Add per-tenant alert generator URL template for customizable alert source links by CharlieTLe · Pull Request #7302 · cortexproject/cortex

CharlieTLe · 2026-02-27T23:39:06Z

Summary

Replace Grafana-specific config fields with a single generic ruler_alert_generator_url_template field that accepts a Go text/template string
Users can construct any URL format (Grafana Explore, Perses, or any metrics explorer) without Cortex needing to understand specific UI formats
Keep per-tenant ruler_external_url override so tenants can have different external URLs without changing the global ruler config

When ruler_alert_generator_url_template is set, the ruler evaluates the template with .ExternalURL and .Expression variables to produce the alert's GeneratorURL. Built-in Go template functions like urlquery are available for URL encoding. If the template is empty, the default Prometheus /graph format is used.

New per-tenant config options

Setting	Description	Default
`ruler_external_url`	Per-tenant external URL override for the ruler	`""` (uses global)
`ruler_alert_generator_url_template`	Go text/template for alert generator URLs	`""` (Prometheus format)

Example runtime config

overrides:
  # Grafana Explore links
  tenant-a:
    ruler_external_url: "http://grafana:3000"
    ruler_alert_generator_url_template: >-
      {{ .ExternalURL }}/explore?schemaVersion=1&panes=%7B%22default%22:%7B%22datasource%22:%22my-ds%22,%22queries%22:%5B%7B%22refId%22:%22A%22,%22expr%22:%22{{ urlquery .Expression }}%22%7D%5D,%22range%22:%7B%22from%22:%22now-1h%22,%22to%22:%22now%22%7D%7D%7D&orgId=1

  # Perses explore links
  tenant-b:
    ruler_external_url: "http://perses:8080"
    ruler_alert_generator_url_template: >-
      {{ .ExternalURL }}/explore?expr={{ urlquery .Expression }}

  # Default Prometheus /graph format (no template needed)
  tenant-c: {}

Getting-started docker-compose example

The PR includes a working docker-compose example in docs/getting-started/ with:

Cortex with runtime config enabling per-tenant URL templates
Grafana with per-tenant Alertmanager datasources (tenant-a, tenant-b)
Perses v0.53.1 with explorer enabled and per-tenant Prometheus datasources
Demo alert rules that fire immediately for both tenants
Clicking "See source" on tenant-a alerts opens Grafana Explore; tenant-b alerts open Perses explore

Test plan

Unit tests for executeGeneratorURLTemplate with various expressions, urlquery, and invalid templates
Unit tests for userExternalURL tracking per-tenant URL changes
Unit tests for SendAlerts with custom generator URL function (default + template formats)
Template parse validation in Limits.Validate() catches invalid templates at config load time
Validated exporter test updated for removed Grafana-specific limit fields
Manual e2e test: Docker Compose with Cortex + Grafana + Perses, always-firing alert rules for tenant-a and tenant-b, confirmed "See source" links open the correct explorer with correct datasource and query

🤖 Generated with Claude Code

Add support for tenants to configure alert GeneratorURL to use Grafana Explore format instead of the default Prometheus /graph format. This is controlled by three new per-tenant settings: ruler_alert_generator_url_format, ruler_grafana_datasource_uid, and ruler_grafana_org_id. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: Charlie Le <charlie_le@apple.com>

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: Charlie Le <charlie_le@apple.com>

yeya24

@rajagopalanand Can you help take a look?

friedrichg

I gave a review, but I think we do not care what parameters are send to the url. We don't care about grafana org IDs

I think we should make this general enough to be used by any metrics explorer.
Maybe consider renaming the function as explore and not have any grafana

I am thinking of something that would support for example:
explore

what do you think?

friedrichg · 2026-03-16T17:06:40Z

pkg/util/validation/exporter_test.go

 		cortex_overrides{limit_name="reject_old_samples",user="tenant-a"} 0
 		cortex_overrides{limit_name="reject_old_samples_max_age",user="tenant-a"} 1.2096e+06
 		cortex_overrides{limit_name="ruler_evaluation_delay_duration",user="tenant-a"} 0
+		cortex_overrides{limit_name="ruler_grafana_org_id",user="tenant-a"} 0


I think this should never be 0. The default org id as zero should be rejected as invalid.

Consider putting

if l.RulerGrafanaOrgID < 1 {

return errors.New("ruler_grafana_org_id must be greater than or equal to 1")

}

in UnmarshalYaml and UnmarshallJSON in limits.go

friedrichg · 2026-03-16T17:13:15Z

pkg/ruler/compat.go

+					if orgID == 0 {
+						orgID = 1
+					}


Suggested change

if orgID == 0 {

orgID = 1

}

Should not be needed after my other comment

friedrichg · 2026-03-16T17:34:55Z

pkg/ruler/compat.go

+					if orgID == 0 {
+						orgID = 1
+					}
+					return grafanaExploreLink(externalURL, expr, datasourceUID, orgID)


I think is better we make grafanaExploreLink return errors if the url cannot be generated. For example: empty datasource
and if get errors we should log the error and fallback to prometheus still

friedrichg · 2026-03-16T18:01:42Z

docs/configuration/config-file-reference.md

+
+# Grafana datasource UID for alert generator URLs when format is
+# grafana-explore.
+[ruler_grafana_datasource_uid: <string> | default = ""]


it's not clear looking at this configuration that empty is not valid.

friedrichg · 2026-03-16T18:17:57Z

pkg/ruler/ruler.go

+				},
+			},
+			"range": map[string]string{
+				"from": "now-1h",


this looks like it should be configurable. Maybe there is a way to configure the whole pane as a configurable json. At the end of the day, we don't care for this

…nerator URLs Replace the 3 Grafana-specific per-tenant config fields (ruler_alert_generator_url_format, ruler_grafana_datasource_uid, ruler_grafana_org_id) with a single generic field: ruler_alert_generator_url_template. This field accepts a Go text/template string with .ExternalURL and .Expression variables, plus built-in functions like urlquery. Users can construct any URL format (Grafana, Perses, etc.) without Cortex needing to understand specific UI formats. The ruler_external_url per-tenant override and SendAlerts signature (func(expr string) string) are kept unchanged. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Signed-off-by: Charlie Le <charlie_le@apple.com>

Add per-tenant Alertmanager datasources (tenant-a, tenant-b) to Grafana provisioning so alerts are visible in Grafana's alerting UI. Add runtime-config.yaml with per-tenant overrides: - tenant-a: Grafana Explore URL template with full pane JSON - tenant-b: Perses explore URL template with PrometheusTimeSeriesQuery Update Perses from v0.49 to v0.53.1 and enable the explorer feature (frontend.explorer.enable: true). Rename project from "default" to "cortex" to match template URLs. Add Step 7 to the getting-started guide with instructions for: - Configuring per-tenant alert generator URL templates - Loading alertmanager configs and demo alert rules - Viewing alerts in Grafana at /alerting/groups?groupBy=alertname - Verifying generator URLs via the API Also configure ruler.alertmanager_url and ruler.external_url, and set an explicit UID on the Grafana Cortex datasource for use in templates. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Signed-off-by: Charlie Le <charlie_le@apple.com>

Add support for tenants to configure alert GeneratorURL to use Grafana Explore format instead of the default Prometheus /graph format. This is controlled by three new per-tenant settings: ruler_alert_generator_url_format, ruler_grafana_datasource_uid, and ruler_grafana_org_id. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: Charlie Le <charlie_le@apple.com>

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: Charlie Le <charlie_le@apple.com>

…nerator URLs Replace the 3 Grafana-specific per-tenant config fields (ruler_alert_generator_url_format, ruler_grafana_datasource_uid, ruler_grafana_org_id) with a single generic field: ruler_alert_generator_url_template. This field accepts a Go text/template string with .ExternalURL and .Expression variables, plus built-in functions like urlquery. Users can construct any URL format (Grafana, Perses, etc.) without Cortex needing to understand specific UI formats. The ruler_external_url per-tenant override and SendAlerts signature (func(expr string) string) are kept unchanged. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Signed-off-by: Charlie Le <charlie_le@apple.com>

Add per-tenant Alertmanager datasources (tenant-a, tenant-b) to Grafana provisioning so alerts are visible in Grafana's alerting UI. Add runtime-config.yaml with per-tenant overrides: - tenant-a: Grafana Explore URL template with full pane JSON - tenant-b: Perses explore URL template with PrometheusTimeSeriesQuery Update Perses from v0.49 to v0.53.1 and enable the explorer feature (frontend.explorer.enable: true). Rename project from "default" to "cortex" to match template URLs. Add Step 7 to the getting-started guide with instructions for: - Configuring per-tenant alert generator URL templates - Loading alertmanager configs and demo alert rules - Viewing alerts in Grafana at /alerting/groups?groupBy=alertname - Verifying generator URLs via the API Also configure ruler.alertmanager_url and ruler.external_url, and set an explicit UID on the Grafana Cortex datasource for use in templates. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Signed-off-by: Charlie Le <charlie_le@apple.com>

Resolve conflict in schemas/cortex-config-schema.json: keep both the upstream results_cache_ttl entry and our ruler_alert_generator_url_template. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Signed-off-by: Charlie Le <charlie_le@apple.com>

Resolve conflict in schemas/cortex-config-schema.json: keep both results_cache_ttl and ruler_alert_generator_url_template entries. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Signed-off-by: Charlie Le <charlie_le@apple.com>

pull-request-size bot added the size/L label Feb 27, 2026

dosubot bot added the component/rules Bits & bobs todo with rules and alerts: the ruler, config service etc. label Feb 27, 2026

Fix gofmt alignment in ruler and validation packages

52b69c6

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: Charlie Le <charlie_le@apple.com>

CharlieTLe linked an issue Mar 1, 2026 that may be closed by this pull request

Alert GeneratorURLs generated by the ruler do not indicate tenant #809

Open

yeya24 reviewed Mar 1, 2026

View reviewed changes

friedrichg reviewed Mar 16, 2026

View reviewed changes

CharlieTLe and others added 2 commits March 30, 2026 11:51

pull-request-size bot added size/XL and removed size/L labels Mar 30, 2026

CharlieTLe requested a review from friedrichg March 30, 2026 22:23

CharlieTLe changed the title ~~Add per-tenant Grafana Explore URL format for alert GeneratorURL~~ Add per-tenant alert generator URL template for customizable alert source links Mar 30, 2026

CharlieTLe and others added 6 commits March 30, 2026 15:54

Fix gofmt alignment in ruler and validation packages

a26eea7

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: Charlie Le <charlie_le@apple.com>

Merge origin/809 into local 809

71f9121

Resolve conflict in schemas/cortex-config-schema.json: keep both results_cache_ttl and ruler_alert_generator_url_template entries. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Signed-off-by: Charlie Le <charlie_le@apple.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add per-tenant alert generator URL template for customizable alert source links#7302

Add per-tenant alert generator URL template for customizable alert source links#7302
CharlieTLe wants to merge 10 commits intocortexproject:masterfrom
CharlieTLe:809

CharlieTLe commented Feb 27, 2026 •

edited

Loading

Uh oh!

yeya24 left a comment

Uh oh!

friedrichg left a comment

Uh oh!

friedrichg Mar 16, 2026

Uh oh!

friedrichg Mar 16, 2026

Uh oh!

friedrichg Mar 16, 2026

Uh oh!

friedrichg Mar 16, 2026

Uh oh!

friedrichg Mar 16, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

CharlieTLe commented Feb 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

New per-tenant config options

Example runtime config

Getting-started docker-compose example

Test plan

Uh oh!

yeya24 left a comment

Choose a reason for hiding this comment

Uh oh!

friedrichg left a comment

Choose a reason for hiding this comment

Uh oh!

friedrichg Mar 16, 2026

Choose a reason for hiding this comment

Uh oh!

friedrichg Mar 16, 2026

Choose a reason for hiding this comment

Uh oh!

friedrichg Mar 16, 2026

Choose a reason for hiding this comment

Uh oh!

friedrichg Mar 16, 2026

Choose a reason for hiding this comment

Uh oh!

friedrichg Mar 16, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

CharlieTLe commented Feb 27, 2026 •

edited

Loading