Skip to main content
This guide walks you through configuring the alerting system end to end. For conceptual background, see Alerting overview.

Step 1: Create alert channels

Alert channels are the destinations where notifications get sent. Create them first, then reference them in notification policies.
# Slack
devhelm alert-channels create \
  --name "Engineering Slack" \
  --type SLACK \
  --webhook-url "https://hooks.slack.com/services/T.../B.../xxx"

# PagerDuty
devhelm alert-channels create \
  --name "On-Call PagerDuty" \
  --type PAGERDUTY \
  --routing-key "your-events-api-v2-routing-key"

# Email
devhelm alert-channels create \
  --name "Team Email" \
  --type EMAIL \
  --recipients "oncall@example.com,team@example.com"

Test a channel

Verify your channel is configured correctly:
devhelm alert-channels test <channel-id>
This sends a test notification through the channel and reports success or failure.

Step 2: Create notification policies

Notification policies connect incidents to alert channels through match rules and escalation chains.

Simple policy (notify Slack for all incidents)

curl -X POST https://api.devhelm.io/api/v1/notification-policies \
  -H "Authorization: Bearer $DEVHELM_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "All incidents to Slack",
    "matchRules": [],
    "escalation": {
      "steps": [
        {
          "delayMinutes": 0,
          "channelIds": ["<slack-channel-id>"]
        }
      ]
    },
    "priority": 0
  }'
An empty matchRules array is a catch-all — it matches every incident.

Tiered escalation

Route critical incidents through an escalation chain:
{
  "name": "Critical escalation",
  "matchRules": [
    { "type": "severity_gte", "value": "DOWN" }
  ],
  "escalation": {
    "steps": [
      {
        "delayMinutes": 0,
        "channelIds": ["<slack-channel-id>"],
        "requireAck": false
      },
      {
        "delayMinutes": 15,
        "channelIds": ["<pagerduty-channel-id>"],
        "requireAck": true,
        "repeatIntervalSeconds": 300
      },
      {
        "delayMinutes": 30,
        "channelIds": ["<email-channel-id>"]
      }
    ],
    "onResolve": "notify_all_steps",
    "onReopen": "restart_from_beginning"
  },
  "priority": 10
}
This policy:
  1. Immediately notifies Slack
  2. After 15 minutes (if not acknowledged), pages PagerDuty and repeats every 5 minutes
  3. After 30 minutes, emails the team

Scoped policies

Use match rules to target specific monitors or conditions:
{
  "name": "Production HTTP monitors only",
  "matchRules": [
    { "type": "monitor_type_in", "values": ["HTTP"] },
    { "type": "monitor_tag_in", "values": ["production"] }
  ],
  "escalation": {
    "steps": [
      { "delayMinutes": 0, "channelIds": ["<channel-id>"] }
    ]
  },
  "priority": 5
}

Available match rules

RuleMatches onFields
severity_gteSeverity ≥ valuevalue: DOWN, DEGRADED, MAINTENANCE
monitor_id_inSpecific monitorsmonitorIds: list of UUIDs
monitor_type_inMonitor typesvalues: HTTP, TCP, DNS, etc.
monitor_tag_inMonitor tagsvalues: tag names
region_inAffected regionsregions: region codes
incident_statusEvent typevalue: created, resolved, reopened
service_id_inStatus data servicesvalues: service IDs
resource_group_id_inResource groupsvalues: group IDs

Step 3: Set policy priority

Policies are evaluated by priority (highest first). All matching policies execute — there’s no “first match wins”. Set higher priority on more specific policies:
PriorityPolicyMatch rules
10Critical escalationseverity_gte: DOWN
5Production HTTP to Slackmonitor_type_in: HTTP + monitor_tag_in: production
0Everything to email(catch-all)

Testing your setup

  1. Test individual channels: devhelm alert-channels test <id>
  2. Create a test monitor with a low frequency that will fail:
devhelm monitors create \
  --name "Alert Test" \
  --type HTTP \
  --url https://httpstat.us/500 \
  --frequency 30 \
  --regions us-east
  1. Wait for the incident to trigger and verify notifications arrive
  2. Delete the test monitor when done

Next steps

Integration setup

Detailed setup guides for each alert channel type.

Incidents guide

Manage incidents and maintenance windows.