Splunk On-Call

Setup

Get your REST endpoint API key and routing key

In Splunk On-Call, go to Settings → Integrations → REST Endpoint
Enable the integration and copy the API key from the endpoint URL
Go to Settings → Routing Keys and copy an existing key — or create a new one (e.g. devhelm) mapped to the team that should receive alerts

Create the alert channel in DevHelm

devhelm alert-channels create \
  --name "Splunk On-Call" \
  --type splunk_oncall \
  --config '{"channelType":"splunk_oncall","apiKey":"your-rest-endpoint-api-key","routingKey":"devhelm"}'

alertChannels:
  - name: Splunk On-Call
    config:
      channelType: splunk_oncall
      apiKey: ${SPLUNK_ONCALL_API_KEY}
      routingKey: ${SPLUNK_ONCALL_ROUTING_KEY}

Test the channel

devhelm alert-channels test <channel-id>

Field	Description	Required
`apiKey`	Splunk On-Call REST endpoint API key	Yes
`routingKey`	Routing key that maps alerts to a team	Yes

Field

Description

Required

apiKey

Splunk On-Call REST endpoint API key

Yes

routingKey

Routing key that maps alerts to a team

Yes

Lifecycle behavior

Splunk On-Call uses a trigger-resolve lifecycle keyed on the incident, so state stays in sync with DevHelm:

DevHelm event	Splunk On-Call action
Incident created	Triggers an incident that follows your escalation policy
Incident resolved	Auto-resolves the corresponding incident
Incident reopened	Triggers a new incident

Troubleshooting

Alerts not creating (HTTP 4xx)

Confirm you used the REST endpoint API key, not a personal API token from Settings → API. The two are different credentials — only the REST endpoint key works with this integration. Then run devhelm alert-channels test <channel-id>.

Incidents created but no one is paged

The routing key must be mapped to a team with an active escalation policy. In Settings → Routing Keys, verify the key points to the right team, then check that team has an on-call schedule.

Overview

Alert Channels

CI/CD

Setup

Configuration

Lifecycle behavior

Troubleshooting

​Setup

​Configuration

​Lifecycle behavior

​Troubleshooting

Setup

Configuration

Lifecycle behavior

Troubleshooting