Manual Incident Management

By the end of this guide, you’ll know how to create manual incidents, post updates, and resolve them — covering the full workflow for issues that automated monitors don’t catch.

Prerequisites

DevHelm CLI installed or an API token
Familiarity with the incident lifecycle

For conceptual background, see Manual incidents.

When to create a manual incident

A customer reports an issue that your monitors haven’t detected
A deployment caused problems visible to users but not to health checks
You’re proactively tracking a known degradation
An internal system is down but not covered by monitoring yet

Full workflow

Create the incident

devhelm incidents create \
  --title "Checkout flow returning 500 for EU users" \
  --severity DOWN \
  --body "Customer reports started at 14:00 UTC. Affecting payment processing."

curl -X POST https://api.devhelm.io/api/v1/incidents \
  -H "Authorization: Bearer $DEVHELM_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "title": "Checkout flow returning 500 for EU users",
    "severity": "DOWN",
    "body": "Customer reports started at 14:00 UTC. Affecting payment processing."
  }'

Manual incidents are immediately set to CONFIRMED — no waiting for trigger rules or confirmation.

Investigate and post updates

Keep your team informed as you investigate. Timeline updates are posted via the API:

curl -X POST https://api.devhelm.io/api/v1/incidents/<incident-id>/updates \
  -H "Authorization: Bearer $DEVHELM_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "body": "Identified root cause — EU payment gateway connection pool exhaustion",
    "notifySubscribers": true
  }'

Setting notifySubscribers to true sends the update through matched notification policies.

Associate with a monitor (optional)

If you find a related monitor, associate it for Dashboard visibility:

curl -X POST https://api.devhelm.io/api/v1/incidents \
  -H "Authorization: Bearer $DEVHELM_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "title": "Checkout flow 500s",
    "severity": "DOWN",
    "monitorId": "<checkout-monitor-id>"
  }'

Resolve the incident

devhelm incidents resolve <incident-id> \
  --message "Deployed hotfix v2.4.1 — connection pool limits increased"

curl -X POST https://api.devhelm.io/api/v1/incidents/<incident-id>/resolve \
  -H "Authorization: Bearer $DEVHELM_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "body": "Deployed hotfix v2.4.1 — connection pool limits increased"
  }'

Severity selection

Severity	When to use
`DOWN`	Complete failure — users cannot use the feature
`DEGRADED`	Partial failure — feature works but is slow or unreliable
`MAINTENANCE`	Planned issue — you’re aware and working on it

Alerting for manual incidents

Manual incidents flow through notification policies just like automated ones. If the incident is associated with a monitor, policies matching that monitor’s tags, type, or ID apply. Otherwise, only catch-all policies (empty matchRules) and policies with incident_status rules match.

Next steps

Incident timeline

Review the full event history for an incident.

Incidents guide

Day-to-day incident management workflow.

Alerting guide

Configure how manual incident alerts are routed.

Overview

Getting Started

Monitoring Guides

Alerting Guides

Incident Guides

Status Data Guides

Monitoring as Code Guides

Manual Incident Management

When to create a manual incident

Full workflow

Severity selection

Alerting for manual incidents

Next steps

Incident timeline

Incidents guide

Alerting guide

​When to create a manual incident

​Full workflow

​Severity selection

​Alerting for manual incidents

​Next steps

Incident timeline

Incidents guide

Alerting guide

When to create a manual incident

Full workflow

Severity selection

Alerting for manual incidents

Next steps