Manual Incidents

Manual incidents let you track problems that automated monitors don’t cover — deployment issues, customer-reported outages, or known degradations that haven’t triggered a monitor yet.

When to use manual incidents

Deployment failures — a bad deploy causes user-facing issues before monitors detect it
Customer reports — users report problems that your checks haven’t caught
Partial outages — a subset of users is affected but aggregate metrics look healthy
Proactive tracking — you know something is wrong and want to start the incident timeline before automation kicks in

Creating a manual incident

devhelm incidents create \
  --title "Database latency spike" \
  --severity DOWN

curl -X POST https://api.devhelm.io/api/v1/incidents \
  -H "Authorization: Bearer $DEVHELM_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "title": "Database latency spike",
    "severity": "DOWN"
  }'

Request fields

Field	Type	Required	Description
`title`	string	Yes	Short summary of the issue
`severity`	string	Yes	`DOWN`, `DEGRADED`, or `MAINTENANCE`
`monitorId`	UUID	No	Associate with a specific monitor
`body`	string	No	Detailed description or context

Manual incidents have source MANUAL and are immediately set to CONFIRMED status — they skip the watching and trigger phases since a human has already confirmed the problem.

Associating a monitorId links the incident to a specific monitor in the Dashboard and timeline. The monitor’s notification policies will apply for alert routing.

Adding updates

Post updates to the incident timeline to keep your team informed. Timeline updates are posted via the API:

API

curl -X POST https://api.devhelm.io/api/v1/incidents/<incident-id>/updates \
  -H "Authorization: Bearer $DEVHELM_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "body": "Identified root cause — connection pool exhaustion",
    "notifySubscribers": true
  }'

Update fields

Field	Type	Required	Description
`body`	string	Yes	Update message or context
`newStatus`	string	No	Transition to a new status (e.g., `RESOLVED`)
`notifySubscribers`	boolean	No	Send notification to subscribed channels

Resolving a manual incident

devhelm incidents resolve <incident-id> \
  --message "Connection pool limits increased, latency back to normal"

curl -X POST https://api.devhelm.io/api/v1/incidents/<incident-id>/resolve \
  -H "Authorization: Bearer $DEVHELM_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "body": "Connection pool limits increased, latency back to normal"
  }'

Resolved manual incidents get resolutionReason: MANUAL.

Manual vs automated incidents

Aspect	Automated	Manual
Source	`MONITORS`, `STATUS_DATA`, `RESOURCE_GROUP`	`MANUAL`
Initial status	`WATCHING` → `TRIGGERED` → `CONFIRMED`	`CONFIRMED` immediately
Recovery	Auto-resolves when recovery policy is met	Must be resolved by a user
Reopening	Reopens automatically if the monitor fails again	Does not reopen automatically
Alerting	Routes through notification policies based on the monitor	Routes through policies matching the associated monitor, or catch-all policies

Next steps

Incident timeline

Track all status changes and updates.

Incidents overview

Understand the full incident lifecycle.

Alerting overview

Configure where manual incident notifications are sent.

Getting Started

Monitoring as Code

Monitoring

Incidents

Alerting

Status Data

Platform

Common Patterns

Manual Incidents

When to use manual incidents

Creating a manual incident

Request fields

Adding updates

Update fields

Resolving a manual incident

Manual vs automated incidents

Next steps

Incident timeline

Incidents overview

Alerting overview

​When to use manual incidents

​Creating a manual incident

​Request fields

​Adding updates

​Update fields

​Resolving a manual incident

​Manual vs automated incidents

​Next steps

Incident timeline

Incidents overview

Alerting overview

When to use manual incidents

Creating a manual incident

Request fields

Adding updates

Update fields

Resolving a manual incident

Manual vs automated incidents

Next steps