> ## Documentation Index
> Fetch the complete documentation index at: https://docs.devhelm.io/llms.txt
> Use this file to discover all available pages before exploring further.
# Incidents Overview
> How DevHelm incidents work — lifecycle, statuses, severities, and automatic resolution
An incident represents an ongoing problem detected by a monitor, reported by a third-party service, or created manually. DevHelm automates the full lifecycle from detection through confirmation, alerting, and recovery.
**Define this in code.** Manage incident policies as part of your monitoring-as-code workflow:
[YAML format](/mac/yaml/monitors) · [Terraform](/mac/terraform/monitors) · [CI/CD patterns](/mac/ci-cd/overview)
## Incident lifecycle
Every incident follows a predictable state machine:
```
Check fails → Trigger rule fires → WATCHING → Regions confirm → TRIGGERED → Multi-region confirmed → CONFIRMED
↓
Alerts sent
↓
Checks start passing
↓
Recovery policy met → RESOLVED
↓
Cooldown period
```
1. **Detection** — A monitor's checks fail and match a [trigger rule](/incidents/policies#trigger-rules)
2. **Watching** — The system observes the failure while waiting for enough data to confirm
3. **Triggered** — The trigger rule threshold is met in at least one region
4. **Confirmed** — The [confirmation policy](/incidents/policies#confirmation) validates failures across multiple regions, promoting the incident to active status and firing alerts
5. **Resolved** — The [recovery policy](/incidents/policies#recovery) detects consecutive passing checks across enough regions, or a user resolves manually
6. **Cooldown** — A configurable quiet period prevents the same monitor from immediately reopening a new incident
## Statuses
| Status | Meaning |
| ----------- | ----------------------------------------------------------------------------- |
| `WATCHING` | Failure detected but not yet confirmed — waiting for additional check results |
| `TRIGGERED` | Trigger rule threshold met in at least one region |
| `CONFIRMED` | Active incident — failure confirmed across regions, alerts have been sent |
| `RESOLVED` | Incident closed — monitor recovered or manually resolved |
## Severities
Each incident carries a severity that determines urgency and drives [notification policy matching](/alerting/notification-policies).
| Severity | When it's used |
| ------------- | -------------------------------------------------------------------------------------- |
| `DOWN` | Complete failure — endpoint unreachable or critical assertions failing |
| `DEGRADED` | Partial failure — response time thresholds exceeded or non-critical assertions failing |
| `MAINTENANCE` | Planned downtime — created by [maintenance windows](/incidents/maintenance-windows) |
A single monitor can have trigger rules at different severities. For example, a response time threshold might open a `DEGRADED` incident, while consecutive failures open a `DOWN` incident.
## Sources
Incidents can originate from four different sources:
| Source | Created by |
| ---------------- | ------------------------------------------------------------------------------------------------------------- |
| `MONITORS` | Automatic detection when a monitor's check results match its trigger rules |
| `MANUAL` | User-created via Dashboard, CLI, or API for issues not covered by automation |
| `STATUS_DATA` | Propagated from a third-party service incident when you track it as a [dependency](/status-data/dependencies) |
| `RESOURCE_GROUP` | Aggregated from multiple monitors or services within a [resource group](/guides/resource-groups) |
## Resolution reasons
When an incident resolves, DevHelm records the reason:
| Reason | Meaning |
| ---------------- | ------------------------------------------------------------------------------------------ |
| `AUTO_RECOVERED` | Monitor checks started passing and met the [recovery policy](/incidents/policies#recovery) |
| `MANUAL` | Resolved by a user via Dashboard, CLI, or API |
| `AUTO_RESOLVED` | Resolved by system logic (e.g., an upstream Status Data incident resolved) |
## Reopening
If a monitor fails again **within** the cooldown period after an incident resolves, DevHelm **reopens** the existing incident rather than creating a new one. Once the cooldown period has passed, a subsequent failure opens a **new** incident. The `reopenCount` field tracks how many times an incident has been reopened.
Reopening behavior interacts with [escalation chains](/alerting/escalation-chains#reopen-behavior) — you can configure whether escalation restarts from the beginning or resumes from the current step.
## Incident fields
Key fields on every incident object:
| Field | Type | Description |
| ------------------ | --------- | ----------------------------------------------------------------------- |
| `id` | UUID | Unique incident identifier |
| `status` | string | Current lifecycle status |
| `severity` | string | DOWN, DEGRADED, or MAINTENANCE |
| `source` | string | How the incident was created |
| `title` | string | Short summary (auto-generated or user-provided) |
| `monitorId` | UUID | Monitor that triggered the incident (null for manual/service incidents) |
| `affectedRegions` | string\[] | Probe regions that observed the failure |
| `reopenCount` | integer | Number of times the incident was reopened |
| `resolutionReason` | string | How the incident was resolved |
| `startedAt` | datetime | When the incident was first detected |
| `confirmedAt` | datetime | When multi-region confirmation completed |
| `resolvedAt` | datetime | When the incident was resolved |
| `cooldownUntil` | datetime | End of cooldown period after resolution |
| `shortlink` | string | Short URL to the incident detail page |
For the full incident schema including all fields, see the [API Reference](/api-reference).
## Next steps
Configure trigger rules, confirmation, and recovery behavior.
Create incidents for issues not caught by automated monitoring.
Route incident notifications to your team.
Track status changes and event history.