> ## Documentation Index
> Fetch the complete documentation index at: https://docs.devhelm.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Your First Incident

> Understand and resolve your first DevHelm incident — lifecycle, timeline, and resolution

By the end of this guide, you'll understand what happens when a monitor fails, how to investigate an incident, and how to resolve it.

<Accordion title="Prerequisites">
  * At least one monitor running — see [First HTTP monitor](/guides/first-http-monitor)
  * Familiarity with the Dashboard or CLI
</Accordion>

## When an incident appears

An incident appears in your Dashboard when a monitor's check failures match its [incident policy](/incidents/policies). Here's the typical flow:

<Steps>
  <Step title="Checks start failing">
    A monitor's check returns an error or fails an assertion. DevHelm starts watching the situation.
  </Step>

  <Step title="Trigger rule matched">
    After enough consecutive failures (default: 2), the trigger rule fires. The incident enters `TRIGGERED` status.
  </Step>

  <Step title="Multi-region confirmation">
    If the monitor runs from multiple regions, DevHelm waits for confirmation from additional regions. This reduces false positives from single-region network issues.
  </Step>

  <Step title="Incident confirmed">
    The incident moves to `CONFIRMED`. Alerts fire through your [notification policies](/alerting/notification-policies).
  </Step>
</Steps>

## Investigate the incident

### View incident details

<CodeGroup>
  ```bash CLI theme={null}
  devhelm incidents list
  devhelm incidents get <incident-id>
  ```

  ```bash API theme={null}
  curl https://api.devhelm.io/api/v1/incidents \
    -H "Authorization: Bearer $DEVHELM_API_TOKEN"

  curl https://api.devhelm.io/api/v1/incidents/<incident-id> \
    -H "Authorization: Bearer $DEVHELM_API_TOKEN"
  ```
</CodeGroup>

The incident detail shows:

* **Severity** — `DOWN`, `DEGRADED`, or `MAINTENANCE`
* **Affected regions** — which probe regions are seeing failures
* **Timeline** — every status change and update since detection
* **Trigger rule** — which rule fired and why
* **Duration** — how long the incident has been active

### Check the timeline

The [incident timeline](/incidents/timeline) shows exactly what happened and when:

```bash theme={null}
devhelm incidents get <incident-id>
```

Look at the `updates` array — each entry records a status change, user update, or system event with a timestamp.

### View failing check results

Drill into the monitor's recent check results to understand what's failing:

```bash theme={null}
devhelm monitors results <monitor-id> --limit 10
```

Look for assertion failures, HTTP error codes, timeouts, or connection errors.

## Resolve the incident

### Automatic resolution

Most incidents resolve automatically. When the monitor starts passing again:

1. Consecutive passing checks accumulate (default: 2 required)
2. Enough regions must be healthy (default: 2)
3. The incident moves to `RESOLVED`
4. A cooldown period prevents immediate reopening (default: 5 minutes)

### Manual resolution

If you've fixed the issue and don't want to wait for automatic recovery:

<CodeGroup>
  ```bash CLI theme={null}
  devhelm incidents resolve <incident-id> \
    --message "Deployed hotfix to restore API endpoint"
  ```

  ```bash API theme={null}
  curl -X POST https://api.devhelm.io/api/v1/incidents/<incident-id>/resolve \
    -H "Authorization: Bearer $DEVHELM_API_TOKEN" \
    -H "Content-Type: application/json" \
    -d '{"body": "Deployed hotfix to restore API endpoint"}'
  ```
</CodeGroup>

### Add context

Post updates during investigation to keep your team informed. Timeline updates are posted via the API:

```bash theme={null}
curl -X POST https://api.devhelm.io/api/v1/incidents/<incident-id>/updates \
  -H "Authorization: Bearer $DEVHELM_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "body": "Investigating — seeing 503s from upstream dependency",
    "notifySubscribers": true
  }'
```

## Key concepts to know

| Concept                                              | What to know                                                             |
| ---------------------------------------------------- | ------------------------------------------------------------------------ |
| **[Incident policy](/incidents/policies)**           | Controls when incidents open (trigger rules) and close (recovery policy) |
| **[Confirmation](/incidents/policies#confirmation)** | Multi-region validation that reduces false positives                     |
| **[Cooldown](/incidents/policies#recovery)**         | Quiet period after resolution that prevents flapping                     |
| **[Reopening](/incidents/timeline#reopening)**       | If the monitor fails again after cooldown, the same incident reopens     |

## Next steps

<CardGroup cols={2}>
  <Card title="Incidents overview" icon="circle-exclamation" href="/incidents/overview">
    Full lifecycle, statuses, severities, and sources.
  </Card>

  <Card title="Incident policies" icon="sliders" href="/incidents/policies">
    Customize trigger rules and recovery behavior.
  </Card>

  <Card title="First alert" icon="bell" href="/guides/first-alert">
    Get notified when incidents happen.
  </Card>

  <Card title="Incidents guide" icon="wrench" href="/guides/incidents">
    Day-to-day incident management workflows.
  </Card>
</CardGroup>