> ## Documentation Index
> Fetch the complete documentation index at: https://docs.mezmo.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Request Lifecycle

> Request flow, timeout configuration, cancellation, and graceful shutdown behavior.

## Overview

Each request to AURA starts a run. The run owns the request's state: a cancellation token and an event channel that carries tool, MCP progress, usage, and approval events to the response stream. That state is created when the run starts and released when it ends. This document covers the lifecycle, timeout configuration, and known limitations.

***

## Request Flow

```
Client POST /v1/chat/completions
         │
         ▼
┌─────────────────────────────────────┐
│ Shutdown middleware check           │
│   (503 if shutdown_token cancelled) │
└─────────────────────────────────────┘
         │
         ▼
┌─────────────────────────────────────┐
│ Generate request_id (UUID)          │
└─────────────────────────────────────┘
         │
         ▼
┌─────────────────────────────────────┐
│ Spawn producer task                 │
│   - Start the run (own cancellation │
│     token and event channel)        │
│   - Read the run's event channel    │
└─────────────────────────────────────┘
         │
         ▼
┌─────────────────────────────────────┐
│ Stream chat with TimeoutHook        │
│   - Tool calls set thread context   │
│   - Tool results clear context      │
└─────────────────────────────────────┘
         │
         ├──────────────────┬──────────────────┐
         ▼                  ▼                  ▼
┌────────────────┐  ┌──────────────────┐  ┌──────────────────┐
│ Normal         │  │ Timeout/         │  │ Shutdown         │
│ completion     │  │ Disconnect       │  │ (grace expired)  │
└────────────────┘  └──────────────────┘  └──────────────────┘
         │                  │                  │
         │                  ▼                  ▼
         │          ┌──────────────────┐  ┌──────────────────┐
         │          │ Cancel run token │  │ Cancel run token │
         │          │ Send MCP cancel  │  │ Send [DONE]      │
         │          │ Evict from pool  │  │ Then MCP cleanup │
         │          └──────────────────┘  │ (no pool evict)  │
         │                  │             └──────────────────┘
         └──────────┬───────┴──────────────────┘
                    ▼
┌─────────────────────────────────────┐
│ Cleanup (RAII guard)                │
│   - Cancel pending HITL approvals   │
└─────────────────────────────────────┘
```

***

## Timeout Configuration

### Production Defaults

| Setting | Default | Env Variable | Purpose |
| - | - | - | - |
| First chunk timeout | 90 sec | `FIRST_CHUNK_TIMEOUT_SECS` | Max wait for first provider chunk |
| Stream timeout | 15 min | `STREAMING_TIMEOUT_SECS` | Max request duration |
| Stream inactivity timeout | Disabled (`0`) | `STREAM_INACTIVITY_TIMEOUT_SECS` | Max silence between stream items after the first chunk |
| Shutdown grace period | 30 sec | `SHUTDOWN_TIMEOUT_SECS` | Time for in-flight requests to finish on shutdown |
| Heartbeat | 15 sec | - | Disconnect detection |

### Rationale

* **First chunk timeout (90 sec)**: Catches provider connection failures early. If the LLM hasn't sent any data within this window, the request is aborted rather than hanging for the full stream timeout. The window is sized for reasoning models, which can legitimately take over a minute before the first chunk.
* **Stream inactivity timeout (disabled by default)**: Closes the gap between the first-chunk timeout, which guards only the opening chunk, and the stream timeout, which is the whole-request budget. Once streaming starts, a provider that goes silent between chunks runs all the way to the stream timeout by default. Enable this to fail a stalled stream sooner.
* **Stream timeout (15 min)**: Supports long-running MCP tools. It's the only setting that caps total request length. Set it to `0` to remove the cap (not recommended). Without the cap, no setting limits how long a request runs. The first chunk timeout stops applying once output starts, and the stream inactivity timeout, when enabled, bounds only the gaps between stream items. Non-streaming requests have neither window, so the stream timeout is their only time limit.
* **Heartbeat (15 sec)**: Standard SSE keepalive. Detects disconnect during silent tool execution.

### Tuning Stream Inactivity Timeouts

There are two knobs at two layers, both disabled by default (`0`). The TOML `stream_inactivity_timeout_secs` (in `[orchestration.timeouts]`, see [configuration reference](/aura/configuration-reference)) governs coordinator and worker orchestration streams. The server `STREAM_INACTIVITY_TIMEOUT_SECS` (see [web server reference](/aura/web-server-reference)) governs single-agent streaming requests.

The TOML knob exempts tool execution from its timer. The TOML deadline suspends during tool execution and re-arms on each stream item, so slow MCP tool calls and paused HITL approvals do not trip it. This makes it the reliable knob for orchestrated deployments.

When the timer fires, the behavior depends on the layer. For orchestration, the affected task fails with a no-stream-progress error (an agent-timeout failure) and the run completes with partial results instead of dying mid-wave. For the server single-agent path, the request terminates as a timeout.

<Warning>
  Orchestrated worker activity reaches the server layer as tool and progress events. Those events re-arm the server deadline but cannot suspend it, so a worker tool call that goes quiet past the window can still trip the server-layer timeout. For orchestrated deployments, rely on the TOML `stream_inactivity_timeout_secs`. If you also set the server `STREAM_INACTIVITY_TIMEOUT_SECS`, size it above the TOML window.
</Warning>

<Note>
  Aura pins OpenAI to Chat Completions, which streams nothing while the model is thinking, so a long think reads as inactivity and can trip the deadline. Anthropic, Gemini, and OpenAI-compatible `reasoning_content` streams re-arm while thinking, so this affects first-party OpenAI only. Size the window above worst-case think time for those models, or leave the knob off.
</Note>

***

## Tool Event Correlation

Rig spawns tool execution in separate tokio tasks. The hook context (where the LLM decides to call a tool) and the execution context (where MCP actually runs) are decoupled, requiring a mechanism to correlate `tool_call_id` across these boundaries. AURA does this with a FIFO queue that the run holds, so the queue lives and ends with the run. This relies on Rig's streaming mode executing tools sequentially within a request; see [Rig Fork Changes](https://github.com/mezmo/aura/blob/main/docs/rig-fork-changes.md) for the validation methodology if you're tracking Rig upstream compatibility.

***

## Cleanup Mechanism

The run's cancellation token, event channel, and tool-call queue belong to the run, so they're released when the run ends. An RAII guard cancels the request's pending HITL approvals, and runs even on panic. If the async runtime is already gone (process exit), cleanup is skipped. Resources are reclaimed with the process.

***

## MCP Cancellation

On client disconnect or timeout:

1. The run's cancellation token signals all waiting code, including tool calls paused for a HITL approval
2. Aura sends `notifications/cancelled` to MCP servers
3. The agent is evicted from its connection pool to prevent stale connections

MCP servers receive the cancellation notification and can abort in-progress operations.

***

## Graceful Shutdown

The server signals shutdown through two cancellation tokens:

| Token | Cancelled | Purpose |
| - | - | - |
| `shutdown_token` | Immediately on signal | Middleware rejects new requests with 503 |
| `stream_shutdown_token` | After `SHUTDOWN_TIMEOUT_SECS` grace period | Terminates remaining in-flight streams |

### Shutdown Sequence

1. **SIGTERM/SIGINT** received
2. **Phase 1 (immediate)**: `shutdown_token` cancelled. Middleware returns 503 for all new requests
3. **Grace period**: In-flight streams continue running for up to `SHUTDOWN_TIMEOUT_SECS` (default 30s). Streams that complete naturally during this window are unaffected. If every in-flight request finishes within this window, the server skips to step 6.
4. **Phase 2 (drain)**: `stream_shutdown_token` cancelled. Remaining streams:
   * Cancel the run's token (stops in-flight MCP tool execution and pending HITL approvals)
   * Send `[DONE]` to client (before MCP cleanup, so client gets clean termination)
   * Run MCP cleanup (send `notifications/cancelled`, close connections)
   * No pool eviction (pool is dying with the server)
5. **Phase 3 (abort)**: Remaining streams have 5s to finish Phase 2. If request tasks are still running after that, the server logs `Stream shutdown grace period expired, aborting stragglers` and aborts them, so their spans close before the telemetry flush.
6. **Shutdown complete**: The server logs `shutdown complete` with two fields. `clean_shutdown` is `true` when no request had to be aborted. `unaborted_tasks` is `true` when the abort step didn't finish within 400 ms.
7. **Telemetry flush**: The server stops and flushes buffered OpenTelemetry spans, waiting up to 5s for the export.

### Shutdown vs Disconnect/Timeout

| Behavior | Disconnect/Timeout | Shutdown |
| - | - | - |
| Pool eviction | Yes | No (pool is dying) |
| `[DONE]` timing | After MCP cleanup (Timeout) / skipped (Disconnect) | Before MCP cleanup |
| Grace period | None: immediate cancel | Configurable (`SHUTDOWN_TIMEOUT_SECS`) |