Files
turnstone/docs/api-reference.md
T
Patrick Buckley a6e929b0a0 Add channel integrations with Discord adapter and atomic session resu… (#24)
* Add channel integrations with Discord adapter and atomic session resume (#24)

Bidirectional channel adapter framework connecting external messaging
platforms to turnstone workstreams via Redis MQ. Discord ships as the
first adapter; the protocol supports future Slack/Teams integrations.

Channel framework:
- ChannelAdapter protocol and ChannelRouter for channel↔workstream mapping
- AsyncRedisBroker with single dispatch loop and per-channel ordered workers
- channel_routes table (migration 003) for persistent route storage
- 9 new StorageBackend methods (4 channel_user + 5 channel_route CRUD)
- Unified turnstone-channel gateway entry point, loads adapters by config
- Message chunking, approval formatting, plan review formatting

Discord adapter:
- discord.py v2.4+ bot with thread-per-@mention model
- Slash commands: /link (modal), /unlink, /ask, /status, /close
- Persistent button views for tool approval and plan review
- Streaming responses via edit-in-place (1.5s interval)
- Stale route detection and atomic session resume via resume_session field
- SessionResumedEvent confirmation back to channel
- Auto-approve support (blanket + per-tool list)

Atomic session resume:
- resume_session field on CreateWorkstreamMessage for single-request resume
- Server resumes session during POST /v1/api/workstreams/new atomically
- Bridge emits SessionResumedEvent to per-workstream channel
- WorkstreamCreatedEvent extended with resumed/session_id/message_count
- Server UI dashboardResumeSession simplified to single request
- Pruned sessions fall back gracefully to fresh start

Service auth:
- Bridge and console auto-mint service JWTs from TURNSTONE_JWT_SECRET
- Bridge: approve scope (1 week). Console collector: read. Proxy: write.

Console admin:
- Channels tab with per-user view, force-link modal, unlink
- 3 admin API endpoints for channel user management
- Styled confirm modals replacing browser confirm() dialogs

Bug fixes:
- AsyncRedisBroker: replaced per-channel listener tasks with single
  dispatch loop + per-channel queue workers (fixes message stealing race)
- Bridge: approval/plan review dedup guard prevents SSE reconnect duplicates
- Bridge: _active_sends tracked for initial messages (fixes missing
  TurnCompleteEvent and unfinalized streaming messages)
- Bridge: HTTP calls moved outside lock scope in approval handlers
- Bridge: _handle_send cleans up _active_sends on HTTP/server errors
- Formatter: reads server SSE format (func_name/preview) with fallback

Docs, SDK, tests:
- docs/channels.md setup guide, architecture diagram 16
- Updated api-reference.md, architecture.md, console.md, docker.md
- Python SDK: resume_session param on create_workstream (async + sync)
- TypeScript SDK: updated CreateWorkstreamRequest/Response interfaces
- OpenAPI schema: resume_session request, resumed/message_count response
- 91 new tests (19 storage, 15 broker, 22 protocol, 6 routing,
  18 discord, 12 resume flow) — 1120 total passing

* Fix CI lint/typecheck failures and address Copilot review feedback (#24)

Lint: fix import ordering, remove unused imports, use contextlib.suppress.
Mypy: explicit postgresql dialect import, add discord module overrides for
optional-dependency CI environments.
Copilot: fix double-escaping in admin confirm modals, return resolved
session_id from server resume response, fix channel_routes diagram schema,
use atomic setdefault for routing locks, add post-insert race guard in
admin channel create, support SSE format in auto-approve check, update
identity linking note in architecture diagram.

* Fix remaining mypy call-arg errors for discord.py optional dependency

Add type: ignore[call-arg] on Modal(title=) and Cog(name=) class
definitions that fail when discord.py is not installed in CI.
2026-03-04 13:02:58 -08:00

966 lines
30 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# turnstone Web Server API Reference
## Overview
> See also: [MQ Protocol diagram](diagrams/png/06-mq-protocol.png) | [Message Routing diagram](diagrams/png/07-message-routing.png) | [Redis Key Schema diagram](diagrams/png/08-redis-key-schema.png)
`turnstone-server` exposes a browser-based chat UI backed by a
**Starlette** ASGI application served by **uvicorn**. The server uses
**Server-Sent Events (SSE)** via `sse-starlette` for real-time streaming
and **HTTP POST** for user actions.
All API responses use `Content-Type: application/json` unless otherwise noted.
CORS headers (`Access-Control-Allow-Origin: *`) are included on every response.
The server supports multiple concurrent **workstreams** (tabs), each backed by
an independent `ChatSession` and event queue.
---
## API Versioning
All API endpoints use the `/v1/` prefix. Non-API endpoints (`/`, `/health`, `/metrics`, `/openapi.json`, `/docs`, `/static/*`, `/shared/*`) are unversioned.
### Interactive Documentation
- **OpenAPI spec**: `GET /openapi.json` — machine-readable OpenAPI 3.1 schema
- **Swagger UI**: `GET /docs` — interactive API explorer (loads from CDN)
### Client SDKs
Typed client libraries for programmatic access to both the server and console APIs.
**Python** (included in the `turnstone` package):
```python
from turnstone.sdk import TurnstoneServer
with TurnstoneServer("http://localhost:8080", token="tok_xxx") as client:
ws = client.create_workstream(name="demo")
result = client.send_and_wait("Hello!", ws.ws_id)
print(result.content)
```
Async variant: `AsyncTurnstoneServer` / `AsyncTurnstoneConsole`.
**TypeScript** (`sdk/typescript/`):
```typescript
import { TurnstoneServer } from "@turnstone/sdk";
const client = new TurnstoneServer({ baseUrl: "http://localhost:8080", token: "tok_xxx" });
const ws = await client.createWorkstream({ name: "demo" });
const result = await client.sendAndWait("Hello!", ws.ws_id);
console.log(result.content);
```
---
## Authentication
When auth is enabled (`[auth].enabled = true` or `TURNSTONE_AUTH_ENABLED=1`), all API endpoints except public paths require a valid token.
### Sending Credentials
Include a token in one of two ways:
- **Bearer header**: `Authorization: Bearer <token>`
- **Cookie**: `turnstone_auth=<token>` (set automatically by the login endpoint)
The server accepts three token types:
| Type | Format | Example |
|------|--------|---------|
| JWT | Base64 segments separated by dots | `eyJhbG...` |
| API token | `ts_` prefix + 64 hex chars | `ts_a1b2c3d4...` |
| Config token | Arbitrary string from `config.toml` | `my-secret-token` |
JWTs are the recommended credential for browser sessions. API tokens are suitable for programmatic access and CI/CD. Config tokens are a simple option for single-node deployments.
### `POST /v1/api/auth/login`
Authenticate with credentials and receive a JWT. Accepts two credential formats:
**Username + password:**
```json
{"username": "alice", "password": "hunter2"}
```
**API token:**
```json
{"token": "ts_a1b2c3d4e5f6..."}
```
**Response (success):** `200`
```json
{
"status": "ok",
"role": "full",
"scopes": "approve,read,write",
"jwt": "eyJhbGciOiJIUzI1NiIs...",
"user_id": "u_abc123"
}
```
The response also sets a `turnstone_auth` HttpOnly cookie containing the JWT.
**Response (failure):** `401`
```json
{"error": "Invalid credentials"}
```
---
### `POST /v1/api/auth/logout`
Clears the `turnstone_auth` cookie. No request body required.
**Response:** `200`
```json
{"status": "ok"}
```
The response includes a `Set-Cookie` header that expires the auth cookie.
---
### `GET /v1/api/auth/status`
Returns the current authentication state. Works with or without a valid token.
**Response (authenticated):** `200`
```json
{
"authenticated": true,
"user_id": "u_abc123",
"scopes": ["approve", "read", "write"],
"source": "jwt"
}
```
**Response (not authenticated):** `200`
```json
{
"authenticated": false,
"user_id": null,
"scopes": [],
"source": null
}
```
**Response (auth disabled):** `200`
```json
{
"authenticated": false,
"auth_enabled": false
}
```
---
### `POST /v1/api/auth/setup`
Creates the first admin user when no users exist in the database. This is a
public endpoint (no authentication required) that only succeeds when auth is
enabled and the user database is empty. Both the server and console expose
this endpoint.
**Request body:**
```json
{
"username": "admin",
"display_name": "Admin",
"password": "strongpass"
}
```
| Field | Type | Required | Validation |
|----------------|--------|----------|-----------------------------|
| `username` | string | yes | 1-64 ASCII characters |
| `display_name` | string | yes | Non-empty |
| `password` | string | yes | Minimum 8 characters |
**Response (success):** `200`
```json
{
"status": "ok",
"user_id": "u_abc123",
"username": "admin",
"role": "full",
"scopes": "approve,read,write",
"jwt": "eyJhbGciOiJIUzI1NiIs..."
}
```
The response also sets a `turnstone_auth` HttpOnly cookie containing the JWT.
**Response (already set up):** `409`
```json
{"error": "Setup already completed"}
```
Returned when one or more users already exist in the database.
**Response (auth disabled):** `400`
```json
{"error": "Auth is not enabled"}
```
---
## Endpoints
### `GET /`
Serves the embedded single-page application (HTML, CSS, and JavaScript inlined
in a single document). The SPA connects to the SSE and POST endpoints listed
below.
**Response:** `text/html; charset=utf-8`
---
### `GET /v1/api/events?ws_id=<id>`
Opens a Server-Sent Events stream scoped to a single workstream. The connection
remains open indefinitely; the server pushes events as they occur.
**Query parameters:**
| Parameter | Type | Required | Description |
|-----------|--------|----------|----------------------------|
| `ws_id` | string | yes | Workstream identifier |
**Error:** Returns `404` with `{"error": "Unknown workstream"}` if `ws_id` is
not recognized.
#### Connection lifecycle
1. **`connected`** -- sent immediately on connect.
```json
{
"type": "connected",
"model": "kappa_20b_131k",
"model_alias": "default",
"skip_permissions": false
}
```
`skip_permissions` reflects the workstream's current auto-approve state. It is
`true` if the server was started with `--skip-permissions` or if the user chose
"Always approve" via the approval prompt during the session.
2. **`history`** -- replays the full conversation history so the client can
rebuild its UI.
```json
{
"type": "history",
"messages": [
{"role": "user", "content": "Hello"},
{"role": "assistant", "content": "Hi there!", "tool_calls": null},
{"role": "tool", "content": "..."}
]
}
```
Each message in the `messages` array has:
| Field | Type | Description |
|--------------|-------------------|-----------------------------------------------|
| `role` | string | `"user"`, `"assistant"`, or `"tool"` |
| `content` | string or null | Text content of the message |
| `tool_calls` | array or null | Present only on assistant messages with calls |
Each entry in `tool_calls`:
| Field | Type | Description |
|-------------|--------|------------------------------------|
| `name` | string | Function name (e.g. `"bash"`) |
| `arguments` | string | JSON-encoded argument string |
#### Streaming events
After the initial `connected` and `history` frames, the server streams
real-time events as the model generates a response:
**`thinking_start`** -- the model has begun generating (shown as a spinner).
```json
{"type": "thinking_start"}
```
**`thinking_stop`** -- the spinner phase is over.
```json
{"type": "thinking_stop"}
```
**`reasoning`** -- a chunk of chain-of-thought reasoning text.
```json
{"type": "reasoning", "text": "Let me think about this..."}
```
**`content`** -- a chunk of the assistant's visible reply.
```json
{"type": "content", "text": "Here is the answer: "}
```
**`stream_end`** -- the model has finished generating. The client should
finalize any in-progress assistant message.
```json
{"type": "stream_end"}
```
**`tool_info`** -- one or more tool calls that were auto-approved (no user
action required).
```json
{
"type": "tool_info",
"items": [
{
"call_id": "call_abc123",
"header": "bash: ls -la",
"preview": "",
"func_name": "bash",
"approval_label": "bash",
"needs_approval": false,
"error": null
}
]
}
```
**`approve_request`** -- one or more tool calls that require user approval. The
client must respond via `POST /v1/api/approve`.
```json
{
"type": "approve_request",
"items": [
{
"call_id": "call_def456",
"header": "bash: rm -rf /tmp/build",
"preview": "",
"func_name": "bash",
"approval_label": "bash",
"needs_approval": true,
"error": null
}
]
}
```
Each item in `items` (shared by `tool_info` and `approve_request`):
| Field | Type | Description |
|------------------|-------------|--------------------------------------------------|
| `call_id` | string | Unique tool call ID (links chunks to results) |
| `header` | string | Human-readable header line for the tool call |
| `preview` | string | Diff or argument preview (may be empty) |
| `func_name` | string | Function name (e.g. `"bash"`, `"edit_file"`) |
| `approval_label` | string | Display label for the approval prompt |
| `needs_approval` | bool | Whether this call requires explicit approval |
| `error` | string/null | Error description if the call was malformed |
**`tool_output_chunk`** -- incremental streaming output from a bash tool execution. Sent line-by-line as stdout is produced. The `call_id` identifies the specific tool invocation (multiple bash tools may run in parallel).
```json
{"type": "tool_output_chunk", "call_id": "call_abc123", "chunk": "Building project...\n"}
```
**`tool_result`** -- final output from a completed tool execution. The `call_id` matches the corresponding `tool_info`/`approve_request` item and any preceding `tool_output_chunk` events. For bash tools, this arrives after all streaming chunks and includes both stdout and stderr.
```json
{"type": "tool_result", "call_id": "call_abc123", "name": "bash", "output": "file1.py\nfile2.py\n"}
```
**`status`** -- token usage statistics, sent after each model turn.
```json
{
"type": "status",
"prompt_tokens": 1024,
"completion_tokens": 256,
"total_tokens": 1280,
"context_window": 131072,
"pct": 1.0,
"effort": "medium"
}
```
| Field | Type | Description |
|---------------------|--------|----------------------------------------------|
| `prompt_tokens` | int | Tokens in the prompt |
| `completion_tokens` | int | Tokens generated by the model |
| `total_tokens` | int | `prompt_tokens + completion_tokens` |
| `context_window` | int | Total context window size in tokens |
| `pct` | float | Percentage of context window used |
| `effort` | string | Reasoning effort level (`low`/`medium`/`high`) |
**`plan_review`** -- the model is proposing a plan and wants feedback. The
client must respond via `POST /v1/api/plan`.
```json
{"type": "plan_review", "content": "Step 1: ...\nStep 2: ..."}
```
**`info`** -- an informational message (e.g. command output).
```json
{"type": "info", "message": "Session cleared."}
```
**`error`** -- an error message.
```json
{"type": "error", "message": "Error: connection timed out"}
```
**`busy_error`** -- sent when a new message arrives while the model is already
processing.
```json
{"type": "busy_error", "message": "Already processing a request. Please wait."}
```
**`clear_ui`** -- instructs the client to clear all displayed messages (sent
after `/clear` or `/new` commands).
```json
{"type": "clear_ui"}
```
#### Keepalive
The server sends an SSE comment every 5 seconds when no events are pending:
```
: keepalive
```
This prevents proxies and browsers from closing the connection due to
inactivity.
#### Generation mechanism
Each new SSE connection to a workstream increments an internal
`_sse_generation` counter. The previous SSE handler detects the generation
mismatch and exits its event loop, ensuring only one active SSE connection per
workstream at a time. The event queue is drained of stale events before the new
connection begins streaming.
---
### `GET /v1/api/events/global`
Opens a Server-Sent Events stream that broadcasts state-change events across
all workstreams. This is used by the tab bar to display per-workstream activity
indicators.
**Events:**
```json
{"type": "ws_state", "ws_id": "abc123", "state": "thinking"}
```
| Field | Type | Description |
|---------|--------|--------------------------|
| `ws_id` | string | Workstream identifier |
| `state` | string | Current workstream state |
Possible `state` values:
| State | Description |
|-------------|-------------------------------------------------|
| `idle` | No active processing |
| `thinking` | Model is generating a response |
| `running` | Tool execution in progress |
| `attention` | Waiting for user input (approval or plan review)|
| `error` | An error occurred |
**Fan-out pattern:** Each connected client receives its own bounded queue
(`maxsize=500`). A dedicated fan-out thread reads from the shared global queue
and copies each event to every client queue. If a client queue is full, the
event is silently dropped for that client.
**Keepalive:** Same as `/v1/api/events` -- an SSE comment every 5 seconds.
---
### `GET /v1/api/workstreams`
Returns a list of all active workstreams.
**Response:**
```json
{
"workstreams": [
{"id": "abc123", "name": "default", "state": "idle", "session_id": "a1b2c3d4e5f6"},
{"id": "def456", "name": "hacker-news", "state": "thinking", "session_id": "c5d6e7f8a9b0"}
]
}
```
Each workstream object:
| Field | Type | Description |
|--------------|-------------|--------------------------------------------------------|
| `id` | string | Unique workstream routing identifier |
| `name` | string | Display name (alias if set, otherwise `ws-xxxx`) |
| `state` | string | Current state (see state values above) |
| `session_id` | string/null | Session ID of the workstream's `ChatSession`, used for deduplication against `/v1/api/sessions` |
---
### `GET /v1/api/sessions`
Returns a list of saved sessions from the database, ordered by most recently
updated.
**Response:**
```json
{
"sessions": [
{
"session_id": "a1b2c3d4e5f6",
"alias": "refactor",
"title": "JWT Authentication Refactor",
"created": "2026-03-01 10:00:00",
"updated": "2026-03-01 11:30:00",
"message_count": 42
}
]
}
```
Each session object:
| Field | Type | Description |
|-----------------|-------------|--------------------------------------------|
| `session_id` | string | Unique 32-char hex UUID session identifier |
| `alias` | string/null | User-assigned short name |
| `title` | string/null | LLM-generated title |
| `created` | string | ISO timestamp of session creation |
| `updated` | string | ISO timestamp of last message |
| `message_count` | int | Number of messages in the session |
| `node_id` | string/null | Server node that created the session |
| `ws_id` | string/null | Workstream the session belongs to |
---
### `POST /v1/api/send`
Sends a user message to a workstream. Spawns a daemon worker thread that calls
`session.send()` and streams results back via the SSE channel.
**Request body:**
```json
{"message": "Explain how the server works", "ws_id": "abc123"}
```
| Field | Type | Required | Description |
|-----------|--------|----------|-------------------------|
| `message` | string | yes | The user's message text |
| `ws_id` | string | yes | Target workstream ID |
**Response (success):**
```json
{"status": "ok"}
```
**Response (busy):** Returned if the workstream's worker thread is still alive
from a previous request. Also pushes a `busy_error` event to the SSE stream.
```json
{"status": "busy"}
```
**Error responses:**
| Status | Body | Condition |
|--------|------------------------------------|------------------------|
| 400 | `{"error": "Empty message"}` | Message is empty |
| 404 | `{"error": "Unknown workstream"}` | `ws_id` not found |
---
### `POST /v1/api/approve`
Responds to a tool approval request. The SSE stream must have previously sent
an `approve_request` event for the given workstream.
**Request body:**
```json
{"approved": true, "feedback": null, "always": false, "ws_id": "abc123"}
```
| Field | Type | Required | Description |
|------------|-------------|----------|--------------------------------------------------|
| `approved` | bool | yes | `true` to approve, `false` to deny |
| `feedback` | string/null | no | Optional feedback text (sent as denial reason) |
| `always` | bool | no | If `true` and `approved`, enables auto-approve |
| `ws_id` | string | yes | Target workstream ID |
When `always` is `true` and `approved` is `true`, the workstream's WebUI
instance sets `auto_approve = True`, causing all subsequent tool calls to be
automatically approved without prompting.
**Response:**
```json
{"status": "ok"}
```
**Error:** `404` with `{"error": "Unknown workstream"}` if `ws_id` is invalid.
---
### `POST /v1/api/plan`
Responds to a plan review dialog. The SSE stream must have previously sent a
`plan_review` event for the given workstream.
**Request body:**
```json
{"feedback": "", "ws_id": "abc123"}
```
| Field | Type | Required | Description |
|------------|--------|----------|---------------------------------------------------------|
| `feedback` | string | yes | Feedback text; empty string means approval |
| `ws_id` | string | yes | Target workstream ID |
To approve the plan, send an empty string for `feedback`. To reject or request
changes, send a non-empty feedback string (e.g. `"reject"` or specific
revision instructions).
**Response:**
```json
{"status": "ok"}
```
**Error:** `404` with `{"error": "Unknown workstream"}` if `ws_id` is invalid.
---
### `POST /v1/api/command`
Executes a slash command in the given workstream.
**Request body:**
```json
{"command": "/clear", "ws_id": "abc123"}
```
| Field | Type | Required | Description |
|-----------|--------|----------|------------------------------------|
| `command` | string | yes | The slash command (e.g. `/clear`) |
| `ws_id` | string | yes | Target workstream ID |
If the command is `/clear` or `/new`, the server pushes a `clear_ui` SSE event
to instruct the client to reset its message display. If the command is
`/resume`, the server pushes `clear_ui` followed by a `history` event
containing the resumed session's messages.
**Response:**
```json
{"status": "ok"}
```
**Error responses:**
| Status | Body | Condition |
|--------|------------------------------------|----------------------|
| 400 | `{"error": "Empty command"}` | Command is empty |
| 404 | `{"error": "Unknown workstream"}` | `ws_id` not found |
---
### `POST /v1/api/workstreams/new`
Creates a new workstream. The server supports up to 10 concurrent workstreams.
**Request body:**
```json
{"name": "my-ws", "model": "openai"}
```
All fields are optional. The body can be empty or an empty JSON object.
| Field | Type | Default | Description |
|------------------|--------|---------|----------------------------------------------------------------|
| `name` | string | auto | Workstream display name |
| `model` | string | default | Model alias from the registry (`[models.*]`) |
| `auto_approve` | bool | false | Auto-approve all tool calls for this workstream |
| `resume_session` | string | "" | Session ID to resume atomically during creation (empty = fresh)|
**Response (success):**
```json
{"ws_id": "ghi789", "name": "ws-3", "resumed": false, "message_count": 0}
```
| Field | Type | Description |
|-----------------|--------|-----------------------------------------------------|
| `ws_id` | string | Unique ID of the new workstream |
| `name` | string | Auto-generated workstream name |
| `resumed` | bool | Whether a previous session was successfully resumed |
| `message_count` | int | Number of messages in the resumed session (0 if fresh) |
**Error (limit reached):**
```json
{"error": "Maximum of 10 workstreams reached"}
```
Status code: `400`
---
### `POST /v1/api/workstreams/close`
Closes and removes a workstream. The last remaining workstream cannot be
closed.
**Request body:**
```json
{"ws_id": "abc123"}
```
| Field | Type | Required | Description |
|---------|--------|----------|---------------------------|
| `ws_id` | string | yes | Workstream ID to close |
**Response (success):**
```json
{"status": "ok"}
```
**Error (last workstream):**
```json
{"error": "Cannot close last workstream"}
```
Status code: `400`
---
### `OPTIONS` (any path)
Handles CORS preflight requests.
**Response headers:**
```
Access-Control-Allow-Origin: *
Access-Control-Allow-Methods: GET, POST, OPTIONS
Access-Control-Allow-Headers: Content-Type
```
Status code: `200` with an empty body.
---
## Error Handling
| Condition | Behavior |
|------------------------------------|------------------------------------------------------------|
| Malformed or unparseable JSON body | Treated as an empty dict `{}`; missing fields use defaults |
| Unknown `ws_id` | `404` with `{"error": "Unknown workstream"}` |
| Unknown path (GET or POST) | `404` with plain-text body `Not found` |
| Empty `message` on `/v1/api/send` | `400` with `{"error": "Empty message"}` |
| Empty `command` on `/v1/api/command` | `400` with `{"error": "Empty command"}` |
| Rate limit exceeded | `429` with `Retry-After` header (see below) |
### `429 Too Many Requests`
Returned when the per-IP rate limiter rejects a request. `/health` and
`/metrics` are exempt.
**Response headers:**
```
Retry-After: 2
```
**Response body:**
```json
{"error": "Rate limit exceeded", "retry_after": 2}
```
| Field | Type | Description |
|---------------|--------|------------------------------------------------|
| `error` | string | `"Rate limit exceeded"` |
| `retry_after` | number | Seconds until the client should retry |
---
## SSE Reconnection
The embedded JavaScript client implements exponential backoff for SSE
reconnection:
| Parameter | Value |
|--------------------|-------------------------------------------|
| Base delay | 1 second |
| Backoff multiplier | 2x on each consecutive failure |
| Maximum delay | 30 seconds |
| Reset | Delay resets to 1 second on first success |
On reconnect, the server replays the full conversation history via the
`history` event, so the client can rebuild its UI state without data loss. The
same reconnection strategy applies to both the per-workstream SSE stream
(`/v1/api/events`) and the global state stream (`/v1/api/events/global`).
---
## Observability
### `GET /health`
Returns server health status. Always returns `200 OK` while the server process
is running. `"status": "degraded"` indicates the server is up but the LLM
backend is unreachable. Suitable for load-balancer health checks and Kubernetes
liveness probes.
**Response:** `application/json`
```json
{
"status": "ok",
"version": "0.3.5",
"node_id": "worker-01_a3f2",
"uptime_seconds": 3614.72,
"model": "llama-3.1-70b-instruct",
"workstreams": {
"total": 2,
"idle": 1,
"thinking": 1,
"running": 0,
"attention": 0,
"error": 0
},
"backend": {
"status": "up",
"circuit_state": "closed"
}
}
```
| Field | Type | Description |
|-------|------|-------------|
| `status` | string | `"ok"` or `"degraded"` (degraded when backend unreachable) |
| `version` | string | turnstone server version |
| `node_id` | string | Server-generated node identity (`{hostname}_{4hex}`) |
| `uptime_seconds` | number | Seconds since the server process started |
| `model` | string | Model name detected or configured at startup |
| `workstreams.total` | integer | Total active workstreams |
| `workstreams.idle` | integer | Workstreams waiting for user input |
| `workstreams.thinking` | integer | Workstreams with LLM currently streaming |
| `workstreams.running` | integer | Workstreams executing tools |
| `workstreams.attention` | integer | Workstreams blocked on approval or plan review |
| `workstreams.error` | integer | Workstreams in error state |
| `backend.status` | string | `"up"` or `"down"` — LLM backend reachability |
| `backend.circuit_state` | string | `"closed"`, `"open"`, or `"half_open"` |
---
### `GET /metrics`
Returns operational metrics in **Prometheus text exposition format v0.0.4**.
Compatible with Prometheus `scrape_configs`, VictoriaMetrics, Grafana Agent,
and any other OpenMetrics-compatible collector.
**Response:** `text/plain; version=0.0.4; charset=utf-8`
#### Prometheus scrape config example
```yaml
scrape_configs:
- job_name: turnstone
static_configs:
- targets: ["localhost:8080"]
metrics_path: /metrics
```
#### Metrics reference
| Metric | Type | Labels | Description |
|--------|------|--------|-------------|
| `turnstone_build_info` | gauge | `version`, `model` | Always 1; carries version/model as labels |
| `turnstone_uptime_seconds` | gauge | — | Seconds since server start |
| `turnstone_workstreams_active_total` | gauge | — | Number of active workstreams |
| `turnstone_workstreams_by_state` | gauge | `state` | Workstream count per state (`idle`, `thinking`, `running`, `attention`, `error`) |
| `turnstone_http_requests_total` | counter | `method`, `endpoint`, `status_code` | Total HTTP requests handled |
| `turnstone_http_request_duration_seconds` | histogram | `method`, `endpoint` | Request latency distribution (11 buckets: 5ms10s) |
| `turnstone_messages_sent_total` | counter | — | User messages dispatched to the AI |
| `turnstone_tokens_total` | counter | `type` | Tokens consumed (`type="prompt"` or `type="completion"`) |
| `turnstone_tool_calls_total` | counter | `tool` | Tool executions by name (e.g. `tool="bash"`) |
| `turnstone_errors_total` | counter | — | Errors reported by workstreams |
| `turnstone_context_window_used_ratio` | gauge | — | Last known fraction of context window in use (0.01.0) |
| `turnstone_sse_connections_active` | gauge | — | Number of open SSE connections |
| `turnstone_ratelimit_rejected_total` | counter | — | Requests rejected by the per-IP rate limiter |
| `turnstone_backend_up` | gauge | — | LLM backend reachability (1 = up, 0 = down) |
| `turnstone_circuit_state` | gauge | — | Circuit breaker state (0 = closed, 1 = open, 2 = half_open) |
| `turnstone_workstreams_evicted_total` | counter | — | Workstreams auto-evicted when at capacity |
#### Example output
```
# HELP turnstone_build_info Server version and model info
# TYPE turnstone_build_info gauge
turnstone_build_info{version="0.2.0",model="llama-3.1-70b-instruct"} 1
# HELP turnstone_uptime_seconds Server uptime in seconds
# TYPE turnstone_uptime_seconds gauge
turnstone_uptime_seconds 3614.72
# HELP turnstone_workstreams_active_total Number of active workstreams
# TYPE turnstone_workstreams_active_total gauge
turnstone_workstreams_active_total 1
# HELP turnstone_http_requests_total Total HTTP requests handled
# TYPE turnstone_http_requests_total counter
turnstone_http_requests_total{method="GET",endpoint="/health",status_code="200"} 42
turnstone_http_requests_total{method="GET",endpoint="/metrics",status_code="200"} 7
turnstone_http_requests_total{method="POST",endpoint="/v1/api/send",status_code="200"} 18
# HELP turnstone_tokens_total Total tokens consumed
# TYPE turnstone_tokens_total counter
turnstone_tokens_total{type="prompt"} 84320
turnstone_tokens_total{type="completion"} 12150
# HELP turnstone_tool_calls_total Total tool executions by name
# TYPE turnstone_tool_calls_total counter
turnstone_tool_calls_total{tool="bash"} 7
turnstone_tool_calls_total{tool="read_file"} 3
```