Compare commits

..

7 Commits

Author SHA1 Message Date
Patrick Buckley 2ab60853f5 chore: bump version to 1.2.2 2026-04-12 20:41:42 -07:00
Patrick Buckley fed5b96a6f fix: universal tool_call/tool_result orphan detection for OpenAI-comp… (#346)
* fix: universal tool_call/tool_result orphan detection for OpenAI-compat providers

The Anthropic provider had orphan detection for mismatched tool_call ↔
tool_result pairs, but OpenAI-compatible providers (Chat Completions,
Google, Responses API) had none. When an Anthropic model runs behind
an OpenAI-compat API (e.g. Azure) or cancellation creates orphans,
the API rejects the malformed request.

- Rewrite sanitize_messages() with orphan detection: synthesize error
  tool results for unmatched tool_calls, drop tool results with no
  matching tool_call, fill empty tool_call IDs with positional remap
- Call sanitize_messages() from Responses API _convert_messages()

* fix: address review feedback on orphan detection

- Track answered IDs per-turn (local_answered) instead of scanning
  all of out, preventing false matches from reused IDs across turns
- Drop empty-ID tool results that have no remap entry instead of
  passing them through with invalid empty tool_call_id
- Increment empty_result_idx for every empty result, not just remapped
- Remove dead result_ids peek-ahead code
- Add test for repeated tool_call IDs across turns
2026-04-12 20:41:28 -07:00
Patrick Buckley 4a65535e00 fix: accurate token usage tracking for compaction across all providers (#345)
* fix: accurate token usage tracking for compaction across all providers

Anthropic's input_tokens excluded cached tokens, causing massive
under-reporting (e.g. 327 vs 9000 actual) when prompt caching was
active. This prevented auto-compaction from triggering.

- Normalize Anthropic prompt_tokens to total input (input_tokens +
  cache_creation + cache_read), matching OpenAI semantics
- Reset _last_usage per API call so tool-chain iterations get fresh
  usage instead of max()-merging with stale values
- Add mid-turn compaction check during tool chains to prevent context
  overflow before end-of-turn
- Anchor _remaining_token_budget() on provider-reported prompt_tokens
  with local estimates only for the delta since last API call
- Improve _msg_char_count() to include structural overhead (role,
  tool_call_id, tool call IDs) and handle image tokens in calibration
- Emit status after every API call, not just end of turn

* fix: defensive null coercion and index clamping from review feedback

- Add `or 0` to all getattr calls for input_tokens/output_tokens in
  Anthropic provider (streaming + non-streaming) to handle SDK nulls
- Use getattr for non-streaming input_tokens/output_tokens instead of
  direct attribute access for consistency
- Clamp _calibrated_msg_count with min() in _remaining_token_budget()
  to prevent stale state from over-slicing after compaction
2026-04-12 20:41:28 -07:00
Patrick Buckley 519b86f56e chore: bump version to 1.2.1 2026-04-08 18:06:12 -07:00
Patrick Buckley 024a2e98d2 fix(ui): remove broken hint animation and restore card toggle
The ws-check-hint animation clobbered the fadein's forwards fill,
making the checkbox invisible for 0.6s on card-body click — appearing
as a deselect-then-reselect. Remove the hint, the unused role=checkbox
on the card, and restore the original symmetric toggle behavior.
2026-04-08 18:05:41 -07:00
Patrick Buckley e9c141aba5 fix(ui): improve delete workstream UX and accessibility (#339)
* fix(ui): improve delete workstream UX and accessibility

Card body click no longer deselects (prevents confusing red border loss);
checkbox pulse hint guides users to deselect affordance. Adds keyboard
navigation, aria-labels, hover feedback, animations, and neutral Close
button styling after deletion.

* fix(ui): remove duplicate a11y checkbox from delete-mode cards

Hide the visual checkbox from the a11y tree and tab order so the card
(role=checkbox) is the sole keyboard/screen-reader target. Addresses
Copilot review feedback about nested interactive elements.
2026-04-08 17:20:45 -07:00
Patrick Buckley b038dbdd5b chore(deps): bump lacme to >=1.0.5 (cryptography security update) 2026-04-08 16:48:35 -07:00
484 changed files with 18381 additions and 160694 deletions
+2 -2
View File
@@ -20,10 +20,10 @@ TURNSTONE_JWT_SECRET=changeme-to-32-bytes-of-hex
# -- Database ------------------------------------------------------------------
# Single-node default is SQLite (zero config). Set these for PostgreSQL:
# TURNSTONE_DB_BACKEND=postgresql
# DB_BACKEND=postgresql
# POSTGRES_USER=turnstone
# POSTGRES_PASSWORD=changeme
# TURNSTONE_DB_URL=postgresql+psycopg://turnstone:changeme@postgres:5432/turnstone
# DATABASE_URL=postgresql+psycopg://turnstone:changeme@postgres:5432/turnstone
# -- Ports ---------------------------------------------------------------------
# SERVER_PORT=8080
+4 -14
View File
@@ -43,16 +43,9 @@ jobs:
- uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6
with:
python-version: ${{ matrix.python-version }}
# Node is required by tests/test_renderer_js.py — without
# explicit setup, that suite silently skips if the runner
# image happens not to ship Node, masking regressions in
# the browser-side renderer.
- uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6
with:
node-version: "24"
- run: pip install -e ".[test]"
- run: pytest tests/ -m "not live" --cov=turnstone --cov-report=term-missing --cov-report=xml -q
- uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
- uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7
if: always()
with:
name: coverage-${{ matrix.python-version }}
@@ -79,9 +72,6 @@ jobs:
- uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6
with:
python-version: "3.14"
- uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6
with:
node-version: "24"
- run: pip install -e ".[test,postgres]"
- run: pytest tests/ -m "not live" --storage-backend=postgresql -q
env:
@@ -138,7 +128,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
- uses: astral-sh/setup-uv@08807647e7069bb48b6ef5acd8ec9567f424441b # v8.1.0
- uses: astral-sh/setup-uv@37802adc94f370d6bfd71619e3f0bf239e1f3b78 # v7
with:
uv-version: "0.9.18"
- run: uv lock --check
@@ -147,7 +137,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
- uses: astral-sh/setup-uv@08807647e7069bb48b6ef5acd8ec9567f424441b # v8.1.0
- uses: astral-sh/setup-uv@37802adc94f370d6bfd71619e3f0bf239e1f3b78 # v7
with:
uv-version: "0.9.18"
- uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6
@@ -165,7 +155,7 @@ jobs:
working-directory: sdk/typescript
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
- uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6
- uses: actions/setup-node@53b83947a5a98c8d113130e565377fae1a50d02f # v6
with:
node-version: "24"
- run: npm ci
+1 -1
View File
@@ -72,7 +72,7 @@ jobs:
- name: Build and push
if: steps.tag.outputs.skip == 'false'
uses: docker/build-push-action@bcafcacb16a39f128d818304e6c9c0c18556b85f # v7
uses: docker/build-push-action@d08e5c354a6adb9ed34480a06d141179aa583294 # v7
with:
context: .
push: true
+22
View File
@@ -0,0 +1,22 @@
name: Docker Security Scan
on:
push:
branches: [main, "stable/*"]
schedule:
- cron: "0 6 * * 1" # Weekly Monday 06:00 UTC
permissions:
contents: read
jobs:
scan:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
- run: docker build -t turnstone:scan .
- uses: aquasecurity/trivy-action@57a97c7e7821a5776cebc9bb87c984fa69cba8f1 # 0.35.0
with:
image-ref: "turnstone:scan"
severity: "HIGH,CRITICAL"
exit-code: "1"
+2 -2
View File
@@ -44,12 +44,12 @@ jobs:
if: steps.tag.outputs.skip == 'false'
- run: python -m build
if: steps.tag.outputs.skip == 'false'
- uses: pypa/gh-action-pypi-publish@cef221092ed1bacb1cc03d23a2d87d1d172e277b # release/v1
- uses: pypa/gh-action-pypi-publish@ed0c53931b1dc9bd32cbe73a98c7f6766f8a527e # release/v1
if: steps.tag.outputs.skip == 'false'
- name: Create GitHub Release
if: steps.tag.outputs.skip == 'false'
uses: softprops/action-gh-release@b4309332981a82ec1c5618f44dd2e27cc8bfbfda # v3
uses: softprops/action-gh-release@153bb8e04406b158c6c84fc1615b65b24149a1fe # v2
with:
tag_name: ${{ steps.tag.outputs.tag }}
generate_release_notes: true
+1 -1
View File
@@ -48,7 +48,7 @@ jobs:
id: detect
run: |
updates=()
for lib in katex hljs mermaid hls; do
for lib in katex hljs mermaid; do
version=$(grep -oE "${lib}-[0-9.]+" pyproject.toml | head -1 | sed "s/${lib}-//")
[[ -z "$version" ]] && continue
[[ -d "turnstone/shared_static/${lib}-${version}" ]] && continue
-2
View File
@@ -21,5 +21,3 @@ PROGRESS.md
.coverage
tools/skill_audit_analysis/data/
tools/skill_audit_analysis/output/
design_ideas/
.claude/
+40
View File
@@ -0,0 +1,40 @@
# libexpat integer overflow — no fix available in Debian repos yet
# https://avd.aquasec.com/nvd/cve-2026-25210
# Review: remove this entry once a patched libexpat1 is published
CVE-2026-25210
# ncurses buffer overflow — no fix in Debian 13 repos yet
# Affects libncursesw6, libtinfo6, ncurses-base, ncurses-bin
# https://avd.aquasec.com/nvd/cve-2025-69720
CVE-2025-69720
# nghttp2 DoS via malformed HTTP/2 frames — no fix in Debian 13 repos yet
# Affects libnghttp2-14
# https://avd.aquasec.com/nvd/cve-2026-27135
CVE-2026-27135
# systemd arbitrary code execution via spurious IPC — no fix in Debian 13 repos yet
# Affects libsystemd0, libudev1
# https://avd.aquasec.com/nvd/cve-2026-29111
CVE-2026-29111
# glibc iconv() DoS — fix_deferred, no patched libc in Debian 13 yet
# Affects libc-bin, libc6
# https://avd.aquasec.com/nvd/cve-2026-4046
CVE-2026-4046
# minimatch ReDoS — transitive npm dep (MCP server), no direct exposure
# https://avd.aquasec.com/nvd/cve-2026-27903
CVE-2026-27903
# https://avd.aquasec.com/nvd/cve-2026-27904
CVE-2026-27904
# picomatch ReDoS — transitive npm dep, no direct exposure
# https://avd.aquasec.com/nvd/cve-2026-33671
CVE-2026-33671
# node-tar path traversal — transitive npm dep, not used to extract untrusted archives
# https://avd.aquasec.com/nvd/cve-2026-29786
CVE-2026-29786
# https://avd.aquasec.com/nvd/cve-2026-31802
CVE-2026-31802
-1901
View File
File diff suppressed because it is too large Load Diff
+1 -9
View File
@@ -26,15 +26,7 @@ transferring ownership.
```
python -m venv .venv
source .venv/bin/activate
pip install -e ".[test,dev]"
```
The `dev` extra installs `ruff` and `mypy`. Before pushing, run:
```
ruff check turnstone tests
mypy turnstone
pytest
pip install -e ".[test]"
```
## Guidelines
+3 -6
View File
@@ -8,17 +8,14 @@ FROM python:3.14-slim
LABEL org.opencontainers.image.title="turnstone" \
org.opencontainers.image.description="Multi-node AI orchestration platform"
COPY --from=ghcr.io/astral-sh/uv:0.11.14 /uv /usr/local/bin/uv
COPY --from=ghcr.io/astral-sh/uv:0.11.3 /uv /usr/local/bin/uv
# Remove the slim image's man page exclusion so man-db has actual content
RUN rm -f /etc/dpkg/dpkg.cfg.d/docker
# System dependencies: psycopg (libpq5), developer tooling for agent workflows.
# ripgrep is the preferred backend for the search tool — natively bounds
# per-line, per-file, and per-filesize so pathological inputs (minified
# bundles, training-data JSONL with multi-MB single records) can't OOM us.
# System dependencies: psycopg (libpq5), developer tooling for agent workflows
RUN apt-get update && apt-get upgrade -y && apt-get install -y --no-install-recommends \
libpq5 git curl jq man-db manpages procps file ripgrep \
libpq5 git curl jq man-db manpages procps file \
&& rm -rf /var/lib/apt/lists/*
# Node.js LTS (for npx-based MCP servers like @modelcontextprotocol/server-github)
+4 -4
View File
@@ -55,7 +55,7 @@ The wizard supports two deployment modes:
```
$ turnstone-bootstrap
Turnstone Bootstrap Wizard v1.5.0
Turnstone Bootstrap Wizard v0.5.4
────────────────────────────────────────────────
Which provider for this wizard?
@@ -87,6 +87,6 @@ $ turnstone-bootstrap
## See Also
- [Docker Deployment](docs/docker.md) — manual compose setup and profiles
- [Security](docs/security.md) — auth architecture and token types
- [Governance](docs/governance.md) — roles, policies, and templates
- [Docker Deployment](docker.md) — manual compose setup and profiles
- [Security](security.md) — auth architecture and token types
- [Governance](governance.md) — roles, policies, and templates
+6 -15
View File
@@ -8,7 +8,7 @@
Multi-node AI orchestration platform. Deploy tool-using AI agents across a cluster of servers with direct HTTP routing, interactive interfaces, and enterprise governance.
<p align="center">
<img src="docs/assets/hero.png" alt="Turnstone coordinator — parallel tool batches with judge-graded approval and child workstream tracking" width="960"/>
<img src="docs/assets/hero.png" alt="Turnstone console — multi-workstream AI orchestration with mermaid diagrams" width="960"/>
</p>
Named after the [Ruddy Turnstone](https://en.wikipedia.org/wiki/Ruddy_turnstone) (*Arenaria interpres*) — a shorebird that flips stones to discover what's hiding underneath.
@@ -53,15 +53,6 @@ pip install turnstone[console]
turnstone-console --port 8090
```
For PostgreSQL (recommended for production):
```bash
pip install turnstone[postgres]
export TURNSTONE_DB_BACKEND=postgresql
export TURNSTONE_DB_URL="postgresql+psycopg://user:pass@localhost:5432/turnstone"
turnstone-server --port 8080 --base-url http://localhost:8000/v1
```
### Docker
```bash
@@ -84,20 +75,20 @@ with TurnstoneServer("http://localhost:8080", token="tok_xxx") as client:
## Tools
Built-in tools for shell, files, search, web, memory, notifications, and autonomous sub-agents — plus external tools via [MCP](https://modelcontextprotocol.io/) with native deferred loading. See [docs/tools.md](docs/tools.md) for the full reference and [docs/mcp-registry.md](docs/mcp-registry.md) for MCP configuration.
Built-in tools for shell, files, search, web, memory, notifications, and autonomous sub-agents — plus external tools via [MCP](https://modelcontextprotocol.io/) with native deferred loading. See [docs/tools.md](docs/tools.md) for the full reference and [docs/mcp.md](docs/mcp.md) for MCP configuration.
## Architecture
**Single-node**: Client → Server (direct HTTP + SSE). No external dependencies beyond the database.
**Multi-node**: Client → Console (rendezvous routing proxy) → Server nodes. The console picks the target node for each workstream via rendezvous (HRW) hashing over the live service registry — pure function of `(ws_id, live_nodes)`, no stored bucket state, deterministic across readers. A node join or drop only re-routes the keys that score highest on the affected node.
**Multi-node**: Client → Console (hash ring routing proxy) → Server nodes. The console maintains a 65536-entry bucket cache for O(1) workstream routing. A rebalancer daemon redistributes buckets when nodes join or leave.
| Component | Purpose |
|-----------|---------|
| `turnstone` | Terminal CLI (REPL) |
| `turnstone-server` | Web UI + REST API + SSE events |
| `turnstone-console` | Cluster dashboard + routing proxy + admin panel |
| `turnstone-channel` | Channel gateway (Discord and Slack adapters) |
| `turnstone-channel` | Channel gateway (Discord, with adapters for Slack/Teams planned) |
| `turnstone-admin` | User/token management CLI |
| `turnstone-eval` | Eval harness for prompt/tool optimization |
| `turnstone-bootstrap` | LLM-guided setup wizard |
@@ -117,7 +108,7 @@ UML diagrams in [`docs/diagrams/`](docs/diagrams/):
| [Console Data Flow](docs/diagrams/png/11-console-data-flow.png) | Dashboard data collection |
| [Deployment](docs/diagrams/png/12-deployment.png) | Docker Compose topology |
| [Auth](docs/diagrams/png/15-auth-architecture.png) | JWT, scopes, login flows |
| [Channels](docs/diagrams/png/16-channel-architecture.png) | Discord / Slack adapters + routing |
| [Channels](docs/diagrams/png/16-channel-architecture.png) | Discord adapter + routing |
| [Judge](docs/diagrams/png/22-judge-architecture.png) | Intent validation pipeline |
| [OIDC](docs/diagrams/png/25-oidc-architecture.png) | SSO authorization code flow |
@@ -136,7 +127,7 @@ UML diagrams in [`docs/diagrams/`](docs/diagrams/):
| Console dashboard | [docs/console.md](docs/console.md) |
| Eval harness | [docs/eval.md](docs/eval.md) |
| Tools reference | [docs/tools.md](docs/tools.md) |
| MCP integration | [docs/mcp-registry.md](docs/mcp-registry.md) |
| MCP integration | [docs/mcp.md](docs/mcp.md) |
## Requirements
+12 -12
View File
@@ -9,7 +9,7 @@
# Usage:
# Infra only: docker compose up
# Single node: docker compose --profile production up
# Production (PG): TURNSTONE_DB_BACKEND=postgresql docker compose --profile production up
# Production (PG): DB_BACKEND=postgresql docker compose --profile production up
# 10-node cluster: docker compose --profile cluster up
# =============================================================================
@@ -57,8 +57,8 @@ services:
deploy:
resources:
limits:
memory: 4G
cpus: '4.0'
memory: 1G
cpus: '1.0'
restart: unless-stopped
# -------------------------------------------------------------------
@@ -94,8 +94,8 @@ services:
- TURNSTONE_JWT_SECRET=${TURNSTONE_JWT_SECRET:?Set TURNSTONE_JWT_SECRET in .env}
- MODEL=${MODEL:-}
- MCP_CONFIG=${MCP_CONFIG:-}
- TURNSTONE_DB_BACKEND=${TURNSTONE_DB_BACKEND:-sqlite}
- TURNSTONE_DB_URL=${TURNSTONE_DB_URL:-}
- TURNSTONE_DB_BACKEND=${DB_BACKEND:-sqlite}
- TURNSTONE_DB_URL=${DATABASE_URL:-}
- TURNSTONE_NODE_ID=${TURNSTONE_NODE_ID:-}
- TURNSTONE_ADVERTISE_URL=${TURNSTONE_ADVERTISE_URL:-http://server:8080}
extra_hosts:
@@ -131,8 +131,8 @@ services:
environment:
# Generate with: python -c "import secrets; print(secrets.token_hex(32))"
- TURNSTONE_JWT_SECRET=${TURNSTONE_JWT_SECRET:?Set TURNSTONE_JWT_SECRET in .env}
- TURNSTONE_DB_BACKEND=${TURNSTONE_DB_BACKEND:-sqlite}
- TURNSTONE_DB_URL=${TURNSTONE_DB_URL:-}
- TURNSTONE_DB_BACKEND=${DB_BACKEND:-sqlite}
- TURNSTONE_DB_URL=${DATABASE_URL:-}
- TURNSTONE_CONSOLE_URL=http://console:8090
networks:
- turnstone-net
@@ -165,8 +165,8 @@ services:
- TURNSTONE_DISCORD_GUILD=${TURNSTONE_DISCORD_GUILD:-0}
# Generate with: python -c "import secrets; print(secrets.token_hex(32))"
- TURNSTONE_JWT_SECRET=${TURNSTONE_JWT_SECRET:?Set TURNSTONE_JWT_SECRET in .env}
- TURNSTONE_DB_BACKEND=${TURNSTONE_DB_BACKEND:-postgresql}
- TURNSTONE_DB_URL=${TURNSTONE_DB_URL:-postgresql+psycopg://${POSTGRES_USER:-turnstone}:${POSTGRES_PASSWORD:-turnstone}@postgres:5432/turnstone}
- TURNSTONE_DB_BACKEND=${DB_BACKEND:-postgresql}
- TURNSTONE_DB_URL=${DATABASE_URL:-postgresql+psycopg://${POSTGRES_USER:-turnstone}:${POSTGRES_PASSWORD:-turnstone}@postgres:5432/turnstone}
- TURNSTONE_CHANNEL_ADVERTISE_URL=http://channel:8091
networks:
- turnstone-net
@@ -215,8 +215,8 @@ services:
TURNSTONE_JWT_SECRET: ${TURNSTONE_JWT_SECRET:?Set TURNSTONE_JWT_SECRET in .env}
MODEL: ${MODEL:-}
MCP_CONFIG: ${MCP_CONFIG:-}
TURNSTONE_DB_BACKEND: ${TURNSTONE_DB_BACKEND:-postgresql}
TURNSTONE_DB_URL: ${TURNSTONE_DB_URL:-postgresql+psycopg://${POSTGRES_USER:-turnstone}:${POSTGRES_PASSWORD:?}@postgres:5432/turnstone}
TURNSTONE_DB_BACKEND: ${DB_BACKEND:-postgresql}
TURNSTONE_DB_URL: ${DATABASE_URL:-postgresql+psycopg://${POSTGRES_USER:-turnstone}:${POSTGRES_PASSWORD:?}@postgres:5432/turnstone}
TURNSTONE_NODE_ID: node-1
TURNSTONE_ADVERTISE_URL: http://server-1:8080
extra_hosts: ["host.docker.internal:host-gateway"]
@@ -231,7 +231,7 @@ services:
start_period: 60s
deploy:
resources:
limits: { memory: 4G, cpus: '4' }
limits: { memory: 384M, cpus: '0.5' }
restart: unless-stopped
server-2:
+1 -1
View File
@@ -7,6 +7,6 @@ appVersion: "0.3.0"
dependencies:
- name: postgresql
version: ~18.6.0
version: ~18.5.0
repository: https://charts.bitnami.com/bitnami
condition: postgresql.enabled
+37 -211
View File
@@ -229,12 +229,12 @@ below.
---
### `GET /v1/api/workstreams/{ws_id}/events`
### `GET /v1/api/events?ws_id=<id>`
Opens a Server-Sent Events stream scoped to a single workstream. The connection
remains open indefinitely; the server pushes events as they occur.
**Path parameters:**
**Query parameters:**
| Parameter | Type | Required | Description |
|-----------|--------|----------|----------------------------|
@@ -281,7 +281,6 @@ Each message in the `messages` array has:
| `role` | string | `"user"`, `"assistant"`, or `"tool"` |
| `content` | string or null | Text content of the message |
| `tool_calls` | array or null | Present only on assistant messages with calls |
| `reasoning` | string (optional) | Concatenated reasoning / chain-of-thought text on assistant turns whose `provider_data` carried reasoning-bearing blocks (Anthropic `thinking`, OpenAI Responses `reasoning`, or synthetic `reasoning_text` from local-model servers). Present only when the active model's `surface_persisted_reasoning` flag is True. |
Each entry in `tool_calls`:
@@ -326,44 +325,6 @@ finalize any in-progress assistant message.
{"type": "stream_end"}
```
**`state_change`** -- the worker thread transitioned to a new state. Drives
the client's busy-mode (composer in send vs. stop, spinner indicators,
auto-focus on idle). Sent live during normal operation AND on every fresh
SSE subscribe (so a mid-stream page refresh restores the correct composer
state without waiting for the next live transition).
```json
{"type": "state_change", "state": "running"}
```
| Field | Type | Description |
|----------|--------|----------------------------------------------------------------------|
| `state` | string | One of `"running"`, `"thinking"`, `"attention"`, `"idle"`, `"error"` |
**`in_progress_snapshot`** -- one-shot replay of the in-progress turn's
content + reasoning text-so-far when this client connects mid-stream.
Lets a refreshing browser tab restore partial assistant text immediately
instead of waiting for the response to complete. Yielded once after the
kind-specific replay phase (history + pending), only when at least one
of `content` / `reasoning` is non-empty. Both halves render into the same
assistant bubble the live `content` / `reasoning` events would target;
clients should treat the snapshot as idempotent (skip overwrite if the
current local buffer is already a superset prefix — covers EventSource
auto-reconnect re-replays).
```json
{
"type": "in_progress_snapshot",
"content": "Here is the answer so far: it depends on ",
"reasoning": "The user is asking about a comparison; let me think about..."
}
```
| Field | Type | Description |
|--------------|--------|------------------------------------------------------------|
| `content` | string | Joined assistant content text accumulated this turn |
| `reasoning` | string | Joined reasoning / chain-of-thought text accumulated |
**`tool_info`** -- one or more tool calls that were auto-approved (no user
action required).
@@ -385,7 +346,7 @@ action required).
```
**`approve_request`** -- one or more tool calls that require user approval. The
client must respond via `POST /v1/api/workstreams/{ws_id}/approve`.
client must respond via `POST /v1/api/approve`.
```json
{
@@ -489,7 +450,7 @@ after `/clear` or `/new` commands).
```
**`cancelled`** -- a cancel request was acknowledged (via the Stop button or
`POST /v1/api/workstreams/{ws_id}/cancel`). This signals that cancellation is in progress, not
`POST /v1/api/cancel`). This signals that cancellation is in progress, not
that it is complete. The worker thread may still be finishing — wait for
`stream_end` before transitioning to a ready state. The client should clear
any in-progress assistant rendering but not re-enable the send button until
@@ -561,13 +522,7 @@ Each SSE connection to a workstream receives its own delivery queue. Events
produced by the worker thread are fanned out to all registered listener queues,
so multiple consumers (browser, console proxy, SDK) can connect
simultaneously and each receives every event. On reconnect the client receives
the kind-specific replay (`connected` + `status` + `history` + pending
approval / plan for interactive; `connected` + `status` + pending for coord)
followed by a `state_change` carrying the current worker state and an
optional `in_progress_snapshot` carrying any partial content / reasoning
buffered for the in-progress turn — so a mid-stream refresh restores both
the busy-mode UI and the partial assistant text without waiting for the
response to complete.
a full history replay, so no catch-up mechanism is needed.
---
@@ -603,7 +558,7 @@ Possible `state` values:
and copies each event to every client queue. If a client queue is full, the
event is silently dropped for that client.
**Keepalive:** Same as `/v1/api/workstreams/{ws_id}/events` -- an SSE comment every 5 seconds.
**Keepalive:** Same as `/v1/api/events` -- an SSE comment every 5 seconds.
---
@@ -616,8 +571,8 @@ Returns a list of all active workstreams.
```json
{
"workstreams": [
{"ws_id": "abc123", "name": "default", "state": "idle"},
{"ws_id": "def456", "name": "hacker-news", "state": "thinking"}
{"id": "abc123", "name": "default", "state": "idle"},
{"id": "def456", "name": "hacker-news", "state": "thinking"}
]
}
```
@@ -626,7 +581,7 @@ Each workstream object:
| Field | Type | Description |
|--------------|-------------|--------------------------------------------------------|
| `ws_id` | string | Unique workstream routing identifier |
| `id` | string | Unique workstream routing identifier |
| `name` | string | Display name (alias if set, otherwise `ws-xxxx`) |
| `state` | string | Current state (see state values above) |
@@ -699,26 +654,21 @@ Each skill summary:
---
### `POST /v1/api/workstreams/{ws_id}/send`
### `POST /v1/api/send`
Sends a user message to a workstream. Spawns a daemon worker thread that calls
`session.send()` and streams results back via the SSE channel.
**Path parameters:**
| Parameter | Type | Required | Description |
|-----------|--------|----------|----------------------|
| `ws_id` | string | yes | Target workstream ID |
**Request body:**
```json
{"message": "Explain how the server works"}
{"message": "Explain how the server works", "ws_id": "abc123"}
```
| Field | Type | Required | Description |
|-----------|--------|----------|-------------------------|
| `message` | string | yes | The user's message text |
| `ws_id` | string | yes | Target workstream ID |
**Response (success):**
@@ -742,21 +692,15 @@ from a previous request. Also pushes a `busy_error` event to the SSE stream.
---
### `POST /v1/api/workstreams/{ws_id}/approve`
### `POST /v1/api/approve`
Responds to a tool approval request. The SSE stream must have previously sent
an `approve_request` event for the given workstream.
**Path parameters:**
| Parameter | Type | Required | Description |
|-----------|--------|----------|----------------------|
| `ws_id` | string | yes | Target workstream ID |
**Request body:**
```json
{"approved": true, "feedback": null, "always": false}
{"approved": true, "feedback": null, "always": false, "ws_id": "abc123"}
```
| Field | Type | Required | Description |
@@ -764,6 +708,7 @@ an `approve_request` event for the given workstream.
| `approved` | bool | yes | `true` to approve, `false` to deny |
| `feedback` | string/null | no | Optional feedback text (sent as denial reason) |
| `always` | bool | no | If `true` and `approved`, enables auto-approve |
| `ws_id` | string | yes | Target workstream ID |
When `always` is `true` and `approved` is `true`, the workstream's WebUI
instance sets `auto_approve = True`, causing all subsequent tool calls to be
@@ -844,7 +789,7 @@ containing the resumed session's messages.
---
### `POST /v1/api/workstreams/{ws_id}/cancel`
### `POST /v1/api/cancel`
Cancels the active generation in a workstream. Sets a cooperative cancellation
flag that is checked at multiple points in the generation loop (per streaming
@@ -867,20 +812,15 @@ for the orphaned thread. Use force cancel when cooperative cancel has not
resolved within a few seconds — the web UI offers this as a "Force Stop"
button automatically.
**Path parameters:**
| Parameter | Type | Required | Description |
|-----------|--------|----------|----------------------|
| `ws_id` | string | yes | Target workstream ID |
**Request body:**
```json
{"force": false}
{"ws_id": "abc123", "force": false}
```
| Field | Type | Required | Description |
|--------|--------|----------|----------------------|
| `ws_id`| string | yes | Target workstream ID |
| `force`| bool | no | Abandon stuck worker immediately (default: `false`) |
**Response:**
@@ -902,15 +842,6 @@ button automatically.
Creates a new workstream. The server supports up to 10 concurrent workstreams.
The endpoint accepts **either** `application/json` (legacy shape) **or**
`multipart/form-data` when you want to upload attachments at creation
time. Multipart requests carry one `meta` field containing the JSON body
shown below plus zero-or-more `file` parts; each file is validated and
reserved onto the new workstream's first turn before the dispatch worker
runs, so queued multimodal turns cannot lose files to racing sends. If
validation fails the fresh workstream is rolled back so no orphan rows
leak.
**Request body:**
```json
@@ -953,32 +884,20 @@ Status code: `400`
---
### `POST /v1/api/workstreams/{ws_id}/close`
### `POST /v1/api/workstreams/close`
Closes and removes a workstream. The last remaining workstream cannot be
closed.
**Path parameters:**
| Parameter | Type | Required | Description |
|-----------|--------|----------|------------------------|
| `ws_id` | string | yes | Workstream ID to close |
**Request body:**
The body must be valid JSON. If you are not supplying any optional
fields, send `{}` — an empty / non-JSON body is rejected with a
`400`.
```json
{"ws_id": "abc123"}
```
| Field | Type | Required | Description |
|----------|--------|----------|----------------------------------------------------------|
| `reason` | string | no | Optional close reason persisted to `workstream_config`. |
The `reason` is capped at **512 UTF-8 bytes** (multibyte-safe — the
cap holds for CJK and emoji payloads), and the output guard's
credential-redaction pass strips secrets before the value is
persisted. A non-string `reason` is silently coerced to empty and
the close proceeds without writing the field.
| Field | Type | Required | Description |
|---------|--------|----------|---------------------------|
| `ws_id` | string | yes | Workstream ID to close |
**Response (success):**
@@ -996,100 +915,6 @@ Status code: `400`
---
### `POST /v1/api/workstreams/{ws_id}/attachments`
Upload an image or text document and attach it to the caller's next user
turn on this workstream.
- Images (png/jpeg/gif/webp) are capped at **4 MiB** and validated via
magic-byte sniff on upload.
- Text documents (any `text/*` MIME, allow-listed application MIMEs, or
known text extensions) are capped at **512 KiB** and must be UTF-8.
- Per-(workstream, user) pending cap is **10** attachments.
The attachment moves through three states: `pending → reserved →
consumed`. Reservation tokens are threaded through
`POST /v1/api/workstreams/{ws_id}/send` so a queued multimodal turn cannot lose its file to
an overlapping send.
Ownership failures are masked as `404` so non-owners cannot enumerate
workstream existence.
**Content-Type:** `multipart/form-data` with a single `file` field.
**Response (success):** `200`
```json
{
"attachment_id": "att_abc123",
"kind": "image",
"mime_type": "image/png",
"size_bytes": 73240,
"filename": "screenshot.png",
"state": "pending"
}
```
**Errors:**
| Code | Meaning |
|------|---------------------------------------------------------|
| 400 | Missing/invalid form, unsupported MIME, not UTF-8, etc. |
| 403 | Auth/scope failure |
| 404 | Workstream not found / not owned by caller |
| 409 | Pending-cap reached |
| 413 | Payload exceeds size cap |
---
### `GET /v1/api/workstreams/{ws_id}/attachments`
List the caller's **pending** (unconsumed) attachments for this
workstream. Ownership failures are masked as `404`.
**Response:** `200`
```json
{
"attachments": [
{
"attachment_id": "att_abc123",
"kind": "image",
"mime_type": "image/png",
"size_bytes": 73240,
"filename": "screenshot.png",
"state": "pending"
}
]
}
```
---
### `GET /v1/api/workstreams/{ws_id}/attachments/{attachment_id}/content`
Returns the raw bytes of an attachment with its stored `Content-Type`.
Useful for previewing an image or replaying a document. Ownership
failures are masked as `404`.
**Response:** `200` — binary body, original `Content-Type`.
---
### `DELETE /v1/api/workstreams/{ws_id}/attachments/{attachment_id}`
Remove a pending attachment. Consumed attachments return `404` (they
are part of a committed conversation turn). Ownership failures are also
masked as `404`.
**Response:** `200`
```json
{"deleted": "att_abc123"}
```
---
### `POST /v1/api/workstreams/{ws_id}/delete`
Permanently delete a saved workstream and all its messages from storage.
@@ -1683,7 +1508,7 @@ version. Requires the `admin.skills` permission.
```json
{
"risk_level": "medium",
"scan_status": "medium",
"scan_report": "{\"composite\": 1.75, \"details\": {...}}",
"scan_version": "1"
}
@@ -2024,7 +1849,7 @@ Status code: `200` with an empty body.
| Malformed or unparseable JSON body | Treated as an empty dict `{}`; missing fields use defaults |
| Unknown `ws_id` | `404` with `{"error": "Unknown workstream"}` |
| Unknown path (GET or POST) | `404` with plain-text body `Not found` |
| Empty `message` on `/v1/api/workstreams/{ws_id}/send` | `400` with `{"error": "Empty message"}` |
| Empty `message` on `/v1/api/send` | `400` with `{"error": "Empty message"}` |
| Empty `command` on `/v1/api/command` | `400` with `{"error": "Empty command"}` |
| Rate limit exceeded | `429` with `Retry-After` header (see below) |
@@ -2067,7 +1892,7 @@ reconnection:
On reconnect, the server replays the full conversation history via the
`history` event, so the client can rebuild its UI state without data loss. The
same reconnection strategy applies to both the per-workstream SSE stream
(`/v1/api/workstreams/{ws_id}/events`) and the global state stream (`/v1/api/events/global`).
(`/v1/api/events`) and the global state stream (`/v1/api/events/global`).
---
@@ -2177,7 +2002,7 @@ turnstone_workstreams_active_total 1
# TYPE turnstone_http_requests_total counter
turnstone_http_requests_total{method="GET",endpoint="/health",status_code="200"} 42
turnstone_http_requests_total{method="GET",endpoint="/metrics",status_code="200"} 7
turnstone_http_requests_total{method="POST",endpoint="/v1/api/workstreams/{ws_id}/send",status_code="200"} 18
turnstone_http_requests_total{method="POST",endpoint="/v1/api/send",status_code="200"} 18
# HELP turnstone_tokens_total Total tokens consumed
# TYPE turnstone_tokens_total counter
turnstone_tokens_total{type="prompt"} 84320
@@ -2193,15 +2018,15 @@ turnstone_tool_calls_total{tool="read_file"} 3
## Console Routing Proxy Endpoints
These endpoints are served by the console (`turnstone-console`) and proxy
requests to the correct server node via rendezvous (HRW) hashing over the
live service registry. In multi-node deployments, clients (SDK, channel
gateway) talk to the console instead of individual server nodes.
requests to the correct server node via the hash ring bucket cache. In
multi-node deployments, clients (SDK, channel gateway) talk to the console
instead of individual server nodes.
### `POST /v1/api/route/workstreams/new`
Create a workstream via rendezvous routing. The console generates the `ws_id`,
routes to the rendezvous-selected node, and includes `node_url` in the
response for direct SSE connections.
Create a workstream via hash-ring routing. The console generates the `ws_id`,
routes to the assigned node, and includes `node_url` in the response for
direct SSE connections.
### `POST /v1/api/route/send`
@@ -2236,4 +2061,5 @@ Used by channel adapters to open direct SSE connections to the correct server no
Prometheus metrics for the console routing layer. Includes:
`turnstone_router_requests_total`, `turnstone_router_request_duration_seconds`,
`turnstone_router_membership_size`, `turnstone_router_refresh_total`.
`turnstone_ring_membership_size`, `turnstone_ring_version`,
`turnstone_ring_rebalance_total`, `turnstone_ring_migrations_total`.
+60 -142
View File
@@ -21,8 +21,7 @@ plugs in.
| `turnstone-console` | `turnstone.console.server` | ClusterCollector | Cluster dashboard (aggregates all nodes) |
| `turnstone-eval` | `turnstone.eval` | `NullUI` | Headless evaluation and prompt optimization |
| `turnstone-channel` | `turnstone.channels.cli` | ChannelAdapter | Channel gateway (Discord, Slack, etc.) |
| `turnstone-admin` | `turnstone.admin` | — | Offline user and API token management |
| `turnstone-bootstrap` | `turnstone.bootstrap` | — | LLM-guided setup wizard |
| `turnstone-admin` | `turnstone.core.admin_cli` | — | Offline user and API token management |
---
@@ -37,10 +36,7 @@ turnstone/
session.py ChatSession engine, SessionUI protocol, tool dispatch
providers/ LLM provider adapters (pluggable backend layer)
_protocol.py LLMProvider protocol, ModelCapabilities, StreamChunk, CompletionResult
_openai.py OpenAIProvider facade (re-exports Chat/Responses providers)
_openai_chat.py OpenAIChatCompletionsProvider — vLLM, llama.cpp, local compatible APIs
_openai_responses.py OpenAIResponsesProvider — commercial OpenAI Responses API
_openai_common.py Shared ModelCapabilities table + helpers
_openai.py OpenAIProvider — OpenAI, vLLM, llama.cpp, any compatible API
_anthropic.py AnthropicProvider — Anthropic Messages API, native streaming, thinking
_google.py GoogleProvider — Google Gemini via OpenAI-compat endpoint
__init__.py create_provider() + create_client() factory functions
@@ -85,13 +81,12 @@ turnstone/
static/ Cluster dashboard web UI (page-specific HTML, CSS, JS)
channels/
cli.py Unified channel gateway entry point (turnstone-channel)
_protocol.py ChannelAdapter protocol
_protocol.py ChannelAdapter protocol, ChannelEvent dataclass
_routing.py ChannelRouter — channel/thread ↔ workstream mapping via HTTP
_config.py Base ChannelConfig dataclass
discord/ Discord adapter (bot, cog, views, streaming, config)
slack/ Slack adapter (Socket Mode bot, DM routing, approval buttons)
shared_static/ Shared design system (base.css, auth.js, theme.js, toast.js, utils.js, kb.js)
katex-0.16.47/ Vendored KaTeX math rendering library (MIT, woff2 fonts)
katex-0.16.45/ Vendored KaTeX math rendering library (MIT, woff2 fonts)
ui/
colors.py ANSI color constants with NO_COLOR support
markdown.py Streaming terminal markdown renderer (line-buffered)
@@ -102,7 +97,7 @@ turnstone/
renderer.js Markdown + LaTeX renderer (tables, nested lists, blockquotes, KaTeX math)
app.js Split-pane UI (Pane class, binary layout tree, SSE, tool approval)
tools/
*.json 19 tool schemas (OpenAI function-calling format + turnstone metadata)
*.json 15 tool schemas (OpenAI function-calling format + turnstone metadata)
```
Both UIs share a common design system extracted into `turnstone/shared_static/`: design tokens, login overlay, toast notifications, theme toggle, keyboard shortcuts, and utility functions. Each UI imports `base.css` and the shared JS modules at `/shared/`, then adds only page-specific code at `/static/`.
@@ -231,13 +226,11 @@ The engine emits state changes via `_emit_state()` which calls
> See also: [Core Engine Classes diagram](diagrams/png/03-core-engine-classes.png)
Defined in `turnstone.core.session.SessionUI` as a `typing.Protocol` with 16
Defined in `turnstone.core.session.SessionUI` as a `typing.Protocol` with 14
methods. Every frontend must implement all of them.
```python
class SessionUI(Protocol):
def on_turn_start(self) -> None: ...
def on_turn_committed(self) -> None: ...
def on_thinking_start(self) -> None: ...
def on_thinking_stop(self) -> None: ...
def on_reasoning_token(self, text: str) -> None: ...
@@ -254,14 +247,6 @@ class SessionUI(Protocol):
def on_rename(self, name: str) -> None: ... # propagate alias to tab/UI label
```
`on_turn_start` fires at the top of each iteration of the send-loop;
`on_turn_committed` fires immediately after `messages.append(assistant_msg)`.
`SessionUIBase` uses both to reset the per-turn inflight buffers
(`_ws_inflight_content` / `_ws_inflight_reasoning` / `_ws_inflight_seq`)
that fuel the SSE refresh-resume `in_progress_snapshot` event — see
the per-workstream events stream in
[`docs/api-reference.md`](api-reference.md#get-v1apiworkstreamsws_idevents).
`on_rename` is called by the `/name` command (on success) and after a successful `/resume` (if the resumed session has an alias or title). `WebUI.on_rename` broadcasts a `ws_rename` event on the global SSE channel and updates the in-memory `Workstream.name`; `TerminalUI.on_rename` is a no-op.
### Three Implementations
@@ -394,11 +379,11 @@ non-idle background workstreams above the input prompt.
(`Ctrl+\`, `Ctrl+Shift+\`). Max 6 panes; no duplicate workstreams across panes.
Layout persisted to `localStorage`.
- **Per-pane SSE**: `Pane.connectSSE(wsId)` opens
`/v1/api/workstreams/{ws_id}/events` for each pane's event stream independently.
`/v1/api/events?ws_id=<id>` for each pane's event stream independently.
- **Global SSE**: `connectGlobalSSE()` opens `/v1/api/events/global` which
receives `ws_state` broadcasts from all workstreams, used to update tab
indicators and pane headers without switching.
- **New tab / close**: POST `/v1/api/workstreams/new`, POST `/v1/api/workstreams/{ws_id}/close`.
- **New tab / close**: POST `/v1/api/workstreams/new`, POST `/v1/api/workstreams/close`.
### Thread Safety
@@ -458,15 +443,13 @@ from each schema and builds:
- `PRIMARY_KEY_MAP` -- `{name: primary_key}` for JSON fallback recovery
- `merge_mcp_tools(builtin, mcp_tools)` -- merges built-in + MCP tools at session init
### 19 Tools by Category
### 13 Tools by Category
**Read-only (auto-approve)**:
- `read_file` -- read file contents with optional offset/limit
- `diff_file` -- show diff between two files / versions
- `search` -- ripgrep-based codebase search
- `man` -- read man pages
- `recall` -- search conversation history
- `read_resource` -- read an MCP resource by URI
**Write (requires approval)**:
- `bash` -- execute shell commands (with safety checks via `turnstone.core.safety`)
@@ -475,20 +458,13 @@ from each schema and builds:
- `math` -- execute Python in sandboxed subprocess (via `turnstone.core.sandbox`)
- `web_fetch` -- fetch a URL (with SSRF protection via `turnstone.core.web`)
- `web_search` -- search the web (provider-native for Anthropic/OpenAI, Tavily fallback for local models)
- `notify` -- send a user-facing notification (Discord/Slack, optional reply routing)
- `watch` -- schedule a recurring poll with condition DSL
**Agent (delegated sub-sessions)**:
- `task_agent` -- delegate to a sub-agent with full tool access (`TASK_AGENT_TOOLS`)
- `plan_agent` -- explore codebase and write a structured plan (`AGENT_TOOLS`)
- `task` -- delegate to a sub-agent with full tool access (`TASK_AGENT_TOOLS`)
- `plan` -- explore codebase and write a structured plan (`AGENT_TOOLS`)
**Memory / skills / prompts**:
**Memory (structured persistent store)**:
- `memory` -- save, search, delete, or list memories (typed and scoped)
- `skill` -- invoke a skill (governed, versioned procedure)
- `use_prompt` -- fetch and apply a prompt template
Tool names are `plan_agent` / `task_agent` (not `plan` / `task`); bare words
collide with chat-template channels on some local models.
### Prepare / Execute Pattern
@@ -507,14 +483,14 @@ separation allows the UI to show previews before any side effects occur.
### Agent Tools
`task_agent` and `plan_agent` invoke `_run_agent()`, which runs a multi-turn
loop with a subset of tools and its own system prompt. The sub-agent runs
`task` and `plan` invoke `_run_agent()`, which runs a multi-turn loop with
a subset of tools and its own system prompt. The sub-agent runs
independently, then returns the final content as the tool result.
- **task_agent**: uses `self._task_tools` (`TASK_AGENT_TOOLS` + MCP tools)
- **plan_agent**: uses `self._agent_tools` (`AGENT_TOOLS` + MCP tools). Writes output
- **task**: uses `self._task_tools` (`TASK_AGENT_TOOLS` + MCP tools)
- **plan**: uses `self._agent_tools` (`AGENT_TOOLS` + MCP tools). Writes output
to `.plan-<ws_id>.md` — unique per `ChatSession` so concurrent workstreams
don't collide. On repeat invocations the prior `plan_agent` tool call and its result
don't collide. On repeat invocations the prior `plan` tool call and its result
are forwarded from `self.messages` so the agent refines the existing plan rather
than starting over. Planning instructions are injected as a developer message
prepended to the agent's conversation.
@@ -556,9 +532,11 @@ adds, removes, or reconnects servers as needed.
6. `_exec_mcp_tool()` calls `call_tool_sync()` which dispatches to the async loop
via `asyncio.run_coroutine_threadsafe()`
**Tool refresh:** Two mechanisms keep tools up-to-date without restart:
**Tool refresh:** Three mechanisms keep tools up-to-date without restart:
- **Push:** Servers declaring `tools.listChanged` send `ToolListChangedNotification`;
the registered `message_handler` triggers immediate single-server refresh.
- **Periodic:** Servers without push support are polled on a staggered interval
(default 4 h, configurable via `[mcp] refresh_interval` or `--mcp-refresh-interval`).
- **Manual:** `/mcp refresh [server]` calls `refresh_sync()` for on-demand refresh
(also attempts reconnection for disconnected servers).
@@ -580,11 +558,10 @@ from a healthy connection do not trip the breaker. When the cooldown expires
(`call_tool_sync`, `read_resource_sync`, `get_prompt_sync`, `refresh_sync`)
cancel orphaned futures on timeout to prevent coroutine accumulation on the
background event loop. Push notification refreshes are debounced (5 s per
server) to protect against notification storms. Operators can force a
catalog refresh or full reconnect from the admin panel; reconnects clear
the circuit breaker and run a fresh handshake. Transport stream references
are pre-closed before stack teardown to work around the MCP SDK's anyio
cancel-scope CPU busy-loop (SDK #2147).
server) to protect against notification storms. The periodic refresh loop
attempts reconnection for disconnected servers with exponential backoff
(60 s1 h). Transport stream references are pre-closed before stack teardown to
work around the MCP SDK's anyio cancel-scope CPU busy-loop (SDK #2147).
**Error isolation:** Per-server connection/refresh failures are caught and logged; other
servers are unaffected. Tool execution errors return error strings to the LLM
@@ -629,15 +606,14 @@ LLMProvider (protocol)
| `get_capabilities()` | Per-model flags (`ModelCapabilities`) |
| `convert_tools()` | Translate OpenAI tool schemas to provider format |
| `retryable_error_names` | Exception class names that trigger retry |
| `extract_reasoning_text()` | Walk stored `provider_blocks`, return concatenated reasoning text for UI rehydration (per-provider block-type knowledge: Anthropic `thinking`, OpenAI Responses `reasoning`, OpenAI Chat synthetic `reasoning_text`) |
**Normalized data types:**
| Type | Fields |
|------|--------|
| `StreamChunk` | `content_delta`, `reasoning_delta`, `tool_call_deltas`, `info_delta`, `usage`, `finish_reason`, `provider_blocks` |
| `CompletionResult` | `content`, `tool_calls`, `finish_reason`, `usage`, `provider_blocks` |
| `ModelCapabilities` | `context_window`, `max_output_tokens`, `supports_temperature`, `token_param`, `thinking_mode`, `supports_effort`, `supports_web_search`, `supports_tool_search`, `supports_vision`, `supports_reasoning_replay` |
| `StreamChunk` | `content_delta`, `reasoning_delta`, `tool_call_deltas`, `info_delta`, `usage`, `finish_reason` |
| `CompletionResult` | `content`, `tool_calls`, `finish_reason`, `usage` |
| `ModelCapabilities` | `context_window`, `max_output_tokens`, `supports_temperature`, `token_param`, `thinking_mode`, `supports_effort`, `supports_web_search`, `supports_tool_search`, `supports_vision` |
| `UsageInfo` | `prompt_tokens`, `completion_tokens`, `total_tokens`, `cache_creation_tokens`, `cache_read_tokens` |
**OpenAIProvider** (`_openai.py`): passes messages through unchanged (they are
@@ -720,55 +696,6 @@ agent_model = "claude"
Each `[models.*]` entry produces a `ModelConfig` with a `provider` field
(default: `"openai"`). Supported values: `"openai"`, `"anthropic"`, `"google"`,
and `"openai-compatible"`.
**Per-model sampling overrides:** Each model can specify `temperature`,
`max_tokens`, and `reasoning_effort` to override the global defaults from
ConfigStore. When unset (`NULL`), the global default is used.
**Per-model reasoning persistence:** Two booleans on `model_definitions`
(migration 052) control how reasoning text round-trips:
* `surface_persisted_reasoning` (default `True`) — gates whether stored
reasoning text is surfaced on `/history` payloads for UI rehydration.
**Storage of reasoning bytes happens regardless of this flag** — they
ride in `provider_data` independently. Phase-1 admin UI label "Surface
persisted reasoning."
* `replay_reasoning_to_model` (default `False`) — gates whether stored
reasoning blocks are sent back to the provider on subsequent turns.
Capability-gated: `ModelCapabilities.supports_reasoning_replay` must
also be `True` for the wire path to actually replay (canonical OpenAI
gpt-5*/o-series and Anthropic Claude entries set it; unknown / local-
server models default to `False`).
Three reasoning paths are recognised:
| Path | Provider | Capture | Persist | Replay |
|------|----------|---------|---------|--------|
| 1 | Anthropic Messages API | `thinking_delta` | `provider_blocks` (`type="thinking"`) | Verbatim via `_provider_content` |
| 2 | OpenAI Responses (gpt-5*, o-series) | `response.reasoning_text.delta` events | `provider_blocks` (`type="reasoning"`) — only when `include=["reasoning.encrypted_content"]` | `ResponseReasoningItemParam` input items |
| 3 | OpenAI Chat Completions (vLLM, llama.cpp, Gemini-compat) | `delta.reasoning_content` Pydantic extras | Synthetic `{type: "reasoning_text", text, source}` block stamped at end-of-stream | None — no API surface for replay on Chat Completions |
Cross-provider safety is enforced by `ANTHROPIC_VALID_BLOCK_TYPES` (a
shape filter in `_anthropic.py:_convert_messages`): foreign blocks
(OpenAI `reasoning`, synthetic `reasoning_text`) fall through to the
text+tool_calls rebuild path rather than reaching Anthropic's input
boundary as malformed content.
```toml
[models.local]
base_url = "http://localhost:8000/v1"
model = "qwen3-32b"
temperature = 0.7
max_tokens = 8192
[models.o3]
base_url = "https://api.openai.com/v1"
api_key = "sk-..."
model = "o3"
reasoning_effort = "high"
# temperature omitted — uses global default
```
An optional `[models.*.capabilities]` sub-table overrides per-model
`ModelCapabilities` flags (useful for local models whose capabilities
cannot be detected programmatically):
@@ -782,15 +709,9 @@ model = "qwen-3.5-vl"
supports_vision = true
```
**Database model definitions:** On server entry points, models can also be
defined in the `model_definitions` table (admin Models tab). DB models support
the same per-model sampling overrides. Config.toml models override DB models
with the same alias in-memory (the DB rows are never modified).
**Lifecycle:**
1. `load_model_registry()` loads DB model definitions (if storage available),
then overlays `[models.*]` from config.toml, then builds a `"default"` entry
from CLI `--base-url`/`--model`/`--api-key` args
1. `load_model_registry()` reads `[models.*]` sections from config.toml and
builds a `"default"` entry from CLI `--base-url`/`--model`/`--api-key` args
2. The registry is passed to the session factory closure in both `cli.py` and
`server.py`; each workstream resolves its model on creation
3. `ModelRegistry.get_client()` lazily creates SDK client instances via
@@ -799,8 +720,7 @@ with the same alias in-memory (the DB rows are never modified).
4. `ModelRegistry.get_provider()` lazily creates `LLMProvider` instances via
`create_provider()` (also cached and thread-safe)
5. `/model` command shows available models; `/model <alias>` switches the
active workstream's client, model, context window, and per-model sampling
parameters
active workstream's client, model, and context window
6. `_create_stream_with_retry()` tries the primary model, then each fallback
alias in order if the primary is unreachable
7. `_run_agent()` resolves `registry.agent_model` (if set) for plan/task
@@ -923,7 +843,7 @@ and are the single source of truth for both backends and Alembic migrations.
| `update_workstream_title(ws_id, title)` | Set/update LLM-generated title |
| `update_workstream_state(ws_id, state)` | Update workstream state and bump timestamp |
| `update_workstream_name(ws_id, name)` | Update workstream display name |
| `list_workstreams(node_id, limit, *, parent_ws_id, kind, user_id)` | List workstreams, optionally filtered by node, parent, kind, or owning user |
| `list_workstreams(node_id, limit)` | List workstreams, optionally by node |
| `kv_get(key)` / `kv_set(key, value)` / `kv_delete(key)` | Generic key-value store (backs memories table) |
| `kv_list()` / `kv_search(query)` | List or search key-value pairs |
| `search_history(query, limit)` | Full-text search (FTS5 on SQLite, tsvector on PostgreSQL) |
@@ -1139,8 +1059,8 @@ Three hierarchical scopes control endpoint access:
| Scope | Grants | Endpoints |
|-------|--------|-----------|
| `read` | SSE streams, workstream listing, history | GET endpoints |
| `write` | `read` + send, command, workstream create/close | POST to `/api/workstreams/{ws_id}/send`, `/api/command`, etc. |
| `approve` | `write` + tool approval, admin operations | POST to `/api/workstreams/{ws_id}/approve`, `/api/admin/*` |
| `write` | `read` + send, command, workstream create/close | POST to `/api/send`, `/api/command`, etc. |
| `approve` | `write` + tool approval, admin operations | POST to `/api/approve`, `/api/admin/*` |
### Middleware Flow
@@ -1164,9 +1084,8 @@ Three hierarchical scopes control endpoint access:
- **Console** is the auth management hub — it hosts the admin endpoints for
creating users, issuing API tokens, and managing channel mappings. User
records and token hashes live in the shared storage backend. The console
dashboard includes an **admin panel** (18 tabs) for managing
credentials, governance, MCP servers, models, node metadata, and runtime
settings through the browser.
dashboard includes an **admin panel** (14 tabs) for managing
credentials, governance, MCP servers, and runtime settings through the browser.
- **Server** is a JWT validator only — it validates tokens on each request but
never creates users or tokens. Both processes share the same `jwt_secret`
(via `TURNSTONE_JWT_SECRET` env var or `[auth].jwt_secret` config).
@@ -1237,12 +1156,12 @@ stderr so it does not interfere with readline. Tool execution may use a
Starlette ASGI app (served by uvicorn)
|
+-- Async request handlers (all under /v1/ prefix)
| POST /v1/api/workstreams/{ws_id}/send -> starts worker thread per workstream
| POST /v1/api/workstreams/{ws_id}/approve -> unblocks WebUI._approval_event
| POST /v1/api/plan -> unblocks WebUI._plan_event
| POST /v1/api/workstreams/new -> creates workstream + worker
| GET /v1/api/workstreams/{ws_id}/events -> SSE via EventSourceResponse (per workstream)
| GET /v1/api/events/global -> SSE via EventSourceResponse (fan-out)
| POST /v1/api/send -> starts worker thread per workstream
| POST /v1/api/approve -> unblocks WebUI._approval_event
| POST /v1/api/plan -> unblocks WebUI._plan_event
| POST /v1/api/workstreams/new -> creates workstream + worker
| GET /v1/api/events -> SSE via EventSourceResponse (per workstream)
| GET /v1/api/events/global -> SSE via EventSourceResponse (fan-out)
|
+-- ASGI middleware stack
| MetricsMiddleware -> CORSMiddleware -> AuthMiddleware -> RateLimitMiddleware
@@ -1316,10 +1235,10 @@ Monitoring (2 daemon threads) Control + Proxy (async Starlette)
| SSE manager | | GET /node/{node_id}/ |
| asyncio loop | | → httpx.AsyncClient |
| 1 task per node | | proxy to server_url |
| /events/global | | GET /node/{id}/v1/api/workstreams/{ws_id}/events |
| snapshot+deltas | | → SSE stream proxy |
+------------------+ | POST /node/{id}/v1/api/workstreams/{ws_id}/send |
| → forwarded to server |
| /events/global | | GET /node/{id}/v1/api/events |
| snapshot+deltas | | → SSE stream proxy |
+------------------+ | POST /node/{id}/v1/api/send |
| → forwarded to server |
+----------------------------+
```
@@ -1401,10 +1320,9 @@ setup, auth headers, `_request()` (REST) and `_stream_sse()` (SSE). Sync
clients delegate through `_SyncRunner` which maintains a persistent background
event loop on a daemon thread.
**Event types**: 38 standalone dataclasses in `events.py` with a type-registry
dispatch (`from_json()` on each event). Events are decoupled from server
internals — the SDK parses SSE frames directly from the `/v1/api/events`
streams.
**Event types**: 27 standalone dataclasses in `events.py` with a type-registry
pattern matching `OutboundEvent.from_json()` from `mq/protocol.py`. Events are
decoupled from server internals.
**TypeScript SDK**: `sdk/typescript/` — separate npm package with the same API
surface. Zero browser dependencies, SSE via `fetch` + `ReadableStream` parsing.
@@ -1426,8 +1344,7 @@ with TurnstoneServer("http://localhost:8080", token="tok_xxx") as client:
> See also: [Channel Integrations guide](channels.md)
The `turnstone-channel` gateway connects external messaging platforms
(Discord and Slack today, with an adapter protocol for future platforms) to
the turnstone cluster via HTTP. Each
(Discord, Slack, Teams) to the turnstone cluster via HTTP. Each
platform adapter implements the `ChannelAdapter` protocol and translates
between platform-native events and turnstone server API calls.
@@ -1440,7 +1357,7 @@ workstream is reactivated, the router uses atomic resume via the
the old workstream's conversation during creation in a single HTTP
request, eliminating ordering fragility.
Discord and Slack adapters ship today. See [channels.md](channels.md) for
Discord ships as the first adapter. See [channels.md](channels.md) for
setup instructions, configuration reference, and the adapter development
guide.
@@ -1461,11 +1378,11 @@ retries up to 3 times with backoff, re-querying the service registry on
each attempt. See [Notification Flow diagram](diagrams/png/17-notify-flow.png).
**Bidirectional replies:** When a user replies to a notification DM, the
channel adapter (Discord or Slack) looks up the originating `ws_id` from the
tracked message ID, verifies the replying user matches the notification
recipient, and routes the reply to the workstream via `router.send_message()`.
The workstream's response is forwarded back to the DM via a temporary entry
in `_notify_reply_channels`. On `TurnCompleteEvent`, the response message is
Discord bot looks up the originating `ws_id` from the tracked message ID,
verifies the replying user matches the notification recipient, and routes
the reply to the workstream via `router.send_message()`. The workstream's
response is forwarded back to the DM via a temporary entry in
`_notify_reply_channels`. On `TurnCompleteEvent`, the response message is
itself tracked for further replies, enabling multi-turn DM conversations
without requiring the user to open the web UI. Tracking entries are capped
at 100 (FIFO eviction) and cleaned up on workstream close.
@@ -1498,10 +1415,11 @@ and workstreams record which skill and version spawned them. Token budget
enforcement tracks consumption in `session.send()` with 80% warning and
100% approval gate via the `__budget_override__` synthetic tool name.
The console admin panel exposes these capabilities as 18 permission-gated
tabs: Users, API Tokens, Channels, Schedules, Watches, Roles, Policies,
Prompts, Judge, Skills, MCP Servers, Usage, Audit, Memories, Models, Nodes,
Settings, and TLS.
The console admin panel adds 5 governance tabs (Roles, Policies, Skills,
Usage, Audit), a Memories tab, a Settings tab (form-based editor for all
ConfigStore settings), and an MCP Servers tab (database-backed server
definitions with live connection status and cluster-wide reload) for a
total of 13 tabs, all permission-gated.
Both Python and TypeScript SDKs expose governance methods on the console
client.
+2 -2
View File
@@ -1,3 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:5d500479d3be2363d4f594042a27e2ef5e2974750f580f6c4037a1fe85868ed9
size 251904
oid sha256:75c1832b6079e8628f4bbf4ce98d37880c4de133636b7555e3869990b046ddc6
size 567704
-244
View File
@@ -1,244 +0,0 @@
# Bulk endpoint shape contract
Turnstone exposes several endpoints and tool calls that take multiple
ids and return a per-id outcome. Over the last few phases two
**distinct** response shapes have settled, one per semantic category.
This doc codifies both so a future endpoint author can pick the right
shape by semantics instead of by coin-flip.
Existing bulk endpoints at time of writing:
| Endpoint / tool | Category | Response shape |
|---------------------------------------------------------|--------------------------|------------------------------------------|
| `GET /v1/api/cluster/ws/live?ids=a,b,c` | bulk read | `{results, denied, truncated}` |
| model tool `spawn_batch` | bulk create (per-item) | `{results, denied}` |
| `POST /v1/api/workstreams/{ws_id}/stop_cascade` | cascade mutation | `{cancelled, failed, skipped}` |
| `POST /v1/api/workstreams/{ws_id}/close_all_children` | cascade mutation | `{closed, failed, skipped}` |
---
## Why two shapes
The ask-to-outcome mapping is fundamentally different between the
two categories, and a one-size-fits-all envelope ends up papering
over distinctions the caller genuinely needs to branch on.
**Bulk read / bulk create-with-payload.** Each input id (or batch
index) carries a *request-side* concept — "give me the live block
for this ws_id" or "spawn a child with this spec" — and each
successful output carries a *payload* — the live block, or the new
workstream's identifying triple. The interesting distinction on
failure is *ownership / validation* (caller can't see that id, spec
was malformed) — independent of the storage state.
**Cascade mutation.** The action is uniform across every id (cancel
this subtree, close this child). The interesting distinctions on
outcome are *did it reach the terminal state?* (succeeded / already
was there / the dispatch itself failed) — driven by the storage
state plus transport reliability, not by the caller's input.
Trying to unify these forces either:
- a stateless `denied` bucket that has to carry "already gone"
*and* "you don't have permission" *and* "transport failed" with a
separate reason string — reviewers end up string-matching to branch.
- or a per-item-payload map for cascade mutations where every
successful value is the same sentinel — carrier with no payload.
So: two shapes, one per category. The rest of this doc spells out
each.
---
## Shape A — bulk read / bulk create-with-payload
```json
{
"results": { "<key>": <value-or-null>, ... },
"denied": [ "<key>", ... ],
"truncated": false
}
```
**`results`** is a key-indexed map of the positive-path payload.
The key is the input id for read endpoints (`cluster/ws/live` uses
the ws_id), or the input-array index (stringified) for create
endpoints that want ordering preserved (`spawn_batch` uses `"0"`,
`"1"`, ...). The value is whatever the endpoint produces per
success — a live block, a `{ws_id, name, node_id, status}` triple,
etc. A `null` value (read endpoints only) means "the id existed and
you own it, but the live block wasn't available" — distinct from
"denied".
**`denied`** is the negative-path list. For read endpoints it's a
flat list of ids (preserves input order so callers can re-zip
against their input). For create endpoints with per-item payloads
it's a list of `{idx, reason}` objects (`spawn_batch`'s validation
and spawn-error rows; also the operator-reject surface when per-item
selective-deny ships). Include every reason that's *not* the
positive path — authz, ownership, validation, already-consumed,
spawn failure — so callers don't branch on status codes.
**`truncated`** is a boolean set to `true` when the server's
per-endpoint input cap was exceeded and the tail was dropped. The
endpoint docs each spell out the cap (50 for `cluster/ws/live`).
`spawn_batch` hard-errors on overflow instead of silently
truncating — it omits the field entirely rather than carry a
permanently-false flag.
### Example — `cluster/ws/live`
```http
GET /v1/api/cluster/ws/live?ids=a1b2,c3d4,nonexistent,foreign HTTP/1.1
```
```json
{
"results": {
"a1b2": {"state": "running", "tokens": 12843, "activity": "..."},
"c3d4": null
},
"denied": ["nonexistent", "foreign"],
"truncated": false
}
```
Callers that need ordered output zip their original id list against
this map; ids in `denied` drop out of the zip cleanly. A live-block
`null` doesn't route to `denied` — the row exists and the caller
owns it; the node is just currently unreachable.
### Example — `spawn_batch`
This is the coordinator-tool result shape (the JSON the LLM receives),
not an HTTP API response — the table above keys it under "model tool"
to distinguish it from the `/v1/api/...` endpoints in the same table.
The underlying HTTP spawn endpoint still returns `ws_id`; the tool
result re-keys it to `child_ws_id` to defuse a coordinator-LLM recency
bias (see `docs/coordinator-skills.md`).
```json
{
"results": {
"0": {"child_ws_id": "d4e5f6...", "name": "csrf-audit", "node_id": "gpu-3"},
"2": {"child_ws_id": "f1a2b3...", "name": "xss-audit", "node_id": "gpu-1"}
},
"denied": [
{"idx": 1, "reason": "skill not found: nonexistent-skill"}
]
}
```
Indexes are stringified to keep the envelope JSON-safe and
consistently-typed across the read and create cases.
---
## Shape B — cascade mutation
```json
{
"status": "ok",
"<bucket>": [ "<ws_id>", ... ],
"failed": [ "<ws_id>", ... ],
"skipped": [ "<ws_id>", ... ]
}
```
Where `<bucket>` is the endpoint-specific name for "succeeded" —
`cancelled` for `stop_cascade`, `closed` for `close_all_children`.
The three buckets partition the input set exactly once:
| Bucket | Meaning |
|---------------|-------------------------------------------------------------------------------|
| `<bucket>` | Action dispatch accepted; target reached the intended terminal state. |
| `failed` | Dispatch returned a non-404 error (transport issue, upstream 5xx, exception). |
| `skipped` | Upstream 404 — stale registry entry, row already deleted, or peer gone. |
The split between `failed` and `skipped` is load-bearing. `failed`
is actionable — the operator may want to retry, or the cascade may
be partial. `skipped` is pre-resolved — the target is already in
the terminal state the cascade was aiming at, so it's neither a
win to report nor a fault to fix.
### Example — `stop_cascade`
```json
{
"status": "ok",
"cancelled": ["child-1", "child-3"],
"failed": [],
"skipped": ["child-2"]
}
```
A subsequent retry would target only `failed` ids, not `skipped`
ones — the latter are already done.
### Example — `close_all_children`
```json
{
"status": "ok",
"closed": ["child-1", "child-3"],
"failed": ["child-2"],
"skipped": []
}
```
Same partition, different success-bucket name. When `coord_client`
is unavailable (session loaded but no HTTP client attached — a
construction bug) every id goes to `failed` so the operator notices
rather than getting a silent all-skipped response.
---
## Guidance for future bulk endpoints
1. **Pick by semantics, not by "what shape is nearby."**
- Mutation that's uniform across ids + terminal-state outcome? →
**Shape B** (cascade mutation).
- Read or create where the input id carries payload, or where the
denial axis is independent of storage state? → **Shape A**
(bulk read / bulk create-with-payload).
2. **Cap the input.** Both shapes assume a bounded input — the
server rejects or silently truncates past the cap. Document the
cap in the endpoint's OpenAPI description. Shape A uses
`truncated: true` on quiet truncation; Shape B hard-errors on
overflow.
3. **Match existing bucket names for the same semantic.** Use
`failed` and `skipped` verbatim in Shape B — the per-endpoint
success bucket is the only slot that varies. Use `results` and
`denied` verbatim in Shape A; the per-endpoint `<key>` /
`<value>` types vary.
4. **Audit the verbose shape.** Both endpoints emit a corresponding
audit event with the full before/after bucket lists — the SSE
stream and the in-process response give live feedback, but a
postmortem operator will read the audit row. Use
`_emit_coord_audit` (coordinator-scoped) or `record_audit`
directly; don't inline.
5. **Don't mix shapes within one endpoint.** If a bulk endpoint
wants both partial-success creation AND per-item failure reasons
(like `spawn_batch` with its `{idx, reason}` denial rows), that's
Shape A with a richer denial element — not a blend with Shape B.
---
## History
- **Phase 6** shipped `cluster/ws/live` as the first Shape A endpoint
(`{results, denied, truncated}`).
- **Phase 7** shipped `stop_cascade` as the first Shape B endpoint
(`{cancelled, failed, skipped}`).
- **Phase 8 PR A** shipped `spawn_batch` (Shape A, keyed by idx) and
`close_all_children` (Shape B, twin of `stop_cascade`), which
crystallised the two-shape-per-semantic-category policy codified
here.
Before adding a third shape, read this doc and argue for why the
new surface doesn't fit either A or B. Two idioms in the cluster
API is a finite operator tax; three is one too many.
+21 -95
View File
@@ -7,35 +7,31 @@ platform-native events (messages, button clicks, slash commands) into
turnstone API calls, and renders workstream output back into the
platform's UI.
Discord and Slack adapters ship today. The adapter protocol is designed
so new platforms can be added with only a new package under
`turnstone/channels/<platform>/`.
Discord ships as the first adapter. The adapter protocol is designed for
future Slack and Teams integrations.
---
## Architecture
```
Discord Gateway Slack (Socket Mode WebSocket)
\ /
v v
turnstone-channel (one or more adapters)
|
v
turnstone-server (direct HTTP)
or
turnstone-console (routing proxy, multi-node)
Discord Gateway
|
v
turnstone-channel (Discord adapter)
|
v
turnstone-server (direct HTTP)
or
turnstone-console (routing proxy, multi-node)
```
A single `turnstone-channel` process can run multiple adapters
simultaneously (e.g. Discord + Slack) — pass the tokens for each
platform you want to enable.
Key components:
- **ChannelAdapter protocol** (`turnstone/channels/_protocol.py`) — generic
interface for any messaging platform. Defines `start()`, `stop()`,
`send()`, and `send_notification()`.
`send()`, `send_notification()`, `edit_message()`,
`send_approval_request()`, `send_plan_review()`, and `create_thread()`.
- **ChannelRouter** (`turnstone/channels/_routing.py`) — maps
channel/thread IDs to turnstone workstream IDs. Handles workstream
creation via HTTP, stale route detection, and user identity resolution.
@@ -124,68 +120,6 @@ An admin can also force-link or unlink users via the console admin panel
---
## Slack Setup
Slack uses **Socket Mode**, so no public URL or API Gateway is required — Slack
connects outbound to the bot via a WebSocket. Install with:
```bash
pip install 'turnstone[slack]'
```
### 1. Create a Slack App
1. Go to https://api.slack.com/apps and click **Create New App**
2. Under **Settings > Socket Mode**, enable Socket Mode. This generates an
**App-Level Token** (prefix `xapp-`) — copy it.
3. Under **OAuth & Permissions**, add these **Bot Token Scopes**:
`chat:write`, `chat:write.public`, `channels:history`, `im:history`,
`groups:history`, `mpim:history`, `reactions:write`, `commands`
4. Under **Event Subscriptions** (Socket Mode delivers events), subscribe
to bot events: `message.channels`, `message.im`, `message.groups`
5. Under **Slash Commands**, create a command (default `/turnstone`)
6. Install the app to your workspace to generate the **Bot User OAuth
Token** (prefix `xoxb-`).
### 2. Configure Turnstone
**Environment variables** (recommended for Docker):
```bash
TURNSTONE_SLACK_TOKEN=xoxb-... # Bot User OAuth Token
TURNSTONE_SLACK_APP_TOKEN=xapp-... # App-Level Token (Socket Mode)
TURNSTONE_SLACK_CHANNELS= # optional, comma-separated channel IDs
TURNSTONE_SLACK_SLASH_COMMAND=/turnstone
```
**CLI flags** (bare-metal):
```bash
turnstone-channel \
--slack-token "xoxb-..." \
--slack-app-token "xapp-..." \
--slack-slash-command /turnstone \
--server-url http://localhost:8080
```
The Slack and Discord adapters can be enabled together — pass tokens for
both and the gateway hosts both adapters in one process.
### 3. Usage
- **DM the bot**: messages sent directly to the bot create a workstream
scoped to that DM; the slash command is not required.
- **Slash command**: `/turnstone <message>` in any channel the bot can
see starts a per-user channel session.
- Tool approvals render as Slack **Block Kit** buttons; only the user
who owns the workstream can approve/reject.
- Plan reviews render as a modal with approve / request-changes actions.
- Notifications and reply routing work identically to Discord.
- Session recovery: persisted channel routes are re-subscribed when the
bot restarts, so existing Slack conversations keep flowing.
---
## Usage
### Conversations
@@ -250,13 +184,9 @@ Plan review requests are displayed as a blue embed with:
| CLI Flag | Env Var | Default | Description |
|----------|---------|---------|-------------|
| `--discord-token` | `TURNSTONE_DISCORD_TOKEN` | — | Discord bot token (required to enable Discord) |
| `--discord-token` | `TURNSTONE_DISCORD_TOKEN` | — | Bot token (required to enable Discord) |
| `--discord-guild` | — | `0` (all guilds) | Restrict to a single Discord guild |
| `--discord-channels` | — | empty (all) | Comma-separated Discord channel IDs to allow |
| `--slack-token` | `TURNSTONE_SLACK_TOKEN` | — | Slack Bot User OAuth token (`xoxb-…`, required to enable Slack) |
| `--slack-app-token` | `TURNSTONE_SLACK_APP_TOKEN` | — | Slack App-Level token (`xapp-…`, required with `--slack-token`) |
| `--slack-channels` | `TURNSTONE_SLACK_CHANNELS` | empty (all) | Comma-separated Slack channel IDs to allow |
| `--slack-slash-command` | `TURNSTONE_SLACK_SLASH_COMMAND` | `/turnstone` | Slash command name registered in the Slack app |
| `--discord-channels` | — | empty (all) | Comma-separated channel IDs to allow |
| `--server-url` | `TURNSTONE_SERVER_URL` | `http://localhost:8080` | Server URL (single-node) |
| `--console-url` | `TURNSTONE_CONSOLE_URL` | — | Console URL (multi-node routing proxy) |
| `--model` | — | server default | Default model for new workstreams |
@@ -266,9 +196,6 @@ Plan review requests are displayed as a blue embed with:
| `--log-level` | `TURNSTONE_LOG_LEVEL` | `INFO` | Log level |
| `--log-format` | `TURNSTONE_LOG_FORMAT` | `auto` | Log format (`auto`/`json`/`text`) |
At least one of `--discord-token` or `--slack-token` must be supplied.
Passing both starts both adapters in the same process.
---
## User Identity
@@ -322,8 +249,8 @@ waiting for them to check in.
Two modes:
- **Username** — provide a turnstone `username`. The gateway resolves
it via the `channel_users` table and sends to every linked platform
the user has (e.g. Discord + Slack).
it via the `channel_users` table and sends to all linked channels
(e.g. Discord + future Slack).
- **Direct** — provide `channel_type` + `channel_id` to target a
specific platform channel or user DM.
@@ -424,6 +351,10 @@ class ChannelAdapter(Protocol):
async def stop(self) -> None: ...
async def send(self, channel_id: str, content: str) -> str: ...
async def send_notification(self, channel_id: str, content: str, ws_id: str) -> str: ...
async def edit_message(self, channel_id: str, message_id: str, content: str) -> None: ...
async def send_approval_request(self, channel_id: str, ws_id: str, correlation_id: str, items: list[dict]) -> None: ...
async def send_plan_review(self, channel_id: str, ws_id: str, correlation_id: str, content: str) -> None: ...
async def create_thread(self, parent_channel_id: str, name: str, message_id: str = "") -> str: ...
```
`send_notification()` is like `send()` but associates the outgoing
@@ -431,11 +362,6 @@ message with a `ws_id` so that user replies can be routed back to the
originating workstream. Adapters must track the mapping from outgoing
message ID to `(ws_id, target_user_id)` and handle DM replies.
Platform-specific concerns — approval prompts, plan reviews, message
edits, thread creation — live inside the adapter implementation and are
not part of the protocol surface. Each adapter drives those via its
own `_on_ws_event` dispatcher using SDK-native APIs.
To add a new platform:
1. Create `turnstone/channels/<platform>/` package
+6 -15
View File
@@ -334,7 +334,7 @@ The console reverse-proxies each node's server UI at `/node/{node_id}/`. This al
### URL Rewriting
The server UI uses root-relative URLs (`/v1/api/workstreams/{ws_id}/send`, `/static/app.js`, `/shared/base.css`, etc.). Since `<base>` tags cannot rewrite root-relative URLs, the console uses a JS shim approach:
The server UI uses root-relative URLs (`/v1/api/send`, `/static/app.js`, `/shared/base.css`, etc.). Since `<base>` tags cannot rewrite root-relative URLs, the console uses a JS shim approach:
1. **HTML rewriting** — when serving `index.html`, replaces `href=` and `src=` references to both `/static/` and `/shared/` with the proxy prefix (`/node/{node_id}/static/` and `/node/{node_id}/shared/` respectively).
@@ -344,7 +344,7 @@ The server UI uses root-relative URLs (`/v1/api/workstreams/{ws_id}/send`, `/sta
### SSE Proxy
SSE streams (`/v1/api/workstreams/{ws_id}/events`, `/v1/api/events/global`) are proxied as raw byte passthrough — the console opens an `httpx.AsyncClient.stream()` to the upstream server (with `read=None` and `pool=None` timeouts since SSE connections are long-lived) and relays every byte via `StreamingResponse`. This preserves server-side ping comments, event framing, and keepalives verbatim without parsing or re-encoding.
SSE streams (`/v1/api/events`, `/v1/api/events/global`) are proxied as raw byte passthrough — the console opens an `httpx.AsyncClient.stream()` to the upstream server (with `read=None` and `pool=None` timeouts since SSE connections are long-lived) and relays every byte via `StreamingResponse`. This preserves server-side ping comments, event framing, and keepalives verbatim without parsing or re-encoding.
### Authentication
@@ -396,19 +396,10 @@ The browser maintains a local `clusterState` object that mirrors the cluster sna
Accessed via the "admin" button in the header (visible when authenticated
with `approve` scope). Provides user, API token, channel link, MCP server,
and skill management with 18 tabs (Users, API Tokens, Channels, Schedules,
Watches, Roles, Policies, Prompts, Judge, Skills, MCP Servers, Usage,
Audit, Memories, Models, Nodes, Settings, TLS). See also
[Governance](governance.md) for the Roles, Policies, Skills, Usage, and
Audit tabs, and [Settings](settings.md) for the database-backed
configuration editor.
The **Channels** tab links users to either a Discord or Slack account
via a per-row channel-type selector. The **Models** tab is a CRUD
editor for `model_definitions`, the **Nodes** tab edits per-node
metadata, and the **TLS** tab manages CA and leaf certificates for the
internal mTLS fabric. The **Settings** tab edits ConfigStore values
live; edits apply without restart.
and skill management with 13 tabs (see also
[Governance](governance.md) for
the Roles, Policies, Skills, Usage, and Audit tabs, and
[Settings](settings.md) for the database-backed configuration editor):
**Users tab:**
-392
View File
@@ -1,392 +0,0 @@
# Coordinator API tour
Turnstone's **coordinator workstream** is a session hosted on the
console whose job is to orchestrate other workstreams. It runs an LLM
that can spawn child workstreams on any node, watch their progress,
wait for them to finish, steer them mid-flight, and tear them down.
This doc walks the full lifecycle — one request, one response, and the
relevant SSE events at each step.
Aimed at integrators driving a coordinator from a custom UI or SDK
without reverse-engineering the built-in console page. The shapes
here match the live OpenAPI spec served at `/openapi.json` and
rendered at `/docs` on every `turnstone-console` process. Every
step references the operation id from that spec so doc updates track
schema changes.
> **Auth throughout.** Every endpoint below sits behind bearer-token
> auth and the `admin.coordinator` permission. A session-scoped JWT
> is minted per login (see [docs/oidc.md](oidc.md) / [docs/security.md](security.md));
> a service token may call the read paths but destructive governance
> paths (`/restrict`, `/stop_cascade`, `/close_all_children`) require
> the explicit `admin.coordinator` grant — a service-token owner
> match isn't enough.
---
## The 9 steps
> **URL convergence (1.5.0).** Pre-1.5 coord-only endpoints lived
> under `/v1/api/coordinator/...`. The Stage 2 verb-shape lift
> consolidated coord and interactive onto the unified
> `/v1/api/workstreams/{ws_id}/<verb>` tree; coord still distinguishes
> itself via the `kind=coordinator` row classifier rather than a
> separate URL space. The endpoints below reflect the post-lift
> surface served by `turnstone-console`.
| # | Action | Operation |
|---|------------------------------|-------------------------------------------------------------|
| 1 | Create | `POST /v1/api/workstreams/new` |
| 2 | Subscribe to events | `GET /v1/api/workstreams/{ws_id}/events` (SSE) |
| 3 | Send a user message | `POST /v1/api/workstreams/{ws_id}/send` |
| 4 | Inspect children | `GET /v1/api/workstreams/{ws_id}/children` |
| 5 | Inspect one workstream | `GET /v1/api/cluster/ws/{ws_id}/detail` |
| 6 | Wait for fan-out | model-side tool `wait_for_workstream` |
| 7 | Govern | `POST /v1/api/workstreams/{ws_id}/trust` |
| | | `POST /v1/api/workstreams/{ws_id}/restrict` |
| | | `POST /v1/api/workstreams/{ws_id}/stop_cascade` |
| | | `POST /v1/api/workstreams/{ws_id}/close_all_children` |
| 8 | Approve / cancel | `POST /v1/api/workstreams/{ws_id}/approve` |
| | | `POST /v1/api/workstreams/{ws_id}/cancel` |
| 9 | Close | `POST /v1/api/workstreams/{ws_id}/close` |
Refer to `/openapi.json` (Swagger UI at `/docs`) on any
`turnstone-console` process for the authoritative operation ids and
schemas. Coordinator-only verbs (`/children`, `/trust`, `/restrict`,
`/stop_cascade`, `/close_all_children`) 404 against `kind=interactive`
rows; the shared verbs (`/send`, `/approve`, `/cancel`, `/events`,
`/history`, `/open`, `/close`, etc.) work on both kinds.
---
## 1. Create a coordinator
```http
POST /v1/api/workstreams/new
Content-Type: application/json
Authorization: Bearer <token>
{
"name": "release-coord",
"skill": "engineer-orchestrator",
"initial_message": "audit /auth for CSRF handling across all active routes"
}
```
```http
HTTP/1.1 201 Created
Content-Type: application/json
{"ws_id": "a1b2c3d4e5f6...", "name": "release-coord"}
```
All three body fields are optional — an empty body still creates a
coordinator with an auto-generated name and no initial message.
Returns **503** with a remediation message when the cluster isn't
configured with a coordinator model; see
[`coordinator.model_alias`](settings.md) to set one.
**SSE implication:** the `ws_created` event fires on the cluster-wide
stream (`/v1/api/cluster/events`) once the row is committed. Per-ws
subscribers (step 2) see the session warm up as token traffic starts.
---
## 2. Subscribe to the per-coordinator event stream
```http
GET /v1/api/workstreams/{ws_id}/events HTTP/1.1
Accept: text/event-stream
Authorization: Bearer <token>
```
One persistent SSE connection per browser tab / SDK caller — the
console fans each event out to every listener queue (cap 500 events
per queue, put_nowait drop on overflow). Events come in flat JSON
with a `type` field. The recurring shapes a UI has to handle:
| `type` | Emitted when | Payload highlights |
|---------------------|--------------------------------------------------------------------------------------------|--------------------|
| `thinking_start` / `thinking_stop` | Model has entered / exited a reasoning block | — |
| `reasoning` | Reasoning-token stream chunk (when the model exposes it) | `text` |
| `content` | Assistant-content stream chunk | `text` |
| `stream_end` | End of a single provider stream | — |
| `tool_result` | A tool call completed (success or error) | `call_id`, `name`, `output`, `is_error?` |
| `tool_output_chunk` | Streaming tool output (e.g. long bash command) | `call_id`, `chunk` |
| `approve_request` | One or more tool calls need operator approval | `items: [{call_id, header, preview, func_name, approval_label, needs_approval}]` |
| `approval_resolved` | Operator answered the approval prompt | `approved`, `feedback` |
| `state_change` | Worker-thread state transition (also re-emitted with the current state on every fresh subscribe so refresh-mid-stream restores composer mode) | `state``running`, `thinking`, `attention`, `idle`, `error` |
| `in_progress_snapshot` | One-shot replay of the in-progress turn's content + reasoning when this client connects mid-stream | `content`, `reasoning` |
| `status` | Token usage + context-window snapshot (fires on every streaming tick) | `prompt_tokens`, `completion_tokens`, `total_tokens`, `context_window`, `pct`, `effort`, `cache_creation_tokens`, `cache_read_tokens` |
| `rename` | Session's display name changed | `name` |
| `intent_verdict` | Intent judge produced a verdict on a pending tool call | `risk_level`, `recommendation`, `reasons` |
| `output_warning` | Output guard flagged a tool result | `call_id`, `risk_level`, `flags` |
| `child_ws_created` | A direct child of this coord was just created (fan-out from the cluster bus) | `child_ws_id`, `node_id`, `name`, `parent_ws_id` (`ws_id` in the envelope is always the coord's own id) |
| `child_ws_state` | A direct child transitioned state | `child_ws_id`, `state` |
| `child_ws_closed` | A direct child closed | `child_ws_id` |
| `child_ws_rename` | A direct child's name changed | `child_ws_id`, `name` |
| `wait_started` / `wait_progress` / `wait_ended` | `wait_for_workstream` tool lifecycle (see §6) | `call_id`, `ws_ids`, `elapsed`, `results`, `complete` |
| `batch_started` / `batch_ended` | `spawn_batch` / `close_all_children` tool lifecycle | `call_id`, `op`, `total`/`succeeded`/`denied`/`closed`/`failed`/`skipped` |
| `info` / `error` | Operational messages | `message` |
**Reconnection contract:** a freshly-opened SSE connection receives
the current snapshot of any pending tool approval (`approve_request`
is re-sent if unresolved), any in-flight `wait_*` / `batch_*`
indicator, the worker's current `state_change`, and an
`in_progress_snapshot` carrying any partial content / reasoning the
model has produced for the in-progress turn — so a tab refresh
mid-approval, mid-tool-execution, or mid-stream restores both the
correct composer mode and the partial assistant text without waiting
for the response to complete.
---
## 3. Send the first user message
```http
POST /v1/api/workstreams/{ws_id}/send
Content-Type: application/json
{"message": "audit /auth for CSRF handling across all active routes"}
```
```http
HTTP/1.1 200 OK
{"status": "ok"}
```
The message is queued for the worker thread at its next tool-result
seam (so you can send follow-ups mid-conversation without corrupting
the in-progress turn). On the SSE stream you'll see `state_change`
`thinking_start` → streaming `reasoning` / `content` / `tool_result`
events, finishing with `state_change → idle` or an
`approve_request` when the model invokes a gated tool.
---
## 4. Inspect direct children
```http
GET /v1/api/workstreams/{ws_id}/children HTTP/1.1
```
```json
{
"items": [
{"ws_id": "d4e5f6...", "name": "csrf-audit", "state": "running", "node_id": "gpu-3"},
{"ws_id": "e1f2a3...", "name": "xss-audit", "state": "idle", "node_id": "gpu-1"}
],
"truncated": false
}
```
The response key is `items`, not `children` — the endpoint shape
follows the cluster-wide workstream-list idiom rather than the
coordinator `list_workstreams` tool's (which uses `children`).
Rows include every state stored for the parent (`running`, `idle`,
`closed`, ...); the endpoint does not accept a state query param,
so clients should inspect each row's `state` field and filter
locally if they want to hide closed/deleted children. Nested
coordinator rows are dropped server-side so only interactive
descendants appear.
---
## 5. Inspect one workstream (storage + live block + tail)
```http
GET /v1/api/cluster/ws/{ws_id}/detail?message_limit=20 HTTP/1.1
```
```json
{
"persisted": { "ws_id": "...", "state": "running", "parent_ws_id": "...", "kind": "interactive", ... },
"live": { "state": "thinking", "tokens": 12843, "activity": "...", "pending_approval": null },
"tail": [ {"role": "assistant", "content": "...", "tokens": 128}, ... ]
}
```
Works for any workstream the caller has `admin.cluster.inspect` on,
not just children of a single coordinator — useful for a cluster
admin panel watching multiple coordinators at once. `live` is
`null` when the owning node is unreachable or has dropped the row
from its dashboard cache; callers should degrade gracefully, not
treat it as an error.
For fan-out views, prefer
[`GET /v1/api/cluster/ws/live?ids=a,b,c`](bulk-endpoints.md) — it
collapses N per-row round-trips into one, returning the live block
for every id in a `{results, denied, truncated}` envelope.
---
## 6. Wait for fan-out (`wait_for_workstream`)
`wait_for_workstream` is a **model-side tool**, not an HTTP endpoint
— the coordinator's LLM invokes it with a list of child ws_ids, the
session's worker thread blocks inside the tool, and a sequence of
`wait_started` / `wait_progress` / `wait_ended` SSE events is emitted
for the UI to drive a "waiting on N children" indicator.
![wait_for_workstream sequence](diagrams/png/27-coordinator-wait-for-workstream.png)
Key properties:
- **Caps** — up to 32 ws_ids per call, up to 600 seconds per call.
A coordinator that needs to wait on more children re-invokes the
tool with a fresh timeout.
- **Modes**`mode="any"` returns as soon as one child reaches a
real terminal state (`idle` / `error` / `closed` / `deleted`);
`mode="all"` waits for every polled child.
- **Progress throttling** — the poll loop runs every 500 ms but the
SSE emission is diff-on-state-change plus a 5-second heartbeat. A
600 s wait generates O(dozens) of progress events, not 1200.
- **Denied rows** — an id the caller doesn't own (cross-tenant) or a
missing row is reported as a `denied` state in the results dict;
`mode="any"` won't satisfy on a pure-denied list (the LLM should
treat it as a config error, not a completion).
Prefer `wait_for_workstream` over polling `inspect_workstream` in a
loop — a wait consumes one assistant turn regardless of how long the
children take, whereas each `inspect_workstream` poll costs a full
turn (plus judge, plus tokens). On a fan-out of 3+ children this
rounds to a 10× token-efficiency win.
---
## 7. Governance — trust, restrict, stop_cascade, close_all_children
These four endpoints let an operator steer a live coordinator session
mid-flight. All four emit an audit event tagged
`coordinator.<action>` via the dedicated audit executor so a cascade
burst can't starve audit writes.
### `POST /trust` — auto-approve own-subtree sends
```http
POST /v1/api/workstreams/{ws_id}/trust
{"send": true}
```
Flips `trust_send=true` on the live session. Subsequent
`send_to_workstream` calls that target a ws_id in the coordinator's
own subtree skip the approval prompt; foreign ws_ids and other tool
calls still go through the normal flow. Requires both
`admin.coordinator` AND `coordinator.trust.send` permissions (the
second grants a service token the opt-in it otherwise wouldn't get).
### `POST /restrict` — revoke tool access mid-session
```http
POST /v1/api/workstreams/{ws_id}/restrict
{"revoke": ["spawn_workstream", "delete_workstream"]}
```
Unions the names into the session's revoked-tools set. Additive and
idempotent — calling twice with overlapping lists converges to the
union. Revocations don't survive a session close/reopen; operators
opt in per session. Cap 256 tool names per request, 128 chars each.
### `POST /stop_cascade` — cancel the subtree
```http
POST /v1/api/workstreams/{ws_id}/stop_cascade
{}
```
Cancels the coordinator's in-flight generation AND dispatches
`cancel_workstream` through the routing proxy for every direct
child in the in-memory registry. Returns:
```json
{"status": "ok", "cancelled": ["child-1", "child-3"], "failed": [], "skipped": ["child-2"]}
```
Response uses the [cascade-mutation bulk shape](bulk-endpoints.md):
`cancelled` = accepted, `failed` = dispatch error worth retrying,
`skipped` = upstream 404 (already gone — stale registry entry or
the row was deleted between snapshot and dispatch). Grandchildren
aren't touched directly; they sit behind their parent's cancel and
propagate via the child's SSE stream.
### `POST /close_all_children` — soft-close the direct fan-out
```http
POST /v1/api/workstreams/{ws_id}/close_all_children
{"reason": "audit round complete"}
```
Response:
```json
{"status": "ok", "closed": ["c-1", "c-2"], "failed": [], "skipped": []}
```
Soft-close cascade bounded by the same semaphore as `stop_cascade`.
The `reason` (up to 512 chars) propagates into each closed child's
audit + `workstream_config` for postmortem. Unlike `stop_cascade`
this does NOT recurse into grandchildren — the model-facing tool
that pairs with this endpoint asks for a bounded teardown of the
coordinator's own fan-out. For a full-subtree teardown, use
`stop_cascade`.
See [bulk-endpoints.md](bulk-endpoints.md) for why both endpoints
share the cascade-mutation shape and how it differs from the
`spawn_batch` / `cluster/ws/live` shape.
---
## 8. Approve / cancel
The `approve` endpoint is what resolves an `approve_request` SSE
event. The coordinator's worker thread is blocked inside
`ui.approve_tools` waiting for this POST.
```http
POST /v1/api/workstreams/{ws_id}/approve
{"approved": true, "feedback": null, "always": false}
{"approved": false, "feedback": "spawn count looks too high — try 3 not 10"}
{"approved": true, "feedback": null, "always": true} // always-approve this tool name
```
`cancel` drops the in-flight generation but leaves the coordinator
idle and open for a fresh `send`:
```http
POST /v1/api/workstreams/{ws_id}/cancel
{}
```
---
## 9. Close
```http
POST /v1/api/workstreams/{ws_id}/close
{}
```
Soft-closes the session — state persists, children keep running (use
`close_all_children` or `stop_cascade` first to wind them down), the
worker thread exits, SSE streams send a final `stream_end` and
disconnect. The row is reopenable via
`POST /v1/api/workstreams/{ws_id}/open` so long as it hasn't been
deleted.
---
## Further reading
- [coordinator-skills.md](coordinator-skills.md) — writing a skill
that runs on a coordinator session (orchestrator persona,
workflow patterns, `SkillKind` classifier).
- [bulk-endpoints.md](bulk-endpoints.md) — the two bulk-shape
idioms (`{results, denied, truncated}` vs
`{<bucket>, failed, skipped}`) used by `cluster/ws/live`,
`spawn_batch`, `stop_cascade`, and `close_all_children`.
- [architecture.md](architecture.md) — cluster-wide architecture
including how coordinator sessions fit next to node-hosted
interactive workstreams.
- The live OpenAPI spec (`/openapi.json` on any console process)
and Swagger UI (`/docs`) — authoritative schemas for every
endpoint above.
-330
View File
@@ -1,330 +0,0 @@
# Writing a coordinator-specific skill
Skills are prompt-level personas that steer a Turnstone session
toward a narrow task. Most skills target **interactive** sessions —
the single-workstream "do this thing" surface where the model wields
`bash`, `edit_file`, `web_fetch`, and the rest of the maker toolset.
A **coordinator skill** is different. It runs on a session whose job
is to orchestrate other sessions. The toolset is smaller and
narrower, the persona is an orchestrator instead of a maker, and the
success metric is "did the plan resolve" instead of "did the code
compile". This doc covers the differences a skill author has to
care about.
---
## The two-surface model
A row in `prompt_templates` carries a `kind` column (see
[`turnstone/core/skill_kind.py`](../turnstone/core/skill_kind.py);
migration 044 added the column). Three values:
| `SkillKind` enum | Stored as | Visible in |
|----------------------|-----------------------------|---------------------------------------------------------------------------|
| `SkillKind.INTERACTIVE` | `"interactive"` | Only the interactive-session activation path. `list_skills` on a coord won't show it. |
| `SkillKind.COORDINATOR` | `"coordinator"` | Only the coordinator's `list_skills` tool. Hidden from interactive activation pickers. |
| `SkillKind.ANY` | `"any"` | Both surfaces. Default for legacy rows predating the classifier. |
The `kind` field is a `StrEnum` — drop-in ``str`` compatible — so
DB rows, JSON payloads, and `==` comparisons all work without
translation at the edge.
When a coordinator calls `list_skills`, the SQL filter narrows to
`kind IN ('coordinator', 'any')`. When an interactive session picks
a skill at activation, the filter narrows to
`kind IN ('interactive', 'any')`. A skill author tags once at
creation; the two surfaces stay partitioned without any
per-call filtering on the LLM side.
**Tagging a new skill as coordinator-only** — set `kind` to
`SkillKind.COORDINATOR` (or the literal string `"coordinator"`) when
you POST to `/v1/api/admin/skills`. Existing rows default to
`SkillKind.ANY`; bump them to `COORDINATOR` if you've rewritten the
prompt around the orchestrator toolset.
---
## Tool surface differences
Coordinator sessions receive a **fixed** tool set, defined in
`turnstone/core/tools.py` as `COORDINATOR_TOOLS`. Nothing a skill
or MCP config can do adds to it. Current members:
| Tool | Category | Notes |
|---------------------------|-----------------|---------------------------------------------------------------------|
| `spawn_workstream` | delegate | Create one child. Requires approval. |
| `spawn_batch` | delegate | Create up to 10 children in one approval. Partial-success shape. |
| `inspect_workstream` | observe | Read state + tail of one child. Auto-approved (no mutation). |
| `list_workstreams` | observe | List the direct children (same shape as `/children` endpoint). |
| `wait_for_workstream` | block | Block until one/all listed children hit a terminal state. |
| `send_to_workstream` | steer | Queue a follow-up message to a running child. |
| `close_workstream` | wind-down | Soft-close one child. Requires approval. |
| `close_all_children` | wind-down | Soft-close every direct child in one approval. Partial-success shape. |
| `cancel_workstream` | wind-down | Drop the in-flight generation; leaves child idle for a fresh send. |
| `delete_workstream` | wind-down | Hard-delete one child. Requires approval. |
| `list_nodes` | discover | Enumerate live cluster nodes + capabilities. |
| `list_skills` | discover | Coordinator-visible skills only (SkillKind filter above). |
| `tasks` | plan | Orchestrator-only scratchpad. Children don't see it. |
Explicitly **not** in the coordinator set:
- `bash` / `edit_file` / `write_file` / `append_file` / `diff_file` — no local FS.
- `read_file` / `search` — no local FS reads.
- `web_fetch` / `web_search` — no direct web access.
- `task_agent` / `plan_agent` — sub-agent tools are zeroed on coord sessions.
- `memory` / `recall` / `notify` / `watch` / `read_resource` / `use_prompt` / `skill` — the orchestrator's "memory" is its children's outputs; these UX / persistence tools belong to interactive sessions.
If your skill needs a coordinator to "run a command" or "read a
file", write the delegate pattern instead: spawn a child with an
appropriate skill, `wait_for_workstream`, then `inspect_workstream`
for the output. The coordinator stays the orchestrator.
---
## Persona differences
Interactive skills compose on top of `base_interactive.md` — a
"maker" persona: get the work done, use the tools, edit the code,
close the loop.
Coordinator skills compose on top of
[`base_coordinator.md`](../turnstone/prompts/base_coordinator.md) —
an "orchestrator" persona: decompose, delegate, monitor, synthesise.
The base text is short but sets the tone every coordinator skill
inherits:
> You are a coordinator on a small, focused infrastructure team.
> Your role is to orchestrate work across the cluster... You do
> not edit files, run shell commands, browse the web, or manipulate
> the codebase directly. Children do that.
Write your skill's system prompt to *add* task-specific orchestration
hints on top — don't re-explain the role, don't paste tool JSON,
don't try to override the "no direct action" contract. Keep the
additions to: (a) the specific kind of work this skill delegates;
(b) the preferred skill tags for children; (c) the synthesis shape
the skill should end on.
---
## `tasks` integration
`tasks` is the coordinator's scratchpad — a persisted, ordered
list of rows with fields `{id, title, status, child_ws_id, created,
updated}` that only this coordinator sees. Children don't see it;
the user does via the sidebar. Five actions: `add`, `update`,
`remove`, `reorder`, `list` (only `list` is auto-approved; the
mutators go through the approval flow).
The input schema refers to rows by `task_id`; the persisted row
object exposes the same id as `id`. The `child_ws_id` field is a
free-form label the skill sets to link a task to a spawned
workstream — it is NOT validated against the workstreams table, so
a skill can set it to a placeholder before `spawn_workstream`
returns or keep it pointing at a closed child for later audit.
A skill's initial prompt can seed the task list by calling
`tasks(action="add", title=...)` as its very first tool calls —
the user gets a visible plan before any child is spawned, and the
coordinator's future self has something concrete to iterate on.
Status transitions (`pending``in_progress``done` / `blocked`)
are the skill's main feedback loop: mutate the task when the child
covering it finishes, not when the child starts. Use
`tasks(action="update", task_id=..., child_ws_id=<ws_id>)` to
link a task to the child that owns it once spawn returns.
A final gotcha: parallel tool dispatch does NOT serialise reads
after writes in the same batch. If a skill issues an `update` and
a `list` in one parallel tool batch, the `list` response may reflect
the pre-update state. Dispatch mutate and list serially (one
tool_use turn each) when the list must observe the mutation.
Keep the tasks coarse-grained — one per child, roughly. A 20-task
list for a 3-child fan-out is noise; a 1-task list for a 5-child
fan-out loses the plan. The sidebar renders tasks as the operator's
mental model of "what the coord thinks it's doing".
---
## Referencing children by `ws_id`
Every ws_id returned by `spawn_workstream` / `spawn_batch` is a
**full 32-char hex string**. The skill's system prompt must not
invent ws_ids — a model that hallucinates `"child-1"` or `"ws-abc"`
hits the tenant guard in `CoordinatorClient._is_own_subtree`, which
validates ws_id against `parent_ws_id=coord_ws_id` AND
`user_id=owner` in storage. The rejection shape varies by tool:
- **Mutating ops** (`send_to_workstream`, `close_workstream`,
`cancel_workstream`, `delete_workstream`) return
`{"error": "workstream not in coordinator subtree: <ws_id>", "status": 404}`
— the skill should treat this as a tool error, not an empty result.
- **`inspect_workstream`** returns `{"error": "workstream not found", "ws_id": "<ws_id>"}`
(same shape as a genuinely missing row, so the guard can't be
used as an existence oracle).
- **`wait_for_workstream`** reports the offending id with
`state="denied"` in its `results` dict; `mode="any"` won't
satisfy on a pure-denied list, so a hallucinated id won't trick
the wait into reporting "complete".
Pattern: capture each spawn result in the next tool call's input.
The JSON tool-result carries `{"child_ws_id": "...", "name": "...",
"node_id": "...", "routing_strategy": "..."}`; the model should
extract the `child_ws_id` and pass it as `ws_id` (or in the `ws_ids`
list) to `inspect_workstream` / `wait_for_workstream` /
`send_to_workstream` / `close_workstream` verbatim. The asymmetry
— spawn returns `child_ws_id` but the other tools accept `ws_id` /
`ws_ids` — is intentional: it defuses a coordinator-LLM recency
bias where seeing `ws_id` in a spawn return primed re-spawn loops
instead of progression to the wait phase.
A UI that wants human-readable identifiers should render the `name`
field and keep the workstream id as the click-through key — note
that the id *value* is the same regardless of whether it arrived
under the `child_ws_id` key (spawn return) or the `ws_id` key
(every other tool's input/output); only the field name differs.
---
## `wait_for_workstream` vs `inspect_workstream`
Two distinct semantics, different cost profiles:
- **`wait_for_workstream(ws_ids=[...], timeout=60, mode="any")`** —
blocks inside a single tool call until one (or all, for `mode="all"`)
of the listed children reaches a terminal state (`idle`, `error`,
`closed`, `deleted`). The worker thread blocks up to `timeout`
seconds; the assistant turn remains a single round-trip regardless
of how long the wait actually takes. Prefer this for "the plan
needs child X to finish before the next step."
- **`inspect_workstream(ws_id=...)`** — single read of the child's
state + tail. Costs a full assistant turn (judge, tokens, stream).
Prefer this for "what does the final message say?" after the child
has already resolved (via `wait_for_workstream` or a known
transition).
Rule of thumb: wait once for a fan-out, then inspect once per
child for the content. A loop of inspect-every-few-seconds is a
token-burning antipattern — on 3+ children it rounds to a 10×
efficiency hit over a wait+inspect pair.
---
## Common coordinator patterns
Three patterns cover most coordinator skills. Pick the one that
matches the task, or combine them deliberately.
### Pattern 1 — delegate-and-summarise
One specialist child, one focused brief, one synthesis message back
to the user. Appropriate when the user's request is "run the thing
and tell me what happened" and the work fits in one workstream.
```
tasks(action='add', title='audit /auth for CSRF')
spawn_workstream(skill='engineer', initial_message='audit /auth ...')
wait_for_workstream(ws_ids=[<child>], timeout=300)
inspect_workstream(ws_id=<child>)
→ synthesise the final message into a user-facing response
tasks(action='update', task_id='t_01', status='done')
close_workstream(ws_id=<child>, reason='audit complete')
```
### Pattern 2 — fan-out-and-synthesise
N children running in parallel, each with a distinct brief, all
waited-on together, then synthesised. Appropriate when the user's
request naturally decomposes into independent subtasks.
```
tasks seeds:
t_01 benchmark Anthropic 4.7 latency on summarisation
t_02 benchmark OpenAI GPT-5.2 latency on summarisation
t_03 benchmark Gemini 2.5 latency on summarisation
spawn_batch(children=[...3 briefs...])
wait_for_workstream(ws_ids=[c1, c2, c3], mode='all', timeout=600)
inspect_workstream(ws_id=c1); ...(c2); ...(c3)
→ synthesise head-to-head comparison
tasks → all done
close_all_children(reason='benchmark complete')
```
Prefer `spawn_batch` over 3 individual `spawn_workstream` calls —
one approval instead of three, one audit trail, deterministic
sibling ordering. Pair with `wait_for_workstream(mode='all')` and
`close_all_children(reason=...)` to wind the fan-out down in one
approval each.
### Pattern 3 — plan-then-delegate
The coordinator first uses its own reasoning to carve the plan,
records it in `tasks`, then spawns children that each own one
task. Appropriate when the user's request is "figure out how to X"
and the coordinator's planning step is itself valuable.
```
→ coord reasons about the shape of the work
tasks(action='add', title='...') × N # the plan, visible in the sidebar
for task in tasks:
spawn_workstream(skill=..., initial_message=task.brief)
tasks(action='update', task_id=task.id, notes='ws=<child_ws_id>')
wait_for_workstream(ws_ids=[...], mode='all', timeout=...)
for child in children:
inspect_workstream(ws_id=child)
tasks(action='update', task_id=..., status='done', notes='result summary')
→ synthesise
```
The key distinction from Pattern 2: the plan is an artifact the user
can see and interact with (via the sidebar). If the coordinator's
reasoning-pass was wrong about the decomposition, the user can
course-correct before any child runs.
---
## Testing a coordinator skill
Coordinator sessions are hosted on the console, not on a node.
Integration tests that drive a real coord session live under
`tests/test_coordinator_end_to_end.py` — they spin a console with
an in-memory SQLite backend and a fake upstream node, then drive
the session through its HTTP surface.
For a new coordinator skill:
1. Write the skill prompt as a string and pass it to the
`coord_session` fixture's `skill=` kwarg (see
`tests/test_coordinator_tools.py` for the pattern).
2. Build a small fake cluster: one node + two children via
the `_seed_children` helper in `tests/_coord_test_helpers.py`
(``_seed_children(mgr._adapter, coord.id, ["child-1", "child-2"])``).
3. Drive the session with seeded tool_call dicts matching the
provider layer's shape. The unit-level tests in
`tests/test_coordinator_tools.py` show the helper (`_tc(name,
args, call_id)`).
4. Assert the skill's decision shape — which tools fire in what
order, what the tasks looks like at the end, which
`_error` reasons appear on the denied-path.
A full end-to-end test isn't required for every skill; a
prepare-step unit test that asserts "given this initial message, the
first tool call is X with Y args" is usually sufficient to catch
persona drift without a real LLM in the loop.
---
## Further reading
- [coordinator-api-tour.md](coordinator-api-tour.md) — the HTTP
surface every coordinator skill indirectly drives.
- [bulk-endpoints.md](bulk-endpoints.md) — the response shape
`spawn_batch` and `close_all_children` use, so your skill can
parse results / denied arrays correctly.
- [governance.md](governance.md) — the broader governance surface
(`/trust`, `/restrict`, `/stop_cascade`, role-based permissions)
that wraps every coord session.
- [settings.md](settings.md) — `coordinator.model_alias` and
`coordinator.reasoning_effort` settings that gate which LLM runs
the coordinator session at all.
+33 -27
View File
@@ -1,27 +1,34 @@
# Consistent Hash Ring — Reference Design
**Status**: Reference — alternative routing strategy
**Status**: Reference (not currently in the hot path)
**Date**: 2026-03-30
Live routing uses **rendezvous (HRW) hashing** in
`turnstone/core/rendezvous.py` and `turnstone/console/router.py`. This
document captures a vnode-ring approach as a reference for future
evaluation if the cluster outgrows rendezvous's O(N)-per-route
characteristic.
## Overview
The FNV-1a-32 hash function specified below is bit-identical to the
hash used by the live rendezvous implementation; cross-language clients
can rely on these test vectors.
This document describes a consistent hash ring algorithm evaluated during
the design of the direct HTTP transport routing system. The current
implementation uses weight-proportional bucket assignment with a
donor/recipient rebalancing algorithm (see `direct-http-transport.md`).
The consistent hash ring is documented here as a reference for future
scalability work — if the cluster grows beyond the point where the
weight-proportional approach is sufficient, the ring provides a
proven alternative with stronger stability guarantees.
## When the ring approach becomes interesting
## When to consider the ring approach
The vnode ring becomes preferable to rendezvous hashing when:
The current weight-proportional seeding + donor/recipient rebalancer works
well when:
- Cluster size is moderate (< 50 nodes)
- Nodes join/leave infrequently
- The rebalancer runs centrally (in the console)
- Cluster size grows large (50+ nodes) and the per-route O(N) hash
computation becomes visible against downstream HTTP cost.
- Decentralised routing is needed (each node computes the ring locally,
no central console required).
- A precomputed flat-array lookup is desired so the routing hot path
avoids hashing entirely.
The consistent hash ring becomes advantageous when:
- Cluster size grows large (50+ nodes) and frequent membership changes
cause the donor/recipient algorithm to churn
- Decentralized routing is needed (each node computes the ring locally,
no central console required)
- Cross-language determinism is important (multiple implementations must
agree on the same assignment without sharing state)
## Algorithm
@@ -126,17 +133,16 @@ class HashRing:
# Precompute all 65536 bucket assignments
```
## Comparison with rendezvous (HRW) hashing
## Comparison with current approach
| Aspect | Rendezvous (live) | Consistent hash ring (this doc) |
|--------|-------------------|---------------------------------|
| Per-route cost | O(N) hash computes | O(log V) bisect against precomputed array |
| Seeding | None — pure function | Build vnode array on every membership change |
| Node addition | Pure function moves ~1/N keys | Ring moves ~1/N buckets |
| Node removal | Surviving nodes' keys unchanged | Surviving nodes' buckets unchanged |
| Decentralised | Yes — pure function over services | Yes each node computes locally |
| Persistent state | None | None on the hot path; precomputed array in memory |
| Complexity | ~20 LOC | Virtual-node construction + bisect |
| Aspect | Weight-proportional (current) | Consistent hash ring |
|--------|------------------------------|---------------------|
| Seeding | Exact weight split, deterministic | Hash-based, ~3% variance |
| Node addition | Donor/recipient moves only excess | Ring moves ~1/N buckets |
| Node removal | Dead buckets → most underloaded | Ring redistributes to clockwise neighbors |
| Cross-node churn | Zero (only donor→recipient) | Zero (ring stability guarantee) |
| Decentralized | No (needs central rebalancer) | Yes (each node computes locally) |
| Complexity | Simple weight arithmetic | Virtual node construction + bisect |
## Test vectors
+1 -1
View File
@@ -43,7 +43,7 @@ eval --> sqlite : SQLite
console --> server : HTTP proxy\n(hash-ring bucket lookup,\nproxy /node/{id}/* traffic)
channel --> server : HTTP + SSE\n(POST /v1/api/workstreams/{ws_id}/send,\nGET /v1/api/workstreams/{ws_id}/events)
channel --> server : HTTP + SSE\n(POST /v1/api/send,\nGET /v1/api/events)
' Notes
note right of console
+4 -9
View File
@@ -19,8 +19,7 @@ package "Entry Points" <<Rectangle>> {
component [cli.py\nturnstone] as cli <<entry>>
component [server.py\nturnstone-server] as server <<entry>>
component [eval.py\nturnstone-eval] as eval <<entry>>
component [admin.py\nturnstone-admin] as admin <<entry>>
component [bootstrap.py\nturnstone-bootstrap] as bootstrap <<entry>>
component [chat.py\n(re-exports)] as chat <<entry>>
}
' Core engine
@@ -40,7 +39,7 @@ package "turnstone/core/" <<Rectangle>> {
component [auth.py\nAuthentication] as auth <<core>>
component [healthcheck.py\nBackendHealthMonitor] as healthcheck <<core>>
component [ratelimit.py\nRateLimiter] as ratelimit <<core>>
component [mcp_client.py\nMCPClientManager\n(push + manual refresh)] as mcp <<core>>
component [mcp_client.py\nMCPClientManager\n(push + periodic refresh)] as mcp <<core>>
component [tool_search.py\nToolSearchManager, BM25] as toolsearch <<core>>
component [model_registry.py\nModelRegistry] as registry <<core>>
}
@@ -49,8 +48,7 @@ package "turnstone/core/" <<Rectangle>> {
package "turnstone/channels/" <<Rectangle>> {
component [_routing.py\nChannelRouter] as router <<channel>>
component [discord/bot.py\nDiscordBot] as discordbot <<channel>>
component [slack/bot.py\nSlackBot (Socket Mode)] as slackbot <<channel>>
component [cli.py\nturnstone-channel] as gateway <<channel>>
component [gateway.py\nturnstone-channel] as gateway <<channel>>
}
' Console
@@ -114,8 +112,7 @@ eval --> memory
eval --> config
eval --> tools
admin --> auth
bootstrap --> providers
chat --> session
' Core internal deps
session --> providers
@@ -138,10 +135,8 @@ tools --> schemas
' Channel dependencies
gateway --> discordbot
gateway --> slackbot
gateway --> router
discordbot --> sdkserver : HTTP + SSE
slackbot --> sdkserver : HTTP + SSE
router --> storage : channel_routes
' Console dependencies
+4 -9
View File
@@ -69,10 +69,9 @@ class "NullUI" as NullUI {
interface "LLMProvider" as LLMProvider <<Protocol>> {
+ provider_name: str {property}
+ get_capabilities(model) → ModelCapabilities
+ create_streaming(client, model, messages, ..., replay_reasoning_to_model) → Iterator[StreamChunk]
+ create_completion(client, model, messages, ..., replay_reasoning_to_model) → CompletionResult
+ create_streaming(client, model, messages, ...) → Iterator[StreamChunk]
+ create_completion(client, model, messages, ...) → CompletionResult
+ convert_tools(tools) → list[dict]
+ extract_reasoning_text(provider_blocks) → str
+ retryable_error_names: frozenset[str] {property}
--
core/providers/_protocol.py
@@ -127,7 +126,6 @@ class "ModelCapabilities" as ModelCaps <<frozen>> {
+ supports_web_search: bool
+ supports_tool_search: bool
+ supports_vision: bool
+ supports_reasoning_replay: bool
}
' ChatSession
@@ -255,7 +253,7 @@ class "MCPClientManager" as MCPMgr {
Background asyncio event loop
bridges async MCP SDK to
sync ChatSession dispatch.
Push + manual refresh.
Push + periodic + manual refresh.
Resources + prompts discovered
alongside tools at startup.
--
@@ -297,7 +295,7 @@ class "ModelRegistry" as ModelReg {
--
Thread-safe lazy client + provider
creation. Loaded by load_model_registry()
from DB + [models.*] config + CLI args.
from CLI args + [models.*] config.
--
core/model_registry.py
}
@@ -308,9 +306,6 @@ class "ModelConfig" as ModelCfg <<frozen>> {
+ base_url: str
+ model: str
+ context_window: int
+ temperature: float | None
+ max_tokens: int | None
+ reasoning_effort: str | None
}
' Circuit breaker state
-16
View File
@@ -24,14 +24,6 @@ CS -> DB : save_message(ws_id, "user", input)
group loop [while tool_calls present]
CS -> UI : on_turn_start()
note right of UI
SessionUIBase resets the per-turn inflight
buffers (_ws_inflight_content / reasoning /
seq) that fuel the SSE in_progress_snapshot
event for mid-stream refresh resume.
end note
CS -> UI : on_state_change("thinking")
CS -> UI : on_thinking_start()
@@ -81,14 +73,6 @@ group loop [while tool_calls present]
CS -> CS : _update_token_table()\ncalibrate chars_per_token ratio
CS -> CS : messages.append(assistant_msg)
CS -> UI : on_turn_committed()
note right of UI
Drops the per-turn inflight buffers — the
assistant message is now in the history
list, so the in_progress_snapshot must
not re-render it during the next tool-
execution window or the next streaming turn.
end note
CS -> DB : save_message(ws_id, "assistant", content)
CS -> DB : save_message(ws_id, "tool_call", ...) ×N
+6 -6
View File
@@ -170,15 +170,15 @@ Server --> Browser : Shimmed app.js
deactivate Server
note right of Browser
All fetch("/v1/api/workstreams/{ws_id}/send") calls in the
server UI now become fetch("/node/nodeA/v1/api/workstreams/{ws_id}/send"),
All fetch("/v1/api/send") calls in the
server UI now become fetch("/node/nodeA/v1/api/send"),
routed through the console proxy.
end note
Browser -> Server : GET /node/nodeA/v1/api/workstreams/ws789/events
Browser -> Server : GET /node/nodeA/v1/api/events?ws_id=ws789
activate Server #FFF9C4
Server -> NodeA : GET http://10.0.1.1:8080/v1/api/workstreams/ws789/events\n(SSE stream via httpx.AsyncClient timeout=None)
Server -> NodeA : GET http://10.0.1.1:8080/v1/api/events?ws_id=ws789\n(SSE stream via httpx.AsyncClient timeout=None)
activate NodeA
loop SSE streaming
@@ -189,10 +189,10 @@ end
deactivate NodeA
deactivate Server
Browser -> Server : POST /node/nodeA/v1/api/workstreams/ws789/send\n{message:"hello"}
Browser -> Server : POST /node/nodeA/v1/api/send\n{message:"hello", ws_id:"ws789"}
activate Server #FFF9C4
Server -> NodeA : POST http://10.0.1.1:8080/v1/api/workstreams/ws789/send\n(body forwarded)
Server -> NodeA : POST http://10.0.1.1:8080/v1/api/send\n(body forwarded)
activate NodeA
NodeA --> Server : {status:"ok"}
deactivate NodeA
+1 -1
View File
@@ -23,7 +23,7 @@ interface "StorageBackend" as SB <<protocol>> {
+resolve_workstream(alias_or_id) → str | None
+delete_workstream(ws_id) → bool
+prune_workstreams(retention_days) → (int, int)
+list_workstreams(node_id, limit, *, parent_ws_id, kind, user_id) → list
+list_workstreams(node_id, limit) → list
+save_workstream_config(ws_id, config)
+load_workstream_config(ws_id) → dict
+kv_get(key) → str | None
+1 -1
View File
@@ -79,7 +79,7 @@ class "Scope Hierarchy" as SH <<scope>> {
--
GET → read
POST write paths → write
POST /api/workstreams/{ws_id}/approve → approve
POST /api/approve → approve
/api/admin/* → approve
}
+13 -34
View File
@@ -20,14 +20,11 @@ class "Discord" as Discord <<platform>> {
asyncio event loop
}
class "Slack" as Slack <<platform>> {
Socket Mode WebSocket
class "Slack (future)" as Slack <<platform>> {
Socket Mode / Events API
Block Kit messages
Slash command (default /turnstone)
DM + channel events
--
slack-bolt (Python)
asyncio event loop
Planned integration
}
class "Teams (future)" as Teams <<platform>> {
@@ -41,7 +38,7 @@ class "Teams (future)" as Teams <<platform>> {
class "turnstone-channel" as ChannelService <<service>> {
entry point: turnstone-channel
--
One process — hosts one or more adapters
One process per platform
asyncio event loop
Structured logging (structlog)
--log-level, --log-format
@@ -50,19 +47,6 @@ class "turnstone-channel" as ChannelService <<service>> {
GET /health
}
class "SlackBot" as SlackBot <<service>> {
+on_message(event)
+on_action(action) (Block Kit buttons)
+send(channel_id, content)
+send_notification(channel_id, content, ws_id)
+run(bot_token, app_token)
--
slack-bolt AsyncApp
Socket Mode client
Per-user channel sessions via slash command
DM routing without slash command
}
class "DiscordBot" as Bot <<service>> {
+on_message(msg)
+on_interaction(interaction)
@@ -95,10 +79,10 @@ class "ChannelRouter" as Router <<service>> {
' -- Server --
class "turnstone-server" as Server <<server>> {
POST /v1/api/workstreams/{ws_id}/send
POST /v1/api/workstreams/{ws_id}/approve
POST /v1/api/send
POST /v1/api/approve
POST /v1/api/workstreams/new
GET /v1/api/workstreams/{ws_id}/events
GET /v1/api/events?ws_id=
--
LLM execution + tool use
SSE event stream
@@ -148,21 +132,16 @@ Bot --> Router : on_message\non_interaction
Router --> CU : resolve identity
Router --> CR : resolve / register route
Router --> Server : POST /v1/api/workstreams/{ws_id}/send\nPOST /v1/api/workstreams/{ws_id}/approve\nPOST /v1/api/workstreams/new
Bot --> Server : GET /v1/api/workstreams/{ws_id}/events\n(SSE via httpx-sse)
Router --> Server : POST /v1/api/send\nPOST /v1/api/approve\nPOST /v1/api/workstreams/new
Bot --> Server : GET /v1/api/events?ws_id=\n(SSE via httpx-sse)
Server --> Bot : SSE event stream
Bot --> Discord : reply / embed\nbutton callback
Slack --> SlackBot : socket-mode\nevents
SlackBot --> Router : on_message / on_action
SlackBot --> Server : POST /v1/api/workstreams/{ws_id}/send\nGET /v1/api/workstreams/{ws_id}/events
SlackBot --> Slack : post / update\nBlock Kit button callbacks
Slack .[hidden]. Discord
Teams .[hidden]. Slack
ChannelService --> Bot : creates + runs
ChannelService --> SlackBot : creates + runs
ChannelService --> Router : creates
ChannelService --> SVC : register / heartbeat /\nderegister
@@ -179,7 +158,7 @@ note right of Bot
(or creates new workstream)
4. ChannelRouter resolves platform user -> user_id
via channel_users table
5. Router sends POST /v1/api/workstreams/{ws_id}/send to server
5. Router sends POST /v1/api/send to server
**Workstream Resume (evicted workstreams)**
1. Stale route detected (no active SSE listener)
@@ -193,7 +172,7 @@ end note
note right of Server
**Outbound Flow**
1. Server emits SSE events on
GET /v1/api/workstreams/{ws_id}/events
GET /v1/api/events?ws_id=
2. Bot subscribes via httpx-sse
3. Bot formats and sends to Discord thread
end note
@@ -204,7 +183,7 @@ note bottom of CR
2. Bot renders Discord buttons (Approve / Deny)
3. User clicks button -> on_interaction()
4. Router builds ApproveMessage
5. Router sends POST /v1/api/workstreams/{ws_id}/approve to server
5. Router sends POST /v1/api/approve to server
end note
note bottom of CU
+11 -15
View File
@@ -190,25 +190,21 @@ group Push Notifications (debounced 5s per server)
MCPMgr -> Storage : sync_prompts_to_storage()
end
group Manual Refresh
Session -> MCPMgr : refresh_sync()
group Periodic Polling (default 4h)
MCPMgr -> MCPMgr : _periodic_refresh()
note right
/mcp refresh [server] —
re-fetches catalog and
attempts reconnect for
disconnected servers.
Only polls capabilities
without push support.
Staggered per-server.
Disconnected servers get
reconnect attempts with
exponential backoff (60s-1h).
end note
end
group Manual Reconnect
Session -> MCPMgr : reconnect_sync(name)
note right
Operator-driven via the
console admin panel —
tears down session, clears
circuit breaker, runs a
fresh handshake.
end note
group Manual Refresh
Session -> MCPMgr : refresh_sync()
note right: /mcp refresh [server]
end
== Policy Evaluation ==
+1 -1
View File
@@ -211,7 +211,7 @@ note over Session, Judge
**Storage:**
intent_verdicts table (migration 012), output_assessments table
(migration 022). Both queryable via admin API endpoints
(requires admin.judge permission). Skills store risk_level,
(requires admin.judge permission). Skills store scan_status,
scan_report, scan_version for install-time risk assessment.
end note
@@ -1,89 +0,0 @@
@startuml
title Turnstone - coordinator wait_for_workstream lifecycle
skinparam sequenceArrowThickness 1.5
skinparam noteBackgroundColor #FDF6E3
participant "Coordinator\nLLM" as LLM
participant "ChatSession\n(worker thread)" as CS
participant "CoordinatorClient" as CC
participant "SessionUI\n(SSE fanout)" as UI
participant "Console routing\nproxy" as RP
database "Storage\n(workstreams row)" as DB
participant "Child\nnode" as NODE
== Spawn ==
LLM -> CS : tool_call spawn_workstream(...)
activate CS
CS -> CC : spawn(initial_message=...,\nparent_ws_id=coord, user_id=...)
CC -> RP : POST /v1/api/route/workstreams/new
RP -> NODE : dispatch (rendezvous)
NODE -> DB : insert workstreams row\nstate='running'
RP --> CC : {ws_id, node_id, name, status: 200}
CC --> CS : {ws_id, ...}
CS -> UI : on_tool_result\n("spawn_workstream", ws_id)
deactivate CS
note right of LLM
Model now knows the child ws_id.
It can inspect / send / wait, and
the parent registry tracks it.
end note
== Wait (blocking) ==
LLM -> CS : tool_call wait_for_workstream\n(ws_ids=[child], mode="any", timeout=60)
activate CS
CS -> CS : _prepare_wait_for_workstream\n(validate ws_ids, timeout, mode)
CS -> UI : emit wait_started\n{call_id, ws_ids, mode, timeout}
CS -> CC : wait_for_workstream(ws_ids, timeout,\nmode, progress_callback)
activate CC
loop every 500ms up to timeout
CC -> DB : read workstreams row(s)
DB --> CC : {state, updated, tokens, ...}
alt state in {idle, error, closed, deleted}
note over CC
real-terminal state ->
completion condition met
end note
else still running / thinking / attention
CC -> CS : progress_callback(snap)\n(diff-on-change or 5s heartbeat)
CS -> UI : emit wait_progress\n{call_id, elapsed, results?}
end
end
CC --> CS : {complete, elapsed,\nresults: {ws_id: snap}}
deactivate CC
CS -> UI : emit wait_ended\n{call_id, complete, elapsed, results}
CS -> UI : on_tool_result\n("wait_for_workstream",\n"complete after Ns (R/N resolved)")
CS --> LLM : tool_result (full results dict)
deactivate CS
note left of UI
Sidebar "waiting on N children" indicator
keys on call_id - started / progress / ended
scope to a single wait invocation so
nested waits render independent badges.
end note
== After wait: inspect + close ==
LLM -> CS : tool_call inspect_workstream(ws_id=child)
CS -> CC : inspect(ws_id)
CC -> DB : read row + tail
CC --> CS : {state, messages, tokens, ...}
CS --> LLM : tool_result (serialised)
LLM -> CS : tool_call close_workstream\n(ws_id=child, reason="...")
CS -> CC : close_workstream(ws_id, reason)
CC -> RP : POST /v1/api/route/workstreams/close
RP -> NODE : dispatch
NODE -> DB : state='closed',\nclose_reason='...'
RP --> CC : {status: 200}
CC --> CS : {closed: true, status: 200, reason: ...}
CS --> LLM : tool_result
@enduml
+2 -2
View File
@@ -1,3 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:95dd5ebc899a1261d516686a5aa3319a7f45015d411302825fa28afbfc82e1ce
size 326766
oid sha256:75da80e6bd205e45b9fe48aa2f87197110c908f44e1d98c52eee69f956274711
size 400402
+2 -2
View File
@@ -1,3 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:9857db23fe3c4316d492073aac69c7e7558b1abe3b95ad7756d4a5933bd0ece7
size 620214
oid sha256:6471e611beebf647f3a191eb16588571a404cc52a43067883a2b6f06dd936376
size 594676
+2 -2
View File
@@ -1,3 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:c14dfbb2db8dcb22cd332b2cf0e53ba75141adb213dae47dd9dbfd389ed482fe
size 354799
oid sha256:3aa8d972bba40d78152f9f0c762b9f5ec616d8052c45fa52b7dd1c679ed81d61
size 325245
@@ -1,3 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:ae4f79fb22600106f8cb0af4ba5586bb26ea5d57e27ef382fdc59b6549fdbd21
size 415473
oid sha256:8a5957b71822656098cbe779ca619b7e56e9b11b0f1c4eac4fc880ed62b71a2a
size 358670
+2 -2
View File
@@ -1,3 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:d6aff446a062aa08f316985d00c2183148694f786d7f22172bc50b30046c728b
size 379259
oid sha256:7623df33be9baf7647ca1c2450640df57e1cd73e8be1f8168aae16e546ad683c
size 459941
@@ -1,3 +0,0 @@
version https://git-lfs.github.com/spec/v1
oid sha256:aa12d81dc578f7e65bf4df3152b3de1736289c422f83d0b0cd32107726722357
size 172028
+4 -19
View File
@@ -22,7 +22,7 @@ Console dashboard: http://localhost:8090
|---------|------|---------|-------------|
| `server` | 8080 | default | Web UI + chat workstreams + LLM |
| `console` | 8090 | default | Cluster dashboard |
| `channel` | — | production | Channel gateway (Discord and/or Slack adapters) |
| `channel` | — | production | Channel gateway (Discord, Slack, etc.) |
| `server-1``server-10` | — | cluster | 10-node server fleet (PostgreSQL required) |
## Profiles
@@ -83,16 +83,11 @@ Auth is always enabled. `TURNSTONE_JWT_SECRET` is required.
| Variable | Default | Description |
|----------|---------|-------------|
| `TURNSTONE_DB_BACKEND` | `sqlite` | Storage backend: `sqlite` or `postgresql` |
| `TURNSTONE_DB_URL` | — | Database URL (e.g. `postgresql+psycopg://user:pass@postgres:5432/turnstone`). For SQLite, defaults to `/data/.turnstone.db` |
| `TURNSTONE_DB_LISTEN_URL` | (falls back to `TURNSTONE_DB_URL`) | Direct-to-PostgreSQL URL for the console's dedicated `LISTEN` connection. Set this when `TURNSTONE_DB_URL` points at PgBouncer in transaction pooling mode — LISTEN is session state and the transaction-pooled connection can't hold it. See [pgbouncer.md](pgbouncer.md). |
| `TURNSTONE_DB_URL` | — | Database URL (e.g. `postgresql://user:pass@db:5432/turnstone`). For SQLite, defaults to `/data/.turnstone.db` |
| `TURNSTONE_DB_POOL_SIZE` | `2` | PostgreSQL connection pool size per process (default: 2 base + 3 overflow = 5 max) |
| `POSTGRES_USER` | `turnstone` | PostgreSQL container username (used in default `TURNSTONE_DB_URL` for cluster/channel) |
| `POSTGRES_PASSWORD` | — | PostgreSQL container password (required for production and cluster profiles) |
The database stores workstream history, user accounts, and API tokens. When using JWT auth, a database backend is required for user storage.
> **Upgrading from <1.3.0a4:** Earlier versions used `DB_BACKEND` and `DATABASE_URL` in `.env`, which `compose.yaml` mapped to the `TURNSTONE_`-prefixed names internally. These short aliases have been removed. Rename `DB_BACKEND``TURNSTONE_DB_BACKEND` and `DATABASE_URL``TURNSTONE_DB_URL` in your `.env` file.
> **Large clusters:** Each turnstone process maintains a small connection pool (5 max). At hundreds of nodes this adds up — use [PgBouncer](pgbouncer.md) in transaction pooling mode between turnstone and PostgreSQL.
> **First-time setup:** After deploying with auth enabled, create an initial admin user by running `turnstone-admin create-user` inside the container:
@@ -109,16 +104,8 @@ The database stores workstream history, user accounts, and API tokens. When usin
|----------|---------|-------------|
| `TURNSTONE_DISCORD_TOKEN` | — | Discord bot token (required to enable Discord adapter) |
| `TURNSTONE_DISCORD_GUILD` | `0` | Restrict to a single Discord guild (0 = all guilds) |
| `TURNSTONE_SLACK_TOKEN` | — | Slack Bot User OAuth token `xoxb-…` (required to enable Slack adapter) |
| `TURNSTONE_SLACK_APP_TOKEN` | — | Slack App-Level token `xapp-…` (required with `TURNSTONE_SLACK_TOKEN`) |
| `TURNSTONE_SLACK_CHANNELS` | — | Comma-separated Slack channel IDs to allow (empty = all) |
| `TURNSTONE_SLACK_SLASH_COMMAND` | `/turnstone` | Slash command registered in the Slack app |
The channel service runs in the `production` profile. When
`TURNSTONE_DISCORD_TOKEN` or the Slack pair is set the gateway starts the
corresponding adapter; both can run in one process. See
[Channel Integrations](channels.md) for platform app setup and user
account linking.
The channel service runs in the `production` profile. When `TURNSTONE_DISCORD_TOKEN` is set, the Discord adapter connects to the Discord Gateway and routes messages to the server via HTTP. See [Channel Integrations](channels.md) for full setup instructions including Discord application creation and user account linking.
## Scaling
@@ -150,9 +137,7 @@ docker compose build
docker compose build --no-cache
```
All entry points are installed in a single image: `turnstone`,
`turnstone-server`, `turnstone-console`, `turnstone-channel`,
`turnstone-admin`, `turnstone-eval`, and `turnstone-bootstrap`.
All entry points are installed in a single image: `turnstone-server`, `turnstone-console`, `turnstone-channel`, `turnstone-admin`, `turnstone-eval`.
## Cleanup
+6 -12
View File
@@ -62,14 +62,14 @@ etc.) since workstream templates were merged into the skills system in v0.8.0.
- **Default skills**: All `is_default=true` skills auto-apply to new
workstreams, concatenated in alphabetical order by name. Use name prefixes
(e.g. `01-safety`, `02-style`) to control ordering.
- **Explicit selection**: `--skill <name>` CLI flag, `skill` field on
- **Explicit selection**: `--template <name>` CLI flag, `template` field on
`POST /v1/api/workstreams/new`, console creation modal dropdown, scheduled task
config, and channel adapter config. An explicit skill *replaces* defaults.
- **Variables**: Three built-in placeholders resolved at load time:
`{{model}}` (active model name), `{{ws_id}}` (workstream ID),
`{{node_id}}` (server node ID). Unrecognized placeholders are kept as-is.
- **Runtime switching**: `/skill <name>` to switch, `/skill clear` to revert
to defaults, `/skill` to show current. Persisted across resume.
- **Runtime switching**: `/template <name>` to switch, `/template clear` to revert
to defaults, `/template` to show current. Persisted across resume.
- **Model-driven loading**: The `skill` built-in tool lets the model
discover and activate skills mid-conversation. `search` action finds skills
by query (auto-approved); `load` action activates by name (requires user
@@ -91,7 +91,7 @@ etc.) since workstream templates were merged into the skills system in v0.8.0.
time. The scanner evaluates four risk axes: content risk (command execution,
data exfiltration), supply chain risk (pipe-to-shell, transitive installs),
vulnerability risk (prompt injection, insecure credentials), and declared
capability risk (from `allowed-tools` in SKILL.md). Results populate the `risk_level`
capability risk (from `allowed-tools` in SKILL.md). Results populate the `scan_status`
(safe/low/medium/high/critical) and `scan_report` (JSON breakdown) columns.
These fields are system-managed and cannot be overwritten via the admin API.
- **Discovery**: External skills can be discovered and installed from registries:
@@ -186,21 +186,15 @@ Full OpenAPI spec at `/openapi.json` and Swagger UI at `/docs`.
## Admin Console UI
Governance-related tabs within the 18-tab admin panel:
6 new tabs added to the admin panel (11 total):
- **Roles** — CRUD roles, permission checkbox grid, user role assignment modal
- **Policies** — CRUD tool policies with colored action badges (green/red/amber)
- **Prompts**Prompt-policy editor (heuristics for admin guardrails)
- **Skills** — CRUD skills with wide modal, textarea editor; Discover pill for
installing from skills.sh / GitHub; per-row scan badges (safe/low/med/high/critical)
- **Judge** — Intent validation configuration and verdict history
- **Skills**CRUD skills with wide modal, textarea editor
- **Usage** — Summary readouts + CSS bar chart, time range + group-by selectors
- **Audit** — Filterable log with relative timestamps, load-more pagination
Tabs are permission-gated: hidden if the user lacks the required permission.
See [docs/console.md](console.md) for the full tab list and
[docs/settings.md](settings.md) for the Settings tab that edits live
ConfigStore values.
## SDK
+4 -4
View File
@@ -315,7 +315,7 @@ four independent risk axes:
`Bash(*)` (unrestricted shell) is high risk. `Bash(git:*)` is low.
Read-only tools are safe.
Results are stored in `risk_level` (tier: safe/low/medium/high/critical) and
Results are stored in `scan_status` (tier: safe/low/medium/high/critical) and
`scan_report` (JSON breakdown) on the `prompt_templates` table. These fields are
system-managed and not editable via the admin API.
@@ -400,11 +400,11 @@ level, annotations, output length, redaction status).
### Session-level skill scan warning
When a skill with `risk_level` of `high` or `critical` is loaded into a
When a skill with `scan_status` of `high` or `critical` is loaded into a
session, a warning is emitted via `on_info`:
```
⚠ Skill 'my-skill' has risk level: high.
⚠ Skill 'my-skill' has scan status: high.
Review scan report in admin panel before enabling in production.
```
@@ -421,7 +421,7 @@ All three evaluation systems persist their assessments for future calibration:
|-------|--------|-------------|
| `intent_verdicts` | Intent judge (heuristic + LLM) | `func_name`, `risk_level`, `confidence`, `user_decision` |
| `output_assessments` | Output guard | `func_name`, `risk_level`, `flags`, `redacted` |
| `prompt_templates` | Skill scanner | `risk_level`, `scan_report`, `scan_version` |
| `prompt_templates` | Skill scanner | `scan_status`, `scan_report`, `scan_version` |
Run v1 with all tools requiring manual approval to build a local dataset.
In v2, calibration tooling will analyze this data to:
-117
View File
@@ -1,117 +0,0 @@
# MCP OAuth — per-user authorization for MCP servers
Turnstone supports **per-(user, MCP server) OAuth 2.1 + PKCE** delegation so each Turnstone user authorizes a remote MCP server with their own identity, rather than sharing a single bearer token across the deployment. This is the right shape for MCP servers that expose user-specific data (a personal CRM, an email inbox, a calendar) and for MCP servers that want per-user audit attribution.
Per-user OAuth is opt-in per `mcp_servers` row. Local-auth Turnstone installs with no `oauth_user` rows exercise zero new code paths — the entire feature is dark by default.
> **Note**: This is a separate authorization layer from Turnstone's own user authentication. A user who logs into Turnstone with a local username + password can still authorize a per-server OAuth MCP server. OIDC SSO and per-server OAuth are orthogonal.
---
## When to use which `auth_type`
The MCP server admin form exposes three authorization modes ("Multitenant Authorization"):
| `auth_type` | What it means | When to use |
|---|---|---|
| `none` | No headers attached. Open MCP server (or one gated by network policy only). | Internal MCP servers on a trusted network. |
| `static` | One static bearer token, configured per server, sent on every request from every user. | Service-to-service MCP servers where per-user attribution doesn't matter, or single-tenant deployments. |
| `oauth_user` *(recommended for user-data servers)* | Each user authorizes separately via OAuth 2.1 + PKCE; Turnstone stores per-user tokens encrypted at rest. | MCP servers that expose user-specific data or that want per-user audit attribution. |
Switching `auth_type` away from `oauth_user` orphans existing per-user tokens. Use the admin **bulk-revoke** affordance on the server row (Phase 9) to clear them, or let them expire naturally — they're inert without the matching `auth_type` value.
---
## Prerequisites for `auth_type=oauth_user`
1. **Encryption key**. Tokens are stored encrypted with Fernet. Set `[security] mcp_token_encryption_key` in `config.toml` (Turnstone won't start with an `oauth_user` row configured but no key installed). Rotate via `MultiFernet` — add the new key first, then later remove the old one once all rows have been re-encrypted.
2. **MCP server publishes RFC 9728 PRM and RFC 8414 AS metadata** *or* you configure the AS URL override on the server row. PKCE S256 is mandatory; Turnstone refuses to connect to authorization servers that don't advertise `code_challenge_methods_supported: ["S256"]`.
3. **OAuth client registration**. Two paths:
- **Pre-registered** (most common): you create an OAuth client at the authorization server (manually, via admin console, or via Terraform), then paste the `client_id` / `client_secret` into the Turnstone admin form.
- **Dynamic client registration** (RFC 7591): if the AS supports it and you select that mode in the admin form, Turnstone registers a client at first use and persists the `client_id` automatically.
4. **Redirect URI** registered at the authorization server: `https://your-turnstone-host/v1/api/mcp/oauth/callback`.
---
## Configuration
### Per-server fields (admin UI)
| Field | Required | Description |
|---|---|---|
| Server URL | Yes | The MCP server's `streamable-http` base URL. |
| Multitenant Authorization | Yes | `none` / `static` / `oauth_user` (recommended). |
| Authorization Server URL | No | Override for RFC 9728 PRM discovery. Set when your AS endpoint differs from the MCP server URL (e.g., corporate AS protecting a third-party MCP). When unset, Turnstone falls back to PRM discovery against the MCP server itself. |
| Client Registration | Yes (oauth_user) | `preregistered` or `dynamic`. |
| Client ID | Yes (preregistered) | OAuth 2.0 client ID. Stored unencrypted. |
| Client Secret | Optional (write-only) | OAuth 2.0 client secret (confidential client). Encrypted at rest. Written but never re-read by the API; field stays masked. |
| Scopes | No | Space-separated default scope set requested at the authorize endpoint. Per-tool step-up may union additional scopes from a server's `insufficient_scope` response. |
| Audience | No | RFC 8707 `resource=` parameter sent on every authorize and token request. Defaults to the MCP server URL when unset. Validate against the `aud` claim in returned JWT tokens. |
### Encryption key
```toml
[security]
mcp_token_encryption_key = "base64-fernet-key"
# For rotation, list the keys in priority order — first is used for new
# writes, all are tried for reads.
# mcp_token_encryption_keys = ["new-key", "old-key"]
```
Keep this in `config.toml` rather than environment variables. An in-process LLM with shell-tool access can read the server's environment via `env` / `os.environ` and exfiltrate any secret stored there; secrets in `config.toml` are only loaded into the server at startup and never re-read on a tool-driven path, so a prompt-injection attack against the agent cannot reach them.
---
## Lifecycle
1. **First tool call** for a user against an `oauth_user` MCP server: pool dispatch finds no stored token, returns `mcp_consent_required` to the agent. Dashboard renders an inline "Connect" action card.
2. **User clicks Connect**: opens `/v1/api/mcp/oauth/start?server=<name>` in a popup. Browser redirects through the AS authorize endpoint, user grants consent, AS redirects back to `/v1/api/mcp/oauth/callback`. Turnstone exchanges code → tokens via PKCE, validates audience, encrypts, persists in `mcp_user_tokens`, redirects user back to the originating URL.
3. **Subsequent tool calls** by the same user against the same server reuse the persisted token via the per-(user, server) session pool. Tokens auto-refresh via the refresh-token grant when expired; failed refresh emits `mcp_consent_required` to drive re-consent.
4. **Step-up scope**: when a tool call hits `403` with `WWW-Authenticate: error="insufficient_scope"`, Turnstone emits `mcp_insufficient_scope` with the parsed scope set; the dashboard offers a "Connect with additional scopes" affordance that opens `/v1/api/mcp/oauth/start?server=<name>&scopes=<extra>` so the union of original + new scopes flows into the AS authorize request.
5. **User revoke** (settings modal): `DELETE /v1/api/mcp/oauth/connections/{server_name}` runs the authoritative local delete + best-effort RFC 7009 upstream revoke (fire-and-forget, capped at 256 concurrent in-flight tasks).
6. **Admin bulk-revoke** (Phase 9): `POST /v1/api/admin/mcp-servers/{name}/bulk-revoke` drops every user's token for the server. Upstream RFC 7009 revoke is intentionally **not** attempted in bulk (avoids N upstream HTTP calls per admin click); tokens at the AS expire naturally. Use the per-user revoke endpoint if you need guaranteed upstream invalidation.
---
## Admin status indicators
The MCP Servers admin tab shows per-server status pills (Phase 9):
- **Consented users count** — distinct users with a non-expired token for this server. Surfaced as a `bulk-revoke (N)` button when ≥1; clicking it opens a confirmation dialog. Hidden when 0.
- **Last refresh** — timestamp + outcome (`ok` / `error:ClassName`) of the most recent manual or auto-reconnect refresh. Per node. Absent until at least one refresh has occurred (renders as "never" in the admin UI).
Additional indicators (circuit-breaker state, encryption-key mismatch) are exposed via `get_server_status` on the API but do not yet have a dedicated admin pill — operators see them today via the per-server status text + error tooltip and in audit logs. A future phase may surface these as discrete pills.
---
## Auth-type transitions
| From | To | What happens |
|---|---|---|
| `none` / `static``oauth_user` | — | New code path activates for this server. Existing static headers (if any) are no longer sent. Users must authorize on first use. |
| `oauth_user``none` / `static` | — | Existing `mcp_user_tokens` rows are **orphaned** — inert without a matching `auth_type`. Use admin bulk-revoke to drop them, or let them expire. Switching back to `oauth_user` later re-activates the orphaned rows if they haven't been deleted. |
| OAuth `client_id` or `client_secret` rotated | — | Existing tokens may stop refreshing if the AS treats them as bound to the previous client. Bulk-revoke after rotation. |
The orphan-by-default behavior is chosen so switching back to `oauth_user` is non-destructive. Bulk-revoke is the explicit cleanup path.
---
## Troubleshooting
| Symptom | Likely cause | Action |
|---|---|---|
| `mcp_consent_required` even after consenting | Token persistence failed, or refresh-token rejected by AS | Check audit log for `mcp_server.oauth.persist_failed` or `mcp_server.oauth.token_revoked`. Re-consent via settings modal. |
| `mcp_token_undecryptable_key_unknown` | Encryption key rotated without keeping the previous key in the keyring | Add the previous key back to `mcp_token_encryption_keys` until all rows have been re-encrypted, then drop. |
| `mcp_oauth_url_insecure` | MCP server URL is `http://` (not `https://`) on a non-loopback host | Use `https://`. Per-user bearers must not transit cleartext. |
| Tools fail in scheduled / Discord / Slack runs | OAuth-MCP requires browser-based consent | Users must pre-consent via the web UI. Phase 9 dashboard badge surfaces deferred consents from these runs on next login. |
| Circuit breaker open repeatedly | Transport-level errors on the MCP server (DNS, TLS, 5xx) | Check the per-server error pill; auth errors do not trip the breaker. |
See also: `docs/operations/mcp-oauth-headless.md` for the cron / channel-driven run caveat.
+1 -1
View File
@@ -146,7 +146,7 @@ with TurnstoneConsole("http://localhost:8081", token="...") as client:
### TypeScript
```typescript
import { TurnstoneConsole } from "@turnstone/sdk";
import { TurnstoneConsole } from "@anthropic/turnstone-sdk";
const client = new TurnstoneConsole({
baseUrl: "http://localhost:8081",
+13 -82
View File
@@ -39,19 +39,18 @@ are set.
| `TURNSTONE_OIDC_ROLE_CLAIM` | No | — | ID token claim containing role/group values (see [Role Mapping](#role-mapping)) |
| `TURNSTONE_OIDC_ROLE_MAP` | No | — | Mapping from claim values to Turnstone role IDs (see [Role Mapping](#role-mapping)) |
| `TURNSTONE_OIDC_PASSWORD_ENABLED` | No | `true` | Set to `false` to hide the password form and block all username/password logins (including admin). API tokens continue to work. |
| `TURNSTONE_OIDC_REDIRECT_BASE` | Yes | — | Externally-reachable origin for the OIDC redirect URI (e.g. `https://app.example.com`). Without this, OIDC will refuse to start. The previous Host-header fallback was unsafe under permissive reverse proxies. |
| `TURNSTONE_OIDC_TRUSTED_ENDPOINT_HOSTS` | No | — | Comma-separated list of additional hostnames whose endpoints the IdP discovery document is allowed to reference. See [Cross-host endpoints](#cross-host-endpoints). |
| `TURNSTONE_OIDC_REDIRECT_BASE` | No | — | Externally-reachable origin for the OIDC redirect URI (e.g. `https://app.example.com`). Recommended when running behind a reverse proxy. When unset, derived from the request Host header. |
All four required fields issuer, client ID, client secret, and
`TURNSTONE_OIDC_REDIRECT_BASE` — must be set. If any are missing OIDC
is disabled at startup (an error is logged when only `redirect_base`
is missing) and the login screen shows only the password form.
OIDC is enabled when all three required fields (issuer, client ID, client
secret) are non-empty. If any is missing, OIDC is silently disabled and
the login screen shows only the password form.
### Redirect base (required)
### Reverse Proxy / Load Balancer
`TURNSTONE_OIDC_REDIRECT_BASE` pins the redirect URI sent to the identity
provider to a known externally-visible origin. Set it to the public origin
of your Turnstone deployment:
When Turnstone runs behind a reverse proxy, the internal `Host` header may
not match the externally-reachable URL. Set `TURNSTONE_OIDC_REDIRECT_BASE`
to the public origin so the redirect URI sent to the identity provider is
correct:
```bash
TURNSTONE_OIDC_REDIRECT_BASE=https://app.example.com
@@ -61,44 +60,6 @@ The resulting callback URL will be
`https://app.example.com/v1/api/auth/oidc/callback` — register this as the
authorized redirect URI in your identity provider.
OIDC will refuse to start when this variable is unset. There is no
Host-header fallback: a permissive reverse proxy or direct backend access
would otherwise let an attacker spoof `Host` and steer the IdP redirect
to a callback origin they control.
### Cross-host endpoints
By default, every endpoint in the IdP discovery document
(`token_endpoint`, `jwks_uri`, `userinfo_endpoint`) must share the
issuer's `(scheme, host, port)`. This prevents a hostile or compromised
IdP from redirecting the token-exchange POST (which carries
`client_secret`) to an arbitrary host, and prevents JWKS fetches from
being aimed at internal services.
A few public IdPs legitimately split endpoints across hostnames. Google
is the canonical example:
| Field | Hostname |
|-------|----------|
| issuer | `accounts.google.com` |
| token_endpoint | `oauth2.googleapis.com` |
| jwks_uri | `www.googleapis.com` |
| userinfo_endpoint | `openidconnect.googleapis.com` |
Google's set is built in — operators using `https://accounts.google.com`
need no extra configuration.
For other IdPs whose discovery document references a non-issuer host,
extend the allow-list explicitly:
```bash
TURNSTONE_OIDC_TRUSTED_ENDPOINT_HOSTS=token.example.com,keys.example.com
```
The same scheme / no-userinfo / SSRF rules apply to allow-listed hosts —
this knob only relaxes the same-origin check, not the security gates.
Each entry is a hostname (no scheme, no path).
### config.toml alternative
```toml
@@ -237,19 +198,6 @@ TURNSTONE_OIDC_ROLE_MAP="admin:builtin-admin,engineering:builtin-operator,viewer
the user authenticates via OIDC, so new group memberships are picked
up on the next login.
### `assigned_by` markers
Role assignments record an `assigned_by` value that controls how the
sync logic treats them. OIDC-driven flows use two distinct markers:
- `oidc` — set by claim-driven role mapping; revoked automatically on
the next login when the corresponding claim value is no longer
present.
- `oidc-default` — applied to brand-new OIDC users who have no
claim-mapped roles, as a safety net so they still get
`builtin-viewer` access on first login. Survives subsequent logins
regardless of claim contents and is never revoked by `apply_role_mapping`.
### Built-in Roles
| Role ID | Permissions |
@@ -427,27 +375,10 @@ callback validation. Entries are automatically cleaned up after 5 minutes.
### "OIDC not configured"
All four required environment variables must be set:
`TURNSTONE_OIDC_ISSUER`, `TURNSTONE_OIDC_CLIENT_ID`,
`TURNSTONE_OIDC_CLIENT_SECRET`, and `TURNSTONE_OIDC_REDIRECT_BASE`.
Check that none are empty or whitespace-only.
### "OIDC enabled but TURNSTONE_OIDC_REDIRECT_BASE is unset"
This error is logged when the three credential variables are set but
`TURNSTONE_OIDC_REDIRECT_BASE` is missing. OIDC is disabled at startup
to prevent Host-header-derived redirect URI spoofing. Set the variable
to your service's externally-visible origin (e.g.
`https://app.example.com`) and restart the server. See
[Redirect base](#redirect-base-required) for the rationale.
### Discovery silently disables OIDC with "host does not match issuer"
The IdP discovery document points `token_endpoint`, `jwks_uri`, or
`userinfo_endpoint` at a hostname that doesn't share the issuer's
origin. If the IdP is legitimate, add the additional hostname(s) to
`TURNSTONE_OIDC_TRUSTED_ENDPOINT_HOSTS`. Google is allow-listed
automatically; see [Cross-host endpoints](#cross-host-endpoints).
All three required environment variables must be set:
`TURNSTONE_OIDC_ISSUER`, `TURNSTONE_OIDC_CLIENT_ID`, and
`TURNSTONE_OIDC_CLIENT_SECRET`. Check that none are empty or
whitespace-only.
### "Login session expired"
-29
View File
@@ -1,29 +0,0 @@
# MCP OAuth in headless / scheduled / channel-driven runs
**Constraint**: OAuth-MCP servers (`auth_type=oauth_user`) require browser-based user consent. Users must pre-consent via the web UI before any run that cannot drive a browser redirect.
**Affected surfaces**:
- Scheduled workstreams (`turnstone-console` task scheduler).
- Discord adapter runs.
- Slack adapter runs.
- Any future channel adapter without an interactive browser session.
**What happens when consent is missing**:
A tool call against an `oauth_user` server returns a structured `mcp_consent_required` error to the agent. The agent surfaces the deferred work in its output. Turnstone persists a record to `mcp_pending_consent` so the dashboard badge surfaces the deferred consent need to the user on next login.
**Recovery**:
The user opens the dashboard, sees the gear-icon badge counting pending consents, opens the settings modal, clicks Connect for each affected server, and completes the OAuth dance. The pending-consent record is cleared by the OAuth callback handler on success. Subsequent scheduled / channel runs use the freshly-stored token.
**Pre-consent recipe**:
Before scheduling a workstream that depends on an `oauth_user` MCP server, the user should:
1. Open the dashboard.
2. Open the settings modal (gear icon).
3. Click Connect on each MCP server the schedule will use.
4. Confirm consent in the popup.
This stores tokens that the scheduled run will reuse. Refresh-token rotation is handled transparently on the run side; only the first consent requires browser interaction.
+14 -40
View File
@@ -40,20 +40,18 @@ Add PgBouncer between turnstone services and PostgreSQL:
```yaml
services:
pgbouncer:
image: edoburu/pgbouncer:latest
image: bitnami/pgbouncer:latest
environment:
DB_HOST: postgres
DB_PORT: "5432"
DB_NAME: ${POSTGRES_DB:-turnstone}
DB_USER: ${POSTGRES_USER:-turnstone}
DB_PASSWORD: ${POSTGRES_PASSWORD:?}
LISTEN_PORT: "6432"
AUTH_TYPE: ${POSTGRES_AUTH_TYPE:-scram-sha-256}
POOL_MODE: transaction
DEFAULT_POOL_SIZE: "40"
MAX_CLIENT_CONN: "5000"
MAX_DB_CONNECTIONS: "80"
SERVER_IDLE_TIMEOUT: "300"
POSTGRESQL_HOST: postgres
POSTGRESQL_PORT: "5432"
POSTGRESQL_DATABASE: turnstone
POSTGRESQL_USERNAME: ${POSTGRES_USER:-turnstone}
POSTGRESQL_PASSWORD: ${POSTGRES_PASSWORD:?}
PGBOUNCER_POOL_MODE: transaction
PGBOUNCER_DEFAULT_POOL_SIZE: "40"
PGBOUNCER_MAX_CLIENT_CONN: "5000"
PGBOUNCER_MAX_DB_CONNECTIONS: "80"
PGBOUNCER_SERVER_IDLE_TIMEOUT: "300"
ports:
- "6432:6432"
networks:
@@ -69,7 +67,7 @@ services:
```
Then point turnstone services at PgBouncer instead of PostgreSQL
directly by changing `TURNSTONE_DB_URL`:
directly by changing the `DATABASE_URL` (or `TURNSTONE_DB_URL`):
```bash
# Before (direct)
@@ -84,7 +82,7 @@ TURNSTONE_DB_URL=postgresql://turnstone:secret@pgbouncer:6432/turnstone
## Helm / Kubernetes
Add a PgBouncer deployment or use a Helm chart like
[edoburu/pgbouncer](https://github.com/edoburu/docker-pgbouncer/tree/master/examples/kubernetes).
[bitnami/pgbouncer](https://github.com/bitnami/charts/tree/main/bitnami/pgbouncer).
In `values.yaml`, point the database at PgBouncer:
@@ -108,7 +106,7 @@ pgbouncer:
maxClientConn: 5000
maxDbConnections: 80
```
:
---
## Configuration reference
@@ -199,28 +197,4 @@ does not support prepared statements. Turnstone's SQLAlchemy layer does
not use server-side prepared statements by default, so this is not an
issue.
**LISTEN / NOTIFY not supported in transaction mode** — PgBouncer's
transaction pooling assigns a real server connection only for the
duration of each transaction, then returns it to the pool. PostgreSQL
`LISTEN` is session state — a transaction-pooled client can't hold the
multi-statement session a long-lived `LISTEN` needs. The console's
`NotifyDispatcher` (reactive node discovery via the `services` channel)
therefore opens a **dedicated, direct-to-Postgres** connection that
bypasses PgBouncer.
Configure via `config.toml` `[database] listen_url` (preferred —
co-located with the main `url`) or the `TURNSTONE_DB_LISTEN_URL` env var
(config.toml wins when both are set). Defaults to the main DB URL when
unset.
| Setting | Behaviour |
|---|---|
| unset | Listener uses `TURNSTONE_DB_URL` as-is. Fine when PgBouncer is in **session** mode, or when there's no pooler in front of Postgres. With transaction-mode PgBouncer the listener's `LISTEN` will fail and the dispatcher retries with exponential backoff (1 s → 30 s cap) without ever succeeding. Reactive NOTIFY-driven node discovery is silently lost; the cluster collector's 60 s `_discovery_loop` is the only remaining backstop. |
| set to direct-to-PG URL (e.g. `postgresql://…/turnstone`) | Listener bypasses PgBouncer for its one dedicated connection. Reactive discovery latency drops from up-to-60 s to ~500 ms. The rest of the storage layer continues to go through PgBouncer in transaction mode. |
Set this whenever PgBouncer is in transaction mode (the recommended
setting per this doc). The override only adds one long-lived PG
connection per console process — sized into the cluster's
`max_connections` budget alongside the pool.
See also: [Docker deployment](docker.md) · [Security](security.md)
+15 -24
View File
@@ -1,24 +1,17 @@
# Release Process
Turnstone ships several parallel release tracks from a single PyPI package.
Turnstone uses two parallel release tracks published from a single PyPI package.
## Release Tracks
| Track | Versions | Branch | Docker tags | PyPI install |
|-------|----------|--------|-------------|--------------|
| **Legacy 1.0** | `1.0.x` | `stable/1.0` | `:1.0.x`, `:1.0` | `pip install 'turnstone==1.0.*'` |
| **Stable 1.3** | `1.3.x` | `stable/1.3` | `:1.3.x`, `:1.3` | `pip install 'turnstone==1.3.*'` |
| **Stable 1.4** | `1.4.x` | `stable/1.4` | `:1.4.x`, `:1.4`, `:stable`, `:latest` | `pip install turnstone` |
| **Experimental** | `1.5.0aN` | `main` | `:1.5.0aN`, `:experimental` | `pip install turnstone --pre` |
| **Stable** | `1.1.0`, `1.1.1` | `stable/1.1` | `:1.1.0`, `:1.1`, `:stable`, `:latest` | `pip install turnstone` |
| **Experimental** | `1.2.0a1`, `1.2.0a2` | `main` | `:1.2.0a1`, `:experimental` | `pip install turnstone --pre` |
- **Stable** tracks receive bugfixes only. The most-recent stable minor
owns the `:stable` / `:latest` Docker tags and the default PyPI
install.
- **Experimental** (always on `main`) receives new features. May be
rough around the edges.
- When experimental matures, it is promoted to a new stable minor via
a `stable/X.Y` branch; older stable branches continue to receive
security fixes until explicitly retired.
- **Stable** receives bugfixes only. Production-grade.
- **Experimental** receives new features. May be rough around the edges.
- When experimental matures, it is promoted to stable. The previous stable branch stops receiving patches.
## Version Scheme
@@ -33,17 +26,17 @@ Turnstone ships several parallel release tracks from a single PyPI package.
## Releasing an Experimental Version (from main)
```bash
scripts/release.sh 1.5.0a2 --push
scripts/release.sh 1.1.0a2 --push
```
This bumps `pyproject.toml` + `turnstone/__init__.py`, regenerates `uv.lock`, commits, tags `v1.5.0a2`, and pushes. CI runs, then publish + Docker workflows fire automatically.
This bumps `pyproject.toml` + `turnstone/__init__.py`, regenerates `uv.lock`, commits, tags `v1.1.0a2`, and pushes. CI runs, then publish + Docker workflows fire automatically.
## Releasing a Stable Patch (from stable/X.Y)
```bash
git checkout stable/1.4
git checkout stable/1.0
git cherry-pick <commit-hash> # bugfix from main
scripts/release.sh 1.4.1 --push
scripts/release.sh 1.0.2 --push
```
## Promoting Experimental to Stable
@@ -52,19 +45,17 @@ When `main` is ready for a stable release:
```bash
# 1. Tag the stable release on main
scripts/release.sh 1.5.0 --push
scripts/release.sh 1.1.0 --push
# 2. Create the stable maintenance branch from that tag
git branch stable/1.5 v1.5.0
git push origin stable/1.5
git branch stable/1.1 v1.1.0
git push origin stable/1.1
# 3. Start the next experimental cycle on main
scripts/release.sh 1.6.0a1 --push
scripts/release.sh 1.2.0a1 --push
```
The previous stable branch (`stable/1.4`) continues to receive
security-only patches; older tracks (`stable/1.0`, `stable/1.3`) are
retired when they fall out of support.
The previous `stable/1.0` branch stops receiving patches at this point.
## CI/CD Pipeline
+2 -39
View File
@@ -69,12 +69,8 @@ Both `TurnstoneServer` (sync) and `AsyncTurnstoneServer` (async) expose:
|----------|--------|---------|
| **Workstreams** | `list_workstreams()` | `ListWorkstreamsResponse` |
| | `dashboard()` | `DashboardResponse` |
| | `create_workstream(*, name, model, auto_approve, skill, initial_message, attachments)` | `CreateWorkstreamResponse` |
| | `create_workstream(*, name, model, auto_approve, skill)` | `CreateWorkstreamResponse` |
| | `close_workstream(ws_id)` | `StatusResponse` |
| **Attachments** | `upload_attachment(ws_id, filename, data, *, mime_type=...)` | `UploadAttachmentResponse` |
| | `list_attachments(ws_id)` | `ListAttachmentsResponse` |
| | `get_attachment_content(ws_id, attachment_id)` | `bytes` |
| | `delete_attachment(ws_id, attachment_id)` | `StatusResponse` |
| **Chat** | `send(message, ws_id)` | `SendResponse` |
| | `approve(*, ws_id, approved, feedback, always)` | `StatusResponse` |
| | `plan_feedback(*, ws_id, feedback)` | `StatusResponse` |
@@ -138,9 +134,6 @@ SSE events are deserialized into typed dataclasses. Use `event.type` to discrimi
| `error` | `ErrorEvent` | `message` |
| `info` | `InfoEvent` | `message` |
| `stream_end` | `StreamEndEvent` | — |
| `state_change` | `StateChangeEvent` | `state``running`/`thinking`/`attention`/`idle`/`error` |
| `in_progress_snapshot` | `InProgressSnapshotEvent` | `content`, `reasoning` (one-shot mid-stream refresh resume) |
| `approval_resolved` | `ApprovalResolvedEvent` | `approved`, `feedback` |
| `cancelled` | `CancelledEvent` | — |
**Global events** (from `stream_global_events()`):
@@ -178,36 +171,6 @@ result.ok # True if no errors and not timed out
result.timed_out # True if timeout expired
```
### Attachments
Upload files to a workstream and attach them to the next user turn:
```python
# Upload separately, then send a message — attachments auto-attach
with open("screenshot.png", "rb") as f:
att = client.upload_attachment(ws.ws_id, "screenshot.png",
f.read(),
mime_type="image/png")
client.send("What's wrong in this screenshot?", ws.ws_id)
# Or attach at workstream-creation time (multipart upload)
from turnstone.sdk import AttachmentUpload
with open("notes.txt", "rb") as f:
ws = client.create_workstream(
name="triage",
initial_message="Summarize the notes",
attachments=[AttachmentUpload(data=f.read(),
filename="notes.txt",
mime_type="text/plain")],
)
```
Limits: images ≤ 4 MiB (png/jpeg/gif/webp), text ≤ 512 KiB (UTF-8),
10 pending per (workstream, user). The SDK auto-generates `ws_id` on the
client so cluster-routed callers bind attachments to the owning node
before the request lands.
### Error Handling
Non-2xx responses raise `TurnstoneAPIError`:
@@ -321,7 +284,7 @@ turnstone/sdk/ Python SDK (sub-package)
_base.py Shared httpx async client, auth, error handling
_sync.py Background event loop for sync wrappers
_types.py TurnResult + TurnstoneAPIError
events.py 38 SSE event dataclasses with type registry
events.py 27 SSE event dataclasses with type registry
server.py AsyncTurnstoneServer + TurnstoneServer
console.py AsyncTurnstoneConsole + TurnstoneConsole
+8 -11
View File
@@ -1,10 +1,8 @@
# Security and Authentication
Turnstone uses a layered authentication system with two token types
(database-backed API tokens + HMAC-SHA256 JWTs), hierarchical scopes,
and a split architecture where the console manages credentials while
individual server nodes validate JWTs locally. Inter-service traffic
uses short-lived service JWTs minted by `ServiceTokenManager`.
Turnstone uses a layered authentication system with three token types,
hierarchical scopes, and a split architecture where the console manages
credentials while individual server nodes validate JWTs locally.
---
@@ -39,7 +37,7 @@ Claims:
|-------|-------------|
| `sub` | User ID |
| `scopes` | Comma-separated scope list (`read,write,approve`) |
| `src` | Token source (`password`, `database`, `oidc`, or a service origin like `console`, `cli`, or `channel`) |
| `src` | Token source (`password`, `api_token`, `config`, `oidc`) |
| `iss` | Issuer — always `turnstone` |
| `aud` | Audience — `turnstone-server` or `turnstone-console` |
| `iat` | Issued-at timestamp |
@@ -67,11 +65,10 @@ Scopes are hierarchical — higher scopes imply all lower ones.
| Method | Path pattern | Required scope |
|--------|-------------|----------------|
| GET | Any protected path | `read` |
| POST | `/api/plan`, `/api/command` | `write` |
| POST | `/api/workstreams/new`, `/api/cluster/workstreams/new` | `write` |
| POST | `/api/workstreams/{ws_id}/{send,cancel,close,delete,open,refresh-title,title,attachments}` | `write` |
| DELETE | `/api/workstreams/{ws_id}/send` (dequeue), `/api/workstreams/{ws_id}/attachments/{attachment_id}` | `write` |
| POST | `/api/workstreams/{ws_id}/approve` | `approve` |
| POST | `/api/send`, `/api/plan`, `/api/command` | `write` |
| POST | `/api/workstreams/new`, `/api/workstreams/close` | `write` |
| POST | `/api/cluster/workstreams/new` | `write` |
| POST | `/api/approve` | `approve` |
| Any | `/api/admin/*` | `approve` |
Public paths bypass authentication entirely: `/`, `/health`, `/metrics`,
+4 -60
View File
@@ -36,62 +36,6 @@ users to the admin Settings API.
---
## Per-Model Sampling Overrides
The global `model.temperature`, `model.max_tokens`, and `model.reasoning_effort`
settings serve as cluster-wide defaults. Individual models can override these
via per-model settings in the `model_definitions` table (admin Models tab).
Resolution order for sampling parameters:
| Priority | Source |
|----------|--------|
| 1 (highest) | Per-model override (set in Models tab) |
| 2 | Global default (set in Settings tab) |
| 3 | Registry default (code) |
When a per-model override is `NULL` (empty in the UI), the global default is
used. Switching models via `/model <alias>` re-resolves sampling parameters
from the new model's overrides or global defaults.
**Removed settings:** `model.name` and `model.context_window` have been removed
from ConfigStore. Model names and context windows are now configured per-model
in the Models tab. A startup warning is logged if these keys appear in
`config.toml`.
### Reasoning persistence (per-model)
Two boolean flags on `model_definitions` (migration 052) control how
reasoning text round-trips per model:
| Flag | Default | Effect |
|------|---------|--------|
| `surface_persisted_reasoning` | `True` | Surface stored reasoning text on `/history` payloads so a page reload re-renders the reasoning bubble. **Storage of reasoning bytes is independent of this flag** — they ride in `provider_data` regardless. |
| `replay_reasoning_to_model` | `False` | Send stored reasoning blocks back to the provider on subsequent turns. Capability-gated: only takes effect when the model's `ModelCapabilities.supports_reasoning_replay` is also `True`. Set on canonical OpenAI gpt-5*/o-series and Anthropic Claude entries; unknown / local-server models default to `False` so an operator who flips the flag on a model whose API doesn't understand reasoning replay silently no-ops rather than 400-ing. |
Edit both via the admin Models tab. See the architecture doc for the
provider-side mechanics (Anthropic `thinking`, OpenAI Responses
`reasoning` + `include=["reasoning.encrypted_content"]`, synthetic
`reasoning_text` for Chat Completions / vLLM / llama.cpp / Gemini-compat).
### Plan / task agent overrides
`plan_agent` and `task_agent` sub-sessions resolve independently from the
conversation model so operators can pick a cheaper/faster model for
autonomous loops:
| Setting | Purpose |
|---------|---------|
| `model.plan_alias` | Alias used for `plan_agent` sub-sessions. Falls back to `[model].plan_model` in config.toml, then `[model].agent_model`, then the session's active model. |
| `model.task_alias` | Alias used for `task_agent` sub-sessions. Same fallback chain as `plan_alias`. |
| `model.plan_effort` | Reasoning effort for `plan_agent` (`none` / `minimal` / `low` / `medium` / `high` / `xhigh` / `max`). Defaults to `high`. |
| `model.task_effort` | Reasoning effort for `task_agent`. Empty string means "inherit from the session". |
All four are live-editable from the Settings tab and take effect on the
next sub-agent invocation — no restart required.
---
## Bootstrap vs ConfigStore
**Bootstrap settings** are required before storage is available (database
@@ -105,17 +49,17 @@ connection, Redis, auth secrets, server bind address). These stay in
| Auth | `[auth]` | config.toml / env |
| Console bind | `[console]` | config.toml / env |
**ConfigStore settings** are loaded from the database after storage
initialization:
**ConfigStore settings** (51 settings) are loaded from the database after
storage initialization:
| Section | Settings |
|---------|----------|
| `model` | default_alias, temperature, max_tokens, reasoning_effort, plan_alias, task_alias, plan_effort, task_effort |
| `model` | name, temperature, max_tokens, reasoning_effort, context_window |
| `session` | instructions, retention_days, compact_max_tokens, auto_compact_pct |
| `tools` | timeout, truncation, agent_max_turns, skip_permissions, search, search_threshold, search_max_results |
| `server` | workstream_idle_timeout, max_workstreams |
| `cluster` | node_fan_out_limit, mcp_max_servers |
| `mcp` | config_path, registry_url |
| `mcp` | config_path, refresh_interval, registry_url |
| `ratelimit` | enabled, requests_per_second, burst, trusted_proxies |
| `health` | backend_probe_interval, backend_probe_timeout, circuit_breaker_threshold, circuit_breaker_cooldown |
| `judge` | enabled, model, provider, base_url, api_key, confidence_threshold, max_context_ratio, timeout, read_only_tools, output_guard, redact_secrets, cancel_on_approval |
@@ -1,233 +0,0 @@
---
name: import-conversation-history
description: Use this skill when the user wants to import or migrate conversation history from another LLM chat or coding tool (e.g. ChatGPT, Claude.ai, Cursor, Copilot Chat, Aider, Gemini, a custom JSON export) into Turnstone. The skill teaches Turnstone's destination contracts — workstream identity, the OpenAI-shaped message rows, tool-call/result pairing, provider-fidelity blobs, attachments, and archive-vs-resumable choice — so the agent can map any source format onto them. Trigger phrases: "import my chats", "migrate this transcript into Turnstone", "bring my Claude.ai history over", "load this export as a workstream".
version: 1.0.0
---
# Importing Conversation History into Turnstone
## Overview
Source formats vary; the destination does not. Your job is to translate whatever the user hands you (JSON dump, ZIP export, scraped HTML, screenshot OCR, raw transcript) into Turnstone's internal shape: **one workstream row** plus an ordered sequence of **conversation rows** in OpenAI message format. This skill documents the destination so you can write a correct mapper for any source.
Two questions to settle with the user before writing anything:
1. **Archive or resumable?** An archive ("saved" workstream — `state="closed"`) is read-only history. A resumable workstream (`state="idle"`) lets the user continue the conversation; this only works cleanly when the source LLM matches a Turnstone-supported provider/model and tool definitions still resolve.
2. **One workstream per source thread, or merge?** Default to one-to-one unless the user explicitly asks to merge.
Default to **archive** when in doubt — resuming a foreign transcript with mismatched tool schemas or stale provider signatures will fail at the next turn.
## Turnstone Data Model (the destination)
Two tables carry the conversation:
### `workstreams` (one row per imported thread)
| Column | Required | Notes |
|---|---|---|
| `ws_id` | yes | 32-char lowercase hex. Auto-generate with `secrets.token_hex(16)` if you don't already have one. **First 4 hex chars are the routing bucket** — see "Identity & Routing" below. |
| `name` | yes | Short title. Pull from source thread title; fall back to first ~60 chars of first user message. |
| `state` | yes | `"closed"` for archive, `"idle"` for resumable. Never set `"running"` on import. |
| `kind` | yes | `"interactive"` for normal threads. Do NOT use `"coordinator"` for imports — that's reserved for cluster-spawned coordinator workstreams. |
| `parent_ws_id` | no | Leave NULL. Only set if you're importing a coordinator-spawned subtree and re-parenting it; rare. |
| `user_id` | yes | Owner. Must exist in `users`; importer must know which Turnstone user owns the imported history. |
| `node_id` | yes (multi-node) | Denormalized cache of the node that owns this `ws_id`'s bucket. Single-node deployments can leave it NULL or set it to the only node. |
| `alias` | no | Human-typeable short name. Optional; must be unique cluster-wide if set. |
| `title` | no | Auto-titled later by the LLM; safe to leave NULL on import. |
| `skill_id`, `skill_version` | yes | Default `""` and `0` unless the source thread was scoped to a Turnstone skill. |
| `created`, `updated` | yes | ISO8601 strings. Use the source's first/last message timestamps when available. |
### `conversations` (many rows per thread, ordered by `id`/`timestamp`)
| Column | Notes |
|---|---|
| `ws_id` | The workstream this row belongs to. |
| `timestamp` | ISO8601 string. Preserve source timestamps; fall back to monotonically increasing values if unknown. **Order is canonical via `id` (autoincrement), not `timestamp`** — but always insert in conversational order so both agree. |
| `role` | One of `system`, `user`, `assistant`, `tool`, `developer`. See role mapping below. |
| `content` | Text. May be NULL for assistant rows that are *only* tool calls. |
| `tool_name` | Set on `role="tool"` rows (the tool whose result this is). NULL otherwise. |
| `tool_call_id` | Set on `role="tool"` rows (matches the assistant row's `tool_calls[].id`). NULL otherwise. |
| `tool_calls` | JSON-encoded list, on `role="assistant"` rows that issued tool calls. OpenAI shape — see "Tool Calls" below. |
| `provider_data` | JSON blob preserving provider-native content blocks (Anthropic `signature`, Gemini `thought_signature`, etc.). Optional; only matters for **resumable** imports against the same provider. Skip for archives. |
The internal format is **OpenAI-shaped**, even when the source was Anthropic or Gemini. Providers translate at their own API boundary; storage stays uniform.
## Identity & Routing (`ws_id`)
- `ws_id` is **32-char lowercase hex** (i.e. `secrets.token_hex(16)`).
- The **routing bucket** is `int(ws_id[:4], 16)` — the first 4 hex chars place this workstream on a specific node via the consistent hash ring.
- For multi-node imports: either insert through the console's routing proxy (which forwards to the owning node), or generate `ws_id`s and write directly to each node's database in batches grouped by bucket.
- For single-node imports: bucket math is irrelevant; any `ws_id` works.
- **Do not reuse the source platform's IDs as `ws_id`** unless they happen to be 32-char hex. Generate fresh; if you need the old ID for traceability, store it in `workstream_config` under a key like `import.source_id`.
## Recommended Import Path
Three options, in order of preference:
### 1. Storage protocol (recommended for full history)
Use `turnstone.core.storage.Storage.save_messages_bulk(rows)`. This is the canonical bulk-insert primitive and bypasses the LLM round-trip entirely.
```python
from turnstone.core.storage import get_storage # construct via the same path the server uses
storage = get_storage(...) # see turnstone.core.storage.__init__ for the project's wiring
storage.create_workstream( # or whatever the project's exposed creator is — check turnstone/core/storage/_protocol.py
ws_id=ws_id,
user_id=user_id,
name=name,
state="closed",
kind="interactive",
...
)
storage.save_messages_bulk([
{"ws_id": ws_id, "role": "user", "content": "Hello"},
{"ws_id": ws_id, "role": "assistant", "content": "Hi! What can I help with?"},
{"ws_id": ws_id, "role": "assistant", "content": None,
"tool_calls": json.dumps([{"id": "call_1", "type": "function",
"function": {"name": "search", "arguments": "{\"q\":\"x\"}"}}])},
{"ws_id": ws_id, "role": "tool", "tool_name": "search", "tool_call_id": "call_1",
"content": "result text"},
# ...
])
```
`save_messages_bulk` handles `timestamp` and the workstream's `updated` column internally, so you don't need to compute them per row. **Verify the exact creator signature** by reading `turnstone/core/storage/_protocol.py` — table layout has shifted across migrations and the Storage protocol is the source of truth.
### 2. SDK `create_workstream(resume_ws=...)` (when the source is already a Turnstone workstream)
Only useful for *Turnstone → Turnstone* re-parenting. Not relevant for foreign sources.
### 3. SDK `create_workstream(initial_message=...)` + `send()` per turn (last resort)
Only fits archives where the source had **no tool calls** and you don't care about preserving assistant turns verbatim. Each `send()` triggers a real LLM round-trip, which is expensive and rewrites assistant content. Don't use this for full history.
## Role Mapping
Common source-role conventions and how they map to Turnstone:
| Source role | Turnstone `role` | Notes |
|---|---|---|
| `user`, `human` | `user` | Direct map. |
| `assistant`, `ai`, `model`, `bot` | `assistant` | Direct map. |
| `system` | `system` | Preserve only if it's content the user wrote (custom instructions). Drop boilerplate provider preambles — Turnstone composes its own system message. |
| `developer` (OpenAI o-series) | `developer` | Preserve. |
| `tool`, `function`, `tool_result` | `tool` | Must carry `tool_name` and `tool_call_id` matching the prior assistant row's `tool_calls[].id`. |
| `tool_use` (Anthropic) | `assistant` with `tool_calls` | Anthropic emits tool calls *inside* an assistant message; flatten to OpenAI shape. |
| `human_feedback`, `revision` | `user` | Treat as a follow-up user turn. |
## Tool Calls (the most error-prone part)
Turnstone stores tool calls in OpenAI's nested-function shape on the assistant row, and matches them with `role="tool"` result rows by `tool_call_id`.
### Assistant row with tool calls
```json
{
"role": "assistant",
"content": null,
"tool_calls": [
{
"id": "call_abc123",
"type": "function",
"function": {
"name": "search_web",
"arguments": "{\"query\":\"turnstone import\"}"
}
}
]
}
```
`tool_calls[].function.arguments` is **a JSON-encoded string**, not an object. Source formats commonly get this wrong — Anthropic stores arguments as a parsed object, Gemini as a struct. Always re-serialize to a string.
### Tool result row
```json
{
"role": "tool",
"tool_name": "search_web",
"tool_call_id": "call_abc123",
"content": "..."
}
```
Pairing rules:
- Every assistant `tool_calls[].id` MUST be followed by exactly one `role="tool"` row with the matching `tool_call_id`, before the next user/assistant turn.
- If the source dropped the tool result (cut-off transcript), insert a synthetic `role="tool"` row with `content="[tool result missing in source]"` to keep the chain valid. An assistant row with an unanswered `tool_calls[].id` will break replay and any LLM round-trip.
- Multi-tool assistant turns: one `role="tool"` row per call, in any order, all before the next non-tool row.
### Tool ID generation
If the source used opaque tool IDs that aren't unique within a thread (some platforms reuse them), regenerate with a stable scheme like `f"call_{i}"` where `i` is a per-thread counter. Update both the assistant and tool rows together.
## Provider Fidelity (`provider_data`)
Skip this entirely for **archive** imports.
For **resumable** imports against the same provider, populate `provider_data` to preserve provider-specific tool-call metadata that the next API round-trip will require:
- **Anthropic**: `signature` field on thinking blocks; required for round-tripping extended-thinking responses.
- **Gemini**: `thought_signature` on tool calls; required for fidelity.
- **OpenAI**: typically nothing to preserve.
The runtime-side dict key is `_provider_content` (a list of provider-native blocks); the persisted column is `provider_data` (the same list, JSON-encoded). If you don't have provider-native blocks from the source — and you usually won't, because a foreign export won't include them — leave `provider_data` NULL. The first new turn will succeed without it, but the previous assistant turn's reasoning won't replay back to the model.
## Attachments
If the source thread had image or file attachments:
- **Size limits**: images ≤ 4 MiB, text documents ≤ 512 KiB. Reject or downsample anything bigger.
- **Allowed types**: server validates magic bytes for images and UTF-8-decodes for text. Binary blobs that aren't images won't pass.
- **Lifecycle**: pending → reserved → consumed. For imports, the cleanest path is to upload as pending and immediately consume by attaching to the relevant `conversations.id`.
Two import paths:
1. **Bulk-insert + post-attach**: insert messages first, get back the assistant/user `conversations.id`, then write `workstream_attachments` rows linking the file to `message_id`.
2. **SDK multipart create**: `create_workstream(attachments=[...], initial_message=...)` for the *first* turn only — the server reserves and consumes them onto that turn. Doesn't help for mid-thread attachments.
For full-history imports with multiple attachments at different turns, path (1) is the only option.
## Validation Checklist
Before declaring success, verify:
- [ ] `ws_id` is 32-char lowercase hex.
- [ ] `workstreams` row exists with the right `user_id`, `state`, `kind`.
- [ ] Conversation rows are inserted **in order** (autoincrement `id` will reflect insert order).
- [ ] Every assistant `tool_calls[].id` has a matching `role="tool"` row with the same `tool_call_id`.
- [ ] `tool_calls[].function.arguments` is a JSON-encoded **string**, not a parsed object.
- [ ] First message is typically `role="user"` (not `system`) — Turnstone composes its own system prompt at runtime.
- [ ] No empty assistant rows (`content=NULL` AND `tool_calls=NULL` is invalid).
- [ ] If multi-node: the `ws_id`'s bucket maps to a node that exists; `workstreams.node_id` matches.
- [ ] Round-trip test: run `Storage.load_messages(ws_id)` and confirm the reconstructed list matches what you inserted (modulo timestamps).
## Anti-patterns
- **Don't import the source provider's system prompt verbatim.** Provider boilerplate ("You are Claude...", "You are ChatGPT...") will conflict with Turnstone's composed system message and confuse the model on resume. Drop it; preserve only user-authored custom instructions.
- **Don't preserve foreign tool definitions as Turnstone tools.** If the source had custom tools that don't exist in Turnstone, the assistant rows that called them are still valid history (archive), but the workstream is **not resumable** — mark `state="closed"`.
- **Don't fabricate `tool_call_id`s without re-pairing.** Mismatched ids silently break the replay chain on the next turn.
- **Don't skip the `tool_name` field on `role="tool"` rows.** Some load paths use it for display and audit; NULL there will render as "unknown tool".
- **Don't write through the LLM (`send()` per turn) for full history.** It's expensive, rewrites assistant turns, and rate-limits will bite long imports.
## Quick Reference
| Task | Path |
|---|---|
| Generate ws_id | `secrets.token_hex(16)` |
| Bulk insert messages | `Storage.save_messages_bulk(rows)` |
| Archive (read-only) | `state="closed"`, skip `provider_data` |
| Resumable | `state="idle"`, populate `provider_data` if same provider |
| Tool call id | OpenAI shape: `{"id": ..., "type": "function", "function": {"name": ..., "arguments": "<json string>"}}` |
| Tool result row | `role="tool"`, `tool_name`, `tool_call_id`, `content` |
| Source role → Turnstone role | See "Role Mapping" table |
| Per-thread metadata | Store source IDs in `workstream_config` under `import.*` keys |
## Files to read before writing the importer
- `turnstone/core/storage/_schema.py` — authoritative table definitions.
- `turnstone/core/storage/_protocol.py``save_message`, `save_messages_bulk`, `load_messages` signatures.
- `turnstone/core/session.py` (around the message-save section) — how the runtime constructs in-memory message dicts; mirror this shape on import to round-trip cleanly.
- `turnstone/api/server_schemas.py` — Pydantic shapes for the SDK paths if you go through HTTP.
+22 -16
View File
@@ -169,8 +169,8 @@ Every tool defines a `primary_key`. The mapping is:
| `man` | `page` |
| `web_fetch` | `url` |
| `web_search` | `query` |
| `task_agent` | `prompt` |
| `plan_agent` | `goal` |
| `task` | `prompt` |
| `plan` | `prompt` |
| `memory` | `name` |
| `recall` | `query` |
| `notify` | `message` |
@@ -357,10 +357,7 @@ Search the web using a text query.
## Agent
Tool names use the `_agent` suffix — bare `plan` / `task` collide with
chat-template channel names on some local models.
### task_agent
### task
Delegate a general-purpose task to an autonomous sub-agent.
@@ -374,7 +371,7 @@ Delegate a general-purpose task to an autonomous sub-agent.
---
### plan_agent
### plan
Plan before implementing -- an autonomous agent explores the codebase and writes a structured plan.
@@ -546,11 +543,11 @@ pre-configure skills at workstream creation.
- `load` — Activate a skill by name. Calls `set_skill()` which handles content
rendering with `{{model}}`/`{{ws_id}}`/`{{node_id}}` variables, system message
reinitialization, and config persistence. Returns the skill name, description,
and security risk level. Warns on high/critical risk level.
and security scan tier. Warns on high/critical scan status.
- `search` — Find available skills by query. Uses BM25 relevance ranking over
name, description, tags, and category (same `BM25Index` used by memory
relevance and tool search). Returns up to 10 results with name, description,
category, risk level, and activation type.
category, scan status, and activation type.
- **Auto-approve**: `load` requires approval (changes session behavior); `search`
is auto-approved (read-only).
@@ -571,8 +568,8 @@ pre-configure skills at workstream creation.
| `man` | Info | Yes | Yes | Yes | `page` |
| `web_fetch` | Info | No | Yes | Yes | `url` |
| `web_search` | Info | No | Yes | Yes | `query` |
| `task_agent` | Agent | No | No | No | `prompt` |
| `plan_agent` | Agent | No | No | No | `goal` |
| `task` | Agent | No | No | No | `prompt` |
| `plan` | Agent | No | No | No | `prompt` |
| `memory` | Memory | Yes | No | No | `name` |
| `recall` | Memory | Yes | No | No | `query` |
| `notify` | Notify | Yes | Yes | Yes | `message` |
@@ -758,18 +755,22 @@ MCP tools (3):
### Dynamic tool refresh
MCP tool lists stay up-to-date without restart through two mechanisms:
MCP tool lists stay up-to-date without restart through three mechanisms:
1. **Push notifications** -- MCP servers that declare `tools.listChanged: true` in
their capabilities send `notifications/tools/list_changed` when their tool list
changes. `MCPClientManager` registers a `message_handler` on each `ClientSession`
that triggers an immediate refresh for that server.
2. **Manual** -- `/mcp refresh` re-fetches tools from all servers immediately.
2. **Periodic timer** -- Servers that do *not* support push notifications are polled
on a configurable interval (default 4 hours). The timer is staggered using a
launch-time seed (`monotonic_ns ^ pid`) so cluster nodes don't all hit MCP
servers simultaneously. Configure via `[mcp] refresh_interval` in `config.toml`
or `--mcp-refresh-interval SECONDS` on the CLI. Set to `0` to disable.
3. **Manual** -- `/mcp refresh` re-fetches tools from all servers immediately.
`/mcp refresh <server>` targets a single server. If a server has disconnected,
manual refresh attempts reconnection. The console admin panel exposes the
same controls (refresh / reconnect buttons per server) for cluster-wide
fan-out.
manual refresh attempts reconnection.
When tools change, `MCPClientManager` rebuilds its merged tool list using copy-on-write
(new list/dict objects assigned atomically) and notifies all active `ChatSession`
@@ -777,6 +778,11 @@ instances via registered listener callbacks. Each session rebuilds its `_tools`,
`_task_tools`, `_agent_tools`, and reconstructs its `ToolSearchManager` (if active),
preserving the set of previously expanded (discovered) tools.
```toml
[mcp]
refresh_interval = 14400 # seconds (default 4h), 0 to disable
```
```
/mcp refresh
MCP refresh complete:
+1 -38
View File
@@ -2,48 +2,11 @@
An MCP server that exposes tools for executing commands across a Turnstone cluster. Serves as a reference implementation for both MCP server patterns and Turnstone SDK usage.
> [!NOTE]
> **Superseded by the built-in coordinator workstream in Turnstone 1.5.**
>
> This MCP side-car is the pre-1.5 pattern for cluster-wide orchestration.
> Turnstone 1.5 promotes coordinator behaviour to a first-class workstream
> kind hosted inside `turnstone-console` — no external MCP server to
> install or operate, proper per-user audit attribution, and a dedicated
> UI at `/coordinator/{ws_id}`.
>
> The extension continues to work for 1.4-and-earlier clusters. On 1.5+:
> grant the `admin.coordinator` permission, set `coordinator.model_alias`
> in the admin Settings tab, and create sessions via the dashboard's
> "new coordinator" button or `POST /v1/api/coordinator/new`. Full
> removal of this example (including docker / compose references) is
> planned once 1.5 is confirmed in production.
>
> | Concern | Built-in coordinator (1.5+) | This MCP extension (1.4-and-earlier) |
> |---|---|---|
> | Install | None — shipped in-tree | `pip install -e examples/mcp-cluster-ops` + MCP client config |
> | Auth | Real creator's `user_id` + `admin.coordinator` permission | Shared service token |
> | Audit | `coordinator.create` / `close` / `cancel` events on the console; `src="coordinator"` preserved on upstream hops | Service identity only |
> | UI | `/coordinator/{ws_id}` one-pane HTML | No UI — model-only |
> | Tool approvals | Inline approval bar in the coordinator pane | MCP approval flow |
> | Configuration | `coordinator.model_alias`, `coordinator.max_active`, `coordinator.reasoning_effort`, `coordinator.session_jwt_ttl_seconds` | MCP server config file |
>
> Minimal 1.5 migration:
>
> ```bash
> curl -X POST https://console.example/v1/api/coordinator/new \
> -H "Authorization: Bearer $TOKEN" \
> -H "Content-Type: application/json" \
> -d '{"name":"planner","initial_message":"Spawn a worker to check the build"}'
> ```
>
> The response carries `ws_id`; open
> `https://console.example/coordinator/{ws_id}` to watch the session.
## How it works
This server uses the Turnstone console SDK (`TurnstoneConsole`) for node discovery and routing, and `TurnstoneServer` for per-node SSE streaming. The dispatch flow for each command is:
1. **Route**`TurnstoneConsole.route_create_workstream(target_node=..., auto_approve=True)` creates a workstream pinned to the target node via the console's rendezvous routing proxy, returning `ws_id` and `node_url`.
1. **Route**`TurnstoneConsole.route_create_workstream(target_node=..., auto_approve=True)` creates a workstream pinned to the target node via the console's hash-ring routing proxy, returning `ws_id` and `node_url`.
2. **Execute**`TurnstoneServer(node_url, token=...)` connects directly to the node's SSE stream using the same `TURNSTONE_API_TOKEN`. `send_and_wait(prompt, ws_id)` runs the command and the raw bash output is captured from the `ToolResultEvent` — bypassing the costly "agent reads output then re-generates output as completion tokens" round-trip.
3. **Cleanup**`TurnstoneConsole.route_close(ws_id)` closes the workstream.
+9 -15
View File
@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
[project]
name = "turnstone"
version = "1.5.18"
version = "1.2.2"
description = "Multi-node AI orchestration platform with tool use, agent routing, and cluster simulation."
readme = "README.md"
license = "BUSL-1.1"
@@ -24,7 +24,7 @@ classifiers = [
dependencies = [
"openai>=2.24",
"httpx>=0.28",
"mcp>=1.27",
"mcp>=1.6",
"starlette>=0.45",
"uvicorn>=0.34",
"sse-starlette>=2.0",
@@ -35,7 +35,6 @@ dependencies = [
"structlog>=24.1",
"PyJWT>=2.8",
"bcrypt>=4.0",
"cryptography>=42",
"python-frontmatter>=1.0",
]
@@ -45,7 +44,7 @@ Repository = "https://github.com/turnstonelabs/turnstone"
Issues = "https://github.com/turnstonelabs/turnstone/issues"
[project.optional-dependencies]
test = ["pytest>=9.0", "pytest-cov>=6.0", "croniter>=3.0", "slack-bolt>=1.18", "aiohttp>=3.9"]
test = ["pytest>=9.0", "pytest-cov>=6.0", "croniter>=3.0"]
dev = ["ruff>=0.9", "mypy>=1.14"]
console = ["croniter>=3.0"]
anthropic = ["anthropic>=0.39"]
@@ -54,8 +53,7 @@ ddg = ["ddgs>=9.0"]
discord = ["discord.py>=2.4"]
tls = ["lacme>=1.0.5"]
sandbox = ["sympy>=1.13", "numpy>=2.0", "scipy>=1.14", "pytest>=9.0"]
slack = ["slack-bolt>=1.18", "aiohttp>=3.9"]
all = ["turnstone[console,anthropic,postgres,discord,ddg,tls,sandbox,slack]"]
all = ["turnstone[console,anthropic,postgres,discord,ddg,tls,sandbox]"]
[project.scripts]
turnstone = "turnstone.cli:main"
@@ -77,15 +75,12 @@ include = [
"turnstone/console/static/*.html",
"turnstone/console/static/*.css",
"turnstone/console/static/*.js",
"turnstone/console/static/coordinator/*.html",
"turnstone/console/static/coordinator/*.css",
"turnstone/console/static/coordinator/*.js",
"turnstone/shared_static/*.css",
"turnstone/shared_static/*.js",
"turnstone/shared_static/katex-0.16.47/**/*",
"turnstone/shared_static/katex-0.16.45/**/*",
"turnstone/shared_static/hljs-11.11.1/**/*",
"turnstone/shared_static/mermaid-11.15.0/**/*",
"turnstone/shared_static/hls-1.6.16/**/*",
"turnstone/shared_static/mermaid-11.14.0/**/*",
"turnstone/shared_static/hls-1.6.15/**/*",
"turnstone/sdk/py.typed",
"turnstone/deploy/*.yaml",
]
@@ -186,6 +181,5 @@ disallow_untyped_decorators = false
warn_unused_ignores = false
[[tool.mypy.overrides]]
module = ["slack_bolt", "slack_bolt.*", "slack_sdk", "slack_sdk.*"]
ignore_missing_imports = true
disallow_untyped_calls = false
module = "tests.*"
disallow_untyped_defs = false
File diff suppressed because it is too large Load Diff
+3 -6
View File
@@ -50,14 +50,11 @@ update_refs() {
local old_pattern="$1" # e.g. katex-0.16.38
local new_pattern="$2" # e.g. katex-0.16.39
# Find all files with version references. Excludes the old versioned vendor
# directory itself (about to be rm -rf'd anyway) so we don't bother rewriting
# self-references inside it — but does NOT exclude all of shared_static/,
# because shared_static/renderer.js loads the vendored libs and needs the bump.
# Find all files with version references (excludes vendored JS and worktrees)
local files
files=$(grep -rl --include='*.toml' --include='*.html' --include='*.js' --include='*.md' --include='*.py' \
files=$(grep -rl --include='*.toml' --include='*.html' --include='*.js' --include='*.md' \
-F "$old_pattern" . \
--exclude-dir='.claude' --exclude-dir='node_modules' --exclude-dir="$old_pattern" \
--exclude-dir='.claude' --exclude-dir='node_modules' --exclude-dir='shared_static' \
2>/dev/null || true)
for f in $files; do
sed -i "s|${old_pattern}|${new_pattern}|g" "$f"
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
+164 -201
View File
@@ -1,12 +1,12 @@
{
"name": "@turnstone/sdk",
"version": "0.4.0",
"version": "0.3.0",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "@turnstone/sdk",
"version": "0.4.0",
"version": "0.3.0",
"license": "BUSL-1.1",
"devDependencies": {
"typescript": "^6.0.0",
@@ -14,24 +14,26 @@
}
},
"node_modules/@emnapi/core": {
"version": "1.10.0",
"resolved": "https://registry.npmjs.org/@emnapi/core/-/core-1.10.0.tgz",
"integrity": "sha512-yq6OkJ4p82CAfPl0u9mQebQHKPJkY7WrIuk205cTYnYe+k2Z8YBh11FrbRG/H6ihirqcacOgl2BIO8oyMQLeXw==",
"version": "1.9.2",
"resolved": "https://registry.npmjs.org/@emnapi/core/-/core-1.9.2.tgz",
"integrity": "sha512-UC+ZhH3XtczQYfOlu3lNEkdW/p4dsJ1r/bP7H8+rhao3TTTMO1ATq/4DdIi23XuGoFY+Cz0JmCbdVl0hz9jZcA==",
"dev": true,
"license": "MIT",
"optional": true,
"peer": true,
"dependencies": {
"@emnapi/wasi-threads": "1.2.1",
"tslib": "^2.4.0"
}
},
"node_modules/@emnapi/runtime": {
"version": "1.10.0",
"resolved": "https://registry.npmjs.org/@emnapi/runtime/-/runtime-1.10.0.tgz",
"integrity": "sha512-ewvYlk86xUoGI0zQRNq/mC+16R1QeDlKQy21Ki3oSYXNgLb45GV1P6A0M+/s6nyCuNDqe5VpaY84BzXGwVbwFA==",
"version": "1.9.2",
"resolved": "https://registry.npmjs.org/@emnapi/runtime/-/runtime-1.9.2.tgz",
"integrity": "sha512-3U4+MIWHImeyu1wnmVygh5WlgfYDtyf0k8AbLhMFxOipihf6nrWC4syIm/SwEeec0mNSafiiNnMJwbza/Is6Lw==",
"dev": true,
"license": "MIT",
"optional": true,
"peer": true,
"dependencies": {
"tslib": "^2.4.0"
}
@@ -43,6 +45,7 @@
"dev": true,
"license": "MIT",
"optional": true,
"peer": true,
"dependencies": {
"tslib": "^2.4.0"
}
@@ -55,9 +58,9 @@
"license": "MIT"
},
"node_modules/@napi-rs/wasm-runtime": {
"version": "1.1.4",
"resolved": "https://registry.npmjs.org/@napi-rs/wasm-runtime/-/wasm-runtime-1.1.4.tgz",
"integrity": "sha512-3NQNNgA1YSlJb/kMH1ildASP9HW7/7kYnRI2szWJaofaS1hWmbGI4H+d3+22aGzXXN9IJ+n+GiFVcGipJP18ow==",
"version": "1.1.2",
"resolved": "https://registry.npmjs.org/@napi-rs/wasm-runtime/-/wasm-runtime-1.1.2.tgz",
"integrity": "sha512-sNXv5oLJ7ob93xkZ1XnxisYhGYXfaG9f65/ZgYuAu3qt7b3NadcOEhLvx28hv31PgX8SZJRYrAIPQilQmFpLVw==",
"dev": true,
"license": "MIT",
"optional": true,
@@ -74,9 +77,9 @@
}
},
"node_modules/@oxc-project/types": {
"version": "0.130.0",
"resolved": "https://registry.npmjs.org/@oxc-project/types/-/types-0.130.0.tgz",
"integrity": "sha512-ibD2usx9JRu7f5pu2tMKMI4cpA4NgXJQoYRP4pQ7Pxmn1l6k/53qWtQWZayhYy3X4QZkt90Ot+mJEaeXouio6Q==",
"version": "0.122.0",
"resolved": "https://registry.npmjs.org/@oxc-project/types/-/types-0.122.0.tgz",
"integrity": "sha512-oLAl5kBpV4w69UtFZ9xqcmTi+GENWOcPF7FCrczTiBbmC0ibXxCwyvZGbO39rCVEuLGAZM84DH0pUIyyv/YJzA==",
"dev": true,
"license": "MIT",
"funding": {
@@ -84,9 +87,9 @@
}
},
"node_modules/@rolldown/binding-android-arm64": {
"version": "1.0.1",
"resolved": "https://registry.npmjs.org/@rolldown/binding-android-arm64/-/binding-android-arm64-1.0.1.tgz",
"integrity": "sha512-fJI3I0r3C3Oj/zdBCpaCmBRZYf07xpaq4yCfDDoSFm+beWNzbIl26puW8RraUdugoJw/95zerNOn6jasAhzSmg==",
"version": "1.0.0-rc.12",
"resolved": "https://registry.npmjs.org/@rolldown/binding-android-arm64/-/binding-android-arm64-1.0.0-rc.12.tgz",
"integrity": "sha512-pv1y2Fv0JybcykuiiD3qBOBdz6RteYojRFY1d+b95WVuzx211CRh+ytI/+9iVyWQ6koTh5dawe4S/yRfOFjgaA==",
"cpu": [
"arm64"
],
@@ -101,9 +104,9 @@
}
},
"node_modules/@rolldown/binding-darwin-arm64": {
"version": "1.0.1",
"resolved": "https://registry.npmjs.org/@rolldown/binding-darwin-arm64/-/binding-darwin-arm64-1.0.1.tgz",
"integrity": "sha512-cKnAhWEsV7TPcA/5EAteDp6KcJZBQ2G+BqE7zayMMi7kMvwRsbv7WT9aOnn0WNl4SKEIf43vjS31iUPu80nzXg==",
"version": "1.0.0-rc.12",
"resolved": "https://registry.npmjs.org/@rolldown/binding-darwin-arm64/-/binding-darwin-arm64-1.0.0-rc.12.tgz",
"integrity": "sha512-cFYr6zTG/3PXXF3pUO+umXxt1wkRK/0AYT8lDwuqvRC+LuKYWSAQAQZjCWDQpAH172ZV6ieYrNnFzVVcnSflAg==",
"cpu": [
"arm64"
],
@@ -118,9 +121,9 @@
}
},
"node_modules/@rolldown/binding-darwin-x64": {
"version": "1.0.1",
"resolved": "https://registry.npmjs.org/@rolldown/binding-darwin-x64/-/binding-darwin-x64-1.0.1.tgz",
"integrity": "sha512-YKrVwQjIRBPo+5G/u03wGjbdy4q7pyzCe93DK9VJ7zkVmeg8LJ7GbgsiHWdR4xSoe4CAXRD7Bcjgbtr64bkXNg==",
"version": "1.0.0-rc.12",
"resolved": "https://registry.npmjs.org/@rolldown/binding-darwin-x64/-/binding-darwin-x64-1.0.0-rc.12.tgz",
"integrity": "sha512-ZCsYknnHzeXYps0lGBz8JrF37GpE9bFVefrlmDrAQhOEi4IOIlcoU1+FwHEtyXGx2VkYAvhu7dyBf75EJQffBw==",
"cpu": [
"x64"
],
@@ -135,9 +138,9 @@
}
},
"node_modules/@rolldown/binding-freebsd-x64": {
"version": "1.0.1",
"resolved": "https://registry.npmjs.org/@rolldown/binding-freebsd-x64/-/binding-freebsd-x64-1.0.1.tgz",
"integrity": "sha512-z/oBsREo46SsFqBwYtFe0kpJeBijAT48O/WXLI4suiCLBkr03RTtTJMCzSdDd2znlh8VJizL09XVkQgk8IZonw==",
"version": "1.0.0-rc.12",
"resolved": "https://registry.npmjs.org/@rolldown/binding-freebsd-x64/-/binding-freebsd-x64-1.0.0-rc.12.tgz",
"integrity": "sha512-dMLeprcVsyJsKolRXyoTH3NL6qtsT0Y2xeuEA8WQJquWFXkEC4bcu1rLZZSnZRMtAqwtrF/Ib9Ddtpa/Gkge9Q==",
"cpu": [
"x64"
],
@@ -152,9 +155,9 @@
}
},
"node_modules/@rolldown/binding-linux-arm-gnueabihf": {
"version": "1.0.1",
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-arm-gnueabihf/-/binding-linux-arm-gnueabihf-1.0.1.tgz",
"integrity": "sha512-ik8q7GM11zxvYxFc2PeDcT6TBvhCQMaUxfph/M5l9sKuTs/Sjg3L+Byw0F7w0ZVLBZmx30P+gG0ECzzN+MFcmQ==",
"version": "1.0.0-rc.12",
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-arm-gnueabihf/-/binding-linux-arm-gnueabihf-1.0.0-rc.12.tgz",
"integrity": "sha512-YqWjAgGC/9M1lz3GR1r1rP79nMgo3mQiiA+Hfo+pvKFK1fAJ1bCi0ZQVh8noOqNacuY1qIcfyVfP6HoyBRZ85Q==",
"cpu": [
"arm"
],
@@ -169,16 +172,13 @@
}
},
"node_modules/@rolldown/binding-linux-arm64-gnu": {
"version": "1.0.1",
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-arm64-gnu/-/binding-linux-arm64-gnu-1.0.1.tgz",
"integrity": "sha512-QoSx2EkyrrdZ6kcyE8stqZ62t0Yra8Fs5ia9lOxJrh6TMQJK7gQKmscdTHf7pOXKREKrVwOtJcQG3qVSfc866A==",
"version": "1.0.0-rc.12",
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-arm64-gnu/-/binding-linux-arm64-gnu-1.0.0-rc.12.tgz",
"integrity": "sha512-/I5AS4cIroLpslsmzXfwbe5OmWvSsrFuEw3mwvbQ1kDxJ822hFHIx+vsN/TAzNVyepI/j/GSzrtCIwQPeKCLIg==",
"cpu": [
"arm64"
],
"dev": true,
"libc": [
"glibc"
],
"license": "MIT",
"optional": true,
"os": [
@@ -189,16 +189,13 @@
}
},
"node_modules/@rolldown/binding-linux-arm64-musl": {
"version": "1.0.1",
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-arm64-musl/-/binding-linux-arm64-musl-1.0.1.tgz",
"integrity": "sha512-uwNwFpwKeNiZawfAWBgg0VIztPTV3ihhh1vV334h9ivnNLorxnQMU6Fz8wG1Zb4Qh9LC1/MkcyT3YlDXG3Rsgg==",
"version": "1.0.0-rc.12",
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-arm64-musl/-/binding-linux-arm64-musl-1.0.0-rc.12.tgz",
"integrity": "sha512-V6/wZztnBqlx5hJQqNWwFdxIKN0m38p8Jas+VoSfgH54HSj9tKTt1dZvG6JRHcjh6D7TvrJPWFGaY9UBVOaWPw==",
"cpu": [
"arm64"
],
"dev": true,
"libc": [
"musl"
],
"license": "MIT",
"optional": true,
"os": [
@@ -209,16 +206,13 @@
}
},
"node_modules/@rolldown/binding-linux-ppc64-gnu": {
"version": "1.0.1",
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-ppc64-gnu/-/binding-linux-ppc64-gnu-1.0.1.tgz",
"integrity": "sha512-zY1bul7OWr7DFBiJ++wofXvnr8B45ce3QsQUhKrIhXsygAh7bTkwyeM1bi1a2g5C/yC/N8TZyGDEoMfm/l9mpg==",
"version": "1.0.0-rc.12",
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-ppc64-gnu/-/binding-linux-ppc64-gnu-1.0.0-rc.12.tgz",
"integrity": "sha512-AP3E9BpcUYliZCxa3w5Kwj9OtEVDYK6sVoUzy4vTOJsjPOgdaJZKFmN4oOlX0Wp0RPV2ETfmIra9x1xuayFB7g==",
"cpu": [
"ppc64"
],
"dev": true,
"libc": [
"glibc"
],
"license": "MIT",
"optional": true,
"os": [
@@ -229,16 +223,13 @@
}
},
"node_modules/@rolldown/binding-linux-s390x-gnu": {
"version": "1.0.1",
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-s390x-gnu/-/binding-linux-s390x-gnu-1.0.1.tgz",
"integrity": "sha512-0frlsT/f4Ft6I7SMESTKnF3cZsdicQn1dCMkF/jT9wDLE+gGoiQfv1nmT9e+s7s/fekvvy6tZM2jHvI2tkbJDQ==",
"version": "1.0.0-rc.12",
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-s390x-gnu/-/binding-linux-s390x-gnu-1.0.0-rc.12.tgz",
"integrity": "sha512-nWwpvUSPkoFmZo0kQazZYOrT7J5DGOJ/+QHHzjvNlooDZED8oH82Yg67HvehPPLAg5fUff7TfWFHQS8IV1n3og==",
"cpu": [
"s390x"
],
"dev": true,
"libc": [
"glibc"
],
"license": "MIT",
"optional": true,
"os": [
@@ -249,16 +240,13 @@
}
},
"node_modules/@rolldown/binding-linux-x64-gnu": {
"version": "1.0.1",
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-x64-gnu/-/binding-linux-x64-gnu-1.0.1.tgz",
"integrity": "sha512-XABVmGp9Tg0WspTVvwduTc4fpqy6JnAUrSQe6OuyqD/03nI7r0O9OWUkMIwFrjKAIqolvqoA4ZrJppgwE0Gxmw==",
"version": "1.0.0-rc.12",
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-x64-gnu/-/binding-linux-x64-gnu-1.0.0-rc.12.tgz",
"integrity": "sha512-RNrafz5bcwRy+O9e6P8Z/OCAJW/A+qtBczIqVYwTs14pf4iV1/+eKEjdOUta93q2TsT/FI0XYDP3TCky38LMAg==",
"cpu": [
"x64"
],
"dev": true,
"libc": [
"glibc"
],
"license": "MIT",
"optional": true,
"os": [
@@ -269,16 +257,13 @@
}
},
"node_modules/@rolldown/binding-linux-x64-musl": {
"version": "1.0.1",
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-x64-musl/-/binding-linux-x64-musl-1.0.1.tgz",
"integrity": "sha512-bV4fzswuzVcKD90o/VM6QqKxnxlDq0g2BISDLNVmxrnhpv1DDbyPhCIjYfvzYLV+MvkKKnQt2Q6AO86SEBULUQ==",
"version": "1.0.0-rc.12",
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-x64-musl/-/binding-linux-x64-musl-1.0.0-rc.12.tgz",
"integrity": "sha512-Jpw/0iwoKWx3LJ2rc1yjFrj+T7iHZn2JDg1Yny1ma0luviFS4mhAIcd1LFNxK3EYu3DHWCps0ydXQ5i/rrJ2ig==",
"cpu": [
"x64"
],
"dev": true,
"libc": [
"musl"
],
"license": "MIT",
"optional": true,
"os": [
@@ -289,9 +274,9 @@
}
},
"node_modules/@rolldown/binding-openharmony-arm64": {
"version": "1.0.1",
"resolved": "https://registry.npmjs.org/@rolldown/binding-openharmony-arm64/-/binding-openharmony-arm64-1.0.1.tgz",
"integrity": "sha512-/Mh0Zhq3OP7fVs0kcQHZP6lZEthMGTaSf8UBQYSFEZDWGXXlEC+nJ6EqenaK2t4LBXMe3A+K/G2BVXXdtOr4PQ==",
"version": "1.0.0-rc.12",
"resolved": "https://registry.npmjs.org/@rolldown/binding-openharmony-arm64/-/binding-openharmony-arm64-1.0.0-rc.12.tgz",
"integrity": "sha512-vRugONE4yMfVn0+7lUKdKvN4D5YusEiPilaoO2sgUWpCvrncvWgPMzK00ZFFJuiPgLwgFNP5eSiUlv2tfc+lpA==",
"cpu": [
"arm64"
],
@@ -306,9 +291,9 @@
}
},
"node_modules/@rolldown/binding-wasm32-wasi": {
"version": "1.0.1",
"resolved": "https://registry.npmjs.org/@rolldown/binding-wasm32-wasi/-/binding-wasm32-wasi-1.0.1.tgz",
"integrity": "sha512-+1xc9X45l8ufsBAm6Gjvx2qDRIY9lTVt0cgWNcJ+1gdhXvkbxePA60yRTwSTuXL09CMhyJmjpV7E3NoyxbqFQQ==",
"version": "1.0.0-rc.12",
"resolved": "https://registry.npmjs.org/@rolldown/binding-wasm32-wasi/-/binding-wasm32-wasi-1.0.0-rc.12.tgz",
"integrity": "sha512-ykGiLr/6kkiHc0XnBfmFJuCjr5ZYKKofkx+chJWDjitX+KsJuAmrzWhwyOMSHzPhzOHOy7u9HlFoa5MoAOJ/Zg==",
"cpu": [
"wasm32"
],
@@ -316,18 +301,16 @@
"license": "MIT",
"optional": true,
"dependencies": {
"@emnapi/core": "1.10.0",
"@emnapi/runtime": "1.10.0",
"@napi-rs/wasm-runtime": "^1.1.4"
"@napi-rs/wasm-runtime": "^1.1.1"
},
"engines": {
"node": "^20.19.0 || >=22.12.0"
"node": ">=14.0.0"
}
},
"node_modules/@rolldown/binding-win32-arm64-msvc": {
"version": "1.0.1",
"resolved": "https://registry.npmjs.org/@rolldown/binding-win32-arm64-msvc/-/binding-win32-arm64-msvc-1.0.1.tgz",
"integrity": "sha512-1D+UqZdfnuR+Jy1GgMJwi85bD40H21uNmOPRWQhw4oRSuolZ/B5rixZ45DK2KXOTCvmVCecauWgEhbw8bI7tOw==",
"version": "1.0.0-rc.12",
"resolved": "https://registry.npmjs.org/@rolldown/binding-win32-arm64-msvc/-/binding-win32-arm64-msvc-1.0.0-rc.12.tgz",
"integrity": "sha512-5eOND4duWkwx1AzCxadcOrNeighiLwMInEADT0YM7xeEOOFcovWZCq8dadXgcRHSf3Ulh1kFo/qvzoFiCLOL1Q==",
"cpu": [
"arm64"
],
@@ -342,9 +325,9 @@
}
},
"node_modules/@rolldown/binding-win32-x64-msvc": {
"version": "1.0.1",
"resolved": "https://registry.npmjs.org/@rolldown/binding-win32-x64-msvc/-/binding-win32-x64-msvc-1.0.1.tgz",
"integrity": "sha512-INAycaWuhlOK3wk4mRHGsdgwYWmd9cChdPdE9bwWmy6rn9VqVNYNFGhOdXrofXUxwHIncSiPNb8tNm8knDVIeQ==",
"version": "1.0.0-rc.12",
"resolved": "https://registry.npmjs.org/@rolldown/binding-win32-x64-msvc/-/binding-win32-x64-msvc-1.0.0-rc.12.tgz",
"integrity": "sha512-PyqoipaswDLAZtot351MLhrlrh6lcZPo2LSYE+VDxbVk24LVKAGOuE4hb8xZQmrPAuEtTZW8E6D2zc5EUZX4Lw==",
"cpu": [
"x64"
],
@@ -359,9 +342,9 @@
}
},
"node_modules/@rolldown/pluginutils": {
"version": "1.0.1",
"resolved": "https://registry.npmjs.org/@rolldown/pluginutils/-/pluginutils-1.0.1.tgz",
"integrity": "sha512-2j9bGt5Jh8hj+vPtgzPtl72j0yRxHAyumoo6TNfAjsLB04UtpSvPbPcDcBMxz7n+9CYB0c1GxQFxYRg2jimqGw==",
"version": "1.0.0-rc.12",
"resolved": "https://registry.npmjs.org/@rolldown/pluginutils/-/pluginutils-1.0.0-rc.12.tgz",
"integrity": "sha512-HHMwmarRKvoFsJorqYlFeFRzXZqCt2ETQlEDOb9aqssrnVBB1/+xgTGtuTrIk5vzLNX1MjMtTf7W9z3tsSbrxw==",
"dev": true,
"license": "MIT"
},
@@ -373,9 +356,9 @@
"license": "MIT"
},
"node_modules/@tybys/wasm-util": {
"version": "0.10.2",
"resolved": "https://registry.npmjs.org/@tybys/wasm-util/-/wasm-util-0.10.2.tgz",
"integrity": "sha512-RoBvJ2X0wuKlWFIjrwffGw1IqZHKQqzIchKaadZZfnNpsAYp2mM0h36JtPCjNDAHGgYez/15uMBpfGwchhiMgg==",
"version": "0.10.1",
"resolved": "https://registry.npmjs.org/@tybys/wasm-util/-/wasm-util-0.10.1.tgz",
"integrity": "sha512-9tTaPJLSiejZKx+Bmog4uSubteqTvFrVrURwkmHixBo0G4seD0zUxp98E1DzUBJxLQ3NPwXrGKDiVjwx/DpPsg==",
"dev": true,
"license": "MIT",
"optional": true,
@@ -402,23 +385,23 @@
"license": "MIT"
},
"node_modules/@types/estree": {
"version": "1.0.9",
"resolved": "https://registry.npmjs.org/@types/estree/-/estree-1.0.9.tgz",
"integrity": "sha512-GhdPgy1el4/ImP05X05Uw4cw2/M93BCUmnEvWZNStlCzEKME4Fkk+YpoA5OiHNQmoS7Cafb8Xa3Pya8m1Qrzeg==",
"version": "1.0.8",
"resolved": "https://registry.npmjs.org/@types/estree/-/estree-1.0.8.tgz",
"integrity": "sha512-dWHzHa2WqEXI/O1E9OjrocMTKJl2mSrEolh1Iomrv6U+JuNwaHXsXx9bLu5gG7BUWFIN0skIQJQ/L1rIex4X6w==",
"dev": true,
"license": "MIT"
},
"node_modules/@vitest/expect": {
"version": "4.1.6",
"resolved": "https://registry.npmjs.org/@vitest/expect/-/expect-4.1.6.tgz",
"integrity": "sha512-7EHDquPthALSV0jhhjgEW8FXaviMx7rSqu8W6oqCoAuOhKov814P99QDV1pxMA3QPv21YudvJngIhjrNI4opLg==",
"version": "4.1.2",
"resolved": "https://registry.npmjs.org/@vitest/expect/-/expect-4.1.2.tgz",
"integrity": "sha512-gbu+7B0YgUJ2nkdsRJrFFW6X7NTP44WlhiclHniUhxADQJH5Szt9mZ9hWnJPJ8YwOK5zUOSSlSvyzRf0u1DSBQ==",
"dev": true,
"license": "MIT",
"dependencies": {
"@standard-schema/spec": "^1.1.0",
"@types/chai": "^5.2.2",
"@vitest/spy": "4.1.6",
"@vitest/utils": "4.1.6",
"@vitest/spy": "4.1.2",
"@vitest/utils": "4.1.2",
"chai": "^6.2.2",
"tinyrainbow": "^3.1.0"
},
@@ -427,13 +410,13 @@
}
},
"node_modules/@vitest/mocker": {
"version": "4.1.6",
"resolved": "https://registry.npmjs.org/@vitest/mocker/-/mocker-4.1.6.tgz",
"integrity": "sha512-MCFc63czMjEInOlcY2cpQCvCN+KgbAn+60xu9cMgP4sKaLC5JNAKw7JH8QdAnoAC88hW1IiSNZ+GgVXlN1UcMQ==",
"version": "4.1.2",
"resolved": "https://registry.npmjs.org/@vitest/mocker/-/mocker-4.1.2.tgz",
"integrity": "sha512-Ize4iQtEALHDttPRCmN+FKqOl2vxTiNUhzobQFFt/BM1lRUTG7zRCLOykG/6Vo4E4hnUdfVLo5/eqKPukcWW7Q==",
"dev": true,
"license": "MIT",
"dependencies": {
"@vitest/spy": "4.1.6",
"@vitest/spy": "4.1.2",
"estree-walker": "^3.0.3",
"magic-string": "^0.30.21"
},
@@ -454,9 +437,9 @@
}
},
"node_modules/@vitest/pretty-format": {
"version": "4.1.6",
"resolved": "https://registry.npmjs.org/@vitest/pretty-format/-/pretty-format-4.1.6.tgz",
"integrity": "sha512-h5SxD/IzNhZYnrSZRsUZQIC+vD0GY8cUvq0iwsmkFKixRCKLLWqCXa/FIQ4S1R+sI+PGoojkHsdNrbZiM9Qpgw==",
"version": "4.1.2",
"resolved": "https://registry.npmjs.org/@vitest/pretty-format/-/pretty-format-4.1.2.tgz",
"integrity": "sha512-dwQga8aejqeuB+TvXCMzSQemvV9hNEtDDpgUKDzOmNQayl2OG241PSWeJwKRH3CiC+sESrmoFd49rfnq7T4RnA==",
"dev": true,
"license": "MIT",
"dependencies": {
@@ -467,13 +450,13 @@
}
},
"node_modules/@vitest/runner": {
"version": "4.1.6",
"resolved": "https://registry.npmjs.org/@vitest/runner/-/runner-4.1.6.tgz",
"integrity": "sha512-nOPCmn2+yD0ZNmKdsXGv/UxMMWbMuKeD6GyYncNwdkYDxpQvrPSKYj2rWuDjC2Y4b6w6hjip5dBKFzEUuZe3vA==",
"version": "4.1.2",
"resolved": "https://registry.npmjs.org/@vitest/runner/-/runner-4.1.2.tgz",
"integrity": "sha512-Gr+FQan34CdiYAwpGJmQG8PgkyFVmARK8/xSijia3eTFgVfpcpztWLuP6FttGNfPLJhaZVP/euvujeNYar36OQ==",
"dev": true,
"license": "MIT",
"dependencies": {
"@vitest/utils": "4.1.6",
"@vitest/utils": "4.1.2",
"pathe": "^2.0.3"
},
"funding": {
@@ -481,14 +464,14 @@
}
},
"node_modules/@vitest/snapshot": {
"version": "4.1.6",
"resolved": "https://registry.npmjs.org/@vitest/snapshot/-/snapshot-4.1.6.tgz",
"integrity": "sha512-YhsdE6xAVfTDmzjxL2ZDUvjj+ZsgyOKe+TdQzqkD72wIOmHka8NuGQ6NpTNZv9D2Z63fbwWKJPeVpEw4EQgYxw==",
"version": "4.1.2",
"resolved": "https://registry.npmjs.org/@vitest/snapshot/-/snapshot-4.1.2.tgz",
"integrity": "sha512-g7yfUmxYS4mNxk31qbOYsSt2F4m1E02LFqO53Xpzg3zKMhLAPZAjjfyl9e6z7HrW6LvUdTwAQR3HHfLjpko16A==",
"dev": true,
"license": "MIT",
"dependencies": {
"@vitest/pretty-format": "4.1.6",
"@vitest/utils": "4.1.6",
"@vitest/pretty-format": "4.1.2",
"@vitest/utils": "4.1.2",
"magic-string": "^0.30.21",
"pathe": "^2.0.3"
},
@@ -497,9 +480,9 @@
}
},
"node_modules/@vitest/spy": {
"version": "4.1.6",
"resolved": "https://registry.npmjs.org/@vitest/spy/-/spy-4.1.6.tgz",
"integrity": "sha512-JFKxMx6udhwKh/Ldo270e17QX710vgunMkuPAvXjHSvC6oqLWAHhVhjg/I71q0u0CBSErIODV1Kjv0FQNSWjdg==",
"version": "4.1.2",
"resolved": "https://registry.npmjs.org/@vitest/spy/-/spy-4.1.2.tgz",
"integrity": "sha512-DU4fBnbVCJGNBwVA6xSToNXrkZNSiw59H8tcuUspVMsBDBST4nfvsPsEHDHGtWRRnqBERBQu7TrTKskmjqTXKA==",
"dev": true,
"license": "MIT",
"funding": {
@@ -507,13 +490,13 @@
}
},
"node_modules/@vitest/utils": {
"version": "4.1.6",
"resolved": "https://registry.npmjs.org/@vitest/utils/-/utils-4.1.6.tgz",
"integrity": "sha512-FxIY+U81R3LGKCxaHHFRQ5+g6/iRgGLmeHWdp2Amj4ljQRrEIWHmZyDfDYBRZlpyqA7qKxtS9DD1dhk8RnRIVQ==",
"version": "4.1.2",
"resolved": "https://registry.npmjs.org/@vitest/utils/-/utils-4.1.2.tgz",
"integrity": "sha512-xw2/TiX82lQHA06cgbqRKFb5lCAy3axQ4H4SoUFhUsg+wztiet+co86IAMDtF6Vm1hc7J6j09oh/rgDn+JdKIQ==",
"dev": true,
"license": "MIT",
"dependencies": {
"@vitest/pretty-format": "4.1.6",
"@vitest/pretty-format": "4.1.2",
"convert-source-map": "^2.0.0",
"tinyrainbow": "^3.1.0"
},
@@ -559,9 +542,9 @@
}
},
"node_modules/es-module-lexer": {
"version": "2.1.0",
"resolved": "https://registry.npmjs.org/es-module-lexer/-/es-module-lexer-2.1.0.tgz",
"integrity": "sha512-n27zTYMjYu1aj4MjCWzSP7G9r75utsaoc8m61weK+W8JMBGGQybd43GstCXZ3WNmSFtGT9wi59qQTW6mhTR5LQ==",
"version": "2.0.0",
"resolved": "https://registry.npmjs.org/es-module-lexer/-/es-module-lexer-2.0.0.tgz",
"integrity": "sha512-5POEcUuZybH7IdmGsD8wlf0AI55wMecM9rVBTI/qEAy2c1kTOm3DjFYjrBdI2K3BaJjJYfYFeRtM0t9ssnRuxw==",
"dev": true,
"license": "MIT"
},
@@ -761,9 +744,6 @@
"arm64"
],
"dev": true,
"libc": [
"glibc"
],
"license": "MPL-2.0",
"optional": true,
"os": [
@@ -785,9 +765,6 @@
"arm64"
],
"dev": true,
"libc": [
"musl"
],
"license": "MPL-2.0",
"optional": true,
"os": [
@@ -809,9 +786,6 @@
"x64"
],
"dev": true,
"libc": [
"glibc"
],
"license": "MPL-2.0",
"optional": true,
"os": [
@@ -833,9 +807,6 @@
"x64"
],
"dev": true,
"libc": [
"musl"
],
"license": "MPL-2.0",
"optional": true,
"os": [
@@ -902,9 +873,9 @@
}
},
"node_modules/nanoid": {
"version": "3.3.12",
"resolved": "https://registry.npmjs.org/nanoid/-/nanoid-3.3.12.tgz",
"integrity": "sha512-ZB9RH/39qpq5Vu6Y+NmUaFhQR6pp+M2Xt76XBnEwDaGcVAqhlvxrl3B2bKS5D3NH3QR76v3aSrKaF/Kiy7lEtQ==",
"version": "3.3.11",
"resolved": "https://registry.npmjs.org/nanoid/-/nanoid-3.3.11.tgz",
"integrity": "sha512-N8SpfPUnUp1bK+PMYW8qSWdl9U+wwNWI4QKxOYDy9JAro3WMX7p2OeVRF9v+347pnakNevPmiHhNmZ2HbFA76w==",
"dev": true,
"funding": [
{
@@ -959,9 +930,9 @@
}
},
"node_modules/postcss": {
"version": "8.5.14",
"resolved": "https://registry.npmjs.org/postcss/-/postcss-8.5.14.tgz",
"integrity": "sha512-SoSL4+OSEtR99LHFZQiJLkT59C5B1amGO1NzTwj7TT1qCUgUO6hxOvzkOYxD+vMrXBM3XJIKzokoERdqQq/Zmg==",
"version": "8.5.8",
"resolved": "https://registry.npmjs.org/postcss/-/postcss-8.5.8.tgz",
"integrity": "sha512-OW/rX8O/jXnm82Ey1k44pObPtdblfiuWnrd8X7GJ7emImCOstunGbXUpp7HdBrFQX6rJzn3sPT397Wp5aCwCHg==",
"dev": true,
"funding": [
{
@@ -988,14 +959,14 @@
}
},
"node_modules/rolldown": {
"version": "1.0.1",
"resolved": "https://registry.npmjs.org/rolldown/-/rolldown-1.0.1.tgz",
"integrity": "sha512-X0KQHljNnEkWNqqiz9zJrGunh1B0HgOxLXvnFpCOcadzcy5qohZ3tqMEUg00vncoRovXuK3ZqCT9KnnKzoInFQ==",
"version": "1.0.0-rc.12",
"resolved": "https://registry.npmjs.org/rolldown/-/rolldown-1.0.0-rc.12.tgz",
"integrity": "sha512-yP4USLIMYrwpPHEFB5JGH1uxhcslv6/hL0OyvTuY+3qlOSJvZ7ntYnoWpehBxufkgN0cvXxppuTu5hHa/zPh+A==",
"dev": true,
"license": "MIT",
"dependencies": {
"@oxc-project/types": "=0.130.0",
"@rolldown/pluginutils": "^1.0.0"
"@oxc-project/types": "=0.122.0",
"@rolldown/pluginutils": "1.0.0-rc.12"
},
"bin": {
"rolldown": "bin/cli.mjs"
@@ -1004,21 +975,21 @@
"node": "^20.19.0 || >=22.12.0"
},
"optionalDependencies": {
"@rolldown/binding-android-arm64": "1.0.1",
"@rolldown/binding-darwin-arm64": "1.0.1",
"@rolldown/binding-darwin-x64": "1.0.1",
"@rolldown/binding-freebsd-x64": "1.0.1",
"@rolldown/binding-linux-arm-gnueabihf": "1.0.1",
"@rolldown/binding-linux-arm64-gnu": "1.0.1",
"@rolldown/binding-linux-arm64-musl": "1.0.1",
"@rolldown/binding-linux-ppc64-gnu": "1.0.1",
"@rolldown/binding-linux-s390x-gnu": "1.0.1",
"@rolldown/binding-linux-x64-gnu": "1.0.1",
"@rolldown/binding-linux-x64-musl": "1.0.1",
"@rolldown/binding-openharmony-arm64": "1.0.1",
"@rolldown/binding-wasm32-wasi": "1.0.1",
"@rolldown/binding-win32-arm64-msvc": "1.0.1",
"@rolldown/binding-win32-x64-msvc": "1.0.1"
"@rolldown/binding-android-arm64": "1.0.0-rc.12",
"@rolldown/binding-darwin-arm64": "1.0.0-rc.12",
"@rolldown/binding-darwin-x64": "1.0.0-rc.12",
"@rolldown/binding-freebsd-x64": "1.0.0-rc.12",
"@rolldown/binding-linux-arm-gnueabihf": "1.0.0-rc.12",
"@rolldown/binding-linux-arm64-gnu": "1.0.0-rc.12",
"@rolldown/binding-linux-arm64-musl": "1.0.0-rc.12",
"@rolldown/binding-linux-ppc64-gnu": "1.0.0-rc.12",
"@rolldown/binding-linux-s390x-gnu": "1.0.0-rc.12",
"@rolldown/binding-linux-x64-gnu": "1.0.0-rc.12",
"@rolldown/binding-linux-x64-musl": "1.0.0-rc.12",
"@rolldown/binding-openharmony-arm64": "1.0.0-rc.12",
"@rolldown/binding-wasm32-wasi": "1.0.0-rc.12",
"@rolldown/binding-win32-arm64-msvc": "1.0.0-rc.12",
"@rolldown/binding-win32-x64-msvc": "1.0.0-rc.12"
}
},
"node_modules/siginfo": {
@@ -1046,9 +1017,9 @@
"license": "MIT"
},
"node_modules/std-env": {
"version": "4.1.0",
"resolved": "https://registry.npmjs.org/std-env/-/std-env-4.1.0.tgz",
"integrity": "sha512-Rq7ybcX2RuC55r9oaPVEW7/xu3tj8u4GeBYHBWCychFtzMIr86A7e3PPEBPT37sHStKX3+TiX/Fr/ACmJLVlLQ==",
"version": "4.0.0",
"resolved": "https://registry.npmjs.org/std-env/-/std-env-4.0.0.tgz",
"integrity": "sha512-zUMPtQ/HBY3/50VbpkupYHbRroTRZJPRLvreamgErJVys0ceuzMkD44J/QjqhHjOzK42GQ3QZIeFG1OYfOtKqQ==",
"dev": true,
"license": "MIT"
},
@@ -1060,9 +1031,9 @@
"license": "MIT"
},
"node_modules/tinyexec": {
"version": "1.1.2",
"resolved": "https://registry.npmjs.org/tinyexec/-/tinyexec-1.1.2.tgz",
"integrity": "sha512-dAqSqE/RabpBKI8+h26GfLq6Vb3JVXs30XYQjdMjaj/c2tS8IYYMbIzP599KtRj7c57/wYApb3QjgRgXmrCukA==",
"version": "1.0.4",
"resolved": "https://registry.npmjs.org/tinyexec/-/tinyexec-1.0.4.tgz",
"integrity": "sha512-u9r3uZC0bdpGOXtlxUIdwf9pkmvhqJdrVCH9fapQtgy/OeTTMZ1nqH7agtvEfmGui6e1XxjcdrlxvxJvc3sMqw==",
"dev": true,
"license": "MIT",
"engines": {
@@ -1070,14 +1041,14 @@
}
},
"node_modules/tinyglobby": {
"version": "0.2.16",
"resolved": "https://registry.npmjs.org/tinyglobby/-/tinyglobby-0.2.16.tgz",
"integrity": "sha512-pn99VhoACYR8nFHhxqix+uvsbXineAasWm5ojXoN8xEwK5Kd3/TrhNn1wByuD52UxWRLy8pu+kRMniEi6Eq9Zg==",
"version": "0.2.15",
"resolved": "https://registry.npmjs.org/tinyglobby/-/tinyglobby-0.2.15.tgz",
"integrity": "sha512-j2Zq4NyQYG5XMST4cbs02Ak8iJUdxRM0XI5QyxXuZOzKOINmWurp3smXu3y5wDcJrptwpSjgXHzIQxR0omXljQ==",
"dev": true,
"license": "MIT",
"dependencies": {
"fdir": "^6.5.0",
"picomatch": "^4.0.4"
"picomatch": "^4.0.3"
},
"engines": {
"node": ">=12.0.0"
@@ -1105,9 +1076,9 @@
"optional": true
},
"node_modules/typescript": {
"version": "6.0.3",
"resolved": "https://registry.npmjs.org/typescript/-/typescript-6.0.3.tgz",
"integrity": "sha512-y2TvuxSZPDyQakkFRPZHKFm+KKVqIisdg9/CZwm9ftvKXLP8NRWj38/ODjNbr43SsoXqNuAisEf1GdCxqWcdBw==",
"version": "6.0.2",
"resolved": "https://registry.npmjs.org/typescript/-/typescript-6.0.2.tgz",
"integrity": "sha512-bGdAIrZ0wiGDo5l8c++HWtbaNCWTS4UTv7RaTH/ThVIgjkveJt83m74bBHMJkuCbslY8ixgLBVZJIOiQlQTjfQ==",
"dev": true,
"license": "Apache-2.0",
"bin": {
@@ -1119,17 +1090,17 @@
}
},
"node_modules/vite": {
"version": "8.0.13",
"resolved": "https://registry.npmjs.org/vite/-/vite-8.0.13.tgz",
"integrity": "sha512-MFtjBYgzmSxmgA4RAfjIyXWpGe1oALnjgUTzzV7QLx/TKxCzjtMH6Fd9/eVK+5Fg1qNoz5VAwsmMs/NofrmJvw==",
"version": "8.0.5",
"resolved": "https://registry.npmjs.org/vite/-/vite-8.0.5.tgz",
"integrity": "sha512-nmu43Qvq9UopTRfMx2jOYW5l16pb3iDC1JH6yMuPkpVbzK0k+L7dfsEDH4jRgYFmsg0sTAqkojoZgzLMlwHsCQ==",
"dev": true,
"license": "MIT",
"dependencies": {
"lightningcss": "^1.32.0",
"picomatch": "^4.0.4",
"postcss": "^8.5.14",
"rolldown": "1.0.1",
"tinyglobby": "^0.2.16"
"postcss": "^8.5.8",
"rolldown": "1.0.0-rc.12",
"tinyglobby": "^0.2.15"
},
"bin": {
"vite": "bin/vite.js"
@@ -1145,7 +1116,7 @@
},
"peerDependencies": {
"@types/node": "^20.19.0 || >=22.12.0",
"@vitejs/devtools": "^0.1.18",
"@vitejs/devtools": "^0.1.0",
"esbuild": "^0.27.0 || ^0.28.0",
"jiti": ">=1.21.0",
"less": "^4.0.0",
@@ -1197,19 +1168,19 @@
}
},
"node_modules/vitest": {
"version": "4.1.6",
"resolved": "https://registry.npmjs.org/vitest/-/vitest-4.1.6.tgz",
"integrity": "sha512-6lvjbS3p9b4CrdCmguzbh2/4uoXhGE2q71R4OX5sqF9R1bo9Xd6fGrMAfvp5wnCzlBnFVdCOp6onuTQVbo8iUQ==",
"version": "4.1.2",
"resolved": "https://registry.npmjs.org/vitest/-/vitest-4.1.2.tgz",
"integrity": "sha512-xjR1dMTVHlFLh98JE3i/f/WePqJsah4A0FK9cc8Ehp9Udk0AZk6ccpIZhh1qJ/yxVWRZ+Q54ocnD8TXmkhspGg==",
"dev": true,
"license": "MIT",
"dependencies": {
"@vitest/expect": "4.1.6",
"@vitest/mocker": "4.1.6",
"@vitest/pretty-format": "4.1.6",
"@vitest/runner": "4.1.6",
"@vitest/snapshot": "4.1.6",
"@vitest/spy": "4.1.6",
"@vitest/utils": "4.1.6",
"@vitest/expect": "4.1.2",
"@vitest/mocker": "4.1.2",
"@vitest/pretty-format": "4.1.2",
"@vitest/runner": "4.1.2",
"@vitest/snapshot": "4.1.2",
"@vitest/spy": "4.1.2",
"@vitest/utils": "4.1.2",
"es-module-lexer": "^2.0.0",
"expect-type": "^1.3.0",
"magic-string": "^0.30.21",
@@ -1237,12 +1208,10 @@
"@edge-runtime/vm": "*",
"@opentelemetry/api": "^1.9.0",
"@types/node": "^20.0.0 || ^22.0.0 || >=24.0.0",
"@vitest/browser-playwright": "4.1.6",
"@vitest/browser-preview": "4.1.6",
"@vitest/browser-webdriverio": "4.1.6",
"@vitest/coverage-istanbul": "4.1.6",
"@vitest/coverage-v8": "4.1.6",
"@vitest/ui": "4.1.6",
"@vitest/browser-playwright": "4.1.2",
"@vitest/browser-preview": "4.1.2",
"@vitest/browser-webdriverio": "4.1.2",
"@vitest/ui": "4.1.2",
"happy-dom": "*",
"jsdom": "*",
"vite": "^6.0.0 || ^7.0.0 || ^8.0.0"
@@ -1266,12 +1235,6 @@
"@vitest/browser-webdriverio": {
"optional": true
},
"@vitest/coverage-istanbul": {
"optional": true
},
"@vitest/coverage-v8": {
"optional": true
},
"@vitest/ui": {
"optional": true
},
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@turnstone/sdk",
"version": "0.4.0",
"version": "0.3.0",
"description": "TypeScript client SDK for the turnstone AI orchestration platform",
"type": "module",
"main": "./dist/index.js",
+24 -77
View File
@@ -29,12 +29,6 @@ export interface ClientOptions {
export interface RequestOptions {
json?: object;
params?: Record<string, string | number>;
/**
* When set, send as multipart form-data with this body. The runtime's
* fetch sets the Content-Type + boundary itself, so we deliberately do
* not include a Content-Type header in this case.
*/
form?: FormData;
}
export class BaseClient {
@@ -53,82 +47,17 @@ export class BaseClient {
path: string,
options?: RequestOptions,
): Promise<T> {
const headers: Record<string, string> = {};
if (!options?.form) {
headers["Content-Type"] = "application/json";
}
const headers: Record<string, string> = {
"Content-Type": "application/json",
};
if (this.token) {
headers["Authorization"] = `Bearer ${this.token}`;
}
const url = this._buildUrl(path, options?.params);
let body: BodyInit | undefined;
if (options?.form) {
body = options.form;
} else if (options?.json) {
body = JSON.stringify(options.json);
}
const resp = await this.fetchFn(url, {
method,
headers,
body,
});
if (!resp.ok) {
let msg = "";
try {
const errBody = (await resp.json()) as Record<string, unknown>;
msg = (errBody.error as string) ?? (errBody.detail as string) ?? "";
} catch {
msg = await resp.text().catch(() => "");
}
throw new TurnstoneAPIError(resp.status, msg || `HTTP ${resp.status}`);
}
return (await resp.json()) as T;
}
protected async requestBytes(
method: string,
path: string,
options?: { params?: Record<string, string | number> },
): Promise<{ bytes: Uint8Array; contentType: string; filename: string }> {
const headers: Record<string, string> = {};
if (this.token) {
headers["Authorization"] = `Bearer ${this.token}`;
}
const url = this._buildUrl(path, options?.params);
const resp = await this.fetchFn(url, { method, headers });
if (!resp.ok) {
let msg = "";
try {
const errBody = (await resp.json()) as Record<string, unknown>;
msg = (errBody.error as string) ?? (errBody.detail as string) ?? "";
} catch {
msg = await resp.text().catch(() => "");
}
throw new TurnstoneAPIError(resp.status, msg || `HTTP ${resp.status}`);
}
const contentType =
resp.headers.get("content-type") ?? "application/octet-stream";
const disposition = resp.headers.get("content-disposition") ?? "";
const match = /filename="?([^";]+)"?/.exec(disposition);
const filename = match ? match[1] : "";
const buf = await resp.arrayBuffer();
return { bytes: new Uint8Array(buf), contentType, filename };
}
private _buildUrl(
path: string,
params?: Record<string, string | number>,
): string {
let url = `${this.baseUrl}${path}`;
if (params) {
if (options?.params) {
const searchParams = new URLSearchParams();
for (const [key, value] of Object.entries(params)) {
for (const [key, value] of Object.entries(options.params)) {
if (value !== undefined && value !== "") {
searchParams.set(key, String(value));
}
@@ -136,7 +65,25 @@ export class BaseClient {
const qs = searchParams.toString();
if (qs) url += `?${qs}`;
}
return url;
const resp = await this.fetchFn(url, {
method,
headers,
body: options?.json ? JSON.stringify(options.json) : undefined,
});
if (!resp.ok) {
let msg = "";
try {
const body = (await resp.json()) as Record<string, unknown>;
msg = (body.error as string) ?? (body.detail as string) ?? "";
} catch {
msg = await resp.text().catch(() => "");
}
throw new TurnstoneAPIError(resp.status, msg || `HTTP ${resp.status}`);
}
return (await resp.json()) as T;
}
protected async *streamSSE<T = Record<string, unknown>>(
-116
View File
@@ -4,8 +4,6 @@ import type {
AdminListMemoriesOptions,
AdminMemoryInfo,
AdminSearchMemoriesOptions,
AttachmentContent,
AttachmentUpload,
AuditQueryOptions,
AuditResponse,
AuthLoginResponse,
@@ -18,9 +16,6 @@ import type {
ConsoleCreateWsRequest,
ConsoleCreateWsResponse,
ConsoleHealthResponse,
CreateWorkstreamRequest,
CreateWorkstreamResponse,
ListAttachmentsResponse,
CreateMcpServerRequest,
CreatePolicyOptions,
CreateRoleOptions,
@@ -60,37 +55,12 @@ import type {
UpdateScheduleRequest,
UpdateSettingOptions,
UpdateSkillRequest,
UploadAttachmentResponse,
UsageQueryOptions,
UsageResponse,
UserRoleInfo,
WorkstreamsOptions,
} from "./types.js";
function generateConsoleWsId(): string {
// 16 bytes => 32 hex chars; matches `secrets.token_hex(16)` server-side.
const buf = new Uint8Array(16);
crypto.getRandomValues(buf);
return Array.from(buf, (b) => b.toString(16).padStart(2, "0")).join("");
}
function consoleAttachmentToBlob(att: AttachmentUpload): Blob {
if (att.data instanceof Blob) {
return att.mimeType
? new Blob([att.data], { type: att.mimeType })
: att.data;
}
// Copy bytes into a fresh ArrayBuffer-backed Uint8Array. The Blob
// BlobPart type rejects ArrayBufferLike views (could be backed by
// SharedArrayBuffer); a freshly allocated buffer is plainly ArrayBuffer.
const src = att.data;
const fresh = new Uint8Array(new ArrayBuffer(src.byteLength));
fresh.set(src);
return new Blob([fresh], {
type: att.mimeType ?? "application/octet-stream",
});
}
/** Async client for the turnstone console API. */
export class TurnstoneConsole extends BaseClient {
constructor(options: ClientOptions) {
@@ -143,92 +113,6 @@ export class TurnstoneConsole extends BaseClient {
});
}
// -- Routing proxy --------------------------------------------------------
/**
* Create a workstream via the console rendezvous router.
*
* When `attachments` is non-empty the request is sent as
* multipart/form-data and the console routes via `?ws_id=<hex>`
* (auto-generated when not supplied) so the body lands on the
* owning node directly.
*/
async routeCreateWorkstream(
opts?: CreateWorkstreamRequest & { target_node?: string },
): Promise<
CreateWorkstreamResponse & { node_url?: string; node_id?: string }
> {
const attachments = opts?.attachments;
if (attachments && attachments.length > 0) {
// The console's multipart route_create routes by `?ws_id=` only —
// it does not parse the body to honor `target_node`. Refuse the
// combination at the SDK boundary so callers don't silently get
// routed to the wrong node.
if (opts?.target_node) {
throw new Error(
"target_node is not supported with attachments; " +
"use ws_id (caller-generated to hash to the desired node) instead",
);
}
const meta: Record<string, unknown> = { ...opts };
delete (meta as { attachments?: unknown }).attachments;
let wsId = (meta.ws_id as string | undefined) ?? "";
if (!wsId) {
wsId = generateConsoleWsId();
meta.ws_id = wsId;
}
const form = new FormData();
form.append("meta", JSON.stringify(meta));
for (const att of attachments) {
form.append("file", consoleAttachmentToBlob(att), att.filename);
}
return this.request("POST", "/v1/api/route/workstreams/new", {
form,
params: { ws_id: wsId },
});
}
return this.request("POST", "/v1/api/route/workstreams/new", {
json: opts ?? {},
});
}
async routeUploadAttachment(
wsId: string,
file: AttachmentUpload,
): Promise<UploadAttachmentResponse> {
const form = new FormData();
form.append("file", consoleAttachmentToBlob(file), file.filename);
return this.request(
"POST",
`/v1/api/route/workstreams/${wsId}/attachments`,
{ form },
);
}
async routeListAttachments(wsId: string): Promise<ListAttachmentsResponse> {
return this.request("GET", `/v1/api/route/workstreams/${wsId}/attachments`);
}
async routeGetAttachmentContent(
wsId: string,
attachmentId: string,
): Promise<AttachmentContent> {
return this.requestBytes(
"GET",
`/v1/api/route/workstreams/${wsId}/attachments/${attachmentId}/content`,
);
}
async routeDeleteAttachment(
wsId: string,
attachmentId: string,
): Promise<StatusResponse> {
return this.request(
"DELETE",
`/v1/api/route/workstreams/${wsId}/attachments/${attachmentId}`,
);
}
// -- Streaming ------------------------------------------------------------
async *clusterEvents(): AsyncIterableIterator<ClusterEvent> {
-39
View File
@@ -13,20 +13,6 @@ export interface ConnectedEvent {
export interface HistoryEvent {
type: "history";
/**
* Per-message dicts the frontend consumes directly. Common optional keys:
* - `role`: "user" | "assistant" | "tool"
* - `content`: string or list (image/document parts)
* - `tool_calls`: assistant turns list of `{id, name, arguments, verdict?, output_assessment?}`
* - `tool_call_id`: tool turns id of the originating call
* - `reminders`: metacognitive nudge bubbles (user/tool channels)
* - `advisories`: extracted `UserInterjection` payloads on tool turns
* - `reasoning`: concatenated reasoning text for assistant turns whose
* `provider_data` carried reasoning-bearing blocks (Anthropic
* `thinking`, OpenAI Responses `reasoning`, or synthetic
* `reasoning_text` from path-3 servers). Present only when the
* active model's `surface_persisted_reasoning` flag is true.
*/
messages: Array<Record<string, unknown>>;
}
@@ -52,14 +38,6 @@ export interface StreamEndEvent {
type: "stream_end";
}
/** One-shot replay of the in-progress turn's content + reasoning emitted
* by the events SSE handler when a fresh subscriber connects mid-stream. */
export interface InProgressSnapshotEvent {
type: "in_progress_snapshot";
content: string;
reasoning: string;
}
export interface StateChangeEvent {
type: "state_change";
state: "idle" | "thinking" | "running" | "attention" | "error";
@@ -114,11 +92,6 @@ export interface PlanReviewEvent {
content: string;
}
export interface PlanResolvedEvent {
type: "plan_resolved";
feedback: string;
}
export interface InfoEvent {
type: "info";
message: string;
@@ -184,7 +157,6 @@ export type ServerEvent =
| ContentEvent
| ReasoningEvent
| StreamEndEvent
| InProgressSnapshotEvent
| StateChangeEvent
| ToolInfoEvent
| ApproveRequestEvent
@@ -193,7 +165,6 @@ export type ServerEvent =
| ToolOutputChunkEvent
| StatusEvent
| PlanReviewEvent
| PlanResolvedEvent
| InfoEvent
| ErrorEvent
| BusyErrorEvent
@@ -284,12 +255,6 @@ export function isStreamEndEvent(e: ServerEvent): e is StreamEndEvent {
return e.type === "stream_end";
}
export function isInProgressSnapshotEvent(
e: ServerEvent,
): e is InProgressSnapshotEvent {
return e.type === "in_progress_snapshot";
}
export function isStateChangeEvent(e: ServerEvent): e is StateChangeEvent {
return e.type === "state_change";
}
@@ -318,10 +283,6 @@ export function isPlanReviewEvent(e: ServerEvent): e is PlanReviewEvent {
return e.type === "plan_review";
}
export function isPlanResolvedEvent(e: ServerEvent): e is PlanResolvedEvent {
return e.type === "plan_resolved";
}
export function isCancelledEvent(e: ServerEvent): e is CancelledEvent {
return e.type === "cancelled";
}
-6
View File
@@ -183,12 +183,6 @@ export type {
SkillInstallRequest,
SkillInstallResponse,
SkillInstallSkipped,
// Attachment types
AttachmentUpload,
AttachmentInfo,
UploadAttachmentResponse,
ListAttachmentsResponse,
AttachmentContent,
} from "./types.js";
// SSE parser (for advanced usage)
+21 -132
View File
@@ -1,8 +1,6 @@
import { BaseClient, type ClientOptions } from "./base.js";
import type { ServerEvent } from "./events.js";
import type {
AttachmentContent,
AttachmentUpload,
AuthLoginResponse,
AuthSetupResponse,
AuthStatusResponse,
@@ -11,46 +9,20 @@ import type {
DashboardResponse,
DeleteMemoryOptions,
HealthResponse,
ListAttachmentsResponse,
ListMemoriesOptions,
ListMemoriesResponse,
ListSavedWorkstreamsResponse,
SkillSummary,
ListWorkstreamsResponse,
MemoryInfo,
SaveMemoryRequest,
SearchMemoriesRequest,
SendAndWaitOptions,
SendResponse,
SkillSummary,
StatusResponse,
TurnResult,
UploadAttachmentResponse,
} from "./types.js";
function generateWsId(): string {
// 16 bytes => 32 hex chars; matches `secrets.token_hex(16)` server-side.
const buf = new Uint8Array(16);
crypto.getRandomValues(buf);
return Array.from(buf, (b) => b.toString(16).padStart(2, "0")).join("");
}
function attachmentToBlob(att: AttachmentUpload): Blob {
if (att.data instanceof Blob) {
return att.mimeType
? new Blob([att.data], { type: att.mimeType })
: att.data;
}
// Copy bytes into a fresh ArrayBuffer-backed Uint8Array. The Blob
// BlobPart type rejects ArrayBufferLike views (could be backed by
// SharedArrayBuffer); a freshly allocated buffer is plainly ArrayBuffer.
const src = att.data;
const fresh = new Uint8Array(new ArrayBuffer(src.byteLength));
fresh.set(src);
return new Blob([fresh], {
type: att.mimeType ?? "application/octet-stream",
});
}
/** Async client for the turnstone server API. */
export class TurnstoneServer extends BaseClient {
constructor(options: ClientOptions) {
@@ -70,114 +42,37 @@ export class TurnstoneServer extends BaseClient {
async createWorkstream(
opts?: CreateWorkstreamRequest,
): Promise<CreateWorkstreamResponse> {
const attachments = opts?.attachments;
if (attachments && attachments.length > 0) {
// Multipart variant: pre-generate ws_id so cluster routers can
// hash to the owning node before this body lands. Server accepts
// either a server-generated id (when meta.ws_id is empty) or the
// caller-supplied one.
const meta: Record<string, unknown> = { ...opts };
delete (meta as { attachments?: unknown }).attachments;
if (!meta.ws_id) {
meta.ws_id = generateWsId();
}
const form = new FormData();
form.append("meta", JSON.stringify(meta));
for (const att of attachments) {
form.append("file", attachmentToBlob(att), att.filename);
}
return this.request("POST", "/v1/api/workstreams/new", { form });
}
return this.request("POST", "/v1/api/workstreams/new", {
json: opts ?? {},
});
return this.request("POST", "/v1/api/workstreams/new", { json: opts });
}
async closeWorkstream(
wsId: string,
opts?: { reason?: string },
): Promise<StatusResponse> {
const body: Record<string, unknown> = {};
if (opts?.reason !== undefined) body.reason = opts.reason;
return this.request(
"POST",
`/v1/api/workstreams/${encodeURIComponent(wsId)}/close`,
{ json: body },
);
async closeWorkstream(wsId: string): Promise<StatusResponse> {
return this.request("POST", "/v1/api/workstreams/close", {
json: { ws_id: wsId },
});
}
// -- Chat interaction -----------------------------------------------------
async send(
message: string,
wsId: string,
opts?: { attachmentIds?: string[] },
): Promise<SendResponse> {
const body: Record<string, unknown> = { message };
if (opts?.attachmentIds !== undefined) {
body.attachment_ids = opts.attachmentIds;
}
return this.request(
"POST",
`/v1/api/workstreams/${encodeURIComponent(wsId)}/send`,
{ json: body },
);
}
// -- Attachments ----------------------------------------------------------
async uploadAttachment(
wsId: string,
file: AttachmentUpload,
): Promise<UploadAttachmentResponse> {
const form = new FormData();
form.append("file", attachmentToBlob(file), file.filename);
return this.request("POST", `/v1/api/workstreams/${wsId}/attachments`, {
form,
async send(message: string, wsId: string): Promise<SendResponse> {
return this.request("POST", "/v1/api/send", {
json: { message, ws_id: wsId },
});
}
async listAttachments(wsId: string): Promise<ListAttachmentsResponse> {
return this.request("GET", `/v1/api/workstreams/${wsId}/attachments`);
}
async getAttachmentContent(
wsId: string,
attachmentId: string,
): Promise<AttachmentContent> {
return this.requestBytes(
"GET",
`/v1/api/workstreams/${wsId}/attachments/${attachmentId}/content`,
);
}
async deleteAttachment(
wsId: string,
attachmentId: string,
): Promise<StatusResponse> {
return this.request(
"DELETE",
`/v1/api/workstreams/${wsId}/attachments/${attachmentId}`,
);
}
async approve(opts: {
wsId: string;
approved?: boolean;
feedback?: string | null;
always?: boolean;
}): Promise<StatusResponse> {
return this.request(
"POST",
`/v1/api/workstreams/${encodeURIComponent(opts.wsId)}/approve`,
{
json: {
approved: opts.approved ?? true,
feedback: opts.feedback,
always: opts.always,
},
return this.request("POST", "/v1/api/approve", {
json: {
ws_id: opts.wsId,
approved: opts.approved ?? true,
feedback: opts.feedback,
always: opts.always,
},
);
});
}
async planFeedback(opts: {
@@ -202,21 +97,15 @@ export class TurnstoneServer extends BaseClient {
wsId: string,
opts?: { force?: boolean },
): Promise<StatusResponse> {
const body: Record<string, unknown> = {};
const body: Record<string, unknown> = { ws_id: wsId };
if (opts?.force) body.force = true;
return this.request(
"POST",
`/v1/api/workstreams/${encodeURIComponent(wsId)}/cancel`,
{ json: body },
);
return this.request("POST", "/v1/api/cancel", { json: body });
}
// -- Streaming ------------------------------------------------------------
async *streamEvents(wsId: string): AsyncIterableIterator<ServerEvent> {
yield* this.streamSSE<ServerEvent>(
`/v1/api/workstreams/${encodeURIComponent(wsId)}/events`,
);
yield* this.streamSSE<ServerEvent>("/v1/api/events", { ws_id: wsId });
}
async *streamGlobalEvents(): AsyncIterableIterator<ServerEvent> {
@@ -256,8 +145,8 @@ export class TurnstoneServer extends BaseClient {
try {
// Start consuming the per-workstream SSE stream first
const events = this.streamSSE<ServerEvent>(
`/v1/api/workstreams/${encodeURIComponent(wsId)}/events`,
undefined,
"/v1/api/events",
{ ws_id: wsId },
controller.signal,
);
+4 -107
View File
@@ -50,67 +50,10 @@ export interface AuthSetupResponse {
export interface SendRequest {
message: string;
ws_id: string;
/**
* Explicit list of pending attachment ids to inject into this turn.
* When omitted, any pending attachments for the caller on the
* workstream are auto-consumed; an empty list disables auto-consume.
*/
attachment_ids?: string[];
}
export interface SendResponse {
/** "ok" | "busy" | "queued" | "queue_full". */
status: string;
/**
* Attachment ids actually reserved onto this turn. Subset of the
* request's `attachment_ids` (or the auto-consumed pending set).
*/
attached_ids?: string[];
/**
* Attachment ids the caller requested that the server could not
* reserve (lost a race, already consumed, or cross-scope). The
* request still proceeds with whatever was reserved.
*/
dropped_attachment_ids?: string[];
/** Set on "queued" responses: relative priority of the queued message. */
priority?: string | null;
/** Set on "queued" responses: id used to dequeue the message. */
msg_id?: string | null;
}
// ---------------------------------------------------------------------------
// Server API — Attachments
// ---------------------------------------------------------------------------
/** A file to upload as an attachment. */
export interface AttachmentUpload {
filename: string;
/** Raw file bytes; use a `Blob` in browsers and a `Uint8Array` in Node. */
data: Blob | Uint8Array;
/** Optional advisory MIME type; the server applies its own validation. */
mimeType?: string;
}
export interface AttachmentInfo {
attachment_id: string;
filename: string;
mime_type: string;
size_bytes: number;
/** "image" or "text". */
kind: string;
}
export type UploadAttachmentResponse = AttachmentInfo;
export interface ListAttachmentsResponse {
attachments: AttachmentInfo[];
}
/** Raw bytes returned from the attachment `/content` endpoint. */
export interface AttachmentContent {
bytes: Uint8Array;
contentType: string;
filename: string;
}
export interface ApproveRequest {
@@ -136,20 +79,6 @@ export interface CreateWorkstreamRequest {
auto_approve?: boolean;
resume_ws?: string;
skill?: string;
/** First user message dispatched in a background worker after creation. */
initial_message?: string;
/**
* Caller-supplied workstream id (32-hex). Auto-generated when omitted.
* Required for cluster-routed multipart creates so the console can
* hash to the owning node before the body lands.
*/
ws_id?: string;
/**
* Files to attach to the first turn. When non-empty the request is
* sent as multipart/form-data and (with `initial_message`) reserved
* onto that turn before the worker dispatches.
*/
attachments?: AttachmentUpload[];
}
export interface CreateWorkstreamResponse {
@@ -157,56 +86,24 @@ export interface CreateWorkstreamResponse {
name: string;
resumed?: boolean;
message_count?: number;
/** Ids of attachments saved by this request (multipart variant only). */
attachment_ids?: string[];
}
export interface CloseWorkstreamRequest {
/**
* Optional close reason persisted to `workstream_config` for
* postmortem. Capped at 512 UTF-8 bytes server-side; credential
* redaction is applied via the output guard.
*/
reason?: string;
ws_id: string;
}
export interface WorkstreamInfo {
// Renamed `id` → `ws_id` and added kind/parent_ws_id/user_id in
// the Stage 2 list-verb lift. Pre-1.5 readers branching on
// `row.id` should swap to `row.ws_id`.
ws_id: string;
id: string;
name: string;
state: string;
kind: string;
parent_ws_id: string | null;
user_id: string;
}
export interface ListWorkstreamsResponse {
workstreams: WorkstreamInfo[];
}
export interface WorkstreamDetailResponse {
// Lifted from coord-only into a shared verb in the Stage 2
// history/detail verb lift. Both kinds populate every field; SDK
// consumers don't branch on kind.
ws_id: string;
name: string;
state: string;
user_id: string;
kind: string;
}
export interface WorkstreamHistoryResponse {
ws_id: string;
// Tail of the workstream's reconstructed message history
// (provider-fidelity OpenAI-like shape). Bounded by the ?limit=
// query param (default 100, max 500).
messages: Record<string, unknown>[];
}
export interface DashboardWorkstream {
ws_id: string;
id: string;
name: string;
state: string;
title?: string;
@@ -982,7 +879,7 @@ export interface SkillDiscoverListing {
install_count: number;
tags: string[];
installed: boolean;
risk_level?: string;
scan_status?: string;
template_id?: string;
}
-22
View File
@@ -62,28 +62,6 @@ describe("TurnstoneConsole", () => {
expect(url).toContain("page=2");
});
it("routeCreateWorkstream rejects attachments + target_node", async () => {
const fetchFn = vi.fn().mockResolvedValue(
new Response("{}", {
status: 500,
headers: { "content-type": "application/json" },
}),
);
const client = new TurnstoneConsole({
baseUrl: "http://test",
fetch: fetchFn,
});
const data = new TextEncoder().encode("hi");
await expect(
client.routeCreateWorkstream({
name: "x",
target_node: "n1",
attachments: [{ filename: "a.txt", data }],
}),
).rejects.toThrow(/target_node/);
expect(fetchFn).not.toHaveBeenCalled();
});
it("health returns parsed response", async () => {
const fetchFn = mockFetch({
status: "ok",
-6
View File
@@ -8,7 +8,6 @@ import {
isApproveRequestEvent,
isApprovalResolvedEvent,
isPlanReviewEvent,
isPlanResolvedEvent,
isReasoningEvent,
} from "../src/events.js";
import type { ServerEvent } from "../src/events.js";
@@ -77,9 +76,4 @@ describe("event type guards", () => {
const e: ServerEvent = { type: "plan_review", content: "## Plan" };
expect(isPlanReviewEvent(e)).toBe(true);
});
it("isPlanResolvedEvent", () => {
const e: ServerEvent = { type: "plan_resolved", feedback: "approved" };
expect(isPlanResolvedEvent(e)).toBe(true);
});
});
@@ -1,168 +0,0 @@
import { describe, expect, it, vi } from "vitest";
import { TurnstoneServer } from "../src/server.js";
function mockFetch(response: object, status = 200): typeof globalThis.fetch {
return vi.fn().mockResolvedValue(
new Response(JSON.stringify(response), {
status,
headers: { "content-type": "application/json" },
}),
);
}
function mockFetchBytes(
body: Uint8Array,
contentType: string,
filename = "",
): typeof globalThis.fetch {
const headers: Record<string, string> = { "content-type": contentType };
if (filename)
headers["content-disposition"] = `inline; filename="${filename}"`;
return vi
.fn()
.mockResolvedValue(new Response(body, { status: 200, headers }));
}
describe("TurnstoneServer attachments", () => {
it("uploadAttachment sends multipart with filename", async () => {
const fetchFn = mockFetch({
attachment_id: "att-1",
filename: "a.txt",
mime_type: "text/plain",
size_bytes: 5,
kind: "text",
});
const client = new TurnstoneServer({
baseUrl: "http://test",
fetch: fetchFn,
});
const data = new TextEncoder().encode("hello");
const result = await client.uploadAttachment("ws-X", {
filename: "a.txt",
data,
mimeType: "text/plain",
});
expect(result.attachment_id).toBe("att-1");
const [url, init] = (fetchFn as ReturnType<typeof vi.fn>).mock.calls[0];
expect(url).toBe("http://test/v1/api/workstreams/ws-X/attachments");
expect(init.method).toBe("POST");
expect(init.body).toBeInstanceOf(FormData);
// Browser/Node fetch sets the Content-Type header from FormData itself
expect(init.headers["Content-Type"]).toBeUndefined();
});
it("listAttachments hits the GET endpoint", async () => {
const fetchFn = mockFetch({ attachments: [] });
const client = new TurnstoneServer({
baseUrl: "http://test",
fetch: fetchFn,
});
const resp = await client.listAttachments("ws-X");
expect(resp.attachments).toEqual([]);
const [url, init] = (fetchFn as ReturnType<typeof vi.fn>).mock.calls[0];
expect(url).toBe("http://test/v1/api/workstreams/ws-X/attachments");
expect(init.method).toBe("GET");
});
it("getAttachmentContent returns raw bytes + parsed headers", async () => {
const bytes = new TextEncoder().encode("hello world");
const fetchFn = mockFetchBytes(
bytes,
"text/plain; charset=utf-8",
"notes.md",
);
const client = new TurnstoneServer({
baseUrl: "http://test",
fetch: fetchFn,
});
const result = await client.getAttachmentContent("ws-X", "att-1");
expect(new TextDecoder().decode(result.bytes)).toBe("hello world");
expect(result.contentType).toBe("text/plain; charset=utf-8");
expect(result.filename).toBe("notes.md");
});
it("deleteAttachment hits the DELETE endpoint", async () => {
const fetchFn = mockFetch({ status: "deleted" });
const client = new TurnstoneServer({
baseUrl: "http://test",
fetch: fetchFn,
});
const resp = await client.deleteAttachment("ws-X", "att-1");
expect(resp.status).toBe("deleted");
const [, init] = (fetchFn as ReturnType<typeof vi.fn>).mock.calls[0];
expect(init.method).toBe("DELETE");
});
it("send threads attachment_ids when provided", async () => {
const fetchFn = mockFetch({ status: "ok" });
const client = new TurnstoneServer({
baseUrl: "http://test",
fetch: fetchFn,
});
await client.send("hi", "ws-X", { attachmentIds: ["a1", "a2"] });
const [, init] = (fetchFn as ReturnType<typeof vi.fn>).mock.calls[0];
expect(JSON.parse(init.body)).toEqual({
message: "hi",
ws_id: "ws-X",
attachment_ids: ["a1", "a2"],
});
});
it("send omits attachment_ids when not supplied", async () => {
const fetchFn = mockFetch({ status: "ok" });
const client = new TurnstoneServer({
baseUrl: "http://test",
fetch: fetchFn,
});
await client.send("hi", "ws-X");
const [, init] = (fetchFn as ReturnType<typeof vi.fn>).mock.calls[0];
expect(JSON.parse(init.body)).toEqual({ message: "hi", ws_id: "ws-X" });
});
it("createWorkstream with attachments sends multipart and auto-generates ws_id", async () => {
const fetchFn = mockFetch({
ws_id: "00ff00000000000000000000000000ff",
name: "demo",
attachment_ids: ["att-1"],
});
const client = new TurnstoneServer({
baseUrl: "http://test",
fetch: fetchFn,
});
const data = new TextEncoder().encode("hello");
const resp = await client.createWorkstream({
name: "demo",
initial_message: "describe",
attachments: [{ filename: "a.txt", data, mimeType: "text/plain" }],
});
expect(resp.attachment_ids).toEqual(["att-1"]);
const [url, init] = (fetchFn as ReturnType<typeof vi.fn>).mock.calls[0];
expect(url).toBe("http://test/v1/api/workstreams/new");
expect(init.method).toBe("POST");
expect(init.body).toBeInstanceOf(FormData);
const form = init.body as FormData;
const meta = JSON.parse(form.get("meta") as string);
expect(meta.name).toBe("demo");
expect(meta.initial_message).toBe("describe");
expect(meta.ws_id).toMatch(/^[0-9a-f]{32}$/);
expect(meta.attachments).toBeUndefined();
const file = form.get("file");
expect(file).toBeInstanceOf(Blob);
});
it("createWorkstream without attachments uses JSON body", async () => {
const fetchFn = mockFetch({ ws_id: "ws-json", name: "j" });
const client = new TurnstoneServer({
baseUrl: "http://test",
fetch: fetchFn,
});
await client.createWorkstream({ name: "j" });
const [, init] = (fetchFn as ReturnType<typeof vi.fn>).mock.calls[0];
expect(init.headers["Content-Type"]).toBe("application/json");
expect(JSON.parse(init.body)).toEqual({ name: "j" });
});
});
+2 -13
View File
@@ -26,16 +26,7 @@ function mockFetchError(
describe("TurnstoneServer", () => {
it("listWorkstreams returns parsed response", async () => {
const fetchFn = mockFetch({
workstreams: [
{
ws_id: "ws1",
name: "test",
state: "idle",
kind: "interactive",
parent_ws_id: null,
user_id: "u1",
},
],
workstreams: [{ id: "ws1", name: "test", state: "idle" }],
});
const client = new TurnstoneServer({
baseUrl: "http://test",
@@ -43,9 +34,7 @@ describe("TurnstoneServer", () => {
});
const resp = await client.listWorkstreams();
expect(resp.workstreams).toHaveLength(1);
// Row key renamed id → ws_id in the Stage 2 list-verb lift.
expect(resp.workstreams[0].ws_id).toBe("ws1");
expect(resp.workstreams[0].kind).toBe("interactive");
expect(resp.workstreams[0].id).toBe("ws1");
expect(fetchFn).toHaveBeenCalledWith(
"http://test/v1/api/workstreams",
expect.objectContaining({ method: "GET" }),
-129
View File
@@ -1,129 +0,0 @@
"""Shared builders for the coordinator-endpoint test files.
The four coordinator test modules each ship a copy of the same
``_AuthMiddleware`` / ``_FakeConfigStore`` / ``_fake_registry`` /
``_build_mgr`` helpers this module is the single home for them so
future edits land once. Named with a leading underscore so pytest
does not collect it.
``_make_client`` stays local to each test module because the route
list differs per file.
"""
from __future__ import annotations
from typing import TYPE_CHECKING, Any
from unittest.mock import MagicMock
from starlette.middleware.base import BaseHTTPMiddleware
from turnstone.console.collector import ClusterCollector
from turnstone.console.coordinator_adapter import CoordinatorAdapter
from turnstone.console.coordinator_ui import ConsoleCoordinatorUI
from turnstone.core.auth import AuthResult
from turnstone.core.session_manager import SessionManager
if TYPE_CHECKING:
from collections.abc import Iterable
def _seed_children(
adapter: CoordinatorAdapter, coord_ws_id: str, child_ws_ids: Iterable[str]
) -> None:
"""Seed the coordinator adapter's children registry directly.
The production path populates the registry via the cluster-event
fan-out thread observing ``ws_created`` events. These tests just
need a known-children set for the endpoint handlers to iterate
inject directly via the registry's bulk-merge surface rather than
spinning up the collector + fan-out plumbing.
"""
adapter._registry.merge_children(coord_ws_id, child_ws_ids)
class _AuthMiddleware(BaseHTTPMiddleware):
"""Inject a configurable AuthResult from a header-based contract.
Tests set ``X-Test-Perms`` to a comma-separated permission list, and
``X-Test-User`` to the user id. Empty or missing no auth.
"""
async def dispatch(self, request, call_next): # type: ignore[no-untyped-def]
perms = request.headers.get("X-Test-Perms", "")
user_id = request.headers.get("X-Test-User", "")
if perms or user_id:
request.state.auth_result = AuthResult(
user_id=user_id,
scopes=frozenset({"approve"}),
token_source="test",
permissions=frozenset(p for p in perms.split(",") if p),
)
return await call_next(request)
class _FakeConfigStore:
"""Minimal ConfigStore stub — returns values from a dict."""
def __init__(self, values: dict[str, Any]) -> None:
self._values = values
def get(self, key: str, default: Any = None) -> Any:
return self._values.get(key, default)
def _fake_registry() -> MagicMock:
"""MagicMock whose ``.resolve()`` succeeds so the 503 gate passes."""
reg = MagicMock()
reg.resolve.return_value = (MagicMock(), "gpt-4", MagicMock())
return reg
def _build_mgr_with_factory(storage: Any, session_factory: Any) -> SessionManager:
"""Build a SessionManager(CoordinatorAdapter) with a caller-supplied factory.
Used by tests that need to capture or assert factory kwargs (e.g.
per-call ``model`` / ``judge_model`` overrides). Plain :func:`_build_mgr`
is the right entry point when the test doesn't care about the
factory.
"""
adapter = CoordinatorAdapter(
collector=MagicMock(),
ui_factory=lambda ws: ConsoleCoordinatorUI(ws_id=ws.id, user_id=ws.user_id or ""),
session_factory=session_factory,
)
mgr = SessionManager(
adapter,
storage=storage,
max_active=3,
node_id=ClusterCollector.CONSOLE_PSEUDO_NODE_ID,
event_emitter=adapter,
)
adapter.attach(mgr)
return mgr
def _build_mgr(storage: Any) -> SessionManager:
"""Build a SessionManager(CoordinatorAdapter) with stub factories (test default)."""
def _sf(ui, model_alias=None, ws_id=None, **kw): # type: ignore[no-untyped-def]
s = MagicMock()
s.send.return_value = None
return s
return _build_mgr_with_factory(storage, _sf)
class MockStorage:
"""Minimal storage mock that implements ``list_services``.
Used by the collector tests + the console route-walk tests. The
collector calls ``list_services("turnstone-server", ...)`` to
discover nodes; tests that don't care about discovery push an
empty list (the default).
"""
def __init__(self) -> None:
self.services: list[dict[str, str]] = []
def list_services(self, service_type: str, max_age_seconds: int = 120) -> list[dict[str, str]]:
return list(self.services)
-53
View File
@@ -1,53 +0,0 @@
"""Shared test helpers — kept out of conftest.py since these are factories,
not fixtures, and several test files want to import them directly."""
from __future__ import annotations
from typing import Any
from unittest.mock import MagicMock
def make_chat_session(**overrides: Any) -> Any:
"""Build a minimal ``ChatSession`` with sane test defaults.
Caller passes any constructor arg as a kwarg to override the default
e.g. ``make_chat_session(memory_config=MemoryConfig(fetch_limit=5))``.
"""
from turnstone.core.session import ChatSession
defaults: dict[str, Any] = {
"client": MagicMock(),
"model": "test-model",
"ui": MagicMock(),
"instructions": None,
"temperature": 0.5,
"max_tokens": 4096,
"tool_timeout": 30,
}
defaults.update(overrides)
return ChatSession(**defaults)
def patch_session_storage(
monkeypatch: Any,
*,
active: bool = True,
raise_on_is_active: bool = False,
) -> list[str]:
"""Patch ``session.get_storage`` to a stub whose ``is_watch_active``
returns *active* (or raises if *raise_on_is_active*). Returns the
list of ``watch_id``s the predicate was called with.
"""
from turnstone.core import session as session_mod
calls: list[str] = []
class _Stub:
def is_watch_active(self, watch_id: str) -> bool:
calls.append(watch_id)
if raise_on_is_active:
raise RuntimeError("storage down")
return active
monkeypatch.setattr(session_mod, "get_storage", lambda: _Stub())
return calls
-58
View File
@@ -1,58 +0,0 @@
"""Shared mock factory for ``events_replay`` tests.
Both interactive (:func:`turnstone.server._interactive_events_replay`)
and coord (:func:`turnstone.console.server._coord_events_replay`) drive
the same shared preamble at
:func:`turnstone.core.session_replay.session_replay_preamble`. Their
test suites share the underlying mock surface (session.model,
session.model_alias, session._last_usage, ui._pending_*, ui._ws_lock,
counters); this module is the single home for that shape so a future
field add lands once.
"""
from __future__ import annotations
import threading
from typing import Any
from unittest.mock import MagicMock
def make_replay_mocks(
*,
last_usage: dict[str, Any] | None = None,
**ui_overrides: Any,
) -> tuple[Any, Any, Any]:
"""Build ``(ws, ui, request)`` MagicMocks for events-replay tests.
Defaults match a fresh workstream that hasn't completed a turn
(no ``last_usage``, no pending prompts).
Args:
last_usage: Sets ``ws.session._last_usage`` directly so tests
don't have to reach into the nested mock; when ``None``
(default), the status replay branch stays inert.
**ui_overrides: Additional attributes set directly on the ``ui``
mock (e.g. ``_pending_approval``, ``_pending_plan_review``,
``_llm_verdicts``, ``_ws_turn_tool_calls``, ``_ws_messages``).
"""
session = MagicMock()
session.model = "gpt-5"
session.model_alias = "default"
session._last_usage = last_usage
session.context_window = 100000
session.reasoning_effort = "medium"
session.messages = []
ui = MagicMock()
ui.auto_approve = False
ui._pending_approval = None
ui._pending_plan_review = None
ui._llm_verdicts = {}
ui._ws_lock = threading.Lock()
ui._ws_turn_tool_calls = 0
ui._ws_messages = 0
for key, value in ui_overrides.items():
setattr(ui, key, value)
ws = MagicMock()
ws.session = session
request = MagicMock()
return ws, ui, request
-45
View File
@@ -1,45 +0,0 @@
"""Shared session-test helpers.
Two reasoning-test modules (``test_session_replay_reasoning.py`` and
``test_session_synth_reasoning_block.py``) need the same minimal
``ChatSession`` factory + a ``SessionUIBase`` no-op subclass. Hoisting
keeps a future third caller from drifting on the defaults the third
existing ``_make_session`` (``test_model_registry.py``) deliberately
takes a different signature (registry / model_alias / reasoning_effort
+ ``_FakeUI``) and is NOT a candidate for sharing this helper.
Module is named with a leading underscore so pytest doesn't try to
collect it as a test file it's an importable utility, not a test.
"""
from __future__ import annotations
from typing import Any
from unittest.mock import MagicMock
from turnstone.core.session import ChatSession
from turnstone.core.session_ui_base import SessionUIBase
class NullUI(SessionUIBase):
"""Bare-bones UI satisfying the SessionUIBase contract for tests
that don't care about UI side effects."""
def __init__(self) -> None:
super().__init__()
def make_session(**kwargs: Any) -> ChatSession:
"""Build a ChatSession with minimal defaults; tests override
individual fields via kwargs."""
defaults: dict[str, Any] = {
"client": MagicMock(),
"model": "test-model",
"ui": NullUI(),
"instructions": None,
"temperature": 0.5,
"max_tokens": 4096,
"tool_timeout": 30,
}
defaults.update(kwargs)
return ChatSession(**defaults)
-87
View File
@@ -1,79 +1,10 @@
from __future__ import annotations
import os
from typing import TYPE_CHECKING, Any
from unittest.mock import MagicMock
import pytest
if TYPE_CHECKING:
from turnstone.core.mcp_client import MCPClientManager, StaticServerState
from turnstone.core.mcp_crypto import MCPTokenCipher
from turnstone.core.oidc import OIDCConfig
def make_mcp_token_cipher() -> MCPTokenCipher:
"""Build a single-key MCP token cipher for tests.
Used by test files that need to exercise ``MCPTokenStore`` round-
trips without the lifespan-side configuration loader; centralised
here so the key/material defaults stay aligned across files.
"""
import base64
from cryptography.fernet import Fernet
from turnstone.core.mcp_crypto import MCPTokenCipher, MCPTokenCipherConfig
raw = base64.urlsafe_b64decode(Fernet.generate_key())
return MCPTokenCipher(MCPTokenCipherConfig(keys=(raw,)))
def _seed_static_state(mgr: MCPClientManager, name: str, **overrides: Any) -> StaticServerState:
"""Get-or-create a ``StaticServerState`` on ``mgr`` and apply ``overrides``.
Shared across MCP test files so the helper stays in one place. Imported
where needed; ``StaticServerState`` is constructed lazily so non-MCP
tests don't pay the import cost.
"""
from turnstone.core.mcp_client import StaticServerState
state = mgr._static_servers.get(name)
if state is None:
state = StaticServerState(name=name)
mgr._static_servers[name] = state
for k, v in overrides.items():
setattr(state, k, v)
return state
def make_oidc_test_config(**overrides: Any) -> OIDCConfig:
"""Build a test ``OIDCConfig`` with sensible defaults.
Shared between ``test_oidc.py`` and ``test_oidc_handlers.py`` so the
defaults (including the now-required ``redirect_base``) stay aligned.
"""
from turnstone.core.oidc import OIDCConfig
defaults: dict[str, Any] = {
"enabled": True,
"issuer": "https://idp.example.com",
"client_id": "my-client",
"client_secret": "my-secret",
"scopes": "openid email profile",
"provider_name": "TestIDP",
"role_claim": "",
"role_map": {},
"password_enabled": True,
"redirect_base": "https://app.example.com",
"authorization_endpoint": "https://idp.example.com/authorize",
"token_endpoint": "https://idp.example.com/token",
"userinfo_endpoint": "https://idp.example.com/userinfo",
"jwks_uri": "https://idp.example.com/.well-known/jwks.json",
}
defaults.update(overrides)
return OIDCConfig(**defaults)
def pytest_addoption(parser: pytest.Parser) -> None:
parser.addoption(
@@ -164,21 +95,3 @@ def mock_openai_client():
client = MagicMock()
client.models.list.return_value.data = [MagicMock(id="test-model")]
return client
@pytest.fixture(autouse=True)
def _clear_policy_cache():
"""Drop the in-process tool-policy cache between tests.
The cache is keyed by org_id (default ``""``), so without this
autouse hook a policy created in test A would leak into test B's
``evaluate_tool_policy`` call distinct storage instances, same
cache slot. Production singleton storage doesn't see the leak
because there's only one storage instance for the process lifetime;
the test isolation requirement is what motivates the autouse.
"""
from turnstone.core.policy import invalidate_policy_cache
invalidate_policy_cache()
yield
invalidate_policy_cache()
-391
View File
@@ -1,391 +0,0 @@
"""Spike 1 — validate MCP SDK behavior for the per-(user, server) session pool.
Three scenarios:
1. N=20 concurrent ClientSession instances to the same URL.
Verifies: no FD blow-up, no shared transport state, each session's
tools/list returns independently.
2. Two concurrent tools/call on a shared ClientSession with interleaving
payloads. Verifies: request_id demux works under contention.
3. Per-session Authorization header isolation. Verifies: different Bearer
tokens per ClientSession reach the server with the expected
Authorization header i.e. httpx connection pooling does not cross
headers between sessions.
Run: uv run python tests/spike_sdk_concurrency.py
Outcome gates Phase 5's pool architecture; if any scenario fails, fall
back to per-call header injection (Alternative F in the OAuth-MCP RFC).
"""
from __future__ import annotations
import asyncio
import contextlib
import logging
import os
import socket
import sys
import threading
import time
from collections import defaultdict
from typing import TYPE_CHECKING
import uvicorn
from mcp import ClientSession
from mcp.client.streamable_http import streamablehttp_client
from mcp.server.fastmcp import FastMCP
from starlette.middleware.base import BaseHTTPMiddleware
if TYPE_CHECKING:
from collections.abc import Callable
from starlette.requests import Request
from starlette.responses import Response
# Reduce uvicorn / mcp log noise so spike output is readable.
logging.getLogger("uvicorn.error").setLevel(logging.WARNING)
logging.getLogger("uvicorn.access").setLevel(logging.WARNING)
logging.getLogger("mcp").setLevel(logging.WARNING)
# Records (auth_header, tool_name) per request — populated by the
# AuthHeaderRecorder middleware below. Indexed by call sequence.
SERVER_OBSERVATIONS: list[tuple[str | None, str | None]] = []
# Tool-call payloads observed (for request_id demux verification).
TOOL_CALL_PAYLOADS: list[str] = []
class AuthHeaderRecorder(BaseHTTPMiddleware):
"""Records the Authorization header on every request the server sees."""
async def dispatch(self, request: Request, call_next: Callable) -> Response:
auth = request.headers.get("authorization")
# We only record the auth header here; tool name comes from the
# body payload which we can't read non-destructively. The tool
# handler logs the payload it received.
SERVER_OBSERVATIONS.append((auth, None))
return await call_next(request)
def find_free_port() -> int:
"""Bind to port 0, return the assigned port."""
s = socket.socket()
s.bind(("127.0.0.1", 0))
port = s.getsockname()[1]
s.close()
return port
def build_server(port: int) -> uvicorn.Server:
"""Create a minimal FastMCP server with one echo tool."""
mcp = FastMCP(name="spike-target", streamable_http_path="/mcp")
@mcp.tool()
async def echo(payload: str) -> str:
"""Echo the payload back. Records the payload server-side."""
TOOL_CALL_PAYLOADS.append(payload)
# Add a small await so two concurrent calls can interleave
# on the wire if the SDK pools the requests.
await asyncio.sleep(0.05)
return f"echoed:{payload}"
app = mcp.streamable_http_app()
app.add_middleware(AuthHeaderRecorder)
config = uvicorn.Config(
app,
host="127.0.0.1",
port=port,
log_level="warning",
access_log=False,
)
return uvicorn.Server(config)
def run_server_in_thread(server: uvicorn.Server) -> threading.Thread:
"""Boot the server in a background thread on its own asyncio loop."""
def _run() -> None:
loop = asyncio.new_event_loop()
asyncio.set_event_loop(loop)
loop.run_until_complete(server.serve())
t = threading.Thread(target=_run, daemon=True, name="spike-server")
t.start()
return t
async def wait_for_server_ready(url: str, timeout: float = 5.0) -> None:
"""Poll the server until it accepts connections."""
import urllib.parse
parsed = urllib.parse.urlparse(url)
deadline = time.monotonic() + timeout
while time.monotonic() < deadline:
try:
reader, writer = await asyncio.open_connection(parsed.hostname, parsed.port)
writer.close()
await writer.wait_closed()
return
except OSError:
await asyncio.sleep(0.05)
raise TimeoutError(f"server at {url} not ready within {timeout}s")
def fd_count() -> int:
"""Count open file descriptors for the current process."""
try:
return len(os.listdir(f"/proc/{os.getpid()}/fd"))
except OSError:
return -1
# ---------------------------------------------------------------------------
# Scenario 1: N=20 concurrent ClientSession instances
# ---------------------------------------------------------------------------
async def scenario_1_concurrent_sessions(url: str, n: int = 20) -> dict:
"""Open N concurrent ClientSession instances and call tools/list on each."""
print(f"\n=== Scenario 1: {n} concurrent ClientSession instances ===")
fd_before = fd_count()
async def one_session(idx: int) -> dict:
headers = {"Authorization": f"Bearer test-token-{idx}"}
async with (
streamablehttp_client(url=url, headers=headers) as (read, write, _),
ClientSession(read, write) as session,
):
await session.initialize()
tools = await session.list_tools()
return {
"idx": idx,
"tool_count": len(tools.tools),
"tool_names": [t.name for t in tools.tools],
}
start = time.monotonic()
results = await asyncio.gather(*[one_session(i) for i in range(n)], return_exceptions=True)
elapsed = time.monotonic() - start
fd_after = fd_count()
# Allow some settling time for FDs to release.
await asyncio.sleep(0.5)
fd_settled = fd_count()
successes = [r for r in results if isinstance(r, dict)]
failures = [r for r in results if isinstance(r, Exception)]
# Verify every session got the same tool catalog.
catalog_consistent = (
len(successes) == n and len({tuple(r["tool_names"]) for r in successes}) == 1
)
return {
"scenario": "concurrent_sessions",
"n": n,
"successes": len(successes),
"failures": len(failures),
"elapsed_seconds": round(elapsed, 3),
"fd_before": fd_before,
"fd_during_peak": fd_after,
"fd_settled": fd_settled,
"fd_growth_during": fd_after - fd_before,
"fd_growth_settled": fd_settled - fd_before,
"catalog_consistent": catalog_consistent,
"first_failure": str(failures[0]) if failures else None,
}
# ---------------------------------------------------------------------------
# Scenario 2: 2 concurrent tools/call on a shared session
# ---------------------------------------------------------------------------
async def scenario_2_concurrent_calls_shared_session(url: str) -> dict:
"""Two concurrent tools/call on one ClientSession with interleaving payloads.
The echo tool sleeps 50ms, so concurrent calls overlap on the wire.
Each call passes a distinct payload (~10KB) to make request bodies
spannable across multiple stream frames.
"""
print("\n=== Scenario 2: 2 concurrent tools/call on shared session ===")
# Generous-size payloads so both bodies live during the await.
payload_a = "A" * 10000
payload_b = "B" * 10000
headers = {"Authorization": "Bearer shared-session-token"}
async with (
streamablehttp_client(url=url, headers=headers) as (read, write, _),
ClientSession(read, write) as session,
):
await session.initialize()
TOOL_CALL_PAYLOADS.clear()
start = time.monotonic()
results = await asyncio.gather(
session.call_tool("echo", {"payload": payload_a}),
session.call_tool("echo", {"payload": payload_b}),
return_exceptions=True,
)
elapsed = time.monotonic() - start
successes = [r for r in results if not isinstance(r, Exception)]
failures = [r for r in results if isinstance(r, Exception)]
# Each result.content[0].text should be "echoed:{payload}".
response_payloads: list[str] = []
if len(successes) == 2:
for r in successes:
text = r.content[0].text if r.content else ""
response_payloads.append(text)
# Order may not match call order — what matters is both payloads echo.
expected = {f"echoed:{payload_a}", f"echoed:{payload_b}"}
received = set(response_payloads)
demux_ok = received == expected
# Did both calls actually overlap? If sequential, elapsed ~= 0.1+s;
# if concurrent, ~0.05s.
concurrent_observed = elapsed < 0.09
return {
"scenario": "concurrent_calls_shared_session",
"successes": len(successes),
"failures": len(failures),
"elapsed_seconds": round(elapsed, 3),
"demux_ok": demux_ok,
"expected_payloads_received": list(received) if demux_ok else None,
"actual_payloads_received_count": len(received),
"appears_concurrent_on_wire": concurrent_observed,
"first_failure": str(failures[0]) if failures else None,
}
# ---------------------------------------------------------------------------
# Scenario 3: per-session header isolation
# ---------------------------------------------------------------------------
async def scenario_3_header_isolation(url: str, n: int = 5) -> dict:
"""Open N sessions with distinct Authorization headers, call echo on each.
Verifies the server sees each session's own header — i.e. httpx
connection pooling does not cross headers between concurrent
ClientSession instances against the same URL.
"""
print(f"\n=== Scenario 3: {n}-session Authorization-header isolation ===")
SERVER_OBSERVATIONS.clear()
async def one_session(idx: int) -> str | None:
headers = {"Authorization": f"Bearer iso-token-{idx}"}
async with (
streamablehttp_client(url=url, headers=headers) as (read, write, _),
ClientSession(read, write) as session,
):
await session.initialize()
# One call per session.
await session.call_tool("echo", {"payload": f"session-{idx}"})
return f"Bearer iso-token-{idx}"
start = time.monotonic()
expected_tokens = await asyncio.gather(*[one_session(i) for i in range(n)])
elapsed = time.monotonic() - start
# Tally observed Authorization headers, ignoring None entries (initial
# handshake sometimes lacks auth).
observed_auth = [auth for auth, _ in SERVER_OBSERVATIONS if auth]
expected_set = set(expected_tokens)
observed_set = set(observed_auth)
# Every expected token must show up at least once on the server.
all_present = expected_set.issubset(observed_set)
# No spurious tokens.
no_extras = observed_set.issubset(expected_set)
# Frequency: at least one observation per token.
counts = defaultdict(int)
for a in observed_auth:
counts[a] += 1
each_seen = all(counts[t] >= 1 for t in expected_tokens)
return {
"scenario": "header_isolation",
"n": n,
"elapsed_seconds": round(elapsed, 3),
"expected_tokens": sorted(expected_set),
"observed_tokens": sorted(observed_set),
"all_expected_present": all_present,
"no_extra_tokens_observed": no_extras,
"each_token_seen_at_least_once": each_seen,
"header_counts_per_token": dict(counts),
"total_requests_observed": len(observed_auth),
}
# ---------------------------------------------------------------------------
# Driver
# ---------------------------------------------------------------------------
async def main() -> None:
port = find_free_port()
url = f"http://127.0.0.1:{port}/mcp"
server = build_server(port)
server_thread = run_server_in_thread(server)
try:
await wait_for_server_ready(url)
print(f"server up at {url}\n")
result_1 = await scenario_1_concurrent_sessions(url, n=20)
print_scenario_result(result_1)
result_2 = await scenario_2_concurrent_calls_shared_session(url)
print_scenario_result(result_2)
result_3 = await scenario_3_header_isolation(url, n=5)
print_scenario_result(result_3)
# Final verdict
verdict_1 = (
result_1["successes"] == result_1["n"]
and result_1["catalog_consistent"]
and result_1["fd_growth_settled"] < 30 # 20 sessions, generous bound
)
verdict_2 = result_2["demux_ok"] and result_2["successes"] == 2
verdict_3 = (
result_3["all_expected_present"]
and result_3["no_extra_tokens_observed"]
and result_3["each_token_seen_at_least_once"]
)
print("\n=== VERDICT ===")
print(f" Scenario 1 (concurrent sessions): {'PASS' if verdict_1 else 'FAIL'}")
print(f" Scenario 2 (concurrent calls shared): {'PASS' if verdict_2 else 'FAIL'}")
print(f" Scenario 3 (header isolation): {'PASS' if verdict_3 else 'FAIL'}")
all_pass = verdict_1 and verdict_2 and verdict_3
print(
f"\n Phase 5 per-(user, server) pool architecture: "
f"{'VIABLE' if all_pass else 'NEEDS REWORK (Alternative F fallback)'}"
)
sys.exit(0 if all_pass else 1)
finally:
server.should_exit = True
server_thread.join(timeout=5)
def print_scenario_result(result: dict) -> None:
print(f"\nresult[{result['scenario']}]:")
for k, v in result.items():
if k == "scenario":
continue
print(f" {k}: {v}")
if __name__ == "__main__":
with contextlib.suppress(KeyboardInterrupt):
asyncio.run(main())
-220
View File
@@ -1,220 +0,0 @@
"""Tests for turnstone-admin DB configuration precedence.
Locks in the alignment with turnstone-server:
CLI / config.toml [database] > TURNSTONE_DB_* env > hardcoded default
The motivation is to keep DB secrets in config.toml (see
feedback_secrets_not_in_env) rather than forcing operators to export
TURNSTONE_DB_URL before every admin invocation.
"""
from __future__ import annotations
import argparse
from typing import TYPE_CHECKING
from unittest.mock import patch
import pytest
if TYPE_CHECKING:
from collections.abc import Iterator
from pathlib import Path
import turnstone.core.config as config_mod
from turnstone.admin import _get_storage
def _reset_cache() -> None:
config_mod._cache = None
config_mod._config_path = None
def _build_args(config_path: str | None) -> argparse.Namespace:
"""Build an args namespace the way admin.main() does.
Skips ``add_config_arg`` (which reads ``sys.argv``) the test
constructs the args programmatically instead.
"""
config_mod.set_config_path(config_path or "/nonexistent/turnstone-admin-test.toml")
parser = argparse.ArgumentParser()
config_mod.apply_config(parser, ["database"])
sub = parser.add_subparsers(dest="command")
sub.add_parser("list-users")
return parser.parse_args(["list-users"])
@pytest.fixture(autouse=True)
def _clear_db_env(monkeypatch: pytest.MonkeyPatch) -> Iterator[None]:
"""Clean slate: no TURNSTONE_DB_* env vars unless a test sets them."""
for var in (
"TURNSTONE_DB_BACKEND",
"TURNSTONE_DB_URL",
"TURNSTONE_DB_PATH",
"TURNSTONE_DB_POOL_SIZE",
"TURNSTONE_DB_SSLMODE",
"TURNSTONE_DB_SSLROOTCERT",
"TURNSTONE_DB_SSLCERT",
"TURNSTONE_DB_SSLKEY",
"TURNSTONE_CONFIG",
):
monkeypatch.delenv(var, raising=False)
_reset_cache()
yield
_reset_cache()
def test_defaults_to_sqlite_when_neither_config_nor_env_set() -> None:
args = _build_args(None)
with patch("turnstone.core.storage.init_storage") as init:
_get_storage(args)
assert init.call_args.args == ("sqlite",)
assert init.call_args.kwargs["url"] == ""
assert init.call_args.kwargs["path"] == ""
assert init.call_args.kwargs["pool_size"] == 2
def test_config_toml_database_section_drives_init_storage(tmp_path: Path) -> None:
cfg = tmp_path / "config.toml"
cfg.write_text(
"[database]\n"
'backend = "postgresql"\n'
'url = "postgresql+psycopg://fromconfig:x@host/db"\n'
"pool_size = 5\n"
'sslmode = "verify-full"\n'
'sslrootcert = "/etc/ssl/ca.pem"\n'
'sslcert = "/etc/ssl/client.pem"\n'
'sslkey = "/etc/ssl/client.key"\n'
)
args = _build_args(str(cfg))
with patch("turnstone.core.storage.init_storage") as init:
_get_storage(args)
assert init.call_args.args == ("postgresql",)
kw = init.call_args.kwargs
assert kw["url"] == "postgresql+psycopg://fromconfig:x@host/db"
assert kw["pool_size"] == 5
assert kw["sslmode"] == "verify-full"
assert kw["sslrootcert"] == "/etc/ssl/ca.pem"
assert kw["sslcert"] == "/etc/ssl/client.pem"
assert kw["sslkey"] == "/etc/ssl/client.key"
def test_env_used_as_fallback_when_config_absent(monkeypatch: pytest.MonkeyPatch) -> None:
monkeypatch.setenv("TURNSTONE_DB_BACKEND", "postgresql")
monkeypatch.setenv("TURNSTONE_DB_URL", "postgresql+psycopg://fromenv:x@host/db")
monkeypatch.setenv("TURNSTONE_DB_POOL_SIZE", "7")
monkeypatch.setenv("TURNSTONE_DB_SSLMODE", "require")
args = _build_args(None)
with patch("turnstone.core.storage.init_storage") as init:
_get_storage(args)
assert init.call_args.args == ("postgresql",)
kw = init.call_args.kwargs
assert kw["url"] == "postgresql+psycopg://fromenv:x@host/db"
assert kw["pool_size"] == 7
assert kw["sslmode"] == "require"
def test_config_toml_wins_over_env(monkeypatch: pytest.MonkeyPatch, tmp_path: Path) -> None:
"""config.toml beats env — operators should put secrets in TOML."""
monkeypatch.setenv("TURNSTONE_DB_BACKEND", "sqlite")
monkeypatch.setenv("TURNSTONE_DB_URL", "postgresql+psycopg://fromenv:x@host/db")
monkeypatch.setenv("TURNSTONE_DB_SSLMODE", "require")
cfg = tmp_path / "config.toml"
cfg.write_text(
"[database]\n"
'backend = "postgresql"\n'
'url = "postgresql+psycopg://fromconfig:x@host/db"\n'
'sslmode = "verify-full"\n'
)
args = _build_args(str(cfg))
with patch("turnstone.core.storage.init_storage") as init:
_get_storage(args)
assert init.call_args.args == ("postgresql",)
kw = init.call_args.kwargs
assert kw["url"] == "postgresql+psycopg://fromconfig:x@host/db"
assert kw["sslmode"] == "verify-full"
def test_partial_config_falls_through_to_env_per_key(
monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
"""A key missing from [database] should fall back to its env var."""
monkeypatch.setenv("TURNSTONE_DB_SSLMODE", "require")
monkeypatch.setenv("TURNSTONE_DB_POOL_SIZE", "9")
cfg = tmp_path / "config.toml"
cfg.write_text(
'[database]\nbackend = "postgresql"\nurl = "postgresql+psycopg://fromconfig:x@host/db"\n'
)
args = _build_args(str(cfg))
with patch("turnstone.core.storage.init_storage") as init:
_get_storage(args)
kw = init.call_args.kwargs
assert kw["url"] == "postgresql+psycopg://fromconfig:x@host/db"
assert kw["sslmode"] == "require"
assert kw["pool_size"] == 9
def test_empty_string_in_config_beats_env(monkeypatch: pytest.MonkeyPatch, tmp_path: Path) -> None:
"""`url = ""` in config.toml beats an env var.
Locks in the `is not None` guard a falsy-but-present TOML value
should NOT silently fall through to the env fallback.
"""
monkeypatch.setenv("TURNSTONE_DB_URL", "postgresql+psycopg://fromenv:x@host/db")
cfg = tmp_path / "config.toml"
cfg.write_text('[database]\nbackend = "sqlite"\nurl = ""\n')
args = _build_args(str(cfg))
with patch("turnstone.core.storage.init_storage") as init:
_get_storage(args)
assert init.call_args.kwargs["url"] == ""
def test_main_threads_config_toml_through_real_argv(
monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
"""End-to-end: ``turnstone-admin --config <toml> list-users`` honors TOML.
Covers the ``add_config_arg`` -> ``apply_config`` -> ``_get_storage``
chain that the programmatic ``_build_args`` helper skips.
"""
cfg = tmp_path / "config.toml"
cfg.write_text(
'[database]\nbackend = "postgresql"\nurl = "postgresql+psycopg://fromcli:x@host/db"\n'
)
monkeypatch.setattr("sys.argv", ["turnstone-admin", "--config", str(cfg), "list-users"])
fake_storage = patch("turnstone.core.storage.init_storage").start()
fake_storage.return_value.list_users.return_value = []
try:
from turnstone.admin import main
main()
finally:
patch.stopall()
assert fake_storage.call_args.args == ("postgresql",)
assert fake_storage.call_args.kwargs["url"] == "postgresql+psycopg://fromcli:x@host/db"
def test_get_storage_initializes_real_sqlite_backend(tmp_path: Path) -> None:
"""Drives the real ``init_storage`` boundary on a fresh sqlite file.
Mock-only tests would miss a kwarg-name typo (sslmode -> ssl_mode).
This test trips on any such drift because Alembic + the backend
actually run.
"""
from turnstone.core.storage import reset_storage
db_file = tmp_path / "admin.db"
cfg = tmp_path / "config.toml"
cfg.write_text(f'[database]\nbackend = "sqlite"\npath = "{db_file}"\n')
args = _build_args(str(cfg))
reset_storage()
try:
storage = _get_storage(args)
assert storage.list_users() == []
finally:
reset_storage()
File diff suppressed because it is too large Load Diff
+2 -2
View File
@@ -49,7 +49,7 @@ class TestServerVersioning:
mock_mgr = MagicMock()
mock_mgr.list_all.return_value = []
mock_mgr.max_active = 10
mock_mgr.max_workstreams = 10
app = create_app(
workstreams=mock_mgr,
global_queue=queue.Queue(),
@@ -76,7 +76,7 @@ class TestServerVersioning:
assert resp.status_code == 200
spec = resp.json()
assert spec["openapi"] == "3.1.0"
assert "/v1/api/workstreams/{ws_id}/send" in spec["paths"]
assert "/v1/api/send" in spec["paths"]
def test_docs_page(self, client):
resp = client.get("/docs")
-528
View File
@@ -1,528 +0,0 @@
"""Static smoke guards for ``turnstone/ui/static/app.js``.
The interactive WebUI's app.js has no JS test framework on the
project side. This file holds Python-side string-presence assertions
that catch regressions on critical paths the kind of one-line
deletion or rename that breaks the UI silently and only surfaces in
manual testing.
"""
from __future__ import annotations
import re
from pathlib import Path
_APP_JS = Path(__file__).resolve().parent.parent / "turnstone/ui/static/app.js"
def test_switch_tab_bootstraps_pane_when_none_exists() -> None:
"""``switchTab`` must create a pane when none exists. A fresh-
loaded interactive UI with no workstreams shows the dashboard
and creates no panes (per ``initWorkstreams``); the user's first
``create`` or ``open`` then calls ``switchTab(newWsId)``. Pre-fix,
the early ``if (!pane) return;`` left switchTab with nowhere to
attach the chat UI never connected SSE for the freshly-created
workstream, and only a page refresh fixed it. This test guards
against accidentally re-introducing the early-return."""
body = _APP_JS.read_text(encoding="utf-8")
start = body.index("function switchTab(wsId) {")
# Bound the search to the function body — switchTab is short.
fn = body[start : start + 2000]
assert "if (!pane) return;" not in fn, (
"switchTab must not early-return when no pane exists — that's "
"the no-chat-after-first-create bug. Bootstrap a pane instead."
)
# Affirmatively check the bootstrap path exists.
assert "createPane(wsId)" in fn, (
"switchTab must call createPane(wsId) to bootstrap the first "
"pane when getFocusedPane returns null"
)
def test_tool_error_does_not_overwrite_approval_badge() -> None:
"""When an approved tool subsequently errors, the existing
`` approved`` (or `` auto-approved``) pill must remain visible
the error indicator is appended as a sibling pill, not by mutating
the approval pill in place. Pre-fix, both ``appendToolOutput``
(live) and ``replayHistory`` (history reconstruction) located the
existing approval badge via ``querySelector(".ts-approval-badge")``
and overwrote its className + textContent with the ``--error``
state, so the user lost the record that they had approved the
call. This test pins the new append-sibling behaviour."""
body = _APP_JS.read_text(encoding="utf-8")
# Affirmatively check that an idempotency guard exists somewhere:
# a ``querySelector(".ts-approval-badge--error")`` lookup is the
# structural marker of the fix. Pre-fix the modifier never appeared
# in app.js at all. Loose on quote style and surrounding form (the
# guard might be a negated ``if (!q) {build...}`` block at a call
# site, or a positive ``if (q) return;`` early-exit inside an
# extracted helper) so a later refactor doesn't trip CI on
# cosmetics.
error_guard_re = re.compile(
r"""querySelector\(\s*['"]\.ts-approval-badge--error['"]\s*\)""",
)
assert error_guard_re.search(body), (
"The error-badge code path must guard creation with a "
"querySelector for .ts-approval-badge--error so duplicate fires "
"(live + history re-render) do not stack badges."
)
# Forbid the mutate-existing-badge sequence: a generic
# ``.ts-approval-badge`` lookup followed within a handful of lines
# by mutating that same handle into the ``--error`` state. Two
# unrelated call sites (history rendering + live tool-output
# insertion) legitimately query ``.ts-approval-badge`` to position
# output above it, so the bare query alone is not the anti-pattern;
# the close pairing with an ``--error`` class mutation is. Accept
# either quote style and catch both ``className = "..."`` and
# ``classList.add("ts-approval-badge--error")`` forms.
overwrite_re = re.compile(
r"""(\w+)\s*=\s*\w+\.querySelector\(\s*(["'])\.ts-approval-badge\2\s*\)\s*;"""
r""".{0,200}?"""
r"""(?:"""
r"""\1\.className\s*=\s*(["'])[^"']*\bts-approval-badge--error\b[^"']*\3"""
r"""|"""
r"""\1\.classList\.add\([^)]*(["'])ts-approval-badge--error\4[^)]*\)"""
r""")""",
re.DOTALL,
)
assert not overwrite_re.search(body), (
"Found the badge-overwrite anti-pattern: a queried "
".ts-approval-badge handle is mutated into the --error variant "
"(via className overwrite or classList.add). Append a sibling "
"badge instead so the approval verdict stays visible alongside "
"the error."
)
def test_replay_history_renders_content_before_tool_block() -> None:
"""In ``replayHistory``'s ``role === "assistant"`` branch, the
``msg.content`` render must precede the ``msg.tool_calls`` render.
Two reasons, both load-bearing:
1. **Structural** the next loop iteration's ``role === "tool"``
message anchors to ``lastToolBlock``. The tool-block branch sets
that anchor; the content branch clears it. If content runs after
the tool block, the clear silently drops the upcoming tool
result. Pre-fix, every interactive tool result was missing from
saved-workstream replays whenever the assistant turn carried
both narration and tool calls (very common output shape).
2. **Visual** the live SSE path renders content first
(``stream_text`` streams before ``tool_info`` /
``approve_request``), so replay should match.
The test pins the order via the offsets of the ``msg.content`` and
``msg.tool_calls`` branch headers inside the function body."""
body = _APP_JS.read_text(encoding="utf-8")
start = body.index("Pane.prototype.replayHistory = function")
end = body.index("Pane.prototype._attachRetryToLastAssistant", start)
fn = body[start:end]
# Locate the assistant branch and bound the search to its body —
# the function also handles user / tool roles which would otherwise
# confuse the offset comparison.
asst_start = fn.index('msg.role === "assistant"')
asst_end = fn.index('msg.role === "tool"', asst_start)
asst = fn[asst_start:asst_end]
# ``if (msg.content && msg.content.trim())`` guards against a
# whitespace-only content row (Qwen-style "\n\n" left over after a
# reasoning-parser model strips ``<think>…</think>`` and emits
# nothing else before the tool call). Pre-trim guard, those rows
# rendered as a visible-but-empty ``.msg.assistant`` card on
# replay. Match the substring up to ``msg.content`` so the test
# tolerates either guard shape without locking the trim() in.
content_idx = asst.index("if (msg.content")
tool_calls_idx = asst.index("if (msg.tool_calls && msg.tool_calls.length)")
assert content_idx < tool_calls_idx, (
"replayHistory must render msg.content BEFORE msg.tool_calls "
"inside the assistant branch — otherwise the lastToolBlock "
"anchor is clobbered before the next iteration's tool result "
"can attach to it (and the visual order also drifts from the "
"live SSE flow)."
)
def test_replay_history_renders_persisted_verdict_badge() -> None:
"""Saved-workstream replays must paint the persisted intent verdict
next to each tool div, using the same ``renderVerdictBadge`` helper
the live ``showInlineToolBlock`` path uses. Pre-fix the audit trail
was complete in storage (``intent_verdicts`` table) but never
surfaced on replay operators reviewing a saved workstream
couldn't see what the heuristic / LLM judge thought of any tool
call. This test pins the call site so a refactor that drops the
decoration regresses the audit surface."""
body = _APP_JS.read_text(encoding="utf-8")
start = body.index("Pane.prototype.replayHistory = function")
end = body.index("Pane.prototype._attachRetryToLastAssistant", start)
fn = body[start:end]
# Match a `renderVerdictBadge(<something>.verdict, ...)` call inside
# the replay loop. Loose on whitespace + identifier so a future
# rename of the iteration variable doesn't trip CI.
badge_call_re = re.compile(
r"renderVerdictBadge\(\s*\w+\.verdict\b",
)
assert badge_call_re.search(fn), (
"replayHistory must call renderVerdictBadge(tc.verdict, ...) "
"when a persisted verdict is attached to a tool_call entry — "
"otherwise the audit-trail data persisted to intent_verdicts "
"doesn't surface on saved-workstream replays."
)
def test_shared_utils_defines_replay_advisories_after_tool() -> None:
"""The shared ``replayAdvisoriesAfterTool`` helper in
``shared_static/utils.js`` is the single source of advisory-walk +
type-filter logic for both ``app.js`` (interactive) and
``coordinator.js`` (coord). A refactor that drops the helper
breaks both surfaces, so guard its definition + filter shape here.
"""
utils_js = Path(__file__).resolve().parent.parent / "turnstone/shared_static/utils.js"
body = utils_js.read_text(encoding="utf-8")
assert "function replayAdvisoriesAfterTool" in body, (
"shared/utils.js must define replayAdvisoriesAfterTool — "
"interactive and coord both invoke it."
)
# The type filter — ``adv.type !== 'user_interjection'`` — must
# remain in the helper so a future advisory shape (output_guard,
# metacognitive nudge, etc.) doesn't silently render as a user
# bubble.
assert 'adv.type !== "user_interjection"' in body, (
"replayAdvisoriesAfterTool must filter by advisory type so a "
"future non-user_interjection advisory shape doesn't silently "
"render as a user bubble."
)
def test_replay_renders_user_interjection_advisory_after_tool_block() -> None:
"""Queued user messages spliced into the last tool-result envelope
of a batch (Seam 1) persist on the tool DB row as a wrapped
``<tool_output>`` envelope. ``decorate_history_messages`` extracts
the advisory back out and the wire layer projects it onto
``msg.advisories``; ``replayHistory`` must invoke the shared
``replayAdvisoriesAfterTool`` helper (defined in
``shared/utils.js``) so each ``user_interjection`` renders through
``addUserMessage`` and the bubble looks identical to a Seam 2/3
user row.
This test pins the call site so a refactor that drops the helper
invocation regresses the queued-during-batch replay shape
silently."""
body = _APP_JS.read_text(encoding="utf-8")
start = body.index("Pane.prototype.replayHistory = function")
end = body.index("Pane.prototype._attachRetryToLastAssistant", start)
fn = body[start:end]
# The replay loop must invoke the shared helper, passing
# ``msg.advisories`` and a renderer that routes through
# ``addUserMessage``. The helper itself filters on
# ``adv.type !== "user_interjection"``; that branch lives in
# ``shared/utils.js`` (test_shared_utils_js or runtime smoke covers
# the helper's body).
assert "replayAdvisoriesAfterTool(msg.advisories" in fn, (
"replayHistory must invoke replayAdvisoriesAfterTool with "
"msg.advisories so queued messages spliced into the tool "
"envelope render as user bubbles after the tool block."
)
assert "addUserMessage(text" in fn, (
"replayHistory's renderer callback must route the extracted "
"advisory text through addUserMessage so the rendered bubble "
"matches a normal user-row replay."
)
# ---------------------------------------------------------------------------
# Phase 8 — Chunk D: MCP error embed + settings panel UX
# ---------------------------------------------------------------------------
_INDEX_HTML = Path(__file__).resolve().parent.parent / "turnstone/ui/static/index.html"
_STYLE_CSS = Path(__file__).resolve().parent.parent / "turnstone/ui/static/style.css"
# The Phase-8 D-chunk pins the absence of an unsafe DOM-write API
# in two regions of app.js. Spell the property name out of literal
# concatenation so the tooling that flags occurrences in code
# strings doesn't false-positive on the test source.
_UNSAFE_DOM_WRITE_RE = re.compile(r"\.inner" + r"HTML\s*=")
def test_phase8_mcp_error_helpers_defined_in_app_js() -> None:
"""The Phase 8 dashboard renderer adds three load-bearing helpers
next to the existing media-embed pattern: ``tryParseMcpError``
(envelope detector), ``buildMcpErrorEmbed`` (interactive card),
and the ``_pendingConsentServers`` set that drives the gear-icon
badge. A regression that drops any of them silently degrades the
OAuth consent UX to a plain JSON dump, so guard their existence
here."""
body = _APP_JS.read_text(encoding="utf-8")
assert "function tryParseMcpError" in body, (
"tryParseMcpError must remain defined — appendToolOutput's "
"error branch depends on it to detect the MCP error envelope."
)
assert "function buildMcpErrorEmbed" in body, (
"buildMcpErrorEmbed must remain defined — it renders the "
"interactive consent / forbidden / operator card."
)
assert "_pendingConsentServers" in body, (
"_pendingConsentServers state must remain — it backs the "
"gear-icon badge so a user who scrolls past a consent prompt "
"still has a stable signal that consent is pending."
)
# The buildMcpErrorEmbed pattern must also wire the "actionable"
# branch (consent_required / insufficient_scope) into the badge
# via _onConsentDetected; pin the helper name.
assert "_onConsentDetected" in body, (
"_onConsentDetected must remain — buildMcpErrorEmbed calls it "
"for the actionable category to surface the gear-icon badge."
)
def test_phase8_settings_panel_handlers_defined() -> None:
"""The settings modal exposes four entry points that the inline
``onclick`` attributes in index.html depend on. Renaming or
deleting any of them breaks the modal silently (the buttons are
still rendered but click-to-action is dead). Catch that here."""
body = _APP_JS.read_text(encoding="utf-8")
for name in [
"function openSettingsPanel",
"function closeSettingsPanel",
"function confirmRevokeMcp",
"function cancelRevokeMcp",
]:
assert name in body, f"Missing required handler: {name}"
# The connections list is fetched against the Phase-7 endpoint —
# pin the URL so a server-side rename forces an explicit UI bump.
assert "/v1/api/mcp/oauth/connections" in body, (
"Settings panel must fetch /v1/api/mcp/oauth/connections — "
"a server-side rename needs an explicit UI update."
)
def test_phase8_appendtooloutput_dispatches_mcp_error_before_renderer() -> None:
"""``appendToolOutput`` must call ``tryParseMcpError`` inside its
``isError`` branch BEFORE falling through to the plain
``renderToolOutput`` path. The ordering is what makes the
interactive consent card replace the JSON dump; reverse the calls
and the user sees the raw error envelope as text again."""
body = _APP_JS.read_text(encoding="utf-8")
start = body.index("Pane.prototype.appendToolOutput = function")
end = body.index("Pane.prototype.", start + 10)
fn = body[start:end]
parse_idx = fn.find("tryParseMcpError(")
render_idx = fn.find("renderToolOutput(")
assert parse_idx >= 0, (
"appendToolOutput must call tryParseMcpError on the error path "
"before renderToolOutput, otherwise the consent card never "
"replaces the plain JSON output."
)
assert render_idx >= 0, "renderToolOutput call must remain present"
assert parse_idx < render_idx, (
"tryParseMcpError must run BEFORE renderToolOutput so the "
"interactive card path takes precedence over plain rendering."
)
def test_phase8_no_unsafe_dom_write_in_settings_panel() -> None:
"""Defensive XSS guard: the settings panel renders user-controlled
server names, scope strings, and timestamp values into the DOM.
The whole section MUST go through ``textContent``-style APIs; an
unsafe-DOM-write assignment would be a regression vector. Bound
the check to the section 15 body to avoid false positives
elsewhere."""
body = _APP_JS.read_text(encoding="utf-8")
start = body.index("// 15. MCP server connections settings panel")
# Bound to the full settings section (terminates at the next
# top-level keydown handler block).
end = body.index('document.addEventListener("keydown"', start)
section = body[start:end]
assert not _UNSAFE_DOM_WRITE_RE.search(section), (
"Section 15 must not assign to the unsafe DOM-write property — "
"server names and scope values flow through here and would be "
"XSS-injectable. Use textContent / DOM APIs instead."
)
def test_phase8_settings_button_in_index_html() -> None:
"""The gear-icon entry-point for the settings menu must remain
in the appbar's actions span. The console proxy IIFE prepends a
node pill to ``header.firstChild`` (turnstone/console/server.py:
202); our button is appended inside ``<span class='appbar-actions'>``
on the right, so they don't collide. Pin both shape constraints
here so a future appbar refactor keeps them disjoint."""
body = _INDEX_HTML.read_text(encoding="utf-8")
assert 'id="settings-btn"' in body, (
"index.html must keep the #settings-btn — onclick handlers "
"and the consent badge target it by id."
)
assert 'onclick="toggleSettingsMenu(this)"' in body, (
"settings-btn must wire onclick=toggleSettingsMenu(this) — "
"the gear opens a dropdown with MCP connections + Logout; "
"losing the binding leaves the menu unreachable."
)
# The button must live inside <span class="appbar-actions"> so the
# console proxy's header.insertBefore(pill, header.firstChild)
# leaves it untouched.
actions_open = body.index('class="appbar-actions"')
actions_close = body.index("</span>", actions_open)
assert 'id="settings-btn"' in body[actions_open:actions_close], (
"settings-btn must be inside <span class='appbar-actions'> "
"so the console proxy's firstChild prepend doesn't shift it."
)
def test_settings_menu_handlers_defined() -> None:
"""The gear-icon dropdown exposes a toggle/open/close trio that the
inline ``onclick="toggleSettingsMenu(this)"`` in index.html depends
on, plus the menu items themselves must wire to existing entry
points (``openSettingsPanel`` for MCP connections, ``logout`` for
sign-out). Pin all four so a rename or deletion fails loudly here
instead of silently leaving the gear's menu broken or wired to a
stale function."""
body = _APP_JS.read_text(encoding="utf-8")
for name in [
"function toggleSettingsMenu",
"function openSettingsMenu",
"function closeSettingsMenu",
]:
assert name in body, f"Missing required handler: {name}"
# Bound to the settings-menu region so we don't accidentally match
# an unrelated openSettingsPanel/logout call elsewhere in the file.
start = body.index("function openSettingsMenu(")
end = body.index("function closeSettingsMenu(", start)
section = body[start:end]
assert "openSettingsPanel()" in section, (
"Settings menu's MCP-connections item must call openSettingsPanel() "
"— otherwise the existing settings overlay is unreachable from the "
"new dropdown."
)
assert "logout()" in section, (
"Settings menu's Logout item must call logout() — that's the "
"shared auth.js entry point that clears the cookie + session state."
)
def test_dashboard_overlay_is_region_not_dialog() -> None:
"""The dashboard overlay must be role='region' (not role='dialog' +
aria-modal='true'). The role downgrade is what allows ui-header to
stay interactive while the dashboard is open see the comment at
showDashboard() in app.js. A revert to role='dialog' + aria-modal
would re-trap focus and break the gear/theme buttons + the console
proxy's node-picker pill while the dashboard is open."""
body = _INDEX_HTML.read_text(encoding="utf-8")
idx = body.index('id="dashboard"')
# Bound to ~600 chars after the tag so we only check this element's
# attributes — same shape as test_phase8_settings_modal_in_index_html.
chunk = body[idx : idx + 600]
assert 'role="region"' in chunk, (
"dashboard must be role='region' — see showDashboard() comment."
)
assert "aria-modal" not in chunk, (
"dashboard must NOT be aria-modal — re-trapping focus breaks "
"the appbar's interactive controls (theme toggle, settings menu, "
"proxy node-picker pill) while the dashboard is open."
)
def test_close_settings_menu_resets_aria() -> None:
"""closeSettingsMenu must reset aria-expanded='false' AND remove
aria-controls from the gear trigger. Without the reset the gear
keeps reporting 'expanded' to assistive tech after the menu closes;
without the removal aria-controls points at a dead DOM id."""
body = _APP_JS.read_text(encoding="utf-8")
start = body.index("function closeSettingsMenu(")
# Bound to ~600 chars so we don't catch unrelated handlers.
section = body[start : start + 600]
assert 'setAttribute("aria-expanded", "false")' in section, (
"closeSettingsMenu must set aria-expanded='false' on the gear."
)
assert 'removeAttribute("aria-controls")' in section, (
"closeSettingsMenu must remove aria-controls from the gear."
)
def test_phase8_settings_modal_in_index_html() -> None:
"""Both the settings overlay and the revoke-confirmation overlay
must remain in the modal area. The Escape-key deferral list in
app.js targets these ids, so removing them silently breaks the
handler chain."""
body = _INDEX_HTML.read_text(encoding="utf-8")
assert 'id="settings-overlay"' in body
assert 'id="revoke-mcp-overlay"' in body
# Each overlay must have role="dialog" + aria-modal="true" so
# screen readers and the existing modal-deferral handlers can
# treat them like the rest of the modal stack.
for overlay_id in ("settings-overlay", "revoke-mcp-overlay"):
idx = body.index(f'id="{overlay_id}"')
# Bound to ~600 chars after the open tag so we only check this
# overlay's attributes.
chunk = body[idx : idx + 600]
assert 'role="dialog"' in chunk, f"{overlay_id} missing role=dialog"
assert 'aria-modal="true"' in chunk, f"{overlay_id} missing aria-modal=true"
def test_phase8_xss_safe_render_in_build_mcp_error_embed() -> None:
"""Adversarial input — the renderer for an MCP error envelope
must use ``textContent`` (not the unsafe DOM-write API) for every
field that flows from the server: ``err.detail``, ``err.server``,
scopes list. The card builder uses createElement + textContent
throughout so a script-tag server name renders harmlessly. Pin
the absence of the unsafe-write inside ``buildMcpErrorEmbed``."""
body = _APP_JS.read_text(encoding="utf-8")
start = body.index("function buildMcpErrorEmbed(")
# Bound to the function body — find its closing brace at column 0.
rest = body[start:]
# Closing function brace at line start (matches existing functions)
end_match = re.search(r"\n}\n", rest)
assert end_match is not None
fn = rest[: end_match.end()]
assert not _UNSAFE_DOM_WRITE_RE.search(fn), (
"buildMcpErrorEmbed must not use the unsafe-DOM-write API — "
"server names and detail strings flow through here. An "
"adversarial server name must render harmlessly via "
"textContent."
)
def test_phase8_css_classes_present_in_stylesheet() -> None:
"""The card / badge / modal classes referenced from app.js must
have CSS rules. Without them the DOM still works but the visual
treatment is gone, which would silently degrade the consent UX."""
css = _STYLE_CSS.read_text(encoding="utf-8")
for selector in [
".mcp-error-card",
".mcp-error-icon",
".mcp-error-action-btn",
".mcp-scope-pill",
"#settings-overlay",
"#settings-box",
".settings-revoke-btn",
".settings-consent-badge",
"#revoke-mcp-overlay",
]:
assert selector in css, f"Missing CSS rule for {selector}"
def test_phase8_consent_url_prefix_check_in_click_handler() -> None:
"""Defence-in-depth: the consent button's click handler must reject
any ``consent_url`` that doesn't start with the dispatcher's known
prefix (``/v1/api/mcp/oauth/start``). ``_build_consent_url`` always
emits a path-relative URL with that exact prefix; a non-prefix
value implies the producer drifted (or was compromised) and a
``window.open("javascript:...")`` would be catastrophic.
The renderer is the last line of defence before ``window.open`` and
must not rely on the producer-side guarantee alone. Pin the prefix
string and the ``startsWith`` form so a future refactor can't
silently weaken the guard.
"""
body = _APP_JS.read_text(encoding="utf-8")
# Bound the search to the click handler region (between the
# ``buildMcpErrorEmbed`` function and the next top-level helper) to
# avoid false positives from unrelated string occurrences.
start = body.index("function buildMcpErrorEmbed(")
end = body.index("\n}\n", start) + 1
fn = body[start:end]
assert 'consentUrl.startsWith("/v1/api/mcp/oauth/start")' in fn, (
"Click handler must guard window.open with "
'consentUrl.startsWith("/v1/api/mcp/oauth/start"). Without it '
"a future producer drift to a non-path-relative URL (or a "
'"javascript:" injection) would be passed straight to '
"window.open."
)
-152
View File
@@ -56,155 +56,3 @@ def test_record_audit_generates_unique_ids(storage):
events = storage.list_audit_events()
assert len(events) == 2
assert events[0]["event_id"] != events[1]["event_id"]
# ---------------------------------------------------------------------------
# Credential redaction at the audit boundary
# ---------------------------------------------------------------------------
def test_record_audit_redacts_passwords_by_default(storage):
"""Detail strings go through redact_credentials by default."""
record_audit(
storage,
"u1",
"coordinator.spawn",
detail={
"initial_message": "connect via postgresql://alice:s3cret@db.example.com/app",
},
)
event = storage.list_audit_events()[0]
detail = json.loads(event["detail"])
# Exact redaction text comes from output_guard._redact_credentials —
# assert the token is stripped rather than the exact marker so
# this test doesn't break if the marker format evolves.
assert "s3cret" not in detail["initial_message"]
assert "REDACTED" in detail["initial_message"]
def test_record_audit_redacts_nested_strings(storage):
"""Walker descends into lists / nested dicts."""
record_audit(
storage,
"u1",
"tasks.update",
detail={
"tasks": [
{"title": "normal task"},
{"title": "pull secret from AWS_SECRET_ACCESS_KEY=AKIAEXAMPLE123"},
],
},
)
event = storage.list_audit_events()[0]
detail = json.loads(event["detail"])
assert detail["tasks"][0]["title"] == "normal task"
assert "AKIAEXAMPLE123" not in detail["tasks"][1]["title"]
def test_record_audit_raw_detail_preserves_payload(storage):
"""`raw_detail=True` bypasses the scrub — operator-originated detail only."""
secret_like = "postgresql://alice:s3cret@db.example.com/app"
record_audit(
storage,
"admin-1",
"investigation.note",
detail={"note": secret_like},
raw_detail=True,
)
event = storage.list_audit_events()[0]
detail = json.loads(event["detail"])
assert detail["note"] == secret_like
def test_record_audit_strips_control_chars(storage):
"""CR/LF/NUL/DEL and C0 controls are replaced with spaces so a
downstream exporter that prints raw detail strings can't re-surface
log-injection. Tab/newline are deliberately preserved."""
record_audit(
storage,
"u1",
"coordinator.note",
detail={
"msg": "hello\r\nInjected: bad\x00 escape \x1b[31mred\x1b[0m\x7f",
"ok_tab": "a\tb\nc",
},
)
event = storage.list_audit_events()[0]
detail = json.loads(event["detail"])
# CR / NUL / ESC / DEL scrubbed to spaces; tab + newline kept.
assert "\r" not in detail["msg"]
assert "\x00" not in detail["msg"]
assert "\x1b" not in detail["msg"]
assert "\x7f" not in detail["msg"]
assert "hello" in detail["msg"]
assert detail["ok_tab"] == "a\tb\nc"
def test_record_audit_clean_strings_roundtrip_unchanged(storage):
"""Detail strings with no credential patterns and no control chars
pass through unchanged the fast-path / scrub must not corrupt the
common case."""
clean = {"note": "hello world", "code": "import foo", "state": "ok"}
record_audit(storage, "u1", "coordinator.note", detail=clean)
event = storage.list_audit_events()[0]
assert json.loads(event["detail"]) == clean
def test_record_audit_fast_path_skips_no_string_detail(storage):
"""A detail carrying only scalars (no strings anywhere) must persist
identically exercises the ``_has_any_string`` fast path."""
record_audit(
storage,
"u1",
"coordinator.metric",
detail={"spawned": 5, "ok": True, "parent": None, "tail": [1, 2, 3]},
)
event = storage.list_audit_events()[0]
assert json.loads(event["detail"]) == {
"spawned": 5,
"ok": True,
"parent": None,
"tail": [1, 2, 3],
}
def test_record_audit_redacts_dict_keys(storage):
"""Walker descends into dict keys too — a caller using
model-controlled text as a key can't leak it verbatim."""
record_audit(
storage,
"u1",
"coordinator.note",
detail={"postgresql://alice:s3cret@db.example.com/app": True},
)
event = storage.list_audit_events()[0]
detail = json.loads(event["detail"])
assert all("s3cret" not in k for k in detail)
def test_record_audit_walks_set_and_frozenset(storage):
"""Walker handles set/frozenset values (docstring promise)."""
record_audit(
storage,
"u1",
"coordinator.note",
detail={"tags": frozenset({"ak_" + "x" * 40, "plain"})},
)
event = storage.list_audit_events()[0]
detail = json.loads(event["detail"])
# The credential-looking AK token gets scrubbed; the plain one survives.
tags = detail["tags"]
assert "plain" in tags
def test_record_audit_leaves_non_string_scalars_alone(storage):
"""Non-string scalars (int / bool / None) pass through unchanged."""
record_audit(
storage,
"u1",
"coordinator.spawn",
detail={"budget_ok": True, "spawned": 5, "parent": None},
)
event = storage.list_audit_events()[0]
detail = json.loads(event["detail"])
assert detail == {"budget_ok": True, "spawned": 5, "parent": None}
+37 -337
View File
@@ -53,8 +53,8 @@ class TestIsPublicPath:
def test_api_workstreams_not_public(self):
assert is_public_path("/api/workstreams") is False
def test_api_workstreams_send_not_public(self):
assert is_public_path("/api/workstreams/abc/send") is False
def test_api_send_not_public(self):
assert is_public_path("/api/send") is False
def test_api_cluster_overview_not_public(self):
assert is_public_path("/api/cluster/overview") is False
@@ -71,8 +71,8 @@ class TestIsPublicPath:
def test_v1_api_workstreams_not_public(self):
assert is_public_path("/v1/api/workstreams") is False
def test_v1_api_workstreams_send_not_public(self):
assert is_public_path("/v1/api/workstreams/abc/send") is False
def test_v1_api_send_not_public(self):
assert is_public_path("/v1/api/send") is False
def test_openapi_json_public(self):
assert is_public_path("/openapi.json") is True
@@ -83,39 +83,6 @@ class TestIsPublicPath:
def test_shared_static_public(self):
assert is_public_path("/shared/base.css") is True
# Console proxy: a public proxied path must still be public, otherwise
# the login modal can never re-authenticate from inside a ``/node/{id}/``
# proxied page once the cookie expires.
def test_proxy_v1_login_public(self):
assert is_public_path("/node/node-a/v1/api/auth/login") is True
def test_proxy_no_v1_login_public(self):
assert is_public_path("/node/node-a/api/auth/login") is True
def test_proxy_v1_status_public(self):
assert is_public_path("/node/node-a/v1/api/auth/status") is True
def test_proxy_v1_setup_public(self):
assert is_public_path("/node/node-a/v1/api/auth/setup") is True
def test_proxy_v1_logout_public(self):
assert is_public_path("/node/node-a/v1/api/auth/logout") is True
def test_proxy_v1_oidc_authorize_public(self):
assert is_public_path("/node/node-a/v1/api/auth/oidc/authorize") is True
def test_proxy_v1_oidc_callback_public(self):
assert is_public_path("/node/node-a/v1/api/auth/oidc/callback") is True
def test_proxy_v1_workstreams_still_not_public(self):
"""Proxy prefix must not turn protected paths into public ones."""
assert is_public_path("/node/node-a/v1/api/workstreams") is False
def test_proxy_v1_refresh_still_requires_auth(self):
"""Refresh isn't in PUBLIC_PATHS — the caller must already have
a valid cookie. Proxy-prefix shouldn't change that."""
assert is_public_path("/node/node-a/v1/api/auth/refresh") is False
# ---------------------------------------------------------------------------
# TestRequiredRole
@@ -130,22 +97,10 @@ class TestRequiredScope:
assert required_scope("GET", "/api/events") == "read"
def test_post_send_needs_write(self):
assert required_scope("POST", "/api/workstreams/abc/send") == "write"
def test_delete_send_needs_write(self):
assert required_scope("DELETE", "/api/workstreams/abc/send") == "write"
assert required_scope("POST", "/api/send") == "write"
def test_post_approve_needs_approve(self):
assert required_scope("POST", "/api/workstreams/abc/approve") == "approve"
def test_post_cancel_needs_write(self):
assert required_scope("POST", "/api/workstreams/abc/cancel") == "write"
def test_post_close_needs_write(self):
assert required_scope("POST", "/api/workstreams/abc/close") == "write"
def test_get_events_per_ws_needs_read(self):
assert required_scope("GET", "/api/workstreams/abc/events") == "read"
assert required_scope("POST", "/api/approve") == "approve"
def test_post_plan_needs_write(self):
assert required_scope("POST", "/api/plan") == "write"
@@ -156,6 +111,9 @@ class TestRequiredScope:
def test_post_workstreams_new_needs_write(self):
assert required_scope("POST", "/api/workstreams/new") == "write"
def test_post_workstreams_close_needs_write(self):
assert required_scope("POST", "/api/workstreams/close") == "write"
def test_all_write_paths_need_write(self):
for path in WRITE_PATHS:
scope = required_scope("POST", path)
@@ -165,10 +123,10 @@ class TestRequiredScope:
assert required_scope("POST", "/api/unknown") == "read"
def test_v1_post_send_needs_write(self):
assert required_scope("POST", "/v1/api/workstreams/abc/send") == "write"
assert required_scope("POST", "/v1/api/send") == "write"
def test_v1_post_approve_needs_approve(self):
assert required_scope("POST", "/v1/api/workstreams/abc/approve") == "approve"
assert required_scope("POST", "/v1/api/approve") == "approve"
def test_v1_get_workstreams_needs_read(self):
assert required_scope("GET", "/v1/api/workstreams") == "read"
@@ -177,10 +135,10 @@ class TestRequiredScope:
assert required_scope("POST", "/v1/api/cluster/workstreams/new") == "write"
def test_proxy_v1_send_needs_write(self):
assert required_scope("POST", "/node/node-a/v1/api/workstreams/abc/send") == "write"
assert required_scope("POST", "/node/node-a/v1/api/send") == "write"
def test_proxy_v1_approve_needs_approve(self):
assert required_scope("POST", "/node/node-a/v1/api/workstreams/abc/approve") == "approve"
assert required_scope("POST", "/node/node-a/v1/api/approve") == "approve"
def test_proxy_v1_read_endpoint_needs_read(self):
assert required_scope("GET", "/node/node-a/v1/api/workstreams") == "read"
@@ -240,30 +198,6 @@ class TestRequiredScope:
"""Only POST is elevated — GET falls through to read."""
assert required_scope("GET", "/api/_internal/mcp-reload") == "read"
def test_internal_mcp_refresh_one_needs_approve(self):
assert required_scope("POST", "/api/_internal/mcp-refresh/srv") == "approve"
def test_v1_internal_mcp_refresh_one_needs_approve(self):
assert required_scope("POST", "/v1/api/_internal/mcp-refresh/srv") == "approve"
def test_proxy_internal_mcp_refresh_one_needs_approve(self):
assert required_scope("POST", "/node/n1/v1/api/_internal/mcp-refresh/srv") == "approve"
def test_proxy_no_v1_internal_mcp_refresh_one_needs_approve(self):
assert required_scope("POST", "/node/n1/api/_internal/mcp-refresh/srv") == "approve"
def test_internal_mcp_reconnect_one_needs_approve(self):
assert required_scope("POST", "/api/_internal/mcp-reconnect/srv") == "approve"
def test_v1_internal_mcp_reconnect_one_needs_approve(self):
assert required_scope("POST", "/v1/api/_internal/mcp-reconnect/srv") == "approve"
def test_proxy_internal_mcp_reconnect_one_needs_approve(self):
assert required_scope("POST", "/node/n1/v1/api/_internal/mcp-reconnect/srv") == "approve"
def test_proxy_no_v1_internal_mcp_reconnect_one_needs_approve(self):
assert required_scope("POST", "/node/n1/api/_internal/mcp-reconnect/srv") == "approve"
# Workstream sub-resource mutations (parametric paths)
def test_ws_delete_needs_write(self):
assert required_scope("POST", "/api/workstreams/abc123/delete") == "write"
@@ -468,7 +402,7 @@ class TestCheckRequest:
def test_write_read_token_403(self, read_jwt):
allowed, status, msg, _result = check_request(
"POST", "/api/workstreams/abc/send", read_jwt, jwt_secret=self._SECRET
"POST", "/api/send", read_jwt, jwt_secret=self._SECRET
)
assert allowed is False
assert status == 403
@@ -476,14 +410,14 @@ class TestCheckRequest:
def test_write_full_token_ok(self, full_jwt):
allowed, status, msg, _result = check_request(
"POST", "/api/workstreams/abc/send", full_jwt, jwt_secret=self._SECRET
"POST", "/api/send", full_jwt, jwt_secret=self._SECRET
)
assert allowed is True
assert status == 200
def test_approve_read_token_403(self, read_jwt):
allowed, status, msg, _result = check_request(
"POST", "/api/workstreams/abc/approve", read_jwt, jwt_secret=self._SECRET
"POST", "/api/approve", read_jwt, jwt_secret=self._SECRET
)
assert allowed is False
assert status == 403
@@ -491,10 +425,7 @@ class TestCheckRequest:
def test_proxy_write_read_token_403(self, read_jwt):
"""Read tokens cannot escalate to write ops via proxy routes."""
allowed, status, msg, _result = check_request(
"POST",
"/node/node-a/api/workstreams/abc/send",
read_jwt,
jwt_secret=self._SECRET,
"POST", "/node/node-a/api/send", read_jwt, jwt_secret=self._SECRET
)
assert allowed is False
assert status == 403
@@ -502,10 +433,7 @@ class TestCheckRequest:
def test_proxy_write_trailing_slash_read_token_403(self, read_jwt):
"""Trailing slash must not bypass write-role check on proxy routes."""
allowed, status, msg, _result = check_request(
"POST",
"/node/node-a/api/workstreams/abc/send/",
read_jwt,
jwt_secret=self._SECRET,
"POST", "/node/node-a/api/send/", read_jwt, jwt_secret=self._SECRET
)
assert allowed is False
assert status == 403
@@ -513,7 +441,7 @@ class TestCheckRequest:
def test_direct_write_trailing_slash_read_token_403(self, read_jwt):
"""Trailing slash must not bypass write-role check on direct routes."""
allowed, status, msg, _result = check_request(
"POST", "/api/workstreams/abc/send/", read_jwt, jwt_secret=self._SECRET
"POST", "/api/send/", read_jwt, jwt_secret=self._SECRET
)
assert allowed is False
assert status == 403
@@ -521,20 +449,14 @@ class TestCheckRequest:
def test_proxy_write_full_token_ok(self, full_jwt):
"""Full tokens pass through proxy write routes."""
allowed, status, msg, _result = check_request(
"POST",
"/node/node-a/api/workstreams/abc/send",
full_jwt,
jwt_secret=self._SECRET,
"POST", "/node/node-a/api/send", full_jwt, jwt_secret=self._SECRET
)
assert allowed is True
def test_proxy_v1_write_read_token_403(self, read_jwt):
"""Read tokens cannot escalate to write ops via v1 proxy routes."""
allowed, status, msg, _result = check_request(
"POST",
"/node/node-a/v1/api/workstreams/abc/send",
read_jwt,
jwt_secret=self._SECRET,
"POST", "/node/node-a/v1/api/send", read_jwt, jwt_secret=self._SECRET
)
assert allowed is False
assert status == 403
@@ -542,10 +464,7 @@ class TestCheckRequest:
def test_proxy_v1_write_full_token_ok(self, full_jwt):
"""Full tokens pass through v1 proxy write routes."""
allowed, status, msg, _result = check_request(
"POST",
"/node/node-a/v1/api/workstreams/abc/send",
full_jwt,
jwt_secret=self._SECRET,
"POST", "/node/node-a/v1/api/send", full_jwt, jwt_secret=self._SECRET
)
assert allowed is True
@@ -577,7 +496,7 @@ class TestCheckRequest:
def test_approve_full_token_ok(self, full_jwt):
allowed, status, msg, _result = check_request(
"POST", "/api/workstreams/abc/approve", full_jwt, jwt_secret=self._SECRET
"POST", "/api/approve", full_jwt, jwt_secret=self._SECRET
)
assert allowed is True
@@ -619,7 +538,7 @@ class TestCheckRequestWithCookie:
def test_bearer_takes_precedence_over_cookie(self, read_jwt, full_jwt):
allowed, status, _, _r = check_request(
"POST",
"/api/workstreams/abc/send",
"/api/send",
f"Bearer {full_jwt}",
cookie_header=f"turnstone_auth={read_jwt}",
jwt_secret=self._SECRET,
@@ -640,7 +559,7 @@ class TestCheckRequestWithCookie:
def test_cookie_read_on_write_403(self, read_jwt):
allowed, status, _, _r = check_request(
"POST",
"/api/workstreams/abc/send",
"/api/send",
None,
cookie_header=f"turnstone_auth={read_jwt}",
jwt_secret=self._SECRET,
@@ -651,7 +570,7 @@ class TestCheckRequestWithCookie:
def test_cookie_full_on_write_ok(self, full_jwt):
allowed, status, _, _r = check_request(
"POST",
"/api/workstreams/abc/send",
"/api/send",
None,
cookie_header=f"turnstone_auth={full_jwt}",
jwt_secret=self._SECRET,
@@ -716,15 +635,9 @@ class TestServerAuth:
mock_ws.name = "test"
mock_ws.state = WorkstreamState.IDLE
mock_ws.session = mock_session
# Set kind / parent_ws_id / user_id explicitly so list_workstreams
# JSON-serializes them — a bare MagicMock attribute returns another
# MagicMock that fails json.dumps and surfaces as 500.
mock_ws.kind = "interactive"
mock_ws.parent_ws_id = None
mock_ws.user_id = "u1"
mock_mgr = MagicMock()
mock_mgr.list_all.return_value = [mock_ws]
mock_mgr.max_active = 10
mock_mgr.max_workstreams = 10
from turnstone.core.auth import JWT_AUD_SERVER
@@ -781,25 +694,25 @@ class TestServerAuth:
def test_api_send_read_token_403(self):
resp = self.client.post(
"/v1/api/workstreams/x/send",
"/v1/api/send",
headers=self._read_hdr,
json={"message": "hello"},
json={"message": "hello", "ws_id": "x"},
)
assert resp.status_code == 403
assert "Forbidden" in resp.json().get("error", "")
def test_api_send_full_token_passes_auth(self):
resp = self.client.post(
"/v1/api/workstreams/nonexistent/send",
"/v1/api/send",
headers=self._full_hdr,
json={"message": "hello"},
json={"message": "hello", "ws_id": "nonexistent"},
)
assert resp.status_code not in (401, 403)
def test_api_send_no_token_401(self):
resp = self.client.post(
"/v1/api/workstreams/x/send",
json={"message": "hello"},
"/v1/api/send",
json={"message": "hello", "ws_id": "x"},
)
assert resp.status_code == 401
@@ -812,7 +725,7 @@ class TestServerAuth:
def test_options_no_auth_required(self):
resp = self.client.options(
"/v1/api/workstreams/x/send",
"/v1/api/send",
headers={
"Origin": "http://example.com",
"Access-Control-Request-Method": "POST",
@@ -938,15 +851,9 @@ class TestServerLogin:
mock_ws.name = "test"
mock_ws.state = WorkstreamState.IDLE
mock_ws.session = mock_session
# Set kind / parent_ws_id / user_id explicitly so list_workstreams
# JSON-serializes them — a bare MagicMock attribute returns another
# MagicMock that fails json.dumps and surfaces as 500.
mock_ws.kind = "interactive"
mock_ws.parent_ws_id = None
mock_ws.user_id = "u1"
mock_mgr = MagicMock()
mock_mgr.list_all.return_value = [mock_ws]
mock_mgr.max_active = 10
mock_mgr.max_workstreams = 10
# Mock storage with a test user for password login
from turnstone.core.auth import hash_password
@@ -1034,163 +941,6 @@ class TestServerLogin:
resp = self.test_client.get("/v1/api/workstreams")
assert resp.status_code == 401
def test_whoami_includes_exp(self):
"""whoami exposes the JWT exp so the frontend can schedule refresh."""
import time
self.test_client.post(
"/v1/api/auth/login",
json={"username": "testuser", "password": "testpass"},
)
resp = self.test_client.get("/v1/api/auth/whoami")
assert resp.status_code == 200
data = resp.json()
assert "exp" in data
# Default JWT TTL is 24h; exp should be > now and < now + 25h.
now = int(time.time())
assert now < data["exp"] < now + 25 * 3600
def test_refresh_returns_new_jwt_and_cookie(self):
"""POST /api/auth/refresh re-mints the cookie with a fresh exp."""
from turnstone.core.auth import AUTH_COOKIE
# Storage needs get_user_permissions for the refresh re-resolve path.
# Mock is shared across tests in the class — re-arm here in case a
# prior test left it default.
self.test_client.app.state.auth_storage.get_user_permissions.return_value = {
"read",
"write",
"approve",
}
login = self.test_client.post(
"/v1/api/auth/login",
json={"username": "testuser", "password": "testpass"},
)
assert login.status_code == 200
refresh = self.test_client.post("/v1/api/auth/refresh")
assert refresh.status_code == 200
body = refresh.json()
assert body["status"] == "ok"
assert body["user_id"] == "uid_test"
assert "jwt" in body
# Set-Cookie header must be present so the browser updates. Don't
# assert the new JWT differs from the original — sub-second login
# and refresh produce identical iat/exp claims and therefore an
# identical token, which is fine: the cookie still gets re-set.
cookie_hdr = refresh.headers.get("set-cookie", "")
assert AUTH_COOKIE in cookie_hdr
assert "HttpOnly" in cookie_hdr
# The refreshed cookie must keep working.
resp = self.test_client.get("/v1/api/workstreams")
assert resp.status_code == 200
def test_refresh_response_includes_exp_and_permissions(self):
"""Refresh response shape must match whoami so the frontend can
populate sessionStorage + reschedule the next refresh off the
single round-trip without a follow-up /whoami call."""
import time
self.test_client.app.state.auth_storage.get_user_permissions.return_value = {
"read",
"write",
"approve",
}
login = self.test_client.post(
"/v1/api/auth/login",
json={"username": "testuser", "password": "testpass"},
)
assert login.status_code == 200
refresh = self.test_client.post("/v1/api/auth/refresh")
assert refresh.status_code == 200
body = refresh.json()
# exp present + within the expected default JWT TTL window
assert "exp" in body, body
now = int(time.time())
assert now < body["exp"] < now + 25 * 3600, body
# permissions present + non-empty (matches the seeded role set)
assert body.get("permissions"), body
assert "write" in body["permissions"].split(",")
def test_refresh_unauthenticated_401(self):
"""Refresh requires a currently-valid cookie — no cookie → 401."""
# Clear cookies on the test client
self.test_client.cookies.clear()
resp = self.test_client.post("/v1/api/auth/refresh")
assert resp.status_code == 401
def test_refresh_storage_failure_falls_back(self):
"""Transient storage error → fall back to in-token claims, not 403.
The earlier implementation called _load_user_permissions() which
swallows exceptions and returns set(); that path was
indistinguishable from a deleted user (legitimate 403). The
handler now calls storage.get_user_permissions() directly so
DB hiccups fall through to in-token perms.
"""
# Re-arm the storage so login works first
self.test_client.app.state.auth_storage.get_user_permissions.return_value = {
"read",
"write",
"approve",
}
login = self.test_client.post(
"/v1/api/auth/login",
json={"username": "testuser", "password": "testpass"},
)
assert login.status_code == 200
# Now make storage raise on the refresh re-resolve
self.test_client.app.state.auth_storage.get_user_permissions.side_effect = RuntimeError(
"db down"
)
try:
resp = self.test_client.post("/v1/api/auth/refresh")
assert resp.status_code == 200, resp.text
body = resp.json()
# Permissions should still be present (fell back to in-token claims)
assert body.get("permissions"), body
finally:
# Restore for any subsequent tests
self.test_client.app.state.auth_storage.get_user_permissions.side_effect = None
self.test_client.app.state.auth_storage.get_user_permissions.return_value = {
"read",
"write",
"approve",
}
def test_refresh_user_with_no_perms_403(self):
"""Storage returns empty (user deleted/role-stripped) → 403.
Distinguished from the storage-failure case above because
get_user_permissions returned a value (the empty set) without
raising that's an authoritative "no roles", not a hiccup.
"""
self.test_client.app.state.auth_storage.get_user_permissions.return_value = {
"read",
"write",
"approve",
}
login = self.test_client.post(
"/v1/api/auth/login",
json={"username": "testuser", "password": "testpass"},
)
assert login.status_code == 200
self.test_client.app.state.auth_storage.get_user_permissions.return_value = set()
try:
resp = self.test_client.post("/v1/api/auth/refresh")
assert resp.status_code == 403
finally:
self.test_client.app.state.auth_storage.get_user_permissions.return_value = {
"read",
"write",
"approve",
}
class TestConsoleLogin:
"""Test login/logout cookie flow on turnstone-console."""
@@ -1378,56 +1128,6 @@ class TestJWTAudienceIssuer:
result = validate_jwt(token, self.SECRET, audience="")
assert result is not None
def test_validate_jwt_accepts_within_leeway_after_expiry(self):
"""validate_jwt has 30s leeway for clock skew across hosts/processes."""
import time
import jwt as pyjwt
from turnstone.core.auth import JWT_ISSUER, validate_jwt
# Mint a token that "expired" 10 seconds ago — still within 30s leeway.
now = int(time.time())
token = pyjwt.encode(
{
"sub": "user1",
"scopes": "read",
"src": "test",
"iss": JWT_ISSUER,
"iat": now - 100,
"exp": now - 10,
},
self.SECRET,
algorithm="HS256",
)
result = validate_jwt(token, self.SECRET, audience="")
assert result is not None
assert result.user_id == "user1"
def test_validate_jwt_rejects_past_leeway(self):
"""Tokens expired beyond the 30s leeway must still be rejected."""
import time
import jwt as pyjwt
from turnstone.core.auth import JWT_ISSUER, validate_jwt
now = int(time.time())
token = pyjwt.encode(
{
"sub": "user1",
"scopes": "read",
"src": "test",
"iss": JWT_ISSUER,
"iat": now - 200,
"exp": now - 60,
},
self.SECRET,
algorithm="HS256",
)
result = validate_jwt(token, self.SECRET, audience="")
assert result is None
def test_create_jwt_expiry_seconds(self):
import jwt as pyjwt
@@ -1752,7 +1452,7 @@ class TestCorsConfigurable:
mgr = MagicMock()
mgr.list_all.return_value = []
mgr.max_active = 10
mgr.max_workstreams = 10
app = srv_mod.create_app(
workstreams=mgr,
global_queue=queue.Queue(),
@@ -1773,7 +1473,7 @@ class TestCorsConfigurable:
mgr = MagicMock()
mgr.list_all.return_value = []
mgr.max_active = 10
mgr.max_workstreams = 10
app = srv_mod.create_app(
workstreams=mgr,
global_queue=queue.Queue(),
+11 -11
View File
@@ -175,10 +175,10 @@ class TestRequiredScope:
assert required_scope("GET", "/api/workstreams") == "read"
def test_post_write(self):
assert required_scope("POST", "/api/workstreams/abc/send") == "write"
assert required_scope("POST", "/api/send") == "write"
def test_post_approve(self):
assert required_scope("POST", "/api/workstreams/abc/approve") == "approve"
assert required_scope("POST", "/api/approve") == "approve"
def test_admin_prefix(self):
assert required_scope("GET", "/api/admin/users") == "approve"
@@ -186,14 +186,14 @@ class TestRequiredScope:
assert required_scope("DELETE", "/api/admin/users/abc") == "approve"
def test_versioned_path(self):
assert required_scope("POST", "/v1/api/workstreams/abc/send") == "write"
assert required_scope("POST", "/v1/api/workstreams/abc/approve") == "approve"
assert required_scope("POST", "/v1/api/send") == "write"
assert required_scope("POST", "/v1/api/approve") == "approve"
def test_proxy_write(self):
assert required_scope("POST", "/node/n1/api/workstreams/abc/send") == "write"
assert required_scope("POST", "/node/n1/api/send") == "write"
def test_proxy_approve(self):
assert required_scope("POST", "/node/n1/api/workstreams/abc/approve") == "approve"
assert required_scope("POST", "/node/n1/api/approve") == "approve"
# ---------------------------------------------------------------------------
@@ -270,7 +270,7 @@ class TestCheckRequestScopes:
jwt_tok = create_jwt("u1", frozenset({"read"}), "test", self._SECRET)
allowed, status, msg, _ = check_request(
"POST",
"/api/workstreams/abc/send",
"/api/send",
f"Bearer {jwt_tok}",
jwt_secret=self._SECRET,
)
@@ -282,7 +282,7 @@ class TestCheckRequestScopes:
jwt_tok = create_jwt("u1", frozenset({"read"}), "test", self._SECRET)
allowed, status, msg, _ = check_request(
"POST",
"/api/workstreams/abc/approve",
"/api/approve",
f"Bearer {jwt_tok}",
jwt_secret=self._SECRET,
)
@@ -294,7 +294,7 @@ class TestCheckRequestScopes:
jwt_tok = create_jwt("u1", frozenset({"read", "write", "approve"}), "test", self._SECRET)
allowed, status, msg, result = check_request(
"POST",
"/api/workstreams/abc/approve",
"/api/approve",
f"Bearer {jwt_tok}",
jwt_secret=self._SECRET,
)
@@ -306,7 +306,7 @@ class TestCheckRequestScopes:
jwt_tok = create_jwt("u1", frozenset({"read", "write"}), "db", self._SECRET)
allowed, status, msg, result = check_request(
"POST",
"/api/workstreams/abc/send",
"/api/send",
f"Bearer {jwt_tok}",
jwt_secret=self._SECRET,
)
@@ -318,7 +318,7 @@ class TestCheckRequestScopes:
jwt_tok = create_jwt("u1", frozenset({"read"}), "db", self._SECRET)
allowed, status, msg, _ = check_request(
"POST",
"/api/workstreams/abc/send",
"/api/send",
f"Bearer {jwt_tok}",
jwt_secret=self._SECRET,
)
-266
View File
@@ -1,266 +0,0 @@
"""Tests for ``turnstone.server._build_history`` reminder + source surfacing.
The replay path (``_build_history``) projects the ``_source`` and
``_reminders`` side-channels onto the wire entry the frontend
consumes. Persisted via migration 050 (Commit 1) so multi-tab /
multi-device replay sees the same metacognitive bubble shape the
originating tab saw live.
"""
from __future__ import annotations
from types import SimpleNamespace
from typing import Any
from unittest.mock import patch
from turnstone.server import _build_history
def _make_stub_session(messages: list[dict[str, Any]]) -> Any:
"""Minimal ChatSession-shaped stub. ``_build_history`` only reads
``session.messages`` plus calls ``_load_verdict_indexes(ws_id)``
the latter we patch out below.
"""
return SimpleNamespace(messages=messages, _ws_id="ws-test")
def _build(messages: list[dict[str, Any]]) -> list[dict[str, Any]]:
"""Run ``_build_history`` against a stub session, bypassing the
verdicts / output-assessment storage round-trip (no tool_calls in
these tests, so the indexes are unused anyway).
"""
session = _make_stub_session(messages)
with patch(
"turnstone.server._load_verdict_indexes",
return_value=({}, {}),
):
return _build_history(session)
class TestSourceSurfacing:
def test_source_surfaces_when_set(self) -> None:
msg = {
"role": "user",
"content": "",
"_source": "system_nudge",
}
history = _build([msg])
assert len(history) == 1
assert history[0]["source"] == "system_nudge"
def test_source_absent_when_unset(self) -> None:
msg = {"role": "user", "content": "hello"}
history = _build([msg])
assert "source" not in history[0]
class TestRemindersWidening:
def test_watch_triggered_optional_fields_propagate(self) -> None:
"""The widened payload (Commit 2) carries watch_name / command /
poll_count / max_polls / is_final on each ``watch_triggered``
reminder so the frontend renders ``.msg.watch-result``.
"""
msg = {
"role": "user",
"content": "",
"_source": "system_nudge",
"_reminders": [
{
"type": "watch_triggered",
"text": "$ ls\nfile.txt",
"watch_name": "w1",
"command": "ls",
"poll_count": 2,
"max_polls": 100,
"is_final": False,
}
],
}
history = _build([msg])
assert history[0]["source"] == "system_nudge"
assert history[0]["reminders"] == [
{
"type": "watch_triggered",
"text": "$ ls\nfile.txt",
"watch_name": "w1",
"command": "ls",
"poll_count": 2,
"max_polls": 100,
"is_final": False,
}
]
def test_legacy_two_field_reminders_still_work(self) -> None:
"""Producers without optional fields (correction / denial /
idle_children) keep the legacy ``{type, text}`` shape the
widened filter just doesn't add anything beyond that."""
msg = {
"role": "user",
"content": "noted",
"_reminders": [{"type": "correction", "text": "watch out"}],
}
history = _build([msg])
assert history[0]["reminders"] == [{"type": "correction", "text": "watch out"}]
def test_unknown_keys_are_dropped(self) -> None:
"""The wire-layer filter projects on a known set of keys so a
future producer accidentally stuffing arbitrary fields can't
leak them through replay.
"""
msg = {
"role": "user",
"content": "x",
"_reminders": [
{
"type": "correction",
"text": "hi",
"secret": "leak-me",
"internal_id": 42,
}
],
}
history = _build([msg])
clean = history[0]["reminders"][0]
assert "secret" not in clean
assert "internal_id" not in clean
assert clean == {"type": "correction", "text": "hi"}
def test_malformed_reminder_skipped(self) -> None:
"""A non-dict / empty entry is filtered out instead of breaking
the rest of the list (mirrors the defensive filter in
``_apply_reminders_for_provider``).
"""
msg = {
"role": "user",
"content": "x",
"_reminders": [
"garbage string",
{"type": "", "text": ""}, # empty type + text → drop
{"type": "denial", "text": "ok"},
],
}
history = _build([msg])
assert history[0]["reminders"] == [{"type": "denial", "text": "ok"}]
class _StubRegistry:
"""Minimal model registry — only ``get_config`` is read by
``_build_history``."""
def __init__(self, surface_persisted_reasoning: bool = True) -> None:
self._cfg = SimpleNamespace(surface_persisted_reasoning=surface_persisted_reasoning)
def get_config(self, alias: str) -> Any:
return self._cfg
def _build_with_registry(
messages: list[dict[str, Any]],
surface_persisted_reasoning: bool = True,
) -> list[dict[str, Any]]:
session = SimpleNamespace(
messages=messages,
_ws_id="ws-test",
_registry=_StubRegistry(surface_persisted_reasoning=surface_persisted_reasoning),
_model_alias="claude-opus-4-7",
)
with patch(
"turnstone.server._load_verdict_indexes",
return_value=({}, {}),
):
return _build_history(session)
class TestReasoningSurfacing:
"""Phase 1 — surface stored Anthropic thinking blocks on the
history payload so refresh-the-page rehydrates the reasoning bubble.
Drives through the real ``AnthropicProvider`` extractor (no mock-of-
extractor) only the model registry is stubbed.
"""
def test_reasoning_surfaces_for_anthropic_thinking_msg(self) -> None:
msg = {
"role": "assistant",
"content": "Final answer.",
"_provider_content": [
{"type": "thinking", "thinking": "let me think", "signature": "s"},
{"type": "text", "text": "Final answer."},
],
}
history = _build_with_registry([msg], surface_persisted_reasoning=True)
assert len(history) == 1
assert history[0]["reasoning"] == "let me think"
def test_reasoning_empty_when_persist_flag_false(self) -> None:
msg = {
"role": "assistant",
"content": "Final answer.",
"_provider_content": [
{"type": "thinking", "thinking": "hidden", "signature": "s"},
],
}
history = _build_with_registry([msg], surface_persisted_reasoning=False)
assert "reasoning" not in history[0]
def test_provider_content_never_in_wire_entry(self) -> None:
# The build path does not copy ``_provider_content`` into the
# entry dict regardless of flag — wire payload stays tight.
msg = {
"role": "assistant",
"content": "Final answer.",
"_provider_content": [
{"type": "thinking", "thinking": "x", "signature": "s"},
],
}
history = _build_with_registry([msg], surface_persisted_reasoning=True)
assert "_provider_content" not in history[0]
def test_no_reasoning_field_when_provider_content_missing(self) -> None:
msg = {"role": "assistant", "content": "plain answer"}
history = _build_with_registry([msg], surface_persisted_reasoning=True)
assert "reasoning" not in history[0]
def test_no_reasoning_field_for_non_assistant_messages(self) -> None:
# Defensive — user/tool messages with a stray _provider_content
# do not get the reasoning field stamped.
msgs: list[dict[str, Any]] = [
{"role": "user", "content": "hi"},
{
"role": "tool",
"tool_call_id": "c1",
"content": "out",
"_provider_content": [{"type": "thinking", "thinking": "leak", "signature": "s"}],
},
]
history = _build_with_registry(msgs, surface_persisted_reasoning=True)
assert "reasoning" not in history[0]
assert "reasoning" not in history[1]
def test_default_true_when_registry_lookup_raises(self) -> None:
# Conservative default — Phase 1 spec mandates rehydration on
# refresh. A registry/alias mismatch must not silently kill the
# bubble.
class BrokenRegistry:
def get_config(self, alias: str) -> Any:
raise KeyError(alias)
session = SimpleNamespace(
messages=[
{
"role": "assistant",
"content": "x",
"_provider_content": [
{"type": "thinking", "thinking": "still works", "signature": "s"}
],
}
],
_ws_id="ws-test",
_registry=BrokenRegistry(),
_model_alias="missing-alias",
)
with patch(
"turnstone.server._load_verdict_indexes",
return_value=({}, {}),
):
history = _build_history(session)
assert history[0]["reasoning"] == "still works"
+4 -127
View File
@@ -19,12 +19,6 @@ class NullUI:
self.infos = []
self.stream_ends = 0
def on_turn_start(self):
pass
def on_turn_committed(self):
pass
def on_thinking_start(self):
pass
@@ -174,18 +168,10 @@ class TestCancelDuringStreaming:
assert ui.states[-1] == "idle"
# Check that "[Generation cancelled]" was emitted
assert any("cancelled" in i.lower() for i in ui.infos)
# The partial content should be preserved as an assistant
# message AND annotated with a marker that downstream readers
# (inspect_workstream, the next coord turn) can use to
# distinguish a cancelled fragment from a completed turn — the
# raw "Hello world" without a marker would look like the
# final assistant answer to a coord LLM reading the child's
# transcript.
# The partial content should be preserved as an assistant message
assistant_msgs = [m for m in session.messages if m["role"] == "assistant"]
assert len(assistant_msgs) == 1
content = assistant_msgs[0]["content"]
assert content.startswith("Hello world")
assert "[generation cancelled before completion]" in content
assert assistant_msgs[0]["content"] == "Hello world"
# No tool_calls in the partial message
assert "tool_calls" not in assistant_msgs[0]
@@ -525,13 +511,10 @@ class TestStreamAbort:
# Should complete as cancelled, not error
assert "idle" in ui.states
assert any("cancelled" in i.lower() for i in ui.infos)
# Partial content preserved AND annotated with the
# cancelled-before-completion marker.
# Partial content preserved
assistant_msgs = [m for m in session.messages if m["role"] == "assistant"]
assert len(assistant_msgs) == 1
content = assistant_msgs[0]["content"]
assert content.startswith("Hello")
assert "[generation cancelled before completion]" in content
assert assistant_msgs[0]["content"] == "Hello"
def test_non_cancel_exception_not_swallowed(self, tmp_db):
"""Exceptions during streaming that aren't caused by cancel
@@ -826,109 +809,3 @@ class TestForceCancelThreaded:
assert "idle" in ui.states
assistant_msgs = [m for m in session.messages if m["role"] == "assistant"]
assert any("Fresh response" in m.get("content", "") for m in assistant_msgs)
class TestSynthesizeCancelledResults:
"""Regression coverage for ``_synthesize_cancelled_results`` — must
fire ``on_tool_result`` for each synthesized cancellation so live
SSE listeners (e.g. coord's ``--running`` indicator added by
tool_info) can complete the in-DOM tool batch. Without this, the
coord JS would spin the running indicator forever on cancelled
batches because ``state_change`` doesn't strip ``--running`` from
individual batches."""
def _ui_with_tool_result_tracking(self):
class _TrackingUI(NullUI):
def __init__(self) -> None:
super().__init__()
self.tool_results: list[tuple[str, str, str, bool]] = []
def on_tool_result(self, call_id, name, output, **kwargs):
self.tool_results.append(
(call_id, name, output, bool(kwargs.get("is_error", False))),
)
return _TrackingUI()
def test_synthesizes_tool_result_for_unanswered_calls(self, tmp_db):
ui = self._ui_with_tool_result_tracking()
session = _make_session(ui=ui)
session.messages.append(
{
"role": "assistant",
"content": "calling tools",
"tool_calls": [
{"id": "call_a", "function": {"name": "search", "arguments": "{}"}},
{"id": "call_b", "function": {"name": "compute", "arguments": "{}"}},
],
},
)
session._msg_tokens.append(1)
session._synthesize_cancelled_results("Cancelled by user.")
# Both unanswered calls fired ``on_tool_result``.
assert len(ui.tool_results) == 2
ids = {tr[0] for tr in ui.tool_results}
assert ids == {"call_a", "call_b"}
# All emitted as errors so the live UI renders them as
# ``coord-tool-row-result--error``.
assert all(tr[3] is True for tr in ui.tool_results)
# Reason text propagates as the synthetic tool output.
assert all(tr[2] == "Cancelled by user." for tr in ui.tool_results)
# And the message list has the synthesized tool entries
# (preserves the prior contract).
tool_msgs = [m for m in session.messages if m.get("role") == "tool"]
assert len(tool_msgs) == 2
def test_skips_calls_already_answered(self, tmp_db):
ui = self._ui_with_tool_result_tracking()
session = _make_session(ui=ui)
session.messages.append(
{
"role": "assistant",
"tool_calls": [
{"id": "call_a", "function": {"name": "search", "arguments": "{}"}},
{"id": "call_b", "function": {"name": "compute", "arguments": "{}"}},
],
},
)
session._msg_tokens.append(1)
# call_a already answered.
session.messages.append(
{"role": "tool", "tool_call_id": "call_a", "content": "result"},
)
session._msg_tokens.append(1)
session._synthesize_cancelled_results("Cancelled by user.")
# Only call_b synthesized.
assert len(ui.tool_results) == 1
assert ui.tool_results[0][0] == "call_b"
def test_ui_emit_failure_does_not_break_synthesis(self, tmp_db):
"""The UI hook is wrapped in try/except — a hook failure
during cancel must NOT compound the problem. Synthesis still
appends to messages + storage."""
class _ExplodingUI(NullUI):
def on_tool_result(self, call_id, name, output, **kwargs):
raise RuntimeError("ui hook blew up")
ui = _ExplodingUI()
session = _make_session(ui=ui)
session.messages.append(
{
"role": "assistant",
"tool_calls": [
{"id": "call_a", "function": {"name": "search", "arguments": "{}"}},
],
},
)
session._msg_tokens.append(1)
# Must not raise.
session._synthesize_cancelled_results("Cancelled by user.")
tool_msgs = [m for m in session.messages if m.get("role") == "tool"]
assert len(tool_msgs) == 1
+47 -381
View File
@@ -28,21 +28,6 @@ def _run(coro):
return asyncio.run(coro)
def _bind_ws_event_handlers(bot, cls):
"""Bind ``_on_ws_event`` + every ``_handle_*`` method from *cls* to *bot*.
``MagicMock(spec=cls)`` stubs async methods as ``AsyncMock`` no-ops,
so dispatcher tests that invoke the real ``_on_ws_event`` must also
bind the per-event handlers it delegates to.
"""
bot._on_ws_event = cls._on_ws_event.__get__(bot, cls)
for name in dir(cls):
if name.startswith("_handle_"):
attr = getattr(cls, name)
if callable(attr):
setattr(bot, name, attr.__get__(bot, cls))
def _make_message(*, bot=False, guild=True, content="hello", channel=None, reference=None):
"""Build a mock ``discord.Message``."""
msg = MagicMock(spec=discord.Message)
@@ -143,7 +128,7 @@ class TestStreamingMessage:
_run(sm.append("hello "))
_run(sm.append("world"))
assert sm.accumulated_text == "hello world"
assert "".join(sm._buffer) == "hello world"
def test_finalize_sends_when_no_prior_message(self):
from turnstone.channels.discord.bot import StreamingMessage
@@ -168,7 +153,7 @@ class TestStreamingMessage:
# First append triggers flush (interval=0) which creates the message.
_run(sm.append("hi"))
assert sm.message is sent_msg
assert sm._message is sent_msg
_run(sm.append(" there"))
_run(sm.finalize())
@@ -367,20 +352,20 @@ class TestAskModelSelection:
class TestParseFooter:
"""Tests for _parse_footer in views.py."""
def test_valid_footer_with_owner(self):
def test_valid_footer(self):
from turnstone.channels.discord.views import _parse_footer
interaction = _make_interaction(footer_text="ws_abc|corr_123|12345")
result = _parse_footer(interaction)
assert result == ("ws_abc", "corr_123", "12345")
def test_footer_without_owner_returns_empty_owner(self):
from turnstone.channels.discord.views import _parse_footer
# Legacy footer without an owner field (pre-upgrade posts).
interaction = _make_interaction(footer_text="ws_abc|corr_123")
result = _parse_footer(interaction)
assert result == ("ws_abc", "corr_123", "")
assert result == ("ws_abc", "corr_123")
def test_footer_with_pipe_in_correlation(self):
from turnstone.channels.discord.views import _parse_footer
interaction = _make_interaction(footer_text="ws_abc|corr|extra")
result = _parse_footer(interaction)
# split("|", 1) means the second part includes everything after first pipe.
assert result == ("ws_abc", "corr|extra")
def test_no_message_returns_none(self):
from turnstone.channels.discord.views import _parse_footer
@@ -443,7 +428,7 @@ class TestWsEventFinalization:
bot._notify_reply_channels = {}
# Use the real _on_ws_event method
_bind_ws_event_handlers(bot, TurnstoneBot)
bot._on_ws_event = TurnstoneBot._on_ws_event.__get__(bot, TurnstoneBot)
thread = AsyncMock()
@@ -472,7 +457,7 @@ class TestWsEventFinalization:
bot._tool_info_msgs = {}
bot._pending_approval_msgs = {}
bot._notify_reply_channels = {}
_bind_ws_event_handlers(bot, TurnstoneBot)
bot._on_ws_event = TurnstoneBot._on_ws_event.__get__(bot, TurnstoneBot)
thread = AsyncMock()
@@ -493,7 +478,6 @@ class TestApprovalVerdictDisplay:
def _make_bot(self):
"""Build a mock TurnstoneBot with _on_ws_event bound."""
from turnstone.channels._routing import PolicyVerdict
from turnstone.channels.discord.bot import TurnstoneBot
bot = MagicMock(spec=TurnstoneBot)
@@ -509,9 +493,7 @@ class TestApprovalVerdictDisplay:
bot._pending_approval_msgs = {}
bot._notify_reply_channels = {}
bot._should_auto_approve = MagicMock(return_value=False)
bot.router = MagicMock()
bot.router.evaluate_tool_policies = AsyncMock(return_value=PolicyVerdict(kind="none"))
_bind_ws_event_handlers(bot, TurnstoneBot)
bot._on_ws_event = TurnstoneBot._on_ws_event.__get__(bot, TurnstoneBot)
return bot
def test_approval_with_heuristic_verdict(self):
@@ -630,7 +612,7 @@ class TestApprovalVerdictDisplay:
bot._tool_info_msgs = {}
bot._pending_approval_msgs = {"ws-1": MagicMock()}
bot._notify_reply_channels = {}
_bind_ws_event_handlers(bot, TurnstoneBot)
bot._on_ws_event = TurnstoneBot._on_ws_event.__get__(bot, TurnstoneBot)
thread = AsyncMock()
event = StreamEndEvent(ws_id="ws-1")
@@ -656,7 +638,7 @@ class TestStreamEndBehavior:
bot._tool_info_msgs = {}
bot._pending_approval_msgs = {}
bot._notify_reply_channels = {}
_bind_ws_event_handlers(bot, TurnstoneBot)
bot._on_ws_event = TurnstoneBot._on_ws_event.__get__(bot, TurnstoneBot)
return bot
def test_stream_end_no_streaming_no_send(self):
@@ -692,88 +674,38 @@ class TestStreamEndBehavior:
class TestNotificationTracking:
"""Tests for notification message tracking and DM reply routing."""
def _make_dm_bot(self, *, sent_message_id: int):
"""Build a MagicMock bot whose notification target resolves to a DM."""
def test_send_notification_tracks_message(self):
"""send_notification should store message_id -> (ws_id, target_user) mapping."""
from turnstone.channels.discord.bot import TurnstoneBot
bot = MagicMock(spec=TurnstoneBot)
bot.config = MagicMock()
bot.config.max_message_length = 2000
sent_msg = MagicMock()
sent_msg.id = sent_message_id
dm_channel = MagicMock()
dm_channel.send = AsyncMock(return_value=sent_msg)
user = MagicMock()
user.id = 7777
user.create_dm = AsyncMock(return_value=dm_channel)
inner_bot = MagicMock()
inner_bot.get_channel = MagicMock(return_value=None)
inner_bot.fetch_user = AsyncMock(return_value=user)
bot._bot = inner_bot
bot.send_notification = TurnstoneBot.send_notification.__get__(bot, TurnstoneBot)
bot._track_notification = TurnstoneBot._track_notification.__get__(bot, TurnstoneBot)
return bot
def test_send_notification_tracks_dm_with_user_id(self):
"""send_notification for a DM records (ws_id, resolved_user_id)."""
bot = self._make_dm_bot(sent_message_id=12345)
bot._notify_ws_map = {}
bot._MAX_NOTIFY_TRACKING = 100
bot.send = AsyncMock(return_value="12345")
bot.send_notification = TurnstoneBot.send_notification.__get__(bot, TurnstoneBot)
bot._track_notification = TurnstoneBot._track_notification.__get__(bot, TurnstoneBot)
_run(bot.send_notification("7777", "Hello", "ws-abc"))
_run(bot.send_notification("chan-1", "Hello", "ws-abc"))
# Tracked under the resolved Discord user ID, not the raw argument.
assert 12345 in bot._notify_ws_map
assert bot._notify_ws_map[12345] == ("ws-abc", "7777")
def test_send_notification_to_guild_channel_is_not_tracked(self):
"""Notifications delivered to a guild channel must not register reply tracking.
The reply-channel_id check treats the stored value as a Discord
user ID, so storing a channel ID would reject every legitimate
reply.
"""
from turnstone.channels.discord.bot import TurnstoneBot
bot = MagicMock(spec=TurnstoneBot)
bot.config = MagicMock()
bot.config.max_message_length = 2000
bot._notify_ws_map = {}
bot._MAX_NOTIFY_TRACKING = 100
sent_msg = MagicMock()
sent_msg.id = 99999
channel = MagicMock()
channel.send = AsyncMock(return_value=sent_msg)
inner_bot = MagicMock()
inner_bot.get_channel = MagicMock(return_value=channel)
bot._bot = inner_bot
bot.send_notification = TurnstoneBot.send_notification.__get__(bot, TurnstoneBot)
bot._track_notification = TurnstoneBot._track_notification.__get__(bot, TurnstoneBot)
_run(bot.send_notification("888888", "Hello", "ws-abc"))
assert bot._notify_ws_map == {}
assert bot._notify_ws_map[12345] == ("ws-abc", "chan-1")
def test_send_notification_evicts_old_entries(self):
"""Oldest notification tracking entries are evicted when cap is reached."""
bot = self._make_dm_bot(sent_message_id=4)
from turnstone.channels.discord.bot import TurnstoneBot
bot = MagicMock(spec=TurnstoneBot)
bot._MAX_NOTIFY_TRACKING = 3
bot._notify_ws_map = {
1: ("ws-1", "u1"),
2: ("ws-2", "u2"),
3: ("ws-3", "u3"),
}
bot.send = AsyncMock(return_value="4")
bot.send_notification = TurnstoneBot.send_notification.__get__(bot, TurnstoneBot)
bot._track_notification = TurnstoneBot._track_notification.__get__(bot, TurnstoneBot)
_run(bot.send_notification("7777", "Hello", "ws-4"))
_run(bot.send_notification("chan-1", "Hello", "ws-4"))
assert 4 in bot._notify_ws_map
assert 1 not in bot._notify_ws_map # oldest evicted
@@ -946,7 +878,7 @@ class TestNotificationTracking:
sent_msg.id = 88888
dm_channel.send = AsyncMock(return_value=sent_msg)
bot._notify_reply_channels = {"ws-1": (dm_channel, "u123")}
_bind_ws_event_handlers(bot, TurnstoneBot)
bot._on_ws_event = TurnstoneBot._on_ws_event.__get__(bot, TurnstoneBot)
bot._track_notification = TurnstoneBot._track_notification.__get__(bot, TurnstoneBot)
thread = AsyncMock()
@@ -981,7 +913,7 @@ class TestNotificationTracking:
dm_channel = AsyncMock()
bot._notify_reply_channels = {"ws-1": (dm_channel, "u123")}
_bind_ws_event_handlers(bot, TurnstoneBot)
bot._on_ws_event = TurnstoneBot._on_ws_event.__get__(bot, TurnstoneBot)
thread = AsyncMock()
@@ -1120,83 +1052,40 @@ class TestTryParseMedia:
class TestIsSafeImageUrl:
"""Tests for _is_safe_image_url in _formatter.py."""
@staticmethod
def _patch_resolver(monkeypatch, ips):
"""Replace socket.getaddrinfo with a stub returning *ips*."""
import socket
def fake(host, port, family=0, *args, **kwargs): # noqa: ARG001
return [(family, 0, 0, "", (ip, 0)) for ip in ips]
monkeypatch.setattr(socket, "getaddrinfo", fake)
def test_http_url(self, monkeypatch):
def test_http_url(self):
from turnstone.channels._formatter import _is_safe_image_url
self._patch_resolver(monkeypatch, ["203.0.113.5"])
assert _run(_is_safe_image_url("http://jellyfin:8096/Items/abc/Images/Primary")) is True
assert _is_safe_image_url("http://jellyfin:8096/Items/abc/Images/Primary") is True
def test_https_url(self, monkeypatch):
def test_https_url(self):
from turnstone.channels._formatter import _is_safe_image_url
self._patch_resolver(monkeypatch, ["203.0.113.5"])
assert (
_run(_is_safe_image_url("https://jellyfin.example.com/Items/abc/Images/Primary"))
is True
)
assert _is_safe_image_url("https://jellyfin.example.com/Items/abc/Images/Primary") is True
def test_ftp_rejected(self):
from turnstone.channels._formatter import _is_safe_image_url
assert _run(_is_safe_image_url("ftp://evil.com/image.jpg")) is False
assert _is_safe_image_url("ftp://evil.com/image.jpg") is False
def test_file_rejected(self):
from turnstone.channels._formatter import _is_safe_image_url
assert _run(_is_safe_image_url("file:///etc/passwd")) is False
assert _is_safe_image_url("file:///etc/passwd") is False
def test_userinfo_rejected(self):
from turnstone.channels._formatter import _is_safe_image_url
assert _run(_is_safe_image_url("http://user:pass@jellyfin:8096/image")) is False
assert _is_safe_image_url("http://user:pass@jellyfin:8096/image") is False
def test_empty_rejected(self):
from turnstone.channels._formatter import _is_safe_image_url
assert _run(_is_safe_image_url("")) is False
assert _is_safe_image_url("") is False
def test_private_ip_allowed(self):
from turnstone.channels._formatter import _is_safe_image_url
assert _run(_is_safe_image_url("http://192.168.0.6:8096/Items/abc/Images/Primary")) is True
def test_dns_rebinding_rejected(self, monkeypatch):
"""Hostname that resolves to a loopback IP must be rejected."""
from turnstone.channels._formatter import _is_safe_image_url
self._patch_resolver(monkeypatch, ["127.0.0.1"])
assert _run(_is_safe_image_url("http://rebind.example.com/image")) is False
def test_metadata_endpoint_rejected(self):
"""AWS/GCP metadata IP is link-local → rejected."""
from turnstone.channels._formatter import _is_safe_image_url
assert _run(_is_safe_image_url("http://169.254.169.254/latest/meta-data/")) is False
def test_ipv6_aws_nitro_metadata_rejected(self, monkeypatch):
"""fd00:ec2::254 is IPv6 ULA (is_private) but must be blocked —
the IPv4 169.254.169.254 check left this analogue open."""
from turnstone.channels._formatter import _is_safe_image_url
self._patch_resolver(monkeypatch, ["fd00:ec2::254"])
assert _run(_is_safe_image_url("http://nitro.example.com/")) is False
def test_ipv6_ecs_task_metadata_rejected(self, monkeypatch):
"""ECS Task Metadata lives in the same fd00:ec2::/32 prefix."""
from turnstone.channels._formatter import _is_safe_image_url
self._patch_resolver(monkeypatch, ["fd00:ec2::23"])
assert _run(_is_safe_image_url("http://ecs-meta.example.com/")) is False
assert _is_safe_image_url("http://192.168.0.6:8096/Items/abc/Images/Primary") is True
class TestBuildMediaEmbed:
@@ -1298,7 +1187,7 @@ class TestThinkingIndicator:
bot._pending_approval_msgs = {}
bot._notify_reply_channels = {}
bot._should_auto_approve = MagicMock(return_value=False)
_bind_ws_event_handlers(bot, TurnstoneBot)
bot._on_ws_event = TurnstoneBot._on_ws_event.__get__(bot, TurnstoneBot)
return bot
def test_thinking_start_sends_message(self):
@@ -1355,7 +1244,7 @@ class TestThinkingIndicator:
# Thinking message becomes the StreamingMessage base — no delete.
assert "ws-1" not in bot._thinking_msgs
sm = bot._streaming["ws-1"]
assert sm.message is thinking_msg
assert sm._message is thinking_msg
def test_stream_end_clears_thinking_message(self):
from turnstone.sdk.events import StreamEndEvent
@@ -1398,7 +1287,7 @@ class TestToolInfoEvent:
bot._pending_approval_msgs = {}
bot._notify_reply_channels = {}
bot._should_auto_approve = MagicMock(return_value=False)
_bind_ws_event_handlers(bot, TurnstoneBot)
bot._on_ws_event = TurnstoneBot._on_ws_event.__get__(bot, TurnstoneBot)
return bot
def test_sends_per_item_embed(self):
@@ -1493,7 +1382,7 @@ class TestToolResultEvent:
bot._notify_reply_channels = {}
bot._http_client = MagicMock()
bot._should_auto_approve = MagicMock(return_value=False)
_bind_ws_event_handlers(bot, TurnstoneBot)
bot._on_ws_event = TurnstoneBot._on_ws_event.__get__(bot, TurnstoneBot)
return bot
def test_marks_info_done_and_sends_result(self):
@@ -1648,7 +1537,7 @@ class TestApprovalResolved:
bot._pending_approval_msgs = {}
bot._notify_reply_channels = {}
bot._should_auto_approve = MagicMock(return_value=False)
_bind_ws_event_handlers(bot, TurnstoneBot)
bot._on_ws_event = TurnstoneBot._on_ws_event.__get__(bot, TurnstoneBot)
return bot
def test_disables_buttons_on_timeout(self):
@@ -1716,226 +1605,3 @@ class TestChannelCLI:
main()
assert exc_info.value.code == 1
# ---------------------------------------------------------------------------
# Approval / plan-review interaction views — owner-check regression tests
# ---------------------------------------------------------------------------
def _make_view_interaction(user_id: int, footer: str | None) -> MagicMock:
"""Build a minimal interaction for ApprovalView / PlanReviewView tests."""
interaction = MagicMock(spec=discord.Interaction)
interaction.user = MagicMock()
interaction.user.id = user_id
interaction.response = MagicMock()
interaction.response.send_message = AsyncMock()
interaction.response.defer = AsyncMock()
interaction.response.send_modal = AsyncMock()
interaction.followup = MagicMock()
interaction.followup.send = AsyncMock()
interaction.message = MagicMock()
if footer is None:
interaction.message.embeds = []
else:
embed = MagicMock()
embed.footer.text = footer
interaction.message.embeds = [embed]
return interaction
def _make_view_bot() -> MagicMock:
"""Build a TurnstoneBot double with just the surface the views read."""
from turnstone.channels.discord.bot import TurnstoneBot
bot = MagicMock(spec=TurnstoneBot)
bot.router = MagicMock()
bot.router.resolve_user = AsyncMock(return_value="turnstone-user-1")
bot.router.send_approval = AsyncMock()
bot.router.send_plan_feedback = AsyncMock()
bot._pending_approval_msgs = {}
return bot
class TestApprovalViewOwnerCheck:
"""ApprovalView rejects clicks from anyone other than the session owner."""
def test_owner_approve_allowed(self, monkeypatch):
from turnstone.channels.discord.views import ApprovalView
# Avoid real disable_message_buttons (touches discord.ui internals).
monkeypatch.setattr(
"turnstone.channels.discord.views._disable_buttons",
AsyncMock(),
)
view = ApprovalView(_make_view_bot())
interaction = _make_view_interaction(user_id=42, footer="ws-1|corr-1|42")
_run(view._handle(interaction, approved=True, always=False))
view.bot.router.send_approval.assert_awaited_once_with(
ws_id="ws-1",
correlation_id="corr-1",
approved=True,
always=False,
)
def test_non_owner_rejected(self):
from turnstone.channels.discord.views import ApprovalView
view = ApprovalView(_make_view_bot())
interaction = _make_view_interaction(user_id=999, footer="ws-1|corr-1|42")
_run(view._handle(interaction, approved=True, always=False))
view.bot.router.send_approval.assert_not_awaited()
interaction.response.send_message.assert_awaited_once()
msg_kwargs = interaction.response.send_message.call_args
assert "Only the session owner" in msg_kwargs.args[0]
assert msg_kwargs.kwargs.get("ephemeral") is True
def test_legacy_footer_without_owner_rejected(self):
from turnstone.channels.discord.views import ApprovalView
view = ApprovalView(_make_view_bot())
# Pre-upgrade footer with only ws_id|correlation_id — fail closed.
interaction = _make_view_interaction(user_id=42, footer="ws-1|corr-1")
_run(view._handle(interaction, approved=True, always=False))
view.bot.router.send_approval.assert_not_awaited()
class TestPlanReviewViewOwnerCheck:
"""PlanReviewView rejects clicks from anyone other than the session owner."""
def test_owner_approve_allowed(self, monkeypatch):
from turnstone.channels.discord.views import PlanReviewView
monkeypatch.setattr(
"turnstone.channels.discord.views._disable_buttons",
AsyncMock(),
)
view = PlanReviewView(_make_view_bot())
interaction = _make_view_interaction(user_id=42, footer="ws-1|corr-1|42")
_run(view._handle_approve(interaction))
view.bot.router.send_plan_feedback.assert_awaited_once_with(
ws_id="ws-1",
correlation_id="corr-1",
feedback="",
)
def test_non_owner_approve_rejected(self):
from turnstone.channels.discord.views import PlanReviewView
view = PlanReviewView(_make_view_bot())
interaction = _make_view_interaction(user_id=999, footer="ws-1|corr-1|42")
_run(view._handle_approve(interaction))
view.bot.router.send_plan_feedback.assert_not_awaited()
interaction.response.send_message.assert_awaited_once()
def test_non_owner_changes_modal_rejected(self):
from turnstone.channels.discord.views import PlanReviewView
view = PlanReviewView(_make_view_bot())
interaction = _make_view_interaction(user_id=999, footer="ws-1|corr-1|42")
_run(view._handle_changes(interaction))
interaction.response.send_modal.assert_not_awaited()
interaction.response.send_message.assert_awaited_once()
class TestDiscordThreadOwnerCheck:
"""Sec-3 gate: only the thread creator can send messages into the workstream."""
@staticmethod
def _make_cog_and_ts():
"""Build a MessageCog wired to a minimal TurnstoneBot double."""
from turnstone.channels.discord.cog import MessageCog
bot = MagicMock()
bot.user = MagicMock()
bot.user.id = 99999
bot.user.mentioned_in = MagicMock(return_value=False)
ts = MagicMock()
ts._is_allowed_channel = MagicMock(return_value=True)
ts.storage = MagicMock()
ts.router = MagicMock()
ts.router.lookup_ws_id = AsyncMock(return_value="ws-1")
ts.router.resolve_user = AsyncMock(return_value="turnstone-user-1")
ts.router.send_message = AsyncMock()
ts.router.get_or_create_workstream = AsyncMock(return_value=("ws-1", False))
ts.config = MagicMock()
ts._ws_tasks = {}
ts._subscribed_ws = {"ws-1"}
ts._notify_ws_map = {}
ts._notify_reply_channels = {}
ts.get_thread_invoker = MagicMock(return_value=None)
ts.subscribe_ws = AsyncMock()
bot.turnstone = ts
return MessageCog(bot), ts
def test_non_owner_thread_message_dropped(self):
"""A linked user who is NOT the thread creator gets their message
silently dropped router.send_message must not fire."""
cog, ts = self._make_cog_and_ts()
# Build a thread whose owner_id is different from the message author.
thread = MagicMock(spec=discord.Thread)
thread.id = 555
thread.parent_id = 111
thread.owner_id = 42 # thread creator
thread.name = "some-thread"
msg = _make_message(guild=True, channel=thread)
msg.author.id = 999 # non-owner trying to inject
_run(cog._on_message(msg))
ts.router.send_message.assert_not_awaited()
ts.router.get_or_create_workstream.assert_not_awaited()
def test_ask_thread_followup_allowed_when_invoker_registered(self):
"""/ask creates threads with owner_id=bot; follow-ups from the
registered invoker must still reach the workstream."""
cog, ts = self._make_cog_and_ts()
# Simulate what _cmd_ask does after channel.create_thread().
ts.get_thread_invoker = MagicMock(return_value=111)
thread = MagicMock(spec=discord.Thread)
thread.id = 555
thread.parent_id = 222
thread.owner_id = 99999 # bot owns the thread after channel.create_thread
thread.name = "ask-thread"
msg = _make_message(guild=True, channel=thread)
msg.author.id = 111 # the human who ran /ask
_run(cog._on_message(msg))
ts.router.send_message.assert_awaited_once_with("ws-1", msg.content)
def test_ask_thread_rejects_other_user_even_when_invoker_registered(self):
"""Registered invoker lock: only that user's follow-ups pass."""
cog, ts = self._make_cog_and_ts()
ts.get_thread_invoker = MagicMock(return_value=111)
thread = MagicMock(spec=discord.Thread)
thread.id = 555
thread.parent_id = 222
thread.owner_id = 99999 # bot-owned
thread.name = "ask-thread"
msg = _make_message(guild=True, channel=thread)
msg.author.id = 222 # someone other than the recorded invoker
_run(cog._on_message(msg))
ts.router.send_message.assert_not_awaited()
+55 -1
View File
@@ -1,13 +1,55 @@
"""Tests for turnstone.channels._formatter."""
"""Tests for turnstone.channels._protocol and turnstone.channels._formatter."""
from __future__ import annotations
from turnstone.channels._formatter import (
chunk_message,
format_approval_request,
format_plan_review,
format_verdict,
truncate,
)
from turnstone.channels._protocol import ChannelEvent
# ---------------------------------------------------------------------------
# ChannelEvent
# ---------------------------------------------------------------------------
class TestChannelEvent:
def test_construction(self) -> None:
evt = ChannelEvent(
channel_type="discord",
channel_id="ch-1",
channel_user_id="u-42",
message="hello",
parent_channel_id="parent",
metadata={"key": "val"},
)
assert evt.channel_type == "discord"
assert evt.channel_id == "ch-1"
assert evt.channel_user_id == "u-42"
assert evt.message == "hello"
assert evt.parent_channel_id == "parent"
assert evt.metadata == {"key": "val"}
def test_defaults(self) -> None:
evt = ChannelEvent(
channel_type="slack",
channel_id="ch-2",
channel_user_id="u-7",
message="hi",
)
assert evt.parent_channel_id == ""
assert evt.metadata == {}
def test_metadata_independence(self) -> None:
"""Default metadata dicts are independent across instances."""
a = ChannelEvent(channel_type="x", channel_id="1", channel_user_id="u", message="m")
b = ChannelEvent(channel_type="x", channel_id="2", channel_user_id="u", message="m")
a.metadata["key"] = "val"
assert "key" not in b.metadata
# ---------------------------------------------------------------------------
# chunk_message
@@ -130,6 +172,18 @@ class TestFormatApprovalRequest:
assert "/etc/hosts" in result
# ---------------------------------------------------------------------------
# format_plan_review
# ---------------------------------------------------------------------------
class TestFormatPlanReview:
def test_format(self) -> None:
result = format_plan_review("Step 1: do stuff")
assert result.startswith("**Plan review requested:**")
assert "Step 1: do stuff" in result
# ---------------------------------------------------------------------------
# format_verdict
# ---------------------------------------------------------------------------
File diff suppressed because it is too large Load Diff
-397
View File
@@ -1,397 +0,0 @@
"""Tests for the shared SSE reconnect helper in turnstone.channels._sse."""
from __future__ import annotations
import asyncio
import contextlib
import json
from types import SimpleNamespace
from unittest.mock import AsyncMock, MagicMock
import httpx
import pytest
def _run(coro): # type: ignore[no-untyped-def]
return asyncio.run(coro)
class _FakeSSEEvent:
"""A fake ``httpx_sse.ServerSentEvent`` with the subset we read."""
def __init__(self, event: str, data: str) -> None:
self.event = event
self.data = data
class _FakeEventSource:
"""Context manager returned by our fake ``aconnect_sse``.
Captures the (status_code, events) the test wants to deliver.
``aiter_sse`` yields the events then returns; the caller then hits
the outer ``while True`` loop again, which will pick up the next
queued response via the shared iterator state on _FakeConnect.
"""
def __init__(self, *, status_code: int, events: list[_FakeSSEEvent]) -> None:
self.response = SimpleNamespace(
status_code=status_code,
request=MagicMock(),
)
self._events = events
async def __aenter__(self) -> _FakeEventSource:
return self
async def __aexit__(self, exc_type, exc, tb) -> None: # noqa: ANN001
return None
async def aiter_sse(self): # type: ignore[no-untyped-def]
for event in self._events:
yield event
class _FakeConnect:
"""Drop-in replacement for ``httpx_sse.aconnect_sse``.
On each call, pops the next ``_FakeEventSource`` from *queue*. When
the queue is empty, raises ``asyncio.CancelledError`` so the loop
terminates cleanly in tests.
"""
def __init__(self, queue: list[_FakeEventSource]) -> None:
self._queue = queue
self.call_count = 0
def __call__(self, *args, **kwargs): # noqa: ANN001, ANN204
self.call_count += 1
if not self._queue:
raise asyncio.CancelledError
return self._queue.pop(0)
@pytest.fixture
def _fast_sleep(monkeypatch):
"""Patch asyncio.sleep so backoff doesn't actually wait; record calls."""
sleeps: list[float] = []
async def fake_sleep(delay: float) -> None:
sleeps.append(delay)
monkeypatch.setattr("turnstone.channels._sse.asyncio.sleep", fake_sleep)
return sleeps
def _valid_event_data(ws_id: str = "ws-1") -> str:
"""A payload ``ServerEvent.from_dict`` will accept (a ContentEvent)."""
return json.dumps(
{
"type": "content",
"ws_id": ws_id,
"text": "hello",
}
)
# ---------------------------------------------------------------------------
# 404 → on_stale + exit
# ---------------------------------------------------------------------------
class TestStaleRoute:
def test_404_calls_on_stale_and_returns(self, monkeypatch, _fast_sleep):
from turnstone.channels import _sse
queue = [_FakeEventSource(status_code=404, events=[])]
fake_connect = _FakeConnect(queue)
monkeypatch.setattr(_sse.httpx_sse, "aconnect_sse", fake_connect)
on_stale = AsyncMock()
on_event = AsyncMock()
async def node_url_fn(ws_id: str) -> str:
return "http://node"
_run(
_sse.run_sse_stream(
http_client=MagicMock(),
log_prefix="test",
ws_id="ws-1",
node_url_fn=node_url_fn,
token_factory=None,
on_event=on_event,
on_stale=on_stale,
)
)
on_stale.assert_awaited_once()
on_event.assert_not_awaited()
# No reconnect after 404.
assert fake_connect.call_count == 1
assert _fast_sleep == []
def test_on_stale_exception_still_exits(self, monkeypatch, _fast_sleep):
"""If on_stale raises, the loop must not reconnect."""
from turnstone.channels import _sse
queue = [_FakeEventSource(status_code=404, events=[])]
fake_connect = _FakeConnect(queue)
monkeypatch.setattr(_sse.httpx_sse, "aconnect_sse", fake_connect)
on_stale = AsyncMock(side_effect=RuntimeError("storage down"))
async def node_url_fn(ws_id: str) -> str:
return "http://node"
_run(
_sse.run_sse_stream(
http_client=MagicMock(),
log_prefix="test",
ws_id="ws-1",
node_url_fn=node_url_fn,
token_factory=None,
on_event=AsyncMock(),
on_stale=on_stale,
)
)
on_stale.assert_awaited_once()
# Still a single connect — no livelock.
assert fake_connect.call_count == 1
# ---------------------------------------------------------------------------
# 500+ → exponential backoff
# ---------------------------------------------------------------------------
class TestBackoff:
def test_500_triggers_backoff_and_retries(self, monkeypatch, _fast_sleep):
from turnstone.channels import _sse
queue = [
_FakeEventSource(status_code=503, events=[]),
_FakeEventSource(status_code=503, events=[]),
_FakeEventSource(status_code=503, events=[]),
]
fake_connect = _FakeConnect(queue)
monkeypatch.setattr(_sse.httpx_sse, "aconnect_sse", fake_connect)
async def node_url_fn(ws_id: str) -> str:
return "http://node"
with contextlib.suppress(asyncio.CancelledError):
_run(
_sse.run_sse_stream(
http_client=MagicMock(),
log_prefix="test",
ws_id="ws-1",
node_url_fn=node_url_fn,
token_factory=None,
on_event=AsyncMock(),
on_stale=AsyncMock(),
)
)
assert fake_connect.call_count >= 3
# First three recorded sleeps are 2s, 4s, 8s (starts at
# SSE_RECONNECT_DELAY, doubles each time, capped at
# SSE_MAX_RECONNECT_DELAY).
assert _fast_sleep[0] == _sse.SSE_RECONNECT_DELAY
assert _fast_sleep[1] == _sse.SSE_RECONNECT_DELAY * 2
assert _fast_sleep[2] == _sse.SSE_RECONNECT_DELAY * 4
def test_backoff_resets_after_successful_dispatch(self, monkeypatch, _fast_sleep):
"""After a 200 + successful event dispatch, the next error
restarts backoff at the initial delay."""
from turnstone.channels import _sse
good_event = _FakeSSEEvent(event="message", data=_valid_event_data())
queue = [
_FakeEventSource(status_code=503, events=[]),
_FakeEventSource(status_code=200, events=[good_event]),
_FakeEventSource(status_code=503, events=[]),
]
fake_connect = _FakeConnect(queue)
monkeypatch.setattr(_sse.httpx_sse, "aconnect_sse", fake_connect)
on_event = AsyncMock()
async def node_url_fn(ws_id: str) -> str:
return "http://node"
with contextlib.suppress(asyncio.CancelledError):
_run(
_sse.run_sse_stream(
http_client=MagicMock(),
log_prefix="test",
ws_id="ws-1",
node_url_fn=node_url_fn,
token_factory=None,
on_event=on_event,
on_stale=AsyncMock(),
)
)
on_event.assert_awaited()
# Sleep sequence: 2 (after first 503), 2 (reset after 200/event),
# then CancelledError exits. First two sleeps are both the base
# delay — the reset did its job.
assert len(_fast_sleep) >= 2
assert _fast_sleep[0] == _sse.SSE_RECONNECT_DELAY
assert _fast_sleep[1] == _sse.SSE_RECONNECT_DELAY
# ---------------------------------------------------------------------------
# Event dispatch
# ---------------------------------------------------------------------------
class TestEventDispatch:
def test_invalid_json_is_skipped(self, monkeypatch, _fast_sleep):
from turnstone.channels import _sse
bad = _FakeSSEEvent(event="message", data="{not json")
good = _FakeSSEEvent(event="message", data=_valid_event_data())
queue = [_FakeEventSource(status_code=200, events=[bad, good])]
fake_connect = _FakeConnect(queue)
monkeypatch.setattr(_sse.httpx_sse, "aconnect_sse", fake_connect)
on_event = AsyncMock()
async def node_url_fn(ws_id: str) -> str:
return "http://node"
with contextlib.suppress(asyncio.CancelledError):
_run(
_sse.run_sse_stream(
http_client=MagicMock(),
log_prefix="test",
ws_id="ws-1",
node_url_fn=node_url_fn,
token_factory=None,
on_event=on_event,
on_stale=AsyncMock(),
)
)
# Good event delivered, bad one silently dropped.
assert on_event.await_count == 1
def test_on_event_exception_does_not_kill_stream(self, monkeypatch, _fast_sleep):
from turnstone.channels import _sse
e1 = _FakeSSEEvent(event="message", data=_valid_event_data())
e2 = _FakeSSEEvent(event="message", data=_valid_event_data())
queue = [_FakeEventSource(status_code=200, events=[e1, e2])]
fake_connect = _FakeConnect(queue)
monkeypatch.setattr(_sse.httpx_sse, "aconnect_sse", fake_connect)
on_event = AsyncMock(side_effect=[RuntimeError("boom"), None])
async def node_url_fn(ws_id: str) -> str:
return "http://node"
with contextlib.suppress(asyncio.CancelledError):
_run(
_sse.run_sse_stream(
http_client=MagicMock(),
log_prefix="test",
ws_id="ws-1",
node_url_fn=node_url_fn,
token_factory=None,
on_event=on_event,
on_stale=AsyncMock(),
)
)
# Both events attempted — first raised but second still delivered.
assert on_event.await_count == 2
# ---------------------------------------------------------------------------
# Token factory
# ---------------------------------------------------------------------------
class TestTokenFactory:
def test_header_refreshed_per_connection(self, monkeypatch, _fast_sleep):
"""token_factory is called once per reconnect so rotating service
JWTs stay fresh."""
from turnstone.channels import _sse
# Two reconnects followed by CancelledError to exit.
queue = [
_FakeEventSource(status_code=503, events=[]),
_FakeEventSource(status_code=503, events=[]),
]
fake_connect = _FakeConnect(queue)
monkeypatch.setattr(_sse.httpx_sse, "aconnect_sse", fake_connect)
tokens: list[str] = []
def factory() -> str:
tok = f"tok-{len(tokens)}"
tokens.append(tok)
return tok
async def node_url_fn(ws_id: str) -> str:
return "http://node"
with contextlib.suppress(asyncio.CancelledError):
_run(
_sse.run_sse_stream(
http_client=MagicMock(),
log_prefix="test",
ws_id="ws-1",
node_url_fn=node_url_fn,
token_factory=factory,
on_event=AsyncMock(),
on_stale=AsyncMock(),
)
)
assert len(tokens) >= 2
assert tokens[0] != tokens[1]
# ---------------------------------------------------------------------------
# httpx errors
# ---------------------------------------------------------------------------
class TestTransportErrors:
def test_connect_error_falls_through_to_backoff(self, monkeypatch, _fast_sleep):
"""ConnectError is caught and treated as retryable."""
from turnstone.channels import _sse
call_order = {"n": 0}
def fake_connect(*args, **kwargs): # noqa: ANN001, ANN003
call_order["n"] += 1
if call_order["n"] == 1:
raise httpx.ConnectError("boom")
# Second attempt: signal the loop to exit.
raise asyncio.CancelledError
monkeypatch.setattr(_sse.httpx_sse, "aconnect_sse", fake_connect)
async def node_url_fn(ws_id: str) -> str:
return "http://node"
with contextlib.suppress(asyncio.CancelledError):
_run(
_sse.run_sse_stream(
http_client=MagicMock(),
log_prefix="test",
ws_id="ws-1",
node_url_fn=node_url_fn,
token_factory=None,
on_event=AsyncMock(),
on_stale=AsyncMock(),
)
)
assert call_order["n"] == 2
# Backoff ran once after the ConnectError.
assert _fast_sleep == [_sse.SSE_RECONNECT_DELAY]
@@ -1,66 +0,0 @@
"""ChatSession interactivity flag tests (Phase 9).
Validates that ``ChatSession._is_interactive_for_consent`` is computed
correctly from ``client_type`` on construction. This is the front of
the Phase 9 plumb-through: the flag flows from here to
``_dispatch_pool_sync`` to the structured-error pending-consent
write path.
"""
from __future__ import annotations
from tests._session_helpers import make_session
from turnstone.prompts import INTERACTIVE_CONSENT_CLIENT_TYPES, ClientType
def test_web_is_interactive() -> None:
s = make_session(client_type=ClientType.WEB)
assert s._is_interactive_for_consent is True
def test_cli_is_interactive() -> None:
s = make_session(client_type=ClientType.CLI)
assert s._is_interactive_for_consent is True
def test_chat_is_not_interactive() -> None:
# Discord / Slack adapters cannot drive a browser redirect from
# inside the channel — consent prompts must be deferred to the
# dashboard badge.
s = make_session(client_type=ClientType.CHAT)
assert s._is_interactive_for_consent is False
def test_scheduled_is_not_interactive() -> None:
# The scheduler runs autonomously; the user isn't online to
# complete the OAuth redirect.
s = make_session(client_type=ClientType.SCHEDULED)
assert s._is_interactive_for_consent is False
def test_interactive_set_matches_module_constant() -> None:
# Pin the module-level frozenset against the flag computation —
# a future reorganisation that drifts the set vs the per-session
# logic would silently break the gating.
for ct in ClientType:
s = make_session(client_type=ct)
assert s._is_interactive_for_consent == (ct in INTERACTIVE_CONSENT_CLIENT_TYPES), ct
def test_default_client_type_is_cli_interactive() -> None:
# Defaults preserved — make_session uses ChatSession's default
# which is CLI. Sanity check that the default user experience
# stays interactive-for-consent.
s = make_session()
assert s._client_type == ClientType.CLI
assert s._is_interactive_for_consent is True
def test_scheduled_env_file_exists() -> None:
"""The SCHEDULED env module must exist; otherwise
``compose_system_message`` for a scheduled session would 500."""
from turnstone.prompts import _load
text = _load("env/scheduled.md")
assert "Output Environment" in text
assert "consent" in text.lower()
-215
View File
@@ -1,215 +0,0 @@
"""Unit tests for :class:`turnstone.core.child_event_bus.ChildEventBus`.
The bus is the in-process wakeup primitive for ``wait_for_workstream``
(see :mod:`turnstone.console.coordinator_client`). It's a small dict
of ws_id set[threading.Event] under a lock focused tests for
register/notify symmetry, no-subscriber notify, multi-waiter fan-out,
multi-child waiter, and concurrent register/notify (smoke). End-to-end
integration with the dispatch sink lives in
``test_coordinator_adapter.py`` and ``test_coordinator_client.py``.
"""
from __future__ import annotations
import threading
import time
import pytest
from turnstone.core.child_event_bus import ChildEventBus
def test_register_returns_event_that_starts_unset() -> None:
"""A waiter must not see leftover state from before it registered —
a fresh wait should always block until the first notify."""
bus = ChildEventBus()
event = bus.register_waiter(["ws-1"])
assert isinstance(event, threading.Event)
assert not event.is_set()
def test_notify_wakes_waiter_on_matching_ws_id() -> None:
bus = ChildEventBus()
event = bus.register_waiter(["ws-1"])
bus.notify("ws-1")
assert event.is_set()
def test_notify_does_not_wake_waiter_on_unrelated_ws_id() -> None:
"""Different ws_ids must keep independent waiter sets — a notify on
a stranger ws can't wake the wait or the bus stops being keyed."""
bus = ChildEventBus()
event = bus.register_waiter(["ws-1"])
bus.notify("ws-other")
assert not event.is_set()
def test_notify_with_no_subscribers_is_noop() -> None:
"""The dispatch sink calls notify on every translated event; the
steady state has no wait tool active. Must not raise."""
bus = ChildEventBus()
bus.notify("ws-nobody-cares") # no exception
def test_multi_waiter_each_gets_independent_event() -> None:
"""Two waits on the same ws_id must wake independently — clearing
one Event must not silence the other."""
bus = ChildEventBus()
e1 = bus.register_waiter(["ws-1"])
e2 = bus.register_waiter(["ws-1"])
assert e1 is not e2
bus.notify("ws-1")
assert e1.is_set()
assert e2.is_set()
def test_multi_child_waiter_fires_on_any_listed_ws_id() -> None:
"""A wait on [A, B, C] returns a single Event registered against
all three. Notify on ANY of A/B/C must wake the wait the
caller's snapshot re-read disambiguates which one changed."""
bus = ChildEventBus()
event = bus.register_waiter(["ws-a", "ws-b", "ws-c"])
bus.notify("ws-b")
assert event.is_set()
def test_unregister_removes_event_from_all_listed_ws_ids() -> None:
"""After unregister, notify on any of the previously-watched ws_ids
must NOT wake the Event leaks would mean every future notify on
that ws_id wakes a long-dead wait."""
bus = ChildEventBus()
event = bus.register_waiter(["ws-a", "ws-b"])
bus.unregister_waiter(["ws-a", "ws-b"], event)
bus.notify("ws-a")
bus.notify("ws-b")
assert not event.is_set()
def test_unregister_is_idempotent() -> None:
"""A double-unregister must silently no-op — finally blocks may
run twice in odd shutdown paths, the bus must not raise."""
bus = ChildEventBus()
event = bus.register_waiter(["ws-1"])
bus.unregister_waiter(["ws-1"], event)
bus.unregister_waiter(["ws-1"], event) # no exception
def test_unregister_pops_empty_buckets() -> None:
"""Empty per-ws_id buckets must be popped so a long-lived bus
doesn't accumulate dead keys after many waits have churned through.
Reaches into the private state the property is structural, not
behavioral, so the assertion is also."""
bus = ChildEventBus()
event = bus.register_waiter(["ws-1"])
assert "ws-1" in bus._waiters
bus.unregister_waiter(["ws-1"], event)
assert "ws-1" not in bus._waiters
def test_unregister_keeps_bucket_with_remaining_waiters() -> None:
"""Removing one waiter from a multi-waiter bucket must not drop
the others popping the bucket would silently disable notifies
for every concurrent wait on the same ws_id."""
bus = ChildEventBus()
e1 = bus.register_waiter(["ws-1"])
e2 = bus.register_waiter(["ws-1"])
bus.unregister_waiter(["ws-1"], e1)
bus.notify("ws-1")
assert not e1.is_set()
assert e2.is_set()
def test_empty_and_falsy_ws_ids_are_skipped_on_register() -> None:
"""Defensive: ``wait_for_workstream`` cleans its inputs but the bus
is reachable from other callers in future use; falsy ids should be
silently dropped, not registered against an empty-string key."""
bus = ChildEventBus()
event = bus.register_waiter(["", "ws-1", ""])
# Only the real ws_id should bucket the waiter.
assert list(bus._waiters.keys()) == ["ws-1"]
bus.notify("") # no crash, no spurious wake
assert not event.is_set()
bus.notify("ws-1")
assert event.is_set()
def test_notify_wakes_waiter_blocking_on_event_wait() -> None:
"""End-to-end wake-up latency: a wait blocked on ``Event.wait``
must return promptly after a notify on a watched ws_id. This is
the property that retires the 0.5s polling cadence."""
bus = ChildEventBus()
event = bus.register_waiter(["ws-1"])
woken_at = [0.0]
def _waiter() -> None:
event.wait(timeout=2.0)
woken_at[0] = time.monotonic()
t = threading.Thread(target=_waiter, daemon=True)
t.start()
# Give the waiter a beat to enter Event.wait, then notify.
time.sleep(0.05)
notified_at = time.monotonic()
bus.notify("ws-1")
t.join(timeout=1.0)
assert not t.is_alive(), "waiter did not wake within 1s of notify"
# Latency budget is generous; the contract is "well under the legacy
# 0.5s poll cadence", not microsecond timing.
assert woken_at[0] - notified_at < 0.2
def test_clear_before_check_race_does_not_lose_wake() -> None:
"""The wait-loop pattern is ``clear(); snapshot(); ...; wait()``.
A notify between clear and wait must leave the Event set, so the
next wait returns immediately and the loop re-snapshots. Same
standard subscribe/check race the wait loop guards against."""
bus = ChildEventBus()
event = bus.register_waiter(["ws-1"])
# Simulate wait-loop ordering: clear, then notify "between" clear
# and the next wait.
event.clear()
bus.notify("ws-1")
# The next wait must return True immediately (set is sticky until
# the next clear).
assert event.wait(timeout=0.1) is True
def test_concurrent_register_and_notify_is_safe() -> None:
"""Smoke test: many threads registering / notifying / unregistering
in parallel must not raise or deadlock. Doesn't assert specific
interleavings only structural safety of the lock discipline."""
bus = ChildEventBus()
stop = threading.Event()
errors: list[BaseException] = []
def _worker(ws_id: str) -> None:
try:
for _ in range(200):
if stop.is_set():
return
ev = bus.register_waiter([ws_id])
bus.notify(ws_id)
bus.unregister_waiter([ws_id], ev)
except BaseException as e: # noqa: BLE001
errors.append(e)
threads = [threading.Thread(target=_worker, args=(f"ws-{i}",), daemon=True) for i in range(8)]
for t in threads:
t.start()
for t in threads:
t.join(timeout=5.0)
stop.set()
assert not errors, f"worker threads raised: {errors!r}"
# All buckets should have been popped (every register paired with
# unregister).
assert bus._waiters == {}
@pytest.mark.parametrize("ws_id", ["", None])
def test_notify_silently_ignores_falsy_ws_id(ws_id: object) -> None:
"""Defensive: the dispatch sink already guards against empty
ws_ids, but a falsy slip-through must not raise."""
bus = ChildEventBus()
event = bus.register_waiter(["ws-1"])
bus.notify(ws_id) # type: ignore[arg-type]
assert not event.is_set()
-270
View File
@@ -1,270 +0,0 @@
"""Unit tests for :mod:`turnstone.core.child_source`.
Covers both strategies in isolation against fakes no live collector,
no live SessionManager. Adapter-level integration coverage continues to
live in ``test_coordinator_adapter.py``.
"""
from __future__ import annotations
import contextlib
import time
from typing import TYPE_CHECKING, Any
from turnstone.core.child_source import ClusterChildSource, SameNodeChildSource
from turnstone.core.children_registry import ChildrenRegistry
from turnstone.core.workstream import WorkstreamState
if TYPE_CHECKING:
import queue
# ---------------------------------------------------------------------------
# SameNodeChildSource
# ---------------------------------------------------------------------------
class _FakeManager:
"""Minimal SessionManager stand-in implementing the subscribe API."""
def __init__(self) -> None:
self.subscribers: list[Any] = []
def subscribe_to_state(self, callback: Any) -> None:
self.subscribers.append(callback)
def unsubscribe_from_state(self, callback: Any) -> None:
with contextlib.suppress(ValueError):
self.subscribers.remove(callback)
def fire(self, ws_id: str, state: WorkstreamState) -> None:
for cb in self.subscribers:
cb(ws_id, state)
class TestSameNodeChildSource:
def test_start_subscribes_to_manager(self) -> None:
mgr = _FakeManager()
registry = ChildrenRegistry()
src = SameNodeChildSource(mgr, registry)
sink_calls: list[dict[str, Any]] = []
src.start(sink=sink_calls.append)
assert len(mgr.subscribers) == 1
def test_state_change_for_known_child_pushes_to_sink(self) -> None:
mgr = _FakeManager()
registry = ChildrenRegistry()
registry.install("p1", object())
registry.add_child("p1", "c1")
src = SameNodeChildSource(mgr, registry)
sink_calls: list[dict[str, Any]] = []
src.start(sink=sink_calls.append)
mgr.fire("c1", WorkstreamState.RUNNING)
assert len(sink_calls) == 1
ev = sink_calls[0]
assert ev["type"] == "cluster_state"
assert ev["ws_id"] == "c1"
assert ev["state"] == "running"
def test_state_change_for_unknown_workstream_is_dropped(self) -> None:
mgr = _FakeManager()
registry = ChildrenRegistry()
src = SameNodeChildSource(mgr, registry)
sink_calls: list[dict[str, Any]] = []
src.start(sink=sink_calls.append)
# No registry entry — pre-filter drops the event without
# invoking the sink.
mgr.fire("ws-unknown", WorkstreamState.IDLE)
assert sink_calls == []
def test_shutdown_unsubscribes(self) -> None:
mgr = _FakeManager()
registry = ChildrenRegistry()
src = SameNodeChildSource(mgr, registry)
src.start(sink=lambda ev: None)
assert len(mgr.subscribers) == 1
src.shutdown()
assert mgr.subscribers == []
def test_start_is_idempotent(self) -> None:
mgr = _FakeManager()
registry = ChildrenRegistry()
src = SameNodeChildSource(mgr, registry)
src.start(sink=lambda ev: None)
src.start(sink=lambda ev: None)
# Second start is a no-op; only one subscription.
assert len(mgr.subscribers) == 1
def test_sink_exception_does_not_propagate(self) -> None:
mgr = _FakeManager()
registry = ChildrenRegistry()
registry.install("p1", object())
registry.add_child("p1", "c1")
src = SameNodeChildSource(mgr, registry)
def bad_sink(ev: dict[str, Any]) -> None:
raise RuntimeError("sink boom")
src.start(sink=bad_sink)
# Should not raise — the strategy catches sink failures and logs.
mgr.fire("c1", WorkstreamState.RUNNING)
# ---------------------------------------------------------------------------
# ClusterChildSource
# ---------------------------------------------------------------------------
class _FakeCollector:
"""Minimal ClusterCollector stand-in providing the listener API."""
def __init__(self, snapshot: dict[str, Any] | None = None) -> None:
self._snapshot = snapshot or {"nodes": []}
self.queues: list[queue.Queue[dict[str, Any]]] = []
self.unregistered: list[queue.Queue[dict[str, Any]]] = []
def get_snapshot_and_register(self, q: queue.Queue[dict[str, Any]]) -> dict[str, Any]:
self.queues.append(q)
return self._snapshot
def unregister_listener(self, q: queue.Queue[dict[str, Any]]) -> None:
self.unregistered.append(q)
def emit(self, event: dict[str, Any]) -> None:
"""Push an event to all registered listener queues."""
for q in self.queues:
q.put(event)
class TestClusterChildSource:
def test_start_subscribes_to_collector(self) -> None:
coll = _FakeCollector()
registry = ChildrenRegistry()
src = ClusterChildSource(
collector=coll,
registry=registry,
parents_provider=list,
)
try:
src.start(sink=lambda ev: None)
assert len(coll.queues) == 1
finally:
src.shutdown()
def test_start_primes_registry_from_snapshot(self) -> None:
snapshot = {
"nodes": [
{
"workstreams": [
{"id": "c1", "parent_ws_id": "p1"},
{"id": "c2", "parent_ws_id": "p1"},
# Unknown parent — dropped
{"id": "x", "parent_ws_id": "p-unknown"},
],
},
],
}
coll = _FakeCollector(snapshot)
registry = ChildrenRegistry()
registry.install("p1", object())
src = ClusterChildSource(
collector=coll,
registry=registry,
parents_provider=lambda: ["p1"],
)
try:
src.start(sink=lambda ev: None)
assert set(registry.children_of("p1")) == {"c1", "c2"}
assert registry.parent_for("x") is None
finally:
src.shutdown()
def test_event_dispatched_to_sink(self) -> None:
coll = _FakeCollector()
registry = ChildrenRegistry()
src = ClusterChildSource(
collector=coll,
registry=registry,
parents_provider=list,
)
sink_calls: list[dict[str, Any]] = []
try:
src.start(sink=sink_calls.append)
coll.emit({"type": "cluster_state", "ws_id": "c1", "state": "running"})
# Daemon thread loop has 1.0s queue timeout; poll briefly.
for _ in range(20):
if sink_calls:
break
time.sleep(0.05)
assert len(sink_calls) == 1
assert sink_calls[0]["ws_id"] == "c1"
finally:
src.shutdown()
def test_shutdown_unregisters_and_joins_thread(self) -> None:
coll = _FakeCollector()
registry = ChildrenRegistry()
src = ClusterChildSource(
collector=coll,
registry=registry,
parents_provider=list,
)
src.start(sink=lambda ev: None)
src.shutdown()
assert coll.unregistered == coll.queues
# Second shutdown is a no-op (idempotent).
src.shutdown()
def test_start_is_idempotent(self) -> None:
coll = _FakeCollector()
registry = ChildrenRegistry()
src = ClusterChildSource(
collector=coll,
registry=registry,
parents_provider=list,
)
try:
src.start(sink=lambda ev: None)
src.start(sink=lambda ev: None)
assert len(coll.queues) == 1
finally:
src.shutdown()
def test_sink_exception_does_not_kill_thread(self) -> None:
coll = _FakeCollector()
registry = ChildrenRegistry()
src = ClusterChildSource(
collector=coll,
registry=registry,
parents_provider=list,
)
survived_calls: list[dict[str, Any]] = []
call_count = [0]
def flaky_sink(ev: dict[str, Any]) -> None:
call_count[0] += 1
if call_count[0] == 1:
raise RuntimeError("first one boom")
survived_calls.append(ev)
try:
src.start(sink=flaky_sink)
coll.emit({"type": "cluster_state", "ws_id": "c1", "state": "x"})
coll.emit({"type": "cluster_state", "ws_id": "c2", "state": "y"})
for _ in range(40):
if survived_calls:
break
time.sleep(0.05)
assert len(survived_calls) == 1
assert survived_calls[0]["ws_id"] == "c2"
finally:
src.shutdown()
# Multi-subscriber observer tests for ``SessionManager.subscribe_to_state``
# / ``unsubscribe_from_state`` live in ``test_session_manager.py`` where
# the proper FakeAdapter / FakeStorage construction helpers already exist.
-239
View File
@@ -1,239 +0,0 @@
"""Unit tests for :class:`turnstone.core.children_registry.ChildrenRegistry`.
The registry was lifted from ``CoordinatorAdapter`` in Stage 3 Step 1.
Adapter-level coverage for the integrated behavior already lives in
``test_coordinator_adapter.py``; this file pins the data structure
invariants in isolation so the registry can be reused by future
``ChildSource`` strategies (Step 2) without re-deriving the behavior
from the adapter test surface.
"""
from __future__ import annotations
import threading
import pytest
from turnstone.core.children_registry import ChildrenRegistry
class _Sentinel:
"""Lightweight UI stand-in; identity-comparable, no behavior."""
@pytest.fixture
def registry() -> ChildrenRegistry:
return ChildrenRegistry()
# ---------------------------------------------------------------------------
# install / uninstall
# ---------------------------------------------------------------------------
class TestInstallUninstall:
def test_install_seeds_empty_child_set_and_presence(self, registry: ChildrenRegistry) -> None:
ui = _Sentinel()
registry.install("p1", ui)
assert registry.children_of("p1") == []
assert registry.ui_for("p1") is ui
assert registry.parents() == ["p1"]
def test_install_is_idempotent_repoints_ui_keeps_children(
self, registry: ChildrenRegistry
) -> None:
ui_a = _Sentinel()
ui_b = _Sentinel()
registry.install("p1", ui_a)
registry.merge_children("p1", ["c1", "c2"])
registry.install("p1", ui_b)
assert registry.ui_for("p1") is ui_b
assert set(registry.children_of("p1")) == {"c1", "c2"}
def test_uninstall_clears_forward_reverse_and_presence(
self, registry: ChildrenRegistry
) -> None:
ui = _Sentinel()
registry.install("p1", ui)
registry.merge_children("p1", ["c1", "c2"])
registry.uninstall("p1")
assert registry.children_of("p1") == []
assert registry.ui_for("p1") is None
assert registry.parents() == []
assert registry.parent_for("c1") is None
assert registry.parent_for("c2") is None
def test_uninstall_unknown_parent_is_noop(self, registry: ChildrenRegistry) -> None:
registry.uninstall("never-installed") # must not raise
def test_uninstall_does_not_clobber_other_parents(self, registry: ChildrenRegistry) -> None:
registry.install("p1", _Sentinel())
registry.install("p2", _Sentinel())
registry.merge_children("p1", ["c1"])
registry.merge_children("p2", ["c2"])
registry.uninstall("p1")
assert registry.parent_for("c1") is None
assert registry.parent_for("c2") == "p2"
assert registry.parents() == ["p2"]
# ---------------------------------------------------------------------------
# add_child — atomic check-and-route
# ---------------------------------------------------------------------------
class TestAddChild:
def test_add_child_returns_ui_on_success(self, registry: ChildrenRegistry) -> None:
ui = _Sentinel()
registry.install("p1", ui)
assert registry.add_child("p1", "c1") is ui
assert registry.parent_for("c1") == "p1"
assert registry.children_of("p1") == ["c1"]
def test_add_child_returns_none_when_parent_not_installed(
self, registry: ChildrenRegistry
) -> None:
assert registry.add_child("absent", "c1") is None
assert registry.parent_for("c1") is None
def test_add_child_returns_none_on_duplicate(self, registry: ChildrenRegistry) -> None:
ui = _Sentinel()
registry.install("p1", ui)
assert registry.add_child("p1", "c1") is ui
# second add for same child returns None — caller must not
# double-dispatch.
assert registry.add_child("p1", "c1") is None
assert registry.children_of("p1") == ["c1"]
# ---------------------------------------------------------------------------
# merge_children — bulk seeding
# ---------------------------------------------------------------------------
class TestMergeChildren:
def test_merge_seeds_forward_and_reverse(self, registry: ChildrenRegistry) -> None:
registry.merge_children("p1", ["c1", "c2", "c3"])
assert set(registry.children_of("p1")) == {"c1", "c2", "c3"}
for cid in ("c1", "c2", "c3"):
assert registry.parent_for(cid) == "p1"
def test_merge_is_idempotent(self, registry: ChildrenRegistry) -> None:
registry.merge_children("p1", ["c1"])
registry.merge_children("p1", ["c1"])
assert registry.children_of("p1") == ["c1"]
def test_merge_skips_empty_or_falsy_ids(self, registry: ChildrenRegistry) -> None:
registry.merge_children("p1", ["", "c1", "", "c2"])
assert set(registry.children_of("p1")) == {"c1", "c2"}
def test_merge_does_not_require_install(self, registry: ChildrenRegistry) -> None:
# Snapshot-priming may run before the parent's install fires —
# the merge still seeds the forward set so the install picks
# the children up. (Storage-seeded rebuild relies on this.)
registry.merge_children("p1", ["c1"])
assert registry.children_of("p1") == ["c1"]
# ui_for is still None because install hasn't run
assert registry.ui_for("p1") is None
# ---------------------------------------------------------------------------
# Lookups — return copies, not live refs
# ---------------------------------------------------------------------------
class TestLookups:
def test_children_of_returns_copy(self, registry: ChildrenRegistry) -> None:
registry.install("p1", _Sentinel())
registry.merge_children("p1", ["c1", "c2"])
snap = registry.children_of("p1")
snap.append("c3-injected")
assert "c3-injected" not in registry.children_of("p1")
def test_children_of_unknown_parent_returns_empty(self, registry: ChildrenRegistry) -> None:
assert registry.children_of("absent") == []
def test_parent_for_unknown_child_returns_none(self, registry: ChildrenRegistry) -> None:
assert registry.parent_for("absent") is None
def test_parents_returns_copy(self, registry: ChildrenRegistry) -> None:
registry.install("p1", _Sentinel())
snap = registry.parents()
snap.append("p2-injected")
assert "p2-injected" not in registry.parents()
# ---------------------------------------------------------------------------
# Concurrency — concurrent add_child must not exceed the unique-set
# invariant or leave a half-installed reverse-index entry.
# ---------------------------------------------------------------------------
class TestConcurrency:
def test_concurrent_add_child_returns_ui_exactly_once_per_unique(
self, registry: ChildrenRegistry
) -> None:
ui = _Sentinel()
registry.install("p1", ui)
results: list[object] = []
results_lock = threading.Lock()
def attempt_add(child_id: str) -> None:
r = registry.add_child("p1", child_id)
with results_lock:
results.append(r)
threads = [threading.Thread(target=attempt_add, args=("c1",)) for _ in range(20)]
for t in threads:
t.start()
for t in threads:
t.join()
# Exactly one thread sees the UI; the remaining 19 see None
# (duplicate). The forward + reverse indexes carry exactly one
# entry for c1.
successes = [r for r in results if r is ui]
nones = [r for r in results if r is None]
assert len(successes) == 1
assert len(nones) == 19
assert registry.children_of("p1") == ["c1"]
assert registry.parent_for("c1") == "p1"
def test_concurrent_install_and_add_child_no_resurrect(
self, registry: ChildrenRegistry
) -> None:
# add_child racing with uninstall: either lands first (registry
# populated) or the parent is gone (returns None). Must NOT
# leave a forward-set entry without presence — that would be
# the "resurrected after close" leak the locked dispatch path
# was guarding against.
ui = _Sentinel()
registry.install("p1", ui)
outcomes: list[object] = []
def adder() -> None:
outcomes.append(registry.add_child("p1", "c1"))
def uninstaller() -> None:
registry.uninstall("p1")
threads = [
threading.Thread(target=adder),
threading.Thread(target=uninstaller),
]
for t in threads:
t.start()
for t in threads:
t.join()
# If add_child landed first: c1 is in the forward set, then
# uninstall clears everything. End state: nothing.
# If uninstall landed first: add_child sees no presence,
# returns None, no entry added. End state: nothing.
# Either way, the leak invariant holds: child set is empty or
# parent is gone, never "child set populated but no presence".
children = registry.children_of("p1")
ui_present = registry.ui_for("p1") is not None
if children:
assert ui_present, "registry leaked: children set without presence"
-200
View File
@@ -1,200 +0,0 @@
"""Server-side tests for the close_workstream handler's close_reason
persistence guards the seam that lets coordinator inspect surface
why a workstream was retired without scraping the audit log.
"""
from __future__ import annotations
import queue
import threading
from typing import Any
from unittest.mock import MagicMock
import pytest
from starlette.testclient import TestClient
import turnstone.server as srv_mod
from turnstone.core.auth import JWT_AUD_SERVER, create_jwt
from turnstone.core.metrics import MetricsCollector
from turnstone.core.storage._sqlite import SQLiteBackend
from turnstone.core.workstream import WorkstreamState
_JWT_SECRET = "test-jwt-secret-minimum-32-chars!"
def _full_hdr() -> dict[str, str]:
return {
"Authorization": (
f"Bearer {create_jwt('u1', frozenset({'read', 'write', 'approve'}), 'test', _JWT_SECRET, audience=JWT_AUD_SERVER)}"
)
}
@pytest.fixture(autouse=True)
def _isolate_metrics(monkeypatch):
"""Swap ``turnstone.server._metrics`` for a fresh collector
per-test, with auto-restore.
Bare ``srv_mod._metrics = MetricsCollector()`` (the prior
pattern) leaks into any test file that already bound the name
via ``from turnstone.server import _metrics`` at import time
those tests' patches then operate on a different instance from
the one the live ``_publish_models_metadata`` reads, and the
monkeypatch silently no-ops. ``monkeypatch.setattr`` restores
after the test, so the leak is contained.
"""
fresh = MetricsCollector()
fresh.model = "test-model"
monkeypatch.setattr(srv_mod, "_metrics", fresh)
def _make_app(storage: Any) -> TestClient:
mock_session = MagicMock()
mock_ws = MagicMock()
mock_ws.id = "ws-target"
mock_ws.name = "test"
mock_ws.state = WorkstreamState.IDLE
mock_ws.session = mock_session
# Tenant gate (#375) checks ws.user_id == JWT subject; explicit set
# so MagicMock's auto-generated truthy attribute doesn't reject the
# request before the persistence path runs. kind / parent_ws_id
# land in the audit_detail dict alongside ``reason``.
mock_ws.user_id = "u1"
mock_ws.kind = "interactive"
mock_ws.parent_ws_id = None
mock_mgr = MagicMock()
mock_mgr.get.return_value = mock_ws
mock_mgr.close.return_value = True
mock_mgr.list_all.return_value = [mock_ws]
mock_mgr.max_active = 10
app = srv_mod.create_app(
workstreams=mock_mgr,
global_queue=queue.Queue(),
global_listeners=[],
global_listeners_lock=threading.Lock(),
skip_permissions=False,
jwt_secret=_JWT_SECRET,
auth_storage=storage,
cors_origins=["*"],
)
return TestClient(app, raise_server_exceptions=False)
@pytest.fixture
def storage(tmp_path):
return SQLiteBackend(str(tmp_path / "close.db"))
def test_close_with_reason_persists_to_workstream_config(storage):
client = _make_app(storage)
resp = client.post(
"/v1/api/workstreams/ws-target/close",
json={"reason": "task complete"},
headers=_full_hdr(),
)
assert resp.status_code == 200
cfg = storage.load_workstream_config("ws-target")
assert cfg.get("close_reason") == "task complete"
def test_close_without_reason_does_not_touch_config(storage):
client = _make_app(storage)
resp = client.post(
"/v1/api/workstreams/ws-target/close",
json={},
headers=_full_hdr(),
)
assert resp.status_code == 200
cfg = storage.load_workstream_config("ws-target")
assert "close_reason" not in cfg
def test_close_reason_capped_at_512_bytes(storage):
"""A model that dumps a multi-KB blob (or a captured secret) into the
close reason must not be able to grow the workstream_config row
without bound the handler enforces a 512-byte ceiling. Tested
with ASCII (1B/char) so the byte cap and char count coincide."""
huge = "x" * 5000
client = _make_app(storage)
resp = client.post(
"/v1/api/workstreams/ws-target/close",
json={"reason": huge},
headers=_full_hdr(),
)
assert resp.status_code == 200
cfg = storage.load_workstream_config("ws-target")
stored = cfg.get("close_reason")
assert stored is not None
assert len(stored.encode("utf-8")) <= 512
def test_close_reason_byte_cap_holds_for_multibyte_utf8(storage):
"""Repro for the char-cap-vs-byte-cap mismatch: a CJK-only payload
of 600 chars would have leaked through a code-point slice at
600*3=1800 bytes. The byte-aware cap holds it at <=512 bytes."""
huge = "\u6f22" * 600 # 3 bytes/char in UTF-8
client = _make_app(storage)
resp = client.post(
"/v1/api/workstreams/ws-target/close",
json={"reason": huge},
headers=_full_hdr(),
)
assert resp.status_code == 200
cfg = storage.load_workstream_config("ws-target")
stored = cfg.get("close_reason")
assert stored is not None
assert len(stored.encode("utf-8")) <= 512
def test_close_with_non_string_reason_drops_silently(storage):
"""A malformed body (reason=dict / list / int) should not crash the
handler non-string reasons are coerced to empty and the close
proceeds without writing to workstream_config."""
client = _make_app(storage)
resp = client.post(
"/v1/api/workstreams/ws-target/close",
json={"reason": {"unexpected": "shape"}},
headers=_full_hdr(),
)
assert resp.status_code == 200
cfg = storage.load_workstream_config("ws-target")
assert "close_reason" not in cfg
def test_close_reason_redacts_credentials(storage):
"""A model under prompt injection that captures a secret and stuffs
it into ``reason`` must not get to plant the plaintext secret in
audit logs / workstream_config. The output guard's credential-
redaction pass runs at the close handler boundary."""
client = _make_app(storage)
secret = "AKIAIOSFODNN7EXAMPLE" # AWS access key — output guard catches.
resp = client.post(
"/v1/api/workstreams/ws-target/close",
json={"reason": f"task done; key={secret}"},
headers=_full_hdr(),
)
assert resp.status_code == 200
cfg = storage.load_workstream_config("ws-target")
stored = cfg.get("close_reason")
assert stored is not None
assert secret not in stored
assert "[REDACTED:" in stored
def test_close_reason_persistence_failure_does_not_block_close(storage):
"""If the storage save raises, the close still succeeds — persistence
is best-effort; a transient storage error must not block the user
from closing a workstream."""
client = _make_app(storage)
def _boom(*args, **kwargs):
raise RuntimeError("storage down")
storage.save_workstream_config = _boom # type: ignore[method-assign]
resp = client.post(
"/v1/api/workstreams/ws-target/close",
json={"reason": "task complete"},
headers=_full_hdr(),
)
assert resp.status_code == 200
-35
View File
@@ -50,41 +50,6 @@ def test_load_config_invalid_toml(tmp_path):
assert load_config() == {}
def test_load_config_warns_when_world_readable(tmp_path, caplog):
"""Secrets in config.toml — warn if anyone but the owner can read it."""
import logging
import os
_reset_cache()
cfg = tmp_path / "config.toml"
cfg.write_text('[database]\nurl = "postgresql+psycopg://u:secret@h/d"\n')
os.chmod(cfg, 0o644)
set_config_path(str(cfg))
with caplog.at_level(logging.WARNING, logger="turnstone.core.config"):
load_config()
messages = [r.getMessage() for r in caplog.records]
assert any("group/world-readable" in m for m in messages)
def test_load_config_quiet_when_mode_0600(tmp_path, caplog):
import logging
import os
_reset_cache()
cfg = tmp_path / "config.toml"
cfg.write_text('[database]\nurl = "postgresql+psycopg://u:secret@h/d"\n')
os.chmod(cfg, 0o600)
set_config_path(str(cfg))
with caplog.at_level(logging.WARNING, logger="turnstone.core.config"):
load_config()
messages = [r.getMessage() for r in caplog.records]
assert not any("group/world-readable" in m for m in messages)
def test_load_config_caches(tmp_path):
_reset_cache()
cfg = tmp_path / "config.toml"
+5 -17
View File
@@ -87,20 +87,8 @@ class TestSetGetRoundTrip:
assert store.get("tools.skip_permissions") is False
def test_str(self, store):
store.set("model.default_alias", "gpt5-prod")
assert store.get("model.default_alias") == "gpt5-prod"
def test_plan_task_alias(self, store):
store.set("model.plan_alias", "smart")
store.set("model.task_alias", "fast")
assert store.get("model.plan_alias") == "smart"
assert store.get("model.task_alias") == "fast"
def test_plan_task_effort(self, store):
store.set("model.plan_effort", "max")
store.set("model.task_effort", "low")
assert store.get("model.plan_effort") == "max"
assert store.get("model.task_effort") == "low"
store.set("model.name", "gpt-5")
assert store.get("model.name") == "gpt-5"
# ---------------------------------------------------------------------------
@@ -177,10 +165,10 @@ class TestStoredKeys:
assert store.stored_keys() == frozenset()
store.set("tools.timeout", 30)
assert store.stored_keys() == frozenset({"tools.timeout"})
store.set("model.default_alias", "gpt5-prod")
assert store.stored_keys() == frozenset({"tools.timeout", "model.default_alias"})
store.set("model.name", "gpt-5")
assert store.stored_keys() == frozenset({"tools.timeout", "model.name"})
store.delete("tools.timeout")
assert store.stored_keys() == frozenset({"model.default_alias"})
assert store.stored_keys() == frozenset({"model.name"})
# ---------------------------------------------------------------------------
+37 -593
View File
@@ -3,13 +3,11 @@
import asyncio
import json
import queue
from typing import Any
from unittest.mock import MagicMock
import pytest
from turnstone.console.collector import ClusterCollector, NodeSnapshot
from turnstone.console.server import _PROXY_AUTH_LOCAL_HANDLERS
# Shared test auth — JWT-based
_TEST_JWT_SECRET = "test-jwt-secret-minimum-32-chars!"
@@ -33,7 +31,16 @@ _TEST_AUTH_HEADERS = {"Authorization": f"Bearer {_test_jwt()}"}
# Mock storage for collector tests
# ---------------------------------------------------------------------------
from tests._coord_test_helpers import MockStorage # noqa: E402, F401
class MockStorage:
"""Minimal storage mock that implements list_services for collector tests."""
def __init__(self):
self.services: list[dict[str, str]] = []
def list_services(self, service_type: str, max_age_seconds: int = 120) -> list[dict[str, str]]:
return list(self.services)
# ---------------------------------------------------------------------------
# Helpers
@@ -151,78 +158,6 @@ class TestCollectorDiscovery:
assert c._nodes["node-a"].started == 1234567890.0
class TestCollectorNotifyWireIn:
"""NotifyDispatcher-driven discovery — reactive node visibility."""
def test_start_subscribes_to_services_channel(self):
# Stub dispatcher records subscriptions without spawning threads.
class _StubDispatcher:
def __init__(self):
self.subscriptions: list[tuple[str, Any]] = []
def subscribe(self, channel, handler):
self.subscriptions.append((channel, handler))
return lambda: None
stub = _StubDispatcher()
storage = MockStorage()
c = ClusterCollector(
storage=storage,
discovery_interval=999,
notify_dispatcher=stub,
)
try:
c.start()
assert len(stub.subscriptions) == 1
channel, handler = stub.subscriptions[0]
assert channel == "services"
assert handler == c._on_services_notify
finally:
c.stop()
def test_no_dispatcher_means_no_subscribe(self):
# Collector without a dispatcher (single-node / SQLite dev) just
# falls back to the 60 s discovery-loop polling — no error.
c = _make_collector(MockStorage())
try:
c.start()
assert c._notify_unsubscribe is None
finally:
c.stop()
def test_on_notify_runs_discovery(self):
# Construct a synthetic Notify and invoke the handler directly —
# asserts the wire-in delegates back to ``_discover_nodes``.
from turnstone.core.storage._notify import Notify
storage = MockStorage()
c = _make_collector(storage)
c._running = True # bypass start() so we don't spawn threads
q: queue.Queue[dict[str, Any]] = queue.Queue()
c.register_listener(q)
storage.services = [
{"service_id": "node-z", "url": "http://z:8080", "metadata": "{}"},
]
c._on_services_notify(Notify(channel="services", payload="{}", pid=0))
event = q.get_nowait()
assert event["type"] == "node_joined"
assert event["node_id"] == "node-z"
def test_on_notify_when_not_running_is_noop(self):
# If a stray notify arrives after stop, the handler doesn't run
# discovery on a half-torn-down collector.
from turnstone.core.storage._notify import Notify
storage = MockStorage()
storage.services = [{"service_id": "node-y", "url": "http://y:8080", "metadata": "{}"}]
c = _make_collector(storage)
# _running stays False (never called start()).
c._on_services_notify(Notify(channel="services", payload="{}", pid=0))
assert c.get_overview()["nodes"] == 0
class TestCollectorSnapshot:
"""Applying node_snapshot SSE events."""
@@ -388,46 +323,6 @@ class TestCollectorSnapshot:
assert event["ws_id"] == "ws1"
assert event["state"] == "running"
def test_apply_snapshot_state_change_does_not_carry_pending_approval_detail(self):
"""Stage 3 cleanup — the snapshot-resync cluster_state event no
longer piggybacks ``pending_approval_detail`` (the field is
gone from cluster_state entirely). On reconnect the browser's
bulk fetch triggered by the ``activity_state="approval"``
transition in the reducer pulls the items directly from
``ui.serialize_pending_approval_detail()`` via the dashboard
endpoint."""
c = _make_collector()
c._nodes["node-a"] = NodeSnapshot(
node_id="node-a",
server_url="http://a:8080",
workstreams={"ws1": {"id": "ws1", "name": "same", "state": "idle"}},
)
q: queue.Queue[dict] = queue.Queue()
c.register_listener(q)
c._apply_snapshot(
"node-a",
{
"type": "node_snapshot",
"node_id": "node-a",
"workstreams": [
{
"id": "ws1",
"name": "same",
"state": "running",
"activity_state": "approval",
}
],
"health": {},
"aggregate": {},
},
)
event = q.get_nowait()
assert event["type"] == "cluster_state"
assert event["activity_state"] == "approval"
assert "pending_approval_detail" not in event
def test_apply_snapshot_skips_empty_id_workstream(self):
c = _make_collector()
c._nodes["node-a"] = NodeSnapshot(node_id="node-a", server_url="http://a:8080")
@@ -473,36 +368,6 @@ class TestCollectorDelta:
# Verify in-memory state was updated
assert c._nodes["node-a"].workstreams["ws1"]["state"] == "running"
def test_apply_delta_ws_state_does_not_carry_pending_approval_detail(self):
"""Stage 3 cleanup — ``cluster_state`` no longer carries the
``pending_approval_detail`` piggyback. Approval items now arrive
via bulk fetch on activity_state transition; verdicts via the
explicit ``intent_verdict`` event class. Symmetric event flow,
no piggyback to dedupe against."""
c = _make_collector()
c._nodes["node-a"] = NodeSnapshot(
node_id="node-a",
server_url="http://a:8080",
workstreams={"ws1": {"id": "ws1", "name": "test", "state": "idle"}},
)
q: queue.Queue[dict] = queue.Queue()
c.register_listener(q)
c._apply_delta(
"node-a",
{
"type": "ws_state",
"ws_id": "ws1",
"state": "running",
"activity_state": "approval",
},
)
event = q.get_nowait()
assert event["type"] == "cluster_state"
assert event["activity_state"] == "approval"
assert "pending_approval_detail" not in event
def test_apply_delta_ws_created(self):
c = _make_collector()
c._nodes["node-a"] = NodeSnapshot(node_id="node-a", server_url="http://a:8080")
@@ -548,139 +413,6 @@ class TestCollectorDelta:
assert event["name"] == "new-name"
assert c._nodes["node-a"].workstreams["ws1"]["name"] == "new-name"
def test_apply_delta_intent_verdict_forwards_verbatim(self):
"""Stage 3 Step 5 — node-emitted intent_verdict events flow
through _apply_delta to cluster fan-out so coord adapters can
re-emit as child_ws_intent_verdict on the parent's SSE."""
c = _make_collector()
c._nodes["node-a"] = NodeSnapshot(
node_id="node-a",
server_url="http://a:8080",
workstreams={"ws1": {"id": "ws1", "name": "test", "state": "idle"}},
)
q: queue.Queue[dict] = queue.Queue()
c.register_listener(q)
verdict = {
"call_id": "c1",
"risk_level": "low",
"confidence": 0.9,
"recommendation": "approve",
}
c._apply_delta(
"node-a",
{"type": "intent_verdict", "ws_id": "ws1", "verdict": verdict},
)
event = q.get_nowait()
assert event["type"] == "intent_verdict"
assert event["ws_id"] == "ws1"
assert event["node_id"] == "node-a"
assert event["verdict"] == verdict
def test_apply_delta_intent_verdict_drops_when_ws_id_missing(self):
c = _make_collector()
c._nodes["node-a"] = NodeSnapshot(node_id="node-a", server_url="http://a:8080")
q: queue.Queue[dict] = queue.Queue()
c.register_listener(q)
c._apply_delta("node-a", {"type": "intent_verdict", "verdict": {}})
assert q.empty()
def test_apply_delta_approval_resolved_forwards_verbatim(self):
"""Stage 3 Step 5 — paired with intent_verdict; clears the
coord tree's pending-approval pill in lockstep with the
actual decision rather than waiting for the state-change
piggyback."""
c = _make_collector()
c._nodes["node-a"] = NodeSnapshot(
node_id="node-a",
server_url="http://a:8080",
workstreams={"ws1": {"id": "ws1", "name": "test", "state": "idle"}},
)
q: queue.Queue[dict] = queue.Queue()
c.register_listener(q)
c._apply_delta(
"node-a",
{
"type": "approval_resolved",
"ws_id": "ws1",
"approved": True,
"feedback": "lgtm",
"always": False,
},
)
event = q.get_nowait()
assert event["type"] == "approval_resolved"
assert event["ws_id"] == "ws1"
assert event["node_id"] == "node-a"
assert event["approved"] is True
assert event["feedback"] == "lgtm"
assert event["always"] is False
def test_apply_delta_approve_request_forwards_detail(self):
"""Push path for the initial approval items — eliminates the
bulk-fetch race that left the coord row stuck on a loading
placeholder when the bulk fetch landed in the gap between
_emit_state(ATTENTION) and approve_tools setting _pending_approval."""
c = _make_collector()
c._nodes["node-a"] = NodeSnapshot(
node_id="node-a",
server_url="http://a:8080",
workstreams={"ws1": {"id": "ws1", "name": "test", "state": "idle"}},
)
q: queue.Queue[dict] = queue.Queue()
c.register_listener(q)
detail = {
"type": "approve_request",
"items": [{"call_id": "c1", "header": "tool x"}],
"judge_pending": True,
}
c._apply_delta(
"node-a",
{"type": "approve_request", "ws_id": "ws1", "detail": detail},
)
event = q.get_nowait()
assert event["type"] == "approve_request"
assert event["ws_id"] == "ws1"
assert event["node_id"] == "node-a"
assert event["detail"] == detail
def test_apply_delta_approve_request_drops_when_ws_id_missing(self):
c = _make_collector()
c._nodes["node-a"] = NodeSnapshot(node_id="node-a", server_url="http://a:8080")
q: queue.Queue[dict] = queue.Queue()
c.register_listener(q)
c._apply_delta("node-a", {"type": "approve_request", "detail": {}})
assert q.empty()
def test_apply_delta_approval_resolved_coerces_missing_fields(self):
"""Defensive: ``approved`` / ``always`` / ``feedback`` may be
omitted by older nodes mid-rolling-upgrade; collector coerces
to safe defaults."""
c = _make_collector()
c._nodes["node-a"] = NodeSnapshot(
node_id="node-a",
server_url="http://a:8080",
workstreams={"ws1": {"id": "ws1", "name": "test", "state": "idle"}},
)
q: queue.Queue[dict] = queue.Queue()
c.register_listener(q)
c._apply_delta("node-a", {"type": "approval_resolved", "ws_id": "ws1"})
event = q.get_nowait()
assert event["approved"] is False
assert event["feedback"] == ""
assert event["always"] is False
def test_apply_delta_health_changed(self):
c = _make_collector()
c._nodes["node-a"] = NodeSnapshot(
@@ -1045,7 +777,6 @@ class TestConsoleHTTPEndpoints:
sort_by="state",
page=1,
per_page=25,
extra_rows=[],
)
def test_get_workstreams_per_page_capped(self, client, mock_collector):
@@ -1102,18 +833,16 @@ class TestConsoleHTTPEndpoints:
resp = client.get("/nonexistent")
assert resp.status_code == 404
def test_index_landing_surfaces(self, client):
def test_index_has_new_ws_button(self, client):
status, body, ct = self._get_raw(client, "/")
assert status == 200
# Coordinator-first landing keeps the node list always-visible.
assert 'id="view-overview"' in body
assert 'id="node-table"' in body
# Removed in the 1.5.0 landing-page cleanup — guard against
# accidental reintroduction.
assert 'id="new-ws-overlay"' not in body
assert 'id="new-ws-btn"' not in body
assert 'id="cluster-summary-compact"' not in body
assert 'id="view-node"' not in body
assert 'id="new-ws-btn"' in body
assert "showNewWsModal" in body
def test_index_has_new_ws_modal(self, client):
status, body, ct = self._get_raw(client, "/")
assert 'id="new-ws-overlay"' in body
assert 'id="new-ws-node"' in body
# ---------------------------------------------------------------------------
@@ -1471,234 +1200,6 @@ class TestConsoleProxy:
)
assert resp.status_code == 404
def test_proxy_api_per_ws_events_routes_to_sse_handler(self, client, mock_collector):
"""``/node/{node_id}/v1/api/workstreams/{ws_id}/events`` is the
per-workstream SSE stream the interactive WebUI subscribes to.
Without explicit detection, the path falls through to the
regular GET branch and the EventSource API can't consume the
one-shot response Firefox surfaces it as "can't establish a
connection". Regression guard for the legacy URL surface
removal (#422) that moved per-ws SSE under
``/workstreams/{ws_id}/events`` without updating the proxy."""
from unittest.mock import AsyncMock, patch
from starlette.responses import Response
mock_collector.get_node_detail.return_value = {
"node_id": "node-a",
"server_url": "http://a:8080",
"reachable": True,
}
ws_id = "a" * 32
with (
patch(
"turnstone.console.server._proxy_sse",
new_callable=AsyncMock,
return_value=Response("ok", status_code=200),
) as sse_mock,
patch(
"turnstone.console.server._proxy_get",
new_callable=AsyncMock,
return_value=Response("ok", status_code=200),
) as get_mock,
):
client.get(f"/node/node-a/v1/api/workstreams/{ws_id}/events")
assert sse_mock.await_count == 1, (
"per-ws events path must route to _proxy_sse, not _proxy_get"
)
assert get_mock.await_count == 0
# Path passed to _proxy_sse must be the workstreams-prefixed
# form so the upstream URL is reconstructed correctly.
sse_args = sse_mock.await_args
assert sse_args.args[2] == f"workstreams/{ws_id}/events"
def test_proxy_api_global_events_still_routes_to_sse(self, client, mock_collector):
"""The bare ``events/global`` path was the only SSE path the
proxy recognized before the per-ws fix. Verify it still routes
correctly so the new branch didn't regress the existing case."""
from unittest.mock import AsyncMock, patch
from starlette.responses import Response
mock_collector.get_node_detail.return_value = {
"node_id": "node-a",
"server_url": "http://a:8080",
"reachable": True,
}
with patch(
"turnstone.console.server._proxy_sse",
new_callable=AsyncMock,
return_value=Response("ok", status_code=200),
) as sse_mock:
client.get("/node/node-a/v1/api/events/global")
assert sse_mock.await_count == 1
# events/global must use the console's service token —
# the upstream gates this path on `service` scope and
# end-user JWTs don't carry it. Without this, the
# browser's interactive UI 403-loops on every retry.
assert sse_mock.await_args.kwargs.get("use_service_auth") is True
def test_proxy_api_per_ws_events_uses_user_auth_not_service(self, client, mock_collector):
"""Per-ws events route uses the user's re-minted JWT, not the
service token the upstream per-ws SSE handler scopes by
user identity for tenant filtering, and a service-scoped
call would bypass that gate. Only ``events/global``
(cross-tenant inventory by design) opts into service auth."""
from unittest.mock import AsyncMock, patch
from starlette.responses import Response
mock_collector.get_node_detail.return_value = {
"node_id": "node-a",
"server_url": "http://a:8080",
"reachable": True,
}
ws_id = "b" * 32
with patch(
"turnstone.console.server._proxy_sse",
new_callable=AsyncMock,
return_value=Response("ok", status_code=200),
) as sse_mock:
client.get(f"/node/node-a/v1/api/workstreams/{ws_id}/events")
assert sse_mock.await_count == 1
assert sse_mock.await_args.kwargs.get("use_service_auth") is False
# -------------------------------------------------------------------
# Proxied auth endpoints — handled locally by the console, not
# forwarded to the upstream node. Cases derive directly from
# ``_PROXY_AUTH_LOCAL_HANDLERS`` so a new dispatch entry can't be
# added without a matching test (or vice versa). See proxy_api's
# docstring for the JWT-audience reasoning.
# -------------------------------------------------------------------
@pytest.mark.parametrize(
("method", "path", "handler_name"),
[
(method, path, handler_name)
for (method, path), handler_name in sorted(_PROXY_AUTH_LOCAL_HANDLERS.items())
],
)
def test_proxy_auth_endpoint_dispatches_to_local_handler(
self, client, method, path, handler_name
):
"""Every entry in ``_PROXY_AUTH_LOCAL_HANDLERS`` must route to its
local console handler and never reach the upstream proxy. The
lockout class of bug this dispatch was added to fix is exactly
what a regression here would reintroduce silently covering all
eight branches keeps each path tied to its handler."""
from unittest.mock import AsyncMock, patch
from starlette.responses import JSONResponse
with (
patch(
f"turnstone.console.server.{handler_name}",
new_callable=AsyncMock,
return_value=JSONResponse({"status": "ok"}),
) as local_mock,
patch(
"turnstone.console.server._proxy_post",
new_callable=AsyncMock,
return_value=JSONResponse({"status": "should-not-be-called"}),
) as post_mock,
patch(
"turnstone.console.server._proxy_get",
new_callable=AsyncMock,
return_value=JSONResponse({"status": "should-not-be-called"}),
) as get_mock,
):
resp = client.request(method, f"/node/node-a/v1/api/{path}")
assert resp.status_code == 200
assert local_mock.await_count == 1
assert post_mock.await_count == 0
assert get_mock.await_count == 0
def test_proxy_auth_login_works_without_cookie(self, mock_collector):
"""Without this fix the AuthMiddleware 401s before any handler
runs the user is locked out of the proxied UI once the cookie
expires. Test bypasses _TEST_AUTH_HEADERS to reproduce."""
from unittest.mock import AsyncMock, patch
from starlette.responses import JSONResponse
from starlette.testclient import TestClient
from turnstone.console.server import _load_static, create_app
_load_static()
app = create_app(collector=mock_collector, jwt_secret=_TEST_JWT_SECRET)
unauth_client = TestClient(app, raise_server_exceptions=False)
try:
with patch(
"turnstone.console.server.auth_login",
new_callable=AsyncMock,
return_value=JSONResponse({"status": "ok"}),
) as local_mock:
resp = unauth_client.post(
"/node/node-a/v1/api/auth/login",
json={"username": "x", "password": "y"},
)
# AuthMiddleware must classify the proxied login path as
# public (is_public_path change) AND proxy_api must
# dispatch to the local handler (proxy_api change).
assert resp.status_code == 200, (
f"login locked out: got {resp.status_code}, body={resp.text}"
)
assert local_mock.await_count == 1
finally:
unauth_client.close()
def test_proxy_auth_wrong_method_returns_405_not_forwarded(self, client):
"""A non-canonical method on an auth path (e.g. PUT on auth/login)
must short-circuit with 405 instead of falling through to the
upstream proxy falling through would forward the request
authenticated as the console's service token (``_proxy_auth_headers``
fallback)."""
from unittest.mock import AsyncMock, patch
from starlette.responses import JSONResponse
with (
patch(
"turnstone.console.server._proxy_post",
new_callable=AsyncMock,
return_value=JSONResponse({"status": "should-not-be-called"}),
) as post_mock,
patch(
"turnstone.console.server._proxy_get",
new_callable=AsyncMock,
return_value=JSONResponse({"status": "should-not-be-called"}),
) as get_mock,
):
# PUT on a POST-only auth path → 405
put_resp = client.put("/node/node-a/v1/api/auth/login")
assert put_resp.status_code == 405
# POST on a GET-only auth path → 405
post_resp = client.post("/node/node-a/v1/api/auth/status")
assert post_resp.status_code == 405
assert post_mock.await_count == 0
assert get_mock.await_count == 0
def test_proxy_non_auth_endpoint_still_forwarded(self, client, mock_collector):
"""Sanity: only auth/* paths intercept. Other API paths still
forward to the upstream node."""
from unittest.mock import AsyncMock, patch
from starlette.responses import JSONResponse
mock_collector.get_node_detail.return_value = {
"node_id": "node-a",
"server_url": "http://a:8080",
"reachable": True,
}
with patch(
"turnstone.console.server._proxy_get",
new_callable=AsyncMock,
return_value=JSONResponse({"ok": True}),
) as proxy_mock:
resp = client.get("/node/node-a/v1/api/workstreams")
assert resp.status_code == 200
assert proxy_mock.await_count == 1
# ---------------------------------------------------------------------------
# Proxy URL rewriting unit tests (no HTTP needed)
@@ -1722,62 +1223,11 @@ class TestProxyRewriting:
assert "window.fetch" in _JS_PROXY_SHIM
assert "window.EventSource" in _JS_PROXY_SHIM
def test_js_shim_carries_node_id_placeholder(self):
"""The picker reads the current node_id from the shim's _nodeId
closure variable; the placeholder must be present and substitutable."""
from turnstone.console.server import _JS_PROXY_SHIM
def test_console_banner_contains_placeholder(self):
from turnstone.console.server import _CONSOLE_BANNER_TEMPLATE
assert "NODE_ID_PLACEHOLDER" in _JS_PROXY_SHIM
replaced = _JS_PROXY_SHIM.replace("NODE_ID_PLACEHOLDER", "node-a")
assert "node-a" in replaced
assert "NODE_ID_PLACEHOLDER" not in replaced
def test_js_shim_includes_picker_pieces(self):
"""Picker logic ships in the same IIFE as the prefix shim — verify
the moving parts are present so a future refactor doesn't silently
drop them. /v1/api/cluster/nodes is the lazy-fetch target;
#ui-header is the DOM anchor; console-node-pill is the trigger
class; ws-tab-dropdown is the menu shell we share with the
workstream chevron menu (style + behaviour parity); ArrowDown is
the keyboard-nav primitive that disambiguates this from a plain
click-only menu."""
from turnstone.console.server import _JS_PROXY_SHIM
# limit=1000 matches the collector's hard cap; without it the
# picker would silently drop nodes past the 100-default in
# clusters with >100 nodes.
assert "/v1/api/cluster/nodes?limit=1000" in _JS_PROXY_SHIM
assert "ui-header" in _JS_PROXY_SHIM
assert "console-node-pill" in _JS_PROXY_SHIM
assert "ws-tab-dropdown" in _JS_PROXY_SHIM
assert "ArrowDown" in _JS_PROXY_SHIM
assert "DOMContentLoaded" in _JS_PROXY_SHIM
def test_proxy_style_drops_banner_styles(self):
"""The legacy banner CSS classes (.console-banner, .ts-header-back-link
offsets, .dashboard-overlay top:32px hack) should be gone the new
picker lives inside #ui-header and doesn't need overlay offsets."""
from turnstone.console.server import _CONSOLE_PROXY_STYLE
assert ".console-banner" not in _CONSOLE_PROXY_STYLE
assert "dashboard-overlay" not in _CONSOLE_PROXY_STYLE
assert ".console-node-pill" in _CONSOLE_PROXY_STYLE
assert ".console-node-menu" in _CONSOLE_PROXY_STYLE
def test_proxy_style_uses_canonical_degraded_color(self):
"""Degraded health dot must use --accent (the canonical "needs
attention" token used by the cluster-overview node table at
console/static/style.css:548) and not --yellow. Yellow is reserved
for the dash-state attention dot, a stronger signal."""
from turnstone.console.server import _CONSOLE_PROXY_STYLE
assert "console-node-menu-item-dot--degraded" in _CONSOLE_PROXY_STYLE
# The degraded rule sits on its own line; assert it uses --accent
# by checking the CSS substring has --accent and not --yellow.
idx = _CONSOLE_PROXY_STYLE.find("console-node-menu-item-dot--degraded")
rule = _CONSOLE_PROXY_STYLE[idx : idx + 200]
assert "var(--accent)" in rule
assert "var(--yellow)" not in rule
assert "NODE_ID_PLACEHOLDER" in _CONSOLE_BANNER_TEMPLATE
assert "Console" in _CONSOLE_BANNER_TEMPLATE
def test_html_rewriting_changes_static_paths(self):
"""Simulate the proxy_index rewriting logic."""
@@ -1794,24 +1244,16 @@ class TestProxyRewriting:
assert 'href="/static/' not in rewritten
assert 'src="/static/' not in rewritten
def test_shim_injection_after_body(self):
"""Simulate the proxy shim injection — the shim ships the node-id
and prefix as JS literals and renders the picker at runtime, so
we assert the substituted JS literals land in the page."""
from turnstone.console.server import _CONSOLE_PROXY_STYLE, _JS_PROXY_SHIM
def test_banner_injection_after_body(self):
"""Simulate the banner injection logic."""
from turnstone.console.server import _CONSOLE_BANNER_TEMPLATE
sample_html = "<html><body><div>content</div></body></html>"
prefix = "/node/node-a"
shim_js = _JS_PROXY_SHIM.replace('"PREFIX_PLACEHOLDER"', json.dumps(prefix)).replace(
'"NODE_ID_PLACEHOLDER"', json.dumps("node-a")
)
injection = _CONSOLE_PROXY_STYLE + "<script>" + shim_js + "</script>"
result = sample_html.replace("<body>", "<body>" + injection, 1)
assert '"node-a"' in result
assert '"/node/node-a"' in result
assert "PREFIX_PLACEHOLDER" not in result
assert "NODE_ID_PLACEHOLDER" not in result
assert result.startswith("<html><body><style>")
banner = _CONSOLE_BANNER_TEMPLATE.replace("NODE_ID_PLACEHOLDER", "node-a")
result = sample_html.replace("<body>", "<body>" + banner, 1)
assert "node-a" in result
assert "Console" in result
assert result.startswith("<html><body><div")
# ---------------------------------------------------------------------------
@@ -2046,15 +1488,17 @@ class TestProxySharedStatic:
def test_proxy_shim_injected_in_html(self):
"""Verify shim is injected as inline script in proxied HTML."""
from turnstone.console.server import _JS_PROXY_SHIM
from turnstone.console.server import _CONSOLE_BANNER_TEMPLATE, _JS_PROXY_SHIM
sample_html = "<html><body><div>content</div></body></html>"
prefix = "/node/test-node"
shim_js = _JS_PROXY_SHIM.replace('"PREFIX_PLACEHOLDER"', json.dumps(prefix)).replace(
'"NODE_ID_PLACEHOLDER"', json.dumps("test-node")
banner = _CONSOLE_BANNER_TEMPLATE.replace("NODE_ID_PLACEHOLDER", "test-node")
shim = (
"<script>"
+ _JS_PROXY_SHIM.replace('"PREFIX_PLACEHOLDER"', json.dumps(prefix))
+ "</script>"
)
shim = "<script>" + shim_js + "</script>"
result = sample_html.replace("<body>", "<body>" + shim, 1)
result = sample_html.replace("<body>", "<body>" + banner + shim, 1)
assert "<script>" in result
assert "/node/test-node" in result
assert "window.fetch" in result

Some files were not shown because too many files have changed in this diff Show More