Four issues raised on the merged PR #542, evaluated and fixed: 1. **Truncated-path snap_seq bug (Copilot low-confidence, VALID).** make_events_handler's truncated branch set snap_seq = 0, disabling the live-drain _seq <= snap_seq dedup filter. Any token writer racing between register_listener_with_replay returning and the live drain's first read would land in BOTH the listener queue AND the captured snapshot text (the snapshot is emitted via in_progress_snapshot as the recovery floor), so the client double-renders. Fix lifts the snapshot capture INTO register_listener_with_replay under the same nested-lock acquire as the listener registration + buffer slice + counter read, so the returned snapshot["seq"] is the exact high-water mark the snapshot text corresponds to. Handler now uses snapshot["seq"] as snap_seq on truncated, dropping any token event with _seq <= snap_seq from the live emit. 2. **Lock-held string join in truncated path (Copilot, VALID).** "".join(ui_base._ws_inflight_content) ran inside the with ui_base._ws_lock: block, holding the lock for the duration of the join and blocking on-token writers. Fix (folded into #1's refactor): the new register_listener_with_replay copies the inflight lists under lock and joins outside, matching the existing pattern in register_listener_with_in_progress_snapshot. 3. **_strip_js_comments docstring misclaim (Copilot, VALID).** Docstring claimed the helper preserves "string/regex literals" but the implementation only tracks string delimiters. Fix: docstring updated to call out the regex-literal limitation explicitly + note that current callers don't scan regions containing regex literals. Extending the tracker is left for a future caller that needs it. 4. **Coord scheduleReconnect dead-code regression (Copilot, VALID).** After the PR-D refactor, scheduleReconnect() had no remaining call sites — which meant reconnectAttempts never incremented, wasReconnecting was always false, AND there was no fallback when the browser transitioned the source to CLOSED (hard 4xx after retries, intermediary tearing the connection down with prejudice, etc.). The first failure mode silently broke the post-gap replace-mode refresh of children / tasks / wait indicator / live-badge cache; the second left the coord permanently disconnected on non-transient failures. Fix: - Introduce disconnectedSinceLastOpen flag set in onerror, cleared in onopen. wasReconnecting reads it (with the legacy reconnectAttempts > 0 fallback for the scheduleReconnect-driven case), so the post-gap refresh fires after every reconnect including the common native-reconnect path. - Re-introduce CLOSED-state recovery: onerror schedules a 5 s delayed check via reconnectTimer; if the source is still CLOSED at that point, call scheduleReconnect(), which opens a new EventSource (threading the saved lastEventId via the URL query param so replay still works across the manual reconnect). Cancel/replace successive timers so onerror floods don't pile up multiple checks for the same source. Tests: - New test_truncated_path_snapshot_captures_real_snap_seq pins the snap_seq fix at the helper boundary. - New test_truncated_path_filters_already_in_snapshot_tokens pins the end-to-end dedup invariant — would have caught the double-render under the old code. - Existing test_sse_reconnect_replay.py call sites updated for the new 6-tuple return of register_listener_with_replay. - All 86 tests in those two files pass; full non-live suite (6347 tests) passes; ruff + mypy clean on changed files; both JS files parse-check.
Turnstone
Multi-node AI orchestration platform. Deploy tool-using AI agents across a cluster of servers with direct HTTP routing, interactive interfaces, and enterprise governance.
Named after the Ruddy Turnstone (Arenaria interpres) — a shorebird that flips stones to discover what's hiding underneath.
Release Tracks
| Track | Install | Docker | Description |
|---|---|---|---|
| Stable | pip install turnstone |
ghcr.io/turnstonelabs/turnstone:stable |
Production-grade. Bugfixes only. |
| Experimental | pip install turnstone --pre |
ghcr.io/turnstonelabs/turnstone:experimental |
New features. May have rough edges. |
See docs/releasing.md for the full release process.
What it does
Turnstone gives LLMs tools — shell, files, search, web, planning — and orchestrates multi-turn conversations where the model investigates, acts, and reports.
- Interactive sessions — terminal CLI or browser UI with parallel workstreams
- Cluster dashboard — real-time view of all nodes and workstreams with console routing proxy
- Intent validation — LLM judge evaluates every tool call with risk assessments and evidence
- Governance — RBAC, OIDC SSO, tool policies, skills, usage tracking, audit logs
- Multi-provider — OpenAI-compatible APIs (vLLM, llama.cpp, NIM), Anthropic Messages API, and Google Gemini
- MCP support — external tool servers with native deferred loading (Anthropic/OpenAI) or BM25 fallback
Quickstart
pip install turnstone
# Terminal REPL
turnstone --base-url http://localhost:8000/v1
# Browser UI
turnstone-server --port 8080 --base-url http://localhost:8000/v1
# Cluster dashboard
pip install turnstone[console]
turnstone-console --port 8090
For PostgreSQL (recommended for production):
pip install turnstone[postgres]
export TURNSTONE_DB_BACKEND=postgresql
export TURNSTONE_DB_URL="postgresql+psycopg://user:pass@localhost:5432/turnstone"
turnstone-server --port 8080 --base-url http://localhost:8000/v1
Docker
cp .env.example .env # edit LLM_BASE_URL, OPENAI_API_KEY, etc.
docker compose --profile production up
See QUICKSTART.md for the bootstrap wizard and docs/docker.md for Docker configuration and profiles.
Programmatic (SDK)
from turnstone.sdk import TurnstoneServer
with TurnstoneServer("http://localhost:8080", token="tok_xxx") as client:
ws = client.create_workstream(name="demo")
result = client.send_and_wait("Analyze the error logs", ws.ws_id, auto_approve=True)
print(result.content)
Tools
Built-in tools for shell, files, search, web, memory, notifications, and autonomous sub-agents — plus external tools via MCP with native deferred loading. See docs/tools.md for the full reference and docs/mcp-registry.md for MCP configuration.
Architecture
Single-node: Client → Server (direct HTTP + SSE). No external dependencies beyond the database.
Multi-node: Client → Console (rendezvous routing proxy) → Server nodes. The console picks the target node for each workstream via rendezvous (HRW) hashing over the live service registry — pure function of (ws_id, live_nodes), no stored bucket state, deterministic across readers. A node join or drop only re-routes the keys that score highest on the affected node.
| Component | Purpose |
|---|---|
turnstone |
Terminal CLI (REPL) |
turnstone-server |
Web UI + REST API + SSE events |
turnstone-console |
Cluster dashboard + routing proxy + admin panel |
turnstone-channel |
Channel gateway (Discord and Slack adapters) |
turnstone-admin |
User/token management CLI |
turnstone-eval |
Eval harness for prompt/tool optimization |
turnstone-bootstrap |
LLM-guided setup wizard |
Diagrams
UML diagrams in docs/diagrams/:
| Diagram | Description |
|---|---|
| System Context | Components and external dependencies |
| Package Structure | Python modules and dependency graph |
| Core Engine | SessionUI, ChatSession, LLMProvider |
| Conversation Turn | Message lifecycle through the engine |
| Tool Pipeline | Prepare / approve / execute |
| Workstream States | State machine transitions |
| Console Data Flow | Dashboard data collection |
| Deployment | Docker Compose topology |
| Auth | JWT, scopes, login flows |
| Channels | Discord / Slack adapters + routing |
| Judge | Intent validation pipeline |
| OIDC | SSO authorization code flow |
Documentation
| Topic | Link |
|---|---|
| Configuration reference | docs/settings.md |
| API reference | docs/api-reference.md |
| Docker deployment | docs/docker.md |
| Intent validation (judge) | docs/judge.md |
| Governance & RBAC | docs/governance.md |
| OIDC SSO | docs/oidc.md |
| TLS / mTLS | docs/tls.md |
| Channel integrations | docs/channels.md |
| Console dashboard | docs/console.md |
| Eval harness | docs/eval.md |
| Tools reference | docs/tools.md |
| MCP integration | docs/mcp-registry.md |
Requirements
- Python 3.11+
- An OpenAI-compatible API endpoint, Anthropic API key, or Google Gemini API key
- Optional: PostgreSQL (
pip install turnstone[postgres]), Anthropic (pip install turnstone[anthropic]) - Git LFS for cloning (diagram PNGs)
License
Business Source License 1.1 — free for all use except hosting as a managed service. Converts to Apache 2.0 on 2030-03-01.
