Long sessions (5000+ messages, several compactions) degraded steadily and could stop rendering entirely while the backend stayed healthy. Four hard failure mechanisms, each sufficient on its own: - Unguarded event pipeline: one throw escaping onmessage/handleEvent (e.g. renderMarkdown stack overflow on a few KB of nested "> ") stranded the streaming refs, so every later delta painted into the poisoned segment. stream_end now resets segment refs BEFORE the finalize render with a plain-text fallback (the coordinator pane's existing pattern); onmessage guards both parse and dispatch; renderMarkdown is depth-capped with throw-safe footnote-scope accounting; the streaming buffer is marked rendered only on success. - Rebuild-vs-live races: clear_ui/replay_truncated re-renders wiped events painted in the snapshot->replaceChildren window (never redelivered) and left deltas writing into detached nodes. Rebuilds now quiesce the event stream behind a token-owned queue flushed after the render; streaming refs reset on every rebuild path including refetch FAILURE; a mid-stream replay_truncated defers its re-sync to the idle edge instead of dropping the repair. - Ignored recovery floor: the global stream now handles node_snapshot and replay_truncated. Roster eviction (with a "Session ended" toast for open panes) happens only from the stream-ordered snapshot; the REST resync is merge-only and r.ok-gated so a mid-restart 503 body cannot read as an authoritative empty roster. - Unbounded growth: _agentCards released on rebuild — deliberately NOT on transport-only reconnects, which must preserve the maps or the next child event builds a duplicate card; orphan grace timers cancelled on full reload/destroy; toast queue capped with duplicate coalescing; diff previews capped at 400 rendered lines (the spread-append could throw RangeError before the approval gate painted) with the omission notice below the scroll box; raw results clamped at 64KiB. Per-event O(N) work removed from the hot paths: thinking-indicator instance ref; near-bottom cached from a passive scroll listener and re-checked at rAF pin time (a user scroll-up landing in the coalescing window wins; ResizeObserver re-engages follow after layout changes); rAF-coalesced outer and per-stream scroll pins; self-healing call_id->row/stream lookup caches; verdict lookup scoped to the row's batch; tracked retry holder; queue-controller Set replaces the whole-transcript idle sweep; rail renders rAF-coalesced; coordinator child_ws_state ticks routed to single-row updates (full render only on terminal-boundary crossings) with observer unobserve on replace. Also: the coordinator SSE-error 401 probe is un-deadened (raw fetch — authFetch never resolves a 401 — with the body inspected so a version_mismatch still takes auth.js's upgrade-reload path via the new noteVersionMismatch export); the console cluster-SSE reconnect timer is tracked across logout; the mermaid render chain is rejection-proof per link and paints errors on the containers the failing link had already claimed. Measured with scripts/livepass.py --perf (n=3000 history + 20-turn live storm): full replay 1060ms -> 238ms; re-render cycles 836-1071ms -> ~94ms flat; chunk path now flat vs transcript size; worst longtask 1080ms -> ~500ms; agent-card retention across rebuilds 4 -> 0. Known limit (needs a server-side event watermark on /history): a turn completing inside the refetch window can paint twice after the quiesce flush — rare, visible, and strictly better than the silent loss it replaces.
Turnstone
Self-hosted, local-first orchestration for tool-using AI agents. Give LLMs real tools — shell, files, search, web — and run them across your own cluster with direct HTTP routing and interactive interfaces. Your code, your models, your data stay on hardware you control: no telemetry, no phone-home.
Named after the Ruddy Turnstone (Arenaria interpres) — a shorebird that flips stones to discover what's hiding underneath.
What is a harness?
ℋ : s_{n+1} ~ T(s_n) for n < τ*, T = ρ ∘ (M_W ∘ π, E)
Release Tracks
| Track | Install | Docker | Description |
|---|---|---|---|
| Stable | pip install turnstone |
ghcr.io/turnstonelabs/turnstone:stable |
Production-grade. Bugfixes only. |
| Experimental | pip install turnstone --pre |
ghcr.io/turnstonelabs/turnstone:experimental |
New features. May have rough edges. |
See docs/releasing.md for the full release process.
What it does
Turnstone gives LLMs tools — shell, files, search, web, planning — and orchestrates multi-turn conversations where the model investigates, acts, and reports.
- Local-first & private — runs entirely on hardware you control, with no telemetry and no phone-home. Point it at local models (vLLM, llama.cpp) or commercial APIs you hold the keys to — your prompts and data never transit a third party you didn't choose.
- Bring your own models — OpenAI-compatible APIs (vLLM, llama.cpp, NIM), the Anthropic Messages API, and Google Gemini, mixed freely per role
- Interactive sessions — terminal CLI or browser UI with parallel workstreams
- Cluster dashboard — real-time view of every node and workstream, with a rendezvous routing proxy
- Intent validation — an LLM judge (your model) grades every tool call with a risk assessment and evidence before it runs
- MCP support — external tool servers with native deferred loading (Anthropic/OpenAI) or BM25 fallback
- Team controls when you need them — optional RBAC, SSO, tool policies, and audit logs, all stored in your own database
Quickstart
pip install turnstone
# Terminal REPL
turnstone --base-url http://localhost:8000/v1
# Browser UI
turnstone-server --port 8080 --base-url http://localhost:8000/v1
# Cluster dashboard
turnstone-console --port 8090
For PostgreSQL (recommended for production):
export TURNSTONE_DB_BACKEND=postgresql
export TURNSTONE_DB_URL="postgresql+psycopg://user:pass@localhost:5432/turnstone"
turnstone-server --port 8080 --base-url http://localhost:8000/v1
Docker
One-line install — autodetects Ubuntu/Debian, Fedora/RHEL, Arch, and WSL, installs git + Docker if missing, generates secrets, and starts the stack:
curl -fsSL https://raw.githubusercontent.com/turnstonelabs/turnstone/main/run.sh | bash
Or, if you already have Docker, clone the repo and run it yourself:
docker compose up
That builds one image and brings up a full local cluster — PostgreSQL, console,
Caddy, channel gateway, and 10 server nodes — with no .env required (it ships
with insecure dev defaults). Open the dashboard at https://localhost:8443 (Caddy
serves it over TLS with its own local CA — trust it once). Nodes boot without an
LLM; add model backends from the console UI.
For production (released images from ghcr.io, real secrets required), use the
bundled stack: docker compose -f turnstone/deploy/compose.yaml up.
See QUICKSTART.md for the install + troubleshooting walkthrough and docs/docker.md for Docker configuration.
Programmatic (SDK)
from turnstone.sdk import TurnstoneServer
with TurnstoneServer("http://localhost:8080", token="tok_xxx") as client:
ws = client.create_workstream(name="demo")
result = client.send_and_wait("Analyze the error logs", ws.ws_id, auto_approve=True)
print(result.content)
Tools
Built-in tools for shell, files, search, web, memory, notifications, and autonomous sub-agents — plus external tools via MCP with native deferred loading. See docs/tools.md for the full reference and docs/mcp-registry.md for MCP configuration.
Architecture
Single-node: Client → Server (direct HTTP + SSE). No external dependencies beyond the database.
Multi-node: Client → Console (rendezvous routing proxy) → Server nodes. The console picks the target node for each workstream via rendezvous (HRW) hashing over the live service registry — pure function of (ws_id, live_nodes), no stored bucket state, deterministic across readers. A node join or drop only re-routes the keys that score highest on the affected node.
| Component | Purpose |
|---|---|
turnstone |
Terminal CLI (REPL) |
turnstone-server |
Web UI + REST API + SSE events |
turnstone-console |
Cluster dashboard + routing proxy + admin panel |
turnstone-channel |
Channel gateway (Discord and Slack adapters) |
turnstone-admin |
User/token management CLI |
turnstone-eval |
Eval harness for prompt/tool optimization |
turnstone-doctor |
LLM-backed cluster diagnostics |
Diagrams
UML diagrams in docs/diagrams/:
| Diagram | Description |
|---|---|
| System Context | Components and external dependencies |
| Package Structure | Python modules and dependency graph |
| Core Engine | SessionUI, ChatSession, LLMProvider |
| Conversation Turn | Message lifecycle through the engine |
| Tool Pipeline | Prepare / approve / execute |
| Workstream States | State machine transitions |
| Console Data Flow | Dashboard data collection |
| Deployment | Docker Compose topology |
| Auth | JWT, scopes, login flows |
| Channels | Discord / Slack adapters + routing |
| Judge | Intent validation pipeline |
| OIDC | SSO authorization code flow |
Documentation
| Topic | Link |
|---|---|
| Configuration reference | docs/settings.md |
| API reference | docs/api-reference.md |
| Docker deployment | docs/docker.md |
| Intent validation (judge) | docs/judge.md |
| Governance & RBAC | docs/governance.md |
| OIDC SSO | docs/oidc.md |
| TLS / mTLS | docs/tls.md |
| Channel integrations | docs/channels.md |
| Console dashboard | docs/console.md |
| Eval harness | docs/eval.md |
| Tools reference | docs/tools.md |
| MCP integration | docs/mcp-registry.md |
Requirements
- Python 3.11+
- An OpenAI-compatible API endpoint, Anthropic API key, or Google Gemini API key
- Optional: Discord / Slack channel integrations (
pip install turnstone[discord,slack]) - Git LFS for cloning (diagram PNGs)
Community
Questions, ideas, or want to show what you're building? Join us on Discord: discord.gg/Nh3bWMacaq.
License
Apache License 2.0, as of version 1.6.0. Versions 1.5.x and earlier remain under the Business Source License 1.1 they shipped with.
