Patrick Buckley 5b449e502d fix(sse): address PR #542 review findings
Four issues raised on the merged PR #542, evaluated and fixed:

1. **Truncated-path snap_seq bug (Copilot low-confidence, VALID).**
   make_events_handler's truncated branch set snap_seq = 0,
   disabling the live-drain _seq <= snap_seq dedup filter.  Any
   token writer racing between register_listener_with_replay
   returning and the live drain's first read would land in BOTH the
   listener queue AND the captured snapshot text (the snapshot is
   emitted via in_progress_snapshot as the recovery floor), so
   the client double-renders.  Fix lifts the snapshot capture INTO
   register_listener_with_replay under the same nested-lock
   acquire as the listener registration + buffer slice + counter
   read, so the returned snapshot["seq"] is the exact
   high-water mark the snapshot text corresponds to.  Handler now
   uses snapshot["seq"] as snap_seq on truncated, dropping
   any token event with _seq <= snap_seq from the live emit.

2. **Lock-held string join in truncated path (Copilot, VALID).**
   "".join(ui_base._ws_inflight_content) ran inside the
   with ui_base._ws_lock: block, holding the lock for the
   duration of the join and blocking on-token writers.  Fix (folded
   into #1's refactor): the new register_listener_with_replay
   copies the inflight lists under lock and joins outside, matching
   the existing pattern in
   register_listener_with_in_progress_snapshot.

3. **_strip_js_comments docstring misclaim (Copilot, VALID).**
   Docstring claimed the helper preserves "string/regex literals"
   but the implementation only tracks string delimiters.  Fix:
   docstring updated to call out the regex-literal limitation
   explicitly + note that current callers don't scan regions
   containing regex literals.  Extending the tracker is left for
   a future caller that needs it.

4. **Coord scheduleReconnect dead-code regression (Copilot, VALID).**
   After the PR-D refactor, scheduleReconnect() had no remaining
   call sites — which meant reconnectAttempts never incremented,
   wasReconnecting was always false, AND there was no
   fallback when the browser transitioned the source to CLOSED
   (hard 4xx after retries, intermediary tearing the connection
   down with prejudice, etc.).  The first failure mode silently
   broke the post-gap replace-mode refresh of children / tasks /
   wait indicator / live-badge cache; the second left the coord
   permanently disconnected on non-transient failures.  Fix:
   - Introduce disconnectedSinceLastOpen flag set in onerror,
     cleared in onopen.  wasReconnecting reads it (with the
     legacy reconnectAttempts > 0 fallback for the
     scheduleReconnect-driven case), so the post-gap refresh fires
     after every reconnect including the common native-reconnect
     path.
   - Re-introduce CLOSED-state recovery: onerror schedules a 5 s
     delayed check via reconnectTimer; if the source is still
     CLOSED at that point, call scheduleReconnect(), which
     opens a new EventSource (threading the saved
     lastEventId via the URL query param so replay still works
     across the manual reconnect).  Cancel/replace successive
     timers so onerror floods don't pile up multiple checks for
     the same source.

Tests:
- New test_truncated_path_snapshot_captures_real_snap_seq pins
  the snap_seq fix at the helper boundary.
- New test_truncated_path_filters_already_in_snapshot_tokens
  pins the end-to-end dedup invariant — would have caught the
  double-render under the old code.
- Existing test_sse_reconnect_replay.py call sites updated for
  the new 6-tuple return of register_listener_with_replay.
- All 86 tests in those two files pass; full non-live suite (6347
  tests) passes; ruff + mypy clean on changed files; both JS files
  parse-check.
2026-05-22 19:34:52 -07:00

Turnstone

CI PyPI Python License

Multi-node AI orchestration platform. Deploy tool-using AI agents across a cluster of servers with direct HTTP routing, interactive interfaces, and enterprise governance.

Turnstone coordinator — parallel tool batches with judge-graded approval and child workstream tracking

Named after the Ruddy Turnstone (Arenaria interpres) — a shorebird that flips stones to discover what's hiding underneath.

Release Tracks

Track Install Docker Description
Stable pip install turnstone ghcr.io/turnstonelabs/turnstone:stable Production-grade. Bugfixes only.
Experimental pip install turnstone --pre ghcr.io/turnstonelabs/turnstone:experimental New features. May have rough edges.

See docs/releasing.md for the full release process.

What it does

Turnstone gives LLMs tools — shell, files, search, web, planning — and orchestrates multi-turn conversations where the model investigates, acts, and reports.

  • Interactive sessions — terminal CLI or browser UI with parallel workstreams
  • Cluster dashboard — real-time view of all nodes and workstreams with console routing proxy
  • Intent validation — LLM judge evaluates every tool call with risk assessments and evidence
  • Governance — RBAC, OIDC SSO, tool policies, skills, usage tracking, audit logs
  • Multi-provider — OpenAI-compatible APIs (vLLM, llama.cpp, NIM), Anthropic Messages API, and Google Gemini
  • MCP support — external tool servers with native deferred loading (Anthropic/OpenAI) or BM25 fallback

Turnstone system architecture

Quickstart

pip install turnstone

# Terminal REPL
turnstone --base-url http://localhost:8000/v1

# Browser UI
turnstone-server --port 8080 --base-url http://localhost:8000/v1

# Cluster dashboard
pip install turnstone[console]
turnstone-console --port 8090

For PostgreSQL (recommended for production):

pip install turnstone[postgres]
export TURNSTONE_DB_BACKEND=postgresql
export TURNSTONE_DB_URL="postgresql+psycopg://user:pass@localhost:5432/turnstone"
turnstone-server --port 8080 --base-url http://localhost:8000/v1

Docker

cp .env.example .env  # edit LLM_BASE_URL, OPENAI_API_KEY, etc.
docker compose --profile production up

See QUICKSTART.md for the bootstrap wizard and docs/docker.md for Docker configuration and profiles.

Programmatic (SDK)

from turnstone.sdk import TurnstoneServer

with TurnstoneServer("http://localhost:8080", token="tok_xxx") as client:
    ws = client.create_workstream(name="demo")
    result = client.send_and_wait("Analyze the error logs", ws.ws_id, auto_approve=True)
    print(result.content)

Tools

Built-in tools for shell, files, search, web, memory, notifications, and autonomous sub-agents — plus external tools via MCP with native deferred loading. See docs/tools.md for the full reference and docs/mcp-registry.md for MCP configuration.

Architecture

Single-node: Client → Server (direct HTTP + SSE). No external dependencies beyond the database.

Multi-node: Client → Console (rendezvous routing proxy) → Server nodes. The console picks the target node for each workstream via rendezvous (HRW) hashing over the live service registry — pure function of (ws_id, live_nodes), no stored bucket state, deterministic across readers. A node join or drop only re-routes the keys that score highest on the affected node.

Component Purpose
turnstone Terminal CLI (REPL)
turnstone-server Web UI + REST API + SSE events
turnstone-console Cluster dashboard + routing proxy + admin panel
turnstone-channel Channel gateway (Discord and Slack adapters)
turnstone-admin User/token management CLI
turnstone-eval Eval harness for prompt/tool optimization
turnstone-bootstrap LLM-guided setup wizard

Diagrams

UML diagrams in docs/diagrams/:

Diagram Description
System Context Components and external dependencies
Package Structure Python modules and dependency graph
Core Engine SessionUI, ChatSession, LLMProvider
Conversation Turn Message lifecycle through the engine
Tool Pipeline Prepare / approve / execute
Workstream States State machine transitions
Console Data Flow Dashboard data collection
Deployment Docker Compose topology
Auth JWT, scopes, login flows
Channels Discord / Slack adapters + routing
Judge Intent validation pipeline
OIDC SSO authorization code flow

Documentation

Topic Link
Configuration reference docs/settings.md
API reference docs/api-reference.md
Docker deployment docs/docker.md
Intent validation (judge) docs/judge.md
Governance & RBAC docs/governance.md
OIDC SSO docs/oidc.md
TLS / mTLS docs/tls.md
Channel integrations docs/channels.md
Console dashboard docs/console.md
Eval harness docs/eval.md
Tools reference docs/tools.md
MCP integration docs/mcp-registry.md

Requirements

  • Python 3.11+
  • An OpenAI-compatible API endpoint, Anthropic API key, or Google Gemini API key
  • Optional: PostgreSQL (pip install turnstone[postgres]), Anthropic (pip install turnstone[anthropic])
  • Git LFS for cloning (diagram PNGs)

License

Business Source License 1.1 — free for all use except hosting as a managed service. Converts to Apache 2.0 on 2030-03-01.

S
Description
No description provided
Readme Apache-2.0 114 MiB
Languages
Python 87.6%
JavaScript 8.7%
CSS 1.8%
HTML 1%
TypeScript 0.5%
Other 0.3%