mirror of
https://github.com/turnstonelabs/turnstone.git
synced 2026-08-23 20:34:49 -06:00
Patrick Buckley
ba3bc9d989
Cleanup pass on the metacognitive nudge stack — restores pre-split errored-counts-toward-repeat behaviour and tightens the is_error plumbing through the per-batch advisory hook. The per-batch hook in ``_run_loop`` was duplicating the is_error signal: ``self._tool_error_flags`` (set by ``_report_tool_result``) and a string-prefix tuple (``Error`` / ``JSON parse error`` / …). Two truth sources is what got us here — bash commands that exit non-zero with normal stdout matched the flag but not the prefix, the deny path matched the prefix but not the flag, and the result was that stuck-loop detection silently broke for the most common failure mode (the model bashing the same broken command). Single source of truth now: - ``_execute_tools.run_one`` deny branch routes through ``_report_tool_result(is_error=True)`` so denied calls populate ``_tool_error_flags`` like every other error path. - The error-prefix tuple is gone; the write-success-clear gate and the tool-error-nudge gate both read ``_tool_error_flags`` only. Repeat-detection state moves from a ``set[str]`` (fired on the second identical call, ignored errors entirely) to a ``RepeatDetector`` helper in ``metacognition.py`` with consecutive-streak semantics: - Threshold raised from 2 to 3 — two-in-a-row was noisy on legitimate transient retries; three is the cheapest stuck-loop signal. - Recording a different signature resets the count, so [A, A, B, A] is two short streaks of 2 and not a streak of 4. Bounded by O(1) state regardless of session length. - Errored calls now count toward the streak (the split into a separate metacog module unintentionally introduced a "skip errors" branch — restored). While there: - ``metacognition._COOLDOWN_SECS`` default aligned to 300s (matches ``MemoryConfig.nudge_cooldown`` and the ``memory.nudge_cooldown`` config-store default; was set to 30 by an earlier investigation). - The per-batch advisory block (~80 lines of mixed orchestration inside ``_run_loop``) is extracted to ``ChatSession._apply_post_execute_advisories`` so the wired behaviour is testable without driving ``_run_loop`` end-to-end. Producer extraction to a dedicated module is deferred to a follow-up; advisory producers all live on ``ChatSession`` for now per existing convention. - Frontend ``appendToolOutput`` (turnstone/ui/static/app.js) now skips rendering when the parent approval block is denied or the output starts with ``Denied by user`` / ``Blocked``, mirroring the history-replay guard at ``_build_history``. Previously the live SSE path didn't need this guard because the deny path never emitted a ``tool_result`` event; the is_error routing change above means it does now, so without this guard the badge from ``resolveApproval`` and the SSE output would both render. Tests: 8 unit tests for ``RepeatDetector`` covering streak, threshold, clear, and intervening-sig reset; 9 integration tests for ``_apply_post_execute_advisories`` covering the wired behaviour (3-identical fires warning + advisory + UI line, errored calls count toward streak as a regression guard, intervening sig resets streak, successful write clears, failed write does not, JSON outputs tracked but not inline-warned, tool_error nudge gates on memory_count, repeat UI line emitted on streak fire).
Turnstone
Multi-node AI orchestration platform. Deploy tool-using AI agents across a cluster of servers with direct HTTP routing, interactive interfaces, and enterprise governance.
Named after the Ruddy Turnstone (Arenaria interpres) — a shorebird that flips stones to discover what's hiding underneath.
Release Tracks
| Track | Install | Docker | Description |
|---|---|---|---|
| Stable | pip install turnstone |
ghcr.io/turnstonelabs/turnstone:stable |
Production-grade. Bugfixes only. |
| Experimental | pip install turnstone --pre |
ghcr.io/turnstonelabs/turnstone:experimental |
New features. May have rough edges. |
See docs/releasing.md for the full release process.
What it does
Turnstone gives LLMs tools — shell, files, search, web, planning — and orchestrates multi-turn conversations where the model investigates, acts, and reports.
- Interactive sessions — terminal CLI or browser UI with parallel workstreams
- Cluster dashboard — real-time view of all nodes and workstreams with console routing proxy
- Intent validation — LLM judge evaluates every tool call with risk assessments and evidence
- Governance — RBAC, OIDC SSO, tool policies, skills, usage tracking, audit logs
- Multi-provider — OpenAI-compatible APIs (vLLM, llama.cpp, NIM), Anthropic Messages API, and Google Gemini
- MCP support — external tool servers with native deferred loading (Anthropic/OpenAI) or BM25 fallback
Quickstart
pip install turnstone
# Terminal REPL
turnstone --base-url http://localhost:8000/v1
# Browser UI
turnstone-server --port 8080 --base-url http://localhost:8000/v1
# Cluster dashboard
pip install turnstone[console]
turnstone-console --port 8090
For PostgreSQL (recommended for production):
pip install turnstone[postgres]
export TURNSTONE_DB_BACKEND=postgresql
export TURNSTONE_DB_URL="postgresql+psycopg://user:pass@localhost:5432/turnstone"
turnstone-server --port 8080 --base-url http://localhost:8000/v1
Docker
cp .env.example .env # edit LLM_BASE_URL, OPENAI_API_KEY, etc.
docker compose --profile production up
See QUICKSTART.md for the bootstrap wizard and docs/docker.md for Docker configuration and profiles.
Programmatic (SDK)
from turnstone.sdk import TurnstoneServer
with TurnstoneServer("http://localhost:8080", token="tok_xxx") as client:
ws = client.create_workstream(name="demo")
result = client.send_and_wait("Analyze the error logs", ws.ws_id, auto_approve=True)
print(result.content)
Tools
Built-in tools for shell, files, search, web, memory, notifications, and autonomous sub-agents — plus external tools via MCP with native deferred loading. See docs/tools.md for the full reference and docs/mcp-registry.md for MCP configuration.
Architecture
Single-node: Client → Server (direct HTTP + SSE). No external dependencies beyond the database.
Multi-node: Client → Console (rendezvous routing proxy) → Server nodes. The console picks the target node for each workstream via rendezvous (HRW) hashing over the live service registry — pure function of (ws_id, live_nodes), no stored bucket state, deterministic across readers. A node join or drop only re-routes the keys that score highest on the affected node.
| Component | Purpose |
|---|---|
turnstone |
Terminal CLI (REPL) |
turnstone-server |
Web UI + REST API + SSE events |
turnstone-console |
Cluster dashboard + routing proxy + admin panel |
turnstone-channel |
Channel gateway (Discord and Slack adapters) |
turnstone-admin |
User/token management CLI |
turnstone-eval |
Eval harness for prompt/tool optimization |
turnstone-bootstrap |
LLM-guided setup wizard |
Diagrams
UML diagrams in docs/diagrams/:
| Diagram | Description |
|---|---|
| System Context | Components and external dependencies |
| Package Structure | Python modules and dependency graph |
| Core Engine | SessionUI, ChatSession, LLMProvider |
| Conversation Turn | Message lifecycle through the engine |
| Tool Pipeline | Prepare / approve / execute |
| Workstream States | State machine transitions |
| Console Data Flow | Dashboard data collection |
| Deployment | Docker Compose topology |
| Auth | JWT, scopes, login flows |
| Channels | Discord / Slack adapters + routing |
| Judge | Intent validation pipeline |
| OIDC | SSO authorization code flow |
Documentation
| Topic | Link |
|---|---|
| Configuration reference | docs/settings.md |
| API reference | docs/api-reference.md |
| Docker deployment | docs/docker.md |
| Intent validation (judge) | docs/judge.md |
| Governance & RBAC | docs/governance.md |
| OIDC SSO | docs/oidc.md |
| TLS / mTLS | docs/tls.md |
| Channel integrations | docs/channels.md |
| Console dashboard | docs/console.md |
| Eval harness | docs/eval.md |
| Tools reference | docs/tools.md |
| MCP integration | docs/mcp-registry.md |
Requirements
- Python 3.11+
- An OpenAI-compatible API endpoint, Anthropic API key, or Google Gemini API key
- Optional: PostgreSQL (
pip install turnstone[postgres]), Anthropic (pip install turnstone[anthropic]) - Git LFS for cloning (diagram PNGs)
License
Business Source License 1.1 — free for all use except hosting as a managed service. Converts to Apache 2.0 on 2030-03-01.