Patrick Buckley b368bdeecc feat(mcp): per-user resource + prompt pool dispatch (Phase 7b)
Extends the Phase 7 per-(user, server) ClientSession pool to cover
RFC §3.2 (resources/read) and §3.3 (prompts/get) on the same shape
already proven for tools/call. Pool discovery is capability-gated so
servers without resources/ or prompts/ stay free of extra round-trips.

API additions / widenings (MCPClientManager):
- ``read_resource_sync(uri, *, user_id=None, timeout=120)`` —
  per-user-first dispatch; falls through to the byte-identical static
  path when ``user_id`` is None or the URI doesn't resolve to an
  ``oauth_user`` pool entry.
- ``get_prompt_sync(prefixed_name, arguments=None, *, user_id=None,
  timeout=30)`` — same dispatch shape; structured-error responses
  surface via ``RuntimeError`` so the agent-loop's ``except Exception``
  block renders the JSON without polluting the prompt-protocol return
  shape.
- ``get_resources(user_id=None)`` / ``get_prompts(user_id=None)`` —
  per-user merged catalogs (admin/global call still passes None).
- ``add_{resource,prompt}_listener`` /
  ``remove_{resource,prompt}_listener`` —  ``user_id`` keyword scopes
  the listener so a pool-only catalog change for one user does not
  wake another user's session.
- ``resource_count_for_user(user_id=None)`` /
  ``prompt_count_for_user(user_id=None)`` — method-form variants used
  by ChatSession's ``read_resource`` / ``use_prompt`` tool gating; the
  legacy ``resource_count`` / ``prompt_count`` properties remain
  static-only for admin paths.
- ``_dispatch_pool_resource`` / ``_dispatch_pool_prompt`` async coros
  — mirror ``_dispatch_pool`` for the new SDK calls; share the
  carrier-race-and-cancel core via ``_dispatch_pool_with_entry_call``.
- ``_handle_auth_403`` extended with ``kind=Literal["tool",
  "resource", "prompt"]`` so the per-operation ``mcp_*_forbidden``
  code surfaces (kind="tool" remains the default for back-compat).
- Pool notification handler now refreshes resources / prompts on
  ``ResourceListChangedNotification`` / ``PromptListChangedNotification``
  via ``_refresh_pool_server_resources`` / ``_refresh_pool_server_prompts``.

ChatSession (``turnstone/core/session.py``) call-site updates:
- 12 sites threaded the session-bound ``user_id`` through
  ``add_*_listener`` / ``remove_*_listener``, ``get_resources`` /
  ``get_prompts``, gating, ``read_resource_sync`` /
  ``get_prompt_sync``, and ``is_mcp_prompt`` so the per-user merged
  catalog drives both the visible-tool set and dispatch.
- ``/mcp`` slash command now lists this user's pool resources and
  prompts alongside tools (Phase 7 already scoped tools).

Scope decisions:
- Per-user-first URI ordering (decision 0.1): the dispatcher attempts
  the user's pool catalog first, falling back to the static catalog
  only when no pool entry resolves the URI / prefixed name. Pool-only
  users never see the static catalog leak into their resolution.
- Method-form ``*_count_for_user`` (vs property) keeps the legacy
  ``resource_count`` / ``prompt_count`` properties intact for admin
  endpoints whose contract is "static catalog size only".
- Shared ``_dispatch_pool_with_entry_call`` helper accepts an
  ``sdk_call: Callable[[ClientSession], Awaitable[Any]]`` closure,
  keeping the entry-locked carrier-race / classification / retry
  plumbing single-source instead of a 3x copy across tool / resource
  / prompt paths.

R6 (anyio uniformity): every pool-side list / read / get path uses
``async with asyncio.timeout(...)`` — ``asyncio.wait_for`` is
forbidden in those paths because it wraps the inner awaitable in a
fresh task and surfaces ``CancelledError`` from inside
``streamablehttp_client``'s anyio TaskGroup on Python 3.11
(per ``feedback_asyncio_timeout_vs_wait_for.md``).

Tests:
- ``test_mcp_pool_auth_resource_integration.py`` — 9 real-transport
  resource tests (FastMCP upstream + ``BehaviorMiddleware``):
  401-refresh-retry success, persistent 401 -> consent_required,
  403+insufficient_scope, 403 generic -> mcp_resource_read_forbidden,
  breaker-isolation under repeated auth failures, missing-token,
  decrypt-failure, http:// URL guard, unknown-URI ValueError.
- ``test_mcp_pool_auth_prompt_integration.py`` — 9 mirror tests for
  the prompt path; structured-error responses verified via
  ``RuntimeError`` payload shape.
- ``test_mcp_user_catalog.py`` — extended unit coverage for per-user
  resource / prompt rebuild + collision policy + symmetric eviction.
- ``test_sessions.py::TestMCPToolGating`` — pool-only-user canary
  asserts ``read_resource`` / ``use_prompt`` stay visible when the
  static catalog is empty but the user has pool entries.

Round-1 review fixes (4-finder review applied, no push yet):
- bug-1: ``_exec_use_prompt`` was hardcoding ``"MCP prompt error: failed
  to invoke prompt"`` — discarding the structured-error JSON that
  ``_dispatch_pool_prompt_sync`` raises via ``RuntimeError``. Now uses
  ``f"MCP prompt error: {e}"`` mirroring ``_exec_mcp_tool``; pool-prompt
  consent_required / insufficient_scope / forbidden errors now reach
  the LLM as intended.
- bug-2 + bug-3: resource template discovery was uncapped —
  ``_cap_server_resources`` covered ``res_result.resources`` but the
  separate ``tmpl_result.resourceTemplates`` loop appended every
  template a server returned. Added ``_MAX_RESOURCE_TEMPLATES_PER_SERVER``
  (1000) + ``_cap_server_resource_templates`` helper, applied at both
  the initial discovery site (``_connect_one_pool``) and the refresh
  site (``_refresh_pool_server_resources``). Mirrors the existing
  ``_MAX_TOOLS_PER_SERVER`` / ``_MAX_PROMPTS_PER_SERVER`` defensive
  ceilings.
- sec-1 + sec-2: ``emit_insufficient_scope_audit`` generalized to
  ``emit_oauth_failure_audit(kind, code, ...)``, called from both the
  insufficient_scope branch AND the previously-silent generic 403
  branch. Audit detail now records ``{"kind": kind, "code": code,
  "scopes_required": [...]}`` so operators can distinguish tool-call
  vs resource-read vs prompt-get 403s in audit logs and so cross-
  tenant probing on the generic 403 path leaves a trail. The Phase 7
  inherited gap (``mcp_tool_call_forbidden`` had the same silence) is
  closed in the same refactor.
- perf-1: pool resource discovery now uses ``asyncio.gather(
  list_resources, list_resource_templates)`` inside the existing
  ``async with asyncio.timeout(...)`` budget — disjoint catalogs, no
  ordering dependency. Typical-case 2-RTT cold-connect resource block
  collapses to 1-RTT. Same change applied at ``_refresh_pool_server_resources``.
- q-1: ``_rebuild_user_prompt_map`` docstring corrected RFC §3.2 →
  §3.3 (resources are §3.2; prompts are §3.3).
- q-2: ``_refresh_pool_server_prompts`` docstring now carries the
  R6 / mcp-loop note that the resource sibling already had — both
  refresh paths now declare the asyncio.timeout invariant explicitly.
- q-5: added the ``_user_resource_map`` / DB-mismatch guard to
  ``read_resource_sync`` for parity with ``get_prompt_sync``. A stale
  per-user map entry with no matching oauth_user row now raises a
  specific ValueError instead of silently falling through to a
  generic ``Unknown MCP resource``.
- q-6: ``_dispatch_pool_with_entry`` (now a single-caller wrapper
  after the ``_dispatch_pool_with_entry_call`` extraction) gains a
  one-line docstring explaining why the wrapper is preserved
  (tool-decode localization + stack-trace identity for debugging).
- q-7: added 1 resource + 1 prompt end-to-end integration test that
  drive REAL discovery + dispatch in the same connect (no
  ``_seed_pool_*_map`` shortcuts), mirroring the tool path's
  ``test_integration_pool_reuse_401_refresh_and_retry_succeeds``.
  The seeded-map tests stay (faster, focused on dispatch); the new
  e2e tests cover the connect-discover-dispatch composition that
  caught Phase 6's carrier-on-entry bug.

Pre-push round-1 review fixes (3-finder review on the final state —
the lesson from Phase 7 round-3's q-1 regression: round-2 catches
what the round-1 apply pass missed):
- q-1 (MAJOR): the bug-1 sibling that round-1 missed —
  ``_exec_read_resource`` was hardcoding ``"MCP resource error: failed
  to read resource"`` while ``_exec_use_prompt`` (post-bug-1) preserved
  the structured-error JSON via ``f"... error: {e}"``. The round-1
  apply pass patched the prompt side but not the resource side. q-5's
  per-user-map / DB-mismatch ValueError was being swallowed at the
  agent loop boundary, defeating the operator-diagnostic intent. Now
  ``_exec_read_resource`` mirrors ``_exec_mcp_tool`` and ``_exec_use_prompt``.
- q-6 (nit): defensive-cap comment block at module-level cited
  "(RFC §3.2)" while covering both resource and prompt list paths;
  prompts are §3.3. Now reads "(RFC §3.2 for resources, §3.3 for
  prompts)" matching the convention the q-1 apply established.
- q-5 (rejected with better justification): the reviewer flagged
  ``_dispatch_pool_with_entry`` as a single-caller wrapper that should
  be inlined. After examination — the autouse fixture
  ``tests/test_mcp_pool_auth_introspection.py::_install_capture_intercept``
  monkeypatches this method to stash ``entry.auth_capture`` for the
  fake call_tool stubs in dispatcher-asserting tests. Inlining would
  redirect the patch to ``_dispatch_pool_with_entry_call`` (different
  kwargs shape) and require re-validating every test that depends on
  the interception. The wrapper IS load-bearing; q-6 docstring updated
  to cite the test-fixture rationale instead of the thin "stack-trace
  identity" claim.

Deferred to follow-up (documented rationale):
- perf-2: single-pass partition for system-message resource list
  (concrete vs templates). Sub-microsecond at expected scale;
  opportunistic-only.
- q-2 (pre-push): ~200 lines of fixture infrastructure
  (``BehaviorMiddleware``, ``_build_server``, ``_seed_oauth_server``,
  ``running_loop_mgr``, etc.) duplicated across three pool-integration
  test files. Real maintenance cost, but a 200-line conftest extraction
  is a focused refactor that earns its own commit / PR. Tracking as
  follow-up rather than balloon Phase 7b's diff further.
- q-3 / q-4 (refactor): extract shared dispatcher / scheduler
  helpers to compress three near-identical 90-line bodies (round-1
  q-3 was the same root cause; the pre-push q-3/q-4 reviewer
  reaffirmed it concretely). Three named methods preserve readability
  for the codebase's hottest correctness path; follow-up if
  duplication grows further or if a per-path divergence ships.
- q-4 (round-1, distinct from pre-push q-4): split pool concerns
  into ``mcp_pool.py``. Out-of-scope per finder; future refactor as
  the file approaches the navigation/merge-conflict threshold.

3.13: 5590 passed (5541 baseline -> +49 net; pre-review +47, q-7
e2e tests added +2). Existing audit-detail tests updated in-place
to expect the new ``kind`` and ``code`` fields.
3.11: 5590 passed (parity gate per ``feedback_pytest_env_parity.md``).

(cherry picked from commit 124615cce0)
2026-05-07 17:35:22 -07:00

Turnstone

CI PyPI Python License

Multi-node AI orchestration platform. Deploy tool-using AI agents across a cluster of servers with direct HTTP routing, interactive interfaces, and enterprise governance.

Turnstone coordinator — parallel tool batches with judge-graded approval and child workstream tracking

Named after the Ruddy Turnstone (Arenaria interpres) — a shorebird that flips stones to discover what's hiding underneath.

Release Tracks

Track Install Docker Description
Stable pip install turnstone ghcr.io/turnstonelabs/turnstone:stable Production-grade. Bugfixes only.
Experimental pip install turnstone --pre ghcr.io/turnstonelabs/turnstone:experimental New features. May have rough edges.

See docs/releasing.md for the full release process.

What it does

Turnstone gives LLMs tools — shell, files, search, web, planning — and orchestrates multi-turn conversations where the model investigates, acts, and reports.

  • Interactive sessions — terminal CLI or browser UI with parallel workstreams
  • Cluster dashboard — real-time view of all nodes and workstreams with console routing proxy
  • Intent validation — LLM judge evaluates every tool call with risk assessments and evidence
  • Governance — RBAC, OIDC SSO, tool policies, skills, usage tracking, audit logs
  • Multi-provider — OpenAI-compatible APIs (vLLM, llama.cpp, NIM), Anthropic Messages API, and Google Gemini
  • MCP support — external tool servers with native deferred loading (Anthropic/OpenAI) or BM25 fallback

Turnstone system architecture

Quickstart

pip install turnstone

# Terminal REPL
turnstone --base-url http://localhost:8000/v1

# Browser UI
turnstone-server --port 8080 --base-url http://localhost:8000/v1

# Cluster dashboard
pip install turnstone[console]
turnstone-console --port 8090

For PostgreSQL (recommended for production):

pip install turnstone[postgres]
export TURNSTONE_DB_BACKEND=postgresql
export TURNSTONE_DB_URL="postgresql+psycopg://user:pass@localhost:5432/turnstone"
turnstone-server --port 8080 --base-url http://localhost:8000/v1

Docker

cp .env.example .env  # edit LLM_BASE_URL, OPENAI_API_KEY, etc.
docker compose --profile production up

See QUICKSTART.md for the bootstrap wizard and docs/docker.md for Docker configuration and profiles.

Programmatic (SDK)

from turnstone.sdk import TurnstoneServer

with TurnstoneServer("http://localhost:8080", token="tok_xxx") as client:
    ws = client.create_workstream(name="demo")
    result = client.send_and_wait("Analyze the error logs", ws.ws_id, auto_approve=True)
    print(result.content)

Tools

Built-in tools for shell, files, search, web, memory, notifications, and autonomous sub-agents — plus external tools via MCP with native deferred loading. See docs/tools.md for the full reference and docs/mcp-registry.md for MCP configuration.

Architecture

Single-node: Client → Server (direct HTTP + SSE). No external dependencies beyond the database.

Multi-node: Client → Console (rendezvous routing proxy) → Server nodes. The console picks the target node for each workstream via rendezvous (HRW) hashing over the live service registry — pure function of (ws_id, live_nodes), no stored bucket state, deterministic across readers. A node join or drop only re-routes the keys that score highest on the affected node.

Component Purpose
turnstone Terminal CLI (REPL)
turnstone-server Web UI + REST API + SSE events
turnstone-console Cluster dashboard + routing proxy + admin panel
turnstone-channel Channel gateway (Discord and Slack adapters)
turnstone-admin User/token management CLI
turnstone-eval Eval harness for prompt/tool optimization
turnstone-bootstrap LLM-guided setup wizard

Diagrams

UML diagrams in docs/diagrams/:

Diagram Description
System Context Components and external dependencies
Package Structure Python modules and dependency graph
Core Engine SessionUI, ChatSession, LLMProvider
Conversation Turn Message lifecycle through the engine
Tool Pipeline Prepare / approve / execute
Workstream States State machine transitions
Console Data Flow Dashboard data collection
Deployment Docker Compose topology
Auth JWT, scopes, login flows
Channels Discord / Slack adapters + routing
Judge Intent validation pipeline
OIDC SSO authorization code flow

Documentation

Topic Link
Configuration reference docs/settings.md
API reference docs/api-reference.md
Docker deployment docs/docker.md
Intent validation (judge) docs/judge.md
Governance & RBAC docs/governance.md
OIDC SSO docs/oidc.md
TLS / mTLS docs/tls.md
Channel integrations docs/channels.md
Console dashboard docs/console.md
Eval harness docs/eval.md
Tools reference docs/tools.md
MCP integration docs/mcp-registry.md

Requirements

  • Python 3.11+
  • An OpenAI-compatible API endpoint, Anthropic API key, or Google Gemini API key
  • Optional: PostgreSQL (pip install turnstone[postgres]), Anthropic (pip install turnstone[anthropic])
  • Git LFS for cloning (diagram PNGs)

License

Business Source License 1.1 — free for all use except hosting as a managed service. Converts to Apache 2.0 on 2030-03-01.

S
Description
No description provided
Readme Apache-2.0 132 MiB
Languages
Python 87.8%
JavaScript 8.6%
CSS 1.8%
HTML 0.9%
TypeScript 0.5%
Other 0.3%