* feat(deps): add altair + vl-convert-python viz stack
The standard, ui://-ready visualization stack: one Vega-Lite spec renders to
static SVG via vl-convert (a bundled Rust renderer — no browser, GDAL, or
chromium) and drops into vega-embed for interactive ui:// panels. The first
consumer is the civic-records choropleth map; future ui:// surfaces build on
the same stack.
The dependency closure is fully permissive (BSD-3 + the OFL font + MIT/ISC JS) —
clean for Apache-2.0 and commercial use. Adds a mypy override for the untyped
vl_convert wheel.
* fix(deps): bump pydantic-settings to 2.14.2 (GHSA-4xgf-cpjx-pc3j)
Clears the pip-audit --strict advisory on the transitive pydantic-settings
2.14.1. Pinned as an explicit security floor in [project.dependencies]
(matching the starlette/cryptography CVE-floor convention) even though it is
transitive-only, so the floor is documented and survives re-resolution.
* Potential fix for pull request finding
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* Potential fix for pull request finding
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* chore: regenerate uv.lock to pass lock check
---------
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Background daemons, event loops, and test servers that outlived their test
bled into later tests' captured output — an intermittent "I/O operation on
closed file" heisenbug, and the same class behind a past multi-day CI-hang
investigation.
- conftest: a fail-on-leak autouse guard (`_no_leaked_threads`) snapshots
threads at setup and fails any test that leaves one running past teardown,
with an `allow_thread_leak` opt-out — so the next leak is caught in minutes,
not days. Plus `logging.raiseExceptions = False` to mute the benign
logging-vs-capture-teardown race, and shared loop/server teardown helpers
(`stop_loop_thread`, `serve_until_exit`).
- collector (PRODUCT FIX): the node-discovery loop slept uninterruptibly, so
`ClusterCollector.stop()` couldn't join the `console-discovery` thread until
the full interval elapsed — a real shutdown hang in production (up to
`discovery_interval`). It now sleeps on an interruptible Event that `stop()`
sets and `start()` clears.
- test fixtures: docker_healthcheck's HTTP servers, the MCP background event
loops (shutdown_default_executor + close), and the FastMCP uvicorn upstreams
(timeout_graceful_shutdown=0 + force_exit) now tear down cleanly instead of
leaking.
Full non-live suite: 7456 passed, 0 closed-file errors, 0 leaked threads, and
~1.5 min faster (the leaks were dragging it).
main's lockfile was already on the patched versions (cryptography 49.0.0,
starlette 1.3.1) via renovate, but the pyproject floors (>=42, >=1.0.1) still
permitted a regression to vulnerable versions. Raise the floors to match
stable/1.6's v1.6.7 security fix:
- cryptography >=48.0.1 (GHSA-537c-gmf6-5ccf — bundled OpenSSL vulnerable <48.0.1)
- starlette >=1.3.1 (CVE-2026-54282 host spoof + CVE-2026-54283 url-encoded form DoS)
A vision-capable model that can't ingest PDF natively now gets the PDF
rendered to one image per page instead of extracted text; falls back to
text extraction when rendering yields nothing.
- core/pdf.py: rasterize_pdf via pypdfium2 render + Pillow PNG (page-capped
at 10, never raises)
- session._wire_content_part: pdf + !supports_pdf + supports_vision ->
rasterized image parts; else text extraction
- trajectory.resolve_attachment_parts: a placeholder can now expand to a
list of parts (1->N); the resolve_attachments callback return type widened
to dict[str, Any] across the provider protocol + 4 providers
- pyproject: pillow dependency
- tests: rasterize_pdf, vision-rasterize gate path, 1->N materialization
When the active model can't ingest a kind natively, the wire resolver
converts it client-side instead of sending a part the model can't read.
Per-kind ownership, no shared machinery: PDF text-extraction is a
pure-local PDF concern; audio transcription is an STT concern memoized
in the audio domain.
- core/pdf.py: extract_pdf_text via pypdfium2 (pure-local, no network, no
cache — re-run per build; page-capped)
- core/audio.py: transcribe_cached — non-raising, memoized by
(alias, content-hash); backend failures not cached
- session._wire_content_part: per-kind dispatch — native where the model
supports the kind (supports_pdf / supports_audio_input), else fallback;
display/export resolve natively so no conversion fires on a render
- image left ungated (pre-existing behavior unchanged)
- pyproject: pypdfium2 dependency + mypy untyped-import override
- tests: pdf extraction, transcript memoization, per-kind gate dispatch
121 warnings -> 0. Two upstream deprecations get narrowly-scoped
filterwarnings entries (the mcp streamablehttp_client rename — adoption
deliberately rides the v2 migration since the new entry point's call
shape changes again there; the starlette httpx TestClient notice). The
one real RuntimeWarning is fixed at the source: tests that mock
asyncio.run_coroutine_threadsafe handed real coroutines to a stub that
never awaited them, GC-firing 'coroutine was never awaited' inside
whatever unrelated test ran later (the same cross-test bleed mechanism
as the CI closed-stream spew — per-test filterwarnings markers cannot
catch it, which is why two such markers existed and still leaked). A
shared _dispatch_stub now closes real coroutines before returning the
canned future; the obsolete markers are removed.
mcp 2.0.0a1 shipped 2026-06-11 (stable targeted ~2026-07-27). v2 removes
streamablehttp_client, changes the transport tuple arity, and renames
mcp.types fields to snake_case — all of which our client imports. The
maintainers' release note asks downstream packages to add an upper
bound now (their worked example is this exact constraint). Floor stays
at 1.27: nothing newer adds anything our surface needs, and the #2147
shutdown busy-loop we wrap remains unfixed at every released version.
Resolution is unchanged (1.27.2); lockfile re-pinned metadata only.
* chore: relicense BUSL-1.1 -> Apache 2.0 for 1.6.0
Flips every license artifact in the tree; 1.5.x and earlier remain
BUSL-1.1 per their release-time LICENSE files. Contributor consent
record: #548 (rationale: #546).
- LICENSE: canonical Apache 2.0 text
- NOTICE: new; copyright line + pointer to THIRD-PARTY-NOTICES
- pyproject.toml: SPDX expression + explicit license-files trio
- Dockerfile: COPY the license trio (hatchling needs them at build)
- THIRD-PARTY-NOTICES: BUSL line reworded; bundled-version drift
fixed (KaTeX 0.17.0, Mermaid 11.15.0, hls.js 1.6.16)
- README badge + License section, CONTRIBUTING inbound-license line,
TS SDK package(+lock), example pyproject
- docs/pgbouncer.md: drop stray ':' introduced in #353
* docs: add CONTRIBUTORS.md
* chore: drop LICENSE leading blank line
The apache.org LICENSE-2.0.txt begins with a newline; the SPDX
canonical text and GitHub license templates do not. Use the
conventional form — detection is whitespace-normalized either way.
- claude-fable-5 capability entry: 1M context / 128K output, adaptive
thinking (summarized display), effort low..max incl. xhigh, no
sampling params, web + tool search, vision, reasoning replay, native
mid-conversation system messages
- document the Fable 5 wire quirk at the capability table: an explicit
thinking={"type": "disabled"} is a 400 on this model; the adaptive
branch never emits "disabled", so adaptive-or-omitted is preserved
- widen the native mid-conversation-system comments from opus-4-8-only
to opus-4-8 + fable-5 (protocol, provider, tool_advisory, prompts,
session)
- raise the anthropic SDK floor 0.39 -> 0.108: 0.39 predates every
named kwarg the provider sends (output_config 0.77, top-level
cache_control 0.83, mid-conversation system blocks 0.105); 0.108
adds claude-fable-5
- tests: capability assertions for claude-fable-5 + dated-variant
prefix match
The Anthropic SDK provider was the lone first-class provider gated behind
an optional extra, while OpenAI ships in core and Google rides the
OpenAI-compatible path. Fold anthropic, psycopg (postgres), croniter
(console), and lacme (tls) into the base dependency set so a default
`pip install turnstone` yields a complete single- or multi-node
deployment; only the Discord/Slack channel gateways stay optional.
- pyproject: four extras → base deps; `all` is now discord+slack; drop the
redundant croniter from the `test` extra; regenerate uv.lock.
- ci: the postgres test job installs `.[test]` (psycopg is base now).
- providers: `_ensure_anthropic` becomes a thin SDK accessor for
`create_client`; drop the now-redundant eager import-guard calls from
the streaming/completion hot path (anthropic is always present).
- bootstrap: import anthropic directly.
- tests/docs: drop the anthropic importorskips and stale extra-install hints.
`man` and `math` duplicated capabilities already reachable through
`bash`; `plan_agent` is better expressed as a `task_agent` running a
planning skill, and carried a large amount of special-case machinery
(plan-review gate, refinement loop, per-kind model routing). Removing
all three shrinks the tool surface and cuts per-call token cost.
Also removed, as dead-once-the-tools-are-gone:
- the `math` sandbox executor (`turnstone.core.sandbox`) and its
`[sandbox]` extra; the eval analyst now runs bash-only
- the read-only `AGENT_TOOLS` sub-agent tool set and the `agent`
tool-metadata key (`task_agent`/`TASK_AGENT_TOOLS` retained)
- the plan-review protocol end to end: the `on_plan_review` UI hook,
`resolve_plan`, `POST /v1/api/plan` + `POST /v1/api/route/plan`,
the `plan_review`/`plan_resolved` SSE events, and their Python SDK /
TypeScript SDK / OpenAPI / frontend / Discord+Slack bindings
- the `model.plan_alias` / `model.plan_effort` settings and the
registry `plan_model` / `plan_effort` routing fields
TOOLS 31->28, TASK_AGENT_TOOLS 13->11; COORDINATOR_TOOLS unchanged.
BREAKING CHANGE: removes the `man`, `math`, `plan_agent` tools, the
plan-review SSE/HTTP/SDK surface, and the plan_* model-routing settings
from the experimental 1.6 line.
Drop the Tavily and DuckDuckGo (ddgs) web_search backends for a single
self-hosted SearxNG service bundled into the docker-compose stacks.
Core:
- New SearXNGClient + _format_searxng; rewrite resolve_web_search_client to
(backend, searxng_url, searxng_engines, ...). MCP backend + oauth_user guard
unchanged. _resolve_search_client follows storage -> toml -> env -> default
precedence (explicit "" disables, via ConfigStore.stored_keys()).
- Drop the Tavily-era topic=finance (no SearxNG category); topic is now
general/news.
Settings/config:
- Remove tools.tavily_api_key, get_tavily_key, $TAVILY_API_KEY, [api].tavily_key.
- Add tools.searxng_url (default http://searxng:8080) + tools.searxng_engines,
with get_searxng_url/get_searxng_engines.
Compose + bundled config:
- Internal-only searxng service (no published API port, :ro config, /healthz
healthcheck, persistent searxng-cache volume) in both stacks; bundle
turnstone/deploy/searxng/settings.yml (JSON output on, limiter off).
- Caddy serves the SearxNG web UI on :8444 (dev: localhost-only; prod: opt-in).
- bootstrap extractor + wheel packaging updated.
Deps: drop the ddg extra + ddgs mypy override (regenerates uv.lock, removing the
lxml/h2/brotli transitives).
Docs: tools/docker/architecture/openshell + diagrams + config example + CHANGELOG;
docs/docker.md carries the AGPL-3.0 §13 operator note.
BREAKING: tools.web_search_backend no longer accepts "tavily"/"ddg";
tools.tavily_api_key and the ddg extra are removed. Run the bundled SearxNG (ships
in the compose stacks) or set TURNSTONE_SEARXNG_URL to an external instance.
Closes#545
`docker compose up` from a clone builds one image and brings up the whole stack
— PostgreSQL, console, Caddy, channel, and 10 server nodes — sharing one
Postgres so the console discovers every node. The dashboard is reachable only
through Caddy (HTTP/2 avoids the browser's 6-connection cap on the dashboard's
SSE streams); the console's plain-HTTP port is no longer published. Postgres
binds 127.0.0.1 so a bare-metal turnstone-server can join the cluster — the
bare-metal overlay is folded in and removed. Insecure dev defaults keep it
zero-config; the bundled production stack mirrors the shape but pulls ghcr
images and requires real secrets.
Move the Caddyfile under turnstone/deploy so it ships in the wheel; update docs,
QUICKSTART, and .env.example to match.
Starlette 1.0.0 reconstructs request URLs without validating the Host
header, allowing path-injection that can bypass authentication on apps
comparing reconstructed URL paths instead of `request.url.path`. Fixed
in 1.0.1.
- pyproject.toml: bump `starlette>=0.45` to `starlette>=1.0.1` so the
CVE floor is explicit at the dependency declaration, not just in the
lockfile. Annotated with the advisory ID so the rationale survives
a future floor relax.
- uv.lock: regenerated via `uv lock --upgrade-package starlette`;
starlette 1.0.0 -> 1.0.1, no transitive bumps.
Locally verified `pip-audit --strict` returns clean after the bump and
the auth + service-boundary test suites (250 tests covering the URL/
host-header reconstruction surface) continue to pass.
* feat(reasoning): Phase 5 — vLLM Chat Completions reasoning-field replay
Multi-turn CoT replay for vLLM-served reasoning models (Qwen3, DeepSeek-R1)
via the non-standard `reasoning` field on assistant messages. Closes the
PR #498 gap claiming Chat Completions has no replay surface — vLLM's
ChatMessage.reasoning input field is that surface (verified in
vllm/entrypoints/openai/chat_completion/protocol.py:54-64).
Session-level attach (no provider class changes). Three-gate composite:
provider isinstance OpenAIChatCompletionsProvider AND
server_compat.server_type == "vllm" AND operator-set
ModelConfig.replay_reasoning_to_model. Deliberately drops the
supports_reasoning_replay capability gate that protects Paths 1+2 —
vLLM's failure mode is silent (template-drop), not loud (server 400),
so the static gate would add operator friction without preventing the
silent failure. Server-type pin bounds blast radius — canonical OpenAI,
llama.cpp, sglang never see the non-standard field.
Also fixes a pre-existing _resolve_server_type bug: it read
cfg.capabilities.get("server_compat") but the model_registry loader pops
server_compat OUT of capabilities into the dedicated cfg.server_compat
dataclass field (model_registry.py:401, 485). Pre-fix the function
returned "" for every production ModelConfig, silently degrading PR #498
Path 3's synth-block source tag and would have made Phase 5 dead-on-
arrival. Test stubs across 3 files updated to mirror production shape
(empty capabilities + populated top-level server_compat) so the same
stub-drift can't hide future regressions.
The agent _run_agent path is deliberately excluded from Phase 5 hoists:
agent assistant messages don't carry _provider_content (rebuilt per
invocation from CompletionResult.content + tool_calls), so the helper
would no-op every turn. Comment at session.py inside _api_call documents
the exclusion.
OpenAI SDK version pin raised to >=2.37 to match the version verified
by the cross-boundary regression test
(test_reasoning_field_present_in_wire_body_when_attached) — drives a
real OpenAI client through httpx MockTransport and asserts the
non-standard field reaches the captured POST body, catching any future
SDK version that adds runtime field filtering.
Tests: 10 helper unit + 17 session integration (incl. SDK boundary
round-trip + per-gate negative tests + call-site wiring tests) + 2
audit-log discipline tests extending the PR #498 logging contract.
* docs(reasoning): apply PR #537 review on Phase 5 docstrings
Two nits from PR #537 review:
1. `_resolve_server_type` docstring claimed Phase 5 (`_maybe_attach_vllm_chat_reasoning`) called it; in fact Phase 5 reads `cfg.server_compat["server_type"]` directly off the single cfg it fetches for the operator-flag check, to avoid a second `registry.get_config` round-trip. Rewrite the paragraph: name `_maybe_synth_reasoning_block` as the sole caller (informational metadata for UI rehydration), then a separate paragraph noting Phase 5 reads the same field path directly and that both readers MUST stay aligned on changes.
2. `_maybe_attach_vllm_chat_reasoning` docstring referenced `project_reasoning_replay_capability_gate.md` which lives in personal memory store, not the repo. Replace the dead-link reference with an inline summary of the asymmetry rationale (Paths 1+2 keep the dual-gate because loud server-side failures; Path C drops the static gate because vLLM's failure mode is template-drop silent).
Two pre-existing defects in the Phase 6 pool dispatch path that only
manifest when a pooled session is reused for a second dispatch:
1. The per-dispatch _AuthCapture allocated in _dispatch_pool was wired
into the httpx response hook only at first connect (via
_connect_one_pool). On a reused session no fresh connect runs, so
the hook continues writing to the original-connect's carrier while
the new dispatch inspects an empty carrier — auth_401/403 silently
misclassified to "other", refresh-and-retry never fires.
2. Even with the carrier on the entry (so the hook writes to a stable
reachable object), session.call_tool itself hangs forever on
upstream 4xx for reused sessions. Trace: SDK's spawned
handle_request_async raises HTTPStatusError, the outer
streamablehttp_client TaskGroup cancels post_writer, post_writer's
finally aclose's read_stream_writer, BaseSession's _receive_loop
exits and enters its CONNECTION_CLOSED-fanout finally. anyio's
send_nowait skips waiting receivers with pending_cancellation; the
dispatch task (created by run_coroutine_threadsafe for the reuse
case) is NOT in any cancel-scope chain, so the send "delivers" but
the receiver's Event is set on stale state — receive() never
wakes. Test 21 doesn't hit this because its 401 happens during
initialize, in the same task that opens streamablehttp_client, so
the cancel scope DOES propagate.
Fix:
- Move _AuthCapture ownership to PoolEntryState (and asyncio.Event
alongside, allocated lazily on the mcp-loop). The hook closes over
entry.auth_capture at first connect and stays valid across
dispatches; reset under open_lock before each call_tool.
- Race session.call_tool against the carrier's fired_event in
_dispatch_pool_with_entry. If the event wins (hook captured 4xx
before SDK propagated), cancel call_tool and raise an internal
_CarrierAuthSignal — _classify_failure resolves to auth_401/403
via the carrier's status, the dispatcher evicts the broken
session, and the cross-task retry handshake reconnects on a fresh
bearer.
Adds tests/test_mcp_pool_auth_integration.py::test_integration_pool_reuse_401_refresh_and_retry_succeeds
which drives the reuse path through real upstream + real SDK and is
the structural gate against this class regressing. Negative-tested
twice: revert PoolEntryState.auth_capture → test fails (carrier
empty); revert the race → test times out (SDK hang).
Also drops the @pytest.mark.asyncio decorator (replaced with
@pytest.mark.anyio) on four tests in test_mcp_pool_auth_introspection.py.
The project depends on anyio's pytest plugin (anyio is in deps);
pytest-asyncio is NOT a project dep and CI's test (3.13) failed on
those four. Local pytest happened to pick it up via system Python.
Found via Copilot review on PR #481.
Phase 3 of docs/design/oauth-mcp.md. Adds the Fernet/MultiFernet wrapper,
[security] config loader with rotation support, MCPTokenStore CRUD facade,
typed MCPTokenDecryptError that maps to the RFC's mcp_token_undecryptable_
key_unknown class, and a startup gate that fails loud when auth_type=
'oauth_user' rows exist without a configured encryption key.
Crypto module (turnstone/core/mcp_crypto.py):
- MCPTokenCipher wraps cryptography.fernet.Fernet + MultiFernet for
rotation; encrypt with first key, decrypt by trying each in order
- load_mcp_token_cipher_config reads [security] mcp_token_encryption_keys
(plural list) or mcp_token_encryption_key (singular), validates each
key is base64-decodable to exactly 32 bytes
- MCPTokenCipherConfig is repr=False with custom __repr__ that redacts
raw key bytes (defense in depth against accidental log/traceback leak)
- _key_fingerprint produces an 8-hex-char SHA-256 prefix for audit
attribution without exposing the key
- MCPTokenStore handles encrypt-on-write / decrypt-on-read for
mcp_user_tokens and mcp_servers.oauth_client_secret_ct
- get_user_token MUST NOT auto-delete the row on MCPTokenDecryptError
(test_get_user_token_with_wrong_key_raises_decrypt_error verifies
the row stays intact across a key-mismatch read)
- initialize_mcp_crypto_state / close_mcp_crypto_state lifespan helpers
shared between server and console
Storage protocol (5 new ciphertext-only methods):
- set_mcp_oauth_client_secret_ct (dedicated writer; deliberately NOT
added to MCP_SERVER_MUTABLE so generic update_mcp_server cannot write
the secret column)
- create_mcp_user_token, get_mcp_user_token,
update_mcp_user_token_after_refresh, delete_mcp_user_token
Server + console lifespans (turnstone/server.py + console/server.py):
- after OIDC init, count auth_type='oauth_user' rows; if any exist and
no encryption key is configured, log an actionable error and
raise SystemExit(1)
- without oauth_user rows, missing key is fine (lazy validation; admin
flip without restart returns 503 from the admin handler)
- app.state.mcp_token_cipher / .mcp_token_store populated when key
configured; None otherwise
Admin handlers:
- _require_token_store_for_oauth_secret pre-mutation gate validates
token_store availability and oauth_client_secret type BEFORE
storage.create_mcp_server / update_mcp_server runs, so a 503 from a
missing key never leaves an orphan row or partial-update state
- _apply_oauth_client_secret encapsulates the encrypt + audit write
used after the storage mutation; rolled out across both create and
update handlers
- 503 message references both mcp_token_encryption_key (singular) and
mcp_token_encryption_keys (plural for rotation)
- non-string oauth_client_secret payloads (false / 0 / lists / dicts)
are rejected with 400 instead of being str()-coerced
- when auth_type transitions away from oauth_user, the encrypted
secret column is cleared in the same admin call (with audit), so
flipping back doesn't silently resurrect a stale credential
Audit events (mcp_server.oauth.* per audit.py taxonomy; RFC's
mcp.oauth.* renamed for consistency):
- mcp_server.oauth.client_secret_set fired from admin handlers with
cleared:bool and key_fingerprint
- mcp_server.oauth.token_decrypt_failure fired from MCPTokenStore
.get_user_token when no installed key can decrypt; carries
key_fingerprints_attempted
Tests: 35 new tests across test_mcp_crypto, test_mcp_token_store,
test_server_lifespan_mcp_crypto, plus 6 admin-API tests covering the
no-orphan-row, no-partial-update, secret-clear-on-transition, and
non-string-secret-rejection invariants. Suite at 5337 (Phase 3 added
~50 tests including the rebase-imported skill suite).
cryptography>=42 promoted from transitive (lacme[tls]) to direct dep
since the encryption layer is now core, not optional.
Phase 4 (OAuth flow) wires the actual callers; Phase 3 adds only the
crypto layer and is exercised entirely by tests.
* refactor(css): unify design system, eliminate data-design="v1" gating
Strip the [data-design="v1"] attribute that was wrapping every DS rule
since #389 and never came back out. Result: every styled element on
coord/ui pages had two CSS rules (default + v1-gated), reviewers
couldn't tell which one rendered, and the bundle shipped duplicates.
Changes:
* Strip [data-design="v1"] prefix from ~400 gated rules. Remove the
attribute from coordinator/index.html, ui/index.html, preview.html.
* Merge shared_static/design/* into pre-v1 sheets:
tokens + typography → base.css (:root, dark default)
appbar + panel + buttons + pills + field → ui-base.css
message primitives → chat.css
sidebar + approval-dock → console/static/coordinator/coordinator.css
(new file linked from coord only — admin no longer ships ~9KB of
coordinator-only chrome on first load).
* Delete preview.html + 5 preview-only orphan stylesheets (topbar /
stats / feed / fleet-grid / live-feed). Drop the empty
shared_static/design/ directory.
* Drop unused primitives the merge dragged in: .pill / .k-badge /
.chip / .field / .t-* utilities / .side-item / .shell. Drop unused
tokens (--accent-c / --accent-l / --row-h / --density / --gap /
--font-display alias). Find-replace var(--font-display) →
var(--font-ui) across 6 files (103 sites).
* Standardize on the DS font stack: Inter body, JetBrains Mono code.
Admin's body shifts from IBM Plex Mono → Inter via the alias rename.
* Re-tune legacy --bg/--bg-surface/--bg-highlight/--bg-elevated from
steel-blue to neutral charcoal so admin and v1 pages share one
palette. Drop the cyan radial-gradient overlay on body that added a
blue tint to the formerly-blue bg.
* Polish:
- .msg.tool / .ts-msg--tool / .ts-approval / inline-approval all use
--cyan instead of amber, removing the user/tool colour collision.
- .msg-action-btn reverts to icon-button styling (transparent,
28x24) after the merge gave it text-button chrome that dwarfed the
13-14px icon glyphs inside.
- Light-mode composer contrast: flip .ts-composer surface roles
(wrapper recessed, textarea elevated) so the textarea reads
against its container; bump .dashboard-composer textarea
border-bottom + options panel surface so they're visible on white.
- .pane-messages padding 20px -> 16px 12px and gap 14px -> 0 (the
flexbox gap was stacking with .ts-msg margin-bottom for ~18px
inter-card spacing); .ts-msg/.msg margin-bottom 8px -> 4px.
- Restore WCAG 2.5.5 36x36 touch target on .msg-action-btn.
- Rename ui's <div id="header"> to id="ui-header" so the legacy
#header chrome no longer outranks the .appbar primitive on the
per-node page (3 getElementById calls in app.js updated).
- Drop chat.css link from coord (coord renders pure DS classes; the
composer.js consumer of chat.css is on admin + ui only).
Verified: ruff + mypy clean (175 files); 4653 non-live tests pass;
zero data-design / --font-display / shared_static/design hits remain.
Net source change: -1885 lines (924 added, 2809 deleted across 28 files).
* build: include coordinator.css in wheel; drop dead design/ glob
The previous commit added turnstone/console/static/coordinator/coordinator.css
(coord-only chrome moved out of console/static/style.css) but didn't
update the [tool.hatch.build.targets.wheel] include list, so CI's
wheel-completeness check failed.
Also drop the now-stale 'turnstone/shared_static/design/**/*' glob —
that directory was deleted in the same v1-elimination commit.
Verified locally: replicating the CI step's source-vs-wheel diff
returns MISSING: none.
* fix(css): address Copilot review feedback on PR #431
* coordinator/index.html — comment now correctly points to the moved
.sidebar rules at console/static/coordinator/coordinator.css (was
console/static/style.css before the perf-1 split-out).
* ui/static/index.html — restore <h1 class="appbar-title">; the cascade
conflict that motivated the h1→div change is gone now that the
wrapper id was renamed away from #header (the legacy #header h1 rule
no longer matches). Page semantics + accessibility regain the
top-level heading.
* governance.js — drop the inline font-family:var(--font-ui) on the
config-key <code> elements; let them inherit the global mono default
from base.css. The inline style was an artifact of the
--font-display → --font-ui find-replace; the original Outfit was
already odd on a <code> tag.
* ui-base.css — typography-helpers comment said "Body text still
inherits var(--font-mono) at 13px" but base.css now sets var(--font-ui)
at 14px. Reword to match current defaults.
* feat(design-system): DS-A — tokens + typography scaffold
Adds turnstone/shared_static/design/{tokens.css,typography.css} as the
first phase of a multi-PR design refactor seeded by Claude Design.
- tokens.css: full palette + shape + rhythm, light default with
[data-theme="dark"] override. oklch() raw colours, color-mix kept out
of DS-A entirely (reserved for primitives in DS-B).
- typography.css: Inter + JetBrains Mono via Google Fonts; six-step
scale (10/11/12/13/14/20-24). Utility classes .t-kicker/.t-meta/
.t-btn/.t-row/.t-body/.t-stat/.t-h1.
Signature accent stays warm amber (oklch hue 70) rather than Claude
Design's teal — preserves turnstone's "Instrument Panel" identity.
All other tokens match the spec verbatim.
Additive: both files gate under [data-design="v1"] so existing views
(base.css, per-view stylesheets) are untouched. DS-B will opt views in
one at a time.
* feat(design-system): DS-B — chrome + primitives + preview page
Adds the reusable primitive kit that DS-C and DS-Cluster will build on:
primitives/
panel.css .panel, .panel-head (.tools pinned right), .ghost
buttons.css .btn (pill 999px), .primary, .deny, .approve (amber)
pills.css .pill (running/thinking/attn/idle/err), .k-badge
(glyph-prefixed per WCAG 1.4.1), .chip, .risk
stats.css .stat + .stat-row, .mini-bar, .spark
feed.css .feed-item (grid ts/body/acts + .evi callout)
chrome/
topbar.css 48px sticky, conic-gradient brand mark
sidebar.css 240px sticky, .shell layout, semantic swatches
preview.html renders every primitive in both themes with an
in-page theme toggle (tracks prefers-color-scheme)
Additive: every selector scopes under [data-design="v1"] so existing
views (base.css + per-view stylesheets) stay untouched.
Spec deviations from the Claude Design prototype:
- `color-mix(in srgb, …)` throughout; prototype had two `in oklab`
usages — srgb per the spec's hard rule
- `.btn.approve` is warn-tinted amber, not green
(approvals signal "needs attention"; amber resolves on approval)
- k-badge tint uses `color-mix` instead of oklch relative-colour syntax
for broader browser support
- `@keyframes pulse/spin` renamed to `ts-pulse/ts-spin` to avoid
clashing with keyframes in base.css on pages that load both
- `prefers-reduced-motion` disables pulse + spin animations
- Text-on-accent-soft + text-on-warn-tinted darkened via color-mix
with ink to pass WCAG AA at 12px (fixes the classic same-hue trap)
- `.risk.crit` uses `#fff` text (dark-mode --panel on bright err fails)
- `.feed-item .acts button:not(.btn)` — compact action styling now
skips .btn-classed buttons so they keep their pill shape
- Focus-visible rings on .btn, .ghost, .stat, .topnav, .side-item
* feat(design-system): DS-C — patterns (approval-dock, fleet-grid, live-feed)
Completes the design library with three patterns that compose primitives
into the signature product surfaces described in the Claude Design handoff.
patterns/
approval-dock.css bottom-pinned approval strip. 1.5px-border,
--r-md squared action cluster: amber Approve
(primary), dashed Always, red Deny. kbd hints
and focus-visible rings on all three acts.
Call row (.dcall) framed as an inline code-
panel to emphasize "this is the exact call."
fleet-grid.css 14-col grid of .node squares. State modifiers
(.s-ok/.s-thinking/.s-attn/.s-err/.s-idle/
.s-unreach) + --pct load fill. Hover uses
outline, not box-shadow (neighbour bleed is
the intended density cue). .fleet-legend
swatch row below.
live-feed.css thin scroll-container wrapper over the
.feed-item primitive with a sticky top fade.
preview.html imports the three patterns, extends the
fleet demo to use the real .fleet class +
legend, adds a live-feed panel, renders the
approval dock fixed at the bottom with
aria-live="polite".
Spec notes:
- Approve button is amber (warn-tinted), never green
- Dock action buttons are 1.5px-bordered 6px-radius squares — NOT
pills — signaling "primary-action surface"
- All three dock actions clear WCAG AA in both themes via the same
color-mix-with-ink darkening pattern used in .btn.approve
- kbd hint color matches primitives/buttons.css (--ink-3, not --ink-4)
View-level rewrites (coordinator.html + coordinator.js opt-in,
admin/cluster dashboard rebuild) are follow-up PRs — they need a
running server to test SSE streams + the approval POST contract.
* fix(design-system): scope DS-A tokens to [data-design="v1"]
Co-authored-by: eous <13773563+eous@users.noreply.github.com>
* fix(design-system): scope DS-A font vars to [data-design="v1"]
Co-authored-by: eous <13773563+eous@users.noreply.github.com>
* fix(design-system): align dark-mode selector with theme.js convention
Co-authored-by: eous <13773563+eous@users.noreply.github.com>
* fix(packaging): add shared_static/design/** to wheel includes
Agent-Logs-Url: https://github.com/turnstonelabs/turnstone/sessions/74d0939a-c55f-46b0-92f4-14d0cbfb7084
Co-authored-by: eous <13773563+eous@users.noreply.github.com>
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: eous <13773563+eous@users.noreply.github.com>
* feat(coordinator): coordinator workstream kind — phase 1
Adds a new ``kind="coordinator"`` workstream that runs inside the
``turnstone-console`` process (first ChatSession hosted on the console)
with a dedicated tool set for spawning and driving child workstreams.
Supersedes the external ``turnstone-coordinator`` MCP side-car for new
installs; the extension is marked deprecated in
``examples/mcp-cluster-ops/README.md`` but still works on 1.4-and-earlier
clusters.
Phase 1 ships: the workstream class, 6 lifecycle tools, console hosting,
9 HTTP endpoints, per-user audit attribution, and a one-pane web UI at
``/coordinator/{ws_id}``. Node/skill discovery tools, task-list tool,
tree-view UI, and routing-proxy audit middleware follow in a later PR.
## Schema
Migration 039 adds ``kind`` / ``parent_ws_id`` columns + indexes to
``workstreams``. Both SQLite and PostgreSQL backends take the new
kwargs on ``register_workstream``; empty-string ``parent_ws_id``
normalises to ``NULL`` at the storage edge. PostgreSQL uses
``INSERT ... ON CONFLICT DO NOTHING`` to match SQLite's ``OR IGNORE``
and close a pre-existing SELECT-then-INSERT TOCTOU window.
``list_workstreams`` gains optional ``parent_ws_id`` / ``kind`` filters;
new ``get_workstream(ws_id)`` returns the full row (the existing
``get_workstream_metadata`` stays untouched for back-compat).
## Core session + kind routing
- ``ChatSession.__init__`` accepts ``kind`` / ``parent_ws_id`` /
``coord_client``. On ``kind="coordinator"`` it swaps
``_tools = COORDINATOR_TOOLS`` and zeros sub-agent tool lists.
- ``Workstream`` dataclass extended with ``user_id`` / ``kind`` /
``parent_ws_id``. Both ``WorkstreamManager`` and the new
``CoordinatorManager`` use the same type — no parallel hierarchy.
- ``_SessionFactory`` Protocol + server / cli factory closures thread
the new kwargs. ``POST /v1/api/workstreams/new`` rejects
``kind != "interactive"`` with 400; ``POST
/v1/api/workstreams/{ws_id}/open`` refuses coordinator rows so a
server node can't accidentally rehydrate one.
## Coordinator tool set
Six tools (``spawn``, ``inspect``, ``send``, ``close``, ``delete``,
``list_workstreams``) with a ``coordinator: true`` metadata flag,
scoped to coordinator-kind sessions only. ``inspect`` and ``list`` are
auto-approved reads; the four mutators need approval. ``list`` returns
``{"children": [...], "truncated": bool}`` so the model can detect
post-filter under-fill and paginate.
## CoordinatorClient (in-process, sync)
Mutating ops HTTP-POST to the console's own ``/v1/api/route/*`` on the
local bind URL so every existing middleware (auth, rate-limit) runs.
Read ops hit ``storage.list_workstreams`` / ``get_workstream`` /
``load_messages`` directly — the routing proxy doesn't expose
list/inspect paths. URL paths are a validated constant table (avoids
an httpx ``base_url``-merge trap). A new
``/v1/api/route/workstreams/delete`` proxy handler joins the existing
route-proxy endpoints.
## Per-session coordinator JWT
``CoordinatorTokenManager`` mints short-lived JWTs with ``sub=<real
user>`` (attribution preserved), ``src="coordinator"``,
``aud="turnstone-console"``, ``coord_ws_id=<ws>`` custom claim.
``_proxy_auth_headers`` preserves ``src`` + ``coord_ws_id`` across the
upstream re-mint so server-side middleware sees coordinator-origin,
not ``console-proxy``. ``AuthResult.extra_claims`` carries
non-reserved claims through validate→remint; ``create_jwt``'s
reserved-claim set (now including ``nbf`` / ``jti``) is symmetric with
``validate_jwt``.
## Console hosts the ChatSession
- New ConfigStore settings: ``coordinator.model_alias`` (required),
``reasoning_effort``, ``max_active`` (default 5),
``session_jwt_ttl_seconds``.
- Console lifespan builds a ``ModelRegistry`` +
``CoordinatorManager``. Missing / unresolvable alias returns **503**
with remediation text — never 500.
- ``CoordinatorManager``: placeholder-slot reservation under lock,
rollback on factory failure, per-ws_id rehydration lock to serialise
concurrent lazy-opens, ``max_active`` enforced via ``close_idle``
eviction semantics.
- ``ConsoleCoordinatorUI`` is a thin ``SessionUI`` implementation — no
global broadcast, no per-node metrics, shared
``_APPROVAL_WAIT_TIMEOUT`` constant across approval + plan paths.
- No eager startup rehydration: persisted coordinator rows load lazily
on first ``GET /v1/api/coordinator/{ws_id}``.
## Console coordinator API
Nine endpoints under ``/v1/api/coordinator/*`` gated by ``approve``
scope + new **``admin.coordinator``** permission (added to
``_VALID_PERMISSIONS``; not in any builtin role — operators opt in
explicitly). Ownership failures return **404, not 403** and use
strict equality so empty-owner rows don't leak across tenants.
Correlation-id masking on every factory-raising path
(``coordinator_create`` + ``coordinator_detail`` lazy rehydrate) — no
stack traces to the client.
## Audit attribution
Three console-side events (``coordinator.create`` / ``.close`` /
``.cancel``) with the real creator's ``user_id`` plus
``detail={coord_ws_id, src="coordinator"}``. No schema migration
required. Per-tool-call audit across the routing proxy is deferred
(needs either a ``source`` column on ``audit_events`` or
``record_audit`` calls wired into the route-proxy handlers).
## Web UI (``/coordinator/{ws_id}``)
One-pane chat served by the console. Reuses ``shared_static``
(``base.css``, ``auth.js``, ``theme.js``, ``toast.js``, ``utils.js``,
``kb.js``) and the server UI's ``renderer.js`` pipeline (KaTeX, Mermaid,
highlight.js already bundled).
- SSE to ``/v1/api/coordinator/{ws_id}/events`` with exponential-
backoff reconnect; status line carries a leading glyph
(● / ○ / ⚠) so state isn't conveyed by colour alone.
- Renders content, reasoning (dimmed italic
``.role-reasoning``), tool_result, approve_request, intent_verdict,
output_warning.
- Child ws_id references auto-wrap to
``/node/{node_id}/?ws_id={child}`` links — both ids regex-validated
before interpolation, everything else HTML-escaped.
- Non-modal approval bar (``role="region"``) with a batch header
("Approve N tool calls"), initial focus on the approve button,
buttons disabled during the in-flight POST, red-bordered deny.
``aria-live`` flips to ``off`` during streaming.
- "New coordinator" button on the dashboard header — permission-gated
on the UI side, matching the backend 403.
- Mobile composer capped under ``@media (max-width: 700px)``.
## Tests
~120 new tests across 8 files: workstream-kind storage + dataclass
semantics, CoordinatorClient URL map + token minting + storage reads +
truncation signalling, tool prepare/exec dispatch and approval gating,
CoordinatorManager create / rollback / eviction / lazy rehydration +
concurrency, HTTP endpoint auth + 404-on-ownership + 503-on-misconfig,
proxy-auth ``src`` preservation, full lifecycle end-to-end, coordinator
page HTML-injection guard. ``test_tools_schema.py`` widened to 25
tools (19 existing + 6 coordinator).
Verification: ``ruff check`` clean, ``mypy turnstone`` clean
(156 files), ``pytest`` 4054 passed (5 pre-existing failures unrelated
to this change — confirmed against ``main``).
* polish(coordinator): address PR review + CI + tool-namespace isolation
CI:
- `ruff format`: two files reformatted, matches the in-repo pre-commit config.
- `wheel-completeness`: add `turnstone/console/static/coordinator/*.html` +
`*.js` to the hatch wheel-include list. Without this the coordinator UI
was missing from published wheels.
- `test (3.11/3.12/3.13)` + `test-postgres`: three `TestExecReadImage`
tests were masking a real bug — my 6 new tool JSONs pushed tool count
19→25, crossing the default `tool_search.auto` threshold (20), which
made `ChatSession.__init__` construct a `ToolSearchManager` and cache
`_cached_capabilities` during init. Tests that later patched
`session._provider.get_capabilities` saw the cached value instead.
Root-cause fix: the tool-search threshold code path now reads
capabilities through `_resolve_capabilities(...)` directly — no cache
populate — so the patch takes.
Tool-namespace isolation (bigger fix than CI symptoms suggested):
- `TOOLS` was the union of all loaded tool JSONs including the 6 new
coordinator tools. Interactive sessions were getting coordinator
tools in their function-calling surface (which is nonsense — they
require a console-hosted `coord_client`), and coordinator sessions
counted against the interactive tool-search threshold. Fix:
- New `INTERACTIVE_TOOLS` / `INTERACTIVE_TOOL_NAMES` in
`turnstone/core/tools.py` exclude anything with `coordinator: true`
metadata. `TOOLS` stays as the union for schema introspection +
eval catalog.
- `ChatSession.__init__` selects tool set by kind: coordinator gets
fixed `COORDINATOR_TOOLS` (no MCP merge, no listeners registered);
interactive gets `INTERACTIVE_TOOLS` (+ MCP if configured).
Coordinators are meta-orchestrators that spawn child workstreams;
MCP tools / resources / prompts live on the children, not on the
coordinator's own surface.
- `_on_mcp_tools_changed` no-ops for coordinator sessions
(defence-in-depth in case listeners were registered).
- `always_on_names` on `ToolSearchManager` is now the set of builtin
tools actually present in the session (kind-aware) rather than the
full `BUILTIN_TOOL_NAMES` frozenset.
- `turnstone/eval.py` uses `INTERACTIVE_TOOLS` (coordinator tools
aren't in scope for the eval harness which tests interactive agent
behaviour).
- Regression tests in `tests/test_workstream_kind.py`:
- `INTERACTIVE_TOOLS ∩ COORDINATOR_TOOLS == ∅` and their union is
`TOOLS`.
- Interactive `ChatSession._tools` does not include any
coordinator tool name.
- Coordinator `ChatSession._tools` contains `spawn_workstream` but
not `bash` / `edit_file` / `memory`; sub-agent lists are empty.
- Coordinator `ChatSession` with an MCP client attached does NOT
merge MCP tools and does NOT register any MCP listeners.
PR review findings:
- **#10 / #11** (Copilot): coordinator UI claimed to reuse the server
renderer pipeline but loaded none of its JS. Mirrored
`turnstone/ui/static/renderer.js` into
`turnstone/console/static/coordinator/renderer.js` (flagged in-file
as a cleanup candidate to promote into `shared_static/`), added
`katex.min.js` / `highlight.min.js` / `renderer.js` script tags to
`coordinator/index.html`. `coordinator.js` now buffers raw markdown
via `textContent` during streaming, then swaps to `renderMarkdown` +
`postRenderMarkdown` on `stream_end`.
- **#7** (Copilot): N+1 query pattern in
`CoordinatorClient.list_children()` — per-row `storage.get_workstream`
just to read `skill_id`. Pushed `skill_id` + `skill_version` into
the `list_workstreams` SELECT projection on both backends; the
client reads them from `row._mapping` directly. New
`test_list_children_skill_filter_avoids_n_plus_one` pins the
behaviour (asserts `storage.get_workstream` call count is 0).
- **#8 / #9** (Copilot): `spawn_workstream` tool JSON said "if empty,
the workstream is created idle" but the prepare method rejected
empty and the field was marked required. Resolved by allowing
empty end-to-end: removed from `required`, prepare builds a
"spawn idle workstream" header + empty preview when empty,
updated `test_spawn_prepare_allows_empty_initial_message`.
- **#1–#5** (github-code-quality): five asserts with side-effecting
method calls in `test_coordinator_manager.py` (`mgr.close`,
`mgr.open`, `mgr.create` in a dead `_c = ...`). Extracted each
call to a local variable so `python -O` can't strip the side
effect.
Verification:
- `ruff check turnstone tests` clean.
- `mypy turnstone` clean (156 source files).
- `pytest -m "not live"` — 4063 passed, 3 deselected (live-backend
tests), 0 failed. The 3 image tests that were failing on this
branch now pass; wheel + lint both green locally.
* polish(coordinator): address Copilot re-review findings
Two findings from the re-review of #368 after the first polish commit.
**user_id wired into `mgr.create()` at the server handlers.** Phase 1
added ``user_id`` to the ``Workstream`` dataclass and
``WorkstreamManager.create()`` signature, but the two call sites in
``turnstone/server.py`` forgot to pass the authenticated caller
through. Result: interactive workstreams created via
``POST /v1/api/workstreams/new`` (including coordinator-spawned
children, which route through this handler) were landing with blank
``user_id``, defeating ownership-based access control on subsequent
sends / approvals / closes (``_require_ws_access`` treats blank
owners as legacy/allowed). Two changes:
- ``server.py:create_workstream`` forwards ``user_id=uid`` — the same
``uid`` already resolved from the auth result (with trusted-service
forwarding preserved).
- ``server.py:open_workstream`` prefers the persisted owner on the
workstream row over the rehydrating caller so reloading someone
else's workstream doesn't silently re-parent it. Falls back to
the authenticated caller when the stored row has no owner
recorded (pre-phase-1 rows).
Regression test in ``tests/test_workstream.py`` pins
``WorkstreamManager.create(user_id=X)`` → ``ws.user_id == X`` so the
manager seam can't regress silently on a future refactor.
**Malformed-JSON recovery allowlist expanded for coordinator args.**
``_prepare_tool()`` has a two-stage salvage path for models that
emit malformed JSON: a regex-extract (fallback 1) and a bare-string
→ primary_key wrap (fallback 2). The fallback-1 key list didn't
include coordinator argument names, so a slightly malformed
``spawn_workstream`` / ``send_to_workstream`` / etc. call would
hard-fail instead of salvaging into a minimal-args dict for retry.
Added ``ws_id`` / ``message`` / ``initial_message`` / ``parent_ws_id``
to the allowlist (kept alphabetised) so the coordinator tools get
the same model-self-correction behaviour as the interactive tools.
Fallback 2 already covers the ``ws_id``-primary-key tools via
``PRIMARY_KEY_MAP``; the regex path matters when the model emits
``{"ws_id": "abc", "message": "..."}`` with a trailing syntax error.
Verification: ``ruff check`` clean, ``mypy turnstone`` clean
(156 source files), ``pytest -m "not live"`` → 4065 passed, 3
deselected (live-backend), 0 failed.
* fix(coordinator): address ultrareview findings on coordinator workstream kind
Security
- Cross-tenant leak: CoordinatorClient.inspect/list_children now constrain
to the coordinator's own ws_id + direct children; an LLM coerced via
prompt injection can no longer exfiltrate other tenants' workstreams.
- Empty-owner short-circuit bypass: strict equality at coordinator.py
ownership gate and at the storage-fallback branch in coordinator_history;
orphan/system-owned coordinator rows can no longer be rehydrated by
arbitrary holders of admin.coordinator (DoS + history disclosure vector).
- Closed coordinators no longer silently resurrect on subsequent GET —
the Close button is now actually durable across URL revisits and tab
refreshes; rows with state in {closed, deleted} refuse rehydration.
Correctness
- ChatSession.close() now releases the CoordinatorClient httpx.Client
pool; previously every closed/evicted coordinator dropped a connection
pool on the floor until non-deterministic GC.
- open_workstream rehydration now forwards parent_ws_id + kind, so
coordinator-spawned children survive node restart / idle eviction
with their parent link intact instead of becoming silent orphans.
- list_children truncated flag now signals whenever the SQL fetch hit
the page cap (previously permanently False in the no-filter case,
causing confident-but-incomplete summaries from the coordinator).
- ConsoleCoordinatorUI.approve_tools: per-tool auto-approve now checks
auto_approve_tools independently of the blanket auto_approve flag,
so 'Always approve this tool' actually works on the next invocation.
Concurrency
- _spawn_worker no longer falls through to start a second concurrent
worker thread on the same ChatSession when queue.Full fires; instead
send() returns False and the endpoint surfaces HTTP 429.
- _open_locks entries are now refcounted under self._lock and only
popped when the last waiter releases — eliminates the race where a
rehydration-failure path lets two threads serialize on different lock
instances for the same ws_id and trip the "already tracked" guard.
Tests: +6 regression cases covering closed-coordinator refusal,
empty-owner non-admin refusal, queue.Full no-duplicate-worker,
inspect/list_children cross-tenant rejection, and truncated semantics.
- cli.py: ChannelAdapter import is annotation-only; move into
TYPE_CHECKING block and switch the two cast() calls to string-form
so the runtime import isn't required (TC001).
- slack/{config,routes}.py: ruff format fixes (whitespace + drop
redundant string-form annotation now that __future__ annotations
is in effect).
- pyproject.toml: drop the unused `tests.*` mypy override — `mypy
turnstone` (the only invocation in CI + local) never matches it,
so it was pure noise in the "unused section(s)" report. Other
optional-dep overrides stay; they're real safety nets when running
mypy without the [all] extras (e.g. on the test job).
- uv.lock: regenerate to match the slack-bolt + transitive deps the
pyproject changes resolve to (lock-check was failing on stale hash).
renovate[bot]