mirror of
https://github.com/turnstonelabs/turnstone.git
synced 2026-08-16 08:52:27 -06:00
e60c19befd5e31376bb606cd380c3564ac4e27df
10 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Patrick Buckley
|
110d44b07e |
refactor(tools): remove man, math, and plan_agent built-in tools
`man` and `math` duplicated capabilities already reachable through `bash`; `plan_agent` is better expressed as a `task_agent` running a planning skill, and carried a large amount of special-case machinery (plan-review gate, refinement loop, per-kind model routing). Removing all three shrinks the tool surface and cuts per-call token cost. Also removed, as dead-once-the-tools-are-gone: - the `math` sandbox executor (`turnstone.core.sandbox`) and its `[sandbox]` extra; the eval analyst now runs bash-only - the read-only `AGENT_TOOLS` sub-agent tool set and the `agent` tool-metadata key (`task_agent`/`TASK_AGENT_TOOLS` retained) - the plan-review protocol end to end: the `on_plan_review` UI hook, `resolve_plan`, `POST /v1/api/plan` + `POST /v1/api/route/plan`, the `plan_review`/`plan_resolved` SSE events, and their Python SDK / TypeScript SDK / OpenAPI / frontend / Discord+Slack bindings - the `model.plan_alias` / `model.plan_effort` settings and the registry `plan_model` / `plan_effort` routing fields TOOLS 31->28, TASK_AGENT_TOOLS 13->11; COORDINATOR_TOOLS unchanged. BREAKING CHANGE: removes the `man`, `math`, `plan_agent` tools, the plan-review SSE/HTTP/SDK surface, and the plan_* model-routing settings from the experimental 1.6 line. |
||
|
Patrick Buckley
|
3d12798315 |
feat(audio): voice I/O — speech-to-text + text-to-speech via model roles (#618)
* feat(audio): voice I/O — speech-to-text + text-to-speech via model roles
Browser voice input/output over the OpenAI audio wire protocol, selected
through the existing model-roles system so the same code path serves OpenAI,
vLLM/vLLM-Omni, or any compatible backend — pure registry config, no new
in-process deps. Anthropic has no audio API, so it is capability-gated out of
the audio roles while remaining valid as the agent model.
Backend
- core/audio.py: role resolution + capability gating + transcribe()/synthesize()
over a registry-resolved client. Typed AudioUnavailableError (503) /
AudioBackendError (502 — body masked, SDK detail logged). Optional STT prompt.
- Endpoints POST /v1/api/workstreams/{ws_id}/speech-to-text and POST /v1/api/tts,
registered in v1_routes, write-scoped (direct + proxied), offloaded with
asyncio.to_thread. Silence -> 422; configured-but-failed backend -> masked 502.
- Model roles: audio.stt_model_alias / audio.tts_model_alias / audio.tts_voice /
audio.stt_prompt settings; Models -> Roles entries (capability-gated dropdowns,
"(disabled — voice off)" when unset). /v1/api/models exposes resolved
stt_default_alias / tts_default_alias + per-model capabilities.
- Capabilities: supports_transcription / supports_speech_synthesis on
ModelCapabilities; current OpenAI audio lineup (whisper-1, gpt-4o[-mini]-
transcribe, tts-1[-hd], gpt-4o-mini-tts) registered as known models, with a
name-inference backstop for local/openai-compatible aliases.
Frontend (interactive UI)
- Mic dictation (record -> transcribe -> fill composer for review) and
per-message playback, shown only when the role is configured.
- CSS-mask icon set, aria-pressed + live-region announcements, recording timer,
reduced-motion cue, error-typed toasts + persistent denial, mic disabled while
busy, code/math stripped before TTS.
Tests: new test_audio.py plus STT/TTS endpoint, settings, openapi, available-
models, and OpenAI-lineup capability coverage. ruff + mypy + node --check clean.
* fix(audio): use const for AUDIO_MODEL_HINTS (var-sweep invariant)
|
||
|
Patrick Buckley
|
5bd7af6b73 |
feat(session): path-key /rewind + /retry into shared verb handlers (#549)
Lift the conversation-modifying /rewind and /retry verbs out of the body-keyed POST /v1/api/command into path-keyed POST /v1/api/workstreams/{ws_id}/rewind ({turns:N}) and /retry, as make_rewind_handler/make_retry_handler in SharedSessionVerbHandlers (template: make_close_handler/make_cancel_handler), wired on both interactive and coordinator kinds. Closes the last unlifted conversation-modifying surface — coordinator workstreams gain rewind/retry where they had none — and removes the surviving exception to the post-#422 path-keyed URL convention.
Handler shape: auth gate (coord -> admin.coordinator via permission_gate; interactive -> conversation.modify via accepted_permissions) -> busy-gate -> session.rewind(n)/retry() -> always emit clear_ui (incl. rewind-to-zero, carries #503) -> audit (conversation.rewind/retry on both kinds). Retry re-dispatch reuses the shared session_worker.send via a per-kind dispatch_retry closure (hard-reject on busy), not a third hand-rolled thread.
The web /command handler now rejects /rewind+/retry with a pointer to the path-keyed endpoint (BREAKING; 1.6.0aN-tolerant); session.handle_command's branches stay for the terminal CLI. auth.py adds the verbs to both write suffix-sets; Python + TS SDKs, OpenAPI (RewindRequest + server/console specs), the /route/ proxy mounts + audit actions, and coordinator_client all gain them.
Interactive frontend (app.js): the 3 /command POST sites + the hand-typed-slash reroute now hit the path-keyed endpoints; the bare .msg.user rewind selector is kept (matches the server's _find_turn_boundaries, which counts system-nudge user turns). The coordinator frontend rewind UX lands in a follow-up commit (browser-verified).
Tests: route-walk mount/order, /route/ audit rows, required_scope, OpenAPI catalog, SDK body-inspection, and HTTP-level handler behavior (busy-gate, turns validation, clear_ui emit, retry dispatch, audit invocation + swallow).
|
||
|
Patrick Buckley
|
1358121d52 |
chore(tests): refresh fixtures for path-keyed URL family
Mechanical updates across the test suite to swap legacy
/v1/api/{send,approve,cancel,events,workstreams/close} URLs for the
path-keyed equivalents under /v1/api/workstreams/{ws_id}/<verb>, and
to drop ws_id from request bodies (the path provides it now).
Per file:
- test_session_routes.py: deletes test_close_legacy_mounts_when_handler_provided
(the close_legacy slot is gone); test_send_mounts_post_and_delete_when_dequeue_provided
(added in PR commit 1) stays.
- test_openapi.py: expected-paths set swaps to path-keyed shape;
test_send_endpoint_has_request_body now asserts the OpenAPI for
/v1/api/workstreams/{ws_id}/send.
- test_auth.py / test_auth_identity.py: required_scope and
check_request fixtures swap to path-keyed shape; new tests cover
write/approve/read scope assignment for the path-keyed verbs +
the /node/* proxy mirror.
- test_sdk_server.py / test_sdk_console.py: mock-transport URL keys
swap; bodies drop ws_id.
- test_server_attachments_endpoints.py: ~17 send sites migrated to
/v1/api/workstreams/<ws>/send (a small Python script ran the bulk
rewrite — body ws_id stripped, URL rebuilt).
- test_server_authz.py: cross-tenant approve/close/cancel/events
tests retargeted to path-keyed URLs;
test_events_legacy_query_keyed_url_still_resolves_to_404_for_unknown_ws
renamed to test_events_path_keyed_url_resolves_to_404_for_unknown_ws
with the docstring updated to note the legacy adapter is gone.
- test_close_reason_persistence.py: 7 close sites all swap.
- test_console_routing_proxy.py: route-proxy tests swap to
/v1/api/route/workstreams/{ws_id}/<verb>; the upstream-URL
assertion now reads from .request (route_proxy uses
client.request(method, url, ...) for method passthrough); _wire_proxy
helper installs both .post and .request mocks for compatibility.
- test_route_proxy_audit.py: parametrized URLs migrated;
_make_proxy now also exposes a .request side-effect that delegates
to .post for the same compatibility surface.
- test_api_versioning.py: openapi.json path assertion swaps to the
path-keyed shape.
4557 passing under -m "not live"; ruff + mypy clean.
|
||
|
Patrick Buckley
|
d555816016 |
refactor(core): lift history + detail verb bodies across both kinds (Stage 2 verb lift)
Last verb-shape lift before v1.5.0 stable can tag. Adds two new
factories to ``turnstone/core/session_routes.py``:
- ``make_history_handler(cfg)`` — body lifted from coord's
``coordinator_history`` near-verbatim. ``?limit=`` query param
defaults to 100, clamps to [1, 500], malformed values fall back
to 100. Storage operations (``get_workstream`` on the
storage-fallback path, ``load_messages`` for the row read) now
run via ``asyncio.to_thread`` (was inline pre-lift on coord).
- ``make_detail_handler(cfg)`` — body lifted from coord's
``coordinator_detail``. Lazy-rehydrates a closed/evicted
workstream via ``mgr.open()`` on miss; mirrors
:func:`make_open_handler`'s exception envelope (``ValueError``
→ 503 with the session-factory's remediation text; bare
``Exception`` → correlation_id'd 500 with the per-kind noun
via ``cfg.audit_action_prefix``).
NO new ``SessionEndpointConfig`` fields — the factories reuse
``permission_gate``, ``manager_lookup``, ``not_found_label``,
``audit_action_prefix``, and (for history's storage-fallback
kind check) ``list_kind`` — all already wired by both production
lifespans for the list/saved factories.
Coord side: ``coordinator_history`` and ``coordinator_detail``
standalone handler bodies removed from ``console/server.py``;
``register_session_routes`` now wires
``history=make_history_handler(coord_endpoint_config)`` and
``detail=make_detail_handler(coord_endpoint_config)``.
Interactive side: GAINS both endpoints as a feature gain. Pre-lift
interactive had no ``GET /v1/api/workstreams/{ws_id}`` and no
``GET /v1/api/workstreams/{ws_id}/history`` — SDK consumers had to
subscribe to ``/events`` SSE just to read display fields or
message rows. The same lifted factories are wired with the
interactive endpoint config; cross-kind isolation is preserved on
both sides (history via ``cfg.list_kind`` storage-fallback gate
+ fail-loud-on-misconfig 500; detail via ``mgr.open()``'s internal
kind check).
Pydantic schemas: ``CoordinatorDetailResponse`` /
``CoordinatorHistoryResponse`` removed from ``console_schemas.py``;
``WorkstreamDetailResponse`` / ``WorkstreamHistoryResponse`` added
to ``server_schemas.py`` (mirrors the list lift's pattern for
``WorkstreamInfo``). Both server and console OpenAPI specs
reference the unified schemas; ``server_spec.py`` gains
``EndpointSpec`` entries for the new interactive endpoints. TS
SDK gains both interfaces in ``sdk/typescript/src/types.ts``;
``openapi-{server,console}.json`` regenerated.
Tests: 6 new coord regression/parity tests in
``test_coordinator_endpoints.py`` (limit clamping, cross-kind 404
on storage fallback, storage-only history, detail 503 on
session-factory misconfig, detail 500 with correlation_id on
unexpected rehydrate failure, history swallows
``load_messages`` exception → 200 with empty messages). 10 new
interactive parity tests in ``test_workstream_endpoints.py``
(``TestHistoryInteractive`` + ``TestDetailInteractive``). 1 new
openapi spec test pinning the server-side ``?limit=`` query param.
Total: ``4490 → 4491`` after the new exception-swallow
regression test landed. ``ruff check`` clean, ``mypy`` clean on
touched files.
/review pipeline (4 finders → verify → dedupe) caught 1 Minor
defense-in-depth (bug-1/sec-1, merged: ``make_history_handler``
fail-closed gate when ``cfg.list_kind is None``, mirroring
``make_saved_handler``'s same gate) + 1 Minor test-helper rename
(q-1: ``_interactive_history_cfg`` → ``_interactive_endpoint_cfg``)
+ 4 Nits (q-2 unused fixture parameter, q-3 CHANGELOG TS SDK
mention, q-4 missing exception-swallow regression test, q-5
misleading test comment) — all addressed in the same commit.
|
||
|
Patrick Buckley
|
9ed8b1e0b5 |
refactor(core): lift create verb body across both kinds (Stage 2 verb lift)
New ``make_create_handler(cfg, *, audit_emit=None)`` factory in ``turnstone/core/session_routes.py`` consumes five new ``SessionEndpointConfig`` fields (``create_supports_attachments``, ``create_supports_user_id_override``, ``create_validate_request``, ``create_build_kwargs``, ``create_post_install``) and replaces both ``create_workstream`` and ``coordinator_create`` bodies. Same factory + capability-flag pattern as the merged cancel / open / events lifts. ``_validate_and_save_uploaded_files`` lifted to ``turnstone.core.attachments`` so both processes call one kind-agnostic implementation. Coord parity gains (§ Post-P3 reckoning item #1 + carry-forward): - Create-time attachments: multipart parsing, validate+save+rollback, ``attachment_ids`` on the response. Coord adapter ``send`` doesn't yet reserve attachments at create time, so the rows save as pending and the next ``/send`` picks them up via the standard send-with-attachments path. - Disabled-skill rejection (matches interactive's pre-lift gate). - Always-include response shape ``{ws_id, name, resumed, message_count, attachment_ids}`` populated with default ``False``/``0``/``[]`` on the fields coord doesn't fill. - 200 status (was 201). - Audit-emit failures swallow + warning log instead of 500. Both kinds converge on the manager-at-capacity 429, factory-misconfig 503, and correlation_id'd 500 for unexpected ``mgr.create`` failure (interactive lifted up to coord's safer error envelope). Three /review fixes folded in: - ``notify_targets`` malformed input gates at the validator (400) instead of bubbling out of post_install as a 500 — pre-fix the workstream had already been created + audited + broadcast by the time the validation raised. - Skill-lookup storage failures now share the correlation_id'd 500 path with ``mgr.create`` (was masquerading as 400 "Skill not found"). - Whitespace-only ``skill`` field treated as empty (matches pre-lift coord). CHANGELOG entry under [Unreleased] documents every observable behaviour change. OpenAPI spec regenerated. Three new coord regression tests (create-time-attachments save pending rows, always-include parity fields, disabled-skill rejection) plus one interactive regression test (notify_targets 400). 4500 tests passing. |
||
|
Patrick Buckley
|
df7c0c2f44 |
refactor(server): delete legacy /v1/api/coordinator/ URL tree
Stage 2 Priority 0 Steps 0.4–0.7 — collapses the four migration
steps into one commit since they have to land together. The legacy
``/v1/api/coordinator/`` URL prefix never shipped in a stable release
(it appeared in 1.5.0aN experimental), so there's no compat carry-
forward — just rip and replace.
What moves:
- Step 0.4: deletes the eighteen ``Route("/api/coordinator/...")``
entries from ``console/server.py``. Coord traffic now flows
exclusively through the unified ``/v1/api/workstreams/`` shape
mounted via ``register_session_routes`` + ``register_coord_verbs``
(Steps 0.2 and 0.3).
- Step 0.5: rewrites the OpenAPI spec (``console_spec.py``) and
schemas (``console_schemas.py``, ``server_schemas.py``) to
document the new paths. ``test_openapi.py`` parity assertions
swap with them.
- Step 0.6: mechanical URL sweep across the frontend
(``console/static/app.js`` — 9 sites; ``coordinator/coordinator.js``
— 16 sites; ``index.html`` — 1 comment).
- Step 0.7: same sweep across the test suite
(``test_coordinator_endpoints.py``, ``test_coordinator_end_to_end.py``,
``test_coordinator_governance.py``, ``test_coordinator_close_all_children.py``,
``test_coordinator_client.py``, ``test_phase6_endpoints.py``).
Also touched:
- Server-side ``CoordinatorClient`` (``coordinator_client.py``) —
the coord agent's HTTP path for ``close_all_children`` updates
to the new shape.
- Handler docstrings in ``console/server.py`` say
``POST /v1/api/workstreams/...`` not ``/coordinator/...`` so a
``grep`` for a verb's URL lands on the right line.
- ``settings_registry.py`` setting descriptions, ``server.py``
cross-process error message, migration 042 docstring — all
updated to the unified shape.
The handler functions stay named ``coordinator_*`` until the
body-convergence follow-on lifts them into ``session_routes`` with
kind branching behind ``SessionRouteConfig`` flags. URL surface is
the only thing that changes here.
The ``test_session_routes`` route-walk now asserts the legacy paths
are GONE — previously it asserted both shapes coexisted. A future
accidental remount of ``/api/coordinator/`` would fail that test.
|
||
|
Patrick Buckley
|
553d73109b |
feat(coordinator): phase 5 — harness-test polish + wait_for_workstrea… (#378)
* feat(coordinator): phase 5 — harness-test polish + wait_for_workstream + judge fix
Closes the bug list surfaced by the 2026-04-17 coordinator harness test
plus the post-phase-4 wait_for_workstream ask, and folds in three
adjacent cleanups that landed in the same window. Tightens defense-in-
depth on the model-invoked mutating ops, fixes the LLM judge silent
no-op, kills the inspect-poll token burn, and rounds out a handful of
observability / docstring / spec gaps.
The session-factory pre-resolve at console/session_factory.py and
server.py was rewriting `judge.model` from an alias (e.g. `judge-mini`)
to the resolved underlying id (e.g. `gpt-5-mini`). IntentJudge then
checked `model_registry.has_alias(config.model)`, found nothing, and
fell back to the SESSION's provider/client with that bare model id —
silent `llm_fallback / "did not return a verdict"` whenever the
coordinator and judge alias resolved to different providers.
Pass the alias through unchanged; IntentJudge's existing alias-
resolution path picks up the matching client + provider. Validate
the alias exists so an obvious typo still surfaces, but don't replace
the model field.
Regression: `test_alias_uses_registry_provider_not_session_provider`
constructs an alias whose provider differs from the session's and
asserts the judge picks up the alias's provider/client/model;
`test_coordinator_tool_call_returns_llm_verdict_not_fallback` asserts
the verdict tier is `llm` (not `llm_fallback`) on the happy path.
New `cancel_workstream` tool (approval required, primary_key=ws_id) —
cancels in-flight generation, unblocks any pending approval / plan,
moves the child to idle, leaves the row in storage so a fresh
send_to_workstream lands cleanly. Re-uses the existing
`/v1/api/route/cancel` route + `route.cancel` audit namespace; no
new server endpoint.
`CoordinatorClient.cancel/close_workstream/delete/send` now enforce
a tenant guard inline (`_is_own_subtree`) — only the coordinator
itself or one of its own children is targetable. Foreign ids return
the same 404-shape inspect/wait_for_workstream use, so the model
can't distinguish foreign from missing (no existence oracle).
Defense-in-depth — the upstream node enforcement is the perimeter,
this is the second line.
`list_workstreams` advertised `state="deleted"` and an
`include_closed=true` that surfaced deleted rows. Hard-deletes
cascade the workstream + conversation rows out of storage, so
deleted is unreachable in normal operation. Doc-only fix; the
synthetic-test path that registers `state="deleted"` rows still
works (terminal-state filter still excludes them via
`_terminal_states = {"closed", "deleted"}` in list_children).
Documented that the 120s service-registry heartbeat window means a
node returned by list_nodes can drop out before a follow-up
`spawn_workstream(target_node=…)` lands — the spawn fails with "No
available node for routing" rather than falling back. Two-line
clarification on each tool. No code change (a code fallback is a
bigger discussion deferred to 1.6).
`close_workstream` accepts `reason`; the upstream server handler now
persists it to `workstream_config.close_reason` (capped at 512 BYTES,
sliced on UTF-8 not code points so a CJK / emoji-heavy payload can't
4× the documented budget). `CoordinatorClient.inspect()` reads it
and surfaces as `close_reason` in the result dict — only for
terminal-state children (closed/error/deleted) so the live-child hot
path doesn't pay a per-inspect DB round-trip.
Tests: server-side persistence covers success / no-reason /
length-cap / non-string / storage-failure / multi-byte-utf8 paths;
client-side surface covers terminal vs. live workstreams.
For idle children whose node-dashboard live counter is 0 (the live
block only surfaces in-flight token counters), fall back to
`SUM(prompt_tokens + completion_tokens)` from `usage_events` so the
inspect output reflects cumulative spend.
New `storage.sum_workstream_tokens(ws_id) -> int` on the protocol +
both backends. The fallback is folded INTO `_fetch_cluster_live` so
the merged live block (with persisted total applied) is what gets
cached — back-to-back inspects of an idle child amortize through
the existing 2s LRU cache instead of each firing a fresh aggregation.
`CoordinatorClient.list_skills()` now projects `allowed_tools` per
skill — capped at 20 with a `+N more` sentinel so a skill that
whitelists a wide MCP surface doesn't bloat the per-row payload.
Reads the existing `prompt_templates.allowed_tools` column; no
storage change. Coordinators no longer have to guess what tools a
skill brings.
`route_create` now sets `routing_strategy: "hash_ring" | "target_node"
| "resume"` on the spawn response so the coordinator's spawn
response (and the `spawn_workstream` tool output) carries why a
given node was chosen. 3 lines + 3 covering tests in
test_console_routing_proxy.py.
New coordinator tool `wait_for_workstream(ws_ids, timeout=60,
mode='any'|'all')` that absorbs the wait into a single tool call —
the model sees one call + one result regardless of how long the
children take. Kills the busy-poll inspect loop that burned 20+
turns on a 3-child fan-out.
Storage-poll loop with batched primitives —
`get_workstreams_batch` + `sum_workstream_tokens_batch` issue exactly
two storage calls per tick regardless of N. At the cap (32 ws_ids /
600s / 0.5s tick) that's ~2400 round-trips for a full wait, down
from ~38k under the naive per-id shape.
Validation single-source-of-truth: the client owns mode whitelist,
ws_ids dedup + cap, timeout coerce + clamp. The session preparer
is a thin pass-through that builds the header + dispatches; bad
input surfaces at exec time as a tool error via `result.get("error")`.
Tenant-isolation collapse: missing-row and cross-tenant cases both
return `state="denied"` so wait can't be used as an existence oracle
(matches the 404-mask contract `inspect` uses).
Prompt-side: tools_coordinator.md adds a `wait_for_workstream`
pattern + an explicit "PREFER wait_for_workstream OVER a loop of
inspect_workstream" line in the workflow-shape section.
Replaces the quote-bracketed substring LIKE/ILIKE pattern with proper
JSON-array containment. The previous shape effectively did
`LOWER(tags) LIKE '%"<lower-tag>"%'`, which broke for tag values
containing `"` (the JSON encoder escapes it to `\"` and the literal-
substring search misses), `\` (encoded as `\\`), or non-ASCII
characters that the encoder rendered as `\uXXXX`. Also exposed a
small spoofing surface — `tags=["foo\","bar"]` would have matched a
query for `bar`. Real-world tag values are alphanumeric+dash today
so it hadn't fired in production, but the fix is small.
- SQLite: `EXISTS (SELECT 1 FROM json_each(prompt_templates.tags)
WHERE lower(value) = lower(:tag))` (JSON1 extension; SQLite 3.38+).
- PostgreSQL: `EXISTS (SELECT 1 FROM jsonb_array_elements_text(
prompt_templates.tags::jsonb) AS jat(elem) WHERE lower(jat.elem) =
lower(:tag))`.
Three new tests prove the substring pattern was broken for
quoted / backslash / unicode tag values; the existing case-fold +
wildcard tests continue to pin the contract.
Phase 1 added the coordinator workstream API; phase 2 added only
`/open` to the OpenAPI catalog and missed every other coordinator
endpoint plus phase 3's `/children`, `/tasks`, and the
`/cluster/ws/{ws_id}/detail` aggregator. SDK consumers + operators
browsing `/docs` couldn't discover the surface. Doc-only addition:
12 endpoints + 9 new Pydantic models, all under the `Coordinator`
OpenAPI tag so /docs groups them together.
Sidebar re-fetches `GET /tasks` on every `task_list` `tool_result`
SSE event. A model that runs `add → list` (or any back-to-back
mutation pair) double-fetches the same envelope. Coalesced into
one fetch per 150ms window via a new `loadTasksDebounced` wrapper;
direct UI actions (refresh button, page load) keep calling
`loadTasks` directly so user clicks aren't delayed.
- `ruff check turnstone tests` — clean
- `mypy turnstone` — clean (157 source files)
- `pytest -m "not live"` — 4284 passed, 3 deselected (was 4226 on
main; +58 new tests across coordinator client, tools, judge,
storage, console routing proxy, server close-handler,
storage_skills_filtered, OpenAPI catalog, server close-reason
persistence)
- New tools added: 2 (cancel_workstream, wait_for_workstream) —
TOOLS count 28 → 30; coordinator subset 9 → 11; auto_approve adds
wait_for_workstream; primary_key adds cancel_workstream
- New OpenAPI endpoints: 12 (every phase-1/2/3 coordinator route +
the cluster-inspect aggregator)
- New storage protocol methods: 3 (sum_workstream_tokens,
sum_workstream_tokens_batch, get_workstreams_batch)
All phase 1 / 2 / 3 / 4 invariants preserved: COORDINATOR_TOOLS /
INTERACTIVE_TOOLS disjoint; coordinator sessions have no MCP surface;
list-style tools return {items, truncated}; route-proxy emits
route.<action> audit on 2xx; 404-mask on ownership failures; tenant
filters pushed into SQL; per-coordinator JWT carries scope context.
* fix(coordinator): address Copilot review on PR #378
Three valid Copilot findings on the wait_for_workstream surface:
1. ``wait_for_workstream.json`` description claimed the tool returns a
top-level mapping ``ws_id -> {state, tokens, updated}`` plus
elapsed/complete/mode at the same level, but the actual shape is
``{results: {ws_id: {...}}, elapsed, complete, mode}``. Description
now matches the implementation. Also adds ``deleted`` to the
advertised terminal-state list (it's in ``_WAIT_REAL_TERMINAL_STATES``;
the doc and runtime now agree).
2. ``CoordinatorClient.wait_for_workstream`` docstring listed
``idle / error / closed`` as the real terminal set but the constant
includes ``deleted``. Same fix — list ``deleted`` with a parenthetical
noting it's unreachable in normal operation (hard-delete cascades the
row).
3. Storage protocol docstring math: ``sum_workstream_tokens_batch``
claimed "from ~38k to ~1200" round-trips per wait at the cap, but
``wait_for_workstream`` issues TWO storage calls per tick
(``get_workstreams_batch`` + this one), so 1200 ticks × 2 = ~2400.
Updated to "~2400" with the math spelled out.
Also a clean rebase onto today's main (PR #377 — the rebalancer node_id
snapshot doc — landed since phase 5's last push). Single conflict in
``inspect_workstream.json`` resolved by keeping both notes (rebalancer
node_id binding semantics + the new ``close_reason`` surface from phase
5); ``spawn_workstream.json`` auto-merged.
The github-code-quality bot also flagged three items on
``_protocol.py`` asking to replace ``...`` with ``pass`` in Protocol
method bodies. Refuted: ``...`` is the canonical PEP 544 idiom for
Protocol method bodies and the rest of the file uses it consistently.
The bot's lint rule misfires for ``Protocol`` classes.
Verification:
- ``ruff check turnstone tests`` clean
- ``mypy turnstone`` clean (158 source files)
- ``pytest -m "not live"`` — 4308 passed, 3 deselected (no test count
change; pure doc/comment edits)
|
||
|
Patrick Buckley
|
fb190f8977 |
Normalize session_id into ws_id as sole persistent identity (#29)
* Normalize session_id into ws_id as sole persistent identity Eliminate the separate session_id concept. The workstream ID (ws_id) is now the single identity used for both real-time routing and conversation persistence, removing a layer of indirection that was 1:1 in practice and buggy on resume (stale pointers, orphaned rows). Schema changes (migration 006): - Drop sessions table; add alias/title columns to workstreams - Rename conversations.session_id → ws_id - Rename session_config table → workstream_config (ws_id column) - Data migration remaps existing conversations to ws_id Storage/API renames: - register_session → register_workstream (already existed, merged) - save_message/load_messages now keyed by ws_id - resolve_session → resolve_workstream - ChatSession.session_id property → ws_id - ChatSession.resume_session() → resume() - resume_session field → resume_ws - SessionResumedEvent → WorkstreamResumedEvent - /api/sessions → /api/workstreams/saved - /sessions slash command → /workstreams - --session-retention-days → --retention-days Channel eviction recovery simplified: reuses old ws_id directly instead of get_session_id_by_ws() reverse lookup. * Fix Copilot review feedback: stale session wording in docs, regenerate OpenAPI spec - docs/channels.md: "resumes the session" → "resumes the workstream", "Session resumed:" → "Resumed:", "old session was pruned" → "old workstream was pruned" - docs/api-reference.md: "Each session object" → "Each saved workstream object", field descriptions updated, removed stale node_id field - sdk/typescript/openapi-server.json: fully regenerated from Python models — removes all stale session_id properties from WorkstreamInfo, DashboardWorkstream, CreateWorkstreamResponse schemas |
||
|
Patrick Buckley
|
62a4ceac96 |
Dev/api versioning openapi (#18)
* Add API versioning under /v1/ prefix with OpenAPI 3.1 spec
All API endpoints move to /v1/api/* (clean break, no unversioned
aliases). Non-API routes (/, /health, /metrics, /static, /shared,
/node proxy) stay unversioned.
New turnstone/api/ package:
- Pydantic v2 models for all request/response schemas (server +
console) used for OpenAPI spec generation
- Programmatic OpenAPI 3.1 spec builder with EndpointSpec catalog
- /openapi.json serves machine-readable spec, /docs serves Swagger UI
Route changes:
- Both servers use Mount("/v1", routes=[...API routes...])
- Auth middleware strips /v1/ prefix before path classification
(PUBLIC_PATHS/WRITE_PATHS stay unversioned internally)
- Console proxy handles /node/{id}/v1/api/ upstream forwarding
- Bridge and CLI HTTP clients updated to /v1/api/ paths
- /openapi.json and /docs added to PUBLIC_PATHS and rate limiter
EXEMPT_PATHS
Security fix from review: required_role() now correctly handles
/node/{id}/v1/api/{path} proxy routes (previously the v1 segment
caused write-path detection to fail, allowing read-only token
escalation).
42 new tests (830 total). All frontend JS, docs, and diagrams updated.
* Fix mypy type errors in turnstone/api/ package
- Add generic type params to dict fields in console_schemas.py
- Add return type annotations to docs.py handler factories
- Move type-only imports (BaseModel, Callable, Awaitable) into
TYPE_CHECKING blocks to satisfy TC002/TC003 ruff rules
* Address PR #18 review feedback + fix mypy errors
Review fixes:
- Add pydantic>=2.0 as explicit dependency in pyproject.toml
(was only transitively available via openai/mcp)
- Auto-detect path parameters from {param} segments in OpenAPI
spec builder (fixes missing required path params)
- Use startswith() with concrete prefix for proxy version
detection instead of fragile substring check
- Make Swagger UI base URL configurable via swagger_ui_base_url
parameter for air-gapped deployments
Mypy fixes:
- Add generic type params to dict fields in console_schemas
- Add return type annotations to docs.py handler factories
- Move type-only imports into TYPE_CHECKING blocks
|