Compare commits

...

162 Commits

Author SHA1 Message Date
Patrick Buckley d8f2e43edb chore: bump version to 1.5.0a4 2026-04-23 18:46:15 -07:00
Patrick Buckley 4fe6e8678e fix(server): trusted-team workstream visibility on listing endpoints (#400)
* fix(server): trusted-team workstream visibility on listing endpoints

The per-user filter on /v1/api/workstreams, /v1/api/dashboard, and
/v1/api/workstreams/saved (PR #375's _visible_workstreams helper) was
written for a multi-tenant SaaS threat model that doesn't match how
turnstone gets deployed.  In a self-hosted, trusted-team install the
filter created friction without preventing the relevant threats — and
hid the auto-created name="default" startup workstream from every
web user, leaving fresh installs staring at a blank dashboard.

Listing endpoints now return the cluster-wide set to any authenticated
caller.  Per-workstream MUTATIONS (/send, /close, /open, /title,
/delete, /refresh-title) keep their independent ownership checks — the
cross-tenant guards from PR #375 stay in force on those handlers (see
TestCrossTenant{Delete,Approve,Close,Title,Open}).  Listing only
exposes metadata (name, state, kind, message_count); message history
still requires the per-workstream gate on /history.

Resuming a saved workstream still goes through /open's owner check, so
the metadata-leak surface ends at "you can see workstream X exists" —
not at any actionable cross-user capability.

The console collector's service-scope is now load-bearing only for the
SSE event stream gate (/v1/api/events/global); kept anyway as belt-
and-braces.

If turnstone is ever deployed as a true multi-tenant SaaS, the right
boundary is a real ``tenant_id`` column with row-level filtering at
the storage layer, not the empty-user_id heuristic this used to apply.

Tests updated to assert the new contract: listing returns all owners;
mutation gates unchanged.

* fix(server): repair test mocks + tighten docstrings on listing endpoints

- tests/test_auth.py: TestServerAuth + TestServerLogin mocks now set
  kind / parent_ws_id / user_id explicitly so /v1/api/workstreams JSON-
  serializes them.  Bare MagicMock attributes return another MagicMock
  that fails json.dumps and surfaces as 500.

- turnstone/server.py: list_saved_workstreams docstring corrected to
  describe what the endpoint actually returns (summary metadata, not
  history) and to spell out that ownerless persisted rows are claimable
  by any authenticated caller via /open — consistent with the trusted-
  team model the listing endpoints assume.  Same callout added next
  to the open_workstream ownership-gate block.  Comments throughout
  rewritten to be timeless (no "previously" / PR-number references).

- tests/test_server_authz.py: TestSaved... docstring matches the actual
  /open behavior for orphan rows (claimable by any authenticated
  caller, not a separate admin path).
2026-04-23 18:44:46 -07:00
Patrick Buckley 96abaf32b2 fix(chat): collapse phantom whitespace + tighten paragraph rhythm (#401)
* fix(chat): collapse phantom whitespace + tighten paragraph rhythm in markdown body

The assistant chat body was rendering 30-50px gaps between every
section.  Two compounding causes:

1. ``.ts-msg-body`` had ``white-space: pre-wrap`` on the markdown
   container.  The custom regex-based markdown converter
   (renderer.js) leaves ``\n`` text nodes between block siblings —
   pre-wrap rendered every one of those as visible vertical space,
   stacking ~14-16px between every heading/paragraph/katex-display.

2. No ``.ts-msg-body p`` margin override, so paragraphs fell back to
   browser-default 1em top + 1em bottom (~28px stacked between any
   two paragraphs).  Headings already had a tight ``8px 0 4px`` rule;
   paragraphs were the outlier.

Switched the body to ``white-space: normal`` and added a
``.ts-msg-body p { margin: 6px 0 }`` rule that matches the heading /
list / blockquote rhythm.  Mirrored the paragraph rule on the design-
v1 ``.msg-body`` selector so both legacy and v1 surfaces stay in sync.

``<pre>`` blocks have ``white-space: pre`` built in so fenced code
still preserves formatting.  Mid-stream partial fences (before the
closing ``\`\`\`` arrives) render as collapsed text for one frame and
then snap back when the next render tick wraps them in ``<pre>`` —
acceptable trade vs. the persistent gap regression.

User-typed messages render through ``.msg-user-text`` (a separate
DOM path), so this only affects assistant markdown output.

* fix(chat): preserve inline <code> whitespace under white-space: normal body

The body's ``white-space: normal`` (which collapses phantom inter-block
``\n`` text nodes from the markdown converter) inherits to inline
``<code>`` and silently collapses multiple spaces inside backtick
spans.  ``<pre>`` blocks rely on the user-agent ``pre { white-space:
pre }`` rule and are unaffected; only bare inline code needs an
explicit override.

Adds ``white-space: pre-wrap`` to ``.ts-msg-body code`` (chat.css) and
the design-v1 ``.msg-body code`` selector so backtick-wrapped code
spans render verbatim while still wrapping on long lines.

Addresses Copilot review feedback on PR #401.
2026-04-23 18:44:30 -07:00
Patrick Buckley 436ce5630b feat(coord): saved coordinators surface + shared session-card primitives (#399)
* feat(coord): saved coordinators surface + shared session-card primitives

The console home view now lists explicitly-closed coordinators in a
"Saved Coordinators" card grid below the active list.  Click a card →
POST /v1/api/coordinator/{ws_id}/open then navigate; capacity issues
surface as a toast instead of a broken detail page.  Card click is
de-duped by an `is-busy` class so rapid double-clicks don't fire
parallel resurrects.

GET /v1/api/coordinator/saved is the new backend endpoint (mirrors the
interactive list_saved_workstreams shape).  Filters at the SQL layer
to state='closed' via a new optional `state` parameter on
list_workstreams_with_history (added to the protocol + both backends);
also drops any rows currently loaded into coord_mgr as defence in
depth.  The blocking storage call + the lock-acquiring list_all are
offloaded via asyncio.to_thread to match coordinator_create's pattern.

CoordinatorManager._open_impl now allows resurrect of state='closed'
rows (deleted is still a tombstone).  The DB state-flip on resurrect
that the first cut had is gone — it raced concurrent close()s and the
next set_state() call syncs the DB naturally; the saved list filters
already keep a still-loaded coordinator from appearing as a saved
card even when its on-disk state lags.

Frontend dedup that paid for the saved surface ships in the same diff:

  - shared_static/cards.css: lifted from ui/static/style.css so both
    surfaces share the basic card primitive (delete-mode rules stay
    interactive-only until coordinator gets the same UX)
  - shared_static/cards.js: new renderSessionCard(sess, opts) helper
    used by both renderSavedWorkstreams (interactive) and
    renderSavedCoordinators (console)
  - shared_static/utils.js: formatRelativeTime moved here from
    ui/static/app.js

Coordinator landing visual fixes folded in:
  - .home-section-title now uses var(--accent) so the COORDINATORS
    heading reads as a peer of the NODES heading
  - .home-panel dropped its bg/border/padding so the composer is no
    longer double-framed (matching the dashboard-composer feel)
  - "Active coordinators" → "Saved Coordinators" rename + "Coordinators"
    on the active list

ws_closed SSE handler now gates on the closed ws's kind so interactive
closes don't spam /v1/api/coordinator/saved on busy clusters.
loadSavedCoordinators in-flight de-dup coalesces close-event bursts to
one fetch instead of N.

Tests cover: caller-scoping, admin sees-all, blank-uid fail-closed,
loaded-coordinator filtering, state filter (idle rows excluded), plus
the manager-level open-resurrect / open-refuses-deleted contracts.

Closes the bug-{1,2,3}, perf-{1,2,3,4}, sec-{1,2}, q-{1,2,3,4,5,6,7}
findings from the prior multi-stage review.

* fix(design): restore amber accent on the v1 design system

The Claude Design handoff swapped the accent hue to teal (h=182).
Walking back to amber (h=75) — turnstone's original brand colour.
Lightness + chroma bumped slightly (0.62→0.7, 0.10→0.13) so the
restored gold matches the visual weight of the legacy #e5a042 token.

Hue map header comment updated to record what happened so the next
person doesn't repeat the swap.  Only surfaces with data-design="v1"
on <html> pick this up — currently just turnstone-server's webui.

* chore: gitignore design_ideas/ and .claude/ dev directories

design_ideas/ holds personal Claude Design handoff scratch + reference
HTML; .claude/ holds per-user Claude Code state (worktrees, settings,
plugin caches).  Neither belongs in version control.

* fix(coord): address PR #399 review nits

- tests/test_coordinator_endpoints.py: split `assert mgr.close(ws.id)`
  in `_seed_closed_coord_with_history` so the close call always runs
  even under `python -O` (asserts stripped).  Same fix in
  test_coordinator_manager.py's `test_open_refuses_deleted_coordinator`
  for the open() and open_admin() calls.
- shared_static/cards.css: `.card-wsid` now reads `var(--font-mono, "IBM
  Plex Mono", monospace)` so design-v1 surfaces pick up the JetBrains
  Mono token while console (still pre-v1) keeps the literal fallback.
2026-04-23 17:48:19 -07:00
Patrick Buckley f510699a4f feat(auth): inline refresh response + sessionStorage rehydrate hardening (#398)
* feat(auth): inline refresh response + sessionStorage rehydrate hardening

The proactive refresh path now consumes the /refresh response body
inline (permissions + exp), eliminating the chained /whoami round-trip
and the brief stale-sessionStorage window after refresh succeeds but
before whoami completes.

Adds AbortController + _loggedOut guards to the whoami fetch so a
logout fired mid-flight cannot re-populate sessionStorage after it
clears.  A non-OK whoami on tab restore now explicitly clears
sessionStorage instead of silently leaving stale cosmetic permissions
(server-side identity gone → UI gating reflects it on next render).

Surfaces window.permissionsReady (one-shot promise) so permission-
gated UI can await the initial whoami's completion instead of guessing
a setTimeout duration.

Tests cover the new refresh response shape, the existing leeway path,
the storage-failure fallback, and the no-perms 403 path.

Closes the bug-3 / perf-4 / sec-1 / q-6 findings from the multi-stage
review of the prior uncommitted change set.

* fix(auth): guard whoami superseding race in _scheduleRefreshFromWhoami

_scheduleRefreshFromWhoami is invoked from several entry points
(initial page load, _onSuccess, BroadcastChannel "login"/"refresh",
_tryRefresh fallback).  Two firing in quick succession could let an
older slow whoami land after a newer one and clobber its effects —
clearing permissions right after a successful login, or rescheduling
the refresh timer off stale exp.

Now aborts any prior _whoamiAbort before starting a new request and
guards the .then's _storePermissions / _scheduleRefreshAt with a
`_whoamiAbort === ctrl` check so a late arrival from a superseded
call is fully neutralised.

Addresses Copilot review feedback on PR #398.
2026-04-23 17:43:48 -07:00
Patrick Buckley fa53b414ed feat(providers): add gpt-5.5 and gpt-5.5-pro capability entries (#396)
* feat(providers): add gpt-5.5 and gpt-5.5-pro capability entries

OpenAI announced gpt-5.5 on 2026-04-23 (ChatGPT/Codex first, API
"very soon"). Mirror the gpt-5.4 / 5.4-pro capability shape: 1M
context, native tool search, vision, xhigh effort; pro is
always-reasoning with no temperature and medium/high/xhigh only.

No provider-logic changes needed — OpenAI announced no API-surface
changes vs 5.4. Cache retention already covers 5.5 via the existing
startswith("gpt-5") prefix rule.

* test(providers): cover gpt-5.4-pro and gpt-5.5-pro in cache retention test

Pro variants share the same gpt-5 prefix and should keep 24h
retention; explicit coverage guards against regressions if the
prefix rule narrows in the future.
2026-04-23 15:20:59 -07:00
Patrick Buckley e4070c2f8c chore(ci): remove trivy docker security scan (#397)
Remove the weekly Trivy scan job and the .trivyignore exclusion file.
The scanner has been flagging base-image CVEs that require no action
on our part (upstream-only fixes) and has provided no actionable
signal, while breaking CI on an ongoing basis.
2026-04-23 15:12:55 -07:00
Patrick Buckley 42d22bb6b4 feat(auth): cookie refresh endpoint, JWT leeway, coord-token observability (#395)
* feat(auth): cookie refresh endpoint, JWT leeway, coord-token observability

Three robustness wins around the auth/JWT layer.

1. POST /v1/api/auth/refresh — handle_auth_refresh in core/auth.py,
   wired in both console/server.py and server.py.  Sliding-window
   re-mint of the auth cookie.  Re-resolves the user's permissions
   from storage so a role change propagates within one refresh cycle
   instead of persisting until the original cookie's natural expiry.
   Returns the same JSON shape as /api/auth/login plus a fresh
   Set-Cookie header.  Refuses to extend a session for a deleted /
   role-stripped user (403).

   Resolves the user-visible "401 after browser tab open >24h"
   symptom: previously the only refresh path was a full re-login,
   now a single POST extends the session.

2. validate_jwt now passes leeway=30 to PyJWT.  Absorbs minor
   clock skew between hosts (multi-replica console deployments) and
   between mint-time and validate-time within the same process.
   Standard tolerance for short-lived tokens.

3. CoordinatorTokenManager._mint logs at debug.  Mirrors the pattern
   in ServiceTokenManager._mint (auth.py).  Premature-401 diagnostics
   would have been an order of magnitude faster with this in place
   the first time around.

Frontend (shared_static/auth.js):

- _scheduleRefreshFromWhoami() reads the JWT exp surfaced via /whoami
  and sets a setTimeout at 90% of remaining cookie life to call
  /refresh.  Floor 30s, ceiling 24h.  Fires on initial page load
  (silent if not authenticated) and after every successful login.
- _tryRefresh() de-dupes concurrent callers via a shared in-flight
  promise — many parallel authFetch's hitting 401 at once still only
  fire one /refresh.
- authFetch on-401 now attempts a single reactive refresh-then-retry
  before falling through to the login overlay.  Covers cases where
  the proactive timer didn't fire (tab restored from disk-cache after
  expiry, system clock jump, page first-load with stale cookie).
- BroadcastChannel "refresh" message keeps sibling tabs in sync so
  they don't redundantly hit /refresh themselves.
- logout() cancels the proactive timer.

Tests:

- validate_jwt accepts 10s-expired tokens (within 30s leeway).
- validate_jwt rejects 60s-expired tokens (past leeway).
- /whoami includes exp claim with sane bounds.
- /refresh returns ok + Set-Cookie + the refreshed cookie keeps
  working on subsequent authenticated requests.
- /refresh without a cookie returns 401.

Not addressed: the coordinator.session_jwt_ttl_seconds ceiling
(currently 1h) — that's a separate, preventative concern for very-
quiet long-running coordinators, orthogonal to the user-visible 401
this PR fixes.  Can bump in a follow-up if it actually surfaces.

* fix(auth): address Copilot PR #395 feedback

Two real bugs caught by Copilot, both fixed.

1. Storage failure was indistinguishable from "user deleted" in
   handle_auth_refresh.  _load_user_permissions() swallows exceptions
   and returns set(), so a transient DB hiccup looked like
   "user has no permissions" and returned 403 — logging the user out.

   Now calls storage.get_user_permissions() directly with try/except.
   - Exception → log + fall through to in-token claims (refresh succeeds
     with stale-but-valid permissions; better than fail-closed mid-
     session for a hiccup).
   - Empty set returned (no exception) → 403 (legitimate signal: user
     deleted or role-stripped).

   Tests:
   - test_refresh_storage_failure_falls_back: storage raises → 200 +
     in-token permissions.
   - test_refresh_user_with_no_perms_403: storage returns empty → 403.

2. Logout race: a /refresh in flight when the user clicks Logout could
   land AFTER /logout's clear-cookie response and re-set the cookie
   from /refresh's Set-Cookie header, silently undoing the logout.

   Fix in shared_static/auth.js:
   - Add a _loggedOut latch + _refreshAbort AbortController.
   - logout() sets _loggedOut = true synchronously and aborts any
     in-flight /refresh BEFORE the /logout fetch fires.
   - _tryRefresh() bails on its post-fetch effects (don't store perms,
     don't reschedule, don't broadcast) when _loggedOut is set.  The
     stale Set-Cookie from /refresh is harmless because /logout's
     response overwrites it on the way back.
   - _onSuccess() (re-login) clears the latch so subsequent refreshes
     work again.

   Race window is small but real on slow networks / contested CPU.
2026-04-22 20:36:00 -07:00
Patrick Buckley eedf700d3b chore: bump version to 1.5.0a3 2026-04-20 20:13:23 -07:00
Patrick Buckley 4d667a2cdc feat(design-system): DS phase 2 — opt server chat UI into v1 primitives (#392)
* feat(design-system): DS phase 2 — opt server chat UI into v1 primitives

ui/static/index.html:
  - data-design="v1" on <html> opts this view into design system tokens
    and primitives scoped under the attribute selector.
  - Link DS stylesheets after the legacy cascade: tokens + typography +
    appbar (chrome) + panel / buttons / pills / message / field
    (primitives). Legacy /shared/base.css, /shared/ui-base.css,
    /shared/chat.css, and /static/style.css stay linked to handle
    anything not yet migrated (rich markdown, tabs, dashboard, split
    panes, approvals, modals).
  - Header <div id="header"> picks up .appbar + .appbar-title +
    .appbar-status + .appbar-spacer + .appbar-actions alongside the
    legacy .ts-header classes. Theme-toggle gets .btn for DS pill shape
    while keeping .header-btn for palette continuity.

ui/static/app.js:
  - Chat message elements emit both legacy and DS class names so the
    DS primitive picks up the message surface while legacy .ts-msg--*
    rules keep view-specific markdown styling (tables, callouts, katex,
    mermaid, hljs). Pairs:
      ts-msg ts-msg--user       → + msg user
      ts-msg ts-msg--assistant  → + msg assistant
      ts-msg ts-msg--reasoning  → + msg reasoning
      ts-msg ts-msg--info       → + msg info
      ts-msg ts-msg--error      → + msg error
      ts-msg-body               → + msg-body
  - Approval blocks keep legacy-only styling — their shape is distinct
    from the DS .msg primitive (the DS approval-dock pattern is a
    fixed bottom dock, not inline-in-chat).

No backend or wire-format changes. SSE events, POST bodies, endpoint
URLs, ARIA attributes, and keyboard shortcuts all unchanged.

* feat(ui/static): DS-skin tool-call + approval + verdict internals

The outer .ts-msg.ts-approval--inline picked up DS .msg styling via
PR #2's dual-class approach, but the inner structure kept rendering
with legacy yellow/green/red colours and legacy chip shapes. Result:
a DS-accent-bordered card containing a mustard tool-name, a clunky
uppercase-yellow verdict chip, and a mismatched auto-approved pill.

Add [data-design="v1"]-scoped overrides that reskin the inner
vocabulary onto DS tokens:

  .ts-approval-tool          panel-over-panel-2 card with hair border
  .tool-name                 accent (teal) for tool-kind identity
  .tool-cmd / .tool-diff     ink-2 text; diff-del/add/warn → err/ok/warn
  .verdict-badge.verdict-*   chip aesthetic matching DS k-badge —
                             low=ok-tinted, medium=warn-tinted,
                             high/critical=err-tinted, with a
                             3px left-border semantic stripe
  .verdict-detail            panel-2 callout with structured rows
  .verdict-judge-spinner     ts-pulse animation (reuses primitive)
  .ts-approval-badge--*      pill shape hugging max-content, matches
                             DS approve-button-family colour palette
                             (ok-text, err-text-mix)
  .tool-output               panel bg, hair border, accent stream
                             left-border, fade-gradient on collapse
  .ts-verdict-glow--*        soft ring on the corresponding action
                             button (approve=ok, deny=err, review=warn)

No JS changes; DOM shape unchanged. CSS-only reskin so approval flow,
tool streaming, and verdict expand/collapse behaviour all stay intact.

* fix(ui/static): consistent tool-card width + flat badge aesthetic

Two fixes to the DS-skinned tool-call rendering:

1. Tool-call cards were sizing to their content (short output →
   narrow card, long output → full-width), producing a jagged column.
   Force .ts-msg.ts-approval--inline to width: 100%; align-self:
   stretch; box-sizing: border-box; so the chat column reads evenly.

2. The "approved" / "auto-approved" pill was styled as a button
   (pilled shape, 1px bg-tinted border, 4x10 padding) which read
   as clickable.  Switched to a flat badge aesthetic matching the
   .risk primitive: 3px-squared, 2x6 padding, 10px mono uppercase
   on a --ok-soft / --err-soft tinted surface, no border.  Reads
   as a status tag, not a call-to-action.

* fix(ui/static): address Copilot PR #392 feedback

Copilot findings, all applied:

- Drop the legacy Outfit + IBM Plex Mono Google Fonts link.  DS
  typography.css @imports Inter + JetBrains Mono; loading both stacks
  on opted-in pages wastes downloads and triggers FOIT/FOUT differences.

- Drop the .ts-header-title class on the <h1>.  Its legacy rule forces
  font-family: var(--font-display) (Outfit) which overrides the DS
  appbar typography.  .appbar-title alone is sufficient under v1.

- Override .ts-msg font-family under [data-design="v1"] when .msg is
  also present (and not the .tool variant).  Legacy .ts-msg forces
  mono; DS user/assistant/reasoning/info/error messages should use
  the UI font.  .msg.tool keeps mono via the primitive's own rule.

- Replace the inline name.style.color = "var(--red)" in buildToolDiv
  with a .tool-name--error class.  Inline styles win over CSS rules
  and broke the DS token mapping (legacy --red is not the DS --err).

- Correct the header comment in style.css for the approval-block
  overrides.  Prior comment claimed the outer wrapper picks up DS .msg
  styling; it doesn't — the dual-class approach wasn't extended to
  approval blocks.  Updated comment to match actual DOM.
2026-04-20 20:10:26 -07:00
Patrick Buckley 581a8c41b1 feat(design-system): DS phase 3 — coordinator chat migration (#393)
* feat(design-system): DS phase 3 — coordinator chat migration

Opt the per-session coordinator view into data-design="v1" and migrate
its rendering to the DS primitives + patterns shipped in phase 1.  This
is the larger of the two parallel chat migrations (the other being the
server UI under turnstone/ui/static/).

Scope — this PR touches two files only:

  turnstone/console/static/coordinator/index.html
    - data-design="v1" on <html>; DS stylesheets linked after the legacy
      base so primitives win on specificity and legacy styles keep
      covering anything not-yet-migrated.
    - Header rewired from .ts-header to .appbar with .appbar-back,
      .appbar-title + .dim subtitle, .appbar-spacer, .appbar-status for
      SSE state, and .appbar-actions wrapping the cancel / end / theme
      buttons (now .btn pills).
    - Approval bar replaced with the .approval-dock pattern.  Signature
      change: amber Approve becomes an ok-family (green) filled button
      with 1.5px border + --r-md squared shape.  .dcall rows frame each
      pending call like a mini inspectable code line.  Action cluster
      sits in a .drow with Deny (.act.danger) / Always (.act.always) /
      Approve (.act.primary) and the preview's kbd affordances
      (D / ⇧A / ⏎).  role="region" + aria-live="assertive" preserved;
      the dock stays non-modal (no focus trap), focus moves to the
      primary Approve button on open via the existing handler.
    - Sidebar shell adopts .sidebar + .side-section + .side-label +
      .ghost refresh buttons.  Coordinator-only .sidebar overrides unset
      the DS sticky-left-column defaults (which assume an admin-shell
      grid) so the aside continues to flex into the right column of
      #coord-body.  Tree-row + task-row styling stays view-local,
      rehomed to DS tokens (--hair-2 hover, --accent focus, --ok/--warn/
      --err + -soft task-status tints).
    - Inline <style> trimmed of rules now covered by DS primitives;
      only the coordinator-specific flex wiring, tree-row visuals, and
      <700px responsive accordion remain.

  turnstone/console/static/coordinator/coordinator.js
    - appendMsg() emits .msg + role variant (.msg.user / .msg.assistant /
      .msg.reasoning / .msg.tool / .msg.error / .msg.info) and .msg-body.
      _TS_ROLE_VARIANTS renamed _MSG_VARIANTS.
    - Streaming helpers query .msg-body; SSE dedup-by-call-id query
      updated to .msg[data-call-id=...].
    - showApproval() renders the .approval-dock DOM shape: .dhead count
      in a .dcount, one .dcall per pending call with .risk index pill +
      .dfn function name + .dargs preview.  approvalBar.hidden toggles
      visibility (the DS pattern is position: fixed and always-rendered;
      [hidden] is the show/hide hook).
    - setSseStatus() keeps .appbar-status as the base; semantic colour
      tracks OK / ERR via inline --ok / --err.  Leading glyph (●/○/⚠)
      preserves the WCAG 1.4.1 non-colour-only cue.
    - Wait indicator uses .appbar-status instead of the legacy
      .ts-header-status BEM; styling from the inline page rules colours
      it --think.

Contracts preserved:
  - SSE wire format and event names unchanged (approve_request,
    child_ws_created, wait_progress, batch_started, state_change,
    stream_end, ...).
  - POST /approve body shape unchanged: {approved, always, call_id}.
    No per-item feedback field is added (that's phase 9 PR C).
  - Keyboard behaviour unchanged: Enter continues to approve via the
    primary-button focus shift in showApproval(); the D and ⇧A kbd
    labels are rendered per the pattern spec but the global key
    handlers (if any) remain untouched.
  - ARIA attributes (role, aria-label, aria-live) preserved on the
    approval dock, messages log, and sidebar.
  - All shared-static JS imports and order unchanged; composer module
    continues to own its own DOM inside #coord-composer-mount.

No backend changes.  Legacy CSS (/shared/base.css, /shared/ui-base.css,
/shared/chat.css, /static/style.css) stays linked as the compatibility
layer — DS selectors [data-design="v1"] beat legacy where applied.

* fix(coordinator): inline approval dock above composer, not viewport-pinned

The .approval-dock DS pattern defaults to position: fixed; bottom: 22px
— designed for the fleet dashboard where the dock overlays content. In
the coordinator chat that rule pinned the dock to the viewport bottom,
covering the composer input area.

Move the dock DOM back inside #coord-main between #coord-messages and
the composer mount so it flex-stacks naturally above the input. Add a
view-local override that neutralises the fixed positioning (position:
static, z-index/box-shadow auto) while preserving the visual pattern
(warm top stripe, head/call/actions rows, dashed Always button).

Drop the 160px bottom-padding hack on #coord-messages since the dock
is now in-flow and naturally pushes the message log up.

Also likely resolves the Firefox initial-render issue — position:fixed
+ [hidden] toggle had cross-browser quirks where the dock wouldn't
appear on first SSE approval event until a separate DOM mutation
forced a reflow. In-flow layout makes it boring and predictable.

* fix(coordinator): integrate judge verdicts into approval dock, not chat

The judge's intent_verdict is evaluation context for the pending
approval, not a chat message. Previously each verdict appended a
"[judge] deny (risk=low)" tool message into the transcript even when
the corresponding approval was visible in the dock — two separate
surfaces showing related decision context, neither one complete.

Now:
- Each .dcall row gets data-call-id from the approve_request item
- intent_verdict looks up the matching row and renders a .dctx sibling
  below it with "judge: <recommendation> (risk: <level>)" + optional
  "confidence: <score>" chips. Reasoning attaches as title tooltip.
- Verdicts cache in a Map<call_id, verdict> so late-arriving
  approve_request events can still apply verdicts that came early
- Fallback to the old chat-message surface only when the approval isn't
  visible (call_id missing, or resolved before we could render) so the
  verdict isn't silently dropped

* feat(coordinator): judge verdict polish — colour-coded chips, spinner, reasoning

Three refinements to the approval-dock judge integration:

1. Colour-code verdict chips by recommendation — approve=green (--ok),
   review=amber (--warn), deny=red (--err). Reviewers can triage at a
   glance without reading the chip text; complements the text label
   for WCAG 1.4.1 (non-colour-only signaling).

2. Spinner while evaluating — when showApproval builds a .dcall row
   without a cached verdict, render a "judge evaluating…" chip with
   a spinner. Replaced in-place when intent_verdict arrives. Reuses
   the ts-spin keyframe from primitives/feed.css.

3. Justification inline — judge.reasoning is delivered in every
   intent_verdict event but was hidden behind a title tooltip. Now
   renders as a wrapped prose block (.drationale) below the .dctx
   chips, styled like the .msg-body .evi callout (left-rule + mono
   + --ink-3). Full text, no truncation — justification is the whole
   point.

View-local styling; the approval-dock pattern itself is unchanged.
If these patterns turn out to be broadly useful, they can promote to
shared_static/design/patterns/approval-dock.css in a later PR.

* fix(coordinator): defer approve-button focus until judge verdict arrives

The Approve button was getting focus the instant the approval dock
opened, which lit up the green focus ring and made the filled-green
button look pre-confirmed.  A reviewer could mistake that for "already
approved" before the judge has even returned a verdict.

Now focus is deferred until the intent_verdict for the first-pending
call arrives, then moves to:
  - Deny   button when judge recommends "deny"  (safety default)
  - Approve button for "approve" / "review" / anything else

Fallback timer (3s) claims focus anyway if no verdict arrives — covers
disabled judge and slow judge cases so keyboard users still land on a
button within a beat.

Focus claim is idempotent so batch approvals don't bounce focus across
buttons as trickling verdicts arrive.  hideApproval clears the timer
and the claimed flag so re-open cycles start fresh.

* fix(coordinator): drop approve-focus fallback timer

Previous commit added a 3s fallback that focused Approve if no verdict
arrived.  Ambiguous — a focus ring that lands "eventually" looks the
same as one that lands because the judge recommended approve.

Now focus only ever moves when a real intent_verdict arrives.  If the
judge is disabled or the verdict never comes, focus stays put and
keyboard users tab from the composer to reach the buttons.  An absent
focus ring is a clearer signal than an ambiguous one.

* fix(coordinator): address Copilot PR #393 feedback

Copilot findings, applied:

- Restore <h2> for Children / Tasks sidebar section labels (were
  changed to <span>).  .side-label class still applies; screen readers
  recover heading-level structure + rotor navigation.

- Mount the wait-indicator into #coord-header (the appbar container)
  instead of #coord-status.  #coord-status is reset via
  statusEl.textContent = ... on every state_change event, which was
  clobbering the wait indicator between ticks.  As a sibling inside
  the appbar, it survives state updates.

- Route `info` SSE events to appendText("info", ...) so they render
  with .msg.info (think-indigo) styling.  Prior routing to "tool"
  gave info events accent-tinted tool-call styling, miscategorising
  them visually.

- Define @keyframes ts-spin locally in the coordinator's <style>.
  Canonical definition lives in primitives/feed.css but this page
  doesn't link feed.css (no .feed-item usage), so the "judge
  evaluating…" spinner wasn't animating.

- Clear judgeVerdicts Map in hideApproval.  Map was growing unbounded
  across resolve cycles — fine for short sessions, leaks memory on
  long-lived coordinators with many approvals.

Not applied: Copilot's suggestion to restore focus-on-open or add a
fallback timer.  User explicitly requested no fallback — the design
decision is that the focus ring should only ever appear when the
judge has returned a verdict, so an absent ring reliably means "no
recommendation yet."  An auto-focus fallback would produce an
ambiguous ring that could be misread as "judge approved."
2026-04-20 20:10:03 -07:00
Patrick Buckley 412df28fe4 feat(design-system): DS phase 1 — chat primitives (.msg, .field, .appbar) (#391)
* feat(design-system): DS phase 1 — chat primitives for view migrations

Three new primitives enabling the chat-surface migrations (server UI +
coordinator):

  primitives/message.css   .msg + variants (user / assistant /
                           reasoning / tool / error / info / system),
                           .msg-meta author/timestamp slot, .msg-body
                           markdown target, .msg-actions hover-revealed
                           row, data-streaming="true" blinking caret.
                           Replaces .ts-msg* family in chat.css.

  primitives/field.css     .field wrapper with label/help/error, element
                           selectors for text/email/password/url/number/
                           search/tel/date/time/datetime/month/week +
                           textarea + select. .field.inline for checkbox/
                           radio rows, .field.invalid for error state.
                           Native-control focus-visible handled for
                           checkbox+radio so box-shadow ring remains
                           visible on unframed controls.

  chrome/appbar.css        chat-app header: back link + title + status +
                           action cluster. Distinct from the admin-style
                           .topbar (brand mark + nav + env metadata).
                           min-width:0 on .appbar-title so .dim subtitle
                           ellipsis fires under narrow viewports.

Preview.html extended with three demo sections exercising every variant
(plus a data-streaming example with live caret).

Fixes carried in from code review:
  - @media (hover: none) and (pointer: coarse) to match chat.css
    convention (hover-none alone is too broad, catches styluses)
  - .field-help uses --ink-3 (not --ink-4 which fails AA on --panel)
  - .msg-meta slot added so downstream PRs don't invent a custom class
  - Tool message pre/code on --panel-2 (parent is --panel; same-bg
    would make inline code disappear)
  - Checkbox/radio :focus-visible override (native controls lack a
    border for the default box-shadow ring to wrap)
  - Message.css comment corrected: "accent-tinted" not "cyan"

All rules scoped under [data-design="v1"]. Nothing existing modified.

* fix(design-system): address Copilot PR #391 feedback

- .msg-actions: add pointer-events: none when hidden, auto when visible.
  opacity:0 alone still intercepts clicks in the top-right corner —
  broke text selection on short one-line messages. Toggle applied in
  both hover/focus-within and the touch-media-query visible states.

- .field.inline comment: rewrite to match behaviour. Old comment said
  ".field stays flex-column" but the rule sets flex-direction: row.

- preview.html appbar demo: swap <a tabindex="0"> back-link to
  <button type="button">. tabindex-only anchors without href have
  inconsistent focus + screen-reader semantics; button is the correct
  native element for "navigate back via JS."
2026-04-20 16:50:28 -07:00
renovate[bot] 29d3953e52 chore(deps): update actions/setup-node digest to 48b55a0 (#390)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-04-20 16:44:59 -07:00
renovate[bot] a5d3e1b83c chore(deps): lock file maintenance (#372)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-04-20 16:44:40 -07:00
Patrick Buckley 2cfe6c23a0 refactor(design-system): post-#389 iteration — palette + glyphs + tint tokens
Live-preview-driven tuning pass following PR #389:

Palette
  - Accent hue 70 (amber) → 182 (teal). Amber collided with warn on
    same-surface k-badges; teal gives the brand accent its own hue.
  - ok / warn / err / think unified at L=0.50 light / L=0.68 dark and
    C=0.13-0.17 for palette coherence. err holds higher chroma so red
    doesn't wash; warn stays in the gold 80 lane (never 90+ / "puke").
  - Soft variants unified at L=0.94 / L=0.29, C=0.05-0.07.

New tokens
  --ok-live       brighter green for liveness signals (running dot)
  --ok-text       theme-aware text colour for filled green surfaces,
                  dark forest in light / bright mint in dark, ~7.5:1
                  against the approve-button bg in both themes
  --err-fill      darker red specifically for filled destructive
                  surfaces (.risk.crit) — bright --err as a fill
                  reads as alarm-loud
  --warn-tint,    directly-defined gold tints for k-tools k-badge —
    -tint-border  skips the color-mix-through-dark-cool-panel mud
                  that would otherwise render warm low-L mixes brown

Approve / Always / Deny
  - Approve filled green (color-mix --ok 28% into panel); text uses
    --ok-text for theme-correct contrast. Matches the pre-refactor
    turnstone/shared_static/chat.css convention where approve = green.
    Deviates from the Claude Design spec which had warn-tinted approve.
  - Always outlined dashed green (same --ok hue family); four non-colour
    cues for WCAG 1.4.1: fill state, border style, label, position.
  - Deny unchanged (err-outlined).

k-badge glyphs
  Replaced generic shapes with semantic symbols:
    tools  ⚙   approval ⚠\FE0E   policy §   role  ◉
    oidc   ⌘   token    ◆        judge  ⚖\FE0E  query ?
    step   ⇧   session  ◈        skill  ★   workstream ⇉
    fanout ⇶   default  ·
  ⚠ and ⚖ carry \FE0E to force text-presentation (avoid emoji
  promotion to coloured yellow triangle / blue scales on iOS Safari).
  token uses ◆ instead of ⬢ for universal font coverage.

k-approval split from k-tools
  k-tools stays gold (--warn family) — "tool call" kind.
  k-approval moves to green (--ok family) — matches the Approve button
  visually, completing the "⚠ approval → Approve" same-family story.

Running pill
  Text uses --ok (passes AA on pale --ok-soft); dot uses --ok-live +
  pulse. Liveness signal lives in the dot, not the text.

All changes stay under [data-design="v1"] — existing views untouched.
2026-04-20 16:30:39 -07:00
Patrick Buckley 78865c75d6 feat(design-system): DS-A + DS-B + DS-C — tokens, primitives, patterns (#389)
* feat(design-system): DS-A — tokens + typography scaffold

Adds turnstone/shared_static/design/{tokens.css,typography.css} as the
first phase of a multi-PR design refactor seeded by Claude Design.

- tokens.css: full palette + shape + rhythm, light default with
  [data-theme="dark"] override. oklch() raw colours, color-mix kept out
  of DS-A entirely (reserved for primitives in DS-B).
- typography.css: Inter + JetBrains Mono via Google Fonts; six-step
  scale (10/11/12/13/14/20-24). Utility classes .t-kicker/.t-meta/
  .t-btn/.t-row/.t-body/.t-stat/.t-h1.

Signature accent stays warm amber (oklch hue 70) rather than Claude
Design's teal — preserves turnstone's "Instrument Panel" identity.
All other tokens match the spec verbatim.

Additive: both files gate under [data-design="v1"] so existing views
(base.css, per-view stylesheets) are untouched. DS-B will opt views in
one at a time.

* feat(design-system): DS-B — chrome + primitives + preview page

Adds the reusable primitive kit that DS-C and DS-Cluster will build on:

  primitives/
    panel.css      .panel, .panel-head (.tools pinned right), .ghost
    buttons.css    .btn (pill 999px), .primary, .deny, .approve (amber)
    pills.css      .pill (running/thinking/attn/idle/err), .k-badge
                   (glyph-prefixed per WCAG 1.4.1), .chip, .risk
    stats.css      .stat + .stat-row, .mini-bar, .spark
    feed.css       .feed-item (grid ts/body/acts + .evi callout)
  chrome/
    topbar.css     48px sticky, conic-gradient brand mark
    sidebar.css    240px sticky, .shell layout, semantic swatches
  preview.html     renders every primitive in both themes with an
                   in-page theme toggle (tracks prefers-color-scheme)

Additive: every selector scopes under [data-design="v1"] so existing
views (base.css + per-view stylesheets) stay untouched.

Spec deviations from the Claude Design prototype:
- `color-mix(in srgb, …)` throughout; prototype had two `in oklab`
  usages — srgb per the spec's hard rule
- `.btn.approve` is warn-tinted amber, not green
  (approvals signal "needs attention"; amber resolves on approval)
- k-badge tint uses `color-mix` instead of oklch relative-colour syntax
  for broader browser support
- `@keyframes pulse/spin` renamed to `ts-pulse/ts-spin` to avoid
  clashing with keyframes in base.css on pages that load both
- `prefers-reduced-motion` disables pulse + spin animations
- Text-on-accent-soft + text-on-warn-tinted darkened via color-mix
  with ink to pass WCAG AA at 12px (fixes the classic same-hue trap)
- `.risk.crit` uses `#fff` text (dark-mode --panel on bright err fails)
- `.feed-item .acts button:not(.btn)` — compact action styling now
  skips .btn-classed buttons so they keep their pill shape
- Focus-visible rings on .btn, .ghost, .stat, .topnav, .side-item

* feat(design-system): DS-C — patterns (approval-dock, fleet-grid, live-feed)

Completes the design library with three patterns that compose primitives
into the signature product surfaces described in the Claude Design handoff.

  patterns/
    approval-dock.css   bottom-pinned approval strip. 1.5px-border,
                        --r-md squared action cluster: amber Approve
                        (primary), dashed Always, red Deny. kbd hints
                        and focus-visible rings on all three acts.
                        Call row (.dcall) framed as an inline code-
                        panel to emphasize "this is the exact call."
    fleet-grid.css      14-col grid of .node squares. State modifiers
                        (.s-ok/.s-thinking/.s-attn/.s-err/.s-idle/
                        .s-unreach) + --pct load fill. Hover uses
                        outline, not box-shadow (neighbour bleed is
                        the intended density cue). .fleet-legend
                        swatch row below.
    live-feed.css       thin scroll-container wrapper over the
                        .feed-item primitive with a sticky top fade.

  preview.html          imports the three patterns, extends the
                        fleet demo to use the real .fleet class +
                        legend, adds a live-feed panel, renders the
                        approval dock fixed at the bottom with
                        aria-live="polite".

Spec notes:
- Approve button is amber (warn-tinted), never green
- Dock action buttons are 1.5px-bordered 6px-radius squares — NOT
  pills — signaling "primary-action surface"
- All three dock actions clear WCAG AA in both themes via the same
  color-mix-with-ink darkening pattern used in .btn.approve
- kbd hint color matches primitives/buttons.css (--ink-3, not --ink-4)

View-level rewrites (coordinator.html + coordinator.js opt-in,
admin/cluster dashboard rebuild) are follow-up PRs — they need a
running server to test SSE streams + the approval POST contract.

* fix(design-system): scope DS-A tokens to [data-design="v1"]

Co-authored-by: eous <13773563+eous@users.noreply.github.com>

* fix(design-system): scope DS-A font vars to [data-design="v1"]

Co-authored-by: eous <13773563+eous@users.noreply.github.com>

* fix(design-system): align dark-mode selector with theme.js convention

Co-authored-by: eous <13773563+eous@users.noreply.github.com>

* fix(packaging): add shared_static/design/** to wheel includes

Agent-Logs-Url: https://github.com/turnstonelabs/turnstone/sessions/74d0939a-c55f-46b0-92f4-14d0cbfb7084

Co-authored-by: eous <13773563+eous@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: eous <13773563+eous@users.noreply.github.com>
2026-04-19 18:01:55 -07:00
Patrick Buckley a76d93b6c6 docs(coordinator): phase 8 PR C — API tour, skills guide, bulk-endpoints contract + wait diagram (#388)
* docs(coordinator): phase 8 PR C — API tour, skills guide, bulk-endpoints contract

Four deliverables that close out the phase 8 doc debt carried since
phase 1:

- docs/coordinator-api-tour.md — 9-step lifecycle walkthrough
  (create → subscribe → send → inspect children / detail → wait for
  fan-out → govern (trust / restrict / stop_cascade / close_all_children)
  → approve / cancel → close), one request + response per step, every
  SSE event type the UI has to handle, and every operation id cross-
  referenced against the live /openapi.json.  Integrators driving a
  coord session from a custom UI or SDK can work end-to-end from this
  doc without reverse-engineering the console page.

- docs/coordinator-skills.md — writing a SkillKind=COORDINATOR skill.
  Tool-surface diff (13 orchestration tools, no bash / edit / web /
  sub-agent), persona diff (orchestrator vs maker, composing on
  base_coordinator.md), SkillKind enum + migration 044, task_list
  integration, ws_id handling, wait vs inspect cost profile, three
  orchestration patterns (delegate-and-summarise, fan-out-and-
  synthesise, plan-then-delegate), testing surface.

- docs/bulk-endpoints.md — codifies the two shape idioms that shipped
  across phases 6–8: {results, denied, truncated} for bulk-read /
  bulk-create-with-payload (cluster/ws/live, spawn_batch); {<bucket>,
  failed, skipped} for cascade-mutation (stop_cascade,
  close_all_children).  Picks-by-semantics guidance so the next bulk
  endpoint author doesn't coin a third shape.

- docs/diagrams/27-coordinator-wait-for-workstream.puml + rendered
  PNG — sequence diagram covering spawn → wait (blocking, with
  bounded progress emission) → inspect → close.  Embedded in the
  API tour doc's §6 so the "why is my coord session blocking?"
  question has a visible answer.

No code changes.  All operation ids in the API tour verified against
a live build of the console spec; all markdown internal links
resolve; PlantUML renders clean on the system plantuml jar.

* docs(coordinator): address PR #388 copilot review

- api-tour.md child-event payload keys: events stamp `ws_id` as the
  coord's own id and carry the child's id separately as
  `child_ws_id`.  Doc previously listed `ws_id` as the child
  identifier on all four child_ws_* events, which would send SDK /
  UI implementers parsing the wrong field.
- api-tour.md SSE table: add the `status` event emitted by
  ConsoleCoordinatorUI.on_status (token usage + context_window +
  effort snapshot; fires on every streaming tick).  Previously
  omitted from the "every event type a UI has to handle" list.
- api-tour.md /children response key: server returns `{items,
  truncated}`, not `{children, truncated}`.  Also drop the
  `state=closed` query-param claim — the endpoint has no state
  filter; clients filter locally on the returned `state` field.
- skills.md task_list shape: the persisted row uses `id` (not
  `task_id` — the input schema uses `task_id`, the row uses `id`),
  has `child_ws_id` / `created` / `updated` (no `notes` field),
  and supports a 5th `reorder` action alongside add/update/remove/
  list.  Adds the parallel-dispatch caveat from the tool
  description.
- skills.md tenant-guard behaviour: foreign / hallucinated ws_ids
  don't return an empty result — they return explicit
  error/not-found/denied shapes that differ by op (mutating ops
  return `{error, status: 404}`; inspect returns `{error}`; wait
  reports state=denied).  Important distinction — a skill that
  expects empty on mismatch will mishandle every single case.

Docs-only; no code / schema / SDK changes.  All internal links
still resolve.
2026-04-19 10:15:08 -07:00
Patrick Buckley 67aaa236e8 feat(coordinator): phase 8 PR B — spawn budget + rate limit + /quota endpoint (#387)
* feat(coordinator): phase 8 PR B — spawn budget + rate limit + /quota endpoint

Adds two complementary controls so a runaway coordinator can't saturate a
cluster's max_active without anyone noticing:

- **Spawn budget** (hard quota) — cap on concurrently active children.
  Default 20 per coord.  spawn_workstream returns a tool error guiding
  the model to close idle children; spawn_batch routes overflow rows to
  `denied[]` with partial-success semantics.
- **Spawn rate limit** (soft pacing) — classic token bucket, defaults
  5 tokens/minute with burst 10.  A rate-limited spawn surfaces a tool
  error carrying `retry after Ns` so the model paces itself.  Zero
  refill rate is honoured as "disable refill" (bucket still honors the
  initial burst).

Shipped infra:

- `turnstone/core/spawn_quota.py` — thread-safe `SpawnBudget` +
  `TokenBucket`.  15 unit tests.
- `turnstone/core/session.py` — coord-only state built from settings at
  __init__.  Shared `_eval_spawn_quota(active)` helper drives both the
  single-spawn path (wraps the denial reason in `_coord_tool_error`) and
  the batch path (annotates `spec["_error"]`).  `_count_active_children`
  routes through `coord_client.list_children(include_closed=False)` and
  fails *open* on lookup error (budget is operator-safety, not security).
- `POST/GET /v1/api/coordinator/{ws_id}/quota` — partial-update admin
  endpoint mirroring the /trust + /restrict shape.  Accepts either the
  nested `spawn_rate` object or flat aliases — supplying both for the
  same field returns 400 so the admin UI can't half-migrate silently.
  Overrides are in-memory only (die on session reopen).  Audits via
  `coordinator.quota.updated` with before/after snapshots.
- Settings: `coordinator.spawn_budget`, `coordinator.spawn_rate.tokens_per_minute`,
  `coordinator.spawn_rate.burst` with ranges 1..500 / 0..600 / 1..500.
  The range bounds are the single source of truth — the endpoint
  validators and Pydantic schema both import from `settings_registry.SETTINGS`
  so bumping a cap in one place lights up everywhere.
- OpenAPI: `CoordinatorQuotaRequest` / `CoordinatorQuotaResponse` /
  `CoordinatorSpawnRateState` schemas + endpoint specs.  TS SDK regenerated.

Tests: +15 unit (SpawnBudget + TokenBucket), +17 endpoint (GET + POST
happy paths, range edges, mixed-body rejection, non-object spawn_rate,
service-token refusal), +11 session-side (budget blocks single spawn,
budget batch partial-success, rate batch partial-success, empty-body
reject, mutator live-update, non-coord session has no quota state).

Deferred (not this PR): per-skill scoping via migration 047 +
`prompt_templates.spawn_budget` column.  Count-only storage helper
(opportunistic — list_children at budget ≤ 500 is fine behind a
human-gated approval flow).

* fix(coordinator): address PR #387 copilot review

- Budget undercount: _count_active_children used list_children's
  LIMIT-then-Python-filter path, so a fan-out with many recently-closed
  children could push live rows past the SQL LIMIT and silently
  undercount, leaking spawn slots past the budget.  Replace with a new
  CoordinatorClient.count_active_children that uses
  storage.count_workstreams_by_state (SQL aggregate, no pagination,
  sums non-terminal states).  Tenant-guarded; fails open on storage
  error (budget is operator-safety, not a security gate).  New client
  tests cover the non-terminal count, the closed/deleted exclusion,
  the foreign-parent guard, and the fail-open path.
- Service-token bypass on /quota: both GET and POST used the default
  allow_service_bypass=True, so a service token whose user_id matched
  the coord owner could read or *raise* spawn capacity without the
  explicit admin.coordinator grant.  Flip both to
  allow_service_bypass=False for consistency with /restrict,
  /stop_cascade, and /close_all_children.
- OpenAPI contract leak: CoordinatorSpawnRateState was used for both
  the request and response shapes, which let generated SDKs imply
  clients could POST tokens_available (a read-only bucket reading the
  handler ignores).  Split into CoordinatorSpawnRateInput (request:
  tokens_per_minute + burst only) and CoordinatorSpawnRateState
  (response: adds tokens_available).  No runtime behaviour change;
  SDKs regenerate with two distinct types.

Drops the _ACTIVE_COUNT_SLACK / _ACTIVE_COUNT_MIN_LIMIT constants in
session.py — no longer needed since the new helper takes no limit
argument.  Updates the 5 session-side quota tests to stub
count_active_children instead of list_children.
2026-04-19 10:14:26 -07:00
Patrick Buckley 7d61f9a37c feat(coordinator): phase 8 PR A — spawn_batch + close_all_children batch tools (#386)
* feat(coordinator): phase 8 PR A — spawn_batch + close_all_children batch tools

Adds two model-facing batch tools so a coordinator can fan out without burning one approval per child:

- `spawn_batch` — create up to 10 child workstreams in a single approval. Serialised
  spawns so sibling ordering (by created_at) stays deterministic. Returns
  `{results: {idx: {ws_id, name, node_id, status}}, denied: [{idx, reason}]}`.
  Per-item validation / spawn failures surface in `denied[]`; the batch hard-errors
  on >10 rather than silent truncation.
- `close_all_children` — soft-close every direct child in one approval. Server-side
  Sem(16) fan-out via `coord_client.close_workstream`; `reason` propagates to every
  closed child's audit + workstream_config. Response mirrors `stop_cascade`'s cascade
  idiom: `{closed, failed, skipped}` where `skipped` is upstream-404 / already-gone.

Shipped infra:

- New console endpoint `POST /v1/api/coordinator/{ws_id}/close_all_children`
  (gated `admin.coordinator`, `allow_service_bypass=False`, 512-char reason cap,
  `coordinator.closed_all_children` audit).
- Shared `_fanout_on_children` helper — both `stop_cascade` and `close_all_children`
  now delegate to it (one place to own the snapshot → semaphore-gather → bucket-split
  skeleton).
- `CoordinatorClient.close_all_children(reason)` plus a `_post_url` seam that
  `_post` now reuses (no more duplicated transport-error handling).
- `_emit_batch_event` — best-effort SSE emitter modelled on `_emit_wait_event`.
  Emits `batch_started` / `batch_ended` pairs keyed by call_id. Throttled
  `batch_progress` deferred to a follow-up.
- OpenAPI request + response schemas, endpoint spec entry, TS SDK regenerated.
- Persona doc (`tools_coordinator.md`) covers the two new patterns.

Bulk-endpoint shape policy (codified in PR C later): split by semantic category —
`{results, denied, truncated}` for bulk-read / bulk-create-with-payload (cluster/ws/live,
spawn_batch), `{<bucket>, failed, skipped}` for cascade-mutation (stop_cascade,
close_all_children). No retrofit needed on stop_cascade.

Tests: new `test_coordinator_close_all_children.py` (8 endpoint tests), expanded
`test_coordinator_tools.py` (session-side prepare/exec, coord_client=None guards,
batch SSE events), expanded `test_coordinator_client.py` (route map, client method,
transport errors), tool-count assertions updated.

Deferred (not this PR): per-item selective-deny approval UI, throttled batch_progress
SSE, coordinator-skills doc + bulk-endpoints doc (PR C), spawn budget / rate limit (PR B).

* fix(coordinator): address PR #386 copilot review

- coordinator_client.close_all_children: pass the unformatted path template
  as log_path so telemetry aggregates don't fragment per session (ws_id
  still lives in the real URL).
- session.py: drop dead spawned_ids accumulator in _exec_spawn_batch —
  leftover from an eager-register path that got removed earlier.
- close_all_children tool JSON: document the 512-char server-side cap on
  reason and that reason is echoed back in the response payload.  Added
  maxLength:512 on the schema property so the LLM sees the constraint.
- CoordinatorCloseAllChildrenRequest: add Field(max_length=512) so the
  OpenAPI schema reflects the runtime 400-on-overflow constraint.
2026-04-18 23:50:25 -07:00
Patrick Buckley a7be0e1610 chore: bump version to 1.5.0a2 2026-04-18 21:15:24 -07:00
Patrick Buckley 7f40141c16 Feat/composer ux squashed (#385)
* refactor(ui): shared composer widget (pane / coordinator / coord-create)

The interactive workstream pane (turnstone-server), the coordinator
session view (turnstone-console), and the console home's "start a new
orchestration task" form had drifted into three unrelated composer
implementations with different DOM, different class names, and
different behaviour sets.  All three now build on a single
`shared_static/composer.js` widget parameterised by feature flags.

The widget owns the textarea, send button, optional stop button,
optional attach button + file input + chip container, optional
drag-drop / paste-image wiring, optional queue-while-busy send-label
rotation, optional touch-aware Enter-to-send, and an optional
collapsible Options panel with input / select fields, live summary
chip, and localStorage-persisted open/closed state.  A stacked layout
puts the textarea above the action row for creation-form consumers;
the inline layout keeps the chat-style single row for send composers.

Consumer wiring:

- Pane: attachments + stopBtn + queueWhileBusy + drag-drop; keeps
  its own attachment-upload pipeline and routes file events through
  the composer's onAttach callback.  Pane-specific CSS (.pane-stop
  / .pane-send.queue-mode) retired; `.ts-composer-stop` / `.ts-
  composer-send--queue` in shared/chat.css take their place.
- Coordinator send: just textarea + send with touchEnterSends=true
  to preserve the pre-refactor tap-to-send behaviour on tablets.
  The header-mounted coord-cancel-btn stays (different semantics
  than a per-generation stop).
- Coord-create: stacked layout, rows=3, Start-labelled send, Options
  dropdown holding Name + Skill; Ctrl/Cmd+Enter handler scoped to
  the composer mount.  `_createCoordinator` lost its DOM-ref shape
  in favour of raw values + a setBusy callback; a single
  `_refreshHomeCoordSubmitEnabled` reconciler owns the submit
  button's disabled flag so the 503 probe and in-flight submit
  can't race each other.

Shared chat.css grew the .ts-composer-stop, .ts-composer-options-*,
and .ts-composer--stacked blocks; the coord-create consumer dropped
its custom .home-composer-task/-row/-name/-skill/-submit selectors
and the "Start a new orchestration task" panel title so the
placeholder text carries its own context, matching the webui
dashboard's clean look.

The visible behaviour on each surface is intentionally the same as
before; the change is structural — the three composers can no longer
drift apart silently.

* fix(composer): review fixes — Enter guard, single disable owner, widget-owned stop reset

Review of the squashed whole caught issues that the piecewise reviews
missed because they only become visible with all three consumers
together:

- **Enter bypassed sendBtn.disabled.**  Composer's Enter keydown
  handler called _fireSend() without checking sendBtn.disabled.  In
  the coord-create flow submitHomeCoord doesn't clear the textarea
  before the POST completes (it redirects on success), so two rapid
  Enter presses both fired _createCoordinator and could create two
  coordinators.  Enter now mirrors the click path.
- **Two writers to sendBtn.disabled.**  Composer.setBusy and
  _refreshHomeCoordSubmitEnabled both wrote the flag.  They agreed
  in sequence today but it was the exact drift hazard the reconciler
  was meant to eliminate.  Added an externalDisable option; when
  true Composer's setBusy rotates labels / placeholder / stop button
  but leaves sendBtn.disabled to the caller's reconciler.  The
  coord-create composer opts in.
- **setSendLabel dead weight.**  Called on every busy transition
  with static "Start" / "Starting…".  Composer.setBusy now rotates
  labels universally (not just in queueWhileBusy mode); the busy
  label goes at construction via the existing busyLabel option and
  setSendLabel is removed.
- **Pane reached through composer to reset stopBtn.**  Pane.setBusy
  was writing stopBtn.textContent / aria-label / dataset after
  delegating — internals leaking through.  Composer.setBusy now
  resets the stop button's standard label + clears forceCancel on
  every transition (matching the comment that used to live in Pane);
  Pane drops the reach-through.
- **destroy() left detached DOM reachable.**  Back-refs (inputEl,
  sendBtn, etc.) are nulled out so post-destroy access fails loudly
  instead of silently mutating detached nodes.
- **_maybeAutoResize dead indirection.**  The enabled-check folded
  into autoResize itself.

* fix(composer): busyLabel context-sensitive default + busyPlaceholder universal

Round-two review caught three related loose ends:

- Default `busyLabel="Queue"` was fine when label rotation was queue-
  mode-only, but became misleading after the earlier review fix made
  rotation universal: non-queue consumers calling setBusy(true)
  without explicit busyLabel would flash "Queue" on the disabled
  button.  Default is now context-sensitive — "Queue" when
  queueWhileBusy=true, sendLabel otherwise (no rotation).  The
  coord-send composer no longer needs to touch the label at all.
- `busyPlaceholder` JSDoc implied universal swap on busy but the
  implementation gated it on queueWhileBusy.  Decoupled — the
  placeholder swaps whenever busy, with callers that don't set
  busyPlaceholder seeing no visible change because it defaults to
  the idle placeholder.
- `options.toggleLabel` and `options.onChange` were supported by the
  implementation but undocumented.  JSDoc for the options shape
  enumerates every supported key + its default.
2026-04-18 21:11:08 -07:00
Patrick Buckley 7c16b0dfa8 refactor(routing): replace hash-ring rebalancer with rendezvous (HRW)… (#384)
* refactor(routing): replace hash-ring rebalancer with rendezvous (HRW) hashing

Routing was a stored bucket table maintained by a central rebalancer
daemon, which shared its liveness primitive (services.last_heartbeat)
with the collector — when a heartbeat-fresh node went into a zombie
HTTP-handler-broken state, neither the collector nor the rebalancer
could self-correct, and the router kept directing traffic at it.
Rendezvous hashing makes the route a pure function of (ws_id,
live_services) so the heartbeat is the single source of truth and any
liveness-eviction propagates to the next route call without a separate
state-publication step.

The rebalancer's central state has no analogue: the new router computes
the per-key node winner on every call, the collector pushes membership
updates into the router cache from its discovery thread, and per-route
overrides survive on workstream_overrides. Eager workstream migration
goes away; in-flight workstreams lazily rehydrate from storage on the
new owner — already the dead-node behaviour.

* fix(tools): describe rendezvous re-routing on spawn/inspect node_id

The first pass overclaimed `node_id` "stays canonical for this
workstream's lifetime" — under rendezvous routing the active owner
re-derives per-call from live membership, so a node join/drop after
spawn can shift it.  Tool descriptions now say `node_id` is the
spawn-time binding; subsequent ops re-route via rendezvous over the
current live-node set; the new owner lazily rehydrates from shared
storage; coordinators should re-read with inspect_workstream rather
than caching the value.
2026-04-18 19:02:52 -07:00
Patrick Buckley 9826ea15c5 feat(coordinator): phase 7 — governance + skill metadata + cross-cutt… (#383)
* feat(coordinator): phase 7 — governance + skill metadata + cross-cutting invariants

Combines three stacked sub-PRs into a single coordinator phase-7
shipment against the phase-7 plan doc.  The sub-PR structure (0 / A /
B) preserved on individual branches for reviewer drill-down; this
branch is the one reviewers should merge.

## Sub-PR 0 — service-auth boundary invariants

Shared helpers and contracts that lock the console ↔ node service-auth
boundary so later authz surfaces use them by construction.

- ``_effective_user_filter(request)`` in both ``turnstone.console.server``
  and ``turnstone.server`` with a shared ``DENY_EMPTY_SUB`` sentinel
  on ``turnstone.core.auth``.  Three-way return — admin/service
  bypass, scoped caller uid, or fail-closed sentinel on blank sub.
  Four callsite migrations (``_coordinator_rows``,
  ``coordinator_children``, ``coordinator_metrics``,
  ``cluster_ws_live_bulk``).

- ``StorageBackend`` class docstring codifies the tenancy contract
  (every list/count/aggregate method must accept ``user_id: str |
  None = None`` and push ``WHERE user_id = :user_id`` into SQL) and
  the ``_mapping`` row-access contract.  New
  ``turnstone.testing.row_contract`` ships ``assert_row_like()``.

- ``_verify_collector_service_scope`` probes an upstream node at boot
  with ``expected_node_id=_scope-probe_``; a 409 proves the scope
  gate was passed, a 403/401 sets ``collector_scope_error`` and
  causes ``cluster_snapshot`` / ``cluster_events_sse`` to return 503
  with a remediation hint.  Probe URL allowlist rejects non-http(s)
  schemes and 169.254.0.0/16 hosts.

- 4xx log-level floor on ``_NodeDashboardCache.get``,
  ``_fetch_live_block``, and ``_proxy_sse`` — dotted-hierarchy
  prefixes with bounded body previews.  ``_bounded_body_preview`` and
  ``_bounded_stream_preview`` strip control chars.

## Sub-PR A — coordinator governance core

Mid-session governance surface for coordinator workstreams.

- **Trusted-session mode.**  New ``coordinator.trust.send``
  permission (migration 042).  ``ChatSession.set_trust_send`` /
  ``revoke_tools`` methods with a ``_governance_lock``.  ``POST
  /v1/api/coordinator/{ws_id}/trust {send: bool}`` double-gated on
  ``admin.coordinator`` AND ``coordinator.trust.send`` with
  ``allow_service_bypass=False`` so service tokens can't escalate.
  ``_prepare_send_to_workstream`` auto-approves sends whose target is
  in the coordinator's own subtree; foreign ws_ids still require
  approval.  ``_is_own_subtree`` checks both ``parent_ws_id`` AND
  ``user_id`` to defend against cross-tenant row corruption.

- **Audit-layer credential redaction.**  ``record_audit`` walks
  ``detail`` (dicts, lists, tuples, sets, frozensets; keys too)
  and routes every string through ``redact_credentials`` + a C0
  control-char scrub.  New kw-only ``raw_detail=True`` opt-out.
  ``_has_any_string`` fast-path.  Audit action registry extended
  with the four new governance sub-prefixes.

- **Mid-session revocation + cascading stop.**  ``POST
  /v1/api/coordinator/{ws_id}/restrict {revoke: [...]}`` caps 256
  entries / 128 chars; ``_prepare_tool`` short-circuits with a
  tool-error.  ``POST /v1/api/coordinator/{ws_id}/stop_cascade``
  cancels the coord's in-flight generation then dispatches
  ``cancel_workstream`` for every direct child in parallel via
  ``asyncio.gather`` bounded by ``Semaphore(16)``.  Per-child
  outcomes split into ``cancelled`` / ``failed`` / ``skipped``
  (404 = already-gone rather than dispatch-broken).  Both endpoints
  apply ``allow_service_bypass=False`` on the admin gate.

- **Shared plumbing.**  ``_resolve_coord_session`` helper collapses
  the handler prelude three endpoints shared.  ``_emit_coord_audit``
  wraps ``record_audit`` in a dedicated ``ThreadPoolExecutor``
  (``app.state.audit_executor``) so audit bursts don't starve cancel
  dispatches.  ``_require_json_object`` guards body parsing so non-
  object JSON returns 400 instead of 500.

## Sub-PR B — skill metadata governance

- **Description validator (migration 043).**  ``prompt_templates``
  rows now require a non-empty ``description``.  Existing empty rows
  get backfilled with a ``"Skill: <name>"`` placeholder on upgrade.
  The installer (``admin_skill_discover``) and MCP prompt sync both
  synthesise a placeholder when the upstream description is blank
  so non-admin write paths satisfy the invariant.

- **Skill kind classifier (migration 044).**  New
  ``prompt_templates.kind`` column (``interactive`` / ``coordinator``
  / ``any``; defaults to ``any``).  New
  ``turnstone.core.skill_kind.SkillKind`` StrEnum is the single
  source of truth; Pydantic schemas type ``kind`` as ``SkillKind``
  (OpenAPI advertises the enum) and the handler validator catches
  the ValueError.  ``list_skills_filtered`` gains a
  ``kinds: list[str] | None = None`` SQL filter.
  ``CoordinatorClient.list_skills`` defaults to
  ``kinds=["coordinator", "any"]`` so interactive-only skills are
  hidden from the orchestrator.

- **``scan_status`` → ``risk_level`` rename (migration 045).**
  Lossless column rename to align with ``IntentVerdict.risk_level``
  terminology.  Swept storage (both backends + schema + protocol),
  handlers, API schemas, tool JSON, generated OpenAPI specs,
  TypeScript SDK types, frontend (``governance.js``), tests, and
  English prose in ``docs/judge.md`` + ``docs/tools.md``.  The
  user-facing on-load warning now reads ``has risk level:
  {risk_tier}``.  Tool JSON's ``risk_level`` enum corrected to the
  scanner's actual taxonomy (``safe / low / medium / high /
  critical``; was the never-shipped ``clean / flagged / unscanned /
  pending``).  Historical migration 021 left untouched.

## Migrations

042 (``coordinator.trust.send`` perm — PR A)
043 (description backfill — PR B)
044 (``kind`` column add — PR B)
045 (``scan_status`` → ``risk_level`` rename — PR B)

All four use position-anchored permission strings / host-side
parse-filter-rejoin on downgrade where SQL ``REPLACE`` could
corrupt prefix-overlapping values.

## Verification

- ``ruff check turnstone tests`` clean.
- ``mypy turnstone`` clean on 165 source files.
- ``pytest -m "not live"``: 4431 passed (+85 over the phase-6
  baseline).  Includes +32 tests in ``tests/test_service_auth_boundary.py``
  and +38 in ``tests/test_coordinator_governance.py``; shared fixtures
  extracted to ``tests/_coord_test_helpers.py``.
- Generated OpenAPI JSON (``sdk/typescript/openapi-{console,server}.json``)
  regenerated via ``sdk/typescript/scripts/generate-types.py``; zero
  ``scan_status`` occurrences remaining outside the historical
  migration 021 and the rename migration 045.

## Security reviews

Both reviews flagged by the phase-7 plan (items 1 + 5, plus 0a's
refuse-to-serve gate) ran through the multi-stage ``/review``
pipeline twice per sub-PR; all confirmed findings landed in-branch.

* fixup(phase-7): CI lint + PR #383 review fixups

Addresses the lint CI failure (ruff format) plus 12 findings from the
two automated PR reviewers.

Copilot:
- ``_sqlite.list_installed_skill_urls`` / ``_postgresql.list_installed_skill_urls``
  used positional row indexing (``r[0]``/``r[1]``/``r[2]``) while this
  same PR's ``StorageBackend`` class docstring forbids it.  Switched
  both to ``r._mapping["..."]`` access.
- ``list_skills.json`` previously advertised ``risk_level=""`` as a
  filter for unscanned skills, but the implementation treats empty
  strings as "no filter".  Clarified the tool description to say
  omit the filter entirely to include unscanned rows, and added an
  explicit ``enum`` on the parameter restricting it to the scanner
  tiers.  ``_prepare_list_skills`` keeps the ``strip() or None``
  normalisation — unscanned filtering now has an unambiguous contract.
- ``test_storage_skills_filtered.test_risk_level_filter`` used the
  legacy ``clean`` / ``flagged`` values from the pre-rename column.
  Rewritten with the scanner's actual taxonomy (``safe`` / ``high``).

github-code-quality (CodeQL):
- ``test_deny_sentinel_is_singleton`` previously asserted
  ``cs.DENY_EMPTY_SUB is cs.DENY_EMPTY_SUB`` — an identical-expression
  comparison.  Rewritten as two separate ``from ... import ... as`` aliases
  (``FIRST_READ`` / ``SECOND_READ``) so the identity check is between
  distinct bindings.
- ``test_restrict_empty_revoke_is_noop_but_audits`` unpacked ``state``
  without using it.  Renamed to ``_state``.
- Mixed import styles in ``test_service_auth_boundary.py`` — the
  file previously used both ``import turnstone.console.server as cs``
  and ``from turnstone.console.server import ...`` for the same
  module (same story for ``turnstone.core.auth`` and
  ``turnstone.server``).  Consolidated to the ``from X import Y`` style
  used elsewhere in the file; the ``_fetch_live_block`` test now
  patches via pytest's ``monkeypatch`` fixture instead of a manual
  rebind through a module alias.

CI:
- ``ruff format`` reformatted one line in
  ``tests/test_coordinator_endpoints.py``.

Verification: ruff check + mypy clean (166 files); 4459 non-live
pytest pass.

* fix(tests): swap asyncio marker for anyio in service-auth boundary tests

PR #383 CI caught that the 13 ``@pytest.mark.asyncio`` decorators I
added in ``test_service_auth_boundary.py`` are an off-convention
choice — the rest of the repo uses ``@pytest.mark.anyio`` (148 sites
vs my 13).  The CI environment pulls in ``anyio`` but not
``pytest-asyncio``, so every async test in this one file was failing
with "async def functions are not natively supported".  It passed
locally by accident — my dev venv happens to have pytest-asyncio
installed ambiently.

Swapped all 13 marker sites to ``@pytest.mark.anyio``.  No functional
change; the tests run under the same default asyncio backend anyio
provides.

Verification: ruff + mypy clean (166 files); 4459 non-live pytest
pass.
2026-04-18 10:20:19 -07:00
Patrick Buckley cab57f244d refactor(channels): backfill review of Slack/Discord adapters (#382)
* refactor(channels): backfill review of Slack/Discord adapters

Retrospective multi-stage review of the Slack (PR #355) and Discord
channel adapters — they shipped before the review pipeline existed,
so this pass goes back and fixes everything the pipeline would have
caught plus a follow-up round of ultrareview findings.

## Security (8 fixes)

- Adapter-side owner checks on all interactive flows: Discord
  ApprovalView / PlanReviewView encode the owner Discord user ID in
  the embed footer (`{ws_id}|{corr_id}|{owner_id}`) and reject
  non-owner clicks; Slack plan-approve / request-changes /
  feedback-modal gain owner tracking in `_pending_plan_review_ts`
  and a shared `_ensure_plan_review_owner` gate.  These closed the
  two critical authz gaps where the gateway's service-scoped JWT
  bypassed server-side ownership checks.
- Discord thread-message gate: only the registered invoker can
  drive the workstream (prevents a linked user posting in another
  user's public thread from injecting into their assistant).
  Invoker recorded explicitly so `/ask` follow-ups survive the
  `channel.create_thread` bot-as-owner quirk.
- Slack /link flow + per-user identity gate: unlinked Slack users
  see an ephemeral `/turnstone link <token>` prompt on every
  message instead of silently creating workstreams under the
  shared gateway identity.  Rate-limited (5/hour) to block online
  token enumeration.
- Gateway `/v1/api/notify` requires `write` scope on the validated
  JWT; low-scope tokens get 403 + audit.
- Thumbnail URL validator DNS-resolves the hostname before fetch
  and rejects any resolved IP that's loopback / link-local /
  multicast / reserved, plus an explicit deny-list for IPv6 cloud
  metadata (`fd00:ec2::/32` — AWS Nitro IMDS + ECS task metadata)
  that would otherwise slip past the `is_private` allowance.
- Per-user rate limit (10 msgs / 60s) + 8 KiB inbound size cap on
  Slack DMs / channels / notification-reply threads so one user
  can't exhaust the shared LLM budget.
- Discord /link rate limit (5/hour) for token-enumeration defense.

## Bug fixes (9 correctness issues)

- Slack DM routing: each top-level DM no longer spawns a fresh
  workstream (was using per-message `ts` as the route key).
- Multi-chunk Slack responses thread correctly under the first
  chunk's ts instead of fragmenting as independent top-level
  messages.
- Finalize the outgoing StreamingMessage before swapping channel /
  thread_ts mid-stream, so buffered tokens still land on the old
  thread.
- Redundant `chat_update` on approve/deny eliminated by popping
  `_pending_approval[ws_id]` after local resolution.
- Notification reply tracking on Discord only registers for DMs
  (guild-channel targets were storing channel IDs where user IDs
  were expected, so legitimate replies were always rejected).
- `get_channel_default_alias` rolls `_channel_default_ts` back on
  `list_models()` failure so the next caller retries instead of
  serving an empty alias for the full TTL.
- Slack `subscribe_ws` purges dead SSE tasks before the
  membership short-circuit (previously an unhandled exception left
  the ws_id in `_subscribed_ws` forever, silently no-opping
  subsequent subscribes).
- ChannelRouter `_create_locks` is now an LRU-bounded OrderedDict
  that evicts only unheld locks (original dict grew unbounded;
  naive LRU could evict a held lock and let a second caller race
  through the critical section, creating duplicate workstreams).
- Slack `_parse_ts` pads the fractional field to 6 digits so
  `"1.2"` and `"1.000002"` stop colliding as `(1, 2)` in the
  latest-session tiebreaker.

## Performance (6 fixes)

- StreamingMessage keeps a rolling truncated display string capped
  at `max_length` so per-flush cost is O(max_length) instead of
  O(total_streamed_chars) — long streaming responses no longer do
  quadratic work every edit interval.
- `StreamingMessage.finalize()` caches the joined content so the
  Discord stream-end DM-forward path doesn't re-join a multi-MB
  buffer twice.
- `PendingApproval` stores the Block Kit payload posted to Slack;
  `IntentVerdictEvent` appends the verdict in-place and
  `chat_update`s, skipping an extra `conversations_history`
  round-trip.
- ChannelRouter `lookup_ws_id()` TTL-caches the channel →
  ws_id resolution (30s TTL, 4096-entry LRU); hot inbound paths
  skip storage on every message.
- Service-discovery startup retry uses exponential backoff
  (1s → 8s cap) with a 30s deadline instead of 30 × 1s fixed
  sleep.
- `_archive_session` now calls `router.close_workstream` so the
  `_node_urls` cache entry is dropped (was leaking one entry per
  archived session).

## Quality / refactors (19 improvements)

- `cli.main()` extracted from a 365-line function into focused
  helpers; imports carefully kept lazy where test patches target
  source-module paths.
- `_run_gateway` finally block now awaits `adapter.stop()` on
  every adapter so SSE tasks, httpx clients, and the Slack socket
  handler close cleanly on shutdown.
- Shared SSE reconnect loop extracted to `turnstone/channels/_sse.py`
  (`run_sse_stream` with `on_event` + `on_stale` callbacks); both
  adapters' `_sse_listener` methods just wire up callbacks. The
  "404 stops reconnect" invariant is enforced inside the helper
  so a broken `on_stale` can't livelock.
- `_on_ws_event` god-dispatchers split into per-event `_handle_*`
  methods with a thin isinstance dispatcher at the top.
- Slack `_on_approve` / `_on_deny` collapsed into a single
  `_resolve_approval(*, approved: bool)`.
- `ApproveRequestEvent` policy evaluation hoisted into
  `ChannelRouter.evaluate_tool_policies` returning a
  `PolicyVerdict`; adapters switch on the verdict kind.
- `ChannelAdapter` protocol trimmed to the four methods adapters
  actually implement; unused `ChannelEvent` dataclass removed.
- Shared constants lifted to `turnstone/channels/_config.py`.
- `_cleanup_stale_route` and `unsubscribe_ws` share a
  `_clear_ws_state` helper.
- `StreamingMessage` private attrs promoted to `message` /
  `message_ts` / `accumulated_text` properties so callers don't
  reach past the `_`-prefix.
- Various cleanups: dead var, noqa'd lambdas, renamed
  `_policy_handled` → `policy_handled`, inlined single-use
  helpers, added module docstrings, documented
  `SlackRoute.parse` edge cases.
- `chunk_message` plain-text fast path (no backticks → skip
  fence bookkeeping).

## Test coverage

Added 45 tests (178 → 223):

- `tests/test_channel_sse.py` (new) — SSE reconnect / backoff /
  404-stale-route / on-stale-exception / invalid-JSON-skip /
  on-event-exception-doesn't-kill-stream / per-connection token
  refresh / ConnectError retry.
- ApprovalView + PlanReviewView owner-check regression tests
  (owner allowed, non-owner rejected, legacy 2-pipe footer fails
  closed, modal path rejected for non-owner, `/ask`
  bot-as-thread-owner follow-up allowed).
- Slack `_recover_routes` latest-ts-wins, `_archive_session`
  drops route + closes workstream.
- SSRF tests: DNS rebinding rejected, IPv4 link-local metadata
  rejected, IPv6 ULA metadata (fd00:ec2::254 / fd00:ec2::23)
  rejected.
- Slack link prefix match (natural-language prompts don't
  hijack), link rate-limit ceiling.
- SlackRoute round-trip across all three shapes + lax-parse
  behaviour.

Lint (ruff) + mypy clean; 210 channel-focused tests pass.

* chore(channels): address PR #382 review-bot feedback

Three line-level findings from github-code-quality on the backfill
review PR.  Copilot had no line-level comments.

- _sse.py:132 — the `except httpx.HTTPStatusError: pass` branch was
  flagged as an empty except.  The original status was already logged
  at WARNING inside the try block (we re-raise ourselves after
  logging), so the handler has real intent.  Added a debug log of the
  exception text + a comment explaining the control flow, so the
  empty-except lint stops firing and the next reader sees why we
  fall through to backoff.
- discord/bot.py:430, cli.py:354, slack/bot.py:1127 — `await task`
  inside `contextlib.suppress` was flagged as "statement has no
  effect".  It's a false positive (await is an effect) and the
  alternative try/except/pass triggers ruff SIM105.  Kept the
  contextlib.suppress pattern and added an explanatory comment above
  each call so the intent (await CancelledError propagation before
  state cleanup) is obvious; will reply on the PR thread noting the
  false positive.

No behavior change.  Lint + mypy clean; 210 channel tests pass.
2026-04-18 05:49:54 -07:00
Patrick Buckley bd6670d748 feat(coordinator): phase 6 — polish, observability, active-coords via SSE, frontend cleanup (#381)
* feat(coordinator): phase 6 — polish, observability, active-coords via SSE, frontend cleanup

Squashed from two working commits:
  1. phase-6 backend polish + active-coords SSE
  2. phase-6 frontend cleanup (legacy chat-view classes + designer nits)

Both tier-A/B observability items and tier-C frontend consolidation
ship together — the shared-vocabulary migration touches surfaces the
backend polish already had its hands in, so one combined commit keeps
the diff reviewable as a coherent phase.

Observability
-------------

- **Coordinator-side wait dashboard** — `_exec_wait_for_workstream`
  emits `wait_started` / `wait_progress` / `wait_ended` SSE events
  via a new `progress_callback` hook on
  `CoordinatorClient.wait_for_workstream`; coordinator.js renders a
  "⧗ waiting · N ws · Ts" header indicator keyed by call_id so
  overlapping waits coexist.  Progress throttled to emit only on
  snapshot-diff or 5s heartbeat; full results dict attached only on
  transitions so a 600s wait doesn't flood SSE listener queues.
  Indicator only attaches when a proper header host exists (no
  floating document.body fallback) and is cleared on SSE reconnect
  so a dropped `wait_ended` can't pin the badge.
- **`cancel_workstream` forensics** — `server.cancel_generation`
  captures `ui._pending_approval` tool names +
  `session._queued_messages` count / preview before invoking
  `session.cancel`, returning the snapshot as `dropped`; routing
  proxy passes it through to the tool result.  Preview runs through
  `redact_credentials` before the 120-char truncate so pasted
  secrets / connection strings don't land verbatim in the
  coordinator's conversation history.
- **Per-coordinator metrics** — `GET /v1/api/coordinator/{ws_id}/metrics`
  returns `spawns_total` / `spawns_last_hour` / `child_state_counts`
  / `judge_fallback_rate` (substring match on verdict.tier) plus
  zero placeholders for wait_* pending dedicated instrumentation.
  Derived from new `storage.count_workstreams_by_state` +
  `count_workstreams_since` aggregate helpers — no 10k-row
  hydrated-select to compute a histogram.  Ownership 404-mask
  matches `coordinator_detail`.
- **Coordinator skill in inspect** — `CoordinatorManager.create`
  resolves `skill` → `template_id` / `applied_version` via
  `get_skill_by_name` + new `storage.count_skill_versions`
  (replacing the SELECT-all-for-COUNT anti-pattern) and persists
  them on the workstreams row.  `/new` handler dispatches via
  `asyncio.to_thread` so blocking storage calls don't stall the
  event loop.
- **`wait_for_workstream(since=…)`** — optional prior-snapshot
  hint; when supplied, the wait loop diffs each polled ws_id that
  IS in `since_map` and exits on any change, independent of mode.
  ws_ids absent from `since_map` fall through to the normal mode
  condition — a disjoint since dict no longer silently exits the
  wait on tick one.
- **`task_list.child_ws_id` referential cleanup** —
  `CoordinatorClient.cleanup_dead_task_child_refs(ws_id)` holds the
  same per-ws `_task_lock` as `task_list_*` so a close racing a
  task_list write can't lose the mutation.  `CoordinatorManager.close`
  delegates.  Final save-failure logs at `warning` instead of
  `debug`.

Home view live-updates
----------------------

- **Active-coordinators via SSE** instead of a 5s poll —
  `ClusterCollector.ensure_console_pseudo_node` +
  `emit_console_ws_created / _closed / _state / _rename` plumbing;
  `CoordinatorManager.create / open / close / eviction` +
  `ConsoleCoordinatorUI.on_state_change / on_rename` all fan out
  through the collector.  The pseudo-node is exempt from the
  discovery-loop eviction; rehydrate-path eviction now also emits
  `console_ws_closed` for the evicted row so other tabs drop it
  live.  `app.js` reads coordinators from
  `clusterState.nodes["console"]`; poller + back-compat shims
  deleted (9 call sites).  Overview / nodes list skip the
  pseudo-node so it doesn't inflate cluster totals.  Tenant-filtering
  preserved by excluding the pseudo-node from
  `collector.get_workstreams` so `/v1/api/cluster/workstreams` still
  uses the existing tenant-filtered `_coordinator_rows` path.
  `CoordinatorManager.NODE_ID` bound from
  `ClusterCollector.CONSOLE_PSEUDO_NODE_ID` so the two literals
  can't drift.

Frontend perf
-------------

- **Bulk cluster-ws live endpoint** — `GET /v1/api/cluster/ws/live?ids=`
  returns `{results, denied, truncated}` (cap 50); coordinator.js
  batches visible-row live-badge fetches into one bulk request per
  ~250ms window (replaces per-row /detail polling).  Ownership
  check routes through the empty-string-safe pattern (non-admin
  with empty `caller_uid` doesn't match empty-owner rows).

Legacy chat-view class cleanup
------------------------------

- Drop the `.msg` / `.msg-user` / `.msg-assistant` / `.msg-tool` /
  `.msg-error` / `.msg-info` / `.approval-block` / `.approval-tool`
  / `.approval-btn` / `.approval-badge` / `.approval-prompt` /
  `.approval-feedback-input` / `.approval-actions` / `.pane-input` /
  `.pane-input-area` / `.pane-input-row` / `.pane-attach` /
  `.pane-attach-chip` / `.coord-msg` / `.coord-body` / `role-*` /
  `btn-approve` / `btn-deny` / `btn-always` / `verdict-glow-*`
  legacy dual-class names left over from the phase-4 migration.
  Every JS className concatenation + querySelector + CSS selector
  now uses the `ts-*` vocabulary from `shared_static/chat.css` (and
  `ui/static/style.css` where the interactive-page extensions
  live).  Feature-specific class names that don't map to `ts-*`
  stay — `msg-queued` / `msg-editing` / `msg-actions` / `msg-edit-*`
  / `msg-user-attach*` / `msg-user-text` / `queued-badge` /
  `queued-dismiss` / `tool-name` / `tool-cmd` / `tool-diff` /
  `tool-header` / `tool-preview`.

Designer nits
-------------

- **`.ui-btn--icon:focus-visible`** — new rule matching `.ui-btn`'s
  `outline: 2px solid var(--accent); outline-offset: 1px` so the
  compact icon variant gets the accent ring instead of the
  browser-default outline.
- **Dropped speculative 701-880px composer wrap rule** — the flex
  math at ≥701px fits comfortably in every desktop viewport, so the
  mid-zone break rule was forcing a 2-line layout where the browser
  wouldn't have wrapped naturally.  The existing `<700px` full-stack
  covers the original wrap observation.
- **`.verdict-badge` border-top** + **`.ch-row.highlight`
  prefers-reduced-motion** — confirmed already in main; no
  additional code change needed for phase 6.

Follow-up designer-review findings
----------------------------------

- Dropped `border-top` from `.ts-approval-badge` + `.ts-approval-body`
  (chat.css's max-content width / flex-gap made them read as
  truncated / floating lines).
- `var(--muted)` → `var(--fg-dim)` on denied tool names (undefined
  token was silently failing).
- Dropped 3 dead `.ts-approval-badge.badge-*` rules + duplicated
  `.ts-approval-btn:focus-visible` + dead `.reasoning` CSS rule +
  `contains("reasoning")` JS guard.
- Dropped `tool-row` / `approval-header` / `btn-row` / `label` dead
  legacy classes in the coordinator.
- Added `:focus-visible` to `.ch-row a.ws-link` + `.task-row` so
  keyboard users get the accent ring on sidebar rows.

Cleanups
--------

- `_WAIT_REAL_TERMINAL_STATES` / `_WAIT_TERMINAL_STATES` /
  `_WAIT_MAX_*` / `_WAIT_POLL_INTERVAL` hoisted to module level on
  `coordinator_client` so `session.py` no longer reads a class
  internal; ClassVar aliases kept for back-compat.
- `ConsoleCoordinatorUI` state/rename observers typed as
  `Callable[[str], None] | None` instead of `Any`.
- SSE error renderer verified end-to-end (coordinator.js already
  handles `case "error"` → `appendText`; no code change).

Tests
-----

- 26 new test cases: 11 for `_diff_since` + `cleanup_dead_task_child_refs`,
  15 for `cluster_ws_live_bulk` + `coordinator_metrics`.  Full suite
  4345 passing (4319 base + 26 phase-6).

Gate: ruff + mypy + pytest -m "not live" (4345 passed) all clean.

* fix(coordinator): address PR #381 review feedback

Copilot comments:

- Cross-tenant aggregate leak in coordinator_metrics — the new
  count_workstreams_by_state / count_workstreams_since aggregates
  took parent_ws_id but not user_id, so a non-admin caller could
  observe drifted / forged child rows that share parent_ws_id with
  their coord but whose user_id drifted to another tenant.  The
  404-mask on coord ownership (_resolve_coordinator_or_404) is the
  primary defense; this is defense-in-depth inside the aggregate
  queries.  Pass filter_user_id (None for admin, caller_uid for
  non-admin) — matches coordinator_children's tenant-push-into-SQL
  pattern.

- wait_for_workstream(since=…) docstring + tool schema were stale —
  claimed "A missing entry counts as changed on first observation"
  but the implementation ignores ws_ids absent from since_map to
  prevent a disjoint since dict from silently exiting on tick one.
  Rewrote both doc sites to match the actual semantics: only ws_ids
  present in `since` participate in the diff-exit check; others fall
  back to the normal mode-based completion condition.

- WAIT_TERMINAL_STATES comment drift — the comment claimed it was
  "used by the resolved-count summary" but the summary counts only
  WAIT_REAL_TERMINAL_STATES (denied is a rejection, not a
  resolution).  Rewrote the comment to describe the real usage:
  mode='any' pure-denied short-circuit + mode='all' settle check.

github-code-quality (CodeQL):

- coordinator.js — dropped the dead typeof _renderWaitIndicator
  guard + the typeof activeWaits guard around the reconnect clear.
  Both symbols are defined in the same IIFE; the onopen handler
  fires strictly AFTER IIFE execution finishes, so the guards
  always evaluated to true.  Removing the dead branching also
  removes a CodeQL nit.

- Protocol-method `...` statements — the bot flagged the three new
  methods (count_workstreams_by_state / count_workstreams_since /
  count_skill_versions) with "statement has no effect".  Left as
  `...` to match the file's universal convention (216 `...` bodies
  / 0 `pass` bodies pre-change); swapping just the new methods to
  `pass` would introduce inconsistency with every other Protocol
  method.  Resolved as non-actionable.

Test: new test_metrics_tenant_filter_excludes_forged_cross_tenant_child
covering the aggregate-query tenant filter with both a legitimate
alice child and a forged bob child sharing parent_ws_id.  Non-admin
alice sees 1; admin sees 2.

Gate: ruff + mypy + pytest -m "not live" (4346 passed) all clean.
2026-04-18 03:49:56 -07:00
Patrick Buckley 334edbd580 fix(server,console): kind filter on saved-workstreams + closed coords on landing (#380)
* fix(server,console): kind filter on saved-workstreams + closed coords on landing

Two independent bugs folded into one hotfix:

1. Coordinators leaking into the interactive UI's "saved workstreams"
   sidebar.  ``list_workstreams_with_history`` (SQLite + postgres) was
   kind-agnostic — every coordinator row with conversation history came
   back alongside interactive rows, and ``list_saved_workstreams``
   serialized them uniformly with no kind field so the interactive UI
   rendered coordinators as regular interactive entries.

   Fix: add optional ``kind: WorkstreamKind | str | None = None`` kwarg
   on ``list_workstreams_with_history`` (storage protocol + both
   backends + the ``turnstone.core.memory`` helper).  Pass
   ``kind=WorkstreamKind.INTERACTIVE`` from the /v1/api/workstreams/saved
   handler so the interactive surface only sees interactive rows.
   Default ``None`` preserves legacy all-kinds behaviour for any
   other caller that wants both.

2. Closed coordinators vanish from the console landing page.
   ``_coordinator_rows`` in console/server.py built dashboard rows
   exclusively from the in-memory ``CoordinatorManager`` registry,
   which pops rows on ``close()``.  The persisted storage row stays
   (state='closed') but never reached the landing-page poller at
   /v1/api/cluster/workstreams?node=console.

   Fix: two-lane merge in ``_coordinator_rows``.  The in-memory lane
   (manager) stays authoritative for live session state (model /
   model_alias / current state / tokens).  A new persisted lane queries
   ``storage.list_workstreams(kind=COORDINATOR, user_id=uid, limit=200)``
   and appends rows NOT already in the in-memory set — surfacing
   closed / error / deleted coordinators so the operator can still
   see them on the landing page.  Ownership semantics unchanged —
   non-admin callers only see their own tenant, admin-bypass via
   admin.users/admin.roles honored on both lanes, empty-string
   defense-in-depth matches _check_row_owner_or_404.

Tests:
- tests/test_storage_sqlite.py — two new tests: kind filter excludes
  coordinators from the history list; string form of kind accepted
  (matches the memory.py forwarding shape).
- tests/test_coordinator_endpoints.py — four new tests:
  - closed coordinators from storage surface alongside active ones.
  - in-memory row wins on ws_id dedup (live state authoritative).
  - persisted rows respect tenant filter (non-admin, admin bypass).
  - orphan rows (empty user_id) never leak to empty-sub callers.

Gate: ruff + mypy + pytest -m "not live" (4315 passed) all clean.

* fix(server,console): address Copilot review on PR #380

Three review comments folded in:

1. Tenancy leak in /v1/api/workstreams/saved — the handler called
   list_workstreams_with_history without a user_id filter, so any
   authenticated user could see every other user's saved workstream
   aliases / titles / names.  Fix:

   - Add ``user_id: str | None = None`` kwarg to
     list_workstreams_with_history on the protocol + both backends
     (SQLite + postgres).  Pushes the filter into SQL.
   - memory.py helper forwards the kwarg.
   - /v1/api/workstreams/saved reads ``_auth_scopes(request)``: a
     service-scoped caller gets cluster-wide visibility (None), a
     non-service caller with a blank ``sub`` returns an empty list,
     otherwise the SQL filter is scoped to the caller's uid.  Matches
     the _visible_workstreams pattern used on /workstreams and
     /dashboard.

2. Loose type annotation on the memory.py helper — ``kind: Any``
   tightened to ``WorkstreamKind | str | None`` so mypy catches
   invalid callers.  WorkstreamKind was already imported in the
   module.

3. Brittle positional indexing in _coordinator_rows persisted-rows
   lane — ``row[10]`` for user_id encoded a column offset that would
   silently corrupt the projection on any future SELECT reorder.
   Drop the test-double fallback entirely; the storage-protocol
   contract already requires SQLAlchemy Row with _mapping, and every
   real caller (SQLite + postgres) provides it.

Tests:
- test_server_authz.py TestSavedWorkstreamsTenantScoping — four new
  regression tests covering: non-service caller sees only own rows,
  service scope sees cluster-wide, blank-sub non-service returns
  empty, and coordinator rows excluded even for service callers.

Gate: ruff + mypy + pytest -m "not live" (4319 passed) all clean.
2026-04-18 03:10:19 -07:00
Patrick Buckley c17eddbbd8 fix(console): service scope on collector token + surface upstream 4xx (#379)
* fix(console): service scope on collector token + surface upstream 4xx

CRITICAL: the console's ClusterCollector ServiceTokenManager was
configured with only frozenset({"read"}) scope, but every upstream
node's /v1/api/events/global hard-gates on "service" scope (added in
PR #375 for cross-tenant authz hardening).  Every console→upstream
SSE connect 403'd, the collector never populated node state, and the
failure was silent — node health, idle workstreams, and interactive-
kind workstream rows all disappeared from the console dashboard with
no user-visible error.  The only surface was a log.debug line in the
collector's _node_sse_task that operators had to opt into via DEBUG
logging or browser DevTools.

Fix:

- Add "service" to the collector_token_mgr scopes
  (turnstone/console/server.py).  Matches the proxy_token_mgr (which
  already has it) and the existing cli / admin / channel-gateway
  service tokens.  Restores /v1/api/events/global SSE subscription
  and /v1/api/dashboard visibility (which silently tenant-filters
  non-service callers to zero rows).

- Upgrade the 4xx path in _node_sse_task to log.warning with the
  status code + 200-char body preview, so configuration-level
  failures (scope misconfig, JWT secret mismatch, expired token)
  show up in operator logs instead of being masked by the generic
  except-block debug line.  Keep transient network errors
  (CancelledError, ConnectError) at debug so the log doesn't flood
  during brief node restarts.

- Add reachable_reason field to NodeSnapshot + surface via
  get_nodes / get_node_detail / get_snapshot (and the browser's
  buildNodeInfoFromSnapshot).  Operators now see the failure cause
  on the cluster node list without tailing the log.  Cleared on
  successful reconnect in _apply_snapshot.

- Test coverage: test_server_authz.py TestGlobalEventsServiceGate
  gains a positive-path test asserting that a token with exactly
  the collector's scope set ({"read", "service"}) is accepted by
  /v1/api/events/global.  Locks in the scope contract so any future
  rename breaks the test before it breaks the dashboard.

Gate: ruff + mypy + pytest -m "not live" (4309 passed) all clean.

* fix(console): address Copilot review on PR #379

Two review comments folded in:

- collector.py — bounded body read for 4xx SSE error previews.  The
  prior ``await source.response.aread()`` buffered the entire
  upstream error body into memory just to log a 200-char preview; a
  malicious / oversized upstream response (HTML error page, proxy-
  generated body) could have forced the collector to download an
  arbitrary amount of bytes.  Iterate ``aiter_bytes()`` and stop once
  the preview cap (256 bytes, ~200 chars after UTF-8 decode) is
  satisfied.

- test_server_authz.py — tighten the service-scope positive test.
  The prior ``assert resp.status_code != 403`` could pass on
  unrelated 500s AND left an SSE stream open indefinitely.  Send
  ``?expected_node_id=definitely-wrong-node-id`` so the handler
  passes the scope gate, hits the post-auth node-identity check, and
  returns 409.  Now ``assert resp.status_code == 409`` proves the
  scope contract precisely and terminates the request immediately.

Gate: ruff + mypy + pytest -m "not live" (4309 passed) all clean.
2026-04-18 02:48:57 -07:00
Patrick Buckley 553d73109b feat(coordinator): phase 5 — harness-test polish + wait_for_workstrea… (#378)
* feat(coordinator): phase 5 — harness-test polish + wait_for_workstream + judge fix

Closes the bug list surfaced by the 2026-04-17 coordinator harness test
plus the post-phase-4 wait_for_workstream ask, and folds in three
adjacent cleanups that landed in the same window.  Tightens defense-in-
depth on the model-invoked mutating ops, fixes the LLM judge silent
no-op, kills the inspect-poll token burn, and rounds out a handful of
observability / docstring / spec gaps.

The session-factory pre-resolve at console/session_factory.py and
server.py was rewriting `judge.model` from an alias (e.g. `judge-mini`)
to the resolved underlying id (e.g. `gpt-5-mini`).  IntentJudge then
checked `model_registry.has_alias(config.model)`, found nothing, and
fell back to the SESSION's provider/client with that bare model id —
silent `llm_fallback / "did not return a verdict"` whenever the
coordinator and judge alias resolved to different providers.

Pass the alias through unchanged; IntentJudge's existing alias-
resolution path picks up the matching client + provider.  Validate
the alias exists so an obvious typo still surfaces, but don't replace
the model field.

Regression: `test_alias_uses_registry_provider_not_session_provider`
constructs an alias whose provider differs from the session's and
asserts the judge picks up the alias's provider/client/model;
`test_coordinator_tool_call_returns_llm_verdict_not_fallback` asserts
the verdict tier is `llm` (not `llm_fallback`) on the happy path.

New `cancel_workstream` tool (approval required, primary_key=ws_id) —
cancels in-flight generation, unblocks any pending approval / plan,
moves the child to idle, leaves the row in storage so a fresh
send_to_workstream lands cleanly.  Re-uses the existing
`/v1/api/route/cancel` route + `route.cancel` audit namespace; no
new server endpoint.

`CoordinatorClient.cancel/close_workstream/delete/send` now enforce
a tenant guard inline (`_is_own_subtree`) — only the coordinator
itself or one of its own children is targetable.  Foreign ids return
the same 404-shape inspect/wait_for_workstream use, so the model
can't distinguish foreign from missing (no existence oracle).
Defense-in-depth — the upstream node enforcement is the perimeter,
this is the second line.

`list_workstreams` advertised `state="deleted"` and an
`include_closed=true` that surfaced deleted rows.  Hard-deletes
cascade the workstream + conversation rows out of storage, so
deleted is unreachable in normal operation.  Doc-only fix; the
synthetic-test path that registers `state="deleted"` rows still
works (terminal-state filter still excludes them via
`_terminal_states = {"closed", "deleted"}` in list_children).

Documented that the 120s service-registry heartbeat window means a
node returned by list_nodes can drop out before a follow-up
`spawn_workstream(target_node=…)` lands — the spawn fails with "No
available node for routing" rather than falling back.  Two-line
clarification on each tool.  No code change (a code fallback is a
bigger discussion deferred to 1.6).

`close_workstream` accepts `reason`; the upstream server handler now
persists it to `workstream_config.close_reason` (capped at 512 BYTES,
sliced on UTF-8 not code points so a CJK / emoji-heavy payload can't
4× the documented budget).  `CoordinatorClient.inspect()` reads it
and surfaces as `close_reason` in the result dict — only for
terminal-state children (closed/error/deleted) so the live-child hot
path doesn't pay a per-inspect DB round-trip.

Tests: server-side persistence covers success / no-reason /
length-cap / non-string / storage-failure / multi-byte-utf8 paths;
client-side surface covers terminal vs. live workstreams.

For idle children whose node-dashboard live counter is 0 (the live
block only surfaces in-flight token counters), fall back to
`SUM(prompt_tokens + completion_tokens)` from `usage_events` so the
inspect output reflects cumulative spend.

New `storage.sum_workstream_tokens(ws_id) -> int` on the protocol +
both backends.  The fallback is folded INTO `_fetch_cluster_live` so
the merged live block (with persisted total applied) is what gets
cached — back-to-back inspects of an idle child amortize through
the existing 2s LRU cache instead of each firing a fresh aggregation.

`CoordinatorClient.list_skills()` now projects `allowed_tools` per
skill — capped at 20 with a `+N more` sentinel so a skill that
whitelists a wide MCP surface doesn't bloat the per-row payload.
Reads the existing `prompt_templates.allowed_tools` column; no
storage change.  Coordinators no longer have to guess what tools a
skill brings.

`route_create` now sets `routing_strategy: "hash_ring" | "target_node"
| "resume"` on the spawn response so the coordinator's spawn
response (and the `spawn_workstream` tool output) carries why a
given node was chosen.  3 lines + 3 covering tests in
test_console_routing_proxy.py.

New coordinator tool `wait_for_workstream(ws_ids, timeout=60,
mode='any'|'all')` that absorbs the wait into a single tool call —
the model sees one call + one result regardless of how long the
children take.  Kills the busy-poll inspect loop that burned 20+
turns on a 3-child fan-out.

Storage-poll loop with batched primitives —
`get_workstreams_batch` + `sum_workstream_tokens_batch` issue exactly
two storage calls per tick regardless of N.  At the cap (32 ws_ids /
600s / 0.5s tick) that's ~2400 round-trips for a full wait, down
from ~38k under the naive per-id shape.

Validation single-source-of-truth: the client owns mode whitelist,
ws_ids dedup + cap, timeout coerce + clamp.  The session preparer
is a thin pass-through that builds the header + dispatches; bad
input surfaces at exec time as a tool error via `result.get("error")`.

Tenant-isolation collapse: missing-row and cross-tenant cases both
return `state="denied"` so wait can't be used as an existence oracle
(matches the 404-mask contract `inspect` uses).

Prompt-side: tools_coordinator.md adds a `wait_for_workstream`
pattern + an explicit "PREFER wait_for_workstream OVER a loop of
inspect_workstream" line in the workflow-shape section.

Replaces the quote-bracketed substring LIKE/ILIKE pattern with proper
JSON-array containment.  The previous shape effectively did
`LOWER(tags) LIKE '%"<lower-tag>"%'`, which broke for tag values
containing `"` (the JSON encoder escapes it to `\"` and the literal-
substring search misses), `\` (encoded as `\\`), or non-ASCII
characters that the encoder rendered as `\uXXXX`.  Also exposed a
small spoofing surface — `tags=["foo\","bar"]` would have matched a
query for `bar`.  Real-world tag values are alphanumeric+dash today
so it hadn't fired in production, but the fix is small.

- SQLite: `EXISTS (SELECT 1 FROM json_each(prompt_templates.tags)
  WHERE lower(value) = lower(:tag))` (JSON1 extension; SQLite 3.38+).
- PostgreSQL: `EXISTS (SELECT 1 FROM jsonb_array_elements_text(
  prompt_templates.tags::jsonb) AS jat(elem) WHERE lower(jat.elem) =
  lower(:tag))`.

Three new tests prove the substring pattern was broken for
quoted / backslash / unicode tag values; the existing case-fold +
wildcard tests continue to pin the contract.

Phase 1 added the coordinator workstream API; phase 2 added only
`/open` to the OpenAPI catalog and missed every other coordinator
endpoint plus phase 3's `/children`, `/tasks`, and the
`/cluster/ws/{ws_id}/detail` aggregator.  SDK consumers + operators
browsing `/docs` couldn't discover the surface.  Doc-only addition:
12 endpoints + 9 new Pydantic models, all under the `Coordinator`
OpenAPI tag so /docs groups them together.

Sidebar re-fetches `GET /tasks` on every `task_list` `tool_result`
SSE event.  A model that runs `add → list` (or any back-to-back
mutation pair) double-fetches the same envelope.  Coalesced into
one fetch per 150ms window via a new `loadTasksDebounced` wrapper;
direct UI actions (refresh button, page load) keep calling
`loadTasks` directly so user clicks aren't delayed.

- `ruff check turnstone tests` — clean
- `mypy turnstone` — clean (157 source files)
- `pytest -m "not live"` — 4284 passed, 3 deselected (was 4226 on
  main; +58 new tests across coordinator client, tools, judge,
  storage, console routing proxy, server close-handler,
  storage_skills_filtered, OpenAPI catalog, server close-reason
  persistence)
- New tools added: 2 (cancel_workstream, wait_for_workstream) —
  TOOLS count 28 → 30; coordinator subset 9 → 11; auto_approve adds
  wait_for_workstream; primary_key adds cancel_workstream
- New OpenAPI endpoints: 12 (every phase-1/2/3 coordinator route +
  the cluster-inspect aggregator)
- New storage protocol methods: 3 (sum_workstream_tokens,
  sum_workstream_tokens_batch, get_workstreams_batch)

All phase 1 / 2 / 3 / 4 invariants preserved: COORDINATOR_TOOLS /
INTERACTIVE_TOOLS disjoint; coordinator sessions have no MCP surface;
list-style tools return {items, truncated}; route-proxy emits
route.<action> audit on 2xx; 404-mask on ownership failures; tenant
filters pushed into SQL; per-coordinator JWT carries scope context.

* fix(coordinator): address Copilot review on PR #378

Three valid Copilot findings on the wait_for_workstream surface:

1. ``wait_for_workstream.json`` description claimed the tool returns a
   top-level mapping ``ws_id -> {state, tokens, updated}`` plus
   elapsed/complete/mode at the same level, but the actual shape is
   ``{results: {ws_id: {...}}, elapsed, complete, mode}``.  Description
   now matches the implementation.  Also adds ``deleted`` to the
   advertised terminal-state list (it's in ``_WAIT_REAL_TERMINAL_STATES``;
   the doc and runtime now agree).

2. ``CoordinatorClient.wait_for_workstream`` docstring listed
   ``idle / error / closed`` as the real terminal set but the constant
   includes ``deleted``.  Same fix — list ``deleted`` with a parenthetical
   noting it's unreachable in normal operation (hard-delete cascades the
   row).

3. Storage protocol docstring math: ``sum_workstream_tokens_batch``
   claimed "from ~38k to ~1200" round-trips per wait at the cap, but
   ``wait_for_workstream`` issues TWO storage calls per tick
   (``get_workstreams_batch`` + this one), so 1200 ticks × 2 = ~2400.
   Updated to "~2400" with the math spelled out.

Also a clean rebase onto today's main (PR #377 — the rebalancer node_id
snapshot doc — landed since phase 5's last push).  Single conflict in
``inspect_workstream.json`` resolved by keeping both notes (rebalancer
node_id binding semantics + the new ``close_reason`` surface from phase
5); ``spawn_workstream.json`` auto-merged.

The github-code-quality bot also flagged three items on
``_protocol.py`` asking to replace ``...`` with ``pass`` in Protocol
method bodies.  Refuted: ``...`` is the canonical PEP 544 idiom for
Protocol method bodies and the rest of the file uses it consistently.
The bot's lint rule misfires for ``Protocol`` classes.

Verification:
- ``ruff check turnstone tests`` clean
- ``mypy turnstone`` clean (158 source files)
- ``pytest -m "not live"`` — 4308 passed, 3 deselected (no test count
  change; pure doc/comment edits)
2026-04-17 23:56:10 -07:00
Patrick Buckley b0a040c8fa docs(tools): clarify node_id snapshot vs current-binding semantics (#377)
Phase 3 fixed spawn_workstream's response to return the storage-
authoritative node_id at spawn time, but neither tool description
mentioned that the cluster rebalancer can migrate the workstream to
a different node afterwards.  A coordinator that cached the
spawn-time node_id for a long-running callback would silently dispatch
to a node that no longer owns the workstream.

- spawn_workstream: ``node_id`` is a POINT-IN-TIME snapshot at spawn;
  re-read with inspect_workstream when you need the current binding.
- inspect_workstream: ``node_id`` is the CURRENT (storage-authoritative)
  binding; reflects any rebalancer migration that happened since spawn.

Pure description edit — no schema or runtime change.
2026-04-17 23:25:45 -07:00
Patrick Buckley 3bdcf9870e fix(server): close review cleanup items from PRs #374 / #375 review (#376)
Third and final PR of the retrospective-review series.  Addresses the
remaining bug / perf / doc findings from the original multi-stage review
plus the three inline comments left on #374 and #375.

From the original review:

- bug-3: delete_workstream now nulls out parent_ws_id on every child
  row before dropping the target — previously, deleting a coordinator
  left orphaned parent_ws_id pointers and list_workstreams(parent_ws_id=
  <deleted>) kept returning ghost-parented rows.  Fix lives at the
  storage edge so both SQLite and PostgreSQL benefit without a schema
  migration.
- perf-1 / perf-2 / perf-3: new migration 041 drops the low-cardinality
  idx_workstreams_kind outright, rebuilds idx_workstreams_parent as a
  partial index (WHERE parent_ws_id IS NOT NULL) to halve its btree,
  and uses CREATE INDEX CONCURRENTLY on postgres so the rebuild
  doesn't take ACCESS EXCLUSIVE on populated tables.  Dialect-guarded;
  sqlite path is a straight partial CREATE INDEX.
- perf-5: _rebuild_children_from_storage bumps its limit sentinel to
  10_000 and logs a warning when the cap is hit instead of silently
  truncating the tail on every console cold-start.
- q-2: turnstone.core.memory.list_workstreams wrapper deleted (zero
  live callers; PR #374 kept it forward-compatible with the new
  kwargs as a stepping stone).
- q-5: migration 039's docstring now warns operators that downgrade
  drops parent_ws_id irreversibly and notes the 041 dependency.
- q-7: GET /v1/api/workstreams row shape now includes kind +
  parent_ws_id to match /v1/api/dashboard; the Pydantic
  WorkstreamInfo schema follows so SDK consumers see the same fields.

Inline review comments:

- #374 (copilot): console/server.py::coordinator_children now pushes
  user_id into the SQL filter for non-admin callers, so forged /
  migration-era rows with matching parent_ws_id but a different
  owner can't leak through.  Admins bypass the filter — they're
  expected to see the full subtree.
- #375 (copilot, delete handler): storage.get_workstream(ws_id) for
  the audit snapshot moved inside the try: block so a transient DB
  error surfaces through the endpoint's redacted 500 handler instead
  of an unhandled exception.
- #375 (copilot, _require_ws_access): added optional mgr= kwarg —
  when the workstream is live in the in-memory manager, trust its
  cached user_id instead of round-tripping storage.  In-memory-only
  handlers (approve / plan / cancel / command / close / events_sse /
  refresh-title / set-title) pass mgr= so they stay functional
  during transient DB outages and skip one query on the hot path.
  Storage-backed handlers (/delete, /open) omit mgr= and keep the
  storage path for persisted-but-not-loaded rows.

Tests:

- tests/test_workstream_kind.py adds regression tests for the cascade
  null-out on delete and the new user_id SQL filter.
- tests/test_workstream_endpoints.py updated so the title-handler
  tests exercise the in-memory fast path (MagicMock manager returning
  None falls through to storage; explicit ws.user_id set where the
  mock ws is used).

Lint (ruff), typecheck (strict mypy), pytest -m 'not live' all green
(4209 passing).
2026-04-17 22:42:46 -07:00
Patrick Buckley 294d6f5766 fix(server): close cross-tenant authz gaps on interactive-ws handlers (#375)
Second of three PRs addressing the retrospective review of the
turnstone-server interactive-kind feature.  The first (PR #374) put
the structural pieces in place — WorkstreamKind enum + user_id
kwarg on the storage protocol.  This PR uses them to close the
handler-level ownership gaps that shipped under the prior design.

- sec-1: approve / plan_feedback / cancel_generation / command now
  call _require_ws_access before touching the target UI.  Previously
  any authenticated user could resolve pending tool-approvals on
  another tenant's workstream — RCE-adjacent because the attacker
  could approve destructive operations the victim would have denied.
- sec-2: /v1/api/workstreams/{ws_id}/delete now gates on ownership
  AND writes a workstream.deleted audit event.  Previously any
  authenticated user could destroy any other tenant's workstream,
  conversations, and attachments in one call with no tamper-evident
  trail.
- sec-3: /v1/api/events (per-ws SSE) gates before _register_listener
  so non-owners can't subscribe to another tenant's message / tool /
  approval stream.
- sec-4 / sec-5: /v1/api/workstreams and /v1/api/dashboard filter
  to the caller's tenant view via a new _visible_workstreams helper;
  service-scoped tokens (cluster / routing proxy) keep the full view.
- sec-6: /v1/api/events/global requires service scope.  The global
  snapshot carries cross-tenant workstream inventory and was never
  intended for end-user browsers.
- sec-7: /v1/api/workstreams/{ws_id}/open verifies the caller is
  the stored owner (or holds service scope) before rehydrating.
  Returns 404 on mismatch — existence isn't enumerable by response
  code.
- sec-8 / sec-9: /workstreams/close, /refresh-title, /title all gate
  on ownership.  Cross-tenant close aborts the victim's running
  generation; cross-tenant rename is a phishing / denial-of-use
  vector in list / dashboard responses.
- sec-11: workstream.created / .deleted / .closed / .opened now
  land in the audit_events table with kind + parent_ws_id detail,
  so forensic review can reconstruct lifecycle even after the row
  is gone.
- q-4: new tests/test_server_authz.py covers every gate above via
  TestClient, plus the PR #1 HTTP-boundary kind-validation branches
  that had no regression coverage (coordinator / unknown-kind / 400,
  cross-tenant parent_ws_id / 403, non-interactive open / 400).
- q-3: test_workstream_kind.py now uses the conftest storage fixture
  so it runs against both SQLite and PostgreSQL under
  --storage-backend=postgresql, closing the sqlite↔postgres drift
  risk the prior review flagged.  Added storage-edge ValueError and
  user_id SQL filter tests alongside.

Tests, lint (ruff), typecheck (strict mypy) all green.  Stacked on
PR #374 — merges after that lands.
2026-04-17 22:22:03 -07:00
Patrick Buckley 37ed6bbf5b feat(core): WorkstreamKind enum + list_workstreams user_id filter (#374)
Foundation PR for the multi-stage-review follow-up.  Introduces a
single source of truth for workstream kind values and pushes tenant
scoping into the storage protocol so list callers can't forget to
filter client-side.

- WorkstreamKind(StrEnum) replaces bare "interactive" / "coordinator"
  literals across 17 production modules.  Strict mypy narrows every
  internal call site; raw strings still work at wide boundaries
  (HTTP body, DB row) via WorkstreamKind(raw) parse at the edge.
- StorageBackend.list_workstreams(..., user_id=None) adds a SQL-level
  WHERE user_id = :user_id gate on both sqlite and postgres impls.
  Memory wrapper forwards the new filters.
- register_workstream now validates kind at the storage edge so SDK /
  restore / internal callers can't silently corrupt the NOT NULL
  column with empty / mis-cased / unknown values.
- WebUI.__init__ normalizes empty-string parent_ws_id to None, matching
  the storage-edge and WorkstreamManager invariants.
- POST /v1/api/workstreams/new parses body["kind"] through the enum
  and returns 400 on unknown kinds instead of silent coercion.

Absorbs bug-1, bug-2, bug-4/q-6, q-1, q-8, and partial q-2 (wrapper
signature forwards the new filters; full deletion of the unused
wrapper stays in the cleanup PR).
2026-04-17 22:10:57 -07:00
Patrick Buckley d3f6514e11 feat(ui): phase 4 — chat-UX unification + coordinator-first console landing (#373)
* feat(ui): phase 4 — chat-UX unification + coordinator-first console landing

Phase 4 unifies the three turnstone UIs (server-node chat, console
dashboard, coordinator page) around a shared design-system layer,
promotes coordinator sessions to first-class citizens on the console
landing, and folds the chat-view itself onto a shared vocabulary so
the two chat pages no longer reinvent messages / approvals / composer /
header / sidebar chrome from scratch.

## Shared static consolidation

- turnstone/shared_static/renderer.js — consolidates the two copies
  (ui/static/ + console/static/coordinator/) into one.  Adds
  streamingRender / streamingRenderFinalize helpers with
  requestAnimationFrame coalescing + per-element buffer cache so both
  chat views re-render the streamed markdown smoothly without the
  prior "plain-text → final pop" on the coordinator page and without
  thrashing renderMarkdown + DOM replacement faster than the paint
  cycle.  renderMarkdown stays the trust boundary for innerHTML
  assignment (escapeHtml internal); postRenderMarkdown (syntax
  highlighting, mermaid, KaTeX) is deferred to finalize.
- turnstone/shared_static/ui-base.css — flat form-control + button +
  state-glyph + pill + panel vocabulary on top of base.css.  Sizes in
  px to match the 11/12/13px scale used elsewhere.  Namespace rubric
  documented inline (.ui-* shared controls; .dash-* dashboard legacy;
  .ts-* chat vocabulary; page-local stays unprefixed).
- turnstone/shared_static/chat.css (new) — chat-view component
  vocabulary: .ts-msg (user / assistant / reasoning / tool / error /
  info), .ts-msg-actions floating toolbar, .ts-approval (inline +
  batch layout hooks sharing a visual language), .ts-verdict-badge,
  .ts-composer shell, .ts-header shell, .ts-sidebar shell.  Mobile +
  reduced-motion covered.

## Interactive server UI migration

- turnstone/ui/static/app.js — dual-class adoption of .ts-msg + .ts-
  approval + .ts-composer + .ts-msg-actions alongside existing class
  names so feature-specific rules (.msg-user-text, .msg-queued,
  .msg-editing, .msg-action-btn toolbar, attachment chips, verdict
  details, media embeds, plan inline) keep working while the shared
  chat.css baseline takes over padding / border / typography.
- Left-aligned user messages: .msg-user loses align-self: flex-end
  and .msg-assistant loses align-self: flex-start.  Both roles now
  render as single-column blocks distinguished by left-border colour
  (amber for user, neutral for assistant, dashed for reasoning,
  mono + code-bg for tool, red for error) per the locked design.
- style.css trimmed: .msg / .msg-info / .msg-error baseline rules
  dropped (chat.css provides); all other feature rules intact.
- index.html links /shared/chat.css and tags the header with
  .ts-header + .ts-header-title.

## Coordinator page migration

- coordinator.js appendMsg drops the visible .role-label <div> per
  the hybrid no-labels design, preserving the role text on
  data-ts-role + aria-label so screen readers and SSE dedup-by-call-
  id continue to see meaningful labels.  Adds .ts-msg + .ts-msg--*
  variants + .ts-msg-body onto the existing .coord-msg / .coord-body
  elements.
- coordinator/index.html adopts .ts-header, .ts-header-title,
  .ts-header-spacer, .ts-header-status on the header; .ts-approval +
  .ts-approval--batch on the pinned bar with .ts-approval-btn
  variants on the buttons; .ts-composer + .ts-composer-input +
  .ts-composer-send on the composer; .ts-sidebar + .ts-sidebar-
  section + .ts-sidebar-section-heading on the children + tasks
  sidebar.  Inline <style> pared from ~300 to ~130 lines — only
  genuinely coordinator-specific layout (flex wiring, sidebar list
  rows, mobile accordion breakpoint) remains.
- Nits fixed along the way: .ch-row .glyph-thinking recoloured cyan
  to match the shared .ui-glyph vocabulary; .task-row .status-done
  lost its 0.7 opacity (colour already signals done; opacity
  reduced contrast for no gain).

## Console landing + admin panel redesign (phase 4 scope-expansion)

Replaces the node-list-first console landing with a coordinator-first
layout on a new #view-home pane:

- #coord-composer-panel — persistent "Start a new coordinator task"
  composer (textarea + optional name + skill dropdown + submit).
  Permission-gated on admin.coordinator (same rule the +coordinator
  header button uses).  Pre-probes GET /v1/api/coordinator on init
  and after login (bug-1 fix) so a 503 (no coordinator.model_alias
  resolvable) surfaces as a remediation banner linking to Admin →
  Models instead of failing on submit.  Probe gates on r.ok instead
  of r.status !== 503 (bug-2 fix) so auth/permission errors don't
  incorrectly flip the banner to ready.  Composer and modal share a
  _createCoordinator helper (q-1 fix) — POST + redirect + error-
  handling tail is not forked.
- #active-coordinators — SSE-driven list of kind=="coordinator"
  workstreams, rendered through the shared _renderWsRow helper so
  state glyphs + child-count badges match the existing tree view.
- #cluster-summary-compact — one-line aggregate.  Clicking expands
  into the legacy #view-overview via showOverview() so deep-link
  callers of ?view=overview / ?view=node / ?view=filtered keep
  working unchanged.

View switching consolidated into a _setLandingView helper so every
show* / drillDown* function toggles the four landing panes through
one call path.  Default currentView flipped from "overview" to
"home"; popstate + init history.replaceState land on {view: "home"}.

patchClusterState preserves kind / parent_ws_id / user_id on
ws_created events (phase 3 invariant) so the active-coordinators
list picks up new coordinators immediately without a snapshot
refetch.

Header H1 is now a home link so operators have a single-click path
back to the coordinator landing from any drill-down / admin view.

## Design polish (review pipeline fixes)

- .home-panel-title dropped from 13px/accent to 11px/fg-dim so it
  sits in the same heading tier as .ui-section-heading / .dash-
  header-title / .home-section-title instead of outweighing them
  (dsn-3).
- .home-composer-banner recoloured from amber-on-amber-glow to
  bg-surface + 1px yellow border + fg-bright text + accent link
  with thicker underline (dsn-1).
- .ui-pill--done dropped the 0.75 opacity — colour signals done,
  opacity reduced contrast for no gain (dsn-6).
- .ui-heading fleshed out with --sm/--md/--lg tiers so the utility
  actually conveys size (dsn-12).
- ui-base.css size scale moved from rem to px matching 11/12/13px
  (dsn-2).

* fixup: address Copilot feedback on PR #373

- admin.js: drop the stale `#view-overview` display:none mutation in
  showAdmin.  #view-overview is now nested inside #view-home and
  toggled via the `hidden` attribute; inline display:none here would
  stick after returning to home and suppress the cluster-details
  expand.
- app.js: reword the _renderHomeView token-bucket fingerprint comment
  to match the actual `Math.floor(tokens / 100)` bucketing — the
  prior comment said "thousands / sub-thousand drift".
2026-04-17 18:53:14 -07:00
renovate[bot] d056e375ef chore(deps): update dependency typescript to v6.0.3 (#371)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-04-18 01:36:09 +00:00
Patrick Buckley c397668d21 feat(coordinator): tree-view UI, cluster-wide live inspect, dashboard… (#370)
* feat(coordinator): tree-view UI, cluster-wide live inspect, dashboard grouping — phase 3

Closes out the 1.5 coordinator UX surface: a right-sidebar tree view at
/coordinator/{ws_id} showing spawned children + task list, a new
cluster-wide live inspect endpoint that powers the tree's live badges,
and 2-level dashboard tree grouping that nests spawned children under
their coordinator parent.

## Cluster-wide live `inspect_workstream`

New `GET /v1/api/cluster/ws/{ws_id}/detail` on the console, gated by a
new `admin.cluster.inspect` permission (unassigned to any builtin role;
operators opt in).  Aggregates `storage.get_workstream` with a
short-timeout (2s) HTTP fetch against the owning node's
`/v1/api/dashboard`.  Coordinator-hosted workstreams get their `live`
block from the in-process `CoordinatorManager` instead of a proxy hop.

Response shape `{persisted, live, messages}` — `live: null` on node
unreachability / 5xx / missing-entry with status 200 so the UI can
degrade gracefully without an error state.  Correlation-id masks
unexpected exceptions.  404-masks cross-tenant reads (non-admin
callers see only their own workstreams).

`CoordinatorClient.inspect()` best-effort merges the `live` block onto
its storage snapshot so the model-facing `inspect_workstream` tool
gains a `live` key without any schema change.  Model-facing tool
schema stays identical.

## Tree-view UI

New right sidebar at `/coordinator/{ws_id}` with a 2-level children
tree + the phase-2 task list.

Backend:
- New `GET /v1/api/coordinator/{ws_id}/children` returns
  `{items, truncated}` — identical row shape to the `list_children`
  tool — filtered via `storage.list_workstreams(parent_ws_id=..., kind=None)`.
- New `GET /v1/api/coordinator/{ws_id}/tasks` returns the
  `{version, tasks}` envelope via the shared module-level
  `load_task_envelope` decoder (extracted from `CoordinatorClient`
  so both the tool path and the UI read share corruption semantics).
  Corrupt envelopes return an empty list for UI resilience — the
  `task_list` tool remains the authoritative write + error path.
- `CoordinatorManager` subscribes to the `ClusterCollector`'s
  listener channel from the console lifespan and dispatches filtered
  `child_ws_created / child_ws_state / child_ws_closed / child_ws_rename`
  events onto each coordinator's SSE stream.  Filter authoritative
  on the server via a per-coordinator child-ws_id registry populated
  lazily on `open()` from storage and incrementally on `ws_created`
  events; cleared on `close()` / eviction.  One SSE connection per
  client, no client-side filtering.

Frontend:
- DOM-method-only child-row rendering (no innerHTML of user content).
- State glyph vocabulary (● running / ◐ thinking / ⚠ attention /
  ✗ error / ○ idle) plus text labels — WCAG 1.4.1 carries info in
  both glyph and label.
- Live badges (tokens + pending-approval pip) fetched via
  `/cluster/ws/{ws_id}/detail` with a 5s TTL cache and 250ms debounce
  per child.  One request per state change, not per second.
- SSE child events update in place; renderChildren() re-sorts.
- Mobile (<700px) sidebar collapses to an accordion above the chat
  with a toggle button flipping aria-expanded; a `.highlight` flash
  marks task→child scroll targets; `prefers-reduced-motion` respected.
- Deep-link child rows to `/node/{node_id}/?ws_id=<child>` via
  `<a target="_blank" rel="noopener">` with encodeURIComponent on
  regex-validated ids.

## Dashboard tree grouping

Cluster dashboard rows now group by `parent_ws_id`.  Coordinator rows
(`kind == "coordinator"` or children present) get an expand/collapse
caret (button with `aria-expanded`); collapsed shows "(N children)".
Expanded renders children indented as sibling rows with a left-border
gutter.  Orphaned children (parent missing or closed) render at top
level with a muted "orphan" badge.  Expansion state persisted in
`localStorage` keyed per coordinator ws_id so operator preference
survives reloads.  Coordinator rows deep-link to `/coordinator/{id}`;
node-backed workstreams keep their existing proxy deep-link.

Per-node `ws_created / ws_state / ws_activity` SSE event payloads
gained `parent_ws_id` + `kind` so the collector can propagate them
through its fan-out to browser clients without a second lookup;
`_build_node_snapshot` and `/v1/api/dashboard` rows include the
same.  Coordinators (which don't live on cluster nodes) merge into
`/cluster/workstreams` via a new `_coordinator_rows` helper that
threads them through the collector's `get_workstreams(extra_rows=...)`
parameter — extras share the filter / sort / paginate pipeline with
node-backed rows.

## Tests

- `tests/test_coordinator_endpoints.py` — 19 new cases covering
  children (empty / populated / ownership 404 / admin bypass /
  invalid ws_id / truncation), tasks (empty / round-trip / corrupt /
  ownership), and cluster-inspect (auth gates / 400 / 404 / ownership /
  coordinator self-path / unloaded-live-null / message-limit clamp).
- `tests/test_coordinator_manager.py` — 8 new cases covering registry
  bootstrap on create + open, dispatch for each event type,
  unrelated-parent filtering, shutdown idempotency.
- `tests/test_console.py` — existing `cluster_workstreams` assert
  updated for the new `extra_rows` kwarg.

## Verification

- `ruff check turnstone tests` clean.
- `mypy turnstone` clean.
- `pytest -m "not live"` — 4184 passed, 3 deselected.

* fix(coordinator): race in dispatch + ui_factory kwarg filtering — PR #370 review

Addresses feedback from the GitHub Copilot + code-quality bot review
passes on PR #370.

## Race in _dispatch_child_event ws_created branch

Copilot flagged a TOCTOU where the lock-free read of
``self._active_coords`` (line 912) could see the parent coordinator,
then ``close()`` / eviction pops ``_children[parent]`` + drops the
coord from ``_active_coords`` before we acquire ``_children_lock``,
and then ``setdefault(parent, set())`` resurrects the entry —
leaking the registry key forever and fanning events to a closed UI.

Fix: re-check ``parent in self._active_coords`` inside
``_children_lock``.  The reference swap is still atomic; holding
``_children_lock`` and re-reading the snapshot catches the race
without serializing back through ``self._lock``.

Regression test: create → close → dispatch a ws_created → assert
neither ``_children`` nor ``_active_coords`` regained the entry.

## ui_factory kwarg filtering via inspect.signature

code-quality bot flagged that the previous ``try ui_factory(…, kind=,
parent_ws_id=) except TypeError`` dance fired on every call with
legacy test factories (``lambda wid: WebUI(ws_id=wid)``) — wasteful
and masks real signature mismatches.

Fix: inspect the factory's signature and only pass kwargs it
actually accepts (explicit param name OR ``**kwargs`` absorber).
Keep a conservative ``except TypeError`` fallback for C-callables
and odd signatures ``inspect`` can't introspect.

Copilot also flagged a comment mismatch (the old comment said
"KeyError on **kwargs" — it's ``TypeError``, which is what the code
caught).  The rewritten comment is correct.

## Nit: side-effect in assert

code-quality bot flagged ``assert mgr.close(ws.id)`` in
test_coordinator_manager.py.  Split into two statements.

## Verification

- ``ruff check`` clean.
- ``mypy turnstone`` clean.
- ``pytest -m "not live"`` — 4223 passed, 3 deselected, 0 failed.
2026-04-17 14:38:13 -07:00
Patrick Buckley 8650370790 Feat/coordinator phase2 audit (#369)
* feat(coordinator): audit middleware on routing proxy — phase 2

Adds per-tool-call audit attribution to the multi-node routing proxy
handlers so coordinator → server hops land observable rows in
``audit_events``.  Phase 1 preserved the ``src="coordinator"`` claim
through ``_proxy_auth_headers``'s upstream re-mint; this commit
closes the recording side.  Was the last real security gap from
phase 1 — an enterprise deployment with ``admin.coordinator``
granted got only the three console-side
``coordinator.{create,close,cancel}`` rows; per-tool-call
attribution was missing.

## Action-naming scheme

  route.workstream.create   POST /v1/api/route/workstreams/new
  route.workstream.send     POST /v1/api/route/send
  route.workstream.close    POST /v1/api/route/workstreams/close
  route.workstream.delete   POST /v1/api/route/workstreams/delete
  route.approve             POST /v1/api/route/approve
  route.cancel              POST /v1/api/route/cancel
  route.command             POST /v1/api/route/command
  route.plan                POST /v1/api/route/plan

Action-name conventions documented in ``turnstone/core/audit.py``
module docstring alongside the existing namespaces — the docstring
is now ``<resource>.<verb>`` shaped (non-exhaustive) rather than
trying to enumerate every prefix.

## Recording rules

- ``record_audit()`` fires only on a 2xx upstream response.  4xx/5xx
  are observable via ``_record_route``'s metrics path; doubling the
  audit-events table size for failure rows would dilute signal
  without giving operators much extra value.
- ``detail`` JSON carries ``{src, node_id, coord_ws_id?}`` — ``src``
  lands verbatim from ``auth.token_source`` so non-coordinator
  origins (``"jwt"``, ``"console-proxy"``) also get attribution;
  ``coord_ws_id`` only appears when the inbound JWT carried it.
- Wrapped in ``try/except`` + ``log.debug("route.audit_failed", ...)``
  defence-in-depth.  ``record_audit`` itself is fire-and-forget;
  the outer try guards against a programmer error in the call site.

## Routing-proxy specifics

- ``route_create``: emits at the post-multipart/JSON convergence
  ``if resp.status_code == 200`` block.  Both branches set
  ``audit_ws_id`` correctly — multipart from the query-string ws_id,
  JSON from ``body["ws_id"]`` (post-503-retry) or ``body["resume_ws"]``.
- ``route_proxy``: emits the URL-method-mapped action.  ``ref`` is
  reassigned to ``new_ref`` after a successful 404→cache-refresh
  retry so audit attribution uses the retried node, not the failed
  first node.
- ``route_workstream_delete``: emits on 2xx using the ws_id from
  the request body.
- ``route_attachment_proxy``: out of scope (upstream attachment
  endpoints emit their own ``workstream.attachment.*`` rows;
  auditing here would double-count).

## Tests

16 new tests in ``tests/test_route_proxy_audit.py`` covering:
- Coordinator-origin emission with full detail payload.
- 502 / 400 / 503-retry-final-node-id paths.
- Parametrised method→action mapping for the 6 ``route_proxy`` URLs.
- Plain-JWT origin (no ``coord_ws_id`` in detail).
- Delete handler 2xx + 502.
- Audit-storage exception swallowed (proxied response unchanged).
- ``auth_storage`` absent → no-op (existing route-handler tests
  unaffected).

Verification: ``ruff check`` clean, ``mypy turnstone`` clean
(156 source files), ``pytest -m "not live"`` 4087 passed,
3 deselected (live-backend), 0 failed.

* feat(coordinator): discovery tools and /open parity — list_nodes, list_skills, POST /coordinator/{ws_id}/open

Adds the read-side surface coordinators need to make informed
orchestration decisions plus an explicit rehydration endpoint
matching the server's ``POST /v1/api/workstreams/{ws_id}/open``.

## list_nodes (auto-approved)

``list_nodes(filters={key: value, ...})`` reads ``node_metadata`` via
``storage.filter_nodes_by_metadata`` + ``get_all_node_metadata`` —
one query each, no N+1.  Each row carries its full metadata dict so
the coordinator has both auto keys (``arch`` / ``cpu_count`` /
``fqdn`` / ``hostname`` / ``os`` / ``os_release`` / ``python``;
always present) and operator-supplied user keys (``capability`` /
``region`` / ``tenant`` / ``role``) without a second round-trip.
Tool description enumerates the auto keys explicitly so the model
knows what's always available vs deployment-specific.

Storage stores metadata values as JSON-encoded strings (the write
path in ``server.py`` / ``admin.py`` / ``console/server.py`` all go
through ``json.dumps``).  The client re-encodes filter values
before the stored-text comparison and decodes stored values before
returning them to the model — so ``{"capability": "gpu"}`` is the
natural form the model uses, not ``{"capability": "\"gpu\""}``.
Ints round-trip as ints.

Returns ``{nodes, truncated}``; ``truncated=True`` when the page
was full.

## list_skills (auto-approved)

``list_skills(category?, tag?, scan_status?, enabled_only?, limit?)``
surfaces the skill registry so coordinators can discover worker
profiles.  New storage protocol method ``list_skills_filtered(...)``
on both SQLite and PostgreSQL backends pushes filters into SQL.
``tag`` filter matches against the JSON-array ``tags`` column with
quote-bracketed substring (``%"tag"%``) — quote-safe against
``foo`` vs ``foobar`` collisions on both backends.

Returns ``{skills, truncated}`` with ``name`` / ``category`` /
``tags`` (decoded to list) / ``version`` / ``description`` /
``model`` / ``enabled`` / ``scan_status`` / ``activation`` — the
discovery projection, not the full row.

## POST /v1/api/coordinator/{ws_id}/open

Explicit rehydration endpoint.  Lazy ``GET`` rehydration works for
the UI; this gives SDK callers and operators a way to warm a
coordinator without browsing to it.  Same ownership / 404-on-
mismatch / correlation-id-masked error semantics as
``coordinator_detail``.  Returns ``{ws_id, name, already_loaded?}``.
Registered in ``turnstone/api/console_spec.py`` with a dedicated
``CoordinatorOpenResponse`` Pydantic model so the OpenAPI schema
matches the wire shape.

## Tests

- ``tests/test_storage_skills_filtered.py`` — 8 cases validated on
  BOTH SQLite and PostgreSQL backends (``pytest --storage-backend
  postgresql``).  Covers no-filter ordering, category exact-match,
  tag quote-safety (``"foo"`` matches ``["foo","bar"]`` but not
  ``["foobar"]``), scan_status, enabled_only, limit, AND semantics,
  empty result.
- ``tests/test_coordinator_client.py`` — 11 new cases covering
  node/skill shape decoding, JSON-encoded filter round-trip (the
  ``"gpu"`` vs ``'"gpu"'`` case), int filter encoding, truncation,
  no-match empty, no N+1 (``get_prompt_template`` /
  ``get_node_metadata`` call counts asserted zero).
- ``tests/test_coordinator_tools.py`` — 11 new cases for
  ``_prepare``/``_exec`` dispatch, filter type-drop, limit clamping
  (``limit=0`` falls back to 100, negatives clamp to 1),
  truncation-signal summary.
- ``tests/test_coordinator_endpoints.py`` — 8 new cases for
  ``/open``: ``already_loaded`` on in-memory hit, 404 on ownership
  mismatch, lazy rehydrate on miss, admin bypass, unknown ws_id,
  503 on ``coord_mgr`` unavailable, 500 with correlation-id mask on
  factory failure, 503 passthrough on ``ValueError``.
- ``tests/test_workstream_kind.py`` / ``test_tools_schema.py``
  updated to include ``list_nodes`` and ``list_skills`` in the
  disjoint-namespace regression guard and the tool-count check.

Verification: ``ruff check`` clean, ``mypy turnstone`` clean
(156 source files), ``pytest -m "not live"`` 4122 passed, 3
deselected (live-backend), 0 failed.  Postgres backend storage
tests green (``pytest --storage-backend postgresql
tests/test_storage_skills_filtered.py`` 8 passed).

* feat(coordinator): task_list tool — persistent planning state

Adds a coordinator-only ``task_list`` tool persisted on the
coordinator's own ``workstream_config`` row.  Gives coordinators a
scratch surface for work decomposition that survives restarts so the
UI can render planned-vs-done state once the tree view lands.

## Tool surface

``task_list(action, ...)`` with five actions:

- ``list``     auto-approved read.  Returns ``{tasks, truncated}``;
               truncated=True when the list exceeded the 200-row
               page cap.
- ``add``      needs approval.  ``title`` required; optional
               ``status`` and ``child_ws_id``.  Title clamped at
               200 chars.  Capacity cap at 500 tasks — hitting the
               cap is an explicit signal to prune done/blocked rows.
- ``update``   needs approval.  Mutate by ``task_id``; fields
               ``title`` / ``status`` / ``child_ws_id`` optional.
- ``remove``   needs approval.  Drop by ``task_id``.
- ``reorder``  needs approval.  Pass ``task_ids``; validated as an
               exact permutation of the current set (rejects
               partial, extra, or substituted ids — prevents silent
               task loss).

Status enum: ``pending`` / ``in_progress`` / ``done`` / ``blocked``.
``child_ws_id`` links a task to the child workstream spawned for it
(no enforcement; the coordinator owns the relationship).

## Persistence

Stored as a single JSON-envelope value on ``workstream_config`` —
``{"version": 1, "tasks": [...]}``.  No new table; the kanban v2
work will supersede this row via a format migration keyed on
``version``.  ``_save_task_list`` writes only the ``tasks`` key so
concurrent writers to other ``workstream_config`` keys (e.g. the
admin Settings UI updating ``reasoning_effort``) aren't clobbered
by a read-modify-write on the full row.

## Corrupt-envelope safety

A hand-edited or legacy config row that doesn't parse as the
expected shape logs a warning and returns an empty envelope from
``task_list_get``.  Mutators refuse to overwrite corrupt data —
they detect the sentinel and return a clear error so the operator
can inspect or clear the row rather than losing work silently.

## Concurrency

Per-(ws) ``threading.Lock`` cached on the client.  The worker
thread is single-threaded for tool execs so this is mostly
defence-in-depth against future maintenance-script call sites.
Cache never grows beyond one entry per coordinator session because
the scope guard short-circuits foreign ``ws_id`` before the lock
is acquired.

## Malformed-JSON recovery

``_prepare_tool`` fallback-1 regex-extract allowlist extended with
``action`` / ``status`` / ``task_id`` / ``title`` (alphabetized) so
slightly-malformed ``task_list`` calls get the same
self-correction behaviour as the other coordinator tools.

## Tests

- ``tests/test_coordinator_client.py`` — 15 new cases covering:
  fresh-envelope shape, add/get roundtrip, empty-title + invalid-
  status rejection, 200-char title clamp, update by id + missing
  id, remove semantics, reorder permutation validation (partial +
  extra + wrong id + valid), cross-ws scope violation, corrupt-
  JSON read recovery, corrupt-envelope write refusal (all four
  mutators), 500-task capacity cap, workstream_config key
  preservation across ``_save_task_list``.
- ``tests/test_coordinator_tools.py`` — 12 new cases covering the
  dispatch layer: list auto-approved, each mutating action needs
  approval, unknown-action / missing-required-arg errors, list
  returns tasks, page-cap at 200 with truncated signal, add
  dispatches to client, reorder surfaces permutation error,
  remove-not-found.
- ``tests/test_tools_schema.py`` / ``tests/test_workstream_kind.py``
  extend the tool-count + disjoint-namespace + primary-key
  regression guards with ``task_list``.

Verification: ``ruff check`` clean, ``mypy turnstone`` clean
(156 source files), ``pytest -m "not live"`` 4148 passed,
3 deselected (live-backend), 0 failed.
2026-04-16 23:44:28 -07:00
Patrick Buckley e42add1b77 feat(coordinator): coordinator workstream kind — phase 1 (#368)
* feat(coordinator): coordinator workstream kind — phase 1

Adds a new ``kind="coordinator"`` workstream that runs inside the
``turnstone-console`` process (first ChatSession hosted on the console)
with a dedicated tool set for spawning and driving child workstreams.
Supersedes the external ``turnstone-coordinator`` MCP side-car for new
installs; the extension is marked deprecated in
``examples/mcp-cluster-ops/README.md`` but still works on 1.4-and-earlier
clusters.

Phase 1 ships: the workstream class, 6 lifecycle tools, console hosting,
9 HTTP endpoints, per-user audit attribution, and a one-pane web UI at
``/coordinator/{ws_id}``.  Node/skill discovery tools, task-list tool,
tree-view UI, and routing-proxy audit middleware follow in a later PR.

## Schema

Migration 039 adds ``kind`` / ``parent_ws_id`` columns + indexes to
``workstreams``.  Both SQLite and PostgreSQL backends take the new
kwargs on ``register_workstream``; empty-string ``parent_ws_id``
normalises to ``NULL`` at the storage edge.  PostgreSQL uses
``INSERT ... ON CONFLICT DO NOTHING`` to match SQLite's ``OR IGNORE``
and close a pre-existing SELECT-then-INSERT TOCTOU window.
``list_workstreams`` gains optional ``parent_ws_id`` / ``kind`` filters;
new ``get_workstream(ws_id)`` returns the full row (the existing
``get_workstream_metadata`` stays untouched for back-compat).

## Core session + kind routing

- ``ChatSession.__init__`` accepts ``kind`` / ``parent_ws_id`` /
  ``coord_client``.  On ``kind="coordinator"`` it swaps
  ``_tools = COORDINATOR_TOOLS`` and zeros sub-agent tool lists.
- ``Workstream`` dataclass extended with ``user_id`` / ``kind`` /
  ``parent_ws_id``.  Both ``WorkstreamManager`` and the new
  ``CoordinatorManager`` use the same type — no parallel hierarchy.
- ``_SessionFactory`` Protocol + server / cli factory closures thread
  the new kwargs.  ``POST /v1/api/workstreams/new`` rejects
  ``kind != "interactive"`` with 400; ``POST
  /v1/api/workstreams/{ws_id}/open`` refuses coordinator rows so a
  server node can't accidentally rehydrate one.

## Coordinator tool set

Six tools (``spawn``, ``inspect``, ``send``, ``close``, ``delete``,
``list_workstreams``) with a ``coordinator: true`` metadata flag,
scoped to coordinator-kind sessions only.  ``inspect`` and ``list`` are
auto-approved reads; the four mutators need approval.  ``list`` returns
``{"children": [...], "truncated": bool}`` so the model can detect
post-filter under-fill and paginate.

## CoordinatorClient (in-process, sync)

Mutating ops HTTP-POST to the console's own ``/v1/api/route/*`` on the
local bind URL so every existing middleware (auth, rate-limit) runs.
Read ops hit ``storage.list_workstreams`` / ``get_workstream`` /
``load_messages`` directly — the routing proxy doesn't expose
list/inspect paths.  URL paths are a validated constant table (avoids
an httpx ``base_url``-merge trap).  A new
``/v1/api/route/workstreams/delete`` proxy handler joins the existing
route-proxy endpoints.

## Per-session coordinator JWT

``CoordinatorTokenManager`` mints short-lived JWTs with ``sub=<real
user>`` (attribution preserved), ``src="coordinator"``,
``aud="turnstone-console"``, ``coord_ws_id=<ws>`` custom claim.
``_proxy_auth_headers`` preserves ``src`` + ``coord_ws_id`` across the
upstream re-mint so server-side middleware sees coordinator-origin,
not ``console-proxy``.  ``AuthResult.extra_claims`` carries
non-reserved claims through validate→remint; ``create_jwt``'s
reserved-claim set (now including ``nbf`` / ``jti``) is symmetric with
``validate_jwt``.

## Console hosts the ChatSession

- New ConfigStore settings: ``coordinator.model_alias`` (required),
  ``reasoning_effort``, ``max_active`` (default 5),
  ``session_jwt_ttl_seconds``.
- Console lifespan builds a ``ModelRegistry`` +
  ``CoordinatorManager``.  Missing / unresolvable alias returns **503**
  with remediation text — never 500.
- ``CoordinatorManager``: placeholder-slot reservation under lock,
  rollback on factory failure, per-ws_id rehydration lock to serialise
  concurrent lazy-opens, ``max_active`` enforced via ``close_idle``
  eviction semantics.
- ``ConsoleCoordinatorUI`` is a thin ``SessionUI`` implementation — no
  global broadcast, no per-node metrics, shared
  ``_APPROVAL_WAIT_TIMEOUT`` constant across approval + plan paths.
- No eager startup rehydration: persisted coordinator rows load lazily
  on first ``GET /v1/api/coordinator/{ws_id}``.

## Console coordinator API

Nine endpoints under ``/v1/api/coordinator/*`` gated by ``approve``
scope + new **``admin.coordinator``** permission (added to
``_VALID_PERMISSIONS``; not in any builtin role — operators opt in
explicitly).  Ownership failures return **404, not 403** and use
strict equality so empty-owner rows don't leak across tenants.
Correlation-id masking on every factory-raising path
(``coordinator_create`` + ``coordinator_detail`` lazy rehydrate) — no
stack traces to the client.

## Audit attribution

Three console-side events (``coordinator.create`` / ``.close`` /
``.cancel``) with the real creator's ``user_id`` plus
``detail={coord_ws_id, src="coordinator"}``.  No schema migration
required.  Per-tool-call audit across the routing proxy is deferred
(needs either a ``source`` column on ``audit_events`` or
``record_audit`` calls wired into the route-proxy handlers).

## Web UI (``/coordinator/{ws_id}``)

One-pane chat served by the console.  Reuses ``shared_static``
(``base.css``, ``auth.js``, ``theme.js``, ``toast.js``, ``utils.js``,
``kb.js``) and the server UI's ``renderer.js`` pipeline (KaTeX, Mermaid,
highlight.js already bundled).

- SSE to ``/v1/api/coordinator/{ws_id}/events`` with exponential-
  backoff reconnect; status line carries a leading glyph
  (● / ○ / ⚠) so state isn't conveyed by colour alone.
- Renders content, reasoning (dimmed italic
  ``.role-reasoning``), tool_result, approve_request, intent_verdict,
  output_warning.
- Child ws_id references auto-wrap to
  ``/node/{node_id}/?ws_id={child}`` links — both ids regex-validated
  before interpolation, everything else HTML-escaped.
- Non-modal approval bar (``role="region"``) with a batch header
  ("Approve N tool calls"), initial focus on the approve button,
  buttons disabled during the in-flight POST, red-bordered deny.
  ``aria-live`` flips to ``off`` during streaming.
- "New coordinator" button on the dashboard header — permission-gated
  on the UI side, matching the backend 403.
- Mobile composer capped under ``@media (max-width: 700px)``.

## Tests

~120 new tests across 8 files: workstream-kind storage + dataclass
semantics, CoordinatorClient URL map + token minting + storage reads +
truncation signalling, tool prepare/exec dispatch and approval gating,
CoordinatorManager create / rollback / eviction / lazy rehydration +
concurrency, HTTP endpoint auth + 404-on-ownership + 503-on-misconfig,
proxy-auth ``src`` preservation, full lifecycle end-to-end, coordinator
page HTML-injection guard.  ``test_tools_schema.py`` widened to 25
tools (19 existing + 6 coordinator).

Verification: ``ruff check`` clean, ``mypy turnstone`` clean
(156 files), ``pytest`` 4054 passed (5 pre-existing failures unrelated
to this change — confirmed against ``main``).

* polish(coordinator): address PR review + CI + tool-namespace isolation

CI:
- `ruff format`: two files reformatted, matches the in-repo pre-commit config.
- `wheel-completeness`: add `turnstone/console/static/coordinator/*.html` +
  `*.js` to the hatch wheel-include list.  Without this the coordinator UI
  was missing from published wheels.
- `test (3.11/3.12/3.13)` + `test-postgres`: three `TestExecReadImage`
  tests were masking a real bug — my 6 new tool JSONs pushed tool count
  19→25, crossing the default `tool_search.auto` threshold (20), which
  made `ChatSession.__init__` construct a `ToolSearchManager` and cache
  `_cached_capabilities` during init.  Tests that later patched
  `session._provider.get_capabilities` saw the cached value instead.
  Root-cause fix: the tool-search threshold code path now reads
  capabilities through `_resolve_capabilities(...)` directly — no cache
  populate — so the patch takes.

Tool-namespace isolation (bigger fix than CI symptoms suggested):
- `TOOLS` was the union of all loaded tool JSONs including the 6 new
  coordinator tools.  Interactive sessions were getting coordinator
  tools in their function-calling surface (which is nonsense — they
  require a console-hosted `coord_client`), and coordinator sessions
  counted against the interactive tool-search threshold.  Fix:
  - New `INTERACTIVE_TOOLS` / `INTERACTIVE_TOOL_NAMES` in
    `turnstone/core/tools.py` exclude anything with `coordinator: true`
    metadata.  `TOOLS` stays as the union for schema introspection +
    eval catalog.
  - `ChatSession.__init__` selects tool set by kind: coordinator gets
    fixed `COORDINATOR_TOOLS` (no MCP merge, no listeners registered);
    interactive gets `INTERACTIVE_TOOLS` (+ MCP if configured).
    Coordinators are meta-orchestrators that spawn child workstreams;
    MCP tools / resources / prompts live on the children, not on the
    coordinator's own surface.
  - `_on_mcp_tools_changed` no-ops for coordinator sessions
    (defence-in-depth in case listeners were registered).
  - `always_on_names` on `ToolSearchManager` is now the set of builtin
    tools actually present in the session (kind-aware) rather than the
    full `BUILTIN_TOOL_NAMES` frozenset.
  - `turnstone/eval.py` uses `INTERACTIVE_TOOLS` (coordinator tools
    aren't in scope for the eval harness which tests interactive agent
    behaviour).
  - Regression tests in `tests/test_workstream_kind.py`:
    - `INTERACTIVE_TOOLS ∩ COORDINATOR_TOOLS == ∅` and their union is
      `TOOLS`.
    - Interactive `ChatSession._tools` does not include any
      coordinator tool name.
    - Coordinator `ChatSession._tools` contains `spawn_workstream` but
      not `bash` / `edit_file` / `memory`; sub-agent lists are empty.
    - Coordinator `ChatSession` with an MCP client attached does NOT
      merge MCP tools and does NOT register any MCP listeners.

PR review findings:
- **#10 / #11** (Copilot): coordinator UI claimed to reuse the server
  renderer pipeline but loaded none of its JS.  Mirrored
  `turnstone/ui/static/renderer.js` into
  `turnstone/console/static/coordinator/renderer.js` (flagged in-file
  as a cleanup candidate to promote into `shared_static/`), added
  `katex.min.js` / `highlight.min.js` / `renderer.js` script tags to
  `coordinator/index.html`.  `coordinator.js` now buffers raw markdown
  via `textContent` during streaming, then swaps to `renderMarkdown` +
  `postRenderMarkdown` on `stream_end`.
- **#7** (Copilot): N+1 query pattern in
  `CoordinatorClient.list_children()` — per-row `storage.get_workstream`
  just to read `skill_id`.  Pushed `skill_id` + `skill_version` into
  the `list_workstreams` SELECT projection on both backends; the
  client reads them from `row._mapping` directly.  New
  `test_list_children_skill_filter_avoids_n_plus_one` pins the
  behaviour (asserts `storage.get_workstream` call count is 0).
- **#8 / #9** (Copilot): `spawn_workstream` tool JSON said "if empty,
  the workstream is created idle" but the prepare method rejected
  empty and the field was marked required.  Resolved by allowing
  empty end-to-end: removed from `required`, prepare builds a
  "spawn idle workstream" header + empty preview when empty,
  updated `test_spawn_prepare_allows_empty_initial_message`.
- **#1–#5** (github-code-quality): five asserts with side-effecting
  method calls in `test_coordinator_manager.py` (`mgr.close`,
  `mgr.open`, `mgr.create` in a dead `_c = ...`).  Extracted each
  call to a local variable so `python -O` can't strip the side
  effect.

Verification:
- `ruff check turnstone tests` clean.
- `mypy turnstone` clean (156 source files).
- `pytest -m "not live"` — 4063 passed, 3 deselected (live-backend
  tests), 0 failed.  The 3 image tests that were failing on this
  branch now pass; wheel + lint both green locally.

* polish(coordinator): address Copilot re-review findings

Two findings from the re-review of #368 after the first polish commit.

**user_id wired into `mgr.create()` at the server handlers.** Phase 1
added ``user_id`` to the ``Workstream`` dataclass and
``WorkstreamManager.create()`` signature, but the two call sites in
``turnstone/server.py`` forgot to pass the authenticated caller
through.  Result: interactive workstreams created via
``POST /v1/api/workstreams/new`` (including coordinator-spawned
children, which route through this handler) were landing with blank
``user_id``, defeating ownership-based access control on subsequent
sends / approvals / closes (``_require_ws_access`` treats blank
owners as legacy/allowed).  Two changes:

- ``server.py:create_workstream`` forwards ``user_id=uid`` — the same
  ``uid`` already resolved from the auth result (with trusted-service
  forwarding preserved).
- ``server.py:open_workstream`` prefers the persisted owner on the
  workstream row over the rehydrating caller so reloading someone
  else's workstream doesn't silently re-parent it.  Falls back to
  the authenticated caller when the stored row has no owner
  recorded (pre-phase-1 rows).

Regression test in ``tests/test_workstream.py`` pins
``WorkstreamManager.create(user_id=X)`` → ``ws.user_id == X`` so the
manager seam can't regress silently on a future refactor.

**Malformed-JSON recovery allowlist expanded for coordinator args.**
``_prepare_tool()`` has a two-stage salvage path for models that
emit malformed JSON: a regex-extract (fallback 1) and a bare-string
→ primary_key wrap (fallback 2).  The fallback-1 key list didn't
include coordinator argument names, so a slightly malformed
``spawn_workstream`` / ``send_to_workstream`` / etc. call would
hard-fail instead of salvaging into a minimal-args dict for retry.
Added ``ws_id`` / ``message`` / ``initial_message`` / ``parent_ws_id``
to the allowlist (kept alphabetised) so the coordinator tools get
the same model-self-correction behaviour as the interactive tools.
Fallback 2 already covers the ``ws_id``-primary-key tools via
``PRIMARY_KEY_MAP``; the regex path matters when the model emits
``{"ws_id": "abc", "message": "..."}`` with a trailing syntax error.

Verification: ``ruff check`` clean, ``mypy turnstone`` clean
(156 source files), ``pytest -m "not live"`` → 4065 passed, 3
deselected (live-backend), 0 failed.

* fix(coordinator): address ultrareview findings on coordinator workstream kind

Security
- Cross-tenant leak: CoordinatorClient.inspect/list_children now constrain
  to the coordinator's own ws_id + direct children; an LLM coerced via
  prompt injection can no longer exfiltrate other tenants' workstreams.
- Empty-owner short-circuit bypass: strict equality at coordinator.py
  ownership gate and at the storage-fallback branch in coordinator_history;
  orphan/system-owned coordinator rows can no longer be rehydrated by
  arbitrary holders of admin.coordinator (DoS + history disclosure vector).
- Closed coordinators no longer silently resurrect on subsequent GET —
  the Close button is now actually durable across URL revisits and tab
  refreshes; rows with state in {closed, deleted} refuse rehydration.

Correctness
- ChatSession.close() now releases the CoordinatorClient httpx.Client
  pool; previously every closed/evicted coordinator dropped a connection
  pool on the floor until non-deterministic GC.
- open_workstream rehydration now forwards parent_ws_id + kind, so
  coordinator-spawned children survive node restart / idle eviction
  with their parent link intact instead of becoming silent orphans.
- list_children truncated flag now signals whenever the SQL fetch hit
  the page cap (previously permanently False in the no-filter case,
  causing confident-but-incomplete summaries from the coordinator).
- ConsoleCoordinatorUI.approve_tools: per-tool auto-approve now checks
  auto_approve_tools independently of the blanket auto_approve flag,
  so 'Always approve this tool' actually works on the next invocation.

Concurrency
- _spawn_worker no longer falls through to start a second concurrent
  worker thread on the same ChatSession when queue.Full fires; instead
  send() returns False and the endpoint surfaces HTTP 429.
- _open_locks entries are now refcounted under self._lock and only
  popped when the last waiter releases — eliminates the race where a
  rehydration-failure path lets two threads serialize on different lock
  instances for the same ws_id and trip the "already tracked" guard.

Tests: +6 regression cases covering closed-coordinator refusal,
empty-owner non-admin refusal, queue.Full no-duplicate-worker,
inspect/list_children cross-tenant rejection, and truncated semantics.
2026-04-16 21:40:50 -07:00
Patrick Buckley a917bf2690 docs: apply Copilot review feedback on PR #367
All eight suggestions verified against source before applying:

- docs/settings.md — ConfigStore key names are `model.plan_alias` /
  `model.task_alias` (not `plan_model` / `task_model`); updated in
  both the overview list and the plan/task overrides table.
- docs/security.md — `src` claim values now reflect what actually
  gets minted: `password`, `database` (from API-token exchange),
  `oidc`, plus service origins `console`, `cli`, `channel`.
- docs/sdk.md — `upload_attachment(ws_id, filename, data, *,
  mime_type=...)` matches the real SDK signature; `bytes`-returning
  helper is `get_attachment_content` (not `download_attachment`);
  code example reordered so it doesn't collide on `filename=` kwarg.
- docs/architecture.md — "prior `plan` tool call" → "prior
  `plan_agent` tool call" so wording stays consistent with the
  renamed tool.
- docs/tools.md — `plan_agent` `primary_key` is `goal`, not
  `prompt`, in both the primary-key table and the summary table
  (matches the JSON schema in turnstone/tools/plan_agent.json).
2026-04-16 16:01:24 -07:00
Patrick Buckley 471d1a3311 docs: audit documentation for 1.4 / 1.5 state
Systematic pass over every doc under docs/, the root-level README /
QUICKSTART / CONTRIBUTING, and the PlantUML diagrams.  Memory and docs
had drifted against the code since 1.2 — this catches them up to the
1.4.0 release and the 1.5.0a1 experimental line.

User-facing fixes
- README: fix broken docs/mcp.md link (→ mcp-registry.md); channel
  gateway entry reflects shipped Discord + Slack adapters instead of
  "Slack/Teams planned"; diagrams table mentions both.
- QUICKSTART: docs/*.md relative links were wrong from the repo root;
  wizard version bumped from 0.5.4.
- CONTRIBUTING: add dev extra plus the ruff / mypy / pytest commands
  we actually expect before push.

Reference docs
- architecture.md: 19 tool schemas (was 15), 18 admin tabs (was 14),
  turnstone-bootstrap added to entry-points table, OpenAI provider
  file split (chat/responses/common) documented, 38 SDK event
  dataclasses (was 27 and referenced deleted mq/protocol.py), Slack
  adapter + multi-adapter gateway, plan_agent/task_agent naming,
  governance admin-panel rewrite.
- api-reference.md: full attachment endpoints (POST/GET/content/
  DELETE on /v1/api/workstreams/{ws_id}/attachments) plus the
  multipart mode on POST /v1/api/workstreams/new.
- channels.md: Slack Setup section (Socket Mode app creation, OAuth
  scopes, tokens), Slack CLI/env reference in config table, combined-
  adapter architecture diagram.
- console.md: 18-tab listing (was 13) with Channels/Models/Nodes/TLS
  descriptions and ConfigStore live-edit note.
- docker.md: Slack env vars block; image entry-point list now
  includes turnstone / turnstone-bootstrap.
- sdk.md: attachments methods on the server client, attachments
  example (upload-then-send and at-creation), event count fixed.
- releasing.md: four-track table (stable/1.0, 1.3, 1.4 + main 1.5);
  promotion workflow uses 1.5 / 1.6 numbering.
- settings.md: plan_model / task_model / plan_effort / task_effort
  overrides section.
- governance.md: skill naming (/skill, `skill` field — not /template),
  Prompts/Judge tabs called out.
- security.md: two-token-types wording; src claim values match the
  AuthResult source strings actually emitted.
- mcp-registry.md: SDK package name is @turnstone/sdk.
- tools.md: plan / task renamed to plan_agent / task_agent in the
  section headings and summary table; primary-key table matched.
- design/consistent-hash-ring.md: dead direct-http-transport.md
  pointer redirected to architecture.md.

Diagrams
- 02-package-structure: drop phantom chat.py entry point, add admin
  and bootstrap, add slack/bot.py, rename channels/gateway.py →
  channels/cli.py.
- 16-channel-architecture: Slack is no longer "(future)", add a
  SlackBot class and the slack-bolt Socket Mode edges; wire the new
  bot into ChannelService.  PNGs regenerated from both puml sources.
2026-04-16 16:01:24 -07:00
Patrick Buckley 879be89bbd docs: enrich CHANGELOG and add Contributors section for 1.4.0
Audit pass against the actual commit messages between v1.3.0 and
v1.4.0 turned up several substantive items the initial CHANGELOG
under-described or omitted entirely.  Fix-forward expansion plus a
Contributors section recognizing external contributors.

Added detail / coverage:

- New "Server compatibility layer for local model servers" entry —
  the vLLM / llama.cpp profiles + admin UI fields shipped in #352
  alongside the capabilities passthrough; previously buried under one
  bullet.
- Per-call plan/task model selection split into three sub-bullets
  (backend split, runtime configurability without restart via
  ConfigStore admin tab, per-call override) — three PRs that build on
  each other deserve to be discoverable independently.
- Opus 4.7 entry expanded with 1M ctx / 128K output, the new
  thinking_display capability field, xhigh effort level, and admin
  dropdown updates.
- Dashboard composer note: tab-bar `+` modal also gained the paperclip
  + chip strip + first-message field.
- Slack adapter: explicit "session recovery via persisted recoverable
  route keys" — ops-relevant promise for restart behaviour.
- pgbouncer swap: helm chart link + ports updates noted.
- Provider capabilities entry: defensive shallow-copy + chat_template
  deep-merge follow-ups.

New Fixed entries:

- Cross-user attachment-fetch hardening (get_attachment_content
  scopes by user_id).
- Attachment-list DoS guard on /v1/api/send.
- Bounded LRU for upload locks.
- 3.12 CI deadlock root-cause writeup (asyncio.Lock vs Starlette
  TestClient loop teardown).

New SDK entry:

- PlanResolvedEvent type + guard, dispatched cross-client when one
  client resolves a plan so others dismiss in sync.

New Operational subsection:

- vendor-js workflow now auto-downloads hls.js for future Renovate
  bumps so they're merge-ready without manual file fetches.

Contributors:

- Recognise @daoxley (Slack adapter, #355) and @pizzaandcheese
  (pgbouncer swap, #353) — the two external contributors with
  meaningful net-new work in this release — plus the Renovate bot.
- Pointer to channel-attachment ingest as the headline 1.4.1 feature
  for would-be contributors.
2026-04-16 15:16:19 -07:00
Patrick Buckley 86d981eb73 chore: bump version to 1.5.0a1 2026-04-16 15:07:53 -07:00
Patrick Buckley b1e7c82e95 docs: add CHANGELOG.md for the 1.4.0 release
Repo previously had no CHANGELOG.  Establishes the file with full
1.4.0 coverage (attachments end-to-end, dashboard composer refactor,
Slack adapter, per-call plan/task model, provider capability
passthrough, Opus 4.7) plus a one-line 1.3.1 entry for the Opus 4.7
backport.  Format follows Keep a Changelog 1.1.0; release-track
guidance up top covers the three stable branches + main.

Operator-relevant call-out at the top of [1.4.0]: migrations 037 +
038 must be applied before starting 1.4.0 against an existing 1.3.x
database.  Both are additive and idempotent.
2026-04-16 15:05:39 -07:00
Patrick Buckley aff449116e feat(ui): dashboard composer polish from PR #362 designer review (#366)
* feat(ui): dashboard composer polish from PR #362 designer review

Three deferred items from the prior designer pass on the unified
dashboard composer.  Pure UX affordances; no server change.

- **Persist Options open/closed in localStorage.**  Power users who
  routinely set non-default model/skill don't have to click "Options"
  on every page load.  Key: `turnstone.dashboard.options_open`.
  Defaults closed for first-time users.  Falls back gracefully when
  localStorage is unavailable (private mode, quota).

- **Active-options summary chip.**  Renders the non-default model /
  judge / skill values inline next to the Options button (mono, dim,
  separated by middots).  Hidden via `[hidden]` when everything is at
  default — no chrome cost in the common case.  Updates on any select
  change via a single delegated handler on the panel.  Hidden on
  narrow viewports (the action row stacks vertically there and the
  chip would push the layout further).

- **"Drop to attach" overlay during drag.**  CSS pseudo-element on
  `.dashboard-composer-drop` overlays a centered "Drop to attach"
  label so dragging a file makes the action explicit instead of just
  showing the dashed-border highlight.  pointer-events: none keeps
  the underlying composer controls reachable; visual only.

* fix(ui): address Copilot review on dashboard composer polish

- _restoreDashboardOptionsState() forced the panel closed every time
  showDashboard() ran when localStorage was unavailable (private mode,
  storage quota), contradicting the comment that promised a per-session
  fallback.  Add a module-scoped _dashOptionsOpenSession variable
  updated by _setDashboardOptionsOpen / _toggleDashboardOptions, and
  only override the visible state from localStorage when the read
  genuinely succeeded.  The session value now preserves the user's
  choice across hide/show cycles in environments where localStorage
  throws.

- Fold the duplicated `.dashboard-composer { position: relative; }`
  block into the existing rule above.  The position context is needed
  for the .dashboard-composer-drop::before overlay; the comment now
  says so.
2026-04-16 14:54:25 -07:00
renovate[bot] ddf7b3c2f0 chore(deps): lock file maintenance 2026-04-16 14:45:22 -07:00
Patrick Buckley a6c6b71d66 feat(console-ui): support Slack channel_type in admin UX
PR #355 added the Slack adapter on the server but missed the console
admin surfaces that talk to channel_type.  Three concrete gaps + a
designer-review polish pass.

Functional bug + UI parity:

- _collectNotifyTargets() in admin.js hardcoded `channel_type: "discord"`
  — even on a Slack-only deployment the skill notify-on-complete form
  always wrote Discord targets, sending notifications to the wrong
  adapter (or nowhere).  Add a per-row channel-type <select> driven by
  a small _NOTIFY_CHANNEL_TYPES table that's the one place to register
  a new platform; collector and populator both read from the dropdown.
  ID-input placeholder updates dynamically when the platform changes.

- The "Link Channel Account" modal only offered Discord — users
  couldn't link a Slack account through the UI at all.  Add a Slack
  <option> and reuse the same dynamic-placeholder helper.  Drop the
  static Discord-shaped HTML placeholder so the JS-driven hint doesn't
  flash a Discord example before the dropdown initializes.

- Skill create/edit modals only showed Discord in the notify-on-complete
  placeholder example.  Show both adapters.

- Per-platform .scope-discord / .scope-slack badge classes so the
  linked-accounts list distinguishes platforms visually instead of all
  rendering as the generic .scope-channel magenta.  Falls back to
  .scope-channel for any future channel_type the stylesheet doesn't
  yet know about.

Designer review polish:

- Theme-aware --discord / --slack / --discord-glow / --slack-glow
  tokens in base.css.  The first pass shipped raw hex (#818cf8 /
  #f472b6) that fails WCAG AA on light theme (1.8:1 and 2.4:1); the
  light variants (#4f46e5 indigo, #be185d rose) pass.  Badge classes
  now reference tokens, matching every other .scope-* rule.

- Notify-row mobile layout: three controls in a row left ~80px for
  the ID input at 360px viewport, truncating snowflakes.  Tighten
  platform select to 76px (labels are short), add flex-wrap, and at
  ≤700px drop the ID input to its own row so it gets full width.

- Per-platform classes apply alone (not co-classed with scope-channel)
  so winning the cascade doesn't depend on stylesheet source order.

- Replace "Discord snowflake" jargon with "Discord ID"; give Slack
  ids concrete examples (C01234567 / U01234567) instead of an
  ambiguous "C0…".
2026-04-16 14:45:01 -07:00
Patrick Buckley 0d3516d6e0 fix(slack): post-merge fixes from Copilot + eous review
Combines the substantive bot.py fixes flagged in both review trails on
PR #355.  Discord parity items grouped here too since they're the same
surface (slack/bot.py).

From Copilot:

- _notify_reply_routes was read on StreamEndEvent but never popped on
  the success path.  Result: one notification reply pinned every later
  response for that ws_id to the notification thread until the bot
  restarted.  Pop after read; combine the surrounding ifs (SIM102).

- PlanReviewEvent embedded raw event.content inside a triple-backtick
  mrkdwn fence without escaping.  A plan with ``` (very common — plans
  often quote code) would break the fence and let later content render
  as live markup, including unintended Slack mentions/links.  Rewrite
  _sanitize_slack_preview to splice a zero-width space inside any ```
  sequence (Slack stops recognizing it as a delimiter) instead of
  escaping every single backtick — keeps single-backtick code snippets
  readable while still protecting the fence.  Apply to plan-review.

- _send_approval_request joined unbounded tool_lines into one mrkdwn
  section, but Slack section.text caps at 3000 chars.  Multi-tool
  batches with large previews silently failed chat_postMessage,
  leaving the user unable to approve/deny.  Cap each preview to 600
  chars under a 2700-char total budget; append "+N more" when truncated.

From eous (parity with Discord):

- Pass `client_type="chat"` from both `get_or_create_workstream` call
  sites (slash-command session + DM).  Without it Slack-routed
  workstreams loaded the web-default prompt; the chat-specific
  system prompt now applies as it does for Discord.

- Add `exc_info=True` to the eleven `log.debug(...)` exception handlers
  so underlying tracebacks are available when debug logging is on
  instead of being silently dropped.  Level stays debug — these are
  benign-by-default sites (chat_update on a deleted message, etc.) so
  only the visibility changes.  Typed-exception handlers
  (RemoteProtocolError, etc.) keep their bare debug log.

- Module docstring on slack/__init__.py so pydoc / import errors have
  human-readable context.

Tests: rewrite the sanitizer test to match the new (more permissive)
single-backtick behaviour; add coverage for the triple-backtick
neutralization + short-input passthrough; patch httpx.AsyncClient at
all five TurnstoneSlackBot construction sites so each test doesn't
leak an unclosed real client.
2026-04-16 14:45:01 -07:00
Patrick Buckley a8dcccafa3 chore(slack): unblock CI + dependency cleanup after #355
- cli.py: ChannelAdapter import is annotation-only; move into
  TYPE_CHECKING block and switch the two cast() calls to string-form
  so the runtime import isn't required (TC001).
- slack/{config,routes}.py: ruff format fixes (whitespace + drop
  redundant string-form annotation now that __future__ annotations
  is in effect).
- pyproject.toml: drop the unused `tests.*` mypy override — `mypy
  turnstone` (the only invocation in CI + local) never matches it,
  so it was pure noise in the "unused section(s)" report.  Other
  optional-dep overrides stay; they're real safety nets when running
  mypy without the [all] extras (e.g. on the test job).
- uv.lock: regenerate to match the slack-bolt + transitive deps the
  pyproject changes resolve to (lock-check was failing on stale hash).
2026-04-16 14:45:01 -07:00
renovate[bot] e19032f369 chore(deps): update ghcr.io/astral-sh/uv docker tag to v0.11.7 (#364)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-04-16 14:28:22 -07:00
daoxley d3ff5e5ac7 Add Slack channel adapter with Socket Mode support (#355)
* Add Slack channel adapter with Socket Mode support

Adds a Slack channel adapter mirroring the Discord adapter pattern:

- Socket Mode connection
- Per-user session management in channels via configurable slash command
- SSE-based event consumption from server nodes
- Tool approval buttons with policy evaluation support
- DM routing without slash command (requires Slack app DM permissions; not validated in current workspace)
- Session recovery after restart via recoverable route keys

New files:
- turnstone/channels/slack/bot.py
- turnstone/channels/slack/config.py
- turnstone/channels/slack/__init__.py
- tests/test_channel_slack.py

Updated:
- turnstone/channels/cli.py — adds Slack CLI arguments
- pyproject.toml — adds slack extra and mypy overrides

Usage:
- Install with Slack support:

* Fix lint issues and update lock file

* fix: unify channel startup, fix Slack notification reply routing, add plan review UI/actions

* Fixes to notification responses

* Add turnstone/channels/slack/routes.py
Update bot.py to import shared SlackRoute
Update _http.py Slack notify validation

* Add approval guards

* Only creators can approve

* Updated slack test suite

* use integer tuple comparison for Slack timestamp ordering

* suppress mypy no-untyped-call for slack_bolt socket mode handler
2026-04-16 13:45:46 -07:00
Patrick Buckley a0b3c35d28 Feat/attachment followups (#363)
* fix(ui): rehydrate chip strip after queued-message dequeue not_found

The dequeue handler only refreshed the per-pane chip strip when the
DELETE returned status="removed". On status="not_found" (the queued
message already dispatched), chips stayed stale: any reservations that
raced the dispatch could leave the UI showing a different pending set
than the server actually had.

Re-fetch on both paths so the chip strip always reflects the
authoritative server state. The queued-message bubble itself stays
visible on not_found, same as before — the promote loop strips the
queued styling on idle.

* feat: sweep orphan attachment reservations periodically

Process crashes between reserve_attachments and consume/unreserve can
leave attachment rows soft-locked forever (reserved_for_msg_id NOT NULL
with no consumer ever coming back). The worker-thread exception path
in /v1/api/send already handles in-process failures, but a hard kill
or oom mid-send escapes that.

Add sweep_orphan_reservations(older_than_seconds) to the storage
protocol — clears reserved_for_msg_id on rows where message_id IS NULL
and created < now() - threshold. Implemented for SQLite + PostgreSQL
using the same string-comparison form (created is ISO-8601 text in
both backends, lexicographic order matches chronological).

Wire into the server lifespan: run once at startup (catches anything
left over from the previous process), then every 30 minutes as
defense-in-depth. Threshold is 4 hours so we don't race a long-running
dispatch and unreserve rows the worker is still about to consume.

Tests cover sweep semantics: clears old reserved rows, leaves fresh
ones alone, skips already-consumed rows, no-ops on zero/negative
threshold.

* fix: track reserved_at for orphan-reservation sweep

Copilot review on PR #363 flagged a real correctness bug: the sweep
used the attachment row's `created` timestamp (upload time) as the
staleness signal. An attachment uploaded hours ago but reserved fresh
could be unreserved mid-send, after which mark_attachments_consumed
silently drops the row because reserved_for_msg_id no longer matches
the send_id.

Add a dedicated `reserved_at` column (migration 038) set on
reserve_attachments and cleared on mark_attachments_consumed /
unreserve_attachments. The sweep now scopes by `reserved_at < cutoff`,
so reservation age is what's measured, not upload age. Backed by a
partial index `(reserved_at) WHERE reserved_at IS NOT NULL` so the
periodic scan stays cheap as the consumed-history grows.

Threshold dropped from 4h to 1h since it now means "longest realistic
single send" rather than "longest plausible time between upload and
send" — a tighter, more defensible bound.

Tests cover the regression (uploaded long ago + reserved fresh must
not be swept), plus reserved_at clearing on both consume and unreserve.
2026-04-16 13:39:32 -07:00
Patrick Buckley 6cbd3eb2c1 feat: workstream attachments at creation time + SDK + UI parity (#362)
* feat: workstream attachments at creation time + SDK + UI parity

Closes the two big deferred items from PR #356: attaching files as part
of the initial workstream-creation request, and full SDK coverage of the
attachment surface.

Server: POST /v1/api/workstreams/new now accepts multipart/form-data
(meta JSON + 0..N file parts).  Files are validated and saved as pending
under the new ws; when initial_message is also set the create handler
reserves them onto that turn before the dispatch worker fires, mirroring
the /v1/api/send pattern.  Validation failure rolls back the workstream
via delete_workstream so we don't leak orphan rows or emit a phantom
ws_created/ws_closed pair on SSE.  JSON path is unchanged.

Console routing: route_create accepts multipart with ?ws_id=<hex> as a
query parameter (the console hashes the id before the body lands).
Added /v1/api/route/workstreams/{ws_id}/attachments POST/GET/DELETE +
.../{attachment_id}/content GET proxies that forward raw bytes and
preserve upstream headers (Content-Disposition, X-Content-Type-Options,
CSP sandbox).

Python + TypeScript SDKs: AttachmentUpload type, upload_attachment,
list_attachments, get_attachment_content, delete_attachment, and
send(attachment_ids=...).  create_workstream(attachments=...) sends
multipart and pre-generates a ws_id client-side so cluster routing
works.  SDKs reject attachments+target_node combinations since the
multipart route doesn't honor target_node.

Web UI: dashboard composer refactored to a single unified create flow.
Replaced the inconsistent split (Enter created+sent raw, "New Chat"
opened a modal) with one rich composer carrying a textarea, paperclip
+ chip strip, drag-drop, paste-image, and a collapsible Options panel
for model/judge_model/skill.  Submit button dynamically labels Create
vs Send.  New-workstream modal also gained the same paperclip + chip
strip + first-message field for the tab-bar + entry point.

Tests: 30 new tests across server multipart create, console route
multipart + attachment proxies, Python + TS SDK attachment surfaces,
plus regressions for the three review-flagged bugs (Content-Type
boundary preservation, attachments+target_node rejection, no phantom
ws_created on validation failure).

* fix: address Copilot review feedback on PR #362

- web_helpers: docstring now matches behaviour — read_multipart_create_or_400
  does enforce the optional max_per_file_bytes cap as defense-in-depth.
- app.js: drop the duplicated _formatAttachSize definition (one already
  exists earlier for pane chips); add a shared _isAttachmentAllowed helper
  that mirrors the server's classifier (png/jpeg/gif/webp images, text/*
  MIMEs, allowlisted application/* MIMEs, known text extensions) and call
  it from both _newWsAddFiles and _addDashboardFiles so unsupported files
  fail fast client-side instead of after a server roundtrip.
- app.js: dashboardSubmit catch now suppresses the redundant error toast
  on authFetch's "auth" Error and falls back to a generic message when
  err.message is undefined, instead of rendering "Connection error: undefined".
- SendResponse (Pydantic + TS): document and expose attached_ids,
  dropped_attachment_ids, priority, and msg_id so attachment-aware SDK
  callers can detect partial reservations and dequeue queued messages.
- test_server_attachments_on_create: drop the dual `import turnstone.server`
  + `from turnstone.server import` style — use monkeypatch.setattr by
  dotted path for module-level mutation and `from … import …` for the
  helpers, keeping a single import style.
2026-04-16 13:30:25 -07:00
Patrick Buckley 551fc43c15 feat: per-call model selection on plan_agent / task_agent (#361)
* feat: per-call model selection on plan_agent / task_agent

The calling LLM can now pass `model="<alias>"` to plan_agent or
task_agent to override the operator-configured per-kind model for
that one invocation.  Useful when subtask difficulty varies within a
session: the model can downgrade to a cheap alias for trivial work
and reach for a stronger one when the problem is hard.

Tool descriptions list the live registered aliases (refreshed when
the operator hits "sync to nodes" / internal_model_reload), so the
calling LLM always sees the current options.  Bad aliases return a
corrective error dict with the available choices so the LLM retries
cleanly rather than failing silently.

No whitelist — any alias the registry knows is acceptable; cost
control is intentionally ceded to the model.  No per-call effort
override (out of scope; effort stays operator-configured).

Resolution precedence in _run_agent: explicit per-call agent_alias
override > registry per-kind (plan_model/task_model) > legacy
agent_model > session model.  The plan retry path (when
_validate_plan fails) reuses the same alias so coaching reflects
real model behaviour rather than a different model masking the
signal.

Implementation:
- plan_agent.json / task_agent.json: optional `model` parameter.
- ChatSession._validate_agent_model_override extracts and validates
  the arg; mirrors the existing empty-prompt error pattern.
- _prepare_plan / _prepare_task stash the override in
  item["model_override"]; _exec_* pass it through.
- _run_agent gains agent_alias kwarg with defence-in-depth
  ValueError on unknown alias.
- _render_agent_tool_descriptions deep-copies plan/task entries
  before mutating description so the module-level TOOLS constant
  stays untouched across sessions; rebuilds the BM25 tool-search
  index when active so its text matches what the LLM sees.
- server._broadcast_agent_tool_schema_refresh walks active
  workstreams on internal_model_reload so descriptions update
  without restart.

* fix: clarify no-registry placeholder + avoid double BM25 rebuild

Addresses Copilot feedback on PR #361.

1. plan_agent.json / task_agent.json placeholder said the parameter
   falls back to the "operator-configured plan/task model".  That
   text is what no-registry sessions see (registry-bearing sessions
   get the templated description with the live alias list); for
   those single-model sessions, omitting the param falls back to
   the current session model, not an operator-configured one.
   Reword so the no-registry user gets accurate guidance.

2. _on_mcp_tools_changed already calls _rebuild_tool_search after
   merging MCP tools.  _render_agent_tool_descriptions also
   rebuilt the BM25 index when active, so the MCP refresh path
   was rebuilding twice per refresh.  Move the BM25 rebuild out
   of the private render helper into the public
   refresh_agent_tool_schemas wrapper — _on_mcp_tools_changed
   keeps calling the render helper directly (no double rebuild),
   and registry-reload callers go through the wrapper which
   still keeps the index in sync.
2026-04-16 11:50:13 -07:00
Patrick Buckley 6c026710ff feat: ConfigStore + admin UI for plan/task agent model and effort (#360)
* feat: ConfigStore + admin UI for plan/task agent model and effort

Per-kind sub-agent routing was added in #359 but only via config.toml.
Operators can now switch the plan_agent / task_agent model and reasoning
effort at runtime from the admin Model tab without restarting.

Adds four ConfigStore-backed settings:
  model.plan_alias    — alias for plan_agent
  model.task_alias    — alias for task_agent
  model.plan_effort   — reasoning effort for plan_agent
  model.task_effort   — reasoning effort for task_agent

Server startup and internal_model_reload both apply these as overrides
on top of the registry's config.toml-loaded values; the new logic
computes "effective" values for all five model-routing fields and only
calls registry.reload() when at least one differs.

Admin UI: extracts ALIAS_SETTING_KEYS to a const used by both the
dynamic-alias-choice injection and the empty-option label rendering.
Adds INHERIT_EMPTY_LABEL_KEYS so plan_effort / task_effort show
"(inherit)" for empty — distinct from the literal "none" choice (which
actually disables reasoning, very different from leaving unset).

Also fixes Copilot review feedback from #359:
  - _validate_effort treats empty / whitespace as unset rather than
    warning on benign explicit-empty configs (with .strip().lower()
    normalisation; "HIGH" and " low " now parse correctly)
  - turnstone.example.toml's reasoning_effort comment lists the full
    set of accepted values (none, minimal, low, medium, high, xhigh, max)

* fix: apply routing overrides on config-reload + skip no-op model-reload

Addresses Copilot feedback on PR #360.

1. Admin settings updates fan out via /_internal/config-reload, which
   only reloaded the ConfigStore — plan/task routing changes weren't
   visible until a model-reload or restart, defeating the runtime
   configurability this PR is meant to add.

2. /_internal/model-reload always called registry.reload(), churning
   cached clients even when nothing changed. Risky when fanned out
   across nodes (could close in-flight clients).

Extracts two helpers in server.py:
  - _effective_routing(cs, ...)  pure function: overlay CS values on base
  - _apply_routing_overrides(reg, cs)  reload only when something differs

Used by the startup path, config_reload (new), and model_reload (now
short-circuits with a noop response when models + routing are unchanged).
2026-04-16 11:08:26 -07:00
Patrick Buckley 54dd557476 feat: split plan_model and task_model, configurable agent reasoning effort
plan_agent and task_agent previously shared a single agent_model knob and
plan_agent hardcoded reasoning_effort="high" in three call sites. They
have different cost/latency profiles — plan is rare and benefits from a
stronger model, task is frequent and benefits from a cheaper one — so
sharing the knob undertunes both.

ModelRegistry gains plan_model, task_model, plan_effort, task_effort.
Per-kind overrides win over the legacy agent_model, which still works
as the single-knob fallback for both. resolve_agent_alias(kind) and
resolve_agent_effort(kind) centralise the resolution; PLAN_DEFAULT_EFFORT
captures the back-compat "high" default in one place rather than at
every call site.

session._run_agent delegates resolution by label ("plan" vs "task").
The three hardcoded reasoning_effort="high" arguments are removed —
behaviour is identical when no plan_effort is configured.

Loader validates effort against {none,minimal,low,medium,high,xhigh,max}
and warns + drops typos rather than passing them to the provider.

ConfigStore parity and admin UI for the new knobs are deferred to a
follow-up — config.toml-only is enough for the backend split.
2026-04-16 10:26:19 -07:00
Patrick Buckley 87a9af1075 fix: broadcast plan_resolved SSE so other clients dismiss in sync
Previously, resolving a plan on one client (e.g. phone) cleared the
server's pending state and unblocked the worker, but emitted no event
to other connected clients. Their plan-approval modal stayed stuck.

resolve_plan() now enqueues a plan_resolved frame (mirroring the
approval_resolved pattern in resolve_approval) before clearing
_pending_plan_review, so a reconnecting client cannot receive both
the replayed plan_review and the live plan_resolved. Skips the frame
on the cancel-with-no-plan path.

Client adds a plan_resolved handler that dismisses the modal without
re-firing /v1/api/plan, restores keyboard context (skipped on touch
to avoid soft-keyboard pop on mobile), labels the inline plan summary
"(synced)" so remote dismissal is unambiguous, announces via the
existing aria-live #toast for screen-reader parity, and falls back
to an info message if plan_resolved races ahead of plan_review.

Adds PlanResolvedEvent to the Python and TypeScript SDKs with
deserialization and type-guard tests.
2026-04-16 09:56:17 -07:00
Patrick Buckley a6c4abe82a chore: bump version to 1.4.0a4 2026-04-16 09:15:03 -07:00
Patrick Buckley 30c89f46c6 feat: add Claude Opus 4.7 support (#357)
- Add claude-opus-4-7 capability entry (1M ctx, 128K output, adaptive
  thinking, supports_temperature=False, thinking_display=summarized)
- Suppress temperature param for Opus 4.7 (API returns 400)
- Add thinking display opt-in via new ModelCapabilities.thinking_display
  field - Opus 4.7 omits thinking by default, always send summarized
- Add xhigh effort level to mapping and Opus 4.7 effort_levels
- Add xhigh/max options to skill template dropdowns in admin console
- Align reasoning effort label capitalization across all console dropdowns
- Update example config to reference claude-opus-4-7
- 10 new tests with regression guards for Opus 4.6 backward compat

Verified against live API: streaming and completion calls succeed.
2026-04-16 08:55:32 -07:00
Patrick Buckley aaea4d302d chore(security): ignore unfixable jq CVEs in Debian 13.4 base image
Trivy flags two HIGH CVEs in jq/libjq1 1.7.1-6+deb13u1 with no fixed
version yet from Debian:

- CVE-2026-39979: out-of-bounds read in jv_parse_sized() on non-NUL-
  terminated buffers
- CVE-2026-40164: DoS via crafted JSON causing hash collisions

jq is invoked only on trusted CLI/admin paths against
process-controlled JSON input in turnstone — never on untrusted
network bytes — so the NUL-terminated invariant holds and the DoS
vector is not reachable.

Will revisit when Debian publishes a patched libjq1.
2026-04-15 13:41:15 -07:00
Patrick Buckley b8daeb3be2 chore: bump version to 1.4.0a3 2026-04-15 13:36:18 -07:00
Patrick Buckley 97fbfb9f8e feat: workstream attachments (images + text documents) (#356)
* feat: workstream attachments (images + text documents)

Adds end-to-end support for attaching images (png/jpeg/gif/webp) and
plain-text documents (markdown, source, JSON, etc.) to a workstream's
next user turn via the web UI.

Storage: new workstream_attachments table (migration 037) with a
three-state lifecycle — pending → reserved → consumed — scoped by
(ws_id, user_id) and linked to conversations.id on consume. Rewind/
truncation cascades attachment rows; delete_workstream does too.

Session: ChatSession.send(attachments, send_id) builds multipart user
content (text + image_url + document parts) and persists text-only to
conversations with attachments joined on load via message_id. Queue
path carries ordered attachment_ids plus a reservation token so
queued multimodal turns can't lose files to overlapping sends.

Providers: internal document content parts translate at the API
boundary — Anthropic emits native document blocks (text/plain
coerced, original MIME folded into title); OpenAI Chat Completions
and the Google OpenAI-compat endpoint inline them as escaped
<document> text blocks (XML-attr escape + </document> neutralization);
Responses API emits input_text with the same wrapper.

Server: POST/GET/DELETE /v1/api/workstreams/{ws_id}/attachments with
multipart upload (magic-byte image sniffing, UTF-8 enforcement for
text, per-kind size caps, Content-Length pre-check, per-(ws,user)
pending cap + TOCTOU lock). /v1/api/send reserves before dispatch
using a full-UUID token, threads it into session.send / queue_message,
releases on worker-thread failure, and reports attached/dropped ids
so the UI can reflect partial reservations. GET /content sets
X-Content-Type-Options, CSP sandbox, inline Content-Disposition, and
forces text/plain for text kinds. Ownership failures mask as 404.

UI: paperclip button, hidden file input with accept allowlist, chip
strip above textarea, drag/drop + paste-image handlers. Chips
rehydrate on ws switch and on queued-message dequeue; send clears
only attached ids and shows a toast when some dropped. Historical
user messages render filename pills via a _attachments_meta sibling
populated on both live-send and reconstruct paths.

530 tests covering CRUD, reservation lifecycle, races (TOCTOU cap,
reserve-then-dispatch overlap), provider translation, XSS headers,
cascade delete, history round-trip, and service-scoped actor flow.

* fix(attachments): address PR review feedback

- get_attachment_content now scopes the row by user_id too, so an
  unowned workstream can't be a vector for cross-user blob fetches
  via attachment_id guessing (Copilot, server.py:2676)
- send_message rejects attachment_ids lists longer than the pending
  cap with 400 — prevents hostile clients from blowing up the
  storage IN (...) clause (Copilot, server.py:1515)
- _attachment_upload_locks switched to a bounded LRU OrderedDict;
  evicts the oldest unlocked entries past the soft cap so the map
  can't grow unboundedly on long-running nodes (Copilot, server.py:2417)
- Pane.dragleave handler uses relatedTarget instead of target so the
  drop-zone styling clears correctly when the cursor moves through
  child elements; dragend listener added as a fallback for cancelled
  drags (Copilot, app.js:297)
- uploadAttachment always cleans up the placeholder chip on failure,
  including auth errors — no more stuck "uploading..." chips after
  re-auth (Copilot, app.js:427)
- New _swapPlaceholderChip / _removeAttachmentChip helpers preserve
  user-selection order through the placeholder→real-id swap; the
  pendingAttachments Map is rebuilt in place rather than naïvely
  delete+set, which would have moved the entry to iteration end
  (Copilot, app.js:420)
- Drop unused `var self = this;` in removeAttachment (github-code-quality)
- Two regression tests: cross-user fetch on an unowned workstream,
  and oversized attachment_ids list rejection

* fix(attachments): switch upload-lock to threading.Lock to avoid 3.12 CI hang

The per-(ws, user) upload lock was a module-cached asyncio.Lock.
Starlette's TestClient runs each request on a fresh anyio task /
event loop, so the cached lock's internal _waiters bind to the first
loop that acquired it.  When a later request runs in a different
loop, await lock.acquire() blocks on a Future from a closed loop —
silent deadlock.

This surfaced as test (3.12) hanging indefinitely in CI on one push
while the same suite passed on 3.11/3.13 and on the next push.  Same
root cause is reproducible against any Starlette TestClient harness
on 3.10+; 3.12 just happens to surface it more often given changes
in how anyio + asyncio.Future interact across loop teardown.

Switched to threading.Lock — loop-agnostic, and the critical section
is one COUNT + one INSERT, short enough that briefly blocking the
event loop is fine.  Updated the LRU-eviction probe accordingly
(threading.Lock has no public .locked(), so use a non-blocking
acquire+release as the "is it free?" probe).

TOCTOU pending-cap test still passes; full attachment suite passes
on both 3.12 and 3.13.
2026-04-15 13:30:22 -07:00
pizzaandcheese 4da751c1c6 replace bitnami pgbouncer with edoburu pgbouncer (#353)
* replace bitnami pgbouncer wit edoburu

replaced bitnami pgbouncer with edoburu pgbouncer container and updated environment variables to fit

* updated ports & Kubernetes

Updated ports to fit existing documentation. Also updated the Kubernetes Helm Chart link to use the same container.
2026-04-14 17:45:53 -07:00
Patrick Buckley 8068ae105d chore: bump version to 1.4.0a2 2026-04-14 11:17:06 -07:00
renovate[bot] 6e99bb8b0b chore(deps): update dependency hls.js to v1.6.16 (#354)
* chore(deps): update dependency hls.js to v1.6.16

* chore: download vendored hls.js files + add hls to workflow detection loop

The wheel-completeness check failed on the Renovate bump because
vendor-js.yml only iterated katex/hljs/mermaid — so hls.js PRs
never got their files auto-downloaded. Adding hls to the loop so
future Renovate bumps are merge-ready without manual intervention.

Also running the update now to fix this specific PR.

---------

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Patrick Buckley <buckleypm@gmail.com>
2026-04-14 11:15:36 -07:00
Patrick Buckley eb59cdefda feat: pass resolved capabilities through to providers, add server com… (#352)
* feat: pass resolved capabilities through to providers, add server compat layer

The LLMProvider protocol previously forced providers to re-derive
capabilities from static lookup tables, ignoring config overrides set
via the admin UI or config.toml (e.g. thinking_mode, token_param).
This adds an optional capabilities parameter to create_streaming and
create_completion so the session can pass its config-merged
ModelCapabilities through to providers.

On top of this, adds a server compatibility layer for local model
servers (vLLM, llama.cpp). Profiles suggest thinking mode and server
workarounds (skip_special_tokens for vLLM, reasoning_format for
llama.cpp) during model detection, with structured admin UI fields
for server type, thinking mode, and extra body params.

Verified against real vLLM (Gemma 4 31B) and llama.cpp (Gemma 4 E4B)
servers.

* fix: defensive copy in _finalize_extra_body, expose thinking_param in UI

Shallow-copy extra_params and its chat_template_kwargs in the provider
before _apply_thinking_mode mutates them, so callers that reuse the
same dict across models are safe.

Replace the hidden thinking_param input with a visible text field
that appears when thinking mode is enabled. Shows the default
"enable_thinking" and hints that Granite/DeepSeek use "thinking".

* fix: address Copilot review feedback on admin UI and server compat

- Preserve unrepresentable thinking_mode values (e.g. "adaptive") in
  raw capabilities JSON instead of silently dropping on edit round-trip
- Validate capabilities and extra body JSON are plain objects, not
  arrays or primitives
- Deep-merge chat_template_kwargs from extra_body instead of silently
  dropping, so operators can extend/override template kwargs

* fix: hide server compat section for non-local providers

The Server Compatibility fields (server type, thinking mode, extra
body) only apply to openai-compatible (local model servers). Hide
the entire section when the provider is openai, anthropic, or google.

* fix: normalize capsObj to plain object on edit load

Defend against DB rows where capabilities is a JSON literal null,
an array, or a primitive — previous code would crash on the
capsObj.server_compat / capsObj.thinking_mode reads. Same defensive
check also applied to the server_compat nested value.

* refactor: extract _isPlainObject helper for JSON type checks

Consolidates the null/array/typeof check that was inlined at three
different call sites into a single helper. Keeps the intent obvious
at each use site and avoids the awkward multi-condition ternary.
2026-04-14 11:05:51 -07:00
Patrick Buckley 06d7cf8896 chore: bump version to 1.4.0a1 2026-04-13 17:19:22 -07:00
Patrick Buckley 934cb075d6 feat: per-model sampling parameters (temperature, max_tokens, reasoni… (#350)
* feat: per-model sampling parameters (temperature, max_tokens, reasoning_effort)

Model sampling parameters were global-only settings applied uniformly to
all models. Different models have fundamentally different requirements
(o-series needs no temperature, Anthropic needs temp=1.0 with thinking,
local models may need different max_tokens). This adds per-model overrides
with global fallback so each model definition can specify its own defaults.

Migration 036 adds nullable temperature, max_tokens, reasoning_effort
columns to model_definitions. NULL inherits the global default from
ConfigStore. The session factory and /model switch command both resolve
per-model override → global fallback consistently.

The admin UI model create/edit modal now has dedicated form fields for
these parameters with client-side validation, a visual section divider,
and per-model override hints in the model table rows.

Removes vestigial model.name and model.context_window global settings
(now handled per-model by the model registry) with startup warnings for
existing config.toml users.

* fix: defensive parsing for config.toml per-model sampling params

Wrap temperature/max_tokens conversions in try/except with range
validation. Invalid values log a warning and fall back to None
(inherit global default) instead of aborting registry load.
2026-04-13 17:14:58 -07:00
Patrick Buckley a793d009fd fix: use gethostname() instead of getfqdn() for advertise URLs (#349)
* fix: use gethostname() instead of getfqdn() for advertise URLs

socket.getfqdn() does a reverse DNS lookup that often returns a
truncated hostname (e.g. "flat" instead of "flat-blck-io"). Use
gethostname() for advertise URLs in both server and console. For TLS
SANs, include both names so certs cover all variations.

* docs: clarify advertise URL comment re Docker/k8s
2026-04-13 14:52:12 -07:00
Patrick Buckley 2a05ba5915 fix: standardize database env vars on TURNSTONE_DB_* naming (#348)
* fix: standardize database env vars on TURNSTONE_DB_* naming

compose.yaml used DB_BACKEND/DATABASE_URL in .env which got mapped to
TURNSTONE_DB_BACKEND/TURNSTONE_DB_URL inside containers. Running bare-
metal required the TURNSTONE_ prefix, but docs didn't explain this.
Eliminate the indirection — use TURNSTONE_DB_BACKEND and TURNSTONE_DB_URL
everywhere (compose, .env, bare-metal, docs, bootstrap wizard).

* fix: update .env.example to use TURNSTONE_DB_* naming
2026-04-13 14:48:51 -07:00
Patrick Buckley cba379d994 chore: bump version to 1.3.0a3 2026-04-12 20:43:35 -07:00
Patrick Buckley 50e6e64c3d fix: universal tool_call/tool_result orphan detection for OpenAI-comp… (#346)
* fix: universal tool_call/tool_result orphan detection for OpenAI-compat providers

The Anthropic provider had orphan detection for mismatched tool_call ↔
tool_result pairs, but OpenAI-compatible providers (Chat Completions,
Google, Responses API) had none. When an Anthropic model runs behind
an OpenAI-compat API (e.g. Azure) or cancellation creates orphans,
the API rejects the malformed request.

- Rewrite sanitize_messages() with orphan detection: synthesize error
  tool results for unmatched tool_calls, drop tool results with no
  matching tool_call, fill empty tool_call IDs with positional remap
- Call sanitize_messages() from Responses API _convert_messages()

* fix: address review feedback on orphan detection

- Track answered IDs per-turn (local_answered) instead of scanning
  all of out, preventing false matches from reused IDs across turns
- Drop empty-ID tool results that have no remap entry instead of
  passing them through with invalid empty tool_call_id
- Increment empty_result_idx for every empty result, not just remapped
- Remove dead result_ids peek-ahead code
- Add test for repeated tool_call IDs across turns
2026-04-12 20:40:12 -07:00
Patrick Buckley 440e93846d fix: accurate token usage tracking for compaction across all providers (#345)
* fix: accurate token usage tracking for compaction across all providers

Anthropic's input_tokens excluded cached tokens, causing massive
under-reporting (e.g. 327 vs 9000 actual) when prompt caching was
active. This prevented auto-compaction from triggering.

- Normalize Anthropic prompt_tokens to total input (input_tokens +
  cache_creation + cache_read), matching OpenAI semantics
- Reset _last_usage per API call so tool-chain iterations get fresh
  usage instead of max()-merging with stale values
- Add mid-turn compaction check during tool chains to prevent context
  overflow before end-of-turn
- Anchor _remaining_token_budget() on provider-reported prompt_tokens
  with local estimates only for the delta since last API call
- Improve _msg_char_count() to include structural overhead (role,
  tool_call_id, tool call IDs) and handle image tokens in calibration
- Emit status after every API call, not just end of turn

* fix: defensive null coercion and index clamping from review feedback

- Add `or 0` to all getattr calls for input_tokens/output_tokens in
  Anthropic provider (streaming + non-streaming) to handle SDK nulls
- Use getattr for non-streaming input_tokens/output_tokens instead of
  direct attribute access for consistency
- Clamp _calibrated_msg_count with min() in _remaining_token_budget()
  to prevent stale state from over-slicing after compaction
2026-04-12 19:53:38 -07:00
renovate[bot] 0dd31e45ca chore(deps): update softprops/action-gh-release action to v3 (#343)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-04-12 19:09:16 -07:00
renovate[bot] 8c64ea0687 chore(deps): lock file maintenance (#344)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-04-12 18:57:03 -07:00
renovate[bot] bacb72a880 chore(deps): update ghcr.io/astral-sh/uv docker tag to v0.11.6 (#342)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-04-12 18:54:42 -07:00
renovate[bot] c75b66a630 chore(deps): update dependency vitest to v4.1.4 (#341)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-04-12 18:54:30 -07:00
renovate[bot] 6559976f2b chore(deps): update github actions (#340)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-04-12 18:54:19 -07:00
Patrick Buckley 83cfea36b0 chore: bump version to 1.3.0a2 2026-04-08 18:07:12 -07:00
Patrick Buckley 12516ffa04 fix(ui): remove broken hint animation and restore card toggle
The ws-check-hint animation clobbered the fadein's forwards fill,
making the checkbox invisible for 0.6s on card-body click — appearing
as a deselect-then-reselect. Remove the hint, the unused role=checkbox
on the card, and restore the original symmetric toggle behavior.
2026-04-08 18:06:54 -07:00
Patrick Buckley c33ad168c7 fix(ui): improve delete workstream UX and accessibility (#339)
* fix(ui): improve delete workstream UX and accessibility

Card body click no longer deselects (prevents confusing red border loss);
checkbox pulse hint guides users to deselect affordance. Adds keyboard
navigation, aria-labels, hover feedback, animations, and neutral Close
button styling after deletion.

* fix(ui): remove duplicate a11y checkbox from delete-mode cards

Hide the visual checkbox from the a11y tree and tab order so the card
(role=checkbox) is the sole keyboard/screen-reader target. Addresses
Copilot review feedback about nested interactive elements.
2026-04-08 17:18:59 -07:00
Patrick Buckley fd1fb7d849 chore(deps): bump lacme to >=1.0.5 (cryptography security update) 2026-04-08 16:48:06 -07:00
renovate[bot] 58b2d01b1c chore(deps): lock file maintenance (#338)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-04-08 16:10:27 -07:00
renovate[bot] b8440d70ac chore(deps): update ghcr.io/astral-sh/uv docker tag to v0.11.5 (#337)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-04-08 16:08:22 -07:00
renovate[bot] b2206337fe chore(deps): update dependency vitest to v4.1.3 (#336)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-04-08 16:08:09 -07:00
renovate[bot] fadb198898 chore(deps): update pypa/gh-action-pypi-publish digest to cef2210 (#335)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-04-08 16:07:50 -07:00
Patrick Buckley b1e78b79fb chore: bump version to 1.3.0a1 2026-04-07 00:38:16 -07:00
Patrick Buckley 98d3289852 chore: bump version to 1.2.0 2026-04-07 00:38:05 -07:00
Patrick Buckley 2025bf8a6f perf: reduce initial rebalance from ~1.5s to ~50ms on PostgreSQL (#334)
* perf: reduce initial rebalance from ~1.5s to ~50ms on PostgreSQL

Increase seed_ring_buckets chunk sizes (PG 500→16k, SQLite 500→8k) to
cut network round-trips from 131 to 5. Add ConsoleRouter.populate_from_assignments()
to build the routing cache directly from computed assignments, eliminating the
65 536-row DB read-back. Router becomes ready in <1ms; DB persistence follows.

* fix: address review — populate after seed write, sync router version

Move router cache population after seed_ring_buckets() so the router
is never "ready" with an unpersisted ring. Pass the new rebalancer
version to populate_from_assignments() so check_version() on the
collector thread does not trigger a redundant 65 536-row refresh.
2026-04-07 00:35:55 -07:00
Patrick Buckley 100bb02e3b fix: stale ARIA attrs after promote, deferred DELETE on pre-ID dismiss
- Remove role="status" and aria-label during _promoteQueuedMessages
  so screen readers don't announce stale "queued" context
- Mark element with pendingDismiss when user dismisses before msg_id
  arrives; send deferred DELETE when the send response provides the ID
2026-04-06 22:35:23 -07:00
Patrick Buckley 2b3b229da6 fix: flush queued messages on normal completion (no tool calls)
If the model responds without tool calls, the main loop exits
immediately — no tool-result seam exists for advisory injection.
Queued messages were silently orphaned in the OrderedDict. Now
flushed as regular user messages before emitting idle state.
2026-04-06 22:34:00 -07:00
Patrick Buckley 76ecb99374 fix: queued message promote loop and dismiss behavior
Bug 1: Extract _promoteQueuedMessages() — removes badge, dismiss
button, queued classes, and data-msgId. Called from setBusy(false)
on state_change: idle.

Bug 2: _dequeueMessage no longer removes the DOM element when server
returns not_found (message already injected). Only removes on
"removed" (actually dequeued). Network errors also preserve the
element. The promote loop handles cleanup on idle instead.
2026-04-06 22:28:21 -07:00
Patrick Buckley c578051cb8 feat: tool result advisory system with user message queuing (#333)
* feat: tool result advisory system with user message queuing

General-purpose advisory injection for tool results — when advisories
are present, tool output is wrapped in <tool_output> tags with
<system-reminder> blocks appended. Two initial producers:

- Output guard advisories: model sees why content was flagged/redacted
- User message interjections: users can queue messages mid-execution
  via the web UI, injected at the next tool-call seam

Queued messages use !!! prefix for important priority. Advisory
injection is gated by ModelCapabilities.supports_tool_advisories
(default true for commercial models, false for local/vLLM).

On cancel/error, queued messages are flushed as regular user messages
so nothing is silently lost. Raw tool output (pre-wrap) is persisted
to the DB to keep history clean of ephemeral advisory XML.

* fix: frontend UX for queued messages — rollback, discoverability, a11y

- Send button changes to "Queue" (outline style) during busy state,
  visually distinct from filled red Stop button
- Placeholder updates to hint at !!! priority convention
- addQueuedMessage returns element ref for optimistic UI rollback
- Remove queued element on queue_full, busy, or connection error
- Add role="status" and aria-label to queued message elements
- Promote queued messages to normal appearance when generation ends

* feat: queued message removal via dismiss button

Switch backing store from queue.Queue to OrderedDict + Lock for O(1)
removal by ID. Each queued message gets a UUID, returned to the
frontend and stored as data-msg-id on the DOM element.

Dismiss button (x) on queued messages calls DELETE /v1/api/send with
the msg_id. If the message was already injected (race), server returns
not_found and the UI removes the element anyway.

No new endpoint — DELETE method added to the existing /v1/api/send
route. dequeue_message() on ChatSession is O(1) under the lock.

* fix: address PR review — escaping, types, list output, message cap

- Escape </tool_output> and <system-reminder> in tool output to prevent
  wrapper tag injection from untrusted tool results
- Change _collect_advisories return type from list[Any] to list[ToolAdvisory]
- Drain queued messages on list/structured output (append as text part)
  so they aren't silently stuck until a str result appears
- Cap queued message length at 2000 chars to prevent context bloat
- Remove unused var in _dequeueMessage
2026-04-06 21:51:47 -07:00
Patrick Buckley 701c3fc717 chore: bump version to 1.2.0a5 2026-04-06 15:52:05 -07:00
Patrick Buckley 92ad5bd439 Feat/tab action dropdown (#332)
* feat: replace workstream action buttons with per-tab dropdown menu

Move refresh-title, edit-title, fork, close, and delete actions from
the header toolbar into a dropdown menu on each workstream tab,
triggered by a ▾ chevron that replaces the × close button.

Dropdown follows the existing pane context menu pattern: keyboard
navigation, mutual exclusion, click-outside/Escape dismiss, toggle
on re-click, aria-expanded + aria-haspopup, and focus restoration.

Delete is visually distinct (red text + wash + red focus ring, 6px
separator). Mobile hides "Refresh title" and sizes the chevron to
36px touch targets.

Removes updateWsActionButtons(), _applyTitleButtonState(), and
_wsTitleState tracking (dead code after button removal).

* fix: remove Ctrl+Shift+R shortcut that overrides browser hard refresh

Refresh title is a low-frequency action accessible from the tab
dropdown; no replacement keybind needed.

* fix: address tab dropdown review findings

- Pass wsId through dropdown actions so they target the correct
  workstream even when opened on a non-active tab
- Fix setTimeout race where closeTabDropdown before timeout fires
  could leave stale listeners
- Guard Close and Delete on last workstream (dropdown, keyboard
  shortcuts, and defense-in-depth in confirmDeleteWorkstream)
- Use aria-disabled instead of disabled so screen reader users can
  discover unavailable items via arrow keys
- Enlarge chevron hit target, add hover affordance with subtle
  background highlight
- Add 0.1s dropdown open animation (respects prefers-reduced-motion)
2026-04-06 15:48:13 -07:00
Patrick Buckley 58c81b2b46 fix: resolve CodeQL double-import findings in test files (#331) 2026-04-06 14:18:02 -07:00
Patrick Buckley a2d4598012 fix: address CodeQL findings — BaseException and empty except (#330)
- server.py: catch (Exception, GenerationCancelled) instead of
  BaseException so KeyboardInterrupt/SystemExit propagate normally
- judge.py: log client close failures instead of bare pass
2026-04-06 13:53:26 -07:00
renovate[bot] 4f83dba1b9 chore(deps): lock file maintenance (#326)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-04-06 13:37:33 -07:00
dependabot[bot] 2629f217d2 chore(deps-dev): bump vite from 8.0.4 to 8.0.5 in /sdk/typescript (#329)
Bumps [vite](https://github.com/vitejs/vite/tree/HEAD/packages/vite) from 8.0.4 to 8.0.5.
- [Release notes](https://github.com/vitejs/vite/releases)
- [Changelog](https://github.com/vitejs/vite/blob/main/packages/vite/CHANGELOG.md)
- [Commits](https://github.com/vitejs/vite/commits/v8.0.5/packages/vite)

---
updated-dependencies:
- dependency-name: vite
  dependency-version: 8.0.5
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-04-06 13:14:32 -07:00
Patrick Buckley d1162b2eb9 fix: preserve Gemini thought_signature via provider_blocks fidelity lane (#328)
Gemini's OpenAI-compat endpoint requires thought_signature to survive
the tool-call round-trip. Previously dropped because the Chat Completions
provider cherry-picks only standard fields (id, type, function).

Fix: GoogleProvider now captures raw tool-call dicts (including
thought_signature) via provider_blocks — the same fidelity lane the
Anthropic provider uses for signature round-tripping. On the next turn,
_prepare_messages reconstructs tool_calls from the stored raw data and
strips _provider_content so it never reaches the wire.

Changes:
- _openai_chat.py: add _prepare_messages and _extract_tool_calls hooks
- _google.py: override hooks + tap-pattern _iter_stream for streaming
- model_registry.py: auto-detect .googleapis.com → google provider
- session.py: read cancel_on_approval from ConfigStore
- console/server.py: add PUT/DELETE to proxy route methods
- server.py: fix fork naming (don't inherit source display name)
2026-04-06 13:13:38 -07:00
Patrick Buckley 217688547e fix: expose channel gateway port for bare-metal deploys
The channel gateway registers with its Docker-internal hostname
(e.g. http://channel:8091) which is unreachable from a host-side
server. Publish port 8091 and set TURNSTONE_CHANNEL_ADVERTISE_URL
to localhost so the server can reach it for schedule notifications.
2026-04-06 10:47:50 -07:00
Patrick Buckley 5dc98f75fb fix: scheduled task notifications not delivered on cancellation
GenerationCancelled extends BaseException, not Exception, so it bypassed
the except handler in _run_initial. The finally block ran but
_extract_last_assistant_content returned "" (response never appended to
messages), and _fire_notify_targets bailed on the empty content guard.

Fixes:
- Catch BaseException (not just Exception) in _run_initial so
  GenerationCancelled is handled and the UI state is cleaned up
- Remove the empty-content suppression in _fire_notify_targets —
  scheduled tasks should always deliver, even with a fallback message
  when no output was captured
2026-04-06 09:54:03 -07:00
renovate[bot] 6980ba5aae chore(deps): lock file maintenance (#325)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-04-06 04:39:13 -07:00
Patrick Buckley 57912faa52 chore: bump version to 1.2.0a4 2026-04-06 03:58:42 -07:00
Patrick Buckley 0625fac87b fix: shorten judge model dropdown default label 2026-04-06 03:57:21 -07:00
Patrick Buckley dc3a1b7a64 fix: workstream toolbar UX — relocate to tab bar, fix visibility and theme sync
- Move action buttons (refresh/edit/fork/delete) from header to tab bar,
  grouped in #ws-action-group with separators. Contextually adjacent to
  the workstream tabs they operate on.
- Toggle group visibility via CSS class (.hidden) instead of per-button
  inline style.display — makes media query overrides reliable.
- Call updateWsActionButtons() from renderTabBar() so buttons appear on
  initial load and ws_created, not just on tab switch.
- Fix theme loss between nodes: loadInterfaceSettings no longer overwrites
  localStorage with server defaults — preserves user's theme choice when
  switching nodes via console proxy.
- Add flex-shrink:0 on +/split buttons to prevent squeeze with many tabs.
2026-04-06 03:54:03 -07:00
Patrick Buckley a3140da3a5 docs: update documentation for PRs #312-#316 (#324)
- README: add Google Gemini to multi-provider feature list and requirements
- architecture.md: add GoogleProvider, update supported provider values,
  file listing, config example
- judge.md: document cancel_on_approval, fresh-client lifecycle, fallback
  delivery, Google compatibility
- settings.md: add judge.cancel_on_approval, new interface.* section
  (close_tab_action, theme), update total count
- api-reference.md: document 6 new workstream/settings endpoints,
  add judge_model to workstreams/new
- console.md: add judge model to modal fields, add keyboard shortcuts
- console_schemas.py: add judge_model field to ConsoleCreateWsRequest
- server_spec.py: add 6 new EndpointSpec entries
- diagrams: add GoogleProvider to package structure and class diagram
2026-04-06 03:43:12 -07:00
Patrick Buckley 8838bd0f8d fix: apply model.default_alias on model-reload by refreshing ConfigStore
The model-reload handler read model.default_alias from ConfigStore's
in-memory cache, which could be stale if the earlier best-effort
config-reload notification failed or hadn't arrived yet. Force a
cs.reload() from DB before reading the alias. Also publish config
changes from the console before dispatching model-reload, and
downgrade the misleading "No 'default' model alias" log to debug.
2026-04-06 03:37:35 -07:00
Patrick Buckley 7f63cd2d33 feat: add keyboard shortcuts for workstream actions (#323)
Ctrl+Shift+R  Refresh title (regenerate via LLM)
Ctrl+Shift+E  Edit title
Ctrl+Shift+F  Fork workstream
Ctrl+Shift+X  Delete workstream (X not D — avoids Chrome DevTools conflict)

Shortcuts are blocked when any modal is open (edit-title, delete-ws,
batch-delete, new-ws). Help dialog (?) updated with the new bindings.
2026-04-06 03:11:06 -07:00
Patrick Buckley 24f59a6c53 feat: add per-node metadata with auto-collection, admin API, and cons… (#318)
* feat: add per-node metadata with auto-collection, admin API, and console UI

Adds a normalized node_metadata table for structured per-node key/value
metadata with source tracking (auto/user/config).  Auto-populated fields
(hostname, OS, arch, interfaces, cpu_count) are collected at server startup
via stdlib; user-defined fields are managed through the admin API, CLI, or
config.toml [metadata] section.

Storage: migration 035, 7 new protocol methods (get, get_all, set,
set_bulk, delete, delete_by_source, filter), both SQLite and PostgreSQL
backends.  Filtering uses single-query GROUP BY/HAVING for efficiency.

Console API: GET/PUT/DELETE endpoints under /admin/nodes/{node_id}/metadata
with auto-source protection.  cluster_nodes gains meta.* query param
filtering; cluster_node_detail attaches metadata to responses.

Frontend: new Nodes admin tab with collapsible per-node sections, inline
add form, delete with confirmation.  Read-only metadata panel in node
detail drill-down.  Proper design token usage, accessibility (ARIA,
keyboard nav, screen reader labels), and mobile responsiveness.

CLI: turnstone-admin list-node-metadata, set-node-metadata, and
delete-node-metadata subcommands.

64 tests (25 storage, 19 node_info, 20 existing unaffected).

* fix: resolve CI typecheck and test failures

- Fix mypy error: use %-style format string instead of structlog kwargs
  for standard Logger.warning() in console server
- Fix test_get_nodes assertion to include new node_ids=None parameter
- Add debug logging to _collect_interfaces empty except block

* fix: address Copilot review feedback on node metadata

- Clear stale auto/config metadata before upserting on startup
- Wrap metadata filter in try/except with graceful fallback
- Add metadata field to NodeDetailResponse schema
- Use _VALID_NODE_ID regex for consistent node_id validation
- Defensive JSON decode in admin_get_node_metadata
- Switch to read_json_or_400 and require_storage_or_503 helpers
- Add SetNodeMetadataValueRequest for single-key PUT endpoint
- Add bulk GET /admin/node-metadata endpoint (replaces N+1 fetches)
- Update frontend to use single bulk metadata fetch

* feat: add admin.nodes permission scope for node metadata

- Add admin.nodes to builtin-admin role via migration 035
- Switch all node metadata handlers from admin.settings to admin.nodes
- Register admin.nodes in the admin panel permission set
- Node detail metadata panel fetches from cluster endpoint (no admin
  permission needed) instead of admin endpoint

* fix: address second round of Copilot feedback

- Replace inline onclick handlers with data-* attributes and event
  delegation to prevent JS string context XSS
- Move NodeMetadataEntry before NodeDetailResponse and use it as the
  typed metadata field (was list[dict[str, Any]])
- Clean up config metadata on shutdown (was only cleaning auto)
2026-04-06 03:08:19 -07:00
Patrick Buckley 5cbc4bc87c feat: bulk message insert for fork performance + endpoint tests (#322)
Add save_messages_bulk() to StorageBackend protocol and both backends.
Fork path now inserts all messages in a single transaction instead of
N individual save_message() calls — for a 200-message workstream this
goes from 200 connection/insert/commit cycles to 1.

FTS5 indexing is intentionally skipped for bulk fork data (historical
messages indexed on rebuild). Ordering preserved via auto-increment id
with a shared timestamp across all rows in the batch.

Also adds 22 endpoint tests covering the 6 new workstream management
endpoints (delete, open, title, refresh-title, list/update interface
settings) and 4 storage-level tests for the bulk insert path.
2026-04-06 02:54:34 -07:00
renovate[bot] eba2f29cd1 chore(deps): lock file maintenance (#320)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-04-06 02:40:55 -07:00
Patrick Buckley 66c856eb6e fix: post-merge follow-ups for PRs #312-#316 (#319)
Security:
- Add write scope rules for 4 new workstream POST endpoints
  (delete, open, refresh-title, title) in required_scope() —
  both direct and console-proxied paths

Judge:
- Restore cancel_event check in inner poll loop (was removed)
- Fix fallback delivery off-by-one: items[idx+1:] not items[idx:]
- Skip empty-response retry when finish_reason=="length"
- Reset empty_retries counter after non-empty response
- Document per-turn timeout semantics in JudgeConfig

Google provider:
- Add default base_url for Gemini endpoint in create_client()
- Bump max_output_tokens 8192→65536, set token_param="max_tokens"
- Add api_key detection for googleapis.com in console detect
- Add provider badge CSS (green) and openai-compatible (dim)

Theme:
- Fix POST→PUT for settings persistence (was silently 405-ing)
- Consolidate dual localStorage keys with backwards-compat read
- Lower banner z-index 9999→200, raise login overlay to 10001
- Fix undefined --bg-input, banner contrast for WCAG AA
- Add smooth theme transition with prefers-reduced-motion override
- Console onThemeChange: add title + aria-label updates

Workstream backend:
- Restore close_workstream 400 for last-ws case (was changed to 404)
- Thread-safe _llm_verdicts via _ws_lock on all mutation sites
- Fork: persist tool_calls + provider_data in save_message
- Add get_workstream_metadata to StorageBackend protocol
- Add ChatSession.request_title_refresh() public API
- Use cs.stored_keys() instead of cs._cache
- Redact exception text in delete 500 response
- web_helpers: catch-all logs and returns 500 not 400
- Live-stream ws_created SSE includes title field

Workstream UI:
- Focus traps + Escape on edit-title and delete-ws modals
- Tab close aria-label, mobile breakpoint for action buttons
- Restore name priority (live SSE over stale API)
- Fix double-delete, fork button text, batch delete handler leak
- Optimistic title update, close-last-tab error toast
- ws_id badge show-on-hover, hover states, aria-live, emoji a11y

Console admin:
- Banner aria-labels, judge dropdown wording, detect button class
- New-ws modal Escape handler, provider defaults cross-reference
2026-04-06 02:23:00 -07:00
Patrick Buckley 40a560b39c Merge pull request #316 from sillyWillieBilly/feat/console-enhancements
feat(console): theme-aware banner, judge model support, Google provider in admin
2026-04-06 00:56:39 -07:00
Patrick Buckley bc945852f7 Merge pull request #315 from sillyWillieBilly/feat/ui-enhancements
feat: UI enhancements — workstream management, title editing, fork, delete, theme sync
2026-04-06 00:56:36 -07:00
Patrick Buckley ca70e79d43 Merge pull request #314 from sillyWillieBilly/feat/workstream-management
feat: workstream management — fork, rename, delete, open, interface settings
2026-04-06 00:56:34 -07:00
Patrick Buckley ebcfb56f0e Merge pull request #313 from sillyWillieBilly/feat/judge-improvements
feat: harden judge with fresh-client lifecycle, fallback delivery, and Google compatibility
2026-04-06 00:56:26 -07:00
Patrick Buckley 33d29e3316 Merge pull request #312 from sillyWillieBilly/feat/google-provider
feat: add Google (Gemini) provider adapter
2026-04-06 00:56:08 -07:00
renovate[bot] bfda91cd25 chore(deps): lock file maintenance (#317)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-04-06 00:19:40 -07:00
William 6fe9f75c3c feat(console): theme-aware banner, judge model support, Google provider in admin
- Replace inline-style console banner with CSS classes + light/dark theme
- Node ID in banner is now a clickable link back to the node UI
- Add judge_model parameter to create_workstream flow
- Add Google to model provider list with default URL
- Provider-specific placeholder hints in model editor
- Detect results populate model name suggestions datalist
- Theme changes in admin settings apply immediately
- Persist theme selection to server via settings API
- Use workstream title field (with name fallback) in collector SSE events
- Add judge model dropdown to new-workstream modal
2026-04-06 08:14:23 +02:00
William c093df274d feat: workstream management — fork, rename, delete, open, interface settings
Add workstream forking (resume with fork=True keeps new ws_id), custom
naming via aliases, title refresh via LLM, and workstream deletion.

New server endpoints: delete, refresh-title, set-title, open-workstream,
list/update interface settings.  Verdict caching with SSE replay on
reconnect, display name fallback (alias→title→name) across all
endpoints, judge_model override per workstream, and settings_changed
broadcast on config reload.

New settings: judge.cancel_on_approval, interface.close_tab_action,
interface.theme.  Storage backends updated with name in
list_workstreams_with_history and new get_workstream_metadata method.
2026-04-06 08:14:18 +02:00
William 49cdb3d0d3 feat: UI enhancements — workstream management, title editing, fork, delete, theme sync
Add workstream action buttons in header (refresh title, edit title, fork,
delete) with supporting modals and keyboard shortcuts.

Workstream tabs: always-visible close button, ws_id badge, configurable
close-tab-action (last_used/nearest/dashboard) via interface settings.

Dashboard: batch delete mode with multi-select, saved workstream cards
with ws_id badge, open endpoint for resuming sessions.

Judge display: late-arriving verdict toast when DOM element is gone,
worst-case verdict glow across all tool calls in approval block.

Theme: server-persisted via admin settings API, real-time sync across
clients via SSE settings_changed events.

New workstream modal: judge model dropdown for per-workstream judge
model selection.
2026-04-06 08:14:14 +02:00
William 04c62f90ff feat: harden judge with fresh-client lifecycle, fallback delivery, and Google compatibility
- Create fresh HTTP client per evaluation run to avoid stale connections
- Store client factory args instead of client instance for on-demand creation
- Add cancel_on_approval config: when True, abort remaining items on user
  approval; when False (default), run all evaluations to completion
- Always deliver LLM verdicts via callback (or fallback when LLM returns None)
- Add _deliver_fallbacks helper for cancelled/incomplete evaluations
- Skip read-only tools for Google provider (requires thought_signature)
- Flatten conversation history to plaintext transcript in _prepare_context
  to avoid multi-turn role sequence errors with strict providers like Google
- Use per-turn timeout instead of shared budget so slow turns don't starve
  later ones
- Add empty-response retry logic (up to 3 retries without consuming turns)
- Enhanced structured logging throughout judge pipeline
- Update tests to match new signatures and behavioral changes
2026-04-06 08:14:09 +02:00
William 1bbaf50214 feat: add Google (Gemini) provider adapter
Add GoogleProvider that extends OpenAIChatCompletionsProvider for
Gemini models via the OpenAI-compatible /v1beta/openai/ endpoint.

- New _google.py with 2M context window defaults and vision support
- Lazy-initialized singleton in create_provider() (thread-safe)
- Route 'google' through OpenAI SDK in create_client()
- Return empty list from list_known_models() (Google models change frequently)
2026-04-06 08:14:05 +02:00
renovate[bot] 38e49b6f9c chore(deps): lock file maintenance (#311)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-04-05 22:10:44 -07:00
Patrick Buckley 99b0e8db12 chore: bump version to 1.2.0a3 2026-04-05 18:21:22 -07:00
Patrick Buckley d22f5a4baf feat: reconcile judge admin rule UX with edit, disable, and reset act… (#310)
* feat: reconcile judge admin rule UX with edit, disable, and reset actions

Replace the misleading "Customize" button on built-in rules with a
logically consistent 4-state action model: pure built-in (Disable/Edit),
overridden built-in (Disable/Edit/Reset), disabled built-in
(Enable/Edit/Reset), and custom rule (Enable-Disable/Edit/Delete).

Add edit modals for both heuristic rules and output guard patterns,
reusing the existing create modal form structure. Introduce amber
"Reset" button styling to visually distinguish reversible resets from
permanent deletes. Fix source badge redundancy (disabled built-ins now
show grey "built-in" in SOURCE, red "disabled" in STATUS only). Add
aria-labels and role="listitem" for screen reader support.

* fix: preserve built-in pattern_flags and priority on override

Derive pattern_flags from compiled regex for built-in output guard
patterns in the list API so IGNORECASE and other flags survive the
disable/edit/override round-trip. Carry priority through edit modals
via hidden fields so built-in evaluation order is preserved.
2026-04-05 18:19:47 -07:00
renovate[bot] da5eae5352 chore(deps): update dependency katex to v0.16.45 (#309)
* chore(deps): update dependency katex to v0.16.45

* chore: download vendored JS files

---------

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-04-05 17:57:43 -07:00
Patrick Buckley adb42c66da feat: deliver scheduled workstream results to Discord on completion (#308)
When a scheduled workstream finishes execution, deliver the final
assistant response to configured Discord channels/users via the
existing channel gateway notify infrastructure.

- Add notify_targets column to scheduled_tasks (migration 034)
- Add notify_targets field to Workstream dataclass
- Storage: accept/return/update notify_targets in protocol, SQLite, PostgreSQL
- Server: validate targets, extract last assistant content, deliver via
  gateway with retry, post-completion hook in _run_initial finally block
- Schedule targets override skill notify_on_complete (dedup rule)
- SDK: notify_targets param on async + sync create_workstream
- Console scheduler: pass notify_targets through dispatch
- Console server: schedule CRUD accepts/validates/returns notify_targets
- API schemas: notify_targets on schedule + workstream request/response
- Admin UI: notify textarea in schedule create/edit modals with JSON
  validation, monospace font, aria-describedby hints
- Governance UI: notify_on_complete textarea in skill create/edit with
  client-side JSON validation and field reset on create
- Bounds: max 10 targets, 256 char field limit, gateway response body
  verification matching _exec_notify pattern
- Gateway: 30s asyncio.wait_for timeout on adapter.send to prevent
  hung Discord API calls from blocking the notify endpoint indefinitely
- 39 new tests covering validation, extraction, delivery, dispatch,
  CRUD, and adapter timeout
2026-04-05 17:18:26 -07:00
Patrick Buckley 7968f1b361 feat: auto-invalidate JWT and static assets on version upgrade (#307)
* feat: auto-invalidate JWT and static assets on version upgrade

Add a `ver` claim (major.minor) to user-facing JWTs so tokens from
previous versions are rejected after upgrade, triggering re-login.
Service tokens are excluded for rolling-deployment safety. Tokens
without a `ver` claim (pre-upgrade) are accepted for backward compat.

Inject `?v={__version__}` query strings into static asset URLs at
startup so browsers fetch fresh JS/CSS after any release. Vendored
libraries (KaTeX, Highlight.js, etc.) are skipped since they already
carry version numbers in directory paths. HTML responses now include
`Cache-Control: no-cache` to ensure browsers always revalidate.

Frontend detects upgrade-specific 401s and shows a contextual subtitle
("The server was updated — please sign in again"), then performs a full
page reload after re-auth to load the new versioned assets.

* refactor: address PR review — public API name, single decode, idempotent regex

Rename _version_slot() → jwt_version_slot() to make the cross-module
import explicit rather than relying on a private name.

Move version gating from validate_jwt() into check_request() via a new
AuthResult.token_version field. This eliminates the double JWT decode
that occurred on version-mismatch detection — the token is now decoded
once and the version compared afterward.

Guard version_html() regex against double-apply by excluding URLs that
already contain a query string ([^"?]+ instead of [^"]+).

* feat: structured version_mismatch code, ETag, cross-tab auth sync

Add structured "code": "version_mismatch" field to the 401 response
so the frontend detects upgrade-triggered re-auth without string
matching on the error message.

Add ETag headers to HTML index responses (server, console, and proxied
node UI). Combined with Cache-Control: no-cache, browsers send
conditional GETs and receive 304 between upgrades, saving bandwidth.

Add BroadcastChannel-based cross-tab auth sync so logging in on one
tab dismisses the login modal on all other tabs (and vice-versa for
logout).

Add a reminder to the vendored JS update script about the
version_html() regex lookahead.

* fix: remove unused import in test_web_helpers
2026-04-05 16:25:53 -07:00
Patrick Buckley 8de53f5cc1 feat: Discord /ask model alias, channel default setting, admin UX (#306)
* feat: Discord /ask model alias, channel default setting, admin UX

Add optional 'model' parameter to Discord /ask command with
autocomplete from available aliases. Model precedence:
explicit > channels.default_model_alias > CLI --model > server default.

- Add channels.default_model_alias to settings registry
- Extend /v1/api/models response with default_alias and
  channel_default_alias fields (both server and console)
- Add list_models() to async + sync SDK clients and ChannelRouter
- TTL-cached channel default in ChannelRouter (5min, fail-open)
- @mention path also respects channel default
- Admin Settings tab: model alias settings render as dropdowns
  populated from enabled model definitions
- Admin Settings tab: is_secret settings render as write-only
  password inputs with save button (replaces static label)
- Update OpenAPI schemas for new response fields
- Validate alias defaults against enabled models on both endpoints

* fix: address PR #306 review feedback

- Move TTL timestamp update before await in get_channel_default_alias
  to prevent concurrent duplicate fetches
- Add 30s TTL cache for list_models() to avoid per-keystroke HTTP
  traffic during Discord autocomplete
- Type SDK list_models() with ListAvailableModelsResponse instead
  of raw dict (both server and console, async + sync)
2026-04-05 15:08:21 -07:00
Patrick Buckley 8808a56801 Add tavily api key to config store and change is_secret tests 2026-04-05 13:09:36 -07:00
Patrick Buckley c071236927 chore: bump version to 1.2.0a2 2026-04-05 12:43:39 -07:00
Patrick Buckley 035ccb0603 fix: remove stale JudgeConfig field references and fix font sizing
- Fix IntentJudge.__init__() control flow: model override block was
  dangling inside try/except instead of being a separate branch
- Remove provider/base_url/api_key kwargs from server.py and cli.py
  JudgeConfig construction (fields removed in prior commit)
- Remove stale TOML mapping entries from config.py
- Remove --judge-provider CLI argument
- Fix Judge settings font sizes to match Settings tab (12px keys,
  11px descriptions, tighter spacing, --fg instead of --accent)
2026-04-05 12:38:21 -07:00
Patrick Buckley 2c050b2520 refactor: remove duplicate judge provider/base_url/api_key fields
Judge model config now uses model aliases exclusively via ModelRegistry.
The separate provider, base_url, and api_key fields on JudgeConfig were
redundant with what's already stored in model definitions. Removes the
fields from JudgeConfig, the explicit-provider resolution path from
IntentJudge.__init__(), and the 3 settings from the registry.
2026-04-05 12:15:38 -07:00
Patrick Buckley 72dd7b50bd fix: authFetch, r.ok checks, model picker race, mypy Mapping type
- Replace all raw fetch() + _adminToken with authFetch() helper
- Fix URL paths to use /v1/api/admin/judge/ prefix
- Add r.ok checks on all GET fetches (match existing tab pattern)
- Load model definitions before settings to fix picker race condition
- Escape secret input values with escapeHtml
- Use Mapping type for evaluate_output patterns param (mypy)
- Clean up stale blank lines and comment references
2026-04-05 02:14:58 -07:00
Patrick Buckley d7cac3716f Worktree feat configurable output guard (#305)
* feat: configurable judge rules with dedicated admin tab

Externalize heuristic intent validation rules and output guard patterns
from hard-coded module constants into the storage abstraction with full
admin UI CRUD. Introduces a dedicated Judge tab in the admin panel that
consolidates all judge configuration (scalar settings, heuristic rules,
output guard patterns) under a single admin.judge permission scope.

- Add heuristic_rules and output_guard_patterns tables (migration 033)
- Add RuleRegistry with thread-safe merge of built-in + DB rules
- Refactor output_guard.py patterns into structured OutputGuardPatternDef
- evaluate_heuristic() and evaluate_output() accept optional rules/patterns
- IntentJudge resolves model aliases via ModelRegistry
- 15 admin API endpoints under /api/admin/judge/ with regex validation
- Judge tab with Settings, Heuristic Rules, and Output Guard sub-panels
- Filter judge.* settings from generic Settings tab
- ConfigStore.storage public property for backend access

* fix: align Judge tab with admin panel design system

- Replace raw <table> with grid-based admin-row/admin-colheaders pattern
- Replace dynamic innerHTML modals with static overlays using focus traps
- Replace confirm() with styled showConfirmModal()
- Replace inline badge styles with scope-badge classes
- Add mobile responsive breakpoints for Judge tab grids

* fix: Judge tab accessibility and polish

- Extract sub-section switcher inline styles to CSS classes
- Add focus-visible outline and reduced-motion support
- Add tab button IDs and fix aria-labelledby on tabpanels
- Add tabindex roving and arrow key navigation for sub-tabs
- Add role=list and aria-live to table containers
- Replace status text with scope-badge classes for scannability

* fix: address CodeQL and Copilot review feedback

- Remove unused validation constants from rule_registry.py (CodeQL)
- Return MappingProxyType from output_patterns for immutability
- Fix ThreadPoolExecutor shutdown(wait=False) to prevent hangs
- Use separate _VALID_OG_RISK_LEVELS (no "critical") for output guard
- Pass pattern_flags to regex validation in update endpoint
- Chain redactions in configurable mode (compose pattern + complex)
- Initialize RuleRegistry on console app.state
- Fix test fixtures to use valid enum values (approve/review/deny)

* fix: use Mapping type for evaluate_output patterns param (mypy)
2026-04-05 01:53:33 -07:00
Patrick Buckley 2b93598d68 feat: multi-model health tracking with runtime default and DB-only st… (#304)
* feat: multi-model health tracking with runtime default and DB-only startup

Replace active-probe circuit breaker with passive per-backend health
tracking.  Backends are marked degraded after consecutive failures and
recover when a request succeeds — requests are never blocked.

- Add model.default_alias ConfigStore setting for runtime default model
- Make load_model_registry CLI args optional for DB-only startup
- Per-(provider, base_url) health trackers via HealthTrackerRegistry
- Two-pass fallback: prefer healthy backends, then try degraded
- Remove BackendHealthMonitor, CircuitState, probe threads, cooldown
- Remove circuit_state from API schema, SDK events, metrics, frontends

* feat: add "Set Default" button to Model Definitions admin panel

Show a "default" badge on the current default model alias and a
"set default" action button on all other models. Clicking it writes
model.default_alias via the settings API. The list endpoint now
includes default_alias in the response so the UI can highlight it.

* fix: address review feedback — metric scoping, effective default, session alias

- Move turnstone_backend_up metric out of BackendHealthTracker into
  server callback; only the effective default backend drives the gauge
- _build_health_dict resolves effective default via ConfigStore override
- session_factory computes selected_alias once before registry.resolve
- admin model-definitions endpoint returns effective default (not just
  override) so UI shows correct badge when ConfigStore is empty
- Rename circuitTitle → healthTitle in console JS
- Fix ruff SIM117 lint in test

* fix: validate effective default against enabled models, degraded label, log normalization

- admin model-definitions endpoint validates default_alias against
  enabled models using same fallback rules as load_model_registry
- UI text "backend down" → "backend degraded" to match advisory semantics
- Health tracker log uses normalized base_url from key, not raw argument
2026-04-04 23:44:29 -07:00
Patrick Buckley 9d4d7a5346 fix: add admin.prompt_policies to valid permissions and builtin-admin role (#303)
Migration 031 created the prompt_policies table but never registered
admin.prompt_policies in _VALID_PERMISSIONS or granted it to the
builtin-admin role, causing 403 on all prompt-policy admin endpoints.
2026-04-04 22:19:03 -07:00
Patrick Buckley 0e3788a54f docs: update release tracks table for 1.1.0 stable / 1.2.0a1 experimental 2026-04-04 19:19:32 -07:00
Patrick Buckley 7f3d6c4da1 chore: bump version to 1.2.0a1 2026-04-04 19:18:53 -07:00
Patrick Buckley b30e1394e0 chore: bump version to 1.1.0 2026-04-04 19:18:22 -07:00
Patrick Buckley af0bf5270c chore: update bootstrap example version to 1.1.0 2026-04-04 19:18:08 -07:00
Patrick Buckley d100ac92d9 fix: capacity-aware tool output truncation and context overflow recovery (#301)
* fix: capacity-aware tool output truncation and context overflow recovery

Large tool results (e.g. 593K-char search output) could overflow the
context window in a single turn when the conversation was already
partially full.  The fixed 50%-of-context truncation limit didn't
account for current usage.

Changes:
- _truncate_output() now accepts remaining token budget and uses
  min(tool_truncation, remaining_budget_chars) as the effective limit
- _remaining_token_budget() helper calculates available capacity with
  reserves for max_tokens response and 5% safety margin
- Safety truncation at tool-result append: every string tool result is
  clamped to remaining budget before entering the message array
- _exec_web_search() now calls _truncate_output() (was missing)
- Context overflow recovery: catches provider errors indicating context
  length exceeded (OpenAI + Anthropic patterns), auto-compacts, retries
  once.  Falls back to original error if compact-and-retry fails.

* fix: address review — zero-budget floor, nested spinner, Anthropic patterns, tests

- Remove 256-char floor from budget truncation — zero budget now returns
  a placeholder instead of allowing 256 chars through
- Stop thinking spinner before compact to avoid nested start/stop
- Add Anthropic error patterns (prompt is too long, input tokens)
- Wrap compact-and-retry so failures re-raise the original error
- Add 15 tests covering budget calculation, capacity-aware truncation,
  and overflow recovery for both providers

* fix: cap response reservation at 25% of context window

Reserving the full max_tokens in _remaining_token_budget() zeroed the
budget for common configs like max_tokens=32768 on a 32K context,
collapsing all tool output to a placeholder.  max_tokens is a ceiling,
not guaranteed consumption — cap the reserve at context_window // 4.

Adds regression test for max_tokens >= context_window.
2026-04-04 19:11:07 -07:00
renovate[bot] 57df445224 chore(deps): lock file maintenance (#302)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-04-04 19:06:59 -07:00
Patrick Buckley f978e7facd fix: skip chat_template_kwargs for commercial OpenAI API (#297)
* fix: skip chat_template_kwargs for commercial OpenAI API

OpenAI rejects chat_template_kwargs as an unknown parameter — it's only
meaningful for local model servers (vLLM, llama.cpp, SGLang).

Split OpenAIProvider into separate singletons for "openai" vs
"openai-compatible" so _provider_extra_params can gate on provider_name
instead of inspecting base_url. Also deduplicates agent inline code into
the same method and fixes pre-existing test pollution where
get_capabilities was mutated on the singleton without cleanup.

* feat: add OpenAI Responses API provider for commercial models

Split the OpenAI provider into three concrete implementations behind the
LLMProvider protocol:

- _openai_chat.py: Chat Completions API for local model servers
  (vLLM, llama.cpp, SGLang)
- _openai_responses.py: Responses API for commercial OpenAI
  (GPT-5.x, O-series)
- _openai_common.py: shared capability table, temperature/reasoning
  gating, cache retention, citations, usage extraction

The Responses API handles reasoning_effort as a {"effort": value} dict,
system messages as an instructions field, and tool format translation at
the provider boundary. ChatSession is unchanged — the provider abstracts
the API difference.

Also fixes diff_file direction when comparing against provided content.

* fix: Responses API input format and local model provider routing

- Assistant input messages use plain string content (not output_text)
- Tool call argument deltas match on item_id, not call_id
- Auto-detect openai-compatible provider for non-api.openai.com URLs
- Fix diff_file direction when comparing against provided content

* fix: resolve env vars before provider auto-detection in config.toml models

Config-file model entries using ${ENV_VAR} placeholders in base_url were
not resolving env vars before _resolve_openai_provider(), causing
commercial OpenAI configs to be misclassified as openai-compatible.
2026-04-04 18:36:37 -07:00
Patrick Buckley 0872f5f5ba fix: add intent comments to intentionally-empty except blocks (#300)
Annotate 20 empty except-pass blocks with brief explanations so
CodeQL's empty-except rule recognizes them as deliberate: optional
imports, JSON parse fallback chains, SSE poll timeouts, best-effort
fetches, and defensive datetime/float parsing.
2026-04-04 18:13:24 -07:00
Patrick Buckley 205e7818f8 Fix/codeql quality findings (#299)
* fix: replace empty except blocks with diagnostic logging

Add log.debug/warning to 7 bare except-pass blocks that silenced
failures in security-relevant or operationally-important paths:
- Channel route lookup, CLI policy evaluation, OIDC JWKS fetch,
  prompt policy loading, plan file write, routing override, username
  resolution.

Plan write now reports failure to user instead of falsely claiming
"Plan saved."

* fix: replace assert-with-side-effect and narrow BaseException catch

- Convert 4 assert isinstance() to explicit TypeError raises — assertions
  are stripped under python -O, removing runtime type checks
- Narrow except BaseException to except Exception in fallback handler —
  KeyboardInterrupt/SystemExit should not record as health failures
- Plan write failure now reports error to user instead of "Plan saved"

* fix: wire up toast error type and remove useless conditional

- showToast() now accepts optional type param ("error") with red border
  styling — 3 call sites were passing "error" that was silently ignored
- Remove always-true if (q) guard after early-return on empty query

* fix: remove unreachable return None after return self._judge

* fix: parenthesize multi-line string concatenations in dev_parts list

Explicit parens make intentional concatenation unambiguous to static
analysis (CodeQL implicit-string-concatenation-in-list rule).

* fix: remove constant-true filter in test mock — return list directly

* fix: extract side-effecting calls from assert in tests

store.delete() and mgr.close() have side effects that would be
stripped under python -O. Assign to variable first, then assert.

* fix: remove unused local variables in tests

Drop assignments to unused workstream/variable references created
solely for side effects. Use _ for unused tuple unpacking.

* fix: use admin.prompt_policies permission for prompt policy endpoints

All 5 prompt-policy endpoints (list, create, get, update, delete)
were checking admin.policies (the tool-policy permission) instead of
admin.prompt_policies. This caused a mismatch with the admin UI which
gates the tab on admin.prompt_policies — users could see the tab but
get 403, or reach the endpoint but never see the tab.

* fix: use caplog instead of capsys for structlog warning assertion

structlog output goes through the logging system, not stdout/stderr.

* fix: address review — remove dead isinstance, module-level import, unnecessary lambdas

- session.py: remove unreachable isinstance check (has_batch already
  validates raw_edits is a list)
- cli.py: move logging import to module level
- test_workstream.py: replace lambda wid: FakeUI(wid) with FakeUI
2026-04-04 17:53:20 -07:00
Patrick Buckley caf449e048 fix: address code scanning alerts — URL sanitization, workflow harden… (#298)
* fix: address code scanning alerts — URL sanitization, workflow hardening, XSS

- CI workflow: add top-level permissions (contents: read)
- Docker publish: gate on head_repository == self to block fork-based pwn
- URL checks: replace substring matching with proper hostname parsing
  (eval.py, model_registry.py, console/server.py)
- renderer.js: allowlist URL schemes (http/https) for images and links
- app.js: escape backslashes before quotes in CSS selector construction

* fix: break CodeQL taint chain — normalize image URL via URL constructor

* fix: address review — scheme-less URL handling, protocol-relative rejection, data:image allowlist

- Normalize scheme-less base URLs before hostname parsing (eval, model_registry,
  console/server) so api.openai.com without https:// still matches
- Reject protocol-relative URLs (//host) in image and link allowlists
- Allow data:image/ URIs for inline MCP resource images
- Tighten image source to https:// only (no relative paths)

* fix: route data: URIs through URL constructor to break CodeQL taint chain
2026-04-04 16:52:46 -07:00
Patrick Buckley 2bfc0f2c5d fix: harden MCP client against misbehaving servers (#296)
* fix: harden MCP client against misbehaving servers

Misbehaving/failed/misconfigured MCP servers could peg CPU at 100% due
to anyio cancel-scope busy-loops (SDK #2147), uncancelled orphaned
futures, and missing application-layer resilience.

Five fixes:

1. Cancel orphaned futures on timeout — future.cancel() in all sync
   bridge methods prevents coroutine accumulation on the event loop

2. Per-server circuit breaker — 3-failure threshold with exponential
   cooldown (30s–5min), per-server jitter, auto-reconnect on half-open
   probe, McpError excluded (protocol errors from healthy servers)

3. Safe transport stream pre-close — store stream refs and close them
   before stack teardown in all error/shutdown paths, preventing the
   anyio zero-buffer CPU busy-loop

4. Notification debounce — 5s per-server rate limit on list_changed
   refresh storms from buggy servers

5. Periodic refresh backoff with auto-reconnect — disconnected servers
   get reconnection attempts with exponential backoff (60s–1hr) instead
   of being silently skipped forever

* docs: add MCP resilience section to architecture docs and diagram

Document the circuit breaker, future cancellation, stream pre-close,
notification debounce, and periodic refresh backoff in the architecture
guide and the MCP architecture PlantUML diagram.

* fix: address review — stack leak on transport error, half-open comment

- Widen _connect_one guard to check _per_server_stacks too, not just
  _sessions. Transport errors in sync dispatch methods evict the session
  but left the stack behind, leaking anyio tasks on reconnect.
- Clarify half-open design: multiple callers are intentionally allowed
  through (reconnects serialize on the event loop, first failure re-trips).
2026-04-04 16:06:42 -07:00
Patrick Buckley c67aba0127 fix: mobile UX for console sidebar drawer and server chat input (#295)
* fix: mobile UX for console sidebar drawer and server chat input

Console admin sidebar: add box-shadow elevation, close button with
focus return, 44px touch targets, focus-into-drawer on open, flip
active indicator to left border, cubic-bezier easing, aria-expanded,
fix resize handler state desync, guard toggle injection for panels
without toolbars.

Server chat input: on touch devices Enter inserts newline (tap Send
button to send), hide Shift+Enter hint from placeholder.

* fix: preserve first group label spacing when close header is injected

Add sibling combinator selector so the first sidebar group keeps its
reduced top padding regardless of whether the close header div is
present as first-child.
2026-04-04 14:52:37 -07:00
Patrick Buckley db0baefeb2 feat: render rich media embeds for MCP tool results (#292)
* feat: render rich media embeds for MCP tool results

Detect structured media JSON (stream_url, results, sessions) in MCP
tool output and render interactive cards instead of plain text.

Web UI: media cards with thumbnail, title, metadata, and click-to-play
video/audio. HLS via lazy-loaded hls.js with direct-stream preference.
Collapsed raw JSON (API keys redacted) for inspection.

Discord: rich embeds with proxied thumbnail images (fetched by the bot
since Discord CDN cannot reach private media servers). Search results
as numbered lists, session state as "Now Playing" cards. Stream URLs
never exposed in embeds — web_url used for safe clickable links.

CI: vendor hls.js 1.6.15 with renovate tracking and update script.

* fix: address PR #292 review — SSRF guards, streaming fetch, tests

- URL validation: reject non-http(s) schemes and userinfo in thumbnail
  URLs. Private IPs intentionally allowed (media servers are on LAN).
- Streaming fetch: use http.stream() with aiter_bytes() and a running
  byte count to enforce the 2MB cap without buffering the full response.
  Validate content-type is image/* before downloading.
- Resilience: wrap try_build_media_embed in try/except in bot.py so a
  media embed failure falls through to the code-block path.
- LICENSE: download hls.js LICENSE from npm on update instead of only
  copying from old dir.
- Tests: add 19 new tests — try_parse_media (8 cases), _is_safe_image_url
  (7 cases), embed builders (4 cases including stream_url exclusion and
  string season/episode safety).

* chore: add LICENSE file for vendored hls.js

* fix: remove ANSI escape codes from tool preview fields

Preview text (tool args, URLs, queries) was wrapped in DIM/RESET ANSI
codes at the source in session.py, which leaked into SSE events and
rendered as raw escape sequences in Discord and the web UI.

Move ANSI styling to the CLI consumer (cli.py) where it belongs. Also
escape markdown in Discord tool name titles to prevent __ from being
interpreted as underline formatting.

* fix: drop [MCP: server] prefix from tool descriptions

The prefix made MCP tools look second-class compared to builtins,
causing models to hesitate using them. The server name is already
encoded in the tool name (mcp__server__tool).

* feat: pretty-print JSON tool output, player error state, broader key redaction

- JSON tool results are detected and pretty-printed with 2-space indent
  instead of rendering as a wall of text
- API key redaction extended to cover api_key, apiKey, api-key, and
  token query params across all tool output (not just media embeds)
- Video/audio player shows styled error message when stream fails to
  load instead of leaving a broken player element
- Both appendToolOutput and replayHistory use shared renderToolOutput()

* fix: designer review — player error retry, contrast, tool-cmd cap

- Player error: role="alert" for screen readers, retry button that
  reuses existing play handler, includes media title in error message
- Light theme: darken --red from #dc2626 to #b91c1c (5.7:1 contrast
  on --code-bg, was 4.3:1 failing WCAG AA at 12px)
- Pretty-print collapsed raw JSON in media embeds (was missed earlier)
- Cap .tool-cmd at 120px to prevent tools with many args from making
  approval blocks disproportionately tall in history replay
- Dedicated .media-player-error class instead of reusing .tool-output

* fix: Discord tool info name matching regression, suppress deprecation warning

The escape_markdown call on tool names was stored for matching against
ToolResultEvent.name, but event.name is raw/unescaped. The escaped name
never matched, so the "Running → Done" transition silently failed and
previews disappeared from the status embed.

Fix: store raw name for matching, use escaped name only for display.

Also suppress discord.py's re.sub count deprecation warning (Python
3.13+ issue, fixed upstream).

* fix: update MCP tool description tests to match prefix removal

* fix: address PR #292 review round 2

- Retry button: handle missing span children in click handler so retry
  buttons from player error state don't throw
- Footer count: use len(lines) instead of min(len(results), 10) to
  reflect actual rendered count after char budget truncation
- Null display: use "null" instead of "None" in JS tool arg preview
- Broader redaction: also redact JSON "api_key": "..." patterns
- SSRF hardening: block loopback and link-local IPs plus cloud metadata
  hostnames in thumbnail fetch (private LAN IPs still allowed)
2026-04-04 14:47:25 -07:00
Patrick Buckley 38fc933c1d fix: bundle production compose.yaml for pipx users (#293) (#294)
* fix: bundle production compose.yaml for pipx users (#293)

Users who install via pipx don't have a git clone, so there's no
compose.yaml or Dockerfile. Bootstrap now extracts a bundled production
compose file that uses pre-built ghcr.io images instead of local builds.

- Add turnstone/deploy/compose.yaml (ghcr.io images, no build blocks,
  single-node production profile only)
- Add write_compose tool to bootstrap wizard
- Update bootstrap system prompt to check for and write compose.yaml
- Remove stale ddgCluster profile references from system prompt
- Include turnstone/deploy/*.yaml in wheel

* fix: use postgresql+psycopg:// DSN scheme in compose fallbacks

The Docker image ships psycopg3, not psycopg2, so the bare
postgresql:// scheme fails. Also clarify PG usage comment in
production compose.
2026-04-04 12:26:10 -07:00
Patrick Buckley 39b39fb79d chore: bump version to 1.1.0a3 2026-04-03 15:41:15 -07:00
Patrick Buckley 0923add7db Fix/web fetch reliability (#290)
* fix: improve web_fetch reliability — strip scripts, dynamic truncation, more tokens

- strip_html() now removes <script>, <style>, <template>, <noscript>
  element content instead of just their tags
- Truncation budget scales with context window (75% in chars, 50k floor)
  and takes from the beginning only instead of head+tail splice
- max_tokens bumped from 2000 to 8192 so thinking models don't starve
  the visible extraction answer
- reasoning_effort="low" on summarization call to avoid wasting tokens
- Empty responses and empty extractions now report as tool errors

* refactor: extract _utility_completion to fix reasoning_effort duplication

Callers previously had to pass reasoning_effort both as a direct keyword
(for commercial providers) and via _provider_extra_params (for local
model servers).  This duplication was easy to get wrong — web_fetch was
already missing the direct keyword.

_utility_completion threads it through both paths from a single call,
used by title generation, compaction, and web_fetch extraction.

* fix: disable thinking when max_tokens too small, cap extraction at 500k

_reasoning_params now returns empty dict when max_tokens can't fit a
thinking budget (e.g. title gen with max_tokens=200).  Previously
produced budget_tokens >= max_tokens which is an API error on
manual-thinking Anthropic models.

Also caps web_fetch content truncation at 500k chars — the dynamic
context-window calc was producing 3M chars on 1M-context models.

* fix: clamp utility max_tokens to model output limit, add strip_html tests

_utility_completion now clamps max_tokens to the model's advertised
max_output_tokens so small/local models don't reject 8192-token
requests.

Adds 8 tests for invisible element stripping (script, style, template,
noscript) including multiline, case-insensitive, and attribute cases.

* fix: mock get_capabilities in title retry tests for _utility_completion

_utility_completion calls _get_capabilities to clamp max_tokens.  The
existing title tests mocked _provider as a bare MagicMock, so
caps.max_output_tokens was a truthy MagicMock instead of an int.  Set
get_capabilities to return a real ModelCapabilities instance.
2026-04-03 15:36:20 -07:00
Patrick Buckley 01cec062d9 fix: share single Docker image across all compose services
Build the image once via the profileless console service and reference
it as turnstone:local from server/channel.  Prevents stale images when
users run docker compose build without --profile.
2026-04-03 15:34:38 -07:00
Patrick Buckley 830eb8ba00 fix: include prompt .md files in wheel, add wheel-completeness CI (#289) (#291)
* fix: include prompt .md files in wheel, add wheel-completeness CI (#289)

Prompt markdown files were missing from PyPI wheels since the modular
prompts refactor, causing FileNotFoundError on startup for pip-installed
users.  Add the missing include pattern and a new CI job that diffs
source-tree data files against wheel contents so omissions are caught
before merge.

* fix: sanitise ALLOW patterns in wheel-completeness check

Strip blank lines and leading whitespace from the allowlist before
passing to grep -vFxf so empty patterns cannot silently match all lines.
2026-04-03 15:32:04 -07:00
Patrick Buckley 5bbf2e65eb fix: log clean one-liner when PostgreSQL becomes unavailable (#288)
* fix: log clean one-liner when PostgreSQL becomes unavailable

Wrap all 174 connection sites in PostgreSQLBackend through a _conn()
context manager that catches OperationalError, emits a single
database.unavailable log line (with connection URL), and suppresses
repeats until the connection is restored (database.connection_restored).

* fix: add StorageUnavailableError and cover all heartbeat loops

Address review feedback:
- Separate connect-phase from execution-phase in _conn() so that
  OperationalError during caller code (e.g. BEGIN IMMEDIATE lock
  contention) is not misclassified as a connectivity failure.
- Add StorageUnavailableError exception class so callers can
  distinguish transient DB outages without redundant tracebacks.
- Apply the same _conn() wrapper to SQLiteBackend for consistency.
- Catch StorageUnavailableError in all 7 periodic loops: watch
  runner, server heartbeat, channel heartbeat, console heartbeat,
  collector discovery, rebalancer, and scheduler.
- Guard dedup flag with threading.Lock.
- Add tests for dedup logging and PostgreSQL path.
2026-04-03 13:11:59 -07:00
Patrick Buckley 4d402fea6b chore: bump version to 1.1.0a2 2026-04-02 20:30:03 -07:00
Patrick Buckley 46d14ddd86 fix: chunk IN clauses to stay within DB parameter limits (#286)
* fix: chunk IN clauses to stay within DB parameter limits

psycopg caps query parameters at 65 535 and SQLite defaults to 999.
assign_buckets, prune_workstreams, and count_skill_resources_bulk were
passing unbounded lists into single IN(...) clauses, causing
OperationalError during rebalancer runs on full-size hash rings.

Chunk sizes: 10 000 (PostgreSQL), 500 (SQLite).

* fix: deduplicate assign_buckets input, add chunking regression tests

Address review feedback: deduplicate bucket list before chunking to
prevent inflated rowcount from cross-chunk duplicates. Add tests that
exercise the multi-chunk path (1200 buckets > SQLite chunk_size of 500)
and verify dedup preserves accurate counts.
2026-04-02 19:25:57 -07:00
renovate[bot] 7c4157f78d chore(deps): lock file maintenance (#285)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-04-02 17:28:00 -07:00
renovate[bot] 3856d80709 chore(deps): update github actions (#284)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-04-02 17:27:29 -07:00
Patrick Buckley 8142d2f1ad fix: add concurrency groups to publish workflows
Multiple CI completions for the same commit (tag push + branch push)
caused duplicate publish and docker runs. Concurrency group keyed on
head_sha ensures only one publish runs per commit.
2026-04-02 17:23:01 -07:00
Patrick Buckley c234d66ebf chore: bump version to 1.1.0a1 2026-04-02 17:10:06 -07:00
374 changed files with 84190 additions and 9543 deletions
+2 -2
View File
@@ -20,10 +20,10 @@ TURNSTONE_JWT_SECRET=changeme-to-32-bytes-of-hex
# -- Database ------------------------------------------------------------------
# Single-node default is SQLite (zero config). Set these for PostgreSQL:
# DB_BACKEND=postgresql
# TURNSTONE_DB_BACKEND=postgresql
# POSTGRES_USER=turnstone
# POSTGRES_PASSWORD=changeme
# DATABASE_URL=postgresql+psycopg://turnstone:changeme@postgres:5432/turnstone
# TURNSTONE_DB_URL=postgresql+psycopg://turnstone:changeme@postgres:5432/turnstone
# -- Ports ---------------------------------------------------------------------
# SERVER_PORT=8080
+9 -1
View File
@@ -41,6 +41,14 @@
"matchStrings": ["mermaid-(?<currentValue>[\\d.]+)/"],
"depNameTemplate": "mermaid",
"datasourceTemplate": "npm"
},
{
"customType": "regex",
"description": "Track vendored hls.js version",
"managerFilePatterns": ["/pyproject\\.toml$/"],
"matchStrings": ["hls-(?<currentValue>[\\d.]+)/"],
"depNameTemplate": "hls.js",
"datasourceTemplate": "npm"
}
],
"packageRules": [
@@ -91,7 +99,7 @@
{
"description": "Vendored JS — CI workflow downloads files automatically",
"groupName": "Vendored JS",
"matchPackageNames": ["katex", "highlight.js", "mermaid"],
"matchPackageNames": ["katex", "highlight.js", "mermaid", "hls.js"],
"schedule": ["before 9am on the first day of the month"],
"automerge": false
},
+52 -2
View File
@@ -7,6 +7,9 @@ on:
pull_request:
branches: [main, "stable/*"]
permissions:
contents: read
jobs:
lint:
runs-on: ubuntu-latest
@@ -42,7 +45,7 @@ jobs:
python-version: ${{ matrix.python-version }}
- run: pip install -e ".[test]"
- run: pytest tests/ -m "not live" --cov=turnstone --cov-report=term-missing --cov-report=xml -q
- uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7
- uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
if: always()
with:
name: coverage-${{ matrix.python-version }}
@@ -74,6 +77,53 @@ jobs:
env:
TURNSTONE_TEST_PG_URL: postgresql+psycopg://postgres:postgres@localhost:5432/turnstone_test
wheel-completeness:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
- uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6
with:
python-version: "3.14"
- run: pip install build
- run: python -m build --wheel
- name: Check all data files are in wheel
run: |
SOURCE=$(find turnstone -type f \
! -name '*.py' ! -name '*.pyc' ! -path '*__pycache__*' \
| sort)
WHEEL=$(python -m zipfile -l dist/*.whl \
| awk '{print $1}' \
| grep -v '\.py$' | grep -v '\.dist-info' | grep -v '\.pyc' | grep -v '^File$' \
| sort)
# Files intentionally excluded from the wheel (one per line)
ALLOW="
turnstone/core/storage/migrations/script.py.mako
"
MISSING=$(comm -23 <(echo "$SOURCE") <(echo "$WHEEL") \
| grep -vFxf <(echo "$ALLOW" | sed '/^[[:space:]]*$/d; s/^[[:space:]]*//' ) || true)
if [ -n "$MISSING" ]; then
echo "::error::Data files in source tree but missing from wheel:"
echo "$MISSING"
echo ""
echo "Add them to [tool.hatch.build.targets.wheel] in pyproject.toml"
echo "or to the ALLOW list in this job if intentionally excluded."
exit 1
fi
echo "All source data files present in wheel"
- name: Smoke-test entry points from installed wheel
run: |
python -m venv /tmp/smoke
/tmp/smoke/bin/pip install dist/*.whl
/tmp/smoke/bin/turnstone --help
/tmp/smoke/bin/turnstone-server --help
/tmp/smoke/bin/turnstone-console --help
/tmp/smoke/bin/turnstone-admin --help
/tmp/smoke/bin/turnstone-channel --help
/tmp/smoke/bin/turnstone-bootstrap --help
lock-check:
runs-on: ubuntu-latest
steps:
@@ -105,7 +155,7 @@ jobs:
working-directory: sdk/typescript
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
- uses: actions/setup-node@53b83947a5a98c8d113130e565377fae1a50d02f # v6
- uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6
with:
node-version: "24"
- run: npm ci
+10 -4
View File
@@ -5,6 +5,10 @@ on:
workflows: ["CI"]
types: [completed]
concurrency:
group: docker-${{ github.event.workflow_run.head_sha }}
cancel-in-progress: true
permissions:
contents: read
packages: write
@@ -15,7 +19,9 @@ env:
jobs:
docker:
if: github.event.workflow_run.conclusion == 'success'
if: >-
github.event.workflow_run.conclusion == 'success' &&
github.event.workflow_run.head_repository.full_name == github.repository
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
@@ -37,7 +43,7 @@ jobs:
- name: Log in to GHCR
if: steps.tag.outputs.skip == 'false'
uses: docker/login-action@74a5d142397b4f367a81961eba4e8cd7edddf772 # v3
uses: docker/login-action@4907a6ddec9925e35a0a9e82d7399ccc52663121 # v4
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
@@ -61,12 +67,12 @@ jobs:
fi
echo "tags=${TAGS}" >> "$GITHUB_OUTPUT"
- uses: docker/setup-buildx-action@b5ca514318bd6ebac0fb2aedd5d36ec1b5c232a2 # v3
- uses: docker/setup-buildx-action@4d04d5d9486b7bd6fa91e7baf45bbb4f8b9deedd # v4
if: steps.tag.outputs.skip == 'false'
- name: Build and push
if: steps.tag.outputs.skip == 'false'
uses: docker/build-push-action@14487ce63c7a62a4a324b0bfb37086795e31c6c1 # v6
uses: docker/build-push-action@bcafcacb16a39f128d818304e6c9c0c18556b85f # v7
with:
context: .
push: true
-22
View File
@@ -1,22 +0,0 @@
name: Docker Security Scan
on:
push:
branches: [main, "stable/*"]
schedule:
- cron: "0 6 * * 1" # Weekly Monday 06:00 UTC
permissions:
contents: read
jobs:
scan:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
- run: docker build -t turnstone:scan .
- uses: aquasecurity/trivy-action@57a97c7e7821a5776cebc9bb87c984fa69cba8f1 # 0.35.0
with:
image-ref: "turnstone:scan"
severity: "HIGH,CRITICAL"
exit-code: "1"
+6 -2
View File
@@ -5,6 +5,10 @@ on:
workflows: ["CI"]
types: [completed]
concurrency:
group: publish-${{ github.event.workflow_run.head_sha }}
cancel-in-progress: true
permissions:
contents: write
id-token: write
@@ -40,12 +44,12 @@ jobs:
if: steps.tag.outputs.skip == 'false'
- run: python -m build
if: steps.tag.outputs.skip == 'false'
- uses: pypa/gh-action-pypi-publish@ed0c53931b1dc9bd32cbe73a98c7f6766f8a527e # release/v1
- uses: pypa/gh-action-pypi-publish@cef221092ed1bacb1cc03d23a2d87d1d172e277b # release/v1
if: steps.tag.outputs.skip == 'false'
- name: Create GitHub Release
if: steps.tag.outputs.skip == 'false'
uses: softprops/action-gh-release@153bb8e04406b158c6c84fc1615b65b24149a1fe # v2
uses: softprops/action-gh-release@b4309332981a82ec1c5618f44dd2e27cc8bfbfda # v3
with:
tag_name: ${{ steps.tag.outputs.tag }}
generate_release_notes: true
+1 -1
View File
@@ -48,7 +48,7 @@ jobs:
id: detect
run: |
updates=()
for lib in katex hljs mermaid; do
for lib in katex hljs mermaid hls; do
version=$(grep -oE "${lib}-[0-9.]+" pyproject.toml | head -1 | sed "s/${lib}-//")
[[ -z "$version" ]] && continue
[[ -d "turnstone/shared_static/${lib}-${version}" ]] && continue
+2
View File
@@ -21,3 +21,5 @@ PROGRESS.md
.coverage
tools/skill_audit_analysis/data/
tools/skill_audit_analysis/output/
design_ideas/
.claude/
-40
View File
@@ -1,40 +0,0 @@
# libexpat integer overflow — no fix available in Debian repos yet
# https://avd.aquasec.com/nvd/cve-2026-25210
# Review: remove this entry once a patched libexpat1 is published
CVE-2026-25210
# ncurses buffer overflow — no fix in Debian 13 repos yet
# Affects libncursesw6, libtinfo6, ncurses-base, ncurses-bin
# https://avd.aquasec.com/nvd/cve-2025-69720
CVE-2025-69720
# nghttp2 DoS via malformed HTTP/2 frames — no fix in Debian 13 repos yet
# Affects libnghttp2-14
# https://avd.aquasec.com/nvd/cve-2026-27135
CVE-2026-27135
# systemd arbitrary code execution via spurious IPC — no fix in Debian 13 repos yet
# Affects libsystemd0, libudev1
# https://avd.aquasec.com/nvd/cve-2026-29111
CVE-2026-29111
# glibc iconv() DoS — fix_deferred, no patched libc in Debian 13 yet
# Affects libc-bin, libc6
# https://avd.aquasec.com/nvd/cve-2026-4046
CVE-2026-4046
# minimatch ReDoS — transitive npm dep (MCP server), no direct exposure
# https://avd.aquasec.com/nvd/cve-2026-27903
CVE-2026-27903
# https://avd.aquasec.com/nvd/cve-2026-27904
CVE-2026-27904
# picomatch ReDoS — transitive npm dep, no direct exposure
# https://avd.aquasec.com/nvd/cve-2026-33671
CVE-2026-33671
# node-tar path traversal — transitive npm dep, not used to extract untrusted archives
# https://avd.aquasec.com/nvd/cve-2026-29786
CVE-2026-29786
# https://avd.aquasec.com/nvd/cve-2026-31802
CVE-2026-31802
+308
View File
@@ -0,0 +1,308 @@
# Changelog
All notable changes to turnstone are documented here.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [PEP 440](https://peps.python.org/pep-0440/) for
version numbers (`X.Y.Z`, with `X.Y.ZaN` / `bN` / `rcN` for pre-releases).
Three release tracks are maintained:
- **`stable/1.0`** — patch-only (`v1.0.x`)
- **`stable/1.3`** — patch-only (`v1.3.x`)
- **`stable/1.4`** — patch-only (`v1.4.x`)
- **`main`** — experimental (`v1.5.0aN`)
## [Unreleased]
## [1.4.0]
User-visible additions: a full attachment system (images + text documents,
including pre-creation uploads), a unified dashboard composer, a Slack
channel adapter, per-call plan/task model selection with an admin UI, and
provider capability passthrough.
This release introduces two forward-only schema migrations
(`037_workstream_attachments`, `038_workstream_attachments_reserved_at`)
that the server applies automatically on first startup against an
existing 1.3.x database. Both are additive; no data loss. See
**Database migrations** below for details.
### Added
- **Workstream attachments** — images (png/jpeg/gif/webp, 4 MiB cap) and
text documents (any `text/*` MIME, allowlisted application MIMEs, or
known text extensions; 512 KiB cap; UTF-8 enforced). Magic-byte image
sniffing on upload; per-(ws, user) pending cap of 10. Three-state
lifecycle (`pending → reserved → consumed`) with reservation tokens
threaded through `/v1/api/send` so queued multimodal turns can't lose
files to overlapping sends. Provider-side translation: Anthropic
emits native document blocks; OpenAI Chat Completions inlines them as
escaped `<document>` text blocks; Responses API emits `input_text`
with the same wrapper. (#356)
- **Attachments at workstream-creation time** —
`POST /v1/api/workstreams/new` accepts `multipart/form-data` (one
`meta` JSON field plus 0..N `file` parts). Files are validated and
reserved onto the first turn before the dispatch worker fires; failure
rolls back the fresh workstream so no orphan rows leak. Web UI
(new-workstream modal + dashboard composer), Python SDK, and
TypeScript SDK all gained attachment support. Cluster routing
(`/v1/api/route/workstreams/{ws_id}/attachments`) extended to forward
multipart bodies + preserve upstream headers (CSP, Content-Disposition).
SDKs auto-generate `ws_id` client-side so cluster-routed callers can
bind the body to the owning node before it lands. (#362)
- **Slack channel adapter** (Socket Mode) — mirrors the Discord adapter:
per-user channel sessions via configurable slash command, DM routing
without slash command, SSE event consumption, tool approval buttons
with per-user owner enforcement, plan-review approve / request-changes
modal, notification reply routing back into the workstream, and
session recovery after restart via persisted recoverable route keys
(the bot re-subscribes to existing Slack-routed workstreams when it
comes back). Install with `pip install 'turnstone[slack]'`. (#355)
- **Console admin UX support for Slack** — channel-link modal offers
Slack alongside Discord; skill notify-on-complete forms expose a
per-row channel-type dropdown (and no longer hardcode `discord`);
per-platform `.scope-discord` / `.scope-slack` badge classes with
theme-aware tokens (`--discord` / `--slack`) so light theme passes
WCAG AA. (#365)
- **Per-call plan/task model selection** — `plan_model` and `task_model`
are now distinct from the conversation model and from each other,
with configurable reasoning effort per agent. Three layers:
- **Backend split** (`#54dd557`) — `ModelRegistry` gains `plan_model`,
`task_model`, `plan_effort`, `task_effort`; per-kind overrides win
over the legacy `agent_model`, which still works as the single-knob
fallback. `resolve_agent_alias(kind)` and `resolve_agent_effort(kind)`
centralise resolution. Loader validates effort against
`{none, minimal, low, medium, high, xhigh, max}` with warn+drop on
typos.
- **Runtime configurability** (`#360`) — `ConfigStore` admin tab in
the console UI lets operators switch alias and reasoning effort per
agent **without restarting**. `INHERIT_EMPTY_LABEL_KEYS` shows
`(inherit)` for empty effort selections — distinct from the literal
`none` choice which actually disables reasoning. Routing overrides
apply on `/v1/api/_internal/config-reload` (admin saves), and
`model-reload` short-circuits when nothing changed so no in-flight
clients churn.
- **Per-call override** (`#361`) — the calling LLM can pass
`model="<alias>"` to `plan_agent` or `task_agent` to override the
operator-configured per-kind model for that one invocation. Tool
descriptions list the live registered aliases (refreshed when the
operator hits "sync to nodes"), so the LLM always sees current
options. Bad aliases return a corrective error dict listing the
available choices. No whitelist — cost control is intentionally
ceded to the model. Plan-retry path reuses the alias so coaching
reflects real model behaviour. (#360, #361)
- **Provider capability passthrough** — resolved per-model capabilities
(vision, reasoning, native web search, thinking_mode, token_param,
etc.) flow through to provider clients via a new `capabilities`
parameter on `create_streaming` / `create_completion`, so feature
gating no longer relies on string matching and admin-UI / config.toml
overrides actually reach the provider. Defensive shallow-copy in
`_finalize_extra_body` so callers reusing the same dict across models
are safe; deep-merge of `chat_template_kwargs` so operators can
extend instead of silently overwriting. (#352)
- **Server compatibility layer for local model servers** — vLLM and
llama.cpp profiles suggest the right thinking mode and per-server
workarounds (`skip_special_tokens` for vLLM, `reasoning_format` for
llama.cpp) during model detection. Admin UI gains structured fields
for server type, thinking mode, and extra body params, hidden for
non-local providers (openai/anthropic/google). New `thinking_param`
text field surfaces the alias name (default `enable_thinking`;
Granite/DeepSeek use `thinking`). Verified end-to-end against real
vLLM (Gemma 4 31B) and llama.cpp (Gemma 4 E4B) servers. (#352)
- **Claude Opus 4.7 support** — `claude-opus-4-7` capability entry
(1M ctx, 128K output, adaptive thinking, `supports_temperature=False`,
`thinking_display=summarized`). New `ModelCapabilities.thinking_display`
field — Opus 4.7 omits thinking by default but always sends summarized
blocks back through the provider boundary. Adds `xhigh` effort level
to the global mapping and to Opus 4.7's `effort_levels`; admin-console
skill-template dropdowns gained `xhigh` and `max` options. Reasoning
effort label capitalization aligned across all console dropdowns.
(#357 — also in 1.3.1)
- **Dashboard composer refactor** — unified single-flow create from the
per-node dashboard. Multi-line textarea + collapsible Options panel
(model / judge / skill) + paperclip + drag-drop / paste-image + chip
strip. Submit-button label dynamically toggles between `Create`
(empty) and `Send` (text or attachments staged); Enter and click both
go through the same `dashboardSubmit()`. Replaces the inconsistent
prior split where Enter created+sent raw and the button opened a
separate modal. Options panel state persists in `localStorage`;
active non-default selections render as an inline summary chip beside
the Options button; drag-over shows an explicit "Drop to attach"
overlay. The tab-bar `+` new-workstream modal also gained a paperclip
+ chip strip + first-message field so the same flow is reachable from
both entry points. (#362, #366)
- **Workstream attachments — orphan reservation sweep** — periodic
background sweep clears `reserved_for_msg_id` on rows whose
`reserved_at` exceeds a 1-hour threshold, self-healing reservations
leaked by process crashes between reserve and consume. Backed by a
partial index on `(reserved_at) WHERE reserved_at IS NOT NULL` so the
scan stays cheap as the consumed-history grows. Threshold tracks
reservation age, not upload age, so a long-pending fresh send can't
be racially unreserved. (#363)
- **`SendResponse` extended** — `attached_ids`,
`dropped_attachment_ids`, `priority`, `msg_id` fields exposed in
Pydantic + TypeScript SDKs so attachment-aware clients can detect
partial reservations and dequeue queued messages. (#365)
### Changed
- **`plan_model` and `task_model` now split** from the conversation
model and from each other — operators who rely on a single model for
all three should set both `plan_model` and `task_model` explicitly in
their config; otherwise both default to the conversation model so
behaviour is unchanged. (#54dd557)
- **Channel notify-on-complete `channel_type` is no longer hardcoded
in the admin UI** — operators creating notify targets through the
skill admin form previously got `channel_type: "discord"` regardless
of what they wanted. Existing skill JSON values are unaffected; only
newly created targets through the form differ. (#365)
- **Slack adapter approval previews** — capped at 600 chars per item
with a 2700-char total budget so multi-tool approval batches never
exceed Slack's 3000-char `section.text` limit. Truncated batches
show a `…and N more (preview truncated)` suffix. (#365)
- **PostgreSQL deployment image** swapped from `bitnami/pgbouncer` to
`edoburu/pgbouncer` to track upstream releases and reduce image size.
Environment variables remapped to the edoburu naming, ports updated
to match documented expectations, and the Kubernetes Helm Chart link
in the deployment docs now points at the same container. Review
your helm values if you depend on `bitnami`-specific environment
variable conventions. (#353)
### Fixed
- **`plan_resolved` SSE broadcast** — when one client resolved a plan
approval, other clients viewing the same workstream now have the
approval card dismissed in sync. (#87a9af1)
- **Slack notification reply routing** — one notification reply
previously pinned every later assistant response for that workstream
to the notification thread until the bot restarted. Reply-route
override now clears on `StreamEndEvent`. (#365)
- **Slack plan-review mrkdwn fence** — plan content containing triple
backticks (very common — plans often quote code) no longer breaks the
surrounding fence and lets later content render as live markup. The
shared `_sanitize_slack_preview` helper splices a zero-width space
inside any ``` ``` `` sequence while keeping single backticks
readable. (#365)
- **Slack-routed workstreams now load the chat-specific system prompt**
via `client_type="chat"`, matching Discord. (#365)
- **`/v1/api/workstreams/new` no longer emits a phantom
`ws_created`/`ws_closed` SSE pair** when attachment validation
rejects a multipart create. Validation runs before the broadcast so
failed creates are silent on dashboards. (#362)
- **Multipart Content-Type boundary preservation** in console routing
proxy — `boundary=` parameter is case-sensitive and was being
lowercased before forwarding to the upstream node, breaking parsing
for clients that used mixed-case boundaries (most browsers). (#362)
- **Local-theme contrast for new badge colors** — `.scope-discord` and
`.scope-slack` first shipped with raw hex that failed WCAG AA on
light theme (1.8:1 / 2.4:1). Theme-aware `--discord` / `--slack`
tokens with proper light variants now pass. (#365)
- **Cross-user attachment fetch hardening** — `get_attachment_content`
now scopes the row by `user_id` in addition to `ws_id`, so an
unowned workstream can't be a vector for cross-user blob fetches via
attachment-id guessing. (#356)
- **Attachment-list DoS guard** — `/v1/api/send` rejects
`attachment_ids` lists longer than the per-(ws, user) pending cap
with a 400, preventing hostile clients from blowing up the storage
`IN (...)` clause. (#356)
- **Bounded LRU for upload locks** — the per-(ws, user) attachment
upload-lock map now evicts the oldest unlocked entries past a soft
cap, so the in-process map can't grow unbounded on long-running
nodes. (#356)
- **3.12 CI deadlock on attachment uploads** — the upload-lock was
initially an `asyncio.Lock`, but Starlette's `TestClient` runs each
request on a fresh anyio task / event loop, so the cached lock's
`_waiters` bound to the first loop and a later request would block
on a Future from a closed loop (silent deadlock). Switched to
`threading.Lock` — loop-agnostic, and the critical section is one
COUNT + one INSERT. Same root cause is reproducible against any
Starlette TestClient harness on Python ≥ 3.10; 3.12 surfaces it
more often. Production users on a single event loop weren't
affected, but the test environment was. (#356)
### Security
- **Slack approval per-user authentication** — only the session owner
can click Approve/Deny on a Slack tool-approval card. Without this,
any channel member with view access could approve dangerous tool
calls initiated by someone else. (#355)
- **Attachment ownership masking** — cross-user/cross-workstream
attachment ID lookups return 404 (not 403) so non-owners can't
enumerate workstream existence by response code. (#356)
- Bumped Debian base image; remaining unfixable `jq` CVEs are
documented and exception-listed. (#aaea4d3)
### Database migrations
- **`037_workstream_attachments`** — new `workstream_attachments` table
with the lifecycle columns described above. Indexes for ws_id,
pending lookups, message linkage, and reservation scoping.
- **`038_workstream_attachments_reserved_at`** — adds `reserved_at`
column for the orphan-sweep staleness signal, plus a partial index
on `reserved_at IS NOT NULL` so the periodic scan is cheap.
Both migrations are additive and idempotent, and the server applies
them automatically on first startup against an existing 1.3.x database.
No manual `alembic upgrade` step is required — though running it
manually beforehand (e.g. as part of a phased deploy) remains safe.
### SDK
Python + TypeScript clients gained:
- `AttachmentUpload` type
- `upload_attachment(ws_id, filename, data, mime_type=None)`
- `list_attachments(ws_id)`
- `get_attachment_content(ws_id, attachment_id) → bytes / Blob`
- `delete_attachment(ws_id, attachment_id)`
- `send(message, ws_id, attachment_ids=...)` (extended)
- `create_workstream(..., attachments=[...])` — multipart variant with
client-side `ws_id` generation for cluster-routed callers
- Console SDK: `route_create_workstream(attachments=...)`,
`route_upload_attachment`, `route_list_attachments`,
`route_get_attachment_content`, `route_delete_attachment`
- Refusal of `attachments + target_node` combination at the SDK
boundary (the multipart routing layer doesn't honor `target_node`,
so silently picking the wrong node is now an explicit error)
- `PlanResolvedEvent` SSE event with type guard, dispatched when one
client (e.g. mobile) resolves a plan so other connected clients can
dismiss their plan-approval modal in sync. Available in both the
Python and TypeScript SDKs. (#87a9af1)
### Operational
- **CI vendor-asset auto-download covers `hls.js`** — the
`vendor-js.yml` workflow previously only iterated katex/hljs/mermaid,
so Renovate bumps for `hls.js` failed the wheel-completeness check
and required manual file downloads. Detection loop now includes
`hls`, so future Renovate bumps are merge-ready without intervention.
(#354)
### Contributors
Thanks to the people who made this release happen — especially the
external contributors who picked up substantial pieces of work:
- **[@daoxley](https://github.com/daoxley)** — designed and shipped
the Slack channel adapter (Socket Mode bot, per-user sessions,
approvals, plan-review, notification routing). Major new feature
surface in #355.
- **[@pizzaandcheese](https://github.com/pizzaandcheese)** — replaced
the deprecated bitnami pgbouncer image with the edoburu image,
remapped environment variables, ports, and helm chart references.
Operationally important for anyone running our reference Postgres
deployment (#353).
- Renovate kept dependencies and the JS vendor tree current via
several automated bumps.
If you're interested in contributing, channel-attachment ingest from
Discord + Slack is the headline 1.4.1 feature and a solid place to
start — see the open issues on GitHub or open one to scope a piece.
## [1.3.1]
### Added
- Backport: Claude Opus 4.7 support (provider capabilities, tokenizer,
adaptive thinking). (#357)
+9 -1
View File
@@ -26,7 +26,15 @@ transferring ownership.
```
python -m venv .venv
source .venv/bin/activate
pip install -e ".[test]"
pip install -e ".[test,dev]"
```
The `dev` extra installs `ruff` and `mypy`. Before pushing, run:
```
ruff check turnstone tests
mypy turnstone
pytest
```
## Guidelines
+1 -1
View File
@@ -8,7 +8,7 @@ FROM python:3.14-slim
LABEL org.opencontainers.image.title="turnstone" \
org.opencontainers.image.description="Multi-node AI orchestration platform"
COPY --from=ghcr.io/astral-sh/uv:0.11.3 /uv /usr/local/bin/uv
COPY --from=ghcr.io/astral-sh/uv:0.11.7 /uv /usr/local/bin/uv
# Remove the slim image's man page exclusion so man-db has actual content
RUN rm -f /etc/dpkg/dpkg.cfg.d/docker
+4 -4
View File
@@ -55,7 +55,7 @@ The wizard supports two deployment modes:
```
$ turnstone-bootstrap
Turnstone Bootstrap Wizard v0.5.4
Turnstone Bootstrap Wizard v1.5.0
────────────────────────────────────────────────
Which provider for this wizard?
@@ -87,6 +87,6 @@ $ turnstone-bootstrap
## See Also
- [Docker Deployment](docker.md) — manual compose setup and profiles
- [Security](security.md) — auth architecture and token types
- [Governance](governance.md) — roles, policies, and templates
- [Docker Deployment](docs/docker.md) — manual compose setup and profiles
- [Security](docs/security.md) — auth architecture and token types
- [Governance](docs/governance.md) — roles, policies, and templates
+16 -7
View File
@@ -30,7 +30,7 @@ Turnstone gives LLMs tools — shell, files, search, web, planning — and orche
- **Cluster dashboard** — real-time view of all nodes and workstreams with console routing proxy
- **Intent validation** — LLM judge evaluates every tool call with risk assessments and evidence
- **Governance** — RBAC, OIDC SSO, tool policies, skills, usage tracking, audit logs
- **Multi-provider** — OpenAI-compatible APIs (vLLM, llama.cpp, NIM) and Anthropic Messages API
- **Multi-provider** — OpenAI-compatible APIs (vLLM, llama.cpp, NIM), Anthropic Messages API, and Google Gemini
- **MCP support** — external tool servers with native deferred loading (Anthropic/OpenAI) or BM25 fallback
<p align="center">
@@ -53,6 +53,15 @@ pip install turnstone[console]
turnstone-console --port 8090
```
For PostgreSQL (recommended for production):
```bash
pip install turnstone[postgres]
export TURNSTONE_DB_BACKEND=postgresql
export TURNSTONE_DB_URL="postgresql+psycopg://user:pass@localhost:5432/turnstone"
turnstone-server --port 8080 --base-url http://localhost:8000/v1
```
### Docker
```bash
@@ -75,20 +84,20 @@ with TurnstoneServer("http://localhost:8080", token="tok_xxx") as client:
## Tools
Built-in tools for shell, files, search, web, memory, notifications, and autonomous sub-agents — plus external tools via [MCP](https://modelcontextprotocol.io/) with native deferred loading. See [docs/tools.md](docs/tools.md) for the full reference and [docs/mcp.md](docs/mcp.md) for MCP configuration.
Built-in tools for shell, files, search, web, memory, notifications, and autonomous sub-agents — plus external tools via [MCP](https://modelcontextprotocol.io/) with native deferred loading. See [docs/tools.md](docs/tools.md) for the full reference and [docs/mcp-registry.md](docs/mcp-registry.md) for MCP configuration.
## Architecture
**Single-node**: Client → Server (direct HTTP + SSE). No external dependencies beyond the database.
**Multi-node**: Client → Console (hash ring routing proxy) → Server nodes. The console maintains a 65536-entry bucket cache for O(1) workstream routing. A rebalancer daemon redistributes buckets when nodes join or leave.
**Multi-node**: Client → Console (rendezvous routing proxy) → Server nodes. The console picks the target node for each workstream via rendezvous (HRW) hashing over the live service registry — pure function of `(ws_id, live_nodes)`, no stored bucket state, deterministic across readers. A node join or drop only re-routes the keys that score highest on the affected node.
| Component | Purpose |
|-----------|---------|
| `turnstone` | Terminal CLI (REPL) |
| `turnstone-server` | Web UI + REST API + SSE events |
| `turnstone-console` | Cluster dashboard + routing proxy + admin panel |
| `turnstone-channel` | Channel gateway (Discord, with adapters for Slack/Teams planned) |
| `turnstone-channel` | Channel gateway (Discord and Slack adapters) |
| `turnstone-admin` | User/token management CLI |
| `turnstone-eval` | Eval harness for prompt/tool optimization |
| `turnstone-bootstrap` | LLM-guided setup wizard |
@@ -108,7 +117,7 @@ UML diagrams in [`docs/diagrams/`](docs/diagrams/):
| [Console Data Flow](docs/diagrams/png/11-console-data-flow.png) | Dashboard data collection |
| [Deployment](docs/diagrams/png/12-deployment.png) | Docker Compose topology |
| [Auth](docs/diagrams/png/15-auth-architecture.png) | JWT, scopes, login flows |
| [Channels](docs/diagrams/png/16-channel-architecture.png) | Discord adapter + routing |
| [Channels](docs/diagrams/png/16-channel-architecture.png) | Discord / Slack adapters + routing |
| [Judge](docs/diagrams/png/22-judge-architecture.png) | Intent validation pipeline |
| [OIDC](docs/diagrams/png/25-oidc-architecture.png) | SSO authorization code flow |
@@ -127,12 +136,12 @@ UML diagrams in [`docs/diagrams/`](docs/diagrams/):
| Console dashboard | [docs/console.md](docs/console.md) |
| Eval harness | [docs/eval.md](docs/eval.md) |
| Tools reference | [docs/tools.md](docs/tools.md) |
| MCP integration | [docs/mcp.md](docs/mcp.md) |
| MCP integration | [docs/mcp-registry.md](docs/mcp-registry.md) |
## Requirements
- Python 3.11+
- An OpenAI-compatible API endpoint or Anthropic API key
- An OpenAI-compatible API endpoint, Anthropic API key, or Google Gemini API key
- Optional: PostgreSQL (`pip install turnstone[postgres]`), Anthropic (`pip install turnstone[anthropic]`)
- [Git LFS](https://git-lfs.com/) for cloning (diagram PNGs)
+19
View File
@@ -95,3 +95,22 @@ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
================================================================================
hls.js 1.6.15
https://github.com/video-dev/hls.js
Copyright 2017 Dailymotion
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
+18 -16
View File
@@ -1,10 +1,15 @@
# =============================================================================
# Turnstone Docker Compose Stack
# Turnstone Docker Compose Stack — Development
#
# This file is for local development from a git clone. It builds images
# locally from the Dockerfile. If you installed via pip/pipx, run
# `turnstone-bootstrap` instead — it writes a production compose.yaml
# that pulls pre-built images from ghcr.io.
#
# Usage:
# Infra only: docker compose up
# Single node: docker compose --profile production up
# Production (PG): DB_BACKEND=postgresql docker compose --profile production up
# Production (PG): TURNSTONE_DB_BACKEND=postgresql docker compose --profile production up
# 10-node cluster: docker compose --profile cluster up
# =============================================================================
@@ -60,9 +65,7 @@ services:
# turnstone-server — Web UI + chat workstreams + LLM interaction
# -------------------------------------------------------------------
server:
build:
context: .
dockerfile: Dockerfile
image: turnstone:local
profiles:
- production
command:
@@ -91,8 +94,8 @@ services:
- TURNSTONE_JWT_SECRET=${TURNSTONE_JWT_SECRET:?Set TURNSTONE_JWT_SECRET in .env}
- MODEL=${MODEL:-}
- MCP_CONFIG=${MCP_CONFIG:-}
- TURNSTONE_DB_BACKEND=${DB_BACKEND:-sqlite}
- TURNSTONE_DB_URL=${DATABASE_URL:-}
- TURNSTONE_DB_BACKEND=${TURNSTONE_DB_BACKEND:-sqlite}
- TURNSTONE_DB_URL=${TURNSTONE_DB_URL:-}
- TURNSTONE_NODE_ID=${TURNSTONE_NODE_ID:-}
- TURNSTONE_ADVERTISE_URL=${TURNSTONE_ADVERTISE_URL:-http://server:8080}
extra_hosts:
@@ -115,6 +118,7 @@ services:
# turnstone-console — Cluster dashboard
# -------------------------------------------------------------------
console:
image: turnstone:local
build:
context: .
dockerfile: Dockerfile
@@ -127,8 +131,8 @@ services:
environment:
# Generate with: python -c "import secrets; print(secrets.token_hex(32))"
- TURNSTONE_JWT_SECRET=${TURNSTONE_JWT_SECRET:?Set TURNSTONE_JWT_SECRET in .env}
- TURNSTONE_DB_BACKEND=${DB_BACKEND:-sqlite}
- TURNSTONE_DB_URL=${DATABASE_URL:-}
- TURNSTONE_DB_BACKEND=${TURNSTONE_DB_BACKEND:-sqlite}
- TURNSTONE_DB_URL=${TURNSTONE_DB_URL:-}
- TURNSTONE_CONSOLE_URL=http://console:8090
networks:
- turnstone-net
@@ -145,9 +149,7 @@ services:
# Requires TURNSTONE_DISCORD_TOKEN to enable Discord adapter
# -------------------------------------------------------------------
channel:
build:
context: .
dockerfile: Dockerfile
image: turnstone:local
profiles:
- production
- cluster
@@ -163,8 +165,8 @@ services:
- TURNSTONE_DISCORD_GUILD=${TURNSTONE_DISCORD_GUILD:-0}
# Generate with: python -c "import secrets; print(secrets.token_hex(32))"
- TURNSTONE_JWT_SECRET=${TURNSTONE_JWT_SECRET:?Set TURNSTONE_JWT_SECRET in .env}
- TURNSTONE_DB_BACKEND=${DB_BACKEND:-postgresql}
- TURNSTONE_DB_URL=${DATABASE_URL:-postgresql://${POSTGRES_USER:-turnstone}:${POSTGRES_PASSWORD:-turnstone}@postgres:5432/turnstone}
- TURNSTONE_DB_BACKEND=${TURNSTONE_DB_BACKEND:-postgresql}
- TURNSTONE_DB_URL=${TURNSTONE_DB_URL:-postgresql+psycopg://${POSTGRES_USER:-turnstone}:${POSTGRES_PASSWORD:-turnstone}@postgres:5432/turnstone}
- TURNSTONE_CHANNEL_ADVERTISE_URL=http://channel:8091
networks:
- turnstone-net
@@ -213,8 +215,8 @@ services:
TURNSTONE_JWT_SECRET: ${TURNSTONE_JWT_SECRET:?Set TURNSTONE_JWT_SECRET in .env}
MODEL: ${MODEL:-}
MCP_CONFIG: ${MCP_CONFIG:-}
TURNSTONE_DB_BACKEND: ${DB_BACKEND:-postgresql}
TURNSTONE_DB_URL: ${DATABASE_URL:-postgresql://${POSTGRES_USER:-turnstone}:${POSTGRES_PASSWORD:?}@postgres:5432/turnstone}
TURNSTONE_DB_BACKEND: ${TURNSTONE_DB_BACKEND:-postgresql}
TURNSTONE_DB_URL: ${TURNSTONE_DB_URL:-postgresql+psycopg://${POSTGRES_USER:-turnstone}:${POSTGRES_PASSWORD:?}@postgres:5432/turnstone}
TURNSTONE_NODE_ID: node-1
TURNSTONE_ADVERTISE_URL: http://server-1:8080
extra_hosts: ["host.docker.internal:host-gateway"]
+40
View File
@@ -0,0 +1,40 @@
# Bare-metal overlay — expose PostgreSQL and let the console reach
# a turnstone-server running outside Docker on the host machine.
#
# Requires TURNSTONE_HOST_IP set to the host's routable IP address.
#
# Usage:
# export TURNSTONE_HOST_IP="$(hostname -I | awk '{print $1}')"
# docker compose --profile production \
# -f compose.yaml -f deploy/docker-compose.bare-metal.yml up
#
# Then on the host:
# export TURNSTONE_JWT_SECRET="<same as .env>"
# export TURNSTONE_DB_BACKEND=postgresql
# export TURNSTONE_DB_URL="postgresql://turnstone:<pw>@localhost:5432/turnstone"
# export TURNSTONE_NODE_ID="bare-metal-1"
# export TURNSTONE_ADVERTISE_URL="http://${TURNSTONE_HOST_IP}:8080"
# python -m turnstone.server --host 0.0.0.0 --port 8080 \
# --base-url http://localhost:8000/v1 --api-key "$OPENAI_API_KEY"
services:
postgres:
ports:
- "${POSTGRES_PORT:-5432}:5432"
console:
extra_hosts:
- "host.docker.internal:host-gateway"
environment:
# Console needs to reach the bare-metal server on the host
TURNSTONE_SERVER_URL: "http://${TURNSTONE_HOST_IP}:${SERVER_PORT:-8080}"
channel:
ports:
- "${CHANNEL_PORT:-8091}:8091"
environment:
# Channel gateway advertises with host-routable IP so the
# bare-metal server can reach it for schedule notifications
TURNSTONE_CHANNEL_ADVERTISE_URL: "http://${TURNSTONE_HOST_IP}:${CHANNEL_PORT:-8091}"
# Channel needs to reach the bare-metal server on the host
TURNSTONE_SERVER_URL: "http://${TURNSTONE_HOST_IP}:${SERVER_PORT:-8080}"
+267 -9
View File
@@ -842,6 +842,15 @@ button automatically.
Creates a new workstream. The server supports up to 10 concurrent workstreams.
The endpoint accepts **either** `application/json` (legacy shape) **or**
`multipart/form-data` when you want to upload attachments at creation
time. Multipart requests carry one `meta` field containing the JSON body
shown below plus zero-or-more `file` parts; each file is validated and
reserved onto the new workstream's first turn before the dispatch worker
runs, so queued multimodal turns cannot lose files to racing sends. If
validation fails the fresh workstream is rolled back so no orphan rows
leak.
**Request body:**
```json
@@ -857,6 +866,7 @@ All fields are optional. The body can be empty or an empty JSON object.
| `auto_approve` | bool | false | Auto-approve all tool calls for this workstream |
| `resume_ws` | string | "" | Workstream ID to resume atomically during creation (empty = fresh)|
| `skill` | string | "" | Skill name. Applies content (system prompt), model, temperature, reasoning effort, max tokens, auto-approve policy, token budget, and other session config from the skill. Returns 400 if not found or disabled. Ignored when `resume_ws` is set (resumed sessions restore their own skill). |
| `judge_model` | string | "" | Optional model alias for the judge (overrides default judge model for this workstream) |
> **Skill behavior:** When `skill` is specified, the skill's content is injected as a system message and its session config fields (model, temperature, auto-approve, token budget, etc.) override system defaults for the new workstream.
@@ -914,6 +924,255 @@ Status code: `400`
---
### `POST /v1/api/workstreams/{ws_id}/attachments`
Upload an image or text document and attach it to the caller's next user
turn on this workstream.
- Images (png/jpeg/gif/webp) are capped at **4 MiB** and validated via
magic-byte sniff on upload.
- Text documents (any `text/*` MIME, allow-listed application MIMEs, or
known text extensions) are capped at **512 KiB** and must be UTF-8.
- Per-(workstream, user) pending cap is **10** attachments.
The attachment moves through three states: `pending → reserved →
consumed`. Reservation tokens are threaded through
`POST /v1/api/send` so a queued multimodal turn cannot lose its file to
an overlapping send.
Ownership failures are masked as `404` so non-owners cannot enumerate
workstream existence.
**Content-Type:** `multipart/form-data` with a single `file` field.
**Response (success):** `200`
```json
{
"attachment_id": "att_abc123",
"kind": "image",
"mime_type": "image/png",
"size_bytes": 73240,
"filename": "screenshot.png",
"state": "pending"
}
```
**Errors:**
| Code | Meaning |
|------|---------------------------------------------------------|
| 400 | Missing/invalid form, unsupported MIME, not UTF-8, etc. |
| 403 | Auth/scope failure |
| 404 | Workstream not found / not owned by caller |
| 409 | Pending-cap reached |
| 413 | Payload exceeds size cap |
---
### `GET /v1/api/workstreams/{ws_id}/attachments`
List the caller's **pending** (unconsumed) attachments for this
workstream. Ownership failures are masked as `404`.
**Response:** `200`
```json
{
"attachments": [
{
"attachment_id": "att_abc123",
"kind": "image",
"mime_type": "image/png",
"size_bytes": 73240,
"filename": "screenshot.png",
"state": "pending"
}
]
}
```
---
### `GET /v1/api/workstreams/{ws_id}/attachments/{attachment_id}/content`
Returns the raw bytes of an attachment with its stored `Content-Type`.
Useful for previewing an image or replaying a document. Ownership
failures are masked as `404`.
**Response:** `200` — binary body, original `Content-Type`.
---
### `DELETE /v1/api/workstreams/{ws_id}/attachments/{attachment_id}`
Remove a pending attachment. Consumed attachments return `404` (they
are part of a committed conversation turn). Ownership failures are also
masked as `404`.
**Response:** `200`
```json
{"deleted": "att_abc123"}
```
---
### `POST /v1/api/workstreams/{ws_id}/delete`
Permanently delete a saved workstream and all its messages from storage.
**Path parameters:**
| Parameter | Type | Description |
|-----------|--------|----------------------|
| `ws_id` | string | Workstream ID |
**Response (success):** `200`
```json
{"deleted": "a1b2c3d4"}
```
**Response (not found):** `404`
```json
{"error": "Workstream not found"}
```
---
### `POST /v1/api/workstreams/{ws_id}/open`
Load a saved workstream into memory with its original `ws_id`. If the
workstream is already loaded, returns immediately with `already_loaded: true`.
**Path parameters:**
| Parameter | Type | Description |
|-----------|--------|----------------------|
| `ws_id` | string | Workstream ID |
**Response (success):** `200`
```json
{"ws_id": "a1b2c3d4", "name": "refactor"}
```
**Response (already loaded):** `200`
```json
{"ws_id": "a1b2c3d4", "name": "refactor", "already_loaded": true}
```
---
### `POST /v1/api/workstreams/{ws_id}/title`
Set a workstream title manually. The title is stored as the workstream alias.
**Path parameters:**
| Parameter | Type | Description |
|-----------|--------|----------------------|
| `ws_id` | string | Workstream ID |
**Request body:**
```json
{"title": "JWT Authentication Refactor"}
```
| Field | Type | Required | Description |
|---------|--------|----------|------------------------|
| `title` | string | yes | New workstream title |
**Response (success):** `200`
```json
{"status": "ok", "title": "JWT Authentication Refactor"}
```
**Response (conflict):** `409`
```json
{"error": "That name is already used by another workstream"}
```
---
### `POST /v1/api/workstreams/{ws_id}/refresh-title`
Regenerate the workstream title via LLM based on conversation content.
**Path parameters:**
| Parameter | Type | Description |
|-----------|--------|----------------------|
| `ws_id` | string | Workstream ID |
**Response (success):** `200`
```json
{"status": "ok"}
```
---
### `GET /v1/api/admin/settings`
List `interface.*` settings with their current values and sources. Requires
`read` scope on the server.
**Response:** `200`
```json
{
"settings": [
{
"key": "interface.close_tab_action",
"value": "last_used",
"source": "default",
"type": "str",
"description": "Determines which workstream to switch to after closing a tab."
}
]
}
```
---
### `POST|PUT /v1/api/admin/settings/{key}`
Update an `interface.*` setting. Only keys in the `interface` section are
accepted; other keys return `400`.
**Path parameters:**
| Parameter | Type | Description |
|-----------|--------|-------------------------------------|
| `key` | string | Setting key (e.g. `interface.theme`) |
**Request body:**
```json
{"value": "light"}
```
| Field | Type | Required | Description |
|---------|------|----------|----------------|
| `value` | any | yes | New value |
**Response (success):** `200`
```json
{"status": "ok", "key": "interface.theme", "value": "light"}
```
**Error:** `400` if the key is not in the `interface` section.
---
### `GET /v1/api/watches`
List active watches on this server node. Optionally filter by workstream.
@@ -1352,7 +1611,7 @@ version. Requires the `admin.skills` permission.
```json
{
"scan_status": "medium",
"risk_level": "medium",
"scan_report": "{\"composite\": 1.75, \"details\": {...}}",
"scan_version": "1"
}
@@ -1862,15 +2121,15 @@ turnstone_tool_calls_total{tool="read_file"} 3
## Console Routing Proxy Endpoints
These endpoints are served by the console (`turnstone-console`) and proxy
requests to the correct server node via the hash ring bucket cache. In
multi-node deployments, clients (SDK, channel gateway) talk to the console
instead of individual server nodes.
requests to the correct server node via rendezvous (HRW) hashing over the
live service registry. In multi-node deployments, clients (SDK, channel
gateway) talk to the console instead of individual server nodes.
### `POST /v1/api/route/workstreams/new`
Create a workstream via hash-ring routing. The console generates the `ws_id`,
routes to the assigned node, and includes `node_url` in the response for
direct SSE connections.
Create a workstream via rendezvous routing. The console generates the `ws_id`,
routes to the rendezvous-selected node, and includes `node_url` in the
response for direct SSE connections.
### `POST /v1/api/route/send`
@@ -1905,5 +2164,4 @@ Used by channel adapters to open direct SSE connections to the correct server no
Prometheus metrics for the console routing layer. Includes:
`turnstone_router_requests_total`, `turnstone_router_request_duration_seconds`,
`turnstone_ring_membership_size`, `turnstone_ring_version`,
`turnstone_ring_rebalance_total`, `turnstone_ring_migrations_total`.
`turnstone_router_membership_size`, `turnstone_router_refresh_total`.
+108 -36
View File
@@ -21,7 +21,8 @@ plugs in.
| `turnstone-console` | `turnstone.console.server` | ClusterCollector | Cluster dashboard (aggregates all nodes) |
| `turnstone-eval` | `turnstone.eval` | `NullUI` | Headless evaluation and prompt optimization |
| `turnstone-channel` | `turnstone.channels.cli` | ChannelAdapter | Channel gateway (Discord, Slack, etc.) |
| `turnstone-admin` | `turnstone.core.admin_cli` | — | Offline user and API token management |
| `turnstone-admin` | `turnstone.admin` | — | Offline user and API token management |
| `turnstone-bootstrap` | `turnstone.bootstrap` | — | LLM-guided setup wizard |
---
@@ -36,8 +37,12 @@ turnstone/
session.py ChatSession engine, SessionUI protocol, tool dispatch
providers/ LLM provider adapters (pluggable backend layer)
_protocol.py LLMProvider protocol, ModelCapabilities, StreamChunk, CompletionResult
_openai.py OpenAIProvider — OpenAI, vLLM, llama.cpp, any compatible API
_openai.py OpenAIProvider facade (re-exports Chat/Responses providers)
_openai_chat.py OpenAIChatCompletionsProvider — vLLM, llama.cpp, local compatible APIs
_openai_responses.py OpenAIResponsesProvider — commercial OpenAI Responses API
_openai_common.py Shared ModelCapabilities table + helpers
_anthropic.py AnthropicProvider — Anthropic Messages API, native streaming, thinking
_google.py GoogleProvider — Google Gemini via OpenAI-compat endpoint
__init__.py create_provider() + create_client() factory functions
workstream.py Parallel workstream manager (WorkstreamState, Workstream, WorkstreamManager)
tools.py Tool schema loader (JSON -> OpenAI function-calling format)
@@ -80,12 +85,13 @@ turnstone/
static/ Cluster dashboard web UI (page-specific HTML, CSS, JS)
channels/
cli.py Unified channel gateway entry point (turnstone-channel)
_protocol.py ChannelAdapter protocol, ChannelEvent dataclass
_protocol.py ChannelAdapter protocol
_routing.py ChannelRouter — channel/thread ↔ workstream mapping via HTTP
_config.py Base ChannelConfig dataclass
discord/ Discord adapter (bot, cog, views, streaming, config)
slack/ Slack adapter (Socket Mode bot, DM routing, approval buttons)
shared_static/ Shared design system (base.css, auth.js, theme.js, toast.js, utils.js, kb.js)
katex-0.16.44/ Vendored KaTeX math rendering library (MIT, woff2 fonts)
katex-0.16.45/ Vendored KaTeX math rendering library (MIT, woff2 fonts)
ui/
colors.py ANSI color constants with NO_COLOR support
markdown.py Streaming terminal markdown renderer (line-buffered)
@@ -96,7 +102,7 @@ turnstone/
renderer.js Markdown + LaTeX renderer (tables, nested lists, blockquotes, KaTeX math)
app.js Split-pane UI (Pane class, binary layout tree, SSE, tool approval)
tools/
*.json 15 tool schemas (OpenAI function-calling format + turnstone metadata)
*.json 19 tool schemas (OpenAI function-calling format + turnstone metadata)
```
Both UIs share a common design system extracted into `turnstone/shared_static/`: design tokens, login overlay, toast notifications, theme toggle, keyboard shortcuts, and utility functions. Each UI imports `base.css` and the shared JS modules at `/shared/`, then adds only page-specific code at `/static/`.
@@ -442,13 +448,15 @@ from each schema and builds:
- `PRIMARY_KEY_MAP` -- `{name: primary_key}` for JSON fallback recovery
- `merge_mcp_tools(builtin, mcp_tools)` -- merges built-in + MCP tools at session init
### 13 Tools by Category
### 19 Tools by Category
**Read-only (auto-approve)**:
- `read_file` -- read file contents with optional offset/limit
- `diff_file` -- show diff between two files / versions
- `search` -- ripgrep-based codebase search
- `man` -- read man pages
- `recall` -- search conversation history
- `read_resource` -- read an MCP resource by URI
**Write (requires approval)**:
- `bash` -- execute shell commands (with safety checks via `turnstone.core.safety`)
@@ -457,13 +465,20 @@ from each schema and builds:
- `math` -- execute Python in sandboxed subprocess (via `turnstone.core.sandbox`)
- `web_fetch` -- fetch a URL (with SSRF protection via `turnstone.core.web`)
- `web_search` -- search the web (provider-native for Anthropic/OpenAI, Tavily fallback for local models)
- `notify` -- send a user-facing notification (Discord/Slack, optional reply routing)
- `watch` -- schedule a recurring poll with condition DSL
**Agent (delegated sub-sessions)**:
- `task` -- delegate to a sub-agent with full tool access (`TASK_AGENT_TOOLS`)
- `plan` -- explore codebase and write a structured plan (`AGENT_TOOLS`)
- `task_agent` -- delegate to a sub-agent with full tool access (`TASK_AGENT_TOOLS`)
- `plan_agent` -- explore codebase and write a structured plan (`AGENT_TOOLS`)
**Memory (structured persistent store)**:
**Memory / skills / prompts**:
- `memory` -- save, search, delete, or list memories (typed and scoped)
- `skill` -- invoke a skill (governed, versioned procedure)
- `use_prompt` -- fetch and apply a prompt template
Tool names are `plan_agent` / `task_agent` (not `plan` / `task`); bare words
collide with chat-template channels on some local models.
### Prepare / Execute Pattern
@@ -482,14 +497,14 @@ separation allows the UI to show previews before any side effects occur.
### Agent Tools
`task` and `plan` invoke `_run_agent()`, which runs a multi-turn loop with
a subset of tools and its own system prompt. The sub-agent runs
`task_agent` and `plan_agent` invoke `_run_agent()`, which runs a multi-turn
loop with a subset of tools and its own system prompt. The sub-agent runs
independently, then returns the final content as the tool result.
- **task**: uses `self._task_tools` (`TASK_AGENT_TOOLS` + MCP tools)
- **plan**: uses `self._agent_tools` (`AGENT_TOOLS` + MCP tools). Writes output
- **task_agent**: uses `self._task_tools` (`TASK_AGENT_TOOLS` + MCP tools)
- **plan_agent**: uses `self._agent_tools` (`AGENT_TOOLS` + MCP tools). Writes output
to `.plan-<ws_id>.md` — unique per `ChatSession` so concurrent workstreams
don't collide. On repeat invocations the prior `plan` tool call and its result
don't collide. On repeat invocations the prior `plan_agent` tool call and its result
are forwarded from `self.messages` so the agent refines the existing plan rather
than starting over. Planning instructions are injected as a developer message
prepended to the agent's conversation.
@@ -547,6 +562,21 @@ expanded tools).
**Tool naming:** `mcp__{server}__{tool}` — double underscore delimiter, validated
at connection time (server names with `__` are rejected).
**Resilience:** Each MCP server has an independent circuit breaker that opens
after 3 consecutive transport failures (timeouts, broken pipes, connection
resets). Cooldown uses capped exponential backoff (30 s base, 5 min max) with
per-server jitter to avoid thundering herd. Protocol-level errors (`McpError`)
from a healthy connection do not trip the breaker. When the cooldown expires
(half-open), the next operation attempt triggers automatic reconnection. Manual
`/mcp refresh` also clears the circuit on success. All sync bridge methods
(`call_tool_sync`, `read_resource_sync`, `get_prompt_sync`, `refresh_sync`)
cancel orphaned futures on timeout to prevent coroutine accumulation on the
background event loop. Push notification refreshes are debounced (5 s per
server) to protect against notification storms. The periodic refresh loop
attempts reconnection for disconnected servers with exponential backoff
(60 s1 h). Transport stream references are pre-closed before stack teardown to
work around the MCP SDK's anyio cancel-scope CPU busy-loop (SDK #2147).
**Error isolation:** Per-server connection/refresh failures are caught and logged; other
servers are unaffected. Tool execution errors return error strings to the LLM
rather than crashing the session.
@@ -578,6 +608,7 @@ LLMProvider (protocol)
|
+--- OpenAIProvider --- OpenAI, vLLM, llama.cpp, any /v1/chat/completions API
+--- AnthropicProvider --- Anthropic Messages API (native streaming, thinking)
+--- GoogleProvider --- Google Gemini via /v1beta/openai/ (extends OpenAIProvider)
```
**Protocol methods:**
@@ -631,6 +662,13 @@ both streaming and non-streaming responses. The `anthropic` SDK is imported
lazily so it remains an optional dependency (`pip install
turnstone[anthropic]`).
**GoogleProvider** (`_google.py`): extends `OpenAIChatCompletionsProvider` for
the Gemini `/v1beta/openai/` endpoint. Uses a single default
`ModelCapabilities` (2M context window, 65K max output tokens,
`token_param=max_tokens`) since Google updates models frequently. No static
per-model capability table. Google's endpoint is wire-compatible with the
OpenAI SDK, so no extra dependency is needed.
**Factory functions** (`__init__.py`): `create_provider(name)` returns a
singleton provider instance (thread-safe). `create_client(name, base_url,
api_key)` creates the appropriate SDK client.
@@ -659,6 +697,10 @@ api_key = "sk-..."
model = "gpt-5"
context_window = 400000
[models.gemini]
provider = "google"
model = "gemini-2.5-pro"
[model]
default = "local"
fallback = ["claude", "openai"]
@@ -666,7 +708,28 @@ agent_model = "claude"
```
Each `[models.*]` entry produces a `ModelConfig` with a `provider` field
(default: `"openai"`). Supported values: `"openai"` and `"anthropic"`.
(default: `"openai"`). Supported values: `"openai"`, `"anthropic"`, `"google"`,
and `"openai-compatible"`.
**Per-model sampling overrides:** Each model can specify `temperature`,
`max_tokens`, and `reasoning_effort` to override the global defaults from
ConfigStore. When unset (`NULL`), the global default is used.
```toml
[models.local]
base_url = "http://localhost:8000/v1"
model = "qwen3-32b"
temperature = 0.7
max_tokens = 8192
[models.o3]
base_url = "https://api.openai.com/v1"
api_key = "sk-..."
model = "o3"
reasoning_effort = "high"
# temperature omitted — uses global default
```
An optional `[models.*.capabilities]` sub-table overrides per-model
`ModelCapabilities` flags (useful for local models whose capabilities
cannot be detected programmatically):
@@ -680,9 +743,15 @@ model = "qwen-3.5-vl"
supports_vision = true
```
**Database model definitions:** On server entry points, models can also be
defined in the `model_definitions` table (admin Models tab). DB models support
the same per-model sampling overrides. Config.toml models override DB models
with the same alias in-memory (the DB rows are never modified).
**Lifecycle:**
1. `load_model_registry()` reads `[models.*]` sections from config.toml and
builds a `"default"` entry from CLI `--base-url`/`--model`/`--api-key` args
1. `load_model_registry()` loads DB model definitions (if storage available),
then overlays `[models.*]` from config.toml, then builds a `"default"` entry
from CLI `--base-url`/`--model`/`--api-key` args
2. The registry is passed to the session factory closure in both `cli.py` and
`server.py`; each workstream resolves its model on creation
3. `ModelRegistry.get_client()` lazily creates SDK client instances via
@@ -691,7 +760,8 @@ supports_vision = true
4. `ModelRegistry.get_provider()` lazily creates `LLMProvider` instances via
`create_provider()` (also cached and thread-safe)
5. `/model` command shows available models; `/model <alias>` switches the
active workstream's client, model, and context window
active workstream's client, model, context window, and per-model sampling
parameters
6. `_create_stream_with_retry()` tries the primary model, then each fallback
alias in order if the primary is unreachable
7. `_run_agent()` resolves `registry.agent_model` (if set) for plan/task
@@ -814,7 +884,7 @@ and are the single source of truth for both backends and Alembic migrations.
| `update_workstream_title(ws_id, title)` | Set/update LLM-generated title |
| `update_workstream_state(ws_id, state)` | Update workstream state and bump timestamp |
| `update_workstream_name(ws_id, name)` | Update workstream display name |
| `list_workstreams(node_id, limit)` | List workstreams, optionally by node |
| `list_workstreams(node_id, limit, *, parent_ws_id, kind, user_id)` | List workstreams, optionally filtered by node, parent, kind, or owning user |
| `kv_get(key)` / `kv_set(key, value)` / `kv_delete(key)` | Generic key-value store (backs memories table) |
| `kv_list()` / `kv_search(query)` | List or search key-value pairs |
| `search_history(query, limit)` | Full-text search (FTS5 on SQLite, tsvector on PostgreSQL) |
@@ -1055,8 +1125,9 @@ Three hierarchical scopes control endpoint access:
- **Console** is the auth management hub — it hosts the admin endpoints for
creating users, issuing API tokens, and managing channel mappings. User
records and token hashes live in the shared storage backend. The console
dashboard includes an **admin panel** (14 tabs) for managing
credentials, governance, MCP servers, and runtime settings through the browser.
dashboard includes an **admin panel** (18 tabs) for managing
credentials, governance, MCP servers, models, node metadata, and runtime
settings through the browser.
- **Server** is a JWT validator only — it validates tokens on each request but
never creates users or tokens. Both processes share the same `jwt_secret`
(via `TURNSTONE_JWT_SECRET` env var or `[auth].jwt_secret` config).
@@ -1291,9 +1362,10 @@ setup, auth headers, `_request()` (REST) and `_stream_sse()` (SSE). Sync
clients delegate through `_SyncRunner` which maintains a persistent background
event loop on a daemon thread.
**Event types**: 27 standalone dataclasses in `events.py` with a type-registry
pattern matching `OutboundEvent.from_json()` from `mq/protocol.py`. Events are
decoupled from server internals.
**Event types**: 38 standalone dataclasses in `events.py` with a type-registry
dispatch (`from_json()` on each event). Events are decoupled from server
internals — the SDK parses SSE frames directly from the `/v1/api/events`
streams.
**TypeScript SDK**: `sdk/typescript/` — separate npm package with the same API
surface. Zero browser dependencies, SSE via `fetch` + `ReadableStream` parsing.
@@ -1315,7 +1387,8 @@ with TurnstoneServer("http://localhost:8080", token="tok_xxx") as client:
> See also: [Channel Integrations guide](channels.md)
The `turnstone-channel` gateway connects external messaging platforms
(Discord, Slack, Teams) to the turnstone cluster via HTTP. Each
(Discord and Slack today, with an adapter protocol for future platforms) to
the turnstone cluster via HTTP. Each
platform adapter implements the `ChannelAdapter` protocol and translates
between platform-native events and turnstone server API calls.
@@ -1328,7 +1401,7 @@ workstream is reactivated, the router uses atomic resume via the
the old workstream's conversation during creation in a single HTTP
request, eliminating ordering fragility.
Discord ships as the first adapter. See [channels.md](channels.md) for
Discord and Slack adapters ship today. See [channels.md](channels.md) for
setup instructions, configuration reference, and the adapter development
guide.
@@ -1349,11 +1422,11 @@ retries up to 3 times with backoff, re-querying the service registry on
each attempt. See [Notification Flow diagram](diagrams/png/17-notify-flow.png).
**Bidirectional replies:** When a user replies to a notification DM, the
Discord bot looks up the originating `ws_id` from the tracked message ID,
verifies the replying user matches the notification recipient, and routes
the reply to the workstream via `router.send_message()`. The workstream's
response is forwarded back to the DM via a temporary entry in
`_notify_reply_channels`. On `TurnCompleteEvent`, the response message is
channel adapter (Discord or Slack) looks up the originating `ws_id` from the
tracked message ID, verifies the replying user matches the notification
recipient, and routes the reply to the workstream via `router.send_message()`.
The workstream's response is forwarded back to the DM via a temporary entry
in `_notify_reply_channels`. On `TurnCompleteEvent`, the response message is
itself tracked for further replies, enabling multi-turn DM conversations
without requiring the user to open the web UI. Tracking entries are capped
at 100 (FIFO eviction) and cleaned up on workstream close.
@@ -1386,11 +1459,10 @@ and workstreams record which skill and version spawned them. Token budget
enforcement tracks consumption in `session.send()` with 80% warning and
100% approval gate via the `__budget_override__` synthetic tool name.
The console admin panel adds 5 governance tabs (Roles, Policies, Skills,
Usage, Audit), a Memories tab, a Settings tab (form-based editor for all
ConfigStore settings), and an MCP Servers tab (database-backed server
definitions with live connection status and cluster-wide reload) for a
total of 13 tabs, all permission-gated.
The console admin panel exposes these capabilities as 18 permission-gated
tabs: Users, API Tokens, Channels, Schedules, Watches, Roles, Policies,
Prompts, Judge, Skills, MCP Servers, Usage, Audit, Memories, Models, Nodes,
Settings, and TLS.
Both Python and TypeScript SDKs expose governance methods on the console
client.
+237
View File
@@ -0,0 +1,237 @@
# Bulk endpoint shape contract
Turnstone exposes several endpoints and tool calls that take multiple
ids and return a per-id outcome. Over the last few phases two
**distinct** response shapes have settled, one per semantic category.
This doc codifies both so a future endpoint author can pick the right
shape by semantics instead of by coin-flip.
Existing bulk endpoints at time of writing:
| Endpoint / tool | Category | Response shape |
|---------------------------------------------------------|--------------------------|------------------------------------------|
| `GET /v1/api/cluster/ws/live?ids=a,b,c` | bulk read | `{results, denied, truncated}` |
| model tool `spawn_batch` | bulk create (per-item) | `{results, denied}` |
| `POST /v1/api/coordinator/{ws_id}/stop_cascade` | cascade mutation | `{cancelled, failed, skipped}` |
| `POST /v1/api/coordinator/{ws_id}/close_all_children` | cascade mutation | `{closed, failed, skipped}` |
---
## Why two shapes
The ask-to-outcome mapping is fundamentally different between the
two categories, and a one-size-fits-all envelope ends up papering
over distinctions the caller genuinely needs to branch on.
**Bulk read / bulk create-with-payload.** Each input id (or batch
index) carries a *request-side* concept — "give me the live block
for this ws_id" or "spawn a child with this spec" — and each
successful output carries a *payload* — the live block, or the new
workstream's identifying triple. The interesting distinction on
failure is *ownership / validation* (caller can't see that id, spec
was malformed) — independent of the storage state.
**Cascade mutation.** The action is uniform across every id (cancel
this subtree, close this child). The interesting distinctions on
outcome are *did it reach the terminal state?* (succeeded / already
was there / the dispatch itself failed) — driven by the storage
state plus transport reliability, not by the caller's input.
Trying to unify these forces either:
- a stateless `denied` bucket that has to carry "already gone"
*and* "you don't have permission" *and* "transport failed" with a
separate reason string — reviewers end up string-matching to branch.
- or a per-item-payload map for cascade mutations where every
successful value is the same sentinel — carrier with no payload.
So: two shapes, one per category. The rest of this doc spells out
each.
---
## Shape A — bulk read / bulk create-with-payload
```json
{
"results": { "<key>": <value-or-null>, ... },
"denied": [ "<key>", ... ],
"truncated": false
}
```
**`results`** is a key-indexed map of the positive-path payload.
The key is the input id for read endpoints (`cluster/ws/live` uses
the ws_id), or the input-array index (stringified) for create
endpoints that want ordering preserved (`spawn_batch` uses `"0"`,
`"1"`, ...). The value is whatever the endpoint produces per
success — a live block, a `{ws_id, name, node_id, status}` triple,
etc. A `null` value (read endpoints only) means "the id existed and
you own it, but the live block wasn't available" — distinct from
"denied".
**`denied`** is the negative-path list. For read endpoints it's a
flat list of ids (preserves input order so callers can re-zip
against their input). For create endpoints with per-item payloads
it's a list of `{idx, reason}` objects (`spawn_batch`'s validation
and spawn-error rows; also the operator-reject surface when per-item
selective-deny ships). Include every reason that's *not* the
positive path — authz, ownership, validation, already-consumed,
spawn failure — so callers don't branch on status codes.
**`truncated`** is a boolean set to `true` when the server's
per-endpoint input cap was exceeded and the tail was dropped. The
endpoint docs each spell out the cap (50 for `cluster/ws/live`).
`spawn_batch` hard-errors on overflow instead of silently
truncating — it omits the field entirely rather than carry a
permanently-false flag.
### Example — `cluster/ws/live`
```http
GET /v1/api/cluster/ws/live?ids=a1b2,c3d4,nonexistent,foreign HTTP/1.1
```
```json
{
"results": {
"a1b2": {"state": "running", "tokens": 12843, "activity": "..."},
"c3d4": null
},
"denied": ["nonexistent", "foreign"],
"truncated": false
}
```
Callers that need ordered output zip their original id list against
this map; ids in `denied` drop out of the zip cleanly. A live-block
`null` doesn't route to `denied` — the row exists and the caller
owns it; the node is just currently unreachable.
### Example — `spawn_batch`
```json
{
"results": {
"0": {"ws_id": "d4e5f6...", "name": "csrf-audit", "node_id": "gpu-3", "status": 200},
"2": {"ws_id": "f1a2b3...", "name": "xss-audit", "node_id": "gpu-1", "status": 200}
},
"denied": [
{"idx": 1, "reason": "skill not found: nonexistent-skill"}
]
}
```
Indexes are stringified to keep the envelope JSON-safe and
consistently-typed across the read and create cases.
---
## Shape B — cascade mutation
```json
{
"status": "ok",
"<bucket>": [ "<ws_id>", ... ],
"failed": [ "<ws_id>", ... ],
"skipped": [ "<ws_id>", ... ]
}
```
Where `<bucket>` is the endpoint-specific name for "succeeded" —
`cancelled` for `stop_cascade`, `closed` for `close_all_children`.
The three buckets partition the input set exactly once:
| Bucket | Meaning |
|---------------|-------------------------------------------------------------------------------|
| `<bucket>` | Action dispatch accepted; target reached the intended terminal state. |
| `failed` | Dispatch returned a non-404 error (transport issue, upstream 5xx, exception). |
| `skipped` | Upstream 404 — stale registry entry, row already deleted, or peer gone. |
The split between `failed` and `skipped` is load-bearing. `failed`
is actionable — the operator may want to retry, or the cascade may
be partial. `skipped` is pre-resolved — the target is already in
the terminal state the cascade was aiming at, so it's neither a
win to report nor a fault to fix.
### Example — `stop_cascade`
```json
{
"status": "ok",
"cancelled": ["child-1", "child-3"],
"failed": [],
"skipped": ["child-2"]
}
```
A subsequent retry would target only `failed` ids, not `skipped`
ones — the latter are already done.
### Example — `close_all_children`
```json
{
"status": "ok",
"closed": ["child-1", "child-3"],
"failed": ["child-2"],
"skipped": []
}
```
Same partition, different success-bucket name. When `coord_client`
is unavailable (session loaded but no HTTP client attached — a
construction bug) every id goes to `failed` so the operator notices
rather than getting a silent all-skipped response.
---
## Guidance for future bulk endpoints
1. **Pick by semantics, not by "what shape is nearby."**
- Mutation that's uniform across ids + terminal-state outcome? →
**Shape B** (cascade mutation).
- Read or create where the input id carries payload, or where the
denial axis is independent of storage state? → **Shape A**
(bulk read / bulk create-with-payload).
2. **Cap the input.** Both shapes assume a bounded input — the
server rejects or silently truncates past the cap. Document the
cap in the endpoint's OpenAPI description. Shape A uses
`truncated: true` on quiet truncation; Shape B hard-errors on
overflow.
3. **Match existing bucket names for the same semantic.** Use
`failed` and `skipped` verbatim in Shape B — the per-endpoint
success bucket is the only slot that varies. Use `results` and
`denied` verbatim in Shape A; the per-endpoint `<key>` /
`<value>` types vary.
4. **Audit the verbose shape.** Both endpoints emit a corresponding
audit event with the full before/after bucket lists — the SSE
stream and the in-process response give live feedback, but a
postmortem operator will read the audit row. Use
`_emit_coord_audit` (coordinator-scoped) or `record_audit`
directly; don't inline.
5. **Don't mix shapes within one endpoint.** If a bulk endpoint
wants both partial-success creation AND per-item failure reasons
(like `spawn_batch` with its `{idx, reason}` denial rows), that's
Shape A with a richer denial element — not a blend with Shape B.
---
## History
- **Phase 6** shipped `cluster/ws/live` as the first Shape A endpoint
(`{results, denied, truncated}`).
- **Phase 7** shipped `stop_cascade` as the first Shape B endpoint
(`{cancelled, failed, skipped}`).
- **Phase 8 PR A** shipped `spawn_batch` (Shape A, keyed by idx) and
`close_all_children` (Shape B, twin of `stop_cascade`), which
crystallised the two-shape-per-semantic-category policy codified
here.
Before adding a third shape, read this doc and argue for why the
new surface doesn't fit either A or B. Two idioms in the cluster
API is a finite operator tax; three is one too many.
+95 -21
View File
@@ -7,31 +7,35 @@ platform-native events (messages, button clicks, slash commands) into
turnstone API calls, and renders workstream output back into the
platform's UI.
Discord ships as the first adapter. The adapter protocol is designed for
future Slack and Teams integrations.
Discord and Slack adapters ship today. The adapter protocol is designed
so new platforms can be added with only a new package under
`turnstone/channels/<platform>/`.
---
## Architecture
```
Discord Gateway
|
v
turnstone-channel (Discord adapter)
|
v
turnstone-server (direct HTTP)
or
turnstone-console (routing proxy, multi-node)
Discord Gateway Slack (Socket Mode WebSocket)
\ /
v v
turnstone-channel (one or more adapters)
|
v
turnstone-server (direct HTTP)
or
turnstone-console (routing proxy, multi-node)
```
A single `turnstone-channel` process can run multiple adapters
simultaneously (e.g. Discord + Slack) — pass the tokens for each
platform you want to enable.
Key components:
- **ChannelAdapter protocol** (`turnstone/channels/_protocol.py`) — generic
interface for any messaging platform. Defines `start()`, `stop()`,
`send()`, `send_notification()`, `edit_message()`,
`send_approval_request()`, `send_plan_review()`, and `create_thread()`.
`send()`, and `send_notification()`.
- **ChannelRouter** (`turnstone/channels/_routing.py`) — maps
channel/thread IDs to turnstone workstream IDs. Handles workstream
creation via HTTP, stale route detection, and user identity resolution.
@@ -120,6 +124,68 @@ An admin can also force-link or unlink users via the console admin panel
---
## Slack Setup
Slack uses **Socket Mode**, so no public URL or API Gateway is required — Slack
connects outbound to the bot via a WebSocket. Install with:
```bash
pip install 'turnstone[slack]'
```
### 1. Create a Slack App
1. Go to https://api.slack.com/apps and click **Create New App**
2. Under **Settings > Socket Mode**, enable Socket Mode. This generates an
**App-Level Token** (prefix `xapp-`) — copy it.
3. Under **OAuth & Permissions**, add these **Bot Token Scopes**:
`chat:write`, `chat:write.public`, `channels:history`, `im:history`,
`groups:history`, `mpim:history`, `reactions:write`, `commands`
4. Under **Event Subscriptions** (Socket Mode delivers events), subscribe
to bot events: `message.channels`, `message.im`, `message.groups`
5. Under **Slash Commands**, create a command (default `/turnstone`)
6. Install the app to your workspace to generate the **Bot User OAuth
Token** (prefix `xoxb-`).
### 2. Configure Turnstone
**Environment variables** (recommended for Docker):
```bash
TURNSTONE_SLACK_TOKEN=xoxb-... # Bot User OAuth Token
TURNSTONE_SLACK_APP_TOKEN=xapp-... # App-Level Token (Socket Mode)
TURNSTONE_SLACK_CHANNELS= # optional, comma-separated channel IDs
TURNSTONE_SLACK_SLASH_COMMAND=/turnstone
```
**CLI flags** (bare-metal):
```bash
turnstone-channel \
--slack-token "xoxb-..." \
--slack-app-token "xapp-..." \
--slack-slash-command /turnstone \
--server-url http://localhost:8080
```
The Slack and Discord adapters can be enabled together — pass tokens for
both and the gateway hosts both adapters in one process.
### 3. Usage
- **DM the bot**: messages sent directly to the bot create a workstream
scoped to that DM; the slash command is not required.
- **Slash command**: `/turnstone <message>` in any channel the bot can
see starts a per-user channel session.
- Tool approvals render as Slack **Block Kit** buttons; only the user
who owns the workstream can approve/reject.
- Plan reviews render as a modal with approve / request-changes actions.
- Notifications and reply routing work identically to Discord.
- Session recovery: persisted channel routes are re-subscribed when the
bot restarts, so existing Slack conversations keep flowing.
---
## Usage
### Conversations
@@ -184,9 +250,13 @@ Plan review requests are displayed as a blue embed with:
| CLI Flag | Env Var | Default | Description |
|----------|---------|---------|-------------|
| `--discord-token` | `TURNSTONE_DISCORD_TOKEN` | — | Bot token (required to enable Discord) |
| `--discord-token` | `TURNSTONE_DISCORD_TOKEN` | — | Discord bot token (required to enable Discord) |
| `--discord-guild` | — | `0` (all guilds) | Restrict to a single Discord guild |
| `--discord-channels` | — | empty (all) | Comma-separated channel IDs to allow |
| `--discord-channels` | — | empty (all) | Comma-separated Discord channel IDs to allow |
| `--slack-token` | `TURNSTONE_SLACK_TOKEN` | — | Slack Bot User OAuth token (`xoxb-…`, required to enable Slack) |
| `--slack-app-token` | `TURNSTONE_SLACK_APP_TOKEN` | — | Slack App-Level token (`xapp-…`, required with `--slack-token`) |
| `--slack-channels` | `TURNSTONE_SLACK_CHANNELS` | empty (all) | Comma-separated Slack channel IDs to allow |
| `--slack-slash-command` | `TURNSTONE_SLACK_SLASH_COMMAND` | `/turnstone` | Slash command name registered in the Slack app |
| `--server-url` | `TURNSTONE_SERVER_URL` | `http://localhost:8080` | Server URL (single-node) |
| `--console-url` | `TURNSTONE_CONSOLE_URL` | — | Console URL (multi-node routing proxy) |
| `--model` | — | server default | Default model for new workstreams |
@@ -196,6 +266,9 @@ Plan review requests are displayed as a blue embed with:
| `--log-level` | `TURNSTONE_LOG_LEVEL` | `INFO` | Log level |
| `--log-format` | `TURNSTONE_LOG_FORMAT` | `auto` | Log format (`auto`/`json`/`text`) |
At least one of `--discord-token` or `--slack-token` must be supplied.
Passing both starts both adapters in the same process.
---
## User Identity
@@ -249,8 +322,8 @@ waiting for them to check in.
Two modes:
- **Username** — provide a turnstone `username`. The gateway resolves
it via the `channel_users` table and sends to all linked channels
(e.g. Discord + future Slack).
it via the `channel_users` table and sends to every linked platform
the user has (e.g. Discord + Slack).
- **Direct** — provide `channel_type` + `channel_id` to target a
specific platform channel or user DM.
@@ -351,10 +424,6 @@ class ChannelAdapter(Protocol):
async def stop(self) -> None: ...
async def send(self, channel_id: str, content: str) -> str: ...
async def send_notification(self, channel_id: str, content: str, ws_id: str) -> str: ...
async def edit_message(self, channel_id: str, message_id: str, content: str) -> None: ...
async def send_approval_request(self, channel_id: str, ws_id: str, correlation_id: str, items: list[dict]) -> None: ...
async def send_plan_review(self, channel_id: str, ws_id: str, correlation_id: str, content: str) -> None: ...
async def create_thread(self, parent_channel_id: str, name: str, message_id: str = "") -> str: ...
```
`send_notification()` is like `send()` but associates the outgoing
@@ -362,6 +431,11 @@ message with a `ws_id` so that user replies can be routed back to the
originating workstream. Adapters must track the mapping from outgoing
message ID to `(ws_id, target_user_id)` and handle DM replies.
Platform-specific concerns — approval prompts, plan reviews, message
edits, thread creation — live inside the adapter implementation and are
not part of the protocol surface. Each adapter drives those via its
own `_on_ws_event` dispatcher using SDK-native APIs.
To add a new platform:
1. Create `turnstone/channels/<platform>/` package
+16 -4
View File
@@ -382,6 +382,9 @@ Triggered by the "+ new" header button. A modal dialog with:
- **Profile** — optional dropdown listing enabled skills. Applies the skill's model, auto-approve policy, token budget, and other behavioral settings at creation time.
- **Name** — optional text input. Auto-generated if left empty.
- **Model** — optional text input for a model alias from the target node's registry.
- **Judge Model** — optional text input for the judge model alias (overrides the default judge model for this workstream).
Keyboard shortcuts: Ctrl+Shift+R (refresh title), Ctrl+Shift+E (edit title), Ctrl+Shift+F (fork), Ctrl+Shift+X (delete). Press ? for full shortcut help.
On submit, `POST /v1/api/cluster/workstreams/new` dispatches the creation request. A toast confirms success; the SSE stream delivers the `ws_created` event to update the dashboard.
@@ -393,10 +396,19 @@ The browser maintains a local `clusterState` object that mirrors the cluster sna
Accessed via the "admin" button in the header (visible when authenticated
with `approve` scope). Provides user, API token, channel link, MCP server,
and skill management with 13 tabs (see also
[Governance](governance.md) for
the Roles, Policies, Skills, Usage, and Audit tabs, and
[Settings](settings.md) for the database-backed configuration editor):
and skill management with 18 tabs (Users, API Tokens, Channels, Schedules,
Watches, Roles, Policies, Prompts, Judge, Skills, MCP Servers, Usage,
Audit, Memories, Models, Nodes, Settings, TLS). See also
[Governance](governance.md) for the Roles, Policies, Skills, Usage, and
Audit tabs, and [Settings](settings.md) for the database-backed
configuration editor.
The **Channels** tab links users to either a Discord or Slack account
via a per-row channel-type selector. The **Models** tab is a CRUD
editor for `model_definitions`, the **Nodes** tab edits per-node
metadata, and the **TLS** tab manages CA and leaf certificates for the
internal mTLS fabric. The **Settings** tab edits ConfigStore values
live; edits apply without restart.
**Users tab:**
+372
View File
@@ -0,0 +1,372 @@
# Coordinator API tour
Turnstone's **coordinator workstream** is a session hosted on the
console whose job is to orchestrate other workstreams. It runs an LLM
that can spawn child workstreams on any node, watch their progress,
wait for them to finish, steer them mid-flight, and tear them down.
This doc walks the full lifecycle — one request, one response, and the
relevant SSE events at each step.
Aimed at integrators driving a coordinator from a custom UI or SDK
without reverse-engineering the built-in console page. The shapes
here match the live OpenAPI spec served at `/openapi.json` and
rendered at `/docs` on every `turnstone-console` process. Every
step references the operation id from that spec so doc updates track
schema changes.
> **Auth throughout.** Every endpoint below sits behind bearer-token
> auth and the `admin.coordinator` permission. A session-scoped JWT
> is minted per login (see [docs/oidc.md](oidc.md) / [docs/security.md](security.md));
> a service token may call the read paths but destructive governance
> paths (`/restrict`, `/stop_cascade`, `/close_all_children`) require
> the explicit `admin.coordinator` grant — a service-token owner
> match isn't enough.
---
## The 9 steps
| # | Action | Operation | Operation id |
|---|------------------------------|-------------------------------------------------------------|-------------------------------------------------------------|
| 1 | Create | `POST /v1/api/coordinator/new` | `v1_api_coordinator_new_post` |
| 2 | Subscribe to events | `GET /v1/api/coordinator/{ws_id}/events` (SSE) | `v1_api_coordinator_{ws_id}_events_get` |
| 3 | Send a user message | `POST /v1/api/coordinator/{ws_id}/send` | `v1_api_coordinator_{ws_id}_send_post` |
| 4 | Inspect children | `GET /v1/api/coordinator/{ws_id}/children` | `v1_api_coordinator_{ws_id}_children_get` |
| 5 | Inspect one workstream | `GET /v1/api/cluster/ws/{ws_id}/detail` | `v1_api_cluster_ws_{ws_id}_detail_get` |
| 6 | Wait for fan-out | model-side tool `wait_for_workstream` | — (tool call, not HTTP) |
| 7 | Govern | `POST /v1/api/coordinator/{ws_id}/trust` | `v1_api_coordinator_{ws_id}_trust_post` |
| | | `POST /v1/api/coordinator/{ws_id}/restrict` | `v1_api_coordinator_{ws_id}_restrict_post` |
| | | `POST /v1/api/coordinator/{ws_id}/stop_cascade` | `v1_api_coordinator_{ws_id}_stop_cascade_post` |
| | | `POST /v1/api/coordinator/{ws_id}/close_all_children` | `v1_api_coordinator_{ws_id}_close_all_children_post` |
| 8 | Approve / cancel | `POST /v1/api/coordinator/{ws_id}/approve` | `v1_api_coordinator_{ws_id}_approve_post` |
| | | `POST /v1/api/coordinator/{ws_id}/cancel` | `v1_api_coordinator_{ws_id}_cancel_post` |
| 9 | Close | `POST /v1/api/coordinator/{ws_id}/close` | `v1_api_coordinator_{ws_id}_close_post` |
---
## 1. Create a coordinator
```http
POST /v1/api/coordinator/new
Content-Type: application/json
Authorization: Bearer <token>
{
"name": "release-coord",
"skill": "engineer-orchestrator",
"initial_message": "audit /auth for CSRF handling across all active routes"
}
```
```http
HTTP/1.1 201 Created
Content-Type: application/json
{"ws_id": "a1b2c3d4e5f6...", "name": "release-coord"}
```
All three body fields are optional — an empty body still creates a
coordinator with an auto-generated name and no initial message.
Returns **503** with a remediation message when the cluster isn't
configured with a coordinator model; see
[`coordinator.model_alias`](settings.md) to set one.
**SSE implication:** the `ws_created` event fires on the cluster-wide
stream (`/v1/api/cluster/events`) once the row is committed. Per-ws
subscribers (step 2) see the session warm up as token traffic starts.
---
## 2. Subscribe to the per-coordinator event stream
```http
GET /v1/api/coordinator/{ws_id}/events HTTP/1.1
Accept: text/event-stream
Authorization: Bearer <token>
```
One persistent SSE connection per browser tab / SDK caller — the
console fans each event out to every listener queue (cap 500 events
per queue, put_nowait drop on overflow). Events come in flat JSON
with a `type` field. The recurring shapes a UI has to handle:
| `type` | Emitted when | Payload highlights |
|---------------------|--------------------------------------------------------------------------------------------|--------------------|
| `thinking_start` / `thinking_stop` | Model has entered / exited a reasoning block | — |
| `reasoning` | Reasoning-token stream chunk (when the model exposes it) | `text` |
| `content` | Assistant-content stream chunk | `text` |
| `stream_end` | End of a single provider stream | — |
| `tool_result` | A tool call completed (success or error) | `call_id`, `name`, `output`, `is_error?` |
| `tool_output_chunk` | Streaming tool output (e.g. long bash command) | `call_id`, `chunk` |
| `approve_request` | One or more tool calls need operator approval | `items: [{call_id, header, preview, func_name, approval_label, needs_approval}]` |
| `approval_resolved` | Operator answered the approval prompt | `approved`, `feedback` |
| `state_change` | Worker-thread state transition | `state``running`, `thinking`, `attention`, `idle`, `error` |
| `status` | Token usage + context-window snapshot (fires on every streaming tick) | `prompt_tokens`, `completion_tokens`, `total_tokens`, `context_window`, `pct`, `effort`, `cache_creation_tokens`, `cache_read_tokens` |
| `rename` | Session's display name changed | `name` |
| `intent_verdict` | Intent judge produced a verdict on a pending tool call | `risk_level`, `recommendation`, `reasons` |
| `output_warning` | Output guard flagged a tool result | `call_id`, `risk_level`, `flags` |
| `child_ws_created` | A direct child of this coord was just created (fan-out from the cluster bus) | `child_ws_id`, `node_id`, `name`, `parent_ws_id` (`ws_id` in the envelope is always the coord's own id) |
| `child_ws_state` | A direct child transitioned state | `child_ws_id`, `state` |
| `child_ws_closed` | A direct child closed | `child_ws_id` |
| `child_ws_rename` | A direct child's name changed | `child_ws_id`, `name` |
| `wait_started` / `wait_progress` / `wait_ended` | `wait_for_workstream` tool lifecycle (see §6) | `call_id`, `ws_ids`, `elapsed`, `results`, `complete` |
| `batch_started` / `batch_ended` | `spawn_batch` / `close_all_children` tool lifecycle | `call_id`, `op`, `total`/`succeeded`/`denied`/`closed`/`failed`/`skipped` |
| `info` / `error` | Operational messages | `message` |
**Reconnection contract:** a freshly-opened SSE connection receives
the current snapshot of any pending tool approval (`approve_request`
is re-sent if unresolved) and any in-flight `wait_*` / `batch_*`
indicator — so a tab refresh mid-approval doesn't strand the
operator.
---
## 3. Send the first user message
```http
POST /v1/api/coordinator/{ws_id}/send
Content-Type: application/json
{"message": "audit /auth for CSRF handling across all active routes"}
```
```http
HTTP/1.1 200 OK
{"status": "ok"}
```
The message is queued for the worker thread at its next tool-result
seam (so you can send follow-ups mid-conversation without corrupting
the in-progress turn). On the SSE stream you'll see `state_change`
`thinking_start` → streaming `reasoning` / `content` / `tool_result`
events, finishing with `state_change → idle` or an
`approve_request` when the model invokes a gated tool.
---
## 4. Inspect direct children
```http
GET /v1/api/coordinator/{ws_id}/children HTTP/1.1
```
```json
{
"items": [
{"ws_id": "d4e5f6...", "name": "csrf-audit", "state": "running", "node_id": "gpu-3"},
{"ws_id": "e1f2a3...", "name": "xss-audit", "state": "idle", "node_id": "gpu-1"}
],
"truncated": false
}
```
The response key is `items`, not `children` — the endpoint shape
follows the cluster-wide workstream-list idiom rather than the
coordinator `list_workstreams` tool's (which uses `children`).
Rows include every state stored for the parent (`running`, `idle`,
`closed`, ...); the endpoint does not accept a state query param,
so clients should inspect each row's `state` field and filter
locally if they want to hide closed/deleted children. Nested
coordinator rows are dropped server-side so only interactive
descendants appear.
---
## 5. Inspect one workstream (storage + live block + tail)
```http
GET /v1/api/cluster/ws/{ws_id}/detail?message_limit=20 HTTP/1.1
```
```json
{
"persisted": { "ws_id": "...", "state": "running", "parent_ws_id": "...", "kind": "interactive", ... },
"live": { "state": "thinking", "tokens": 12843, "activity": "...", "pending_approval": null },
"tail": [ {"role": "assistant", "content": "...", "tokens": 128}, ... ]
}
```
Works for any workstream the caller has `admin.cluster.inspect` on,
not just children of a single coordinator — useful for a cluster
admin panel watching multiple coordinators at once. `live` is
`null` when the owning node is unreachable or has dropped the row
from its dashboard cache; callers should degrade gracefully, not
treat it as an error.
For fan-out views, prefer
[`GET /v1/api/cluster/ws/live?ids=a,b,c`](bulk-endpoints.md) — it
collapses N per-row round-trips into one, returning the live block
for every id in a `{results, denied, truncated}` envelope.
---
## 6. Wait for fan-out (`wait_for_workstream`)
`wait_for_workstream` is a **model-side tool**, not an HTTP endpoint
— the coordinator's LLM invokes it with a list of child ws_ids, the
session's worker thread blocks inside the tool, and a sequence of
`wait_started` / `wait_progress` / `wait_ended` SSE events is emitted
for the UI to drive a "waiting on N children" indicator.
![wait_for_workstream sequence](diagrams/png/27-coordinator-wait-for-workstream.png)
Key properties:
- **Caps** — up to 32 ws_ids per call, up to 600 seconds per call.
A coordinator that needs to wait on more children re-invokes the
tool with a fresh timeout.
- **Modes**`mode="any"` returns as soon as one child reaches a
real terminal state (`idle` / `error` / `closed` / `deleted`);
`mode="all"` waits for every polled child.
- **Progress throttling** — the poll loop runs every 500 ms but the
SSE emission is diff-on-state-change plus a 5-second heartbeat. A
600 s wait generates O(dozens) of progress events, not 1200.
- **Denied rows** — an id the caller doesn't own (cross-tenant) or a
missing row is reported as a `denied` state in the results dict;
`mode="any"` won't satisfy on a pure-denied list (the LLM should
treat it as a config error, not a completion).
Prefer `wait_for_workstream` over polling `inspect_workstream` in a
loop — a wait consumes one assistant turn regardless of how long the
children take, whereas each `inspect_workstream` poll costs a full
turn (plus judge, plus tokens). On a fan-out of 3+ children this
rounds to a 10× token-efficiency win.
---
## 7. Governance — trust, restrict, stop_cascade, close_all_children
These four endpoints let an operator steer a live coordinator session
mid-flight. All four emit an audit event tagged
`coordinator.<action>` via the dedicated audit executor so a cascade
burst can't starve audit writes.
### `POST /trust` — auto-approve own-subtree sends
```json
POST /v1/api/coordinator/{ws_id}/trust
{"send": true}
```
Flips `trust_send=true` on the live session. Subsequent
`send_to_workstream` calls that target a ws_id in the coordinator's
own subtree skip the approval prompt; foreign ws_ids and other tool
calls still go through the normal flow. Requires both
`admin.coordinator` AND `coordinator.trust.send` permissions (the
second grants a service token the opt-in it otherwise wouldn't get).
### `POST /restrict` — revoke tool access mid-session
```json
POST /v1/api/coordinator/{ws_id}/restrict
{"revoke": ["spawn_workstream", "delete_workstream"]}
```
Unions the names into the session's revoked-tools set. Additive and
idempotent — calling twice with overlapping lists converges to the
union. Revocations don't survive a session close/reopen; operators
opt in per session. Cap 256 tool names per request, 128 chars each.
### `POST /stop_cascade` — cancel the subtree
```json
POST /v1/api/coordinator/{ws_id}/stop_cascade
{}
```
Cancels the coordinator's in-flight generation AND dispatches
`cancel_workstream` through the routing proxy for every direct
child in the in-memory registry. Returns:
```json
{"status": "ok", "cancelled": ["child-1", "child-3"], "failed": [], "skipped": ["child-2"]}
```
Response uses the [cascade-mutation bulk shape](bulk-endpoints.md):
`cancelled` = accepted, `failed` = dispatch error worth retrying,
`skipped` = upstream 404 (already gone — stale registry entry or
the row was deleted between snapshot and dispatch). Grandchildren
aren't touched directly; they sit behind their parent's cancel and
propagate via the child's SSE stream.
### `POST /close_all_children` — soft-close the direct fan-out
```json
POST /v1/api/coordinator/{ws_id}/close_all_children
{"reason": "audit round complete"}
```
Response:
```json
{"status": "ok", "closed": ["c-1", "c-2"], "failed": [], "skipped": []}
```
Soft-close cascade bounded by the same semaphore as `stop_cascade`.
The `reason` (up to 512 chars) propagates into each closed child's
audit + `workstream_config` for postmortem. Unlike `stop_cascade`
this does NOT recurse into grandchildren — the model-facing tool
that pairs with this endpoint asks for a bounded teardown of the
coordinator's own fan-out. For a full-subtree teardown, use
`stop_cascade`.
See [bulk-endpoints.md](bulk-endpoints.md) for why both endpoints
share the cascade-mutation shape and how it differs from the
`spawn_batch` / `cluster/ws/live` shape.
---
## 8. Approve / cancel
The `approve` endpoint is what resolves an `approve_request` SSE
event. The coordinator's worker thread is blocked inside
`ui.approve_tools` waiting for this POST.
```json
POST /v1/api/coordinator/{ws_id}/approve
{"approved": true, "feedback": null, "always": false}
{"approved": false, "feedback": "spawn count looks too high — try 3 not 10"}
{"approved": true, "feedback": null, "always": true} // always-approve this tool name
```
`cancel` drops the in-flight generation but leaves the coordinator
idle and open for a fresh `send`:
```json
POST /v1/api/coordinator/{ws_id}/cancel
{}
```
---
## 9. Close
```json
POST /v1/api/coordinator/{ws_id}/close
{}
```
Soft-closes the session — state persists, children keep running (use
`close_all_children` or `stop_cascade` first to wind them down), the
worker thread exits, SSE streams send a final `stream_end` and
disconnect. The row is reopenable via
`POST /v1/api/coordinator/{ws_id}/open` so long as it hasn't been
deleted.
---
## Further reading
- [coordinator-skills.md](coordinator-skills.md) — writing a skill
that runs on a coordinator session (orchestrator persona,
workflow patterns, `SkillKind` classifier).
- [bulk-endpoints.md](bulk-endpoints.md) — the two bulk-shape
idioms (`{results, denied, truncated}` vs
`{<bucket>, failed, skipped}`) used by `cluster/ws/live`,
`spawn_batch`, `stop_cascade`, and `close_all_children`.
- [architecture.md](architecture.md) — cluster-wide architecture
including how coordinator sessions fit next to node-hosted
interactive workstreams.
- The live OpenAPI spec (`/openapi.json` on any console process)
and Swagger UI (`/docs`) — authoritative schemas for every
endpoint above.
+321
View File
@@ -0,0 +1,321 @@
# Writing a coordinator-specific skill
Skills are prompt-level personas that steer a Turnstone session
toward a narrow task. Most skills target **interactive** sessions —
the single-workstream "do this thing" surface where the model wields
`bash`, `edit_file`, `web_fetch`, and the rest of the maker toolset.
A **coordinator skill** is different. It runs on a session whose job
is to orchestrate other sessions. The toolset is smaller and
narrower, the persona is an orchestrator instead of a maker, and the
success metric is "did the plan resolve" instead of "did the code
compile". This doc covers the differences a skill author has to
care about.
---
## The two-surface model
A row in `prompt_templates` carries a `kind` column (see
[`turnstone/core/skill_kind.py`](../turnstone/core/skill_kind.py);
migration 044 added the column). Three values:
| `SkillKind` enum | Stored as | Visible in |
|----------------------|-----------------------------|---------------------------------------------------------------------------|
| `SkillKind.INTERACTIVE` | `"interactive"` | Only the interactive-session activation path. `list_skills` on a coord won't show it. |
| `SkillKind.COORDINATOR` | `"coordinator"` | Only the coordinator's `list_skills` tool. Hidden from interactive activation pickers. |
| `SkillKind.ANY` | `"any"` | Both surfaces. Default for legacy rows predating the classifier. |
The `kind` field is a `StrEnum` — drop-in ``str`` compatible — so
DB rows, JSON payloads, and `==` comparisons all work without
translation at the edge.
When a coordinator calls `list_skills`, the SQL filter narrows to
`kind IN ('coordinator', 'any')`. When an interactive session picks
a skill at activation, the filter narrows to
`kind IN ('interactive', 'any')`. A skill author tags once at
creation; the two surfaces stay partitioned without any
per-call filtering on the LLM side.
**Tagging a new skill as coordinator-only** — set `kind` to
`SkillKind.COORDINATOR` (or the literal string `"coordinator"`) when
you POST to `/v1/api/admin/skills`. Existing rows default to
`SkillKind.ANY`; bump them to `COORDINATOR` if you've rewritten the
prompt around the orchestrator toolset.
---
## Tool surface differences
Coordinator sessions receive a **fixed** tool set, defined in
`turnstone/core/tools.py` as `COORDINATOR_TOOLS`. Nothing a skill
or MCP config can do adds to it. Current members:
| Tool | Category | Notes |
|---------------------------|-----------------|---------------------------------------------------------------------|
| `spawn_workstream` | delegate | Create one child. Requires approval. |
| `spawn_batch` | delegate | Create up to 10 children in one approval. Partial-success shape. |
| `inspect_workstream` | observe | Read state + tail of one child. Auto-approved (no mutation). |
| `list_workstreams` | observe | List the direct children (same shape as `/children` endpoint). |
| `wait_for_workstream` | block | Block until one/all listed children hit a terminal state. |
| `send_to_workstream` | steer | Queue a follow-up message to a running child. |
| `close_workstream` | wind-down | Soft-close one child. Requires approval. |
| `close_all_children` | wind-down | Soft-close every direct child in one approval. Partial-success shape. |
| `cancel_workstream` | wind-down | Drop the in-flight generation; leaves child idle for a fresh send. |
| `delete_workstream` | wind-down | Hard-delete one child. Requires approval. |
| `list_nodes` | discover | Enumerate live cluster nodes + capabilities. |
| `list_skills` | discover | Coordinator-visible skills only (SkillKind filter above). |
| `task_list` | plan | Orchestrator-only scratchpad. Children don't see it. |
Explicitly **not** in the coordinator set:
- `bash` / `edit_file` / `write_file` / `append_file` / `diff_file` — no local FS.
- `read_file` / `search` — no local FS reads.
- `web_fetch` / `web_search` — no direct web access.
- `task_agent` / `plan_agent` — sub-agent tools are zeroed on coord sessions.
- `memory` / `recall` / `notify` / `watch` / `read_resource` / `use_prompt` / `skill` — the orchestrator's "memory" is its children's outputs; these UX / persistence tools belong to interactive sessions.
If your skill needs a coordinator to "run a command" or "read a
file", write the delegate pattern instead: spawn a child with an
appropriate skill, `wait_for_workstream`, then `inspect_workstream`
for the output. The coordinator stays the orchestrator.
---
## Persona differences
Interactive skills compose on top of `base_interactive.md` — a
"maker" persona: get the work done, use the tools, edit the code,
close the loop.
Coordinator skills compose on top of
[`base_coordinator.md`](../turnstone/prompts/base_coordinator.md) —
an "orchestrator" persona: decompose, delegate, monitor, synthesise.
The base text is short but sets the tone every coordinator skill
inherits:
> You are a coordinator on a small, focused infrastructure team.
> Your role is to orchestrate work across the cluster... You do
> not edit files, run shell commands, browse the web, or manipulate
> the codebase directly. Children do that.
Write your skill's system prompt to *add* task-specific orchestration
hints on top — don't re-explain the role, don't paste tool JSON,
don't try to override the "no direct action" contract. Keep the
additions to: (a) the specific kind of work this skill delegates;
(b) the preferred skill tags for children; (c) the synthesis shape
the skill should end on.
---
## `task_list` integration
`task_list` is the coordinator's scratchpad — a persisted, ordered
list of rows with fields `{id, title, status, child_ws_id, created,
updated}` that only this coordinator sees. Children don't see it;
the user does via the sidebar. Five actions: `add`, `update`,
`remove`, `reorder`, `list` (only `list` is auto-approved; the
mutators go through the approval flow).
The input schema refers to rows by `task_id`; the persisted row
object exposes the same id as `id`. The `child_ws_id` field is a
free-form label the skill sets to link a task to a spawned
workstream — it is NOT validated against the workstreams table, so
a skill can set it to a placeholder before `spawn_workstream`
returns or keep it pointing at a closed child for later audit.
A skill's initial prompt can seed the task list by calling
`task_list(action="add", title=...)` as its very first tool calls —
the user gets a visible plan before any child is spawned, and the
coordinator's future self has something concrete to iterate on.
Status transitions (`pending``in_progress``done` / `blocked`)
are the skill's main feedback loop: mutate the task when the child
covering it finishes, not when the child starts. Use
`task_list(action="update", task_id=..., child_ws_id=<ws_id>)` to
link a task to the child that owns it once spawn returns.
A final gotcha: parallel tool dispatch does NOT serialise reads
after writes in the same batch. If a skill issues an `update` and
a `list` in one parallel tool batch, the `list` response may reflect
the pre-update state. Dispatch mutate and list serially (one
tool_use turn each) when the list must observe the mutation.
Keep the tasks coarse-grained — one per child, roughly. A 20-task
list for a 3-child fan-out is noise; a 1-task list for a 5-child
fan-out loses the plan. The sidebar renders tasks as the operator's
mental model of "what the coord thinks it's doing".
---
## Referencing children by `ws_id`
Every ws_id returned by `spawn_workstream` / `spawn_batch` is a
**full 32-char hex string**. The skill's system prompt must not
invent ws_ids — a model that hallucinates `"child-1"` or `"ws-abc"`
hits the tenant guard in `CoordinatorClient._is_own_subtree`, which
validates ws_id against `parent_ws_id=coord_ws_id` AND
`user_id=owner` in storage. The rejection shape varies by tool:
- **Mutating ops** (`send_to_workstream`, `close_workstream`,
`cancel_workstream`, `delete_workstream`) return
`{"error": "workstream not in coordinator subtree: <ws_id>", "status": 404}`
— the skill should treat this as a tool error, not an empty result.
- **`inspect_workstream`** returns `{"error": "workstream not found: <ws_id>"}`
(same shape as a genuinely missing row, so the guard can't be
used as an existence oracle).
- **`wait_for_workstream`** reports the offending id with
`state="denied"` in its `results` dict; `mode="any"` won't
satisfy on a pure-denied list, so a hallucinated id won't trick
the wait into reporting "complete".
Pattern: capture each spawn result in the next tool call's input.
The JSON tool-result carries `{"ws_id": "...", "name": "...",
"node_id": "...", "status": 200}`; the model should extract the
ws_id and pass it to `inspect_workstream` / `wait_for_workstream` /
`send_to_workstream` / `close_workstream` verbatim.
A UI that wants human-readable identifiers should render the `name`
field and keep the ws_id as the click-through key.
---
## `wait_for_workstream` vs `inspect_workstream`
Two distinct semantics, different cost profiles:
- **`wait_for_workstream(ws_ids=[...], timeout=60, mode="any")`** —
blocks inside a single tool call until one (or all, for `mode="all"`)
of the listed children reaches a terminal state (`idle`, `error`,
`closed`, `deleted`). The worker thread blocks up to `timeout`
seconds; the assistant turn remains a single round-trip regardless
of how long the wait actually takes. Prefer this for "the plan
needs child X to finish before the next step."
- **`inspect_workstream(ws_id=...)`** — single read of the child's
state + tail. Costs a full assistant turn (judge, tokens, stream).
Prefer this for "what does the final message say?" after the child
has already resolved (via `wait_for_workstream` or a known
transition).
Rule of thumb: wait once for a fan-out, then inspect once per
child for the content. A loop of inspect-every-few-seconds is a
token-burning antipattern — on 3+ children it rounds to a 10×
efficiency hit over a wait+inspect pair.
---
## Common coordinator patterns
Three patterns cover most coordinator skills. Pick the one that
matches the task, or combine them deliberately.
### Pattern 1 — delegate-and-summarise
One specialist child, one focused brief, one synthesis message back
to the user. Appropriate when the user's request is "run the thing
and tell me what happened" and the work fits in one workstream.
```
task_list(action='add', title='audit /auth for CSRF')
spawn_workstream(skill='engineer', initial_message='audit /auth ...')
wait_for_workstream(ws_ids=[<child>], timeout=300)
inspect_workstream(ws_id=<child>)
→ synthesise the final message into a user-facing response
task_list(action='update', task_id='t_01', status='done')
close_workstream(ws_id=<child>, reason='audit complete')
```
### Pattern 2 — fan-out-and-synthesise
N children running in parallel, each with a distinct brief, all
waited-on together, then synthesised. Appropriate when the user's
request naturally decomposes into independent subtasks.
```
task_list seeds:
t_01 benchmark Anthropic 4.7 latency on summarisation
t_02 benchmark OpenAI GPT-5.2 latency on summarisation
t_03 benchmark Gemini 2.5 latency on summarisation
spawn_batch(children=[...3 briefs...])
wait_for_workstream(ws_ids=[c1, c2, c3], mode='all', timeout=600)
inspect_workstream(ws_id=c1); ...(c2); ...(c3)
→ synthesise head-to-head comparison
task_list → all done
close_all_children(reason='benchmark complete')
```
Prefer `spawn_batch` over 3 individual `spawn_workstream` calls —
one approval instead of three, one audit trail, deterministic
sibling ordering. Pair with `wait_for_workstream(mode='all')` and
`close_all_children(reason=...)` to wind the fan-out down in one
approval each.
### Pattern 3 — plan-then-delegate
The coordinator first uses its own reasoning to carve the plan,
records it in `task_list`, then spawns children that each own one
task. Appropriate when the user's request is "figure out how to X"
and the coordinator's planning step is itself valuable.
```
→ coord reasons about the shape of the work
task_list(action='add', title='...') × N # the plan, visible in the sidebar
for task in tasks:
spawn_workstream(skill=..., initial_message=task.brief)
task_list(action='update', task_id=task.id, notes='ws=<child_ws_id>')
wait_for_workstream(ws_ids=[...], mode='all', timeout=...)
for child in children:
inspect_workstream(ws_id=child)
task_list(action='update', task_id=..., status='done', notes='result summary')
→ synthesise
```
The key distinction from Pattern 2: the plan is an artifact the user
can see and interact with (via the sidebar). If the coordinator's
reasoning-pass was wrong about the decomposition, the user can
course-correct before any child runs.
---
## Testing a coordinator skill
Coordinator sessions are hosted on the console, not on a node.
Integration tests that drive a real coord session live under
`tests/test_coordinator_end_to_end.py` — they spin a console with
an in-memory SQLite backend and a fake upstream node, then drive
the session through its HTTP surface.
For a new coordinator skill:
1. Write the skill prompt as a string and pass it to the
`coord_session` fixture's `skill=` kwarg (see
`tests/test_coordinator_tools.py` for the pattern).
2. Build a small fake cluster: one node + two children via
`mgr.register_children(coord.id, ["child-1", "child-2"])`.
3. Drive the session with seeded tool_call dicts matching the
provider layer's shape. The unit-level tests in
`tests/test_coordinator_tools.py` show the helper (`_tc(name,
args, call_id)`).
4. Assert the skill's decision shape — which tools fire in what
order, what the task_list looks like at the end, which
`_error` reasons appear on the denied-path.
A full end-to-end test isn't required for every skill; a
prepare-step unit test that asserts "given this initial message, the
first tool call is X with Y args" is usually sufficient to catch
persona drift without a real LLM in the loop.
---
## Further reading
- [coordinator-api-tour.md](coordinator-api-tour.md) — the HTTP
surface every coordinator skill indirectly drives.
- [bulk-endpoints.md](bulk-endpoints.md) — the response shape
`spawn_batch` and `close_all_children` use, so your skill can
parse results / denied arrays correctly.
- [governance.md](governance.md) — the broader governance surface
(`/trust`, `/restrict`, `/stop_cascade`, role-based permissions)
that wraps every coord session.
- [settings.md](settings.md) — `coordinator.model_alias` and
`coordinator.reasoning_effort` settings that gate which LLM runs
the coordinator session at all.
+27 -33
View File
@@ -1,34 +1,27 @@
# Consistent Hash Ring — Reference Design
**Status**: Reference (not currently in the hot path)
**Date**: 2026-03-30
**Status**: Reference — alternative routing strategy
## Overview
Live routing uses **rendezvous (HRW) hashing** in
`turnstone/core/rendezvous.py` and `turnstone/console/router.py`. This
document captures a vnode-ring approach as a reference for future
evaluation if the cluster outgrows rendezvous's O(N)-per-route
characteristic.
This document describes a consistent hash ring algorithm evaluated during
the design of the direct HTTP transport routing system. The current
implementation uses weight-proportional bucket assignment with a
donor/recipient rebalancing algorithm (see `direct-http-transport.md`).
The consistent hash ring is documented here as a reference for future
scalability work — if the cluster grows beyond the point where the
weight-proportional approach is sufficient, the ring provides a
proven alternative with stronger stability guarantees.
The FNV-1a-32 hash function specified below is bit-identical to the
hash used by the live rendezvous implementation; cross-language clients
can rely on these test vectors.
## When to consider the ring approach
## When the ring approach becomes interesting
The current weight-proportional seeding + donor/recipient rebalancer works
well when:
- Cluster size is moderate (< 50 nodes)
- Nodes join/leave infrequently
- The rebalancer runs centrally (in the console)
The vnode ring becomes preferable to rendezvous hashing when:
The consistent hash ring becomes advantageous when:
- Cluster size grows large (50+ nodes) and frequent membership changes
cause the donor/recipient algorithm to churn
- Decentralized routing is needed (each node computes the ring locally,
no central console required)
- Cross-language determinism is important (multiple implementations must
agree on the same assignment without sharing state)
- Cluster size grows large (50+ nodes) and the per-route O(N) hash
computation becomes visible against downstream HTTP cost.
- Decentralised routing is needed (each node computes the ring locally,
no central console required).
- A precomputed flat-array lookup is desired so the routing hot path
avoids hashing entirely.
## Algorithm
@@ -133,16 +126,17 @@ class HashRing:
# Precompute all 65536 bucket assignments
```
## Comparison with current approach
## Comparison with rendezvous (HRW) hashing
| Aspect | Weight-proportional (current) | Consistent hash ring |
|--------|------------------------------|---------------------|
| Seeding | Exact weight split, deterministic | Hash-based, ~3% variance |
| Node addition | Donor/recipient moves only excess | Ring moves ~1/N buckets |
| Node removal | Dead buckets → most underloaded | Ring redistributes to clockwise neighbors |
| Cross-node churn | Zero (only donor→recipient) | Zero (ring stability guarantee) |
| Decentralized | No (needs central rebalancer) | Yes (each node computes locally) |
| Complexity | Simple weight arithmetic | Virtual node construction + bisect |
| Aspect | Rendezvous (live) | Consistent hash ring (this doc) |
|--------|-------------------|---------------------------------|
| Per-route cost | O(N) hash computes | O(log V) bisect against precomputed array |
| Seeding | None — pure function | Build vnode array on every membership change |
| Node addition | Pure function moves ~1/N keys | Ring moves ~1/N buckets |
| Node removal | Surviving nodes' keys unchanged | Surviving nodes' buckets unchanged |
| Decentralised | Yes — pure function over services | Yes each node computes locally |
| Persistent state | None | None on the hot path; precomputed array in memory |
| Complexity | ~20 LOC | Virtual-node construction + bisect |
## Test vectors
+9 -4
View File
@@ -19,13 +19,14 @@ package "Entry Points" <<Rectangle>> {
component [cli.py\nturnstone] as cli <<entry>>
component [server.py\nturnstone-server] as server <<entry>>
component [eval.py\nturnstone-eval] as eval <<entry>>
component [chat.py\n(re-exports)] as chat <<entry>>
component [admin.py\nturnstone-admin] as admin <<entry>>
component [bootstrap.py\nturnstone-bootstrap] as bootstrap <<entry>>
}
' Core engine
package "turnstone/core/" <<Rectangle>> {
component [session.py\nChatSession, SessionUI] as session <<core>>
component [providers/\nLLMProvider, OpenAI, Anthropic] as providers <<core>>
component [providers/\nLLMProvider, OpenAI, Anthropic, Google] as providers <<core>>
component [workstream.py\nWorkstreamManager] as workstream <<core>>
component [tools.py\nTool loader] as tools <<core>>
component [memory.py\nPersistence facade] as memory <<core>>
@@ -48,7 +49,8 @@ package "turnstone/core/" <<Rectangle>> {
package "turnstone/channels/" <<Rectangle>> {
component [_routing.py\nChannelRouter] as router <<channel>>
component [discord/bot.py\nDiscordBot] as discordbot <<channel>>
component [gateway.py\nturnstone-channel] as gateway <<channel>>
component [slack/bot.py\nSlackBot (Socket Mode)] as slackbot <<channel>>
component [cli.py\nturnstone-channel] as gateway <<channel>>
}
' Console
@@ -112,7 +114,8 @@ eval --> memory
eval --> config
eval --> tools
chat --> session
admin --> auth
bootstrap --> providers
' Core internal deps
session --> providers
@@ -135,8 +138,10 @@ tools --> schemas
' Channel dependencies
gateway --> discordbot
gateway --> slackbot
gateway --> router
discordbot --> sdkserver : HTTP + SSE
slackbot --> sdkserver : HTTP + SSE
router --> storage : channel_routes
' Console dependencies
+17 -1
View File
@@ -103,6 +103,18 @@ class "AnthropicProvider" as AnthropicProv {
core/providers/_anthropic.py
}
class "GoogleProvider" as GoogleProv {
+ provider_name: str
+ get_capabilities(model) -> ModelCapabilities
--
Extends OpenAIChatCompletionsProvider
for Gemini /v1beta/openai/ endpoint.
Single default ModelCapabilities
(2M context, 65K output).
--
core/providers/_google.py
}
' ModelCapabilities
class "ModelCapabilities" as ModelCaps <<frozen>> {
+ context_window: int
@@ -283,7 +295,7 @@ class "ModelRegistry" as ModelReg {
--
Thread-safe lazy client + provider
creation. Loaded by load_model_registry()
from CLI args + [models.*] config.
from DB + [models.*] config + CLI args.
--
core/model_registry.py
}
@@ -294,6 +306,9 @@ class "ModelConfig" as ModelCfg <<frozen>> {
+ base_url: str
+ model: str
+ context_window: int
+ temperature: float | None
+ max_tokens: int | None
+ reasoning_effort: str | None
}
' Circuit breaker state
@@ -360,6 +375,7 @@ SessionUI <|.. NullUI
LLMProvider <|.. OpenAIProv
LLMProvider <|.. AnthropicProv
OpenAIProv <|-- GoogleProv
ChatSession --> SessionUI : uses
ChatSession --> LLMProvider : delegates LLM calls
+1 -1
View File
@@ -23,7 +23,7 @@ interface "StorageBackend" as SB <<protocol>> {
+resolve_workstream(alias_or_id) → str | None
+delete_workstream(ws_id) → bool
+prune_workstreams(retention_days) → (int, int)
+list_workstreams(node_id, limit) → list
+list_workstreams(node_id, limit, *, parent_ws_id, kind, user_id) → list
+save_workstream_config(ws_id, config)
+load_workstream_config(ws_id) → dict
+kv_get(key) → str | None
+26 -5
View File
@@ -20,11 +20,14 @@ class "Discord" as Discord <<platform>> {
asyncio event loop
}
class "Slack (future)" as Slack <<platform>> {
Socket Mode / Events API
class "Slack" as Slack <<platform>> {
Socket Mode WebSocket
Block Kit messages
Slash command (default /turnstone)
DM + channel events
--
Planned integration
slack-bolt (Python)
asyncio event loop
}
class "Teams (future)" as Teams <<platform>> {
@@ -38,7 +41,7 @@ class "Teams (future)" as Teams <<platform>> {
class "turnstone-channel" as ChannelService <<service>> {
entry point: turnstone-channel
--
One process per platform
One process — hosts one or more adapters
asyncio event loop
Structured logging (structlog)
--log-level, --log-format
@@ -47,6 +50,19 @@ class "turnstone-channel" as ChannelService <<service>> {
GET /health
}
class "SlackBot" as SlackBot <<service>> {
+on_message(event)
+on_action(action) (Block Kit buttons)
+send(channel_id, content)
+send_notification(channel_id, content, ws_id)
+run(bot_token, app_token)
--
slack-bolt AsyncApp
Socket Mode client
Per-user channel sessions via slash command
DM routing without slash command
}
class "DiscordBot" as Bot <<service>> {
+on_message(msg)
+on_interaction(interaction)
@@ -138,10 +154,15 @@ Server --> Bot : SSE event stream
Bot --> Discord : reply / embed\nbutton callback
Slack .[hidden]. Discord
Slack --> SlackBot : socket-mode\nevents
SlackBot --> Router : on_message / on_action
SlackBot --> Server : POST /v1/api/send\nGET /v1/api/events?ws_id=
SlackBot --> Slack : post / update\nBlock Kit button callbacks
Teams .[hidden]. Slack
ChannelService --> Bot : creates + runs
ChannelService --> SlackBot : creates + runs
ChannelService --> Router : creates
ChannelService --> SVC : register / heartbeat /\nderegister
+28 -1
View File
@@ -152,10 +152,34 @@ MCPMgr -> MCPSrv : prompts/get
MCPSrv --> MCPMgr : GetPromptResult
MCPMgr --> Session : messages [{role, content}]
== Resilience: Circuit Breaker & Stream Safety ==
note over MCPMgr
**Per-server circuit breaker**
CLOSED --(3 failures)--> OPEN
OPEN --(cooldown expires)--> half-open probe
Probe success --> CLOSED (trip_count decays by 1)
Probe failure --> OPEN (cooldown doubles, max 5 min)
McpError (protocol) does NOT trip breaker.
BrokenPipeError / EOFError evicts dead session.
All sync methods cancel orphaned futures on timeout.
Transport streams pre-closed before stack teardown
to avoid anyio cancel-scope CPU busy-loop (SDK #2147).
end note
Session -> MCPMgr : call_tool_sync()
MCPMgr -> MCPMgr : _cb_gate(server)\n[reject if circuit open]
MCPMgr -> MCPMgr : _cb_auto_reconnect()\n[if session gone + cooldown expired]
MCPMgr -> MCPSrv : tools/call
MCPSrv --> MCPMgr : result or error
MCPMgr -> MCPMgr : _cb_record_success()\nor _cb_record_failure()
== Three-Tier Refresh ==
group Push Notifications
group Push Notifications (debounced 5s per server)
MCPSrv -> MCPMgr : ToolListChangedNotification
MCPMgr -> MCPMgr : debounce check\n(skip if < 5s since last)
MCPMgr -> MCPMgr : _refresh_server_tools()
MCPSrv -> MCPMgr : ResourceListChangedNotification
@@ -172,6 +196,9 @@ group Periodic Polling (default 4h)
Only polls capabilities
without push support.
Staggered per-server.
Disconnected servers get
reconnect attempts with
exponential backoff (60s-1h).
end note
end
+1 -1
View File
@@ -211,7 +211,7 @@ note over Session, Judge
**Storage:**
intent_verdicts table (migration 012), output_assessments table
(migration 022). Both queryable via admin API endpoints
(requires admin.judge permission). Skills store scan_status,
(requires admin.judge permission). Skills store risk_level,
scan_report, scan_version for install-time risk assessment.
end note
@@ -0,0 +1,89 @@
@startuml
title Turnstone - coordinator wait_for_workstream lifecycle
skinparam sequenceArrowThickness 1.5
skinparam noteBackgroundColor #FDF6E3
participant "Coordinator\nLLM" as LLM
participant "ChatSession\n(worker thread)" as CS
participant "CoordinatorClient" as CC
participant "SessionUI\n(SSE fanout)" as UI
participant "Console routing\nproxy" as RP
database "Storage\n(workstreams row)" as DB
participant "Child\nnode" as NODE
== Spawn ==
LLM -> CS : tool_call spawn_workstream(...)
activate CS
CS -> CC : spawn(initial_message=...,\nparent_ws_id=coord, user_id=...)
CC -> RP : POST /v1/api/route/workstreams/new
RP -> NODE : dispatch (rendezvous)
NODE -> DB : insert workstreams row\nstate='running'
RP --> CC : {ws_id, node_id, name, status: 200}
CC --> CS : {ws_id, ...}
CS -> UI : on_tool_result\n("spawn_workstream", ws_id)
deactivate CS
note right of LLM
Model now knows the child ws_id.
It can inspect / send / wait, and
the parent registry tracks it.
end note
== Wait (blocking) ==
LLM -> CS : tool_call wait_for_workstream\n(ws_ids=[child], mode="any", timeout=60)
activate CS
CS -> CS : _prepare_wait_for_workstream\n(validate ws_ids, timeout, mode)
CS -> UI : emit wait_started\n{call_id, ws_ids, mode, timeout}
CS -> CC : wait_for_workstream(ws_ids, timeout,\nmode, progress_callback)
activate CC
loop every 500ms up to timeout
CC -> DB : read workstreams row(s)
DB --> CC : {state, updated, tokens, ...}
alt state in {idle, error, closed, deleted}
note over CC
real-terminal state ->
completion condition met
end note
else still running / thinking / attention
CC -> CS : progress_callback(snap)\n(diff-on-change or 5s heartbeat)
CS -> UI : emit wait_progress\n{call_id, elapsed, results?}
end
end
CC --> CS : {complete, elapsed,\nresults: {ws_id: snap}}
deactivate CC
CS -> UI : emit wait_ended\n{call_id, complete, elapsed, results}
CS -> UI : on_tool_result\n("wait_for_workstream",\n"complete after Ns (R/N resolved)")
CS --> LLM : tool_result (full results dict)
deactivate CS
note left of UI
Sidebar "waiting on N children" indicator
keys on call_id - started / progress / ended
scope to a single wait invocation so
nested waits render independent badges.
end note
== After wait: inspect + close ==
LLM -> CS : tool_call inspect_workstream(ws_id=child)
CS -> CC : inspect(ws_id)
CC -> DB : read row + tail
CC --> CS : {state, messages, tokens, ...}
CS --> LLM : tool_result (serialised)
LLM -> CS : tool_call close_workstream\n(ws_id=child, reason="...")
CS -> CC : close_workstream(ws_id, reason)
CC -> RP : POST /v1/api/route/workstreams/close
RP -> NODE : dispatch
NODE -> DB : state='closed',\nclose_reason='...'
RP --> CC : {status: 200}
CC --> CS : {closed: true, status: 200, reason: ...}
CS --> LLM : tool_result
@enduml
+2 -2
View File
@@ -1,3 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:75da80e6bd205e45b9fe48aa2f87197110c908f44e1d98c52eee69f956274711
size 400402
oid sha256:a3b5c59403a6febd81667fc8fd2a7d22bc59da6130eba0dea5449c42668d0ede
size 387044
+2 -2
View File
@@ -1,3 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:6471e611beebf647f3a191eb16588571a404cc52a43067883a2b6f06dd936376
size 594676
oid sha256:474b900448ec04d1117b48a2b55614524721b2f04ac4bda66170bd0a06aae0f2
size 624573
@@ -1,3 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:8a5957b71822656098cbe779ca619b7e56e9b11b0f1c4eac4fc880ed62b71a2a
size 358670
oid sha256:ae4f79fb22600106f8cb0af4ba5586bb26ea5d57e27ef382fdc59b6549fdbd21
size 415473
+2 -2
View File
@@ -1,3 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:a6b7769aa7e732ffbeb1eb7f5b65273a135fb3a78d9802ec36d3b92801c34f6b
size 427745
oid sha256:7623df33be9baf7647ca1c2450640df57e1cd73e8be1f8168aae16e546ad683c
size 459941
@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:aa12d81dc578f7e65bf4df3152b3de1736289c422f83d0b0cd32107726722357
size 172028
+18 -4
View File
@@ -22,7 +22,7 @@ Console dashboard: http://localhost:8090
|---------|------|---------|-------------|
| `server` | 8080 | default | Web UI + chat workstreams + LLM |
| `console` | 8090 | default | Cluster dashboard |
| `channel` | — | production | Channel gateway (Discord, Slack, etc.) |
| `channel` | — | production | Channel gateway (Discord and/or Slack adapters) |
| `server-1``server-10` | — | cluster | 10-node server fleet (PostgreSQL required) |
## Profiles
@@ -83,11 +83,15 @@ Auth is always enabled. `TURNSTONE_JWT_SECRET` is required.
| Variable | Default | Description |
|----------|---------|-------------|
| `TURNSTONE_DB_BACKEND` | `sqlite` | Storage backend: `sqlite` or `postgresql` |
| `TURNSTONE_DB_URL` | — | Database URL (e.g. `postgresql://user:pass@db:5432/turnstone`). For SQLite, defaults to `/data/.turnstone.db` |
| `TURNSTONE_DB_URL` | — | Database URL (e.g. `postgresql+psycopg://user:pass@postgres:5432/turnstone`). For SQLite, defaults to `/data/.turnstone.db` |
| `TURNSTONE_DB_POOL_SIZE` | `2` | PostgreSQL connection pool size per process (default: 2 base + 3 overflow = 5 max) |
| `POSTGRES_USER` | `turnstone` | PostgreSQL container username (used in default `TURNSTONE_DB_URL` for cluster/channel) |
| `POSTGRES_PASSWORD` | — | PostgreSQL container password (required for production and cluster profiles) |
The database stores workstream history, user accounts, and API tokens. When using JWT auth, a database backend is required for user storage.
> **Upgrading from <1.3.0a4:** Earlier versions used `DB_BACKEND` and `DATABASE_URL` in `.env`, which `compose.yaml` mapped to the `TURNSTONE_`-prefixed names internally. These short aliases have been removed. Rename `DB_BACKEND``TURNSTONE_DB_BACKEND` and `DATABASE_URL``TURNSTONE_DB_URL` in your `.env` file.
> **Large clusters:** Each turnstone process maintains a small connection pool (5 max). At hundreds of nodes this adds up — use [PgBouncer](pgbouncer.md) in transaction pooling mode between turnstone and PostgreSQL.
> **First-time setup:** After deploying with auth enabled, create an initial admin user by running `turnstone-admin create-user` inside the container:
@@ -104,8 +108,16 @@ The database stores workstream history, user accounts, and API tokens. When usin
|----------|---------|-------------|
| `TURNSTONE_DISCORD_TOKEN` | — | Discord bot token (required to enable Discord adapter) |
| `TURNSTONE_DISCORD_GUILD` | `0` | Restrict to a single Discord guild (0 = all guilds) |
| `TURNSTONE_SLACK_TOKEN` | — | Slack Bot User OAuth token `xoxb-…` (required to enable Slack adapter) |
| `TURNSTONE_SLACK_APP_TOKEN` | — | Slack App-Level token `xapp-…` (required with `TURNSTONE_SLACK_TOKEN`) |
| `TURNSTONE_SLACK_CHANNELS` | — | Comma-separated Slack channel IDs to allow (empty = all) |
| `TURNSTONE_SLACK_SLASH_COMMAND` | `/turnstone` | Slash command registered in the Slack app |
The channel service runs in the `production` profile. When `TURNSTONE_DISCORD_TOKEN` is set, the Discord adapter connects to the Discord Gateway and routes messages to the server via HTTP. See [Channel Integrations](channels.md) for full setup instructions including Discord application creation and user account linking.
The channel service runs in the `production` profile. When
`TURNSTONE_DISCORD_TOKEN` or the Slack pair is set the gateway starts the
corresponding adapter; both can run in one process. See
[Channel Integrations](channels.md) for platform app setup and user
account linking.
## Scaling
@@ -137,7 +149,9 @@ docker compose build
docker compose build --no-cache
```
All entry points are installed in a single image: `turnstone-server`, `turnstone-console`, `turnstone-channel`, `turnstone-admin`, `turnstone-eval`.
All entry points are installed in a single image: `turnstone`,
`turnstone-server`, `turnstone-console`, `turnstone-channel`,
`turnstone-admin`, `turnstone-eval`, and `turnstone-bootstrap`.
## Cleanup
+12 -6
View File
@@ -62,14 +62,14 @@ etc.) since workstream templates were merged into the skills system in v0.8.0.
- **Default skills**: All `is_default=true` skills auto-apply to new
workstreams, concatenated in alphabetical order by name. Use name prefixes
(e.g. `01-safety`, `02-style`) to control ordering.
- **Explicit selection**: `--template <name>` CLI flag, `template` field on
- **Explicit selection**: `--skill <name>` CLI flag, `skill` field on
`POST /v1/api/workstreams/new`, console creation modal dropdown, scheduled task
config, and channel adapter config. An explicit skill *replaces* defaults.
- **Variables**: Three built-in placeholders resolved at load time:
`{{model}}` (active model name), `{{ws_id}}` (workstream ID),
`{{node_id}}` (server node ID). Unrecognized placeholders are kept as-is.
- **Runtime switching**: `/template <name>` to switch, `/template clear` to revert
to defaults, `/template` to show current. Persisted across resume.
- **Runtime switching**: `/skill <name>` to switch, `/skill clear` to revert
to defaults, `/skill` to show current. Persisted across resume.
- **Model-driven loading**: The `skill` built-in tool lets the model
discover and activate skills mid-conversation. `search` action finds skills
by query (auto-approved); `load` action activates by name (requires user
@@ -91,7 +91,7 @@ etc.) since workstream templates were merged into the skills system in v0.8.0.
time. The scanner evaluates four risk axes: content risk (command execution,
data exfiltration), supply chain risk (pipe-to-shell, transitive installs),
vulnerability risk (prompt injection, insecure credentials), and declared
capability risk (from `allowed-tools` in SKILL.md). Results populate the `scan_status`
capability risk (from `allowed-tools` in SKILL.md). Results populate the `risk_level`
(safe/low/medium/high/critical) and `scan_report` (JSON breakdown) columns.
These fields are system-managed and cannot be overwritten via the admin API.
- **Discovery**: External skills can be discovered and installed from registries:
@@ -186,15 +186,21 @@ Full OpenAPI spec at `/openapi.json` and Swagger UI at `/docs`.
## Admin Console UI
6 new tabs added to the admin panel (11 total):
Governance-related tabs within the 18-tab admin panel:
- **Roles** — CRUD roles, permission checkbox grid, user role assignment modal
- **Policies** — CRUD tool policies with colored action badges (green/red/amber)
- **Skills**CRUD skills with wide modal, textarea editor
- **Prompts**Prompt-policy editor (heuristics for admin guardrails)
- **Skills** — CRUD skills with wide modal, textarea editor; Discover pill for
installing from skills.sh / GitHub; per-row scan badges (safe/low/med/high/critical)
- **Judge** — Intent validation configuration and verdict history
- **Usage** — Summary readouts + CSS bar chart, time range + group-by selectors
- **Audit** — Filterable log with relative timestamps, load-more pagination
Tabs are permission-gated: hidden if the user lacks the required permission.
See [docs/console.md](console.md) for the full tab list and
[docs/settings.md](settings.md) for the Settings tab that edits live
ConfigStore values.
## SDK
+16 -4
View File
@@ -41,6 +41,7 @@ confidence_threshold = 0.7 # reserved for v2 smart approvals (not used in v1)
max_context_ratio = 0.5 # max % of judge context window for history
timeout = 60.0 # seconds (generous for local models)
read_only_tools = true # judge can use read_file/list_directory
cancel_on_approval = false # stop judging remaining tool calls once user decides
```
All fields are optional. The judge is enabled by default; use `enabled = false`
@@ -72,6 +73,17 @@ CLI flags override `config.toml` values.
- **Cross-provider**: When both `model` and `provider` are set, the judge
creates its own LLM client. You can optionally specify `base_url` and
`api_key` for non-default endpoints.
- **Google models**: The judge supports `google` as a provider. Note that
read-only tools are disabled for Google models (the Gemini API requires
`thought_signature` in tool call round-trips which the judge's normalized
format does not preserve).
The judge creates a fresh HTTP client for each evaluation run and closes it
when done, avoiding stale connection issues across runs.
If the LLM judge fails or returns no verdict, a fallback verdict with tier
`llm_fallback` is delivered via the callback, ensuring the UI always receives
a result.
---
@@ -303,7 +315,7 @@ four independent risk axes:
`Bash(*)` (unrestricted shell) is high risk. `Bash(git:*)` is low.
Read-only tools are safe.
Results are stored in `scan_status` (tier: safe/low/medium/high/critical) and
Results are stored in `risk_level` (tier: safe/low/medium/high/critical) and
`scan_report` (JSON breakdown) on the `prompt_templates` table. These fields are
system-managed and not editable via the admin API.
@@ -388,11 +400,11 @@ level, annotations, output length, redaction status).
### Session-level skill scan warning
When a skill with `scan_status` of `high` or `critical` is loaded into a
When a skill with `risk_level` of `high` or `critical` is loaded into a
session, a warning is emitted via `on_info`:
```
⚠ Skill 'my-skill' has scan status: high.
⚠ Skill 'my-skill' has risk level: high.
Review scan report in admin panel before enabling in production.
```
@@ -409,7 +421,7 @@ All three evaluation systems persist their assessments for future calibration:
|-------|--------|-------------|
| `intent_verdicts` | Intent judge (heuristic + LLM) | `func_name`, `risk_level`, `confidence`, `user_decision` |
| `output_assessments` | Output guard | `func_name`, `risk_level`, `flags`, `redacted` |
| `prompt_templates` | Skill scanner | `scan_status`, `scan_report`, `scan_version` |
| `prompt_templates` | Skill scanner | `risk_level`, `scan_report`, `scan_version` |
Run v1 with all tools requiring manual approval to build a local dataset.
In v2, calibration tooling will analyze this data to:
+1 -1
View File
@@ -146,7 +146,7 @@ with TurnstoneConsole("http://localhost:8081", token="...") as client:
### TypeScript
```typescript
import { TurnstoneConsole } from "@anthropic/turnstone-sdk";
import { TurnstoneConsole } from "@turnstone/sdk";
const client = new TurnstoneConsole({
baseUrl: "http://localhost:8081",
+16 -14
View File
@@ -40,18 +40,20 @@ Add PgBouncer between turnstone services and PostgreSQL:
```yaml
services:
pgbouncer:
image: bitnami/pgbouncer:latest
image: edoburu/pgbouncer:latest
environment:
POSTGRESQL_HOST: postgres
POSTGRESQL_PORT: "5432"
POSTGRESQL_DATABASE: turnstone
POSTGRESQL_USERNAME: ${POSTGRES_USER:-turnstone}
POSTGRESQL_PASSWORD: ${POSTGRES_PASSWORD:?}
PGBOUNCER_POOL_MODE: transaction
PGBOUNCER_DEFAULT_POOL_SIZE: "40"
PGBOUNCER_MAX_CLIENT_CONN: "5000"
PGBOUNCER_MAX_DB_CONNECTIONS: "80"
PGBOUNCER_SERVER_IDLE_TIMEOUT: "300"
DB_HOST: postgres
DB_PORT: "5432"
DB_NAME: ${POSTGRES_DB:-turnstone}
DB_USER: ${POSTGRES_USER:-turnstone}
DB_PASSWORD: ${POSTGRES_PASSWORD:?}
LISTEN_PORT: "6432"
AUTH_TYPE: ${POSTGRES_AUTH_TYPE:-scram-sha-256}
POOL_MODE: transaction
DEFAULT_POOL_SIZE: "40"
MAX_CLIENT_CONN: "5000"
MAX_DB_CONNECTIONS: "80"
SERVER_IDLE_TIMEOUT: "300"
ports:
- "6432:6432"
networks:
@@ -67,7 +69,7 @@ services:
```
Then point turnstone services at PgBouncer instead of PostgreSQL
directly by changing the `DATABASE_URL` (or `TURNSTONE_DB_URL`):
directly by changing `TURNSTONE_DB_URL`:
```bash
# Before (direct)
@@ -82,7 +84,7 @@ TURNSTONE_DB_URL=postgresql://turnstone:secret@pgbouncer:6432/turnstone
## Helm / Kubernetes
Add a PgBouncer deployment or use a Helm chart like
[bitnami/pgbouncer](https://github.com/bitnami/charts/tree/main/bitnami/pgbouncer).
[edoburu/pgbouncer](https://github.com/edoburu/docker-pgbouncer/tree/master/examples/kubernetes).
In `values.yaml`, point the database at PgBouncer:
@@ -106,7 +108,7 @@ pgbouncer:
maxClientConn: 5000
maxDbConnections: 80
```
:
---
## Configuration reference
+24 -15
View File
@@ -1,17 +1,24 @@
# Release Process
Turnstone uses two parallel release tracks published from a single PyPI package.
Turnstone ships several parallel release tracks from a single PyPI package.
## Release Tracks
| Track | Versions | Branch | Docker tags | PyPI install |
|-------|----------|--------|-------------|--------------|
| **Stable** | `1.0.0`, `1.0.1` | `stable/1.0` | `:1.0.1`, `:1.0`, `:stable`, `:latest` | `pip install turnstone` |
| **Experimental** | `1.1.0a1`, `1.1.0a2` | `main` | `:1.1.0a1`, `:experimental` | `pip install turnstone --pre` |
| **Legacy 1.0** | `1.0.x` | `stable/1.0` | `:1.0.x`, `:1.0` | `pip install 'turnstone==1.0.*'` |
| **Stable 1.3** | `1.3.x` | `stable/1.3` | `:1.3.x`, `:1.3` | `pip install 'turnstone==1.3.*'` |
| **Stable 1.4** | `1.4.x` | `stable/1.4` | `:1.4.x`, `:1.4`, `:stable`, `:latest` | `pip install turnstone` |
| **Experimental** | `1.5.0aN` | `main` | `:1.5.0aN`, `:experimental` | `pip install turnstone --pre` |
- **Stable** receives bugfixes only. Production-grade.
- **Experimental** receives new features. May be rough around the edges.
- When experimental matures, it is promoted to stable. The previous stable branch stops receiving patches.
- **Stable** tracks receive bugfixes only. The most-recent stable minor
owns the `:stable` / `:latest` Docker tags and the default PyPI
install.
- **Experimental** (always on `main`) receives new features. May be
rough around the edges.
- When experimental matures, it is promoted to a new stable minor via
a `stable/X.Y` branch; older stable branches continue to receive
security fixes until explicitly retired.
## Version Scheme
@@ -26,17 +33,17 @@ Turnstone uses two parallel release tracks published from a single PyPI package.
## Releasing an Experimental Version (from main)
```bash
scripts/release.sh 1.1.0a2 --push
scripts/release.sh 1.5.0a2 --push
```
This bumps `pyproject.toml` + `turnstone/__init__.py`, regenerates `uv.lock`, commits, tags `v1.1.0a2`, and pushes. CI runs, then publish + Docker workflows fire automatically.
This bumps `pyproject.toml` + `turnstone/__init__.py`, regenerates `uv.lock`, commits, tags `v1.5.0a2`, and pushes. CI runs, then publish + Docker workflows fire automatically.
## Releasing a Stable Patch (from stable/X.Y)
```bash
git checkout stable/1.0
git checkout stable/1.4
git cherry-pick <commit-hash> # bugfix from main
scripts/release.sh 1.0.2 --push
scripts/release.sh 1.4.1 --push
```
## Promoting Experimental to Stable
@@ -45,17 +52,19 @@ When `main` is ready for a stable release:
```bash
# 1. Tag the stable release on main
scripts/release.sh 1.1.0 --push
scripts/release.sh 1.5.0 --push
# 2. Create the stable maintenance branch from that tag
git branch stable/1.1 v1.1.0
git push origin stable/1.1
git branch stable/1.5 v1.5.0
git push origin stable/1.5
# 3. Start the next experimental cycle on main
scripts/release.sh 1.2.0a1 --push
scripts/release.sh 1.6.0a1 --push
```
The previous `stable/1.0` branch stops receiving patches at this point.
The previous stable branch (`stable/1.4`) continues to receive
security-only patches; older tracks (`stable/1.0`, `stable/1.3`) are
retired when they fall out of support.
## CI/CD Pipeline
+36 -2
View File
@@ -69,8 +69,12 @@ Both `TurnstoneServer` (sync) and `AsyncTurnstoneServer` (async) expose:
|----------|--------|---------|
| **Workstreams** | `list_workstreams()` | `ListWorkstreamsResponse` |
| | `dashboard()` | `DashboardResponse` |
| | `create_workstream(*, name, model, auto_approve, skill)` | `CreateWorkstreamResponse` |
| | `create_workstream(*, name, model, auto_approve, skill, initial_message, attachments)` | `CreateWorkstreamResponse` |
| | `close_workstream(ws_id)` | `StatusResponse` |
| **Attachments** | `upload_attachment(ws_id, filename, data, *, mime_type=...)` | `UploadAttachmentResponse` |
| | `list_attachments(ws_id)` | `ListAttachmentsResponse` |
| | `get_attachment_content(ws_id, attachment_id)` | `bytes` |
| | `delete_attachment(ws_id, attachment_id)` | `StatusResponse` |
| **Chat** | `send(message, ws_id)` | `SendResponse` |
| | `approve(*, ws_id, approved, feedback, always)` | `StatusResponse` |
| | `plan_feedback(*, ws_id, feedback)` | `StatusResponse` |
@@ -171,6 +175,36 @@ result.ok # True if no errors and not timed out
result.timed_out # True if timeout expired
```
### Attachments
Upload files to a workstream and attach them to the next user turn:
```python
# Upload separately, then send a message — attachments auto-attach
with open("screenshot.png", "rb") as f:
att = client.upload_attachment(ws.ws_id, "screenshot.png",
f.read(),
mime_type="image/png")
client.send("What's wrong in this screenshot?", ws.ws_id)
# Or attach at workstream-creation time (multipart upload)
from turnstone.sdk import AttachmentUpload
with open("notes.txt", "rb") as f:
ws = client.create_workstream(
name="triage",
initial_message="Summarize the notes",
attachments=[AttachmentUpload(data=f.read(),
filename="notes.txt",
mime_type="text/plain")],
)
```
Limits: images ≤ 4 MiB (png/jpeg/gif/webp), text ≤ 512 KiB (UTF-8),
10 pending per (workstream, user). The SDK auto-generates `ws_id` on the
client so cluster-routed callers bind attachments to the owning node
before the request lands.
### Error Handling
Non-2xx responses raise `TurnstoneAPIError`:
@@ -284,7 +318,7 @@ turnstone/sdk/ Python SDK (sub-package)
_base.py Shared httpx async client, auth, error handling
_sync.py Background event loop for sync wrappers
_types.py TurnResult + TurnstoneAPIError
events.py 27 SSE event dataclasses with type registry
events.py 38 SSE event dataclasses with type registry
server.py AsyncTurnstoneServer + TurnstoneServer
console.py AsyncTurnstoneConsole + TurnstoneConsole
+6 -4
View File
@@ -1,8 +1,10 @@
# Security and Authentication
Turnstone uses a layered authentication system with three token types,
hierarchical scopes, and a split architecture where the console manages
credentials while individual server nodes validate JWTs locally.
Turnstone uses a layered authentication system with two token types
(database-backed API tokens + HMAC-SHA256 JWTs), hierarchical scopes,
and a split architecture where the console manages credentials while
individual server nodes validate JWTs locally. Inter-service traffic
uses short-lived service JWTs minted by `ServiceTokenManager`.
---
@@ -37,7 +39,7 @@ Claims:
|-------|-------------|
| `sub` | User ID |
| `scopes` | Comma-separated scope list (`read,write,approve`) |
| `src` | Token source (`password`, `api_token`, `config`, `oidc`) |
| `src` | Token source (`password`, `database`, `oidc`, or a service origin like `console`, `cli`, or `channel`) |
| `iss` | Issuer — always `turnstone` |
| `aud` | Audience — `turnstone-server` or `turnstone-console` |
| `iat` | Issued-at timestamp |
+46 -4
View File
@@ -36,6 +36,47 @@ users to the admin Settings API.
---
## Per-Model Sampling Overrides
The global `model.temperature`, `model.max_tokens`, and `model.reasoning_effort`
settings serve as cluster-wide defaults. Individual models can override these
via per-model settings in the `model_definitions` table (admin Models tab).
Resolution order for sampling parameters:
| Priority | Source |
|----------|--------|
| 1 (highest) | Per-model override (set in Models tab) |
| 2 | Global default (set in Settings tab) |
| 3 | Registry default (code) |
When a per-model override is `NULL` (empty in the UI), the global default is
used. Switching models via `/model <alias>` re-resolves sampling parameters
from the new model's overrides or global defaults.
**Removed settings:** `model.name` and `model.context_window` have been removed
from ConfigStore. Model names and context windows are now configured per-model
in the Models tab. A startup warning is logged if these keys appear in
`config.toml`.
### Plan / task agent overrides
`plan_agent` and `task_agent` sub-sessions resolve independently from the
conversation model so operators can pick a cheaper/faster model for
autonomous loops:
| Setting | Purpose |
|---------|---------|
| `model.plan_alias` | Alias used for `plan_agent` sub-sessions. Falls back to `[model].plan_model` in config.toml, then `[model].agent_model`, then the session's active model. |
| `model.task_alias` | Alias used for `task_agent` sub-sessions. Same fallback chain as `plan_alias`. |
| `model.plan_effort` | Reasoning effort for `plan_agent` (`none` / `minimal` / `low` / `medium` / `high` / `xhigh` / `max`). Defaults to `high`. |
| `model.task_effort` | Reasoning effort for `task_agent`. Empty string means "inherit from the session". |
All four are live-editable from the Settings tab and take effect on the
next sub-agent invocation — no restart required.
---
## Bootstrap vs ConfigStore
**Bootstrap settings** are required before storage is available (database
@@ -49,12 +90,12 @@ connection, Redis, auth secrets, server bind address). These stay in
| Auth | `[auth]` | config.toml / env |
| Console bind | `[console]` | config.toml / env |
**ConfigStore settings** (48 settings) are loaded from the database after
storage initialization:
**ConfigStore settings** are loaded from the database after storage
initialization:
| Section | Settings |
|---------|----------|
| `model` | name, temperature, max_tokens, reasoning_effort, context_window |
| `model` | default_alias, temperature, max_tokens, reasoning_effort, plan_alias, task_alias, plan_effort, task_effort |
| `session` | instructions, retention_days, compact_max_tokens, auto_compact_pct |
| `tools` | timeout, truncation, agent_max_turns, skip_permissions, search, search_threshold, search_max_results |
| `server` | workstream_idle_timeout, max_workstreams |
@@ -62,7 +103,8 @@ storage initialization:
| `mcp` | config_path, refresh_interval, registry_url |
| `ratelimit` | enabled, requests_per_second, burst, trusted_proxies |
| `health` | backend_probe_interval, backend_probe_timeout, circuit_breaker_threshold, circuit_breaker_cooldown |
| `judge` | enabled, model, provider, base_url, api_key, confidence_threshold, max_context_ratio, timeout, read_only_tools, output_guard, redact_secrets |
| `judge` | enabled, model, provider, base_url, api_key, confidence_threshold, max_context_ratio, timeout, read_only_tools, output_guard, redact_secrets, cancel_on_approval |
| `interface` | close_tab_action, theme |
| `skills` | discovery_url |
| `memory` | relevance_k, fetch_limit, max_content, nudge_cooldown, nudges |
+11 -8
View File
@@ -169,8 +169,8 @@ Every tool defines a `primary_key`. The mapping is:
| `man` | `page` |
| `web_fetch` | `url` |
| `web_search` | `query` |
| `task` | `prompt` |
| `plan` | `prompt` |
| `task_agent` | `prompt` |
| `plan_agent` | `goal` |
| `memory` | `name` |
| `recall` | `query` |
| `notify` | `message` |
@@ -357,7 +357,10 @@ Search the web using a text query.
## Agent
### task
Tool names use the `_agent` suffix — bare `plan` / `task` collide with
chat-template channel names on some local models.
### task_agent
Delegate a general-purpose task to an autonomous sub-agent.
@@ -371,7 +374,7 @@ Delegate a general-purpose task to an autonomous sub-agent.
---
### plan
### plan_agent
Plan before implementing -- an autonomous agent explores the codebase and writes a structured plan.
@@ -543,11 +546,11 @@ pre-configure skills at workstream creation.
- `load` — Activate a skill by name. Calls `set_skill()` which handles content
rendering with `{{model}}`/`{{ws_id}}`/`{{node_id}}` variables, system message
reinitialization, and config persistence. Returns the skill name, description,
and security scan tier. Warns on high/critical scan status.
and security risk level. Warns on high/critical risk level.
- `search` — Find available skills by query. Uses BM25 relevance ranking over
name, description, tags, and category (same `BM25Index` used by memory
relevance and tool search). Returns up to 10 results with name, description,
category, scan status, and activation type.
category, risk level, and activation type.
- **Auto-approve**: `load` requires approval (changes session behavior); `search`
is auto-approved (read-only).
@@ -568,8 +571,8 @@ pre-configure skills at workstream creation.
| `man` | Info | Yes | Yes | Yes | `page` |
| `web_fetch` | Info | No | Yes | Yes | `url` |
| `web_search` | Info | No | Yes | Yes | `query` |
| `task` | Agent | No | No | No | `prompt` |
| `plan` | Agent | No | No | No | `prompt` |
| `task_agent` | Agent | No | No | No | `prompt` |
| `plan_agent` | Agent | No | No | No | `goal` |
| `memory` | Memory | Yes | No | No | `name` |
| `recall` | Memory | Yes | No | No | `query` |
| `notify` | Notify | Yes | Yes | Yes | `message` |
+38 -1
View File
@@ -2,11 +2,48 @@
An MCP server that exposes tools for executing commands across a Turnstone cluster. Serves as a reference implementation for both MCP server patterns and Turnstone SDK usage.
> [!NOTE]
> **Superseded by the built-in coordinator workstream in Turnstone 1.5.**
>
> This MCP side-car is the pre-1.5 pattern for cluster-wide orchestration.
> Turnstone 1.5 promotes coordinator behaviour to a first-class workstream
> kind hosted inside `turnstone-console` — no external MCP server to
> install or operate, proper per-user audit attribution, and a dedicated
> UI at `/coordinator/{ws_id}`.
>
> The extension continues to work for 1.4-and-earlier clusters. On 1.5+:
> grant the `admin.coordinator` permission, set `coordinator.model_alias`
> in the admin Settings tab, and create sessions via the dashboard's
> "new coordinator" button or `POST /v1/api/coordinator/new`. Full
> removal of this example (including docker / compose references) is
> planned once 1.5 is confirmed in production.
>
> | Concern | Built-in coordinator (1.5+) | This MCP extension (1.4-and-earlier) |
> |---|---|---|
> | Install | None — shipped in-tree | `pip install -e examples/mcp-cluster-ops` + MCP client config |
> | Auth | Real creator's `user_id` + `admin.coordinator` permission | Shared service token |
> | Audit | `coordinator.create` / `close` / `cancel` events on the console; `src="coordinator"` preserved on upstream hops | Service identity only |
> | UI | `/coordinator/{ws_id}` one-pane HTML | No UI — model-only |
> | Tool approvals | Inline approval bar in the coordinator pane | MCP approval flow |
> | Configuration | `coordinator.model_alias`, `coordinator.max_active`, `coordinator.reasoning_effort`, `coordinator.session_jwt_ttl_seconds` | MCP server config file |
>
> Minimal 1.5 migration:
>
> ```bash
> curl -X POST https://console.example/v1/api/coordinator/new \
> -H "Authorization: Bearer $TOKEN" \
> -H "Content-Type: application/json" \
> -d '{"name":"planner","initial_message":"Spawn a worker to check the build"}'
> ```
>
> The response carries `ws_id`; open
> `https://console.example/coordinator/{ws_id}` to watch the session.
## How it works
This server uses the Turnstone console SDK (`TurnstoneConsole`) for node discovery and routing, and `TurnstoneServer` for per-node SSE streaming. The dispatch flow for each command is:
1. **Route**`TurnstoneConsole.route_create_workstream(target_node=..., auto_approve=True)` creates a workstream pinned to the target node via the console's hash-ring routing proxy, returning `ws_id` and `node_url`.
1. **Route**`TurnstoneConsole.route_create_workstream(target_node=..., auto_approve=True)` creates a workstream pinned to the target node via the console's rendezvous routing proxy, returning `ws_id` and `node_url`.
2. **Execute**`TurnstoneServer(node_url, token=...)` connects directly to the node's SSE stream using the same `TURNSTONE_API_TOKEN`. `send_and_wait(prompt, ws_id)` runs the command and the raw bash output is captured from the `ToolResultEvent` — bypassing the costly "agent reads output then re-generates output as completion tokens" round-trip.
3. **Cleanup**`TurnstoneConsole.route_close(ws_id)` closes the workstream.
+15 -7
View File
@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
[project]
name = "turnstone"
version = "1.0.0"
version = "1.5.0a4"
description = "Multi-node AI orchestration platform with tool use, agent routing, and cluster simulation."
readme = "README.md"
license = "BUSL-1.1"
@@ -44,16 +44,17 @@ Repository = "https://github.com/turnstonelabs/turnstone"
Issues = "https://github.com/turnstonelabs/turnstone/issues"
[project.optional-dependencies]
test = ["pytest>=9.0", "pytest-cov>=6.0", "croniter>=3.0"]
test = ["pytest>=9.0", "pytest-cov>=6.0", "croniter>=3.0", "slack-bolt>=1.18", "aiohttp>=3.9"]
dev = ["ruff>=0.9", "mypy>=1.14"]
console = ["croniter>=3.0"]
anthropic = ["anthropic>=0.39"]
postgres = ["psycopg[binary]>=3.2"]
ddg = ["ddgs>=9.0"]
discord = ["discord.py>=2.4"]
tls = ["lacme>=1.0.4"]
tls = ["lacme>=1.0.5"]
sandbox = ["sympy>=1.13", "numpy>=2.0", "scipy>=1.14", "pytest>=9.0"]
all = ["turnstone[console,anthropic,postgres,discord,ddg,tls,sandbox]"]
slack = ["slack-bolt>=1.18", "aiohttp>=3.9"]
all = ["turnstone[console,anthropic,postgres,discord,ddg,tls,sandbox,slack]"]
[project.scripts]
turnstone = "turnstone.cli:main"
@@ -67,6 +68,7 @@ turnstone-bootstrap = "turnstone.bootstrap:main"
[tool.hatch.build.targets.wheel]
include = [
"turnstone/**/*.py",
"turnstone/prompts/**/*.md",
"turnstone/tools/*.json",
"turnstone/ui/static/*.html",
"turnstone/ui/static/*.css",
@@ -74,12 +76,17 @@ include = [
"turnstone/console/static/*.html",
"turnstone/console/static/*.css",
"turnstone/console/static/*.js",
"turnstone/console/static/coordinator/*.html",
"turnstone/console/static/coordinator/*.js",
"turnstone/shared_static/*.css",
"turnstone/shared_static/*.js",
"turnstone/shared_static/katex-0.16.44/**/*",
"turnstone/shared_static/design/**/*",
"turnstone/shared_static/katex-0.16.45/**/*",
"turnstone/shared_static/hljs-11.11.1/**/*",
"turnstone/shared_static/mermaid-11.14.0/**/*",
"turnstone/shared_static/hls-1.6.16/**/*",
"turnstone/sdk/py.typed",
"turnstone/deploy/*.yaml",
]
[tool.pytest.ini_options]
@@ -178,5 +185,6 @@ disallow_untyped_decorators = false
warn_unused_ignores = false
[[tool.mypy.overrides]]
module = "tests.*"
disallow_untyped_defs = false
module = ["slack_bolt", "slack_bolt.*", "slack_sdk", "slack_sdk.*"]
ignore_missing_imports = true
disallow_untyped_calls = false
+32 -1
View File
@@ -5,6 +5,7 @@
# scripts/update-vendored-js.sh katex 0.16.39
# scripts/update-vendored-js.sh hljs 11.12.0
# scripts/update-vendored-js.sh mermaid 11.14.0
# scripts/update-vendored-js.sh hls 1.6.15
#
# This script:
# 1. Downloads the new version from CDN
@@ -18,7 +19,7 @@ STATIC_DIR="turnstone/shared_static"
CDN="https://cdn.jsdelivr.net/npm"
usage() {
echo "Usage: $0 <katex|hljs|mermaid> <version>"
echo "Usage: $0 <katex|hljs|mermaid|hls> <version>"
echo "Example: $0 katex 0.16.39"
exit 1
}
@@ -147,12 +148,42 @@ case "$LIB" in
echo "Done. Old directory removed: ${OLD_DIR}"
;;
hls)
OLD_VERSION=$(detect_old_version "hls")
check_same_version "$OLD_VERSION" "$VERSION" "hls"
OLD_DIR="${STATIC_DIR}/hls-${OLD_VERSION}"
NEW_DIR="${STATIC_DIR}/hls-${VERSION}"
echo "Updating hls.js ${OLD_VERSION} -> ${VERSION}"
mkdir -p "${NEW_DIR}"
echo " Downloading hls.min.js..."
curl -sSfL "${CDN}/hls.js@${VERSION}/dist/hls.min.js" -o "${NEW_DIR}/hls.min.js"
echo " Downloading LICENSE..."
if ! curl -sSfL "${CDN}/hls.js@${VERSION}/LICENSE" -o "${NEW_DIR}/LICENSE" 2>/dev/null; then
if [[ -f "${OLD_DIR}/LICENSE" ]]; then
cp "${OLD_DIR}/LICENSE" "${NEW_DIR}/LICENSE"
else
echo " WARNING: Could not obtain LICENSE for hls.js ${VERSION}"
fi
fi
update_refs "hls-${OLD_VERSION}" "hls-${VERSION}"
rm -rf "${OLD_DIR}"
echo "Done. Old directory removed: ${OLD_DIR}"
;;
*)
echo "Unknown library: ${LIB}"
usage
;;
esac
echo ""
echo "NOTE: If you added a NEW library (not just updating a version), also update"
echo " the _ASSET_RE regex in turnstone/core/web_helpers.py — its negative lookahead"
echo " skips vendored directories to avoid double-versioning static asset URLs."
echo ""
echo "Verify the update:"
echo " git diff --stat"
File diff suppressed because it is too large Load Diff
+828 -16
View File
@@ -2,7 +2,7 @@
"openapi": "3.1.0",
"info": {
"title": "turnstone Server API",
"version": "0.9.2",
"version": "1.5.0a2",
"description": "Single-node workstream management, chat interaction, and real-time streaming."
},
"paths": {
@@ -55,6 +55,7 @@
"tags": [
"Workstreams"
],
"description": "Accepts two content types. Default is `application/json` with a `CreateWorkstreamRequest` body. Alternatively, `multipart/form-data` with one `meta` field (JSON-encoded `CreateWorkstreamRequest` shape) plus zero-or-more `file` parts saves each file as an attachment under the new workstream. When `initial_message` is also set, attachments are reserved onto that turn before the worker thread dispatches; otherwise they remain pending for a follow-up `POST /v1/api/send`.",
"requestBody": {
"required": true,
"content": {
@@ -85,6 +86,26 @@
}
}
}
},
"409": {
"description": "Error 409",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ErrorResponse"
}
}
}
},
"413": {
"description": "Error 413",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ErrorResponse"
}
}
}
}
}
}
@@ -408,7 +429,7 @@
"tags": [
"Streaming"
],
"description": "Global Server-Sent Events stream for state-change broadcasts across all workstreams. Returns text/event-stream.",
"description": "Server-Sent Events stream for node-level state broadcasts. Emits a node_snapshot event on connect (workstreams, health, aggregate), followed by real-time delta events (ws_state, ws_activity, ws_created, ws_closed, ws_rename, health_changed, aggregate). Pass ?expected_node_id=X for identity verification (returns 409 on mismatch).",
"responses": {
"200": {
"description": "Success"
@@ -416,6 +437,426 @@
}
}
},
"/v1/api/workstreams/{ws_id}/delete": {
"post": {
"summary": "Permanently delete a saved workstream",
"operationId": "v1_api_workstreams_{ws_id}_delete_post",
"tags": [
"Workstreams"
],
"parameters": [
{
"name": "ws_id",
"in": "path",
"required": true,
"schema": {
"type": "string"
}
}
],
"responses": {
"200": {
"description": "Success"
},
"400": {
"description": "Error 400",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ErrorResponse"
}
}
}
},
"404": {
"description": "Error 404",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ErrorResponse"
}
}
}
},
"500": {
"description": "Error 500",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ErrorResponse"
}
}
}
}
}
}
},
"/v1/api/workstreams/{ws_id}/open": {
"post": {
"summary": "Load a saved workstream into memory",
"operationId": "v1_api_workstreams_{ws_id}_open_post",
"tags": [
"Workstreams"
],
"parameters": [
{
"name": "ws_id",
"in": "path",
"required": true,
"schema": {
"type": "string"
}
}
],
"responses": {
"200": {
"description": "Success"
},
"400": {
"description": "Error 400",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ErrorResponse"
}
}
}
},
"404": {
"description": "Error 404",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ErrorResponse"
}
}
}
},
"500": {
"description": "Error 500",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ErrorResponse"
}
}
}
}
}
}
},
"/v1/api/workstreams/{ws_id}/title": {
"post": {
"summary": "Set workstream title manually",
"operationId": "v1_api_workstreams_{ws_id}_title_post",
"tags": [
"Workstreams"
],
"parameters": [
{
"name": "ws_id",
"in": "path",
"required": true,
"schema": {
"type": "string"
}
}
],
"responses": {
"200": {
"description": "Success"
},
"400": {
"description": "Error 400",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ErrorResponse"
}
}
}
},
"409": {
"description": "Error 409",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ErrorResponse"
}
}
}
}
}
}
},
"/v1/api/workstreams/{ws_id}/refresh-title": {
"post": {
"summary": "Regenerate workstream title via LLM",
"operationId": "v1_api_workstreams_{ws_id}_refresh-title_post",
"tags": [
"Workstreams"
],
"parameters": [
{
"name": "ws_id",
"in": "path",
"required": true,
"schema": {
"type": "string"
}
}
],
"responses": {
"200": {
"description": "Success"
},
"404": {
"description": "Error 404",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ErrorResponse"
}
}
}
}
}
}
},
"/v1/api/workstreams/{ws_id}/attachments": {
"post": {
"summary": "Upload a file (multipart/form-data, field 'file') and attach it to the caller's next user turn on this workstream. Validates size, MIME, and UTF-8 for text; magic-byte sniff for images. Ownership failures are masked as 404 so non-owners cannot enumerate workstream existence; a 403 indicates a scope/auth failure from the middleware layer.",
"operationId": "v1_api_workstreams_{ws_id}_attachments_post",
"tags": [
"Attachments"
],
"parameters": [
{
"name": "ws_id",
"in": "path",
"required": true,
"schema": {
"type": "string"
}
}
],
"responses": {
"200": {
"description": "Success",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/UploadAttachmentResponse"
}
}
}
},
"400": {
"description": "Error 400",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ErrorResponse"
}
}
}
},
"403": {
"description": "Error 403",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ErrorResponse"
}
}
}
},
"404": {
"description": "Error 404",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ErrorResponse"
}
}
}
},
"409": {
"description": "Error 409",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ErrorResponse"
}
}
}
},
"413": {
"description": "Error 413",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ErrorResponse"
}
}
}
}
}
},
"get": {
"summary": "List the caller's pending (unconsumed) attachments for this workstream. Ownership failures are masked as 404.",
"operationId": "v1_api_workstreams_{ws_id}_attachments_get",
"tags": [
"Attachments"
],
"parameters": [
{
"name": "ws_id",
"in": "path",
"required": true,
"schema": {
"type": "string"
}
}
],
"responses": {
"200": {
"description": "Success",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ListAttachmentsResponse"
}
}
}
},
"403": {
"description": "Error 403",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ErrorResponse"
}
}
}
},
"404": {
"description": "Error 404",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ErrorResponse"
}
}
}
}
}
}
},
"/v1/api/workstreams/{ws_id}/attachments/{attachment_id}/content": {
"get": {
"summary": "Return raw bytes of an attachment with its stored Content-Type. Ownership failures are masked as 404.",
"operationId": "v1_api_workstreams_{ws_id}_attachments_{attachment_id}_content_get",
"tags": [
"Attachments"
],
"parameters": [
{
"name": "ws_id",
"in": "path",
"required": true,
"schema": {
"type": "string"
}
},
{
"name": "attachment_id",
"in": "path",
"required": true,
"schema": {
"type": "string"
}
}
],
"responses": {
"200": {
"description": "Success"
},
"403": {
"description": "Error 403",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ErrorResponse"
}
}
}
},
"404": {
"description": "Error 404",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ErrorResponse"
}
}
}
}
}
}
},
"/v1/api/workstreams/{ws_id}/attachments/{attachment_id}": {
"delete": {
"summary": "Remove a pending attachment (consumed attachments return 404). Ownership failures are also masked as 404.",
"operationId": "v1_api_workstreams_{ws_id}_attachments_{attachment_id}_delete",
"tags": [
"Attachments"
],
"parameters": [
{
"name": "ws_id",
"in": "path",
"required": true,
"schema": {
"type": "string"
}
},
{
"name": "attachment_id",
"in": "path",
"required": true,
"schema": {
"type": "string"
}
}
],
"responses": {
"200": {
"description": "Success"
},
"403": {
"description": "Error 403",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ErrorResponse"
}
}
}
},
"404": {
"description": "Error 404",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ErrorResponse"
}
}
}
}
}
}
},
"/v1/api/workstreams/saved": {
"get": {
"summary": "List saved workstreams",
@@ -891,6 +1332,106 @@
}
}
},
"/v1/api/admin/settings": {
"get": {
"summary": "List interface.* settings with values and sources",
"operationId": "v1_api_admin_settings_get",
"tags": [
"Admin"
],
"responses": {
"200": {
"description": "Success"
}
}
}
},
"/v1/api/admin/settings/{key}": {
"put": {
"summary": "Update an interface.* setting",
"operationId": "v1_api_admin_settings_{key}_put",
"tags": [
"Admin"
],
"parameters": [
{
"name": "key",
"in": "path",
"required": true,
"schema": {
"type": "string"
}
}
],
"responses": {
"200": {
"description": "Success"
},
"400": {
"description": "Error 400",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ErrorResponse"
}
}
}
},
"503": {
"description": "Error 503",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ErrorResponse"
}
}
}
}
}
},
"post": {
"summary": "Update an interface.* setting (alias for PUT)",
"operationId": "v1_api_admin_settings_{key}_post",
"tags": [
"Admin"
],
"parameters": [
{
"name": "key",
"in": "path",
"required": true,
"schema": {
"type": "string"
}
}
],
"responses": {
"200": {
"description": "Success"
},
"400": {
"description": "Error 400",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ErrorResponse"
}
}
}
},
"503": {
"description": "Error 503",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ErrorResponse"
}
}
}
}
}
}
},
"/health": {
"get": {
"summary": "Server health check",
@@ -1132,6 +1673,22 @@
"description": "Target workstream ID",
"title": "Ws Id",
"type": "string"
},
"attachment_ids": {
"anyOf": [
{
"items": {
"type": "string"
},
"type": "array"
},
{
"type": "null"
}
],
"default": null,
"description": "Explicit list of attachment ids to inject into this turn. When omitted, any pending attachments for the caller on this workstream are auto-consumed. An empty list disables auto-consumption for this send.",
"title": "Attachment Ids"
}
},
"required": [
@@ -1144,13 +1701,57 @@
"SendResponse": {
"properties": {
"status": {
"description": "'ok' or 'busy'",
"description": "'ok', 'busy', 'queued', or 'queue_full'",
"examples": [
"ok",
"busy"
"busy",
"queued",
"queue_full"
],
"title": "Status",
"type": "string"
},
"attached_ids": {
"description": "Attachment ids actually reserved onto this turn. Subset of the request's `attachment_ids` (or the auto-consumed pending set). Empty when the send carries no attachments.",
"items": {
"type": "string"
},
"title": "Attached Ids",
"type": "array"
},
"dropped_attachment_ids": {
"description": "Attachment ids the caller requested that the server could not reserve (lost a race, already consumed, or cross-scope). The request still proceeds with whatever was reserved; the client can retry uploads or surface a partial-attach warning.",
"items": {
"type": "string"
},
"title": "Dropped Attachment Ids",
"type": "array"
},
"priority": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"description": "Set on `queued` responses: relative priority of the queued message.",
"title": "Priority"
},
"msg_id": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"description": "Set on `queued` responses: id used to dequeue the message.",
"title": "Msg Id"
}
},
"required": [
@@ -1289,11 +1890,75 @@
"description": "Skill name (replaces default skills)",
"title": "Skill",
"type": "string"
},
"notify_targets": {
"anyOf": [
{
"type": "string"
},
{
"items": {
"additionalProperties": {
"type": "string"
},
"type": "object"
},
"type": "array"
}
],
"default": "[]",
"description": "Notification targets, accepted as either a JSON string or a structured array of objects containing channel_type + channel_id/user_id",
"title": "Notify Targets"
},
"client_type": {
"default": "",
"description": "Client surface type (web, cli, chat). Defaults to web for server-created sessions.",
"title": "Client Type",
"type": "string"
},
"initial_message": {
"default": "",
"description": "Optional first user message dispatched as a background turn after the workstream is created. When attachments are also provided (via the multipart variant), they are reserved onto this turn.",
"title": "Initial Message",
"type": "string"
},
"ws_id": {
"default": "",
"description": "Optional caller-supplied workstream id (32-hex). Required when creating with attachments via the cluster routing layer so the console can hash to the owning node before the multipart body lands. Auto-generated when omitted.",
"title": "Ws Id",
"type": "string"
},
"kind": {
"$ref": "#/components/schemas/WorkstreamKind",
"default": "interactive",
"description": "Workstream kind \u2014 'interactive' (default) or 'coordinator'. Coordinator workstreams are created by the console's own /v1/api/coordinator/new endpoint; clients hitting /v1/api/workstreams/new should leave this at the default."
},
"parent_ws_id": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"description": "Optional parent workstream id. Populated on children spawned by a coordinator so the parent/child relationship survives restart and appears in audit / list views.",
"title": "Parent Ws Id"
}
},
"title": "CreateWorkstreamRequest",
"type": "object"
},
"WorkstreamKind": {
"description": "Classifier for which manager hosts a workstream.\n\nStrEnum so members are drop-in ``str`` replacements for the DB column,\nJSON payloads, and existing ``==`` comparisons against raw strings.\nNarrow internal annotations to this type; wide boundaries (HTTP body,\nDB row) stay ``str`` and parse via ``WorkstreamKind(raw)`` / ``from_raw``\nat the edge.",
"enum": [
"interactive",
"coordinator"
],
"title": "WorkstreamKind",
"type": "string"
},
"CreateWorkstreamResponse": {
"properties": {
"ws_id": {
@@ -1317,6 +1982,14 @@
"description": "Number of messages in the resumed workstream",
"title": "Message Count",
"type": "integer"
},
"attachment_ids": {
"description": "Ids of attachments saved by this request (multipart variant only). Already reserved onto the initial_message turn when one was provided; otherwise left pending for a follow-up POST /v1/api/send.",
"items": {
"type": "string"
},
"title": "Attachment Ids",
"type": "array"
}
},
"required": [
@@ -1369,6 +2042,22 @@
"state": {
"title": "State",
"type": "string"
},
"kind": {
"$ref": "#/components/schemas/WorkstreamKind",
"default": "interactive"
},
"parent_ws_id": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Parent Ws Id"
}
},
"required": [
@@ -1493,6 +2182,27 @@
"default": "",
"title": "Model Alias",
"type": "string"
},
"kind": {
"$ref": "#/components/schemas/WorkstreamKind",
"default": "interactive"
},
"parent_ws_id": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Parent Ws Id"
},
"user_id": {
"default": "",
"title": "User Id",
"type": "string"
}
},
"required": [
@@ -1571,6 +2281,108 @@
"title": "SavedWorkstreamInfo",
"type": "object"
},
"UploadAttachmentResponse": {
"description": "Returned after a successful upload.",
"properties": {
"attachment_id": {
"description": "Opaque id for this attachment",
"title": "Attachment Id",
"type": "string"
},
"filename": {
"description": "Original upload filename",
"title": "Filename",
"type": "string"
},
"mime_type": {
"description": "Canonicalized MIME type",
"title": "Mime Type",
"type": "string"
},
"size_bytes": {
"description": "Payload size in bytes",
"title": "Size Bytes",
"type": "integer"
},
"kind": {
"description": "'image' or 'text'",
"examples": [
"image",
"text"
],
"title": "Kind",
"type": "string"
}
},
"required": [
"attachment_id",
"filename",
"mime_type",
"size_bytes",
"kind"
],
"title": "UploadAttachmentResponse",
"type": "object"
},
"ListAttachmentsResponse": {
"properties": {
"attachments": {
"description": "Pending (unconsumed) attachments for caller+workstream",
"items": {
"$ref": "#/components/schemas/AttachmentInfo"
},
"title": "Attachments",
"type": "array"
}
},
"required": [
"attachments"
],
"title": "ListAttachmentsResponse",
"type": "object"
},
"AttachmentInfo": {
"properties": {
"attachment_id": {
"description": "Opaque id for this attachment",
"title": "Attachment Id",
"type": "string"
},
"filename": {
"description": "Original upload filename",
"title": "Filename",
"type": "string"
},
"mime_type": {
"description": "Canonicalized MIME type",
"title": "Mime Type",
"type": "string"
},
"size_bytes": {
"description": "Payload size in bytes",
"title": "Size Bytes",
"type": "integer"
},
"kind": {
"description": "'image' or 'text'",
"examples": [
"image",
"text"
],
"title": "Kind",
"type": "string"
}
},
"required": [
"attachment_id",
"filename",
"mime_type",
"size_bytes",
"kind"
],
"title": "AttachmentInfo",
"type": "object"
},
"HealthResponse": {
"properties": {
"status": {
@@ -1656,20 +2468,10 @@
],
"title": "Status",
"type": "string"
},
"circuit_state": {
"examples": [
"closed",
"open",
"half_open"
],
"title": "Circuit State",
"type": "string"
}
},
"required": [
"status",
"circuit_state"
"status"
],
"title": "BackendStatus",
"type": "object"
@@ -2036,6 +2838,16 @@
},
"title": "Models",
"type": "array"
},
"default_alias": {
"default": "",
"title": "Default Alias",
"type": "string"
},
"channel_default_alias": {
"default": "",
"title": "Channel Default Alias",
"type": "string"
}
},
"title": "ListAvailableModelsResponse",
@@ -2043,4 +2855,4 @@
}
}
}
}
}
+158 -151
View File
@@ -14,38 +14,35 @@
}
},
"node_modules/@emnapi/core": {
"version": "1.9.1",
"resolved": "https://registry.npmjs.org/@emnapi/core/-/core-1.9.1.tgz",
"integrity": "sha512-mukuNALVsoix/w1BJwFzwXBN/dHeejQtuVzcDsfOEsdpCumXb/E9j8w11h5S54tT1xhifGfbbSm/ICrObRb3KA==",
"version": "1.9.2",
"resolved": "https://registry.npmjs.org/@emnapi/core/-/core-1.9.2.tgz",
"integrity": "sha512-UC+ZhH3XtczQYfOlu3lNEkdW/p4dsJ1r/bP7H8+rhao3TTTMO1ATq/4DdIi23XuGoFY+Cz0JmCbdVl0hz9jZcA==",
"dev": true,
"license": "MIT",
"optional": true,
"peer": true,
"dependencies": {
"@emnapi/wasi-threads": "1.2.0",
"@emnapi/wasi-threads": "1.2.1",
"tslib": "^2.4.0"
}
},
"node_modules/@emnapi/runtime": {
"version": "1.9.1",
"resolved": "https://registry.npmjs.org/@emnapi/runtime/-/runtime-1.9.1.tgz",
"integrity": "sha512-VYi5+ZVLhpgK4hQ0TAjiQiZ6ol0oe4mBx7mVv7IflsiEp0OWoVsp/+f9Vc1hOhE0TtkORVrI1GvzyreqpgWtkA==",
"version": "1.9.2",
"resolved": "https://registry.npmjs.org/@emnapi/runtime/-/runtime-1.9.2.tgz",
"integrity": "sha512-3U4+MIWHImeyu1wnmVygh5WlgfYDtyf0k8AbLhMFxOipihf6nrWC4syIm/SwEeec0mNSafiiNnMJwbza/Is6Lw==",
"dev": true,
"license": "MIT",
"optional": true,
"peer": true,
"dependencies": {
"tslib": "^2.4.0"
}
},
"node_modules/@emnapi/wasi-threads": {
"version": "1.2.0",
"resolved": "https://registry.npmjs.org/@emnapi/wasi-threads/-/wasi-threads-1.2.0.tgz",
"integrity": "sha512-N10dEJNSsUx41Z6pZsXU8FjPjpBEplgH24sfkmITrBED1/U2Esum9F3lfLrMjKHHjmi557zQn7kR9R+XWXu5Rg==",
"version": "1.2.1",
"resolved": "https://registry.npmjs.org/@emnapi/wasi-threads/-/wasi-threads-1.2.1.tgz",
"integrity": "sha512-uTII7OYF+/Mes/MrcIOYp5yOtSMLBWSIoLPpcgwipoiKbli6k322tcoFsxoIIxPDqW01SQGAgko4EzZi2BNv2w==",
"dev": true,
"license": "MIT",
"optional": true,
"peer": true,
"dependencies": {
"tslib": "^2.4.0"
}
@@ -58,9 +55,9 @@
"license": "MIT"
},
"node_modules/@napi-rs/wasm-runtime": {
"version": "1.1.2",
"resolved": "https://registry.npmjs.org/@napi-rs/wasm-runtime/-/wasm-runtime-1.1.2.tgz",
"integrity": "sha512-sNXv5oLJ7ob93xkZ1XnxisYhGYXfaG9f65/ZgYuAu3qt7b3NadcOEhLvx28hv31PgX8SZJRYrAIPQilQmFpLVw==",
"version": "1.1.4",
"resolved": "https://registry.npmjs.org/@napi-rs/wasm-runtime/-/wasm-runtime-1.1.4.tgz",
"integrity": "sha512-3NQNNgA1YSlJb/kMH1ildASP9HW7/7kYnRI2szWJaofaS1hWmbGI4H+d3+22aGzXXN9IJ+n+GiFVcGipJP18ow==",
"dev": true,
"license": "MIT",
"optional": true,
@@ -77,9 +74,9 @@
}
},
"node_modules/@oxc-project/types": {
"version": "0.122.0",
"resolved": "https://registry.npmjs.org/@oxc-project/types/-/types-0.122.0.tgz",
"integrity": "sha512-oLAl5kBpV4w69UtFZ9xqcmTi+GENWOcPF7FCrczTiBbmC0ibXxCwyvZGbO39rCVEuLGAZM84DH0pUIyyv/YJzA==",
"version": "0.124.0",
"resolved": "https://registry.npmjs.org/@oxc-project/types/-/types-0.124.0.tgz",
"integrity": "sha512-VBFWMTBvHxS11Z5Lvlr3IWgrwhMTXV+Md+EQF0Xf60+wAdsGFTBx7X7K/hP4pi8N7dcm1RvcHwDxZ16Qx8keUg==",
"dev": true,
"license": "MIT",
"funding": {
@@ -87,9 +84,9 @@
}
},
"node_modules/@rolldown/binding-android-arm64": {
"version": "1.0.0-rc.12",
"resolved": "https://registry.npmjs.org/@rolldown/binding-android-arm64/-/binding-android-arm64-1.0.0-rc.12.tgz",
"integrity": "sha512-pv1y2Fv0JybcykuiiD3qBOBdz6RteYojRFY1d+b95WVuzx211CRh+ytI/+9iVyWQ6koTh5dawe4S/yRfOFjgaA==",
"version": "1.0.0-rc.15",
"resolved": "https://registry.npmjs.org/@rolldown/binding-android-arm64/-/binding-android-arm64-1.0.0-rc.15.tgz",
"integrity": "sha512-YYe6aWruPZDtHNpwu7+qAHEMbQ/yRl6atqb/AhznLTnD3UY99Q1jE7ihLSahNWkF4EqRPVC4SiR4O0UkLK02tA==",
"cpu": [
"arm64"
],
@@ -104,9 +101,9 @@
}
},
"node_modules/@rolldown/binding-darwin-arm64": {
"version": "1.0.0-rc.12",
"resolved": "https://registry.npmjs.org/@rolldown/binding-darwin-arm64/-/binding-darwin-arm64-1.0.0-rc.12.tgz",
"integrity": "sha512-cFYr6zTG/3PXXF3pUO+umXxt1wkRK/0AYT8lDwuqvRC+LuKYWSAQAQZjCWDQpAH172ZV6ieYrNnFzVVcnSflAg==",
"version": "1.0.0-rc.15",
"resolved": "https://registry.npmjs.org/@rolldown/binding-darwin-arm64/-/binding-darwin-arm64-1.0.0-rc.15.tgz",
"integrity": "sha512-oArR/ig8wNTPYsXL+Mzhs0oxhxfuHRfG7Ikw7jXsw8mYOtk71W0OkF2VEVh699pdmzjPQsTjlD1JIOoHkLP1Fg==",
"cpu": [
"arm64"
],
@@ -121,9 +118,9 @@
}
},
"node_modules/@rolldown/binding-darwin-x64": {
"version": "1.0.0-rc.12",
"resolved": "https://registry.npmjs.org/@rolldown/binding-darwin-x64/-/binding-darwin-x64-1.0.0-rc.12.tgz",
"integrity": "sha512-ZCsYknnHzeXYps0lGBz8JrF37GpE9bFVefrlmDrAQhOEi4IOIlcoU1+FwHEtyXGx2VkYAvhu7dyBf75EJQffBw==",
"version": "1.0.0-rc.15",
"resolved": "https://registry.npmjs.org/@rolldown/binding-darwin-x64/-/binding-darwin-x64-1.0.0-rc.15.tgz",
"integrity": "sha512-YzeVqOqjPYvUbJSWJ4EDL8ahbmsIXQpgL3JVipmN+MX0XnXMeWomLN3Fb+nwCmP/jfyqte5I3XRSm7OfQrbyxw==",
"cpu": [
"x64"
],
@@ -138,9 +135,9 @@
}
},
"node_modules/@rolldown/binding-freebsd-x64": {
"version": "1.0.0-rc.12",
"resolved": "https://registry.npmjs.org/@rolldown/binding-freebsd-x64/-/binding-freebsd-x64-1.0.0-rc.12.tgz",
"integrity": "sha512-dMLeprcVsyJsKolRXyoTH3NL6qtsT0Y2xeuEA8WQJquWFXkEC4bcu1rLZZSnZRMtAqwtrF/Ib9Ddtpa/Gkge9Q==",
"version": "1.0.0-rc.15",
"resolved": "https://registry.npmjs.org/@rolldown/binding-freebsd-x64/-/binding-freebsd-x64-1.0.0-rc.15.tgz",
"integrity": "sha512-9Erhx956jeQ0nNTyif1+QWAXDRD38ZNjr//bSHrt6wDwB+QkAfl2q6Mn1k6OBPerznjRmbM10lgRb1Pli4xZPw==",
"cpu": [
"x64"
],
@@ -155,9 +152,9 @@
}
},
"node_modules/@rolldown/binding-linux-arm-gnueabihf": {
"version": "1.0.0-rc.12",
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-arm-gnueabihf/-/binding-linux-arm-gnueabihf-1.0.0-rc.12.tgz",
"integrity": "sha512-YqWjAgGC/9M1lz3GR1r1rP79nMgo3mQiiA+Hfo+pvKFK1fAJ1bCi0ZQVh8noOqNacuY1qIcfyVfP6HoyBRZ85Q==",
"version": "1.0.0-rc.15",
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-arm-gnueabihf/-/binding-linux-arm-gnueabihf-1.0.0-rc.15.tgz",
"integrity": "sha512-cVwk0w8QbZJGTnP/AHQBs5yNwmpgGYStL88t4UIaqcvYJWBfS0s3oqVLZPwsPU6M0zlW4GqjP0Zq5MnAGwFeGA==",
"cpu": [
"arm"
],
@@ -172,9 +169,9 @@
}
},
"node_modules/@rolldown/binding-linux-arm64-gnu": {
"version": "1.0.0-rc.12",
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-arm64-gnu/-/binding-linux-arm64-gnu-1.0.0-rc.12.tgz",
"integrity": "sha512-/I5AS4cIroLpslsmzXfwbe5OmWvSsrFuEw3mwvbQ1kDxJ822hFHIx+vsN/TAzNVyepI/j/GSzrtCIwQPeKCLIg==",
"version": "1.0.0-rc.15",
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-arm64-gnu/-/binding-linux-arm64-gnu-1.0.0-rc.15.tgz",
"integrity": "sha512-eBZ/u8iAK9SoHGanqe/jrPnY0JvBN6iXbVOsbO38mbz+ZJsaobExAm1Iu+rxa4S1l2FjG0qEZn4Rc6X8n+9M+w==",
"cpu": [
"arm64"
],
@@ -192,9 +189,9 @@
}
},
"node_modules/@rolldown/binding-linux-arm64-musl": {
"version": "1.0.0-rc.12",
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-arm64-musl/-/binding-linux-arm64-musl-1.0.0-rc.12.tgz",
"integrity": "sha512-V6/wZztnBqlx5hJQqNWwFdxIKN0m38p8Jas+VoSfgH54HSj9tKTt1dZvG6JRHcjh6D7TvrJPWFGaY9UBVOaWPw==",
"version": "1.0.0-rc.15",
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-arm64-musl/-/binding-linux-arm64-musl-1.0.0-rc.15.tgz",
"integrity": "sha512-ZvRYMGrAklV9PEkgt4LQM6MjQX2P58HPAuecwYObY2DhS2t35R0I810bKi0wmaYORt6m/2Sm+Z+nFgb0WhXNcQ==",
"cpu": [
"arm64"
],
@@ -212,9 +209,9 @@
}
},
"node_modules/@rolldown/binding-linux-ppc64-gnu": {
"version": "1.0.0-rc.12",
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-ppc64-gnu/-/binding-linux-ppc64-gnu-1.0.0-rc.12.tgz",
"integrity": "sha512-AP3E9BpcUYliZCxa3w5Kwj9OtEVDYK6sVoUzy4vTOJsjPOgdaJZKFmN4oOlX0Wp0RPV2ETfmIra9x1xuayFB7g==",
"version": "1.0.0-rc.15",
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-ppc64-gnu/-/binding-linux-ppc64-gnu-1.0.0-rc.15.tgz",
"integrity": "sha512-VDpgGBzgfg5hLg+uBpCLoFG5kVvEyafmfxGUV0UHLcL5irxAK7PKNeC2MwClgk6ZAiNhmo9FLhRYgvMmedLtnQ==",
"cpu": [
"ppc64"
],
@@ -232,9 +229,9 @@
}
},
"node_modules/@rolldown/binding-linux-s390x-gnu": {
"version": "1.0.0-rc.12",
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-s390x-gnu/-/binding-linux-s390x-gnu-1.0.0-rc.12.tgz",
"integrity": "sha512-nWwpvUSPkoFmZo0kQazZYOrT7J5DGOJ/+QHHzjvNlooDZED8oH82Yg67HvehPPLAg5fUff7TfWFHQS8IV1n3og==",
"version": "1.0.0-rc.15",
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-s390x-gnu/-/binding-linux-s390x-gnu-1.0.0-rc.15.tgz",
"integrity": "sha512-y1uXY3qQWCzcPgRJATPSOUP4tCemh4uBdY7e3EZbVwCJTY3gLJWnQABgeUetvED+bt1FQ01OeZwvhLS2bpNrAQ==",
"cpu": [
"s390x"
],
@@ -252,9 +249,9 @@
}
},
"node_modules/@rolldown/binding-linux-x64-gnu": {
"version": "1.0.0-rc.12",
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-x64-gnu/-/binding-linux-x64-gnu-1.0.0-rc.12.tgz",
"integrity": "sha512-RNrafz5bcwRy+O9e6P8Z/OCAJW/A+qtBczIqVYwTs14pf4iV1/+eKEjdOUta93q2TsT/FI0XYDP3TCky38LMAg==",
"version": "1.0.0-rc.15",
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-x64-gnu/-/binding-linux-x64-gnu-1.0.0-rc.15.tgz",
"integrity": "sha512-023bTPBod7J3Y/4fzAN6QtpkSABR0rigtrwaP+qSEabUh5zf6ELr9Nc7GujaROuPY3uwdSIXWrvhn1KxOvurWA==",
"cpu": [
"x64"
],
@@ -272,9 +269,9 @@
}
},
"node_modules/@rolldown/binding-linux-x64-musl": {
"version": "1.0.0-rc.12",
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-x64-musl/-/binding-linux-x64-musl-1.0.0-rc.12.tgz",
"integrity": "sha512-Jpw/0iwoKWx3LJ2rc1yjFrj+T7iHZn2JDg1Yny1ma0luviFS4mhAIcd1LFNxK3EYu3DHWCps0ydXQ5i/rrJ2ig==",
"version": "1.0.0-rc.15",
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-x64-musl/-/binding-linux-x64-musl-1.0.0-rc.15.tgz",
"integrity": "sha512-witB2O0/hU4CgfOOKUoeFgQ4GktPi1eEbAhaLAIpgD6+ZnhcPkUtPsoKKHRzmOoWPZue46IThdSgdo4XneOLYw==",
"cpu": [
"x64"
],
@@ -292,9 +289,9 @@
}
},
"node_modules/@rolldown/binding-openharmony-arm64": {
"version": "1.0.0-rc.12",
"resolved": "https://registry.npmjs.org/@rolldown/binding-openharmony-arm64/-/binding-openharmony-arm64-1.0.0-rc.12.tgz",
"integrity": "sha512-vRugONE4yMfVn0+7lUKdKvN4D5YusEiPilaoO2sgUWpCvrncvWgPMzK00ZFFJuiPgLwgFNP5eSiUlv2tfc+lpA==",
"version": "1.0.0-rc.15",
"resolved": "https://registry.npmjs.org/@rolldown/binding-openharmony-arm64/-/binding-openharmony-arm64-1.0.0-rc.15.tgz",
"integrity": "sha512-UCL68NJ0Ud5zRipXZE9dF5PmirzJE4E4BCIOOssEnM7wLDsxjc6Qb0sGDxTNRTP53I6MZpygyCpY8Aa8sPfKPg==",
"cpu": [
"arm64"
],
@@ -309,9 +306,9 @@
}
},
"node_modules/@rolldown/binding-wasm32-wasi": {
"version": "1.0.0-rc.12",
"resolved": "https://registry.npmjs.org/@rolldown/binding-wasm32-wasi/-/binding-wasm32-wasi-1.0.0-rc.12.tgz",
"integrity": "sha512-ykGiLr/6kkiHc0XnBfmFJuCjr5ZYKKofkx+chJWDjitX+KsJuAmrzWhwyOMSHzPhzOHOy7u9HlFoa5MoAOJ/Zg==",
"version": "1.0.0-rc.15",
"resolved": "https://registry.npmjs.org/@rolldown/binding-wasm32-wasi/-/binding-wasm32-wasi-1.0.0-rc.15.tgz",
"integrity": "sha512-ApLruZq/ig+nhaE7OJm4lDjayUnOHVUa77zGeqnqZ9pn0ovdVbbNPerVibLXDmWeUZXjIYIT8V3xkT58Rm9u5Q==",
"cpu": [
"wasm32"
],
@@ -319,16 +316,18 @@
"license": "MIT",
"optional": true,
"dependencies": {
"@napi-rs/wasm-runtime": "^1.1.1"
"@emnapi/core": "1.9.2",
"@emnapi/runtime": "1.9.2",
"@napi-rs/wasm-runtime": "^1.1.3"
},
"engines": {
"node": ">=14.0.0"
}
},
"node_modules/@rolldown/binding-win32-arm64-msvc": {
"version": "1.0.0-rc.12",
"resolved": "https://registry.npmjs.org/@rolldown/binding-win32-arm64-msvc/-/binding-win32-arm64-msvc-1.0.0-rc.12.tgz",
"integrity": "sha512-5eOND4duWkwx1AzCxadcOrNeighiLwMInEADT0YM7xeEOOFcovWZCq8dadXgcRHSf3Ulh1kFo/qvzoFiCLOL1Q==",
"version": "1.0.0-rc.15",
"resolved": "https://registry.npmjs.org/@rolldown/binding-win32-arm64-msvc/-/binding-win32-arm64-msvc-1.0.0-rc.15.tgz",
"integrity": "sha512-KmoUoU7HnN+Si5YWJigfTws1jz1bKBYDQKdbLspz0UaqjjFkddHsqorgiW1mxcAj88lYUE6NC/zJNwT+SloqtA==",
"cpu": [
"arm64"
],
@@ -343,9 +342,9 @@
}
},
"node_modules/@rolldown/binding-win32-x64-msvc": {
"version": "1.0.0-rc.12",
"resolved": "https://registry.npmjs.org/@rolldown/binding-win32-x64-msvc/-/binding-win32-x64-msvc-1.0.0-rc.12.tgz",
"integrity": "sha512-PyqoipaswDLAZtot351MLhrlrh6lcZPo2LSYE+VDxbVk24LVKAGOuE4hb8xZQmrPAuEtTZW8E6D2zc5EUZX4Lw==",
"version": "1.0.0-rc.15",
"resolved": "https://registry.npmjs.org/@rolldown/binding-win32-x64-msvc/-/binding-win32-x64-msvc-1.0.0-rc.15.tgz",
"integrity": "sha512-3P2A8L+x75qavWLe/Dll3EYBJLQmtkJN8rfh+U/eR3MqMgL/h98PhYI+JFfXuDPgPeCB7iZAKiqii5vqOvnA0g==",
"cpu": [
"x64"
],
@@ -360,9 +359,9 @@
}
},
"node_modules/@rolldown/pluginutils": {
"version": "1.0.0-rc.12",
"resolved": "https://registry.npmjs.org/@rolldown/pluginutils/-/pluginutils-1.0.0-rc.12.tgz",
"integrity": "sha512-HHMwmarRKvoFsJorqYlFeFRzXZqCt2ETQlEDOb9aqssrnVBB1/+xgTGtuTrIk5vzLNX1MjMtTf7W9z3tsSbrxw==",
"version": "1.0.0-rc.15",
"resolved": "https://registry.npmjs.org/@rolldown/pluginutils/-/pluginutils-1.0.0-rc.15.tgz",
"integrity": "sha512-UromN0peaE53IaBRe9W7CjrZgXl90fqGpK+mIZbA3qSTeYqg3pqpROBdIPvOG3F5ereDHNwoHBI2e50n1BDr1g==",
"dev": true,
"license": "MIT"
},
@@ -410,16 +409,16 @@
"license": "MIT"
},
"node_modules/@vitest/expect": {
"version": "4.1.2",
"resolved": "https://registry.npmjs.org/@vitest/expect/-/expect-4.1.2.tgz",
"integrity": "sha512-gbu+7B0YgUJ2nkdsRJrFFW6X7NTP44WlhiclHniUhxADQJH5Szt9mZ9hWnJPJ8YwOK5zUOSSlSvyzRf0u1DSBQ==",
"version": "4.1.4",
"resolved": "https://registry.npmjs.org/@vitest/expect/-/expect-4.1.4.tgz",
"integrity": "sha512-iPBpra+VDuXmBFI3FMKHSFXp3Gx5HfmSCE8X67Dn+bwephCnQCaB7qWK2ldHa+8ncN8hJU8VTMcxjPpyMkUjww==",
"dev": true,
"license": "MIT",
"dependencies": {
"@standard-schema/spec": "^1.1.0",
"@types/chai": "^5.2.2",
"@vitest/spy": "4.1.2",
"@vitest/utils": "4.1.2",
"@vitest/spy": "4.1.4",
"@vitest/utils": "4.1.4",
"chai": "^6.2.2",
"tinyrainbow": "^3.1.0"
},
@@ -428,13 +427,13 @@
}
},
"node_modules/@vitest/mocker": {
"version": "4.1.2",
"resolved": "https://registry.npmjs.org/@vitest/mocker/-/mocker-4.1.2.tgz",
"integrity": "sha512-Ize4iQtEALHDttPRCmN+FKqOl2vxTiNUhzobQFFt/BM1lRUTG7zRCLOykG/6Vo4E4hnUdfVLo5/eqKPukcWW7Q==",
"version": "4.1.4",
"resolved": "https://registry.npmjs.org/@vitest/mocker/-/mocker-4.1.4.tgz",
"integrity": "sha512-R9HTZBhW6yCSGbGQnDnH3QHfJxokKN4KB+Yvk9Q1le7eQNYwiCyKxmLmurSpFy6BzJanSLuEUDrD+j97Q+ZLPg==",
"dev": true,
"license": "MIT",
"dependencies": {
"@vitest/spy": "4.1.2",
"@vitest/spy": "4.1.4",
"estree-walker": "^3.0.3",
"magic-string": "^0.30.21"
},
@@ -455,9 +454,9 @@
}
},
"node_modules/@vitest/pretty-format": {
"version": "4.1.2",
"resolved": "https://registry.npmjs.org/@vitest/pretty-format/-/pretty-format-4.1.2.tgz",
"integrity": "sha512-dwQga8aejqeuB+TvXCMzSQemvV9hNEtDDpgUKDzOmNQayl2OG241PSWeJwKRH3CiC+sESrmoFd49rfnq7T4RnA==",
"version": "4.1.4",
"resolved": "https://registry.npmjs.org/@vitest/pretty-format/-/pretty-format-4.1.4.tgz",
"integrity": "sha512-ddmDHU0gjEUyEVLxtZa7xamrpIefdEETu3nZjWtHeZX4QxqJ7tRxSteHVXJOcr8jhiLoGAhkK4WJ3WqBpjx42A==",
"dev": true,
"license": "MIT",
"dependencies": {
@@ -468,13 +467,13 @@
}
},
"node_modules/@vitest/runner": {
"version": "4.1.2",
"resolved": "https://registry.npmjs.org/@vitest/runner/-/runner-4.1.2.tgz",
"integrity": "sha512-Gr+FQan34CdiYAwpGJmQG8PgkyFVmARK8/xSijia3eTFgVfpcpztWLuP6FttGNfPLJhaZVP/euvujeNYar36OQ==",
"version": "4.1.4",
"resolved": "https://registry.npmjs.org/@vitest/runner/-/runner-4.1.4.tgz",
"integrity": "sha512-xTp7VZ5aXP5ZJrn15UtJUWlx6qXLnGtF6jNxHepdPHpMfz/aVPx+htHtgcAL2mDXJgKhpoo2e9/hVJsIeFbytQ==",
"dev": true,
"license": "MIT",
"dependencies": {
"@vitest/utils": "4.1.2",
"@vitest/utils": "4.1.4",
"pathe": "^2.0.3"
},
"funding": {
@@ -482,14 +481,14 @@
}
},
"node_modules/@vitest/snapshot": {
"version": "4.1.2",
"resolved": "https://registry.npmjs.org/@vitest/snapshot/-/snapshot-4.1.2.tgz",
"integrity": "sha512-g7yfUmxYS4mNxk31qbOYsSt2F4m1E02LFqO53Xpzg3zKMhLAPZAjjfyl9e6z7HrW6LvUdTwAQR3HHfLjpko16A==",
"version": "4.1.4",
"resolved": "https://registry.npmjs.org/@vitest/snapshot/-/snapshot-4.1.4.tgz",
"integrity": "sha512-MCjCFgaS8aZz+m5nTcEcgk/xhWv0rEH4Yl53PPlMXOZ1/Ka2VcZU6CJ+MgYCZbcJvzGhQRjVrGQNZqkGPttIKw==",
"dev": true,
"license": "MIT",
"dependencies": {
"@vitest/pretty-format": "4.1.2",
"@vitest/utils": "4.1.2",
"@vitest/pretty-format": "4.1.4",
"@vitest/utils": "4.1.4",
"magic-string": "^0.30.21",
"pathe": "^2.0.3"
},
@@ -498,9 +497,9 @@
}
},
"node_modules/@vitest/spy": {
"version": "4.1.2",
"resolved": "https://registry.npmjs.org/@vitest/spy/-/spy-4.1.2.tgz",
"integrity": "sha512-DU4fBnbVCJGNBwVA6xSToNXrkZNSiw59H8tcuUspVMsBDBST4nfvsPsEHDHGtWRRnqBERBQu7TrTKskmjqTXKA==",
"version": "4.1.4",
"resolved": "https://registry.npmjs.org/@vitest/spy/-/spy-4.1.4.tgz",
"integrity": "sha512-XxNdAsKW7C+FLydqFJLb5KhJtl3PGCMmYwFRfhvIgxJvLSXhhVI1zM8f1qD3Zg7RCjTSzDVyct6sghs9UEgBEQ==",
"dev": true,
"license": "MIT",
"funding": {
@@ -508,13 +507,13 @@
}
},
"node_modules/@vitest/utils": {
"version": "4.1.2",
"resolved": "https://registry.npmjs.org/@vitest/utils/-/utils-4.1.2.tgz",
"integrity": "sha512-xw2/TiX82lQHA06cgbqRKFb5lCAy3axQ4H4SoUFhUsg+wztiet+co86IAMDtF6Vm1hc7J6j09oh/rgDn+JdKIQ==",
"version": "4.1.4",
"resolved": "https://registry.npmjs.org/@vitest/utils/-/utils-4.1.4.tgz",
"integrity": "sha512-13QMT+eysM5uVGa1rG4kegGYNp6cnQcsTc67ELFbhNLQO+vgsygtYJx2khvdt4gVQqSSpC/KT5FZZxUpP3Oatw==",
"dev": true,
"license": "MIT",
"dependencies": {
"@vitest/pretty-format": "4.1.2",
"@vitest/pretty-format": "4.1.4",
"convert-source-map": "^2.0.0",
"tinyrainbow": "^3.1.0"
},
@@ -960,9 +959,9 @@
}
},
"node_modules/postcss": {
"version": "8.5.8",
"resolved": "https://registry.npmjs.org/postcss/-/postcss-8.5.8.tgz",
"integrity": "sha512-OW/rX8O/jXnm82Ey1k44pObPtdblfiuWnrd8X7GJ7emImCOstunGbXUpp7HdBrFQX6rJzn3sPT397Wp5aCwCHg==",
"version": "8.5.10",
"resolved": "https://registry.npmjs.org/postcss/-/postcss-8.5.10.tgz",
"integrity": "sha512-pMMHxBOZKFU6HgAZ4eyGnwXF/EvPGGqUr0MnZ5+99485wwW41kW91A4LOGxSHhgugZmSChL5AlElNdwlNgcnLQ==",
"dev": true,
"funding": [
{
@@ -989,14 +988,14 @@
}
},
"node_modules/rolldown": {
"version": "1.0.0-rc.12",
"resolved": "https://registry.npmjs.org/rolldown/-/rolldown-1.0.0-rc.12.tgz",
"integrity": "sha512-yP4USLIMYrwpPHEFB5JGH1uxhcslv6/hL0OyvTuY+3qlOSJvZ7ntYnoWpehBxufkgN0cvXxppuTu5hHa/zPh+A==",
"version": "1.0.0-rc.15",
"resolved": "https://registry.npmjs.org/rolldown/-/rolldown-1.0.0-rc.15.tgz",
"integrity": "sha512-Ff31guA5zT6WjnGp0SXw76X6hzGRk/OQq2hE+1lcDe+lJdHSgnSX6nK3erbONHyCbpSj9a9E+uX/OvytZoWp2g==",
"dev": true,
"license": "MIT",
"dependencies": {
"@oxc-project/types": "=0.122.0",
"@rolldown/pluginutils": "1.0.0-rc.12"
"@oxc-project/types": "=0.124.0",
"@rolldown/pluginutils": "1.0.0-rc.15"
},
"bin": {
"rolldown": "bin/cli.mjs"
@@ -1005,21 +1004,21 @@
"node": "^20.19.0 || >=22.12.0"
},
"optionalDependencies": {
"@rolldown/binding-android-arm64": "1.0.0-rc.12",
"@rolldown/binding-darwin-arm64": "1.0.0-rc.12",
"@rolldown/binding-darwin-x64": "1.0.0-rc.12",
"@rolldown/binding-freebsd-x64": "1.0.0-rc.12",
"@rolldown/binding-linux-arm-gnueabihf": "1.0.0-rc.12",
"@rolldown/binding-linux-arm64-gnu": "1.0.0-rc.12",
"@rolldown/binding-linux-arm64-musl": "1.0.0-rc.12",
"@rolldown/binding-linux-ppc64-gnu": "1.0.0-rc.12",
"@rolldown/binding-linux-s390x-gnu": "1.0.0-rc.12",
"@rolldown/binding-linux-x64-gnu": "1.0.0-rc.12",
"@rolldown/binding-linux-x64-musl": "1.0.0-rc.12",
"@rolldown/binding-openharmony-arm64": "1.0.0-rc.12",
"@rolldown/binding-wasm32-wasi": "1.0.0-rc.12",
"@rolldown/binding-win32-arm64-msvc": "1.0.0-rc.12",
"@rolldown/binding-win32-x64-msvc": "1.0.0-rc.12"
"@rolldown/binding-android-arm64": "1.0.0-rc.15",
"@rolldown/binding-darwin-arm64": "1.0.0-rc.15",
"@rolldown/binding-darwin-x64": "1.0.0-rc.15",
"@rolldown/binding-freebsd-x64": "1.0.0-rc.15",
"@rolldown/binding-linux-arm-gnueabihf": "1.0.0-rc.15",
"@rolldown/binding-linux-arm64-gnu": "1.0.0-rc.15",
"@rolldown/binding-linux-arm64-musl": "1.0.0-rc.15",
"@rolldown/binding-linux-ppc64-gnu": "1.0.0-rc.15",
"@rolldown/binding-linux-s390x-gnu": "1.0.0-rc.15",
"@rolldown/binding-linux-x64-gnu": "1.0.0-rc.15",
"@rolldown/binding-linux-x64-musl": "1.0.0-rc.15",
"@rolldown/binding-openharmony-arm64": "1.0.0-rc.15",
"@rolldown/binding-wasm32-wasi": "1.0.0-rc.15",
"@rolldown/binding-win32-arm64-msvc": "1.0.0-rc.15",
"@rolldown/binding-win32-x64-msvc": "1.0.0-rc.15"
}
},
"node_modules/siginfo": {
@@ -1047,9 +1046,9 @@
"license": "MIT"
},
"node_modules/std-env": {
"version": "4.0.0",
"resolved": "https://registry.npmjs.org/std-env/-/std-env-4.0.0.tgz",
"integrity": "sha512-zUMPtQ/HBY3/50VbpkupYHbRroTRZJPRLvreamgErJVys0ceuzMkD44J/QjqhHjOzK42GQ3QZIeFG1OYfOtKqQ==",
"version": "4.1.0",
"resolved": "https://registry.npmjs.org/std-env/-/std-env-4.1.0.tgz",
"integrity": "sha512-Rq7ybcX2RuC55r9oaPVEW7/xu3tj8u4GeBYHBWCychFtzMIr86A7e3PPEBPT37sHStKX3+TiX/Fr/ACmJLVlLQ==",
"dev": true,
"license": "MIT"
},
@@ -1061,9 +1060,9 @@
"license": "MIT"
},
"node_modules/tinyexec": {
"version": "1.0.4",
"resolved": "https://registry.npmjs.org/tinyexec/-/tinyexec-1.0.4.tgz",
"integrity": "sha512-u9r3uZC0bdpGOXtlxUIdwf9pkmvhqJdrVCH9fapQtgy/OeTTMZ1nqH7agtvEfmGui6e1XxjcdrlxvxJvc3sMqw==",
"version": "1.1.1",
"resolved": "https://registry.npmjs.org/tinyexec/-/tinyexec-1.1.1.tgz",
"integrity": "sha512-VKS/ZaQhhkKFMANmAOhhXVoIfBXblQxGX1myCQ2faQrfmobMftXeJPcZGp0gS07ocvGJWDLZGyOZDadDBqYIJg==",
"dev": true,
"license": "MIT",
"engines": {
@@ -1071,14 +1070,14 @@
}
},
"node_modules/tinyglobby": {
"version": "0.2.15",
"resolved": "https://registry.npmjs.org/tinyglobby/-/tinyglobby-0.2.15.tgz",
"integrity": "sha512-j2Zq4NyQYG5XMST4cbs02Ak8iJUdxRM0XI5QyxXuZOzKOINmWurp3smXu3y5wDcJrptwpSjgXHzIQxR0omXljQ==",
"version": "0.2.16",
"resolved": "https://registry.npmjs.org/tinyglobby/-/tinyglobby-0.2.16.tgz",
"integrity": "sha512-pn99VhoACYR8nFHhxqix+uvsbXineAasWm5ojXoN8xEwK5Kd3/TrhNn1wByuD52UxWRLy8pu+kRMniEi6Eq9Zg==",
"dev": true,
"license": "MIT",
"dependencies": {
"fdir": "^6.5.0",
"picomatch": "^4.0.3"
"picomatch": "^4.0.4"
},
"engines": {
"node": ">=12.0.0"
@@ -1106,9 +1105,9 @@
"optional": true
},
"node_modules/typescript": {
"version": "6.0.2",
"resolved": "https://registry.npmjs.org/typescript/-/typescript-6.0.2.tgz",
"integrity": "sha512-bGdAIrZ0wiGDo5l8c++HWtbaNCWTS4UTv7RaTH/ThVIgjkveJt83m74bBHMJkuCbslY8ixgLBVZJIOiQlQTjfQ==",
"version": "6.0.3",
"resolved": "https://registry.npmjs.org/typescript/-/typescript-6.0.3.tgz",
"integrity": "sha512-y2TvuxSZPDyQakkFRPZHKFm+KKVqIisdg9/CZwm9ftvKXLP8NRWj38/ODjNbr43SsoXqNuAisEf1GdCxqWcdBw==",
"dev": true,
"license": "Apache-2.0",
"bin": {
@@ -1120,16 +1119,16 @@
}
},
"node_modules/vite": {
"version": "8.0.3",
"resolved": "https://registry.npmjs.org/vite/-/vite-8.0.3.tgz",
"integrity": "sha512-B9ifbFudT1TFhfltfaIPgjo9Z3mDynBTJSUYxTjOQruf/zHH+ezCQKcoqO+h7a9Pw9Nm/OtlXAiGT1axBgwqrQ==",
"version": "8.0.8",
"resolved": "https://registry.npmjs.org/vite/-/vite-8.0.8.tgz",
"integrity": "sha512-dbU7/iLVa8KZALJyLOBOQ88nOXtNG8vxKuOT4I2mD+Ya70KPceF4IAmDsmU0h1Qsn5bPrvsY9HJstCRh3hG6Uw==",
"dev": true,
"license": "MIT",
"dependencies": {
"lightningcss": "^1.32.0",
"picomatch": "^4.0.4",
"postcss": "^8.5.8",
"rolldown": "1.0.0-rc.12",
"rolldown": "1.0.0-rc.15",
"tinyglobby": "^0.2.15"
},
"bin": {
@@ -1147,7 +1146,7 @@
"peerDependencies": {
"@types/node": "^20.19.0 || >=22.12.0",
"@vitejs/devtools": "^0.1.0",
"esbuild": "^0.27.0",
"esbuild": "^0.27.0 || ^0.28.0",
"jiti": ">=1.21.0",
"less": "^4.0.0",
"sass": "^1.70.0",
@@ -1198,19 +1197,19 @@
}
},
"node_modules/vitest": {
"version": "4.1.2",
"resolved": "https://registry.npmjs.org/vitest/-/vitest-4.1.2.tgz",
"integrity": "sha512-xjR1dMTVHlFLh98JE3i/f/WePqJsah4A0FK9cc8Ehp9Udk0AZk6ccpIZhh1qJ/yxVWRZ+Q54ocnD8TXmkhspGg==",
"version": "4.1.4",
"resolved": "https://registry.npmjs.org/vitest/-/vitest-4.1.4.tgz",
"integrity": "sha512-tFuJqTxKb8AvfyqMfnavXdzfy3h3sWZRWwfluGbkeR7n0HUev+FmNgZ8SDrRBTVrVCjgH5cA21qGbCffMNtWvg==",
"dev": true,
"license": "MIT",
"dependencies": {
"@vitest/expect": "4.1.2",
"@vitest/mocker": "4.1.2",
"@vitest/pretty-format": "4.1.2",
"@vitest/runner": "4.1.2",
"@vitest/snapshot": "4.1.2",
"@vitest/spy": "4.1.2",
"@vitest/utils": "4.1.2",
"@vitest/expect": "4.1.4",
"@vitest/mocker": "4.1.4",
"@vitest/pretty-format": "4.1.4",
"@vitest/runner": "4.1.4",
"@vitest/snapshot": "4.1.4",
"@vitest/spy": "4.1.4",
"@vitest/utils": "4.1.4",
"es-module-lexer": "^2.0.0",
"expect-type": "^1.3.0",
"magic-string": "^0.30.21",
@@ -1238,10 +1237,12 @@
"@edge-runtime/vm": "*",
"@opentelemetry/api": "^1.9.0",
"@types/node": "^20.0.0 || ^22.0.0 || >=24.0.0",
"@vitest/browser-playwright": "4.1.2",
"@vitest/browser-preview": "4.1.2",
"@vitest/browser-webdriverio": "4.1.2",
"@vitest/ui": "4.1.2",
"@vitest/browser-playwright": "4.1.4",
"@vitest/browser-preview": "4.1.4",
"@vitest/browser-webdriverio": "4.1.4",
"@vitest/coverage-istanbul": "4.1.4",
"@vitest/coverage-v8": "4.1.4",
"@vitest/ui": "4.1.4",
"happy-dom": "*",
"jsdom": "*",
"vite": "^6.0.0 || ^7.0.0 || ^8.0.0"
@@ -1265,6 +1266,12 @@
"@vitest/browser-webdriverio": {
"optional": true
},
"@vitest/coverage-istanbul": {
"optional": true
},
"@vitest/coverage-v8": {
"optional": true
},
"@vitest/ui": {
"optional": true
},
+69 -16
View File
@@ -29,6 +29,12 @@ export interface ClientOptions {
export interface RequestOptions {
json?: object;
params?: Record<string, string | number>;
/**
* When set, send as multipart form-data with this body. The runtime's
* fetch sets the Content-Type + boundary itself, so we deliberately do
* not include a Content-Type header in this case.
*/
form?: FormData;
}
export class BaseClient {
@@ -47,36 +53,34 @@ export class BaseClient {
path: string,
options?: RequestOptions,
): Promise<T> {
const headers: Record<string, string> = {
"Content-Type": "application/json",
};
const headers: Record<string, string> = {};
if (!options?.form) {
headers["Content-Type"] = "application/json";
}
if (this.token) {
headers["Authorization"] = `Bearer ${this.token}`;
}
let url = `${this.baseUrl}${path}`;
if (options?.params) {
const searchParams = new URLSearchParams();
for (const [key, value] of Object.entries(options.params)) {
if (value !== undefined && value !== "") {
searchParams.set(key, String(value));
}
}
const qs = searchParams.toString();
if (qs) url += `?${qs}`;
const url = this._buildUrl(path, options?.params);
let body: BodyInit | undefined;
if (options?.form) {
body = options.form;
} else if (options?.json) {
body = JSON.stringify(options.json);
}
const resp = await this.fetchFn(url, {
method,
headers,
body: options?.json ? JSON.stringify(options.json) : undefined,
body,
});
if (!resp.ok) {
let msg = "";
try {
const body = (await resp.json()) as Record<string, unknown>;
msg = (body.error as string) ?? (body.detail as string) ?? "";
const errBody = (await resp.json()) as Record<string, unknown>;
msg = (errBody.error as string) ?? (errBody.detail as string) ?? "";
} catch {
msg = await resp.text().catch(() => "");
}
@@ -86,6 +90,55 @@ export class BaseClient {
return (await resp.json()) as T;
}
protected async requestBytes(
method: string,
path: string,
options?: { params?: Record<string, string | number> },
): Promise<{ bytes: Uint8Array; contentType: string; filename: string }> {
const headers: Record<string, string> = {};
if (this.token) {
headers["Authorization"] = `Bearer ${this.token}`;
}
const url = this._buildUrl(path, options?.params);
const resp = await this.fetchFn(url, { method, headers });
if (!resp.ok) {
let msg = "";
try {
const errBody = (await resp.json()) as Record<string, unknown>;
msg = (errBody.error as string) ?? (errBody.detail as string) ?? "";
} catch {
msg = await resp.text().catch(() => "");
}
throw new TurnstoneAPIError(resp.status, msg || `HTTP ${resp.status}`);
}
const contentType =
resp.headers.get("content-type") ?? "application/octet-stream";
const disposition = resp.headers.get("content-disposition") ?? "";
const match = /filename="?([^";]+)"?/.exec(disposition);
const filename = match ? match[1] : "";
const buf = await resp.arrayBuffer();
return { bytes: new Uint8Array(buf), contentType, filename };
}
private _buildUrl(
path: string,
params?: Record<string, string | number>,
): string {
let url = `${this.baseUrl}${path}`;
if (params) {
const searchParams = new URLSearchParams();
for (const [key, value] of Object.entries(params)) {
if (value !== undefined && value !== "") {
searchParams.set(key, String(value));
}
}
const qs = searchParams.toString();
if (qs) url += `?${qs}`;
}
return url;
}
protected async *streamSSE<T = Record<string, unknown>>(
path: string,
params?: Record<string, string | number>,
+116
View File
@@ -4,6 +4,8 @@ import type {
AdminListMemoriesOptions,
AdminMemoryInfo,
AdminSearchMemoriesOptions,
AttachmentContent,
AttachmentUpload,
AuditQueryOptions,
AuditResponse,
AuthLoginResponse,
@@ -16,6 +18,9 @@ import type {
ConsoleCreateWsRequest,
ConsoleCreateWsResponse,
ConsoleHealthResponse,
CreateWorkstreamRequest,
CreateWorkstreamResponse,
ListAttachmentsResponse,
CreateMcpServerRequest,
CreatePolicyOptions,
CreateRoleOptions,
@@ -55,12 +60,37 @@ import type {
UpdateScheduleRequest,
UpdateSettingOptions,
UpdateSkillRequest,
UploadAttachmentResponse,
UsageQueryOptions,
UsageResponse,
UserRoleInfo,
WorkstreamsOptions,
} from "./types.js";
function generateConsoleWsId(): string {
// 16 bytes => 32 hex chars; matches `secrets.token_hex(16)` server-side.
const buf = new Uint8Array(16);
crypto.getRandomValues(buf);
return Array.from(buf, (b) => b.toString(16).padStart(2, "0")).join("");
}
function consoleAttachmentToBlob(att: AttachmentUpload): Blob {
if (att.data instanceof Blob) {
return att.mimeType
? new Blob([att.data], { type: att.mimeType })
: att.data;
}
// Copy bytes into a fresh ArrayBuffer-backed Uint8Array. The Blob
// BlobPart type rejects ArrayBufferLike views (could be backed by
// SharedArrayBuffer); a freshly allocated buffer is plainly ArrayBuffer.
const src = att.data;
const fresh = new Uint8Array(new ArrayBuffer(src.byteLength));
fresh.set(src);
return new Blob([fresh], {
type: att.mimeType ?? "application/octet-stream",
});
}
/** Async client for the turnstone console API. */
export class TurnstoneConsole extends BaseClient {
constructor(options: ClientOptions) {
@@ -113,6 +143,92 @@ export class TurnstoneConsole extends BaseClient {
});
}
// -- Routing proxy --------------------------------------------------------
/**
* Create a workstream via the console rendezvous router.
*
* When `attachments` is non-empty the request is sent as
* multipart/form-data and the console routes via `?ws_id=<hex>`
* (auto-generated when not supplied) so the body lands on the
* owning node directly.
*/
async routeCreateWorkstream(
opts?: CreateWorkstreamRequest & { target_node?: string },
): Promise<
CreateWorkstreamResponse & { node_url?: string; node_id?: string }
> {
const attachments = opts?.attachments;
if (attachments && attachments.length > 0) {
// The console's multipart route_create routes by `?ws_id=` only —
// it does not parse the body to honor `target_node`. Refuse the
// combination at the SDK boundary so callers don't silently get
// routed to the wrong node.
if (opts?.target_node) {
throw new Error(
"target_node is not supported with attachments; " +
"use ws_id (caller-generated to hash to the desired node) instead",
);
}
const meta: Record<string, unknown> = { ...opts };
delete (meta as { attachments?: unknown }).attachments;
let wsId = (meta.ws_id as string | undefined) ?? "";
if (!wsId) {
wsId = generateConsoleWsId();
meta.ws_id = wsId;
}
const form = new FormData();
form.append("meta", JSON.stringify(meta));
for (const att of attachments) {
form.append("file", consoleAttachmentToBlob(att), att.filename);
}
return this.request("POST", "/v1/api/route/workstreams/new", {
form,
params: { ws_id: wsId },
});
}
return this.request("POST", "/v1/api/route/workstreams/new", {
json: opts ?? {},
});
}
async routeUploadAttachment(
wsId: string,
file: AttachmentUpload,
): Promise<UploadAttachmentResponse> {
const form = new FormData();
form.append("file", consoleAttachmentToBlob(file), file.filename);
return this.request(
"POST",
`/v1/api/route/workstreams/${wsId}/attachments`,
{ form },
);
}
async routeListAttachments(wsId: string): Promise<ListAttachmentsResponse> {
return this.request("GET", `/v1/api/route/workstreams/${wsId}/attachments`);
}
async routeGetAttachmentContent(
wsId: string,
attachmentId: string,
): Promise<AttachmentContent> {
return this.requestBytes(
"GET",
`/v1/api/route/workstreams/${wsId}/attachments/${attachmentId}/content`,
);
}
async routeDeleteAttachment(
wsId: string,
attachmentId: string,
): Promise<StatusResponse> {
return this.request(
"DELETE",
`/v1/api/route/workstreams/${wsId}/attachments/${attachmentId}`,
);
}
// -- Streaming ------------------------------------------------------------
async *clusterEvents(): AsyncIterableIterator<ClusterEvent> {
+10
View File
@@ -92,6 +92,11 @@ export interface PlanReviewEvent {
content: string;
}
export interface PlanResolvedEvent {
type: "plan_resolved";
feedback: string;
}
export interface InfoEvent {
type: "info";
message: string;
@@ -165,6 +170,7 @@ export type ServerEvent =
| ToolOutputChunkEvent
| StatusEvent
| PlanReviewEvent
| PlanResolvedEvent
| InfoEvent
| ErrorEvent
| BusyErrorEvent
@@ -283,6 +289,10 @@ export function isPlanReviewEvent(e: ServerEvent): e is PlanReviewEvent {
return e.type === "plan_review";
}
export function isPlanResolvedEvent(e: ServerEvent): e is PlanResolvedEvent {
return e.type === "plan_resolved";
}
export function isCancelledEvent(e: ServerEvent): e is CancelledEvent {
return e.type === "cancelled";
}
+6
View File
@@ -183,6 +183,12 @@ export type {
SkillInstallRequest,
SkillInstallResponse,
SkillInstallSkipped,
// Attachment types
AttachmentUpload,
AttachmentInfo,
UploadAttachmentResponse,
ListAttachmentsResponse,
AttachmentContent,
} from "./types.js";
// SSE parser (for advanced usage)
+96 -5
View File
@@ -1,6 +1,8 @@
import { BaseClient, type ClientOptions } from "./base.js";
import type { ServerEvent } from "./events.js";
import type {
AttachmentContent,
AttachmentUpload,
AuthLoginResponse,
AuthSetupResponse,
AuthStatusResponse,
@@ -9,20 +11,46 @@ import type {
DashboardResponse,
DeleteMemoryOptions,
HealthResponse,
ListAttachmentsResponse,
ListMemoriesOptions,
ListMemoriesResponse,
ListSavedWorkstreamsResponse,
SkillSummary,
ListWorkstreamsResponse,
MemoryInfo,
SaveMemoryRequest,
SearchMemoriesRequest,
SendAndWaitOptions,
SendResponse,
SkillSummary,
StatusResponse,
TurnResult,
UploadAttachmentResponse,
} from "./types.js";
function generateWsId(): string {
// 16 bytes => 32 hex chars; matches `secrets.token_hex(16)` server-side.
const buf = new Uint8Array(16);
crypto.getRandomValues(buf);
return Array.from(buf, (b) => b.toString(16).padStart(2, "0")).join("");
}
function attachmentToBlob(att: AttachmentUpload): Blob {
if (att.data instanceof Blob) {
return att.mimeType
? new Blob([att.data], { type: att.mimeType })
: att.data;
}
// Copy bytes into a fresh ArrayBuffer-backed Uint8Array. The Blob
// BlobPart type rejects ArrayBufferLike views (could be backed by
// SharedArrayBuffer); a freshly allocated buffer is plainly ArrayBuffer.
const src = att.data;
const fresh = new Uint8Array(new ArrayBuffer(src.byteLength));
fresh.set(src);
return new Blob([fresh], {
type: att.mimeType ?? "application/octet-stream",
});
}
/** Async client for the turnstone server API. */
export class TurnstoneServer extends BaseClient {
constructor(options: ClientOptions) {
@@ -42,7 +70,27 @@ export class TurnstoneServer extends BaseClient {
async createWorkstream(
opts?: CreateWorkstreamRequest,
): Promise<CreateWorkstreamResponse> {
return this.request("POST", "/v1/api/workstreams/new", { json: opts });
const attachments = opts?.attachments;
if (attachments && attachments.length > 0) {
// Multipart variant: pre-generate ws_id so cluster routers can
// hash to the owning node before this body lands. Server accepts
// either a server-generated id (when meta.ws_id is empty) or the
// caller-supplied one.
const meta: Record<string, unknown> = { ...opts };
delete (meta as { attachments?: unknown }).attachments;
if (!meta.ws_id) {
meta.ws_id = generateWsId();
}
const form = new FormData();
form.append("meta", JSON.stringify(meta));
for (const att of attachments) {
form.append("file", attachmentToBlob(att), att.filename);
}
return this.request("POST", "/v1/api/workstreams/new", { form });
}
return this.request("POST", "/v1/api/workstreams/new", {
json: opts ?? {},
});
}
async closeWorkstream(wsId: string): Promise<StatusResponse> {
@@ -53,12 +101,55 @@ export class TurnstoneServer extends BaseClient {
// -- Chat interaction -----------------------------------------------------
async send(message: string, wsId: string): Promise<SendResponse> {
return this.request("POST", "/v1/api/send", {
json: { message, ws_id: wsId },
async send(
message: string,
wsId: string,
opts?: { attachmentIds?: string[] },
): Promise<SendResponse> {
const body: Record<string, unknown> = { message, ws_id: wsId };
if (opts?.attachmentIds !== undefined) {
body.attachment_ids = opts.attachmentIds;
}
return this.request("POST", "/v1/api/send", { json: body });
}
// -- Attachments ----------------------------------------------------------
async uploadAttachment(
wsId: string,
file: AttachmentUpload,
): Promise<UploadAttachmentResponse> {
const form = new FormData();
form.append("file", attachmentToBlob(file), file.filename);
return this.request("POST", `/v1/api/workstreams/${wsId}/attachments`, {
form,
});
}
async listAttachments(wsId: string): Promise<ListAttachmentsResponse> {
return this.request("GET", `/v1/api/workstreams/${wsId}/attachments`);
}
async getAttachmentContent(
wsId: string,
attachmentId: string,
): Promise<AttachmentContent> {
return this.requestBytes(
"GET",
`/v1/api/workstreams/${wsId}/attachments/${attachmentId}/content`,
);
}
async deleteAttachment(
wsId: string,
attachmentId: string,
): Promise<StatusResponse> {
return this.request(
"DELETE",
`/v1/api/workstreams/${wsId}/attachments/${attachmentId}`,
);
}
async approve(opts: {
wsId: string;
approved?: boolean;
+74 -2
View File
@@ -50,10 +50,67 @@ export interface AuthSetupResponse {
export interface SendRequest {
message: string;
ws_id: string;
/**
* Explicit list of pending attachment ids to inject into this turn.
* When omitted, any pending attachments for the caller on the
* workstream are auto-consumed; an empty list disables auto-consume.
*/
attachment_ids?: string[];
}
export interface SendResponse {
/** "ok" | "busy" | "queued" | "queue_full". */
status: string;
/**
* Attachment ids actually reserved onto this turn. Subset of the
* request's `attachment_ids` (or the auto-consumed pending set).
*/
attached_ids?: string[];
/**
* Attachment ids the caller requested that the server could not
* reserve (lost a race, already consumed, or cross-scope). The
* request still proceeds with whatever was reserved.
*/
dropped_attachment_ids?: string[];
/** Set on "queued" responses: relative priority of the queued message. */
priority?: string | null;
/** Set on "queued" responses: id used to dequeue the message. */
msg_id?: string | null;
}
// ---------------------------------------------------------------------------
// Server API — Attachments
// ---------------------------------------------------------------------------
/** A file to upload as an attachment. */
export interface AttachmentUpload {
filename: string;
/** Raw file bytes; use a `Blob` in browsers and a `Uint8Array` in Node. */
data: Blob | Uint8Array;
/** Optional advisory MIME type; the server applies its own validation. */
mimeType?: string;
}
export interface AttachmentInfo {
attachment_id: string;
filename: string;
mime_type: string;
size_bytes: number;
/** "image" or "text". */
kind: string;
}
export type UploadAttachmentResponse = AttachmentInfo;
export interface ListAttachmentsResponse {
attachments: AttachmentInfo[];
}
/** Raw bytes returned from the attachment `/content` endpoint. */
export interface AttachmentContent {
bytes: Uint8Array;
contentType: string;
filename: string;
}
export interface ApproveRequest {
@@ -79,6 +136,20 @@ export interface CreateWorkstreamRequest {
auto_approve?: boolean;
resume_ws?: string;
skill?: string;
/** First user message dispatched in a background worker after creation. */
initial_message?: string;
/**
* Caller-supplied workstream id (32-hex). Auto-generated when omitted.
* Required for cluster-routed multipart creates so the console can
* hash to the owning node before the body lands.
*/
ws_id?: string;
/**
* Files to attach to the first turn. When non-empty the request is
* sent as multipart/form-data and (with `initial_message`) reserved
* onto that turn before the worker dispatches.
*/
attachments?: AttachmentUpload[];
}
export interface CreateWorkstreamResponse {
@@ -86,6 +157,8 @@ export interface CreateWorkstreamResponse {
name: string;
resumed?: boolean;
message_count?: number;
/** Ids of attachments saved by this request (multipart variant only). */
attachment_ids?: string[];
}
export interface CloseWorkstreamRequest {
@@ -284,7 +357,6 @@ export interface CreateSkillResourceRequest {
export interface BackendStatus {
status: string;
circuit_state: string;
}
export interface WorkstreamCounts {
@@ -880,7 +952,7 @@ export interface SkillDiscoverListing {
install_count: number;
tags: string[];
installed: boolean;
scan_status?: string;
risk_level?: string;
template_id?: string;
}
+22
View File
@@ -62,6 +62,28 @@ describe("TurnstoneConsole", () => {
expect(url).toContain("page=2");
});
it("routeCreateWorkstream rejects attachments + target_node", async () => {
const fetchFn = vi.fn().mockResolvedValue(
new Response("{}", {
status: 500,
headers: { "content-type": "application/json" },
}),
);
const client = new TurnstoneConsole({
baseUrl: "http://test",
fetch: fetchFn,
});
const data = new TextEncoder().encode("hi");
await expect(
client.routeCreateWorkstream({
name: "x",
target_node: "n1",
attachments: [{ filename: "a.txt", data }],
}),
).rejects.toThrow(/target_node/);
expect(fetchFn).not.toHaveBeenCalled();
});
it("health returns parsed response", async () => {
const fetchFn = mockFetch({
status: "ok",
+6
View File
@@ -8,6 +8,7 @@ import {
isApproveRequestEvent,
isApprovalResolvedEvent,
isPlanReviewEvent,
isPlanResolvedEvent,
isReasoningEvent,
} from "../src/events.js";
import type { ServerEvent } from "../src/events.js";
@@ -76,4 +77,9 @@ describe("event type guards", () => {
const e: ServerEvent = { type: "plan_review", content: "## Plan" };
expect(isPlanReviewEvent(e)).toBe(true);
});
it("isPlanResolvedEvent", () => {
const e: ServerEvent = { type: "plan_resolved", feedback: "approved" };
expect(isPlanResolvedEvent(e)).toBe(true);
});
});
@@ -0,0 +1,168 @@
import { describe, expect, it, vi } from "vitest";
import { TurnstoneServer } from "../src/server.js";
function mockFetch(response: object, status = 200): typeof globalThis.fetch {
return vi.fn().mockResolvedValue(
new Response(JSON.stringify(response), {
status,
headers: { "content-type": "application/json" },
}),
);
}
function mockFetchBytes(
body: Uint8Array,
contentType: string,
filename = "",
): typeof globalThis.fetch {
const headers: Record<string, string> = { "content-type": contentType };
if (filename)
headers["content-disposition"] = `inline; filename="${filename}"`;
return vi
.fn()
.mockResolvedValue(new Response(body, { status: 200, headers }));
}
describe("TurnstoneServer attachments", () => {
it("uploadAttachment sends multipart with filename", async () => {
const fetchFn = mockFetch({
attachment_id: "att-1",
filename: "a.txt",
mime_type: "text/plain",
size_bytes: 5,
kind: "text",
});
const client = new TurnstoneServer({
baseUrl: "http://test",
fetch: fetchFn,
});
const data = new TextEncoder().encode("hello");
const result = await client.uploadAttachment("ws-X", {
filename: "a.txt",
data,
mimeType: "text/plain",
});
expect(result.attachment_id).toBe("att-1");
const [url, init] = (fetchFn as ReturnType<typeof vi.fn>).mock.calls[0];
expect(url).toBe("http://test/v1/api/workstreams/ws-X/attachments");
expect(init.method).toBe("POST");
expect(init.body).toBeInstanceOf(FormData);
// Browser/Node fetch sets the Content-Type header from FormData itself
expect(init.headers["Content-Type"]).toBeUndefined();
});
it("listAttachments hits the GET endpoint", async () => {
const fetchFn = mockFetch({ attachments: [] });
const client = new TurnstoneServer({
baseUrl: "http://test",
fetch: fetchFn,
});
const resp = await client.listAttachments("ws-X");
expect(resp.attachments).toEqual([]);
const [url, init] = (fetchFn as ReturnType<typeof vi.fn>).mock.calls[0];
expect(url).toBe("http://test/v1/api/workstreams/ws-X/attachments");
expect(init.method).toBe("GET");
});
it("getAttachmentContent returns raw bytes + parsed headers", async () => {
const bytes = new TextEncoder().encode("hello world");
const fetchFn = mockFetchBytes(
bytes,
"text/plain; charset=utf-8",
"notes.md",
);
const client = new TurnstoneServer({
baseUrl: "http://test",
fetch: fetchFn,
});
const result = await client.getAttachmentContent("ws-X", "att-1");
expect(new TextDecoder().decode(result.bytes)).toBe("hello world");
expect(result.contentType).toBe("text/plain; charset=utf-8");
expect(result.filename).toBe("notes.md");
});
it("deleteAttachment hits the DELETE endpoint", async () => {
const fetchFn = mockFetch({ status: "deleted" });
const client = new TurnstoneServer({
baseUrl: "http://test",
fetch: fetchFn,
});
const resp = await client.deleteAttachment("ws-X", "att-1");
expect(resp.status).toBe("deleted");
const [, init] = (fetchFn as ReturnType<typeof vi.fn>).mock.calls[0];
expect(init.method).toBe("DELETE");
});
it("send threads attachment_ids when provided", async () => {
const fetchFn = mockFetch({ status: "ok" });
const client = new TurnstoneServer({
baseUrl: "http://test",
fetch: fetchFn,
});
await client.send("hi", "ws-X", { attachmentIds: ["a1", "a2"] });
const [, init] = (fetchFn as ReturnType<typeof vi.fn>).mock.calls[0];
expect(JSON.parse(init.body)).toEqual({
message: "hi",
ws_id: "ws-X",
attachment_ids: ["a1", "a2"],
});
});
it("send omits attachment_ids when not supplied", async () => {
const fetchFn = mockFetch({ status: "ok" });
const client = new TurnstoneServer({
baseUrl: "http://test",
fetch: fetchFn,
});
await client.send("hi", "ws-X");
const [, init] = (fetchFn as ReturnType<typeof vi.fn>).mock.calls[0];
expect(JSON.parse(init.body)).toEqual({ message: "hi", ws_id: "ws-X" });
});
it("createWorkstream with attachments sends multipart and auto-generates ws_id", async () => {
const fetchFn = mockFetch({
ws_id: "00ff00000000000000000000000000ff",
name: "demo",
attachment_ids: ["att-1"],
});
const client = new TurnstoneServer({
baseUrl: "http://test",
fetch: fetchFn,
});
const data = new TextEncoder().encode("hello");
const resp = await client.createWorkstream({
name: "demo",
initial_message: "describe",
attachments: [{ filename: "a.txt", data, mimeType: "text/plain" }],
});
expect(resp.attachment_ids).toEqual(["att-1"]);
const [url, init] = (fetchFn as ReturnType<typeof vi.fn>).mock.calls[0];
expect(url).toBe("http://test/v1/api/workstreams/new");
expect(init.method).toBe("POST");
expect(init.body).toBeInstanceOf(FormData);
const form = init.body as FormData;
const meta = JSON.parse(form.get("meta") as string);
expect(meta.name).toBe("demo");
expect(meta.initial_message).toBe("describe");
expect(meta.ws_id).toMatch(/^[0-9a-f]{32}$/);
expect(meta.attachments).toBeUndefined();
const file = form.get("file");
expect(file).toBeInstanceOf(Blob);
});
it("createWorkstream without attachments uses JSON body", async () => {
const fetchFn = mockFetch({ ws_id: "ws-json", name: "j" });
const client = new TurnstoneServer({
baseUrl: "http://test",
fetch: fetchFn,
});
await client.createWorkstream({ name: "j" });
const [, init] = (fetchFn as ReturnType<typeof vi.fn>).mock.calls[0];
expect(init.headers["Content-Type"]).toBe("application/json");
expect(JSON.parse(init.body)).toEqual({ name: "j" });
});
});
+75
View File
@@ -0,0 +1,75 @@
"""Shared builders for the coordinator-endpoint test files.
The four coordinator test modules each ship a copy of the same
``_AuthMiddleware`` / ``_FakeConfigStore`` / ``_fake_registry`` /
``_build_mgr`` helpers this module is the single home for them so
future edits land once. Named with a leading underscore so pytest
does not collect it.
``_make_client`` stays local to each test module because the route
list differs per file.
"""
from __future__ import annotations
from typing import Any
from unittest.mock import MagicMock
from starlette.middleware.base import BaseHTTPMiddleware
from turnstone.console.coordinator import CoordinatorManager
from turnstone.console.coordinator_ui import ConsoleCoordinatorUI
from turnstone.core.auth import AuthResult
class _AuthMiddleware(BaseHTTPMiddleware):
"""Inject a configurable AuthResult from a header-based contract.
Tests set ``X-Test-Perms`` to a comma-separated permission list, and
``X-Test-User`` to the user id. Empty or missing no auth.
"""
async def dispatch(self, request, call_next): # type: ignore[no-untyped-def]
perms = request.headers.get("X-Test-Perms", "")
user_id = request.headers.get("X-Test-User", "")
if perms or user_id:
request.state.auth_result = AuthResult(
user_id=user_id,
scopes=frozenset({"approve"}),
token_source="test",
permissions=frozenset(p for p in perms.split(",") if p),
)
return await call_next(request)
class _FakeConfigStore:
"""Minimal ConfigStore stub — returns values from a dict."""
def __init__(self, values: dict[str, Any]) -> None:
self._values = values
def get(self, key: str, default: Any = None) -> Any:
return self._values.get(key, default)
def _fake_registry() -> MagicMock:
"""MagicMock whose ``.resolve()`` succeeds so the 503 gate passes."""
reg = MagicMock()
reg.resolve.return_value = (MagicMock(), "gpt-4", MagicMock())
return reg
def _build_mgr(storage: Any) -> CoordinatorManager:
"""Build a CoordinatorManager with stub factories (test default)."""
def _sf(ui, model_alias=None, ws_id=None, **kw): # type: ignore[no-untyped-def]
s = MagicMock()
s.send.return_value = None
return s
return CoordinatorManager(
session_factory=_sf,
ui_factory=lambda w, u: ConsoleCoordinatorUI(ws_id=w, user_id=u),
storage=storage,
max_active=3,
)
+152
View File
@@ -56,3 +56,155 @@ def test_record_audit_generates_unique_ids(storage):
events = storage.list_audit_events()
assert len(events) == 2
assert events[0]["event_id"] != events[1]["event_id"]
# ---------------------------------------------------------------------------
# Credential redaction at the audit boundary
# ---------------------------------------------------------------------------
def test_record_audit_redacts_passwords_by_default(storage):
"""Detail strings go through redact_credentials by default."""
record_audit(
storage,
"u1",
"coordinator.spawn",
detail={
"initial_message": "connect via postgresql://alice:s3cret@db.example.com/app",
},
)
event = storage.list_audit_events()[0]
detail = json.loads(event["detail"])
# Exact redaction text comes from output_guard._redact_credentials —
# assert the token is stripped rather than the exact marker so
# this test doesn't break if the marker format evolves.
assert "s3cret" not in detail["initial_message"]
assert "REDACTED" in detail["initial_message"]
def test_record_audit_redacts_nested_strings(storage):
"""Walker descends into lists / nested dicts."""
record_audit(
storage,
"u1",
"task_list.update",
detail={
"tasks": [
{"title": "normal task"},
{"title": "pull secret from AWS_SECRET_ACCESS_KEY=AKIAEXAMPLE123"},
],
},
)
event = storage.list_audit_events()[0]
detail = json.loads(event["detail"])
assert detail["tasks"][0]["title"] == "normal task"
assert "AKIAEXAMPLE123" not in detail["tasks"][1]["title"]
def test_record_audit_raw_detail_preserves_payload(storage):
"""`raw_detail=True` bypasses the scrub — operator-originated detail only."""
secret_like = "postgresql://alice:s3cret@db.example.com/app"
record_audit(
storage,
"admin-1",
"investigation.note",
detail={"note": secret_like},
raw_detail=True,
)
event = storage.list_audit_events()[0]
detail = json.loads(event["detail"])
assert detail["note"] == secret_like
def test_record_audit_strips_control_chars(storage):
"""CR/LF/NUL/DEL and C0 controls are replaced with spaces so a
downstream exporter that prints raw detail strings can't re-surface
log-injection. Tab/newline are deliberately preserved."""
record_audit(
storage,
"u1",
"coordinator.note",
detail={
"msg": "hello\r\nInjected: bad\x00 escape \x1b[31mred\x1b[0m\x7f",
"ok_tab": "a\tb\nc",
},
)
event = storage.list_audit_events()[0]
detail = json.loads(event["detail"])
# CR / NUL / ESC / DEL scrubbed to spaces; tab + newline kept.
assert "\r" not in detail["msg"]
assert "\x00" not in detail["msg"]
assert "\x1b" not in detail["msg"]
assert "\x7f" not in detail["msg"]
assert "hello" in detail["msg"]
assert detail["ok_tab"] == "a\tb\nc"
def test_record_audit_clean_strings_roundtrip_unchanged(storage):
"""Detail strings with no credential patterns and no control chars
pass through unchanged the fast-path / scrub must not corrupt the
common case."""
clean = {"note": "hello world", "code": "import foo", "state": "ok"}
record_audit(storage, "u1", "coordinator.note", detail=clean)
event = storage.list_audit_events()[0]
assert json.loads(event["detail"]) == clean
def test_record_audit_fast_path_skips_no_string_detail(storage):
"""A detail carrying only scalars (no strings anywhere) must persist
identically exercises the ``_has_any_string`` fast path."""
record_audit(
storage,
"u1",
"coordinator.metric",
detail={"spawned": 5, "ok": True, "parent": None, "tail": [1, 2, 3]},
)
event = storage.list_audit_events()[0]
assert json.loads(event["detail"]) == {
"spawned": 5,
"ok": True,
"parent": None,
"tail": [1, 2, 3],
}
def test_record_audit_redacts_dict_keys(storage):
"""Walker descends into dict keys too — a caller using
model-controlled text as a key can't leak it verbatim."""
record_audit(
storage,
"u1",
"coordinator.note",
detail={"postgresql://alice:s3cret@db.example.com/app": True},
)
event = storage.list_audit_events()[0]
detail = json.loads(event["detail"])
assert all("s3cret" not in k for k in detail)
def test_record_audit_walks_set_and_frozenset(storage):
"""Walker handles set/frozenset values (docstring promise)."""
record_audit(
storage,
"u1",
"coordinator.note",
detail={"tags": frozenset({"ak_" + "x" * 40, "plain"})},
)
event = storage.list_audit_events()[0]
detail = json.loads(event["detail"])
# The credential-looking AK token gets scrubbed; the plain one survives.
tags = detail["tags"]
assert "plain" in tags
def test_record_audit_leaves_non_string_scalars_alone(storage):
"""Non-string scalars (int / bool / None) pass through unchanged."""
record_audit(
storage,
"u1",
"coordinator.spawn",
detail={"budget_ok": True, "spawned": 5, "parent": None},
)
event = storage.list_audit_events()[0]
detail = json.loads(event["detail"])
assert detail == {"budget_ok": True, "spawned": 5, "parent": None}
+393 -6
View File
@@ -14,6 +14,7 @@ from turnstone.core.auth import (
check_request,
create_jwt,
is_public_path,
load_jwt_secret,
make_clear_cookie,
make_set_cookie,
required_scope,
@@ -197,6 +198,35 @@ class TestRequiredScope:
"""Only POST is elevated — GET falls through to read."""
assert required_scope("GET", "/api/_internal/mcp-reload") == "read"
# Workstream sub-resource mutations (parametric paths)
def test_ws_delete_needs_write(self):
assert required_scope("POST", "/api/workstreams/abc123/delete") == "write"
def test_ws_open_needs_write(self):
assert required_scope("POST", "/api/workstreams/abc123/open") == "write"
def test_ws_refresh_title_needs_write(self):
assert required_scope("POST", "/api/workstreams/abc123/refresh-title") == "write"
def test_ws_title_needs_write(self):
assert required_scope("POST", "/api/workstreams/abc123/title") == "write"
def test_v1_ws_delete_needs_write(self):
assert required_scope("POST", "/v1/api/workstreams/abc123/delete") == "write"
def test_proxy_ws_delete_needs_write(self):
assert required_scope("POST", "/node/n1/v1/api/workstreams/abc123/delete") == "write"
def test_proxy_ws_open_needs_write(self):
assert required_scope("POST", "/node/n1/v1/api/workstreams/abc123/open") == "write"
def test_proxy_ws_title_needs_write(self):
assert required_scope("POST", "/node/n1/v1/api/workstreams/abc123/title") == "write"
def test_ws_get_is_still_read(self):
"""GET on workstream sub-resource is not elevated."""
assert required_scope("GET", "/api/workstreams/abc123/delete") == "read"
# ---------------------------------------------------------------------------
# TestExtractBearer
@@ -605,6 +635,12 @@ class TestServerAuth:
mock_ws.name = "test"
mock_ws.state = WorkstreamState.IDLE
mock_ws.session = mock_session
# Set kind / parent_ws_id / user_id explicitly so list_workstreams
# JSON-serializes them — a bare MagicMock attribute returns another
# MagicMock that fails json.dumps and surfaces as 500.
mock_ws.kind = "interactive"
mock_ws.parent_ws_id = None
mock_ws.user_id = "u1"
mock_mgr = MagicMock()
mock_mgr.list_all.return_value = [mock_ws]
mock_mgr.max_workstreams = 10
@@ -821,6 +857,12 @@ class TestServerLogin:
mock_ws.name = "test"
mock_ws.state = WorkstreamState.IDLE
mock_ws.session = mock_session
# Set kind / parent_ws_id / user_id explicitly so list_workstreams
# JSON-serializes them — a bare MagicMock attribute returns another
# MagicMock that fails json.dumps and surfaces as 500.
mock_ws.kind = "interactive"
mock_ws.parent_ws_id = None
mock_ws.user_id = "u1"
mock_mgr = MagicMock()
mock_mgr.list_all.return_value = [mock_ws]
mock_mgr.max_workstreams = 10
@@ -911,6 +953,163 @@ class TestServerLogin:
resp = self.test_client.get("/v1/api/workstreams")
assert resp.status_code == 401
def test_whoami_includes_exp(self):
"""whoami exposes the JWT exp so the frontend can schedule refresh."""
import time
self.test_client.post(
"/v1/api/auth/login",
json={"username": "testuser", "password": "testpass"},
)
resp = self.test_client.get("/v1/api/auth/whoami")
assert resp.status_code == 200
data = resp.json()
assert "exp" in data
# Default JWT TTL is 24h; exp should be > now and < now + 25h.
now = int(time.time())
assert now < data["exp"] < now + 25 * 3600
def test_refresh_returns_new_jwt_and_cookie(self):
"""POST /api/auth/refresh re-mints the cookie with a fresh exp."""
from turnstone.core.auth import AUTH_COOKIE
# Storage needs get_user_permissions for the refresh re-resolve path.
# Mock is shared across tests in the class — re-arm here in case a
# prior test left it default.
self.test_client.app.state.auth_storage.get_user_permissions.return_value = {
"read",
"write",
"approve",
}
login = self.test_client.post(
"/v1/api/auth/login",
json={"username": "testuser", "password": "testpass"},
)
assert login.status_code == 200
refresh = self.test_client.post("/v1/api/auth/refresh")
assert refresh.status_code == 200
body = refresh.json()
assert body["status"] == "ok"
assert body["user_id"] == "uid_test"
assert "jwt" in body
# Set-Cookie header must be present so the browser updates. Don't
# assert the new JWT differs from the original — sub-second login
# and refresh produce identical iat/exp claims and therefore an
# identical token, which is fine: the cookie still gets re-set.
cookie_hdr = refresh.headers.get("set-cookie", "")
assert AUTH_COOKIE in cookie_hdr
assert "HttpOnly" in cookie_hdr
# The refreshed cookie must keep working.
resp = self.test_client.get("/v1/api/workstreams")
assert resp.status_code == 200
def test_refresh_response_includes_exp_and_permissions(self):
"""Refresh response shape must match whoami so the frontend can
populate sessionStorage + reschedule the next refresh off the
single round-trip without a follow-up /whoami call."""
import time
self.test_client.app.state.auth_storage.get_user_permissions.return_value = {
"read",
"write",
"approve",
}
login = self.test_client.post(
"/v1/api/auth/login",
json={"username": "testuser", "password": "testpass"},
)
assert login.status_code == 200
refresh = self.test_client.post("/v1/api/auth/refresh")
assert refresh.status_code == 200
body = refresh.json()
# exp present + within the expected default JWT TTL window
assert "exp" in body, body
now = int(time.time())
assert now < body["exp"] < now + 25 * 3600, body
# permissions present + non-empty (matches the seeded role set)
assert body.get("permissions"), body
assert "write" in body["permissions"].split(",")
def test_refresh_unauthenticated_401(self):
"""Refresh requires a currently-valid cookie — no cookie → 401."""
# Clear cookies on the test client
self.test_client.cookies.clear()
resp = self.test_client.post("/v1/api/auth/refresh")
assert resp.status_code == 401
def test_refresh_storage_failure_falls_back(self):
"""Transient storage error → fall back to in-token claims, not 403.
The earlier implementation called _load_user_permissions() which
swallows exceptions and returns set(); that path was
indistinguishable from a deleted user (legitimate 403). The
handler now calls storage.get_user_permissions() directly so
DB hiccups fall through to in-token perms.
"""
# Re-arm the storage so login works first
self.test_client.app.state.auth_storage.get_user_permissions.return_value = {
"read",
"write",
"approve",
}
login = self.test_client.post(
"/v1/api/auth/login",
json={"username": "testuser", "password": "testpass"},
)
assert login.status_code == 200
# Now make storage raise on the refresh re-resolve
self.test_client.app.state.auth_storage.get_user_permissions.side_effect = RuntimeError(
"db down"
)
try:
resp = self.test_client.post("/v1/api/auth/refresh")
assert resp.status_code == 200, resp.text
body = resp.json()
# Permissions should still be present (fell back to in-token claims)
assert body.get("permissions"), body
finally:
# Restore for any subsequent tests
self.test_client.app.state.auth_storage.get_user_permissions.side_effect = None
self.test_client.app.state.auth_storage.get_user_permissions.return_value = {
"read",
"write",
"approve",
}
def test_refresh_user_with_no_perms_403(self):
"""Storage returns empty (user deleted/role-stripped) → 403.
Distinguished from the storage-failure case above because
get_user_permissions returned a value (the empty set) without
raising that's an authoritative "no roles", not a hiccup.
"""
self.test_client.app.state.auth_storage.get_user_permissions.return_value = {
"read",
"write",
"approve",
}
login = self.test_client.post(
"/v1/api/auth/login",
json={"username": "testuser", "password": "testpass"},
)
assert login.status_code == 200
self.test_client.app.state.auth_storage.get_user_permissions.return_value = set()
try:
resp = self.test_client.post("/v1/api/auth/refresh")
assert resp.status_code == 403
finally:
self.test_client.app.state.auth_storage.get_user_permissions.return_value = {
"read",
"write",
"approve",
}
class TestConsoleLogin:
"""Test login/logout cookie flow on turnstone-console."""
@@ -1098,6 +1297,56 @@ class TestJWTAudienceIssuer:
result = validate_jwt(token, self.SECRET, audience="")
assert result is not None
def test_validate_jwt_accepts_within_leeway_after_expiry(self):
"""validate_jwt has 30s leeway for clock skew across hosts/processes."""
import time
import jwt as pyjwt
from turnstone.core.auth import JWT_ISSUER, validate_jwt
# Mint a token that "expired" 10 seconds ago — still within 30s leeway.
now = int(time.time())
token = pyjwt.encode(
{
"sub": "user1",
"scopes": "read",
"src": "test",
"iss": JWT_ISSUER,
"iat": now - 100,
"exp": now - 10,
},
self.SECRET,
algorithm="HS256",
)
result = validate_jwt(token, self.SECRET, audience="")
assert result is not None
assert result.user_id == "user1"
def test_validate_jwt_rejects_past_leeway(self):
"""Tokens expired beyond the 30s leeway must still be rejected."""
import time
import jwt as pyjwt
from turnstone.core.auth import JWT_ISSUER, validate_jwt
now = int(time.time())
token = pyjwt.encode(
{
"sub": "user1",
"scopes": "read",
"src": "test",
"iss": JWT_ISSUER,
"iat": now - 200,
"exp": now - 60,
},
self.SECRET,
algorithm="HS256",
)
result = validate_jwt(token, self.SECRET, audience="")
assert result is None
def test_create_jwt_expiry_seconds(self):
import jwt as pyjwt
@@ -1145,6 +1394,132 @@ class TestJWTAudienceIssuer:
create_jwt("user1", frozenset({"read"}), "test", self.SECRET, expiry_seconds=-1)
class TestJWTVersionClaim:
SECRET = "test-secret-that-is-at-least-32-chars"
def test_create_jwt_with_version(self):
import jwt as pyjwt
from turnstone.core.auth import create_jwt
token = create_jwt("user1", frozenset({"read"}), "test", self.SECRET, version="1.2")
payload = pyjwt.decode(
token, self.SECRET, algorithms=["HS256"], options={"verify_aud": False}
)
assert payload["ver"] == "1.2"
def test_create_jwt_without_version(self):
import jwt as pyjwt
from turnstone.core.auth import create_jwt
token = create_jwt("user1", frozenset({"read"}), "test", self.SECRET)
payload = pyjwt.decode(
token, self.SECRET, algorithms=["HS256"], options={"verify_aud": False}
)
assert "ver" not in payload
def test_validate_jwt_carries_token_version(self):
from turnstone.core.auth import create_jwt, validate_jwt
token = create_jwt("user1", frozenset({"read"}), "test", self.SECRET, version="1.2")
result = validate_jwt(token, self.SECRET)
assert result is not None
assert result.user_id == "user1"
assert result.token_version == "1.2"
def test_validate_jwt_no_ver_returns_empty_token_version(self):
from turnstone.core.auth import create_jwt, validate_jwt
token = create_jwt("user1", frozenset({"read"}), "test", self.SECRET)
result = validate_jwt(token, self.SECRET)
assert result is not None
assert result.token_version == ""
def test_check_request_accepts_matching_version(self):
from turnstone.core.auth import JWT_AUD_SERVER, check_request, create_jwt
token = create_jwt(
"user1",
frozenset({"read"}),
"test",
self.SECRET,
audience=JWT_AUD_SERVER,
version="1.2",
)
allowed, _status, _msg, result = check_request(
"GET",
"/v1/api/workstreams",
f"Bearer {token}",
jwt_secret=self.SECRET,
jwt_audience=JWT_AUD_SERVER,
jwt_version="1.2",
)
assert allowed
assert result is not None
def test_check_request_accepts_no_ver_backward_compat(self):
from turnstone.core.auth import JWT_AUD_SERVER, check_request, create_jwt
# Token without ver claim should be accepted (backward compat)
token = create_jwt(
"user1",
frozenset({"read"}),
"test",
self.SECRET,
audience=JWT_AUD_SERVER,
)
allowed, _status, _msg, _result = check_request(
"GET",
"/v1/api/workstreams",
f"Bearer {token}",
jwt_secret=self.SECRET,
jwt_audience=JWT_AUD_SERVER,
jwt_version="1.2",
)
assert allowed
def test_check_request_rejects_old_version_jwt(self):
from turnstone.core.auth import JWT_AUD_SERVER, check_request, create_jwt
token = create_jwt(
"user1",
frozenset({"read"}),
"test",
self.SECRET,
audience=JWT_AUD_SERVER,
version="1.1",
)
allowed, status, msg, _result = check_request(
"GET",
"/v1/api/workstreams",
f"Bearer {token}",
jwt_secret=self.SECRET,
jwt_audience=JWT_AUD_SERVER,
jwt_version="1.2",
)
assert not allowed
assert status == 401
assert msg == "version_mismatch"
class TestVersionSlot:
def test_returns_major_minor(self):
from turnstone.core.auth import jwt_version_slot
slot = jwt_version_slot()
parts = slot.split(".")
assert len(parts) == 2
def test_strips_patch_and_prerelease(self):
from unittest.mock import patch
with patch("turnstone.__version__", "2.3.1a5"):
from turnstone.core.auth import jwt_version_slot
assert jwt_version_slot() == "2.3"
class TestServiceTokenManager:
SECRET = "test-secret-that-is-at-least-32-chars"
@@ -1224,6 +1599,22 @@ class TestServiceTokenManager:
)
assert payload["aud"] == JWT_AUD_SERVER
def test_service_token_no_version_claim(self):
import jwt as pyjwt
from turnstone.core.auth import ServiceTokenManager
mgr = ServiceTokenManager(
user_id="svc",
scopes=frozenset({"read"}),
source="test",
secret=self.SECRET,
)
payload = pyjwt.decode(
mgr.token, self.SECRET, algorithms=["HS256"], options={"verify_aud": False}
)
assert "ver" not in payload
class TestIsSecureRequest:
def test_https_scheme(self):
@@ -1249,13 +1640,11 @@ class TestIsSecureRequest:
class TestSecretStrength:
def test_short_secret_exits(self):
import turnstone.core.auth as auth_mod
old = os.environ.get("TURNSTONE_JWT_SECRET", "")
os.environ["TURNSTONE_JWT_SECRET"] = "short"
try:
with pytest.raises(SystemExit):
auth_mod.load_jwt_secret()
load_jwt_secret()
finally:
if old:
os.environ["TURNSTONE_JWT_SECRET"] = old
@@ -1263,14 +1652,12 @@ class TestSecretStrength:
os.environ.pop("TURNSTONE_JWT_SECRET", None)
def test_missing_secret_exits(self):
import turnstone.core.auth as auth_mod
with (
patch("turnstone.core.config.load_config", return_value={}),
patch.dict(os.environ, {}, clear=True),
pytest.raises(SystemExit),
):
auth_mod.load_jwt_secret()
load_jwt_secret()
class TestCorsConfigurable:
+48 -1
View File
@@ -19,6 +19,7 @@ from turnstone.bootstrap import (
_tool_generate_secret,
_tool_read_file,
_tool_validate_api_key,
_tool_write_compose,
_tool_write_file,
execute_tool,
)
@@ -103,6 +104,52 @@ class TestWriteFile:
assert (tmp_path / "changed.txt").read_text() == "new\n"
class TestWriteCompose:
def test_writes_compose_file(self, tmp_path: Path) -> None:
with patch("builtins.input", return_value="y"):
result = _tool_write_compose(tmp_path, {})
assert "written successfully" in result
assert "ghcr.io" in result
content = (tmp_path / "compose.yaml").read_text()
assert "ghcr.io/turnstonelabs/turnstone" in content
assert "TURNSTONE_IMAGE_TAG" in content
def test_user_declines(self, tmp_path: Path) -> None:
with patch("builtins.input", return_value="n"):
result = _tool_write_compose(tmp_path, {})
assert "declined" in result
assert not (tmp_path / "compose.yaml").exists()
def test_identical_content_skipped(self, tmp_path: Path) -> None:
# Write it once
with patch("builtins.input", return_value="y"):
_tool_write_compose(tmp_path, {})
# Second call should skip
result = _tool_write_compose(tmp_path, {})
assert "already exists" in result
def test_no_build_blocks(self, tmp_path: Path) -> None:
with patch("builtins.input", return_value="y"):
_tool_write_compose(tmp_path, {})
content = (tmp_path / "compose.yaml").read_text()
assert "build:" not in content
assert "dockerfile:" not in content.lower()
def test_overwrites_different_content(self, tmp_path: Path) -> None:
(tmp_path / "compose.yaml").write_text("old content\n")
with patch("builtins.input", return_value="y"):
result = _tool_write_compose(tmp_path, {})
assert "written successfully" in result
content = (tmp_path / "compose.yaml").read_text()
assert "ghcr.io" in content
def test_no_local_image_references(self, tmp_path: Path) -> None:
with patch("builtins.input", return_value="y"):
_tool_write_compose(tmp_path, {})
content = (tmp_path / "compose.yaml").read_text()
assert "turnstone:local" not in content
class TestGenerateSecret:
def test_default_length(self) -> None:
secret = _tool_generate_secret({})
@@ -620,7 +667,7 @@ class TestConstants:
assert func["parameters"]["type"] == "object"
def test_tool_count(self) -> None:
assert len(TOOLS) == 7
assert len(TOOLS) == 8
def test_all_tools_have_implementations(self) -> None:
from turnstone.bootstrap import TOOL_FUNCTIONS
+649 -37
View File
@@ -8,6 +8,13 @@ from unittest.mock import AsyncMock, MagicMock, patch
import pytest
# discord.utils.escape_markdown passes 'count' as positional to re.sub,
# which is deprecated in Python 3.13+. This is a discord.py bug (fixed
# in newer releases); suppress here to keep the test output clean.
pytestmark = pytest.mark.filterwarnings(
"ignore:.*'count' is passed as positional argument:DeprecationWarning"
)
discord = pytest.importorskip("discord")
@@ -21,6 +28,21 @@ def _run(coro):
return asyncio.run(coro)
def _bind_ws_event_handlers(bot, cls):
"""Bind ``_on_ws_event`` + every ``_handle_*`` method from *cls* to *bot*.
``MagicMock(spec=cls)`` stubs async methods as ``AsyncMock`` no-ops,
so dispatcher tests that invoke the real ``_on_ws_event`` must also
bind the per-event handlers it delegates to.
"""
bot._on_ws_event = cls._on_ws_event.__get__(bot, cls)
for name in dir(cls):
if name.startswith("_handle_"):
attr = getattr(cls, name)
if callable(attr):
setattr(bot, name, attr.__get__(bot, cls))
def _make_message(*, bot=False, guild=True, content="hello", channel=None, reference=None):
"""Build a mock ``discord.Message``."""
msg = MagicMock(spec=discord.Message)
@@ -121,7 +143,7 @@ class TestStreamingMessage:
_run(sm.append("hello "))
_run(sm.append("world"))
assert "".join(sm._buffer) == "hello world"
assert sm.accumulated_text == "hello world"
def test_finalize_sends_when_no_prior_message(self):
from turnstone.channels.discord.bot import StreamingMessage
@@ -146,7 +168,7 @@ class TestStreamingMessage:
# First append triggers flush (interval=0) which creates the message.
_run(sm.append("hi"))
assert sm._message is sent_msg
assert sm.message is sent_msg
_run(sm.append(" there"))
_run(sm.finalize())
@@ -253,6 +275,90 @@ class TestMessageCog:
ts.router.send_message.assert_not_awaited()
# ---------------------------------------------------------------------------
# /ask command — model selection
# ---------------------------------------------------------------------------
class TestAskModelSelection:
"""Tests for the /ask command's model parameter and channel default."""
def _make_cog_and_interaction(self):
from turnstone.channels.discord.cog import MessageCog
bot = MagicMock()
bot.user = MagicMock()
bot.user.id = 99999
ts = MagicMock()
ts.router = MagicMock()
ts.router.resolve_user = AsyncMock(return_value="u_abc")
ts.router.get_or_create_workstream = AsyncMock(return_value=("ws-1", True))
ts.router.send_message = AsyncMock()
ts.router.get_channel_default_alias = AsyncMock(return_value="")
ts.subscribe_ws = AsyncMock()
ts.config = MagicMock()
ts.config.model = "cli-model"
ts.config.thread_auto_archive = 1440
bot.turnstone = ts
cog = MessageCog(bot)
interaction = MagicMock(spec=discord.Interaction)
interaction.user = MagicMock()
interaction.user.id = 67890
interaction.response = MagicMock()
interaction.response.defer = AsyncMock()
interaction.followup = MagicMock()
interaction.followup.send = AsyncMock()
thread = AsyncMock(spec=discord.Thread)
thread.id = 111
thread.mention = "<#111>"
channel = MagicMock(spec=discord.TextChannel)
channel.create_thread = AsyncMock(return_value=thread)
interaction.channel = channel
return cog, ts, interaction
def test_explicit_model_overrides_all(self):
cog, ts, interaction = self._make_cog_and_interaction()
ts.router.get_channel_default_alias = AsyncMock(return_value="channel-default")
_run(cog._cmd_ask(interaction, "hello", model="explicit-model"))
_, kwargs = ts.router.get_or_create_workstream.call_args
assert kwargs["model"] == "explicit-model"
def test_channel_default_used_when_no_explicit_model(self):
cog, ts, interaction = self._make_cog_and_interaction()
ts.router.get_channel_default_alias = AsyncMock(return_value="channel-default")
_run(cog._cmd_ask(interaction, "hello"))
_, kwargs = ts.router.get_or_create_workstream.call_args
assert kwargs["model"] == "channel-default"
def test_cli_model_fallback(self):
cog, ts, interaction = self._make_cog_and_interaction()
# Channel default is empty → fall back to CLI --model.
ts.router.get_channel_default_alias = AsyncMock(return_value="")
_run(cog._cmd_ask(interaction, "hello"))
_, kwargs = ts.router.get_or_create_workstream.call_args
assert kwargs["model"] == "cli-model"
def test_empty_model_when_no_defaults(self):
cog, ts, interaction = self._make_cog_and_interaction()
ts.router.get_channel_default_alias = AsyncMock(return_value="")
ts.config.model = ""
_run(cog._cmd_ask(interaction, "hello"))
_, kwargs = ts.router.get_or_create_workstream.call_args
assert kwargs["model"] == ""
# ---------------------------------------------------------------------------
# _parse_footer (views.py)
# ---------------------------------------------------------------------------
@@ -261,20 +367,20 @@ class TestMessageCog:
class TestParseFooter:
"""Tests for _parse_footer in views.py."""
def test_valid_footer(self):
def test_valid_footer_with_owner(self):
from turnstone.channels.discord.views import _parse_footer
interaction = _make_interaction(footer_text="ws_abc|corr_123|12345")
result = _parse_footer(interaction)
assert result == ("ws_abc", "corr_123", "12345")
def test_footer_without_owner_returns_empty_owner(self):
from turnstone.channels.discord.views import _parse_footer
# Legacy footer without an owner field (pre-upgrade posts).
interaction = _make_interaction(footer_text="ws_abc|corr_123")
result = _parse_footer(interaction)
assert result == ("ws_abc", "corr_123")
def test_footer_with_pipe_in_correlation(self):
from turnstone.channels.discord.views import _parse_footer
interaction = _make_interaction(footer_text="ws_abc|corr|extra")
result = _parse_footer(interaction)
# split("|", 1) means the second part includes everything after first pipe.
assert result == ("ws_abc", "corr|extra")
assert result == ("ws_abc", "corr_123", "")
def test_no_message_returns_none(self):
from turnstone.channels.discord.views import _parse_footer
@@ -337,7 +443,7 @@ class TestWsEventFinalization:
bot._notify_reply_channels = {}
# Use the real _on_ws_event method
bot._on_ws_event = TurnstoneBot._on_ws_event.__get__(bot, TurnstoneBot)
_bind_ws_event_handlers(bot, TurnstoneBot)
thread = AsyncMock()
@@ -366,7 +472,7 @@ class TestWsEventFinalization:
bot._tool_info_msgs = {}
bot._pending_approval_msgs = {}
bot._notify_reply_channels = {}
bot._on_ws_event = TurnstoneBot._on_ws_event.__get__(bot, TurnstoneBot)
_bind_ws_event_handlers(bot, TurnstoneBot)
thread = AsyncMock()
@@ -387,6 +493,7 @@ class TestApprovalVerdictDisplay:
def _make_bot(self):
"""Build a mock TurnstoneBot with _on_ws_event bound."""
from turnstone.channels._routing import PolicyVerdict
from turnstone.channels.discord.bot import TurnstoneBot
bot = MagicMock(spec=TurnstoneBot)
@@ -402,7 +509,9 @@ class TestApprovalVerdictDisplay:
bot._pending_approval_msgs = {}
bot._notify_reply_channels = {}
bot._should_auto_approve = MagicMock(return_value=False)
bot._on_ws_event = TurnstoneBot._on_ws_event.__get__(bot, TurnstoneBot)
bot.router = MagicMock()
bot.router.evaluate_tool_policies = AsyncMock(return_value=PolicyVerdict(kind="none"))
_bind_ws_event_handlers(bot, TurnstoneBot)
return bot
def test_approval_with_heuristic_verdict(self):
@@ -521,7 +630,7 @@ class TestApprovalVerdictDisplay:
bot._tool_info_msgs = {}
bot._pending_approval_msgs = {"ws-1": MagicMock()}
bot._notify_reply_channels = {}
bot._on_ws_event = TurnstoneBot._on_ws_event.__get__(bot, TurnstoneBot)
_bind_ws_event_handlers(bot, TurnstoneBot)
thread = AsyncMock()
event = StreamEndEvent(ws_id="ws-1")
@@ -547,7 +656,7 @@ class TestStreamEndBehavior:
bot._tool_info_msgs = {}
bot._pending_approval_msgs = {}
bot._notify_reply_channels = {}
bot._on_ws_event = TurnstoneBot._on_ws_event.__get__(bot, TurnstoneBot)
_bind_ws_event_handlers(bot, TurnstoneBot)
return bot
def test_stream_end_no_streaming_no_send(self):
@@ -583,38 +692,88 @@ class TestStreamEndBehavior:
class TestNotificationTracking:
"""Tests for notification message tracking and DM reply routing."""
def test_send_notification_tracks_message(self):
"""send_notification should store message_id -> (ws_id, target_user) mapping."""
def _make_dm_bot(self, *, sent_message_id: int):
"""Build a MagicMock bot whose notification target resolves to a DM."""
from turnstone.channels.discord.bot import TurnstoneBot
bot = MagicMock(spec=TurnstoneBot)
bot.config = MagicMock()
bot.config.max_message_length = 2000
sent_msg = MagicMock()
sent_msg.id = sent_message_id
dm_channel = MagicMock()
dm_channel.send = AsyncMock(return_value=sent_msg)
user = MagicMock()
user.id = 7777
user.create_dm = AsyncMock(return_value=dm_channel)
inner_bot = MagicMock()
inner_bot.get_channel = MagicMock(return_value=None)
inner_bot.fetch_user = AsyncMock(return_value=user)
bot._bot = inner_bot
bot.send_notification = TurnstoneBot.send_notification.__get__(bot, TurnstoneBot)
bot._track_notification = TurnstoneBot._track_notification.__get__(bot, TurnstoneBot)
return bot
def test_send_notification_tracks_dm_with_user_id(self):
"""send_notification for a DM records (ws_id, resolved_user_id)."""
bot = self._make_dm_bot(sent_message_id=12345)
bot._notify_ws_map = {}
bot._MAX_NOTIFY_TRACKING = 100
bot.send = AsyncMock(return_value="12345")
_run(bot.send_notification("7777", "Hello", "ws-abc"))
# Tracked under the resolved Discord user ID, not the raw argument.
assert 12345 in bot._notify_ws_map
assert bot._notify_ws_map[12345] == ("ws-abc", "7777")
def test_send_notification_to_guild_channel_is_not_tracked(self):
"""Notifications delivered to a guild channel must not register reply tracking.
The reply-channel_id check treats the stored value as a Discord
user ID, so storing a channel ID would reject every legitimate
reply.
"""
from turnstone.channels.discord.bot import TurnstoneBot
bot = MagicMock(spec=TurnstoneBot)
bot.config = MagicMock()
bot.config.max_message_length = 2000
bot._notify_ws_map = {}
bot._MAX_NOTIFY_TRACKING = 100
sent_msg = MagicMock()
sent_msg.id = 99999
channel = MagicMock()
channel.send = AsyncMock(return_value=sent_msg)
inner_bot = MagicMock()
inner_bot.get_channel = MagicMock(return_value=channel)
bot._bot = inner_bot
bot.send_notification = TurnstoneBot.send_notification.__get__(bot, TurnstoneBot)
bot._track_notification = TurnstoneBot._track_notification.__get__(bot, TurnstoneBot)
_run(bot.send_notification("chan-1", "Hello", "ws-abc"))
_run(bot.send_notification("888888", "Hello", "ws-abc"))
assert 12345 in bot._notify_ws_map
assert bot._notify_ws_map[12345] == ("ws-abc", "chan-1")
assert bot._notify_ws_map == {}
def test_send_notification_evicts_old_entries(self):
"""Oldest notification tracking entries are evicted when cap is reached."""
from turnstone.channels.discord.bot import TurnstoneBot
bot = MagicMock(spec=TurnstoneBot)
bot = self._make_dm_bot(sent_message_id=4)
bot._MAX_NOTIFY_TRACKING = 3
bot._notify_ws_map = {
1: ("ws-1", "u1"),
2: ("ws-2", "u2"),
3: ("ws-3", "u3"),
}
bot.send = AsyncMock(return_value="4")
bot.send_notification = TurnstoneBot.send_notification.__get__(bot, TurnstoneBot)
bot._track_notification = TurnstoneBot._track_notification.__get__(bot, TurnstoneBot)
_run(bot.send_notification("chan-1", "Hello", "ws-4"))
_run(bot.send_notification("7777", "Hello", "ws-4"))
assert 4 in bot._notify_ws_map
assert 1 not in bot._notify_ws_map # oldest evicted
@@ -787,7 +946,7 @@ class TestNotificationTracking:
sent_msg.id = 88888
dm_channel.send = AsyncMock(return_value=sent_msg)
bot._notify_reply_channels = {"ws-1": (dm_channel, "u123")}
bot._on_ws_event = TurnstoneBot._on_ws_event.__get__(bot, TurnstoneBot)
_bind_ws_event_handlers(bot, TurnstoneBot)
bot._track_notification = TurnstoneBot._track_notification.__get__(bot, TurnstoneBot)
thread = AsyncMock()
@@ -822,7 +981,7 @@ class TestNotificationTracking:
dm_channel = AsyncMock()
bot._notify_reply_channels = {"ws-1": (dm_channel, "u123")}
bot._on_ws_event = TurnstoneBot._on_ws_event.__get__(bot, TurnstoneBot)
_bind_ws_event_handlers(bot, TurnstoneBot)
thread = AsyncMock()
@@ -886,6 +1045,235 @@ class TestFormatToolResult:
assert result.count("```") == 2
# ---------------------------------------------------------------------------
# Media embed detection and rendering
# ---------------------------------------------------------------------------
class TestTryParseMedia:
"""Tests for try_parse_media in _formatter.py."""
def test_stream_url_detected(self):
import json
from turnstone.channels._formatter import try_parse_media
data = json.dumps({"stream_url": "http://jf:8096/Videos/abc/stream", "container": "mp4"})
result = try_parse_media(data)
assert result is not None
assert result["stream_url"] == "http://jf:8096/Videos/abc/stream"
def test_media_details_detected(self):
import json
from turnstone.channels._formatter import try_parse_media
data = json.dumps({"id": "abc", "name": "Test Movie", "type": "Movie", "year": 2024})
result = try_parse_media(data)
assert result is not None
assert result["name"] == "Test Movie"
def test_search_results_detected(self):
import json
from turnstone.channels._formatter import try_parse_media
data = json.dumps({"results": [{"id": "1", "name": "Hit"}], "total_count": 1})
result = try_parse_media(data)
assert result is not None
assert len(result["results"]) == 1
def test_sessions_detected(self):
import json
from turnstone.channels._formatter import try_parse_media
data = json.dumps({"sessions": [{"id": "s1", "user_name": "ptrck"}]})
result = try_parse_media(data)
assert result is not None
def test_empty_results_returns_none(self):
import json
from turnstone.channels._formatter import try_parse_media
assert try_parse_media(json.dumps({"results": []})) is None
def test_plain_text_returns_none(self):
from turnstone.channels._formatter import try_parse_media
assert try_parse_media("just a string") is None
def test_non_dict_json_returns_none(self):
from turnstone.channels._formatter import try_parse_media
assert try_parse_media("[1, 2, 3]") is None
def test_unrelated_dict_returns_none(self):
import json
from turnstone.channels._formatter import try_parse_media
assert try_parse_media(json.dumps({"foo": "bar"})) is None
class TestIsSafeImageUrl:
"""Tests for _is_safe_image_url in _formatter.py."""
@staticmethod
def _patch_resolver(monkeypatch, ips):
"""Replace socket.getaddrinfo with a stub returning *ips*."""
import socket
def fake(host, port, family=0, *args, **kwargs): # noqa: ARG001
return [(family, 0, 0, "", (ip, 0)) for ip in ips]
monkeypatch.setattr(socket, "getaddrinfo", fake)
def test_http_url(self, monkeypatch):
from turnstone.channels._formatter import _is_safe_image_url
self._patch_resolver(monkeypatch, ["203.0.113.5"])
assert _run(_is_safe_image_url("http://jellyfin:8096/Items/abc/Images/Primary")) is True
def test_https_url(self, monkeypatch):
from turnstone.channels._formatter import _is_safe_image_url
self._patch_resolver(monkeypatch, ["203.0.113.5"])
assert (
_run(_is_safe_image_url("https://jellyfin.example.com/Items/abc/Images/Primary"))
is True
)
def test_ftp_rejected(self):
from turnstone.channels._formatter import _is_safe_image_url
assert _run(_is_safe_image_url("ftp://evil.com/image.jpg")) is False
def test_file_rejected(self):
from turnstone.channels._formatter import _is_safe_image_url
assert _run(_is_safe_image_url("file:///etc/passwd")) is False
def test_userinfo_rejected(self):
from turnstone.channels._formatter import _is_safe_image_url
assert _run(_is_safe_image_url("http://user:pass@jellyfin:8096/image")) is False
def test_empty_rejected(self):
from turnstone.channels._formatter import _is_safe_image_url
assert _run(_is_safe_image_url("")) is False
def test_private_ip_allowed(self):
from turnstone.channels._formatter import _is_safe_image_url
assert _run(_is_safe_image_url("http://192.168.0.6:8096/Items/abc/Images/Primary")) is True
def test_dns_rebinding_rejected(self, monkeypatch):
"""Hostname that resolves to a loopback IP must be rejected."""
from turnstone.channels._formatter import _is_safe_image_url
self._patch_resolver(monkeypatch, ["127.0.0.1"])
assert _run(_is_safe_image_url("http://rebind.example.com/image")) is False
def test_metadata_endpoint_rejected(self):
"""AWS/GCP metadata IP is link-local → rejected."""
from turnstone.channels._formatter import _is_safe_image_url
assert _run(_is_safe_image_url("http://169.254.169.254/latest/meta-data/")) is False
def test_ipv6_aws_nitro_metadata_rejected(self, monkeypatch):
"""fd00:ec2::254 is IPv6 ULA (is_private) but must be blocked —
the IPv4 169.254.169.254 check left this analogue open."""
from turnstone.channels._formatter import _is_safe_image_url
self._patch_resolver(monkeypatch, ["fd00:ec2::254"])
assert _run(_is_safe_image_url("http://nitro.example.com/")) is False
def test_ipv6_ecs_task_metadata_rejected(self, monkeypatch):
"""ECS Task Metadata lives in the same fd00:ec2::/32 prefix."""
from turnstone.channels._formatter import _is_safe_image_url
self._patch_resolver(monkeypatch, ["fd00:ec2::23"])
assert _run(_is_safe_image_url("http://ecs-meta.example.com/")) is False
class TestBuildMediaEmbed:
"""Tests for try_build_media_embed and embed builders."""
def test_single_item_embed_uses_web_url_not_stream_url(self):
import json
from turnstone.channels._formatter import try_parse_media
data = {
"name": "Test Movie",
"type": "Movie",
"year": 2024,
"stream_url": "http://jf:8096/Videos/abc/stream?api_key=SECRET",
"web_url": "http://jf:8096/web/#/details?id=abc",
"overview": "A test movie.",
}
parsed = try_parse_media(json.dumps(data))
assert parsed is not None
from turnstone.channels._formatter import _build_single_media_embed
embed = _build_single_media_embed(parsed, "mcp__mediamcp__get_stream_url")
# web_url should be the embed URL, never stream_url
assert embed.url == "http://jf:8096/web/#/details?id=abc"
assert "SECRET" not in str(embed.to_dict())
def test_search_results_embed_format(self):
import json
from turnstone.channels._formatter import try_parse_media
data = {
"results": [
{"name": "Movie A", "year": 2020, "type": "Movie", "runtime_minutes": 120},
{"name": "Movie B", "year": 2021, "type": "Movie"},
],
"total_count": 2,
}
parsed = try_parse_media(json.dumps(data))
from turnstone.channels._formatter import _build_search_results_embed
embed = _build_search_results_embed(parsed)
assert "Movie A" in embed.description
assert "Movie B" in embed.description
assert "2 of 2" in embed.footer.text
def test_build_media_embed_returns_none_for_plain_text(self):
from turnstone.channels._formatter import try_build_media_embed
http = MagicMock()
result = _run(try_build_media_embed("tool", "plain text", http=http))
assert result is None
def test_season_episode_string_values(self):
"""Season/episode numbers as strings should not raise."""
from turnstone.channels._formatter import _build_search_results_embed
data = {
"results": [
{
"name": "Pilot",
"type": "Episode",
"series_name": "Show",
"season_number": "1",
"episode_number": "1",
},
],
"total_count": 1,
}
embed = _build_search_results_embed(data)
assert "S01E01" in embed.description
# ---------------------------------------------------------------------------
# Thinking indicator lifecycle
# ---------------------------------------------------------------------------
@@ -910,7 +1298,7 @@ class TestThinkingIndicator:
bot._pending_approval_msgs = {}
bot._notify_reply_channels = {}
bot._should_auto_approve = MagicMock(return_value=False)
bot._on_ws_event = TurnstoneBot._on_ws_event.__get__(bot, TurnstoneBot)
_bind_ws_event_handlers(bot, TurnstoneBot)
return bot
def test_thinking_start_sends_message(self):
@@ -967,7 +1355,7 @@ class TestThinkingIndicator:
# Thinking message becomes the StreamingMessage base — no delete.
assert "ws-1" not in bot._thinking_msgs
sm = bot._streaming["ws-1"]
assert sm._message is thinking_msg
assert sm.message is thinking_msg
def test_stream_end_clears_thinking_message(self):
from turnstone.sdk.events import StreamEndEvent
@@ -1010,7 +1398,7 @@ class TestToolInfoEvent:
bot._pending_approval_msgs = {}
bot._notify_reply_channels = {}
bot._should_auto_approve = MagicMock(return_value=False)
bot._on_ws_event = TurnstoneBot._on_ws_event.__get__(bot, TurnstoneBot)
_bind_ws_event_handlers(bot, TurnstoneBot)
return bot
def test_sends_per_item_embed(self):
@@ -1103,8 +1491,9 @@ class TestToolResultEvent:
bot._tool_info_msgs = {}
bot._pending_approval_msgs = {}
bot._notify_reply_channels = {}
bot._http_client = MagicMock()
bot._should_auto_approve = MagicMock(return_value=False)
bot._on_ws_event = TurnstoneBot._on_ws_event.__get__(bot, TurnstoneBot)
_bind_ws_event_handlers(bot, TurnstoneBot)
return bot
def test_marks_info_done_and_sends_result(self):
@@ -1259,7 +1648,7 @@ class TestApprovalResolved:
bot._pending_approval_msgs = {}
bot._notify_reply_channels = {}
bot._should_auto_approve = MagicMock(return_value=False)
bot._on_ws_event = TurnstoneBot._on_ws_event.__get__(bot, TurnstoneBot)
_bind_ws_event_handlers(bot, TurnstoneBot)
return bot
def test_disables_buttons_on_timeout(self):
@@ -1327,3 +1716,226 @@ class TestChannelCLI:
main()
assert exc_info.value.code == 1
# ---------------------------------------------------------------------------
# Approval / plan-review interaction views — owner-check regression tests
# ---------------------------------------------------------------------------
def _make_view_interaction(user_id: int, footer: str | None) -> MagicMock:
"""Build a minimal interaction for ApprovalView / PlanReviewView tests."""
interaction = MagicMock(spec=discord.Interaction)
interaction.user = MagicMock()
interaction.user.id = user_id
interaction.response = MagicMock()
interaction.response.send_message = AsyncMock()
interaction.response.defer = AsyncMock()
interaction.response.send_modal = AsyncMock()
interaction.followup = MagicMock()
interaction.followup.send = AsyncMock()
interaction.message = MagicMock()
if footer is None:
interaction.message.embeds = []
else:
embed = MagicMock()
embed.footer.text = footer
interaction.message.embeds = [embed]
return interaction
def _make_view_bot() -> MagicMock:
"""Build a TurnstoneBot double with just the surface the views read."""
from turnstone.channels.discord.bot import TurnstoneBot
bot = MagicMock(spec=TurnstoneBot)
bot.router = MagicMock()
bot.router.resolve_user = AsyncMock(return_value="turnstone-user-1")
bot.router.send_approval = AsyncMock()
bot.router.send_plan_feedback = AsyncMock()
bot._pending_approval_msgs = {}
return bot
class TestApprovalViewOwnerCheck:
"""ApprovalView rejects clicks from anyone other than the session owner."""
def test_owner_approve_allowed(self, monkeypatch):
from turnstone.channels.discord.views import ApprovalView
# Avoid real disable_message_buttons (touches discord.ui internals).
monkeypatch.setattr(
"turnstone.channels.discord.views._disable_buttons",
AsyncMock(),
)
view = ApprovalView(_make_view_bot())
interaction = _make_view_interaction(user_id=42, footer="ws-1|corr-1|42")
_run(view._handle(interaction, approved=True, always=False))
view.bot.router.send_approval.assert_awaited_once_with(
ws_id="ws-1",
correlation_id="corr-1",
approved=True,
always=False,
)
def test_non_owner_rejected(self):
from turnstone.channels.discord.views import ApprovalView
view = ApprovalView(_make_view_bot())
interaction = _make_view_interaction(user_id=999, footer="ws-1|corr-1|42")
_run(view._handle(interaction, approved=True, always=False))
view.bot.router.send_approval.assert_not_awaited()
interaction.response.send_message.assert_awaited_once()
msg_kwargs = interaction.response.send_message.call_args
assert "Only the session owner" in msg_kwargs.args[0]
assert msg_kwargs.kwargs.get("ephemeral") is True
def test_legacy_footer_without_owner_rejected(self):
from turnstone.channels.discord.views import ApprovalView
view = ApprovalView(_make_view_bot())
# Pre-upgrade footer with only ws_id|correlation_id — fail closed.
interaction = _make_view_interaction(user_id=42, footer="ws-1|corr-1")
_run(view._handle(interaction, approved=True, always=False))
view.bot.router.send_approval.assert_not_awaited()
class TestPlanReviewViewOwnerCheck:
"""PlanReviewView rejects clicks from anyone other than the session owner."""
def test_owner_approve_allowed(self, monkeypatch):
from turnstone.channels.discord.views import PlanReviewView
monkeypatch.setattr(
"turnstone.channels.discord.views._disable_buttons",
AsyncMock(),
)
view = PlanReviewView(_make_view_bot())
interaction = _make_view_interaction(user_id=42, footer="ws-1|corr-1|42")
_run(view._handle_approve(interaction))
view.bot.router.send_plan_feedback.assert_awaited_once_with(
ws_id="ws-1",
correlation_id="corr-1",
feedback="",
)
def test_non_owner_approve_rejected(self):
from turnstone.channels.discord.views import PlanReviewView
view = PlanReviewView(_make_view_bot())
interaction = _make_view_interaction(user_id=999, footer="ws-1|corr-1|42")
_run(view._handle_approve(interaction))
view.bot.router.send_plan_feedback.assert_not_awaited()
interaction.response.send_message.assert_awaited_once()
def test_non_owner_changes_modal_rejected(self):
from turnstone.channels.discord.views import PlanReviewView
view = PlanReviewView(_make_view_bot())
interaction = _make_view_interaction(user_id=999, footer="ws-1|corr-1|42")
_run(view._handle_changes(interaction))
interaction.response.send_modal.assert_not_awaited()
interaction.response.send_message.assert_awaited_once()
class TestDiscordThreadOwnerCheck:
"""Sec-3 gate: only the thread creator can send messages into the workstream."""
@staticmethod
def _make_cog_and_ts():
"""Build a MessageCog wired to a minimal TurnstoneBot double."""
from turnstone.channels.discord.cog import MessageCog
bot = MagicMock()
bot.user = MagicMock()
bot.user.id = 99999
bot.user.mentioned_in = MagicMock(return_value=False)
ts = MagicMock()
ts._is_allowed_channel = MagicMock(return_value=True)
ts.storage = MagicMock()
ts.router = MagicMock()
ts.router.lookup_ws_id = AsyncMock(return_value="ws-1")
ts.router.resolve_user = AsyncMock(return_value="turnstone-user-1")
ts.router.send_message = AsyncMock()
ts.router.get_or_create_workstream = AsyncMock(return_value=("ws-1", False))
ts.config = MagicMock()
ts._ws_tasks = {}
ts._subscribed_ws = {"ws-1"}
ts._notify_ws_map = {}
ts._notify_reply_channels = {}
ts.get_thread_invoker = MagicMock(return_value=None)
ts.subscribe_ws = AsyncMock()
bot.turnstone = ts
return MessageCog(bot), ts
def test_non_owner_thread_message_dropped(self):
"""A linked user who is NOT the thread creator gets their message
silently dropped router.send_message must not fire."""
cog, ts = self._make_cog_and_ts()
# Build a thread whose owner_id is different from the message author.
thread = MagicMock(spec=discord.Thread)
thread.id = 555
thread.parent_id = 111
thread.owner_id = 42 # thread creator
thread.name = "some-thread"
msg = _make_message(guild=True, channel=thread)
msg.author.id = 999 # non-owner trying to inject
_run(cog._on_message(msg))
ts.router.send_message.assert_not_awaited()
ts.router.get_or_create_workstream.assert_not_awaited()
def test_ask_thread_followup_allowed_when_invoker_registered(self):
"""/ask creates threads with owner_id=bot; follow-ups from the
registered invoker must still reach the workstream."""
cog, ts = self._make_cog_and_ts()
# Simulate what _cmd_ask does after channel.create_thread().
ts.get_thread_invoker = MagicMock(return_value=111)
thread = MagicMock(spec=discord.Thread)
thread.id = 555
thread.parent_id = 222
thread.owner_id = 99999 # bot owns the thread after channel.create_thread
thread.name = "ask-thread"
msg = _make_message(guild=True, channel=thread)
msg.author.id = 111 # the human who ran /ask
_run(cog._on_message(msg))
ts.router.send_message.assert_awaited_once_with("ws-1", msg.content)
def test_ask_thread_rejects_other_user_even_when_invoker_registered(self):
"""Registered invoker lock: only that user's follow-ups pass."""
cog, ts = self._make_cog_and_ts()
ts.get_thread_invoker = MagicMock(return_value=111)
thread = MagicMock(spec=discord.Thread)
thread.id = 555
thread.parent_id = 222
thread.owner_id = 99999 # bot-owned
thread.name = "ask-thread"
msg = _make_message(guild=True, channel=thread)
msg.author.id = 222 # someone other than the recorded invoker
_run(cog._on_message(msg))
ts.router.send_message.assert_not_awaited()
+1 -55
View File
@@ -1,55 +1,13 @@
"""Tests for turnstone.channels._protocol and turnstone.channels._formatter."""
"""Tests for turnstone.channels._formatter."""
from __future__ import annotations
from turnstone.channels._formatter import (
chunk_message,
format_approval_request,
format_plan_review,
format_verdict,
truncate,
)
from turnstone.channels._protocol import ChannelEvent
# ---------------------------------------------------------------------------
# ChannelEvent
# ---------------------------------------------------------------------------
class TestChannelEvent:
def test_construction(self) -> None:
evt = ChannelEvent(
channel_type="discord",
channel_id="ch-1",
channel_user_id="u-42",
message="hello",
parent_channel_id="parent",
metadata={"key": "val"},
)
assert evt.channel_type == "discord"
assert evt.channel_id == "ch-1"
assert evt.channel_user_id == "u-42"
assert evt.message == "hello"
assert evt.parent_channel_id == "parent"
assert evt.metadata == {"key": "val"}
def test_defaults(self) -> None:
evt = ChannelEvent(
channel_type="slack",
channel_id="ch-2",
channel_user_id="u-7",
message="hi",
)
assert evt.parent_channel_id == ""
assert evt.metadata == {}
def test_metadata_independence(self) -> None:
"""Default metadata dicts are independent across instances."""
a = ChannelEvent(channel_type="x", channel_id="1", channel_user_id="u", message="m")
b = ChannelEvent(channel_type="x", channel_id="2", channel_user_id="u", message="m")
a.metadata["key"] = "val"
assert "key" not in b.metadata
# ---------------------------------------------------------------------------
# chunk_message
@@ -172,18 +130,6 @@ class TestFormatApprovalRequest:
assert "/etc/hosts" in result
# ---------------------------------------------------------------------------
# format_plan_review
# ---------------------------------------------------------------------------
class TestFormatPlanReview:
def test_format(self) -> None:
result = format_plan_review("Step 1: do stuff")
assert result.startswith("**Plan review requested:**")
assert "Step 1: do stuff" in result
# ---------------------------------------------------------------------------
# format_verdict
# ---------------------------------------------------------------------------
File diff suppressed because it is too large Load Diff
+397
View File
@@ -0,0 +1,397 @@
"""Tests for the shared SSE reconnect helper in turnstone.channels._sse."""
from __future__ import annotations
import asyncio
import contextlib
import json
from types import SimpleNamespace
from unittest.mock import AsyncMock, MagicMock
import httpx
import pytest
def _run(coro): # type: ignore[no-untyped-def]
return asyncio.run(coro)
class _FakeSSEEvent:
"""A fake ``httpx_sse.ServerSentEvent`` with the subset we read."""
def __init__(self, event: str, data: str) -> None:
self.event = event
self.data = data
class _FakeEventSource:
"""Context manager returned by our fake ``aconnect_sse``.
Captures the (status_code, events) the test wants to deliver.
``aiter_sse`` yields the events then returns; the caller then hits
the outer ``while True`` loop again, which will pick up the next
queued response via the shared iterator state on _FakeConnect.
"""
def __init__(self, *, status_code: int, events: list[_FakeSSEEvent]) -> None:
self.response = SimpleNamespace(
status_code=status_code,
request=MagicMock(),
)
self._events = events
async def __aenter__(self) -> _FakeEventSource:
return self
async def __aexit__(self, exc_type, exc, tb) -> None: # noqa: ANN001
return None
async def aiter_sse(self): # type: ignore[no-untyped-def]
for event in self._events:
yield event
class _FakeConnect:
"""Drop-in replacement for ``httpx_sse.aconnect_sse``.
On each call, pops the next ``_FakeEventSource`` from *queue*. When
the queue is empty, raises ``asyncio.CancelledError`` so the loop
terminates cleanly in tests.
"""
def __init__(self, queue: list[_FakeEventSource]) -> None:
self._queue = queue
self.call_count = 0
def __call__(self, *args, **kwargs): # noqa: ANN001, ANN204
self.call_count += 1
if not self._queue:
raise asyncio.CancelledError
return self._queue.pop(0)
@pytest.fixture
def _fast_sleep(monkeypatch):
"""Patch asyncio.sleep so backoff doesn't actually wait; record calls."""
sleeps: list[float] = []
async def fake_sleep(delay: float) -> None:
sleeps.append(delay)
monkeypatch.setattr("turnstone.channels._sse.asyncio.sleep", fake_sleep)
return sleeps
def _valid_event_data(ws_id: str = "ws-1") -> str:
"""A payload ``ServerEvent.from_dict`` will accept (a ContentEvent)."""
return json.dumps(
{
"type": "content",
"ws_id": ws_id,
"text": "hello",
}
)
# ---------------------------------------------------------------------------
# 404 → on_stale + exit
# ---------------------------------------------------------------------------
class TestStaleRoute:
def test_404_calls_on_stale_and_returns(self, monkeypatch, _fast_sleep):
from turnstone.channels import _sse
queue = [_FakeEventSource(status_code=404, events=[])]
fake_connect = _FakeConnect(queue)
monkeypatch.setattr(_sse.httpx_sse, "aconnect_sse", fake_connect)
on_stale = AsyncMock()
on_event = AsyncMock()
async def node_url_fn(ws_id: str) -> str:
return "http://node"
_run(
_sse.run_sse_stream(
http_client=MagicMock(),
log_prefix="test",
ws_id="ws-1",
node_url_fn=node_url_fn,
token_factory=None,
on_event=on_event,
on_stale=on_stale,
)
)
on_stale.assert_awaited_once()
on_event.assert_not_awaited()
# No reconnect after 404.
assert fake_connect.call_count == 1
assert _fast_sleep == []
def test_on_stale_exception_still_exits(self, monkeypatch, _fast_sleep):
"""If on_stale raises, the loop must not reconnect."""
from turnstone.channels import _sse
queue = [_FakeEventSource(status_code=404, events=[])]
fake_connect = _FakeConnect(queue)
monkeypatch.setattr(_sse.httpx_sse, "aconnect_sse", fake_connect)
on_stale = AsyncMock(side_effect=RuntimeError("storage down"))
async def node_url_fn(ws_id: str) -> str:
return "http://node"
_run(
_sse.run_sse_stream(
http_client=MagicMock(),
log_prefix="test",
ws_id="ws-1",
node_url_fn=node_url_fn,
token_factory=None,
on_event=AsyncMock(),
on_stale=on_stale,
)
)
on_stale.assert_awaited_once()
# Still a single connect — no livelock.
assert fake_connect.call_count == 1
# ---------------------------------------------------------------------------
# 500+ → exponential backoff
# ---------------------------------------------------------------------------
class TestBackoff:
def test_500_triggers_backoff_and_retries(self, monkeypatch, _fast_sleep):
from turnstone.channels import _sse
queue = [
_FakeEventSource(status_code=503, events=[]),
_FakeEventSource(status_code=503, events=[]),
_FakeEventSource(status_code=503, events=[]),
]
fake_connect = _FakeConnect(queue)
monkeypatch.setattr(_sse.httpx_sse, "aconnect_sse", fake_connect)
async def node_url_fn(ws_id: str) -> str:
return "http://node"
with contextlib.suppress(asyncio.CancelledError):
_run(
_sse.run_sse_stream(
http_client=MagicMock(),
log_prefix="test",
ws_id="ws-1",
node_url_fn=node_url_fn,
token_factory=None,
on_event=AsyncMock(),
on_stale=AsyncMock(),
)
)
assert fake_connect.call_count >= 3
# First three recorded sleeps are 2s, 4s, 8s (starts at
# SSE_RECONNECT_DELAY, doubles each time, capped at
# SSE_MAX_RECONNECT_DELAY).
assert _fast_sleep[0] == _sse.SSE_RECONNECT_DELAY
assert _fast_sleep[1] == _sse.SSE_RECONNECT_DELAY * 2
assert _fast_sleep[2] == _sse.SSE_RECONNECT_DELAY * 4
def test_backoff_resets_after_successful_dispatch(self, monkeypatch, _fast_sleep):
"""After a 200 + successful event dispatch, the next error
restarts backoff at the initial delay."""
from turnstone.channels import _sse
good_event = _FakeSSEEvent(event="message", data=_valid_event_data())
queue = [
_FakeEventSource(status_code=503, events=[]),
_FakeEventSource(status_code=200, events=[good_event]),
_FakeEventSource(status_code=503, events=[]),
]
fake_connect = _FakeConnect(queue)
monkeypatch.setattr(_sse.httpx_sse, "aconnect_sse", fake_connect)
on_event = AsyncMock()
async def node_url_fn(ws_id: str) -> str:
return "http://node"
with contextlib.suppress(asyncio.CancelledError):
_run(
_sse.run_sse_stream(
http_client=MagicMock(),
log_prefix="test",
ws_id="ws-1",
node_url_fn=node_url_fn,
token_factory=None,
on_event=on_event,
on_stale=AsyncMock(),
)
)
on_event.assert_awaited()
# Sleep sequence: 2 (after first 503), 2 (reset after 200/event),
# then CancelledError exits. First two sleeps are both the base
# delay — the reset did its job.
assert len(_fast_sleep) >= 2
assert _fast_sleep[0] == _sse.SSE_RECONNECT_DELAY
assert _fast_sleep[1] == _sse.SSE_RECONNECT_DELAY
# ---------------------------------------------------------------------------
# Event dispatch
# ---------------------------------------------------------------------------
class TestEventDispatch:
def test_invalid_json_is_skipped(self, monkeypatch, _fast_sleep):
from turnstone.channels import _sse
bad = _FakeSSEEvent(event="message", data="{not json")
good = _FakeSSEEvent(event="message", data=_valid_event_data())
queue = [_FakeEventSource(status_code=200, events=[bad, good])]
fake_connect = _FakeConnect(queue)
monkeypatch.setattr(_sse.httpx_sse, "aconnect_sse", fake_connect)
on_event = AsyncMock()
async def node_url_fn(ws_id: str) -> str:
return "http://node"
with contextlib.suppress(asyncio.CancelledError):
_run(
_sse.run_sse_stream(
http_client=MagicMock(),
log_prefix="test",
ws_id="ws-1",
node_url_fn=node_url_fn,
token_factory=None,
on_event=on_event,
on_stale=AsyncMock(),
)
)
# Good event delivered, bad one silently dropped.
assert on_event.await_count == 1
def test_on_event_exception_does_not_kill_stream(self, monkeypatch, _fast_sleep):
from turnstone.channels import _sse
e1 = _FakeSSEEvent(event="message", data=_valid_event_data())
e2 = _FakeSSEEvent(event="message", data=_valid_event_data())
queue = [_FakeEventSource(status_code=200, events=[e1, e2])]
fake_connect = _FakeConnect(queue)
monkeypatch.setattr(_sse.httpx_sse, "aconnect_sse", fake_connect)
on_event = AsyncMock(side_effect=[RuntimeError("boom"), None])
async def node_url_fn(ws_id: str) -> str:
return "http://node"
with contextlib.suppress(asyncio.CancelledError):
_run(
_sse.run_sse_stream(
http_client=MagicMock(),
log_prefix="test",
ws_id="ws-1",
node_url_fn=node_url_fn,
token_factory=None,
on_event=on_event,
on_stale=AsyncMock(),
)
)
# Both events attempted — first raised but second still delivered.
assert on_event.await_count == 2
# ---------------------------------------------------------------------------
# Token factory
# ---------------------------------------------------------------------------
class TestTokenFactory:
def test_header_refreshed_per_connection(self, monkeypatch, _fast_sleep):
"""token_factory is called once per reconnect so rotating service
JWTs stay fresh."""
from turnstone.channels import _sse
# Two reconnects followed by CancelledError to exit.
queue = [
_FakeEventSource(status_code=503, events=[]),
_FakeEventSource(status_code=503, events=[]),
]
fake_connect = _FakeConnect(queue)
monkeypatch.setattr(_sse.httpx_sse, "aconnect_sse", fake_connect)
tokens: list[str] = []
def factory() -> str:
tok = f"tok-{len(tokens)}"
tokens.append(tok)
return tok
async def node_url_fn(ws_id: str) -> str:
return "http://node"
with contextlib.suppress(asyncio.CancelledError):
_run(
_sse.run_sse_stream(
http_client=MagicMock(),
log_prefix="test",
ws_id="ws-1",
node_url_fn=node_url_fn,
token_factory=factory,
on_event=AsyncMock(),
on_stale=AsyncMock(),
)
)
assert len(tokens) >= 2
assert tokens[0] != tokens[1]
# ---------------------------------------------------------------------------
# httpx errors
# ---------------------------------------------------------------------------
class TestTransportErrors:
def test_connect_error_falls_through_to_backoff(self, monkeypatch, _fast_sleep):
"""ConnectError is caught and treated as retryable."""
from turnstone.channels import _sse
call_order = {"n": 0}
def fake_connect(*args, **kwargs): # noqa: ANN001, ANN003
call_order["n"] += 1
if call_order["n"] == 1:
raise httpx.ConnectError("boom")
# Second attempt: signal the loop to exit.
raise asyncio.CancelledError
monkeypatch.setattr(_sse.httpx_sse, "aconnect_sse", fake_connect)
async def node_url_fn(ws_id: str) -> str:
return "http://node"
with contextlib.suppress(asyncio.CancelledError):
_run(
_sse.run_sse_stream(
http_client=MagicMock(),
log_prefix="test",
ws_id="ws-1",
node_url_fn=node_url_fn,
token_factory=None,
on_event=AsyncMock(),
on_stale=AsyncMock(),
)
)
assert call_order["n"] == 2
# Backoff ran once after the ConnectError.
assert _fast_sleep == [_sse.SSE_RECONNECT_DELAY]
+184
View File
@@ -0,0 +1,184 @@
"""Server-side tests for the close_workstream handler's close_reason
persistence guards the seam that lets coordinator inspect surface
why a workstream was retired without scraping the audit log.
"""
from __future__ import annotations
import queue
import threading
from typing import Any
from unittest.mock import MagicMock
import pytest
from starlette.testclient import TestClient
import turnstone.server as srv_mod
from turnstone.core.auth import JWT_AUD_SERVER, create_jwt
from turnstone.core.metrics import MetricsCollector
from turnstone.core.storage._sqlite import SQLiteBackend
from turnstone.core.workstream import WorkstreamState
_JWT_SECRET = "test-jwt-secret-minimum-32-chars!"
def _full_hdr() -> dict[str, str]:
return {
"Authorization": (
f"Bearer {create_jwt('u1', frozenset({'read', 'write', 'approve'}), 'test', _JWT_SECRET, audience=JWT_AUD_SERVER)}"
)
}
def _make_app(storage: Any) -> TestClient:
srv_mod._metrics = MetricsCollector()
srv_mod._metrics.model = "test-model"
mock_session = MagicMock()
mock_ws = MagicMock()
mock_ws.id = "ws-target"
mock_ws.name = "test"
mock_ws.state = WorkstreamState.IDLE
mock_ws.session = mock_session
# Tenant gate (#375) checks ws.user_id == JWT subject; explicit set
# so MagicMock's auto-generated truthy attribute doesn't reject the
# request before the persistence path runs. kind / parent_ws_id
# land in the audit_detail dict alongside ``reason``.
mock_ws.user_id = "u1"
mock_ws.kind = "interactive"
mock_ws.parent_ws_id = None
mock_mgr = MagicMock()
mock_mgr.get.return_value = mock_ws
mock_mgr.close.return_value = True
mock_mgr.list_all.return_value = [mock_ws]
mock_mgr.max_workstreams = 10
app = srv_mod.create_app(
workstreams=mock_mgr,
global_queue=queue.Queue(),
global_listeners=[],
global_listeners_lock=threading.Lock(),
skip_permissions=False,
jwt_secret=_JWT_SECRET,
auth_storage=storage,
cors_origins=["*"],
)
return TestClient(app, raise_server_exceptions=False)
@pytest.fixture
def storage(tmp_path):
return SQLiteBackend(str(tmp_path / "close.db"))
def test_close_with_reason_persists_to_workstream_config(storage):
client = _make_app(storage)
resp = client.post(
"/v1/api/workstreams/close",
json={"ws_id": "ws-target", "reason": "task complete"},
headers=_full_hdr(),
)
assert resp.status_code == 200
cfg = storage.load_workstream_config("ws-target")
assert cfg.get("close_reason") == "task complete"
def test_close_without_reason_does_not_touch_config(storage):
client = _make_app(storage)
resp = client.post(
"/v1/api/workstreams/close",
json={"ws_id": "ws-target"},
headers=_full_hdr(),
)
assert resp.status_code == 200
cfg = storage.load_workstream_config("ws-target")
assert "close_reason" not in cfg
def test_close_reason_capped_at_512_bytes(storage):
"""A model that dumps a multi-KB blob (or a captured secret) into the
close reason must not be able to grow the workstream_config row
without bound the handler enforces a 512-byte ceiling. Tested
with ASCII (1B/char) so the byte cap and char count coincide."""
huge = "x" * 5000
client = _make_app(storage)
resp = client.post(
"/v1/api/workstreams/close",
json={"ws_id": "ws-target", "reason": huge},
headers=_full_hdr(),
)
assert resp.status_code == 200
cfg = storage.load_workstream_config("ws-target")
stored = cfg.get("close_reason")
assert stored is not None
assert len(stored.encode("utf-8")) <= 512
def test_close_reason_byte_cap_holds_for_multibyte_utf8(storage):
"""Repro for the char-cap-vs-byte-cap mismatch: a CJK-only payload
of 600 chars would have leaked through a code-point slice at
600*3=1800 bytes. The byte-aware cap holds it at <=512 bytes."""
huge = "\u6f22" * 600 # 3 bytes/char in UTF-8
client = _make_app(storage)
resp = client.post(
"/v1/api/workstreams/close",
json={"ws_id": "ws-target", "reason": huge},
headers=_full_hdr(),
)
assert resp.status_code == 200
cfg = storage.load_workstream_config("ws-target")
stored = cfg.get("close_reason")
assert stored is not None
assert len(stored.encode("utf-8")) <= 512
def test_close_with_non_string_reason_drops_silently(storage):
"""A malformed body (reason=dict / list / int) should not crash the
handler non-string reasons are coerced to empty and the close
proceeds without writing to workstream_config."""
client = _make_app(storage)
resp = client.post(
"/v1/api/workstreams/close",
json={"ws_id": "ws-target", "reason": {"unexpected": "shape"}},
headers=_full_hdr(),
)
assert resp.status_code == 200
cfg = storage.load_workstream_config("ws-target")
assert "close_reason" not in cfg
def test_close_reason_redacts_credentials(storage):
"""A model under prompt injection that captures a secret and stuffs
it into ``reason`` must not get to plant the plaintext secret in
audit logs / workstream_config. The output guard's credential-
redaction pass runs at the close handler boundary."""
client = _make_app(storage)
secret = "AKIAIOSFODNN7EXAMPLE" # AWS access key — output guard catches.
resp = client.post(
"/v1/api/workstreams/close",
json={"ws_id": "ws-target", "reason": f"task done; key={secret}"},
headers=_full_hdr(),
)
assert resp.status_code == 200
cfg = storage.load_workstream_config("ws-target")
stored = cfg.get("close_reason")
assert stored is not None
assert secret not in stored
assert "[REDACTED:" in stored
def test_close_reason_persistence_failure_does_not_block_close(storage):
"""If the storage save raises, the close still succeeds — persistence
is best-effort; a transient storage error must not block the user
from closing a workstream."""
client = _make_app(storage)
def _boom(*args, **kwargs):
raise RuntimeError("storage down")
storage.save_workstream_config = _boom # type: ignore[method-assign]
resp = client.post(
"/v1/api/workstreams/close",
json={"ws_id": "ws-target", "reason": "task complete"},
headers=_full_hdr(),
)
assert resp.status_code == 200
+4 -1
View File
@@ -3,7 +3,10 @@
import argparse
import turnstone.core.config as config_mod
from turnstone.core.config import apply_config, load_config, set_config_path
apply_config = config_mod.apply_config
load_config = config_mod.load_config
set_config_path = config_mod.set_config_path
def _reset_cache():
+19 -6
View File
@@ -87,8 +87,20 @@ class TestSetGetRoundTrip:
assert store.get("tools.skip_permissions") is False
def test_str(self, store):
store.set("model.name", "gpt-5")
assert store.get("model.name") == "gpt-5"
store.set("model.default_alias", "gpt5-prod")
assert store.get("model.default_alias") == "gpt5-prod"
def test_plan_task_alias(self, store):
store.set("model.plan_alias", "smart")
store.set("model.task_alias", "fast")
assert store.get("model.plan_alias") == "smart"
assert store.get("model.task_alias") == "fast"
def test_plan_task_effort(self, store):
store.set("model.plan_effort", "max")
store.set("model.task_effort", "low")
assert store.get("model.plan_effort") == "max"
assert store.get("model.task_effort") == "low"
# ---------------------------------------------------------------------------
@@ -105,7 +117,8 @@ class TestDelete:
assert store.get("tools.timeout") == defn.default
def test_returns_false_for_non_existent(self, store):
assert store.delete("tools.timeout") is False
result = store.delete("tools.timeout")
assert result is False
def test_rejects_unknown_key(self, store):
with pytest.raises(ValueError, match="Unknown setting"):
@@ -164,10 +177,10 @@ class TestStoredKeys:
assert store.stored_keys() == frozenset()
store.set("tools.timeout", 30)
assert store.stored_keys() == frozenset({"tools.timeout"})
store.set("model.name", "gpt-5")
assert store.stored_keys() == frozenset({"tools.timeout", "model.name"})
store.set("model.default_alias", "gpt5-prod")
assert store.stored_keys() == frozenset({"tools.timeout", "model.default_alias"})
store.delete("tools.timeout")
assert store.stored_keys() == frozenset({"model.name"})
assert store.stored_keys() == frozenset({"model.default_alias"})
# ---------------------------------------------------------------------------
+22 -6
View File
@@ -39,7 +39,7 @@ class MockStorage:
self.services: list[dict[str, str]] = []
def list_services(self, service_type: str, max_age_seconds: int = 120) -> list[dict[str, str]]:
return [s for s in self.services if True] # all services match
return list(self.services)
# ---------------------------------------------------------------------------
@@ -418,13 +418,12 @@ class TestCollectorDelta:
c._nodes["node-a"] = NodeSnapshot(
node_id="node-a",
server_url="http://a:8080",
health={"status": "ok", "backend": {"status": "up", "circuit_state": "closed"}},
health={"status": "ok", "backend": {"status": "up"}},
)
c._apply_delta("node-a", {"type": "health_changed", "circuit_state": "open"})
c._apply_delta("node-a", {"type": "health_changed", "backend_status": "degraded"})
health = c._nodes["node-a"].health
assert health["backend"]["circuit_state"] == "open"
assert health["backend"]["status"] == "down"
assert health["status"] == "degraded"
@@ -758,7 +757,9 @@ class TestConsoleHTTPEndpoints:
assert status == 200
assert len(data["nodes"]) == 1
assert data["total"] == 1
mock_collector.get_nodes.assert_called_once_with(sort_by="activity", limit=10, offset=0)
mock_collector.get_nodes.assert_called_once_with(
sort_by="activity", limit=10, offset=0, node_ids=None
)
def test_get_workstreams(self, client, mock_collector):
status, data = self._get(
@@ -776,6 +777,7 @@ class TestConsoleHTTPEndpoints:
sort_by="state",
page=1,
per_page=25,
extra_rows=[],
)
def test_get_workstreams_per_page_capped(self, client, mock_collector):
@@ -1428,7 +1430,7 @@ class TestSharedStatic:
def test_index_imports_shared_base_css(self, client):
resp = client.get("/")
assert resp.status_code == 200
assert '/shared/base.css"' in resp.text
assert "/shared/base.css?v=" in resp.text
def test_index_imports_shared_scripts(self, client):
resp = client.get("/")
@@ -1446,6 +1448,20 @@ class TestSharedStatic:
app_pos = body.find("/static/app.js")
assert shared_pos < app_pos
def test_index_cache_control_no_cache(self, client):
resp = client.get("/")
assert resp.headers.get("cache-control") == "no-cache"
def test_index_etag_present(self, client):
resp = client.get("/")
assert resp.headers.get("etag")
def test_index_etag_304(self, client):
resp = client.get("/")
etag = resp.headers.get("etag")
resp2 = client.get("/", headers={"If-None-Match": etag})
assert resp2.status_code == 304
class TestProxySharedStatic:
"""Tests for proxy rewriting of /shared/ paths."""
+16 -65
View File
@@ -37,61 +37,22 @@ class TestRecordRoute:
assert "turnstone_router_request_duration_seconds_sum" in text
class TestRingInfo:
"""Ring membership and version gauges."""
class TestRouterInfo:
"""Live-membership gauge + refresh counter."""
def test_defaults_zero(self) -> None:
m = ConsoleMetrics()
text = m.generate_text()
assert "turnstone_ring_membership_size 0" in text
assert "turnstone_ring_version 0" in text
assert "turnstone_router_membership_size 0" in text
assert "turnstone_router_refresh_total 0" in text
def test_set_ring_info(self) -> None:
def test_set_router_info(self) -> None:
m = ConsoleMetrics()
m.set_ring_info(3, 7)
m.set_router_info(3, 7)
text = m.generate_text()
assert "turnstone_ring_membership_size 3" in text
assert "turnstone_ring_version 7" in text
class TestRebalance:
"""Rebalance and migration counters."""
def test_noop(self) -> None:
m = ConsoleMetrics()
m.record_rebalance("noop")
text = m.generate_text()
assert 'turnstone_ring_rebalance_total{result="noop"} 1' in text
def test_seeded(self) -> None:
m = ConsoleMetrics()
m.record_rebalance("seeded")
text = m.generate_text()
assert 'turnstone_ring_rebalance_total{result="seeded"} 1' in text
def test_rebalanced(self) -> None:
m = ConsoleMetrics()
m.record_rebalance("rebalanced")
m.record_rebalance("rebalanced")
text = m.generate_text()
assert 'turnstone_ring_rebalance_total{result="rebalanced"} 2' in text
def test_migrations(self) -> None:
m = ConsoleMetrics()
m.record_migrations(5)
m.record_migrations(3)
text = m.generate_text()
assert "turnstone_ring_migrations_total 8" in text
def test_migrations_default_zero(self) -> None:
m = ConsoleMetrics()
text = m.generate_text()
assert "turnstone_ring_migrations_total 0" in text
assert "turnstone_router_membership_size 3" in text
assert "turnstone_router_refresh_total 7" in text
class TestGenerateText:
@@ -103,10 +64,8 @@ class TestGenerateText:
expected = [
"turnstone_router_requests_total",
"turnstone_router_request_duration_seconds",
"turnstone_ring_membership_size",
"turnstone_ring_version",
"turnstone_ring_rebalance_total",
"turnstone_ring_migrations_total",
"turnstone_router_membership_size",
"turnstone_router_refresh_total",
]
for name in expected:
assert name in text, f"Missing metric: {name}"
@@ -116,8 +75,8 @@ class TestGenerateText:
text = m.generate_text()
assert "# HELP turnstone_router_requests_total" in text
assert "# TYPE turnstone_router_requests_total counter" in text
assert "# HELP turnstone_ring_membership_size" in text
assert "# TYPE turnstone_ring_membership_size gauge" in text
assert "# HELP turnstone_router_membership_size" in text
assert "# TYPE turnstone_router_membership_size gauge" in text
def test_ends_with_newline(self) -> None:
m = ConsoleMetrics()
@@ -125,24 +84,16 @@ class TestGenerateText:
assert text.endswith("\n")
def test_combined_scenario(self) -> None:
"""Full scenario: routes, ring info, rebalances, migrations."""
"""Full scenario: routes + router info."""
m = ConsoleMetrics()
m.record_route("create", 200, 0.1)
m.record_route("send", 200, 0.05)
m.record_route("send", 502, 1.2)
m.set_ring_info(3, 12)
m.record_rebalance("seeded")
m.record_rebalance("noop")
m.record_rebalance("rebalanced")
m.record_migrations(4)
m.set_router_info(3, 12)
text = m.generate_text()
assert 'turnstone_router_requests_total{method="create",status="2xx"} 1' in text
assert 'turnstone_router_requests_total{method="send",status="2xx"} 1' in text
assert 'turnstone_router_requests_total{method="send",status="5xx"} 1' in text
assert "turnstone_ring_membership_size 3" in text
assert "turnstone_ring_version 12" in text
assert 'turnstone_ring_rebalance_total{result="noop"} 1' in text
assert 'turnstone_ring_rebalance_total{result="rebalanced"} 1' in text
assert 'turnstone_ring_rebalance_total{result="seeded"} 1' in text
assert "turnstone_ring_migrations_total 4" in text
assert "turnstone_router_membership_size 3" in text
assert "turnstone_router_refresh_total 12" in text
+353
View File
@@ -0,0 +1,353 @@
"""Tests for console routing of attachment endpoints + multipart route_create.
Covers the cluster-routing surface added alongside the workstream
attachment-on-create feature: the multipart variant of route_create and
the four ws-id-keyed attachment proxies under /v1/api/route/.
"""
from __future__ import annotations
from typing import Any
from unittest.mock import MagicMock
import httpx
from starlette.testclient import TestClient
from turnstone.console.collector import ClusterCollector
from turnstone.console.router import ConsoleRouter, NodeRef
_TEST_JWT_SECRET = "test-jwt-secret-minimum-32-chars!"
def _test_jwt() -> str:
from turnstone.core.auth import JWT_AUD_CONSOLE, create_jwt
return create_jwt(
user_id="test-routing",
scopes=frozenset({"read", "write", "approve", "service"}),
source="test",
secret=_TEST_JWT_SECRET,
audience=JWT_AUD_CONSOLE,
)
_AUTH: dict[str, str] = {"Authorization": f"Bearer {_test_jwt()}"}
def _make_app(router: Any) -> Any:
from turnstone.console.server import _load_static, create_app
_load_static()
collector = MagicMock(spec=ClusterCollector)
return create_app(
collector=collector,
jwt_secret=_TEST_JWT_SECRET,
router=router,
)
def _make_router() -> MagicMock:
router = MagicMock(spec=ConsoleRouter)
router.is_ready.return_value = True
router.route.return_value = NodeRef("node-a", "http://a:8080")
return router
# ---------------------------------------------------------------------------
# route_create multipart
# ---------------------------------------------------------------------------
class TestRouteCreateMultipart:
def test_multipart_requires_ws_id_query(self):
router = _make_router()
app = _make_app(router=router)
app.state.proxy_client = MagicMock(spec=httpx.AsyncClient)
client = TestClient(app, raise_server_exceptions=False)
try:
resp = client.post(
"/v1/api/route/workstreams/new",
files=[("file", ("a.txt", b"hello", "text/plain"))],
data={"meta": "{}"},
headers=_AUTH,
)
assert resp.status_code == 400
assert "ws_id" in resp.json()["error"]
finally:
client.close()
def test_multipart_forwards_raw_body_to_routed_node(self):
router = _make_router()
app = _make_app(router=router)
captured: dict[str, Any] = {}
async def _mock_post(*args: Any, **kwargs: Any) -> httpx.Response:
captured["url"] = args[0] if args else ""
captured["headers"] = kwargs.get("headers") or {}
captured["content"] = kwargs.get("content")
return httpx.Response(
200,
json={"ws_id": "00ff" + "0" * 28, "name": "demo"},
request=httpx.Request("POST", args[0] if args else "http://test"),
)
mock_proxy = MagicMock(spec=httpx.AsyncClient)
mock_proxy.post = MagicMock(side_effect=_mock_post)
app.state.proxy_client = mock_proxy
client = TestClient(app, raise_server_exceptions=False)
try:
ws_id = "00ff" + "0" * 28
resp = client.post(
f"/v1/api/route/workstreams/new?ws_id={ws_id}",
files=[("file", ("a.txt", b"hello", "text/plain"))],
data={"meta": '{"name":"demo"}'},
headers=_AUTH,
)
assert resp.status_code == 200, resp.text
data = resp.json()
assert data["node_id"] == "node-a"
# Forwarded multipart Content-Type
assert captured["headers"].get("Content-Type", "").startswith("multipart/form-data")
# Body bytes were forwarded raw
assert isinstance(captured["content"], (bytes, bytearray))
assert b"hello" in bytes(captured["content"])
router.route.assert_called_with(ws_id)
finally:
client.close()
def test_multipart_preserves_mixed_case_boundary(self):
"""The boundary= param is case-sensitive — must match body bytes verbatim.
Regression for an earlier bug where route_create lowercased the
whole Content-Type header before forwarding, mangling boundaries
like ``WebKitFormBoundary7MA4YWxkTrZu0gW``.
"""
router = _make_router()
app = _make_app(router=router)
captured: dict[str, Any] = {}
async def _mock_post(*args: Any, **kwargs: Any) -> httpx.Response:
captured["headers"] = kwargs.get("headers") or {}
captured["content"] = kwargs.get("content")
return httpx.Response(
200,
json={"ws_id": "00ff" + "0" * 28, "name": "ok"},
request=httpx.Request("POST", args[0] if args else "http://test"),
)
mock_proxy = MagicMock(spec=httpx.AsyncClient)
mock_proxy.post = MagicMock(side_effect=_mock_post)
app.state.proxy_client = mock_proxy
client = TestClient(app, raise_server_exceptions=False)
try:
ws_id = "00ff" + "0" * 28
boundary = "WebKitFormBoundary7MA4YWxkTrZu0gW" # mixed-case
body = (
f"--{boundary}\r\n"
f'Content-Disposition: form-data; name="meta"\r\n\r\n'
f'{{"name":"demo"}}\r\n'
f"--{boundary}\r\n"
f'Content-Disposition: form-data; name="file"; filename="a.txt"\r\n'
f"Content-Type: text/plain\r\n\r\n"
f"hello\r\n"
f"--{boundary}--\r\n"
).encode()
resp = client.post(
f"/v1/api/route/workstreams/new?ws_id={ws_id}",
content=body,
headers={
**_AUTH,
"Content-Type": f"multipart/form-data; boundary={boundary}",
},
)
assert resp.status_code == 200, resp.text
forwarded = captured["headers"].get("Content-Type", "")
assert boundary in forwarded, (
f"boundary mangled in upstream Content-Type: {forwarded!r}"
)
# Body bytes still contain the mixed-case boundary
assert boundary.encode() in bytes(captured["content"])
finally:
client.close()
def test_json_path_unchanged(self):
"""Existing JSON callers should continue to work as before."""
router = _make_router()
app = _make_app(router=router)
async def _mock_post(*args: Any, **kwargs: Any) -> httpx.Response:
return httpx.Response(
200,
json={"ws_id": "abc123", "name": "json"},
request=httpx.Request("POST", args[0] if args else "http://test"),
)
mock_proxy = MagicMock(spec=httpx.AsyncClient)
mock_proxy.post = MagicMock(side_effect=_mock_post)
app.state.proxy_client = mock_proxy
client = TestClient(app, raise_server_exceptions=False)
try:
resp = client.post(
"/v1/api/route/workstreams/new",
json={"name": "json"},
headers=_AUTH,
)
assert resp.status_code == 200
assert resp.json()["ws_id"] == "abc123"
# JSON path uses json= kwarg, not content=
call_kwargs = mock_proxy.post.call_args.kwargs
assert "json" in call_kwargs
assert "content" not in call_kwargs
finally:
client.close()
# ---------------------------------------------------------------------------
# route_attachment_proxy
# ---------------------------------------------------------------------------
class TestRouteAttachmentProxy:
def _wire(self, mock_request_fn) -> tuple[Any, MagicMock]:
router = _make_router()
app = _make_app(router=router)
mock_proxy = MagicMock(spec=httpx.AsyncClient)
mock_proxy.request = MagicMock(side_effect=mock_request_fn)
mock_proxy.get = MagicMock(side_effect=mock_request_fn)
mock_proxy.post = MagicMock(side_effect=mock_request_fn)
app.state.proxy_client = mock_proxy
return app, mock_proxy
def test_upload_proxies_multipart(self):
captured: dict[str, Any] = {}
async def _mock(*args: Any, **kwargs: Any) -> httpx.Response:
captured["method"] = args[0] if args else kwargs.get("method")
captured["url"] = args[1] if len(args) > 1 else kwargs.get("url", "")
captured["headers"] = kwargs.get("headers") or {}
captured["content"] = kwargs.get("content")
return httpx.Response(
200,
json={
"attachment_id": "att-1",
"filename": "a.txt",
"mime_type": "text/plain",
"size_bytes": 5,
"kind": "text",
},
request=httpx.Request("POST", "http://a:8080/x"),
)
app, _ = self._wire(_mock)
client = TestClient(app, raise_server_exceptions=False)
try:
resp = client.post(
"/v1/api/route/workstreams/ws-X/attachments",
files=[("file", ("a.txt", b"hello", "text/plain"))],
headers=_AUTH,
)
assert resp.status_code == 200
assert resp.json()["attachment_id"] == "att-1"
assert "/v1/api/workstreams/ws-X/attachments" in captured["url"]
assert "/route/" not in captured["url"]
assert captured["headers"].get("Content-Type", "").startswith("multipart/form-data")
finally:
client.close()
def test_list_proxies_get(self):
async def _mock(*args: Any, **kwargs: Any) -> httpx.Response:
return httpx.Response(
200,
json={"attachments": []},
request=httpx.Request("GET", "http://a:8080/x"),
)
app, mock_proxy = self._wire(_mock)
client = TestClient(app, raise_server_exceptions=False)
try:
resp = client.get(
"/v1/api/route/workstreams/ws-X/attachments",
headers=_AUTH,
)
assert resp.status_code == 200
assert resp.json() == {"attachments": []}
mock_proxy.get.assert_called()
finally:
client.close()
def test_get_content_preserves_upstream_headers(self):
async def _mock(*args: Any, **kwargs: Any) -> httpx.Response:
return httpx.Response(
200,
content=b"hello world",
headers={
"Content-Type": "text/plain; charset=utf-8",
"Content-Disposition": 'inline; filename="notes.md"',
"X-Content-Type-Options": "nosniff",
},
request=httpx.Request("GET", "http://a:8080/x"),
)
app, _ = self._wire(_mock)
client = TestClient(app, raise_server_exceptions=False)
try:
resp = client.get(
"/v1/api/route/workstreams/ws-X/attachments/att-1/content",
headers=_AUTH,
)
assert resp.status_code == 200
assert resp.content == b"hello world"
assert resp.headers.get("X-Content-Type-Options") == "nosniff"
assert "filename" in resp.headers.get("Content-Disposition", "")
finally:
client.close()
def test_delete_proxies_method(self):
captured: dict[str, Any] = {}
async def _mock(*args: Any, **kwargs: Any) -> httpx.Response:
captured["method"] = args[0] if args else ""
captured["url"] = args[1] if len(args) > 1 else ""
return httpx.Response(
200,
json={"status": "deleted"},
request=httpx.Request("DELETE", "http://a:8080/x"),
)
app, _ = self._wire(_mock)
client = TestClient(app, raise_server_exceptions=False)
try:
resp = client.delete(
"/v1/api/route/workstreams/ws-X/attachments/att-1",
headers=_AUTH,
)
assert resp.status_code == 200
assert resp.json() == {"status": "deleted"}
assert captured["method"] == "DELETE"
finally:
client.close()
# ---------------------------------------------------------------------------
# Routing-failure paths
# ---------------------------------------------------------------------------
class TestRoutingFailures:
def test_router_not_ready_returns_503(self):
router = MagicMock(spec=ConsoleRouter)
router.is_ready.return_value = False
router.refresh_cache.return_value = None
app = _make_app(router=router)
app.state.proxy_client = MagicMock(spec=httpx.AsyncClient)
client = TestClient(app, raise_server_exceptions=False)
try:
resp = client.get(
"/v1/api/route/workstreams/ws-X/attachments",
headers=_AUTH,
)
assert resp.status_code == 503
finally:
client.close()
+178 -138
View File
@@ -1,17 +1,13 @@
"""Tests for turnstone.console.router."""
"""Tests for turnstone.console.router (rendezvous routing)."""
from __future__ import annotations
from typing import Any
import secrets
import pytest
from turnstone.console.router import ConsoleRouter, NodeRef
from turnstone.core.hash_ring import RING_SIZE, NoAvailableNodeError
# ---------------------------------------------------------------------------
# Fake storage
# ---------------------------------------------------------------------------
from turnstone.core.rendezvous import NoAvailableNodeError
class FakeStorage:
@@ -19,26 +15,14 @@ class FakeStorage:
def __init__(self) -> None:
self.services: list[dict[str, str]] = []
self.buckets: list[dict[str, Any]] = []
self.overrides: list[dict[str, str]] = []
self.settings: dict[str, dict[str, Any]] = {}
def list_services(self, service_type: str, max_age_seconds: int = 120) -> list[dict[str, str]]:
return list(self.services)
def list_ring_buckets(self) -> list[dict[str, Any]]:
return list(self.buckets)
def list_workstream_overrides(self) -> list[dict[str, str]]:
return list(self.overrides)
def get_system_setting(self, key: str, node_id: str = "") -> dict[str, Any] | None:
return self.settings.get(key)
# ---------------------------------------------------------------------------
# Helpers
# ---------------------------------------------------------------------------
NODE_A = {"service_id": "node-a", "url": "http://a:8080", "metadata": "{}"}
NODE_B = {"service_id": "node-b", "url": "http://b:8080", "metadata": "{}"}
@@ -50,206 +34,262 @@ def _make_router(storage: FakeStorage | None = None) -> tuple[ConsoleRouter, Fak
return ConsoleRouter(s), s # type: ignore[arg-type]
def _ws_id_for_bucket(bucket: int) -> str:
"""Build a 32-char hex ws_id whose first 4 chars encode *bucket*."""
return f"{bucket:04x}" + "0" * 28
# ---------------------------------------------------------------------------
# TestRouteBasic
# ---------------------------------------------------------------------------
def _random_ws_id() -> str:
return secrets.token_hex(16)
class TestRouteBasic:
"""Basic routing through the bucket cache."""
def test_route_returns_correct_node(self) -> None:
def test_route_returns_a_live_node(self) -> None:
router, storage = _make_router()
storage.services = [NODE_A, NODE_B, NODE_C]
storage.buckets = [
{"bucket": 0x0000, "node_id": "node-a"},
{"bucket": 0x0001, "node_id": "node-b"},
{"bucket": 0x0002, "node_id": "node-c"},
]
router.refresh_cache()
assert router.route(_ws_id_for_bucket(0x0000)) == NodeRef("node-a", "http://a:8080")
assert router.route(_ws_id_for_bucket(0x0001)) == NodeRef("node-b", "http://b:8080")
assert router.route(_ws_id_for_bucket(0x0002)) == NodeRef("node-c", "http://c:8080")
ref = router.route(_random_ws_id())
assert ref.node_id in {"node-a", "node-b", "node-c"}
def test_route_is_deterministic_for_same_ws_id(self) -> None:
"""Same ws_id + same membership → same target every time."""
router, storage = _make_router()
storage.services = [NODE_A, NODE_B, NODE_C]
router.refresh_cache()
ws_id = _random_ws_id()
first = router.route(ws_id)
for _ in range(50):
assert router.route(ws_id) == first
def test_route_override_priority(self) -> None:
router, storage = _make_router()
storage.services = [NODE_A, NODE_B]
storage.buckets = [{"bucket": 0x0000, "node_id": "node-a"}]
ws_id = _ws_id_for_bucket(0x0000)
ws_id = _random_ws_id()
storage.overrides = [{"ws_id": ws_id, "node_id": "node-b"}]
router.refresh_cache()
# Override wins over bucket assignment
# Override wins regardless of HRW score.
assert router.route(ws_id) == NodeRef("node-b", "http://b:8080")
def test_route_empty_cache_raises(self) -> None:
def test_route_empty_membership_raises(self) -> None:
router, _ = _make_router()
with pytest.raises(NoAvailableNodeError):
router.route(_random_ws_id())
with pytest.raises(NoAvailableNodeError, match="not assigned"):
router.route(_ws_id_for_bucket(0x0000))
def test_route_empty_ws_id_raises(self) -> None:
router, storage = _make_router()
storage.services = [NODE_A]
router.refresh_cache()
with pytest.raises(NoAvailableNodeError, match="empty"):
router.route("")
def test_route_url_convenience(self) -> None:
router, storage = _make_router()
storage.services = [NODE_A]
storage.buckets = [{"bucket": 0x0010, "node_id": "node-a"}]
router.refresh_cache()
assert router.route_url(_random_ws_id()) == "http://a:8080"
class TestMembershipConvergence:
"""Rendezvous gives the minimal-moves property; pin it."""
def test_node_join_only_steals_some_keys(self) -> None:
"""Adding a 4th node moves ~1/4 of keys to it; the other 3
nodes' kept keys are unchanged."""
router, storage = _make_router()
storage.services = [NODE_A, NODE_B, NODE_C]
router.refresh_cache()
assert router.route_url(_ws_id_for_bucket(0x0010)) == "http://a:8080"
sample = [_random_ws_id() for _ in range(2000)]
before = {ws: router.route(ws).node_id for ws in sample}
storage.services = [
NODE_A,
NODE_B,
NODE_C,
{"service_id": "node-d", "url": "http://d:8080", "metadata": "{}"},
]
router.refresh_cache()
after = {ws: router.route(ws).node_id for ws in sample}
# ---------------------------------------------------------------------------
# TestRefreshCache
# ---------------------------------------------------------------------------
moved = sum(1 for ws in sample if before[ws] != after[ws])
moved_to_new = sum(1 for ws in sample if after[ws] == "node-d")
# Every move must be onto the new node — no churn between
# existing nodes.
assert moved == moved_to_new
# Should be roughly 1/4 of keys; allow a wide band for variance.
assert 0.15 < moved / len(sample) < 0.35
class TestRefreshCache:
"""Cache loading from storage."""
def test_refresh_loads_from_storage(self) -> None:
def test_node_leave_only_redistributes_dead_node_keys(self) -> None:
"""Removing node-a sends node-a's keys to b/c only; keys that
were on b/c stay put."""
router, storage = _make_router()
storage.services = [NODE_A]
storage.buckets = [{"bucket": 100, "node_id": "node-a"}]
storage.services = [NODE_A, NODE_B, NODE_C]
router.refresh_cache()
ref = router.route(_ws_id_for_bucket(100))
assert ref.node_id == "node-a"
sample = [_random_ws_id() for _ in range(2000)]
before = {ws: router.route(ws).node_id for ws in sample}
def test_refresh_handles_dead_nodes(self) -> None:
storage.services = [NODE_B, NODE_C]
router.refresh_cache()
after = {ws: router.route(ws).node_id for ws in sample}
for ws in sample:
if before[ws] in ("node-b", "node-c"):
assert after[ws] == before[ws], (
f"key {ws} moved from {before[ws]} to {after[ws]} "
"even though its old owner is still live"
)
else: # was on node-a
assert after[ws] in ("node-b", "node-c")
class TestWeights:
def test_weight_2_node_gets_more_keys_than_weight_1(self) -> None:
router, storage = _make_router()
# node-b is in buckets but not in services (dead/expired)
storage.services = [NODE_A]
storage.buckets = [
{"bucket": 0x0000, "node_id": "node-a"},
{"bucket": 0x0001, "node_id": "node-b"},
storage.services = [
{"service_id": "node-a", "url": "http://a:8080", "metadata": '{"weight": 2}'},
{"service_id": "node-b", "url": "http://b:8080", "metadata": '{"weight": 1}'},
]
router.refresh_cache()
assert router.route(_ws_id_for_bucket(0x0000)).node_id == "node-a"
with pytest.raises(NoAvailableNodeError):
router.route(_ws_id_for_bucket(0x0001))
sample = [_random_ws_id() for _ in range(5000)]
on_a = sum(1 for ws in sample if router.route(ws).node_id == "node-a")
# Heavier node should win clearly more than half; exact ratio
# depends on the simple hash×weight formulation but a/b > 1.4
# for weight 2:1 across 5k samples is reliable.
assert on_a / len(sample) > 0.55
def test_refresh_returns_true_on_change(self) -> None:
def test_invalid_metadata_falls_back_to_weight_1(self) -> None:
router, storage = _make_router()
storage.services = [NODE_A]
storage.buckets = [{"bucket": 0, "node_id": "node-a"}]
assert router.refresh_cache() is True
def test_refresh_returns_false_on_no_change(self) -> None:
router, storage = _make_router()
storage.services = [NODE_A]
storage.buckets = [{"bucket": 0, "node_id": "node-a"}]
storage.services = [
{"service_id": "node-a", "url": "http://a:8080", "metadata": "not json"},
]
router.refresh_cache()
assert router.refresh_cache() is False
# Just confirms it doesn't blow up.
router.route(_random_ws_id())
# ---------------------------------------------------------------------------
# TestCheckVersion
# ---------------------------------------------------------------------------
class TestCheckVersion:
"""Version-gated refresh."""
def test_version_change_triggers_refresh(self) -> None:
class TestRefreshLifecycle:
def test_refresh_cache_publishes_new_membership_immediately(self) -> None:
"""refresh_cache() reloads on the calling thread — the next
route() sees the new membership without any further trigger."""
router, storage = _make_router()
storage.services = [NODE_A]
storage.buckets = [{"bucket": 0, "node_id": "node-a"}]
storage.settings["rebalancer_version"] = {"value": "1"}
router.refresh_cache()
assert router.node_count() == 1
assert router.check_version() is True
assert router.is_ready()
storage.services = [NODE_A, NODE_B]
router.refresh_cache()
assert router.node_count() == 2
def test_concurrent_refresh_returns_false_on_lock_contention(self) -> None:
"""refresh_cache uses a non-blocking lock acquire — if another
thread is already refreshing, the second caller bails so the
in-flight refresh's result is the one that publishes."""
def test_same_version_skips(self) -> None:
router, storage = _make_router()
# Default version is 0; setting absent also means 0
storage.services = [NODE_A]
storage.buckets = [{"bucket": 0, "node_id": "node-a"}]
# First call: version=0 matches self._version=0 -> no refresh
assert router.check_version() is False
assert not router.is_ready() # cache was never loaded
with router._refresh_lock:
# Lock held by this thread → the call below can't acquire.
assert router.refresh_cache() is False
def test_force_refresh_blocks_until_in_flight_refresh_releases(self) -> None:
"""force_refresh acquires the refresh lock blocking — used by the
404-retry path to guarantee a fresh view even under contention."""
import threading
def test_version_none_treated_as_zero(self) -> None:
router, storage = _make_router()
# settings dict is empty -> get_system_setting returns None
assert router.check_version() is False
storage.services = [NODE_A]
# Hold the refresh lock from another thread.
lock_held = threading.Event()
release = threading.Event()
# ---------------------------------------------------------------------------
# TestGenerateWsId
# ---------------------------------------------------------------------------
def hold_lock() -> None:
with router._refresh_lock:
lock_held.set()
release.wait(timeout=2)
holder = threading.Thread(target=hold_lock, daemon=True)
holder.start()
assert lock_held.wait(timeout=1)
# force_refresh should block, not bail.
result_box: list[bool] = []
def call_force() -> None:
result_box.append(router.force_refresh())
caller = threading.Thread(target=call_force, daemon=True)
caller.start()
caller.join(timeout=0.2)
assert caller.is_alive(), "force_refresh returned without acquiring lock"
release.set()
holder.join(timeout=1)
caller.join(timeout=1)
assert not caller.is_alive()
# Membership changed from empty → 1 live node.
assert result_box == [True]
assert router.node_count() == 1
def test_force_refresh_always_reloads(self) -> None:
"""force_refresh skips the non-blocking-lock bail and always
publishes a fresh view back-to-back calls each pick up the
latest storage state."""
router, storage = _make_router()
storage.services = [NODE_A]
router.force_refresh()
assert router.node_count() == 1
storage.services = [NODE_A, NODE_B]
router.force_refresh()
assert router.node_count() == 2
def test_version_is_monotonic_across_refreshes(self) -> None:
router, storage = _make_router()
storage.services = [NODE_A]
router.refresh_cache()
v1 = router.version
router.refresh_cache()
v2 = router.version
assert v2 > v1
router.force_refresh()
assert router.version > v2
class TestGenerateWsId:
"""Workstream ID generation targeting a specific node."""
def test_generates_routable_id(self) -> None:
router, storage = _make_router()
storage.services = [NODE_A, NODE_B]
storage.buckets = [
{"bucket": 0x00FF, "node_id": "node-a"},
{"bucket": 0x0100, "node_id": "node-b"},
]
storage.services = [NODE_A, NODE_B, NODE_C]
router.refresh_cache()
ws_id = router.generate_ws_id_for_node("node-a")
ws_id = router.generate_ws_id_for_node("node-b")
assert len(ws_id) == 32
assert router.route(ws_id).node_id == "node-a"
assert router.route(ws_id).node_id == "node-b"
def test_unknown_node_raises(self) -> None:
router, storage = _make_router()
storage.services = [NODE_A]
storage.buckets = [{"bucket": 0, "node_id": "node-a"}]
router.refresh_cache()
with pytest.raises(NoAvailableNodeError, match="node-z"):
router.generate_ws_id_for_node("node-z")
# ---------------------------------------------------------------------------
# TestIsReady
# ---------------------------------------------------------------------------
class TestIsReady:
"""Readiness checks."""
def test_false_when_empty(self) -> None:
router, _ = _make_router()
assert router.is_ready() is False
def test_true_after_refresh(self) -> None:
def test_true_after_membership_loads(self) -> None:
router, storage = _make_router()
storage.services = [NODE_A]
storage.buckets = [{"bucket": 0, "node_id": "node-a"}]
router.refresh_cache()
assert router.is_ready() is True
# ---------------------------------------------------------------------------
# TestNodeCount
# ---------------------------------------------------------------------------
class TestNodeCount:
"""Distinct node counting."""
def test_count_distinct_nodes(self) -> None:
def test_count_matches_live_services(self) -> None:
router, storage = _make_router()
storage.services = [NODE_A, NODE_B, NODE_C]
# Spread all 65536 buckets across 3 nodes
storage.buckets = [
{"bucket": b, "node_id": f"node-{['a', 'b', 'c'][b % 3]}"} for b in range(RING_SIZE)
]
router.refresh_cache()
assert router.node_count() == 3
+45 -2
View File
@@ -11,7 +11,7 @@ from starlette.testclient import TestClient
from turnstone.console.collector import ClusterCollector
from turnstone.console.router import ConsoleRouter, NodeRef
from turnstone.core.hash_ring import NoAvailableNodeError
from turnstone.core.rendezvous import NoAvailableNodeError
# Shared test auth — JWT-based
_TEST_JWT_SECRET = "test-jwt-secret-minimum-32-chars!"
@@ -102,7 +102,7 @@ def _wire_proxy(app: Any, mock_post: MagicMock | None = None) -> None:
class TestRouteCreate:
"""POST /v1/api/route/workstreams/new — create via hash-ring routing."""
"""POST /v1/api/route/workstreams/new — create via rendezvous routing."""
@pytest.fixture()
def client(self):
@@ -178,6 +178,49 @@ class TestRouteCreate:
router.generate_ws_id_for_node.assert_called_with("node-c")
client.close()
def test_route_create_routing_strategy_rendezvous(self, client):
"""Default fan-out (no resume_ws / no target_node) reports
routing_strategy='rendezvous' so the coordinator's spawn tool
can explain why the node was chosen."""
resp = client.post(
"/v1/api/route/workstreams/new",
json={"name": "test-ws"},
headers=_TEST_AUTH_HEADERS,
)
assert resp.status_code == 200
assert resp.json()["routing_strategy"] == "rendezvous"
def test_route_create_routing_strategy_target_node(self):
router = _make_mock_router()
router.generate_ws_id_for_node.return_value = "00ff" + "0" * 28
router.route.return_value = NodeRef("node-c", "http://c:8080")
app = _make_app(router=router)
_wire_proxy(app, _make_proxy_post(json_data={"ws_id": "00ff" + "0" * 28, "name": "pinned"}))
client = TestClient(app, raise_server_exceptions=False)
resp = client.post(
"/v1/api/route/workstreams/new",
json={"target_node": "node-c"},
headers=_TEST_AUTH_HEADERS,
)
assert resp.status_code == 200
assert resp.json()["routing_strategy"] == "target_node"
client.close()
def test_route_create_routing_strategy_resume(self):
router = _make_mock_router()
router.route.return_value = NodeRef("node-b", "http://b:8080")
app = _make_app(router=router)
_wire_proxy(app, _make_proxy_post(json_data={"ws_id": "old_ws_resumed", "name": "resumed"}))
client = TestClient(app, raise_server_exceptions=False)
resp = client.post(
"/v1/api/route/workstreams/new",
json={"resume_ws": "old_ws_id"},
headers=_TEST_AUTH_HEADERS,
)
assert resp.status_code == 200
assert resp.json()["routing_strategy"] == "resume"
client.close()
class TestRouteCreate503Retry:
"""503 retry logic in route_create."""
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,234 @@
"""Tests for the coordinator ``close_all_children`` endpoint.
Near-twin of the ``stop_cascade`` tests in
``test_coordinator_governance.py``. Keeps the close-cascade surface in
its own file so PR A's review surface stays tight.
"""
from __future__ import annotations
import json
from unittest.mock import MagicMock
import pytest
from starlette.applications import Starlette
from starlette.middleware import Middleware
from starlette.routing import Route
from starlette.testclient import TestClient
from tests._coord_test_helpers import (
_AuthMiddleware,
_build_mgr,
_fake_registry,
_FakeConfigStore,
)
from turnstone.console.server import coordinator_close_all_children
from turnstone.core.storage._sqlite import SQLiteBackend
@pytest.fixture
def storage(tmp_path):
return SQLiteBackend(str(tmp_path / "coord.db"))
_COORD_HEADERS = {"X-Test-User": "user-1", "X-Test-Perms": "admin.coordinator"}
def _make_client(storage, *, coord_mgr, alias="my-model", registry=None) -> TestClient:
app = Starlette(
routes=[
Route(
"/v1/api/coordinator/{ws_id}/close_all_children",
coordinator_close_all_children,
methods=["POST"],
),
],
middleware=[Middleware(_AuthMiddleware)],
)
app.state.coord_mgr = coord_mgr
app.state.config_store = _FakeConfigStore({"coordinator.model_alias": alias})
app.state.coord_registry = registry
app.state.coord_registry_error = "" if coord_mgr else "registry missing"
app.state.auth_storage = storage
app.state.jwt_secret = "x" * 64
return TestClient(app)
def test_close_all_children_closes_each_child_and_audits(storage):
mgr = _build_mgr(storage)
coord = mgr.create(user_id="user-1", name="coord-a")
mgr.register_children(coord.id, ["child-1", "child-2", "child-3"])
def _close(wid, reason):
if wid == "child-2":
return {"error": "gateway_timeout", "status": 502}
return {"status": "ok"}
coord_client = MagicMock()
coord_client.close_workstream.side_effect = _close
coord.session = MagicMock()
coord.session._coord_client = coord_client
client = _make_client(storage, coord_mgr=mgr, registry=_fake_registry())
resp = client.post(
f"/v1/api/coordinator/{coord.id}/close_all_children",
json={"reason": "tests done"},
headers=_COORD_HEADERS,
)
assert resp.status_code == 200
body = resp.json()
assert set(body["closed"] + body["failed"] + body["skipped"]) == {
"child-1",
"child-2",
"child-3",
}
assert body["failed"] == ["child-2"]
assert set(body["closed"]) == {"child-1", "child-3"}
assert body["skipped"] == []
assert coord_client.close_workstream.call_count == 3
# Reason must propagate to each per-child close call.
for call in coord_client.close_workstream.call_args_list:
assert call.args[1] == "tests done"
events = [
e for e in storage.list_audit_events() if e["action"] == "coordinator.closed_all_children"
]
assert len(events) == 1
detail = json.loads(events[0]["detail"])
assert detail["reason"] == "tests done"
assert set(detail["closed"] + detail["failed"] + detail["skipped"]) == {
"child-1",
"child-2",
"child-3",
}
def test_close_all_children_routes_404_to_skipped_bucket(storage):
"""An upstream 404 (child row already deleted, stale registry entry)
is 'already gone', not a dispatch failure. Route to skipped."""
mgr = _build_mgr(storage)
coord = mgr.create(user_id="user-1", name="coord-a")
mgr.register_children(coord.id, ["stale-child"])
coord_client = MagicMock()
coord_client.close_workstream.return_value = {
"error": "workstream not in coordinator subtree: stale-child",
"status": 404,
}
coord.session = MagicMock()
coord.session._coord_client = coord_client
client = _make_client(storage, coord_mgr=mgr, registry=_fake_registry())
resp = client.post(
f"/v1/api/coordinator/{coord.id}/close_all_children",
json={},
headers=_COORD_HEADERS,
)
assert resp.status_code == 200
body = resp.json()
assert body["closed"] == []
assert body["failed"] == []
assert body["skipped"] == ["stale-child"]
def test_close_all_children_empty_children_still_audits(storage):
mgr = _build_mgr(storage)
coord = mgr.create(user_id="user-1", name="coord-a")
coord.session = MagicMock()
coord.session._coord_client = MagicMock()
client = _make_client(storage, coord_mgr=mgr, registry=_fake_registry())
resp = client.post(
f"/v1/api/coordinator/{coord.id}/close_all_children",
json={},
headers=_COORD_HEADERS,
)
assert resp.status_code == 200
body = resp.json()
assert body == {"status": "ok", "closed": [], "failed": [], "skipped": []}
assert [
e for e in storage.list_audit_events() if e["action"] == "coordinator.closed_all_children"
]
def test_close_all_children_without_coord_client_marks_all_failed(storage):
mgr = _build_mgr(storage)
coord = mgr.create(user_id="user-1", name="coord-a")
mgr.register_children(coord.id, ["child-a", "child-b"])
coord.session = MagicMock()
coord.session._coord_client = None
client = _make_client(storage, coord_mgr=mgr, registry=_fake_registry())
resp = client.post(
f"/v1/api/coordinator/{coord.id}/close_all_children",
json={},
headers=_COORD_HEADERS,
)
assert resp.status_code == 200
body = resp.json()
assert body["closed"] == []
assert body["skipped"] == []
assert set(body["failed"]) == {"child-a", "child-b"}
def test_close_all_children_rejects_non_string_reason(storage):
mgr = _build_mgr(storage)
coord = mgr.create(user_id="user-1", name="coord-a")
coord.session = MagicMock()
coord.session._coord_client = MagicMock()
client = _make_client(storage, coord_mgr=mgr, registry=_fake_registry())
resp = client.post(
f"/v1/api/coordinator/{coord.id}/close_all_children",
json={"reason": 123},
headers=_COORD_HEADERS,
)
assert resp.status_code == 400
def test_close_all_children_rejects_overlong_reason(storage):
mgr = _build_mgr(storage)
coord = mgr.create(user_id="user-1", name="coord-a")
coord.session = MagicMock()
coord.session._coord_client = MagicMock()
client = _make_client(storage, coord_mgr=mgr, registry=_fake_registry())
resp = client.post(
f"/v1/api/coordinator/{coord.id}/close_all_children",
json={"reason": "x" * 600},
headers=_COORD_HEADERS,
)
assert resp.status_code == 400
def test_close_all_children_404_when_session_not_loaded(storage):
mgr = _build_mgr(storage)
coord = mgr.create(user_id="user-1", name="coord-a")
coord.session = None
client = _make_client(storage, coord_mgr=mgr, registry=_fake_registry())
resp = client.post(
f"/v1/api/coordinator/{coord.id}/close_all_children",
json={},
headers=_COORD_HEADERS,
)
assert resp.status_code == 404
def test_close_all_children_service_token_cannot_bypass_admin_coordinator(storage):
"""Destructive endpoint — a service token matching the coord owner
still needs the explicit ``admin.coordinator`` grant. Mirrors the
stop_cascade treatment."""
mgr = _build_mgr(storage)
coord = mgr.create(user_id="user-1", name="coord-a")
coord.session = MagicMock()
coord.session._coord_client = MagicMock()
# Service token without admin.coordinator should be rejected.
headers = {"X-Test-User": "user-1", "X-Test-Perms": ""}
client = _make_client(storage, coord_mgr=mgr, registry=_fake_registry())
resp = client.post(
f"/v1/api/coordinator/{coord.id}/close_all_children",
json={},
headers=headers,
)
assert resp.status_code in (401, 403)
+435
View File
@@ -0,0 +1,435 @@
"""End-to-end integration tests for the coordinator workstream feature.
Tests cover the full create inspect list close lifecycle using
real in-process components:
1. Create + list + detail round-trip via the Starlette TestClient.
2. CoordinatorClient against a MockTransport "server node" stub.
3. list_children storage read flow (kind filtering, parent scoping).
4. Lazy rehydration via GET /v1/api/coordinator/{ws_id}.
Intentionally no real LLM infrastructure session factories return
MagicMock-backed stubs. All four tests run in < 2 s total.
"""
from __future__ import annotations
import json
from typing import Any
from unittest.mock import MagicMock
import httpx
import pytest
from starlette.applications import Starlette
from starlette.middleware import Middleware
from starlette.middleware.base import BaseHTTPMiddleware
from starlette.routing import Route
from starlette.testclient import TestClient
from turnstone.console.coordinator import CoordinatorManager
from turnstone.console.coordinator_client import CoordinatorClient
from turnstone.console.coordinator_ui import ConsoleCoordinatorUI
from turnstone.console.server import (
coordinator_close,
coordinator_create,
coordinator_detail,
coordinator_list,
)
from turnstone.core.auth import AuthResult
from turnstone.core.storage._sqlite import SQLiteBackend
# ---------------------------------------------------------------------------
# Shared auth-injection middleware (mirrors test_coordinator_endpoints.py)
# ---------------------------------------------------------------------------
class _AuthMiddleware(BaseHTTPMiddleware):
"""Inject an ``AuthResult`` from ``X-Test-Perms`` / ``X-Test-User``."""
async def dispatch(self, request, call_next):
perms = request.headers.get("X-Test-Perms", "")
user_id = request.headers.get("X-Test-User", "")
if perms or user_id:
request.state.auth_result = AuthResult(
user_id=user_id,
scopes=frozenset({"approve"}),
token_source="test",
permissions=frozenset(p for p in perms.split(",") if p),
)
return await call_next(request)
# ---------------------------------------------------------------------------
# Shared stubs
# ---------------------------------------------------------------------------
class _FakeConfigStore:
"""Minimal ConfigStore stub returning values from a dict."""
def __init__(self, values: dict[str, Any]) -> None:
self._values = values
def get(self, key: str, default: Any = None) -> Any:
return self._values.get(key, default)
def _fake_registry() -> MagicMock:
"""Registry stub that always succeeds on .resolve() so the 503 gate passes."""
reg = MagicMock()
reg.resolve.return_value = (MagicMock(), "gpt-test", MagicMock())
return reg
def _build_mgr(storage: SQLiteBackend) -> CoordinatorManager:
"""Build a CoordinatorManager backed by stub factories."""
def _sf(ui, model_alias=None, ws_id=None, **kw):
s = MagicMock()
s.ws_id = ws_id
s.send.return_value = None
return s
return CoordinatorManager(
session_factory=_sf,
ui_factory=lambda w, u: ConsoleCoordinatorUI(ws_id=w, user_id=u),
storage=storage,
max_active=5,
)
def _make_client(
storage: SQLiteBackend,
*,
coord_mgr: CoordinatorManager | None = None,
alias: str = "my-model",
registry: Any = None,
) -> TestClient:
"""Build a Starlette TestClient exposing the coordinator routes."""
app = Starlette(
routes=[
Route(
"/v1/api/coordinator/new",
coordinator_create,
methods=["POST"],
),
Route("/v1/api/coordinator", coordinator_list, methods=["GET"]),
Route(
"/v1/api/coordinator/{ws_id}/close",
coordinator_close,
methods=["POST"],
),
Route(
"/v1/api/coordinator/{ws_id}",
coordinator_detail,
methods=["GET"],
),
],
middleware=[Middleware(_AuthMiddleware)],
)
app.state.coord_mgr = coord_mgr
app.state.config_store = _FakeConfigStore({"coordinator.model_alias": alias})
app.state.coord_registry = registry
app.state.coord_registry_error = "" if coord_mgr else "registry missing"
app.state.auth_storage = storage
app.state.jwt_secret = "x" * 64
return TestClient(app)
# ---------------------------------------------------------------------------
# Test 1 — Create + list + detail round-trip
# ---------------------------------------------------------------------------
_COORD_HEADERS = {"X-Test-User": "user-1", "X-Test-Perms": "admin.coordinator"}
def test_create_list_detail_lifecycle(tmp_path):
"""POST /new → appears in GET / → GET /{ws_id} returns correct detail."""
storage = SQLiteBackend(str(tmp_path / "coord.db"))
mgr = _build_mgr(storage)
client = _make_client(storage, coord_mgr=mgr, registry=_fake_registry())
# --- Create ---
resp = client.post(
"/v1/api/coordinator/new",
json={"name": "e2e-coord"},
headers=_COORD_HEADERS,
)
assert resp.status_code == 201, resp.text
body = resp.json()
ws_id = body["ws_id"]
assert ws_id
assert "e2e-coord" in body["name"]
# --- List: caller sees their own coordinator ---
resp = client.get("/v1/api/coordinator", headers=_COORD_HEADERS)
assert resp.status_code == 200, resp.text
coordinators = resp.json()["coordinators"]
ids = {c["ws_id"] for c in coordinators}
assert ws_id in ids
# Coordinator created by a different user is invisible to our caller.
mgr.create(user_id="other-user", name="not-mine")
resp = client.get("/v1/api/coordinator", headers=_COORD_HEADERS)
assert resp.status_code == 200
names = {c["name"] for c in resp.json()["coordinators"]}
assert "not-mine" not in names
# --- Detail ---
resp = client.get(f"/v1/api/coordinator/{ws_id}", headers=_COORD_HEADERS)
assert resp.status_code == 200, resp.text
detail = resp.json()
assert detail["ws_id"] == ws_id
assert detail["kind"] == "coordinator"
assert detail["user_id"] == "user-1"
# --- Close ---
resp = client.post(f"/v1/api/coordinator/{ws_id}/close", headers=_COORD_HEADERS)
assert resp.status_code == 200
# Manager no longer tracks it after close.
assert mgr.get(ws_id) is None
# Storage row reflects closed state.
row = storage.get_workstream(ws_id)
assert row is not None
assert row["state"] == "closed"
# Detail endpoint returns 404 after close (not in memory, not rehydratable
# from a "closed" row — well, the manager would rehydrate it but let's verify
# the row is gone from the in-memory index).
assert mgr.get(ws_id) is None
# ---------------------------------------------------------------------------
# Test 2 — CoordinatorClient against a MockTransport "server node" stub
# ---------------------------------------------------------------------------
def test_coordinator_client_spawn_close_delete(tmp_path):
"""CoordinatorClient.spawn / close_workstream / delete produce correct
upstream HTTP requests to the mocked server node."""
storage = SQLiteBackend(str(tmp_path / "client.db"))
# Register the coordinator + the soon-to-be-spawned child so the
# client-side tenant guard on close/delete passes. In production
# the spawn route adds the child row before the model can call
# close on it; the test stub doesn't run that side-effect, so we
# set it up here.
storage.register_workstream("coord-42", kind="coordinator", user_id="user-1")
storage.register_workstream(
"child-99", kind="interactive", parent_ws_id="coord-42", user_id="user-1"
)
captured: list[httpx.Request] = []
def _handler(req: httpx.Request) -> httpx.Response:
captured.append(req)
path = req.url.path
if path == "/v1/api/route/workstreams/new":
return httpx.Response(
201,
json={"ws_id": "child-99", "name": "spawned", "node_id": "node-a"},
)
# close and delete both return a generic ok
return httpx.Response(200, json={"status": "ok"})
transport = httpx.MockTransport(_handler)
http = httpx.Client(transport=transport)
coord_client = CoordinatorClient(
console_base_url="http://console",
storage=storage,
token_factory=lambda: "bearer-test-token",
coord_ws_id="coord-42",
user_id="user-1",
http_client=http,
)
# spawn ---------------------------------------------------------------
result = coord_client.spawn(
initial_message="analyse data",
parent_ws_id="coord-42",
user_id="user-1",
skill="data-skill",
target_node="node-a",
)
assert result["ws_id"] == "child-99"
spawn_req = captured[0]
assert spawn_req.method == "POST"
assert spawn_req.url.path == "/v1/api/route/workstreams/new"
assert spawn_req.headers["Authorization"] == "Bearer bearer-test-token"
spawn_body = json.loads(spawn_req.content)
assert spawn_body["kind"] == "interactive"
assert spawn_body["parent_ws_id"] == "coord-42"
assert spawn_body["user_id"] == "user-1"
assert spawn_body["initial_message"] == "analyse data"
assert spawn_body["skill"] == "data-skill"
assert spawn_body["target_node"] == "node-a"
# close_workstream ----------------------------------------------------
captured.clear()
close_result = coord_client.close_workstream("child-99")
assert close_result.get("status") in (200, "ok"), close_result
close_req = captured[0]
assert close_req.url.path == "/v1/api/route/workstreams/close"
close_body = json.loads(close_req.content)
assert close_body["ws_id"] == "child-99"
# delete --------------------------------------------------------------
captured.clear()
del_result = coord_client.delete("child-99")
assert del_result.get("status") in (200, "ok"), del_result
del_req = captured[0]
assert del_req.url.path == "/v1/api/route/workstreams/delete"
del_body = json.loads(del_req.content)
assert del_body["ws_id"] == "child-99"
# ---------------------------------------------------------------------------
# Test 3 — list_children storage read: kind filtering + parent scoping
# ---------------------------------------------------------------------------
@pytest.fixture()
def seeded_storage(tmp_path):
"""SQLiteBackend with a coordinator + 2 interactive children + extras."""
st = SQLiteBackend(str(tmp_path / "seed.db"))
# Parent coordinator.
st.register_workstream("coord-root", kind="coordinator", user_id="user-1")
# Two interactive children — one idle, one running. Children inherit
# the coord's user_id by construction (server-side create gate), which
# the list_children SQL filter now enforces.
st.register_workstream(
"child-idle",
kind="interactive",
parent_ws_id="coord-root",
state="idle",
skill_id="skill-alpha",
user_id="user-1",
)
st.register_workstream(
"child-running",
kind="interactive",
parent_ws_id="coord-root",
state="running",
skill_id="skill-beta",
user_id="user-1",
)
# Coordinator child — MUST be excluded from list_children results.
st.register_workstream(
"child-coord",
kind="coordinator",
parent_ws_id="coord-root",
user_id="user-1",
)
# Unrelated workstream with no parent — MUST be excluded.
st.register_workstream("unrelated-ws", kind="interactive", user_id="user-1")
return st
def _read_client(storage: SQLiteBackend) -> CoordinatorClient:
"""Build a CoordinatorClient whose HTTP transport is a no-op stub."""
transport = httpx.MockTransport(lambda r: httpx.Response(200))
http = httpx.Client(transport=transport)
return CoordinatorClient(
console_base_url="http://x",
storage=storage,
token_factory=lambda: "t",
coord_ws_id="coord-root",
user_id="user-1",
http_client=http,
)
def test_list_children_excludes_coordinator_and_unrelated_rows(seeded_storage):
"""list_children returns only interactive children of the given parent."""
client = _read_client(seeded_storage)
result = client.list_children("coord-root")
rows = result["children"]
ws_ids = {r["ws_id"] for r in rows}
# The two interactive children are present.
assert ws_ids == {"child-idle", "child-running"}
# Every returned row must be interactive and linked to coord-root.
for r in rows:
assert r["kind"] == "interactive"
assert r["parent_ws_id"] == "coord-root"
# Coordinator child and unrelated ws are absent.
assert "child-coord" not in ws_ids
assert "unrelated-ws" not in ws_ids
assert result["truncated"] is False
def test_list_children_state_filter(seeded_storage):
"""list_children(state='running') filters to only running children."""
client = _read_client(seeded_storage)
result = client.list_children("coord-root", state="running")
assert {r["ws_id"] for r in result["children"]} == {"child-running"}
def test_list_children_skill_filter(seeded_storage):
"""list_children(skill='skill-alpha') returns the matching child only."""
client = _read_client(seeded_storage)
result = client.list_children("coord-root", skill="skill-alpha")
rows = result["children"]
assert {r["ws_id"] for r in rows} == {"child-idle"}
assert rows[0].get("skill_id") == "skill-alpha"
# ---------------------------------------------------------------------------
# Test 4 — Lazy rehydration via GET /v1/api/coordinator/{ws_id}
# ---------------------------------------------------------------------------
def test_lazy_rehydration_on_detail_get(tmp_path):
"""A persisted coordinator row rehydrates into the manager on GET /{ws_id}.
Sequence:
1. Pre-seed storage with a coordinator row (simulating a previous process).
2. Build a CoordinatorManager that doesn't know about it yet.
3. Hit GET /v1/api/coordinator/{ws_id} expect 200.
4. Manager now tracks the rehydrated session.
5. The response body carries the correct kind / user_id metadata.
"""
storage = SQLiteBackend(str(tmp_path / "rehydrate.db"))
# Seed the row directly — the manager has never seen it.
storage.register_workstream(
"persisted-coord",
node_id="console",
user_id="user-1",
name="old-coord",
kind="coordinator",
)
mgr = _build_mgr(storage)
# Confirm: not tracked in memory yet.
assert mgr.get("persisted-coord") is None
client = _make_client(storage, coord_mgr=mgr, registry=_fake_registry())
resp = client.get("/v1/api/coordinator/persisted-coord", headers=_COORD_HEADERS)
assert resp.status_code == 200, resp.text
body = resp.json()
assert body["ws_id"] == "persisted-coord"
assert body["kind"] == "coordinator"
assert body["user_id"] == "user-1"
# The endpoint triggers lazy rehydration — manager now tracks it.
assert mgr.get("persisted-coord") is not None
# Non-owner cannot reach the same endpoint (returns 404 — no existence leak).
resp_stranger = client.get(
"/v1/api/coordinator/persisted-coord",
headers={"X-Test-User": "stranger", "X-Test-Perms": "admin.coordinator"},
)
assert resp_stranger.status_code == 404
# A workstream with kind='interactive' is not reachable via the coordinator
# endpoint even when it exists in storage.
storage.register_workstream("interactive-ws", kind="interactive", user_id="user-1")
resp_int = client.get("/v1/api/coordinator/interactive-ws", headers=_COORD_HEADERS)
assert resp_int.status_code == 404
File diff suppressed because it is too large Load Diff
+810
View File
@@ -0,0 +1,810 @@
"""Tests for the coordinator governance endpoints and session hooks.
Covers the three console endpoints that let an operator steer a live
coordinator session mid-flight (``/trust``, ``/restrict``,
``/stop_cascade``), the two ``ChatSession`` methods the endpoints
toggle (``set_trust_send`` / ``revoke_tools``), the audit rows the
handlers emit, and the ``_prepare_tool`` revocation gate.
"""
from __future__ import annotations
import json
from typing import Any
from unittest.mock import MagicMock
import pytest
from starlette.applications import Starlette
from starlette.middleware import Middleware
from starlette.middleware.base import BaseHTTPMiddleware
from starlette.routing import Route
from starlette.testclient import TestClient
from tests._coord_test_helpers import (
_AuthMiddleware,
_build_mgr,
_fake_registry,
_FakeConfigStore,
)
from turnstone.console.server import (
coordinator_restrict,
coordinator_stop_cascade,
coordinator_trust,
)
from turnstone.core.auth import AuthResult
from turnstone.core.storage._sqlite import SQLiteBackend
@pytest.fixture
def storage(tmp_path):
return SQLiteBackend(str(tmp_path / "coord.db"))
def _make_client(storage, *, coord_mgr, alias="my-model", registry=None) -> TestClient:
"""Starlette app exposing only the three governance endpoints."""
app = Starlette(
routes=[
Route(
"/v1/api/coordinator/{ws_id}/trust",
coordinator_trust,
methods=["POST"],
),
Route(
"/v1/api/coordinator/{ws_id}/restrict",
coordinator_restrict,
methods=["POST"],
),
Route(
"/v1/api/coordinator/{ws_id}/stop_cascade",
coordinator_stop_cascade,
methods=["POST"],
),
],
middleware=[Middleware(_AuthMiddleware)],
)
app.state.coord_mgr = coord_mgr
app.state.config_store = _FakeConfigStore({"coordinator.model_alias": alias})
app.state.coord_registry = registry
app.state.coord_registry_error = "" if coord_mgr else "registry missing"
app.state.auth_storage = storage
app.state.jwt_secret = "x" * 64
return TestClient(app)
def _make_session_mock(*, trust_send: bool = False, revoked: frozenset[str] = frozenset()):
"""Build a MagicMock ``session`` that honours the new ChatSession
governance surface (``set_trust_send`` / ``get_trust_send`` /
``revoke_tools`` / ``get_revoked_tools``) so handler tests exercise
the real method calls rather than reaching into attributes."""
state: dict[str, Any] = {"trust_send": trust_send, "revoked": revoked}
def _set_trust_send(value: bool) -> None:
state["trust_send"] = bool(value)
def _get_trust_send() -> bool:
return bool(state["trust_send"])
def _revoke_tools(names):
state["revoked"] = state["revoked"] | frozenset(names)
return state["revoked"]
def _get_revoked_tools():
return state["revoked"]
session = MagicMock()
session.set_trust_send.side_effect = _set_trust_send
session.get_trust_send.side_effect = _get_trust_send
session.revoke_tools.side_effect = _revoke_tools
session.get_revoked_tools.side_effect = _get_revoked_tools
return session, state
_COORD_HEADERS = {"X-Test-User": "user-1", "X-Test-Perms": "admin.coordinator"}
_TRUST_HEADERS = {
"X-Test-User": "user-1",
"X-Test-Perms": "admin.coordinator,coordinator.trust.send",
}
# ---------------------------------------------------------------------------
# /trust endpoint — trusted-session mode (item 1)
# ---------------------------------------------------------------------------
def test_trust_toggle_requires_trust_send_permission(storage):
"""Double-gated: admin.coordinator alone is insufficient — the
trust-send perm is an explicit opt-in."""
mgr = _build_mgr(storage)
coord = mgr.create(user_id="user-1", name="coord-a")
client = _make_client(storage, coord_mgr=mgr, registry=_fake_registry())
resp = client.post(
f"/v1/api/coordinator/{coord.id}/trust",
json={"send": True},
headers=_COORD_HEADERS,
)
assert resp.status_code == 403
def test_trust_toggle_flips_session_flag_and_audits(storage):
mgr = _build_mgr(storage)
coord = mgr.create(user_id="user-1", name="coord-a")
session, state = _make_session_mock()
coord.session = session
client = _make_client(storage, coord_mgr=mgr, registry=_fake_registry())
resp = client.post(
f"/v1/api/coordinator/{coord.id}/trust",
json={"send": True},
headers=_TRUST_HEADERS,
)
assert resp.status_code == 200
assert resp.json() == {"status": "ok", "trust_send": True}
assert state["trust_send"] is True
events = [e for e in storage.list_audit_events() if e["action"] == "coordinator.trust.toggled"]
assert len(events) == 1
detail = json.loads(events[0]["detail"])
assert detail["send_before"] is False
assert detail["send_after"] is True
def _service_token_client(
storage,
coord_mgr,
*,
user_id: str,
permissions: frozenset[str],
) -> TestClient:
"""Build a TestClient whose middleware injects a service-scoped token.
Used to verify that the capability-escalating endpoints (``/trust``,
``/restrict``, ``/stop_cascade``) do NOT honor the normal
``require_permission`` service-scope bypass when the caller lacks
the specific grant they need.
"""
app = Starlette(
routes=[
Route(
"/v1/api/coordinator/{ws_id}/trust",
coordinator_trust,
methods=["POST"],
),
Route(
"/v1/api/coordinator/{ws_id}/restrict",
coordinator_restrict,
methods=["POST"],
),
Route(
"/v1/api/coordinator/{ws_id}/stop_cascade",
coordinator_stop_cascade,
methods=["POST"],
),
],
)
app.state.coord_mgr = coord_mgr
app.state.config_store = _FakeConfigStore({"coordinator.model_alias": "my-model"})
app.state.coord_registry = _fake_registry()
app.state.coord_registry_error = ""
app.state.auth_storage = storage
app.state.jwt_secret = "x" * 64
captured_perms = permissions
class _ServiceAuth(BaseHTTPMiddleware):
async def dispatch(self, request, call_next):
request.state.auth_result = AuthResult(
user_id=user_id,
scopes=frozenset({"read", "write", "approve", "service"}),
token_source="test",
permissions=captured_perms,
)
return await call_next(request)
app.user_middleware = [Middleware(_ServiceAuth)]
app.middleware_stack = app.build_middleware_stack()
return TestClient(app)
def test_trust_toggle_service_token_cannot_bypass_permission(storage):
"""Service token without coordinator.trust.send is 403'd even when
its user_id matches the coord owner."""
mgr = _build_mgr(storage)
coord = mgr.create(user_id="svc-user", name="coord-a")
coord.session, _ = _make_session_mock()
client = _service_token_client(
storage,
mgr,
user_id="svc-user",
permissions=frozenset({"admin.coordinator"}),
)
resp = client.post(
f"/v1/api/coordinator/{coord.id}/trust",
json={"send": True},
)
assert resp.status_code == 403
assert "coordinator.trust.send" in resp.json()["error"]
def test_trust_toggle_service_token_with_permission_succeeds(storage):
"""Service token WITH the explicit coordinator.trust.send grant IS
allowed through locks the intended invariant: bypass is off, but
an explicit perm still works."""
mgr = _build_mgr(storage)
coord = mgr.create(user_id="svc-user", name="coord-a")
session, state = _make_session_mock()
coord.session = session
client = _service_token_client(
storage,
mgr,
user_id="svc-user",
permissions=frozenset({"admin.coordinator", "coordinator.trust.send"}),
)
resp = client.post(
f"/v1/api/coordinator/{coord.id}/trust",
json={"send": True},
)
assert resp.status_code == 200
assert resp.json() == {"status": "ok", "trust_send": True}
assert state["trust_send"] is True
def test_restrict_service_token_cannot_bypass_admin_coordinator(storage):
"""/restrict is destructive — a service token WITHOUT explicit
admin.coordinator grant must be 403'd rather than letting the
service-scope bypass open the endpoint up."""
mgr = _build_mgr(storage)
coord = mgr.create(user_id="svc-user", name="coord-a")
coord.session, _ = _make_session_mock()
client = _service_token_client(
storage,
mgr,
user_id="svc-user",
permissions=frozenset(), # no admin.coordinator
)
resp = client.post(
f"/v1/api/coordinator/{coord.id}/restrict",
json={"revoke": ["bash"]},
)
assert resp.status_code == 403
def test_stop_cascade_service_token_cannot_bypass_admin_coordinator(storage):
"""/stop_cascade mirrors /restrict — same destructive treatment."""
mgr = _build_mgr(storage)
coord = mgr.create(user_id="svc-user", name="coord-a")
coord.session, _ = _make_session_mock()
client = _service_token_client(
storage,
mgr,
user_id="svc-user",
permissions=frozenset(),
)
resp = client.post(
f"/v1/api/coordinator/{coord.id}/stop_cascade",
json={},
)
assert resp.status_code == 403
def test_trust_toggle_rejects_non_bool(storage):
mgr = _build_mgr(storage)
coord = mgr.create(user_id="user-1", name="coord-a")
coord.session, _ = _make_session_mock()
client = _make_client(storage, coord_mgr=mgr, registry=_fake_registry())
resp = client.post(
f"/v1/api/coordinator/{coord.id}/trust",
json={"send": "yes"},
headers=_TRUST_HEADERS,
)
assert resp.status_code == 400
def test_trust_toggle_rejects_non_object_body(storage):
"""A valid-JSON-but-non-object body (null / list / scalar) must
400 cleanly rather than AttributeError 500."""
mgr = _build_mgr(storage)
coord = mgr.create(user_id="user-1", name="coord-a")
coord.session, _ = _make_session_mock()
client = _make_client(storage, coord_mgr=mgr, registry=_fake_registry())
# Non-dict JSON values — all must 400. Different bodies may hit
# `read_json_or_400`'s own parse error ("Invalid JSON body") or the
# downstream dict-shape guard ("body must be a JSON object"); we
# only care that none 500.
for body in ([], 42, "string"):
resp = client.post(
f"/v1/api/coordinator/{coord.id}/trust",
json=body,
headers=_TRUST_HEADERS,
)
assert resp.status_code == 400, body
assert "JSON object" in resp.json()["error"], resp.json()
def test_restrict_rejects_non_object_body(storage):
mgr = _build_mgr(storage)
coord = mgr.create(user_id="user-1", name="coord-a")
coord.session, _ = _make_session_mock()
client = _make_client(storage, coord_mgr=mgr, registry=_fake_registry())
resp = client.post(
f"/v1/api/coordinator/{coord.id}/restrict",
json=[],
headers=_COORD_HEADERS,
)
assert resp.status_code == 400
def test_trust_toggle_tenant_404_on_foreign_coord(storage):
mgr = _build_mgr(storage)
coord = mgr.create(user_id="user-owner", name="coord-a")
coord.session, _ = _make_session_mock()
client = _make_client(storage, coord_mgr=mgr, registry=_fake_registry())
resp = client.post(
f"/v1/api/coordinator/{coord.id}/trust",
json={"send": True},
headers={
"X-Test-User": "user-other",
"X-Test-Perms": "admin.coordinator,coordinator.trust.send",
},
)
assert resp.status_code == 404
def test_trust_toggle_404_when_session_not_loaded(storage):
"""Persisted-but-not-loaded coordinator: runtime session state can't
be mutated, so the endpoint 404s. Matches the tenant-miss shape
so non-admins can't probe for closed rows via this endpoint."""
mgr = _build_mgr(storage)
coord = mgr.create(user_id="user-1", name="coord-a")
coord.session = None # simulate a closed / lazy-rehydrate coord
client = _make_client(storage, coord_mgr=mgr, registry=_fake_registry())
resp = client.post(
f"/v1/api/coordinator/{coord.id}/trust",
json={"send": True},
headers=_TRUST_HEADERS,
)
assert resp.status_code == 404
# ---------------------------------------------------------------------------
# _prepare_send_to_workstream — trust gate (item 1, unit-level)
# ---------------------------------------------------------------------------
def test_prepare_send_to_workstream_trust_skips_approval_for_own_child():
from turnstone.core.session import ChatSession
session = ChatSession.__new__(ChatSession)
session._coord_client = MagicMock()
session._trust_send = True
session._coord_client._is_own_subtree.return_value = True
item = session._prepare_send_to_workstream(call_id="c1", args={"ws_id": "abc", "message": "hi"})
assert item["needs_approval"] is False
assert item["trust_auto_approved"] is True
def test_prepare_send_to_workstream_trust_holds_for_foreign_ws():
from turnstone.core.session import ChatSession
session = ChatSession.__new__(ChatSession)
session._coord_client = MagicMock()
session._trust_send = True
session._coord_client._is_own_subtree.return_value = False
item = session._prepare_send_to_workstream(
call_id="c2", args={"ws_id": "foreign-ws", "message": "hi"}
)
assert item["needs_approval"] is True
assert item["trust_auto_approved"] is False
def test_prepare_send_to_workstream_without_trust_always_requires_approval():
from turnstone.core.session import ChatSession
session = ChatSession.__new__(ChatSession)
session._coord_client = MagicMock()
session._trust_send = False
session._coord_client._is_own_subtree.return_value = True
item = session._prepare_send_to_workstream(call_id="c3", args={"ws_id": "abc", "message": "hi"})
assert item["needs_approval"] is True
assert item["trust_auto_approved"] is False
def test_exec_send_to_workstream_records_trust_audit(storage):
"""The audit row fires before the HTTP send so a downstream failure
can't suppress the trail."""
from turnstone.console.coordinator_client import CoordinatorClient
from turnstone.core.session import ChatSession
client = CoordinatorClient.__new__(CoordinatorClient)
client._storage = storage
client._user_id = "user-1"
client._coord_ws_id = "coord-1"
session = ChatSession.__new__(ChatSession)
session._coord_client = client
session.ui = MagicMock()
send_mock = MagicMock(return_value={"status": "ok"})
client.send = send_mock # type: ignore[method-assign]
session._exec_send_to_workstream(
{
"call_id": "c1",
"ws_id": "child-ws-1",
"message": "please summarise",
"trust_auto_approved": True,
}
)
events = [
e for e in storage.list_audit_events() if e["action"] == "coordinator.send.auto_approved"
]
assert len(events) == 1
detail = json.loads(events[0]["detail"])
assert detail["src"] == "coordinator"
assert detail["trust"] is True
assert detail["ws_id"] == "child-ws-1"
assert "please summarise" in detail["message_preview"]
# ---------------------------------------------------------------------------
# /restrict endpoint + _prepare_tool revocation gate (item 5a)
# ---------------------------------------------------------------------------
def test_restrict_adds_to_revoked_tools_and_audits(storage):
mgr = _build_mgr(storage)
coord = mgr.create(user_id="user-1", name="coord-a")
session, state = _make_session_mock()
coord.session = session
client = _make_client(storage, coord_mgr=mgr, registry=_fake_registry())
resp = client.post(
f"/v1/api/coordinator/{coord.id}/restrict",
json={"revoke": ["spawn_workstream", "delete_workstream"]},
headers=_COORD_HEADERS,
)
assert resp.status_code == 200
body = resp.json()
assert set(body["revoked_tools"]) == {"spawn_workstream", "delete_workstream"}
assert state["revoked"] == frozenset({"spawn_workstream", "delete_workstream"})
events = [e for e in storage.list_audit_events() if e["action"] == "coordinator.restricted"]
assert len(events) == 1
detail = json.loads(events[0]["detail"])
assert set(detail["revoked"]) == {"spawn_workstream", "delete_workstream"}
def test_restrict_is_additive_across_calls(storage):
mgr = _build_mgr(storage)
coord = mgr.create(user_id="user-1", name="coord-a")
coord.session, _ = _make_session_mock()
client = _make_client(storage, coord_mgr=mgr, registry=_fake_registry())
client.post(
f"/v1/api/coordinator/{coord.id}/restrict",
json={"revoke": ["spawn_workstream"]},
headers=_COORD_HEADERS,
)
resp = client.post(
f"/v1/api/coordinator/{coord.id}/restrict",
json={"revoke": ["delete_workstream"]},
headers=_COORD_HEADERS,
)
assert resp.status_code == 200
assert set(resp.json()["revoked_tools"]) == {
"spawn_workstream",
"delete_workstream",
}
def test_restrict_empty_revoke_is_noop_but_audits(storage):
"""Empty list is accepted as a no-op write — still emits the audit
row so operators can see 'operator poked the restrict endpoint but
didn't actually revoke anything' events."""
mgr = _build_mgr(storage)
coord = mgr.create(user_id="user-1", name="coord-a")
coord.session, _state = _make_session_mock(revoked=frozenset({"spawn_workstream"}))
client = _make_client(storage, coord_mgr=mgr, registry=_fake_registry())
resp = client.post(
f"/v1/api/coordinator/{coord.id}/restrict",
json={"revoke": []},
headers=_COORD_HEADERS,
)
assert resp.status_code == 200
# Pre-existing revocations are preserved; no new entries were added.
assert set(resp.json()["revoked_tools"]) == {"spawn_workstream"}
events = [e for e in storage.list_audit_events() if e["action"] == "coordinator.restricted"]
assert len(events) == 1
detail = json.loads(events[0]["detail"])
assert detail["revoked"] == []
def test_restrict_rejects_non_list_body(storage):
mgr = _build_mgr(storage)
coord = mgr.create(user_id="user-1", name="coord-a")
coord.session, _ = _make_session_mock()
client = _make_client(storage, coord_mgr=mgr, registry=_fake_registry())
resp = client.post(
f"/v1/api/coordinator/{coord.id}/restrict",
json={"revoke": "spawn_workstream"},
headers=_COORD_HEADERS,
)
assert resp.status_code == 400
def test_restrict_rejects_oversize_list(storage):
"""Defense-in-depth cap — an admin-sized list can't blow up the
session frozenset or the audit row's detail column."""
mgr = _build_mgr(storage)
coord = mgr.create(user_id="user-1", name="coord-a")
coord.session, _ = _make_session_mock()
client = _make_client(storage, coord_mgr=mgr, registry=_fake_registry())
resp = client.post(
f"/v1/api/coordinator/{coord.id}/restrict",
json={"revoke": [f"tool_{i}" for i in range(500)]},
headers=_COORD_HEADERS,
)
assert resp.status_code == 400
def test_restrict_rejects_oversize_name(storage):
mgr = _build_mgr(storage)
coord = mgr.create(user_id="user-1", name="coord-a")
coord.session, _ = _make_session_mock()
client = _make_client(storage, coord_mgr=mgr, registry=_fake_registry())
resp = client.post(
f"/v1/api/coordinator/{coord.id}/restrict",
json={"revoke": ["x" * 1000]},
headers=_COORD_HEADERS,
)
assert resp.status_code == 400
def test_restrict_404_when_session_not_loaded(storage):
mgr = _build_mgr(storage)
coord = mgr.create(user_id="user-1", name="coord-a")
coord.session = None
client = _make_client(storage, coord_mgr=mgr, registry=_fake_registry())
resp = client.post(
f"/v1/api/coordinator/{coord.id}/restrict",
json={"revoke": ["bash"]},
headers=_COORD_HEADERS,
)
assert resp.status_code == 404
def test_prepare_tool_blocks_revoked_tool():
"""Revocation short-circuits BEFORE the preparer dispatch so the
model sees a clear 'revoked' error rather than a preparer-level
validation message."""
from turnstone.core.session import ChatSession
session = ChatSession.__new__(ChatSession)
session._revoked_tools = frozenset({"spawn_workstream"})
session._mcp_client = None
session.ui = MagicMock()
tc = {
"id": "call-1",
"function": {
"name": "spawn_workstream",
"arguments": '{"initial_message": "x"}',
},
}
item = session._prepare_tool(tc)
assert item["needs_approval"] is False
assert "revoked" in item["header"].lower()
assert "revoked" in item["error"].lower()
def test_prepare_tool_allows_non_revoked_tool():
"""The revocation gate must not fire on a tool name that isn't in
the revoked set. We pick a name that's also not in the preparers
dict so we can assert the 'unknown tool' result shape without
exercising a real preparer."""
from turnstone.core.session import ChatSession
session = ChatSession.__new__(ChatSession)
session._revoked_tools = frozenset({"spawn_workstream"})
session._mcp_client = None
session.ui = MagicMock()
tc = {
"id": "call-2",
"function": {"name": "this_tool_is_not_registered", "arguments": "{}"},
}
item = session._prepare_tool(tc)
# Unknown tool path — not the revocation error path.
err = str(item.get("error") or "")
assert "revoked" not in err.lower()
# ---------------------------------------------------------------------------
# /stop_cascade endpoint (item 5b)
# ---------------------------------------------------------------------------
def test_stop_cascade_cancels_coord_and_each_child(storage):
mgr = _build_mgr(storage)
coord = mgr.create(user_id="user-1", name="coord-a")
mgr.register_children(coord.id, ["child-1", "child-2", "child-3"])
def _cancel(wid: str) -> dict:
if wid == "child-2":
return {"error": "gateway_timeout", "status": 502}
return {"status": "ok"}
coord_client = MagicMock()
coord_client.cancel.side_effect = _cancel
coord.session = MagicMock()
coord.session._coord_client = coord_client
client = _make_client(storage, coord_mgr=mgr, registry=_fake_registry())
resp = client.post(
f"/v1/api/coordinator/{coord.id}/stop_cascade",
json={},
headers=_COORD_HEADERS,
)
assert resp.status_code == 200
body = resp.json()
assert set(body["cancelled"] + body["failed"] + body["skipped"]) == {
"child-1",
"child-2",
"child-3",
}
assert body["failed"] == ["child-2"]
assert set(body["cancelled"]) == {"child-1", "child-3"}
assert body["skipped"] == []
assert coord_client.cancel.call_count == 3
events = [
e for e in storage.list_audit_events() if e["action"] == "coordinator.stopped_cascade"
]
assert len(events) == 1
detail = json.loads(events[0]["detail"])
assert set(detail["cancelled"] + detail["failed"] + detail["skipped"]) == {
"child-1",
"child-2",
"child-3",
}
def test_stop_cascade_routes_404_to_skipped_bucket(storage):
"""A stale registry entry (child row already deleted from storage)
or an upstream-404 on cancel is semantically 'already gone', not a
dispatch failure. Report it in ``skipped`` so operators can tell
them apart."""
mgr = _build_mgr(storage)
coord = mgr.create(user_id="user-1", name="coord-a")
mgr.register_children(coord.id, ["stale-child"])
coord_client = MagicMock()
coord_client.cancel.return_value = {
"error": "workstream not in coordinator subtree: stale-child",
"status": 404,
}
coord.session = MagicMock()
coord.session._coord_client = coord_client
client = _make_client(storage, coord_mgr=mgr, registry=_fake_registry())
resp = client.post(
f"/v1/api/coordinator/{coord.id}/stop_cascade",
json={},
headers=_COORD_HEADERS,
)
assert resp.status_code == 200
body = resp.json()
assert body["cancelled"] == []
assert body["failed"] == []
assert body["skipped"] == ["stale-child"]
def test_stop_cascade_empty_children_still_audits(storage):
mgr = _build_mgr(storage)
coord = mgr.create(user_id="user-1", name="coord-a")
coord.session = MagicMock()
coord.session._coord_client = MagicMock()
client = _make_client(storage, coord_mgr=mgr, registry=_fake_registry())
resp = client.post(
f"/v1/api/coordinator/{coord.id}/stop_cascade",
json={},
headers=_COORD_HEADERS,
)
assert resp.status_code == 200
body = resp.json()
assert body == {"status": "ok", "cancelled": [], "failed": [], "skipped": []}
assert [e for e in storage.list_audit_events() if e["action"] == "coordinator.stopped_cascade"]
def test_stop_cascade_without_coord_client_marks_all_failed(storage):
"""If the coord session has no attached coord_client (unexpected
state for a loaded session), every child routes to ``failed`` so
the operator can investigate."""
mgr = _build_mgr(storage)
coord = mgr.create(user_id="user-1", name="coord-a")
mgr.register_children(coord.id, ["child-a", "child-b"])
coord.session = MagicMock()
coord.session._coord_client = None
client = _make_client(storage, coord_mgr=mgr, registry=_fake_registry())
resp = client.post(
f"/v1/api/coordinator/{coord.id}/stop_cascade",
json={},
headers=_COORD_HEADERS,
)
assert resp.status_code == 200
body = resp.json()
assert body["cancelled"] == []
assert body["skipped"] == []
assert set(body["failed"]) == {"child-a", "child-b"}
def test_stop_cascade_404_when_session_not_loaded(storage):
mgr = _build_mgr(storage)
coord = mgr.create(user_id="user-1", name="coord-a")
coord.session = None
client = _make_client(storage, coord_mgr=mgr, registry=_fake_registry())
resp = client.post(
f"/v1/api/coordinator/{coord.id}/stop_cascade",
json={},
headers=_COORD_HEADERS,
)
assert resp.status_code == 404
def test_children_snapshot_returns_copy_not_live_set(storage):
mgr = _build_mgr(storage)
coord = mgr.create(user_id="user-1", name="coord-a")
mgr.register_children(coord.id, ["a", "b", "c"])
snap = mgr.children_snapshot(coord.id)
assert set(snap) == {"a", "b", "c"}
mgr.register_children(coord.id, ["d"])
assert set(snap) == {"a", "b", "c"}
# ---------------------------------------------------------------------------
# ChatSession governance methods (q-14) — unit-level
# ---------------------------------------------------------------------------
def test_set_and_get_trust_send_round_trip():
from turnstone.core.session import ChatSession
session = ChatSession.__new__(ChatSession)
import threading as _t
session._trust_send = False
session._governance_lock = _t.Lock()
assert session.get_trust_send() is False
session.set_trust_send(True)
assert session.get_trust_send() is True
session.set_trust_send(False)
assert session.get_trust_send() is False
def test_revoke_tools_is_additive_and_returns_post_state():
from turnstone.core.session import ChatSession
session = ChatSession.__new__(ChatSession)
import threading as _t
session._revoked_tools = frozenset()
session._governance_lock = _t.Lock()
after = session.revoke_tools(["bash", "read_file"])
assert after == frozenset({"bash", "read_file"})
after2 = session.revoke_tools(["write_file"])
assert after2 == frozenset({"bash", "read_file", "write_file"})
# Re-revoking is a no-op (idempotent).
after3 = session.revoke_tools(["bash"])
assert after3 == after2
assert session.get_revoked_tools() == after3
+955
View File
@@ -0,0 +1,955 @@
"""Tests for :class:`turnstone.console.coordinator.CoordinatorManager`.
Covers the lifecycle semantics without standing up a full ModelRegistry
or ChatSession: a stub session factory returns a MagicMock-backed
session so tests stay fast.
"""
from __future__ import annotations
from typing import Any
from unittest.mock import MagicMock
import pytest
from turnstone.console.coordinator import CoordinatorManager
from turnstone.console.coordinator_ui import ConsoleCoordinatorUI
from turnstone.core.storage._sqlite import SQLiteBackend
from turnstone.core.workstream import WorkstreamState
@pytest.fixture
def storage(tmp_path):
return SQLiteBackend(str(tmp_path / "coord.db"))
@pytest.fixture
def built_mgr(storage):
"""Build a CoordinatorManager with a stub session factory.
The factory records its calls and returns a MagicMock-backed
session so ``_spawn_worker`` can run without hitting real LLM
infrastructure.
"""
call_log: list[dict] = []
def _session_factory(ui, model_alias=None, ws_id=None, **kwargs):
call_log.append(
{
"ui": ui,
"model_alias": model_alias,
"ws_id": ws_id,
**kwargs,
}
)
mock_session = MagicMock()
mock_session.ws_id = ws_id
# send() is the worker thread target; make it a fast no-op.
mock_session.send.return_value = None
return mock_session
def _ui_factory(ws_id, user_id):
return ConsoleCoordinatorUI(ws_id=ws_id, user_id=user_id)
mgr = CoordinatorManager(
session_factory=_session_factory,
ui_factory=_ui_factory,
storage=storage,
max_active=3,
)
return mgr, call_log, storage
# ---------------------------------------------------------------------------
# create
# ---------------------------------------------------------------------------
def test_create_registers_row_with_coordinator_kind(built_mgr):
mgr, _calls, storage = built_mgr
ws = mgr.create(user_id="user-1", name="c1")
row = storage.get_workstream(ws.id)
assert row is not None
assert row["kind"] == "coordinator"
assert row["user_id"] == "user-1"
assert row["node_id"] == "console"
assert row["parent_ws_id"] is None
def test_create_passes_kind_to_factory(built_mgr):
mgr, calls, _s = built_mgr
mgr.create(user_id="user-1")
assert calls[-1]["kind"] == "coordinator"
assert calls[-1]["parent_ws_id"] is None
def test_create_dispatches_initial_message(built_mgr):
import time
mgr, _calls, _s = built_mgr
ws = mgr.create(user_id="user-1", initial_message="hello")
# Give the worker a brief window to run send() on the mock.
for _ in range(20):
if ws.session.send.called:
break
time.sleep(0.01)
ws.session.send.assert_called_once_with("hello")
def test_create_no_initial_message_skips_worker(built_mgr):
mgr, _calls, _s = built_mgr
ws = mgr.create(user_id="user-1")
assert ws.session.send.call_count == 0
# ---------------------------------------------------------------------------
# max_active + eviction
# ---------------------------------------------------------------------------
def test_max_active_enforced_evicts_idle(built_mgr):
mgr, _calls, _s = built_mgr
ws_a = mgr.create(user_id="u1")
ws_b = mgr.create(user_id="u2")
ws_c = mgr.create(user_id="u3")
# All three at capacity. The next create should evict the oldest
# IDLE — ws_a has the oldest last_active.
ws_d = mgr.create(user_id="u4")
# ws_a got evicted from the dict; b/c/d are still present.
assert mgr.get(ws_a.id) is None
for w in (ws_b, ws_c, ws_d):
assert mgr.get(w.id) is not None
def test_max_active_raises_when_all_non_idle(built_mgr):
mgr, _calls, _s = built_mgr
ws_a = mgr.create(user_id="u1")
ws_b = mgr.create(user_id="u2")
ws_c = mgr.create(user_id="u3")
# Force all into a non-idle state so no eviction candidate exists.
for w in (ws_a, ws_b, ws_c):
w.state = WorkstreamState.RUNNING
with pytest.raises(RuntimeError) as exc_info:
mgr.create(user_id="u4")
assert "slots are active" in str(exc_info.value)
def test_rollback_on_factory_failure(storage):
"""If the session factory raises, the slot + persisted row are rolled back."""
def _factory_explodes(*args, **kwargs):
raise RuntimeError("session construction failed")
mgr = CoordinatorManager(
session_factory=_factory_explodes,
ui_factory=lambda w, u: ConsoleCoordinatorUI(ws_id=w, user_id=u),
storage=storage,
max_active=3,
)
with pytest.raises(RuntimeError):
mgr.create(user_id="u1")
# No leaked in-memory workstream.
assert mgr.list_all() == []
# ---------------------------------------------------------------------------
# send / cancel / close
# ---------------------------------------------------------------------------
def test_send_returns_false_when_not_loaded(built_mgr):
mgr, _calls, _s = built_mgr
assert mgr.send("nonexistent", "hello") is False
def test_send_returns_false_on_queue_full_without_spawning_duplicate(storage):
"""If queue_message raises queue.Full, _spawn_worker must NOT fall
through and start a second concurrent worker on the same ChatSession
that would corrupt history / cursors / approvals. Instead, send()
returns False so the endpoint can surface 429."""
import queue
import threading
entered = threading.Event()
block = threading.Event()
def _slow_send(msg):
entered.set()
block.wait(timeout=5.0)
def _session_factory(ui, model_alias=None, ws_id=None, **kwargs):
sess = MagicMock()
sess.send.side_effect = _slow_send
sess.queue_message.side_effect = queue.Full()
return sess
mgr = CoordinatorManager(
session_factory=_session_factory,
ui_factory=lambda w, u: ConsoleCoordinatorUI(ws_id=w, user_id=u),
storage=storage,
max_active=3,
)
ws = mgr.create(user_id="u1", initial_message="first")
try:
assert entered.wait(timeout=2.0), "worker didn't start"
original_thread = ws.worker_thread
assert mgr.send(ws.id, "second") is False
# Must NOT have replaced worker_thread with a fresh second worker.
assert ws.worker_thread is original_thread
finally:
block.set()
if ws.worker_thread:
ws.worker_thread.join(timeout=2.0)
def test_send_enqueues_on_live_worker(storage):
"""When a worker thread is already processing, send() routes through
queue_message instead of spawning a duplicate worker."""
import threading
import time
entered = threading.Event()
block = threading.Event()
def _slow_send(msg):
entered.set()
block.wait(timeout=5.0)
def _session_factory(ui, model_alias=None, ws_id=None, **kwargs):
sess = MagicMock()
sess.send.side_effect = _slow_send
return sess
mgr = CoordinatorManager(
session_factory=_session_factory,
ui_factory=lambda w, u: ConsoleCoordinatorUI(ws_id=w, user_id=u),
storage=storage,
max_active=3,
)
ws = mgr.create(user_id="u1", initial_message="first")
try:
# Wait until the worker is actually inside session.send.
assert entered.wait(timeout=2.0), "worker didn't start"
# Now the worker is alive — mgr.send should route through queue_message.
for _ in range(20):
if ws.worker_thread and ws.worker_thread.is_alive():
break
time.sleep(0.01)
sent = mgr.send(ws.id, "second")
assert sent
ws.session.queue_message.assert_called_with("second")
finally:
block.set()
if ws.worker_thread:
ws.worker_thread.join(timeout=2.0)
def test_cancel_resolves_pending_approval(built_mgr):
mgr, _calls, _s = built_mgr
ws = mgr.create(user_id="u1")
assert ws.ui is not None
assert isinstance(ws.ui, ConsoleCoordinatorUI)
# Put ui into a pending-approval state.
ws.ui._pending_approval = {"type": "approve_request", "items": []}
ws.ui._approval_event.clear()
assert mgr.cancel(ws.id) is True
# resolve_approval should have been called with approved=False.
assert ws.ui._approval_event.is_set()
assert ws.ui._approval_result == (False, "cancelled")
def test_cancel_unblocks_worker_blocked_on_approval(built_mgr):
"""Cancel fires while a worker thread is blocked inside
ui.approve_tools() waiting on _approval_event. The worker must
unblock with approved=False and return."""
import threading
import time
mgr, _calls, _s = built_mgr
ws = mgr.create(user_id="u1")
ui = ws.ui
assert isinstance(ui, ConsoleCoordinatorUI)
# Simulate the session worker entering approve_tools. We call it
# directly on its own thread so the test can observe the unblock.
result_holder: list[tuple[bool, str | None]] = []
def _worker() -> None:
outcome = ui.approve_tools(
[
{
"call_id": "c1",
"func_name": "spawn_workstream",
"approval_label": "spawn_workstream",
"needs_approval": True,
}
]
)
result_holder.append(outcome)
t = threading.Thread(target=_worker, daemon=True)
t.start()
# Give the worker time to enter the approval wait.
for _ in range(50):
if ui._pending_approval is not None:
break
time.sleep(0.01)
assert ui._pending_approval is not None, "worker didn't reach approve_tools"
# Cancel fires — worker should unblock with approved=False.
assert mgr.cancel(ws.id) is True
t.join(timeout=2.0)
assert not t.is_alive()
assert result_holder == [(False, "cancelled")]
def test_close_removes_and_updates_state(built_mgr):
mgr, _calls, storage = built_mgr
ws = mgr.create(user_id="u1")
# Extract side-effectful call from the assert expression so
# python -O (which strips asserts) can't drop the close().
closed = mgr.close(ws.id)
assert closed is True
assert mgr.get(ws.id) is None
row = storage.get_workstream(ws.id)
assert row["state"] == "closed"
# ---------------------------------------------------------------------------
# list_for_user + list_all
# ---------------------------------------------------------------------------
def test_list_for_user_filters_by_owner(built_mgr):
mgr, _calls, _s = built_mgr
a = mgr.create(user_id="user-1")
b = mgr.create(user_id="user-1")
mgr.create(user_id="user-2") # non-owner — existence matters, value doesn't
user1_rows = mgr.list_for_user("user-1")
ids = {r.id for r in user1_rows}
assert ids == {a.id, b.id}
def test_list_all_returns_every_loaded(built_mgr):
mgr, _calls, _s = built_mgr
mgr.create(user_id="u1")
mgr.create(user_id="u2")
assert len(mgr.list_all()) == 2
# ---------------------------------------------------------------------------
# Lazy rehydration
# ---------------------------------------------------------------------------
def test_open_rehydrates_from_storage(built_mgr):
mgr, _calls, storage = built_mgr
# Simulate a coordinator persisted from a previous console process.
storage.register_workstream(
"coord-persisted",
node_id="console",
user_id="user-1",
kind="coordinator",
)
# Initially not loaded in memory.
assert mgr.get("coord-persisted") is None
ws = mgr.open("coord-persisted", "user-1")
assert ws is not None
assert ws.kind == "coordinator"
assert ws.user_id == "user-1"
# Now tracked.
assert mgr.get("coord-persisted") is not None
def test_open_rejects_non_coordinator_kind(built_mgr):
mgr, _calls, storage = built_mgr
storage.register_workstream("interactive-ws", kind="interactive", user_id="user-1")
# open() has side effects (factory call, slot reservation); keep it
# out of the assert expression so python -O can't strip it.
opened = mgr.open("interactive-ws", "user-1")
assert opened is None
def test_open_enforces_ownership(built_mgr):
mgr, _calls, storage = built_mgr
storage.register_workstream("coord-x", kind="coordinator", user_id="owner")
# Non-owner gets None.
stranger_ws = mgr.open("coord-x", "stranger")
assert stranger_ws is None
# Owner gets the row.
owner_ws = mgr.open("coord-x", "owner")
assert owner_ws is not None
def test_open_admin_ignores_ownership(built_mgr):
mgr, _calls, storage = built_mgr
storage.register_workstream("coord-x", kind="coordinator", user_id="owner")
ws = mgr.open_admin("coord-x")
assert ws is not None
def test_open_resurrects_closed_coordinator(built_mgr):
"""A coordinator that was closed (state='closed' in storage) IS now
resurrectable via open(). Restore is an explicit user action via
the Saved Coordinators landing UI; ``_reserve_and_install_locked``
still enforces ``max_active`` (evicts an idle peer or 429s). The
old "URL revisit silently undoes Close" safety lives in the slot
accounting now, not in a flat refusal at the open path."""
mgr, _calls, storage = built_mgr
ws = mgr.create(user_id="u1")
mgr.close(ws.id)
assert storage.get_workstream(ws.id)["state"] == "closed"
reopened = mgr.open(ws.id, "u1")
assert reopened is not None
assert reopened.id == ws.id
# Re-loaded into memory.
assert mgr.get(ws.id) is reopened
# Admin path also resurrects.
mgr.close(ws.id)
assert mgr.open_admin(ws.id) is not None
def test_open_refuses_deleted_coordinator(built_mgr):
"""A coordinator marked state='deleted' is a tombstone — open() must
refuse to resurrect even though closed-state is now resurrectable."""
mgr, _calls, storage = built_mgr
ws = mgr.create(user_id="u1")
mgr.close(ws.id)
storage.update_workstream_state(ws.id, "deleted")
user_open = mgr.open(ws.id, "u1")
assert user_open is None
admin_open = mgr.open_admin(ws.id)
assert admin_open is None
def test_open_refuses_empty_owner_for_non_admin(built_mgr):
"""Empty-owner rows (orphan / pre-002 migrated) must not be
rehydrated by non-admin callers would consume a max_active slot
and let any user evict another tenant's IDLE coordinator."""
mgr, _calls, storage = built_mgr
storage.register_workstream("coord-orphan", kind="coordinator", user_id=None)
# Non-admin caller — empty owner must NOT short-circuit the gate.
assert mgr.open("coord-orphan", "any-user") is None
# Admin path can still rehydrate (e.g. cleanup tooling).
assert mgr.open_admin("coord-orphan") is not None
def test_open_returns_existing_when_loaded(built_mgr):
mgr, _calls, _s = built_mgr
ws1 = mgr.create(user_id="u1")
ws2 = mgr.open(ws1.id, "u1")
assert ws2 is ws1
# ---------------------------------------------------------------------------
# Concurrency regressions — blockers 1 & 2 from review
# ---------------------------------------------------------------------------
def test_concurrent_open_for_same_ws_id_constructs_one_session(storage):
"""Two threads calling open() for the same persisted-but-unloaded
ws_id must not each spin up a session. Per-ws_id serialization
ensures the second thread picks up the first thread's session."""
import threading
import time
construct_count = {"n": 0}
construct_lock = threading.Lock()
first_in = threading.Event()
release_first = threading.Event()
def _slow_factory(ui, model_alias=None, ws_id=None, **kwargs):
with construct_lock:
construct_count["n"] += 1
my_idx = construct_count["n"]
if my_idx == 1:
first_in.set()
# Block so the second thread can race past the storage read.
release_first.wait(timeout=5.0)
sess = MagicMock()
sess.ws_id = ws_id
sess.send.return_value = None
return sess
mgr = CoordinatorManager(
session_factory=_slow_factory,
ui_factory=lambda w, u: ConsoleCoordinatorUI(ws_id=w, user_id=u),
storage=storage,
max_active=5,
)
storage.register_workstream(
"coord-shared",
node_id="console",
user_id="user-1",
kind="coordinator",
)
results: list[Any] = [None, None]
def _open_one(idx: int) -> None:
results[idx] = mgr.open("coord-shared", "user-1")
t1 = threading.Thread(target=_open_one, args=(0,))
t2 = threading.Thread(target=_open_one, args=(1,))
t1.start()
assert first_in.wait(timeout=2.0), "first thread didn't enter factory"
t2.start()
# Give t2 a chance to reach the per-ws lock and block.
time.sleep(0.1)
release_first.set()
t1.join(timeout=5.0)
t2.join(timeout=5.0)
assert construct_count["n"] == 1, (
f"expected exactly 1 session construction, got {construct_count['n']}"
)
assert results[0] is not None
assert results[1] is not None
# Both threads must see the same installed Workstream instance.
assert results[0] is results[1]
# Manager tracks exactly one entry.
assert len(mgr.list_all()) == 1
def test_concurrent_create_respects_max_active(storage):
"""max_active + 2 concurrent creates → exactly max_active succeed
and the overflow raises RuntimeError. Regression for the
check-then-install gap that previously let all creates pass the gate."""
import threading
slow_entered = threading.Event()
release = threading.Event()
def _slow_factory(ui, model_alias=None, ws_id=None, **kwargs):
# Block after construction to widen the race window between
# slot reservation and final install. Only the first N reach
# here — the rest must trip on the capacity gate earlier.
slow_entered.set()
release.wait(timeout=5.0)
sess = MagicMock()
sess.send.return_value = None
return sess
max_active = 3
mgr = CoordinatorManager(
session_factory=_slow_factory,
ui_factory=lambda w, u: ConsoleCoordinatorUI(ws_id=w, user_id=u),
storage=storage,
max_active=max_active,
)
successes: list[bool] = []
failures: list[Exception] = []
successes_lock = threading.Lock()
def _create_one(user_suffix: int) -> None:
try:
mgr.create(user_id=f"u{user_suffix}")
with successes_lock:
successes.append(True)
except RuntimeError as exc:
with successes_lock:
failures.append(exc)
threads = [threading.Thread(target=_create_one, args=(i,)) for i in range(max_active + 2)]
for t in threads:
t.start()
# Wait until at least one creation is blocked inside the factory.
assert slow_entered.wait(timeout=2.0)
release.set()
for t in threads:
t.join(timeout=5.0)
assert len(successes) == max_active, f"expected {max_active} successes, got {len(successes)}"
assert len(failures) == 2
for exc in failures:
assert "slots are active" in str(exc)
assert len(mgr.list_all()) == max_active
# ---------------------------------------------------------------------------
# Cross-tenant leak — blocker 3 from review
# ---------------------------------------------------------------------------
def test_list_for_user_excludes_empty_owner_rows(built_mgr):
"""A coordinator whose user_id is empty (system-created, migration
artifact, or lazily rehydrated from a NULL owner) must NOT appear
in list_for_user() output for other callers doing so would leak
ws_id + name + state across tenants."""
mgr, _calls, storage = built_mgr
# Real user's coordinator.
owned = mgr.create(user_id="alice")
# Simulate a rogue empty-owner session by creating one with
# user_id="" directly. Matches what a rehydrate of a NULL-owner
# row would produce, or a system-created coordinator.
empty_owner = mgr.create(user_id="")
rows = mgr.list_for_user("alice")
ids = {ws.id for ws in rows}
assert owned.id in ids
assert empty_owner.id not in ids, (
"list_for_user must not expose empty-owner coordinators to other callers"
)
# ---------------------------------------------------------------------------
# Phase 3 — child-event fan-out
# ---------------------------------------------------------------------------
def _seed_child_row(storage, *, parent_ws_id: str, ws_id: str, state: str = "idle") -> None:
storage.register_workstream(
ws_id,
node_id="node-a",
user_id="user-1",
name=f"c-{ws_id[:4]}",
kind="interactive",
parent_ws_id=parent_ws_id,
)
if state != "idle":
storage.update_workstream_state(ws_id, state)
def _drain(listener, *, wait: float = 0.5):
"""Drain a ConsoleCoordinatorUI listener queue with a short timeout."""
import queue as _q
items = []
try:
while True:
items.append(listener.get(timeout=wait))
except _q.Empty:
return items
def test_children_registry_bootstrapped_from_storage_on_create(built_mgr):
mgr, _calls, storage = built_mgr
ws = mgr.create(user_id="user-1")
# The registry starts empty — no children yet.
assert mgr._children.get(ws.id, set()) == set()
def test_children_registry_bootstrapped_from_storage_on_open(built_mgr):
mgr, _calls, storage = built_mgr
# Seed a persisted coordinator row + two children directly in storage
# so open() rehydrates them without create() being called.
coord_id = "a" * 32
storage.register_workstream(
coord_id,
node_id="console",
user_id="user-1",
name="persisted",
kind="coordinator",
parent_ws_id=None,
)
_seed_child_row(storage, parent_ws_id=coord_id, ws_id="b" * 32)
_seed_child_row(storage, parent_ws_id=coord_id, ws_id="c" * 32)
ws = mgr.open(coord_id, "user-1")
assert ws is not None
assert mgr._children[coord_id] == {"b" * 32, "c" * 32}
def test_dispatch_ws_created_fans_out_to_parent(built_mgr):
mgr, _calls, _storage = built_mgr
ws = mgr.create(user_id="user-1")
listener = ws.ui._register_listener()
mgr._dispatch_child_event(
{
"type": "ws_created",
"ws_id": "d" * 32,
"parent_ws_id": ws.id,
"node_id": "node-a",
"name": "new-child",
"title": "",
"user_id": "user-1",
}
)
events = _drain(listener)
child_created = [e for e in events if e.get("type") == "child_ws_created"]
assert len(child_created) == 1
assert child_created[0]["child_ws_id"] == "d" * 32
assert child_created[0]["parent_ws_id"] == ws.id
assert "d" * 32 in mgr._children[ws.id]
def test_dispatch_ws_created_ignores_unrelated_parent(built_mgr):
mgr, _calls, _storage = built_mgr
ws = mgr.create(user_id="user-1")
listener = ws.ui._register_listener()
# A ws_created for a parent this coordinator doesn't own.
mgr._dispatch_child_event(
{
"type": "ws_created",
"ws_id": "e" * 32,
"parent_ws_id": "f" * 32,
"node_id": "node-a",
"name": "stranger-child",
"title": "",
"user_id": "user-1",
}
)
events = _drain(listener, wait=0.1)
assert not any(e.get("type") == "child_ws_created" for e in events)
def test_dispatch_ws_created_cross_tenant_dropped(built_mgr):
"""A ws_created event whose user_id does not match the coordinator's
owner must NOT reach the coordinator's SSE stream — prevents the
cross-tenant info-leak via spoofed parent_ws_id (sec-1)."""
mgr, _calls, _storage = built_mgr
ws = mgr.create(user_id="alice")
listener = ws.ui._register_listener()
# A mallory-owned workstream claiming alice's coordinator as parent.
mgr._dispatch_child_event(
{
"type": "ws_created",
"ws_id": "d" * 32,
"parent_ws_id": ws.id,
"node_id": "node-a",
"name": "spoofed-child",
"title": "",
"user_id": "mallory",
}
)
events = _drain(listener, wait=0.1)
assert not any(e.get("type") == "child_ws_created" for e in events)
# Registry must not have gained mallory's ws_id either.
assert "d" * 32 not in mgr._children.get(ws.id, set())
def test_dispatch_ws_created_empty_user_id_dropped(built_mgr):
"""An event with empty/missing user_id fails closed — we can't
prove tenancy, so we refuse to route it."""
mgr, _calls, _storage = built_mgr
ws = mgr.create(user_id="alice")
listener = ws.ui._register_listener()
mgr._dispatch_child_event(
{
"type": "ws_created",
"ws_id": "d" * 32,
"parent_ws_id": ws.id,
"node_id": "node-a",
"name": "no-owner-child",
"title": "",
# user_id intentionally absent
}
)
events = _drain(listener, wait=0.1)
assert not any(e.get("type") == "child_ws_created" for e in events)
assert "d" * 32 not in mgr._children.get(ws.id, set())
def test_dispatch_cluster_state_fans_out_when_child_tracked(built_mgr):
mgr, _calls, _storage = built_mgr
ws = mgr.create(user_id="user-1")
child_id = "a" * 32
mgr._add_child(ws.id, child_id)
listener = ws.ui._register_listener()
mgr._dispatch_child_event(
{
"type": "cluster_state",
"ws_id": child_id,
"state": "running",
"tokens": 42,
"node_id": "node-a",
}
)
events = _drain(listener)
state_events = [e for e in events if e.get("type") == "child_ws_state"]
assert len(state_events) == 1
assert state_events[0]["child_ws_id"] == child_id
assert state_events[0]["state"] == "running"
assert state_events[0]["tokens"] == 42
def test_dispatch_ws_closed_fans_out(built_mgr):
mgr, _calls, _storage = built_mgr
ws = mgr.create(user_id="user-1")
child_id = "a" * 32
mgr._add_child(ws.id, child_id)
listener = ws.ui._register_listener()
mgr._dispatch_child_event({"type": "ws_closed", "ws_id": child_id, "reason": "closed"})
events = _drain(listener)
close_events = [e for e in events if e.get("type") == "child_ws_closed"]
assert len(close_events) == 1
assert close_events[0]["child_ws_id"] == child_id
assert close_events[0]["reason"] == "closed"
def test_dispatch_unrelated_state_ignored(built_mgr):
mgr, _calls, _storage = built_mgr
ws = mgr.create(user_id="user-1")
listener = ws.ui._register_listener()
# No _add_child called — ws_id is not in anyone's registry.
mgr._dispatch_child_event({"type": "cluster_state", "ws_id": "a" * 32, "state": "running"})
events = _drain(listener, wait=0.1)
assert not any(e.get("type", "").startswith("child_ws_") for e in events)
def test_shutdown_is_idempotent(built_mgr):
mgr, _calls, _storage = built_mgr
# No fanout started — shutdown must not raise.
mgr.shutdown()
mgr.shutdown()
# ---------------------------------------------------------------------------
# Phase 3 — review-pass-2 regression tests
# ---------------------------------------------------------------------------
def test_rebuild_registry_unions_with_concurrent_adds(built_mgr):
"""A ws_created event that arrives during open() must survive the
subsequent _rebuild_children_registry call the rebuild must UNION
its storage read with whatever the fan-out thread already added."""
mgr, _calls, storage = built_mgr
coord_id = "a" * 32
# Seed a persisted coordinator row — open() will rehydrate it.
storage.register_workstream(
coord_id,
node_id="console",
user_id="user-1",
name="persisted",
kind="coordinator",
parent_ws_id=None,
)
# Persist one child (will show up in rebuild's storage query).
_seed_child_row(storage, parent_ws_id=coord_id, ws_id="b" * 32)
# Simulate the fan-out thread pre-adding a different child_ws_id
# between the placeholder install and the rebuild call. Calling
# open() in this test runs synchronously, so we emulate the race
# by pre-populating the registry for the coord before open.
mgr._add_child(coord_id, "c" * 32)
ws = mgr.open(coord_id, "user-1")
assert ws is not None
# Both the persisted child (from rebuild) AND the pre-added one
# (from the simulated fan-out race) should be present.
assert "b" * 32 in mgr._children[coord_id]
assert "c" * 32 in mgr._children[coord_id]
def test_dispatch_ws_created_atomic_against_close(built_mgr):
"""Concurrent close() during a ws_created dispatch must not leave
the evicted coordinator's registry entry behind.
Regression for a race where the dispatch reads _active_coords
lock-free, close() runs (pops _children[parent]) between the
snapshot read and the _children_lock acquisition, then setdefault
resurrects the entry leaking the registry key forever."""
mgr, _calls, _storage = built_mgr
ws = mgr.create(user_id="user-1")
# Close the coordinator — _children[ws.id] gets popped and
# _active_coords loses the entry.
closed = mgr.close(ws.id)
assert closed
# A ws_created event still arriving for the now-closed parent
# must NOT resurrect the registry entry via setdefault.
mgr._dispatch_child_event(
{
"type": "ws_created",
"ws_id": "d" * 32,
"parent_ws_id": ws.id,
"node_id": "node-a",
"user_id": "user-1",
}
)
assert ws.id not in mgr._children
assert ws.id not in mgr._active_coords
def test_open_impl_eviction_clears_children_registry(built_mgr):
"""When _open_impl evicts an idle coordinator to make room, the
evicted coordinator's _children entry must be popped — matching
the create() eviction path."""
mgr, _calls, storage = built_mgr
# Fill the manager to capacity (max_active=3) with owned coords,
# then pre-seed a 4th as persisted-only so open() triggers eviction.
for i in range(3):
mgr.create(user_id=f"u{i}")
# Record which coord is idlest (oldest create) — it's the eviction
# candidate.
victim_id = mgr._order[0]
# Pre-seed the victim's _children to prove the pop works.
mgr._add_child(victim_id, "z" * 32)
assert victim_id in mgr._children
# Persist a 4th coord row so open() will rehydrate + evict.
fourth_id = "f" * 32
storage.register_workstream(
fourth_id,
node_id="console",
user_id="u3",
name="fourth",
kind="coordinator",
parent_ws_id=None,
)
# Force open() — it must evict the idle victim and clear its
# registry entry in the process.
result = mgr.open_admin(fourth_id)
assert result is not None
assert victim_id not in mgr._workstreams, "victim should have been evicted to make room"
assert victim_id not in mgr._children, (
"_open_impl must pop the evicted coordinator's _children entry "
"(mirrors create() eviction path)"
)
def test_child_to_coord_reverse_index_maintained(built_mgr):
"""_coord_for_child uses the reverse index for O(1) lookup. The
index must stay in sync with the forward set across add/close
paths this test pokes each maintenance point."""
mgr, _calls, _storage = built_mgr
ws = mgr.create(user_id="user-1")
# _add_child path — populates both sides.
assert mgr._add_child(ws.id, "child-1")
assert mgr._coord_for_child("child-1") == ws.id
assert mgr._child_to_coord["child-1"] == ws.id
# close() path — pops both sides.
mgr.close(ws.id)
assert mgr._coord_for_child("child-1") is None
assert "child-1" not in mgr._child_to_coord
def test_prime_children_from_snapshot(built_mgr):
"""start_child_event_fanout uses the collector snapshot to prime
the child registry so a just-opened coordinator sees already-live
children without waiting for the next ws_state event. Simulate
by calling the helper directly."""
mgr, _calls, _storage = built_mgr
ws = mgr.create(user_id="user-1")
snapshot = {
"nodes": [
{
"node_id": "node-a",
"workstreams": [
{"id": "child-1", "parent_ws_id": ws.id, "state": "running"},
{"id": "child-2", "parent_ws_id": ws.id, "state": "idle"},
# Unrelated — parent isn't a tracked coordinator.
{
"id": "foreign-1",
"parent_ws_id": "some-other-coord",
"state": "idle",
},
],
}
]
}
mgr._prime_children_from_snapshot(snapshot)
assert mgr._children[ws.id] == {"child-1", "child-2"}
assert mgr._coord_for_child("child-1") == ws.id
assert mgr._coord_for_child("child-2") == ws.id
# Foreign children with parents we don't track stay out of the
# registry — we only care about live coordinators.
assert mgr._coord_for_child("foreign-1") is None
def test_prime_children_from_empty_snapshot_noop(built_mgr):
"""No nodes → no state changes. Defensive: snapshot shape can
legitimately be missing the ``nodes`` key right after startup."""
mgr, _calls, _storage = built_mgr
ws = mgr.create(user_id="user-1")
mgr._prime_children_from_snapshot({})
mgr._prime_children_from_snapshot({"nodes": []})
assert mgr._children[ws.id] == set()
+54
View File
@@ -0,0 +1,54 @@
"""Tests for the /coordinator/{ws_id} HTML page handler.
The handler serves the shared template with the ws_id injected as a
``data-ws-id`` attribute. It does NOT enforce auth on the page itself
auth gating happens on the API endpoints the page calls (an unauthenticated
visitor lands on the page but all API calls fail).
"""
from __future__ import annotations
import pytest
from starlette.applications import Starlette
from starlette.routing import Route
from starlette.testclient import TestClient
from turnstone.console.server import coordinator_page
@pytest.fixture
def client():
app = Starlette(routes=[Route("/coordinator/{ws_id}", coordinator_page, methods=["GET"])])
return TestClient(app)
def test_valid_ws_id_injects_data_attr(client):
ws_id = "a" * 32
resp = client.get(f"/coordinator/{ws_id}")
assert resp.status_code == 200
assert "text/html" in resp.headers["content-type"]
body = resp.text
# ws_id is injected into the html data-ws-id attribute.
assert f'data-ws-id="{ws_id}"' in body
# Template placeholder is fully substituted.
assert "{{WS_ID}}" not in body
# Sanity: the shared static imports are wired.
assert "/shared/base.css" in body
assert "/static/coordinator/coordinator.js" in body
def test_non_hex_ws_id_returns_400(client):
"""Only hex chars are allowed to avoid HTML injection."""
resp = client.get("/coordinator/not-hex-chars-here")
assert resp.status_code == 400
def test_ws_id_too_long_returns_400(client):
resp = client.get("/coordinator/" + "a" * 65)
assert resp.status_code == 400
def test_uppercase_hex_rejected(client):
# Our ws_ids are lowercase hex; reject mixed/upper to avoid surprises.
resp = client.get("/coordinator/" + "A" * 32)
assert resp.status_code == 400
+96
View File
@@ -0,0 +1,96 @@
"""Tests for console _proxy_auth_headers preserving the coordinator src claim.
Verifies C8 of the coordinator plan: when a console handler processes an
inbound request authenticated with a coordinator-minted JWT (``src ==
"coordinator"``), the upstream JWT the console mints for the proxied
request preserves that source plus the ``coord_ws_id`` custom claim.
For non-coordinator inbound tokens the re-mint still uses
``"console-proxy"`` as before the existing behaviour is unchanged.
"""
from __future__ import annotations
from types import SimpleNamespace
from unittest.mock import MagicMock
import jwt as pyjwt
from turnstone.console.server import _proxy_auth_headers
from turnstone.core.auth import JWT_AUD_SERVER, AuthResult
_SECRET = "x" * 64
def _build_request(auth_result: AuthResult | None):
"""Minimal Request-alike for _proxy_auth_headers."""
state = SimpleNamespace(auth_result=auth_result)
app_state = SimpleNamespace(jwt_secret=_SECRET, proxy_token_mgr=None)
app = MagicMock()
app.state = app_state
req = MagicMock()
req.state = state
req.app = app
return req
def _decode(headers: dict[str, str]) -> dict:
token = headers["Authorization"].removeprefix("Bearer ")
return pyjwt.decode(token, _SECRET, algorithms=["HS256"], audience=JWT_AUD_SERVER)
def test_console_proxy_uses_console_proxy_source_by_default():
"""Non-coordinator inbound tokens still mint src='console-proxy'."""
auth = AuthResult(
user_id="user-1",
scopes=frozenset({"write"}),
token_source="jwt",
permissions=frozenset(),
)
headers = _proxy_auth_headers(_build_request(auth))
payload = _decode(headers)
assert payload["src"] == "console-proxy"
assert "coord_ws_id" not in payload
def test_coordinator_source_is_preserved_on_remint():
"""Inbound src='coordinator' → outbound src='coordinator'."""
auth = AuthResult(
user_id="user-1",
scopes=frozenset({"approve"}),
token_source="coordinator",
permissions=frozenset({"admin.coordinator"}),
extra_claims={"coord_ws_id": "coord-42"},
)
headers = _proxy_auth_headers(_build_request(auth))
payload = _decode(headers)
assert payload["src"] == "coordinator"
assert payload["coord_ws_id"] == "coord-42"
def test_coord_ws_id_absent_when_not_in_inbound_claims():
"""Defensive: if the inbound token is src=coordinator but missing the
coord_ws_id claim (shouldn't happen in practice), the re-mint skips
the custom claim rather than panicking."""
auth = AuthResult(
user_id="user-1",
scopes=frozenset({"write"}),
token_source="coordinator",
permissions=frozenset(),
)
headers = _proxy_auth_headers(_build_request(auth))
payload = _decode(headers)
assert payload["src"] == "coordinator"
assert "coord_ws_id" not in payload
def test_empty_auth_falls_back_to_service_token_or_empty():
"""Without auth_result.user_id, falls through to ServiceTokenManager."""
auth = AuthResult(
user_id="",
scopes=frozenset(),
token_source="config",
permissions=frozenset(),
)
# No proxy_token_mgr configured → empty headers.
headers = _proxy_auth_headers(_build_request(auth))
assert headers == {}
+395
View File
@@ -0,0 +1,395 @@
"""Tests for the coordinator ``/quota`` GET + POST endpoints.
Covers the admin partial-update surface for spawn-budget and
spawn-rate parallel to the /trust + /restrict shape in
``test_coordinator_governance.py``. Kept in its own file so PR B's
review surface stays tight.
"""
from __future__ import annotations
import json
from unittest.mock import MagicMock
import pytest
from starlette.applications import Starlette
from starlette.middleware import Middleware
from starlette.routing import Route
from starlette.testclient import TestClient
from tests._coord_test_helpers import (
_AuthMiddleware,
_build_mgr,
_fake_registry,
_FakeConfigStore,
)
from turnstone.console.server import (
coordinator_quota_get,
coordinator_quota_post,
)
from turnstone.core.spawn_quota import SpawnBudget, TokenBucket
from turnstone.core.storage._sqlite import SQLiteBackend
@pytest.fixture
def storage(tmp_path):
return SQLiteBackend(str(tmp_path / "coord.db"))
_COORD_HEADERS = {"X-Test-User": "user-1", "X-Test-Perms": "admin.coordinator"}
def _make_client(storage, *, coord_mgr, alias="my-model", registry=None) -> TestClient:
app = Starlette(
routes=[
Route(
"/v1/api/coordinator/{ws_id}/quota",
coordinator_quota_get,
methods=["GET"],
),
Route(
"/v1/api/coordinator/{ws_id}/quota",
coordinator_quota_post,
methods=["POST"],
),
],
middleware=[Middleware(_AuthMiddleware)],
)
app.state.coord_mgr = coord_mgr
app.state.config_store = _FakeConfigStore({"coordinator.model_alias": alias})
app.state.coord_registry = registry
app.state.coord_registry_error = "" if coord_mgr else "registry missing"
app.state.auth_storage = storage
app.state.jwt_secret = "x" * 64
return TestClient(app)
def _install_quota(coord) -> tuple[SpawnBudget, TokenBucket]:
"""Attach a real budget + bucket to the coord session under test."""
budget = SpawnBudget(20)
bucket = TokenBucket(5.0, 10)
session = MagicMock()
session._spawn_budget = budget
session._spawn_bucket = bucket
session._coord_client = MagicMock()
def _get_state():
return {
"spawn_budget": budget.budget,
"spawn_rate": {
"tokens_per_minute": bucket.tokens_per_minute,
"burst": bucket.burst,
"tokens_available": bucket.tokens,
},
}
def _set_budget(n):
budget.set_budget(int(n))
def _set_rate(tpm, brst):
bucket.set_rate(float(tpm), int(brst))
session.get_quota_state.side_effect = _get_state
session.set_spawn_budget.side_effect = _set_budget
session.set_spawn_rate.side_effect = _set_rate
coord.session = session
return budget, bucket
# ---------------------------------------------------------------------------
# GET
# ---------------------------------------------------------------------------
def test_quota_get_returns_live_snapshot(storage):
mgr = _build_mgr(storage)
coord = mgr.create(user_id="user-1", name="coord-a")
_install_quota(coord)
client = _make_client(storage, coord_mgr=mgr, registry=_fake_registry())
resp = client.get(
f"/v1/api/coordinator/{coord.id}/quota",
headers=_COORD_HEADERS,
)
assert resp.status_code == 200
body = resp.json()
assert body["status"] == "ok"
assert body["spawn_budget"] == 20
assert body["spawn_rate"]["tokens_per_minute"] == 5.0
assert body["spawn_rate"]["burst"] == 10
assert 0 <= body["spawn_rate"]["tokens_available"] <= 10
def test_quota_get_404_when_session_not_loaded(storage):
mgr = _build_mgr(storage)
coord = mgr.create(user_id="user-1", name="coord-a")
coord.session = None
client = _make_client(storage, coord_mgr=mgr, registry=_fake_registry())
resp = client.get(
f"/v1/api/coordinator/{coord.id}/quota",
headers=_COORD_HEADERS,
)
assert resp.status_code == 404
# ---------------------------------------------------------------------------
# POST — happy path
# ---------------------------------------------------------------------------
def test_quota_post_updates_budget_only_and_audits(storage):
mgr = _build_mgr(storage)
coord = mgr.create(user_id="user-1", name="coord-a")
budget, bucket = _install_quota(coord)
client = _make_client(storage, coord_mgr=mgr, registry=_fake_registry())
resp = client.post(
f"/v1/api/coordinator/{coord.id}/quota",
json={"spawn_budget": 42},
headers=_COORD_HEADERS,
)
assert resp.status_code == 200
body = resp.json()
assert body["spawn_budget"] == 42
# Rate left untouched — the partial update didn't widen it.
assert body["spawn_rate"]["tokens_per_minute"] == 5.0
assert body["spawn_rate"]["burst"] == 10
assert budget.budget == 42
events = [e for e in storage.list_audit_events() if e["action"] == "coordinator.quota.updated"]
assert len(events) == 1
detail = json.loads(events[0]["detail"])
assert detail["before"]["spawn_budget"] == 20
assert detail["after"]["spawn_budget"] == 42
def test_quota_post_accepts_nested_spawn_rate(storage):
mgr = _build_mgr(storage)
coord = mgr.create(user_id="user-1", name="coord-a")
_budget, bucket = _install_quota(coord)
client = _make_client(storage, coord_mgr=mgr, registry=_fake_registry())
resp = client.post(
f"/v1/api/coordinator/{coord.id}/quota",
json={"spawn_rate": {"tokens_per_minute": 30.0, "burst": 15}},
headers=_COORD_HEADERS,
)
assert resp.status_code == 200
body = resp.json()
assert body["spawn_rate"]["tokens_per_minute"] == 30.0
assert body["spawn_rate"]["burst"] == 15
assert bucket.burst == 15
def test_quota_post_accepts_flat_aliases(storage):
"""The admin UI may flatten the rate object — both shapes must work."""
mgr = _build_mgr(storage)
coord = mgr.create(user_id="user-1", name="coord-a")
_budget, bucket = _install_quota(coord)
client = _make_client(storage, coord_mgr=mgr, registry=_fake_registry())
resp = client.post(
f"/v1/api/coordinator/{coord.id}/quota",
json={"tokens_per_minute": 12.0, "burst": 4},
headers=_COORD_HEADERS,
)
assert resp.status_code == 200
assert bucket.tokens_per_minute == 12.0
assert bucket.burst == 4
def test_quota_post_burst_only_preserves_refill_rate(storage):
"""Changing only burst shouldn't zero the refill rate — a previous
bug-prone shape in partial-update handlers that overwrite missing
fields with defaults. Here the handler must read current state
for the missing dimension."""
mgr = _build_mgr(storage)
coord = mgr.create(user_id="user-1", name="coord-a")
_budget, bucket = _install_quota(coord)
client = _make_client(storage, coord_mgr=mgr, registry=_fake_registry())
resp = client.post(
f"/v1/api/coordinator/{coord.id}/quota",
json={"burst": 3},
headers=_COORD_HEADERS,
)
assert resp.status_code == 200
assert bucket.tokens_per_minute == 5.0 # unchanged
assert bucket.burst == 3
def test_quota_post_updates_all_three_knobs_at_once(storage):
mgr = _build_mgr(storage)
coord = mgr.create(user_id="user-1", name="coord-a")
budget, bucket = _install_quota(coord)
client = _make_client(storage, coord_mgr=mgr, registry=_fake_registry())
resp = client.post(
f"/v1/api/coordinator/{coord.id}/quota",
json={"spawn_budget": 50, "tokens_per_minute": 0.0, "burst": 1},
headers=_COORD_HEADERS,
)
assert resp.status_code == 200
assert budget.budget == 50
assert bucket.tokens_per_minute == 0.0
assert bucket.burst == 1
# ---------------------------------------------------------------------------
# POST — validation failures
# ---------------------------------------------------------------------------
def test_quota_post_rejects_empty_body(storage):
mgr = _build_mgr(storage)
coord = mgr.create(user_id="user-1", name="coord-a")
_install_quota(coord)
client = _make_client(storage, coord_mgr=mgr, registry=_fake_registry())
resp = client.post(
f"/v1/api/coordinator/{coord.id}/quota",
json={},
headers=_COORD_HEADERS,
)
assert resp.status_code == 400
def test_quota_post_rejects_out_of_range_budget(storage):
mgr = _build_mgr(storage)
coord = mgr.create(user_id="user-1", name="coord-a")
_install_quota(coord)
client = _make_client(storage, coord_mgr=mgr, registry=_fake_registry())
for bad in (0, -5, 10_000):
resp = client.post(
f"/v1/api/coordinator/{coord.id}/quota",
json={"spawn_budget": bad},
headers=_COORD_HEADERS,
)
assert resp.status_code == 400, f"expected 400 for {bad}"
def test_quota_post_rejects_non_numeric_rate(storage):
mgr = _build_mgr(storage)
coord = mgr.create(user_id="user-1", name="coord-a")
_install_quota(coord)
client = _make_client(storage, coord_mgr=mgr, registry=_fake_registry())
resp = client.post(
f"/v1/api/coordinator/{coord.id}/quota",
json={"tokens_per_minute": "fast"},
headers=_COORD_HEADERS,
)
assert resp.status_code == 400
def test_quota_post_rejects_out_of_range_rate(storage):
mgr = _build_mgr(storage)
coord = mgr.create(user_id="user-1", name="coord-a")
_install_quota(coord)
client = _make_client(storage, coord_mgr=mgr, registry=_fake_registry())
for bad_tpm in (-1.0, 1_000.0):
resp = client.post(
f"/v1/api/coordinator/{coord.id}/quota",
json={"tokens_per_minute": bad_tpm},
headers=_COORD_HEADERS,
)
assert resp.status_code == 400
def test_quota_post_rejects_out_of_range_burst(storage):
mgr = _build_mgr(storage)
coord = mgr.create(user_id="user-1", name="coord-a")
_install_quota(coord)
client = _make_client(storage, coord_mgr=mgr, registry=_fake_registry())
for bad in (0, -1, 10_000):
resp = client.post(
f"/v1/api/coordinator/{coord.id}/quota",
json={"burst": bad},
headers=_COORD_HEADERS,
)
assert resp.status_code == 400
def test_quota_post_rejects_mixed_nested_and_flat_body(storage):
"""Schema description says 'don't mix' — the handler enforces it with 400.
Silently picking one side would make the admin UI's behaviour
unpredictable when it accidentally sends both shapes (e.g. during
a form-rewrite transition).
"""
mgr = _build_mgr(storage)
coord = mgr.create(user_id="user-1", name="coord-a")
_install_quota(coord)
client = _make_client(storage, coord_mgr=mgr, registry=_fake_registry())
resp = client.post(
f"/v1/api/coordinator/{coord.id}/quota",
json={"spawn_rate": {"burst": 5}, "burst": 9},
headers=_COORD_HEADERS,
)
assert resp.status_code == 400
assert "conflicting" in resp.json()["error"]
def test_quota_post_rejects_non_object_spawn_rate(storage):
mgr = _build_mgr(storage)
coord = mgr.create(user_id="user-1", name="coord-a")
_install_quota(coord)
client = _make_client(storage, coord_mgr=mgr, registry=_fake_registry())
resp = client.post(
f"/v1/api/coordinator/{coord.id}/quota",
json={"spawn_rate": "not-an-object"},
headers=_COORD_HEADERS,
)
assert resp.status_code == 400
def test_quota_post_rejects_bool_as_numeric_field(storage):
"""``True`` passes ``isinstance(x, int)`` in Python — explicit reject."""
mgr = _build_mgr(storage)
coord = mgr.create(user_id="user-1", name="coord-a")
_install_quota(coord)
client = _make_client(storage, coord_mgr=mgr, registry=_fake_registry())
for payload in (
{"spawn_budget": True},
{"burst": True},
{"tokens_per_minute": True},
):
resp = client.post(
f"/v1/api/coordinator/{coord.id}/quota",
json=payload,
headers=_COORD_HEADERS,
)
assert resp.status_code == 400, f"expected 400 for {payload}"
def test_quota_post_404_when_session_not_loaded(storage):
mgr = _build_mgr(storage)
coord = mgr.create(user_id="user-1", name="coord-a")
coord.session = None
client = _make_client(storage, coord_mgr=mgr, registry=_fake_registry())
resp = client.post(
f"/v1/api/coordinator/{coord.id}/quota",
json={"spawn_budget": 5},
headers=_COORD_HEADERS,
)
assert resp.status_code == 404
def test_quota_post_without_admin_coordinator_is_rejected(storage):
mgr = _build_mgr(storage)
coord = mgr.create(user_id="user-1", name="coord-a")
_install_quota(coord)
client = _make_client(storage, coord_mgr=mgr, registry=_fake_registry())
resp = client.post(
f"/v1/api/coordinator/{coord.id}/quota",
json={"spawn_budget": 5},
headers={"X-Test-User": "user-1", "X-Test-Perms": ""},
)
assert resp.status_code in (401, 403)
File diff suppressed because it is too large Load Diff
+11 -11
View File
@@ -154,7 +154,7 @@ class TestSingleEdit:
)
assert result["needs_approval"]
call_id, msg = session._exec_edit_file(result)
_, msg = session._exec_edit_file(result)
assert "applied 1 edit" in msg
with open(path) as f:
assert f.read() == "foo\nbar\nbaz\n"
@@ -172,7 +172,7 @@ class TestSingleEdit:
assert result["needs_approval"]
assert "deletion" in result["preview"]
call_id, msg = session._exec_edit_file(result)
_, msg = session._exec_edit_file(result)
with open(sample_file) as f:
assert f.read() == "line1\nline2\nline4\nline5\n"
@@ -196,7 +196,7 @@ class TestBatchEdit:
assert result["needs_approval"]
assert "2 edits" in result["header"]
call_id, msg = session._exec_edit_file(result)
_, msg = session._exec_edit_file(result)
assert "applied 2 edits" in msg
with open(sample_file) as f:
assert f.read() == "first\nline2\nline3\nline4\nlast\n"
@@ -216,7 +216,7 @@ class TestBatchEdit:
)
assert result["needs_approval"]
call_id, msg = session._exec_edit_file(result)
_, msg = session._exec_edit_file(result)
assert "applied 3 edits" in msg
with open(sample_file) as f:
assert f.read() == "line1\nsecond\nthird\nfourth\nline5\n"
@@ -238,7 +238,7 @@ class TestBatchEdit:
)
assert result["needs_approval"]
call_id, msg = session._exec_edit_file(result)
_, msg = session._exec_edit_file(result)
assert "overlap" in msg.lower()
# File should be untouched
with open(path) as f:
@@ -305,7 +305,7 @@ class TestBatchEdit:
)
assert result["needs_approval"]
call_id, msg = session._exec_edit_file(result)
_, msg = session._exec_edit_file(result)
assert "applied 2 edits" in msg
with open(path) as f:
assert f.read() == "first_foo\nbar\nsecond_foo\nbaz\n"
@@ -324,7 +324,7 @@ class TestBatchEdit:
)
assert result["needs_approval"]
call_id, msg = session._exec_edit_file(result)
_, msg = session._exec_edit_file(result)
with open(sample_file) as f:
assert f.read() == "line1\nline3\nline5\n"
@@ -344,7 +344,7 @@ class TestBatchEdit:
# Single edit — no "(N edits)" count in header
assert "edits)" not in result["header"]
call_id, msg = session._exec_edit_file(result)
_, msg = session._exec_edit_file(result)
assert "applied 1 edit" in msg
with open(sample_file) as f:
assert f.read() == "line1\nline2\nmiddle\nline4\nline5\n"
@@ -414,7 +414,7 @@ class TestExecEdgeCases:
with open(sample_file, "w") as f:
f.write("completely different content\n")
call_id, msg = session._exec_edit_file(result)
_, msg = session._exec_edit_file(result)
assert "no longer found" in msg
def test_file_deleted_between_prepare_and_exec(self, session, sample_file):
@@ -431,7 +431,7 @@ class TestExecEdgeCases:
os.unlink(sample_file)
call_id, msg = session._exec_edit_file(result)
_, msg = session._exec_edit_file(result)
assert "Error" in msg
def test_batch_file_changed_partial_match(self, session, sample_file):
@@ -453,7 +453,7 @@ class TestExecEdgeCases:
with open(sample_file, "w") as f:
f.write("line1\nline2\nline3\nline4\n")
call_id, msg = session._exec_edit_file(result)
_, msg = session._exec_edit_file(result)
assert "no longer found" in msg
# line1 should NOT have been edited (atomic failure)
with open(sample_file) as f:
+1
View File
@@ -57,6 +57,7 @@ class _InjectAuthMiddleware(BaseHTTPMiddleware):
"admin.users",
"admin.orgs",
"admin.policies",
"admin.prompt_policies",
"admin.skills",
"admin.usage",
"admin.audit",
-15
View File
@@ -1,15 +0,0 @@
"""Tests for turnstone.core.hash_ring."""
from turnstone.core.hash_ring import bucket_of
class TestBucketOf:
def test_known_vectors(self):
assert bucket_of("a3f1" + "0" * 28) == 0xA3F1
assert bucket_of("0000" + "a" * 28) == 0
assert bucket_of("ffff" + "b" * 28) == 65535
def test_hex_prefix(self):
# Only the first 4 hex chars matter — the rest is ignored.
assert bucket_of("abcd0000") == bucket_of("abcdffff")
assert bucket_of("abcd0000") == 0xABCD
-159
View File
@@ -1,159 +0,0 @@
"""Tests for the hash ring routing storage methods."""
from __future__ import annotations
class TestHashRingBuckets:
def test_list_empty(self, storage):
assert storage.list_ring_buckets() == []
def test_seed_and_list(self, storage):
storage.seed_ring_buckets([(0, "node-a"), (1, "node-b"), (2, "node-a")])
rows = storage.list_ring_buckets()
assert len(rows) == 3
assert rows[0] == {"bucket": 0, "node_id": "node-a"}
assert rows[1] == {"bucket": 1, "node_id": "node-b"}
assert rows[2] == {"bucket": 2, "node_id": "node-a"}
def test_seed_idempotent(self, storage):
storage.seed_ring_buckets([(0, "node-a"), (1, "node-b")])
# Re-seed with conflicting assignment: should keep original
storage.seed_ring_buckets([(0, "node-x"), (2, "node-c")])
rows = storage.list_ring_buckets()
by_bucket = {r["bucket"]: r["node_id"] for r in rows}
assert by_bucket[0] == "node-a" # original preserved
assert by_bucket[1] == "node-b"
assert by_bucket[2] == "node-c" # new bucket added
def test_assign_buckets(self, storage):
storage.seed_ring_buckets([(0, "node-a"), (1, "node-a"), (2, "node-b")])
storage.assign_buckets([0, 1], "node-c")
rows = storage.list_ring_buckets()
by_bucket = {r["bucket"]: r["node_id"] for r in rows}
assert by_bucket[0] == "node-c"
assert by_bucket[1] == "node-c"
assert by_bucket[2] == "node-b"
def test_assign_returns_count(self, storage):
storage.seed_ring_buckets([(0, "node-a"), (1, "node-a")])
count = storage.assign_buckets([0, 1], "node-b")
assert count == 2
# Empty list returns 0
assert storage.assign_buckets([], "node-x") == 0
class TestBucketStats:
def test_increment_creates_row(self, storage):
storage.increment_bucket_count(42)
stats = storage.list_bucket_stats()
assert len(stats) == 1
assert stats[0]["bucket"] == 42
assert stats[0]["ws_count"] == 1
assert stats[0]["active_count"] == 0
def test_increment_active(self, storage):
storage.increment_bucket_count(10, active=True)
stats = storage.list_bucket_stats()
assert stats[0]["ws_count"] == 1
assert stats[0]["active_count"] == 1
# Increment again without active
storage.increment_bucket_count(10)
stats = storage.list_bucket_stats()
assert stats[0]["ws_count"] == 2
assert stats[0]["active_count"] == 1
def test_decrement(self, storage):
storage.increment_bucket_count(5, active=True)
storage.increment_bucket_count(5, active=True)
storage.decrement_bucket_count(5, active=True)
stats = storage.list_bucket_stats()
assert stats[0]["ws_count"] == 1
assert stats[0]["active_count"] == 1
def test_decrement_clamps_at_zero(self, storage):
storage.increment_bucket_count(7)
storage.decrement_bucket_count(7)
storage.decrement_bucket_count(7) # already at 0
stats = storage.list_bucket_stats()
# ws_count is 0, so should not appear (filter ws_count > 0)
assert len(stats) == 0
def test_adjust_active_only(self, storage):
storage.increment_bucket_count(20, active=True)
storage.increment_bucket_count(20, active=True)
# Decrease active without changing ws_count
storage.adjust_bucket_active(20, -1)
stats = storage.list_bucket_stats()
assert stats[0]["ws_count"] == 2
assert stats[0]["active_count"] == 1
# Clamp at zero
storage.adjust_bucket_active(20, -5)
stats = storage.list_bucket_stats()
assert stats[0]["active_count"] == 0
def test_list_sparse(self, storage):
storage.increment_bucket_count(100)
storage.increment_bucket_count(200)
storage.increment_bucket_count(300)
# Decrement 200 to zero
storage.decrement_bucket_count(200)
stats = storage.list_bucket_stats()
buckets = [s["bucket"] for s in stats]
assert 100 in buckets
assert 200 not in buckets
assert 300 in buckets
def test_set_bucket_stat_creates(self, storage):
"""set_bucket_stat upserts a new row."""
storage.set_bucket_stat(42, 5, 2)
stats = storage.list_bucket_stats()
row = next(s for s in stats if s["bucket"] == 42)
assert row["ws_count"] == 5
assert row["active_count"] == 2
def test_set_bucket_stat_overwrites(self, storage):
"""set_bucket_stat overwrites existing values."""
storage.set_bucket_stat(42, 10, 3)
storage.set_bucket_stat(42, 2, 0)
stats = storage.list_bucket_stats()
row = next(s for s in stats if s["bucket"] == 42)
assert row["ws_count"] == 2
assert row["active_count"] == 0
def test_set_bucket_stat_zero_removes_from_sparse(self, storage):
"""Setting ws_count=0 means list_bucket_stats excludes it (sparse)."""
storage.set_bucket_stat(42, 5, 1)
storage.set_bucket_stat(42, 0, 0)
stats = storage.list_bucket_stats()
assert not any(s["bucket"] == 42 for s in stats)
class TestWorkstreamOverrides:
def test_set_and_list(self, storage):
storage.set_workstream_override("ws-001", "node-a", reason="affinity")
overrides = storage.list_workstream_overrides()
assert len(overrides) == 1
assert overrides[0]["ws_id"] == "ws-001"
assert overrides[0]["node_id"] == "node-a"
assert overrides[0]["reason"] == "affinity"
def test_upsert(self, storage):
storage.set_workstream_override("ws-002", "node-a")
storage.set_workstream_override("ws-002", "node-b", reason="migration")
overrides = storage.list_workstream_overrides()
assert len(overrides) == 1
assert overrides[0]["node_id"] == "node-b"
assert overrides[0]["reason"] == "migration"
def test_delete(self, storage):
storage.set_workstream_override("ws-003", "node-a")
result = storage.delete_workstream_override("ws-003")
assert result is True
assert storage.list_workstream_overrides() == []
def test_delete_nonexistent(self, storage):
result = storage.delete_workstream_override("ws-nope")
assert result is False
def test_list_empty(self, storage):
assert storage.list_workstream_overrides() == []
+157 -235
View File
@@ -1,4 +1,4 @@
"""Tests for turnstone.core.healthcheck — backend health monitor with circuit breaker."""
"""Tests for turnstone.core.healthcheck — passive backend health tracking."""
from __future__ import annotations
@@ -10,39 +10,16 @@ import pytest
if TYPE_CHECKING:
from collections.abc import Generator
from turnstone.core.healthcheck import BackendHealthMonitor, CircuitState
from turnstone.core.healthcheck import BackendHealthTracker, HealthTrackerRegistry
# ---------------------------------------------------------------------------
# CircuitState enum
# Fixtures
# ---------------------------------------------------------------------------
class TestCircuitState:
def test_closed(self) -> None:
assert CircuitState.CLOSED.value == "closed"
def test_open(self) -> None:
assert CircuitState.OPEN.value == "open"
def test_half_open(self) -> None:
assert CircuitState.HALF_OPEN.value == "half_open"
# ---------------------------------------------------------------------------
# BackendHealthMonitor
# ---------------------------------------------------------------------------
@pytest.fixture
def mock_client() -> MagicMock:
client = MagicMock()
client.models.list.return_value.data = [MagicMock(id="test-model")]
return client
@pytest.fixture
def mock_metrics() -> Generator[MagicMock]:
"""Patch the metrics singleton so set_backend_status / set_circuit_state exist."""
"""Patch the metrics singleton so set_backend_status exists."""
m = MagicMock()
with (
patch("turnstone.core.healthcheck.metrics", m, create=True),
@@ -51,240 +28,185 @@ def mock_metrics() -> Generator[MagicMock]:
yield m
def _make_monitor(
client: MagicMock,
failure_threshold: int = 3,
cooldown: float = 60.0,
) -> BackendHealthMonitor:
return BackendHealthMonitor(
client=client,
probe_interval=1.0,
probe_timeout=1.0,
failure_threshold=failure_threshold,
cooldown=cooldown,
)
def _make_tracker(failure_threshold: int = 3) -> BackendHealthTracker:
return BackendHealthTracker(failure_threshold=failure_threshold)
class TestBackendHealthMonitor:
def test_starts_closed(self, mock_client: MagicMock) -> None:
mon = _make_monitor(mock_client)
assert mon.circuit_state == CircuitState.CLOSED
assert mon.is_healthy is True
# ---------------------------------------------------------------------------
# BackendHealthTracker
# ---------------------------------------------------------------------------
def test_record_failure_increments(
self, mock_client: MagicMock, mock_metrics: MagicMock
) -> None:
"""Failures below threshold do not open the circuit."""
mon = _make_monitor(mock_client, failure_threshold=5)
class TestBackendHealthTracker:
def test_starts_healthy(self) -> None:
t = _make_tracker()
assert t.is_healthy is True
assert t.is_degraded is False
assert t.consecutive_failures == 0
def test_failures_below_threshold(self, mock_metrics: MagicMock) -> None:
"""Failures below threshold do not degrade."""
t = _make_tracker(failure_threshold=5)
for _ in range(4):
mon.record_failure()
assert mon.circuit_state == CircuitState.CLOSED
t.record_failure()
assert t.is_healthy is True
assert t.consecutive_failures == 4
def test_opens_after_threshold(self, mock_client: MagicMock, mock_metrics: MagicMock) -> None:
mon = _make_monitor(mock_client, failure_threshold=3)
def test_degrades_at_threshold(self, mock_metrics: MagicMock) -> None:
t = _make_tracker(failure_threshold=3)
for _ in range(3):
mon.record_failure()
assert mon.circuit_state == CircuitState.OPEN
assert mon.is_healthy is False
t.record_failure()
assert t.is_degraded is True
assert t.is_healthy is False
def test_should_reject_when_open(self, mock_client: MagicMock, mock_metrics: MagicMock) -> None:
mon = _make_monitor(mock_client, failure_threshold=1, cooldown=9999.0)
mon.record_failure()
assert mon.circuit_state == CircuitState.OPEN
assert mon.acquire_request_permit() is False
def test_stays_degraded_on_more_failures(self, mock_metrics: MagicMock) -> None:
t = _make_tracker(failure_threshold=2)
for _ in range(5):
t.record_failure()
assert t.is_degraded is True
assert t.consecutive_failures == 5
@patch("turnstone.core.healthcheck.time")
def test_half_open_after_cooldown(
self, mock_time: MagicMock, mock_client: MagicMock, mock_metrics: MagicMock
) -> None:
"""After cooldown elapses, should_allow_request transitions to HALF_OPEN."""
t = 1000.0
mock_time.monotonic.return_value = t
def test_success_clears_degraded(self, mock_metrics: MagicMock) -> None:
t = _make_tracker(failure_threshold=2)
t.record_failure()
t.record_failure()
assert t.is_degraded is True
t.record_success()
assert t.is_healthy is True
assert t.consecutive_failures == 0
mon = _make_monitor(mock_client, failure_threshold=1, cooldown=60.0)
# Override _last_state_change to use our mocked time
mon._last_state_change = t
mon.record_failure()
assert mon.circuit_state == CircuitState.OPEN
def test_success_resets_failure_count(self, mock_metrics: MagicMock) -> None:
t = _make_tracker(failure_threshold=5)
for _ in range(4):
t.record_failure()
t.record_success()
assert t.consecutive_failures == 0
# Should need 5 more failures to degrade
for _ in range(4):
t.record_failure()
assert t.is_healthy is True
# Advance past cooldown
mock_time.monotonic.return_value = t + 61.0
assert mon.acquire_request_permit() is True
assert mon.circuit_state == CircuitState.HALF_OPEN # type: ignore[comparison-overlap]
def test_state_changed_callback_on_degrade(self, mock_metrics: MagicMock) -> None:
events: list[str] = []
t = BackendHealthTracker(failure_threshold=2, on_state_changed=events.append)
t.record_failure()
assert events == []
t.record_failure()
assert events == ["degraded"]
def test_success_resets(self, mock_client: MagicMock, mock_metrics: MagicMock) -> None:
"""record_success resets failures and closes circuit from any state."""
mon = _make_monitor(mock_client, failure_threshold=1)
mon.record_failure()
assert mon.circuit_state == CircuitState.OPEN
def test_state_changed_callback_on_recover(self, mock_metrics: MagicMock) -> None:
events: list[str] = []
t = BackendHealthTracker(failure_threshold=1, on_state_changed=events.append)
t.record_failure()
assert events == ["degraded"]
t.record_success()
assert events == ["degraded", "healthy"]
mon.record_success()
assert mon.circuit_state == CircuitState.CLOSED # type: ignore[comparison-overlap]
assert mon.is_healthy is True
# Internal counter should be reset
assert mon._consecutive_failures == 0
def test_no_callback_when_already_degraded(self, mock_metrics: MagicMock) -> None:
"""Extra failures after degraded don't fire again."""
events: list[str] = []
t = BackendHealthTracker(failure_threshold=1, on_state_changed=events.append)
t.record_failure()
t.record_failure()
t.record_failure()
assert events == ["degraded"] # only once
def test_should_allow_when_closed(self, mock_client: MagicMock) -> None:
mon = _make_monitor(mock_client)
assert mon.acquire_request_permit() is True
def test_no_callback_when_already_healthy(self, mock_metrics: MagicMock) -> None:
"""Success while healthy doesn't fire."""
events: list[str] = []
t = BackendHealthTracker(failure_threshold=3, on_state_changed=events.append)
t.record_success()
t.record_success()
assert events == []
def test_half_open_allows_only_one_request(
self, mock_client: MagicMock, mock_metrics: MagicMock
) -> None:
"""HALF_OPEN permits exactly one probe; subsequent callers are blocked."""
mon = _make_monitor(mock_client, failure_threshold=1)
mon.record_failure()
assert mon.circuit_state == CircuitState.OPEN
def test_no_direct_metrics_calls(self) -> None:
"""Tracker does not touch metrics — the server callback handles it."""
t = _make_tracker(failure_threshold=1)
t.record_failure()
t.record_success()
# No assertion on metrics — the tracker delegates metric updates
# to the server-level callback via on_state_changed
# Force into HALF_OPEN with permit
with mon._lock:
mon._state = CircuitState.HALF_OPEN
mon._half_open_permit = True
# First caller gets through
assert mon.acquire_request_permit() is True
# Second caller is blocked
assert mon.acquire_request_permit() is False
# Third caller is also blocked
assert mon.acquire_request_permit() is False
# ---------------------------------------------------------------------------
# HealthTrackerRegistry
# ---------------------------------------------------------------------------
def test_half_open_success_reopens_to_all(
self, mock_client: MagicMock, mock_metrics: MagicMock
) -> None:
"""After probe succeeds in HALF_OPEN, circuit closes and all requests pass."""
mon = _make_monitor(mock_client, failure_threshold=1)
mon.record_failure()
with mon._lock:
mon._state = CircuitState.HALF_OPEN
mon._half_open_permit = False # permit already consumed
# Probe succeeds
mon.record_success()
assert mon.circuit_state == CircuitState.CLOSED # type: ignore[comparison-overlap]
# All callers pass now
assert mon.acquire_request_permit() is True
assert mon.acquire_request_permit() is True
class TestHealthTrackerRegistry:
def test_same_backend_shares_tracker(self, mock_metrics: MagicMock) -> None:
"""Two aliases on the same (provider, base_url) share a tracker."""
reg = HealthTrackerRegistry(failure_threshold=5)
t1 = reg.get_tracker("openai", "https://api.openai.com/v1")
t2 = reg.get_tracker("openai", "https://api.openai.com/v1")
assert t1 is t2
def test_half_open_failure_blocks_all(
self, mock_client: MagicMock, mock_metrics: MagicMock
) -> None:
"""After probe fails in HALF_OPEN, circuit reopens and all requests blocked."""
mon = _make_monitor(mock_client, failure_threshold=1, cooldown=9999.0)
mon.record_failure()
with mon._lock:
mon._state = CircuitState.HALF_OPEN
mon._half_open_permit = False
def test_different_backends_independent(self, mock_metrics: MagicMock) -> None:
"""Different (provider, base_url) pairs get independent trackers."""
reg = HealthTrackerRegistry(failure_threshold=5)
t_cloud = reg.get_tracker("openai", "https://api.openai.com/v1")
t_local = reg.get_tracker("openai-compatible", "http://localhost:8000/v1")
assert t_cloud is not t_local
# Probe fails
mon.record_failure()
assert mon.circuit_state == CircuitState.OPEN
assert mon.acquire_request_permit() is False
def test_trailing_slash_normalized(self, mock_metrics: MagicMock) -> None:
"""Trailing slashes on base_url are normalized away."""
reg = HealthTrackerRegistry(failure_threshold=5)
t1 = reg.get_tracker("openai", "https://api.openai.com/v1/")
t2 = reg.get_tracker("openai", "https://api.openai.com/v1")
assert t1 is t2
def test_half_open_failure_reopens(
self, mock_client: MagicMock, mock_metrics: MagicMock
) -> None:
"""A failure in HALF_OPEN re-opens the circuit immediately."""
mon = _make_monitor(mock_client, failure_threshold=1)
mon.record_failure()
assert mon.circuit_state == CircuitState.OPEN
def test_degraded_isolation(self, mock_metrics: MagicMock) -> None:
"""Degrading one backend does not affect another."""
reg = HealthTrackerRegistry(failure_threshold=2)
t_cloud = reg.get_tracker("openai", "https://api.openai.com/v1")
t_local = reg.get_tracker("openai-compatible", "http://localhost:8000/v1")
# Degrade the cloud tracker
t_cloud.record_failure()
t_cloud.record_failure()
assert t_cloud.is_degraded is True
# Local should be unaffected
assert t_local.is_healthy is True
# Force into HALF_OPEN
with mon._lock:
mon._state = CircuitState.HALF_OPEN
mon._update_metrics()
def test_get_tracker_for_alias(self, mock_metrics: MagicMock) -> None:
"""get_tracker_for_alias looks up by model config's backend."""
from turnstone.core.model_registry import ModelConfig, ModelRegistry
# Another failure should reopen
mon.record_failure()
assert mon.circuit_state == CircuitState.OPEN
models = {
"cloud": ModelConfig(
"cloud", "https://api.openai.com/v1", "sk", "gpt-4o", provider="openai"
),
"local": ModelConfig(
"local", "http://localhost:8000/v1", "x", "qwen", provider="openai-compatible"
),
}
model_reg = ModelRegistry(models=models, default="cloud")
def test_probe_success_closes(self, mock_client: MagicMock, mock_metrics: MagicMock) -> None:
"""A successful probe closes the circuit."""
mon = _make_monitor(mock_client, failure_threshold=1)
mon.record_failure()
assert mon.circuit_state == CircuitState.OPEN
reg = HealthTrackerRegistry(failure_threshold=5)
# No tracker created yet — should return None
assert reg.get_tracker_for_alias(model_reg, "cloud") is None
# Simulate probe success
assert mon._probe_once() is True
mon.record_success()
assert mon.circuit_state == CircuitState.CLOSED # type: ignore[comparison-overlap]
# Create a tracker for the cloud backend
t = reg.get_tracker("openai", "https://api.openai.com/v1")
assert reg.get_tracker_for_alias(model_reg, "cloud") is t
def test_probe_failure_opens(self, mock_client: MagicMock, mock_metrics: MagicMock) -> None:
"""Enough probe failures open the circuit."""
mock_client.with_options.return_value.models.list.side_effect = ConnectionError("down")
mon = _make_monitor(mock_client, failure_threshold=2)
# Local alias should still return None (no tracker for that backend)
assert reg.get_tracker_for_alias(model_reg, "local") is None
assert mon._probe_once() is False
mon.record_failure()
assert mon.circuit_state == CircuitState.CLOSED # only 1 failure
assert mon._probe_once() is False
mon.record_failure()
assert mon.circuit_state == CircuitState.OPEN # type: ignore[comparison-overlap]
def test_probe_loop_autonomous_recovery(
self, mock_client: MagicMock, mock_metrics: MagicMock
) -> None:
"""_probe_loop transitions OPEN → HALF_OPEN → CLOSED without user requests."""
# Use very short intervals so the test is fast
mon = BackendHealthMonitor(
client=mock_client,
probe_interval=0.05,
probe_timeout=1.0,
failure_threshold=1,
cooldown=0.1,
def test_state_changed_callback(self, mock_metrics: MagicMock) -> None:
"""on_state_changed fires with backend key and state."""
events: list[tuple[str, str]] = []
reg = HealthTrackerRegistry(
failure_threshold=2,
on_state_changed=lambda backend, state: events.append((backend, state)),
)
# Trip the circuit
mon.record_failure()
assert mon.circuit_state == CircuitState.OPEN
t = reg.get_tracker("openai", "https://api.openai.com/v1")
t.record_failure()
t.record_failure() # triggers degraded
assert len(events) == 1
assert events[0][0] == "openai:https://api.openai.com/v1"
assert events[0][1] == "degraded"
# Backend is healthy — probe_once will succeed
mock_client.with_options.return_value.models.list.return_value = MagicMock()
# Start the probe loop and wait for autonomous recovery
mon.start()
try:
import time
deadline = time.monotonic() + 5.0
while mon.circuit_state != CircuitState.CLOSED and time.monotonic() < deadline:
time.sleep(0.05)
assert mon.circuit_state == CircuitState.CLOSED
# User requests should flow again without anyone calling acquire_request_permit
assert mon.acquire_request_permit() is True
finally:
mon.stop()
if mon._thread:
mon._thread.join(timeout=2.0)
def test_probe_loop_no_user_permit_during_probe(
self, mock_client: MagicMock, mock_metrics: MagicMock
) -> None:
"""While background probe is in HALF_OPEN, user requests are blocked."""
mon = BackendHealthMonitor(
client=mock_client,
probe_interval=0.05,
probe_timeout=1.0,
failure_threshold=1,
cooldown=0.1,
)
mon.record_failure()
assert mon.circuit_state == CircuitState.OPEN
# Force into HALF_OPEN as the probe loop would
with mon._lock:
mon._state = CircuitState.HALF_OPEN
mon._half_open_permit = False # probe consumes it
# User requests should be blocked — only the probe gets through
assert mon.acquire_request_permit() is False
def test_stop_thread(self, mock_client: MagicMock) -> None:
"""stop() signals the probe loop to exit."""
mon = _make_monitor(mock_client)
mon.start()
assert mon._thread is not None
assert mon._thread.is_alive()
mon.stop()
mon._thread.join(timeout=3.0)
assert not mon._thread.is_alive()
def test_backend_key_static(self) -> None:
"""backend_key is a static method returning normalized tuple."""
key = HealthTrackerRegistry.backend_key("anthropic", "https://api.anthropic.com/")
assert key == ("anthropic", "https://api.anthropic.com")
+52
View File
@@ -37,3 +37,55 @@ class TestStripHtml:
def test_self_closing_tags(self):
result = strip_html("hello<br/>world")
assert result == "helloworld"
# -- invisible element stripping -----------------------------------------
def test_strips_script_content(self):
html = "<p>before</p><script>var x = 1;</script><p>after</p>"
result = strip_html(html)
assert "var x" not in result
assert "before" in result
assert "after" in result
def test_strips_style_content(self):
html = "<style>.foo { color: red; }</style><p>visible</p>"
result = strip_html(html)
assert "color" not in result
assert "visible" in result
def test_strips_template_content(self):
html = "<template><div>hidden</div></template><p>shown</p>"
result = strip_html(html)
assert "hidden" not in result
assert "shown" in result
def test_strips_noscript_content(self):
html = "<noscript>Enable JS</noscript><p>content</p>"
result = strip_html(html)
assert "Enable JS" not in result
assert "content" in result
def test_strips_multiple_script_blocks(self):
html = "<script>a()</script><p>middle</p><script>b()</script>"
result = strip_html(html)
assert "a()" not in result
assert "b()" not in result
assert "middle" in result
def test_strips_multiline_script(self):
html = "<script>\nfunction foo() {\n return 1;\n}\n</script><p>ok</p>"
result = strip_html(html)
assert "function" not in result
assert "ok" in result
def test_strips_script_case_insensitive(self):
html = "<SCRIPT>code()</SCRIPT><p>text</p>"
result = strip_html(html)
assert "code()" not in result
assert "text" in result
def test_strips_script_with_attributes(self):
html = '<script type="text/javascript" src="app.js">init();</script><p>done</p>'
result = strip_html(html)
assert "init()" not in result
assert "done" in result
+159 -12
View File
@@ -24,6 +24,7 @@ def _make_mock_provider(
) -> MagicMock:
"""Create a mock LLM provider that returns a fixed response."""
provider = MagicMock()
provider.provider_name = "openai"
caps = MagicMock()
caps.context_window = 100_000
caps.max_output_tokens = 4096
@@ -63,6 +64,8 @@ def _make_judge(
timeout=timeout,
)
client = MagicMock()
client.base_url = "https://api.openai.com/v1"
client.api_key = "test-key"
return IntentJudge(
config=config,
session_provider=provider,
@@ -186,11 +189,16 @@ class TestErrorHandling:
[{"role": "user", "content": "test"}],
cancel_event=None,
executor=pool,
client=MagicMock(),
)
assert result is None
def test_provider_error_heuristic_still_returned(self):
"""When LLM fails, heuristic verdicts are still returned from evaluate()."""
"""When LLM fails, heuristic verdicts are still returned from evaluate().
With fallback delivery, the callback *will* fire with a fallback
verdict, but heuristic verdicts are always returned synchronously.
"""
provider = _make_mock_provider(side_effect=RuntimeError("API down"))
judge = _make_judge(provider)
@@ -204,8 +212,9 @@ class TestErrorHandling:
assert len(heuristics) == 1
assert heuristics[0].tier == "heuristic"
# Callback should not have been invoked (LLM failed)
assert len(callback_results) == 0
# Fallback verdict delivered via callback
assert len(callback_results) == 1
assert callback_results[0].tier == "llm_fallback"
def test_empty_content_returns_none(self):
"""Provider returns empty content, no tool calls."""
@@ -221,9 +230,31 @@ class TestErrorHandling:
[{"role": "user", "content": "test"}],
cancel_event=None,
executor=pool,
client=MagicMock(),
)
assert result is None
def test_empty_content_length_stop_no_retry(self):
"""When finish_reason is 'length', don't retry — return None immediately."""
provider = _make_mock_provider(response_content="")
result_mock = provider.create_completion.return_value
result_mock.tool_calls = None
result_mock.content = ""
result_mock.finish_reason = "length"
judge = _make_judge(provider)
with ThreadPoolExecutor(max_workers=1) as pool:
result = judge._evaluate_single(
_make_item(),
[{"role": "user", "content": "test"}],
cancel_event=None,
executor=pool,
client=MagicMock(),
)
assert result is None
# Should have been called exactly once — no retries
assert provider.create_completion.call_count == 1
# ---------------------------------------------------------------------------
# Multi-turn tool use
@@ -234,6 +265,7 @@ class TestMultiTurnToolUse:
def test_tool_call_then_verdict(self):
"""Provider requests read_file, then returns verdict."""
provider = MagicMock()
provider.provider_name = "openai"
caps = MagicMock()
caps.context_window = 100_000
caps.max_output_tokens = 4096
@@ -267,6 +299,7 @@ class TestMultiTurnToolUse:
[{"role": "user", "content": "test"}],
cancel_event=None,
executor=pool,
client=MagicMock(),
)
assert verdict is not None
assert verdict.tier == "llm"
@@ -275,6 +308,7 @@ class TestMultiTurnToolUse:
def test_max_turns_reached(self):
"""Provider keeps requesting tools — stops at _JUDGE_MAX_TURNS."""
provider = MagicMock()
provider.provider_name = "openai"
caps = MagicMock()
caps.context_window = 100_000
caps.max_output_tokens = 4096
@@ -315,6 +349,7 @@ class TestMultiTurnToolUse:
[{"role": "user", "content": "test"}],
cancel_event=None,
executor=pool,
client=MagicMock(),
)
# Should have called create_completion exactly _JUDGE_MAX_TURNS times
assert provider.create_completion.call_count == 5
@@ -335,12 +370,12 @@ class TestContextPreparation:
result = judge._prepare_context(_make_item(), messages)
# Should have system message + some truncated history + user message
# Should have system message + single user message with transcript
assert len(result) == 2
assert result[0]["role"] == "system"
assert result[-1]["role"] == "user"
assert "pending human approval" in result[-1]["content"]
# Should be fewer messages than the original 100
assert len(result) < 102 # system + 100 + user
assert result[1]["role"] == "user"
assert "pending human approval" in result[1]["content"]
assert "Conversation context:" in result[1]["content"]
# ---------------------------------------------------------------------------
@@ -369,8 +404,8 @@ class TestConfidenceArbitration:
assert callback_results[0].tier == "llm"
assert callback_results[0].confidence == 0.95
def test_llm_lower_confidence_no_callback(self):
"""LLM confidence < heuristic confidence — no callback."""
def test_llm_lower_confidence_no_arbitration_block(self):
"""LLM confidence < heuristic — callback still invoked (all verdicts delivered)."""
provider = _make_mock_provider(response_content=_good_verdict_json(confidence=0.5))
judge = _make_judge(provider)
@@ -384,8 +419,10 @@ class TestConfidenceArbitration:
time.sleep(0.5)
assert len(heuristics) == 1
# LLM confidence (0.5) < heuristic (0.85), so no callback
assert len(callback_results) == 0
# LLM verdict is always delivered regardless of confidence comparison
assert len(callback_results) == 1
assert callback_results[0].tier == "llm"
assert callback_results[0].confidence == 0.5
# ---------------------------------------------------------------------------
@@ -666,3 +703,113 @@ class TestHeuristicNewLowRules:
def test_web_search(self):
v = evaluate_heuristic("web_search", {"query": "python"}, "web_search")
assert v.risk_level == "low"
# ---------------------------------------------------------------------------
# Alias resolution — regression guard for the "did not return a verdict"
# silent no-op surfaced during coordinator harness testing.
# ---------------------------------------------------------------------------
class TestModelAliasResolution:
"""When ``judge.model`` points at a registry alias whose underlying
provider differs from the session's, the judge MUST resolve through
the registry not fall back to the session provider with the
underlying model id. Pre-resolving the alias to the model id in the
session_factory stranded the alias and made every coordinator tool
verdict come back ``llm_fallback / "did not return a verdict"``.
"""
def _make_alias_registry(
self,
alias: str,
alias_provider: MagicMock,
alias_client: MagicMock,
underlying_model: str,
) -> MagicMock:
registry = MagicMock()
cfg = MagicMock()
cfg.context_window = 50_000
registry.has_alias.side_effect = lambda a: a == alias
registry.resolve.return_value = (alias_client, underlying_model, cfg)
registry.get_provider.return_value = alias_provider
return registry
def test_alias_uses_registry_provider_not_session_provider(self):
"""Judge with model=alias should resolve via registry — provider, client,
and concrete model name all come from the alias."""
# Session provider/client — would be used if resolution falls back.
session_provider = _make_mock_provider(
response_content=_good_verdict_json(intent_summary="from-session"),
)
session_provider.provider_name = "anthropic"
session_client = MagicMock()
session_client.base_url = "https://session.example/v1"
session_client.api_key = "session-key"
# Alias provider/client — what the judge SHOULD use.
alias_provider = _make_mock_provider(
response_content=_good_verdict_json(intent_summary="from-alias"),
)
alias_provider.provider_name = "openai"
alias_client = MagicMock()
alias_client.base_url = "https://alias.example/v1"
alias_client.api_key = "alias-key"
registry = self._make_alias_registry(
"judge-mini", alias_provider, alias_client, "gpt-5-mini-resolved"
)
config = JudgeConfig(enabled=True, model="judge-mini")
judge = IntentJudge(
config=config,
session_provider=session_provider,
session_client=session_client,
session_model="session-default-model",
context_window=100_000,
model_registry=registry,
)
assert judge._provider is alias_provider
assert judge._model == "gpt-5-mini-resolved"
# Client factory args reflect the alias's client, not the session's.
assert judge._client_factory_args["base_url"] == "https://alias.example/v1"
assert judge._client_factory_args["api_key"] == "alias-key"
assert judge._client_factory_args["provider_name"] == "openai"
def test_coordinator_tool_call_returns_llm_verdict_not_fallback(self):
"""Happy-path regression for coordinator tool calls: with a properly
resolved provider, the verdict tier must be ``llm`` the
``llm_fallback`` failure mode flagged in the harness was uniform
across every coordinator tool, so guard the happy path explicitly.
"""
provider = _make_mock_provider(
response_content=_good_verdict_json(
intent_summary="Spawn a child workstream",
risk_level="medium",
recommendation="approve",
),
)
judge = _make_judge(provider)
callback_results: list[IntentVerdict] = []
coord_item = _make_item(
func_name="spawn_workstream",
func_args={"initial_message": "do the thing", "skill": "engineer"},
approval_label="spawn_workstream",
)
judge.evaluate(
[coord_item],
[{"role": "user", "content": "delegate the audit"}],
callback_results.append,
)
# Wait for daemon thread.
for _ in range(20):
if callback_results:
break
time.sleep(0.1)
assert callback_results, "judge never delivered a verdict"
assert callback_results[0].tier == "llm"
assert callback_results[0].tier != "llm_fallback"
assert "did not return a verdict" not in callback_results[0].reasoning
+63
View File
@@ -453,3 +453,66 @@ class TestEdgeCases:
def test_cargo_install(self):
v = evaluate_heuristic("bash", {"command": "cargo install ripgrep"}, "bash")
_assert_verdict(v, risk_level="medium", recommendation="review")
# ---------------------------------------------------------------------------
# Custom rules parameter
# ---------------------------------------------------------------------------
class TestCustomRulesParam:
"""Tests for evaluate_heuristic() with custom rules kwarg."""
def test_custom_rules_override_builtins(self):
"""Custom rules list is used instead of built-in rules."""
from turnstone.core.judge import _HeuristicRule, evaluate_heuristic
custom = [
_HeuristicRule(
name="custom-test",
risk_level="high",
confidence=0.95,
recommendation="deny",
tool_pattern="bash",
arg_patterns=[r"custom_dangerous_cmd"],
intent_template="Custom danger: {arg_snippet}",
reasoning_template="Custom rule matched.",
),
]
# Should match custom rule
verdict = evaluate_heuristic(
"bash",
{"command": "custom_dangerous_cmd --flag"},
"bash",
rules=custom,
)
assert verdict.risk_level == "high"
assert verdict.recommendation == "deny"
assert "custom-test" in verdict.evidence[0]
def test_custom_rules_no_match_default(self):
"""When custom rules don't match, default medium/review verdict returned."""
from turnstone.core.judge import evaluate_heuristic
verdict = evaluate_heuristic(
"bash",
{"command": "ls"},
"bash",
rules=[],
)
assert verdict.risk_level == "medium"
assert verdict.recommendation == "review"
assert verdict.confidence == 0.5
def test_none_rules_uses_builtins(self):
"""When rules=None, built-in rules are used (backward compat)."""
from turnstone.core.judge import evaluate_heuristic
verdict = evaluate_heuristic(
"bash",
{"command": "rm -rf /etc"},
"bash",
rules=None,
)
assert verdict.risk_level == "critical"
assert "rm-root" in verdict.evidence[0]
+429
View File
@@ -0,0 +1,429 @@
"""Tests for heuristic_rules and output_guard_patterns storage CRUD operations."""
from __future__ import annotations
import uuid
from typing import TYPE_CHECKING
if TYPE_CHECKING:
from turnstone.core.storage._sqlite import SQLiteBackend
def _make_id() -> str:
return uuid.uuid4().hex
class TestHeuristicRuleStorage:
def test_create_and_get_heuristic_rule(self, db: SQLiteBackend) -> None:
rid = _make_id()
db.create_heuristic_rule(
rule_id=rid,
name="dangerous-exec",
risk_level="critical",
confidence=0.95,
recommendation="deny",
tool_pattern="execute_code",
arg_patterns='[".*exec.*", ".*eval.*"]',
intent_template="User wants to run code",
reasoning_template="Executing arbitrary code is dangerous",
tier="critical",
priority=100,
builtin=True,
enabled=True,
created_by="admin",
)
r = db.get_heuristic_rule(rid)
assert r is not None
assert r["rule_id"] == rid
assert r["name"] == "dangerous-exec"
assert r["risk_level"] == "critical"
assert r["confidence"] == 0.95
assert r["recommendation"] == "deny"
assert r["tool_pattern"] == "execute_code"
assert r["arg_patterns"] == '[".*exec.*", ".*eval.*"]'
assert r["intent_template"] == "User wants to run code"
assert r["reasoning_template"] == "Executing arbitrary code is dangerous"
assert r["tier"] == "critical"
assert r["priority"] == 100
assert r["builtin"] is True
assert r["enabled"] is True
assert r["created_by"] == "admin"
def test_get_heuristic_rule_by_name(self, db: SQLiteBackend) -> None:
rid = _make_id()
db.create_heuristic_rule(
rule_id=rid,
name="by-name-lookup",
risk_level="high",
confidence=0.8,
recommendation="review",
tool_pattern="file_write",
)
r = db.get_heuristic_rule_by_name("by-name-lookup")
assert r is not None
assert r["rule_id"] == rid
assert r["name"] == "by-name-lookup"
def test_get_heuristic_rule_by_name_not_found(self, db: SQLiteBackend) -> None:
assert db.get_heuristic_rule_by_name("nonexistent") is None
def test_list_heuristic_rules(self, db: SQLiteBackend) -> None:
db.create_heuristic_rule(
rule_id=_make_id(),
name="low-tier-rule",
risk_level="low",
confidence=0.5,
recommendation="approve",
tool_pattern="read_file",
tier="low",
priority=10,
)
db.create_heuristic_rule(
rule_id=_make_id(),
name="critical-tier-rule",
risk_level="critical",
confidence=0.99,
recommendation="deny",
tool_pattern="delete_all",
tier="critical",
priority=50,
)
db.create_heuristic_rule(
rule_id=_make_id(),
name="medium-tier-rule",
risk_level="medium",
confidence=0.7,
recommendation="review",
tool_pattern="web_search",
tier="medium",
priority=20,
)
rules = db.list_heuristic_rules()
assert len(rules) == 3
# Ordered by tier (critical=0, medium=2, low=3) then priority desc
assert rules[0]["name"] == "critical-tier-rule"
assert rules[1]["name"] == "medium-tier-rule"
assert rules[2]["name"] == "low-tier-rule"
def test_list_heuristic_rules_enabled_only(self, db: SQLiteBackend) -> None:
db.create_heuristic_rule(
rule_id=_make_id(),
name="enabled-rule",
risk_level="medium",
confidence=0.7,
recommendation="approve",
tool_pattern="tool_a",
enabled=True,
)
db.create_heuristic_rule(
rule_id=_make_id(),
name="disabled-rule",
risk_level="low",
confidence=0.3,
recommendation="deny",
tool_pattern="tool_b",
enabled=False,
)
enabled = db.list_heuristic_rules(enabled_only=True)
assert len(enabled) == 1
assert enabled[0]["name"] == "enabled-rule"
assert enabled[0]["enabled"] is True
def test_update_heuristic_rule(self, db: SQLiteBackend) -> None:
rid = _make_id()
db.create_heuristic_rule(
rule_id=rid,
name="orig-name",
risk_level="low",
confidence=0.5,
recommendation="review",
tool_pattern="orig_tool",
)
ok = db.update_heuristic_rule(
rid,
name="updated-name",
risk_level="high",
confidence=0.9,
recommendation="deny",
enabled=False,
builtin=True,
)
assert ok is True
r = db.get_heuristic_rule(rid)
assert r is not None
assert r["name"] == "updated-name"
assert r["risk_level"] == "high"
assert r["confidence"] == 0.9
assert r["recommendation"] == "deny"
assert r["enabled"] is False
assert r["builtin"] is True
def test_update_heuristic_rule_not_found(self, db: SQLiteBackend) -> None:
ok = db.update_heuristic_rule("nonexistent", name="x")
assert ok is False
def test_delete_heuristic_rule(self, db: SQLiteBackend) -> None:
rid = _make_id()
db.create_heuristic_rule(
rule_id=rid,
name="delete-me",
risk_level="low",
confidence=0.3,
recommendation="review",
tool_pattern="temp_tool",
)
ok = db.delete_heuristic_rule(rid)
assert ok is True
assert db.get_heuristic_rule(rid) is None
def test_delete_heuristic_rule_not_found(self, db: SQLiteBackend) -> None:
ok = db.delete_heuristic_rule("nonexistent")
assert ok is False
def test_create_duplicate_id_noop(self, db: SQLiteBackend) -> None:
rid = _make_id()
db.create_heuristic_rule(
rule_id=rid,
name="first-insert",
risk_level="high",
confidence=0.8,
recommendation="approve",
tool_pattern="tool_orig",
)
# Second insert with same ID should be no-op (OR IGNORE)
db.create_heuristic_rule(
rule_id=rid,
name="second-insert",
risk_level="low",
confidence=0.1,
recommendation="deny",
tool_pattern="tool_new",
)
r = db.get_heuristic_rule(rid)
assert r is not None
assert r["name"] == "first-insert" # original preserved
assert r["risk_level"] == "high"
def test_defaults(self, db: SQLiteBackend) -> None:
"""Verify default values for optional fields."""
rid = _make_id()
db.create_heuristic_rule(
rule_id=rid,
name="defaults-test",
risk_level="medium",
confidence=0.5,
recommendation="review",
tool_pattern="some_tool",
)
r = db.get_heuristic_rule(rid)
assert r is not None
assert r["arg_patterns"] == "[]"
assert r["intent_template"] == ""
assert r["reasoning_template"] == ""
assert r["tier"] == "medium"
assert r["priority"] == 0
assert r["builtin"] is False
assert r["enabled"] is True
assert r["created_by"] == ""
class TestOutputGuardPatternStorage:
def test_create_and_get_output_guard_pattern(self, db: SQLiteBackend) -> None:
pid = _make_id()
db.create_output_guard_pattern(
pattern_id=pid,
name="aws-key-pattern",
category="credentials",
risk_level="high",
pattern=r"AKIA[0-9A-Z]{16}",
flag_name="aws_access_key",
annotation="AWS access key detected",
pattern_flags="IGNORECASE",
is_credential=True,
redact_label="[AWS_KEY]",
priority=100,
builtin=True,
enabled=True,
created_by="system",
)
p = db.get_output_guard_pattern(pid)
assert p is not None
assert p["pattern_id"] == pid
assert p["name"] == "aws-key-pattern"
assert p["category"] == "credentials"
assert p["risk_level"] == "high"
assert p["pattern"] == r"AKIA[0-9A-Z]{16}"
assert p["flag_name"] == "aws_access_key"
assert p["annotation"] == "AWS access key detected"
assert p["pattern_flags"] == "IGNORECASE"
assert p["is_credential"] is True
assert p["redact_label"] == "[AWS_KEY]"
assert p["priority"] == 100
assert p["builtin"] is True
assert p["enabled"] is True
assert p["created_by"] == "system"
def test_get_output_guard_pattern_by_name(self, db: SQLiteBackend) -> None:
pid = _make_id()
db.create_output_guard_pattern(
pattern_id=pid,
name="lookup-by-name",
category="credentials",
risk_level="high",
pattern=r"ghp_[A-Za-z0-9_]{36}",
flag_name="github_pat",
annotation="GitHub PAT detected",
)
p = db.get_output_guard_pattern_by_name("lookup-by-name")
assert p is not None
assert p["pattern_id"] == pid
assert p["name"] == "lookup-by-name"
def test_get_output_guard_pattern_by_name_not_found(self, db: SQLiteBackend) -> None:
assert db.get_output_guard_pattern_by_name("nonexistent") is None
def test_list_output_guard_patterns(self, db: SQLiteBackend) -> None:
db.create_output_guard_pattern(
pattern_id=_make_id(),
name="secrets-high",
category="credentials",
risk_level="high",
pattern=r"secret_.*",
flag_name="generic_secret",
annotation="Secret detected",
priority=50,
)
db.create_output_guard_pattern(
pattern_id=_make_id(),
name="credentials-high",
category="credentials",
risk_level="high",
pattern=r"password=.*",
flag_name="password",
annotation="Password detected",
priority=100,
)
db.create_output_guard_pattern(
pattern_id=_make_id(),
name="credentials-low",
category="credentials",
risk_level="low",
pattern=r"token=test",
flag_name="test_token",
annotation="Test token",
priority=10,
)
patterns = db.list_output_guard_patterns()
assert len(patterns) == 3
# Ordered by category then priority desc
assert patterns[0]["name"] == "credentials-high"
assert patterns[1]["name"] == "secrets-high"
assert patterns[2]["name"] == "credentials-low"
def test_list_output_guard_patterns_enabled_only(self, db: SQLiteBackend) -> None:
db.create_output_guard_pattern(
pattern_id=_make_id(),
name="active-pattern",
category="credentials",
risk_level="high",
pattern=r"AKIA.*",
flag_name="aws_key",
annotation="AWS key",
enabled=True,
)
db.create_output_guard_pattern(
pattern_id=_make_id(),
name="inactive-pattern",
category="credentials",
risk_level="low",
pattern=r"test_.*",
flag_name="test",
annotation="Test pattern",
enabled=False,
)
enabled = db.list_output_guard_patterns(enabled_only=True)
assert len(enabled) == 1
assert enabled[0]["name"] == "active-pattern"
assert enabled[0]["enabled"] is True
def test_update_output_guard_pattern(self, db: SQLiteBackend) -> None:
pid = _make_id()
db.create_output_guard_pattern(
pattern_id=pid,
name="orig-pattern",
category="credentials",
risk_level="medium",
pattern=r"old_pattern",
flag_name="old_flag",
annotation="Old annotation",
is_credential=False,
)
ok = db.update_output_guard_pattern(
pid,
name="updated-pattern",
category="credentials",
risk_level="high",
pattern=r"new_pattern",
flag_name="new_flag",
annotation="Updated annotation",
is_credential=True,
enabled=False,
builtin=True,
)
assert ok is True
p = db.get_output_guard_pattern(pid)
assert p is not None
assert p["name"] == "updated-pattern"
assert p["category"] == "credentials"
assert p["risk_level"] == "high"
assert p["pattern"] == r"new_pattern"
assert p["flag_name"] == "new_flag"
assert p["annotation"] == "Updated annotation"
assert p["is_credential"] is True
assert p["enabled"] is False
assert p["builtin"] is True
def test_update_output_guard_pattern_not_found(self, db: SQLiteBackend) -> None:
ok = db.update_output_guard_pattern("nonexistent", name="x")
assert ok is False
def test_delete_output_guard_pattern(self, db: SQLiteBackend) -> None:
pid = _make_id()
db.create_output_guard_pattern(
pattern_id=pid,
name="delete-me",
category="credentials",
risk_level="low",
pattern=r"temp",
flag_name="temp_flag",
annotation="Temporary",
)
ok = db.delete_output_guard_pattern(pid)
assert ok is True
assert db.get_output_guard_pattern(pid) is None
def test_delete_output_guard_pattern_not_found(self, db: SQLiteBackend) -> None:
ok = db.delete_output_guard_pattern("nonexistent")
assert ok is False
def test_defaults(self, db: SQLiteBackend) -> None:
"""Verify default values for optional fields."""
pid = _make_id()
db.create_output_guard_pattern(
pattern_id=pid,
name="defaults-test",
category="credentials",
risk_level="medium",
pattern=r"some_pattern",
flag_name="some_flag",
annotation="Some annotation",
)
p = db.get_output_guard_pattern(pid)
assert p is not None
assert p["pattern_flags"] == ""
assert p["is_credential"] is False
assert p["redact_label"] == ""
assert p["priority"] == 0
assert p["builtin"] is False
assert p["enabled"] is True
assert p["created_by"] == ""
+13 -12
View File
@@ -158,7 +158,7 @@ class TestExecLoadSkill:
"name": "code-review",
"description": "Reviews code for quality",
"content": "# Code Review\nReview all code.",
"scan_status": "safe",
"risk_level": "safe",
"category": "engineering",
}
]
@@ -185,7 +185,7 @@ class TestExecLoadSkill:
assert session._set_skill_called == []
def test_load_calls_ui_on_tool_result(self) -> None:
skills = [{"name": "test", "content": "content", "description": "", "scan_status": ""}]
skills = [{"name": "test", "content": "content", "description": "", "risk_level": ""}]
session, _, fake_get = _make_session(skills)
with patch("turnstone.core.session.get_skill_by_name", side_effect=fake_get):
@@ -200,7 +200,7 @@ class TestExecLoadSkill:
"name": "code-review",
"description": "Reviews code",
"category": "eng",
"scan_status": "safe",
"risk_level": "safe",
"tags": "[]",
"activation": "named",
},
@@ -208,7 +208,7 @@ class TestExecLoadSkill:
"name": "docs-writer",
"description": "Writes docs",
"category": "general",
"scan_status": "low",
"risk_level": "low",
"tags": "[]",
"activation": "named",
},
@@ -232,7 +232,7 @@ class TestExecLoadSkill:
"name": f"skill-{i}",
"description": f"Desc {i}",
"category": "general",
"scan_status": "",
"risk_level": "",
"tags": "[]",
"activation": "named",
}
@@ -262,13 +262,13 @@ class TestExecLoadSkill:
assert "no skills found" in result.lower()
def test_search_includes_scan_status(self) -> None:
def test_search_includes_risk_level(self) -> None:
skills = [
{
"name": "risky",
"description": "Risky skill",
"category": "ops",
"scan_status": "high",
"risk_level": "high",
"tags": "[]",
"activation": "named",
},
@@ -301,7 +301,7 @@ class TestExecLoadSkill:
"name": "disabled-skill",
"content": "x",
"description": "",
"scan_status": "",
"risk_level": "",
"enabled": False,
}
]
@@ -315,7 +315,7 @@ class TestExecLoadSkill:
assert session._set_skill_called == []
def test_load_already_active_skill(self) -> None:
skills = [{"name": "active", "content": "x", "description": "", "scan_status": "safe"}]
skills = [{"name": "active", "content": "x", "description": "", "risk_level": "safe"}]
session, _, fake_get = _make_session(skills)
session._skill_name = "active"
@@ -332,7 +332,7 @@ class TestExecLoadSkill:
"name": "enabled-skill",
"description": "Good",
"category": "gen",
"scan_status": "",
"risk_level": "",
"tags": "[]",
"activation": "named",
"enabled": True,
@@ -341,7 +341,7 @@ class TestExecLoadSkill:
"name": "disabled-skill",
"description": "Bad",
"category": "gen",
"scan_status": "",
"risk_level": "",
"tags": "[]",
"activation": "named",
"enabled": False,
@@ -365,7 +365,7 @@ class TestExecLoadSkill:
"name": "code-review",
"description": "Reviews code for quality",
"category": "eng",
"scan_status": "",
"risk_level": "",
"tags": "[]",
"activation": "named",
},
@@ -430,6 +430,7 @@ class TestSkillCatalogDisclosure:
session._tools = []
session._client_type = ClientType.CLI
session._username = ""
session._kind = "interactive"
# Memory stubs
session._memory_config = MagicMock()
+413 -6
View File
@@ -3,7 +3,9 @@
from __future__ import annotations
import asyncio
import concurrent.futures
import json
import time
from contextlib import AsyncExitStack
from typing import Any
from unittest.mock import AsyncMock, MagicMock, patch
@@ -15,7 +17,7 @@ from turnstone.core.mcp_client import (
_mcp_to_openai,
load_mcp_config,
)
from turnstone.core.tools import TOOLS, merge_mcp_tools
from turnstone.core.tools import INTERACTIVE_TOOLS, TOOLS, merge_mcp_tools
# ---------------------------------------------------------------------------
# Helpers
@@ -141,7 +143,7 @@ class TestMcpToOpenai:
assert result["type"] == "function"
func = result["function"]
assert func["name"] == "mcp__github__search_repos"
assert "[MCP: github]" in func["description"]
assert func["description"] == "Search GitHub repos"
assert func["parameters"]["type"] == "object"
assert "query" in func["parameters"]["properties"]
@@ -164,7 +166,7 @@ class TestMcpToOpenai:
tool.description = ""
tool.inputSchema = {"type": "object", "properties": {}}
result = _mcp_to_openai("test", tool)
assert result["function"]["description"] == "[MCP: test] "
assert result["function"]["description"] == ""
# ---------------------------------------------------------------------------
@@ -304,7 +306,7 @@ class TestMCPClientManager:
def test_call_tool_sync_disconnected_server(self):
mgr = MCPClientManager({})
mgr._tool_map["mcp__dead__ping"] = ("dead", "ping")
# No session registered for "dead"
# No session registered for "dead", no config/loop → reconnect fails
with pytest.raises(RuntimeError, match="not connected"):
mgr.call_tool_sync("mcp__dead__ping", {})
@@ -347,14 +349,15 @@ class TestSessionIntegration:
def test_session_without_mcp(self, tmp_db):
session = self._make_session(mcp_client=None)
assert session._tools is TOOLS
# Interactive session surface — coordinator tools excluded.
assert session._tools is INTERACTIVE_TOOLS
assert session._mcp_client is None
def test_session_with_mcp(self, tmp_db):
mock_mcp = MagicMock()
mock_mcp.get_tools.return_value = [_fake_openai_tool()]
session = self._make_session(mcp_client=mock_mcp)
assert len(session._tools) == len(TOOLS) + 1
assert len(session._tools) == len(INTERACTIVE_TOOLS) + 1
assert session._tools[-1]["function"]["name"] == "mcp__test__search"
def test_task_tools_include_mcp(self, tmp_db):
@@ -1553,3 +1556,407 @@ class TestSafeCloseStack:
await MCPClientManager._safe_close_stack(stack)
asyncio.run(_run())
# ---------------------------------------------------------------------------
# Fix 1: Cancel orphaned futures on timeout
# ---------------------------------------------------------------------------
class TestFutureCancellation:
"""Verify future.cancel() is called when sync bridge methods time out."""
def _make_manager_with_session(self) -> MCPClientManager:
mgr = MCPClientManager({"test": {"type": "stdio", "command": "echo"}})
mock_session = MagicMock()
# Prevent auto-spec from creating async coroutines that trigger warnings
mock_session.call_tool = MagicMock(return_value="sentinel")
mock_session.read_resource = MagicMock(return_value="sentinel")
mock_session.get_prompt = MagicMock(return_value="sentinel")
mgr._sessions["test"] = mock_session
mgr._loop = MagicMock()
mgr._tool_map["mcp__test__search"] = ("test", "search")
mgr._resource_map["file:///a.txt"] = ("test", "file:///a.txt")
mgr._prompt_map["mcp__test__review"] = ("test", "review")
return mgr
def test_call_tool_sync_cancels_future_on_timeout(self):
mgr = self._make_manager_with_session()
mock_future = MagicMock()
mock_future.result.side_effect = concurrent.futures.TimeoutError()
with (
patch("asyncio.run_coroutine_threadsafe", return_value=mock_future),
pytest.raises(TimeoutError, match="timed out"),
):
mgr.call_tool_sync("mcp__test__search", {"query": "x"}, timeout=1)
mock_future.cancel.assert_called_once()
def test_read_resource_sync_cancels_future_on_timeout(self):
mgr = self._make_manager_with_session()
mock_future = MagicMock()
mock_future.result.side_effect = concurrent.futures.TimeoutError()
with (
patch("asyncio.run_coroutine_threadsafe", return_value=mock_future),
pytest.raises(TimeoutError, match="timed out"),
):
mgr.read_resource_sync("file:///a.txt", timeout=1)
mock_future.cancel.assert_called_once()
def test_get_prompt_sync_cancels_future_on_timeout(self):
mgr = self._make_manager_with_session()
mock_future = MagicMock()
mock_future.result.side_effect = concurrent.futures.TimeoutError()
with (
patch("asyncio.run_coroutine_threadsafe", return_value=mock_future),
pytest.raises(TimeoutError, match="timed out"),
):
mgr.get_prompt_sync("mcp__test__review", timeout=1)
mock_future.cancel.assert_called_once()
def test_refresh_sync_cancels_future_on_timeout(self):
mgr = MCPClientManager({})
mgr._loop = MagicMock()
mock_future = MagicMock()
mock_future.result.side_effect = concurrent.futures.TimeoutError()
with (
patch.object(mgr, "_refresh_all", return_value=MagicMock()),
patch("asyncio.run_coroutine_threadsafe", return_value=mock_future),
pytest.raises(TimeoutError, match="timed out"),
):
mgr.refresh_sync(timeout=1)
mock_future.cancel.assert_called_once()
# ---------------------------------------------------------------------------
# Fix 2: Per-server circuit breaker
# ---------------------------------------------------------------------------
class TestCircuitBreaker:
"""Verify per-server circuit breaker behavior."""
def test_circuit_stays_closed_below_threshold(self):
mgr = MCPClientManager({})
mgr._cb_record_failure("srv")
mgr._cb_record_failure("srv")
is_open, _ = mgr._cb_check("srv")
assert not is_open
def test_circuit_opens_at_threshold(self):
mgr = MCPClientManager({})
for _ in range(3):
mgr._cb_record_failure("srv")
is_open, cooldown_expired = mgr._cb_check("srv")
assert is_open
assert not cooldown_expired # just opened, cooldown not expired
def test_circuit_half_open_after_cooldown(self):
mgr = MCPClientManager({})
for _ in range(3):
mgr._cb_record_failure("srv")
# Simulate cooldown expiry
mgr._circuit_open_until["srv"] = time.monotonic() - 1
is_open, cooldown_expired = mgr._cb_check("srv")
assert is_open
assert cooldown_expired
def test_circuit_resets_on_success(self):
mgr = MCPClientManager({})
for _ in range(3):
mgr._cb_record_failure("srv")
assert "srv" in mgr._circuit_open_until
mgr._cb_record_success("srv")
is_open, _ = mgr._cb_check("srv")
assert not is_open
assert mgr._consecutive_failures.get("srv") is None
def test_success_decays_trip_count(self):
"""Success decays trip_count by 1 so flapping servers escalate backoff."""
mgr = MCPClientManager({})
mgr._circuit_trip_count["srv"] = 3
mgr._cb_record_success("srv")
assert mgr._circuit_trip_count["srv"] == 2
mgr._cb_record_success("srv")
assert mgr._circuit_trip_count["srv"] == 1
mgr._cb_record_success("srv")
assert "srv" not in mgr._circuit_trip_count
def test_cooldown_is_exponential(self):
mgr = MCPClientManager({})
# First trip (trip_count starts at 0)
for _ in range(3):
mgr._cb_record_failure("srv")
deadline1 = mgr._circuit_open_until["srv"]
base1 = deadline1 - time.monotonic()
# Reset circuit but keep trip_count at 1 (set by first trip)
mgr._cb_record_success("srv")
# trip_count decayed from 1 to 0 — manually set to 1 for test
mgr._circuit_trip_count["srv"] = 1
for _ in range(3):
mgr._cb_record_failure("srv")
deadline2 = mgr._circuit_open_until["srv"]
base2 = deadline2 - time.monotonic()
# Second trip should have longer cooldown (roughly 2x, within jitter)
assert base2 > base1 * 1.5
def test_cooldown_capped_at_max(self):
mgr = MCPClientManager({})
mgr._circuit_trip_count["srv"] = 100 # very high trip count
for _ in range(3):
mgr._cb_record_failure("srv")
deadline = mgr._circuit_open_until["srv"]
cooldown = deadline - time.monotonic()
# Should not exceed max (300s) + 10% jitter = 330s
assert cooldown <= mgr._CB_MAX_COOLDOWN * 1.11
def test_cb_gate_rejects_when_open(self):
mgr = MCPClientManager({})
for _ in range(3):
mgr._cb_record_failure("srv")
with pytest.raises(RuntimeError, match="circuit open"):
mgr._cb_gate("srv")
def test_cb_gate_allows_after_cooldown(self):
mgr = MCPClientManager({})
for _ in range(3):
mgr._cb_record_failure("srv")
mgr._circuit_open_until["srv"] = time.monotonic() - 1
# Should not raise
mgr._cb_gate("srv")
# Deadline should be removed (half-open probe allowed)
assert "srv" not in mgr._circuit_open_until
def test_cb_clear_removes_all_state(self):
mgr = MCPClientManager({})
for _ in range(3):
mgr._cb_record_failure("srv")
mgr._cb_clear("srv")
assert "srv" not in mgr._consecutive_failures
assert "srv" not in mgr._circuit_open_until
assert "srv" not in mgr._circuit_trip_count
@pytest.mark.filterwarnings("ignore::pytest.PytestUnraisableExceptionWarning")
@pytest.mark.filterwarnings("ignore:coroutine.*was never awaited:RuntimeWarning")
def test_call_tool_sync_records_failure_on_timeout(self):
mgr = MCPClientManager({"test": {"type": "stdio", "command": "echo"}})
mock_session = MagicMock()
mock_session.call_tool = MagicMock(return_value="sentinel")
mgr._sessions["test"] = mock_session
mgr._loop = MagicMock()
mgr._tool_map["mcp__test__ping"] = ("test", "ping")
mock_future = MagicMock()
mock_future.result.side_effect = concurrent.futures.TimeoutError()
with (
patch("asyncio.run_coroutine_threadsafe", return_value=mock_future),
pytest.raises(TimeoutError),
):
mgr.call_tool_sync("mcp__test__ping", {}, timeout=1)
assert mgr._consecutive_failures.get("test", 0) == 1
def test_call_tool_sync_records_success(self):
mgr = MCPClientManager({"test": {"type": "stdio", "command": "echo"}})
mock_session = MagicMock()
mock_session.call_tool = MagicMock(return_value="sentinel")
mgr._sessions["test"] = mock_session
mgr._loop = MagicMock()
mgr._tool_map["mcp__test__ping"] = ("test", "ping")
# Pre-set a failure
mgr._consecutive_failures["test"] = 2
mock_result = MagicMock()
mock_result.content = []
mock_result.isError = False
mock_future = MagicMock()
mock_future.result.return_value = mock_result
with patch("asyncio.run_coroutine_threadsafe", return_value=mock_future):
mgr.call_tool_sync("mcp__test__ping", {}, timeout=5)
assert mgr._consecutive_failures.get("test") is None
def test_connection_error_evicts_session(self):
mgr = MCPClientManager({"test": {"type": "stdio", "command": "echo"}})
mock_session = MagicMock()
mock_session.call_tool = MagicMock(return_value="sentinel")
mgr._sessions["test"] = mock_session
mgr._loop = MagicMock()
mgr._tool_map["mcp__test__ping"] = ("test", "ping")
mock_future = MagicMock()
mock_future.result.side_effect = BrokenPipeError("dead")
with (
patch("asyncio.run_coroutine_threadsafe", return_value=mock_future),
pytest.raises(BrokenPipeError),
):
mgr.call_tool_sync("mcp__test__ping", {}, timeout=5)
assert "test" not in mgr._sessions
def test_independent_circuits_per_server(self):
mgr = MCPClientManager({})
for _ in range(3):
mgr._cb_record_failure("a")
is_open_a, _ = mgr._cb_check("a")
is_open_b, _ = mgr._cb_check("b")
assert is_open_a
assert not is_open_b
def test_mcp_error_does_not_trip_circuit(self):
"""Protocol errors (McpError) should not count as transport failures."""
from mcp import McpError
from mcp.types import ErrorData
mgr = MCPClientManager({"test": {"type": "stdio", "command": "echo"}})
mock_session = MagicMock()
mock_session.call_tool = MagicMock(return_value="sentinel")
mgr._sessions["test"] = mock_session
mgr._loop = MagicMock()
mgr._tool_map["mcp__test__ping"] = ("test", "ping")
mock_future = MagicMock()
mock_future.result.side_effect = McpError(ErrorData(code=-32601, message="tool not found"))
with (
patch("asyncio.run_coroutine_threadsafe", return_value=mock_future),
pytest.raises(McpError),
):
mgr.call_tool_sync("mcp__test__ping", {}, timeout=5)
# Circuit should NOT have recorded a failure
assert mgr._consecutive_failures.get("test", 0) == 0
# ---------------------------------------------------------------------------
# Fix 3: Safe transport stream pre-close
# ---------------------------------------------------------------------------
class TestSafeTransportStreams:
"""Verify stream references are stored and pre-closed."""
def test_pre_close_streams_closes_both(self):
mgr = MCPClientManager({})
stream_a = MagicMock()
stream_b = MagicMock()
mgr._server_streams["srv"] = (stream_a, stream_b)
async def _run():
await mgr._pre_close_streams("srv")
asyncio.run(_run())
stream_a.aclose.assert_called_once()
stream_b.aclose.assert_called_once()
assert "srv" not in mgr._server_streams
def test_pre_close_streams_ignores_missing(self):
mgr = MCPClientManager({})
async def _run():
await mgr._pre_close_streams("nonexistent")
asyncio.run(_run()) # should not raise
def test_pre_close_streams_suppresses_errors(self):
mgr = MCPClientManager({})
stream_a = MagicMock()
stream_a.aclose.side_effect = RuntimeError("boom")
stream_b = MagicMock()
mgr._server_streams["srv"] = (stream_a, stream_b)
async def _run():
await mgr._pre_close_streams("srv")
asyncio.run(_run()) # should not raise despite stream_a error
stream_b.aclose.assert_called_once()
def test_shutdown_clears_stream_refs(self):
mgr = MCPClientManager({})
mgr._server_streams["srv"] = (MagicMock(), MagicMock())
mgr.shutdown()
assert len(mgr._server_streams) == 0
# ---------------------------------------------------------------------------
# Fix 4: Notification debounce
# ---------------------------------------------------------------------------
class TestNotificationDebounce:
"""Verify notification-triggered refreshes are debounced."""
def test_debounce_within_window(self):
mgr = MCPClientManager({})
mgr._last_notification_refresh["srv"] = time.monotonic()
# We can't easily call _on_notification (it's a closure), so test
# the debounce logic directly via the timestamp check
now = time.monotonic()
last = mgr._last_notification_refresh.get("srv", 0.0)
assert now - last < mgr._NOTIFICATION_DEBOUNCE
def test_debounce_passes_after_window(self):
mgr = MCPClientManager({})
# Set timestamp well in the past
mgr._last_notification_refresh["srv"] = time.monotonic() - 10
now = time.monotonic()
last = mgr._last_notification_refresh.get("srv", 0.0)
assert now - last >= mgr._NOTIFICATION_DEBOUNCE
def test_debounce_is_per_server(self):
mgr = MCPClientManager({})
mgr._last_notification_refresh["srv_a"] = time.monotonic()
# srv_b has no timestamp — should pass debounce
now = time.monotonic()
last_b = mgr._last_notification_refresh.get("srv_b", 0.0)
assert now - last_b >= mgr._NOTIFICATION_DEBOUNCE
# ---------------------------------------------------------------------------
# Fix 5: Periodic refresh backoff
# ---------------------------------------------------------------------------
class TestPeriodicRefreshBackoff:
"""Verify periodic refresh backoff and auto-reconnect."""
def test_backoff_set_on_failure(self):
mgr = MCPClientManager({})
mgr._refresh_failures["srv"] = 1
# Simulate what _periodic_refresh does on failure
failures = mgr._refresh_failures.get("srv", 0) + 1
mgr._refresh_failures["srv"] = failures
backoff = min(mgr._REFRESH_BACKOFF_BASE * (2 ** (failures - 1)), mgr._REFRESH_BACKOFF_MAX)
mgr._refresh_backoff_until["srv"] = time.monotonic() + backoff
assert mgr._refresh_backoff_until["srv"] > time.monotonic()
assert failures == 2
def test_backoff_doubles(self):
mgr = MCPClientManager({})
b1 = min(mgr._REFRESH_BACKOFF_BASE * (2**0), mgr._REFRESH_BACKOFF_MAX)
b2 = min(mgr._REFRESH_BACKOFF_BASE * (2**1), mgr._REFRESH_BACKOFF_MAX)
b3 = min(mgr._REFRESH_BACKOFF_BASE * (2**2), mgr._REFRESH_BACKOFF_MAX)
assert b1 == 60
assert b2 == 120
assert b3 == 240
def test_backoff_capped(self):
mgr = MCPClientManager({})
b = min(mgr._REFRESH_BACKOFF_BASE * (2**20), mgr._REFRESH_BACKOFF_MAX)
assert b == mgr._REFRESH_BACKOFF_MAX
def test_backoff_clears_on_success(self):
mgr = MCPClientManager({})
mgr._refresh_failures["srv"] = 3
mgr._refresh_backoff_until["srv"] = time.monotonic() + 1000
# Simulate success
mgr._refresh_failures.pop("srv", None)
mgr._refresh_backoff_until.pop("srv", None)
assert "srv" not in mgr._refresh_failures
assert "srv" not in mgr._refresh_backoff_until
def test_server_status_includes_circuit_info(self):
mgr = MCPClientManager({"srv": {"type": "stdio", "command": "echo"}})
status = mgr.get_server_status("srv")
assert "circuit_open" in status
assert "consecutive_failures" in status
assert status["circuit_open"] is False
assert status["consecutive_failures"] == 0
def test_server_status_shows_open_circuit(self):
mgr = MCPClientManager({"srv": {"type": "stdio", "command": "echo"}})
for _ in range(3):
mgr._cb_record_failure("srv")
status = mgr.get_server_status("srv")
assert status["circuit_open"] is True
assert status["consecutive_failures"] == 3
+51
View File
@@ -161,3 +161,54 @@ class TestModelDefinitionStorage:
assert m["capabilities"] == "{}"
assert m["enabled"] is True
assert m["created_by"] == ""
# Per-model sampling params default to None (use global default)
assert m["temperature"] is None
assert m["max_tokens"] is None
assert m["reasoning_effort"] is None
def test_create_with_sampling_params(self, db: SQLiteBackend) -> None:
did = _make_id()
db.create_model_definition(
definition_id=did,
alias="sampling",
model="gpt-5",
temperature=0.7,
max_tokens=8192,
reasoning_effort="high",
)
m = db.get_model_definition(did)
assert m is not None
assert m["temperature"] == 0.7
assert m["max_tokens"] == 8192
assert m["reasoning_effort"] == "high"
def test_create_with_zero_temperature(self, db: SQLiteBackend) -> None:
"""temperature=0.0 is a valid override, distinct from None."""
did = _make_id()
db.create_model_definition(
definition_id=did, alias="zero-temp", model="o3", temperature=0.0
)
m = db.get_model_definition(did)
assert m is not None
assert m["temperature"] == 0.0
def test_update_sampling_params(self, db: SQLiteBackend) -> None:
did = _make_id()
db.create_model_definition(definition_id=did, alias="upd-samp", model="gpt-5")
db.update_model_definition(did, temperature=1.2, max_tokens=4096, reasoning_effort="low")
m = db.get_model_definition(did)
assert m is not None
assert m["temperature"] == 1.2
assert m["max_tokens"] == 4096
assert m["reasoning_effort"] == "low"
def test_clear_sampling_params(self, db: SQLiteBackend) -> None:
"""Setting sampling params to None clears them back to global default."""
did = _make_id()
db.create_model_definition(
definition_id=did, alias="clear-samp", model="gpt-5", temperature=0.9
)
db.update_model_definition(did, temperature=None)
m = db.get_model_definition(did)
assert m is not None
assert m["temperature"] is None

Some files were not shown because too many files have changed in this diff Show More