The interactive pane only auto-scrolled when isNearBottom() was true, but it measured that AFTER the new node was appended. A tool block is a tall one-shot append (batch shell, approval card, or result) that clears the 80px near-bottom threshold in a single step, so the post-append check read false and auto-follow silently disengaged at exactly tool-call time — the view froze at the top of the block and only snapped back at the next stream_end. Token streaming was unaffected because each append stays sub-threshold.
Capture the near-bottom state as the first statement of each tool-render method, before any DOM mutation, and thread it into scrollToBottom(stick). This re-pins when the user was already at the bottom and, unlike the coordinator pane's unconditional pin, leaves the view alone if they deliberately scrolled up while a result was rendering.
Methods fixed: announceToolBlock, showInlineToolBlock, resolveApproval, appendToolOutput (all three exit paths), appendToolOutputChunk.
(cherry picked from commit a628e9f3b4)
The streamable-http server bound to 0.0.0.0/a LAN IP answered TCP and
/watch but returned 421 "Invalid Host header" on /mcp for every remote
node — which broke multi-node play entirely. FastMCP freezes DNS-rebinding
protection (a localhost-only Host allowlist) at CONSTRUCTION, and this
module builds its FastMCP at import time with the default 127.0.0.1 host;
flipping settings.host in _serve afterward never updated the frozen
allowlist, so the LAN Host was always rejected.
When UNDERSTONE_HOST is off localhost, drop the allowlist in _serve before
run() — matching the SDK's own default for a non-localhost bind. The /mcp
and /watch routes are unauthenticated by design, so serve only on a trusted
network (documented).
Regression test pins the mechanism: a default FastMCP 421s a foreign Host,
a protection-disabled one accepts it. Tests 420 -> 421.
(cherry picked from commit 1468ca7972)
The job was named "test", colliding with core CI's "test" matrix so the PR
checks list showed two "test (3.11)" rows. Rename it to "understone" so the
example's checks read unambiguously (understone (3.11) / (3.13)).
(cherry picked from commit efa8664e4d)
- CodeQL (implicit string concatenation in a list): collapse the wrapped
bullets in cli._render_validate_coverage to single literals. The rendered
output is byte-identical (the example's ruff ignores E501); clears all
six alerts and reads cleaner.
- Copilot: packs/README no longer claims the directory ships "effectively
empty" — it ships the bundled Cinder Wastes alternate world.
- Copilot: the Cinder Wastes' ash_flats and caldera_deep zones overlapped
on column x=60 (inclusive bounds + first-match zone_for silently shadowed
the tier-3..5 band onto a 1x5 deep-edge strip). Move caldera_deep to
x0=61 — no overlap, no dead tiles, deep zone still covers the dungeon.
And harden the loader: overlapping zone rectangles are now a
WorldLoadError, so no authored pack can ship that bug unseen (the
cold-author dogfood loop — a generated pack exposed a validator gap).
Tests 419 -> 420 (zone-overlap rejection). Both worlds validate sound and
remain winnable by the sim bot.
(cherry picked from commit a0a097dfa8)
The door-game example is a standalone package (no turnstone-core
dependency) that the root suite does not collect — its
testpaths are scoped to ["tests"], so the example's 419 tests, ruff,
and mypy gates never ran in CI.
Add a path-filtered workflow that installs the example and runs its
full gate (pytest + ruff check + ruff format --check + mypy) whenever
examples/door-game (or this workflow) changes, across the example's
declared Python floor and ceiling (3.11, 3.13). Pinned action SHAs and
contents:read permissions match the existing CI workflows.
(cherry picked from commit 30c09aaf51)
A game-loop mechanics patch: the satchel becomes a real stacking inventory,
forging now demands ore won in combat (not just gold), and a vault lets a
hero protect coin from ambush.
- Stacking satchel: the bag re-encodes from a flat id list to "id:qty"
stacks, so potions stack (three Minor Potions fill one slot, not three)
and materials ride alongside. satchel_max now caps distinct KINDS (3);
per-kind quantity is unbounded. quaff/death-save still pull the strongest
potion and ignore materials. One pure codec (engine/satchel.py) owns the
encoding; the façade, the Watch, and the sim all decode through it — no
three-way drift (the v0.9 single-source lesson). The codec parses a bare
id as qty 1, so it can never silently drop a malformed stack.
- Ore-gated forge: ore is a material that drops from won dungeon-rung
fights (and, less often, forest fights), stacks in the satchel, and is
not buyable or sellable — you earn your edge by fighting for it. Forging
now costs gold AND ore ((plus+1) ore per tier), so a rich-but-idle hero
can no longer buy power at the dice table. The dungeon is now also the
mine.
- The vault: deposit/withdraw at the inn moves coin to a strongbox that
ambush cannot touch and that SURVIVES the Wyrm-win legacy reset — the
carry-vs-protect decision the PvP economy was missing.
- Surfaced on both the /watch lobby TV and the in-chat door_status sheet:
each hero's stacked satchel, carried gold, and vaulted gold.
- Tuning (the sim is the instrument): the ore gate added ~2 days to the
Vale and ~1.6 to the Cinder Wastes; the greedy bot still slays the Wyrm
3/3 on both, fully forged to +3/+3, so the loop is not stalled. Defaults
held — no numbers needed retuning.
Four new banded settings (forge_ore_item, forge_ore_per_plus,
ore_dungeon_drop, ore_forest_chance); both worlds gained an ore item.
Schema mutated in place (banked column, satchel re-encoding) — pre-1.0, no
migration by design; a real migration story is owed at 1.0. Tests 382 ->
419; the vault-survives-rebirth invariant and the codec are revert-verified.
(cherry picked from commit 393a6fc2b2)
Graphics polish: distinct terrain and structures now read by COLOUR on the
Watch, not only by glyph. One unified palette, shared by every world — the
fix is to grow the set of distinct object-type roles, not to fork per-world.
- Roads were the tell: road shared the "floor" green with grass, so a path
vanished into the meadow on the lobby TV. Likewise forest shared "tree",
the three town buildings all shared "town", and the Cinder Wastes' molten
slag borrowed "water" and rendered BLUE. Each is now its own role: road
(stone), forest (lush green) with scrub (its barren ember-brown
counterpart for volcanic/desert dense terrain that must NOT read as
woods), lava (molten orange), barren (wasteland taupe), and inn/shop/
healer split out of the generic town.
- Both worlds remap onto the shared vocabulary; in each, no two distinct
terrain/building types share a colour. A live render caught the Cinder
cinder-fields rendering green under the generic "forest" role — hence the
scrub role, so the volcanic waste reads warm. The text frame renderer
stays monochrome (it never read colour), so frames and goldens are
untouched — this is Watch-only.
- The bug class is now closed by construction: a test asserts the Watch
PALETTE carries a hex for EVERY Color role, so a role can never ship
unpaintable and silently fall back (which is exactly how road hid).
- Color.assignable() is the single source for the overlay-vs-assignable
split (runtime actor/item colours and the DEFAULT fallback are not
author-pickable); the authoring manual's colour vocabulary generates
from it, so it can't drift.
Tests 373 -> 382. floor/tree/forest are three greens kept deliberately
distinct (forest is olive-hued); verified on a real render along with the
scrub fix.
(cherry picked from commit 917e391b1f)
The slice that proves the pipeline: a second world authored entirely by an
LLM from AUTHORING.md and the validator alone, plus the tooling to discover,
theme, and balance-test any world.
- The dogfood: "The Cinder Wastes" — an ashen volcanic underworld (slag
rivers, a caldera mouth, a Magma Wyrm) — was written cold by an agent
given only the generated authoring manual and `understone validate`. It
passed validation on the FIRST run with zero failures. Its stumble log
found six places where the manual stated a rule the validator didn't
enforce; those became permanent hardening (below). It ships in
understone/world/packs/ and glows ember on the lobby TV.
- `understone worlds` lists every bundled world (the Vale + alternates)
with its load status, via one shared discovery path.
- Per-world Watch themes: settings.watch_theme (phosphor/amber/ice/ember,
loader-validated) repaints the spectator page; the Vale's green is
byte-for-byte unchanged.
- The sim harness: a pure, seeded, greedy bot plays the real game façade
over an injected day-stepping clock and emits a balance report —
`understone simulate PATH [--days N] [--seeds K]`. It SLAYS THE WYRM on
both worlds (Vale ~day 13, Cinder ~day 25), so the whole v0.1->v0.7 loop
is proven winnable end-to-end by an unclever bot through the real stack.
- Loader hardening from the dogfood: a rare monster may not occupy a
dungeon-rung guardian slot (it would silently become a fixed foe and
leave the rare pool); exactly one monster may be the boss; and the
boss-tier error now says "no non-boss monster," matching the manual.
AUTHORING gained a generated "what validate checks vs. what it cannot"
section so the rule/guidance boundary is honest.
Review hardened the bot for arbitrary authored packs (a MENU-mode fight
spin and four related robustness gaps that were latent on the shipped
worlds), and documented that final_level reads post-legacy-reset. Tests
359 -> 373; both worlds still win byte-identically after the fixes.
(cherry picked from commit 65e7b404bc)
The depth slice: four standing reasons to return past the daily reset.
- The rung ladder: the dungeon is a descent fought one rung per turn, each
guardian a fixed tier. A loss bounces you home but your depth PERSISTS —
you re-enter where you left off. The Wyrm now gates on BOTH level AND
reaching the floor (the deep has a bottom, and you must have touched it).
- The satchel + the death-save: potions are CARRIED now (up to three),
bought to the satchel, drunk with quaff. The heart of it: when any fight
would kill the active fighter and they carry a draught, the strongest is
drunk automatically — they survive standing at the potion's value, no
bounce. This fires on EVERY fight (forest, rung, and the Wyrm itself —
a potion carried to the climax is a real tactical choice); a Wyrm loss
so saved is "driven back, alive but unproven," not devoured. The sleeping
ambush victim never quaffs (they are asleep). combat.py stays pure — the
satchel and the save live entirely in the façade.
- The forge: the shop spends scaling gold to add a +1 edge to equipped
weapon or armour, capped — the late-game gold sink. Swapping or selling
the piece loses the edge with it (one centralized unequip clears the
bonus and the plus so a stat can never go phantom).
- Rare beasts: a few named foes prowl the forest via weighted selection,
surfacing seldom; felling one is a public Herald flash and always yields
a draught into the satchel. Rung guardians are never rare (fixed foes).
Four new player columns; four new banded settings; dungeon_tiers extended
to three rungs. Tests 283 -> 330; the death-save (all four paths), forge
accounting across forge/buy/sell/legacy, rung math, and weighted rare
selection all pinned, with the death-save and forge invariants
revert-verified.
(cherry picked from commit dcc0e5fb0a)
The look of the next age — the modern equivalent of the ASCII->CP437 leap.
Full Unicode is available now, but the whole stack (text frames, golden
tests, the Watch's 1ch grid) assumes one glyph = one column, so the
enabling piece is a WIDTH RULE, not the glyphs themselves.
- textwidth.is_grid_safe: one code point, printable, East-Asian width not
Wide/Fullwidth, no combining/format/control category. This is the
one-glyph-one-column contract. Ambiguous-width glyphs are ACCEPTED on
purpose — they ARE CP437 (the wall, the club-tree, the up-arrow forest)
and render single-column on the Western-monospace metrics every surface
uses; only genuinely double-width runes are barred. The loader enforces
it on every map glyph; the player-name/free-text sanitizer enforces the
same rule (the narrow ledger), so a wide name can't shear a frame.
- Re-skin: water ~ -> ≋, inn -> ⌂, healer -> ✚, dungeon mouth -> ∩, and
the other adventurer -> ☻ (CP437's own player glyph). The colour field
the renderer has carried unused since v0.1 now has a second consumer.
- Texture variants: grass and water vary by a deterministic per-coordinate
hash, rendered identically in the Python frame builder and the Watch's
JS. The two are kept in lockstep by shared hash constants + an agreement
test that replays the JS arithmetic and asserts it equals the Python
output for every variant over a grid — not a comment-coupled copy.
- Watch glow-up: a Noto Sans Mono font stack and a UTC-hour day/night tint
(the Vale darkens at dusk on the lobby TV).
- The curated SAFE_PALETTE is enforced author-usable: a test asserts no
palette glyph collides with the reserved player markers, so AUTHORING's
generated appendix can't advertise a glyph the loader would reject.
- Resume is identity-preserving: an existing character resumes by exact
stored name without re-validating the width rule (which governs creation
only) — resume must never lock anyone out.
Tests 231 -> 283; width edges (CJK/emoji/combining/fullwidth), the
Python<->JS lockstep, the palette/reserved guard, and resume-vs-create all
pinned and revert-verified.
(cherry picked from commit b76b2a98d0)
The social slice: the shared world gets teeth, letters, and a house game.
- Ambush (async PvP, classic door-game player-kill spirit): waylay an adventurer who has
not yet begun their day. Ordered gates — known target, not yourself, the
gatekeeper shields the young (both >= min level), level band +-2, the
SLEEP RULE (acting today makes you watchful — an active-play defense),
mercy for the downed (hp<=1 cannot be piled on: even bandits have
standards), once per pair per UTC day. Win: capped gold cut transfers,
victim wakes at the spawn-stone with a private note; lose: the sleeper
wakes blade-in-hand and the Herald crows your shame. The attacker wears
the counter-blows the combat log narrates (state matches story). Both
players persist in one transaction.
- The inn mailbox: events carry a target ('' = public). door_log delivers
private notes to the addressee only; the Watch and other players never
see them. Mail is DURABLE past the in-memory tail (SQLite backfill for
cursors older than the resident window) — the broadsheet is ephemeral,
letters are not. Sanitized, daily-capped.
- Inn dice: 2d6 against the house, bet- and count-capped per day, big wins
make the news.
- Six new banded settings; four day-counter columns join the shared lazy
UTC reset; schema stamp stays 1 (pre-1.0 mutates in place by design).
Tests 184 -> 231; sleep rule, mercy gate, band boundary (exact/over),
refusal precedence, attacker wear, zero-gold robbery, mail eviction
survival, and Watch privacy all pinned; guards revert-verified.
(cherry picked from commit 08d46f086f)
The IGM seam realized: world packs are now a first-class authoring target
for models and humans, with a validate loop and a loader hardened for
routinely-untrusted generated content.
- understone newpack DIR scaffolds a pack (the six content JSONs templated
from the shipped Vale) plus AUTHORING.md — a manual written for a model
to follow cold. Its bands table is RENDERED FROM the loader's own band
constants at scaffold time, so documented limits and enforced limits
cannot drift.
- understone validate DIR loads a pack and prints either a pack report
("This pack is sound. The door stands open.") or the loader's
file/index/field-naming error — the authoring feedback loop.
- Loader hardening: glyphs must be one printable column-safe character and
never the frame box-drawing set or the @/& player markers (map content
cannot impersonate players or forge frame chrome); map dims 8..256;
per-file count caps; display-name length caps. All errors instructive.
- The packaged-world path is single-sourced (understone.world.
PACKAGED_WORLD_DIR) for the server default and the scaffold template.
- README "Authoring worlds" section frames the loop: newpack -> write or
generate -> validate -> serve with UNDERSTONE_WORLD=dir.
Review round: bug finder returned zero findings; quality round fixed the
world.json doc example (it showed a zone fragment where an authoring model
would copy a whole-file shape — now a labeled skeleton), the stale Usage
docstring, and the duplicated packaged-path constant.
Tests 166 -> 184. Scaffold round-trips through load_world by test.
(cherry picked from commit d40c4c85ee)
A read-only CRT spectator page served by the game process itself, plus
content depth. Input never flows through the Watch — it is the wall-mounted
terminal in the BBS room; chat remains the only actuator, so there is no
input channel to deadlock and no cross-origin surface (the page polls the
same origin that served it).
- /watch: one self-contained page (inline CSS/JS, no external assets),
phosphor CRT styling. The base map paints once from /watch/world.json
(terrain glyph rows + a glyph->color legend — the palette the text
renderer has deliberately ignored since v0.1 finally gets its first
renderer); players overlay as positioned glyphs repainted from
/watch/state.json every 2s; the sidebar carries the roster with win
stars, the Hall of Legends, and the Herald. SIGNAL LOST on poll failure;
the bootstrap retries so a spectator arriving during a server blip
recovers without a reload.
- Routes ride FastMCP custom_route on the existing process — read-only
handlers with no awaits between reads (handlers and sync tools
interleave on one event loop, so every response is a consistent
snapshot).
- door_join/door_help advertise the Watch URL in http mode (stdio: none).
- Content: +5 monsters (one per tier; the gauntlet's first-in-tier foes
preserved), +3 items smoothing the gear curve, +6 events; fight weight
retuned to hold ~55% of encounter rolls. Zero geography churn.
- Review round: the Herald window is a plain list tail (id arithmetic
under-reported the feed when AUTOINCREMENT ids gap — regression-pinned
with sparse ids), and the bootstrap-retry fix above.
Tests 149 -> 166.
(cherry picked from commit d54110ffcb)
The "make it a game" slice: a win condition with classic-door-game-style legacy, texture
between fights, and a shared broadsheet.
- The Wyrm Below: a boss (flagged in the pack, excluded from random bands)
behind a level-gated `challenge` verb at the dungeon. Victory writes a
Hall of Legends row and the character resets to the fresh-start kit,
keeping a wins counter rendered as ★ on the leaderboard — the classic
race-reset-race loop. Defeat and stalemate flight make the news.
- Forest events: movement encounters weighted-pick from a content-pack
table (fight/gold/heal/trap/lore). Only fights stop the walk or cost
turns; texture is free and private. Trap damage floors at 1 hp.
- The Understone Herald: door_log is a broadsheet with a masthead and
write-time template variety; the public feed is curated to notable beats
(joins, blessings, level-ups, defeats, the Wyrm's fate) — town errands
stay private.
- Reward narration moved from the combat engine to the façade, composed at
the moment gold/xp are actually banked, so the server can never narrate
a reward it did not apply (the Wyrm win previously claimed +400 XP /
+250 gold that the legacy reset wiped).
- Fresh-start hp/atk/def promoted into world.json settings alongside the
starting kit; dungeon-tier validation counts non-boss monsters only,
keeping the validator's no-silent-rung promise true.
Schema mutated in place (players.wins, hall_of_fame) — pre-release, no
migration path by design. Tests 109 -> 149; the challenge level gate is
negative-tested; rank stars survive 24-char names (compact form past 5).
(cherry picked from commit 4b8681db8a)
A shared-world, classic-door-game-style door game in examples/door-game/: a pure-stdlib
game engine (tile overworld + location menus, seeded combat, daily turn
budget, leveling, shop, event log, leaderboard) behind nine sync door_*
FastMCP tools returning monochrome box-drawing frames. The connecting
session's LLM plays dungeon master — tool descriptions plus a door_help
manual teach a cold model to run the game with zero setup, while the server
owns all dice and state, so the DM narrates around facts it cannot bend.
Non-obvious decisions:
- engine/screen/world/persistence import stdlib only; server.py is the only
mcp import. All nine handlers are sync def: on mcp 1.27 they execute
inline on the event loop (verified against func_metadata), so tool bodies
serialize and one SQLite connection (WAL, per-action commit) is safe.
check_same_thread=False exists only because the Store may be constructed
on a different thread than the serving loop.
- Streamable HTTP serves ONE process = one shared world (players appear on
each other's maps; async "while you were away" event feed); stdio is the
solo-world fallback.
- The economy is content, not code: daily_turns, costs, xp curve, bestow
budget, and dungeon tiers live in world.json settings, band-validated by
the loader. door_bestow gives the DM capped, event-audited largesse
(gold/heal only, never turns) so story generosity cannot melt the shared
leaderboard.
- Player names and bestow reasons are sanitized (printable-only, length
caps) because they flow into the shared event log and from there into
other players' DM context — embedded newlines would forge log lines.
- Daily turn/bestow pools lazy-reset per UTC day on every consuming path
(injectable clock); the dungeon gauntlet is a fixed boss ladder by design.
Tests: 109 — engine units with seeded RNG + frozen clock, hand-authored
golden frames paired with structural asserts, loader band rejections, and
one real-wire integration test (uvicorn + streamablehttp_client) with a
two-session shared-world assertion. Negative-tested by reverting the guard
and watching the suite fail: the daily turn-budget guard, the bestow cap,
and the sanitizer's isprintable clause.
(cherry picked from commit 99e7dc17ec)
The coordinator memory scope was keyed by the session's ws_id, so every
new coordinator session started with an empty namespace and its rows
were orphaned on close — coordinator memory never actually persisted.
Re-key the scope to the coordinator's creator user_id: one durable
orchestration namespace per user, shared by all of that user's
coordinator sessions (concurrent ones included; upsert-by-name is the
collision rule).
The child-containment threat model is unchanged: the gate is session
KIND — children are always interactive and share the parent's user_id,
so _validate_scope rejects them before scope resolution, and the REST
memories API still rejects the coordinator scope outright. The implicit
visibility lane now also fails closed on an empty scope_id to match the
explicit search/list lanes (the storage helpers treat a falsy scope_id
as 'no scope_id filter', which would have read every user's rows).
Anonymous coordinators are no longer constructible: ChatSession refuses
kind=COORDINATOR with an empty user_id at the constructor — the single
choke point covering create, rehydration of legacy rows (surfaced by
the open handler as a 503 with remediation text), and any future host —
and the console no longer masks an empty uid as a phantom 'system'
principal when minting coordinator JWTs, per CoordinatorTokenManager's
documented 'sub = the real creator user_id' contract.
Migration 061 carries existing coordinator rows across: rows whose
owning workstream is gone or ownerless are deleted (unreachable under
user keying), same-name collisions within a user keep the newest
updated row (memory_id tiebreak), and survivors re-key to the owner's
user_id.
(cherry picked from commit 30b590fb25)
_buildHandle hard-coded aria-valuemin/max at 10/90 (inherited from the
old ui/static implementation) while the actual drag/keyboard clamp is
_ratioBounds — the cell minimums against the split node's OWN px region
(a 1200px host really clamps at ~17/83; nested splits sit tighter), so
assistive tech was told a wider range than the separator allows.
aria-valuenow/min/max are now all written in _applyLayout's handle loop
from _ratioBounds(h.node) — one writer, refreshed on every drag,
keyboard nudge, and structural change. A bare window resize can stale
the advertised range until the next interaction (no resize listener by
design — % insets make resizes free), still strictly truer than a
constant. The max>=min guard covers a host shrunk below two cell
minimums, where the bounds legitimately cross.
The /coordinator/{ws_id} standalone page is reachable only by direct
URL — all three console navigation sites are shell-fallback else
branches behind openPane. Record that in the sidebar-padding comment
so the scope isn't over-read as a live second surface.
The per-pane ✕/− chip floats at the pane's top-right — exactly where
the coordinator sidebar's toggle row and Children refresh button sit,
so the chip covered them. Pane-hosted coordinators now start the
sidebar content 44px down (padding, not margin, so the column's left
border still runs the full pane height); the standalone coordinator
page has no chip and keeps the 14px default.
Dual designer review (one primed on the branch context, one cold), all
measured findings applied:
- The per-pane chip was a mode-error trap: identical glyph at the
identical locus, reversible in split mode (hide cell) but destructive
single-pane (close pane). Now − hides, ✕ closes, and the close mode
wears a danger hover/focus ring so the irreversible action telegraphs
before the click lands.
- Single-pane chip anchored to the VIEWPORT: an unpositioned section
resolves absolutes to <body>, so the chip only coincidentally landed
near the pane corner. .panes > section.pane is now position:relative
in both modes (all pane-content absolutes verified to anchor to their
own local relative parents).
- Light-theme AA (measured): .shown tab underline 55% mix composited to
2.34:1 -> 80% (~3.7:1 light / ~5:1 dark); focused-cell ring 2.60:1 on
light -> 75% mix override there (dark keeps 55% at 3.75:1).
- Chip: border --hair-2 measured ~1.3:1 (invisible) -> --ink-4; 22px
target under WCAG 2.5.8's 24px floor -> 28px; right offset clears the
message scrollbar gutter; light resting glyph one ink step up.
- Focus bar inset 1px from cell sides (no doubled-accent stripe where
it butted a separator at the T-junction); greyscale font smoothing on
the tail glyphs (subpixel RGB fringed the box-drawing characters).
Rejected with rationale: aria-pressed on the split buttons (they are
one-shot verbs — splitting again nests — not mode toggles).
Four refinements from first live use:
- Per-pane ✕ chip, top-right of every visible pane. Split mode: hide
that cell (closeCell — the tab stays, the sibling absorbs the space).
Single-pane: close the pane outright (withheld from the unclosable
Dashboard). The click decides at click time; the label tracks the
mode. Manager-injected into the pane section — content untouched.
- Coordinator child links open BESIDE the coordinator (openPaneBeside:
split right of the focused cell, seeded with the child pane) instead
of replacing it — the parent stays on screen. Degrades to the plain
focused-cell swap when the split is denied (cap / narrow viewport).
splitFocused() gained an optional explicit-fill parameter for this.
- Tier-1 ws_closed now CLOSES the open interactive pane (tab gone, a
split cell collapses) — the coordinator-closes-its-child flow,
matching the standalone's pane-auto-close. The dead-banner lane
stays for streams that die without a ws_closed (node crash/network),
where the session may still be revivable.
- Paint bug: the focused-cell ring was an inset box-shadow on the
section, which paints in the element's own background layer — UNDER
opaque children touching the edges, so the status bar / composer
strip occluded it. The ring now rides a click-transparent ::after
overlay above pane content; the 2px top bar sits above the ring line.
The livepass shell surface's demo panes grew a .ws-status-bar footer so
the occlusion bug class stays visible to future passes.
Revives the split-pane feature retired with ui/static (step 6), rebuilt
on PaneManager: an optional binary layout tree (null = the one-pane-per-
tab behaviour, unchanged) renders visible panes as %-inset cells — no
reparenting, so live stream DOM, scroll state and media survive layout
changes. Tabs stay global: the active tab is the focused cell, a
backgrounded tab swaps into it, clicking inside a visible pane focuses
its cell, .shown marks visible-unfocused tabs. Separators resize by
pointer-capture drag and arrow keys (role=separator + aria-value*); the
tree persists in the working-set blob and rehydrate prunes leaves whose
pane did not restore. Limits: 6 cells, 200x150 cell minimums, denials
toast the manager's reason.
Affordance: Split right / Split down / Unsplit buttons in the tab-bar
tail replace the redundant [+] (the permanent Dashboard tab is the
launcher) — deliberately no contextmenu override this time. The dead
TS_APP.focusLauncher seam goes with it.
Measured chrome: the focused cell wears a 2px accent top bar (no thin
tinted ring clears 3:1 in both themes) plus a 55%-mix inset ring;
separators rest at --ink-4 with solid-accent hover/drag/focus; .shown
tabs carry an accent underline; the tail cluster is fenced and lifted
to --ink-3.
scripts/livepass.py grows a third surface: shell/livepass.html boots
the real shell.js + pane.js and drives ?split=right|down|three|none
(+ &theme=light), stamping SPLIT-READY-<cells> / SPLIT-FAILED-<reason>.
121 warnings -> 0. Two upstream deprecations get narrowly-scoped
filterwarnings entries (the mcp streamablehttp_client rename — adoption
deliberately rides the v2 migration since the new entry point's call
shape changes again there; the starlette httpx TestClient notice). The
one real RuntimeWarning is fixed at the source: tests that mock
asyncio.run_coroutine_threadsafe handed real coroutines to a stub that
never awaited them, GC-firing 'coroutine was never awaited' inside
whatever unrelated test ran later (the same cross-test bleed mechanism
as the CI closed-stream spew — per-test filterwarnings markers cannot
catch it, which is why two such markers existed and still leaked). A
shared _dispatch_stub now closes real coroutines before returning the
canned future; the obsolete markers are removed.
mcp 2.0.0a1 shipped 2026-06-11 (stable targeted ~2026-07-27). v2 removes
streamablehttp_client, changes the transport tuple arity, and renames
mcp.types fields to snake_case — all of which our client imports. The
maintainers' release note asks downstream packages to add an upper
bound now (their worked example is this exact constraint). Floor stays
at 1.27: nothing newer adds anything our surface needs, and the #2147
shutdown busy-loop we wrap remains unfixed at every released version.
Resolution is unchanged (1.27.2); lockfile re-pinned metadata only.
Copilot review on #661: empty base_url let the SDK fall back to
https://api.anthropic.com, sending compat-shaped requests to the
commercial API. The lane is local-only by definition, and the /v1-strip
edge case already established fail-loudly-over-silent-prod-retarget;
apply the same principle to the empty case. create_client raises an
actionable ValueError; the admin Detect path surfaces it as a clean
error string via probe_model_endpoint's existing handler.
Add provider id "anthropic-compatible": the existing AnthropicProvider
pointed at Anthropic-compatible local servers (vLLM /v1/messages),
mirroring the openai/openai-compatible split. Registry-only — configured
via the admin Models tab or [models.*] toml, not exposed on the bare
--provider flag, so the CLI/server prod-URL defaults are unreachable for
the lane and real-Anthropic behavior is untouched.
Lane behavior (live-verified against vLLM 0.22.1rc1 + DeepSeek-V4-Flash):
- Capability defaults replace the Claude static table: token_param
max_tokens, thinking_mode none, web_search/tool_search/vision off,
reasoning replay on. vLLM rejects Anthropic server-side tool types
(tools require input_schema) and ignores the thinking request param,
so neither is sent; thinking blocks still stream back and round-trip
through the native lane verbatim.
- Reasoning toggles via server_compat extra_body chat_template_kwargs
(first-class vLLM request field; request-level keys beat server
defaults). _build_thinking_and_kwargs forwards non-internal
extra_params as SDK extra_body; thinking_budget_tokens stays internal.
- No temperature force: thinking_mode none skips the Claude-only
temperature=1.0 requirement.
Admin UI: provider option + URL placeholder (base_url without /v1 — the
SDK appends /v1/messages); the server-compat section shows only the
extra-body field for the lane. thinking_mode round-trips through the
form dropdown for every provider except anthropic-compatible, where it
stays in the raw capabilities JSON — the edit-load lift and save restore
use the same predicate so stored overrides are never silently dropped.
Docs: architecture.md gains the lane subsection incl. verified quirks
(thinking param dropped by vLLM; stop_sequences cut inside thinking and
report end_turn; usage has no cache fields; images need a multimodal
model; mid-conversation system turns are per-model opt-in).
Negative-tested: removing the _INTERNAL_EXTRA_PARAMS exclusion fails
test_internal_keys_not_leaked; the live test drives a streamed turn with
the chat_template_kwargs toggle and asserts no reasoning deltas.
Review feedback: (1) gating the drain on a main-thread truthiness check
of _background_tasks could skip cancellation when a spawn queued via
call_soon_threadsafe had not reached the set yet — submit whenever the
loop is RUNNING and snapshot on the loop, where FIFO callback order
guarantees earlier-queued spawns have landed; (2) shutdown stopped the
loop thread but never closed the loop or cleared _loop/_thread, leaking
selector resources for embedders that cycle managers — close + clear
when we own the thread and it actually stopped (loud warning when it
does not); unowned loops (tests wiring _loop directly) stay untouched;
(3) the bare await-in-suppress drain loops become
asyncio.gather(return_exceptions=True) in both the shutdown drain and
the test fixture.
The post-reconnect catalog refresh was scheduled as a bare
asyncio.create_task: no strong reference (the task could be GC'd
mid-flight, so the refresh might silently never run) and no exception
retrieval (failures surfaced as "Task exception was never retrieved"
at GC time — in CI, onto an already-closed pytest capture stream, the
"I/O operation on closed file" spew; a suspected contributor to the
flaky 60-minute CI hangs via cross-test loop/task state bleed).
- _spawn_background(coro, label): tracked-task set + done-callback
that retrieves and logs failures at warning; discard runs LAST so
set-emptiness means "done AND reported"
- shutdown() drains tracked tasks FIRST, so stack teardown can't race
an in-flight refresh; same run_coroutine_threadsafe idiom and
timeouts as the existing close steps
- running_loop_mgr fixture: cancel-pending -> drain -> stop ->
join(5) with a loud assert -> loop.close() (was stop + silent
join(2), never closed)
- the false-property test ("swallows refresh failure" — nothing
swallowed it) now waits for completion and asserts the logged
warning via the patched module logger (structlog; caplog cannot
observe it), polling inside the patch context
Review feedback on the purge's race window: the pre-SELECT re-verify
left a statement-to-statement gap where a concurrent registration could
still lose rows — and the pre-counted refcount release could underflow
when it didn't. Orphan-ness now rides the DELETE itself (correlated
NOT EXISTS) with refcounts released from its RETURNING, so refs are
released for exactly the rows that were deleted. Input is de-duplicated,
IN-lists chunk at the storage layer's 500 convention, and the scan's
per-workstream ref-count loop is now one anti-join pass.
Conversation rows whose workstreams row is gone (historical unregistered
writers; the delete-during-inflight race re-creating rows after
delete_workstream) are invisible cruft that also pins attachment
refcounts. Add a turnstone-admin verb: default = read-only scan report
(ws_id, rows, attachment refs, first/last); --delete [--yes] purges.
- shared find/purge logic in storage/_utils; protocol + both backends
in lockstep (thin wrappers)
- purge re-verifies orphan-ness in-transaction: a ws_id re-registered
between scan and purge is skipped, never deleted
- releases the deleted rows' attachment refcounts through the
delete_workstream GC path and sweeps workstream_config/overrides
- summary reports actual purge results, including the skipped clause
* fix(ui): re-home MCP consent badge on the Manage Connections row
The L-shell renovation retired the standalone settings gear (#settings-btn).
The MCP pending-consent badge anchored to that gear via _refreshConsentBadge,
which null-guarded silently — so since the renovation pending consent requests
had no indicator (the badge was invisible).
Re-home the badge on the rail's Manage row where the MCP/connections surface
lives in both deployments:
- rail.js gains a generic setRowBadge(tabKey, count, label?) hook + a `badge`
builder: a small ⚠-glyph + count chip (never colour alone) using the DS warn
tokens. mountManage registers row + owning-group-head refs and re-applies live
counts across a (re)mount. When the owning group is collapsed, the count also
mirrors onto the group head so a hidden row never hides the signal. rail.js
stays agnostic — it owns the mechanism, the caller owns the meaning.
- shell.js (the ESM bridge) re-exports setRowBadge on window.TS_SHELL so the
classic ui/static/app.js subsystem can drive it without importing the module.
- The standalone consent subsystem keeps its shell-level ownership: _refresh-
ConsentBadge now drives setRowBadge on the Connections tab, fed by both the
loadPendingConsents hydrate/poll load and live onConsentDetected notifications.
- The shared interactive pane host bridges onConsentDetected to the new
window.TS_APP.onConsentDetected seam (undefined on the console, so the console
pane stays a no-op there); panes only notify.
- The dead colour-only gear badge CSS (.settings-consent-badge, red dot) is
removed; the new chip lives in shell.css as token-only .rail-badge so it
flips themes by construction.
Console MCP tab (Extensions > mcp) and standalone Connections tab
(Extensions > connections) both badge correctly. Pins extended in
test_shell_js.py + test_app_js.py.
* fix(ui): drop the unused head ref from the rail badge row map
Review feedback: _rowEls stored each row's group-head element but every
head consumer resolves it through _groupEls; keeping the duplicate DOM
ref made the remount state shape harder to reason about.
Review feedback: the next-case end markers were exact-indentation
string finds that raised a bare ValueError when unmatched. Use
whitespace-tolerant regexes with actionable assertion messages, and
bound the history-replay window structurally (next role branch, with
a generous fallback) instead of a fixed 600 chars.
Review feedback: (1) the Enter keydown re-dispatched through btn.click(),
relying on the disabled-guard to suppress the browser's own
Enter-to-click — preventDefault + direct activation makes the keyboard
path provably single-fire; (2) the branch-scoped Hls instance was
unreachable from the media error handler, leaking its listeners and
loader timers when the player node was replaced with the retry UI —
hoist the ref and destroy it before replacement.
The interactive Pane renders media embeds (buildMediaEmbed / buildPlayButton),
but the Play activation — _loadHls / _isHlsUrl / _activatePlayer and the
click/keydown delegate — stayed behind in the standalone ui/static/app.js as
DOCUMENT-level listeners. The console L-shell mounts the same interactive.js
module but never loads ui/static/app.js, so the Play button was dead in
console-hosted interactive panes.
Lift the activation into shared_static/interactive.js (alongside the existing
buildMediaEmbed/buildPlayButton — media embeds are interactive-pane-only; the
coordinator pane renders none) and wire it as a pane-owned, root-scoped
this.el click/keydown listener, mirroring the approval-keydown pattern the
fork collapse established. The standalone copy is deleted so no duplicate
implementation remains; both deployments now activate through the one shared
handler.
The hls.js vendor is fetched lazily by absolute /shared/ URL (the same
mechanism renderer.js uses for mermaid), and /shared is mounted at the root in
both turnstone/server.py and turnstone/console/server.py, so the vendor —
which ships in shared_static/hls-1.6.16/ — resolves in both deployments with
no HTML change.
Pins: assert the lift + pane-ownership in test_interactive_pane_js.py and the
standalone-stays-clean guard in test_app_js.py.
The live-SSE/history system-turn dedupe (renderedSystemEventIds /
_renderedSystemEventIds) was already in place on both panes and merged
to main (21af6c4 aligned the persisted row event_id with its SSE event;
09e41d1 added the belt-and-braces Set on the coordinator). The existing
pin tests only assert the Set's .has()/.add()/.clear() symbols appear
somewhere in the file, so a refactor that keeps the Set but short-circuits
the live-handler consultation (guard -> false) re-opens the double-render
while the pins stay green.
Scope the new assertions to their blocks: the live system_turn case must
CONSULT and RECORD against the Set, and the history render path
(replayHistory / refetchHistory's system-role branch) must record each
replayed row's event_id. Bounded at the next switch case rather than the
first break; the dedup-skip path itself breaks before the .add(), so a
break-bounded slice would drop the record half.
Verified the new slice checks fail on a dedupe-neutered factory (a
headless-Chrome harness driving the real createCoordinatorPane confirms
that neutering produces two rendered nodes for one event id; intact code
renders one, and the no-event-id legacy path still renders both).
The touch_structured_memories facade and both storage backends were
implemented but had zero call sites, so access_count never moved and
last_accessed never advanced past write time on any deployment.
Wire two touch points:
- proactive composition touches the injected top-k (post-rerank) set,
deduped per turn since _init_system_messages recomposes many times
within a single turn;
- the memory tool's search and get reads touch their returned rows,
counted per call. save/delete/list do not touch.
Touches are best-effort through the facade, which already swallows
storage errors, so a failed touch never breaks composition or a tool
call.
* docs: 1.6.0 changelog — roll up the 1.5→1.6 line for stable
Replaces [Unreleased] with the 1.6.0 section: 320 main-only commits
since the stable/1.5 divergence grouped into theme bullets (license,
trajectory/migration-060, web search, rerank/memory, approvals/judge,
L-shell, shelf, SSE, providers, cluster ops, security). Breaking
changes aggregated up top; migration-060 backup callout reshaped from
discussion #631 for the stable audience.
* docs: add the stable/1.6 track to the changelog preamble
* docs: retire the stable/1.4 track — current + one prior policy
Changelog preamble down to three tracks with the policy stated;
1.4 retirement noted in the 1.6.0 Removed section (final release
v1.4.0; tags/artifacts remain, BUSL-1.1 as shipped). releasing.md
track table, policy bullet, and examples brought up to the 1.6.0
promote cycle — the doc was still describing the 1.4-stable era.
PR #652 review follow-ups:
- not_found snapshot entries now carry the full key set (updated/name
empty) so results[ws_id] is shape-uniform across states; pinned by a
key-set assertion in the sentinel test
- ws_ids param text now distinguishes malformed (fails before any
waiting) from well-formed-but-unobservable (first-tick abort) at
unchanged length — per-param descriptions stay lean by policy
Field incident: the coordinator LLM hand-copied a child ws_id and
collapsed its aaa run to a, producing a 30-char id. inspect said "not
found", wait called it "denied", neither offered recovery, and the model
concluded the child was dead and dropped the lane — silent report
degradation while the child kept working.
- validate model-supplied ws_id args at the tool boundary
(send/close/cancel/delete/inspect/wait): full 32-hex ids pass through
at unchanged storage cost; a child's exact legacy id still resolves;
anything else fails fast with a did-you-mean (capped Levenshtein <=3
over the coord's own children) plus a child roster. Near-misses never
auto-resolve; display names are not addresses (mutable, non-unique) —
a name ref errors with a pointer at the right id
- wait_for_workstream: rename per-entry state "denied" -> "not_found"
with an honest sentinel; malformed refs error before any waiting
(invalid_ws_ids); a well-formed id that is foreign, missing, or
hard-deleted mid-wait aborts the wait on the tick that observes it
instead of burning the timeout (mode=all was unsatisfiable) or riding
along to complete=True (silent lane loss); mode=all completes only
when every id is real-terminal; entries carry the child display name
- one not-found payload across all verbs: foreign and nonexistent stay
byte-identical (no existence oracle), hints reference only the coord's
own children, echoed refs clipped in error strings; invalid_ws_ids and
not_found share one per-ref shape with the roster hoisted top-level
- inspect ownership now requires user_id parity via _row_in_own_subtree,
matching the wait/mutating gates (#506) — closes the forged-parent
cross-tenant read
- session exec serializes the structured recovery payload (results +
did_you_mean + children) on unresolvable-id wait errors instead of
collapsing to the bare error string
- tool JSON descriptions + coordinator docs updated to the new contract;
incident regression test pins the captured aaa-collapse ids
* chore: relicense BUSL-1.1 -> Apache 2.0 for 1.6.0
Flips every license artifact in the tree; 1.5.x and earlier remain
BUSL-1.1 per their release-time LICENSE files. Contributor consent
record: #548 (rationale: #546).
- LICENSE: canonical Apache 2.0 text
- NOTICE: new; copyright line + pointer to THIRD-PARTY-NOTICES
- pyproject.toml: SPDX expression + explicit license-files trio
- Dockerfile: COPY the license trio (hatchling needs them at build)
- THIRD-PARTY-NOTICES: BUSL line reworded; bundled-version drift
fixed (KaTeX 0.17.0, Mermaid 11.15.0, hls.js 1.6.16)
- README badge + License section, CONTRIBUTING inbound-license line,
TS SDK package(+lock), example pyproject
- docs/pgbouncer.md: drop stray ':' introduced in #353
* docs: add CONTRIBUTORS.md
* chore: drop LICENSE leading blank line
The apache.org LICENSE-2.0.txt begins with a newline; the SPDX
canonical text and GitHub license templates do not. Use the
conventional form — detection is whitespace-normalized either way.
Independent re-implementation of the --skip-permissions argparse flag,
OR-ed with the tools.skip_permissions config-store setting at both
consumption sites. Written from the flag's pre-existing spec (the
--help epilog and compose.yaml, which referenced it before #450
existed).
Replaces reverted #450 so that 1.6.0 ships no non-consented
contributions under Apache 2.0. Provenance record in #548.
Copilot round: the 3px literal (carried from the hatch .seg segments)
disagreed with the shared :focus-visible rule, which restates
border-radius as var(--r-sm) — so the corner radius popped on keyboard
focus. One token, no jump.
The dashboard launcher's Coordinator|Interactive radiogroup styled its
active option as a neutral panel highlight — two faint text links that
said nothing about WHAT was being chosen. The active option now takes
the kind vocabulary the rest of the shell already speaks (.ptag.coord
amber / .ptag.int cyan): 15% kind tint, kind-colored label, and a kind
LED dot, in a recessed .seg-style track. Colour is never alone — the
LED + label weight carry the state, and the JS contract (classList
toggle on .active, aria-checked, roving tabindex) is untouched.
Designer-review round on the scroll fix found the harness's dialog-tier
gate silently green: confirm-dialog (and install/coord-delete) markup
lives OUTSIDE #admin-layout, so the fragment extraction never embedded
it — ?open=confirm threw at showConfirmModal and screenshot a normal,
dialog-less page. build() now injects every hatch dialog the fragment
does not already contain, and a driven ?open= that ends with no open
dialog stamps OPEN-FAILED-<state> into the title instead of passing.
Also upstreams the review's probe states: &focuslast=1 focuses the last
shelf-body control (the displaced-dock regression class — only .sh-body
may scroll; head/foot must stay pinned) and &scrolled=bottom shows the
24px scroll tail.
The L-shell height-pins the admin chain and .hatch-host clipped it, so no
box below the pane could scroll: tabs taller than the pane were cut dead,
and the overflow:hidden host doubled as a hidden scroll container that
focus-into-view silently scrolled — visually-hidden toggle/cap/radio
inputs escape the .sh-body scroller (abspos under an unpositioned label),
overhang the shelf, and a Tab keypress shoved the docked hatch off its
head with no scrollbar to recover by.
- .admin-content becomes the manage pane's interior scroller (the #main
precedent); switchAdminTab resets it on real tab changes only
- .hatch-host: overflow hidden -> clip — paint clipping without a scroll
container, so focus can never displace the dock
- position:relative anchors on the three hidden-input labels
(toggle-switch, .sh-body .cap, segmented-option); .settings-toggle
already carried one
- livepass: the console harness wraps the fragment in the REAL L-shell
chain (its bespoke height pin is exactly how this bug class stayed
invisible to the screenshot gates) and gains a ?tall=1/&scrolled=1
scroll state
The wiring-lint test used percent-formatted regex patterns — UP031 under
the ruff 0.15.6 the CI pre-commit pins (the older venv binary let it
through; checked repo-wide against the exact pin now). f-strings with
doubled quantifier braces, plus one over-long fixture line in the
livepass generator split.
Copilot threads, both validated rather than blindly applied:
- closeShelf's scrim-ownership scan now skips detached entries. The
thread's throw scenario doesn't occur on the real removal path (a pane
close detaches an ANCESTOR, so _hostOf still resolves inside the
detached subtree) — but a detached shelf is genuinely not a scrim
owner, so the guard is correct beyond being defensive.
- toast.js drops the popover attribute via removeAttribute instead of
the null assignment. The claim that null leaves popover="null" is
refuted — the IDL is nullable and null removes the attribute (verified
empirically in headless Chrome) — but removeAttribute reads correct
without requiring that spec knowledge.
The livepass harness — the headless-render rig that verified every
converted modal surface and click-drives submits (the dead-Save bug
class) — lived as ad hoc files in /tmp and got wiped once already.
The durable piece is the GENERATOR: the markup is extracted fresh from
the index files at build time (a committed snapshot would drift) and
the stylesheets/scripts are symlinked so edits are live on refresh.
scripts/livepass.py builds both harnesses into /tmp/livepass/ (ui:
all six dialog-tier surfaces incl. the real cards.js batch controller
drive; console: the admin-pane fragment hosting the shelves, with
schedule/model/policy/confirm/token fixtures and the model-save click
drive that flips document.title to PUT-OK-<n>). --serve included;
the chrome screenshot incantation and the ?open= registry are in the
module docstring. Governance fixtures (roles/HR/OGP/memory/skill) are
documented seams for when those surfaces need driving.
The models conversion dropped the legacy onclick= from the submit button
and wired detect/recalibrate/capgrid/thinking in the boot IIFE — but never
the submit itself. submitCreateModel existed with nothing calling it: Save
clicked dead with no error, exactly what a live test surfaced. None of the
gates could see it — the markup lint checks anatomy not wiring, the review
finders verified the submit function's internals, and the livepass renders
never clicked Save.
Audited every id-bearing button inside every dialog.hatch across both apps
for click wiring: model-create-submit was the only true positive (the
batch confirm buttons wire through cards.js's $() prefix helper and
new-ws-submit wires 106 lines from its getElementById — audit false
alarms). The new test_every_hatch_button_is_wired pins the class: direct
getElementById wiring or wiring through the assigned variable, with the
two prefix-built cards.js ids allowlisted; verified it fails against the
pre-fix tree. Livepass now drives the actual click: Save → busy → one PUT
→ shelf closes + toast.
Sixteen-finder review (4 dimensions x 4 subsystem slices) + adversarial
verify: 16/16 findings confirmed, all fixed.
Majors:
- The schedule preview (and create/update via _compute_next_run) 500'd on
syntactically-valid-but-impossible cron dates: croniter.is_valid passes
'0 0 30 2 *' but get_next raises CroniterBadDateError. One _next_cron_runs
helper now owns construction + the guard for both paths; the preview
answers its 200/valid:false contract, and next[] is one shape (the cron
branch now carries the UTC offset the 'at' branch always had).
- The batch-delete results view tore down (exit delete mode, refresh the
stale list) only via the footer Close — header ✕ / Escape / backdrop left
deleted rows on screen and the mode stuck. The teardown moved onto the
dialog's onClose (gated by a resultsShown flag so a pre-delete cancel
keeps the selection), and Close just closes.
- The model shelf's Server-compatibility section was permanently invisible:
the one hidden-attr element still toggled via style.display, which cannot
beat .hatch [hidden] !important — openai-compatible operators could never
reach server type / API surface / extra-body. Now .hidden like its
siblings.
Minors: shelves prune detached entries when a pane closes mid-edit (state
map + Escape-listener leak); the capabilities autofill gains the
_schPreviewSeq stale-response guard; the rename dialog focuses its input
before select() (select() does not move focus per spec — Enter landed on
the ✕); .mcp-install-source-label becomes the fourth protected label
component; nine write-only shelf-handle vars dropped; one alert region
gets one name; orphaned .modal-col-heading CSS, the stale toast z-index
rationale, a dangling divider comment, and a comment chasing the renamed
_submitRoleShelf all cleaned. Regression tests pin the Feb-30 preview,
the create-path guard, and the uniform next[] shape.
Dual review of the dialog-tier work (primed + cold), adjudicated:
- Menu-launched dialogs lost focus return: openPopupMenu's close() removed
the focused item before the action ran, so dialogs captured <body> as
their opener and the close-restore no-op'd. The menu now hands focus to
its return target before invoking the action — repairs every
menu→dialog flow in both apps.
- Rename targeted the wrong workstream: submitEditTitle re-read the
ACTIVE pane's id, so renaming a background tab via its context menu
renamed whichever pane was focused. The dialog now pins its target at
open (pre-existing bug, carried from the legacy overlay).
- Batch-failure rows printed raw HTML error pages verbatim (proxy 502s,
gateway timeouts): bodies are stripped of style/script content and
markup before display, with an HTTP-status fallback.
- The two single confirms behaved differently in flight — delete-ws
closed optimistically while revoke held under the busy lock. Unified on
hold-open-with-busy: failures keep the user's context for retry.
- Batch results: failures announce through the live sh-alert (previously
dead markup), a clean run flips the chrome to the success kind (red
head over '3 deleted, 0 failed' disagreed with the de-dangered foot),
and focus lands on Close after the state swap.
- Ghost-button boundaries measured ~1.2:1 on the foot strip (WCAG
1.4.11): ink-mix borders routed through a variable so kind variants
keep their own border colors; reduced-motion busy gains a static ' …'
cue; dark label-hint opacity brought above the compound 4.5:1 line.
- Revoke voice unified ('Revoke connection', no '?'); batch alertdialogs
gain aria-describedby; dead _settingsTrap machinery removed.
The revoke confirm is no longer a role=dialog/aria-modal overlay with
page-local CSS — it's a hatch dialog-tier alertdialog whose chrome lives
in /shared/hatch.css. The markup pin follows the new shape and the
stylesheet pin list drops the retired #revoke-mcp-overlay rule.
The shared cards.js multi-select controller serves the ui Saved
Workstreams AND the console Saved Coordinators, so the builder and its
two host markups convert as one unit: #ws-delete-dialog and
#coord-delete-dialog are md danger dialogs whose list renders inside the
sh-body (the only scroll region — the 200px inner list cap dies).
Foot grammar: [N selected meta] [Cancel data-close, autofocus] [red
filled "Delete N workstreams"] — the count moves out of the body prose
into the meta and the action label. The fan-out brackets with setBusy
(LED pulse + action lock replace the disabled/'Deleting...' swap); the
results view swaps the action to a neutral Close and hides Cancel (a
Cancel beside a Close is the redundant dismissal pair the foot grammar
forbids). The dormant error region becomes the sh-alert.
The controller's hand-rolled focus trap, prevFocus bookkeeping and
overlay display toggles die — hatch.js owns trap/Escape/backdrop/focus
restore; the post-results Close still hands focus to the section toggle
the bar collapse just rebuilt. The ws-delete-modal-* CSS family leaves
cards.css (only the row treatment survives); the dead window wrappers
(cancelWsDelete/confirmWsDelete + coord twins) and the ui keydown
handler's last legacy-overlay branch go with it.
The ui app has no admin pane host and new-ws is a launcher invokable from
anywhere, so every surface lands on the document-modal DIALOG tier (no
shelves). hatch.css gains the one md dialog width (560) the v1 design
specified; the ui index links hatch.css/hatch.js alongside the other
shared assets.
- New workstream: md create dialog, WS-NEW plate; the fork path keeps its
title/semantics (WS-FORK plate, skill + attach rows hidden via the
hidden attribute). Submit brackets with setBusy; errors land in the
sh-alert with the scroll-into-view rule.
- Rename: a styled prompt() — single field, Enter submits, no plate.
- Delete-workstream + revoke-MCP mirror the console confirm exactly:
danger chrome, prose body, Cancel autofocus, red filled action.
The four hand-rolled focus traps, the Escape/overlay-click dispatch and
the body-overflow lock die (native dialog + hatch.js own all of it); the
global-shortcut handler defers on dialog:modal instead of the overlay-ID
list. The forked legacy modal CSS leaves style.css (~270 lines). The
markup-shape lint now scans both index files, the asset assertion and the
parse-time-bridge guard extend to the ui app.
Cosmetic/consistency findings from the dual shelf review, adjudicated;
accepted items:
- One designation-plate grammar: 2-4 char domain code + closed suffix
vocabulary. SKILL-GH → SKL-IMPORT, PP-* → PPO-*, ROLE-MAP → USR-ROLES,
ROLE-* → ROL-*; HR-/OGP- stay (established judge domain terms).
- Create titles say "New <thing>" (Add model / Add MCP server renamed;
"New prompt" was creating a prompt POLICY); "Edit model" gains the
"— {alias}" suffix every other edit surface carries.
- Shelf primaries uniform single-word Create/Save — the skill shelf's
"Create skill" / "Save config" / "Save changes" collapse.
- label-hint dialect normalized to bare lowercase (wrapping parens
dropped); the em-dash unit form "— UTC" is a different species, kept.
- Placeholder-only format instructions promoted to label-hints (a11y
3.3.2): HR confidence "0.0–1.0", intent "supports {arg_snippet}". The
install dialog's dynamic required asterisk pairs with required on the
control + aria-hidden on the glyph.
- The mcp and schedule adjacent toggle pairs wrap in toggle-stack like
the model shelf; the sch-enabled-row hidden toggle rides inside.
- memory-detail foot grammar: Delete demoted from err-filled to a quiet
destructive text action (.sh-btn--quiet-danger, doubled class so it
beats the later-loaded hatch.css quiet color), Close stays rightmost.
- First-focus: user-roles lands on its first role toggle once rendered,
channel on the type select, builtin-role edit on the first enabled
control, mcp-install autofocuses its primary.
- The watch-cancel confirm's action reads "Stop watch" — no more
Cancel-beside-Cancel.
- A locked skill opens as data-kind=inspect (cyan read-out chrome) and
flips to edit on the in-place unlock re-render.
- mcp-detail healthy node dot pairs with a dim "connected" text token,
mirroring the error-text sibling (state was color-alone).
- etm-scan section converts to the hidden attribute (last
style.display straggler on the skill shelf).
Dual design review of the shelf stack (one primed on intent, one cold)
adjudicated; accepted findings:
- The busy lock held the door for the mouse only: Enter on the focused
primary re-fired submits, scrim clicks closed a shelf mid-flight, and
the dialog tier's native Escape (cancel event) dismissed a busy confirm.
One capture-phase guard + a scrim busy check + cancel interception close
all three for every surface; setBusy now announces aria-busy. Contract
assertions added to the busy test (it previously claimed scrim coverage
it didn't assert).
- The MCP auth radio cards were destroyed by the .sh-body label cadence —
the exact specificity war the toggle-switch/cap exceptions guard
against, missed for .segmented-option. Restated at 0,2,1.
- Light-theme off-state toggle tracks measured ~1.2:1 (WCAG 1.4.11 wants
3:1): real ink fill on light, recessed look kept on dark.
- Failed submits on tall shelves rendered out of view — _showModalError
scrolls the alert into view. user-roles gains the missing busy bracket
and in-shelf errors instead of toast-only.
- Toasts render under the dialog tier's top layer: promote the toast to a
manual popover only while a modal dialog is open (popovers stack above
later dialogs); attribute dropped after so the everyday fade survives.
- Ambient glow now follows the kind accent (cyan/red surfaces no longer
bloom amber); skill editor's content column pins sticky so the SKILL.md
pane stays visible while the meta column scrolls; MCP-detail stacking
re-keyed from viewport to @container pane; dark-body micro-text and the
origin badge brought up to the contrast floor; quiet foot actions
(Validate regex, Detect) lifted from body-copy gray.
With the popover and a shelf both open, Escape closed both at once
(admin.js's popover listener and hatch.js's shelf listener each fired).
The popover listener registers at parse time — always ahead of hatch.js's
first openShelf — so stopImmediatePropagation makes Escape peel one layer
at a time: popover first, shelf on the next press.
With the skill editor converted, nothing renders through the legacy
overlay system any more. Grep-driven deletion:
- admin.js: _modalFocusTrap/_installTrap/_removeTrap, the govOverlays
dispatch table, and the global Escape handler die. The handler's one
still-live block — closing an open settings-help popover — is
extracted into its own small keydown listener (the settings panels
and the shelf form help buttons share that component).
- governance.js: the orphaned template trap/trigger let declarations.
- style.css: the .admin-modal box rules (incl. -wide/-skill), every
.admin-modal-prefixed half of the doubled toggle/segmented/perm/
user-roles selectors (the unscoped twins keep serving the settings
panels and shelf bodies), .admin-details (the shelf uses
details.rawhatch), .modal-buttons/.modal-cancel/.modal-submit,
.modal-section-divider, hr.toggle-group-divider, the now-empty
#…-template-overlay ID rule, and their reduced-motion entries.
Stale comment pointers re-aim at the live rules (.sh-alert,
.sh-body label).
Kept with reason: .modal-columns/.modal-col* — _openMcpDetail still
builds the MCP-detail shelf body with them (their narrow-viewport
stacking rules survive in a rebuilt media block).
Livepass re-verified after the teardown: zero console errors across
create/edit/locked, and the unlock-confirm-over-shelf capture is
pixel-identical to the pre-teardown one.
The last legacy modal pair (create-template + edit-template, ~430 lines
per mode) collapses into one pane-scoped 920px shelf: a hidden skl-id
decides POST vs PUT, the duplicated ctm-/etm- field sets merge into one
skl-/sklc- set, and the skill-spec two-column grid transplants whole into
the scrolling body (content-area rule re-scoped under .sh-body so the
shelf's font:inherit/min-height cadence doesn't flatten it; the mobile
breakpoint becomes a pane container query to match the shelf's own
bottom-sheet degradation).
Everything judgment-bearing carries over: paste-to-parse (now cancelled
via the shelf's onClose so Escape/scrim dismissals abort the inflight
parse too), live {{variable}} detection (re-dressed as match-strip chips
under the textarea, count mirrored into the foot meta when provenance
isn't occupying it), pending-resource rows in create, server-backed
resources + security scan + re-scan in edit, and the runtime-config
field set that stays editable on readonly skills (must keep matching
SKILL_RUNTIME_CONFIG_FIELDS). The readonly lock affordance moves into
the head strip left of the designation plate as a ghost icon button;
origin/locked provenance renders in the foot meta lane (installed/
customized chip + source URL + 'locked — unlock to edit'); the unlock
confirm stacks above the shelf via the native top layer and the
post-unlock re-render mutates the open shelf in place. Progressive
disclosure keeps its <details> semantics on rawhatch chrome, with the
shared-DOM state leaks (disabled spec fields, expanded details) re-armed
on every create open.
Mode-exclusive blocks keep their ctm-/etm- ids (pending vs server
resources, scan) — only true duplicates merged. Inline onclick wiring
moves to a one-shot _skillShelfWire; submits go busy via the shelf LED
lock. The legacy overlay markup and the .skill-lock-btn / .skill-vars-*
rules are deleted; trap machinery teardown follows separately.
Livepass-verified (stubbed authFetch, headless Chrome): create, edit
populated, locked view (disabled spec + editable runtime + scan +
readonly resources), unlock confirm stacked over the shelf, paste
auto-fill, variable chips, pending-resource add/remove.
The memory-detail read-out leaves its body-level overlay for a pane-scoped
inspect shelf (#memory-detail-shelf, cyan, lg) inside #admin-layout. The
detail-grid + content population carries over verbatim; the foot keeps the
Delete action (now a danger sh-btn) wired after the record loads, beside Close.
showMemoryDetailModal/hideMemoryDetailModal keep their names — the row renderer
calls them — and run through window.TurnstoneHatch; the post-delete close check
keys off the dialog's .open. Legacy overlay markup, its _installTrap/Escape
dispatch entries, the trap/trigger lets, and its style.css overlay ID-list row
are deleted.
The three governance create+edit pairs collapse onto pane-scoped lg shelves
inside #admin-layout. Each merges into ONE shelf with a hidden id (plus a
builtin flag on the judge surfaces) deciding the write: PUT for a DB row, an
override-POST for a built-in's first edit, a plain POST for a new row.
Title/tag/data-kind/submit-label flip between create and edit; edit-only chrome
(the prompt-policy Enabled toggle) sits in a hidden-toggled row.
The heuristic-rule and output-guard editors lift out of the Judge tab panel to
sit as direct children of the hatch-host (the shelf's inert containment needs
that); the two built-in-edit entry points share a single populate-and-open
helper. The settings-help-popover buttons carry over verbatim — their document-
delegated toggle is independent of the container. The output-guard "Validate
regex" button moves to the foot as a quiet action while its result strip stays
in the body. Submits go busy via the shelf LED lock; the regex flags input and
the is-credential toggle move onto the field grid. Legacy overlays, their
_installTrap/Escape dispatch entries, the trap-handler/trigger lets, and their
rows in the style.css overlay ID list are deleted.
The governance role surfaces leave their legacy overlays. Create + edit role
collapse into ONE pane-scoped shelf (#role-shelf inside #admin-layout): a hidden
role-id decides POST vs PUT, and title/tag/data-kind/submit-label flip between
"New role"/ROLE-NEW/create/Create and "Edit role — name"/ROLE-EDIT/edit/Save.
The slug-name row is create-only (hidden attr on edit); the display name carries
over and is disabled for builtin rows. The permission checkbox grid renders with
one "role" prefix for both modes — the builtin baseline-vs-rendered diff that
produces {grant, revoke} (and round-trips unknown perms untouched) is preserved
verbatim. Submit goes busy via the shelf LED lock instead of the disable dance.
User-roles becomes an edit shelf carrying its toggle-list population; github-
import a create shelf with the URL as an sh-mono field and the hint folded into
a label-hint span. The public show/hide/submit names the toolbars and row
renderers call are kept — only the bodies are rewired through window.TurnstoneHatch
(handler-time, never at parse time). Legacy overlays, their _installTrap/Escape
dispatch entries, the trap-handler/trigger lets, and their rows in the style.css
overlay ID list are deleted.
The four MCP surfaces leave their legacy overlays. The add/edit server editor
collapses into ONE lg pane-scoped shelf (#mcp-shelf): a hidden mcp-edit-id
decides POST vs PUT, and the title/tag/data-kind/submit-label flip between
"Add MCP server"/MCP-NEW/create/Create and "Edit MCP server — name"/MCP-EDIT/
edit/Save. The transport-conditional stdio/http field groups and the OAuth
subfield block toggle on the hidden attribute instead of style.display (which
.hatch [hidden] enforces); the multitenant-auth segmented radio control carries
over verbatim. The transport/auth onchange and submit move out of inline markup
into _mcpWire (which also installs the audience-autofill listener once).
mcp-import becomes a create shelf with the JSON paste as an sh-mono textarea;
mcp-detail an inspect shelf carrying its setSafeHtml two-column population. The
registry install flow moves onto the document-modal dialog tier per the confirm
precedent: a STATIC #mcp-install-dialog whose summary/source/fields containers
_showInstallMcpModal still populates — the dynamic overlay-shell construction is
gone. _doRegistryInstall is shared by the one-click card path and the dialog
submit, so its busy lock and inline-vs-toast error branch now key off the
dialog's .open. Submits go busy via the LED lock; legacy trap/Escape/ID-list
entries and the trap-handler lets are deleted.
The four admin.js leaf surfaces follow the schedules pilot onto pane-scoped
shelves inside #admin-layout: create-user, create-token, link-channel and the
read-only schedule run history. Each keeps the public show/hide/submit names the
toolbars and row renderers already call (showCreateUserModal, showScheduleRuns,
…) — only the bodies are rewired. Open/close run through window.TurnstoneHatch
(handler-time, never at parse time); submit goes busy via the shelf LED lock
instead of the disable/relabel button dance; errors land in the sh-alert.
The token shelf hands its issued secret to the already-converted token-created
dialog unchanged. Schedule runs is an inspect shelf (cyan, lg width, Close-only
foot) and carries its setSafeHtml run-table population verbatim. The channel
type→placeholder onchange moves out of inline markup into _channelWire. Legacy
overlays, their _installTrap/Escape dispatch entries, the trap-handler lets, and
their rows in the style.css overlay ID list are deleted.
Tool policies join the shelf: one pane-scoped editor for create+edit with a
priority-neighbor read-out computed from the loaded policy list ('evaluates
after deny-rm (900) · before default-ask (0)' — policies run highest-first),
so where a priority lands answers itself while typing. Live tool-pattern
match chips are deferred until a cluster tool-registry endpoint exists.
The reusable confirm and the show-once token dialog move onto the
document-modal hatch tier (native showModal): the confirm keeps its
showConfirmModal(title, message, actionLabel, callback) contract for all
14 call sites, gains the danger chrome (red LED/hairline/title + err-filled
action), and deliberately moves autofocus from the action button to Cancel
— Enter on a fresh destructive confirm no longer fires the action. The
token dialog gets the success chrome + show-once callout, with copy wired
to the primary. Nested confirm-over-shelf now stacks via the top layer;
the z-index 650 special case and four more overlay-ID/dispatch entries die.
Also fixes a real war the livepass caught: display rules on form chrome
(label.toggle-switch's inline-flex) defeat the hidden attribute — hatch
containers now enforce [hidden] with display:none !important.
The model editor moves onto a lg shelf and the hand-written capabilities
JSON requirement dies. Nine LED tiles (tools/streaming/vision/web-search/
temperature/effort/STT/TTS/reranker) display merge(dataclass defaults,
known-model table, explicit overrides) with SPARSE-OVERRIDE persistence:
only keys saved in the row or toggled by the operator are written back, so
known models keep tracking future capability-table updates instead of
being pinned at save time. The known-model lookup that previously dumped
the whole table into the textarea becomes the tile BASELINE refresher with
a provenance banner ('Loaded from the built-in table for X'); the raw JSON
survives as a collapsed advanced hatch holding everything the tiles don't
manage (thinking_display, max_output_tokens, …). supports_rerank is now a
tile — no more hand-JSON to flag a reranker — and drives the Re-calibrate
button + calibration chip (now a foot read-out next to quiet Detect).
Everything regression-prone carries over: server_compat extraction,
write-only api_key sentinel, reranker calibration field re-merge (raw-typed
keys still win), thinking-mode representability guard, detect/calibrate
flows. Inline onclick/onchange wiring moves to the boot IIFE; busy locks
the shelf LED instead of disabling the button. Legacy overlay markup, the
trap/Escape dispatch entries, and the CSS ID-list entry are deleted.
Livepass-verified (stubbed authFetch, headless Chrome): create + edit,
baseline banner, tile extraction, toggle-switch inside .sh-body.
First production surface on the service-hatch shelf. Create + edit collapse
into ONE pane-scoped dialog (#schedule-shelf inside #admin-layout, now the
.hatch-host): a segmented Runs control (Daily/Weekly/Monthly/Interval/Once/
Cron) compiles to schedule_type/cron_expr/at_time — nobody types cron unless
they choose Cron mode, which keeps the raw input as the escape hatch with
the same live read-out. The NEXT RUNS read-out previews the next three
firings through the server's croniter (debounced POST /schedules/preview),
and the foot strip always shows the compiled expression for verification.
Storage is untouched: on edit the saved expression is reverse-parsed back
into the friendly mode when its shape matches (_cronToScheduleMode), else
the editor opens in Cron mode. Notify-row/select-populate helpers carry
over verbatim; submit goes busy via the shelf LED lock instead of button
text swapping. The legacy create/edit overlays, their show/hide/toggle
globals, hand-rolled trap wiring, and their entries in the overlay ID list
and _installTrap/Escape dispatch tables are deleted (-242 lines of markup).
Toggle-switch gets its .sh-body exception in hatch.css (the same
specificity war .admin-modal fights), + .sh-mono utility.
Verified against a stubbed-authFetch livepass harness in headless Chrome:
create/edit/light/busy, weekly reverse-parse round-trip, preview rendering.
One chrome vocabulary (machined head/foot strips on --code-bg, kind LED,
designation plate, center-fading hairline), two mounting points:
- .hatch--shelf: pane-scoped, NON-modal (dialog.show()). Mounts inside the
pane's .hatch-host, docks right, dims only that pane via a lazy sibling
.pane-scrim, and contains focus with inert on the pane's other children —
rail/tabs/other panes stay live. Split panes work by construction; the
bottom-sheet degradation is an @container query on the pane, not a
viewport media query, so a narrow split degrades too. Controller-owned
Escape defers to any document-modal dialog stacked above.
- .hatch--dialog: document-modal (showModal()) confirm/show-once tier;
top layer stacks it above any shelf with no z-index ladder.
Smart-input primitives ship alongside (seg, chips, readout, capgrid,
match-strip, rawhatch, autofill) for the Phase-1 surfaces. data-busy locks
the container while a submit is in flight (LED pulses, dismissal refused).
Light-theme micro-text gets the .tab-menu-key one-step-up contrast pass.
Classic scripts reach the ESM controller via the window.TurnstoneHatch
bridge (toast.js pattern, handler-time only); invariants pinned in
tests/test_hatch_js.py incl. a markup-shape lint over dialog.hatch.
POST /v1/api/admin/schedules/preview validates {schedule_type, cron_expr,
at_time} with the same _validate_schedule_fields the CRUD path uses and
returns the next three croniter firings. Pure compute, no storage touch;
invalid input answers 200 {valid:false, error} because the schedule
editor renders it live while the user types. Registered ahead of the
{task_id} routes so the literal segment wins.
Review feedback: the cancel-path docstrings described undone items as
degrading to 'heuristic fallback verdicts', but the emitted and
persisted tier is llm_fallback (heuristic content relabeled). Aligned
all eight occurrences — including the pre-existing _deliver_fallbacks
docstring — so docs, logs, and audit rows use one vocabulary.
Review follow-up: the ON CONFLICT rationale lived verbatim in three
places (protocol docstring + both backend comments). Keep the prose in
the protocol — the contract's home — and point the backends at it.
Also recommend cancel_on_approval=true in docs for deployments where
the judge shares one local inference backend with the session model.
judge.cancel_on_approval=False (the default) promises the daemon
evaluates every tool call to completion so all verdicts are available
for later review. Two sites conspired to break that: the approval
gate's finally set the cancel event unconditionally the moment a
decision landed, and _evaluate_single's poll loop honors the event
regardless of config — so every item the sequential judge hadn't
reached degraded to a heuristic llm_fallback row. On a 22-call
parallel batch, approving after the third verdict silently downgraded
the other 19; the elaborate late-verdict machinery in
on_intent_verdict was effectively dead code.
Make the event a pure abort signal whose firing policy lives with the
caller: the gate fires it only when cancel_on_approval is enabled,
while generation supersede (next batch) and close() keep firing it
unconditionally, bounding a stale daemon to one batch of real work.
_run_judge drops its own config second-guessing — a fired event always
fast-forwards the remainder to fallbacks (every call still gets
exactly one verdict), and the fallback reason no longer claims 'user
approval' for supersede/close aborts.
The async judge daemon can UPSERT a fallback row — reusing a heuristic
verdict_id from the batch approve_tools is about to bulk-insert —
before the bulk write runs. With a plain INSERT, that single PK
collision aborted the entire statement, and the caller's best-effort
try/except silently discarded every heuristic row in the batch.
Insert ON CONFLICT (verdict_id) DO NOTHING on both backends: siblings
survive a mid-batch collision, and the colliding row keeps the
daemon's llm_fallback tier upgrade instead of regressing to the
heuristic stamp (the documented preferred outcome). Regression test
runs against both storage backends via --storage-backend.
ChatSession._on_verdict guards on judge-generation identity so a stale
verdict can't ride a reused call_id into the Smart-Approvals cache —
but it dropped those verdicts entirely, before persistence. Every
ruling the sequential judge delivered after the next turn began left
intent_verdicts claiming the judge never answered.
Route superseded verdicts to a new persist-only hook
(SessionUIBase.on_superseded_intent_verdict): the row lands with
user_decision="superseded" while every live surface stays untouched
(no SSE, no replay cache, no pending-decision park). The hook is
duck-typed; display-only UIs (CLI/eval) don't define it and keep the
plain drop. upsert_intent_verdict already excludes user_decision from
its on-conflict SET, so a superseded fallback upgrading its heuristic
row in place cannot clobber a decision already stamped there.
The /history decoration layer suppressed intent-verdict rows with
risk_level="none" from the wire payload, on the assumption the client
filtered them anyway. It never did: buildConvVerdict renders a badge
for every verdict it receives, so the live SSE path painted all judge
verdicts while rehydration silently dropped the benign majority — a
22-call parallel batch came back from a restart showing only the 3
flagged calls.
Ship every stored row and let the client render replay exactly as it
rendered the live stream. The output-guard chip pair (showOutputWarning
+ merge-on-clean) already suppresses consistently on BOTH sides and is
unchanged.
attempts < 1 made init() return successfully without fetching CA or
cert — a silent no-op leaving the client uninitialized. Fail fast with
ValueError instead; negative base_delay rejected on the same guard.
A whole-stack restart races every node against the console for the CA
fetch (compose re-enforces depends_on ordering only on `up`): losers
logged one warning and served plain HTTP for their lifetime, while
winners served mTLS that the plain-HTTP container healthcheck could
never probe — leaving "healthy" plaintext nodes and "unhealthy"
working ones.
- TLSClient.init() grows attempts/base_delay retry (server passes 6
attempts, ~31 s backoff) absorbing the boot race; per-attempt CA-fetch
failures log warning + debug traceback instead of error tracebacks.
- healthcheck.py falls back to HTTPS when the plain probe fails,
presenting the node's own cert as the client cert with the cluster CA
pinned; dials localhost because the internal CA issues DNS SANs only.
Default plain-HTTP deployments are unchanged.
- The server writes boot PEMs under a fixed root (TURNSTONE_TLS_PEM_DIR,
default <tmpdir>/turnstone-tls) so the probe can find them; boot
clears stale dirs and refuses a symlinked/foreign-owned root; renewal
rewrites the PEM dir so the probe's client cert never outlives the
served cert.
- /health reports tls: "active"|"fallback" (absent when TLS is
disabled) so a silently downgraded node is observable.
Review feedback (Copilot), both confirmed against source:
- openPopupMenu: when no menu item has focus (a click on a separator or
the menu surface moves focus off the items without closing), ArrowUp's
unguarded modulo landed on the second-to-last item ((-1-1+n)%n == n-2).
Guarded to enter at the bottom; ArrowDown's (-1+1)%n already entered at
the top. Pre-existing in the tab dropdown this helper was extracted
from — the shared chrome means one fix covers both menus.
- The burger and the [+] tail were focusable non-tab children inside the
element PaneManager stamps role=tablist (the [+] violation pre-existed;
the burger doubled it). The tabs now live in their own .tabstrip, which
becomes the tablist PaneManager owns; burger and tail sit outside it in
.tabbar. The strip is also the mobile horizontal scroller, so burger +
[+] stay pinned while tabs scroll.
Verified: 289 static-suite tests, 28 harness self-tests, and both
real-page boot harnesses green; mobile render unchanged.
test_renderer_js drives renderer.js behaviorally through node via
vm.runInThisContext — script semantics, which choke on the import/
export syntax renderer.js and utils.js now carry (all 68 tests failed
at harness setup). The harness now evaluates _demodulize()d source:
imports drop (the shared vm context resolves cross-file bindings as
globals, exactly like the pre-module classic scripts) and export
keywords peel off.
Deliberately NOT switched to dynamic import(): the mermaid harness
pokes renderer-internal state (_mermaidState = 'ready') that script
evaluation exposes but a real module would encapsulate. Module
semantics are covered by test_shell_js's .mjs parse sweep; these
tests pin renderer behavior.
Review follow-ups: the new rail collapse and mobile drawer had no
committed guards (the repo pattern is per-step string assertions in
test_shell_js.py) and openPopupMenu — now load-bearing for both the
tab dropdown and the footer user menu — was unpinned.
- test_rail_collapse_glyph_strip: persistence key, toggle +
aria-controls, class-flip seam, cpill-label/manage-glyph companions,
52px desktop-scoped CSS.
- test_mobile_drawer_off_canvas: burger, scrim, rail-open flip,
pane-activation auto-close, off-canvas translateX + visibility:hidden.
- test_popup_menu_shared_helper: the export + both consumers (the user
menu's prefer-up path included).
- shell.css: the 769/768 media blocks are a matched pair CSS cannot
express as a shared token — both now carry a cross-referencing
change-both comment.
Review finding (critical): the ESM migration made cards.js a deferred
module, but both classic app.js bundles built their saved-list tables at
TOP LEVEL — const COORD_COLUMNS = [SavedColumns.name(), ...] and
const _coordTable/_wsTable = createSavedTable({...}) execute at parse
time, before the window bridges exist. ReferenceError aborted each
bundle before it could define TS_APP.boot, so neither deployment booted.
(The earlier consumer audit caught bare top-level CALLS and IIFE bodies
but excluded declarations — missing initializers with side effects.)
Construction moves into _initSavedCoordTable()/_initSavedWsTable(),
called first from each boot path (substrate modules have evaluated by
then). The two typeof-undefined guards become null-checks (a let
binding passes typeof). Verified end-to-end with real-page load
harnesses: both index.html script chains (real app/admin/governance +
module substrate, network mocked) boot to a mounted shell with zero
uncaught errors — console over loopback HTTP (the coordinator dynamic
import needs real URL resolution), standalone over file://.
Dead-code removal, all provable (no JS creator / no reachable caller):
- interactive.js drops the !this._embedded branches: focus tracking, the
right-click context menu, and the header with split/close buttons all
referenced shell globals (setFocusedPane, splitPane, splitRoot,
showPaneContextMenu, countLeaves, closePane) that exist nowhere since
the step-6 fork collapse — reaching them was a guaranteed
ReferenceError. The embedded flag goes with them (every pane is
L-shell-hosted; pane--embedded is now unconditional), as do the
call-less updateWsName() and the host adapter's getWsName seam.
- ui/static/style.css drops the orphaned split-pane/tab-bar vocabulary:
.ws-tab*, #new-tab-btn, #split-btn, .split-handle, the pane-header/
action-button block, the unused dropdown-in keyframe, and the dead
entries in the reduced-motion list.
- ui/static/app.js drops the retired settings-gear menu remnants (state
vars for a builder that no longer exists + an always-false Escape
guard) and fixes a real crash: hideNewWsModal() focused the removed
#new-tab-btn unguarded, throwing a TypeError on every create/fork
modal close; focus now returns to the shell's [+] new-session button.
- pane.js exports openPopupMenu — items, positioning (flip + clamp),
dismissal, aria-expanded mirroring, and arrow-key roving in one place.
The tab-action dropdown delegates to it, and shell.js's footer user
menu replaces its hand-rolled duplicate (which lacked Tab-close and
arrow roving — it inherits both).
test_interactive_pane_js.py: the embedded-gate pins flip to retired-
symbol pins (the gate is gone, not gated).
utils/toast/kb/cards/auth/renderer/composer/composer_attachments/
composer_queue/status_bar convert from classic scripts (implicit globals,
IIFE wrappers) to ES modules with explicit exports. Parse-time
cross-dependencies become real imports (auth/kb/cards -> utils,
auth/cards -> toast, cards -> auth, renderer -> utils), which deletes the
implicit script-order contract those files relied on. utils stays
import-free (bottom of the graph); its two upward calls (setMarkdown ->
renderer, export -> toast/auth) late-bind through window at call time to
avoid import cycles.
Each module installs a transitional window bridge for the still-classic
bundles (console app/admin/governance, ui app, inline onclick=), which
only touch the globals at boot/event time — verified by a column-0 /
IIFE-body audit of all four consumers, and including the audit-missed
initLogin() that both app.js boot paths call. theme.js stays classic:
deferring it would flash the wrong theme before first paint. Vendored
katex/hljs/mermaid stay classic and lazily typeof-guarded.
interactive.js and shell.js drop their bare-global reads for real imports
(authFetch, showToast, Composer, StatusBar, queue/attachment controllers,
streaming renderer, setMarkdown). The three HTML entries load the
substrate as module tags (same positions, same version_html stamping);
classic admin/governance/app still parse first, modules evaluate before
shell.js calls TS_APP.boot().
Tests: auth/kb/utils move from test_app_js's classic node-check sweep to
test_shell_js's module-semantics sweep, which now covers all 15 shared
modules (sink scan excludes renderer.js, the sanctioned HTML producer;
the no-var ratchet covers the var-free subset). The const-reassign guard
re-includes the converted files plus the shell modules.
The L-shell rail gains its two deferred responsive modes:
- Desktop collapse (user preference, localStorage turnstone_interface.rail):
the rail shrinks to a 52px glyph-only strip — live Tier-1 state glyphs
remain the navigation, cluster pills stack as glyph+count, Manage becomes
one gear row opening the Admin pane, children flatten to peer glyphs.
Title attrs (now set unconditionally) carry the names; aria-labels were
already complete.
- Mobile drawer (max-width 768px): the rail leaves the grid and overlays
off-canvas at full width behind a scrim. Burger in the tab bar opens it
(focus moves into the rail); Escape (focus returns), scrim tap, or any
pane activation closes it. Closed drawer is visibility:hidden so its
buttons leave the Tab order. The collapse preference lies dormant here.
- Tab titles render in an ellipsizing span capped at 240px (48vw mobile)
instead of growing the tab unbounded; tab bar scrolls horizontally on
mobile.
A dead interactive controller's base goes stale once its node loses or
re-homes the ws, but menuBase() returned it first — so the close/delete
404-as-success lanes could silently drop a tab whose session is alive on
the node it re-homed to. Mirror the revive path: when isDead(), lead with
the live Tier-1 node and fall back to the stale base only when the ws is
gone cluster-wide (its 404 then correctly reads as "already closed").
Two reported console bugs, one shared root: a pane can outlive its
session, and nothing brought the two back together.
Reconnect: an interactive pane whose stream died (ws closed/evicted
elsewhere, node restart, re-home) could never reconnect while its tab
existed — openPane() on an existing pane was focus-only, the
controller's connect() is one-shot, and its 5s recovery loop re-dialed
the SAME node forever (infinite 404 polling through the console proxy).
The only workaround was closing the tab before resuming.
- createInteractivePane now tracks terminal failure: 3 consecutive
CLOSED recovery beats -> give up (stream closed, timers + any pending
history load invalidated, status bar "Disconnected", opts.onDead
fired once). host.onStreamOpen (new hook) resets the counter;
isDead()/markDead()/base join the controller surface; onLogin
ignores a dead controller — revive owns recovery, so a deliberately
closed session is never resurrected by a timer.
- PaneManager.openPane fires pane.onReopen(extra) when it targets an
ALREADY-OPEN pane — the explicit-intent signal (saved-list resume,
rail row, child link) that activate() can't carry (hooks no-op on the
active pane, and onActivate also fires on plain tab switches).
getPane() added for cross-cutting lifecycle signals.
- The shell paints a click-to-reconnect banner on give-up — and
immediately on Tier-1 ws_closed via the new
TS_SHELL.notifySessionClosed seam (the console keeps the tab, unlike
the standalone's auto-close, so the conversation stays readable).
Reopen/banner-click revives: tear down the dead controller,
re-resolve through the origin-first POST /open lane, rebuild. The
forced resolve skips BOTH beginConnect fast paths (a stale Tier-1 row
must not bypass /open) while a live node leads the hint chain (an
origin-first /open then reuses a genuinely-live session instead of
loading a duplicate on the old meta node). The standalone lane POSTs
its local /open on revive too — /events 404s on an unloaded ws.
- Coordinator parity: the factory exposes reconnect() (acts only on a
missing/CLOSED stream; OPEN is healthy, CONNECTING is already being
worked) and the pane's onReopen drives it — the saved-list resume
POSTs /open before openPane, so a fresh stream is all it needs.
Tab menu: a node-proxied interactive pane's dropdown gated every verb
on classic globals that only exist in ui/static/app.js, so the console
got a nearly-empty menu whose one surviving verb (Export) hit the
console origin and 404'd. convTabMenu gains a base-aware fallback lane:
verbs POST against the pane's OWN transport base (controller's exact
base -> persisted node hint -> live Tier-1 node; a verb is omitted
while no base is resolvable — never aimed at the wrong origin).
Close/Delete confirm first (window.confirm, the coordinator precedent)
and treat 404 as intent-satisfied (nothing left to stop/delete -> drop
the tab). exportWorkstreamDownload takes the base. The standalone
keeps its globals lane (incl. Fork) byte-identical, and an empty verb
section no longer renders a leading separator.
Verified: 189 JS-pin tests; two headless-Chrome live-DOM harnesses
driving the real modules — console 16/16 (connect -> ws_closed ->
banner -> reopen revives on a new node with the fresh hint -> give-up
stops retrying -> live-node-led resolve), standalone 10/10 (globals
menu intact, revive POSTs /open exactly once, no cluster resolve).
CI lock-check failed: the Fable 5 commit raised the anthropic floor to
>=0.108 in pyproject.toml but uv.lock still recorded >=0.39 / resolved
0.107.1. Regenerate the lock: anthropic 0.107.1 -> 0.108.0, specifier
0.39 -> 0.108 (no transitive changes).
Also address the Copilot review nit: the operator-instruction trust
declaration docstring wrote the fence marker as <system-reminder_<nonce>>;
align it to the emitted and project-standard <system-reminder_{nonce}>
notation.
- claude-fable-5 capability entry: 1M context / 128K output, adaptive
thinking (summarized display), effort low..max incl. xhigh, no
sampling params, web + tool search, vision, reasoning replay, native
mid-conversation system messages
- document the Fable 5 wire quirk at the capability table: an explicit
thinking={"type": "disabled"} is a 400 on this model; the adaptive
branch never emits "disabled", so adaptive-or-omitted is preserved
- widen the native mid-conversation-system comments from opus-4-8-only
to opus-4-8 + fable-5 (protocol, provider, tool_advisory, prompts,
session)
- raise the anthropic SDK floor 0.39 -> 0.108: 0.39 predates every
named kwarg the provider sends (output_config 0.77, top-level
cache_control 0.83, mid-conversation system blocks 0.105); 0.108
adds claude-fable-5
- tests: capability assertions for claude-fable-5 + dated-variant
prefix match
The pattern attribute on the MCP server-name and model-alias inputs used an unescaped hyphen in its character class. Browsers compile the HTML pattern attribute with the RegExp `v` flag, under which a literal `-` must be escaped — the class failed to compile, so the browser silently dropped the constraint and disabled client-side validation (Firefox). Escaping the hyphen leaves the matched set unchanged and consistent with the server-side ^[a-zA-Z0-9._-]+$ validators.
Add an inline-SVG data: URI favicon to the console, coordinator, and ui entry points so page loads no longer 404 on /favicon.ico. A data URI needs no new static route and survives the /node/{id} proxy path rewrite.
All findings validated against source before fixing; behaviour-preserving:
- coordinator.js: drop the unused `stripAnsi` import; `updateStatusBar` early-returns
on a null evt, so `(evt && evt.effort)` is simplified to `evt.effort` (no
redundant guard).
- pane.js `_onTablistKeydown`: drop the dead `let j = i` initial value — every
branch reassigns j before `tabs[j]` is read (the no-match case returns first).
- rail.js: collapse the redundant `ws.parent_ws_id ? "interactive" : "interactive"`
ternary to `ws.kind || "interactive"`. Tier-1 always stamps `kind`
(console/static/app.js defaults it to "interactive"), so the fallback was dead
and the parent-based arm would have mis-tagged a standalone interactive — the
single default mirrors the snapshot's own and is behaviour-identical.
- status_bar.js: correct the stale JSDoc — it's a THREE-cell bar now (tokens /
tools / turns); the model cell moved to the composer chip.
- ui/static/index.html: the tool-approval `a` shortcut help read "Always approve";
align it to the button language "Approve all".
Multi-stage review (find → verify → sanity) of b8914854 found one critical
bug plus four minor + one nit; all confirmed against source and fixed:
- CRITICAL — the interactive launcher's "Specific node" pick was unusable:
selecting a node fired the composer `change` event → onChange →
_applyLauncherFields → _populateLauncherNodes → setOptionChoices, which
rebuilds the <select> and reset it to the placeholder, wiping the selection
the instant it was made (submit then failed "Choose a node…"). Fix:
_populateLauncherNodes snapshots the current pick before the rebuild and
restores it after (setOptionValue does not dispatch `change`, so no loop).
- perf — every interactive open blocked first paint on a POST /open round-trip,
even on the hot rail / active-row paths where the ws is already live. The
pane now connects DIRECTLY when the Tier-1 snapshot already names the owning
node; only the dormant / reload case (snapshot empty) resolves + opens. This
resolves the uniform-vs-gated /open question left open last change; refresh
safety is unchanged (a reload activates before the snapshot lands → nodeForWs
null → resolve path).
- bug — an errored resolve (capacity / no node free) had no in-place retry
(re-clicking the active tab is a no-op); the error status line is now
click-to-retry.
- quality — resolveInteractiveNode surfaced each failure twice (toast + in-pane
line) with drifted wording; dropped the toasts, the in-pane status line is the
single source of truth.
- quality — corrected a setOptionFieldVisible comment that cited a nonexistent
"flex/grid rule" (the row is `display: contents`).
- nit — buildController skips the redundant sessionStorage re-persist when the
resolved node already matches the persisted hint.
Guard tests extended (bug-1 capture/restore, the live-direct path); the
behavioral harnesses were strengthened to fire a real selection rebuild and to
exercise the live-direct vs reload-resolve split that the first round missed.
Workstream-lifecycle bugfixes on the L-shell:
- Node-proxied interactive panes now SURVIVE a browser reload. On first
activate a pane resolves its owning node and (re)opens the session there
before streaming — the node /events stream 404s on a ws not loaded on its
node, so a rehydrated pane could not just connect blind. Resolution is
origin-first via the new TS_APP.resolveInteractiveNode seam (POST /open with
a rendezvous /route fallback). PaneManager now persists a pane's resolved
nodeId as opaque meta and hands it back on rehydrate, so a reload restores
the pane onto the SAME node even before the Tier-1 snapshot has populated —
the exact timing that used to strand it on base="" (the console, not a node).
- Both launcher personas open the new session as a PANE, not a full-page nav
(coordinator -> coordinator pane; interactive -> node-proxied pane); the
full-page nav stays only as the shell-absent fallback. Every interactive
entry point (create, active row, rail, saved row, child link, reload) now
funnels through one resolve-open-connect path, folding away the bespoke
restoreInteractiveSession helper.
- The interactive launcher gains a node-selection strategy (Least loaded |
Specific node, with a live node picker fed from the cluster snapshot) and a
persona-aware task hint — the shared composer no longer shows
"...coordinator orchestrate?" when the interactive persona is selected.
Guards updated to pin the new wiring; the stale console landing test (asserting
the renovation-retired bottom-bar node picker) is corrected to the rail.
The parallel tool-batch kicker strings ("Parallel · N tools", "Evaluating ·
Parallel N", "Running · Parallel N", "⚠ Approval · Parallel N") and the "1/N"
index label were duplicated byte-for-byte across interactive's three render paths
and the coordinator's kicker state machine, with no shared source enforcing the
visual parity the two surfaces require. Extract batchKicker(state, n) +
indexLabel(idx, n) into conversation.js (both files already import it) and route
all 13 sites through them, so a future label tweak can't silently diverge them.
Byte-identical, harness-verified: the rendered kicker + 1/N labels are unchanged.
From the multi-stage review of this session's changes (all minor):
- bug-1: the footer user-menu's deferred document-listener attach now bails if the
menu was already closed (closeUserMenu nulls the cleanup ref), closing a latent
listener-leak window.
- perf-1: fold paintConvTabGlyphs + paintConvTabTitles into one paintConvTabs — a
single findWs per stateful tab per Tier-1 render instead of two scans.
- q-2: remove the dead StatusBar.paint modelEl branch + its modelInfo arg (both
callers dropped it when the model moved to the composer chip) and the orphaned
.ws-sb-model CSS rules.
(q-1, the parallel-head string-helper extraction, follows separately.)
After the stacked composer box, the action row's margin-left:auto on the send
button pushed send to the right edge but left the mic stranded on the left next
to the model chip (the mic inserts before send). When the STT role is confirmed
the action row now gets .has-mic, which puts margin-left:auto on the MIC instead
so mic + send form a right-aligned cluster (send flush after the mic); without
STT, send keeps its own auto margin and sits alone on the right.
Verified (headless): with .has-mic the mic moves from x=427 (by the model chip)
to x=906 (38px left of send at 944); css audit at baseline 10.
Move the interactive + coordinator composers to the mock's layout: a compact
rounded composer box with the borderless textarea on top and the
[+] / model·effort chip / send row below (layout:"stacked"). The bordered inner
textarea and the boxed paperclip are gone — the box is the frame, and the attach
is a plain "+" glyph. Scoped via a composer--chat class (the model-chip hosts)
so the home launcher's taller stacked composer is untouched.
Verified by headless render against the mock; css audit at baseline 10, guards
green.
After the collapse fix, parallel tool calls render but the head read
"TOOL web_fetch + 1 more", which implies the rest are hidden. Match the
coordinator's presentation across all three interactive paths (announce /
inline / replay) + buildToolDiv:
- "Parallel · N tools" kicker (renders "PARALLEL · N TOOLS") instead of "Tool"
- the conv-batch--parallel class (the numbered-row connecting rail)
- per-row "1/N" index labels
so the "+ N more" summary now reads as a label, not hidden calls.
Verified in the real console shell (headless): a 2-call batch shows kicker
"Parallel · 2 tools", rows "1/2"/"2/2", conv-batch--parallel, both calls named,
no JS errors.
The real regression behind "parallel tool calls don't show / tool cards get
overwritten, leaving a thin stripe with a coloured pixel on the left": the
.conv-* convergence put an `overflow:hidden` card (.conv-batch) into the
interactive pane's SCROLLING flex-column message list
(.pane--embedded .pane-messages, overflow-y:auto). An overflow:hidden flex
item's `min-height:auto` resolves to 0, so flexbox squished the tool batch to
~2px (just its left border) once the column filled — while plain .msg blocks
(overflow visible) kept their height. That asymmetry is why the COORDINATOR
(different container) and main's old `.ts-approval` block (a .msg, overflow
visible) were never hit, and why it impacted ALL models — it was never a
local-model id-collision (that earlier theory + fix were reverted).
Fix: pin every message child to flex-shrink:0 so the column scrolls instead of
collapsing cards. Reproduced + verified in the real console shell (headless):
the parallel-bash tool batch went from 2px (collapsed) to 244px (full content)
once a multi-turn conversation fills the column. Guarded in test_conversation_css.
Per the BRIEFING the composer is the sole model location, but the model still
rendered in the per-pane status bar. Add a display-only "model · effort" chip to
the chat composer (next to the attach button) and remove the status-bar model
cell from both the interactive and coordinator panes; the status bar keeps
tokens/tools/turns. The chip repaints from the same model + status events, with
effort silent on the implicit "medium"/none (mirroring the status bar's old
suffix rule). A per-session model/effort PICKER is a separate deferred task
(needs a backend override path).
Verified in a real interactive pane (headless): the chip renders with its em-dash
placeholder, the status-bar model cell is gone (tokens/tools/turns remain), the
send glyph is intact, no JS errors; JS guards green, css audit at baseline 10.
The embedded message list stacked a 5px flex gap ON TOP of each .msg turn box's
4px margin-bottom (~9px of dead space between segments — "too thick"). Trim the
container gap to 2px so segments land at a compact ~6px, matching the mock's
gap-only intent without overriding the shared .msg margin (no specificity-audit
flip).
whoami returned only user_id (an opaque uuid), so the rail footer rendered the
uuid. whoami now resolves the user record by id and returns the human
username/display_name (best-effort — a storage miss just omits it). The client
stores data.username (no fallback to user_id: a uuid is worse than the generic
"account" placeholder). Hardened against a malformed user record (isinstance
dict guard) so a bad row can't 500 whoami; test stubs get_user + asserts the
display name is surfaced.
Local OpenAI-compatible models (e.g. DeepSeek) reuse tool-call ids across turns
(call_0, call_1 each turn). The interactive pane resolved a call's card via a
PANE-WIDE first-match messagesEl.querySelector('[data-call-id=...]'), so a later
turn's tool_result / verdict / warning / output-chunk landed on an EARLIER turn's
card — corrupting it and leaving the current batch's rows empty (a thin stripe).
It also made parallel calls look like they "didn't show" (the head summary sat
over emptied rows).
The pane is strictly serial, so a live result belongs to the MOST-RECENT matching
card. Add a _lastMatch(root, selector) helper and resolve the five live-path
lookups (appendToolOutput x2, appendToolOutputChunk, showOutputWarning,
updateVerdictBadge) to the LAST match instead of the first. The replay/history
path was already scoped to its block and is untouched.
Known limitation: if a model emits ALL parallel calls in ONE turn sharing the
same id, they still collide within the batch — a separate source-level issue.
The chat composers (coordinator + interactive) now render an up-arrow send glyph
instead of the text label, matching the mock. Opt-in via opts.sendGlyph so
creation-form composers keep their text label; the visible glyph is constant
while the textual sendLabel stays the aria-label and drives setBusy's a11y
rotation (setBusy no longer overwrites the glyph).
Verified in a real interactive pane (headless): the send button is the up-arrow
glyph with aria-label "Send message", no JS errors.
Conversational tabs froze at their open-time title — wsTitle(id), which is the
id-slice when the session isn't in the Tier-1 snapshot yet (e.g. a just-restored
saved session) — and never updated, so the tab read as the raw id instead of the
saved name.
Add PaneManager.setTabTitle(paneId, text) (mirrors setTabGlyph: rewrites the tab's
title text node in place + pane.title for a later rebuild) and a
paintConvTabTitles(pm) Tier-1 hook alongside the glyph repaint. It only UPGRADES
a tab to a real name (ws.name || ws.title) — never flickers a known name back to
the id if the ws blips out of a single frame.
Verified in the real console shell (headless): a named ws shows the name at open,
a dormant ws shows the id-slice then upgrades on repaint, no JS errors.
Mock-review batch (4 items):
- Footer: drop the redundant Admin button — Manage already surfaces every
admin tab, so only the theme toggle relocates from the retired header.
- Footer: the user chip now shows the real logged-in user. whoami returns
user_id but _storePermissions only persisted permissions, so the chip was
stuck on the "account" placeholder; it is now stored as ts.username and the
chip repaints once whoami lands (Tier-1 render hook).
- Footer: Log out moves into a click-menu on the user chip (reuses the
.tab-menu popup chrome; the item clicks the hidden #logout-btn so auth.js
stays the single owner of logout and its in-flight-refresh race guards).
- Manage: groups start collapsed instead of auto-expanding the first one —
the rail is a discovery map, not a wall of open links.
Verified end-to-end in the real console shell (headless): chip is a button
showing the user, no admin button in the footer, the menu opens with Log out
which invokes logout, outside-click/Escape close it, no JS errors.
Saved INTERACTIVE sessions opened from the console did nothing useful.
Coordinators rehydrate because their activation POSTs /open first; the
interactive branch only passed the saved DTO's node_id to openPane and
bailed "Session node unknown" when falsy. Even with a node_id nothing
streamed: the per-pane SSE /events 404s on a not-loaded ws and /history
alone does not rehydrate, so the session was never loaded onto a node.
New restoreInteractiveSession (console app.js) is ORIGIN-FIRST: POST /open
to the session's origin node (the DTO node_id, stamped at create) and pin
the pane there. This keeps node affinity and — load-bearing — REUSES a
session already live on its origin instead of loading a duplicate copy
elsewhere; the interactive pane talks directly to /node/{id} for every
verb, so the load-node and the pane-node must match (no split-brain).
Only when the origin is gone (POST /open 404 = not in registry / 502 =
unreachable) do we re-home onto a fresh rendezvous node via
GET /v1/api/route (the router skips dead nodes; persistence is shared
ws_id-keyed Postgres, so any live node is state-safe). Capacity (429) and
permission (403) are surfaced, not silently re-homed. No origin (legacy/
CLI rows) routes straight away. Mirrors the coordinator open-before-
navigate and the standalone dashboardResumeSession; the active-row path
(already-loaded sessions) is untouched.
Bugs surfaced by the live console (the headless harnesses stubbed data, so these
only showed against a real cluster):
- Saved-session AND active/filtered-table row clicks did full-page nav (interactive
-> /node/{node}/?ws_id=, coordinator -> /coordinator/{ws}) instead of opening an
L-shell tab. Route both through window.TS_SHELL.panes.openPane (interactive =
node-proxied pane, coordinator = coordinator pane). Full-page nav stays only as
the shell-absent fallback. (The broader ?ws_id= URL-pattern cleanup is deferred to
its own session.)
- The cluster health pills wrapped ("idle" fell to a second line) in the 266px rail.
Tightened gaps + font + nowrap so all three fit one line (verified at 266px).
- The [+] new-session button was a no-op when the Dashboard was already active
(showHome focuses it, no visible change). It now also focuses the launcher
composer via a new TS_APP.focusLauncher seam — "new session" lands you ready to
type.
- The cluster node list wasn't collapsible (unlike the Manage groups). The "Nodes"
header is now a toggle (button + rotating caret), state persisted across the
rail's Tier-1 re-renders.
Verified: node + prettier clean; CSS audit baseline; 89 JS guards; cluster-pill +
node-collapse render harnesses (pills one-line, toggle hides/shows + caret rotates);
wiring harnesses errs:[] (no regression).
Three shell-spine P3s from the review:
- Consolidated the three near-identical Tier-1 ws-scans (wsTitle / nodeForWs /
stateForWs each re-walked getClusterState -> nodes -> workstreams) into one
findWs(wsId, skipConsole) helper + three thin wrappers. nodeForWs keeps the
console-pseudo-node skip (coordinators live there, must not be node-proxied);
the other two scan all nodes. Also restored the convTabMenu doc comment that an
earlier glyph-helper insertion had orphaned above stateForWs.
- Simplified PaneManager.activate's dead/misleading guard (_activeId===paneId ||
!has, with a nested re-check) to the equivalent `if (!has) return;`.
- rehydrate now counts a pane restored only when openPane actually returns one — an
auth-gated (denied) coordinator pane returned null but still set restored=true,
which would suppress the Dashboard fallback into a blank shell (latent today: the
non-closable Dashboard is always in the persisted set).
Verified: 28 shell guards; mechanism harness 31/31 (activate/rehydrate/gate); both
wiring harnesses errs:[] (the scans drive the verified tab titles / node-proxy /
state glyphs — console running/idle, standalone full menu). node + prettier clean.
Three console front-door P3s from the review:
- Removed dead module state _lastOverviewJson / _lastNodePickerJson (memo caches
for the removed renderStatusBar / renderNodePicker; only declared, never read).
- Dropped the unreachable popstate view==="admin" branch — Admin is a rehydrated
PaneManager pane now, nothing pushes {view:"admin"}, and Back-from-admin already
lands on the dashboard via the home/filtered path.
- _createInteractive: added an else for a 200 without target_node so a server-
contract drift surfaces an error instead of silently stranding the user (the
branch is currently unreachable — the node is validated non-empty server-side —
but it was a silent failure mode).
Verified: node + prettier clean; 28 shell guards; console wiring harness errs:[].
getFocusedPane() has been a permanent `null` stub since the fork collapse
(PaneManager owns focus; interactive.js owns approval keys). That left ~90 LOC of
unreachable pane-dependent code — and it's exactly where the P1 closeTabDropdown
crash hid. Removed all 4 sites:
- the global-keydown Escape-cancel branch + the whole inline-approval keybinding
block (still referencing the retired .ts-approval-feedback / .verdict-* vocab);
every LIVE shortcut (Escape->dashboard, Ctrl+D/T/1-9, Ctrl+Shift+E/F/X, Ctrl+W)
is kept;
- the dashboardSubmit optimistic-echo block (interactive.js echoes its own turn);
- the new-ws modal model prefill (curModel is always "");
- the stub itself.
Also fixed _formatAttachSize: called 4x (chip size + over-cap error) but defined
nowhere -> a ReferenceError that broke file staging (pre-existing on main; flagged
by the review). Defined a local B/KB/MB formatter mirroring composer_attachments's
IIFE-local formatSize.
Verified: node + prettier clean; 61 app guards; standalone harness errs:[]; all
live keyboard-shortcut verbs asserted present after the splice.
The header removal (5e.2e) left the wait_for_workstream progress surface inert:
_waitIndicatorEl() mounts only into the deleted #coord-header and returns null, so
the whole #14 wait-indicator (handleWaitStarted/Progress/Ended, the activeWaits
Map, _renderWaitIndicator, the reconnect-path clear, the 3 SSE switch cases) ran
but rendered nothing. Ripped it (~110 LOC) + the orphaned .coord-wait-indicator
CSS rule. (The observability loss was a deliberate, tested design decision —
test_coordinator_page.py pins the header absence.)
The review's broader coord-chrome dead-CSS list was a false positive on
verification: `.task-row .status-done/.status-blocked` are LIVE (applied via a
dynamic `"status-" + status` class), `ts-spin` is live (coord-chrome.css:209), and
the rest (.coord-tool-*/.judging/.feed-item/.topbar) appear only in prose comments.
Also (review P3-1): clear aria-busy in _unsetBatchRunningIfAllResults so a batch
that completes via tool_result only (judge + gate bypassed, early-paint on) stops
announcing "busy" to screen readers after completion.
Verified: node + prettier clean; 16 coordinator guards; CSS audit baseline;
coord-keys harness all green (approval keys intact after the rip).
Two lower-severity findings from the full-branch review:
- Backend doc divergence (session_routes.py + console/server.py): the comments
justifying interactive's `state=None` saved listing claimed "the storage layer
already excludes state='deleted' tombstones" — but neither storage impl has such
a filter. It's incidentally safe because delete is a HARD delete (no `deleted`
tombstone is ever written), NOT because of a filter. Corrected both comments to
state the real mechanism + flag that a future soft-delete tombstone would need an
explicit `state != 'deleted'` guard here.
- Dead CSS: the entire #cluster-status-bar / .csb-* block (387 lines) was orphaned
— its HTML element + all JS writers were deleted earlier in this branch and
nothing reuses the vocabulary (grep-confirmed across console+shared). Removed the
main block (per-selector verified all-csb before splicing); the 9 residual csb
rules inside two MIXED @media blocks are left as a safe over-keep, matching the
5e.2f dead-CSS method.
Verified: ruff clean; CSS audit at baseline (zero new flips); braces balanced;
prettier clean; console builds errs:[].
The full-branch pre-push review (6 subsystem slices) found 1 P1 + several real P2
bugs; the confirmed functional ones, fixed here:
- P1 (standalone): two LIVE keydown branches called the deleted closeTabDropdown()
-> ReferenceError that silently killed Ctrl+Shift+E/F/X (edit/fork/delete) and
Ctrl+W (close) before reaching the verb. Removed the dead calls.
- coordinator (this branch's step-7 keys): the deny branch fired on any `d` with no
modifier guard, so Cmd+D (bookmark) / Ctrl+D / Alt+D silently DENIED a pending
batch. Early-return on ctrl/meta/alt (Shift+A still resolves).
- shell.js: window.TS_LOGIN was defined AFTER rehydrate(), so a RESTORED
conversational pane silently skipped its re-auth Tier-2 reconnect (onActivate saw
no TS_LOGIN and never re-fired). Moved the fan-out (+ TS_SHELL) above rehydrate.
- shell.js: TS_LOGIN.subscribe had no unsubscribe -> a closed pane leaked its
controller closure across open/close/re-login. Added unsubscribe + call it in
both onClose hooks.
- standalone dashboard: updateTabIndicator dropped its `extra` arg in the fork
collapse, so a watched row's STATE/TOKENS/CTX went stale on every ws_state tick
until a full reload. Ported the in-place row patch from main (sans the retired
.ws-tab indicator).
Also dropped a redundant index.html NODE-gate comment (the CSS rule + loadDashboard
already document it) that had tipped a fragile 4000-char structural guard.
Verified: 105 JS guards green; mechanism (31/31) + coord-keys (all) + wiring
harnesses errs:[]; node + prettier clean.
The merge-gate designer pass on the CONSOLE persona (coordinator + interactive,
caps on) found 0 P1 — ship-quality — and 4 small CSS P2s, applied here:
- P2-1 (non-optional AA): the pending approval card's "APPROVAL NEEDED" kicker was
4.17:1 in light (sub-AA on the operator's primary decision signal). Darken off
raw --warn via a theme-tracking color-mix toward --ink-2 (~5.5:1 light; dark
stays warm + passing). In conversation.css, so both personas benefit.
- P2-2: rail micro-labels (.sec-label/.nlabel/.node-row .ver/.grp-head .gcount)
were --ink-4 <=11px = ~3.9-4.0:1 in light. Light-scoped --ink-3 (~6.5:1), the
same escape hatch as .tab-menu-key; drift versions keep --yellow.
- P2-3: the tab-dropdown separator was imperceptible. Use the MORE-visible hairline
per theme (--hair-2 dark / --hair light) — the designer's suggested tokens were
reversed; corrected against the actual hex values.
- P2-4: the relocated "Reconnecting..." rail-conn read as an alarming top-of-rail
peer of Cluster health. Quiet it (10px, collapses when connected) + --warn (not
the near-error --yellow) on disconnect.
- P3-5: the Manage active-tab marker (inset --hair-2) nearly vanished in light ->
--ink-4 (reads in both themes, still not the amber `.open` of a live session).
Verified: CSS audit at baseline (zero new flips); 33 conversation/shell guards
green; node + prettier clean. Other P3s noted (model-chip disclosure is the locked
"model lives only in the composer" decision; node-row rhythm / verdict-expand minor).
The settings MODAL backdrop (#settings-overlay) died when MCP connections moved to
the Manage > Connections pane (step 6) — the #settings-mcp-* content rules are
reused there, but the overlay wrapper is gone. Remove the closed loop of dead-but-
mutually-alive references: the CSS rule, the stale "settings-overlay" modal-id
array entry, and the guarded getElementById no-op in the settings-close path.
The other fork-collapse dead-code (the getFocusedPane stub + its null-gated
branches, the partially-retired settings-gear) is woven into still-live handlers —
deferred to the merge-gate /review for a systematic sweep with the review findings.
Verified: 0 settings-overlay refs remain; node + prettier clean; CSS audit at
baseline; standalone harness errs:[].
The console twin of the interactive.js approval-key fix: the coordinator's
tool-batch card shows kbd hints (Enter approve / D deny / Shift+A approve-all) but
they did nothing — the keys were never wired (the standalone routed approval keys
through the app.js global keydown + getFocusedPane, retired in the fork collapse).
Add a pane-owned keydown on `root` that resolves the current pending batch:
- _currentPendingBatch() finds the last .conv-batch with a still-pending
[data-needs-approval="1"] row whose actions aren't already disabled — the
in-flight double-fire guard (a second key during the resolve is a no-op).
- Enter -> approve, D/Esc -> deny, Shift+A -> approve-all, routed to the existing
_resolveBatchAction path.
- A focus guard skips when an input/textarea/contenteditable is focused, so the
keys never hijack composer typing (the coordinator has no feedback field, unlike
interactive, so no feedback special-case).
Verified end-to-end against the real coord pane (keydown -> _currentPendingBatch ->
_resolveBatchAction -> approveWorkstream -> postJSON -> authFetch, stubbed at the
HTTP boundary): Enter/D/Shift+A fire the right verb, the double-fire + focus guards
hold, errs:[] + a coordinator JS guard. Live keypress confirm rides the merge gate.
The WORKSTREAMS table showed a NODE column (a multi-node console-ism) on the
single-node standalone server, where every row reads "local". Drop it: remove the
NODE header span + skip the node cell in loadDashboard, and gate just that table to
6 columns by overriding the --dash-grid VARIABLE (not the grid-template-columns
property — so it stays the var's single declaration, no cascade flip), scoped by
id, so the shared --dash-grid and the Saved Workstreams table keep their 7-col
layout. Matches the brief's capability-derived-affordances thesis (the rail drops
Cluster the same way). Designer P2 (pre-existing, not a step-6 regression).
Verified: standalone DOM shows 0 dash-col-node + the saved table intact; CSS audit
at baseline (zero new flips); node + prettier clean.
The brief defers mobile: the rail -> off-canvas drawer matches no current DS scope
(the DS is desktop-only; the console's mobile drawer was retired in step 3b).
Record the decision in-code at the .app layout seam (the brief is local-only) where
a future max-width @media would slot in. Verified narrow viewports (720px wide)
are cramped, not broken — no silent mobile-support claim.
openPane now auth-gates pane CREATION via an optional per-type canOpen predicate
(deny -> no pane; focusing an already-open pane is never re-gated). PaneManager
stays generic — it holds a _gates map and consults canOpen/onDeny; the shell
supplies the gate.
The coordinator type gates on the admin.coordinator scope — the SAME
sessionStorage-backed _hasCoordPermission helper the launcher + saved-list use.
Because every coordinator open path (rail click, child-link, rehydrate, [+]
launcher) routes through openPane, this gates them all at once — closing the gap
where a rail click opened a coordinator pane a user lacked scope for (it then
404'd server-side). Perms live in sessionStorage so they survive a refresh →
rehydrate gates correctly (an operator's persisted coord pane restores, a
non-operator's is skipped). The backend enforces the scope too; this just avoids
opening a doomed pane.
Verified: 31/31 mechanism harness (gate allow/deny/onDeny) + real-stack console
wiring (authorized operator opens the coord pane; a no-permission stub denies a
new coord pane, gateDenied:true, errs:[]) + a shell JS guard + CSS audit baseline.
The right-floated tabbar tail (empty since the scaffold) gets a [+] button that
focuses the persona launcher — the Dashboard pane hosts the unified
coordinator/interactive launcher, and a new session needs a task prompt, so "new
session" composes there. Cross-deployment via window.showHome (both the console
and standalone expose it) with a pm.openPane("dashboard") fallback; reuses the
scaffold's .tab-add styling. Auth stays the launcher's concern (it gates each
persona option), so focusing it is always safe.
Verified: renders in the real shell (standalone harness DOM) + a shell JS guard.
Conversational tabs now show a live shape+colour state glyph (● ◐ ⚠ ✗ ○) instead
of the static ◆/○ placeholders the header removal (5e.2e) left behind — driven by
the SAME Tier-1 source + builder the rail uses, so tab and rail always agree.
- rail.js: export the glyph() builder (one source of truth for the mapping).
- pane.js: ShellPane.stateful + PaneManager.setTabGlyph/statefulTabs — generic
(PaneManager owns no glyph vocabulary; the shell passes the built element).
A stateful pane builds no static glyph; the shell paints a live .ui-glyph.
- shell.js: stateForWs() reads the Tier-1 snapshot; paintConvTabGlyphs() repaints
every stateful tab on each Tier-1 render (subscribed to TS_APP.onRender) + per
pane on activate. Coordinator + interactive panes are now stateful.
- shell.css: .tab .tab-glyph spacing (static + live); live glyphs keep their own
.ui-glyph-* state colour (no .tab .glyph override).
SINGLE WRITER: the tab glyph is written only by the Tier-1 path (the pane's Tier-2
stream drives its body, not the tab) — no two-tier race, no stale open-time
placeholder on reconnect (BRIEFING L144-147). A coordinator-telemetry-parity gap
(open Q#2) would stale tab + rail equally, consistently.
Verified: 27/27 mechanism harness (8 new glyph asserts) + real-stack wiring
harnesses (console coord ui-glyph-running / int ui-glyph-idle; standalone int
ui-glyph-running — matching the stubbed Tier-1 state, errs:[]) + 26 shell JS
guards + CSS audit at baseline.
PaneManager tabs gain a caret opening a generic, keyboard-navigable action
dropdown — recovering the affordances the pane-header removal (5e.2e) dropped.
The mechanism is generic; the item set is pane-type AND deployment derived.
- pane.js: the caret (a <span>, not a nested <button>) + _openTabMenu/_closeTabMenu
— singleton, right-anchored under the caret with overflow flip + viewport clamp,
Arrow/Home/End/Esc/Tab nav, ContextMenu/Shift+F10 + right-click open.
- shell.css: the .tab-menu chrome promoted to the SHARED sheet (both deployments),
recovered from the retired .ws-tab-dropdown design but translated onto the DS
token vocabulary (--panel-2/--hair-2/--ink-*/--err).
- shell.js: convTabMenu wires each type by capability/feature-detection —
coordinator: Export · Close pane · Close workstream (its controller's
closeSession — the Export + end removed from its header land here)
standalone interactive: Refresh/Edit/Fork · Export · Close pane ·
Close workstream · Delete (classic ui/static globals)
console interactive: Export · Close pane (those globals are standalone-only)
admin: Close pane
Three-verb close is load-bearing: Close pane (drop tab) != Close workstream
(stop session) != Delete (destroy + unsave).
Designer-reviewed both personas, dark+light: resting danger cue on Delete (never
colour-alone), elevated --panel-2 surface, accent-wash hover, light key-hint AA,
viewport y-clamp + max-height.
Verified: 19/19 mechanism harness + real-stack wiring harnesses (all three menus,
errs:[]) + 25 shell JS guards + CSS audit at baseline (zero new flips).
The converged .conv-* card advertises y/n/a (+Enter/Esc) kbd hints, but the keys
did nothing in the L-shell: the only handler was the old standalone app.js global
keydown gated on getFocusedPane(), which the fork collapse stubbed to null — so
Approve/Deny/Approve-all were mouse-only (the chips over-promised), and that dead
block also queried the retired .ts-approval-feedback class.
Wire the keys pane-owned on this.el (every embedded L-shell pane), restoring the
pre-regression behavior + using the converged .conv-feedback: when a pending
approval is up, in the feedback field Enter approves (with feedback) / Esc denies
and other keys type; elsewhere y|Enter approve, n|Esc deny, a = approve-all. The
composer is disabled while pending, so the feedback field is the only typing
surface. This fixes both the standalone and the console interactive pane (shared
interactive.js); the coordinator pane's keys (console-only) are a separate
merge-gate item. Verified: node-check, the headless shell harness still builds
clean (errs:[]), 102 JS guards green incl. a new wiring guard. The live keypress
-> resolve confirm rides the owed live-backend pass.
Removes the structurally-dead CSS the L-shell superseded — 794 lines: the tab
bar (.ws-tab*, #tab-bar, #split-btn, .ws-tab-dropdown-*), the binary split-pane
machinery (.split-*, #split-root, .pane-ctx-*), the old approval/verdict card
(.ts-approval-*, .verdict-*, the judge spinner), the fixed .dashboard-overlay,
and the retired appbar/settings-overlay bits — plus their [data-theme=light]
overrides. The standalone now styles its conversation from the shared sheets
(chat/conversation/interactive.css); the dashboard table + saved list were
always shared (base/cards.css).
Method: a conservative token-diff — a rule is dropped only when EVERY selector's
class/id token is absent (word-boundary, comments stripped) from the standalone
runtime (index.html + every JS it loads, incl. the vendored hljs/katex/mermaid
so their runtime-built classes aren't mistaken for dead). Mixed/any-live rules
are kept verbatim (no reformatting), so ~50 dead-but-harmless rules that share a
generic token like `.active` survive — safe over-keep. The markdown / syntax /
math / diagram theme lives ONLY in this style.css (the shared sheets don't carry
it), so the hljs/katex/mermaid families are protected from removal.
Also fixes four dead tab-DOM pokes in app.js (editWorkstreamTitle /
confirmDeleteWorkstream read the title from the workstreams roster now, not the
retired .ws-tab .tab-name; the cancel handlers drop the gone .tab-chevron focus
restore).
Verified: braces balanced (370/370), the headless harness still builds clean
(errs:[]), git diff confirms zero live dashboard/render rules removed, and the
css_specificity_audit (manifest synced to the standalone's new sheet set) shows
the SAME 10 pre-existing findings before/after — zero new cascade flips (removing
a rule for a non-existent selector can't change any live element's cascade).
121 JS guards green, ruff/mypy clean.
The renovation META-GOAL: a standalone turnstone-server now serves the SAME
capability-parameterised L-shell the console serves (caps {cluster:false,
orchestration:false}), collapsing the console/static vs ui/static fork. No server
change was needed — turnstone/server.py already mounts ui/static at /static; this
changes what ui/static CONTAINS.
ui/static/index.html -> the L-shell skeleton: a hidden #header the shell
relocates (status -> rail, theme/logout -> footer), #main as the Dashboard pane
body (launcher + workstreams table + saved list), a one-panel #view-admin hosting
MCP connections (reusing the #settings-mcp-* table ids), the modals, and the caps
block flipped to {cluster:false, orchestration:false, brandSub:server}. The
split-pane chrome (#tab-bar/#split-root/#split-btn), admin.js/governance.js, and
the separate interactive.js module tag are gone (shell.js imports it).
ui/static/app.js -> a single-node TS_APP/TS_ADMIN/showHome provider (-1417 lines):
- TS_APP.{getClusterState, onRender, bucketByParent, boot}: getClusterState
synthesizes a one-node cluster from the flat /v1/api/events/global roster;
boot() is shell-driven (no parse-time auto-run).
- TS_ADMIN: a one-tab Manage IA (Extensions > Connections) whose openTab opens
the Admin pane + renders the MCP table — the floating settings gear is retired.
- The binary split-pane machinery (layout tree, splitPane/renderLayout, tab bar,
context menu, tab dropdown, STANDALONE_HOST, createPane, the gear menu) is
deleted; the keep surfaces (dashboard, global SSE, new-ws modal, MCP
consent/connections, health/theme/kb) are rewired onto PaneManager + the rail
(switchTab/renderTabBar/showDashboard become thin shims; sessions open as
interactive panes).
interactive.js -> the window.InteractivePane bridge is retired (the shell imports
the factory in both deployments; nothing reads the global anymore).
JS guards re-pointed to the L-shell reality (gear/split-pane/window-bridge guards).
126 JS guards green, ruff/mypy clean, node clean. Verified in a headless harness:
the standalone shell builds caps-off (rail = Workspaces + Manage > Connections, no
Cluster), the Dashboard pane adopts #main, TS_APP/TS_ADMIN wired, zero uncaught JS
errors. The owed merge-gate passes (designer both personas + live-backend +
/review) are unchanged.
Two changes so the SAME shell mounts on a standalone turnstone-server (step 6's
META-GOAL: collapse the console/static vs ui/static fork):
- The coordinator pane import is LAZY + gated on caps.orchestration. A static
`import ... from "/static/coordinator/coordinator.js"` 404s on a standalone
server (whose /static is ui/static, no coordinator file) and aborts the whole
shell module. It's now `await import()` inside mountShell, before rehydrate,
registered only when the deployment has orchestration (the console); a
persisted coordinator pane then degrades to a rehydrate skip. mountShell is
now async.
- The interactive pane's nodeId is gated on caps.cluster. Node-proxy transport
only exists in a cluster deployment; on a single-node standalone every session
is LOCAL, so nodeId stays null -> the pane uses base="" (no /node/<id> hop),
even though the synthesized one-node clusterState names a node.
Console behaviour is identical (orchestration:true -> the import runs + the
coordinator registers; cluster:true -> the nodeId ternary's true branch is the
original expression). Verified both personas build clean in a headless harness:
console = [Cluster, Workspaces, Manage] + coordinator registered, no errors;
standalone = caps off, no Cluster, no coordinator, no errors.
Same as the coordinator: in the L-shell the embedded interactive pane's header
content (workstream name + INTERACTIVE persona tag) is redundant — the tab shows
the name and the rail (Workspaces) shows name + state + the INT/COORD persona.
So the embedded pane builds no header and the conversation reclaims the height.
The header build is gated behind !this._embedded (the standalone split-pane,
retired in step 6, keeps its split/close header). The --skip-permissions
SECURITY banner moves off the header to messagesEl (the console host's
warningTarget now matches the default host) so it's preserved, not dropped.
updateWsName null-guards the absent header. Dead .pane--embedded .pane-header /
.pane-ws-name / .pane-persona-tag CSS removed.
Both panes are now header-less in the L-shell — name/state/persona live in the
tab + rail. 103 JS guards green (test_embedded_chrome_is_gated updated for the
no-header reality), node-check clean.
In the L-shell a coordinator is always a pane (no standalone page), and
everything the header showed is redundant: the name + state live in the pane
tab and the rail (Workspaces); end / export moved to the tab dropdown (step 7);
the light/dark toggle is in the rail footer. So drop the header entirely and
give the wasted vertical pixels to the conversation.
buildCoordChrome no longer builds an appbar. The busy/wait indicator
self-disables (its #coord-header mount host is gone; busy shows in the rail
glyph). The per-pane SSE-connection indicator is dropped — reconnect handles
transient drops, matching the interactive pane (which has none); setSseStatus
and the name/state writes are null-guarded. End/export logic stays reachable
for step 7 (exportWorkstreamDownload(wsId); the pane's closeSession() API).
15 coordinator guards green, node-check clean. (Interactive pane header is next
— it hosts the --skip-permissions security banner, which moves to a pane-top
slot so removing the header doesn't drop a security warning.)
The cutover delegated the pane's verdict/warning/status DOM to the shared
conversation.js builders, so three test_app_js guards pinning the old
implementation needed updating to the new reality:
- replayHistory now calls buildConvVerdict(tc.verdict) (was renderVerdictBadge).
- risk normalization moved into the shared builders — the pane carries no raw
`risk_level || "medium"` fallback and builds via buildConvVerdict /
buildConvWarning (which route through normalizeRiskLevel in conversation.js).
- the error pill converged onto .conv-status--error (was .ts-approval-badge--error).
Also dropped the now-dead normalizeRiskLevel import from interactive.js (the
builders own normalization; the pane no longer calls it directly). 123 frontend
JS/CSS guards green.
The emitters switched to .conv-* (conversation.css), so the forked card
vocabularies are now dead (match nothing). Remove them:
- coordinator.css: the whole .coord-tool-* tool-batch construct (-552).
- chat.css: the .ts-approval-* / .ts-verdict-* approval shell (-261); dead
selectors grouped with kept ones in reduced-motion/hover @media blocks were
stripped from the group, not the whole rule.
- interactive.css: the .ts-approval-* / .verdict-* / .output-warning* / tool-div
internals (-436), keeping the .tool-output collapse/stream + .media-* result
subsystem (interactive-only live-execution affordances). Also fixed a latent
malformed-comment bug — the file header had ".ts-approval-*/" whose "*/"
accidentally closed the comment, leaving the rest as stray CSS the browser
silently dropped.
~1249 lines of dead CSS gone. Verified: 0 dead selectors remain across all four
sheets (comments aside), kept anchors present, braces balanced, prettier-clean,
27 JS/CSS guards green. The cards render entirely via conversation.css.
renderApprovalBlock's child-approval pill mapped an unknown/unrecognized
risk_level -> .high (a deliberate "fail-safe over-alert"). Per the user's
2026-06-06 decision, fold it onto the canonical unknown->medium (5e.1b): the
separate "(judge unavailable)" pill already covers the genuinely-unassessed
case, so this path only fired for a malformed risk_level on an otherwise-present
verdict — a rare data edge, not a "we didn't check" signal. Unknown now maps to
.med, consistent with how every other surface (the shared builders via
normalizeRiskLevel) displays it. No remaining inline `|| "medium"` risk
fallbacks in either emitter.
Re-vocabularize interactive.js's approval card onto the shared conversation.js
builders, converging it with the coordinator onto ONE neutral .conv-* card:
buildToolDiv -> buildConvRow + buildConvCmd, renderVerdictBadge ->
buildConvVerdict, _buildOutputWarningEl -> buildConvWarning; showInlineToolBlock
/ announceToolBlock build the .conv-batch shell + head + rows + buildConvActions
(with the inline feedback + recommended glow); resolveApproval / the auto path
-> buildConvStatus; updateVerdictBadge replaces the badge via buildConvVerdict;
the history-replay branch synthesizes the live `item` shape so replay renders
the SAME .conv-row. ~124 ts-approval-* / verdict-* references gone (a final
no-stale-vocab assert guarded it); dead toggleVerdictDetail removed.
The card chrome converges; interactive's richer post-execution result subsystem
(.tool-output collapse/stream + .media-* embeds) is KEPT as-is — those are
live-execution affordances the read-only coordinator history doesn't need.
Block state classes move to the BEM modifiers (.conv-batch--approved/--denied/
--error/--auto); per-pane keybindings (y/n/a) preserved via the builder kbd
hint. The now-dead old card CSS (.coord-tool-*/.ts-approval-*/.verdict-*) is
inert (matches nothing) and is removed in 5e.2f's CSS dedup.
Verified: node --check both emitters; the no-stale asserts; 60 JS guards green
(test_coordinator_page pinned vocab updated coord-tool-batch-> conv-batch). The
builders are behavior-tested (5e.2b). Designer + /review + live-backend run once
at the merge gate.
Re-vocabularize coordinator.js's tool-batch construct onto the shared
conversation.js builders (5e.2b): _renderBatchRow -> buildConvRow,
_appendVerdictLineTo -> buildConvVerdict, _attachOutputWarningChip +
appendGuardFinding -> buildConvWarning, _appendResultToRow -> buildConvResult,
_buildBatchActions -> buildConvActions, _buildStatusPill -> buildConvStatus,
the appendToolBatch shell -> buildConvBatchShell. All 115 .coord-tool-*
references (including the security-critical _resolveBatchAction call_id
selector and the SSE upgrade-in-place handlers) renamed to .conv-* (a final
"no coord-tool- remains" assert guarded the rename). Dead _makeActionButton
removed (buildConvActions replaces it).
The verdict converges on the richer expandable badge; its rationale folds into
the verdict detail (the separate .coord-tool-row-rationale <details> is gone).
The warning rationale is now inline. Coordinator renders via conversation.css
(linked since 5e.2a); the old .coord-tool-* rules in coordinator.css are now
dead and get deleted with the interactive switch. Child-approval block
(.approval-*) untouched here — it converges in 5e.2d.
net -249 lines. Verified: node --check + the no-stray-ref assert; the builders
are behavior-tested (5e.2b, 48 asserts). Holistic both-panes harness + designer
pass land after the interactive switch.
Add the pure leaf DOM builders for the unified `.conv-*` card to
conversation.js (both panes import it): buildConvBatchShell / buildConvRow /
buildConvCmd / buildConvVerdict / buildConvWarning / buildConvButton /
buildConvActions / buildConvStatus / buildConvResult. The builders own only
the DOM + class vocabulary; everything stateful (the toolRows map, idempotent
upgrade-in-place, the early-paint announce shell, SSE routing) stays in each
pane and CALLS these in 5e.2c.
Parameterized by AFFORDANCE, not subclass: buildConvRow takes an indexLabel
(coordinator's parallel idx pill) or defers to buildConvCmd (interactive's
bash `$ cmd` + diff preview); buildConvActions takes per-pane keybinding hints
+ an optional feedback input (interactive) and per-pane resolve callbacks. The
persistent action unifies on "Approve all" (dashed --ok ghost), not the
coordinator's old "Always". Risk routes through normalizeRiskLevel so the
per-site `|| "medium"` fallbacks fold onto the canonical unknown->medium; the
judging spinner withholds the --{risk} class so its stripe stays neutral.
Additive — no emitter calls these yet (the re-vocabularize + delete is 5e.2c).
Verified: 48-assert headless-Chrome behavior harness (DOM shape, crit->critical
normalize, unknown->medium fold, expand toggle, action callbacks, JSON
pretty-print) + tests/test_conversation_js.py extended (11) + node --check.
Author shared_static/conversation.css: ONE neutral `.conv-*` approval-card
vocabulary that both panes will emit, converging the two forked cards
(coordinator's `.coord-tool-*` + interactive's `.ts-approval-*`/`.verdict-*`).
Based on the BRIEFING-blessed `.coord-tool-batch` idiom — neutral surface,
state left-stripe (warn pending / ok approved / err denied), uppercase kicker,
Approve = subtle --ok fill / Approve all = dashed --ok ghost / Deny = --err
(the DS hard-rule: approve uses --ok, never --warn) — with the interactive
affordances folded in (bash `$ cmd`, unified-diff preview, inline feedback,
recommended-button glow, auto-approved tag, the expandable verdict detail).
Converges onto the DS token vocabulary (--ok/--warn/--err, --ink-*, --panel*,
--hair*), not chat.css's legacy --green/--red/--cyan. Self-contained spinner
keyframe (conv-spin) so the sheet doesn't depend on coord-chrome.css's ts-spin,
which the standalone interactive pane never loads.
Additive only — no emitter uses `.conv-*` yet (the re-vocabularize + delete of
the old sheets is 5e.2c). Linked from the console + both standalone pages so
the card is styled the moment 5e.2c switches the emitters over.
Designer-reviewed (rendered both themes): applied the warning-chip wrap fix,
the medium-severity-weight fix (12% mix, not raw --warn-tint), ink-4 -> ink-3
on verdict-detail/tier content for light-mode AA, the bold severity label, and
the neutral judging-row stripe. Guard: tests/test_conversation_css.py (5).
Both panes carried their own risk-level logic that disagreed on the fallback:
interactive's normalizeRiskLevel sent an unknown level to "medium" (and, lacking
the crit/med aliases, rendered a "crit" verdict as medium), while the
coordinator's _riskRank sent unknown to "high". Lift one canonical normalize +
rank into conversation.js and route both panes through it.
- conversation.js: normalizeRiskLevel (aliases crit->critical / med->medium;
unknown -> "medium"), riskRank, maxSeverityItem (keeps the no-verdict -> -1
edge so an unassessed item never wins the max-severity pick).
- interactive.js: import normalizeRiskLevel, drop the local copy (its 3 callers
unchanged); a "crit" verdict now renders critical instead of medium.
- coordinator.js: import maxSeverityItem, drop RISK_SEVERITY / _riskRank /
_maxSeverityItem; an unknown-level item now ranks medium, not high.
- unknown -> medium is the deliberate fallback (per decision), not "high":
medium is the neutral default both panes' displays already used.
- tests: conversation guards for the fallback + aliases + the no-verdict edge;
the two pane guards now check the shared module.
The per-site risk->CSS-class display mappings (coordinator's inline chips and
renderApprovalBlock's deliberate unknown->crit over-alert pill) are left for the
5e.2 vocabulary reconcile.
Stand up shared_static/conversation.js as the deduplicated conversational-pane
substrate both panes import (interactive via ./, the coordinator via /shared/ —
both ES modules since 5e.0). First tenants are the byte-identical duplicates the
in-file comments flagged for the step-5e lift: stripAnsi, the watch-result card
builder, and the system-nudge marker. No visible change — the builders return
the same DOM; each caller still appends + scrolls.
- conversation.js: stripAnsi (null-safe variant), buildWatchResultCard,
buildSystemNudgeMarker.
- interactive.js / coordinator.js: import the three, drop their local copies,
delegate appendWatchResult + the nudge marker through the shared builders.
- stripAnsi unified on the coordinator's null-safe form (interactive's threw on a
non-string arg); identical output for string inputs.
- tests: new test_conversation_js.py pins the module; the two retry-walk guards
now check the watch-result marker in conversation.js (it moved there).
- refreshes interactive.js's header comment, stale since 5e.0 made the
coordinator an ES module too.
Lift coordinator.js off the window.createCoordinatorPane bridge onto a real ESM
export, so the upcoming shared conversational module (5e) is import-consumed on
both sides rather than through a classic window global. The console shell and the
standalone page's bootstrap both import the factory now; zero behaviour change.
- coordinator.js: export the factory, drop the window bridge — no classic
consumer remains (unlike interactive.js, whose ui/static app.js still uses its
global).
- shell.js: import the coordinator factory by URL (mirrors the interactive
import) and call it directly.
- console index.html: stop script-tagging coordinator.js; shell.js's import loads
it (a classic tag chokes on the top-level export).
- coordinator/index.html: the standalone bootstrap becomes a module that imports
the factory — a classic eager IIFE ran before the deferred module loaded it.
- tests: pin the new ESM seam (export, shell import, module bootstrap).
Designer pass on the new console interactive pane. Two clean fixes; the rest of
the findings are scoped to their planned steps (see below).
- Rail Workspaces `.open` marker now tracks the ACTIVE pane instead of being
hardcoded to Dashboard — the rail map and the tab bar were disagreeing about
what's focused (opening a session never moved the rail highlight). PaneManager
gains getActive() + onActiveChange() and fans out on activate/close; the rail
keys `.open` off the active pane's rawId and re-renders on activation (not just
on the next Tier-1 snapshot).
- The active tab's glyph brightens (--ink-4 -> --ink-2) so an open session's `○`
placeholder doesn't read permanently "idle" beside its live (running ●) rail
row. (Tab glyphs go fully live in step 7.)
Deferred (planned elsewhere, not regressions): the tab CLOSE affordance is step 7
(the brief's three-verb `.ws-tab-dropdown`); the interactive/coordinator HEADER
consistency is what the step-5e base lift unifies (a shared header parameterized
by affordances), so partial coordinator-header surgery now would be a half-measure.
Verified: a headless screenshot (rail `.open` now on the active session, slim
header + persona tag render clean) + 107 JS-guard tests (test_shell_js 5d guard),
node --check, prettier, ruff.
Rewire the coordinator's child ws links (deferred from step 4) to open the child
as a node-proxied interactive pane inside the console L-shell, instead of a
full-page new tab to /node/{id}/?ws_id=.
- A delegated click handler on the pane root catches .ws-link (children tree,
renderChildRow) and .coord-ws-link (linkified tool output, renderToolOutput)
clicks; both link types now carry data-ws-id + data-node-id. When a
PaneManager is present it opens openPane('interactive', child_ws_id,
{nodeId: child_node_id}) — the child's OWN node, so its stream proxies to that
node even though the coordinator lives in the console.
- Progressive enhancement: the link's href (/node/{node}/?ws_id=) stays the
standalone fallback — the standalone coordinator page has no PaneManager, so
the new-tab nav stands. No innerHTML introduced (the tool-output linkifier
still returns a string; only data-* attrs were added).
Verified: a harness running the console stack (coordinator.js + shell.js +
interactive.js) — open a coordinator pane, click a child link -> a node-proxied
interactive pane opens (/node/{child_node}/.../events), zero errors; 106
JS-guard tests (test_coordinator_page step-5c guard), node --check, prettier.
Wire the shared interactive Pane (5a) into the console L-shell as a ws_id-keyed,
node-proxied conversational pane.
- shell.js IMPORTS createInteractivePane (interactive is a real ES module, so
the shell consumes it the modern way; the legacy coordinator pane stays on the
window.* seam — the incremental "pulled by the adopting pane" modernization).
registerType('interactive') mirrors the coordinator: build on mount, connect
on activate (idempotent) + login re-arm, deactivate on tab-away (stops
focus-stealing while the stream stays live), destroy on close.
- The node-proxy target is DERIVED from the Tier-1 snapshot (nodeForWs), so a
rehydrated pane needs no persisted node_id; a rail click / child link can pass
{nodeId} as an open-time hint. openPane(type, id, extra) threads that hint to
the factory (not persisted).
- rail.js: interactive session clicks now openPane('interactive', ws.id,
{nodeId: ws.node}) instead of full-page nav to /node/{id}/.
- interactive.css (new, shared): the embedded slim-header layout (scoped to
.pane--embedded so it never collides with the ShellPane's own .pane section —
the brief's namespace watch-out) + the conversational rendering (tool output /
media / MCP-error / verdict / output-guard cards) COPIED from ui/static. The
shared chat.css .msg/.ts-approval base is left untouched, so the coordinator
pane is unaffected; step 5e unifies the vocabularies, and ui/static keeps its
copy for the standalone until step 6.
Verified: an integration harness running the REAL shell.js + rail.js +
interactive.js (register -> rail-open -> embedded chrome -> node-proxy SSE
/node/{id}/.../events -> /history replay into real .msg turns -> destroy, zero
errors) + a screenshot; 105 JS-guard tests (test_shell_js step-5 guard),
node --check, prettier, ruff.
Lift the per-workstream conversational Pane (chat + approval cards + composer +
voice + tool/media/MCP-error/verdict rendering) out of ui/static/app.js into a
new shared ES module shared_static/interactive.js, so BOTH deployments can
mount it: the standalone turnstone-server UI (its split-pane shell stays in
app.js and builds panes via window.InteractivePane) and — next, in step 5b —
the console L-shell over a node-proxied Tier-2 stream.
- Transport seam: a per-pane `base` prefix ("" local, "/node/{id}" proxied)
threads through every request; createInteractivePane derives it from nodeId
(the LOCALITY invariant — an interactive session lives on a cluster node).
- Host seam: the couplings only the surrounding shell knows (workstream name,
focus, stream-error recovery, the --skip-permissions banner target, the MCP
consent badge) route through an injected host adapter; the standalone shell
supplies the real one (refetchWorkstreamsAndReassign + STANDALONE_HOST), the
console factory a Tier-1 / no-op one.
- Embedded chrome: the standalone split-pane affordances (focus tracking,
context menu, split/close buttons) are gated behind !embedded; the embedded
path adds the INTERACTIVE persona tag.
- First legacy pane lifted into a real module: it exports the factory for the
console shell's import and bridges window.* for the still-classic standalone
shell (which builds panes only after the workstream fetch, so the deferred
module has run). coordinator.js + the shared substrate stay classic.
The whole tool-output / media / MCP-error / verdict cluster moved with the Pane
(used only by it); the consent-BADGE subsystem stays in the standalone shell,
reached via host.onConsentDetected.
Verified: 104 JS-guard tests + a headless harness running the real module
(standalone + embedded chrome, node-proxy transport, lifecycle, zero errors),
node --check, prettier, ruff.
A dedicated, formatting-only pass over the renovation's frontend so future edits
inherit a consistent style — LLM/contributor edits pattern-match the surrounding
code, so a clean baseline keeps it clean. Covers every non-conformant
.js/.css/.html under shared_static/ + console/static/ + ui/static/ (vendored
katex/hljs + *.min.* excluded; the other ~22 frontend files were already clean).
No rule, value, or markup-semantic changes — whitespace/wrapping only.
Also fixes a real (browser-tolerated) bug the pass surfaced: a `*/` inside a
coord-chrome.css header comment (`#coord-*/coordinator-class`) closed the CSS
comment early; reworded so the comment is valid.
Files: coord-chrome.css, console/static/{index.html,style.css}, coordinator.css,
shared_static/{auth.js,base.css,chat.css,ui-base.css}, ui/static/index.html.
Four findings from the step-4 designer pass on the coordinator pane.
- P1 (bug): the `end` button ran `window.location.href = "/"`, which inside the
L-shell reloaded the WHOLE console — every other pane destroyed, all their
Tier-2 streams dropped. Thread an `onClose` through the factory; the console
pane passes `() => pm.close(pane.id)` so `end` closes that tab (and runs the
controller teardown via onClose→destroy); the standalone page passes none and
keeps the console redirect.
- P2: the pane root carries both `.pane-body` (overflow:auto) and
`.coord-chrome-root` (flex column), so the generic pane scroller redundantly
wrapped the sticky appbar. `.pane-body.coord-chrome-root { overflow: hidden }`
(scoped to this pane type) — the coord chrome owns its own scroll regions.
- P3: the coordinator tab glyph `●` collided with the rail's running state-dot
vocabulary (a static dot reading as "live"); swap to `◆` (a shape marker that
pairs with dashboard's `◇`), pending the real state-glyph in step 7.
- P3: the destructive `end` button had only a title; add aria-label
"End coordinator session".
Verified via the harness (clicking `end` closes the pane without reloading;
glyph `◆`; aria-label present; zero errors); guards pin the pane-aware close.
test_shell_js + test_coordinator_page (97 green), ruff.
The console can now host coordinator sessions as ws_id-keyed panes alongside
dashboard/admin — step 4 complete (the de-globalization landed in 4a).
- coordinator.js: `buildCoordChrome(root, opts)` builds the coordinator chrome
programmatically (createElement, no innerHTML); the factory builds it on
instantiate, so the SAME factory serves the standalone page and a console pane.
`opts.standalone` adds the page-level bits a pane doesn't want (the Console
back-link, the theme toggle, the shared #toast).
- index.html (standalone): goes thin — a bootstrap calling
createCoordinatorPane(document.body, ws_id, {standalone:true}); the ~500-line
inline <style> is migrated to coord-chrome.css (its lone page-level body rule
scoped to .coord-chrome-root) so the console can load the same chrome CSS.
- shell.js: registerType('coordinator') keyed by ws_id — onMount builds the
controller into the pane body, onActivate opens its Tier-2 SSE once, onClose
destroys it. Plus a window.TS_LOGIN fan-out registry so every pane re-arms its
own stream on re-auth (app.js's single onLoginSuccess becomes one subscriber).
- rail.js: coordinator clicks → openPane('coordinator', ws_id) instead of
full-page nav (interactive sessions stay interim full-page until step 5).
- console/index.html: loads the coordinator controller + chrome CSS + the shared
composer/renderer deps it needs.
Child links → openPane('interactive', ws_id) are deferred to step 5 (the
interactive pane doesn't exist yet); coordinator transport stays console-local
inline (parameterized only when the shared ConversationalPane base is lifted).
Verified end-to-end with a headless harness running the real shell.js + rail.js +
coordinator.js: opening a coordinator pane registers the type, the rail row opens
it, buildCoordChrome populates the pane, the Tier-2 SSE connects, destroy() tears
down — zero uncaught errors; renders cleanly (appbar + chat + children/tasks
sidebar + status bar). test_shell_js + test_coordinator_page (97 green), ruff.
coordinator.js was a page-global IIFE keyed off <html data-ws-id>. Make it
multi-instantiable so the console shell can host coordinator sessions as panes
(one per ws_id) alongside dashboard/admin — the first conversational pane-content.
- IIFE -> `createCoordinatorPane(root, wsId)`: the ~40 module-state vars stay
closure-local (now automatically per-instance), every #coord-* lookup is
root-scoped (27 getElementById -> root.querySelector), ws_id is a constructor arg.
- New lifecycle: `connect` (= init), `destroy` (closes the EventSource + clears the
6 timers + the prune interval + the IntersectionObserver — the IIFE had no
teardown, so a backgrounded pane would leak an SSE and fire into detached DOM),
`onLogin` (re-arm after a 401), `closeSession`.
- Drop the page-global collision points: `window.coordSend`/`coordCloseSession`
-> local fns (the close button binds per-instance; its inline onclick is removed);
`window.onLoginSuccess` -> the returned `onLogin` (the console shell will fan
login out to every pane; standalone keeps the single hook).
- Standalone coordinator page = one pane filling the body: a thin bootstrap calls
`createCoordinatorPane(document.body, ws_id).connect()`.
Console-local transport (the coordinator endpoints) stays inline — coordinators
always live in the console; transport is parameterized only when the shared
ConversationalPane base is lifted (after step 5). The chrome builder, CSS
migration, and console pane registration are 4b.
Verified: node --check; a headless smoke (the real factory instantiates against a
provided root, runs connect()'s snapshot/history/children/tasks/SSE on stubs, then
destroy()s — zero uncaught errors); test_coordinator_page.py (13, incl. a new
factory-shape guard); ruff.
Five findings from the step-3 designer pass; the P3 chevron-rotation (taste) was
skipped — the text-swap is already motion-safe.
- P1: the adopted #view-admin had no inset, so the first admin section-header
butted the tab-bar hairline + rail edge. Add `padding:16px 0 0 16px` on
`.pane-body > #view-admin` (.admin-content keeps its right pad).
- P2: the rail Manage active-marker never seeded from getActiveTab(), so a
PaneManager.rehydrate-restored Admin pane showed no active group/row until a
re-click. mountManage now takes the PaneManager, seeds the marker + expands the
owning group when the Admin pane is already open (new PaneManager.hasPane()).
- P2: the active-row band was byte-identical to the amber `.row.open` of live
sessions (distinct only by a 2px stripe). Give it its own neutral idiom —
`--panel-2` fill + a hairline `inset 2px` marker — so "which admin tab" reads
as different in kind from "which session is live".
- P3: `.gcount` pinned right with `margin-left:auto` (was incidental via flex).
- P3: strip the dangling `role="tabpanel"`/`aria-labelledby="tab-*"` from the 18
adopted admin panels (their sidebar buttons were deleted in 3b); the 9 legit
tabpanels elsewhere are untouched.
Verified via the headless harness (rail / admin-open / rehydrate states) +
test_shell_js.py guards (the aria strip is now pinned).
The rail's Manage groups replaced the in-pane admin sidebar in 3a; this removes
the now-dead markup, JS, and CSS that it leaves behind.
- index.html: drop the #admin-sidebar nav (6 groups / 18 buttons) + the mobile
#admin-sidebar-backdrop; #admin-layout now wraps #admin-content alone.
- admin.js: delete the mobile off-canvas drawer (_mobileSidebarOpen,
_injectMobileToggle, _toggleMobileSidebar, the Escape-to-close + arrow-nav +
resize-sync handlers) and switchAdminTab's now-dead .admin-nav active loop +
breadcrumb write.
- style.css: remove the .admin-sidebar* / .admin-nav* / .admin-mobile-toggle*
rules, the mobile off-canvas @media block, and the dead reduced-motion entries.
- shell.css: drop the .pane-body .admin-sidebar hide rule (nothing to hide now).
.admin-layout / .admin-content / #view-admin stay (the Admin pane adopts them).
Verified: no residual sidebar/mobile refs, CSS braces balanced, admin.js parses,
headless render unchanged, and test_shell_js.py pins the removal.
Admin becomes a singleton pane and the rail's Manage section becomes its
navigation; the in-pane sidebar is retired.
- shell.js registers an `admin` pane type that adopts #view-admin (the 18
tabpanels) on first open; the dashboard pane keeps #main.
- admin.js: new ADMIN_IA seam (window.TS_ADMIN) — the group→tab map, a shared
adminTabAllowed() gate (mirrors the legacy showAdmin permission gate, incl.
the ungated node list), an active-tab subscription, and openTab. showAdmin is
now a thin delegator (openPane('admin') + switchAdminTab); the in-#main view
toggle, breadcrumb write, history push, and mobile-hamburger injection go.
- rail.js: mountManage() builds the six collapsible .grp groups from the seam,
permission-filtered, routing a row click through openTab — never touching
admin DOM.
- app.js: home/drill re-focus the Dashboard pane instead of blanking the moved
#view-admin.
- shell.css: the .grp vocabulary + admin-pane layout (in-pane sidebar hidden,
#view-admin fills the pane).
The legacy #admin-sidebar is hidden via CSS pending its deletion in 3b; this is
the additive, independently-runnable half. Verified with a headless-Chrome
harness driving the real shell.js + rail.js over a stubbed seam, plus the
test_shell_js.py guards (19 passing).
From the designer pass on the live rail + persona launcher:
- rail.js: the version-drift amber now marks only nodes whose version differs from the cluster majority (was painting every node when the cluster drifted — the highlight pointed at everything, so at nothing). Adds a per-node title naming the majority.
- app.js + index.html: the persona toggle honours its role=radiogroup contract — arrow keys move the selection, roving tabindex makes the group a single tab stop (seeded statically + in _setLauncherKind), instead of announcing radios but behaving like plain buttons.
- shell.css: an inset (-2px) :focus-visible ring for the rail rows/pills + persona buttons, so the keyboard focus outline doesn't clip against the 266px rail edge (mirrors .dash-row:focus-visible).
The dashboard body becomes a persona-unified launcher (start a coordinator OR an interactive session from one composer) and the saved list spans both kinds; the redundant active-coordinators table is dropped (the rail covers it now).
Backend — the console /v1/api/workstreams/saved now returns both kinds: session_routes.py extracts _collect_saved_rows (shared by the refactored, behaviour-preserving make_saved_handler) + adds make_unified_saved_handler (merges per-kind queries — run concurrently via asyncio.gather — sorted by updated desc). The operator gate (admin.coordinator) is applied once; no new exposure (operators already see every session). console/server.py mounts it with [coordinator, interactive] cfgs.
Frontend — a persona toggle routes submit by kind: coordinator -> console-local POST /v1/api/workstreams/new; interactive -> node-proxy POST /v1/api/cluster/workstreams/new (auto placement). Each option is scope-gated (admin.coordinator / workstreams.create); attachments stay coordinator-only. The saved list gains a KIND tag column + kind-routed activation (coordinator -> /open + /coordinator; interactive -> /node/{id}/?ws_id=) and stays operator-gated. The active-coordinators table + _renderHomeView/_activeCoordsFromClusterState are removed.
Tests: make_unified_saved_handler coverage (tests/test_saved_handler_unified.py, synthetic fixtures, no DB) + a console-launcher static guard (tests/test_shell_js.py). Reviewed via the multi-stage pipeline; findings applied (client/server gate match, concurrent queries, chip-CSS dedup, static guards, stale-comment cleanup).
The rail's Cluster + Workspaces sections (step-1 stub labels) now render live from the Tier-1 clusterState, and the legacy bottom #cluster-status-bar is retired — the rail replaces it (the L-shell has no bottom bar).
New shared_static/rail.js (ESM): renders Cluster (health pills wired to drillDownByState + a node list with version/drift) and Workspaces (the session tree — coordinators with children nested via the shared _bucketByParent, COORD/INT persona tags, state = shape+colour via ui-base .ui-glyph-*).
app.js exposes a minimal Tier-1 seam on window.TS_APP (getClusterState + onRender + the rail's nav actions); renderFromState fires subscribers. No physical clusterState extraction — the seam closures see the live binding. shell.js builds the Cluster/Workspaces render targets and mounts rail.js before boot so it catches the first snapshot.
Retire the bottom bar: delete the #cluster-status-bar markup + renderStatusBar / renderNodePicker / the node-picker helpers + STATE_ORDER (~310 lines), and the .stale toggles in connectSSE (the rail-conn #status-bar carries connection state now). buildNodeInfoFromSnapshot / recomputeOverview / _bucketByParent stay — the rail reuses them. The dashboard body still carries its active-coordinator table transiently; 2b reshapes it into the persona launcher + unified saved list.
Step 1 of the console renovation: a full-height left rail, a top tab bar, and a generic pane host that shows one pane per tab. Existing console content is hosted unchanged inside it as the default Dashboard pane.
New shared_static ES modules (the first ESM citizens; classic scripts keep loading alongside them): pane.js (PaneManager + ShellPane — typed-window host with openPane/activate/close, sessionStorage rehydrate, a WAI-ARIA tablist with roving tabindex + arrow-key nav, reconcile-in-place tabs); shell.js (builds the rail/tab-bar/pane-host, reparents #main + #status-bar with ids preserved so connectSSE needs no rewire, relocates the header controls into the rail footer, drives the app boot); shell.css (chrome ported from the layout mock to base.css tokens).
console index.html loads the shell module + capability flags + stylesheet; app.js's bottom init is wrapped into window.TS_APP.boot, which the deferred shell module drives (it runs after the classic scripts). Cluster health, the Workspaces tree, admin, and conversational pane types arrive in later steps; the rail sections are labelled stubs.
The interactive pane (app.js) got the event-id dedup that skips an
operator-context system turn already painted from /history when an SSE replay
redelivers it; the coordinator pane (coordinator.js) shares the identical
/history + live system_turn + last_event_id replay seam but was left without
the guard. The backend row/event-id alignment already fixes the actual double
for both panes — this restores the defense-in-depth symmetry.
- Module-scoped renderedSystemEventIds (persists across reconnects like
lastEventId), reset in refetchHistory.
- onmessage tags each event with its SSE id; the live system_turn handler skips
an already-rendered id; the history loop records the ids it paints.
- Parallel static-shape regression test in test_coordinator_page.py.
A first-class operator-context system turn (metacognition nudge, output-guard
finding, interjection, watch result) was persisted stamped with the event-id
counter's PRE-emit value, then its live `on_system_turn` SSE event was emitted
with the post-increment id — so the row sat one below its own event. On an
in-flight-orphan `/history` resume, `_resume_cursor_and_trim` derives the SSE
replay cursor from the row's id; being one low, the replay redelivered the
turn's own `system_turn` event and the frontend (no dedup) painted the
operator bubble twice. Reliable for coordinator-spawned children (opened
mid-task) and self-healing on rehydrate — a non-persisted, live-only double.
- `SessionUIBase._enqueue` returns the monotonic `_event_id` it assigns;
`on_system_turn` returns it; `_append_system_turn` emits the hook first and
persists the row with that id (fallback to the current cursor for non-SSE
UIs / a throwing hook). Now row.event_id == its own SSE event id.
- `project_history_messages` surfaces each row's `event_id` so the frontend
can dedup.
- app.js: tag each SSE event with its id, reset a per-pane rendered-id set on
`replayHistory`, and skip a `system_turn` already painted from `/history`
(belt-and-braces against any future cursor skew).
- Regression tests pin the row/event id alignment, the `/history` emit, and
the FE dedup.
- ruff format on two test modules that had drifted (a stray blank line
and multi-line calls that now fit on one line) — restores a clean
`ruff format --check`.
- test_attachment_buffer: pull `buf.discard(...)` out of the `assert`
expressions into locals so the eviction still runs under `python -O`
(CodeQL: assert statement has a side effect).
The Anthropic SDK provider was the lone first-class provider gated behind
an optional extra, while OpenAI ships in core and Google rides the
OpenAI-compatible path. Fold anthropic, psycopg (postgres), croniter
(console), and lacme (tls) into the base dependency set so a default
`pip install turnstone` yields a complete single- or multi-node
deployment; only the Discord/Slack channel gateways stay optional.
- pyproject: four extras → base deps; `all` is now discord+slack; drop the
redundant croniter from the `test` extra; regenerate uv.lock.
- ci: the postgres test job installs `.[test]` (psycopg is base now).
- providers: `_ensure_anthropic` becomes a thin SDK accessor for
`create_client`; drop the now-redundant eager import-guard calls from
the streaming/completion hot path (anthropic is always present).
- bootstrap: import anthropic directly.
- tests/docs: drop the anthropic importorskips and stale extra-install hints.
The operator-context consolidation persists each metacognition nudge's type as
``_source`` (start / resume / correction / denial / completion / repeat), and
both panes rendered it raw as "operator · start". Add a shared
``operatorSourceLabel`` helper in utils.js (loaded by both UIs) that collapses
the metacognition types to one "metacognition" category and humanizes
``tool_error`` / ``skill_hint``; both panes call the single helper so they can't
drift. Carded kinds (watch / guard / idle / interjection) are unaffected.
The canonical-Turn migration left lowering's fold/drop/repair passes Turn-typed even
though they convert to dicts internally and feed dicts to the translators, so
_prepare_wire_messages round-tripped the whole history Turn->dict->Turn ~7-8x per send
(even on the no-op early-return paths). Make fold_system_turns / drop_empty_user_turns
/ repair_wire_messages dict-native (list[dict]->list[dict]); _prepare_wire_messages now
threads the dict projection _full_messages already produced straight through, with no
Turn round-trip. self.messages stays the canonical Turn trajectory. export.py is
simplified (it converted to dicts immediately after repair anyway). Equivalence-
preserving — test_wire_payload_golden stays byte-identical.
- Buffer (attachment_buffer.py): content-address staged bytes once and track the
per-(ws_id,user_id) references to them, so identical bytes staged from two tabs
dedupe to one copy yet neither scope's send can drop the other's pending upload
(the prior hash-only key let one overwrite the other). Single lock; add a public
clear() that replaces test reaches into the private store.
- GC: lift the byte-identical _release_attachment_refs out of both backends into one
dialect-agnostic storage/_utils.release_attachment_refs with a portable searched-
CASE single-query decrement (was one UPDATE per id in a Python loop).
- _format_messages_for_summary: mark by-reference vision results
({type:image, attachment_id}) as [image], not just inline image_url.
- Security: escape_like() the attachment_referenced_in_ws LIKE needle on both
backends; secrets.compare_digest for the output-guard operator-fence leak check.
- B1: page _backfill_content_addressed_attachments via a composite keyset cursor
(message_id, created, attachment_id) instead of one un-paged fetchall() of every
blob's bytes — bounds peak migration memory regardless of stored blob volume.
Validated on the dev-DB snapshot: upgrade + downgrade clean, 5431 conversations
preserved, blobs deduped + content-addressed.
- B2: _native_from_provider_data strips orphan client tool-call blocks on load when
the row's tool_calls column is empty (the truncated-mid-tool_use legacy hole), so
a same-provider resume can't replay an unanswered tool_use — closing the Anthropic
400 and the Google tool-call resurrection path. Healthy (mirror-holds) rows decode
byte-identically.
- A1: add a shared `operator-context` marker to every operator-context row in
both UIs; the retry-skip walk keys on it, so a trailing watch-result /
guard-finding / idle-children card no longer makes retry regenerate the
wrong turn. Pinned by source-grep tests + a headless-DOM self-test.
- A2: wrap get_content's two sync DB gates (get_attachment + the unbounded
ws-scoped attachment_referenced_in_ws LIKE scan) in asyncio.to_thread so a
long scan can't stall the event loop — matching the module's convention.
- A3: add the HistoryEvent attachments docstring bullet; drop the dead
`interjection` class; document the interactive-only system-context label;
correct the SDK attachments-meta docs to {kind, filename, mime_type}
(size_bytes is not carried through the history projection).
Operator-context system turns (watch results, output-guard findings, idle
children, user interjections) carried their kind (_source) and a flattened
text content, but the structured per-kind fields were dropped at every
persist/deliver boundary — so the UI rendered every kind as one generic
operator bubble and the structured watch-result card was lost.
Wire the structured meta through as the single source of truth:
- Storage: new conversations.meta JSON column (migration 060); threaded
through save_message/save_messages_bulk (facade + protocol + both backends)
and rehydrated in reconstruct_turns onto Turn.meta.extra["source_meta"].
- Canonical: make_system_turn carries meta as one _source_meta dict;
turn_from_dict/turn_to_dict bridge it to/from Turn.meta.extra.
- Live + history: widen on_system_turn(content, source, meta) across all
impls + the SSE payload; surface _source_meta -> meta in the /history
projection. SDK HistoryEvent docs note the field.
- Producers derive both the model-facing content text AND the card from one
meta dict, so they cannot drift: render_output_guard_text, build_watch_
reminder carrying output, idle_children and user_interjection metadata.
- Frontend: addSystemContext / renderSystemTurn dispatch by source to the
watch-result, guard-finding, idle-children, and queued-message cards in
both the interactive and coordinator panes; every untrusted field renders
via textContent.
The meta is a leading-underscore key, stripped before the wire (sanitize_
messages and the native mid-conversation path copy only role+content), so the
per-provider wire payloads stay byte-identical. Additive column, no backfill:
operator turns predating it reload as plain text bubbles.
A deep-dive review of the branch surfaced a budget regression, SDK doc
drift, dead code, and stale docstrings. Each was boundary-spiked before
fixing.
- R1 (regression): by-reference document attachments were invisible to the
token budget — _msg_text_chars returned 0 doc_chars for a
{type:document,attachment_id} placeholder, and the comment's claim that
the budget "lands at calibration" was false (calibration discards
doc_chars). Thread the doc size through _attachments_meta (size_bytes, at
both the live-append and reconstruct build sites) and count it in
_msg_text_chars, guarded against double-counting the inline form.
Regression test added.
- F1: the history-DTO schema description and the TS HistoryEvent docstring
still advertised the removed reminders/advisories keys and omitted the
system role; corrected server_schemas.py + sdk/typescript/src/events.ts to
match the shipped shape. The committed OpenAPI JSON snapshots were already
~679 lines stale on main; their regen is left to its own chore branch.
- D1: removed AttachmentBuffer.take() — dead (no production caller; the
commit path uses discard()) and scope-weak (ws_id only, unlike its
siblings) — with its test and the now-orphaned Iterable import.
- O1: 4 docstrings referenced the moved ChatSession._fold_system_turns →
lowering.fold_system_turns.
The blob store is global content-addressed — identical bytes dedupe across
workstreams and users, so the per-tenant ws_id/user_id scope columns are dead:
nothing reads them, and a committed blob is authorised via the
conversations.attachments ref-list (attachment_referenced_in_ws), not a row
scope. Drop both columns and idx_ws_attachments_ws_id from the schema and from
the save_attachment signature (protocol + both backends + memory wrapper + the
caller); fold the column/index drops and their downgrade into the unshipped
migration 060.
tool_name stays: it is a live denormalised search label (search_history →
recall + /history), not trajectory data — "never rehydrated" held only for the
wire path, which already ignores it.
The by-reference content lane now materializes at the provider translator (the
C layer), not in the session. Each create_streaming / create_completion takes
a resolve_attachments callback and runs materialize_attachments() up front,
expanding {type:kind, attachment_id} placeholders to inline data-URI / document
parts by a content-addressed point-lookup the session hands down
(_resolve_attachments). _full_messages emits placeholders; the dict bridge
carries only placeholders.
RawContentBlock is removed — ContentBlock = TextBlock | AttachmentRef. A
resolved inline part is terminal (the wire payload / display output) and never
re-enters the canonical path, so turn_from_dict drops a stray inline image_url
rather than carrying bytes. resolve_attachment_parts / materialize_attachments
operate on the dict projection. Tool vision output rides by reference too
(_tool_content_by_reference): the turn carries placeholders, the bytes persist
content-addressed. The per-turn token estimate counts a by-ref image as one
fixed image budget; the document char budget lands at send (on resolution).
Wire harness byte-identical (the multipart fixture is a placeholder + a matching
resolver); full non-live suite green (7136).
Non-text content (user uploads, reloaded tool images) rides as AttachmentRef(id,kind)
in the canonical Turn — session.messages carries ids, never bytes. Each output
materializes it to inline data-URI/document parts by point-lookup on the content-
addressed store: the wire (ChatSession._lower_messages_to_wire, in _full_messages),
/history + export (reconstruct_messages resolves), and the per-turn token estimate
(a by-ref image costs one fixed image budget; the doc char budget lands at send).
reconstruct splits: reconstruct_turns = unresolved row→Turn (load_message_turns, the
resume path); reconstruct_messages = resolved dict facade. RawContentBlock is demoted
to the transient carrier for a resolved inline part on the dict↔Turn bridge.
repair_wire_messages / fold_system_turns / drop_empty_user_turns take and
return list[Turn] — the neutral lowering layer (A representation + B validity)
now speaks the canonical type. Their intricate content-merge / orphan-detect
internals run over the dict projection (reading Turn content blocks would only
duplicate turn_to_dict's content logic), so each bridges
dicts_from_turns ↔ turns_from_dicts at its boundary; byte-identical.
ChatSession._prepare_wire_messages lifts the wire dicts into Turns, runs the
lowering passes, and lowers the result back to the dict projection the provider
translators (the C layer) consume — the dict bridge now lives in the wire layer,
not in _full_messages. Export runs the same repair, reordered before the
non-canonical reasoning-content attach (a key the Turn model does not carry).
The provider translators keep their dict input by design: they are the format
layer that emits provider bytes, the vLLM reasoning-attach is a non-canonical
wire concern that sits between lowering and the provider on dicts, and feeding
the converters the lowered projection is equivalent to — and simpler than —
threading Turn content through them. Wire harness byte-identical; full
non-live suite green (7130).
ChatSession.messages flips from list[dict] to list[Turn] — the in-memory
canonical trajectory. Reads migrate to typed fields (turn.role, turn.text,
turn.tool_calls); appends and assignments go through turn_from_dict /
turns_from_dicts; the fork bulk-save and retry's multipart check read via
turn_to_dict. _full_messages lowers Turns→dicts at the wire boundary — the
fold/repair and provider translators still consume dicts until the next slice.
The token-accounting helpers accept a dict or a Turn.
Non-session consumers migrate too: coordinator_idle_observer and eval to typed
fields (mypy-enumerated), and server's last-assistant extractor via turn_to_dict
(an Any-typed call site mypy could not flag). An all-text multipart content
list (the unreadable-attachment placeholder path) now round-trips faithfully
through the adapter (single text block → str, multiple → list).
Tests that inspected session.messages as dicts read it through the
dicts_from_turns / turn_to_dict bridge; those that built it pass dicts through
turns_from_dicts / turn_from_dict. Byte-identical wire harness; full non-live
suite green (7130).
reconstruct_turns is the pure row→Turn deserialize: one positional unpack of
the row tuple, one Turn per row, no wire-validity correction. The scattered
per-role dict-building and the side-channel keys collapse into typed Turn
fields (native ← {producer,blocks}, source ← _source, …); the dead tool_name
column is unpacked but unused. recover_trajectory(turns) is the load-time
trailing-strip policy, lifted out as its own function (one of lowering's three
orphan policies).
reconstruct_messages stays the dict-returning facade for now —
dicts_from_turns(recover_trajectory? · reconstruct_turns) — so every consumer
is unchanged and byte-identical (verified across the storage + reconstruct +
export + wire-payload suites, 7129 green). developer collapses into
Role.SYSTEM (zero writers, wire-identical); a bare-dict provider_data (never a
real native shape — the lane is a block list) no longer round-trips, which the
storage test now reflects.
turn_from_dict / turn_to_dict losslessly bridge the OpenAI-like message dict
(plus its _-prefixed side channels) and the typed Turn, so the migration to
Turn can proceed one boundary at a time: a dict-producing layer can be read as
Turns, and a Turn-holding layer can hand dicts to a not-yet-migrated consumer.
The _ side channels become typed fields: _source→source, _provider_content
(+_producer)→native, _event_id→meta.event_id, _attachments_meta→meta.extra.
A transitional RawContentBlock carries image/document parts verbatim until the
by-reference AttachmentRef wiring (§2/§6) relocates byte-resolution to the
translator; text parts become TextBlock so .text/FTS stays faithful.
turn_to_dict(turn_from_dict(d)) == d for every shape reconstruct and the wire
path emit (test_trajectory). No consumers yet — wiring is the next slices.
reconstruct_messages(repair=True) did two things: strip a trailing incomplete
tool-call turn AND synthesize cancellation results for mid-conversation
orphans. The mid-orphan synth was a near-duplicate of
lowering.repair_wire_messages — same detector, same contiguous insert past
interspersed system turns, same cancellation string — running at the wrong
layer (storage, on every load).
Drop it: load is now trailing-strip only (boot-crash recovery), and the
mid-orphan synth happens once, at send, in lowering.repair_wire_messages — the
single place the wire path fills orphans. The session send path gets it via
_prepare_wire_messages; export, which bypasses that path, now runs
repair_wire_messages itself (otherwise a mid-conversation orphan would
serialize as an unanswered tool_call). The duplicated cancellation string
goes with the synth — CANCELLED_TOOL_RESULT lives only in lowering now.
Safe: a bare mid-orphan is harmless between load and send (token count is
additive, /history reads repair=False, compaction summarizes to text), and
every wire path repairs it. Reconstruct tests updated to the new load
contract; an export mid-orphan test added.
The fold (representation) joins repair (validity) in the shared lowering
sibling module: fold_system_turns / _neutralize_host / _append_text_block /
drop_empty_user_turns move out of ChatSession as free functions.
_prepare_wire_messages now composes the two neutral passes plus repair, so
session.py owns zero wire-shape mutation.
The nonce stays session-minted and session-owned (_envelope_nonce binds three
consumers: the fold, the cached-prefix trust declaration, and the output-guard
forgery check) — lowering borrows it as a parameter and never mints its own.
The capability gate (supports_mid_conversation_system) is a parameter too, so
native-passthrough is unit-testable without monkeypatching a session; the
provider-None case is handled by the caller.
Pure relocation: the fold algorithm, the once-per-host neutralize ordering,
the read-only contract, and drop-after-fold are unchanged. Wire harness
byte-identical; fold unit + _prepare_wire_messages integration tests green.
Synthesizing a cancellation result for an assistant tool_call with no
matching tool result was triplicated across the translators: Anthropic's
verbatim-replay (pc_tool_ids) and rebuild branches, and sanitize_messages
for the OpenAI-compatible lanes (Chat, Responses, Google). The Anthropic
pc_tool_ids branch was also the sole repairer of a native tool_use orphan.
Lift it to one neutral policy — lowering.repair_wire_messages — run once in
ChatSession._prepare_wire_messages before the translator. It reads tool_calls
only, which is sound because the native/tool_calls mirror is enforced at save
(normalize_native_for_save): a verbatim-replay orphan is caught via its
mirrored top-level call. The translators become pure format translation and
carry no orphan synthesis.
The neutral cancellation turn carries is_error=True; Anthropic renders it on
the tool_result block, the OpenAI-compatible tool message has no such field
so sanitize_messages drops it (the C-layer translation of the flag).
sanitize_messages keeps one orphan synth of its own: a back-filled empty-id
tool_call (local servers that omit ids) is id-less when the upstream repair
runs and so invisible to it, so that lane owns its cancellation — preserving
the pre-refactor behavior for local servers.
reconstruct's load-time strip and the runtime-cancel persist-synth are
unchanged. Proven byte-identical against the per-provider wire-payload golden
harness (including a new native_orphan fixture); the harness applies the same
send-side repair the session does.
Freeze the wire payload for an unanswered native tool_use whose id is
mirrored top-level in tool_calls (the P1 invariant). This pins the
verbatim-replay orphan path each provider repairs today — Anthropic via
the provider_content tool_use synthesis, the OpenAI-compatible lane via
sanitize_messages — as the baseline the repair-unification change is
proven byte-identical against.
Replace the persisted pending/reserved/consumed upload lifecycle (and its orphan-sweep
and per-user cap) with a content-addressed, refcounted blob store fronted by the per-node
in-memory pending buffer:
- Upload stages bytes in the buffer (keyed by sha256); send-commit drains the referenced
handles, writes each blob content-addressed (INSERT-OR-IGNORE then refcount += 1, so a
stored blob is born referenced and dedupes across messages/workstreams), and records the
ordered conversations.attachments ref-list — the sole message->blob link.
- reconstruct rebuilds inline image_url/document multipart content from the ref-list,
role-agnostically (so tool-produced images via _exec_read_image now persist + rehydrate
instead of being flattened to text and lost). Output shape unchanged.
- GC is reference counting: delete_messages_after / delete_workstream decrement once per
reference and prune a blob at 0; a deduped blob shared with a kept turn (or another ws)
survives.
- get_content for a committed blob is gated by reference-ownership (the requester owns a
turn in the ws whose ref-list names the id), replacing the dropped ws_id/user_id scope.
- Migration 060 re-keys legacy consumed attachments to their content hash, dedups, sets
refcounts, writes the ref-lists, and drops message_id/reserved_*; pending legacy rows are
dropped (pending now lives only in the buffer).
Both backends symmetric; the reservation methods, cap, and orphan-sweep are removed across
storage/facade/protocol/endpoints/coordinator. Wire harness byte-identical; full suite green.
The content-addressed model writes blob bytes to workstream_attachments only at
send-commit (so every stored blob is born referenced). Pending uploads — between the
upload request and the send that references them — live in this per-node, content-hash-
keyed buffer, scoped to (ws_id, user_id) and bounded by a TTL + total-size ceiling
(OOM-safety, not the removed per-user cap). ws->node affinity (HRW routing) keeps it
process-local. Losing an unsent upload on crash/re-route is acceptable transient state.
This replaces the persisted pending/reserved/consumed lifecycle + orphan-sweep. No
consumers yet — the upload-endpoint rework and the send-commit drain wire onto it as the
attachment cutover lands.
Additive schema for the attachment cutover: workstream_attachments gains refcount +
origin, conversations gains the attachments ref-list column (migration 060 + _schema in
lockstep). Columns sit unused until the cutover, which fills them and retires the
message_id/reserved_* upload-lifecycle in favour of a content-addressed, refcounted blob
store keyed by the conversations ref-list.
Also registers the coordinator test's backend via init_storage: the attachment handlers
resolve storage through the global registry, so a bare SQLiteBackend left get_attachment
hitting a stale default db — latent until the new column made the schema drift bite.
060 now tags legacy bare-list provider_data rows with the {producer, blocks} envelope,
inferring the producer from block types — and the inference yields the exact provider_name
strings the live save writes (anthropic / google / openai / openai-compatible) so a
backfilled row compares equal to a freshly-saved one under the lowering layer's
producer==active rule. Google is keyed on a 'function' block carrying thought_signature;
xAI is byte-identical to OpenAI-Responses in the stored blocks so legacy xAI rows tag as
'openai' (bounded, self-healing). Un-inferable rows are left bare (reconstruct dual-reads
them). Paged like the envelope rewrite. Completes the producer story: 2a tags new rows,
this tags legacy. Sub-commit 2b of the canonical-trajectory storage cut.
Persist provider_data as a {producer, blocks} envelope (producer = the generating
provider's name) so the lowering layer can later replay the native lane verbatim only
to its producer and rebuild from neutral fields for any other.
The envelope is storage-only: prepare_provider_data_for_save runs the P1 mirror on the
bare block list and then wraps; reconstruct_messages dual-reads (new envelope OR legacy
bare list), unwraps to a bare _provider_content list (every consumer's contract), and
surfaces the producer on the stripped-before-wire _producer side channel. The producer
threads the same four save layers as is_error (facade -> protocol -> SQLite + Postgres);
the live assistant save tags it from self._provider.provider_name and the fork carries
_producer. Legacy rows need no migration to keep working (dual-read); the one-shot
backfill that tags them is a follow-up.
Sub-commit 2a of the canonical-trajectory storage cut.
Tool-result error state was an in-memory-only message key, lost on reload. Add an
is_error column to conversations (migration 060, backfilled False) and thread it through
the four save layers (memory facade → StorageBackend protocol → SQLite + PostgreSQL):
save_message/save_messages_bulk persist it, reconstruct_messages emits it on tool rows,
and the session tool-result + synthetic-cancel saves + the fork bulk-copy pass it. It
rides as the last conversations column so reconstruct's row-tuple positions stay stable
(legacy fixtures default False). history_decoration already prefers the persisted flag
over its text heuristic, so reload fidelity improves immediately.
First sub-commit of the canonical-trajectory storage cut (folds into rev 060).
The provider-neutral typed Turn (flat, role-discriminated; uniform tuple[ContentBlock]
content + .text; AttachmentRef by-reference content; ToolCall raw-arg str;
ProviderNative producer-tagged opaque lane; TurnMeta sidecar). In-memory foundation
for the wire-shape narrow waist — no consumers yet; storage deserialization, the
lowering layer, and the provider translators wire onto it in subsequent steps.
A max-tokens truncation mid-tool_use can leave an orphan tool_use in the native lane
(provider_data / _provider_content) with no matching tool_calls; on a same-provider
resume that replays as a tool call with no result and the API rejects it.
normalize_native_for_save strips orphan client tool-call blocks (tool_use /
function_call / function) when tool_calls is empty, applied by save_message and
save_messages_bulk in both backends; strip_orphan_client_tool_blocks enforces the
same mirror in memory at message assembly (the truncation path). The mirror now holds
by construction, so the orphan-repair pass can read tool_calls alone and the Anthropic
pc_tool_ids fallback can be retired.
Captures the exact request kwargs each provider hands to its SDK seam (Anthropic
messages.stream, OpenAI chat/responses create, Google OpenAI-compat) for a
representative set of trajectories, asserted against committed goldens. This is the
behavior-equivalence net the canonical-trajectory wire-shape refactor is proven
against. Regenerate the baseline with UPDATE_WIRE_GOLDENS=1.
Phase-2 review follow-ups:
- sec-1 (major): the skills find-zero hint interpolated the model-supplied
filter values (query/category/tag/…) into the system_reminder, which now
rides a TRUSTED operator system turn (fold fence / native system role). Under
an indirect prompt injection the model could be steered to call
skills(find, query='<directive>', category='nonexistent') so 0 rows match,
laundering the attacker text into operator authority. Drop the filter echo —
the count is harness-derived and the model already knows its own filters.
- q-1 (minor): refresh the stale :func:`escape_wrapper_tags` cross-reference in
metacognition.sanitize_payload's docstring (the function was removed; fold-time
fence.neutralize is the current marker defense).
_skill_hint spliced its guidance into the tool result as a bare <system-reminder>
block — but the operator declaration now tells the model to treat bare markers
as untrusted, silently demoting the hint. Make the hint first-class instead:
- _skill_hint returns the tool result verbatim and queues the guidance via
_queue_tool_advisory("skill_hint", ...); _collect_advisories drains it into a
{role:system, _source:"skill_hint"} turn after the clean result — folded in
the trusted nonce fence for non-native models, inline for native. (Queuing
no-ops mid-wake, like the other tool-channel advisories.)
- skill_hint added to SYSTEM_TURN_SOURCES (an advisory-producer source).
- escape_wrapper_tags removed outright: it was the last consumer, and its job
(defang a marker next to the bare block) is now covered at fold time by
_neutralize_host. The result message rides through verbatim. This also
collapses the two-escaping-mechanism confusion the review flagged.
Tests assert the clean result + the queued/drained hint, plus wake suppression.
Operator context moved to first-class system turns, leaving _reminders written
by nothing and read by nothing. Nulling it (the prior 060 step) left a writable
dead column — a foot-gun inviting accidental reuse. Drop it outright and remove
every reference in one shot so there is no half-alive state:
- migration 060: replace the wholesale null with batch_alter_table drop_column
(per migration 027); downgrade re-adds the empty column to match the 059
schema (the envelope un-wrap stays irreversible).
- _schema.py: remove the column.
- _sqlite / _postgresql: drop the reminders save param, the INSERT/bulk values,
and both SELECT columns.
- reconstruct_messages: the row tuple is now 8/9-tuple (event_id shifts from
index 9 to 8); _utils + the _row test helper updated.
- _protocol / memory save_message: drop the reminders param + docstrings.
- tests: replace the reminders-roundtrip tests with a _source-only file and a
060 drop-column assertion; remove the obsolete legacy-reminders wire test.
No production caller passed reminders=, and the SELECT no longer reads the
column, so an un-migrated DB simply ignores any residual values.
Phase-2 follow-ups to the mid-conversation-system consolidation:
- user_interjection framing (known #2): a queued message that drains mid-turn is
re-framed via render_user_interjection ("The user sent … User message: …") so
the user's words keep USER authority, not operator authority — the regression
mattered most on the native path, where the turn enters as a real role=system
message. Empty/whitespace interjections (e.g. a bare "!!!") are dropped (bug-2).
- empty-content user turns dropped at the wire boundary after the fold
(known #3): the wake pipeline's synthetic empty send("") leaves an empty user
turn on the native path (the nudge stays inline); an empty user message is
invalid on every provider. The drop runs after the fold so the fold-path wake
turn, which the nudge fills, survives.
- leading-system guard (_anthropic): a turn that converts to nothing no longer
lets a system message become messages[0] (the API requires messages[0]=user).
Newly reachable now that the empty-turn drop can expose it on a fresh-session
native wake.
- refresh stale .msg.watch-result comments (the card was removed) to describe
the current operator-bubble rendering.
Phase-1 review follow-ups:
- neutralize() now tolerates whitespace between '<' and the slash ('< /tag',
'< /tag'), matching output_guard's detection regex so a marker can no longer
be detected-but-not-defanged (a leaked-nonce break-out gap).
- Add direct tests for the sec-1 forge-in defence: _neutralize_host defangs a
forged <system-reminder_{nonce}> in both string- and list-content untrusted
hosts before the real fence is appended, and the host is defanged exactly once
so consecutive folds don't corrupt the first appended fence.
PlanReviewView was removed alongside the plan_agent built-in tool (110d44b0),
but its Discord owner-check tests were left behind importing a class that no
longer exists. They raise ImportError wherever discord.py is installed (green in
CI only because discord.py is absent there). Remove the dead test class and the
now-unused send_plan_feedback router mock from the shared bot double.
060 un-wrapped legacy <tool_output> envelopes with a loose guard (open + close)
that could irreversibly mis-rewrite a bare tool row resembling the open, and
entity-decoded the wrapper tags back to live form — re-activating injection the
old escape had neutralised (and downgrade cannot undo it).
- Require the full legacy signature (the exact </tool_output>\n\n<system-
reminder>\n join plus a trailing </system-reminder>), which wrap_tool_result
only ever emitted with advisories. A bare row with a matching close but no
advisory is left byte-for-byte untouched.
- Reverse only & -> & ; leave the wrapper-tag entities escaped so a
previously-defanged injection stays defanged.
Adds false-positive guard tests (open+close without advisory; missing tail).
Both the operator fold and the output-guard judge wrap spans in nonce-delimited
fences, but the two had drifted: the operator path minted a 32-bit nonce reused
per session with no body escaping, while the judge used a 64-bit per-call nonce
plus closing-tag escaping. Extract the shared mechanism (mint/neutralise/wrap)
into turnstone/core/fence.py and put both callers on it so they cannot diverge
again.
- Operator fold (sec-1): 64-bit nonce; fence.wrap neutralises the operator
body's close marker, and _fold_system_turns neutralises the untrusted host
turn's <system-reminder> markers once before the first fold, so a leaked or
guessed per-session nonce still cannot forge a trusted block. Per-session +
cached declaration kept (the declaration pins the exact value, so per-turn
rotation would bust the prompt cache). Marker is now <system-reminder_{nonce}>.
- Judge: refactored onto fence (behaviour-preserving; still per-call).
- Forgery detection: output_guard scans tool output for trust-fence markers —
an exact session-nonce match is HIGH (operator_marker_leak: the token has
leaked and is being replayed), any other marker LOW (operator_marker_forgery).
Removes mint_envelope_nonce / wrap_system_context (folded into fence.wrap).
Replace the two operator-context hacks (the <tool_output>/<system-reminder> content envelope and the transient _reminders side-channel) with one persistent {role: system, _source} trajectory turn. Adds supports_mid_conversation_system (claude-opus-4-8): native models take the turn inline; all others fold it into the preceding turn as a nonce-delimited <system-reminder> block declared in the system prompt as the sole trusted marker. Producers (advisories, metacog nudges, user interjections, idle/watch) emit system turns; the envelope/_reminders machinery, escaping round-trip, replay parser, and reminder SSE events are removed. Eager 060 migration un-wraps legacy envelopes. Net -1662 lines.
Known follow-ups from review (unfixed here): (1) the 060 un-wrap heuristic can irreversibly mis-rewrite bare tool rows that resemble the envelope, so do not run the migration until it is tightened; (2) user_interjection turns lost the user-framing/priority preamble (a regression, and a native-path authority-framing concern); (3) native-path wake nudge can emit empty user content.
- rerank_config.py: the runtime instruction fallback had a dead tail
(`get_rerank_instruction() or str(cs.get(...))` -- the cs.get term can only
return the registry default ""), via a stored_keys() branch that also diverged
from the calibrate CLI / endpoint. Collapse to the sibling idiom
(`cs.get(...) or get_rerank_instruction()`) so the instruction used at
calibration time matches the one used at runtime. Correct the module docstring:
ChatSession is the sole caller (the CLI/endpoint share only the instruction
precedence, not this function).
- session.py: the deferred first-turn memory recompose was gated on the flag
alone, so a synthetic wake send (empty user content -> flag stays False) re-ran
the full compose on every wake before the first real turn. Gate on a non-empty
query too, so wakes don't re-pay it and the real turn still fires exactly once
(+ test).
Accepted as-is: the __init__ compose (kept so system_messages/_agent_system_messages
are valid for early readers; one cheap extra compose per fresh session) and the
orphaned tools.rerank_* config rows (inert -- no read path, never listed or
redacted; a purge migration would collide with the 060 in flight on another branch).
Proactive memory selection scores candidates against the recent-user-message
query (extract_recent_context), but a fresh session composes the system prefix
once in __init__ while self.messages is still empty. That empty query takes the
no-context path: _select_memory_candidates returns recency order and
score_memories returns memories[:k] verbatim -- the 5 most recently UPDATED
memories, with BM25 and the reranker never invoked. send() never recomposes, so
those recency-only memories are what the model sees for the whole session (until
an unrelated event -- skill / MCP / model refresh / resume / memory write /
command -- happens to rebuild the prefix). Net effect: the injected memories are
unrelated to the actual question.
Fix: defer the memory-bearing compose to the first real user turn.
- Track _system_composed_with_context, set once extract_recent_context is
non-empty in _init_system_messages.
- send() recomposes once, right after _append_user_turn, while the flag is still
False -- so the opening turn's memory block is selected (and reranked) against
the real message. The flag then stays True, so the prefix is composed once and
stays cache-stable exactly as before (no per-turn prompt-cache churn).
This is the targeted fix; per-turn memory refresh (so later topic shifts also
re-rank) is the larger tail-injection redesign tracked on another branch. Two
adjacent gaps are left as-is for now: the reranker/BM25 only see content[:200],
and build_memory_context flat-truncates each memory to 500 chars (max_content is
the save cap, not an injection budget).
Tests: flag is False on a fresh session and after a whitespace-only wake turn,
flips True on a real query; send() runs the deferred recompose with the user
message in the query.
The reranker_alias -> model-definition path (added when reranking became a model
role) made the older global endpoint settings redundant. Resolve reranking
solely through the Reranker role and remove the parallel global config.
- Removed settings tools.rerank_url / rerank_model / rerank_api_key, their
config.py getters (+ $TURNSTONE_RERANK_URL / $TURNSTONE_RERANK_MODEL and the
module caches), and the fallback branch in resolve_rerank_client_from. The
resolver now returns a client only when a Reranker model (capability
supports_rerank, base_url = its /rerank endpoint) is selected, else None.
- Kept as global knobs: reranker_alias (the selector), rerank_web_search,
rerank_bm25, rerank_bm25_threshold, and rerank_instruction -- a task-level
query knob (Qwen3-style), not endpoint identity.
- The Settings tab is registry-driven, so the three fields disappear with their
SettingDefs. Updated the Reranker role help, example config, and docs/tools.md.
BREAKING: a reranker configured via [tools] rerank_url (config.toml / env /
Settings tab) no longer works -- add the reranker in the admin Models tab and
pick it under Models -> Roles -> Reranker. No migration: reranking is days old
and disabled by default, so any orphaned tools.rerank_* config rows are inert.
Tests: the resolver covers no-store / no-alias / non-rerank-alias -> None and the
model-definition happy path; the obsolete global-fallback tests are removed.
Adding a reranker model through the admin modal was impossible: Detect
(admin_detect_model) always ran the OpenAI /v1/models probe first and gated
calibration on its `reachable` result. A Cohere/Jina /rerank endpoint can't
answer /v1/models, so it failed both ways -- `$host/v1` passed the probe but
calibration POSTed to the wrong path, `$host/rerank` 404'd the probe outright.
- console/server.py: branch on `supports_rerank` BEFORE the probe and calibrate
the endpoint directly; that round-trip IS the reachability check (there is no
independent /v1/models signal for a rerank-only endpoint). Success
autopopulates the three calibration fields the way context_window does;
calibration failure -> reachable:False + error; empty base_url -> 400 with the
/rerank hint; reachable-but-no-clean-separation -> a note. Drops the now-dead
post-probe calibrate-on-detect block.
Reranker selection stays per-model (reranker_alias -> registry); recalibrate on
a saved reranker was already correct (it calibrates directly). Flagging a model
as a reranker still rides the capabilities JSON (supports_rerank).
Negative-tested: rerank detect skips the probe, autopopulates on success, notes
no-separation, reports unreachable on calibration failure, and 400s on an empty
base_url; the non-rerank detect path is unchanged.
Phase 3. Stores reranker calibration per-model on the model definition's
capabilities (rerank_threshold/rerank_scale/rerank_separated; a non-empty
rerank_scale is the "has been calibrated" marker) instead of a single global
threshold, populated automatically when a reranker endpoint is detected.
- ChatSession._bm25_rerank_threshold precedence: the active reranker model's
calibrated rerank_threshold (when separated) wins; calibrated-but-not-
separated -> 0 (no floor); else the global tools.rerank_bm25_threshold
fallback. Reads the raw caps dict, in-memory per turn.
- Detect (admin_detect_model) calibrates a supports_rerank endpoint and
autopopulates the three fields like context window; the create-model UI shows
a verdict chip (calibrated / no-clean-separation / not-calibrated).
- POST /api/admin/model-definitions/{id}/calibrate backs the Re-calibrate
button; calibration runs off the event loop (run_in_executor, bounded by
asyncio.timeout(90)), persists the fields + refreshes the registry, and is
graceful on a down/slow endpoint (never 500). Shared
merge_calibration_into_caps helper used by the endpoint and the CLI.
- turnstone-admin rerank-calibrate is now per-model: --model <alias> required;
--apply writes that model's caps (not the global setting). A no-separation
result records the marker (calibrated, no floor) consistently across CLI and
endpoint.
The serving lesson stays documented: Qwen3-Reranker needs vLLM --chat-template
or its scores are near-random (live-validated 0.6B + 4B: the calibrated floor
came out 0.95 vs 0.33 for the same task -- why per-model calibration exists).
Negative-tested: the floor-precedence branches (calibrated+separated -> per-
model, calibrated+!separated -> 0, uncalibrated -> global, no-alias -> global,
registry-must-not-be-consulted), endpoint persist/refresh + graceful failure,
detect-skips-non-rerankers, the CLI per-model write, and the caps-merge
preserving supports_rerank. Chip states verified via headless Chrome.
Phase 2 of BM25 reranking (follows #627). Makes the rerank_bm25_threshold floor
usable across reranker models and adds tooling to pick it.
- normalize_scores (rerank.py): map a rerank batch into a 0-1 relevance
probability -- sigmoid when any score falls outside [0,1] (logit endpoints
like bge/TEI), identity otherwise (Cohere/Jina/Qwen already 0-1). Applied in
the _bm25_reranker closure AND calibration so the threshold means the same on
every endpoint. Monotonic, so ranking order is unchanged.
- rerank_calibrate.py + `turnstone-admin rerank-calibrate [--apply]`: probe the
endpoint with labelled relevant/irrelevant groups, normalise, and recommend a
recall-biased floor -- or report "no clean separation" (a mis-served/weak
reranker). A warmup loop absorbs a cold endpoint's first-request compile so
calibration doesn't time out. Validated live against Qwen3-Reranker 0.6B and
4B: the calibrated floor differs sharply per model (~0.95 vs ~0.33 for the
same task) -- exactly why per-endpoint calibration exists.
- rerank_config.py: extract resolve_rerank_client_from(config_store, registry);
the alias/url precedence now lives in one place, shared by ChatSession (which
delegates) and the CLI.
- tools.rerank_instruction (config + setting + client): wrap the query as
<Instruct>:/<Query>: for instruction-aware rerankers (Qwen3) on endpoints that
don't apply the model's own chat template. Docs note the critical vLLM serving
detail: Qwen3-Reranker needs --chat-template or its scores are near-random and
reranking hurts retrieval.
Negative-tested: normalize sigmoid/identity branches, closure-normalises-before-
floor, calibration separation/recall-bias/warmup-absorbs-cold-start, the CLI
apply/no-apply/no-separation paths, and instruction query-wrapping through the
real httpx boundary.
Reuse the shipped Cohere/Jina rerank client as an optional post-process on
the BM25 surfaces (tool search, skill search, memory composition) via one
seam: BM25Index gains an injected reranker + a two-stage search (BM25 recall
top-50 -> rerank -> top-k). No new storage.
Gated on a configured endpoint plus tools.rerank_bm25 (default on, matching
rerank_web_search). tools.rerank_bm25_threshold (default 0.0 = off) is a
relevance FLOOR for proactive memory surfacing: BM25 always returns something,
so without a floor every-turn memory injection spends tokens on the top-k of
whatever lexically matched; the reranker score is what makes a meaningful
"inject nothing" gate possible.
Two reranker modes (BM25Index rerank_filters):
- REORDER (reactive tool/skill search): the reranker must never drop results
-> fall back to BM25 order on empty, backfill omitted pool items, so a
misbehaving endpoint can't silently lose tools.
- FILTER (memory, rerank_filters = threshold > 0): a clean empty/short result
is honoured (inject nothing) -- a deliberate divergence from
web_search._rerank_results.
Parse/endpoint failure is a discrete branch from the floor: an empty result
for non-empty input means an unparseable response (a conforming reranker
scores every doc), so the closure raises RerankError and BM25Index falls back
to BM25 order in BOTH modes -- the floor only acts on valid scores.
Also: cap the rerank client timeout at 15s (the per-turn memory path can't
afford tools.timeout's 120s default); move the Reranker alias to rerank.py
(shared, no import cycle); document the endpoint egress in the rerank_bm25
help, the admin Reranker-role description, and docs/tools.md; add
scripts/bench_bm25_rerank.py (manual, needs a live endpoint) to measure
precision@k/MRR lift and recommend a threshold default.
Negative-tested: reorder fallback-on-empty and omitted-item backfill,
filter-mode honor-empty, singleton-still-floored, the parse-fail RerankError
raise, the >= floor boundary, and pool-position-to-doc-index mapping -- each
guard reverted to confirm its test fails, then restored.
list()-materialize the reranker's output inside the guarded block so a
None / non-iterable / lazily-raising reranker falls back to native order
instead of raising out of web_search, and reject bool indices (an int
subclass) the same way _parse_hits already does.
Drop leftover web_fetch references from the rerank settings and Reranker
role help (reranking is wired into web_search only), and document both
endpoint paths (tools.reranker_alias and tools.rerank_url).
Reranking is delegated to an external Cohere/Jina-compatible /rerank endpoint
(self-hosted vLLM/TEI/llama.cpp, or hosted Cohere/Jina/Voyage); Turnstone runs
no reranker model itself. Disabled until an endpoint is configured.
- core/rerank.py: CohereJinaRerankClient (tolerant of results-wrapped and
bare-list responses) + resolver.
- web_search: rerank the SearxNG result pool by query relevance before top-k,
with a native-order fallback on error; answers/infoboxes untouched.
- Reranker as a model definition: add a model with the supports_rerank
capability and pick it under Models -> Roles -> Reranker
(tools.reranker_alias); takes precedence over the tools.rerank_url settings.
Settings: tools.rerank_url/model/api_key, tools.rerank_web_search,
tools.reranker_alias. Docs: docs/tools.md, turnstone.example.toml.
(web_fetch reranking was evaluated and dropped: for single-document chunk
selection it did not reliably beat head-truncation. Reranking is reserved for
multi-item ranking.)
`man` and `math` duplicated capabilities already reachable through
`bash`; `plan_agent` is better expressed as a `task_agent` running a
planning skill, and carried a large amount of special-case machinery
(plan-review gate, refinement loop, per-kind model routing). Removing
all three shrinks the tool surface and cuts per-call token cost.
Also removed, as dead-once-the-tools-are-gone:
- the `math` sandbox executor (`turnstone.core.sandbox`) and its
`[sandbox]` extra; the eval analyst now runs bash-only
- the read-only `AGENT_TOOLS` sub-agent tool set and the `agent`
tool-metadata key (`task_agent`/`TASK_AGENT_TOOLS` retained)
- the plan-review protocol end to end: the `on_plan_review` UI hook,
`resolve_plan`, `POST /v1/api/plan` + `POST /v1/api/route/plan`,
the `plan_review`/`plan_resolved` SSE events, and their Python SDK /
TypeScript SDK / OpenAPI / frontend / Discord+Slack bindings
- the `model.plan_alias` / `model.plan_effort` settings and the
registry `plan_model` / `plan_effort` routing fields
TOOLS 31->28, TASK_AGENT_TOOLS 13->11; COORDINATOR_TOOLS unchanged.
BREAKING CHANGE: removes the `man`, `math`, `plan_agent` tools, the
plan-review SSE/HTTP/SDK surface, and the plan_* model-routing settings
from the experimental 1.6 line.
Rename the web_search tool's `topic` parameter to `category` and expand the
enum to general/news/it/science, mapped to SearxNG `categories=`. The model can
now target the right corpus per query (e.g. `it` for code, `science` for
papers) — useful when generic engines rate-limit. The Tavily-era `finance`
topic (no SearxNG equivalent) is dropped. Threaded consistently through
_prepare_web_search / _exec_web_search / both search clients.
BREAKING: the web_search `topic` argument is now `category`.
The SearxNG change reworded _tool_write_compose's duplicate-skip return and
dropped the "identical content" phrase that test_identical_content_skipped
asserts on, failing CI (which then fail-fast-cancelled the parallel matrix
jobs). Restore the phrase (now covering all three bundled files) and assert
the searxng/settings.yml extraction in test_writes_compose_file.
Drop the Tavily and DuckDuckGo (ddgs) web_search backends for a single
self-hosted SearxNG service bundled into the docker-compose stacks.
Core:
- New SearXNGClient + _format_searxng; rewrite resolve_web_search_client to
(backend, searxng_url, searxng_engines, ...). MCP backend + oauth_user guard
unchanged. _resolve_search_client follows storage -> toml -> env -> default
precedence (explicit "" disables, via ConfigStore.stored_keys()).
- Drop the Tavily-era topic=finance (no SearxNG category); topic is now
general/news.
Settings/config:
- Remove tools.tavily_api_key, get_tavily_key, $TAVILY_API_KEY, [api].tavily_key.
- Add tools.searxng_url (default http://searxng:8080) + tools.searxng_engines,
with get_searxng_url/get_searxng_engines.
Compose + bundled config:
- Internal-only searxng service (no published API port, :ro config, /healthz
healthcheck, persistent searxng-cache volume) in both stacks; bundle
turnstone/deploy/searxng/settings.yml (JSON output on, limiter off).
- Caddy serves the SearxNG web UI on :8444 (dev: localhost-only; prod: opt-in).
- bootstrap extractor + wheel packaging updated.
Deps: drop the ddg extra + ddgs mypy override (regenerates uv.lock, removing the
lxml/h2/brotli transitives).
Docs: tools/docker/architecture/openshell + diagrams + config example + CHANGELOG;
docs/docker.md carries the AGPL-3.0 §13 operator note.
BREAKING: tools.web_search_backend no longer accepts "tavily"/"ddg";
tools.tavily_api_key and the ddg extra are removed. Run the bundled SearxNG (ships
in the compose stacks) or set TURNSTONE_SEARXNG_URL to an external instance.
Closes#545
run.sh autodetects Ubuntu/Debian, Fedora/RHEL, Arch, and WSL; ensures git and
Docker; clones, builds, picks free ports (Caddy prefers 443, Postgres 5432),
generates a .env with a JWT secret and Postgres password, asks how many nodes to
run, and starts the stack. The node count persists via an auto-loaded
compose.override.yaml, so a later plain `docker compose up -d` keeps it; a fresh
clone with no override still starts all 10. Ignores the generated override.
Lead with privacy, local-first, and no-telemetry; demote governance to an
optional team-controls line. Fix the dashboard URL (Caddy on 8443), add the
one-line installer, and link the Discord community.
The bundled production compose mounts ./Caddyfile, so write_compose has to write
it alongside compose.yaml or `docker compose up` fails to start Caddy. Update
the wizard's system prompt for the new model (no profiles, Caddy-fronted
dashboard, current ports).
Without a Discord/Slack token the channel gateway exited, which crash-loops
under `restart: unless-stopped`. Run the HTTP server and service heartbeat with
zero adapters instead — registered and idle — until a token is set. The Slack
token-pair mismatch stays a hard error.
A node refused to start with no model configured and no LLM reachable, so a
fresh cluster couldn't come up to be configured. load_model_registry now
accepts allow_empty and returns an empty registry; ModelRegistry permits the
empty state (default unset); the server passes allow_empty so a node registers
and shows in the console, then picks up models added in the admin UI live. The
CLI keeps failing fast — a REPL with no model is unusable.
Migrations run on every node at boot, and one migration rebuilds an index with
CREATE INDEX CONCURRENTLY, which can't run in a transaction and waits for all
concurrent transactions to drain. The advisory lock that serialises migrations
was held inside an open transaction, so the lock-holder's own idle-in-
transaction connection deadlocked the concurrent build when several nodes
started together. Acquire the lock on an AUTOCOMMIT connection and poll
pg_try_advisory_lock so no waiter pins a snapshot. Adds a Postgres concurrency
regression test (skipped on SQLite).
`docker compose up` from a clone builds one image and brings up the whole stack
— PostgreSQL, console, Caddy, channel, and 10 server nodes — sharing one
Postgres so the console discovers every node. The dashboard is reachable only
through Caddy (HTTP/2 avoids the browser's 6-connection cap on the dashboard's
SSE streams); the console's plain-HTTP port is no longer published. Postgres
binds 127.0.0.1 so a bare-metal turnstone-server can join the cluster — the
bare-metal overlay is folded in and removed. Insecure dev defaults keep it
zero-config; the bundled production stack mirrors the shape but pulls ghcr
images and requires real secrets.
Move the Caddyfile under turnstone/deploy so it ships in the wheel; update docs,
QUICKSTART, and .env.example to match.
The guard tests were appended via heredoc, bypassing the editor's
auto-format; ruff-format collapses one wrapped call onto a single line.
No behaviour change.
The intent-validation judge runs before the approval gate resolves, and
Smart Approvals (judge.smart_approvals) parks approve_tools on the async LLM
verdict for up to judge.timeout — so the tool-call card never reached the UI
until the judge had ruled. An operator could not see a committed call, let
alone Stop it, during that window.
approve_tools now emits a tool_pending event carrying the serialized batch at
the top of the gate, before the tool-policy lookup, the verdict wait, and the
human prompt. It is a UI paint only — no persistence, audit, or verdict
bookkeeping — so it cannot perturb the gate's accounting. The authoritative
tool_info / approve_request / tool_result events that follow upgrade the same
construct in place, keyed by call_id, and the Last-Event-ID replay slice
reconstructs it on reconnect. A ToolPendingEvent joins the SDK registry.
Coordinator: appendToolBatch was already idempotent on call_ids; the new
handler reuses the --running placeholder it already upgrades, with an
"Evaluating" kicker that swaps to "Running" on the auto-approve upgrade.
Interactive: showInlineToolBlock was create-only, so a second card would
duplicate. Added announceToolBlock + _takeAnnouncedBlock to reuse the
announced shell (matched on its call_id set) instead. The announced rail is
dashed amber and must out-specify the .msg.ts-approval--inline cyan-hold
(specificity 0,2,0) — at 0,1,0 it rendered cyan, indistinguishable from a
normal card — so the announced card is the one visually distinct surface in
the stream.
Screen-reader parity: the early paint announces politely through dedicated
off-screen aria-live regions on both surfaces (the messages log is
aria-live=off mid-stream, so the appended shell alone is inaudible), and the
announced shell carries aria-busy until the upgrade clears it. Polite, not
assertive — the human gate keeps its assertive announcement.
Tests cover the gate ordering (tool_pending precedes tool_info and the Smart
Approvals gate) plus string-guards on both UIs' wiring, the announced-rail
specificity, and the screen-reader regions.
risk_level is server-supplied and was interpolated straight into className
and data-risk at three sites — updateVerdictBadge, _buildOutputWarningEl,
renderVerdictBadge — as `risk_level || "medium"`. Whitespace, a stray case,
or a future relaxed-validation value would pass into the class string and
silently break the selectors that updateVerdictBadge, toggleVerdictDetail,
and the d-key handler rely on. It is not an XSS vector (className assignment
is text-typed), but a broken selector is a real failure.
Funnel all three through a normalizeRiskLevel() chokepoint backed by a
{low, medium, high, critical} allowlist; unknown or blank falls back to the
neutral "medium" default. Pre-existing; surfaced while rendering verdict
badges from the early-paint path.
The floor blocks only explicit heuristic deny/critical verdicts — it is
not a general "never lower the heuristic" rule. The heuristic default for
an unmatched tool is `review`, and letting a confident LLM `approve`
upgrade a `review` is the feature's purpose. Matches the implementation
and addresses PR review feedback.
Opt-in judge.smart_approvals (default off): when the intent-validation
LLM judge returns a high-confidence "approve" verdict, the tool batch is
approved automatically with no operator prompt. review/deny recommendations,
low confidence, judge errors (llm_fallback), and a deterministic heuristic
deny/critical finding all still require a human. Requires judge.enabled.
- Batch-atomic: a parallel tool batch auto-approves only if every call
qualifies; one non-qualifying call holds the whole batch for a human.
- Gate: tier==llm + recommendation==approve + confidence >=
judge.confidence_threshold (default raised 0.7 -> 0.95), with a floor
that never clears an explicit heuristic deny/critical verdict.
- approve_tools waits for the async LLM verdicts, finalises the audit
trail (AutoApproveReason.smart_approval), and re-emits verdicts after
the card so the live chip updates; the auto-approved row renders the
LLM verdict rather than the cautious heuristic carry-over.
- judge: always deliver exactly one verdict per call (fallback on error);
reject non-finite confidence so NaN can't clear the bar.
- Drop verdicts from a superseded judge generation so a reused call_id
from a prior turn's still-running daemon can't satisfy the gate's wait.
Config plumbed through the server/console/CLI builders and the live
_judge_cfg; admin Judge tab renders the toggle. Docs + example config
updated. ~35 tests covering the gate matrix, batch-atomicity, the
heuristic floor, audit stamping, the streaming re-emit, NaN/duplicate-id
defenses, and the cross-turn generation guard.
- Surface the degraded state in each node item's aria-label (only
unreachable was included), so screen readers announce it alongside
the visible DEGRADED word.
- Give the trigger an initial aria-label="Nodes" so it isn't an unnamed
control before the first snapshot render populates it; renderNodePicker
overwrites it with the live count/version once data arrives.
The trigger only showed a border on hover/open, so at rest it read as
plain text rather than a control. Add a persistent recessed box (subtle
fill + border) so it's visibly clickable, and brighten the border to
accent on hover and when open.
The always-visible NODES table dominated the coordinator-first landing
page for information most users glance at rarely. Replace it with a
compact node picker in the cluster status bar: the rightmost segment
shows "N nodes" + the cluster version (or a DRIFT chip on mixed
versions), and clicking it opens a dropdown of every compute node with
its live workstream count. Selecting a node navigates to /node/{id}/ —
the same destination the table rows linked to.
The picker reads the same /v1/api/cluster/snapshot + SSE data the table
did (via the retained buildNodeInfoFromSnapshot), so no backend change
was needed; the table was a pure client-side render. The node-grouping
/ prefix-collapsing JS and all the table CSS are removed.
Accessibility / design:
- Status is encoded by shape and colour (round = healthy, diamond =
degraded, square = unreachable), mirroring the .csb-state-dot
vocabulary, plus a spelled-out DEGRADED/DOWN word — colour alone
fails at 7px for color-blind users.
- DRIFT renders as a solid amber chip (dark text on fill) so it reads
as a real alert rather than yellow-on-yellow.
- role="menu"/menuitem (navigation, not selection), aria-haspopup,
aria-expanded; Escape / outside-click / Arrow / Home / End handling;
full node id surfaced via title when the name ellipsizes; menu height
clamped to the viewport so a long list never touches the top edge.
tests/test_console.py: assert the picker markup is served and the old
table markup (#view-overview / #node-table) stays gone.
Address PR review threads:
- _on_renewed now wraps the client-context hot-reload in try/except, so a
load_cert_chain failure can't abort the renewal callback before the
frontend-bundle update (matching the node-side reload hook).
- The startup gc_expired_certs() sweep is contained like the periodic one,
so a malformed legacy cert row can't abort the TLS block and skip the
proxy/collector mTLS client setup that follows it.
Enabling mTLS broke the cluster in three layered ways:
- Service certs were keyed on socket.gethostname() (the container ID) and
never carried the advertised service name as a SAN, so every collector and
routing-proxy handshake failed the hostname check. build_cert_hostnames()
now puts the advertised host first: it becomes the cert's primary domain
(hence a SAN) and a stable store key that survives container recreation.
- lacme's RenewalManager renews everything in the store; with the store shared
cluster-wide, every node renewed every other node's (and every dead
container's) cert — an N×M renewal storm. _SingleDomainStore scopes each
node's sweep to its own cert, and the console adds a periodic GC for the
certs of long-departed nodes.
- uvicorn loads its cert once at boot and never reloads, so renewed certs
never reached the listener and the served cert expired mid-process.
swap_context_cert() hot-swaps renewed material into the live SSL context
(server listener and console client context) via load_cert_chain.
Observability and browser access:
- The collector logged connection/TLS failures at DEBUG, so a persistent
mTLS-verify failure was invisible. It now logs the first failure per node
(reachable->unreachable) at WARNING and stays at DEBUG on retries.
- The console serves plain HTTP (it is the ACME bootstrap endpoint) and no
longer rewrites its advertised URL to https://. Browser->console TLS is
terminated by a reverse proxy: the cluster profile gains a caddy service
(browser h2/HTTPS -> caddy -> console h1.1/HTTP) plus browser-TLS docs.
Tests: tests/test_tls_san_renewal.py, tests/test_collector_reachability.py.
* feat(audio): voice I/O — speech-to-text + text-to-speech via model roles
Browser voice input/output over the OpenAI audio wire protocol, selected
through the existing model-roles system so the same code path serves OpenAI,
vLLM/vLLM-Omni, or any compatible backend — pure registry config, no new
in-process deps. Anthropic has no audio API, so it is capability-gated out of
the audio roles while remaining valid as the agent model.
Backend
- core/audio.py: role resolution + capability gating + transcribe()/synthesize()
over a registry-resolved client. Typed AudioUnavailableError (503) /
AudioBackendError (502 — body masked, SDK detail logged). Optional STT prompt.
- Endpoints POST /v1/api/workstreams/{ws_id}/speech-to-text and POST /v1/api/tts,
registered in v1_routes, write-scoped (direct + proxied), offloaded with
asyncio.to_thread. Silence -> 422; configured-but-failed backend -> masked 502.
- Model roles: audio.stt_model_alias / audio.tts_model_alias / audio.tts_voice /
audio.stt_prompt settings; Models -> Roles entries (capability-gated dropdowns,
"(disabled — voice off)" when unset). /v1/api/models exposes resolved
stt_default_alias / tts_default_alias + per-model capabilities.
- Capabilities: supports_transcription / supports_speech_synthesis on
ModelCapabilities; current OpenAI audio lineup (whisper-1, gpt-4o[-mini]-
transcribe, tts-1[-hd], gpt-4o-mini-tts) registered as known models, with a
name-inference backstop for local/openai-compatible aliases.
Frontend (interactive UI)
- Mic dictation (record -> transcribe -> fill composer for review) and
per-message playback, shown only when the role is configured.
- CSS-mask icon set, aria-pressed + live-region announcements, recording timer,
reduced-motion cue, error-typed toasts + persistent denial, mic disabled while
busy, code/math stripped before TTS.
Tests: new test_audio.py plus STT/TTS endpoint, settings, openapi, available-
models, and OpenAI-lineup capability coverage. ruff + mypy + node --check clean.
* fix(audio): use const for AUDIO_MODEL_HINTS (var-sweep invariant)
Saved Workstreams / Saved Coordinators (shared createSavedTable):
- Re-add the pagination retired by #611, capped at 20 rows/page, in the
shared component so both surfaces stay consistent. The list is fetched
whole and sliced client-side; the delete controller only sees the visible
page so Select-All stays bounded. Page resets on filter/sort, clamps on
shrink, hides on a single page or in delete mode.
- Footer is range-aware ("Showing 1-20 of N"); footer + pager share one
justified row (range left, pager right) so they read as one region.
- Saved rows carry the pointer cursor in the shared cards.css. The console
only set it on .dash-row.has-link, which the shared row builder never adds,
so saved-coordinator rows had fallen back to the default cursor.
Active Coordinators (console home):
- Give the active-coordinators block the full card chrome matching the
server's Workstreams block: a dash-header bar with an "N active / M total"
summary, the shared dash-colheaders band (was missing entirely), the rows,
and a dash-footer count line.
- Share .dash-footer into base.css (was server-only); the server keeps its
bottom-margin override.
- Make both coordinator cards contiguous by dropping the console-only
home-section gap, matching the server which ships both cards contiguous.
Frontend only -- no API, DTO, or migration changes. Pagination logic
covered by a DOM-stub harness; two designer passes applied.
make_history_handler is shared by interactive and coord, so coord
/history already trims the executing in-flight orphan turn and returns
a cursor. But coordinator.js never read it -- it connected fresh, so the
trimmed turn was neither in /history nor delta-replayed and vanished
from the dashboard (a regression vs the prior #610 in-flight render).
Mirror the ui/static/app.js fix in coordinator.js: refetchHistory takes
a seedCursor flag (default false) and seeds lastEventId from hist.cursor
only on the initial-connect path; connectSSE gates ?last_event_id= on
!= null so a cursor of 0 isn't dropped. The clear_ui / replay_truncated
re-render callers leave seedCursor false (they run on a live stream and
must not rewind the live cursor). Adds a coordinator.js static guard.
When the active UI is a MagicMock test double, _ui_event_id() returned
the auto-vivified _event_id mock (getattr finds it, so the None default
never applies). That mock reached the conversations INSERT and failed
to bind ("type 'MagicMock' is not supported"), so save_message raised,
the row was dropped, and tests on the real-storage + mock-UI path broke
(CI: test_session_attachments::test_db_row_stores_text_only).
Coerce a non-int _event_id to None so mock UIs -- and counterless
CLI/eval/placeholder UIs -- stamp NULL (the synthetic-snapshot floor),
matching the documented contract. Production UIs always carry an int,
so behaviour there is unchanged.
Also drop two redundant local `import json` in the new /history
integration tests; the module-level import already covers them.
A fresh browser connect during a parallel tool batch (e.g. several
web_fetch) left completed siblings' tool blocks empty until a manual
refresh: each tool_result SSE event fires the instant a sibling
finishes, but the result messages persist only after the whole batch
returns, so a fresh connect replayed neither the already-fired event
(a fresh connect doesn't replay the ring buffer) nor a /history row.
Route the fresh connect through the same delta replay a reconnect
already uses. Persist the per-ws SSE ring-buffer high-water mark
(_event_id) onto each saved conversation row. /history returns the
committed snapshot up to a resolved-turn-boundary cursor and omits the
trailing executing in-flight turn; the client opens its initial SSE
with that cursor (Last-Event-ID) so the existing replay_ok path
fast-forwards the in-flight turn whole -- tool blocks, results, and
approve/plan prompts all rebuild from the ring buffer.
The cut sits at the last resolved-turn boundary (not max(saved
event_id)), so out-of-order result saves in the post-batch loop can't
move it or strand a sibling. Gated on buffer-liveness (can_replay_from):
reloaded / evicted / awaiting-approval cases keep the in-flight turn in
/history and return a null cursor, falling back to the synthetic
snapshot floor -- preserving the existing in-flight render and never
leaving a turn unrenderable.
- Migration 059: nullable event_id BIGINT on conversations + a
(ws_id, event_id) index (keeps the cold-open high-water reseed a seek).
- save_message(event_id=) across the storage wrapper / protocol /
sqlite / postgres backends; get_max_event_id; reconstruct_messages
surfaces the _event_id side-channel.
- SessionUIBase: reseed _event_id from storage on construction (so the
id space stays monotonic across restarts); can_replay_from() gate.
- make_history_handler: _resume_cursor_and_trim() + cursor in the
response (WorkstreamHistoryResponse.cursor). The shared projection,
export, and coord-rebuild paths are untouched.
- app.js: seed the resume cursor on the initial-connect path only, and
gate the last_event_id param on != null so a cursor of 0 (a brand-new
workstream's first-turn boundary) is not dropped.
Tests: helper, storage round-trip, and seed unit tests; two
make_history_handler integration tests (cursor + orphan-trim when
replayable, null cursor + orphan kept when not); app.js static guards.
Migration applies up and down on SQLite.
strip_html deleted every HTML tag with no separator, gluing paragraphs,
headings, list items, and table cells into a structureless run of text
("<p>a</p><p>b</p>" -> "ab"). This degrades web_fetch, which feeds the
cleaned page to a summarising agent — and it flattens the structure any
downstream chunking/retrieval would rely on.
Block-level tags and <br> now become newlines so structure survives
("<p>a</p><p>b</p>" -> "a\n\nb"); inline tags are still dropped.
The conversion is a single linear tag scan: one pass over `<[^>]++>` with
a possessive quantifier, dispatching each tag name against a frozenset.
This replaces three full-document passes plus a 24-way alternation, and:
- Removes catastrophic backtracking (ReDoS). The earlier `<\s*/?\s*` and
`<\s*br\s*/?\s*>` patterns were quadratic on '<' + a long whitespace
run (~2s at 4k chars); the scan is now linear (~3ms at 1M chars) on the
untrusted, up-to-10MB web_fetch input. The possessive quantifier also
neutralises the pre-existing quadratic in the old `<[^>]+>` pass.
- Matches <br> carrying attributes (e.g. `<br clear="all">`), which the
first cut missed.
Tests cover block separation, inline-tag joining, uppercase tags, <br>
with attributes, lookalike tag names, and a pathological-whitespace
regression guard.
Note (pre-existing, not changed here): in _exec_web_fetch the 10 MB cap is
applied after strip_html, so the stripper sees the full fetched body. With
the scan now linear this is no longer a CPU concern; capping the raw input
before stripping remains a worthwhile defence-in-depth follow-up.
Add a workstream conversation export on three surfaces, all sharing one
serializer (turnstone/core/export.py):
- `turnstone-admin export <ws_id> [--children] [-o FILE|-]` — offline,
direct-DB. `--children` bundles a coordinator's parent conversation
plus one JSON per child into a zip (parent.json + children/<id>.json,
no manifest).
- `GET /v1/api/workstreams/{ws_id}/export` — conversation-only file
download, mounted on both the node (interactive) and console
(coordinator) lifespans via `make_export_handler(cfg)`, reusing the
/history gate ladder (permission_gate, tenant_check, list_kind
cross-kind isolation) so ownership and isolation come for free.
- Web UI — an "Export conversation" item in the interactive per-tab
dropdown (scoped to that tab's workstream) and an Export button on the
coordinator appbar.
Format is OpenAI Chat Completions messages JSON (`{"messages": [...]}`),
built from `sanitize_messages(load_messages(repair=True))`. Persisted
reasoning is surfaced on assistant messages as a flat `reasoning_content`
field (the convention OpenAI-compatible inference servers use) via a
dedicated helper that runs before sanitize strips the internal
_provider_content lane. Attachments ride along as the standard image_url
/ inlined-document content parts.
Lets users get conversations out in a portable interchange format
(backup, fine-tuning datasets, sharing, interop) without lock-in.
Closes#613.
Non-obvious decisions:
- Single format (openai-json); children/zip is CLI-only. The HTTP
endpoint and web UI are conversation-only, keeping the served surface
— and its security surface (no child rows read through the coordinator
handler) — small.
- `reasoning_content`, not the `reasoning` field /history and the
reasoning-replay path use: export targets the chat-completions
convention. Documented in export.py to prevent a "consistency fix".
- list_workstreams exposes no cursor, so the child walk passes an
explicit high limit rather than inheriting the default 100, which
would silently drop a coordinator's children past 100.
- Interactive export lives in the per-tab menu (interactive is
per-tab/pane — avoids focused-workstream ambiguity); the coordinator
is one conversation, so it keeps an appbar button.
Tested: 25 new tests through real storage + handlers (TestClient), incl.
cross-kind isolation 404, misconfig 500, the reasoning + attachment
pipeline, and the coordinator children zip. The shared frontend helper
is verified by a node sandbox harness (re-entrancy guard, button
disable/aria-busy, no-button tab-menu path). Full non-live suite green
(6714 passed); ruff + format + mypy clean; OpenAPI spec updated.
PR #612 review (Copilot): the synthetic `error` event surfaced on a fresh
connect carried no SSE `id:`, so the client's `lastEventId` never advanced.
The client's `error` handler is append-only (not idempotent like
`state_change` / `in_progress_snapshot`), and a terminal-errored idle ws
emits no live event to set a cursor — so a native EventSource reconnect sent
no `Last-Event-ID`, re-ran the fresh path, and appended a DUPLICATE error
bubble on every reconnect cycle (proxy idle-timeout, network blip).
Attach `id: str(snap_seq)` (the registration-time buffer cursor already in
scope) to the surfaced error. The reconnect then sends that `Last-Event-ID`
→ `register_listener_with_replay` returns `replay_ok` (nothing buffered past
snap_seq on an idle ws) → the handler's replay_ok branch skips the synthetic
surface. No duplicate.
Test asserts the surfaced error carries `id: snap_seq`; the existing
`test_handler_replay_ok_does_not_resurface_last_error` pins the
reconnect-skips half.
A browser connecting fresh to a workstream sitting in the error state
saw the error STATE (composer unlock + retry, via the replayed
state_change) but not the error TEXT explaining why — on a fresh connect
there was no source for it. `on_error` is never persisted as a message,
so `/history` can't rebuild it; only the reconnect path (ring buffer)
carried the original `error` event.
Surface the persisted `last_error` in `make_events_handler`'s
fresh/truncated synthetic-replay branch when the workstream is in the
error state. Gated on the error state so a healthy ws skips the storage
read, and confined to the fresh/truncated path — the `replay_ok` branch's
ring buffer already replays the original `error` event, so surfacing here
would double it. The persist (`_record_fatal_error`, sanitized) / clear
(on recovery) lifecycle already exists; this only reconstructs the event
on a fresh connect, reaching parity with reconnect.
Second of the fresh-connect replay-completeness fixes surfaced by the
audit (sibling to the tool-call `pending` fix in this branch). Non-terminal
mid-turn errors (tool parse failures, truncation — not state=error, not
persisted) remain an accepted gap; the queued-message indicator gap is
deferred (needs client-side render-on-replay).
Adds two parity tests: fresh+error → surfaced / fresh+idle → gate skips,
and replay_ok → not double-emitted.
In-flight tool calls did not render when a browser connected fresh to an
in-progress workstream mid-tool-execution; they only reappeared after the
SSE dropped and reconnected.
`project_history_messages` marked the trailing tool-call turn `pending`
from orphan-detection (a tool_call with no result) as a proxy for
"awaiting approval". But an orphan that is *executing* (already approved,
running) is orphan-but-not-awaiting. The renderer skips `pending` turns
because the SSE replay re-emits the interactive approve_request prompt
instead — and during execution `_pending_approval` is None, so nothing
re-emits. The tool call rendered from neither source on a fresh connect,
recovering only on reconnect (ring-buffer replay carries the
tool_info / tool_result events).
Regression from the REST-first history convergence (0ad1ab7f), inherited
by the wire-shape unification (#596): both swapped the `pending` predicate
from the live `_pending_approval` signal to storage orphan-detection,
which diverge exactly during tool execution.
Thread the live awaiting-approval signal from `make_history_handler` into
`project_history_messages` (new `awaiting_approval` param) and gate the
pending mark on it, re-syncing `pending` with the same `_pending_approval`
signal that drives the SSE prompt re-emit. A storage-only / closed ws has
no live session → never pending → trailing orphans render as historical.
Asserted as `dict` to match the detail handler's MagicMock-safe guard.
Adds a projection-level gate test and two handler boundary tests
(execution → renders, awaiting approval → pending). The existing
partial-trailing-turn test now asserts the turn RENDERS, not just that the
row survives — the parity gap that let this regression through.
Replaces the Saved Workstreams (ui/static) and Saved Coordinators
(console/static) card grids with a dense, sortable table that reuses the
active dashboard's row system, via one shared component in
shared_static/cards.js (renderSessionRow, SavedColumns, createSavedTable)
+ cards.css. The two surfaces differ only by column spec (MSGS vs CHILDREN)
and per-app data/ids/delete-request; everything generic is shared.
- NAME flexes to full width (kills the card grid's near-duplicate-name
truncation); client-side filter + sortable headers; scroll-all
(pagination retired); multi-select delete preserved on rows.
- Consumes the enriched saved-list DTO: MODEL, CTX (context-window
occupancy, a frozen last-activity snapshot), SKILL chip, CHILDREN, and a
red left-edge for failed runs.
- Saved rows reuse the dash-table chrome but opt out of the active table's
live-state styling: idle rows aren't dimmed, CTX reads as a snapshot (not
the live gauge), legible zebra + AA-contrast muted text for a long
terminal list, and responsive compact columns keep NAME readable on
narrow viewports.
- a11y: sortable headers exposed to assistive tech (aria-label / aria-sort
+ at-rest carets); footers are live regions.
- Removes the now-dead renderSessionCard + card-grid CSS.
GET /v1/api/workstreams/saved returned only ws_id/alias/title/created/
updated/message_count — too little to drive the planned saved-list table
redesign. Add seven fields, all sourced from already-persisted data (no
migration):
- state, kind, node_id: columns on the workstreams table
- model_alias, launch_skill: from workstream_config via LEFT JOIN
- child_count: COUNT of child workstreams via parent_ws_id
- context_tokens: most recent usage_events prompt size for the workstream
- context_ratio: context-window occupancy (context_tokens / model context
window), computed in the handler so the NULL / zero-window cases stay
explicit and identical across both storage backends
context_window comes from a model_definitions join; aliases defined only in
config.toml are absent there, so context_ratio degrades to 0.0 rather than
reporting bogus occupancy. The Python SDK reuses the Pydantic model; the
TypeScript SDK OpenAPI snapshot and hand-maintained interface are updated.
Tests cover the new storage columns (including NULL-when-absent), the
handler ratio math + zero-window degradation, and the SDK enriched
round-trip.
The Usage dashboard summary cards read the oldest day bucket
(`summary.breakdown[0]`) instead of the window SUM, so every headline
(total/prompt/completion/tool-calls/cache) showed a single day's value —
e.g. 30-day tool-calls reading lower than 7-day. Read `.summary[0]` and
collapse the redundant two-request fetch into one (the response already
carried both `summary` and `breakdown`).
Only the main streaming loop (`on_status`) recorded `usage_events`.
Auxiliary non-streaming calls — title generation, conversation
compaction, web-fetch summarization, and plan/task sub-agents — bypassed
that path and were never counted, undercounting real consumption by a
large factor for agent-heavy workstreams. Add an `on_aux_usage` UI hook
(storage row via a shared `_write_usage_row` helper with `on_status`;
`WebUI` override feeds Prometheus) and route `_utility_completion` and
sub-agent turns through it, attributed to the agent's own model. Judge
token spend remains uncounted — deferred to a follow-up.
GET /v1/api/models blanked default_alias whenever model.default_alias named
an alias absent from the server's live registry — e.g. when a standalone
turnstone-server shares a ConfigStore with a console whose model.default_alias
points at a console-only / DB alias (or the underlying model id rather than
the alias). The interactive dashboard then showed a bare "Default model"
placeholder even though a new workstream launches on a concrete model.
Mirror session_factory's _effective_default_alias / _effective_routing: fall
back to registry.default (which already incorporates a *valid* model.default_alias
override) when the configured alias is unset or foreign, blanking only if
registry.default is itself unresolvable. The endpoint now reports the model
creation actually uses.
openapi-server.json / openapi-console.json had drifted well behind
build_server_spec() / build_console_spec() — the committed snapshots are
regenerated periodically (via sdk/typescript/scripts/generate-types.py)
rather than on every schema-changing PR, so accumulated additions (skill
parsing, pending-approval items, model-definition CRUD, etc.) had not been
captured. This resyncs both with no code changes.
Addresses #603 review: the pagination control is a sibling of the cards
container, so loadDashboard()'s "Loading…" / "Failed to load" states (which
replaceChildren only the cards) left stale Prev/Next visible and still wired
to the previous _wsSavedItems cache — in the error state clicking them would
resurrect the old cards over "Failed to load". Route both transient states
through a _setSavedWsMessage() helper that clears the cards and hides the
pagination in lockstep; a successful load re-renders both via
renderSavedWorkstreams.
The interactive dashboard had drifted from the coordinator launcher in two
ways; this backports both for consistency.
Selectors: the Model / Judge Model dropdowns now show the resolved default
model in the placeholder (e.g. "Default — primary (vendor/primary)") instead
of a generic "Default model" / "Default (agent model)". The server's
GET /v1/api/models now returns judge_default_alias (mirroring the console
endpoint), sourced from the judge.model setting. It is intentionally left
blank when judge.model is unset or points at a disabled/removed alias,
because the judge then follows the per-workstream agent model at runtime
(session_factory: judge_config.model or model) — the UI keeps the honest
"Default (agent model)" wording in that case. This also fixes a latent
mislabel: the judge row previously said "agent model" even when an operator
had configured judge.model.
Pagination: Saved Workstreams now paginates at 24/page (Prev · X / Y · Next),
matching Saved Coordinators — page clamp after deletes, hidden on a single
page and in delete mode, Select-All bounded to the visible page. The empty
branch drops out of delete mode (matching the launcher) so the toolbar can't
linger over an empty grid.
The shared .pagination CSS is lifted from console/static/style.css into
shared/cards.css (loaded by both apps) so the two dashboards keep one source
of truth instead of a third copy. The pagination JS render wiring stays
per-app (it binds per-app DOM ids + controller instances) with a
cross-reference comment to its console twin.
Tests: new tests/test_server_available_models.py pins the judge/model
resolution chain (unset / configured / unknown / whitespace / registry-default
fallback).
The output-guard judge's model (judge.output_guard_model) was only
configurable on the Judge settings tab, while every other model role —
coordinator, intent judge, plan/task agents, channel adapter — lives in
Models → Roles. Add it there as a role (mirroring the intent Judge role)
and skip it on the Judge tab so it renders in exactly one place.
No backend change: the role read/write goes through the generic
/v1/api/admin/settings endpoints, the same path the intent-judge model
role already uses.
- LLM-tier row no longer duplicates the judge reasoning into its
annotations column — reasoning lives in the dedicated reasoning column,
so annotations stays heuristic-only and audit consumers aren't confused.
(The replay merge reads the heuristic row's annotations + the LLM row's
reasoning, never the LLM row's annotations, so this is display-safe.)
- Correct the output-warning chip comment: tier "llm" means the judge
returned a verdict (it may have cleared a heuristic-positive), not that
it owns the displayed finding.
After the heuristic+LLM merge, the LLM verdict no longer "overrides" the regex
verdict; it merges (risk = max, flags = union) and can raise but never lower a
regex finding. Fix the admin Settings help string to match.
Surface the output-guard LLM judge on the inline finding chip and merge it
with the regex heuristic instead of one stage winning outright.
Merge rule (issue #560, "show, annotated"):
- risk_level = max(heuristic, llm); flags = union. The judge can escalate
but never lower a heuristic positive — it evaluates adversarial tool
output, so defeating it must not erase a deterministic regex finding.
Credentials stay heuristic-only and are always redacted.
- The judge's own verdict rides along as a dissent-aware annotation
(judge_risk / confidence / reasoning / judge_model) on the chip in both
the interactive and coordinator UIs, live and on reconnect. One shared
merge_guard_display_payload drives both paths so they cannot drift.
- The model is shown the merged risk + flags but never the judge's "benign"
verdict — a fooled judge must not talk the model out of caution.
Fixes a reconnect bug: a judge that ran but failed wrote a risk="none" row
that won the replay dedup and hid the heuristic finding (it showed live but
vanished on refresh). Failed judges now persist under tier="llm_error",
excluded from the display merge; the max-merge also floors the displayed
risk at the heuristic level so the chip never vanishes.
Also adds a regression test confirming the LLM judge runs on every tool
output, not just heuristic-flagged ones.
Tests: merge unit tests, storage-backed replay regression, live/replay
wire-shape parity, SDK-event drift guard. ruff + mypy clean.
The per-message rewind / edit / retry affordance — the icon glyphs
(.icon-edit / .icon-rewind / .icon-retry) plus the inline edit-in-place
form (.msg-edit-*) and the [data-busy] / .msg-editing states — was
duplicated verbatim in both pane stylesheets: ui/static/style.css
(interactive) and console/static/coordinator/coordinator.css
(coordinator). PR #598 deferred consolidating them to keep that
coord-only change off the shipped interactive stylesheet's cascade.
Move the block into shared_static/chat.css, immediately after the
.msg-actions / .msg-action-btn primitives both panes already share, and
delete both copies (including coordinator.css's now-obsolete FOLLOW-UP
note describing the duplication).
Both index.html files load chat.css before their pane stylesheet, so the
rules land earlier in the cascade; the selectors are unique (defined
nowhere else, confirmed repo-wide) so it is a visual no-op. The block is
moved verbatim — chat.css's sibling rules use a different variable
vocabulary (--r-sm=4px / --font-mono) than the affordance block
(--radius-sm=3px / --font-ui), so renaming would change radii/fonts.
Verified pixel-identical via a headless-Chrome render-diff of both panes,
before vs after, across all four affordance states (edit+rewind, retry,
editing-open, busy): 0 differing pixels.
The capture-phase window scroll listener that dismisses an open overflow
menu also fired for scrolls originating inside the menu itself (the menu can
overflow-y:auto at high browser zoom / short viewports), so a tall menu
closed the instant you tried to scroll it. Skip scroll events whose target
is inside .admin-kebab-menu; page/ancestor scroll still dismisses.
Addresses review feedback on #599.
Admin tables render their per-row actions in an ACTIONS column whose grid
track is fixed-width and inherits `.admin-col`'s overflow:hidden +
text-overflow:ellipsis. Rows with several actions (MCP servers:
refresh/reconnect/edit/del) overflowed the track, so only the first button
showed and the rest were clipped behind a misleading "…". The edit button
was unreachable — the only way to change an MCP server was editing the DB
by hand (#593).
Replace the inline button strips across all 15 admin tables (admin.js +
governance.js) with a shared kebab (⋯) overflow menu:
- _kebabMenu()/_kebabMenuEl() build the menu; a single action degrades to
an inline button. _initKebabMenus() wires open/close, outside-click,
Escape, arrow-key nav and viewport-aware flip-up/flip-left through one
document-level delegated listener. Menu items keep the same data-*
attributes, so the existing per-table click bindings are unchanged.
- Right-align the actions column so the trigger shares an edge with its
menu; pair danger/caution items with warning glyphs (not colour alone);
theme-aware --red-glow/--yellow-glow hover tints.
Fixes#593.
_refreshRetryButton() ran only from finishAssistantStream (live turn ends), so a reloaded coordinator workstream or a clear_ui/replay_truncated re-render after rewind/retry showed assistant turns without the retry button — unlike the interactive replayHistory() path, and inconsistent with the edit/rewind buttons (which DO attach on re-render via appendUserMessageWithAttachments). Call it at the end of refetchHistory() so retry attaches on every render. (Addresses Copilot review on PR #598.)
Give coordinator workstreams the rewind/retry/edit affordance the interactive pane has, completing the #549 verb lift on the frontend. Browser-verified separately (the console won't boot in-sandbox).
- Factor refetchHistory() out of init(): the history-render block moves into a reusable function that clears the message column + tool-tracking state (toolRows / activeBatch) before re-rendering, so a mid-session re-render leaves no stale call_id->row mappings. init() keeps the first-paint-only pending-approval replay + children/tasks/attachments/SSE.
- SSE: case clear_ui re-renders from REST then dispatches the latched edit-and-resend via path-keyed /send; case replay_truncated re-syncs (skipped mid-stream). New _pendingEditSend module latch.
- Per-message affordance mirroring the interactive pane: edit + rewind buttons on every user bubble, a retry button on the last assistant turn (skipped when the turn ended tool-only, gated on .coord-tool-batch). Bare .msg.user turn-count matches the server's _find_turn_boundaries.
- Port the icon-glyph + inline-edit-form CSS into coordinator.css (.msg-actions / .msg-action-btn already live in shared chat.css; consolidating all of it into shared chat.css is a tracked follow-up).
coord-LOCAL mirror (not a shared_static extraction): app.js is class-based over this, coordinator.js is a module closure.
Lift the conversation-modifying /rewind and /retry verbs out of the body-keyed POST /v1/api/command into path-keyed POST /v1/api/workstreams/{ws_id}/rewind ({turns:N}) and /retry, as make_rewind_handler/make_retry_handler in SharedSessionVerbHandlers (template: make_close_handler/make_cancel_handler), wired on both interactive and coordinator kinds. Closes the last unlifted conversation-modifying surface — coordinator workstreams gain rewind/retry where they had none — and removes the surviving exception to the post-#422 path-keyed URL convention.
Handler shape: auth gate (coord -> admin.coordinator via permission_gate; interactive -> conversation.modify via accepted_permissions) -> busy-gate -> session.rewind(n)/retry() -> always emit clear_ui (incl. rewind-to-zero, carries #503) -> audit (conversation.rewind/retry on both kinds). Retry re-dispatch reuses the shared session_worker.send via a per-kind dispatch_retry closure (hard-reject on busy), not a third hand-rolled thread.
The web /command handler now rejects /rewind+/retry with a pointer to the path-keyed endpoint (BREAKING; 1.6.0aN-tolerant); session.handle_command's branches stay for the terminal CLI. auth.py adds the verbs to both write suffix-sets; Python + TS SDKs, OpenAPI (RewindRequest + server/console specs), the /route/ proxy mounts + audit actions, and coordinator_client all gain them.
Interactive frontend (app.js): the 3 /command POST sites + the hand-typed-slash reroute now hit the path-keyed endpoints; the bare .msg.user rewind selector is kept (matches the server's _find_turn_boundaries, which counts system-nudge user turns). The coordinator frontend rewind UX lands in a follow-up commit (browser-verified).
Tests: route-walk mount/order, /route/ audit rows, required_scope, OpenAPI catalog, SDK body-inspection, and HTTP-level handler behavior (busy-gate, turns validation, clear_ui emit, retry dispatch, audit invocation + swallow).
Register `claude-opus-4-8` in the Anthropic capability table. Opus 4.8
shares Opus 4.7's request/response surface exactly — adaptive-thinking
only (`budget_tokens` rejected), sampling params removed, the
low/medium/high/xhigh/max effort levels, `thinking.display` defaulting to
omitted, 1M context, and 128K output — so the entry is a verbatim copy of
the 4.7 row. `_lookup_capabilities` longest-prefix matching then resolves
date-suffixed ids (e.g. `claude-opus-4-8-20260601`) without colliding
with the 4.7 key.
No provider code paths change: the existing 4.7 handling already covers
all of 4.8's behavior. Models are selected via config.toml / the admin
ConfigStore UI, so there is no catalog or dropdown to update.
- _anthropic.py: new claude-opus-4-8 capability entry + effort comment
- tests/test_providers.py: opus 4.8 bare + dated capability tests
- turnstone.example.toml: bump the showcased model example to 4.8
Migrate the coordinator dashboard's init() history rebuild onto the
server-projected wire shape (the prior commit's project_history_messages),
retiring its inline raw-storage-shape handling. Field-sourcing only -- the
batch render + orphan->--running state machine is unchanged:
- callOutcomes reads the server-derived `denied` / `is_error` flags
instead of re-sniffing tool content prefixes;
- tool_calls are read flat (`tc.name` / `tc.arguments`) now that the
projection flattens the nested `function` wrapper;
- user content is a plain string and attachments come from the projected
`attachments` list, replacing the multipart walk + `_attachments_meta`
side-channel read.
Fix two latent reload bugs along the way: coord read `m.reminders` /
`m.source` but the raw shape carried `_reminders` / `_source`, so metacog
reminder bubbles and the system-nudge marker never rendered on a coord
history reload. The projection surfaces both top-level, so coord's
existing (unchanged) render paths now fire.
Repoint the test_coordinator_page deny-classifier guard at the live
`m.denied` / `m.is_error` reads instead of comment prose.
Refs #549.
Collapse the three hand-synced "raw storage -> render shape" projections
into one server-side projection. The projection previously lived in a
test-only `_build_history` (SSE-era reference impl), a client-side JS
normaliser (`history_normalize.js`, the transitional bridge), and coord's
inline `init()` handling -- drifting silently with no parity test.
Add `project_history_messages` to `history_decoration.py` and run it as the
final step of the `make_history_handler` pipeline (load_messages -> decorate
-> extract_reasoning -> project), so `GET /history` emits the canonical
render shape directly: flat tool_calls (with verdict / output_assessment),
top-level source / reminders / attachments, collapsed multipart content,
derived denied / is_error / pending, reasoning, and advisories. Interactive
`replayHistory` now consumes the payload verbatim.
Close two gaps the JS bridge deferred:
- list-content <tool_output> advisory extraction (decorate handles only
string content; the projection extracts list-carrier advisories, then
joins remaining text parts to the string the renderers require);
- orphan->pending marks ONLY the last orphan tool-call turn, so a
mid-conversation cancelled tool still renders instead of vanishing.
Delete `history_normalize.js` (+ its <script> tag and node test) and the
test-only `_build_history` (+ orphaned imports); retarget its direct tests
onto the projection helpers. Update the WorkstreamHistoryResponse
description and the Web UI Resilience architecture note to the projected
shape.
Coord's `init()` still reads the raw side-channels; migrating it to the
projected shape is the next commit, browser-verified separately.
Refs #549.
Two issues on the new REST-first history path (PR #595 review + a user repro):
- Stale history on a fast workstream switch: a slow `_refetchHistory` (e.g. a
large resumed session) could resolve AFTER the pane moved to another ws,
rendering the old ws's history over the new one — and its `.finally`
reconnecting the old stream. Add a per-pane load-generation token:
`_loadHistoryThenConnect` bumps it, and the refetch render, the deferred
`connectSSE`, and the `clear_ui` resend are each discarded when a newer load
supersedes them. Fixes the "open a child workstream, see the previously
resumed workstream's history" repro.
- Stale per-ws SSE replay cursor (Copilot): ws-assign callers set `this.wsId`
before `_loadHistoryThenConnect`, so `connectSSE`'s `wsChanged` is already
false, and `reset()` never cleared `_lastEventId` — so a tab switch / child
open sent ws-A's `last_event_id` to ws-B, mis-triggering the server's
`replay_ok` path and skipping the synthetic replay (connected / status /
in_progress_snapshot). Reset `_lastEventId` + `_lastStatusEvt` in
`_loadHistoryThenConnect` so a ws (re)load always opens a fresh stream;
transient same-ws reconnects (direct `connectSSE`) still reuse the cursor
for `replay_ok`.
test_app_js.py pins the load-generation guard.
Interactive fetched conversation history as an inline SSE `history` event on
every (re)connect — a multi-MB payload — while coord fetches it once via REST
`GET /history` and uses SSE for live deltas only. This converges interactive
onto coord's model so both kinds share one history-delivery pattern, the
prerequisite for lifting the `/command` (rewind/retry) verb to coord.
Backend (server.py, core/session_routes.py):
- `_interactive_events_replay` and the `/command` resume/rewind branches no
longer emit the inline `history` SSE event; the open/create-resume paths
emit `clear_ui` only. The `/events` stream no longer carries conversation
history — REST `GET /history` is the source (acceptable on 1.6.0aN).
- Removed the now-orphaned `events_replay_prepare` hook.
- `_build_history` retained as the canonical wire-shape reference for the
decoration/parity tests (no production callers post-convergence).
Frontend (ui/static/app.js, ui/static/index.html):
- `_loadHistoryThenConnect` fetches REST `/history`, renders, then opens SSE
(mirrors coord's `init()` ordering); wired into the seven ws-assign sites.
- `clear_ui` re-renders via a REST refetch and dispatches the edit-and-resend
latch; `replay_truncated` re-syncs (skipped mid-stream so it cannot clobber
an in-flight turn). The `case "history"` SSE handler is removed.
New shared module (shared_static/history_normalize.js):
- `normalizeHistoryMessages` converts the raw provider-native REST shape
(nested tool_calls, `_source`/`_reminders`/`_attachments_meta` side-channels,
multipart content, no derived flags) into the projected shape `replayHistory`
renders. Pure/DOM-free and node-unit-tested. This is a transitional bridge —
a server-side wire-shape unification (folding this projection back into the
server so interactive, coord, and coord's inline raw-handling collapse onto
one shape) is planned to replace it.
Tests: backend replay-omits-history regression; a node-executed normalizer
projection test (incl. the orphan->pending and denial-propagation edges); and
REST-first wiring guards in test_app_js.py.
notify was interactive-only — a coord with a natural "fan-out complete"
or "batch failed" beat could only post by spawning a child for the
single message, which is a lot of ceremony. Routing is session-kind-
agnostic in _prepare_notify / _exec_notify; this is a metadata flip
that adds the coord flag (plus the explicit interactive flag the loader
needs once coordinator is set) and updates the dual-kind whitelists,
coord tool-set assertions, and skill-author docs accordingly. Adds
two coord-session tests pinning the prepare dispatch contract
(needs_approval=False matches notify.json auto_approve) and the exec
→ channel-gateway path.
The Judge → Settings tab has its own renderer in governance.js separate
from _renderSettingRow in admin.js — it builds card-style rows inline
and was painting ``s.help || s.description`` straight into the row,
which is why the Judge settings still showed long paragraphs inline
after the prior commit on this branch.
Convert renderJudgeSettings to the same shape as the generic Settings
tab:
- Short description (s.description) renders inline below the key — the
always-visible TLDR.
- A ? button next to the key gates the long-form s.help paragraph via
a sibling .settings-help-popover with id ``<key>-help``. Picked up
automatically by the document-delegated click handler in admin.js.
Toggle behavior, save/reset buttons, and Judge-specific save endpoints
are untouched.
Settings-tab buttons assembled in admin.js and the skill-modal buttons in
index.html used two parallel rendering shapes with two binding mechanisms
(per-button addEventListener vs. document delegation). Listeners stacked
on the same DOM whenever the Settings tab re-rendered.
Converge both surfaces on the empty-button + ``data-help-target`` +
document-delegated dispatch pattern:
- ``_renderSettingRow`` emits empty <button> with ``data-help-target`` and
gives the sibling popover a matching ``id``. Per-button listener loop in
``_renderSettings`` removed.
- ``_toggleSettingsHelp`` and ``_closeAllSettingsHelp`` drop their
``.settings-label-col`` fallback branches; ``data-help-target`` reverse
lookup is the only path.
- 12 skill-modal buttons in index.html lose their literal ``?`` text so
every emitter produces empty content.
- ``font-size: 0`` on ``.settings-help-btn`` removed (workaround is no
longer load-bearing now that all emitters are empty).
Also tucks long-form help text in Judge modals behind the same ``?``:
- 8 ``.label-hint`` spans in the Create/Edit Heuristic Rule and Create/Edit
Output Guard Pattern modals (Tool Pattern, Arg Patterns, Confidence,
Pattern Flags) become ``?`` + popover. Short ``.label-hint`` strings
elsewhere in the admin UI are unchanged.
Description rows (the short TLDR under each key) stay inline as before.
The ``?`` popover carries only the long-form ``help`` paragraph and any
``reference_url`` learn-more link.
* feat(rbac): editable builtin role permissions via overlay layer
Adds a ``role_permission_overrides`` table that stores per-(role_id,
permission) grant/revoke deltas, applied on top of the immutable
``roles.permissions`` baseline at permission-load time. Builtin roles
(``builtin-admin/operator/viewer``) become customizable through the
admin Roles UI without losing the "reset to default" guarantee — every
override is auditable and reversible.
Motivating case: ``model.skills.write`` is deliberately default-ungranted
on every role so operators must consciously opt in before a coordinator
session can mutate the skill catalog. Until now there was no UX path to
do that opt-in — the only options were dropping into SQL or running a
fresh migration. The overrides editor closes that gap.
Backend
- Migration 057 + storage methods on both sqlite + postgresql backends
- ``get_user_permissions`` merges baseline ∪ grants − revokes for builtin
rows; custom rows pass through unchanged
- ``GET /v1/api/admin/roles/{id}/effective`` for inspect
- ``PUT /v1/api/admin/roles/{id}/overrides`` for write — admin.roles gated,
audited, validates against ``_VALID_PERMISSIONS``, refuses non-builtin
targets, strips no-op grants/revokes before persisting
- Lockout guard: cannot revoke ``admin.roles`` if doing so would leave
zero users with the permission (returns 409)
- ``coordinator.trust.send`` added to ``_VALID_PERMISSIONS`` — was
seeded into builtin-admin by migration 042 but never registered with
the validator, so the very first round-trip through the editor 400'd
on it. Drift-detection test guards future migrations from recreating
the same gap
Frontend
- Roles tab redesign: chevron + permission-count chip replace the
"..." truncation; expand-on-click drawer groups perms by namespace
with baseline / grant (green +) / revoke (red −) chip variants
- Edit modal opens for builtin rows ("Customize Built-in Role" title);
toggles show baseline-default vs override state; submit diffs against
the rendered toggle universe (not raw baseline) so future taxonomy
drift can't silently strip unknown perms
- "Modified +N/-N" pill on rows with active overrides; "Reset to default"
drawer action clears the override set
- ``_PERMISSION_SECTIONS`` brought up to date with all currently-seeded
perms (admin.coordinator, admin.cluster.inspect, admin.models,
admin.nodes, admin.prompt_policies, conversation.modify,
coordinator.trust.send were missing)
Tests
- 7 storage tests covering set/list/clear/effective + overlay merge into
``get_user_permissions`` for both builtin and custom roles
- 11 endpoint tests covering effective/overrides happy paths, validation,
lockout guard, builtin-only restriction, no-op normalization, list
enrichment
* feat(rbac): enforce workstreams.{create,close} + tools.approve gates
These three permissions were declared in ``_VALID_PERMISSIONS``, seeded
into ``builtin-operator``'s baseline by migration 008/017, surfaced in
the admin Roles UI as toggles, and documented in ``bootstrap.py`` as
the operator role's capabilities — and never enforced anywhere. The
audit that ran out of the overlay PR found zero ``require_permission``
sites for any of them; any authenticated user could create workstreams,
close any workstream, or approve any pending tool regardless of role.
Behaviour change for callers without the perms:
- ``POST /v1/api/workstreams/new`` (node + console proxy variants)
now 403 without ``workstreams.create``
- ``POST /v1/api/workstreams/{ws_id}/close`` (and ``/route/`` proxy)
now 403 without ``workstreams.close``
- ``POST /v1/api/workstreams/{ws_id}/approve`` (and ``/route/`` proxy)
now 403 without ``tools.approve``
The OR-fallback to ``admin.coordinator`` keeps coord sessions spawning
interactive children unblocked without needing operator-style perms.
Service-scoped inter-cluster calls bypass via the existing
``allow_service_bypass`` path on the new ``require_any_permission``
helper. Builtin admin and operator both already carry these perms;
viewer correctly loses workstream create/close/approve (it already
couldn't do those in spirit).
Implementation
- ``require_any_permission`` (core/auth.py) — OR-semantics variant of
``require_permission`` with per-conditional comments documenting the
security policy at the choke point. 403 body names every accepted
perm so operators get an actionable remediation
- ``make_{create,close,approve}_handler`` (core/session_routes.py)
accept ``fallback_permissions: tuple[str, ...]`` — checked only when
``cfg.permission_gate is None`` (interactive case). Coord's
``permission_gate=_require_admin_coordinator`` continues to take
precedence on the coord-config side
- Console-side ``create_workstream`` and ``route_create`` inline the
same OR check before proxying — fail fast on a forbidden request
without burning a cluster round-trip
- ``route_proxy`` adds a verb-scoped gate on ``approve`` and ``close``
only; ``send``/``cancel``/``dequeue``/``command``/``plan`` remain
authenticated-only (pre-existing, out of scope for this audit)
Tests
- New ``TestPermissionGatesOnLifecycle`` (4 tests) in test_server_authz
pinning 403-without-perm + non-403-with-perm at the node lift sites
- New ``TestRouteProxyPermissionGates`` (5 tests) in
test_console_routing_proxy covering 403 paths, OR fallback via
``admin.coordinator``, and that ``send`` remains ungated
- ``_make_jwt`` helpers in test_server_authz, test_close_reason_
persistence, test_server_attachments_on_create updated to embed
operator-shaped perms by default so existing tests continue to
exercise the post-gate logic rather than 403'ing on the new check
Docs
- ``bootstrap.py`` operator role line corrected to list every perm
it actually carries (was missing ``tools.approve`` and
``conversation.modify``)
* fix(rbac): close lockout + escalation gaps in role-overrides editor
Three issues surfaced by /review of the overlay layer and gate uplift —
all in the RBAC/auth surface, treated as zero-days.
**F-1: lockout guard misses the grant-removal path.** PUT-replace
semantics on ``set_role_overrides`` mean an existing grant of
``admin.roles`` (added via override to e.g. builtin-operator) is
silently dropped when the new payload omits it. The previous guard
short-circuited on ``"admin.roles" not in revokes`` and never noticed.
Concrete cluster-bricking scenario: grant admin.roles to operator via
override, unassign builtin-admin, click "Reset to default" on operator
→ all users lose admin.roles, recoverable only via SQL.
The rewritten guard simulates the post-PUT effective set on the target
role directly: if ``(baseline | new_grants) - new_revokes`` lacks
admin.roles AND nobody holds it via another role, refuse the change.
The "via another role" question is answered by one bulk query rather
than the prior O(users × roles) round-trip loop.
**F-3: lockout check blocked the event loop on moderate deployments.**
The prior check called ``storage.list_user_roles`` per user and
``storage.effective_role_permissions`` per (user, role) pair —
synchronous SQL inside an async handler. 200 users × 5 roles = 1000
connection cycles long enough to trip reverse-proxy timeouts on a
permission revoke.
Replaced with ``storage.users_with_permission(perm, *,
exclude_role_id)`` — one join over ``user_roles ⋈ roles`` plus one IN
fetch on overrides for the builtin role ids in the result, folded
in-process. Two queries total, independent of cluster size. The whole
check now runs under ``asyncio.to_thread`` so even the bulk read
doesn't stall the loop.
**F-2 reframed: admin_assign_role's subset check ignored the overlay.**
The check at lines 6321-6328 reads ``target_role.get("permissions",
"")`` (baseline column) when computing the perms it requires the
caller to hold. After this branch, an admin.roles holder can grant
e.g. ``model.skills.write`` to builtin-operator via override; an
admin.users holder (who happens to NOT hold that perm) could then
assign operator to a new user, silently escalating the assignee. The
existing two-person-rule by perm split (admin.roles for catalog edits,
admin.users for assignments) only holds if the assignment-time check
considers the overlay. Switched ``target_perms`` to
``storage.effective_role_permissions(role_id)["effective"]``.
Note: this PR retains the existing model where admin.roles is the
catalog-edit superuser (admin_create_role, admin_update_role, and now
admin_role_overrides all skip the caller-holds-grants check). The
two-person rule against escalation lives at the assignment gate, which
this fix reinforces.
**F-7: delete_role left orphaned override rows.** No FK on
``role_permission_overrides.role_id`` (migration 057 omitted FKs to
match the rest of the governance schema). Added explicit cleanup in
both sqlite + postgresql ``delete_role`` implementations so a
re-seeded role_id (deterministic for builtins on schema reseed) can't
silently inherit stale overrides from the prior occupant.
Tests
- storage: ``test_users_with_permission_bulk`` exercises the new bulk
helper including ``exclude_role_id`` and overlay folding
- storage: ``test_delete_role_cleans_up_overrides`` pins the F-7 fix
- endpoint: ``test_overrides_lockout_guard_blocks_grant_removal`` is
the F-1 reproduction — operator-overlay grants admin.roles, builtin-
admin has it removed, attempting to reset operator's overrides 409s
- endpoint: ``test_assign_role_blocks_escalation_via_overlay_grant``
pins the F-2 reframed fix — overlay-poisoned operator can't be
assigned by a caller missing the overlay perms
* refactor(rbac): cleanup batch from /review (#584)
Five non-security findings folded into one commit so the security
batch stays focused. All consistent with the existing intent of
``feat/builtin-role-overrides``.
**F-4: presence check on ``_effectivePerms``.** ``governance.js`` was
guarding on ``Array.isArray(role.effective) && role.effective.length > 0``,
falling through to splitting ``role.permissions`` (the baseline) when
the array was empty. For a builtin role whose overrides legitimately
revoke every baseline perm, that path silently rendered the baseline
chips with no override indicators — the inspector lied about what the
role can do. ``_enrich_role`` always sets ``effective: []``, so
presence is the right sentinel.
**F-5: JS-side drift detector.** Commit 1 added a Python-side test
asserting ``_VALID_PERMISSIONS`` covers every baseline perm; the
mirror invariant on the frontend went uncaught. A new perm added to
``_VALID_PERMISSIONS`` without a matching entry in
``_PERMISSION_SECTIONS`` becomes silently un-customizable through the
admin UI (the only documented grant/revoke path). Test parses the
JS const out via regex and asserts set-equality both directions —
detects "missing in UI" and "extra in UI" so the toggle catalog and
validator can't fork.
**F-6: bulk enrich for ``admin_list_roles``.** Was ``1 +
2*builtin_count + 1*custom_count`` SELECTs per admin-tab open;
collapsed to one ``IN``-filtered query via new
``storage.effective_role_permissions_bulk(role_ids)``. Implemented
on both sqlite + postgresql backends following the existing
``effective_role_permissions`` shape.
**F-8: rename ``fallback_permissions`` → ``accepted_permissions``.**
The lift body uses ``if cfg.permission_gate / elif accepted_permissions``
— mutually exclusive — so when ``permission_gate`` is None this IS
the primary gate, not a fallback to anything. The "fallback" name
suggested a tier-2-after-tier-1 semantic that didn't exist. Renamed
across ``make_{approve,close,create}_handler`` factories, the three
call sites in ``turnstone/server.py``, and the docstrings.
**F-9: positive lift-level tests for ``admin.coordinator``-only.**
``TestPermissionGatesOnLifecycle`` previously had a single positive
test for ``workstreams.create`` alone, plus negative-403 tests for
each verb without perms. The OR-fallback to ``admin.coordinator``
(which keeps coord sessions spawning interactive children unblocked)
had no positive coverage at the lift code path — only at the proxy,
which exercises a different verb-dict gate. Added three tests
(create / close / approve) that pass ``admin.coordinator`` alone and
assert non-403, so a future tightening of the accepted_permissions
tuple can't silently regress coord-driven child workstreams.
Out of scope: nit perf-4 (event-delegation refactor on
``_renderGovRoles``). ``setSafeHtml`` rebuild is the existing
pattern across every admin tab; rewriting one tab's render path on
this branch would be drive-by inconsistent with the surrounding
codebase. Filed as a separate concern if the Roles tab grows past
the scale where it bites.
* fix(rbac-ui): aria-expanded + row-click on Roles drawer (#585)
Two Copilot review findings on governance.js:
- Expand button was missing aria-expanded — screen readers couldn't
announce drawer state. Now reflects the row's expanded flag.
- Comment said "row + chevron both work" but only the chevron was
wired. Added data-expand-role to the row element too so the
existing handler loop (querySelectorAll on the attribute) picks up
both — clicking anywhere in the role row toggles the drawer.
Edit/Delete handlers already stopPropagation so they aren't
triggered by the row-level click.
* fix(migrations): rebase role_permission_overrides to 058
PR #560 mitigation #1 landed 057_output_assessments_llm_judge.py on
main in parallel; my migration claimed the same number, forking
alembic's head and breaking postgres. Renumbered to 058 and
re-pointed down_revision at 057 so the chain stays linear.
No behaviour change — same DDL. Full sweep clean (6730 passed).
* fix(migrations): update 058 revision strings to match filename
Previous commit (ea86aefc) renamed 057_role_permission_overrides.py to
058_* but the in-file revision = "057" / down_revision = "056"
strings stayed — leftover from when the file shipped as 057. Tests
pass because alembic walks the chain by revision string, and the
strings now correctly read revision = "058" / down_revision = "057"
to make the chain linear with main's 057_output_assessments_llm_judge.
Caught locally before re-running CI; my prior `git mv` + content edit
landed as a staged rename + unstaged modification on the previous
push.
`float("4.20") == 4.2`, so the old version-sort routed `grok-4.20`
under `grok-4.3` despite 4.20 being the newer dated-snapshot line.
Parsing each component as an int via `_version_tuple` makes
`(4, 20) > (4, 3)` as intended.
Applied symmetrically to the openai branch — same shape, same latent
bug against a future `gpt-5.10` vs `gpt-5.2` collision.
Locked in by four tests in `test_provider_xai.py::TestSelectBestModel`.
Spotted by Copilot review on #586.
Adds xAI as a first-class commercial provider through the officially-
documented server-to-server API-key path against https://api.x.ai/v1.
XAIProvider is a thin subclass of OpenAIResponsesProvider; xAI's
Responses surface is OpenAI-shaped, so the only override needed is
_build_kwargs, which merges <tool>_call_output strings into include[]
so xAI's server-side tool outputs (hidden by default) become visible.
GROK_CAPABILITIES covers the five documented chat models; aliases
like grok-4.3-latest resolve via the existing longest-prefix lookup.
Two narrow base-class generalisations earn their keep beyond Grok:
- ModelCapabilities.server_side_tools: tuple[str, ...] drives the
Responses-surface tool injection (previously hardcoded to
web_search). resolve_server_side_tools folds in the legacy
supports_web_search boolean for backward compat.
- extra_headers: dict[str, str] | None threaded through
LLMProvider.create_streaming / create_completion so callers can
pass x-grok-conv-id: <ws_id> for prompt-cache hit-rate. Session-
side population is a follow-up; the plumbing lands here.
Out of scope:
- OAuth (SuperGrok / X-Premium+) — not officially documented.
- Chat Completions surface — deprecated on xAI's comparison page.
- Image / voice / video models.
- argparse --provider xai in cli.py / server.py — Google isn't
there either; both providers configure via config.toml.
Closes#583.
Validated each of the 7 substantive Copilot findings + 5 CodeQL
findings via source spike; applied 7 (CP2-CP7, C3-C5), refuted 2
(CP1, C1+C2) with citation.
* CP2 (session.py:3551) — pre-truncation budget was reused per
output, allowing N parallel tool results to each claim the full
remaining context budget and collectively overflow. Maintain a
running budget that shrinks as each output is sized.
* CP3 (ratelimit.py) — TokenBucket.consume() mutated tokens /
last_refill without a lock; ChatSession._batch_evaluate_outputs
invokes it concurrently from up to 4 worker threads, so the rate
limiter's stated 60-call/min cap was best-effort. Added an
internal threading.Lock that protects consume() and retry_after.
RateLimiter's outer lock still holds for the bucket-dict it owns;
the new lock just makes the class safe-by-default for direct
consumers.
* CP4 (session_ui_base.py:1642) — docstring claimed "llm" rows
were only persisted on success; the session actually persists a
failure row too (with reasoning=error_reason) for audit. Match
the docstring to behavior.
* CP5 (migration 057) — header said "Revises: 055" but
down_revision was "056". Fix header.
* CP6 (_protocol.py:1761) — :func:`intent_verdicts` is a table,
not a callable. Plain reference.
* CP7 (test_output_guard_judge.py:271) — test_client_created_once
was self-contradictory: header comment claimed caching was being
verified, body asserted the patched fake was called once per
evaluate (no caching). The next test
(test_real_lazy_init_caches_real_client) covers actual caching;
drop the misleading one.
* C3-C5 (_extract_json) — three empty except-pass blocks now carry
a justifying one-liner explaining each strategy's expected failure
mode and what falls through.
REFUTED:
* CP1 (output_guard_judge.py:453 catch of TimeoutError vs
concurrent.futures.TimeoutError) — in Python 3.11+ (our minimum,
per pyproject.toml requires-python = ">=3.11") the two are the
same class. Verified at runtime: ``cf.TimeoutError is
TimeoutError`` is True; MRO is (TimeoutError, OSError, Exception,
BaseException, object). The polling loop catches the right
exception.
* C1/C2 (Protocol method bodies and test stubs using ``...``) —
`...` is the idiomatic Python pattern for Protocol method bodies
and one-line stub functions; the surrounding code in
test_model_registry.py uses ``...`` consistently across all stub
methods. CodeQL is flagging a single instance while ignoring
identical patterns nearby.
Adds a second, LLM-driven stage to the output guard so domain-camouflaged
prompt-injection payloads that the regex stage misses (arXiv:2605.22001 —
Llama 3.1 8B evades the existing regex set on ~90% of camouflaged
prompts) get caught before the tool output lands in the assistant's
context.
## Surface
* New `OutputGuardJudge` in `turnstone/core/output_guard_judge.py` —
synchronous, single-shot LLM call. Inlines the alias-resolution +
client-config + JSON-parsing helpers (copied verbatim from
`IntentJudge` at `judge.py:917-969` / `1604-1659`) rather than going
through a shared module — when `IntentJudge` lifts its own helpers,
both copies move together.
* JSON-in-content verdict with a 3-strategy parser (direct / markdown
fence / balanced braces). `IntentJudge` ships a 4th regex-field
fallback; OutputGuardJudge deliberately doesn't, because strategy-4
hits on broken LLM output can extract a "verdict" from the model's
reasoning quote that lands in storage looking identical to a clean
strategy-1 result. Failure of all three returns
`error="unparseable_verdict"` and the heuristic stage stands.
* `OutputJudgeVerdict` is a frozen dataclass with:
`risk_level` (none/low/medium/high — normalises `critical`→`high`
and `info[rmational]`→`low` for IntentJudge-echo safety),
`flags: tuple[str, ...]`, `reasoning`, `confidence: float`
(0.0-1.0, parsed + clamped from the LLM's self-report;
pass-through to audit, no threshold gating), `judge_model`,
`latency_ms`, `error`.
* Real wall-clock timeout via `ThreadPoolExecutor.shutdown(wait=False,
cancel_futures=True)` on the timeout/cancel path — `with ... as ex:`
would block return until the worker drained. 1s `cancel_event`
poll mirrors `IntentJudge._run_judge` at `judge.py:1117-1118`.
* HTTP client lazy-init + reuse for the judge instance's lifetime.
Session-side model swap drops the entire judge, dropping the client
with it.
* Untrusted tool output wrapped in per-call random-nonced
`<tool_output_NONCE>...</tool_output_NONCE>` fence. Closing-tag
substrings in the raw text are case-insensitively backslash-escaped
first (`</tool_output` → `<\/tool_output`) so an attacker can't
break out even if they guess the nonce. System prompt classifies
the fenced region as UNTRUSTED DATA so directives inside are
evaluated as content, not obeyed.
* Judge user prompt carries the heuristic verdict (risk + flags +
annotations), the tool description (looked up from the session's
tools registry), and the tool args (truncated to 500 chars, also
classified UNTRUSTED in the system prompt since they may be
caller-supplied). Lets the judge defer to the regex on credential
leaks and focus on injection signals the regex set misses; also
enables output-vs-request plausibility reasoning.
## Session integration
* `_evaluate_output(call_id, output, func_name, *, tool_args="")` —
heuristic always runs; LLM stage runs when `judge.output_guard_llm`
is enabled. When the LLM produces a usable verdict and the
heuristic didn't detect credentials, the LLM verdict is acted on;
otherwise the heuristic stands.
* Credential redaction is a regex-only signal. When `heuristic.
sanitized` is non-None, the heuristic owns the acted assessment
regardless of what the LLM said — an LLM asked about prompt-
injection can correctly label a credential-bearing output as
"none" risk for injection, but the secret still needs redaction.
* `_batch_evaluate_outputs` runs the per-tool guard concurrently
(4-worker pool) when LLM is enabled and there are ≥2 string
outputs — collapses N×LLM-latency to ⌈N/4⌉×latency on the common
5-20 tool-calls-per-turn turn.
* Per-session `TokenBucket(rate=1.0, burst=60)` caps adversarial
LLM-fan-out cost at 60 calls/min/session.
* Pre-truncation: the per-tool loop truncates output before the
judge sees it, so the judge evaluates exactly what enters the
assistant's context (no wasted tokens on text that won't land).
* Both heuristic and LLM tier rows persisted to `output_assessments`
when the LLM ran (audit completeness); heuristic-only rows skip
when matched-clean to keep the table focused.
## Storage
Migration 057 extends `output_assessments` with five LLM-tier
columns: `tier` (`heuristic` / `llm`, backfilled to `heuristic`),
`reasoning`, `judge_model`, `latency_ms`, `confidence`. Tie-break
on `(created DESC, tier='llm' first)` so downstream consumers see
the acted verdict first when the two rows tie at second resolution.
`StorageBackend.record_output_assessment` + sqlite/pg implementations
+ `SessionUIBase.record_output_assessment` + `SessionUI` protocol +
the test stub overrides (cli, eval, 9 test files) all take the new
LLM-tier kwargs.
## Config surface
Three new judge.* settings in `settings_registry`:
* `judge.output_guard_llm` (bool, default False) — capability gate.
Default off; operators opt in once a small/fast model is pointed
at `output_guard_model`.
* `judge.output_guard_model` (str, default "") — alias for the LLM
stage. Empty inherits the session model (same fallback shape as
`judge.model`).
* `judge.output_guard_llm_timeout` (float, default 30.0, min 1.0) —
wall-clock budget per call.
Both `server.py` and `console/session_factory.py` wire these into
the `JudgeConfig` they hand to `ChatSession`.
## Notes
* No backwards-compatibility shims — the LLM stage is purely additive.
* No reasoning/threshold gating on confidence; it rides as an
audit-only signal per maintainer direction. Surface it in the
`on_output_warning` dict so live UI / cluster broadcast can sort
flagged outputs by judge certainty.
* Tests: 392 lines of judge-only coverage (`test_output_guard_judge.
py`) + 629 lines of session-integration coverage in `test_session.
py`, plus the storage and stub-shape updates.
Two related cleanups landed together because they touch the same surface
(skill-spec uplift PRs #569/#570/#571/#572):
1. Wording: replace "Anthropic spec" / "Anthropic Claude Code skill spec"
with "SKILL.md spec" across admin UI tooltips, code comments, test
docstrings, migration 056's module docstring, and the user-facing
`arguments` description in tools/skills.json. Renames a parser test
`test_anthropic_tags` -> `test_nested_metadata_tags` and consolidates
a parse-API test of the same shape; fixture author renamed
`Anthropic` -> `Acme` to keep the fixture neutral. Legitimate
provider/SDK/API references (provider name, api.anthropic.com,
`_anthropic.py`, capability comments) are intentionally untouched.
2. Admin UX: in the Create + Edit Skill modals, six fields per modal
(Compatibility, Paths, Hide-from-skill-picker, Arguments, Argument
hint, Activation) had long uppercase label-hint spans crammed into
the visible label. Migrated each to the existing
`.settings-help-btn` + `.settings-help-popover` pattern already used
in the Settings tab — short label + inline `?` button that opens a
styled popover with proper `<code>` formatting for technical tokens.
Pattern reuse required two small generalisations in admin.js:
* `_toggleSettingsHelp` now looks up the popover via a new
`data-help-target="<id>"` attribute first, falling back to the
settings-tab `.settings-label-col` ancestor lookup.
* `_closeAllSettingsHelp` mirrors the same dual-path lookup when
resetting `aria-expanded`, so modal buttons don't get stuck on
`aria-expanded="true"` after another popover opens.
* Added a document-delegated click handler that fires only for
buttons with `data-help-target`; existing per-button binding
in the settings-tab render path is unchanged.
CSS: `.settings-help-btn` now paints its `?` via `::after` with the
button's own `font-size: 0`, so prettier-introduced whitespace
inside the new HTML buttons can't off-center the glyph. The same
rule applies to existing admin.js-generated buttons (text content
hidden, pseudo identical). Small additions for
`.settings-help-popover code` / `strong` styling so technical
tokens render with the same monospace pill treatment used elsewhere
in skill UI.
Known follow-ups (intentionally NOT in this PR):
* Migrate the settings-tab `_renderSettingRow` button assembly to the
empty-`<button>` + `data-help-target` form so the per-button
addEventListener loop can be dropped in favour of pure document
delegation, and the `font-size: 0` rule stops being a workaround for
two markup styles.
* The 12 new popover blocks are duplicated verbatim between the
Create and Edit modals (same as the rest of the create/edit modal
pair). A small renderer that emits popovers from a shared data
object would eliminate the drift risk but is unrelated cleanup.
* fix(coord): strip intent-judge verdicts from inspect_workstream output
Coordinator LLMs repeatedly misread `user_decision="policy"` (the label
meaning "auto-approved by an admin policy allow rule") as "blocked,
waiting for policy review" — combined with `recommendation="review"`
(the heuristic judge's risk class, not a workflow state) the verdict
fields read end-to-end as "stuck on policy review" and produced
incorrect cancel-and-respawn reasoning against healthy children.
The blocking signal already lives on `state` (`"attention"`) and the
`live.pending_approval` block, both still in the result. Verdict
history remains queryable through admin / audit surfaces — only the
LLM-facing inspect surface drops them.
Also drops `verdict_count` / `verdicts_by_risk` from the tier-3
skeleton fallback, deletes the now-dead `_serialize_verdicts` helper,
and clears the now-stale `"verdicts": []` keys from 10 fixture sites
that fed `_format_inspect_tiered` test cases.
* fix(coord): correct comment pointer — inline comment, not docstring
Implements the Anthropic Claude Code skill spec's placeholder
substitution end to end. The renderer in ``_substitute_skill_args``
handles every spec form except ``\${CLAUDE_SKILL_DIR}`` (deferred):
* ``\$ARGUMENTS`` — full args string as the user/model typed it
* ``\$ARGUMENTS[N]`` / ``\$N`` — Nth positional arg, ``shlex.split``-parsed
* ``\$<name>`` — named arg from the SKILL.md ``arguments:`` list
* ``\${CLAUDE_SESSION_ID}`` / ``\${CLAUDE_EFFORT}`` — session state
Substitution is single-pass (one combined regex, one ``re.sub``).
Append rule: when args are passed but the body has no bare
``\$ARGUMENTS``, append ``ARGUMENTS: …`` at the end.
## Surface
* Parser: ``arguments:`` (list/space-delim) + ``argument-hint:`` (str)
extracted into ``ParsedSkill``.
* Install: persists both to the pre-allocated columns from migration
056 (PR #574). Install path clamps ``argument_hint`` to 128 chars
to match the admin-create cap (untrusted upstream source).
* Admin: ``CreateSkillRequest`` / ``UpdateSkillRequest`` accept both
fields; create + edit modals get inputs; parse-preview echoes.
* Renderer: ``_substitute_skill_args`` runs AFTER ``_render_template``
in ``_load_skills`` so user-supplied args containing ``{{var}}`` can't
be re-expanded by the legacy renderer.
* Session: ``_skill_arguments`` plumbed through ``__init__``,
``set_skill``, and ``_save_config`` so a resumed workstream re-renders
with the original arg payload.
* Model tool: ``skills(action='load')`` accepts an ``arguments`` string.
Approval label includes a SHA-256 digest of the args so a once-
approved skill name can't grant cover for a future payload; preview
surfaces the args inline.
## ``/review`` findings (addressed)
* ``\${CLAUDE_EFFORT}`` referenced ``self._reasoning_effort`` — wrong
attribute; the real one is ``self.reasoning_effort``. Always rendered
empty. Fixed.
* Two-pass layering let user args containing ``{{var}}`` re-expand.
Render order reversed.
* ``_skill_arguments`` wasn't in ``_save_config`` — resumed workstreams
silently lost their payload. Added.
* Approval label omitted ``arguments``. Digest + preview added.
* Install path didn't bound ``argument_hint``. Clamped.
* Added ``_skill_arg_names`` decode tests + "load same skill,
different args → re-render" invariant test.
## Copilot review findings (addressed)
* ``skills.json`` tool description was inaccurate about ``shlex``
stripping quotes and "empty string disables substitution". Rewrote
to match actual behaviour.
* Named-argument regex was stricter than parser/storage contract.
``arguments: [issue-number]`` would partial-match ``\$issue-number``
as ``\$issue``, leaving ``-number`` as stray text. Broadened the
regex to ``[A-Za-z_][A-Za-z0-9_]*`` AND added validation at
``_skill_arg_names`` decode time so names not matching the regex
are dropped with a warning.
## Tests
* ``tests/test_substitute_skill_args.py`` — placeholder forms,
single-pass guarantee, append-at-end rule, shell-quoted input,
unbalanced-quote fallback, uppercase + underscore-prefix names
* ``tests/test_skill_parser.py::TestArgumentsAndHint`` — parser
extraction
* ``tests/test_skill_parse_api.py`` — HTTP parse-preview echoes
both fields
* ``tests/test_skill_discovery_api.py::test_install_seeds_arguments_and_argument_hint``
— install round-trip
* ``tests/test_skills_tool.py::test_load_forwards_arguments_to_set_skill``
+ ``test_load_same_skill_different_args_triggers_resub`` —
wire path through prepare → exec → set_skill
* ``tests/test_skills_tool.py::TestSkillArgNames`` — storage decode
helper including the hyphen/dot/leading-digit filter
Two findings from Copilot's review of PR #577:
* ``_extract_bool`` int branch: Copilot flagged that ``bool(raw)``
treats any non-zero int as True, so ``disable-model-invocation: 2``
silently disables model invocation without warning the author about
the typo. Tightened to accept only ``0`` and ``1`` as integer
boolean forms — anything else falls back to *default*. Matches
spec (which mentions only 0/1) and the broader principle that
ambiguous input should not coerce silently.
* ``hidden_from_menu`` admin body parse: Copilot flagged that
``bool(body.get("hidden_from_menu", False))`` treats non-empty
strings via Python truthiness, so a malformed client sending
``"false"`` would flip the flag to ``True`` — opposite to obvious
intent. Extracted a ``_parse_strict_bool`` helper that accepts
only Python ``bool`` or int ``0``/``1`` and returns a 400 on
anything else. Applied at both admin create and admin update
sites; the install path remains untouched because it derives the
flag from the typed ``ParsedSkill.user_invocable`` field (not raw
HTTP body).
## Tests
* ``test_other_ints_fall_back_to_default`` — ``2`` and ``-1`` no
longer silently coerce
* ``test_create_skill_hidden_from_menu_string_rejected`` — string
``"false"`` returns 400
* ``test_create_skill_hidden_from_menu_int_zero_and_one_accepted``
— 0 / 1 accepted, 2 rejected with 400
No behaviour change to the main surface — both fixes close latent
type-coerce hazards a malformed input could have exploited.
The Anthropic Claude Code skill spec defines two invocation-control
axes Turnstone was parsing but not consuming:
* ``disable-model-invocation: true`` — model can't autoload this skill
(only user can invoke by name). Stored on ``ParsedSkill`` and
echoed on the parse-preview UI; no install consumer because
Turnstone hardcodes ``activation="named"`` on source-installs
already. The dataclass docstring spells out the no-op so a future
reader doesn't try to wire a translation that's already implicit.
* ``user-invocable: false`` — skill stays available to the model but
disappears from the user-facing picker. Mapped to
``hidden_from_menu=true`` on ``prompt_templates`` (column
pre-allocated by PR #574); consumed by ``list_skills_summary``
(both the standalone-server and console-server impls).
## Surface
* Parser: new ``_extract_bool`` helper accepts every YAML 1.1
boolean spelling (true/false/yes/no/on/off/1/0) plus their quoted
variants — caught by ``/review`` as a real gap, since YAML's
``safe_load`` returns ``int`` for unquoted ``1``/``0`` and ``str``
for the YAML 1.1 spellings when quoted.
* Install handler: derives ``hidden_from_menu`` from
``parsed.user_invocable`` on the source-install path.
* Admin: ``CreateSkillRequest`` / ``UpdateSkillRequest`` accept
``hidden_from_menu``; both modals get a checkbox; the parse-preview
auto-fill flips it when the source SKILL.md sets
``user-invocable: false``.
* Runtime config: ``hidden_from_menu`` joined
``SKILL_RUNTIME_CONFIG_FIELDS`` so admin can override on installed
(readonly) skills — same precedent as ``model`` / ``effort``.
## list_skills_summary shared helper
Two identical implementations of ``list_skills_summary`` had
accreted in ``turnstone/server.py`` and ``turnstone/console/server.py``.
Both needed the new ``hidden_from_menu`` filter, so extracted the
shared body to ``turnstone/core/web_helpers.skill_summary_rows``.
Future spec-uplift fields (e.g. #572's ``argument_hint`` for
autocomplete) only touch one place now.
## Tests
* ``TestInvocationControl`` — bool / quoted / YAML 1.1 / int variants
across both fields
* ``test_install_user_invocable_false_sets_hidden_from_menu`` +
default-unhidden case
* ``test_list_skills_summary_excludes_hidden_from_menu`` — picker
filter, admin tab unaffected
* ``test_update_skill_readonly_hidden_from_menu_allowed`` — admin
can hide/unhide installed skills via PUT (pins the runtime-config
membership invariant)
* Existing parse-API fixture extended with both new fields plus
default-case assertions
The Anthropic Claude Code skill spec defines three frontmatter fields
the parser was previously dropping; this PR wires them through to the
existing storage shape so the SKILL.md author's intent survives the
import.
* ``when_to_use`` — concatenated into ``description`` at parse time
with a ``\n\nWhen to use: `` separator. Kept as its own field on
``ParsedSkill`` so the admin parse-preview UI can surface it
separately.
* ``model`` — passed through to ``create_prompt_template(model=...)``
on the source-install path, seeding the existing
``prompt_templates.model`` column.
* ``effort`` — same shape, translates to the existing
``reasoning_effort`` column at the install handler boundary.
Re-install short-circuits at the source_url dedup, so admin overrides
to either column survive an upstream re-install — covered by a new
``test_reinstall_preserves_admin_model_override`` test that pins the
load-bearing invariant.
## Description length cap
``_MAX_DESCRIPTION_LEN`` exported as ``MAX_SKILL_DESCRIPTION_LEN``
(public name) and raised from 1024 to 1536 to match the spec's
combined ``description`` + ``when_to_use`` listing budget. All five
write surfaces now import the same constant rather than each carrying
their own magic number:
* ``skill_parser.MAX_SKILL_DESCRIPTION_LEN`` — parse-time cap
* ``console_schemas.CreateSkillRequest.description`` — Pydantic
* ``console_schemas.UpdateSkillRequest.description`` — Pydantic
* ``console/server.admin_create_skill`` — handler slice
* ``console/server.admin_update_skill`` — handler slice
* ``core/session._exec_skills_create`` — coordinator tool slice
* ``core/session._exec_skills_update`` — coordinator tool slice
The coordinator sites (last two) were the bug ``/review`` caught:
they still capped at 1024 after the rest of the surface bumped to
1536, so a model-issued ``skills(action='create')`` with a 1025-1536
char description would silently truncate. Sharing the constant
closes that desync.
## when_to_use truncation guard
The ``when_to_use`` concat reserves room for the separator + at
least one character of the appended value; below that budget, the
addition is dropped entirely. Previously the naive concat could
truncate mid-separator and leave the description ending in a
dangling ``\n\nWhen ``.
## Tests
* ``TestWhenToUse`` — concat semantics, no-description fallback,
1536 truncation
* ``TestModelAndEffort`` — extraction + defaults
* ``test_install_seeds_model_and_effort_from_frontmatter`` — install
path persists both columns
* ``test_install_no_model_or_effort_leaves_columns_empty`` — bare
SKILL.md doesn't invent values
* ``test_reinstall_preserves_admin_model_override`` — admin edits
survive an upstream re-install (dedup invariant)
* ``test_parses_full_frontmatter`` / ``test_parses_minimal_frontmatter``
extended with the new field assertions
Three findings from Copilot's review of PR #574:
* `update_prompt_template` (both backends) coerces every other
INTEGER-as-bool field (`is_default`, `auto_approve`, `enabled`) but
not `hidden_from_menu`. Without coercion, a caller updating with
``hidden_from_menu=True`` writes a Python bool to a SQLAlchemy
Integer column, which is driver-dependent on PostgreSQL and a
consistency hazard. Coerce to int alongside the existing trio.
Added a focused round-trip test that updates with ``True`` /
``False`` and asserts the read-back bool transitions.
* `_canonicalize_skill_string_list` docstring describes its own
``None``-collapses-to-``"[]"`` rule but doesn't mention that
`admin_update_skill` intercepts ``None`` before the helper is
called. Added a note documenting the layered contract: the helper
defines normalization (create semantics), the update endpoint
layers no-op semantics on top.
* HTML label hints used Markdown-style ``paths:`` backticks inside
plain HTML, which render as literal backticks in the browser.
Replaced with `<code>paths:</code>` on both the create and edit
modal Paths inputs.
No behaviour change to PR #574's main surface — the bool coercion
addresses a latent bug a future consumer would have hit; the
docstring + HTML fix are purely cosmetic.
Implements PR1 of issue #569 — parser + storage + admin UI for the
Anthropic Claude Code skill spec `paths:` SKILL.md frontmatter field
(glob patterns gating model-initiated autoload). The autoload filter
that consumes `paths` is deferred to a follow-up PR pending the
workstream-CWD design discussion.
Migration 056 bundles three additional columns whose consumer PRs are
filed but not yet implemented:
* `hidden_from_menu` (boolean) — backs the spec's `user-invocable:
false` (issue #571).
* `arguments` (JSON list) — backs spec `arguments:` named arg slots
(issue #572).
* `argument_hint` (string) — autocomplete display string (issue #572).
The deferred columns surface in `SkillInfo` (response) so consumers can
read them, but are deliberately absent from `CreateSkillRequest` and
`UpdateSkillRequest` — the create/update handlers don't yet read them
and advertising a writable field the handler would silently ignore
would be an OpenAPI lie.
Surface
- Parser: `ParsedSkill.paths` populated from frontmatter; accepts the
spec's YAML-list-or-CSV-string shape via the existing
`_extract_list` machinery.
- Storage: 4 new columns on `prompt_templates`; `SKILL_MUTABLE`
extended; `_row_to_dict` calls extended to cast the new bool;
protocol + SQLite + PostgreSQL `create_prompt_template` signatures
threaded.
- HTTP: admin create/update/install/parse handlers plumb `paths`
through. Pydantic schemas extended accordingly.
- Admin UI: `skill-paths` and `etm-paths` inputs on the create + edit
modals; field map and read/write helpers wired across paste-parse,
reset, create-send, edit-load, edit-send, and the readonly-disable
list.
Notable
- `_canonicalize_skill_string_list` collapses the list-or-CSV-or-JSON-
string normalization shared between admin_create_skill and
admin_update_skill. Treats `None` as no-value so a body containing
`{"paths": null}` doesn't CSV-split through `str(None)` and store
the literal `["None"]`. Will back `arguments` once #572 wires its
consumer.
Tests
- Parser: TestPaths covers YAML list, CSV string, empty, full-
frontmatter integration (tests/test_skill_parser.py).
- Storage: round-trip suite covers create + read + update for each
of the four new columns on both backends
(tests/test_storage_skill_spec_uplift.py).
- Helper: focused unit tests for the canonicalizer including the
regression-net case for the null-corruption bug
(tests/test_canonicalize_skill_string_list.py).
- HTTP boundary: extended test_parses_full_frontmatter +
test_parses_minimal_frontmatter to assert `paths` survives the
admin parse endpoint.
* fix(output_guard): harden against domain-camouflaged injection (#560)
Three layered mitigations against the camouflage attack class described in
arXiv:2605.22001 (Pai, May 2026), which demonstrates 90.3% evasion on Llama
3.1 8B and 44.4% on Gemini 2.0 Flash against pattern-based detectors:
- Sub-agent synthesis is now scanned by output_guard at the sub-agent
boundary in _run_agent, in addition to the existing scan at the parent's
tool-result loop. Covers all four return paths (clean exit, truncation,
context-limit recovery, turn-limit forced synthesis), closing the
cross-workstream summary laundering surface.
- Adds pair-of-signals camouflage detection: imperative recommendation
phrase combined with either an authority frame ("consistent with our
risk framework") or a caps action verb (SELL/BUY/TRANSFER/...). New
flag camouflaged_injection at medium risk; deliberately partial — the
paper's augmented-detector approach recovers only ~10% on Llama-class
models, so this is duct-tape pending a semantic-evaluator follow-up.
- Bumps output_guard's wall-clock budget default from 5s to 30s and
exposes it as judge.output_guard_budget_seconds in ConfigStore, so the
expanded regex set has headroom on large tool outputs.
* Fix test_budget_kwarg_is_honored to exercise deadline logic path
The test previously passed an empty string which short-circuited
evaluate_output() before budget_seconds was used. Now uses a non-empty
input and monkeypatches time.monotonic() to deterministically verify
the deadline path is exercised.
Single-$ inline math is too ambiguous in conversational text: currency
amounts ("$5 and $10 each"), shell variables ("$HOME and $PATH"), and
shell prompts all produced false-positive KaTeX spans because the
regex matched any non-$/non-newline span between two dollar signs.
Inline math now requires the unambiguous \(...\) form, which is what
GPT-5 / o-series / Claude with reasoning effort emit by default anyway.
Display math ($$...$$ and \[...\]) is unchanged — the doubled
delimiter has enough mass that ambiguity is not a practical problem.
Three former positive tests are inverted into regression guards so a
future regex change can't quietly resurrect the bug, and new tests
name the currency and env-var cases explicitly. The web env prompt is
updated to advertise \(...\) and to tell the model why $...$ is gone.
5 follow-up comments from Copilot, all valid:
1. **CRITICAL — snap_seq race with split writer** (concurrency, 001).
Round-1's fix lifted snapshot capture into
register_listener_with_replay under nested locks, but the
WRITER side (on_content_token / on_reasoning_token) still
released _ws_lock before calling _enqueue (which bumps
_event_id under _listeners_lock). A reader could
interleave between writer's release and writer's _enqueue:
capture inflight WITH the new text, read STALE _event_id,
return snap_seq < new_event_id. The new event's live emit
then has _seq > snap_seq, slips past the dedup filter, and
double-renders text the snapshot already contained.
Fix: move self._enqueue(...) INSIDE the with self._ws_lock:
block in both token writers. The inflight mutation and the
_event_id advancement are now atomic against any snapshot
reader. Lock order _ws_lock (outer) → _listeners_lock
(inner via _enqueue) matches the snapshot helpers, so no
deadlock. Fan-out's put_nowait calls happen under
_ws_lock for token writers — microsecond cost per listener,
acceptable for the correctness guarantee.
2. **NIT — stale comment ref to buffered[-1]._event_id** (docs, 002).
The comment referenced a local var (buffered) that lives in
register_listener_with_replay, not in the events handler.
Reworded to describe the cutoff in terms of the last replayed
event id and the atomic-against-writers registration.
3. **MODERATE — 401 branch leaves reconnect loop** (bug, 003).
The coord's onerror schedules a 5 s CLOSED-state recovery timer
unconditionally. In the 401-expired-session branch we close
evtSource and showLogin — but the timer still fires 5 s later,
observes !evtSource, and calls scheduleReconnect(),
which opens a new EventSource that 401s again → infinite
reconnect loop while the login overlay is up. Fix: cancel
reconnectTimer in the 401 branch.
4. **MODERATE — race test was vacuous** (test_coverage, 004).
The previous regression test drained the listener queue after
register_listener_with_replay returned, but the helper
doesn't backfill buffered events into the queue, so the loop
was almost always a no-op and the assertion never executed.
Rewrote with a monkey-patched _enqueue that sleeps 50 ms
before bumping _event_id — widens the race window
deterministically. Verified: the test FAILS on pre-fix code
(snap.content has marker but snap.seq=0 < final_event_id=1)
and PASSES on post-fix code (writer holds _ws_lock through
_enqueue, so the reader blocks until writer fully done).
Also pinned the no-backfill contract so a future change adding
listener-queue backfill remembers to keep snap_seq the
high-water mark.
5. **NIT — except Exception too broad in test** (best_practices, 005).
Tightened except Exception: to except queue.Empty: so
unexpected exceptions aren't silently swallowed in the drain
loop.
Tests:
- 86 tests in test_sse_reconnect_replay.py + test_session_ui_base.py
pass (existing 84 + 2 new race regressions).
- Full non-live suite: 6347 passed, 15 skipped, no regressions.
- Ruff + mypy clean on changed .py files; JS parses.
Four issues raised on the merged PR #542, evaluated and fixed:
1. **Truncated-path snap_seq bug (Copilot low-confidence, VALID).**
make_events_handler's truncated branch set snap_seq = 0,
disabling the live-drain _seq <= snap_seq dedup filter. Any
token writer racing between register_listener_with_replay
returning and the live drain's first read would land in BOTH the
listener queue AND the captured snapshot text (the snapshot is
emitted via in_progress_snapshot as the recovery floor), so
the client double-renders. Fix lifts the snapshot capture INTO
register_listener_with_replay under the same nested-lock
acquire as the listener registration + buffer slice + counter
read, so the returned snapshot["seq"] is the exact
high-water mark the snapshot text corresponds to. Handler now
uses snapshot["seq"] as snap_seq on truncated, dropping
any token event with _seq <= snap_seq from the live emit.
2. **Lock-held string join in truncated path (Copilot, VALID).**
"".join(ui_base._ws_inflight_content) ran inside the
with ui_base._ws_lock: block, holding the lock for the
duration of the join and blocking on-token writers. Fix (folded
into #1's refactor): the new register_listener_with_replay
copies the inflight lists under lock and joins outside, matching
the existing pattern in
register_listener_with_in_progress_snapshot.
3. **_strip_js_comments docstring misclaim (Copilot, VALID).**
Docstring claimed the helper preserves "string/regex literals"
but the implementation only tracks string delimiters. Fix:
docstring updated to call out the regex-literal limitation
explicitly + note that current callers don't scan regions
containing regex literals. Extending the tracker is left for
a future caller that needs it.
4. **Coord scheduleReconnect dead-code regression (Copilot, VALID).**
After the PR-D refactor, scheduleReconnect() had no remaining
call sites — which meant reconnectAttempts never incremented,
wasReconnecting was always false, AND there was no
fallback when the browser transitioned the source to CLOSED
(hard 4xx after retries, intermediary tearing the connection
down with prejudice, etc.). The first failure mode silently
broke the post-gap replace-mode refresh of children / tasks /
wait indicator / live-badge cache; the second left the coord
permanently disconnected on non-transient failures. Fix:
- Introduce disconnectedSinceLastOpen flag set in onerror,
cleared in onopen. wasReconnecting reads it (with the
legacy reconnectAttempts > 0 fallback for the
scheduleReconnect-driven case), so the post-gap refresh fires
after every reconnect including the common native-reconnect
path.
- Re-introduce CLOSED-state recovery: onerror schedules a 5 s
delayed check via reconnectTimer; if the source is still
CLOSED at that point, call scheduleReconnect(), which
opens a new EventSource (threading the saved
lastEventId via the URL query param so replay still works
across the manual reconnect). Cancel/replace successive
timers so onerror floods don't pile up multiple checks for
the same source.
Tests:
- New test_truncated_path_snapshot_captures_real_snap_seq pins
the snap_seq fix at the helper boundary.
- New test_truncated_path_filters_already_in_snapshot_tokens
pins the end-to-end dedup invariant — would have caught the
double-render under the old code.
- Existing test_sse_reconnect_replay.py call sites updated for
the new 6-tuple return of register_listener_with_replay.
- All 86 tests in those two files pass; full non-live suite (6347
tests) passes; ruff + mypy clean on changed files; both JS files
parse-check.
Four /review findings collapsed to one code chokepoint + two
documentation fixes:
1. find's `kind` arg now validated against ``SkillKind`` (matching
create / update's existing pattern at session.py:8298 / :8512).
Closes two failure modes that shared the same root:
- typos (`kind="interactivee"`) silently produced
`kinds=["interactivee", "any"]` filtering to literal-`any` rows
only and masquerading as a narrowed catalog — now returns an
explicit "kind must be one of: ..." error;
- the documented enum value `kind="any"` degenerated to
`kinds=["any", "any"]` which narrowed to literal-`any` rows
instead of returning "every kind" — now collapses to ``None``
so the documented semantic holds.
2. docs/coordinator-skills.md "two-surface model" section rewritten
to reflect the post-flatten reality: kind is metadata, not an
enforcement boundary. The line-67 tools-table row updated from
the long-dead `list_skills` to `skills (action=find)` with the
opt-in kind-filter framing.
3. Three stale "interactive-only" comments in session.py
(:5514, :7857, :8210) that directly contradicted the
`_prepare_skills_load` docstring ("Both kinds can load") — drop
the qualifier so future grep-and-encode hazards don't reintroduce
the rejection.
Tests:
- test_find_kind_invalid_errors — typo case (replaces the silent
degenerate to literal-any-only)
- test_find_kind_any_means_no_filter — documented enum value matches
documented semantic (collapses to None at prepare)
- test_find_kind_narrow_passes_through — valid narrowing values
reach exec as expected
Deferred to release notes (no code change, intentional policy shift):
- skills(action='get') / load can now read full content + scan_report +
allowed_tools on cross-kind rows from any session. Operators with
pre-existing kind=coordinator skills authored under the prior
implicit visibility contract should audit those bodies for
sensitive content (allowed_tools allowlists, embedded credentials,
internal hostnames in examples) before upgrade.
Closes#557. SkillKind was authored audience metadata that the
discoverability filter dressed up as a runtime visibility gate. Real
access control is allowed_tools + auto_approve, which apply identically
across kinds. The kind-scoping chokepoints scaled linearly with every
new model-write surface for zero security payoff.
Drop kind consultation from:
- ChatSession._skills_kinds (deleted) and ._lookup_visible_skill
(deleted; callers inlined to storage.get_prompt_template_by_name).
- _exec_skills_find: no longer auto-threads kinds=. The opt-in `kind`
arg is a passable filter (threads [<kind>, "any"]) so the
discoverability win survives without enforcement.
- _exec_skills_get / _exec_skills_load: row lookup is name-only.
Disabled-row gate stays on load (admin quarantine is the actual
boundary). _prepare_task already uses unscoped get_skill_by_name;
session_routes.py already calls storage directly with no kind
check. Both confirmed by the spike, no source change needed.
- tools/skills.json: drop "Coord sessions see / interactive sees"
language; kind arg description re-cast as opt-in discoverability
narrowing.
- storage Protocol docstring + console_schemas.py kind field
description: refresh to reflect passive-metadata role.
Keep:
- SkillKind enum, kind column on prompt_templates, admin Skills tab
editing, kind field in skills.find / skills.get projection. The
field is useful for sorting/grouping at the model layer and as
authored intent.
- storage.list_skills_filtered(kinds=...) parameter — admin-filter
only now; docstring updated to note it's no longer auto-threaded
from the model-tool path.
Design calls:
1. find accepts opt-in `kind` arg: YES. ~5 lines on prepare + exec.
Threads kinds=[<kind>, "any"] only when supplied. Preserves the
model's ability to narrow a browse without enforcing.
2. kind field stays in find/get projection: YES. Already pulled
directly from the row dict in _skills_project_row (session.py
line 8187); the projection survives the flatten unchanged.
Tests:
- Delete TestLookupVisibleSkill (helper gone), the two
TestExecSkillsLoadKindScoping cross-kind reject branches, the two
test_find_kind_scoping_* tests, and test_get_cross_kind_returns_not_found
— the rejections those pinned are gone.
- Add test_find_default_threads_no_kind_filter (kinds=None by default
for both session kinds), test_find_returns_all_kinds_for_session
(interactive sees both interactive- and coord-tagged rows),
test_find_filters_by_kind_when_supplied (opt-in narrowing works),
test_get_returns_row_across_kinds (cross-kind get succeeds),
test_load_works_across_kinds (cross-kind load succeeds in both
directions — the flatten contract), test_load_rejects_missing_skill
(missing-row hint coverage). Keep test_load_rejects_disabled_skill
(admin quarantine still applies), test_load_works_for_coord_on_*
(coord-side load still works on more kinds now).
Storage tests untouched: tests/test_storage_skills_filtered.py
keeps its kinds= coverage (the parameter still works, just no longer
auto-threaded from the model-tool path).
Closes review findings from PR #555 that motivated the rethink:
sec-1 (_prepare_task unscoped) moot, sec-2 (HTTP create unscoped)
moot, sec-3 (no audit on cross-kind probes) moot — there is no
cross-kind concept anymore.
Boundary spike (verified against fresh main at 9a98d07d):
- _skills_kinds defined at session.py:7933, callers exactly two:
_lookup_visible_skill (7978) + _exec_skills_find (8057, 8109).
Verified via grep across turnstone/.
- _lookup_visible_skill defined at session.py:7943, callers exactly
two: _exec_skills_get (8213) + _exec_skills_load (8277). Verified
via grep.
- _skills_project_row reads kind directly from the row dict
(r.get("kind") or "any") at session.py:8187 — no helper call;
projection survives flatten.
- _prepare_task at session.py:6353 calls unscoped get_skill_by_name
— no kind check, no change needed.
- session_routes.py:1985 calls storage.get_prompt_template_by_name
directly with no kind check — already flat.
- storage.list_skills_filtered(kinds=...) parameter is identical
across _sqlite.py:2986, _postgresql.py:2827, and _protocol.py:1364.
- tests/test_skills_tool.py: TestLookupVisibleSkill (5 tests, lines
481-536) and TestExecSkillsLoadKindScoping (5 tests, lines 539-639)
pre-flatten. Total: 61 collected → 57 collected post-flatten.
Net production LOC: -15.
The local-escape posture from the prior commit double-encoded values
that inlineMarkdown had already escaped: leading escapeHtml(text)
turns `&` into `&`, the local escapeHtml(url) then turned that
into `&amp;`, which breaks query-string URLs after browser parse
+ getAttribute + new URL round-trip.
Switch to convention-rename: regex callback params renamed to
safeAlt / safeUrl / safeLabel to signal the upstream-escape
invariant. The attribute-context lint enforces all future
attribute-context concat sites maintain the safe* convention or
call escapeHtml explicitly — defense-in-depth preserved without
the regression. Two added pin tests verify `&` survives with
single (not double) entity encoding through image data-src and
link href.
Also addresses two test issues from the same review:
- Docstring listed `safe[A-Z_]…` but code only checked isupper().
Drop the underscore option (JS uses camelCase anyway).
- `_all_attr_names` only recorded attribute-bearing tags, so a
bare `<script>` injection would have false-negatived the link-
label pin test. Refactored to `_parse_renderer_html` returning
both start tags and (tag, attr) pairs.
inlineMarkdown's image and link renderers now escapeHtml each
interpolated value (url, alt, label, domain) at the call site
instead of relying on the upstream escape pass. Defence-in-depth:
a future refactor calling those renderers from outside
inlineMarkdown would otherwise silently regress.
New CI lint scans renderer.js for `attr="' + ident` patterns; ident
must be escapeHtml(...), safe*, or in the reviewer-approved
allowlist. Four pin tests use html.parser.HTMLParser to verify
attacker URLs and labels don't materialize event-handler attributes
on rendered DOM.
Three independent fixes flagged by Copilot's review on PR #555:
2. ``update`` auto_approve self-escalation warning false-positive
(turnstone/core/session.py:_prepare_skills_update)
- The warning was computed against ``existing_auto_approve or
proposed_auto_approve`` — meaning an update that explicitly
turned auto_approve OFF still triggered the warning because the
existing row had it ON. Now computes against the *final state*
(``updates["auto_approve"]`` if present, else
``existing.get("auto_approve")``) combined with the final
``allowed_tools`` value. False-positives gone; the inverse case
(existing auto_approve=False, update turns it ON without
touching allowed_tools) now correctly fires the warning against
the inherited allowlist.
3. ``temperature`` validator silent-coerce → explicit error
(turnstone/core/skill_field_validation.py:parse_skill_session_config)
- Non-numeric temperature input silently coerced to ``None``,
unlike ``max_tokens`` / ``token_budget`` which return an error.
Numeric-field consistency: temperature now errors on
unparseable input with "temperature must be a number between 0
and 2". Range check unchanged; blank / None still → None.
4. Version-snapshot uses max+1, not count+1
(turnstone/core/session.py:_exec_skills_update)
- ``count_skill_versions + 1`` re-uses version numbers when any
row has been deleted via the existing
``storage.delete_skill_versions`` method, and the schema has no
``(skill_id, version)`` unique constraint to catch the
collision. Switched to ``max(list_skill_versions)`` + 1,
matching the ``storage.unlock_skill`` pattern. A storage-side
atomic allocator is the right architectural fix and is tracked
for a future PR.
Tests cover both the false-positive and inverse-positive auto_approve
cases, the new temperature error path, and the version-numbering edge
case where prior versions have been deleted (max diverges from count).
Two changes that share the same kind-scoping touch point.
Lookup unification (closes the bypass Copilot flagged on _exec_skills_load):
- New ChatSession._lookup_visible_skill(name) — single source of truth for
"find me a skill by name, if it's visible to this session". Combines
storage.get_prompt_template_by_name with the kind filter in one call;
returns None for both the missing-row and out-of-kind cases so callers
don't have to branch on the reason.
- _exec_skills_get refactored from inline two-step to one helper call.
- _exec_skills_load refactored from the unscoped memory.get_skill_by_name
to the new helper — the kind-scoping bypass it had (interactive could
load a kind=coordinator skill by name) goes away by construction
because the unscoped path no longer exists on the model-tool surface.
- memory.get_skill_by_name stays available for admin / sub-agent /
rehydrate paths that need full-catalog visibility — those are
deliberate cross-kind callers, not bypass surfaces. Storage exceptions
now propagate from _lookup_visible_skill by design (distinct from the
legacy swallow-and-return-None) so the operator gets a clear signal on
DB outage rather than a misleading "not found".
Coord-side load support:
- _prepare_skills_load no longer rejects coordinator sessions. Parity
with the admin / HTTP create path that already accepts a `skill` body
field on kind=coordinator workstreams — what the operator can do at
create time, the model can now do on its own session. Visibility is
still kind-scoped via _lookup_visible_skill at exec (a coord can only
load {coordinator, any}-tagged skills; interactive can only load
{interactive, any}), matching what `find` / `get` enforce.
The kind-scoping itself is queued for a separate cleanup PR: the marker
turned out to be a discoverability hint that never gated runtime
capability, and the combinatorial complexity (every new model-tool /
HTTP path needs kind awareness) isn't worth the squeeze at this team
size. Follow-up issue to land.
Test coverage:
- TestLookupVisibleSkill — 5 cases: visible / cross-kind / missing /
storage-unavailable / kind=any-on-both-surfaces.
- TestExecSkillsLoadKindScoping — kind-rejection from both directions
(interactive→coord-only, coord→interactive-only), disabled-skill
caller-side gate, and the two new positive coord-load cases (coord
loads kind=coordinator and kind=any).
- Removed test_load_on_coord_session_errors (the rejection it pinned
is gone).
Plus the /review-suggested doc fixes that came with the unification:
- Comment in _exec_skills_load now correctly attributes the disabled
collapse to the caller's enabled check rather than implying the
helper handles it.
- _lookup_visible_skill docstring documents the deliberate
exception-propagation behavior.
Replaces the legacy `skill` (load + search) and `list_skills` tools with a
single `skills(action=...)` tool serving both interactive and coordinator
sessions. Stacks on the model.skills.write permission introduced in PR 1.
Tool surface
- `find`: filter by category/tag/risk_level/enabled_only/limit with
optional BM25 query ranking; auto-approved on both kinds; kind-scoped at
the storage filter (interactive sees interactive+any, coord sees
coordinator+any).
- `get`: fetch a single skill including content; cross-kind misses
collapse to "not found" so a model can't enumerate the other surface
by name-probing.
- `load`: activate a skill in the current session (interactive-only;
coord sessions get an explicit hint pointing at spawn_workstream).
- `create`/`update`/`enable`/`disable`: require approval AND
model.skills.write; permission re-checked at exec time to catch a
revocation between approval and write.
- No `delete` — hard-delete stays admin-UI exclusive; tool description
documents the soft-delete-via-disable pattern.
Defenses on the write surface
- Approval cards surface projected risk_level (scanner re-run against
the proposed final state) and warn explicitly when allowed_tools +
auto_approve combine (auto-fire-on-load consequence is spelled out,
not just shown as raw field values).
- Toggle preview surfaces existing risk_level + allowed_tools count so
re-enabling a critical-tier skill is never a one-click bypass.
- Update path now re-fetches the row at exec to catch a readonly flip
between approval and write, filters updates back to the runtime-only
set if so, refuses if no fields survive.
- Update path rejects empty content (hollow-out via emptying bypassed
the soft-delete-via-disable invariant), non-list tags, and empty
category — failures are loud rather than silent.
- Permission denials audit `skill.write_denied` with actor_source=model
so probing the permission state leaves a trail. Audit failures log
at error (not warning) — a successful write without a row is the
exact gap the trail exists to surface.
- `_skill_hint` routes both message and system_reminder through
escape_wrapper_tags so caller-controlled values can't close the
<system-reminder> envelope and let the model fabricate directives in
its own future context.
Shared validation
- `parse_skill_session_config` lifted from console/server.py to
turnstone/core/skill_field_validation.py; both the HTTP admin path and
the model-tool path consume it. Single source of truth so field rules
can't drift between layers.
- `SKILL_RUNTIME_CONFIG_FIELDS` lifted similarly (was duplicated as
_SKILL_RUNTIME_CONFIG_FIELDS in server.py and _SKILLS_READONLY_FIELDS
on ChatSession).
- `notify_on_complete` validator now accepts list input from the JSON
schema's `array` type — previously rejected because str() of a list
yields Python repr that json.loads then refuses.
Performance
- Update prepare skips the projected-risk scan when neither content nor
allowed_tools is changing (storage re-scans on write authoritatively).
Metadata-only updates no longer pay the ~25 regex-pass scan cost.
Cleanup
- CoordinatorClient.list_skills deleted (-91 lines); model-tool path
talks to storage directly via list_skills_filtered.
- Roles admin UI gains a Model section exposing model.skills.write.
- tests/test_load_skill.py renamed to tests/test_skills_tool.py and
rewritten for the new tool — 48 tests covering registration, prepare
dispatch, permission gating (including TOCTOU-revoked exec deny),
audit actor_source on create + disable + permission-denied probe,
BM25 ranking, invalid-kind branches, audit-failure swallow, and
<system-reminder> envelope injection resistance.
In-process permission check for model-facing tool exec paths that need
to gate a write capability without HTTP middleware in the loop. Foundation
for the upcoming skills tool refactor: the merged
skills(action=create|update|enable|disable) tool will gate on
model.skills.write before reaching storage.
- Add model.skills.write to _VALID_PERMISSIONS (default-ungranted on every
role including builtin-admin — operators opt themselves in explicitly)
- Add user_has_permission(user_id, permission, *, storage=None) helper
that fails-closed on storage outages and short-circuits on empty user_id
- Document service-scope asymmetry with require_permission (no AuthResult
in the model-tool path → no bypass; explicit guidance if a legitimate
service-scope caller ever needs to reach here)
- Pin the "no implicit cache" contract with a regression test asserting
every helper call hits storage (call_count == 2 after two calls)
- Lock the "builtin-admin default-ungranted" invariant with an alembic
migration test that drives the chain to head and asserts the role's
permission string omits model.skills.write
- Plus the role-create end-to-end test proving the constant flows through
the admin endpoint's validator
Roles admin UI changes deferred to the PR that lands the gated tool — no
operator action needed until the capability exists.
Per-call DB hit + warning-log spam on outage deferred to a follow-up PR;
the helper is dead code in this commit, so cache TTL would be sized
against guesswork — better to wait for a real call-rate signal from the
first caller.
Starlette 1.0.0 reconstructs request URLs without validating the Host
header, allowing path-injection that can bypass authentication on apps
comparing reconstructed URL paths instead of `request.url.path`. Fixed
in 1.0.1.
- pyproject.toml: bump `starlette>=0.45` to `starlette>=1.0.1` so the
CVE floor is explicit at the dependency declaration, not just in the
lockfile. Annotated with the advisory ID so the rationale survives
a future floor relax.
- uv.lock: regenerated via `uv lock --upgrade-package starlette`;
starlette 1.0.0 -> 1.0.1, no transitive bumps.
Locally verified `pip-audit --strict` returns clean after the bump and
the auth + service-boundary test suites (250 tests covering the URL/
host-header reconstruction surface) continue to pass.
2000 was sized for the cloud-provider regime (50–200 events/sec)
and was too small for the two regimes that actually shape PR-D's
recovery floor:
1. **Local inference**: vLLM / llama.cpp hit 500–2000 tok/s per
active stream. Each token is an _enqueue call, so a single
busy workstream burns through 2000 events in ~1 s. Reconnects
after any disconnect longer than a network blip immediately
fall through to the replay_truncated recovery path on a
stream that was supposed to be transparently resumable.
2. **Backgrounded tabs**: Chrome (and Firefox to a lesser extent)
throttle the SSE-drain microtask aggressively when a tab isn't
visible — Chrome's background-tab budget drops to ~1 wake/min
after ~5 min hidden, so a backgrounded pane can legitimately
sit on tens of seconds of un-drained events. PR-G (drop-pings-
let-it-die) deliberately closes those connections on hide and
re-opens on focus return; reconnect-with-replay is the only
recovery path, and if the buffer evicted in the interim, the
snapshot floor is all that's left for past-turn structural
events (tool calls, state changes, approvals).
50000 at the 2000-tok/s local-inference rate buys ~25 s of pure
token streaming before truncation; at cloud rates it's minutes of
coverage. Memory cost is ~200–500 bytes per event (deque node +
dict + payload), so 50000 × 100-ws design ceiling caps at roughly
2.5 GB worst-case — and practically nowhere close because the cap
is per-ws ceiling, not per-ws steady-state. Operators on heavier
workloads can raise via TURNSTONE_SSE_EVENT_BUFFER_MAX.
Considered and rejected: in-buffer coalescing of consecutive
content/reasoning tokens. A naive text-merge breaks the replay-
slice semantic — a coalesced entry has the latest _event_id
but text that includes content the client already received under
an earlier id, so any consumer with last_event_id falling
INSIDE the coalesced span would double-render on replay. A
correctness-preserving coalesce would need a per-consumer high-
water tracker we deliberately don't maintain. Bigger cap +
simple per-event storage avoids the trap; the rationale is
captured inline in _resolve_event_buffer_max.
The browser-side completion of PR-D reconnect-with-replay. Today's
`onerror` handlers on `Pane.connectSSE`, `connectGlobalSSE`, and
the coordinator's `connectSSE` all explicitly call
`evtSource.close()` on the transient-error path — that forces the
source into the terminal CLOSED state, defeating EventSource's
native auto-reconnect (which would otherwise reconnect with the
`Last-Event-ID` header that PR-D commit 1 now honours server-side).
Three handler refactors share the same shape:
- Remove the unconditional `close()` from the transient-error
branch. Native EventSource handles CONNECTING -> CONNECTING ->
OPEN with replay automatically.
- Keep UI updates (status bar dim, Reconnecting… text) — those
are orthogonal visualizations of the disconnected state.
- Keep terminal-branch closes: a 401 expired-session still does an
explicit close + showLogin (the user must re-authenticate); a
workstream-reassignment to a different ws still disconnects +
connects on the new wsId (it's a different stream, not a same-
stream replay).
- Capture `lastEventId` in `onmessage` BEFORE `JSON.parse` so a
malformed event doesn't desync the manual-reconnect fallback
from native auto-reconnect.
- Thread `?last_event_id=N` on the URL when constructing a fresh
`new EventSource(url)` — the constructor can't set custom
headers so the query-param fallback covers the manual-reconnect
path (initial connect with a saved id, scheduleReconnect after
an explicit close, etc.).
For `Pane.connectSSE`, the long focused-pane workstream-refetch
body inside `onerror` is lifted to a dedicated
`_refetchWorkstreamsAndReassign` method so it survives the
refactor as an orthogonal trigger (handles the workstream-evicted-
during-disconnect recovery case, which is independent of the SSE
reconnect mechanics). The reassignment branch's existing
`disconnectSSE + connectSSE(newWsId)` sequence stays — different
workstream genuinely needs a fresh stream. When reassigning, the
saved `_lastEventId` is dropped because replay is per-ws and an
id from ws-A is meaningless against ws-B.
Tests in `tests/test_app_js.py` add 3 static lint guards that
fail loudly if any future refactor reintroduces a naked
`evtSource.close()` in a transient-error path of any of the three
handlers. The guards understand the allowed terminal-branch
exceptions (401, login overlay, reassignment) and ship with an
escape hatch (functions that explicitly reference `last_event_id`
have taken explicit responsibility for the replay header and are
exempt). A small `_strip_js_comments` helper handles the
apostrophe-in-comment hazard that pre-existing
`_slice_balanced_body` doesn't (comments are stripped before
brace-walking; offsets preserved by space substitution).
The console SSE proxy (`_proxy_sse`) is the inbound SSE path for
multi-node deployments — every browser EventSource that targets a
per-node route traverses it. Today's proxy strips client request
headers (only `Accept`, `Cache-Control`, and the re-minted auth
token make it upstream), so the per-ws / global SSE handlers'
`Last-Event-ID` resume (PR-D commit 1) never sees the header in
the multi-node shape — every reconnect would be a fresh connect
and silently drop events from the disconnect window.
Builds the upstream headers dict conditionally: copy `Last-Event-ID`
from the incoming request when present, omit otherwise (no
fabricated value on fresh connects). Starlette's header dict is
case-insensitive so the `request.headers.get("last-event-id")`
lookup catches both the spec-recommended capitalization and any
intermediary normalisation.
The query-param fallback (`?last_event_id=N`) needs no proxy
change — `request.url.query` is already forwarded verbatim at the
top of the function.
Tests in `tests/test_service_auth_boundary.py::TestProxySseLastEventIdForwarding`:
- Positive: browser header → upstream header (value preserved).
- Negative: browser sends nothing → upstream gets nothing (no
fabricated value).
Adds the server-side foundation for SSE reconnect-with-replay (PR-D
in issue #540's sequencing): a per-ws monotonic ring buffer that
holds the last N events for replay against a client's
`Last-Event-ID` header (or `?last_event_id=N` query-param
fallback for manual reconnect paths that can't set custom headers).
Per-ws lane (SessionUIBase + make_events_handler):
- `_event_buffer` deque (cap 2000, env-overridable via
`TURNSTONE_SSE_EVENT_BUFFER_MAX`) holds (event_id, event_dict)
tuples; `maxlen` evicts the oldest automatically.
- Existing `_ws_inflight_seq` renamed to `_event_id` and lifted
to live alongside the listeners — one monotonic counter drives
both the new replay slice AND the existing `_seq`/`snap_seq`
snapshot dedup (byte-identical contract on token events).
- `_enqueue` now stamps every event with `_event_id` (and `_seq`
on `content`/`reasoning` token events) under
`_listeners_lock`, so the buffer append + listener fan-out + new
listener registration are all atomic against each other.
- New `register_listener_with_replay` returns
(queue, replay_events, status, lost_count, earliest_id) where
status ∈ {replay_ok, truncated}. `make_events_handler` reads
`Last-Event-ID` (header or query), branches three ways
(fresh / replay_ok / truncated), and emits the SSE `id:` field
on every event sourced from the buffer. On `replay_ok` the
in-progress snapshot is skipped (the buffered events already
cover it); on `truncated` an explicit envelope precedes the
fresh-style recovery path.
- Every events stream emits a jittered `retry:` in [2500, 4500] ms
on first yield so 6-pane reconnects don't lockstep on
EventSource's default ~3 s interval.
Global lane (server.py / _global_fanout_thread / global_events_sse):
- Parallel buffer + counter on `app.state.global_event_buffer` and
`app.state.global_event_id_holder`; fanout thread stamps each
event with `_event_id` and appends to the buffer under
`global_listeners_lock`. `global_events_sse` branches on
`Last-Event-ID` with the same three shapes.
Tests:
- 16 new tests in `tests/test_sse_reconnect_replay.py` cover the
ring buffer semantics (empty-listeners hold, last_event_id
slicing, truncation, atomic registration), the counter
invariants (monotonic under concurrent writers, no skip on
queue.Full, persists across turn boundaries, cross-thread
consistency), and the handler branching (retry on first yield,
id: on buffered events, snapshot-skip on replay_ok, envelope on
truncated, query-param fallback, malformed header → fresh).
- Existing `tests/test_session_ui_base.py` updated for the
`_ws_inflight_seq` → `_event_id` rename and the new
`_event_id` field on enqueued events.
Backward-compat: all consumers that don't send `Last-Event-ID`
(today's browser, Python SDK, TypeScript SDK, channel adapter) see
behaviour identical to pre-PR — the server change is purely
additive on the request side.
2026-05-22 15:13:39 -07:00
533 changed files with 89051 additions and 34389 deletions
Multi-node AI orchestration platform. Deploy tool-using AI agents across a cluster of servers with direct HTTP routing, interactive interfaces, and enterprise governance.
Self-hosted, local-first orchestration for tool-using AI agents. Give LLMs real tools — shell, files, search, web — and run them across your own cluster with direct HTTP routing and interactive interfaces. Your code, your models, your data stay on hardware you control: no telemetry, no phone-home.
<p align="center">
<img src="docs/assets/hero.png" alt="Turnstone coordinator — parallel tool batches with judge-graded approval and child workstream tracking" width="960"/>
@@ -26,12 +27,13 @@ See [docs/releasing.md](docs/releasing.md) for the full release process.
Turnstone gives LLMs tools — shell, files, search, web, planning — and orchestrates multi-turn conversations where the model investigates, acts, and reports.
- **Local-first & private** — runs entirely on hardware you control, with no telemetry and no phone-home. Point it at local models (vLLM, llama.cpp, Ollama) or commercial APIs you hold the keys to — your prompts and data never transit a third party you didn't choose.
- **Bring your own models** — OpenAI-compatible APIs (vLLM, llama.cpp, NIM), the Anthropic Messages API, and Google Gemini, mixed freely per role
- **Interactive sessions** — terminal CLI or browser UI with parallel workstreams
- **Cluster dashboard** — real-time view of all nodes and workstreams with console routing proxy
- **Intent validation** — LLM judge evaluates every tool call with risk assessments and evidence
*.json 19 tool schemas (OpenAI function-calling format + turnstone metadata)
*.json 16 tool schemas (OpenAI function-calling format + turnstone metadata)
```
Both UIs share a common design system extracted into `turnstone/shared_static/`: design tokens, login overlay, toast notifications, theme toggle, keyboard shortcuts, and utility functions. Each UI imports `base.css` and the shared JS modules at `/shared/`, then adds only page-specific code at `/static/`.
@@ -190,7 +189,6 @@ Phase 3: EXECUTE (parallel)
(cancel_event also checked per line — kills process group on cancel)
Final output (stdout + stderr) delivered via ui.on_tool_result(call_id, name, output)
call_id links tool_info items → streaming chunks → final result
For plan tool: post-execution gate via ui.on_plan_review()
```
### State Transitions
@@ -209,7 +207,7 @@ The engine emits state changes via `_emit_state()` which calls
"running" ---> tool execution
|
v
"attention" ---> waiting for user approval / plan review
"attention" ---> waiting for user approval
|
v
"running" ---> executing approved tools
@@ -231,7 +229,7 @@ The engine emits state changes via `_emit_state()` which calls
> See also: [Core Engine Classes diagram](diagrams/png/03-core-engine-classes.png)
Defined in `turnstone.core.session.SessionUI` as a `typing.Protocol` with 16
Defined in `turnstone.core.session.SessionUI` as a `typing.Protocol` with 15
methods. Every frontend must implement all of them.
| `WebUI` | `turnstone.server` | SSE event queue per workstream + global broadcast, `threading.Event` for blocking on approval/plan. `on_state_change` sends to both per-workstream and global SSE (the browser UI uses per-workstream `state_change` events to manage busy/idle transitions; `stream_end` only finalizes markdown rendering). |
| `WebUI` | `turnstone.server` | SSE event queue per workstream + global broadcast, `threading.Event` for blocking on approval. `on_state_change` sends to both per-workstream and global SSE (the browser UI uses per-workstream `state_change` events to manage busy/idle transitions; `stream_end` only finalizes markdown rendering). |
| `notify` | broadcast | Post a status update to a human channel at a narrative beat. |
| `tasks` | plan | Orchestrator-only scratchpad. Children don't see it. |
Explicitly **not** in the coordinator set:
@@ -72,8 +86,8 @@ Explicitly **not** in the coordinator set:
- `bash` / `edit_file` / `write_file` / `append_file` / `diff_file` — no local FS.
- `read_file` / `search` — no local FS reads.
- `web_fetch` / `web_search` — no direct web access.
- `task_agent` / `plan_agent` — sub-agent tools are zeroed on coord sessions.
- `memory` / `recall` / `notify` / `watch` / `read_resource` / `use_prompt`/ `skill` — the orchestrator's "memory" is its children's outputs; these UX / persistence tools belong to interactive sessions.
- `task_agent` — sub-agent tool is zeroed on coord sessions.
- `recall` / `watch` / `read_resource` / `use_prompt`— UX / persistence tools that belong to interactive sessions. The dual-kind `memory` / `skills` / `notify` tools are available on both kinds (see the table above).
If your skill needs a coordinator to "run a command" or "read a
file", write the delegate pattern instead: spawn a child with an
@@ -154,19 +168,33 @@ Every ws_id returned by `spawn_workstream` / `spawn_batch` is a
invent ws_ids — a model that hallucinates `"child-1"` or `"ws-abc"`
hits the tenant guard in `CoordinatorClient._is_own_subtree`, which
validates ws_id against `parent_ws_id=coord_ws_id` AND
`user_id=owner` in storage. The rejection shape varies by tool:
`user_id=owner` in storage. The rejection shape is uniform and
**Default** (no flag) — starts `server` and `console`. Requires an OpenAI-compatible LLM API running on the host (default: `http://localhost:8000/v1`).
Open the dashboard at **https://localhost:8443**. It's served by Caddy with its
own local CA, so trust the root certificate once (or click through the browser
The dashboard is at **https://localhost:8443** (Caddy, same as the dev stack);
the console's HTTP port isn't published. For a real domain and a publicly
trusted cert, edit `turnstone/deploy/Caddyfile` to point Caddy at Let's Encrypt
(see [tls.md](tls.md)). Pin the image with `TURNSTONE_IMAGE_TAG` (default:
`latest`).
### mTLS
Layer the TLS overlay on the production stack to enable mutual TLS between
services. A bootstrap container creates a CA and every service auto-provisions
certs via the console's ACME endpoint:
```bash
docker compose -f turnstone/deploy/compose.yaml -f deploy/docker-compose.tls.yml up
```
See [tls.md](tls.md) for details.
## Configuration
All configuration is via environment variables in `.env` (copy from`.env.example`):
Everything is configured with environment variables in `.env` (copy from
[`.env.example`](../.env.example)). The dev stack needs none of them — they're
overrides.
### LLM Backend
### LLM backend
| Variable | Default | Description |
|----------|---------|-------------|
| `LLM_BASE_URL` | `http://host.docker.internal:8000/v1` | OpenAI-compatible API URL |
| `LLM_BASE_URL` | `http://host.docker.internal:8000/v1` | Bootstrap OpenAI-compatible API URL (real backends go in the UI) |
| `OPENAI_API_KEY` | `dummy` | API key (`dummy` for local servers) |
| `TAVILY_API_KEY` | — | Web search API key (only needed for local/vLLM models; Anthropic and OpenAI search models use native search) |
| `TURNSTONE_SEARXNG_URL` | `http://searxng:8080` | SearxNG URL for the `web_search` tool (local/vLLM models only; Anthropic/OpenAI use native search). Defaults to the bundled `searxng` service; set to an external instance's URL. To turn web search off, clear `tools.searxng_url` in the admin Settings tab. |
| `SEARXNG_IMAGE_TAG` | `latest` | Tag for the bundled `searxng/searxng` image |
| `MODEL` | — | Override the default model alias |
### Server
### Auth & database
| Variable | Default (dev / prod) | Description |
|----------|----------------------|-------------|
| `TURNSTONE_JWT_SECRET` | insecure default / **required** | JWT signing secret. Every service must share one value. |
| `TURNSTONE_DB_URL` | — | Database URL (e.g. `postgresql+psycopg://user:pass@postgres:5432/turnstone`). For SQLite, defaults to `/data/.turnstone.db` |
| `TURNSTONE_DB_LISTEN_URL` | (falls back to `TURNSTONE_DB_URL`) | Direct-to-PostgreSQL URL for the console's dedicated `LISTEN` connection. Set this when `TURNSTONE_DB_URL` points at PgBouncer in transaction pooling mode — LISTEN is session state and the transaction-pooled connection can't hold it. See [pgbouncer.md](pgbouncer.md). |
| `TURNSTONE_DB_POOL_SIZE` | `2` | PostgreSQL connection pool size per process (default: 2 base + 3 overflow = 5 max) |
| `POSTGRES_USER` | `turnstone` | PostgreSQL container username (used in default `TURNSTONE_DB_URL` for cluster/channel) |
| `POSTGRES_PASSWORD` | — | PostgreSQL container password (required for production and cluster profiles) |
The database stores workstream history, user accounts, and API tokens. When using JWT auth, a database backend is required for user storage.
> **Upgrading from <1.3.0a4:** Earlier versions used `DB_BACKEND` and `DATABASE_URL` in `.env`, which `compose.yaml` mapped to the `TURNSTONE_`-prefixed names internally. These short aliases have been removed. Rename `DB_BACKEND` → `TURNSTONE_DB_BACKEND` and `DATABASE_URL` → `TURNSTONE_DB_URL` in your `.env` file.
> **Large clusters:** Each turnstone process maintains a small connection pool (5 max). At hundreds of nodes this adds up — use [PgBouncer](pgbouncer.md) in transaction pooling mode between turnstone and PostgreSQL.
> **First-time setup:** After deploying with auth enabled, create an initial admin user by running `turnstone-admin create-user` inside the container:
> You will be prompted to set a password. Use it to log in via the UI or SDK, then create additional users through the admin API. Pass `--token --scopes read,write,approve` to also generate an initial API token.
| `TURNSTONE_SLACK_SLASH_COMMAND` | `/turnstone` | Slash command registered in the Slack app |
The channel service runs in the `production` profile. When
`TURNSTONE_DISCORD_TOKEN` or the Slack pair is set the gateway starts the
corresponding adapter; both can run in one process. See
[Channel Integrations](channels.md) for platform app setup and user
account linking.
## Scaling
For multi-node testing, use the `cluster` profile which provides 10 server instances with unique node IDs (`node-1` through `node-10`), resource limits, and shared PostgreSQL:
```bash
POSTGRES_PASSWORD=secret docker compose --profile cluster up
```
The default `server` also runs alongside the cluster nodes (11 total). All nodes are accessible via the console dashboard at `:8090`.
For production clusters beyond ~50 nodes, add PgBouncer between turnstone services and PostgreSQL. See [PgBouncer Connection Pooling](pgbouncer.md) for Docker Compose and Helm configuration.
- **Stable** tracks receive bugfixes only. The most-recent stable minor
owns the `:stable` / `:latest` Docker tags and the default PyPI
@@ -17,8 +16,10 @@ Turnstone ships several parallel release tracks from a single PyPI package.
- **Experimental** (always on `main`) receives new features. May be
rough around the edges.
- When experimental matures, it is promoted to a new stable minor via
a `stable/X.Y` branch; older stable branches continue to receive
security fixes until explicitly retired.
a `stable/X.Y` branch. One prior stable track is maintained alongside
the current one; at each promotion the oldest track is retired — its
branch is deleted, while its tags and released artifacts remain
available.
## Version Scheme
@@ -33,17 +34,17 @@ Turnstone ships several parallel release tracks from a single PyPI package.
## Releasing an Experimental Version (from main)
```bash
scripts/release.sh 1.5.0a2 --push
scripts/release.sh 1.7.0a2 --push
```
This bumps `pyproject.toml` + `turnstone/__init__.py`, regenerates `uv.lock`, commits, tags `v1.5.0a2`, and pushes. CI runs, then publish + Docker workflows fire automatically.
This bumps `pyproject.toml` + `turnstone/__init__.py`, regenerates `uv.lock`, commits, tags `v1.7.0a2`, and pushes. CI runs, then publish + Docker workflows fire automatically.
## Releasing a Stable Patch (from stable/X.Y)
```bash
git checkout stable/1.4
git checkout stable/1.6
git cherry-pick <commit-hash> # bugfix from main
scripts/release.sh 1.4.1 --push
scripts/release.sh 1.6.1 --push
```
## Promoting Experimental to Stable
@@ -52,19 +53,19 @@ When `main` is ready for a stable release:
```bash
# 1. Tag the stable release on main
scripts/release.sh 1.5.0 --push
scripts/release.sh 1.6.0 --push
# 2. Create the stable maintenance branch from that tag
git branch stable/1.5 v1.5.0
git push origin stable/1.5
git branch stable/1.6 v1.6.0
git push origin stable/1.6
# 3. Start the next experimental cycle on main
scripts/release.sh 1.6.0a1 --push
scripts/release.sh 1.7.0a1 --push
```
The previous stable branch (`stable/1.4`) continues to receive
security-only patches; older tracks (`stable/1.0`, `stable/1.3`) are
retired when they fall out of support.
The previous stable branch continues to receive security-only patches;
the track before it is retired at each promotion (at 1.6.0:
`stable/1.5` stays maintained, `stable/1.4` is retired).
`reasoning_text` for Chat Completions / vLLM / llama.cpp / Gemini-compat).
### Plan / task agent overrides
### Task agent overrides
`plan_agent` and `task_agent` sub-sessions resolve independently from the
conversation model so operators can pick a cheaper/faster model for
autonomous loops:
`task_agent` sub-sessions resolve independently from the conversation model
so operators can pick a cheaper/faster model for autonomous loops:
| Setting | Purpose |
|---------|---------|
| `model.plan_alias` | Alias used for `plan_agent` sub-sessions. Falls back to `[model].plan_model` in config.toml, then `[model].agent_model`, then the session's active model. |
| `model.task_alias` | Alias used for `task_agent` sub-sessions. Same fallback chain as `plan_alias`. |
| `model.task_alias` | Alias used for `task_agent` sub-sessions. Falls back to `[model].agent_model` in config.toml, then the session's active model. |
| `model.task_effort` | Reasoning effort for `task_agent`. Empty string means "inherit from the session". |
All four are live-editable from the Settings tab and take effect on the
Both are live-editable from the Settings tab and take effect on the
turnstone exposes 19 built-in tools plus any number of external MCP tools to the
turnstone exposes 16 built-in tools plus any number of external MCP tools to the
LLM via the OpenAI function-calling interface. Built-in tools are defined as JSON
files under `turnstone/tools/` and loaded at startup by `turnstone/core/tools.py`.
MCP tools are discovered from configured MCP servers at startup by
@@ -22,7 +22,6 @@ schema plus turnstone-specific metadata keys:
"properties": { ... },
"required": ["param1"]
},
"agent": true,
"task_agent": true,
"auto_approve": true,
"primary_key": "param1"
@@ -33,8 +32,7 @@ schema plus turnstone-specific metadata keys:
| Key | Type | Meaning |
|----------------|------|---------|
| `agent` | bool | Tool is available to plan/task sub-agents (read-only subset). |
| `task_agent` | bool | Tool is available to task sub-agents (broader subset). |
| `task_agent` | bool | Tool is available to task sub-agents. |
| `auto_approve` | bool | Tool runs without user confirmation (read-only, safe operations). |
| `primary_key` | str | When the model sends a bare string instead of JSON args, map it to this parameter name. |
@@ -46,12 +44,10 @@ schema plus turnstone-specific metadata keys:
| Name | Description |
|---------------------|-------------|
| `TOOLS` | All 19 tool definitions (sent to the model). |
| `AGENT_TOOLS` | Tools with `agent: true` -- available to plan sub-agents. Read-only tools. |
| `TOOLS` | All 28 loaded built-in tool definitions (interactive + coordinator union). Sessions send a kind-specific subset (`INTERACTIVE_TOOLS` or `COORDINATOR_TOOLS`). |
| `TASK_AGENT_TOOLS` | Tools with `task_agent: true` -- available to task sub-agents. Includes write operations. |
| `AGENT_AUTO_TOOLS` | Set of tool names with `auto_approve: true` -- no user confirmation needed. |
| `TASK_AUTO_TOOLS` | Same as `AGENT_AUTO_TOOLS` (identical filter). |
| `BUILTIN_TOOL_NAMES`| Frozenset of all 19 built-in tool names. Used by tool search to distinguish always-on tools from deferrable MCP tools. |
| `TASK_AUTO_TOOLS`| Set of all tool names with `auto_approve: true` -- used by task-agent sub-sessions to skip confirmation for matching available tools. |
| `BUILTIN_TOOL_NAMES`| Frozenset of all 28 built-in tool names (interactive + coordinator union). Used by tool search to distinguish always-on tools from deferrable MCP tools. |
| `PRIMARY_KEY_MAP` | Dict mapping tool name to its `primary_key` parameter name. |
- `notify` -- sends notifications to linked channels (time-sensitive, auto-approved for urgency)
@@ -130,16 +122,14 @@ Each item's `execute` callable is invoked:
- `bash` -- arbitrary command execution
- `write_file` -- creates or overwrites files
- `edit_file` -- modifies file content
- `math` -- sandboxed computation (confirmation required despite being sandboxed)
- `web_fetch` -- fetches a URL (SSRF-protected, but makes network requests)
- `web_search` -- web search via Tavily API (makes network requests)
- `task` -- spawns an autonomous sub-agent
- `plan` -- spawns a planning sub-agent, plus post-execution review gate
- `web_search` -- web search via self-hosted SearxNG (makes network requests)
- `task_agent` -- spawns an autonomous sub-agent
Note: The JSON schema metadata key `auto_approve` controls membership in
`AGENT_AUTO_TOOLS`/`TASK_AUTO_TOOLS` (used for agent sub-sessions). The actual
runtime approval behavior is determined by the `needs_approval` field set in
each `_prepare_*` method on `ChatSession`. These two mechanisms can differ.
`TASK_AUTO_TOOLS` (used for task agent sub-sessions). The actual runtime
approval behavior is determined by the `needs_approval` field set in each
`_prepare_*` method on `ChatSession`. These two mechanisms can differ.
---
@@ -165,12 +155,9 @@ Every tool defines a `primary_key`. The mapping is:
| `write_file` | `content` |
| `edit_file` | `old_string`|
| `search` | `query` |
| `math` | `code` |
| `man` | `page` |
| `web_fetch` | `url` |
| `web_search` | `query` |
| `task_agent` | `prompt` |
| `plan_agent` | `goal` |
| `memory` | `name` |
| `recall` | `query` |
| `notify` | `message` |
@@ -194,7 +181,7 @@ Execute a bash command and return stdout + stderr.
- **What it does**: Runs the command in a subprocess with a configurable timeout. Commands are sanitized and checked against a blocklist (e.g. `rm -rf /`). Environment variables containing secrets are scrubbed (`*_KEY`, `*_SECRET`, `*_TOKEN`, etc.).
- **Output format**: Stdout is returned directly. Stderr lines are prefixed with `[stderr]` so the model can distinguish them. When the command itself redirects stderr to stdout (`2>&1`), no prefix is added. Output exceeding 256KB is truncated (head + tail preserved, middle replaced with a truncation notice).
- **Auto-approve**: No -- requires user confirmation.
- **Agent availability**: `task_agent` only (not available to plan sub-agents).
- **Agent availability**: `task_agent` only.
---
@@ -212,7 +199,7 @@ base64-encoded image data for supported image formats.
- **What it does**: For text files, reads and returns content with line numbers. For image files (PNG, JPEG, GIF, WebP, BMP, TIFF, ICO), returns image data as multi-part content when the model supports vision, or a text description when it does not. SVG files are read as text. Images larger than 4 MB are rejected. Must be called before `edit_file` on the same path (the session tracks which files have been read).
- **Vision support**: Controlled by `ModelCapabilities.supports_vision`. All commercial OpenAI and Anthropic models have vision enabled. Local models (vLLM, llama.cpp, NIM) default to off — enable via `[models.*.capabilities] supports_vision = true` in config.toml.
- **Auto-approve**: Yes.
- **Agent availability**: `agent` and `task_agent`.
- **Agent availability**: `task_agent`.
---
@@ -268,7 +255,7 @@ Show a unified diff between two files, or between a file and a provided string.
- **What it does**: Returns unified diff output using Python's `difflib`. Binary files (containing null bytes) are rejected with a clear error. Files read through `diff_file` satisfy `edit_file`'s read guard — you can diff then edit without a separate `read_file` call. Large diffs are streamed with early cutoff at the tool truncation limit.
- **Auto-approve**: Yes (read-only).
- **Agent availability**: `agent` and `task_agent`.
- **Agent availability**: `task_agent`.
---
@@ -283,44 +270,12 @@ Search file contents for a regex pattern.
- **What it does**: Recursively searches for the pattern using `grep -rn`. Returns matching lines with file paths and line numbers.
- **Auto-approve**: Yes.
- **Agent availability**: `agent` and `task_agent`.
---
## Computation
### math
Execute Python code for math and computation in a sandbox.
| Parameter | Type | Required | Description |
|-----------|--------|----------|-------------|
| `code` | string | yes | Python code to execute. Must use `print()` for output. |
- **What it does**: Runs Python code in a sandboxed environment with pre-imported libraries: `sympy`, `numpy`, `scipy`, `math`, `fractions`, `itertools`, `functools`, `collections`, `decimal`, `operator`, `random`, `re`, `string`. Common sympy names (`symbols`, `solve`, `simplify`, `sqrt`, `Matrix`, etc.) are pre-imported. `pytest` is also available for import.
- **Installation**: `sympy`, `numpy`, `scipy`, and `pytest` require the `[sandbox]` extras group: `pip install turnstone[sandbox]` (included in `[all]`).
- **Auto-approve**: Yes.
- **Agent availability**: `agent` and `task_agent`.
- **Agent availability**: `task_agent`.
---
## Information
### man
Read a man page.
| Parameter | Type | Required | Description |
|-----------|--------|----------|-------------|
| `page` | string | yes | The man page name (e.g. `grep`, `socket`, `printf`). |
- **What it does**: Returns the full formatted manual entry. Preferred over `bash('man ...')` or `web_search` for command/API documentation.
- **Auto-approve**: Yes.
- **Agent availability**: `agent` and `task_agent`.
---
### web_fetch
Fetch a URL and extract specific information from it.
@@ -332,7 +287,7 @@ Fetch a URL and extract specific information from it.
- **What it does**: Fetches the URL, strips HTML to plain text, and uses the LLM to extract the answer to the question from the page content. Protected against SSRF (blocks private/internal IPs).
- **Auto-approve**: No -- requires user confirmation (makes network requests).
- **Agent availability**: `agent` and `task_agent`.
- **Agent availability**: `task_agent`.
---
@@ -344,20 +299,55 @@ Search the web using a text query.
| `max_results` | integer | no | Max results to return (default 5, max 20). |
| `topic` | string | no | Search topic: `general`, `news`, or `finance` (default `general`). |
| `category` | string | no | Search category: `general` (default), `news`, `it` (code/tech), or `science`. Maps to SearxNG categories; the model picks per query. |
- **What it does**: Searches the web and returns ranked results with titles, URLs, and content snippets. Uses provider-native search when available:
- **Anthropic**: Replaced at the API boundary with Anthropic's `web_search_20250305` server-side tool. Claude decides when to search; the API executes it and returns results with citations inline. No Tavily key needed.
- **Anthropic**: Replaced at the API boundary with Anthropic's `web_search_20250305` server-side tool. Claude decides when to search; the API executes it and returns results with citations inline. No backend needed.
- **OpenAI search models** (`gpt-5-search-api`): Replaced with `web_search_options` parameter. The model always searches and returns `url_citation` annotations.
- **Local/vLLM models**: Falls back to the Tavily API. Requires `tavily_key` in `config.toml` or `$TAVILY_API_KEY`.
- **Local/vLLM models**: Falls back to a self-hosted [SearxNG](https://searxng.org) instance. Set `searxng_url` in `config.toml``[tools]` or `$TURNSTONE_SEARXNG_URL` (the docker-compose stack bundles a `searxng` service and points at it by default). Operators with a custom MCP search server can instead set `web_search_backend = "mcp:server:tool"`.
- **Auto-approve**: Yes (auto-approved for all tool dispatch paths).
- **Agent availability**: `agent` and `task_agent`.
- **Agent availability**: `task_agent`.
---
### Reranking (optional)
`web_search` can use an external **reranker** to re-order the backend's result pool by relevance to the query before returning the top hits. Turnstone runs no reranker model itself; it POSTs to a Cohere/Jina-compatible `/rerank` endpoint (self-hosted [vLLM](https://docs.vllm.ai) / [TEI](https://github.com/huggingface/text-embeddings-inference) / llama.cpp, or hosted Cohere/Jina/Voyage).
**Disabled by default.** In the console **Models** tab, add a model definition whose `base_url` is a Cohere/Jina-compatible `/rerank` endpoint and whose capabilities include `{"supports_rerank": true}`, then select it under **Models → Roles → Reranker**. It's managed like every other model (write-only key, enable/disable, calibration). The reranker is purely this per-model definition — there is no global `rerank_url`-style endpoint setting.
The `rerank_web_search` toggle defaults on once a reranker is selected. If the endpoint is unreachable or errors, web_search falls back silently to the backend's native result order — reranking never makes a search fail.
When `rerank_bm25` is enabled, the candidate text for memory, tool, and skill retrieval (memory name/description/content and tool/skill names + descriptions) is also sent to the rerank endpoint — a self-hosted endpoint (vLLM/TEI/llama.cpp) keeps it on your infrastructure, a hosted provider (Cohere/Jina/Voyage) sends it off-box.
**Serving a Qwen3-Reranker with vLLM.** The model is instruction-aware, so vLLM **must** apply its chat template — pass `--chat-template` explicitly. Without it the bare query produces near-random scores and reranking actively *hurts* retrieval (verified: an irrelevant passage outscored the correct one):
Then add a reranker model in the **Models** tab with `base_url``http://vllm:8000/rerank` (model name `qwen3-reranker`) and select it under **Models → Roles → Reranker**.
For an endpoint that does *not* apply the model's template, set `rerank_instruction` instead — Turnstone then wraps each query as `<Instruct>: {instruction}` / `<Query>: {query}` (Qwen3's own default is `Given a web search query, retrieve relevant passages that answer the query`). Use the chat template **or** the instruction, not both (they double-wrap).
**Picking `rerank_bm25_threshold`.** The relevance floor that gates proactive memory injection is a probability in `[0, 1]`, but the right value differs per model (a sharp 0.6B reranker may want ~0.95; a broader 4B ~0.33). Calibrate it against your endpoint:
```bash
turnstone-admin rerank-calibrate # probe the endpoint, recommend a floor
It reports the score scale, whether the endpoint cleanly separates relevant from irrelevant probes (a **"no clean separation"** result flags a mis-served or weak reranker), and the suggested floor. Leave the threshold at `0` to rerank-without-filtering.
---
## Agent
Tool names use the `_agent` suffix — bare `plan` / `task` collide with
The tool name uses the `_agent` suffix — bare `task` collides with
chat-template channel names on some local models.
### task_agent
@@ -368,23 +358,9 @@ Delegate a general-purpose task to an autonomous sub-agent.
|-----------|--------|----------|-------------|
| `prompt` | string | yes | Complete task description for the sub-agent. |
- **What it does**: Spawns a sub-agent that inherits the `TASK_AGENT_TOOLS` set (read, write, edit, search, bash, math, man, web tools, memory tools). The sub-agent runs autonomously to completion. Use for work that requires file modifications or command execution.
- **What it does**: Spawns a sub-agent that inherits the `TASK_AGENT_TOOLS` set (read, write, edit, search, bash, web tools, memory tools). The sub-agent runs autonomously to completion. Use for work that requires file modifications or command execution.
- **Auto-approve**: No -- requires user confirmation.
- **Agent availability**: Not available to sub-agents (top-level only).
---
### plan_agent
Plan before implementing -- an autonomous agent explores the codebase and writes a structured plan.
| Parameter | Type | Required | Description |
|-----------|--------|----------|-------------|
| `prompt` | string | yes | What to plan -- the goal, constraints, and scope. |
- **What it does**: Spawns a planning sub-agent with `AGENT_TOOLS` (read-only tools: `read_file`, `search`, `math`, `man`, `web_fetch`, `web_search`). The agent explores the codebase and writes a structured plan to `.plan-<ws_id>.md` (unique per workstream, so concurrent workstreams never collide). If the `plan` tool has been called before in the same session, the prior plan is passed to the agent as context so it refines rather than restarts. After completion, the user is prompted to review and can accept, reject, or annotate the plan.
- **Auto-approve**: No -- requires user confirmation, plus post-execution review gate.
- **Agent availability**: Not available to sub-agents (top-level only).
- **Agent availability**: Top-level only.
---
@@ -445,7 +421,7 @@ Provide either `username` for user-based targeting or `channel_type` +
- **What it does**: Sends a notification via the channel gateway's HTTP endpoint (`POST /v1/api/notify`). The server queries the `services` table for healthy channel gateways, authenticates with a service JWT (`aud: turnstone-channel`), and delivers to the first healthy gateway. On failure, retries up to 2 additional times with backoff (1s, 3s). Rate-limited to 5 notifications per turn (counter only increments on success).
- **Auto-approve**: Yes — notifications are time-sensitive and auto-approved so the model can alert users urgently.
- **Agent availability**: `agent` and `task_agent`.
- **Agent availability**: `task_agent`.
> See [Channel Integrations: Notifications](channels.md#notifications)
> for the full delivery flow, service registry details, and security
@@ -521,7 +497,7 @@ data.get("mergedAt") is not None
- Duplicate names rejected within the same workstream.
- **Auto-approve**: `create` requires approval; `list` and `cancel` are auto-approved.
- **Agent availability**: Main session only — not available to plan/task sub-agents.
- **Agent availability**: Main session only — not available to task sub-agents.
> See [Watch Architecture](diagrams/png/18-watch-architecture.png) for the
> full poll → evaluate → dispatch flow.
@@ -554,33 +530,30 @@ pre-configure skills at workstream creation.
- **Task sub-agents** — via `self._task_tools` (merged list)
- **Plan sub-agents** — via `self._agent_tools` (merged list)
### Naming convention
@@ -774,7 +747,7 @@ MCP tool lists stay up-to-date without restart through two mechanisms:
When tools change, `MCPClientManager` rebuilds its merged tool list using copy-on-write
(new list/dict objects assigned atomically) and notifies all active `ChatSession`
instances via registered listener callbacks. Each session rebuilds its `_tools`,
`_task_tools`,`_agent_tools`, and reconstructs its `ToolSearchManager` (if active),
`_task_tools`, and reconstructs its `ToolSearchManager` (if active),
preserving the set of previously expanded (discovered) tools.
```
@@ -830,7 +803,7 @@ Use read_resource(uri='...') to access the resources listed above.
- **What it does**: Reads the resource from its MCP server via `MCPClientManager.read_resource_sync()`. Returns text content for text resources or base64-encoded data for binary resources. Output is truncated by the standard tool output limiter.
- **Auto-approve**: No -- requires user confirmation (reads external data).
- **Agent availability**: `agent` and `task_agent`.
- **Agent availability**: `task_agent`.
### Capability guards
@@ -872,7 +845,7 @@ the `initialize` handshake. Each prompt is stored with its prefixed name
- **What it does**: Invokes an MCP prompt by name via `MCPClientManager.get_prompt_sync()`, expanding it into messages. Returns the expanded prompt content formatted as `[role]: content` blocks joined with blank lines. The prompt catalog is listed in the system message so the model knows which prompts are available. Output is truncated by the standard tool output limiter.
- **Auto-approve**: No -- requires user confirmation (invokes external prompt servers).
- **Agent availability**: `agent` and `task_agent`.
f"* **Economy and progression bands** — every one of the {settings_count} `settings` fields must sit in its allowed range (the table above), and `growth` must be present and non-negative.",
f"* **Glyph safety** — every terrain, location, and legend glyph must render exactly one column and must not be a reserved marker ({reserved}).",
"* **Map integrity** — `width`/`height` in band, every `terrain_rows` row exactly `width` long with `height` rows, and every row character in the `legend`.",
"* **Walkability** — `spawn` and every placed location must sit on walkable terrain (and no two locations share a cell).",
f"* **Display-name length** — every monster, item, and location name within `{loader.MAX_NAME_LEN}` printable characters; content lists within their caps.",
"* **The fight row** — `events.json` must hold at least one `fight` entry, with weights `> 0`, `min <= max`, and amounts in their per-kind band.",
'* **Cross-references** — `legend` → terrain key, location placements → `locations.json` keys, `starting_weapon`/`starting_armor` → item ids, `boss_monster` → a monster flagged `"boss": true`, `rare_drop_item` → a consumable item id, and `forge_ore_item` → a `material` item id.',
"* **Zone tiers** — every zone's tier band must overlap at least one monster tier.",
"* **Dungeon ladder** — every `dungeon_tiers` tier must have a non-boss monster, and that tier's FIRST monster (its fixed rung guardian) must not be `rare`.",
'* **Exactly one boss** — at most one monster may carry `"boss": true`.',
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Patrick Buckley
