Copilot: bash_output's schema promises the exit code once the shell has
exited, but the formatting attached it only to 'completed' — a killed
shell has one too (the negated signal number). Attach it to any exited
state.
Code-quality: the registry test file mixed a top-level from-import with
function-local 'import ... as bg_mod' for monkeypatching module
attributes; one from-style module alias at the top now serves all of
them.
Restore 'start a dev server, use it in a later call' as an explicit opt-in
after #816 made bash reap its whole process group on return. The surface
mirrors the dominant coding-agent convention: bash(run_in_background=true)
returns a bash_N handle immediately; bash_output(id, filter?) returns only
output produced since the previous read plus status and exit code;
kill_shell(id) terminates the shell's whole process group.
- Per-session BackgroundShellRegistry: capped rolling line buffer with
drop-oldest gap accounting, exit-order record pruning, owner scoping for
task_agents (shells reaped when the agent finishes), liveness-guarded
group kills (a stale pgid is never signalled), budgeted teardown joins.
- Exit notices ride a shared external-event rail (sanitize, soft cap,
channel 'any', idle wake) now common to watch fires; a new 'quiet'
NudgeQueue channel lets a user cancel defer pending notices without
letting them re-wake the stopped workstream, and failed wake delivery
re-queues external notices seq- and predicate-intact without re-arming
the wake gate.
- The bash_output filter runs in a killable subprocess: sre holds the GIL
for an entire search, so no in-process timeout can bound a hostile
pattern. Scrubbed child env, pinned UTF-8 pipes, honest timeout-vs-
helper-failure error taxonomy, per-line match window with explicit
clipping notes; a failed filter never consumes the delta.
- run_in_background rides the bash intent-judge projection; bash_output is
exempt from the repeat warning but still recorded so interleaved polls
keep breaking other tools' streaks; all bash boolean args share one
lenient coercion dialect.
- Shells survive generation cancel and die with the workstream: every
teardown path funnels through ChatSession.close(); CLI exit and the
server lifespan now close every loaded session, signal-first and
Ctrl-C-safe, so nothing detached outlives a graceful shutdown.
CodeQL flagged tempfile.mktemp as an insecure temporary file and Copilot flagged the same call as race-prone (the path is not reserved). Use the pytest tmp_path fixture, which reserves a unique per-test directory and is cleaned up automatically.
main was missing the 1.7.1 through 1.7.3 sections and the two-track preamble that shipped on stable/1.7; bring them in and add an Unreleased entry for the bash background-hang fix.
A bash command that leaves a process running in the background (server &, a daemon) could wedge the whole workstream forever: the tool read stdout/stderr to EOF, which never arrives because the child inherits the pipe, and the timeout watchdog bailed the moment the tracked bash exited.
Wait on the tracked process bounded by the tool timeout (keyed on process exit, not pipe EOF) and terminate its whole session group on every exit path, reaping any backgrounded survivor, forcing the drain threads to EOF, and leaving nothing to leak. Decode with errors=replace so undecodable output is preserved instead of dropped, and pre-bind proc so a Popen failure surfaces the real error.
Behavior change: a process the command backgrounds no longer survives the call. First-class opt-in backgrounding is left as a separate change.
engineer.md is the default BASE module for non-coordinator sessions.
Rework it from posture-level guidance to explicit process discipline:
phased work (understand, design, plan, edit, verify) with ceremony
scaled to the size of the change, red-green as the default for
testable work, minimal-diff scoping, a thrash-stop after repeated
failed attempts, and reporting only observed results. Exploration
delegates to task agents; push-back happens once, then defers with
the disagreement stated for the record.
Review follow-ups: the s-shorthand convention and its glossary echo now
cover Q_E's own state argument (s -> w where Q_E reads it), and the Q_E
glossary row carries the factored (w, a) ~> (w', o) reading so the
symbol table no longer reintroduces the environment-reads-all-of-s
interpretation the outer-kernel note warns against. PRIMER: the
top-alone-widens bullet keeps owner language anchored to the
simple-case top; success is defined as an accepted end, consistent
with the declared-vs-actually-right distinction two sentences later.
HYPOTHESIS.md:
- carry the initial law mu_0 in the tuple (and its displayed signature);
split the rejection symbol into parse failure vs authorization
refusal, with gamma(s, bot_Y) = bot_A as an axiom and a positional
convention for the remaining bare bots
- factor the state s = (q, w) and retype Q_E to (w, a) ~> (w', o) so the
latent world has a generator and the displayed T is its stated
projection; quantify fail-closed over a rejection-invariant safe set K
- read H_ok as operational acceptance (H_acc) against analysis-only
success G, with a convention for which claims read which side; score
C6's ceiling against G and pin C5's slack to the correct-halting
drift, resolving the tension with its own falsifier
- state ledger integrity relative to an attestation assumption (reported
vs actual effects); split cancellation into safe vs unresolved and
count unresolved as possibly-bad; add the realizability clause to
C1/C3; admit multi-principal trust tops as deployment choices
- reversibility is declared in the tool contract the gate reads at
authorization; the returned record's mark is confirmation, not source
PRIMER.md: mirror the same corrections in plain language -- ceiling not
cliff for the desk wall, contract-first reversibility, the multi-party
trust top (including the summary line), declared-vs-actual success on
dashboards, reported-vs-actual ledger honesty, cancel is not
automatically safe.
Onboard the GPT-5.6 family (GA 2026-07-09) to the OpenAI Responses lane.
- Capability rows for gpt-5.6 (= Sol alias/catch-all), gpt-5.6-terra, and
gpt-5.6-luna: 1.05M context, 128K output, tool_search/vision/pdf/reasoning
replay, default effort medium, temperature only at effort=none.
- "max" reasoning effort, Sol-only; Terra/Luna cap at xhigh (the knob's "max"
snaps to the xhigh ceiling). First commercial OpenAI use of "max" — the
ordinal knob already ranked it, so no effort-ladder change was needed.
- Verbosity and pro mode as operator-declared capability fields
(supports_verbosity/verbosity, supports_pro_mode/reasoning_mode), merged
from the model-definition capabilities JSON and emitted on the Responses
wire as text.verbosity and reasoning.mode. Both are gated by a supports
flag plus an enum guard that drops unknown values with a warning. Pro mode
is Sol-only. There is no gpt-5.6-pro model — "pro" is the reasoning.mode
param, not a separate model id.
- Raise the openai floor to >=2.44 for the 5.6 Responses params.
Unit and wire-golden tests cover the rows, max->xhigh snapping, the two
levers, and the enum guards. Validated live against the OpenAI API: gpt-5.6
accepts the model id, effort "max", text.verbosity, and reasoning.mode="pro".
The #805 server-side fixes (emit-time batching, _ListenerQueue poison,
out-of-band closing) already cover every SSE stream, but the client-side
companions lived only in the interactive pane. Port them to coordinator.js
and extract the drift-prone pure core into a shared module (closes#806).
- shared_static/sse_overflow.js (new): storm-guard constants +
overflowWindowTripped + degradedCooldownStep, imported by both panes so the
trip threshold and cooldown ladder have one source of truth. interactive.js
imports these instead of holding local copies; the two node runtime probes
move to tests/test_sse_overflow_js.py.
- coordinator.js: handle the stream_overflow frame (storm guard -> degraded
catch-up with a doubling cooldown; the reconnect replays from the ring, or
falls to the replay_truncated -> /history floor); add the close-on-hide /
replay-on-show visibilitychange handler plus a document.hidden guard at the
connectSSE chokepoint; add drop-vs-render-wedge counters (onmessage now wraps
the dispatch in try/catch -- the coordinator previously had no wedge guard,
so a handler throw silently poisoned every later turn).
- After a stream gap the children/tasks sidebar re-syncs only when the ring
replay cannot cover it: no resume cursor, a replay_truncated envelope, a gap
beyond the cursor-trust window, or a live event id below the saved cursor (a
process restart reset the counter, which the replay path reports as a false
replay_ok). child_ws_*/task events are ordinary ring entries, so an ordinary
short reconnect heals the sidebar through the live handlers with no REST
rebuild -- a momentary blur/focus under close-on-hide rebuilds nothing.
- Close-session teardown detaches the visibility handler before the close POST
so a hide/show mid-close can't resurrect a dying stream. A replay_truncated
seen mid-stream is deferred (not dropped) and re-synced from /history on the
next idle -- repairing both a ring-evicted gap and a turn stranded by
close-on-hide (stream_end evicted while hidden), matching interactive.js's
_pendingTruncatedResync.
The extraction stops at the pure core: interactive.js's stateful glue is
hard-pinned by its source-assertion suite, so its class-method shape stays put
and the coordinator reimplements the equivalent glue as closure functions.
Tests: new test_sse_overflow_js.py (module exports + the two runtime probes);
coordinator parity + lifecycle pins in test_app_js.py (replay-aware sidebar
refresh, restart detection, truncated-resync deferral, close-session
visibility detach); interactive's moved probes replaced by an extraction pin.
All JS-source suites green.
HYPOTHESIS.md:
- New appendix entry "Daemons (the recurrent harness)": a daemon as the
regenerative process of concatenated runs — ready-set recurrence,
renewal-reward lifting exactly at regeneration points, accumulation as
what breaks regeneration (cross-cycle provenance meet, renewal events
that reset accumulated risk), and authority under intermittence
(owner contact as a renewal point for authority; TOCTOU at cycle
scale).
- New body section "The loop": the task-dispatching outer loop as the
harness construction applied one level out — the composition
correspondence read at the top level, the daemon as its single-agent
special case, the bare while-loop as the trivial-group harness one
level up. Flagged as a sketch; outer fail-closed/reach-avoid
treatment deferred to later rounds.
- Veto caveat threaded to match: judge-as-veto safety scoped to the
authority lattice, and the nonblocking escape degrades to an
always-enabled safe halt when the principal is unreachable.
- Consistency: Grounding's Asserted tier now covers "The loop";
"always-enabled escalation" -> "escape" (the appendix's own term, now
that the escape has an unattended form); brace the one unbraced \bot
subscript (linter section-B HIT).
PRIMER.md: new plain-language companion — same object, no symbols, the
formal doc wins every disagreement. README's entry link now points at
the primer, which links onward to HYPOTHESIS.md.
highlight.js, KaTeX and Mermaid all run on every surface via the shared
renderer (renderer.js), but their theme/wrapper CSS lived only in
ui/static/style.css. The console and coordinator load /static/style.css from
console/static/ — a different file on a different server — so hljs token spans
fell back to --fg (flat monospace for several releases), and the KaTeX/Mermaid
wrappers lacked their overflow containers, letting wide equations/diagrams
overflow the pane.
Move the hljs theme, .katex-display/.katex-error and the .mermaid-* wrappers
into shared_static/chat.css, which every surface loads via /shared/chat.css.
Restate the mermaid width-clamp for the preview pane (.preview-markdown) too,
since its content isn't a .msg.assistant message.
Drop the redundant background on .msg.assistant pre code.hljs so the <pre>
carries the code surface on every surface — otherwise the console/coordinator
(where the pre is --panel, not --code-bg) showed a darker box inside a lighter
padding band.
A scheduled task could pin the model and skill of the workstream each
firing creates; it can now also pin its persona and project, so a
schedule can run under, e.g., the researcher persona attached to a
specific project's memory bucket.
The two values live on scheduled_tasks (migration 066, Text NOT NULL
default '') and are passed verbatim to create_workstream at dispatch,
where the node resolves the persona for the workstream kind and gates
the project attach. Empty means "kind-default persona / no project",
resolved late at each firing (mirrors how empty model/skill already
behave) -- existing schedules keep byte-identical dispatch behaviour,
so there is no backfill.
Also fixes a latent bug this feature depends on: admin_create_schedule
read created_by from request.state.user_id, which AuthMiddleware never
sets, so every scheduled task stored created_by=''. It now reads
auth_result.user_id like every other console endpoint. This is now
load-bearing -- the scheduler dispatches under created_by and the node
gates the project attach against it. admin_update_schedule adopts the
editing admin as owner when a project is assigned to a pre-fix orphaned
('') schedule, and re-validates persona/project only when they change
so a since-disabled persona or lost membership does not block unrelated
edits (the node re-checks at dispatch either way).
Wired through: schema + migration (up/down + parity tested), both
storage backends, API schemas, SDK create_workstream and console
create_schedule/update_schedule, scheduler dispatch, and the admin
schedule shelf (persona + project pickers, current value preserved so
an edit cannot silently clear a filtered-out selection).
The URL-extraction call hard-coded max_tokens=8192 and rode the "low"
reasoning default, which broke local-inference models whose registry entry
advertises a tighter output limit or a different reasoning config. Inherit
the session/registry max_tokens and reasoning_effort instead (temperature
already was) — the same knobs the main turn uses.
max_tokens is capped to context_window // 4, the ~25% output slice Phase 2
already reserves, matching the main turn's response reserve
(_remaining_token_budget), so a large operator budget can't push
prompt + output past a small context window on strict runtimes.
github-code-quality flagged 8 spots where tests imported
turnstone.core.session_ui_base both as `from ... import` and `import ... as
suib` (the alias was only there to monkeypatch the module-level batch
constants). Drop the alias and patch via string target
(`monkeypatch.setattr("turnstone.core.session_ui_base._TOKEN_BATCH_WINDOW_SECS", ...)`),
which resolves to the same module global — behavior-identical. The one test
that READS the constant imports the symbol directly. Test-only, no
production change.
PR #805 review (Copilot + the round-3 finding it corroborates):
- connectSSE opened a new EventSource even when the tab was already hidden
(e.g. a first load in a background tab), where the close-on-hide handler
never fires because there is no open stream to close — so a throttled
hidden tab could still become the slow consumer this PR prevents. Add the
document.hidden guard at the single connect chokepoint, after the wsId
assignment + visibilitychange-handler install (so the show edge reconnects)
and before new EventSource (so nothing opens). The timer callbacks keep
their own pre-checks (the recover beat's also gates failCount); this closes
the fresh-connect path they never covered.
- Fix the stale _ListenerQueue.closing docstring: it claimed the drain loop
checks closing BEFORE poisoned, but the round-2 fix moved that check INSIDE
the poison branch (poisoned+closing -> clean close; a healthy closing queue
drains its tail to the ws_closed sentinel). Wording now matches the code.
A long live session driven by a fast local model (500-2000 tok/s) showed
corrupted / missing spans of assistant text while the backend stayed
healthy. Root cause: on_content_token/on_reasoning_token enqueued one SSE
event per model delta, so the per-listener queue (cap 500) overflowed
against any slow consumer; put_nowait on a full queue silently dropped the
newest event. Once saturated, drops scatter (the consumer keeps freeing
single slots), so the client's lastEventId sails past the holes and
reconnect-replay (eid > last_event_id) can never heal them. A dropped
fence-closer reshapes all downstream markdown -> reads as heavy corruption.
Fix B (primary) - emit-time micro-batching:
Coalesce content/reasoning fragments over a ~25 ms window (or 4 KB) into
one _enqueue, cutting the wire event rate ~10-20x at local-inference
speeds. A batch is assembled before it gets an _event_id, so it is one
ordinary ring entry no cursor can fall inside (unlike the forbidden
in-ring coalesce). Two conditions are load-bearing and pinned:
1. The inflight-buffer append and the enqueue are one _ws_lock section,
so a snapshot's snap_seq stays a true high-water mark for its text.
Splitting them lets a straddling snapshot double-render (the client
content path is a blind +=, no dedup).
2. Every non-token emit flushes the pending batch first, enforced at the
single _enqueue choke point, so stream_end/tool_*/state_change can't
overtake trailing content and repaint it into a new bubble.
Fix A (recovery net) - poison-at-first-overflow:
_ListenerQueue latches `poisoned` atomically at the FIRST rejected put and
refuses every later put, freezing its contents as a contiguous prefix; the
drain loop closes the stream after an id-less stream_overflow frame and the
native EventSource reconnect replays the whole gap from the ring buffer.
Poisoning at the first full (not after N) is required: any deliver-while-
dropping window advances lastEventId past interior holes that reconnect
can't replay. A ws teardown that races the overflow sets an out-of-band
`closing` flag (mark_closing), checked inside the drain loop's poison
branch: a poisoned+closing queue returns clean (no false overflow frame),
while a healthy closing queue still drains its full tail FIFO to the in-band
ws_closed sentinel -- so a slow-but-unpoisoned client never loses the turn's
final content batch + stream_end at teardown.
Client (interactive.js):
- Reconnect storm guard: after 3 overflow closes in 60 s the pane drops to
a degraded catch-up (stop live streaming, "connection is slow" state,
reconnect after a doubling 15->120 s cooldown that resyncs from the ring
or the uncapped /history floor). The cooldown ladder is keyed off a
last-trip timestamp, not the overflow-window array (which the trip
clears), so the escalation survives its own backoff.
- Close-on-hide / replay-on-show: a visibilitychange handler closes the
EventSource on tab-hide (a throttled hidden tab is the likeliest slow
consumer) and reconnects with the saved Last-Event-ID on show. The
factory recovery beat defers when hidden, and giveUp() detaches the
handler, so a dead or backgrounded controller can't reopen a stream.
- Drop-vs-render-wedge counters distinguish this bug (server overflow
closes) from the handler-wedge class (render/finalize throws) in the
field. No global gap-detector: live ids are not strictly monotonic
across concurrent tool+content emit, so a naive id!=last+1 check would
false-positive; recovery is server-signalled instead.
Corrects the stale _resolve_event_buffer_max comment that justified the
50k ring on a "PR-G closes connections on hide" mitigation that never
existed (the close-on-hide handler above is the real one).
Negative-tested (revert the guarantee, confirm the pin fails, restore):
per-token inflight append -> snapshot straddle double-render; removed
choke-point flush -> stream_end split; no poison latch -> silent drops;
top-of-loop closing check -> healthy-close tail loss; missing mark_closing
wiring / drain closing check -> clean close mis-reported as overflow;
_noteStreamOverflow cooldown reset -> ladder never escalates; removed
hidden-tab recovery guard / giveUp handler removal -> hidden-tab reconnect.
Copilot review on PR #804:
- An indented closing fence line (" ```") left its leading spaces as a
trailing whitespace-only line inside the rendered code block: the content
capture runs up to the backtick run and the close-line indent precedes it, so
it was captured as content. Strip a trailing newline PLUS any trailing indent
(/\n[ \t]*$/ instead of /\n$/); a column-0 close is unaffected. Red-green
pinned (content is exactly " x = 1", no trailing whitespace line).
- Correct a stale test docstring claiming the fence open anchor allows "up to 3
spaces" of indent — it allows arbitrary indent (the 4-space case is pinned
separately).
The markdown renderer protects structural blocks with in-band NUL-framed
sentinels (chr(0)+tag+index+chr(0)). escapeHtml preserves U+0000, so
model/tool text could forge sentinels, and recursively-rendered <details>
bodies re-rendered against fresh block arrays and lost their content. This
lands the ordered containment fixes from the render-containment brief.
Fixes (each pinned in tests/test_renderer_js.py; all NUL-sensitive cases also
confirmed in real headless Chrome, which drops a U+0000 token the node harness
preserves):
- B1/B2/B3 — forged sentinels: strip U+0000 at the TOP-LEVEL render entry only
(_fnDepth === 0). renderer.js is the sole NUL producer and every restore
regex is NUL-framed, so removing NUL closes every forgery path (block
duplication/relocation, out-of-range "undefined", cross-container injection)
while generated sentinels in recursive frames survive. Only NUL is stripped,
so a code fence still shows pasted control bytes (ESC/FF/VT/DEL) verbatim.
- B4 — blockquote-in-fence (the common one): the code-fence pass now runs
before the line-based blockquote pass. Its open matches at line start after
optional indent and an optional list marker (`- `, `1. `), and re-emits that
indent+marker before the sentinel so the fence keeps its document position
(a nested-list item stays nested; a fence continuing a footnote definition
keeps the indent its continuation scan needs). A blockquoted fence (`> ```)
is not matched (`>` is neither indent nor a list marker), so the blockquote
pass extracts that `> ` run and its recursion renders the fence. A `> ` line
inside a plain fence stays literal.
- B5 — <details> open anchored to line start (^[ \t]*), so a `<details>`
mentioned mid-line inside inline code no longer starts a block.
- NEW-1 — recursive-frame content loss: <details> extraction runs AFTER fence
protection and restores a fenced body from a saved raw-source array
(codeBlockRaw) back to raw markdown before the recursive render, so
code-in-details renders in-frame instead of restoring to "undefined". Running
after fence also means a </details> shown as example code inside a fence
can't close the block early, and a <details> shown inside a fence stays
literal — no offset-based fence-awareness needed. Inline-code/math in footnote
definitions render via the restore round-trip the undefined-guard enables
(documented at the append site).
- NEW-3 — code blocks gained the <p>SENTINEL</p> unwrap variant DT/BQ/MB/TB
already had, removing a stray empty <p> before a standalone <pre>. The CB
unwrap is whitespace-tolerant so an indented own-line fence (whose indent the
fence pass re-emits) also doesn't leave a stray <p>.
- Defense-in-depth: every restore callback returns the matched sentinel
(inert; the browser drops the NUL) instead of the array's `undefined`.
Non-obvious decisions:
- Control chars are authored as literal \xNN hex escapes (byte-verified: only
\uXXXX decodes to raw bytes in this toolchain; \xNN matches the file's
existing \x00 sentinel convention).
- Open anchors allow arbitrary leading indent (the fence open also allows a
list marker), not CommonMark's ^ {0,3}: the renderer has no indented-code
fallback, so preserving the prior behaviour of matching indented/list-nested
fences beats CommonMark strictness, while still excluding `> ``` and mid-line
forms.
- codeBlockRaw (the <details> raw-fence array) and the restore callbacks are
factored through a _restorer(arr) helper; codeBlockRaw is only populated when
the text contains a <details> tag (its sole reader).
- The entry strip is depth-0-only on purpose: an unconditional strip would
shred the generated sentinels recursive frames carry, foreclosing NEW-1.
Negative-tested (reverted the production line, confirmed the pin fails):
- fence anchor: unanchored swallows a blockquoted fence.
- NEW-1 codeBlockRaw restore: without it, code inside <details> is lost.
Deferred (called out per the brief):
- B6/NEW-4 bidi controls (U+202A–202E, U+2066–2069, U+200E/F) still pass
through unescaped; they are not C0 so the entry strip misses them. Left to a
follow-up — stripping risks corrupting legitimate RTL text and <bdi>
isolation is involved for a string renderer.
Review follow-up: the helper returned `fallback` verbatim when the name
sanitized to empty, so a future caller passing an unsafe fallback (non-latin-1,
control chars, quote, backslash) could reintroduce the header crash/corruption
the helper exists to prevent. Not reachable today — all call sites pass safe
ASCII literals — but the helper is a shared safety primitive whose contract is
wire-safe output.
Run the fallback through the same cleaning, backed by a safe constant if even
that is empty, so the return is always wire-safe and never filename="". Adds a
test.
Attachment `/content`, preview, and workstream-export downloads built the
Content-Disposition `filename="..."` value straight from a user-supplied
name, stripping only quotes and CR/LF. Three input classes still broke the
header:
- Non-latin-1 names (CJK, em dash): Starlette encodes header values as
latin-1 and raised, 500-ing the serving route. (The original get_content
bug.)
- ASCII control bytes (NUL, form-feed, VT, DEL): latin-1-encodable, so they
passed Starlette, but the HTTP server layer rejects control characters in a
header value and 500s one layer later.
- Backslash: the RFC 6266 quoted-pair escape. A trailing backslash escaped
the closing quote and corrupted the download filename (not a 500, but wrong
output; Windows-origin uploads carry it legitimately).
Extract one `latin1_safe_filename()` helper in web_helpers that drops every
non-printable character plus the double-quote and backslash quoted-string
metacharacters, folds any surviving non-latin-1 codepoint to '?', and falls
back to a non-empty name so the header never emits an empty filename. Route
get_content, preview_response_headers, and the export handler through it,
replacing three near-duplicate inline strips.
Adds unit tests for the helper (non-latin-1 fold, control-char and backslash
stripping, per-site fallback) and an endpoint regression test.
unsplit() closed each doomed (ephemeral) pane via close() — which already
renders/persists/notifies — then repeated that trio, firing intermediate
persist/notify passes mid-operation. A 2-cell split fully collapses inside
close(), so bail there; only a 3+-cell split (or an empty doom list) still
needs the trailing exit + refresh. The all-conversation path is unchanged.
Also reword the cell-chip CSS comment so it names the reversible hide vs
destructive close glyphs, now that an ephemeral pane can show the close glyph
in split mode.
The preview pane opens beside the conversation as a split cell. Dismissing
that cell — the per-cell chip, or Unsplit from the other pane — ran
closeCell(), which hides the pane but keeps it in _panes/_order, leaving an
orphan tab with no meaningful reopen (the reopen affordance is the transcript
chip, not the tab bar).
Add an `ephemeral` flag on ShellPane. For an ephemeral pane the cell chip and
Unsplit route to close() — destroying the pane and its tab — and the chip's
glyph/label read as a destructive close rather than a reversible hide. Unsplit
still spares the focused survivor even when it is ephemeral ("keep the focused
pane"). The preview pane sets the flag; conversational panes do not, so an
all-conversation split is unchanged (Unsplit reduces to the prior
_exitLayout(_activeId)).
The wake gate documented exactly one info line per call past its
gates, but a send() refusal (the authoritative under-lock _closed
re-check catching a teardown the gate's lockless peek missed) emitted
nothing — a dropped wake should stay traceable to its trigger, so the
refusal now logs nudge_wake.refused.
The already-dispatched branch's comment claimed a held reminder can
coexist with the terminal mark via a redelivery whose commit raised —
impossible with the current control flow (_redeliver_pending clears
the hold before committing). Reworded to what the clear actually is:
the last line of defense against any coexisting hold leaking forever
once this branch deactivates the row, since inactive rows never
re-list. Test comment updated to match.
A session whose _nudge_queue answers has_pending truthily while its
deliver_wake_nudge_from_queue consumes nothing turns the worker-exit
backstop into an infinite respawn loop: the gate passes, the wake
worker no-ops, the exit backstop re-runs the gate, forever.
Mock-backed test sessions riding real Workstreams are exactly that
shape, and one worker on such a pairing is enough to ignite a
wake-thread storm that trips the leaked-thread guard in every
subsequent test. The wake contract requires real drain semantics —
the spawned worker must CONSUME what the gate saw — so the gate now
refuses on type, not just presence.
str | None under-specified the field: the implementation and the TS SDK
union both constrain it to queue_full / refused_closed, and the Literal
projects a proper enum into the generated OpenAPI spec so clients
reject unexpected values. Specs regenerated.
Wake path:
- Denial metacog nudge moves to the tool channel so it drains with the
denied tool batch instead of the next user-message seam.
- wake_workstream_if_pending: shared wake gate for watch fires on
already-idle workstreams (no IDLE transition for the watcher to
observe), wired as wake_fn at every set_watch_runner site via the
shared _watch_fire_wake_fn helper (closes over the Workstream OBJECT
— after eviction+restore an id-keyed manager lookup would miss).
- session_worker exit backstop re-runs the wake gate the moment worker
ownership clears: IDLE fans out on the worker thread, so
transition-time wakes always landed on the reuse path and no-op'd
(the coordinator idle_children strand).
- deliver_wake_nudge_from_queue contains GenerationCancelled — it is
the wake worker's run() closure and only Exception is caught
downstream.
Watch delivery:
- Terminal fires that cannot reach their workstream are HELD and
redelivered on min(interval, 60s) without re-running the command,
bounded by MAX_DELIVERY_ATTEMPTS per cycle and the watch's own
max_polls across cycles; the poll charge commits durably at hold
time so restarts stay budget-bounded.
- Restore admission control: per-ws dedup + MAX_CONCURRENT_RESTORES
cap, presence-only re-check under the lock, detection-only stall
alerts (reclaiming a wedged admission would trade capped degradation
for total poll-pool collapse).
- Permanent-vs-transient restore taxonomy: corrupt persona stamp and
genuinely-missing history (confirmed by a raising storage probe —
the resume loader swallows read blips into []) deactivate the watch
immediately; everything else holds and retries.
- Cancel-race defense: delivery paths re-check is_watch_active before
stashing/dispatching, cancel paths write the row BEFORE
forget_terminal_dispatched, the HTTP cancel endpoint clears runner
state, and a per-tick sweep bounds the residual stash-after-clear
interleaving to one check_interval.
- Abandon/exhaustion commits are write-then-clear so storage that can
read but not write retries the row write instead of re-running the
command every cycle; the fresh-fire unrestorable path stashes before
its deactivation write for the same reason.
Registry follows identity:
- The dispatch registry is keyed by _ws_id at registration time; every
rebind now moves it: non-fork resume() and /new go through
_follow_watch_registration (new key live before the old is removed,
never stealing a registration another live session holds), removals
are owner-checked so tearing down a watch-restore shell or a
resumed-away session cannot unregister a live pane, the restore
shell yields to a registration that appears mid-restore, CLI
--resume registers after the successful resume, and both the open
path and the detail-GET lazy rehydrate wire the registration.
Teardown gating and backpressure honesty:
- cleanup_session_ui marks ws._closed FIRST under ws._lock — every
teardown path (close, close_idle, evict, delete, discard) funnels
through it — and session_worker.send re-checks under the same lock,
so a wake can never spawn a worker on a torn-down workstream.
- Create responses carry initial_message_status when the initial
message could not be delivered (queue_full / refused_closed) instead
of reading as success; staged attachments survive for the retry;
/send surfaces a closed workstream as 404 rather than queue_full.
Docs/spec: OpenAPI artifacts regenerated; api-reference documents the
new create-response field; TS SDK type extended.
Tests: ~30 new pins (cancel races, budget durability across restarts,
owner-checked registry moves, teardown gating, stall alerts,
backpressure surfaces, wait_until final re-check); wide subsystem
sweep green (2353 passed).
Review feedback (PR #800): the URL lane hard-capped fetched bodies at
10 MB before kind resolution, making the 32 MiB pdf cap unreachable for
URL targets while path targets honored it. The flat pre-check is gone;
the guarded fetch's max_bytes now tracks max(PREVIEW_SIZE_CAPS.values())
- mirroring the path lane's stat pre-check - and the per-kind caps after
resolution stay authoritative.
Also drops a redundant function-local asyncio import in test_console.py.
fetch_with_ssrf_guard now streams the response under a max_bytes budget
(default 32 MiB, counted on decoded bytes so gzip cannot expand past it)
instead of buffering blind - an unbounded body previously filled memory
before any caller-side size cap could run. Redirect-hop bodies are no
longer read at all, and the realized response drops stale wire-framing
headers (content-encoding/content-length/transfer-encoding) that no
longer describe the decoded content it carries.
Preview blob ids are salted out of the model-visible attachment
namespace (sha256("preview:" + body)): uploads use bare sha256(body)
and save_attachment freezes kind at first insert, so a byte-identical
preview/upload pair would otherwise share a row - whichever landed
second inherited the other's kind, silently hiding an upload from model
context or materializing preview bytes into a tool turn.
turnstone's primary audience self-hosts it beside other lab services —
a web_fetch or open_preview aimed at Grafana, Home Assistant, or a dev
node on the local network is the operator using their own network, not
an attack. The hard SSRF refusal made those targets unreachable.
New runtime setting tools.allow_private_network (settings registry,
default off, rendered in console Settings → Tools; hot — read per tool
call, no restart). When enabled, a call NAMING a private address
becomes approvable: the approval prompt tags it "(private network)" so
the operator approves it as what it is, and the human gate stays.
The redirect side-door stays closed either way: a PUBLIC target that
302s into private address space is refused regardless of the opt-in —
that address never appeared on the approval card, so it is never
fetched. Only a chain whose approved origin was itself private skips
hop screening (its redirects are the operator's own network).
Refusals now teach the knob (mirrors the oidc opt-in hint): the error
names tools.allow_private_network and where to enable it. Surfaces
without a ConfigStore (bare CLI, eval) stay strict — there is no admin
surface to have opted in on.
Four follow-ups to the preview pane:
- Probe-mode preflight: the pane preflights src-loaded kinds with
GET ?probe=1 (204, real hardening headers, no body) instead of HEAD —
the console reverse proxy forwards HEAD as a full GET, so the old
preflight dragged the whole blob across the node→console hop twice.
Ownership gate + renderable-type check still run on probes.
- Legacy-charset text: table/text/markdown now transcode to UTF-8 at
store time (declared charset → UTF-8 → cp1252-replace ladder), same
model the web kind already used. The ladder applies only when the
text kind was DECLARED (MIME/extension/override); the bare no-hint
fallback stays strict UTF-8 and NUL bytes still hard-reject, so
binary rejection is unchanged.
- Remote assets default OFF: previewed pages are now served under
"sandbox; default-src 'none'; style-src 'unsafe-inline'; img-src
data:; font-src data:" — they render with inline styling but cannot
contact their origin site (no viewer IP/traffic disclosure). A
per-pane "Load remote images & styles" checkbox (web previews only,
sticky, not persisted) reloads with ?assets=1 for the permissive
bare-sandbox mode.
- Markdown vendor parity: preview markdown now runs renderer.js's
postRenderMarkdown (hljs token coloring + lazy mermaid diagrams)
like the conversation pane, with preview-scoped code-block/KaTeX
chrome (the conversation theme is .msg.assistant-scoped).
Tests: probe/assets HTTP + policy coverage, charset ladder units +
stored-bytes round-trip, JS static guards for the probe form, the
default-off toggle, and the post-pass; headless-chrome harness grew to
41 assertions (probe-not-HEAD, toggle visibility/default, fenced-code
render). Full suite green.
Tool results only ever rendered as plain text in the transcript. This
adds the model-driven rich-preview lane every comparable surface has,
in turnstone's developer-tool idiom: a preview pane that opens BESIDE
the conversation, keyboard-operable, sandboxed, never replacing the
transcript that spawned it.
Backend
- New built-in open_preview(target, kind?, title?): resolves an http(s)
URL, a file path, or attachment:<id> to bytes; classifies into
web/pdf/image/table/text/markdown (magic bytes > MIME hint >
extension > UTF-8 fallback, legacy-charset pages transcoded); caps
size per kind; persists content-addressed with kind="preview" —
refcounted and GC'd with the workstream, skipped by trajectory
reconstruction so preview bytes can never materialize onto the wire.
URL targets gate like web_fetch (network egress); paths/attachments
run unprompted like read_file.
- New core.web.fetch_with_ssrf_guard: manual redirect walk that
SSRF-screens every hop BEFORE requesting it (follow_redirects=True
checked nothing between hops); adopted by both open_preview and
web_fetch. URL userinfo is stripped before the descriptor or the
stored bytes see it; <base href> is injected doctype-safely so
relative assets resolve without quirks mode.
- The preview descriptor rides the tool turn's meta side channel with
ONE shape on every boundary: the live tool_result SSE event, the
conversations.meta column, and the /history projection. Cancelled
batches commit an already-announced preview (blob + meta) instead of
stranding the open pane on a permanent 404.
- New GET {ws}/attachments/{id}/preview (read scope, same ownership
gate as /content) serves the STORED type with per-MIME hardening:
bare CSP sandbox for text/html (renderable, scriptless, opaque
origin), no CSP for application/pdf (Chromium's viewer refuses
sandboxed contexts), full default-src 'none' otherwise; filenames
fold to latin-1-safe ASCII. The console /node proxy now forwards
CSP/nosniff/disposition/cache-control instead of dropping them.
- History loads exclude preview blobs from the bulk content fetch at
the query (they were read and discarded on every load).
Frontend
- New "preview" pane type registered in the shared shell (server +
console): openPaneBeside placement, per-kind renderers — fully
sandboxed iframe for pages, browser PDF viewer, sortable tables
(CSV/TSV/JSON, ragged-file safe, 5k-row cap), rendered markdown,
text — plus back/forward history with arrow keys, reload persistence
via pane meta, and backoff auto-retry (0.9s..7.2s) bridging the gap
between the live descriptor and the batch fold that commits its blob.
- Tool results carrying a descriptor render a credential-redacted
preview chip (the reopen + replay affordance); live results auto-open
the pane only while the originating pane holds focus.
Docs: docs/tools.md + prompts/tools.md. Tests: policy unit tests, tool
prepare/exec (mocked fetch), serving route + proxy header pass-through,
storage exclusion on both backends, cancel-path commit, JS static
guards; a headless-Chrome harness drives the real module graph (32 DOM
assertions).
The discovered-endpoint wrapper converted every OAuthSSRFError to a
bare OIDCError, so a private-resolving endpoint or trusted host got the
non-public message without the allow_private_network remediation even
though the same knob fixes it. Hoist the hint into a module constant
and append it in both wrappers.
Also name "unspecified" in the refused-even-with-opt-in message so
0.0.0.0/:: rejections read unambiguously.
The SSRF guard on OIDC endpoint URLs hard-refused any hostname
resolving to a non-public address, which made it impossible to use a
self-hosted IdP (Keycloak, Authentik, Dex) on an internal network —
even though the login-flow issuer is operator-configured, i.e. trusted
input.
Add [oidc] allow_private_network in config.toml (or
TURNSTONE_OIDC_ALLOW_PRIVATE_NETWORK), default off. When set, the
issuer and its discovered endpoints may resolve to private-range,
unique-local, CGNAT, and loopback addresses. Link-local, multicast,
unspecified, and reserved ranges stay refused regardless — cloud
metadata services live on link-local and no legitimate IdP does. The
HTTPS requirement and same-origin endpoint checks are unchanged.
The private-address refusal now raises OAuthSSRFPrivateAddressError,
and the OIDC wrapper appends the remediation hint to the error message
so the failure is self-service. mcp_oauth call sites — where endpoint
URLs come from untrusted remote-server metadata — do not get the knob
and keep the strict public-address rule.
RFC 3986 schemes are case-insensitive, so POSTGRESQL+PSYCOPG2:// or
HTTPS://user:pass@host in tool output leaked the password past the
case-sensitive scheme alternation. Compile with IGNORECASE on both
sides of the FE/backend mirror; the structural userinfo requirement
is unchanged. Uppercase-scheme cases added to both test suites.
Connection-string redaction (the output_guard pattern and its frontend
mirror) only enumerated bare dialects plus +psycopg, so SQLAlchemy
dialect+driver URLs — postgresql+psycopg2://, postgresql+asyncpg://,
mysql+pymysql:// — leaked the password through every redaction surface.
The scheme now takes an optional +suffix instead of enumerating drivers.
redactCredentials() also gains a single early-exit prefilter scan ahead
of its sixteen replace passes, for plain-log tool output on card render.
The prefilter is documented and pinned as a superset of the pattern
set's required substrings, so a miss is provably a no-op: new smoke
cases assert bare sk-/AKIA/Bearer credentials with no '=', quote or '@'
anywhere in the text still redact, alongside the fast-path no-op and
the driver-scheme URLs on both sides of the mirror.
- Proven: name supervisory control's controllability and nonblocking
conditions as the ancestors of gate-early-on-irreversibles and the
always-enabled escalation required behind a learned veto; cite TCSEC
covert-channel analysis (NCSC-TG-030) for the verdict channel.
- Asserted: add the narrow-only rule's influence-side twin (verdict
payloads to the plant selected, never generated).
- New caveat paragraph: a denial is free only in the authority lattice;
in the dynamics it is an input (selection + targeted-liveness
channels), so a learned veto needs a nonblocking escape it cannot
disable, verdict payloads are selected rather than generated with the
symbols/tokens/language thresholds bounding the alphabet, and the
strongest form dissolves the verdict into scheduling over
deterministic checks.
Review: the wrapped ValueError is echoed in 503 bodies, and arbitrary
SDK exception text can embed filesystem paths. Echo the exception type
only; log the full exception with traceback at the raise site.
SDK client construction can fail on environment problems the config
never sees (e.g. httpx resolving a certifi CA path deleted by a venv
rebuild). Those escaped as bare exceptions and turned every workstream
open/create into an opaque 500; re-type them as ValueError in
ModelRegistry.get_client so routes answer 503 with the message and the
alias.
to_tsvector was computed inline over full row content, so one row
whose tsvector exceeds PostgreSQL's 1MB limit aborted every
search_history scan. Cap the FTS input at 250K chars (worst-case
tsvector expansion stays under the limit; giant rows remain findable
by their head). The ILIKE fallback also never ran on postgres: the
failed statement leaves the autobegun transaction aborted, so roll it
back before falling back.
Address review on #794. The prior gate deepcopied every agent tool, then compared, then discarded the copy on a no-op render; and its comment framed the equality case as 'no personas' when it also covers an idempotent re-render of unchanged aliases/personas. Compute the target model/persona descriptions from the current (read-only) schema first and only deepcopy when one differs — so a no-op render is genuinely allocation-free, not just fork-free. Behaviour is unchanged: identity preserved when nothing differs, stale text still cleared on reload.
_render_agent_tool_descriptions rebuilt self._tools and reassigned it on every session init, deep-copying task_agent even with no model aliases and no personas to inject — the single-model CLI case the docstring says is skipped. This regressed after the persona-discoverability change removed the early 'if self._registry is None: return' guard, breaking the session._tools is INTERACTIVE_TOOLS invariant (test_session_without_mcp).
Gate the reassignment on whether a description actually changed: keep the original tool object when the render is a no-op, fork self._tools only when something was injected. Restores the shared-constant invariant, makes repeated renders idempotent, and preserves clear-stale-on-reload (an emptied registry still takes the changed path).
Add .github/FUNDING.yml to enable the native GitHub Sponsor button, plus a Sponsor badge and a Support section in the README. Primary CTA is GitHub Sponsors (eous); PayPal (paypal.me/eousphoros) is offered as a one-off fallback.
Review feedback on #792: the "not found or disabled" branch
interpolated the raw input unquoted, so the whitespace-only and
trailing-space inputs the forgiving lookup explicitly handles rendered
invisibly in CLI output and logs. Use {name!r} like the other two
resolver errors already do.
Coordinators and interactive agents had no way to enumerate valid
persona names: task_agent / spawn_workstream / spawn_batch described
`persona=` but nothing listed what it accepts, and resolution was an
exact case-sensitive slug match - users reaching for the display name
or a case variant got an unexplained failure.
- Inject the live persona catalog (enabled, interactive-kind; children
and sub-agents are always interactive) into the `persona` parameter
description of task_agent / spawn_workstream / spawn_batch, riding
the same render path as the model-alias injection. Rebuilt from the
pristine TOOLS base every render, so repeated renders are idempotent
and archived personas drop out instead of lingering. Entries carry
name + default marker + <=96-char description; names-only past 25
personas. spawn_batch's persona property is nested per-child under
children.items.properties (located via _persona_property, null-safe
against name-colliding MCP tools). Storage-less sessions keep the
base text: the render runs at session construction, so it gates on
is_storage_initialized() rather than get_storage(), which would
auto-init SQLite as a side effect.
- resolve_persona_for_kind - the ONE shared rule behind the HTTP
create handler, CLI --persona, the coordinator spawn precheck, and
task_agent prep - is now forgiving: exact slug, then the lowercased
input (created names are regex-enforced lowercase slugs), then a
case-insensitive display-name match accepted only when unique among
the kind's enabled personas. Duplicates refuse loudly naming the
candidate slugs; a same-label persona of another kind neither blocks
nor wins (the label the caller saw came from a kind-filtered
surface); whitespace-only input never matches blank display names
(display_name defaults to ""). Every failure now enumerates the
kind's valid names, so a stale injected list or a typo self-corrects
on the next attempt.
- The canonical slug is stamped everywhere: task_agent prep rewrites
its arg from the resolved snapshot, _validate_child_persona returns
(canonical, error) and both spawn call sites adopt it - approval
chrome, the wire, and workstream_config never carry a forgiven
variant.
- Create-persona shelf: label hint under Name explaining agents and
the CLI launch the persona by this name (case-insensitive) and the
display name is only a list label. docs/personas.md gains a "How
agents discover personas" section and drops the stale claim that
task_agent has no persona parameter.
Tests: resolver unit suite (case/display/ambiguity/cross-kind/
whitespace/disabled/storage-failure) + guards for injection content
and ordering, idempotent re-render, archive drop, the 25-persona
prose cutoff, coordinator-kind exclusion, and canonical stamping
through spawn_workstream / spawn_batch / task_agent.
- _coordinator_tenant_check and _coord_attachment_owner resolved storage from
the global registry (get_workstream_row / for_request without a storage arg),
which can evaluate the project-tenancy decision against a different or
auto-initialised backend and fail OPEN on a missing project row. Use
request.app.state.auth_storage explicitly, matching cluster_ws_detail and
_resolve_coordinator_or_404; fail closed (404) when it is unset.
- reject_unassignable_scopes now derives its allowed-scope error message from
ASSIGNABLE_SCOPES so validation and the message can't drift.
Workstreams attached to a private project were visible -- including their
conversation content -- to holders of admin.cluster.inspect / admin.coordinator
(both default builtin-admin permissions), defeating the project's confidentiality
boundary. Enforce that a private project's resources are visible only to people
IN the project (owner, workstream creator, or an explicit member), even for admins.
Surfaces closed:
- WorkstreamProjectVisibility bypass narrowed to service scope only (node->console
machine plumbing, re-filtered per-user at the console edge). No human principal
bypasses; admin.cluster.inspect gates the inspect surfaces, not tenancy. This
flows to /dashboard, session listings, the attachment row-gate, cluster_workstreams,
cluster_node_detail, and cluster_snapshot/SSE.
- cluster_ws_detail 404-masks a workstream in a private project the caller can't
see; cluster_ws_live_bulk routes such ids to the denied list (no private-project
oracle).
- Coordinator operator verbs (history/export/detail/send/approve/set_title/open/
children/tasks/attachments) now enforce project tenancy: _coordinator_tenant_check
on coord_endpoint_config, the gate in _resolve_coordinator_or_404 (children/tasks),
the tenant_check now run in make_open_handler before rehydrate, and a
project-visibility check in _coord_attachment_owner. admin.coordinator gates the
surface cluster-wide, but a non-member is 404-masked. The tenant-check mirrors the
manager-first + coordinator-kind ladder so kind-isolation is preserved.
- service scope is no longer user-assignable: admin_create_token and both
turnstone-admin CLI mint paths reject it via reject_unassignable_scopes, so an
admin.users holder cannot self-mint a service token and restore the bypass. Service
scope is minted only by ServiceTokenManager / the JWT secret.
- The events/global node proxy (service-elevated cross-tenant firehose) is gated on
admin.cluster.inspect so a plain authenticated user cannot reach it through the
console proxy.
Updates the OpenAPI description, the row-gate/tenancy-filter docstrings, and adds
tests for every surface (visibility predicate + cluster detail/bulk + coordinator
history/export/children/open/attachments + events/global proxy + scope-mint
rejection); inverts the tests that pinned the old admin-bypass contract.
* fix(mcp): route static transport lifecycles through per-server owner tasks
A crash-looping MCP server drove the mcp-loop thread to a sustained,
climbing 100%+ CPU spin. Root cause: anyio cancel scopes are
host-task-bound, and the static path entered the SDK's transport /
ClientSession task-group scopes from short-lived connect tasks (every
health tick is a new task since #768). Once such a scope was cancelled
after its host task had finished - by anyio's task_done when a
transport child died with the server, or by ClientSession.__aexit__
during a cross-task teardown - CancelScope._deliver_cancellation could
never make progress (task.cancel() on a done task is a no-op) and
re-armed itself via call_soon every loop iteration, forever: ~900k
callbacks/s per zombie scope, one more per flap cycle (verified against
anyio 4.14.1; no upstream fix exists as of that release).
Fix: each static server's transport + session cms are now entered,
parked, and exited by ONE long-lived owner task
(_static_transport_owner), so scopes always have a live host and always
exit in the task that entered them. Teardown follows a one-cancel close
protocol (signal the close event before the first await, graceful
grace, then at most ONE cancel - never a second, which would abandon a
scope exit mid-flight). Connect timeouts now cancel only the waiting
caller; connect failures are delivered through a readiness future;
unrequested owner death (server died under a live session) evicts the
session immediately via a done-callback instead of waiting for the next
liveness ping. A rate-limited, mcp-loop-scoped gc-walk backstop
(_maybe_disarm_orphaned_scopes) disarms any zombie minted by paths not
yet migrated (the oauth_user pool keeps the old cross-task-close shape;
follow-up).
Also fixed: BaseExceptionGroup (BaseException-derived, as raised by
anyio task groups wrapping a stray CancelledError, e.g. an
accept-then-RST server) escaped `except Exception` in _connect_all and
killed it before the health/sweep loops were created - silently
disabling all autonomous recovery. Handled there and in the
health/sweep/eviction loops and the reconnect/refresh callers.
Verified: a live SIGKILL-flap repro went from 130%+ CPU (climbing, one
armed scope per cycle) to 0.3% flat with zero armed scopes; the RST
repro now leaves both background loops alive (previously both silently
dead). New tests: owner-lifecycle + close-protocol units (incl. an
exactly-one-cancel pin), a _connect_all BaseExceptionGroup regression,
a discriminating disarm-sweep test, and a ~10s live SIGKILL-flap smoke
test (real FastMCP subprocess, skips on environment gaps) asserting
zero armed scopes, exactly one live owner, and a post-recovery tool
call. Full 8470-test suite green; ruff+mypy clean.
* fix(mcp): route pool transport lifecycles through per-entry owner tasks
Completes the owner-task migration started for the static path: the
oauth_user pool path had the same latent anyio cancel-scope exposure
(host-task-bound scopes entered by short-lived connect tasks; a scope
cancelled after its host finished re-delivers cancellation via
call_soon forever - the 100%-CPU zombie), previously covered only by
the disarm backstop.
Each (user, server) pool entry's transport + ClientSession cms are now
entered, parked, and exited by ONE long-lived owner task
(_pool_transport_owner). The caller keeps building client_kwargs (the
per-user bearer and, when an auth-capture carrier is active, the
httpx_client_factory response hook) so 401/WWW-Authenticate capture
semantics are unchanged. Teardown is the shared one-cancel close
protocol (_teardown_pool_entry: signal before first await, graceful
grace, at most ONE cancel), used by the connect stale-guard, idle/LRU
eviction, and shutdown (parallel signal-then-reap). Unrequested owner
death evicts the session but keeps the entry and its discovered
catalog, matching the existing evict-session-keep-entry semantics the
auth_401 retry relies on.
Discovery still runs in the connecting caller while the transport is
hosted by the owner, so a transport collapse mid-discovery (e.g. the
SDK tearing its task group down on an upstream 401) cancels the OWNER,
not the caller - a bare await on the response stream would hang until
the 30s phase timeout. _await_pool_discovery races each discovery
await against owner completion and converts owner death into a prompt
ConnectionError (the owner is never cancelled there; teardown owns its
lifecycle). Carrier-first failure classification preserves auth_401
semantics for captured 401s.
With no cross-task stack closes left, _safe_close_stack and
_safe_teardown_on_connect_failure are deleted (zero callers).
Tests: new tests/test_mcp_pool_owner.py pins the pool close protocol
(graceful event-before-await close, exactly-one-cancel escalation,
owner-death eviction retaining entry+catalog, caller-cancel-mid-connect
cm-exit guarantee, factory-present-iff-capture, and the
owner-death-during-discovery fast-fail). 1010 mcp tests and the full
8471-test suite green, including the historical cross-task-anyio
sentinel test_integration_pool_reuse_401_refresh_and_retry_succeeds;
ruff+mypy clean; zero destroyed-task warnings.
* fix(mcp): harden disarm-sweep loop guard and owner BaseException arm
Review follow-ups on the owner-task migration:
- _maybe_disarm_orphaned_scopes now enforces its mcp-loop requirement
instead of trusting callers: it returns without walking (and without
advancing the rate-limit clock) unless the currently running loop IS
self._loop. A suppressed close can fire before start() or after
shutdown(), where the walk would be wasted at best and a cross-thread
reach at worst.
- The transport owner's BaseException arm now re-raises non-Exception,
non-group escapees (KeyboardInterrupt, SystemExit) after delivering
them to the readiness future - failure delivery is the arm's job;
swallowing an interpreter-level exit was not.
* fix(mcp): extend owner-death discovery fast-fail to the static path
The static connect path had the same exposure the pool's discovery race
closed: discovery runs in the connecting caller while the transport is
hosted by the owner task, so a transport collapse mid-discovery cancels
the OWNER and the caller's bare await on the response stream hung until
the caller-side attempt timeout (~45s) instead of failing promptly.
_await_pool_discovery is renamed to _await_owner_discovery (it is now
path-neutral) and wired into _connect_one_locked's four discovery
awaits. The helper also converts a discovery future that completes
CANCELLED without the race's own reap (an SDK-internal cancellation
shape) into the same ConnectionError, instead of leaking a bare
CancelledError the caller would misread as its own cancellation.
The pool transport owner's BaseException arm gains the same refinement
the static owner received in review: interpreter-level exits
(KeyboardInterrupt, SystemExit) re-raise after delivery to the
readiness future instead of being swallowed.
Tests: static owner-death-during-discovery fast-fail (<1s vs the ~45s
hang), and a direct pin on the cancelled-discovery-future conversion.
* fix(mcp): replace owner BaseException arm with targeted catch + finally delivery
The owner's failure arm now catches only (BaseExceptionGroup, Exception);
waiter delivery for everything else moves to a finally that resolves the
readiness future with a clean transport-failure ConnectionError before
the task unwinds. Interpreter exits and BaseException-derived library
control-flow escapes propagate from the owner exactly once, uncaught -
and the waiter can never be left hanging on an unresolved future (the
initial _connect_all connect has no outer bound). For SystemExit /
KeyboardInterrupt asyncio additionally stops the loop right after, so
the delivery is load-bearing for the non-exit BaseException shapes and
free for the exits.
Pinned by a test driving a BaseException-derived escape through the
owner: the waiter resolves promptly with ConnectionError while the
escape propagates unswallowed.
* fix(mcp): mirror targeted-catch + finally delivery in the pool owner
Same shape the static owner received in review: the failure arm catches
only (BaseExceptionGroup, Exception), and waiter delivery for anything
else moves to a finally that resolves the readiness future with a clean
ConnectionError before the task unwinds - interpreter exits and
BaseException-derived library escapes propagate exactly once, uncaught,
and the waiter can never be left hanging.
* test(mcp): narrow the escape test's waiter catch to explicit types
* test(mcp): narrow discovery-race waiter catches to explicit types
* refactor(mcp): make reap/synchronization awaits explicit to analyzers
Full-absorb reaps (cancel-then-drain of a future whose outcome is
deliberately consumed) become `await asyncio.gather(x,
return_exceptions=True)` - one line, self-describing, and in the
owner-died discovery reap it is also a small semantic improvement: a
caller cancellation arriving during the reap now propagates instead of
being masked by the ConnectionError. Bare synchronization awaits and
selective suppress blocks in tests keep their raise-through semantics
via throwaway assignment. Applied uniformly across the owner-task
test files, including sites introduced by the static-path PR.
Patrick Buckley