Compare commits

...

21 Commits

Author SHA1 Message Date
Patrick Buckley a4c35e9e29 chore: bump version to 1.7.4 2026-07-11 19:13:24 -07:00
Patrick Buckley 862eb99cdb docs(changelog): add 1.7.4 release notes 2026-07-11 19:13:14 -07:00
Patrick Buckley 25b97bebdf fix(install): gate get.docker.com by $ID instead of trapping all failures
Deciding the installer up front — get.docker.com for the IDs it recognizes,
Docker's repo directly for unrecognized derivatives — avoids treating a
transient get.docker.com failure (network, apt lock, EOL sleep) on a supported
distro as an "unsupported distro" and silently routing it into the repo path.

Recognized IDs now surface the real failure via die instead of masking it;
unrecognized derivatives (Nobara, Mint, …) skip the doomed call and its
"Unsupported distribution" output entirely rather than running it to fail.

Addresses review feedback on #829.

(cherry picked from commit f4e54ce814)
2026-07-11 19:06:23 -07:00
Patrick Buckley ee5ca9a242 fix(install): install Docker on distros get.docker.com rejects
run.sh delegates Docker installation to get.docker.com, which detects the
distro from $ID alone and aborts with "Unsupported distribution '<id>'" on
any derivative it doesn't hardcode — Nobara (the reported case), Linux Mint,
Pop!_OS, AlmaLinux, Oracle Linux, and so on. run.sh's own detection already
resolves these via ID_LIKE/fallback, so the family is known; only the
delegated install fails.

When get.docker.com exits non-zero, fall back to adding Docker's official CE
repo for the upstream the family maps to and installing the same packages
(including the compose plugin the rest of run.sh depends on). Upstream is
chosen from PLATFORM_ID for the dnf family — Fedora is platform:fNN, Enterprise
Linux platform:elN, which ID_LIKE cannot distinguish (Nobara's is
"rhel centos fedora" yet it is pure Fedora) — and from UBUNTU_CODENAME for the
apt family, which is present only on Ubuntu lineage and is the exact codename
Docker's repo expects (Mint's VERSION_CODENAME is not).

Fixes #822.

(cherry picked from commit 4d423913b1)
2026-07-11 19:06:23 -07:00
Patrick Buckley dd8543fce9 fix(admin): add create-admin CLI; stop run.sh onboarding into a role-less user
The installer's "Finish setup" told users to run `turnstone-admin create-user`,
which creates a user with no role. Web login derives scopes solely from assigned
roles (empty perms -> read only), so that account logs in read-only and every
admin action fails with "Forbidden: token lacks 'approve' scope". Creating any
user also flips setup_required to false, so the browser first-run wizard -- the
only path that assigns the builtin-admin role -- never appears.

- run.sh: point "Finish setup" at the web setup wizard; use create-admin as the
  headless fallback instead of create-user
- admin.py: add `create-admin` -- creates a user + assigns builtin-admin, or
  promotes an existing role-less user (idempotent); guards on the seeded admin
  role and enforces the wizard's 8-char password floor for fresh accounts
- tests: cover fresh-grant (approve reaches the derived login scope), the
  promote/recovery path, idempotency, and both validation exits

Fixes #824

(cherry picked from commit a4876c00e2)
2026-07-11 19:06:23 -07:00
Patrick Buckley 667942024f fix(judge): thread model-definition capabilities into judge completions
The intent judge and output-guard judge were the only create_completion
callers that never passed model-definition capabilities, so operator-declared
capabilities (effort passthrough, tool support, temperature, verbosity) were
silently ignored on judge calls. Every in-ChatSession lane threads them via
_resolve_capabilities; the judges live outside the session and never reached
it.

Add a shared _resolve_model_capabilities() helper mirroring
ChatSession._resolve_capabilities, and have both judges resolve
self._capabilities — from the judge alias's model definition, or the injected
session capabilities on the session-model fallback — and pass capabilities=
into create_completion. Replace each judge's context_window int arg with
session_capabilities: the fallback window now derives from the resolved caps
(identical to what the session passed before), while the alias path keeps
reading ModelConfig.context_window, a separate field the capability merge must
not touch.

Refresh the stale docs/judge.md note claiming sub-agents are exempt from intent
validation — task agents have been judge-gated since #773.

Refs #823

(cherry picked from commit 6a94dc1d57)
2026-07-11 19:06:23 -07:00
Patrick Buckley 78831bbe91 docs(task-agent): record the decided durable-sub-turn id strategy at the mint site
If sub-turns ever persist: Turn-IR verbatim, re-mint at load (run_seq is
session-scoped), rebuild the wire map from the native lane's structural
1:1 pairing with the mirror; turns without native client tool blocks
need no entries. The map itself is never persisted — it is derivable,
and a second durable source of truth would have to be kept in lockstep
with the turns. Also documents why the mint must never be string-split
(not injective: parent and original may contain the delimiter).

(cherry picked from commit 8e2657248d)
2026-07-11 19:06:23 -07:00
Patrick Buckley d44d7eb1a8 fix(task-agent): guard the Google swap against partial lanes; fix a stale comment
The fidelity swap now requires the raw lane to be a faithful counterpart
of the mirror — same length, every id present — before replacing
tool_calls; a partially-corrupted lane (filtered non-dict elements)
would otherwise swap a shorter list over the mirror and orphan a
mirrored call whose tool result remains in history. The _run_agent
call-site comment now matches the builder's reasoning_text-only
blank-id rule.

(cherry picked from commit 660aff6f1e)
2026-07-11 19:06:23 -07:00
Patrick Buckley 876c7d8cb3 fix(task-agent): simplify the blank-id rule to reasoning_text-only and heal historical rows
The blank-id gate's strip-then-filter semantics left two residual
hazards (surviving Responses reasoning items whose pairing contract
needs their original sibling items; an asymmetric Messages-shaped lane
surviving when no client block was actually stripped). The rule is now
total and simpler: on a blank-id turn only the loose-text
reasoning_text synth block survives — it carries no id and is
shape-invalid on the Messages translator by design, and real-world
blank-id servers are Chat-Completions locals whose reasoning IS that
loose text. This also removes the builder's per-call provider import.

The Google fidelity swap now skips raw rows carrying a blank id
(historical captures that predate the gate would otherwise resurrect
the blank id on every replay — the sanitized mirror stays), guards
against non-dict lane elements, and legalizes via the new shared
lowering.legalize_tool_call_entry — the ONE per-entry legalizer the
sanitize pass also uses, so the two seats cannot drift on semantics or
the wire.tool_args_legalized breadcrumb.

(cherry picked from commit dc52bc2b96)
2026-07-11 19:06:23 -07:00
Patrick Buckley 98823eb769 fix(task-agent): move the blank-id gate into the shared native-lane builder
The blank-provider-id gate lived only at the _run_agent call site while
the main-loop stream accumulator has the identical back-fill-then-carry
seam — and it over-dropped, discarding the reasoning lane for exactly
the servers that emit blank ids. The gate now lives in
_finalize_provider_blocks as a had_blank_ids parameter both harnesses
thread: client tool blocks (which keep the blank id the mirror back-fill
never reached) are stripped, and when any were present the remaining
Messages-shaped blocks go with them (a surviving native lane REPLACES
the rebuilt content on the Anthropic translator, so a lane missing its
tool_use would orphan every mirrored call) — while shape-invalid
reasoning residuals (reasoning_text, Responses reasoning items) are
kept. This also closes the pre-existing main-loop case: a Gemini
openai-compat turn with a blank tool id no longer persists a raw
fidelity dict whose blank id the swap would resurrect on every replay.

The Google fidelity-swap legalization now reuses the canonical
lowering.legalized_arguments (made public) instead of a hand-rolled
narrower copy: dict-shaped arguments are serialized rather than
collapsed to {}, the standard wire.tool_args_legalized breadcrumb is
logged, and a degenerate non-dict function entry passes through
untouched instead of raising.

(cherry picked from commit 98cefc3660)
2026-07-11 19:06:23 -07:00
Patrick Buckley 4d708c30ac fix(task-agent): review fixes for the native-lane carry
- Skip the native lane on a turn whose provider left a tool-call id
  blank: the uuid back-fill reaches only the tool_calls mirror, so a
  carried native tool_use block would replay the blank id and desync
  from the restored tool_result (Anthropic orphans the result; Google
  re-fills a fresh uuid). The rebuild path keeps every representation
  on the back-filled id — the pre-native behaviour, for exactly the
  degenerate case.
- Extract _reasoning_text as the ONE Chat-Completions reasoning
  extractor shared by the streaming and non-streaming paths: first
  non-empty STRING of reasoning/reasoning_content wins, so a server
  putting a structured object in reasoning can neither shadow valid
  text in reasoning_content nor leak a non-str into the session's
  reasoning accumulator.
- Legalize arguments when GoogleProvider's fidelity swap replaces the
  sanitized tool_calls mirror with the raw provider dicts — the swap
  could resurrect a malformed arguments string the upstream sanitize
  pass had fixed (pre-existing on the main loop; ids and
  thought_signature untouched).
- Drop the redundant emptiness guard on the agent seam's
  reasoning_parts (the shared finalize helper already guards) and
  document the wire_id_map lifetime invariant for future
  resumable/background agents.

(cherry picked from commit 646bceed52)
2026-07-11 19:06:23 -07:00
Patrick Buckley 6d60ff7634 feat(task-agent): carry the provider-native reasoning lane in the sub-harness
A task agent's replayed turns now carry the native reasoning lane the
model produced (Anthropic thinking blocks + signatures, OpenAI Responses
reasoning items, Gemini thought_signature blocks, vLLM/llama.cpp parsed
reasoning text) instead of being rebuilt from content + tool_calls with
the reasoning dropped — restoring reasoning continuity across the
agent's own multi-turn tool loop on every provider lane.

The prerequisite is the id half: replace legalize_tool_call_ids with
restore_provider_tool_ids, a lowering pass that maps the session-minted
sub-tool ids back to the provider's own ids on the transient wire copy
(from the per-run mint map, never by string-splitting). The native
tool_use block is replayed verbatim — its id and signature untouched —
and the top-level mirror and tool_result agree with it on every request.
The minted id stays the sole internal key (registry, DOM, recall,
cancel ledger), #820 unchanged.

Chat-Completions lane: non-streaming create_completion now surfaces
reasoning/reasoning_content as CompletionResult.reasoning (the twin of
the streaming reasoning_delta extraction), and the agent seam runs the
Phase 5 vLLM reasoning-field replay against the agent's own provider
and alias. The native lane is finalized by a shared helper
(_finalize_provider_blocks) so the main loop and the sub-harness cannot
drift; replay honors the per-model replay_reasoning_to_model flag on
every lane, and llama.cpp stays capture-only, matching the main loop.

(cherry picked from commit d660819142)
2026-07-11 19:06:22 -07:00
Patrick Buckley be662c6134 fix(task-agent): address Copilot review on the id projection
- wire_safe_tool_call_id: SHA-256 not SHA-1 for the deterministic token —
  matches the codebase convention for fingerprints (attachments, auth,
  session) and drops the SHA-1 scanner flag. Non-crypto use, ids unchanged
  in shape (tid_ + 32 hex); no test pins the literal value.
- interactive.js: the two sub-agent child-id example comments now show the
  real minted shape (<parent>::r{run}s{step}::<id>), not a stale <seq> form.

(cherry picked from commit a5c3dc00fc)
2026-07-11 19:06:02 -07:00
Patrick Buckley 3ef3f24c7f fix(task-agent): mint session-unique sub-tool ids
Sub-agent tool ids were namespaced {parent}::{provider_id} — unique
across concurrent agents but not across turns within one agent. A local
provider reissuing "call_0" every response minted the same id twice, so
the live card's DOM row lookup collapsed distinct calls onto one row
while FIFO recall kept them apart: two views of one trajectory disagreed
on identical input (the bug-3 id-consistency defect). When the provider
also reuses the PARENT call id, sequential runs repeated the collision
one level up.

Mint {parent}::r{run}s{step}::{provider_id} at the single rewrite point:
a session-monotonic run tag (lock-allocated; runs start concurrently on
the 4-wide task pool) plus a per-run step tag make each id unique within
the session, and every consumer — nesting registry, error flags, DOM
data-call-id, recall projection, cancel ledger — keys on that one id.
The FIFO pairing helper stays as honest pairing for un-minted input
(unparented runs, direct construction), with its rationale rewritten.

The agent wire seam (_run_agent's _api_call) also runs the same two
validity passes the main loop already ran — sanitize_tool_call_arguments
(a documented vLLM deepseek_v4 renders malformed args and 400s; agents
hit the same backends) and legalize_tool_call_ids (projects the long,
::-containing ids to plain tokens, call/result pairing preserved). The
id projection is DEFENSIVE hardening, not a fix for an observed break:
the ids replay fine on the lenient anthropic-compatible deployment (the
prior ::-containing format ran reliably), it just keeps an agent's
self-built history valid on a hypothetically stricter backend. Applied
at the agent seam only — main-loop assistant turns carry a provider-
native block lane whose id must stay byte-identical to the mirrored
tool_calls, so the projection cannot run there without desyncing them.

Follow-ups: parent-level card aliasing under a reused parent id; the same
id hygiene for the main conversation loop / native lane.

(cherry picked from commit 110d6b4fc0)
2026-07-11 19:06:02 -07:00
Patrick Buckley db903f482a fix(console): scope response-control dirty flag to the identity that set it
A dirty flag set by touching a verbosity/reasoning-mode select survives a
model/provider/surface change, so the merge-side delete could destroy a key
hand-typed into the Advanced JSON for the renamed row. Honor the dirty
override only while the identity still matches the row that made it dirty.

Also document the captured-value fallback contract at both sites (the
baseline is deliberately not consulted: it arrives async or never on the
compat lane, capture has already lifted the value out of the row JSON, and
emission is gated server-side on the merged supports_* flag) and pin the
fallback plus the scoped dirty-delete in test_app_js.

(cherry picked from commit 062a260c88)
2026-07-11 19:05:48 -07:00
Patrick Buckley 6aeffd1845 feat(console): verbosity and reasoning-mode controls in the model shelf
- capability-gated "Response controls" on the Models create/edit
  shelf: Output verbosity (low/medium/high) and Reasoning mode
  (Standard/Pro), shown only for Responses-surface models; the empty
  selection means provider default and omits the capability key
- values lift out of the capabilities JSON into the selects on edit
  and merge back on save with identity tracking, so changing the
  provider/model/surface resets them instead of carrying a value
  across models; the Advanced JSON textarea wins unless the select
  was touched last
- known GPT-5.6 models inherit support from the static table without
  persisting redundant support flags; OpenAI-compatible models pinned
  to the Responses surface opt in via the supports_verbosity /
  supports_pro_mode tiles
- invalidate in-flight capability lookups on any identity field
  change and on modal open so a stale response cannot clobber a fresh
  shelf; API-surface changes now run the full field-change path
- model list rows surface verbosity= / mode= override chips

(cherry picked from commit 03861e0cf5)
2026-07-11 19:05:48 -07:00
Patrick Buckley a02b093733 fix(providers): align GPT-5.6 with the GA API surface
- every 5.6 tier accepts effort "max" and reasoning.mode
  "standard"/"pro" (GA docs: pro is a request mode on any GPT-5.6
  model) -- drop the Sol-only gating
- GPT-5.6 deprecates prompt_cache_retention; send
  prompt_cache_options={"ttl": "30m"} (its only supported lifetime)
  and keep the 24h retention policy for pre-5.6 models
- never inject commercial cache params into local lanes: dropped from
  the Chat Completions lane (which serves only openai-compatible and
  google) and gated off the compat-pinned Responses lane -- a gpt-5*
  served-model name is not an OpenAI account
- account cache writes: usage *_tokens_details.cache_write_tokens
  flows into cache_creation_tokens (5.6 bills writes at 1.25x the
  uncached input rate)
- drop non-string verbosity/reasoning_mode overrides with a warning
  instead of raising on unhashable capability-JSON values
- keep ModelCapabilities' public positional prefix stable by appending
  the verbosity/pro fields at the tail; pin it with a constructor test
- openai floor 2.44 -> 2.45, the first release with the typed
  prompt_cache_options kwarg

(cherry picked from commit b450b9ad20)
2026-07-11 19:05:48 -07:00
Patrick Buckley f311555026 fix(bash): report exit code for killed shells; single import style in registry tests
Copilot: bash_output's schema promises the exit code once the shell has
exited, but the formatting attached it only to 'completed' — a killed
shell has one too (the negated signal number). Attach it to any exited
state.

Code-quality: the registry test file mixed a top-level from-import with
function-local 'import ... as bg_mod' for monkeypatching module
attributes; one from-style module alias at the top now serves all of
them.

(cherry picked from commit dc647f4d63)
2026-07-11 19:05:48 -07:00
Patrick Buckley 45d95a2c1f feat(bash): opt-in background shells with delta output reader and kill tool (#817)
Restore 'start a dev server, use it in a later call' as an explicit opt-in
after #816 made bash reap its whole process group on return. The surface
mirrors the dominant coding-agent convention: bash(run_in_background=true)
returns a bash_N handle immediately; bash_output(id, filter?) returns only
output produced since the previous read plus status and exit code;
kill_shell(id) terminates the shell's whole process group.

- Per-session BackgroundShellRegistry: capped rolling line buffer with
  drop-oldest gap accounting, exit-order record pruning, owner scoping for
  task_agents (shells reaped when the agent finishes), liveness-guarded
  group kills (a stale pgid is never signalled), budgeted teardown joins.
- Exit notices ride a shared external-event rail (sanitize, soft cap,
  channel 'any', idle wake) now common to watch fires; a new 'quiet'
  NudgeQueue channel lets a user cancel defer pending notices without
  letting them re-wake the stopped workstream, and failed wake delivery
  re-queues external notices seq- and predicate-intact without re-arming
  the wake gate.
- The bash_output filter runs in a killable subprocess: sre holds the GIL
  for an entire search, so no in-process timeout can bound a hostile
  pattern. Scrubbed child env, pinned UTF-8 pipes, honest timeout-vs-
  helper-failure error taxonomy, per-line match window with explicit
  clipping notes; a failed filter never consumes the delta.
- run_in_background rides the bash intent-judge projection; bash_output is
  exempt from the repeat warning but still recorded so interleaved polls
  keep breaking other tools' streaks; all bash boolean args share one
  lenient coercion dialect.
- Shells survive generation cancel and die with the workstream: every
  teardown path funnels through ChatSession.close(); CLI exit and the
  server lifespan now close every loaded session, signal-first and
  Ctrl-C-safe, so nothing detached outlives a graceful shutdown.

(cherry picked from commit ab7d56e0ba)
2026-07-11 19:05:36 -07:00
Patrick Buckley a2d9d9832a test(bash): use tmp_path fixture instead of tempfile.mktemp
CodeQL flagged tempfile.mktemp as an insecure temporary file and Copilot flagged the same call as race-prone (the path is not reserved). Use the pytest tmp_path fixture, which reserves a unique per-test directory and is cleaned up automatically.

(cherry picked from commit bec757a96b)
2026-07-11 19:05:22 -07:00
Patrick Buckley ab123c6cfc fix(bash): do not hang when a command backgrounds a long-lived process
A bash command that leaves a process running in the background (server &, a daemon) could wedge the whole workstream forever: the tool read stdout/stderr to EOF, which never arrives because the child inherits the pipe, and the timeout watchdog bailed the moment the tracked bash exited.

Wait on the tracked process bounded by the tool timeout (keyed on process exit, not pipe EOF) and terminate its whole session group on every exit path, reaping any backgrounded survivor, forcing the drain threads to EOF, and leaving nothing to leak. Decode with errors=replace so undecodable output is preserved instead of dropped, and pre-bind proc so a Popen failure surfaces the real error.

Behavior change: a process the command backgrounds no longer survives the call. First-class opt-in backgrounding is left as a separate change.

(cherry picked from commit f1f488aa55)
2026-07-11 19:05:22 -07:00
54 changed files with 6321 additions and 377 deletions
+46
View File
@@ -14,6 +14,52 @@ experimental line:
Earlier stable lines (`stable/1.6`, `stable/1.5`) are frozen.
## [1.7.4]
A feature-bearing patch for the 1.7 line, rolling up work that had stabilised
on `main`. No schema migrations (head stays 066) and no new configuration knobs.
### Added
- **Background shells for the `bash` tool** — `run_in_background=true` starts a
command as a detached shell and returns a `bash_N` handle; new `bash_output`
(delta output since last read, optional regex filter, status/exit code) and
`kill_shell` (terminates the shell's process group) tools manage it. Output is
buffered with a drop-oldest cap, a system notice lands when a shell exits, and
shells die with their workstream — never outliving a `task_agent` that started
them.
- **`task_agent` carries the model's native reasoning across its own tool loop** —
a task agent's replayed turns now preserve the provider-native reasoning lane
(Anthropic thinking blocks with signatures, OpenAI reasoning items, Gemini
`thought_signature`, vLLM/llama.cpp reasoning text) instead of rebuilding each
turn from text alone, restoring reasoning continuity for thinking models.
- **Model-shelf response controls** — the console model shelf exposes verbosity
and reasoning-mode controls per identity.
### Changed
- **GPT-5.6 aligned with the GA API surface** — the Responses provider matches
GPT-5.6's GA shape (typed `reasoning.mode`, `prompt_cache_options`,
cache-write accounting); the `openai` floor moves to `>=2.45`.
### Fixed
- **`bash` never hangs on a backgrounded child** — a command that left a
long-lived process running no longer wedges the workstream; the tool waits on
the tracked process (bounded by the timeout) and reaps its whole process group.
- **`task_agent` sub-tool ids are session-unique** — ids are minted
`{parent}::r{run}s{step}::{id}` so a local model reissuing sequential ids
(`call_0` each turn) no longer aliases two steps onto one live-card row while
`/history` keeps them apart.
- **Judge completions honour model-definition capabilities** — a judge's
completion now threads its model's declared capabilities instead of assuming a
default surface.
- **`create-admin` CLI** — adds an explicit admin-creation command; `run.sh` no
longer onboards into a role-less user.
- **Install script Docker handling** — installs Docker on distros
`get.docker.com` rejects, and gates that path by `$ID` instead of trapping all
failures.
## [1.7.3]
A small feature and maintenance patch for the 1.7 line. No schema migrations
+1 -1
View File
@@ -458,7 +458,7 @@ Each item in `items` (shared by `tool_info` and `approve_request`):
| `context_window` | int | Total context window size in tokens |
| `pct` | float | Percentage of context window used |
| `effort` | string | Reasoning effort level (`low`/`medium`/`high`) |
| `cache_creation_tokens` | int | Tokens written to prompt cache (Anthropic) |
| `cache_creation_tokens` | int | Tokens written to prompt cache (Anthropic + OpenAI) |
| `cache_read_tokens` | int | Tokens served from prompt cache (Anthropic + OpenAI) |
**`info`** -- an informational message (e.g. command output).
+9 -8
View File
@@ -622,19 +622,19 @@ LLMProvider (protocol)
|------|--------|
| `StreamChunk` | `content_delta`, `reasoning_delta`, `tool_call_deltas`, `info_delta`, `usage`, `finish_reason`, `provider_blocks` |
| `CompletionResult` | `content`, `tool_calls`, `finish_reason`, `usage`, `provider_blocks` |
| `ModelCapabilities` | `context_window`, `max_output_tokens`, `supports_temperature`, `token_param`, `thinking_mode`, `supports_effort`, `supports_web_search`, `supports_tool_search`, `supports_vision`, `supports_reasoning_replay` |
| `ModelCapabilities` | `context_window`, `max_output_tokens`, `supports_temperature`, `token_param`, `thinking_mode`, `supports_effort`, `supports_web_search`, `supports_tool_search`, `supports_vision`, `supports_reasoning_replay`, `supports_verbosity`, `verbosity`, `supports_pro_mode`, `reasoning_mode` |
| `UsageInfo` | `prompt_tokens`, `completion_tokens`, `total_tokens`, `cache_creation_tokens`, `cache_read_tokens` |
**OpenAIProvider** (`_openai.py`): passes messages through unchanged (they are
already in OpenAI format), including multi-part content blocks (text + images)
in tool results. Model capability lookup table covers GPT-5/5.1/5.2/5.3/5.4,
in tool results. Model capability lookup covers GPT-5 through GPT-5.6,
O-series, and search models (`gpt-5-search-api`) — all with `supports_vision`.
For search models, injects `web_search_options` and removes the `web_search`
function tool (the model always searches). Citations from `url_citation`
annotations are formatted as footnotes. Extended prompt cache retention
(`prompt_cache_retention: "24h"`) is enabled for GPT-5.x models at no
additional cost. Cached token counts are extracted from
`usage.prompt_tokens_details.cached_tokens`. Unknown models get permissive
annotations are formatted as footnotes. Pre-5.6 GPT-5 models request extended
prompt-cache retention (`prompt_cache_retention: "24h"`); GPT-5.6 uses
`prompt_cache_options.ttl: "30m"`. Cache reads and writes are extracted from
`cached_tokens` and `cache_write_tokens`. Unknown models get permissive
defaults with `supports_vision=False` and use SearxNG for web search. The
`openai-compatible` lane never consults this table at all — on either API
surface (the responses pin is served by a compat-mode
@@ -642,8 +642,9 @@ surface (the responses pin is served by a compat-mode
local server serves whatever the operator named it (vLLM
`--served-model-name` is a free string), so a prefix collision with a cloud
model id must not inherit that model's sampling/effort contract — every
local model gets the plain defaults, and anything beyond them is declared on
the model definition (capabilities JSON + `server_compat`), matching the
local model gets the plain defaults, commercial prompt-cache controls are not
injected by model-name prefix, and anything beyond those defaults is declared
on the model definition (capabilities JSON + `server_compat`), matching the
`anthropic-compatible` lane.
**AnthropicProvider** (`_anthropic.py`): converts OpenAI-format messages to
+6 -3
View File
@@ -131,9 +131,12 @@ Per-LLM-request token and tool call metrics:
LLM response with prompt/completion tokens, cache tokens, tool call count,
model, ws_id
- **Prompt caching**: Anthropic automatic caching (`cache_control: ephemeral`)
and OpenAI extended retention (`prompt_cache_retention: 24h` for GPT-5.x)
are enabled by default. `cache_creation_tokens` and `cache_read_tokens` are
tracked per request in `usage_events` and surfaced in the Usage admin tab
and OpenAI caching are enabled by default. Pre-5.6 GPT-5 models request
`prompt_cache_retention: 24h`; GPT-5.6 uses
`prompt_cache_options: {"ttl": "30m"}`. GPT-5.6 cache writes use the
provider's 1.25× input-token rate. `cache_creation_tokens` and
`cache_read_tokens` are tracked per request in `usage_events` and surfaced
in the Usage admin tab
- **Querying**: `GET /v1/api/admin/usage` with `group_by` (day/hour/model/user)
and time range filtering — includes cache token aggregates
- **Prometheus**: `turnstone_tokens_total{type="cache_creation|cache_read"}`
+8 -2
View File
@@ -249,8 +249,14 @@ are withheld from the live surfaces (a reused call_id must never ride a stale
`approve` into Smart Approvals) but still persist with
`user_decision = "superseded"` so the audit trail records the judge's answer.
Sub-agents (plan agent, task agent) are exempt from intent validation -- they
always get full tool visibility without judge evaluation.
Sub-agent (task agent) tool calls are judge-gated too. Each runs the same
intent pipeline as its own `agent_gate` generation, grounded in that sub-agent's
own trajectory -- its task prompt is the delegation contract the operator
approved, so "does this call serve the task" is the right local question.
Agent-gate generations never occupy the main loop's supersede slot (parallel
siblings would otherwise make each other's verdicts look stale); per-cycle
generation checks enforce staleness instead, and `judge.cancel_on_approval`
fires per gate exactly like the main loop.
---
+17
View File
@@ -54,6 +54,23 @@ When a per-model override is `NULL` (empty in the UI), the global default is
used. Switching models via `/model <alias>` re-resolves sampling parameters
from the new model's overrides or global defaults.
### Responses output controls (per-model)
Models whose capability table declares Responses output controls expose two
additional fields in the Models create/edit shelf:
| Field | Stored capability | Values | Effect |
|-------|-------------------|--------|--------|
| Output verbosity | `verbosity` | `low`, `medium`, `high` | Controls answer length independently of reasoning effort. |
| Reasoning mode | `reasoning_mode` | `standard`, `pro` | Selects standard or higher-compute Pro execution without changing the model ID. |
An empty selection means provider default and omits the capability key. Known
GPT-5.6 models inherit support from the built-in table without persisting
redundant support flags. An OpenAI-compatible model pinned to the Responses API
can opt in with the `supports_verbosity` and `supports_pro_mode` capability
tiles. Chat Completions and non-Responses providers do not surface or submit
these controls.
**Removed settings:** `model.name` and `model.context_window` have been removed
from ConfigStore. Model names and context windows are now configured per-model
in the Models tab. A startup warning is logged if these keys appear in
+2 -2
View File
@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
[project]
name = "turnstone"
version = "1.7.3"
version = "1.7.4"
description = "Multi-node AI orchestration platform with tool use, agent routing, and cluster simulation."
readme = "README.md"
license = "Apache-2.0"
@@ -23,7 +23,7 @@ classifiers = [
"Topic :: Scientific/Engineering :: Artificial Intelligence",
]
dependencies = [
"openai>=2.44", # GPT-5.6 (Sol/Terra/Luna): Responses reasoning.mode + effort "max" + text.verbosity
"openai>=2.45", # GPT-5.6: typed reasoning.mode, prompt_cache_options, and cache_write_tokens
"anthropic>=0.108", # claude-fable-5 support; hard runtime floor is 0.105 (mid-conversation system blocks)
"httpx>=0.28",
"mcp>=1.27,<2", # v2 is a breaking rewrite (2.0.0a1 live 2026-06-11; stable ~2026-07-27) — streamablehttp_client removed, 2-tuple transport, snake_case types; migrate deliberately
+89 -7
View File
@@ -4,8 +4,9 @@
#
# curl -fsSL https://raw.githubusercontent.com/turnstonelabs/turnstone/main/run.sh | bash
#
# Autodetects your distro (Ubuntu/Debian, Fedora/RHEL, Arch, and WSL on any of
# them) and:
# Autodetects your distro Ubuntu/Debian, Fedora/RHEL, Arch, their common
# derivatives (Mint, Pop!_OS, Nobara, AlmaLinux, …), and WSL on any of them —
# and:
# 1. ensures git is installed, then clones the repo
# 2. ensures Docker + the compose plugin are installed and the daemon is usable
# 3. asks how many server nodes to run (1-10)
@@ -65,12 +66,18 @@ ask() {
# -- distro / package manager detection --------------------------------------
OS_ID=""; OS_LIKE=""; PKG=""; IS_WSL=0; SUDO=""
# Extra os-release fields, captured only to pick Docker's upstream repo when
# get.docker.com refuses a derivative it doesn't recognize (see install_docker).
OS_PLATFORM_ID=""; OS_CODENAME=""; OS_UBUNTU_CODENAME=""
detect_os() {
if [ -r /etc/os-release ]; then
# shellcheck disable=SC1091
. /etc/os-release
OS_ID="${ID:-}"; OS_LIKE="${ID_LIKE:-}"
OS_PLATFORM_ID="${PLATFORM_ID:-}"
OS_CODENAME="${VERSION_CODENAME:-}"
OS_UBUNTU_CODENAME="${UBUNTU_CODENAME:-}"
fi
if grep -qiE 'microsoft|wsl' /proc/version 2>/dev/null || [ -n "${WSL_DISTRO_NAME:-}" ]; then
IS_WSL=1
@@ -130,11 +137,83 @@ clone_repo() {
# -- docker -------------------------------------------------------------------
DOCKER="docker"
# Fallback when get.docker.com won't install here. That script keys off $ID alone
# (never ID_LIKE), so it aborts with "Unsupported distribution '<id>'" on every
# derivative — Nobara, Linux Mint, Pop!_OS, AlmaLinux, Oracle Linux, … — even
# though the family is clear. We already know the family from detect_os, so we add
# Docker's official CE repo for the matching upstream and install the same
# packages get.docker.com would (including the compose plugin the rest of run.sh
# relies on).
install_docker_ce_repo() {
local up
case "$PKG" in
apt)
local codename arch
# UBUNTU_CODENAME is set by Ubuntu and every Ubuntu-derived distro
# (Mint/Pop!_OS/Zorin/…) and never by pure Debian, so it both routes
# the family and gives the exact codename Docker's repo expects.
if [ -n "$OS_UBUNTU_CODENAME" ]; then
up=ubuntu; codename="$OS_UBUNTU_CODENAME"
else
up=debian; codename="$OS_CODENAME"
fi
[ -n "$codename" ] || die "couldn't determine the $up release codename for Docker's repo — install Docker manually and re-run."
arch="$(dpkg --print-architecture 2>/dev/null || echo amd64)"
info "Adding Docker's $up repository ($codename)."
$SUDO install -m 0755 -d /etc/apt/keyrings
curl -fsSL "https://download.docker.com/linux/$up/gpg" | $SUDO tee /etc/apt/keyrings/docker.asc >/dev/null
$SUDO chmod a+r /etc/apt/keyrings/docker.asc
printf 'deb [arch=%s signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/%s %s stable\n' \
"$arch" "$up" "$codename" | $SUDO tee /etc/apt/sources.list.d/docker.list >/dev/null
$SUDO apt-get update -y
$SUDO apt-get install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
;;
dnf|yum)
# A Fedora spin and a RHEL clone can both carry "fedora" in ID_LIKE
# (Nobara's is "rhel centos fedora"), so ID_LIKE can't separate them.
# PLATFORM_ID can: Fedora is platform:fNN, Enterprise Linux platform:elN.
case "$OS_PLATFORM_ID" in
platform:f*) up=fedora ;;
platform:el*) up=centos ;;
*) if [ -e /etc/fedora-release ]; then up=fedora; else up=centos; fi ;;
esac
info "Adding Docker's $up repository."
$SUDO curl -fsSL "https://download.docker.com/linux/$up/docker-ce.repo" \
-o /etc/yum.repos.d/docker-ce.repo \
|| die "couldn't add Docker's $up repository — install Docker manually and re-run."
pkg_install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
;;
esac
}
# The distro IDs get.docker.com installs directly: it matches $ID against this
# exact set (ignoring ID_LIKE) and aborts on anything else. Mirrors the dispatch
# in get.docker.com, including its fedora-asahi-remix -> fedora alias.
get_docker_com_supports() {
case "$1" in
ubuntu|debian|raspbian|centos|fedora|rhel|rocky|sles|fedora-asahi-remix) return 0 ;;
*) return 1 ;;
esac
}
install_docker() {
case "$PKG" in
apt|dnf|yum)
info "Installing Docker via the official get.docker.com script"
curl -fsSL https://get.docker.com | $SUDO sh ;;
# Decide up front which installer applies, rather than treating every
# get.docker.com failure as "unsupported distro": for an ID it knows,
# let it run and surface any real failure (network, apt lock, EOL) via
# die instead of masking it with the repo path. Only unrecognized
# derivatives (Nobara, Mint, …) — which it would just abort on — skip
# straight to adding Docker's repo ourselves.
if [ -n "$OS_ID" ] && ! get_docker_com_supports "$OS_ID"; then
info "get.docker.com doesn't support '$OS_ID' — using Docker's official repository directly."
install_docker_ce_repo
else
info "Installing Docker via the official get.docker.com script"
curl -fsSL https://get.docker.com | $SUDO sh \
|| die "get.docker.com failed to install Docker (see the output above). Fix the issue and re-run — the script resumes."
fi
;;
pacman)
pkg_install docker docker-compose ;;
esac
@@ -366,12 +445,15 @@ ${GREEN}${BOLD}Turnstone is running${RESET} (${NODE_COUNT} node$([ "$NODE_COUNT"
${DIM}cd $INSTALL_DIR && $DOCKER compose exec caddy cat /data/caddy/pki/authorities/local/root.crt${RESET}
Finish setup
1. Create the first admin user:
${DIM}cd $INSTALL_DIR && $DOCKER compose exec node-1 turnstone-admin create-user --username admin --name "Admin"${RESET}
2. Open ${url}, log in, and add a model backend in the ${BOLD}Models${RESET} tab —
1. Open ${BOLD}${url}${RESET} and create the admin account when prompted —
the first user created there gets full admin access.
2. Log in, then add a model backend in the ${BOLD}Models${RESET} tab —
a local server (vLLM / llama.cpp) or an OpenAI / Anthropic / Gemini key.
Nodes boot without a model and pick it up live; no restart needed.
${DIM}No browser? Create the admin from the CLI instead:
cd $INSTALL_DIR && $DOCKER compose exec node-1 turnstone-admin create-admin --username admin --name "Admin"${RESET}
Scale Running ${scale}
Manage ${DIM}cd $INSTALL_DIR${RESET}
+43
View File
@@ -0,0 +1,43 @@
"""Shared process/polling helpers for the bash + background-shell suites.
One copy instead of three: ``test_bash_tool_background_hang``,
``test_background_shells`` and ``test_bash_background_tool`` all assert on
process liveness and poll for asynchronous state. Leading underscore so
pytest doesn't collect it.
"""
from __future__ import annotations
import contextlib
import os
import signal
import time
def pid_alive(pid: int) -> bool:
try:
os.kill(pid, 0)
except ProcessLookupError:
return False
except PermissionError:
return True
return True
def kill_pid(pid: int) -> None:
with contextlib.suppress(OSError):
os.kill(pid, signal.SIGKILL)
def poll_until(predicate, timeout=10.0, interval=0.05):
"""Poll ``predicate`` until truthy or ``timeout``; RETURNS the last value
(falsy on timeout assert at the call site). Deliberately named apart
from ``tests/_helpers.wait_until``, which RAISES on timeout: two
same-named helpers with opposite failure semantics invite silently-green
tests."""
deadline = time.monotonic() + timeout
value = predicate()
while not value and time.monotonic() < deadline:
time.sleep(interval)
value = predicate()
return value
@@ -21,7 +21,9 @@
],
"max_output_tokens": 4096,
"model": "gpt-5.6-sol",
"prompt_cache_retention": "24h",
"prompt_cache_options": {
"ttl": "30m"
},
"reasoning": {
"effort": "max"
},
@@ -27,7 +27,9 @@
],
"max_output_tokens": 4096,
"model": "gpt-5.6-sol",
"prompt_cache_retention": "24h",
"prompt_cache_options": {
"ttl": "30m"
},
"reasoning": {
"effort": "max"
},
@@ -21,7 +21,9 @@
],
"max_output_tokens": 4096,
"model": "gpt-5.6-sol",
"prompt_cache_retention": "24h",
"prompt_cache_options": {
"ttl": "30m"
},
"reasoning": {
"effort": "high",
"mode": "pro"
+170
View File
@@ -0,0 +1,170 @@
"""Tests for ``turnstone-admin create-admin`` (issue #824).
``create-user`` creates a role-less user; the web UI derives a login's scopes
purely from assigned roles, so that account logs in read-only and hits
"Forbidden: token lacks 'approve' scope" on any admin action. ``create-admin``
assigns the built-in admin role mirroring the web setup wizard
(``POST /api/auth/setup``) and promotes an existing role-less user, which is
the recovery path for anyone already stuck.
Each test drives the real ``_cmd_create_admin`` handler against a real,
fully-migrated SQLite DB: the ``builtin-admin`` role is seeded by migration
008, so the DB must be migrated (not just ``create_all``-built) for the role
to exist.
"""
from __future__ import annotations
import argparse
from typing import TYPE_CHECKING, Any
import pytest
from turnstone.admin import _cmd_create_admin, _cmd_create_user
from turnstone.core.auth import _load_user_permissions, _permissions_to_scopes
from turnstone.core.storage import init_storage, reset_storage
if TYPE_CHECKING:
from collections.abc import Iterator
from pathlib import Path
@pytest.fixture(autouse=True)
def _reset_storage_singleton() -> Iterator[None]:
"""Keep the module-global storage singleton from leaking across tests."""
reset_storage()
yield
reset_storage()
def _db_args(db_path: str, **overrides: Any) -> argparse.Namespace:
"""Build the Namespace ``_cmd_create_admin`` (and ``_cmd_create_user``) expect.
Pins every DB field so ``_get_storage`` resolves to the tmp sqlite file and
never leaks a ``TURNSTONE_DB_*`` env var (it only falls back when the attr
``is None``). ``token``/``scopes`` are only read by ``_cmd_create_user``.
"""
base: dict[str, Any] = {
"username": "admin",
"name": "",
"password": "",
"token": False,
"scopes": "read,write,approve",
"db_backend": "sqlite",
"db_path": db_path,
"db_url": "",
"db_pool_size": 2,
"db_sslmode": "",
"db_sslrootcert": "",
"db_sslcert": "",
"db_sslkey": "",
}
base.update(overrides)
return argparse.Namespace(**base)
def _migrated_storage(db_path: str) -> Any:
"""Return a fully-migrated storage singleton (seeds the ``builtin-admin`` role)."""
return init_storage("sqlite", path=db_path, run_migrations=True)
def _has_admin_role(storage: Any, user_id: str) -> bool:
return any(r.get("role_id") == "builtin-admin" for r in storage.list_user_roles(user_id))
def _login_scopes(storage: Any, user_id: str) -> frozenset[str]:
"""Scopes a password login would grant this user — the real lockout surface."""
return _permissions_to_scopes(_load_user_permissions(storage, user_id))
def test_create_admin_fresh_user_gets_approve_scope(tmp_path: Path) -> None:
db_path = str(tmp_path / "admin.db")
storage = _migrated_storage(db_path)
_cmd_create_admin(_db_args(db_path, username="admin", name="Admin", password="hunter2!pw"))
user = storage.get_user_by_username("admin")
assert user is not None
assert _has_admin_role(storage, user["user_id"])
# The exact bug surface: a web login for this account must carry `approve`.
assert "approve" in _login_scopes(storage, user["user_id"])
def test_create_admin_defaults_display_name_to_username(tmp_path: Path) -> None:
db_path = str(tmp_path / "admin.db")
storage = _migrated_storage(db_path)
_cmd_create_admin(_db_args(db_path, username="root", name="", password="hunter2!pw"))
user = storage.get_user_by_username("root")
assert user is not None
assert user["display_name"] == "root"
def test_create_admin_promotes_existing_read_only_user(
tmp_path: Path, capsys: pytest.CaptureFixture[str]
) -> None:
"""Issue #824 recovery path: a role-less create-user account, then create-admin."""
db_path = str(tmp_path / "admin.db")
storage = _migrated_storage(db_path)
# Reproduce the locked-out account exactly (role-less create-user).
_cmd_create_user(_db_args(db_path, username="admin", name="Admin", password="hunter2!pw"))
user = storage.get_user_by_username("admin")
assert user is not None
assert not _has_admin_role(storage, user["user_id"])
assert "approve" not in _login_scopes(storage, user["user_id"]) # locked out
# Unstick without recreating the user.
_cmd_create_admin(_db_args(db_path, username="admin"))
assert _has_admin_role(storage, user["user_id"])
assert "approve" in _login_scopes(storage, user["user_id"])
assert "Granted the admin role" in capsys.readouterr().out
def test_create_admin_already_admin_is_idempotent(
tmp_path: Path, capsys: pytest.CaptureFixture[str]
) -> None:
db_path = str(tmp_path / "admin.db")
storage = _migrated_storage(db_path)
_cmd_create_admin(_db_args(db_path, username="admin", name="Admin", password="hunter2!pw"))
capsys.readouterr() # drop first-run output
_cmd_create_admin(_db_args(db_path, username="admin"))
user = storage.get_user_by_username("admin")
assert user is not None
admin_rows = [
r for r in storage.list_user_roles(user["user_id"]) if r.get("role_id") == "builtin-admin"
]
assert len(admin_rows) == 1 # not duplicated
assert "already an admin" in capsys.readouterr().out
def test_create_admin_short_password_rejected(
tmp_path: Path, capsys: pytest.CaptureFixture[str]
) -> None:
db_path = str(tmp_path / "admin.db")
storage = _migrated_storage(db_path)
with pytest.raises(SystemExit) as exc_info:
_cmd_create_admin(_db_args(db_path, username="admin", name="Admin", password="short"))
assert exc_info.value.code == 1
assert "at least 8" in capsys.readouterr().err
assert storage.get_user_by_username("admin") is None # nothing created
def test_create_admin_invalid_username_rejected(
tmp_path: Path, capsys: pytest.CaptureFixture[str]
) -> None:
db_path = str(tmp_path / "admin.db")
_migrated_storage(db_path)
with pytest.raises(SystemExit) as exc_info:
_cmd_create_admin(_db_args(db_path, username="bad user!", name="X", password="hunter2!pw"))
assert exc_info.value.code == 1
assert "invalid username" in capsys.readouterr().err
+76
View File
@@ -677,6 +677,82 @@ def test_audio_roles_gated_to_openai_sdk_providers() -> None:
assert '_providerCarriesAudio((md && md.provider) || "openai")' in body
def test_model_response_controls_are_capability_driven_and_sparse() -> None:
"""The model shelf surfaces Responses-only scalar controls without
hard-coding GPT-5.6 IDs or pinning inherited capability-table values."""
html = _CONSOLE_INDEX.read_text(encoding="utf-8")
admin = _CONSOLE_ADMIN_JS.read_text(encoding="utf-8")
assert 'id="model-response-controls"' in html
assert 'aria-labelledby="model-response-controls-title"' in html
assert 'id="model-output-verbosity"' in html
assert 'for="model-output-verbosity"' in html
assert 'id="model-reasoning-mode"' in html
assert 'for="model-reasoning-mode"' in html
for value in ("low", "medium", "high"):
assert f'<option value="{value}">' in html
for value in ("standard", "pro"):
assert f'<option value="{value}">' in html
assert 'data-cap="supports_verbosity"' in html
assert 'data-cap="supports_pro_mode"' in html
assert '"supports_verbosity"' in admin
assert '"supports_pro_mode"' in admin
surface = _slice_function_body(admin, "_modelUsesResponsesSurface")
assert surface is not None
assert 'provider === "openai"' in surface
assert 'provider === "openai-compatible"' in surface
assert 'value === "responses"' in surface
visibility = _slice_function_body(admin, "_updateModelResponseControls")
assert visibility is not None
assert "_modelGetTile(spec.supportKey)" in visibility
assert 'supportKey: "supports_verbosity"' in admin
assert 'supportKey: "supports_pro_mode"' in admin
assert "gpt-5.6" not in visibility, "visibility must come from capabilities, not model IDs"
assert "function _captureModelResponseControls(" in admin
assert "function _mergeModelResponseControls(" in admin
assert "_captureModelResponseControls(capsObj)" in admin
assert "_mergeModelResponseControls(caps)" in admin
assert "let _modelResponseCaptured = {};" in admin
assert "let _modelResponseDirty = {};" in admin
assert "_modelResponseCaptured[spec.key] = value" in admin
assert "nextIdentity === _modelResponseInitialIdentity" in admin
identity = _slice_function_body(admin, "_modelIdentity")
assert identity is not None
assert 'provider === "openai-compatible"' in identity
assert ': ""' in identity
merge = _slice_function_body(admin, "_mergeModelResponseControls")
assert merge is not None
# The dirty flag (select touched) may only override Advanced JSON for
# the identity that made it dirty — a stale flag from a renamed row
# must not delete a hand-typed JSON key.
assert "if (_modelResponseDirty[spec.key] && sameIdentity) delete caps[spec.key]" in merge
# The captured-value fallback is load-bearing, not a gating bug: a value
# lifted out of the row JSON on edit-open must stay visible and re-save
# for the same identity even when the baseline table says unsupported.
# The baseline arrives async (or never, on the compat lane); yielding to
# it would silently drop the pinned value on an unrelated edit-save.
# Wire safety lives server-side (emission gates on merged supports_*).
for body in (visibility, merge):
assert "_modelGetTile(spec.supportKey) || capturedFallback" in body
assert "sameIdentity" in body
assert "!(spec.supportKey in _modelCapsExplicit)" in body
create = _slice_function_body(admin, "showCreateModelModal")
assert create is not None
assert "_modelCapsSeq++" in create, "a fresh shelf must invalidate prior lookups"
assert "displayCaps.supports_verbosity !== false" in admin
assert "displayCaps.supports_pro_mode !== false" in admin
change = _slice_function_body(admin, "_onModelFieldChange")
assert change is not None
assert "_modelCapsSeq++" in change, "model changes must invalidate in-flight baselines"
assert "_modelCapsBaseline = {}" in change
assert 'apiSurfEl.addEventListener("change", _onModelFieldChange)' in admin
def test_shared_utils_defines_set_markdown_helper() -> None:
"""The ``setMarkdown`` helper in ``shared/utils.js`` is the single
audited entry point for rendering markdown content into a DOM
+670
View File
@@ -0,0 +1,670 @@
"""Unit tests for the per-session background-shell registry (#817).
The registry backs the ``bash(run_in_background=true)`` / ``bash_output`` /
``kill_shell`` tool surface: it spawns detached shells (``bash_N`` handles),
buffers their merged output in a capped rolling buffer, serves delta reads
(only lines since the last read), and reaps whole session groups on kill /
owner reap / close the #816 rule (the tracked command defines the lifetime,
nothing escapes its process group) extended to explicit backgrounding.
Pure registry tests no ChatSession. Session wiring is covered in
``test_bash_background_tool.py``.
"""
import re
import threading
import time
import pytest
from tests._proc_helpers import kill_pid as _kill_pid
from tests._proc_helpers import pid_alive as _pid_alive
from tests._proc_helpers import poll_until as _wait_until
# Module alias (from-style, matching the symbol imports below) for tests
# that monkeypatch module attributes (os.killpg, subprocess.Popen, ...).
from turnstone.core import background_shells as bg_mod
from turnstone.core.background_shells import (
BackgroundShellRegistry,
FilterExecError,
FilterTimeoutError,
TooManyShellsError,
UnknownShellError,
)
def _wait_status(shell, status, timeout=10.0):
return _wait_until(lambda: shell.status == status, timeout=timeout)
@pytest.fixture
def registry():
reg = BackgroundShellRegistry()
yield reg
reg.close()
# ---------------------------------------------------------------------------
# Handles + spawning
# ---------------------------------------------------------------------------
def test_spawn_returns_incrementing_bash_handles(registry):
s1 = registry.spawn("sleep 30")
s2 = registry.spawn("sleep 30")
assert s1.shell_id == "bash_1"
assert s2.shell_id == "bash_2"
def test_spawned_shell_is_running_with_live_pid(registry):
shell = registry.spawn("sleep 30")
assert shell.status == "running"
assert _pid_alive(shell.pid)
def test_spawn_records_command(registry):
shell = registry.spawn("sleep 30")
assert shell.command == "sleep 30"
def test_spawn_after_close_is_refused():
reg = BackgroundShellRegistry()
reg.close()
with pytest.raises(RuntimeError):
reg.spawn("echo hi")
def test_max_live_shells_cap():
reg = BackgroundShellRegistry(max_shells=2)
try:
reg.spawn("sleep 30")
s2 = reg.spawn("sleep 30")
with pytest.raises(TooManyShellsError):
reg.spawn("sleep 30")
# Cap counts LIVE shells: killing one frees a slot.
reg.kill(s2.shell_id)
s3 = reg.spawn("sleep 30")
assert s3.status == "running"
finally:
reg.close()
def test_completed_shells_do_not_count_toward_cap():
reg = BackgroundShellRegistry(max_shells=1)
try:
s1 = reg.spawn("true")
assert _wait_status(s1, "completed")
s2 = reg.spawn("sleep 30")
assert s2.status == "running"
finally:
reg.close()
# ---------------------------------------------------------------------------
# Exit tracking
# ---------------------------------------------------------------------------
def test_natural_exit_sets_completed_and_exit_code(registry):
shell = registry.spawn("exit 7")
assert _wait_status(shell, "completed")
assert shell.exit_code == 7
def test_output_is_complete_once_completed(registry):
"""Status flips to completed only after the drains finish: a read at
completed must see everything the command wrote."""
shell = registry.spawn("echo alpha; echo beta")
assert _wait_status(shell, "completed")
read = registry.read(shell.shell_id)
assert [ln.strip() for ln in read.lines] == ["alpha", "beta"]
def test_leader_exit_reaps_backgrounded_grandchild(registry, tmp_path):
"""#816 consistency: the tracked command defines the lifetime. When the
leader exits, the whole session group is killed a child the command
backgrounded does not outlive it."""
pidfile = tmp_path / "bg.pid"
shell = registry.spawn(f"sleep 60 & echo $! > {pidfile}; echo done")
bg_pid = None
try:
assert _wait_status(shell, "completed")
bg_pid = int(pidfile.read_text().strip())
assert _wait_until(lambda: not _pid_alive(bg_pid)), (
f"grandchild {bg_pid} leaked past leader exit"
)
read = registry.read(shell.shell_id)
assert "done" in "".join(read.lines)
finally:
if bg_pid is not None:
_kill_pid(bg_pid)
def test_stderr_lines_are_tagged_inline(registry):
shell = registry.spawn("echo out; echo err >&2")
assert _wait_status(shell, "completed")
lines = [ln.strip() for ln in registry.read(shell.shell_id).lines]
assert "out" in lines
assert "[stderr] err" in lines
# ---------------------------------------------------------------------------
# Delta reads
# ---------------------------------------------------------------------------
def test_read_returns_only_new_lines_since_last_read(registry):
"""The load-bearing convention: consecutive reads never overlap and never
drop a line collecting across polls yields each line exactly once."""
shell = registry.spawn("echo one; echo two; sleep 0.4; echo three; sleep 30")
collected: list[str] = []
def _collect():
collected.extend(ln.strip() for ln in registry.read(shell.shell_id).lines)
return "three" in collected
assert _wait_until(_collect)
assert collected == ["one", "two", "three"]
registry.kill(shell.shell_id)
def test_read_after_exit_then_again_reports_no_new_output(registry):
shell = registry.spawn("echo hi")
assert _wait_status(shell, "completed")
first = registry.read(shell.shell_id)
assert [ln.strip() for ln in first.lines] == ["hi"]
second = registry.read(shell.shell_id)
assert second.lines == []
assert second.status == "completed"
assert second.exit_code == 0
def test_read_reports_status_and_exit_code(registry):
shell = registry.spawn("sleep 30")
read = registry.read(shell.shell_id)
assert read.shell_id == shell.shell_id
assert read.status == "running"
assert read.exit_code is None
registry.kill(shell.shell_id)
def test_read_unknown_id_raises_with_live_ids(registry):
registry.spawn("sleep 30")
with pytest.raises(UnknownShellError) as excinfo:
registry.read("bash_99")
assert "bash_99" in str(excinfo.value)
assert "bash_1" in str(excinfo.value)
def test_read_unknown_id_when_registry_empty(registry):
with pytest.raises(UnknownShellError):
registry.read("bash_1")
# ---------------------------------------------------------------------------
# Filter
# ---------------------------------------------------------------------------
def test_filter_selects_matching_lines_only(registry):
shell = registry.spawn("echo match-a; echo skip-b; echo match-c")
assert _wait_status(shell, "completed")
read = registry.read(shell.shell_id, filter_pattern="^match")
assert [ln.strip() for ln in read.lines] == ["match-a", "match-c"]
def test_filter_is_display_only_and_consumes_the_delta(registry):
"""Filtered-out lines are consumed, not deferred — the cursor advances
past the whole delta (Claude Code ``BashOutput`` semantics)."""
shell = registry.spawn("echo match-a; echo skip-b")
assert _wait_status(shell, "completed")
first = registry.read(shell.shell_id, filter_pattern="^match")
assert [ln.strip() for ln in first.lines] == ["match-a"]
assert first.new_line_count == 2 # both lines were new, one shown
second = registry.read(shell.shell_id)
assert second.lines == []
assert second.new_line_count == 0
def test_filter_uses_search_not_match(registry):
shell = registry.spawn("echo prefix-needle-suffix")
assert _wait_status(shell, "completed")
read = registry.read(shell.shell_id, filter_pattern="needle")
assert len(read.lines) == 1
def test_invalid_filter_regex_raises(registry):
shell = registry.spawn("echo hi")
assert _wait_status(shell, "completed")
with pytest.raises(re.error):
registry.read(shell.shell_id, filter_pattern="[unclosed")
# ---------------------------------------------------------------------------
# Buffer cap
# ---------------------------------------------------------------------------
def test_buffer_cap_drops_oldest_and_reports_gap():
reg = BackgroundShellRegistry(max_buffer_chars=200)
try:
shell = reg.spawn('for i in $(seq 1 50); do echo "line-$i-padded-to-length"; done')
assert _wait_status(shell, "completed")
read = reg.read(shell.shell_id)
assert read.dropped_lines > 0
# Newest output survives; the tail is intact.
assert read.lines, "cap must retain the newest lines, not drop everything"
assert read.lines[-1].strip() == "line-50-padded-to-length"
finally:
reg.close()
def test_unread_lines_excludes_buffer_evicted():
"""The exit notice's line count must not promise evicted output."""
reg = BackgroundShellRegistry(max_buffer_chars=200)
try:
shell = reg.spawn('for i in $(seq 1 50); do echo "line-$i-padded-to-length"; done')
assert _wait_status(shell, "completed")
with shell.lock:
retained = len(shell._buffer)
assert shell.unread_lines == retained
finally:
reg.close()
def test_buffer_gap_is_relative_to_cursor():
"""Lines dropped BEFORE being read are a reported gap; lines already
read and then dropped are not."""
reg = BackgroundShellRegistry(max_buffer_chars=10_000)
try:
shell = reg.spawn("echo early; sleep 30")
# Each poll consumes whatever has arrived; stop once something did.
assert _wait_until(lambda: bool(reg.read(shell.shell_id).lines))
# Everything emitted so far is read; nothing has been dropped.
read = reg.read(shell.shell_id)
assert read.dropped_lines == 0
reg.kill(shell.shell_id)
finally:
reg.close()
# ---------------------------------------------------------------------------
# Kill / reap / close
# ---------------------------------------------------------------------------
def test_kill_marks_killed_and_reaps_group(registry, tmp_path):
pidfile = tmp_path / "bg.pid"
shell = registry.spawn(f"sleep 60 & echo $! > {pidfile}; sleep 60")
assert _wait_until(pidfile.exists)
bg_pid = int(pidfile.read_text().strip())
try:
killed = registry.kill(shell.shell_id)
assert killed.status == "killed"
assert _wait_until(lambda: not _pid_alive(shell.pid))
assert _wait_until(lambda: not _pid_alive(bg_pid)), "grandchild survived kill"
finally:
_kill_pid(bg_pid)
def test_kill_unknown_id_raises(registry):
with pytest.raises(UnknownShellError):
registry.kill("bash_7")
def test_killed_shell_output_remains_readable(registry, tmp_path):
"""Output that arrived before the kill survives it: the record keeps its
buffer, and ``kill`` returns only after the drains have flushed."""
sentinel = tmp_path / "started"
shell = registry.spawn(f"echo before-kill; touch {sentinel}; sleep 60")
assert _wait_until(sentinel.exists)
registry.kill(shell.shell_id)
read = registry.read(shell.shell_id)
assert read.status == "killed"
assert "before-kill" in "".join(read.lines)
def test_signal_all_kills_live_shells_without_closing(registry):
"""signal_all is the instant half of teardown: every live group dies,
but the registry stays open (records intact, spawns still allowed)
close() remains the complete teardown."""
s1 = registry.spawn("sleep 60")
s2 = registry.spawn("sleep 60")
registry.signal_all()
assert _wait_until(lambda: not _pid_alive(s1.pid))
assert _wait_until(lambda: not _pid_alive(s2.pid))
assert registry.has(s1.shell_id), "signal_all must not drop records"
s3 = registry.spawn("true")
assert _wait_status(s3, "completed"), "registry must remain usable after signal_all"
def test_close_kills_everything_and_is_idempotent():
reg = BackgroundShellRegistry()
s1 = reg.spawn("sleep 60")
s2 = reg.spawn("sleep 60")
reg.close()
assert not _pid_alive(s1.pid)
assert not _pid_alive(s2.pid)
reg.close() # second close is a no-op
def test_reap_owner_kills_only_that_owners_shells(registry):
mine = registry.spawn("sleep 60", owner="agent-1")
other = registry.spawn("sleep 60", owner="agent-2")
main = registry.spawn("sleep 60")
registry.reap(owner="agent-1")
assert _wait_until(lambda: not _pid_alive(mine.pid))
assert _pid_alive(other.pid)
assert _pid_alive(main.pid)
# ---------------------------------------------------------------------------
# Owner scoping
# ---------------------------------------------------------------------------
def test_owner_scoped_lookup_isolates_shells(registry):
agent_shell = registry.spawn("sleep 30", owner="agent-1")
main_shell = registry.spawn("sleep 30")
# Main scope cannot see the agent's shell...
with pytest.raises(UnknownShellError):
registry.read(agent_shell.shell_id)
# ...and the agent scope cannot see the main shell.
with pytest.raises(UnknownShellError):
registry.read(main_shell.shell_id, owner="agent-1")
# Each side reads its own.
assert registry.read(agent_shell.shell_id, owner="agent-1").status == "running"
assert registry.read(main_shell.shell_id).status == "running"
def test_shells_snapshot_is_owner_scoped(registry):
registry.spawn("sleep 30", owner="agent-1")
registry.spawn("sleep 30")
assert [s.owner for s in registry.shells(owner="agent-1")] == ["agent-1"]
assert [s.owner for s in registry.shells()] == [None]
def test_handles_are_unique_across_owners(registry):
a = registry.spawn("sleep 30", owner="agent-1")
b = registry.spawn("sleep 30")
assert a.shell_id != b.shell_id
# ---------------------------------------------------------------------------
# Exit callback (the notice hook)
# ---------------------------------------------------------------------------
def test_on_exit_fires_once_on_natural_exit():
fired = threading.Event()
seen = []
def _on_exit(shell):
seen.append(shell)
fired.set()
reg = BackgroundShellRegistry(on_exit=_on_exit)
try:
shell = reg.spawn("echo done")
assert fired.wait(10)
assert len(seen) == 1
assert seen[0].shell_id == shell.shell_id
assert seen[0].exit_code == 0
finally:
reg.close()
def test_on_exit_not_fired_for_kill():
seen = []
reg = BackgroundShellRegistry(on_exit=seen.append)
try:
shell = reg.spawn("sleep 60")
reg.kill(shell.shell_id)
assert _wait_until(lambda: not _pid_alive(shell.pid))
time.sleep(0.2) # give a buggy late callback a chance to land
assert seen == []
finally:
reg.close()
def test_on_exit_not_fired_for_close():
seen = []
reg = BackgroundShellRegistry(on_exit=seen.append)
shell = reg.spawn("sleep 60")
reg.close()
assert not _pid_alive(shell.pid)
time.sleep(0.2)
assert seen == []
# ---------------------------------------------------------------------------
# Review-hardening regressions (#817 code review)
# ---------------------------------------------------------------------------
def test_kill_on_completed_shell_does_not_signal_group(registry, monkeypatch):
"""A completed shell's pgid is a stale snapshot the OS may have recycled
to an unrelated process group kill() must not signal it (the waiter's
own group kill already ran at exit, when the pgid was fresh)."""
shell = registry.spawn("true")
assert _wait_status(shell, "completed")
calls = []
monkeypatch.setattr(bg_mod.os, "killpg", lambda *a: calls.append(a))
killed = registry.kill(shell.shell_id)
assert calls == [], "killpg must not fire for an already-exited shell"
assert killed.status == "completed", "a natural exit must not be relabelled 'killed'"
def test_close_is_time_bounded_with_pipe_holding_escapee(registry, tmp_path):
"""An escaped-group grandchild that holds the output pipes wedges the
drain threads. close() must still return within its total budget
it can run under the server's async close route, where an unbounded
join would freeze the whole node's event loop."""
pidfile = tmp_path / "holder.pid"
# ``setsid`` puts the sleep in a NEW session (outside our kill group)
# while it still inherits our stdout/stderr pipes — the accepted
# leaked-daemon case from the module docstring.
shell = registry.spawn(f"setsid sleep 60 & echo $! > {pidfile}; echo started")
assert _wait_until(pidfile.exists)
holder_pid = int(pidfile.read_text().strip())
try:
start = time.monotonic()
registry.close()
elapsed = time.monotonic() - start
assert elapsed < 8, f"close() took {elapsed:.1f}s — teardown must be budget-bounded"
finally:
_kill_pid(holder_pid)
# The holder is dead, so the wedged drains EOF promptly; wait for
# them here so the conftest leak guard sees a clean teardown.
assert _wait_until(lambda: not any(t.is_alive() for t in shell._threads))
def test_exited_records_are_pruned_at_cap():
reg = BackgroundShellRegistry(max_exited_records=2)
try:
shells = [reg.spawn(f"echo job-{i}") for i in range(3)]
for s in shells:
assert _wait_status(s, "completed")
# Eviction happens on each exit; poll until the oldest is gone
# (waiter threads race, prune runs per-exit).
assert _wait_until(lambda: not reg.has(shells[0].shell_id))
assert reg.has(shells[1].shell_id)
assert reg.has(shells[2].shell_id)
with pytest.raises(UnknownShellError):
reg.read(shells[0].shell_id)
finally:
reg.close()
def test_catastrophic_filter_times_out_without_consuming(registry):
"""A backtracking-bomb filter must error within the bound and consume
NOTHING the retry without a filter still gets the output. The match
runs in a killable child process: sre holds the GIL, so an in-process
bomb would freeze the whole interpreter, watchdogs included."""
# One ~3000-char line of a's ending in 'b' — the classic (a+)+$ bomb
# subject — followed by a sentinel line.
shell = registry.spawn("printf 'a%.0s' $(seq 1 3000); echo b; echo tail-line")
assert _wait_status(shell, "completed")
start = time.monotonic()
with pytest.raises(FilterTimeoutError):
registry.read(shell.shell_id, filter_pattern=r"(a+)+$")
assert time.monotonic() - start < 10, "filter timeout must be bounded"
# Nothing was consumed: an unfiltered read sees the whole delta.
read = registry.read(shell.shell_id)
assert any("tail-line" in ln for ln in read.lines)
def test_overlong_filter_pattern_is_rejected(registry):
shell = registry.spawn("echo hi")
assert _wait_status(shell, "completed")
with pytest.raises(re.error):
registry.read(shell.shell_id, filter_pattern="x" * 600)
def test_cap_error_is_owner_scope_honest():
"""The cap is registry-wide, but the advice must only name shells the
caller can actually kill kill_shell is owner-scoped."""
reg = BackgroundShellRegistry(max_shells=1)
try:
reg.spawn("sleep 30") # main scope fills the cap
with pytest.raises(TooManyShellsError) as excinfo:
reg.spawn("sleep 30", owner="agent-1")
msg = str(excinfo.value)
assert "bash_1" not in msg, "must not advise killing another scope's shell"
assert "other agents" in msg
# The same-scope variant names the killable shell.
with pytest.raises(TooManyShellsError) as excinfo2:
reg.spawn("sleep 30")
assert "bash_1" in str(excinfo2.value)
assert "kill_shell" in str(excinfo2.value)
finally:
reg.close()
def test_prune_evicts_by_exit_order_not_spawn_order():
"""A long-lived first-spawned server must never be evicted by its OWN
exit's prune once enough later jobs have finished — eviction follows
exit order, so the just-exited shell is always the newest record."""
reg = BackgroundShellRegistry(max_exited_records=2)
try:
server = reg.spawn("sleep 30") # bash_1, exits LAST
jobs = [reg.spawn(f"echo job-{i}") for i in range(3)]
for job in jobs:
assert _wait_status(job, "completed")
reg.kill(server.shell_id)
assert reg.has(server.shell_id), "the just-exited shell must survive its own exit's prune"
# The earliest-EXITED job is the eviction victim, not bash_1.
assert _wait_until(lambda: len(reg.shells()) <= 3)
assert reg.read(server.shell_id).status == "killed"
finally:
reg.close()
def test_thread_start_failure_leaves_no_orphan_record(registry, monkeypatch, tmp_path):
"""If Thread.start raises (thread exhaustion), the record must be
unregistered and the fresh group reaped an orphan with never-started
Thread objects would make every later close()/reap() join raise and
abort session teardown."""
pidfile = tmp_path / "leader.pid"
real_thread = bg_mod.threading.Thread
class FailingWaiterThread(real_thread):
def start(self):
if "bg-shell-wait" in (self.name or ""):
raise RuntimeError("can't start new thread")
super().start()
monkeypatch.setattr(bg_mod.threading, "Thread", FailingWaiterThread)
with pytest.raises(RuntimeError):
registry.spawn(f"echo $$ > {pidfile}; sleep 60")
assert registry.shells() == [], "failed spawn must not strand a record"
if pidfile.exists():
leader_pid = int(pidfile.read_text().strip())
assert _wait_until(lambda: not _pid_alive(leader_pid)), "fresh group leaked"
monkeypatch.undo()
registry.close() # must not raise on the (empty) registry
def test_filter_helper_failure_reports_exec_error_not_timeout(registry, monkeypatch):
"""A crashed helper must not tell the model its (fine) pattern was too
slow and must not consume the delta."""
shell = registry.spawn("echo hello")
assert _wait_status(shell, "completed")
monkeypatch.setattr(bg_mod.sys, "executable", "/bin/false")
with pytest.raises(FilterExecError) as excinfo:
registry.read(shell.shell_id, filter_pattern="hello")
assert "not a problem with your pattern" in str(excinfo.value)
monkeypatch.undo()
read = registry.read(shell.shell_id)
assert [ln.strip() for ln in read.lines] == ["hello"]
def test_filter_matches_only_within_line_cap_and_reports_clipping(registry):
"""Lines are truncated parent-side before shipping to the helper: a
match beyond the per-line cap is not found (a filter targets log
lines), and a huge retained line cannot burn the time budget on I/O.
The clipping is NEVER silent the read reports how many lines were
only partially visible to the pattern."""
shell = registry.spawn("printf 'x%.0s' $(seq 1 5000); echo needle-suffix")
assert _wait_status(shell, "completed")
read = registry.read(shell.shell_id, filter_pattern="needle")
assert read.lines == []
assert read.new_line_count == 1
assert read.clipped_lines == 1
def test_concurrent_reads_never_double_deliver(registry):
"""Two simultaneous reads of one shell must SPLIT the delta between
them, never both return it the whole pass (snapshot commit)
serializes per shell. Without that, a parallel tool batch reading the
same handle gets every line twice."""
shell = registry.spawn("seq 1 200")
assert _wait_status(shell, "completed")
results: list[list[str]] = [[], []]
barrier = threading.Barrier(2)
def _reader(slot: int) -> None:
barrier.wait()
results[slot] = [ln.strip() for ln in registry.read(shell.shell_id).lines]
threads = [threading.Thread(target=_reader, args=(i,)) for i in range(2)]
for t in threads:
t.start()
for t in threads:
t.join(timeout=10)
combined = results[0] + results[1]
assert len(combined) == 200, f"expected each line exactly once, got {len(combined)}"
assert sorted(combined, key=int) == [str(i) for i in range(1, 201)]
def test_filter_helper_spawn_failure_is_exec_error(registry, monkeypatch):
"""A helper that fails to LAUNCH (fork pressure) must land in the same
honest FilterExecError as a crashed helper not escape as a raw
OSError blaming nothing and must not consume the delta."""
shell = registry.spawn("echo hello")
assert _wait_status(shell, "completed")
def _boom(*args, **kwargs):
raise BlockingIOError("Resource temporarily unavailable")
monkeypatch.setattr(bg_mod.subprocess, "Popen", _boom)
with pytest.raises(FilterExecError):
registry.read(shell.shell_id, filter_pattern="hello")
monkeypatch.undo()
read = registry.read(shell.shell_id)
assert [ln.strip() for ln in read.lines] == ["hello"]
def test_on_exit_exception_does_not_wedge_the_shell():
def _boom(shell):
raise RuntimeError("callback bug")
reg = BackgroundShellRegistry(on_exit=_boom)
try:
shell = reg.spawn("echo hi")
# The waiter thread must survive the callback raising: status still
# lands and output is still readable.
assert _wait_status(shell, "completed")
assert [ln.strip() for ln in reg.read(shell.shell_id).lines] == ["hi"]
finally:
reg.close()
+801
View File
@@ -0,0 +1,801 @@
"""Session-level tests for the background-shell tool surface (#817).
Covers the wiring around :class:`BackgroundShellRegistry`:
* ``bash`` gains ``run_in_background: true`` (alias ``is_background``)
same approval gate, returns immediately with a ``bash_N`` handle.
* ``bash_output`` auto-approved delta reader (status + exit code + only
new output since the last call, optional ``filter`` regex).
* ``kill_shell`` auto-approved kill of a registered shell's whole group.
* Exit notices ride the NudgeQueue on channel ``"any"`` (the watch rail) so
they drain at the next seam and can wake an idle workstream.
* Lifecycle: ``close()`` reaps everything; generation-``cancel()`` does NOT
(a deliberately-detached server survives a stopped turn); shells spawned
inside a task_agent are owner-scoped and reaped when the agent finishes.
"""
import time
import pytest
from tests._proc_helpers import pid_alive as _pid_alive
from tests._proc_helpers import poll_until as _wait_until
from tests._session_helpers import make_session
@pytest.fixture
def session():
s = make_session()
yield s
s.close()
def _start_background(session, command, call_id="bg1", **extra_args):
"""Prepare + execute a backgrounded bash call; return the result text."""
args = {"command": command, "run_in_background": True, **extra_args}
prepared = session._prepare_bash(call_id, args)
assert "error" not in prepared, prepared.get("error")
_cid, output = prepared["execute"](prepared)
return output
def _only_shell(session):
shells = session._background_shells.shells()
assert len(shells) == 1
return shells[0]
# ---------------------------------------------------------------------------
# bash: run_in_background routing
# ---------------------------------------------------------------------------
def test_prepare_bash_background_keeps_approval_gate(session):
prepared = session._prepare_bash("c1", {"command": "sleep 30", "run_in_background": True})
assert prepared["needs_approval"] is True
assert prepared["approval_label"] == "bash"
def test_prepare_bash_background_header_says_background(session):
prepared = session._prepare_bash("c1", {"command": "sleep 30", "run_in_background": True})
assert "background" in prepared["header"]
def test_background_bash_returns_immediately_with_handle(session):
start = time.monotonic()
output = _start_background(session, "sleep 30")
elapsed = time.monotonic() - start
assert elapsed < 5, f"backgrounded call blocked for {elapsed:.1f}s"
assert "bash_1" in output
shell = _only_shell(session)
assert shell.status == "running"
assert _pid_alive(shell.pid)
def test_background_start_mentions_reader_and_killer(session):
"""The immediate result must teach the follow-up tools — weak-prior
models (GPT-5.6) only reach for the poll pattern if the result names it."""
output = _start_background(session, "sleep 30")
assert "bash_output" in output
assert "kill_shell" in output
def test_is_background_alias_accepted(session):
output = _start_background(session, "sleep 30", is_background=True)
assert "bash_1" in output
assert _only_shell(session).status == "running"
def test_foreground_bash_routing_unchanged(session):
prepared = session._prepare_bash("c1", {"command": "echo hi"})
assert prepared["execute"] == session._exec_bash
prepared_false = session._prepare_bash("c2", {"command": "echo hi", "run_in_background": False})
assert prepared_false["execute"] == session._exec_bash
def test_background_respects_command_blocklist(session):
prepared = session._prepare_bash("c1", {"command": "shutdown now", "run_in_background": True})
assert "error" in prepared
assert session._background_shells.shells() == []
def test_background_ignores_timeout(session):
"""No bounded wait exists to time out — a 1s timeout must not kill the
detached shell."""
_start_background(session, "sleep 30", timeout=1)
shell = _only_shell(session)
time.sleep(1.5)
assert shell.status == "running"
assert _pid_alive(shell.pid)
def test_background_spawn_failure_reports_error(session, monkeypatch):
from turnstone.core import background_shells as bg_mod
def _boom(*args, **kwargs):
raise OSError("cannot fork")
monkeypatch.setattr(bg_mod.subprocess, "Popen", _boom)
prepared = session._prepare_bash("c1", {"command": "echo hi", "run_in_background": True})
_cid, output = prepared["execute"](prepared)
assert "cannot fork" in output
def test_too_many_background_shells_reports_error(session, monkeypatch):
monkeypatch.setattr(session._background_shells, "_max_shells", 1)
_start_background(session, "sleep 30", call_id="bg1")
output = _start_background(session, "sleep 30", call_id="bg2")
assert "bash_1" in output # the live shell is named so the model can kill it
assert len(session._background_shells.shells()) == 1
# ---------------------------------------------------------------------------
# bash_output
# ---------------------------------------------------------------------------
def test_bash_output_is_auto_approved(session):
prepared = session._prepare_bash_output("c1", {"id": "bash_1"})
assert prepared["needs_approval"] is False
def test_bash_output_missing_id_errors(session):
prepared = session._prepare_bash_output("c1", {})
assert "error" in prepared
def test_bash_output_returns_delta_then_no_new_output(session):
_start_background(session, "echo hello; sleep 30")
shell = _only_shell(session)
assert _wait_until(lambda: shell.status == "running")
def _read():
prepared = session._prepare_bash_output("r", {"id": shell.shell_id})
assert "error" not in prepared
return prepared["execute"](prepared)[1]
assert _wait_until(lambda: "hello" in _read())
again = _read()
assert "hello" not in again
assert "no new output" in again.lower()
assert "running" in again.lower()
def test_bash_output_reports_exit_code_when_completed(session):
_start_background(session, "exit 3")
shell = _only_shell(session)
assert _wait_until(lambda: shell.status == "completed")
prepared = session._prepare_bash_output("r", {"id": shell.shell_id})
_cid, output = prepared["execute"](prepared)
assert "completed" in output.lower()
assert "3" in output
def test_bash_output_filter_applies(session):
_start_background(session, "echo match-a; echo skip-b")
shell = _only_shell(session)
assert _wait_until(lambda: shell.status == "completed")
prepared = session._prepare_bash_output("r", {"id": shell.shell_id, "filter": "^match"})
_cid, output = prepared["execute"](prepared)
assert "match-a" in output
assert "skip-b" not in output
def test_bash_output_invalid_filter_reports_error(session):
_start_background(session, "sleep 30")
shell = _only_shell(session)
prepared = session._prepare_bash_output("r", {"id": shell.shell_id, "filter": "[bad"})
_cid, output = prepared["execute"](prepared)
assert "regex" in output.lower() or "filter" in output.lower()
def test_bash_output_unknown_id_lists_live_shells(session):
_start_background(session, "sleep 30")
prepared = session._prepare_bash_output("r", {"id": "bash_42"})
_cid, output = prepared["execute"](prepared)
assert "bash_42" in output
assert "bash_1" in output
# ---------------------------------------------------------------------------
# kill_shell
# ---------------------------------------------------------------------------
def test_kill_shell_is_auto_approved(session):
prepared = session._prepare_kill_shell("c1", {"id": "bash_1"})
assert prepared["needs_approval"] is False
def test_kill_shell_missing_id_errors(session):
prepared = session._prepare_kill_shell("c1", {})
assert "error" in prepared
def test_kill_shell_kills_and_reports(session):
_start_background(session, "sleep 60")
shell = _only_shell(session)
prepared = session._prepare_kill_shell("k", {"id": shell.shell_id})
_cid, output = prepared["execute"](prepared)
assert "killed" in output.lower()
assert _wait_until(lambda: not _pid_alive(shell.pid))
# The schema promises the exit code for ANY exited state, killed included.
read_prepared = session._prepare_bash_output("r", {"id": shell.shell_id})
_cid, read_output = read_prepared["execute"](read_prepared)
assert "exit code" in read_output
def test_kill_shell_unknown_id_reports_error(session):
prepared = session._prepare_kill_shell("k", {"id": "bash_9"})
_cid, output = prepared["execute"](prepared)
assert "bash_9" in output
# ---------------------------------------------------------------------------
# Exit notices (NudgeQueue, channel "any", wake)
# ---------------------------------------------------------------------------
def test_natural_exit_enqueues_any_channel_notice(session):
_start_background(session, "echo done")
assert _wait_until(
lambda: any(t == "background_shell_exit" for t, _ in session._nudge_queue.pending())
)
entries = session._nudge_queue.pending(channel="any")
texts = [text for t, text in entries if t == "background_shell_exit"]
assert texts, "notice must ride channel 'any' so it can wake an idle workstream"
assert "bash_1" in texts[0]
assert "bash_output" in texts[0]
def test_exit_notice_carries_metadata(session):
_start_background(session, "exit 5")
assert _wait_until(
lambda: any(t == "background_shell_exit" for t, _ in session._nudge_queue.pending())
)
metadata = [
meta
for t, _text, meta in session._nudge_queue.pending_with_metadata()
if t == "background_shell_exit"
][0]
assert metadata["shell_id"] == "bash_1"
assert metadata["exit_code"] == 5
def test_exit_notice_triggers_wake_fn(session):
wakes = []
session._watch_wake_fn = lambda: wakes.append(1)
_start_background(session, "echo done")
assert _wait_until(lambda: wakes), "natural exit must wake an idle workstream"
def test_kill_shell_suppresses_exit_notice(session):
_start_background(session, "sleep 60")
shell = _only_shell(session)
prepared = session._prepare_kill_shell("k", {"id": shell.shell_id})
prepared["execute"](prepared)
assert _wait_until(lambda: not _pid_alive(shell.pid))
time.sleep(0.3) # a buggy late notice would land within this window
assert not any(t == "background_shell_exit" for t, _ in session._nudge_queue.pending())
def test_close_drops_pending_exit_notice_via_valid_until(session):
"""A notice for a shell that no longer exists (registry closed) must not
deliver the valid_until predicate drops it at drain time."""
_start_background(session, "echo done")
assert _wait_until(
lambda: any(t == "background_shell_exit" for t, _ in session._nudge_queue.pending())
)
session.close()
from turnstone.core.nudge_queue import USER_DRAIN
drained = session._nudge_queue.drain(USER_DRAIN)
assert not any(t == "background_shell_exit" for t, _text, _m in drained)
# ---------------------------------------------------------------------------
# Lifecycle
# ---------------------------------------------------------------------------
def test_close_reaps_background_shells(session):
_start_background(session, "sleep 60")
shell = _only_shell(session)
session.close()
assert not _pid_alive(shell.pid)
def test_generation_cancel_does_not_reap_background_shells(session):
"""cancel() fires on mere stop-generation — a deliberately-detached
server must survive it. Only close()/kill_shell end it."""
_start_background(session, "sleep 60")
shell = _only_shell(session)
session.cancel()
time.sleep(0.3)
assert _pid_alive(shell.pid), "generation cancel must not kill detached shells"
# ---------------------------------------------------------------------------
# Review-hardening regressions (#817 code review)
# ---------------------------------------------------------------------------
def test_string_typed_background_flag_is_honored(session):
"""Providers intermittently send booleans as strings; 'true' must not
silently fall through to the foreground executor (where the group kill
would reap the server the model believed it detached)."""
for call_id, args in (
("s1", {"command": "sleep 30", "run_in_background": "true"}),
("s2", {"command": "sleep 30", "is_background": "True"}),
):
prepared = session._prepare_bash(call_id, args)
assert prepared["execute"] == session._exec_bash_background, args
def test_kill_shell_on_completed_shell_reports_already_exited(session):
_start_background(session, "true")
shell = _only_shell(session)
assert _wait_until(lambda: shell.status == "completed")
prepared = session._prepare_kill_shell("k", {"id": shell.shell_id})
_cid, output = prepared["execute"](prepared)
assert "already exited" in output.lower()
def test_exit_notice_survives_generation_abandon_without_waking(session):
"""cancel/interrupt/exception clear generation-scoped advisories, but an
external event (a background shell exited) still happened its notice
must survive to the next seam or the model keeps talking to a dead
server. It survives DEMOTED to 'quiet': still deliverable, but no
longer wake-eligible, so the workstream the user just stopped cannot
resume itself over it."""
from turnstone.core.nudge_queue import USER_DRAIN, WAKE_PENDING
_start_background(session, "echo done")
assert _wait_until(
lambda: any(t == "background_shell_exit" for t, _ in session._nudge_queue.pending())
)
session._queue_tool_advisory("tool_error", "3 consecutive tool errors")
session._drain_pending_advisories()
kinds = [t for t, _ in session._nudge_queue.pending()]
assert "background_shell_exit" in kinds
assert "tool_error" not in kinds
# Post-cancel quiescence: nothing is wake-eligible...
assert not session._nudge_queue.has_pending(WAKE_PENDING)
# ...yet the notice still delivers at the next legitimate seam.
drained = session._nudge_queue.drain(USER_DRAIN)
assert any(t == "background_shell_exit" for t, _x, _m in drained)
def test_int_typed_background_flag_is_honored(session):
prepared = session._prepare_bash("i1", {"command": "sleep 30", "run_in_background": 1})
assert prepared["execute"] == session._exec_bash_background
prepared_zero = session._prepare_bash("i2", {"command": "echo hi", "run_in_background": 0})
assert prepared_zero["execute"] == session._exec_bash
def test_bash_output_non_string_filter_errors_without_consuming(session):
_start_background(session, "echo hello; sleep 30")
shell = _only_shell(session)
prepared = session._prepare_bash_output("r", {"id": shell.shell_id, "filter": 123})
assert "error" in prepared
assert "filter" in prepared["error"].lower()
# Nothing was consumed by the refused call.
assert _wait_until(lambda: shell.unread_lines > 0)
def test_filter_timeout_reports_error_without_consuming(session, monkeypatch):
from turnstone.core.background_shells import FilterTimeoutError
_start_background(session, "sleep 30")
shell = _only_shell(session)
def _boom(*a, **kw):
raise FilterTimeoutError("filter regex took longer than 2s to run")
monkeypatch.setattr(session._background_shells, "read", _boom)
prepared = session._prepare_bash_output("r", {"id": shell.shell_id, "filter": "(a+)+$"})
_cid, output = prepared["execute"](prepared)
assert "filter" in output.lower()
assert "error" in output.lower()
def test_registries_are_isolated_per_session():
"""Workstream isolation: a handle from one session must be unresolvable
from another buffers, ids, and kills never cross ChatSessions."""
session_a = make_session()
session_b = make_session()
try:
_start_background(session_a, "sleep 30")
shell_a = _only_shell(session_a)
read_b = session_b._prepare_bash_output("r", {"id": shell_a.shell_id})
_cid, output = read_b["execute"](read_b)
assert "no background shell" in output.lower()
kill_b = session_b._prepare_kill_shell("k", {"id": shell_a.shell_id})
_cid, kill_output = kill_b["execute"](kill_b)
assert "no background shell" in kill_output.lower()
assert _pid_alive(shell_a.pid), "another session must not be able to kill the shell"
finally:
session_a.close()
session_b.close()
def test_bash_output_polling_is_repeat_exempt(session):
"""Repeated identical bash_output calls ARE the documented monitoring
pattern the repeat detector must not brand them 'identical repeat'
(the delta result differs by construction) nor queue a repeat nudge."""
import json as _json
_start_background(session, "sleep 30")
shell = _only_shell(session)
args = _json.dumps({"id": shell.shell_id})
for i in range(5):
tool_calls = [{"id": f"t{i}", "function": {"name": "bash_output", "arguments": args}}]
results = [(f"t{i}", "bash_1 (running)\nNo new output since the last read.")]
session._apply_post_execute_advisories(tool_calls, results)
assert "identical repeat" not in results[0][1]
assert not any(t == "repeat" for t, _ in session._nudge_queue.pending())
def test_repeat_exempt_calls_still_break_other_streaks(session):
"""The exemption suppresses the WARNING, not the recording: a
bash_output poll interleaved between identical bash calls must reset
the bash streak otherwise the documented monitor-and-probe loop
(poll, curl health, poll, curl health) draws a false 'identical
repeat' on the probe."""
import json as _json
_start_background(session, "sleep 30")
shell = _only_shell(session)
poll_args = _json.dumps({"id": shell.shell_id})
probe_args = _json.dumps({"command": "curl -s localhost:8080/health"})
for i in range(6):
probe = [{"id": f"p{i}", "function": {"name": "bash", "arguments": probe_args}}]
probe_results = [(f"p{i}", "ok")]
session._apply_post_execute_advisories(probe, probe_results)
assert "identical repeat" not in probe_results[0][1], (
"interleaved probes are not a stuck loop"
)
poll = [{"id": f"q{i}", "function": {"name": "bash_output", "arguments": poll_args}}]
session._apply_post_execute_advisories(poll, [(f"q{i}", "no new output")])
def test_bash_repeats_still_warn(session):
"""The exemption is bash_output-specific: a genuinely stuck identical
bash loop still gets the warning."""
import json as _json
args = _json.dumps({"command": "echo test"})
warned = False
for i in range(5):
tool_calls = [{"id": f"b{i}", "function": {"name": "bash", "arguments": args}}]
results = [(f"b{i}", "test")]
session._apply_post_execute_advisories(tool_calls, results)
warned = warned or "identical repeat" in results[0][1]
assert warned
def test_quiet_only_entries_do_not_trigger_wake_delivery(session, monkeypatch):
"""A dispatched wake whose wake-eligible entries all evaporated must be
a no-op: quiet entries alone never resume a stopped workstream, and
they stay queued for the next legitimate seam."""
calls = []
monkeypatch.setattr(session, "send", lambda *a, **k: calls.append(1))
session._nudge_queue.enqueue("background_shell_exit", "old news", "quiet")
session.deliver_wake_nudge_from_queue()
assert calls == []
assert session._nudge_queue.pending(channel="quiet") == [("background_shell_exit", "old news")]
def test_wake_delivers_quiet_alongside_eligible_in_insertion_order(session, monkeypatch):
"""Quiet entries ride the wake AND cross-channel chronology holds: an
older demoted notice renders before the newer fire that earned the
wake (a poll counter must never run backwards)."""
seen = {}
def _fake_send(*a, **k):
seen["reminders"] = list(session._wake_drained_reminders or [])
session._wake_drained_reminders = None # emulate emission consuming
monkeypatch.setattr(session, "send", _fake_send)
session._nudge_queue.enqueue("background_shell_exit", "old", "quiet")
session._nudge_queue.enqueue("watch_triggered", "new", "any")
session.deliver_wake_nudge_from_queue()
types = [e["type"] for e in seen["reminders"]]
assert types == ["background_shell_exit", "watch_triggered"], (
"older quiet entry must precede the newer wake-eligible one"
)
assert session._nudge_queue.pending() == []
def test_failed_wake_reenqueue_preserves_valid_until(session, monkeypatch):
"""The re-enqueued notice keeps its staleness predicate — a stale
notice re-queued by a failed wake must still be droppable at its next
drain, not delivered against a gone shell."""
from turnstone.core.nudge_queue import USER_DRAIN
alive = {"value": True}
def _fail(*a, **k):
raise RuntimeError("storage down")
monkeypatch.setattr(session, "send", _fail)
session._nudge_queue.enqueue(
"background_shell_exit",
"server died",
"any",
valid_until=lambda: alive["value"],
)
with pytest.raises(RuntimeError):
session.deliver_wake_nudge_from_queue()
assert session._nudge_queue.pending(channel="quiet"), "notice must be re-queued"
alive["value"] = False # the shell record is gone now
drained = session._nudge_queue.drain(USER_DRAIN)
assert drained == [], "stale re-queued notice must drop via its predicate"
def test_mid_emit_failure_restashes_unemitted_tail(session, monkeypatch):
"""A failure while emitting reminder k of n must leave k..n recoverable
the wake caller's finally re-enqueues them instead of losing the
suffix."""
calls = {"n": 0}
def _append(source, text, **meta):
calls["n"] += 1
if calls["n"] == 2:
raise RuntimeError("storage down")
monkeypatch.setattr(session, "_append_system_turn", _append)
session._wake_drained_reminders = [
{"type": "a", "text": "1"},
{"type": "b", "text": "2"},
{"type": "c", "text": "3"},
]
with pytest.raises(RuntimeError):
session._emit_pending_user_nudges()
assert session._wake_drained_reminders == [
{"type": "b", "text": "2"},
{"type": "c", "text": "3"},
]
def test_failed_wake_reenqueues_undelivered_as_quiet(session, monkeypatch):
"""A wake send that dies before emitting its drained reminders must not
eat them a shell's exit notice fires exactly once."""
def _fail(*a, **k):
raise RuntimeError("storage down")
monkeypatch.setattr(session, "send", _fail)
session._nudge_queue.enqueue(
"background_shell_exit", "server died", "any", metadata={"shell_id": "bash_1"}
)
with pytest.raises(RuntimeError):
session.deliver_wake_nudge_from_queue()
pending = session._nudge_queue.pending_with_metadata(channel="quiet")
assert [(t, x) for t, x, _m in pending] == [("background_shell_exit", "server died")]
assert pending[0][2] == {"shell_id": "bash_1"}
def test_failed_wake_preserves_chronology_and_stays_wake_quiescent(session, monkeypatch):
"""Failed-wake recovery invariants: (a) the re-queued external notice
keeps its seq, so the retry renders it BEFORE a newer event that
arrived during the failure; (b) NOTHING wake-eligible remains after
the failure external notices demote to quiet and user-channel
advisories are dropped outright, because a re-armed WAKE_PENDING gate
plus the zero-backoff worker-exit retry would respawn wake workers in
an unbounded hot loop against a persistent failure."""
from turnstone.core.nudge_queue import WAKE_PENDING
calls = {"n": 0}
seen = {}
def _send(*a, **k):
calls["n"] += 1
if calls["n"] == 1:
raise RuntimeError("transient storage failure")
seen["reminders"] = list(session._wake_drained_reminders or [])
session._wake_drained_reminders = None
monkeypatch.setattr(session, "send", _send)
session._nudge_queue.enqueue("watch_triggered", "poll-4", "any")
session._nudge_queue.enqueue("correction", "user advisory", "user")
with pytest.raises(RuntimeError):
session.deliver_wake_nudge_from_queue()
# (b) bounded: nothing left that could re-trigger the wake gate.
assert not session._nudge_queue.has_pending(WAKE_PENDING), (
"a failed wake must not leave wake-eligible entries (respawn hot loop)"
)
assert [t for t, _x in session._nudge_queue.pending(channel="quiet")] == ["watch_triggered"]
# A NEWER event lands after the failure...
session._nudge_queue.enqueue("watch_triggered", "poll-5", "any")
session.deliver_wake_nudge_from_queue()
texts = [e["text"] for e in seen["reminders"]]
# (a) ...and the retry renders old-before-new despite the round trip.
assert texts.index("poll-4") < texts.index("poll-5")
def test_exit_notice_emits_end_to_end_as_system_turn(session):
"""THE test whose absence hid an undeliverable notice for six review
rounds: drive the notice through REAL emission (make_system_turn +
_append_system_turn), not just queue assertions an unregistered
``_source`` raises ValueError only at this layer."""
_start_background(session, "echo done")
assert _wait_until(
lambda: any(t == "background_shell_exit" for t, _ in session._nudge_queue.pending())
)
from turnstone.core.trajectory import Role
before = len(session.messages)
session._emit_pending_user_nudges() # must not raise
new_turns = session.messages[before:]
assert any(
turn.role is Role.SYSTEM and turn.source == "background_shell_exit" for turn in new_turns
), f"exit notice must land as a first-class system turn, got {new_turns!r}"
def test_cli_exit_closes_every_loaded_session():
"""CLI exit must reap background shells in EVERY workstream, not just
the active one a server started before /new must not outlive /exit."""
from unittest.mock import MagicMock
from turnstone.cli import _close_all_sessions
ws_a, ws_b, ws_never_loaded = MagicMock(), MagicMock(), MagicMock()
ws_never_loaded.session = None
ws_a.session.close.side_effect = RuntimeError("bad teardown")
manager = MagicMock()
manager.list_all.return_value = [ws_a, ws_b, ws_never_loaded]
_close_all_sessions(manager) # must not raise
ws_a.session.close.assert_called_once()
ws_b.session.close.assert_called_once(), "one bad teardown must not stop the rest"
# Signal phase ran for every loaded session, before any close.
ws_a.session._background_shells.signal_all.assert_called_once()
ws_b.session._background_shells.signal_all.assert_called_once()
def test_cli_exit_ctrl_c_does_not_abort_the_reap():
"""Ctrl-C during the close phase must not escape the helper: the kill
signals already landed on every session in phase 1, and an escaping
KeyboardInterrupt would also skip MCP/registry shutdown in main()."""
from unittest.mock import MagicMock
from turnstone.cli import _close_all_sessions
ws_a, ws_b = MagicMock(), MagicMock()
ws_a.session.close.side_effect = KeyboardInterrupt
manager = MagicMock()
manager.list_all.return_value = [ws_a, ws_b]
_close_all_sessions(manager) # must not raise
ws_a.session._background_shells.signal_all.assert_called_once()
(
ws_b.session._background_shells.signal_all.assert_called_once(),
("signals must land on every session before the interruptible close phase"),
)
def test_non_string_reminder_text_drops_silently(session):
"""A dict reminder with non-str text must drop at the rail, not
TypeError out of the dispatch closure (WatchRunner would re-fire the
row every tick)."""
runner = type(
"R",
(),
{
"set_dispatch_fn": lambda self, ws, fn: None,
"remove_dispatch_fn": lambda self, ws, owner=None: None,
},
)()
session.set_watch_runner(runner)
session._watch_dispatch_fn({"text": 123, "watch_name": "w"}, "watch-1") # must not raise
assert session._nudge_queue.pending() == []
def test_string_typed_stop_on_error_is_honored(session):
"""One coercion dialect for every bash boolean: a string-typed
stop_on_error must add set -e in both branches, not silently drop it."""
fg = session._prepare_bash("f1", {"command": "echo hi", "stop_on_error": "true"})
assert fg["stop_on_error"] is True
bg = session._prepare_bash(
"b1", {"command": "echo hi", "run_in_background": True, "stop_on_error": "true"}
)
assert bg["stop_on_error"] is True
def test_non_dict_watch_reminder_drops_silently(session):
"""The rebuilt dispatch closure must drop a non-dict reminder like the
old code did a TypeError would make WatchRunner hold and re-fire the
row every tick."""
runner = type(
"R",
(),
{
"set_dispatch_fn": lambda self, ws, fn: None,
"remove_dispatch_fn": lambda self, ws, owner=None: None,
},
)()
session.set_watch_runner(runner)
dispatch = session._watch_dispatch_fn
dispatch("not a dict", "watch-1") # must not raise
assert session._nudge_queue.pending() == []
def test_truthy_flag_dialect_is_unified():
"""One coercion dialect file-wide — 'on' and nonzero numbers count, so a
provider quirk honored on coordinator tools is honored on bash too."""
from turnstone.core.session import _is_truthy_flag
assert _is_truthy_flag(True)
assert _is_truthy_flag("on")
assert _is_truthy_flag(2)
assert not _is_truthy_flag("off")
assert not _is_truthy_flag(0)
assert not _is_truthy_flag(None)
assert not _is_truthy_flag(False)
def test_bash_output_notes_clipped_lines_under_filter(session):
_start_background(session, "printf 'x%.0s' $(seq 1 5000); echo tail")
shell = _only_shell(session)
assert _wait_until(lambda: shell.status == "completed")
prepared = session._prepare_bash_output("r", {"id": shell.shell_id, "filter": "zzz"})
_cid, output = prepared["execute"](prepared)
assert "partially visible" in output
# ---------------------------------------------------------------------------
# task_agent scoping
# ---------------------------------------------------------------------------
def test_task_agent_shells_are_owner_scoped_and_reaped(session, monkeypatch):
seen = {}
def fake_run_agent(agent_turns, label="task", **kwargs):
out = _start_background(session, "sleep 60", call_id="sub-bash")
seen["start_output"] = out
agent_shells = session._background_shells.shells(owner="task-1")
seen["agent_shells"] = list(agent_shells)
seen["pid"] = agent_shells[0].pid if agent_shells else None
# The sub-agent's shell is invisible to the main scope.
seen["visible_to_parent"] = [s.shell_id for s in session._background_shells.shells()]
return "agent done"
monkeypatch.setattr(session, "_run_agent", fake_run_agent)
call_id, result = session._exec_task({"call_id": "task-1", "prompt": "start a server"})
assert "agent done" in result
assert seen["agent_shells"], "shell spawned inside the agent must carry its owner"
# Scope honesty in the start message: the sub-agent must not promise its
# caller a server that dies the moment it returns.
assert "terminated when the agent finishes" in seen["start_output"]
assert seen["visible_to_parent"] == []
assert seen["pid"] is not None
assert _wait_until(lambda: not _pid_alive(seen["pid"])), (
"sub-agent shells must be reaped when the agent finishes"
)
def test_task_agent_cannot_touch_parent_shells(session, monkeypatch):
_start_background(session, "sleep 60", call_id="parent-bash")
parent_shell = _only_shell(session)
seen = {}
def fake_run_agent(agent_turns, label="task", **kwargs):
prepared = session._prepare_bash_output("r", {"id": parent_shell.shell_id})
seen["read_output"] = prepared["execute"](prepared)[1]
prepared_kill = session._prepare_kill_shell("k", {"id": parent_shell.shell_id})
seen["kill_output"] = prepared_kill["execute"](prepared_kill)[1]
return "done"
monkeypatch.setattr(session, "_run_agent", fake_run_agent)
session._exec_task({"call_id": "task-1", "prompt": "snoop"})
assert "no background shell" in seen["read_output"].lower()
assert "no background shell" in seen["kill_output"].lower()
assert _pid_alive(parent_shell.pid), "agent must not be able to kill a parent shell"
def test_parent_scope_restored_after_task_agent(session, monkeypatch):
monkeypatch.setattr(session, "_run_agent", lambda *a, **k: "done")
session._exec_task({"call_id": "task-1", "prompt": "noop"})
output = _start_background(session, "sleep 30", call_id="after-task")
assert "bash_1" in output
assert _only_shell(session).owner is None
+165
View File
@@ -0,0 +1,165 @@
"""Regression tests for the bash tool hanging on a backgrounded child.
A bash command that backgrounds a long-lived process (``server &``,
``python -m http.server &``, any daemon) used to wedge the whole workstream
forever: the child inherits the tool's stdout/stderr pipe, so the foreground
read never hit EOF, and the timeout watchdog bailed the moment the tracked
``bash`` exited. ``_exec_bash`` now waits on the tracked process (not pipe
EOF) bounded by ``tool_timeout`` and kills the whole session group on exit, so
the call always returns and never leaks the background child.
"""
import threading
import time
from tests._proc_helpers import kill_pid as _kill_pid
from tests._proc_helpers import pid_alive as _pid_alive
from tests._session_helpers import NullUI, make_session
from turnstone.core.trajectory import EffectStatus
def _run_in_thread(fn, timeout):
"""Run ``fn`` in a daemon thread; return ``(finished, result)``."""
box = {}
def _target():
box["result"] = fn()
t = threading.Thread(target=_target, daemon=True)
t.start()
t.join(timeout)
return (not t.is_alive()), box.get("result")
def test_backgrounded_child_does_not_hang_and_is_reaped(tmp_path):
"""Foreground exits immediately but leaves ``sleep 60 &`` holding the pipe.
Old behaviour: infinite hang (EOF never arrives, watchdog bails once the
tracked bash exits). New behaviour: returns promptly and the background
child is reaped by the session-group kill.
"""
pidfile = str(tmp_path / "bg.pid")
# A generous tool_timeout proves the return comes from foreground-exit, not
# from the deadline firing.
session = make_session(tool_timeout=30)
command = f"sleep 60 & echo $! > {pidfile}; echo done"
bg_pid = None
try:
finished, result = _run_in_thread(
lambda: session._exec_bash({"call_id": "c1", "command": command}),
timeout=15,
)
assert finished, "_exec_bash hung on a backgrounded child"
assert result is not None
call_id, output = result
assert call_id == "c1"
assert "done" in output
# The backgrounded process must have been reaped by the group kill.
with open(pidfile) as f:
bg_pid = int(f.read().strip())
deadline = time.monotonic() + 5
while _pid_alive(bg_pid) and time.monotonic() < deadline:
time.sleep(0.05)
assert not _pid_alive(bg_pid), f"backgrounded child {bg_pid} leaked"
finally:
if bg_pid is not None:
_kill_pid(bg_pid)
def test_timeout_still_fires_with_backgrounded_child():
"""A silent foreground command plus a backgrounded child still hits the
deadline: the watchdog kills the whole group and the result reads UNKNOWN
(the ``unknown, never none`` timeout discipline)."""
session = make_session(tool_timeout=1)
command = "sleep 60 & sleep 60"
finished, result = _run_in_thread(
lambda: session._exec_bash({"call_id": "c1", "command": command}),
timeout=10,
)
assert finished, "_exec_bash did not return at its deadline"
assert result is not None
call_id, output = result
assert call_id == "c1"
assert "timed out" in output.lower()
assert "UNKNOWN" in output
assert session._tool_status.get("c1") is EffectStatus.UNKNOWN
def test_undecodable_output_is_preserved_not_swallowed():
"""Undecodable bytes on stdout must not silently vanish.
The drain's broad ``except (ValueError, OSError)`` would otherwise catch the
``UnicodeDecodeError`` (a ``ValueError``) and kill the thread before any line
was yielded dropping ALL output and reporting a clean success. ``Popen``
now decodes with ``errors="replace"`` so output always survives.
"""
session = make_session(tool_timeout=30)
# Valid lines bracketing a raw invalid-UTF-8 byte sequence.
command = r"printf 'before\n'; printf '\xff\xfe'; printf 'after\n'"
finished, result = _run_in_thread(
lambda: session._exec_bash({"call_id": "c1", "command": command}),
timeout=15,
)
assert finished
assert result is not None
_call_id, output = result
assert output != "(no output)"
assert "before" in output
assert "after" in output
def test_stdout_streams_to_ui_from_drain_thread():
"""stdout chunks are now emitted from the drain thread; they must still reach
``on_tool_output_chunk``."""
chunks: list[str] = []
class RecordingUI(NullUI):
def on_tool_output_chunk(self, call_id, chunk):
chunks.append(chunk)
session = make_session(tool_timeout=30, ui=RecordingUI())
finished, result = _run_in_thread(
lambda: session._exec_bash({"call_id": "c1", "command": "echo streamed-line"}),
timeout=15,
)
assert finished
assert any("streamed-line" in c for c in chunks)
def test_cancel_midbash_reports_unknown():
"""An external ``cancel()`` during a running bash unblocks the process-bounded
wait and reports UNKNOWN (unknown-never-none), not a clean result."""
session = make_session(tool_timeout=30)
def _cancel_soon():
time.sleep(0.5)
session.cancel()
threading.Thread(target=_cancel_soon, daemon=True).start()
finished, result = _run_in_thread(
lambda: session._exec_bash({"call_id": "c1", "command": "sleep 30"}),
timeout=15,
)
assert finished, "cancel did not unblock _exec_bash"
assert result is not None
_call_id, output = result
assert "cancelled" in output.lower()
assert session._tool_status.get("c1") is EffectStatus.UNKNOWN
def test_popen_failure_reports_cleanly(monkeypatch):
"""If ``Popen`` itself raises, the ``finally`` must not mask the real error
with ``UnboundLocalError`` ``proc`` is pre-bound to ``None``."""
from turnstone.core import session as session_mod
session = make_session(tool_timeout=30)
def _boom(*args, **kwargs):
raise OSError("cannot fork")
monkeypatch.setattr(session_mod.subprocess, "Popen", _boom)
call_id, output = session._exec_bash({"call_id": "c1", "command": "echo hi"})
assert call_id == "c1"
assert "cannot fork" in output
+71 -6
View File
@@ -71,7 +71,7 @@ def _make_judge(
session_provider=provider,
session_client=client,
session_model="test-model",
context_window=100_000,
session_capabilities=MagicMock(context_window=100_000),
)
@@ -892,15 +892,83 @@ class TestModelAliasResolution:
alias_provider: MagicMock,
alias_client: MagicMock,
underlying_model: str,
*,
capabilities: dict[str, Any] | None = None,
) -> MagicMock:
registry = MagicMock()
cfg = MagicMock()
cfg.context_window = 50_000
cfg.capabilities = capabilities if capabilities is not None else {}
registry.has_alias.side_effect = lambda a: a == alias
registry.resolve.return_value = (alias_client, underlying_model, cfg)
registry.get_provider.return_value = alias_provider
return registry
def test_alias_capabilities_merged_and_threaded_to_wire(self):
"""#823: a judge alias's model-definition ``capabilities`` are merged
onto the provider base AND passed to ``create_completion`` the same
contract as the session / utility / sub-agent lanes. Without threading,
operator overrides (effort passthrough, tool support) were silently
ignored on judge calls; deleting ``capabilities=self._capabilities`` from
the call site, or breaking the merge, must fail here."""
from turnstone.core.providers._protocol import ModelCapabilities
base = ModelCapabilities(supports_tools=True, effort_passthrough=False)
alias_provider = _make_mock_provider(response_content=_good_verdict_json())
alias_provider.get_capabilities = MagicMock(return_value=base)
registry = self._make_alias_registry(
"judge-mini",
alias_provider,
MagicMock(base_url="https://a/v1", api_key="k"),
"local-9b",
capabilities={"supports_tools": False, "effort_passthrough": True},
)
judge = IntentJudge(
config=JudgeConfig(enabled=True, model="judge-mini"),
session_provider=_make_mock_provider(),
session_client=MagicMock(base_url="https://s/v1", api_key="s"),
session_model="session-model",
session_capabilities=MagicMock(context_window=100_000),
model_registry=registry,
)
# Merged at construction: overrides applied, untouched fields survive.
assert judge._capabilities.supports_tools is False
assert judge._capabilities.effort_passthrough is True
assert judge._capabilities.context_window == base.context_window
# ...and the SAME merged object reaches the wire.
judge._evaluate_single(
_make_item(),
[{"role": "user", "content": "x"}],
cancel_event=None,
client=MagicMock(),
)
passed = alias_provider.create_completion.call_args.kwargs["capabilities"]
assert passed is judge._capabilities
def test_fallback_threads_session_capabilities_to_wire(self):
"""No judge alias → the judge inherits the session model AND the
session's resolved capabilities, threaded to ``create_completion``."""
from turnstone.core.providers._protocol import ModelCapabilities
sess_caps = ModelCapabilities(context_window=54_321, effort_passthrough=True)
provider = _make_mock_provider(response_content=_good_verdict_json())
judge = IntentJudge(
config=JudgeConfig(enabled=True, model=""), # no alias → fallback
session_provider=provider,
session_client=MagicMock(base_url="https://s/v1", api_key="s"),
session_model="session-model",
session_capabilities=sess_caps,
)
assert judge._capabilities is sess_caps
assert judge._judge_context_window == 54_321
judge._evaluate_single(
_make_item(),
[{"role": "user", "content": "x"}],
cancel_event=None,
client=MagicMock(),
)
assert provider.create_completion.call_args.kwargs["capabilities"] is sess_caps
def test_alias_uses_registry_provider_not_session_provider(self):
"""Judge with model=alias should resolve via registry — provider, client,
and concrete model name all come from the alias."""
@@ -932,7 +1000,6 @@ class TestModelAliasResolution:
session_provider=session_provider,
session_client=session_client,
session_model="session-default-model",
context_window=100_000,
model_registry=registry,
)
@@ -959,7 +1026,6 @@ class TestModelAliasResolution:
session_provider=_make_mock_provider(),
session_client=MagicMock(base_url="https://s/v1", api_key="s"),
session_model="session-model",
context_window=100_000,
model_registry=registry,
)
assert judge._judge_context_window == 50_000
@@ -980,7 +1046,7 @@ class TestModelAliasResolution:
session_provider=_make_mock_provider(),
session_client=MagicMock(base_url="http://s", api_key="s"),
session_model="session-model",
context_window=100_000,
session_capabilities=MagicMock(context_window=100_000),
model_registry=registry,
)
assert judge._judge_context_window == 100_000 # session window, not 0
@@ -1008,7 +1074,7 @@ class TestModelAliasResolution:
session_provider=session_provider,
session_client=session_client,
session_model="session-default-model",
context_window=100_000,
session_capabilities=MagicMock(context_window=100_000),
model_registry=registry,
)
@@ -1031,7 +1097,6 @@ class TestModelAliasResolution:
session_provider=session_provider,
session_client=session_client,
session_model="session-default-model",
context_window=100_000,
)
assert judge._provider is session_provider
+88
View File
@@ -18,6 +18,7 @@ from turnstone.core.lowering import (
CANCELLED_TOOL_RESULT,
_find_orphaned_tool_calls,
repair_wire_messages,
restore_provider_tool_ids,
sanitize_tool_call_arguments,
tool_args_preview,
wire_valid_arguments,
@@ -340,3 +341,90 @@ def test_pipeline_every_emitted_arguments_is_a_json_object() -> None:
for m in out:
for tc in m.get("tool_calls", []):
assert isinstance(json.loads(tc["function"]["arguments"]), dict)
# --------------------------------------------------------------------------- #
# restore_provider_tool_ids — the agent-wire id map (minted → provider-original).
#
# Sub-agent tool ids are minted "{parent}::r{run}s{step}::{provider_id}" for
# session-unique correlation (registry / DOM / recall). On the wire the pass
# maps them BACK to the provider's own ids from the per-run mint map, so the
# provider-native tool_use block (replayed verbatim, id never rewritten), the
# top-level tool_calls mirror, and the tool_result all agree on every request.
# --------------------------------------------------------------------------- #
def test_restore_ids_identity_on_empty_map() -> None:
msgs = [_assistant_calls(_call("task-1::r1s1::call_0", "{}")), _tool("task-1::r1s1::call_0")]
assert restore_provider_tool_ids(msgs, {}) is msgs
def test_restore_ids_identity_when_nothing_matches() -> None:
msgs = [_assistant_calls(_call("call_1", "{}")), _tool("call_1")]
assert restore_provider_tool_ids(msgs, {"task-1::r1s1::call_0": "call_0"}) is msgs
def test_restore_ids_maps_call_and_result_to_provider_original() -> None:
minted = "task-1::r1s1::toolu_01AB"
msgs = [_assistant_calls(_call(minted, "{}")), _tool(minted)]
out = restore_provider_tool_ids(msgs, {minted: "toolu_01AB"})
assert out[0]["tool_calls"][0]["id"] == "toolu_01AB"
assert out[1]["tool_call_id"] == "toolu_01AB" # pairing restored on both sides
# Copy-on-write: the input messages (the canonical-adjacent dicts) are unmutated.
assert msgs[0]["tool_calls"][0]["id"] == minted
assert msgs[1]["tool_call_id"] == minted
def test_restore_ids_recovers_originals_containing_the_mint_delimiter() -> None:
# Recovery is by MAP, not by string-splitting the mint suffix: a provider
# id that itself contains "::" round-trips exactly.
original = "srv::call::0"
minted = f"task-1::r1s1::{original}"
msgs = [_assistant_calls(_call(minted, "{}")), _tool(minted)]
out = restore_provider_tool_ids(msgs, {minted: original})
assert out[0]["tool_calls"][0]["id"] == original
assert out[1]["tool_call_id"] == original
def test_restore_ids_duplicate_originals_across_turns() -> None:
# A local server reissuing "call_0" every turn: two distinct minted ids
# both restore to "call_0" — the proven prior wire shape, each round
# pairing with its adjacent result.
m1, m2 = "task-1::r1s1::call_0", "task-1::r1s2::call_0"
msgs = [
_assistant_calls(_call(m1, "{}")),
_tool(m1),
_assistant_calls(_call(m2, "{}")),
_tool(m2),
]
out = restore_provider_tool_ids(msgs, {m1: "call_0", m2: "call_0"})
assert out[0]["tool_calls"][0]["id"] == "call_0"
assert out[1]["tool_call_id"] == "call_0"
assert out[2]["tool_calls"][0]["id"] == "call_0"
assert out[3]["tool_call_id"] == "call_0"
def test_restore_ids_leaves_unmapped_siblings_untouched() -> None:
minted = "task-1::r1s2::call_1"
msgs = [
_assistant_calls(_call("call_ok", "{}"), _call(minted, "{}")),
_tool("call_ok"),
_tool(minted),
]
out = restore_provider_tool_ids(msgs, {minted: "call_1"})
assert out[0]["tool_calls"][0]["id"] == "call_ok"
assert out[1]["tool_call_id"] == "call_ok"
assert out[0]["tool_calls"][1]["id"] == "call_1"
assert out[2]["tool_call_id"] == "call_1"
def test_restore_ids_skips_empty_and_non_string() -> None:
# Empty ids belong to repair_wire_messages' back-fill; non-strings are
# someone else's malformation — neither is this pass's to invent.
msgs = [
_assistant_calls(
{"id": "", "type": "function", "function": {"name": "b", "arguments": "{}"}}
),
{"role": "tool", "tool_call_id": None, "content": "x"},
]
out = restore_provider_tool_ids(msgs, {"task-1::r1s1::x": "x"})
assert out[0]["tool_calls"][0]["id"] == ""
assert out[1]["tool_call_id"] is None
+92
View File
@@ -98,6 +98,98 @@ class TestLenAndClear:
q = NudgeQueue()
assert q.clear() == 0
def test_clear_channels_drops_only_matching(self):
"""The abandoned-generation path drops advisory channels but must
preserve ``"any"``-channel external events (watch fires,
background-shell exits) in order."""
q = NudgeQueue()
q.enqueue("tool_error", "1", "tool")
q.enqueue("watch_triggered", "2", "any")
q.enqueue("correction", "3", "user")
q.enqueue("background_shell_exit", "4", "any")
assert q.clear_channels({"tool", "user"}) == 2
assert q.pending() == [("watch_triggered", "2"), ("background_shell_exit", "4")]
def test_clear_channels_empty_returns_zero(self):
q = NudgeQueue()
assert q.clear_channels({"tool", "user"}) == 0
def test_demote_channel_retags_preserving_order_and_metadata(self):
"""Cancel demotes 'any''quiet': same entries, same order, same
metadata/valid_until only wake eligibility changes."""
q = NudgeQueue()
q.enqueue("watch_triggered", "w", "any", metadata={"watch_name": "ci"})
q.enqueue("correction", "c", "user")
q.enqueue("background_shell_exit", "b", "any", valid_until=lambda: True)
assert q.demote_channel("any", "quiet") == 2
assert q.pending(channel="any") == []
assert q.pending(channel="quiet") == [
("watch_triggered", "w"),
("background_shell_exit", "b"),
]
# Metadata and valid_until ride the demotion; USER_DRAIN delivers.
from turnstone.core.nudge_queue import USER_DRAIN, WAKE_PENDING
assert not q.has_pending(WAKE_PENDING - {"user"}) # no 'any' left
drained = q.drain(USER_DRAIN)
assert [(t, x, m) for t, x, m in drained] == [
("watch_triggered", "w", {"watch_name": "ci"}),
("correction", "c", None),
("background_shell_exit", "b", None),
]
def test_cap_channel_none_sees_demoted_entries(self):
"""The watch soft cap counts across channels: entries a cancel
demoted to 'quiet' still occupy the budget, and drop-oldest evicts
the stalest regardless of channel."""
q = NudgeQueue()
q.enqueue("watch_triggered", "1", "any")
q.demote_channel("any", "quiet")
q.enqueue("watch_triggered", "2", "any")
assert q.cap_at_or_drop_oldest("watch_triggered", 2, channel=None) is True
assert q.pending() == [("watch_triggered", "2")]
def test_requeue_preserves_seq_for_chronology(self):
"""A failed delivery gives entries back with their ORIGINAL seq, so
a re-queued poll-4 still sorts before the poll-5 that arrived during
the failed attempt counters never run backwards."""
q = NudgeQueue()
q.enqueue("watch_triggered", "poll-4", "any")
(drained_entry,) = q.drain_entries({"any"})
q.enqueue("watch_triggered", "poll-5", "any") # newer event lands
q.requeue(drained_entry, channel="quiet")
entries = q.drain_entries({"any", "quiet"})
entries.sort(key=lambda e: e.seq)
assert [e.text for e in entries] == ["poll-4", "poll-5"]
def test_requeue_positions_by_seq_for_fifo_drains(self):
"""Positioned insertion: plain (unsorted) drains also see the
re-queued older entry first."""
q = NudgeQueue()
q.enqueue("a", "old", "quiet")
(old_entry,) = q.drain_entries({"quiet"})
q.enqueue("b", "new", "quiet")
q.requeue(old_entry)
assert [text for _t, text in q.pending()] == ["old", "new"]
def test_requeue_preserves_valid_until_and_metadata(self):
alive = {"value": True}
q = NudgeQueue()
q.enqueue("n", "x", "any", valid_until=lambda: alive["value"], metadata={"k": 1})
(entry,) = q.drain_entries({"any"})
q.requeue(entry, channel="quiet")
alive["value"] = False
assert q.drain({"quiet"}) == [] # predicate survived the round-trip
def test_quiet_is_outside_the_wake_gate(self):
from turnstone.core.nudge_queue import TOOL_DRAIN, USER_DRAIN, WAKE_PENDING
q = NudgeQueue()
q.enqueue("background_shell_exit", "b", "quiet")
assert not q.has_pending(WAKE_PENDING)
assert q.has_pending(USER_DRAIN)
assert q.has_pending(TOOL_DRAIN)
class TestDropOldestByType:
def test_drop_oldest_by_type_removes_earliest_match(self):
+76 -3
View File
@@ -15,6 +15,7 @@ from turnstone.core.output_guard_judge import (
OutputJudgeVerdict,
_extract_json,
)
from turnstone.core.providers._protocol import ModelCapabilities
def _make_provider(
@@ -68,6 +69,76 @@ def _make_judge(
return judge
class TestCapabilityThreading:
"""#823: the output-guard judge threads resolved capabilities to
create_completion, like every other create_completion caller."""
@staticmethod
def _recording_provider() -> tuple[Any, dict[str, Any]]:
captured: dict[str, Any] = {}
def _cc(**kwargs: Any) -> Any:
captured.update(kwargs)
result = MagicMock()
result.content = '{"risk_level": "none", "flags": []}'
return result
provider = MagicMock()
provider.provider_name = "openai"
provider.get_capabilities = MagicMock(
return_value=ModelCapabilities(context_window=200_000)
)
provider.create_completion = MagicMock(side_effect=_cc)
return provider, captured
def test_fallback_threads_session_capabilities(self) -> None:
provider, captured = self._recording_provider()
sess_caps = ModelCapabilities(context_window=40_000, effort_passthrough=True)
client = MagicMock(base_url="http://s", api_key="k")
judge = OutputGuardJudge(
config=JudgeConfig(output_guard_llm=True), # no alias → fallback
session_provider=provider,
session_client=client,
session_model="m",
session_capabilities=sess_caps,
)
judge._create_client = lambda: client # type: ignore[method-assign]
assert judge._capabilities is sess_caps
v = judge.evaluate("a small, safe output", func_name="bash", call_id="c1")
assert v.succeeded
assert captured["capabilities"] is sess_caps
def test_alias_merges_operator_capabilities(self) -> None:
provider, captured = self._recording_provider()
provider.get_capabilities = MagicMock(return_value=ModelCapabilities(supports_tools=True))
cfg = MagicMock()
cfg.context_window = 64_000
cfg.capabilities = {"supports_tools": False}
registry = MagicMock()
registry.has_alias.return_value = True
registry.resolve.return_value = (
MagicMock(base_url="http://a", api_key="k"),
"local-9b",
cfg,
)
registry.get_provider.return_value = provider
client = MagicMock(base_url="http://s", api_key="k")
judge = OutputGuardJudge(
config=JudgeConfig(output_guard_llm=True, output_guard_model="og"),
session_provider=_make_provider(),
session_client=client,
session_model="m",
session_capabilities=MagicMock(context_window=100_000),
model_registry=registry,
)
judge._create_client = lambda: client # type: ignore[method-assign]
assert judge._capabilities.supports_tools is False # operator override applied
v = judge.evaluate("a small, safe output", func_name="bash", call_id="c1")
assert v.succeeded
assert captured["capabilities"] is judge._capabilities
assert captured["capabilities"].supports_tools is False
class TestVerdictDataclass:
def test_default_verdict_with_no_error_succeeds(self) -> None:
# A default OutputJudgeVerdict has risk_level='none' and error=''
@@ -291,14 +362,16 @@ class TestOversizeGuard:
session_provider=provider,
session_client=MagicMock(base_url="http://test", api_key="k"),
session_model="test-model",
context_window=40_000, # the session's real window
# The session's real window rides in the resolved caps the caller
# passes; the guard must key off it, not provider.get_capabilities().
session_capabilities=MagicMock(context_window=40_000),
)
assert judge._judge_context_window == 40_000
def test_zero_window_coerced_away_on_both_paths(self) -> None:
"""A config.toml context_window=0 (present but unusable) must not zero
the guard: coerce to the session window (alias path) / the default."""
from turnstone.core.output_guard_judge import _DEFAULT_JUDGE_CONTEXT_WINDOW
from turnstone.core.judge import _DEFAULT_JUDGE_CONTEXT_WINDOW
# Alias path: ModelConfig.context_window == 0 → session window.
cfg = MagicMock()
@@ -313,7 +386,7 @@ class TestOversizeGuard:
session_client=MagicMock(base_url="http://s", api_key="s"),
session_model="m",
model_registry=registry,
context_window=64_000,
session_capabilities=MagicMock(context_window=64_000),
)
assert alias_judge._judge_context_window == 64_000
@@ -277,6 +277,41 @@ class TestConvertMessagesReasoningReplay:
types = [it.get("type") for it in items]
assert "reasoning" not in types
def test_agent_shaped_turn_pairs_reasoning_with_restored_call_ids(
self, provider: OpenAIResponsesProvider
) -> None:
# The sub-agent wire shape (native lane carried, minted ids already
# restored to the provider originals by the lowering map): the stored
# reasoning item rides immediately before the function_call rebuilt
# from the SAME original call id, and the function_call_output pairs
# to it — the ordering + id agreement the Responses API requires when
# replaying reasoning across an agent's own tool loop.
messages = [
{"role": "user", "content": "go"},
{
"role": "assistant",
"content": None,
"tool_calls": [{"id": "call_orig1", "function": {"name": "f", "arguments": "{}"}}],
"_provider_content": [
{"type": "reasoning", "id": "rs_1", "summary": [], "encrypted_content": "enc"},
{
"type": "function_call",
"call_id": "call_orig1",
"name": "f",
"arguments": "{}",
},
],
},
{"role": "tool", "tool_call_id": "call_orig1", "content": "out"},
]
_, items = provider._convert_messages(messages, replay_reasoning_to_model=True)
types = [it.get("type") for it in items]
assert types == ["message", "reasoning", "function_call", "function_call_output"]
assert items[1]["id"] == "rs_1"
assert items[1]["encrypted_content"] == "enc"
assert items[2]["call_id"] == "call_orig1"
assert items[3]["call_id"] == "call_orig1"
def test_no_reasoning_items_when_provider_content_lacks_reasoning(
self, provider: OpenAIResponsesProvider
) -> None:
+516 -25
View File
@@ -16,6 +16,7 @@ from turnstone.core.providers._openai_common import (
apply_cache_retention,
apply_temperature_and_effort,
apply_tool_search,
extract_usage,
format_citations,
lookup_openai_capabilities,
sanitize_messages,
@@ -1902,6 +1903,201 @@ class TestGoogleProviderFidelity:
assert len(cleaned) == 2
assert cleaned[0]["content"] == "hello"
def test_prepare_messages_swap_cannot_resurrect_malformed_arguments(self) -> None:
# The raw fidelity dicts carry the model's ORIGINAL arguments string;
# the sanitized top-level mirror is what the swap replaces. A raw
# dict whose arguments are malformed must be legalized during the
# swap (thought_signature and id untouched) — otherwise every replay
# resurrects the malformed string the upstream sanitize pass fixed.
from turnstone.core.providers._google import GoogleProvider
prov = GoogleProvider()
msgs = [
{
"role": "assistant",
"content": "",
"tool_calls": [
# Mirror already legalized upstream.
{"id": "c1", "type": "function", "function": {"name": "f", "arguments": "{}"}},
{
"id": "c2",
"type": "function",
"function": {"name": "g", "arguments": '{"ok": 1}'},
},
],
"_provider_content": [
{
"id": "c1",
"type": "function",
# Raw, unterminated — the model's original output.
"function": {"name": "f", "arguments": '{"path": "/tmp'},
"thought_signature": "sig123",
},
{
"id": "c2",
"type": "function",
"function": {"name": "g", "arguments": '{"ok": 1}'},
"thought_signature": "sig456",
},
],
},
{"role": "tool", "tool_call_id": "c1", "content": "ok"},
{"role": "tool", "tool_call_id": "c2", "content": "ok"},
]
cleaned = prov._prepare_messages(msgs)
tcs = cleaned[0]["tool_calls"]
assert tcs[0]["function"]["arguments"] == "{}" # legalized
assert tcs[0]["thought_signature"] == "sig123" # fidelity preserved
assert tcs[0]["id"] == "c1"
# The valid sibling passes through byte-identical.
assert tcs[1]["function"]["arguments"] == '{"ok": 1}'
assert tcs[1]["thought_signature"] == "sig456"
def test_prepare_messages_swap_serializes_dict_arguments(self) -> None:
# The internal-shape case the shared legalize helper handles: a raw
# fidelity dict whose arguments landed as an unserialized dict is
# json.dumps'd — content preserved, not collapsed to "{}".
from turnstone.core.providers._google import GoogleProvider
prov = GoogleProvider()
msgs = [
{
"role": "assistant",
"content": "",
"tool_calls": [
{"id": "c1", "type": "function", "function": {"name": "f", "arguments": "{}"}},
],
"_provider_content": [
{
"id": "c1",
"type": "function",
"function": {"name": "f", "arguments": {"path": "/tmp/x"}},
"thought_signature": "sig1",
},
],
},
{"role": "tool", "tool_call_id": "c1", "content": "ok"},
]
cleaned = prov._prepare_messages(msgs)
tc = cleaned[0]["tool_calls"][0]
assert json.loads(tc["function"]["arguments"]) == {"path": "/tmp/x"}
assert tc["thought_signature"] == "sig1"
def test_prepare_messages_blank_id_raw_row_keeps_sanitized_mirror(self) -> None:
# A historical fidelity row whose raw dict carries a blank id (saved
# before the capture-time blank-id gate existed): swapping it in
# would resurrect the blank id on every replay, so the swap is
# skipped and the sanitized mirror — with its back-filled id — stays.
from turnstone.core.providers._google import GoogleProvider
prov = GoogleProvider()
msgs = [
{
"role": "assistant",
"content": "",
"tool_calls": [
{
"id": "call_backfilled",
"type": "function",
"function": {"name": "f", "arguments": "{}"},
},
],
"_provider_content": [
{
"id": "",
"type": "function",
"function": {"name": "f", "arguments": "{}"},
"thought_signature": "sig",
},
],
},
{"role": "tool", "tool_call_id": "call_backfilled", "content": "ok"},
]
cleaned = prov._prepare_messages(msgs)
tc = cleaned[0]["tool_calls"][0]
assert tc["id"] == "call_backfilled" # mirror kept, raw lane not swapped
assert "thought_signature" not in tc
def test_prepare_messages_ignores_non_dict_provider_content_elements(self) -> None:
# A corrupted persisted lane with a non-dict element must not crash
# the request build.
from turnstone.core.providers._google import GoogleProvider
prov = GoogleProvider()
msgs = [
{
"role": "assistant",
"content": "x",
"_provider_content": ["garbage-string"],
},
]
cleaned = prov._prepare_messages(msgs)
assert cleaned[0]["content"] == "x"
assert "_provider_content" not in cleaned[0]
def test_prepare_messages_partial_lane_keeps_sanitized_mirror(self) -> None:
# A partially-corrupted lane (one valid raw dict + one garbage
# element) must not swap a SHORTER list over the mirror — that would
# drop a mirrored call whose tool result remains in history and
# orphan it. The sanitized mirror stays.
from turnstone.core.providers._google import GoogleProvider
prov = GoogleProvider()
msgs = [
{
"role": "assistant",
"content": "",
"tool_calls": [
{
"id": "call_A",
"type": "function",
"function": {"name": "f", "arguments": "{}"},
},
{
"id": "call_B",
"type": "function",
"function": {"name": "g", "arguments": "{}"},
},
],
"_provider_content": [
{
"id": "call_A",
"type": "function",
"function": {"name": "f", "arguments": "{}"},
"thought_signature": "sig",
},
"garbage-string",
],
},
{"role": "tool", "tool_call_id": "call_A", "content": "ok"},
{"role": "tool", "tool_call_id": "call_B", "content": "ok"},
]
cleaned = prov._prepare_messages(msgs)
ids = [tc["id"] for tc in cleaned[0]["tool_calls"]]
assert ids == ["call_A", "call_B"] # mirror kept — no orphaned call_B
def test_prepare_messages_swap_passes_non_dict_function_through(self) -> None:
# A degenerate fidelity block with function=None must pass through
# untouched (the prior behaviour), not raise.
from turnstone.core.providers._google import GoogleProvider
prov = GoogleProvider()
msgs = [
{
"role": "assistant",
"content": "x",
"tool_calls": [
{"id": "c1", "type": "function", "function": {"name": "f", "arguments": "{}"}},
],
"_provider_content": [
{"id": "c1", "type": "function", "function": None},
],
},
{"role": "tool", "tool_call_id": "c1", "content": "ok"},
]
cleaned = prov._prepare_messages(msgs)
assert cleaned[0]["tool_calls"][0]["function"] is None
def test_non_streaming_captures_provider_blocks(self) -> None:
from turnstone.core.providers._google import GoogleProvider
@@ -2236,15 +2432,14 @@ class TestOpenAIParameterGating:
assert "max" in lookup_openai_capabilities("gpt-5.6-sol").reasoning_effort_values
assert "max" in lookup_openai_capabilities("gpt-5.6-2026-07-09").reasoning_effort_values
def test_gpt56_terra_luna_max_snaps_to_xhigh_ceiling(self) -> None:
"""GPT-5.6 Terra and Luna have no "max" (Sol-only); the knob's "max"
snaps DOWN to the declared "xhigh" ceiling rather than being dropped."""
def test_gpt56_terra_luna_support_max_effort(self) -> None:
"""Every GPT-5.6 tier accepts the documented "max" effort."""
for tier in ("gpt-5.6-terra", "gpt-5.6-luna"):
caps = lookup_openai_capabilities(tier)
assert "max" not in caps.reasoning_effort_values, tier
assert "max" in caps.reasoning_effort_values, tier
kwargs: dict[str, Any] = {}
apply_temperature_and_effort(kwargs, caps, temperature=0.7, reasoning_effort="max")
assert kwargs["reasoning_effort"] == "xhigh", tier
assert kwargs["reasoning_effort"] == "max", tier
class TestAnthropicOrphanedToolUse:
@@ -3174,6 +3369,80 @@ class TestAnthropicProviderBlocks:
assert assistant_msg["role"] == "assistant"
assert assistant_msg["content"] == [{"type": "text", "text": "Hi there"}]
def test_agent_native_lane_with_restore_map_is_wire_consistent(self) -> None:
"""The sub-agent wire shape: an assistant Turn carrying the provider-
native lane, its minted tool id restored to the provider original by
the lowering map. The native blocks replay verbatim (thinking +
signature untouched) and the native tool_use id, the top-level
mirror, and the tool_result all agree."""
from turnstone.core.lowering import restore_provider_tool_ids
from turnstone.core.trajectory import (
ProviderNative,
ToolCall,
Turn,
dicts_from_turns,
)
thinking = {"type": "thinking", "thinking": "look first", "signature": "sig_1"}
tool_use = {"type": "tool_use", "id": "toolu_01X", "name": "f", "input": {}}
minted = "task-1::r1s1::toolu_01X"
turns = [
Turn.user("go"),
Turn.assistant(
"using f",
tool_calls=(ToolCall(id=minted, name="f", arguments="{}"),),
native=ProviderNative(
producer="anthropic",
blocks=(thinking, {"type": "text", "text": "using f"}, tool_use),
),
),
Turn.tool(minted, "out"),
]
wire = restore_provider_tool_ids(dicts_from_turns(turns), {minted: "toolu_01X"})
_, converted = self.provider._convert_messages(wire, replay_reasoning_to_model=True)
assistant = converted[1]
assert [b["type"] for b in assistant["content"]] == ["thinking", "text", "tool_use"]
assert assistant["content"][0]["signature"] == "sig_1"
assert assistant["content"][2]["id"] == "toolu_01X"
tool_results = [b for b in converted[2]["content"] if b.get("type") == "tool_result"]
assert tool_results and tool_results[0]["tool_use_id"] == "toolu_01X"
def test_agent_native_lane_without_restore_map_orphans_the_result(self) -> None:
"""Documents why the id map is a PREREQUISITE of carrying the native
lane, not hygiene: without it the tool_result arrives with the minted
id, matches no native tool_use, and the converter drops it as an
orphan leaving an unanswered tool_use on the wire (a provider
rejection)."""
from turnstone.core.trajectory import (
ProviderNative,
ToolCall,
Turn,
dicts_from_turns,
)
tool_use = {"type": "tool_use", "id": "toolu_01X", "name": "f", "input": {}}
minted = "task-1::r1s1::toolu_01X"
turns = [
Turn.user("go"),
Turn.assistant(
"",
tool_calls=(ToolCall(id=minted, name="f", arguments="{}"),),
native=ProviderNative(producer="anthropic", blocks=(tool_use,)),
),
Turn.tool(minted, "out"),
]
_, converted = self.provider._convert_messages(
dicts_from_turns(turns), replay_reasoning_to_model=True
)
all_results = [
b
for m in converted
if isinstance(m.get("content"), list)
for b in m["content"]
if isinstance(b, dict) and b.get("type") == "tool_result"
]
assert all_results == []
def test_block_to_dict_with_model_dump(self) -> None:
"""_block_to_dict uses model_dump(exclude_none=True) when available."""
from turnstone.core.providers._anthropic import _block_to_dict
@@ -3495,6 +3764,31 @@ class TestModelCapabilitiesToolSearch:
caps = ModelCapabilities()
assert caps.supports_tool_search is False
def test_public_positional_prefix_remains_stable(self) -> None:
"""New optional fields must not shift the exported constructor's existing slots."""
caps = ModelCapabilities(
100000,
10000,
False,
False,
False,
"max_tokens",
"manual",
"thinking",
"reasoning_effort",
True,
("low",),
("low",),
"low",
True,
True,
True,
True,
)
assert caps.supports_web_search is True
assert caps.supports_tool_search is True
assert caps.supports_vision is True
class TestMidConversationSystemCapability:
"""supports_mid_conversation_system — NextOpus (claude-opus-4-8) only."""
@@ -3943,8 +4237,50 @@ class TestOpenAIPromptCaching:
def setup_method(self) -> None:
self.provider = OpenAIProvider()
def test_cache_retention_set_for_gpt5(self) -> None:
"""GPT-5.x models get prompt_cache_retention=24h."""
@pytest.mark.parametrize("model", ("gpt-5.5-local-lora", "gpt-5.6-local-lora"))
def test_chat_compat_streaming_omits_commercial_cache_params(self, model: str) -> None:
"""A local model name must not activate commercial OpenAI cache controls."""
client = MagicMock()
client.chat.completions.create.return_value = iter(())
list(
self.provider.create_streaming(
client=client,
model=model,
messages=[{"role": "user", "content": "hi"}],
)
)
sent = client.chat.completions.create.call_args.kwargs
assert "prompt_cache_retention" not in sent
assert "prompt_cache_options" not in sent
@pytest.mark.parametrize("model", ("gpt-5.5-local-lora", "gpt-5.6-local-lora"))
def test_chat_compat_completion_omits_commercial_cache_params(self, model: str) -> None:
"""The non-streaming local lane has the same cache-parameter isolation."""
response = MagicMock()
response.choices = [
MagicMock(
message=MagicMock(content="hello", tool_calls=None, annotations=None),
finish_reason="stop",
)
]
response.usage = None
client = MagicMock()
client.chat.completions.create.return_value = response
self.provider.create_completion(
client=client,
model=model,
messages=[{"role": "user", "content": "hi"}],
)
sent = client.chat.completions.create.call_args.kwargs
assert "prompt_cache_retention" not in sent
assert "prompt_cache_options" not in sent
def test_cache_retention_set_for_pre_gpt56_models(self) -> None:
"""Pre-5.6 GPT-5 models retain the legacy 24-hour cache policy."""
for model in (
"gpt-5",
"gpt-5.1",
@@ -3953,16 +4289,21 @@ class TestOpenAIPromptCaching:
"gpt-5.4-pro",
"gpt-5.5",
"gpt-5.5-pro",
"gpt-5.6",
"gpt-5.6-sol",
"gpt-5.6-terra",
"gpt-5.6-luna",
"gpt-5-mini",
"gpt-5-pro",
):
kwargs: dict[str, Any] = {}
apply_cache_retention(kwargs, model)
assert kwargs.get("prompt_cache_retention") == "24h", f"Failed for {model}"
assert "prompt_cache_options" not in kwargs
def test_gpt56_uses_prompt_cache_options(self) -> None:
"""GPT-5.6 uses the replacement cache API introduced in SDK 2.45."""
for model in ("gpt-5.6", "gpt-5.6-sol", "gpt-5.6-terra", "gpt-5.6-luna"):
kwargs: dict[str, Any] = {}
apply_cache_retention(kwargs, model)
assert kwargs.get("prompt_cache_options") == {"ttl": "30m"}, model
assert "prompt_cache_retention" not in kwargs
def test_cache_retention_not_set_for_non_gpt5(self) -> None:
"""Non-GPT-5 models do not get cache retention."""
@@ -3970,6 +4311,38 @@ class TestOpenAIPromptCaching:
kwargs: dict[str, Any] = {}
apply_cache_retention(kwargs, model)
assert "prompt_cache_retention" not in kwargs, f"Unexpected retention for {model}"
assert "prompt_cache_options" not in kwargs, f"Unexpected options for {model}"
def test_cache_write_tokens_from_responses_usage(self) -> None:
"""GPT-5.6 cache writes flow into normalized usage accounting."""
usage = MagicMock()
usage.prompt_tokens = None
usage.input_tokens = 100
usage.completion_tokens = None
usage.output_tokens = 20
usage.total_tokens = 120
usage.prompt_tokens_details = None
usage.input_tokens_details = MagicMock(cached_tokens=30, cache_write_tokens=70)
normalized = extract_usage(usage)
assert normalized is not None
assert normalized.cache_read_tokens == 30
assert normalized.cache_creation_tokens == 70
def test_cache_write_tokens_from_chat_usage(self) -> None:
"""The Chat Completions usage shape reports the same cache-write metric."""
usage = MagicMock()
usage.prompt_tokens = 100
usage.completion_tokens = 20
usage.total_tokens = 120
usage.prompt_tokens_details = MagicMock(cached_tokens=30, cache_write_tokens=70)
normalized = extract_usage(usage)
assert normalized is not None
assert normalized.cache_read_tokens == 30
assert normalized.cache_creation_tokens == 70
def test_streaming_cached_tokens_from_usage(self) -> None:
"""Streaming usage extracts cached_tokens from prompt_tokens_details."""
@@ -4131,6 +4504,78 @@ class TestOpenAIResponsesProvider:
assert caps.supports_tool_search is True
class TestOpenAIChatReasoningCapture:
"""Non-streaming ``create_completion`` surfaces the Chat-Completions
lane's non-canonical reasoning (vLLM ``--reasoning-parser``, llama.cpp
``reasoning_format``) as ``CompletionResult.reasoning`` the twin of the
streaming path's ``reasoning_delta`` extraction, same attribute pair and
precedence."""
@staticmethod
def _client(*, reasoning: Any = None, reasoning_content: Any = None) -> MagicMock:
msg = MagicMock()
msg.content = "ok"
msg.tool_calls = None
msg.annotations = None
msg.reasoning = reasoning
msg.reasoning_content = reasoning_content
choice = MagicMock()
choice.message = msg
choice.finish_reason = "stop"
resp = MagicMock()
resp.choices = [choice]
resp.usage = None
client = MagicMock()
client.chat.completions.create.return_value = resp
return client
def _complete(self, client: MagicMock):
provider = OpenAIChatCompletionsProvider()
return provider.create_completion(
client=client, model="m", messages=[{"role": "user", "content": "hi"}]
)
def test_reasoning_content_captured(self) -> None:
result = self._complete(self._client(reasoning_content="thought text"))
assert result.reasoning == "thought text"
def test_reasoning_attribute_takes_precedence(self) -> None:
result = self._complete(self._client(reasoning="direct", reasoning_content="parsed"))
assert result.reasoning == "direct"
def test_absent_reasoning_is_empty(self) -> None:
result = self._complete(self._client())
assert result.reasoning == ""
def test_non_string_reasoning_collapses_to_empty(self) -> None:
# A server surfacing a structured reasoning object (not text) must not
# leak a non-str into the result.
result = self._complete(self._client(reasoning={"odd": True}))
assert result.reasoning == ""
def test_structured_reasoning_does_not_shadow_reasoning_content(self) -> None:
# A truthy non-string in ``reasoning`` must not shadow valid text in
# ``reasoning_content`` — the first non-empty STRING wins.
result = self._complete(
self._client(reasoning={"content": "structured"}, reasoning_content="parsed text")
)
assert result.reasoning == "parsed text"
def test_streaming_delta_shares_the_same_guard(self) -> None:
# The streaming twin: a structured object in ``reasoning`` must not
# leak into reasoning_delta (it would TypeError the session's
# ``"".join`` accumulator) nor shadow the parsed string.
provider = OpenAIChatCompletionsProvider()
chunk = _openai_stream_chunk(
reasoning={"content": "structured"}, # type: ignore[arg-type] — the hostile input under test
reasoning_content="parsed text",
finish_reason="stop",
)
chunks = list(provider._iter_stream(iter([chunk])))
assert any(c.reasoning_delta == "parsed text" for c in chunks)
assert all(isinstance(c.reasoning_delta, str) for c in chunks)
class TestResponsesMessageConversion:
"""Tests for _convert_messages — Chat Completions format to Responses API."""
@@ -4393,8 +4838,7 @@ class TestResponsesParamBuilding:
assert kwargs["reasoning"] == {"effort": "high", "mode": "pro"}
def test_pro_mode_rejected_when_unsupported(self) -> None:
"""A pro reasoning_mode on Terra/Luna (supports_pro_mode False) is
dropped effort still rides, mode does not."""
"""A pro mode on a model without reasoning-mode support is dropped."""
caps = ModelCapabilities(
supports_pro_mode=False,
reasoning_mode="pro",
@@ -4410,6 +4854,16 @@ class TestResponsesParamBuilding:
kwargs = self._build(caps, reasoning_effort="medium")
assert kwargs["reasoning"] == {"mode": "pro"}
def test_standard_reasoning_mode_is_accepted(self) -> None:
"""The SDK's explicit standard mode is valid even though omission is equivalent."""
caps = ModelCapabilities(
supports_pro_mode=True,
reasoning_mode="standard",
reasoning_effort_values=("low", "medium", "high"),
)
kwargs = self._build(caps, reasoning_effort="high")
assert kwargs["reasoning"] == {"effort": "high", "mode": "standard"}
def test_verbosity_unknown_value_dropped(self) -> None:
"""A verbosity outside {low,medium,high} is dropped, not sent — an
operator typo must not 400 every request."""
@@ -4426,9 +4880,23 @@ class TestResponsesParamBuilding:
kwargs = self._build(caps, reasoning_effort="high")
assert kwargs["reasoning"] == {"effort": "high"}
def test_gpt56_terra_max_snaps_to_xhigh_on_responses_wire(self) -> None:
"""Terra's knob "max" snaps to the xhigh ceiling on the ACTUAL
Responses wire path (_build_kwargs), not only the shared resolver."""
def test_verbosity_non_string_value_dropped(self) -> None:
"""Malformed operator JSON must not crash request construction."""
kwargs = self._build(ModelCapabilities(supports_verbosity=True, verbosity=["low"]))
assert "text" not in kwargs
def test_pro_mode_non_string_value_dropped(self) -> None:
"""Malformed operator JSON must not crash request construction."""
caps = ModelCapabilities(
supports_pro_mode=True,
reasoning_mode=["pro"],
reasoning_effort_values=("low", "medium", "high"),
)
kwargs = self._build(caps, reasoning_effort="high")
assert kwargs["reasoning"] == {"effort": "high"}
def test_gpt56_terra_max_reaches_responses_wire(self) -> None:
"""Terra sends the documented max effort on the actual Responses path."""
kwargs = self.provider._build_kwargs(
model="gpt-5.6-terra",
messages=[{"role": "user", "content": "Hi"}],
@@ -4438,19 +4906,14 @@ class TestResponsesParamBuilding:
reasoning_effort="max",
deferred_names=None,
)
assert kwargs["reasoning"] == {"effort": "xhigh"}
assert kwargs["reasoning"] == {"effort": "max"}
def test_gpt56_verbosity_and_pro_flags(self) -> None:
"""The static rows carry the right capability flags: verbosity on all
three tiers, pro mode on Sol/alias only."""
sol = lookup_openai_capabilities("gpt-5.6-sol")
assert sol.supports_verbosity is True
assert sol.supports_pro_mode is True
assert lookup_openai_capabilities("gpt-5.6").supports_pro_mode is True
for tier in ("gpt-5.6-terra", "gpt-5.6-luna"):
"""Every GPT-5.6 tier supports verbosity and pro reasoning mode."""
for tier in ("gpt-5.6", "gpt-5.6-sol", "gpt-5.6-terra", "gpt-5.6-luna"):
caps = lookup_openai_capabilities(tier)
assert caps.supports_verbosity is True
assert caps.supports_pro_mode is False
assert caps.supports_pro_mode is True
def _kwargs_with(self, tools: list[dict[str, Any]], caps: ModelCapabilities) -> dict[str, Any]:
return self.provider._build_kwargs(
@@ -4503,6 +4966,34 @@ class TestResponsesParamBuilding:
)
assert kwargs["prompt_cache_retention"] == "24h"
def test_compat_responses_omits_commercial_cache_params(self) -> None:
provider = type(self.provider)(compat=True)
for model in ("gpt-5.5-local-lora", "gpt-5.6-local-lora"):
kwargs = provider._build_kwargs(
model=model,
messages=[{"role": "user", "content": "Hi"}],
tools=None,
max_tokens=4096,
temperature=0.5,
reasoning_effort="medium",
deferred_names=None,
)
assert "prompt_cache_retention" not in kwargs, model
assert "prompt_cache_options" not in kwargs, model
def test_cache_options_for_gpt56(self) -> None:
kwargs = self.provider._build_kwargs(
model="gpt-5.6-sol",
messages=[{"role": "user", "content": "Hi"}],
tools=None,
max_tokens=4096,
temperature=0.5,
reasoning_effort="medium",
deferred_names=None,
)
assert kwargs["prompt_cache_options"] == {"ttl": "30m"}
assert "prompt_cache_retention" not in kwargs
def test_instructions_from_system_messages(self) -> None:
kwargs = self.provider._build_kwargs(
model="gpt-5.4",
+600 -3
View File
@@ -1002,6 +1002,27 @@ class TestEvaluateIntentProjection:
assert fa["edits"][0]["near_line"] == 42
assert fa["replace_all"] is False
# -- bash: backgrounding is part of the intent (#817) -------------------
def test_bash_background_projects_run_in_background(self) -> None:
"""The judge must know a bash command will run detached — a
backgrounded server/miner is a different intent than a bounded run.
Built via the real preparer so the prepared item can't silently drop
the flag before the projection reads it."""
session = _make_session()
item = session._prepare_bash(
"c1", {"command": "python -m http.server 8000", "run_in_background": True}
)
fa = _project_func_args(item)
assert fa["run_in_background"] is True
assert fa["command"] == "python -m http.server 8000"
def test_bash_foreground_projects_run_in_background_false(self) -> None:
session = _make_session()
item = session._prepare_bash("c1", {"command": "echo hi"})
fa = _project_func_args(item)
assert fa["run_in_background"] is False
# -- skills: the dead-assignment bug -----------------------------------
def test_skills_create_projection_is_not_empty(self) -> None:
@@ -2429,9 +2450,582 @@ class TestAgentChildRegistration:
parent_call_id="task-1",
)
# Sub-agent tool ids are namespaced by the parent so the UI registry
# can't collide across concurrent task agents (local sequential ids).
session.ui.note_agent_child.assert_called_once_with("task-1::call_1", "task-1")
# Sub-agent tool ids are minted ``{parent}::r{run}s{step}::{provider_id}``
# so the UI registry can't collide across concurrent task agents, across
# turns within one agent (local sequential ids like "call_0"), or across
# runs whose PARENT id was itself reused.
session.ui.note_agent_child.assert_called_once_with("task-1::r1s1::call_1", "task-1")
def test_cross_turn_reused_provider_ids_stay_distinct(self):
# A local provider reuses "call_0" verbatim every response. The minted
# id carries a per-agent step sequence, so the registry, the wire, the
# recall projection, and the cancel ledger all see two DISTINCT calls.
# Pre-mint both mapped to "task-1::call_0": the live card collapsed the
# rows (bug-3) while FIFO recall kept them apart — the two disagreed on
# identical input.
from turnstone.core.providers._openai_chat import OpenAIChatCompletionsProvider
session = _make_session()
session._provider = OpenAIChatCompletionsProvider()
session.ui.note_agent_child = MagicMock()
call_count = [0]
def fake_create(**_kwargs):
call_count[0] += 1
resp = MagicMock()
choice = MagicMock()
if call_count[0] <= 2:
choice.finish_reason = "tool_calls"
tc = MagicMock()
tc.id = "call_0" # reused verbatim across turns
tc.function.name = "read_file"
tc.function.arguments = f'{{"path": "/tmp/f{call_count[0]}"}}'
choice.message.tool_calls = [tc]
choice.message.content = None
else:
choice.finish_reason = "stop"
choice.message.tool_calls = None
choice.message.content = "done"
resp.choices = [choice]
resp.usage = MagicMock(prompt_tokens=10, completion_tokens=5)
return resp
session.client.chat.completions.create = fake_create
def fake_prepare(tc_dict, **_kwargs):
n = call_count[0]
return {
"call_id": tc_dict["id"],
"func_name": "read_file",
"needs_approval": False,
"execute": lambda p, n=n: (p["call_id"], f"contents-{n}"),
}
agent_turns = [Turn.user("x")]
with patch.object(session, "_prepare_tool", side_effect=fake_prepare):
session._run_agent(
agent_turns,
tools=[{"type": "function", "function": {"name": "read_file"}}],
label="task",
parent_call_id="task-1",
)
# Registry: two registrations, distinct minted ids, same parent.
assert [c.args for c in session.ui.note_agent_child.call_args_list] == [
("task-1::r1s1::call_0", "task-1"),
("task-1::r1s2::call_0", "task-1"),
]
# Recall projection: two steps, each paired to its OWN result.
steps = ChatSession._project_agent_steps(agent_turns)
assert [s["id"] for s in steps] == ["task-1::r1s1::call_0", "task-1::r1s2::call_0"]
assert [s["output"] for s in steps] == ["contents-1", "contents-2"]
# Cancel ledger agrees: both calls answered, no in-flight gap.
issued, first_gap = ChatSession._cancel_ledger(agent_turns)
assert issued == [("read_file", True), ("read_file", True)]
assert first_gap is None
@staticmethod
def _reusing_provider(session, tool_turns: int = 1):
"""Fake create() reissuing id "call_0" for ``tool_turns`` turns, then
stopping the local-server id-reuse shape. Returns the counter."""
call_count = [0]
def fake_create(**_kwargs):
call_count[0] += 1
resp = MagicMock()
choice = MagicMock()
if call_count[0] <= tool_turns:
choice.finish_reason = "tool_calls"
tc = MagicMock()
tc.id = "call_0"
tc.function.name = "read_file"
tc.function.arguments = '{"path": "/tmp/x"}'
choice.message.tool_calls = [tc]
choice.message.content = None
else:
choice.finish_reason = "stop"
choice.message.tool_calls = None
choice.message.content = "done"
resp.choices = [choice]
resp.usage = MagicMock(prompt_tokens=10, completion_tokens=5)
return resp
session.client.chat.completions.create = fake_create
return call_count
def test_parent_id_reuse_across_runs_mints_distinct_child_ids(self):
# A local provider reuses "call_0" for the PARENT task_agent call too:
# two sequential runs share parent_call_id "call_0". The session-level
# run counter keeps their minted CHILD ids distinct — with only the
# per-run step seq (the intermediate fix, before the run counter) both
# runs minted "call_0::s1::call_0" and the second agent's sub-tool
# steps grafted onto the first agent's DOM rows.
#
# SCOPE: this fixes child (sub-tool) ids only. The parent CARD still
# keys on the raw reused parent id ("call_0") — stash_agent_trajectory,
# _tool_status, the card's own data-call-id row — so two runs with the
# same parent id still alias at the card level. Parent ids are
# main-loop ids; de-colliding them is the main-loop id-hygiene
# follow-up, not this change.
from turnstone.core.providers._openai_chat import OpenAIChatCompletionsProvider
session = _make_session()
session._provider = OpenAIChatCompletionsProvider()
session.ui.note_agent_child = MagicMock()
def fake_prepare(tc_dict, **_kwargs):
return {
"call_id": tc_dict["id"],
"func_name": "read_file",
"needs_approval": False,
"execute": lambda p: (p["call_id"], "contents"),
}
minted: list[str] = []
for _run in range(2):
self._reusing_provider(session)
with patch.object(session, "_prepare_tool", side_effect=fake_prepare):
session._run_agent(
[Turn.user("x")],
tools=[{"type": "function", "function": {"name": "read_file"}}],
label="task",
parent_call_id="call_0",
)
minted.append(session.ui.note_agent_child.call_args.args[0])
assert minted == ["call_0::r1s1::call_0", "call_0::r2s1::call_0"]
assert len(set(minted)) == 2
def test_agent_wire_restores_provider_ids_and_sanitizes_args(self):
# The agent seam bypasses the main-loop wire prep and builds its own
# history, so it runs its own validity passes. Drive one tool turn
# whose call carries a minted "::" id (mapped back to the provider's
# own id on the wire) and malformed non-object arguments (a strict
# renderer json.loads and 400s them), then assert the REPLAY request
# the second _api_call sends carries the PROVIDER-ORIGINAL id on both
# the call and its result, and object-shaped arguments. The internal
# id keeps the minted "::" form.
import json as _json
from turnstone.core.providers._openai_chat import OpenAIChatCompletionsProvider
session = _make_session()
session._provider = OpenAIChatCompletionsProvider()
session.ui.note_agent_child = MagicMock()
seen_messages: list[list[dict]] = []
call_count = [0]
def fake_create(**kwargs):
seen_messages.append(kwargs.get("messages") or [])
call_count[0] += 1
resp = MagicMock()
choice = MagicMock()
if call_count[0] == 1:
choice.finish_reason = "tool_calls"
tc = MagicMock()
tc.id = "call_0"
tc.function.name = "read_file"
# Malformed: unterminated JSON with a non-"length" finish
# reason — the sanitize pass's reason to exist.
tc.function.arguments = '{"path": "/tmp/x"'
choice.message.tool_calls = [tc]
choice.message.content = None
else:
choice.finish_reason = "stop"
choice.message.tool_calls = None
choice.message.content = "done"
resp.choices = [choice]
resp.usage = MagicMock(prompt_tokens=10, completion_tokens=5)
return resp
session.client.chat.completions.create = fake_create
def fake_prepare(tc_dict, **_kwargs):
return {
"call_id": tc_dict["id"],
"func_name": "read_file",
"needs_approval": False,
"execute": lambda p: (p["call_id"], "contents"),
}
with patch.object(session, "_prepare_tool", side_effect=fake_prepare):
session._run_agent(
[Turn.user("x")],
tools=[{"type": "function", "function": {"name": "read_file"}}],
label="task",
parent_call_id="task-1",
)
# Internal id (registry) keeps the minted "::" form.
internal = session.ui.note_agent_child.call_args.args[0]
assert internal == "task-1::r1s1::call_0"
# The SECOND request replays the tool turn: the wire carries the
# provider's own id, consistent between the call and its result (the
# shape the provider-native tool_use block also holds, so a native
# replay and a rebuild agree); arguments are legalized to a JSON
# object.
replay = seen_messages[1]
wire_calls = [tc for m in replay if m.get("tool_calls") for tc in m["tool_calls"]]
wire_results = [m for m in replay if m.get("role") == "tool"]
assert wire_calls and wire_results
assert wire_calls[0]["id"] == "call_0"
assert wire_results[0]["tool_call_id"] == "call_0"
assert isinstance(_json.loads(wire_calls[0]["function"]["arguments"]), dict)
def test_agent_carries_native_lane_and_replays_thinking_anthropic(self):
# The load-bearing fidelity pin: a thinking-model agent's SECOND
# request must carry the prior assistant turn's native lane verbatim
# — thinking block and signature untouched — with the provider's own
# tool_use id agreeing across the native block, the restored
# top-level mirror, and the tool_result. Pre-native-lane, the seam
# rebuilt the turn from content + tool_calls and the model re-reasoned
# from scratch every tool turn (and commercial Anthropic rejects a
# thinking-enabled tool_use turn without its thinking block).
from turnstone.core.providers._anthropic import AnthropicProvider
class _Block:
def __init__(self, **d):
self._d = d
for k, v in d.items():
setattr(self, k, v)
def model_dump(self, **_kw):
return dict(self._d)
session = _make_session()
session._provider = AnthropicProvider()
session.ui.note_agent_child = MagicMock()
seen: list[dict] = []
call_count = [0]
def fake_stream(**kwargs):
seen.append(kwargs)
call_count[0] += 1
resp = MagicMock()
if call_count[0] == 1:
resp.content = [
_Block(type="thinking", thinking="check the file first", signature="sig_v1"),
_Block(type="text", text="reading"),
_Block(type="tool_use", id="toolu_01AB", name="read_file", input={"path": "x"}),
]
resp.stop_reason = "tool_use"
else:
resp.content = [_Block(type="text", text="done")]
resp.stop_reason = "end_turn"
resp.usage = None
mgr = MagicMock()
mgr.__enter__ = MagicMock(
return_value=MagicMock(get_final_message=MagicMock(return_value=resp))
)
mgr.__exit__ = MagicMock(return_value=False)
return mgr
session.client.messages.stream = fake_stream
def fake_prepare(tc_dict, **_kwargs):
return {
"call_id": tc_dict["id"],
"func_name": "read_file",
"needs_approval": False,
"execute": lambda p: (p["call_id"], "contents"),
}
with (
patch.object(session, "_prepare_tool", side_effect=fake_prepare),
patch.object(session, "_resolve_replay_reasoning_to_model", return_value=True),
):
session._run_agent(
[Turn.user("x")],
tools=[{"type": "function", "function": {"name": "read_file", "parameters": {}}}],
label="task",
parent_call_id="task-1",
)
# Internal key stays minted — the nesting registry saw the "::" id.
assert session.ui.note_agent_child.call_args.args[0] == "task-1::r1s1::toolu_01AB"
# Second request: the assistant wire turn IS the native lane.
replay = seen[1]["messages"]
assistant = next(
m for m in replay if m["role"] == "assistant" and isinstance(m.get("content"), list)
)
kinds = [b.get("type") for b in assistant["content"]]
assert kinds == ["thinking", "text", "tool_use"]
assert assistant["content"][0]["thinking"] == "check the file first"
assert assistant["content"][0]["signature"] == "sig_v1" # byte-untouched
assert assistant["content"][2]["id"] == "toolu_01AB" # provider-original
tool_results = [
b
for m in replay
if m["role"] == "user" and isinstance(m.get("content"), list)
for b in m["content"]
if isinstance(b, dict) and b.get("type") == "tool_result"
]
assert tool_results and tool_results[0]["tool_use_id"] == "toolu_01AB"
def test_agent_blank_provider_id_skips_native_lane(self):
# A server that leaves a tool-call id blank gets a uuid back-fill in
# the tool_calls mirror (_ensure_tool_call_ids) — but the native
# tool_use block keeps the blank id verbatim. Carrying the lane for
# that turn would replay a native tool_use whose id matches no
# tool_result (Anthropic orphans the result and 400s). The shared
# builder must drop the whole Messages-shaped lane for exactly that
# turn (a residual thinking block would REPLACE the rebuilt content
# and lose the tool_use) and fall back to the rebuild path, where
# every wire representation uses the back-filled id consistently.
from turnstone.core.providers._anthropic import AnthropicProvider
class _Block:
def __init__(self, **d):
self._d = d
for k, v in d.items():
setattr(self, k, v)
def model_dump(self, **_kw):
return dict(self._d)
session = _make_session()
session._provider = AnthropicProvider()
session.ui.note_agent_child = MagicMock()
seen: list[dict] = []
call_count = [0]
def fake_stream(**kwargs):
seen.append(kwargs)
call_count[0] += 1
resp = MagicMock()
if call_count[0] == 1:
resp.content = [
_Block(type="thinking", thinking="hm", signature="sig_b"),
# Blank provider id — the back-fill case.
_Block(type="tool_use", id="", name="read_file", input={"path": "x"}),
]
resp.stop_reason = "tool_use"
else:
resp.content = [_Block(type="text", text="done")]
resp.stop_reason = "end_turn"
resp.usage = None
mgr = MagicMock()
mgr.__enter__ = MagicMock(
return_value=MagicMock(get_final_message=MagicMock(return_value=resp))
)
mgr.__exit__ = MagicMock(return_value=False)
return mgr
session.client.messages.stream = fake_stream
def fake_prepare(tc_dict, **_kwargs):
return {
"call_id": tc_dict["id"],
"func_name": "read_file",
"needs_approval": False,
"execute": lambda p: (p["call_id"], "contents"),
}
turns = [Turn.user("x")]
with (
patch.object(session, "_prepare_tool", side_effect=fake_prepare),
patch.object(session, "_resolve_replay_reasoning_to_model", return_value=True),
):
session._run_agent(
turns,
tools=[{"type": "function", "function": {"name": "read_file", "parameters": {}}}],
label="task",
parent_call_id="task-1",
)
# The back-filled turn carries NO native lane.
assert turns[1].native is None
# The replay request rebuilds the turn: tool_use and tool_result agree
# on the back-filled uuid — no blank id, no orphan.
replay = seen[1]["messages"]
tool_uses = [
b
for m in replay
if m["role"] == "assistant" and isinstance(m.get("content"), list)
for b in m["content"]
if isinstance(b, dict) and b.get("type") == "tool_use"
]
tool_results = [
b
for m in replay
if m["role"] == "user" and isinstance(m.get("content"), list)
for b in m["content"]
if isinstance(b, dict) and b.get("type") == "tool_result"
]
assert tool_uses and tool_results
assert tool_uses[0]["id"] # non-blank (uuid back-fill, restored)
assert tool_results[0]["tool_use_id"] == tool_uses[0]["id"]
def test_agent_blank_provider_id_keeps_synthesized_reasoning(self):
# The over-drop guard: a Chat-Completions server that BOTH leaves
# tool-call ids blank AND surfaces reasoning_content (llama.cpp,
# older vLLM) must still get its reasoning carried — the blank-id
# gate drops only the blocks a back-fill desyncs, and the
# synthesized reasoning_text lane has no client tool blocks at all.
from types import SimpleNamespace
from turnstone.core.providers._openai_chat import OpenAIChatCompletionsProvider
from turnstone.core.providers._openai_common import OPENAI_COMPAT_DEFAULT
session = _make_session()
session._provider = OpenAIChatCompletionsProvider()
session._model_alias = "loc"
session._registry = MagicMock()
session._registry.resolve_agent_alias.return_value = None
session._registry.resolve_agent_effort.return_value = None
session._registry.get_config.return_value = SimpleNamespace(
server_compat={"server_type": "vllm"}, replay_reasoning_to_model=True
)
session.ui.note_agent_child = MagicMock()
call_count = [0]
def fake_create(**kwargs):
call_count[0] += 1
resp = MagicMock()
choice = MagicMock()
if call_count[0] == 1:
choice.finish_reason = "tool_calls"
tc = MagicMock()
tc.id = "" # blank — the back-fill case
tc.function.name = "read_file"
tc.function.arguments = '{"path": "x"}'
choice.message.tool_calls = [tc]
choice.message.content = None
choice.message.reasoning = None
choice.message.reasoning_content = "work it out"
else:
choice.finish_reason = "stop"
choice.message.tool_calls = None
choice.message.content = "done"
choice.message.reasoning = None
choice.message.reasoning_content = None
resp.choices = [choice]
resp.usage = MagicMock(prompt_tokens=1, completion_tokens=1)
return resp
session.client.chat.completions.create = fake_create
def fake_prepare(tc_dict, **_kwargs):
return {
"call_id": tc_dict["id"],
"func_name": "read_file",
"needs_approval": False,
"execute": lambda p: (p["call_id"], "contents"),
}
turns = [Turn.user("x")]
with (
patch.object(session, "_prepare_tool", side_effect=fake_prepare),
patch.object(session, "_resolve_capabilities", return_value=OPENAI_COMPAT_DEFAULT),
patch.object(session, "_provider_extra_params", return_value={}),
):
session._run_agent(
turns,
tools=[{"type": "function", "function": {"name": "read_file"}}],
label="task",
parent_call_id="task-1",
)
# Reasoning survives the blank-id turn.
assert turns[1].native is not None
assert [b["type"] for b in turns[1].native.blocks] == ["reasoning_text"]
assert turns[1].native.blocks[0]["text"] == "work it out"
def test_agent_synthesizes_reasoning_and_attaches_vllm_replay_field(self):
# Chat-Completions lane (vLLM): non-streaming ``reasoning_content`` is
# captured into CompletionResult.reasoning, synthesized into the agent
# turn's native lane as a ``reasoning_text`` block by the SAME
# finalize helper the main loop uses — source-tagged from the AGENT
# alias — and replayed on the next request as vLLM's non-standard
# ``reasoning`` field (Phase 5 at the agent seam; the internal
# ``_provider_content`` key itself never reaches the wire).
from types import SimpleNamespace
from turnstone.core.providers._openai_chat import OpenAIChatCompletionsProvider
from turnstone.core.providers._openai_common import OPENAI_COMPAT_DEFAULT
session = _make_session()
session._provider = OpenAIChatCompletionsProvider()
session._model_alias = "loc-qwen"
session._registry = MagicMock()
session._registry.resolve_agent_alias.return_value = None
session._registry.resolve_agent_effort.return_value = None
session._registry.get_config.return_value = SimpleNamespace(
server_compat={"server_type": "vllm"}, replay_reasoning_to_model=True
)
session.ui.note_agent_child = MagicMock()
seen_messages: list[list[dict]] = []
call_count = [0]
def fake_create(**kwargs):
seen_messages.append(kwargs.get("messages") or [])
call_count[0] += 1
resp = MagicMock()
choice = MagicMock()
if call_count[0] == 1:
choice.finish_reason = "tool_calls"
tc = MagicMock()
tc.id = "call_0"
tc.function.name = "read_file"
tc.function.arguments = '{"path": "x"}'
choice.message.tool_calls = [tc]
choice.message.content = None
choice.message.reasoning = None
choice.message.reasoning_content = "scan the repo first"
else:
choice.finish_reason = "stop"
choice.message.tool_calls = None
choice.message.content = "done"
choice.message.reasoning = None
choice.message.reasoning_content = None
resp.choices = [choice]
resp.usage = MagicMock(prompt_tokens=10, completion_tokens=5)
return resp
session.client.chat.completions.create = fake_create
def fake_prepare(tc_dict, **_kwargs):
return {
"call_id": tc_dict["id"],
"func_name": "read_file",
"needs_approval": False,
"execute": lambda p: (p["call_id"], "contents"),
}
turns = [Turn.user("x")]
with (
patch.object(session, "_prepare_tool", side_effect=fake_prepare),
patch.object(session, "_resolve_capabilities", return_value=OPENAI_COMPAT_DEFAULT),
patch.object(session, "_provider_extra_params", return_value={}),
):
session._run_agent(
turns,
tools=[{"type": "function", "function": {"name": "read_file"}}],
label="task",
parent_call_id="task-1",
)
# The agent Turn carries the synthesized native lane, source-tagged
# via the agent alias (alias threading through the shared helper).
assistant_turn = turns[1]
assert assistant_turn.native is not None
assert assistant_turn.native.producer == "openai-compatible"
assert assistant_turn.native.blocks == (
{"type": "reasoning_text", "text": "scan the repo first", "source": "vllm"},
)
# The replay request carries the vLLM ``reasoning`` field on the
# assistant turn; the internal ``_provider_content`` key is stripped
# by the provider's sanitize before the wire.
replay = seen_messages[1]
assistant_wire = next(m for m in replay if m.get("role") == "assistant")
assert assistant_wire.get("reasoning") == "scan the repo first"
assert "_provider_content" not in assistant_wire
class TestRunAgentDenialMessage:
@@ -2612,6 +3206,9 @@ class TestProjectAgentSteps:
def test_colliding_ids_paired_fifo_not_last_wins(self):
# A local provider reuses id "call_0" across turns; FIFO pairing gives
# each call its OWN result, not last-wins (which would show out-B twice).
# Parented runs can no longer produce this input (_run_agent mints
# unique ids), but the FIFO stays as honest pairing for input a mint
# never touched — an unparented run, or turns constructed directly.
from turnstone.core.trajectory import ToolCall, Turn
turns = [
@@ -361,3 +361,94 @@ class TestResolveServerType:
session._registry = BrokenRegistry()
session._model_alias = "x"
assert session._resolve_server_type() == ""
class TestFinalizeProviderBlocks:
"""Direct unit tests for the shared native-lane builder
``ChatSession._finalize_provider_blocks`` in particular the
``had_blank_ids`` gate (a uuid back-fill reaches only the tool_calls
mirror, so blocks that would replay the blank id must be dropped while
the reasoning lane survives)."""
def test_passthrough_without_blank_ids(self) -> None:
session = _make_session()
blocks = [
{"type": "thinking", "thinking": "x", "signature": "s"},
{"type": "tool_use", "id": "toolu_1", "name": "f", "input": {}},
]
out = session._finalize_provider_blocks(blocks, [], has_tool_calls=True)
assert out is blocks
def test_no_tool_calls_strips_orphan_client_blocks(self) -> None:
session = _make_session()
blocks = [
{"type": "thinking", "thinking": "x", "signature": "s"},
{"type": "tool_use", "id": "toolu_1", "name": "f", "input": {}},
]
out = session._finalize_provider_blocks(blocks, [], has_tool_calls=False)
assert [b["type"] for b in out] == ["thinking"]
def test_blank_ids_drop_messages_shaped_lane_entirely(self) -> None:
# Anthropic-shaped lane with a blank-id tool_use: only reasoning_text
# may survive a blank-id turn, so the whole Messages-shaped lane goes
# — on that translator a surviving native lane REPLACES the rebuilt
# content, so a lane missing its tool_use would orphan the mirror's
# calls.
session = _make_session()
blocks = [
{"type": "thinking", "thinking": "x", "signature": "s"},
{"type": "text", "text": "using f"},
{"type": "tool_use", "id": "", "name": "f", "input": {}},
]
out = session._finalize_provider_blocks(blocks, [], has_tool_calls=True, had_blank_ids=True)
assert out == []
def test_blank_ids_drop_asymmetric_thinking_lane_without_tool_blocks(self) -> None:
# Asymmetric capture (thinking/text present, tool_use absent, mirror
# blank-id): the rule is total — no client block needs to be present
# for the Messages-shaped lane to be dropped on a blank-id turn.
session = _make_session()
blocks = [
{"type": "thinking", "thinking": "x", "signature": "s"},
{"type": "text", "text": "t"},
]
out = session._finalize_provider_blocks(blocks, [], has_tool_calls=True, had_blank_ids=True)
assert out == []
def test_blank_ids_drop_responses_reasoning_items(self) -> None:
# Responses reasoning items pair with their original sibling items;
# on a blank-id turn the function_call siblings are rebuilt from the
# back-filled mirror, so the reasoning items must go too.
session = _make_session()
blocks = [
{"type": "reasoning", "id": "rs_1", "summary": [], "encrypted_content": "enc"},
{"type": "function_call", "call_id": "", "name": "f", "arguments": "{}"},
]
out = session._finalize_provider_blocks(blocks, [], has_tool_calls=True, had_blank_ids=True)
assert out == []
def test_blank_ids_keep_only_reasoning_text(self) -> None:
# Google-shaped lane: the raw function dict (blank id) is dropped;
# the synthesized reasoning_text block survives — it carries no id
# and is shape-invalid on the Messages translator by design, and the
# Google swap simply finds no function blocks and keeps the
# sanitized mirror.
session = _make_session()
blocks = [
{"id": "", "type": "function", "function": {"name": "f", "arguments": "{}"}},
]
out = session._finalize_provider_blocks(
blocks, ["thinking text"], has_tool_calls=True, had_blank_ids=True
)
assert [b["type"] for b in out] == ["reasoning_text"]
assert out[0]["text"] == "thinking text"
def test_blank_ids_without_client_blocks_keep_the_lane(self) -> None:
# llama.cpp / older vLLM: blank tool ids AND loose reasoning text,
# but no client tool blocks at all — nothing can desync, so the
# synthesized reasoning lane must be kept (the over-drop case).
session = _make_session()
out = session._finalize_provider_blocks(
[], ["step by step"], has_tool_calls=True, had_blank_ids=True
)
assert [b["type"] for b in out] == ["reasoning_text"]
+11 -3
View File
@@ -60,11 +60,11 @@ class TestToolsMetadata:
"""Validate the metadata extracted from JSON files."""
def test_tool_count(self):
# 17 interactive tools + 12 coordinator-only tools.
assert len(TOOLS) == 29
# 19 interactive tools + 12 coordinator-only tools.
assert len(TOOLS) == 31
def test_task_agent_tools_count(self):
assert len(TASK_AGENT_TOOLS) == 11
assert len(TASK_AGENT_TOOLS) == 13
def test_coordinator_tools_count(self):
from turnstone.core.tools import COORDINATOR_TOOLS
@@ -109,6 +109,12 @@ class TestToolsMetadata:
"web_fetch",
"web_search",
"notify",
# Background-shell follow-ups: ``bash_output`` is read-only;
# ``kill_shell`` only signals process groups the session itself
# spawned via an approved bash call — strictly risk-reducing,
# so gating cleanup behind approval adds friction, not safety.
"bash_output",
"kill_shell",
# Coordinator read-only tools (no-mutation, safe to auto-approve):
"inspect_workstream",
"list_workstreams",
@@ -128,6 +134,8 @@ class TestToolsMetadata:
"web_search": "query",
"open_preview": "target",
"task_agent": "prompt",
"bash_output": "id",
"kill_shell": "id",
"memory": "name",
"recall": "query",
"notify": "message",
+5 -5
View File
@@ -201,9 +201,9 @@ class TestSoftCap:
# Oldest ("body-0") gone; newest ("overflow") present.
assert "body-0" not in bodies
assert "overflow" in bodies
# Warning logged.
assert any("watch_dispatch.queue_full" in r.message for r in caplog.records), (
"expected a watch_dispatch.queue_full warning record"
# Warning logged (the shared external-event rail owns the event now).
assert any("external_event.queue_full" in r.message for r in caplog.records), (
"expected an external_event.queue_full warning record"
)
def test_dispatch_soft_cap_does_not_evict_other_types(self, tmp_db):
@@ -457,8 +457,8 @@ class TestWakeFn:
# Entry survived; the failure surfaced as a warning, not a raise
# up into the poll loop.
assert len(session._nudge_queue) == 1
assert any("watch_dispatch.wake_failed" in r.message for r in caplog.records), (
"expected a watch_dispatch.wake_failed warning record"
assert any("external_event.wake_failed" in r.message for r in caplog.records), (
"expected an external_event.wake_failed warning record"
)
+2 -2
View File
@@ -311,8 +311,8 @@ def test_wire_payload_anthropic_compat(fixture_id: str) -> None:
_assert_golden(f"anthropic_compat__{fixture_id}", payload)
# GPT-5.6 Sol is the first COMMERCIAL OpenAI model to expose the "max"
# reasoning effort (Terra/Luna cap at "xhigh"; see OPENAI_CAPABILITIES).
# GPT-5.6 is the first commercial OpenAI family to expose the "max"
# reasoning effort across Sol, Terra, and Luna (see OPENAI_CAPABILITIES).
# The base matrix above pins only the default-effort Responses shape
# (gpt-5 → "medium"), so freeze a max-effort request to prove the new
# level compiles onto the native ``reasoning={"effort": "max"}`` param.
+1 -1
View File
@@ -1,3 +1,3 @@
"""turnstone - Multi-node AI orchestration platform with tool use, agent routing, and cluster simulation."""
__version__ = "1.7.3"
__version__ = "1.7.4"
+80
View File
@@ -101,6 +101,73 @@ def _cmd_create_user(args: argparse.Namespace) -> None:
print(" (Save this token now — it cannot be retrieved again)")
def _cmd_create_admin(args: argparse.Namespace) -> None:
"""Create an admin user (or promote an existing one) with full access.
Unlike ``create-user`` which creates a role-less user that logs into the
web UI read-only this assigns the built-in admin role, mirroring the web
first-run setup wizard (``POST /api/auth/setup``). Use it for headless
installs, or to unstick a ``create-user`` account that logs in only to hit
"Forbidden: token lacks 'approve' scope".
"""
import getpass
from turnstone.core.auth import hash_password, is_valid_username
if not is_valid_username(args.username):
print("Error: invalid username (1-64 chars: letters, digits, . _ -)", file=sys.stderr)
sys.exit(1)
storage = _get_storage(args)
# The admin role is seeded by DB migrations; without it we'd leave the
# account read-only — the exact lockout this command exists to prevent.
if storage.get_role("builtin-admin") is None:
print(
"Error: the built-in admin role is missing — run database migrations first.",
file=sys.stderr,
)
sys.exit(1)
# Promote an existing user (recovery path: create-user assigns no role).
existing = storage.get_user_by_username(args.username)
if existing is not None:
user_id = existing["user_id"]
already_admin = any(
r.get("role_id") == "builtin-admin" for r in storage.list_user_roles(user_id)
)
storage.assign_role(user_id, "builtin-admin", "")
if already_admin:
print(f"User '{args.username}' is already an admin (user {user_id}); no change.")
else:
print(f"Granted the admin role to existing user '{args.username}' (user {user_id}).")
print(" Log out and back in for the new access to take effect.")
return
# Create a fresh admin user.
password = args.password
if not password:
password = getpass.getpass("Password: ")
confirm = getpass.getpass("Confirm password: ")
if password != confirm:
print("Error: passwords do not match", file=sys.stderr)
sys.exit(1)
# Match the web setup wizard's floor for the most privileged account.
if len(password) < 8:
print("Error: password must be at least 8 characters", file=sys.stderr)
sys.exit(1)
display_name = args.name or args.username
user_id = uuid.uuid4().hex
pw_hash = hash_password(password)
storage.create_user(user_id, args.username, display_name, pw_hash)
storage.assign_role(user_id, "builtin-admin", "")
print(f"Created admin user: {user_id}")
print(f" Username: {args.username}")
print(f" Name: {display_name}")
print(" Role: admin (full access)")
def _cmd_create_token(args: argparse.Namespace) -> None:
from turnstone.core.auth import (
generate_token,
@@ -596,6 +663,18 @@ def main() -> None:
p_cu.add_argument("--token", action="store_true", help="Also create an initial API token")
p_cu.add_argument("--scopes", default="read,write,approve", help="Scopes for initial token")
p_ca = sub.add_parser(
"create-admin",
help="Create an admin user (or promote an existing one) with full access",
)
p_ca.add_argument("--username", required=True, help="Login username")
p_ca.add_argument("--name", default="", help="Display name (defaults to the username)")
p_ca.add_argument(
"--password",
default="",
help="Password (prompted if omitted; ignored when promoting an existing user)",
)
p_ct = sub.add_parser("create-token", help="Create an API token for a user")
p_ct.add_argument("--user", required=True, help="User ID")
p_ct.add_argument("--name", default="", help="Human label for the token")
@@ -690,6 +769,7 @@ def main() -> None:
dispatch = {
"create-user": _cmd_create_user,
"create-admin": _cmd_create_admin,
"create-token": _cmd_create_token,
"list-users": _cmd_list_users,
"list-tokens": _cmd_list_tokens,
+37 -3
View File
@@ -7,6 +7,7 @@ model auto-detection, workstream management, and the main() REPL entry point.
from __future__ import annotations
import argparse
import contextlib
import logging
import os
import readline
@@ -920,6 +921,41 @@ def resolve_cli_persona_kwargs(
return {}
def _close_all_sessions(manager: SessionManager) -> None:
"""Close EVERY loaded session at CLI exit — not just the active one.
``ChatSession.close()`` removes MCP listeners AND reaps the workstream's
background shells (#817). An active-only close would let a dev server
started in workstream 1 survive ``/new`` + ``/exit`` forever: its
detached process group outlives this process the exact leaked-server
class #816 removed.
Two phases so total exit latency doesn't stack per workstream: the kill
signals land on EVERY session's shells first (microseconds each — after
which nothing can outlive us), then the per-session closes pay their
join budgets, which are near-zero once the kills have landed. A Ctrl-C
during the close phase degrades gracefully instead of aborting the
sweep: the signals are already delivered, the remaining joins are
skipped, and the caller still runs MCP/registry shutdown. Best-effort
per workstream either way one bad teardown must not stop the rest.
"""
loaded = [(ws.id, ws.session) for ws in manager.list_all() if ws.session is not None]
for _ws_id, session in loaded:
with contextlib.suppress(Exception):
session._background_shells.signal_all()
try:
for ws_id, session in loaded:
try:
session.close()
except Exception:
print(dim(f" (workstream {ws_id[:8]} teardown error, continuing)"))
except KeyboardInterrupt:
# The kills above already landed; skipping the remaining joins
# leaks nothing — it only abandons wedged drain threads that die
# with this process anyway.
print(dim(" (interrupted — background shells already signalled)"))
def main() -> None:
parser = argparse.ArgumentParser(
description="Interactive CLI for OpenAI-compatible models with tool calling.",
@@ -1389,9 +1425,7 @@ def main() -> None:
except Exception as e:
print(f"\n{red(f'Error: {e}')}")
# Close active session (removes MCP listener) before shutting down MCP
if active and active.session:
active.session.close()
_close_all_sessions(manager)
if mcp_client:
mcp_client.shutdown()
registry.shutdown()
+227 -6
View File
@@ -6185,7 +6185,7 @@ let _modelDefaultAlias = "";
// and are re-merged on save. Reset per modal open.
let _rerankCalFields = {};
// Capability tile matrix — sparse-override semantics. The 9 tiles display
// Capability tile matrix — sparse-override semantics. The tiles display
// merge(dataclass defaults, known-model table baseline, explicit overrides);
// only EXPLICIT keys persist (saved keys + tiles the user toggled), so a
// known model keeps tracking future table updates instead of being pinned.
@@ -6197,6 +6197,8 @@ const _MODEL_CAP_KEYS = [
"supports_web_search",
"supports_temperature",
"supports_effort",
"supports_verbosity",
"supports_pro_mode",
"supports_transcription",
"supports_speech_synthesis",
"supports_audio_input",
@@ -6210,6 +6212,8 @@ const _MODEL_CAP_DEFAULTS = {
supports_web_search: false,
supports_temperature: true,
supports_effort: false,
supports_verbosity: false,
supports_pro_mode: false,
supports_transcription: false,
supports_speech_synthesis: false,
supports_audio_input: false,
@@ -6233,6 +6237,152 @@ function _modelRenderTiles() {
else if (k in _modelCapsBaseline) el.checked = !!_modelCapsBaseline[k];
else el.checked = _MODEL_CAP_DEFAULTS[k];
});
_updateModelResponseControls();
}
const _MODEL_RESPONSE_CONTROLS = [
{
key: "verbosity",
supportKey: "supports_verbosity",
elementId: "model-output-verbosity",
fieldId: "model-output-verbosity-field",
values: ["low", "medium", "high"],
},
{
key: "reasoning_mode",
supportKey: "supports_pro_mode",
elementId: "model-reasoning-mode",
fieldId: "model-reasoning-mode-field",
values: ["standard", "pro"],
},
];
let _modelResponseInitialIdentity = "";
let _modelResponseCurrentIdentity = "";
let _modelResponseCaptured = {};
let _modelResponseDirty = {};
function _modelIdentity() {
const provider = document.getElementById("model-provider").value;
const model = document.getElementById("model-name").value.trim();
const surface =
provider === "openai-compatible"
? document.getElementById("model-api-surface").value
: "";
return provider + "\n" + model + "\n" + surface;
}
function _modelUsesResponsesSurface() {
const provider = document.getElementById("model-provider").value;
if (provider === "openai") return true;
return (
provider === "openai-compatible" &&
document.getElementById("model-api-surface").value === "responses"
);
}
function _modelResponseValueValid(spec, value) {
return typeof value === "string" && spec.values.indexOf(value) !== -1;
}
function _updateModelResponseControls() {
const group = document.getElementById("model-response-controls");
if (!group) return;
const responseSurface = _modelUsesResponsesSurface();
const sameIdentity =
_modelResponseInitialIdentity &&
_modelIdentity() === _modelResponseInitialIdentity;
let anyVisible = false;
_MODEL_RESPONSE_CONTROLS.forEach(function (spec) {
const field = document.getElementById(spec.fieldId);
const select = document.getElementById(spec.elementId);
if (!field || !select) return;
// A value _captureModelResponseControls lifted out of the row's JSON
// stays visible (and re-saveable) while the identity still matches the
// row being edited, deliberately NOT consulting the capability
// baseline: the baseline arrives async (or never, on the compat lane),
// and yielding to it would hide the pinned value and silently drop it
// on save — the same lift-then-restore contract as server_compat,
// rerank calibration, and thinking_param. Wire safety is server-side:
// emission gates on the merged supports_* flag, so a pinned value on
// an unsupported model is inert; "Provider default" explicitly clears
// it. An explicit tile override (either polarity) supersedes the
// fallback — unchecking the tile is the operator's way to retire it.
const capturedFallback =
sameIdentity &&
!(spec.supportKey in _modelCapsExplicit) &&
_modelResponseValueValid(spec, select.value);
const visible =
responseSurface && (_modelGetTile(spec.supportKey) || capturedFallback);
field.hidden = !visible;
anyVisible = anyVisible || visible;
});
group.hidden = !anyVisible;
}
function _resetModelResponseControls() {
_MODEL_RESPONSE_CONTROLS.forEach(function (spec) {
const select = document.getElementById(spec.elementId);
if (select) select.value = "";
});
_modelResponseInitialIdentity = "";
_modelResponseCurrentIdentity = _modelIdentity();
_modelResponseCaptured = {};
_modelResponseDirty = {};
_updateModelResponseControls();
}
function _captureModelResponseControls(capsObj) {
if (!_modelUsesResponsesSurface()) return;
_MODEL_RESPONSE_CONTROLS.forEach(function (spec) {
const select = document.getElementById(spec.elementId);
if (!select) return;
const explicitlyUnsupported =
spec.supportKey in _modelCapsExplicit &&
!_modelCapsExplicit[spec.supportKey];
const value = capsObj[spec.key];
if (!explicitlyUnsupported && _modelResponseValueValid(spec, value)) {
select.value = value;
_modelResponseCaptured[spec.key] = value;
delete capsObj[spec.key];
}
});
}
function _mergeModelResponseControls(caps) {
if (!_modelUsesResponsesSurface()) return;
const sameIdentity =
_modelResponseInitialIdentity &&
_modelIdentity() === _modelResponseInitialIdentity;
_MODEL_RESPONSE_CONTROLS.forEach(function (spec) {
// Dirty (select touched this session) lets the select override a
// stale JSON key, but only for the identity that made it dirty —
// after a model/provider/surface change the flag describes the OLD
// row, and honoring it would delete a key hand-typed into the
// Advanced JSON for the new one.
if (_modelResponseDirty[spec.key] && sameIdentity) delete caps[spec.key];
else if (spec.key in caps) return; // Advanced JSON wins.
const select = document.getElementById(spec.elementId);
if (!select || !_modelResponseValueValid(spec, select.value)) return;
// Same capturedFallback contract as _updateModelResponseControls
// (rationale there): a lifted same-identity value must re-save, or an
// unrelated edit silently drops it from the row.
const capturedFallback =
sameIdentity && !(spec.supportKey in _modelCapsExplicit);
if (_modelGetTile(spec.supportKey) || capturedFallback) {
caps[spec.key] = select.value;
}
});
}
function _rememberModelResponseControl(spec) {
_modelResponseDirty[spec.key] = true;
if (_modelIdentity() !== _modelResponseInitialIdentity) return;
const select = document.getElementById(spec.elementId);
if (select && _modelResponseValueValid(spec, select.value)) {
_modelResponseCaptured[spec.key] = select.value;
} else {
delete _modelResponseCaptured[spec.key];
}
}
// Roles surfaced in the Models → Roles sub-tab. Each entry maps a
@@ -6768,6 +6918,25 @@ function _renderModels(items) {
if (m.max_tokens != null) overrides.push("max_tok=" + m.max_tokens);
if (m.reasoning_effort != null)
overrides.push("effort=" + m.reasoning_effort);
let displayCaps = m.capabilities;
if (typeof displayCaps === "string") {
try {
displayCaps = JSON.parse(displayCaps || "{}");
} catch (e) {
displayCaps = {};
}
}
if (!_isPlainObject(displayCaps)) displayCaps = {};
if (
displayCaps.supports_verbosity !== false &&
["low", "medium", "high"].indexOf(displayCaps.verbosity) !== -1
)
overrides.push("verbosity=" + displayCaps.verbosity);
if (
displayCaps.supports_pro_mode !== false &&
["standard", "pro"].indexOf(displayCaps.reasoning_mode) !== -1
)
overrides.push("mode=" + displayCaps.reasoning_mode);
// Reasoning persistence flags surface only when non-default
// (persist=False is the operator opt-out; replay=True is the
// operator opt-in). Default values are silent.
@@ -6985,8 +7154,10 @@ function showCreateModelModal() {
if (_calChip) _calChip.style.display = "none";
const _recalBtn = document.getElementById("model-recalibrate-btn");
if (_recalBtn) _recalBtn.hidden = true;
_modelCapsSeq++; // invalidate lookups from a prior shelf lifecycle
_modelCapsBaseline = {};
_modelCapsExplicit = {};
_resetModelResponseControls();
_modelRenderTiles();
document.getElementById("model-autofill").hidden = true;
_refreshModelSuggestions();
@@ -7082,7 +7253,7 @@ function showEditModelModal(definitionId) {
}
},
);
// Lift the 9 matrix keys out of the JSON into the tiles — they are
// Lift the capability keys out of the JSON into the tiles — they are
// the row's explicit overrides and the textarea holds the remainder.
_modelCapsExplicit = {};
_MODEL_CAP_KEYS.forEach(function (k) {
@@ -7091,6 +7262,9 @@ function showEditModelModal(definitionId) {
delete capsObj[k];
}
});
_modelResponseInitialIdentity = _modelIdentity();
_modelResponseCurrentIdentity = _modelResponseInitialIdentity;
_captureModelResponseControls(capsObj);
_modelRenderTiles();
_modelCapsRefreshBaseline();
_scheduleEffortLadder();
@@ -7259,6 +7433,7 @@ function submitCreateModel() {
Object.keys(_modelCapsExplicit).forEach(function (k) {
if (!(k in caps)) caps[k] = _modelGetTile(k);
});
_mergeModelResponseControls(caps);
// Re-merge reranker calibration fields extracted on edit so an unrelated edit
// doesn't silently drop the calibration. A field typed directly into the
@@ -7688,12 +7863,34 @@ function recalibrateModel() {
}
/* Capability auto-fill: when the user types a known model name or
changes the provider, look up static capabilities and pre-fill
context_window and the capabilities textarea. */
changes the provider, look up static capabilities and refresh the
context window, capability tiles, and conditional response controls. */
let _capsTimer = null;
let _modelCapsSeq = 0;
function _onModelFieldChange() {
clearTimeout(_capsTimer);
const nextIdentity = _modelIdentity();
if (
_modelResponseCurrentIdentity &&
nextIdentity !== _modelResponseCurrentIdentity
) {
_MODEL_RESPONSE_CONTROLS.forEach(function (spec) {
const select = document.getElementById(spec.elementId);
if (!select) return;
const captured = _modelResponseCaptured[spec.key];
select.value =
nextIdentity === _modelResponseInitialIdentity &&
_modelResponseValueValid(spec, captured)
? captured
: "";
});
}
_modelResponseCurrentIdentity = nextIdentity;
_modelCapsSeq++; // invalidate any capability lookup already in flight
_modelCapsBaseline = {};
const banner = document.getElementById("model-autofill");
if (banner) banner.hidden = true;
_modelRenderTiles();
_capsTimer = setTimeout(_modelCapsRefreshBaseline, 500);
_scheduleEffortLadder();
}
@@ -7822,6 +8019,7 @@ function _modelCapsRefreshBaseline() {
const provider = document.getElementById("model-provider").value;
const modelName = document.getElementById("model-name").value.trim();
const banner = document.getElementById("model-autofill");
const seq = ++_modelCapsSeq;
if (
!modelName ||
provider === "openai-compatible" ||
@@ -7834,7 +8032,6 @@ function _modelCapsRefreshBaseline() {
}
// Two type-then-pause cycles can have both fetches in flight; a reordered
// older response must not clobber the tiles (the _schPreviewSeq pattern).
const seq = ++_modelCapsSeq;
authFetch(
"/v1/api/admin/model-capabilities?provider=" +
encodeURIComponent(provider) +
@@ -7918,6 +8115,7 @@ function _applyProviderDefaults() {
if (serverFieldsRow) {
serverFieldsRow.hidden = provider === "anthropic-compatible";
}
_updateModelResponseControls();
}
/* Populate the model name datalist with known model prefixes for the
@@ -7957,14 +8155,26 @@ function _refreshModelSuggestions() {
const tmEl = document.getElementById("model-thinking-mode");
if (tmEl) tmEl.addEventListener("change", _toggleThinkingParam);
if (tmEl) tmEl.addEventListener("change", _scheduleEffortLadder);
_MODEL_RESPONSE_CONTROLS.forEach(function (spec) {
const select = document.getElementById(spec.elementId);
if (select)
select.addEventListener("change", function () {
_rememberModelResponseControl(spec);
});
});
["model-thinking-param", "model-effort-param", "model-capabilities"].forEach(
function (id) {
const el = document.getElementById(id);
if (el) el.addEventListener("input", _scheduleEffortLadder);
},
);
const rawCapsEl = document.getElementById("model-capabilities");
if (rawCapsEl)
rawCapsEl.addEventListener("input", function () {
_modelResponseDirty = {};
});
const apiSurfEl = document.getElementById("model-api-surface");
if (apiSurfEl) apiSurfEl.addEventListener("change", _scheduleEffortLadder);
if (apiSurfEl) apiSurfEl.addEventListener("change", _onModelFieldChange);
const grid = document.getElementById("model-capgrid");
if (grid) {
grid.addEventListener("change", function (e) {
@@ -7972,6 +8182,17 @@ function _refreshModelSuggestions() {
if (!cap) return;
// a toggle IS the override decision — the key persists from here on
_modelCapsExplicit[cap] = e.target.checked;
if (cap === "supports_verbosity" || cap === "supports_pro_mode") {
const spec = _MODEL_RESPONSE_CONTROLS.find(function (item) {
return item.supportKey === cap;
});
if (spec && !e.target.checked) {
const select = document.getElementById(spec.elementId);
if (select) select.value = "";
delete _modelResponseCaptured[spec.key];
}
_updateModelResponseControls();
}
if (cap === "supports_rerank") {
const recalBtn = document.getElementById("model-recalibrate-btn");
if (recalBtn)
+54
View File
@@ -1744,6 +1744,46 @@
<option value="max">Max</option>
</select>
<div
id="model-response-controls"
role="group"
aria-labelledby="model-response-controls-title"
hidden
>
<div class="sh-section" id="model-response-controls-title">
Response controls
</div>
<div class="field-pair">
<div id="model-output-verbosity-field" hidden>
<label for="model-output-verbosity"
>Output verbosity
<span class="label-hint"
>answer length, independent of effort</span
></label
>
<select id="model-output-verbosity">
<option value="">Provider default</option>
<option value="low">Low — concise</option>
<option value="medium">Medium — balanced</option>
<option value="high">High — detailed</option>
</select>
</div>
<div id="model-reasoning-mode-field" hidden>
<label for="model-reasoning-mode"
>Reasoning mode
<span class="label-hint"
>Pro applies more work before answering</span
></label
>
<select id="model-reasoning-mode">
<option value="">Provider default</option>
<option value="standard">Standard</option>
<option value="pro">Pro</option>
</select>
</div>
</div>
</div>
<div id="model-server-compat-section" hidden>
<div class="sh-section">Server compatibility</div>
<div class="field-pair" id="model-server-fields-row">
@@ -1882,6 +1922,20 @@
></span
><span class="cap-name">Reasoning-effort control</span></label
>
<label class="cap"
><input
type="checkbox"
data-cap="supports_verbosity"
/><span class="cap-led"></span
><span class="cap-name">Output verbosity</span></label
>
<label class="cap"
><input
type="checkbox"
data-cap="supports_pro_mode"
/><span class="cap-led"></span
><span class="cap-name">Standard / Pro mode</span></label
>
<label class="cap"
><input
type="checkbox"
+833
View File
@@ -0,0 +1,833 @@
"""Per-session registry for explicitly backgrounded bash shells (#817).
#816 made the ``bash`` tool terminate its whole process group when the call
returns no leaked servers, no hangs, but also no way to keep a dev server
alive across calls. This registry restores that as an explicit opt-in with
the model-facing shape the frontier coding agents converged on: a boolean on
the shell tool, a short ``bash_N`` handle, a delta-output reader that returns
only lines produced since the previous read, and a kill tool.
Lifetime rules (the #816 rule, extended):
* The tracked command defines the shell's lifetime. When it exits —
naturally, by ``kill``, or by registry teardown its whole session group
is SIGKILLed, so nothing the command backgrounded can outlive it.
* Shells survive generation-cancel (they are deliberately detached) and die
with the owning session: :meth:`BackgroundShellRegistry.close` runs from
``ChatSession.close()``, which every workstream-teardown path funnels
through.
* Shells spawned inside a task_agent carry that agent's ``owner`` tag; the
agent's ``finally`` reaps them, and owner-scoped lookup keeps parallel
agents (and the parent) from touching each other's handles.
Output is buffered per shell as a rolling deque of lines (stderr tagged
``[stderr] `` inline, arrival order) capped by total characters with
drop-oldest semantics a chatty server cannot grow a session's memory
unbounded. Reads advance a cursor over the *logical* line stream, so a
line dropped before it was ever read surfaces as an explicit gap count
rather than silently vanishing.
"""
from __future__ import annotations
import contextlib
import itertools
import json
import os
import re
import signal
import subprocess
import sys
import tempfile
import threading
import time
from collections import deque
from dataclasses import dataclass
from typing import TYPE_CHECKING, Any, Literal
from turnstone.core.log import get_logger
if TYPE_CHECKING:
from collections.abc import Callable
log = get_logger(__name__)
ShellStatus = Literal["running", "completed", "killed"]
# Live (status == "running") shells per session. A hard backstop against a
# runaway loop of spawns on a multi-tenant node, not an operator knob.
_DEFAULT_MAX_SHELLS = 8
# Rolling per-shell buffer cap, in characters. Oldest whole lines drop
# first; the newest line always survives even if it alone exceeds the cap.
_DEFAULT_MAX_BUFFER_CHARS = 200_000
# How long to wait for the drain threads after the group kill forces their
# pipes to EOF. A grandchild that double-``setsid``-escaped the group can
# hold a pipe open past this — the drain is a daemon thread and leaks
# (logged) until that process dies, same acceptance as the foreground tool.
_DRAIN_JOIN_TIMEOUT_S = 5
# TOTAL join budget for ``kill``/``reap`` across all of a shell's threads
# (not per-thread — a wedged drain must not stack timeouts).
_WAITER_JOIN_TIMEOUT_S = 10
# TOTAL join budget for ``close()`` across ALL shells. close() runs on the
# workstream-teardown funnel, which the server can reach from an async
# handler — an unbounded (or per-shell-stacking) wait here would freeze the
# node's event loop, not just this workstream. Threads still alive past the
# budget are daemons: logged and abandoned, they die with their pipes.
_CLOSE_JOIN_BUDGET_S = 5
# Exited records retained per registry (drop-oldest). Keeps a long-lived
# workstream that backgrounds thousands of short jobs from accumulating
# dead records (each can pin up to ``max_buffer_chars`` of buffer) while
# still letting the model read recently-exited shells' output.
_MAX_EXITED_RECORDS = 32
# Bounds on the model-supplied ``filter`` regex: pattern length, how much of
# each line the pattern sees, and wall-clock for the whole filter pass. The
# pass runs in a SUBPROCESS, not a thread: CPython's sre engine holds the
# GIL for the entire duration of one ``search`` call, so a catastrophic-
# backtracking pattern freezes every thread in the interpreter — no
# in-process timeout (thread join, signal, anything) can fire. A child
# process is killable from outside the GIL; on timeout the read errors
# WITHOUT consuming the delta (the cursor only commits on a completed pass).
_MAX_FILTER_PATTERN_CHARS = 512
_FILTER_MAX_LINE_CHARS = 4096
_FILTER_TIMEOUT_S = 2.0
# Runs inside ``sys.executable -c``: reads {pattern, lines} as JSON on
# stdin (lines already truncated parent-side), writes the MATCHING INDEXES
# as JSON on stdout (indexes, not lines — no need to echo a 200K buffer
# back through a pipe).
_FILTER_HELPER_SRC = (
"import json, re, sys\n"
"d = json.load(sys.stdin)\n"
"p = re.compile(d['pattern'])\n"
"sys.stdout.write(json.dumps([i for i, ln in enumerate(d['lines']) if p.search(ln)]))\n"
)
class UnknownShellError(LookupError):
"""No shell with that id is visible in the caller's owner scope."""
class TooManyShellsError(RuntimeError):
"""The per-session live-shell cap would be exceeded."""
class FilterTimeoutError(ValueError):
"""The ``filter`` regex did not finish within the time bound."""
class FilterExecError(RuntimeError):
"""The filter helper process failed for a non-pattern reason."""
def _filter_lines_bounded(pattern: re.Pattern[str], lines: list[str], shell_id: str) -> list[str]:
"""Apply ``pattern`` per line with a wall-clock bound.
A catastrophic-backtracking pattern would wedge the (auto-approved)
tool call the exact never-returns class #816 removed and it cannot
be bounded IN-PROCESS: sre holds the GIL for the whole ``search`` call,
freezing every interpreter thread including any watchdog. So the pass
runs in a small child process (killable from the OS): each line
truncated PARENT-side to :data:`_FILTER_MAX_LINE_CHARS` before
serialization (a filter targets log lines; shipping a retained multi-MB
line through the pipe would spend the time budget on I/O and misreport
a fine pattern as slow), the whole pass bounded by
:data:`_FILTER_TIMEOUT_S`, SIGKILL on the child's group past that.
Raises :class:`FilterTimeoutError` on timeout and
:class:`FilterExecError` on a helper failure that is NOT the pattern's
fault (fork/OOM/env) distinct messages, so the model doesn't
"simplify" an innocent regex. Either way the caller consumes nothing.
The ~tens-of-ms interpreter startup is paid only on filtered reads.
Threat model for the auto-approved path (``bash_output`` runs without
operator approval): the only model-controlled inputs are the PATTERN
and, transitively, the buffered text. The pattern is compiled
parent-side before the fork (a non-regex payload fails there), the
child executes only the fixed ``_FILTER_HELPER_SRC`` the pattern is
DATA on stdin, never code , the child gets a scrubbed environment, no
shell, read-only work, and a SIGKILL at the time bound. Worst case a
hostile pattern buys ~2s of one core.
"""
from turnstone.core.env import scrubbed_env
payload = json.dumps(
{
"pattern": pattern.pattern,
"lines": [ln[:_FILTER_MAX_LINE_CHARS] for ln in lines],
},
ensure_ascii=False,
)
try:
proc = subprocess.Popen(
[sys.executable, "-c", _FILTER_HELPER_SRC],
stdin=subprocess.PIPE,
stdout=subprocess.PIPE,
stderr=subprocess.DEVNULL,
# Pin BOTH pipe directions to UTF-8: ``text=True`` alone uses
# the locale encoding, and on a C/POSIX-locale node a single
# U+FFFD (from the drain's ``errors="replace"``) would raise
# UnicodeEncodeError out of communicate() — escaping the
# Timeout/Exec error taxonomy as a generic crash. The child's
# own stdio decode is pinned via PYTHONIOENCODING.
encoding="utf-8",
errors="replace",
# scrubbed_env, not os.environ: the helper needs no secrets (it
# runs only our trusted source over already-buffered text), and
# every other fork in this codebase strips API keys/tokens —
# this one must not be the exception.
env={**scrubbed_env(), "PYTHONIOENCODING": "utf-8"},
start_new_session=True,
)
except OSError as e:
# Fork pressure (EAGAIN) / exec failure — same containment class as
# spawn()'s thread-start guard, and by contract NOT the pattern's
# fault.
log.warning("bg_shell.filter_helper_spawn_failed", shell_id=shell_id, error=str(e))
raise FilterExecError(
"the filter could not be applied (helper failed to start); this "
"is not a problem with your pattern — no output was consumed; "
"retry, or read without a filter"
) from e
try:
out, _ = proc.communicate(payload, timeout=_FILTER_TIMEOUT_S)
except subprocess.TimeoutExpired:
with contextlib.suppress(OSError, ProcessLookupError):
os.killpg(proc.pid, signal.SIGKILL)
with contextlib.suppress(subprocess.TimeoutExpired):
proc.wait(timeout=5)
log.warning("bg_shell.filter_timeout", shell_id=shell_id, pattern=pattern.pattern[:80])
raise FilterTimeoutError(
f"filter regex took longer than {_FILTER_TIMEOUT_S:g}s to run; no "
"output was consumed — simplify the pattern or retry without a filter"
) from None
if proc.returncode != 0:
# The parent validated the compile, so a child failure is exotic
# (fork pressure, interpreter env) — NOT the pattern's fault.
log.warning(
"bg_shell.filter_helper_failed",
shell_id=shell_id,
returncode=proc.returncode,
)
raise FilterExecError(
f"the filter could not be applied (helper exited {proc.returncode}); "
"this is not a problem with your pattern — no output was consumed; "
"retry, or read without a filter"
)
try:
indexes = json.loads(out)
except ValueError:
log.warning("bg_shell.filter_helper_bad_output", shell_id=shell_id)
raise FilterExecError(
"the filter could not be applied (helper returned malformed data); "
"no output was consumed — retry, or read without a filter"
) from None
return [lines[i] for i in indexes if isinstance(i, int) and 0 <= i < len(lines)]
def drain_pipe_lines(pipe: Any, on_line: Callable[[str], None]) -> None:
"""Read ``pipe`` line-by-line until EOF, forwarding each to ``on_line``.
The drain half of the shared bash recipe (see :func:`spawn_group_leader`
for the spawn half): both variants of the tool tolerate the same two
end-of-stream shapes. A pipe torn down by the session-group kill is the
expected end; anything else must not kill the drain silently. (The
``errors="replace"`` on the shared Popen pre-empts UnicodeDecodeError
a ValueError that would otherwise end the drain early and drop ALL
remaining output while reporting a clean success.)
"""
try:
for line in pipe:
on_line(line)
except (ValueError, OSError):
log.debug("bash.drain_read_error", exc_info=True)
def spawn_group_leader(
command: str, *, stop_on_error: bool, env: dict[str, str] | None
) -> tuple[subprocess.Popen[str], int, str]:
"""Write the script, fork the detached group leader, snapshot its pgid.
THE shared prologue for both runs of the model-facing bash tool the
foreground executor (``ChatSession._exec_bash``) and this registry so
the two variants of one tool cannot drift: same ``pipefail``/``set -e``
preamble, same decode policy (``errors="replace"``), same session-group
discipline. The script file exists because bash reads scripts lazily
(robust to quoting/length; unlinking early could truncate a long script
mid-run) the CALLER owns the unlink on its own exit path. On a
failed fork the script is unlinked here and the error propagates. The
pgid snapshot happens while the leader is alive (``start_new_session``
makes ``pgid == pid``); the microseconds-wide pid-wraparound TOCTOU is
the same accepted one as always.
"""
with tempfile.NamedTemporaryFile(mode="w", suffix=".sh", delete=False) as f:
preamble = "set -o pipefail\n"
if stop_on_error:
preamble += "set -e\n"
f.write(preamble + command)
script_path = f.name
try:
proc = subprocess.Popen(
["bash", script_path],
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
text=True,
errors="replace",
start_new_session=True,
env=env,
)
except BaseException:
with contextlib.suppress(OSError):
os.unlink(script_path)
raise
try:
pgid = os.getpgid(proc.pid)
except OSError:
pgid = proc.pid
return proc, pgid, script_path
@dataclass
class ShellRead:
"""One delta read: lines since the previous read, plus shell state.
``lines`` is post-filter (what the caller shows); ``new_line_count`` is
the pre-filter delta size the cursor advanced past all of them, so a
filtered-out line is consumed, never deferred to a later read.
``dropped_lines`` counts lines lost to the buffer cap before they were
ever read (an explicit gap, not silence).
"""
shell_id: str
status: ShellStatus
exit_code: int | None
lines: list[str]
new_line_count: int
dropped_lines: int
# Lines in this delta longer than the per-line filter window — their
# tails were invisible to the pattern. Only populated on filtered
# reads; the caller surfaces it so a "none matching" answer over
# clipped evidence is never silent.
clipped_lines: int = 0
class BackgroundShell:
"""One detached shell: process handles, rolling buffer, read cursor.
Mutable state is guarded by ``self.lock`` the drain threads append
while reads snapshot; the waiter thread flips ``status`` exactly once.
"""
def __init__(
self,
shell_id: str,
command: str,
proc: subprocess.Popen[str],
pgid: int,
owner: str | None,
script_path: str,
max_buffer_chars: int,
) -> None:
self.shell_id = shell_id
self.command = command
self.proc = proc
self.pid = proc.pid
self.pgid = pgid
self.owner = owner
self.status: ShellStatus = "running"
self.exit_code: int | None = None
self.lock = threading.Lock()
self._script_path = script_path
self._max_buffer_chars = max_buffer_chars
# Rolling buffer over the logical line stream: ``_buffer`` holds the
# retained tail; ``_dropped_total``/``_total_lines`` are absolute
# line counts so the cursor survives drop-oldest evictions.
self._buffer: deque[str] = deque()
self._buffered_chars = 0
self._dropped_total = 0
self._total_lines = 0
self._read_cursor = 0
# Set (under ``lock``) before the group kill on every deliberate
# termination path so the waiter can distinguish "killed" from
# "completed" and suppress the exit callback.
self._killed = False
self._threads: list[threading.Thread] = []
# Serializes whole read passes (snapshot → filter → commit). The
# buffer lock alone leaves a window where two concurrent reads of
# the same shell snapshot the same cursor and BOTH return the delta
# as new — double-delivering every line. Held across the filter
# subprocess too: correctness over parallel reads of one shell.
self.read_serial = threading.Lock()
# Monotonic EXIT order (registry-assigned by the waiter), None while
# running. Dead-record eviction sorts on this, never on spawn
# order: a long-lived first-spawned server must not be the first
# record evicted — least of all by its own exit's prune, which
# would drop its promised exit notice and crash output unread.
self._exit_seq: int | None = None
@property
def unread_lines(self) -> int:
"""Lines still READABLE that the cursor hasn't consumed — excludes
lines the buffer cap already evicted, so an exit notice never
promises more output than ``bash_output`` can actually return."""
with self.lock:
return self._total_lines - max(self._read_cursor, self._dropped_total)
def _append(self, line: str) -> None:
with self.lock:
self._buffer.append(line)
self._buffered_chars += len(line)
self._total_lines += 1
# Drop oldest whole lines past the cap, but always keep the
# newest — a single oversized line must not empty the buffer.
while self._buffered_chars > self._max_buffer_chars and len(self._buffer) > 1:
dropped = self._buffer.popleft()
self._buffered_chars -= len(dropped)
self._dropped_total += 1
def _snapshot_delta(self) -> tuple[list[str], int, int, ShellStatus, int | None]:
"""Snapshot unread lines WITHOUT consuming them.
Returns ``(delta, gap, new_cursor, status, exit_code)``. The caller
commits ``new_cursor`` via :meth:`_commit_cursor` only after any
filtering succeeded a failed/timed-out filter must not eat output.
"""
with self.lock:
start = max(self._read_cursor, self._dropped_total)
gap = start - self._read_cursor
delta = list(itertools.islice(self._buffer, start - self._dropped_total, None))
return delta, gap, self._total_lines, self.status, self.exit_code
def _commit_cursor(self, new_cursor: int) -> None:
with self.lock:
# max(): monotonic under concurrent reads of the same scope.
self._read_cursor = max(self._read_cursor, new_cursor)
class BackgroundShellRegistry:
"""Session-scoped table of background shells, ``bash_N``-keyed.
Thread-safe: tool calls (spawn/read/kill), waiter threads (exit
transitions), and teardown (close/reap) may interleave freely.
``on_exit`` fires from the waiter thread on NATURAL exit only never
for ``kill``/``reap``/``close`` after the drains have flushed, so a
read triggered by the callback sees the complete output.
"""
def __init__(
self,
*,
max_shells: int = _DEFAULT_MAX_SHELLS,
max_buffer_chars: int = _DEFAULT_MAX_BUFFER_CHARS,
max_exited_records: int = _MAX_EXITED_RECORDS,
on_exit: Callable[[BackgroundShell], None] | None = None,
) -> None:
self._max_shells = max_shells
self._max_buffer_chars = max_buffer_chars
self._max_exited_records = max_exited_records
self._on_exit = on_exit
self._shells: dict[str, BackgroundShell] = {}
self._lock = threading.Lock()
self._counter = 0
self._exit_counter = 0
self._closed = False
# -- Spawning -----------------------------------------------------------
def spawn(
self,
command: str,
*,
env: dict[str, str] | None = None,
owner: str | None = None,
stop_on_error: bool = False,
) -> BackgroundShell:
"""Start ``command`` as a detached shell; return its record.
Raises ``RuntimeError`` after :meth:`close`, :class:`TooManyShellsError`
at the live-shell cap, and propagates ``OSError`` from a failed spawn.
"""
if env is None:
from turnstone.core.env import scrubbed_env
env = scrubbed_env()
# Fast-fail before paying disk + fork; re-checked authoritatively
# under the lock after the fork (spawn stays lock-free through the
# slow syscalls so close()/reap() — which serialize on the registry
# lock with a total time budget — can never be blocked behind a
# stalled filesystem write or fork).
with self._lock:
self._check_capacity_locked(owner)
# Shared prologue with the foreground bash tool — the waiter unlinks
# the script after exit.
proc, pgid, script_path = spawn_group_leader(command, stop_on_error=stop_on_error, env=env)
try:
with self._lock:
# Authoritative re-check: a concurrent spawn/close may have
# won the race while we were forking. Refusal lands in the
# outer handler, which reaps the freshly-forked group —
# nothing may outlive a failed call (#816 rule).
self._check_capacity_locked(owner)
self._counter += 1
shell = BackgroundShell(
shell_id=f"bash_{self._counter}",
command=command,
proc=proc,
pgid=pgid,
owner=owner,
script_path=script_path,
max_buffer_chars=self._max_buffer_chars,
)
# Publish, wire and START the threads under the registry
# lock: close()/reap() take the same lock, so they can never
# observe a registered shell whose threads aren't started
# (they would "join" nothing and return while the drains /
# waiter start up behind them). Registration is popped on a
# start failure IN the same hold, so a thread-exhausted node
# (RLIMIT_NPROC) can't strand an orphan record whose
# never-started Thread objects would make every later
# ``join`` — hence every teardown — raise. The thread
# bodies only ever take ``shell.lock`` or re-take the
# registry lock AFTER this hold is released (the waiter's
# prune), so starting them here cannot deadlock.
self._shells[shell.shell_id] = shell
assert proc.stdout is not None and proc.stderr is not None
out_thread = threading.Thread(
target=self._drain,
args=(proc.stdout, shell, False),
name=f"bg-shell-out-{shell.shell_id}",
daemon=True,
)
err_thread = threading.Thread(
target=self._drain,
args=(proc.stderr, shell, True),
name=f"bg-shell-err-{shell.shell_id}",
daemon=True,
)
waiter = threading.Thread(
target=self._wait_for_exit,
args=(shell, out_thread, err_thread),
name=f"bg-shell-wait-{shell.shell_id}",
daemon=True,
)
shell._threads = [out_thread, err_thread, waiter]
try:
out_thread.start()
err_thread.start()
waiter.start()
except BaseException:
self._shells.pop(shell.shell_id, None)
raise
except BaseException:
# Refused post-fork or thread start failed: reap the fresh group
# (any started drain then EOFs and exits on its own) and surface
# the original error to the tool layer.
with contextlib.suppress(OSError, ProcessLookupError):
os.killpg(pgid, signal.SIGKILL)
with contextlib.suppress(subprocess.TimeoutExpired):
proc.wait(timeout=5)
with contextlib.suppress(OSError):
os.unlink(script_path)
raise
log.info(
"bg_shell.spawned",
shell_id=shell.shell_id,
pid=shell.pid,
owner=owner or "",
)
return shell
def _check_capacity_locked(self, owner: str | None) -> None:
"""Raise if closed or at the live-shell cap. Caller holds the lock."""
if self._closed:
raise RuntimeError("background shells unavailable: session is closing")
live = [s for s in self._shells.values() if s.status == "running"]
if len(live) < self._max_shells:
return
# The cap is registry-wide (it protects the node), but the advice
# must be scope-honest: kill_shell is owner-scoped, so naming
# another scope's ids would send the caller in circles.
mine = [s.shell_id for s in live if s.owner == owner]
others = len(live) - len(mine)
if mine:
detail = f"In your scope: {', '.join(mine)} — stop one with kill_shell"
if others:
detail += f"; {others} more belong to other agents"
detail += "."
else:
detail = (
f"All {others} belong to other agents' scopes and end when "
"those agents finish; wait and retry."
)
raise TooManyShellsError(
f"Background shell limit reached ({self._max_shells} running). {detail}"
)
@staticmethod
def _drain(pipe: Any, shell: BackgroundShell, is_stderr: bool) -> None:
drain_pipe_lines(
pipe, lambda line: shell._append(f"[stderr] {line}" if is_stderr else line)
)
def _wait_for_exit(
self,
shell: BackgroundShell,
out_thread: threading.Thread,
err_thread: threading.Thread,
) -> None:
"""Waiter thread: block on the leader, then tear down the group.
The kill-on-exit is what keeps the #816 guarantee: a child the
command backgrounded dies with the command, and the drains hit EOF
promptly instead of hanging on an inherited pipe write-end.
"""
shell.proc.wait()
with contextlib.suppress(OSError, ProcessLookupError):
os.killpg(shell.pgid, signal.SIGKILL)
out_thread.join(timeout=_DRAIN_JOIN_TIMEOUT_S)
err_thread.join(timeout=_DRAIN_JOIN_TIMEOUT_S)
if out_thread.is_alive() or err_thread.is_alive():
log.warning("bg_shell.drain_leaked", shell_id=shell.shell_id, pid=shell.pid)
with contextlib.suppress(OSError):
os.unlink(shell._script_path)
with shell.lock:
shell.exit_code = shell.proc.returncode
shell.status = "killed" if shell._killed else "completed"
notify = not shell._killed
with self._lock:
self._exit_counter += 1
shell._exit_seq = self._exit_counter
log.info(
"bg_shell.exited",
shell_id=shell.shell_id,
exit_code=shell.exit_code,
status=shell.status,
)
self._prune_exited()
if notify and self._on_exit is not None:
try:
self._on_exit(shell)
except Exception:
log.warning("bg_shell.on_exit_failed", shell_id=shell.shell_id, exc_info=True)
def _prune_exited(self) -> None:
"""Drop the OLDEST-EXITED records past ``max_exited_records``.
Exited records are kept so the model can read a finished shell's
output later, but a workstream that backgrounds thousands of short
jobs must not accumulate them (each can pin ``max_buffer_chars`` of
buffer). Eviction sorts on exit order, NOT spawn order the shell
whose exit triggered this prune is by definition the newest-exited
and therefore never its own victim (its exit notice and unread
output survive). An exited shell whose ``_exit_seq`` isn't
assigned yet (waiter mid-transition) sorts as newest for the same
reason.
"""
with self._lock:
if self._closed:
return
exited = sorted(
(s for s in self._shells.values() if s.status != "running"),
key=lambda s: s._exit_seq if s._exit_seq is not None else float("inf"),
)
for stale in exited[: max(0, len(exited) - self._max_exited_records)]:
self._shells.pop(stale.shell_id, None)
# -- Lookup / reads ------------------------------------------------------
def _get(self, shell_id: str, owner: str | None) -> BackgroundShell:
with self._lock:
shell = self._shells.get(shell_id)
if shell is not None and shell.owner == owner:
return shell
visible = [
f"{s.shell_id} ({s.status})" for s in self._shells.values() if s.owner == owner
]
known = (
f" Known shells: {', '.join(visible)}." if visible else " No background shells exist."
)
raise UnknownShellError(f"No background shell with id '{shell_id}'.{known}")
def has(self, shell_id: str) -> bool:
with self._lock:
return shell_id in self._shells
def shells(self, owner: str | None = None) -> list[BackgroundShell]:
"""Snapshot of the given scope's shells, in spawn order."""
with self._lock:
return [s for s in self._shells.values() if s.owner == owner]
def read(
self, shell_id: str, *, owner: str | None = None, filter_pattern: str | None = None
) -> ShellRead:
"""Return output produced since the last read of ``shell_id``.
``filter_pattern`` (a regex, ``search`` semantics per line the
tool-facing ``filter`` arg) narrows what is RETURNED, not what is
consumed: on a successful read the cursor advances past the whole
delta. A failed or timed-out filter consumes NOTHING the model
can retry without the filter and still get its output. Raises
:class:`UnknownShellError` outside the caller's scope, ``re.error``
for a bad pattern, and :class:`FilterTimeoutError` for a pattern
that blows the time bound (catastrophic backtracking).
"""
shell = self._get(shell_id, owner)
pattern: re.Pattern[str] | None = None
if filter_pattern:
if len(filter_pattern) > _MAX_FILTER_PATTERN_CHARS:
raise re.error( # noqa: TRY003 — mirrors re.compile's own error type
f"filter pattern too long ({len(filter_pattern)} chars, "
f"max {_MAX_FILTER_PATTERN_CHARS})"
)
pattern = re.compile(filter_pattern)
# Serialize the whole pass: concurrent reads of one shell (a
# parallel tool batch) would otherwise snapshot the same cursor and
# each return the full delta as "new".
with shell.read_serial:
delta, gap, new_cursor, status, exit_code = shell._snapshot_delta()
clipped = 0
if pattern is None:
shown = delta
else:
# Raises FilterTimeoutError / FilterExecError BEFORE the
# commit below — a failed filter consumes nothing.
shown = _filter_lines_bounded(pattern, delta, shell.shell_id)
clipped = sum(1 for ln in delta if len(ln) > _FILTER_MAX_LINE_CHARS)
shell._commit_cursor(new_cursor)
return ShellRead(
shell_id=shell.shell_id,
status=status,
exit_code=exit_code,
lines=shown,
new_line_count=len(delta),
dropped_lines=gap,
clipped_lines=clipped,
)
# -- Termination ---------------------------------------------------------
@staticmethod
def _signal_group(shell: BackgroundShell) -> None:
"""SIGKILL the shell's group IFF its leader is still running.
The liveness guard is load-bearing: a completed shell's ``pgid`` is
an hours-stale snapshot the OS may have recycled to an unrelated
process group signalling it unconditionally would let the
auto-approved ``kill_shell`` (or a routine ``close()``) SIGKILL
another tenant's processes. A leader that exits between the
``poll()`` and the ``killpg`` leaves the same microseconds-wide
pid-wraparound TOCTOU as the foreground tool accepted there,
accepted here. The guard also keeps a kill racing a natural exit
honest: the waiter labels the shell ``completed`` (with its real
exit code and notice) instead of ``killed``.
"""
with shell.lock:
if shell.proc.poll() is not None:
return # already exited — the waiter's own group kill ran/runs
shell._killed = True
# killpg INSIDE the lock: poll-and-signal is atomic wrt our own
# bookkeeping (nothing can observe _killed without the signal
# having been attempted). The lock is never held around other
# locks, so this cannot deadlock; the OS-level microseconds
# pid-wraparound TOCTOU is the same accepted one as always.
with contextlib.suppress(OSError, ProcessLookupError):
os.killpg(shell.pgid, signal.SIGKILL)
@staticmethod
def _join_threads(shells: list[BackgroundShell], budget_s: float) -> bool:
"""Join every shell thread under ONE shared deadline; True if all done.
The budget is total, not per-thread: teardown latency must not stack
by shell count (``close()`` can run under the server's async close
route see :data:`_CLOSE_JOIN_BUDGET_S`). Stragglers are daemons;
the caller logs and abandons them.
"""
deadline = time.monotonic() + budget_s
done = True
for shell in shells:
for t in shell._threads:
# suppress: joining a never-started Thread raises
# RuntimeError. spawn() unregisters on a start failure, so
# this is pure belt — teardown must never die on a join.
with contextlib.suppress(RuntimeError):
t.join(timeout=max(0.0, deadline - time.monotonic()))
done = done and not t.is_alive()
return done
def kill(self, shell_id: str, *, owner: str | None = None) -> BackgroundShell:
"""SIGKILL ``shell_id``'s whole group; return its (updated) record.
Suppresses the exit callback the caller asked for this exit, so
there is nothing to announce. Killing an already-exited shell
signals nothing (see :meth:`_signal_group`) and returns the record
unchanged. On return the record is usually terminal; a leader in
uninterruptible sleep can still read ``running`` after the join
budget callers report that honestly rather than assuming.
"""
shell = self._get(shell_id, owner)
self._signal_group(shell)
if not self._join_threads([shell], _WAITER_JOIN_TIMEOUT_S):
log.warning("bg_shell.kill_join_timeout", shell_id=shell.shell_id, pid=shell.pid)
return shell
def reap(self, *, owner: str | None) -> None:
"""Kill every shell belonging to ``owner`` and drop their records.
Used by the task_agent teardown: a sub-agent's shells are bound to
the sub-agent's lifetime (never handed to the parent), and dropping
the records keeps dead ``bash_N`` handles from cluttering scope
listings. Suppresses the exit callback for the shells it kills,
same as :meth:`kill` teardown is the caller's own act, there is
nothing to announce.
"""
with self._lock:
mine = [s for s in self._shells.values() if s.owner == owner]
for shell in mine:
self._signal_group(shell)
if not self._join_threads(mine, _WAITER_JOIN_TIMEOUT_S):
log.warning("bg_shell.reap_join_timeout", owner=owner or "")
with self._lock:
for shell in mine:
self._shells.pop(shell.shell_id, None)
def signal_all(self) -> None:
"""SIGKILL every live shell's group WITHOUT joining or unregistering.
The instant half of teardown, separated so multi-session frontends
can bound their total exit latency: signal every session's groups
first (microseconds each), then pay the join budgets or, on an
impatient Ctrl-C, signal alone still guarantees no process outlives
the frontend even though the joins are skipped. Liveness-guarded
per shell (:meth:`_signal_group`), so completed shells' stale pgids
are never touched. Idempotent; :meth:`close` remains the complete
teardown.
"""
with self._lock:
shells = list(self._shells.values())
for shell in shells:
self._signal_group(shell)
def close(self) -> None:
"""Kill everything, join threads under a total budget, refuse spawns.
Idempotent; called from ``ChatSession.close()`` (the funnel every
workstream-teardown path runs through). Signals are issued to all
groups first (instant), then ONE shared join budget covers every
thread a pathological shell (escaped-group grandchild holding the
pipes, D-state leader) delays teardown by at most
:data:`_CLOSE_JOIN_BUDGET_S`, with the stragglers logged and left
to die as daemons. Records are dropped so a queued exit notice's
``valid_until`` predicate (``has(shell_id)``) goes stale and the
drain discards it nobody is left to read it.
"""
with self._lock:
if self._closed:
return
self._closed = True
shells = list(self._shells.values())
for shell in shells:
self._signal_group(shell)
if not self._join_threads(shells, _CLOSE_JOIN_BUDGET_S):
leaked = [t.name for s in shells for t in s._threads if t.is_alive()]
log.warning("bg_shell.close_join_timeout", leaked=",".join(leaked))
with self._lock:
self._shells.clear()
+7 -4
View File
@@ -26,7 +26,7 @@ from typing import TYPE_CHECKING, Any
from turnstone.core import session_worker
from turnstone.core.log import get_logger
from turnstone.core.nudge_queue import USER_DRAIN, NudgeQueue
from turnstone.core.nudge_queue import WAKE_PENDING, NudgeQueue
from turnstone.core.workstream import WorkstreamState
if TYPE_CHECKING:
@@ -75,7 +75,7 @@ def wake_workstream_if_pending(ws: Workstream, *, trigger: str = "unspecified")
queue at its own seams (``ATTENTION``/``THINKING``/``RUNNING``
all imply a live worker), and ``ERROR`` stays parked for the
operator rather than burning inference unattended.
* nothing drainable under ``USER_DRAIN`` tool-only entries
* nothing gate-eligible under ``WAKE_PENDING`` tool-only/quiet entries
belong to the next tool-result seam, not a synthetic empty user
turn (``deliver_wake_nudge_from_queue`` would no-op on them).
@@ -102,7 +102,10 @@ def wake_workstream_if_pending(ws: Workstream, *, trigger: str = "unspecified")
if session is None or ws._closed or ws.state is not WorkstreamState.IDLE:
return False
nudge_queue = getattr(session, "_nudge_queue", None)
if not isinstance(nudge_queue, NudgeQueue) or not nudge_queue.has_pending(USER_DRAIN):
# Gate on WAKE_PENDING, not USER_DRAIN: ``"quiet"`` entries (external
# events demoted by a user cancel) deliver at the next legitimate seam
# but must never themselves wake the workstream the user just stopped.
if not isinstance(nudge_queue, NudgeQueue) or not nudge_queue.has_pending(WAKE_PENDING):
return False
deferred = False
@@ -135,7 +138,7 @@ class IdleNudgeWatcher:
:func:`wake_workstream_if_pending` (the shared gate see its
docstring for the full gate order). If the workstream's
:class:`NudgeQueue` has any drainable entry for the wake's drain
filter (``USER_DRAIN`` channels ``"user"`` or ``"any"``), the
gate (``WAKE_PENDING`` channels ``"user"`` or ``"any"``), the
gate dispatches via ``session_worker.send`` with a no-op
``enqueue`` callback. Tool-only entries don't fire the wake —
they belong to the next tool-result seam, not a synthetic empty
+59 -6
View File
@@ -15,7 +15,7 @@ import re
import threading
import time
import uuid
from dataclasses import dataclass, field
from dataclasses import dataclass, field, fields, replace
from functools import partial
from pathlib import Path
from typing import TYPE_CHECKING, Any
@@ -30,7 +30,7 @@ from turnstone.core.log import get_logger
if TYPE_CHECKING:
from collections.abc import Callable
from turnstone.core.providers._protocol import LLMProvider
from turnstone.core.providers._protocol import LLMProvider, ModelCapabilities
log = get_logger(__name__)
@@ -844,6 +844,33 @@ def _positive_window(*candidates: Any, floor: int = _DEFAULT_JUDGE_CONTEXT_WINDO
return floor
def _resolve_model_capabilities(provider: LLMProvider, model: str, cfg: Any) -> ModelCapabilities:
"""Provider base capabilities with a model definition's ``capabilities``
overrides applied the same lowering ``ChatSession._resolve_capabilities``
performs for the session, utility, and sub-agent completion lanes.
The judges are the only completion callers that live outside ``ChatSession``,
so they cannot reach ``self._resolve_capabilities``; this mirrors it so a
judge alias honors operator-declared capabilities (effort passthrough, tool
support, temperature, verbosity) exactly like the main loop. ``cfg`` is the
alias's ``ModelConfig``; a missing or non-dict ``capabilities`` is ignored
rather than raised capability resolution must never crash a judge turn.
It does NOT fold in ``ModelConfig.context_window``: that is a separate field,
not part of the capabilities JSON, and the caller sizes the judge's window
budget off it directly (the static caps table reports 200000 for local
models, which would silently over-budget them).
"""
caps = provider.get_capabilities(model)
overrides = getattr(cfg, "capabilities", None)
if isinstance(overrides, dict) and overrides:
names = {f.name for f in fields(type(caps))}
applied = {k: v for k, v in overrides.items() if k in names}
if applied:
caps = replace(caps, **applied)
return caps
def honest_truncate(text: str, budget: int) -> str:
"""Return *text* untouched when it fits *budget* characters, otherwise the
leading ``budget`` characters followed by an explicit note of exactly how
@@ -971,13 +998,21 @@ class IntentJudge:
session_provider: LLMProvider,
session_client: Any,
session_model: str,
context_window: int = 200_000,
session_capabilities: ModelCapabilities | None = None,
rule_registry: Any | None = None,
model_registry: Any | None = None,
) -> None:
self._config = config
self._context_window = context_window
self._rule_registry = rule_registry
# The caller (ChatSession) resolves the session model's real caps from
# _get_capabilities (config/registry-aware) and passes them in; they are
# this judge's wire capabilities and window when it inherits the session
# model. The window is taken ONLY from these resolved caps (else a
# floor) — NEVER provider.get_capabilities(), whose static 200000 for a
# local model would blind the budget to overflow.
session_window = (
session_capabilities.context_window if session_capabilities is not None else None
)
# Resolve judge model via ModelRegistry alias, otherwise self-
# consistency on the session model. ``judge.model`` is alias-only
@@ -1001,6 +1036,9 @@ class IntentJudge:
self._provider.provider_name,
)
self._model = model_name
self._capabilities = _resolve_model_capabilities(
self._provider, self._model, model_cfg
)
# Use the registry's per-model context window, NOT
# ``provider.get_capabilities().context_window``: the static
# capability table returns 200000 for every model absent
@@ -1013,7 +1051,8 @@ class IntentJudge:
# the session ``context_window`` then a floor, so it neither
# aborts resolution nor zeroes the budgets.
self._judge_context_window = _positive_window(
getattr(model_cfg, "context_window", None), context_window
getattr(model_cfg, "context_window", None),
session_window,
)
resolved = True
except Exception:
@@ -1034,8 +1073,15 @@ class IntentJudge:
session_provider.provider_name,
)
self._model = session_model
# Wire caps: the caller's resolved session caps, or the provider's
# static table as a last resort for degraded / legacy callers.
self._capabilities = (
session_capabilities
if session_capabilities is not None
else session_provider.get_capabilities(session_model)
)
# Coerce here too, defensively against a non-positive session window.
self._judge_context_window = _positive_window(context_window)
self._judge_context_window = _positive_window(session_window)
# -- Client lifecycle helpers -------------------------------------------
@@ -1336,6 +1382,13 @@ class IntentJudge:
max_tokens=2048,
temperature=0.0,
reasoning_effort="medium",
# Thread the judge model's operator-declared capabilities
# onto the wire like every other lane — resolved from the
# judge alias's model definition, or the session model on
# fallback. Without this the provider would fall back to
# its static capability table and silently ignore the
# definition's overrides on judge calls alone.
capabilities=self._capabilities,
),
timeout=per_call_timeout,
cancel_event=cancel_event,
+101 -14
View File
@@ -18,7 +18,15 @@ This module owns the three provider-neutral lowering passes:
``deepseek_v4``, which ``json.loads`` the arguments at request-render time)
can't reject the whole request. Mutates the transient wire copy only — the
canonical trajectory keeps the raw output. See
:func:`sanitize_tool_call_arguments`.
:func:`sanitize_tool_call_arguments`. The id sibling,
:func:`restore_provider_tool_ids`, maps session-minted sub-agent tool ids
(``{parent}::r{run}s{step}::{provider_id}``) back to the provider's OWN ids
on the wire copy, so the provider-native block lane whose ``tool_use``
blocks carry the provider id verbatim, under a reasoning signature that must
never be touched stays id-consistent with the top-level mirror and the
tool results. It runs at the AGENT wire seam (``ChatSession._run_agent``'s
``_api_call``) only main-loop ids are provider-issued or uuid-filled and
already consistent with their native lane.
* **repair** (validity) synthesizing cancellation results for orphaned client
tool calls. See :func:`repair_wire_messages`.
@@ -224,7 +232,7 @@ def tool_args_preview(arguments: Any) -> str:
return _ARGS_PREVIEW_CONTROL_RE.sub(" ", redact_credentials(text))[:120]
def _legalized_arguments(arguments: Any) -> str | None:
def legalized_arguments(arguments: Any) -> str | None:
"""A wire-valid replacement for *arguments*, or ``None`` if already valid.
A raw ``dict`` (an internal shape that reached the wire seat) is serialized;
@@ -243,6 +251,33 @@ def _legalized_arguments(arguments: Any) -> str | None:
return "{}"
def legalize_tool_call_entry(tc: dict[str, Any]) -> dict[str, Any] | None:
"""A legalized copy of one wire tool-call entry, or ``None`` when its
``arguments`` are already wire-valid or its ``function`` is not a dict
(someone else's malformation to surface, not silently rename).
Emits the standard ``wire.tool_args_legalized`` breadcrumb when it fixes
an entry. The ONE per-entry legalizer: shared by
:func:`sanitize_tool_call_arguments` and the Google fidelity swap
(``providers/_google.py``), which re-applies the same floor to the raw
provider dicts it swaps over the sanitized mirror so the two seats
cannot drift on what "wire-valid" means or on the diagnosis trail.
"""
fn = tc.get("function")
if not isinstance(fn, dict):
return None
replacement = legalized_arguments(fn.get("arguments"))
if replacement is None:
return None # already wire-valid — leave byte-for-byte untouched
log.debug(
"wire.tool_args_legalized",
tool=fn.get("name", "?"),
call_id=tc.get("id", ""),
raw_preview=tool_args_preview(fn.get("arguments")),
)
return {**tc, "function": {**fn, "arguments": replacement}}
def sanitize_tool_call_arguments(messages: list[dict[str, Any]]) -> list[dict[str, Any]]:
"""Return *messages* with every assistant tool call's ``arguments`` made
wire-valid the legalize pass (see the module docstring).
@@ -265,21 +300,12 @@ def sanitize_tool_call_arguments(messages: list[dict[str, Any]]) -> list[dict[st
continue
repaired: list[dict[str, Any]] | None = None
for ci, tc in enumerate(msg["tool_calls"]):
fn = tc.get("function")
if not isinstance(fn, dict):
fixed = legalize_tool_call_entry(tc)
if fixed is None:
continue
replacement = _legalized_arguments(fn.get("arguments"))
if replacement is None:
continue # already wire-valid — leave byte-for-byte untouched
if repaired is None:
repaired = list(msg["tool_calls"])
log.debug(
"wire.tool_args_legalized",
tool=fn.get("name", "?"),
call_id=tc.get("id", ""),
raw_preview=tool_args_preview(fn.get("arguments")),
)
repaired[ci] = {**tc, "function": {**fn, "arguments": replacement}}
repaired[ci] = fixed
if repaired is not None:
if out is None:
out = list(messages)
@@ -287,6 +313,67 @@ def sanitize_tool_call_arguments(messages: list[dict[str, Any]]) -> list[dict[st
return messages if out is None else out
def restore_provider_tool_ids(
messages: list[dict[str, Any]], id_map: dict[str, str]
) -> list[dict[str, Any]]:
"""Return *messages* with session-minted sub-agent tool ids mapped back to
the provider's own ids — the id half of the legalize pass, applied at the
AGENT wire seam.
*id_map* is the per-``_run_agent`` ``{minted_id: provider_original_id}``
record built at the mint site. Rewriting assistant ``tool_calls[*].id``
and tool ``tool_call_id`` back to the provider originals makes every wire
representation agree: the provider-native ``tool_use`` block (which holds
the provider id verbatim and must never be rewritten its bytes sit under
the turn's reasoning signature), the top-level ``tool_calls`` mirror, and
the ``tool_result``. A translator that replays the native lane and one
that rebuilds from ``tool_calls`` therefore emit the same ids, so the
pairing holds on both paths. The minted id stays the internal key
(registry / DOM / recall / cancel ledger) untouched only the transient
wire copy is mapped.
Replaying the provider's own ids to the producing provider is the proven
prior behaviour, including the duplicate-ish ids a local server that
reissues per-response ids ("call_0") produces an agent run is pinned to
one provider, so the ids always return to the backend that issued them.
Ids not in the map (uuid back-fills, an unparented run that never minted)
pass through untouched; recovery is by MAP ONLY, never by string-splitting
the mint suffix (provider ids can contain surprising characters,
including the mint's own delimiter).
Copy-on-write + identity-preserving, exactly like
:func:`sanitize_tool_call_arguments`: an empty map or a conversation with
no minted id returns the same object.
"""
if not id_map:
return messages
out: list[dict[str, Any]] | None = None # copy-on-write: None until first fix
for idx, msg in enumerate(messages):
role = msg.get("role")
if role == "assistant" and msg.get("tool_calls"):
repaired: list[dict[str, Any]] | None = None
for ci, tc in enumerate(msg["tool_calls"]):
tc_id = tc.get("id")
original = id_map.get(tc_id) if isinstance(tc_id, str) else None
if original is None or original == tc_id:
continue
if repaired is None:
repaired = list(msg["tool_calls"])
repaired[ci] = {**tc, "id": original}
if repaired is not None:
if out is None:
out = list(messages)
out[idx] = {**msg, "tool_calls": repaired}
elif role == "tool":
tc_id = msg.get("tool_call_id")
original = id_map.get(tc_id) if isinstance(tc_id, str) else None
if original is not None and original != tc_id:
if out is None:
out = list(messages)
out[idx] = {**msg, "tool_call_id": original}
return messages if out is None else out
# --------------------------------------------------------------------------- #
# Fold — operator-context representation (A); runs BEFORE repair on the wire.
# --------------------------------------------------------------------------- #
+4
View File
@@ -158,6 +158,10 @@ _NUDGE_MAP: dict[str, str] = {
# consumers recognise the type.
"idle_children": "",
"watch_triggered": "",
# background_shell_exit (#817) likewise: per-fire text is composed by
# ``ChatSession._on_background_shell_exit`` and rides the shared
# external-event rail, never :func:`format_nudge`.
"background_shell_exit": "",
# participant_joined likewise carries no static body — the per-fire text
# ("<name> has joined this shared workstream…") is composed by its producer
# (``ChatSession._maybe_note_new_participant``) and emitted via
+112 -14
View File
@@ -15,7 +15,13 @@ Channels:
* ``"tool"`` only drains at tool-result seams.
* ``"any"`` drains at whichever seam fires first (used for
wake-trigger-driven nudges that should not be pinned to a
specific drain seam).
specific drain seam) AND counts toward the idle-wake gate
(:data:`WAKE_PENDING`).
* ``"quiet"`` drains at whichever seam fires first, but does NOT
count toward the idle-wake gate. A user cancel demotes pending
``"any"`` entries here: the external event (watch fire,
background-shell exit) is still delivered at the next seam, but it
must not wake the workstream the user just stopped.
Drain preserves FIFO order; non-matching entries stay queued. Each
entry can carry an optional ``valid_until`` predicate that drain
@@ -40,19 +46,38 @@ if TYPE_CHECKING:
log = get_logger(__name__)
Channel = Literal["user", "tool", "any"]
_VALID_CHANNELS: frozenset[str] = frozenset({"user", "tool", "any"})
Channel = Literal["user", "tool", "any", "quiet"]
_VALID_CHANNELS: frozenset[str] = frozenset({"user", "tool", "any", "quiet"})
# Module-level filter constants — most callers want one of these and
# pre-allocating spares us a frozenset construction at every drain seam.
USER_DRAIN: frozenset[str] = frozenset({"user", "any"})
TOOL_DRAIN: frozenset[str] = frozenset({"tool", "any"})
USER_DRAIN: frozenset[str] = frozenset({"user", "any", "quiet"})
TOOL_DRAIN: frozenset[str] = frozenset({"tool", "any", "quiet"})
# The idle-wake GATE (``IdleNudgeWatcher``): which pending channels justify
# waking an idle workstream. Deliberately excludes ``"quiet"`` — entries a
# user cancel demoted must ride the next legitimate seam/wake, never cause
# one, or Stop is followed seconds later by an autonomous resume.
WAKE_PENDING: frozenset[str] = frozenset({"user", "any"})
# The quiet channel, named once: the demotion target for external events a
# user cancel must not let re-wake the workstream, and the ride-along drain
# the wake path uses after its WAKE_PENDING pass.
QUIET_CHANNEL: Channel = "quiet"
QUIET_DRAIN: frozenset[str] = frozenset({QUIET_CHANNEL})
class _Entry(NamedTuple):
class Entry(NamedTuple):
"""One queued nudge. Public so consumers of
:meth:`NudgeQueue.drain_entries` can give entries back via
:meth:`NudgeQueue.requeue` which preserves ``valid_until`` AND the
original ``seq`` (a plain :meth:`NudgeQueue.enqueue` would assign a
fresh seq and re-order a recovered older notice after newer events).
``seq`` is the queue-global insertion number multi-channel drains sort
on to restore chronology."""
nudge_type: str
text: str
channel: Channel
seq: int = 0
valid_until: Callable[[], bool] | None = None
# Producer-supplied optional fields that ride alongside ``text`` when
# drained — used by ``watch_triggered`` to carry ``watch_name`` /
@@ -70,7 +95,8 @@ class NudgeQueue:
"""Single-session FIFO queue with channel-tagged entries."""
def __init__(self) -> None:
self._items: deque[_Entry] = deque()
self._items: deque[Entry] = deque()
self._seq = 0
self._lock = threading.Lock()
def enqueue(
@@ -105,7 +131,8 @@ class NudgeQueue:
if channel not in _VALID_CHANNELS:
raise ValueError(f"channel={channel!r}; expected one of {sorted(_VALID_CHANNELS)}")
with self._lock:
self._items.append(_Entry(nudge_type, text, channel, valid_until, metadata))
self._seq += 1
self._items.append(Entry(nudge_type, text, channel, self._seq, valid_until, metadata))
def drain(
self, channels: frozenset[str] | set[str]
@@ -122,6 +149,15 @@ class NudgeQueue:
without delivering it. Already-removed-from-queue either way
dropped entries don't ride a future drain.
"""
return [(e.nudge_type, e.text, e.metadata) for e in self.drain_entries(channels)]
def drain_entries(self, channels: frozenset[str] | set[str]) -> list[Entry]:
"""Like :meth:`drain` but returns the surviving :class:`Entry`
records whole ``seq`` for cross-channel chronology merges and
``valid_until`` so a consumer that must give an entry back (the
wake path's failed-send re-enqueue) can do so without stripping
its staleness predicate.
"""
with self._lock:
if not self._items:
return []
@@ -131,10 +167,10 @@ class NudgeQueue:
# ``USER_DRAIN`` / ``TOOL_DRAIN`` (channel + "any") and
# most queues hold only one channel's entries at a time.
if all(entry.channel in channels for entry in self._items):
candidates: list[_Entry] = list(self._items)
candidates: list[Entry] = list(self._items)
self._items = deque()
else:
kept: deque[_Entry] = deque()
kept: deque[Entry] = deque()
candidates = []
for entry in self._items:
if entry.channel in channels:
@@ -150,14 +186,14 @@ class NudgeQueue:
# every child closed) and logs at ``info``; a raised exception
# is a wiring bug (predicate is misbehaving) and stays at
# ``warning`` with ``exc_info`` so the traceback surfaces.
out: list[tuple[str, str, dict[str, Any] | None]] = []
out: list[Entry] = []
for entry in candidates:
if entry.valid_until is None:
out.append((entry.nudge_type, entry.text, entry.metadata))
out.append(entry)
continue
try:
if entry.valid_until():
out.append((entry.nudge_type, entry.text, entry.metadata))
out.append(entry)
continue
log.info(
"nudge_queue.predicate_dropped",
@@ -187,12 +223,74 @@ class NudgeQueue:
return len(self._items)
def clear(self) -> int:
"""Drop every entry; return the count cleared. Used in cancel paths."""
"""Drop every entry regardless of channel; return the count cleared.
No longer on the cancel path abandoned generations use
:meth:`clear_channels` + :meth:`demote_channel` so external events
survive. Kept for tests and for full-reset callers that truly mean
"everything".
"""
with self._lock:
n = len(self._items)
self._items.clear()
return n
def requeue(self, entry: Entry, *, channel: Channel | None = None) -> None:
"""Give a drained :class:`Entry` back to the queue, KEEPING its seq.
A plain :meth:`enqueue` would assign a fresh (higher) seq, so a
failed delivery's re-queued OLDER notice would sort after events
that arrived during the failed attempt running poll counters
backwards at the next seq-merged wake. Insertion is positioned by
seq so plain FIFO drains stay chronological too. ``channel``
overrides the entry's channel (the wake path demotes ``"any"`` →
``"quiet"``); ``valid_until`` and ``metadata`` ride unchanged.
"""
dst = channel if channel is not None else entry.channel
if dst not in _VALID_CHANNELS:
raise ValueError(f"channel={dst!r}; expected one of {sorted(_VALID_CHANNELS)}")
restored = entry._replace(channel=dst)
with self._lock:
for i, existing in enumerate(self._items):
if existing.seq > restored.seq:
self._items.insert(i, restored)
return
self._items.append(restored)
def demote_channel(self, src: Channel, dst: Channel) -> int:
"""Atomically re-tag every ``src``-channel entry as ``dst``; return
the count. Order, text, metadata and ``valid_until`` are preserved
only drain/wake eligibility changes. The cancel path uses this to
take ``"any"`` entries out of the idle-wake gate ( ``"quiet"``)
without dropping the external events they announce.
"""
if dst not in _VALID_CHANNELS:
raise ValueError(f"channel={dst!r}; expected one of {sorted(_VALID_CHANNELS)}")
with self._lock:
demoted = 0
for i, entry in enumerate(self._items):
if entry.channel == src:
self._items[i] = entry._replace(channel=dst)
demoted += 1
return demoted
def clear_channels(self, channels: frozenset[str] | set[str]) -> int:
"""Drop entries whose channel is in ``channels``; return the count.
The abandoned-generation paths use this instead of :meth:`clear`:
``"tool"``/``"user"`` advisories are generation-scoped commentary
(a stale ``repeat`` nudge must not bleed into the next send), but
``"any"``-channel entries are EXTERNAL events a watch fire or a
background-shell exit that happened during the doomed generation
still happened, and dropping it would silently break the "you will
be notified" contract those producers promised the model.
"""
with self._lock:
kept = deque(e for e in self._items if e.channel not in channels)
n = len(self._items) - len(kept)
self._items = kept
return n
def count_by_type(self, nudge_type: str, channel: Channel | None = None) -> int:
"""Return the number of queued entries matching ``nudge_type``.
+32 -10
View File
@@ -50,8 +50,8 @@ from turnstone.core.deadline import (
)
from turnstone.core.judge import (
_CHARS_PER_TOKEN,
_DEFAULT_JUDGE_CONTEXT_WINDOW,
_positive_window,
_resolve_model_capabilities,
)
from turnstone.core.log import get_logger
@@ -59,7 +59,7 @@ if TYPE_CHECKING:
import threading
from turnstone.core.judge import JudgeConfig
from turnstone.core.providers._protocol import LLMProvider
from turnstone.core.providers._protocol import LLMProvider, ModelCapabilities
log = get_logger(__name__)
@@ -247,7 +247,7 @@ class OutputGuardJudge:
Construction resolves the configured ``judge.output_guard_model``
alias inline; on resolution failure (alias unset or unknown) the
session model is used as a fallback. Mirrors :class:`IntentJudge`'s
own resolution at ``judge.py:917-960``.
own alias resolution.
The HTTP client is lazy-initialised on the first ``evaluate()`` call
and reused for the lifetime of the judge instance see
@@ -267,16 +267,23 @@ class OutputGuardJudge:
session_client: Any,
session_model: str,
model_registry: Any | None = None,
context_window: int = _DEFAULT_JUDGE_CONTEXT_WINDOW,
session_capabilities: ModelCapabilities | None = None,
) -> None:
self._config = config
# Alias resolution mirrors IntentJudge.__init__ at judge.py:917-960.
# Caller's resolved session-model caps (config/registry-aware): the wire
# capabilities + window when this judge inherits the session model, and
# the alias path's window fallback. The window comes ONLY from these
# (else a floor), never provider.get_capabilities() — see below.
session_window = (
session_capabilities.context_window if session_capabilities is not None else None
)
# Alias resolution mirrors IntentJudge.__init__.
# An empty / unset alias falls through to the session model silently;
# a set-but-unknown alias logs a warning and also falls through.
# Judge model's context window drives the oversize-output guard in
# ``evaluate``. It comes from the registry's ModelConfig on the alias
# path and the session's real window (``context_window``, resolved by
# the caller from _get_capabilities) on the fallback path — NEVER
# path and the session's real window (``session_capabilities``, resolved
# by the caller from _get_capabilities) on the fallback path — NEVER
# ``provider.get_capabilities()``, which returns a static 200000 for
# every model absent from its table (i.e. every local / self-hosted
# judge), so a guard keyed off it would never trip for the small-window
@@ -296,8 +303,12 @@ class OutputGuardJudge:
)
self._model = model_name
self._judge_model_alias = config.output_guard_model
self._capabilities = _resolve_model_capabilities(
self._provider, self._model, model_cfg
)
self._judge_context_window = _positive_window(
getattr(model_cfg, "context_window", None), context_window
getattr(model_cfg, "context_window", None),
session_window,
)
resolved = True
except Exception:
@@ -321,12 +332,19 @@ class OutputGuardJudge:
)
self._model = session_model
self._judge_model_alias = ""
# Wire caps: the caller's resolved session caps, or the provider's
# static table as a last resort for degraded / legacy callers.
self._capabilities = (
session_capabilities
if session_capabilities is not None
else session_provider.get_capabilities(session_model)
)
# Session-model fallback: use the session's real context window
# (the caller resolved it from _get_capabilities, config/registry-
# aware) — NOT provider.get_capabilities(), which reports 200000 for
# a local session model and would leave the guard blind to overflow,
# the very failure this fixes. Mirrors IntentJudge's fallback.
self._judge_context_window = _positive_window(context_window)
self._judge_context_window = _positive_window(session_window)
# Lazy-init in _create_client(); reused across evaluate() calls.
# Session swaps the entire OutputGuardJudge on credential / model
@@ -341,7 +359,7 @@ class OutputGuardJudge:
Reads ``base_url`` and ``api_key`` from the client and returns
the dict ``turnstone.core.providers.create_client`` accepts.
Inlined from IntentJudge's helper at ``judge.py:965-969``.
Inlined from IntentJudge's ``_extract_client_config``.
"""
base_url = str(getattr(client, "base_url", getattr(client, "_base_url", "")))
api_key = getattr(client, "api_key", "") or ""
@@ -491,6 +509,10 @@ class OutputGuardJudge:
max_tokens=512,
temperature=0.0,
reasoning_effort="low",
# Operator-declared capabilities reach the wire like every
# other lane — from the output_guard alias's definition, or
# the session model on fallback. See IntentJudge for why.
capabilities=self._capabilities,
),
timeout=timeout,
cancel_event=cancel_event,
+26 -3
View File
@@ -23,6 +23,7 @@ from typing import TYPE_CHECKING, Any
if TYPE_CHECKING:
from collections.abc import Iterator
from turnstone.core.lowering import legalize_tool_call_entry
from turnstone.core.providers._openai_chat import OpenAIChatCompletionsProvider
from turnstone.core.providers._openai_common import sanitize_messages
from turnstone.core.providers._protocol import ModelCapabilities, StreamChunk
@@ -98,9 +99,31 @@ class GoogleProvider(OpenAIChatCompletionsProvider):
# Only type=="function" is expected today; if Gemini adds
# other tool types (e.g. code_execution) they will need
# their own round-trip handling here.
raw_tcs = [b for b in pc if b.get("type") == "function"]
if raw_tcs:
msg["tool_calls"] = raw_tcs
raw_tcs = [b for b in pc if isinstance(b, dict) and b.get("type") == "function"]
# Swap ONLY when the raw lane is a faithful counterpart of
# the mirror: every raw dict carries an id (a blank id
# predates the capture-time blank-id gate — swapping it in
# would resurrect the blank id on every replay of that
# historical row), and the raw list is the same length as
# the mirror (a shorter list — a corrupted lane whose
# non-dict elements the extraction filtered — would DROP
# mirrored calls whose tool results remain in history and
# orphan them). A turn failing either check keeps the
# sanitized mirror — losing the raw lane, exactly what the
# capture-time gate now produces for new degenerate turns.
if (
raw_tcs
and len(raw_tcs) == len(msg.get("tool_calls") or [])
and all(b.get("id") for b in raw_tcs)
):
# The raw dicts carry the model's ORIGINAL arguments;
# the mirror this swap replaces may have been legalized
# upstream (lowering.sanitize_tool_call_arguments), so
# re-apply the SAME per-entry legalizer — otherwise the
# fidelity swap resurrects a malformed arguments value
# on every replay. Copy-on-write per offending entry;
# ids and ``thought_signature`` stay untouched.
msg["tool_calls"] = [legalize_tool_call_entry(b) or b for b in raw_tcs]
cleaned.append(msg)
return sanitize_messages(cleaned)
+28 -4
View File
@@ -16,7 +16,6 @@ import structlog
from turnstone.core.providers._openai_common import (
OPENAI_COMPAT_DEFAULT,
RETRYABLE_ERROR_NAMES,
apply_cache_retention,
apply_temperature_and_effort,
apply_tool_search,
extract_usage,
@@ -33,6 +32,27 @@ from turnstone.core.providers._protocol import (
)
from turnstone.core.trajectory import materialize_attachments
def _reasoning_text(obj: Any) -> str:
"""The non-canonical reasoning text off a Chat-Completions message or
streaming delta ``reasoning`` (vLLM) preferred over
``reasoning_content`` (llama.cpp, other parsers), first non-empty
STRING wins.
The type guard matters twice over: a server that puts a structured
object in ``reasoning`` must not shadow valid text sitting in
``reasoning_content``, and a non-``str`` must never leak into the
session's reasoning accumulator (``"".join(...)`` downstream). One
helper for both the streaming and non-streaming paths so the two
lanes cannot drift on precedence or guarding.
"""
for attr in ("reasoning", "reasoning_content"):
value = getattr(obj, attr, None)
if isinstance(value, str) and value:
return value
return ""
log = structlog.get_logger(__name__)
@@ -178,7 +198,6 @@ class OpenAIChatCompletionsProvider:
"stream_options": {"include_usage": True},
}
apply_temperature_and_effort(kwargs, caps, temperature, reasoning_effort)
apply_cache_retention(kwargs, model)
tools = self._apply_web_search(kwargs, caps, tools)
tools = apply_tool_search(caps, tools, deferred_names)
if tools:
@@ -232,7 +251,7 @@ class OpenAIChatCompletionsProvider:
delta = chunk.choices[0].delta
# Reasoning field (vLLM --reasoning-parser, llama.cpp)
rc = getattr(delta, "reasoning", None) or getattr(delta, "reasoning_content", None)
rc = _reasoning_text(delta)
if rc:
sc.reasoning_delta = rc
@@ -313,7 +332,6 @@ class OpenAIChatCompletionsProvider:
"stream": False,
}
apply_temperature_and_effort(kwargs, caps, temperature, reasoning_effort)
apply_cache_retention(kwargs, model)
tools = self._apply_web_search(kwargs, caps, tools)
tools = apply_tool_search(caps, tools, deferred_names)
if tools:
@@ -347,6 +365,11 @@ class OpenAIChatCompletionsProvider:
if annotations:
content = format_citations(content, annotations)
# Non-canonical reasoning text (vLLM ``--reasoning-parser``, llama.cpp
# ``reasoning_format``) — the shared extractor also serves the
# streaming delta path, so the two lanes cannot drift.
reasoning = _reasoning_text(msg)
usage = extract_usage(getattr(response, "usage", None))
result = CompletionResult(
@@ -355,6 +378,7 @@ class OpenAIChatCompletionsProvider:
finish_reason=choice.finish_reason or "stop",
usage=usage,
provider_blocks=provider_blocks,
reasoning=reasoning,
)
log.debug(
"openai.chat.response",
+30 -30
View File
@@ -173,22 +173,11 @@ OPENAI_CAPABILITIES: dict[str, ModelCapabilities] = {
supports_reasoning_replay=True,
),
# GPT-5.6 (Sol / Terra / Luna) — released 2026-07-09. The bare
# "gpt-5.6" alias routes to Sol (developers.openai.com/api/docs/guides/
# latest-model, 2026-07 check), so this catch-all row carries Sol's
# caps and also covers dated Sol snapshots ("gpt-5.6-2026-..") and the
# explicit "gpt-5.6-sol" id by longest-prefix match. Sol is the ONLY
# 5.6 tier that unlocks the new "max" reasoning effort — the first
# COMMERCIAL OpenAI model to use it (KNOB_EFFORT_ORDER already ranks
# "max" for the Anthropic lane, so the ordinal snap and effort ladder
# need no change). Sol also has a Sol-only "ultra" multi-agent mode
# that Turnstone does NOT expose (only "pro" is wired — see below).
# Default effort is "medium" like gpt-5.5; temperature is accepted only at
# reasoning_effort="none" (the "none"-in-values gate). There is NO
# gpt-5.6-pro model: "pro" is now a reasoning.mode="pro" request param,
# not a separate model id. Context window is not yet on the model page
# (limited preview); 1.05M mirrors the 5.4/5.5 lineage — override via
# the DB model definition if OpenAI publishes a different window (a
# smaller Luna window has been reported but is unconfirmed).
# "gpt-5.6" alias routes to Sol, so this catch-all row also covers the
# explicit "gpt-5.6-sol" id by longest-prefix match. Every tier supports
# the family reasoning ladder through "max", output verbosity, and
# reasoning.mode="pro"; there is no separate gpt-5.6-pro model. Default
# effort is "medium" and temperature is accepted only when effort="none".
"gpt-5.6": ModelCapabilities(
context_window=1050000,
max_output_tokens=128000,
@@ -199,32 +188,33 @@ OPENAI_CAPABILITIES: dict[str, ModelCapabilities] = {
supports_pdf=True,
supports_reasoning_replay=True,
supports_verbosity=True,
supports_pro_mode=True, # Sol-only reasoning.mode="pro"
supports_pro_mode=True,
),
# GPT-5.6 Terra — balanced tier; Sol's ladder minus "max" (Sol-only),
# so the knob's "max" snaps to the "xhigh" ceiling. No pro mode.
# GPT-5.6 Terra — balanced intelligence/cost tier.
"gpt-5.6-terra": ModelCapabilities(
context_window=1050000,
max_output_tokens=128000,
reasoning_effort_values=("none", "low", "medium", "high", "xhigh"),
reasoning_effort_values=("none", "low", "medium", "high", "xhigh", "max"),
default_reasoning_effort="medium",
supports_tool_search=True,
supports_vision=True,
supports_pdf=True,
supports_reasoning_replay=True,
supports_verbosity=True,
supports_pro_mode=True,
),
# GPT-5.6 Luna — fastest/cheapest tier; no "max" effort, no pro mode.
# GPT-5.6 Luna — cost-sensitive, high-volume tier.
"gpt-5.6-luna": ModelCapabilities(
context_window=1050000,
max_output_tokens=128000,
reasoning_effort_values=("none", "low", "medium", "high", "xhigh"),
reasoning_effort_values=("none", "low", "medium", "high", "xhigh", "max"),
default_reasoning_effort="medium",
supports_tool_search=True,
supports_vision=True,
supports_pdf=True,
supports_reasoning_replay=True,
supports_verbosity=True,
supports_pro_mode=True,
),
# O-series reasoning models
"o1": ModelCapabilities(
@@ -415,14 +405,15 @@ def apply_temperature_and_effort(
def apply_cache_retention(kwargs: dict[str, Any], model: str) -> None:
"""Enable 24-hour extended prompt cache retention for GPT-5.x models.
"""Configure the prompt-cache lifetime supported by each GPT-5 generation.
OpenAI caching is automatic (no code changes for basic caching), but
the default TTL is only 5-10 minutes. Extended retention keeps cached
KV tensors for up to 24 hours at no additional cost, which is valuable
for workstreams with bursty activity patterns.
GPT-5.6 replaces the deprecated ``prompt_cache_retention`` field with
``prompt_cache_options.ttl``; 30 minutes is currently its only accepted
minimum lifetime. Earlier GPT-5 models retain the 24-hour policy.
"""
if model.startswith("gpt-5"):
if model.startswith("gpt-5.6"):
kwargs["prompt_cache_options"] = {"ttl": "30m"}
elif model.startswith("gpt-5"):
kwargs["prompt_cache_retention"] = "24h"
@@ -439,7 +430,7 @@ def apply_cache_retention(kwargs: dict[str, Any], model: str) -> None:
# The emission sites drop unknown values with a warning instead, mirroring
# how ``model_registry`` clamps out-of-range temperature / max_tokens.
VERBOSITY_LEVELS: frozenset[str] = frozenset({"low", "medium", "high"})
REASONING_MODES: frozenset[str] = frozenset({"pro"})
REASONING_MODES: frozenset[str] = frozenset({"standard", "pro"})
def apply_verbosity(kwargs: dict[str, Any], caps: ModelCapabilities) -> None:
@@ -456,7 +447,14 @@ def apply_verbosity(kwargs: dict[str, Any], caps: ModelCapabilities) -> None:
``apply_temperature``); a value outside ``VERBOSITY_LEVELS`` is dropped
with a warning (an operator typo must not 400 every request).
"""
if not (caps.supports_verbosity and caps.verbosity):
if not caps.supports_verbosity or caps.verbosity == "":
return
if not isinstance(caps.verbosity, str):
log.warning(
"openai.responses: ignoring non-string verbosity",
value=caps.verbosity,
expected=sorted(VERBOSITY_LEVELS),
)
return
if caps.verbosity not in VERBOSITY_LEVELS:
log.warning(
@@ -818,11 +816,13 @@ def extract_usage(usage_obj: Any) -> UsageInfo | None:
if ptd is None:
ptd = getattr(usage_obj, "input_tokens_details", None)
cached = getattr(ptd, "cached_tokens", 0) if ptd is not None else 0
cache_written = getattr(ptd, "cache_write_tokens", 0) if ptd is not None else 0
return UsageInfo(
prompt_tokens=pt,
completion_tokens=ct,
total_tokens=tt if isinstance(tt, int) else (pt + ct),
cache_creation_tokens=cache_written if isinstance(cache_written, int) else 0,
cache_read_tokens=cached if isinstance(cached, int) else 0,
)
+11 -4
View File
@@ -438,7 +438,7 @@ class OpenAIResponsesProvider:
apply_temperature(kwargs, caps, temperature, reasoning_effort)
# Reasoning params → {"effort": ..., "mode": ...} (Responses format).
# "mode": "pro" (GPT-5.6 Sol) applies more model work before a single
# "mode": "pro" (GPT-5.6) applies more model work before a single
# final answer; it rides with or without an effort level (effort
# defaults to medium in pro mode), and effort still rides without a
# mode. Both are operator-declared and gated by their static
@@ -447,8 +447,14 @@ class OpenAIResponsesProvider:
effort = resolve_reasoning_effort(caps, reasoning_effort)
if effort:
reasoning["effort"] = effort
if caps.supports_pro_mode and caps.reasoning_mode:
if caps.reasoning_mode in REASONING_MODES:
if caps.supports_pro_mode and caps.reasoning_mode != "":
if not isinstance(caps.reasoning_mode, str):
log.warning(
"openai.responses: ignoring non-string reasoning mode",
value=caps.reasoning_mode,
expected=sorted(REASONING_MODES),
)
elif caps.reasoning_mode in REASONING_MODES:
reasoning["mode"] = caps.reasoning_mode
else:
log.warning(
@@ -460,7 +466,8 @@ class OpenAIResponsesProvider:
kwargs["reasoning"] = reasoning
apply_verbosity(kwargs, caps)
apply_cache_retention(kwargs, model)
if not self._compat:
apply_cache_retention(kwargs, model)
return kwargs
# -- streaming -----------------------------------------------------------
+20 -17
View File
@@ -59,6 +59,12 @@ class CompletionResult:
finish_reason: str = "stop"
usage: UsageInfo | None = None
provider_blocks: list[dict[str, Any]] = field(default_factory=list)
# Non-canonical reasoning text surfaced by Chat-Completions-lane servers
# (vLLM ``--reasoning-parser``, llama.cpp ``reasoning_format``) — the
# non-streaming twin of ``StreamChunk.reasoning_delta``. Lanes whose
# reasoning rides ``provider_blocks`` natively (Anthropic ``thinking``,
# OpenAI Responses ``reasoning`` items) leave it empty.
reasoning: str = ""
@dataclass(frozen=True)
@@ -102,23 +108,6 @@ class ModelCapabilities:
# this False: there, an empty values list means the model has no
# effort control at all (o1-mini) and the param must be omitted.
effort_passthrough: bool = False
# Responses-API output-length control (GPT-5 family): "low"/"medium"/
# "high", separate from reasoning effort. ``supports_verbosity`` is the
# static capability; ``verbosity`` is the operator-declared value
# (model-definition capabilities JSON, merged via
# ``ChatSession._resolve_capabilities``), "" = omit. Nests under
# ``text.verbosity`` on the Responses wire (a top-level ``verbosity``
# 400s there); the Chat/compat lane never emits it. A value set on a
# model whose ``supports_verbosity`` is False is dropped, not sent.
supports_verbosity: bool = False
verbosity: str = ""
# Responses-API ``reasoning.mode`` (GPT-5.6 Sol): "pro" applies more
# model work before a single final answer. ``supports_pro_mode`` is the
# static capability (Sol-only); ``reasoning_mode`` is the
# operator-declared value, "" = omit (normal reasoning). There is no
# gpt-5.6-pro *model* — "pro" is this request-level mode instead.
supports_pro_mode: bool = False
reasoning_mode: str = ""
supports_web_search: bool = False
supports_tool_search: bool = False
supports_vision: bool = False
@@ -170,6 +159,20 @@ class ModelCapabilities:
rerank_threshold: float = 0.0
rerank_scale: str = ""
rerank_separated: bool = False
# Responses-API output-length control (GPT-5 family): "low"/"medium"/
# "high", separate from reasoning effort. Appended rather than inserted
# above to preserve the public dataclass constructor's positional order.
# ``supports_verbosity`` is the static capability; ``verbosity`` is the
# operator-declared value (model-definition capabilities JSON, merged via
# ``ChatSession._resolve_capabilities``), "" = omit. Nests under
# ``text.verbosity`` on the Responses wire.
supports_verbosity: bool = False
verbosity: str = ""
# Responses-API ``reasoning.mode`` for GPT-5.6. ``supports_pro_mode`` is
# the static capability; ``reasoning_mode`` is the operator-declared value,
# "" = omit (standard reasoning). There is no gpt-5.6-pro model.
supports_pro_mode: bool = False
reasoning_mode: str = ""
# The session effort knob is ORDINAL — snapping must respect this order.
+874 -175
View File
File diff suppressed because it is too large Load Diff
+8 -5
View File
@@ -926,11 +926,14 @@ class SessionUIBase:
with ``parent_call_id``. Called by the session for each sub-tool a
``_run_agent`` issues, before the tool emits anything.
The session namespaces each sub-agent's child ids by parent
(``f"{parent_call_id}::{tc_id}"``) before registering them here, so the
key is unique even for local servers that assign per-response sequential
ids (``call_0``) two task agents in the parent's 4-wide pool can't
collide and mis-nest steps."""
The session mints each sub-agent child id session-unique
(``f"{parent_call_id}::r{run}s{step}::{tc_id}"``) before registering
it here: the run tag de-collides agent runs (concurrent in the
parent's 4-wide pool, or sequential runs whose PARENT id a local
server reused), and the step tag de-collides that server's
per-response sequential sub-tool ids (``call_0``) across the SAME
agent's turns — one key names one call, so steps can't mis-nest or
collapse."""
if not child_call_id or not parent_call_id:
return
with self._agent_children_lock:
+4
View File
@@ -125,6 +125,10 @@ SYSTEM_TURN_SOURCES: Final = frozenset(
"compaction_pending",
"idle_children",
"watch_triggered",
# Background-shell exit notice (#817) — rides the same external-event
# rail as ``watch_triggered``; carries ``shell_id`` / ``command`` /
# ``exit_code`` / ``unread_lines`` metadata.
"background_shell_exit",
"participant_joined",
}
)
+24
View File
@@ -4179,6 +4179,30 @@ async def _lifespan(app: Starlette) -> AsyncGenerator[None, None]:
state_writer = getattr(app.state, "state_writer", None)
if state_writer is not None:
await asyncio.to_thread(state_writer.shutdown)
# Reap every loaded session's background shells (#817) before MCP
# teardown — a GRACEFUL server shutdown must not orphan detached
# process groups (the leaked-server class #816 removed; a hard crash
# remains the documented acceptance). Two phases like the CLI exit:
# signal every session's shells first (instant — after this nothing
# can outlive us), then pay the bounded per-session join budgets off
# the event loop. Session close() also removes MCP listeners, hence
# the ordering before mcp_client.shutdown().
mgr = WebUI._workstream_mgr
if mgr is not None:
loaded = [(ws.id, ws.session) for ws in mgr.list_all() if ws.session is not None]
for _ws_id, session in loaded:
with contextlib.suppress(Exception):
session._background_shells.signal_all()
def _close_loaded() -> None:
for ws_id, session in loaded:
try:
session.close()
except Exception:
log.exception("server.session_close_failed", ws_id=ws_id[:8])
if loaded:
await asyncio.to_thread(_close_loaded)
# health_registry is stateless (no background threads) — nothing to stop
if app.state.mcp_client:
app.state.mcp_client.shutdown()
+2 -2
View File
@@ -653,7 +653,7 @@ class Pane {
if (!el) {
let target = this._toolRow(callId);
if (!target) {
// A namespaced sub-agent child id ("<parent>::<id>") whose row hasn't
// A minted sub-agent child id ("<parent>::r{run}s{step}::<id>") whose row hasn't
// nested yet must NOT graft its stream onto the last top-level batch —
// that mislabels a sub-tool's output as a main-harness tool's. Its row
// arrives via the orphan flush; skip the chunk until then.
@@ -3491,7 +3491,7 @@ class Pane {
}
let target = this._toolRow(callId);
if (!target) {
// A namespaced sub-agent child id ("<parent>::<id>") whose row hasn't
// A minted sub-agent child id ("<parent>::r{run}s{step}::<id>") whose row hasn't
// nested yet must NOT graft its output onto the last top-level batch row
// — that mislabels a sub-tool's result as a main-harness tool's. Its row
// arrives via the orphan flush; skip until then.
+5 -1
View File
@@ -1,6 +1,6 @@
{
"name": "bash",
"description": "Execute a bash command and return stdout + stderr. Use for running programs, git, tests, system commands, installing packages, etc. Environment questions ('What Python version?', 'Is X installed?') are tool-use tasks — e.g. bash(command='python --version'). For file creation use write_file instead; for man pages use man instead. Long output is truncated (head+tail preserved, middle elided). Stderr lines prefixed with [stderr].",
"description": "Execute a bash command and return stdout + stderr. Use for running programs, git, tests, system commands, installing packages, etc. Environment questions ('What Python version?', 'Is X installed?') are tool-use tasks — e.g. bash(command='python --version'). For file creation use write_file instead; for man pages use man instead. Long output is truncated (head+tail preserved, middle elided). Stderr lines prefixed with [stderr]. Runs to completion and returns: any process the command leaves running in the background (e.g. 'server &') is terminated when the command returns — nothing persists across calls. To keep a long-lived process (dev server, watcher) running across calls, set run_in_background=true instead of using '&'.",
"parameters": {
"type": "object",
"properties": {
@@ -15,6 +15,10 @@
"stop_on_error": {
"type": "boolean",
"description": "If true, enables 'set -e' so the script exits on the first command failure. Default false. Use for multi-step scripts where intermediate failures should halt execution."
},
"run_in_background": {
"type": "boolean",
"description": "If true, start the command as a detached background shell and return immediately with a shell id (e.g. bash_1). Default false. Read new output later with bash_output(id=...); stop it with kill_shell(id=...). The shell runs until it exits, is killed, or the workstream closes; in the main session a system notice announces its exit. Inside a task agent there is no exit notice — poll bash_output — and the shell is also terminated when the agent finishes. The timeout parameter does not apply. Use for long-lived processes like dev servers — not for ordinary commands whose result you want now."
}
},
"required": ["command"]
+21
View File
@@ -0,0 +1,21 @@
{
"name": "bash_output",
"description": "Read new output from a background shell started with bash(run_in_background=true). Returns only output produced since your previous bash_output call for that shell, plus its status (running / completed / killed) and exit code once it has exited. Stderr lines are prefixed with [stderr]. Poll this to monitor a long-running process. In the main session a system notice announces when the shell exits, so you do not need to poll a shell you are merely waiting on; inside a task agent there is no notice — poll before you finish.",
"parameters": {
"type": "object",
"properties": {
"id": {
"type": "string",
"description": "Background shell id, e.g. bash_1 (returned when the shell was started)."
},
"filter": {
"type": "string",
"description": "Optional regular expression; only new lines matching it are returned. Non-matching lines in this read are consumed and will not be returned by later calls."
}
},
"required": ["id"]
},
"task_agent": true,
"auto_approve": true,
"primary_key": "id"
}
+17
View File
@@ -0,0 +1,17 @@
{
"name": "kill_shell",
"description": "Terminate a background shell started with bash(run_in_background=true), killing its whole process group. Use it when the process is no longer needed or is misbehaving. Output already produced remains readable via bash_output afterwards.",
"parameters": {
"type": "object",
"properties": {
"id": {
"type": "string",
"description": "Background shell id to terminate, e.g. bash_1."
}
},
"required": ["id"]
},
"task_agent": true,
"auto_approve": true,
"primary_key": "id"
}
Generated
+5 -5
View File
@@ -1473,7 +1473,7 @@ wheels = [
[[package]]
name = "openai"
version = "2.44.0"
version = "2.45.0"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "anyio" },
@@ -1485,9 +1485,9 @@ dependencies = [
{ name = "tqdm" },
{ name = "typing-extensions" },
]
sdist = { url = "https://files.pythonhosted.org/packages/49/f5/7c7cb955305cb41f7f3c5fd7e0e38bf6bbf2658468863d4b7b868a5cb8df/openai-2.44.0.tar.gz", hash = "sha256:68a5a5ffad82b8ff7d451c437529fb64f7c3b8123aaf0c021966a882d9e3947d", size = 988753, upload-time = "2026-06-24T20:56:02.293Z" }
sdist = { url = "https://files.pythonhosted.org/packages/78/60/d4219875289b11d2c2f7da93c36283da224a2e55865ed865ab64e0ce9217/openai-2.45.0.tar.gz", hash = "sha256:10d34ca9c5643bce775852fddbfc172505cb1d4de1ccd101696c3ecff358765d", size = 1109653, upload-time = "2026-07-09T18:02:44.091Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/ae/f4/561ed79fd94876160018a5e75254cfcb9b0e62d4dded9dcb20072e86d623/openai-2.44.0-py3-none-any.whl", hash = "sha256:0a2a3ab2e29aeda368700f662ff9ba0f9df17ba4c54577a64e08b8115a3cc0ad", size = 1366216, upload-time = "2026-06-24T20:55:58.882Z" },
{ url = "https://files.pythonhosted.org/packages/f1/b0/2291689e3ec4723fbf5bbf3b54afcd7b160f9ddc98ca7aedfd0132af5677/openai-2.45.0-py3-none-any.whl", hash = "sha256:5df105f5f8c9b711fcb9d06d2d3888cebc82506db216484c14a4e53cdf651777", size = 1629470, upload-time = "2026-07-09T18:02:42.21Z" },
]
[[package]]
@@ -2475,7 +2475,7 @@ wheels = [
[[package]]
name = "turnstone"
version = "1.7.3"
version = "1.7.4"
source = { editable = "." }
dependencies = [
{ name = "alembic" },
@@ -2544,7 +2544,7 @@ requires-dist = [
{ name = "lacme", specifier = ">=1.0.5" },
{ name = "mcp", specifier = ">=1.27,<2" },
{ name = "mypy", marker = "extra == 'dev'", specifier = ">=1.14" },
{ name = "openai", specifier = ">=2.44" },
{ name = "openai", specifier = ">=2.45" },
{ name = "pillow", specifier = ">=10" },
{ name = "psycopg", extras = ["binary"], specifier = ">=3.2" },
{ name = "pydantic", specifier = ">=2.0" },