mirror of
https://github.com/turnstonelabs/turnstone.git
synced 2026-08-13 07:22:24 -06:00
Compare commits
51 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| ee94ae8ba1 | |||
| 357d00400e | |||
| 3615f98c19 | |||
| 801d5dfb59 | |||
| a352b20786 | |||
| 6f8efaa44e | |||
| 9c90fe2722 | |||
| 1035fe05eb | |||
| 530958e06b | |||
| e136237b63 | |||
| 41e9907803 | |||
| 7c34d859b4 | |||
| 16ee4e12ef | |||
| 976c07d047 | |||
| 8da5dc3f5a | |||
| 7f0e0406b3 | |||
| 6c94514106 | |||
| 0d0fe8dd71 | |||
| 18c3301428 | |||
| 185dcc2960 | |||
| 0fe8e4106f | |||
| fd65a490dc | |||
| 3607517814 | |||
| a0e04a8588 | |||
| ffe8214cfe | |||
| 1f63f622c9 | |||
| f4701bf0f9 | |||
| 06cc184227 | |||
| 59a527f2f2 | |||
| d7941c88be | |||
| c64dc16319 | |||
| d564cee43d | |||
| 2cf23b6fe2 | |||
| 68b22adfa3 | |||
| 9289693730 | |||
| deff44bcea | |||
| 217d3a3a9b | |||
| bcf509a440 | |||
| 10f726f83d | |||
| acc262c405 | |||
| 62034378c6 | |||
| bcb8c5ab88 | |||
| fec5067fcd | |||
| b0ed67aa60 | |||
| c023272b16 | |||
| 3568a6db50 | |||
| 9bf8d5699b | |||
| c0ff00a1ff | |||
| 45010f5890 | |||
| 845df69031 | |||
| d47d528d9a |
@@ -34,7 +34,7 @@ jobs:
|
||||
|
||||
- name: Run Claude Code Review
|
||||
id: claude-review
|
||||
uses: anthropics/claude-code-action@6c0083bb7289c31716797a039b6367b3079cc46e # v1
|
||||
uses: anthropics/claude-code-action@01872ccc02bf66740207fb338a783ce028216758 # v1
|
||||
with:
|
||||
claude_code_oauth_token: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }}
|
||||
allowed_bots: 'renovate[bot]' # let Renovate PRs get reviewed
|
||||
|
||||
@@ -45,7 +45,7 @@ jobs:
|
||||
|
||||
- name: Run Claude Code
|
||||
id: claude
|
||||
uses: anthropics/claude-code-action@6c0083bb7289c31716797a039b6367b3079cc46e # v1
|
||||
uses: anthropics/claude-code-action@01872ccc02bf66740207fb338a783ce028216758 # v1
|
||||
with:
|
||||
claude_code_oauth_token: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }}
|
||||
|
||||
|
||||
@@ -54,7 +54,7 @@ jobs:
|
||||
|
||||
- name: Log in to GHCR
|
||||
if: steps.tag.outputs.skip == 'false'
|
||||
uses: docker/login-action@650006c6eb7dba73a995cc03b0b2d7f5ca915bee # v4
|
||||
uses: docker/login-action@af1e73f918a031802d376d3c8bbc3fe56130a9b0 # v4
|
||||
with:
|
||||
registry: ${{ env.REGISTRY }}
|
||||
username: ${{ github.actor }}
|
||||
@@ -78,7 +78,7 @@ jobs:
|
||||
fi
|
||||
echo "tags=${TAGS}" >> "$GITHUB_OUTPUT"
|
||||
|
||||
- uses: docker/setup-buildx-action@d7f5e7f509e45cec5c76c4d5afdd7de93d0b3df5 # v4
|
||||
- uses: docker/setup-buildx-action@bb05f3f5519dd87d3ba754cc423b652a5edd6d2c # v4
|
||||
if: steps.tag.outputs.skip == 'false'
|
||||
|
||||
- name: Build and push
|
||||
|
||||
@@ -28,3 +28,4 @@ tools/skill_audit_analysis/data/
|
||||
tools/skill_audit_analysis/output/
|
||||
design_ideas/
|
||||
.claude/
|
||||
docs/design/
|
||||
|
||||
+131
-4
@@ -13,7 +13,28 @@ stable, and the experimental line:
|
||||
- **`stable/1.6`** — patch-only (`v1.6.x`)
|
||||
- **`main`** — experimental (next major)
|
||||
|
||||
## [Unreleased]
|
||||
## [1.7.0]
|
||||
|
||||
The headline of the 1.7 line is **Personas** — operator-authored control
|
||||
over how each workstream composes its system message and capability
|
||||
envelope. The rest of the release hardens the pieces a persona leans on:
|
||||
concurrent approvals, cross-provider reasoning-effort control, cooperative
|
||||
compaction, multi-user session safety, and MCP resilience for unattended
|
||||
work.
|
||||
|
||||
> **⚠️ Before upgrading:** 1.7.0 adds Alembic migrations `062`–`065`,
|
||||
> applied automatically on first start (projects, personas, and two
|
||||
> smaller schema tidy-ups). Migration `063` creates the `personas` table
|
||||
> with its six seed personas and converts existing `creative_mode`
|
||||
> workstreams to the `writer` persona in place. The changes are additive
|
||||
> to your conversation data, but — as always — back up your storage before
|
||||
> upgrading (`pg_dump` for PostgreSQL; copy the database file for SQLite).
|
||||
|
||||
**Breaking changes at a glance** (details in the sections below): the
|
||||
`/creative` REPL toggle removed (replaced by the `writer` persona), the
|
||||
`turnstone-bootstrap` entry point renamed to `turnstone-doctor`, and the
|
||||
approval-status API/SDK field `pending_approval_details` changed from a
|
||||
single object to a list (one entry per concurrent approval cycle).
|
||||
|
||||
### Added
|
||||
|
||||
@@ -31,12 +52,106 @@ stable, and the experimental line:
|
||||
`turnstone --persona <name>`); authored in the console's new
|
||||
Governance → Personas tab (`persona.{create,read,write}` perms,
|
||||
archive-only lifecycle). See `docs/personas.md`.
|
||||
- **Projects — governed resource containers** (#724) — group workstreams
|
||||
and their resources under a project (migration `062`), with
|
||||
project-scoped memory, a per-project resources view, a project column on
|
||||
the saved list, and server-enforced private-project workstream
|
||||
visibility.
|
||||
- **Task-agent sub-harness** (#732) — a spawned task agent now runs on its
|
||||
own Turn-IR sub-harness with parent-tagged step events: its sub-tool
|
||||
steps nest inside an expandable card in the parent trajectory, its
|
||||
sub-trajectory is recallable, and each agent gets read isolation from
|
||||
its siblings.
|
||||
- **MCP static-server autonomous reconnect** (#768) — statically
|
||||
configured MCP servers are now kept live by a health loop
|
||||
(capped-jittered backoff, ping-based liveness) instead of silently
|
||||
staying dead after the first transport drop.
|
||||
- **Attachments — capability-gated client-side fallback** — when the
|
||||
active model can't natively handle an attachment, the client degrades
|
||||
gracefully (PDF → extracted text, audio → transcript) instead of
|
||||
failing the turn.
|
||||
- **Eval measurement / optimizer split** (#763, #765) — `turnstone-eval`
|
||||
is now a measure-only substrate with the prompt optimizer factored out,
|
||||
plus a new skill-adherence measurement mode.
|
||||
- **Deployment examples** — a vLLM + LiteLLM unified-memory inference
|
||||
example showing a 3-model co-resident stack with an HF loader (#686,
|
||||
#688), and an Altair + `vl-convert-python` visualization stack (#685).
|
||||
- **Concurrent approvals and a long-session frontend overhaul** (#754,
|
||||
#755, #773, #775) — the live-session frontend was reworked for long
|
||||
runs (the pipeline is wedge-proofed and its hot paths de-O(N)'d), and on
|
||||
top of it a workstream can now hold more than one tool call awaiting
|
||||
approval at a time. Each parallel batch gets its own approval cycle,
|
||||
with one card per pending call in the interactive and coordinator UIs,
|
||||
cycle-keyed tracking in Slack and Discord, and cycle-routed resolution
|
||||
across the server/console/SDK APIs; sub-agent tool gates run the
|
||||
intent-judge pipeline as their own generation. The send button no longer
|
||||
sticks disabled after a batch resolves — orphaned approval cycles are
|
||||
pruned and the app is the sole owner of the button state.
|
||||
*(BREAKING: the `pending_approval_details` field is now a list, oldest
|
||||
first.)*
|
||||
- **Reasoning-effort control on every provider lane** (#771, #774) — the
|
||||
session effort knob now reaches local backends too: it drives
|
||||
`chat_template_kwargs` on the anthropic-compatible and openai-compatible
|
||||
lanes and threads through to Gemini and xAI, alongside the commercial
|
||||
providers that handle effort natively. The console surfaces each model's
|
||||
effective effort ladder in plain words and adds an always-on
|
||||
thinking-mode option to the model form. Effort snapping is ordinal —
|
||||
it rounds up and caps at the model's ceiling rather than silently
|
||||
dropping.
|
||||
|
||||
### Changed
|
||||
|
||||
- **Skills are capability-context, not identity** (#762) — a task agent's
|
||||
identity now comes from its persona; an applied skill's body is demoted
|
||||
to capability context and moved out of the identity system message.
|
||||
Skill-body substitution is unified across every invocation context so
|
||||
the same skill renders identically whether loaded interactively, by the
|
||||
model, or inside a sub-agent.
|
||||
- **`turnstone-doctor` replaces `turnstone-bootstrap`** (#718)
|
||||
*(BREAKING)* — the setup/diagnostics entry point is renamed; update any
|
||||
scripts or service units that invoke `turnstone-bootstrap`.
|
||||
- **Honest cancellation dispositions** — cancelled or timed-out
|
||||
side-effecting tools now report an `UNKNOWN` disposition rather than a
|
||||
flat failure, tool dispositions are typed (not just prose), and a
|
||||
coordinator cancel propagates down the sub-tree.
|
||||
- **Multi-user shared-workstream context** (#750) — in a shared
|
||||
workstream, send is gated to the acting participant while a turn is in
|
||||
flight (both the interactive and coordinator surfaces), cross-user
|
||||
mid-turn interjections are blocked, and shared-workstream state plus
|
||||
fork sender attribution are now durable.
|
||||
- **Cooperative compaction** (#730) — the context budget is anchored to
|
||||
the provider's true capacity, the summary call is chunked so it can't
|
||||
overflow, and the active plan and the outstanding ask are carried across
|
||||
compaction verbatim. The `recall` tool is scoped to the compacted-away
|
||||
past.
|
||||
- **Intent judge sees the full tool arguments** (#760) — the judge's
|
||||
argument projection is no longer narrowed, so it stops issuing confident
|
||||
false denials on a partial view. The output-guard judge sources its real
|
||||
context window, and `context_window = 0` in `config.toml` now means
|
||||
auto-detect.
|
||||
|
||||
### Fixed
|
||||
|
||||
- **Compaction resume hardening** (#731) — checkpoint markers are
|
||||
persisted so resume rehydration is bounded, context-overflow on resume
|
||||
is recovered across providers, and a recognized rate-limit is no longer
|
||||
misclassified as context overflow.
|
||||
- **MCP unattended-work resilience** (#706, #742, #767) — dead-transport
|
||||
handling is completed, consented OAuth (OBO) tokens are refreshed
|
||||
proactively so autonomous runs don't strand on an expired grant, the
|
||||
Entra ID on-behalf-of impersonation flow blockers are closed (migration
|
||||
`065` adds the OIDC `oid`), and OAuth refresh failures are classified so
|
||||
a transient blip never revokes consent nor a dead grant strands the
|
||||
user.
|
||||
- **Memory writes** (#735) — save/update is a single atomic upsert, and
|
||||
writing a memory no longer recomposes the system prefix mid-session.
|
||||
|
||||
### Removed
|
||||
|
||||
- **`/creative` removed** *(BREAKING)* — the REPL toggle (and its tab
|
||||
completion) is gone; the `writer` seed persona replaces it — start a
|
||||
session with `turnstone --persona writer` or pick *Writer* in the web
|
||||
- **`/creative` removed** *(BREAKING)* — subsumed by the Personas feature
|
||||
above: the REPL toggle (and its tab completion) is gone, and the
|
||||
`writer` seed persona replaces it — start a session with
|
||||
`turnstone --persona writer` or pick *Writer* in the web
|
||||
pickers. Unlike the old fork, the writer persona composes the full
|
||||
system message, so session context and mandatory prompt policies now
|
||||
apply to prose-only sessions too. The `creative_mode` key in
|
||||
@@ -45,6 +160,18 @@ stable, and the experimental line:
|
||||
automatically, so they resume as writing sessions rather than as
|
||||
legacy defaults.
|
||||
|
||||
### Security
|
||||
|
||||
- **High-risk skill activation is gated** (#762) — a model-initiated load
|
||||
of a `high`- or `critical`-risk skill is gated and fails closed when the
|
||||
backing storage is unavailable, so an untrusted turn can't silently
|
||||
pull in a dangerous capability.
|
||||
- **Dependency security floors** — `cryptography` and `starlette` are
|
||||
pinned to security-fixed minimums.
|
||||
- **CI publish hardening** — the vendored-JS dispatch path refuses fork
|
||||
PRs, and `workflow_run` publishing is gated to same-repo tag pushes, so
|
||||
a fork can't trigger a release build.
|
||||
|
||||
## [1.6.0]
|
||||
|
||||
The first stable release of the 1.6 line — and the first under Apache 2.0.
|
||||
|
||||
+108
-9
@@ -634,8 +634,17 @@ function tool (the model always searches). Citations from `url_citation`
|
||||
annotations are formatted as footnotes. Extended prompt cache retention
|
||||
(`prompt_cache_retention: "24h"`) is enabled for GPT-5.x models at no
|
||||
additional cost. Cached token counts are extracted from
|
||||
`usage.prompt_tokens_details.cached_tokens`. Unknown models (local servers) get
|
||||
permissive defaults with `supports_vision=False` and use SearxNG for web search.
|
||||
`usage.prompt_tokens_details.cached_tokens`. Unknown models get permissive
|
||||
defaults with `supports_vision=False` and use SearxNG for web search. The
|
||||
`openai-compatible` lane never consults this table at all — on either API
|
||||
surface (the responses pin is served by a compat-mode
|
||||
`OpenAIResponsesProvider`, mirroring `AnthropicProvider(compat=True)`): a
|
||||
local server serves whatever the operator named it (vLLM
|
||||
`--served-model-name` is a free string), so a prefix collision with a cloud
|
||||
model id must not inherit that model's sampling/effort contract — every
|
||||
local model gets the plain defaults, and anything beyond them is declared on
|
||||
the model definition (capabilities JSON + `server_compat`), matching the
|
||||
`anthropic-compatible` lane.
|
||||
|
||||
**AnthropicProvider** (`_anthropic.py`): converts OpenAI-format messages to
|
||||
Anthropic content blocks, maps `system`/`developer` roles to the `system`
|
||||
@@ -798,15 +807,105 @@ model = "deepseek-ai/DeepSeek-V4-Flash"
|
||||
supports_vision = true # multimodal checkpoints only
|
||||
supports_mid_conversation_system = true # template-dependent
|
||||
context_window = 131072
|
||||
thinking_mode = "manual" # session effort knob drives the template toggle
|
||||
thinking_param = "enable_thinking" # Qwen/Gemma key; "thinking" for Granite/DeepSeek
|
||||
```
|
||||
|
||||
The reasoning toggle does NOT use Anthropic's `thinking` request param.
|
||||
Toggle it through the chat template instead: set `{"chat_template_kwargs":
|
||||
{"thinking": false}}` as extra body params in the admin Models
|
||||
server-compat section (for this provider the section shows only the
|
||||
extra-body field — server type, API surface, and thinking mode are
|
||||
openai-compatible-only knobs); the provider forwards it via the SDK's
|
||||
`extra_body`.
|
||||
Reasoning control does NOT use Anthropic's `thinking` request param —
|
||||
the levers live in the chat template, reached through
|
||||
`chat_template_kwargs` in the request body. Two channels, dynamic first:
|
||||
|
||||
* **Session effort knob (dynamic).** Set the model's thinking mode to
|
||||
"Effort-knob controlled" in the admin Models form (or
|
||||
`thinking_mode = "manual"` + `thinking_param` under
|
||||
`[models.*.capabilities]`) and the provider maps the session's
|
||||
reasoning-effort knob onto the template toggle per-request: effort
|
||||
`none` sends `{<thinking_param>: false}`, any other level sends
|
||||
`true` — the same contract as the real lane's manual mode. ("Always
|
||||
on" / `thinking_mode = "adaptive"` instead always sends `true`: the
|
||||
model self-regulates, so the knob never force-disables — mirroring
|
||||
the native adaptive branch.) The graded effort value always rides
|
||||
alongside the toggle: under `effort_param` when the operator names
|
||||
the template's key, else under the conventional fallback key
|
||||
(`reasoning_effort`) on the anthropic-compatible lane — the user's
|
||||
effort setting always reaches the wire, and a template that doesn't
|
||||
reference the kwarg ignores it. On the openai-compatible lane the
|
||||
undeclared-key case rides the flat top-level `reasoning_effort`
|
||||
param instead (the documented compat field), forwarded verbatim.
|
||||
Optional `reasoning_effort_values` / `default_reasoning_effort`
|
||||
validate the knob before it reaches the server; without declared
|
||||
values the knob is forwarded as-is. The knob is ordinal, and validation
|
||||
respects that: an off-list knob value rounds UP onto the declared
|
||||
list and a value above the ceiling rides the ceiling
|
||||
(`snap_reasoning_effort`) — asking for more effort than the model
|
||||
declares never falls back to a lower default tier. The knob's
|
||||
`none` position is forwarded verbatim when the model declares an
|
||||
explicit `none` level (gpt-5.1+, grok-4.3) — omitting it there would
|
||||
leave a reasoning-on server default (e.g. gpt-5.5's `medium`) in
|
||||
charge of a knob that promises off — and omitted otherwise; `none`
|
||||
is never a snap target for other positions.
|
||||
`default_reasoning_effort` only catches values the ordinal snap
|
||||
cannot rank (custom strings). Declare values that match the
|
||||
template's documented vocabulary: for DeepSeek-V4, which officially
|
||||
accepts `high`/`max` (Think High is the default thinking tier;
|
||||
`low`/`medium` alias to `high`, `xhigh` to `max`), a
|
||||
`("high", "max")` values list reproduces the official aliasing
|
||||
exactly — `low`/`medium` round up to `high`, `xhigh` to `max` —
|
||||
and freeform passthrough matches it too. To map an undocumented
|
||||
template, probe with per-request `chat_template_kwargs` and compare
|
||||
`input_tokens`. Setting `effort_param` also suppresses the
|
||||
flat top-level `reasoning_effort` request param on the
|
||||
openai-compatible lane — the template channel replaces it, never
|
||||
doubles it. With the default `thinking_mode = "none"` nothing is
|
||||
injected and the server's template default decides.
|
||||
|
||||
Upgrade note: before 1.7.0a7 the openai-compatible lane sent the
|
||||
toggle unconditionally `true` whenever thinking mode was enabled. A
|
||||
stored per-model `reasoning_effort = "none"` now disables thinking
|
||||
on such models — pick any real level (or clear the override) to keep
|
||||
it on. Also since 1.7.0a7 the effort level itself always reaches the
|
||||
wire on the local lanes (previously dropped unless
|
||||
`reasoning_effort_values` was declared): flat `reasoning_effort` on
|
||||
openai-compatible, the `effort_param`-or-fallback template key on
|
||||
anthropic-compatible when reasoning control is engaged.
|
||||
* **Operator pin (static).** Entries under `{"chat_template_kwargs":
|
||||
...}` in the admin Models extra-body field ride the SDK's
|
||||
`extra_body` unconditionally and win over the knob mapping on key
|
||||
collision — e.g. pin `{"enable_thinking": true}` to keep thinking on
|
||||
regardless of the session knob. (Server type and API surface remain
|
||||
openai-compatible-only knobs and stay hidden for this provider.)
|
||||
|
||||
The same knob mapping drives the `openai-compatible` lane's Chat
|
||||
Completions requests — `merge_reasoning_template_kwargs` is shared by
|
||||
both local-server lanes, so `thinking_mode`/`thinking_param`/
|
||||
`effort_param` mean the same thing whichever endpoint serves the model.
|
||||
Only the Responses API surface (native reasoning) ignores it.
|
||||
|
||||
The console surfaces this projection as an *effective effort ladder*:
|
||||
the admin model form's per-model effort select and the skill
|
||||
launch-config effort select annotate each position with what the
|
||||
request will carry, in plain words — a position whose delivered level
|
||||
matches its name stays plain ("Max"), a snapped position says so
|
||||
("Low — sends high"), the adaptive lanes' none position warns
|
||||
"thinking stays on", and budget detail lives in the tooltip. A
|
||||
position is never labeled after a sibling that shares its wire (that
|
||||
rendered "Max (= minimal)", implying a downgrade the wire doesn't
|
||||
contain). Computed server-side by `providers/effort_ladder.py` from
|
||||
the same mapping functions the providers use at request time and
|
||||
shipped on `/v1/api/models` rows (every row carries `effort_ladder`,
|
||||
empty when the capabilities column fails to parse) and
|
||||
`POST /v1/api/admin/models/effort-ladder`. The ladder describes what
|
||||
Turnstone sends — a server-side template may alias further (DeepSeek-V4
|
||||
folds `low`/`medium` into its default `high` tier).
|
||||
|
||||
The `anthropic-compatible` lane never sends Anthropic's native
|
||||
`thinking`/`output_config` params — they are not in vLLM's request
|
||||
schema. The real `anthropic` provider is unaffected: official Claude
|
||||
models keep native thinking, budget mapping, and `output_config`
|
||||
effort. A gateway fronting *real* Claude on a Messages-shaped URL
|
||||
(e.g. a LiteLLM `anthropic/` route to the Claude API) should use
|
||||
`provider = "anthropic"` with a custom `base_url`, which keeps the
|
||||
native thinking params.
|
||||
|
||||
Verified quirks of vLLM's Anthropic endpoint:
|
||||
|
||||
|
||||
@@ -0,0 +1,56 @@
|
||||
{
|
||||
"defaults": {
|
||||
"n_runs": 3
|
||||
},
|
||||
"cases": [
|
||||
{
|
||||
"id": "search-first",
|
||||
"skill": {
|
||||
"name": "search-first",
|
||||
"content": "# Search First\n\nBefore answering ANY question about where something lives in the codebase, you MUST call the `search` tool first. Never answer from memory."
|
||||
},
|
||||
"user_prompt": "Where is JWT token validation implemented in this project?",
|
||||
"expected_actions": [{ "tool": "search" }],
|
||||
"match_mode": "ordered_subset",
|
||||
"max_turns": 4
|
||||
},
|
||||
{
|
||||
"id": "test-after-edit",
|
||||
"skill": {
|
||||
"name": "test-after-edit",
|
||||
"content": "# Test After Edit\n\nAfter editing or writing ANY file, you MUST run the test suite with `python -m pytest` via bash before you finish. Do not report done until tests have run."
|
||||
},
|
||||
"user_prompt": "Add a function `clamp(x, lo, hi)` that clamps x to [lo, hi] in utils.py.",
|
||||
"setup": {
|
||||
"files": {
|
||||
"utils.py": ""
|
||||
}
|
||||
},
|
||||
"expected_actions": [
|
||||
{ "tool": "write_file" },
|
||||
{ "tool": "bash", "args_pattern": { "command": "pytest" } }
|
||||
],
|
||||
"match_mode": "ordered_subset",
|
||||
"max_turns": 8
|
||||
},
|
||||
{
|
||||
"id": "changelog-update",
|
||||
"skill": {
|
||||
"name": "changelog-update",
|
||||
"content": "# Changelog Discipline\n\nWhenever you modify a file, you MUST also append a one-line entry to CHANGELOG.md describing the change in the same task."
|
||||
},
|
||||
"user_prompt": "Fix the off-by-one so pager.py shows the last page. Edit pager.py.",
|
||||
"setup": {
|
||||
"files": {
|
||||
"pager.py": "def last_page(total_items, per_page):\n # off-by-one: drops the final partial page\n return total_items // per_page\n",
|
||||
"CHANGELOG.md": "# Changelog\n"
|
||||
}
|
||||
},
|
||||
"expected_actions": [
|
||||
{ "tool": "edit_file", "args_pattern": { "path": "CHANGELOG.md" } }
|
||||
],
|
||||
"match_mode": "subset",
|
||||
"max_turns": 8
|
||||
}
|
||||
]
|
||||
}
|
||||
+1
-1
@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
|
||||
|
||||
[project]
|
||||
name = "turnstone"
|
||||
version = "1.7.0a6"
|
||||
version = "1.7.0"
|
||||
description = "Multi-node AI orchestration platform with tool use, agent routing, and cluster simulation."
|
||||
readme = "README.md"
|
||||
license = "Apache-2.0"
|
||||
|
||||
@@ -11201,6 +11201,7 @@
|
||||
}
|
||||
],
|
||||
"default": null,
|
||||
"description": "Inline BASE override \u2014 required. Every persona must name a prompt source; built-in file-backed personas are seeded by migration, not created here, so an operator-created persona must supply base_prompt.",
|
||||
"title": "Base Prompt"
|
||||
},
|
||||
"tool_allowlist": {
|
||||
@@ -11259,7 +11260,7 @@
|
||||
"type": "object"
|
||||
},
|
||||
"UpdatePersonaRequest": {
|
||||
"description": "PATCH body \u2014 absent fields are left unchanged.\n\nExplicit ``null`` is meaningful only on the two resettable fields:\n``base_prompt: null`` clears the override back to the kind's stock\nBASE, and ``tool_allowlist: null`` resets to unrestricted. ``null``\non the boolean flags or ``applies_to_kinds`` is ignored (treated as\nabsent), so a client serializing unset optionals as null cannot\narchive a persona or flip levers by accident.\n\nArchive = ``{\"enabled\": false}``; default flip = ``{\"is_default\": true}``\non the successor (storage demotes the incumbent atomically). ``name``\nis immutable; existing workstreams are never affected by edits.",
|
||||
"description": "PATCH body \u2014 absent fields are left unchanged.\n\nExplicit ``null`` resets ``tool_allowlist`` to unrestricted, and \u2014 on a\nBUILT-IN persona only \u2014 clears ``base_prompt`` (the operator override),\nreverting to that persona's file-backed prompt. An OPERATOR persona has no\nfallback source, so ``base_prompt: null`` on one is rejected: every persona\nmust name a prompt source. ``null`` on the boolean flags or\n``applies_to_kinds`` is ignored (treated as absent), so a client serializing\nunset optionals as null cannot archive a persona or flip levers by accident.\n\nArchive = ``{\"enabled\": false}``; default flip = ``{\"is_default\": true}``\non the successor (storage demotes the incumbent atomically). ``name``\nis immutable; existing workstreams are never affected by edits.",
|
||||
"properties": {
|
||||
"display_name": {
|
||||
"anyOf": [
|
||||
@@ -13305,21 +13306,17 @@
|
||||
},
|
||||
"pending_approval": {
|
||||
"default": false,
|
||||
"description": "True when the workstream is parked on ``_approval_event`` awaiting an operator approve/deny. Mirrors the same field on ``DashboardWorkstream`` / cluster live projections so a freshly-loaded chat tab can render the inline approval gate from the detail snapshot before SSE replay arrives.",
|
||||
"description": "True when at least one approval cycle is live (a gate thread parked awaiting an operator approve/deny). Mirrors the same field on ``DashboardWorkstream`` / cluster live projections so a freshly-loaded chat tab can render the inline approval gate from the detail snapshot before SSE replay arrives.",
|
||||
"title": "Pending Approval",
|
||||
"type": "boolean"
|
||||
},
|
||||
"pending_approval_detail": {
|
||||
"anyOf": [
|
||||
{
|
||||
"$ref": "#/components/schemas/PendingApprovalDetail"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
],
|
||||
"default": null,
|
||||
"description": "Inline approval payload \u2014 same shape as ``DashboardWorkstream.pending_approval_detail``. ``None`` when no approval is pending. Lets a reload paint the action row + judge verdicts immediately instead of relying on the SSE approve_request replay timing window."
|
||||
"pending_approval_details": {
|
||||
"description": "Inline approval payloads, one per live cycle, oldest first \u2014 same shape as ``DashboardWorkstream.pending_approval_details``. Empty when no approval is pending. Lets a reload paint every action row + judge verdicts immediately instead of relying on the SSE approve_request replay timing window. Replaces 1.6's ``pending_approval_detail`` single-object field (breaking, 1.7).",
|
||||
"items": {
|
||||
"$ref": "#/components/schemas/PendingApprovalDetail"
|
||||
},
|
||||
"title": "Pending Approval Details",
|
||||
"type": "array"
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
@@ -13332,8 +13329,14 @@
|
||||
"type": "object"
|
||||
},
|
||||
"PendingApprovalDetail": {
|
||||
"description": "Inline approval payload merged into ``DashboardWorkstream``.\n\nSet when a workstream's ``approve_tools`` is parked on\n``_approval_event``; ``None`` (omitted) otherwise. Cross-tenant\nexposure here follows the same trusted-team posture as\n``activity`` / ``tokens`` \u2014 see ``server.py``'s ``dashboard``\nhandler comment.",
|
||||
"description": "Inline approval payload merged into ``DashboardWorkstream``.\n\nOne entry per live approval CYCLE \u2014 a gate thread parked in\n``approve_tools`` awaiting the operator. Parallel task agents run\nconcurrent gates, so a workstream can have several of these at\nonce (``pending_approval_details``, oldest first). Cross-tenant\nexposure here follows the same trusted-team posture as\n``activity`` / ``tokens`` \u2014 see ``server.py``'s ``dashboard``\nhandler comment.",
|
||||
"properties": {
|
||||
"cycle_id": {
|
||||
"default": "",
|
||||
"description": "Identity of this approval cycle. Echo it back on ``POST /v1/api/workstreams/{ws_id}/approve`` to resolve exactly this round \u2014 required for correctness when several cycles are live (parallel task agents).",
|
||||
"title": "Cycle Id",
|
||||
"type": "string"
|
||||
},
|
||||
"call_id": {
|
||||
"default": "",
|
||||
"description": "Primary call_id \u2014 first non-empty call_id in items list order. Matches the 409 ``current_call_id`` response from ``POST /v1/api/workstreams/{ws_id}/approve`` so the UI can render the same identifier the server reports as current.",
|
||||
|
||||
@@ -2692,21 +2692,17 @@
|
||||
},
|
||||
"pending_approval": {
|
||||
"default": false,
|
||||
"description": "True when the workstream is parked on ``_approval_event`` awaiting an operator approve/deny. Mirrors the same field on ``DashboardWorkstream`` / cluster live projections so a freshly-loaded chat tab can render the inline approval gate from the detail snapshot before SSE replay arrives.",
|
||||
"description": "True when at least one approval cycle is live (a gate thread parked awaiting an operator approve/deny). Mirrors the same field on ``DashboardWorkstream`` / cluster live projections so a freshly-loaded chat tab can render the inline approval gate from the detail snapshot before SSE replay arrives.",
|
||||
"title": "Pending Approval",
|
||||
"type": "boolean"
|
||||
},
|
||||
"pending_approval_detail": {
|
||||
"anyOf": [
|
||||
{
|
||||
"$ref": "#/components/schemas/PendingApprovalDetail"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
],
|
||||
"default": null,
|
||||
"description": "Inline approval payload \u2014 same shape as ``DashboardWorkstream.pending_approval_detail``. ``None`` when no approval is pending. Lets a reload paint the action row + judge verdicts immediately instead of relying on the SSE approve_request replay timing window."
|
||||
"pending_approval_details": {
|
||||
"description": "Inline approval payloads, one per live cycle, oldest first \u2014 same shape as ``DashboardWorkstream.pending_approval_details``. Empty when no approval is pending. Lets a reload paint every action row + judge verdicts immediately instead of relying on the SSE approve_request replay timing window. Replaces 1.6's ``pending_approval_detail`` single-object field (breaking, 1.7).",
|
||||
"items": {
|
||||
"$ref": "#/components/schemas/PendingApprovalDetail"
|
||||
},
|
||||
"title": "Pending Approval Details",
|
||||
"type": "array"
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
@@ -2719,8 +2715,14 @@
|
||||
"type": "object"
|
||||
},
|
||||
"PendingApprovalDetail": {
|
||||
"description": "Inline approval payload merged into ``DashboardWorkstream``.\n\nSet when a workstream's ``approve_tools`` is parked on\n``_approval_event``; ``None`` (omitted) otherwise. Cross-tenant\nexposure here follows the same trusted-team posture as\n``activity`` / ``tokens`` \u2014 see ``server.py``'s ``dashboard``\nhandler comment.",
|
||||
"description": "Inline approval payload merged into ``DashboardWorkstream``.\n\nOne entry per live approval CYCLE \u2014 a gate thread parked in\n``approve_tools`` awaiting the operator. Parallel task agents run\nconcurrent gates, so a workstream can have several of these at\nonce (``pending_approval_details``, oldest first). Cross-tenant\nexposure here follows the same trusted-team posture as\n``activity`` / ``tokens`` \u2014 see ``server.py``'s ``dashboard``\nhandler comment.",
|
||||
"properties": {
|
||||
"cycle_id": {
|
||||
"default": "",
|
||||
"description": "Identity of this approval cycle. Echo it back on ``POST /v1/api/workstreams/{ws_id}/approve`` to resolve exactly this round \u2014 required for correctness when several cycles are live (parallel task agents).",
|
||||
"title": "Cycle Id",
|
||||
"type": "string"
|
||||
},
|
||||
"call_id": {
|
||||
"default": "",
|
||||
"description": "Primary call_id \u2014 first non-empty call_id in items list order. Matches the 409 ``current_call_id`` response from ``POST /v1/api/workstreams/{ws_id}/approve`` so the UI can render the same identifier the server reports as current.",
|
||||
@@ -3004,17 +3006,13 @@
|
||||
"default": null,
|
||||
"title": "Project Id"
|
||||
},
|
||||
"pending_approval_detail": {
|
||||
"anyOf": [
|
||||
{
|
||||
"$ref": "#/components/schemas/PendingApprovalDetail"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
],
|
||||
"default": null,
|
||||
"description": "Inline approval payload for the coordinator children-tree UI. Carries the merged ``_pending_approval`` items list + per-call_id LLM verdict cache so a coord can render approve/deny buttons + judge pill without a separate per-child round-trip. ``None`` when no approval is pending. Also surfaced (verbatim) on ``GET /v1/api/cluster/ws/live`` via the ``_CLUSTER_WS_LIVE_KEYS`` projection."
|
||||
"pending_approval_details": {
|
||||
"description": "Inline approval payload for the coordinator children-tree UI: EVERY live approval cycle, oldest first \u2014 parallel task agents gate concurrently, so a workstream can hold several prompts at once. Each entry carries the cycle's items + per-call_id LLM verdict cache so a coord can render approve/deny buttons + judge pill without a separate per-child round-trip; resolve each with its ``cycle_id``. Empty when no approval is pending. Also surfaced (verbatim) on ``GET /v1/api/cluster/ws/live`` via the ``_CLUSTER_WS_LIVE_KEYS`` projection. Replaces 1.6's ``pending_approval_detail`` single-object field (breaking, 1.7).",
|
||||
"items": {
|
||||
"$ref": "#/components/schemas/PendingApprovalDetail"
|
||||
},
|
||||
"title": "Pending Approval Details",
|
||||
"type": "array"
|
||||
},
|
||||
"recent_auto_approvals": {
|
||||
"description": "Per-ws ring buffer (cap 10) of recent tool calls that bypassed the operator approval gate. Surfaces ``WebUI._recent_auto_approvals`` so the coord-tree row can render an 'auto-approved by ...' pill when the child's skill / blanket / admin-policy rules silently let a tool through. Also projected onto ``GET /v1/api/cluster/ws/live`` via ``_CLUSTER_WS_LIVE_KEYS``.",
|
||||
|
||||
@@ -75,15 +75,35 @@ export interface ToolInfoEvent {
|
||||
items: Array<Record<string, unknown>>;
|
||||
}
|
||||
|
||||
/** One approval CYCLE awaiting the operator. Several can be outstanding
|
||||
* at once (parallel task agents each gate their own tool calls) — key
|
||||
* prompt UI by `cycle_id` and echo it back on the approve POST.
|
||||
*
|
||||
* `cycle_id` is optional because it was added in 1.7: a pre-1.7 server
|
||||
* omits it on the wire, so a current SDK talking to an older node sees
|
||||
* `undefined`. Resolve those the legacy way (no selector → oldest
|
||||
* cycle). A current server always sends it. */
|
||||
export interface ApproveRequestEvent {
|
||||
type: "approve_request";
|
||||
cycle_id?: string;
|
||||
items: Array<Record<string, unknown>>;
|
||||
judge_pending?: boolean;
|
||||
}
|
||||
|
||||
/** A specific approval cycle resolved; `cycle_id`/`call_ids` identify
|
||||
* which prompt to dismiss.
|
||||
*
|
||||
* Both are optional for the same reason as `ApproveRequestEvent.cycle_id`
|
||||
* — a pre-1.7 server emits neither, so a bare "something resolved"
|
||||
* dismisses the sole tracked prompt (the legacy fallback the UI and
|
||||
* channel adapters keep). A current server always sends both. */
|
||||
export interface ApprovalResolvedEvent {
|
||||
type: "approval_resolved";
|
||||
approved: boolean;
|
||||
feedback: string;
|
||||
always?: boolean;
|
||||
cycle_id?: string;
|
||||
call_ids?: string[];
|
||||
}
|
||||
|
||||
export interface ToolResultEvent {
|
||||
|
||||
@@ -166,6 +166,13 @@ export class TurnstoneServer extends BaseClient {
|
||||
approved?: boolean;
|
||||
feedback?: string | null;
|
||||
always?: boolean;
|
||||
/** Resolve exactly this approval cycle (from ApproveRequestEvent.cycle_id).
|
||||
* Omitting it resolves the OLDEST live cycle — ambiguous when parallel
|
||||
* task agents have several prompts outstanding, so pass it whenever the
|
||||
* triggering event is known. */
|
||||
cycleId?: string;
|
||||
/** Alternative selector: any call_id inside the target cycle. */
|
||||
callId?: string;
|
||||
}): Promise<StatusResponse> {
|
||||
return this.request(
|
||||
"POST",
|
||||
@@ -175,6 +182,8 @@ export class TurnstoneServer extends BaseClient {
|
||||
approved: opts.approved ?? true,
|
||||
feedback: opts.feedback,
|
||||
always: opts.always,
|
||||
cycle_id: opts.cycleId,
|
||||
call_id: opts.callId,
|
||||
},
|
||||
},
|
||||
);
|
||||
|
||||
@@ -104,7 +104,6 @@ describe("TurnstoneServer attachments", () => {
|
||||
const [, init] = (fetchFn as ReturnType<typeof vi.fn>).mock.calls[0];
|
||||
expect(JSON.parse(init.body)).toEqual({
|
||||
message: "hi",
|
||||
ws_id: "ws-X",
|
||||
attachment_ids: ["a1", "a2"],
|
||||
});
|
||||
});
|
||||
@@ -117,7 +116,7 @@ describe("TurnstoneServer attachments", () => {
|
||||
});
|
||||
await client.send("hi", "ws-X");
|
||||
const [, init] = (fetchFn as ReturnType<typeof vi.fn>).mock.calls[0];
|
||||
expect(JSON.parse(init.body)).toEqual({ message: "hi", ws_id: "ws-X" });
|
||||
expect(JSON.parse(init.body)).toEqual({ message: "hi" });
|
||||
});
|
||||
|
||||
it("createWorkstream with attachments sends multipart and auto-generates ws_id", async () => {
|
||||
|
||||
@@ -74,8 +74,8 @@ describe("TurnstoneServer", () => {
|
||||
await client.send("Hello", "ws1");
|
||||
|
||||
const [url, init] = (fetchFn as ReturnType<typeof vi.fn>).mock.calls[0];
|
||||
expect(url).toBe("http://test/v1/api/send");
|
||||
expect(JSON.parse(init.body)).toEqual({ message: "Hello", ws_id: "ws1" });
|
||||
expect(url).toBe("http://test/v1/api/workstreams/ws1/send");
|
||||
expect(JSON.parse(init.body)).toEqual({ message: "Hello" });
|
||||
});
|
||||
|
||||
it("injects auth header when token provided", async () => {
|
||||
|
||||
@@ -51,6 +51,12 @@ def make_replay_mocks(
|
||||
ui._ws_messages = 0
|
||||
for key, value in ui_overrides.items():
|
||||
setattr(ui, key, value)
|
||||
# Both replay paths read cycle cards via ``pending_approval_cards()``
|
||||
# (one card per concurrent approval cycle). Model it from the
|
||||
# single-slot ``_pending_approval`` override so tests keep seeding
|
||||
# the one field; a bare MagicMock here would iterate empty and
|
||||
# silently drop the approve_request from the replay.
|
||||
ui.pending_approval_cards = lambda: [ui._pending_approval] if ui._pending_approval else []
|
||||
ws = MagicMock()
|
||||
ws.session = session
|
||||
request = MagicMock()
|
||||
|
||||
@@ -0,0 +1,76 @@
|
||||
"""Recording fake SDK client — captures the kwargs at each provider's seam.
|
||||
|
||||
Every provider's ``create_streaming`` assembles its kwargs and calls the
|
||||
SDK *eagerly* before returning the stream iterator (Anthropic
|
||||
``client.messages.stream``, OpenAI ``client.chat.completions.create``,
|
||||
Responses ``client.responses.create/stream``), so driving a provider
|
||||
against a :class:`RecordingClient` captures the full composed request
|
||||
payload without a network round-trip.
|
||||
|
||||
Shared by the wire-payload golden harness (``test_wire_payload_golden``)
|
||||
and the effort-ladder parity harness (``test_effort_ladder_wire_parity``)
|
||||
so both assert against the same capture seam.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from collections.abc import Iterator
|
||||
|
||||
|
||||
class _EmptyStream:
|
||||
"""Stand-in for an SDK stream / stream-manager: empty iterable AND no-op CM."""
|
||||
|
||||
def __iter__(self) -> Iterator[Any]:
|
||||
return iter(())
|
||||
|
||||
def __enter__(self) -> _EmptyStream:
|
||||
return self
|
||||
|
||||
def __exit__(self, *exc: object) -> None:
|
||||
return None
|
||||
|
||||
|
||||
class _Seam:
|
||||
"""Records the kwargs of a single SDK call, returns an empty stream stub."""
|
||||
|
||||
def __init__(self, sink: dict[str, Any]) -> None:
|
||||
self._sink = sink
|
||||
|
||||
def __call__(self, **kwargs: Any) -> _EmptyStream:
|
||||
# Last write wins; only one seam is exercised per provider call.
|
||||
self._sink["payload"] = kwargs
|
||||
return _EmptyStream()
|
||||
|
||||
|
||||
class _Completions:
|
||||
def __init__(self, sink: dict[str, Any]) -> None:
|
||||
self.create = _Seam(sink)
|
||||
|
||||
|
||||
class _Chat:
|
||||
def __init__(self, sink: dict[str, Any]) -> None:
|
||||
self.completions = _Completions(sink)
|
||||
|
||||
|
||||
class _Messages:
|
||||
def __init__(self, sink: dict[str, Any]) -> None:
|
||||
self.stream = _Seam(sink)
|
||||
|
||||
|
||||
class _Responses:
|
||||
def __init__(self, sink: dict[str, Any]) -> None:
|
||||
self.create = _Seam(sink)
|
||||
self.stream = _Seam(sink)
|
||||
|
||||
|
||||
class RecordingClient:
|
||||
"""Fake SDK client exposing every provider's call seam, recording kwargs."""
|
||||
|
||||
def __init__(self) -> None:
|
||||
self.captured: dict[str, Any] = {}
|
||||
self.messages = _Messages(self.captured)
|
||||
self.chat = _Chat(self.captured)
|
||||
self.responses = _Responses(self.captured)
|
||||
+69
-1
@@ -52,8 +52,76 @@ def serve_until_exit(server: Any) -> None:
|
||||
loop.close()
|
||||
|
||||
|
||||
class _PendingResolver:
|
||||
"""Race-free drop-in for ``threading.Timer(delay, ui.resolve_approval)``.
|
||||
|
||||
``approve_tools`` runs ``_approval_event.clear()`` -> register
|
||||
``_pending_approval`` -> ``_approval_event.wait(_APPROVAL_WAIT_TIMEOUT)``
|
||||
(3600s). A *fixed-delay* timer can fire ``resolve_approval``
|
||||
(``_approval_event.set()``) BEFORE that ``.clear()`` on a slow/loaded
|
||||
runner, so the set is wiped by the clear and ``approve_tools`` blocks the
|
||||
full hour -- surfacing as a CI hang. This instead waits until the approval
|
||||
is actually registered (which happens *after* the clear), then resolves, so
|
||||
the wakeup can never be lost. ``start()`` / ``cancel()`` mirror
|
||||
``threading.Timer`` so it drops into existing scaffolding. ``cancel()``
|
||||
signals the worker to stop and joins it, so a test that errors *before* the
|
||||
approval registers can't leak the thread or resolve late into a finished
|
||||
test. ``before`` runs just before resolving -- e.g. to snapshot
|
||||
pending-state fields the test asserts on.
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
ui: Any,
|
||||
*args: Any,
|
||||
before: Callable[[], None] | None = None,
|
||||
deadline: float = 10.0,
|
||||
**kwargs: Any,
|
||||
) -> None:
|
||||
self._ui = ui
|
||||
self._args = args
|
||||
self._kwargs = kwargs
|
||||
self._before = before
|
||||
self._deadline = deadline
|
||||
self._cancelled = threading.Event()
|
||||
self._started = False
|
||||
self._thread = threading.Thread(target=self._run, name="resolve-when-pending", daemon=True)
|
||||
|
||||
def _run(self) -> None:
|
||||
end = time.monotonic() + self._deadline
|
||||
while time.monotonic() < end:
|
||||
if self._cancelled.is_set():
|
||||
return
|
||||
# getattr (not a bare read) so a UI without _pending_approval can't
|
||||
# crash the worker into a silent death that leaves approve_tools
|
||||
# blocked for the full _APPROVAL_WAIT_TIMEOUT.
|
||||
if getattr(self._ui, "_pending_approval", None) is not None:
|
||||
if self._before is not None:
|
||||
self._before()
|
||||
self._ui.resolve_approval(*self._args, **self._kwargs)
|
||||
return
|
||||
time.sleep(0.001)
|
||||
# Deadline without registration: approve_tools isn't parked on the
|
||||
# approval event (returned early, or never reached it) -- don't resolve
|
||||
# into an unknown state; let the test's own assertions speak.
|
||||
|
||||
def start(self) -> None:
|
||||
self._started = True
|
||||
self._thread.start()
|
||||
|
||||
def cancel(self) -> None:
|
||||
self._cancelled.set()
|
||||
if self._started:
|
||||
self._thread.join(timeout=5)
|
||||
|
||||
|
||||
def resolve_when_pending(ui: Any, *args: Any, **kwargs: Any) -> _PendingResolver:
|
||||
"""Build a race-free approval resolver (see :class:`_PendingResolver`)."""
|
||||
return _PendingResolver(ui, *args, **kwargs)
|
||||
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from collections.abc import Iterator
|
||||
from collections.abc import Callable, Iterator
|
||||
|
||||
from turnstone.core.mcp_client import MCPClientManager, StaticServerState
|
||||
from turnstone.core.mcp_crypto import MCPTokenCipher
|
||||
|
||||
@@ -0,0 +1,78 @@
|
||||
{
|
||||
"cache_control": {
|
||||
"type": "ephemeral"
|
||||
},
|
||||
"extra_body": {
|
||||
"chat_template_kwargs": {
|
||||
"enable_thinking": true,
|
||||
"reasoning_effort": "high"
|
||||
}
|
||||
},
|
||||
"max_tokens": 4096,
|
||||
"messages": [
|
||||
{
|
||||
"content": "Weather in Paris and London?",
|
||||
"role": "user"
|
||||
},
|
||||
{
|
||||
"content": [
|
||||
{
|
||||
"id": "call_1",
|
||||
"input": {
|
||||
"city": "Paris"
|
||||
},
|
||||
"name": "get_weather",
|
||||
"type": "tool_use"
|
||||
},
|
||||
{
|
||||
"id": "call_2",
|
||||
"input": {
|
||||
"city": "London"
|
||||
},
|
||||
"name": "get_weather",
|
||||
"type": "tool_use"
|
||||
}
|
||||
],
|
||||
"role": "assistant"
|
||||
},
|
||||
{
|
||||
"content": [
|
||||
{
|
||||
"content": "18C, clear.",
|
||||
"tool_use_id": "call_1",
|
||||
"type": "tool_result"
|
||||
},
|
||||
{
|
||||
"content": "Tool execution was cancelled. Outcome UNKNOWN — this call may have begun executing before the generation was stopped; do not assume it did not run, and reconcile before re-issuing it.",
|
||||
"is_error": true,
|
||||
"tool_use_id": "call_2",
|
||||
"type": "tool_result"
|
||||
},
|
||||
{
|
||||
"text": "Actually, never mind London.",
|
||||
"type": "text"
|
||||
}
|
||||
],
|
||||
"role": "user"
|
||||
}
|
||||
],
|
||||
"model": "qwen3.6-27b",
|
||||
"temperature": 0.5,
|
||||
"tools": [
|
||||
{
|
||||
"description": "Look up the weather for a city.",
|
||||
"input_schema": {
|
||||
"properties": {
|
||||
"city": {
|
||||
"type": "string"
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
"city"
|
||||
],
|
||||
"type": "object"
|
||||
},
|
||||
"name": "get_weather"
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,33 @@
|
||||
{
|
||||
"cache_control": {
|
||||
"type": "ephemeral"
|
||||
},
|
||||
"extra_body": {
|
||||
"chat_template_kwargs": {
|
||||
"enable_thinking": true,
|
||||
"reasoning_effort": "high"
|
||||
}
|
||||
},
|
||||
"max_tokens": 4096,
|
||||
"messages": [
|
||||
{
|
||||
"content": [
|
||||
{
|
||||
"text": "What's in this image?",
|
||||
"type": "text"
|
||||
},
|
||||
{
|
||||
"source": {
|
||||
"data": "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mP8z8BQDwAEhQGAhKmMIQAAAABJRU5ErkJggg==",
|
||||
"media_type": "image/png",
|
||||
"type": "base64"
|
||||
},
|
||||
"type": "image"
|
||||
}
|
||||
],
|
||||
"role": "user"
|
||||
}
|
||||
],
|
||||
"model": "qwen3.6-27b",
|
||||
"temperature": 0.5
|
||||
}
|
||||
@@ -0,0 +1,70 @@
|
||||
{
|
||||
"cache_control": {
|
||||
"type": "ephemeral"
|
||||
},
|
||||
"extra_body": {
|
||||
"chat_template_kwargs": {
|
||||
"enable_thinking": true,
|
||||
"reasoning_effort": "high"
|
||||
}
|
||||
},
|
||||
"max_tokens": 4096,
|
||||
"messages": [
|
||||
{
|
||||
"content": "Think about the weather.",
|
||||
"role": "user"
|
||||
},
|
||||
{
|
||||
"content": [
|
||||
{
|
||||
"signature": "sig-abc",
|
||||
"thinking": "The user wants weather.",
|
||||
"type": "thinking"
|
||||
},
|
||||
{
|
||||
"text": "Let me check.",
|
||||
"type": "text"
|
||||
},
|
||||
{
|
||||
"id": "call_1",
|
||||
"input": {
|
||||
"city": "Paris"
|
||||
},
|
||||
"name": "get_weather",
|
||||
"type": "tool_use"
|
||||
}
|
||||
],
|
||||
"role": "assistant"
|
||||
},
|
||||
{
|
||||
"content": [
|
||||
{
|
||||
"content": "Tool execution was cancelled. Outcome UNKNOWN — this call may have begun executing before the generation was stopped; do not assume it did not run, and reconcile before re-issuing it.",
|
||||
"is_error": true,
|
||||
"tool_use_id": "call_1",
|
||||
"type": "tool_result"
|
||||
}
|
||||
],
|
||||
"role": "user"
|
||||
}
|
||||
],
|
||||
"model": "qwen3.6-27b",
|
||||
"temperature": 0.5,
|
||||
"tools": [
|
||||
{
|
||||
"description": "Look up the weather for a city.",
|
||||
"input_schema": {
|
||||
"properties": {
|
||||
"city": {
|
||||
"type": "string"
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
"city"
|
||||
],
|
||||
"type": "object"
|
||||
},
|
||||
"name": "get_weather"
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,69 @@
|
||||
{
|
||||
"cache_control": {
|
||||
"type": "ephemeral"
|
||||
},
|
||||
"extra_body": {
|
||||
"chat_template_kwargs": {
|
||||
"enable_thinking": true,
|
||||
"reasoning_effort": "high"
|
||||
}
|
||||
},
|
||||
"max_tokens": 4096,
|
||||
"messages": [
|
||||
{
|
||||
"content": "Think about the weather.",
|
||||
"role": "user"
|
||||
},
|
||||
{
|
||||
"content": [
|
||||
{
|
||||
"signature": "sig-abc",
|
||||
"thinking": "The user wants weather.",
|
||||
"type": "thinking"
|
||||
},
|
||||
{
|
||||
"text": "Let me check.",
|
||||
"type": "text"
|
||||
},
|
||||
{
|
||||
"id": "call_1",
|
||||
"input": {
|
||||
"city": "Paris"
|
||||
},
|
||||
"name": "get_weather",
|
||||
"type": "tool_use"
|
||||
}
|
||||
],
|
||||
"role": "assistant"
|
||||
},
|
||||
{
|
||||
"content": [
|
||||
{
|
||||
"content": "18C, clear.",
|
||||
"tool_use_id": "call_1",
|
||||
"type": "tool_result"
|
||||
}
|
||||
],
|
||||
"role": "user"
|
||||
}
|
||||
],
|
||||
"model": "qwen3.6-27b",
|
||||
"temperature": 0.5,
|
||||
"tools": [
|
||||
{
|
||||
"description": "Look up the weather for a city.",
|
||||
"input_schema": {
|
||||
"properties": {
|
||||
"city": {
|
||||
"type": "string"
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
"city"
|
||||
],
|
||||
"type": "object"
|
||||
},
|
||||
"name": "get_weather"
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,63 @@
|
||||
{
|
||||
"cache_control": {
|
||||
"type": "ephemeral"
|
||||
},
|
||||
"extra_body": {
|
||||
"chat_template_kwargs": {
|
||||
"enable_thinking": true,
|
||||
"reasoning_effort": "high"
|
||||
}
|
||||
},
|
||||
"max_tokens": 4096,
|
||||
"messages": [
|
||||
{
|
||||
"content": "Run the deploy.",
|
||||
"role": "user"
|
||||
},
|
||||
{
|
||||
"content": [
|
||||
{
|
||||
"id": "call_1",
|
||||
"input": {},
|
||||
"name": "deploy",
|
||||
"type": "tool_use"
|
||||
}
|
||||
],
|
||||
"role": "assistant"
|
||||
},
|
||||
{
|
||||
"content": [
|
||||
{
|
||||
"content": "deployed",
|
||||
"tool_use_id": "call_1",
|
||||
"type": "tool_result"
|
||||
},
|
||||
{
|
||||
"text": "Great, what's next?",
|
||||
"type": "text"
|
||||
}
|
||||
],
|
||||
"role": "user"
|
||||
}
|
||||
],
|
||||
"model": "qwen3.6-27b",
|
||||
"system": "Output-guard: deploy output looked clean.",
|
||||
"temperature": 0.5,
|
||||
"tools": [
|
||||
{
|
||||
"description": "Look up the weather for a city.",
|
||||
"input_schema": {
|
||||
"properties": {
|
||||
"city": {
|
||||
"type": "string"
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
"city"
|
||||
],
|
||||
"type": "object"
|
||||
},
|
||||
"name": "get_weather"
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,33 @@
|
||||
{
|
||||
"cache_control": {
|
||||
"type": "ephemeral"
|
||||
},
|
||||
"extra_body": {
|
||||
"chat_template_kwargs": {
|
||||
"enable_thinking": true,
|
||||
"reasoning_effort": "high"
|
||||
}
|
||||
},
|
||||
"max_tokens": 4096,
|
||||
"messages": [
|
||||
{
|
||||
"content": "Hi there.",
|
||||
"role": "user"
|
||||
},
|
||||
{
|
||||
"content": [
|
||||
{
|
||||
"text": "Hello! How can I help?",
|
||||
"type": "text"
|
||||
}
|
||||
],
|
||||
"role": "assistant"
|
||||
},
|
||||
{
|
||||
"content": "What's the weather in Paris?",
|
||||
"role": "user"
|
||||
}
|
||||
],
|
||||
"model": "qwen3.6-27b",
|
||||
"temperature": 0.5
|
||||
}
|
||||
@@ -0,0 +1,69 @@
|
||||
{
|
||||
"cache_control": {
|
||||
"type": "ephemeral"
|
||||
},
|
||||
"extra_body": {
|
||||
"chat_template_kwargs": {
|
||||
"enable_thinking": true,
|
||||
"reasoning_effort": "high"
|
||||
}
|
||||
},
|
||||
"max_tokens": 4096,
|
||||
"messages": [
|
||||
{
|
||||
"content": "Weather in Paris?",
|
||||
"role": "user"
|
||||
},
|
||||
{
|
||||
"content": [
|
||||
{
|
||||
"id": "call_1",
|
||||
"input": {
|
||||
"city": "Paris"
|
||||
},
|
||||
"name": "get_weather",
|
||||
"type": "tool_use"
|
||||
}
|
||||
],
|
||||
"role": "assistant"
|
||||
},
|
||||
{
|
||||
"content": [
|
||||
{
|
||||
"content": "18C, clear.",
|
||||
"tool_use_id": "call_1",
|
||||
"type": "tool_result"
|
||||
}
|
||||
],
|
||||
"role": "user"
|
||||
},
|
||||
{
|
||||
"content": [
|
||||
{
|
||||
"text": "It's 18C and clear in Paris.",
|
||||
"type": "text"
|
||||
}
|
||||
],
|
||||
"role": "assistant"
|
||||
}
|
||||
],
|
||||
"model": "qwen3.6-27b",
|
||||
"temperature": 0.5,
|
||||
"tools": [
|
||||
{
|
||||
"description": "Look up the weather for a city.",
|
||||
"input_schema": {
|
||||
"properties": {
|
||||
"city": {
|
||||
"type": "string"
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
"city"
|
||||
],
|
||||
"type": "object"
|
||||
},
|
||||
"name": "get_weather"
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,61 @@
|
||||
{
|
||||
"cache_control": {
|
||||
"type": "ephemeral"
|
||||
},
|
||||
"extra_body": {
|
||||
"chat_template_kwargs": {
|
||||
"enable_thinking": true,
|
||||
"reasoning_effort": "high"
|
||||
}
|
||||
},
|
||||
"max_tokens": 4096,
|
||||
"messages": [
|
||||
{
|
||||
"content": "Weather in Paris?",
|
||||
"role": "user"
|
||||
},
|
||||
{
|
||||
"content": [
|
||||
{
|
||||
"id": "call_1",
|
||||
"input": {
|
||||
"city": "Paris"
|
||||
},
|
||||
"name": "get_weather",
|
||||
"type": "tool_use"
|
||||
}
|
||||
],
|
||||
"role": "assistant"
|
||||
},
|
||||
{
|
||||
"content": [
|
||||
{
|
||||
"content": "Tool execution was cancelled. Outcome UNKNOWN — this call may have begun executing before the generation was stopped; do not assume it did not run, and reconcile before re-issuing it.",
|
||||
"is_error": true,
|
||||
"tool_use_id": "call_1",
|
||||
"type": "tool_result"
|
||||
}
|
||||
],
|
||||
"role": "user"
|
||||
}
|
||||
],
|
||||
"model": "qwen3.6-27b",
|
||||
"temperature": 0.5,
|
||||
"tools": [
|
||||
{
|
||||
"description": "Look up the weather for a city.",
|
||||
"input_schema": {
|
||||
"properties": {
|
||||
"city": {
|
||||
"type": "string"
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
"city"
|
||||
],
|
||||
"type": "object"
|
||||
},
|
||||
"name": "get_weather"
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -43,6 +43,7 @@
|
||||
}
|
||||
],
|
||||
"model": "gemini-2.5-pro",
|
||||
"reasoning_effort": "medium",
|
||||
"stream": true,
|
||||
"stream_options": {
|
||||
"include_usage": true
|
||||
|
||||
@@ -18,6 +18,7 @@
|
||||
}
|
||||
],
|
||||
"model": "gemini-2.5-pro",
|
||||
"reasoning_effort": "medium",
|
||||
"stream": true,
|
||||
"stream_options": {
|
||||
"include_usage": true
|
||||
|
||||
@@ -26,6 +26,7 @@
|
||||
}
|
||||
],
|
||||
"model": "gemini-2.5-pro",
|
||||
"reasoning_effort": "medium",
|
||||
"stream": true,
|
||||
"stream_options": {
|
||||
"include_usage": true
|
||||
|
||||
@@ -26,6 +26,7 @@
|
||||
}
|
||||
],
|
||||
"model": "gemini-2.5-pro",
|
||||
"reasoning_effort": "medium",
|
||||
"stream": true,
|
||||
"stream_options": {
|
||||
"include_usage": true
|
||||
|
||||
@@ -34,6 +34,7 @@
|
||||
}
|
||||
],
|
||||
"model": "gemini-2.5-pro",
|
||||
"reasoning_effort": "medium",
|
||||
"stream": true,
|
||||
"stream_options": {
|
||||
"include_usage": true
|
||||
|
||||
@@ -15,6 +15,7 @@
|
||||
}
|
||||
],
|
||||
"model": "gemini-2.5-pro",
|
||||
"reasoning_effort": "medium",
|
||||
"stream": true,
|
||||
"stream_options": {
|
||||
"include_usage": true
|
||||
|
||||
@@ -30,6 +30,7 @@
|
||||
}
|
||||
],
|
||||
"model": "gemini-2.5-pro",
|
||||
"reasoning_effort": "medium",
|
||||
"stream": true,
|
||||
"stream_options": {
|
||||
"include_usage": true
|
||||
|
||||
@@ -26,6 +26,7 @@
|
||||
}
|
||||
],
|
||||
"model": "gemini-2.5-pro",
|
||||
"reasoning_effort": "medium",
|
||||
"stream": true,
|
||||
"stream_options": {
|
||||
"include_usage": true
|
||||
|
||||
@@ -43,6 +43,7 @@
|
||||
}
|
||||
],
|
||||
"model": "gpt-4o-mini",
|
||||
"reasoning_effort": "medium",
|
||||
"stream": true,
|
||||
"stream_options": {
|
||||
"include_usage": true
|
||||
|
||||
@@ -18,6 +18,7 @@
|
||||
}
|
||||
],
|
||||
"model": "gpt-4o-mini",
|
||||
"reasoning_effort": "medium",
|
||||
"stream": true,
|
||||
"stream_options": {
|
||||
"include_usage": true
|
||||
|
||||
@@ -26,6 +26,7 @@
|
||||
}
|
||||
],
|
||||
"model": "gpt-4o-mini",
|
||||
"reasoning_effort": "medium",
|
||||
"stream": true,
|
||||
"stream_options": {
|
||||
"include_usage": true
|
||||
|
||||
@@ -26,6 +26,7 @@
|
||||
}
|
||||
],
|
||||
"model": "gpt-4o-mini",
|
||||
"reasoning_effort": "medium",
|
||||
"stream": true,
|
||||
"stream_options": {
|
||||
"include_usage": true
|
||||
|
||||
@@ -34,6 +34,7 @@
|
||||
}
|
||||
],
|
||||
"model": "gpt-4o-mini",
|
||||
"reasoning_effort": "medium",
|
||||
"stream": true,
|
||||
"stream_options": {
|
||||
"include_usage": true
|
||||
|
||||
@@ -15,6 +15,7 @@
|
||||
}
|
||||
],
|
||||
"model": "gpt-4o-mini",
|
||||
"reasoning_effort": "medium",
|
||||
"stream": true,
|
||||
"stream_options": {
|
||||
"include_usage": true
|
||||
|
||||
@@ -30,6 +30,7 @@
|
||||
}
|
||||
],
|
||||
"model": "gpt-4o-mini",
|
||||
"reasoning_effort": "medium",
|
||||
"stream": true,
|
||||
"stream_options": {
|
||||
"include_usage": true
|
||||
|
||||
@@ -26,6 +26,7 @@
|
||||
}
|
||||
],
|
||||
"model": "gpt-4o-mini",
|
||||
"reasoning_effort": "medium",
|
||||
"stream": true,
|
||||
"stream_options": {
|
||||
"include_usage": true
|
||||
|
||||
@@ -41,6 +41,11 @@ def _bind_ws_event_handlers(bot, cls):
|
||||
attr = getattr(cls, name)
|
||||
if callable(attr):
|
||||
setattr(bot, name, attr.__get__(bot, cls))
|
||||
# ``_handle_stream_end`` delegates the all-cycles sweep to
|
||||
# ``_pop_ws_approvals``; bind the real method too so dispatcher
|
||||
# tests observe the pop instead of a spec'd AsyncMock no-op.
|
||||
if hasattr(cls, "_pop_ws_approvals"):
|
||||
bot._pop_ws_approvals = cls._pop_ws_approvals.__get__(bot, cls)
|
||||
|
||||
|
||||
def _make_message(*, bot=False, guild=True, content="hello", channel=None, reference=None):
|
||||
@@ -537,7 +542,7 @@ class TestApprovalVerdictDisplay:
|
||||
},
|
||||
}
|
||||
]
|
||||
event = ApproveRequestEvent(ws_id="ws-1", items=items)
|
||||
event = ApproveRequestEvent(ws_id="ws-1", cycle_id="cyc-1", items=items)
|
||||
_run(bot._on_ws_event("ws-1", thread, event))
|
||||
|
||||
# thread.send was called with an embed containing a verdict field
|
||||
@@ -551,8 +556,8 @@ class TestApprovalVerdictDisplay:
|
||||
assert "HIGH" in field.value
|
||||
assert "85%" in field.value
|
||||
|
||||
# Pending approval message tracked
|
||||
assert "ws-1" in bot._pending_approval_msgs
|
||||
# Pending approval message tracked under (ws_id, cycle_id).
|
||||
assert ("ws-1", "cyc-1") in bot._pending_approval_msgs
|
||||
|
||||
def test_approval_without_verdict(self):
|
||||
"""ApproveRequestEvent items without verdict still work normally."""
|
||||
@@ -585,10 +590,11 @@ class TestApprovalVerdictDisplay:
|
||||
embed = MagicMock()
|
||||
msg.embeds = [embed]
|
||||
msg.edit = AsyncMock()
|
||||
bot._pending_approval_msgs["ws-1"] = msg
|
||||
bot._pending_approval_msgs[("ws-1", "cyc-1")] = (msg, frozenset({"c-1"}))
|
||||
|
||||
event = IntentVerdictEvent(
|
||||
ws_id="ws-1",
|
||||
call_id="c-1",
|
||||
func_name="bash",
|
||||
risk_level="high",
|
||||
recommendation="deny",
|
||||
@@ -628,7 +634,10 @@ class TestApprovalVerdictDisplay:
|
||||
bot._streaming = {}
|
||||
bot._thinking_msgs = {}
|
||||
bot._tool_info_msgs = {}
|
||||
bot._pending_approval_msgs = {"ws-1": MagicMock()}
|
||||
bot._pending_approval_msgs = {
|
||||
("ws-1", "cyc-1"): (MagicMock(), frozenset()),
|
||||
("ws-1", "cyc-2"): (MagicMock(), frozenset()),
|
||||
}
|
||||
bot._notify_reply_channels = {}
|
||||
_bind_ws_event_handlers(bot, TurnstoneBot)
|
||||
|
||||
@@ -636,7 +645,8 @@ class TestApprovalVerdictDisplay:
|
||||
event = StreamEndEvent(ws_id="ws-1")
|
||||
_run(bot._on_ws_event("ws-1", thread, event))
|
||||
|
||||
assert "ws-1" not in bot._pending_approval_msgs
|
||||
# ALL of the ws's cycles are swept, not just one entry.
|
||||
assert not bot._pending_approval_msgs
|
||||
|
||||
|
||||
class TestStreamEndBehavior:
|
||||
@@ -1657,19 +1667,21 @@ class TestApprovalResolved:
|
||||
bot = self._make_bot()
|
||||
thread = AsyncMock()
|
||||
|
||||
# Set up a pending approval message with components.
|
||||
# Set up a pending approval message with components. The event
|
||||
# below carries no cycle_id (pre-multi-cycle server) — the
|
||||
# legacy fallback clears the ws's single tracked entry.
|
||||
approval_msg = MagicMock()
|
||||
approval_msg.embeds = [MagicMock()]
|
||||
approval_msg.components = []
|
||||
approval_msg.edit = AsyncMock()
|
||||
bot._pending_approval_msgs["ws-1"] = approval_msg
|
||||
bot._pending_approval_msgs[("ws-1", "cyc-1")] = (approval_msg, frozenset())
|
||||
|
||||
event = ApprovalResolvedEvent(ws_id="ws-1", approved=False, feedback="timeout")
|
||||
_run(bot._on_ws_event("ws-1", thread, event))
|
||||
|
||||
approval_msg.edit.assert_awaited_once()
|
||||
# Pending approval message should be removed.
|
||||
assert "ws-1" not in bot._pending_approval_msgs
|
||||
assert not bot._pending_approval_msgs
|
||||
|
||||
def test_disables_buttons_on_approved(self):
|
||||
from turnstone.sdk.events import ApprovalResolvedEvent
|
||||
@@ -1681,9 +1693,11 @@ class TestApprovalResolved:
|
||||
approval_msg.embeds = [MagicMock()]
|
||||
approval_msg.components = []
|
||||
approval_msg.edit = AsyncMock()
|
||||
bot._pending_approval_msgs["ws-1"] = approval_msg
|
||||
bot._pending_approval_msgs[("ws-1", "cyc-1")] = (approval_msg, frozenset())
|
||||
|
||||
event = ApprovalResolvedEvent(ws_id="ws-1", approved=True)
|
||||
# Cycle-routed resolution: the event's cycle_id selects exactly
|
||||
# this tracked message.
|
||||
event = ApprovalResolvedEvent(ws_id="ws-1", approved=True, cycle_id="cyc-1")
|
||||
_run(bot._on_ws_event("ws-1", thread, event))
|
||||
|
||||
approval_msg.edit.assert_awaited_once()
|
||||
|
||||
@@ -87,7 +87,7 @@ class TestSendApproval:
|
||||
monkeypatch.setattr(router._server, "approve", mock_approve)
|
||||
await router.send_approval("ws-1", "corr-abc", approved=True, feedback="ok")
|
||||
mock_approve.assert_awaited_once_with(
|
||||
ws_id="ws-1", approved=True, feedback="ok", always=False
|
||||
ws_id="ws-1", approved=True, feedback="ok", always=False, cycle_id="corr-abc"
|
||||
)
|
||||
|
||||
@pytest.mark.anyio
|
||||
@@ -99,7 +99,7 @@ class TestSendApproval:
|
||||
monkeypatch.setattr(router._server, "approve", mock_approve)
|
||||
await router.send_approval("ws-1", "corr-abc", approved=False)
|
||||
mock_approve.assert_awaited_once_with(
|
||||
ws_id="ws-1", approved=False, feedback=None, always=False
|
||||
ws_id="ws-1", approved=False, feedback=None, always=False, cycle_id="corr-abc"
|
||||
)
|
||||
|
||||
@pytest.mark.anyio
|
||||
@@ -110,7 +110,9 @@ class TestSendApproval:
|
||||
mock_approve = AsyncMock()
|
||||
monkeypatch.setattr(console_router._console, "route_approve", mock_approve)
|
||||
await console_router.send_approval("ws-1", "corr-abc", approved=True, always=True)
|
||||
mock_approve.assert_awaited_once_with(ws_id="ws-1", approved=True, feedback="", always=True)
|
||||
mock_approve.assert_awaited_once_with(
|
||||
ws_id="ws-1", approved=True, feedback="", always=True, cycle_id="corr-abc"
|
||||
)
|
||||
|
||||
|
||||
class TestDeleteRoute:
|
||||
|
||||
@@ -576,10 +576,11 @@ class TestApprovalOwnership:
|
||||
|
||||
bot, router, client = _make_bot()
|
||||
ws_id = "ws-1"
|
||||
bot._pending_approval[ws_id] = PendingApproval( # type: ignore[attr-defined]
|
||||
bot._pending_approval[(ws_id, "corr-1")] = PendingApproval( # type: ignore[attr-defined]
|
||||
channel="C01SAPU5414",
|
||||
message_ts="111.222",
|
||||
owner_user_id="U_OWNER",
|
||||
cycle_id="corr-1",
|
||||
)
|
||||
|
||||
body = {
|
||||
@@ -598,10 +599,11 @@ class TestApprovalOwnership:
|
||||
|
||||
bot, router, client = _make_bot()
|
||||
ws_id = "ws-1"
|
||||
bot._pending_approval[ws_id] = PendingApproval( # type: ignore[attr-defined]
|
||||
bot._pending_approval[(ws_id, "corr-1")] = PendingApproval( # type: ignore[attr-defined]
|
||||
channel="C01SAPU5414",
|
||||
message_ts="111.222",
|
||||
owner_user_id="U_OWNER",
|
||||
cycle_id="corr-1",
|
||||
)
|
||||
|
||||
body = {
|
||||
@@ -620,10 +622,11 @@ class TestApprovalOwnership:
|
||||
|
||||
bot, router, client = _make_bot()
|
||||
ws_id = "ws-1"
|
||||
bot._pending_approval[ws_id] = PendingApproval( # type: ignore[attr-defined]
|
||||
bot._pending_approval[(ws_id, "corr-1")] = PendingApproval( # type: ignore[attr-defined]
|
||||
channel="C01SAPU5414",
|
||||
message_ts="111.222",
|
||||
owner_user_id="U_OWNER",
|
||||
cycle_id="corr-1",
|
||||
)
|
||||
|
||||
body = {
|
||||
@@ -776,7 +779,9 @@ class TestWsEventDispatch:
|
||||
bot, client = self._make_ws_bot()
|
||||
|
||||
event = ApproveRequestEvent(
|
||||
ws_id="ws-1", items=[{"func_name": "bash", "needs_approval": True}]
|
||||
ws_id="ws-1",
|
||||
cycle_id="cyc-1",
|
||||
items=[{"call_id": "c-1", "func_name": "bash", "needs_approval": True}],
|
||||
)
|
||||
route = SlackRoute(channel="C1", user_id="U12345", thread_ts="123.456")
|
||||
_run(bot._on_ws_event("ws-1", route, event)) # type: ignore[attr-defined]
|
||||
@@ -784,8 +789,12 @@ class TestWsEventDispatch:
|
||||
client.chat_postMessage.assert_awaited_once()
|
||||
call_kwargs = client.chat_postMessage.call_args[1]
|
||||
assert "blocks" in call_kwargs
|
||||
assert "ws-1" in bot._pending_approval # type: ignore[attr-defined]
|
||||
assert bot._pending_approval["ws-1"].owner_user_id == "U12345" # type: ignore[attr-defined]
|
||||
# Tracked under (ws_id, cycle_id) so concurrent cycles each get
|
||||
# their own Slack message.
|
||||
entry = bot._pending_approval[("ws-1", "cyc-1")] # type: ignore[attr-defined]
|
||||
assert entry.owner_user_id == "U12345"
|
||||
assert entry.cycle_id == "cyc-1"
|
||||
assert entry.call_ids == frozenset({"c-1"})
|
||||
|
||||
def test_intent_verdict_updates_approval_message(self) -> None:
|
||||
from turnstone.channels.slack.bot import PendingApproval
|
||||
@@ -797,14 +806,17 @@ class TestWsEventDispatch:
|
||||
return_value={"ok": True, "messages": [{"blocks": []}]}
|
||||
)
|
||||
|
||||
bot._pending_approval["ws-1"] = PendingApproval( # type: ignore[attr-defined]
|
||||
bot._pending_approval[("ws-1", "cyc-1")] = PendingApproval( # type: ignore[attr-defined]
|
||||
channel="C1",
|
||||
message_ts="999.000",
|
||||
owner_user_id="U12345",
|
||||
cycle_id="cyc-1",
|
||||
call_ids=frozenset({"c-1"}),
|
||||
)
|
||||
|
||||
event = IntentVerdictEvent(
|
||||
ws_id="ws-1",
|
||||
call_id="c-1",
|
||||
func_name="bash",
|
||||
risk_level="high",
|
||||
confidence=0.9,
|
||||
@@ -821,17 +833,20 @@ class TestWsEventDispatch:
|
||||
from turnstone.sdk.events import ApprovalResolvedEvent
|
||||
|
||||
bot, client = self._make_ws_bot()
|
||||
bot._pending_approval["ws-1"] = PendingApproval( # type: ignore[attr-defined]
|
||||
bot._pending_approval[("ws-1", "cyc-9")] = PendingApproval( # type: ignore[attr-defined]
|
||||
channel="C1",
|
||||
message_ts="999.000",
|
||||
owner_user_id="U12345",
|
||||
cycle_id="cyc-9",
|
||||
)
|
||||
|
||||
# Event WITHOUT a cycle_id (pre-multi-cycle server): the legacy
|
||||
# fallback clears the ws's single tracked entry, as before.
|
||||
event = ApprovalResolvedEvent(ws_id="ws-1", approved=True)
|
||||
route = SlackRoute(channel="C1", user_id="U12345", thread_ts="123.456")
|
||||
_run(bot._on_ws_event("ws-1", route, event)) # type: ignore[attr-defined]
|
||||
|
||||
assert "ws-1" not in bot._pending_approval # type: ignore[attr-defined]
|
||||
assert not bot._pending_approval # type: ignore[attr-defined]
|
||||
client.chat_update.assert_awaited_once()
|
||||
|
||||
def test_link_prefix_does_not_hijack_regular_prompt(self) -> None:
|
||||
|
||||
@@ -336,10 +336,88 @@ def test_channel_default_alias_blanked_when_disabled(
|
||||
|
||||
|
||||
def test_models_payload_strips_secret_fields(storage: SQLiteBackend) -> None:
|
||||
"""Regression guard: only alias/model/provider land in the response,
|
||||
never api_key / base_url / context_window / capabilities."""
|
||||
"""Regression guard: only alias/model/provider (+ the derived
|
||||
effort_ladder) land in the response, never api_key / base_url /
|
||||
context_window / raw capabilities."""
|
||||
_seed_model(storage, definition_id="m1", alias="primary")
|
||||
body = _get_models(_make_client(storage))
|
||||
assert body["models"] == [
|
||||
{"alias": "primary", "model": "model-x", "provider": "openai-compatible"}
|
||||
]
|
||||
assert len(body["models"]) == 1
|
||||
entry = body["models"][0]
|
||||
assert set(entry) == {"alias", "model", "provider", "effort_ladder"}
|
||||
assert entry["alias"] == "primary"
|
||||
assert entry["model"] == "model-x"
|
||||
assert entry["provider"] == "openai-compatible"
|
||||
|
||||
|
||||
def test_effort_ladder_parses_string_capabilities(storage: SQLiteBackend) -> None:
|
||||
"""The capabilities column is a JSON STRING — the ladder must survive
|
||||
the parse (regression: .items() on the raw string threw and the
|
||||
guard silently dropped the field from every row)."""
|
||||
storage.create_model_definition(
|
||||
definition_id="m1",
|
||||
alias="qwen",
|
||||
model="qwen3.6-27b",
|
||||
provider="anthropic-compatible",
|
||||
base_url="http://localhost:8000",
|
||||
api_key="dummy",
|
||||
context_window=262144,
|
||||
capabilities='{"thinking_mode": "manual", "thinking_param": "enable_thinking"}',
|
||||
enabled=True,
|
||||
created_by="admin",
|
||||
)
|
||||
body = _get_models(_make_client(storage))
|
||||
ladder = {r["value"]: r["effective"] for r in body["models"][0]["effort_ladder"]}
|
||||
assert ladder["none"] == "off"
|
||||
assert ladder["medium"] == "on+medium"
|
||||
assert ladder["max"] == "on+max"
|
||||
|
||||
|
||||
def test_effort_ladder_key_survives_malformed_capabilities(
|
||||
storage: SQLiteBackend,
|
||||
) -> None:
|
||||
"""A capabilities column that fails to parse must not drop the key —
|
||||
every row carries ``effort_ladder`` (empty on failure) so clients can
|
||||
index it unconditionally instead of null-checking per row."""
|
||||
storage.create_model_definition(
|
||||
definition_id="m1",
|
||||
alias="broken",
|
||||
model="model-x",
|
||||
provider="openai-compatible",
|
||||
base_url="http://localhost:8000/v1",
|
||||
api_key="dummy",
|
||||
context_window=131072,
|
||||
capabilities="{not valid json",
|
||||
enabled=True,
|
||||
created_by="admin",
|
||||
)
|
||||
body = _get_models(_make_client(storage))
|
||||
entry = body["models"][0]
|
||||
assert set(entry) == {"alias", "model", "provider", "effort_ladder"}
|
||||
assert entry["effort_ladder"] == []
|
||||
|
||||
|
||||
def test_effort_ladder_honors_responses_api_surface(storage: SQLiteBackend) -> None:
|
||||
"""server_compat.api_surface (namespaced inside the capabilities JSON)
|
||||
switches the projection to the flat-param path — no template toggle."""
|
||||
caps = (
|
||||
'{"thinking_mode": "manual", "thinking_param": "enable_thinking",'
|
||||
' "reasoning_effort_values": ["low", "medium", "high"],'
|
||||
' "server_compat": {"api_surface": "responses"}}'
|
||||
)
|
||||
storage.create_model_definition(
|
||||
definition_id="m1",
|
||||
alias="mistral",
|
||||
model="mistral-medium",
|
||||
provider="openai-compatible",
|
||||
base_url="http://localhost:8000/v1",
|
||||
api_key="dummy",
|
||||
context_window=131072,
|
||||
capabilities=caps,
|
||||
enabled=True,
|
||||
created_by="admin",
|
||||
)
|
||||
body = _get_models(_make_client(storage))
|
||||
ladder = {r["value"]: r["effective"] for r in body["models"][0]["effort_ladder"]}
|
||||
# Responses surface: flat param only — no "on+"/"off" toggle tokens.
|
||||
assert ladder["medium"] == "medium"
|
||||
assert ladder["none"] == "default"
|
||||
|
||||
@@ -0,0 +1,106 @@
|
||||
"""``POST /v1/api/admin/models/effort-ladder`` — live modal projection.
|
||||
|
||||
Pure computation over (provider, model, unsaved capability overrides,
|
||||
api_surface); every malformed input must land as a 400, never a 500 —
|
||||
the body is operator-typed form state.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Any
|
||||
|
||||
from starlette.applications import Starlette
|
||||
from starlette.middleware import Middleware
|
||||
from starlette.routing import Route
|
||||
from starlette.testclient import TestClient
|
||||
|
||||
from tests._coord_test_helpers import _AuthMiddleware
|
||||
from turnstone.console.server import admin_effort_ladder
|
||||
|
||||
|
||||
def _make_client() -> TestClient:
|
||||
app = Starlette(
|
||||
routes=[Route("/v1/api/admin/models/effort-ladder", admin_effort_ladder, methods=["POST"])],
|
||||
middleware=[Middleware(_AuthMiddleware)],
|
||||
)
|
||||
client = TestClient(app)
|
||||
client.headers.update({"X-Test-User": "admin", "X-Test-Perms": "admin.models"})
|
||||
return client
|
||||
|
||||
|
||||
def _post(client: TestClient, body: Any) -> Any:
|
||||
return client.post("/v1/api/admin/models/effort-ladder", json=body)
|
||||
|
||||
|
||||
def test_valid_request_returns_ladder() -> None:
|
||||
resp = _post(
|
||||
_make_client(),
|
||||
{
|
||||
"provider": "anthropic-compatible",
|
||||
"model": "qwen3.6-27b",
|
||||
"capabilities": {"thinking_mode": "manual", "thinking_param": "enable_thinking"},
|
||||
},
|
||||
)
|
||||
assert resp.status_code == 200, resp.text
|
||||
ladder = {r["value"]: r["effective"] for r in resp.json()["ladder"]}
|
||||
assert ladder["none"] == "off"
|
||||
assert ladder["high"] == "on+high"
|
||||
|
||||
|
||||
def test_api_surface_switches_projection() -> None:
|
||||
body = {
|
||||
"provider": "openai-compatible",
|
||||
"model": "m",
|
||||
"capabilities": {
|
||||
"thinking_mode": "manual",
|
||||
"reasoning_effort_values": ["low", "medium", "high"],
|
||||
},
|
||||
}
|
||||
client = _make_client()
|
||||
chat = {r["value"]: r["effective"] for r in _post(client, body).json()["ladder"]}
|
||||
body["api_surface"] = "responses"
|
||||
responses = {r["value"]: r["effective"] for r in _post(client, body).json()["ladder"]}
|
||||
assert chat["medium"] == "on+medium" # toggle + flat on the chat surface
|
||||
assert responses["medium"] == "medium" # flat only on the responses surface
|
||||
|
||||
|
||||
def test_non_dict_json_body_is_400_not_500() -> None:
|
||||
client = _make_client()
|
||||
for body in (None, [], "x", 7):
|
||||
resp = _post(client, body)
|
||||
assert resp.status_code == 400, (body, resp.status_code, resp.text)
|
||||
|
||||
|
||||
def test_unknown_provider_is_400() -> None:
|
||||
resp = _post(_make_client(), {"provider": "nope", "model": "m"})
|
||||
assert resp.status_code == 400
|
||||
|
||||
|
||||
def test_missing_model_is_400() -> None:
|
||||
resp = _post(_make_client(), {"provider": "openai", "model": ""})
|
||||
assert resp.status_code == 400
|
||||
|
||||
|
||||
def test_non_dict_capabilities_is_400() -> None:
|
||||
resp = _post(_make_client(), {"provider": "openai", "model": "m", "capabilities": [1]})
|
||||
assert resp.status_code == 400
|
||||
|
||||
|
||||
def test_garbage_capability_value_types_are_400() -> None:
|
||||
"""Wrong-typed override values raise inside the resolver → clean 400."""
|
||||
resp = _post(
|
||||
_make_client(),
|
||||
{
|
||||
"provider": "anthropic",
|
||||
"model": "claude-fable-5",
|
||||
"capabilities": {"supports_effort": True, "effort_levels": 5},
|
||||
},
|
||||
)
|
||||
assert resp.status_code == 400
|
||||
|
||||
|
||||
def test_requires_admin_models_permission() -> None:
|
||||
client = _make_client()
|
||||
client.headers.update({"X-Test-Perms": "read"})
|
||||
resp = _post(client, {"provider": "openai", "model": "m"})
|
||||
assert resp.status_code in (401, 403)
|
||||
@@ -16,10 +16,10 @@ to ``SessionUIBase`` automatically enables:
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import threading
|
||||
from typing import Any
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
from tests.conftest import resolve_when_pending
|
||||
from turnstone.console.coordinator_ui import ConsoleCoordinatorUI
|
||||
|
||||
|
||||
@@ -153,7 +153,7 @@ def test_coord_heuristic_verdict_persists_to_storage() -> None:
|
||||
items[0]["_heuristic_verdict"] = hv
|
||||
|
||||
storage = MagicMock()
|
||||
timer = threading.Timer(0.05, lambda: ui.resolve_approval(False))
|
||||
timer = resolve_when_pending(ui, False)
|
||||
timer.start()
|
||||
try:
|
||||
with _patch_storage(storage):
|
||||
@@ -246,9 +246,8 @@ def test_coord_pending_approval_sets_activity_tag() -> None:
|
||||
def _capture_activity() -> None:
|
||||
captured["activity"] = ui._ws_current_activity
|
||||
captured["state"] = ui._ws_activity_state
|
||||
ui.resolve_approval(False)
|
||||
|
||||
timer = threading.Timer(0.05, _capture_activity)
|
||||
timer = resolve_when_pending(ui, False, before=_capture_activity)
|
||||
timer.start()
|
||||
try:
|
||||
with _patch_storage(MagicMock()):
|
||||
@@ -292,7 +291,7 @@ def test_coord_judge_pending_flag_dynamic_when_heuristic_present() -> None:
|
||||
captured_events: list[dict[str, Any]] = []
|
||||
ui._enqueue = captured_events.append # type: ignore[method-assign]
|
||||
|
||||
timer = threading.Timer(0.05, lambda: ui.resolve_approval(False))
|
||||
timer = resolve_when_pending(ui, False)
|
||||
timer.start()
|
||||
try:
|
||||
with _patch_storage(MagicMock()):
|
||||
@@ -338,7 +337,7 @@ def test_coord_judge_pending_false_when_no_heuristic_verdict() -> None:
|
||||
captured_events: list[dict[str, Any]] = []
|
||||
ui._enqueue = captured_events.append # type: ignore[method-assign]
|
||||
|
||||
timer = threading.Timer(0.05, lambda: ui.resolve_approval(False))
|
||||
timer = resolve_when_pending(ui, False)
|
||||
timer.start()
|
||||
try:
|
||||
with _patch_storage(MagicMock()):
|
||||
@@ -410,7 +409,7 @@ def test_coord_budget_override_prompts_even_under_blanket_auto_approve() -> None
|
||||
captured_events: list[dict[str, Any]] = []
|
||||
ui._enqueue = captured_events.append # type: ignore[method-assign]
|
||||
|
||||
timer = threading.Timer(0.05, lambda: ui.resolve_approval(True))
|
||||
timer = resolve_when_pending(ui, True)
|
||||
timer.start()
|
||||
try:
|
||||
with _patch_storage(MagicMock()):
|
||||
@@ -453,7 +452,7 @@ def test_coord_budget_override_survives_wildcard_allow_policy() -> None:
|
||||
captured_events: list[dict[str, Any]] = []
|
||||
ui._enqueue = captured_events.append # type: ignore[method-assign]
|
||||
|
||||
timer = threading.Timer(0.05, lambda: ui.resolve_approval(True))
|
||||
timer = resolve_when_pending(ui, True)
|
||||
timer.start()
|
||||
try:
|
||||
with _patch_storage(MagicMock()), _patch_policies({"__budget_override__": "allow"}):
|
||||
@@ -526,12 +525,16 @@ class TestBroadcastApprovalResolved:
|
||||
collector = MagicMock()
|
||||
ConsoleCoordinatorUI._collector = collector
|
||||
try:
|
||||
ui._broadcast_approval_resolved(True, "lgtm", always=True)
|
||||
ui._broadcast_approval_resolved(
|
||||
True, "lgtm", always=True, cycle_id="cyc-1", call_ids=("c-1", "c-2")
|
||||
)
|
||||
collector.emit_console_ws_approval_resolved.assert_called_once_with(
|
||||
"coord-a",
|
||||
approved=True,
|
||||
feedback="lgtm",
|
||||
always=True,
|
||||
cycle_id="cyc-1",
|
||||
call_ids=["c-1", "c-2"],
|
||||
)
|
||||
finally:
|
||||
ConsoleCoordinatorUI._collector = None
|
||||
@@ -547,6 +550,8 @@ class TestBroadcastApprovalResolved:
|
||||
approved=False,
|
||||
feedback="",
|
||||
always=False,
|
||||
cycle_id="",
|
||||
call_ids=[],
|
||||
)
|
||||
finally:
|
||||
ConsoleCoordinatorUI._collector = None
|
||||
|
||||
@@ -195,6 +195,21 @@ def test_emit_tolerates_collector_exception() -> None:
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_cleanup_ui_sweeps_all_approval_cycles_on_registry_uis() -> None:
|
||||
"""The real ConsoleCoordinatorUI carries the approval-cycle
|
||||
registry: cleanup denies + wakes EVERY parked gate via
|
||||
``resolve_all_approvals`` (parallel task agents can hold several),
|
||||
not the pre-cycle single-slot kick."""
|
||||
adapter, _ = _make_adapter()
|
||||
ws = _make_ws()
|
||||
ws.ui.resolve_all_approvals = MagicMock(return_value=2) # type: ignore[attr-defined]
|
||||
adapter.cleanup_ui(ws)
|
||||
ws.ui.resolve_all_approvals.assert_called_once_with( # type: ignore[attr-defined]
|
||||
False, "Workstream closed"
|
||||
)
|
||||
assert ws.ui._fg_event.is_set() # type: ignore[attr-defined]
|
||||
|
||||
|
||||
def test_cleanup_ui_unblocks_events_and_broadcasts_to_listeners() -> None:
|
||||
adapter, _ = _make_adapter()
|
||||
ws = _make_ws()
|
||||
|
||||
@@ -1103,18 +1103,7 @@ def test_approve_resolves_ui_event(storage):
|
||||
mgr = _build_mgr(storage)
|
||||
ws = mgr.create(user_id="user-1")
|
||||
assert isinstance(ws.ui, ConsoleCoordinatorUI)
|
||||
ws.ui._pending_approval = {
|
||||
"type": "approve_request",
|
||||
"items": [
|
||||
{
|
||||
"call_id": "c-1",
|
||||
"func_name": "spawn_workstream",
|
||||
"approval_label": "spawn_workstream",
|
||||
"needs_approval": True,
|
||||
}
|
||||
],
|
||||
}
|
||||
ws.ui._approval_event.clear()
|
||||
cycle = _seed_pending(ws, "c-1")
|
||||
client = _make_client(storage, coord_mgr=mgr, registry=_fake_registry())
|
||||
resp = client.post(
|
||||
f"/v1/api/workstreams/{ws.id}/approve",
|
||||
@@ -1122,34 +1111,46 @@ def test_approve_resolves_ui_event(storage):
|
||||
headers=_COORD_HEADERS,
|
||||
)
|
||||
assert resp.status_code == 200
|
||||
assert ws.ui._approval_event.is_set()
|
||||
assert ws.ui._approval_result == (True, None)
|
||||
assert resp.json()["cycle_id"] == cycle.cycle_id
|
||||
assert cycle.event.is_set()
|
||||
assert cycle.result == (True, None)
|
||||
assert "spawn_workstream" in ws.ui.auto_approve_tools
|
||||
|
||||
|
||||
def _seed_pending(ws, *call_ids: str) -> None:
|
||||
ws.ui._pending_approval = {
|
||||
def _seed_pending(ws, *call_ids: str, func_name: str = "spawn_workstream"):
|
||||
"""Register a live ApprovalCycle on the coord UI the way its
|
||||
``approve_tools`` gate does, returning the cycle for direct
|
||||
event/result assertions (the pre-cycle singleton
|
||||
``_approval_event`` / ``_approval_result`` slots are gone)."""
|
||||
from turnstone.core.session_ui_base import ApprovalCycle
|
||||
|
||||
items = [
|
||||
{
|
||||
"call_id": cid,
|
||||
"func_name": func_name,
|
||||
"approval_label": func_name,
|
||||
"needs_approval": True,
|
||||
}
|
||||
for cid in call_ids
|
||||
]
|
||||
card = {
|
||||
"type": "approve_request",
|
||||
"items": [
|
||||
{
|
||||
"call_id": cid,
|
||||
"func_name": "spawn_workstream",
|
||||
"approval_label": "spawn_workstream",
|
||||
"needs_approval": True,
|
||||
}
|
||||
for cid in call_ids
|
||||
],
|
||||
"cycle_id": f"cyc-{'-'.join(call_ids)}",
|
||||
"items": ws.ui._serialize_approval_items(items),
|
||||
"judge_pending": False,
|
||||
}
|
||||
ws.ui._approval_event.clear()
|
||||
cycle = ApprovalCycle(items, card, None)
|
||||
ws.ui._register_approval_cycle(cycle)
|
||||
return cycle
|
||||
|
||||
|
||||
def test_approve_409_on_stale_call_id(storage):
|
||||
"""Body call_id doesn't match any pending item → 409 with the
|
||||
current primary call_id so the UI can re-render against the
|
||||
new round."""
|
||||
current primary call_id + cycle_id so the UI can re-render
|
||||
against the new round."""
|
||||
mgr = _build_mgr(storage)
|
||||
ws = mgr.create(user_id="user-1")
|
||||
_seed_pending(ws, "c-current")
|
||||
cycle = _seed_pending(ws, "c-current")
|
||||
client = _make_client(storage, coord_mgr=mgr, registry=_fake_registry())
|
||||
resp = client.post(
|
||||
f"/v1/api/workstreams/{ws.id}/approve",
|
||||
@@ -1160,17 +1161,17 @@ def test_approve_409_on_stale_call_id(storage):
|
||||
body = resp.json()
|
||||
assert body["error"] == "stale call_id"
|
||||
assert body["current_call_id"] == "c-current"
|
||||
# Approval event must NOT be set — no resolve_approval ran.
|
||||
assert not ws.ui._approval_event.is_set()
|
||||
assert body["current_cycle_id"] == cycle.cycle_id
|
||||
# The live cycle must NOT have been resolved.
|
||||
assert not cycle.event.is_set()
|
||||
|
||||
|
||||
def test_approve_409_when_no_pending_and_call_id_sent(storage):
|
||||
"""Body sends a call_id but the UI has no pending approval —
|
||||
409 with current_call_id=None so the UI knows to clear the row."""
|
||||
"""Body sends a call_id but the UI has no live cycle — 409 with
|
||||
current_call_id=None so the UI knows to clear the row."""
|
||||
mgr = _build_mgr(storage)
|
||||
ws = mgr.create(user_id="user-1")
|
||||
# No _pending_approval seeded → ui._pending_approval is None.
|
||||
ws.ui._approval_event.clear()
|
||||
# No cycle registered.
|
||||
client = _make_client(storage, coord_mgr=mgr, registry=_fake_registry())
|
||||
resp = client.post(
|
||||
f"/v1/api/workstreams/{ws.id}/approve",
|
||||
@@ -1179,18 +1180,18 @@ def test_approve_409_when_no_pending_and_call_id_sent(storage):
|
||||
)
|
||||
assert resp.status_code == 409
|
||||
body = resp.json()
|
||||
assert body["error"] == "no pending approval"
|
||||
assert body["error"] == "stale call_id"
|
||||
assert body["current_call_id"] is None
|
||||
assert not ws.ui._approval_event.is_set()
|
||||
assert body["current_cycle_id"] is None
|
||||
|
||||
|
||||
def test_approve_no_call_id_preserves_backward_compat(storage):
|
||||
"""Existing clients (CLI, channel adapters) that omit call_id
|
||||
must still resolve approvals — the guard only kicks in when
|
||||
call_id is present in the body."""
|
||||
must still resolve approvals — a selector-less body lands on the
|
||||
oldest live cycle."""
|
||||
mgr = _build_mgr(storage)
|
||||
ws = mgr.create(user_id="user-1")
|
||||
_seed_pending(ws, "c-1")
|
||||
cycle = _seed_pending(ws, "c-1")
|
||||
client = _make_client(storage, coord_mgr=mgr, registry=_fake_registry())
|
||||
resp = client.post(
|
||||
f"/v1/api/workstreams/{ws.id}/approve",
|
||||
@@ -1198,18 +1199,18 @@ def test_approve_no_call_id_preserves_backward_compat(storage):
|
||||
headers=_COORD_HEADERS,
|
||||
)
|
||||
assert resp.status_code == 200
|
||||
assert ws.ui._approval_event.is_set()
|
||||
assert resp.json()["cycle_id"] == cycle.cycle_id
|
||||
assert cycle.event.is_set()
|
||||
|
||||
|
||||
def test_approve_no_call_id_no_pending_falls_through(storage):
|
||||
"""Legacy clients (no call_id) calling approve when pending is
|
||||
None hit the existing resolve_approval no-op path — the new
|
||||
guard must not change that behavior. Regression guard for the
|
||||
legacy code path that the call_id check intentionally bypasses."""
|
||||
def test_approve_no_call_id_no_pending_resolves_nothing(storage):
|
||||
"""Legacy clients (no call_id) calling approve with no live cycle:
|
||||
200 with ``cycle_id: null`` — the handler resolves NOTHING rather
|
||||
than racing a cycle that registers between its lookup and its
|
||||
resolve (the client can't have been looking at one)."""
|
||||
mgr = _build_mgr(storage)
|
||||
ws = mgr.create(user_id="user-1")
|
||||
ws.ui._approval_event.clear()
|
||||
# No _pending_approval seeded.
|
||||
# No cycle registered.
|
||||
client = _make_client(storage, coord_mgr=mgr, registry=_fake_registry())
|
||||
resp = client.post(
|
||||
f"/v1/api/workstreams/{ws.id}/approve",
|
||||
@@ -1217,7 +1218,7 @@ def test_approve_no_call_id_no_pending_falls_through(storage):
|
||||
headers=_COORD_HEADERS,
|
||||
)
|
||||
assert resp.status_code == 200
|
||||
assert ws.ui._approval_event.is_set()
|
||||
assert resp.json()["cycle_id"] is None
|
||||
|
||||
|
||||
def test_approve_call_id_matches_any_item_in_multi_envelope(storage):
|
||||
@@ -1226,7 +1227,7 @@ def test_approve_call_id_matches_any_item_in_multi_envelope(storage):
|
||||
one-boolean semantics of resolve_approval."""
|
||||
mgr = _build_mgr(storage)
|
||||
ws = mgr.create(user_id="user-1")
|
||||
_seed_pending(ws, "c-1", "c-2", "c-3")
|
||||
cycle = _seed_pending(ws, "c-1", "c-2", "c-3")
|
||||
client = _make_client(storage, coord_mgr=mgr, registry=_fake_registry())
|
||||
resp = client.post(
|
||||
f"/v1/api/workstreams/{ws.id}/approve",
|
||||
@@ -1234,7 +1235,61 @@ def test_approve_call_id_matches_any_item_in_multi_envelope(storage):
|
||||
headers=_COORD_HEADERS,
|
||||
)
|
||||
assert resp.status_code == 200
|
||||
assert ws.ui._approval_event.is_set()
|
||||
assert cycle.event.is_set()
|
||||
|
||||
|
||||
def test_selectorless_always_whitelists_only_the_resolved_oldest_cycle(storage):
|
||||
"""sweep-3 regression: with several live cycles, a selector-less
|
||||
"Approve + Always" must whitelist the tools of the cycle it
|
||||
actually resolved (the oldest) — not a sibling's."""
|
||||
mgr = _build_mgr(storage)
|
||||
ws = mgr.create(user_id="user-1")
|
||||
oldest = _seed_pending(ws, "a-1", func_name="spawn_workstream")
|
||||
newer = _seed_pending(ws, "b-1", func_name="send_message")
|
||||
client = _make_client(storage, coord_mgr=mgr, registry=_fake_registry())
|
||||
resp = client.post(
|
||||
f"/v1/api/workstreams/{ws.id}/approve",
|
||||
json={"approved": True, "always": True}, # no selector
|
||||
headers=_COORD_HEADERS,
|
||||
)
|
||||
assert resp.status_code == 200
|
||||
assert resp.json()["cycle_id"] == oldest.cycle_id
|
||||
assert oldest.event.is_set()
|
||||
assert not newer.event.is_set()
|
||||
assert "spawn_workstream" in ws.ui.auto_approve_tools
|
||||
assert "send_message" not in ws.ui.auto_approve_tools
|
||||
|
||||
|
||||
def test_approve_always_skips_whitelist_when_pinned_cycle_lost_the_race(storage):
|
||||
"""sweep-3 regression: the handler collects always-names from the
|
||||
cycle its lookup pinned; if that cycle is resolved by someone else
|
||||
(gate timeout, peer tab) between lookup and resolve, the whitelist
|
||||
must NOT grow — approving a card that already resolved must not
|
||||
auto-approve anything."""
|
||||
mgr = _build_mgr(storage)
|
||||
ws = mgr.create(user_id="user-1")
|
||||
_seed_pending(ws, "a-1", func_name="spawn_workstream")
|
||||
ui = ws.ui
|
||||
real_find = ui.find_approval_cycle
|
||||
|
||||
def racing_find(**kwargs):
|
||||
card = real_find(**kwargs)
|
||||
if card is not None:
|
||||
# A concurrent resolver wins the gap between the handler's
|
||||
# lookup and its (pinned) resolve.
|
||||
ui.resolve_approval(False, "raced", cycle_id=card["cycle_id"])
|
||||
return card
|
||||
|
||||
ui.find_approval_cycle = racing_find
|
||||
client = _make_client(storage, coord_mgr=mgr, registry=_fake_registry())
|
||||
resp = client.post(
|
||||
f"/v1/api/workstreams/{ws.id}/approve",
|
||||
json={"approved": True, "always": True},
|
||||
headers=_COORD_HEADERS,
|
||||
)
|
||||
assert resp.status_code == 200
|
||||
assert resp.json()["cycle_id"] is None
|
||||
assert "spawn_workstream" not in ws.ui.auto_approve_tools
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
@@ -1521,15 +1576,19 @@ def test_export_404_when_kind_interactive(storage):
|
||||
|
||||
|
||||
def test_cancel_resolves_pending_approval(storage):
|
||||
"""Cancel addresses the workstream, not one batch — EVERY live
|
||||
cycle resolves (parallel task agents can hold several gates)."""
|
||||
mgr = _build_mgr(storage)
|
||||
ws = mgr.create(user_id="user-1")
|
||||
assert isinstance(ws.ui, ConsoleCoordinatorUI)
|
||||
ws.ui._pending_approval = {"type": "approve_request", "items": []}
|
||||
ws.ui._approval_event.clear()
|
||||
first = _seed_pending(ws, "c-1")
|
||||
second = _seed_pending(ws, "c-2")
|
||||
client = _make_client(storage, coord_mgr=mgr, registry=_fake_registry())
|
||||
resp = client.post(f"/v1/api/workstreams/{ws.id}/cancel", headers=_COORD_HEADERS)
|
||||
assert resp.status_code == 200
|
||||
assert ws.ui._approval_event.is_set()
|
||||
assert first.event.is_set()
|
||||
assert second.event.is_set()
|
||||
assert first.result == (False, "Cancelled by user")
|
||||
|
||||
|
||||
def test_cancel_response_always_includes_dropped_key(storage):
|
||||
@@ -2399,6 +2458,7 @@ def test_cluster_inspect_node_backed_pending_approval_detail_passes_through(stor
|
||||
ws_id = "f0" * 16
|
||||
_seed_node_workstream(storage, ws_id=ws_id, node_id="node-a")
|
||||
detail = {
|
||||
"cycle_id": "cyc-bash",
|
||||
"call_id": "c-bash",
|
||||
"judge_pending": False,
|
||||
"items": [
|
||||
@@ -2427,7 +2487,7 @@ def test_cluster_inspect_node_backed_pending_approval_detail_passes_through(stor
|
||||
"activity_state": "approval",
|
||||
"activity": "awaiting approval",
|
||||
"tokens": 100,
|
||||
"pending_approval_detail": detail,
|
||||
"pending_approval_details": [detail],
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -2438,7 +2498,7 @@ def test_cluster_inspect_node_backed_pending_approval_detail_passes_through(stor
|
||||
assert resp.status_code == 200
|
||||
live = resp.json()["live"]
|
||||
assert live["pending_approval"] is True # derived bool, existing behavior
|
||||
assert live["pending_approval_detail"] == detail # full payload, new behavior
|
||||
assert live["pending_approval_details"] == [detail] # full payload passthrough
|
||||
|
||||
|
||||
def test_cluster_inspect_node_backed_pending_approval_synthesized(storage):
|
||||
|
||||
@@ -313,17 +313,17 @@ def test_coordinator_js_handle_child_state_no_longer_reads_sse_pending_approval_
|
||||
)
|
||||
|
||||
# The merge body must preserve BOTH pending_approval and
|
||||
# pending_approval_detail from prev — preserving only one would
|
||||
# pending_approval_details from prev — preserving only one would
|
||||
# render a row with a phantom badge but no buttons (or vice versa).
|
||||
merge_body = re.search(
|
||||
r"mergedLive\s*=\s*Object\.assign\(\s*\{\}\s*,\s*live\s*,\s*\{"
|
||||
r"[^}]*pending_approval:\s*prev\.live\.pending_approval[^}]*"
|
||||
r"pending_approval_detail:\s*prev\.live\.pending_approval_detail",
|
||||
r"pending_approval_details:\s*prev\.live\.pending_approval_details",
|
||||
body,
|
||||
)
|
||||
assert merge_body is not None, (
|
||||
"Merge body must preserve both pending_approval AND "
|
||||
"pending_approval_detail from prev.live — preserving only one "
|
||||
"pending_approval_details from prev.live — preserving only one "
|
||||
"creates a half-rendered approval row."
|
||||
)
|
||||
|
||||
|
||||
@@ -198,6 +198,23 @@ def test_spawn_prepare_needs_approval(coord_session):
|
||||
assert item["skill"] == "s"
|
||||
|
||||
|
||||
def test_spawn_prepare_denies_high_risk_skill(coord_session):
|
||||
"""Review fix: the high/critical-risk gate that blocks skills(load) also
|
||||
blocks spawn_workstream(skill=…), so a child spawn can't route around it."""
|
||||
sess, _coord, _ui = coord_session
|
||||
with patch("turnstone.core.session.get_storage") as gs:
|
||||
gs.return_value.get_prompt_template_by_name.return_value = {
|
||||
"name": "danger",
|
||||
"risk_level": "critical",
|
||||
}
|
||||
item = sess._prepare_tool(
|
||||
_tc("spawn_workstream", {"initial_message": "go", "skill": "danger"})
|
||||
)
|
||||
assert "error" in item
|
||||
assert "/skill danger" in item["error"]
|
||||
assert item.get("needs_approval") is not True
|
||||
|
||||
|
||||
def test_spawn_exec_calls_client_and_returns_summary(coord_session):
|
||||
sess, coord, _ui = coord_session
|
||||
coord.spawn.return_value = {
|
||||
|
||||
@@ -0,0 +1,239 @@
|
||||
"""Tests for the effective effort-ladder projection.
|
||||
|
||||
The ladder must mirror the request-time mapping functions exactly —
|
||||
equal ``effective`` tokens promise byte-identical effort behavior on
|
||||
the wire, which is what the UI annotations lean on.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from turnstone.core.providers._protocol import ModelCapabilities
|
||||
from turnstone.core.providers.effort_ladder import (
|
||||
KNOB_VALUES,
|
||||
effort_ladder,
|
||||
effort_ladder_for_model,
|
||||
)
|
||||
|
||||
|
||||
def _as_map(ladder: list[dict[str, str]]) -> dict[str, str]:
|
||||
assert [r["value"] for r in ladder] == list(KNOB_VALUES)
|
||||
return {r["value"]: r["effective"] for r in ladder}
|
||||
|
||||
|
||||
class TestLocalLanes:
|
||||
def test_toggle_engaged_carries_graded_value_per_position(self) -> None:
|
||||
"""No declared effort key: the toggle rides the knob AND the graded
|
||||
value is forwarded under the fallback template key — the user's
|
||||
effort setting always reaches the wire (a template that doesn't
|
||||
reference the kwarg ignores it), so every position is distinct."""
|
||||
caps = ModelCapabilities(thinking_mode="manual", thinking_param="enable_thinking")
|
||||
eff = _as_map(effort_ladder("anthropic-compatible", caps))
|
||||
assert eff["none"] == "off"
|
||||
assert eff["minimal"] == "on+minimal"
|
||||
assert eff["max"] == "on+max"
|
||||
assert len({eff[k] for k in KNOB_VALUES}) == len(KNOB_VALUES)
|
||||
|
||||
def test_freeform_effort_param_forwards_each_value(self) -> None:
|
||||
"""deepseek-style config: toggle + verbatim effort per position."""
|
||||
caps = ModelCapabilities(
|
||||
thinking_mode="manual",
|
||||
thinking_param="thinking",
|
||||
effort_param="reasoning_effort",
|
||||
)
|
||||
eff = _as_map(effort_ladder("anthropic-compatible", caps))
|
||||
assert eff["none"] == "off"
|
||||
assert eff["low"] == "on+low"
|
||||
assert eff["max"] == "on+max"
|
||||
|
||||
def test_validated_effort_param_shows_snapping(self) -> None:
|
||||
"""Off-list positions round up onto the declared values; above the
|
||||
ceiling they ride the ceiling — never the (possibly lower) default."""
|
||||
caps = ModelCapabilities(
|
||||
thinking_mode="manual",
|
||||
thinking_param="enable_thinking",
|
||||
effort_param="reasoning_effort",
|
||||
reasoning_effort_values=("low", "medium", "high"),
|
||||
default_reasoning_effort="medium",
|
||||
)
|
||||
eff = _as_map(effort_ladder("openai-compatible", caps))
|
||||
assert eff["minimal"] == "on+low"
|
||||
assert eff["high"] == "on+high"
|
||||
assert eff["xhigh"] == "on+high"
|
||||
assert eff["max"] == "on+high"
|
||||
|
||||
def test_openai_compatible_flat_param_without_effort_param(self) -> None:
|
||||
caps = ModelCapabilities(
|
||||
reasoning_effort_values=("low", "medium", "high"),
|
||||
default_reasoning_effort="medium",
|
||||
)
|
||||
eff = _as_map(effort_ladder("openai-compatible", caps))
|
||||
assert eff["none"] == "default"
|
||||
assert eff["high"] == "high"
|
||||
assert eff["xhigh"] == "high" # ceiling, not default
|
||||
|
||||
def test_adaptive_local_never_off(self) -> None:
|
||||
caps = ModelCapabilities(thinking_mode="adaptive", thinking_param="enable_thinking")
|
||||
eff = _as_map(effort_ladder("openai-compatible", caps))
|
||||
assert eff["none"] == "on"
|
||||
assert eff["max"] == "on"
|
||||
|
||||
|
||||
class TestNativeAnthropicLane:
|
||||
def test_adaptive_with_effort_levels(self) -> None:
|
||||
caps = ModelCapabilities(
|
||||
thinking_mode="adaptive",
|
||||
supports_effort=True,
|
||||
effort_levels=("low", "medium", "high", "xhigh", "max"),
|
||||
)
|
||||
eff = _as_map(effort_ladder("anthropic", caps))
|
||||
assert eff["none"] == "adaptive" # thinking on, model decides
|
||||
assert eff["minimal"] == "low" # rounds up onto the declared levels
|
||||
assert eff["low"] == "low"
|
||||
assert eff["max"] == "max"
|
||||
|
||||
def test_sonnet_5_registry_row(self) -> None:
|
||||
"""claude-sonnet-5: adaptive + full effort ladder incl. xhigh/max —
|
||||
every knob level above none is a distinct wire behavior."""
|
||||
eff = _as_map(effort_ladder_for_model("anthropic", "claude-sonnet-5", None))
|
||||
assert eff["none"] == "adaptive"
|
||||
assert eff["minimal"] == "low" # rounds up onto declared levels
|
||||
assert eff["low"] == "low"
|
||||
assert eff["xhigh"] == "xhigh"
|
||||
assert eff["max"] == "max"
|
||||
|
||||
def test_sonnet_4_6_xhigh_rides_max(self) -> None:
|
||||
"""Sonnet 4.6 declares (low, medium, high, max) — no xhigh, so the
|
||||
knob's xhigh snaps up onto max rather than down onto high."""
|
||||
eff = _as_map(effort_ladder_for_model("anthropic", "claude-sonnet-4-6", None))
|
||||
assert eff["high"] == "high"
|
||||
assert eff["xhigh"] == "max"
|
||||
assert eff["max"] == "max"
|
||||
|
||||
def test_manual_budget_ladder(self) -> None:
|
||||
"""Budgets are monotone over the whole knob domain."""
|
||||
caps = ModelCapabilities(thinking_mode="manual")
|
||||
eff = _as_map(effort_ladder("anthropic", caps))
|
||||
assert eff["none"] == "off"
|
||||
assert eff["minimal"] == eff["low"] == "budget:1024" # 1024 = API floor
|
||||
assert eff["medium"] == "budget:4096"
|
||||
assert eff["high"] == "budget:16384"
|
||||
assert eff["xhigh"] == "budget:32768"
|
||||
assert eff["max"] == "budget:65536"
|
||||
|
||||
|
||||
class TestFlatParamLanes:
|
||||
def test_google_default_caps(self) -> None:
|
||||
eff = _as_map(effort_ladder_for_model("google", "gemini-3-flash", None))
|
||||
assert eff["none"] == "default"
|
||||
assert eff["minimal"] == "minimal"
|
||||
assert eff["high"] == "high"
|
||||
assert eff["xhigh"] == eff["max"] == "high"
|
||||
|
||||
def test_google_override_routes_through_chat_lane(self) -> None:
|
||||
"""GoogleProvider inherits _finalize_extra_body — a thinking_mode
|
||||
override changes real requests, and the ladder must mirror it."""
|
||||
eff = _as_map(
|
||||
effort_ladder_for_model(
|
||||
"google",
|
||||
"gemini-3-flash",
|
||||
{"thinking_mode": "manual", "thinking_param": "enable_thinking"},
|
||||
)
|
||||
)
|
||||
assert eff["none"] == "off"
|
||||
assert eff["medium"] == "on+medium" # toggle + inherited flat param
|
||||
|
||||
def test_responses_surface_projects_flat_only(self) -> None:
|
||||
caps_overrides = {
|
||||
"thinking_mode": "manual",
|
||||
"reasoning_effort_values": ["low", "medium", "high"],
|
||||
}
|
||||
chat = _as_map(effort_ladder_for_model("openai-compatible", "m", caps_overrides))
|
||||
responses = _as_map(
|
||||
effort_ladder_for_model(
|
||||
"openai-compatible", "m", caps_overrides, api_surface="responses"
|
||||
)
|
||||
)
|
||||
assert chat["medium"] == "on+medium"
|
||||
assert responses["medium"] == "medium"
|
||||
assert responses["none"] == "default"
|
||||
|
||||
def test_xai_projects_flat_only(self) -> None:
|
||||
"""grok-4.3 declares values (none/low/medium/high, default low);
|
||||
knob positions above the ceiling ride the ceiling (high). The
|
||||
declared "none" IS forwarded for the knob's off position (xAI
|
||||
documents it as disabling reasoning) but is never a snap target
|
||||
for other positions."""
|
||||
eff = _as_map(effort_ladder_for_model("xai", "grok-4.3", None))
|
||||
assert eff["none"] == "none" # explicit disable, declared by grok
|
||||
assert eff["minimal"] == "low"
|
||||
assert eff["low"] == "low"
|
||||
assert eff["high"] == "high"
|
||||
assert eff["xhigh"] == eff["max"] == "high"
|
||||
|
||||
def test_xai_ignores_template_overrides(self) -> None:
|
||||
"""XAIProvider subclasses OpenAIResponsesProvider, which drops
|
||||
extra_body — a thinking_mode/effort_param override cannot change
|
||||
an xai request, so it must not change the ladder either."""
|
||||
eff = _as_map(
|
||||
effort_ladder_for_model(
|
||||
"xai",
|
||||
"grok-4.3",
|
||||
{
|
||||
"thinking_mode": "manual",
|
||||
"thinking_param": "enable_thinking",
|
||||
"effort_param": "reasoning_effort",
|
||||
},
|
||||
)
|
||||
)
|
||||
assert eff["none"] == "none" # flat channel, not an "off" toggle
|
||||
assert eff["medium"] == "medium"
|
||||
assert all("+" not in v and v not in ("on", "off") for v in eff.values())
|
||||
|
||||
def test_openai_gpt55_registry_row(self) -> None:
|
||||
"""gpt-5.5 declares none/low/medium/high/xhigh with default medium:
|
||||
knob none sends the explicit "none" level (server default is
|
||||
MEDIUM, so omission would not disable), max rides the xhigh
|
||||
ceiling, minimal rounds up to low."""
|
||||
eff = _as_map(effort_ladder_for_model("openai", "gpt-5.5", None))
|
||||
assert eff["none"] == "none"
|
||||
assert eff["minimal"] == "low"
|
||||
assert eff["xhigh"] == "xhigh"
|
||||
assert eff["max"] == "xhigh"
|
||||
|
||||
def test_openai_o3_registry_row(self) -> None:
|
||||
"""o-series (except o1-mini) accept low/medium/high; no declared
|
||||
"none" level, so the knob's off position omits the param."""
|
||||
eff = _as_map(effort_ladder_for_model("openai", "o3", None))
|
||||
assert eff["none"] == "default"
|
||||
assert eff["minimal"] == "low"
|
||||
assert eff["medium"] == "medium"
|
||||
assert eff["xhigh"] == eff["max"] == "high"
|
||||
|
||||
def test_openai_codex_max_has_xhigh(self) -> None:
|
||||
"""gpt-5.1-codex-max must not prefix-fall onto the gpt-5.1 row
|
||||
(which lacks xhigh) — xhigh reaches the wire verbatim."""
|
||||
eff = _as_map(effort_ladder_for_model("openai", "gpt-5.1-codex-max", None))
|
||||
assert eff["xhigh"] == "xhigh"
|
||||
assert eff["max"] == "xhigh"
|
||||
|
||||
def test_anthropic_effort_applies_even_with_thinking_mode_none(self) -> None:
|
||||
"""output_config gates on supports_effort alone at request time."""
|
||||
caps = ModelCapabilities(
|
||||
thinking_mode="none",
|
||||
supports_effort=True,
|
||||
effort_levels=("low", "medium", "high"),
|
||||
)
|
||||
eff = _as_map(effort_ladder("anthropic", caps))
|
||||
assert eff["high"] == "high"
|
||||
assert eff["none"] == "default"
|
||||
|
||||
def test_overrides_merge_and_unknown_keys_ignored(self) -> None:
|
||||
eff = _as_map(
|
||||
effort_ladder_for_model(
|
||||
"google",
|
||||
"gemini-3-flash",
|
||||
{"reasoning_effort_values": [], "not_a_field": True},
|
||||
)
|
||||
)
|
||||
# Operator cleared the values → nothing effort-related is sent.
|
||||
assert set(eff.values()) == {"default"}
|
||||
@@ -0,0 +1,410 @@
|
||||
"""Ladder↔wire parity harness — the effort ladder must tell the truth.
|
||||
|
||||
``effort_ladder`` *projects* the session effort knob through the same
|
||||
mapping functions the providers use at request time. This suite proves
|
||||
that projection against the REAL request path: for every provider lane
|
||||
and capability shape, each knob position is driven through the actual
|
||||
provider ``create_streaming`` against a recording fake client (the same
|
||||
SDK-seam capture the wire-payload goldens use), the effort-relevant
|
||||
subset of the captured kwargs is extracted, and it must equal what the
|
||||
ladder token decodes to. Two invariants per shape:
|
||||
|
||||
1. **Semantics** — each ladder token decodes to an expected wire subset
|
||||
(``on``/``off`` ⇒ the chat-template toggle, ``budget:N`` ⇒ Anthropic
|
||||
thinking budget, a bare level ⇒ the lane's flat/effort channel) and
|
||||
the observed wire subset must match it exactly.
|
||||
2. **Grouping** — the ladder's core promise: two knob positions carry
|
||||
equal ``effective`` tokens if and only if they produce identical
|
||||
effort-relevant wire payloads.
|
||||
|
||||
A failure here means the UI annotates behavior the wire does not have —
|
||||
the bug class that shipped xai in the ladder's chat-lane set even though
|
||||
``XAIProvider`` rides the Responses surface, which drops ``extra_body``.
|
||||
|
||||
The harness goes through ``create_provider`` (not direct classes) so the
|
||||
provider ROUTING the ladder assumes — e.g. ``api_surface="responses"``
|
||||
selecting the Responses adapter — is itself under test.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import contextlib
|
||||
import dataclasses
|
||||
import itertools
|
||||
from typing import Any
|
||||
|
||||
import pytest
|
||||
|
||||
from tests._wire_capture import RecordingClient
|
||||
from turnstone.core.providers import create_provider
|
||||
from turnstone.core.providers._protocol import (
|
||||
EFFORT_TEMPLATE_FALLBACK_PARAM,
|
||||
ModelCapabilities,
|
||||
)
|
||||
from turnstone.core.providers.effort_ladder import KNOB_VALUES, effort_ladder
|
||||
|
||||
# Above the largest manual-mode thinking budget (max: 65536) so the
|
||||
# request path's budget<max_tokens clamp never fires — the ladder
|
||||
# documents budgets unclamped, so the capture must be too. (At small
|
||||
# per-request max_tokens the clamp can genuinely alias adjacent budget
|
||||
# tiers on the wire; that is the ladder's documented approximation, not
|
||||
# a parity break.)
|
||||
_MAX_TOKENS = 128_000
|
||||
|
||||
|
||||
@dataclasses.dataclass(frozen=True)
|
||||
class Shape:
|
||||
"""One (provider lane, capability shape) point of the parity matrix."""
|
||||
|
||||
id: str
|
||||
provider: str
|
||||
caps: ModelCapabilities
|
||||
api_surface: str = ""
|
||||
model: str = "m"
|
||||
|
||||
|
||||
# Real registry rows for the lanes whose defaults carry effort values —
|
||||
# parity should cover what ships, not only synthetic shapes.
|
||||
_GEMINI_CAPS = create_provider("google").get_capabilities("gemini-3-flash")
|
||||
_GROK_CAPS = create_provider("xai").get_capabilities("grok-4.3")
|
||||
_GPT55_CAPS = create_provider("openai").get_capabilities("gpt-5.5")
|
||||
|
||||
SHAPES: tuple[Shape, ...] = (
|
||||
# -- anthropic-compatible (vLLM /v1/messages): template channel only --
|
||||
Shape(
|
||||
"compat-toggle-manual",
|
||||
"anthropic-compatible",
|
||||
ModelCapabilities(thinking_mode="manual", thinking_param="enable_thinking"),
|
||||
),
|
||||
Shape(
|
||||
"compat-toggle-adaptive",
|
||||
"anthropic-compatible",
|
||||
ModelCapabilities(thinking_mode="adaptive", thinking_param="enable_thinking"),
|
||||
),
|
||||
Shape(
|
||||
"compat-freeform-effort",
|
||||
"anthropic-compatible",
|
||||
ModelCapabilities(
|
||||
thinking_mode="manual",
|
||||
thinking_param="thinking",
|
||||
effort_param="reasoning_effort",
|
||||
),
|
||||
),
|
||||
Shape(
|
||||
# DeepSeek-V4 official contract: toggle + effort in {high, max}.
|
||||
"compat-validated-effort",
|
||||
"anthropic-compatible",
|
||||
ModelCapabilities(
|
||||
thinking_mode="manual",
|
||||
thinking_param="thinking",
|
||||
effort_param="reasoning_effort",
|
||||
reasoning_effort_values=("high", "max"),
|
||||
default_reasoning_effort="high",
|
||||
),
|
||||
),
|
||||
Shape(
|
||||
"compat-inert",
|
||||
"anthropic-compatible",
|
||||
ModelCapabilities(thinking_mode="none"),
|
||||
),
|
||||
# -- openai-compatible on the Chat Completions surface: both channels --
|
||||
Shape(
|
||||
"oc-toggle-only",
|
||||
"openai-compatible",
|
||||
ModelCapabilities(thinking_mode="manual", thinking_param="enable_thinking"),
|
||||
),
|
||||
Shape(
|
||||
"oc-toggle-plus-flat",
|
||||
"openai-compatible",
|
||||
ModelCapabilities(
|
||||
thinking_mode="manual",
|
||||
thinking_param="enable_thinking",
|
||||
reasoning_effort_values=("low", "medium", "high"),
|
||||
default_reasoning_effort="medium",
|
||||
),
|
||||
),
|
||||
Shape(
|
||||
"oc-effort-param-suppresses-flat",
|
||||
"openai-compatible",
|
||||
ModelCapabilities(
|
||||
thinking_mode="manual",
|
||||
thinking_param="enable_thinking",
|
||||
effort_param="reasoning_effort",
|
||||
reasoning_effort_values=("low", "medium", "high"),
|
||||
default_reasoning_effort="medium",
|
||||
),
|
||||
),
|
||||
Shape(
|
||||
"oc-flat-only",
|
||||
"openai-compatible",
|
||||
ModelCapabilities(
|
||||
reasoning_effort_values=("low", "medium", "high"),
|
||||
default_reasoning_effort="medium",
|
||||
),
|
||||
),
|
||||
Shape(
|
||||
"oc-adaptive",
|
||||
"openai-compatible",
|
||||
ModelCapabilities(thinking_mode="adaptive", thinking_param="enable_thinking"),
|
||||
),
|
||||
# -- openai-compatible pinned to the Responses surface: template caps
|
||||
# become inert and only the native flat channel remains --
|
||||
Shape(
|
||||
"oc-responses-surface",
|
||||
"openai-compatible",
|
||||
ModelCapabilities(
|
||||
thinking_mode="manual",
|
||||
thinking_param="enable_thinking",
|
||||
effort_param="reasoning_effort",
|
||||
reasoning_effort_values=("low", "medium", "high"),
|
||||
default_reasoning_effort="medium",
|
||||
),
|
||||
api_surface="responses",
|
||||
),
|
||||
# -- commercial flat lanes --
|
||||
Shape(
|
||||
# Real registry row: none/low/medium/high/xhigh, default medium.
|
||||
# Knob none must send the EXPLICIT "none" level (omission would
|
||||
# leave the server default medium reasoning on); knob max rides
|
||||
# the xhigh ceiling.
|
||||
"openai-gpt-5.5",
|
||||
"openai",
|
||||
_GPT55_CAPS,
|
||||
model="gpt-5.5",
|
||||
),
|
||||
Shape("google-default", "google", _GEMINI_CAPS, model="gemini-3-flash"),
|
||||
Shape(
|
||||
# GoogleProvider subclasses the chat provider, so a template
|
||||
# override DOES change real requests — hybrid toggle + flat.
|
||||
"google-manual-override",
|
||||
"google",
|
||||
dataclasses.replace(_GEMINI_CAPS, thinking_mode="manual", thinking_param="enable_thinking"),
|
||||
model="gemini-3-flash",
|
||||
),
|
||||
Shape("xai-default", "xai", _GROK_CAPS, model="grok-4.3"),
|
||||
Shape(
|
||||
# XAIProvider rides the Responses surface: template overrides are
|
||||
# inert on the wire, and the ladder must not pretend otherwise.
|
||||
"xai-template-override-inert",
|
||||
"xai",
|
||||
dataclasses.replace(
|
||||
_GROK_CAPS,
|
||||
thinking_mode="manual",
|
||||
thinking_param="enable_thinking",
|
||||
effort_param="reasoning_effort",
|
||||
),
|
||||
model="grok-4.3",
|
||||
),
|
||||
# -- native Anthropic --
|
||||
Shape(
|
||||
"anthropic-adaptive-effort",
|
||||
"anthropic",
|
||||
ModelCapabilities(
|
||||
thinking_mode="adaptive",
|
||||
supports_effort=True,
|
||||
effort_levels=("low", "medium", "high", "xhigh", "max"),
|
||||
),
|
||||
model="claude-fable-5",
|
||||
),
|
||||
Shape(
|
||||
"anthropic-adaptive-plain",
|
||||
"anthropic",
|
||||
ModelCapabilities(thinking_mode="adaptive"),
|
||||
model="claude-fable-5",
|
||||
),
|
||||
Shape(
|
||||
"anthropic-manual-budgets",
|
||||
"anthropic",
|
||||
ModelCapabilities(thinking_mode="manual"),
|
||||
model="claude-3-7-sonnet-latest",
|
||||
),
|
||||
Shape(
|
||||
"anthropic-manual-plus-effort",
|
||||
"anthropic",
|
||||
ModelCapabilities(
|
||||
thinking_mode="manual",
|
||||
supports_effort=True,
|
||||
effort_levels=("low", "medium", "high"),
|
||||
),
|
||||
model="claude-3-7-sonnet-latest",
|
||||
),
|
||||
Shape(
|
||||
"anthropic-none-effort",
|
||||
"anthropic",
|
||||
ModelCapabilities(
|
||||
thinking_mode="none",
|
||||
supports_effort=True,
|
||||
effort_levels=("low", "medium", "high"),
|
||||
),
|
||||
model="claude-3-5-haiku-latest",
|
||||
),
|
||||
Shape(
|
||||
"anthropic-inert",
|
||||
"anthropic",
|
||||
ModelCapabilities(thinking_mode="none"),
|
||||
model="claude-3-5-haiku-latest",
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# Wire capture + effort-subset extraction
|
||||
# --------------------------------------------------------------------------- #
|
||||
|
||||
|
||||
def _wire_payload(shape: Shape, knob: str) -> dict[str, Any]:
|
||||
"""Drive the real provider request path; return the captured SDK kwargs."""
|
||||
provider = create_provider(shape.provider, api_surface=shape.api_surface or None)
|
||||
client = RecordingClient()
|
||||
gen = provider.create_streaming(
|
||||
client=client,
|
||||
model=shape.model,
|
||||
messages=[{"role": "user", "content": "hi"}],
|
||||
max_tokens=_MAX_TOKENS,
|
||||
reasoning_effort=knob,
|
||||
capabilities=shape.caps,
|
||||
)
|
||||
# kwargs are recorded eagerly during the call above; close the
|
||||
# unconsumed iterator so stream-manager cleanup runs on the stub.
|
||||
close = getattr(gen, "close", None)
|
||||
if callable(close):
|
||||
with contextlib.suppress(Exception):
|
||||
close()
|
||||
assert "payload" in client.captured, f"{shape.id}: provider made no SDK call"
|
||||
return dict(client.captured["payload"])
|
||||
|
||||
|
||||
def _effort_wire_subset(payload: dict[str, Any], shape: Shape) -> dict[str, Any]:
|
||||
"""Every effort-related lever in *payload*, normalized across lanes.
|
||||
|
||||
Keys: ``thinking`` (native Anthropic param), ``output_effort``
|
||||
(Anthropic ``output_config.effort``), ``flat`` (Chat Completions
|
||||
``reasoning_effort`` / Responses ``reasoning.effort``), ``toggle``
|
||||
and ``template_effort`` (``extra_body.chat_template_kwargs`` — the
|
||||
graded key is ``caps.effort_param``, else the fallback template key
|
||||
on the anthropic-compatible lane, whose only effort channel is the
|
||||
template).
|
||||
"""
|
||||
caps = shape.caps
|
||||
effort_key = caps.effort_param or (
|
||||
EFFORT_TEMPLATE_FALLBACK_PARAM if shape.provider == "anthropic-compatible" else ""
|
||||
)
|
||||
subset: dict[str, Any] = {}
|
||||
if "thinking" in payload:
|
||||
subset["thinking"] = payload["thinking"]
|
||||
output_config = payload.get("output_config")
|
||||
if isinstance(output_config, dict) and "effort" in output_config:
|
||||
subset["output_effort"] = output_config["effort"]
|
||||
if "reasoning_effort" in payload:
|
||||
subset["flat"] = payload["reasoning_effort"]
|
||||
reasoning = payload.get("reasoning")
|
||||
if isinstance(reasoning, dict) and "effort" in reasoning:
|
||||
subset["flat"] = reasoning["effort"]
|
||||
extra_body = payload.get("extra_body")
|
||||
ctk = extra_body.get("chat_template_kwargs") if isinstance(extra_body, dict) else None
|
||||
if isinstance(ctk, dict):
|
||||
known = {caps.thinking_param, effort_key} - {""}
|
||||
unexpected = set(ctk) - known
|
||||
assert not unexpected, f"unexpected chat_template_kwargs keys: {unexpected}"
|
||||
if caps.thinking_param in ctk:
|
||||
subset["toggle"] = ctk[caps.thinking_param]
|
||||
if effort_key and effort_key in ctk:
|
||||
subset["template_effort"] = ctk[effort_key]
|
||||
return subset
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# Ladder-token decoding — the token grammar, made executable
|
||||
# --------------------------------------------------------------------------- #
|
||||
|
||||
|
||||
def _decode_token(shape: Shape, token: str) -> dict[str, Any]:
|
||||
"""Expected effort wire subset for a ladder ``effective`` token."""
|
||||
caps = shape.caps
|
||||
if shape.provider == "anthropic":
|
||||
return _decode_native(caps, token)
|
||||
if shape.provider in ("openai", "xai") or shape.api_surface == "responses":
|
||||
return {} if token == "default" else {"flat": token}
|
||||
return _decode_template(shape.provider, caps, token)
|
||||
|
||||
|
||||
def _decode_native(caps: ModelCapabilities, token: str) -> dict[str, Any]:
|
||||
if caps.thinking_mode == "adaptive":
|
||||
# Thinking is unconditionally adaptive; a non-"adaptive" token is
|
||||
# the output_config effort level riding on top.
|
||||
expected: dict[str, Any] = {"thinking": {"type": "adaptive"}}
|
||||
if token != "adaptive":
|
||||
expected["output_effort"] = token
|
||||
return expected
|
||||
if token in ("default", "off"):
|
||||
return {}
|
||||
effort, sep, budget = token.partition("·budget:")
|
||||
if sep:
|
||||
return {
|
||||
"output_effort": effort,
|
||||
"thinking": {"type": "enabled", "budget_tokens": int(budget)},
|
||||
}
|
||||
if token.startswith("budget:"):
|
||||
budget_tokens = int(token.removeprefix("budget:"))
|
||||
return {"thinking": {"type": "enabled", "budget_tokens": budget_tokens}}
|
||||
return {"output_effort": token}
|
||||
|
||||
|
||||
def _decode_template(provider: str, caps: ModelCapabilities, token: str) -> dict[str, Any]:
|
||||
if token == "default":
|
||||
return {}
|
||||
parts = token.split("+")
|
||||
expected: dict[str, Any] = {}
|
||||
if parts[0] in ("on", "off"):
|
||||
expected["toggle"] = parts[0] == "on"
|
||||
parts = parts[1:]
|
||||
if parts:
|
||||
assert len(parts) == 1, f"unparseable ladder token: {token!r}"
|
||||
if caps.effort_param or provider == "anthropic-compatible":
|
||||
# Declared graded key, or the anthropic-compatible fallback
|
||||
# template key — that lane has no flat channel, so a graded
|
||||
# part there is always template-borne.
|
||||
expected["template_effort"] = parts[0]
|
||||
else:
|
||||
expected["flat"] = parts[0]
|
||||
return expected
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# The parity tests
|
||||
# --------------------------------------------------------------------------- #
|
||||
|
||||
|
||||
@pytest.mark.parametrize("shape", SHAPES, ids=lambda s: s.id)
|
||||
def test_ladder_tokens_match_wire(shape: Shape) -> None:
|
||||
"""Invariant 1: each token's decoded meaning equals the captured wire."""
|
||||
ladder = effort_ladder(shape.provider, shape.caps, shape.api_surface)
|
||||
assert [row["value"] for row in ladder] == list(KNOB_VALUES)
|
||||
for row in ladder:
|
||||
knob, token = row["value"], row["effective"]
|
||||
observed = _effort_wire_subset(_wire_payload(shape, knob), shape)
|
||||
expected = _decode_token(shape, token)
|
||||
assert observed == expected, (
|
||||
f"{shape.id}/knob={knob}: ladder says {token!r} which decodes to "
|
||||
f"{expected}, but the wire carries {observed}"
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("shape", SHAPES, ids=lambda s: s.id)
|
||||
def test_equal_tokens_iff_equal_wire(shape: Shape) -> None:
|
||||
"""Invariant 2: token equality ⇔ effort-wire equality, per shape."""
|
||||
tokens = {
|
||||
row["value"]: row["effective"]
|
||||
for row in effort_ladder(shape.provider, shape.caps, shape.api_surface)
|
||||
}
|
||||
subsets = {knob: _effort_wire_subset(_wire_payload(shape, knob), shape) for knob in KNOB_VALUES}
|
||||
for a, b in itertools.combinations(KNOB_VALUES, 2):
|
||||
same_token = tokens[a] == tokens[b]
|
||||
same_wire = subsets[a] == subsets[b]
|
||||
assert same_token == same_wire, (
|
||||
f"{shape.id}: knobs {a!r}/{b!r} have "
|
||||
f"{'equal' if same_token else 'distinct'} tokens "
|
||||
f"({tokens[a]!r} vs {tokens[b]!r}) but "
|
||||
f"{'identical' if same_wire else 'different'} wire subsets "
|
||||
f"({subsets[a]} vs {subsets[b]})"
|
||||
)
|
||||
@@ -409,3 +409,23 @@ def test_pane_handles_cross_user_409() -> None:
|
||||
assert "r.status === 409" in body
|
||||
assert 'status: "cross_user_interjection"' in body
|
||||
assert 'data.status === "cross_user_interjection"' in body
|
||||
|
||||
|
||||
def test_sync_approval_state_prunes_orphan_cycles() -> None:
|
||||
"""``_syncApprovalState`` prunes cycles whose block elements are no longer
|
||||
in the living DOM (``.isConnected === false``). This covers the rare case
|
||||
where an ``approve_request`` event is processed between a DOM wipe
|
||||
(``clear_ui`` / ``replay_truncated`` / ``replaceChildren``) and the
|
||||
refetch-restore — the cycle card lives in a detached subtree, the matching
|
||||
``approval_resolved`` never arrives, and the send button stays disabled
|
||||
forever without this guard. The pin guards against a future refactor that
|
||||
drops the orphan prune but doesn't otherwise break ``_syncApprovalState``."""
|
||||
body = _INTERACTIVE.read_text(encoding="utf-8")
|
||||
fn_start = body.index("_syncApprovalState() {")
|
||||
assert "entry.blockEls && !entry.blockEls.some((el) => el.isConnected)" in body, (
|
||||
"orphan pruning must check .isConnected on block elements"
|
||||
)
|
||||
tail = body[fn_start : body.index("_oldestCycleId()", fn_start)]
|
||||
assert "this.approvalCycles.delete(cid);" in tail, (
|
||||
"orphan pruning must delete the cycle from the Map"
|
||||
)
|
||||
|
||||
+1077
-21
File diff suppressed because it is too large
Load Diff
@@ -630,8 +630,6 @@ class TestCallback:
|
||||
server_name="srv-oauth",
|
||||
error_code="mcp_consent_required",
|
||||
scopes_required=None,
|
||||
last_ws_id="ws-1",
|
||||
last_tool_call_id="tool-1",
|
||||
now_iso="2026-05-11T12:00:00",
|
||||
)
|
||||
storage.upsert_mcp_pending_consent(
|
||||
@@ -639,8 +637,6 @@ class TestCallback:
|
||||
server_name="srv-oauth",
|
||||
error_code="mcp_consent_required",
|
||||
scopes_required=None,
|
||||
last_ws_id=None,
|
||||
last_tool_call_id=None,
|
||||
now_iso="2026-05-11T12:00:00",
|
||||
)
|
||||
token_store = _make_token_store(storage)
|
||||
|
||||
@@ -408,6 +408,97 @@ class TestRefreshFailureClassification:
|
||||
assert ("user-1", "srv-oauth") not in state.mcp_oauth_refresh_locks
|
||||
|
||||
|
||||
class TestObserveOnlyLookup:
|
||||
"""``revoke_on_failure=False`` (the background token-freshness sweep): still
|
||||
refresh a healthy token, but on failure NEVER delete a token or mutate the
|
||||
shared streak — a timer must not destroy consent or move a foreground user's
|
||||
revoke threshold. A permanent rejection surfaces as ``refresh_failed`` with
|
||||
the row INTACT; an ambiguous one as transient with the streak untouched."""
|
||||
|
||||
def _lookup(self, state: SimpleNamespace) -> Any:
|
||||
from turnstone.core.mcp_oauth import get_user_access_token_classified
|
||||
|
||||
async def _run() -> Any:
|
||||
with _public_addr_patch():
|
||||
return await get_user_access_token_classified(
|
||||
app_state=state,
|
||||
user_id="user-1",
|
||||
server_name="srv-oauth",
|
||||
force_refresh=True,
|
||||
revoke_on_failure=False,
|
||||
)
|
||||
|
||||
return asyncio.run(_run())
|
||||
|
||||
def test_permanent_invalid_grant_does_not_revoke(self, storage: SQLiteBackend) -> None:
|
||||
"""The exact contrast to ``test_permanent_invalid_grant_revokes``: same
|
||||
dead-grant signal, but observe-only leaves the row for the lazy path."""
|
||||
_seed_server(storage)
|
||||
client = MagicMock(spec=httpx.AsyncClient)
|
||||
client.get = AsyncMock(return_value=_mk_response(200, _good_as_metadata_doc()))
|
||||
client.post = AsyncMock(return_value=_mk_response(400, {"error": "invalid_grant"}))
|
||||
state = _make_app_state(storage, http_client=client)
|
||||
_seed_token(state, expires_in_seconds=-1000)
|
||||
|
||||
result = self._lookup(state)
|
||||
|
||||
assert result.kind == "refresh_failed"
|
||||
assert state.mcp_token_store.get_user_token("user-1", "srv-oauth") is not None
|
||||
|
||||
def test_ambiguous_does_not_touch_shared_streak(self, storage: SQLiteBackend) -> None:
|
||||
"""Repeated observe-mode ambiguous failures never bump the shared
|
||||
ambiguous_streak, so a later foreground dispatch is not pushed over the
|
||||
escalation edge by background activity (the finding this guards)."""
|
||||
_seed_server(storage)
|
||||
client = MagicMock(spec=httpx.AsyncClient)
|
||||
client.get = AsyncMock(return_value=_mk_response(200, _good_as_metadata_doc()))
|
||||
client.post = AsyncMock(return_value=_mk_response(400, None))
|
||||
state = _make_app_state(storage, http_client=client)
|
||||
_seed_token(state, expires_in_seconds=-1000)
|
||||
|
||||
with patch("turnstone.core.mcp_oauth._AMBIGUOUS_ESCALATION_THRESHOLD", 2):
|
||||
for _ in range(5):
|
||||
assert self._lookup(state).kind == "refresh_failed_transient"
|
||||
|
||||
backoff = getattr(state, "mcp_oauth_refresh_backoff", {})
|
||||
entry = backoff.get(("user-1", "srv-oauth"))
|
||||
assert entry is None or entry.ambiguous_streak == 0
|
||||
assert state.mcp_token_store.get_user_token("user-1", "srv-oauth") is not None
|
||||
|
||||
def test_expired_no_refresh_does_not_revoke(self, storage: SQLiteBackend) -> None:
|
||||
"""An expired token with no refresh token surfaces as a dead grant but is
|
||||
NOT deleted on the observe path."""
|
||||
_seed_server(storage)
|
||||
client = MagicMock(spec=httpx.AsyncClient)
|
||||
client.get = AsyncMock(return_value=_mk_response(200, _good_as_metadata_doc()))
|
||||
state = _make_app_state(storage, http_client=client)
|
||||
_seed_token(state, expires_in_seconds=-1000, refresh=None)
|
||||
|
||||
result = self._lookup(state)
|
||||
|
||||
assert result.kind == "refresh_failed"
|
||||
assert state.mcp_token_store.get_user_token("user-1", "srv-oauth") is not None
|
||||
|
||||
def test_healthy_token_still_refreshes(self, storage: SQLiteBackend) -> None:
|
||||
"""Observe mode is not read-only: a near-expiry token is still refreshed
|
||||
(only the destructive failure paths change)."""
|
||||
_seed_server(storage)
|
||||
client = MagicMock(spec=httpx.AsyncClient)
|
||||
client.get = AsyncMock(return_value=_mk_response(200, _good_as_metadata_doc()))
|
||||
client.post = AsyncMock(
|
||||
return_value=_mk_response(
|
||||
200, {"access_token": "fresh-bbb", "expires_in": 3600, "token_type": "Bearer"}
|
||||
)
|
||||
)
|
||||
state = _make_app_state(storage, http_client=client)
|
||||
_seed_token(state, expires_in_seconds=-1000)
|
||||
|
||||
result = self._lookup(state)
|
||||
|
||||
assert result.kind == "token"
|
||||
assert result.token == "fresh-bbb"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Happy paths
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
@@ -108,8 +108,6 @@ def _seed_pending(
|
||||
server_name=server_name,
|
||||
error_code=error_code,
|
||||
scopes_required=None,
|
||||
last_ws_id=None,
|
||||
last_tool_call_id=None,
|
||||
now_iso=now_iso,
|
||||
)
|
||||
|
||||
|
||||
@@ -24,8 +24,6 @@ class TestUpsertAndList:
|
||||
server_name="srv-x",
|
||||
error_code="mcp_consent_required",
|
||||
scopes_required="read write",
|
||||
last_ws_id="ws-1",
|
||||
last_tool_call_id="tool-1",
|
||||
now_iso=_iso(),
|
||||
)
|
||||
rows = backend.list_mcp_pending_consent_by_user("user-a")
|
||||
@@ -35,8 +33,6 @@ class TestUpsertAndList:
|
||||
assert r["server_name"] == "srv-x"
|
||||
assert r["error_code"] == "mcp_consent_required"
|
||||
assert r["scopes_required"] == "read write"
|
||||
assert r["last_ws_id"] == "ws-1"
|
||||
assert r["last_tool_call_id"] == "tool-1"
|
||||
assert r["occurrence_count"] == 1
|
||||
assert r["first_seen_at"] == r["last_seen_at"]
|
||||
|
||||
@@ -46,8 +42,6 @@ class TestUpsertAndList:
|
||||
server_name="srv-x",
|
||||
error_code="mcp_consent_required",
|
||||
scopes_required=None,
|
||||
last_ws_id=None,
|
||||
last_tool_call_id=None,
|
||||
now_iso="2026-05-11T12:00:00",
|
||||
)
|
||||
backend.upsert_mcp_pending_consent(
|
||||
@@ -55,8 +49,6 @@ class TestUpsertAndList:
|
||||
server_name="srv-x",
|
||||
error_code="mcp_insufficient_scope",
|
||||
scopes_required="read",
|
||||
last_ws_id="ws-2",
|
||||
last_tool_call_id="tool-2",
|
||||
now_iso="2026-05-11T13:00:00",
|
||||
)
|
||||
rows = backend.list_mcp_pending_consent_by_user("user-a")
|
||||
@@ -66,8 +58,6 @@ class TestUpsertAndList:
|
||||
assert r["occurrence_count"] == 2
|
||||
assert r["error_code"] == "mcp_insufficient_scope"
|
||||
assert r["scopes_required"] == "read"
|
||||
assert r["last_ws_id"] == "ws-2"
|
||||
assert r["last_tool_call_id"] == "tool-2"
|
||||
assert r["last_seen_at"] == "2026-05-11T13:00:00"
|
||||
# first_seen_at preserved — that's the load-bearing audit value.
|
||||
assert r["first_seen_at"] == "2026-05-11T12:00:00"
|
||||
@@ -78,8 +68,6 @@ class TestUpsertAndList:
|
||||
server_name="srv-old",
|
||||
error_code="mcp_consent_required",
|
||||
scopes_required=None,
|
||||
last_ws_id=None,
|
||||
last_tool_call_id=None,
|
||||
now_iso="2026-05-11T10:00:00",
|
||||
)
|
||||
backend.upsert_mcp_pending_consent(
|
||||
@@ -87,8 +75,6 @@ class TestUpsertAndList:
|
||||
server_name="srv-new",
|
||||
error_code="mcp_consent_required",
|
||||
scopes_required=None,
|
||||
last_ws_id=None,
|
||||
last_tool_call_id=None,
|
||||
now_iso="2026-05-11T11:00:00",
|
||||
)
|
||||
rows = backend.list_mcp_pending_consent_by_user("user-a")
|
||||
@@ -100,8 +86,6 @@ class TestUpsertAndList:
|
||||
server_name="srv",
|
||||
error_code="mcp_consent_required",
|
||||
scopes_required=None,
|
||||
last_ws_id=None,
|
||||
last_tool_call_id=None,
|
||||
now_iso=_iso(),
|
||||
)
|
||||
assert backend.list_mcp_pending_consent_by_user("user-b") == []
|
||||
@@ -114,8 +98,6 @@ class TestDelete:
|
||||
server_name="srv-x",
|
||||
error_code="mcp_consent_required",
|
||||
scopes_required=None,
|
||||
last_ws_id=None,
|
||||
last_tool_call_id=None,
|
||||
now_iso=_iso(),
|
||||
)
|
||||
assert backend.delete_mcp_pending_consent("user-a", "srv-x") is True
|
||||
@@ -133,8 +115,6 @@ class TestDelete:
|
||||
server_name=name,
|
||||
error_code="mcp_consent_required",
|
||||
scopes_required=None,
|
||||
last_ws_id=None,
|
||||
last_tool_call_id=None,
|
||||
now_iso=_iso(),
|
||||
)
|
||||
# Cross-user row that must NOT be touched.
|
||||
@@ -143,8 +123,6 @@ class TestDelete:
|
||||
server_name="srv-z",
|
||||
error_code="mcp_consent_required",
|
||||
scopes_required=None,
|
||||
last_ws_id=None,
|
||||
last_tool_call_id=None,
|
||||
now_iso=_iso(),
|
||||
)
|
||||
assert backend.delete_all_mcp_pending_consent_by_user("user-a") == 3
|
||||
|
||||
@@ -1033,7 +1033,9 @@ class TestStaticPathUnchanged:
|
||||
|
||||
from turnstone.core import mcp_client
|
||||
|
||||
source = inspect.getsource(mcp_client.MCPClientManager._connect_one)
|
||||
# The connect body (incl. the streamablehttp_client call site) lives in
|
||||
# ``_connect_one_locked``; ``_connect_one`` is now a per-name-lock wrapper.
|
||||
source = inspect.getsource(mcp_client.MCPClientManager._connect_one_locked)
|
||||
|
||||
# The static path's streamablehttp_client invocation should NOT
|
||||
# mention ``httpx_client_factory``. Pool path keeps it.
|
||||
|
||||
+406
-7
@@ -15,13 +15,14 @@ from __future__ import annotations
|
||||
import asyncio
|
||||
import contextlib
|
||||
import json
|
||||
import logging
|
||||
import threading
|
||||
import time
|
||||
from contextlib import AsyncExitStack
|
||||
from datetime import UTC, datetime, timedelta
|
||||
from types import SimpleNamespace
|
||||
from typing import Any
|
||||
from unittest.mock import AsyncMock, MagicMock
|
||||
from unittest.mock import AsyncMock, MagicMock, patch
|
||||
|
||||
import pytest
|
||||
|
||||
@@ -114,12 +115,13 @@ def running_loop_mgr():
|
||||
# handlers don't fire after pytest has torn its handlers down. Mirrors
|
||||
# the production ``shutdown()`` shape.
|
||||
async def _drain(m: MCPClientManager) -> None:
|
||||
task = m._user_pool_eviction_task
|
||||
if task is not None:
|
||||
task.cancel()
|
||||
with contextlib.suppress(BaseException):
|
||||
await task
|
||||
m._user_pool_eviction_task = None
|
||||
for attr in ("_user_pool_eviction_task", "_user_token_sweep_task"):
|
||||
task = getattr(m, attr)
|
||||
if task is not None:
|
||||
task.cancel()
|
||||
with contextlib.suppress(BaseException):
|
||||
await task
|
||||
setattr(m, attr, None)
|
||||
|
||||
with contextlib.suppress(Exception):
|
||||
asyncio.run_coroutine_threadsafe(_drain(mgr), loop).result(timeout=2)
|
||||
@@ -969,3 +971,400 @@ class TestUserIdThreadThrough:
|
||||
assert result == "static-output"
|
||||
# No pool entries were created.
|
||||
assert mgr._user_pool_entries == {}
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Background token-freshness sweep (oauth_user keep-hot, no connection warming)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestUserTokenFreshnessSweep:
|
||||
"""The background sweep that keeps every consented ``oauth_user`` grant hot
|
||||
for unattended / autonomous work: refresh-on-expiry via the canonical path,
|
||||
proactive dead-grant badging, once-only surfacing, and — the load-bearing
|
||||
property — total invisibility to static / no-auth deployments."""
|
||||
|
||||
def _wire(self, mgr: MCPClientManager, storage: SQLiteBackend, cipher: Any) -> None:
|
||||
mgr.set_storage(storage)
|
||||
mgr.set_app_state(_make_app_state(storage, cipher=cipher))
|
||||
mgr._oauth_user_server_names = {"pool-srv"}
|
||||
|
||||
@staticmethod
|
||||
def _classified(kind: str, token: str | None = None):
|
||||
async def _fake(**kwargs: Any) -> Any:
|
||||
return SimpleNamespace(kind=kind, token=token)
|
||||
|
||||
return _fake
|
||||
|
||||
# -- no-auth / static safety: the sweep must be structurally invisible ----
|
||||
|
||||
def test_sweep_noop_without_oauth_servers(self, running_loop_mgr, storage) -> None:
|
||||
"""A static-only / no-auth deployment: the OBO gate returns before any
|
||||
DB scan or AS round-trip — the single most important property."""
|
||||
mgr, loop, _ = running_loop_mgr
|
||||
cipher = make_mcp_token_cipher()
|
||||
self._wire(mgr, storage, cipher)
|
||||
mgr._oauth_user_server_names = set() # no oauth_user server configured
|
||||
storage.list_mcp_user_token_reconcile_targets = MagicMock(return_value=[]) # type: ignore[method-assign]
|
||||
|
||||
with patch(
|
||||
"turnstone.core.mcp_client.get_user_access_token_classified",
|
||||
new=AsyncMock(),
|
||||
) as classified:
|
||||
_run_on_loop(loop, mgr._sweep_user_token_freshness())
|
||||
storage.list_mcp_user_token_reconcile_targets.assert_not_called() # no token-table scan
|
||||
classified.assert_not_awaited() # no AS round-trip
|
||||
|
||||
def test_sweep_noop_before_storage_wired(self, running_loop_mgr) -> None:
|
||||
mgr, loop, _ = running_loop_mgr
|
||||
mgr._oauth_user_server_names = {"pool-srv"} # oauth configured but app not wired yet
|
||||
with patch(
|
||||
"turnstone.core.mcp_client.get_user_access_token_classified",
|
||||
new=AsyncMock(),
|
||||
) as classified:
|
||||
_run_on_loop(loop, mgr._sweep_user_token_freshness())
|
||||
classified.assert_not_awaited()
|
||||
|
||||
def test_sweep_skips_server_not_in_oauth_set(self, running_loop_mgr, storage) -> None:
|
||||
"""A token row lingering for a since-demoted / renamed server is not
|
||||
reconciled — only pairs whose server is currently ``oauth_user``."""
|
||||
mgr, loop, _ = running_loop_mgr
|
||||
cipher = make_mcp_token_cipher()
|
||||
self._wire(mgr, storage, cipher)
|
||||
_seed_user_token(storage, cipher, user_id="u1", server_name="ghost-srv")
|
||||
|
||||
with patch(
|
||||
"turnstone.core.mcp_client.get_user_access_token_classified",
|
||||
new=AsyncMock(),
|
||||
) as classified:
|
||||
_run_on_loop(loop, mgr._sweep_user_token_freshness())
|
||||
classified.assert_not_awaited() # ghost-srv is not in _oauth_user_server_names
|
||||
|
||||
# -- classification branches --------------------------------------------
|
||||
|
||||
def test_healthy_token_no_badge(self, running_loop_mgr, storage) -> None:
|
||||
mgr, loop, _ = running_loop_mgr
|
||||
cipher = make_mcp_token_cipher()
|
||||
self._wire(mgr, storage, cipher)
|
||||
_seed_user_token(storage, cipher, user_id="u1", server_name="pool-srv")
|
||||
storage.upsert_mcp_pending_consent = MagicMock() # type: ignore[method-assign]
|
||||
|
||||
with patch(
|
||||
"turnstone.core.mcp_client.get_user_access_token_classified",
|
||||
new=self._classified("token", token="access-aaa"),
|
||||
):
|
||||
_run_on_loop(loop, mgr._sweep_user_token_freshness())
|
||||
storage.upsert_mcp_pending_consent.assert_not_called()
|
||||
assert ("u1", "pool-srv") not in mgr._token_sweep_warned
|
||||
|
||||
def test_dead_grant_badges_once_and_dedups(self, running_loop_mgr, storage, caplog) -> None:
|
||||
mgr, loop, _ = running_loop_mgr
|
||||
cipher = make_mcp_token_cipher()
|
||||
self._wire(mgr, storage, cipher)
|
||||
_seed_user_token(storage, cipher, user_id="u1", server_name="pool-srv")
|
||||
storage.upsert_mcp_pending_consent = MagicMock() # type: ignore[method-assign]
|
||||
|
||||
with (
|
||||
patch(
|
||||
"turnstone.core.mcp_client.get_user_access_token_classified",
|
||||
new=self._classified("refresh_failed"),
|
||||
),
|
||||
caplog.at_level(logging.WARNING, logger="turnstone.core.mcp_client"),
|
||||
):
|
||||
_run_on_loop(loop, mgr._sweep_user_token_freshness())
|
||||
_run_on_loop(loop, mgr._sweep_user_token_freshness()) # second tick: no re-badge
|
||||
|
||||
# Badge raised exactly once, proactively, with the dashboard's code.
|
||||
storage.upsert_mcp_pending_consent.assert_called_once()
|
||||
assert (
|
||||
storage.upsert_mcp_pending_consent.call_args.kwargs["error_code"]
|
||||
== "mcp_consent_required"
|
||||
)
|
||||
assert ("u1", "pool-srv") in mgr._token_sweep_warned
|
||||
escalations = [r for r in caplog.records if "needs re-consent" in r.getMessage()]
|
||||
assert len(escalations) == 1 # logged loud-once, not every tick
|
||||
|
||||
def test_decrypt_failure_warns_but_does_not_badge(self, running_loop_mgr, storage) -> None:
|
||||
mgr, loop, _ = running_loop_mgr
|
||||
cipher = make_mcp_token_cipher()
|
||||
self._wire(mgr, storage, cipher)
|
||||
_seed_user_token(storage, cipher, user_id="u1", server_name="pool-srv")
|
||||
storage.upsert_mcp_pending_consent = MagicMock() # type: ignore[method-assign]
|
||||
|
||||
with patch(
|
||||
"turnstone.core.mcp_client.get_user_access_token_classified",
|
||||
new=self._classified("decrypt_failure"),
|
||||
):
|
||||
_run_on_loop(loop, mgr._sweep_user_token_freshness())
|
||||
# Operator-actionable (key unknown) — surfaced in the warned set, but NOT
|
||||
# a user-consent badge (outside the dashboard's scope).
|
||||
storage.upsert_mcp_pending_consent.assert_not_called()
|
||||
assert ("u1", "pool-srv") in mgr._token_sweep_warned
|
||||
|
||||
def test_transient_failure_is_silent(self, running_loop_mgr, storage) -> None:
|
||||
mgr, loop, _ = running_loop_mgr
|
||||
cipher = make_mcp_token_cipher()
|
||||
self._wire(mgr, storage, cipher)
|
||||
_seed_user_token(storage, cipher, user_id="u1", server_name="pool-srv")
|
||||
storage.upsert_mcp_pending_consent = MagicMock() # type: ignore[method-assign]
|
||||
|
||||
with patch(
|
||||
"turnstone.core.mcp_client.get_user_access_token_classified",
|
||||
new=self._classified("refresh_failed_transient"),
|
||||
):
|
||||
_run_on_loop(loop, mgr._sweep_user_token_freshness())
|
||||
storage.upsert_mcp_pending_consent.assert_not_called()
|
||||
assert ("u1", "pool-srv") not in mgr._token_sweep_warned # retryable, not surfaced
|
||||
|
||||
def test_recovery_rearms_and_clears_badge(self, running_loop_mgr, storage) -> None:
|
||||
"""A dead grant that later returns healthy clears its warned pin AND drops
|
||||
the stale badge — the self-heal for a spurious invalid_grant that has
|
||||
since recovered. Production-reachable now that the observe-only sweep no
|
||||
longer deletes the row on refresh_failed, so the pair keeps enumerating."""
|
||||
mgr, loop, _ = running_loop_mgr
|
||||
cipher = make_mcp_token_cipher()
|
||||
self._wire(mgr, storage, cipher)
|
||||
_seed_user_token(storage, cipher, user_id="u1", server_name="pool-srv")
|
||||
storage.delete_mcp_pending_consent = MagicMock(return_value=True) # type: ignore[method-assign]
|
||||
key = ("u1", "pool-srv")
|
||||
|
||||
with patch(
|
||||
"turnstone.core.mcp_client.get_user_access_token_classified",
|
||||
new=self._classified("refresh_failed"),
|
||||
):
|
||||
_run_on_loop(loop, mgr._sweep_user_token_freshness())
|
||||
assert key in mgr._token_sweep_warned
|
||||
with patch(
|
||||
"turnstone.core.mcp_client.get_user_access_token_classified",
|
||||
new=self._classified("token", token="access-aaa"),
|
||||
):
|
||||
_run_on_loop(loop, mgr._sweep_user_token_freshness())
|
||||
assert key not in mgr._token_sweep_warned # recovered → re-armed
|
||||
storage.delete_mcp_pending_consent.assert_called_once_with("u1", "pool-srv")
|
||||
|
||||
def test_dead_grant_not_pinned_when_badge_persist_fails(
|
||||
self, running_loop_mgr, storage
|
||||
) -> None:
|
||||
"""If the badge write fails, the pair is NOT pinned, so the next tick
|
||||
retries — a single failed persist must not permanently lose the only
|
||||
proactive signal for a sweep-detected dead grant."""
|
||||
mgr, loop, _ = running_loop_mgr
|
||||
cipher = make_mcp_token_cipher()
|
||||
self._wire(mgr, storage, cipher)
|
||||
_seed_user_token(storage, cipher, user_id="u1", server_name="pool-srv")
|
||||
storage.upsert_mcp_pending_consent = MagicMock( # type: ignore[method-assign]
|
||||
side_effect=RuntimeError("db down")
|
||||
)
|
||||
key = ("u1", "pool-srv")
|
||||
|
||||
with patch(
|
||||
"turnstone.core.mcp_client.get_user_access_token_classified",
|
||||
new=self._classified("refresh_failed"),
|
||||
):
|
||||
_run_on_loop(loop, mgr._sweep_user_token_freshness())
|
||||
assert key not in mgr._token_sweep_warned # not pinned — will retry
|
||||
_run_on_loop(loop, mgr._sweep_user_token_freshness())
|
||||
# Retried on the second tick rather than deduped away by a phantom pin.
|
||||
assert storage.upsert_mcp_pending_consent.call_count == 2
|
||||
|
||||
def test_sweep_uses_non_revoking_observe_mode(self, running_loop_mgr, storage) -> None:
|
||||
"""The background sweep MUST call the canonical lookup non-destructively:
|
||||
a timer may never delete a token or move a foreground user's revoke
|
||||
threshold."""
|
||||
mgr, loop, _ = running_loop_mgr
|
||||
cipher = make_mcp_token_cipher()
|
||||
self._wire(mgr, storage, cipher)
|
||||
_seed_user_token(storage, cipher, user_id="u1", server_name="pool-srv")
|
||||
seen_kwargs: list[dict[str, Any]] = []
|
||||
|
||||
async def _spy(**kwargs: Any) -> Any:
|
||||
seen_kwargs.append(kwargs)
|
||||
return SimpleNamespace(kind="token", token="access-aaa")
|
||||
|
||||
with patch("turnstone.core.mcp_client.get_user_access_token_classified", new=_spy):
|
||||
_run_on_loop(loop, mgr._sweep_user_token_freshness())
|
||||
assert seen_kwargs and seen_kwargs[0]["revoke_on_failure"] is False
|
||||
assert seen_kwargs[0]["revoke_ambiguous_escalation"] is False
|
||||
|
||||
# -- keepalive refresh (exercise the refresh token before it idles out) ---
|
||||
|
||||
def test_keepalive_refresh_due_logic(self) -> None:
|
||||
mgr = MCPClientManager({})
|
||||
mgr._user_token_refresh_keepalive_s = 3600.0
|
||||
old = (datetime.now(UTC) - timedelta(hours=2)).strftime("%Y-%m-%dT%H:%M:%S")
|
||||
recent = (datetime.now(UTC) - timedelta(minutes=1)).strftime("%Y-%m-%dT%H:%M:%S")
|
||||
assert mgr._keepalive_refresh_due(old) is True # past the window → force
|
||||
assert mgr._keepalive_refresh_due(recent) is False # still warm
|
||||
assert mgr._keepalive_refresh_due(None) is True # unknown → force once, safe
|
||||
assert mgr._keepalive_refresh_due("not-a-date") is True # unparseable → force
|
||||
mgr._user_token_refresh_keepalive_s = 0.0
|
||||
assert mgr._keepalive_refresh_due(old) is False # disabled → never force
|
||||
|
||||
def test_keepalive_due_forces_refresh(self, running_loop_mgr, storage) -> None:
|
||||
"""A grant whose refresh token has idled past the window is force-refreshed
|
||||
even though its access token may be fresh — the [6] fix: keep the refresh
|
||||
token alive so an unattended run never finds it aged out."""
|
||||
mgr, loop, _ = running_loop_mgr
|
||||
cipher = make_mcp_token_cipher()
|
||||
self._wire(mgr, storage, cipher)
|
||||
mgr._user_token_refresh_keepalive_s = 1800.0
|
||||
stale = (datetime.now(UTC) - timedelta(hours=2)).strftime("%Y-%m-%dT%H:%M:%S")
|
||||
storage.list_mcp_user_token_reconcile_targets = MagicMock( # type: ignore[method-assign]
|
||||
return_value=[("u1", "pool-srv", stale)]
|
||||
)
|
||||
seen_kwargs: list[dict[str, Any]] = []
|
||||
|
||||
async def _spy(**kwargs: Any) -> Any:
|
||||
seen_kwargs.append(kwargs)
|
||||
return SimpleNamespace(kind="token", token="access-aaa")
|
||||
|
||||
with patch("turnstone.core.mcp_client.get_user_access_token_classified", new=_spy):
|
||||
_run_on_loop(loop, mgr._sweep_user_token_freshness())
|
||||
assert seen_kwargs and seen_kwargs[0]["force_refresh"] is True
|
||||
|
||||
def test_keepalive_not_due_does_not_force(self, running_loop_mgr, storage) -> None:
|
||||
mgr, loop, _ = running_loop_mgr
|
||||
cipher = make_mcp_token_cipher()
|
||||
self._wire(mgr, storage, cipher)
|
||||
mgr._user_token_refresh_keepalive_s = 1800.0
|
||||
recent = (datetime.now(UTC) - timedelta(minutes=1)).strftime("%Y-%m-%dT%H:%M:%S")
|
||||
storage.list_mcp_user_token_reconcile_targets = MagicMock( # type: ignore[method-assign]
|
||||
return_value=[("u1", "pool-srv", recent)]
|
||||
)
|
||||
seen_kwargs: list[dict[str, Any]] = []
|
||||
|
||||
async def _spy(**kwargs: Any) -> Any:
|
||||
seen_kwargs.append(kwargs)
|
||||
return SimpleNamespace(kind="token", token="access-aaa")
|
||||
|
||||
with patch("turnstone.core.mcp_client.get_user_access_token_classified", new=_spy):
|
||||
_run_on_loop(loop, mgr._sweep_user_token_freshness())
|
||||
assert seen_kwargs and seen_kwargs[0]["force_refresh"] is False # still warm
|
||||
|
||||
def test_warned_set_pruned_to_consented_pairs(self, running_loop_mgr, storage) -> None:
|
||||
"""A warned pair that is no longer consented (row gone) is dropped from
|
||||
the dedup set so it can't grow unbounded across transient dead grants."""
|
||||
mgr, loop, _ = running_loop_mgr
|
||||
cipher = make_mcp_token_cipher()
|
||||
self._wire(mgr, storage, cipher)
|
||||
_seed_user_token(storage, cipher, user_id="u1", server_name="pool-srv")
|
||||
mgr._token_sweep_warned = {("gone-user", "pool-srv"), ("u1", "pool-srv")}
|
||||
|
||||
with patch(
|
||||
"turnstone.core.mcp_client.get_user_access_token_classified",
|
||||
new=self._classified("token", token="access-aaa"),
|
||||
):
|
||||
_run_on_loop(loop, mgr._sweep_user_token_freshness())
|
||||
assert ("gone-user", "pool-srv") not in mgr._token_sweep_warned # pruned
|
||||
assert ("u1", "pool-srv") not in mgr._token_sweep_warned # healthy → cleared
|
||||
|
||||
def test_per_pair_failure_isolated(self, running_loop_mgr, storage) -> None:
|
||||
"""One pair raising must not starve the rest of the pass."""
|
||||
mgr, loop, _ = running_loop_mgr
|
||||
cipher = make_mcp_token_cipher()
|
||||
self._wire(mgr, storage, cipher)
|
||||
mgr._oauth_user_server_names = {"pool-srv"}
|
||||
_seed_user_token(storage, cipher, user_id="u-bad", server_name="pool-srv")
|
||||
_seed_user_token(storage, cipher, user_id="u-ok", server_name="pool-srv")
|
||||
seen: list[str] = []
|
||||
|
||||
async def _flaky(**kwargs: Any) -> Any:
|
||||
uid = kwargs["user_id"]
|
||||
seen.append(uid)
|
||||
if uid == "u-bad":
|
||||
raise RuntimeError("boom")
|
||||
return SimpleNamespace(kind="token", token="access-aaa")
|
||||
|
||||
with patch("turnstone.core.mcp_client.get_user_access_token_classified", new=_flaky):
|
||||
_run_on_loop(loop, mgr._sweep_user_token_freshness())
|
||||
assert {"u-bad", "u-ok"} <= set(seen) # both attempted despite one raising
|
||||
|
||||
def test_sweep_loop_cancel_returns_cleanly(self, running_loop_mgr) -> None:
|
||||
"""The loop body exits on cancellation without raising (mirrors the
|
||||
eviction loop's teardown contract)."""
|
||||
mgr, loop, _ = running_loop_mgr
|
||||
mgr._user_token_sweep_s = 999.0 # park in the sleep
|
||||
|
||||
async def _spawn() -> asyncio.Task[None]:
|
||||
return asyncio.ensure_future(mgr._user_token_sweep_loop())
|
||||
|
||||
task = _run_on_loop(loop, _spawn())
|
||||
|
||||
async def _cancel() -> None:
|
||||
task.cancel()
|
||||
with contextlib.suppress(BaseException):
|
||||
await task
|
||||
|
||||
_run_on_loop(loop, _cancel())
|
||||
assert task.cancelled() or task.done()
|
||||
|
||||
def test_connect_all_starts_the_sweep_task(self, running_loop_mgr) -> None:
|
||||
"""Wiring guard: ``_connect_all`` must start the sweep once, even with no
|
||||
servers configured — otherwise the whole keep-hot mechanism is dead code."""
|
||||
mgr, loop, _ = running_loop_mgr
|
||||
assert mgr._user_token_sweep_task is None
|
||||
|
||||
_run_on_loop(loop, mgr._connect_all())
|
||||
try:
|
||||
task = mgr._user_token_sweep_task
|
||||
assert task is not None and not task.done() # live, single instance
|
||||
finally:
|
||||
|
||||
async def _drain() -> None:
|
||||
t = mgr._user_token_sweep_task
|
||||
if t is not None:
|
||||
t.cancel()
|
||||
with contextlib.suppress(BaseException):
|
||||
await t
|
||||
mgr._user_token_sweep_task = None
|
||||
|
||||
_run_on_loop(loop, _drain())
|
||||
|
||||
def test_disabled_sweep_not_started_by_connect_all(self, running_loop_mgr) -> None:
|
||||
"""Cadence <= 0 disables the sweep entirely — no task is spawned."""
|
||||
mgr, loop, _ = running_loop_mgr
|
||||
mgr._user_token_sweep_s = 0.0
|
||||
_run_on_loop(loop, mgr._connect_all())
|
||||
assert mgr._user_token_sweep_task is None
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("configured", "expected"),
|
||||
[
|
||||
(0, 0.0), # explicit disable
|
||||
(-5, 0.0), # negative disables (no busy-loop)
|
||||
(1, 30.0), # tiny positive floored to _MIN_USER_TOKEN_SWEEP_S
|
||||
(600, 600.0), # normal value passes through
|
||||
],
|
||||
)
|
||||
def test_cadence_clamped_or_disabled(self, configured, expected) -> None:
|
||||
"""The config cadence is floored (positive) or disabled (<= 0) so an
|
||||
``asyncio.sleep(0)`` busy-loop is unreachable."""
|
||||
with patch(
|
||||
"turnstone.core.mcp_client.load_config",
|
||||
return_value={"user_token_sweep_seconds": configured},
|
||||
):
|
||||
mgr = MCPClientManager({})
|
||||
assert mgr._user_token_sweep_s == expected
|
||||
|
||||
# -- storage enumerator --------------------------------------------------
|
||||
|
||||
def test_reconcile_targets_pairs_expiry_unfiltered_with_last_exercised(self, storage) -> None:
|
||||
cipher = make_mcp_token_cipher()
|
||||
# alice consents to two servers → two rows.
|
||||
_seed_user_token(storage, cipher, user_id="alice", server_name="srv-a")
|
||||
_seed_user_token(storage, cipher, user_id="alice", server_name="srv-b")
|
||||
# bob's access token is expired but the refresh token is live — still a
|
||||
# consented, reconcilable grant, so bob must be enumerated.
|
||||
_seed_user_token(
|
||||
storage, cipher, user_id="bob", server_name="srv-a", expires_in_seconds=-999
|
||||
)
|
||||
targets = storage.list_mcp_user_token_reconcile_targets()
|
||||
# (user, server) identity, all three grants present regardless of expiry.
|
||||
assert sorted((u, s) for u, s, _ in targets) == [
|
||||
("alice", "srv-a"),
|
||||
("alice", "srv-b"),
|
||||
("bob", "srv-a"),
|
||||
]
|
||||
# last_exercised = COALESCE(last_refreshed, created); never-refreshed rows
|
||||
# fall back to created, so it is always populated (drives the keepalive).
|
||||
assert all(last_exercised for _, _, last_exercised in targets)
|
||||
|
||||
@@ -0,0 +1,104 @@
|
||||
"""Tests for alembic migration 065 (capture Entra oid/tid on oidc_identities).
|
||||
|
||||
Drives ``command.upgrade``/``downgrade`` against an isolated SQLite database per
|
||||
test (the 060/062/063 harness pattern), then asserts:
|
||||
|
||||
* upgrade adds the ``oid``/``tid`` columns and the ``idx_oidc_identities_oid``
|
||||
index;
|
||||
* a pre-065 row migrates cleanly, gaining ``""`` for the new columns;
|
||||
* downgrade removes the columns + index, returning ``oidc_identities`` to its
|
||||
exact pre-065 shape — this pins the **clean-rollback** guarantee (the change
|
||||
can be backed out with no orphaned state if the upstream PR is rejected).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
import sqlalchemy as sa
|
||||
from alembic import command
|
||||
from alembic.config import Config
|
||||
|
||||
_MIGRATIONS_DIR = str(
|
||||
Path(__file__).resolve().parent.parent / "turnstone" / "core" / "storage" / "migrations"
|
||||
)
|
||||
|
||||
|
||||
def _alembic_cfg(db_path: Path) -> Config:
|
||||
cfg = Config()
|
||||
cfg.set_main_option("script_location", _MIGRATIONS_DIR)
|
||||
cfg.set_main_option("sqlalchemy.url", f"sqlite:///{db_path}")
|
||||
return cfg
|
||||
|
||||
|
||||
class TestMigration065:
|
||||
def test_upgrade_adds_oid_tid_and_index(self, tmp_path: Path) -> None:
|
||||
db_path = tmp_path / "065-up.db"
|
||||
command.upgrade(_alembic_cfg(db_path), "065")
|
||||
|
||||
engine = sa.create_engine(f"sqlite:///{db_path}")
|
||||
try:
|
||||
insp = sa.inspect(engine)
|
||||
cols = {c["name"] for c in insp.get_columns("oidc_identities")}
|
||||
assert {"oid", "tid"} <= cols
|
||||
idx = {i["name"] for i in insp.get_indexes("oidc_identities")}
|
||||
assert "idx_oidc_identities_oid" in idx
|
||||
finally:
|
||||
engine.dispose()
|
||||
|
||||
def test_preexisting_row_migrates_with_empty_default(self, tmp_path: Path) -> None:
|
||||
db_path = tmp_path / "065-default.db"
|
||||
cfg = _alembic_cfg(db_path)
|
||||
# Stop at 064, insert a pre-065 identity, THEN upgrade to 065.
|
||||
command.upgrade(cfg, "064")
|
||||
engine = sa.create_engine(f"sqlite:///{db_path}")
|
||||
try:
|
||||
with engine.begin() as conn:
|
||||
conn.execute(
|
||||
sa.text(
|
||||
"INSERT INTO oidc_identities "
|
||||
"(issuer, subject, user_id, email, created, last_login) "
|
||||
"VALUES ('iss', 'sub', 'u1', '', "
|
||||
"'2026-01-01T00:00:00', '2026-01-01T00:00:00')"
|
||||
)
|
||||
)
|
||||
command.upgrade(cfg, "065")
|
||||
with engine.connect() as conn:
|
||||
row = conn.execute(
|
||||
sa.text("SELECT oid, tid FROM oidc_identities WHERE subject = 'sub'")
|
||||
).fetchone()
|
||||
assert row is not None
|
||||
assert row[0] == "" and row[1] == ""
|
||||
finally:
|
||||
engine.dispose()
|
||||
|
||||
def test_downgrade_removes_oid_tid_and_index(self, tmp_path: Path) -> None:
|
||||
db_path = tmp_path / "065-down.db"
|
||||
cfg = _alembic_cfg(db_path)
|
||||
command.upgrade(cfg, "065")
|
||||
command.downgrade(cfg, "064")
|
||||
|
||||
engine = sa.create_engine(f"sqlite:///{db_path}")
|
||||
try:
|
||||
insp = sa.inspect(engine)
|
||||
cols = {c["name"] for c in insp.get_columns("oidc_identities")}
|
||||
assert "oid" not in cols and "tid" not in cols
|
||||
idx = {i["name"] for i in insp.get_indexes("oidc_identities")}
|
||||
assert "idx_oidc_identities_oid" not in idx
|
||||
finally:
|
||||
engine.dispose()
|
||||
|
||||
def test_downgrade_then_upgrade_round_trip(self, tmp_path: Path) -> None:
|
||||
"""up -> down -> up must land cleanly (no leftover column/index conflict)."""
|
||||
db_path = tmp_path / "065-roundtrip.db"
|
||||
cfg = _alembic_cfg(db_path)
|
||||
command.upgrade(cfg, "065")
|
||||
command.downgrade(cfg, "064")
|
||||
command.upgrade(cfg, "065")
|
||||
|
||||
engine = sa.create_engine(f"sqlite:///{db_path}")
|
||||
try:
|
||||
cols = {c["name"] for c in sa.inspect(engine).get_columns("oidc_identities")}
|
||||
assert {"oid", "tid"} <= cols
|
||||
finally:
|
||||
engine.dispose()
|
||||
@@ -1643,6 +1643,65 @@ class TestProvisionOIDCUser:
|
||||
|
||||
storage.assign_role.assert_not_called()
|
||||
|
||||
def test_provision_oidc_user_null_oid_tid_collapse_to_empty(self):
|
||||
"""A present-but-null oid/tid claim must store "" — never the string "None".
|
||||
|
||||
`claims.get("oid", "")` returns None (not the "" default) when the key is
|
||||
present with a JSON null, and str(None) == "None" would slip past both the
|
||||
server_default and the truthy backfill guard, storing a bogus non-empty
|
||||
sentinel that collides across every null-emitting user. New-user path.
|
||||
"""
|
||||
config = _make_config()
|
||||
storage = _mock_storage()
|
||||
storage.get_user.return_value = {
|
||||
"user_id": "u-new",
|
||||
"username": "bob",
|
||||
"display_name": "Bob",
|
||||
"password_hash": "!oidc",
|
||||
}
|
||||
|
||||
claims = {"sub": "sub-null", "preferred_username": "bob", "oid": None, "tid": None}
|
||||
with patch("turnstone.core.oidc.uuid") as mock_uuid:
|
||||
mock_uuid.uuid4.return_value = MagicMock(hex="u-new-hex-00000000000000000000")
|
||||
provision_oidc_user(storage, config, claims)
|
||||
|
||||
kwargs = storage.create_oidc_user.call_args.kwargs
|
||||
assert kwargs["oid"] == ""
|
||||
assert kwargs["tid"] == ""
|
||||
|
||||
def test_provision_oidc_user_null_oid_tid_not_backfilled_existing(self):
|
||||
"""Existing-identity path: null oid/tid claims must not backfill "None".
|
||||
|
||||
The truthy guard in update_oidc_identity_login only protects against ""; a
|
||||
"None" produced by str(None) is truthy and would be written, clobbering a
|
||||
real value captured on an earlier login.
|
||||
"""
|
||||
config = _make_config()
|
||||
existing_user = {
|
||||
"user_id": "u1",
|
||||
"username": "alice",
|
||||
"display_name": "Alice",
|
||||
"password_hash": "!oidc",
|
||||
}
|
||||
existing_identity = {
|
||||
"issuer": "https://idp.example.com",
|
||||
"subject": "sub-123",
|
||||
"user_id": "u1",
|
||||
"email": "alice@example.com",
|
||||
"created": "2024-01-01T00:00:00",
|
||||
"last_login": "2024-01-01T00:00:00",
|
||||
"oid": "obj-real",
|
||||
"tid": "ten-real",
|
||||
}
|
||||
storage = _mock_storage(identity=existing_identity, user=existing_user)
|
||||
|
||||
claims = {"sub": "sub-123", "email": "alice@example.com", "oid": None, "tid": None}
|
||||
provision_oidc_user(storage, config, claims)
|
||||
|
||||
kwargs = storage.update_oidc_identity_login.call_args.kwargs
|
||||
assert kwargs["oid"] == ""
|
||||
assert kwargs["tid"] == ""
|
||||
|
||||
def test_existing_identity_self_heals_zero_roles(self):
|
||||
"""Existing identity user with zero roles -> safety-net assigns builtin-viewer.
|
||||
|
||||
|
||||
@@ -84,6 +84,40 @@ class TestCreateOIDCUser:
|
||||
assert identity is not None
|
||||
assert identity["user_id"] == "u-other"
|
||||
|
||||
def test_create_oidc_user_captures_oid_tid(self, db):
|
||||
"""Entra oid/tid are persisted and returned on the identity."""
|
||||
db.create_oidc_user(
|
||||
user_id="u-oid",
|
||||
username="carol",
|
||||
display_name="Carol",
|
||||
password_hash="!oidc",
|
||||
issuer="https://idp.example.com",
|
||||
subject="sub-oid",
|
||||
email="carol@example.com",
|
||||
oid="obj-123",
|
||||
tid="tenant-abc",
|
||||
)
|
||||
identity = db.get_oidc_identity("https://idp.example.com", "sub-oid")
|
||||
assert identity is not None
|
||||
assert identity["oid"] == "obj-123"
|
||||
assert identity["tid"] == "tenant-abc"
|
||||
|
||||
def test_create_oidc_user_oid_tid_default_empty(self, db):
|
||||
"""Omitting oid/tid (non-Entra IdP) stores "" — never NULL."""
|
||||
db.create_oidc_user(
|
||||
user_id="u-noid",
|
||||
username="dave",
|
||||
display_name="Dave",
|
||||
password_hash="!oidc",
|
||||
issuer="https://idp.example.com",
|
||||
subject="sub-noid",
|
||||
email="dave@example.com",
|
||||
)
|
||||
identity = db.get_oidc_identity("https://idp.example.com", "sub-noid")
|
||||
assert identity is not None
|
||||
assert identity["oid"] == ""
|
||||
assert identity["tid"] == ""
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# OIDC Identity CRUD
|
||||
@@ -137,6 +171,33 @@ class TestOIDCIdentityCRUD:
|
||||
result = db.update_oidc_identity_login("https://idp.example.com", "sub-999")
|
||||
assert result is False
|
||||
|
||||
def test_update_oidc_identity_login_backfills_oid_tid(self, db):
|
||||
"""A login carrying oid/tid backfills them onto a pre-existing row."""
|
||||
db.create_oidc_identity("https://idp.example.com", "sub-bf", "u1", "a@example.com")
|
||||
before = db.get_oidc_identity("https://idp.example.com", "sub-bf")
|
||||
assert before is not None and before["oid"] == ""
|
||||
|
||||
db.update_oidc_identity_login("https://idp.example.com", "sub-bf", oid="obj-9", tid="ten-9")
|
||||
|
||||
after = db.get_oidc_identity("https://idp.example.com", "sub-bf")
|
||||
assert after is not None
|
||||
assert after["oid"] == "obj-9"
|
||||
assert after["tid"] == "ten-9"
|
||||
|
||||
def test_update_oidc_identity_login_omitted_does_not_clobber_oid_tid(self, db):
|
||||
"""A later login WITHOUT oid/tid must not wipe previously-captured values."""
|
||||
db.create_oidc_identity("https://idp.example.com", "sub-keep", "u1", "a@example.com")
|
||||
db.update_oidc_identity_login(
|
||||
"https://idp.example.com", "sub-keep", oid="obj-keep", tid="ten-keep"
|
||||
)
|
||||
# Simulate a subsequent login where the token omitted oid/tid.
|
||||
db.update_oidc_identity_login("https://idp.example.com", "sub-keep")
|
||||
|
||||
identity = db.get_oidc_identity("https://idp.example.com", "sub-keep")
|
||||
assert identity is not None
|
||||
assert identity["oid"] == "obj-keep"
|
||||
assert identity["tid"] == "ten-keep"
|
||||
|
||||
def test_list_oidc_identities_for_user(self, db):
|
||||
"""Two identities for same user, list returns both."""
|
||||
db.create_oidc_identity("https://idp1.example.com", "sub-A", "u1", "alice@idp1.com")
|
||||
|
||||
@@ -521,16 +521,21 @@ class TestSpawnPersona:
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Guard 7 — task_agent has no persona parameter (sub-agents keep their own
|
||||
# identity; persona is a workstream-level concept).
|
||||
# Guard 7 — task_agent HAS a persona parameter: a sub-agent's identity comes
|
||||
# from a persona (default = the built-in task-agent identity), validated at
|
||||
# prep against the interactive kind. Revises the original "no persona for
|
||||
# task agents" stance now that personas are first-class on every path.
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_task_agent_schema_has_no_persona_param() -> None:
|
||||
def test_task_agent_schema_has_persona_param() -> None:
|
||||
from turnstone.core.tools import TOOLS
|
||||
|
||||
task_agent = next(t for t in TOOLS if t["function"]["name"] == "task_agent")
|
||||
assert "persona" not in task_agent["function"]["parameters"]["properties"]
|
||||
props = task_agent["function"]["parameters"]["properties"]
|
||||
assert "persona" in props
|
||||
# skill= is capability now — the description frames it that way.
|
||||
assert "capability" in props["skill"]["description"].lower()
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
@@ -228,34 +228,42 @@ def test_bulk_live_coordinator_row_uses_manager_snapshot(storage):
|
||||
live = body["results"][ws.id]
|
||||
assert live is not None
|
||||
assert "pending_approval" in live
|
||||
# New field always present on the wire — None when no approval
|
||||
# is pending so the JS can `key in row` without surprise.
|
||||
assert "pending_approval_detail" in live
|
||||
assert live["pending_approval_detail"] is None
|
||||
# The details list is always present on the wire — empty when no
|
||||
# approval is pending so the JS can `key in row` without surprise.
|
||||
# Replaces 1.6's singular ``pending_approval_detail`` null
|
||||
# (breaking, 1.7).
|
||||
assert "pending_approval_details" in live
|
||||
assert live["pending_approval_details"] == []
|
||||
|
||||
|
||||
def test_bulk_live_coordinator_row_includes_pending_approval_detail(storage):
|
||||
"""When _pending_approval is set on a coord UI, the live block
|
||||
surfaces the merged items + judge_verdict payload through the
|
||||
coord-pseudo-node path. End-to-end equivalent of the dashboard
|
||||
test in test_server_authz, but for the console live-bulk
|
||||
endpoint that the coord tree UI actually consumes."""
|
||||
def test_bulk_live_coordinator_row_includes_pending_approval_details(storage):
|
||||
"""When an approval cycle is live on a coord UI, the live block
|
||||
surfaces one detail entry per cycle with merged items +
|
||||
judge_verdict through the coord-pseudo-node path. End-to-end
|
||||
equivalent of the dashboard test in test_server_authz, but for
|
||||
the console live-bulk endpoint that the coord tree UI actually
|
||||
consumes."""
|
||||
from turnstone.core.session_ui_base import ApprovalCycle
|
||||
|
||||
mgr = _build_mgr(storage)
|
||||
ws = mgr.create(user_id="user-1")
|
||||
ws.ui._pending_approval = {
|
||||
items = [
|
||||
{
|
||||
"call_id": "c-99",
|
||||
"header": "spawn_workstream",
|
||||
"preview": "{...}",
|
||||
"func_name": "spawn_workstream",
|
||||
"approval_label": "spawn_workstream",
|
||||
"needs_approval": True,
|
||||
}
|
||||
]
|
||||
card = {
|
||||
"type": "approve_request",
|
||||
"items": [
|
||||
{
|
||||
"call_id": "c-99",
|
||||
"header": "spawn_workstream",
|
||||
"preview": "{...}",
|
||||
"func_name": "spawn_workstream",
|
||||
"approval_label": "spawn_workstream",
|
||||
"needs_approval": True,
|
||||
}
|
||||
],
|
||||
"cycle_id": "cyc-99",
|
||||
"items": ws.ui._serialize_approval_items(items),
|
||||
"judge_pending": False,
|
||||
}
|
||||
ws.ui._register_approval_cycle(ApprovalCycle(items, card, None))
|
||||
ws.ui._llm_verdicts["c-99"] = {
|
||||
"recommendation": "approve",
|
||||
"risk_level": "low",
|
||||
@@ -269,8 +277,10 @@ def test_bulk_live_coordinator_row_includes_pending_approval_detail(storage):
|
||||
assert resp.status_code == 200
|
||||
live = resp.json()["results"][ws.id]
|
||||
assert live["pending_approval"] is True # boolean derived flag
|
||||
detail = live["pending_approval_detail"]
|
||||
assert detail is not None
|
||||
details = live["pending_approval_details"]
|
||||
assert len(details) == 1
|
||||
detail = details[0]
|
||||
assert detail["cycle_id"] == "cyc-99"
|
||||
assert detail["call_id"] == "c-99"
|
||||
assert detail["items"][0]["func_name"] == "spawn_workstream"
|
||||
assert detail["items"][0]["judge_verdict"]["recommendation"] == "approve"
|
||||
|
||||
@@ -88,10 +88,9 @@ def _make_session(**kwargs):
|
||||
|
||||
|
||||
def _sys_content(session: ChatSession) -> str:
|
||||
"""Extract the system message content."""
|
||||
msgs = [m for m in session.system_messages if m["role"] == "system"]
|
||||
assert msgs
|
||||
return msgs[0]["content"]
|
||||
"""Full prompt prefix: identity system message + any skill context message."""
|
||||
assert session.system_messages
|
||||
return "\n".join(m["content"] for m in session.system_messages)
|
||||
|
||||
|
||||
def _create_template(db, template_id, name, content, is_default=False, **kwargs):
|
||||
@@ -187,17 +186,23 @@ class TestDefaultTemplates:
|
||||
content = _sys_content(session)
|
||||
assert "Not default." not in content
|
||||
|
||||
def test_templates_before_instructions(self, tmp_db):
|
||||
def test_default_template_stays_in_identity_system_message(self, tmp_db):
|
||||
"""Default (always-on) templates are the standing baseline and never
|
||||
change mid-session, so they stay in the identity system message (with
|
||||
user instructions) — only a NAMED applied skill moves to a separate
|
||||
context message. Template guidance still precedes user instructions."""
|
||||
from turnstone.core.storage import get_storage
|
||||
|
||||
db = get_storage()
|
||||
_create_template(db, "t1", "tpl", "TEMPLATE_CONTENT", is_default=True)
|
||||
|
||||
session = _make_session(instructions="USER_INSTRUCTIONS")
|
||||
content = _sys_content(session)
|
||||
tpl_pos = content.index("TEMPLATE_CONTENT")
|
||||
instr_pos = content.index("USER_INSTRUCTIONS")
|
||||
assert tpl_pos < instr_pos
|
||||
msgs = session.system_messages
|
||||
# Both the default template and instructions live in the system message,
|
||||
# template first — and no default triggers a user-role context message.
|
||||
assert all(m["role"] == "system" for m in msgs)
|
||||
content = msgs[0]["content"]
|
||||
assert content.index("TEMPLATE_CONTENT") < content.index("USER_INSTRUCTIONS")
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
@@ -12,6 +12,7 @@ toggle.
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import dataclasses
|
||||
import os
|
||||
import sys
|
||||
from typing import Any
|
||||
@@ -21,6 +22,7 @@ import pytest
|
||||
|
||||
from tests._session_helpers import make_session as _make_session
|
||||
from turnstone.core.providers._anthropic import AnthropicProvider
|
||||
from turnstone.core.providers._protocol import ModelCapabilities
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Helpers
|
||||
@@ -166,6 +168,178 @@ class TestCompatWireShape:
|
||||
assert kwargs["thinking"] == {"type": "enabled", "budget_tokens": 2048}
|
||||
|
||||
|
||||
# ===========================================================================
|
||||
# TestCompatReasoningControl
|
||||
# ===========================================================================
|
||||
|
||||
|
||||
class TestCompatReasoningControl:
|
||||
"""Session effort knob → ``chat_template_kwargs`` on the compat lane.
|
||||
|
||||
vLLM's ``/v1/messages`` ignores the native ``thinking`` param — the
|
||||
reasoning levers live in the chat template.
|
||||
``merge_reasoning_template_kwargs`` maps the knob onto
|
||||
``caps.thinking_param`` (manual: knob "none" = off, mirroring
|
||||
``_reasoning_params``; adaptive: always on) and ``caps.effort_param``
|
||||
(graded value for gpt-oss-style templates). Verified live against
|
||||
qwen3.6 on vLLM 2026-07-03: ``{"enable_thinking": false}`` disables
|
||||
thinking, unknown chat_template_kwargs keys are silently ignored.
|
||||
"""
|
||||
|
||||
_MANUAL_CAPS = ModelCapabilities(
|
||||
token_param="max_tokens",
|
||||
thinking_mode="manual",
|
||||
thinking_param="enable_thinking",
|
||||
)
|
||||
|
||||
def setup_method(self) -> None:
|
||||
self.provider = AnthropicProvider(compat=True)
|
||||
|
||||
def _stream_kwargs(
|
||||
self,
|
||||
caps: ModelCapabilities | None,
|
||||
reasoning_effort: str,
|
||||
extra_params: dict[str, Any] | None = None,
|
||||
) -> dict[str, Any]:
|
||||
client = _capture_client()
|
||||
with patch("turnstone.core.providers._anthropic._ensure_anthropic"):
|
||||
list(
|
||||
self.provider.create_streaming(
|
||||
client=client,
|
||||
model="qwen3.6-27b",
|
||||
messages=[{"role": "user", "content": "hi"}],
|
||||
temperature=0.6,
|
||||
reasoning_effort=reasoning_effort,
|
||||
extra_params=extra_params,
|
||||
capabilities=caps,
|
||||
)
|
||||
)
|
||||
return client.messages.stream.call_args[1]
|
||||
|
||||
def test_manual_toggle_on(self) -> None:
|
||||
"""Any non-none effort turns the toggle on AND carries the graded
|
||||
value under the fallback key — the user's effort setting always
|
||||
reaches the wire; a template that doesn't reference the kwarg
|
||||
ignores it."""
|
||||
kwargs = self._stream_kwargs(self._MANUAL_CAPS, "medium")
|
||||
assert kwargs["extra_body"] == {
|
||||
"chat_template_kwargs": {"enable_thinking": True, "reasoning_effort": "medium"}
|
||||
}
|
||||
assert "thinking" not in kwargs
|
||||
assert kwargs["temperature"] == 0.6 # never forced to 1.0 on compat
|
||||
|
||||
@pytest.mark.parametrize("knob", ["none", ""])
|
||||
def test_manual_toggle_off(self, knob: str) -> None:
|
||||
"""Effort "none"/empty disables thinking — native manual-mode parity;
|
||||
no effort key rides when thinking is off."""
|
||||
kwargs = self._stream_kwargs(self._MANUAL_CAPS, knob)
|
||||
assert kwargs["extra_body"] == {"chat_template_kwargs": {"enable_thinking": False}}
|
||||
assert "thinking" not in kwargs
|
||||
|
||||
def test_adaptive_always_on(self) -> None:
|
||||
"""Adaptive never knob-disables — native-adaptive contract, no native
|
||||
dict; the graded value rides for on-positions only."""
|
||||
caps = dataclasses.replace(self._MANUAL_CAPS, thinking_mode="adaptive")
|
||||
kwargs = self._stream_kwargs(caps, "high")
|
||||
assert kwargs["extra_body"] == {
|
||||
"chat_template_kwargs": {"enable_thinking": True, "reasoning_effort": "high"}
|
||||
}
|
||||
assert "thinking" not in kwargs
|
||||
assert kwargs["temperature"] == 0.6
|
||||
kwargs = self._stream_kwargs(caps, "none")
|
||||
assert kwargs["extra_body"] == {"chat_template_kwargs": {"enable_thinking": True}}
|
||||
assert "thinking" not in kwargs
|
||||
assert kwargs["temperature"] == 0.6
|
||||
|
||||
def test_default_caps_inject_nothing(self) -> None:
|
||||
"""Untouched compat defaults (thinking_mode=none) keep today's wire."""
|
||||
kwargs = self._stream_kwargs(None, "medium")
|
||||
assert "extra_body" not in kwargs
|
||||
assert "thinking" not in kwargs
|
||||
|
||||
def test_effort_param_validated_against_values(self) -> None:
|
||||
"""Off-list knob rounds up onto the declared values (ceiling-capped),
|
||||
never sent raw — and never snaps DOWN to the default."""
|
||||
caps = dataclasses.replace(
|
||||
self._MANUAL_CAPS,
|
||||
effort_param="reasoning_effort",
|
||||
reasoning_effort_values=("low", "medium", "high"),
|
||||
default_reasoning_effort="medium",
|
||||
)
|
||||
kwargs = self._stream_kwargs(caps, "xhigh")
|
||||
assert kwargs["extra_body"] == {
|
||||
"chat_template_kwargs": {"enable_thinking": True, "reasoning_effort": "high"}
|
||||
}
|
||||
|
||||
def test_effort_param_freeform_without_values(self) -> None:
|
||||
"""No declared values → knob forwarded as-is, template is authority."""
|
||||
caps = ModelCapabilities(
|
||||
token_param="max_tokens",
|
||||
effort_param="reasoning_effort",
|
||||
)
|
||||
kwargs = self._stream_kwargs(caps, "xhigh")
|
||||
assert kwargs["extra_body"] == {"chat_template_kwargs": {"reasoning_effort": "xhigh"}}
|
||||
|
||||
def test_effort_param_omitted_on_none(self) -> None:
|
||||
"""Knob "none" sends no effort key (and toggles thinking off)."""
|
||||
caps = dataclasses.replace(
|
||||
self._MANUAL_CAPS,
|
||||
effort_param="reasoning_effort",
|
||||
reasoning_effort_values=("low", "medium", "high"),
|
||||
)
|
||||
kwargs = self._stream_kwargs(caps, "none")
|
||||
assert kwargs["extra_body"] == {"chat_template_kwargs": {"enable_thinking": False}}
|
||||
|
||||
def test_operator_override_wins(self) -> None:
|
||||
"""server_compat chat_template_kwargs entries beat the knob mapping."""
|
||||
kwargs = self._stream_kwargs(
|
||||
self._MANUAL_CAPS,
|
||||
"none",
|
||||
extra_params={"chat_template_kwargs": {"enable_thinking": True}, "foo": 1},
|
||||
)
|
||||
assert kwargs["extra_body"] == {
|
||||
"chat_template_kwargs": {"enable_thinking": True},
|
||||
"foo": 1,
|
||||
}
|
||||
|
||||
def test_caller_extra_params_not_mutated(self) -> None:
|
||||
"""The session's extra_params dict must never be written through."""
|
||||
extra = {"chat_template_kwargs": {"foo": 1}}
|
||||
self._stream_kwargs(self._MANUAL_CAPS, "medium", extra_params=extra)
|
||||
assert extra == {"chat_template_kwargs": {"foo": 1}}
|
||||
|
||||
def test_no_output_config_on_compat(self) -> None:
|
||||
"""supports_effort must not leak Anthropic output_config to vLLM."""
|
||||
caps = dataclasses.replace(
|
||||
self._MANUAL_CAPS,
|
||||
supports_effort=True,
|
||||
effort_levels=("low", "medium", "high"),
|
||||
)
|
||||
kwargs = self._stream_kwargs(caps, "high")
|
||||
assert "output_config" not in kwargs
|
||||
assert kwargs["extra_body"] == {
|
||||
"chat_template_kwargs": {"enable_thinking": True, "reasoning_effort": "high"}
|
||||
}
|
||||
|
||||
def test_create_completion_same_injection(self) -> None:
|
||||
"""The non-streaming path shares _build_thinking_and_kwargs."""
|
||||
client = MagicMock()
|
||||
final = MagicMock(content=[], stop_reason="end_turn")
|
||||
stream = MagicMock()
|
||||
stream.get_final_message.return_value = final
|
||||
client.messages.stream.return_value.__enter__.return_value = stream
|
||||
with patch("turnstone.core.providers._anthropic._ensure_anthropic"):
|
||||
self.provider.create_completion(
|
||||
client=client,
|
||||
model="qwen3.6-27b",
|
||||
messages=[{"role": "user", "content": "hi"}],
|
||||
reasoning_effort="none",
|
||||
capabilities=self._MANUAL_CAPS,
|
||||
)
|
||||
kwargs = client.messages.stream.call_args[1]
|
||||
assert kwargs["extra_body"] == {"chat_template_kwargs": {"enable_thinking": False}}
|
||||
|
||||
|
||||
# ===========================================================================
|
||||
# TestCompatFactory
|
||||
# ===========================================================================
|
||||
|
||||
+200
-81
@@ -12,10 +12,12 @@ from turnstone.core.lowering import repair_wire_messages
|
||||
from turnstone.core.providers._openai import OpenAIProvider
|
||||
from turnstone.core.providers._openai_chat import OpenAIChatCompletionsProvider
|
||||
from turnstone.core.providers._openai_common import (
|
||||
OPENAI_COMPAT_DEFAULT,
|
||||
apply_cache_retention,
|
||||
apply_temperature_and_effort,
|
||||
apply_tool_search,
|
||||
format_citations,
|
||||
lookup_openai_capabilities,
|
||||
sanitize_messages,
|
||||
)
|
||||
from turnstone.core.providers._protocol import (
|
||||
@@ -153,51 +155,89 @@ class TestOpenAIProvider:
|
||||
def test_provider_name(self) -> None:
|
||||
assert self.provider.provider_name == "openai-compatible"
|
||||
|
||||
# -- _apply_thinking_mode -------------------------------------------------
|
||||
# -- reasoning template kwargs (_finalize_extra_body) ---------------------
|
||||
|
||||
def test_thinking_mode_none_does_nothing(self) -> None:
|
||||
"""No thinking params injected when thinking_mode is 'none'."""
|
||||
"""No toggle injected when thinking_mode is 'none'; operator keys pass."""
|
||||
caps = ModelCapabilities(thinking_mode="none")
|
||||
extra_body: dict[str, Any] = {"chat_template_kwargs": {"reasoning_effort": "medium"}}
|
||||
OpenAIProvider._apply_thinking_mode(extra_body, caps)
|
||||
assert "enable_thinking" not in extra_body["chat_template_kwargs"]
|
||||
extra_params = {"chat_template_kwargs": {"reasoning_effort": "medium"}}
|
||||
eb = self.provider._finalize_extra_body(extra_params, caps, "medium")
|
||||
assert eb is not None
|
||||
assert "enable_thinking" not in eb["chat_template_kwargs"]
|
||||
assert eb["chat_template_kwargs"]["reasoning_effort"] == "medium"
|
||||
|
||||
def test_thinking_mode_manual_injects_param(self) -> None:
|
||||
"""Manual thinking mode injects enable_thinking into chat_template_kwargs."""
|
||||
caps = ModelCapabilities(thinking_mode="manual")
|
||||
extra_body: dict[str, Any] = {"chat_template_kwargs": {"reasoning_effort": "medium"}}
|
||||
OpenAIProvider._apply_thinking_mode(extra_body, caps)
|
||||
assert extra_body["chat_template_kwargs"]["enable_thinking"] is True
|
||||
assert extra_body["chat_template_kwargs"]["reasoning_effort"] == "medium"
|
||||
extra_params = {"chat_template_kwargs": {"reasoning_effort": "medium"}}
|
||||
eb = self.provider._finalize_extra_body(extra_params, caps, "medium")
|
||||
assert eb is not None
|
||||
assert eb["chat_template_kwargs"]["enable_thinking"] is True
|
||||
assert eb["chat_template_kwargs"]["reasoning_effort"] == "medium"
|
||||
|
||||
def test_thinking_mode_manual_knob_none_disables(self) -> None:
|
||||
"""Effort knob "none" turns the template toggle off, not just quiet."""
|
||||
caps = ModelCapabilities(thinking_mode="manual")
|
||||
eb = self.provider._finalize_extra_body(None, caps, "none")
|
||||
assert eb == {"chat_template_kwargs": {"enable_thinking": False}}
|
||||
|
||||
def test_thinking_mode_custom_param(self) -> None:
|
||||
"""Custom thinking_param (e.g. Granite's 'thinking') is used."""
|
||||
caps = ModelCapabilities(thinking_mode="manual", thinking_param="thinking")
|
||||
extra_body: dict[str, Any] = {"chat_template_kwargs": {}}
|
||||
OpenAIProvider._apply_thinking_mode(extra_body, caps)
|
||||
assert extra_body["chat_template_kwargs"]["thinking"] is True
|
||||
assert "enable_thinking" not in extra_body["chat_template_kwargs"]
|
||||
eb = self.provider._finalize_extra_body(None, caps, "medium")
|
||||
assert eb == {"chat_template_kwargs": {"thinking": True}}
|
||||
|
||||
def test_thinking_mode_does_not_override_explicit(self) -> None:
|
||||
"""If operator explicitly set the param to False, provider respects it."""
|
||||
caps = ModelCapabilities(thinking_mode="manual")
|
||||
extra_body: dict[str, Any] = {"chat_template_kwargs": {"enable_thinking": False}}
|
||||
OpenAIProvider._apply_thinking_mode(extra_body, caps)
|
||||
assert extra_body["chat_template_kwargs"]["enable_thinking"] is False
|
||||
extra_params = {"chat_template_kwargs": {"enable_thinking": False}}
|
||||
eb = self.provider._finalize_extra_body(extra_params, caps, "medium")
|
||||
assert eb is not None
|
||||
assert eb["chat_template_kwargs"]["enable_thinking"] is False
|
||||
|
||||
def test_thinking_mode_creates_ctk_if_missing(self) -> None:
|
||||
"""Creates chat_template_kwargs dict if not present in extra_body."""
|
||||
caps = ModelCapabilities(thinking_mode="manual")
|
||||
extra_body: dict[str, Any] = {}
|
||||
OpenAIProvider._apply_thinking_mode(extra_body, caps)
|
||||
assert extra_body["chat_template_kwargs"]["enable_thinking"] is True
|
||||
|
||||
def test_thinking_mode_adaptive(self) -> None:
|
||||
"""Adaptive thinking mode also injects the param."""
|
||||
def test_thinking_mode_adaptive_never_knob_disables(self) -> None:
|
||||
"""Adaptive = model self-regulates; knob "none" must not force false."""
|
||||
caps = ModelCapabilities(thinking_mode="adaptive")
|
||||
extra_body: dict[str, Any] = {"chat_template_kwargs": {}}
|
||||
OpenAIProvider._apply_thinking_mode(extra_body, caps)
|
||||
assert extra_body["chat_template_kwargs"]["enable_thinking"] is True
|
||||
for knob in ("high", "none", ""):
|
||||
eb = self.provider._finalize_extra_body(None, caps, knob)
|
||||
assert eb == {"chat_template_kwargs": {"enable_thinking": True}}
|
||||
|
||||
def test_effort_param_suppresses_flat_reasoning_effort(self) -> None:
|
||||
"""Declaring the ctk effort channel must not double-send the flat param."""
|
||||
from turnstone.core.providers._openai_common import apply_temperature_and_effort
|
||||
|
||||
caps = ModelCapabilities(
|
||||
effort_param="reasoning_effort",
|
||||
reasoning_effort_values=("low", "medium", "high"),
|
||||
)
|
||||
kwargs: dict[str, Any] = {}
|
||||
apply_temperature_and_effort(kwargs, caps, 0.5, "medium")
|
||||
assert "reasoning_effort" not in kwargs
|
||||
# Without effort_param the flat param still flows (commercial path).
|
||||
flat_caps = ModelCapabilities(reasoning_effort_values=("low", "medium", "high"))
|
||||
kwargs = {}
|
||||
apply_temperature_and_effort(kwargs, flat_caps, 0.5, "medium")
|
||||
assert kwargs["reasoning_effort"] == "medium"
|
||||
|
||||
def test_effort_param_injects_knob_value(self) -> None:
|
||||
"""effort_param carries the knob into chat_template_kwargs (gpt-oss);
|
||||
a knob above the declared ceiling rides the ceiling, not the default."""
|
||||
caps = ModelCapabilities(
|
||||
thinking_mode="none",
|
||||
effort_param="reasoning_effort",
|
||||
reasoning_effort_values=("low", "medium", "high"),
|
||||
default_reasoning_effort="medium",
|
||||
)
|
||||
eb = self.provider._finalize_extra_body(None, caps, "xhigh")
|
||||
assert eb == {"chat_template_kwargs": {"reasoning_effort": "high"}}
|
||||
assert self.provider._finalize_extra_body(None, caps, "none") is None
|
||||
|
||||
def test_caller_extra_params_not_mutated(self) -> None:
|
||||
"""The session dict and its ctk sub-dict survive injection untouched."""
|
||||
caps = ModelCapabilities(thinking_mode="manual")
|
||||
extra_params = {"chat_template_kwargs": {"foo": 1}}
|
||||
self.provider._finalize_extra_body(extra_params, caps, "medium")
|
||||
assert extra_params == {"chat_template_kwargs": {"foo": 1}}
|
||||
|
||||
# -- _sanitize_messages ---------------------------------------------------
|
||||
|
||||
@@ -1633,6 +1673,25 @@ class TestProviderFactory:
|
||||
assert openai_prov.provider_name == "openai"
|
||||
assert compat.provider_name == "openai-compatible"
|
||||
|
||||
def test_openai_compatible_never_consults_commercial_registry(self) -> None:
|
||||
"""Local-lane model ids are operator-chosen strings — a prefix
|
||||
collision with a cloud model id must not inherit that model's
|
||||
sampling/effort contract, on either API surface. Cloud lookups
|
||||
are unaffected."""
|
||||
from turnstone.core.providers import create_provider
|
||||
|
||||
compat = create_provider("openai-compatible")
|
||||
compat_responses = create_provider("openai-compatible", api_surface="responses")
|
||||
for name in ("gpt-5.5-my-finetune", "o3-distill", "deepseek-v4-flash", ""):
|
||||
assert compat.get_capabilities(name) is OPENAI_COMPAT_DEFAULT
|
||||
assert compat_responses.get_capabilities(name) is OPENAI_COMPAT_DEFAULT
|
||||
# The commercial lane keeps resolving its registry rows — through
|
||||
# the factory AND through the non-compat class default.
|
||||
cloud = create_provider("openai").get_capabilities("gpt-5.5")
|
||||
assert cloud.default_reasoning_effort == "medium"
|
||||
assert "xhigh" in cloud.reasoning_effort_values
|
||||
assert create_provider("openai") is not compat_responses
|
||||
|
||||
def test_create_provider_returns_singleton(self) -> None:
|
||||
from turnstone.core.providers import create_provider
|
||||
|
||||
@@ -1765,6 +1824,39 @@ class TestProviderFactory:
|
||||
# ===========================================================================
|
||||
|
||||
|
||||
class TestGoogleEffortKnob:
|
||||
"""The session effort knob reaches Gemini as a flat reasoning_effort."""
|
||||
|
||||
def _create_kwargs(self, reasoning_effort: str) -> dict[str, Any]:
|
||||
from turnstone.core.providers._google import GoogleProvider
|
||||
|
||||
prov = GoogleProvider()
|
||||
client = MagicMock()
|
||||
client.chat.completions.create.return_value = iter([])
|
||||
list(
|
||||
prov.create_streaming(
|
||||
client=client,
|
||||
model="gemini-3-flash",
|
||||
messages=[{"role": "user", "content": "hi"}],
|
||||
reasoning_effort=reasoning_effort,
|
||||
)
|
||||
)
|
||||
return client.chat.completions.create.call_args[1]
|
||||
|
||||
def test_knob_values_forward_verbatim(self) -> None:
|
||||
for knob in ("minimal", "low", "medium", "high"):
|
||||
assert self._create_kwargs(knob)["reasoning_effort"] == knob
|
||||
|
||||
def test_off_list_knob_snaps_to_high(self) -> None:
|
||||
"""xhigh/max are not in Gemini's vocabulary — snap down to high."""
|
||||
for knob in ("xhigh", "max"):
|
||||
assert self._create_kwargs(knob)["reasoning_effort"] == "high"
|
||||
|
||||
def test_none_omits_the_param(self) -> None:
|
||||
"""Knob none never sends "none" — 2.5 Pro / 3.x reject disabling."""
|
||||
assert "reasoning_effort" not in self._create_kwargs("none")
|
||||
|
||||
|
||||
class TestGoogleProviderFidelity:
|
||||
"""Tests for thought_signature round-trip via provider_blocks."""
|
||||
|
||||
@@ -2003,49 +2095,64 @@ class TestOpenAIParameterGating:
|
||||
def setup_method(self) -> None:
|
||||
self.provider = OpenAIProvider()
|
||||
|
||||
def test_unknown_model_no_reasoning_effort(self) -> None:
|
||||
"""Unknown/local models should NOT receive top-level reasoning_effort."""
|
||||
def test_local_model_effort_forwarded_verbatim(self) -> None:
|
||||
"""Local-lane models receive the session knob verbatim on the flat
|
||||
param (effort_passthrough) — the user's effort setting always
|
||||
reaches the wire; "none" stays omitted (nothing to disable
|
||||
beyond the template toggle)."""
|
||||
caps = self.provider.get_capabilities("my-local-model")
|
||||
kwargs: dict[str, Any] = {}
|
||||
apply_temperature_and_effort(kwargs, caps, temperature=0.7, reasoning_effort="medium")
|
||||
assert "reasoning_effort" not in kwargs
|
||||
assert kwargs["reasoning_effort"] == "medium"
|
||||
assert kwargs["temperature"] == 0.7
|
||||
kwargs = {}
|
||||
apply_temperature_and_effort(kwargs, caps, temperature=0.7, reasoning_effort="none")
|
||||
assert "reasoning_effort" not in kwargs
|
||||
|
||||
def test_gpt5_no_temperature_has_reasoning_effort(self) -> None:
|
||||
"""GPT-5 base: no temperature, reasoning_effort sent."""
|
||||
caps = self.provider.get_capabilities("gpt-5")
|
||||
caps = lookup_openai_capabilities("gpt-5")
|
||||
kwargs: dict[str, Any] = {}
|
||||
apply_temperature_and_effort(kwargs, caps, temperature=0.7, reasoning_effort="high")
|
||||
assert "temperature" not in kwargs
|
||||
assert kwargs["reasoning_effort"] == "high"
|
||||
|
||||
def test_gpt51_temperature_when_effort_none(self) -> None:
|
||||
"""GPT-5.1: temperature only when reasoning_effort='none'."""
|
||||
caps = self.provider.get_capabilities("gpt-5.1")
|
||||
"""GPT-5.1: temperature only when reasoning_effort='none'; the
|
||||
declared "none" level is forwarded explicitly (knob = off)."""
|
||||
caps = lookup_openai_capabilities("gpt-5.1")
|
||||
kwargs: dict[str, Any] = {}
|
||||
apply_temperature_and_effort(kwargs, caps, temperature=0.7, reasoning_effort="none")
|
||||
assert kwargs["temperature"] == 0.7
|
||||
assert "reasoning_effort" not in kwargs # "none" is skipped
|
||||
assert kwargs["reasoning_effort"] == "none"
|
||||
|
||||
def test_gpt51_no_temperature_when_reasoning_active(self) -> None:
|
||||
"""GPT-5.1: no temperature when reasoning is active."""
|
||||
caps = self.provider.get_capabilities("gpt-5.1")
|
||||
caps = lookup_openai_capabilities("gpt-5.1")
|
||||
kwargs: dict[str, Any] = {}
|
||||
apply_temperature_and_effort(kwargs, caps, temperature=0.7, reasoning_effort="high")
|
||||
assert "temperature" not in kwargs
|
||||
assert kwargs["reasoning_effort"] == "high"
|
||||
|
||||
def test_o_series_no_temperature_no_reasoning_effort(self) -> None:
|
||||
"""O-series: no temperature, no reasoning_effort."""
|
||||
caps = self.provider.get_capabilities("o3")
|
||||
def test_o_series_no_temperature_but_effort_forwarded(self) -> None:
|
||||
"""O-series: no temperature; low/medium/high ARE valid effort
|
||||
values (all o-series except o1-mini) and the knob reaches them."""
|
||||
caps = lookup_openai_capabilities("o3")
|
||||
kwargs: dict[str, Any] = {}
|
||||
apply_temperature_and_effort(kwargs, caps, temperature=0.7, reasoning_effort="medium")
|
||||
assert "temperature" not in kwargs
|
||||
assert kwargs["reasoning_effort"] == "medium"
|
||||
|
||||
def test_o1_mini_has_no_effort_control(self) -> None:
|
||||
"""o1-mini is the one o-series model without reasoning_effort."""
|
||||
caps = lookup_openai_capabilities("o1-mini")
|
||||
kwargs: dict[str, Any] = {}
|
||||
apply_temperature_and_effort(kwargs, caps, temperature=0.7, reasoning_effort="medium")
|
||||
assert "reasoning_effort" not in kwargs
|
||||
|
||||
def test_gpt5_pro_unsupported_effort_falls_back(self) -> None:
|
||||
"""GPT-5 pro only supports 'high'; unsupported values fall back to default."""
|
||||
caps = self.provider.get_capabilities("gpt-5-pro")
|
||||
caps = lookup_openai_capabilities("gpt-5-pro")
|
||||
kwargs: dict[str, Any] = {}
|
||||
apply_temperature_and_effort(kwargs, caps, temperature=0.7, reasoning_effort="medium")
|
||||
assert "temperature" not in kwargs
|
||||
@@ -2053,19 +2160,19 @@ class TestOpenAIParameterGating:
|
||||
|
||||
def test_gpt5_pro_supported_effort_passes_through(self) -> None:
|
||||
"""GPT-5 pro accepts 'high' directly."""
|
||||
caps = self.provider.get_capabilities("gpt-5-pro")
|
||||
caps = lookup_openai_capabilities("gpt-5-pro")
|
||||
kwargs: dict[str, Any] = {}
|
||||
apply_temperature_and_effort(kwargs, caps, temperature=0.7, reasoning_effort="high")
|
||||
assert kwargs["reasoning_effort"] == "high"
|
||||
|
||||
def test_gpt54_1m_context_and_effort(self) -> None:
|
||||
"""GPT-5.4: 1M context, temperature when effort=none, xhigh supported."""
|
||||
caps = self.provider.get_capabilities("gpt-5.4")
|
||||
caps = lookup_openai_capabilities("gpt-5.4")
|
||||
assert caps.context_window == 1050000
|
||||
kwargs: dict[str, Any] = {}
|
||||
apply_temperature_and_effort(kwargs, caps, temperature=0.7, reasoning_effort="none")
|
||||
assert kwargs["temperature"] == 0.7
|
||||
assert "reasoning_effort" not in kwargs
|
||||
assert kwargs["reasoning_effort"] == "none" # declared level, forwarded
|
||||
kwargs2: dict[str, Any] = {}
|
||||
apply_temperature_and_effort(kwargs2, caps, temperature=0.7, reasoning_effort="xhigh")
|
||||
assert "temperature" not in kwargs2
|
||||
@@ -2073,7 +2180,7 @@ class TestOpenAIParameterGating:
|
||||
|
||||
def test_gpt54_pro_no_temperature_always_reasoning(self) -> None:
|
||||
"""GPT-5.4 pro: no temperature, medium/high/xhigh only."""
|
||||
caps = self.provider.get_capabilities("gpt-5.4-pro")
|
||||
caps = lookup_openai_capabilities("gpt-5.4-pro")
|
||||
assert caps.context_window == 1050000
|
||||
kwargs: dict[str, Any] = {}
|
||||
apply_temperature_and_effort(kwargs, caps, temperature=0.7, reasoning_effort="low")
|
||||
@@ -2082,14 +2189,14 @@ class TestOpenAIParameterGating:
|
||||
|
||||
def test_gpt55_1m_context_and_effort(self) -> None:
|
||||
"""GPT-5.5: 1M context, temperature when effort=none, xhigh supported."""
|
||||
caps = self.provider.get_capabilities("gpt-5.5")
|
||||
caps = lookup_openai_capabilities("gpt-5.5")
|
||||
assert caps.context_window == 1050000
|
||||
assert caps.supports_tool_search is True
|
||||
assert caps.supports_vision is True
|
||||
kwargs: dict[str, Any] = {}
|
||||
apply_temperature_and_effort(kwargs, caps, temperature=0.7, reasoning_effort="none")
|
||||
assert kwargs["temperature"] == 0.7
|
||||
assert "reasoning_effort" not in kwargs
|
||||
assert kwargs["reasoning_effort"] == "none" # declared level, forwarded
|
||||
kwargs2: dict[str, Any] = {}
|
||||
apply_temperature_and_effort(kwargs2, caps, temperature=0.7, reasoning_effort="xhigh")
|
||||
assert "temperature" not in kwargs2
|
||||
@@ -2097,7 +2204,7 @@ class TestOpenAIParameterGating:
|
||||
|
||||
def test_gpt55_pro_no_temperature_always_reasoning(self) -> None:
|
||||
"""GPT-5.5 pro: no temperature, medium/high/xhigh only."""
|
||||
caps = self.provider.get_capabilities("gpt-5.5-pro")
|
||||
caps = lookup_openai_capabilities("gpt-5.5-pro")
|
||||
assert caps.context_window == 1050000
|
||||
assert caps.supports_tool_search is True
|
||||
kwargs: dict[str, Any] = {}
|
||||
@@ -2316,11 +2423,20 @@ class TestAnthropicReasoningNone:
|
||||
result = _map_reasoning_to_effort("xhigh", ("low", "medium", "high", "xhigh", "max"))
|
||||
assert result == "xhigh"
|
||||
|
||||
def test_map_xhigh_rejected_by_model_without_it(self) -> None:
|
||||
def test_map_xhigh_snaps_up_through_gap_to_max(self) -> None:
|
||||
"""Levels with a hole (no xhigh) round the knob UP to the next
|
||||
declared level rather than dropping output_config entirely."""
|
||||
from turnstone.core.providers._anthropic import _map_reasoning_to_effort
|
||||
|
||||
result = _map_reasoning_to_effort("xhigh", ("low", "medium", "high", "max"))
|
||||
assert result is None
|
||||
assert result == "max"
|
||||
|
||||
def test_map_above_ceiling_rides_ceiling(self) -> None:
|
||||
from turnstone.core.providers._anthropic import _map_reasoning_to_effort
|
||||
|
||||
assert _map_reasoning_to_effort("max", ("low", "medium", "high")) == "high"
|
||||
assert _map_reasoning_to_effort("minimal", ("low", "medium", "high")) == "low"
|
||||
assert _map_reasoning_to_effort("none", ("low", "medium", "high")) is None
|
||||
|
||||
|
||||
# ===========================================================================
|
||||
@@ -2704,19 +2820,19 @@ class TestOpenAIWebSearch:
|
||||
|
||||
def test_search_model_capability(self) -> None:
|
||||
"""Search models should have supports_web_search=True."""
|
||||
caps = self.provider.get_capabilities("gpt-5-search-api")
|
||||
caps = lookup_openai_capabilities("gpt-5-search-api")
|
||||
assert caps.supports_web_search is True
|
||||
|
||||
def test_non_search_model_no_web_search(self) -> None:
|
||||
"""Regular models should not have supports_web_search."""
|
||||
caps = self.provider.get_capabilities("gpt-5")
|
||||
caps = lookup_openai_capabilities("gpt-5")
|
||||
assert caps.supports_web_search is False
|
||||
caps = self.provider.get_capabilities("gpt-5.2")
|
||||
caps = lookup_openai_capabilities("gpt-5.2")
|
||||
assert caps.supports_web_search is False
|
||||
|
||||
def test_apply_web_search_injects_options(self) -> None:
|
||||
"""For search models, web_search_options should be added to kwargs."""
|
||||
caps = self.provider.get_capabilities("gpt-5-search-api")
|
||||
caps = lookup_openai_capabilities("gpt-5-search-api")
|
||||
kwargs: dict[str, Any] = {"model": "gpt-5-search-api"}
|
||||
tools: list[dict[str, Any]] = [
|
||||
{"type": "function", "function": {"name": "bash", "description": "Run bash"}},
|
||||
@@ -2733,7 +2849,7 @@ class TestOpenAIWebSearch:
|
||||
|
||||
def test_apply_web_search_no_op_for_regular_models(self) -> None:
|
||||
"""For non-search models, no web_search_options, tools unchanged."""
|
||||
caps = self.provider.get_capabilities("gpt-5")
|
||||
caps = lookup_openai_capabilities("gpt-5")
|
||||
kwargs: dict[str, Any] = {"model": "gpt-5"}
|
||||
tools: list[dict[str, Any]] = [
|
||||
{"type": "function", "function": {"name": "web_search", "description": "Search"}},
|
||||
@@ -2744,7 +2860,7 @@ class TestOpenAIWebSearch:
|
||||
|
||||
def test_apply_web_search_returns_none_when_only_web_search(self) -> None:
|
||||
"""If web_search was the only tool, return None after removing it."""
|
||||
caps = self.provider.get_capabilities("gpt-5-search-api")
|
||||
caps = lookup_openai_capabilities("gpt-5-search-api")
|
||||
kwargs: dict[str, Any] = {}
|
||||
tools: list[dict[str, Any]] = [
|
||||
{"type": "function", "function": {"name": "web_search", "description": "Search"}},
|
||||
@@ -2758,7 +2874,7 @@ class TestOpenAIWebSearch:
|
||||
toolset) must NOT gain native search — the option stays off and the
|
||||
tools pass through untouched. Contrast test_apply_web_search_with_
|
||||
no_tools, which covers the tool-less utility-call case."""
|
||||
caps = self.provider.get_capabilities("gpt-5-search-api")
|
||||
caps = lookup_openai_capabilities("gpt-5-search-api")
|
||||
assert caps.supports_web_search is True
|
||||
kwargs: dict[str, Any] = {"model": "gpt-5-search-api"}
|
||||
tools: list[dict[str, Any]] = [
|
||||
@@ -2833,7 +2949,7 @@ class TestOpenAIWebSearch:
|
||||
visibility set, coordinator toolset, or a tool-less utility call —
|
||||
must not gain native search at the provider layer.
|
||||
"""
|
||||
caps = self.provider.get_capabilities("gpt-5-search-api")
|
||||
caps = lookup_openai_capabilities("gpt-5-search-api")
|
||||
kwargs: dict[str, Any] = {}
|
||||
result = self.provider._apply_web_search(kwargs, caps, None)
|
||||
assert "web_search_options" not in kwargs
|
||||
@@ -2841,7 +2957,7 @@ class TestOpenAIWebSearch:
|
||||
|
||||
def test_apply_web_search_replaces_client_def(self) -> None:
|
||||
"""With the client def present, it is filtered and the option set."""
|
||||
caps = self.provider.get_capabilities("gpt-5-search-api")
|
||||
caps = lookup_openai_capabilities("gpt-5-search-api")
|
||||
kwargs: dict[str, Any] = {}
|
||||
tools = [{"type": "function", "function": {"name": "web_search"}}]
|
||||
result = self.provider._apply_web_search(kwargs, caps, tools)
|
||||
@@ -2867,6 +2983,10 @@ class TestOpenAIWebSearch:
|
||||
"function": {"name": "web_search", "description": "Search"},
|
||||
},
|
||||
],
|
||||
# The local lane resolves no commercial rows — the search
|
||||
# model's capabilities ride in explicitly, as the session
|
||||
# layer would pass them.
|
||||
capabilities=lookup_openai_capabilities("gpt-5-search-api"),
|
||||
)
|
||||
)
|
||||
call_kwargs = client.chat.completions.create.call_args[1]
|
||||
@@ -3287,22 +3407,18 @@ class TestAnthropicToolSearch:
|
||||
|
||||
|
||||
class TestOpenAIToolSearch:
|
||||
"""Test OpenAI provider tool search injection."""
|
||||
"""Test OpenAI tool search injection (registry rows + shared helper)."""
|
||||
|
||||
@pytest.fixture()
|
||||
def provider(self):
|
||||
return OpenAIProvider()
|
||||
|
||||
def test_tool_search_capability_on_gpt54(self, provider):
|
||||
caps = provider.get_capabilities("gpt-5.4")
|
||||
def test_tool_search_capability_on_gpt54(self):
|
||||
caps = lookup_openai_capabilities("gpt-5.4")
|
||||
assert caps.supports_tool_search is True
|
||||
|
||||
def test_tool_search_not_supported_on_gpt5(self, provider):
|
||||
caps = provider.get_capabilities("gpt-5")
|
||||
def test_tool_search_not_supported_on_gpt5(self):
|
||||
caps = lookup_openai_capabilities("gpt-5")
|
||||
assert caps.supports_tool_search is False
|
||||
|
||||
def test_apply_tool_search_marks_deferred(self, provider):
|
||||
caps = provider.get_capabilities("gpt-5.4")
|
||||
def test_apply_tool_search_marks_deferred(self):
|
||||
caps = lookup_openai_capabilities("gpt-5.4")
|
||||
tools = [
|
||||
{"type": "function", "function": {"name": "bash", "description": "Run commands"}},
|
||||
{
|
||||
@@ -3318,16 +3434,16 @@ class TestOpenAIToolSearch:
|
||||
# slack tool deferred
|
||||
assert result[1]["defer_loading"] is True
|
||||
|
||||
def test_apply_tool_search_no_op_without_deferred(self, provider):
|
||||
caps = provider.get_capabilities("gpt-5.4")
|
||||
def test_apply_tool_search_no_op_without_deferred(self):
|
||||
caps = lookup_openai_capabilities("gpt-5.4")
|
||||
tools = [
|
||||
{"type": "function", "function": {"name": "bash", "description": "Run commands"}},
|
||||
]
|
||||
result = apply_tool_search(caps, tools, None)
|
||||
assert result == tools
|
||||
|
||||
def test_apply_tool_search_no_op_on_unsupported_model(self, provider):
|
||||
caps = provider.get_capabilities("gpt-5")
|
||||
def test_apply_tool_search_no_op_on_unsupported_model(self):
|
||||
caps = lookup_openai_capabilities("gpt-5")
|
||||
tools = [
|
||||
{"type": "function", "function": {"name": "bash", "description": "Run commands"}},
|
||||
]
|
||||
@@ -3395,13 +3511,12 @@ class TestVisionCapabilities:
|
||||
assert caps.supports_vision is False
|
||||
|
||||
def test_openai_commercial_supports_vision(self) -> None:
|
||||
provider = OpenAIProvider()
|
||||
for model in ("gpt-5", "gpt-5-mini", "gpt-5.4", "o3", "o4-mini"):
|
||||
caps = provider.get_capabilities(model)
|
||||
caps = lookup_openai_capabilities(model)
|
||||
assert caps.supports_vision is True, f"{model} should support vision"
|
||||
|
||||
def test_openai_default_no_vision(self) -> None:
|
||||
"""Unknown models (local servers) default to no vision."""
|
||||
"""Local-lane models (any name) default to no vision."""
|
||||
provider = OpenAIProvider()
|
||||
caps = provider.get_capabilities("some-local-model")
|
||||
assert caps.supports_vision is False
|
||||
@@ -3626,8 +3741,9 @@ class TestAnthropicPromptCaching:
|
||||
)
|
||||
assert kwargs["output_config"] == {"effort": "xhigh"}
|
||||
|
||||
def test_xhigh_effort_not_applied_to_opus_4_6(self) -> None:
|
||||
"""xhigh is not a valid effort level for Opus 4.6 — should be ignored."""
|
||||
def test_xhigh_effort_snaps_to_max_on_opus_4_6(self) -> None:
|
||||
"""Opus 4.6 declares (low, medium, high, max) — a knob of xhigh
|
||||
rounds up to max instead of silently dropping output_config."""
|
||||
caps = self.provider.get_capabilities("claude-opus-4-6")
|
||||
kwargs = self.provider._build_thinking_and_kwargs(
|
||||
caps=caps,
|
||||
@@ -3640,7 +3756,7 @@ class TestAnthropicPromptCaching:
|
||||
model="claude-opus-4-6",
|
||||
tools=None,
|
||||
)
|
||||
assert "output_config" not in kwargs
|
||||
assert kwargs["output_config"] == {"effort": "max"}
|
||||
|
||||
@patch("turnstone.core.providers._anthropic._ensure_anthropic")
|
||||
def test_streaming_message_start_cache_metrics(self, mock_ensure: MagicMock) -> None:
|
||||
@@ -4173,7 +4289,10 @@ class TestResponsesParamBuilding:
|
||||
assert kwargs["reasoning"] == {"effort": "high"}
|
||||
assert "reasoning_effort" not in kwargs
|
||||
|
||||
def test_no_reasoning_when_none_effort(self) -> None:
|
||||
def test_none_effort_sends_declared_none_level(self) -> None:
|
||||
"""gpt-5.4 declares an explicit "none" level — the knob's off
|
||||
position forwards it rather than omitting (omission would leave
|
||||
the server default in charge on models like gpt-5.5)."""
|
||||
kwargs = self.provider._build_kwargs(
|
||||
model="gpt-5.4",
|
||||
messages=[{"role": "user", "content": "Hi"}],
|
||||
@@ -4183,7 +4302,7 @@ class TestResponsesParamBuilding:
|
||||
reasoning_effort="none",
|
||||
deferred_names=None,
|
||||
)
|
||||
assert "reasoning" not in kwargs
|
||||
assert kwargs["reasoning"] == {"effort": "none"}
|
||||
|
||||
def test_store_is_false(self) -> None:
|
||||
kwargs = self.provider._build_kwargs(
|
||||
|
||||
+34
-21
@@ -88,18 +88,19 @@ class _FakeUI:
|
||||
self._ws_turn_tool_calls = 0
|
||||
self._llm_verdicts: dict[str, dict[str, Any]] = {}
|
||||
|
||||
def serialize_pending_approval_detail(self) -> dict[str, Any] | None:
|
||||
# Mirrors SessionUIBase.serialize_pending_approval_detail —
|
||||
def serialize_pending_approval_details(self) -> list[dict[str, Any]]:
|
||||
# Mirrors SessionUIBase.serialize_pending_approval_details —
|
||||
# the fake is monkeypatched in for ``WebUI`` and the dashboard
|
||||
# handler reads this method during projection. Real subclasses
|
||||
# inherit from ``SessionUIBase``; the fake replicates the
|
||||
# shape directly to stay decoupled.
|
||||
# iterate their approval-cycle registry (one entry per live
|
||||
# cycle); the fake models a single slot, so the list carries
|
||||
# zero or one entries.
|
||||
pending = self._pending_approval
|
||||
if pending is None:
|
||||
return None
|
||||
return []
|
||||
items = pending.get("items") or []
|
||||
if not items:
|
||||
return None
|
||||
return []
|
||||
call_ids = [item.get("call_id", "") for item in items]
|
||||
# Match the real impl's pattern (session_ui_base.py): snapshot
|
||||
# references under the lock, copy after release. Writers only
|
||||
@@ -133,11 +134,14 @@ class _FakeUI:
|
||||
# fake here keeps test-vs-prod behavioural drift from
|
||||
# masking a real-shape regression.
|
||||
primary = next((cid for cid in call_ids if cid), "")
|
||||
return {
|
||||
"call_id": primary,
|
||||
"judge_pending": bool(pending.get("judge_pending", False)),
|
||||
"items": serialized,
|
||||
}
|
||||
return [
|
||||
{
|
||||
"cycle_id": pending.get("cycle_id", ""),
|
||||
"call_id": primary,
|
||||
"judge_pending": bool(pending.get("judge_pending", False)),
|
||||
"items": serialized,
|
||||
}
|
||||
]
|
||||
|
||||
def serialize_recent_auto_approvals(self) -> list[dict[str, Any]]:
|
||||
# Empty buffer for tests that don't exercise the auto-approve
|
||||
@@ -671,28 +675,35 @@ class TestDashboardTrustedTeamVisibility:
|
||||
owners = {w["user_id"] for w in data["workstreams"]}
|
||||
assert {"user-a", "user-b"}.issubset(owners)
|
||||
|
||||
def test_dashboard_pending_approval_detail_default_none(self, app_client):
|
||||
"""No pending approval → field is explicitly null on the wire so
|
||||
consumers can distinguish "not present" from "absent key"."""
|
||||
def test_dashboard_pending_approval_details_default_empty(self, app_client):
|
||||
"""No pending approval → the list field is explicitly empty on
|
||||
the wire so consumers can distinguish "nothing pending" from
|
||||
"absent key". Replaces 1.6's singular ``pending_approval_detail``
|
||||
null (breaking, 1.7)."""
|
||||
client, _mgr = app_client
|
||||
client.post("/v1/api/workstreams/new", json={"name": "a"}, headers=_auth("user-a"))
|
||||
resp = client.get("/v1/api/dashboard", headers=_auth("user-a"))
|
||||
assert resp.status_code == 200
|
||||
rows = resp.json()["workstreams"]
|
||||
assert len(rows) == 1
|
||||
assert "pending_approval_detail" in rows[0]
|
||||
assert rows[0]["pending_approval_detail"] is None
|
||||
assert "pending_approval_details" in rows[0]
|
||||
assert rows[0]["pending_approval_details"] == []
|
||||
# The 1.6 singular field is GONE, not null — a consumer still
|
||||
# reading it should break loudly, not read None forever.
|
||||
assert "pending_approval_detail" not in rows[0]
|
||||
|
||||
def test_dashboard_pending_approval_detail_merges_judge_verdict(self, app_client):
|
||||
def test_dashboard_pending_approval_details_merge_judge_verdict(self, app_client):
|
||||
"""When _pending_approval is set on a ws's UI, /dashboard
|
||||
embeds the merged items + judge_verdict so coord live-bulk
|
||||
callers can render inline approve/deny buttons."""
|
||||
embeds one detail entry per live cycle with merged items +
|
||||
judge_verdict so coord live-bulk callers can render inline
|
||||
approve/deny buttons."""
|
||||
client, mgr = app_client
|
||||
client.post("/v1/api/workstreams/new", json={"name": "a"}, headers=_auth("user-a"))
|
||||
ws_id = next(iter(mgr.list_all())).id
|
||||
ui = mgr.get(ws_id).ui
|
||||
ui._pending_approval = {
|
||||
"type": "approve_request",
|
||||
"cycle_id": "cyc-1",
|
||||
"items": [
|
||||
{
|
||||
"call_id": "c-1",
|
||||
@@ -714,8 +725,10 @@ class TestDashboardTrustedTeamVisibility:
|
||||
resp = client.get("/v1/api/dashboard", headers=_auth("user-a"))
|
||||
assert resp.status_code == 200
|
||||
row = next(w for w in resp.json()["workstreams"] if w["ws_id"] == ws_id)
|
||||
detail = row["pending_approval_detail"]
|
||||
assert detail is not None
|
||||
details = row["pending_approval_details"]
|
||||
assert len(details) == 1
|
||||
detail = details[0]
|
||||
assert detail["cycle_id"] == "cyc-1"
|
||||
assert detail["call_id"] == "c-1"
|
||||
assert detail["judge_pending"] is False
|
||||
item = detail["items"][0]
|
||||
|
||||
+41
-17
@@ -208,7 +208,9 @@ class TestMergeServerCompat:
|
||||
|
||||
|
||||
class TestEndToEndRequestShaping:
|
||||
"""Compose both layers — session builds extra_params, provider applies thinking."""
|
||||
"""Compose both layers — session builds extra_params, provider applies reasoning."""
|
||||
|
||||
provider = OpenAIChatCompletionsProvider()
|
||||
|
||||
def test_vllm_gemma_full_flow(self) -> None:
|
||||
"""Gemma now needs only the thinking param — no server workaround."""
|
||||
@@ -216,18 +218,27 @@ class TestEndToEndRequestShaping:
|
||||
server_compat = {"server_type": "vllm"}
|
||||
# Step 1: session forwards (no auto-injection of reasoning_effort).
|
||||
extra_params = merge_server_compat(None, server_compat)
|
||||
# Step 2: provider injects thinking param into chat_template_kwargs.
|
||||
extra_body = dict(extra_params)
|
||||
OpenAIChatCompletionsProvider._apply_thinking_mode(extra_body, caps)
|
||||
# Step 2: provider folds the effort knob into chat_template_kwargs.
|
||||
extra_body = self.provider._finalize_extra_body(extra_params, caps, "medium")
|
||||
|
||||
assert extra_body == {"chat_template_kwargs": {"enable_thinking": True}}
|
||||
|
||||
def test_knob_none_disables_toggle(self) -> None:
|
||||
"""Session effort "none" turns the template toggle off dynamically."""
|
||||
caps = ModelCapabilities(thinking_mode="manual", thinking_param="enable_thinking")
|
||||
extra_body = self.provider._finalize_extra_body(
|
||||
merge_server_compat(None, {"server_type": "vllm"}), caps, "none"
|
||||
)
|
||||
|
||||
assert extra_body == {"chat_template_kwargs": {"enable_thinking": False}}
|
||||
|
||||
def test_server_workaround_composes_with_thinking(self) -> None:
|
||||
"""A top-level server workaround forwards alongside the injected thinking param."""
|
||||
caps = ModelCapabilities(thinking_mode="manual", thinking_param="enable_thinking")
|
||||
compat = {"server_type": "llama.cpp", "extra_body": {"reasoning_format": "auto"}}
|
||||
extra_body = dict(merge_server_compat(None, compat))
|
||||
OpenAIChatCompletionsProvider._apply_thinking_mode(extra_body, caps)
|
||||
extra_body = self.provider._finalize_extra_body(
|
||||
merge_server_compat(None, compat), caps, "medium"
|
||||
)
|
||||
|
||||
assert extra_body == {
|
||||
"chat_template_kwargs": {"enable_thinking": True},
|
||||
@@ -237,20 +248,20 @@ class TestEndToEndRequestShaping:
|
||||
def test_granite_thinking_key(self) -> None:
|
||||
"""Granite uses 'thinking' instead of 'enable_thinking'."""
|
||||
caps = ModelCapabilities(thinking_mode="manual", thinking_param="thinking")
|
||||
extra_params = merge_server_compat(None, {})
|
||||
extra_body = dict(extra_params)
|
||||
OpenAIChatCompletionsProvider._apply_thinking_mode(extra_body, caps)
|
||||
extra_body = self.provider._finalize_extra_body(
|
||||
merge_server_compat(None, {}), caps, "medium"
|
||||
)
|
||||
|
||||
assert extra_body == {"chat_template_kwargs": {"thinking": True}}
|
||||
|
||||
def test_non_thinking_model_no_injection(self) -> None:
|
||||
"""Non-thinking model gets no chat_template_kwargs at all."""
|
||||
"""Non-thinking model gets no extra_body at all."""
|
||||
caps = ModelCapabilities() # thinking_mode="none"
|
||||
extra_params = merge_server_compat(None, {})
|
||||
extra_body = dict(extra_params)
|
||||
OpenAIChatCompletionsProvider._apply_thinking_mode(extra_body, caps)
|
||||
extra_body = self.provider._finalize_extra_body(
|
||||
merge_server_compat(None, {}), caps, "medium"
|
||||
)
|
||||
|
||||
assert extra_body == {}
|
||||
assert extra_body is None
|
||||
|
||||
def test_operator_reasoning_effort_passthrough(self) -> None:
|
||||
"""Operator-supplied reasoning_effort under chat_template_kwargs is preserved."""
|
||||
@@ -259,9 +270,9 @@ class TestEndToEndRequestShaping:
|
||||
"server_type": "vllm",
|
||||
"extra_body": {"chat_template_kwargs": {"reasoning_effort": "high"}},
|
||||
}
|
||||
extra_params = merge_server_compat(None, compat)
|
||||
extra_body = dict(extra_params)
|
||||
OpenAIChatCompletionsProvider._apply_thinking_mode(extra_body, caps)
|
||||
extra_body = self.provider._finalize_extra_body(
|
||||
merge_server_compat(None, compat), caps, "medium"
|
||||
)
|
||||
|
||||
assert extra_body == {
|
||||
"chat_template_kwargs": {
|
||||
@@ -270,6 +281,19 @@ class TestEndToEndRequestShaping:
|
||||
},
|
||||
}
|
||||
|
||||
def test_operator_pin_beats_effort_param(self) -> None:
|
||||
"""A pinned effort key wins over the knob mapping (setdefault)."""
|
||||
caps = ModelCapabilities(
|
||||
thinking_mode="none",
|
||||
effort_param="reasoning_effort",
|
||||
)
|
||||
compat = {"extra_body": {"chat_template_kwargs": {"reasoning_effort": "low"}}}
|
||||
extra_body = self.provider._finalize_extra_body(
|
||||
merge_server_compat(None, compat), caps, "high"
|
||||
)
|
||||
|
||||
assert extra_body == {"chat_template_kwargs": {"reasoning_effort": "low"}}
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Probe integration: suggest_profile called from _detect_openai_compat
|
||||
|
||||
+395
-51
@@ -272,18 +272,19 @@ class TestChatSessionConstruction:
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Tests — _exec_task (optional skill substitutes the hardcoded identity)
|
||||
# Tests — _exec_task (identity from persona/default; skill = capability turn)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestTaskExec:
|
||||
"""Tests for _exec_task: optional skill= replaces the default persona,
|
||||
but operating guidance (one-shot, tool-use over narration, no follow-ups)
|
||||
is always preserved."""
|
||||
"""Tests for _exec_task: identity comes from ``persona=`` (or the default
|
||||
task-agent identity), NEVER the skill; a ``skill=`` rides a distinct
|
||||
capability turn. Operating guidance (one-shot, tool-use over narration,
|
||||
no follow-ups) always layers on top."""
|
||||
|
||||
@staticmethod
|
||||
def _capture_exec_messages(session, item):
|
||||
"""Run _exec_task with _run_agent patched; return system message text."""
|
||||
def _capture_exec_turns(session, item):
|
||||
"""Run _exec_task with _run_agent patched; return the turns list."""
|
||||
captured: dict = {}
|
||||
|
||||
def fake_run_agent(messages, **kwargs):
|
||||
@@ -292,27 +293,22 @@ class TestTaskExec:
|
||||
|
||||
with patch.object(session, "_run_agent", side_effect=fake_run_agent):
|
||||
session._exec_task(item)
|
||||
return captured["messages"][0].text
|
||||
return captured["messages"]
|
||||
|
||||
def test_known_skill_renders_into_system_message(self, tmp_db) -> None:
|
||||
"""Validated skill content (with template vars resolved) replaces
|
||||
the default '# Task Agent' persona, but the operating guidance
|
||||
(the numbered list) is preserved — those are sub-agent semantics
|
||||
that a persona should layer on top of, not replace.
|
||||
|
||||
Covers the full prepare→exec round-trip so a future regression
|
||||
in either half (skill not stored on the item, or exec ignoring it)
|
||||
is caught."""
|
||||
def test_skill_delivered_as_capability_turn_not_identity(self, tmp_db) -> None:
|
||||
"""A skill= is CAPABILITY, not identity: its body (template vars
|
||||
resolved) rides a distinct turn AFTER the system message, while the
|
||||
default '# Task Agent' identity + operating guidance stay in the
|
||||
system message. Covers the full prepare→exec round-trip."""
|
||||
session = _make_session()
|
||||
skill = {
|
||||
"name": "research",
|
||||
"content": "# Research Agent\nws={{ws_id}} model={{model}} node={{node_id}}",
|
||||
"content": "# Research Skill\nws={{ws_id}} model={{model}} node={{node_id}}",
|
||||
}
|
||||
with patch("turnstone.core.session.get_skill_by_name", return_value=skill):
|
||||
item = session._prepare_task("c1", {"prompt": "investigate X", "skill": "research"})
|
||||
|
||||
# Item carries the minimized projection — name/content/risk_level
|
||||
# only — not the raw prompt_templates row.
|
||||
# Item carries the minimized projection — name/content/risk_level only.
|
||||
assert item["skill"] == {
|
||||
"name": "research",
|
||||
"content": skill["content"],
|
||||
@@ -321,35 +317,302 @@ class TestTaskExec:
|
||||
assert item.get("needs_approval") is True
|
||||
assert "skill: research" in item["header"]
|
||||
|
||||
sys_msg = self._capture_exec_messages(session, item)
|
||||
# Skill persona rendered with template vars resolved
|
||||
assert "# Research Agent" in sys_msg
|
||||
assert f"ws={session._ws_id}" in sys_msg
|
||||
assert f"model={session.model}" in sys_msg
|
||||
# Default persona is gone — skill substitutes for it.
|
||||
assert "# Task Agent" not in sys_msg
|
||||
assert "autonomous task agent with full tool access" not in sys_msg
|
||||
# Operating guidance survives regardless of skill.
|
||||
turns = self._capture_exec_turns(session, item)
|
||||
sys_msg = turns[0].text
|
||||
# Identity stays the DEFAULT — the skill does NOT become identity.
|
||||
assert ChatSession._TASK_DEFAULT_IDENTITY in sys_msg
|
||||
assert "# Task Agent" in sys_msg
|
||||
assert ChatSession._TASK_OPERATING_GUIDANCE in sys_msg
|
||||
# Skill body is NOT fused into the identity system message.
|
||||
assert "# Research Skill" not in sys_msg
|
||||
# It rides a distinct capability turn, template vars resolved.
|
||||
capability = turns[1].text
|
||||
assert ChatSession._TASK_SKILL_CAPABILITY_PREAMBLE in capability
|
||||
assert "# Research Skill" in capability
|
||||
assert f"ws={session._ws_id}" in capability
|
||||
assert f"model={session.model}" in capability
|
||||
# Task prompt is the final turn.
|
||||
assert turns[-1].text == "investigate X"
|
||||
|
||||
def test_omitted_skill_uses_hardcoded_identity(self, tmp_db) -> None:
|
||||
"""Regression guard: without skill=, the default '# Task Agent'
|
||||
persona AND the operating guidance both appear verbatim.
|
||||
|
||||
Pins the no-skill path so the substitution branch can't
|
||||
accidentally swallow the default case."""
|
||||
def test_omitted_skill_uses_default_identity(self, tmp_db) -> None:
|
||||
"""Without skill= or persona=, the default '# Task Agent' identity +
|
||||
operating guidance appear in the system message, and there is NO
|
||||
capability turn — just system + prompt."""
|
||||
session = _make_session()
|
||||
item = session._prepare_task("c1", {"prompt": "do x"})
|
||||
|
||||
assert item["skill"] is None
|
||||
assert item["persona"] == ""
|
||||
assert "skill:" not in item["header"]
|
||||
assert "persona:" not in item["header"]
|
||||
|
||||
sys_msg = self._capture_exec_messages(session, item)
|
||||
turns = self._capture_exec_turns(session, item)
|
||||
sys_msg = turns[0].text
|
||||
assert ChatSession._TASK_DEFAULT_IDENTITY in sys_msg
|
||||
assert ChatSession._TASK_OPERATING_GUIDANCE in sys_msg
|
||||
# Default-persona literals also present (sanity check on the constant).
|
||||
assert "# Task Agent" in sys_msg
|
||||
assert "autonomous task agent with full tool access" in sys_msg
|
||||
# No skill → no capability turn: just system + prompt.
|
||||
assert len(turns) == 2
|
||||
assert turns[-1].text == "do x"
|
||||
|
||||
def test_persona_sets_identity_skill_stays_capability(self, tmp_db) -> None:
|
||||
"""persona= sets the sub-agent identity (base prompt) in place of the
|
||||
default; a skill passed alongside stays a capability turn."""
|
||||
session = _make_session()
|
||||
persona_row = {
|
||||
"name": "engineer",
|
||||
"base_prompt": "# Engineer\nYou are an engineer.",
|
||||
"base_prompt_file": None,
|
||||
"tool_allowlist": None,
|
||||
"mcp_enabled": True,
|
||||
"memory_enabled": True,
|
||||
"enabled": True,
|
||||
"applies_to_kinds": ["interactive"],
|
||||
}
|
||||
skill = {"name": "research", "content": "# Research Skill"}
|
||||
with (
|
||||
patch("turnstone.core.session.get_skill_by_name", return_value=skill),
|
||||
patch("turnstone.core.session.get_storage") as gs,
|
||||
):
|
||||
gs.return_value.get_persona_by_name.return_value = persona_row
|
||||
item = session._prepare_task(
|
||||
"c1", {"prompt": "do x", "skill": "research", "persona": "engineer"}
|
||||
)
|
||||
|
||||
assert item.get("needs_approval") is True
|
||||
assert item["persona"] == "engineer"
|
||||
assert "persona: engineer" in item["header"]
|
||||
assert "skill: research" in item["header"]
|
||||
|
||||
turns = self._capture_exec_turns(session, item)
|
||||
sys_msg = turns[0].text
|
||||
# Identity = persona, not the default and not the skill.
|
||||
assert "# Engineer" in sys_msg
|
||||
assert ChatSession._TASK_DEFAULT_IDENTITY not in sys_msg
|
||||
assert "# Research Skill" not in sys_msg
|
||||
# Operating guidance still layers on the persona identity.
|
||||
assert ChatSession._TASK_OPERATING_GUIDANCE in sys_msg
|
||||
# Skill remains a capability turn.
|
||||
assert "# Research Skill" in turns[1].text
|
||||
|
||||
def test_unknown_persona_returns_error(self, tmp_db) -> None:
|
||||
"""Unknown persona name → clean error item, no approval."""
|
||||
session = _make_session()
|
||||
with patch("turnstone.core.session.get_storage") as gs:
|
||||
gs.return_value.get_persona_by_name.return_value = None
|
||||
item = session._prepare_task("c1", {"prompt": "do x", "persona": "ghost"})
|
||||
assert item.get("needs_approval") is False
|
||||
assert "ghost" in item["error"]
|
||||
assert "Omit `persona`" in item["error"]
|
||||
|
||||
def test_persona_wrong_kind_returns_error(self, tmp_db) -> None:
|
||||
"""A coordinator-only persona can't serve as a task-agent identity."""
|
||||
session = _make_session()
|
||||
coord_row = {
|
||||
"name": "orchestrator",
|
||||
"base_prompt": "# Orchestrator",
|
||||
"base_prompt_file": None,
|
||||
"tool_allowlist": None,
|
||||
"mcp_enabled": True,
|
||||
"memory_enabled": True,
|
||||
"enabled": True,
|
||||
"applies_to_kinds": ["coordinator"],
|
||||
}
|
||||
with patch("turnstone.core.session.get_storage") as gs:
|
||||
gs.return_value.get_persona_by_name.return_value = coord_row
|
||||
item = session._prepare_task("c1", {"prompt": "do x", "persona": "orchestrator"})
|
||||
assert item.get("needs_approval") is False
|
||||
assert "interactive" in item["error"]
|
||||
|
||||
def test_persona_tool_allowlist_restricts_sub_agent_tools(self, tmp_db) -> None:
|
||||
"""A restrictive persona caps the sub-agent's TOOLS (Principle 7 /
|
||||
review fix), not just its identity text — stated identity must match
|
||||
granted authority."""
|
||||
session = _make_session()
|
||||
session._task_tools = [
|
||||
{"function": {"name": "read_file"}},
|
||||
{"function": {"name": "write_file"}},
|
||||
{"function": {"name": "bash"}},
|
||||
]
|
||||
persona_row = {
|
||||
"name": "readonly",
|
||||
"base_prompt": "# Readonly reviewer",
|
||||
"base_prompt_file": None,
|
||||
"tool_allowlist": ["read_file", "search"], # excludes write_file/bash
|
||||
"mcp_enabled": True,
|
||||
"memory_enabled": True,
|
||||
"enabled": True,
|
||||
"applies_to_kinds": ["interactive"],
|
||||
}
|
||||
with patch("turnstone.core.session.get_storage") as gs:
|
||||
gs.return_value.get_persona_by_name.return_value = persona_row
|
||||
item = session._prepare_task("c1", {"prompt": "edit auth", "persona": "readonly"})
|
||||
assert item["persona_tools"] == frozenset({"read_file", "search"})
|
||||
|
||||
captured: dict = {}
|
||||
|
||||
def fake_run_agent(messages, **kwargs):
|
||||
captured.update(kwargs)
|
||||
return "done"
|
||||
|
||||
with patch.object(session, "_run_agent", side_effect=fake_run_agent):
|
||||
session._exec_task(item)
|
||||
tool_names = {t["function"]["name"] for t in captured["tools"]}
|
||||
# write_file + bash dropped by the persona; read_file kept (search was
|
||||
# never in the task tool set to begin with).
|
||||
assert tool_names == {"read_file"}
|
||||
|
||||
def test_no_persona_keeps_full_task_tools(self, tmp_db) -> None:
|
||||
session = _make_session()
|
||||
session._task_tools = [
|
||||
{"function": {"name": "read_file"}},
|
||||
{"function": {"name": "bash"}},
|
||||
]
|
||||
item = session._prepare_task("c1", {"prompt": "do x"})
|
||||
assert item["persona_tools"] is None
|
||||
|
||||
captured: dict = {}
|
||||
|
||||
def fake_run_agent(messages, **kwargs):
|
||||
captured.update(kwargs)
|
||||
return "done"
|
||||
|
||||
with patch.object(session, "_run_agent", side_effect=fake_run_agent):
|
||||
session._exec_task(item)
|
||||
assert {t["function"]["name"] for t in captured["tools"]} == {"read_file", "bash"}
|
||||
|
||||
def test_parent_persona_caps_sub_agent_tools(self, tmp_db) -> None:
|
||||
"""A restricted PARENT session must not escalate authority by spawning:
|
||||
the sub-agent's tools are capped by the parent's own persona grant even
|
||||
with NO child persona (Principle 7 — delegation narrows, never widens;
|
||||
whole-PR review fix)."""
|
||||
session = _make_session()
|
||||
session._tool_search = None
|
||||
session._task_tools = [
|
||||
{"function": {"name": "read_file"}},
|
||||
{"function": {"name": "write_file"}},
|
||||
{"function": {"name": "bash"}},
|
||||
]
|
||||
# Parent runs under a read-only persona.
|
||||
session._persona_tools = frozenset({"read_file", "search"})
|
||||
|
||||
item = session._prepare_task("c1", {"prompt": "edit auth"})
|
||||
assert item["persona_tools"] is None # no CHILD persona
|
||||
|
||||
captured: dict = {}
|
||||
|
||||
def fake_run_agent(messages, **kwargs):
|
||||
captured.update(kwargs)
|
||||
return "done"
|
||||
|
||||
with patch.object(session, "_run_agent", side_effect=fake_run_agent):
|
||||
session._exec_task(item)
|
||||
# Parent's read-only grant caps the sub-agent: write_file + bash dropped.
|
||||
assert {t["function"]["name"] for t in captured["tools"]} == {"read_file"}
|
||||
|
||||
def test_child_persona_mcp_off_drops_mcp_tools(self, tmp_db) -> None:
|
||||
"""A child persona with mcp_enabled=False hides MCP tools (``mcp__*`` and
|
||||
the MCP-access read_resource / use_prompt) from the sub-agent, even when
|
||||
tool_allowlist is null (unrestricted native tools) — the mcp lever must
|
||||
not silently no-op on the task_agent path (whole-PR review fix)."""
|
||||
session = _make_session()
|
||||
session._tool_search = None
|
||||
session._task_tools = [
|
||||
{"function": {"name": "read_file"}},
|
||||
{"function": {"name": "read_resource"}},
|
||||
{"function": {"name": "use_prompt"}},
|
||||
{"function": {"name": "mcp__github__search"}},
|
||||
]
|
||||
persona_row = {
|
||||
"name": "sandboxed",
|
||||
"base_prompt": "# Sandboxed",
|
||||
"base_prompt_file": None,
|
||||
"tool_allowlist": None, # null = unrestricted native tools
|
||||
"mcp_enabled": False, # but MCP is OFF
|
||||
"memory_enabled": True,
|
||||
"enabled": True,
|
||||
"applies_to_kinds": ["interactive"],
|
||||
}
|
||||
with patch("turnstone.core.session.get_storage") as gs:
|
||||
gs.return_value.get_persona_by_name.return_value = persona_row
|
||||
item = session._prepare_task("c1", {"prompt": "do x", "persona": "sandboxed"})
|
||||
assert item["persona_mcp"] is False
|
||||
assert item["persona_tools"] is None
|
||||
|
||||
captured: dict = {}
|
||||
|
||||
def fake_run_agent(messages, **kwargs):
|
||||
captured.update(kwargs)
|
||||
return "done"
|
||||
|
||||
with patch.object(session, "_run_agent", side_effect=fake_run_agent):
|
||||
session._exec_task(item)
|
||||
# MCP tools shed; native read_file kept.
|
||||
assert {t["function"]["name"] for t in captured["tools"]} == {"read_file"}
|
||||
|
||||
def test_child_persona_memory_off_drops_memory_tool(self, tmp_db) -> None:
|
||||
"""A child persona with memory_enabled=False drops the memory tool from
|
||||
the sub-agent's hands (lever 4), matching a main session under the same
|
||||
persona (whole-PR review fix)."""
|
||||
session = _make_session()
|
||||
session._tool_search = None
|
||||
session._task_tools = [
|
||||
{"function": {"name": "read_file"}},
|
||||
{"function": {"name": "memory"}},
|
||||
]
|
||||
persona_row = {
|
||||
"name": "nomem",
|
||||
"base_prompt": "# No memory",
|
||||
"base_prompt_file": None,
|
||||
"tool_allowlist": None,
|
||||
"mcp_enabled": True,
|
||||
"memory_enabled": False,
|
||||
"enabled": True,
|
||||
"applies_to_kinds": ["interactive"],
|
||||
}
|
||||
with patch("turnstone.core.session.get_storage") as gs:
|
||||
gs.return_value.get_persona_by_name.return_value = persona_row
|
||||
item = session._prepare_task("c1", {"prompt": "do x", "persona": "nomem"})
|
||||
assert item["persona_memory"] is False
|
||||
|
||||
captured: dict = {}
|
||||
|
||||
def fake_run_agent(messages, **kwargs):
|
||||
captured.update(kwargs)
|
||||
return "done"
|
||||
|
||||
with patch.object(session, "_run_agent", side_effect=fake_run_agent):
|
||||
session._exec_task(item)
|
||||
assert {t["function"]["name"] for t in captured["tools"]} == {"read_file"}
|
||||
|
||||
def test_evaluate_intent_projects_persona_for_task_agent(self, tmp_db, monkeypatch) -> None:
|
||||
"""Judge/audit projection includes the persona name (review fix): a
|
||||
persona-driven identity shift must be visible to policy + audit, like
|
||||
spawn_workstream."""
|
||||
session = _make_session()
|
||||
fake_verdict = MagicMock()
|
||||
fake_verdict.to_dict.return_value = {"verdict_id": "v0", "tier": "heuristic"}
|
||||
fake_judge = MagicMock()
|
||||
fake_judge.evaluate.side_effect = lambda items, *_a, **_kw: [fake_verdict] * len(items)
|
||||
fake_judge.arg_budget_chars.return_value = 200_000
|
||||
monkeypatch.setattr(session, "_ensure_judge", lambda: fake_judge)
|
||||
|
||||
persona_row = {
|
||||
"name": "engineer",
|
||||
"base_prompt": "# Engineer",
|
||||
"base_prompt_file": None,
|
||||
"tool_allowlist": None,
|
||||
"mcp_enabled": True,
|
||||
"memory_enabled": True,
|
||||
"enabled": True,
|
||||
"applies_to_kinds": ["interactive"],
|
||||
}
|
||||
with patch("turnstone.core.session.get_storage") as gs:
|
||||
gs.return_value.get_persona_by_name.return_value = persona_row
|
||||
item = session._prepare_task("c1", {"prompt": "do x", "persona": "engineer"})
|
||||
session._evaluate_intent([item])
|
||||
assert item["func_args"]["persona"] == "engineer"
|
||||
|
||||
@pytest.mark.parametrize("skill_value", ["", " ", "\t\n"])
|
||||
def test_prepare_task_empty_or_whitespace_skill_treated_as_omitted(
|
||||
@@ -397,12 +660,11 @@ class TestTaskExec:
|
||||
# tell them apart at recovery time.
|
||||
assert "unknown skill" not in item["error"]
|
||||
|
||||
def test_prepare_task_high_risk_skill_surfaces_in_header(self, tmp_db, caplog) -> None:
|
||||
"""High/critical risk skills surface the tier in the approval header
|
||||
and emit a structured warning, mirroring the signal ``_load_skills``
|
||||
emits for session-level skills (session.py:1336)."""
|
||||
import logging
|
||||
|
||||
def test_prepare_task_denies_high_risk_skill(self, tmp_db) -> None:
|
||||
"""High/critical-risk skills are PRINCIPAL-load-only: task_agent(skill=…)
|
||||
DENIES them — the same gate skills(load) / spawn_* enforce, so a model
|
||||
cannot route around it by delegating activation to a sub-agent
|
||||
(whole-PR review fix — task_agent was the un-gated surface)."""
|
||||
session = _make_session()
|
||||
risky_skill = {
|
||||
"name": "danger",
|
||||
@@ -410,16 +672,15 @@ class TestTaskExec:
|
||||
"enabled": True,
|
||||
"risk_level": "critical",
|
||||
}
|
||||
with (
|
||||
caplog.at_level(logging.WARNING, logger="turnstone.core.session"),
|
||||
patch("turnstone.core.session.get_skill_by_name", return_value=risky_skill),
|
||||
):
|
||||
with patch("turnstone.core.session.get_skill_by_name", return_value=risky_skill):
|
||||
item = session._prepare_task("c1", {"prompt": "do x", "skill": "danger"})
|
||||
assert item.get("needs_approval") is True
|
||||
assert "skill: danger" in item["header"]
|
||||
assert "risk: critical" in item["header"]
|
||||
warning_seen = any("high_risk_skill" in r.getMessage() for r in caplog.records)
|
||||
assert warning_seen, "expected task_agent.high_risk_skill warning"
|
||||
assert item.get("needs_approval") is False
|
||||
assert "principal-load-only" in item["header"]
|
||||
assert "/skill danger" in item["error"]
|
||||
# Distinct from the unknown/disabled errors so the model's recovery
|
||||
# path can tell them apart.
|
||||
assert "unknown skill" not in item["error"]
|
||||
assert "disabled" not in item["error"]
|
||||
|
||||
def test_prepare_task_normal_risk_skill_omits_tier_from_header(self, tmp_db) -> None:
|
||||
"""Header only surfaces high/critical — low/medium/safe skills don't
|
||||
@@ -537,6 +798,89 @@ class TestTaskExec:
|
||||
captured[0](fake_verdict) # must not raise
|
||||
session.ui.on_intent_verdict.assert_not_called()
|
||||
|
||||
def test_evaluate_intent_agent_gate_owns_generation_off_the_main_slot(
|
||||
self, tmp_db, monkeypatch
|
||||
) -> None:
|
||||
"""Sub-agent gates run the SAME judge pipeline as the main loop
|
||||
but as their OWN generation (release blocker #1: task_agent
|
||||
calls used to reach the gate judge-blind). The main-loop
|
||||
supersede slot stays untouched — with parallel task agents,
|
||||
publishing into it would make every sibling's verdicts look
|
||||
stale to the previous sibling's callback — while the generation
|
||||
is stamped on the items for the UI's origin checks, registered
|
||||
for ``close()``'s sweep, delivered alongside the verdict, and
|
||||
grounded on the SUB-AGENT's trajectory (its task prompt is the
|
||||
delegation contract), not the parent conversation."""
|
||||
import threading
|
||||
|
||||
from turnstone.core.session_ui_base import SessionUIBase
|
||||
from turnstone.core.trajectory import turns_from_dicts
|
||||
|
||||
class _GateUI(SessionUIBase):
|
||||
pass
|
||||
|
||||
session = _make_session()
|
||||
ui = _GateUI(ws_id="ws-gate", user_id="u1")
|
||||
ui.on_intent_verdict = MagicMock() # shadow: capture delivery kwargs
|
||||
session.ui = ui
|
||||
|
||||
captured: dict[str, Any] = {}
|
||||
fake_verdict = MagicMock()
|
||||
fake_verdict.to_dict.return_value = {"verdict_id": "v0", "call_id": "c1", "tier": "llm"}
|
||||
fake_judge = MagicMock()
|
||||
|
||||
def _eval(items, convo, **kw):
|
||||
captured["convo"] = convo
|
||||
captured["callback"] = kw.get("callback")
|
||||
captured["cancel_event"] = kw.get("cancel_event")
|
||||
captured["done"] = kw.get("done_callback")
|
||||
return [fake_verdict] * len(items)
|
||||
|
||||
fake_judge.evaluate.side_effect = _eval
|
||||
monkeypatch.setattr(session, "_ensure_judge", lambda: fake_judge)
|
||||
|
||||
main_slot = threading.Event()
|
||||
session._judge_cancel_event = main_slot
|
||||
agent_turns = turns_from_dicts([{"role": "user", "content": "Task: reindex the docs tree"}])
|
||||
item = {"call_id": "c1", "func_name": "bash", "needs_approval": True, "command": "ls"}
|
||||
|
||||
ev = session._evaluate_intent([item], conversation=agent_turns, agent_gate=True)
|
||||
|
||||
assert ev is not None and ev is not main_slot
|
||||
# Main-loop slot untouched by the sub-agent spawn.
|
||||
assert session._judge_cancel_event is main_slot
|
||||
# Generation stamped for the UI's origin checks + close() sweep,
|
||||
# and handed to the daemon as its cancel event.
|
||||
assert item["_judge_event"] is ev
|
||||
assert ev in session._judge_cancel_events
|
||||
assert captured["cancel_event"] is ev
|
||||
# Judge grounded on the sub-agent trajectory, not session.messages.
|
||||
assert any("reindex the docs tree" in str(m) for m in captured["convo"])
|
||||
# Delivery rides the generation into the UI.
|
||||
captured["callback"](fake_verdict)
|
||||
assert ui.on_intent_verdict.call_args.kwargs.get("judge_event") is ev
|
||||
# Daemon completion keeps the close()-sweep set exact.
|
||||
captured["done"]()
|
||||
assert ev not in session._judge_cancel_events
|
||||
|
||||
def test_close_fires_agent_gate_judge_generations(self, tmp_db, monkeypatch) -> None:
|
||||
"""``close()`` aborts EVERY in-flight judge daemon — including
|
||||
sub-agent generations that never touched the main slot — so a
|
||||
torn-down session can't leave daemons running against a dead
|
||||
UI."""
|
||||
session = _make_session()
|
||||
fake_verdict = MagicMock()
|
||||
fake_verdict.to_dict.return_value = {"verdict_id": "v0", "call_id": "c1", "tier": "llm"}
|
||||
fake_judge = MagicMock()
|
||||
fake_judge.evaluate.side_effect = lambda items, *_a, **_kw: [fake_verdict] * len(items)
|
||||
monkeypatch.setattr(session, "_ensure_judge", lambda: fake_judge)
|
||||
|
||||
item = {"call_id": "c1", "func_name": "bash", "needs_approval": True, "command": "ls"}
|
||||
ev = session._evaluate_intent([item], conversation=[], agent_gate=True)
|
||||
assert ev is not None and not ev.is_set()
|
||||
session.close()
|
||||
assert ev.is_set()
|
||||
|
||||
def _drive_gate(self, session, monkeypatch, *, cancel_on_approval: bool):
|
||||
"""Run one needs_approval bash item through ``_execute_tools`` with a
|
||||
stubbed judge + approval gate; return the cancel event the judge
|
||||
|
||||
+732
-180
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,226 @@
|
||||
"""Tests for turnstone.eval skill-adherence measurement mode.
|
||||
|
||||
Two levels, neither requires a live model:
|
||||
|
||||
* ``TestSkillComposition`` is the load-bearing plumbing proof — it seeds a
|
||||
named skill, builds ``HeadlessSession`` under natural composition, and
|
||||
asserts the skill body folds into ``system_messages`` for the treatment
|
||||
arm and is absent for the control arm. This is what makes the two arms
|
||||
measure different things.
|
||||
* ``TestAdherenceLift`` unit-tests ``run_skill_adherence``'s lift math with
|
||||
the per-arm runner stubbed out.
|
||||
"""
|
||||
|
||||
import os
|
||||
import tempfile
|
||||
from collections.abc import Iterator
|
||||
from typing import Any
|
||||
|
||||
import pytest
|
||||
from openai import OpenAI
|
||||
|
||||
from turnstone.core.storage import get_storage, init_storage, reset_storage
|
||||
from turnstone.eval import core
|
||||
from turnstone.eval.core import HeadlessSession, run_skill_adherence
|
||||
|
||||
_SKILL = {
|
||||
"name": "search-first",
|
||||
"content": (
|
||||
"# Search First\n\nBefore answering ANY question about where something "
|
||||
"lives in the codebase, you MUST call the `search` tool first. "
|
||||
"SENTINEL_SKILL_BODY_MARKER."
|
||||
),
|
||||
}
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def temp_storage() -> Iterator[None]:
|
||||
"""Fresh sqlite storage in a temp dir, torn down after the test."""
|
||||
workdir = tempfile.mkdtemp(prefix="turnstone_skill_test_")
|
||||
reset_storage()
|
||||
init_storage("sqlite", path=os.path.join(workdir, ".eval.db"), run_migrations=False)
|
||||
try:
|
||||
yield
|
||||
finally:
|
||||
reset_storage()
|
||||
import shutil
|
||||
|
||||
shutil.rmtree(workdir, ignore_errors=True)
|
||||
|
||||
|
||||
def _seed_skill(skill: dict[str, str]) -> None:
|
||||
"""Seed a named skill exactly as the runner does."""
|
||||
get_storage().create_prompt_template(
|
||||
template_id="eval-skill",
|
||||
name=skill["name"],
|
||||
category="eval",
|
||||
content=skill["content"],
|
||||
variables="[]",
|
||||
is_default=False,
|
||||
org_id="",
|
||||
created_by="eval",
|
||||
activation="named",
|
||||
enabled=True,
|
||||
)
|
||||
|
||||
|
||||
def _system_text(session: HeadlessSession) -> str:
|
||||
return "\n".join(m["content"] for m in session.system_messages)
|
||||
|
||||
|
||||
class TestSkillComposition:
|
||||
"""Prove the treatment/control arms compose different system messages."""
|
||||
|
||||
def test_treatment_folds_skill_into_system(self, temp_storage: None) -> None:
|
||||
_seed_skill(_SKILL)
|
||||
client = OpenAI(base_url="http://localhost:9/v1", api_key="dummy")
|
||||
session = HeadlessSession(client=client, model="test-model")
|
||||
try:
|
||||
# Treatment arm activates the seeded skill via the real path.
|
||||
session.set_skill(_SKILL["name"])
|
||||
assert "SENTINEL_SKILL_BODY_MARKER" in _system_text(session)
|
||||
finally:
|
||||
session.close()
|
||||
|
||||
def test_control_omits_skill(self, temp_storage: None) -> None:
|
||||
# Control arm: no skill seeded, no set_skill — natural default only.
|
||||
client = OpenAI(base_url="http://localhost:9/v1", api_key="dummy")
|
||||
session = HeadlessSession(client=client, model="test-model")
|
||||
try:
|
||||
assert "SENTINEL_SKILL_BODY_MARKER" not in _system_text(session)
|
||||
finally:
|
||||
session.close()
|
||||
|
||||
def test_no_system_prompt_override_in_skill_mode(self, temp_storage: None) -> None:
|
||||
# skill_mode must NOT override the base identity — a real base prompt
|
||||
# (persona / composed developer message) must survive, or we'd be
|
||||
# measuring an empty prompt instead of the identity under test.
|
||||
client = OpenAI(base_url="http://localhost:9/v1", api_key="dummy")
|
||||
session = HeadlessSession(client=client, model="test-model")
|
||||
try:
|
||||
assert _system_text(session).strip(), "expected a composed base prompt"
|
||||
finally:
|
||||
session.close()
|
||||
|
||||
|
||||
class TestAdherenceLift:
|
||||
"""Unit-test the lift math with the per-arm runner stubbed."""
|
||||
|
||||
def test_lift_treatment_over_control(self, monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
# Stub _run_iteration: treatment (skill != None) passes 3/3, control
|
||||
# (skill is None) passes 1/3. run_skill_adherence must report the
|
||||
# difference as the lift.
|
||||
def fake_run_iteration(**kwargs: Any) -> dict[str, Any]:
|
||||
rate = 1.0 if kwargs.get("skill") is not None else 1.0 / 3.0
|
||||
return {"aggregate": {"overall_pass_rate": rate}}
|
||||
|
||||
monkeypatch.setattr(core, "_run_iteration", fake_run_iteration)
|
||||
|
||||
cases = [
|
||||
{
|
||||
"id": "search-first",
|
||||
"skill": _SKILL,
|
||||
"user_prompt": "where is X?",
|
||||
"expected_actions": [{"tool": "search"}],
|
||||
}
|
||||
]
|
||||
result = run_skill_adherence(
|
||||
client=None,
|
||||
base_url="http://localhost:9/v1",
|
||||
api_key="dummy",
|
||||
model="test-model",
|
||||
cases=cases,
|
||||
n_runs=3,
|
||||
temperature=0.7,
|
||||
max_tokens=1024,
|
||||
reasoning_effort="medium",
|
||||
context_window=8192,
|
||||
)
|
||||
|
||||
assert len(result["cases"]) == 1
|
||||
row = result["cases"][0]
|
||||
assert row["case_id"] == "search-first"
|
||||
assert row["skill"] == "search-first"
|
||||
assert row["treatment_rate"] == pytest.approx(1.0)
|
||||
assert row["control_rate"] == pytest.approx(1.0 / 3.0)
|
||||
assert row["lift"] == pytest.approx(2.0 / 3.0)
|
||||
assert row["n_runs"] == 3
|
||||
assert result["mean_lift"] == pytest.approx(2.0 / 3.0)
|
||||
|
||||
def test_rejects_malformed_skill(self) -> None:
|
||||
# A skill missing 'content' (or 'name') fails fast with a clear error,
|
||||
# not a KeyError mid-run (Copilot review). Validation raises before any
|
||||
# arm runs, so no _run_iteration stub is needed.
|
||||
cases = [
|
||||
{
|
||||
"id": "bad-skill",
|
||||
"skill": {"name": "x"}, # missing 'content'
|
||||
"user_prompt": "do x",
|
||||
"expected_actions": [{"tool": "search"}],
|
||||
}
|
||||
]
|
||||
with pytest.raises(ValueError, match="non-empty 'name' and 'content'"):
|
||||
run_skill_adherence(
|
||||
client=None,
|
||||
base_url="http://localhost:9/v1",
|
||||
api_key="dummy",
|
||||
model="test-model",
|
||||
cases=cases,
|
||||
n_runs=1,
|
||||
temperature=0.7,
|
||||
max_tokens=1024,
|
||||
reasoning_effort="medium",
|
||||
context_window=8192,
|
||||
)
|
||||
|
||||
def test_skipped_when_no_skill(self, monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
# A case with no skill is not measurable — it must be skipped, not
|
||||
# crash, and must not contribute to the mean.
|
||||
def fake_run_iteration(**kwargs: Any) -> dict[str, Any]:
|
||||
return {"aggregate": {"overall_pass_rate": 1.0}}
|
||||
|
||||
monkeypatch.setattr(core, "_run_iteration", fake_run_iteration)
|
||||
|
||||
cases = [{"id": "no-skill", "user_prompt": "hi", "expected_actions": []}]
|
||||
result = run_skill_adherence(
|
||||
client=None,
|
||||
base_url="http://localhost:9/v1",
|
||||
api_key="dummy",
|
||||
model="test-model",
|
||||
cases=cases,
|
||||
n_runs=3,
|
||||
temperature=0.7,
|
||||
max_tokens=1024,
|
||||
reasoning_effort="medium",
|
||||
context_window=8192,
|
||||
)
|
||||
assert result["cases"] == []
|
||||
assert result["mean_lift"] == 0.0
|
||||
|
||||
def test_mean_lift_averages_multiple_cases(self, monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
# Two skill cases with different lifts average into mean_lift.
|
||||
rates = iter([1.0, 0.0, 1.0, 0.5]) # t1, c1, t2, c2 -> lifts 1.0, 0.5
|
||||
|
||||
def fake_run_iteration(**kwargs: Any) -> dict[str, Any]:
|
||||
return {"aggregate": {"overall_pass_rate": next(rates)}}
|
||||
|
||||
monkeypatch.setattr(core, "_run_iteration", fake_run_iteration)
|
||||
|
||||
cases = [
|
||||
{"id": "a", "skill": _SKILL, "user_prompt": "q", "expected_actions": []},
|
||||
{"id": "b", "skill": _SKILL, "user_prompt": "q", "expected_actions": []},
|
||||
]
|
||||
result = run_skill_adherence(
|
||||
client=None,
|
||||
base_url="http://localhost:9/v1",
|
||||
api_key="dummy",
|
||||
model="test-model",
|
||||
cases=cases,
|
||||
n_runs=2,
|
||||
temperature=0.7,
|
||||
max_tokens=1024,
|
||||
reasoning_effort="medium",
|
||||
context_window=8192,
|
||||
)
|
||||
assert [c["lift"] for c in result["cases"]] == pytest.approx([1.0, 0.5])
|
||||
assert result["mean_lift"] == pytest.approx(0.75)
|
||||
@@ -135,9 +135,8 @@ def _create_skill(db: Any, skill_id: str, name: str, content: str, **kw: Any) ->
|
||||
|
||||
|
||||
def _sys_content(session: ChatSession) -> str:
|
||||
msgs = [m for m in session.system_messages if m["role"] == "system"]
|
||||
assert msgs
|
||||
return msgs[0]["content"]
|
||||
assert session.system_messages
|
||||
return "\n".join(m["content"] for m in session.system_messages)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
@@ -0,0 +1,295 @@
|
||||
"""Skill substitution unification (SKILL.md subsystem refactor, step 1).
|
||||
|
||||
Pins the invariant that skill-body placeholder substitution is IDENTICAL
|
||||
across every invocation context. Interactive load, default skills, and
|
||||
``task_agent`` sub-agents all route through
|
||||
``ChatSession._render_skill_body`` — so a skill reading ``$ARGUMENTS`` or
|
||||
``${TURNSTONE_EFFORT}`` resolves the same everywhere, rather than
|
||||
rendering literally on the ``task_agent`` path (which previously ran
|
||||
``_render_template`` alone).
|
||||
|
||||
Also covers the two behaviours the unified path newly guarantees:
|
||||
|
||||
* ``${TURNSTONE_*}`` env vars (canonical) and their ``${CLAUDE_*}``
|
||||
back-compat aliases both resolve.
|
||||
* ``${TURNSTONE_SKILL_DIR}`` resolves to the concrete materialized bundle
|
||||
path in the rendered body, because resources are materialized BEFORE
|
||||
substitution (the ordering fix).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
from tests._session_helpers import make_session
|
||||
from turnstone.core.storage._registry import get_storage
|
||||
|
||||
if TYPE_CHECKING:
|
||||
import pytest
|
||||
|
||||
|
||||
def _create_skill(db: Any, skill_id: str, name: str, content: str, **kw: Any) -> None:
|
||||
db.create_prompt_template(
|
||||
template_id=skill_id,
|
||||
name=name,
|
||||
category=kw.get("category", "general"),
|
||||
content=content,
|
||||
variables="[]",
|
||||
is_default=kw.get("is_default", False),
|
||||
org_id="",
|
||||
created_by="test",
|
||||
origin="manual",
|
||||
mcp_server="",
|
||||
readonly=False,
|
||||
description="",
|
||||
tags="[]",
|
||||
source_url="",
|
||||
version="1.0.0",
|
||||
author="",
|
||||
activation=kw.get("activation", "named"),
|
||||
token_estimate=0,
|
||||
model="",
|
||||
auto_approve=False,
|
||||
temperature=None,
|
||||
reasoning_effort="",
|
||||
max_tokens=None,
|
||||
token_budget=0,
|
||||
agent_max_turns=None,
|
||||
notify_on_complete="{}",
|
||||
enabled=True,
|
||||
allowed_tools="[]",
|
||||
priority=0,
|
||||
)
|
||||
|
||||
|
||||
class TestRenderSkillBodySharedPath:
|
||||
"""``_render_skill_body`` is the single substitution path — the one
|
||||
``task_agent`` now calls. With ``substitute_args=True`` (arg-capable
|
||||
invocations: interactive /skill, skills(load)) the spec arg forms
|
||||
resolve; with ``substitute_args=False`` (capability contexts: defaults,
|
||||
task_agent) literal ``$N``/``$ARGUMENTS`` are left untouched. Env vars
|
||||
resolve either way."""
|
||||
|
||||
def test_env_vars_resolve(self, tmp_db: str) -> None:
|
||||
session = make_session(reasoning_effort="high")
|
||||
try:
|
||||
out = session._render_skill_body(
|
||||
"id ${TURNSTONE_SESSION_ID} effort ${TURNSTONE_EFFORT}"
|
||||
)
|
||||
assert out == f"id {session._ws_id} effort high"
|
||||
finally:
|
||||
session.close()
|
||||
|
||||
def test_claude_aliases_resolve(self, tmp_db: str) -> None:
|
||||
session = make_session(reasoning_effort="low")
|
||||
try:
|
||||
out = session._render_skill_body("id ${CLAUDE_SESSION_ID} effort ${CLAUDE_EFFORT}")
|
||||
assert out == f"id {session._ws_id} effort low"
|
||||
finally:
|
||||
session.close()
|
||||
|
||||
def test_arg_capable_path_clears_bare_arguments_when_no_args(self, tmp_db: str) -> None:
|
||||
# Arg-capable invocation (interactive /skill, skills(load)) with no
|
||||
# args → bare $ARGUMENTS clears to empty, per the SKILL.md spec.
|
||||
session = make_session()
|
||||
try:
|
||||
assert session._render_skill_body("before $ARGUMENTS after") == "before after"
|
||||
finally:
|
||||
session.close()
|
||||
|
||||
def test_curly_and_spec_passes_both_apply(self, tmp_db: str) -> None:
|
||||
# Legacy ``{{model}}`` AND spec ``${TURNSTONE_EFFORT}`` in one body —
|
||||
# both passes run through the shared path.
|
||||
session = make_session(model="my-model", reasoning_effort="high")
|
||||
try:
|
||||
out = session._render_skill_body("model {{model}} effort ${TURNSTONE_EFFORT}")
|
||||
assert out == "model my-model effort high"
|
||||
finally:
|
||||
session.close()
|
||||
|
||||
def test_arg_capable_path_clears_positional_tokens_when_no_args(self, tmp_db: str) -> None:
|
||||
# Arg-capable path with no args → positional forms clear to empty (spec).
|
||||
session = make_session()
|
||||
try:
|
||||
out = session._render_skill_body("step $1 / $0 / $ARGUMENTS[2] done")
|
||||
assert out == "step / / done"
|
||||
finally:
|
||||
session.close()
|
||||
|
||||
def test_capability_context_preserves_literal_arg_tokens(self, tmp_db: str) -> None:
|
||||
# Capability contexts (task_agent, defaults) never receive invocation
|
||||
# args, so substitute_args=False leaves literal $ARGUMENTS/$N/$name
|
||||
# untouched (they are prose/shell text) while env vars still resolve.
|
||||
# Pins the review fix that stopped blanking such tokens for sub-agents.
|
||||
session = make_session(reasoning_effort="high")
|
||||
try:
|
||||
out = session._render_skill_body(
|
||||
"run ./deploy.sh $1 $2 at ${TURNSTONE_EFFORT}; process $ARGUMENTS",
|
||||
substitute_args=False,
|
||||
)
|
||||
assert out == "run ./deploy.sh $1 $2 at high; process $ARGUMENTS"
|
||||
finally:
|
||||
session.close()
|
||||
|
||||
def test_skill_dir_literal_on_sub_agent_path(self, tmp_db: str) -> None:
|
||||
# task_agent calls _render_skill_body with no skill_dir (sub-agent
|
||||
# bundles aren't materialized yet), so ${TURNSTONE_SKILL_DIR} stays
|
||||
# literal on this path — unchanged from before, resolved in a later
|
||||
# step. The env vars that DO have values still resolve.
|
||||
session = make_session(reasoning_effort="high")
|
||||
try:
|
||||
out = session._render_skill_body(
|
||||
"dir ${TURNSTONE_SKILL_DIR} effort ${TURNSTONE_EFFORT}"
|
||||
)
|
||||
assert out == "dir ${TURNSTONE_SKILL_DIR} effort high"
|
||||
finally:
|
||||
session.close()
|
||||
|
||||
|
||||
class TestSkillDirResolvesInBody:
|
||||
"""Materialize-before-substitute: ``${TURNSTONE_SKILL_DIR}`` in a skill
|
||||
body resolves to the concrete on-disk bundle path after a full load."""
|
||||
|
||||
def test_turnstone_skill_dir_in_body(self, tmp_db: str) -> None:
|
||||
db = get_storage()
|
||||
_create_skill(db, "s1", "dir-skill", "Scripts under ${TURNSTONE_SKILL_DIR}/scripts.")
|
||||
db.create_skill_resource("r1", "s1", "scripts/go.py", "print('x')")
|
||||
|
||||
session = make_session(skill="dir-skill")
|
||||
try:
|
||||
base = session._skill_resources_dir
|
||||
assert base is not None
|
||||
assert session._skill_content == f"Scripts under {base}/scripts."
|
||||
finally:
|
||||
session.close()
|
||||
|
||||
def test_claude_skill_dir_not_aliased_in_body(self, tmp_db: str) -> None:
|
||||
# CLAUDE_SKILL_DIR is NOT a turnstone-owned alias: it stays a literal
|
||||
# placeholder even with a materialized bundle (that name belongs to the
|
||||
# host in bash; turnstone claims neither surface). The canonical
|
||||
# TURNSTONE_SKILL_DIR does resolve.
|
||||
db = get_storage()
|
||||
_create_skill(
|
||||
db,
|
||||
"s1",
|
||||
"dir-alias-skill",
|
||||
"Bundle at ${CLAUDE_SKILL_DIR} vs ${TURNSTONE_SKILL_DIR}",
|
||||
)
|
||||
db.create_skill_resource("r1", "s1", "references/a.md", "# a")
|
||||
|
||||
session = make_session(skill="dir-alias-skill")
|
||||
try:
|
||||
base = session._skill_resources_dir
|
||||
assert base is not None
|
||||
assert session._skill_content == f"Bundle at ${{CLAUDE_SKILL_DIR}} vs {base}"
|
||||
finally:
|
||||
session.close()
|
||||
|
||||
def test_skill_dir_literal_without_resources(self, tmp_db: str) -> None:
|
||||
# No bundled resources → no dir → placeholder stays literal
|
||||
# (graceful degradation), not an empty path.
|
||||
db = get_storage()
|
||||
_create_skill(db, "s1", "no-res-skill", "Path ${TURNSTONE_SKILL_DIR} here.")
|
||||
|
||||
session = make_session(skill="no-res-skill")
|
||||
try:
|
||||
assert session._skill_resources_dir is None
|
||||
assert session._skill_content == "Path ${TURNSTONE_SKILL_DIR} here."
|
||||
finally:
|
||||
session.close()
|
||||
|
||||
|
||||
class TestSkillResourceEnvAliases:
|
||||
"""Bash env exposes the materialized bundle dir under turnstone-owned
|
||||
names (``TURNSTONE_SKILL_DIR`` / ``SKILL_RESOURCES_DIR``) unconditionally,
|
||||
and never under the foreign ``CLAUDE_SKILL_DIR`` — that name is the host's,
|
||||
so turnstone leaves it untouched whether or not the host has set it."""
|
||||
|
||||
def test_turnstone_owned_names_present_claude_absent(
|
||||
self, tmp_db: str, monkeypatch: pytest.MonkeyPatch
|
||||
) -> None:
|
||||
monkeypatch.delenv("CLAUDE_SKILL_DIR", raising=False)
|
||||
db = get_storage()
|
||||
_create_skill(db, "s1", "env-alias-skill", "content")
|
||||
db.create_skill_resource("r1", "s1", "scripts/t.py", "code")
|
||||
|
||||
session = make_session(skill="env-alias-skill")
|
||||
try:
|
||||
env = session._skill_resource_env()
|
||||
d = session._skill_resources_dir
|
||||
assert env["SKILL_RESOURCES_DIR"] == d
|
||||
assert env["TURNSTONE_SKILL_DIR"] == d
|
||||
# turnstone never supplies CLAUDE_SKILL_DIR (the host's namespace),
|
||||
# even when the host hasn't set it.
|
||||
assert "CLAUDE_SKILL_DIR" not in env
|
||||
finally:
|
||||
session.close()
|
||||
|
||||
def test_host_claude_skill_dir_untouched(
|
||||
self, tmp_db: str, monkeypatch: pytest.MonkeyPatch
|
||||
) -> None:
|
||||
# turnstone as a node inside Claude Code: the host's CLAUDE_SKILL_DIR
|
||||
# must survive. turnstone never injects CLAUDE_SKILL_DIR into the
|
||||
# extra-env, so scrubbed_env's passthrough keeps the host value.
|
||||
monkeypatch.setenv("CLAUDE_SKILL_DIR", "/host/claude/skill")
|
||||
db = get_storage()
|
||||
_create_skill(db, "s1", "env-host-skill", "content")
|
||||
db.create_skill_resource("r1", "s1", "scripts/t.py", "code")
|
||||
|
||||
session = make_session(skill="env-host-skill")
|
||||
try:
|
||||
env = session._skill_resource_env()
|
||||
d = session._skill_resources_dir
|
||||
assert env["TURNSTONE_SKILL_DIR"] == d
|
||||
assert env["SKILL_RESOURCES_DIR"] == d
|
||||
assert "CLAUDE_SKILL_DIR" not in env
|
||||
finally:
|
||||
session.close()
|
||||
|
||||
|
||||
class TestSkillContextPlacement:
|
||||
"""Step 3: an applied skill's body rides its own capability context
|
||||
message (user role), separate from the identity system message — so it
|
||||
never occupies the cached identity prefix or reads as identity, and it
|
||||
does not leak into the task_agent base."""
|
||||
|
||||
def test_skill_body_in_context_message_not_identity(self, tmp_db: str) -> None:
|
||||
db = get_storage()
|
||||
_create_skill(db, "s1", "place-skill", "PLACEMENT_MARKER body text")
|
||||
|
||||
session = make_session(skill="place-skill")
|
||||
try:
|
||||
msgs = session.system_messages
|
||||
# Identity system message is first, role=system, and skill-free.
|
||||
assert msgs[0]["role"] == "system"
|
||||
assert "PLACEMENT_MARKER" not in msgs[0]["content"]
|
||||
# Skill rides exactly one separate user-role capability message.
|
||||
skill_msgs = [m for m in msgs if m["role"] == "user"]
|
||||
assert len(skill_msgs) == 1
|
||||
assert "PLACEMENT_MARKER" in skill_msgs[0]["content"]
|
||||
# The intro names the active skill so the model knows what it is.
|
||||
assert "place-skill" in skill_msgs[0]["content"]
|
||||
finally:
|
||||
session.close()
|
||||
|
||||
def test_no_skill_no_context_message(self, tmp_db: str) -> None:
|
||||
# No applied skill and no defaults → only the identity system message.
|
||||
session = make_session()
|
||||
try:
|
||||
assert all(m["role"] == "system" for m in session.system_messages)
|
||||
finally:
|
||||
session.close()
|
||||
|
||||
def test_agent_prefix_excludes_skill_context(self, tmp_db: str) -> None:
|
||||
# task_agent base = the identity system block only; the parent's
|
||||
# applied skill does NOT leak into the sub-agent prefix.
|
||||
db = get_storage()
|
||||
_create_skill(db, "s1", "leak-skill", "SHOULD_NOT_LEAK body")
|
||||
|
||||
session = make_session(skill="leak-skill")
|
||||
try:
|
||||
assert len(session._agent_system_messages) == 1
|
||||
assert session._agent_system_messages[0]["role"] == "system"
|
||||
assert "SHOULD_NOT_LEAK" not in session._agent_system_messages[0]["content"]
|
||||
finally:
|
||||
session.close()
|
||||
@@ -111,10 +111,9 @@ def _make_session(**kwargs):
|
||||
|
||||
|
||||
def _sys_content(session: ChatSession) -> str:
|
||||
"""Extract the system message content."""
|
||||
msgs = [m for m in session.system_messages if m["role"] == "system"]
|
||||
assert msgs
|
||||
return msgs[0]["content"]
|
||||
"""Full prompt prefix: identity system message + any skill context message."""
|
||||
assert session.system_messages
|
||||
return "\n".join(m["content"] for m in session.system_messages)
|
||||
|
||||
|
||||
def _create_template(db, template_id, name, content, **kwargs):
|
||||
|
||||
@@ -1444,3 +1444,98 @@ class TestSkillCatalogDisclosure:
|
||||
# human-facing path); the tool-facing path is the new
|
||||
# ``skills(action='find')`` flow.
|
||||
assert "/skill" in content
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Tests — model-initiated load risk gate (design §5.5 / Principle 7)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestSkillsLoadRiskGate:
|
||||
"""A high/critical-risk skill is PRINCIPAL-load-only: the model cannot
|
||||
activate it through skills(action='load') — that path is denied outright,
|
||||
forcing an explicit operator /skill. Lower-risk skills load as before.
|
||||
The scanner-computed risk_level is the gate signal (it already escalates
|
||||
for the auto_approve + allowed_tools authority the create path warns about),
|
||||
so no new schema is needed. The principal path (set_skill via /skill or
|
||||
cli --skill) is not routed through _prepare_skills_load and is not gated."""
|
||||
|
||||
@staticmethod
|
||||
def _load_item(session, name: str, risk: str):
|
||||
row = {"name": name, "risk_level": risk, "enabled": True, "content": "x"}
|
||||
with patch("turnstone.core.session.get_storage") as gs:
|
||||
gs.return_value.get_prompt_template_by_name.return_value = row
|
||||
return session._prepare_skills("c1", {"action": "load", "name": name})
|
||||
|
||||
def test_high_risk_load_denied_for_model(self) -> None:
|
||||
item = self._load_item(_make_session(), "danger", "high")
|
||||
assert item.get("needs_approval") is False
|
||||
assert "/skill danger" in item["error"]
|
||||
assert "high" in item["error"]
|
||||
|
||||
def test_critical_risk_load_denied_for_model(self) -> None:
|
||||
item = self._load_item(_make_session(), "nuke", "critical")
|
||||
assert item.get("needs_approval") is False
|
||||
assert "critical" in item["error"]
|
||||
|
||||
def test_low_risk_load_allowed_for_model(self) -> None:
|
||||
item = self._load_item(_make_session(), "safe", "low")
|
||||
assert item.get("needs_approval") is True
|
||||
assert item["action"] == "load"
|
||||
assert item["name"] == "safe"
|
||||
|
||||
def test_no_risk_load_allowed_for_model(self) -> None:
|
||||
item = self._load_item(_make_session(), "plain", "")
|
||||
assert item.get("needs_approval") is True
|
||||
|
||||
def test_missing_skill_falls_through_to_exec_not_found(self) -> None:
|
||||
# Unknown name → the gate finds no row and passes through; the
|
||||
# not-found error is the exec path's job, so prep still asks for
|
||||
# approval rather than erroring on the gate.
|
||||
session = _make_session()
|
||||
with patch("turnstone.core.session.get_storage") as gs:
|
||||
gs.return_value.get_prompt_template_by_name.return_value = None
|
||||
item = session._prepare_skills("c1", {"action": "load", "name": "ghost"})
|
||||
assert item.get("needs_approval") is True
|
||||
|
||||
def test_shared_helper_denies_high_and_critical_only(self) -> None:
|
||||
# The one shared gate used by skills(load) AND the spawn paths, so a
|
||||
# child spawn cannot route around it.
|
||||
session = _make_session()
|
||||
for tier in ("high", "critical"):
|
||||
with patch("turnstone.core.session.get_storage") as gs:
|
||||
gs.return_value.get_prompt_template_by_name.return_value = {
|
||||
"name": "x",
|
||||
"risk_level": tier,
|
||||
}
|
||||
assert "/skill x" in session._high_risk_skill_denied("x")
|
||||
for row in (
|
||||
{"name": "y", "risk_level": "low"},
|
||||
{"name": "y", "risk_level": ""},
|
||||
None,
|
||||
):
|
||||
with patch("turnstone.core.session.get_storage") as gs:
|
||||
gs.return_value.get_prompt_template_by_name.return_value = row
|
||||
assert session._high_risk_skill_denied("y") == ""
|
||||
|
||||
def test_risk_gate_fails_closed_on_storage_error(self) -> None:
|
||||
# A risk gate that can't read the row must DENY, never wave the skill
|
||||
# through (fail closed). Returning a denial (not "") also keeps
|
||||
# spawn_batch's per-row partial-success intact under a storage blip.
|
||||
session = _make_session()
|
||||
with patch("turnstone.core.session.get_storage") as gs:
|
||||
gs.return_value.get_prompt_template_by_name.side_effect = RuntimeError("db down")
|
||||
denial = session._high_risk_skill_denied("z")
|
||||
assert denial != ""
|
||||
assert "z" in denial
|
||||
assert "/skill z" in denial
|
||||
|
||||
def test_risk_gate_fails_closed_on_no_storage(self) -> None:
|
||||
# Storage-unavailable (get_storage() is None) is treated the same as a
|
||||
# lookup fault — DENY, not allow. A gate that can't verify the risk
|
||||
# tier must not wave the skill through (Copilot review nit).
|
||||
session = _make_session()
|
||||
with patch("turnstone.core.session.get_storage", return_value=None):
|
||||
denial = session._high_risk_skill_denied("z")
|
||||
assert denial != ""
|
||||
assert "/skill z" in denial
|
||||
|
||||
@@ -1,10 +1,14 @@
|
||||
"""Unit tests for ``_substitute_skill_args`` — SKILL.md spec placeholder
|
||||
substitution applied to skill bodies at load time.
|
||||
|
||||
Covers every placeholder form Turnstone implements (``${CLAUDE_SKILL_DIR}``
|
||||
is deferred — see #572) plus the spec's "append ARGUMENTS at end if no
|
||||
placeholder" rule and the single-pass guarantee against re-expansion of
|
||||
user-supplied values that happen to contain placeholder syntax.
|
||||
Covers every placeholder form Turnstone implements — including the
|
||||
``${TURNSTONE_*}`` canonical env vars and the ``${CLAUDE_SESSION_ID}`` /
|
||||
``${CLAUDE_EFFORT}`` back-compat aliases (there is deliberately NO
|
||||
``CLAUDE_SKILL_DIR`` alias), ``${TURNSTONE_SKILL_DIR}`` resolution when the
|
||||
caller supplies a materialized bundle path, plus the spec's "append
|
||||
ARGUMENTS at end if no placeholder" rule and the single-pass guarantee
|
||||
against re-expansion of user-supplied values that happen to contain
|
||||
placeholder syntax.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
@@ -136,6 +140,64 @@ class TestEnvironment:
|
||||
assert _sub("${CLAUDE_UNKNOWN_FOO}") == "${CLAUDE_UNKNOWN_FOO}"
|
||||
|
||||
|
||||
class TestEnvironmentAliases:
|
||||
"""``TURNSTONE_*`` is the canonical, vendor-neutral spelling;
|
||||
``CLAUDE_*`` is a permanent back-compat alias so skills imported from
|
||||
Claude Code / skills.sh keep resolving. Both map to one value."""
|
||||
|
||||
def test_turnstone_session_id(self) -> None:
|
||||
assert _sub("session ${TURNSTONE_SESSION_ID}") == "session ws-abc"
|
||||
|
||||
def test_turnstone_effort(self) -> None:
|
||||
assert _sub("effort ${TURNSTONE_EFFORT}") == "effort high"
|
||||
|
||||
def test_canonical_and_alias_agree(self) -> None:
|
||||
assert _sub("${CLAUDE_SESSION_ID}") == _sub("${TURNSTONE_SESSION_ID}") == "ws-abc"
|
||||
assert _sub("${CLAUDE_EFFORT}") == _sub("${TURNSTONE_EFFORT}") == "high"
|
||||
|
||||
def test_unknown_turnstone_var_left_as_literal(self) -> None:
|
||||
assert _sub("${TURNSTONE_UNKNOWN_FOO}") == "${TURNSTONE_UNKNOWN_FOO}"
|
||||
|
||||
|
||||
class TestSkillDir:
|
||||
"""``${TURNSTONE_SKILL_DIR}`` resolves to the materialized bundle path when
|
||||
the caller supplies one (it materializes resources BEFORE substituting) and
|
||||
degrades to a literal placeholder when the skill bundles no resources.
|
||||
``${CLAUDE_SKILL_DIR}`` is NOT a turnstone alias — it always stays literal
|
||||
(that name is the host's in bash; turnstone claims neither surface)."""
|
||||
|
||||
def test_turnstone_skill_dir_resolves(self) -> None:
|
||||
out = _substitute_skill_args(
|
||||
"cd ${TURNSTONE_SKILL_DIR}/scripts",
|
||||
arguments_str="",
|
||||
arg_names=[],
|
||||
ws_id="ws-abc",
|
||||
effort="high",
|
||||
skill_dir="/tmp/skill-xyz",
|
||||
)
|
||||
assert out == "cd /tmp/skill-xyz/scripts"
|
||||
|
||||
def test_claude_skill_dir_not_aliased(self) -> None:
|
||||
# Even with a materialized bundle, ${CLAUDE_SKILL_DIR} is left literal:
|
||||
# turnstone owns TURNSTONE_SKILL_DIR only, so the two never diverge from
|
||||
# the bash env (which likewise never sets CLAUDE_SKILL_DIR).
|
||||
out = _substitute_skill_args(
|
||||
"cd ${CLAUDE_SKILL_DIR}",
|
||||
arguments_str="",
|
||||
arg_names=[],
|
||||
ws_id="ws-abc",
|
||||
effort="high",
|
||||
skill_dir="/tmp/skill-xyz",
|
||||
)
|
||||
assert out == "cd ${CLAUDE_SKILL_DIR}"
|
||||
|
||||
def test_skill_dir_left_literal_when_unset(self) -> None:
|
||||
# Default skill_dir="" → placeholder stays literal (graceful),
|
||||
# not an empty path.
|
||||
assert _sub("${TURNSTONE_SKILL_DIR}") == "${TURNSTONE_SKILL_DIR}"
|
||||
assert _sub("${CLAUDE_SKILL_DIR}") == "${CLAUDE_SKILL_DIR}"
|
||||
|
||||
|
||||
class TestSinglePassGuarantee:
|
||||
"""A placeholder VALUE containing another placeholder must not be
|
||||
re-expanded — matches spec's "Substitution runs once" rule."""
|
||||
@@ -177,3 +239,45 @@ class TestIntegration:
|
||||
"Named: 123 resolved on main.\n"
|
||||
"Full: 123 main"
|
||||
)
|
||||
|
||||
|
||||
class TestSubstituteArgsToggle:
|
||||
"""``substitute_args=False`` (capability contexts: defaults, task_agent)
|
||||
leaves every invocation-arg form LITERAL while still resolving env vars,
|
||||
so literal ``$1`` / ``$ARGUMENTS`` prose or shell text isn't blanked."""
|
||||
|
||||
def test_arg_forms_left_literal(self) -> None:
|
||||
out = _substitute_skill_args(
|
||||
"run $0 $1 $ARGUMENTS $ARGUMENTS[2] $named",
|
||||
arguments_str="a b c", # present, but ignored under substitute_args=False
|
||||
arg_names=["named"],
|
||||
ws_id="ws-abc",
|
||||
effort="high",
|
||||
substitute_args=False,
|
||||
)
|
||||
assert out == "run $0 $1 $ARGUMENTS $ARGUMENTS[2] $named"
|
||||
|
||||
def test_env_still_resolves(self) -> None:
|
||||
out = _substitute_skill_args(
|
||||
"id ${TURNSTONE_SESSION_ID} at ${TURNSTONE_EFFORT} in ${TURNSTONE_SKILL_DIR}",
|
||||
arguments_str="",
|
||||
arg_names=[],
|
||||
ws_id="ws-abc",
|
||||
effort="high",
|
||||
skill_dir="/tmp/skill-x",
|
||||
substitute_args=False,
|
||||
)
|
||||
assert out == "id ws-abc at high in /tmp/skill-x"
|
||||
|
||||
def test_no_append_when_args_disabled(self) -> None:
|
||||
# The append-ARGUMENTS-at-end rule must not fire when arg substitution
|
||||
# is off, even if arguments_str is non-empty.
|
||||
out = _substitute_skill_args(
|
||||
"body with no placeholder",
|
||||
arguments_str="x y",
|
||||
arg_names=[],
|
||||
ws_id="ws-abc",
|
||||
effort="high",
|
||||
substitute_args=False,
|
||||
)
|
||||
assert out == "body with no placeholder"
|
||||
|
||||
@@ -21,12 +21,12 @@ exactly the case the visibility fix is meant to surface.
|
||||
from __future__ import annotations
|
||||
|
||||
import queue
|
||||
import threading
|
||||
from typing import Any
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
import pytest
|
||||
|
||||
from tests.conftest import resolve_when_pending
|
||||
from turnstone.server import WebUI
|
||||
|
||||
|
||||
@@ -107,11 +107,11 @@ def test_policy_partial_allow_then_prompt_records_allowed_items() -> None:
|
||||
ui = WebUI(ws_id="ws-test")
|
||||
items = _make_items(("c1", "read_file"), ("c2", "bash"))
|
||||
|
||||
# ``approve_tools`` blocks on ``_approval_event.wait`` for the
|
||||
# prompt path. Schedule a deny-by-operator on a tiny timer so
|
||||
# the wait returns promptly; this test asserts on ring-buffer
|
||||
# state, not the verdict outcome, so a deny is fine.
|
||||
timer = threading.Timer(0.05, lambda: ui.resolve_approval(False))
|
||||
# ``approve_tools`` blocks on ``_approval_event.wait`` for the prompt
|
||||
# path. Resolve (deny) once the approval is registered so the wait
|
||||
# returns promptly without the lost-wakeup race a fixed-delay timer has;
|
||||
# this test asserts on ring-buffer state, not the verdict, so a deny is fine.
|
||||
timer = resolve_when_pending(ui, False)
|
||||
timer.start()
|
||||
|
||||
storage = MagicMock()
|
||||
|
||||
@@ -30,86 +30,37 @@ from typing import TYPE_CHECKING, Any, cast
|
||||
|
||||
import pytest
|
||||
|
||||
from tests._wire_capture import RecordingClient
|
||||
from turnstone.core.lowering import repair_wire_messages
|
||||
from turnstone.core.providers._anthropic import AnthropicProvider
|
||||
from turnstone.core.providers._google import GoogleProvider
|
||||
from turnstone.core.providers._openai_chat import OpenAIChatCompletionsProvider
|
||||
from turnstone.core.providers._openai_responses import OpenAIResponsesProvider
|
||||
from turnstone.core.providers._protocol import ModelCapabilities
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from collections.abc import Iterator
|
||||
|
||||
from turnstone.core.providers._protocol import LLMProvider
|
||||
|
||||
GOLDEN_DIR = Path(__file__).parent / "data" / "wire_payloads"
|
||||
_UPDATE = os.environ.get("UPDATE_WIRE_GOLDENS") == "1"
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# Recording fake client — captures the kwargs at each provider's SDK seam.
|
||||
# --------------------------------------------------------------------------- #
|
||||
class _EmptyStream:
|
||||
"""Stand-in for an SDK stream / stream-manager: empty iterable AND no-op CM."""
|
||||
|
||||
def __iter__(self) -> Iterator[Any]:
|
||||
return iter(())
|
||||
|
||||
def __enter__(self) -> _EmptyStream:
|
||||
return self
|
||||
|
||||
def __exit__(self, *exc: object) -> None:
|
||||
return None
|
||||
|
||||
|
||||
class _Seam:
|
||||
"""Records the kwargs of a single SDK call, returns an empty stream stub."""
|
||||
|
||||
def __init__(self, sink: dict[str, Any]) -> None:
|
||||
self._sink = sink
|
||||
|
||||
def __call__(self, **kwargs: Any) -> _EmptyStream:
|
||||
# Last write wins; only one seam is exercised per provider call.
|
||||
self._sink["payload"] = kwargs
|
||||
return _EmptyStream()
|
||||
|
||||
|
||||
class _Completions:
|
||||
def __init__(self, sink: dict[str, Any]) -> None:
|
||||
self.create = _Seam(sink)
|
||||
|
||||
|
||||
class _Chat:
|
||||
def __init__(self, sink: dict[str, Any]) -> None:
|
||||
self.completions = _Completions(sink)
|
||||
|
||||
|
||||
class _Messages:
|
||||
def __init__(self, sink: dict[str, Any]) -> None:
|
||||
self.stream = _Seam(sink)
|
||||
|
||||
|
||||
class _Responses:
|
||||
def __init__(self, sink: dict[str, Any]) -> None:
|
||||
self.create = _Seam(sink)
|
||||
self.stream = _Seam(sink)
|
||||
|
||||
|
||||
class RecordingClient:
|
||||
"""Fake SDK client exposing every provider's call seam, recording kwargs."""
|
||||
|
||||
def __init__(self) -> None:
|
||||
self.captured: dict[str, Any] = {}
|
||||
self.messages = _Messages(self.captured)
|
||||
self.chat = _Chat(self.captured)
|
||||
self.responses = _Responses(self.captured)
|
||||
|
||||
|
||||
def _capture(
|
||||
provider: LLMProvider, *, model: str, messages: list[dict[str, Any]], **opts: Any
|
||||
provider: LLMProvider,
|
||||
*,
|
||||
model: str,
|
||||
messages: list[dict[str, Any]],
|
||||
caps: ModelCapabilities | None = None,
|
||||
**opts: Any,
|
||||
) -> dict[str, Any]:
|
||||
"""Drive ``create_streaming`` against a recording client; return the SDK kwargs."""
|
||||
"""Drive ``create_streaming`` against a recording client; return the SDK kwargs.
|
||||
|
||||
*caps* overrides the provider's own capability lookup — required for the
|
||||
anthropic-compatible lane, which has no static table (an operator-run model
|
||||
definition supplies its capabilities).
|
||||
"""
|
||||
client = RecordingClient()
|
||||
caps = provider.get_capabilities(model)
|
||||
caps = caps or provider.get_capabilities(model)
|
||||
# Mirror the session's wire prep: orphan repair runs once on the canonical
|
||||
# Turns (``ChatSession._prepare_wire_messages``), then the result is lowered
|
||||
# to the dict projection the translator consumes. Fixtures arrive
|
||||
@@ -145,7 +96,10 @@ def _assert_golden(name: str, payload: dict[str, Any]) -> None:
|
||||
norm = _normalize(payload)
|
||||
if _UPDATE:
|
||||
GOLDEN_DIR.mkdir(parents=True, exist_ok=True)
|
||||
path.write_text(json.dumps(norm, indent=2, sort_keys=True) + "\n")
|
||||
# ensure_ascii=False keeps non-ASCII (em dashes in repair
|
||||
# messages) literal, matching the existing baselines — a regen
|
||||
# must not churn unrelated lines into \uXXXX escapes.
|
||||
path.write_text(json.dumps(norm, indent=2, sort_keys=True, ensure_ascii=False) + "\n")
|
||||
return
|
||||
assert path.exists(), f"missing golden {name!r}; run UPDATE_WIRE_GOLDENS=1 to baseline"
|
||||
assert norm == json.loads(path.read_text()), f"wire-payload drift for {name!r}"
|
||||
@@ -321,3 +275,36 @@ def test_wire_payload(
|
||||
provider = factory()
|
||||
payload = _capture(provider, model=model, messages=[dict(m) for m in messages], **opts)
|
||||
_assert_golden(f"{provider_id}__{fixture_id}", payload)
|
||||
|
||||
|
||||
# The anthropic-compatible lane (vLLM /v1/messages) has no static capability
|
||||
# table and carries reasoning control in ``extra_body.chat_template_kwargs``
|
||||
# rather than the native ``thinking`` param — a wire shape the matrix above
|
||||
# never exercises (both AnthropicProvider rows are the native lane). Freeze it
|
||||
# with the capabilities a manual-mode model definition supplies and a real
|
||||
# effort level, so the graded ``reasoning_effort`` key is pinned in the golden
|
||||
# (never the native ``thinking`` param, and no forced ``temperature=1.0``).
|
||||
_COMPAT_CAPS = ModelCapabilities(
|
||||
context_window=262144,
|
||||
max_output_tokens=64000,
|
||||
token_param="max_tokens",
|
||||
thinking_mode="manual",
|
||||
thinking_param="enable_thinking",
|
||||
supports_reasoning_replay=True,
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("fixture_id", sorted(_FIXTURES))
|
||||
def test_wire_payload_anthropic_compat(fixture_id: str) -> None:
|
||||
messages, opts = _FIXTURES[fixture_id]
|
||||
provider = AnthropicProvider(compat=True)
|
||||
payload = _capture(
|
||||
provider,
|
||||
model="qwen3.6-27b",
|
||||
messages=[dict(m) for m in messages],
|
||||
caps=_COMPAT_CAPS,
|
||||
reasoning_effort="high",
|
||||
**opts,
|
||||
)
|
||||
assert "thinking" not in payload, "compat lane must never send the native thinking param"
|
||||
_assert_golden(f"anthropic_compat__{fixture_id}", payload)
|
||||
|
||||
@@ -1598,7 +1598,7 @@ class TestDetailInteractive:
|
||||
"user_id": "test-user",
|
||||
"kind": "interactive",
|
||||
"pending_approval": False,
|
||||
"pending_approval_detail": None,
|
||||
"pending_approval_details": [],
|
||||
}
|
||||
|
||||
def test_pending_approval_fields_propagate_from_ui(self):
|
||||
@@ -1631,23 +1631,26 @@ class TestDetailInteractive:
|
||||
],
|
||||
"judge_pending": True,
|
||||
}
|
||||
loaded_ws.ui.serialize_pending_approval_detail = MagicMock(
|
||||
return_value={
|
||||
"call_id": "c-1",
|
||||
"judge_pending": True,
|
||||
"items": [
|
||||
{
|
||||
"call_id": "c-1",
|
||||
"func_name": "spawn_workstream",
|
||||
"needs_approval": True,
|
||||
"heuristic_verdict": {
|
||||
"recommendation": "approve",
|
||||
"risk_level": "low",
|
||||
"confidence": 0.9,
|
||||
},
|
||||
}
|
||||
],
|
||||
}
|
||||
loaded_ws.ui.serialize_pending_approval_details = MagicMock(
|
||||
return_value=[
|
||||
{
|
||||
"cycle_id": "cyc-1",
|
||||
"call_id": "c-1",
|
||||
"judge_pending": True,
|
||||
"items": [
|
||||
{
|
||||
"call_id": "c-1",
|
||||
"func_name": "spawn_workstream",
|
||||
"needs_approval": True,
|
||||
"heuristic_verdict": {
|
||||
"recommendation": "approve",
|
||||
"risk_level": "low",
|
||||
"confidence": 0.9,
|
||||
},
|
||||
}
|
||||
],
|
||||
}
|
||||
]
|
||||
)
|
||||
mock_mgr = MagicMock()
|
||||
mock_mgr.get.return_value = loaded_ws
|
||||
@@ -1657,9 +1660,12 @@ class TestDetailInteractive:
|
||||
assert r.status_code == 200
|
||||
body = r.json()
|
||||
assert body["pending_approval"] is True
|
||||
assert body["pending_approval_detail"]["call_id"] == "c-1"
|
||||
assert body["pending_approval_detail"]["judge_pending"] is True
|
||||
items = body["pending_approval_detail"]["items"]
|
||||
details = body["pending_approval_details"]
|
||||
assert len(details) == 1
|
||||
assert details[0]["cycle_id"] == "cyc-1"
|
||||
assert details[0]["call_id"] == "c-1"
|
||||
assert details[0]["judge_pending"] is True
|
||||
items = details[0]["items"]
|
||||
assert len(items) == 1
|
||||
assert items[0]["func_name"] == "spawn_workstream"
|
||||
assert items[0]["needs_approval"] is True
|
||||
@@ -1680,7 +1686,7 @@ class TestDetailInteractive:
|
||||
loaded_ws.user_id = "test-user"
|
||||
loaded_ws.kind = "coordinator"
|
||||
loaded_ws.ui._pending_approval = {"items": []}
|
||||
loaded_ws.ui.serialize_pending_approval_detail = MagicMock(
|
||||
loaded_ws.ui.serialize_pending_approval_details = MagicMock(
|
||||
side_effect=RuntimeError("verdict object is malformed"),
|
||||
)
|
||||
mock_mgr = MagicMock()
|
||||
@@ -1691,7 +1697,7 @@ class TestDetailInteractive:
|
||||
assert r.status_code == 200
|
||||
body = r.json()
|
||||
assert body["pending_approval"] is True
|
||||
assert body["pending_approval_detail"] is None
|
||||
assert body["pending_approval_details"] == []
|
||||
|
||||
def test_lazy_rehydrates_on_miss(self):
|
||||
"""``mgr.get`` miss → ``mgr.open`` rehydrate. Same flow as coord;
|
||||
@@ -1801,7 +1807,7 @@ class TestTenantCheckOnReadEndpoints:
|
||||
# Sensitive fields the PR added must not surface for a
|
||||
# non-owning caller.
|
||||
assert "name" not in body
|
||||
assert "pending_approval_detail" not in body
|
||||
assert "pending_approval_details" not in body
|
||||
assert "user_id" not in body
|
||||
# And mgr.get was NEVER consulted — the gate fires first.
|
||||
mock_mgr.get.assert_not_called()
|
||||
@@ -1832,7 +1838,7 @@ class TestTenantCheckOnReadEndpoints:
|
||||
body = r.json()
|
||||
assert body["ws_id"] == ws_id
|
||||
assert body["pending_approval"] is False
|
||||
assert body["pending_approval_detail"] is None
|
||||
assert body["pending_approval_details"] == []
|
||||
|
||||
def test_history_404s_when_tenant_check_rejects(self, _inject_storage):
|
||||
"""A non-owning interactive caller reading another user's ws_id
|
||||
|
||||
@@ -1,3 +1,3 @@
|
||||
"""turnstone - Multi-node AI orchestration platform with tool use, agent routing, and cluster simulation."""
|
||||
|
||||
__version__ = "1.7.0a6"
|
||||
__version__ = "1.7.0"
|
||||
|
||||
@@ -287,8 +287,8 @@ class ListWorkstreamsResponse(BaseModel):
|
||||
class PendingApprovalItem(BaseModel):
|
||||
"""One pending tool-call inside a ``PendingApprovalDetail`` envelope.
|
||||
|
||||
Mirrors the dict ``SessionUIBase.serialize_pending_approval_detail``
|
||||
emits per item. ``heuristic_verdict`` / ``judge_verdict`` are kept
|
||||
Mirrors the dict ``SessionUIBase.serialize_pending_approval_details``
|
||||
emits per item inside each cycle entry. ``heuristic_verdict`` / ``judge_verdict`` are kept
|
||||
loosely-typed because the underlying verdict shape varies by tier;
|
||||
consumers that want the full structure can decode against
|
||||
:class:`turnstone.sdk.events.IntentVerdictEvent`.
|
||||
@@ -339,13 +339,24 @@ class RecentAutoApproval(BaseModel):
|
||||
class PendingApprovalDetail(BaseModel):
|
||||
"""Inline approval payload merged into ``DashboardWorkstream``.
|
||||
|
||||
Set when a workstream's ``approve_tools`` is parked on
|
||||
``_approval_event``; ``None`` (omitted) otherwise. Cross-tenant
|
||||
One entry per live approval CYCLE — a gate thread parked in
|
||||
``approve_tools`` awaiting the operator. Parallel task agents run
|
||||
concurrent gates, so a workstream can have several of these at
|
||||
once (``pending_approval_details``, oldest first). Cross-tenant
|
||||
exposure here follows the same trusted-team posture as
|
||||
``activity`` / ``tokens`` — see ``server.py``'s ``dashboard``
|
||||
handler comment.
|
||||
"""
|
||||
|
||||
cycle_id: str = Field(
|
||||
default="",
|
||||
description=(
|
||||
"Identity of this approval cycle. Echo it back on "
|
||||
"``POST /v1/api/workstreams/{ws_id}/approve`` to resolve "
|
||||
"exactly this round — required for correctness when "
|
||||
"several cycles are live (parallel task agents)."
|
||||
),
|
||||
)
|
||||
call_id: str = Field(
|
||||
default="",
|
||||
description=(
|
||||
@@ -388,16 +399,21 @@ class DashboardWorkstream(BaseModel):
|
||||
parent_ws_id: str | None = None
|
||||
user_id: str = ""
|
||||
project_id: str | None = None
|
||||
pending_approval_detail: PendingApprovalDetail | None = Field(
|
||||
default=None,
|
||||
pending_approval_details: list[PendingApprovalDetail] = Field(
|
||||
default_factory=list,
|
||||
description=(
|
||||
"Inline approval payload for the coordinator children-tree "
|
||||
"UI. Carries the merged ``_pending_approval`` items list + "
|
||||
"per-call_id LLM verdict cache so a coord can render "
|
||||
"approve/deny buttons + judge pill without a separate "
|
||||
"per-child round-trip. ``None`` when no approval is pending. "
|
||||
"Also surfaced (verbatim) on ``GET /v1/api/cluster/ws/live`` "
|
||||
"via the ``_CLUSTER_WS_LIVE_KEYS`` projection."
|
||||
"UI: EVERY live approval cycle, oldest first — parallel "
|
||||
"task agents gate concurrently, so a workstream can hold "
|
||||
"several prompts at once. Each entry carries the cycle's "
|
||||
"items + per-call_id LLM verdict cache so a coord can "
|
||||
"render approve/deny buttons + judge pill without a "
|
||||
"separate per-child round-trip; resolve each with its "
|
||||
"``cycle_id``. Empty when no approval is pending. Also "
|
||||
"surfaced (verbatim) on ``GET /v1/api/cluster/ws/live`` "
|
||||
"via the ``_CLUSTER_WS_LIVE_KEYS`` projection. Replaces "
|
||||
"1.6's ``pending_approval_detail`` single-object field "
|
||||
"(breaking, 1.7)."
|
||||
),
|
||||
)
|
||||
recent_auto_approvals: list[RecentAutoApproval] = Field(
|
||||
@@ -482,21 +498,25 @@ class WorkstreamDetailResponse(BaseModel):
|
||||
pending_approval: bool = Field(
|
||||
default=False,
|
||||
description=(
|
||||
"True when the workstream is parked on ``_approval_event`` "
|
||||
"awaiting an operator approve/deny. Mirrors the same field "
|
||||
"on ``DashboardWorkstream`` / cluster live projections so a "
|
||||
"freshly-loaded chat tab can render the inline approval gate "
|
||||
"from the detail snapshot before SSE replay arrives."
|
||||
"True when at least one approval cycle is live (a gate "
|
||||
"thread parked awaiting an operator approve/deny). Mirrors "
|
||||
"the same field on ``DashboardWorkstream`` / cluster live "
|
||||
"projections so a freshly-loaded chat tab can render the "
|
||||
"inline approval gate from the detail snapshot before SSE "
|
||||
"replay arrives."
|
||||
),
|
||||
)
|
||||
pending_approval_detail: PendingApprovalDetail | None = Field(
|
||||
default=None,
|
||||
pending_approval_details: list[PendingApprovalDetail] = Field(
|
||||
default_factory=list,
|
||||
description=(
|
||||
"Inline approval payload — same shape as ``DashboardWorkstream"
|
||||
".pending_approval_detail``. ``None`` when no approval is "
|
||||
"pending. Lets a reload paint the action row + judge "
|
||||
"Inline approval payloads, one per live cycle, oldest "
|
||||
"first — same shape as ``DashboardWorkstream"
|
||||
".pending_approval_details``. Empty when no approval is "
|
||||
"pending. Lets a reload paint every action row + judge "
|
||||
"verdicts immediately instead of relying on the SSE "
|
||||
"approve_request replay timing window."
|
||||
"approve_request replay timing window. Replaces 1.6's "
|
||||
"``pending_approval_detail`` single-object field "
|
||||
"(breaking, 1.7)."
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
@@ -11,7 +11,7 @@ import asyncio
|
||||
import time
|
||||
from collections import OrderedDict
|
||||
from dataclasses import dataclass, field
|
||||
from typing import TYPE_CHECKING, Any, Literal
|
||||
from typing import TYPE_CHECKING, Any, Literal, TypeVar
|
||||
|
||||
from turnstone.channels._config import CREATE_LOCK_CAP
|
||||
from turnstone.core.log import get_logger
|
||||
@@ -19,6 +19,8 @@ from turnstone.sdk._types import TurnstoneAPIError
|
||||
from turnstone.sdk.console import AsyncTurnstoneConsole
|
||||
from turnstone.sdk.server import AsyncTurnstoneServer
|
||||
|
||||
_V = TypeVar("_V")
|
||||
|
||||
|
||||
@dataclass
|
||||
class PolicyVerdict:
|
||||
@@ -43,7 +45,7 @@ class PolicyVerdict:
|
||||
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from collections.abc import Callable
|
||||
from collections.abc import Callable, Mapping, MutableMapping
|
||||
|
||||
from turnstone.core.storage import StorageBackend
|
||||
|
||||
@@ -56,6 +58,57 @@ _ROUTE_CACHE_TTL = 30.0 # cache (channel_type, channel_id) → ws_id lookups
|
||||
_ROUTE_CACHE_CAP = 4096 # LRU bound on the lookup cache
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Pending-approval bookkeeping shared by the channel adapters.
|
||||
#
|
||||
# Every adapter tracks its posted approval prompts in a dict keyed by
|
||||
# ``(ws_id, cycle_id)`` — one entry per concurrent approval cycle — and
|
||||
# needs the same two lookups: "this exact cycle, falling back to the
|
||||
# workstream's single entry when the cycle_id is empty" (events from a
|
||||
# pre-multi-cycle server, or buttons posted before an in-flight
|
||||
# upgrade), and "everything for this workstream" (stream end / close /
|
||||
# unsubscribe sweeps). Centralised here so the legacy-fallback
|
||||
# semantics can't drift between adapters.
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def get_cycle_entry(
|
||||
entries: Mapping[tuple[str, str], _V],
|
||||
ws_id: str,
|
||||
cycle_id: str,
|
||||
) -> _V | None:
|
||||
"""Exact ``(ws_id, cycle_id)`` lookup with the pre-multi-cycle fallback.
|
||||
|
||||
An empty *cycle_id* falls back to the workstream's single tracked
|
||||
entry; a NON-empty one never falls back (a stale cycle must not
|
||||
resolve an unrelated prompt).
|
||||
"""
|
||||
entry = entries.get((ws_id, cycle_id))
|
||||
if entry is None and not cycle_id:
|
||||
entry = next((v for (wid, _), v in entries.items() if wid == ws_id), None)
|
||||
return entry
|
||||
|
||||
|
||||
def pop_cycle_entry(
|
||||
entries: MutableMapping[tuple[str, str], _V],
|
||||
ws_id: str,
|
||||
cycle_id: str,
|
||||
) -> _V | None:
|
||||
"""Like :func:`get_cycle_entry`, but removes the matched entry."""
|
||||
entry = entries.pop((ws_id, cycle_id), None)
|
||||
if entry is None and not cycle_id:
|
||||
key = next((k for k in entries if k[0] == ws_id), None)
|
||||
if key is not None:
|
||||
entry = entries.pop(key, None)
|
||||
return entry
|
||||
|
||||
|
||||
def pop_ws_entries(entries: MutableMapping[tuple[str, str], _V], ws_id: str) -> None:
|
||||
"""Drop every entry tracked under *ws_id* (all cycles)."""
|
||||
for key in [k for k in entries if k[0] == ws_id]:
|
||||
entries.pop(key, None)
|
||||
|
||||
|
||||
class ChannelRouter:
|
||||
"""Manage channel-to-workstream routing via SDK clients.
|
||||
|
||||
@@ -428,15 +481,33 @@ class ChannelRouter:
|
||||
feedback: str = "",
|
||||
always: bool = False,
|
||||
) -> None:
|
||||
"""Approve or deny a pending tool call via the server API."""
|
||||
"""Approve or deny a pending tool call via the server API.
|
||||
|
||||
``correlation_id`` is the cycle_id captured from the
|
||||
``approve_request`` event the adapter displayed; forwarding it
|
||||
makes the decision land on exactly that cycle. With parallel
|
||||
task agents a workstream can hold several prompts — a
|
||||
selector-less approve would resolve the OLDEST, which may not
|
||||
be the message the user answered. Empty string (policy /
|
||||
auto-approve sweeps that act on "whatever is pending") keeps
|
||||
the legacy oldest-first behavior.
|
||||
"""
|
||||
if self._console:
|
||||
await self._console.route_approve(
|
||||
ws_id=ws_id, approved=approved, feedback=feedback, always=always
|
||||
ws_id=ws_id,
|
||||
approved=approved,
|
||||
feedback=feedback,
|
||||
always=always,
|
||||
cycle_id=correlation_id,
|
||||
)
|
||||
else:
|
||||
assert self._server is not None
|
||||
await self._server.approve(
|
||||
ws_id=ws_id, approved=approved, feedback=feedback or None, always=always
|
||||
ws_id=ws_id,
|
||||
approved=approved,
|
||||
feedback=feedback or None,
|
||||
always=always,
|
||||
cycle_id=correlation_id or None,
|
||||
)
|
||||
log.debug(
|
||||
"channel_router.send_approval",
|
||||
|
||||
@@ -23,7 +23,7 @@ import httpx
|
||||
|
||||
from turnstone.channels._config import MAX_NOTIFY_TRACKING
|
||||
from turnstone.channels._formatter import chunk_message
|
||||
from turnstone.channels._routing import ChannelRouter
|
||||
from turnstone.channels._routing import ChannelRouter, pop_cycle_entry, pop_ws_entries
|
||||
from turnstone.channels._sse import run_sse_stream
|
||||
from turnstone.core.log import get_logger
|
||||
from turnstone.sdk.events import (
|
||||
@@ -235,10 +235,15 @@ class TurnstoneBot:
|
||||
# List preserves call order for FIFO matching when the same tool name
|
||||
# appears more than once in a single turn.
|
||||
self._tool_info_msgs: dict[str, list[tuple[str, str, str, discord.Message]]] = {}
|
||||
# Track the Discord message containing the pending approval embed per
|
||||
# workstream so that IntentVerdictEvent can update it with LLM judge
|
||||
# results.
|
||||
self._pending_approval_msgs: dict[str, discord.Message] = {}
|
||||
# Track the Discord message containing each pending approval embed,
|
||||
# keyed by (ws_id, cycle_id) — parallel task agents make several
|
||||
# approval cycles live per workstream at once, each its own embed.
|
||||
# The value carries the cycle's member call_ids so
|
||||
# IntentVerdictEvent (keyed by call_id) updates the RIGHT embed
|
||||
# with LLM judge results.
|
||||
self._pending_approval_msgs: dict[
|
||||
tuple[str, str], tuple[discord.Message, frozenset[str]]
|
||||
] = {}
|
||||
# Notification reply tracking: maps Discord message ID ->
|
||||
# (ws_id, target_discord_user_id) so that DM replies can be routed
|
||||
# back to the originating workstream. The target user ID is checked
|
||||
@@ -413,13 +418,17 @@ class TurnstoneBot:
|
||||
with contextlib.suppress(Exception):
|
||||
await thinking_msg.delete()
|
||||
self._tool_info_msgs.pop(ws_id, None)
|
||||
self._pending_approval_msgs.pop(ws_id, None)
|
||||
self._pop_ws_approvals(ws_id)
|
||||
self._notify_reply_channels.pop(ws_id, None)
|
||||
# Purge stale notification tracking entries for this workstream.
|
||||
stale = [mid for mid, entry in self._notify_ws_map.items() if entry[0] == ws_id]
|
||||
for mid in stale:
|
||||
del self._notify_ws_map[mid]
|
||||
|
||||
def _pop_ws_approvals(self, ws_id: str) -> None:
|
||||
"""Drop every tracked approval embed for *ws_id* (all cycles)."""
|
||||
pop_ws_entries(self._pending_approval_msgs, ws_id)
|
||||
|
||||
async def unsubscribe_ws(self, ws_id: str) -> None:
|
||||
"""Cancel the SSE listener for *ws_id* and clean up streaming state."""
|
||||
task = self._sse_tasks.pop(ws_id, None)
|
||||
@@ -680,6 +689,10 @@ class TurnstoneBot:
|
||||
from turnstone.channels._formatter import format_approval_request, format_verdict
|
||||
from turnstone.channels.discord.views import ApprovalView
|
||||
|
||||
# Every decision below is about THIS event's cycle — forward its
|
||||
# cycle_id so the resolution can't land on a sibling round
|
||||
# (parallel task agents can have several prompts outstanding).
|
||||
cycle_id = event.cycle_id
|
||||
# Evaluate admin tool policies before auto-approve.
|
||||
policy_verdict = await self.router.evaluate_tool_policies(event.items)
|
||||
policy_handled = False
|
||||
@@ -687,22 +700,19 @@ class TurnstoneBot:
|
||||
denied = ", ".join(policy_verdict.denied_tools)
|
||||
await self.router.send_approval(
|
||||
ws_id,
|
||||
"",
|
||||
cycle_id,
|
||||
approved=False,
|
||||
feedback=f"Blocked by tool policy: {denied}",
|
||||
)
|
||||
await thread.send(f"*Tool blocked by admin policy: {denied}*")
|
||||
policy_handled = True
|
||||
elif policy_verdict.kind == "allow":
|
||||
await self.router.send_approval(ws_id, "", approved=True)
|
||||
await self.router.send_approval(ws_id, cycle_id, approved=True)
|
||||
await thread.send("*Tool approved by policy.*")
|
||||
policy_handled = True
|
||||
|
||||
if not policy_handled and (self.config.auto_approve or self._should_auto_approve(event)):
|
||||
# correlation_id is empty because the server's /api/approve
|
||||
# endpoint resolves approvals by ws_id alone (one pending
|
||||
# approval per workstream at a time).
|
||||
await self.router.send_approval(ws_id, "", approved=True)
|
||||
await self.router.send_approval(ws_id, cycle_id, approved=True)
|
||||
await thread.send("*Tool auto-approved.*")
|
||||
elif not policy_handled:
|
||||
text = format_approval_request(event.items)
|
||||
@@ -721,9 +731,15 @@ class TurnstoneBot:
|
||||
value=format_verdict(verdict),
|
||||
inline=False,
|
||||
)
|
||||
embed.set_footer(text=f"{ws_id}||{_thread_owner_id(thread)}")
|
||||
# Footer format ws_id|cycle_id|owner — the persistent view
|
||||
# parses it back on click and routes the decision to exactly
|
||||
# this cycle.
|
||||
embed.set_footer(text=f"{ws_id}|{cycle_id}|{_thread_owner_id(thread)}")
|
||||
msg = await thread.send(embed=embed, view=ApprovalView(self)._view)
|
||||
self._pending_approval_msgs[ws_id] = msg
|
||||
call_ids = frozenset(
|
||||
str(it.get("call_id", "")) for it in event.items if it.get("call_id")
|
||||
)
|
||||
self._pending_approval_msgs[(ws_id, cycle_id)] = (msg, call_ids)
|
||||
|
||||
async def _handle_intent_verdict(
|
||||
self,
|
||||
@@ -734,8 +750,18 @@ class TurnstoneBot:
|
||||
|
||||
from turnstone.channels._formatter import format_verdict
|
||||
|
||||
# LLM judge verdict arrived — update the pending approval embed.
|
||||
approval_msg = self._pending_approval_msgs.get(ws_id)
|
||||
# LLM judge verdict arrived — update the pending approval embed
|
||||
# whose cycle contains this call_id (several can be live at once
|
||||
# under parallel task agents). Legacy entries (empty call_ids)
|
||||
# accept any verdict for the ws, as before.
|
||||
approval_msg = next(
|
||||
(
|
||||
msg
|
||||
for (wid, _), (msg, call_ids) in self._pending_approval_msgs.items()
|
||||
if wid == ws_id and (not call_ids or event.call_id in call_ids)
|
||||
),
|
||||
None,
|
||||
)
|
||||
if approval_msg and approval_msg.embeds:
|
||||
embed = approval_msg.embeds[0]
|
||||
verdict_data = {
|
||||
@@ -771,9 +797,12 @@ class TurnstoneBot:
|
||||
event: ApprovalResolvedEvent,
|
||||
) -> None:
|
||||
# Server resolved the approval (timeout, external approve/reject).
|
||||
# Disable the buttons so they can't be clicked stale.
|
||||
approval_msg = self._pending_approval_msgs.pop(ws_id, None)
|
||||
if approval_msg is not None:
|
||||
# Disable the buttons so they can't be clicked stale. Route by
|
||||
# the event's cycle_id; a pre-multi-cycle server (no cycle_id)
|
||||
# clears the ws's single tracked entry, as before.
|
||||
entry = pop_cycle_entry(self._pending_approval_msgs, ws_id, event.cycle_id)
|
||||
if entry is not None:
|
||||
approval_msg = entry[0]
|
||||
from turnstone.channels.discord.views import disable_message_buttons
|
||||
|
||||
label = "Approved" if event.approved else "Denied"
|
||||
@@ -809,8 +838,8 @@ class TurnstoneBot:
|
||||
# for multi-turn DM conversations.
|
||||
if last_msg is not None:
|
||||
self._track_notification(last_msg.id, ws_id, target_user_id)
|
||||
# Clean up pending approval message tracking.
|
||||
self._pending_approval_msgs.pop(ws_id, None)
|
||||
# Clean up pending approval message tracking (all cycles).
|
||||
self._pop_ws_approvals(ws_id)
|
||||
|
||||
# -- helpers -------------------------------------------------------------
|
||||
|
||||
|
||||
@@ -9,6 +9,7 @@ from __future__ import annotations
|
||||
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
from turnstone.channels._routing import pop_cycle_entry
|
||||
from turnstone.core.log import get_logger
|
||||
|
||||
if TYPE_CHECKING:
|
||||
@@ -187,7 +188,9 @@ class ApprovalView:
|
||||
|
||||
label = "Always Approved" if always else ("Approved" if approved else "Rejected")
|
||||
# Pop pending approval so ApprovalResolvedEvent doesn't double-update.
|
||||
self.bot._pending_approval_msgs.pop(ws_id, None)
|
||||
# Keyed by (ws_id, cycle_id); a legacy footer without a cycle_id
|
||||
# clears the ws's single tracked entry.
|
||||
pop_cycle_entry(self.bot._pending_approval_msgs, ws_id, correlation_id)
|
||||
await _disable_buttons(interaction, label)
|
||||
await interaction.followup.send(
|
||||
f"Tool execution **{label.lower()}**.",
|
||||
|
||||
@@ -36,7 +36,12 @@ from slack_sdk.web.async_client import AsyncWebClient
|
||||
|
||||
from turnstone.channels._config import MAX_NOTIFY_TRACKING
|
||||
from turnstone.channels._formatter import chunk_message
|
||||
from turnstone.channels._routing import ChannelRouter
|
||||
from turnstone.channels._routing import (
|
||||
ChannelRouter,
|
||||
get_cycle_entry,
|
||||
pop_cycle_entry,
|
||||
pop_ws_entries,
|
||||
)
|
||||
from turnstone.channels._sse import run_sse_stream
|
||||
from turnstone.channels.slack.routes import SlackRoute
|
||||
from turnstone.core.log import get_logger
|
||||
@@ -128,6 +133,13 @@ class PendingApproval:
|
||||
# approval message (matches the pre-refactor "fetch live blocks
|
||||
# and append" behavior).
|
||||
blocks: list[dict[str, Any]] = field(default_factory=list)
|
||||
# Approval-cycle identity + member call_ids. A workstream can hold
|
||||
# several concurrent cycles (parallel task agents), each its own
|
||||
# Slack message; the cycle_id routes the button click to exactly
|
||||
# this round and call_ids route IntentVerdictEvents (keyed by
|
||||
# call_id) onto the right message.
|
||||
cycle_id: str = ""
|
||||
call_ids: frozenset[str] = frozenset()
|
||||
|
||||
|
||||
@dataclass
|
||||
@@ -275,7 +287,11 @@ class TurnstoneSlackBot:
|
||||
self._sse_tasks: dict[str, asyncio.Task[None]] = {}
|
||||
self._streaming: dict[str, StreamingMessage] = {}
|
||||
|
||||
self._pending_approval: dict[str, PendingApproval] = {}
|
||||
# Keyed by (ws_id, cycle_id) — one entry per approval MESSAGE.
|
||||
# Parallel task agents make several cycles live per ws at once;
|
||||
# the old ws_id-only key silently replaced the tracked message
|
||||
# so verdict edits and resolution updates hit the wrong prompt.
|
||||
self._pending_approval: dict[tuple[str, str], PendingApproval] = {}
|
||||
self._notify_ws_map: dict[str, tuple[str, SlackRoute]] = {}
|
||||
# Per-workstream override used to route the next streamed assistant
|
||||
# response back into a Slack notification reply thread instead of the
|
||||
@@ -865,7 +881,11 @@ class TurnstoneSlackBot:
|
||||
return
|
||||
|
||||
ws_id, correlation_id = parts
|
||||
entry = self._pending_approval.get(ws_id)
|
||||
# correlation_id = the cycle_id stamped into the button value at
|
||||
# post time; pre-multi-cycle messages carry "" — fall back to
|
||||
# the ws's single tracked entry so an in-flight upgrade doesn't
|
||||
# orphan an already-posted prompt.
|
||||
entry = get_cycle_entry(self._pending_approval, ws_id, correlation_id)
|
||||
actor_user_id = body.get("user", {}).get("id", "")
|
||||
channel = body["container"]["channel_id"]
|
||||
verb = "approve" if approved else "deny"
|
||||
@@ -899,12 +919,12 @@ class TurnstoneSlackBot:
|
||||
)
|
||||
return
|
||||
|
||||
await self.router.send_approval(ws_id, correlation_id, approved=approved)
|
||||
await self.router.send_approval(ws_id, entry.cycle_id, approved=approved)
|
||||
# Drop the pending entry now that we've handled it locally. Otherwise
|
||||
# the subsequent ApprovalResolvedEvent will rewrite the message a
|
||||
# second time ("Tool approved" → "Approved") — wasted chat_update and
|
||||
# a visible edit flicker.
|
||||
self._pending_approval.pop(ws_id, None)
|
||||
self._pending_approval.pop((ws_id, entry.cycle_id), None)
|
||||
ts = body["container"]["message_ts"]
|
||||
update_text = "Tool approved" if approved else "Tool denied"
|
||||
event_key = "approve" if approved else "deny"
|
||||
@@ -941,6 +961,10 @@ class TurnstoneSlackBot:
|
||||
self._subscribed_ws.add(ws_id)
|
||||
log.info("slack.subscribed", ws_id=ws_id, channel_id=channel_id)
|
||||
|
||||
def _pop_ws_approvals(self, ws_id: str) -> None:
|
||||
"""Drop every tracked approval message for *ws_id* (all cycles)."""
|
||||
pop_ws_entries(self._pending_approval, ws_id)
|
||||
|
||||
def _clear_ws_state(self, ws_id: str) -> None:
|
||||
"""Drop all in-memory state keyed by *ws_id*.
|
||||
|
||||
@@ -950,7 +974,7 @@ class TurnstoneSlackBot:
|
||||
"""
|
||||
self._subscribed_ws.discard(ws_id)
|
||||
self._streaming.pop(ws_id, None)
|
||||
self._pending_approval.pop(ws_id, None)
|
||||
self._pop_ws_approvals(ws_id)
|
||||
self._clear_notification_tracking_for_ws(ws_id)
|
||||
|
||||
async def unsubscribe_ws(self, ws_id: str) -> None:
|
||||
@@ -1062,13 +1086,17 @@ class TurnstoneSlackBot:
|
||||
thread_ts = route.thread_ts or ""
|
||||
owner_user_id = route.user_id
|
||||
|
||||
# Every decision this handler takes is about THIS event's cycle —
|
||||
# forward its cycle_id so the resolution can't land on a sibling
|
||||
# round (parallel task agents can have several outstanding).
|
||||
cycle_id = event.cycle_id
|
||||
verdict = await self.router.evaluate_tool_policies(event.items)
|
||||
policy_handled = False
|
||||
if verdict.kind == "deny":
|
||||
denied = ", ".join(verdict.denied_tools)
|
||||
await self.router.send_approval(
|
||||
ws_id,
|
||||
"",
|
||||
cycle_id,
|
||||
approved=False,
|
||||
feedback=f"Blocked by tool policy: {denied}",
|
||||
)
|
||||
@@ -1079,7 +1107,7 @@ class TurnstoneSlackBot:
|
||||
)
|
||||
policy_handled = True
|
||||
elif verdict.kind == "allow":
|
||||
await self.router.send_approval(ws_id, "", approved=True)
|
||||
await self.router.send_approval(ws_id, cycle_id, approved=True)
|
||||
await self._client.chat_postMessage(
|
||||
channel=slack_channel,
|
||||
thread_ts=thread_ts or None,
|
||||
@@ -1088,7 +1116,7 @@ class TurnstoneSlackBot:
|
||||
policy_handled = True
|
||||
|
||||
if not policy_handled and (self.config.auto_approve or self._should_auto_approve(event)):
|
||||
await self.router.send_approval(ws_id, "", approved=True)
|
||||
await self.router.send_approval(ws_id, cycle_id, approved=True)
|
||||
await self._client.chat_postMessage(
|
||||
channel=slack_channel,
|
||||
thread_ts=thread_ts or None,
|
||||
@@ -1097,7 +1125,7 @@ class TurnstoneSlackBot:
|
||||
elif not policy_handled:
|
||||
await self._send_approval_request(
|
||||
ws_id,
|
||||
"",
|
||||
cycle_id,
|
||||
event.items,
|
||||
slack_channel,
|
||||
thread_ts,
|
||||
@@ -1105,7 +1133,18 @@ class TurnstoneSlackBot:
|
||||
)
|
||||
|
||||
async def _handle_intent_verdict(self, ws_id: str, event: IntentVerdictEvent) -> None:
|
||||
entry = self._pending_approval.get(ws_id)
|
||||
# Route the verdict onto the message whose cycle contains this
|
||||
# call_id — with several prompts live, editing "the" entry would
|
||||
# stack the judge section on the wrong message. Legacy entries
|
||||
# (empty call_ids) accept any verdict for the ws, as before.
|
||||
entry = next(
|
||||
(
|
||||
v
|
||||
for (wid, _), v in self._pending_approval.items()
|
||||
if wid == ws_id and (not v.call_ids or event.call_id in v.call_ids)
|
||||
),
|
||||
None,
|
||||
)
|
||||
if entry is None:
|
||||
return
|
||||
|
||||
@@ -1138,7 +1177,9 @@ class TurnstoneSlackBot:
|
||||
log.debug("slack.verdict_message_update_failed", ws_id=ws_id, exc_info=True)
|
||||
|
||||
async def _handle_approval_resolved(self, ws_id: str, event: ApprovalResolvedEvent) -> None:
|
||||
entry = self._pending_approval.pop(ws_id, None)
|
||||
# Route by the event's cycle_id; a pre-multi-cycle server (no
|
||||
# cycle_id) clears the ws's single tracked entry, as before.
|
||||
entry = pop_cycle_entry(self._pending_approval, ws_id, event.cycle_id)
|
||||
if entry is None:
|
||||
return
|
||||
|
||||
@@ -1174,7 +1215,7 @@ class TurnstoneSlackBot:
|
||||
):
|
||||
self._track_notification(sm.message_ts, ws_id, reply_route)
|
||||
|
||||
self._pending_approval.pop(ws_id, None)
|
||||
self._pop_ws_approvals(ws_id)
|
||||
|
||||
async def _handle_error(self, route: SlackRoute, event: ErrorEvent) -> None:
|
||||
safe_msg = event.message[:500] if event.message else "An error occurred"
|
||||
@@ -1254,15 +1295,18 @@ class TurnstoneSlackBot:
|
||||
blocks=cast("list[dict[str, Any]]", blocks),
|
||||
)
|
||||
if resp.get("ok"):
|
||||
self._pending_approval[ws_id] = PendingApproval(
|
||||
self._pending_approval[(ws_id, correlation_id)] = PendingApproval(
|
||||
channel=channel,
|
||||
message_ts=resp["ts"],
|
||||
owner_user_id=owner_user_id,
|
||||
blocks=list(cast("list[dict[str, Any]]", blocks)),
|
||||
cycle_id=correlation_id,
|
||||
call_ids=frozenset(str(it.get("call_id", "")) for it in items if it.get("call_id")),
|
||||
)
|
||||
log.info(
|
||||
"slack.pending_approval_stored",
|
||||
ws_id=ws_id,
|
||||
cycle_id=correlation_id,
|
||||
channel=channel,
|
||||
thread_ts=thread_ts,
|
||||
owner_user_id=owner_user_id,
|
||||
|
||||
+7
-2
@@ -324,8 +324,13 @@ class TerminalUI(SessionUI):
|
||||
def on_state_change(self, state: str) -> None:
|
||||
pass # base TerminalUI ignores state changes
|
||||
|
||||
def on_intent_verdict(self, verdict: dict[str, Any]) -> None:
|
||||
"""Display LLM judge verdict — called from daemon thread while approval is pending."""
|
||||
def on_intent_verdict(self, verdict: dict[str, Any], judge_event: object | None = None) -> None:
|
||||
"""Display LLM judge verdict — called from daemon thread while approval is pending.
|
||||
|
||||
``judge_event`` (the delivering judge generation) is a
|
||||
cycle-registry concern — the terminal renders every verdict it
|
||||
receives and ignores it.
|
||||
"""
|
||||
risk = verdict.get("risk_level", "medium")
|
||||
rec = verdict.get("recommendation", "review")
|
||||
summary = verdict.get("intent_summary", "")
|
||||
|
||||
@@ -733,6 +733,9 @@ class ClusterCollector:
|
||||
# coordinator's tree UI can clear the pending-approval
|
||||
# pill the moment the user decides, rather than waiting
|
||||
# for the subsequent state-change piggyback.
|
||||
# ``cycle_id`` / ``call_ids`` name WHICH cycle resolved —
|
||||
# a child running parallel task agents can have several
|
||||
# approval blocks live at once.
|
||||
ws_id = data.get("ws_id", "")
|
||||
if ws_id:
|
||||
pending_events.append(
|
||||
@@ -743,6 +746,8 @@ class ClusterCollector:
|
||||
"approved": bool(data.get("approved", False)),
|
||||
"feedback": data.get("feedback", "") or "",
|
||||
"always": bool(data.get("always", False)),
|
||||
"cycle_id": data.get("cycle_id", "") or "",
|
||||
"call_ids": data.get("call_ids") or [],
|
||||
}
|
||||
)
|
||||
|
||||
@@ -1391,6 +1396,8 @@ class ClusterCollector:
|
||||
approved: bool,
|
||||
feedback: str = "",
|
||||
always: bool = False,
|
||||
cycle_id: str = "",
|
||||
call_ids: list[str] | None = None,
|
||||
) -> None:
|
||||
"""Fan an ``approval_resolved`` decision for a console-pseudo-node ws.
|
||||
|
||||
@@ -1407,6 +1414,8 @@ class ClusterCollector:
|
||||
"approved": approved,
|
||||
"feedback": feedback,
|
||||
"always": always,
|
||||
"cycle_id": cycle_id,
|
||||
"call_ids": call_ids or [],
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
@@ -670,6 +670,11 @@ class CoordinatorAdapter:
|
||||
"approved": bool(event.get("approved", False)),
|
||||
"feedback": event.get("feedback", "") or "",
|
||||
"always": bool(event.get("always", False)),
|
||||
# Which cycle resolved — the tree row can hold several
|
||||
# approval blocks when the child runs parallel task
|
||||
# agents; empty (legacy node) clears them all.
|
||||
"cycle_id": event.get("cycle_id", "") or "",
|
||||
"call_ids": event.get("call_ids") or [],
|
||||
}
|
||||
else: # approve_request
|
||||
# Push path for the initial approval items —
|
||||
|
||||
@@ -5,8 +5,8 @@ Mirrors ``turnstone.server.WebUI`` but scoped to the console's needs:
|
||||
- Per-session SSE listener fan-out (inherited from
|
||||
:class:`SessionUIBase` — same ``threading.Lock`` + queue list
|
||||
pattern WebUI uses).
|
||||
- ``threading.Event`` + ``_approval_result`` for blocking the worker
|
||||
thread until a console endpoint delivers the decision (inherited).
|
||||
- Per-cycle ``ApprovalCycle`` registry for blocking each gate thread
|
||||
until a console endpoint delivers its decision (inherited).
|
||||
- Per-ws metric tracking + turn-content accumulator + activity
|
||||
bookkeeping (inherited from :class:`SessionUIBase` post the rich
|
||||
``ws_state`` payload lift). Coord populates the same
|
||||
@@ -196,6 +196,8 @@ class ConsoleCoordinatorUI(SessionUIBase):
|
||||
feedback: str | None = None,
|
||||
*,
|
||||
always: bool = False,
|
||||
cycle_id: str = "",
|
||||
call_ids: tuple[str, ...] = (),
|
||||
) -> None:
|
||||
"""Fan an ``approval_resolved`` decision to the cluster collector.
|
||||
|
||||
@@ -213,6 +215,8 @@ class ConsoleCoordinatorUI(SessionUIBase):
|
||||
approved=approved,
|
||||
feedback=feedback or "",
|
||||
always=always,
|
||||
cycle_id=cycle_id,
|
||||
call_ids=list(call_ids),
|
||||
)
|
||||
except Exception:
|
||||
log.debug(
|
||||
@@ -317,8 +321,12 @@ class ConsoleCoordinatorUI(SessionUIBase):
|
||||
return
|
||||
fire_judge_verdict_metric(cm, verdict, "heuristic")
|
||||
|
||||
def on_intent_verdict(self, verdict: dict[str, Any]) -> None:
|
||||
super().on_intent_verdict(verdict)
|
||||
def on_intent_verdict(
|
||||
self,
|
||||
verdict: dict[str, Any],
|
||||
judge_event: object | None = None,
|
||||
) -> None:
|
||||
super().on_intent_verdict(verdict, judge_event)
|
||||
cm = ConsoleCoordinatorUI._console_metrics
|
||||
if cm is None:
|
||||
return
|
||||
|
||||
+129
-28
@@ -1047,12 +1047,14 @@ _CLUSTER_WS_LIVE_KEYS = (
|
||||
"model_alias",
|
||||
"title",
|
||||
"name",
|
||||
# Carries the inline approve/deny payload (items + judge_verdict)
|
||||
# so coord live-bulk callers can render row-level UI without a
|
||||
# per-child round-trip. ``None`` when no approval is pending.
|
||||
# Cross-tenant exposure follows the trusted-team posture documented
|
||||
# on ``SessionUIBase.serialize_pending_approval_detail``.
|
||||
"pending_approval_detail",
|
||||
# Carries the inline approve/deny payloads (one per live cycle,
|
||||
# items + judge_verdict each) so coord live-bulk callers can render
|
||||
# row-level UI without a per-child round-trip. ``[]`` when no
|
||||
# approval is pending; several entries when parallel task agents
|
||||
# gate concurrently. Cross-tenant exposure follows the trusted-team
|
||||
# posture documented on
|
||||
# ``SessionUIBase.serialize_pending_approval_details``.
|
||||
"pending_approval_details",
|
||||
# Ring buffer of the child's recent auto-approves (last 10) for
|
||||
# the coord-tree's "auto-approved by skill X" pill. Without this
|
||||
# the operator has no surface to see WHICH tool calls bypassed
|
||||
@@ -1170,16 +1172,16 @@ def _coordinator_live_snapshot(ws: Any) -> dict[str, Any]:
|
||||
val = getattr(obj, name, "") if obj else ""
|
||||
return val if isinstance(val, str) else ""
|
||||
|
||||
# Coord rows synthesize the same ``pending_approval_detail`` shape
|
||||
# Coord rows synthesize the same ``pending_approval_details`` shape
|
||||
# the node-side dashboard produces — single source of truth via
|
||||
# ``SessionUIBase.serialize_pending_approval_detail``. The console
|
||||
# ``SessionUIBase.serialize_pending_approval_details``. The console
|
||||
# coord LLM judge isn't wired today (``coordinator_ui.py:138``
|
||||
# hardcodes ``judge_pending=False``), so ``judge_verdict`` will
|
||||
# always be ``None`` for these rows; the coord-self stretch in
|
||||
# the plan covers that follow-up. ``ui`` may be ``None`` in
|
||||
# transient states (newly-created ws before activation); every
|
||||
# active coord UI is a ``SessionUIBase`` and supports the method.
|
||||
pending_approval_detail = ui.serialize_pending_approval_detail() if ui is not None else None
|
||||
pending_approval_details = ui.serialize_pending_approval_details() if ui is not None else []
|
||||
recent_auto_approvals = ui.serialize_recent_auto_approvals() if ui is not None else []
|
||||
|
||||
return {
|
||||
@@ -1194,7 +1196,7 @@ def _coordinator_live_snapshot(ws: Any) -> dict[str, Any]:
|
||||
"title": "",
|
||||
"name": getattr(ws, "name", "") or "",
|
||||
"pending_approval": pending_approval,
|
||||
"pending_approval_detail": pending_approval_detail,
|
||||
"pending_approval_details": pending_approval_details,
|
||||
"recent_auto_approvals": recent_auto_approvals,
|
||||
}
|
||||
|
||||
@@ -1286,15 +1288,15 @@ async def _fetch_live_block(
|
||||
# ``activity_state="approval"`` is set inside approve_tools
|
||||
# AFTER the state transition fires, so a bulk fetch that
|
||||
# races with that window can see state=attention and
|
||||
# activity_state="" simultaneously. A non-null
|
||||
# ``pending_approval_detail`` is also a definitive signal
|
||||
# (the serializer only emits non-None when ``_pending_approval``
|
||||
# is set on the UI). Any of the three flips this true; the
|
||||
# frontend reducer mirrors the same disjunction.
|
||||
# activity_state="" simultaneously. A non-empty
|
||||
# ``pending_approval_details`` is also a definitive signal
|
||||
# (the serializer emits entries only for live cycles). Any
|
||||
# of the three flips this true; the frontend reducer
|
||||
# mirrors the same disjunction.
|
||||
live["pending_approval"] = (
|
||||
live.get("activity_state") == "approval"
|
||||
or entry.get("state") == "attention"
|
||||
or live.get("pending_approval_detail") is not None
|
||||
or bool(live.get("pending_approval_details"))
|
||||
)
|
||||
return live
|
||||
return None
|
||||
@@ -1908,8 +1910,45 @@ async def list_available_models(request: Request) -> JSONResponse:
|
||||
return err
|
||||
|
||||
rows = storage.list_model_definitions(enabled_only=True)
|
||||
# Only expose alias/model/provider — rows also contain api_key, base_url, etc.
|
||||
models = [{"alias": r["alias"], "model": r["model"], "provider": r["provider"]} for r in rows]
|
||||
# Only expose alias/model/provider (+ the derived effort ladder) —
|
||||
# rows also contain api_key, base_url, etc.
|
||||
from turnstone.core.providers.effort_ladder import effort_ladder_for_model
|
||||
|
||||
models = []
|
||||
for r in rows:
|
||||
# ``effort_ladder`` starts as the empty list so the row schema is
|
||||
# stable even when the try block below bails on a malformed
|
||||
# capabilities column — clients can index the key unconditionally.
|
||||
entry: dict[str, Any] = {
|
||||
"alias": r["alias"],
|
||||
"model": r["model"],
|
||||
"provider": r["provider"],
|
||||
"effort_ladder": [],
|
||||
}
|
||||
try:
|
||||
# ``capabilities`` is a JSON string (sa.Text column) with
|
||||
# ``server_compat`` namespaced inside — parse it the same way
|
||||
# the model_registry loader does. Read (don't pop) the
|
||||
# namespace: the ladder's field filter drops non-capability
|
||||
# keys, so the parsed dict can stay unmutated.
|
||||
caps: dict[str, Any] = {}
|
||||
raw_caps = r.get("capabilities")
|
||||
if raw_caps:
|
||||
parsed = json.loads(raw_caps)
|
||||
if isinstance(parsed, dict):
|
||||
caps = parsed
|
||||
server_compat = caps.get("server_compat", {})
|
||||
api_surface = (
|
||||
server_compat.get("api_surface", "") if isinstance(server_compat, dict) else ""
|
||||
)
|
||||
entry["effort_ladder"] = effort_ladder_for_model(
|
||||
r["provider"], r["model"], caps, api_surface=str(api_surface or "")
|
||||
)
|
||||
except Exception:
|
||||
# Unknown provider string / malformed capabilities row must
|
||||
# not take down the picker — the ladder is an annotation.
|
||||
log.debug("models.effort_ladder_failed alias=%s", r.get("alias"), exc_info=True)
|
||||
models.append(entry)
|
||||
|
||||
# Include effective defaults for clients (web UI, channel gateway).
|
||||
default_alias = ""
|
||||
@@ -3528,16 +3567,18 @@ def _coord_events_replay(
|
||||
"""
|
||||
yield from session_replay_preamble(ws.session, ui)
|
||||
|
||||
pending_approval = getattr(ui, "_pending_approval", None)
|
||||
if pending_approval is not None:
|
||||
yield pending_approval
|
||||
# Cached LLM verdicts that fired since the approval prompt
|
||||
# — without this replay, a reconnecting / refreshing tab
|
||||
# sees the approve_request prompt but no judge chip, and
|
||||
# since intent_verdict only fires once per call_id (no
|
||||
# push to a late subscriber), the chip would never appear
|
||||
# until the operator re-invokes the action. Mirrors the
|
||||
# interactive path at ``turnstone/server.py:875-878``.
|
||||
# EVERY live approval cycle replays (parallel task agents can have
|
||||
# several outstanding), each card followed once by the cached LLM
|
||||
# verdicts — without this replay, a reconnecting / refreshing tab
|
||||
# sees the approve_request prompts but no judge chips, and since
|
||||
# intent_verdict only fires once per call_id (no push to a late
|
||||
# subscriber), the chips would never appear until the operator
|
||||
# re-invokes the action. Mirrors the interactive path in
|
||||
# ``turnstone/server.py`` (fresh-connect replay).
|
||||
cards_fn = getattr(ui, "pending_approval_cards", None)
|
||||
pending_cards = cards_fn() if callable(cards_fn) else []
|
||||
if pending_cards:
|
||||
yield from pending_cards
|
||||
llm_verdicts = getattr(ui, "_llm_verdicts", None)
|
||||
ws_lock = getattr(ui, "_ws_lock", None)
|
||||
if llm_verdicts and ws_lock is not None:
|
||||
@@ -11592,6 +11633,61 @@ async def admin_model_capabilities(request: Request) -> JSONResponse:
|
||||
)
|
||||
|
||||
|
||||
async def admin_effort_ladder(request: Request) -> JSONResponse:
|
||||
"""POST /v1/api/admin/models/effort-ladder — knob→wire projection.
|
||||
|
||||
Body: ``{"provider": ..., "model": ..., "capabilities": {...}}`` —
|
||||
capabilities are the (possibly unsaved) overrides from the model
|
||||
form, so the modal can annotate its effort select live while the
|
||||
operator edits thinking mode / effort param. Pure computation; no
|
||||
stored state is read or written.
|
||||
"""
|
||||
from turnstone.core.auth import require_permission
|
||||
from turnstone.core.providers.effort_ladder import effort_ladder_for_model
|
||||
|
||||
err = require_permission(request, "admin.models")
|
||||
if err:
|
||||
return err
|
||||
|
||||
try:
|
||||
body = await request.json()
|
||||
except Exception:
|
||||
return JSONResponse({"error": "invalid JSON body"}, status_code=400)
|
||||
if not isinstance(body, dict):
|
||||
# json() happily parses null/arrays/strings — .get() on those
|
||||
# would 500 where the contract says 400.
|
||||
return JSONResponse({"error": "body must be a JSON object"}, status_code=400)
|
||||
provider = str(body.get("provider") or "").strip()
|
||||
model = str(body.get("model") or "").strip()
|
||||
capabilities = body.get("capabilities")
|
||||
api_surface = str(body.get("api_surface") or "").strip()
|
||||
|
||||
if provider not in _MODEL_PROVIDERS:
|
||||
return JSONResponse({"error": f"Unknown provider: {provider!r}"}, status_code=400)
|
||||
if not model:
|
||||
return JSONResponse({"error": "model is required"}, status_code=400)
|
||||
if capabilities is not None and not isinstance(capabilities, dict):
|
||||
return JSONResponse({"error": "capabilities must be an object"}, status_code=400)
|
||||
|
||||
try:
|
||||
ladder = effort_ladder_for_model(
|
||||
provider, model, capabilities or {}, api_surface=api_surface
|
||||
)
|
||||
except Exception:
|
||||
# Garbage override values (wrong types for capability fields)
|
||||
# surface as a clean 400 rather than a 500 — the form's raw
|
||||
# JSON is operator-typed. Log it: the same catch would
|
||||
# otherwise mask a genuine resolver bug as a silent 400.
|
||||
log.warning(
|
||||
"models.effort_ladder_resolve_failed provider=%s model=%s",
|
||||
provider,
|
||||
model,
|
||||
exc_info=True,
|
||||
)
|
||||
return JSONResponse({"error": "could not resolve capabilities"}, status_code=400)
|
||||
return JSONResponse({"provider": provider, "model": model, "ladder": ladder})
|
||||
|
||||
|
||||
async def admin_known_models(request: Request) -> JSONResponse:
|
||||
"""GET /v1/api/admin/model-capabilities/known — list known model name prefixes."""
|
||||
from turnstone.core.auth import require_permission
|
||||
@@ -13988,6 +14084,11 @@ def create_app(
|
||||
"/api/admin/model-capabilities/known",
|
||||
admin_known_models,
|
||||
),
|
||||
Route(
|
||||
"/api/admin/models/effort-ladder",
|
||||
admin_effort_ladder,
|
||||
methods=["POST"],
|
||||
),
|
||||
# Governance: Prompt Policies
|
||||
Route("/api/admin/prompt-policies", admin_list_prompt_policies),
|
||||
Route(
|
||||
|
||||
@@ -6904,6 +6904,11 @@ function showCreateModelModal() {
|
||||
document.getElementById("model-thinking-mode").value = "";
|
||||
document.getElementById("model-thinking-param").value = "";
|
||||
document.getElementById("model-thinking-param-row").hidden = true;
|
||||
document.getElementById("model-effort-param").value = "";
|
||||
_annotateEffortSelect(
|
||||
document.getElementById("model-reasoning-effort"),
|
||||
null,
|
||||
);
|
||||
document.getElementById("model-extra-body").value = "";
|
||||
document.getElementById("model-capabilities").value = "";
|
||||
// Clear validation error styling from prior submit attempts
|
||||
@@ -6979,15 +6984,15 @@ function showEditModelModal(definitionId) {
|
||||
? capsObj.server_compat
|
||||
: {};
|
||||
// Only extract thinking_mode into the dropdown when the UI can
|
||||
// represent it ("manual" or "") AND the provider round-trips the
|
||||
// dropdown on save (every provider except anthropic-compatible —
|
||||
// see submitCreateModel). Unrepresentable values like "adaptive"
|
||||
// and anthropic-compatible rows keep thinking_mode in the raw
|
||||
// capabilities JSON so it isn't silently lost on save.
|
||||
// represent it ("", "manual" = effort-knob controlled, "adaptive" =
|
||||
// always on). Unrepresentable/garbage values keep thinking_mode in
|
||||
// the raw capabilities JSON so they aren't silently lost on save.
|
||||
// Both compat lanes round-trip the dropdown: it drives the
|
||||
// effort-knob → chat_template_kwargs mapping
|
||||
// (merge_reasoning_template_kwargs).
|
||||
const tmVal = capsObj.thinking_mode || "";
|
||||
const tmRepresentable = tmVal === "" || tmVal === "manual";
|
||||
const tmCaptured =
|
||||
tmRepresentable && (m.provider || "openai") !== "anthropic-compatible";
|
||||
tmVal === "" || tmVal === "manual" || tmVal === "adaptive";
|
||||
if (tmCaptured) {
|
||||
document.getElementById("model-thinking-mode").value = tmVal;
|
||||
document.getElementById("model-thinking-param").value =
|
||||
@@ -6997,6 +7002,10 @@ function showEditModelModal(definitionId) {
|
||||
document.getElementById("model-thinking-param").value = "";
|
||||
}
|
||||
_toggleThinkingParam();
|
||||
// effort_param is a plain string — always representable, so it lifts
|
||||
// into its field unconditionally (stripped from the raw JSON below).
|
||||
document.getElementById("model-effort-param").value =
|
||||
capsObj.effort_param || "";
|
||||
// Server compat: server_type, api_surface, and extra_body workarounds
|
||||
document.getElementById("model-server-type").value = sc.server_type || "";
|
||||
document.getElementById("model-api-surface").value = sc.api_surface || "";
|
||||
@@ -7029,12 +7038,27 @@ function showEditModelModal(definitionId) {
|
||||
});
|
||||
_modelRenderTiles();
|
||||
_modelCapsRefreshBaseline();
|
||||
_scheduleEffortLadder();
|
||||
// Remove structured fields from capabilities display — only delete
|
||||
// thinking_mode/thinking_param when the UI successfully captured them.
|
||||
delete capsObj.server_compat;
|
||||
if (tmCaptured) {
|
||||
delete capsObj.thinking_mode;
|
||||
delete capsObj.thinking_param;
|
||||
// Strip thinking_param only when a mode value round-trips through
|
||||
// the dropdown. With mode unset the save path writes neither key,
|
||||
// so a stored thinking_param must stay in the raw JSON — deleting
|
||||
// it here would silently drop it on the next unrelated edit-save.
|
||||
if (tmVal) delete capsObj.thinking_param;
|
||||
}
|
||||
// Strip effort_param only for the lanes whose save path re-adds it
|
||||
// (the same provider gate) — on other providers the field is
|
||||
// hidden and the save path never writes it, so deleting it here
|
||||
// would silently drop a stored key on an unrelated edit-save.
|
||||
if (
|
||||
m.provider === "openai-compatible" ||
|
||||
m.provider === "anthropic-compatible"
|
||||
) {
|
||||
delete capsObj.effort_param;
|
||||
}
|
||||
const capsText = JSON.stringify(capsObj, null, 2);
|
||||
document.getElementById("model-capabilities").value =
|
||||
@@ -7107,19 +7131,33 @@ function submitCreateModel() {
|
||||
const providerVal = document.getElementById("model-provider").value;
|
||||
|
||||
// Thinking mode → capabilities. thinking_mode round-trips through the
|
||||
// dropdown for every provider EXCEPT anthropic-compatible, where it
|
||||
// stays in the raw capabilities JSON (mirroring the edit-load lift):
|
||||
// that lane hides the dropdown row and drives reasoning via extra-body
|
||||
// chat_template_kwargs, so a lingering dropdown value must never be
|
||||
// persisted.
|
||||
// dropdown for every provider, including anthropic-compatible (where
|
||||
// manual mode maps the session effort knob onto the template's
|
||||
// thinking toggle via chat_template_kwargs —
|
||||
// merge_reasoning_template_kwargs).
|
||||
const thinkingMode = document.getElementById("model-thinking-mode").value;
|
||||
if (providerVal !== "anthropic-compatible" && thinkingMode) {
|
||||
if (thinkingMode) {
|
||||
caps.thinking_mode = thinkingMode;
|
||||
// Preserve thinking_param so Granite/DeepSeek "thinking" key
|
||||
// isn't silently reverted to the default "enable_thinking".
|
||||
const savedParam = document.getElementById("model-thinking-param").value;
|
||||
if (savedParam) caps.thinking_param = savedParam;
|
||||
}
|
||||
// Effort param (graded chat-template effort key, e.g. gpt-oss
|
||||
// "reasoning_effort") round-trips like thinking_param: lifted out of
|
||||
// the raw JSON on edit-load, re-added here when the field is set.
|
||||
// Gated on the local-server lanes (like the server_compat block
|
||||
// below): the field is hidden for other providers, so a lingering
|
||||
// value from a provider switch must never persist.
|
||||
if (
|
||||
providerVal === "openai-compatible" ||
|
||||
providerVal === "anthropic-compatible"
|
||||
) {
|
||||
const effortParam = document
|
||||
.getElementById("model-effort-param")
|
||||
.value.trim();
|
||||
if (effortParam) caps.effort_param = effortParam;
|
||||
}
|
||||
|
||||
// Build server_compat from structured fields. Only meaningful for the
|
||||
// compat lanes (openai-compatible: all fields; anthropic-compatible: the
|
||||
@@ -7602,6 +7640,121 @@ let _modelCapsSeq = 0;
|
||||
function _onModelFieldChange() {
|
||||
clearTimeout(_capsTimer);
|
||||
_capsTimer = setTimeout(_modelCapsRefreshBaseline, 500);
|
||||
_scheduleEffortLadder();
|
||||
}
|
||||
|
||||
/* Effort-ladder annotation: each knob position states, in plain words,
|
||||
what the request will carry — from the server-computed projection
|
||||
(providers/effort_ladder.py — equal "effective" tokens ⇒ identical
|
||||
requests). A position whose delivered level matches its name stays
|
||||
plain ("Max"); a snapped position says so ("Low — sends high"); the
|
||||
adaptive none position warns "thinking stays on"; budget detail
|
||||
lives in the tooltip. Never label a position after a sibling that
|
||||
shares its token — that rendered "Max (= minimal)", implying a
|
||||
downgrade the wire doesn't contain. Defined here and shared as a
|
||||
page global with governance.js (skill launch config), which loads
|
||||
after this file. */
|
||||
function _annotateEffortSelect(sel, ladder) {
|
||||
if (!sel) return;
|
||||
const byVal = {};
|
||||
(ladder || []).forEach(function (row) {
|
||||
byVal[row.value] = row.effective;
|
||||
});
|
||||
Array.from(sel.options).forEach(function (opt) {
|
||||
if (!opt.value) return; // "" = inherit-the-default option
|
||||
if (!opt.dataset.baseLabel) opt.dataset.baseLabel = opt.textContent;
|
||||
let label = opt.dataset.baseLabel;
|
||||
let title = "";
|
||||
const eff = byVal[opt.value];
|
||||
if (eff !== undefined) {
|
||||
title = "sends: " + eff;
|
||||
if (eff === "default") {
|
||||
label += " — model default";
|
||||
} else if (eff === "on") {
|
||||
// Local adaptive lane, none position: we send only the thinking
|
||||
// toggle (no effort grade exists), so the knob can't turn
|
||||
// thinking off — that is the whole story here.
|
||||
label += " — thinking stays on";
|
||||
} else if (eff === "adaptive") {
|
||||
// Native Anthropic adaptive: thinking is ALWAYS on and an effort
|
||||
// level rides output_config on every graded position, so
|
||||
// "thinking stays on" is true everywhere and not what sets the
|
||||
// none position apart. What "none" uniquely means is that no
|
||||
// effort is pinned — the model self-regulates it.
|
||||
label += " — model sets effort";
|
||||
} else if (eff !== "off") {
|
||||
// "off" needs no echo on the none position. Otherwise strip
|
||||
// the toggle prefix and budget qualifier ("on+high" → "high",
|
||||
// "high·budget:16384" → "high", bare "budget:4096" → "" = no
|
||||
// suffix); the tooltip keeps the full token.
|
||||
const wire = eff.replace(/^on\+/, "").replace(/(^|·)budget:\d+$/, "");
|
||||
if (wire && wire !== opt.value) label += " — sends " + wire;
|
||||
}
|
||||
}
|
||||
opt.textContent = label;
|
||||
opt.title = title;
|
||||
});
|
||||
}
|
||||
|
||||
let _effortLadderTimer = null;
|
||||
let _effortLadderSeq = 0;
|
||||
function _scheduleEffortLadder() {
|
||||
clearTimeout(_effortLadderTimer);
|
||||
_effortLadderTimer = setTimeout(_refreshModelEffortLadder, 500);
|
||||
}
|
||||
function _refreshModelEffortLadder() {
|
||||
const shelf = document.getElementById("model-shelf");
|
||||
const sel = document.getElementById("model-reasoning-effort");
|
||||
if (!shelf || !shelf.open || !sel) return;
|
||||
const provider = document.getElementById("model-provider").value;
|
||||
const modelName = document.getElementById("model-name").value.trim();
|
||||
if (!modelName) {
|
||||
_effortLadderSeq++; // invalidate any in-flight response
|
||||
_annotateEffortSelect(sel, null);
|
||||
return;
|
||||
}
|
||||
// Assemble the same capabilities the save path would persist: raw
|
||||
// JSON base, structured thinking/effort fields overlaid. Mid-edit
|
||||
// invalid JSON annotates from the structured fields alone.
|
||||
let caps = {};
|
||||
const rawText = document.getElementById("model-capabilities").value.trim();
|
||||
if (rawText) {
|
||||
try {
|
||||
const parsed = JSON.parse(rawText);
|
||||
if (_isPlainObject(parsed)) caps = parsed;
|
||||
} catch (e) {
|
||||
/* fall through */
|
||||
}
|
||||
}
|
||||
const tm = document.getElementById("model-thinking-mode").value;
|
||||
if (tm) {
|
||||
caps.thinking_mode = tm;
|
||||
const tp = document.getElementById("model-thinking-param").value;
|
||||
if (tp) caps.thinking_param = tp;
|
||||
}
|
||||
const ep = document.getElementById("model-effort-param").value.trim();
|
||||
if (ep) caps.effort_param = ep;
|
||||
const seq = ++_effortLadderSeq;
|
||||
authFetch("/v1/api/admin/models/effort-ladder", {
|
||||
method: "POST",
|
||||
headers: { "content-type": "application/json" },
|
||||
body: JSON.stringify({
|
||||
provider: provider,
|
||||
model: modelName,
|
||||
capabilities: caps,
|
||||
api_surface: document.getElementById("model-api-surface").value || "",
|
||||
}),
|
||||
})
|
||||
.then(function (r) {
|
||||
return r.ok ? r.json() : null;
|
||||
})
|
||||
.then(function (d) {
|
||||
if (seq !== _effortLadderSeq) return; // superseded by a newer edit
|
||||
_annotateEffortSelect(sel, d && d.ladder);
|
||||
})
|
||||
.catch(function () {
|
||||
/* silent — annotation only */
|
||||
});
|
||||
}
|
||||
|
||||
/* Known-model lookup feeding the tile matrix: the table becomes the display
|
||||
@@ -7701,15 +7854,15 @@ function _applyProviderDefaults() {
|
||||
scSection.hidden =
|
||||
provider !== "openai-compatible" && provider !== "anthropic-compatible";
|
||||
}
|
||||
// Within the section, server type / API surface / thinking mode are
|
||||
// openai-compatible knobs — the anthropic-compatible lane is configured
|
||||
// through the extra-body JSON alone, so collapse the section to just
|
||||
// that field.
|
||||
const hideOpenaiOnlyRows = provider === "anthropic-compatible";
|
||||
["model-server-fields-row", "model-thinking-mode-row"].forEach(function (id) {
|
||||
const row = document.getElementById(id);
|
||||
if (row) row.hidden = hideOpenaiOnlyRows;
|
||||
});
|
||||
// Within the section, server type / API surface are openai-compatible
|
||||
// knobs (Detect heuristics + chat-vs-responses surface pick) and stay
|
||||
// hidden on the anthropic-compatible lane. Thinking mode applies to
|
||||
// BOTH compat lanes: on anthropic-compatible it opts the model into
|
||||
// the effort-knob → chat_template_kwargs mapping.
|
||||
const serverFieldsRow = document.getElementById("model-server-fields-row");
|
||||
if (serverFieldsRow) {
|
||||
serverFieldsRow.hidden = provider === "anthropic-compatible";
|
||||
}
|
||||
}
|
||||
|
||||
/* Populate the model name datalist with known model prefixes for the
|
||||
@@ -7748,6 +7901,15 @@ function _refreshModelSuggestions() {
|
||||
const provEl = document.getElementById("model-provider");
|
||||
const tmEl = document.getElementById("model-thinking-mode");
|
||||
if (tmEl) tmEl.addEventListener("change", _toggleThinkingParam);
|
||||
if (tmEl) tmEl.addEventListener("change", _scheduleEffortLadder);
|
||||
["model-thinking-param", "model-effort-param", "model-capabilities"].forEach(
|
||||
function (id) {
|
||||
const el = document.getElementById(id);
|
||||
if (el) el.addEventListener("input", _scheduleEffortLadder);
|
||||
},
|
||||
);
|
||||
const apiSurfEl = document.getElementById("model-api-surface");
|
||||
if (apiSurfEl) apiSurfEl.addEventListener("change", _scheduleEffortLadder);
|
||||
const grid = document.getElementById("model-capgrid");
|
||||
if (grid) {
|
||||
grid.addEventListener("change", function (e) {
|
||||
|
||||
@@ -429,6 +429,15 @@ function handleClusterEvent(data) {
|
||||
if (typeof _populateHomeModelDropdowns === "function") {
|
||||
_populateHomeModelDropdowns();
|
||||
}
|
||||
if (typeof _sklcInvalidateModelsCache === "function") {
|
||||
_sklcInvalidateModelsCache();
|
||||
}
|
||||
if (typeof _sklcScheduleEffortLadder === "function") {
|
||||
// Re-annotate from the fresh cache — an edited model's ladder
|
||||
// would otherwise stay stale until the next keystroke in the
|
||||
// launch-config form.
|
||||
_sklcScheduleEffortLadder();
|
||||
}
|
||||
if (
|
||||
typeof _adminTab !== "undefined" &&
|
||||
_adminTab === "models" &&
|
||||
|
||||
@@ -1310,6 +1310,7 @@ function createCoordinatorPane(root, wsId, opts) {
|
||||
approved: !!approved,
|
||||
always: !!always,
|
||||
call_id: callId,
|
||||
cycle_id: batch.dataset.cycleId || null,
|
||||
});
|
||||
if (!resp.ok) throw new Error("approve failed: HTTP " + resp.status);
|
||||
} catch (e) {
|
||||
@@ -1465,6 +1466,9 @@ function createCoordinatorPane(root, wsId, opts) {
|
||||
}
|
||||
if (allMapped) {
|
||||
const existing = toolRows.get(items[0].call_id).batch;
|
||||
// Late cycle identity (SSE approve_request upgrading a replay /
|
||||
// early-paint shell) — stamp it so the approve POST can route.
|
||||
if (opts.cycleId) existing.dataset.cycleId = opts.cycleId;
|
||||
// Upgrade-in-place: when SSE arrives with a more specific state
|
||||
// than the placeholder history replay rendered, morph the
|
||||
// existing shell instead of leaving stale chrome. The two real
|
||||
@@ -1578,6 +1582,10 @@ function createCoordinatorPane(root, wsId, opts) {
|
||||
summaryText,
|
||||
tierText: _pickBatchTier(items),
|
||||
});
|
||||
// Approval-cycle identity — _resolveBatchAction posts it back so
|
||||
// the decision lands on exactly this round when several batches
|
||||
// are pending (parallel task agents).
|
||||
if (opts.cycleId) batch.dataset.cycleId = opts.cycleId;
|
||||
if (opts.pending) batch.classList.add("conv-batch--pending");
|
||||
else if (opts.auto) batch.classList.add("conv-batch--auto");
|
||||
else if (opts.resolved) {
|
||||
@@ -1732,12 +1740,13 @@ function createCoordinatorPane(root, wsId, opts) {
|
||||
// directly, and inline action buttons drive _resolveBatchAction.
|
||||
// ------------------------------------------------------------------
|
||||
|
||||
function showApproval(items, judgePending) {
|
||||
function showApproval(items, judgePending, cycleId) {
|
||||
const list = (items || []).filter(Boolean);
|
||||
if (list.length === 0) return;
|
||||
const batch = appendToolBatch(list, {
|
||||
pending: true,
|
||||
judgePending: !!judgePending,
|
||||
cycleId: cycleId || "",
|
||||
});
|
||||
const firstPending = list.find((it) => it.needs_approval);
|
||||
if (batch && firstPending && firstPending.call_id) {
|
||||
@@ -2233,7 +2242,7 @@ function createCoordinatorPane(root, wsId, opts) {
|
||||
loadChildren({ replace: true });
|
||||
loadTasks();
|
||||
// Drop the live-badge cache too — entries within the 5s TTL
|
||||
// can carry stale pending_approval_detail (the child may
|
||||
// can carry stale pending_approval_details (the child may
|
||||
// have resolved its approval during the SSE gap). Without
|
||||
// this clear, inline approve/deny buttons could render on
|
||||
// a row whose approval was resolved elsewhere; the next
|
||||
@@ -2448,26 +2457,36 @@ function createCoordinatorPane(root, wsId, opts) {
|
||||
break;
|
||||
case "approve_request":
|
||||
// appendToolBatch is idempotent on call_ids — the console replays
|
||||
// _pending_approval into every new SSE subscriber, so reconnect
|
||||
// won't double-render the construct.
|
||||
showApproval(ev.items, !!ev.judge_pending);
|
||||
// every live cycle's card into every new SSE subscriber, so
|
||||
// reconnect won't double-render the construct.
|
||||
showApproval(ev.items, !!ev.judge_pending, ev.cycle_id || "");
|
||||
break;
|
||||
case "approval_resolved": {
|
||||
// Server-driven resolution. Morph the active pending batch
|
||||
// (the construct that posted the approve POST). The server
|
||||
// event carries `approved` + `feedback` only — the "always"
|
||||
// intent. Server now echoes ``always`` on the SSE payload
|
||||
// (post-PR-447) so cross-tab resolution renders the right
|
||||
// status pill on every subscribed tab — not just the one
|
||||
// that clicked. Fall back to this tab's stashed dataset
|
||||
// flag for backward compat with a server hot-deploy where
|
||||
// the SSE event might briefly omit the field.
|
||||
// Fall back to a DOM lookup if activeBatch was never set
|
||||
// (e.g. cross-tab resolution where this tab never rendered
|
||||
// the approval gate before the resolved event landed).
|
||||
const target =
|
||||
activeBatch ||
|
||||
messagesEl.querySelector(".conv-batch.conv-batch--pending");
|
||||
// Server-driven resolution. Route to the batch whose rows
|
||||
// carry one of the resolved call_ids — several batches can be
|
||||
// pending at once (parallel task agents), and morphing "the
|
||||
// active" one would resolve the wrong construct. Server now
|
||||
// echoes ``always`` on the SSE payload (post-PR-447) so
|
||||
// cross-tab resolution renders the right status pill on every
|
||||
// subscribed tab — not just the one that clicked. Fall back
|
||||
// to this tab's stashed dataset flag for backward compat with
|
||||
// a server hot-deploy where the SSE event might briefly omit
|
||||
// the field. Legacy events without call_ids fall back to
|
||||
// activeBatch / the last pending batch (single-cycle server).
|
||||
let target = null;
|
||||
const resolvedIds = Array.isArray(ev.call_ids) ? ev.call_ids : [];
|
||||
for (const cid of resolvedIds) {
|
||||
const mapped = toolRows.get(cid);
|
||||
if (mapped && mapped.batch) {
|
||||
target = mapped.batch;
|
||||
break;
|
||||
}
|
||||
}
|
||||
if (!target) {
|
||||
target =
|
||||
activeBatch ||
|
||||
messagesEl.querySelector(".conv-batch.conv-batch--pending");
|
||||
}
|
||||
if (target) {
|
||||
const wasAlways =
|
||||
ev.always === true ||
|
||||
@@ -2971,11 +2990,18 @@ function createCoordinatorPane(root, wsId, opts) {
|
||||
// \u2014 without it the operator sees a "demand for action" badge
|
||||
// with no actionable content.
|
||||
if (cached && cached.live && cached.live.pending_approval) {
|
||||
const detail = cached.live.pending_approval_detail;
|
||||
const block = detail
|
||||
? renderApprovalBlock(child, detail)
|
||||
: renderApprovalPlaceholder(child);
|
||||
if (block) row.appendChild(block);
|
||||
// One block per live cycle — a child running parallel task agents
|
||||
// can gate several batches at once, each independently resolvable.
|
||||
const details = _liveApprovalDetails(cached.live);
|
||||
if (details.length) {
|
||||
details.forEach((detail) => {
|
||||
const block = renderApprovalBlock(child, detail);
|
||||
if (block) row.appendChild(block);
|
||||
});
|
||||
} else {
|
||||
const block = renderApprovalPlaceholder(child);
|
||||
if (block) row.appendChild(block);
|
||||
}
|
||||
}
|
||||
// Recent auto-approves — tools that bypassed the operator gate
|
||||
// (skill ``allowed_tools`` allowlist / blanket / admin policy /
|
||||
@@ -3132,6 +3158,16 @@ function createCoordinatorPane(root, wsId, opts) {
|
||||
return block;
|
||||
}
|
||||
|
||||
// Normalize the live block's approval payload to a list of cycle
|
||||
// details (``pending_approval_details``). Defensive against a
|
||||
// malformed entry: non-arrays fold to [].
|
||||
function _liveApprovalDetails(live) {
|
||||
if (!live) return [];
|
||||
return Array.isArray(live.pending_approval_details)
|
||||
? live.pending_approval_details.filter(Boolean)
|
||||
: [];
|
||||
}
|
||||
|
||||
function renderApprovalBlock(child, detail) {
|
||||
if (!detail || !Array.isArray(detail.items) || detail.items.length === 0) {
|
||||
return null;
|
||||
@@ -3397,7 +3433,7 @@ function createCoordinatorPane(root, wsId, opts) {
|
||||
}
|
||||
|
||||
// Submit the approve POST + handle the result. On success, locally
|
||||
// clear pending_approval_detail so the row re-renders without
|
||||
// clear the resolved cycle from pending_approval_details so the row re-renders without
|
||||
// buttons immediately (optimistic update \u2014 the next live-bulk poll
|
||||
// confirms). On 409 (stale call_id), refresh the live block so the
|
||||
// row re-renders against the new round.
|
||||
@@ -3423,6 +3459,7 @@ function createCoordinatorPane(root, wsId, opts) {
|
||||
approved: !!approved,
|
||||
always: false,
|
||||
call_id: callId,
|
||||
cycle_id: (detail && detail.cycle_id) || null,
|
||||
});
|
||||
if (resp.status === 409) {
|
||||
// Stale call_id \u2014 server has rolled to a new round, or
|
||||
@@ -3477,9 +3514,15 @@ function createCoordinatorPane(root, wsId, opts) {
|
||||
// pill flickers back. (Caught by /review bug-4.)
|
||||
const cached = liveBadgeCache.get(targetWsId);
|
||||
if (cached && cached.live) {
|
||||
// Optimistically remove ONLY the resolved cycle — sibling
|
||||
// cycles (parallel task agents) keep their buttons. The
|
||||
// pending flag clears only when no cycles remain.
|
||||
const remaining = _liveApprovalDetails(cached.live).filter(
|
||||
(d) => d !== detail && d.cycle_id !== (detail && detail.cycle_id),
|
||||
);
|
||||
cached.live = Object.assign({}, cached.live, {
|
||||
pending_approval: false,
|
||||
pending_approval_detail: null,
|
||||
pending_approval: remaining.length > 0,
|
||||
pending_approval_details: remaining,
|
||||
});
|
||||
cached.sseUpdatedAt = Date.now();
|
||||
_liveBadgeCacheSet(targetWsId, cached);
|
||||
@@ -3488,8 +3531,14 @@ function createCoordinatorPane(root, wsId, opts) {
|
||||
// badge in .meta disappears immediately too — without this, the
|
||||
// row shows the badge with no buttons for ~50-150ms until the
|
||||
// child_ws_approval_resolved push or next state event lands.
|
||||
// Only when NO cycles remain: a sibling prompt keeps the badge.
|
||||
const cachedAfter = liveBadgeCache.get(targetWsId);
|
||||
const anyLeft =
|
||||
cachedAfter &&
|
||||
cachedAfter.live &&
|
||||
_liveApprovalDetails(cachedAfter.live).length > 0;
|
||||
const childState = childrenState.get(targetWsId);
|
||||
if (childState && childState.activity_state === "approval") {
|
||||
if (childState && childState.activity_state === "approval" && !anyLeft) {
|
||||
childState.activity_state = "";
|
||||
}
|
||||
renderChildren();
|
||||
@@ -3895,7 +3944,7 @@ function createCoordinatorPane(root, wsId, opts) {
|
||||
) {
|
||||
mergedLive = Object.assign({}, live, {
|
||||
pending_approval: prev.live.pending_approval,
|
||||
pending_approval_detail: prev.live.pending_approval_detail,
|
||||
pending_approval_details: prev.live.pending_approval_details,
|
||||
});
|
||||
}
|
||||
_liveBadgeCacheSet(id, {
|
||||
@@ -3996,10 +4045,20 @@ function createCoordinatorPane(root, wsId, opts) {
|
||||
// misses 30+ children all sitting in attention. (Caught manual
|
||||
// testing: 30 children all state=attention rendered with no
|
||||
// approval blocks because pendingApproval was always false.)
|
||||
const pendingApproval =
|
||||
// The ws-level activity signal is COARSE under parallel task
|
||||
// agents: one gate resolving (activity flips to "tool") while a
|
||||
// sibling is still parked would read as "no approval pending".
|
||||
// The per-cycle details list is the authoritative surface — a
|
||||
// non-empty list keeps the row pending regardless of the
|
||||
// activity flicker; per-cycle removal happens in
|
||||
// handleChildApprovalResolved, and the bulk fetch reconciles a
|
||||
// dropped resolution event within its ~2s TTL.
|
||||
const coarsePending =
|
||||
existing.state === "attention" || existing.activity_state === "approval";
|
||||
const cached = liveBadgeCache.get(childId);
|
||||
const cachedLive = (cached && cached.live) || {};
|
||||
const pendingApproval =
|
||||
coarsePending || _liveApprovalDetails(cachedLive).length > 0;
|
||||
// Rising-edge detection BEFORE we mutate the cache. The chat-pane
|
||||
// tool batches already announce assertively
|
||||
// (renderApprovalDock / appendToolBatch); the children-tree was
|
||||
@@ -4018,18 +4077,18 @@ function createCoordinatorPane(root, wsId, opts) {
|
||||
// authoritatively writes a value the bulk fetch must not
|
||||
// resurrect (a stale bulk-fetch landing within
|
||||
// SSE_AUTHORITATIVE_MS would otherwise re-render the cleared
|
||||
// approval block). Setting ``pending_approval=true`` does NOT
|
||||
// approval blocks). Setting ``pending_approval=true`` does NOT
|
||||
// claim cache authority — the bulk fetch is the source of
|
||||
// truth for ``pending_approval_detail``, and bumping
|
||||
// truth for ``pending_approval_details``, and bumping
|
||||
// sseUpdatedAt here makes the merge guard in flushLiveFetches
|
||||
// preserve our stale (often null) detail over the bulk
|
||||
// preserve our stale (often empty) details over the bulk
|
||||
// fetch's actual data, leaving the row stuck on the loading
|
||||
// placeholder. (Caught when the screenshot showed buttons
|
||||
// briefly then loading replaced them.)
|
||||
const detailClearedAuthoritatively =
|
||||
!pendingApproval && cachedLive.pending_approval === true;
|
||||
if (!pendingApproval) {
|
||||
nextLive.pending_approval_detail = null;
|
||||
nextLive.pending_approval_details = [];
|
||||
}
|
||||
_liveBadgeCacheSet(childId, {
|
||||
live: nextLive,
|
||||
@@ -4071,8 +4130,8 @@ function createCoordinatorPane(root, wsId, opts) {
|
||||
const existing = childrenState.get(childId);
|
||||
if (!existing) return;
|
||||
existing.state = ev.reason === "deleted" ? "deleted" : "closed";
|
||||
// Clearing the live cache eagerly on close prevents a stale
|
||||
// pending_approval_detail from continuing to render approve/deny
|
||||
// Clearing the live cache eagerly on close prevents stale
|
||||
// pending_approval_details from continuing to render approve/deny
|
||||
// buttons on a closed row (its TTL would otherwise survive into
|
||||
// the closed/deleted lifecycle until natural expiry).
|
||||
invalidateLiveBadge(childId);
|
||||
@@ -4114,17 +4173,23 @@ function createCoordinatorPane(root, wsId, opts) {
|
||||
if (!callId) return;
|
||||
const cached = liveBadgeCache.get(childId);
|
||||
const cachedLive = (cached && cached.live) || {};
|
||||
const detail = cachedLive.pending_approval_detail;
|
||||
// Stamp onto the CYCLE containing this call_id — several details
|
||||
// can be live at once (parallel task agents).
|
||||
const detail = _liveApprovalDetails(cachedLive).find(
|
||||
(d) =>
|
||||
Array.isArray(d.items) &&
|
||||
d.items.some((it) => it && it.call_id === callId),
|
||||
);
|
||||
if (!detail) {
|
||||
// No pending_approval_detail to stamp the verdict onto. The
|
||||
// verdict is still durable in storage; the next bulk fetch
|
||||
// will hydrate the detail and include the verdict via the
|
||||
// existing serialize path.
|
||||
// No pending detail to stamp the verdict onto. The verdict is
|
||||
// still durable in storage; the next bulk fetch will hydrate
|
||||
// the details and include the verdict via the existing
|
||||
// serialize path.
|
||||
return;
|
||||
}
|
||||
// Stamp on the matching item (UI render reads judge_verdict per
|
||||
// item) AND on the by-call_id map (matches the
|
||||
// serialize_pending_approval_detail shape).
|
||||
// serialize_pending_approval_details shape).
|
||||
const items = Array.isArray(detail.items) ? detail.items : [];
|
||||
for (const item of items) {
|
||||
if (item && item.call_id === callId) {
|
||||
@@ -4153,8 +4218,15 @@ function createCoordinatorPane(root, wsId, opts) {
|
||||
if (!childId) return;
|
||||
const cached = liveBadgeCache.get(childId);
|
||||
const cachedLive = (cached && cached.live) || {};
|
||||
cachedLive.pending_approval = false;
|
||||
cachedLive.pending_approval_detail = null;
|
||||
// Remove ONLY the resolved cycle; siblings keep their buttons.
|
||||
// Legacy events without a cycle_id (pre-multi-cycle node) clear
|
||||
// everything, matching the old single-slot behavior.
|
||||
const details = _liveApprovalDetails(cachedLive);
|
||||
const remaining = ev.cycle_id
|
||||
? details.filter((d) => d.cycle_id !== ev.cycle_id)
|
||||
: [];
|
||||
cachedLive.pending_approval = remaining.length > 0;
|
||||
cachedLive.pending_approval_details = remaining;
|
||||
_liveBadgeCacheSet(childId, {
|
||||
live: cachedLive,
|
||||
fetched: cached ? cached.fetched : 0,
|
||||
@@ -4179,8 +4251,17 @@ function createCoordinatorPane(root, wsId, opts) {
|
||||
if (!detail) return;
|
||||
const cached = liveBadgeCache.get(childId);
|
||||
const cachedLive = (cached && cached.live) || {};
|
||||
// Append (or replace, keyed by cycle_id) — several cycles can be
|
||||
// outstanding under parallel task agents; a sibling's push must
|
||||
// not clobber ours. Legacy nodes without cycle_id fold to a
|
||||
// single-slot replace via the shared "" key.
|
||||
const key = detail.cycle_id || "";
|
||||
const details = _liveApprovalDetails(cachedLive).filter(
|
||||
(d) => (d.cycle_id || "") !== key,
|
||||
);
|
||||
details.push(detail);
|
||||
cachedLive.pending_approval = true;
|
||||
cachedLive.pending_approval_detail = detail;
|
||||
cachedLive.pending_approval_details = details;
|
||||
_liveBadgeCacheSet(childId, {
|
||||
live: cachedLive,
|
||||
fetched: cached ? cached.fetched : 0,
|
||||
@@ -4530,19 +4611,20 @@ function createCoordinatorPane(root, wsId, opts) {
|
||||
// NOT re-run this (SSE re-delivers approve_request live, and replaying a
|
||||
// stale pre-rewind pending batch would be wrong).
|
||||
try {
|
||||
const pendingDetail =
|
||||
const pendingDetails =
|
||||
wsSnapshot &&
|
||||
wsSnapshot.pending_approval &&
|
||||
wsSnapshot.pending_approval_detail &&
|
||||
Array.isArray(wsSnapshot.pending_approval_detail.items)
|
||||
? wsSnapshot.pending_approval_detail
|
||||
: null;
|
||||
if (pendingDetail) {
|
||||
Array.isArray(wsSnapshot.pending_approval_details)
|
||||
? wsSnapshot.pending_approval_details
|
||||
: [];
|
||||
pendingDetails.forEach((pendingDetail) => {
|
||||
if (!pendingDetail || !Array.isArray(pendingDetail.items)) return;
|
||||
appendToolBatch(pendingDetail.items, {
|
||||
pending: true,
|
||||
judgePending: !!pendingDetail.judge_pending,
|
||||
cycleId: pendingDetail.cycle_id || "",
|
||||
});
|
||||
}
|
||||
});
|
||||
} catch (e) {
|
||||
console.warn("pending-approval replay failed", e);
|
||||
}
|
||||
|
||||
@@ -1965,6 +1965,7 @@ function showEditTemplateModal(tmplId) {
|
||||
document.getElementById("skl-default").checked = tmpl.is_default;
|
||||
// Session config fields
|
||||
document.getElementById("sklc-model").value = tmpl.model || "";
|
||||
_sklcScheduleEffortLadder();
|
||||
document.getElementById("sklc-temperature").value =
|
||||
tmpl.temperature != null ? tmpl.temperature : "";
|
||||
document.getElementById("sklc-reasoning-effort").value =
|
||||
@@ -5044,3 +5045,55 @@ function _submitOGPShelf() {
|
||||
errEl.classList.add("is-visible");
|
||||
});
|
||||
}
|
||||
|
||||
/* Effort-ladder annotation for the skill launch-config effort select.
|
||||
Resolves the typed alias against /v1/api/models (each row carries a
|
||||
server-computed effort_ladder) and reuses the page-global
|
||||
_annotateEffortSelect from admin.js, which loads before this file. */
|
||||
let _sklcModelsPromise = null;
|
||||
let _sklcLadderTimer = null;
|
||||
/* Called from app.js on the models_changed SSE event so edited/added
|
||||
models don't serve a stale ladder until page reload. */
|
||||
function _sklcInvalidateModelsCache() {
|
||||
_sklcModelsPromise = null;
|
||||
}
|
||||
function _sklcRefreshEffortLadder() {
|
||||
const sel = document.getElementById("sklc-reasoning-effort");
|
||||
const aliasEl = document.getElementById("sklc-model");
|
||||
if (!sel || !aliasEl || typeof _annotateEffortSelect !== "function") return;
|
||||
const alias = aliasEl.value.trim();
|
||||
if (!alias) {
|
||||
_annotateEffortSelect(sel, null);
|
||||
return;
|
||||
}
|
||||
if (!_sklcModelsPromise) {
|
||||
_sklcModelsPromise = authFetch("/v1/api/models")
|
||||
.then(function (r) {
|
||||
return r.json();
|
||||
})
|
||||
.catch(function (e) {
|
||||
// A cached rejection would block every retry (the truthy guard
|
||||
// above) — reset so the next keystroke can refetch.
|
||||
_sklcModelsPromise = null;
|
||||
throw e;
|
||||
});
|
||||
}
|
||||
_sklcModelsPromise
|
||||
.then(function (d) {
|
||||
const row = (d.models || []).find(function (m) {
|
||||
return m.alias === alias;
|
||||
});
|
||||
_annotateEffortSelect(sel, row ? row.effort_ladder : null);
|
||||
})
|
||||
.catch(function () {
|
||||
/* silent — annotation only */
|
||||
});
|
||||
}
|
||||
function _sklcScheduleEffortLadder() {
|
||||
clearTimeout(_sklcLadderTimer);
|
||||
_sklcLadderTimer = setTimeout(_sklcRefreshEffortLadder, 500);
|
||||
}
|
||||
(function () {
|
||||
const el = document.getElementById("sklc-model");
|
||||
if (el) el.addEventListener("input", _sklcScheduleEffortLadder);
|
||||
})();
|
||||
|
||||
@@ -1771,7 +1771,8 @@
|
||||
>
|
||||
<select id="model-thinking-mode">
|
||||
<option value="">None</option>
|
||||
<option value="manual">Enabled</option>
|
||||
<option value="manual">Effort-knob controlled</option>
|
||||
<option value="adaptive">Always on</option>
|
||||
</select>
|
||||
<div id="model-thinking-param-row" hidden>
|
||||
<label for="model-thinking-param"
|
||||
@@ -1789,6 +1790,21 @@
|
||||
/>
|
||||
</div>
|
||||
</div>
|
||||
<div id="model-effort-param-row">
|
||||
<label for="model-effort-param"
|
||||
>Effort param
|
||||
<span class="label-hint"
|
||||
>chat-template key for graded effort — empty = template
|
||||
has none</span
|
||||
></label
|
||||
>
|
||||
<input
|
||||
type="text"
|
||||
id="model-effort-param"
|
||||
class="sh-mono"
|
||||
placeholder="reasoning_effort / reasoning"
|
||||
/>
|
||||
</div>
|
||||
<label for="model-extra-body"
|
||||
>Extra body params
|
||||
<span class="label-hint"
|
||||
@@ -1887,7 +1903,7 @@
|
||||
<textarea
|
||||
id="model-capabilities"
|
||||
rows="3"
|
||||
placeholder='{"max_output_tokens": 32000}'
|
||||
placeholder='{"max_output_tokens": 32000, "reasoning_effort_values": ["low", "high"], "default_reasoning_effort": "low"}'
|
||||
></textarea>
|
||||
</div>
|
||||
</details>
|
||||
|
||||
@@ -33,8 +33,18 @@ def cleanup_session_ui(ws: Workstream) -> None:
|
||||
ws.session.cancel()
|
||||
ui = ws.ui
|
||||
if ui is not None:
|
||||
if hasattr(ui, "_approval_event"):
|
||||
ui._approval_result = False, None # type: ignore[attr-defined]
|
||||
if hasattr(ui, "resolve_all_approvals"):
|
||||
# Deny + unblock EVERY live approval cycle — with parallel
|
||||
# task agents several gate threads can be parked at once,
|
||||
# and each must wake with its own (denied) result.
|
||||
with contextlib.suppress(Exception):
|
||||
ui.resolve_all_approvals(False, "Workstream closed")
|
||||
elif hasattr(ui, "_approval_event"):
|
||||
# Pre-cycle external SessionUI impls: the old single-slot
|
||||
# contract. Without this kick their gate thread stays
|
||||
# parked on a workstream that no longer exists.
|
||||
if hasattr(ui, "_approval_result"):
|
||||
ui._approval_result = (False, None)
|
||||
ui._approval_event.set()
|
||||
if hasattr(ui, "_fg_event"):
|
||||
ui._fg_event.set()
|
||||
|
||||
@@ -6,7 +6,7 @@ surface is asymmetric on the interactive side and the asymmetry is
|
||||
load-bearing — see ``InteractiveAdapter`` docstring.
|
||||
|
||||
Uses ``WebUI``'s per-UI listener set for ``cleanup_ui`` — the same
|
||||
hooks (``_approval_event`` / ``_fg_event`` + the ``ws_closed``
|
||||
hooks (``resolve_all_approvals`` / ``_fg_event`` + the ``ws_closed``
|
||||
broadcast) the old ``WorkstreamManager._cleanup_ui`` touched.
|
||||
"""
|
||||
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user