mirror of
https://github.com/turnstonelabs/turnstone.git
synced 2026-08-13 15:32:24 -06:00
Compare commits
20 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 84fd5dc859 | |||
| ba8b1d9126 | |||
| 685b1e3d9b | |||
| 91300060fa | |||
| 688f047ce1 | |||
| 87893aa4ab | |||
| 0988142303 | |||
| 40ecebf012 | |||
| c91869c7e5 | |||
| 53b52092f9 | |||
| 0d1a009a4c | |||
| e8352bd8e5 | |||
| de4cc568c4 | |||
| b477c85ddc | |||
| 00bd80a658 | |||
| 47df9d23c5 | |||
| 16fc7efce2 | |||
| e8eca2ec9b | |||
| 57563b0c12 | |||
| 43e622840b |
+46
-1
@@ -281,6 +281,7 @@ Each message in the `messages` array has:
|
||||
| `role` | string | `"user"`, `"assistant"`, or `"tool"` |
|
||||
| `content` | string or null | Text content of the message |
|
||||
| `tool_calls` | array or null | Present only on assistant messages with calls |
|
||||
| `reasoning` | string (optional) | Concatenated reasoning / chain-of-thought text on assistant turns whose `provider_data` carried reasoning-bearing blocks (Anthropic `thinking`, OpenAI Responses `reasoning`, or synthetic `reasoning_text` from local-model servers). Present only when the active model's `surface_persisted_reasoning` flag is True. |
|
||||
|
||||
Each entry in `tool_calls`:
|
||||
|
||||
@@ -325,6 +326,44 @@ finalize any in-progress assistant message.
|
||||
{"type": "stream_end"}
|
||||
```
|
||||
|
||||
**`state_change`** -- the worker thread transitioned to a new state. Drives
|
||||
the client's busy-mode (composer in send vs. stop, spinner indicators,
|
||||
auto-focus on idle). Sent live during normal operation AND on every fresh
|
||||
SSE subscribe (so a mid-stream page refresh restores the correct composer
|
||||
state without waiting for the next live transition).
|
||||
|
||||
```json
|
||||
{"type": "state_change", "state": "running"}
|
||||
```
|
||||
|
||||
| Field | Type | Description |
|
||||
|----------|--------|----------------------------------------------------------------------|
|
||||
| `state` | string | One of `"running"`, `"thinking"`, `"attention"`, `"idle"`, `"error"` |
|
||||
|
||||
**`in_progress_snapshot`** -- one-shot replay of the in-progress turn's
|
||||
content + reasoning text-so-far when this client connects mid-stream.
|
||||
Lets a refreshing browser tab restore partial assistant text immediately
|
||||
instead of waiting for the response to complete. Yielded once after the
|
||||
kind-specific replay phase (history + pending), only when at least one
|
||||
of `content` / `reasoning` is non-empty. Both halves render into the same
|
||||
assistant bubble the live `content` / `reasoning` events would target;
|
||||
clients should treat the snapshot as idempotent (skip overwrite if the
|
||||
current local buffer is already a superset prefix — covers EventSource
|
||||
auto-reconnect re-replays).
|
||||
|
||||
```json
|
||||
{
|
||||
"type": "in_progress_snapshot",
|
||||
"content": "Here is the answer so far: it depends on ",
|
||||
"reasoning": "The user is asking about a comparison; let me think about..."
|
||||
}
|
||||
```
|
||||
|
||||
| Field | Type | Description |
|
||||
|--------------|--------|------------------------------------------------------------|
|
||||
| `content` | string | Joined assistant content text accumulated this turn |
|
||||
| `reasoning` | string | Joined reasoning / chain-of-thought text accumulated |
|
||||
|
||||
**`tool_info`** -- one or more tool calls that were auto-approved (no user
|
||||
action required).
|
||||
|
||||
@@ -522,7 +561,13 @@ Each SSE connection to a workstream receives its own delivery queue. Events
|
||||
produced by the worker thread are fanned out to all registered listener queues,
|
||||
so multiple consumers (browser, console proxy, SDK) can connect
|
||||
simultaneously and each receives every event. On reconnect the client receives
|
||||
a full history replay, so no catch-up mechanism is needed.
|
||||
the kind-specific replay (`connected` + `status` + `history` + pending
|
||||
approval / plan for interactive; `connected` + `status` + pending for coord)
|
||||
followed by a `state_change` carrying the current worker state and an
|
||||
optional `in_progress_snapshot` carrying any partial content / reasoning
|
||||
buffered for the in-progress turn — so a mid-stream refresh restores both
|
||||
the busy-mode UI and the partial assistant text without waiting for the
|
||||
response to complete.
|
||||
|
||||
---
|
||||
|
||||
|
||||
+44
-4
@@ -231,11 +231,13 @@ The engine emits state changes via `_emit_state()` which calls
|
||||
|
||||
> See also: [Core Engine Classes diagram](diagrams/png/03-core-engine-classes.png)
|
||||
|
||||
Defined in `turnstone.core.session.SessionUI` as a `typing.Protocol` with 14
|
||||
Defined in `turnstone.core.session.SessionUI` as a `typing.Protocol` with 16
|
||||
methods. Every frontend must implement all of them.
|
||||
|
||||
```python
|
||||
class SessionUI(Protocol):
|
||||
def on_turn_start(self) -> None: ...
|
||||
def on_turn_committed(self) -> None: ...
|
||||
def on_thinking_start(self) -> None: ...
|
||||
def on_thinking_stop(self) -> None: ...
|
||||
def on_reasoning_token(self, text: str) -> None: ...
|
||||
@@ -252,6 +254,14 @@ class SessionUI(Protocol):
|
||||
def on_rename(self, name: str) -> None: ... # propagate alias to tab/UI label
|
||||
```
|
||||
|
||||
`on_turn_start` fires at the top of each iteration of the send-loop;
|
||||
`on_turn_committed` fires immediately after `messages.append(assistant_msg)`.
|
||||
`SessionUIBase` uses both to reset the per-turn inflight buffers
|
||||
(`_ws_inflight_content` / `_ws_inflight_reasoning` / `_ws_inflight_seq`)
|
||||
that fuel the SSE refresh-resume `in_progress_snapshot` event — see
|
||||
the per-workstream events stream in
|
||||
[`docs/api-reference.md`](api-reference.md#get-v1apiworkstreamsws_idevents).
|
||||
|
||||
`on_rename` is called by the `/name` command (on success) and after a successful `/resume` (if the resumed session has an alias or title). `WebUI.on_rename` broadcasts a `ws_rename` event on the global SSE channel and updates the in-memory `Workstream.name`; `TerminalUI.on_rename` is a no-op.
|
||||
|
||||
### Three Implementations
|
||||
@@ -619,14 +629,15 @@ LLMProvider (protocol)
|
||||
| `get_capabilities()` | Per-model flags (`ModelCapabilities`) |
|
||||
| `convert_tools()` | Translate OpenAI tool schemas to provider format |
|
||||
| `retryable_error_names` | Exception class names that trigger retry |
|
||||
| `extract_reasoning_text()` | Walk stored `provider_blocks`, return concatenated reasoning text for UI rehydration (per-provider block-type knowledge: Anthropic `thinking`, OpenAI Responses `reasoning`, OpenAI Chat synthetic `reasoning_text`) |
|
||||
|
||||
**Normalized data types:**
|
||||
|
||||
| Type | Fields |
|
||||
|------|--------|
|
||||
| `StreamChunk` | `content_delta`, `reasoning_delta`, `tool_call_deltas`, `info_delta`, `usage`, `finish_reason` |
|
||||
| `CompletionResult` | `content`, `tool_calls`, `finish_reason`, `usage` |
|
||||
| `ModelCapabilities` | `context_window`, `max_output_tokens`, `supports_temperature`, `token_param`, `thinking_mode`, `supports_effort`, `supports_web_search`, `supports_tool_search`, `supports_vision` |
|
||||
| `StreamChunk` | `content_delta`, `reasoning_delta`, `tool_call_deltas`, `info_delta`, `usage`, `finish_reason`, `provider_blocks` |
|
||||
| `CompletionResult` | `content`, `tool_calls`, `finish_reason`, `usage`, `provider_blocks` |
|
||||
| `ModelCapabilities` | `context_window`, `max_output_tokens`, `supports_temperature`, `token_param`, `thinking_mode`, `supports_effort`, `supports_web_search`, `supports_tool_search`, `supports_vision`, `supports_reasoning_replay` |
|
||||
| `UsageInfo` | `prompt_tokens`, `completion_tokens`, `total_tokens`, `cache_creation_tokens`, `cache_read_tokens` |
|
||||
|
||||
**OpenAIProvider** (`_openai.py`): passes messages through unchanged (they are
|
||||
@@ -714,6 +725,35 @@ and `"openai-compatible"`.
|
||||
`max_tokens`, and `reasoning_effort` to override the global defaults from
|
||||
ConfigStore. When unset (`NULL`), the global default is used.
|
||||
|
||||
**Per-model reasoning persistence:** Two booleans on `model_definitions`
|
||||
(migration 052) control how reasoning text round-trips:
|
||||
|
||||
* `surface_persisted_reasoning` (default `True`) — gates whether stored
|
||||
reasoning text is surfaced on `/history` payloads for UI rehydration.
|
||||
**Storage of reasoning bytes happens regardless of this flag** — they
|
||||
ride in `provider_data` independently. Phase-1 admin UI label "Surface
|
||||
persisted reasoning."
|
||||
* `replay_reasoning_to_model` (default `False`) — gates whether stored
|
||||
reasoning blocks are sent back to the provider on subsequent turns.
|
||||
Capability-gated: `ModelCapabilities.supports_reasoning_replay` must
|
||||
also be `True` for the wire path to actually replay (canonical OpenAI
|
||||
gpt-5*/o-series and Anthropic Claude entries set it; unknown / local-
|
||||
server models default to `False`).
|
||||
|
||||
Three reasoning paths are recognised:
|
||||
|
||||
| Path | Provider | Capture | Persist | Replay |
|
||||
|------|----------|---------|---------|--------|
|
||||
| 1 | Anthropic Messages API | `thinking_delta` | `provider_blocks` (`type="thinking"`) | Verbatim via `_provider_content` |
|
||||
| 2 | OpenAI Responses (gpt-5*, o-series) | `response.reasoning_text.delta` events | `provider_blocks` (`type="reasoning"`) — only when `include=["reasoning.encrypted_content"]` | `ResponseReasoningItemParam` input items |
|
||||
| 3 | OpenAI Chat Completions (vLLM, llama.cpp, Gemini-compat) | `delta.reasoning_content` Pydantic extras | Synthetic `{type: "reasoning_text", text, source}` block stamped at end-of-stream | None — no API surface for replay on Chat Completions |
|
||||
|
||||
Cross-provider safety is enforced by `ANTHROPIC_VALID_BLOCK_TYPES` (a
|
||||
shape filter in `_anthropic.py:_convert_messages`): foreign blocks
|
||||
(OpenAI `reasoning`, synthetic `reasoning_text`) fall through to the
|
||||
text+tool_calls rebuild path rather than reaching Anthropic's input
|
||||
boundary as malformed content.
|
||||
|
||||
```toml
|
||||
[models.local]
|
||||
base_url = "http://localhost:8000/v1"
|
||||
|
||||
@@ -115,7 +115,8 @@ with a `type` field. The recurring shapes a UI has to handle:
|
||||
| `approve_request` | One or more tool calls need operator approval | `items: [{call_id, header, preview, func_name, approval_label, needs_approval}]` |
|
||||
| `approval_resolved` | Operator answered the approval prompt | `approved`, `feedback` |
|
||||
| `state_change` | Worker-thread state transition (also re-emitted with the current state on every fresh subscribe so refresh-mid-stream restores composer mode) | `state` ∈ `running`, `thinking`, `attention`, `idle`, `error` |
|
||||
| `status` | Token usage + context-window snapshot (fires on every streaming tick) | `prompt_tokens`, `completion_tokens`, `total_tokens`, `context_window`, `pct`, `effort`, `cache_creation_tokens`, `cache_read_tokens` |
|
||||
| `in_progress_snapshot` | One-shot replay of the in-progress turn's content + reasoning when this client connects mid-stream | `content`, `reasoning` |
|
||||
| `status` | Token usage + context-window snapshot (fires on every streaming tick) | `prompt_tokens`, `completion_tokens`, `total_tokens`, `context_window`, `pct`, `effort`, `cache_creation_tokens`, `cache_read_tokens` |
|
||||
| `rename` | Session's display name changed | `name` |
|
||||
| `intent_verdict` | Intent judge produced a verdict on a pending tool call | `risk_level`, `recommendation`, `reasons` |
|
||||
| `output_warning` | Output guard flagged a tool result | `call_id`, `risk_level`, `flags` |
|
||||
@@ -130,9 +131,13 @@ with a `type` field. The recurring shapes a UI has to handle:
|
||||
**Reconnection contract:** a freshly-opened SSE connection receives
|
||||
the current snapshot of any pending tool approval (`approve_request`
|
||||
is re-sent if unresolved), any in-flight `wait_*` / `batch_*`
|
||||
indicator — so a tab refresh mid-approval doesn't strand the
|
||||
operator.
|
||||
|
||||
indicator, the worker's current `state_change`, and an
|
||||
`in_progress_snapshot` carrying any partial content / reasoning the
|
||||
model has produced for the in-progress turn — so a tab refresh
|
||||
mid-approval, mid-tool-execution, or mid-stream restores both the
|
||||
correct composer mode and the partial assistant text without waiting
|
||||
for the response to complete.
|
||||
|
||||
---
|
||||
|
||||
## 3. Send the first user message
|
||||
|
||||
@@ -69,9 +69,10 @@ class "NullUI" as NullUI {
|
||||
interface "LLMProvider" as LLMProvider <<Protocol>> {
|
||||
+ provider_name: str {property}
|
||||
+ get_capabilities(model) → ModelCapabilities
|
||||
+ create_streaming(client, model, messages, ...) → Iterator[StreamChunk]
|
||||
+ create_completion(client, model, messages, ...) → CompletionResult
|
||||
+ create_streaming(client, model, messages, ..., replay_reasoning_to_model) → Iterator[StreamChunk]
|
||||
+ create_completion(client, model, messages, ..., replay_reasoning_to_model) → CompletionResult
|
||||
+ convert_tools(tools) → list[dict]
|
||||
+ extract_reasoning_text(provider_blocks) → str
|
||||
+ retryable_error_names: frozenset[str] {property}
|
||||
--
|
||||
core/providers/_protocol.py
|
||||
@@ -126,6 +127,7 @@ class "ModelCapabilities" as ModelCaps <<frozen>> {
|
||||
+ supports_web_search: bool
|
||||
+ supports_tool_search: bool
|
||||
+ supports_vision: bool
|
||||
+ supports_reasoning_replay: bool
|
||||
}
|
||||
|
||||
' ChatSession
|
||||
|
||||
@@ -24,6 +24,14 @@ CS -> DB : save_message(ws_id, "user", input)
|
||||
|
||||
group loop [while tool_calls present]
|
||||
|
||||
CS -> UI : on_turn_start()
|
||||
note right of UI
|
||||
SessionUIBase resets the per-turn inflight
|
||||
buffers (_ws_inflight_content / reasoning /
|
||||
seq) that fuel the SSE in_progress_snapshot
|
||||
event for mid-stream refresh resume.
|
||||
end note
|
||||
|
||||
CS -> UI : on_state_change("thinking")
|
||||
CS -> UI : on_thinking_start()
|
||||
|
||||
@@ -73,6 +81,14 @@ group loop [while tool_calls present]
|
||||
|
||||
CS -> CS : _update_token_table()\ncalibrate chars_per_token ratio
|
||||
CS -> CS : messages.append(assistant_msg)
|
||||
CS -> UI : on_turn_committed()
|
||||
note right of UI
|
||||
Drops the per-turn inflight buffers — the
|
||||
assistant message is now in the history
|
||||
list, so the in_progress_snapshot must
|
||||
not re-render it during the next tool-
|
||||
execution window or the next streaming turn.
|
||||
end note
|
||||
CS -> DB : save_message(ws_id, "assistant", content)
|
||||
CS -> DB : save_message(ws_id, "tool_call", ...) ×N
|
||||
|
||||
|
||||
@@ -1,3 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:25b5448bbb7da8ddafe4f65c6c5e6cbcaa9cb9f31746ca46d3a2241bc47b1956
|
||||
size 259687
|
||||
oid sha256:9857db23fe3c4316d492073aac69c7e7558b1abe3b95ad7756d4a5933bd0ece7
|
||||
size 620214
|
||||
|
||||
@@ -1,3 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:3aa8d972bba40d78152f9f0c762b9f5ec616d8052c45fa52b7dd1c679ed81d61
|
||||
size 325245
|
||||
oid sha256:c14dfbb2db8dcb22cd332b2cf0e53ba75141adb213dae47dd9dbfd389ed482fe
|
||||
size 354799
|
||||
|
||||
@@ -138,6 +138,9 @@ SSE events are deserialized into typed dataclasses. Use `event.type` to discrimi
|
||||
| `error` | `ErrorEvent` | `message` |
|
||||
| `info` | `InfoEvent` | `message` |
|
||||
| `stream_end` | `StreamEndEvent` | — |
|
||||
| `state_change` | `StateChangeEvent` | `state` ∈ `running`/`thinking`/`attention`/`idle`/`error` |
|
||||
| `in_progress_snapshot` | `InProgressSnapshotEvent` | `content`, `reasoning` (one-shot mid-stream refresh resume) |
|
||||
| `approval_resolved` | `ApprovalResolvedEvent` | `approved`, `feedback` |
|
||||
| `cancelled` | `CancelledEvent` | — |
|
||||
|
||||
**Global events** (from `stream_global_events()`):
|
||||
|
||||
@@ -59,6 +59,21 @@ from ConfigStore. Model names and context windows are now configured per-model
|
||||
in the Models tab. A startup warning is logged if these keys appear in
|
||||
`config.toml`.
|
||||
|
||||
### Reasoning persistence (per-model)
|
||||
|
||||
Two boolean flags on `model_definitions` (migration 052) control how
|
||||
reasoning text round-trips per model:
|
||||
|
||||
| Flag | Default | Effect |
|
||||
|------|---------|--------|
|
||||
| `surface_persisted_reasoning` | `True` | Surface stored reasoning text on `/history` payloads so a page reload re-renders the reasoning bubble. **Storage of reasoning bytes is independent of this flag** — they ride in `provider_data` regardless. |
|
||||
| `replay_reasoning_to_model` | `False` | Send stored reasoning blocks back to the provider on subsequent turns. Capability-gated: only takes effect when the model's `ModelCapabilities.supports_reasoning_replay` is also `True`. Set on canonical OpenAI gpt-5*/o-series and Anthropic Claude entries; unknown / local-server models default to `False` so an operator who flips the flag on a model whose API doesn't understand reasoning replay silently no-ops rather than 400-ing. |
|
||||
|
||||
Edit both via the admin Models tab. See the architecture doc for the
|
||||
provider-side mechanics (Anthropic `thinking`, OpenAI Responses
|
||||
`reasoning` + `include=["reasoning.encrypted_content"]`, synthetic
|
||||
`reasoning_text` for Chat Completions / vLLM / llama.cpp / Gemini-compat).
|
||||
|
||||
### Plan / task agent overrides
|
||||
|
||||
`plan_agent` and `task_agent` sub-sessions resolve independently from the
|
||||
|
||||
+1
-1
@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
|
||||
|
||||
[project]
|
||||
name = "turnstone"
|
||||
version = "1.5.10"
|
||||
version = "1.5.11"
|
||||
description = "Multi-node AI orchestration platform with tool use, agent routing, and cluster simulation."
|
||||
readme = "README.md"
|
||||
license = "BUSL-1.1"
|
||||
|
||||
@@ -13,6 +13,20 @@ export interface ConnectedEvent {
|
||||
|
||||
export interface HistoryEvent {
|
||||
type: "history";
|
||||
/**
|
||||
* Per-message dicts the frontend consumes directly. Common optional keys:
|
||||
* - `role`: "user" | "assistant" | "tool"
|
||||
* - `content`: string or list (image/document parts)
|
||||
* - `tool_calls`: assistant turns — list of `{id, name, arguments, verdict?, output_assessment?}`
|
||||
* - `tool_call_id`: tool turns — id of the originating call
|
||||
* - `reminders`: metacognitive nudge bubbles (user/tool channels)
|
||||
* - `advisories`: extracted `UserInterjection` payloads on tool turns
|
||||
* - `reasoning`: concatenated reasoning text for assistant turns whose
|
||||
* `provider_data` carried reasoning-bearing blocks (Anthropic
|
||||
* `thinking`, OpenAI Responses `reasoning`, or synthetic
|
||||
* `reasoning_text` from path-3 servers). Present only when the
|
||||
* active model's `surface_persisted_reasoning` flag is true.
|
||||
*/
|
||||
messages: Array<Record<string, unknown>>;
|
||||
}
|
||||
|
||||
@@ -38,6 +52,14 @@ export interface StreamEndEvent {
|
||||
type: "stream_end";
|
||||
}
|
||||
|
||||
/** One-shot replay of the in-progress turn's content + reasoning emitted
|
||||
* by the events SSE handler when a fresh subscriber connects mid-stream. */
|
||||
export interface InProgressSnapshotEvent {
|
||||
type: "in_progress_snapshot";
|
||||
content: string;
|
||||
reasoning: string;
|
||||
}
|
||||
|
||||
export interface StateChangeEvent {
|
||||
type: "state_change";
|
||||
state: "idle" | "thinking" | "running" | "attention" | "error";
|
||||
@@ -162,6 +184,7 @@ export type ServerEvent =
|
||||
| ContentEvent
|
||||
| ReasoningEvent
|
||||
| StreamEndEvent
|
||||
| InProgressSnapshotEvent
|
||||
| StateChangeEvent
|
||||
| ToolInfoEvent
|
||||
| ApproveRequestEvent
|
||||
@@ -261,6 +284,12 @@ export function isStreamEndEvent(e: ServerEvent): e is StreamEndEvent {
|
||||
return e.type === "stream_end";
|
||||
}
|
||||
|
||||
export function isInProgressSnapshotEvent(
|
||||
e: ServerEvent,
|
||||
): e is InProgressSnapshotEvent {
|
||||
return e.type === "in_progress_snapshot";
|
||||
}
|
||||
|
||||
export function isStateChangeEvent(e: ServerEvent): e is StateChangeEvent {
|
||||
return e.type === "state_change";
|
||||
}
|
||||
|
||||
@@ -0,0 +1,45 @@
|
||||
"""Shared session-test helpers.
|
||||
|
||||
Two reasoning-test modules (``test_session_replay_reasoning.py`` and
|
||||
``test_session_synth_reasoning_block.py``) need the same minimal
|
||||
``ChatSession`` factory + a ``SessionUIBase`` no-op subclass. Hoisting
|
||||
keeps a future third caller from drifting on the defaults — the third
|
||||
existing ``_make_session`` (``test_model_registry.py``) deliberately
|
||||
takes a different signature (registry / model_alias / reasoning_effort
|
||||
+ ``_FakeUI``) and is NOT a candidate for sharing this helper.
|
||||
|
||||
Module is named with a leading underscore so pytest doesn't try to
|
||||
collect it as a test file — it's an importable utility, not a test.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Any
|
||||
from unittest.mock import MagicMock
|
||||
|
||||
from turnstone.core.session import ChatSession
|
||||
from turnstone.core.session_ui_base import SessionUIBase
|
||||
|
||||
|
||||
class NullUI(SessionUIBase):
|
||||
"""Bare-bones UI satisfying the SessionUIBase contract for tests
|
||||
that don't care about UI side effects."""
|
||||
|
||||
def __init__(self) -> None:
|
||||
super().__init__()
|
||||
|
||||
|
||||
def make_session(**kwargs: Any) -> ChatSession:
|
||||
"""Build a ChatSession with minimal defaults; tests override
|
||||
individual fields via kwargs."""
|
||||
defaults: dict[str, Any] = {
|
||||
"client": MagicMock(),
|
||||
"model": "test-model",
|
||||
"ui": NullUI(),
|
||||
"instructions": None,
|
||||
"temperature": 0.5,
|
||||
"max_tokens": 4096,
|
||||
"tool_timeout": 30,
|
||||
}
|
||||
defaults.update(kwargs)
|
||||
return ChatSession(**defaults)
|
||||
@@ -141,3 +141,126 @@ class TestRemindersWidening:
|
||||
}
|
||||
history = _build([msg])
|
||||
assert history[0]["reminders"] == [{"type": "denial", "text": "ok"}]
|
||||
|
||||
|
||||
class _StubRegistry:
|
||||
"""Minimal model registry — only ``get_config`` is read by
|
||||
``_build_history``."""
|
||||
|
||||
def __init__(self, surface_persisted_reasoning: bool = True) -> None:
|
||||
self._cfg = SimpleNamespace(surface_persisted_reasoning=surface_persisted_reasoning)
|
||||
|
||||
def get_config(self, alias: str) -> Any:
|
||||
return self._cfg
|
||||
|
||||
|
||||
def _build_with_registry(
|
||||
messages: list[dict[str, Any]],
|
||||
surface_persisted_reasoning: bool = True,
|
||||
) -> list[dict[str, Any]]:
|
||||
session = SimpleNamespace(
|
||||
messages=messages,
|
||||
_ws_id="ws-test",
|
||||
_registry=_StubRegistry(surface_persisted_reasoning=surface_persisted_reasoning),
|
||||
_model_alias="claude-opus-4-7",
|
||||
)
|
||||
with patch(
|
||||
"turnstone.server._load_verdict_indexes",
|
||||
return_value=({}, {}),
|
||||
):
|
||||
return _build_history(session)
|
||||
|
||||
|
||||
class TestReasoningSurfacing:
|
||||
"""Phase 1 — surface stored Anthropic thinking blocks on the
|
||||
history payload so refresh-the-page rehydrates the reasoning bubble.
|
||||
Drives through the real ``AnthropicProvider`` extractor (no mock-of-
|
||||
extractor) — only the model registry is stubbed.
|
||||
"""
|
||||
|
||||
def test_reasoning_surfaces_for_anthropic_thinking_msg(self) -> None:
|
||||
msg = {
|
||||
"role": "assistant",
|
||||
"content": "Final answer.",
|
||||
"_provider_content": [
|
||||
{"type": "thinking", "thinking": "let me think", "signature": "s"},
|
||||
{"type": "text", "text": "Final answer."},
|
||||
],
|
||||
}
|
||||
history = _build_with_registry([msg], surface_persisted_reasoning=True)
|
||||
assert len(history) == 1
|
||||
assert history[0]["reasoning"] == "let me think"
|
||||
|
||||
def test_reasoning_empty_when_persist_flag_false(self) -> None:
|
||||
msg = {
|
||||
"role": "assistant",
|
||||
"content": "Final answer.",
|
||||
"_provider_content": [
|
||||
{"type": "thinking", "thinking": "hidden", "signature": "s"},
|
||||
],
|
||||
}
|
||||
history = _build_with_registry([msg], surface_persisted_reasoning=False)
|
||||
assert "reasoning" not in history[0]
|
||||
|
||||
def test_provider_content_never_in_wire_entry(self) -> None:
|
||||
# The build path does not copy ``_provider_content`` into the
|
||||
# entry dict regardless of flag — wire payload stays tight.
|
||||
msg = {
|
||||
"role": "assistant",
|
||||
"content": "Final answer.",
|
||||
"_provider_content": [
|
||||
{"type": "thinking", "thinking": "x", "signature": "s"},
|
||||
],
|
||||
}
|
||||
history = _build_with_registry([msg], surface_persisted_reasoning=True)
|
||||
assert "_provider_content" not in history[0]
|
||||
|
||||
def test_no_reasoning_field_when_provider_content_missing(self) -> None:
|
||||
msg = {"role": "assistant", "content": "plain answer"}
|
||||
history = _build_with_registry([msg], surface_persisted_reasoning=True)
|
||||
assert "reasoning" not in history[0]
|
||||
|
||||
def test_no_reasoning_field_for_non_assistant_messages(self) -> None:
|
||||
# Defensive — user/tool messages with a stray _provider_content
|
||||
# do not get the reasoning field stamped.
|
||||
msgs: list[dict[str, Any]] = [
|
||||
{"role": "user", "content": "hi"},
|
||||
{
|
||||
"role": "tool",
|
||||
"tool_call_id": "c1",
|
||||
"content": "out",
|
||||
"_provider_content": [{"type": "thinking", "thinking": "leak", "signature": "s"}],
|
||||
},
|
||||
]
|
||||
history = _build_with_registry(msgs, surface_persisted_reasoning=True)
|
||||
assert "reasoning" not in history[0]
|
||||
assert "reasoning" not in history[1]
|
||||
|
||||
def test_default_true_when_registry_lookup_raises(self) -> None:
|
||||
# Conservative default — Phase 1 spec mandates rehydration on
|
||||
# refresh. A registry/alias mismatch must not silently kill the
|
||||
# bubble.
|
||||
class BrokenRegistry:
|
||||
def get_config(self, alias: str) -> Any:
|
||||
raise KeyError(alias)
|
||||
|
||||
session = SimpleNamespace(
|
||||
messages=[
|
||||
{
|
||||
"role": "assistant",
|
||||
"content": "x",
|
||||
"_provider_content": [
|
||||
{"type": "thinking", "thinking": "still works", "signature": "s"}
|
||||
],
|
||||
}
|
||||
],
|
||||
_ws_id="ws-test",
|
||||
_registry=BrokenRegistry(),
|
||||
_model_alias="missing-alias",
|
||||
)
|
||||
with patch(
|
||||
"turnstone.server._load_verdict_indexes",
|
||||
return_value=({}, {}),
|
||||
):
|
||||
history = _build_history(session)
|
||||
assert history[0]["reasoning"] == "still works"
|
||||
|
||||
@@ -19,6 +19,12 @@ class NullUI:
|
||||
self.infos = []
|
||||
self.stream_ends = 0
|
||||
|
||||
def on_turn_start(self):
|
||||
pass
|
||||
|
||||
def on_turn_committed(self):
|
||||
pass
|
||||
|
||||
def on_thinking_start(self):
|
||||
pass
|
||||
|
||||
@@ -820,3 +826,109 @@ class TestForceCancelThreaded:
|
||||
assert "idle" in ui.states
|
||||
assistant_msgs = [m for m in session.messages if m["role"] == "assistant"]
|
||||
assert any("Fresh response" in m.get("content", "") for m in assistant_msgs)
|
||||
|
||||
|
||||
class TestSynthesizeCancelledResults:
|
||||
"""Regression coverage for ``_synthesize_cancelled_results`` — must
|
||||
fire ``on_tool_result`` for each synthesized cancellation so live
|
||||
SSE listeners (e.g. coord's ``--running`` indicator added by
|
||||
tool_info) can complete the in-DOM tool batch. Without this, the
|
||||
coord JS would spin the running indicator forever on cancelled
|
||||
batches because ``state_change`` doesn't strip ``--running`` from
|
||||
individual batches."""
|
||||
|
||||
def _ui_with_tool_result_tracking(self):
|
||||
class _TrackingUI(NullUI):
|
||||
def __init__(self) -> None:
|
||||
super().__init__()
|
||||
self.tool_results: list[tuple[str, str, str, bool]] = []
|
||||
|
||||
def on_tool_result(self, call_id, name, output, **kwargs):
|
||||
self.tool_results.append(
|
||||
(call_id, name, output, bool(kwargs.get("is_error", False))),
|
||||
)
|
||||
|
||||
return _TrackingUI()
|
||||
|
||||
def test_synthesizes_tool_result_for_unanswered_calls(self, tmp_db):
|
||||
ui = self._ui_with_tool_result_tracking()
|
||||
session = _make_session(ui=ui)
|
||||
session.messages.append(
|
||||
{
|
||||
"role": "assistant",
|
||||
"content": "calling tools",
|
||||
"tool_calls": [
|
||||
{"id": "call_a", "function": {"name": "search", "arguments": "{}"}},
|
||||
{"id": "call_b", "function": {"name": "compute", "arguments": "{}"}},
|
||||
],
|
||||
},
|
||||
)
|
||||
session._msg_tokens.append(1)
|
||||
|
||||
session._synthesize_cancelled_results("Cancelled by user.")
|
||||
|
||||
# Both unanswered calls fired ``on_tool_result``.
|
||||
assert len(ui.tool_results) == 2
|
||||
ids = {tr[0] for tr in ui.tool_results}
|
||||
assert ids == {"call_a", "call_b"}
|
||||
# All emitted as errors so the live UI renders them as
|
||||
# ``coord-tool-row-result--error``.
|
||||
assert all(tr[3] is True for tr in ui.tool_results)
|
||||
# Reason text propagates as the synthetic tool output.
|
||||
assert all(tr[2] == "Cancelled by user." for tr in ui.tool_results)
|
||||
# And the message list has the synthesized tool entries
|
||||
# (preserves the prior contract).
|
||||
tool_msgs = [m for m in session.messages if m.get("role") == "tool"]
|
||||
assert len(tool_msgs) == 2
|
||||
|
||||
def test_skips_calls_already_answered(self, tmp_db):
|
||||
ui = self._ui_with_tool_result_tracking()
|
||||
session = _make_session(ui=ui)
|
||||
session.messages.append(
|
||||
{
|
||||
"role": "assistant",
|
||||
"tool_calls": [
|
||||
{"id": "call_a", "function": {"name": "search", "arguments": "{}"}},
|
||||
{"id": "call_b", "function": {"name": "compute", "arguments": "{}"}},
|
||||
],
|
||||
},
|
||||
)
|
||||
session._msg_tokens.append(1)
|
||||
# call_a already answered.
|
||||
session.messages.append(
|
||||
{"role": "tool", "tool_call_id": "call_a", "content": "result"},
|
||||
)
|
||||
session._msg_tokens.append(1)
|
||||
|
||||
session._synthesize_cancelled_results("Cancelled by user.")
|
||||
|
||||
# Only call_b synthesized.
|
||||
assert len(ui.tool_results) == 1
|
||||
assert ui.tool_results[0][0] == "call_b"
|
||||
|
||||
def test_ui_emit_failure_does_not_break_synthesis(self, tmp_db):
|
||||
"""The UI hook is wrapped in try/except — a hook failure
|
||||
during cancel must NOT compound the problem. Synthesis still
|
||||
appends to messages + storage."""
|
||||
|
||||
class _ExplodingUI(NullUI):
|
||||
def on_tool_result(self, call_id, name, output, **kwargs):
|
||||
raise RuntimeError("ui hook blew up")
|
||||
|
||||
ui = _ExplodingUI()
|
||||
session = _make_session(ui=ui)
|
||||
session.messages.append(
|
||||
{
|
||||
"role": "assistant",
|
||||
"tool_calls": [
|
||||
{"id": "call_a", "function": {"name": "search", "arguments": "{}"}},
|
||||
],
|
||||
},
|
||||
)
|
||||
session._msg_tokens.append(1)
|
||||
|
||||
# Must not raise.
|
||||
session._synthesize_cancelled_results("Cancelled by user.")
|
||||
|
||||
tool_msgs = [m for m in session.messages if m.get("role") == "tool"]
|
||||
assert len(tool_msgs) == 1
|
||||
|
||||
@@ -0,0 +1,345 @@
|
||||
"""``GET /v1/api/models`` resolution-chain coverage.
|
||||
|
||||
The console handler resolves four defaults from settings + the enabled
|
||||
model list:
|
||||
|
||||
* ``default_alias`` ← ``model.default_alias``
|
||||
* ``channel_default_alias`` ← ``channels.default_model_alias``
|
||||
* ``coordinator_default_alias`` ← ``coordinator.model_alias``, falling
|
||||
back to ``default_alias`` when empty *or* pointing at a disabled /
|
||||
removed alias (mirrors :mod:`turnstone.console.session_factory`).
|
||||
* ``judge_default_alias`` ← ``judge.model``, falling back to the
|
||||
resolved coordinator alias when empty *or* pointing at a value that
|
||||
isn't an enabled alias. ``judge.model`` is alias-only — same
|
||||
contract as the other model roles — and
|
||||
:class:`turnstone.core.judge.IntentJudge` silently inherits the
|
||||
session model when an unknown value is configured, so the API
|
||||
surfaces the resolved coordinator alias rather than echoing the
|
||||
misconfigured string.
|
||||
|
||||
These tests pin each branch so the home composer's resolved-alias
|
||||
placeholder stays correct as the precedence rules evolve.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Any
|
||||
|
||||
import pytest
|
||||
from starlette.applications import Starlette
|
||||
from starlette.middleware import Middleware
|
||||
from starlette.routing import Route
|
||||
from starlette.testclient import TestClient
|
||||
|
||||
from tests._coord_test_helpers import _AuthMiddleware, _FakeConfigStore
|
||||
from turnstone.console.server import list_available_models
|
||||
from turnstone.core.storage._sqlite import SQLiteBackend
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def storage(tmp_path: Any) -> SQLiteBackend:
|
||||
return SQLiteBackend(str(tmp_path / "available_models.db"))
|
||||
|
||||
|
||||
def _seed_model(
|
||||
storage: SQLiteBackend,
|
||||
*,
|
||||
definition_id: str,
|
||||
alias: str,
|
||||
model: str = "model-x",
|
||||
enabled: bool = True,
|
||||
) -> None:
|
||||
storage.create_model_definition(
|
||||
definition_id=definition_id,
|
||||
alias=alias,
|
||||
model=model,
|
||||
provider="openai-compatible",
|
||||
base_url="http://localhost:8000/v1",
|
||||
api_key="sk-test",
|
||||
context_window=8192,
|
||||
capabilities="{}",
|
||||
enabled=enabled,
|
||||
created_by="admin",
|
||||
)
|
||||
|
||||
|
||||
class _StubRegistry:
|
||||
"""Mimics the surface ``resolve_coordinator_alias`` reads from
|
||||
``coord_registry``: ``.default`` and ``.has_alias()``.
|
||||
|
||||
Production wires this through ``ModelRegistry``, which in turn
|
||||
pulls aliases from both DB rows and config.toml. The fixture
|
||||
mirrors the storage's enabled-row set so ``has_alias()`` agrees
|
||||
with what the placeholder's enabled-row filter would accept —
|
||||
without that alignment the helper rejects every tier-2 candidate
|
||||
and the placeholder goes blank in cases that production handles
|
||||
fine."""
|
||||
|
||||
def __init__(self, *, default: str, known: set[str]) -> None:
|
||||
self.default = default
|
||||
self._known = known
|
||||
|
||||
def has_alias(self, alias: str) -> bool:
|
||||
return alias in self._known
|
||||
|
||||
|
||||
def _make_client(
|
||||
storage: SQLiteBackend,
|
||||
*,
|
||||
settings: dict[str, str] | None = None,
|
||||
registry_default: str = "",
|
||||
config_store: bool = True,
|
||||
) -> TestClient:
|
||||
app = Starlette(
|
||||
routes=[Route("/v1/api/models", list_available_models)],
|
||||
middleware=[Middleware(_AuthMiddleware)],
|
||||
)
|
||||
app.state.auth_storage = storage
|
||||
if config_store:
|
||||
app.state.config_store = _FakeConfigStore(dict(settings or {}))
|
||||
# ``coord_registry`` is always set in production after lifespan
|
||||
# startup; mirror that here. ``has_alias`` answers from the same
|
||||
# enabled-rows set the handler filters against.
|
||||
enabled = {r["alias"] for r in storage.list_model_definitions(enabled_only=True)}
|
||||
app.state.coord_registry = _StubRegistry(default=registry_default, known=enabled)
|
||||
client = TestClient(app)
|
||||
client.headers.update({"X-Test-User": "admin", "X-Test-Perms": ""})
|
||||
return client
|
||||
|
||||
|
||||
def _get_models(client: TestClient) -> dict[str, Any]:
|
||||
resp = client.get("/v1/api/models")
|
||||
assert resp.status_code == 200, resp.text
|
||||
return resp.json()
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Coordinator resolution
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_no_settings_leaves_all_defaults_blank(storage: SQLiteBackend) -> None:
|
||||
"""No model.default_alias, no per-role overrides → every default
|
||||
field is empty and ``models`` is an empty list."""
|
||||
body = _get_models(_make_client(storage))
|
||||
assert body == {
|
||||
"models": [],
|
||||
"default_alias": "",
|
||||
"channel_default_alias": "",
|
||||
"coordinator_default_alias": "",
|
||||
"judge_default_alias": "",
|
||||
}
|
||||
|
||||
|
||||
def test_coordinator_inherits_default_alias_when_unset(
|
||||
storage: SQLiteBackend,
|
||||
) -> None:
|
||||
_seed_model(storage, definition_id="m1", alias="primary")
|
||||
body = _get_models(_make_client(storage, settings={"model.default_alias": "primary"}))
|
||||
assert body["default_alias"] == "primary"
|
||||
assert body["coordinator_default_alias"] == "primary"
|
||||
|
||||
|
||||
def test_coordinator_explicit_enabled_alias_passes_through(
|
||||
storage: SQLiteBackend,
|
||||
) -> None:
|
||||
_seed_model(storage, definition_id="m1", alias="primary")
|
||||
_seed_model(storage, definition_id="m2", alias="fast")
|
||||
body = _get_models(
|
||||
_make_client(
|
||||
storage,
|
||||
settings={
|
||||
"model.default_alias": "primary",
|
||||
"coordinator.model_alias": "fast",
|
||||
},
|
||||
)
|
||||
)
|
||||
assert body["coordinator_default_alias"] == "fast"
|
||||
|
||||
|
||||
def test_coordinator_set_to_disabled_alias_falls_back_to_default(
|
||||
storage: SQLiteBackend,
|
||||
) -> None:
|
||||
"""Operator disabled the alias the coordinator was pinned to —
|
||||
fall back to the registry default rather than advertising a model
|
||||
that workstream creation would refuse to use."""
|
||||
_seed_model(storage, definition_id="m1", alias="primary")
|
||||
_seed_model(storage, definition_id="m2", alias="legacy", enabled=False)
|
||||
body = _get_models(
|
||||
_make_client(
|
||||
storage,
|
||||
settings={
|
||||
"model.default_alias": "primary",
|
||||
"coordinator.model_alias": "legacy",
|
||||
},
|
||||
)
|
||||
)
|
||||
assert body["coordinator_default_alias"] == "primary"
|
||||
|
||||
|
||||
def test_coordinator_set_to_unknown_alias_falls_back_to_default(
|
||||
storage: SQLiteBackend,
|
||||
) -> None:
|
||||
_seed_model(storage, definition_id="m1", alias="primary")
|
||||
body = _get_models(
|
||||
_make_client(
|
||||
storage,
|
||||
settings={
|
||||
"model.default_alias": "primary",
|
||||
"coordinator.model_alias": "ghost",
|
||||
},
|
||||
)
|
||||
)
|
||||
assert body["coordinator_default_alias"] == "primary"
|
||||
|
||||
|
||||
def test_coordinator_falls_back_to_registry_default_when_config_store_empty(
|
||||
storage: SQLiteBackend,
|
||||
) -> None:
|
||||
"""Match ``console/session_factory.py:109-110``: when both
|
||||
``coordinator.model_alias`` and ``model.default_alias`` are unset, new
|
||||
coordinator sessions run on ``registry.default`` (loaded from
|
||||
config.toml ``[model].default``). The placeholder must report the
|
||||
same alias rather than going blank — otherwise the home composer
|
||||
advertises "Default model" while sessions actually launch on a
|
||||
concrete alias."""
|
||||
_seed_model(storage, definition_id="m1", alias="primary")
|
||||
body = _get_models(_make_client(storage, registry_default="primary"))
|
||||
assert body["default_alias"] == ""
|
||||
assert body["coordinator_default_alias"] == "primary"
|
||||
assert body["judge_default_alias"] == "primary"
|
||||
|
||||
|
||||
def test_coordinator_skips_registry_default_when_alias_disabled(
|
||||
storage: SQLiteBackend,
|
||||
) -> None:
|
||||
"""Registry default points at an alias that's been disabled in the DB
|
||||
— the placeholder stays blank rather than advertising a model that
|
||||
workstream creation would refuse to use."""
|
||||
_seed_model(storage, definition_id="m1", alias="legacy", enabled=False)
|
||||
body = _get_models(_make_client(storage, registry_default="legacy"))
|
||||
assert body["coordinator_default_alias"] == ""
|
||||
|
||||
|
||||
def test_coordinator_falls_back_to_registry_default_when_config_store_missing(
|
||||
storage: SQLiteBackend,
|
||||
) -> None:
|
||||
"""Edge case from PR #500 review: lifespan can leave
|
||||
``app.state.config_store`` as None (e.g. a startup exception) while
|
||||
``coord_registry`` still binds successfully. The placeholder must
|
||||
still advertise ``registry.default`` (filtered against enabled rows)
|
||||
rather than going blank — otherwise the home composer is uselessly
|
||||
empty in a degraded-but-recoverable state."""
|
||||
_seed_model(storage, definition_id="m1", alias="primary")
|
||||
body = _get_models(_make_client(storage, registry_default="primary", config_store=False))
|
||||
assert body["default_alias"] == ""
|
||||
assert body["coordinator_default_alias"] == "primary"
|
||||
assert body["judge_default_alias"] == "primary"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Judge resolution
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_judge_empty_inherits_resolved_coordinator_alias(
|
||||
storage: SQLiteBackend,
|
||||
) -> None:
|
||||
_seed_model(storage, definition_id="m1", alias="primary")
|
||||
_seed_model(storage, definition_id="m2", alias="fast")
|
||||
body = _get_models(
|
||||
_make_client(
|
||||
storage,
|
||||
settings={
|
||||
"model.default_alias": "primary",
|
||||
"coordinator.model_alias": "fast",
|
||||
},
|
||||
)
|
||||
)
|
||||
assert body["coordinator_default_alias"] == "fast"
|
||||
assert body["judge_default_alias"] == "fast"
|
||||
|
||||
|
||||
def test_judge_explicit_enabled_alias_passes_through(
|
||||
storage: SQLiteBackend,
|
||||
) -> None:
|
||||
_seed_model(storage, definition_id="m1", alias="primary")
|
||||
_seed_model(storage, definition_id="m2", alias="judge-fast")
|
||||
body = _get_models(
|
||||
_make_client(
|
||||
storage,
|
||||
settings={
|
||||
"model.default_alias": "primary",
|
||||
"judge.model": "judge-fast",
|
||||
},
|
||||
)
|
||||
)
|
||||
assert body["judge_default_alias"] == "judge-fast"
|
||||
|
||||
|
||||
def test_judge_set_to_unknown_value_inherits_coordinator(
|
||||
storage: SQLiteBackend,
|
||||
) -> None:
|
||||
"""``judge.model`` is alias-only — same contract as the other model
|
||||
roles. An unknown value silently inherits the session model in
|
||||
:class:`IntentJudge`, so the API surfaces the resolved coordinator
|
||||
alias rather than echoing the misconfigured string."""
|
||||
_seed_model(storage, definition_id="m1", alias="primary")
|
||||
body = _get_models(
|
||||
_make_client(
|
||||
storage,
|
||||
settings={
|
||||
"model.default_alias": "primary",
|
||||
"judge.model": "anthropic/claude-haiku-4-5", # raw, not an alias
|
||||
},
|
||||
)
|
||||
)
|
||||
assert body["coordinator_default_alias"] == "primary"
|
||||
assert body["judge_default_alias"] == "primary"
|
||||
|
||||
|
||||
def test_judge_set_to_disabled_alias_inherits_coordinator(
|
||||
storage: SQLiteBackend,
|
||||
) -> None:
|
||||
"""Disabled-alias case is handled identically to the unknown-value
|
||||
case — both trip the alias-not-resolved path."""
|
||||
_seed_model(storage, definition_id="m1", alias="primary")
|
||||
_seed_model(storage, definition_id="m2", alias="judge-old", enabled=False)
|
||||
body = _get_models(
|
||||
_make_client(
|
||||
storage,
|
||||
settings={
|
||||
"model.default_alias": "primary",
|
||||
"judge.model": "judge-old",
|
||||
},
|
||||
)
|
||||
)
|
||||
assert body["judge_default_alias"] == "primary"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Pre-existing fields stay correct under the new resolution code
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_channel_default_alias_blanked_when_disabled(
|
||||
storage: SQLiteBackend,
|
||||
) -> None:
|
||||
_seed_model(storage, definition_id="m1", alias="primary", enabled=False)
|
||||
body = _get_models(
|
||||
_make_client(
|
||||
storage,
|
||||
settings={"channels.default_model_alias": "primary"},
|
||||
)
|
||||
)
|
||||
assert body["channel_default_alias"] == ""
|
||||
|
||||
|
||||
def test_models_payload_strips_secret_fields(storage: SQLiteBackend) -> None:
|
||||
"""Regression guard: only alias/model/provider land in the response,
|
||||
never api_key / base_url / context_window / capabilities."""
|
||||
_seed_model(storage, definition_id="m1", alias="primary")
|
||||
body = _get_models(_make_client(storage))
|
||||
assert body["models"] == [
|
||||
{"alias": "primary", "model": "model-x", "provider": "openai-compatible"}
|
||||
]
|
||||
@@ -0,0 +1,219 @@
|
||||
"""``console/session_factory.py`` alias-resolution coverage.
|
||||
|
||||
The console session factory resolves the coordinator alias through a
|
||||
three-tier chain that must stay in lockstep with the placeholder logic
|
||||
in ``console/server.py:list_available_models`` — otherwise the home
|
||||
composer advertises one alias while sessions launch on another.
|
||||
|
||||
Tier order (highest priority first):
|
||||
|
||||
1. Per-call ``model_alias`` arg, or the ``coordinator.model_alias``
|
||||
ConfigStore setting (admin-pinned coordinator-specific override).
|
||||
2. ``model.default_alias`` ConfigStore setting (admin-managed system
|
||||
default surfaced in the Models tab).
|
||||
3. ``registry.default`` (config.toml ``[model].default``, the boot-time
|
||||
fallback).
|
||||
|
||||
These tests pin each branch by intercepting ``registry.resolve`` —
|
||||
they short-circuit before ChatSession construction so the test never
|
||||
has to satisfy ChatSession's full kwarg contract.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Any
|
||||
from unittest.mock import MagicMock
|
||||
|
||||
import pytest
|
||||
|
||||
from tests._coord_test_helpers import _FakeConfigStore
|
||||
from turnstone.console.session_factory import build_console_session_factory
|
||||
|
||||
|
||||
class _StopBeforeChatSessionError(Exception):
|
||||
"""Sentinel raised by the capturing registry to short-circuit
|
||||
factory execution after alias resolution but before ChatSession is
|
||||
built. The factory's outer code path is irrelevant to alias
|
||||
resolution and would force the test to satisfy a long kwarg
|
||||
contract for no extra coverage."""
|
||||
|
||||
|
||||
class _CapturingRegistry:
|
||||
"""Records the alias passed to ``resolve()`` and short-circuits.
|
||||
|
||||
``has_alias`` answers from the configured known set so the
|
||||
``model.default_alias`` validation tier behaves realistically.
|
||||
Mirrors the public surface ``ModelRegistry`` exposes to
|
||||
session_factory: ``has_alias``, ``resolve``, and ``default``.
|
||||
"""
|
||||
|
||||
def __init__(self, *, default: str, known: set[str]) -> None:
|
||||
self.default = default
|
||||
self._known = known
|
||||
self.captured_alias: str | None = None
|
||||
|
||||
def has_alias(self, alias: str) -> bool:
|
||||
return alias in self._known
|
||||
|
||||
def resolve(self, alias: str) -> Any:
|
||||
self.captured_alias = alias
|
||||
raise _StopBeforeChatSessionError()
|
||||
|
||||
|
||||
def _build_factory(
|
||||
*,
|
||||
registry_default: str = "registry-default",
|
||||
known_aliases: set[str] | None = None,
|
||||
settings: dict[str, Any] | None = None,
|
||||
) -> tuple[Any, _CapturingRegistry]:
|
||||
"""Construct the factory with stub deps. Returns ``(factory_callable,
|
||||
registry)`` so tests can read back ``registry.captured_alias``."""
|
||||
|
||||
registry = _CapturingRegistry(
|
||||
default=registry_default,
|
||||
known=known_aliases if known_aliases is not None else {registry_default},
|
||||
)
|
||||
config_store = _FakeConfigStore(dict(settings or {}))
|
||||
factory = build_console_session_factory(
|
||||
registry=registry, # type: ignore[arg-type]
|
||||
config_store=config_store, # type: ignore[arg-type]
|
||||
node_id="console",
|
||||
coord_client_factory=lambda ws_id, uid: MagicMock(),
|
||||
)
|
||||
return factory, registry
|
||||
|
||||
|
||||
def _invoke(factory: Any, **factory_kwargs: Any) -> None:
|
||||
"""Call the factory with a stub UI and absorb the sentinel.
|
||||
|
||||
Forwards ``factory_kwargs`` to the factory so per-call overrides
|
||||
(e.g. ``model_alias``) can flow through. Raises if any other
|
||||
exception comes out — the test should fail loudly when alias
|
||||
resolution itself errors rather than swallowing it.
|
||||
"""
|
||||
ui = MagicMock()
|
||||
ui._user_id = "" # skip storage-backed username lookup branch
|
||||
with pytest.raises(_StopBeforeChatSessionError):
|
||||
factory(ui, **factory_kwargs)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Tier 1 — explicit pin (per-call arg or coordinator.model_alias)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_per_call_model_alias_arg_wins_over_everything() -> None:
|
||||
"""The ``model_alias`` kwarg on the factory call (e.g. body field on
|
||||
POST /workstreams/new) wins over both ConfigStore tiers and the
|
||||
registry default."""
|
||||
factory, registry = _build_factory(
|
||||
known_aliases={"per-call", "coord-pin", "admin-default", "registry-default"},
|
||||
settings={
|
||||
"coordinator.model_alias": "coord-pin",
|
||||
"model.default_alias": "admin-default",
|
||||
},
|
||||
)
|
||||
_invoke(factory, model_alias="per-call")
|
||||
assert registry.captured_alias == "per-call"
|
||||
|
||||
|
||||
def test_coordinator_model_alias_wins_when_no_per_call_override() -> None:
|
||||
factory, registry = _build_factory(
|
||||
known_aliases={"coord-pin", "admin-default", "registry-default"},
|
||||
settings={
|
||||
"coordinator.model_alias": "coord-pin",
|
||||
"model.default_alias": "admin-default",
|
||||
},
|
||||
)
|
||||
_invoke(factory)
|
||||
assert registry.captured_alias == "coord-pin"
|
||||
|
||||
|
||||
def test_coordinator_model_alias_passed_through_unvalidated() -> None:
|
||||
"""Tier 1 is an *explicit* operator pin — when it's stale or typoed
|
||||
we deliberately pass it through to ``registry.resolve`` so the
|
||||
request layer turns it into a 503 with the alias surfaced in the
|
||||
error. Falling through silently would mask the misconfiguration."""
|
||||
factory, registry = _build_factory(
|
||||
known_aliases={"admin-default", "registry-default"},
|
||||
settings={
|
||||
"coordinator.model_alias": "ghost", # unknown
|
||||
"model.default_alias": "admin-default",
|
||||
},
|
||||
)
|
||||
_invoke(factory)
|
||||
assert registry.captured_alias == "ghost"
|
||||
|
||||
|
||||
def test_per_call_model_alias_arg_passed_through_unvalidated() -> None:
|
||||
"""The per-call ``model_alias`` kwarg (POST body field — the more
|
||||
common production trigger) is the same kind of explicit pin as the
|
||||
ConfigStore setting, so a stale value passes through to
|
||||
``registry.resolve`` rather than silently falling through to the
|
||||
system default."""
|
||||
factory, registry = _build_factory(
|
||||
known_aliases={"registry-default"},
|
||||
settings={"model.default_alias": "registry-default"},
|
||||
)
|
||||
_invoke(factory, model_alias="ghost")
|
||||
assert registry.captured_alias == "ghost"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Tier 2 — model.default_alias (admin-managed system default)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_model_default_alias_used_when_coordinator_unset() -> None:
|
||||
"""Regression for the historical drift: admin sets the system
|
||||
default in the Models tab, the home composer advertises it, and new
|
||||
coordinator sessions must launch on the same alias rather than
|
||||
silently falling through to ``registry.default``."""
|
||||
factory, registry = _build_factory(
|
||||
known_aliases={"admin-default", "registry-default"},
|
||||
settings={"model.default_alias": "admin-default"},
|
||||
)
|
||||
_invoke(factory)
|
||||
assert registry.captured_alias == "admin-default"
|
||||
|
||||
|
||||
def test_unknown_model_default_alias_falls_through_to_registry_default() -> None:
|
||||
"""Tier 2 is *not* an explicit pin — operators set
|
||||
``model.default_alias`` once in the UI and forget about it; an alias
|
||||
that's later disabled or typo'd should not 503 the coordinator,
|
||||
since tier 3 (``registry.default``) is guaranteed to resolve."""
|
||||
factory, registry = _build_factory(
|
||||
known_aliases={"registry-default"}, # admin-default got removed
|
||||
settings={"model.default_alias": "admin-default"},
|
||||
)
|
||||
_invoke(factory)
|
||||
assert registry.captured_alias == "registry-default"
|
||||
|
||||
|
||||
def test_blank_model_default_alias_falls_through_to_registry_default() -> None:
|
||||
factory, registry = _build_factory(
|
||||
settings={"model.default_alias": ""},
|
||||
)
|
||||
_invoke(factory)
|
||||
assert registry.captured_alias == "registry-default"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Tier 3 — registry.default (config.toml [model].default)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_no_settings_uses_registry_default() -> None:
|
||||
factory, registry = _build_factory()
|
||||
_invoke(factory)
|
||||
assert registry.captured_alias == "registry-default"
|
||||
|
||||
|
||||
def test_whitespace_only_coord_alias_falls_through() -> None:
|
||||
"""``" "`` is not an explicit pin — ``.strip()`` reduces it to
|
||||
"", which the chain should treat as unset."""
|
||||
factory, registry = _build_factory(
|
||||
settings={"coordinator.model_alias": " "},
|
||||
)
|
||||
_invoke(factory)
|
||||
assert registry.captured_alias == "registry-default"
|
||||
@@ -453,3 +453,225 @@ class TestDecorateAdvisoryExtraction:
|
||||
assert tool_msg["advisories"] == [
|
||||
{"type": "user_interjection", "text": "check the logs", "priority": "notice"}
|
||||
]
|
||||
|
||||
|
||||
class TestExtractReasoningForHistory:
|
||||
"""``extract_reasoning_for_history`` — Phase 1 surfaces stored
|
||||
Anthropic thinking blocks on assistant messages and strips
|
||||
``_provider_content`` from the wire payload.
|
||||
|
||||
Drives through the real ``AnthropicProvider.extract_reasoning_text``
|
||||
(no mock-of-extractor) — the helper test and the provider unit
|
||||
test (``tests/test_provider_anthropic_reasoning.py``) together
|
||||
catch a regression at either layer distinctly.
|
||||
"""
|
||||
|
||||
def _anthropic_thinking_msg(self, text: str = "let me think") -> dict[str, object]:
|
||||
return {
|
||||
"role": "assistant",
|
||||
"content": "Final answer.",
|
||||
"_provider_content": [
|
||||
{"type": "thinking", "thinking": text, "signature": "sig"},
|
||||
{"type": "text", "text": "Final answer."},
|
||||
],
|
||||
}
|
||||
|
||||
def test_extract_thinking_surfaces_reasoning_field(self) -> None:
|
||||
from turnstone.core.history_decoration import extract_reasoning_for_history
|
||||
|
||||
messages = [self._anthropic_thinking_msg("let me think")]
|
||||
extract_reasoning_for_history(messages, surface_persisted_reasoning_flag=True)
|
||||
assert messages[0]["reasoning"] == "let me think"
|
||||
|
||||
def test_strips_provider_content_after_extraction(self) -> None:
|
||||
from turnstone.core.history_decoration import extract_reasoning_for_history
|
||||
|
||||
messages = [self._anthropic_thinking_msg("anything")]
|
||||
extract_reasoning_for_history(messages, surface_persisted_reasoning_flag=True)
|
||||
assert "_provider_content" not in messages[0]
|
||||
|
||||
def test_strips_provider_content_when_flag_false(self) -> None:
|
||||
from turnstone.core.history_decoration import extract_reasoning_for_history
|
||||
|
||||
messages = [self._anthropic_thinking_msg("anything")]
|
||||
extract_reasoning_for_history(messages, surface_persisted_reasoning_flag=False)
|
||||
# Strip is unconditional; reasoning is the conditional bit.
|
||||
assert "_provider_content" not in messages[0]
|
||||
assert "reasoning" not in messages[0]
|
||||
|
||||
def test_first_block_thinking_dispatches_to_anthropic(self) -> None:
|
||||
# Even when text and tool_use blocks follow, the first-block-type
|
||||
# discriminator routes thinking-prefixed payloads correctly.
|
||||
from turnstone.core.history_decoration import extract_reasoning_for_history
|
||||
|
||||
messages = [
|
||||
{
|
||||
"role": "assistant",
|
||||
"content": "x",
|
||||
"_provider_content": [
|
||||
{"type": "thinking", "thinking": "first", "signature": "s"},
|
||||
{"type": "text", "text": "spoken"},
|
||||
{"type": "tool_use", "id": "t1", "name": "f", "input": {}},
|
||||
],
|
||||
}
|
||||
]
|
||||
extract_reasoning_for_history(messages, surface_persisted_reasoning_flag=True)
|
||||
assert messages[0]["reasoning"] == "first"
|
||||
|
||||
def test_first_block_reasoning_dispatches_to_openai_responses(self) -> None:
|
||||
# Phase 3: dispatcher routes type=="reasoning" to the
|
||||
# OpenAI Responses extractor, which now returns the
|
||||
# summary[*].text concatenation. Pre-Phase-3 this asserted
|
||||
# "" (the stub); the assertion was tightened once the wire
|
||||
# path landed.
|
||||
from turnstone.core.history_decoration import extract_reasoning_for_history
|
||||
|
||||
messages = [
|
||||
{
|
||||
"role": "assistant",
|
||||
"content": "x",
|
||||
"_provider_content": [
|
||||
{"type": "reasoning", "summary": [{"type": "summary_text", "text": "s"}]}
|
||||
],
|
||||
}
|
||||
]
|
||||
extract_reasoning_for_history(messages, surface_persisted_reasoning_flag=True)
|
||||
assert messages[0]["reasoning"] == "s"
|
||||
assert "_provider_content" not in messages[0]
|
||||
|
||||
def test_unknown_first_block_type_no_op(self) -> None:
|
||||
from turnstone.core.history_decoration import extract_reasoning_for_history
|
||||
|
||||
messages = [
|
||||
{
|
||||
"role": "assistant",
|
||||
"content": "x",
|
||||
"_provider_content": [{"type": "text", "text": "no reasoning here"}],
|
||||
}
|
||||
]
|
||||
extract_reasoning_for_history(messages, surface_persisted_reasoning_flag=True)
|
||||
assert "reasoning" not in messages[0]
|
||||
assert "_provider_content" not in messages[0]
|
||||
|
||||
def test_skips_messages_without_provider_content(self) -> None:
|
||||
from turnstone.core.history_decoration import extract_reasoning_for_history
|
||||
|
||||
messages = [{"role": "assistant", "content": "plain"}]
|
||||
extract_reasoning_for_history(messages, surface_persisted_reasoning_flag=True)
|
||||
assert "reasoning" not in messages[0]
|
||||
assert messages[0]["content"] == "plain"
|
||||
|
||||
def test_user_and_tool_messages_untouched(self) -> None:
|
||||
from turnstone.core.history_decoration import extract_reasoning_for_history
|
||||
|
||||
messages: list[dict[str, object]] = [
|
||||
{"role": "user", "content": "hi"},
|
||||
{"role": "tool", "tool_call_id": "c1", "content": "out"},
|
||||
self._anthropic_thinking_msg("only this one"),
|
||||
]
|
||||
extract_reasoning_for_history(messages, surface_persisted_reasoning_flag=True)
|
||||
assert "reasoning" not in messages[0]
|
||||
assert "reasoning" not in messages[1]
|
||||
assert messages[2]["reasoning"] == "only this one"
|
||||
|
||||
def test_empty_provider_content_no_extraction(self) -> None:
|
||||
from turnstone.core.history_decoration import extract_reasoning_for_history
|
||||
|
||||
messages = [{"role": "assistant", "content": "x", "_provider_content": []}]
|
||||
extract_reasoning_for_history(messages, surface_persisted_reasoning_flag=True)
|
||||
assert "reasoning" not in messages[0]
|
||||
# Empty-list provider_content is still stripped from the wire.
|
||||
assert "_provider_content" not in messages[0]
|
||||
|
||||
def test_first_block_not_a_dict_skipped(self) -> None:
|
||||
from turnstone.core.history_decoration import extract_reasoning_for_history
|
||||
|
||||
messages: list[dict[str, object]] = [
|
||||
{
|
||||
"role": "assistant",
|
||||
"content": "x",
|
||||
"_provider_content": ["bogus"],
|
||||
}
|
||||
]
|
||||
extract_reasoning_for_history(messages, surface_persisted_reasoning_flag=True)
|
||||
assert "reasoning" not in messages[0]
|
||||
assert "_provider_content" not in messages[0]
|
||||
|
||||
def test_first_block_reasoning_text_dispatches_to_openai_chat(self) -> None:
|
||||
# Phase 3 path 3: synthetic ``reasoning_text`` blocks (stamped
|
||||
# by ChatSession._maybe_synth_reasoning_block for vLLM /
|
||||
# llama.cpp / Gemini-compat conversations) dispatch to
|
||||
# OpenAIChatCompletionsProvider.extract_reasoning_text.
|
||||
from turnstone.core.history_decoration import extract_reasoning_for_history
|
||||
|
||||
messages = [
|
||||
{
|
||||
"role": "assistant",
|
||||
"content": "answer",
|
||||
"_provider_content": [
|
||||
{"type": "reasoning_text", "text": "synth thought", "source": "vllm"},
|
||||
],
|
||||
}
|
||||
]
|
||||
extract_reasoning_for_history(messages, surface_persisted_reasoning_flag=True)
|
||||
assert messages[0]["reasoning"] == "synth thought"
|
||||
assert "_provider_content" not in messages[0]
|
||||
|
||||
def test_dispatcher_scans_past_unrecognized_first_blocks(self) -> None:
|
||||
# Regression for Copilot finding: dispatcher used to inspect
|
||||
# only provider_content[0]['type']. OpenAI Responses captures
|
||||
# EVERY output_item.done event into provider_blocks (not just
|
||||
# reasoning), so a hypothetical [message, reasoning, ...]
|
||||
# ordering would have silently dropped the reasoning. Now
|
||||
# walks the list for the first recognised reasoning-bearing
|
||||
# type and dispatches the whole list to that provider.
|
||||
from turnstone.core.history_decoration import extract_reasoning_for_history
|
||||
|
||||
messages = [
|
||||
{
|
||||
"role": "assistant",
|
||||
"content": "answer",
|
||||
"_provider_content": [
|
||||
# First block is a non-reasoning OpenAI Responses item.
|
||||
{"type": "message", "role": "assistant", "content": "answer"},
|
||||
# Reasoning sits later in the list.
|
||||
{
|
||||
"type": "reasoning",
|
||||
"id": "r_1",
|
||||
"summary": [{"type": "summary_text", "text": "deferred"}],
|
||||
},
|
||||
],
|
||||
}
|
||||
]
|
||||
extract_reasoning_for_history(messages, surface_persisted_reasoning_flag=True)
|
||||
assert messages[0]["reasoning"] == "deferred"
|
||||
assert "_provider_content" not in messages[0]
|
||||
|
||||
def test_first_block_redacted_thinking_dispatches_to_anthropic(self) -> None:
|
||||
# Anthropic's extended-thinking API documents that
|
||||
# ``redacted_thinking`` blocks (sealed by the safety system)
|
||||
# can appear before, after, or interleaved with regular
|
||||
# ``thinking`` blocks. When the redacted block lands first,
|
||||
# the dispatcher must still route to AnthropicProvider so the
|
||||
# surrounding real thinking text surfaces — without this the
|
||||
# reasoning bubble silently disappears on history rehydration.
|
||||
# Pinned by registering "redacted_thinking" as a second key
|
||||
# in _BLOCK_TYPE_PROVIDER_FACTORY pointing at the Anthropic
|
||||
# factory; Anthropic's extractor's type=="thinking" filter
|
||||
# already correctly skips the redacted block.
|
||||
from turnstone.core.history_decoration import extract_reasoning_for_history
|
||||
|
||||
messages = [
|
||||
{
|
||||
"role": "assistant",
|
||||
"content": "answer",
|
||||
"_provider_content": [
|
||||
{"type": "redacted_thinking", "data": "sealed-blob"},
|
||||
{"type": "thinking", "thinking": "real thought", "signature": "s"},
|
||||
{"type": "text", "text": "answer"},
|
||||
],
|
||||
}
|
||||
]
|
||||
extract_reasoning_for_history(messages, surface_persisted_reasoning_flag=True)
|
||||
assert messages[0]["reasoning"] == "real thought"
|
||||
assert "_provider_content" not in messages[0]
|
||||
|
||||
@@ -55,6 +55,12 @@ class _FakeUI:
|
||||
pass
|
||||
|
||||
# ChatSession callbacks (no-op for this test)
|
||||
def on_turn_start(self) -> None:
|
||||
pass
|
||||
|
||||
def on_turn_committed(self) -> None:
|
||||
pass
|
||||
|
||||
def on_thinking_start(self) -> None:
|
||||
pass
|
||||
|
||||
|
||||
@@ -777,6 +777,58 @@ class TestModelAliasResolution:
|
||||
assert judge._client_factory_args["api_key"] == "alias-key"
|
||||
assert judge._client_factory_args["provider_name"] == "openai"
|
||||
|
||||
def test_unknown_alias_inherits_session_model(self):
|
||||
"""``judge.model`` is alias-only. A value that doesn't resolve
|
||||
through the registry inherits the session model (same path as
|
||||
an empty config.model) rather than getting pinned onto the
|
||||
session provider as a raw model id — that legacy behavior
|
||||
silently broke whenever the session provider didn't speak the
|
||||
configured model id (Anthropic session, ``judge.model =
|
||||
"gpt-5-mini"`` → every verdict came back as ``llm_fallback``)."""
|
||||
session_provider = _make_mock_provider()
|
||||
session_provider.provider_name = "anthropic"
|
||||
session_client = MagicMock()
|
||||
session_client.base_url = "https://session.example/v1"
|
||||
session_client.api_key = "session-key"
|
||||
|
||||
registry = MagicMock()
|
||||
registry.has_alias.return_value = False # judge.model isn't an alias
|
||||
|
||||
config = JudgeConfig(enabled=True, model="gpt-5-mini")
|
||||
judge = IntentJudge(
|
||||
config=config,
|
||||
session_provider=session_provider,
|
||||
session_client=session_client,
|
||||
session_model="session-default-model",
|
||||
context_window=100_000,
|
||||
model_registry=registry,
|
||||
)
|
||||
|
||||
assert judge._provider is session_provider
|
||||
assert judge._model == "session-default-model"
|
||||
# Context window mirrors the session, not the (uncalled) caps lookup.
|
||||
assert judge._judge_context_window == 100_000
|
||||
|
||||
def test_empty_model_inherits_session_model(self):
|
||||
"""Empty ``config.model`` is the documented self-consistency path."""
|
||||
session_provider = _make_mock_provider()
|
||||
session_provider.provider_name = "openai"
|
||||
session_client = MagicMock()
|
||||
session_client.base_url = "https://session.example/v1"
|
||||
session_client.api_key = "session-key"
|
||||
|
||||
config = JudgeConfig(enabled=True, model="")
|
||||
judge = IntentJudge(
|
||||
config=config,
|
||||
session_provider=session_provider,
|
||||
session_client=session_client,
|
||||
session_model="session-default-model",
|
||||
context_window=100_000,
|
||||
)
|
||||
|
||||
assert judge._provider is session_provider
|
||||
assert judge._model == "session-default-model"
|
||||
|
||||
def test_coordinator_tool_call_returns_llm_verdict_not_fallback(self):
|
||||
"""Happy-path regression for coordinator tool calls: with a properly
|
||||
resolved provider, the verdict tier must be ``llm`` — the
|
||||
|
||||
@@ -212,3 +212,73 @@ class TestModelDefinitionStorage:
|
||||
m = db.get_model_definition(did)
|
||||
assert m is not None
|
||||
assert m["temperature"] is None
|
||||
|
||||
def test_reasoning_flags_default(self, db: SQLiteBackend) -> None:
|
||||
"""surface_persisted_reasoning defaults True; replay_reasoning_to_model defaults False."""
|
||||
did = _make_id()
|
||||
db.create_model_definition(definition_id=did, alias="reason-default", model="gpt-5")
|
||||
m = db.get_model_definition(did)
|
||||
assert m is not None
|
||||
assert m["surface_persisted_reasoning"] is True
|
||||
assert m["replay_reasoning_to_model"] is False
|
||||
|
||||
def test_create_with_explicit_reasoning_flags(self, db: SQLiteBackend) -> None:
|
||||
did = _make_id()
|
||||
db.create_model_definition(
|
||||
definition_id=did,
|
||||
alias="reason-explicit",
|
||||
model="claude-opus-4-7",
|
||||
surface_persisted_reasoning=False,
|
||||
replay_reasoning_to_model=True,
|
||||
)
|
||||
m = db.get_model_definition(did)
|
||||
assert m is not None
|
||||
assert m["surface_persisted_reasoning"] is False
|
||||
assert m["replay_reasoning_to_model"] is True
|
||||
# Same values must round-trip via the alias lookup too.
|
||||
m_alias = db.get_model_definition_by_alias("reason-explicit")
|
||||
assert m_alias is not None
|
||||
assert m_alias["surface_persisted_reasoning"] is False
|
||||
assert m_alias["replay_reasoning_to_model"] is True
|
||||
|
||||
def test_update_surface_persisted_reasoning(self, db: SQLiteBackend) -> None:
|
||||
did = _make_id()
|
||||
db.create_model_definition(definition_id=did, alias="upd-persist", model="gpt-5")
|
||||
ok = db.update_model_definition(did, surface_persisted_reasoning=False)
|
||||
assert ok is True
|
||||
m = db.get_model_definition(did)
|
||||
assert m is not None
|
||||
assert m["surface_persisted_reasoning"] is False
|
||||
assert m["replay_reasoning_to_model"] is False # untouched
|
||||
|
||||
def test_update_replay_reasoning_to_model(self, db: SQLiteBackend) -> None:
|
||||
did = _make_id()
|
||||
db.create_model_definition(definition_id=did, alias="upd-replay", model="gpt-5")
|
||||
ok = db.update_model_definition(did, replay_reasoning_to_model=True)
|
||||
assert ok is True
|
||||
m = db.get_model_definition(did)
|
||||
assert m is not None
|
||||
assert m["surface_persisted_reasoning"] is True # untouched
|
||||
assert m["replay_reasoning_to_model"] is True
|
||||
|
||||
def test_list_returns_reasoning_flags(self, db: SQLiteBackend) -> None:
|
||||
db.create_model_definition(
|
||||
definition_id=_make_id(),
|
||||
alias="list-a",
|
||||
model="gpt-5",
|
||||
surface_persisted_reasoning=True,
|
||||
replay_reasoning_to_model=False,
|
||||
)
|
||||
db.create_model_definition(
|
||||
definition_id=_make_id(),
|
||||
alias="list-b",
|
||||
model="claude-opus-4-7",
|
||||
surface_persisted_reasoning=False,
|
||||
replay_reasoning_to_model=True,
|
||||
)
|
||||
models = db.list_model_definitions()
|
||||
by_alias = {m["alias"]: m for m in models}
|
||||
assert by_alias["list-a"]["surface_persisted_reasoning"] is True
|
||||
assert by_alias["list-a"]["replay_reasoning_to_model"] is False
|
||||
assert by_alias["list-b"]["surface_persisted_reasoning"] is False
|
||||
assert by_alias["list-b"]["replay_reasoning_to_model"] is True
|
||||
|
||||
@@ -76,6 +76,23 @@ class TestModelConfig:
|
||||
assert cfg.temperature == 0.0
|
||||
assert cfg.temperature is not None
|
||||
|
||||
def test_reasoning_flags_default(self) -> None:
|
||||
cfg = ModelConfig(alias="x", base_url="x", api_key="x", model="x")
|
||||
assert cfg.surface_persisted_reasoning is True
|
||||
assert cfg.replay_reasoning_to_model is False
|
||||
|
||||
def test_reasoning_flags_set(self) -> None:
|
||||
cfg = ModelConfig(
|
||||
alias="x",
|
||||
base_url="x",
|
||||
api_key="x",
|
||||
model="x",
|
||||
surface_persisted_reasoning=False,
|
||||
replay_reasoning_to_model=True,
|
||||
)
|
||||
assert cfg.surface_persisted_reasoning is False
|
||||
assert cfg.replay_reasoning_to_model is True
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# ModelRegistry
|
||||
@@ -686,6 +703,53 @@ class TestLoadModelRegistryWithDB:
|
||||
assert cfg.max_tokens is None
|
||||
assert cfg.reasoning_effort is None
|
||||
|
||||
def test_db_reasoning_flags_loaded(self) -> None:
|
||||
"""Per-model reasoning flags from DB are carried in ModelConfig."""
|
||||
storage = _MockStorage(
|
||||
[
|
||||
{
|
||||
"alias": "anth-thinking",
|
||||
"model": "claude-opus-4-7",
|
||||
"provider": "anthropic",
|
||||
"base_url": "",
|
||||
"api_key": "sk-anth",
|
||||
"context_window": 200000,
|
||||
"capabilities": "{}",
|
||||
"enabled": True,
|
||||
"surface_persisted_reasoning": False,
|
||||
"replay_reasoning_to_model": True,
|
||||
}
|
||||
]
|
||||
)
|
||||
with patch("turnstone.core.model_registry.load_config", return_value={}):
|
||||
reg = load_model_registry("http://x/v1", "x", "x", storage=storage)
|
||||
cfg = reg.get_config("anth-thinking")
|
||||
assert cfg.surface_persisted_reasoning is False
|
||||
assert cfg.replay_reasoning_to_model is True
|
||||
|
||||
def test_db_reasoning_flags_default_when_absent(self) -> None:
|
||||
"""Pre-052 rows without the columns degrade to dataclass defaults."""
|
||||
storage = _MockStorage(
|
||||
[
|
||||
{
|
||||
"alias": "legacy-row",
|
||||
"model": "gpt-5",
|
||||
"provider": "openai",
|
||||
"base_url": "",
|
||||
"api_key": "",
|
||||
"context_window": 32768,
|
||||
"capabilities": "{}",
|
||||
"enabled": True,
|
||||
# surface_persisted_reasoning + replay_reasoning_to_model intentionally absent
|
||||
}
|
||||
]
|
||||
)
|
||||
with patch("turnstone.core.model_registry.load_config", return_value={}):
|
||||
reg = load_model_registry("http://x/v1", "x", "x", storage=storage)
|
||||
cfg = reg.get_config("legacy-row")
|
||||
assert cfg.surface_persisted_reasoning is True
|
||||
assert cfg.replay_reasoning_to_model is False
|
||||
|
||||
def test_db_default_alias_not_clobbered(self) -> None:
|
||||
"""DB model with alias='default' is not overwritten by CLI args."""
|
||||
storage = _MockStorage(
|
||||
@@ -869,6 +933,8 @@ class _FakeUI:
|
||||
self.infos: list[str] = []
|
||||
self.errors: list[str] = []
|
||||
|
||||
def on_turn_start(self) -> None: ...
|
||||
def on_turn_committed(self) -> None: ...
|
||||
def on_thinking_start(self) -> None: ...
|
||||
def on_thinking_stop(self) -> None: ...
|
||||
def on_reasoning_token(self, text: str) -> None: ...
|
||||
|
||||
@@ -197,6 +197,7 @@ _EXPECTED_AFFECTING_KEYS = frozenset(
|
||||
"coordinator.model_alias",
|
||||
"coordinator.reasoning_effort",
|
||||
"judge.model",
|
||||
"channels.default_model_alias",
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
@@ -11,6 +11,12 @@ from turnstone.core.session import ChatSession, _render_template
|
||||
class NullUI:
|
||||
"""UI adapter that discards all output."""
|
||||
|
||||
def on_turn_start(self):
|
||||
pass
|
||||
|
||||
def on_turn_committed(self):
|
||||
pass
|
||||
|
||||
def on_thinking_start(self):
|
||||
pass
|
||||
|
||||
|
||||
@@ -0,0 +1,125 @@
|
||||
"""Tests for ``AnthropicProvider.extract_reasoning_text``.
|
||||
|
||||
Phase 1 of the optional-reasoning-persistence feature: provider-side
|
||||
extractor that walks stored ``provider_blocks`` and returns the
|
||||
concatenated thinking text, capped at the operator-friendly UI display
|
||||
size.
|
||||
|
||||
These tests drive through the real ``AnthropicProvider`` instance — no
|
||||
mocks of the extractor itself — using fixture-shaped blocks that match
|
||||
what ``_iter_anthropic_stream`` actually accumulates at
|
||||
``_anthropic.py:713-724`` (``thinking_delta`` + ``signature_delta``
|
||||
combined into ``{"type": "thinking", "thinking": <text>, "signature":
|
||||
<sig>}``).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import pytest
|
||||
|
||||
from turnstone.core.providers._anthropic import AnthropicProvider
|
||||
from turnstone.core.providers._openai_chat import OpenAIChatCompletionsProvider
|
||||
from turnstone.core.providers._openai_responses import OpenAIResponsesProvider
|
||||
from turnstone.core.providers._protocol import (
|
||||
MAX_REASONING_DISPLAY_CHARS as _MAX_REASONING_DISPLAY_CHARS,
|
||||
)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def anthropic() -> AnthropicProvider:
|
||||
return AnthropicProvider()
|
||||
|
||||
|
||||
class TestExtractReasoningText:
|
||||
def test_none_input_returns_empty_string(self, anthropic: AnthropicProvider) -> None:
|
||||
assert anthropic.extract_reasoning_text(None) == ""
|
||||
|
||||
def test_empty_list_returns_empty_string(self, anthropic: AnthropicProvider) -> None:
|
||||
assert anthropic.extract_reasoning_text([]) == ""
|
||||
|
||||
def test_no_thinking_blocks_returns_empty(self, anthropic: AnthropicProvider) -> None:
|
||||
blocks: list[dict[str, object]] = [
|
||||
{"type": "text", "text": "hello"},
|
||||
{"type": "tool_use", "id": "t1", "name": "x", "input": {}},
|
||||
]
|
||||
assert anthropic.extract_reasoning_text(blocks) == ""
|
||||
|
||||
def test_single_thinking_block_returns_text(self, anthropic: AnthropicProvider) -> None:
|
||||
blocks = [{"type": "thinking", "thinking": "Let me think about this.", "signature": "abc"}]
|
||||
assert anthropic.extract_reasoning_text(blocks) == "Let me think about this."
|
||||
|
||||
def test_multiple_thinking_blocks_joined_with_newline(
|
||||
self, anthropic: AnthropicProvider
|
||||
) -> None:
|
||||
blocks = [
|
||||
{"type": "thinking", "thinking": "first thought", "signature": "s1"},
|
||||
{"type": "thinking", "thinking": "second thought", "signature": "s2"},
|
||||
]
|
||||
assert anthropic.extract_reasoning_text(blocks) == "first thought\nsecond thought"
|
||||
|
||||
def test_mixed_blocks_extracts_only_thinking(self, anthropic: AnthropicProvider) -> None:
|
||||
blocks = [
|
||||
{"type": "thinking", "thinking": "reason A", "signature": "s"},
|
||||
{"type": "text", "text": "visible answer"},
|
||||
{"type": "tool_use", "id": "t1", "name": "x", "input": {}},
|
||||
{"type": "thinking", "thinking": "reason B", "signature": "s"},
|
||||
]
|
||||
assert anthropic.extract_reasoning_text(blocks) == "reason A\nreason B"
|
||||
|
||||
def test_thinking_block_without_thinking_field_skipped(
|
||||
self, anthropic: AnthropicProvider
|
||||
) -> None:
|
||||
blocks = [{"type": "thinking", "signature": "s"}]
|
||||
assert anthropic.extract_reasoning_text(blocks) == ""
|
||||
|
||||
def test_thinking_block_with_empty_text_skipped(self, anthropic: AnthropicProvider) -> None:
|
||||
blocks = [{"type": "thinking", "thinking": "", "signature": "s"}]
|
||||
assert anthropic.extract_reasoning_text(blocks) == ""
|
||||
|
||||
def test_truncation_at_64kib_cap(self, anthropic: AnthropicProvider) -> None:
|
||||
long_text = "x" * (_MAX_REASONING_DISPLAY_CHARS + 1024)
|
||||
blocks = [{"type": "thinking", "thinking": long_text, "signature": "s"}]
|
||||
result = anthropic.extract_reasoning_text(blocks)
|
||||
assert len(result) == _MAX_REASONING_DISPLAY_CHARS
|
||||
|
||||
def test_just_under_cap_not_truncated(self, anthropic: AnthropicProvider) -> None:
|
||||
text = "y" * (_MAX_REASONING_DISPLAY_CHARS - 1)
|
||||
blocks = [{"type": "thinking", "thinking": text, "signature": "s"}]
|
||||
assert anthropic.extract_reasoning_text(blocks) == text
|
||||
|
||||
def test_malformed_block_entry_skipped(self, anthropic: AnthropicProvider) -> None:
|
||||
# A defensive sanity check — we should not crash if some
|
||||
# entry isn't a dict (e.g. a corrupted JSON payload).
|
||||
blocks = [
|
||||
"not a dict", # type: ignore[list-item]
|
||||
{"type": "thinking", "thinking": "good one", "signature": "s"},
|
||||
]
|
||||
assert anthropic.extract_reasoning_text(blocks) == "good one" # type: ignore[arg-type]
|
||||
|
||||
def test_non_list_input_returns_empty(self, anthropic: AnthropicProvider) -> None:
|
||||
# Defensive against a corrupted provider_data payload.
|
||||
assert anthropic.extract_reasoning_text("not a list") == "" # type: ignore[arg-type]
|
||||
assert anthropic.extract_reasoning_text({"type": "thinking"}) == "" # type: ignore[arg-type]
|
||||
|
||||
|
||||
class TestOtherProvidersDefault:
|
||||
"""Non-Anthropic providers return "" for the same fixture shapes."""
|
||||
|
||||
def test_openai_chat_returns_empty(self) -> None:
|
||||
provider = OpenAIChatCompletionsProvider()
|
||||
blocks = [{"type": "thinking", "thinking": "would-be-text", "signature": "s"}]
|
||||
assert provider.extract_reasoning_text(blocks) == ""
|
||||
|
||||
def test_openai_responses_returns_empty(self) -> None:
|
||||
provider = OpenAIResponsesProvider()
|
||||
blocks = [{"type": "thinking", "thinking": "would-be-text", "signature": "s"}]
|
||||
assert provider.extract_reasoning_text(blocks) == ""
|
||||
|
||||
def test_openai_responses_extracts_reasoning_summary(self) -> None:
|
||||
# Phase 3: extractor now walks reasoning items captured via
|
||||
# include=["reasoning.encrypted_content"] and returns the
|
||||
# summary[*].text concatenation. Pre-Phase-3 this returned
|
||||
# "" — the stub was replaced once the wire path landed.
|
||||
provider = OpenAIResponsesProvider()
|
||||
blocks = [{"type": "reasoning", "summary": [{"type": "summary_text", "text": "x"}]}]
|
||||
assert provider.extract_reasoning_text(blocks) == "x"
|
||||
@@ -0,0 +1,532 @@
|
||||
"""Tests for Phase 2 wire-build replay flag + shape filter on AnthropicProvider.
|
||||
|
||||
Phase 2 of optional reasoning persistence wraps the verbatim
|
||||
``_provider_content`` replay path at ``_anthropic.py:_convert_messages``
|
||||
with two gates:
|
||||
|
||||
1. ``ANTHROPIC_VALID_BLOCK_TYPES`` per-block shape filter — foreign-
|
||||
shaped blocks (OpenAI Responses ``type="reasoning"``, Gemini thought
|
||||
parts, the synthetic ``reasoning_text`` from path-3 capture) are
|
||||
dropped individually; valid Anthropic blocks in the same message
|
||||
still ride the verbatim path. When NO valid blocks survive, the
|
||||
converter falls through to the text+tool_calls rebuild path.
|
||||
2. ``replay_reasoning_to_model`` operator flag — when False (the
|
||||
``model_definitions`` server_default), thinking blocks are
|
||||
stripped before the wire payload is built. Tool_use /
|
||||
server_tool_use / web_search_tool_result blocks (which carry
|
||||
web-search ``encrypted_content``) intentionally survive — the
|
||||
strip predicate is narrow by design.
|
||||
|
||||
Drives through the real ``AnthropicProvider._convert_messages`` with
|
||||
fixture-shaped messages, no mocks of the converter. Edge cases come
|
||||
from the briefing's "Edges & validation memo" sections.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import pytest
|
||||
|
||||
from turnstone.core.providers._anthropic import (
|
||||
ANTHROPIC_REASONING_BLOCK_TYPES,
|
||||
ANTHROPIC_VALID_BLOCK_TYPES,
|
||||
AnthropicProvider,
|
||||
)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def provider() -> AnthropicProvider:
|
||||
return AnthropicProvider()
|
||||
|
||||
|
||||
def _assistant_with_thinking(content: str = "Final answer.") -> dict[str, object]:
|
||||
"""Build an assistant message with a thinking + text + tool_use shape
|
||||
matching what the streaming layer captures at ``_anthropic.py:713-724``."""
|
||||
return {
|
||||
"role": "assistant",
|
||||
"content": content,
|
||||
"_provider_content": [
|
||||
{"type": "thinking", "thinking": "let me think", "signature": "sig"},
|
||||
{"type": "text", "text": content},
|
||||
{
|
||||
"type": "tool_use",
|
||||
"id": "call_abc",
|
||||
"name": "search",
|
||||
"input": {"q": "x"},
|
||||
},
|
||||
],
|
||||
"tool_calls": [
|
||||
{
|
||||
"id": "call_abc",
|
||||
"type": "function",
|
||||
"function": {"name": "search", "arguments": '{"q": "x"}'},
|
||||
}
|
||||
],
|
||||
}
|
||||
|
||||
|
||||
class TestReplayFlagStripsThinking:
|
||||
"""``replay_reasoning_to_model=False`` strips thinking; ``True`` preserves."""
|
||||
|
||||
def test_replay_true_preserves_thinking_block(self, provider: AnthropicProvider) -> None:
|
||||
msg = _assistant_with_thinking()
|
||||
_, converted = provider._convert_messages([msg], replay_reasoning_to_model=True)
|
||||
assistant = next(m for m in converted if m["role"] == "assistant")
|
||||
types_present = [b["type"] for b in assistant["content"]]
|
||||
assert "thinking" in types_present
|
||||
assert "text" in types_present
|
||||
assert "tool_use" in types_present
|
||||
|
||||
def test_replay_false_strips_thinking_block(self, provider: AnthropicProvider) -> None:
|
||||
msg = _assistant_with_thinking()
|
||||
_, converted = provider._convert_messages([msg], replay_reasoning_to_model=False)
|
||||
assistant = next(m for m in converted if m["role"] == "assistant")
|
||||
types_present = [b["type"] for b in assistant["content"]]
|
||||
assert "thinking" not in types_present
|
||||
assert "text" in types_present # final answer survives
|
||||
assert "tool_use" in types_present # tool dispatch survives
|
||||
|
||||
def test_replay_false_strips_redacted_thinking_too(self, provider: AnthropicProvider) -> None:
|
||||
# Anthropic emits redacted_thinking blocks when the safety system
|
||||
# rewrites a thinking block. Phase 2 strip predicate must include
|
||||
# both shapes.
|
||||
msg = {
|
||||
"role": "assistant",
|
||||
"content": "Answer.",
|
||||
"_provider_content": [
|
||||
{"type": "redacted_thinking", "data": "redacted-blob"},
|
||||
{"type": "text", "text": "Answer."},
|
||||
],
|
||||
}
|
||||
_, converted = provider._convert_messages([msg], replay_reasoning_to_model=False)
|
||||
assistant = next(m for m in converted if m["role"] == "assistant")
|
||||
types_present = [b["type"] for b in assistant["content"]]
|
||||
assert "redacted_thinking" not in types_present
|
||||
assert "text" in types_present
|
||||
|
||||
def test_default_kwarg_preserves_existing_behaviour(self, provider: AnthropicProvider) -> None:
|
||||
"""Pre-Phase-2 callers that don't pass the kwarg get the verbatim
|
||||
replay (default True), matching the behaviour all production
|
||||
Anthropic-with-thinking turns shipped with for months."""
|
||||
msg = _assistant_with_thinking()
|
||||
_, converted = provider._convert_messages([msg]) # no kwarg
|
||||
assistant = next(m for m in converted if m["role"] == "assistant")
|
||||
types_present = [b["type"] for b in assistant["content"]]
|
||||
assert "thinking" in types_present
|
||||
|
||||
|
||||
class TestWebSearchBlocksSurviveStrip:
|
||||
"""Edge 14: Anthropic web-search ``encrypted_content`` rides on
|
||||
``server_tool_use`` / ``web_search_tool_result`` blocks (NOT
|
||||
thinking blocks). Strip predicate is intentionally narrow."""
|
||||
|
||||
def test_server_tool_use_survives(self, provider: AnthropicProvider) -> None:
|
||||
msg = {
|
||||
"role": "assistant",
|
||||
"content": "From search: ...",
|
||||
"_provider_content": [
|
||||
{"type": "thinking", "thinking": "I should search", "signature": "s"},
|
||||
{
|
||||
"type": "server_tool_use",
|
||||
"id": "stu_1",
|
||||
"name": "web_search",
|
||||
"input": {"query": "turnstone bird"},
|
||||
},
|
||||
{
|
||||
"type": "web_search_tool_result",
|
||||
"tool_use_id": "stu_1",
|
||||
"content": [{"type": "web_search_result", "url": "https://e.com"}],
|
||||
"encrypted_content": "abc123encrypted",
|
||||
"encrypted_index": "idx456encrypted",
|
||||
},
|
||||
{"type": "text", "text": "From search: ..."},
|
||||
],
|
||||
}
|
||||
_, converted = provider._convert_messages([msg], replay_reasoning_to_model=False)
|
||||
assistant = next(m for m in converted if m["role"] == "assistant")
|
||||
types_present = [b["type"] for b in assistant["content"]]
|
||||
assert "thinking" not in types_present # stripped
|
||||
assert "server_tool_use" in types_present # survives
|
||||
assert "web_search_tool_result" in types_present # survives
|
||||
assert "text" in types_present # survives
|
||||
# encrypted_content rides through intact — required for round-trip continuity
|
||||
wsr = next(b for b in assistant["content"] if b["type"] == "web_search_tool_result")
|
||||
assert wsr["encrypted_content"] == "abc123encrypted"
|
||||
|
||||
def test_tool_use_block_survives(self, provider: AnthropicProvider) -> None:
|
||||
# Plain tool_use (not server-side) — used by client-side function
|
||||
# tools. Strip predicate must not touch these.
|
||||
msg = {
|
||||
"role": "assistant",
|
||||
"content": "Calling tool",
|
||||
"_provider_content": [
|
||||
{"type": "thinking", "thinking": "I should call tool", "signature": "s"},
|
||||
{"type": "tool_use", "id": "tu_1", "name": "f", "input": {"a": 1}},
|
||||
],
|
||||
"tool_calls": [
|
||||
{
|
||||
"id": "tu_1",
|
||||
"type": "function",
|
||||
"function": {"name": "f", "arguments": '{"a": 1}'},
|
||||
}
|
||||
],
|
||||
}
|
||||
# Provide the tool result so orphan-tool detection doesn't synthesize
|
||||
msgs = [
|
||||
msg,
|
||||
{"role": "tool", "tool_call_id": "tu_1", "content": "ok"},
|
||||
]
|
||||
_, converted = provider._convert_messages(msgs, replay_reasoning_to_model=False)
|
||||
assistant = next(m for m in converted if m["role"] == "assistant")
|
||||
types_present = [b["type"] for b in assistant["content"]]
|
||||
assert "thinking" not in types_present
|
||||
assert "tool_use" in types_present
|
||||
|
||||
def test_orphan_tool_use_synthesized_after_strip(self, provider: AnthropicProvider) -> None:
|
||||
"""Pin the post-strip orphan-tool branch at _anthropic.py:397-433.
|
||||
|
||||
The implementation comment specifically calls out reading
|
||||
``provider_content`` (not ``wire_blocks``) for the orphan-tool
|
||||
ID walk after the strip — keeping the read on the source-of-
|
||||
truth list so a future refactor that swapped them would still
|
||||
get the same set of tool_use IDs. This test exercises that
|
||||
branch end-to-end: replay=False strips the thinking block,
|
||||
AND the message has a tool_use whose result is missing. The
|
||||
converter must synthesize a 'cancelled' tool_result for the
|
||||
orphaned tool_use ID (matching the existing pre-Phase-2
|
||||
behaviour for the verbatim path).
|
||||
"""
|
||||
msg = {
|
||||
"role": "assistant",
|
||||
"content": "Calling tool",
|
||||
"_provider_content": [
|
||||
{"type": "thinking", "thinking": "let me think", "signature": "s"},
|
||||
{"type": "tool_use", "id": "orphan_tu", "name": "f", "input": {"a": 1}},
|
||||
],
|
||||
"tool_calls": [
|
||||
{
|
||||
"id": "orphan_tu",
|
||||
"type": "function",
|
||||
"function": {"name": "f", "arguments": '{"a": 1}'},
|
||||
}
|
||||
],
|
||||
}
|
||||
# NO tool result follows — orphan branch must synthesize one.
|
||||
_, converted = provider._convert_messages([msg], replay_reasoning_to_model=False)
|
||||
# Synthetic tool_result lands as a user-role message immediately
|
||||
# after the assistant turn (per existing behaviour at
|
||||
# _anthropic.py:421-430).
|
||||
assistant = next(m for m in converted if m["role"] == "assistant")
|
||||
# Stripped: thinking gone, tool_use survives.
|
||||
a_types = [b["type"] for b in assistant["content"]]
|
||||
assert "thinking" not in a_types
|
||||
assert "tool_use" in a_types
|
||||
# Synthesized: cancelled tool_result for orphan_tu attached to a
|
||||
# following user-role message.
|
||||
user_msgs_after = [m for m in converted if m["role"] == "user"]
|
||||
assert user_msgs_after, (
|
||||
"Expected a synthetic user message carrying the cancelled "
|
||||
"tool_result for the orphaned tool_use"
|
||||
)
|
||||
flat_results = [
|
||||
block
|
||||
for um in user_msgs_after
|
||||
if isinstance(um["content"], list)
|
||||
for block in um["content"]
|
||||
if isinstance(block, dict) and block.get("type") == "tool_result"
|
||||
]
|
||||
synth = next(
|
||||
(b for b in flat_results if b.get("tool_use_id") == "orphan_tu"),
|
||||
None,
|
||||
)
|
||||
assert synth is not None, f"Expected synthetic tool_result for orphan_tu in {flat_results}"
|
||||
assert synth.get("is_error") is True
|
||||
assert "cancelled" in synth.get("content", "").lower()
|
||||
|
||||
|
||||
class TestShapeFilterFallthrough:
|
||||
"""Foreign / empty / mixed-shape ``_provider_content`` falls through
|
||||
to the text+tool_calls rebuild path rather than reaching the wire
|
||||
as a malformed block."""
|
||||
|
||||
def test_foreign_shape_openai_reasoning_falls_through(
|
||||
self, provider: AnthropicProvider
|
||||
) -> None:
|
||||
# OpenAI Responses style block (Phase 3 will land this shape into
|
||||
# _provider_content via include=["reasoning.encrypted_content"]).
|
||||
# Mid-workstream model switch from OpenAI -> Anthropic must NOT
|
||||
# reach the API with an OpenAI-shaped block (which would 400).
|
||||
msg = {
|
||||
"role": "assistant",
|
||||
"content": "Final answer from openai turn.",
|
||||
"_provider_content": [
|
||||
{
|
||||
"type": "reasoning",
|
||||
"summary": [{"type": "summary_text", "text": "I reasoned..."}],
|
||||
"encrypted_content": "openai-encrypted",
|
||||
}
|
||||
],
|
||||
"tool_calls": [],
|
||||
}
|
||||
_, converted = provider._convert_messages([msg], replay_reasoning_to_model=True)
|
||||
assistant = next(m for m in converted if m["role"] == "assistant")
|
||||
# Rebuilt from text — no foreign block reached the wire.
|
||||
for b in assistant["content"]:
|
||||
assert b.get("type") in ANTHROPIC_VALID_BLOCK_TYPES, (
|
||||
f"Foreign block type leaked through: {b}"
|
||||
)
|
||||
# And the foreign block specifically is NOT present.
|
||||
types_present = [b["type"] for b in assistant["content"]]
|
||||
assert "reasoning" not in types_present
|
||||
|
||||
def test_mixed_shape_drops_foreign_keeps_valid(self, provider: AnthropicProvider) -> None:
|
||||
# Per-block filter: a single foreign block in a mostly-Anthropic
|
||||
# payload no longer forces fall-through. Valid Anthropic blocks
|
||||
# ride the verbatim path; the foreign block is dropped.
|
||||
msg = {
|
||||
"role": "assistant",
|
||||
"content": "Mixed.",
|
||||
"_provider_content": [
|
||||
{"type": "thinking", "thinking": "anth shape", "signature": "s"},
|
||||
{"type": "text", "text": "Mixed."},
|
||||
{"type": "reasoning", "summary": []}, # foreign
|
||||
],
|
||||
}
|
||||
_, converted = provider._convert_messages([msg], replay_reasoning_to_model=True)
|
||||
assistant = next(m for m in converted if m["role"] == "assistant")
|
||||
types_present = [b["type"] for b in assistant["content"]]
|
||||
assert "reasoning" not in types_present # foreign dropped
|
||||
assert "thinking" in types_present # valid + replay=True kept
|
||||
assert "text" in types_present
|
||||
for b in assistant["content"]:
|
||||
assert b.get("type") in ANTHROPIC_VALID_BLOCK_TYPES
|
||||
|
||||
def test_mixed_shape_preserves_web_search_encrypted_content(
|
||||
self, provider: AnthropicProvider
|
||||
) -> None:
|
||||
# The motivating case for per-block (vs all-or-nothing) filter:
|
||||
# cross-model resumption stamps a foreign ``reasoning`` block
|
||||
# alongside Anthropic web-search blocks carrying encrypted
|
||||
# citations. An all-or-nothing filter would discard the whole
|
||||
# message and rebuild from text+tool_calls — silently losing
|
||||
# the encrypted_content the API needs for round-trip continuity.
|
||||
msg = {
|
||||
"role": "assistant",
|
||||
"content": "From search: ...",
|
||||
"_provider_content": [
|
||||
{"type": "reasoning", "summary": []}, # foreign (e.g. OpenAI)
|
||||
{
|
||||
"type": "server_tool_use",
|
||||
"id": "stu_1",
|
||||
"name": "web_search",
|
||||
"input": {"query": "x"},
|
||||
},
|
||||
{
|
||||
"type": "web_search_tool_result",
|
||||
"tool_use_id": "stu_1",
|
||||
"content": [{"type": "web_search_result", "url": "https://e.com"}],
|
||||
"encrypted_content": "encrypted-blob-must-survive",
|
||||
"encrypted_index": "encrypted-idx-must-survive",
|
||||
},
|
||||
{"type": "text", "text": "From search: ..."},
|
||||
],
|
||||
}
|
||||
_, converted = provider._convert_messages([msg], replay_reasoning_to_model=False)
|
||||
assistant = next(m for m in converted if m["role"] == "assistant")
|
||||
types_present = [b["type"] for b in assistant["content"]]
|
||||
assert "reasoning" not in types_present # foreign dropped
|
||||
assert "server_tool_use" in types_present
|
||||
assert "web_search_tool_result" in types_present
|
||||
assert "text" in types_present
|
||||
wsr = next(b for b in assistant["content"] if b["type"] == "web_search_tool_result")
|
||||
assert wsr["encrypted_content"] == "encrypted-blob-must-survive"
|
||||
assert wsr["encrypted_index"] == "encrypted-idx-must-survive"
|
||||
|
||||
def test_all_foreign_blocks_fall_through_to_rebuild(self, provider: AnthropicProvider) -> None:
|
||||
# When every block is foreign-shaped (no Anthropic-valid block
|
||||
# survives the per-block filter), the converter still falls
|
||||
# through to text+tool_calls rebuild rather than emitting an
|
||||
# empty assistant turn.
|
||||
msg = {
|
||||
"role": "assistant",
|
||||
"content": "Final answer.",
|
||||
"_provider_content": [
|
||||
{"type": "reasoning", "summary": []},
|
||||
{"type": "reasoning_text", "text": "synthetic"}, # path-3 shape
|
||||
],
|
||||
}
|
||||
_, converted = provider._convert_messages([msg], replay_reasoning_to_model=True)
|
||||
assistant = next(m for m in converted if m["role"] == "assistant")
|
||||
# Rebuild path: msg.content lifted into a single text block.
|
||||
assert assistant["content"] == [{"type": "text", "text": "Final answer."}]
|
||||
|
||||
def test_empty_provider_content_falls_through(self, provider: AnthropicProvider) -> None:
|
||||
msg = {
|
||||
"role": "assistant",
|
||||
"content": "Plain text answer.",
|
||||
"_provider_content": [],
|
||||
}
|
||||
_, converted = provider._convert_messages([msg])
|
||||
assistant = next(m for m in converted if m["role"] == "assistant")
|
||||
# Falls through to text rebuild
|
||||
assert assistant["content"] == [{"type": "text", "text": "Plain text answer."}]
|
||||
|
||||
def test_none_provider_content_falls_through(self, provider: AnthropicProvider) -> None:
|
||||
msg = {
|
||||
"role": "assistant",
|
||||
"content": "Plain text answer.",
|
||||
"_provider_content": None,
|
||||
}
|
||||
_, converted = provider._convert_messages([msg])
|
||||
assistant = next(m for m in converted if m["role"] == "assistant")
|
||||
assert assistant["content"] == [{"type": "text", "text": "Plain text answer."}]
|
||||
|
||||
def test_provider_content_not_a_list_falls_through(self, provider: AnthropicProvider) -> None:
|
||||
# Defensive against a corrupted provider_data deserialization.
|
||||
msg = {
|
||||
"role": "assistant",
|
||||
"content": "Plain.",
|
||||
"_provider_content": "not a list",
|
||||
}
|
||||
_, converted = provider._convert_messages([msg])
|
||||
assistant = next(m for m in converted if m["role"] == "assistant")
|
||||
assert assistant["content"] == [{"type": "text", "text": "Plain."}]
|
||||
|
||||
def test_non_dict_and_missing_type_blocks_are_dropped(
|
||||
self, provider: AnthropicProvider
|
||||
) -> None:
|
||||
# Defensive branches in the per-block walk: a stray non-dict
|
||||
# element (corrupted JSON) or a dict with no/None ``type`` key
|
||||
# (provider drift) must be silently dropped without raising.
|
||||
# Valid blocks in the same list still ride the verbatim path.
|
||||
msg = {
|
||||
"role": "assistant",
|
||||
"content": "ok",
|
||||
"_provider_content": [
|
||||
{"type": "text", "text": "ok"},
|
||||
"stray-string", # non-dict
|
||||
{"type": None, "text": "huh"}, # None type
|
||||
{"no_type_key": 1}, # missing type
|
||||
{"type": "thinking", "thinking": "t", "signature": "s"},
|
||||
],
|
||||
}
|
||||
_, converted = provider._convert_messages([msg], replay_reasoning_to_model=True)
|
||||
assistant = next(m for m in converted if m["role"] == "assistant")
|
||||
types_present = [b.get("type") for b in assistant["content"]]
|
||||
assert types_present == ["text", "thinking"]
|
||||
|
||||
|
||||
class TestLegacyAnthropicRowsNoRegression:
|
||||
"""Critical property: rows persisted before Phase 2 carry valid
|
||||
Anthropic-shape _provider_content (only Anthropic captured this lane
|
||||
historically). They must stay in the verbatim path and keep their
|
||||
thinking context across the migration boundary when replay=True
|
||||
(the legacy default).
|
||||
"""
|
||||
|
||||
def test_legacy_thinking_row_preserved_with_default_kwarg(
|
||||
self, provider: AnthropicProvider
|
||||
) -> None:
|
||||
"""No kwarg passed (matches the pre-Phase-2 production call site)."""
|
||||
msg = {
|
||||
"role": "assistant",
|
||||
"content": "Old answer from months ago.",
|
||||
"_provider_content": [
|
||||
{"type": "thinking", "thinking": "old reasoning", "signature": "s"},
|
||||
{"type": "text", "text": "Old answer from months ago."},
|
||||
],
|
||||
}
|
||||
_, converted = provider._convert_messages([msg])
|
||||
assistant = next(m for m in converted if m["role"] == "assistant")
|
||||
types_present = [b["type"] for b in assistant["content"]]
|
||||
assert "thinking" in types_present # preserved -> no regression
|
||||
# The thinking block IS the same dict as the source (verbatim path).
|
||||
assert assistant["content"][0]["thinking"] == "old reasoning"
|
||||
|
||||
def test_replay_false_only_strips_when_explicitly_requested(
|
||||
self, provider: AnthropicProvider
|
||||
) -> None:
|
||||
# Operator flips persist+replay flags off. Strip fires.
|
||||
# Pinning that the strip is gated on the explicit flag value,
|
||||
# not silently triggered by some other condition.
|
||||
msg = {
|
||||
"role": "assistant",
|
||||
"content": "Answer.",
|
||||
"_provider_content": [
|
||||
{"type": "thinking", "thinking": "stripped", "signature": "s"},
|
||||
{"type": "text", "text": "Answer."},
|
||||
],
|
||||
}
|
||||
_, converted_default = provider._convert_messages([msg])
|
||||
_, converted_strip = provider._convert_messages([msg], replay_reasoning_to_model=False)
|
||||
default_types = [b["type"] for b in converted_default[0]["content"]]
|
||||
strip_types = [b["type"] for b in converted_strip[0]["content"]]
|
||||
assert "thinking" in default_types
|
||||
assert "thinking" not in strip_types
|
||||
|
||||
|
||||
class TestStripAllBlocksFallthrough:
|
||||
"""When the message is 100% thinking (no text, no tool_use) and
|
||||
replay=False strips everything, the message falls through to the
|
||||
text+tool_calls rebuild path. If both are also empty, the assistant
|
||||
turn is silently skipped — correct: stripped reasoning has nothing
|
||||
to replay."""
|
||||
|
||||
def test_only_thinking_strip_falls_to_rebuild_with_text(
|
||||
self, provider: AnthropicProvider
|
||||
) -> None:
|
||||
# Provider_content = only thinking; msg.content has the spoken text.
|
||||
# Strip drops thinking; rebuild path picks up the content as a
|
||||
# text block. No information lost.
|
||||
msg = {
|
||||
"role": "assistant",
|
||||
"content": "Spoken answer.",
|
||||
"_provider_content": [
|
||||
{"type": "thinking", "thinking": "internal", "signature": "s"},
|
||||
],
|
||||
}
|
||||
_, converted = provider._convert_messages([msg], replay_reasoning_to_model=False)
|
||||
assistant = next(m for m in converted if m["role"] == "assistant")
|
||||
assert assistant["content"] == [{"type": "text", "text": "Spoken answer."}]
|
||||
|
||||
def test_only_thinking_strip_with_no_content_skips_message(
|
||||
self, provider: AnthropicProvider
|
||||
) -> None:
|
||||
# Edge: provider_content was 100% thinking AND msg.content is
|
||||
# empty AND no tool_calls. The rebuild path sees nothing to
|
||||
# emit — assistant turn silently skipped. Anthropic's API
|
||||
# would reject an empty assistant content array anyway.
|
||||
msg = {
|
||||
"role": "assistant",
|
||||
"content": "",
|
||||
"_provider_content": [
|
||||
{"type": "thinking", "thinking": "only", "signature": "s"},
|
||||
],
|
||||
}
|
||||
_, converted = provider._convert_messages([msg], replay_reasoning_to_model=False)
|
||||
# Assistant turn skipped — no entry for it in `converted`.
|
||||
assert all(m["role"] != "assistant" for m in converted)
|
||||
|
||||
|
||||
class TestConstants:
|
||||
"""Pin the constant contents so a future edit doesn't accidentally
|
||||
widen the strip set or narrow the valid set."""
|
||||
|
||||
def test_reasoning_block_types_is_narrow(self) -> None:
|
||||
# Strip predicate MUST cover only reasoning shapes. Adding
|
||||
# tool_use here would break web-search round-trip.
|
||||
assert frozenset({"thinking", "redacted_thinking"}) == ANTHROPIC_REASONING_BLOCK_TYPES
|
||||
|
||||
def test_valid_block_types_includes_web_search(self) -> None:
|
||||
# Without server_tool_use / web_search_tool_result, Anthropic
|
||||
# web-search results would fall through to the rebuild path
|
||||
# and lose their encrypted_content.
|
||||
assert "server_tool_use" in ANTHROPIC_VALID_BLOCK_TYPES
|
||||
assert "web_search_tool_result" in ANTHROPIC_VALID_BLOCK_TYPES
|
||||
assert "tool_use" in ANTHROPIC_VALID_BLOCK_TYPES
|
||||
assert "tool_result" in ANTHROPIC_VALID_BLOCK_TYPES
|
||||
|
||||
def test_reasoning_subset_of_valid(self) -> None:
|
||||
# The strip set must be a subset of the valid set — otherwise
|
||||
# the strip predicate would never match anything (we only
|
||||
# strip after shape validity passes).
|
||||
assert ANTHROPIC_REASONING_BLOCK_TYPES.issubset(ANTHROPIC_VALID_BLOCK_TYPES)
|
||||
@@ -0,0 +1,320 @@
|
||||
"""Tests for OpenAI Responses reasoning capture + replay (Phase 3 path 2).
|
||||
|
||||
Phase 3 wires:
|
||||
1. ``include=["reasoning.encrypted_content"]`` on the request when
|
||||
the operator flag AND the model capability both allow.
|
||||
2. ``_convert_messages`` round-tripping stored reasoning items as
|
||||
``ResponseReasoningItemParam`` input items on subsequent turns.
|
||||
3. ``OpenAIResponsesProvider.extract_reasoning_text`` walking
|
||||
reasoning items and returning concatenated summary + content text.
|
||||
|
||||
All tests drive through the real ``OpenAIResponsesProvider`` — no
|
||||
mocks of the converter/build_kwargs themselves; only the SDK boundary
|
||||
is mocked where relevant.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import pytest
|
||||
|
||||
from turnstone.core.providers._openai_responses import (
|
||||
OpenAIResponsesProvider,
|
||||
_reasoning_item_for_input,
|
||||
)
|
||||
from turnstone.core.providers._protocol import (
|
||||
MAX_REASONING_DISPLAY_CHARS as _MAX_REASONING_DISPLAY_CHARS,
|
||||
)
|
||||
from turnstone.core.providers._protocol import ModelCapabilities
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def provider() -> OpenAIResponsesProvider:
|
||||
return OpenAIResponsesProvider()
|
||||
|
||||
|
||||
def _capable_caps() -> ModelCapabilities:
|
||||
"""Capability fixture for a reasoning-replay-capable model."""
|
||||
return ModelCapabilities(
|
||||
context_window=400000,
|
||||
max_output_tokens=128000,
|
||||
supports_temperature=False,
|
||||
reasoning_effort_values=("low", "medium", "high"),
|
||||
default_reasoning_effort="medium",
|
||||
supports_reasoning_replay=True,
|
||||
)
|
||||
|
||||
|
||||
class TestExtractReasoningText:
|
||||
def test_none_returns_empty(self, provider: OpenAIResponsesProvider) -> None:
|
||||
assert provider.extract_reasoning_text(None) == ""
|
||||
|
||||
def test_empty_list_returns_empty(self, provider: OpenAIResponsesProvider) -> None:
|
||||
assert provider.extract_reasoning_text([]) == ""
|
||||
|
||||
def test_no_reasoning_items_returns_empty(self, provider: OpenAIResponsesProvider) -> None:
|
||||
blocks = [
|
||||
{"type": "message", "role": "assistant", "content": "hi"},
|
||||
{"type": "function_call", "call_id": "c1", "name": "x", "arguments": "{}"},
|
||||
]
|
||||
assert provider.extract_reasoning_text(blocks) == ""
|
||||
|
||||
def test_summary_text_extracted(self, provider: OpenAIResponsesProvider) -> None:
|
||||
# Per ResponseReasoningItem (response_reasoning_item.py:31-62):
|
||||
# summary is always present; content is optional.
|
||||
blocks = [
|
||||
{
|
||||
"type": "reasoning",
|
||||
"id": "r_1",
|
||||
"summary": [
|
||||
{"type": "summary_text", "text": "I considered X"},
|
||||
{"type": "summary_text", "text": "then Y"},
|
||||
],
|
||||
}
|
||||
]
|
||||
assert provider.extract_reasoning_text(blocks) == "I considered X\nthen Y"
|
||||
|
||||
def test_content_text_extracted_alongside_summary(
|
||||
self, provider: OpenAIResponsesProvider
|
||||
) -> None:
|
||||
blocks = [
|
||||
{
|
||||
"type": "reasoning",
|
||||
"id": "r_1",
|
||||
"summary": [{"type": "summary_text", "text": "summary line"}],
|
||||
"content": [{"type": "reasoning_text", "text": "raw reasoning"}],
|
||||
}
|
||||
]
|
||||
# Order: summary first, then content (matches the order the SDK
|
||||
# surfaces them via streaming events).
|
||||
result = provider.extract_reasoning_text(blocks)
|
||||
assert "summary line" in result
|
||||
assert "raw reasoning" in result
|
||||
|
||||
def test_truncation_at_64kib_cap(self, provider: OpenAIResponsesProvider) -> None:
|
||||
long_text = "x" * (_MAX_REASONING_DISPLAY_CHARS + 1024)
|
||||
blocks = [
|
||||
{
|
||||
"type": "reasoning",
|
||||
"id": "r_1",
|
||||
"summary": [{"type": "summary_text", "text": long_text}],
|
||||
}
|
||||
]
|
||||
result = provider.extract_reasoning_text(blocks)
|
||||
assert len(result) == _MAX_REASONING_DISPLAY_CHARS
|
||||
|
||||
def test_malformed_summary_entry_skipped(self, provider: OpenAIResponsesProvider) -> None:
|
||||
blocks = [
|
||||
{
|
||||
"type": "reasoning",
|
||||
"id": "r_1",
|
||||
"summary": [
|
||||
"not a dict",
|
||||
{"type": "summary_text"}, # missing text
|
||||
{"type": "summary_text", "text": ""}, # empty text
|
||||
{"type": "summary_text", "text": "good"},
|
||||
],
|
||||
}
|
||||
]
|
||||
assert provider.extract_reasoning_text(blocks) == "good"
|
||||
|
||||
def test_non_list_input_returns_empty(self, provider: OpenAIResponsesProvider) -> None:
|
||||
assert provider.extract_reasoning_text("not a list") == "" # type: ignore[arg-type]
|
||||
|
||||
def test_other_block_types_skipped_in_walk(self, provider: OpenAIResponsesProvider) -> None:
|
||||
# Mixed payload: only the reasoning block contributes.
|
||||
blocks = [
|
||||
{"type": "message", "role": "assistant", "content": "hi"},
|
||||
{
|
||||
"type": "reasoning",
|
||||
"id": "r_1",
|
||||
"summary": [{"type": "summary_text", "text": "thought"}],
|
||||
},
|
||||
{"type": "function_call", "call_id": "c1", "name": "x", "arguments": "{}"},
|
||||
]
|
||||
assert provider.extract_reasoning_text(blocks) == "thought"
|
||||
|
||||
|
||||
class TestReasoningItemForInput:
|
||||
"""``_reasoning_item_for_input`` projects a stored ``ResponseReasoningItem``
|
||||
dict into ``ResponseReasoningItemParam`` shape (drops server-only
|
||||
``status``)."""
|
||||
|
||||
def test_minimal_item_round_trip(self) -> None:
|
||||
stored = {
|
||||
"type": "reasoning",
|
||||
"id": "r_1",
|
||||
"summary": [{"type": "summary_text", "text": "x"}],
|
||||
"status": "completed",
|
||||
}
|
||||
result = _reasoning_item_for_input(stored)
|
||||
assert result["type"] == "reasoning"
|
||||
assert result["id"] == "r_1"
|
||||
assert result["summary"] == [{"type": "summary_text", "text": "x"}]
|
||||
# status NOT round-tripped (server-only field per
|
||||
# ResponseReasoningItemParam at response_reasoning_item_param.py).
|
||||
assert "status" not in result
|
||||
|
||||
def test_encrypted_content_round_trips_when_present(self) -> None:
|
||||
stored = {
|
||||
"type": "reasoning",
|
||||
"id": "r_1",
|
||||
"summary": [{"type": "summary_text", "text": "x"}],
|
||||
"encrypted_content": "opaque-blob",
|
||||
}
|
||||
result = _reasoning_item_for_input(stored)
|
||||
assert result["encrypted_content"] == "opaque-blob"
|
||||
|
||||
def test_encrypted_content_omitted_when_absent(self) -> None:
|
||||
stored = {
|
||||
"type": "reasoning",
|
||||
"id": "r_1",
|
||||
"summary": [{"type": "summary_text", "text": "x"}],
|
||||
}
|
||||
result = _reasoning_item_for_input(stored)
|
||||
assert "encrypted_content" not in result
|
||||
|
||||
def test_content_round_trips_when_present(self) -> None:
|
||||
stored = {
|
||||
"type": "reasoning",
|
||||
"id": "r_1",
|
||||
"summary": [{"type": "summary_text", "text": "s"}],
|
||||
"content": [{"type": "reasoning_text", "text": "raw"}],
|
||||
}
|
||||
result = _reasoning_item_for_input(stored)
|
||||
assert result["content"] == [{"type": "reasoning_text", "text": "raw"}]
|
||||
|
||||
|
||||
class TestBuildKwargsInclude:
|
||||
"""``_build_kwargs`` adds ``include=["reasoning.encrypted_content"]``
|
||||
when the resolved operator flag is True. The capability AND-gate
|
||||
lives upstream in ``ChatSession._resolve_replay_reasoning_to_model``
|
||||
(single source of truth across providers); the provider trusts the
|
||||
bool it receives. See
|
||||
``test_session_replay_reasoning.py::TestSessionToOpenAIResponsesBoundaryIntegration``
|
||||
for the end-to-end gate test."""
|
||||
|
||||
def test_include_added_when_flag_true(self, provider: OpenAIResponsesProvider) -> None:
|
||||
kwargs = provider._build_kwargs(
|
||||
model="gpt-5",
|
||||
messages=[{"role": "user", "content": "hi"}],
|
||||
tools=None,
|
||||
max_tokens=1024,
|
||||
temperature=0.5,
|
||||
reasoning_effort="medium",
|
||||
deferred_names=None,
|
||||
capabilities=_capable_caps(),
|
||||
replay_reasoning_to_model=True,
|
||||
)
|
||||
assert kwargs.get("include") == ["reasoning.encrypted_content"]
|
||||
|
||||
def test_include_omitted_when_flag_false(self, provider: OpenAIResponsesProvider) -> None:
|
||||
kwargs = provider._build_kwargs(
|
||||
model="gpt-5",
|
||||
messages=[{"role": "user", "content": "hi"}],
|
||||
tools=None,
|
||||
max_tokens=1024,
|
||||
temperature=0.5,
|
||||
reasoning_effort="medium",
|
||||
deferred_names=None,
|
||||
capabilities=_capable_caps(),
|
||||
replay_reasoning_to_model=False,
|
||||
)
|
||||
assert "include" not in kwargs
|
||||
|
||||
|
||||
class TestConvertMessagesReasoningReplay:
|
||||
"""``_convert_messages`` round-trips stored reasoning items as input."""
|
||||
|
||||
def test_reasoning_item_emitted_before_assistant_when_replay_true(
|
||||
self, provider: OpenAIResponsesProvider
|
||||
) -> None:
|
||||
messages = [
|
||||
{"role": "user", "content": "explain"},
|
||||
{
|
||||
"role": "assistant",
|
||||
"content": "Final answer.",
|
||||
"_provider_content": [
|
||||
{
|
||||
"type": "reasoning",
|
||||
"id": "r_1",
|
||||
"summary": [{"type": "summary_text", "text": "I thought"}],
|
||||
"encrypted_content": "abc",
|
||||
}
|
||||
],
|
||||
},
|
||||
{"role": "user", "content": "follow up"},
|
||||
]
|
||||
_, items = provider._convert_messages(messages, replay_reasoning_to_model=True)
|
||||
# Find the reasoning input item.
|
||||
types = [it.get("type") for it in items]
|
||||
# Expected: user, reasoning, message (assistant), user.
|
||||
assert types == ["message", "reasoning", "message", "message"]
|
||||
reasoning_idx = types.index("reasoning")
|
||||
r_item = items[reasoning_idx]
|
||||
assert r_item["id"] == "r_1"
|
||||
assert r_item["encrypted_content"] == "abc"
|
||||
# And the reasoning item appears immediately BEFORE the
|
||||
# assistant message it belongs to.
|
||||
assert items[reasoning_idx + 1]["role"] == "assistant"
|
||||
|
||||
def test_reasoning_item_dropped_when_replay_false(
|
||||
self, provider: OpenAIResponsesProvider
|
||||
) -> None:
|
||||
messages = [
|
||||
{
|
||||
"role": "assistant",
|
||||
"content": "Answer.",
|
||||
"_provider_content": [
|
||||
{
|
||||
"type": "reasoning",
|
||||
"id": "r_1",
|
||||
"summary": [{"type": "summary_text", "text": "thought"}],
|
||||
}
|
||||
],
|
||||
},
|
||||
]
|
||||
_, items = provider._convert_messages(messages, replay_reasoning_to_model=False)
|
||||
types = [it.get("type") for it in items]
|
||||
assert "reasoning" not in types
|
||||
|
||||
def test_no_reasoning_items_when_provider_content_lacks_reasoning(
|
||||
self, provider: OpenAIResponsesProvider
|
||||
) -> None:
|
||||
# Anthropic-shaped _provider_content reaching OpenAI Responses
|
||||
# (cross-provider — operator switch from Anthropic to GPT-5):
|
||||
# no type=="reasoning" items, so nothing emitted.
|
||||
messages = [
|
||||
{
|
||||
"role": "assistant",
|
||||
"content": "x",
|
||||
"_provider_content": [
|
||||
{"type": "thinking", "thinking": "anth", "signature": "s"},
|
||||
],
|
||||
},
|
||||
]
|
||||
_, items = provider._convert_messages(messages, replay_reasoning_to_model=True)
|
||||
types = [it.get("type") for it in items]
|
||||
assert "reasoning" not in types
|
||||
|
||||
def test_default_replay_reasoning_false_omits_reasoning(
|
||||
self, provider: OpenAIResponsesProvider
|
||||
) -> None:
|
||||
# Pre-Phase-3 callers (no kwarg) get the back-compat behaviour:
|
||||
# reasoning items are silently dropped (sanitize_messages was
|
||||
# already stripping _provider_content anyway).
|
||||
messages = [
|
||||
{
|
||||
"role": "assistant",
|
||||
"content": "x",
|
||||
"_provider_content": [
|
||||
{
|
||||
"type": "reasoning",
|
||||
"id": "r_1",
|
||||
"summary": [{"type": "summary_text", "text": "x"}],
|
||||
}
|
||||
],
|
||||
},
|
||||
]
|
||||
_, items = provider._convert_messages(messages) # no kwarg
|
||||
types = [it.get("type") for it in items]
|
||||
assert "reasoning" not in types
|
||||
@@ -0,0 +1,333 @@
|
||||
"""Audit-log discipline test for reasoning text.
|
||||
|
||||
Phase 1 of optional reasoning persistence surfaces stored thinking
|
||||
blocks on the ``/history`` payload (UI rehydration). The bytes ride
|
||||
through the helper (``extract_reasoning_for_history``), through the
|
||||
provider extractor (``AnthropicProvider.extract_reasoning_text``), and
|
||||
through the server build path (``_build_history``).
|
||||
|
||||
This test pins the security-sensitive contract:
|
||||
|
||||
Reasoning text MAY land on ``msg["reasoning"]`` (UI-bound),
|
||||
but MUST NOT appear in any ``Logger.info`` / ``warning`` /
|
||||
``error`` payload at any layer in the pipeline.
|
||||
|
||||
The test mocks the standard-library ``logging.Logger`` info/warning/
|
||||
error methods, runs a thinking-bearing turn through the relevant
|
||||
extractors and history build, then asserts no captured log call's
|
||||
positional args or kwargs contain the unique marker string. Replaces
|
||||
the v4 grep-the-output approach (fragile when log strings are
|
||||
formatted) with a structural mock-and-assert (tests the actual
|
||||
contract rather than the rendered text).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
from types import SimpleNamespace
|
||||
from typing import Any
|
||||
from unittest.mock import patch
|
||||
|
||||
from tests._session_helpers import make_session
|
||||
from turnstone.core.history_decoration import (
|
||||
extract_reasoning_for_history,
|
||||
extract_reasoning_text_from_provider_content,
|
||||
)
|
||||
from turnstone.core.providers._anthropic import AnthropicProvider
|
||||
from turnstone.core.providers._openai_chat import OpenAIChatCompletionsProvider
|
||||
from turnstone.core.providers._openai_responses import OpenAIResponsesProvider
|
||||
from turnstone.core.providers._protocol import StreamChunk, UsageInfo
|
||||
from turnstone.server import _build_history
|
||||
|
||||
_MARKER = "SECRET_REASONING_MARKER_xyz123_unlikely_collision"
|
||||
|
||||
|
||||
def _payload_contains_marker(args: tuple[Any, ...], kwargs: dict[str, Any]) -> bool:
|
||||
"""Walk a captured log call's args + kwargs for the marker string.
|
||||
|
||||
Logger.info-style calls accept a format string + positional substitution
|
||||
args; the marker could appear in either the format string itself or
|
||||
the substitution values. Format-time strings (``%`` substitution) are
|
||||
NOT inspected because they're a stdlib formatting concern, not a
|
||||
callable our pipeline reaches into. The structural check is "no
|
||||
user-controlled marker appears in any arg slot we passed".
|
||||
"""
|
||||
for a in args:
|
||||
if isinstance(a, str) and _MARKER in a:
|
||||
return True
|
||||
# Defensive — a list/dict/exception arg might carry the marker too.
|
||||
try:
|
||||
if _MARKER in repr(a):
|
||||
return True
|
||||
except Exception:
|
||||
continue
|
||||
for v in kwargs.values():
|
||||
if isinstance(v, str) and _MARKER in v:
|
||||
return True
|
||||
try:
|
||||
if _MARKER in repr(v):
|
||||
return True
|
||||
except Exception:
|
||||
continue
|
||||
return False
|
||||
|
||||
|
||||
def _capture_log_calls():
|
||||
"""Capture every Logger.info / warning / error call into a single list."""
|
||||
captured: list[tuple[str, tuple[Any, ...], dict[str, Any]]] = []
|
||||
|
||||
def make_recorder(level: str):
|
||||
def _rec(*args: Any, **kwargs: Any) -> None:
|
||||
captured.append((level, args, kwargs))
|
||||
|
||||
return _rec
|
||||
|
||||
return captured, [
|
||||
patch.object(logging.Logger, "info", side_effect=make_recorder("info"), autospec=True),
|
||||
patch.object(
|
||||
logging.Logger, "warning", side_effect=make_recorder("warning"), autospec=True
|
||||
),
|
||||
patch.object(logging.Logger, "error", side_effect=make_recorder("error"), autospec=True),
|
||||
]
|
||||
|
||||
|
||||
class TestReasoningAuditLogDiscipline:
|
||||
"""Reasoning text never lands at INFO+ severity on any logger."""
|
||||
|
||||
def _thinking_msg(self, text: str = _MARKER) -> dict[str, Any]:
|
||||
return {
|
||||
"role": "assistant",
|
||||
"content": "Final answer.",
|
||||
"_provider_content": [
|
||||
{"type": "thinking", "thinking": text, "signature": "sig"},
|
||||
{"type": "text", "text": "Final answer."},
|
||||
],
|
||||
}
|
||||
|
||||
def test_anthropic_extractor_does_not_log_reasoning(self) -> None:
|
||||
captured, patchers = _capture_log_calls()
|
||||
for p in patchers:
|
||||
p.start()
|
||||
try:
|
||||
provider = AnthropicProvider()
|
||||
text = provider.extract_reasoning_text(
|
||||
[{"type": "thinking", "thinking": _MARKER, "signature": "s"}]
|
||||
)
|
||||
assert text == _MARKER # extractor IS allowed to return it
|
||||
finally:
|
||||
for p in patchers:
|
||||
p.stop()
|
||||
offending = [
|
||||
(lvl, args, kwargs)
|
||||
for lvl, args, kwargs in captured
|
||||
if _payload_contains_marker(args, kwargs)
|
||||
]
|
||||
assert offending == [], (
|
||||
f"AnthropicProvider.extract_reasoning_text leaked reasoning text "
|
||||
f"into INFO+ logs: {offending}"
|
||||
)
|
||||
|
||||
def test_dispatch_helper_does_not_log_reasoning(self) -> None:
|
||||
captured, patchers = _capture_log_calls()
|
||||
for p in patchers:
|
||||
p.start()
|
||||
try:
|
||||
text = extract_reasoning_text_from_provider_content(
|
||||
[{"type": "thinking", "thinking": _MARKER, "signature": "s"}]
|
||||
)
|
||||
assert text == _MARKER
|
||||
finally:
|
||||
for p in patchers:
|
||||
p.stop()
|
||||
offending = [
|
||||
(lvl, args, kwargs)
|
||||
for lvl, args, kwargs in captured
|
||||
if _payload_contains_marker(args, kwargs)
|
||||
]
|
||||
assert offending == [], (
|
||||
f"extract_reasoning_text_from_provider_content leaked reasoning "
|
||||
f"text into INFO+ logs: {offending}"
|
||||
)
|
||||
|
||||
def test_list_helper_does_not_log_reasoning(self) -> None:
|
||||
captured, patchers = _capture_log_calls()
|
||||
for p in patchers:
|
||||
p.start()
|
||||
try:
|
||||
messages = [self._thinking_msg(_MARKER)]
|
||||
extract_reasoning_for_history(messages, surface_persisted_reasoning_flag=True)
|
||||
assert messages[0]["reasoning"] == _MARKER # UI-bound is allowed
|
||||
finally:
|
||||
for p in patchers:
|
||||
p.stop()
|
||||
offending = [
|
||||
(lvl, args, kwargs)
|
||||
for lvl, args, kwargs in captured
|
||||
if _payload_contains_marker(args, kwargs)
|
||||
]
|
||||
assert offending == [], (
|
||||
f"extract_reasoning_for_history leaked reasoning text into INFO+ logs: {offending}"
|
||||
)
|
||||
|
||||
def test_build_history_does_not_log_reasoning(self) -> None:
|
||||
registry = SimpleNamespace(
|
||||
get_config=lambda alias: SimpleNamespace(surface_persisted_reasoning=True)
|
||||
)
|
||||
session = SimpleNamespace(
|
||||
messages=[self._thinking_msg(_MARKER)],
|
||||
_ws_id="ws-audit",
|
||||
_registry=registry,
|
||||
_model_alias="claude-opus-4-7",
|
||||
)
|
||||
captured, patchers = _capture_log_calls()
|
||||
for p in patchers:
|
||||
p.start()
|
||||
try:
|
||||
with patch(
|
||||
"turnstone.server._load_verdict_indexes",
|
||||
return_value=({}, {}),
|
||||
):
|
||||
history = _build_history(session)
|
||||
assert history[0]["reasoning"] == _MARKER # UI-bound is allowed
|
||||
finally:
|
||||
for p in patchers:
|
||||
p.stop()
|
||||
offending = [
|
||||
(lvl, args, kwargs)
|
||||
for lvl, args, kwargs in captured
|
||||
if _payload_contains_marker(args, kwargs)
|
||||
]
|
||||
assert offending == [], f"_build_history leaked reasoning text into INFO+ logs: {offending}"
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Phase 2 + Phase 3 surfaces — added in response to a code-review
|
||||
# finding that the original 4-test coverage missed every code path
|
||||
# introduced after Phase 1. Each new test mirrors the structure
|
||||
# above: capture every Logger.info / warning / error call across
|
||||
# the operation, assert the marker doesn't appear in any captured
|
||||
# payload (UI-bound returns IS allowed; logging at INFO+ is NOT).
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def test_openai_responses_extractor_does_not_log_reasoning(self) -> None:
|
||||
captured, patchers = _capture_log_calls()
|
||||
for p in patchers:
|
||||
p.start()
|
||||
try:
|
||||
provider = OpenAIResponsesProvider()
|
||||
blocks = [
|
||||
{
|
||||
"type": "reasoning",
|
||||
"id": "r_1",
|
||||
"summary": [{"type": "summary_text", "text": _MARKER}],
|
||||
}
|
||||
]
|
||||
text = provider.extract_reasoning_text(blocks)
|
||||
assert _MARKER in text # UI-bound return is allowed
|
||||
finally:
|
||||
for p in patchers:
|
||||
p.stop()
|
||||
offending = [
|
||||
(lvl, args, kwargs)
|
||||
for lvl, args, kwargs in captured
|
||||
if _payload_contains_marker(args, kwargs)
|
||||
]
|
||||
assert offending == [], (
|
||||
f"OpenAIResponsesProvider.extract_reasoning_text leaked reasoning "
|
||||
f"text into INFO+ logs: {offending}"
|
||||
)
|
||||
|
||||
def test_openai_chat_extractor_does_not_log_reasoning(self) -> None:
|
||||
captured, patchers = _capture_log_calls()
|
||||
for p in patchers:
|
||||
p.start()
|
||||
try:
|
||||
provider = OpenAIChatCompletionsProvider()
|
||||
blocks = [{"type": "reasoning_text", "text": _MARKER, "source": "vllm"}]
|
||||
text = provider.extract_reasoning_text(blocks)
|
||||
assert text == _MARKER
|
||||
finally:
|
||||
for p in patchers:
|
||||
p.stop()
|
||||
offending = [
|
||||
(lvl, args, kwargs)
|
||||
for lvl, args, kwargs in captured
|
||||
if _payload_contains_marker(args, kwargs)
|
||||
]
|
||||
assert offending == [], (
|
||||
f"OpenAIChatCompletionsProvider.extract_reasoning_text leaked "
|
||||
f"reasoning text into INFO+ logs: {offending}"
|
||||
)
|
||||
|
||||
def test_synth_reasoning_block_via_stream_response_does_not_log_reasoning(
|
||||
self,
|
||||
) -> None:
|
||||
"""Drives ChatSession._stream_response (which calls
|
||||
_maybe_synth_reasoning_block at end-of-stream) with a fake
|
||||
``reasoning_delta=_MARKER`` chunk; asserts no log call carried
|
||||
the marker text."""
|
||||
session = make_session()
|
||||
chunks = [
|
||||
StreamChunk(reasoning_delta=_MARKER, is_first=True),
|
||||
StreamChunk(content_delta="answer"),
|
||||
StreamChunk(
|
||||
finish_reason="stop",
|
||||
usage=UsageInfo(prompt_tokens=10, completion_tokens=20, total_tokens=30),
|
||||
),
|
||||
]
|
||||
captured, patchers = _capture_log_calls()
|
||||
for p in patchers:
|
||||
p.start()
|
||||
try:
|
||||
msg = session._stream_response(iter(chunks))
|
||||
# Synth block stamped onto _provider_content with the marker.
|
||||
assert msg["_provider_content"][0]["text"] == _MARKER
|
||||
finally:
|
||||
for p in patchers:
|
||||
p.stop()
|
||||
offending = [
|
||||
(lvl, args, kwargs)
|
||||
for lvl, args, kwargs in captured
|
||||
if _payload_contains_marker(args, kwargs)
|
||||
]
|
||||
assert offending == [], (
|
||||
f"_stream_response + _maybe_synth_reasoning_block leaked reasoning "
|
||||
f"text into INFO+ logs: {offending}"
|
||||
)
|
||||
|
||||
def test_anthropic_convert_messages_strip_does_not_log_reasoning(self) -> None:
|
||||
"""Drives the Phase 2 strip predicate
|
||||
(``replay_reasoning_to_model=False``) which walks thinking
|
||||
blocks to filter them out before the wire payload is built;
|
||||
asserts no log call carried the marker text."""
|
||||
captured, patchers = _capture_log_calls()
|
||||
for p in patchers:
|
||||
p.start()
|
||||
try:
|
||||
provider = AnthropicProvider()
|
||||
messages = [
|
||||
{
|
||||
"role": "assistant",
|
||||
"content": "Final answer.",
|
||||
"_provider_content": [
|
||||
{"type": "thinking", "thinking": _MARKER, "signature": "s"},
|
||||
{"type": "text", "text": "Final answer."},
|
||||
],
|
||||
},
|
||||
]
|
||||
_, converted = provider._convert_messages(messages, replay_reasoning_to_model=False)
|
||||
# Strip fired — thinking block dropped from wire.
|
||||
assistant = next(m for m in converted if m["role"] == "assistant")
|
||||
block_types = [b.get("type") for b in assistant["content"]]
|
||||
assert "thinking" not in block_types
|
||||
finally:
|
||||
for p in patchers:
|
||||
p.stop()
|
||||
offending = [
|
||||
(lvl, args, kwargs)
|
||||
for lvl, args, kwargs in captured
|
||||
if _payload_contains_marker(args, kwargs)
|
||||
]
|
||||
assert offending == [], (
|
||||
f"AnthropicProvider._convert_messages strip predicate leaked "
|
||||
f"reasoning text into INFO+ logs: {offending}"
|
||||
)
|
||||
@@ -14,6 +14,12 @@ from turnstone.core.session import ChatSession
|
||||
class NullUI:
|
||||
"""UI adapter that discards all output."""
|
||||
|
||||
def on_turn_start(self):
|
||||
pass
|
||||
|
||||
def on_turn_committed(self):
|
||||
pass
|
||||
|
||||
def on_thinking_start(self):
|
||||
pass
|
||||
|
||||
|
||||
@@ -297,6 +297,49 @@ def test_extra_fields_ignored():
|
||||
assert e.text == "hi"
|
||||
|
||||
|
||||
def test_in_progress_snapshot_event_round_trip():
|
||||
from turnstone.sdk.events import InProgressSnapshotEvent
|
||||
|
||||
payload = {
|
||||
"type": "in_progress_snapshot",
|
||||
"ws_id": "ws1",
|
||||
"content": "Partial content...",
|
||||
"reasoning": "Partial reasoning...",
|
||||
}
|
||||
e = ServerEvent.from_dict(payload)
|
||||
assert isinstance(e, InProgressSnapshotEvent)
|
||||
assert e.ws_id == "ws1"
|
||||
assert e.content == "Partial content..."
|
||||
assert e.reasoning == "Partial reasoning..."
|
||||
|
||||
|
||||
def test_in_progress_snapshot_event_strips_internal_seq():
|
||||
"""``_seq`` is server-internal plumbing — even if a stray copy
|
||||
leaks through, ``from_dict`` must drop it (not a declared field)."""
|
||||
from turnstone.sdk.events import InProgressSnapshotEvent
|
||||
|
||||
e = ServerEvent.from_dict(
|
||||
{
|
||||
"type": "in_progress_snapshot",
|
||||
"ws_id": "ws1",
|
||||
"content": "x",
|
||||
"reasoning": "",
|
||||
"_seq": 42,
|
||||
}
|
||||
)
|
||||
assert isinstance(e, InProgressSnapshotEvent)
|
||||
assert not hasattr(e, "_seq")
|
||||
|
||||
|
||||
def test_state_change_event_round_trip():
|
||||
from turnstone.sdk.events import StateChangeEvent
|
||||
|
||||
e = ServerEvent.from_dict({"type": "state_change", "ws_id": "ws1", "state": "thinking"})
|
||||
assert isinstance(e, StateChangeEvent)
|
||||
assert e.state == "thinking"
|
||||
assert e.ws_id == "ws1"
|
||||
|
||||
|
||||
def test_missing_type_defaults_to_base():
|
||||
e = ServerEvent.from_dict({"ws_id": "ws1"})
|
||||
assert type(e) is ServerEvent
|
||||
|
||||
@@ -70,6 +70,12 @@ class RecordingUI:
|
||||
self.errors: list[str] = []
|
||||
self.infos: list[str] = []
|
||||
|
||||
def on_turn_start(self):
|
||||
self.events.append(("turn_start",))
|
||||
|
||||
def on_turn_committed(self):
|
||||
self.events.append(("turn_committed",))
|
||||
|
||||
def on_thinking_start(self):
|
||||
self.events.append(("thinking_start",))
|
||||
|
||||
|
||||
@@ -14,6 +14,12 @@ from turnstone.core.session import _IMAGE_EXTENSIONS, _IMAGE_SIZE_CAP, ChatSessi
|
||||
class NullUI:
|
||||
"""UI adapter that discards all output. Used for testing."""
|
||||
|
||||
def on_turn_start(self):
|
||||
pass
|
||||
|
||||
def on_turn_committed(self):
|
||||
pass
|
||||
|
||||
def on_thinking_start(self):
|
||||
pass
|
||||
|
||||
|
||||
@@ -0,0 +1,648 @@
|
||||
"""Tests for session-level ``replay_reasoning_to_model`` plumbing.
|
||||
|
||||
Phase 2 of optional reasoning persistence reads the per-model
|
||||
``ModelConfig.replay_reasoning_to_model`` flag at the wire-build call
|
||||
site and threads it through ``provider.create_streaming`` /
|
||||
``provider.create_completion``. These tests pin:
|
||||
|
||||
1. The resolver helper (``ChatSession._resolve_replay_reasoning_to_model``)
|
||||
walks the registry correctly and falls back to ``False`` (the
|
||||
conservative default matching the migration server_default) when
|
||||
the lookup fails.
|
||||
2. The streaming wire-build call site at ``session.py:_try_stream``
|
||||
actually passes the resolved flag down — without this, the Phase
|
||||
2 work is dead code (the strip-when-False predicate never fires).
|
||||
3. The non-streaming wire-build call site at
|
||||
``session.py:_utility_completion`` does the same.
|
||||
|
||||
Drives through the real ``ChatSession._resolve_replay_reasoning_to_model``
|
||||
with a stub registry, then captures the kwarg passed to a mock provider
|
||||
to verify the flow end-to-end.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from types import SimpleNamespace
|
||||
from typing import Any
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
import pytest
|
||||
|
||||
from tests._session_helpers import make_session as _make_session
|
||||
|
||||
|
||||
def _registry_with_flag(persist: bool = True, replay: bool = False) -> Any:
|
||||
"""Stub registry returning a ModelConfig-shaped object with the
|
||||
flags under test."""
|
||||
return SimpleNamespace(
|
||||
get_config=lambda alias: SimpleNamespace(
|
||||
surface_persisted_reasoning=persist,
|
||||
replay_reasoning_to_model=replay,
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
class TestResolveReplayReasoningToModel:
|
||||
"""Direct unit tests for the resolver."""
|
||||
|
||||
def test_returns_false_when_no_registry(self) -> None:
|
||||
session = _make_session()
|
||||
session._registry = None
|
||||
session._model_alias = "anything"
|
||||
assert session._resolve_replay_reasoning_to_model() is False
|
||||
|
||||
def test_returns_false_when_no_alias(self) -> None:
|
||||
session = _make_session()
|
||||
session._registry = _registry_with_flag(replay=True)
|
||||
session._model_alias = ""
|
||||
assert session._resolve_replay_reasoning_to_model() is False
|
||||
|
||||
def test_returns_false_default(self) -> None:
|
||||
session = _make_session()
|
||||
session._registry = _registry_with_flag(replay=False)
|
||||
session._model_alias = "claude-opus-4-7"
|
||||
assert session._resolve_replay_reasoning_to_model() is False
|
||||
|
||||
def test_returns_true_when_flag_set(self) -> None:
|
||||
session = _make_session()
|
||||
session._registry = _registry_with_flag(replay=True)
|
||||
session._model_alias = "claude-opus-4-7"
|
||||
assert session._resolve_replay_reasoning_to_model() is True
|
||||
|
||||
def test_explicit_alias_arg_overrides_default(self) -> None:
|
||||
session = _make_session()
|
||||
|
||||
def per_alias(alias: str) -> Any:
|
||||
return SimpleNamespace(
|
||||
replay_reasoning_to_model=(alias == "needs-replay"),
|
||||
)
|
||||
|
||||
session._registry = SimpleNamespace(get_config=per_alias)
|
||||
session._model_alias = "primary"
|
||||
# Default reads session._model_alias → False.
|
||||
assert session._resolve_replay_reasoning_to_model() is False
|
||||
# Explicit alias arg → True for "needs-replay".
|
||||
assert session._resolve_replay_reasoning_to_model("needs-replay") is True
|
||||
|
||||
def test_returns_false_on_registry_exception(self) -> None:
|
||||
session = _make_session()
|
||||
|
||||
def boom(alias: str) -> Any:
|
||||
raise KeyError(alias)
|
||||
|
||||
session._registry = SimpleNamespace(get_config=boom)
|
||||
session._model_alias = "missing"
|
||||
# Conservative fallback — losing the strip is a UX nuisance,
|
||||
# but accepting wire-side reasoning replay against an unknown
|
||||
# operator preference is a worse default.
|
||||
assert session._resolve_replay_reasoning_to_model() is False
|
||||
|
||||
def test_caps_none_preserves_back_compat(self) -> None:
|
||||
# When ``caps`` is omitted, the resolver returns the operator
|
||||
# flag unchanged — matching pre-PR behaviour for any caller
|
||||
# that hasn't been updated to thread caps yet.
|
||||
session = _make_session()
|
||||
session._registry = _registry_with_flag(replay=True)
|
||||
session._model_alias = "claude-opus-4-7"
|
||||
assert session._resolve_replay_reasoning_to_model() is True
|
||||
assert session._resolve_replay_reasoning_to_model(caps=None) is True
|
||||
|
||||
def test_caps_supports_replay_true_passes_through(self) -> None:
|
||||
from turnstone.core.providers._protocol import ModelCapabilities
|
||||
|
||||
session = _make_session()
|
||||
session._registry = _registry_with_flag(replay=True)
|
||||
session._model_alias = "claude-opus-4-7"
|
||||
caps = ModelCapabilities(supports_reasoning_replay=True)
|
||||
assert session._resolve_replay_reasoning_to_model(caps=caps) is True
|
||||
|
||||
def test_caps_supports_replay_false_blocks_replay(self) -> None:
|
||||
from turnstone.core.providers._protocol import ModelCapabilities
|
||||
|
||||
# Operator flipped replay=True but the model's capability
|
||||
# advertises supports_reasoning_replay=False — AND-gate blocks
|
||||
# replay so the strip predicate runs at the wire build.
|
||||
session = _make_session()
|
||||
session._registry = _registry_with_flag(replay=True)
|
||||
session._model_alias = "hypothetical-no-replay-claude"
|
||||
caps = ModelCapabilities(supports_reasoning_replay=False)
|
||||
assert session._resolve_replay_reasoning_to_model(caps=caps) is False
|
||||
|
||||
def test_caps_supports_replay_true_does_not_force_replay(self) -> None:
|
||||
from turnstone.core.providers._protocol import ModelCapabilities
|
||||
|
||||
# Capability True but operator flag False — result must be
|
||||
# False (the AND has to be False on either side).
|
||||
session = _make_session()
|
||||
session._registry = _registry_with_flag(replay=False)
|
||||
session._model_alias = "claude-opus-4-7"
|
||||
caps = ModelCapabilities(supports_reasoning_replay=True)
|
||||
assert session._resolve_replay_reasoning_to_model(caps=caps) is False
|
||||
|
||||
|
||||
class TestStreamingCallSitePassesFlag:
|
||||
"""Pin that ``_try_stream`` actually passes the resolved flag to
|
||||
``provider.create_streaming`` — without this the Phase 2 work is
|
||||
dead code at the call site."""
|
||||
|
||||
def test_replay_true_propagates_to_provider(self) -> None:
|
||||
session = _make_session()
|
||||
session._registry = _registry_with_flag(replay=True)
|
||||
session._model_alias = "claude-opus-4-7"
|
||||
# Stub provider: capture the kwargs passed to create_streaming.
|
||||
captured: dict[str, Any] = {}
|
||||
|
||||
def capture_streaming(**kwargs: Any) -> Any:
|
||||
captured.update(kwargs)
|
||||
return iter([])
|
||||
|
||||
mock_provider = MagicMock()
|
||||
mock_provider.create_streaming = capture_streaming
|
||||
with (
|
||||
patch.object(session, "_get_active_tools", return_value=None),
|
||||
patch.object(session, "_provider_extra_params", return_value=None),
|
||||
patch.object(session, "_get_deferred_names", return_value=frozenset()),
|
||||
patch.object(session, "_check_cancelled"),
|
||||
):
|
||||
session._try_stream(
|
||||
client=MagicMock(),
|
||||
model="claude-opus-4-7",
|
||||
msgs=[{"role": "user", "content": "hi"}],
|
||||
provider=mock_provider,
|
||||
model_alias="claude-opus-4-7",
|
||||
)
|
||||
assert captured["replay_reasoning_to_model"] is True
|
||||
|
||||
def test_replay_false_propagates_to_provider(self) -> None:
|
||||
session = _make_session()
|
||||
session._registry = _registry_with_flag(replay=False)
|
||||
session._model_alias = "claude-opus-4-7"
|
||||
captured: dict[str, Any] = {}
|
||||
|
||||
def capture_streaming(**kwargs: Any) -> Any:
|
||||
captured.update(kwargs)
|
||||
return iter([])
|
||||
|
||||
mock_provider = MagicMock()
|
||||
mock_provider.create_streaming = capture_streaming
|
||||
with (
|
||||
patch.object(session, "_get_active_tools", return_value=None),
|
||||
patch.object(session, "_provider_extra_params", return_value=None),
|
||||
patch.object(session, "_get_deferred_names", return_value=frozenset()),
|
||||
patch.object(session, "_check_cancelled"),
|
||||
):
|
||||
session._try_stream(
|
||||
client=MagicMock(),
|
||||
model="claude-opus-4-7",
|
||||
msgs=[{"role": "user", "content": "hi"}],
|
||||
provider=mock_provider,
|
||||
model_alias="claude-opus-4-7",
|
||||
)
|
||||
assert captured["replay_reasoning_to_model"] is False
|
||||
|
||||
def test_fallback_alias_uses_its_own_flag(self) -> None:
|
||||
# When the primary fails and we fall back to an alias with a
|
||||
# different flag, the flag MUST track the resolved alias —
|
||||
# not the session's primary alias.
|
||||
session = _make_session()
|
||||
|
||||
def per_alias(alias: str) -> Any:
|
||||
return SimpleNamespace(
|
||||
replay_reasoning_to_model=(alias == "fallback-with-replay"),
|
||||
)
|
||||
|
||||
session._registry = SimpleNamespace(get_config=per_alias)
|
||||
session._model_alias = "primary" # primary has replay=False
|
||||
captured: dict[str, Any] = {}
|
||||
|
||||
def capture_streaming(**kwargs: Any) -> Any:
|
||||
captured.update(kwargs)
|
||||
return iter([])
|
||||
|
||||
mock_provider = MagicMock()
|
||||
mock_provider.create_streaming = capture_streaming
|
||||
with (
|
||||
patch.object(session, "_get_active_tools", return_value=None),
|
||||
patch.object(session, "_provider_extra_params", return_value=None),
|
||||
patch.object(session, "_get_deferred_names", return_value=frozenset()),
|
||||
patch.object(session, "_check_cancelled"),
|
||||
):
|
||||
session._try_stream(
|
||||
client=MagicMock(),
|
||||
model="fallback-model",
|
||||
msgs=[{"role": "user", "content": "hi"}],
|
||||
provider=mock_provider,
|
||||
model_alias="fallback-with-replay",
|
||||
)
|
||||
# Resolved against the FALLBACK alias, not the session's primary.
|
||||
assert captured["replay_reasoning_to_model"] is True
|
||||
|
||||
|
||||
class TestSessionToWireBoundaryIntegration:
|
||||
"""End-to-end integration: session._try_stream -> real
|
||||
AnthropicProvider.create_streaming -> captured Anthropic SDK
|
||||
boundary call. Verifies the strip-when-False predicate actually
|
||||
fires at the wire payload, not just at the captured kwarg.
|
||||
|
||||
The bare-function-stub tests above (TestStreamingCallSitePassesFlag)
|
||||
pin that ``_try_stream`` PASSES the flag; this test pins that the
|
||||
real provider USES it. Together they catch:
|
||||
- kwarg renamed at provider boundary -> stub-tests still pass,
|
||||
this one fails on its real-provider assertion.
|
||||
- _convert_messages stops reading the kwarg -> stub-tests still
|
||||
pass, this one fails because the wire payload still carries
|
||||
the thinking block.
|
||||
- _try_stream stops calling create_streaming -> stub-tests fail
|
||||
on the captured kwarg, this one fails because the SDK boundary
|
||||
was never reached.
|
||||
|
||||
Drives through the real ``AnthropicProvider`` with a mock client
|
||||
whose ``client.messages.stream`` is captured — the smallest possible
|
||||
surface that crosses the session->provider->wire boundary chain.
|
||||
|
||||
Negative-tested: temporarily reverting
|
||||
``_anthropic.py:create_streaming``'s
|
||||
``self._convert_messages(messages, replay_reasoning_to_model=...)``
|
||||
call to drop the kwarg makes the wire payload carry the thinking
|
||||
block again; ``test_replay_false_strips_thinking_at_wire`` then
|
||||
fails with ``Strip predicate did not fire at wire boundary``.
|
||||
Restoring the kwarg makes it pass — confirming the test gates the
|
||||
actual wire-build invariant rather than the captured kwarg.
|
||||
"""
|
||||
|
||||
def _stub_anthropic_client(self) -> tuple[MagicMock, dict[str, object]]:
|
||||
"""Build a mock Anthropic client + captured-kwargs dict.
|
||||
|
||||
``client.messages.stream(**kwargs)`` returns a context manager
|
||||
whose ``__enter__`` yields an iterable of zero events — enough
|
||||
to satisfy the ``_iter_with_cleanup`` shape without exercising
|
||||
actual streaming protocol.
|
||||
"""
|
||||
captured: dict[str, object] = {}
|
||||
|
||||
def stream(**kwargs: object) -> object:
|
||||
captured.update(kwargs)
|
||||
cm = MagicMock()
|
||||
cm.__enter__ = MagicMock(return_value=iter([]))
|
||||
cm.__exit__ = MagicMock(return_value=False)
|
||||
return cm
|
||||
|
||||
client = MagicMock()
|
||||
client.messages.stream = stream
|
||||
return client, captured
|
||||
|
||||
def _drive_session_through_anthropic(
|
||||
self,
|
||||
replay_flag: bool,
|
||||
msgs: list[dict[str, object]],
|
||||
) -> dict[str, object]:
|
||||
"""Run session._try_stream against a real AnthropicProvider with
|
||||
the resolver pre-set to *replay_flag*. Returns the kwargs
|
||||
dict that reached the (mocked) Anthropic SDK boundary.
|
||||
"""
|
||||
pytest.importorskip("anthropic")
|
||||
from turnstone.core.providers._anthropic import AnthropicProvider
|
||||
|
||||
session = _make_session()
|
||||
session._registry = _registry_with_flag(replay=replay_flag)
|
||||
session._model_alias = "claude-opus-4-7"
|
||||
client, captured = self._stub_anthropic_client()
|
||||
real_provider = AnthropicProvider()
|
||||
with (
|
||||
patch.object(session, "_get_active_tools", return_value=None),
|
||||
patch.object(session, "_provider_extra_params", return_value=None),
|
||||
patch.object(session, "_get_deferred_names", return_value=frozenset()),
|
||||
patch.object(session, "_check_cancelled"),
|
||||
):
|
||||
stream = session._try_stream(
|
||||
client=client,
|
||||
model="claude-opus-4-7",
|
||||
msgs=msgs,
|
||||
provider=real_provider,
|
||||
model_alias="claude-opus-4-7",
|
||||
)
|
||||
# Iterate the stream to drain the (empty) generator and ensure
|
||||
# _ensure_anthropic / convert / build_kwargs all ran.
|
||||
list(stream)
|
||||
return captured
|
||||
|
||||
def test_replay_false_strips_thinking_at_wire(self) -> None:
|
||||
msgs: list[dict[str, object]] = [
|
||||
{"role": "user", "content": "hello"},
|
||||
{
|
||||
"role": "assistant",
|
||||
"content": "Final answer.",
|
||||
"_provider_content": [
|
||||
{"type": "thinking", "thinking": "secret reasoning", "signature": "s"},
|
||||
{"type": "text", "text": "Final answer."},
|
||||
],
|
||||
},
|
||||
{"role": "user", "content": "ack"},
|
||||
]
|
||||
captured = self._drive_session_through_anthropic(False, msgs)
|
||||
# Anthropic SDK was called.
|
||||
wire_msgs = captured.get("messages")
|
||||
assert isinstance(wire_msgs, list), (
|
||||
f"Expected messages= list at SDK boundary, got {captured}"
|
||||
)
|
||||
# Walk the wire payload — the thinking block must NOT be present
|
||||
# in the assistant turn's content blocks.
|
||||
assistant = next(m for m in wire_msgs if m["role"] == "assistant")
|
||||
block_types = [b.get("type") for b in assistant["content"] if isinstance(b, dict)]
|
||||
assert "thinking" not in block_types, (
|
||||
f"Strip predicate did not fire at wire boundary: blocks={block_types}"
|
||||
)
|
||||
# Defense-in-depth: the secret reasoning text must not appear
|
||||
# anywhere in the wire payload.
|
||||
flat = repr(captured)
|
||||
assert "secret reasoning" not in flat, "Reasoning text leaked into the SDK boundary payload"
|
||||
|
||||
def test_replay_true_preserves_thinking_at_wire(self) -> None:
|
||||
msgs: list[dict[str, object]] = [
|
||||
{"role": "user", "content": "hello"},
|
||||
{
|
||||
"role": "assistant",
|
||||
"content": "Final answer.",
|
||||
"_provider_content": [
|
||||
{"type": "thinking", "thinking": "kept reasoning", "signature": "s"},
|
||||
{"type": "text", "text": "Final answer."},
|
||||
],
|
||||
},
|
||||
{"role": "user", "content": "ack"},
|
||||
]
|
||||
captured = self._drive_session_through_anthropic(True, msgs)
|
||||
wire_msgs = captured.get("messages")
|
||||
assert isinstance(wire_msgs, list)
|
||||
assistant = next(m for m in wire_msgs if m["role"] == "assistant")
|
||||
block_types = [b.get("type") for b in assistant["content"] if isinstance(b, dict)]
|
||||
assert "thinking" in block_types, (
|
||||
f"Replay-true did not preserve thinking at wire: blocks={block_types}"
|
||||
)
|
||||
|
||||
def test_capability_false_strips_thinking_even_when_operator_flag_true(self) -> None:
|
||||
# Mirror of the OpenAI Responses ``test_capability_false_omits_
|
||||
# include_even_when_flag_true`` test below: operator flips
|
||||
# replay=True but the model's capability advertises
|
||||
# supports_reasoning_replay=False. AND-gate at the resolver
|
||||
# blocks replay, so the strip predicate fires at the wire and
|
||||
# the thinking block does NOT reach the SDK boundary.
|
||||
pytest.importorskip("anthropic")
|
||||
from turnstone.core.providers._anthropic import AnthropicProvider
|
||||
from turnstone.core.providers._protocol import ModelCapabilities
|
||||
|
||||
msgs: list[dict[str, object]] = [
|
||||
{"role": "user", "content": "hello"},
|
||||
{
|
||||
"role": "assistant",
|
||||
"content": "Final answer.",
|
||||
"_provider_content": [
|
||||
{"type": "thinking", "thinking": "secret reasoning", "signature": "s"},
|
||||
{"type": "text", "text": "Final answer."},
|
||||
],
|
||||
},
|
||||
{"role": "user", "content": "ack"},
|
||||
]
|
||||
|
||||
session = _make_session()
|
||||
session._registry = _registry_with_flag(replay=True) # operator opted in
|
||||
session._model_alias = "hypothetical-no-replay-claude"
|
||||
caps = ModelCapabilities(supports_reasoning_replay=False)
|
||||
client, captured = self._stub_anthropic_client()
|
||||
real_provider = AnthropicProvider()
|
||||
with (
|
||||
patch.object(session, "_get_active_tools", return_value=None),
|
||||
patch.object(session, "_provider_extra_params", return_value=None),
|
||||
patch.object(session, "_get_deferred_names", return_value=frozenset()),
|
||||
patch.object(session, "_check_cancelled"),
|
||||
):
|
||||
stream = session._try_stream(
|
||||
client=client,
|
||||
model="hypothetical-no-replay-claude",
|
||||
msgs=msgs,
|
||||
provider=real_provider,
|
||||
capabilities=caps,
|
||||
model_alias="hypothetical-no-replay-claude",
|
||||
)
|
||||
list(stream)
|
||||
|
||||
wire_msgs = captured.get("messages")
|
||||
assert isinstance(wire_msgs, list), (
|
||||
f"Expected messages= list at SDK boundary, got {captured}"
|
||||
)
|
||||
assistant = next(m for m in wire_msgs if m["role"] == "assistant")
|
||||
block_types = [b.get("type") for b in assistant["content"] if isinstance(b, dict)]
|
||||
assert "thinking" not in block_types, (
|
||||
"Capability gate did not block replay: thinking block reached the wire "
|
||||
f"despite supports_reasoning_replay=False (blocks={block_types})"
|
||||
)
|
||||
flat = repr(captured)
|
||||
assert "secret reasoning" not in flat, (
|
||||
"Reasoning text leaked into the SDK boundary payload despite capability gate"
|
||||
)
|
||||
|
||||
|
||||
class TestSessionToOpenAIResponsesBoundaryIntegration:
|
||||
"""End-to-end integration: session._try_stream -> real
|
||||
OpenAIResponsesProvider.create_streaming -> captured Responses
|
||||
SDK boundary call. Mirrors the AnthropicProvider test above
|
||||
but for the path-2 (Responses API) replay flow.
|
||||
|
||||
Pins the include= request kwarg + reasoning input-item emission
|
||||
actually fire at the wire boundary when the operator flag and
|
||||
model capability both allow.
|
||||
"""
|
||||
|
||||
def _stub_responses_client(self) -> tuple[MagicMock, dict[str, object]]:
|
||||
"""Mock OpenAI Responses client. ``client.responses.create``
|
||||
captures kwargs and returns an empty stream iterator."""
|
||||
captured: dict[str, object] = {}
|
||||
|
||||
def create(**kwargs: object) -> object:
|
||||
captured.update(kwargs)
|
||||
return iter([])
|
||||
|
||||
client = MagicMock()
|
||||
client.responses.create = create
|
||||
return client, captured
|
||||
|
||||
def _registry_with_reasoning_capability(
|
||||
self, replay: bool = True, supports_replay: bool = True
|
||||
) -> Any:
|
||||
from turnstone.core.providers._protocol import ModelCapabilities
|
||||
|
||||
return SimpleNamespace(
|
||||
get_config=lambda alias: SimpleNamespace(
|
||||
replay_reasoning_to_model=replay,
|
||||
capabilities={}, # no overrides
|
||||
),
|
||||
_caps=ModelCapabilities(
|
||||
context_window=400000,
|
||||
supports_temperature=False,
|
||||
reasoning_effort_values=("low", "medium", "high"),
|
||||
default_reasoning_effort="medium",
|
||||
supports_reasoning_replay=supports_replay,
|
||||
),
|
||||
)
|
||||
|
||||
def test_replay_true_adds_include_to_responses_request(self) -> None:
|
||||
from turnstone.core.providers._openai_responses import OpenAIResponsesProvider
|
||||
|
||||
registry = self._registry_with_reasoning_capability(replay=True, supports_replay=True)
|
||||
session = _make_session()
|
||||
session._registry = registry
|
||||
session._model_alias = "gpt-5"
|
||||
client, captured = self._stub_responses_client()
|
||||
real_provider = OpenAIResponsesProvider()
|
||||
with (
|
||||
patch.object(session, "_get_active_tools", return_value=None),
|
||||
patch.object(session, "_provider_extra_params", return_value=None),
|
||||
patch.object(session, "_get_deferred_names", return_value=frozenset()),
|
||||
patch.object(session, "_check_cancelled"),
|
||||
):
|
||||
stream = session._try_stream(
|
||||
client=client,
|
||||
model="gpt-5",
|
||||
msgs=[{"role": "user", "content": "hi"}],
|
||||
provider=real_provider,
|
||||
capabilities=registry._caps,
|
||||
model_alias="gpt-5",
|
||||
)
|
||||
list(stream)
|
||||
assert captured.get("include") == ["reasoning.encrypted_content"]
|
||||
|
||||
def test_replay_false_omits_include(self) -> None:
|
||||
from turnstone.core.providers._openai_responses import OpenAIResponsesProvider
|
||||
|
||||
registry = self._registry_with_reasoning_capability(replay=False, supports_replay=True)
|
||||
session = _make_session()
|
||||
session._registry = registry
|
||||
session._model_alias = "gpt-5"
|
||||
client, captured = self._stub_responses_client()
|
||||
real_provider = OpenAIResponsesProvider()
|
||||
with (
|
||||
patch.object(session, "_get_active_tools", return_value=None),
|
||||
patch.object(session, "_provider_extra_params", return_value=None),
|
||||
patch.object(session, "_get_deferred_names", return_value=frozenset()),
|
||||
patch.object(session, "_check_cancelled"),
|
||||
):
|
||||
stream = session._try_stream(
|
||||
client=client,
|
||||
model="gpt-5",
|
||||
msgs=[{"role": "user", "content": "hi"}],
|
||||
provider=real_provider,
|
||||
capabilities=registry._caps,
|
||||
model_alias="gpt-5",
|
||||
)
|
||||
list(stream)
|
||||
assert "include" not in captured
|
||||
|
||||
def test_capability_false_omits_include_even_when_flag_true(self) -> None:
|
||||
from turnstone.core.providers._openai_responses import OpenAIResponsesProvider
|
||||
|
||||
# Operator flips replay=True but the model has
|
||||
# supports_reasoning_replay=False (e.g. gpt-4o via Responses).
|
||||
# Capability gate prevents the include= from being sent.
|
||||
registry = self._registry_with_reasoning_capability(replay=True, supports_replay=False)
|
||||
session = _make_session()
|
||||
session._registry = registry
|
||||
session._model_alias = "gpt-4o"
|
||||
client, captured = self._stub_responses_client()
|
||||
real_provider = OpenAIResponsesProvider()
|
||||
with (
|
||||
patch.object(session, "_get_active_tools", return_value=None),
|
||||
patch.object(session, "_provider_extra_params", return_value=None),
|
||||
patch.object(session, "_get_deferred_names", return_value=frozenset()),
|
||||
patch.object(session, "_check_cancelled"),
|
||||
):
|
||||
stream = session._try_stream(
|
||||
client=client,
|
||||
model="gpt-4o",
|
||||
msgs=[{"role": "user", "content": "hi"}],
|
||||
provider=real_provider,
|
||||
capabilities=registry._caps,
|
||||
model_alias="gpt-4o",
|
||||
)
|
||||
list(stream)
|
||||
assert "include" not in captured
|
||||
|
||||
def test_replay_true_emits_reasoning_input_item(self) -> None:
|
||||
from turnstone.core.providers._openai_responses import OpenAIResponsesProvider
|
||||
|
||||
registry = self._registry_with_reasoning_capability(replay=True, supports_replay=True)
|
||||
session = _make_session()
|
||||
session._registry = registry
|
||||
session._model_alias = "gpt-5"
|
||||
client, captured = self._stub_responses_client()
|
||||
real_provider = OpenAIResponsesProvider()
|
||||
# Multi-turn conversation with stored reasoning on assistant turn.
|
||||
msgs: list[dict[str, object]] = [
|
||||
{"role": "user", "content": "explain"},
|
||||
{
|
||||
"role": "assistant",
|
||||
"content": "Final answer.",
|
||||
"_provider_content": [
|
||||
{
|
||||
"type": "reasoning",
|
||||
"id": "r_xyz",
|
||||
"summary": [{"type": "summary_text", "text": "I thought"}],
|
||||
"encrypted_content": "blob",
|
||||
}
|
||||
],
|
||||
},
|
||||
{"role": "user", "content": "follow-up"},
|
||||
]
|
||||
with (
|
||||
patch.object(session, "_get_active_tools", return_value=None),
|
||||
patch.object(session, "_provider_extra_params", return_value=None),
|
||||
patch.object(session, "_get_deferred_names", return_value=frozenset()),
|
||||
patch.object(session, "_check_cancelled"),
|
||||
):
|
||||
stream = session._try_stream(
|
||||
client=client,
|
||||
model="gpt-5",
|
||||
msgs=msgs,
|
||||
provider=real_provider,
|
||||
capabilities=registry._caps,
|
||||
model_alias="gpt-5",
|
||||
)
|
||||
list(stream)
|
||||
# Walk the wire input items — one of them must be the reasoning
|
||||
# round-trip (id matches what we stored).
|
||||
wire_input = captured.get("input")
|
||||
assert isinstance(wire_input, list)
|
||||
reasoning_items = [it for it in wire_input if it.get("type") == "reasoning"]
|
||||
assert len(reasoning_items) == 1
|
||||
assert reasoning_items[0]["id"] == "r_xyz"
|
||||
assert reasoning_items[0]["encrypted_content"] == "blob"
|
||||
|
||||
|
||||
class TestUtilityCompletionPassesFlag:
|
||||
"""Non-streaming utility path (title gen, compaction, extraction) —
|
||||
same plumbing requirement as streaming."""
|
||||
|
||||
def test_utility_completion_passes_resolved_flag(self) -> None:
|
||||
from turnstone.core.providers._protocol import ModelCapabilities
|
||||
|
||||
session = _make_session()
|
||||
session._registry = _registry_with_flag(replay=True)
|
||||
session._model_alias = "claude-opus-4-7"
|
||||
captured: dict[str, Any] = {}
|
||||
|
||||
def capture_completion(**kwargs: Any) -> Any:
|
||||
captured.update(kwargs)
|
||||
return SimpleNamespace(content="title", finish_reason="stop", usage=None)
|
||||
|
||||
mock_provider = MagicMock()
|
||||
mock_provider.create_completion = capture_completion
|
||||
session._provider = mock_provider
|
||||
caps = ModelCapabilities(max_output_tokens=0, supports_reasoning_replay=True)
|
||||
with (
|
||||
patch.object(session, "_get_capabilities", return_value=caps),
|
||||
patch.object(session, "_provider_extra_params", return_value=None),
|
||||
):
|
||||
session._utility_completion(
|
||||
messages=[{"role": "user", "content": "summarize"}],
|
||||
max_tokens=512,
|
||||
temperature=0.3,
|
||||
)
|
||||
assert captured["replay_reasoning_to_model"] is True
|
||||
@@ -0,0 +1,355 @@
|
||||
"""Tests for ChatSession synthetic ``reasoning_text`` block stamping (Phase 3 path 3).
|
||||
|
||||
Path 3 covers OpenAI Chat Completions endpoints — vLLM with
|
||||
``--reasoning-parser``, llama.cpp with ``reasoning_format``, Gemini's
|
||||
``/v1beta/openai/`` endpoint, and any other server that surfaces
|
||||
``delta.reasoning_content`` Pydantic extras. These have no native
|
||||
provider_blocks shape on the wire, so ``ChatSession._stream_response``
|
||||
captures the streamed reasoning text into ``reasoning_parts`` and
|
||||
``_maybe_synth_reasoning_block`` stamps it onto ``_provider_content``
|
||||
as a synthetic ``{type: "reasoning_text"}`` block at the end of the
|
||||
turn.
|
||||
|
||||
These tests pin:
|
||||
1. The synthesizer fires only when no native blocks were emitted AND
|
||||
reasoning was captured (Anthropic + OpenAI Responses bypass it).
|
||||
2. ``source`` field is tagged with the active model's server_type
|
||||
(informational; pulled from ``server_compat.server_type``).
|
||||
3. ``OpenAIChatCompletionsProvider.extract_reasoning_text`` round-trips
|
||||
the synthetic block on history rehydration.
|
||||
4. The synthetic shape is NOT in ``ANTHROPIC_VALID_BLOCK_TYPES`` so
|
||||
cross-model resumption (local-model → Anthropic) falls through
|
||||
cleanly to the text+tool_calls rebuild path.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from types import SimpleNamespace
|
||||
from typing import Any
|
||||
|
||||
from tests._session_helpers import make_session as _make_session
|
||||
from turnstone.core.providers._anthropic import (
|
||||
ANTHROPIC_VALID_BLOCK_TYPES,
|
||||
AnthropicProvider,
|
||||
)
|
||||
from turnstone.core.providers._openai_chat import OpenAIChatCompletionsProvider
|
||||
|
||||
|
||||
class TestMaybeSynthReasoningBlock:
|
||||
"""Direct unit tests for ``ChatSession._maybe_synth_reasoning_block``."""
|
||||
|
||||
def test_no_synth_when_provider_blocks_present(self) -> None:
|
||||
# Anthropic / OpenAI Responses path — native blocks already
|
||||
# carry the reasoning, no synth needed.
|
||||
session = _make_session()
|
||||
existing = [{"type": "thinking", "thinking": "x"}]
|
||||
out = session._maybe_synth_reasoning_block(existing, ["should not be added"])
|
||||
assert out is existing
|
||||
|
||||
def test_no_synth_when_reasoning_parts_empty(self) -> None:
|
||||
session = _make_session()
|
||||
out = session._maybe_synth_reasoning_block([], [])
|
||||
assert out == []
|
||||
|
||||
def test_no_synth_when_reasoning_parts_only_whitespace(self) -> None:
|
||||
session = _make_session()
|
||||
out = session._maybe_synth_reasoning_block([], [" ", "\n\t"])
|
||||
assert out == []
|
||||
|
||||
def test_synth_creates_reasoning_text_block(self) -> None:
|
||||
session = _make_session()
|
||||
out = session._maybe_synth_reasoning_block([], ["thought ", "process"])
|
||||
assert len(out) == 1
|
||||
assert out[0]["type"] == "reasoning_text"
|
||||
assert out[0]["text"] == "thought process"
|
||||
|
||||
def test_synth_omits_source_when_no_server_type(self) -> None:
|
||||
session = _make_session()
|
||||
# No registry / no server_compat → source field omitted.
|
||||
out = session._maybe_synth_reasoning_block([], ["text"])
|
||||
assert "source" not in out[0]
|
||||
|
||||
def test_synth_includes_source_when_server_type_resolvable(self) -> None:
|
||||
session = _make_session()
|
||||
session._registry = SimpleNamespace(
|
||||
get_config=lambda alias: SimpleNamespace(
|
||||
capabilities={"server_compat": {"server_type": "vllm"}},
|
||||
)
|
||||
)
|
||||
session._model_alias = "qwen3-32b"
|
||||
out = session._maybe_synth_reasoning_block([], ["text"])
|
||||
assert out[0]["source"] == "vllm"
|
||||
|
||||
def test_synth_handles_registry_exception(self) -> None:
|
||||
# _resolve_server_type silently returns "" on any lookup error
|
||||
# — synth still fires but omits the source field.
|
||||
class BrokenRegistry:
|
||||
def get_config(self, alias: str) -> Any:
|
||||
raise KeyError(alias)
|
||||
|
||||
session = _make_session()
|
||||
session._registry = BrokenRegistry()
|
||||
session._model_alias = "missing"
|
||||
out = session._maybe_synth_reasoning_block([], ["text"])
|
||||
assert out[0]["text"] == "text"
|
||||
assert "source" not in out[0]
|
||||
|
||||
def test_synth_appends_when_provider_blocks_are_non_reasoning(self) -> None:
|
||||
# GoogleProvider attaches raw tool_call dicts as provider_blocks
|
||||
# on the finish chunk (for thought_signature round-trip). When
|
||||
# the same turn streamed reasoning_delta (Gemini's reasoning_
|
||||
# content extra), the synthesizer must APPEND the synthetic
|
||||
# reasoning block rather than skip synthesis — otherwise the
|
||||
# reasoning text is shown live but lost on page reload.
|
||||
session = _make_session()
|
||||
existing = [
|
||||
{
|
||||
"id": "call_1",
|
||||
"type": "function",
|
||||
"function": {"name": "search", "arguments": "{}"},
|
||||
"thought_signature": "sig123",
|
||||
}
|
||||
]
|
||||
out = session._maybe_synth_reasoning_block(existing, ["I should search"])
|
||||
assert len(out) == 2
|
||||
assert out[0] is existing[0] # tool_call fidelity block survives intact
|
||||
assert out[1]["type"] == "reasoning_text"
|
||||
assert out[1]["text"] == "I should search"
|
||||
|
||||
def test_no_synth_when_openai_responses_reasoning_already_present(self) -> None:
|
||||
# OpenAI Responses native reasoning item — synth must NOT fire
|
||||
# even though provider_blocks contains ALSO non-reasoning items
|
||||
# (e.g. message blocks). The reasoning-bearing block satisfies
|
||||
# the persistence contract on its own.
|
||||
session = _make_session()
|
||||
existing = [
|
||||
{"type": "reasoning", "summary": [{"text": "openai reasoning"}]},
|
||||
{"type": "message", "role": "assistant", "content": "answer"},
|
||||
]
|
||||
out = session._maybe_synth_reasoning_block(existing, ["live reasoning text"])
|
||||
assert out is existing
|
||||
|
||||
def test_no_synth_when_non_reasoning_blocks_but_reasoning_parts_empty(self) -> None:
|
||||
# Google tool_calls with no reasoning streamed — return as-is.
|
||||
session = _make_session()
|
||||
existing = [
|
||||
{"id": "call_1", "type": "function", "function": {"name": "f", "arguments": "{}"}}
|
||||
]
|
||||
out = session._maybe_synth_reasoning_block(existing, [])
|
||||
assert out is existing
|
||||
|
||||
|
||||
class TestSyntheticBlockShapeContract:
|
||||
"""The synthetic block shape MUST stay outside Anthropic's valid
|
||||
block types so cross-model resumption falls through cleanly."""
|
||||
|
||||
def test_reasoning_text_not_in_anthropic_valid_types(self) -> None:
|
||||
# If this assertion ever fails, the cross-model resumption
|
||||
# safety story breaks: a synthetic block from a local-model
|
||||
# session would reach Anthropic's wire as a malformed block.
|
||||
assert "reasoning_text" not in ANTHROPIC_VALID_BLOCK_TYPES
|
||||
|
||||
def test_synthetic_block_falls_through_anthropic_shape_filter(self) -> None:
|
||||
# Cross-model resumption regression: turn 1 was on a local
|
||||
# model (synthetic block stamped), then the operator switched
|
||||
# to Anthropic. The shape filter must reject the synthetic
|
||||
# block and fall through to text+tool_calls rebuild.
|
||||
provider = AnthropicProvider()
|
||||
msg = {
|
||||
"role": "assistant",
|
||||
"content": "spoken answer",
|
||||
"_provider_content": [
|
||||
{"type": "reasoning_text", "text": "synth thought", "source": "vllm"},
|
||||
],
|
||||
}
|
||||
_, converted = provider._convert_messages([msg])
|
||||
assistant = next(m for m in converted if m["role"] == "assistant")
|
||||
block_types = [b.get("type") for b in assistant["content"] if isinstance(b, dict)]
|
||||
# Foreign block did NOT reach Anthropic's wire. Rebuilt from
|
||||
# text only.
|
||||
assert "reasoning_text" not in block_types
|
||||
assert assistant["content"] == [{"type": "text", "text": "spoken answer"}]
|
||||
|
||||
|
||||
class TestOpenAIChatExtractReasoningText:
|
||||
"""``OpenAIChatCompletionsProvider.extract_reasoning_text`` reads
|
||||
the synthetic block back out for UI rehydration."""
|
||||
|
||||
def test_reads_synthetic_reasoning_text_block(self) -> None:
|
||||
provider = OpenAIChatCompletionsProvider()
|
||||
blocks = [{"type": "reasoning_text", "text": "captured thought"}]
|
||||
assert provider.extract_reasoning_text(blocks) == "captured thought"
|
||||
|
||||
def test_concatenates_multiple_blocks(self) -> None:
|
||||
provider = OpenAIChatCompletionsProvider()
|
||||
blocks = [
|
||||
{"type": "reasoning_text", "text": "first"},
|
||||
{"type": "reasoning_text", "text": "second"},
|
||||
]
|
||||
assert provider.extract_reasoning_text(blocks) == "first\nsecond"
|
||||
|
||||
def test_skips_other_block_types(self) -> None:
|
||||
provider = OpenAIChatCompletionsProvider()
|
||||
blocks = [
|
||||
{"type": "thinking", "thinking": "anth"},
|
||||
{"type": "reasoning", "summary": [{"text": "openai"}]},
|
||||
{"type": "reasoning_text", "text": "chat"},
|
||||
]
|
||||
assert provider.extract_reasoning_text(blocks) == "chat"
|
||||
|
||||
def test_handles_empty_text_field(self) -> None:
|
||||
provider = OpenAIChatCompletionsProvider()
|
||||
blocks = [
|
||||
{"type": "reasoning_text", "text": ""},
|
||||
{"type": "reasoning_text", "text": "kept"},
|
||||
]
|
||||
assert provider.extract_reasoning_text(blocks) == "kept"
|
||||
|
||||
def test_handles_missing_text_field(self) -> None:
|
||||
provider = OpenAIChatCompletionsProvider()
|
||||
blocks = [
|
||||
{"type": "reasoning_text"}, # no text
|
||||
{"type": "reasoning_text", "text": "kept"},
|
||||
]
|
||||
assert provider.extract_reasoning_text(blocks) == "kept"
|
||||
|
||||
def test_returns_empty_for_no_synth_blocks(self) -> None:
|
||||
provider = OpenAIChatCompletionsProvider()
|
||||
blocks = [{"type": "thinking", "thinking": "x"}]
|
||||
assert provider.extract_reasoning_text(blocks) == ""
|
||||
|
||||
|
||||
class TestStreamResponseSynthBlockIntegration:
|
||||
"""Integration test: drives a fake reasoning-emitting stream
|
||||
through ``ChatSession._stream_response`` and asserts the
|
||||
synthesizer wires up correctly. Pins the call site at
|
||||
``session.py`` (where ``_maybe_synth_reasoning_block`` is invoked
|
||||
on the assembled provider_blocks before stamping ``_provider_content``)
|
||||
— without this, a future refactor that drops the synthesizer call
|
||||
would silently break path-3 capture (vLLM/llama.cpp/Gemini-compat
|
||||
reasoning would be visible live but invisible on history reload).
|
||||
"""
|
||||
|
||||
def _make_stream(self, content: str, reasoning: str) -> Any:
|
||||
"""Build an iterator of StreamChunks that mimic a path-3
|
||||
capture (reasoning_delta chunks, content chunks, no
|
||||
provider_blocks emitted).
|
||||
"""
|
||||
from turnstone.core.providers._protocol import StreamChunk, UsageInfo
|
||||
|
||||
chunks = []
|
||||
# Reasoning first (matches live SSE order).
|
||||
if reasoning:
|
||||
chunks.append(StreamChunk(reasoning_delta=reasoning, is_first=True))
|
||||
# Content next.
|
||||
if content:
|
||||
chunks.append(
|
||||
StreamChunk(
|
||||
content_delta=content,
|
||||
is_first=not reasoning,
|
||||
)
|
||||
)
|
||||
# Final chunk with finish_reason + usage.
|
||||
chunks.append(
|
||||
StreamChunk(
|
||||
finish_reason="stop",
|
||||
usage=UsageInfo(prompt_tokens=10, completion_tokens=20, total_tokens=30),
|
||||
)
|
||||
)
|
||||
return iter(chunks)
|
||||
|
||||
def test_stream_response_stamps_synth_block_when_path3_reasoning_captured(
|
||||
self,
|
||||
) -> None:
|
||||
"""Drive a fake stream emitting reasoning_delta chunks (no
|
||||
native provider_blocks) through ``_stream_response``; assert
|
||||
the resulting assistant_msg carries a synthetic reasoning_text
|
||||
block stamped onto ``_provider_content``."""
|
||||
session = _make_session()
|
||||
# No registry → source field omitted from synth block.
|
||||
stream = self._make_stream(content="Final answer.", reasoning="path-3 reasoning")
|
||||
msg = session._stream_response(stream)
|
||||
assert msg["role"] == "assistant"
|
||||
assert msg["content"] == "Final answer."
|
||||
# Synthetic block should be stamped onto _provider_content.
|
||||
provider_content = msg.get("_provider_content")
|
||||
assert isinstance(provider_content, list)
|
||||
assert len(provider_content) == 1
|
||||
assert provider_content[0]["type"] == "reasoning_text"
|
||||
assert provider_content[0]["text"] == "path-3 reasoning"
|
||||
|
||||
def test_stream_response_no_synth_when_no_reasoning_captured(self) -> None:
|
||||
"""Stream emits only content (no reasoning_delta). No synth
|
||||
block stamped — _provider_content key absent on assistant_msg."""
|
||||
session = _make_session()
|
||||
stream = self._make_stream(content="just content", reasoning="")
|
||||
msg = session._stream_response(stream)
|
||||
assert msg["content"] == "just content"
|
||||
# No synth block (and no native blocks either) → key absent.
|
||||
assert "_provider_content" not in msg
|
||||
|
||||
def test_stream_response_synth_block_carries_source_when_server_type_resolvable(
|
||||
self,
|
||||
) -> None:
|
||||
"""When the active model has server_compat.server_type set,
|
||||
the synth block carries it as the ``source`` field."""
|
||||
session = _make_session()
|
||||
session._registry = SimpleNamespace(
|
||||
get_config=lambda alias: SimpleNamespace(
|
||||
capabilities={"server_compat": {"server_type": "vllm"}},
|
||||
)
|
||||
)
|
||||
session._model_alias = "qwen3-32b"
|
||||
stream = self._make_stream(content="answer", reasoning="reasoning text")
|
||||
msg = session._stream_response(stream)
|
||||
provider_content = msg.get("_provider_content")
|
||||
assert isinstance(provider_content, list)
|
||||
assert provider_content[0]["source"] == "vllm"
|
||||
|
||||
|
||||
class TestResolveServerType:
|
||||
"""Direct unit tests for the helper that pulls server_type from
|
||||
the active model's capabilities dict."""
|
||||
|
||||
def test_returns_empty_when_no_registry(self) -> None:
|
||||
session = _make_session()
|
||||
session._registry = None
|
||||
assert session._resolve_server_type() == ""
|
||||
|
||||
def test_returns_empty_when_no_alias(self) -> None:
|
||||
session = _make_session()
|
||||
session._registry = SimpleNamespace(
|
||||
get_config=lambda alias: SimpleNamespace(capabilities={})
|
||||
)
|
||||
session._model_alias = ""
|
||||
assert session._resolve_server_type() == ""
|
||||
|
||||
def test_returns_server_type_when_present(self) -> None:
|
||||
session = _make_session()
|
||||
session._registry = SimpleNamespace(
|
||||
get_config=lambda alias: SimpleNamespace(
|
||||
capabilities={"server_compat": {"server_type": "llama.cpp"}}
|
||||
)
|
||||
)
|
||||
session._model_alias = "local-model"
|
||||
assert session._resolve_server_type() == "llama.cpp"
|
||||
|
||||
def test_returns_empty_when_server_compat_missing(self) -> None:
|
||||
session = _make_session()
|
||||
session._registry = SimpleNamespace(
|
||||
get_config=lambda alias: SimpleNamespace(
|
||||
capabilities={"context_window": 32768},
|
||||
)
|
||||
)
|
||||
session._model_alias = "local-model"
|
||||
assert session._resolve_server_type() == ""
|
||||
|
||||
def test_returns_empty_on_exception(self) -> None:
|
||||
class BrokenRegistry:
|
||||
def get_config(self, alias: str) -> Any:
|
||||
raise RuntimeError("boom")
|
||||
|
||||
session = _make_session()
|
||||
session._registry = BrokenRegistry()
|
||||
session._model_alias = "x"
|
||||
assert session._resolve_server_type() == ""
|
||||
@@ -872,3 +872,413 @@ def test_concurrent_enqueue_and_listener_registration() -> None:
|
||||
# intent survives optimization-mode assertion stripping.
|
||||
assert not producer.is_alive()
|
||||
assert all(not s.is_alive() for s in subscribers)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Per-turn inflight buffers — SSE refresh-resume snapshot path
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_on_content_token_writes_to_both_buffers() -> None:
|
||||
"""``on_content_token`` writes to the multi-turn buffer (IDLE
|
||||
piggyback) AND the per-turn inflight buffer (SSE snapshot)."""
|
||||
ui = _make_ui()
|
||||
ui.on_content_token("hello")
|
||||
assert ui._ws_turn_content == ["hello"]
|
||||
assert ui._ws_inflight_content == ["hello"]
|
||||
assert ui._ws_inflight_seq == 1
|
||||
|
||||
|
||||
def test_on_reasoning_token_writes_to_inflight_buffer_only() -> None:
|
||||
"""Reasoning has no multi-turn IDLE piggyback — only the inflight
|
||||
buffer + the seq counter."""
|
||||
ui = _make_ui()
|
||||
ui.on_reasoning_token("thinking...")
|
||||
assert ui._ws_inflight_reasoning == ["thinking..."]
|
||||
assert ui._ws_inflight_seq == 1
|
||||
# Multi-turn buffer is content-only and untouched by reasoning.
|
||||
assert ui._ws_turn_content == []
|
||||
|
||||
|
||||
def test_inflight_seq_advances_on_every_emit_even_at_cap() -> None:
|
||||
"""Cap-hit content tokens MUST advance ``_ws_inflight_seq``,
|
||||
even though the buffer rejected the append. If seq stalled at
|
||||
high-water-pre-cap, a subscriber registering AFTER the cap is
|
||||
hit would capture ``snap_seq == stalled_seq`` and every
|
||||
subsequent live token (also tagged with the stalled seq) would
|
||||
be filter-dropped by the events handler — silently losing the
|
||||
rest of the stream. The cap is a buffer-size limit, not a
|
||||
"stop streaming" signal."""
|
||||
from turnstone.core.session_ui_base import _MAX_TURN_CONTENT_CHARS
|
||||
|
||||
ui = _make_ui()
|
||||
chunk = "x" * 1024
|
||||
while ui._ws_inflight_content_size < _MAX_TURN_CONTENT_CHARS:
|
||||
ui.on_content_token(chunk)
|
||||
seq_at_cap = ui._ws_inflight_seq
|
||||
|
||||
# Cap-hit token: seq MUST advance (no buffer append, but the
|
||||
# event still gets a fresh seq for the dedup filter).
|
||||
ui.on_content_token(chunk)
|
||||
assert ui._ws_inflight_seq == seq_at_cap + 1
|
||||
# Buffer remains bounded — the cap-hit token is NOT in inflight.
|
||||
assert ui._ws_inflight_content_size <= _MAX_TURN_CONTENT_CHARS + len(chunk)
|
||||
|
||||
|
||||
def test_subscriber_after_cap_hit_receives_subsequent_tokens() -> None:
|
||||
"""Regression for Copilot's cap+seq finding: a subscriber that
|
||||
connects AFTER the inflight buffer is at cap must still receive
|
||||
live tokens past the cap. Past-cap tokens are absent from
|
||||
``snap.content`` (the snapshot text was truncated at cap) but
|
||||
the live stream past them must NOT be filter-dropped."""
|
||||
from turnstone.core.session_ui_base import _MAX_TURN_CONTENT_CHARS
|
||||
|
||||
ui = _make_ui()
|
||||
chunk = "x" * 1024
|
||||
while ui._ws_inflight_content_size < _MAX_TURN_CONTENT_CHARS:
|
||||
ui.on_content_token(chunk)
|
||||
# Stream a few tokens PAST the cap before subscribing.
|
||||
for _ in range(3):
|
||||
ui.on_content_token(chunk)
|
||||
|
||||
lq, snap = ui.register_listener_with_in_progress_snapshot()
|
||||
snap_seq = snap["seq"]
|
||||
|
||||
# Live token past cap.
|
||||
ui.on_content_token(chunk)
|
||||
ev = lq.get_nowait()
|
||||
assert ev["type"] == "content"
|
||||
# The critical invariant: seq advances per-emit, so the new
|
||||
# event's _seq is strictly greater than the snap_seq the
|
||||
# subscriber captured. Without this, the events handler's
|
||||
# ``seq <= snap_seq`` filter would drop every token past the
|
||||
# cap (silent token loss for refresh-past-cap).
|
||||
assert ev["_seq"] > snap_seq, (
|
||||
f"Token past cap has _seq={ev['_seq']} which is <= "
|
||||
f"snap_seq={snap_seq} — would be silently dropped after a "
|
||||
f"refresh past the cap."
|
||||
)
|
||||
|
||||
|
||||
def test_subscriber_after_reasoning_cap_hit_receives_subsequent_tokens() -> None:
|
||||
"""Same invariant as content cap: reasoning subscribers past
|
||||
cap must keep receiving live reasoning tokens."""
|
||||
from turnstone.core.session_ui_base import _MAX_TURN_CONTENT_CHARS
|
||||
|
||||
ui = _make_ui()
|
||||
chunk = "x" * 1024
|
||||
while ui._ws_inflight_reasoning_size < _MAX_TURN_CONTENT_CHARS:
|
||||
ui.on_reasoning_token(chunk)
|
||||
for _ in range(3):
|
||||
ui.on_reasoning_token(chunk)
|
||||
|
||||
lq, snap = ui.register_listener_with_in_progress_snapshot()
|
||||
snap_seq = snap["seq"]
|
||||
|
||||
ui.on_reasoning_token(chunk)
|
||||
ev = lq.get_nowait()
|
||||
assert ev["type"] == "reasoning"
|
||||
assert ev["_seq"] > snap_seq
|
||||
|
||||
|
||||
def test_on_turn_committed_resets_inflight_after_commit() -> None:
|
||||
"""``on_turn_committed`` fires immediately after each
|
||||
``messages.append(assistant_msg)`` in the send loop. Without it,
|
||||
the inflight buffer keeps the just-committed turn's content
|
||||
during the post-commit tool-execution window — and a refresh in
|
||||
that window would show the assistant turn TWICE (history list
|
||||
+ in_progress_snapshot)."""
|
||||
ui = _make_ui()
|
||||
ui.on_content_token("Just-finished turn ")
|
||||
ui.on_reasoning_token("Reasoning for the turn ")
|
||||
# Sanity: buffer is populated pre-commit.
|
||||
assert ui._ws_inflight_content == ["Just-finished turn "]
|
||||
assert ui._ws_inflight_reasoning == ["Reasoning for the turn "]
|
||||
|
||||
ui.on_turn_committed()
|
||||
|
||||
# Inflight content + reasoning reset; seq stays monotonic.
|
||||
assert ui._ws_inflight_content == []
|
||||
assert ui._ws_inflight_reasoning == []
|
||||
# Multi-turn buffer is NOT reset by commit (it drains at idle).
|
||||
assert ui._ws_turn_content == ["Just-finished turn "]
|
||||
|
||||
|
||||
def test_inflight_snapshot_empty_during_post_commit_tool_window() -> None:
|
||||
"""Models the user-reported bug: refresh during a tool-execution
|
||||
window between commit and the next stream. Pre-fix: snapshot has
|
||||
the just-committed turn's text → double-renders against history.
|
||||
Post-fix: snapshot is empty → no double-render. Seq stays
|
||||
monotonic (carries the high-water mark across turn boundaries)."""
|
||||
ui = _make_ui()
|
||||
ui.on_content_token("Calling tool with these args: ")
|
||||
seq_pre_commit = ui._ws_inflight_seq
|
||||
ui.on_turn_committed() # session.py fires this after messages.append
|
||||
# We're now in the tool-execution window. A reconnecting client
|
||||
# would call register_listener_with_in_progress_snapshot.
|
||||
_, snap = ui.register_listener_with_in_progress_snapshot()
|
||||
assert snap["content"] == ""
|
||||
assert snap["reasoning"] == ""
|
||||
# Seq did NOT reset — must remain monotonic across turns.
|
||||
assert snap["seq"] == seq_pre_commit
|
||||
|
||||
|
||||
def test_on_turn_start_resets_inflight_content_and_reasoning() -> None:
|
||||
"""``on_turn_start`` clears the per-turn content + reasoning
|
||||
buffers but does NOT touch the multi-turn ``_ws_turn_content``
|
||||
(which the dashboard's IDLE-piggyback payload depends on) and
|
||||
does NOT reset the seq counter (must remain monotonic across
|
||||
turn boundaries — see ``test_inflight_seq_monotonic_across_turn_boundaries``)."""
|
||||
ui = _make_ui()
|
||||
ui.on_content_token("turn-1 ")
|
||||
ui.on_reasoning_token("reasoning-1 ")
|
||||
multi_pre = list(ui._ws_turn_content)
|
||||
multi_pre_size = ui._ws_turn_content_size
|
||||
|
||||
ui.on_turn_start()
|
||||
|
||||
assert ui._ws_inflight_content == []
|
||||
assert ui._ws_inflight_content_size == 0
|
||||
assert ui._ws_inflight_reasoning == []
|
||||
assert ui._ws_inflight_reasoning_size == 0
|
||||
# Multi-turn untouched.
|
||||
assert ui._ws_turn_content == multi_pre
|
||||
assert ui._ws_turn_content_size == multi_pre_size
|
||||
|
||||
|
||||
def test_register_listener_with_in_progress_snapshot_empty() -> None:
|
||||
ui = _make_ui()
|
||||
lq, snap = ui.register_listener_with_in_progress_snapshot()
|
||||
assert isinstance(lq, queue.Queue)
|
||||
assert lq in ui._listeners
|
||||
assert snap == {"content": "", "reasoning": "", "seq": 0}
|
||||
|
||||
|
||||
def test_register_listener_with_in_progress_snapshot_populated() -> None:
|
||||
ui = _make_ui()
|
||||
ui.on_content_token("Hello, ")
|
||||
ui.on_content_token("world!")
|
||||
ui.on_reasoning_token("planning a greeting")
|
||||
lq, snap = ui.register_listener_with_in_progress_snapshot()
|
||||
assert snap["content"] == "Hello, world!"
|
||||
assert snap["reasoning"] == "planning a greeting"
|
||||
# seq counts every successful append across BOTH buffers.
|
||||
assert snap["seq"] == 3
|
||||
# Listener is registered — later live tokens land in lq.
|
||||
ui.on_content_token(" Goodbye.")
|
||||
ev = lq.get_nowait()
|
||||
assert ev["type"] == "content"
|
||||
assert ev["text"] == " Goodbye."
|
||||
assert ev["_seq"] == 4
|
||||
|
||||
|
||||
def test_register_listener_with_in_progress_snapshot_only_inflight_not_multi_turn() -> None:
|
||||
"""The snapshot reflects the in-progress turn only — anything
|
||||
cleared by ``on_turn_start`` (a prior committed turn within the
|
||||
same send) must NOT appear in the snapshot, even though the
|
||||
multi-turn buffer still has it."""
|
||||
ui = _make_ui()
|
||||
ui.on_content_token("PRIOR_TURN ")
|
||||
ui.on_turn_start() # commit boundary — inflight reset
|
||||
ui.on_content_token("CURRENT")
|
||||
_, snap = ui.register_listener_with_in_progress_snapshot()
|
||||
assert snap["content"] == "CURRENT"
|
||||
# Multi-turn buffer still has both turns (drives the IDLE piggyback).
|
||||
assert "".join(ui._ws_turn_content) == "PRIOR_TURN CURRENT"
|
||||
|
||||
|
||||
def test_seq_filter_dedup_round_trip() -> None:
|
||||
"""End-to-end dedup invariant: every token appears exactly once
|
||||
when reconstructing from snapshot + listener queue under live
|
||||
writes that race the registration. Models the events handler."""
|
||||
ui = _make_ui()
|
||||
for ch in "abcde":
|
||||
ui.on_content_token(ch)
|
||||
lq, snap = ui.register_listener_with_in_progress_snapshot()
|
||||
for ch in "fgh":
|
||||
ui.on_content_token(ch)
|
||||
|
||||
reconstructed = snap["content"]
|
||||
while True:
|
||||
try:
|
||||
ev = lq.get_nowait()
|
||||
except queue.Empty:
|
||||
break
|
||||
if ev.get("_seq", 0) <= snap["seq"]:
|
||||
continue
|
||||
reconstructed += ev["text"]
|
||||
assert reconstructed == "abcdefgh"
|
||||
|
||||
|
||||
def test_seq_filter_drops_overlap_when_register_lands_after_writer() -> None:
|
||||
"""Race: writer appends + emits while a second register snapshots
|
||||
after the writer. The live event has _seq <= snap.seq → must be
|
||||
dropped to avoid double-render."""
|
||||
ui = _make_ui()
|
||||
# Register a first listener so the writer's enqueue lands somewhere.
|
||||
lq1, _ = ui.register_listener_with_in_progress_snapshot()
|
||||
ui.on_content_token("X")
|
||||
# Second register snapshots AFTER the write — snap has "X" AND
|
||||
# the writer's enqueue is in lq1.
|
||||
_, snap2 = ui.register_listener_with_in_progress_snapshot()
|
||||
assert snap2["content"] == "X"
|
||||
# Drain lq1 with the filter against snap2.seq — duplicate dropped.
|
||||
duped: list[str] = []
|
||||
while True:
|
||||
try:
|
||||
ev = lq1.get_nowait()
|
||||
except queue.Empty:
|
||||
break
|
||||
if ev.get("_seq", 0) <= snap2["seq"]:
|
||||
continue
|
||||
duped.append(ev["text"])
|
||||
assert duped == []
|
||||
|
||||
|
||||
def test_inflight_seq_monotonic_across_turn_boundaries() -> None:
|
||||
"""Regression: a subscriber registered mid-turn-N must still
|
||||
receive turn N+1's tokens. The seq counter is monotonic across
|
||||
turn boundaries — resetting it at on_turn_committed/on_turn_start
|
||||
would silently drop turn N+1's first M tokens (M = the snap_seq
|
||||
captured mid-turn-N) via the events handler's `seq <= snap_seq`
|
||||
filter."""
|
||||
ui = _make_ui()
|
||||
# Turn N: stream tokens, register a listener mid-turn.
|
||||
ui.on_content_token("turn-N tok1 ")
|
||||
ui.on_content_token("turn-N tok2 ")
|
||||
lq, snap = ui.register_listener_with_in_progress_snapshot()
|
||||
snap_seq = snap["seq"]
|
||||
assert snap_seq == 2
|
||||
# Turn N completes, turn N+1 begins.
|
||||
ui.on_turn_committed()
|
||||
ui.on_turn_start()
|
||||
# Turn N+1's first content token. With the q-1 fix, seq is
|
||||
# monotonic (3), not reset to 1. The events handler's
|
||||
# `seq <= snap_seq` filter must NOT swallow it.
|
||||
ui.on_content_token("turn-N+1 tok1 ")
|
||||
ev = lq.get_nowait()
|
||||
assert ev["type"] == "content"
|
||||
assert ev["text"] == "turn-N+1 tok1 "
|
||||
assert ev["_seq"] > snap_seq, (
|
||||
f"Token from turn N+1 has _seq={ev['_seq']} which is <= "
|
||||
f"snap_seq={snap_seq} — the events handler's dedup filter "
|
||||
f"would silently drop it on a long-lived SSE subscription."
|
||||
)
|
||||
|
||||
|
||||
def test_snapshot_and_consume_drains_inflight_at_idle() -> None:
|
||||
"""Regression for the cancel/error path: ``on_turn_committed`` is
|
||||
NOT called from cancel handlers, but every exit path eventually
|
||||
fires ``_emit_state("idle")`` (cancel) or ``_emit_state("error")``
|
||||
(exception). The IDLE/ERROR branches of
|
||||
``snapshot_and_consume_state_payload`` must drain the inflight
|
||||
buffers so a refresh post-cancel doesn't double-render the
|
||||
cancelled fragment against history's marker'd version."""
|
||||
ui = _make_ui()
|
||||
ui.on_content_token("partial cancelled text ")
|
||||
ui.on_reasoning_token("partial reasoning ")
|
||||
assert ui._ws_inflight_content_size > 0
|
||||
assert ui._ws_inflight_reasoning_size > 0
|
||||
|
||||
ui.snapshot_and_consume_state_payload("idle")
|
||||
|
||||
assert ui._ws_inflight_content == []
|
||||
assert ui._ws_inflight_content_size == 0
|
||||
assert ui._ws_inflight_reasoning == []
|
||||
assert ui._ws_inflight_reasoning_size == 0
|
||||
|
||||
|
||||
def test_snapshot_and_consume_drains_inflight_at_error() -> None:
|
||||
"""Regression for the exception path: ERROR-branch must drain
|
||||
inflight too (parallel to the IDLE branch)."""
|
||||
ui = _make_ui()
|
||||
ui.on_content_token("partial errored text ")
|
||||
ui.on_reasoning_token("partial errored reasoning ")
|
||||
|
||||
ui.snapshot_and_consume_state_payload("error")
|
||||
|
||||
assert ui._ws_inflight_content == []
|
||||
assert ui._ws_inflight_reasoning == []
|
||||
|
||||
|
||||
def test_snapshot_and_consume_does_not_reset_seq_at_idle_or_error() -> None:
|
||||
"""The IDLE/ERROR drain clears content + reasoning but must NOT
|
||||
reset the seq counter — long-lived subscribers' snap_seq must
|
||||
stay valid across turn boundaries (see the q-1 invariant test)."""
|
||||
ui = _make_ui()
|
||||
ui.on_content_token("a")
|
||||
ui.on_content_token("b")
|
||||
assert ui._ws_inflight_seq == 2
|
||||
|
||||
ui.snapshot_and_consume_state_payload("idle")
|
||||
assert ui._ws_inflight_seq == 2
|
||||
|
||||
ui.snapshot_and_consume_state_payload("error")
|
||||
assert ui._ws_inflight_seq == 2
|
||||
|
||||
|
||||
def test_listeners_share_dict_reference_warning() -> None:
|
||||
"""Pinning the shape that necessitated the events-handler shallow
|
||||
copy: ``_enqueue`` puts ONE dict reference into every listener
|
||||
queue. If multiple SSE coroutines mutate (e.g. ``del event[\"_seq\"]``)
|
||||
without copying first, they corrupt each other's view. The fix
|
||||
in make_events_handler is ``event = dict(event)`` immediately
|
||||
after ``client_queue.get`` — verify the underlying invariant
|
||||
here so a future refactor of ``_enqueue`` can't silently break
|
||||
the assumption the events handler relies on."""
|
||||
ui = _make_ui()
|
||||
lq1, _ = ui.register_listener_with_in_progress_snapshot()
|
||||
lq2, _ = ui.register_listener_with_in_progress_snapshot()
|
||||
ui.on_content_token("X")
|
||||
ev1 = lq1.get_nowait()
|
||||
ev2 = lq2.get_nowait()
|
||||
# Same reference today — consumers MUST shallow-copy before any
|
||||
# mutation. If a future _enqueue change makes this no longer
|
||||
# true, the events handler's defensive copy becomes redundant
|
||||
# but harmless; if this assertion suddenly fails the underlying
|
||||
# invariant has shifted and the handler comment should be updated.
|
||||
assert ev1 is ev2
|
||||
|
||||
|
||||
def test_concurrent_writer_and_register_with_snapshot_no_loss_no_dup() -> None:
|
||||
"""Stress: many tokens streaming + a register_with_snapshot landing
|
||||
at a random point. End state: snapshot ∪ filtered_live == every
|
||||
token written, exactly once."""
|
||||
ui = _make_ui()
|
||||
n_tokens = 500
|
||||
snap_box: dict[str, Any] = {}
|
||||
lq_box: dict[str, queue.Queue[Any]] = {}
|
||||
|
||||
def _writer() -> None:
|
||||
for i in range(n_tokens):
|
||||
ui.on_content_token(f"{i},")
|
||||
|
||||
def _registrar() -> None:
|
||||
# Tiny sleep so the writer is mid-flight.
|
||||
threading.Event().wait(0.001)
|
||||
lq, snap = ui.register_listener_with_in_progress_snapshot()
|
||||
snap_box["snap"] = snap
|
||||
lq_box["lq"] = lq
|
||||
|
||||
w = threading.Thread(target=_writer)
|
||||
r = threading.Thread(target=_registrar)
|
||||
w.start()
|
||||
r.start()
|
||||
w.join()
|
||||
r.join()
|
||||
|
||||
snap = snap_box["snap"]
|
||||
lq = lq_box["lq"]
|
||||
reconstructed = snap["content"]
|
||||
while True:
|
||||
try:
|
||||
ev = lq.get_nowait()
|
||||
except queue.Empty:
|
||||
break
|
||||
if ev.get("_seq", 0) <= snap["seq"]:
|
||||
continue
|
||||
reconstructed += ev["text"]
|
||||
expected = "".join(f"{i}," for i in range(n_tokens))
|
||||
assert reconstructed == expected, (
|
||||
f"reconstruction mismatch: len(rec)={len(reconstructed)}, len(exp)={len(expected)}"
|
||||
)
|
||||
|
||||
@@ -24,6 +24,12 @@ from turnstone.core.storage._registry import get_storage
|
||||
class NullUI:
|
||||
"""UI adapter that discards all output."""
|
||||
|
||||
def on_turn_start(self):
|
||||
pass
|
||||
|
||||
def on_turn_committed(self):
|
||||
pass
|
||||
|
||||
def on_thinking_start(self):
|
||||
pass
|
||||
|
||||
|
||||
@@ -34,6 +34,12 @@ from turnstone.core.storage._sqlite import SQLiteBackend
|
||||
class NullUI:
|
||||
"""UI adapter that discards all output."""
|
||||
|
||||
def on_turn_start(self):
|
||||
pass
|
||||
|
||||
def on_turn_committed(self):
|
||||
pass
|
||||
|
||||
def on_thinking_start(self):
|
||||
pass
|
||||
|
||||
|
||||
@@ -2,7 +2,9 @@
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import queue
|
||||
from types import SimpleNamespace
|
||||
from typing import TYPE_CHECKING, Any
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
@@ -1762,3 +1764,155 @@ class TestTenantCheckOnReadEndpoints:
|
||||
assert cold_check in offloaded, (
|
||||
f"tenant_check must be invoked through asyncio.to_thread; got {offloaded}"
|
||||
)
|
||||
|
||||
|
||||
class TestHistoryReasoningRehydration:
|
||||
"""The lifted ``GET /v1/api/workstreams/{ws_id}/history`` surfaces
|
||||
stored Anthropic thinking blocks on assistant messages so a page
|
||||
refresh re-renders the reasoning bubble. Drives through the real
|
||||
``AnthropicProvider.extract_reasoning_text`` and the storage
|
||||
``reconstruct_messages`` boundary that JSON-decodes
|
||||
``provider_data`` into ``_provider_content``.
|
||||
"""
|
||||
|
||||
def test_history_handler_surfaces_reasoning_for_anthropic_thinking(self, _inject_storage):
|
||||
ws_id = "ws-reason-1"
|
||||
_inject_storage.register_workstream(ws_id, kind="interactive", user_id="test-user")
|
||||
provider_data = json.dumps(
|
||||
[
|
||||
{"type": "thinking", "thinking": "let me reason", "signature": "s"},
|
||||
{"type": "text", "text": "Final answer."},
|
||||
]
|
||||
)
|
||||
_inject_storage.save_message(
|
||||
ws_id, "assistant", "Final answer.", provider_data=provider_data
|
||||
)
|
||||
# No live session — exercises the storage-only path which
|
||||
# falls back to default surface_persisted_reasoning=True.
|
||||
mock_mgr = MagicMock()
|
||||
mock_mgr.get.return_value = None
|
||||
client = _build_history_app(mock_mgr, _inject_storage)
|
||||
|
||||
r = client.get(f"/v1/api/workstreams/{ws_id}/history")
|
||||
assert r.status_code == 200
|
||||
msgs = r.json()["messages"]
|
||||
assistant = next(m for m in msgs if m.get("role") == "assistant")
|
||||
assert assistant["reasoning"] == "let me reason"
|
||||
|
||||
def test_history_handler_strips_provider_content(self, _inject_storage):
|
||||
ws_id = "ws-reason-2"
|
||||
_inject_storage.register_workstream(ws_id, kind="interactive", user_id="test-user")
|
||||
provider_data = json.dumps([{"type": "thinking", "thinking": "x", "signature": "s"}])
|
||||
_inject_storage.save_message(ws_id, "assistant", "Answer.", provider_data=provider_data)
|
||||
mock_mgr = MagicMock()
|
||||
mock_mgr.get.return_value = None
|
||||
client = _build_history_app(mock_mgr, _inject_storage)
|
||||
|
||||
r = client.get(f"/v1/api/workstreams/{ws_id}/history")
|
||||
assert r.status_code == 200
|
||||
for m in r.json()["messages"]:
|
||||
assert "_provider_content" not in m
|
||||
|
||||
def test_history_handler_with_persist_flag_false_via_live_session(self, _inject_storage):
|
||||
"""Operator-flipped ``surface_persisted_reasoning=False`` on the active
|
||||
model suppresses the reasoning field even when the data is
|
||||
stored. ``_provider_content`` is still stripped from the wire.
|
||||
"""
|
||||
ws_id = "ws-reason-3"
|
||||
_inject_storage.register_workstream(ws_id, kind="interactive", user_id="test-user")
|
||||
provider_data = json.dumps([{"type": "thinking", "thinking": "hidden", "signature": "s"}])
|
||||
_inject_storage.save_message(ws_id, "assistant", "Answer.", provider_data=provider_data)
|
||||
live_session = SimpleNamespace(
|
||||
id=ws_id,
|
||||
_registry=SimpleNamespace(
|
||||
get_config=lambda alias: SimpleNamespace(surface_persisted_reasoning=False)
|
||||
),
|
||||
_model_alias="claude-opus-4-7",
|
||||
)
|
||||
mock_mgr = MagicMock()
|
||||
mock_mgr.get.return_value = live_session
|
||||
client = _build_history_app(mock_mgr, _inject_storage)
|
||||
|
||||
r = client.get(f"/v1/api/workstreams/{ws_id}/history")
|
||||
assert r.status_code == 200
|
||||
for m in r.json()["messages"]:
|
||||
if m.get("role") == "assistant":
|
||||
assert "reasoning" not in m
|
||||
assert "_provider_content" not in m
|
||||
|
||||
def test_history_handler_cold_workstream_resolves_via_workstream_config(self, _inject_storage):
|
||||
"""Cold workstream (no live session) — the handler walks
|
||||
``workstream_config.model_alias`` (persisted at first send by
|
||||
the SessionManager rehydrate path) and looks up the active
|
||||
model's ``surface_persisted_reasoning`` flag through the global registry
|
||||
on ``app.state``. Operator flag-flip is honored uniformly
|
||||
across live and cold workstreams.
|
||||
"""
|
||||
ws_id = "ws-reason-cold"
|
||||
_inject_storage.register_workstream(ws_id, kind="interactive", user_id="test-user")
|
||||
# Simulate the model alias persisted by the rehydrate path
|
||||
# (session_manager.py:628-629 reads it back via the same key).
|
||||
_inject_storage.save_workstream_config(ws_id, {"model_alias": "claude-opus-4-7"})
|
||||
provider_data = json.dumps(
|
||||
[{"type": "thinking", "thinking": "should not surface", "signature": "s"}]
|
||||
)
|
||||
_inject_storage.save_message(ws_id, "assistant", "Answer.", provider_data=provider_data)
|
||||
# No live session — handler falls back to workstream_config + registry.
|
||||
mock_mgr = MagicMock()
|
||||
mock_mgr.get.return_value = None
|
||||
|
||||
# Build the app with a global registry that reports persist=False
|
||||
# for the saved alias.
|
||||
cfg = _interactive_endpoint_cfg(mock_mgr)
|
||||
handler = make_history_handler(cfg)
|
||||
app = Starlette(
|
||||
routes=[
|
||||
Mount(
|
||||
"/v1",
|
||||
routes=[
|
||||
Route(
|
||||
"/api/workstreams/{ws_id}/history",
|
||||
handler,
|
||||
methods=["GET"],
|
||||
),
|
||||
],
|
||||
),
|
||||
],
|
||||
middleware=[Middleware(_InjectAuthMiddleware)],
|
||||
)
|
||||
app.state.workstreams = mock_mgr
|
||||
app.state.auth_storage = _inject_storage
|
||||
app.state.registry = SimpleNamespace(
|
||||
get_config=lambda alias: SimpleNamespace(
|
||||
surface_persisted_reasoning=(alias != "claude-opus-4-7"),
|
||||
)
|
||||
)
|
||||
client = TestClient(app)
|
||||
|
||||
r = client.get(f"/v1/api/workstreams/{ws_id}/history")
|
||||
assert r.status_code == 200
|
||||
# Flag-flip on the saved alias is honored: reasoning suppressed.
|
||||
for m in r.json()["messages"]:
|
||||
if m.get("role") == "assistant":
|
||||
assert "reasoning" not in m
|
||||
assert "_provider_content" not in m
|
||||
|
||||
def test_history_handler_cold_workstream_no_alias_defaults_true(self, _inject_storage):
|
||||
"""A workstream that pre-dates the rehydrate-time alias persist
|
||||
(or one that simply has no workstream_config row) falls through
|
||||
to the conservative default ``True``. Reasoning surfaces.
|
||||
"""
|
||||
ws_id = "ws-reason-cold-no-alias"
|
||||
_inject_storage.register_workstream(ws_id, kind="interactive", user_id="test-user")
|
||||
provider_data = json.dumps(
|
||||
[{"type": "thinking", "thinking": "default-true wins", "signature": "s"}]
|
||||
)
|
||||
_inject_storage.save_message(ws_id, "assistant", "Answer.", provider_data=provider_data)
|
||||
mock_mgr = MagicMock()
|
||||
mock_mgr.get.return_value = None
|
||||
client = _build_history_app(mock_mgr, _inject_storage)
|
||||
|
||||
r = client.get(f"/v1/api/workstreams/{ws_id}/history")
|
||||
assert r.status_code == 200
|
||||
assistant = next(m for m in r.json()["messages"] if m.get("role") == "assistant")
|
||||
assert assistant["reasoning"] == "default-true wins"
|
||||
|
||||
@@ -1,3 +1,3 @@
|
||||
"""turnstone - Multi-node AI orchestration platform with tool use, agent routing, and cluster simulation."""
|
||||
|
||||
__version__ = "1.5.10"
|
||||
__version__ = "1.5.11"
|
||||
|
||||
@@ -908,6 +908,8 @@ class ModelDefinitionInfo(BaseModel):
|
||||
temperature: float | None = None
|
||||
max_tokens: int | None = None
|
||||
reasoning_effort: str | None = None
|
||||
surface_persisted_reasoning: bool = True
|
||||
replay_reasoning_to_model: bool = False
|
||||
source: str = ""
|
||||
created_by: str = ""
|
||||
created: str = ""
|
||||
@@ -926,6 +928,8 @@ class CreateModelDefinitionRequest(BaseModel):
|
||||
temperature: float | None = None
|
||||
max_tokens: int | None = None
|
||||
reasoning_effort: str | None = None
|
||||
surface_persisted_reasoning: bool = True
|
||||
replay_reasoning_to_model: bool = False
|
||||
|
||||
|
||||
class UpdateModelDefinitionRequest(BaseModel):
|
||||
@@ -940,6 +944,8 @@ class UpdateModelDefinitionRequest(BaseModel):
|
||||
temperature: float | None = None
|
||||
max_tokens: int | None = None
|
||||
reasoning_effort: str | None = None
|
||||
surface_persisted_reasoning: bool | None = None
|
||||
replay_reasoning_to_model: bool | None = None
|
||||
|
||||
|
||||
class ListModelDefinitionsResponse(BaseModel):
|
||||
@@ -990,6 +996,8 @@ class ListAvailableModelsResponse(BaseModel):
|
||||
models: list[AvailableModelInfo] = Field(default_factory=list)
|
||||
default_alias: str = ""
|
||||
channel_default_alias: str = ""
|
||||
coordinator_default_alias: str = ""
|
||||
judge_default_alias: str = ""
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
@@ -110,6 +110,14 @@ class TerminalUI(SessionUI):
|
||||
self.auto_approve = False
|
||||
self.auto_approve_tools: set[str] = set()
|
||||
|
||||
def on_turn_start(self) -> None:
|
||||
# Terminal UI has no inflight buffer to reset.
|
||||
pass
|
||||
|
||||
def on_turn_committed(self) -> None:
|
||||
# Terminal UI has no inflight buffer to reset.
|
||||
pass
|
||||
|
||||
def on_thinking_start(self) -> None:
|
||||
self.spinner = Spinner("Thinking")
|
||||
self.spinner.start()
|
||||
|
||||
@@ -0,0 +1,88 @@
|
||||
"""Coordinator alias resolution shared by the placeholder API and the
|
||||
session factory.
|
||||
|
||||
Both ``/v1/api/models`` (advertises the resolved default to the home
|
||||
composer) and ``console/session_factory.py:factory`` (resolves the
|
||||
alias new coordinator sessions launch on) walk the same three-tier
|
||||
chain. Centralising it here means the tier names and the tier-2
|
||||
validation policy live once — the prior arrangement was two
|
||||
implementations coupled by a "keep these in sync" comment, which is
|
||||
exactly the drift trap that produced the historical bug where the
|
||||
home composer advertised one alias while sessions ran on another.
|
||||
|
||||
Tiers, in priority order:
|
||||
|
||||
1. **Explicit pin** — per-call ``model_alias`` arg (factory only) or
|
||||
the ``coordinator.model_alias`` ConfigStore setting.
|
||||
2. **System default** — ``model.default_alias`` ConfigStore setting,
|
||||
admin-managed in the Models tab. Validated against
|
||||
``registry.has_alias()`` — a stale or typo'd value falls through
|
||||
with a logged warning rather than 503ing.
|
||||
3. **Registry default** — ``registry.default`` (config.toml
|
||||
``[model].default``), guaranteed by the registry to resolve.
|
||||
|
||||
Tier 1 is intentionally passed through unvalidated by default: an
|
||||
explicit operator pin should surface as 503 at ``registry.resolve``
|
||||
when stale, not silently fall through to a different alias. Callers
|
||||
that need stricter filtering (the placeholder API restricts to
|
||||
enabled DB rows so the home composer doesn't advertise a model the
|
||||
workstream picker can't offer) supply an ``alias_filter`` predicate
|
||||
applied to every tier.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
from turnstone.core.log import get_logger
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from collections.abc import Callable
|
||||
|
||||
from turnstone.core.config_store import ConfigStore
|
||||
from turnstone.core.model_registry import ModelRegistry
|
||||
|
||||
log = get_logger(__name__)
|
||||
|
||||
|
||||
def resolve_coordinator_alias(
|
||||
*,
|
||||
explicit: str | None,
|
||||
config_store: ConfigStore,
|
||||
registry: ModelRegistry,
|
||||
alias_filter: Callable[[str], bool] | None = None,
|
||||
) -> str:
|
||||
"""Resolve the effective coordinator alias through the three tiers.
|
||||
|
||||
See module docstring for the full chain. Returns the concrete
|
||||
alias name, or ``""`` when every tier failed (rare — only when
|
||||
``registry.default`` itself fails the filter).
|
||||
"""
|
||||
|
||||
def _accept(alias: str) -> bool:
|
||||
if not alias:
|
||||
return False
|
||||
return alias_filter(alias) if alias_filter is not None else True
|
||||
|
||||
explicit_alias = (explicit or "").strip()
|
||||
if not explicit_alias:
|
||||
explicit_alias = (config_store.get("coordinator.model_alias") or "").strip()
|
||||
if _accept(explicit_alias):
|
||||
return explicit_alias
|
||||
|
||||
fallback_alias = (config_store.get("model.default_alias") or "").strip()
|
||||
if fallback_alias and not registry.has_alias(fallback_alias):
|
||||
log.warning(
|
||||
"coord_alias.model_default_alias_unknown alias=%r "
|
||||
"— falling through to registry.default",
|
||||
fallback_alias,
|
||||
)
|
||||
fallback_alias = ""
|
||||
if _accept(fallback_alias):
|
||||
return fallback_alias
|
||||
|
||||
registry_default = registry.default or ""
|
||||
if _accept(registry_default):
|
||||
return registry_default
|
||||
|
||||
return ""
|
||||
@@ -41,6 +41,7 @@ from starlette.staticfiles import StaticFiles
|
||||
from turnstone.api.console_spec import build_console_spec
|
||||
from turnstone.api.docs import make_docs_handler, make_openapi_handler
|
||||
from turnstone.console.collector import ClusterCollector
|
||||
from turnstone.console.coordinator_alias import resolve_coordinator_alias
|
||||
from turnstone.console.coordinator_client import load_task_envelope
|
||||
from turnstone.console.metrics import ConsoleMetrics
|
||||
from turnstone.console.router import ConsoleRouter
|
||||
@@ -1671,20 +1672,55 @@ async def list_available_models(request: Request) -> JSONResponse:
|
||||
# Include effective defaults for clients (web UI, channel gateway).
|
||||
default_alias = ""
|
||||
channel_default_alias = ""
|
||||
coordinator_default_alias = ""
|
||||
judge_default_alias = ""
|
||||
cs = getattr(request.app.state, "config_store", None)
|
||||
if cs is not None:
|
||||
default_alias = cs.get("model.default_alias") or ""
|
||||
channel_default_alias = cs.get("channels.default_model_alias") or ""
|
||||
judge_default_alias = (cs.get("judge.model") or "").strip()
|
||||
enabled_aliases = {r["alias"] for r in rows}
|
||||
if default_alias and default_alias not in enabled_aliases:
|
||||
default_alias = ""
|
||||
if channel_default_alias and channel_default_alias not in enabled_aliases:
|
||||
channel_default_alias = ""
|
||||
# Coordinator default walks the standard three-tier chain (see
|
||||
# :func:`turnstone.console.coordinator_alias.resolve_coordinator_alias`).
|
||||
# The placeholder restricts every tier to enabled DB rows so the home
|
||||
# composer doesn't advertise a model the workstream-creation picker
|
||||
# can't actually offer — the session factory uses the same chain
|
||||
# without that filter so explicit operator pins surface as 503 at
|
||||
# ``registry.resolve`` instead of being silently swapped out.
|
||||
coord_registry = getattr(request.app.state, "coord_registry", None)
|
||||
if cs is not None and coord_registry is not None:
|
||||
coordinator_default_alias = resolve_coordinator_alias(
|
||||
explicit=cs.get("coordinator.model_alias"),
|
||||
config_store=cs,
|
||||
registry=coord_registry,
|
||||
alias_filter=lambda a: a in enabled_aliases,
|
||||
)
|
||||
elif coord_registry is not None:
|
||||
# ConfigStore failed lifespan but coord_registry is still bound —
|
||||
# fall through to ``registry.default`` (filtered) so the home
|
||||
# composer isn't blank. Mirrors the helper's tier 3 with the
|
||||
# placeholder's enabled-rows filter applied.
|
||||
registry_default = getattr(coord_registry, "default", "") or ""
|
||||
if registry_default in enabled_aliases:
|
||||
coordinator_default_alias = registry_default
|
||||
# Judge falls back to the resolved coordinator alias when
|
||||
# ``judge.model`` is empty *or* not a registered alias — judge.model
|
||||
# is alias-only (matches IntentJudge.__init__), so an unknown value is
|
||||
# operator misconfiguration that the judge itself silently inherits
|
||||
# the session model on.
|
||||
if not judge_default_alias or judge_default_alias not in enabled_aliases:
|
||||
judge_default_alias = coordinator_default_alias
|
||||
return JSONResponse(
|
||||
{
|
||||
"models": models,
|
||||
"default_alias": default_alias,
|
||||
"channel_default_alias": channel_default_alias,
|
||||
"coordinator_default_alias": coordinator_default_alias,
|
||||
"judge_default_alias": judge_default_alias,
|
||||
}
|
||||
)
|
||||
|
||||
@@ -7884,6 +7920,7 @@ _MODEL_AFFECTING_SETTING_KEYS: frozenset[str] = frozenset(
|
||||
"coordinator.model_alias",
|
||||
"coordinator.reasoning_effort",
|
||||
"judge.model",
|
||||
"channels.default_model_alias",
|
||||
}
|
||||
)
|
||||
|
||||
@@ -9904,6 +9941,9 @@ async def admin_create_model_definition(request: Request) -> JSONResponse:
|
||||
if not reasoning_effort:
|
||||
reasoning_effort = None
|
||||
|
||||
surface_persisted_reasoning = bool(body.get("surface_persisted_reasoning", True))
|
||||
replay_reasoning_to_model = bool(body.get("replay_reasoning_to_model", False))
|
||||
|
||||
storage.create_model_definition(
|
||||
definition_id=definition_id,
|
||||
alias=alias,
|
||||
@@ -9918,6 +9958,8 @@ async def admin_create_model_definition(request: Request) -> JSONResponse:
|
||||
temperature=temperature,
|
||||
max_tokens=max_tokens,
|
||||
reasoning_effort=reasoning_effort,
|
||||
surface_persisted_reasoning=surface_persisted_reasoning,
|
||||
replay_reasoning_to_model=replay_reasoning_to_model,
|
||||
)
|
||||
|
||||
record_audit(
|
||||
@@ -10079,6 +10121,10 @@ async def admin_update_model_definition(request: Request) -> JSONResponse:
|
||||
)
|
||||
else:
|
||||
updates["reasoning_effort"] = re_val
|
||||
if "surface_persisted_reasoning" in body:
|
||||
updates["surface_persisted_reasoning"] = bool(body["surface_persisted_reasoning"])
|
||||
if "replay_reasoning_to_model" in body:
|
||||
updates["replay_reasoning_to_model"] = bool(body["replay_reasoning_to_model"])
|
||||
|
||||
if updates:
|
||||
storage.update_model_definition(definition_id, **updates)
|
||||
|
||||
@@ -21,6 +21,7 @@ from __future__ import annotations
|
||||
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
from turnstone.console.coordinator_alias import resolve_coordinator_alias
|
||||
from turnstone.core.log import get_logger
|
||||
from turnstone.core.session import ChatSession
|
||||
from turnstone.core.workstream import WorkstreamKind
|
||||
@@ -96,18 +97,19 @@ def build_console_session_factory(
|
||||
f"console session factory only supports kind=COORDINATOR, got {kind!r}"
|
||||
)
|
||||
|
||||
# Resolve coordinator.model_alias from settings if caller didn't
|
||||
# override. Unset ``coordinator.model_alias`` falls back to the
|
||||
# model registry's default alias — operators get a working
|
||||
# coordinator on a freshly-provisioned console without an extra
|
||||
# manual setting. Resolve to the CONCRETE alias name
|
||||
# (``registry.default``) rather than passing None downstream:
|
||||
# ``ChatSession.__init__`` reads ``registry.get_provider(alias)``
|
||||
# to pick the right provider class, and passing None makes it
|
||||
# fall through to a generic OpenAI-compat provider — which
|
||||
# mismatches when the default is Anthropic/Google-backed.
|
||||
explicit_alias = model_alias or (config_store.get("coordinator.model_alias") or "").strip()
|
||||
effective_alias = explicit_alias or registry.default
|
||||
# Resolve to the CONCRETE alias name rather than passing None
|
||||
# downstream: ``ChatSession.__init__`` reads
|
||||
# ``registry.get_provider(alias)`` to pick the right provider
|
||||
# class, and passing None makes it fall through to a generic
|
||||
# OpenAI-compat provider — which mismatches when the default is
|
||||
# Anthropic/Google-backed. See
|
||||
# :func:`turnstone.console.coordinator_alias.resolve_coordinator_alias`
|
||||
# for the three-tier chain shared with the placeholder API.
|
||||
effective_alias = resolve_coordinator_alias(
|
||||
explicit=model_alias,
|
||||
config_store=config_store,
|
||||
registry=registry,
|
||||
)
|
||||
|
||||
r_client, r_model, r_cfg = registry.resolve(effective_alias)
|
||||
|
||||
|
||||
@@ -1142,12 +1142,54 @@ function _populateScheduleSelect(selectId, url, labelKey, valueKey, opts) {
|
||||
sel.appendChild(opt);
|
||||
});
|
||||
if (opts && opts.selected) sel.value = opts.selected;
|
||||
// Caller hook for placeholder annotation / other post-load tweaks.
|
||||
// Used by the schedule modals to rewrite the bare "Default model"
|
||||
// placeholder with the resolved alias so the label matches the
|
||||
// home composer (see app.js _populateHomeModelDropdowns).
|
||||
if (opts && typeof opts.afterPopulate === "function") {
|
||||
try {
|
||||
opts.afterPopulate(sel, data, items);
|
||||
} catch (_e) {
|
||||
/* hook errors must not break the dropdown */
|
||||
}
|
||||
}
|
||||
})
|
||||
.catch(function () {
|
||||
/* dropdown stays with placeholder or temporary option */
|
||||
});
|
||||
}
|
||||
|
||||
// Update the schedule-model placeholder option (first <option>) to
|
||||
// "Default — alias (model)" using /v1/api/models's resolved
|
||||
// default_alias, mirroring the home composer. Schedules don't carry
|
||||
// a coordinator/judge split, so they consume the workstream-creation
|
||||
// default rather than coordinator_default_alias / judge_default_alias.
|
||||
// Em-dash separator (rather than nested parens) keeps the alias's
|
||||
// "(model)" suffix legible.
|
||||
function _decorateScheduleModelPlaceholder(sel, data) {
|
||||
if (!sel || sel.options.length === 0) return;
|
||||
var alias = (data && data.default_alias) || "";
|
||||
if (!alias) return;
|
||||
var match = null;
|
||||
var models = (data && data.models) || [];
|
||||
for (var i = 0; i < models.length; i++) {
|
||||
if (models[i].alias === alias) {
|
||||
match = models[i];
|
||||
break;
|
||||
}
|
||||
}
|
||||
var label;
|
||||
if (match) {
|
||||
label =
|
||||
match.alias === match.model
|
||||
? match.alias
|
||||
: match.alias + " (" + match.model + ")";
|
||||
} else {
|
||||
label = alias;
|
||||
}
|
||||
sel.options[0].textContent = "Default — " + label;
|
||||
}
|
||||
|
||||
// Channel platforms shown in admin notify-target rows. Mirror server-side
|
||||
// channel adapters; expand here when a new adapter ships (Discord / Slack
|
||||
// today, MS Teams / etc. later).
|
||||
@@ -1333,6 +1375,7 @@ function showCreateScheduleModal() {
|
||||
display: function (m) {
|
||||
return m.alias === m.model ? m.alias : m.alias + " (" + m.model + ")";
|
||||
},
|
||||
afterPopulate: _decorateScheduleModelPlaceholder,
|
||||
});
|
||||
// Populate skill dropdown
|
||||
_populateScheduleSelect(
|
||||
@@ -1487,6 +1530,7 @@ function showEditScheduleModal(taskId) {
|
||||
display: function (m) {
|
||||
return m.alias === m.model ? m.alias : m.alias + " (" + m.model + ")";
|
||||
},
|
||||
afterPopulate: _decorateScheduleModelPlaceholder,
|
||||
});
|
||||
// Populate skill dropdown with current value pre-selected
|
||||
_populateScheduleSelect(
|
||||
@@ -2637,7 +2681,7 @@ function loadSettings() {
|
||||
|
||||
// Merge values + schema. Skip role-assignment settings owned by
|
||||
// the Models → Roles sub-tab (judge.* settings still live on the
|
||||
// Judge tab; the four model-tab roles render only there).
|
||||
// Judge tab; the model-tab roles render only there).
|
||||
var merged = {};
|
||||
var roleKeys = {
|
||||
"coordinator.model_alias": 1,
|
||||
@@ -2646,6 +2690,7 @@ function loadSettings() {
|
||||
"model.plan_effort": 1,
|
||||
"model.task_alias": 1,
|
||||
"model.task_effort": 1,
|
||||
"channels.default_model_alias": 1,
|
||||
};
|
||||
for (var j = 0; j < valuesArr.length; j++) {
|
||||
var v = valuesArr[j];
|
||||
@@ -4639,6 +4684,14 @@ var _modelCreateTrigger = null;
|
||||
// setting render a second selector inline. Adding a new role (e.g.
|
||||
// ``perception.audio.model``) is purely additive: drop a row here once
|
||||
// the SettingDef lands in turnstone/core/settings_registry.py.
|
||||
// ``fallbackKind`` controls how the empty/blank option in the alias
|
||||
// dropdown is labelled. Coordinator and Judge fall back to a single
|
||||
// well-defined alias (model.default_alias / coordinator alias) so we
|
||||
// surface that concrete model in the placeholder. Plan/Task agents
|
||||
// cascade through ``[model].plan_model → [model].agent_model →
|
||||
// session model`` per turnstone/core/settings_registry.py — there's
|
||||
// no single "default" to advertise, so the blank reads "(inherit)"
|
||||
// to match the vocabulary of the reasoning-effort dropdowns.
|
||||
var MODEL_ROLES = [
|
||||
{
|
||||
label: "Coordinator",
|
||||
@@ -4646,12 +4699,14 @@ var MODEL_ROLES = [
|
||||
"Console-hosted coordinator sessions that drive child workstreams.",
|
||||
aliasKey: "coordinator.model_alias",
|
||||
effortKey: "coordinator.reasoning_effort",
|
||||
fallbackKind: "default",
|
||||
},
|
||||
{
|
||||
label: "Judge",
|
||||
description:
|
||||
"Intent-validation judge that scores tool calls before approval.",
|
||||
aliasKey: "judge.model",
|
||||
fallbackKind: "default",
|
||||
},
|
||||
{
|
||||
label: "Plan agent",
|
||||
@@ -4659,6 +4714,7 @@ var MODEL_ROLES = [
|
||||
"plan_agent sub-agent — produces high-level plans before task dispatch.",
|
||||
aliasKey: "model.plan_alias",
|
||||
effortKey: "model.plan_effort",
|
||||
fallbackKind: "inherit",
|
||||
},
|
||||
{
|
||||
label: "Task agent",
|
||||
@@ -4666,6 +4722,14 @@ var MODEL_ROLES = [
|
||||
"task_agent sub-agent — runs autonomous subtasks dispatched by the parent.",
|
||||
aliasKey: "model.task_alias",
|
||||
effortKey: "model.task_effort",
|
||||
fallbackKind: "inherit",
|
||||
},
|
||||
{
|
||||
label: "Channel adapter",
|
||||
description:
|
||||
"Workstreams created by channel adapters (Discord, Slack) when no model is specified at creation time.",
|
||||
aliasKey: "channels.default_model_alias",
|
||||
fallbackKind: "default",
|
||||
},
|
||||
];
|
||||
|
||||
@@ -4848,11 +4912,40 @@ function _renderModelRoles(container, values, schema) {
|
||||
"aria-label",
|
||||
role.label + " model (empty = default)",
|
||||
);
|
||||
// Format the blank/inherit option in the same "alias (model)" shape
|
||||
// as the other rows so the dropdown reads consistently — without
|
||||
// this the empty row was bare "(default — flatspark)" while every
|
||||
// other row carried a "(/models/...)" suffix. Plan/Task agent
|
||||
// fall back through a multi-step chain (config.toml → agent_model
|
||||
// → session) that has no single concrete "default", so they get
|
||||
// a plain "(inherit)" instead of the misleading
|
||||
// "(default — <coordinator-alias>)".
|
||||
var blank = document.createElement("option");
|
||||
blank.value = "";
|
||||
blank.textContent = _modelDefaultAlias
|
||||
? "(default — " + _modelDefaultAlias + ")"
|
||||
: "(default)";
|
||||
if (role.fallbackKind === "inherit") {
|
||||
blank.textContent = "(inherit)";
|
||||
} else {
|
||||
var defaultDef = null;
|
||||
if (_modelDefaultAlias) {
|
||||
for (var dm = 0; dm < enabledAliases.length; dm++) {
|
||||
if (enabledAliases[dm].alias === _modelDefaultAlias) {
|
||||
defaultDef = enabledAliases[dm];
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
if (defaultDef) {
|
||||
var defLabel =
|
||||
defaultDef.alias === defaultDef.model
|
||||
? defaultDef.alias
|
||||
: defaultDef.alias + " (" + defaultDef.model + ")";
|
||||
blank.textContent = "(default — " + defLabel + ")";
|
||||
} else if (_modelDefaultAlias) {
|
||||
blank.textContent = "(default — " + _modelDefaultAlias + ")";
|
||||
} else {
|
||||
blank.textContent = "(default)";
|
||||
}
|
||||
}
|
||||
aliasSel.appendChild(blank);
|
||||
var currentAlias = aliasInfo.value || "";
|
||||
var matched = false;
|
||||
@@ -5008,6 +5101,11 @@ function _renderModels(items) {
|
||||
if (m.max_tokens != null) overrides.push("max_tok=" + m.max_tokens);
|
||||
if (m.reasoning_effort != null)
|
||||
overrides.push("effort=" + m.reasoning_effort);
|
||||
// Reasoning persistence flags surface only when non-default
|
||||
// (persist=False is the operator opt-out; replay=True is the
|
||||
// operator opt-in). Default values are silent.
|
||||
if (m.surface_persisted_reasoning === false) overrides.push("surface=off");
|
||||
if (m.replay_reasoning_to_model === true) overrides.push("replay=on");
|
||||
if (overrides.length) {
|
||||
var ovrSpan = document.createElement("span");
|
||||
ovrSpan.className = "model-overrides-hint";
|
||||
@@ -5196,6 +5294,8 @@ function showCreateModelModal() {
|
||||
el.style.borderColor = "";
|
||||
});
|
||||
document.getElementById("model-enabled").checked = true;
|
||||
document.getElementById("model-surface-persisted-reasoning").checked = true;
|
||||
document.getElementById("model-replay-reasoning").checked = false;
|
||||
document.getElementById("model-detect-result").style.display = "none";
|
||||
document.getElementById("model-detect-btn").disabled = false;
|
||||
document.getElementById("model-detect-btn").textContent = "Detect";
|
||||
@@ -5278,6 +5378,13 @@ function showEditModelModal(definitionId) {
|
||||
document.getElementById("model-capabilities").value =
|
||||
capsText === "{}" ? "" : capsText;
|
||||
document.getElementById("model-enabled").checked = m.enabled !== false;
|
||||
// Reasoning persistence flags — defaults match the dataclass
|
||||
// defaults (persist=true, replay=false) when the API returns
|
||||
// them as undefined (legacy / pre-052 row).
|
||||
document.getElementById("model-surface-persisted-reasoning").checked =
|
||||
m.surface_persisted_reasoning !== false;
|
||||
document.getElementById("model-replay-reasoning").checked =
|
||||
m.replay_reasoning_to_model === true;
|
||||
_applyProviderDefaults();
|
||||
})
|
||||
.catch(function () {
|
||||
@@ -5419,6 +5526,16 @@ function submitCreateModel() {
|
||||
form.reasoning_effort = null;
|
||||
}
|
||||
|
||||
// Reasoning persistence flags — always serialize so a flip from
|
||||
// default takes effect on PUT (the server's update path keys off
|
||||
// "field present in body").
|
||||
form.surface_persisted_reasoning = document.getElementById(
|
||||
"model-surface-persisted-reasoning",
|
||||
).checked;
|
||||
form.replay_reasoning_to_model = document.getElementById(
|
||||
"model-replay-reasoning",
|
||||
).checked;
|
||||
|
||||
var apiKey = document.getElementById("model-api-key").value;
|
||||
if (apiKey) form.api_key = apiKey;
|
||||
|
||||
|
||||
@@ -1761,11 +1761,11 @@ function _mountHomeCoordComposer() {
|
||||
id: "judge_model",
|
||||
label: "Judge Model",
|
||||
type: "select",
|
||||
// Neutral label — the actual default is ConfigStore
|
||||
// ``judge.model`` when set, IntentJudge's agent-model
|
||||
// fallback when not. "Default judge model" doesn't
|
||||
// mislead either way.
|
||||
choices: [{ value: "", text: "Default judge model" }],
|
||||
// Initial placeholder; _populateHomeModelDropdowns rewrites this
|
||||
// to "Default model (<alias>)" once /v1/api/models reports the
|
||||
// resolved judge alias (judge.model when set, otherwise the
|
||||
// session model — see IntentJudge.__init__).
|
||||
choices: [{ value: "", text: "Default model" }],
|
||||
},
|
||||
],
|
||||
},
|
||||
@@ -1801,6 +1801,21 @@ function _populateHomeSkillDropdown() {
|
||||
});
|
||||
}
|
||||
|
||||
// Format a resolved alias with its model suffix the same way as the
|
||||
// dropdown rows ("alias (model)", or just "alias" when they coincide).
|
||||
// Returns "" when alias is empty or unknown so callers can fall back
|
||||
// to a neutral placeholder.
|
||||
function _resolveModelLabel(alias, models) {
|
||||
if (!alias) return "";
|
||||
for (var i = 0; i < (models || []).length; i++) {
|
||||
var m = models[i];
|
||||
if (m.alias === alias) {
|
||||
return m.alias === m.model ? m.alias : m.alias + " (" + m.model + ")";
|
||||
}
|
||||
}
|
||||
return "";
|
||||
}
|
||||
|
||||
// Populate Model + Judge Model dropdowns from /v1/api/models — same
|
||||
// list the interactive new-ws modal uses. Empty/default option stays
|
||||
// at the top so submitting without a choice falls back to the
|
||||
@@ -1819,6 +1834,30 @@ function _populateHomeModelDropdowns() {
|
||||
});
|
||||
_homeCoordComposer.setOptionChoices("model", choices);
|
||||
_homeCoordComposer.setOptionChoices("judge_model", choices);
|
||||
// Both placeholders use the same "Default — alias (model)"
|
||||
// template — the field-row labels (MODEL / JUDGE MODEL) already
|
||||
// carry the role context, so an asymmetric "Default judge model"
|
||||
// reads awkwardly alongside the plain "Default model" line above
|
||||
// it. Em-dash separator (rather than nested parens) keeps the
|
||||
// alias's "(model)" suffix legible and matches the
|
||||
// ``(default — alias (model))`` pattern used in the admin Roles
|
||||
// tab.
|
||||
var coordDefault = _resolveModelLabel(
|
||||
data.coordinator_default_alias || "",
|
||||
data.models || [],
|
||||
);
|
||||
var judgeDefault = _resolveModelLabel(
|
||||
data.judge_default_alias || "",
|
||||
data.models || [],
|
||||
);
|
||||
_homeCoordComposer.setOptionPlaceholder(
|
||||
"model",
|
||||
coordDefault ? "Default — " + coordDefault : "Default model",
|
||||
);
|
||||
_homeCoordComposer.setOptionPlaceholder(
|
||||
"judge_model",
|
||||
judgeDefault ? "Default — " + judgeDefault : "Default model",
|
||||
);
|
||||
})
|
||||
.catch(function () {
|
||||
/* defaults still work even without the dropdown populated */
|
||||
|
||||
@@ -579,6 +579,13 @@
|
||||
if (callId && toolRows.has(callId)) {
|
||||
const entry = toolRows.get(callId);
|
||||
_appendResultToRow(entry.row, output, isError, opts);
|
||||
// The batch may have been --running (live tool_info auto path,
|
||||
// approval_resolved approved path, or replay-time orphan).
|
||||
// Drop --running once every row in the batch has a result so
|
||||
// the kicker text + visual style flip back to the post-execution
|
||||
// state. Per-row check (not a counter) keeps the logic
|
||||
// resilient to out-of-order replay + late SSE deliveries.
|
||||
_unsetBatchRunningIfAllResults(entry.batch);
|
||||
// Result blocks grow scrollHeight; without this the user pinned
|
||||
// at the bottom loses their pin when the row inflates. appendMsg
|
||||
// already routes through _scheduleScroll on the legacy path; this
|
||||
@@ -1166,6 +1173,37 @@
|
||||
return status;
|
||||
}
|
||||
|
||||
function _setBatchRunning(batch) {
|
||||
if (!batch) return;
|
||||
batch.classList.add("coord-tool-batch--running");
|
||||
const kicker = batch.querySelector(".coord-tool-batch-kicker");
|
||||
if (kicker) {
|
||||
const rowCount = batch.querySelectorAll(".coord-tool-row").length;
|
||||
kicker.textContent =
|
||||
rowCount >= 2 ? "Running · Parallel " + rowCount : "Running";
|
||||
}
|
||||
}
|
||||
|
||||
function _unsetBatchRunningIfAllResults(batch) {
|
||||
// Remove ``--running`` once every row in the batch has rendered a
|
||||
// result block. Caller invokes after each tool_result; the test
|
||||
// is "did THIS result complete the batch?" — cheap DOM walk over
|
||||
// the same handful of rows we already track.
|
||||
if (!batch) return;
|
||||
if (!batch.classList.contains("coord-tool-batch--running")) return;
|
||||
const rows = batch.querySelectorAll(".coord-tool-row");
|
||||
for (const row of rows) {
|
||||
if (!row.querySelector(".coord-tool-row-result")) return;
|
||||
}
|
||||
batch.classList.remove("coord-tool-batch--running");
|
||||
const kicker = batch.querySelector(".coord-tool-batch-kicker");
|
||||
if (kicker && !batch.classList.contains("coord-tool-batch--pending")) {
|
||||
const rowCount = rows.length;
|
||||
kicker.textContent =
|
||||
rowCount >= 2 ? "Parallel · " + rowCount + " tools" : "Tool";
|
||||
}
|
||||
}
|
||||
|
||||
function _morphBatchResolved(batch, opts) {
|
||||
if (!batch) return;
|
||||
batch.classList.remove("coord-tool-batch--pending");
|
||||
@@ -1283,9 +1321,15 @@
|
||||
_announceAssertive(_approvalAriaLabel(items));
|
||||
} else if (
|
||||
opts.auto &&
|
||||
existing.classList.contains("coord-tool-batch--running")
|
||||
existing.classList.contains("coord-tool-batch--running") &&
|
||||
!existing.classList.contains("coord-tool-batch--auto")
|
||||
) {
|
||||
existing.classList.remove("coord-tool-batch--running");
|
||||
// SSE tool_info clarifies an existing --running batch as
|
||||
// auto-approved. Keep --running (the tool is still in
|
||||
// flight; tool_result will remove it) and add --auto so the
|
||||
// batch reflects BOTH "auto-approved" + "running" — historical
|
||||
// behaviour swapped --running out, which lost the running
|
||||
// indicator the moment tool_info clarified the approval state.
|
||||
existing.classList.add("coord-tool-batch--auto");
|
||||
} else if (opts.pending) {
|
||||
// Already pending — keep the action row, just refresh
|
||||
@@ -1328,7 +1372,6 @@
|
||||
);
|
||||
if (opts.pending) batch.classList.add("coord-tool-batch--pending");
|
||||
else if (opts.auto) batch.classList.add("coord-tool-batch--auto");
|
||||
else if (opts.running) batch.classList.add("coord-tool-batch--running");
|
||||
else if (opts.resolved) {
|
||||
batch.classList.add(
|
||||
opts.resolved.approved
|
||||
@@ -1336,6 +1379,12 @@
|
||||
: "coord-tool-batch--denied",
|
||||
);
|
||||
}
|
||||
// ``running`` is additive — coexists with ``auto`` (auto-approved
|
||||
// and currently in flight) or stands alone (replay-time orphan
|
||||
// before SSE clarifies the approval state). Removed by
|
||||
// :func:`_unsetBatchRunningIfAllResults` when every row in the
|
||||
// batch has a tool_result.
|
||||
if (opts.running) batch.classList.add("coord-tool-batch--running");
|
||||
|
||||
const head = document.createElement("div");
|
||||
head.className = "coord-tool-batch-head";
|
||||
@@ -1960,6 +2009,47 @@
|
||||
case "reasoning":
|
||||
appendReasoningToken(ev.text || "");
|
||||
break;
|
||||
case "in_progress_snapshot":
|
||||
// One-shot replay of the in-progress turn's reasoning + content
|
||||
// when this client connects mid-stream (page refresh while the
|
||||
// model is generating). Idempotent on EventSource auto-reconnect:
|
||||
// skip overwrite when the current buffer is already at-or-past
|
||||
// the snapshot length, so a stale replay can't reset the live-
|
||||
// streamed view back to a shorter prefix.
|
||||
if (ev.reasoning && ev.reasoning.length > currentReasoningBuf.length) {
|
||||
if (!currentReasoningEl) {
|
||||
currentReasoningEl = appendMsg("reasoning", "", {
|
||||
label: "reasoning",
|
||||
});
|
||||
messagesEl.setAttribute("aria-live", "off");
|
||||
}
|
||||
currentReasoningBuf = ev.reasoning;
|
||||
var rbody = currentReasoningEl.querySelector(".msg-body");
|
||||
if (rbody) rbody.textContent = currentReasoningBuf;
|
||||
_scheduleScroll();
|
||||
}
|
||||
if (ev.content && ev.content.length > currentAssistantBuf.length) {
|
||||
if (!currentAssistantEl) {
|
||||
currentAssistantEl = appendMsg("assistant", "", {
|
||||
label: "assistant",
|
||||
});
|
||||
messagesEl.setAttribute("aria-live", "off");
|
||||
}
|
||||
currentAssistantBuf = ev.content;
|
||||
var abody = currentAssistantEl.querySelector(".msg-body");
|
||||
if (abody && typeof streamingRender === "function") {
|
||||
try {
|
||||
streamingRender(abody, currentAssistantBuf);
|
||||
} catch (e) {
|
||||
console.warn("coordinator streamingRender failed", e);
|
||||
abody.textContent = currentAssistantBuf;
|
||||
}
|
||||
} else if (abody) {
|
||||
abody.textContent = currentAssistantBuf;
|
||||
}
|
||||
_scheduleScroll();
|
||||
}
|
||||
break;
|
||||
case "stream_end":
|
||||
finishAssistantStream();
|
||||
break;
|
||||
@@ -2007,11 +2097,19 @@
|
||||
const wasAlways =
|
||||
ev.always === true ||
|
||||
(ev.always === undefined && target.dataset.requestedAlways === "1");
|
||||
const approved = ev.approved !== false;
|
||||
_morphBatchResolved(target, {
|
||||
approved: ev.approved !== false,
|
||||
approved,
|
||||
always: wasAlways,
|
||||
feedback: ev.feedback || null,
|
||||
});
|
||||
// Approved batches start running the moment the user clicks
|
||||
// approve — mirror the auto path so the live RUNNING
|
||||
// indicator shows during execution (not just on refresh).
|
||||
// Denied batches don't run at all, so no --running.
|
||||
if (approved) {
|
||||
_setBatchRunning(target);
|
||||
}
|
||||
}
|
||||
break;
|
||||
}
|
||||
@@ -2202,8 +2300,12 @@
|
||||
// for both kinds, matching the interactive payload name. All
|
||||
// items in a single ``tool_info`` envelope share a dispatch
|
||||
// turn, so render them as one batch construct (parallel when
|
||||
// ≥2, solo otherwise) rather than N separate bubbles.
|
||||
appendToolBatch(ev.items || [], { auto: true });
|
||||
// ≥2, solo otherwise) rather than N separate bubbles. ``auto``
|
||||
// marks the approval-state class; ``running`` marks "in flight"
|
||||
// — the tool starts executing the moment auto-approval lands,
|
||||
// and the batch should show the same RUNNING indicator the
|
||||
// replay path renders for an unresolved committed turn.
|
||||
appendToolBatch(ev.items || [], { auto: true, running: true });
|
||||
break;
|
||||
// Child-workstream fan-out routed through the coordinator's own
|
||||
// SSE stream. CoordinatorManager filters the cluster event bus
|
||||
@@ -4093,6 +4195,20 @@
|
||||
appendUserMessageWithAttachments(text, [], { label: "user" });
|
||||
});
|
||||
} else if (role === "assistant") {
|
||||
// Reasoning bubble (Phase 1 reasoning persistence) — render
|
||||
// BEFORE the content card so the visual order matches the
|
||||
// live SSE flow (reasoning_delta arrives before content_delta
|
||||
// for thinking-enabled models). Mirrors the live ":1524" /
|
||||
// snapshot ":2021" call sites — same appendMsg("reasoning")
|
||||
// helper, just driven from history-render rather than the
|
||||
// SSE handler. Only present when the active model's
|
||||
// surface_persisted_reasoning flag is true and the message round-tripped
|
||||
// a thinking lane.
|
||||
if (typeof m.reasoning === "string" && m.reasoning.length) {
|
||||
const rEl = appendMsg("reasoning", "", { label: "reasoning" });
|
||||
const rBody = rEl && rEl.querySelector(".msg-body");
|
||||
if (rBody) rBody.textContent = m.reasoning;
|
||||
}
|
||||
// Render content BEFORE the tool batch so DOM order matches
|
||||
// chronological order (the model emits text first, then
|
||||
// dispatches tools). Whitespace-only content (e.g. "\n\n"
|
||||
|
||||
@@ -143,46 +143,91 @@ function _renderGovRoles(items) {
|
||||
}
|
||||
}
|
||||
|
||||
// All permission names for the checkbox UI
|
||||
var _ALL_PERMISSIONS = [
|
||||
"read",
|
||||
"write",
|
||||
"approve",
|
||||
"admin.users",
|
||||
"admin.roles",
|
||||
"admin.orgs",
|
||||
"admin.policies",
|
||||
"admin.skills",
|
||||
"admin.audit",
|
||||
"admin.usage",
|
||||
"admin.schedules",
|
||||
"admin.watches",
|
||||
"admin.judge",
|
||||
"admin.memories",
|
||||
"admin.settings",
|
||||
"admin.mcp",
|
||||
"tools.approve",
|
||||
"workstreams.create",
|
||||
"workstreams.close",
|
||||
// Permission inventory grouped by namespace so the role modal can
|
||||
// render each section under its own heading. Sectioning prevents the
|
||||
// row-flow grid from slicing a namespace mid-column (e.g. half of
|
||||
// ``admin.*`` ending up in column 1, the rest in column 2) and lets
|
||||
// readers who don't yet know the permission taxonomy scan by
|
||||
// concept. Each section's permissions render as a 2-column grid;
|
||||
// the ``Scopes`` and ``Workstreams & Tools`` sections are short
|
||||
// enough to fit one row, ``Admin`` carries the bulk.
|
||||
var _PERMISSION_SECTIONS = [
|
||||
{
|
||||
label: "Scopes",
|
||||
permissions: ["read", "write", "approve"],
|
||||
},
|
||||
{
|
||||
label: "Admin",
|
||||
permissions: [
|
||||
"admin.users",
|
||||
"admin.roles",
|
||||
"admin.orgs",
|
||||
"admin.policies",
|
||||
"admin.skills",
|
||||
"admin.audit",
|
||||
"admin.usage",
|
||||
"admin.schedules",
|
||||
"admin.watches",
|
||||
"admin.judge",
|
||||
"admin.memories",
|
||||
"admin.settings",
|
||||
"admin.mcp",
|
||||
],
|
||||
},
|
||||
{
|
||||
label: "Workstreams & Tools",
|
||||
permissions: ["workstreams.create", "workstreams.close", "tools.approve"],
|
||||
},
|
||||
];
|
||||
|
||||
function _buildPermCheckboxes(prefix, selected) {
|
||||
var html = '<div class="perm-grid">';
|
||||
for (var i = 0; i < _ALL_PERMISSIONS.length; i++) {
|
||||
var p = _ALL_PERMISSIONS[i];
|
||||
var checked = selected && selected.indexOf(p) >= 0 ? " checked" : "";
|
||||
html +=
|
||||
'<label class="perm-checkbox"><input type="checkbox" value="' +
|
||||
p +
|
||||
'" name="' +
|
||||
prefix +
|
||||
'-perm"' +
|
||||
checked +
|
||||
"> " +
|
||||
escapeHtml(p) +
|
||||
"</label>";
|
||||
// Flat list — kept for any caller that wants the full permission
|
||||
// inventory without caring about sectioning.
|
||||
var _ALL_PERMISSIONS = (function () {
|
||||
var flat = [];
|
||||
for (var i = 0; i < _PERMISSION_SECTIONS.length; i++) {
|
||||
flat = flat.concat(_PERMISSION_SECTIONS[i].permissions);
|
||||
}
|
||||
return flat;
|
||||
})();
|
||||
|
||||
function _buildPermCheckboxes(prefix, selected) {
|
||||
// Emits the toggle-switch component used elsewhere in the admin
|
||||
// modals so each permission reads as a deliberate on/off rather
|
||||
// than a generic checkbox. Sections are wrapped in a
|
||||
// ``.perm-section`` block with a caps-styled heading so the
|
||||
// typographic system inside the role modal stays consistent (the
|
||||
// surrounding label cadence is also caps + 0.08em letter-spacing).
|
||||
// The underlying ``<input type="checkbox" name="{prefix}-perm">``
|
||||
// shape is preserved so ``_collectPermCheckboxes`` still picks
|
||||
// them up regardless of section.
|
||||
var html = "";
|
||||
for (var s = 0; s < _PERMISSION_SECTIONS.length; s++) {
|
||||
var section = _PERMISSION_SECTIONS[s];
|
||||
html +=
|
||||
'<div class="perm-section">' +
|
||||
'<div class="perm-section-label">' +
|
||||
escapeHtml(section.label) +
|
||||
"</div>" +
|
||||
'<div class="perm-grid">';
|
||||
for (var i = 0; i < section.permissions.length; i++) {
|
||||
var p = section.permissions[i];
|
||||
var checked = selected && selected.indexOf(p) >= 0 ? " checked" : "";
|
||||
html +=
|
||||
'<label class="toggle-switch perm-toggle">' +
|
||||
'<input type="checkbox" value="' +
|
||||
p +
|
||||
'" name="' +
|
||||
prefix +
|
||||
'-perm"' +
|
||||
checked +
|
||||
">" +
|
||||
'<span class="toggle-track" aria-hidden="true"></span>' +
|
||||
'<span class="toggle-label">' +
|
||||
escapeHtml(p) +
|
||||
"</span></label>";
|
||||
}
|
||||
html += "</div></div>";
|
||||
}
|
||||
html += "</div>";
|
||||
return html;
|
||||
}
|
||||
|
||||
@@ -346,20 +391,27 @@ function showUserRolesModal(userId) {
|
||||
var assigned = {};
|
||||
for (var i = 0; i < userRoles.length; i++)
|
||||
assigned[userRoles[i].role_id] = true;
|
||||
var html = "";
|
||||
// Role-assignment rows reuse the toggle-switch component for
|
||||
// consistency with the rest of the admin UX. Role display names
|
||||
// are human-readable text, so no monospace override is needed.
|
||||
var html = '<div class="user-roles-list">';
|
||||
for (var j = 0; j < allRoles.length; j++) {
|
||||
var r = allRoles[j];
|
||||
var checked = assigned[r.role_id] ? " checked" : "";
|
||||
html +=
|
||||
'<label class="perm-checkbox"><input type="checkbox" value="' +
|
||||
'<label class="toggle-switch user-role-toggle">' +
|
||||
'<input type="checkbox" value="' +
|
||||
escapeHtml(r.role_id) +
|
||||
'" name="ur-role"' +
|
||||
checked +
|
||||
"> " +
|
||||
">" +
|
||||
'<span class="toggle-track" aria-hidden="true"></span>' +
|
||||
'<span class="toggle-label">' +
|
||||
escapeHtml(r.display_name) +
|
||||
"</label>";
|
||||
"</span></label>";
|
||||
}
|
||||
container.innerHTML = html;
|
||||
html += "</div>";
|
||||
container.innerHTML = html; // values escaped via escapeHtml above
|
||||
})
|
||||
.catch(function () {
|
||||
container.innerHTML =
|
||||
@@ -3199,18 +3251,27 @@ function renderJudgeSettings() {
|
||||
var isDefault = s.source === "default";
|
||||
|
||||
if (s.type === "bool") {
|
||||
// Toggle switch — same component used by the admin modals.
|
||||
// ``onchange`` reads the box's new ``.checked`` and writes via
|
||||
// saveJudgeSetting. The ``.toggle-label`` is a static "Enabled"
|
||||
// because the slider position is the truth — flipping the
|
||||
// caption text on save round-tripped through reload, so the
|
||||
// slider moved instantly while the caption lagged 50-300ms and
|
||||
// looked broken. ``aria-label`` carries the setting name so
|
||||
// screen readers get the row context inline.
|
||||
inputHtml =
|
||||
'<label class="toggle-label" style="display:flex;align-items:center;gap:8px;cursor:pointer">' +
|
||||
'<label class="toggle-switch toggle--flush">' +
|
||||
'<input type="checkbox" data-key="' +
|
||||
s.key +
|
||||
'" aria-label="' +
|
||||
escapeHtml(shortKey) +
|
||||
'" ' +
|
||||
(currentVal ? "checked" : "") +
|
||||
" onchange=\"saveJudgeSetting('" +
|
||||
s.key +
|
||||
'\',this.checked)" style="width:16px;height:16px">' +
|
||||
'<span style="font-size:12px">' +
|
||||
(currentVal ? "Enabled" : "Disabled") +
|
||||
"</span></label>";
|
||||
"',this.checked)\">" +
|
||||
'<span class="toggle-track" aria-hidden="true"></span>' +
|
||||
'<span class="toggle-label">Enabled</span></label>';
|
||||
} else if (s.type === "float") {
|
||||
inputHtml =
|
||||
'<div style="display:flex;gap:8px;align-items:center">' +
|
||||
|
||||
+136
-100
@@ -2507,10 +2507,11 @@
|
||||
rows="3"
|
||||
placeholder="What should the workstream do?"
|
||||
></textarea>
|
||||
<label class="admin-checkbox"
|
||||
><input id="cs-autoapprove" type="checkbox" /> Auto-approve tool
|
||||
calls</label
|
||||
>
|
||||
<label class="toggle-switch">
|
||||
<input id="cs-autoapprove" type="checkbox" />
|
||||
<span class="toggle-track" aria-hidden="true"></span>
|
||||
<span class="toggle-label">Auto-approve tool calls</span>
|
||||
</label>
|
||||
<label
|
||||
>Notify on completion
|
||||
<span class="label-hint">optional</span></label
|
||||
@@ -2609,13 +2610,16 @@
|
||||
</select>
|
||||
<label for="es-message">Initial message</label>
|
||||
<textarea id="es-message" rows="3"></textarea>
|
||||
<label class="admin-checkbox"
|
||||
><input id="es-autoapprove" type="checkbox" /> Auto-approve tool
|
||||
calls</label
|
||||
>
|
||||
<label class="admin-checkbox"
|
||||
><input id="es-enabled" type="checkbox" /> Enabled</label
|
||||
>
|
||||
<label class="toggle-switch">
|
||||
<input id="es-autoapprove" type="checkbox" />
|
||||
<span class="toggle-track" aria-hidden="true"></span>
|
||||
<span class="toggle-label">Auto-approve tool calls</span>
|
||||
</label>
|
||||
<label class="toggle-switch">
|
||||
<input id="es-enabled" type="checkbox" />
|
||||
<span class="toggle-track" aria-hidden="true"></span>
|
||||
<span class="toggle-label">Enabled</span>
|
||||
</label>
|
||||
<label
|
||||
>Notify on completion
|
||||
<span class="label-hint">optional</span></label
|
||||
@@ -2859,9 +2863,11 @@
|
||||
</select>
|
||||
<label for="ep-priority">Priority</label>
|
||||
<input id="ep-priority" type="number" value="0" min="0" max="9999" />
|
||||
<label class="admin-checkbox"
|
||||
><input id="ep-enabled" type="checkbox" checked /> Enabled</label
|
||||
>
|
||||
<label class="toggle-switch">
|
||||
<input id="ep-enabled" type="checkbox" checked />
|
||||
<span class="toggle-track" aria-hidden="true"></span>
|
||||
<span class="toggle-label">Enabled</span>
|
||||
</label>
|
||||
<div class="modal-buttons">
|
||||
<button class="modal-cancel" onclick="hideEditPolicyModal()">
|
||||
Cancel
|
||||
@@ -2963,9 +2969,11 @@
|
||||
<textarea id="epp-content" rows="10" spellcheck="false"></textarea>
|
||||
<label for="epp-priority">Priority</label>
|
||||
<input id="epp-priority" type="number" value="0" min="0" max="9999" />
|
||||
<label class="admin-checkbox"
|
||||
><input id="epp-enabled" type="checkbox" checked /> Enabled</label
|
||||
>
|
||||
<label class="toggle-switch">
|
||||
<input id="epp-enabled" type="checkbox" checked />
|
||||
<span class="toggle-track" aria-hidden="true"></span>
|
||||
<span class="toggle-label">Enabled</span>
|
||||
</label>
|
||||
<div class="modal-buttons">
|
||||
<button class="modal-cancel" onclick="hideEditPromptPolicyModal()">
|
||||
Cancel
|
||||
@@ -3083,10 +3091,11 @@
|
||||
</option>
|
||||
<option value="search">Search — BM25 discoverable</option>
|
||||
</select>
|
||||
<label class="admin-checkbox"
|
||||
><input id="ctm-default" type="checkbox" /> Apply to new
|
||||
workstreams by default</label
|
||||
>
|
||||
<label class="toggle-switch">
|
||||
<input id="ctm-default" type="checkbox" />
|
||||
<span class="toggle-track" aria-hidden="true"></span>
|
||||
<span class="toggle-label">Apply to new workstreams by default</span>
|
||||
</label>
|
||||
</div>
|
||||
</div>
|
||||
<div class="skill-spec-col skill-spec-col-content">
|
||||
@@ -3189,10 +3198,11 @@
|
||||
/>
|
||||
</div>
|
||||
</div>
|
||||
<label class="admin-checkbox"
|
||||
><input id="csk-auto-approve" type="checkbox" /> Auto-approve all
|
||||
tools</label
|
||||
>
|
||||
<label class="toggle-switch">
|
||||
<input id="csk-auto-approve" type="checkbox" />
|
||||
<span class="toggle-track" aria-hidden="true"></span>
|
||||
<span class="toggle-label">Auto-approve all tools</span>
|
||||
</label>
|
||||
<label for="csk-allowed-tools"
|
||||
>Allowed Tools
|
||||
<span class="label-hint"
|
||||
@@ -3222,9 +3232,11 @@
|
||||
style="display: block; margin-top: 3px"
|
||||
>JSON array. Each: channel_type + channel_id or user_id</span
|
||||
>
|
||||
<label class="admin-checkbox"
|
||||
><input id="csk-enabled" type="checkbox" checked /> Enabled</label
|
||||
>
|
||||
<label class="toggle-switch">
|
||||
<input id="csk-enabled" type="checkbox" checked />
|
||||
<span class="toggle-track" aria-hidden="true"></span>
|
||||
<span class="toggle-label">Enabled</span>
|
||||
</label>
|
||||
</details>
|
||||
<details class="admin-details">
|
||||
<summary>
|
||||
@@ -3395,10 +3407,11 @@
|
||||
</option>
|
||||
<option value="search">Search — BM25 discoverable</option>
|
||||
</select>
|
||||
<label class="admin-checkbox"
|
||||
><input id="etm-default" type="checkbox" /> Apply to new
|
||||
workstreams by default</label
|
||||
>
|
||||
<label class="toggle-switch">
|
||||
<input id="etm-default" type="checkbox" />
|
||||
<span class="toggle-track" aria-hidden="true"></span>
|
||||
<span class="toggle-label">Apply to new workstreams by default</span>
|
||||
</label>
|
||||
</div>
|
||||
</div>
|
||||
<div class="skill-spec-col skill-spec-col-content">
|
||||
@@ -3480,10 +3493,11 @@
|
||||
/>
|
||||
</div>
|
||||
</div>
|
||||
<label class="admin-checkbox"
|
||||
><input id="esk-auto-approve" type="checkbox" /> Auto-approve all
|
||||
tools</label
|
||||
>
|
||||
<label class="toggle-switch">
|
||||
<input id="esk-auto-approve" type="checkbox" />
|
||||
<span class="toggle-track" aria-hidden="true"></span>
|
||||
<span class="toggle-label">Auto-approve all tools</span>
|
||||
</label>
|
||||
<label for="esk-allowed-tools"
|
||||
>Allowed Tools
|
||||
<span class="label-hint"
|
||||
@@ -3513,9 +3527,11 @@
|
||||
style="display: block; margin-top: 3px"
|
||||
>JSON array. Each: channel_type + channel_id or user_id</span
|
||||
>
|
||||
<label class="admin-checkbox"
|
||||
><input id="esk-enabled" type="checkbox" checked /> Enabled</label
|
||||
>
|
||||
<label class="toggle-switch">
|
||||
<input id="esk-enabled" type="checkbox" checked />
|
||||
<span class="toggle-track" aria-hidden="true"></span>
|
||||
<span class="toggle-label">Enabled</span>
|
||||
</label>
|
||||
</details>
|
||||
<div id="etm-scan-section" style="display: none" class="admin-field">
|
||||
<span class="admin-field-heading" id="etm-scan-heading"
|
||||
@@ -3699,43 +3715,48 @@
|
||||
<legend style="font-size: 12px; padding: 0 6px">
|
||||
Multitenant Authorization
|
||||
</legend>
|
||||
<label
|
||||
style="display: block; margin: 4px 0; font-weight: 400; font-size: 13px"
|
||||
>
|
||||
<input
|
||||
type="radio"
|
||||
name="mcp-auth-type"
|
||||
id="mcp-auth-none"
|
||||
value="none"
|
||||
onchange="toggleMcpAuthFields()"
|
||||
style="margin-right: 6px"
|
||||
/>No authorization
|
||||
</label>
|
||||
<label
|
||||
style="display: block; margin: 4px 0; font-weight: 400; font-size: 13px"
|
||||
>
|
||||
<input
|
||||
type="radio"
|
||||
name="mcp-auth-type"
|
||||
id="mcp-auth-static"
|
||||
value="static"
|
||||
onchange="toggleMcpAuthFields()"
|
||||
checked
|
||||
style="margin-right: 6px"
|
||||
/>Static headers (single shared identity)
|
||||
</label>
|
||||
<label
|
||||
style="display: block; margin: 4px 0; font-weight: 400; font-size: 13px"
|
||||
>
|
||||
<input
|
||||
type="radio"
|
||||
name="mcp-auth-type"
|
||||
id="mcp-auth-oauth"
|
||||
value="oauth_user"
|
||||
onchange="toggleMcpAuthFields()"
|
||||
style="margin-right: 6px"
|
||||
/>Per-user OAuth 2.1 (recommended)
|
||||
</label>
|
||||
<div class="segmented-control" role="radiogroup" aria-label="Multitenant Authorization">
|
||||
<label class="segmented-option">
|
||||
<input
|
||||
type="radio"
|
||||
name="mcp-auth-type"
|
||||
id="mcp-auth-none"
|
||||
value="none"
|
||||
onchange="toggleMcpAuthFields()"
|
||||
/>
|
||||
<span class="segmented-indicator" aria-hidden="true"></span>
|
||||
<span class="segmented-text">No authorization</span>
|
||||
</label>
|
||||
<label class="segmented-option">
|
||||
<input
|
||||
type="radio"
|
||||
name="mcp-auth-type"
|
||||
id="mcp-auth-static"
|
||||
value="static"
|
||||
onchange="toggleMcpAuthFields()"
|
||||
checked
|
||||
/>
|
||||
<span class="segmented-indicator" aria-hidden="true"></span>
|
||||
<span class="segmented-text"
|
||||
>Static headers
|
||||
<span class="segmented-hint">single shared identity</span></span
|
||||
>
|
||||
</label>
|
||||
<label class="segmented-option">
|
||||
<input
|
||||
type="radio"
|
||||
name="mcp-auth-type"
|
||||
id="mcp-auth-oauth"
|
||||
value="oauth_user"
|
||||
onchange="toggleMcpAuthFields()"
|
||||
/>
|
||||
<span class="segmented-indicator" aria-hidden="true"></span>
|
||||
<span class="segmented-text"
|
||||
>Per-user OAuth 2.1
|
||||
<span class="segmented-hint">recommended</span></span
|
||||
>
|
||||
</label>
|
||||
</div>
|
||||
<div id="mcp-oauth-fields" style="display: none; margin-top: 8px">
|
||||
<label for="mcp-oauth-as-url"
|
||||
>Authorization Server URL
|
||||
@@ -3796,22 +3817,17 @@
|
||||
/>
|
||||
</div>
|
||||
</fieldset>
|
||||
<div style="display: flex; gap: 20px; margin-top: 14px">
|
||||
<label style="margin: 0; font-size: 12px; color: var(--fg-dim)"
|
||||
><input
|
||||
type="checkbox"
|
||||
id="mcp-auto-approve"
|
||||
style="margin-right: 5px"
|
||||
/>Auto-approve tools</label
|
||||
>
|
||||
<label style="margin: 0; font-size: 12px; color: var(--fg-dim)"
|
||||
><input
|
||||
type="checkbox"
|
||||
id="mcp-enabled"
|
||||
checked
|
||||
style="margin-right: 5px"
|
||||
/>Enabled</label
|
||||
>
|
||||
<div class="toggle-stack">
|
||||
<label class="toggle-switch">
|
||||
<input type="checkbox" id="mcp-auto-approve" />
|
||||
<span class="toggle-track" aria-hidden="true"></span>
|
||||
<span class="toggle-label">Auto-approve tools</span>
|
||||
</label>
|
||||
<label class="toggle-switch">
|
||||
<input type="checkbox" id="mcp-enabled" checked />
|
||||
<span class="toggle-track" aria-hidden="true"></span>
|
||||
<span class="toggle-label">Enabled</span>
|
||||
</label>
|
||||
</div>
|
||||
<div class="modal-buttons">
|
||||
<button class="modal-cancel" onclick="hideCreateMcpModal()">
|
||||
@@ -3936,6 +3952,14 @@
|
||||
<h2 id="model-create-title">Add Model</h2>
|
||||
<div id="model-create-error" role="alert" aria-live="assertive" aria-atomic="true"></div>
|
||||
<input type="hidden" id="model-edit-id" value="" />
|
||||
<label
|
||||
class="toggle-switch toggle--flush"
|
||||
title="Disable to hide this alias from every model dropdown (workstreams, schedules, channel adapters, role assignments) without removing the definition."
|
||||
>
|
||||
<input type="checkbox" id="model-enabled" checked />
|
||||
<span class="toggle-track" aria-hidden="true"></span>
|
||||
<span class="toggle-label">Active</span>
|
||||
</label>
|
||||
<label for="model-alias">Alias</label>
|
||||
<input
|
||||
type="text"
|
||||
@@ -4116,15 +4140,27 @@
|
||||
placeholder='{"supports_vision": true}'
|
||||
style="font-family: var(--font-mono); font-size: 11px"
|
||||
></textarea>
|
||||
<div style="display: flex; gap: 20px; margin-top: 14px">
|
||||
<label style="margin: 0; font-size: 12px; color: var(--fg-dim)"
|
||||
><input
|
||||
type="checkbox"
|
||||
id="model-enabled"
|
||||
checked
|
||||
style="margin-right: 5px"
|
||||
/>Enabled</label
|
||||
<div class="toggle-stack">
|
||||
<label
|
||||
class="toggle-switch"
|
||||
title="Surface stored reasoning text on /history responses (UI bubble on page reload). Storage of reasoning bytes is unaffected by this flag — they ride in provider_data regardless."
|
||||
>
|
||||
<input
|
||||
type="checkbox"
|
||||
id="model-surface-persisted-reasoning"
|
||||
checked
|
||||
/>
|
||||
<span class="toggle-track" aria-hidden="true"></span>
|
||||
<span class="toggle-label">Surface persisted reasoning</span>
|
||||
</label>
|
||||
<label
|
||||
class="toggle-switch"
|
||||
title="Replay stored reasoning blocks back to the model on subsequent provider calls. Capability-dependent; off by default for cost/spec compliance."
|
||||
>
|
||||
<input type="checkbox" id="model-replay-reasoning" />
|
||||
<span class="toggle-track" aria-hidden="true"></span>
|
||||
<span class="toggle-label">Replay reasoning to model</span>
|
||||
</label>
|
||||
</div>
|
||||
<div id="model-detect-area" style="margin-top: 14px">
|
||||
<button
|
||||
|
||||
@@ -1572,23 +1572,284 @@
|
||||
padding-left: 12px;
|
||||
}
|
||||
|
||||
/* Checkbox labels inside admin modals */
|
||||
.admin-modal label.admin-checkbox {
|
||||
/* Toggle switch — modern replacement for boolean checkboxes inside
|
||||
* admin modals. The native <input type="checkbox"> stays in the
|
||||
* markup (visually hidden but keyboard-focusable) so existing JS
|
||||
* that reads `.checked` keeps working; the .toggle-track + ::before
|
||||
* pseudo render the slider, .toggle-label carries the caption.
|
||||
*
|
||||
* Usage:
|
||||
* <label class="toggle-switch">
|
||||
* <input type="checkbox" id="..." />
|
||||
* <span class="toggle-track" aria-hidden="true"></span>
|
||||
* <span class="toggle-label">Enabled</span>
|
||||
* </label>
|
||||
*/
|
||||
/* Selector is doubled with .admin-modal so we win the specificity
|
||||
* battle against ``.admin-modal label`` (0,1,1) — without that, the
|
||||
* parent rule's display:block + text-transform:uppercase + margins
|
||||
* cascade and we lose the inline-flex layout.
|
||||
*
|
||||
* Default margin-top: 14px matches the .admin-modal label cadence
|
||||
* for toggles that sit directly between regular labelled rows
|
||||
* (schedule/policy/skill modals). When a toggle is inside an
|
||||
* explicit ``.toggle-stack`` flex container, the stack resets the
|
||||
* margin so the parent's gap controls spacing on its own. */
|
||||
.admin-modal label.toggle-switch,
|
||||
label.toggle-switch {
|
||||
display: inline-flex;
|
||||
align-items: center;
|
||||
gap: 10px;
|
||||
cursor: pointer;
|
||||
user-select: none;
|
||||
margin: 14px 0 0 0;
|
||||
padding: 0;
|
||||
text-transform: none;
|
||||
letter-spacing: 0;
|
||||
font-size: 12px;
|
||||
font-weight: 500;
|
||||
color: var(--fg);
|
||||
}
|
||||
.admin-modal .toggle-stack > label.toggle-switch,
|
||||
.toggle-stack > label.toggle-switch {
|
||||
margin: 0;
|
||||
}
|
||||
/* Modifier for toggles that should sit flush against the surrounding
|
||||
* rhythm rather than carry the default 14px top margin — used when a
|
||||
* toggle is the first row of a modal (under the h2) or when it lives
|
||||
* in a dynamically-rendered row that already supplies its own
|
||||
* spacing (e.g. judge bool settings). */
|
||||
.admin-modal label.toggle-switch.toggle--flush,
|
||||
label.toggle-switch.toggle--flush {
|
||||
margin-top: 0;
|
||||
}
|
||||
.admin-modal .toggle-stack,
|
||||
.toggle-stack {
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
gap: 10px;
|
||||
margin-top: 16px;
|
||||
}
|
||||
/* Hidden-input + track rules also need the .admin-modal prefix to
|
||||
* outrank ``.admin-modal input:not([type="hidden"])`` (same
|
||||
* specificity 0,2,1; that one comes later in source so without the
|
||||
* bump it wins and forces width:100% on the hidden input, popping it
|
||||
* back into the layout). */
|
||||
.admin-modal .toggle-switch input[type="checkbox"],
|
||||
.toggle-switch input[type="checkbox"] {
|
||||
/* Visually hidden but still focusable + click-targetable via the
|
||||
* <label> wrap. Avoids display:none, which would strip the input
|
||||
* from the keyboard tab order. */
|
||||
position: absolute;
|
||||
width: 1px;
|
||||
height: 1px;
|
||||
margin: -1px;
|
||||
padding: 0;
|
||||
border: 0;
|
||||
overflow: hidden;
|
||||
clip: rect(0 0 0 0);
|
||||
white-space: nowrap;
|
||||
}
|
||||
.admin-modal .toggle-switch .toggle-track,
|
||||
.toggle-switch .toggle-track {
|
||||
position: relative;
|
||||
flex: 0 0 auto;
|
||||
/* 40×22 meets WCAG 2.5.5 (Level AAA) target size when combined with
|
||||
* the label's hit area, and reads as a deliberate switch on touch
|
||||
* targets without dominating the form rhythm. */
|
||||
width: 40px;
|
||||
height: 22px;
|
||||
/* Off state: depressed inset on the modal background so the track
|
||||
* silhouette stays visible at 1.5+ contrast ratio. The earlier
|
||||
* ``var(--border-strong)`` solid fill was ~1.18:1 against the modal
|
||||
* surface — visible to most readers, invisible to anyone on a
|
||||
* glare-y screen or at ``prefers-contrast: more``. */
|
||||
background: var(--bg);
|
||||
box-shadow: inset 0 0 0 1px var(--border-strong);
|
||||
border-radius: 11px;
|
||||
transition:
|
||||
background 0.18s ease,
|
||||
box-shadow 0.18s ease;
|
||||
}
|
||||
.admin-modal .toggle-switch .toggle-track::before,
|
||||
.toggle-switch .toggle-track::before {
|
||||
content: "";
|
||||
position: absolute;
|
||||
top: 3px;
|
||||
left: 3px;
|
||||
width: 16px;
|
||||
height: 16px;
|
||||
background: var(--fg);
|
||||
border-radius: 50%;
|
||||
transition: transform 0.18s ease;
|
||||
}
|
||||
.admin-modal .toggle-switch input:checked + .toggle-track,
|
||||
.toggle-switch input:checked + .toggle-track {
|
||||
background: var(--accent);
|
||||
box-shadow: none;
|
||||
}
|
||||
.admin-modal .toggle-switch input:checked + .toggle-track::before,
|
||||
.toggle-switch input:checked + .toggle-track::before {
|
||||
transform: translateX(18px);
|
||||
background: var(--bg);
|
||||
}
|
||||
.admin-modal .toggle-switch input:focus-visible + .toggle-track,
|
||||
.toggle-switch input:focus-visible + .toggle-track {
|
||||
box-shadow: 0 0 0 3px var(--accent-dim);
|
||||
}
|
||||
.admin-modal .toggle-switch input:disabled + .toggle-track,
|
||||
.toggle-switch input:disabled + .toggle-track {
|
||||
opacity: 0.4;
|
||||
}
|
||||
.toggle-switch:has(input:disabled) {
|
||||
cursor: not-allowed;
|
||||
opacity: 0.7;
|
||||
}
|
||||
.toggle-switch .toggle-label {
|
||||
display: inline-block;
|
||||
/* The label text rides at the modal's normal label cadence — caps
|
||||
* + letter-spacing so it sits next to the surrounding form rows
|
||||
* without shifting the visual rhythm. */
|
||||
font-family: var(--font-ui);
|
||||
font-size: 10px;
|
||||
font-weight: 600;
|
||||
text-transform: uppercase;
|
||||
letter-spacing: 0.08em;
|
||||
color: var(--fg-dim);
|
||||
}
|
||||
|
||||
/* Hair divider for separating conceptually-grouped toggles inside a
|
||||
* stack — used between the lone "Enabled" toggle and the paired
|
||||
* Reasoning toggles in the Add Model modal so the grouping reads
|
||||
* without needing a subheading or indent. Margin: 0 because the
|
||||
* .toggle-stack flex container already supplies a 10px gap on each
|
||||
* side; adding a margin on top would visually separate the divider
|
||||
* twice. */
|
||||
.admin-modal hr.toggle-group-divider,
|
||||
hr.toggle-group-divider {
|
||||
border: 0;
|
||||
border-top: 1px solid var(--border);
|
||||
margin: 0;
|
||||
width: 100%;
|
||||
}
|
||||
|
||||
/* Segmented option list — vertical card group for radio choices that
|
||||
* benefit from a strong selected-state highlight. The native
|
||||
* ``<input type="radio">`` is visually hidden but stays focusable;
|
||||
* the ``.segmented-indicator`` pseudo-circle and the row's
|
||||
* background carry the selected state.
|
||||
*
|
||||
* Usage:
|
||||
* <div class="segmented-control" role="radiogroup">
|
||||
* <label class="segmented-option">
|
||||
* <input type="radio" name="..." value="..." />
|
||||
* <span class="segmented-indicator" aria-hidden="true"></span>
|
||||
* <span class="segmented-text">Label
|
||||
* <span class="segmented-hint">(hint)</span>
|
||||
* </span>
|
||||
* </label>
|
||||
* ...
|
||||
* </div>
|
||||
*/
|
||||
.admin-modal .segmented-control,
|
||||
.segmented-control {
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
border: 1px solid var(--border-strong);
|
||||
border-radius: var(--radius-sm);
|
||||
overflow: hidden;
|
||||
background: var(--bg);
|
||||
margin-top: 8px;
|
||||
}
|
||||
.admin-modal .segmented-option,
|
||||
.segmented-option {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 8px;
|
||||
font-size: 12px;
|
||||
gap: 10px;
|
||||
padding: 11px 14px;
|
||||
cursor: pointer;
|
||||
margin: 0;
|
||||
font-family: var(--font-ui);
|
||||
font-size: 13px;
|
||||
font-weight: 500;
|
||||
text-transform: none;
|
||||
letter-spacing: 0;
|
||||
color: var(--fg);
|
||||
cursor: pointer;
|
||||
margin-top: 14px;
|
||||
color: var(--fg-dim);
|
||||
border-top: 1px solid var(--border);
|
||||
transition:
|
||||
background 0.15s ease,
|
||||
color 0.15s ease;
|
||||
}
|
||||
.admin-modal label.admin-checkbox input[type="checkbox"],
|
||||
.admin-modal label.admin-checkbox input[type="radio"] {
|
||||
width: auto;
|
||||
margin: 0;
|
||||
.segmented-option:first-of-type {
|
||||
border-top: none;
|
||||
}
|
||||
.segmented-option input[type="radio"] {
|
||||
/* Visually hidden but focusable + click-targetable via the label
|
||||
* wrap. Keyboard arrows still cycle within the radiogroup. */
|
||||
position: absolute;
|
||||
width: 1px;
|
||||
height: 1px;
|
||||
margin: -1px;
|
||||
padding: 0;
|
||||
border: 0;
|
||||
overflow: hidden;
|
||||
clip: rect(0 0 0 0);
|
||||
}
|
||||
.segmented-option .segmented-indicator {
|
||||
position: relative;
|
||||
flex: 0 0 auto;
|
||||
width: 16px;
|
||||
height: 16px;
|
||||
border-radius: 50%;
|
||||
background: var(--bg);
|
||||
box-shadow: inset 0 0 0 1px var(--border-strong);
|
||||
transition:
|
||||
background 0.15s ease,
|
||||
box-shadow 0.15s ease;
|
||||
}
|
||||
.segmented-option .segmented-indicator::after {
|
||||
content: "";
|
||||
position: absolute;
|
||||
top: 4px;
|
||||
left: 4px;
|
||||
width: 6px;
|
||||
height: 6px;
|
||||
border-radius: 50%;
|
||||
background: transparent;
|
||||
transition: background 0.15s ease;
|
||||
}
|
||||
.segmented-option:hover {
|
||||
background: var(--bg-highlight);
|
||||
color: var(--fg);
|
||||
}
|
||||
.segmented-option:has(input:checked) {
|
||||
background: var(--accent-dim);
|
||||
color: var(--fg);
|
||||
}
|
||||
.segmented-option:has(input:checked) .segmented-indicator {
|
||||
box-shadow: inset 0 0 0 1px var(--accent);
|
||||
}
|
||||
.segmented-option:has(input:checked) .segmented-indicator::after {
|
||||
background: var(--accent);
|
||||
}
|
||||
.segmented-option:has(input:focus-visible) {
|
||||
/* Bright accent (not --accent-dim) so the ring stays visible even
|
||||
* on the currently-selected row, which already paints --accent-dim
|
||||
* as its background. */
|
||||
box-shadow: inset 0 0 0 2px var(--accent);
|
||||
}
|
||||
.segmented-option .segmented-text {
|
||||
flex: 1 1 auto;
|
||||
min-width: 0;
|
||||
}
|
||||
.segmented-option .segmented-hint {
|
||||
font-size: 11px;
|
||||
font-weight: 400;
|
||||
color: var(--fg-dim);
|
||||
margin-left: 4px;
|
||||
}
|
||||
.segmented-option:has(input:checked) .segmented-hint {
|
||||
color: var(--accent);
|
||||
}
|
||||
|
||||
/* Admin modals */
|
||||
@@ -1692,9 +1953,6 @@
|
||||
border-color: var(--border);
|
||||
color: var(--fg-dim);
|
||||
}
|
||||
.admin-modal label.admin-checkbox input:disabled {
|
||||
opacity: 0.4;
|
||||
}
|
||||
.admin-modal input::placeholder,
|
||||
.admin-modal textarea::placeholder {
|
||||
color: var(--fg-dim);
|
||||
@@ -2755,28 +3013,64 @@ textarea.skill-content-area {
|
||||
padding: 0;
|
||||
margin-bottom: 4px;
|
||||
}
|
||||
/* Permission section grouping — keeps each namespace contiguous so
|
||||
* the row-flow grid below doesn't slice ``admin.*`` mid-column. The
|
||||
* caps-styled ``.perm-section-label`` re-anchors the toggles to the
|
||||
* modal's typographic system (matches ``.admin-modal label``: 10px
|
||||
* caps, 0.08em letter-spacing, fg-dim). */
|
||||
.perm-section {
|
||||
margin-top: 12px;
|
||||
}
|
||||
.perm-section:first-of-type {
|
||||
margin-top: 0;
|
||||
}
|
||||
.perm-section-label {
|
||||
font-family: var(--font-ui);
|
||||
font-size: 10px;
|
||||
font-weight: 600;
|
||||
text-transform: uppercase;
|
||||
letter-spacing: 0.08em;
|
||||
color: var(--fg-dim);
|
||||
margin-bottom: 4px;
|
||||
}
|
||||
.perm-grid {
|
||||
display: grid;
|
||||
grid-template-columns: 1fr 1fr;
|
||||
gap: 4px 16px;
|
||||
padding: 8px 0;
|
||||
gap: 8px 16px;
|
||||
padding: 4px 0 0;
|
||||
}
|
||||
.perm-checkbox {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 6px;
|
||||
font-size: 11px;
|
||||
/* Permission rows inherit the toggle-switch component but render the
|
||||
* permission name in monospace + lower case (it's an identifier, not
|
||||
* a heading) — overrides the .toggle-label's caps/letter-spacing
|
||||
* cadence used elsewhere in admin modals. */
|
||||
.admin-modal .perm-grid label.toggle-switch.perm-toggle,
|
||||
.perm-grid label.toggle-switch.perm-toggle {
|
||||
margin: 0;
|
||||
}
|
||||
.perm-grid .toggle-switch.perm-toggle .toggle-label {
|
||||
font-family: var(--font-mono);
|
||||
color: var(--fg);
|
||||
padding: 3px 0;
|
||||
cursor: pointer;
|
||||
font-size: 11px;
|
||||
font-weight: 500;
|
||||
text-transform: none;
|
||||
letter-spacing: normal;
|
||||
color: var(--fg);
|
||||
}
|
||||
.perm-checkbox input[type="checkbox"] {
|
||||
width: auto;
|
||||
|
||||
/* User-role assignment list (Users tab → Manage roles modal). Same
|
||||
* toggle-switch component as elsewhere, but the labels are
|
||||
* human-readable role display names so we keep ui-font + caps-on so
|
||||
* the stack reads like a settings list. Only override the layout
|
||||
* margin (the modal already provides outer padding). */
|
||||
.admin-modal .user-roles-list,
|
||||
.user-roles-list {
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
gap: 6px;
|
||||
margin-top: 8px;
|
||||
}
|
||||
.admin-modal .user-roles-list label.toggle-switch.user-role-toggle,
|
||||
.user-roles-list label.toggle-switch.user-role-toggle {
|
||||
margin: 0;
|
||||
accent-color: var(--accent);
|
||||
}
|
||||
|
||||
/* ==========================================================================
|
||||
|
||||
@@ -19,7 +19,7 @@ either an async caller (via ``asyncio.to_thread``) or a sync hook.
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from typing import Any
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
from turnstone.core.log import get_logger
|
||||
from turnstone.core.tool_advisory import (
|
||||
@@ -268,6 +268,161 @@ def extract_advisories_from_tool_envelope(
|
||||
return _entity_decode_wrapper_tags(inner), advisories
|
||||
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from collections.abc import Callable
|
||||
|
||||
from turnstone.core.providers._protocol import LLMProvider
|
||||
|
||||
|
||||
def _make_provider_factory(module_path: str, class_name: str) -> Callable[[], LLMProvider]:
|
||||
"""Build a thread-unsafe lazy-init factory for a provider singleton.
|
||||
|
||||
Each block-type entry in ``_BLOCK_TYPE_PROVIDER_FACTORY`` closes
|
||||
over its own (module_path, class_name) pair. Adding a fourth
|
||||
provider is a single tuple in the dict, not a new 9-line getter.
|
||||
|
||||
Uses ``nonlocal`` instead of ``functools.lru_cache`` so the cache
|
||||
state stays inside this closure (lru_cache would attach state to
|
||||
the inner function object, which is correct but adds a per-call
|
||||
hash lookup on a bound key for what's effectively a single-slot
|
||||
cache).
|
||||
"""
|
||||
instance: LLMProvider | None = None
|
||||
|
||||
def factory() -> LLMProvider:
|
||||
nonlocal instance
|
||||
if instance is None:
|
||||
import importlib
|
||||
|
||||
module = importlib.import_module(module_path)
|
||||
instance = getattr(module, class_name)()
|
||||
return instance
|
||||
|
||||
return factory
|
||||
|
||||
|
||||
# Block-type → provider factory. Routing is structural — block shape
|
||||
# is non-overlapping across providers by API design. Recognised
|
||||
# block types today:
|
||||
#
|
||||
# * ``"thinking"`` — Anthropic native (Phase 1). Walks the
|
||||
# ``thinking`` field on each block.
|
||||
# * ``"redacted_thinking"`` — Anthropic native (Phase 1). Anthropic's
|
||||
# safety system rewrites a thinking block into a sealed
|
||||
# ``redacted_thinking`` block; the Anthropic docs note these can
|
||||
# appear before, after, or interleaved with regular ``thinking``
|
||||
# blocks. Same factory: AnthropicProvider's extractor walks the
|
||||
# full block list and filters to ``type == "thinking"``, so the
|
||||
# redacted blocks are correctly skipped while the surrounding
|
||||
# real thinking text still surfaces.
|
||||
# * ``"reasoning"`` — OpenAI Responses native (Phase 3). Walks
|
||||
# ``summary[*].text`` (always present) and ``content[*].text``
|
||||
# (present when ``include=["reasoning.encrypted_content"]`` is
|
||||
# requested AND the response carries raw reasoning text).
|
||||
# * ``"reasoning_text"`` — synthetic, stamped by
|
||||
# ``ChatSession._maybe_synth_reasoning_block`` for Chat Completions
|
||||
# paths (vLLM, llama.cpp, Gemini-compat) where reasoning surfaces
|
||||
# only as ``reasoning_delta`` chunks with no native block shape.
|
||||
_anthropic_factory = _make_provider_factory(
|
||||
"turnstone.core.providers._anthropic", "AnthropicProvider"
|
||||
)
|
||||
_BLOCK_TYPE_PROVIDER_FACTORY: dict[str, Callable[[], LLMProvider]] = {
|
||||
"thinking": _anthropic_factory,
|
||||
"redacted_thinking": _anthropic_factory,
|
||||
"reasoning": _make_provider_factory(
|
||||
"turnstone.core.providers._openai_responses", "OpenAIResponsesProvider"
|
||||
),
|
||||
"reasoning_text": _make_provider_factory(
|
||||
"turnstone.core.providers._openai_chat", "OpenAIChatCompletionsProvider"
|
||||
),
|
||||
}
|
||||
|
||||
|
||||
def extract_reasoning_text_from_provider_content(provider_content: Any) -> str:
|
||||
"""Dispatch reasoning extraction by scanning for a recognised block type.
|
||||
|
||||
Walks ``provider_content`` looking for the first block whose
|
||||
``type`` is in :data:`_BLOCK_TYPE_PROVIDER_FACTORY`, then dispatches
|
||||
the WHOLE list to that provider's ``extract_reasoning_text``. Each
|
||||
provider's extractor already filters internally by its own block
|
||||
type (Anthropic walks ``thinking``, OpenAI Responses walks
|
||||
``reasoning``, OpenAI Chat walks ``reasoning_text``), so passing
|
||||
the full list is correct — interleaved foreign blocks are ignored.
|
||||
|
||||
Returns ``""`` for empty / missing / non-list input or when no
|
||||
recognised reasoning-bearing block type appears anywhere in the
|
||||
list.
|
||||
|
||||
Why scan instead of just inspecting ``provider_content[0]``: the
|
||||
OpenAI Responses streaming layer captures EVERY ``output_item.done``
|
||||
event into ``provider_blocks`` (``_openai_responses.py:415-420``),
|
||||
not just reasoning items. In practice the order is usually
|
||||
``[reasoning, message, ...]`` but the API doesn't guarantee that —
|
||||
a hypothetical ``[message, reasoning]`` ordering would silently
|
||||
drop the reasoning under an index-only check. Same robustness
|
||||
point for Anthropic's hypothetical mixed-order outputs.
|
||||
|
||||
Pure transform — safe from any thread. Both history surfaces
|
||||
(interactive ``_build_history`` and lifted ``make_history_handler``)
|
||||
call this directly. See ``_BLOCK_TYPE_PROVIDER_FACTORY`` above
|
||||
for the recognised block types and the providers that own them.
|
||||
"""
|
||||
if not isinstance(provider_content, list) or not provider_content:
|
||||
return ""
|
||||
for block in provider_content:
|
||||
if not isinstance(block, dict):
|
||||
continue
|
||||
block_type = block.get("type")
|
||||
if not isinstance(block_type, str):
|
||||
continue
|
||||
factory = _BLOCK_TYPE_PROVIDER_FACTORY.get(block_type)
|
||||
if factory is not None:
|
||||
return factory().extract_reasoning_text(provider_content)
|
||||
return ""
|
||||
|
||||
|
||||
def extract_reasoning_for_history(
|
||||
messages: list[dict[str, Any]],
|
||||
surface_persisted_reasoning_flag: bool,
|
||||
) -> None:
|
||||
"""Surface stored reasoning text on each assistant message; strip the
|
||||
raw provider content from the wire payload.
|
||||
|
||||
For the ``make_history_handler`` REST path where the response
|
||||
payload IS the messages list returned from ``storage.load_messages``
|
||||
— both extraction source and stamp destination are the same dict.
|
||||
Walks *messages* in place: for every assistant message, dispatches
|
||||
via :func:`extract_reasoning_text_from_provider_content` and stamps
|
||||
``msg["reasoning"]`` when *surface_persisted_reasoning_flag* is True and the
|
||||
dispatcher returned non-empty text. Strips ``_provider_content``
|
||||
unconditionally — the field is internal and never read by either UI.
|
||||
|
||||
The interactive ``_build_history`` surface DOES NOT call this
|
||||
helper; it builds new entry dicts from scratch and calls
|
||||
:func:`extract_reasoning_text_from_provider_content` directly per
|
||||
assistant message, stamping ``entry["reasoning"]`` inline. The two
|
||||
surfaces converge on the same dispatcher; only the mutation shape
|
||||
differs.
|
||||
|
||||
Pure transform. Safe to call from ``asyncio.to_thread``.
|
||||
"""
|
||||
for msg in messages:
|
||||
if msg.get("role") != "assistant":
|
||||
continue
|
||||
provider_content = msg.get("_provider_content")
|
||||
# Always strip the internal lane before the wire payload leaves
|
||||
# the helper, even when surface_persisted_reasoning_flag is False or the
|
||||
# field is empty/missing. The strip is the contract; reasoning
|
||||
# surfacing is conditional on top of it.
|
||||
if "_provider_content" in msg:
|
||||
del msg["_provider_content"]
|
||||
if not surface_persisted_reasoning_flag:
|
||||
continue
|
||||
text = extract_reasoning_text_from_provider_content(provider_content)
|
||||
if text:
|
||||
msg["reasoning"] = text
|
||||
|
||||
|
||||
def decorate_history_messages(
|
||||
messages: list[dict[str, Any]],
|
||||
verdicts_by_call_id: dict[str, dict[str, Any]],
|
||||
|
||||
+20
-13
@@ -910,7 +910,17 @@ class IntentJudge:
|
||||
self._context_window = context_window
|
||||
self._rule_registry = rule_registry
|
||||
|
||||
# Resolve judge model via ModelRegistry alias, falling back to session
|
||||
# Resolve judge model via ModelRegistry alias, otherwise self-
|
||||
# consistency on the session model. ``judge.model`` is alias-only
|
||||
# — same contract as ``coordinator.model_alias`` /
|
||||
# ``model.plan_alias`` / ``model.task_alias``. A non-alias value
|
||||
# used to be accepted as a raw model id pinned onto the session
|
||||
# provider, but that path silently broke whenever the session
|
||||
# provider didn't speak that model id (e.g. coordinator on
|
||||
# Anthropic, ``judge.model = "gpt-5-mini"`` → every judge call
|
||||
# returned ``llm_fallback``). Operators register an alias
|
||||
# instead; an unknown value here logs a warning and inherits the
|
||||
# session model.
|
||||
resolved = False
|
||||
if config.model and model_registry is not None:
|
||||
try:
|
||||
@@ -928,18 +938,15 @@ class IntentJudge:
|
||||
except Exception:
|
||||
log.debug("Model alias resolution failed for %r, falling back", config.model)
|
||||
|
||||
if not resolved and config.model:
|
||||
# Model name override with session provider
|
||||
self._provider = session_provider
|
||||
self._client_factory_args = self._extract_client_config(
|
||||
session_client,
|
||||
session_provider.provider_name,
|
||||
)
|
||||
self._model = config.model
|
||||
caps = self._provider.get_capabilities(self._model)
|
||||
self._judge_context_window = caps.context_window
|
||||
elif not resolved:
|
||||
# Self-consistency: same model as session
|
||||
if not resolved:
|
||||
if config.model:
|
||||
log.warning(
|
||||
"judge.model=%r is not a registered alias — falling back to "
|
||||
"session model %r. Register the model in the Models tab and "
|
||||
"set judge.model to its alias.",
|
||||
config.model,
|
||||
session_model,
|
||||
)
|
||||
self._provider = session_provider
|
||||
self._client_factory_args = self._extract_client_config(
|
||||
session_client,
|
||||
|
||||
@@ -39,6 +39,12 @@ class ModelConfig:
|
||||
temperature: float | None = None
|
||||
max_tokens: int | None = None
|
||||
reasoning_effort: str | None = None
|
||||
# Per-model reasoning-persistence flags (db-backed, admin-toggleable).
|
||||
# surface_persisted_reasoning controls UI rehydration of stored reasoning text in
|
||||
# /history responses; replay_reasoning_to_model controls whether
|
||||
# reasoning blocks ride the wire on subsequent provider calls.
|
||||
surface_persisted_reasoning: bool = True
|
||||
replay_reasoning_to_model: bool = False
|
||||
# Server compatibility settings for openai-compatible backends.
|
||||
# Populated from capabilities["server_compat"] during load.
|
||||
server_compat: dict[str, Any] = field(default_factory=dict)
|
||||
@@ -405,6 +411,10 @@ def load_model_registry(
|
||||
row_temperature = row.get("temperature")
|
||||
row_max_tokens = row.get("max_tokens")
|
||||
row_reasoning_effort = row.get("reasoning_effort")
|
||||
# Per-model reasoning flags. Defaults match the dataclass so a
|
||||
# pre-052 row missing these columns degrades gracefully.
|
||||
row_surface_persisted_reasoning = bool(row.get("surface_persisted_reasoning", True))
|
||||
row_replay_reasoning = bool(row.get("replay_reasoning_to_model", False))
|
||||
configs[alias] = ModelConfig(
|
||||
alias=alias,
|
||||
base_url=row_base_url,
|
||||
@@ -419,6 +429,8 @@ def load_model_registry(
|
||||
reasoning_effort=row_reasoning_effort
|
||||
if row_reasoning_effort is not None
|
||||
else None,
|
||||
surface_persisted_reasoning=row_surface_persisted_reasoning,
|
||||
replay_reasoning_to_model=row_replay_reasoning,
|
||||
server_compat=row_server_compat,
|
||||
)
|
||||
except Exception:
|
||||
|
||||
@@ -17,6 +17,7 @@ from turnstone.core.providers._protocol import (
|
||||
StreamChunk,
|
||||
ToolCallDelta,
|
||||
UsageInfo,
|
||||
_join_reasoning_with_cap,
|
||||
_lookup_capabilities,
|
||||
)
|
||||
|
||||
@@ -80,6 +81,7 @@ _ANTHROPIC_DEFAULT = ModelCapabilities(
|
||||
thinking_mode="manual",
|
||||
supports_web_search=True,
|
||||
supports_vision=True,
|
||||
supports_reasoning_replay=True,
|
||||
)
|
||||
|
||||
_ANTHROPIC_CAPABILITIES: dict[str, ModelCapabilities] = {
|
||||
@@ -95,6 +97,7 @@ _ANTHROPIC_CAPABILITIES: dict[str, ModelCapabilities] = {
|
||||
supports_vision=True,
|
||||
supports_temperature=False,
|
||||
thinking_display="summarized",
|
||||
supports_reasoning_replay=True,
|
||||
),
|
||||
"claude-opus-4-6": ModelCapabilities(
|
||||
context_window=1000000,
|
||||
@@ -106,6 +109,7 @@ _ANTHROPIC_CAPABILITIES: dict[str, ModelCapabilities] = {
|
||||
supports_web_search=True,
|
||||
supports_tool_search=True,
|
||||
supports_vision=True,
|
||||
supports_reasoning_replay=True,
|
||||
),
|
||||
"claude-sonnet-4-6": ModelCapabilities(
|
||||
context_window=1000000,
|
||||
@@ -117,6 +121,7 @@ _ANTHROPIC_CAPABILITIES: dict[str, ModelCapabilities] = {
|
||||
supports_web_search=True,
|
||||
supports_tool_search=True,
|
||||
supports_vision=True,
|
||||
supports_reasoning_replay=True,
|
||||
),
|
||||
"claude-haiku-4-5": ModelCapabilities(
|
||||
context_window=200000,
|
||||
@@ -125,6 +130,7 @@ _ANTHROPIC_CAPABILITIES: dict[str, ModelCapabilities] = {
|
||||
thinking_mode="manual",
|
||||
supports_web_search=True,
|
||||
supports_vision=True,
|
||||
supports_reasoning_replay=True,
|
||||
),
|
||||
"claude-sonnet-4-5": ModelCapabilities(
|
||||
context_window=200000,
|
||||
@@ -133,6 +139,7 @@ _ANTHROPIC_CAPABILITIES: dict[str, ModelCapabilities] = {
|
||||
thinking_mode="manual",
|
||||
supports_web_search=True,
|
||||
supports_vision=True,
|
||||
supports_reasoning_replay=True,
|
||||
),
|
||||
"claude-opus-4-5": ModelCapabilities(
|
||||
context_window=200000,
|
||||
@@ -143,6 +150,7 @@ _ANTHROPIC_CAPABILITIES: dict[str, ModelCapabilities] = {
|
||||
effort_levels=("low", "medium", "high"),
|
||||
supports_web_search=True,
|
||||
supports_vision=True,
|
||||
supports_reasoning_replay=True,
|
||||
),
|
||||
}
|
||||
|
||||
@@ -159,6 +167,39 @@ def _map_reasoning_to_effort(
|
||||
return None
|
||||
|
||||
|
||||
# Anthropic accepts only a closed set of content-block types on the
|
||||
# input boundary. ``_convert_messages`` runs each block in
|
||||
# ``_provider_content`` through this set per-block: foreign-shaped
|
||||
# blocks (OpenAI Responses ``type="reasoning"``, Gemini thought parts,
|
||||
# the synthetic ``reasoning_text`` from path-3 capture) are dropped
|
||||
# individually; valid Anthropic blocks in the same message still ride
|
||||
# the verbatim path. When no valid blocks survive, the converter falls
|
||||
# through to the text+tool_calls rebuild path. Web-search blocks
|
||||
# (``server_tool_use`` / ``web_search_tool_result``) stay in the set
|
||||
# because they carry ``encrypted_content`` the API requires for
|
||||
# round-trip continuity — the per-block filter preserves them even when
|
||||
# they share a message with a foreign block (the prior all-or-nothing
|
||||
# filter would have silently dropped them in that case).
|
||||
ANTHROPIC_VALID_BLOCK_TYPES = frozenset(
|
||||
{
|
||||
"text",
|
||||
"image",
|
||||
"thinking",
|
||||
"redacted_thinking",
|
||||
"tool_use",
|
||||
"tool_result",
|
||||
"server_tool_use",
|
||||
"web_search_tool_result",
|
||||
}
|
||||
)
|
||||
|
||||
# Subset of valid block types whose ``content`` is reasoning text.
|
||||
# When ``replay_reasoning_to_model=False`` the strip predicate drops
|
||||
# these (and only these) before the wire payload is built — narrow by
|
||||
# design so ``tool_use`` / web-search blocks survive.
|
||||
ANTHROPIC_REASONING_BLOCK_TYPES = frozenset({"thinking", "redacted_thinking"})
|
||||
|
||||
|
||||
# -- provider ----------------------------------------------------------------
|
||||
|
||||
|
||||
@@ -283,10 +324,37 @@ class AnthropicProvider:
|
||||
def _convert_messages(
|
||||
self,
|
||||
messages: list[dict[str, Any]],
|
||||
*,
|
||||
replay_reasoning_to_model: bool = True,
|
||||
) -> tuple[str, list[dict[str, Any]]]:
|
||||
"""Convert internal (OpenAI-like) messages to Anthropic format.
|
||||
|
||||
Returns ``(system_prompt, converted_messages)``.
|
||||
|
||||
``replay_reasoning_to_model`` (Phase 2 of the reasoning-
|
||||
persistence feature, mirroring ``ModelConfig.replay_
|
||||
reasoning_to_model``) gates whether stored ``thinking`` blocks
|
||||
survive the verbatim ``_provider_content`` replay path. When
|
||||
the caller passes ``False`` (the operator-side server_default
|
||||
for the ``model_definitions`` row this kwarg is sourced from),
|
||||
thinking blocks are stripped before the wire payload is built;
|
||||
``tool_use`` / ``server_tool_use`` / ``web_search_tool_result``
|
||||
blocks (which carry web-search ``encrypted_content``) are
|
||||
intentionally preserved. The kwarg defaults to ``True`` here
|
||||
purely for back-compat with any direct caller that hasn't been
|
||||
updated to thread the resolver — production call sites
|
||||
(``ChatSession._try_stream`` / ``_utility_completion``) always
|
||||
pass the resolved flag explicitly.
|
||||
|
||||
Foreign-shaped blocks (OpenAI ``reasoning``, Gemini thought
|
||||
parts, the synthetic ``reasoning_text`` from path-3 capture) are
|
||||
dropped per-block; valid Anthropic blocks in the same message
|
||||
still ride the verbatim path. Critical for cross-model
|
||||
resumption: an earlier all-or-nothing filter silently lost
|
||||
web-search ``encrypted_content`` whenever a foreign block
|
||||
shared a message with ``server_tool_use`` /
|
||||
``web_search_tool_result``. If the filter leaves nothing, the
|
||||
converter falls through to the text+tool_calls rebuild path.
|
||||
"""
|
||||
system_parts: list[str] = []
|
||||
converted: list[dict[str, Any]] = []
|
||||
@@ -309,47 +377,88 @@ class AnthropicProvider:
|
||||
if pending_orphan_results:
|
||||
converted.append({"role": "user", "content": pending_orphan_results})
|
||||
pending_orphan_results = []
|
||||
# If raw provider content was preserved, pass it through verbatim
|
||||
# so encrypted_content/encrypted_index from web search are retained
|
||||
# Per-block shape filter: drop foreign blocks individually,
|
||||
# apply the replay strip to valid ``thinking`` /
|
||||
# ``redacted_thinking`` blocks. See ``_convert_messages``
|
||||
# docstring for why this is per-block, not all-or-nothing.
|
||||
provider_content = msg.get("_provider_content")
|
||||
if provider_content:
|
||||
converted.append({"role": "assistant", "content": provider_content})
|
||||
# Check for orphaned tool_use in provider content too
|
||||
if isinstance(provider_content, list):
|
||||
pc_tool_ids = [
|
||||
b["id"]
|
||||
for b in provider_content
|
||||
if isinstance(b, dict) and b.get("type") == "tool_use" and b.get("id")
|
||||
]
|
||||
if pc_tool_ids:
|
||||
j = i + 1
|
||||
result_ids_pc: set[str] = set()
|
||||
while j < len(messages) and messages[j]["role"] == "tool":
|
||||
tc_id = messages[j].get("tool_call_id", "")
|
||||
if tc_id:
|
||||
result_ids_pc.add(tc_id)
|
||||
j += 1
|
||||
orphaned_pc = [uid for uid in pc_tool_ids if uid not in result_ids_pc]
|
||||
if orphaned_pc:
|
||||
log.debug(
|
||||
"Synthesizing %d tool_result(s) for orphaned provider_content tool_use IDs",
|
||||
len(orphaned_pc),
|
||||
)
|
||||
synthetic_pc = [
|
||||
{
|
||||
"type": "tool_result",
|
||||
"tool_use_id": uid,
|
||||
"content": "Tool execution was cancelled.",
|
||||
"is_error": True,
|
||||
}
|
||||
for uid in orphaned_pc
|
||||
]
|
||||
if j == i + 1:
|
||||
converted.append({"role": "user", "content": synthetic_pc})
|
||||
else:
|
||||
pending_orphan_results = synthetic_pc
|
||||
wire_blocks: list[dict[str, Any]] = []
|
||||
valid_blocks: list[dict[str, Any]] = []
|
||||
all_input_valid = True
|
||||
if isinstance(provider_content, list) and provider_content:
|
||||
dropped_foreign: list[str] = []
|
||||
for b in provider_content:
|
||||
if not isinstance(b, dict):
|
||||
all_input_valid = False
|
||||
continue
|
||||
btype = b.get("type")
|
||||
if btype not in ANTHROPIC_VALID_BLOCK_TYPES:
|
||||
dropped_foreign.append(str(btype))
|
||||
all_input_valid = False
|
||||
continue
|
||||
valid_blocks.append(b)
|
||||
if (
|
||||
not replay_reasoning_to_model
|
||||
and btype in ANTHROPIC_REASONING_BLOCK_TYPES
|
||||
):
|
||||
continue
|
||||
wire_blocks.append(b)
|
||||
if dropped_foreign:
|
||||
log.debug(
|
||||
"Dropped %d foreign block(s) from _provider_content "
|
||||
"during Anthropic conversion: %s",
|
||||
len(dropped_foreign),
|
||||
dropped_foreign,
|
||||
)
|
||||
# Identity-preserving fast path: when nothing was filtered
|
||||
# or stripped, reuse the source list reference rather than
|
||||
# the per-block-built copy. Pinned by the ``is`` assertions
|
||||
# in test_providers.py (test_convert_messages_uses_provider_
|
||||
# content + test_thinking_block_multiturn_roundtrip).
|
||||
if all_input_valid and replay_reasoning_to_model:
|
||||
wire_blocks = provider_content
|
||||
if valid_blocks and wire_blocks:
|
||||
converted.append({"role": "assistant", "content": wire_blocks})
|
||||
# Orphan-tool detection reads ``valid_blocks`` (unstripped
|
||||
# valid blocks) so a future widening of the strip predicate
|
||||
# can't accidentally drop tool_use IDs.
|
||||
pc_tool_ids = [
|
||||
b["id"] for b in valid_blocks if b.get("type") == "tool_use" and b.get("id")
|
||||
]
|
||||
if pc_tool_ids:
|
||||
j = i + 1
|
||||
result_ids_pc: set[str] = set()
|
||||
while j < len(messages) and messages[j]["role"] == "tool":
|
||||
tc_id = messages[j].get("tool_call_id", "")
|
||||
if tc_id:
|
||||
result_ids_pc.add(tc_id)
|
||||
j += 1
|
||||
orphaned_pc = [uid for uid in pc_tool_ids if uid not in result_ids_pc]
|
||||
if orphaned_pc:
|
||||
log.debug(
|
||||
"Synthesizing %d tool_result(s) for orphaned provider_content tool_use IDs",
|
||||
len(orphaned_pc),
|
||||
)
|
||||
synthetic_pc = [
|
||||
{
|
||||
"type": "tool_result",
|
||||
"tool_use_id": uid,
|
||||
"content": "Tool execution was cancelled.",
|
||||
"is_error": True,
|
||||
}
|
||||
for uid in orphaned_pc
|
||||
]
|
||||
if j == i + 1:
|
||||
converted.append({"role": "user", "content": synthetic_pc})
|
||||
else:
|
||||
pending_orphan_results = synthetic_pc
|
||||
i += 1
|
||||
continue
|
||||
# Fall through to text+tool_calls rebuild when nothing
|
||||
# survived the filter (missing/empty pc, all-foreign, or
|
||||
# strip removed every remaining thinking block). An empty
|
||||
# rebuild is silently skipped — a turn with only stripped
|
||||
# reasoning has nothing to replay.
|
||||
|
||||
content_blocks: list[dict[str, Any]] = []
|
||||
text = msg.get("content")
|
||||
@@ -618,10 +727,13 @@ class AnthropicProvider:
|
||||
deferred_names: frozenset[str] | None = None,
|
||||
cancel_ref: list[Any] | None = None,
|
||||
capabilities: ModelCapabilities | None = None,
|
||||
replay_reasoning_to_model: bool = True,
|
||||
) -> Iterator[StreamChunk]:
|
||||
_ensure_anthropic()
|
||||
caps = capabilities or self.get_capabilities(model)
|
||||
system_prompt, converted_msgs = self._convert_messages(messages)
|
||||
system_prompt, converted_msgs = self._convert_messages(
|
||||
messages, replay_reasoning_to_model=replay_reasoning_to_model
|
||||
)
|
||||
kwargs = self._build_thinking_and_kwargs(
|
||||
caps,
|
||||
reasoning_effort,
|
||||
@@ -822,10 +934,13 @@ class AnthropicProvider:
|
||||
extra_params: dict[str, Any] | None = None,
|
||||
deferred_names: frozenset[str] | None = None,
|
||||
capabilities: ModelCapabilities | None = None,
|
||||
replay_reasoning_to_model: bool = True,
|
||||
) -> CompletionResult:
|
||||
_ensure_anthropic()
|
||||
caps = capabilities or self.get_capabilities(model)
|
||||
system_prompt, converted_msgs = self._convert_messages(messages)
|
||||
system_prompt, converted_msgs = self._convert_messages(
|
||||
messages, replay_reasoning_to_model=replay_reasoning_to_model
|
||||
)
|
||||
kwargs = self._build_thinking_and_kwargs(
|
||||
caps,
|
||||
reasoning_effort,
|
||||
@@ -922,6 +1037,25 @@ class AnthropicProvider:
|
||||
}
|
||||
)
|
||||
|
||||
# -- reasoning extraction ------------------------------------------------
|
||||
|
||||
def extract_reasoning_text(
|
||||
self,
|
||||
provider_blocks: list[dict[str, Any]] | None,
|
||||
) -> str:
|
||||
if not isinstance(provider_blocks, list):
|
||||
return ""
|
||||
parts: list[str] = []
|
||||
for block in provider_blocks:
|
||||
if not isinstance(block, dict):
|
||||
continue
|
||||
if block.get("type") != "thinking":
|
||||
continue
|
||||
text = block.get("thinking")
|
||||
if isinstance(text, str) and text:
|
||||
parts.append(text)
|
||||
return _join_reasoning_with_cap(parts)
|
||||
|
||||
|
||||
def _normalize_finish_reason(reason: str) -> str:
|
||||
"""Normalize Anthropic stop reasons to OpenAI-compatible strings."""
|
||||
|
||||
@@ -28,6 +28,7 @@ from turnstone.core.providers._protocol import (
|
||||
ModelCapabilities,
|
||||
StreamChunk,
|
||||
ToolCallDelta,
|
||||
_join_reasoning_with_cap,
|
||||
)
|
||||
|
||||
log = structlog.get_logger(__name__)
|
||||
@@ -156,6 +157,12 @@ class OpenAIChatCompletionsProvider:
|
||||
|
||||
# -- streaming -----------------------------------------------------------
|
||||
|
||||
# Phase 2 of the reasoning-persistence feature plumbs an optional
|
||||
# ``replay_reasoning_to_model`` kwarg through every provider's
|
||||
# ``create_streaming`` / ``create_completion``. OpenAI Chat (and
|
||||
# the local-model server flavours that route through this adapter)
|
||||
# have no first-class reasoning shape on the wire, so the kwarg is
|
||||
# accepted for Protocol conformance and ignored here.
|
||||
def create_streaming(
|
||||
self,
|
||||
*,
|
||||
@@ -170,6 +177,7 @@ class OpenAIChatCompletionsProvider:
|
||||
deferred_names: frozenset[str] | None = None,
|
||||
cancel_ref: list[Any] | None = None,
|
||||
capabilities: ModelCapabilities | None = None,
|
||||
replay_reasoning_to_model: bool = True,
|
||||
) -> Iterator[StreamChunk]:
|
||||
caps = capabilities or self.get_capabilities(model)
|
||||
messages = self._prepare_messages(messages)
|
||||
@@ -299,6 +307,8 @@ class OpenAIChatCompletionsProvider:
|
||||
extra_params: dict[str, Any] | None = None,
|
||||
deferred_names: frozenset[str] | None = None,
|
||||
capabilities: ModelCapabilities | None = None,
|
||||
# See create_streaming above for the Phase 2 reasoning-persistence rationale.
|
||||
replay_reasoning_to_model: bool = True,
|
||||
) -> CompletionResult:
|
||||
caps = capabilities or self.get_capabilities(model)
|
||||
messages = self._prepare_messages(messages)
|
||||
@@ -373,3 +383,34 @@ class OpenAIChatCompletionsProvider:
|
||||
@property
|
||||
def retryable_error_names(self) -> frozenset[str]:
|
||||
return RETRYABLE_ERROR_NAMES
|
||||
|
||||
# -- reasoning extraction ------------------------------------------------
|
||||
|
||||
def extract_reasoning_text(
|
||||
self,
|
||||
provider_blocks: list[dict[str, Any]] | None,
|
||||
) -> str:
|
||||
"""Walk synthetic ``reasoning_text`` blocks (Phase 3 path-3
|
||||
capture) and return the concatenated reasoning text.
|
||||
|
||||
OpenAI Chat Completions has no native reasoning shape on the
|
||||
wire — vLLM ``--reasoning-parser``, llama.cpp
|
||||
``reasoning_format``, and Gemini's OpenAI-compat endpoint all
|
||||
surface reasoning as non-canonical ``delta.reasoning_content``
|
||||
Pydantic extras. ``ChatSession._maybe_synth_reasoning_block``
|
||||
captures these into a single ``{type: "reasoning_text", text,
|
||||
source?}`` block when no native ``provider_blocks`` were
|
||||
emitted. This extractor unwraps those for UI rehydration.
|
||||
"""
|
||||
if not isinstance(provider_blocks, list):
|
||||
return ""
|
||||
parts: list[str] = []
|
||||
for block in provider_blocks:
|
||||
if not isinstance(block, dict):
|
||||
continue
|
||||
if block.get("type") != "reasoning_text":
|
||||
continue
|
||||
text = block.get("text")
|
||||
if isinstance(text, str) and text:
|
||||
parts.append(text)
|
||||
return _join_reasoning_with_cap(parts)
|
||||
|
||||
@@ -33,6 +33,7 @@ OPENAI_CAPABILITIES: dict[str, ModelCapabilities] = {
|
||||
reasoning_effort_values=("minimal", "low", "medium", "high"),
|
||||
default_reasoning_effort="medium",
|
||||
supports_vision=True,
|
||||
supports_reasoning_replay=True,
|
||||
),
|
||||
"gpt-5-mini": ModelCapabilities(
|
||||
context_window=400000,
|
||||
@@ -41,6 +42,7 @@ OPENAI_CAPABILITIES: dict[str, ModelCapabilities] = {
|
||||
reasoning_effort_values=("minimal", "low", "medium", "high"),
|
||||
default_reasoning_effort="medium",
|
||||
supports_vision=True,
|
||||
supports_reasoning_replay=True,
|
||||
),
|
||||
"gpt-5-nano": ModelCapabilities(
|
||||
context_window=400000,
|
||||
@@ -49,6 +51,7 @@ OPENAI_CAPABILITIES: dict[str, ModelCapabilities] = {
|
||||
reasoning_effort_values=("minimal", "low", "medium", "high"),
|
||||
default_reasoning_effort="medium",
|
||||
supports_vision=True,
|
||||
supports_reasoning_replay=True,
|
||||
),
|
||||
# GPT-5 pro — high reasoning only, extended output
|
||||
"gpt-5-pro": ModelCapabilities(
|
||||
@@ -58,6 +61,7 @@ OPENAI_CAPABILITIES: dict[str, ModelCapabilities] = {
|
||||
reasoning_effort_values=("high",),
|
||||
default_reasoning_effort="high",
|
||||
supports_vision=True,
|
||||
supports_reasoning_replay=True,
|
||||
),
|
||||
# GPT-5.1 — temperature OK when reasoning_effort=none (default)
|
||||
"gpt-5.1": ModelCapabilities(
|
||||
@@ -66,6 +70,7 @@ OPENAI_CAPABILITIES: dict[str, ModelCapabilities] = {
|
||||
reasoning_effort_values=("none", "low", "medium", "high"),
|
||||
default_reasoning_effort="none",
|
||||
supports_vision=True,
|
||||
supports_reasoning_replay=True,
|
||||
),
|
||||
# GPT-5.2 — adds xhigh
|
||||
"gpt-5.2": ModelCapabilities(
|
||||
@@ -74,6 +79,7 @@ OPENAI_CAPABILITIES: dict[str, ModelCapabilities] = {
|
||||
reasoning_effort_values=("none", "low", "medium", "high", "xhigh"),
|
||||
default_reasoning_effort="none",
|
||||
supports_vision=True,
|
||||
supports_reasoning_replay=True,
|
||||
),
|
||||
# GPT-5.2 pro — always-reasoning variant
|
||||
"gpt-5.2-pro": ModelCapabilities(
|
||||
@@ -83,6 +89,7 @@ OPENAI_CAPABILITIES: dict[str, ModelCapabilities] = {
|
||||
reasoning_effort_values=("medium", "high", "xhigh"),
|
||||
default_reasoning_effort="medium",
|
||||
supports_vision=True,
|
||||
supports_reasoning_replay=True,
|
||||
),
|
||||
# GPT-5.3 — same capabilities as 5.2 (matches gpt-5.3-chat-latest, codex)
|
||||
"gpt-5.3": ModelCapabilities(
|
||||
@@ -91,6 +98,7 @@ OPENAI_CAPABILITIES: dict[str, ModelCapabilities] = {
|
||||
reasoning_effort_values=("none", "low", "medium", "high", "xhigh"),
|
||||
default_reasoning_effort="none",
|
||||
supports_vision=True,
|
||||
supports_reasoning_replay=True,
|
||||
),
|
||||
# GPT-5.4 — 1M context window, native tool search
|
||||
"gpt-5.4": ModelCapabilities(
|
||||
@@ -100,6 +108,7 @@ OPENAI_CAPABILITIES: dict[str, ModelCapabilities] = {
|
||||
default_reasoning_effort="none",
|
||||
supports_tool_search=True,
|
||||
supports_vision=True,
|
||||
supports_reasoning_replay=True,
|
||||
),
|
||||
# GPT-5.4 pro — always-reasoning, 1M context, native tool search
|
||||
"gpt-5.4-pro": ModelCapabilities(
|
||||
@@ -110,6 +119,7 @@ OPENAI_CAPABILITIES: dict[str, ModelCapabilities] = {
|
||||
default_reasoning_effort="medium",
|
||||
supports_tool_search=True,
|
||||
supports_vision=True,
|
||||
supports_reasoning_replay=True,
|
||||
),
|
||||
# GPT-5.5 — 1M context, native tool search, stronger agentic/tool use
|
||||
"gpt-5.5": ModelCapabilities(
|
||||
@@ -119,6 +129,7 @@ OPENAI_CAPABILITIES: dict[str, ModelCapabilities] = {
|
||||
default_reasoning_effort="none",
|
||||
supports_tool_search=True,
|
||||
supports_vision=True,
|
||||
supports_reasoning_replay=True,
|
||||
),
|
||||
# GPT-5.5 pro — always-reasoning, 1M context, native tool search
|
||||
"gpt-5.5-pro": ModelCapabilities(
|
||||
@@ -129,6 +140,7 @@ OPENAI_CAPABILITIES: dict[str, ModelCapabilities] = {
|
||||
default_reasoning_effort="medium",
|
||||
supports_tool_search=True,
|
||||
supports_vision=True,
|
||||
supports_reasoning_replay=True,
|
||||
),
|
||||
# O-series reasoning models
|
||||
"o1": ModelCapabilities(
|
||||
@@ -137,6 +149,7 @@ OPENAI_CAPABILITIES: dict[str, ModelCapabilities] = {
|
||||
supports_temperature=False,
|
||||
supports_streaming=False,
|
||||
supports_vision=True,
|
||||
supports_reasoning_replay=True,
|
||||
),
|
||||
"o1-mini": ModelCapabilities(
|
||||
context_window=128000,
|
||||
@@ -144,18 +157,21 @@ OPENAI_CAPABILITIES: dict[str, ModelCapabilities] = {
|
||||
supports_temperature=False,
|
||||
supports_streaming=False,
|
||||
supports_vision=True,
|
||||
supports_reasoning_replay=True,
|
||||
),
|
||||
"o3": ModelCapabilities(
|
||||
context_window=200000,
|
||||
max_output_tokens=100000,
|
||||
supports_temperature=False,
|
||||
supports_vision=True,
|
||||
supports_reasoning_replay=True,
|
||||
),
|
||||
"o3-mini": ModelCapabilities(
|
||||
context_window=200000,
|
||||
max_output_tokens=100000,
|
||||
supports_temperature=False,
|
||||
supports_vision=True,
|
||||
supports_reasoning_replay=True,
|
||||
),
|
||||
"o3-pro": ModelCapabilities(
|
||||
context_window=200000,
|
||||
@@ -163,12 +179,14 @@ OPENAI_CAPABILITIES: dict[str, ModelCapabilities] = {
|
||||
supports_temperature=False,
|
||||
supports_streaming=False,
|
||||
supports_vision=True,
|
||||
supports_reasoning_replay=True,
|
||||
),
|
||||
"o4-mini": ModelCapabilities(
|
||||
context_window=200000,
|
||||
max_output_tokens=100000,
|
||||
supports_temperature=False,
|
||||
supports_vision=True,
|
||||
supports_reasoning_replay=True,
|
||||
),
|
||||
# Search models — always search on every request, no reasoning_effort
|
||||
"gpt-5-search-api": ModelCapabilities(
|
||||
|
||||
@@ -32,6 +32,7 @@ from turnstone.core.providers._protocol import (
|
||||
ModelCapabilities,
|
||||
StreamChunk,
|
||||
ToolCallDelta,
|
||||
_join_reasoning_with_cap,
|
||||
)
|
||||
|
||||
log = structlog.get_logger(__name__)
|
||||
@@ -92,16 +93,79 @@ class OpenAIResponsesProvider:
|
||||
@staticmethod
|
||||
def _convert_messages(
|
||||
messages: list[dict[str, Any]],
|
||||
*,
|
||||
replay_reasoning_to_model: bool = False,
|
||||
) -> tuple[str | None, list[dict[str, Any]]]:
|
||||
"""Convert Chat Completions messages to Responses API input items.
|
||||
|
||||
Returns ``(instructions, input_items)`` where *instructions* is the
|
||||
concatenated system/developer messages (or ``None``) and *input_items*
|
||||
is the Responses API ``input`` array.
|
||||
|
||||
When *replay_reasoning_to_model* is True, stored ``_provider_content``
|
||||
reasoning items (``type=="reasoning"``, captured via
|
||||
``include=["reasoning.encrypted_content"]`` on a prior turn)
|
||||
are emitted as ``ResponseReasoningItemParam`` input items
|
||||
immediately before the assistant message they belong to. The
|
||||
SDK explicitly documents this round-trip pattern at
|
||||
``response_reasoning_item_param.py:33-37``: "Be sure to include
|
||||
these items in your ``input`` to the Responses API for
|
||||
subsequent turns of a conversation if you are manually managing
|
||||
context". Even with ``store=False``, ``encrypted_content``
|
||||
round-trips correctly per ``response_create_params.py:70-74``.
|
||||
|
||||
When *replay_reasoning_to_model* is False, reasoning items are silently
|
||||
dropped (they were stripped from the wire by ``sanitize_messages``
|
||||
anyway, but we also skip the input-item emission step).
|
||||
|
||||
Default ``False`` differs intentionally from
|
||||
``AnthropicProvider._convert_messages`` (which defaults
|
||||
``True``). Anthropic's default exists for back-compat with
|
||||
pre-Phase-2 callers who never threaded the kwarg; OpenAI
|
||||
Responses replay is brand-new in Phase 3 and has no such
|
||||
legacy. Production callers (``_build_kwargs``) always pass
|
||||
the resolved flag explicitly, so the default only matters in
|
||||
tests. Conservative-default-False keeps the persist-only
|
||||
capture path live without forcing a downstream cost on every
|
||||
unaware caller.
|
||||
"""
|
||||
# Capture ``_provider_content`` reasoning items per ASSISTANT
|
||||
# ORDINAL (not raw message index) BEFORE sanitization strips
|
||||
# the underscore-prefixed key. Position-by-index would be
|
||||
# unsafe: ``sanitize_messages`` drops orphan tool results
|
||||
# (``_openai_common.py:489-498`` / ``:521-535``) and inserts
|
||||
# synthesized error tool messages for orphaned tool_calls
|
||||
# (``:510-517``). Either operation shifts subsequent message
|
||||
# indices, so a pre-vs-post-sanitize index match would
|
||||
# silently miss reasoning attachments after any tool-message
|
||||
# repair. Assistant messages themselves are never dropped or
|
||||
# duplicated by sanitize_messages — only tool messages — so
|
||||
# the n-th assistant in the original list is invariably the
|
||||
# n-th assistant in the sanitized list. Ordinal-keyed lookup
|
||||
# survives any tool-message length change.
|
||||
reasoning_by_assistant_ordinal: dict[int, list[dict[str, Any]]] = {}
|
||||
if replay_reasoning_to_model:
|
||||
ord_pre = 0
|
||||
for raw_msg in messages:
|
||||
if raw_msg.get("role") != "assistant":
|
||||
continue
|
||||
pc = raw_msg.get("_provider_content")
|
||||
if isinstance(pc, list):
|
||||
items_to_replay = [
|
||||
b for b in pc if isinstance(b, dict) and b.get("type") == "reasoning"
|
||||
]
|
||||
if items_to_replay:
|
||||
reasoning_by_assistant_ordinal[ord_pre] = items_to_replay
|
||||
ord_pre += 1
|
||||
|
||||
messages = sanitize_messages(messages)
|
||||
instructions_parts: list[str] = []
|
||||
items: list[dict[str, Any]] = []
|
||||
# Track assistant ordinal in the SANITIZED list so the lookup
|
||||
# into reasoning_by_assistant_ordinal stays aligned with the
|
||||
# original-list ordinal. See the long comment above for why
|
||||
# ordinal is invariant under sanitization.
|
||||
assistant_ordinal_post = 0
|
||||
|
||||
for msg in messages:
|
||||
role = msg.get("role", "")
|
||||
@@ -129,9 +193,15 @@ class OpenAIResponsesProvider:
|
||||
items.append(item)
|
||||
|
||||
elif role == "assistant":
|
||||
# With store=False, provider_blocks cannot be replayed as input
|
||||
# (output format != input format, and IDs aren't persisted).
|
||||
# Rebuild from the normalized content/tool_calls instead.
|
||||
# Phase 3 reasoning replay: emit stored reasoning items
|
||||
# BEFORE the assistant message they belong to. The SDK
|
||||
# expects reasoning items to appear in input order
|
||||
# alongside the assistant turn that produced them.
|
||||
for r_item in reasoning_by_assistant_ordinal.get(assistant_ordinal_post, []):
|
||||
item_for_input = _reasoning_item_for_input(r_item)
|
||||
if item_for_input is not None:
|
||||
items.append(item_for_input)
|
||||
assistant_ordinal_post += 1
|
||||
|
||||
# Text content → assistant message (plain string for input)
|
||||
if content:
|
||||
@@ -239,11 +309,30 @@ class OpenAIResponsesProvider:
|
||||
reasoning_effort: str,
|
||||
deferred_names: frozenset[str] | None,
|
||||
capabilities: ModelCapabilities | None = None,
|
||||
replay_reasoning_to_model: bool = True,
|
||||
) -> dict[str, Any]:
|
||||
"""Build the kwargs dict for ``client.responses.create/stream``."""
|
||||
"""Build the kwargs dict for ``client.responses.create/stream``.
|
||||
|
||||
``replay_reasoning_to_model`` (Phase 3 of the reasoning-
|
||||
persistence feature) gates two things together:
|
||||
1. ``include=["reasoning.encrypted_content"]`` on the request
|
||||
(so the API surfaces ``encrypted_content`` on reasoning
|
||||
items in ``provider_blocks``).
|
||||
2. ``_convert_messages`` round-tripping stored reasoning items
|
||||
from ``_provider_content`` as ``input`` items on subsequent
|
||||
turns (the SDK's ``ResponseReasoningItemParam`` shape).
|
||||
|
||||
The AND-gate against ``caps.supports_reasoning_replay`` lives
|
||||
upstream in ``ChatSession._resolve_replay_reasoning_to_model``
|
||||
(single source of truth across providers). Production callers
|
||||
always thread the session-resolved flag, so this method trusts
|
||||
the bool it receives.
|
||||
"""
|
||||
caps = capabilities or self.get_capabilities(model)
|
||||
|
||||
instructions, input_items = self._convert_messages(messages)
|
||||
instructions, input_items = self._convert_messages(
|
||||
messages, replay_reasoning_to_model=replay_reasoning_to_model
|
||||
)
|
||||
tools = apply_tool_search(caps, tools, deferred_names)
|
||||
converted_tools = self._convert_tools(tools, caps)
|
||||
|
||||
@@ -261,6 +350,13 @@ class OpenAIResponsesProvider:
|
||||
"store": False,
|
||||
}
|
||||
|
||||
if replay_reasoning_to_model:
|
||||
# SDK doc (response_create_params.py:70-74): with
|
||||
# ``include=["reasoning.encrypted_content"]`` the API
|
||||
# surfaces opaque ``encrypted_content`` on reasoning
|
||||
# items, enabling stateless replay even with ``store=False``.
|
||||
kwargs["include"] = ["reasoning.encrypted_content"]
|
||||
|
||||
if instructions:
|
||||
kwargs["instructions"] = instructions
|
||||
|
||||
@@ -293,6 +389,14 @@ class OpenAIResponsesProvider:
|
||||
deferred_names: frozenset[str] | None = None,
|
||||
cancel_ref: list[Any] | None = None,
|
||||
capabilities: ModelCapabilities | None = None,
|
||||
# Phase 3 reasoning-persistence kwarg — gates
|
||||
# ``include=["reasoning.encrypted_content"]`` on the request
|
||||
# AND ``_convert_messages`` round-tripping stored reasoning
|
||||
# items as input. The AND-gate against
|
||||
# ``caps.supports_reasoning_replay`` lives in
|
||||
# ``ChatSession._resolve_replay_reasoning_to_model`` — single
|
||||
# source of truth across providers.
|
||||
replay_reasoning_to_model: bool = True,
|
||||
) -> Iterator[StreamChunk]:
|
||||
if extra_params:
|
||||
log.debug("openai.responses: extra_params ignored (not supported by Responses API)")
|
||||
@@ -305,6 +409,7 @@ class OpenAIResponsesProvider:
|
||||
reasoning_effort,
|
||||
deferred_names,
|
||||
capabilities=capabilities,
|
||||
replay_reasoning_to_model=replay_reasoning_to_model,
|
||||
)
|
||||
kwargs["stream"] = True
|
||||
|
||||
@@ -474,6 +579,8 @@ class OpenAIResponsesProvider:
|
||||
extra_params: dict[str, Any] | None = None,
|
||||
deferred_names: frozenset[str] | None = None,
|
||||
capabilities: ModelCapabilities | None = None,
|
||||
# See create_streaming above for the Phase 3 reasoning-persistence rationale.
|
||||
replay_reasoning_to_model: bool = True,
|
||||
) -> CompletionResult:
|
||||
if extra_params:
|
||||
log.debug("openai.responses: extra_params ignored (not supported by Responses API)")
|
||||
@@ -486,6 +593,7 @@ class OpenAIResponsesProvider:
|
||||
reasoning_effort,
|
||||
deferred_names,
|
||||
capabilities=capabilities,
|
||||
replay_reasoning_to_model=replay_reasoning_to_model,
|
||||
)
|
||||
|
||||
log.debug(
|
||||
@@ -576,3 +684,78 @@ class OpenAIResponsesProvider:
|
||||
@property
|
||||
def retryable_error_names(self) -> frozenset[str]:
|
||||
return RETRYABLE_ERROR_NAMES
|
||||
|
||||
# -- reasoning extraction ------------------------------------------------
|
||||
|
||||
def extract_reasoning_text(
|
||||
self,
|
||||
provider_blocks: list[dict[str, Any]] | None,
|
||||
) -> str:
|
||||
if not isinstance(provider_blocks, list):
|
||||
return ""
|
||||
parts: list[str] = []
|
||||
for block in provider_blocks:
|
||||
if not isinstance(block, dict):
|
||||
continue
|
||||
if block.get("type") != "reasoning":
|
||||
continue
|
||||
# Per ``ResponseReasoningItem`` (response_reasoning_item.py:31-62):
|
||||
# ``summary`` is the human-readable summary list (always
|
||||
# present), ``content`` is the raw reasoning text list
|
||||
# (optional). We surface both — summary is what the model
|
||||
# produces by default; content is only present on certain
|
||||
# configurations.
|
||||
for s in block.get("summary") or []:
|
||||
if isinstance(s, dict) and s.get("type") == "summary_text":
|
||||
text = s.get("text")
|
||||
if isinstance(text, str) and text:
|
||||
parts.append(text)
|
||||
for c in block.get("content") or []:
|
||||
if isinstance(c, dict) and c.get("type") == "reasoning_text":
|
||||
text = c.get("text")
|
||||
if isinstance(text, str) and text:
|
||||
parts.append(text)
|
||||
return _join_reasoning_with_cap(parts)
|
||||
|
||||
|
||||
def _reasoning_item_for_input(stored: dict[str, Any]) -> dict[str, Any] | None:
|
||||
"""Project a stored reasoning item into ``ResponseReasoningItemParam`` shape.
|
||||
|
||||
The output of a Responses API call carries reasoning items shaped
|
||||
like ``ResponseReasoningItem`` (response_reasoning_item.py:31-62);
|
||||
we stored those verbatim into ``provider_blocks`` via
|
||||
``item.model_dump()`` (``_iter_stream`` line 415-420 captures all
|
||||
output items). To replay them as input on the next turn, the
|
||||
Responses API expects ``ResponseReasoningItemParam``
|
||||
(response_reasoning_item_param.py:31-62) which has the same shape
|
||||
minus ``status`` (a server-only field).
|
||||
|
||||
The ``id``, ``summary``, ``content``, ``encrypted_content``, and
|
||||
``type`` fields all round-trip directly. We project explicitly
|
||||
rather than ``del stored["status"]; return stored`` so callers
|
||||
aren't surprised by mutation of the source dict.
|
||||
|
||||
Returns ``None`` when ``id`` is missing or non-string — per the
|
||||
SDK schema (``response_reasoning_item_param.py:39``) ``id`` is
|
||||
``Required[str]``; sending an empty string would emit a malformed
|
||||
input item that the API may either reject (4xx) or silently
|
||||
misroute. Caller skips appending when None is returned. Items
|
||||
captured via the streaming layer always have ``id`` populated, so
|
||||
this guard is defensive against manually-constructed or migrated
|
||||
storage rows.
|
||||
"""
|
||||
item_id = stored.get("id")
|
||||
if not isinstance(item_id, str) or not item_id:
|
||||
return None
|
||||
out: dict[str, Any] = {
|
||||
"type": "reasoning",
|
||||
"id": item_id,
|
||||
"summary": stored.get("summary") or [],
|
||||
}
|
||||
content = stored.get("content")
|
||||
if content:
|
||||
out["content"] = content
|
||||
encrypted = stored.get("encrypted_content")
|
||||
if encrypted:
|
||||
out["encrypted_content"] = encrypted
|
||||
return out
|
||||
|
||||
@@ -87,6 +87,53 @@ class ModelCapabilities:
|
||||
supports_tool_search: bool = False
|
||||
supports_vision: bool = False
|
||||
thinking_display: str = "" # "summarized" for models that omit thinking by default
|
||||
# Phase 3 reasoning-persistence: gate the per-model
|
||||
# ``replay_reasoning_to_model`` flag. When False, the wire-build
|
||||
# path skips replay regardless of the operator flag (defends
|
||||
# against operators flipping the flag on a model whose API has
|
||||
# no reasoning-replay shape — e.g. OpenAI Chat Completions, where
|
||||
# reasoning is purely server-side and never round-trips). Set
|
||||
# True for: Anthropic models with ``thinking_mode != "none"``,
|
||||
# OpenAI Responses o-series + GPT-5+ (``include=
|
||||
# ["reasoning.encrypted_content"]`` round-trip). Path-3 capture
|
||||
# (Chat Completions / vLLM / llama.cpp / Gemini-compat) is
|
||||
# persist-only and doesn't gate on this flag.
|
||||
supports_reasoning_replay: bool = False
|
||||
|
||||
|
||||
# Operator-friendly UI cap on reasoning text returned from
|
||||
# ``LLMProvider.extract_reasoning_text``. Single source of truth so a
|
||||
# tuning change propagates to every provider's display path uniformly.
|
||||
# Larger reasoning bodies are still stored verbatim in
|
||||
# ``provider_data``; only the rehydrated UI display payload is
|
||||
# truncated.
|
||||
#
|
||||
# Named ``_CHARS`` (not ``_BYTES``) because the cap is enforced via
|
||||
# Python ``str`` slicing, which counts code points. Reasoning text
|
||||
# that happens to contain 4-byte UTF-8 glyphs (CJK, emoji) will
|
||||
# serialise to a larger UTF-8 payload than the constant suggests —
|
||||
# fine for the UI display path (browsers handle the encoded length),
|
||||
# but worth knowing if this is ever wired to a byte-quota system.
|
||||
MAX_REASONING_DISPLAY_CHARS = 64 * 1024
|
||||
|
||||
|
||||
def _join_reasoning_with_cap(parts: list[str]) -> str:
|
||||
"""Join collected reasoning text parts with newline; truncate at the
|
||||
operator-friendly UI cap.
|
||||
|
||||
Shared tail of every provider's ``extract_reasoning_text`` —
|
||||
Anthropic walks ``thinking`` blocks, OpenAI Responses walks
|
||||
``reasoning`` items' ``summary`` + ``content``, OpenAI Chat walks
|
||||
synthetic ``reasoning_text`` blocks. All three converge on the
|
||||
same emit pattern: collect strings, drop empties, join with
|
||||
newline, cap at :data:`MAX_REASONING_DISPLAY_CHARS`.
|
||||
"""
|
||||
if not parts:
|
||||
return ""
|
||||
joined = "\n".join(parts)
|
||||
if len(joined) > MAX_REASONING_DISPLAY_CHARS:
|
||||
return joined[:MAX_REASONING_DISPLAY_CHARS]
|
||||
return joined
|
||||
|
||||
|
||||
def _lookup_capabilities(
|
||||
@@ -133,6 +180,7 @@ class LLMProvider(Protocol):
|
||||
deferred_names: frozenset[str] | None = None,
|
||||
cancel_ref: list[Any] | None = None,
|
||||
capabilities: ModelCapabilities | None = None,
|
||||
replay_reasoning_to_model: bool = True,
|
||||
) -> Iterator[StreamChunk]:
|
||||
"""Create a streaming request, yielding normalized StreamChunks.
|
||||
|
||||
@@ -146,6 +194,21 @@ class LLMProvider(Protocol):
|
||||
stream object (which has a ``.close()`` method) before yielding the
|
||||
first chunk. The caller can then close it from another thread to
|
||||
abort a blocked HTTP read immediately.
|
||||
|
||||
``replay_reasoning_to_model`` defaults to ``True`` here (and on
|
||||
every concrete provider's ``create_streaming`` /
|
||||
``create_completion``) for back-compat with direct callers that
|
||||
haven't been updated to thread the resolver — eval scripts,
|
||||
ad-hoc tests, third-party harnesses. This is INTENTIONALLY
|
||||
the opposite of the operator-side default
|
||||
(``ModelConfig.replay_reasoning_to_model = False``,
|
||||
``model_definitions`` server_default ``0``); the resolver in
|
||||
``ChatSession`` reads the operator value and passes it
|
||||
explicitly, so production call sites never rely on the
|
||||
kwarg-omitted path. Provider-internal helpers (e.g.
|
||||
``OpenAIResponsesProvider._convert_messages``) default ``False``
|
||||
because they're called BY the public entry points — once the
|
||||
resolver-driven value lands, it's already explicit.
|
||||
"""
|
||||
...
|
||||
|
||||
@@ -162,8 +225,17 @@ class LLMProvider(Protocol):
|
||||
extra_params: dict[str, Any] | None = None,
|
||||
deferred_names: frozenset[str] | None = None,
|
||||
capabilities: ModelCapabilities | None = None,
|
||||
replay_reasoning_to_model: bool = True,
|
||||
) -> CompletionResult:
|
||||
"""Create a non-streaming request, returning a normalized result."""
|
||||
"""Create a non-streaming request, returning a normalized result.
|
||||
|
||||
``replay_reasoning_to_model`` mirrors the per-model
|
||||
``model_definitions`` operator flag. Anthropic uses it to
|
||||
gate the verbatim ``_provider_content`` replay (Phase 2);
|
||||
other providers accept the kwarg for Protocol conformance and
|
||||
ignore it (chat-template ``<think>`` content isn't part of
|
||||
their wire-side replay path).
|
||||
"""
|
||||
...
|
||||
|
||||
def convert_tools(
|
||||
@@ -177,3 +249,31 @@ class LLMProvider(Protocol):
|
||||
def retryable_error_names(self) -> frozenset[str]:
|
||||
"""Exception class names that should trigger retry."""
|
||||
...
|
||||
|
||||
def extract_reasoning_text(
|
||||
self,
|
||||
provider_blocks: list[dict[str, Any]] | None,
|
||||
) -> str:
|
||||
"""Return concatenated reasoning text from stored ``provider_blocks``.
|
||||
|
||||
Each provider walks the block types it owns:
|
||||
|
||||
* ``AnthropicProvider`` — ``thinking`` blocks (concatenated
|
||||
``thinking`` text).
|
||||
* ``OpenAIResponsesProvider`` — ``reasoning`` items
|
||||
(concatenated ``summary`` + ``content`` text).
|
||||
* ``OpenAIChatCompletionsProvider`` — synthetic
|
||||
``reasoning_text`` blocks stamped by
|
||||
``ChatSession._maybe_synth_reasoning_block`` for vLLM /
|
||||
llama.cpp / Gemini-OpenAI-compat reasoning capture.
|
||||
* ``GoogleProvider`` — inherits the OpenAI Chat extractor
|
||||
(Gemini's ``/v1beta/openai/`` reasoning surfaces as
|
||||
synthetic ``reasoning_text`` blocks too).
|
||||
|
||||
All providers return the joined text capped at
|
||||
:data:`MAX_REASONING_DISPLAY_CHARS` for UI rendering; full
|
||||
bytes remain in ``provider_data`` for replay. Returns ``""``
|
||||
when the input list contains no recognised reasoning-bearing
|
||||
blocks for the implementing provider.
|
||||
"""
|
||||
...
|
||||
|
||||
+206
-2
@@ -579,12 +579,29 @@ def _render_template(content: str, context: dict[str, str]) -> str:
|
||||
return _TEMPLATE_VAR_RE.sub(_replace, content)
|
||||
|
||||
|
||||
# Block types that carry reasoning content across providers. Used by
|
||||
# ``ChatSession._maybe_synth_reasoning_block`` to decide whether
|
||||
# captured ``reasoning_parts`` need a synthetic ``reasoning_text``
|
||||
# block: if any of these types already appear in ``provider_blocks``,
|
||||
# native lane handles persistence and synthesis is a no-op.
|
||||
# - ``thinking`` / ``redacted_thinking`` — Anthropic native
|
||||
# - ``reasoning`` — OpenAI Responses native
|
||||
# - ``reasoning_text`` — synthetic (path-3 capture; included so
|
||||
# re-running this code path against an already-synthesized list is
|
||||
# idempotent).
|
||||
_REASONING_BEARING_BLOCK_TYPES: frozenset[str] = frozenset(
|
||||
{"thinking", "redacted_thinking", "reasoning", "reasoning_text"}
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# SessionUI protocol — the contract every frontend must implement
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class SessionUI(Protocol):
|
||||
def on_turn_start(self) -> None: ...
|
||||
def on_turn_committed(self) -> None: ...
|
||||
def on_thinking_start(self) -> None: ...
|
||||
def on_thinking_stop(self) -> None: ...
|
||||
def on_reasoning_token(self, text: str) -> None: ...
|
||||
@@ -1094,6 +1111,150 @@ class ChatSession:
|
||||
return self._cached_capabilities
|
||||
return self._resolve_capabilities(p, m, "")
|
||||
|
||||
def _resolve_server_type(self, alias: str | None = None) -> str:
|
||||
"""Read ``server_compat.server_type`` for an alias from the registry.
|
||||
|
||||
Used by :meth:`_maybe_synth_reasoning_block` to tag synthetic
|
||||
path-3 reasoning blocks with their origin server (vllm,
|
||||
llama.cpp, sglang, etc.) — informational today, useful for
|
||||
future per-server replay paths. Returns ``""`` on any lookup
|
||||
miss; the synthetic block then omits the ``source`` field.
|
||||
"""
|
||||
target_alias = alias or self._model_alias or ""
|
||||
if not self._registry or not target_alias:
|
||||
return ""
|
||||
try:
|
||||
cfg: ModelConfig = self._registry.get_config(target_alias)
|
||||
sc = (
|
||||
cfg.capabilities.get("server_compat")
|
||||
if isinstance(cfg.capabilities, dict)
|
||||
else None
|
||||
)
|
||||
if isinstance(sc, dict):
|
||||
return str(sc.get("server_type") or "")
|
||||
except Exception:
|
||||
# Best-effort lookup — synth-block source tagging is
|
||||
# informational, never load-bearing. Log at debug so a
|
||||
# repeated registry-lookup failure during a session shows
|
||||
# up under DEBUG triage but doesn't spam normal logs.
|
||||
log.debug(
|
||||
"_resolve_server_type lookup failed for alias=%s; defaulting to empty",
|
||||
target_alias,
|
||||
exc_info=True,
|
||||
)
|
||||
return ""
|
||||
|
||||
def _maybe_synth_reasoning_block(
|
||||
self,
|
||||
provider_blocks: list[dict[str, Any]],
|
||||
reasoning_parts: list[str],
|
||||
) -> list[dict[str, Any]]:
|
||||
"""Stamp captured ``reasoning_parts`` as a synthetic ``reasoning_text``
|
||||
block when no reasoning-bearing block already appears in
|
||||
``provider_blocks``.
|
||||
|
||||
Anthropic emits native ``thinking`` blocks; OpenAI Responses
|
||||
emits native ``reasoning`` items via ``output_item.done``.
|
||||
Both populate ``provider_blocks`` with reasoning-bearing
|
||||
shapes during streaming and need no synthesis here.
|
||||
|
||||
OpenAI Chat Completions (vLLM ``--reasoning-parser``, llama.cpp
|
||||
``reasoning_format``, Gemini's ``/v1beta/openai/`` endpoint
|
||||
when it surfaces ``reasoning_content``) streams reasoning as
|
||||
``reasoning_delta`` chunks but never emits a reasoning-bearing
|
||||
provider block. Without this synthesis the captured text would
|
||||
be dropped at the end of the stream — visible live, invisible
|
||||
on page reload.
|
||||
|
||||
Crucially, GoogleProvider attaches raw tool_call dicts as
|
||||
``provider_blocks`` on the finish chunk for ``thought_signature``
|
||||
round-trip (``_google.py:_iter_stream``). An earlier version
|
||||
bailed out whenever ``provider_blocks`` was non-empty, which
|
||||
silently lost reasoning text on Google + reasoning_delta turns.
|
||||
The fix tests for reasoning-bearing block types specifically
|
||||
(see ``_REASONING_BEARING_BLOCK_TYPES``) and APPENDS the
|
||||
synthetic block to the existing list rather than replacing it
|
||||
— preserving Google's tool-call fidelity blocks alongside the
|
||||
new synthetic reasoning entry.
|
||||
|
||||
The synthetic block uses ``type="reasoning_text"`` (NOT
|
||||
``"thinking"``) so it falls through Phase 2's
|
||||
``ANTHROPIC_VALID_BLOCK_TYPES`` shape filter on cross-model
|
||||
resumption — protecting against operator-switches from a
|
||||
local-model session to Anthropic, which would otherwise hit
|
||||
Anthropic's input boundary with an unsigned ``thinking`` block.
|
||||
|
||||
The optional ``source`` field tags the block with the
|
||||
originating server (``vllm``, ``llamacpp``, ``sglang``, etc.)
|
||||
when ``ModelConfig.capabilities["server_compat"]["server_type"]``
|
||||
is populated. Reserved for future per-server replay paths
|
||||
(e.g. an operator-flagged path that re-injects synthetic
|
||||
reasoning back into a vllm round-trip) — not consumed today;
|
||||
the field is informational metadata, not dead code.
|
||||
"""
|
||||
text = "".join(reasoning_parts)
|
||||
if not text.strip():
|
||||
return provider_blocks
|
||||
# Native reasoning already present — Anthropic / OpenAI
|
||||
# Responses path. No synth needed; return reference unchanged
|
||||
# so the existing identity contract holds.
|
||||
for b in provider_blocks:
|
||||
if isinstance(b, dict) and b.get("type") in _REASONING_BEARING_BLOCK_TYPES:
|
||||
return provider_blocks
|
||||
block: dict[str, Any] = {
|
||||
"type": "reasoning_text",
|
||||
"text": text,
|
||||
}
|
||||
server_type = self._resolve_server_type()
|
||||
if server_type:
|
||||
block["source"] = server_type
|
||||
# Append rather than replace so non-reasoning fidelity blocks
|
||||
# (e.g. Google tool_calls with thought_signature) survive.
|
||||
return [*provider_blocks, block]
|
||||
|
||||
def _resolve_replay_reasoning_to_model(
|
||||
self,
|
||||
alias: str | None = None,
|
||||
*,
|
||||
caps: ModelCapabilities | None = None,
|
||||
) -> bool:
|
||||
"""Read ``ModelConfig.replay_reasoning_to_model`` for an alias.
|
||||
|
||||
Used by the streaming + non-streaming wire-build paths to gate
|
||||
verbatim reasoning-block replay (Phase 2 of the reasoning-
|
||||
persistence feature). The resolver's miss-fallback is
|
||||
``False``: when no registry / alias is available, or the lookup
|
||||
raises, return ``False`` so the provider-side strip path runs.
|
||||
Losing the strip on operator-flagged-on models would be a
|
||||
worse default than losing the replay on operator-flagged-off
|
||||
models — replaying reasoning text against an unknown operator
|
||||
preference shouldn't happen. The False-on-miss matches the
|
||||
``model_definitions`` server-side default for the column, so
|
||||
cold workstreams behave the same as unconfigured ones.
|
||||
|
||||
When ``caps`` is provided, the operator flag is AND-gated with
|
||||
``caps.supports_reasoning_replay`` so a model lacking the
|
||||
capability silently skips replay even when the operator flag
|
||||
is set. Mirrors the gate in
|
||||
``OpenAIResponsesProvider._build_kwargs`` and protects against
|
||||
future Claude entries (or other Anthropic-shaped surfaces)
|
||||
shipping with ``supports_reasoning_replay=False``. When
|
||||
``caps`` is omitted the resolver returns the operator flag
|
||||
unchanged — back-compat for callers that haven't been updated
|
||||
to thread caps yet.
|
||||
"""
|
||||
target_alias = alias or self._model_alias or ""
|
||||
if not self._registry or not target_alias:
|
||||
return False
|
||||
try:
|
||||
cfg: ModelConfig = self._registry.get_config(target_alias)
|
||||
operator_on = bool(cfg.replay_reasoning_to_model)
|
||||
except Exception:
|
||||
return False
|
||||
if caps is None:
|
||||
return operator_on
|
||||
return operator_on and bool(caps.supports_reasoning_replay)
|
||||
|
||||
def _save_config(self) -> None:
|
||||
"""Persist LLM-affecting config so resumed workstreams behave identically."""
|
||||
save_workstream_config(
|
||||
@@ -2462,6 +2623,7 @@ class ChatSession:
|
||||
reasoning_effort=reasoning_effort,
|
||||
extra_params=self._provider_extra_params(),
|
||||
capabilities=caps,
|
||||
replay_reasoning_to_model=self._resolve_replay_reasoning_to_model(caps=caps),
|
||||
)
|
||||
|
||||
# -- tool search helpers --------------------------------------------------
|
||||
@@ -2629,6 +2791,10 @@ class ChatSession:
|
||||
) -> Iterator[StreamChunk]:
|
||||
"""Attempt a streaming API call with retries on transient errors."""
|
||||
prov = provider or self._provider
|
||||
# Resolve once outside the retry loop — caps don't change per
|
||||
# attempt, and the resolver below threads them into the
|
||||
# ``replay_reasoning_to_model`` AND-gate.
|
||||
resolved_caps = capabilities or self._get_capabilities(prov, model)
|
||||
raw_url = str(getattr(client, "base_url", getattr(client, "_base_url", "?")))
|
||||
safe_url = raw_url.split("?")[0] # strip query params (may contain keys)
|
||||
msg_count = len(msgs)
|
||||
@@ -2662,7 +2828,10 @@ class ChatSession:
|
||||
),
|
||||
deferred_names=self._get_deferred_names(),
|
||||
cancel_ref=self._cancel_ref,
|
||||
capabilities=capabilities or self._get_capabilities(prov, model),
|
||||
capabilities=resolved_caps,
|
||||
replay_reasoning_to_model=self._resolve_replay_reasoning_to_model(
|
||||
model_alias, caps=resolved_caps
|
||||
),
|
||||
)
|
||||
except Exception as e:
|
||||
ename = type(e).__name__
|
||||
@@ -2933,6 +3102,13 @@ class ChatSession:
|
||||
if self.debug:
|
||||
self._debug_print_request(msgs)
|
||||
|
||||
# Reset the per-turn inflight buffers BEFORE entering
|
||||
# the streaming phase so the SSE refresh-resume snapshot
|
||||
# only ever represents the CURRENT in-progress turn —
|
||||
# not prior already-committed turns within this send
|
||||
# loop. Distinct from on_thinking_start (which can fire
|
||||
# twice within a single iteration on compact-retry).
|
||||
self.ui.on_turn_start()
|
||||
self._emit_state("thinking")
|
||||
self.ui.on_thinking_start()
|
||||
try:
|
||||
@@ -3002,6 +3178,12 @@ class ChatSession:
|
||||
self._mark_reminders_delivered()
|
||||
self._print_status_line() # Report usage for EVERY API call
|
||||
self.messages.append(assistant_msg)
|
||||
# Clear per-turn inflight buffers — the assistant
|
||||
# message is now in the history list a refresh would
|
||||
# replay, so the in_progress_snapshot shouldn't re-
|
||||
# render the same text during the next tool-execution
|
||||
# window or the next streaming turn.
|
||||
self.ui.on_turn_committed()
|
||||
self._msg_tokens.append(
|
||||
self._assistant_pending_tokens
|
||||
or max(
|
||||
@@ -3364,6 +3546,20 @@ class ChatSession:
|
||||
)
|
||||
self._msg_tokens.append(1)
|
||||
save_message(self._ws_id, "tool", reason, func_name, tool_call_id=tc_id)
|
||||
# Emit synthetic tool_result so live SSE listeners can
|
||||
# complete the in-DOM tool batch — without this the
|
||||
# coord ``--running`` indicator (added by SSE
|
||||
# tool_info) would spin forever on cancelled batches.
|
||||
# Defensive: we're already on a cancel/error path, so
|
||||
# a UI hook failure must not compound the problem.
|
||||
try:
|
||||
self.ui.on_tool_result(tc_id, func_name, reason, is_error=True)
|
||||
except Exception:
|
||||
log.debug(
|
||||
"session.synthesize_cancelled.ui_emit_failed ws=%s",
|
||||
self._ws_id[:8],
|
||||
exc_info=True,
|
||||
)
|
||||
|
||||
# -- Rewind / retry -------------------------------------------------------
|
||||
|
||||
@@ -3737,7 +3933,12 @@ class ChatSession:
|
||||
)
|
||||
|
||||
# Store raw provider content blocks for multi-turn preservation
|
||||
# (e.g. Anthropic web_search_tool_result with encrypted_content)
|
||||
# (e.g. Anthropic web_search_tool_result with encrypted_content).
|
||||
# Phase 3 path-3 capture: when no native blocks were emitted but
|
||||
# ``reasoning_delta`` chunks accumulated text, synthesize a
|
||||
# ``reasoning_text`` block so the captured reasoning survives
|
||||
# past the live stream and surfaces on history reload.
|
||||
provider_blocks = self._maybe_synth_reasoning_block(provider_blocks, reasoning_parts)
|
||||
if provider_blocks:
|
||||
msg["_provider_content"] = provider_blocks
|
||||
|
||||
@@ -8898,6 +9099,9 @@ class ChatSession:
|
||||
reasoning_effort=reasoning_effort or self.reasoning_effort,
|
||||
extra_params=agent_extra,
|
||||
capabilities=agent_caps,
|
||||
replay_reasoning_to_model=self._resolve_replay_reasoning_to_model(
|
||||
agent_alias, caps=agent_caps
|
||||
),
|
||||
)
|
||||
except Exception as e:
|
||||
ename = type(e).__name__
|
||||
|
||||
@@ -47,6 +47,7 @@ if TYPE_CHECKING:
|
||||
from starlette.routing import BaseRoute
|
||||
|
||||
from turnstone.core.session_manager import SessionManager
|
||||
from turnstone.core.session_ui_base import SessionUIBase
|
||||
from turnstone.core.workstream import Workstream, WorkstreamKind
|
||||
|
||||
log = get_logger(__name__)
|
||||
@@ -1438,7 +1439,24 @@ def make_events_handler(cfg: SessionEndpointConfig) -> Handler:
|
||||
# half-built shape.
|
||||
return JSONResponse({"error": "session has no UI"}, status_code=409)
|
||||
|
||||
client_queue = register()
|
||||
# Register the listener AND snapshot the per-turn inflight
|
||||
# buffers in one atomic-against-writers step. The snapshot
|
||||
# (content / reasoning text-so-far for the current turn) is
|
||||
# yielded as a one-shot ``in_progress_snapshot`` event after
|
||||
# the replay phase; it lets a mid-stream page refresh restore
|
||||
# the partial assistant text without waiting for the response
|
||||
# to complete. ``snap.seq`` is captured to dedup live events
|
||||
# whose ``_seq`` is already in the snapshot payload (race-
|
||||
# free composition with ``on_content_token`` /
|
||||
# ``on_reasoning_token`` writers across the two-lock surface
|
||||
# — see ``register_listener_with_in_progress_snapshot``).
|
||||
# The placeholder-UI guard above (which 409s when
|
||||
# ``_register_listener`` is missing) already proves that
|
||||
# ``ui`` is a ``SessionUIBase`` subclass, so the cast is
|
||||
# tightening the type, not weakening it.
|
||||
ui_base = cast("SessionUIBase", ui)
|
||||
client_queue, in_progress_snap = ui_base.register_listener_with_in_progress_snapshot()
|
||||
snap_seq: int = in_progress_snap["seq"]
|
||||
|
||||
# Per-kind executor for the blocking ``client_queue.get``
|
||||
# wait. Interactive returns its dedicated 200-thread
|
||||
@@ -1499,6 +1517,44 @@ def make_events_handler(cfg: SessionEndpointConfig) -> Handler:
|
||||
ws_id[:8],
|
||||
exc_info=True,
|
||||
)
|
||||
# Refresh-resume tail: emit the current workstream
|
||||
# state (so the composer flips to stop-mode on a mid-
|
||||
# stream refresh — ``state_change`` is the only event
|
||||
# the JS busy machine listens to, and the kind-specific
|
||||
# replay above doesn't yield it) and the in-progress
|
||||
# snapshot (so partial content / reasoning re-renders
|
||||
# immediately, instead of waiting for the next live
|
||||
# token). Both are best-effort — a ws.state read
|
||||
# failure or empty buffers just yields nothing extra.
|
||||
try:
|
||||
cur_state = getattr(ws.state, "value", None)
|
||||
if isinstance(cur_state, str) and cur_state:
|
||||
yield {
|
||||
"data": json.dumps(
|
||||
{
|
||||
"type": "state_change",
|
||||
"state": cur_state,
|
||||
"ws_id": ws_id,
|
||||
}
|
||||
)
|
||||
}
|
||||
except Exception:
|
||||
log.debug(
|
||||
"ws.events.state_change_replay_failed ws=%s",
|
||||
ws_id[:8],
|
||||
exc_info=True,
|
||||
)
|
||||
if in_progress_snap["content"] or in_progress_snap["reasoning"]:
|
||||
yield {
|
||||
"data": json.dumps(
|
||||
{
|
||||
"type": "in_progress_snapshot",
|
||||
"content": in_progress_snap["content"],
|
||||
"reasoning": in_progress_snap["reasoning"],
|
||||
"ws_id": ws_id,
|
||||
}
|
||||
)
|
||||
}
|
||||
# Live phase — drain the per-UI listener queue
|
||||
# until either the workstream closes or the client
|
||||
# disconnects. 5s poll matches pre-lift interactive
|
||||
@@ -1506,6 +1562,14 @@ def make_events_handler(cfg: SessionEndpointConfig) -> Handler:
|
||||
# cancel-detection latency the timeout would otherwise
|
||||
# gate; shortening to 1s 5x'd the wakeup rate without
|
||||
# any client-observable benefit).
|
||||
#
|
||||
# ``_seq`` filter: ``on_content_token`` /
|
||||
# ``on_reasoning_token`` tag each emit with the
|
||||
# per-turn inflight seq counter. Events whose seq is
|
||||
# already covered by the snapshot we just yielded get
|
||||
# dropped to avoid double-rendering. ``_seq`` is
|
||||
# internal plumbing — strip before yielding so the
|
||||
# SDK / JS clients never see it.
|
||||
while True:
|
||||
if await request.is_disconnected():
|
||||
return
|
||||
@@ -1518,6 +1582,20 @@ def make_events_handler(cfg: SessionEndpointConfig) -> Handler:
|
||||
continue # ping keeps the connection alive
|
||||
if event.get("type") == "ws_closed":
|
||||
return
|
||||
# ``_enqueue`` puts ONE dict reference into every
|
||||
# listener queue (no per-listener copy). Multiple
|
||||
# SSE coroutines on the same workstream observe the
|
||||
# same dict; ``yield`` is an await point, so one
|
||||
# listener's ``del event["_seq"]`` would race
|
||||
# another listener's seq-filter read. Shallow-copy
|
||||
# before any mutation so each listener can filter
|
||||
# / strip ``_seq`` without disturbing peers.
|
||||
event = dict(event)
|
||||
seq = event.get("_seq")
|
||||
if seq is not None:
|
||||
if seq <= snap_seq:
|
||||
continue
|
||||
del event["_seq"]
|
||||
yield {"data": json.dumps(event)}
|
||||
finally:
|
||||
_metrics.record_sse_disconnect()
|
||||
@@ -2248,7 +2326,8 @@ def make_history_handler(cfg: SessionEndpointConfig) -> Handler:
|
||||
# shares storage with the other kind. ``cfg.list_kind`` is
|
||||
# guaranteed non-None by the misconfig gate above.
|
||||
storage = getattr(request.app.state, "auth_storage", None)
|
||||
if mgr.get(ws_id) is None:
|
||||
live_session = mgr.get(ws_id)
|
||||
if live_session is None:
|
||||
if storage is None:
|
||||
return JSONResponse({"error": cfg.not_found_label}, status_code=404)
|
||||
try:
|
||||
@@ -2286,6 +2365,7 @@ def make_history_handler(cfg: SessionEndpointConfig) -> Handler:
|
||||
try:
|
||||
from turnstone.core.history_decoration import (
|
||||
decorate_history_messages,
|
||||
extract_reasoning_for_history,
|
||||
load_verdict_indexes,
|
||||
)
|
||||
|
||||
@@ -2298,6 +2378,58 @@ def make_history_handler(cfg: SessionEndpointConfig) -> Handler:
|
||||
# shared mutable state beyond the per-call message
|
||||
# list) so the off-loop hop is free.
|
||||
await asyncio.to_thread(decorate_history_messages, messages, indexes[0], indexes[1])
|
||||
# Active-model ``surface_persisted_reasoning`` flag. Three-tier
|
||||
# resolution so the operator's flag-flip takes effect
|
||||
# uniformly — live session, storage-rehydratable cold
|
||||
# workstream, or unknown workstream:
|
||||
#
|
||||
# 1. Live session in memory → read from its registry
|
||||
# (already-warm path).
|
||||
# 2. Cold workstream → ``workstream_config.model_alias``
|
||||
# persisted at first send (see
|
||||
# ``session_manager.py:628`` rehydrate path) →
|
||||
# resolve through the kind-appropriate registry on
|
||||
# ``app.state``.
|
||||
# 3. Neither available → conservative default ``True``,
|
||||
# matching the migration server_default and the
|
||||
# rehydration default in spec.
|
||||
surface_persisted_reasoning = True
|
||||
resolved_alias = ""
|
||||
resolved_registry: Any = None
|
||||
if live_session is not None:
|
||||
resolved_registry = getattr(live_session, "_registry", None)
|
||||
resolved_alias = getattr(live_session, "_model_alias", "") or ""
|
||||
if not resolved_alias and storage is not None:
|
||||
# Off-loop the sync storage call (mirrors get_workstream
|
||||
# / load_messages / load_verdict_indexes / decorate /
|
||||
# extract_reasoning_for_history above). Preserves the
|
||||
# try/except so a DB failure degrades to the
|
||||
# conservative-default branch instead of bubbling out.
|
||||
try:
|
||||
ws_cfg = (
|
||||
await asyncio.to_thread(storage.load_workstream_config, ws_id) or {}
|
||||
)
|
||||
except Exception:
|
||||
ws_cfg = {}
|
||||
resolved_alias = ws_cfg.get("model_alias") or ""
|
||||
if resolved_registry is None:
|
||||
# Interactive server stores the registry as
|
||||
# ``app.state.registry``; console stores its coord
|
||||
# registry as ``app.state.coord_registry``. The
|
||||
# lifted handler is shared, so we try both.
|
||||
resolved_registry = getattr(request.app.state, "registry", None) or getattr(
|
||||
request.app.state, "coord_registry", None
|
||||
)
|
||||
if resolved_registry is not None and resolved_alias:
|
||||
try:
|
||||
surface_persisted_reasoning = bool(
|
||||
resolved_registry.get_config(resolved_alias).surface_persisted_reasoning
|
||||
)
|
||||
except Exception:
|
||||
surface_persisted_reasoning = True
|
||||
await asyncio.to_thread(
|
||||
extract_reasoning_for_history, messages, surface_persisted_reasoning
|
||||
)
|
||||
except Exception:
|
||||
# Operationally interesting: a persistent decoration
|
||||
# failure (missing migration, driver mismatch, schema
|
||||
|
||||
@@ -42,14 +42,17 @@ log = get_logger(__name__)
|
||||
_DEFAULT_LISTENER_QUEUE_MAX = 500
|
||||
|
||||
|
||||
# Cap on the per-turn assistant content accumulator. The accumulator
|
||||
# is piggybacked onto the ``ws_state:idle`` broadcast payload so the
|
||||
# cluster collector / dashboard can render the freshly-emitted assistant
|
||||
# turn without round-tripping storage; capping it keeps a runaway turn
|
||||
# from ballooning the broadcast event past the listener queues' size
|
||||
# budget. Lifted from WebUI in the rich ``ws_state`` payload work so
|
||||
# coord broadcasts hit the same ceiling.
|
||||
_MAX_TURN_CONTENT_CHARS = 256 * 1024
|
||||
# Cap on the assistant content / reasoning accumulators. Used by two
|
||||
# independent buffer pairs:
|
||||
# - ``_ws_turn_content`` (multi-turn, drained at idle/error) — the
|
||||
# IDLE-piggyback payload the cluster collector / dashboard renders
|
||||
# without round-tripping storage.
|
||||
# - ``_ws_inflight_content`` / ``_ws_inflight_reasoning`` (per-turn,
|
||||
# drained at :meth:`on_turn_start`) — the SSE refresh-resume
|
||||
# snapshot a reconnecting client sees for the in-progress turn.
|
||||
# 512 KiB gives headroom for current commercial models; bump if a
|
||||
# single turn legitimately exceeds it.
|
||||
_MAX_TURN_CONTENT_CHARS = 512 * 1024
|
||||
|
||||
|
||||
def fire_judge_verdict_metric(
|
||||
@@ -206,8 +209,29 @@ class SessionUIBase:
|
||||
# the ``ws_state:idle`` broadcast so the dashboard renders the
|
||||
# turn without an extra storage round-trip. Cleared on IDLE /
|
||||
# ERROR transitions by :meth:`snapshot_and_consume_state_payload`.
|
||||
# Multi-turn (per-``send()``) — accumulates across all internal
|
||||
# turns within one user-facing send.
|
||||
self._ws_turn_content: list[str] = []
|
||||
self._ws_turn_content_size: int = 0
|
||||
# Per-turn inflight accumulators: the in-progress turn's content
|
||||
# + reasoning, exposed to a reconnecting SSE client via the
|
||||
# ``in_progress_snapshot`` event so a mid-stream page refresh
|
||||
# restores the partial assistant text. Reset at the start of
|
||||
# each turn by :meth:`on_turn_start` (separate from the multi-
|
||||
# turn IDLE-piggyback buffer above so prior committed turns
|
||||
# don't leak into the snapshot and double-render against the
|
||||
# replayed history). ``_ws_inflight_seq`` is a monotonic
|
||||
# counter incremented on EVERY emit (even when the cap
|
||||
# rejected the buffer append) so a subscriber registering
|
||||
# after the cap is hit doesn't have subsequent live tokens
|
||||
# filter-dropped against a stalled ``snap_seq`` — the events
|
||||
# handler dedups live events whose ``_seq`` is at-or-below
|
||||
# the snapshot's seq (already in the snapshot payload).
|
||||
self._ws_inflight_content: list[str] = []
|
||||
self._ws_inflight_content_size: int = 0
|
||||
self._ws_inflight_reasoning: list[str] = []
|
||||
self._ws_inflight_reasoning_size: int = 0
|
||||
self._ws_inflight_seq: int = 0
|
||||
# Last broadcast (activity, activity_state) tuple — used by
|
||||
# :meth:`_broadcast_activity` overrides to dedup back-to-back
|
||||
# identical activity ticks. Tool-heavy turns can fire many
|
||||
@@ -270,6 +294,56 @@ class SessionUIBase:
|
||||
with self._listeners_lock, contextlib.suppress(ValueError):
|
||||
self._listeners.remove(client_queue)
|
||||
|
||||
def register_listener_with_in_progress_snapshot(
|
||||
self, maxsize: int = _DEFAULT_LISTENER_QUEUE_MAX
|
||||
) -> tuple[queue.Queue[dict[str, Any]], dict[str, Any]]:
|
||||
"""Register a listener AND snapshot the per-turn inflight buffers.
|
||||
|
||||
Used by :func:`make_events_handler` so a fresh SSE subscriber
|
||||
connecting mid-stream can be told the in-progress turn's content
|
||||
and reasoning text-so-far in a one-shot ``in_progress_snapshot``
|
||||
event, on top of the kind-specific replay (history / pending).
|
||||
|
||||
Race-free composition with the on-token writers, even though
|
||||
``on_content_token`` / ``on_reasoning_token`` cross two locks
|
||||
(``_ws_lock`` for the buffer append, ``_listeners_lock`` for
|
||||
the fan-out enqueue). The trick is the seq counter —
|
||||
``_ws_inflight_seq`` is incremented under ``_ws_lock`` on
|
||||
every emit (even when the cap rejected the append, so a
|
||||
subscriber that registers after the cap is hit doesn't have
|
||||
subsequent live tokens filter-dropped against a stalled
|
||||
snap_seq). This method captures it alongside the buffer
|
||||
contents under the same ``_ws_lock``, and the events handler's
|
||||
live drain drops any incoming event whose ``_seq`` is at-or-
|
||||
below the captured ``snap.seq`` (already in the snapshot
|
||||
payload). Lock acquisition order: ``_listeners_lock`` (inside
|
||||
:meth:`_register_listener`) is taken and released first, THEN
|
||||
``_ws_lock`` for the snapshot copy. Sequential — no nesting,
|
||||
no deadlock with the writer's reverse order.
|
||||
|
||||
Returns ``(client_queue, snapshot_dict)`` where ``snapshot_dict``
|
||||
has keys ``content`` (str), ``reasoning`` (str), ``seq`` (int).
|
||||
Caller checks for non-empty content / reasoning to decide
|
||||
whether to yield the event at all (empty snapshots are common
|
||||
between turns and on freshly-opened workstreams).
|
||||
|
||||
Joins the captured fragments OUTSIDE the lock — bounded at
|
||||
``_MAX_TURN_CONTENT_CHARS`` but still O(n) over fragments, so
|
||||
worth not blocking concurrent on-token writers for the
|
||||
duration. The shallow ``list(...)`` copy under the lock means
|
||||
subsequent appends to the live buffer don't mutate our view.
|
||||
"""
|
||||
client_queue = self._register_listener(maxsize=maxsize)
|
||||
with self._ws_lock:
|
||||
captured_content = list(self._ws_inflight_content)
|
||||
captured_reasoning = list(self._ws_inflight_reasoning)
|
||||
snap_seq = self._ws_inflight_seq
|
||||
return client_queue, {
|
||||
"content": "".join(captured_content),
|
||||
"reasoning": "".join(captured_reasoning),
|
||||
"seq": snap_seq,
|
||||
}
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Approval / plan blocking gates
|
||||
# ------------------------------------------------------------------
|
||||
@@ -1165,6 +1239,56 @@ class SessionUIBase:
|
||||
# of the shared writes.
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def _reset_inflight_buffers_locked(self) -> None:
|
||||
"""Clear the per-turn inflight content + reasoning. Caller holds ``_ws_lock``.
|
||||
|
||||
``_ws_inflight_seq`` is INTENTIONALLY not reset — it must
|
||||
remain monotonically increasing for the lifetime of the UI so
|
||||
a long-lived SSE subscriber's ``snap_seq`` cutoff stays a
|
||||
valid high-water mark across turn boundaries. If we reset
|
||||
seq=0 at every turn, turn N+1's first M tokens (M = the
|
||||
snap_seq the subscriber captured mid-turn-N) would all carry
|
||||
``_seq <= snap_seq`` and get silently dropped by the dedup
|
||||
filter in :func:`make_events_handler`. Seq is just a wire-
|
||||
format dedup tag — its absolute value doesn't matter, only
|
||||
that it's monotonic.
|
||||
"""
|
||||
self._ws_inflight_content = []
|
||||
self._ws_inflight_content_size = 0
|
||||
self._ws_inflight_reasoning = []
|
||||
self._ws_inflight_reasoning_size = 0
|
||||
|
||||
def on_turn_start(self) -> None:
|
||||
"""Reset inflight buffers at the top of each ``send()`` iteration.
|
||||
|
||||
Defensive — covers the FIRST iteration of a fresh ``send()``
|
||||
where a prior ``send()`` may have crashed mid-stream and left
|
||||
stale content in the buffers. Steady-state, the buffers are
|
||||
already empty at this point because :meth:`on_turn_committed`
|
||||
cleared them right after the last assistant message committed.
|
||||
"""
|
||||
with self._ws_lock:
|
||||
self._reset_inflight_buffers_locked()
|
||||
|
||||
def on_turn_committed(self) -> None:
|
||||
"""Reset inflight buffers right after the assistant message commits.
|
||||
|
||||
The committed message is now in ``session.messages`` (the
|
||||
history source for SSE replay), so leaving the same text in
|
||||
the inflight buffer would double-render it on a refresh during
|
||||
the post-commit tool-execution window — history shows the
|
||||
committed turn AND the ``in_progress_snapshot`` shows the
|
||||
same text again.
|
||||
|
||||
Future cross-turn reasoning persistence (some commercial
|
||||
models want reasoning preserved across user turns, not just
|
||||
within the current send) will override this hook to copy
|
||||
inflight reasoning to a per-message persistence store BEFORE
|
||||
clearing — keeping the `current vs historical` boundary clean.
|
||||
"""
|
||||
with self._ws_lock:
|
||||
self._reset_inflight_buffers_locked()
|
||||
|
||||
def on_thinking_start(self) -> None:
|
||||
"""Track that the model is thinking; broadcast activity + enqueue."""
|
||||
with self._ws_lock:
|
||||
@@ -1177,25 +1301,72 @@ class SessionUIBase:
|
||||
self._enqueue({"type": "thinking_stop"})
|
||||
|
||||
def on_reasoning_token(self, text: str) -> None:
|
||||
self._enqueue({"type": "reasoning", "text": text})
|
||||
"""Append to the inflight reasoning buffer (capped) + enqueue.
|
||||
|
||||
Mirrors :meth:`on_content_token`'s shape. ``_ws_inflight_seq``
|
||||
advances on EVERY emit — even when the buffer cap rejected
|
||||
the append — so the dedup filter in :func:`make_events_handler`
|
||||
stays correct for subscribers that register after the cap is
|
||||
hit. If seq stalled at the high-water-pre-cap, those late
|
||||
subscribers would capture ``snap_seq == high-water`` and
|
||||
every subsequent live token (with the same stalled seq)
|
||||
would be filter-dropped as "already in your snapshot",
|
||||
silently losing the rest of the stream. The cap is a
|
||||
buffer-size limit, NOT a "stop streaming" signal.
|
||||
|
||||
Tokens past the cap are absent from ``snap.reasoning`` (the
|
||||
snapshot text was truncated at cap) but the live stream
|
||||
continues normally past them — refresh-after-cap renders the
|
||||
snapshot text up to the cap and then live tokens past it,
|
||||
with a visual gap equal to the past-cap chunk. No silent
|
||||
drop of subsequent tokens.
|
||||
"""
|
||||
with self._ws_lock:
|
||||
if self._ws_inflight_reasoning_size < _MAX_TURN_CONTENT_CHARS:
|
||||
self._ws_inflight_reasoning.append(text)
|
||||
self._ws_inflight_reasoning_size += len(text)
|
||||
self._ws_inflight_seq += 1
|
||||
seq = self._ws_inflight_seq
|
||||
self._enqueue({"type": "reasoning", "text": text, "_seq": seq})
|
||||
|
||||
def on_content_token(self, text: str) -> None:
|
||||
"""Append to the turn-content accumulator (capped) + enqueue.
|
||||
"""Append to both turn-content buffers (capped) + enqueue.
|
||||
|
||||
The cap-check + append + size-update run under ``_ws_lock``
|
||||
so a concurrent :meth:`snapshot_and_consume_state_payload`
|
||||
IDLE/ERROR drain can't see a torn list mid-append. In
|
||||
production this is single-writer-per-ws (the worker thread)
|
||||
but the snapshot reader runs from coord's adapter via
|
||||
``mgr.set_state``; without the lock the writer's append
|
||||
could land in an orphaned list reference the snapshot just
|
||||
swapped out. Lock hold is microseconds.
|
||||
Writes under ``_ws_lock`` to two independent buffers:
|
||||
- ``_ws_turn_content`` (multi-turn, drained at idle/error)
|
||||
— fuels the dashboard's IDLE-piggyback content payload.
|
||||
- ``_ws_inflight_content`` (per-turn, drained at
|
||||
:meth:`on_turn_start`) — fuels the SSE ``in_progress_snapshot``
|
||||
event a reconnecting client sees on mid-stream refresh.
|
||||
|
||||
Both caps are checked independently. ``_ws_inflight_seq``
|
||||
advances on EVERY emit — even when the inflight cap rejected
|
||||
the append — so a subscriber that registers after the cap is
|
||||
hit doesn't have every subsequent live token filter-dropped
|
||||
against a stalled ``snap_seq``. See
|
||||
:meth:`on_reasoning_token` for the full rationale.
|
||||
|
||||
The cap-check + append + size-update + seq-bump run under
|
||||
``_ws_lock`` so a concurrent
|
||||
:meth:`snapshot_and_consume_state_payload` IDLE/ERROR drain or
|
||||
a concurrent :meth:`register_listener_with_in_progress_snapshot`
|
||||
can't see a torn list mid-append. In production this is
|
||||
single-writer-per-ws (the worker thread) but the snapshot
|
||||
reader runs from coord's adapter via ``mgr.set_state``;
|
||||
without the lock the writer's append could land in an
|
||||
orphaned list reference the snapshot just swapped out. Lock
|
||||
hold is microseconds.
|
||||
"""
|
||||
with self._ws_lock:
|
||||
if self._ws_turn_content_size < _MAX_TURN_CONTENT_CHARS:
|
||||
self._ws_turn_content.append(text)
|
||||
self._ws_turn_content_size += len(text)
|
||||
self._enqueue({"type": "content", "text": text})
|
||||
if self._ws_inflight_content_size < _MAX_TURN_CONTENT_CHARS:
|
||||
self._ws_inflight_content.append(text)
|
||||
self._ws_inflight_content_size += len(text)
|
||||
self._ws_inflight_seq += 1
|
||||
seq = self._ws_inflight_seq
|
||||
self._enqueue({"type": "content", "text": text, "_seq": seq})
|
||||
|
||||
def on_stream_end(self) -> None:
|
||||
with self._ws_lock:
|
||||
@@ -1460,9 +1631,18 @@ class SessionUIBase:
|
||||
captured_content = self._ws_turn_content
|
||||
self._ws_turn_content = []
|
||||
self._ws_turn_content_size = 0
|
||||
# Drain the per-turn inflight buffers too so a refresh
|
||||
# post-cancel doesn't double-render against history.
|
||||
# On the success path :meth:`on_turn_committed` already
|
||||
# cleared inflight at ``messages.append`` time, so this
|
||||
# is a no-op there. On cancel/error/exception paths
|
||||
# nothing else clears inflight — this single chokepoint
|
||||
# covers them all.
|
||||
self._reset_inflight_buffers_locked()
|
||||
elif state == "error":
|
||||
self._ws_turn_content = []
|
||||
self._ws_turn_content_size = 0
|
||||
self._reset_inflight_buffers_locked()
|
||||
# Join outside the lock — bounded at _MAX_TURN_CONTENT_CHARS but
|
||||
# still O(n) over the captured fragments, so worth not blocking
|
||||
# concurrent on_content_token writers for the duration.
|
||||
|
||||
@@ -419,11 +419,13 @@ def _build_registry() -> dict[str, SettingDef]:
|
||||
"judge.model",
|
||||
"str",
|
||||
"",
|
||||
"Model for LLM judge (empty = same as session)",
|
||||
"Model alias for LLM judge (empty = same as session)",
|
||||
"judge",
|
||||
help="The judge can use a different AI model than the main conversation. Leave empty "
|
||||
"to use the same model (self-consistency), or specify a different model for "
|
||||
"cross-model evaluation.",
|
||||
help="The judge can use a different AI model than the main conversation. "
|
||||
"Specify a registered alias from the Models tab, or leave empty to use the "
|
||||
"same model as the session (self-consistency). Values that aren’t "
|
||||
"registered aliases inherit the session model and log a warning — "
|
||||
"register the model in the Models tab and reference it by alias.",
|
||||
),
|
||||
SettingDef(
|
||||
"judge.confidence_threshold",
|
||||
@@ -634,13 +636,15 @@ def _build_registry() -> dict[str, SettingDef]:
|
||||
"coordinator.reasoning_effort",
|
||||
"str",
|
||||
"medium",
|
||||
"Reasoning effort for coordinator sessions",
|
||||
"Reasoning effort for coordinator sessions (empty = inherit from model.reasoning_effort)",
|
||||
"coordinator",
|
||||
choices=["none", "minimal", "low", "medium", "high", "xhigh", "max"],
|
||||
choices=["", "none", "minimal", "low", "medium", "high", "xhigh", "max"],
|
||||
help="Reasoning effort for coordinator sessions. Coordinators benefit from "
|
||||
"medium-or-higher effort when juggling multiple child workstreams. Use "
|
||||
"'low' only when your coordinator handles simple, one-off dispatch "
|
||||
"workflows.",
|
||||
"workflows. (Empty here means “inherit” — the per-model "
|
||||
"override on the alias wins, otherwise model.reasoning_effort. Use "
|
||||
"‘none’ to actually disable reasoning.)",
|
||||
),
|
||||
SettingDef(
|
||||
"coordinator.max_active",
|
||||
|
||||
@@ -4167,6 +4167,8 @@ class PostgreSQLBackend:
|
||||
temperature: float | None = None,
|
||||
max_tokens: int | None = None,
|
||||
reasoning_effort: str | None = None,
|
||||
surface_persisted_reasoning: bool = True,
|
||||
replay_reasoning_to_model: bool = False,
|
||||
) -> None:
|
||||
from sqlalchemy.dialects import postgresql
|
||||
|
||||
@@ -4187,6 +4189,8 @@ class PostgreSQLBackend:
|
||||
temperature=temperature,
|
||||
max_tokens=max_tokens,
|
||||
reasoning_effort=reasoning_effort,
|
||||
surface_persisted_reasoning=1 if surface_persisted_reasoning else 0,
|
||||
replay_reasoning_to_model=1 if replay_reasoning_to_model else 0,
|
||||
created_by=created_by,
|
||||
created=now,
|
||||
updated=now,
|
||||
@@ -4205,7 +4209,9 @@ class PostgreSQLBackend:
|
||||
).fetchone()
|
||||
if row is None:
|
||||
return None
|
||||
return _row_to_dict(row, "enabled")
|
||||
return _row_to_dict(
|
||||
row, "enabled", "surface_persisted_reasoning", "replay_reasoning_to_model"
|
||||
)
|
||||
|
||||
def get_model_definition_by_alias(self, alias: str) -> dict[str, Any] | None:
|
||||
|
||||
@@ -4215,7 +4221,9 @@ class PostgreSQLBackend:
|
||||
).fetchone()
|
||||
if row is None:
|
||||
return None
|
||||
return _row_to_dict(row, "enabled")
|
||||
return _row_to_dict(
|
||||
row, "enabled", "surface_persisted_reasoning", "replay_reasoning_to_model"
|
||||
)
|
||||
|
||||
def list_model_definitions(self, enabled_only: bool = False) -> list[dict[str, Any]]:
|
||||
|
||||
@@ -4224,7 +4232,12 @@ class PostgreSQLBackend:
|
||||
if enabled_only:
|
||||
q = q.where(model_definitions.c.enabled == 1)
|
||||
rows = conn.execute(q).fetchall()
|
||||
return [_row_to_dict(r, "enabled") for r in rows]
|
||||
return [
|
||||
_row_to_dict(
|
||||
r, "enabled", "surface_persisted_reasoning", "replay_reasoning_to_model"
|
||||
)
|
||||
for r in rows
|
||||
]
|
||||
|
||||
def update_model_definition(self, definition_id: str, **fields: Any) -> bool:
|
||||
|
||||
@@ -4232,6 +4245,12 @@ class PostgreSQLBackend:
|
||||
fields["updated"] = datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%S")
|
||||
if "enabled" in fields:
|
||||
fields["enabled"] = 1 if fields["enabled"] else 0
|
||||
if "surface_persisted_reasoning" in fields:
|
||||
fields["surface_persisted_reasoning"] = (
|
||||
1 if fields["surface_persisted_reasoning"] else 0
|
||||
)
|
||||
if "replay_reasoning_to_model" in fields:
|
||||
fields["replay_reasoning_to_model"] = 1 if fields["replay_reasoning_to_model"] else 0
|
||||
with self._conn() as conn:
|
||||
result = conn.execute(
|
||||
sa.update(model_definitions)
|
||||
|
||||
@@ -1853,6 +1853,8 @@ class StorageBackend(Protocol):
|
||||
temperature: float | None = None,
|
||||
max_tokens: int | None = None,
|
||||
reasoning_effort: str | None = None,
|
||||
surface_persisted_reasoning: bool = True,
|
||||
replay_reasoning_to_model: bool = False,
|
||||
) -> None:
|
||||
"""Create a model definition. No-op if definition_id already exists."""
|
||||
...
|
||||
|
||||
@@ -660,6 +660,8 @@ model_definitions = sa.Table(
|
||||
sa.Column("temperature", sa.Float, nullable=True),
|
||||
sa.Column("max_tokens", sa.Integer, nullable=True),
|
||||
sa.Column("reasoning_effort", sa.Text, nullable=True),
|
||||
sa.Column("surface_persisted_reasoning", sa.Integer, nullable=False, server_default="1"),
|
||||
sa.Column("replay_reasoning_to_model", sa.Integer, nullable=False, server_default="0"),
|
||||
sa.Column("created_by", sa.Text, nullable=False, server_default=""),
|
||||
sa.Column("created", sa.Text, nullable=False),
|
||||
sa.Column("updated", sa.Text, nullable=False),
|
||||
|
||||
@@ -4313,6 +4313,8 @@ class SQLiteBackend:
|
||||
temperature: float | None = None,
|
||||
max_tokens: int | None = None,
|
||||
reasoning_effort: str | None = None,
|
||||
surface_persisted_reasoning: bool = True,
|
||||
replay_reasoning_to_model: bool = False,
|
||||
) -> None:
|
||||
|
||||
now = datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%S")
|
||||
@@ -4332,6 +4334,8 @@ class SQLiteBackend:
|
||||
"temperature": temperature,
|
||||
"max_tokens": max_tokens,
|
||||
"reasoning_effort": reasoning_effort,
|
||||
"surface_persisted_reasoning": 1 if surface_persisted_reasoning else 0,
|
||||
"replay_reasoning_to_model": (1 if replay_reasoning_to_model else 0),
|
||||
"created_by": created_by,
|
||||
"created": now,
|
||||
"updated": now,
|
||||
@@ -4349,7 +4353,9 @@ class SQLiteBackend:
|
||||
).fetchone()
|
||||
if row is None:
|
||||
return None
|
||||
return _row_to_dict(row, "enabled")
|
||||
return _row_to_dict(
|
||||
row, "enabled", "surface_persisted_reasoning", "replay_reasoning_to_model"
|
||||
)
|
||||
|
||||
def get_model_definition_by_alias(self, alias: str) -> dict[str, Any] | None:
|
||||
|
||||
@@ -4359,7 +4365,9 @@ class SQLiteBackend:
|
||||
).fetchone()
|
||||
if row is None:
|
||||
return None
|
||||
return _row_to_dict(row, "enabled")
|
||||
return _row_to_dict(
|
||||
row, "enabled", "surface_persisted_reasoning", "replay_reasoning_to_model"
|
||||
)
|
||||
|
||||
def list_model_definitions(self, enabled_only: bool = False) -> list[dict[str, Any]]:
|
||||
|
||||
@@ -4368,7 +4376,12 @@ class SQLiteBackend:
|
||||
if enabled_only:
|
||||
q = q.where(model_definitions.c.enabled == 1)
|
||||
rows = conn.execute(q).fetchall()
|
||||
return [_row_to_dict(r, "enabled") for r in rows]
|
||||
return [
|
||||
_row_to_dict(
|
||||
r, "enabled", "surface_persisted_reasoning", "replay_reasoning_to_model"
|
||||
)
|
||||
for r in rows
|
||||
]
|
||||
|
||||
def update_model_definition(self, definition_id: str, **fields: Any) -> bool:
|
||||
|
||||
@@ -4376,6 +4389,12 @@ class SQLiteBackend:
|
||||
fields["updated"] = datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%S")
|
||||
if "enabled" in fields:
|
||||
fields["enabled"] = 1 if fields["enabled"] else 0
|
||||
if "surface_persisted_reasoning" in fields:
|
||||
fields["surface_persisted_reasoning"] = (
|
||||
1 if fields["surface_persisted_reasoning"] else 0
|
||||
)
|
||||
if "replay_reasoning_to_model" in fields:
|
||||
fields["replay_reasoning_to_model"] = 1 if fields["replay_reasoning_to_model"] else 0
|
||||
with self._conn() as conn:
|
||||
result = conn.execute(
|
||||
sa.update(model_definitions)
|
||||
|
||||
@@ -194,6 +194,8 @@ MODEL_DEFINITION_MUTABLE = frozenset(
|
||||
"temperature",
|
||||
"max_tokens",
|
||||
"reasoning_effort",
|
||||
"surface_persisted_reasoning",
|
||||
"replay_reasoning_to_model",
|
||||
}
|
||||
)
|
||||
PROMPT_POLICY_MUTABLE = frozenset({"name", "content", "tool_gate", "priority", "enabled"})
|
||||
|
||||
@@ -0,0 +1,67 @@
|
||||
"""Add per-model reasoning-persistence flags to model_definitions.
|
||||
|
||||
Adds two boolean (integer-coded) operator knobs:
|
||||
|
||||
* ``surface_persisted_reasoning`` (default ``1``) — when true, the
|
||||
history-build path extracts stored reasoning text from
|
||||
``provider_data`` and surfaces it on each assistant message dict so a
|
||||
page refresh re-renders the reasoning bubble. **Storage of the
|
||||
reasoning bytes happens regardless of this flag** — it only controls
|
||||
the extract-and-include step in ``_build_history`` /
|
||||
``decorate_history_messages``. The pre-rename column was
|
||||
``persist_reasoning``; the rename to ``surface_persisted_reasoning``
|
||||
happened in the review-fix wave because the original name implied a
|
||||
storage-control switch when the flag is purely about UI rehydration.
|
||||
* ``replay_reasoning_to_model`` (default ``0``) — when true, the
|
||||
wire-build path keeps reasoning blocks in the outgoing
|
||||
``_provider_content`` lane on subsequent provider calls. False is the
|
||||
conservative default: spec compliance, lower per-turn cost, no behaviour
|
||||
change vs. the pre-flag default. **Phase 1 stores the column but does
|
||||
not consume it on the wire**; Phase 2 wires the strip branch in
|
||||
``_anthropic.py``'s ``_convert_messages``.
|
||||
|
||||
Mirrors the ``enabled`` column pattern (``_schema.py:659``):
|
||||
``NOT NULL`` with an integer ``server_default`` so existing rows pick up
|
||||
the conservative defaults silently on upgrade. Distinct from
|
||||
``temperature`` / ``max_tokens`` / ``reasoning_effort`` (migration 036)
|
||||
which are nullable inherit-from-cluster sampling overrides — these are
|
||||
operator-toggle booleans, never NULL.
|
||||
|
||||
Revision ID: 052
|
||||
Revises: 051
|
||||
Create Date: 2026-05-08
|
||||
"""
|
||||
|
||||
import sqlalchemy as sa
|
||||
from alembic import op
|
||||
|
||||
revision = "052"
|
||||
down_revision = "051"
|
||||
branch_labels = None
|
||||
depends_on = None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
with op.batch_alter_table("model_definitions") as batch:
|
||||
batch.add_column(
|
||||
sa.Column(
|
||||
"surface_persisted_reasoning",
|
||||
sa.Integer,
|
||||
nullable=False,
|
||||
server_default="1",
|
||||
)
|
||||
)
|
||||
batch.add_column(
|
||||
sa.Column(
|
||||
"replay_reasoning_to_model",
|
||||
sa.Integer,
|
||||
nullable=False,
|
||||
server_default="0",
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
with op.batch_alter_table("model_definitions") as batch:
|
||||
batch.drop_column("replay_reasoning_to_model")
|
||||
batch.drop_column("surface_persisted_reasoning")
|
||||
@@ -96,6 +96,12 @@ class EvolutionNode:
|
||||
class NullUI:
|
||||
"""UI adapter that discards all output. Used by HeadlessSession."""
|
||||
|
||||
def on_turn_start(self) -> None:
|
||||
pass
|
||||
|
||||
def on_turn_committed(self) -> None:
|
||||
pass
|
||||
|
||||
def on_thinking_start(self) -> None:
|
||||
pass
|
||||
|
||||
|
||||
@@ -49,6 +49,26 @@ class ConnectedEvent(ServerEvent):
|
||||
|
||||
@dataclass
|
||||
class HistoryEvent(ServerEvent):
|
||||
"""SSE replay payload — the per-tab message backlog on connect.
|
||||
|
||||
Each entry in ``messages`` is a per-message dict the frontend
|
||||
consumes directly. Notable optional keys:
|
||||
|
||||
* ``role`` (``"user"`` / ``"assistant"`` / ``"tool"``)
|
||||
* ``content`` — string for text turns, list for image / document parts
|
||||
* ``tool_calls`` — list of ``{id, name, arguments, verdict?,
|
||||
output_assessment?}`` (assistant turns)
|
||||
* ``tool_call_id`` — the originating call's id (tool turns)
|
||||
* ``reminders`` — metacognitive nudge bubbles (user / tool channels)
|
||||
* ``advisories`` — extracted ``UserInterjection`` payloads (tool
|
||||
turns whose envelope wrapped queued-message advisories)
|
||||
* ``reasoning`` — concatenated reasoning text for assistant turns
|
||||
that round-tripped a thinking-block lane (Anthropic-with-thinking
|
||||
today; OpenAI Responses + Gemini in later phases). Present only
|
||||
when the active model's ``surface_persisted_reasoning`` flag is true and
|
||||
the underlying ``provider_data`` carries reasoning blocks.
|
||||
"""
|
||||
|
||||
type: str = "history"
|
||||
messages: list[dict[str, Any]] = field(default_factory=list)
|
||||
|
||||
@@ -75,6 +95,28 @@ class ContentEvent(ServerEvent):
|
||||
text: str = ""
|
||||
|
||||
|
||||
@dataclass
|
||||
class InProgressSnapshotEvent(ServerEvent):
|
||||
"""One-shot replay of the in-progress turn's content + reasoning.
|
||||
|
||||
Emitted by the per-workstream events SSE handler immediately after
|
||||
the kind-specific replay phase (history / pending / state_change)
|
||||
when a fresh subscriber connects mid-stream. Lets a refreshing
|
||||
browser tab restore the partial assistant message without waiting
|
||||
for the response to complete and a full page reload.
|
||||
"""
|
||||
|
||||
type: str = "in_progress_snapshot"
|
||||
content: str = ""
|
||||
reasoning: str = ""
|
||||
|
||||
|
||||
@dataclass
|
||||
class StateChangeEvent(ServerEvent):
|
||||
type: str = "state_change"
|
||||
state: str = ""
|
||||
|
||||
|
||||
@dataclass
|
||||
class StreamEndEvent(ServerEvent):
|
||||
type: str = "stream_end"
|
||||
@@ -358,6 +400,8 @@ _SERVER_REGISTRY: dict[str, type[ServerEvent]] = {
|
||||
ThinkingStopEvent,
|
||||
ReasoningEvent,
|
||||
ContentEvent,
|
||||
InProgressSnapshotEvent,
|
||||
StateChangeEvent,
|
||||
StreamEndEvent,
|
||||
ToolInfoEvent,
|
||||
ApproveRequestEvent,
|
||||
|
||||
@@ -58,6 +58,9 @@ from turnstone.core.history_decoration import (
|
||||
from turnstone.core.history_decoration import (
|
||||
extract_advisories_from_tool_envelope,
|
||||
)
|
||||
from turnstone.core.history_decoration import (
|
||||
extract_reasoning_text_from_provider_content as _extract_reasoning_text,
|
||||
)
|
||||
from turnstone.core.history_decoration import (
|
||||
load_verdict_indexes as _load_verdict_indexes,
|
||||
)
|
||||
@@ -471,6 +474,26 @@ def _build_history(
|
||||
else:
|
||||
ws_id = getattr(session, "_ws_id", "") or ""
|
||||
verdicts_by_call_id, assessments_by_call_id = _load_verdict_indexes(ws_id)
|
||||
# Active-model ``surface_persisted_reasoning`` flag — single-tier
|
||||
# resolution (live session's registry only). This path always has
|
||||
# a live ``ChatSession`` in hand, so the cold-workstream and
|
||||
# app.state-registry tiers used by ``make_history_handler``
|
||||
# (``session_routes.py:2396-2429``) are unreachable here. Default
|
||||
# True mirrors the migration's server_default and matches the
|
||||
# conservative rehydration default in the Phase 1 spec.
|
||||
surface_persisted_reasoning = True
|
||||
registry = getattr(session, "_registry", None)
|
||||
model_alias = getattr(session, "_model_alias", "") or ""
|
||||
if registry is not None and model_alias:
|
||||
try:
|
||||
surface_persisted_reasoning = bool(
|
||||
registry.get_config(model_alias).surface_persisted_reasoning
|
||||
)
|
||||
except Exception:
|
||||
# Unknown alias / partially-built registry / dataclass drift —
|
||||
# fall back to the conservative default rather than failing
|
||||
# the entire history build.
|
||||
surface_persisted_reasoning = True
|
||||
history = []
|
||||
for msg in session.messages:
|
||||
content = msg.get("content")
|
||||
@@ -556,6 +579,15 @@ def _build_history(
|
||||
clean_reminders.append(clean)
|
||||
if clean_reminders:
|
||||
entry["reminders"] = clean_reminders
|
||||
# Surface stored reasoning text on assistant messages for UI
|
||||
# rehydration (page-refresh path). Sourced from the in-memory
|
||||
# ``_provider_content`` lane on ``session.messages`` (set
|
||||
# post-commit at ``session.py:3768-3771``). The lane itself is
|
||||
# never copied into ``entry`` — the wire payload stays tight.
|
||||
if msg.get("role") == "assistant" and surface_persisted_reasoning:
|
||||
reasoning_text = _extract_reasoning_text(msg.get("_provider_content"))
|
||||
if reasoning_text:
|
||||
entry["reasoning"] = reasoning_text
|
||||
if msg.get("tool_calls"):
|
||||
tc_entries: list[dict[str, Any]] = []
|
||||
for tc in msg["tool_calls"]:
|
||||
|
||||
@@ -690,6 +690,18 @@
|
||||
this._refreshOptionsSummary();
|
||||
};
|
||||
|
||||
// Update just the placeholder (first) option's text without disturbing
|
||||
// the rest of the choice list. Callers that resolve the effective
|
||||
// default asynchronously use this to annotate the empty option with the
|
||||
// concrete alias — e.g. "Default model" → "Default model (gpt-5)".
|
||||
Composer.prototype.setOptionPlaceholder = function (id, text) {
|
||||
var ctrl = this._optionFields && this._optionFields[id];
|
||||
if (!ctrl || ctrl.tagName !== "SELECT") return;
|
||||
if (ctrl.options.length === 0) return;
|
||||
ctrl.options[0].textContent = text == null ? "" : String(text);
|
||||
this._refreshOptionsSummary();
|
||||
};
|
||||
|
||||
Composer.prototype._refreshOptionsSummary = function () {
|
||||
if (!this.optionsSummaryEl) return;
|
||||
var summaryFn = this._opts.options && this._opts.options.summary;
|
||||
|
||||
@@ -480,6 +480,49 @@ Pane.prototype.handleEvent = function (evt) {
|
||||
this.scrollToBottom(true);
|
||||
break;
|
||||
|
||||
case "in_progress_snapshot":
|
||||
// One-shot replay of the in-progress turn's reasoning + content
|
||||
// when this client connects mid-stream (page refresh while the
|
||||
// model is generating). Both fields may be empty; render only
|
||||
// the non-empty halves. Idempotent on EventSource auto-reconnect:
|
||||
// skip overwrite when the current buffer is already at-or-past
|
||||
// the snapshot length, so a stale replay can't reset the live-
|
||||
// streamed view back to a shorter prefix.
|
||||
this.removeThinkingIndicator();
|
||||
if (evt.reasoning) {
|
||||
if (!this.currentReasoningEl) {
|
||||
this.currentReasoningEl = document.createElement("div");
|
||||
this.currentReasoningEl.className = "msg reasoning";
|
||||
this.messagesEl.appendChild(this.currentReasoningEl);
|
||||
}
|
||||
var curReason = this.currentReasoningEl.textContent || "";
|
||||
if (curReason.length < evt.reasoning.length) {
|
||||
this.currentReasoningEl.textContent = evt.reasoning;
|
||||
}
|
||||
}
|
||||
if (evt.content) {
|
||||
// Content snapshot supersedes any reasoning bubble — matches
|
||||
// the "case content" invariant of clearing currentReasoningEl
|
||||
// when content begins.
|
||||
if (this.currentReasoningEl) {
|
||||
this.currentReasoningEl = null;
|
||||
}
|
||||
if (!this.currentAssistantEl) {
|
||||
this.currentAssistantEl = document.createElement("div");
|
||||
this.currentAssistantEl.className = "msg assistant";
|
||||
this.currentAssistantBodyEl = document.createElement("div");
|
||||
this.currentAssistantBodyEl.className = "msg-body";
|
||||
this.currentAssistantEl.appendChild(this.currentAssistantBodyEl);
|
||||
this.messagesEl.appendChild(this.currentAssistantEl);
|
||||
}
|
||||
if (this.contentBuffer.length < evt.content.length) {
|
||||
this.contentBuffer = evt.content;
|
||||
streamingRender(this.currentAssistantBodyEl, this.contentBuffer);
|
||||
}
|
||||
}
|
||||
this.scrollToBottom();
|
||||
break;
|
||||
|
||||
case "state_change":
|
||||
if (evt.state === "idle" || evt.state === "error") {
|
||||
this.setBusy(false);
|
||||
@@ -1151,6 +1194,20 @@ Pane.prototype.replayHistory = function (messages) {
|
||||
}
|
||||
lastToolBlock = null;
|
||||
} else if (msg.role === "assistant") {
|
||||
// Reasoning bubble (Phase 1 reasoning persistence) — render
|
||||
// BEFORE the content bubble so the visual order matches the
|
||||
// live SSE flow (reasoning_delta arrives before content_delta
|
||||
// for thinking-enabled models). Mirrors the live-stream
|
||||
// construction at the "case 'reasoning':" branch above. Only
|
||||
// surfaces when the active model's surface_persisted_reasoning flag is
|
||||
// true and the message round-tripped a thinking lane.
|
||||
if (msg.reasoning && msg.reasoning.length) {
|
||||
var reasonEl = document.createElement("div");
|
||||
reasonEl.className = "msg reasoning";
|
||||
reasonEl.textContent = msg.reasoning;
|
||||
self.messagesEl.appendChild(reasonEl);
|
||||
lastToolBlock = null;
|
||||
}
|
||||
// Render content BEFORE the tool block so the visual order
|
||||
// matches the live SSE flow (stream_text streams content first,
|
||||
// then tool_info / approve_request paints the tool block, then
|
||||
|
||||
Reference in New Issue
Block a user