Every single-shot lane (model_turn: judges, titles, compaction, web-fetch
extraction, perception, eval, optimizer) now samples through the provider's
streaming entry and accumulates via a shared drain_stream(), deleting
create_completion from the Protocol and all three adapters (xai/google
inherit). Request shaping can no longer drift between the two consumption
styles, and callers keep the exact CompletionResult contract.
The drain mirrors the main loop's proven chunk semantics: per-field
max-merge for usage (Anthropic splits prompt/completion across
message_start/message_delta), tool-call assembly by delta index,
provider_blocks from the terminal emission, trailing citation info folded
back into content (byte-matching the old format_citations append),
mid-stream status pings dropped.
Also in this change:
- model_turn grows cancel_ref; both judges wire their run_with_deadline
abandon paths to a new StreamAbortRef (deadline.py) that closes the SDK
stream — a timed-out judge call now aborts its HTTP read instead of
pinning a daemon thread until the next upstream chunk. The append hook
covers the arrival race, mirroring ChatSession._CancelRef.
- Responses streaming gains the response.incomplete terminal handler
(truncated runs were mislabeled finish=stop and lost final usage AND
collected provider_blocks) and a refusal handler ([Refused: …] content,
matching the retired non-streaming rendering). Both also fix the main
chat loop, which shared the gaps.
- supports_streaming capability flag deleted (zero readers) along with
its admin capability tile; o1-era models that reject streaming need a
model alias pointing at a current model (release-noted).
- Helpers that existed only for the deleted transport go with it:
Responses._parse_response, chat/google._extract_tool_calls.
Known behavioral deltas (release-noted): OpenAI-compatible servers that
ignore stream_options.include_usage stop producing usage rows on these
lanes; multiple Anthropic text blocks concatenate without the old "\n"
joint (matching the main loop); model_turn lanes no longer risk client
read-timeouts on long generations — the reason the Anthropic adapter
already drained a stream internally.
Tests: new test_drain_stream.py pins the accumulator rules; shared fakes
(as_stream, fake_chat_stream, fake_anthropic_stream) migrate 11 suites to
the streaming transport, with the task-agent and adapter suites now
exercising the real _iter_stream + drain path end to end.
Round-2 review fixes. The round-1 de-pinning collided with
ConfigStore.get's default-on-miss semantics: the registry defaults
(temperature 1.0, effort "medium") were manufactured onto every
store-backed lane's wire, making the documented "unset -> omit"
terminal unreachable. Unset is now representable end to end, and one
scheme governs every lane: per-model alias value > operator-stored
global setting > in-code model definition (effort only: caps
declaration) > field omitted, inference engine's default rules.
- settings_registry: model.temperature default None, model.reasoning_effort
default "" — the registered defaults ARE the unset sentinels, so the
admin UI and the wire agree. Admin webux renders nullable floats blank
("(inherit model default)") and maps blank-save to reset; the "" effort
choice reads "(inherit)".
- model_turn: resolve_temperature_setting/resolve_effort_setting are the
ONE pair of operator-rung resolvers, shared by resolve_lane, both
session factories, and the /model switch (the 4th-copy mirror is gone;
the switch no longer leaks the previous model's override on store-less
sessions). The caps rung moved out of the lane into model_turn's
effective computation, below a new request-shaped default_reasoning_effort
parameter (utility + output guard pass "low": budget coherence with
their small token caps, not sampling policy — any operator or
model-definition value beats it). The hidden "medium" terminal is gone.
- providers: Protocol + all adapters take reasoning_effort: str | None =
None (the Protocol-signature "medium" was the same manufactured pin one
layer down); ModelCapabilities.default_reasoning_effort defaults "" —
commercial rows all declare theirs explicitly, so only local lanes and
Anthropic change, both to match their real serving defaults (Anthropic
manual-thinking models no longer get implicit thinking-on-medium).
reasoning_template_kwargs distinguishes unset (inject nothing; template
default rules) from the explicit "none" off-switch. apply_temperature
skips temperature unless reasoning is EXPLICITLY off on none-declaring
models (unset leaves the server default in charge, possibly reasoning-on).
- session: ctor takes temperature: float | None / reasoning_effort:
str | None = None; _save_config/resume round-trip unset as "" (the
str(None) era guarded); _run_agent relays session temperature AND
effort on the same-alias fall-through only (a task alias's configured
knobs stay reachable in both directions).
- optimizer: the five meta lanes are decoupled from --temperature/
--reasoning-effort (test-model knobs, per their documented meaning);
registry-less meta lanes omit both fields.
- cli: --temperature/--reasoning-effort default unset and fall through
the model config instead of pinning 0.5/"medium" for every CLI session.
- cleanup from the review's below-cap findings: dead resolve_server_type
deleted (tests re-pointed at _server_type_of), stale ChatSession
comments in _openai_responses fixed, _store_get_or_none extracted,
eval system-turn conversion hoisted out of the per-turn loop, dead
_provider_extra_params patch removed, test_perception uses the shared
mock_completion_result, effort_ladder uses apply_capability_overrides
instead of a SimpleNamespace fake config.
Wire goldens regenerated: the only drift is the manufactured "medium"
effort vanishing from unset-effort requests (Responses reasoning.effort,
Chat/Google reasoning_effort, Anthropic output_config.effort) — pure
removals, no additions. Ladder tests now fake ConfigStore with the REAL
get() semantics (registry default on miss) so a forgiving fake can't
mask this class of bug again.
create_completion now has exactly one caller: model_turn. The π-side
lanes migrate off hand-built OpenAI dicts:
- _utility_completion (title gen, compaction, web-fetch extraction)
takes list[Turn] and runs the session's primary lane through
model_turn; its three call sites build Turn.system/Turn.user.
- perception.describe builds a by-reference trajectory (AttachmentRef +
the prebuilt parts via resolve_attachments, reintroduced on
model_turn with its first caller and pinned by tests) — Turn IR never
carries inline media bytes, matching the main loop's wire path. Its
temperature=0.2 pin is gone (house rule).
- eval HeadlessSession's loop lowers system prompts through the
turns_from_dicts bridge and appends result.turn; the parallel-call
cap now also drops the native lane on a capped turn (a capped mirror
with a full native lane would replay orphan tool blocks).
- optimizer: all five sites (diversifier, observer, analyst loop,
tool optimizer, prompt optimizer) build Turn IR through per-function
lanes; every temperature pin (0.8/0.3/0.3/0.3/0.6) removed per house
rule — sampling behavior belongs in the model's configuration.
Test mocks move to the shared full-shape helper where the model_turn
re-ingest now runs; perception/attachment tests assert the
by-reference placeholder + resolver contract instead of inline parts.
_resolve_attachments re-runs on every agentic round-trip (and per fallback model), each time re-fetching every attachment across the full history and re-rasterizing / re-base64'ing it. A 10-page PDF in a 10-cycle tool turn was rendered dozens of times.
Add a per-send memo (self._wire_part_cache) keyed by (attachment_id, caps-signature): the materialized wire part is computed at most once per send. The cache is None outside a send (display/export paths unaffected) and reset per send to bound the heavy rasterized-page parts and pick up any mid-session capability change. Skip the DB fetch entirely when every id is already cached.
Also peek the perception (alias, content_hash) memo before building parts in _perception_fallback_part, so a cross-send describe hit no longer wastes a PDF rasterize. Leaves pdf.py's deliberate no-module-cache stance intact — the per-send scope addresses the round-trip amplification without the durable store it defers.
Adds describe_peek() + per-send-cache and peek tests.
Add a `perception.model_alias` model role: when the primary model can't ingest
an attachment natively and can't be shown a degraded-but-native form, a
configured perception model perceives it and its output is carried as text.
Mirrors the STT role — a role alias plus a module-level memo so the extra LLM
round-trip runs once per attachment, not once per conversation turn. The call
goes through the provider abstraction's create_completion (the path the intent
judge uses), so any vision/omni provider works.
Bottom-tier, universal ladder — perception only fills the remaining gap:
- pdf : native supports_pdf -> rasterize-to-vision-primary -> perception
-> extracted text -> placeholder
- image: native vision -> perception (non-vision primary) -> native image_url
- audio: native supports_audio_input -> STT -> perception (omni) -> placeholder
Folds in two review findings the role subsumes:
- bug-1: thread the active attempt's capabilities into _resolve_attachments
(bound in _try_stream) so a model fallback materializes attachments against
the fallback model's caps, not the primary's.
- bug-2: charge a by-reference pdf/audio a bounded budget min(size_bytes, 16K)
instead of zero, so a large-attachment turn isn't budgeted as ~empty (the
exact materialized size isn't known until wire build).
Patrick Buckley