mirror of
https://github.com/turnstonelabs/turnstone.git
synced 2026-08-12 23:12:23 -06:00
c906776efd253c3aa1326d7dd4458a5c007cbbab
9 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Patrick Buckley
|
c906776efd |
fix(832): the serving lane's capabilities reach the wire fold
The per-attempt prepare_wire closure folded mid-conversation system turns with the PRIMARY binding's capabilities on every lane, so a fallback whose chat template rejects non-leading system roles failed on the self-inflicted wire shape and burned its own health record — the wrong-dialect class the walk's binding snapshot guards against elsewhere. model_turn now passes the serving lane to prepare_wire, and the session's closure folds with that lane's capabilities; callers without a lane in hand (the token-table re-fold) keep the primary default. Pre-fold prepared once with primary caps for every lane, so this is a named improvement, not a parity break. The arm-duties hook rode the same unguarded two-statement supersession window the _CancelRef docstring accepts only for the stream register: a force-cancel claiming a new generation between the superseded read and the hook let an orphan's late registration null the successor's usage slots and record spurious creation health. on_stream_armed now generation-gates itself, shrinking the accepted window's harm back to the register-only class. Test hygiene: the two overflow-compact tests are one parametrized body; arm_session mints a fresh ArmedHandle per create (provider.handles, _armed_handle = latest) matching the one-handle-per-create rule of real adapters. The duplicate sanitize pass stands as designed (accepted for wire parity); its perf note rides #979. All three product fixes are mutation-probed. |
||
|
Patrick Buckley
|
aa4371ea99 |
fix(832): retire the dead attempt's armed state in the re-create window
Between a mid-stream death and the next begin_attempt there is no live attempt, but the consumer kept the dead attempt's armed _CancelRef: a Stop in that window re-emitted the discarded splitter carry as fresh content behind a duplicate stream_end, and a walk-preamble failure was classified as another armed death, replacing the operator-actionable stream-death error. end_attempt() now pronounces the attempt dead at partial-capture; the consumer gains a single per-attempt initializer (_reset_attempt), a lane-free constructor (one resolve_lane walk per turn), and a saw-chunk classifier fallback so a never-arming adapter's mid-stream death still classifies mid-stream instead of silently double-rendering the same lane. Wire-preparation failures are typed at the seam: model_turn wraps prepare_wire raises in WirePreparationError, both walk arms forward it verbatim (no health record, no fallback walk — a session-data fault would otherwise paint every backend degraded), the fatal formatter gets a dedicated branch, and the re-issue ladder's last-death mask exempts it alongside BackendAuthUnavailableError so an auth outage mid-turn is not misdiagnosed as a network flap. Riding fixes: the tag-scan gate gets its single spelling (lane_scans_inline_reasoning) shared by drain and display; the citations fold's separator+gate become a shared pair in _protocol; _build_main_lane stops passing config_store (dead derivation — the session's own knobs replace both values it feeds); the debug wire dump is ruled per-invocation (the overflow-recovery re-print is the dump that diagnoses the recovery) and pinned; dead delegates _ensure_tool_call_ids and _finalize_provider_blocks deleted; the parity runner adapts to the pre-fold seam signature by inspection and refuses to record a harness-shape TypeError as a baseline; the streaming provider fakes move to tests/_session_helpers (their tree-wide home) and test_cancel's duplicate helper is deleted; committed parity pins restate their rulings in full; architecture.md's circuit-breaker section is replaced by the real passive health-tracker story and the send-flow diagram stops attributing tool-call assembly to the display consumer; stale pre-fold names and ragged comment paragraphs cleaned. New pins are mutation-probed: disabling end_attempt, the saw-chunk fallback, the auth exemption, or the WirePreparationError arm each fails its pin. |
||
|
Patrick Buckley
|
e103af94e7 |
test(832): port the seam-coupled suites to the folded architecture
Seventeen files, ~1,300 tests, re-pointed or redesigned per the triage ledger's recipes: wholesale turn-scripting moves to ModelTurnResult fakes; streaming-behavior suites drive the REAL wrapper+consumer+drain path through armed provider fakes (tests/_parity_832.arm_session — the eager cancel_ref append every real adapter performs, exception elements for creation-phase failures, sequential per-turn scripts, and the title lane quieted: a provider-level fake otherwise loses its one-shot script to best-effort title generation, which is why the old tests patched at the session level); kwarg-capture suites assert through model_turn's create_streaming call with system-prepend-aware index math; delegate wrappers retired by the fold re-aim at their model_turn module twins. Old-architecture pins are replaced by their new-world equivalents rather than deleted: no shared cancel ref exists (pinned), the handle slot and per-attempt refs carry the cancel surface, the retry gate reads the serving lane's provider, and a superseded generation's death exits send silently as cancelled — a named delta: no arbitrary exception class escapes an orphaned thread anymore. Full suite: 10651 passed, 10 skipped. The wire-payload goldens pass untouched — the fold's lowering composition is byte-equivalent on every provider's request path, as designed. |
||
|
Patrick Buckley
|
5fb27e8f81 |
fix(session): survive mid-stream transport deaths in interactive turns (#937)
A wire death during body streaming (ReadError on a TLS record failure, peer resets) surfaces after the request has already returned its stream handle, so neither the SDK's request retries nor the creation-time retry ladder ever saw it: the interactive turn died with a bare exception string, the partial output was discarded, and no log trace was left. Utility lanes already survived this through drain_stream's normalization; the interactive loop now gets the same treatment. - transport_guarded() in providers/_protocol.py: drain_stream's transport-death conversion made reusable for consumers that keep streaming semantics. Pre-finish deaths raise the retryable IncompleteStreamError (drain's exact message shape); post-finish blips end the stream cleanly, forfeiting only trailing metadata. - The single-pass chunk consumer renames to _stream_attempt; _stream_response is now the resilient wrapper owning ALL stream acquisition plus a bounded mid-stream re-issue ladder (_MID_STREAM_RETRIES, the shared _stop_retrying predicate with a per-loop cap, cancel-aware exponential backoff). Send()'s overflow compact-and-retry arm now wraps the whole turn and passes re-prepared msgs explicitly. - A dead attempt is finalized across every UI consumer before the retry (stream_end then turn_committed then notice then spinner), so retried text never appends onto the dead attempt's in any surface (browser transcript, CLI markdown fences, Slack/Discord streamed messages, SSE replay ring). - Before re-creating, the session re-resolves its registry binding: a concurrent ModelRegistry.reload() closes cached clients, and the retry must not stream into the closed one. A failing re-create logs stream.retry.recreate_failed and re-raises the ORIGINAL stream-death error rather than masking it. - _format_backend_error gains a stream-death branch naming the provider, endpoint, and model, with a short identity-bearing first sentence. _BACKEND_STREAM_EXC_NAMES joins _BACKEND_KNOWN_EXC_NAMES, which also removes those names from _is_ctx_overflow's text-detection eligibility (deliberate: their texts are fixed transport strings that never carry overflow phrases). - _record_fatal_error now logs session.fatal.recorded (INFO for KeyboardInterrupt, ERROR otherwise) so fatal turns leave a journal trace. - _assistant_pending_tokens resets at stream entry so a post-finish blip that loses the trailing usage chunk cannot append the previous turn's completion count as this turn's estimate. Offline SDK boundary pins (openai/anthropic mid-body death identity and no re-request, cross-thread client close surfacing httpx.ReadError) guard the assumptions the retry gate rests on. |
||
|
Patrick Buckley
|
6c4c848a08 |
fix(session): survive tool-result truncation at zero context budget (#883)
At an exhausted context budget the drain loop replaced every tool result with a placeholder that read as a successful-but-trimmed call. For structural results — spawn_workstream's ws_id, the tasks scratchpad — the model lost the handle orchestration depends on and silently stalled, while the UI (told the real summary before the drain) kept showing success. Worse, the budget zeroes near 70% fullness when max_tokens ≥ context_window/4, well below the 80% auto-compact threshold, so a stalled coordinator could sit in that band indefinitely with no compaction ever firing. Three guarantees at the truncation seam, one renewal trigger at the drain: - structural-tool and error results get a guaranteed 2048-char admission floor (head+tail beyond it) — never the zero-budget drop - any result at or under the floor passes verbatim (denial notices, spawn acks: never destroy what is smaller than the guarantee) - bulky non-structural results get an explicit drop notice stating the call RAN but its output could not be admitted — never a trim impersonation the model cannot distinguish from success - a zero truncation budget triggers one mid-turn compaction (no threshold_pct — none was evaluated, same rule as the ctx-overflow retry), closing the 70-80% band where the budget zeroed but compaction was never owed Background-bash spawn acks ride the small-result pass; a name-keyed floor cannot distinguish them from foreground bash — see #891. |
||
|
Patrick Buckley
|
2dd0688d45 |
fix(compaction): carry the plan and the ask across compaction verbatim
The definition review found the two control-relevant crossings paraphrased:
the model's wind-down spill (recorded on the cooperative advisory, then
handed to the summarizer with everything else) and the user's last message
(clipped to 400 chars in the continuation hint). Both now cross copied.
- carry_spill: when the model stopped because it was advised to wrap up,
its final turn's text is shell-concatenated onto the summary under
'## Wind-down (verbatim)', ahead of '## Continue'. The summarizer still
reads the spill; its paraphrase is no longer the only survivor.
- _carry_budget_chars(carries): ~25% of the window per carry, sized so ALL
concurrent carries fit the spare after the summary output reserve —
spill + hint fire together at the end-of-turn site, and independent
sizing stacked reserve + 2*(cw/4) + margin past the window at default
config. Floored at 2000 chars; oversize content keeps head + tail.
- _truncate_block's marker reports the original size ('truncated — N chars
total'), and a truncated carry adds one line telling the model the full
text remains in history and recall can retrieve it.
- Summary turns carry source="compaction" (in-memory swap and checkpoint
reconstruction); _find_turn_boundaries and _generate_title test the tag
instead of the label string, so a user who literally types
'[Conversation summary]' stays a real turn.
- The send-loop overflow backstop now passes my_generation, closing the
compact-and-swap race every other compaction site already guards.
Tests: tests/test_compaction_crossing.py (tags on both paths, literal-label
boundary, budget arithmetic incl. the double-carry invariant at shipped
defaults, verbatim/truncated carries, spill semantics, forwarding); existing
suites updated for the tagged label turns and the new kwargs.
|
||
|
Patrick Buckley
|
6b6c220986 |
fix(compaction): chunk the summary call so it can't overflow
The compaction summary ran as a single model call sized by the per-message token estimate, which disagreed with the head+tail-capped formatted text, so a long history could overflow the summary call itself; the old prefix-fit also silently dropped the most-recent messages. Summarize the whole selection via _summarize_blocks: greedily pack the formatted blocks into batches that each fit the summary call's own input budget (_summary_input_budget_chars), summarize each, and recursively merge the partials until they collapse to one. The common case (it all fits) stays a single call. Bail to the existing False path when the input is irreducible rather than fabricate a summary; a mid-chunk failure leaves messages untouched (atomic swap only on full success). Bound the summary output reserve to half the context window (_summary_output_tokens), used by BOTH the input-budget sizing and the actual call. compact_max_tokens defaults to the full window (32768); clamped only by max_output_tokens it reserved the entire context for output, flooring the input budget so compaction overflowed (or bailed as irreducible) at the default/small-window config that needs it most. Large windows are unaffected (compact_max_tokens stays binding). Guard an empty summary (keep history instead of swapping in nothing and reporting success). Fold tool-def tokens into the _last_usage-less estimate AND the post-compaction usage anchor, so the compact-before-truncate budget doesn't over-state free space by the tool-def count. Single-source the shared compactor/merge prompt section (_COMPACT_OUTPUT_FORMAT) and the tool-def sizing (_tool_def_chars/_tool_def_tokens). A just-resumed session (no _last_usage) now counts tool-def tokens so it doesn't undercount and skip proactive compaction until its first reply re-anchors the estimate. Prepush review follow-ups: generation-guard the end-of-turn auto-compaction and its resume turn so a force-cancel during the slow summary call can't compact or persist under a new generation (matching the mid-turn and end-of-loop guards); single-source the soft-threshold predicate (_over_soft) shared by the mid-turn policy, _compaction_owed, and the end-of-turn check; add tests for the pre-attempted-compaction guard and the recursion depth ceiling. |
||
|
Patrick Buckley
|
dc88060b79 |
refactor(core): session.messages is the canonical Turn trajectory
ChatSession.messages flips from list[dict] to list[Turn] — the in-memory canonical trajectory. Reads migrate to typed fields (turn.role, turn.text, turn.tool_calls); appends and assignments go through turn_from_dict / turns_from_dicts; the fork bulk-save and retry's multipart check read via turn_to_dict. _full_messages lowers Turns→dicts at the wire boundary — the fold/repair and provider translators still consume dicts until the next slice. The token-accounting helpers accept a dict or a Turn. Non-session consumers migrate too: coordinator_idle_observer and eval to typed fields (mypy-enumerated), and server's last-assistant extractor via turn_to_dict (an Any-typed call site mypy could not flag). An all-text multipart content list (the unreadable-attachment placeholder path) now round-trips faithfully through the adapter (single text block → str, multiple → list). Tests that inspected session.messages as dicts read it through the dicts_from_turns / turn_to_dict bridge; those that built it pass dicts through turns_from_dicts / turn_from_dict. Byte-identical wire harness; full non-live suite green (7130). |
||
|
Patrick Buckley
|
d100ac92d9 |
fix: capacity-aware tool output truncation and context overflow recovery (#301)
* fix: capacity-aware tool output truncation and context overflow recovery Large tool results (e.g. 593K-char search output) could overflow the context window in a single turn when the conversation was already partially full. The fixed 50%-of-context truncation limit didn't account for current usage. Changes: - _truncate_output() now accepts remaining token budget and uses min(tool_truncation, remaining_budget_chars) as the effective limit - _remaining_token_budget() helper calculates available capacity with reserves for max_tokens response and 5% safety margin - Safety truncation at tool-result append: every string tool result is clamped to remaining budget before entering the message array - _exec_web_search() now calls _truncate_output() (was missing) - Context overflow recovery: catches provider errors indicating context length exceeded (OpenAI + Anthropic patterns), auto-compacts, retries once. Falls back to original error if compact-and-retry fails. * fix: address review — zero-budget floor, nested spinner, Anthropic patterns, tests - Remove 256-char floor from budget truncation — zero budget now returns a placeholder instead of allowing 256 chars through - Stop thinking spinner before compact to avoid nested start/stop - Add Anthropic error patterns (prompt is too long, input tokens) - Wrap compact-and-retry so failures re-raise the original error - Add 15 tests covering budget calculation, capacity-aware truncation, and overflow recovery for both providers * fix: cap response reservation at 25% of context window Reserving the full max_tokens in _remaining_token_budget() zeroed the budget for common configs like max_tokens=32768 on a 32K context, collapsing all tool output to a placeholder. max_tokens is a ceiling, not guaranteed consumption — cap the reserve at context_window // 4. Adds regression test for max_tokens >= context_window. |