mirror of
https://github.com/turnstonelabs/turnstone.git
synced 2026-08-12 23:12:23 -06:00
fix(providers): segregate inline reasoning at the drain seam
Passthrough servers (parserless vLLM/llama.cpp, LM Studio, bare gateways) emit reasoning as literal <think>/<reasoning> blocks inside content, and only three of nine drained lanes stripped them: web_fetch tool results persisted raw think blocks into every following turn (#940), judge verdicts parsed through tag noise, and a draft verdict inside a think block could shadow the real one at the output guard. One rule at the seam now. drain_stream accumulates content in RUNS bounded by interleaving signals (provider-parsed reasoning deltas, tool-call deltas) with the interactive consumer's within-chunk ordering — reasoning, then content, then the tool-call close — and splits each run through split_inline_reasoning, the one-shot form of the interactive lane's ThinkTagSplitter: a pure raw split, exactly equivalent to the streaming form on every catalog case. One trim policy exists and the drain owns it: blank edge lines are trimmed once over the joined runs when a tag was consumed, so tag residue dies at the edges while genuine inter-run paragraph separators survive. Extracted text is appended to result.reasoning after any server-parsed reasoning with a blank-line boundary and rides the native lane as the reasoning_text synth block. Orphan CLOSE tags deliberately pass through byte-identical: a close whose open never arrived is indistinguishable from prose QUOTING the tag, and drained lanes routinely quote third-party text — reclassifying would let a malicious page containing the literal tag destroy the extraction that cites it. The title lane keeps a local rfind peel as display-string formatting. The citations footer folds only onto non-blank content — sourcing for an answer that does not exist is dropped rather than handed to emptiness checks as a footer-only "answer". Every private strip is deleted: the title lane's strip, the summarizer strip, _strip_reasoning itself, and the optimizer's five regexes (_strip_markdown_fence is now the one fence rule, applied to normalized model output only, never to or-fallback values). Think-only and whitespace-only responses drain to blank content, and every lane's no-answer fallback gates on blankness: web_fetch returns an honest extraction-error card, the intent judge takes the empty-retry ladder, the task-agent synthesis reports "(no output)", and the optimizer keeps the current observer system and prompt verbatim on no-answer passes. Final-say reads (optimizer analyst, eval final_content, the notify hook) use trajectory.final_assistant_text — the last assistant turn only, never an earlier narration presented as the conclusion — while last_assistant_text is the salvage walk (task_agent partial-work recovery), skipping tool-call-only, all-reasoning, and whitespace-only turns. Perception memoizes every completed description immediately, including an empty one — one perceive per key, ever — under a commit-lock guard so an empty result never overwrites a concurrently memoized real description; an all-reasoning perception model pins the placeholder until restart, and the remediation is server-side (a reasoning parser or the template thinking toggle on the perception alias). A true double-reasoning shape (inline-extracted text alongside a native reasoning block) logs chars-only at the drain, where it is distinguishable from the routine reasoning_delta mirror. The dialect's semantics are pinned as one table (tests/_reasoning_dialect.py) driven through shared fixtures (think_tag_stream, seam_provider): one-shot conformance, the exact one-shot/streaming equivalence property, the drain seam rules including quoted-tag safety, run-boundary and separator-preservation pins, per-lane pins for all nine lanes, and the empty-content assistant wire shape. Closes #965. Closes #940.
This commit is contained in:
@@ -19,7 +19,7 @@ from types import SimpleNamespace
|
||||
from typing import Any
|
||||
from unittest.mock import MagicMock
|
||||
|
||||
from turnstone.core.providers import StreamChunk, ToolCallDelta
|
||||
from turnstone.core.providers import ModelCapabilities, StreamChunk, ToolCallDelta
|
||||
from turnstone.core.session import ChatSession
|
||||
from turnstone.core.session_ui_base import SessionUIBase
|
||||
|
||||
@@ -349,6 +349,43 @@ def as_stream(result: Any) -> list[StreamChunk]:
|
||||
]
|
||||
|
||||
|
||||
def think_tag_stream(utterance: str) -> list[StreamChunk]:
|
||||
"""``create_streaming`` return value simulating a passthrough server
|
||||
that emits *utterance* — typically think-tag-bearing — as plain
|
||||
streamed content.
|
||||
|
||||
The per-lane fixture for inline-reasoning dialect pins: lane tests
|
||||
supply their own utterances (the dialect's SEMANTICS are specified
|
||||
once, in ``tests._reasoning_dialect.CASES``, and pinned by the
|
||||
one-shot suites — lane pins assert lane behavior, not tag grammar).
|
||||
Routes through the real ``drain_stream`` seam exactly like
|
||||
``as_stream``.
|
||||
"""
|
||||
return as_stream(mock_completion_result(content=utterance))
|
||||
|
||||
|
||||
def seam_provider(utterance: str, *, provider_name: str = "openai-compatible") -> MagicMock:
|
||||
"""Provider fake whose ``create_streaming`` replays *utterance* through
|
||||
the REAL drain seam (``think_tag_stream``) — THE lane-suite seam fake.
|
||||
|
||||
One definition so the lane suites cannot drift when the provider
|
||||
surface ``model_turn`` probes grows: real ``ModelCapabilities`` for
|
||||
the clamp math, ``provider_name`` overridable per suite.
|
||||
|
||||
Assign the RETURNED fake to ``session._provider`` — never mutate the
|
||||
provider a session resolved on its own: with a MagicMock client the
|
||||
session resolves the process-wide ``create_provider(...)`` singleton,
|
||||
and writing that shared instance's ``create_streaming`` poisons every
|
||||
later session in the test run (the SSE-recovery e2e servers resolve
|
||||
the same instance).
|
||||
"""
|
||||
provider = MagicMock()
|
||||
provider.provider_name = provider_name
|
||||
provider.get_capabilities.return_value = ModelCapabilities()
|
||||
provider.create_streaming = MagicMock(return_value=think_tag_stream(utterance))
|
||||
return provider
|
||||
|
||||
|
||||
class RecordingUI:
|
||||
"""UI adapter recording the ordered event stream ``send()`` emits."""
|
||||
|
||||
|
||||
Reference in New Issue
Block a user