mirror of
https://github.com/turnstonelabs/turnstone.git
synced 2026-08-12 23:12:23 -06:00
main
9 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Patrick Buckley
|
98e96ab5f3 |
Add per-alias model concurrency admission (#990)
* feat(models): add per-alias concurrency admission Add registry-backed FIFO admission limits with queue-aware deadlines and full-stream leases. Expose max_concurrency through storage, admin configuration, OpenAPI, documentation, and diagrams, with role and live backend count coverage. * fix(api): omit null concurrency schema default Keep max_concurrency optional for presence-keyed updates without advertising a null default for its non-null integer OpenAPI shape. |
||
|
Patrick Buckley
|
7a06f5e8bc |
refactor(session): make ModelLane the provider boundary (#979) (#989)
* refactor(session): make ModelLane the provider boundary (#979) ## Summary This closes the model-lane ownership gap left by #832: `ChatSession` no longer stores raw provider/client handles. `ResolvedModelBinding` now carries the provider, client, model, capabilities, registry generation, and backend-auth configuration as one coherent snapshot. - Atomically rebind existing sessions after model-registry changes while pinning each in-flight send, fallback, judge, output guard, task agent, title, compaction, perception, and voice operation to its initiating principal and binding. - Fence UI publication, canonical trajectory folds, durable writes, streams, retries, child scopes, and judge work by generation. Stop can hand off to a successor without accepting late state; cancelled tools retain typed effect receipts, and concurrent approval batches resolve by exact cycle or call. - Make create, fork, open, close, and delete race-safe with hidden `creating` reservations, incarnation-aware state tails, and an ACL-rechecked transaction that clones checkpoint-bounded history, configuration, project/persona state, and attachment references. - Extend REST/OpenAPI and Python/TypeScript SDK contracts for create/fork inputs, routed-create metadata, live-workstream probes, targeted approvals, and structured cancellation results. - Update architecture, storage, authentication, judge, channel, console, API, and SDK documentation, including regenerated architecture diagrams and OpenAPI artifacts. ## Validation - SQLite suite: 11,188 passed, 9 skipped, 10 deselected - PostgreSQL suite: 11,195 passed, 2 skipped, 10 deselected - Live backend: 3 passed - SSE recovery: 6 passed; browser recovery harness passed all scenarios - Ruff: clean; 595 files correctly formatted - mypy: 243 source files clean - TypeScript: typecheck/build and 35 tests passed - OpenAPI artifacts fresh; all 14 changed diagrams reproduce byte-for-byte - `git diff --check` and Git LFS integrity clean Closes #979. * fix(deps): update nanoid for GHSA-2v37-7h3g-55p8 Refresh the transitive lock entry admitted by PostCSS so the TypeScript security gate no longer resolves the vulnerable custom-generator implementation. Validation: - npm ci - npm audit --audit-level=moderate: 0 vulnerabilities - TypeScript typecheck and build - TypeScript tests: 35 passed * fix(test): assert canonical model registry URLs Replace prefix checks with exact canonical base URL assertions so the tests do not model incomplete URL validation. Validation: tests/test_model_registry.py (185 passed); Ruff check/format; mypy. |
||
|
Patrick Buckley
|
bc3fa60011 |
fix(providers): segregate inline reasoning at the drain seam
Passthrough servers (parserless vLLM/llama.cpp, LM Studio, bare gateways) emit reasoning as literal <think>/<reasoning> blocks inside content, and only three of nine drained lanes stripped them: web_fetch tool results persisted raw think blocks into every following turn (#940), judge verdicts parsed through tag noise, and a draft verdict inside a think block could shadow the real one at the output guard. One rule at the seam now. drain_stream accumulates content in RUNS bounded by interleaving signals (provider-parsed reasoning deltas, tool-call deltas) with the interactive consumer's within-chunk ordering — reasoning, then content, then the tool-call close — and splits each run through split_inline_reasoning, the one-shot form of the interactive lane's ThinkTagSplitter: a pure raw split, exactly equivalent to the streaming form on every catalog case. One trim policy exists and the drain owns it: blank edge lines are trimmed once over the joined runs when a tag was consumed, so tag residue dies at the edges while genuine inter-run paragraph separators survive. Extracted text is appended to result.reasoning after any server-parsed reasoning with a blank-line boundary and rides the native lane as the reasoning_text synth block. Orphan CLOSE tags deliberately pass through byte-identical: a close whose open never arrived is indistinguishable from prose QUOTING the tag, and drained lanes routinely quote third-party text — reclassifying would let a malicious page containing the literal tag destroy the extraction that cites it. The title lane keeps a local rfind peel as display-string formatting. The citations footer folds only onto non-blank content — sourcing for an answer that does not exist is dropped rather than handed to emptiness checks as a footer-only "answer". Every private strip is deleted: the title lane's strip, the summarizer strip, _strip_reasoning itself, and the optimizer's five regexes (_strip_markdown_fence is now the one fence rule, applied to normalized model output only, never to or-fallback values). Think-only and whitespace-only responses drain to blank content, and every lane's no-answer fallback gates on blankness: web_fetch returns an honest extraction-error card, the intent judge takes the empty-retry ladder, the task-agent synthesis reports "(no output)", and the optimizer keeps the current observer system and prompt verbatim on no-answer passes. Final-say reads (optimizer analyst, eval final_content, the notify hook) use trajectory.final_assistant_text — the last assistant turn only, never an earlier narration presented as the conclusion — while last_assistant_text is the salvage walk (task_agent partial-work recovery), skipping tool-call-only, all-reasoning, and whitespace-only turns. Perception memoizes every completed description immediately, including an empty one — one perceive per key, ever — under a commit-lock guard so an empty result never overwrites a concurrently memoized real description; an all-reasoning perception model pins the placeholder until restart, and the remediation is server-side (a reasoning parser or the template thinking toggle on the perception alias). A true double-reasoning shape (inline-extracted text alongside a native reasoning block) logs chars-only at the drain, where it is distinguishable from the routine reasoning_delta mirror. The dialect's semantics are pinned as one table (tests/_reasoning_dialect.py) driven through shared fixtures (think_tag_stream, seam_provider): one-shot conformance, the exact one-shot/streaming equivalence property, the drain seam rules including quoted-tag safety, run-boundary and separator-preservation pins, per-lane pins for all nine lanes, and the empty-content assistant wire shape. Closes #965. Closes #940. |
||
|
metaclassing
|
9adde920d4 |
feat(models): per-alias backend auth via Entra OBO and app identity (#898)
Adds a per-alias `auth_mode` on model definitions so a model backend can authenticate to an Entra-fronted gateway with a per-request minted token instead of one shared static API key, letting the gateway attribute calls to the actual user or to the app as a machine identity. - `static` (default, unchanged) sends the stored `api_key`. - `entra_obo` mints a per-user On-Behalf-Of token for `obo_audience` from the caller's captured refresh credential. - `entra_app` mints an app-identity token via the client-credentials grant, and covers userless turns that OBO cannot. Reuses the existing OBO grant legs, refresh-token rotation CAS, cluster advisory lock and the `mcp_user_tokens` mint-cache, keyed under synthetic `__model_obo__:<audience>` / `__model_app__:<audience>` rows. The token binds at the call site through `client.with_options(api_key=...)` so each SDK emits it on its own auth path rather than through header injection. Migration 068 adds `auth_mode` and `obo_audience`. Both are additive and existing rows default to `static`, so behaviour is unchanged unless an alias opts in. Operator controls: `model.auth_audience_allowlist` is an exact-match allow-list that gates which audiences may be configured and denies all by default, and changing a mode or audience requires `admin.mcp`. `model.auth_fail_closed` decides whether a failed mint may fall back to an explicitly configured static key. A delegated call with no user, or a dynamic alias with no real static key, always refuses. Two changes here apply regardless of whether any alias opts in: - Storage and app state are now wired into the console MCP client manager. This fixes per-user `oauth_user` / `oauth_obo` dispatch for coordinator-hosted sessions, which previously raised `RuntimeError` on first call because `set_app_state` was only ever called on the node. - Unattended watch restores and `--resume` resolve the persisted workstream owner instead of constructing the session under an empty principal. A workstream with no owner is now a permanent refusal rather than an anonymous, auto-approved run. |
||
|
Patrick Buckley
|
1e7ad7bcb6 |
feat(providers): one transport — drain create_streaming, retire create_completion (#831)
Every single-shot lane (model_turn: judges, titles, compaction, web-fetch extraction, perception, eval, optimizer) now samples through the provider's streaming entry and accumulates via a shared drain_stream(), deleting create_completion from the Protocol and all three adapters (xai/google inherit). Request shaping can no longer drift between the two consumption styles, and callers keep the exact CompletionResult contract. The drain mirrors the main loop's proven chunk semantics: per-field max-merge for usage (Anthropic splits prompt/completion across message_start/message_delta), tool-call assembly by delta index, provider_blocks from the terminal emission, trailing citation info folded back into content (byte-matching the old format_citations append), mid-stream status pings dropped. Also in this change: - model_turn grows cancel_ref; both judges wire their run_with_deadline abandon paths to a new StreamAbortRef (deadline.py) that closes the SDK stream — a timed-out judge call now aborts its HTTP read instead of pinning a daemon thread until the next upstream chunk. The append hook covers the arrival race, mirroring ChatSession._CancelRef. - Responses streaming gains the response.incomplete terminal handler (truncated runs were mislabeled finish=stop and lost final usage AND collected provider_blocks) and a refusal handler ([Refused: …] content, matching the retired non-streaming rendering). Both also fix the main chat loop, which shared the gaps. - supports_streaming capability flag deleted (zero readers) along with its admin capability tile; o1-era models that reject streaming need a model alias pointing at a current model (release-noted). - Helpers that existed only for the deleted transport go with it: Responses._parse_response, chat/google._extract_tool_calls. Known behavioral deltas (release-noted): OpenAI-compatible servers that ignore stream_options.include_usage stop producing usage rows on these lanes; multiple Anthropic text blocks concatenate without the old "\n" joint (matching the main loop); model_turn lanes no longer risk client read-timeouts on long generations — the reason the Anthropic adapter already drained a stream internally. Tests: new test_drain_stream.py pins the accumulator rules; shared fakes (as_stream, fake_chat_stream, fake_anthropic_stream) migrate 11 suites to the streaming transport, with the task-agent and adapter suites now exercising the real _iter_stream + drain path end to end. |
||
|
Patrick Buckley
|
b6391d1f90 |
fix(model-turn): one sampling-knob assignment scheme — alias > config > model definition > omit
Round-2 review fixes. The round-1 de-pinning collided with
ConfigStore.get's default-on-miss semantics: the registry defaults
(temperature 1.0, effort "medium") were manufactured onto every
store-backed lane's wire, making the documented "unset -> omit"
terminal unreachable. Unset is now representable end to end, and one
scheme governs every lane: per-model alias value > operator-stored
global setting > in-code model definition (effort only: caps
declaration) > field omitted, inference engine's default rules.
- settings_registry: model.temperature default None, model.reasoning_effort
default "" — the registered defaults ARE the unset sentinels, so the
admin UI and the wire agree. Admin webux renders nullable floats blank
("(inherit model default)") and maps blank-save to reset; the "" effort
choice reads "(inherit)".
- model_turn: resolve_temperature_setting/resolve_effort_setting are the
ONE pair of operator-rung resolvers, shared by resolve_lane, both
session factories, and the /model switch (the 4th-copy mirror is gone;
the switch no longer leaks the previous model's override on store-less
sessions). The caps rung moved out of the lane into model_turn's
effective computation, below a new request-shaped default_reasoning_effort
parameter (utility + output guard pass "low": budget coherence with
their small token caps, not sampling policy — any operator or
model-definition value beats it). The hidden "medium" terminal is gone.
- providers: Protocol + all adapters take reasoning_effort: str | None =
None (the Protocol-signature "medium" was the same manufactured pin one
layer down); ModelCapabilities.default_reasoning_effort defaults "" —
commercial rows all declare theirs explicitly, so only local lanes and
Anthropic change, both to match their real serving defaults (Anthropic
manual-thinking models no longer get implicit thinking-on-medium).
reasoning_template_kwargs distinguishes unset (inject nothing; template
default rules) from the explicit "none" off-switch. apply_temperature
skips temperature unless reasoning is EXPLICITLY off on none-declaring
models (unset leaves the server default in charge, possibly reasoning-on).
- session: ctor takes temperature: float | None / reasoning_effort:
str | None = None; _save_config/resume round-trip unset as "" (the
str(None) era guarded); _run_agent relays session temperature AND
effort on the same-alias fall-through only (a task alias's configured
knobs stay reachable in both directions).
- optimizer: the five meta lanes are decoupled from --temperature/
--reasoning-effort (test-model knobs, per their documented meaning);
registry-less meta lanes omit both fields.
- cli: --temperature/--reasoning-effort default unset and fall through
the model config instead of pinning 0.5/"medium" for every CLI session.
- cleanup from the review's below-cap findings: dead resolve_server_type
deleted (tests re-pointed at _server_type_of), stale ChatSession
comments in _openai_responses fixed, _store_get_or_none extracted,
eval system-turn conversion hoisted out of the per-turn loop, dead
_provider_extra_params patch removed, test_perception uses the shared
mock_completion_result, effort_ladder uses apply_capability_overrides
instead of a SimpleNamespace fake config.
Wire goldens regenerated: the only drift is the manufactured "medium"
effort vanishing from unset-effort requests (Responses reasoning.effort,
Chat/Google reasoning_effort, Anthropic output_config.effort) — pure
removals, no additions. Ladder tests now fake ConfigStore with the REAL
get() semantics (registry default on miss) so a forgiving fake can't
mask this class of bug again.
|
||
|
Patrick Buckley
|
7e07f2ea93 |
feat(core): phase 2 — every single-shot lane speaks Turn IR (#827)
create_completion now has exactly one caller: model_turn. The π-side lanes migrate off hand-built OpenAI dicts: - _utility_completion (title gen, compaction, web-fetch extraction) takes list[Turn] and runs the session's primary lane through model_turn; its three call sites build Turn.system/Turn.user. - perception.describe builds a by-reference trajectory (AttachmentRef + the prebuilt parts via resolve_attachments, reintroduced on model_turn with its first caller and pinned by tests) — Turn IR never carries inline media bytes, matching the main loop's wire path. Its temperature=0.2 pin is gone (house rule). - eval HeadlessSession's loop lowers system prompts through the turns_from_dicts bridge and appends result.turn; the parallel-call cap now also drops the native lane on a capped turn (a capped mirror with a full native lane would replay orphan tool blocks). - optimizer: all five sites (diversifier, observer, analyst loop, tool optimizer, prompt optimizer) build Turn IR through per-function lanes; every temperature pin (0.8/0.3/0.3/0.3/0.6) removed per house rule — sampling behavior belongs in the model's configuration. Test mocks move to the shared full-shape helper where the model_turn re-ingest now runs; perception/attachment tests assert the by-reference placeholder + resolver contract instead of inline parts. |
||
|
Patrick Buckley
|
7da07e2350 |
perf(attachments): per-send wire-part memo to stop re-rasterizing every round-trip
_resolve_attachments re-runs on every agentic round-trip (and per fallback model), each time re-fetching every attachment across the full history and re-rasterizing / re-base64'ing it. A 10-page PDF in a 10-cycle tool turn was rendered dozens of times. Add a per-send memo (self._wire_part_cache) keyed by (attachment_id, caps-signature): the materialized wire part is computed at most once per send. The cache is None outside a send (display/export paths unaffected) and reset per send to bound the heavy rasterized-page parts and pick up any mid-session capability change. Skip the DB fetch entirely when every id is already cached. Also peek the perception (alias, content_hash) memo before building parts in _perception_fallback_part, so a cross-send describe hit no longer wastes a PDF rasterize. Leaves pdf.py's deliberate no-module-cache stance intact — the per-send scope addresses the round-trip amplification without the durable store it defers. Adds describe_peek() + per-send-cache and peek tests. |
||
|
Patrick Buckley
|
558ddadc79 |
feat(attachments): universal perception fallback for non-native modalities
Add a `perception.model_alias` model role: when the primary model can't ingest
an attachment natively and can't be shown a degraded-but-native form, a
configured perception model perceives it and its output is carried as text.
Mirrors the STT role — a role alias plus a module-level memo so the extra LLM
round-trip runs once per attachment, not once per conversation turn. The call
goes through the provider abstraction's create_completion (the path the intent
judge uses), so any vision/omni provider works.
Bottom-tier, universal ladder — perception only fills the remaining gap:
- pdf : native supports_pdf -> rasterize-to-vision-primary -> perception
-> extracted text -> placeholder
- image: native vision -> perception (non-vision primary) -> native image_url
- audio: native supports_audio_input -> STT -> perception (omni) -> placeholder
Folds in two review findings the role subsumes:
- bug-1: thread the active attempt's capabilities into _resolve_attachments
(bound in _try_stream) so a model fallback materializes attachments against
the fallback model's caps, not the primary's.
- bug-2: charge a by-reference pdf/audio a bounded budget min(size_bytes, 16K)
instead of zero, so a large-attachment turn isn't budgeted as ~empty (the
exact materialized size isn't known until wire build).
|