* refactor(session): make ModelLane the provider boundary (#979)
## Summary
This closes the model-lane ownership gap left by #832: `ChatSession` no longer stores raw provider/client handles. `ResolvedModelBinding` now carries the provider, client, model, capabilities, registry generation, and backend-auth configuration as one coherent snapshot.
- Atomically rebind existing sessions after model-registry changes while pinning each in-flight send, fallback, judge, output guard, task agent, title, compaction, perception, and voice operation to its initiating principal and binding.
- Fence UI publication, canonical trajectory folds, durable writes, streams, retries, child scopes, and judge work by generation. Stop can hand off to a successor without accepting late state; cancelled tools retain typed effect receipts, and concurrent approval batches resolve by exact cycle or call.
- Make create, fork, open, close, and delete race-safe with hidden `creating` reservations, incarnation-aware state tails, and an ACL-rechecked transaction that clones checkpoint-bounded history, configuration, project/persona state, and attachment references.
- Extend REST/OpenAPI and Python/TypeScript SDK contracts for create/fork inputs, routed-create metadata, live-workstream probes, targeted approvals, and structured cancellation results.
- Update architecture, storage, authentication, judge, channel, console, API, and SDK documentation, including regenerated architecture diagrams and OpenAPI artifacts.
## Validation
- SQLite suite: 11,188 passed, 9 skipped, 10 deselected
- PostgreSQL suite: 11,195 passed, 2 skipped, 10 deselected
- Live backend: 3 passed
- SSE recovery: 6 passed; browser recovery harness passed all scenarios
- Ruff: clean; 595 files correctly formatted
- mypy: 243 source files clean
- TypeScript: typecheck/build and 35 tests passed
- OpenAPI artifacts fresh; all 14 changed diagrams reproduce byte-for-byte
- `git diff --check` and Git LFS integrity clean
Closes#979.
* fix(deps): update nanoid for GHSA-2v37-7h3g-55p8
Refresh the transitive lock entry admitted by PostCSS so the TypeScript security gate no longer resolves the vulnerable custom-generator implementation.
Validation:
- npm ci
- npm audit --audit-level=moderate: 0 vulnerabilities
- TypeScript typecheck and build
- TypeScript tests: 35 passed
* fix(test): assert canonical model registry URLs
Replace prefix checks with exact canonical base URL assertions so the tests do not model incomplete URL validation.
Validation: tests/test_model_registry.py (185 passed); Ruff check/format; mypy.
The reranker_alias -> model-definition path (added when reranking became a model
role) made the older global endpoint settings redundant. Resolve reranking
solely through the Reranker role and remove the parallel global config.
- Removed settings tools.rerank_url / rerank_model / rerank_api_key, their
config.py getters (+ $TURNSTONE_RERANK_URL / $TURNSTONE_RERANK_MODEL and the
module caches), and the fallback branch in resolve_rerank_client_from. The
resolver now returns a client only when a Reranker model (capability
supports_rerank, base_url = its /rerank endpoint) is selected, else None.
- Kept as global knobs: reranker_alias (the selector), rerank_web_search,
rerank_bm25, rerank_bm25_threshold, and rerank_instruction -- a task-level
query knob (Qwen3-style), not endpoint identity.
- The Settings tab is registry-driven, so the three fields disappear with their
SettingDefs. Updated the Reranker role help, example config, and docs/tools.md.
BREAKING: a reranker configured via [tools] rerank_url (config.toml / env /
Settings tab) no longer works -- add the reranker in the admin Models tab and
pick it under Models -> Roles -> Reranker. No migration: reranking is days old
and disabled by default, so any orphaned tools.rerank_* config rows are inert.
Tests: the resolver covers no-store / no-alias / non-rerank-alias -> None and the
model-definition happy path; the obsolete global-fallback tests are removed.
Phase 3. Stores reranker calibration per-model on the model definition's
capabilities (rerank_threshold/rerank_scale/rerank_separated; a non-empty
rerank_scale is the "has been calibrated" marker) instead of a single global
threshold, populated automatically when a reranker endpoint is detected.
- ChatSession._bm25_rerank_threshold precedence: the active reranker model's
calibrated rerank_threshold (when separated) wins; calibrated-but-not-
separated -> 0 (no floor); else the global tools.rerank_bm25_threshold
fallback. Reads the raw caps dict, in-memory per turn.
- Detect (admin_detect_model) calibrates a supports_rerank endpoint and
autopopulates the three fields like context window; the create-model UI shows
a verdict chip (calibrated / no-clean-separation / not-calibrated).
- POST /api/admin/model-definitions/{id}/calibrate backs the Re-calibrate
button; calibration runs off the event loop (run_in_executor, bounded by
asyncio.timeout(90)), persists the fields + refreshes the registry, and is
graceful on a down/slow endpoint (never 500). Shared
merge_calibration_into_caps helper used by the endpoint and the CLI.
- turnstone-admin rerank-calibrate is now per-model: --model <alias> required;
--apply writes that model's caps (not the global setting). A no-separation
result records the marker (calibrated, no floor) consistently across CLI and
endpoint.
The serving lesson stays documented: Qwen3-Reranker needs vLLM --chat-template
or its scores are near-random (live-validated 0.6B + 4B: the calibrated floor
came out 0.95 vs 0.33 for the same task -- why per-model calibration exists).
Negative-tested: the floor-precedence branches (calibrated+separated -> per-
model, calibrated+!separated -> 0, uncalibrated -> global, no-alias -> global,
registry-must-not-be-consulted), endpoint persist/refresh + graceful failure,
detect-skips-non-rerankers, the CLI per-model write, and the caps-merge
preserving supports_rerank. Chip states verified via headless Chrome.
Phase 2 of BM25 reranking (follows #627). Makes the rerank_bm25_threshold floor
usable across reranker models and adds tooling to pick it.
- normalize_scores (rerank.py): map a rerank batch into a 0-1 relevance
probability -- sigmoid when any score falls outside [0,1] (logit endpoints
like bge/TEI), identity otherwise (Cohere/Jina/Qwen already 0-1). Applied in
the _bm25_reranker closure AND calibration so the threshold means the same on
every endpoint. Monotonic, so ranking order is unchanged.
- rerank_calibrate.py + `turnstone-admin rerank-calibrate [--apply]`: probe the
endpoint with labelled relevant/irrelevant groups, normalise, and recommend a
recall-biased floor -- or report "no clean separation" (a mis-served/weak
reranker). A warmup loop absorbs a cold endpoint's first-request compile so
calibration doesn't time out. Validated live against Qwen3-Reranker 0.6B and
4B: the calibrated floor differs sharply per model (~0.95 vs ~0.33 for the
same task) -- exactly why per-endpoint calibration exists.
- rerank_config.py: extract resolve_rerank_client_from(config_store, registry);
the alias/url precedence now lives in one place, shared by ChatSession (which
delegates) and the CLI.
- tools.rerank_instruction (config + setting + client): wrap the query as
<Instruct>:/<Query>: for instruction-aware rerankers (Qwen3) on endpoints that
don't apply the model's own chat template. Docs note the critical vLLM serving
detail: Qwen3-Reranker needs --chat-template or its scores are near-random and
reranking hurts retrieval.
Negative-tested: normalize sigmoid/identity branches, closure-normalises-before-
floor, calibration separation/recall-bias/warmup-absorbs-cold-start, the CLI
apply/no-apply/no-separation paths, and instruction query-wrapping through the
real httpx boundary.
Reuse the shipped Cohere/Jina rerank client as an optional post-process on
the BM25 surfaces (tool search, skill search, memory composition) via one
seam: BM25Index gains an injected reranker + a two-stage search (BM25 recall
top-50 -> rerank -> top-k). No new storage.
Gated on a configured endpoint plus tools.rerank_bm25 (default on, matching
rerank_web_search). tools.rerank_bm25_threshold (default 0.0 = off) is a
relevance FLOOR for proactive memory surfacing: BM25 always returns something,
so without a floor every-turn memory injection spends tokens on the top-k of
whatever lexically matched; the reranker score is what makes a meaningful
"inject nothing" gate possible.
Two reranker modes (BM25Index rerank_filters):
- REORDER (reactive tool/skill search): the reranker must never drop results
-> fall back to BM25 order on empty, backfill omitted pool items, so a
misbehaving endpoint can't silently lose tools.
- FILTER (memory, rerank_filters = threshold > 0): a clean empty/short result
is honoured (inject nothing) -- a deliberate divergence from
web_search._rerank_results.
Parse/endpoint failure is a discrete branch from the floor: an empty result
for non-empty input means an unparseable response (a conforming reranker
scores every doc), so the closure raises RerankError and BM25Index falls back
to BM25 order in BOTH modes -- the floor only acts on valid scores.
Also: cap the rerank client timeout at 15s (the per-turn memory path can't
afford tools.timeout's 120s default); move the Reranker alias to rerank.py
(shared, no import cycle); document the endpoint egress in the rerank_bm25
help, the admin Reranker-role description, and docs/tools.md; add
scripts/bench_bm25_rerank.py (manual, needs a live endpoint) to measure
precision@k/MRR lift and recommend a threshold default.
Negative-tested: reorder fallback-on-empty and omitted-item backfill,
filter-mode honor-empty, singleton-still-floored, the parse-fail RerankError
raise, the >= floor boundary, and pool-position-to-doc-index mapping -- each
guard reverted to confirm its test fails, then restored.
Reranking is delegated to an external Cohere/Jina-compatible /rerank endpoint
(self-hosted vLLM/TEI/llama.cpp, or hosted Cohere/Jina/Voyage); Turnstone runs
no reranker model itself. Disabled until an endpoint is configured.
- core/rerank.py: CohereJinaRerankClient (tolerant of results-wrapped and
bare-list responses) + resolver.
- web_search: rerank the SearxNG result pool by query relevance before top-k,
with a native-order fallback on error; answers/infoboxes untouched.
- Reranker as a model definition: add a model with the supports_rerank
capability and pick it under Models -> Roles -> Reranker
(tools.reranker_alias); takes precedence over the tools.rerank_url settings.
Settings: tools.rerank_url/model/api_key, tools.rerank_web_search,
tools.reranker_alias. Docs: docs/tools.md, turnstone.example.toml.
(web_fetch reranking was evaluated and dropped: for single-document chunk
selection it did not reliably beat head-truncation. Reranking is reserved for
multi-item ranking.)
Patrick Buckley