Follow-up to #938; closes#941. The unavailable-server advisory fired for
users whose own pool was warm: _pool_discovery_error was keyed by server
name while pool connections are per-(user, server), so one account's
failed prime rendered its exception text into every user's search results.
- mcp_client: re-key _pool_discovery_error to (user_id, server_name).
Written by the failing user's prime (single sanitize-and-cap pipeline
shared with _set_error), cleared by that user's successful connect,
retired with the grant on explicit disconnect / dead-grant convergence,
and swept name-wide on registration lifecycle (removal, reconcile
auth-type flips) via a snapshot-safe helper. Departed users' records
are reaped by the eviction tick's orphan sweep — the single tick-side
reaper; a live user's record survives its stub's eviction because the
advisory has no mid-session re-record path. The eviction loop also
starts on record write, so records written before any pool entry
exists cannot outlive their users. Status reads scope to the
requesting user, with an any-user view under the admin aggregate flag.
- tool_search: _status_reason treats discovery_error as an outage only
when the requesting user's own status is not connected — with per-user
records this is belt-and-braces, since a successful connect clears the
user's record.
- session: the tool-search status snapshot scopes to the EFFECTIVE user
(the acting participant on shared workstreams), matching the get_tools
call that builds the search corpus, so an owner's pool state never
renders into a non-owner's results.
- bm25: with a reranker attached, matches ranked past the recall pool
trail in BM25 order (reorder mode), so tool_search's "top N of M"
count no longer floors at the pool size; the exception fallback is
mode-aware (filter mode keeps its pool bound, byte-for-byte).
Tool discovery for a pool-backed (oauth_user/oauth_obo) MCP server that is
down or 5xx-ing was invisible: the server contributed zero tools to the
catalog, so tool_search returned "No matching tools found" —
indistinguishable from a genuine no-match — and matches past max_results
were silently dropped with no signal.
- tool_search: search() ranks the whole deferred corpus and records the
pre-slice match count so format_search_results can report honest
truncation ("top N of M"). An optional status_provider lets results name
servers that are actually failing (open circuit breaker, recorded error,
recorded discovery failure) instead of masquerading as "no such tool".
Un-primed servers are deliberately not flagged, and a provider that
raises never breaks search.
- mcp_client: the previously swallowed pool prime/connect discovery
failure is recorded per server (single-line, bounded), cleared on the
next successful pool connect, on removal, and on reconcile-observed pool
removal or auth-type flips; exposed via get_server_status
as "discovery_error".
- session: wires get_all_server_status(user_id) into both
ToolSearchManager constructions as a lazily-called status provider.
Reuse the shipped Cohere/Jina rerank client as an optional post-process on
the BM25 surfaces (tool search, skill search, memory composition) via one
seam: BM25Index gains an injected reranker + a two-stage search (BM25 recall
top-50 -> rerank -> top-k). No new storage.
Gated on a configured endpoint plus tools.rerank_bm25 (default on, matching
rerank_web_search). tools.rerank_bm25_threshold (default 0.0 = off) is a
relevance FLOOR for proactive memory surfacing: BM25 always returns something,
so without a floor every-turn memory injection spends tokens on the top-k of
whatever lexically matched; the reranker score is what makes a meaningful
"inject nothing" gate possible.
Two reranker modes (BM25Index rerank_filters):
- REORDER (reactive tool/skill search): the reranker must never drop results
-> fall back to BM25 order on empty, backfill omitted pool items, so a
misbehaving endpoint can't silently lose tools.
- FILTER (memory, rerank_filters = threshold > 0): a clean empty/short result
is honoured (inject nothing) -- a deliberate divergence from
web_search._rerank_results.
Parse/endpoint failure is a discrete branch from the floor: an empty result
for non-empty input means an unparseable response (a conforming reranker
scores every doc), so the closure raises RerankError and BM25Index falls back
to BM25 order in BOTH modes -- the floor only acts on valid scores.
Also: cap the rerank client timeout at 15s (the per-turn memory path can't
afford tools.timeout's 120s default); move the Reranker alias to rerank.py
(shared, no import cycle); document the endpoint egress in the rerank_bm25
help, the admin Reranker-role description, and docs/tools.md; add
scripts/bench_bm25_rerank.py (manual, needs a live endpoint) to measure
precision@k/MRR lift and recommend a threshold default.
Negative-tested: reorder fallback-on-empty and omitted-item backfill,
filter-mode honor-empty, singleton-still-floored, the parse-fail RerankError
raise, the >= floor boundary, and pool-position-to-doc-index mapping -- each
guard reverted to confirm its test fails, then restored.
* Add dynamic tool search with native defer_loading for Anthropic/OpenAI
When MCP tools push the total tool count past a configurable threshold
(default 20), tool definitions are deferred to reduce token overhead and
improve tool selection accuracy. Three-tier approach mirrors the existing
web search pattern:
- Anthropic (Claude 4.x): native defer_loading + server-side BM25 search
- OpenAI (GPT-5.4+): native defer_loading + hosted search
- vLLM/llama/NIM: client-side BM25 fallback via synthetic tool_search tool
New module turnstone/core/tool_search.py with BM25Index (pure-Python,
zero deps) and ToolSearchManager (session-scoped visibility, expansion,
server hint generation). Discovered tools persist for the session lifetime
so the model only searches once per capability needed.
Config: [tools] search/search_threshold/search_max_results
CLI: --tool-search {auto,on,off}, --tool-search-threshold, --tool-search-max-results
Agents (plan/task) exempt — their scoped tool sets are always small.
43 new tests (1253 total). All diagrams regenerated with PlantUML 1.2025.2.
* Fix Copilot review feedback on tool search
- Fix _MCP_PREFIX_RE to handle underscores in server names (non-greedy match)
- Use ordered dict for _expanded to preserve tool discovery order
- Avoid constructing ToolSearchManager when below threshold in auto mode
- Return empty string from _mcp_server_summary when no servers (not "none")
- Fix CLI help text to reference threshold generically, not hardcoded "20"
- Fix agent exemption docs to accurately describe scoped tool sets
- Fix README to not hardcode "30+" threshold number
Patrick Buckley