Compare commits

...

166 Commits

Author SHA1 Message Date
Patrick Buckley d5e86c8493 release: v0.9.5 2026-03-29 23:02:53 -07:00
Patrick Buckley c154ea3966 fix: subscribe to workstream events before sending first Discord message (#250)
The first message sent from Discord was silently dropped because the
cog delegated the initial message to the bridge via CreateWorkstream-
Message, but the bridge published response events to the per-workstream
Redis pub/sub channel before the Discord bot had subscribed to it.
Redis pub/sub is fire-and-forget — events with no subscribers are lost.

Fix: create the workstream with initial_message="" (no delegation),
subscribe to the per-workstream event channel, then send the message
through router.send_message() — the same path the second message
already uses successfully.

Applied to both @mention handler and /ask slash command.
2026-03-29 22:57:56 -07:00
renovate[bot] 755ab51802 chore(deps): lock file maintenance (#249)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-03-29 22:22:49 -07:00
Patrick Buckley 9df8ab836f Feat/per pane status bar (#248)
* feat: per-workstream status bar above input

Move the global token counter and model name from the header into a
per-pane telemetry strip between messages and the text input. Each
workstream pane now independently shows model name, token usage with
context percentage, tool calls this turn, and turn count.

Backend: add _ws_turn_tool_calls counter (reset per user turn, emitted
in SSE status event alongside turn_count). MQ bridge forwards the new
fields. SDK and TypeScript types updated.

Frontend: build .ws-status-bar DOM in _createDOM, rewrite updateStatus
to target per-pane elements, update SSE connect/disconnect handlers.
Remove #model-name and #status-bar from global header. Restore console
#status-bar CSS in its own stylesheet.

Accessibility: aria-atomic, aria-labels on each field, warning symbols
(▲/⚠) at 80%/95% context for color-blind users, placeholder text
before first status event. Disconnect state uses 2px red border with
dimmed stale fields.

* fix: emit status event on SSE connect so status bar populates on resume

When resuming a workstream, the event_generator only sent connected +
history events. The status bar stayed at placeholder values until the
next LLM response. Now replays session._last_usage as a synthetic
status event right after connected, so token count, tool calls, and
turn count render immediately.

* fix: address Copilot review — remove dead function, clarify locals

Remove updateHeaderForFocusedPane() and its call site (no-op since
status moved per-pane). Rename ambiguous ttc/tc locals to
turn_tool_calls/turn_count in the status replay block.
2026-03-29 22:22:06 -07:00
Patrick Buckley 8d88e6a7eb feat: add memory get action, reduce search/list preview to 200 chars (#247)
* feat: add memory get action, reduce search/list preview to 200 chars

search and list truncated memory content to 500 chars with no way to
read the full value.  Two changes:

- New 'get' action retrieves a single memory by name with complete
  untruncated content.  Searches scopes narrowest-first (workstream
  → user → global).
- search/list previews reduced from 500 to 200 chars now that get
  exists for full content.  Both append a hint:
  "Use memory(action='get', name='...') for full content."

Includes get_structured_memory_by_name wrapper in memory.py and
4 tests.

* Update turnstone/tools/memory.json

* fix: include 'get' in _prepare_memory docstring and invalid-action error
2026-03-29 20:53:48 -07:00
Patrick Buckley cce292f793 fix: strip NUL bytes in both storage backends via shared sanitize_text
PostgreSQL text fields cannot store NUL (0x00) bytes, and SQLite
stores them but they cause downstream issues (API payloads, web UI).
Add sanitize_text() to _utils.py and apply it in both backends'
save_message to content and provider_data fields.
2026-03-29 19:20:42 -07:00
Patrick Buckley 6adc577d30 release: v0.9.4 2026-03-29 18:32:44 -07:00
Patrick Buckley 02c50b81c1 docs: update tool counts, add diff_file docs, new params (#244)
* docs: update tool counts, add diff_file docs, new params

- Tool count 17/18 → 19 across tools.md, architecture.md, and
  PlantUML diagrams (02-package-structure, 05-tool-pipeline)
- Add diff_file tool documentation section
- Document new params: bash timeout + stop_on_error, write_file
  mode (append), edit_file replace_all
- Add diff_file, watch, skill to tool pipeline dispatch table
- Regenerate diagram PNGs

* fix: remove slim dpkg exclusion so man pages are actually installed

The python:3.14-slim image excludes /usr/share/man/* via dpkg config.
man-db was installed but had no pages to serve.  Remove the exclusion
before installing packages, and add manpages package for coreutils
documentation.  Dropped info (rarely used, man covers the same).

* fix: redact DB connection strings and URL-based secrets in output guard

The output redactor missed TURNSTONE_DB_URL and DATABASE_URL because
the env secret key pattern only matched SECRET/TOKEN/PASSWORD/KEY,
not URL-based credential keys.  Also the connection string regex
didn't cover the postgresql+psycopg:// scheme used by psycopg3.

- Add DATABASE_URL, TURNSTONE_DB_URL, DB_URL to explicit env key matches
- Add psycopg and sqlite to connection string scheme pattern

* fix: address Copilot review on docs — tool names, counts, approval

- Fix remaining 17→19 count in tools.md execution pipeline section
- Dispatch table: task→task_agent, plan→plan_agent (match actual names)
- Dispatch table: header clarifies "19 built-in + tool_search"
- watch/skill: show conditional approval (create only / load only)
- Regenerate pipeline diagram PNG
2026-03-29 18:28:21 -07:00
Patrick Buckley 753cd04b4e Fix/orphaned tool results (#243)
* fix: drop orphaned tool_results with no matching tool_use in _convert_messages

The context window increase from 200K to 1M for Claude 4.6 means
conversations that previously triggered auto-compaction now send their
full history.  Older messages with orphaned tool_results (from
pre-fix cancels or compaction boundaries) are now visible to the API,
causing "unexpected tool_use_id in tool_result blocks" errors.

The existing repair code handles orphaned tool_use (synthesizes
missing results), but not the reverse.  Now validates each
tool_result against the preceding assistant message's tool_use IDs
and silently drops results with no match.

* fix: filter empty IDs from prev_tool_use_ids, document pass-through

Code review: empty-ID tool_use blocks were added to the filter set,
and the intentional pass-through when prev_tool_use_ids is empty
needed documentation.
2026-03-29 18:26:25 -07:00
Patrick Buckley c3217748dc fix: block math sandbox escape via getattr/setattr/type reflection (#239)
* fix: block math sandbox escape via getattr/setattr/type reflection

getattr() with runtime-constructed strings bypassed the AST validator,
allowing full os/subprocess access from the sandboxed math tool via
module.__builtins__['__import__']('os').

Three-layer fix:
- Block getattr, setattr, delattr, type, __import__ in
  _MATH_BLOCKED_BUILTINS (prevents direct calls)
- Add AST validation for getattr/setattr/delattr call nodes
  (catches them even if builtins dict is bypassed)
- Strip __builtins__ from all pre-imported modules in the execution
  namespace (runtime defense — even if AST is somehow bypassed,
  module.__builtins__ returns empty dict)

Normal math, sympy, numpy, scipy operations unaffected.

* fix: harden _safe_import to strip __builtins__ from runtime imports

Copilot review: modules imported at runtime via _safe_import still
had their original __builtins__ dict, accessible via
operator.attrgetter('__builtins__').  Now _safe_import strips
__builtins__ from every module it returns.  Also blocks
operator.attrgetter/itemgetter at the AST level, and removes the
redundant duplicate getattr check in visit_Call.

* fix: add type ignore for module __builtins__ assignment
2026-03-29 17:37:59 -07:00
Patrick Buckley 8cbff49694 fix: block /proc/*/environ access in bash filter and judge heuristic (#240)
* fix: block /proc/*/environ access in bash filter and judge heuristic

/proc/1/environ leaks the full server environment including DB
credentials, API keys, and JWT secrets.  Env scrubbing in env.py
only affects subprocess calls, not procfs reads.

- Add /proc/1/environ and /proc/self/environ to BLOCKED_PATTERNS
  in safety.py (hard block)
- Add proc-environ-exfil heuristic rule at critical severity with
  deny recommendation (catches /proc/<pid>/environ patterns)

* fix: move proc-environ-exfil rule to _CRITICAL_RULES list

Copilot review: rule had risk_level=critical but was placed in
_HIGH_RULES.  Move to _CRITICAL_RULES for consistency with the
first-match-wins severity ordering.
2026-03-29 17:37:46 -07:00
Patrick Buckley 4f26d63c14 perf: trim judge context to messages from last user turn onward (#241)
The intent judge was receiving up to 50% of the context window in
conversation history (FIFO from end), which grows linearly with
conversation length and causes increasing latency.  The judge only
needs the immediate request context to evaluate a tool call's safety.

Now trims to messages from the last user message onward before
applying the FIFO budget cap.  Keeps the user's request, the
assistant's response with tool calls, and any recent tool results
while discarding earlier conversation that isn't relevant to the
current intent evaluation.
2026-03-29 17:34:41 -07:00
Patrick Buckley 74347fb29f fix: update Claude 4.6 context windows to 1M, remove EOL 4.0 models (#242)
Claude 4.6 (Opus + Sonnet) unified on 1M token context windows.
Update capabilities table from 200K to 1M for both models.  Remove
claude-opus-4 and claude-sonnet-4 entries (end of life).  4.5 models
remain at 200K.  Default fallback stays at 200K for unknown models.
2026-03-29 17:28:45 -07:00
Patrick Buckley 2ace8cccc8 fix: distinguish user cancel from crash in bash tool results (#235)
* fix: distinguish user cancel from crash in bash tool results

When a user cancels a running bash command, the process is killed
with SIGKILL (exit code -9).  Previously this showed as an error,
causing the model to retry.  Now checks cancel.is_set() after proc
exit and returns "Cancelled by user." as a non-error result so the
model knows to stop rather than retry.

* fix: use -signal.SIGKILL instead of magic -9

Copilot review: replace hard-coded -9 with -signal.SIGKILL for
clarity.  Popen.returncode is negative of signal number when killed.
2026-03-29 17:09:44 -07:00
Patrick Buckley e95b8f5ca1 feat: add stop_on_error param to bash tool for set -e behavior (#236)
* feat: add stop_on_error param to bash tool for set -e behavior

New boolean parameter enables 'set -e' in the bash preamble so
multi-step scripts exit on the first command failure instead of
silently continuing.  Default false (existing behavior preserved).
pipefail remains always-on.

* fix: strict bool parsing for stop_on_error, treat exit 1 as error with set -e

Copilot review: bool("false") is True — use `is True` for strict
JSON boolean parsing.  Also, with stop_on_error enabled, any non-zero
exit code is now treated as an error (set -e means the script halted
on failure), whereas without it exit code 1 remains benign.
2026-03-29 17:09:29 -07:00
Patrick Buckley 7a32c51a1c fix: synthesize cancelled tool results instead of stripping turns (#237)
* fix: synthesize cancelled tool results instead of stripping turns

When a user cancels during tool execution, the model previously lost
all context about what was attempted (assistant message + tool_calls
stripped entirely).  Now synthesizes tool_result messages with
is_error=true and "Cancelled by user." content for any tool_calls
that lack matching results.  This keeps the conversation valid for
both providers while preserving the full tool call structure so the
model knows what was tried.

Also applies to KeyboardInterrupt with "Interrupted by user." text.

* fix: persist synthesized cancel results to DB, assert is_error in test

Copilot review: synthesized tool messages were in-memory only,
creating a mismatch with DB that could break rewind/retry.  Now
calls save_message() for each synthesized result.  Also adds
is_error=True assertion to the cancel test.
2026-03-29 17:09:17 -07:00
Patrick Buckley dce663105b feat: add pagination and longer content to recall tool (#238)
* feat: add pagination and longer content to recall tool

- New offset parameter for paginating through recall results
- Content preview increased from 500 to 2000 chars per match with
  total length indicator when truncated
- Output passed through _truncate_output for consistency
- OFFSET clause added to SQLite (FTS5 + LIKE) and PostgreSQL
  (tsvector + ILIKE) search queries

* fix: defensive int coercion for recall offset/limit

Copilot review: offset/limit could arrive as null, float, or other
non-int types from JSON.  Coerce with int() + try/except in prepare,
and int() at the storage layer before binding into SQL OFFSET/LIMIT.
2026-03-29 17:09:05 -07:00
Patrick Buckley cfef3616e6 feat: add diff_file tool for comparing files and content (#234)
* feat: add diff_file tool for comparing files and content

New read-only tool that shows unified diffs between two files or
between a file and provided content.  Useful for verifying edit_file
changes and comparing file versions.  Auto-approved (no side effects).
Configurable context lines (default 3).  Available to task agents.

* refactor: extract _read_text_lines helper, share across read_file and diff_file

Copilot review: diff_file duplicated file-loading and lacked binary
detection.  Extract _read_text_lines() that handles realpath
resolution, null-byte binary detection, and error handling.  Used by
both _exec_read_file and _exec_diff for consistent behavior.

* fix: address code review — agent flag, resolved shadowing, read_files

- Add agent: true to diff_file schema so plan agents can use it
- Fix resolved variable shadowing in _exec_read_file (use _ for
  unused return from _read_text_lines)
- Register diffed files in _read_files so edit_file read guard
  is satisfied after diff_file
- Move difflib import to module level (stdlib, no lazy-load needed)
- Fix description wording ("provided string" not "previous version")

* fix: stream diff with early cutoff, expand paths before header

- Stream difflib output and stop collecting after tool_truncation
  chars to avoid large intermediate allocations on big diffs
- Expand paths with expanduser before building the approval header
  so display matches actual execution paths
2026-03-29 16:17:48 -07:00
Patrick Buckley c22d39a798 docs: tool descriptions, bash timeout param, multi-line preview (#233)
* docs: tool descriptions, bash timeout param, multi-line preview

- task_agent/plan_agent: document the tool subset limitation (no
  memory, recall, watch, skill, or further delegation)
- bash: add per-call timeout parameter (1-600s, defaults to 120s),
  shown in approval header when specified
- bash: show full command in preview for multi-line scripts so the
  approval flow displays the complete command, not just the first line
- bash: document 256KB output cap and stderr prefix in description

* fix: address Copilot review on tool descriptions

- bash: say "truncated" not "256KB" (limit is configurable), document
  timeout clamping range (1-600) and global fallback
- bash preview: fix "1 more lines" → "1 more line" singular
- plan_agent: remove bash from listed tools (not in AGENT_TOOLS)
2026-03-29 16:17:35 -07:00
Patrick Buckley 63921450b1 fix: improve memory save error message, narrow dd command filter (#232)
* fix: improve memory save error message, narrow dd command filter

Two minor fixes from harness shakedown:

- memory save: split "both name and content required" into separate
  errors for missing name vs empty content
- bash safety: replace blanket "dd if=" block with targeted patterns
  for writes to block devices (of=/dev/sd*, /dev/nvme*, /dev/disk/,
  etc.) and redirects to the same.  Legitimate dd use like generating
  test data or benchmarking reads is no longer blocked.

* fix: generalize > /dev/sda redirect pattern to > /dev/sd

Copilot review: only /dev/sda was blocked for redirects while
/dev/sdb, /dev/sdc etc were not.  Generalize to match any /dev/sd*
device, consistent with the of= patterns.
2026-03-29 16:17:22 -07:00
Patrick Buckley 929fad63be feat: edit_file replace_all, write_file append mode, search match count (#231)
* feat: edit_file replace_all, write_file append mode, search match count

Three tool enhancements from harness shakedown feedback:

- edit_file: new replace_all parameter replaces all occurrences of
  old_string instead of requiring a unique match.  Cannot combine with
  near_line or edits array.
- write_file: new mode parameter with "append" option.  Appends
  content to end of file instead of truncating.
- search: output now includes a summary footer showing total match
  count and file count (e.g. "47 matches across 12 files").

* fix: address Copilot review on tool enhancements

- replace_all: skip multi-occurrence rejection in pre-validation so
  the feature actually works; show occurrence count in preview
- write_file mode: coerce non-string types safely via str()
- search footer: append before truncation to respect output limits
- edit_file error: mention replace_all as alternative to near_line
2026-03-29 16:17:11 -07:00
Patrick Buckley 976e9df3b6 ci: suppress CVE-2026-25210 (libexpat1, no fix available) (#230)
Integer overflow in libexpat1 2.7.1-2 with no patched version in
Debian repos yet.  Suppress in Trivy until a fix is published.
2026-03-29 15:35:20 -07:00
Patrick Buckley 7cb21b84f1 fix: detect binary files in read_file instead of silent corruption (#227)
read_file silently converted null bytes to spaces, showing corrupted
content with no warning.  Now samples the first 8KB for null bytes and
returns a clear error directing the user to bash for binary inspection.
2026-03-29 15:32:04 -07:00
Patrick Buckley 7263edd48d fix: memory delete searches all scopes when scope not specified (#228)
* fix: memory delete searches all scopes when scope not specified

Previously delete defaulted to scope=global, so deleting a
workstream-scoped memory without explicitly passing scope=workstream
silently failed.  Now tries narrowest scope first (workstream → user
→ global) and deletes the first match.  Explicit scope still honored
when provided.

* fix: reject invalid scope on memory delete instead of silent fallback

Copilot review: invalid scope values were silently treated as
unspecified, which could cause accidental deletion from the wrong
scope.  Now returns a clear error listing valid scopes.
2026-03-29 15:29:14 -07:00
Patrick Buckley 6742c7e405 fix: exclude build/vendor/VCS directories from search tool (#226)
* fix: exclude build/vendor/VCS directories from search tool

grep -rn recursed into .git, node_modules, target, __pycache__, etc.
producing hundreds of noise hits from generated content.  Add
--exclude-dir flags for common directories that should never appear
in search results.

* fix: glob egg-info pattern and add vendor exclude

Copilot review: .egg-info misses turnstone.egg-info (named dirs),
use *.egg-info glob.  Also add vendor to the exclude list.
2026-03-29 15:28:52 -07:00
Patrick Buckley 1aa6982868 fix: add git, curl, jq, man-db, info to Docker image (#225)
Agent workflows need git for version control, curl for raw HTTP
requests, jq for JSON processing, and man/info for documentation
lookup.  All were missing from the slim base image, leaving the man
tool non-functional and standard dev workflows broken.
2026-03-29 15:28:38 -07:00
Patrick Buckley 42d1abbd04 fix: block IPv6 loopback/link-local/private in SSRF filter (#224)
* fix: block IPv6 loopback/link-local/private in SSRF filter

check_ssrf used gethostbyname which only resolves IPv4.  IPv6 addresses
like ::1, fe80::, fd00:: bypassed the filter entirely.  Switch to
getaddrinfo which resolves both address families and check all results.

* fix: handle IPv4-mapped IPv6 and zone IDs in SSRF filter

Copilot review caught two bypasses: ::ffff:127.0.0.1 (IPv4-mapped
IPv6) wasn't normalized before private/loopback checks, and fe80::1%lo0
(zone ID suffix) caused a ValueError that was silently swallowed.
Now normalizes IPv4-mapped addresses and strips zone IDs before parsing.
2026-03-29 15:28:27 -07:00
Patrick Buckley f543ed714a fix: resolve symlinks before file I/O to prevent path-based bypass (#223)
* fix: resolve symlinks before file I/O to prevent path-based bypass

write_file and edit_file followed symlinks silently — a symlink at
/data/link → /etc/passwd would show the /data path in the approval
header while writing to the real target.  Three changes:

- open() calls in _exec_write_file, _exec_edit_file, _exec_read_file
  now use the resolved (realpath) path instead of the raw symlink
- Approval headers show both paths when a symlink is detected
  (e.g. "⚙ write_file: /data/link → /etc/passwd")
- Judge _get_arg_text includes the resolved path so heuristic rules
  like write-system-path fire even through symlinks

* fix: address Copilot review — expanduser in fallback, pre-read, image paths

- edit_file exec fallback: add expanduser before realpath (tilde bypass)
- judge _get_arg_text: compare resolved against abspath(expanduser(path))
  so ~/ paths don't false-positive as symlinks
- edit_file pre-read: use resolved path instead of raw symlink path
- _exec_read_image: use resolved path for getsize and binary open
2026-03-29 15:28:13 -07:00
Patrick Buckley 2c6abb0fde fix: clear dedup sigs after write tools to avoid false repeat warnings (#229)
* fix: clear dedup sigs after write tools to avoid false repeat warnings

The read→edit→read workflow triggered "identical repeat" warnings
because the dedup tracker compared (tool_name, args) without
considering intervening state changes.  Now clears the signature set
when write_file, edit_file, or bash executes successfully, so
subsequent reads of the same file are not flagged.

* fix: use shared error prefixes for write-success detection in dedup

Copilot review: the error detection for write tools only checked
"Error" prefix, missing "Command timed out", "Blocked:", "Denied",
etc.  Now shares the same _error_prefixes tuple used by the repeat
detection below, ensuring consistent classification.
2026-03-29 15:27:59 -07:00
Patrick Buckley 491fc6748a fix: judge double tool conversion on Anthropic (#222)
The judge pre-converted tool schemas via convert_tools() before
passing them to create_completion(), which internally calls
convert_tools() again. The second conversion tried to extract
function.name from already-converted Anthropic-format tools,
producing empty tool names that the API rejected with
"tools.0.custom.name: String should have at least 1 character".

Fix: pass raw OpenAI-format schemas directly — create_completion
handles the provider-specific conversion.
2026-03-29 14:51:11 -07:00
Patrick Buckley 9a996f0067 release: v0.9.3
- fix: orphaned tool_use followup — ordering, empty IDs, provider_content bypass, universal repair (#220)
- feat: /retry and /rewind commands, message action controls in web UI (#221)
- docs: update tools, architecture, SDK for v0.9.2 changes
2026-03-29 14:19:02 -07:00
Patrick Buckley a465ac6383 Feat/rewind retry (#221)
* feat: add /retry and /rewind commands for conversation history navigation

Allow users to re-send the last message for a new response (/retry) or
drop the last N turns to restore an earlier conversation state (/rewind N).
Both operations sync in-memory state with the persistent database.

Server path includes conversation.modify permission gate, audit trail
(conversation.rewind / conversation.retry events), and thread-safe retry
dispatch. Migration 029 grants the permission to admin and operator roles.

* feat: add message action controls for retry, edit, and rewind in web UI

Hover toolbar on messages with CSS-only icons matching instrument panel
aesthetic. User messages get edit (pencil) and rewind (chevrons) buttons;
last assistant message gets retry (circular arrow). Edit flow uses
event-driven coordination — rewind completes via SSE history event before
send fires. Includes ARIA labels, keyboard nav, touch device support,
reduced motion, and busy-state gating.
2026-03-29 14:18:39 -07:00
Patrick Buckley a4539923e4 fix: orphaned tool_use followup — ordering, empty IDs, universal repair (#220)
Addresses Copilot review feedback on #219:

1. Anthropic _convert_messages: collect tool_use IDs in order (list
   not set), filter empty IDs, defer synthetic results until after
   real tool results so _merge_consecutive produces correct ordering.

2. Universal repair in reconstruct_messages: synthesize tool results
   for mid-conversation orphaned tool calls on DB load. Benefits all
   providers (OpenAI is lenient today but may tighten).

3. Test improvements: assert on is_error flag instead of "cancelled"
   substring, verify real-before-synthetic ordering in partial results.
2026-03-29 13:33:58 -07:00
Patrick Buckley 42e99d6990 docs: update tools, architecture, SDK for v0.9.2 changes
- docs/tools.md: batch edit_file (edits array), bash stderr prefix,
  math sandbox extras, output truncation
- docs/judge.md: JSON secret detection in output guard
- docs/architecture.md: state_change now sent to per-workstream SSE
- README.md: [sandbox] extras group in requirements
- TypeScript SDK: StateChangeEvent type, type guard, exports
- OpenAPI specs regenerated
2026-03-29 06:06:30 -07:00
Patrick Buckley a0ff22e137 release: v0.9.2
- fix: UI busy state during multi-tool-call turns (#216)
- feat: batch edit_file, sandbox packages, stderr labels, JSON secret redaction, model resume (#217)
- fix: Anthropic sub-agent streaming timeout (#218)
- fix: synthesize tool_results for orphaned tool_use blocks on Anthropic (#219)
2026-03-29 05:56:40 -07:00
Patrick Buckley de4b3b3909 fix: synthesize tool_results for orphaned tool_use blocks on Anthropic (#219)
When a cancel interrupts tool execution, the assistant message with
tool_use blocks is saved to DB before tools run, but
GenerationCancelled prevents tool results from being created. The
in-memory rollback removes the orphaned message, but the DB row
persists. On resume, Anthropic rejects the conversation with
"tool_use ids were found without tool_result blocks".

Fix: _convert_messages now peeks ahead after each assistant message
with tool_use blocks. If any tool_use IDs lack matching tool_result
messages, synthetic error results are injected (is_error: true,
"Tool execution was cancelled."). Transparent to all callers,
provider-specific (OpenAI is lenient about this).

5 new tests covering: single orphan, multiple orphans, partial
results, complete results (no synthesis), and trailing orphan.
2026-03-29 05:54:19 -07:00
Patrick Buckley 120d229b5f fix: Anthropic sub-agent streaming timeout (#218)
plan_agent and task_agent fail on Anthropic models with "Streaming is
required for operations that may take longer than 10 minutes" from
the SDK. The non-streaming create_completion path used
client.messages.create() which the SDK rejects for thinking-enabled
models.

Fix: use client.messages.stream() internally and call
get_final_message() to get the same Message object. Transparent to
all callers — fixes sub-agents, title generation, summarization,
web fetch, and judge create_completion calls.
2026-03-29 05:35:27 -07:00
Patrick Buckley c6f4c11870 feat: harness quick wins — batch edit_file, sandbox packages, stderr labels, JSON secret redaction, model resume (#217)
Five improvements from Opus self-evaluation of the turnstone harness:

1. Batch edit_file: edits array parameter for atomic multi-edit in a
   single tool call. Overlap detection, reverse-order application,
   mutual exclusivity with single-edit params.

2. Sandbox packages: new [sandbox] extras group with sympy, numpy,
   scipy, pytest — the sandbox already had graceful ImportError
   fallbacks, now the packages are actually installed.

3. Stderr labeling: bash tool output prefixes stderr lines with
   [stderr] so the model can distinguish errors from stdout.

4. JSON secret redaction: output guard now detects and redacts secrets
   in JSON format ("api_key": "...", "password": "...", etc.) with
   18 key patterns and 8-char minimum value length.

5. Model persisted on resume: workstream config now saves model and
   model_alias. Resume restores the original model via registry
   (same path as /model command), falling back to raw model name
   if the alias is no longer available.

24 new tests (23 in test_edit_file.py, 1 in test_sessions.py).
2026-03-29 05:21:08 -07:00
Patrick Buckley 979fab37a9 fix: UI busy state during multi-tool-call turns (#216)
stream_end fires per-segment (between tool calls), not per-turn.
The UI was using stream_end to transition to idle, causing a window
where the Send button appeared but the server worker thread was still
alive. User messages submitted during this window were silently
dropped. No Stop button was visible, so the user had no cancel path.

Root cause: state_change events (idle/thinking/running/error) were
only broadcast to the global SSE stream (console dashboard), never
to the per-workstream SSE that the browser UI listens to.

Fix: (1) server.py: on_state_change now also enqueues to the
per-workstream SSE listeners. (2) app.js: stream_end no longer
calls setBusy(false) — it only finalizes markdown rendering.
New state_change handler manages busy transitions: idle/error
set busy=false, thinking/running set busy=true.
2026-03-29 05:20:47 -07:00
Patrick Buckley da5bf90a4b feat: model detect button, capabilities API, and model dropdowns (#215)
* feat: model detect button, capabilities API, and model dropdowns

Admin Models tab: add Detect button that probes a model endpoint to
verify reachability, list available models, detect context_window, and
identify server type (llama.cpp/vLLM/SGLang/OpenAI/Anthropic). Add
static capability lookup endpoint for auto-filling form fields when
a known model name is entered. Add known-models endpoint for datalist
autocomplete suggestions.

Add "openai-compatible" as a third provider option for local servers,
keeping the OpenAI SDK under the hood but suppressing capability
auto-fill and known-model suggestions.

Replace free-text model input with a select dropdown in both console
and server new-workstream modals, populated from a new lightweight
GET /v1/api/models endpoint.

New endpoints:
- POST /v1/api/admin/model-definitions/detect
- GET /v1/api/admin/model-capabilities
- GET /v1/api/admin/model-capabilities/known
- GET /v1/api/models (both console and server)

* fix: accumulate signature_delta for Anthropic thinking blocks (#214)

The streaming path captured thinking_delta events but not
signature_delta, leaving the signature empty on round-trip and
causing 400 errors on multi-turn conversations with thinking enabled.

* fix: address PR review — empty base_url, capability leak, response schemas

- Don't pass empty base_url to OpenAI client (falls back to SDK default)
- Return None from lookup_model_capabilities for openai-compatible provider
- Only use static capability table for known models in _detect_openai_compat,
  avoiding misleading 200k default for unknown local models
- Add AvailableModelInfo + ListAvailableModelsResponse schemas to both
  console_spec and server_spec
- Regenerate TypeScript SDK OpenAPI snapshots

* fix: apply same known-model guard to Anthropic context_window detection

Only report context_window from the static capability table when the
Anthropic model is actually known, matching the OpenAI path fix.

* ui: add autocomplete hint to Model ID label in admin modal

* fix: use explicit kwargs for OpenAI() to satisfy strict mypy
2026-03-29 03:50:21 -07:00
Patrick Buckley 70c18467cb fix: accumulate signature_delta for Anthropic thinking blocks (#214)
The streaming path captured thinking_delta events but not
signature_delta, leaving the signature empty on round-trip and
causing 400 errors on multi-turn conversations with thinking enabled.
2026-03-29 03:10:49 -07:00
Patrick Buckley 801774bc4a fix: add diagnostic logging for silent tool call drops (#213)
When a local model generates tool calls that are silently dropped
(truncation, missing tool-call-parser), there was zero server-side
logging — making it impossible to diagnose from docker compose logs.

- OpenAI provider: log request params and response summary (debug)
- Session: log stream completion, tool call presence (info), and
  tool call discard with names when truncated (warning)

CLI unaffected — log level is WARNING there.
2026-03-29 02:25:43 -07:00
Patrick Buckley 497984b452 feat: database-backed model definitions with admin UI (#212)
Add model_definitions table (migration 028) enabling model management
via the admin console without SSH access or server restarts. Models
defined in the database coexist with config.toml models through a
per-node merge strategy — config.toml overrides DB for the same alias,
DB-only models coexist alongside, no cross-node contamination.

Storage layer:
- model_definitions table with CRUD (SQLite + PostgreSQL)
- MODEL_DEFINITION_MUTABLE allowlist, admin.models permission

ModelRegistry integration:
- load_model_registry() merges DB + config.toml + CLI models
- context_window=0 auto-detects from provider capability table
or inherits CLI-detected value (same as config.toml behavior)
- ModelRegistry.reload() with validation, TOCTOU-safe accessors
- internal_model_reload + internal_model_status server endpoints

Admin API + UI:
- 6 console endpoints (list, create, get, update, delete, reload)
with admin.models permission, audit trail, provider validation
- Models tab in System group with sky blue (--blue) accent color
- Provider badges (openai/anthropic), source badges (config/db)
- Write-only API keys (never readable, "***" sentinel on update)
- Sync-pending indicator, mobile responsive, focus-trapped modal

Also changes is_secret settings from write-blocked (403) to write-only
across all settings, making judge.api_key configurable via admin UI.
2026-03-29 01:58:08 -07:00
Patrick Buckley bdc1eba34c cleanup: drop vestigial tool_args column from conversations (migration 027) 2026-03-28 23:36:49 -07:00
Patrick Buckley 028c77cae5 fix: display tool errors inline in CLI (#210)
* fix: display tool errors inline in CLI

* fix: thread-safe stderr write with _print_lock and flush
2026-03-28 23:08:24 -07:00
Patrick Buckley 76d007d83f fix: TypeScript SDK DeleteSettingResponse type drift (#209)
* fix: TypeScript SDK DeleteSettingResponse type drift

* fix: export DeleteSettingResponse from SDK index
2026-03-28 23:08:09 -07:00
Patrick Buckley 3f432b8a42 fix: watch dispatch error handler missing stream_end and state cleanup (#208)
* fix: watch dispatch error handler missing stream_end and state cleanup

The watch dispatch run() closure was missing GenerationCancelled
handling, stream_end emission, on_state_change calls, and the
worker_thread identity guard that the send_message path has. This
left the web UI in a stale state when watch-dispatched sends failed.

* fix: address review feedback - on_stream_end, put_nowait, ws._lock, tests

* fix: ruff lint (unused pytest import)

* fix: send_message() use on_stream_end() instead of raw _enqueue
2026-03-28 23:07:47 -07:00
Patrick Buckley f74aa2264e refactor: add is_error to on_tool_result protocol, remove text heuris… (#207)
* refactor: add is_error to on_tool_result protocol, remove text heuristics

Add is_error keyword arg to SessionUI.on_tool_result() so tools
report errors structurally. Server and JS client no longer guess
from output text prefixes — each tool sets the flag at the source.

Bash tool: exit code >= 2 is error, exit code 1 is ambiguous (grep
no-match). History reconstruction keeps text heuristic as fallback
for pre-migration data.

Update SDKs (Python + TypeScript), test mocks, docs, and diagrams.

* fix: infinite recursion in _report_tool_result, signal exits, stale docs

* fix: add _tool_error_flags to test_load_skill ChatSession stubs
2026-03-28 22:09:52 -07:00
Patrick Buckley d00aae2429 fix: tool UX improvements (bash exit codes, previews, edit guard) (#206)
* fix: tool UX improvements (bash exit codes, previews, edit guard)

- Enable pipefail in bash tool so piped commands surface real exit codes
- Move exit code append before UI callback so web UI shows failures
- Remove preview truncation from edit_file, write_file, and math tools
- Add no-op guard to edit_file when old_string == new_string
- Fix collapsed tool output scroll — "click to expand" stays anchored

* fix: correct stale comment on edit_file preview

* fix: suggest re-reading file when edit_file old_string not found
2026-03-28 21:24:23 -07:00
Patrick Buckley 5e09940745 bump version to 0.9.1 2026-03-28 20:32:20 -07:00
renovate[bot] 72bd62d3d8 chore(deps): update dependency katex to v0.16.44 (#204)
* chore(deps): update dependency katex to v0.16.44

* chore: download vendored JS files

---------

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-03-28 20:31:00 -07:00
Patrick Buckley d31f89b2e3 ci: let Renovate rebase over github-actions[bot] commits 2026-03-28 20:30:23 -07:00
Patrick Buckley 9bae8f1a10 ci: auto-download vendored JS files on Renovate PRs
Add vendor-js workflow that triggers on Renovate PRs touching
pyproject.toml — detects version changes, runs update-vendored-js.sh,
and commits the actual files back to the PR branch.  Supports manual
dispatch via pr_number input for one-off runs.
2026-03-28 20:28:06 -07:00
renovate[bot] a012561195 chore(deps): lock file maintenance (#205)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-03-28 20:24:53 -07:00
Patrick Buckley 4198b59a0f fix: eager cancel_ref registration, SDK type drift, force-cancel tests (#203)
Providers now register the SDK stream handle eagerly (before returning
the iterator) instead of lazily inside a generator body. This closes
the window where cancel() couldn't abort a blocked HTTP read because
the stream handle wasn't populated yet.

- OpenAI: yield from → return (HTTP call + cancel_ref happen eagerly)
- Anthropic: split into eager __enter__ + _iter_with_cleanup generator
  with defensive __exit__ on __enter__ failure
- Console SDK: delete_setting() return type fixed from StatusResponse
  to DeleteSettingResponse (matching actual endpoint response)
- 2 new threaded force-cancel integration tests verifying orphaned
  threads don't mutate messages and new generations succeed
- Updated _CancelRef docstring, test robustness (assert on wait)
2026-03-28 20:05:57 -07:00
Patrick Buckley 4f6ef13ce9 fix: cancel button race condition with stream abort and force cancel (#202)
The cancel endpoint emitted a 'cancelled' SSE event before the worker
thread terminated. The frontend transitioned to "send" mode prematurely,
so the next send got rejected with "Already processing a request."

Backend:
- Providers expose SDK stream handle via cancel_ref parameter so
  cancel() can close the HTTP connection and unblock iteration
- Generation counter prevents orphaned threads from mutating messages
  or clearing cancel state after force cancel
- _check_cancelled() added between retry attempts in _try_stream
- Server polls (async, non-blocking) for cancelled worker to exit
- Force cancel (force:true) abandons stuck worker, keeps cancel event
  set so subprocesses are killed, guards against spurious SSE events

Frontend:
- 'cancelled' shows "Cancelling..." then escalates to "Force Stop"
  after 2s for a harder cancel that abandons the worker immediately
- 10s safety timeout auto-recovers if stream_end never arrives
- busy_error re-enables stop button instead of showing send
- Timeout cleanup in disconnectSSE, stream_end, and force .then()
- Layout shift prevention (min-width, white-space: nowrap)
- aria-label updates for accessibility

Tests:
- 7 new tests: stream close, error suppression, cancel_ref population,
  transport error conversion, non-cancel exception propagation, retry
  cancellation check
2026-03-28 19:26:37 -07:00
Patrick Buckley 52716ed611 feat: detect repeated tool calls and nudge model to try different approach (#201)
When a model calls the same tool with identical arguments as a previous
call, append a warning to the tool result and inject a metacognitive
nudge. This breaks loops where small local models get stuck repeating
the same action (e.g. running the same bash command 3+ times).

The repeat signature set is cleared after a warning fires, giving the
model a clean slate. Also cleared on conversation compaction.

Ref: #186
2026-03-28 16:18:12 -07:00
Patrick Buckley 6c9a7d7351 fix: harden tool call handling for local model servers (#200)
* fix: harden tool call handling for local model servers

Local models (Qwen 3.5 9B, etc.) via llama.cpp produce tool calls with
empty IDs, whitespace-padded names, and malformed JSON arguments. These
defensive gaps caused cascading conversation corruption and silent
failures.

- Strip whitespace from tool names in both main and agent paths
- Generate synthetic UUIDs when tool call IDs are empty/null
- Surface malformed tool call errors to the user via on_error
- Give the model actionable hints (expected JSON format, available tools)
  so it can self-correct on retry
- Surface metacognition nudge types to UI via on_info

Ref: #186, #117

* refactor: extract _ensure_tool_call_ids helper, include MCP tools in error

Address Copilot review feedback on PR #200:
- Extract duplicated ID fixup into _ensure_tool_call_ids() static method
- Tests now exercise the actual helper instead of reimplementing the logic
- Unknown tool error now includes MCP tool names alongside builtins
2026-03-28 15:48:11 -07:00
Patrick Buckley 3518f7953c fix: prevent 100% CPU spin from unreachable HTTP MCP servers (#199)
When an HTTP MCP server is unreachable and TCP connect fails immediately
(ECONNREFUSED, DNS failure), the anyio task group inside
streamablehttp_client produces a CancelledError that escapes
asyncio.wait_for and leaves orphaned cancel-scope tasks in an infinite
_deliver_cancellation loop (~800K callbacks/sec).

Three-part fix:
- TCP pre-flight probe (5s timeout) before entering the anyio transport
  context — fails fast on unreachable servers, avoiding the bug entirely
- Catch CancelledError in _connect_one with current_task().cancelling()
  check to distinguish stray anyio cancels from real shutdown
- _safe_close_stack helper with bounded timeout that never raises,
  preventing cleanup errors from masking the original exception
2026-03-28 15:19:04 -07:00
Patrick Buckley 48769e5a97 fix: gate read_resource and use_prompt tools on MCP server availability (#198)
These built-in tools were always sent to the LLM even when no MCP
servers were connected, wasting model turns on calls that would always
return errors. Now gated per-request in _get_active_tools() — same
pattern as the existing web_search gating — using resource_count and
prompt_count for granular filtering.
2026-03-28 14:38:53 -07:00
Patrick Buckley adb4ff6399 fix: resolve pre-existing test failures, stale type ignores, and warnings (#197)
- TLS: suppress no-any-return + unused-ignore on lacme mtls helpers
  (lacme stubs typed locally but not in CI)
- TLS: catch duplicate Prometheus metric registration specifically
  (not blanket ValueError) so real setup errors still propagate
- Discord: add missing storage=None to _make_bot mock (policy evaluation
  path accesses self.storage)
- Web search: patch _ddg_available in gating test so ddgs availability
  doesn't mask the Tavily-only test path
- Bridge stress: close real httpx client before replacing with mock so
  daemon threads don't make real HTTP calls or leak connections
- README: comment out tavily_key placeholder in example config
2026-03-28 14:24:53 -07:00
Patrick Buckley 131a1ec943 fix: stream tool errors in real-time with visual error indicator (#196)
* fix: stream tool errors in real-time with visual error indicator

Tool executors that hit errors (file not found, timeout, write failure,
etc.) were returning error strings without calling ui.on_tool_result(),
so no SSE event was emitted to the browser during streaming. Errors only
appeared after page refresh via history rebuild. Now all error paths
call on_tool_result() so errors stream in real-time.

Added visual error state: tool blocks with errors get a red left border,
red "✗ error" badge, and red error text — matching the existing denied
state pattern but distinct from it. Error detection uses prefix matching
("Error", "Command timed out", "Search timed out") both server-side
(is_error flag on SSE event + _build_history) and client-side (regex
fallback).

Affected tools: bash, read_file, write_file, edit_file, search, math,
man, memory, web_fetch, web_search, plus the run_one catch-all and
prepare-error paths.

* review: expand error prefix detection per copilot feedback

Add Unknown tool, JSON parse error, MCP prompt timed out, and MCP
prompt error to the is_error prefix list in on_tool_result, _build_history,
and the frontend regex. These error messages were confirmed in the
codebase but missing from the detection heuristic.
2026-03-28 00:21:11 -07:00
Patrick Buckley 1cded9b430 chore: bump version to 0.9.0 2026-03-27 21:24:49 -07:00
Patrick Buckley 62c741eb8a fix: prevent assistant messages with content=None from reaching OpenAI API (#195)
Replayed or cancelled conversations could produce assistant messages
with content=None and no tool_calls, which OpenAI-compatible APIs
reject with a 400.  Fix at three layers for defense in depth:

- session.py: use empty string instead of None when building assistant
  messages (streaming + cancellation paths)
- _utils.py: normalise content on DB load in reconstruct_messages()
- _openai.py: add _sanitize_messages() catch-all at provider boundary

Closes #194
2026-03-27 21:22:39 -07:00
Patrick Buckley 3362917e1e chore(deps): update vendored KaTeX 0.16.42 → 0.16.43 (#193) 2026-03-27 10:54:21 -07:00
renovate[bot] 698cbbf988 chore(deps): lock file maintenance (#192)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-03-26 20:55:46 -07:00
renovate[bot] e47a08b7bc chore(deps): update ghcr.io/astral-sh/uv docker tag to v0.11.2 (#191)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-03-26 20:55:00 -07:00
renovate[bot] aaa427debd chore(deps): update dependency vitest to v4.1.2 (#190)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-03-26 20:54:58 -07:00
renovate[bot] 611af76971 chore(deps): pin dependencies (#188)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-03-26 20:54:49 -07:00
Patrick Buckley bbe28ecab3 fix: cancel LLM judge daemon when user approves/denies tools (#187)
When the LLM judge is enabled and the user makes rapid approval
decisions, judge daemon threads pile up competing for the inference
server, causing timeouts. Pass a threading.Event from session to the
judge — set on approval — so the daemon abandons remaining work
within ~1s, including shutting down the executor to kill in-flight
API calls.
2026-03-26 18:10:23 -07:00
Patrick Buckley 93a9fd3c28 bump: v0.8.9 — mTLS + ACME integration via lacme 2026-03-26 14:33:11 -07:00
Patrick Buckley 62ff3217d0 fix: TLS Docker end-to-end testing fixes (#185)
* fix: TLS Docker end-to-end testing fixes

Fixes discovered during Docker Compose TLS integration testing:

- Dockerfile: use --extra all (prevents missing optional deps)
- lacme 1.0.3: fixes CACertificateIssued event logging crash
- chmod PermissionError: guard for Docker volume mounts
- socket import: moved to top of main() (was inside TLS conditional,
  caused NameError in _default_node_id)
- redis.SSLConnection: ConnectionPool needs explicit connection_class,
  not ssl=True (which only works on Redis() directly)
- Empty redis password: pass None instead of "" to avoid AUTH error
- TURNSTONE_CONSOLE_URL: env var for Docker service discovery
  (0.0.0.0 bind address isn't reachable from other containers)
- HTTP01Handler: ACME client needs a challenge handler even when
  server auto-approves
- Docker overlay: tls-init as root with chmod, Redis conditional
  password, console Redis TLS flags, TURNSTONE_CONSOLE_URL

* feat: full mTLS end-to-end with lacme 1.0.4

Completes the mTLS chain across all services:

lacme 1.0.4:
- Dual EKU certs (serverAuth + clientAuth) — fixes mTLS rejection
- Configurable CA name (name="turnstone") — consistent store key

Bootstrap CA import:
- Console imports bootstrap CA from /certs volume on first boot
- Single trust root: bootstrap CA → console → all service certs

Bridge mTLS:
- TLSClient init when TURNSTONE_TLS_ENABLED set
- Auto-upgrades server URL from http:// to https://
- SSLContext passed to all 3 httpx clients via verify=

Console collector mTLS:
- upgrade_tls() method replaces httpx client with mTLS context
- Called in lifespan after cert issuance alongside proxy upgrade
- Fixes "Failed to poll node" when server serves HTTPS

Docker overlay:
- TURNSTONE_TLS_SANS on all services (Docker service names as SANs)
- TURNSTONE_TLS_ENABLED on bridge
- Channel service with Redis TLS flags
- TURNSTONE_CONSOLE_URL for service discovery
- Server healthcheck disabled (mTLS healthcheck deferred)
- Redis conditional password from env

Verified end-to-end: bootstrap → console CA → server HTTPS →
bridge mTLS → Redis TLS → channel Redis TLS → console collector
polls server over mTLS → workstream creation works through bridge

* fix: lint + copilot feedback on TLS Docker e2e

- SIM105: contextlib.suppress(PermissionError) for chmod
- F401: remove unused get_storage import in bridge
- Redis healthcheck: pass password when REDIS_PASSWORD is set

* fix: sort imports in admin.py and bridge.py

* fix: tls-init key permissions, healthcheck env, collector race

- tls-init: add set -e, chown to turnstone:turnstone with restrictive
  perms (keys 0600, certs 0640, dirs 0750) instead of world-readable
- Redis healthcheck: use container runtime $$REDIS_PASSWORD instead of
  Compose-time interpolation for consistency with --requirepass block
- collector upgrade_tls(): don't close old httpx client while concurrent
  poll threads may still be using it — let GC handle cleanup
2026-03-26 14:24:44 -07:00
Patrick Buckley b086390558 fix: TLS deferred work — wiring, security, tests, Docker overlay (#184)
* fix: TLS deferred work — wiring, security, tests, Docker overlay

Security fixes:
- PostgreSQL SSL: validate sslmode against known values, urlencode
  all params to prevent URL injection
- ConfigStore env seeding: removed redundant type coercion, delegate
  to validate_value() which handles all coercion correctly

Functional wiring:
- Database SSL: init_storage() passes SSL params to PostgreSQL URL
- Server: env var fallbacks for DB SSL (TURNSTONE_DB_SSLMODE etc.)
- Proxy mTLS: re-create proxy clients after TLS cert issuance
- Channel gateway: --ssl-certfile/keyfile/ca-certs CLI args, HTTPS
  advertise URL when SSL configured
- ConfigStore env seeding: TURNSTONE_{SECTION}_{KEY} seeds on first boot
- Console deregistration on shutdown (with debug logging)

Specs, tests, Docker:
- OpenAPI: 5 TLS admin endpoints in console_spec.py
- Auth enforcement test (401 without auth)
- SDK ValueError test (mismatched cert/key)
- Docker overlay: TURNSTONE_TLS_ENABLED, bridge --redis-tls, Redis
  healthcheck with client cert
- Removed stale type:ignore comments (lacme 1.0.2 type stubs)

* review: address copilot feedback on TLS deferred work

- ConfigStore env seeding: use config_store.set() instead of
  storage.set_system_setting() (correct API, updates cache)
- Remove unused defn variable (iterate SETTINGS keys only)
- Fix structlog call-arg error (positional args, not kwargs)
- Channel gateway: validate cert+key provided together
- Restore type:ignore[no-any-return] for CI mypy (lacme 1.0.2
  type stubs not in CI's mypy overrides yet)

* fix: rename _VALID_SSLMODES to lowercase (N806)
2026-03-25 22:43:35 -07:00
Patrick Buckley d08a57dfc2 feat: SDK TLS support, Docker Compose overlay, TLS docs (#183)
Python SDK:
- ca_cert, client_cert, client_key on all 4 client classes
- ValueError if only one of client_cert/client_key provided
- Passed to httpx verify=/cert=

TypeScript SDK:
- TlsOptions type exported (zero runtime code)
- Fix picomatch vulnerability (npm audit fix)

Docker Compose:
- deploy/docker-compose.tls.yml overlay with tls-init bootstrap
- Notes it's an overlay requiring a base compose file

Documentation:
- docs/tls.md: architecture, config, CLI, SDK examples, troubleshooting
- Fixed package name (@turnstone/sdk), Node.js 18+ note
2026-03-25 20:43:34 -07:00
Patrick Buckley 9fdf51ff3d feat: TLS admin UI, CLI cert management, Redis/PG TLS (#182)
Admin API (require admin.settings):
- GET /v1/api/admin/tls/certs, POST .../renew, DELETE .../certs/{domain}
- renew_cert() updates in-memory bundles immediately

Admin UI (instrument panel grid pattern):
- TLS tab with CA status bar, cert grid, renew/delete actions
- Uses admin-row/admin-colheaders grid system (consistent with 12+ tabs)
- showConfirmModal for destructive actions, aria-labels on buttons
- Expired certs show "EXPIRED" text prefix + red color (WCAG 1.4.1)
- Loading state, empty state, error state

CLI (turnstone-admin):
- tls-bootstrap: offline CA + cert issuance (dir perms 0700)
- tls-issue: ACME cert request with key perms 0600
- tls-ca-cert: SHA-256 fingerprint for TOFU verification
- tls-list: auth via --auth-token or config token

Redis/PG TLS:
- RedisBroker + AsyncRedisBroker: ssl params wired through
- broker_from_args + async_broker_from_args: forward TLS kwargs
- add_redis_args: --redis-tls CLI flags
- Config map: [redis] and [database] TLS passthrough
2026-03-25 18:55:11 -07:00
Patrick Buckley 45471894da refactor: adopt lacme 1.0.2 — eliminate loopback client + temp file boilerplate (#181)
lacme 1.0.2 ships four features that simplify turnstone's TLS code:

- RenewalManager CA-direct mode: console renewal now uses ca= param
  instead of a loopback ACME client. No network, no startup ordering
  dependency, no client lifecycle management.
- ACMEResponder serves /ca.pem natively: removed custom route handler
  and route ordering workaround.
- write_pem_files_persistent: replaced manual temp file creation,
  chmod, and atexit cleanup with lacme's secure PEM file helper.
- Removed port param from TLSManager (was only for loopback URL).

Net: ~50 lines removed, two tech debt items resolved.
2026-03-25 18:02:26 -07:00
Patrick Buckley c0b5952573 feat: mTLS service clients with ACME auto-provisioning (#180)
TLSClient class for service nodes — discovers console via services
table, fetches CA cert, requests cert via ACME, provides SSL contexts:

- Console self-registers in services table for discovery
- Services auto-discover console URL from DB (no extra config)
- Initial cert request over plain HTTP (ACME provides integrity)
- Auto-renewal via RenewalManager in server lifespan
- Unauthenticated /acme/ca.pem endpoint for node bootstrapping

RenewalManager fix (was passing client=None):
- Console creates loopback ACME client for self-renewal
- Proper async lifecycle (aenter/aexit) with clean shutdown

Integration points wired:
- Server: TLS init before uvicorn, temp PEM files (0o600, atexit
  cleanup), auto-renewal in lifespan
- Bridge: tls_verify + tls_cert params on all 3 httpx clients
- Console collector: tls_verify + tls_cert params on httpx client
- Console proxy: mTLS context from TLSManager on proxy clients
- Channel gateway: optional SSL params on uvicorn.Config
- Console main(): reads tls.enabled, creates TLSManager, passes
  to create_app with console_url

6 tests for TLSClient (discovery, defaults, backward compat)
2026-03-25 17:48:34 -07:00
Patrick Buckley b9d5b5b671 feat: console CA + ACME server via lacme (#179)
TLSManager class owns the internal CA, ACME responder, and cert
lifecycle:
- CertificateAuthority with DB-backed storage (StorageStore adapter)
- ACMEResponder mounted at /acme (auto_approve for internal network)
- Dual cert issuance: internal CA for mTLS, optional external ACME CA
  for frontend HTTPS (Let's Encrypt via acme_directory setting)
- Auto-renewal via RenewalManager with clean async shutdown
- Expired cert detection on reload (re-issues instead of loading stale)
- EventDispatcher wired to structlog + lacme Prometheus metrics
- GET /v1/api/admin/tls/ca.pem — root cert download
- GET /v1/api/admin/tls/ca — CA status + cert inventory
- Console lifespan: init CA, issue certs, start renewal, stop on shutdown
- 11 async tests covering CA lifecycle, cert persistence, SSL contexts,
  endpoints, and event wiring
2026-03-25 16:35:55 -07:00
Patrick Buckley 274c97135e feat: TLS storage backend + config for lacme integration (#178)
Storage layer for mTLS certificate management via lacme:

- Migration 026: tls_account_keys, tls_ca, tls_certificates tables
- 8 protocol methods on StorageBackend (save/load account keys, CA,
  certs; list/delete certs)
- SQLite and PostgreSQL implementations with dialect-specific upserts
- StorageStore adapter bridging lacme's Store protocol to turnstone
  storage (bytes↔str PEM conversion, CertBundle↔dict mapping)
- Settings registry: tls.enabled (bool), tls.acme_directory (string)
- Config.toml: [redis] TLS and [database] SSL passthrough params
- lacme>=1.0.1 as optional [tls] dependency
- 20 unit tests covering storage CRUD + adapter + crypto roundtrip
2026-03-25 16:07:05 -07:00
Patrick Buckley e87f8e19c2 feat: adopt eval-optimized system prompt for plan_agent pattern
Update plan_agent tool pattern to show codebase exploration before
delegating to the planning agent. This pattern achieved 100% pass
rate (160/160 runs) on the eval suite — up from 98% on the previous
prompt.
2026-03-25 13:32:01 -07:00
Patrick Buckley bb221f4dab chore: bump v0.8.8, update vendored katex 0.16.40 → 0.16.42 2026-03-25 12:00:26 -07:00
Patrick Buckley 361876b17a feat: eval pipeline improvements + tool description optimization (#174)
Eval pipeline:
- --optimize-tools mode freezes system prompt, optimizes tool descriptions only
- Analyst sees available tool list (prevents hallucinated "tool not in schema")
- Analyst sees current tool descriptions in --optimize-tools mode
- Three-layer timeout defense: httpx timeout + _cancelled event + client.close()
- Filter MCP-only tools (read_resource, use_prompt) from headless eval
- Pattern-over-rules framing in optimizer, analyst, and observer prompts
- Tool description diffs logged after each iteration
- Tool optimizer failure retries instead of stopping the loop

Tool renames (avoid chat template channel collision on local models):
- create_plan -> plan_agent
- task -> task_agent

Tool descriptions (from eval-driven optimization, 79% -> 98%):
- bash: environment question examples, disambiguation from write_file/man
- edit_file: multi-file workflow, prerequisite clarification, docstring example
- man: "questions about flags are tool-use tasks" prefix
- math: simple example up front
- plan_agent: "delegate to sub-agent" framing, negative boundary for direct edits
- read_file: multi-file workflow hint
- search: trigger phrases, prerequisite clarification, disambiguation
- write_file: immediate action framing, placeholder example

System prompt: enriched tool patterns from eval results, added identity opener

Test suite: plan-before-refactor -> plan-when-asked (simplified)

Docs: comprehensive eval.md rewrite covering all current features
2026-03-25 11:58:10 -07:00
renovate[bot] 5d26cd6593 chore(deps): update dependency katex to v0.16.42 (#175)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-03-25 11:57:59 -07:00
renovate[bot] 2d4420e00d chore(deps): lock file maintenance (#177)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-03-25 11:57:47 -07:00
renovate[bot] b20548583d chore(deps): update ghcr.io/astral-sh/uv docker tag to v0.11.1 (#176)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-03-25 11:57:44 -07:00
Patrick Buckley f63b2915cc review: address copilot feedback on user_id trust check
Remove console-proxy from trusted_sources — end-user tokens via the
console proxy already carry the real user_id in the JWT, so they must
not be able to override it via the request body (impersonation risk).
Only bridge and console service identities are trusted to forward
user_id on behalf of users. Add 5 tests covering the trust boundary.
2026-03-24 03:13:23 -07:00
Patrick Buckley bf85bbea94 fix: resolve user_id to username in usage and audit displays
Usage tab grouped by user showed raw hex user_id instead of username.
Audit tab showed truncated hex. Now both API endpoints resolve user_id
to username via list_users() lookup before returning the response.
2026-03-24 03:13:23 -07:00
Patrick Buckley 803d8ee8f9 docs: document user_id propagation through MQ path
Update security.md with trusted service user_id forwarding. Update
MQ protocol diagram to include user_id field on CreateWorkstreamMessage.
Update console data flow diagram to show user_id in message and bridge
forwarding.
2026-03-24 03:13:23 -07:00
Patrick Buckley 17b5961a70 fix: propagate user_id through MQ workstream creation path
Console create_workstream was constructing CreateWorkstreamMessage
without setting user_id, so workstreams created via console→MQ→bridge
→server had empty user_id in usage events. Now the console extracts
user_id from auth_result and passes it through the MQ message. The
bridge forwards it in the HTTP payload, and the server accepts it
from trusted service callers (bridge, console-proxy).
2026-03-24 03:13:23 -07:00
Patrick Buckley 037308f3b1 fix: propagate user identity through console proxy
Console proxy previously used a fixed service identity (console-proxy)
with full {read,write,approve} scopes for all proxied requests, losing
the real user's identity at the proxy boundary. Now mints per-request
short-lived JWTs carrying the authenticated user's actual user_id,
scopes, and permissions so upstream servers record correct audit
attribution and enforce scope narrowing as defense in depth.
2026-03-24 03:13:23 -07:00
Patrick Buckley d7a9895855 feat: tool description optimization + OOM and logging fixes (#172)
* feat: tool description optimization + OOM and logging fixes

Tool description optimization (three-phase pipeline):
- Tool optimizer (phase 2) modifies tool descriptions to resolve
  confusion, gated by --optimize-tools and wrong_tool detection
- _apply_tool_overrides deep-copies modified tools, never mutates TOOLS
- _propose_tool_overrides validates JSON, deep-merges parameter overrides
- tool_overrides on EvolutionNode, plumbed through full eval pipeline
- --save-tools writes best overrides back to turnstone/tools/*.json
- Prompt optimizer informed when tool descriptions have been modified
- TSV tool_changes column

OOM fix (session lifecycle):
- HeadlessSession created inside retry loop, not outside — timed-out
  orphan threads no longer pin old sessions in memory
- Session ref cleared immediately after extracting results
- Previous behavior leaked unbounded memory per timeout (~515GB OOM)

Logging fix:
- Removed _suppress_stdout entirely — redirected fd 1 process-wide,
  causing main thread print() to vanish during slow API calls
- NullUI already discards session output; tools return strings

New CLI: --optimize-tools, --tool-optimizer-model/base-url, --save-tools

* review: address copilot feedback on tool optimization
2026-03-23 22:47:06 -07:00
Patrick Buckley 0ba49b8bb7 review: address copilot feedback on eval pipeline
- Record prompt_variant in parallel execution path (was serial-only)
- Handle "Subprocess error:" and "Skipped (fast-fail)" in failure
  classifier instead of mis-bucketing as missing_tool
- Build case_id->case_def dict once in _build_failure_analysis,
  _run_analyst, _propose_prompt_modification (was O(n²) linear scan)
- Enforce original prompt at slot 0 of cached user_prompts variants
- Use wall clock time for TSV elapsed_s instead of sum of run times
2026-03-23 19:44:32 -07:00
Patrick Buckley 51b5b3ee74 feat: multi-agent eval pipeline with tree search, analyst, diversifier
UCB tree search for prompt optimization (arXiv:2603.18620):
- EvolutionNode dataclass, UCB1 selection, rolling mean scores
- Replaces fragile linear chain with backtracking via tree
- Holdout set separation prevents optimizer overfitting
- Improvement-based delta feedback to optimizer

Multi-agent optimization pipeline:
- Analyst agent (phase 1): multi-turn with math/bash tools, identifies
  semantic failure patterns, computes statistics across test results
- Optimizer (phase 2): uses analyst diagnosis to edit developer prompt
- Observer: tunes optimizer strategy every 3 iterations
- Diversifier: generates paraphrased prompt variants for phrasing
  robustness, with dedup, delta generation, and JSON caching

Failure classification:
- 8 failure mode buckets (no_tool_call, wrong_tool, missing_tool,
  wrong_args, extra_tools, timeout, error, json_dump)
- Consistency signals (systematic, flaky, marginal)
- Rule-based pre-analysis feeds into analyst as structured input

Logging and observability:
- Config summary at startup (models, case count, runs)
- Per-case diversifier progress with dedup stats
- UCB selection reasoning, node score updates, tree growth
- Extended TSV: node_score, elapsed_s, prompt_len, iter_tokens,
  cumul_tokens columns plus 4-decimal precision

Infrastructure:
- Thread-safe fd-level stdout suppression (os.dup2)
- Prompt variants plumbed through parallel execution path
- Cached variants auto-detected from tests.json user_prompts field

New CLI flags: --explore-constant, --analyst-model/base-url,
--diversifier-model/base-url, --diversify N, --save-variants
2026-03-23 19:44:32 -07:00
Patrick Buckley c3b0ddeba7 fix: add ddgs to mypy ignore_missing_imports for CI compat 2026-03-23 19:11:50 -07:00
Patrick Buckley a533e1c783 fix: use fd-level stdout redirect in eval to avoid thread race
_suppress_stdout() was setting sys.stdout = StringIO() which is
process-global. When send_headless runs in a ThreadPoolExecutor and
blocks on an API call inside the suppression context, the main
thread's print() calls silently go to the StringIO and vanish.

Switch to os.dup2 fd-level redirect which is thread-safe.
2026-03-23 18:15:45 -07:00
Patrick Buckley d8bc78556f bump: v0.8.7 2026-03-23 17:27:23 -07:00
Patrick Buckley 4f5854e768 fix: remove stale type: ignore on ddgs import 2026-03-23 17:10:27 -07:00
Patrick Buckley bf2dc04cb3 feat: UCB tree search for eval prompt optimization
Replace linear optimization chain with UCB1 evolution tree. Each
iteration selects the most promising node to extend, preventing
irrecoverable collapse from bad edits. Also adds improvement-based
delta feedback to the optimizer and optional holdout set separation.

Inspired by "Learning to Self-Evolve" (arXiv:2603.18620).
2026-03-23 17:09:22 -07:00
Patrick Buckley bb894d073b fix: SQLite migrations use batch_alter_table for compat
Migrations 014, 021-024 used bare op.add_column/alter_column/drop_column
which fails on SQLite (no ALTER of constraints). Switch to
batch_alter_table and enable render_as_batch in env.py. Migration 014
also moved UniqueConstraint inline into create_table.
2026-03-23 17:09:13 -07:00
Patrick Buckley b3934a2d14 bump: v0.8.6
Hotfix: DDG web search returning empty results.

- Switch dependency from duckduckgo-search (deprecated shim, empty
  results) to ddgs>=9.0 (actively maintained successor)
- Include ddg extra in Docker image for free web search fallback
- Update all user-facing install instructions to reference ddgs
2026-03-23 16:17:28 -07:00
Patrick Buckley 3d02cf66b4 review: update stale duckduckgo-search references to ddgs 2026-03-23 16:09:03 -07:00
Patrick Buckley 7631b88792 fix: switch DDG dependency from duckduckgo-search to ddgs
duckduckgo-search 8.x is a deprecated shim that returns empty results
(upstream temporarily disabled HTML/Lite backends, Bing backend broken).
The package was renamed to ddgs in v9.x which works correctly.
2026-03-23 16:09:03 -07:00
Patrick Buckley a07172b0c0 fix: include ddg extra in Docker image for free web search fallback 2026-03-23 15:23:12 -07:00
Patrick Buckley f3d33bf44a bump: v0.8.5
Features:
- Pluggable web search backends — DDG as free default, Tavily, MCP (#166)
- --config flag and $TURNSTONE_CONFIG env var (#160)
- Live session config via ConfigStore point-of-use reads (#154)
- PostgreSQL CI integration tests (#156)
- Skill priority ordering (#144)
- Raise scaling limits for 1000-node clusters (#129)

Security:
- Output guard wired into agent loops — plan + task agents (#168)
- Tool policy enforcement in CLI, bridge, and channel (#168)
- Subprocess environment scrubbing — API keys stripped (#168)
- OIDC issuer SSRF validation (#140)
- MCP registry URL scheme validation (#133)
- Output guard enabled in CLI mode (#134)

Reliability:
- Bridge approval/plan review TOCTOU races fixed (#167)
- SQLite WAL mode, eviction cancel, title retry (#151)
- Health monitor OPEN → HALF_OPEN autonomous probe (#152)
- ConfigStore spec alignment (#153)
- Critical production readiness fixes (#147)
- Server startup stampede prevention (#132)
- Python 3.14 CancelledError guard (#146)

Performance:
- Conversations index, batch config saves, capabilities cache (#149)

Quality:
- Bridge stress tests (6 scenarios, 100 iterations each) (#157)
- Governance SDK, MCP reload, skill config integration tests
- Structlog standardization across 19 modules (#150)
- Dead code removal (#148)
2026-03-23 15:00:08 -07:00
Patrick Buckley fdb1a189e8 fix: show policy deny reason in CLI approval output
Denied tools now print the error text (e.g. "Blocked by tool policy")
in red below the header, so the user sees why a tool was blocked.
2026-03-23 14:55:14 -07:00
Patrick Buckley 58e2d9348f review: fix mypy, tighten env scrub, bridge storage safety
Address Copilot + code review feedback:
- Fix mypy: rename tool_names → _policy_names in CLI to avoid type clash
- Tighten env scrub from substring to suffix matching (_KEY, _TOKEN, etc.)
  to avoid false positives on MONKEYTYPE, KEYBOARD_LAYOUT
- Add DATABASE_URL/TURNSTONE_DB_URL to explicit scrub list
- Bridge: use _storage directly instead of get_storage() which
  auto-initializes a local SQLite DB with no admin policies
- Discord bot: add self.storage None guard
- Add tests for suffix-only matching and false positive avoidance
2026-03-23 14:55:14 -07:00
Patrick Buckley 771d03b8e6 review: fix LESS prefix leak, move policy before auto-approve, add tests
- Move LESS/LESSOPEN/LESSCLOSE/LESSPIPE/LESSCHARSET to _SAFE_NAMES
  instead of prefix matching (prevents LESS_SECRET_TOKEN leak)
- Move bridge policy evaluation before auto-approve check so deny
  policies override auto-approve
- Add storage None guard in Discord bot
- Clean up _policy_handled pattern in Discord bot
- Add debug logging on policy evaluation exceptions
- Add tests: extra overrides scrub, LESS prefix safety
2026-03-23 14:55:14 -07:00
Patrick Buckley d147aaea36 security: scrub secrets from subprocess environments
New turnstone/core/env.py provides scrubbed_env() that strips API keys,
tokens, passwords, and credentials from os.environ before passing to
subprocesses. Applied to all 5 subprocess call sites: _exec_bash,
_exec_search, _exec_man, watch _run_command, and MCP stdio servers.

Pattern-based scrubbing (KEY, SECRET, TOKEN, PASSWORD, CREDENTIAL
substrings) plus explicit blocklist for known secrets. Safe vars
(PATH, HOME, locale, etc.) always preserved. Passthrough list for
operator overrides.
2026-03-23 14:55:14 -07:00
Patrick Buckley 8b747178e0 security: enforce tool policies in CLI, bridge, and channel
evaluate_tool_policies_batch() was only called in the server WebUI.
CLI, bridge, and channel entry points now evaluate admin-defined tool
policies before auto-approve checks. Deny policies block tools, allow
policies auto-approve. Best-effort: gracefully skipped if storage is
unavailable.
2026-03-23 14:55:14 -07:00
Patrick Buckley d57280d807 security: wire output guard into agent loops
_run_agent (plan + task agents) now passes tool results through
_evaluate_output() before appending to context — same as the main
session loop. Runs before truncation so the guard sees full output.
Catches prompt injection, credential leakage, and encoded payloads
in agent tool results that were previously unscanned.
2026-03-23 14:55:14 -07:00
Patrick Buckley b9870f279c fix: bridge approval & plan review TOCTOU races (#158, #159) (#167)
* fix: bridge approval & plan review TOCTOU races (#158, #159)

Replace "pop on completion" with a tombstone pattern — pending entries
are marked resolved=True instead of being removed, eliminating the
window where stale SSE reconnect events bypass the duplicate guard.
Resolved tombstones are cleaned up on ws_state events and ws_closed.

Stress tests now pass reliably (previously ~12-16% failure rate).

* review: extract _mark_resolved helper, use real ws_closed path in test, add refinement loop test

Address code review suggestions:
- Extract _mark_resolved() helper in _wait_plan to reduce duplication
- Document cross-stream ordering assumption for plan review refinement
- Race 6 test now calls _handle_global_event instead of manual dict pops
- New Race 7 test validates plan review refinement loop (tombstone → cleanup → re-entry)

* fix: add TTL fallback for tombstone cleanup when global SSE lags

If the global SSE stream is temporarily down while per-WS SSE continues,
resolved tombstones would block legitimate new approvals/plan reviews.
Add a 30s TTL so stale tombstones are expired in the duplicate guard
as a fallback to the normal ws_state-based cleanup.

Also changes tombstone type from (request_id, bool) to
(request_id, float) where 0.0 = active, >0 = resolved_at monotonic time.

* fix: use 3x approval_timeout for tombstone TTL instead of hardcoded 30s

Tie the TTL to the configurable approval_timeout (default 300s = 900s TTL)
rather than a short hardcoded value. The TTL is only a fallback for when
the global SSE stream is completely down — a conservative value is safer.
2026-03-23 14:04:12 -07:00
Patrick Buckley 71ee340bc6 feat: pluggable web search backends (DDG, Tavily, MCP) (#166)
* feat: pluggable web search backends (DDG, Tavily, MCP)

web_search is now an abstract capability with swappable backends:

- DuckDuckGoClient — free, no API key, uses duckduckgo-search library
- TavilyClient — existing behavior, requires API key
- MCPSearchClient — delegates to any MCP server tool

New tools.web_search_backend setting (ConfigStore + --web-search-backend
CLI flag): '' (auto), 'tavily', 'ddg', or 'mcp:server:tool'.

Auto-detection (default): Tavily if key present, else DDG if installed,
else disabled. This means users with duckduckgo-search installed get
web search for free with local models — no API key needed.

Closes #131

* fix: address Copilot review on pluggable web search

- Unknown backend values now log warning + return None (not silent
  fallthrough to auto-detect)
- Pass timeout to DDGS constructor
- MCP: use math.ceil for timeout, forward topic kwarg
- Fix agent mode web_search gating to use _resolve_search_client()
  instead of get_tavily_key() (was still using old check)
- Update docstring to reflect new backend resolution
- Fix DDG test to patch DDGS import properly
- Add test for unknown backend rejection
2026-03-23 13:21:04 -07:00
Patrick Buckley c5d5d0b7cd fix: update-vendored-js.sh detects old version from filesystem
The script detected the old version from pyproject.toml, which Renovate
had already updated. This caused OLD_DIR == NEW_DIR, so the script
downloaded files then immediately deleted them.

Fix: detect old version from the actual directory on disk. Add a guard
that errors if old == new version to prevent silent data loss.

Also: run the fixed script to vendor katex 0.16.40 (fonts + css + js).
2026-03-23 13:03:49 -07:00
renovate[bot] 1f9d03c3e0 chore(deps): update dependency katex to v0.16.40 2026-03-23 13:03:49 -07:00
renovate[bot] 24e082df05 chore(deps): update postgres docker tag to v18 2026-03-23 12:55:27 -07:00
renovate[bot] 4a78d20eea chore(deps): update dependency typescript to v6 2026-03-23 12:55:18 -07:00
renovate[bot] e0d17e0f99 chore(deps): update ghcr.io/astral-sh/uv docker tag to v0.10.12 2026-03-23 12:55:08 -07:00
renovate[bot] cd6c49dd01 chore(deps): update dependency vitest to v4.1.1 (#162)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-03-23 19:53:13 +00:00
Patrick Buckley 8454e961ba feat: --config flag and $TURNSTONE_CONFIG env var (#160)
* feat: --config flag and $TURNSTONE_CONFIG env var for config.toml path

Add set_config_path() to config.py with three-tier resolution:
  1. --config CLI flag (via set_config_path)
  2. $TURNSTONE_CONFIG environment variable
  3. ~/.config/turnstone/config.toml (default)

--config added to all 5 entry points that load config.toml: CLI,
server, console, bridge, eval. Uses parse_known_args pre-parse so
the path is resolved before apply_config reads the file.

Closes #130

* fix: centralize --config pre-parse, fix help and docstrings

- Add add_config_arg() helper with separate pre-parser (add_help=False)
  so --help still shows config-derived defaults
- Replace duplicated pre-parse blocks in all 5 entry points
- Fix set_config_path docstring (works after load_config too)
- Fix module docstring precedence description
- Remove redundant import os in get_tavily_key
2026-03-23 12:26:41 -07:00
Patrick Buckley 4ae38bc2ae test: bridge race condition stress tests (#157)
* test: bridge race condition stress tests (5 scenarios)

Repetition-based stress harness (100 iterations per scenario) targeting
threading races in bridge.py:

1. Duplicate approval on SSE reconnect — xfail, confirms known TOCTOU
   race where _wait_approval pops pending entry allowing duplicate
2. Duplicate plan review on SSE reconnect — xfail, same pattern
3. approve_set consistency during concurrent update — passes
4. _running flag visibility across threads — passes
5. Workstream closure during blocked pop_response — passes
6. Concurrent approval + workstream close — passes (no orphaned state)

Two real races confirmed (marked xfail with fix descriptions).

* fix: address Copilot feedback on bridge stress tests

- Fix plan review mock to use correct message type ("plan_feedback")
- Replace fixed sleeps with bounded _wait_pending_clear() polling
- Add assert not t.is_alive() after all thread joins
- Update Race 5 description to reflect timeout validation (not
  closure-unblocks-pop)
- Update plan review xfail reason to mention generation counters
2026-03-23 11:36:42 -07:00
Patrick Buckley ab1a71c86c feat: add PostgreSQL CI integration tests (#156)
* feat: add PostgreSQL CI integration tests

Add --storage-backend pytest option and shared storage_backend fixture
in conftest.py that creates SQLiteBackend or PostgreSQLBackend based
on the flag. Migrate 13 storage test files to use shared fixture
instead of local SQLiteBackend fixtures.

Add test-postgres CI job with PostgreSQL 17 service container that
runs the full test suite against real PostgreSQL.

* fix: use TRUNCATE CASCADE for PG cleanup, wrap in try/finally

TRUNCATE is faster than per-table DELETE and resets autoincrement
sequences. try/except ensures reset_storage() always runs even if
cleanup fails due to a corrupted connection from a failing test.

* fix: document _engine coupling in PG cleanup comment
2026-03-23 11:11:40 -07:00
renovate[bot] ce57df6888 chore(deps): lock file maintenance 2026-03-23 11:04:43 -07:00
Patrick Buckley 275f40eebb feat: live session config via ConfigStore point-of-use reads (#154)
* feat: live session config via ConfigStore point-of-use reads

Existing sessions now pick up admin settings changes without requiring
workstream recreation. ChatSession reads MemoryConfig and JudgeConfig
behavioral flags from ConfigStore at point-of-use via _mem_cfg and
_judge_cfg properties. Judge LLM client config (model, provider,
base_url, api_key) stays frozen from creation time.

Falls back to frozen dataclasses when ConfigStore is absent (CLI mode).

* fix: type config_store param, clarify _ensure_judge guard comment

* fix: re-check live judge.enabled on every _ensure_judge call

Copilot correctly identified that the cached judge was returned without
re-checking the live enabled flag. Move the enabled check before the
cache check so disabling the judge via admin takes immediate effect.
Also defer JudgeConfig import to local scope in _judge_cfg property
to avoid pulling in the full judge module at import time.
Add test for disable-after-init scenario.
2026-03-21 19:16:06 -07:00
Patrick Buckley 2c510f8617 fix: medium reliability — SQLite WAL, eviction cancel, title retry (#151)
* fix: medium reliability — SQLite WAL + timeout, eviction cancel, title retry

M1: Increase SQLite busy timeout to 30s and enable WAL journal mode
    for better concurrent read/write. Prevents OperationalError under
    multi-workstream write contention.

M2: Call session.cancel() during workstream eviction cleanup so
    in-flight worker threads stop promptly instead of running to
    completion on an evicted workstream.

M3: Reset _title_generated flag on exception so title generation
    retries on the next successful exchange instead of permanently
    giving up after one failure.

* fix: address review — WAL pragma error handling, title retry ws_id guard

Wrap WAL pragma in try/except and verify returned mode. Log warning if
WAL is not enabled (e.g., filesystem permissions) instead of aborting
connection.

Guard title retry flag reset with ws_id comparison to prevent
re-enabling titling for a different workstream after /resume.

* fix: address review — add title retry tests

Two tests verifying _title_generated flag behavior: reset on failure
(allows retry on next turn), stays True on success. Covers the new
retry logic added in this PR.

* fix: guard title update success path against ws_id change during resume

Use captured ws_id on success path (not just failure path) so a
concurrent resume() can't cause the background title thread to rename
the wrong workstream. Add test for the race scenario.
2026-03-21 17:02:25 -07:00
Patrick Buckley 2afb9c7f72 fix: health monitor probe loop transitions OPEN → HALF_OPEN autonomously (#152)
The probe loop continued probing while the circuit was OPEN but never
transitioned to HALF_OPEN — that only happened inside
acquire_request_permit() which requires a user request. If no user
sends a message during the cooldown, the circuit never recovers.

Now the probe loop checks cooldown elapsed and transitions to HALF_OPEN
before probing, so recovery happens automatically without user
interaction.
2026-03-21 16:35:47 -07:00
Patrick Buckley 5b8ab94446 fix: align ConfigStore implementation with spec (#153)
* fix: align ConfigStore implementation with spec

- Add cluster + skills sections to admin UI settings order and labels
- Return default value in DELETE /v1/api/admin/settings response per spec
- Document 4 missing settings in docs/settings.md (trusted_proxies,
  output_guard, redact_secrets, discovery_url) and correct count to 48
- Wire ConfigStore into console server replacing 4 raw
  get_system_setting() calls with validated/cached config_store.get()
- Reload console ConfigStore on settings mutations via
  _publish_config_change()
- Update registry URL tests for ConfigStore-based resolution

* fix: address Copilot review feedback on ConfigStore PR

- Move config_store.reload() before collector guard in
  _publish_config_change() so cache refreshes even without collector
- Add DeleteSettingResponse schema and update OpenAPI spec to match
  the actual delete response (status + key + default)
- Add test asserting default field in delete response
- Fix stale docstring in test helper
2026-03-21 16:12:50 -07:00
Patrick Buckley 30828e9f9c perf: conversations index, batch config saves, capabilities cache (#149)
* perf: add conversations.timestamp index, batch config saves, cache capabilities

P1: Add idx_conversations_timestamp index (migration 025) to eliminate
    full table scans on search_history_recent ORDER BY timestamp DESC.

P2: Batch save_workstream_config — replace N separate SQL statements
    with single executemany call. SQLite uses INSERT OR REPLACE,
    PostgreSQL uses INSERT ON CONFLICT DO UPDATE.

P3: Cache _get_capabilities() result on ChatSession — called 4-6x per
    turn but deterministic for session lifetime. Invalidated on model
    switch.

* fix: address review — capabilities cache bypassed for fallback models

Cache only applies to the primary session model. Fallback models
(different provider/model passed to _get_capabilities) resolve fresh
to avoid stale capability flags affecting tool selection and web search.
2026-03-21 04:29:43 -07:00
Patrick Buckley 6d0dc6df94 chore: standardize logging to structlog get_logger across 19 modules (#150)
Replace bare `import logging` / `logging.getLogger(__name__)` with
`from turnstone.core.log import get_logger` / `get_logger(__name__)`
across all core modules and server.py. This enables structured log
context injection (node_id, ws_id, request_id) in modules that
previously used plain stdlib logging.

Also adds _ensure_stdlib_factory() to log.py for pytest caplog
compatibility when configure_logging() hasn't been called.

Renames `logger` to `log` in skill_sources.py for naming consistency.
2026-03-21 04:29:40 -07:00
Patrick Buckley 7e680ee883 chore: remove dead code — chat.py, singular touch, unused vars, inline imports (#148)
* chore: remove dead code — chat.py shim, singular touch method, unused vars

- Delete turnstone/chat.py (backward-compat re-export shim, zero importers)
- Remove touch_structured_memory() singular method from protocol + both
  backends + 6 tests (only plural batch form is used)
- Remove unused _last_err variable in _compact_messages
- Remove redundant _AGENT_AUTO_TOOLS / _TASK_AUTO_TOOLS class aliases,
  use module-level constants directly
- Consolidate ~76 inline schema imports to top-level in both storage
  backends (channel_users, channel_routes, oidc_*, scheduled_tasks,
  watches, services)

Net: -219 lines

* fix: address review — remove stale inline timedelta imports in prune_task_runs

timedelta is already imported at module scope in both backends.
2026-03-21 04:29:37 -07:00
Patrick Buckley e950219246 docs: add beta status warning to README
Mark platform as experimental beta with explicit disclaimers: no
guarantees of determinism, reliability, or backward compatibility.
Advise thorough evaluation before deployment.
2026-03-21 03:43:13 -07:00
Patrick Buckley b3764a8035 fix: critical reliability fixes for production readiness (#147)
* fix: critical reliability fixes for production readiness

C1: Add 1-hour timeout to _approval_event.wait() and _plan_event.wait()
    to prevent permanent worker thread hangs when users disconnect.

C2: Atomically check-and-start worker thread under Workstream._lock to
    prevent race condition where two concurrent send_message requests
    spawn duplicate workers on the same non-thread-safe ChatSession.

C3: Bound _watch_pending queue to maxsize=20 to prevent OOM under
    heavy watch load with busy workstreams.

H1: Add timeout to proc.wait() (10s) and stderr_thread.join() (5s)
    after SIGKILL to prevent indefinite hang on D-state processes.

H2: Protect _pending_verdicts with _ws_lock at all three mutation sites
    (reset in approve_tools, append in on_intent_verdict, swap-and-clear
    in resolve_approval) to prevent lost verdicts from concurrent
    judge daemon and approval threads.

H3: Bound global SSE queue to maxsize=10000 with put_nowait() and
    contextlib.suppress(queue.Full) for backpressure. Prevents
    unbounded memory growth when fanout thread is overloaded.

H4: Bridge SSE threads for closed workstreams now check ws_id membership
    in _ws_threads before reconnecting, preventing thread leak on
    workstream close.

* fix: address review — verdict lock consistency, watch queue non-blocking, SSE drop logging

- Move _last_verdict_decision set inside _ws_lock in resolve_approval()
  so swap+decision is atomic with on_intent_verdict() reads
- Read _last_verdict_decision under _ws_lock in on_intent_verdict()
- Build heuristic_verdicts locally then assign under lock in approve_tools()
- Use resolve_approval() for timeout path so verdicts are updated consistently
- Watch queue producer uses put_nowait with log on Full (prevents WatchRunner hang)
- Global SSE state broadcasts log on queue.Full instead of silent suppress
- Plan event wait also gets 1-hour timeout (same class of bug as approval)
2026-03-21 03:28:01 -07:00
Patrick Buckley 756c4d8929 fix: guard against CancelledError on MCP startup future (Python 3.14) (#146)
On Python 3.14, Future.exception() raises CancelledError on cancelled
futures instead of returning None. Check future.cancelled() before
calling exception() to prevent crash when the MCP event loop shuts down
before _connect_all completes.
2026-03-21 01:06:11 -07:00
Patrick Buckley 04c50568e9 feat: add priority column for skill ordering control (#144)
* feat: add priority column for skill ordering control

Add priority INTEGER DEFAULT 0 column to prompt_templates (migration
024). Skills with activation="default" are now ordered by priority ASC,
name ASC instead of name-only. Admins can set priority via create/update
API. Lower values run first. Priority is editable on readonly/installed
skills. 4 new tests. Python SDK, TypeScript SDK, and Pydantic models
updated.

* fix: address review — apply priority ordering to list_default_templates

list_default_templates() still ordered by name only, so priority had
no effect on default skill execution order. Update both SQLite and
PostgreSQL backends to order by (priority, name).

* fix: address review — regenerate OpenAPI snapshot, add default template ordering test

Regenerate openapi-console.json to include priority field on skill
models. Add test_list_default_templates_ordered_by_priority to verify
the execution path for default skills respects priority ordering.
2026-03-21 00:53:51 -07:00
Patrick Buckley 3bf220c503 fix: use approval_label for per-tool always-approve in CLI and bridge (#143)
* fix: use approval_label for per-tool always-approve in CLI and bridge

The server stores approval_label (e.g. mcp__server__tool) for per-tool
auto-approve, but CLI and bridge extracted only func_name (bare tool
name). This caused always-approve decisions to not carry over across
access paths. Align CLI and bridge to prefer approval_label with
func_name fallback, matching the server's WebUI.approve_tools() pattern.

* fix: address review — exclude errored items from bridge auto-approve check

Filter out items with error set from the auto-approve subset check,
matching the server's WebUI.approve_tools() behavior. Prevents
policy-denied items from affecting auto-approve decisions.
2026-03-21 00:42:33 -07:00
Patrick Buckley 4b853e329e test: verify skill_id/skill_version populated in workstreams table (#145)
The workstreams.skill_id and skill_version columns were already being
populated correctly (wired in the skills unification PR #106). Add two
tests confirming: lineage columns set when skill is applied, and
defaults when no skill is used. Check off the PROGRESS.md item.
2026-03-21 00:40:30 -07:00
Patrick Buckley 29ffdc36d0 test: add governance SDK integration tests against real Starlette app (#142)
* test: add governance SDK integration tests against real Starlette app

24 TestClient-based tests verifying round-trip serialization of SDK
governance methods (roles, policies, orgs) against actual route
handlers with SQLite storage. Covers create/list/update/delete
lifecycles, error cases, and Pydantic model field validation.

* fix: address review — close AsyncClient in sdk_client fixture teardown

Convert sdk_client fixture to async context manager so the httpx
AsyncClient is properly closed after tests, avoiding resource leak
warnings.
2026-03-21 00:31:53 -07:00
Patrick Buckley ada8b80509 test: add MCP reload and reconcile endpoint integration tests (#141)
* test: add MCP reload and reconcile endpoint integration tests

11 new tests covering POST /v1/api/admin/mcp-servers/reload (console)
and POST /v1/api/_internal/mcp-reload (node). Verifies reconcile_sync
invocation, fan-out results, permission checks, missing storage
handling, and mixed node error propagation.

* fix: address review — lazy-import internal_mcp_reload to avoid heavy module load

Move turnstone.server import inside _routes_with_internal() helper so
the full server module (which reads UI static assets) is only loaded
when node-side endpoint tests actually run, not during test collection.
2026-03-21 00:14:50 -07:00
Patrick Buckley 1f47ca62de fix: validate OIDC issuer URLs against SSRF before discovery fetch (#140)
* fix: validate OIDC issuer URLs against SSRF before discovery fetch

Add validate_issuer_url() that rejects private/loopback/link-local IPs,
non-HTTPS (except localhost for dev), embedded credentials, and
unresolvable hostnames. Called before the HTTP fetch in discover_oidc()
so the request is never made for invalid URLs. 17 new tests.

* fix: address review — use is_global, redact userinfo, catch ValueError

Use `not addr.is_global` instead of individual range checks to cover
all non-routable addresses (CGNAT, unspecified, multicast). Redact
credentials from error messages to prevent log leakage. Catch ValueError
from ip_address() for zone-indexed IPv6 addresses.
2026-03-21 00:14:46 -07:00
Patrick Buckley 83d9233304 test: skill session config application to workstreams (#139)
* test: skill session config application to workstreams

13 TestClient-based integration tests verifying that skill session
config fields (model, temperature, token_budget, auto_approve,
allowed_tools, reasoning_effort, agent_max_turns) are correctly applied
to ChatSession and WebUI when creating a workstream with a skill.

Covers: individual fields, combined application, disabled/unknown skill
rejection, zero-value no-ops.

* fix: address review — pass skill kwarg, clarify no-op test assertions

Pass skill=kwargs.get("skill") into ChatSession in test factory to
match production behavior. Clarify zero-value no-op test docstrings
to document they verify the handler's guard conditions, not observable
state changes.
2026-03-20 23:09:59 -07:00
Patrick Buckley bf06102d37 fix: memory access tracking and BM25 context caching (#138)
* fix: memory access tracking and BM25 context caching

Add touch_structured_memory/touch_structured_memories to storage
protocol + SQLite/PostgreSQL backends. Bumps last_accessed and
access_count on memory retrieval (BM25 injection + search results).
9 new storage tests.

Cache the scored BM25 memory context string on ChatSession, invalidated
on memory save/delete. Eliminates ~12 redundant storage queries + index
rebuilds per session lifecycle.

* fix: address review — deduplicate keys in touch facade, clarify contract

Deduplicate keys in the memory.py facade before calling storage so each
distinct memory is touched at most once. Update protocol docstring to
clarify per-call increment semantics. Add deduplication unit test.

* fix: replace unused-import test with real batch duplicate test

Replace facade dedup test (which only tested Python set logic) with a
real storage-level test that verifies duplicate keys each increment
access_count. Fixes ruff F401 lint failure.
2026-03-20 23:08:32 -07:00
Patrick Buckley e015b4512d fix: return typed Pydantic models from SDK skill methods (#137)
Skill methods on TurnstoneConsole and AsyncTurnstoneConsole returned
dict[str, Any] instead of validated Pydantic models. Update list_skills,
create_skill, get_skill, update_skill, list_skill_resources,
create_skill_resource, and install_skill to use response_model= with
ListSkillsResponse, SkillInfo, SkillResourceInfo, and
SkillInstallResponse.
2026-03-20 20:02:06 -07:00
Patrick Buckley 0c1afff7fc test: add _get_registry_url three-tier fallback tests (#136)
8 tests covering the DB setting → config.toml → default URL resolution
chain, including storage errors, empty values, malformed JSON, and
documenting that RuntimeError propagates uncaught through the except
clause.
2026-03-20 20:01:58 -07:00
Patrick Buckley a94051a995 fix: add split pane button to tab bar for discoverability (#135)
* fix: add split pane button to tab bar for discoverability

The split pane feature was only accessible via right-click context menu
or Ctrl+\ keyboard shortcut. Add a subtle split icon (⧉) to the tab
bar that appears at low opacity in single-pane mode. Hidden in
multi-pane mode where pane headers already provide split/close controls.

* fix: address review — change tab-bar from tablist to toolbar role

The tab bar contains both tabs and action buttons (new workstream,
split pane), which is invalid for role=tablist. Change to role=toolbar
which correctly describes a container of mixed interactive controls.

* fix: address design review — WCAG contrast, ARIA structure, mobile

- Drop opacity approach, use border: dashed var(--border) matching
  #new-tab-btn pattern (fixes WCAG contrast failure at 35% opacity)
- Nest tabs in #tab-list[role=tablist] inside toolbar (fixes invalid
  role=tab children inside role=toolbar)
- Hide split button on mobile (<600px) where splits can't work
- Add aria-keyshortcuts to both action buttons
2026-03-20 20:01:35 -07:00
Patrick Buckley 2f906ea1f9 fix: enable output guard in CLI mode (#134)
* fix: enable output guard in CLI mode

The heuristic output guard (credential redaction, prompt injection
detection) only ran in server mode because cli.py never constructed a
JudgeConfig. Additionally, the guard condition in session.py required
enabled=True, coupling the zero-cost heuristic (<5ms) to the full LLM
judge.

Wire JudgeConfig from existing CLI args into the session factory.
Decouple the output_guard condition from the enabled flag so the
heuristic guard runs even when the LLM judge is disabled via --no-judge.

* fix: address review — pass config.toml judge fields to CLI JudgeConfig

apply_config() merges [judge] section from config.toml into args as
judge_base_url and judge_api_key. Pass these through to JudgeConfig so
the CLI respects config.toml judge settings (e.g. separate judge
endpoint).
2026-03-20 20:01:31 -07:00
Patrick Buckley b61bfd1aa6 fix: validate URL scheme after MCP registry template substitution (#133)
* fix: validate URL scheme after MCP registry template substitution

resolve_install_config() substitutes user-provided values into URL
templates via string replacement without validating the resulting URL.
Add urlparse check after substitution to reject non-HTTP(S) schemes,
preventing SSRF-style redirection through crafted template variables.

* fix: address review — reject empty hostname and embedded credentials

Add hostname presence check and userinfo rejection after URL scheme
validation. Prevents URLs like https:///path (no host) and
https://user:pass@host (credential leakage in config). Two new tests.
2026-03-20 20:01:26 -07:00
Patrick Buckley 19c3a48b10 fix: server startup stampede — timeout model detection, non-fatal PG … (#132)
* fix: server startup stampede — timeout model detection, non-fatal PG migrations

detect_model() blocked the main thread for up to 400s when the LLM backend
was unreachable (OpenAI SDK default: 600s read timeout × 2 retries × TCP
retransmit). Cap startup detection at 10s with no retries — the
BackendHealthMonitor handles ongoing availability probing after startup.

PostgreSQL migrations via _run_with_pg_lock() crashed the server on lock
contention when 10 containers stampeded the advisory lock simultaneously.
Wrap in try/except matching the SQLite path — the entrypoint script already
runs migrations before the server process starts.

Health check start_period increased from 15s to 60s to accommodate the
startup sequence under load.

* fix: address review — narrow PG migration except, add detect_model test

Narrow the PG migration except clause to (OSError, EOFError) so DDL
errors still propagate. Add two unit tests for detect_model() verifying
with_options(timeout=10, max_retries=0) is called and that connection
errors in non-fatal mode return (None, None).
2026-03-20 20:01:21 -07:00
Patrick Buckley 414eb52d67 feat: raise scaling limits for 1000-node clusters (#129)
* feat: raise scaling limits for 1000-node clusters

Raise hardcoded limits throughout the codebase so clusters up to 1000
nodes work without configuration changes.

Scaling limits:
- max_workstreams default 10 → 50 (configurable via settings)
- Console fan-out concurrency 50 → 200 (configurable: cluster.node_fan_out_limit)
- MCP max servers 50 → 200 (configurable: cluster.mcp_max_servers)
- Console SSE queue 500 → 2000, server global SSE queue 500 → 1000
- httpx proxy pool: explicit max_connections on both proxy clients
- PostgreSQL pool 5+10 → 2+3 per process (right-sized for short-burst queries)
- Redis pool: explicit max_connections=200 on both sync and async brokers

Performance optimizations:
- Redis list_nodes(): replace N+1 SCAN+GET with SCAN+MGET
- Collector poll: raise thread pool to 200 (matches fan-out limit)
- Server SSE: dedicated ThreadPoolExecutor(200) for queue polling
- Fan-out: new get_all_nodes() removes hardcoded limit=1000 ceiling

Bug fixes:
- Settings reload notification was silently failing (called .get() on tuple)
- Watch fan-out only queried 500 nodes instead of full cluster

New cluster settings (configurable via admin Settings tab):
- cluster.node_fan_out_limit (default 200, range 10-1000)
- cluster.mcp_max_servers (default 200, range 1-2000)

Adds docs/pgbouncer.md for PostgreSQL connection pooling at scale.
Adds ddgStressCluster compose profile (100 nodes, 10 groups of 10).
Updates architecture, console, docker, settings, and API reference docs.

* fix: add image tag to compose anchors to avoid redundant builds

All cluster/stress services inherit `build:` from the anchor, causing
Docker to attempt 200+ separate builds. Adding `image: turnstone:local`
means Docker builds once and all services reuse the cached image.

* fix: address Copilot review feedback on scaling PR

- Remove magic number in get_all_nodes (limit=None instead of 2**31)
- Size httpx proxy pool from fan-out limit setting (not hardcoded 250)
- Cap cluster.node_fan_out_limit max_value to 500, mark restart_required
- Convert _publish_config_change from sync to async (was blocking event loop)
- Use shutdown(wait=True, cancel_futures=True) for SSE executor

* fix: add PostgreSQL env vars to cluster bridge anchor

Bridges initialize storage for auth/migrations but the bridge anchor
was missing TURNSTONE_DB_BACKEND and TURNSTONE_DB_URL, causing all
bridges to fall back to SQLite. With 100 bridges sharing the same
volume, concurrent SQLite migrations corrupt the database.

* fix: address Copilot round 2 + PG connection exhaustion at startup

Copilot feedback:
- Raise cluster.node_fan_out_limit max_value to 1000 (matches target)
- Cache fan-out limit on app.state at startup instead of re-reading DB
  per request (pool and semaphore now use the same value consistently)
- Remove unused params from _publish_config_change

Stress cluster fix:
- Raise PG max_connections to 300 (configurable via POSTGRES_MAX_CONNECTIONS)
  to handle 200 processes connecting simultaneously at startup
- Bump PG shared_buffers to 128MB and memory limit to 1G to match
- Add DB env vars to production bridge service

* fix readme

* fix: startup resilience for large clusters

Server no longer crashes when LLM backend is unreachable at startup.
detect_model() accepts fatal=False, returning (None, None) so the
server starts in degraded mode with circuit breaker open. The health
monitor will detect when the backend becomes available.

Migration runner retries with jittered exponential backoff (up to 10
attempts) when PostgreSQL rejects connections during startup stampedes.

Collector httpx pool sized to match poll workers (was using default of
100 connections with 200 workers).

Also addresses Copilot round 2:
- Raise cluster.node_fan_out_limit max_value to 1000
- Cache fan-out limit on app.state at startup
- Remove unused params from _publish_config_change
- Add DB env vars to production bridge service

* fix: replace silent error suppression with structured logging

Audit and fix 30+ instances of silently swallowed exceptions across 8
files. No-raise contracts are preserved — all changes add logging
while keeping the same return-value behavior.

memory.py (26 changes):
  Every storage operation now logs on failure. Previously the entire
  persistence facade had zero logging — messages, workstream state,
  and structured memories could silently stop being saved.

server.py:
  Usage recording failures now log at warning (was pass).
  Global SSE fan-out errors log at debug (was pass).

console/server.py:
  Config reload notification logs per-node failures at warning.
  Settings read fallbacks log at warning with the default value used.

auth.py:
  User existence check logs at warning (was pass).
  Setup rollback failures log at error (was suppress).
  OIDC state cleanup logs at debug (was suppress).

mcp_client.py:
  DB-managed MCP server list failure logs at warning (was pass).

collector.py:
  Node poll failure upgraded from debug to warning with exc_info.
  Health fetch failure logs at debug with exc_info (was silent).

bridge.py:
  Best-effort plan rejection logs at warning (was suppress).
  Malformed SSE data logs at debug (was suppress).

session.py:
  Tool output UI callback failure logs at debug (was suppress).

* fix: stagger collector poll with deterministic per-node jitter

Each node gets a stable offset within the first half of the poll
interval, derived from hashing the node_id against a Mersenne prime
(2^31 - 1). This spreads HTTP requests across the cycle instead of
firing all 100+ at the same instant.

Also raises poll interval from 10s to 15s and HTTP timeout from 5s
to 30s for large-cluster resilience.

* fix: add startup jitter to bridge heartbeat and health monitor probe

Bridge heartbeat: deterministic per-node jitter (from node_id hash)
spreads initial registration across the first quarter of the heartbeat
TTL. At 100 bridges with 60s TTL, heartbeats spread across 15s instead
of all firing at T=0.

Health monitor probe: deterministic per-process jitter (from PID hash)
spreads initial LLM backend probes across half the probe interval. At
100 servers with 30s interval, probes spread across 15s instead of all
hitting the LLM at T=30.

Both use the same Mersenne prime hashing approach as the collector poll
jitter for consistency.

* fix: split collector httpx timeout and raise keepalive pool

Use separate connect/read/write/pool timeouts instead of a single 30s
for all phases. Raise keepalive connections from 50 to 200 so the
collector reuses TCP connections across poll cycles instead of
constantly tearing down and re-establishing them.

* fix: narrow detect_model return type for CLI and eval callers

detect_model() now returns tuple[str | None, int | None] to support
fatal=False. CLI and eval always use fatal=True (the default), which
guarantees a non-None model or SystemExit. Add assert to narrow the
type for mypy.
2026-03-19 04:53:11 -07:00
Patrick Buckley 86b404177b chore: bump version to 0.8.4
- feat: split-pane layout for chat UI (#127)
- fix: enforce CSS min dimensions during split handle drag
- feat: add OpenShell sandbox policy for turnstone-server (#128)
- fix: collector JWT expiry causes silent workstream data wipe (#126)
- fix: auto-titler SSE event + SSE reconnection after restart (#125)
- fix: wire resume_ws through console + expose max_ws in heartbeat (#124)
2026-03-18 18:13:18 -07:00
Patrick Buckley 10165bb8a1 feat: add OpenShell sandbox policy for turnstone-server (#128)
* feat: add OpenShell sandbox policy for turnstone-server

Curated policy for running turnstone-server inside an OpenShell sandbox
with kernel-enforced security boundaries (Landlock, netns, seccomp).

- Filesystem: workdir read-write, /usr+/etc read-only, /tmp+/dev/null
  read-write, Landlock best_effort compatibility
- Network: default-deny with allowlisted LLM APIs (OpenAI, Anthropic),
  Tavily, skills.sh, GitHub (read-only L7), MCP registry (read-only L7),
  Redis localhost, package registries, curated web_fetch domains
- Git: L7-enforced read-only (info/refs + git-upload-pack only)
- Process: privilege drop to sandbox:sandbox
- Inference routing template for credential isolation (real API keys
  never enter the sandbox, resolved at proxy layer)

* fix: address PR #128 review feedback + add integration guide

Review fixes:
- Use python3 (not python) in usage examples to match binary allowlist
- Fix network_policy → network_policies in comment
- Remove /usr/bin/git from github_api (git uses github.com not
  api.github.com; already covered by git_operations policy)
- Remove pip/uv from bash_network_tools (package_registries already
  covers their PyPI access; no need for StackOverflow/Wikipedia reach)
- Restructure routes.yaml so commented blocks are indented under
  routes: key (uncomment without restructuring YAML)

New: docs/openshell.md covering policy customization, inference routing,
domain allowlisting, MCP subprocess inheritance, and the dual-layer
security model.
2026-03-18 18:09:47 -07:00
Patrick Buckley 5d478573cc fix: enforce CSS min dimensions during split handle drag
Drag ratio bounds were hardcoded at 0.1/0.9 which allowed panes to be
resized below their CSS min-width (200px) / min-height (150px), causing
input areas and text to overflow and clip. Now compute bounds dynamically
from the container size and CSS minimums.
2026-03-18 18:08:35 -07:00
Patrick Buckley 1b24e4717f feat: split-pane layout for chat UI (#127)
* feat: split-pane layout for chat UI

Refactor the server UI from a single-pane global-state design to a
multi-pane architecture with per-workstream Pane instances and a binary
layout tree. Each pane has its own SSE connection, message area, input,
and state (busy, approval, streaming).

Phase 1 — Pane class with 25 prototype methods encapsulating all
per-workstream state. Phase 2 — binary split tree (leaf/split nodes)
with recursive flexbox rendering and drag-to-resize handles. Phase 3 —
keyboard shortcuts (Ctrl+\, Ctrl+Shift+\, Ctrl+Shift+W, Ctrl+Alt+Arrow)
and right-click context menu. Phase 4 — layout persistence via
localStorage.

Key design decisions:
- No duplicate workstreams across panes (split refused if no unused ws,
  auto-close redundant pane on ws deletion)
- Max 6 panes to avoid exhausting browser SSE connections
- Viewport guard prevents splitting below min-width/min-height
- Only focused pane refreshes workstream list on SSE reconnect (prevents
  race when multiple panes disconnect simultaneously)
- Tab click focuses existing pane showing that ws in multi-pane mode
- Pointer events on drag handles for mouse + touch support
- Full a11y: ARIA roles/labels, keyboard nav in context menu, focus
  restoration, prefers-reduced-motion coverage

* fix: address PR #127 review feedback

- Add focusin handler so keyboard focus (Tab) updates focusedPaneId
- Context menu skips interactive elements (textarea, input, links,
  buttons) so native copy/paste and link context menus work
- Split handles get ARIA role=separator, aria-orientation, aria-valuenow,
  keyboard resizing (arrow keys, Home/End), and tabindex=0
- Enforce MAX_PANES limit in deserializeLayout to prevent corrupted
  localStorage from creating too many panes/SSE connections
- Update architecture.md to document split-pane layout
2026-03-18 18:03:46 -07:00
Patrick Buckley 9a2db63c07 fix: collector JWT expiry causes silent workstream data wipe (#126)
* fix: collector JWT expiry causes silent workstream data wipe

The console collector baked a one-time JWT snapshot into its httpx
client headers at startup. After 1 hour (JWT expiry), every poll to
server nodes returned 401. The error JSON was silently parsed as valid
empty data, wiping all workstream state while nodes still appeared
reachable — the cluster showed "10 nodes, 0 workstreams."

Root causes fixed:
- Collector: no auth baked into httpx.Client; per-request headers
  from ServiceTokenManager.token (auto-rotating) or static fallback
- Proxy: same pattern — proxy_client/proxy_sse_client created without
  auth headers; _proxy_auth_headers() injects fresh token per-request
- main(): static token snapshot only passed when no token_manager
  exists, preventing stale JWT from being stored anywhere
- _fetch_node: raise_for_status() before .json() so 401s throw
  instead of returning error JSON as "0 workstreams"
- Auth errors (401/403) logged at warning level for operator visibility

* fix: address PR #126 review — type annotation, regression tests, log messages

Tighten token_manager type from Any to ServiceTokenManager | None.
Add two regression tests verifying 401/403 poll responses preserve
existing workstream data and mark nodes unreachable. Fix misleading
log messages: "jwt_minted" → "token_manager_created" since
ServiceTokenManager mints lazily on first .token access.
2026-03-18 15:45:53 -07:00
Patrick Buckley e159837b74 fix: auto-titler SSE event + SSE reconnection after restart (#125)
* fix: auto-titler SSE event + SSE reconnection after restart

_generate_title() now calls self.ui.on_rename() after persisting the
title, so the tab bar, bridge, and console all update in real time.
Also handles multi-part (vision) content and replaces silent except
with log.debug.

SSE onerror handler now parses the workstreams response, replaces the
stale workstreams map, and switches to the first available workstream
if the current ws_id no longer exists (e.g. after server restart).
Previously it retried the stale ws_id forever.

* fix: address PR #125 review — avoid double reconnect + sync tab bar

Return immediately after switchTab/showDashboard on stale ws_id to
prevent scheduling a redundant connectContentSSE via setTimeout.
Always re-render tab bar after replacing the workstreams map so DOM
stays in sync even when currentWsId is still valid.
2026-03-18 15:14:14 -07:00
Patrick Buckley ec3454ee2e fix: wire resume_ws through console + expose max_ws in heartbeat (#124)
* fix: wire resume_ws through console + expose max_ws in heartbeat

Console create_workstream handler now reads resume_ws from the request
body and passes it to CreateWorkstreamMessage on all three dispatch paths
(pool, auto, explicit). Previously resume only worked via channel router
and direct CLI — the console layer never plumbed it through.

Server /health now includes max_ws from WorkstreamManager. Bridge reads
it on startup and includes it in heartbeat metadata so the console's
_pick_best_node gets accurate capacity instead of always defaulting to 10.
Collector also updates max_ws on subsequent heartbeats (not just discovery).

Schemas, Python SDK, TypeScript SDK, and OpenAPI specs updated. Test mocks
fixed for new max_workstreams property access in /health.

* fix: address PR #124 review — resume_ws tests + max_ws fetch on pre-set node_id

Add _fetch_server_metadata() so bridge reads max_ws from /health even
when node_id is pre-set (skipping _fetch_node_id). Without this, heartbeats
would advertise max_ws=10 regardless of actual server config.

Add 3 test cases verifying resume_ws flows through all three console
dispatch paths (directed, pool, auto-select).
2026-03-18 14:24:10 -07:00
renovate[bot] 5cbb832162 chore(deps): lock file maintenance 2026-03-18 13:20:49 -07:00
renovate[bot] 4e4ae2a91d chore(deps): update ghcr.io/astral-sh/uv docker tag to v0.10.11 2026-03-18 13:20:47 -07:00
renovate[bot] c7d0bac638 chore(deps): update astral-sh/setup-uv digest to 37802ad 2026-03-18 13:20:44 -07:00
Patrick Buckley e86305c143 chore: bump version to 0.8.3 2026-03-17 17:06:00 -07:00
Patrick Buckley d0fc42195a chore: remove dead code and fix noisy JWT test warnings
Remove unused methods (ToolSearchManager.should_activate, get_all_tools),
dead attributes (_all_tools, _threshold), unused constant (DEFAULT_INTERVAL),
unused Scenario protocol class, and vestigial parameters (judge._evaluate_single
heuristic, SimEngine.simulate_llm_response turn_number). Lengthen JWT test
secrets to >= 32 bytes to suppress InsecureKeyLengthWarning from PyJWT.
2026-03-17 17:02:25 -07:00
Patrick Buckley 760321f7ee refactor: extract _resolve_capabilities and _without_tool helpers
Extract _resolve_capabilities() shared helper so _get_capabilities()
and _run_agent() use the same config-override logic instead of
duplicating inline. Add _without_tool() module-level helper to
deduplicate the tool-filtering listcomp.
2026-03-17 16:49:11 -07:00
Patrick Buckley 693e51f782 fix: address PR #119 review feedback
Add UI error notification and exc_info logging to run_one() exception
handler so tool failures are visible in the frontend. Apply config.toml
capability overrides when gating web_search in _run_agent(), matching
the pattern used by _get_capabilities().
2026-03-17 16:49:11 -07:00
Patrick Buckley ba07409724 fix: isolate parallel tool exceptions + gate web_search without backend
Two bugs: (1) an uncaught exception in one parallel tool call killed the
entire batch via pool.map(), losing all results including successful ones.
Wrap run_one() in try/except so failures return error strings instead of
propagating. (2) web_search was offered to local models even without a
Tavily API key — the model would attempt it, only to fail at execution
time. Filter web_search from _get_active_tools() and _run_agent() when
neither native support nor Tavily is available.

Closes https://github.com/turnstonelabs/turnstone/issues/117
2026-03-17 16:49:11 -07:00
Patrick Buckley c76a61841e fix: PR #118 round 2 — null-safe parser, docs, consistency
- Null-safe extraction for description, license, and compatibility in
  skill_parser.py — YAML bare keys (e.g. `description:`) no longer
  produce the literal string "None"
- Log warning on skill catalog storage failure instead of silent swallow
- Use `enabled == 1` in list_skills_by_activation for consistency with
  other prompt_templates queries in both storage backends
- Add parser tests for YAML null description, license, and compatibility
- Update governance.md: document runtime config editing on installed
  skills, two-column modal layout, SPDX license dropdown, origin badge
2026-03-17 16:09:06 -07:00
Patrick Buckley 341d2f604f fix: address PR #118 review feedback
- Regenerate OpenAPI snapshots (openapi-console.json) to include license
  and compatibility fields in SkillInfo/CreateSkillRequest/UpdateSkillRequest
- Omit version from create/update payloads when blank so server applies
  default "1.0.0" instead of storing empty string
- Push enabled_only + limit filters into list_skills_by_activation storage
  query (protocol, SQLite, PostgreSQL) instead of loading all rows and
  filtering in Python; session.py now passes enabled_only=True, limit=30
- License length cap ([:128]) was already applied in previous commit
2026-03-17 16:09:06 -07:00
Patrick Buckley 3f7f8495d6 feat: skills modal redesign + runtime config editing for installed skills
Redesigns the create/edit/view skill modal into a two-column spec manifest
layout (Identity/Manifest/Deployment | Skill Content) matching the Agent
Skills spec structure. Installed (readonly) skills can now have their runtime
config (model, temperature, token limits, enabled) edited independently of
the locked spec/content fields.

- Two-column spec layout with section headings (Identity, Manifest, Deployment,
  Skill Content); content textarea uses monospace font and fills the column
- h3 section headings for screen-reader nav; h3 UA stylesheet reset in CSS
- Runtime Config collapsible uses 3-column grid; license field is now a select
  of SPDX identifiers (MIT, Apache-2.0, GPL-3.0, AGPL-3.0, etc.)
- Origin badge (cyan) shows source URL for installed skills in view mode
- server.py: _SKILL_RUNTIME_CONFIG_FIELDS frozenset; readonly skills filter
  updates to config-only fields (spec fields silently dropped); audit action
  distinguishes skill.update.config from skill.update; license field capped
  at 128 chars in both create and update paths
- governance.js: spec fields disabled for readonly; config fields always
  editable; Save button shown for all skills (labeled "Save Config" when
  readonly); collapsible state reset between modal opens prevents state leak;
  esk-allowed-tools disabled state driven by auto_approve not readonly
- Tests: spec-only body on readonly skill → 400; config-only → 200 with
  spec fields unchanged; mixed body → config fields applied, spec dropped
2026-03-17 16:09:06 -07:00
Patrick Buckley dc464ac313 feat: Agent Skills standard compliance + frontend spec fields
Brings skills implementation into full compliance with agentskills.io:

Parser:
- Read `allowed-tools` (hyphenated, standard) only; stored as
  `allowed_tools` internally — no underscore fallback
- Reject consecutive hyphens in skill names
- Extract author/version from standard `metadata:` map with top-level
  fallback; null-safe (no "None" string for bare YAML keys)
- Truncate description at 1024 chars, compatibility at 500 chars (spec
  caps) with log warnings
- Lenient parsing mode (lenient=True) for cross-client import: sanitizes
  names, returns None on skip, malformed-YAML colon-value retry
- Type overloads: strict mode returns ParsedSkill, lenient returns
  ParsedSkill | None

Session:
- `<available-skills>` XML catalog in system messages for
  activation="search" skills (disabled ones filtered out, capped at 30)

Tool rename:
- `load_skill` tool → `skill` (JSON, session preparers/executors,
  approval labels, tests, docs)

Storage (migration 023):
- Add `license` and `compatibility` columns to prompt_templates
- skill_license / compatibility params on create_prompt_template across
  protocol, SQLite, PostgreSQL backends
- Add to SKILL_MUTABLE for update_prompt_template

API + server:
- SkillInfo, CreateSkillRequest, UpdateSkillRequest: license +
  compatibility fields
- Create/update/install endpoints extract and persist both fields
- Install endpoint maps parsed.license + parsed.compatibility from
  imported SKILL.md (previously discarded)
- _skill_to_response() includes both fields

Admin UI:
- Create + edit modals: version, license, compatibility fields
- Readonly (imported) skills: "edit" → "view" button, modal title
  "View Skill", all fields disabled, Save hidden, Cancel → "Close",
  collapsibles auto-expand, focus on Close button
- :disabled CSS for dark-theme modal inputs (bg-highlight, cursor
  not-allowed, dimmed text)
- Fix addEventListener stacking on auto-approve checkboxes → .onchange

SDK: license + compatibility on SkillInfo, CreateSkillRequest,
UpdateSkillRequest TypeScript interfaces

Docs: governance.md, judge.md, tools.md, README, diagram updated
2026-03-17 16:09:06 -07:00
289 changed files with 28616 additions and 3975 deletions
+34 -107
View File
@@ -5,55 +5,40 @@
"helpers:pinGitHubActionDigests",
":separateMajorReleases"
],
"labels": [
"dependencies"
"gitIgnoredAuthors": [
"41898282+github-actions[bot]@users.noreply.github.com"
],
"labels": ["dependencies"],
"prConcurrentLimit": 5,
"prHourlyLimit": 2,
"schedule": [
"before 9am on Monday"
],
"schedule": ["before 9am on Monday"],
"timezone": "America/New_York",
"lockFileMaintenance": {
"enabled": true,
"schedule": [
"before 9am on Monday"
]
"schedule": ["before 9am on Monday"]
},
"customManagers": [
{
"customType": "regex",
"description": "Track vendored KaTeX version",
"managerFilePatterns": [
"/pyproject\\.toml$/"
],
"matchStrings": [
"katex-(?<currentValue>[\\d.]+)/"
],
"managerFilePatterns": ["/pyproject\\.toml$/"],
"matchStrings": ["katex-(?<currentValue>[\\d.]+)/"],
"depNameTemplate": "katex",
"datasourceTemplate": "npm"
},
{
"customType": "regex",
"description": "Track vendored Highlight.js version",
"managerFilePatterns": [
"/pyproject\\.toml$/"
],
"matchStrings": [
"hljs-(?<currentValue>[\\d.]+)/"
],
"managerFilePatterns": ["/pyproject\\.toml$/"],
"matchStrings": ["hljs-(?<currentValue>[\\d.]+)/"],
"depNameTemplate": "highlight.js",
"datasourceTemplate": "npm"
},
{
"customType": "regex",
"description": "Track vendored Mermaid version",
"managerFilePatterns": [
"/pyproject\\.toml$/"
],
"matchStrings": [
"mermaid-(?<currentValue>[\\d.]+)/"
],
"managerFilePatterns": ["/pyproject\\.toml$/"],
"matchStrings": ["mermaid-(?<currentValue>[\\d.]+)/"],
"depNameTemplate": "mermaid",
"datasourceTemplate": "npm"
}
@@ -62,14 +47,8 @@
{
"description": "LLM SDKs — always review manually",
"groupName": "LLM SDKs",
"matchPackageNames": [
"openai",
"anthropic",
"mcp"
],
"schedule": [
"before 9am on Monday"
],
"matchPackageNames": ["openai", "anthropic", "mcp"],
"schedule": ["before 9am on Monday"],
"automerge": false
},
{
@@ -83,73 +62,38 @@
"httpx-sse",
"pydantic"
],
"schedule": [
"before 9am on Wednesday"
],
"schedule": ["before 9am on Wednesday"],
"automerge": true,
"matchUpdateTypes": [
"patch"
]
"matchUpdateTypes": ["patch"]
},
{
"description": "Database layer",
"groupName": "Database",
"matchPackageNames": [
"sqlalchemy",
"alembic",
"psycopg"
],
"schedule": [
"before 9am on Wednesday"
],
"matchPackageNames": ["sqlalchemy", "alembic", "psycopg"],
"schedule": ["before 9am on Wednesday"],
"automerge": true,
"matchUpdateTypes": [
"patch"
]
"matchUpdateTypes": ["patch"]
},
{
"description": "Security-critical — always review manually",
"groupName": "Security",
"matchPackageNames": [
"PyJWT",
"pyjwt",
"bcrypt"
],
"matchPackageNames": ["PyJWT", "pyjwt", "bcrypt"],
"automerge": false
},
{
"description": "Infrastructure dependencies",
"groupName": "Infrastructure",
"matchPackageNames": [
"structlog",
"redis",
"croniter",
"discord.py"
],
"schedule": [
"before 9am on the first day of the month"
],
"matchPackageNames": ["structlog", "redis", "croniter", "discord.py"],
"schedule": ["before 9am on the first day of the month"],
"automerge": true,
"matchUpdateTypes": [
"patch"
]
"matchUpdateTypes": ["patch"]
},
{
"description": "Vendored JS — requires manual file download after merge",
"description": "Vendored JS — CI workflow downloads files automatically",
"groupName": "Vendored JS",
"matchPackageNames": [
"katex",
"highlight.js",
"mermaid"
],
"schedule": [
"before 9am on the first day of the month"
],
"automerge": false,
"prBodyNotes": [
"This PR updates version references only.",
"After merging, run `scripts/update-vendored-js.sh <lib> <version>` to download the actual files."
]
"matchPackageNames": ["katex", "highlight.js", "mermaid"],
"schedule": ["before 9am on the first day of the month"],
"automerge": false
},
{
"description": "Dev/test tooling",
@@ -162,46 +106,29 @@
"pytest-cov",
"pre-commit"
],
"schedule": [
"before 9am on the first day of the month"
],
"schedule": ["before 9am on the first day of the month"],
"automerge": true,
"matchUpdateTypes": [
"patch"
]
"matchUpdateTypes": ["patch"]
},
{
"description": "Docker base images",
"groupName": "Docker Images",
"matchManagers": [
"dockerfile",
"docker-compose"
],
"schedule": [
"before 9am on the first day of the month"
],
"matchManagers": ["dockerfile", "docker-compose"],
"schedule": ["before 9am on the first day of the month"],
"automerge": false
},
{
"description": "TypeScript SDK dev dependencies",
"groupName": "TypeScript SDK",
"matchFileNames": [
"sdk/typescript/**"
],
"schedule": [
"before 9am on the first day of the month"
],
"matchFileNames": ["sdk/typescript/**"],
"schedule": ["before 9am on the first day of the month"],
"automerge": true,
"matchUpdateTypes": [
"patch"
]
"matchUpdateTypes": ["patch"]
},
{
"description": "GitHub Actions — group all action updates",
"groupName": "GitHub Actions",
"matchManagers": [
"github-actions"
],
"matchManagers": ["github-actions"],
"automerge": false
}
]
+28 -2
View File
@@ -47,11 +47,37 @@ jobs:
name: coverage-${{ matrix.python-version }}
path: coverage.xml
test-postgres:
runs-on: ubuntu-latest
services:
postgres:
image: postgres:18
env:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
POSTGRES_DB: turnstone_test
ports:
- 5432:5432
options: >-
--health-cmd="pg_isready -U postgres"
--health-interval=10s
--health-timeout=5s
--health-retries=5
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
- uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6
with:
python-version: "3.14"
- run: pip install -e ".[test,mq,postgres]"
- run: pytest tests/ -m "not live" --storage-backend=postgresql -q
env:
TURNSTONE_TEST_PG_URL: postgresql+psycopg://postgres:postgres@localhost:5432/turnstone_test
lock-check:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
- uses: astral-sh/setup-uv@e06108dd0aef18192324c70427afc47652e63a82 # v7
- uses: astral-sh/setup-uv@37802adc94f370d6bfd71619e3f0bf239e1f3b78 # v7
with:
uv-version: "0.9.18"
- run: uv lock --check
@@ -60,7 +86,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
- uses: astral-sh/setup-uv@e06108dd0aef18192324c70427afc47652e63a82 # v7
- uses: astral-sh/setup-uv@37802adc94f370d6bfd71619e3f0bf239e1f3b78 # v7
with:
uv-version: "0.9.18"
- uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6
+1 -1
View File
@@ -19,7 +19,7 @@ jobs:
python-version: "3.14"
- run: pip install build
- run: python -m build
- uses: pypa/gh-action-pypi-publish@release/v1
- uses: pypa/gh-action-pypi-publish@ed0c53931b1dc9bd32cbe73a98c7f6766f8a527e # release/v1
- name: Create GitHub Release
uses: softprops/action-gh-release@153bb8e04406b158c6c84fc1615b65b24149a1fe # v2
+86
View File
@@ -0,0 +1,86 @@
name: Complete Vendored JS Updates
# When Renovate bumps a vendored JS version in pyproject.toml, this
# workflow downloads the actual files and commits them to the PR branch
# so the PR is merge-ready without manual intervention.
#
# Note: the commit is made with GITHUB_TOKEN, so it won't re-trigger CI
# automatically. The reviewer should re-run CI once this workflow passes,
# or Renovate's next rebase will trigger it.
on:
pull_request:
paths:
- pyproject.toml
workflow_dispatch:
inputs:
pr_number:
description: "PR number to update"
required: true
type: number
permissions:
contents: write
pull-requests: read
jobs:
vendor-js:
if: github.actor == 'renovate[bot]' || github.event_name == 'workflow_dispatch'
runs-on: ubuntu-latest
steps:
- name: Resolve PR head ref
id: ref
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
if [[ "${{ github.event_name }}" == "workflow_dispatch" ]]; then
ref=$(gh pr view "${{ inputs.pr_number }}" --repo "${{ github.repository }}" --json headRefName -q .headRefName)
else
ref="${{ github.head_ref }}"
fi
echo "head_ref=${ref}" >> "$GITHUB_OUTPUT"
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
with:
ref: ${{ steps.ref.outputs.head_ref }}
- name: Detect vendored JS changes
id: detect
run: |
updates=()
for lib in katex hljs mermaid; do
version=$(grep -oE "${lib}-[0-9.]+" pyproject.toml | head -1 | sed "s/${lib}-//")
[[ -z "$version" ]] && continue
[[ -d "turnstone/shared_static/${lib}-${version}" ]] && continue
updates+=("${lib}:${version}")
done
if [[ ${#updates[@]} -eq 0 ]]; then
echo "found=false" >> "$GITHUB_OUTPUT"
else
echo "found=true" >> "$GITHUB_OUTPUT"
printf '%s\n' "${updates[@]}" > /tmp/updates.txt
echo "Libs to update:"
cat /tmp/updates.txt
fi
- name: Download vendored files
if: steps.detect.outputs.found == 'true'
run: |
while IFS=: read -r lib version; do
echo "::group::Updating ${lib} to ${version}"
bash scripts/update-vendored-js.sh "$lib" "$version"
echo "::endgroup::"
done < /tmp/updates.txt
- name: Commit and push
if: steps.detect.outputs.found == 'true'
run: |
git config user.name "github-actions[bot]"
git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
git add -A
if git diff --cached --quiet; then
echo "No changes to commit"
exit 0
fi
git commit -m "chore: download vendored JS files"
git push
+4
View File
@@ -0,0 +1,4 @@
# libexpat integer overflow — no fix available in Debian repos yet
# https://avd.aquasec.com/nvd/cve-2026-25210
# Review: remove this entry once a patched libexpat1 is published
CVE-2026-25210
+9 -5
View File
@@ -8,10 +8,14 @@ FROM python:3.14-slim
LABEL org.opencontainers.image.title="turnstone" \
org.opencontainers.image.description="Multi-node AI orchestration platform"
COPY --from=ghcr.io/astral-sh/uv:0.10.10 /uv /usr/local/bin/uv
COPY --from=ghcr.io/astral-sh/uv:0.11.2 /uv /usr/local/bin/uv
# System dependencies for psycopg (PostgreSQL client library)
RUN apt-get update && apt-get upgrade -y && apt-get install -y --no-install-recommends libpq5 \
# Remove the slim image's man page exclusion so man-db has actual content
RUN rm -f /etc/dpkg/dpkg.cfg.d/docker
# System dependencies: psycopg (libpq5), developer tooling for agent workflows
RUN apt-get update && apt-get upgrade -y && apt-get install -y --no-install-recommends \
libpq5 git curl jq man-db manpages \
&& rm -rf /var/lib/apt/lists/*
# Non-root user
@@ -25,12 +29,12 @@ ENV UV_COMPILE_BYTECODE=1
# Install dependencies first (cached layer — only re-runs when deps change)
COPY pyproject.toml uv.lock README.md LICENSE ./
RUN uv sync --frozen --no-install-project --no-dev \
--extra mq --extra console --extra sim --extra postgres --extra discord --extra anthropic
--extra all
# Install the project itself
COPY turnstone/ turnstone/
RUN uv sync --frozen --no-dev \
--extra mq --extra console --extra sim --extra postgres --extra discord --extra anthropic
--extra all
# Add venv to PATH so entry points are found
ENV PATH="/app/.venv/bin:$PATH"
+11 -8
View File
@@ -5,9 +5,11 @@
[![Python](https://img.shields.io/pypi/pyversions/turnstone)](https://pypi.org/project/turnstone/)
[![License](https://img.shields.io/badge/license-BSL--1.1-blue)](LICENSE)
Multi-node AI orchestration platform. Deploy tool-using AI agents across a cluster of servers, driven by message queues or interactive interfaces.
Experimental multi-node AI orchestration platform. Deploy tool-using AI agents across a cluster of servers, driven by message queues or interactive interfaces.
Named after the [Ruddy Turnstone](https://en.wikipedia.org/wiki/Ruddy_turnstone) — a bird that flips rocks to expose what's hiding underneath.
> **Beta — Use at your own risk.** Turnstone is under active development and has not reached a stable release. APIs, configuration formats, and database schemas may change between versions without migration paths. We make no guarantees of determinism, reliability, or backward compatibility. Evaluate thoroughly before deploying to any environment where these properties matter.
Named after the [Ruddy Turnstone](https://en.wikipedia.org/wiki/Ruddy_turnstone) (*Arenaria interpres*) — a shorebird that flips stones to discover what's hiding underneath.
## What it does
@@ -145,7 +147,7 @@ Turnstone includes a built-in governance layer for enterprise deployments — ma
- **RBAC** — 15 granular permissions, 3 built-in roles (admin / operator / viewer), custom roles, privilege escalation prevention
- **OIDC SSO** — single sign-on via any OpenID Connect provider (Okta, Azure AD, Google, Keycloak); Authorization Code Flow with PKCE, auto-provisioning, claim-based role mapping with demotion propagation; see [docs/oidc.md](docs/oidc.md)
- **Tool policies** — glob-pattern rules (`allow` / `deny` / `ask`) with priority ordering; automate approvals or lock down dangerous tools
- **Skills** — reusable behavioral profiles with system prompts, `{{variable}}` substitution, session config (model, temperature, token budget), install-time security scanning, version history, external discovery (skills.sh / GitHub), and runtime `load_skill` tool for model-driven skill activation
- **Skills** — reusable behavioral profiles with system prompts, `{{variable}}` substitution, session config (model, temperature, token budget), install-time security scanning, version history, external discovery (skills.sh / GitHub), and runtime `skill` tool for model-driven skill activation
- **Usage tracking** — per-request token and tool metrics, aggregation by day / model / user, automatic 90-day pruning
- **Audit logging** — append-only event trail for all admin mutations, IP-aware, 365-day retention
@@ -288,7 +290,7 @@ All entry points read `~/.config/turnstone/config.toml`. CLI flags override conf
[api]
base_url = "http://localhost:8000/v1"
api_key = ""
tavily_key = "" # only needed for local/vLLM models without native search
# tavily_key = "" # only needed for local/vLLM models without native search
[model]
name = "" # empty = auto-detect
@@ -308,7 +310,7 @@ search_max_results = 5 # max tools returned per search query
[server]
host = "0.0.0.0"
port = 8080
max_workstreams = 10 # auto-evicts oldest idle when full
max_workstreams = 50 # auto-evicts oldest idle when full
[redis]
host = "localhost"
@@ -340,7 +342,7 @@ burst = 20
backend = "sqlite" # "sqlite" (default) or "postgresql"
path = ".turnstone.db" # SQLite file path (relative to working directory)
# url = "postgresql+psycopg://user:pass@host:5432/turnstone" # PostgreSQL
# pool_size = 5 # PostgreSQL connection pool size
# pool_size = 2 # PostgreSQL connection pool size (per process)
[judge]
enabled = true # intent validation for tool approvals (--no-judge to disable)
@@ -394,7 +396,7 @@ Idle workstreams are automatically cleaned up after 2 hours (configurable). In m
- `turnstone_judge_llm_latency_seconds` — LLM judge evaluation latency histogram
- `turnstone_judge_enabled` — whether the intent validation judge is active (0/1)
Per-workstream metrics are labeled by `ws_id` (bounded to 10 max workstreams).
Per-workstream metrics are labeled by `ws_id` (bounded by `[server].max_workstreams`).
### Health & Rate Limiting
@@ -404,7 +406,7 @@ Per-workstream metrics are labeled by `ws_id` (bounded to 10 max workstreams).
**Per-IP rate limiting.** When `[ratelimit].enabled` is true, each client IP is tracked with a token-bucket limiter (`requests_per_second` / `burst`). Rate limiting is applied in `do_GET`/`do_POST` after authentication but before route dispatch. `/health` and `/metrics` are exempt. Requests that exceed the limit receive HTTP 429 with a `Retry-After` header.
**Workstream eviction.** When `WorkstreamManager.create()` would exceed `max_workstreams`, the oldest IDLE workstream is automatically evicted and the `turnstone_workstreams_evicted_total` counter is incremented. Configure via `[server].max_workstreams` (default 10).
**Workstream eviction.** When `WorkstreamManager.create()` would exceed `max_workstreams`, the oldest IDLE workstream is automatically evicted and the `turnstone_workstreams_evicted_total` counter is incremented. Configure via `[server].max_workstreams` (default 50).
## Requirements
@@ -413,6 +415,7 @@ Per-workstream metrics are labeled by `ws_id` (bounded to 10 max workstreams).
- Redis (for message queue bridge — `pip install turnstone[mq]`)
- Anthropic provider (optional — `pip install turnstone[anthropic]`)
- PostgreSQL (optional, for production — `pip install turnstone[postgres]`)
- Math sandbox packages (optional — `pip install turnstone[sandbox]` for sympy, numpy, scipy, pytest)
- [Git LFS](https://git-lfs.com/) (for cloning — diagram PNGs are stored in LFS)
## License
+2354 -5
View File
File diff suppressed because it is too large Load Diff
+146
View File
@@ -0,0 +1,146 @@
# TLS overlay — enables mTLS across the turnstone cluster.
#
# Usage (requires base compose.yaml with production profile):
# docker compose -f compose.yaml -f deploy/docker-compose.tls.yml --profile production up
#
# The tls-init service bootstraps a CA and issues a cert for Redis.
# All turnstone services auto-provision their own certs via the
# console's ACME endpoint.
services:
# Bootstrap: create CA + Redis cert before anything starts.
# Runs as root to create directories in the volume, then chowns
# to turnstone:turnstone with restrictive perms (keys 0600).
tls-init:
build: .
user: root
command:
- sh
- -c
- |
set -e
turnstone-admin tls-bootstrap --out /certs --issue redis
chown -R turnstone:turnstone /certs
find /certs -type d -exec chmod 750 {} +
find /certs -type f -name '*key.pem' -exec chmod 600 {} +
find /certs -type f ! -name '*key.pem' -exec chmod 640 {} +
volumes:
- tls-certs:/certs
networks:
- turnstone-net
restart: "no"
# Console: runs the internal CA + ACME server
console:
depends_on:
tls-init:
condition: service_completed_successfully
volumes:
- tls-certs:/certs:ro
environment:
TURNSTONE_TLS_ENABLED: "true"
TURNSTONE_TLS_SANS: "console"
TURNSTONE_CONSOLE_URL: "http://console:8090"
command:
- turnstone-console
- --host=0.0.0.0
- --port=8090
- --redis-host=redis
- --redis-port=6379
- --poll-interval=${CONSOLE_POLL_INTERVAL:-10}
- --redis-tls
- --redis-tls-ca=/certs/ca.pem
# Server: auto-provisions certs via console ACME, serves HTTPS
server:
depends_on:
console:
condition: service_healthy
volumes:
- tls-certs:/certs:ro
environment:
TURNSTONE_TLS_ENABLED: "true"
TURNSTONE_TLS_SANS: "server"
# Disable healthcheck — server serves HTTPS with mTLS which the
# stdlib healthcheck script can't satisfy. The base compose
# healthcheck uses plain HTTP which won't work on an HTTPS listener.
# TODO: wire healthcheck with client cert from /certs volume
healthcheck:
disable: true
# Bridge: mTLS to server + Redis TLS
bridge:
depends_on:
console:
condition: service_healthy
server:
condition: service_started
redis:
condition: service_healthy
volumes:
- tls-certs:/certs:ro
environment:
TURNSTONE_TLS_ENABLED: "true"
TURNSTONE_TLS_SANS: "bridge"
command:
- turnstone-bridge
- --server-url=http://server:8080
- --redis-host=redis
- --redis-port=6379
- --heartbeat-ttl=${HEARTBEAT_TTL:-60}
- --approval-timeout=${APPROVAL_TIMEOUT:-3600}
- --redis-tls
- --redis-tls-ca=/certs/ca.pem
# Channel: Redis TLS
channel:
depends_on:
console:
condition: service_healthy
redis:
condition: service_healthy
volumes:
- tls-certs:/certs:ro
environment:
TURNSTONE_TLS_ENABLED: "true"
TURNSTONE_TLS_SANS: "channel"
command:
- sh
- -c
- >-
turnstone-channel
--redis-host=redis
--redis-port=6379
--redis-tls
--redis-tls-ca=/certs/ca.pem
--http-host=0.0.0.0
$${TURNSTONE_DISCORD_GUILD:+--discord-guild $$TURNSTONE_DISCORD_GUILD}
# Redis: TLS with certs from bootstrap
redis:
depends_on:
tls-init:
condition: service_completed_successfully
volumes:
- tls-certs:/certs:ro
command:
- sh
- -c
- |
ARGS="--tls-port 6379 --port 0 \
--tls-cert-file /certs/certs/redis/cert.pem \
--tls-key-file /certs/certs/redis/key.pem \
--tls-ca-cert-file /certs/ca.pem \
--tls-auth-clients no"
if [ -n "$$REDIS_PASSWORD" ]; then
ARGS="$$ARGS --requirepass $$REDIS_PASSWORD"
fi
exec redis-server $$ARGS
healthcheck:
test: ["CMD-SHELL", "if [ -n \"$$REDIS_PASSWORD\" ]; then redis-cli --tls --cacert /certs/ca.pem -a $$REDIS_PASSWORD ping; else redis-cli --tls --cacert /certs/ca.pem ping; fi"]
interval: 5s
timeout: 3s
retries: 5
volumes:
tls-certs:
+49
View File
@@ -0,0 +1,49 @@
# OpenShell inference routing for Turnstone.
#
# When using inference routing, the sandbox process connects to
# https://inference.local instead of the real LLM API. The OpenShell
# proxy intercepts, rewrites credentials, and forwards to the backend.
#
# This keeps real API keys out of the sandbox entirely — the process
# only sees opaque placeholder tokens in its environment.
#
# Usage:
# openshell sandbox run \
# --inference-routes deploy/openshell/routes.yaml \
# ...
#
# Then start turnstone with:
# python3 -m turnstone.server --base-url https://inference.local
#
# CUSTOMIZE: uncomment one of the provider blocks below.
routes:
# --- OpenAI ---
# - name: inference.local
# endpoint: https://api.openai.com/v1
# model: gpt-5
# provider_type: openai
# protocols:
# - openai_chat_completions
# - model_discovery
# api_key_env: OPENAI_API_KEY
# --- Anthropic ---
# - name: inference.local
# endpoint: https://api.anthropic.com
# model: claude-sonnet-4-6
# provider_type: anthropic
# protocols:
# - anthropic_messages
# api_key_env: ANTHROPIC_API_KEY
# --- Local model server (vLLM / llama.cpp) ---
# No secret resolution needed — local servers typically have no auth.
# Omit both api_key and api_key_env to skip credential injection.
# - name: inference.local
# endpoint: http://localhost:8000/v1
# model: meta-llama/Llama-3.1-70B-Instruct
# protocols:
# - openai_chat_completions
# - model_discovery
+333
View File
@@ -0,0 +1,333 @@
# OpenShell sandbox policy for Turnstone AI orchestration platform.
#
# This policy wraps a turnstone-server process (the primary sandbox target).
# The bridge, console, and channel gateway are separate processes that would
# each need their own sandbox with a tailored policy variant.
#
# Usage:
# openshell sandbox run \
# --policy deploy/openshell/turnstone-policy.yaml \
# --workdir /project \
# -- python3 -m turnstone.server --host 0.0.0.0 --port 8080
#
# For inference routing (keeps real API keys out of the sandbox):
# openshell sandbox run \
# --policy deploy/openshell/turnstone-policy.yaml \
# --inference-routes deploy/openshell/routes.yaml \
# --workdir /project \
# -- python3 -m turnstone.server --host 0.0.0.0 --port 8080 \
# --base-url https://inference.local
#
# Note: inference.local is intercepted by the OpenShell proxy before
# network policy evaluation — no network_policies entry is needed for it.
#
# Customization points (search for "CUSTOMIZE"):
# - OIDC issuer endpoint
# - Redis host/port (if not localhost)
# - MCP HTTP server endpoints
# - Additional tool binaries
# - web_fetch domain allowlist
version: 1
# ---------------------------------------------------------------------------
# Filesystem: Landlock kernel enforcement
# ---------------------------------------------------------------------------
# Static — cannot be changed after sandbox creation.
# include_workdir adds the --workdir path to read_write automatically.
filesystem_policy:
include_workdir: true
read_only:
# Python runtime + installed packages (includes turnstone package)
- /usr
- /lib
- /lib64
# System essentials
- /etc
- /proc
- /dev/urandom
# Turnstone config (read-only — writes go to database)
# CUSTOMIZE: adjust if config lives elsewhere
- /home/sandbox/.config/turnstone
read_write:
# Working directory is added via include_workdir
# Temp files (bash tool scripts, eval workdirs)
- /tmp
# Shell redirections (2>/dev/null)
- /dev/null
# SQLite database (default location is workdir, covered by include_workdir)
# Logs
- /var/log
landlock:
# best_effort: degrade gracefully on kernels without Landlock (< 5.13)
# Change to hard_requirement for production hardened deployments
compatibility: best_effort
# ---------------------------------------------------------------------------
# Process: privilege separation
# ---------------------------------------------------------------------------
process:
run_as_user: sandbox
run_as_group: sandbox
# ---------------------------------------------------------------------------
# Network: per-endpoint, per-binary allowlisting
# ---------------------------------------------------------------------------
# Default-deny. Only listed host:port pairs are reachable.
# Child processes (MCP servers, bash subcommands) inherit the network
# namespace — they cannot bypass the proxy.
network_policies:
# --- LLM API providers ---
openai_api:
name: openai-api
endpoints:
- host: api.openai.com
port: 443
binaries:
- path: /usr/bin/python3*
- path: /usr/local/bin/python3*
anthropic_api:
name: anthropic-api
endpoints:
- host: api.anthropic.com
port: 443
binaries:
- path: /usr/bin/python3*
- path: /usr/local/bin/python3*
# --- Web search fallback (Tavily) ---
tavily_api:
name: tavily-search
endpoints:
- host: api.tavily.com
port: 443
binaries:
- path: /usr/bin/python3*
- path: /usr/local/bin/python3*
# --- Skill discovery ---
skills_registry:
name: skills-registry
endpoints:
- host: skills.sh
port: 443
binaries:
- path: /usr/bin/python3*
- path: /usr/local/bin/python3*
github_api:
name: github-api
endpoints:
- host: api.github.com
port: 443
protocol: rest
tls: terminate
enforcement: enforce
access: read-only
- host: raw.githubusercontent.com
port: 443
binaries:
- path: /usr/bin/python3*
- path: /usr/local/bin/python3*
mcp_registry:
name: mcp-registry
endpoints:
- host: registry.modelcontextprotocol.io
port: 443
protocol: rest
tls: terminate
enforcement: enforce
access: read-only
binaries:
- path: /usr/bin/python3*
- path: /usr/local/bin/python3*
# --- OIDC SSO ---
# CUSTOMIZE: replace with your identity provider's hostname
# oidc_provider:
# name: oidc-provider
# endpoints:
# - host: login.example.com
# port: 443
# binaries:
# - path: /usr/bin/python3*
# - path: /usr/local/bin/python3*
# --- Redis (MQ) ---
# CUSTOMIZE: if Redis is not on localhost, add host + allowed_ips.
# localhost is blocked by default SSRF protection, so we need allowed_ips.
redis:
name: redis-mq
endpoints:
- port: 6379
allowed_ips:
- "127.0.0.1"
binaries:
- path: /usr/bin/python3*
- path: /usr/local/bin/python3*
# --- Discord (channel integration) ---
# Uncomment if using turnstone-channel with Discord adapter.
# discord:
# name: discord
# endpoints:
# - host: discord.com
# port: 443
# - host: gateway.discord.gg
# port: 443
# - host: cdn.discordapp.com
# port: 443
# binaries:
# - path: /usr/bin/python3*
# - path: /usr/local/bin/python3*
# --- web_fetch tool: curated domain allowlist ---
#
# This is the hard tradeoff. Turnstone's web_fetch tool lets the LLM
# fetch arbitrary public URLs. OpenShell cannot allow "all HTTPS" —
# every domain must be enumerated.
#
# Strategy: allowlist the domains your workloads actually need.
# The web_fetch tool will return a connection error for unlisted domains,
# which the LLM handles gracefully (it tells the user it can't reach
# that site).
#
# CUSTOMIZE: add domains your workstreams need to fetch from.
web_fetch_common:
name: web-fetch-common
endpoints:
# Documentation sites
- host: "**.readthedocs.io"
port: 443
- host: docs.python.org
port: 443
- host: "**.github.io"
port: 443
# Package registries (metadata lookups)
- host: pypi.org
port: 443
- host: www.npmjs.com
port: 443
# Stack Overflow / reference
- host: stackoverflow.com
port: 443
- host: "**.stackexchange.com"
port: 443
# Wikipedia
- host: "**.wikipedia.org"
port: 443
binaries:
- path: /usr/bin/python3*
- path: /usr/local/bin/python3*
# --- MCP HTTP servers ---
# CUSTOMIZE: add endpoints for any MCP servers using streamable-http
# transport. stdio-transport MCP servers need no network entry (they
# communicate via stdin/stdout pipes within the sandbox).
# mcp_http_servers:
# name: mcp-http
# endpoints:
# - host: mcp.internal.example.com
# port: 443
# binaries:
# - path: /usr/bin/python3*
# - path: /usr/local/bin/python3*
# --- Bash tool: curl/wget ---
# The bash tool can run curl/wget. These inherit the network namespace
# so they can only reach allowed endpoints. But they need binary entries
# to pass the proxy's identity check.
bash_network_tools:
name: bash-network-tools
endpoints:
# Mirrors web_fetch_common — curl/wget should have the same reach.
- host: "**.readthedocs.io"
port: 443
- host: docs.python.org
port: 443
- host: "**.github.io"
port: 443
- host: pypi.org
port: 443
- host: www.npmjs.com
port: 443
- host: stackoverflow.com
port: 443
- host: "**.stackexchange.com"
port: 443
- host: "**.wikipedia.org"
port: 443
binaries:
- path: /usr/bin/curl
- path: /usr/bin/wget
# --- Package installation ---
# pip install / uv add from the bash tool.
package_registries:
name: package-install
endpoints:
- host: pypi.org
port: 443
- host: files.pythonhosted.org
port: 443
- host: "**.pypi.org"
port: 443
binaries:
- path: /usr/bin/pip*
- path: /usr/local/bin/pip*
- path: /usr/bin/uv
- path: /usr/local/bin/uv
- path: /usr/bin/python3*
- path: /usr/local/bin/python3*
# --- Git operations ---
# read-only: clone, fetch, pull. No push (L7 enforcement).
git_operations:
name: git-read-only
endpoints:
- host: github.com
port: 443
protocol: rest
tls: terminate
enforcement: enforce
rules:
- allow:
method: GET
path: "/**/info/refs*"
- allow:
method: POST
path: "/**/git-upload-pack"
- host: gitlab.com
port: 443
protocol: rest
tls: terminate
enforcement: enforce
rules:
- allow:
method: GET
path: "/**/info/refs*"
- allow:
method: POST
path: "/**/git-upload-pack"
binaries:
- path: /usr/bin/git
+24 -9
View File
@@ -386,10 +386,10 @@ Each item in `items` (shared by `tool_info` and `approve_request`):
{"type": "tool_output_chunk", "call_id": "call_abc123", "chunk": "Building project...\n"}
```
**`tool_result`** -- final output from a completed tool execution. The `call_id` matches the corresponding `tool_info`/`approve_request` item and any preceding `tool_output_chunk` events. For bash tools, this arrives after all streaming chunks and includes both stdout and stderr.
**`tool_result`** -- final output from a completed tool execution. The `call_id` matches the corresponding `tool_info`/`approve_request` item and any preceding `tool_output_chunk` events. For bash tools, this arrives after all streaming chunks and includes both stdout and stderr. The `is_error` field is `true` when the tool execution failed (e.g. bash exit code >= 2 or signal, file not found, timeout). Exit code 1 is ambiguous (e.g. `grep` no-match) and is not flagged. User denials are tracked separately via a `denied` flag. Clients should use `is_error` instead of text-prefix heuristics.
```json
{"type": "tool_result", "call_id": "call_abc123", "name": "bash", "output": "file1.py\nfile2.py\n"}
{"type": "tool_result", "call_id": "call_abc123", "name": "bash", "output": "file1.py\nfile2.py\n", "is_error": false}
```
**`status`** -- token usage statistics, sent after each model turn.
@@ -452,9 +452,12 @@ after `/clear` or `/new` commands).
{"type": "clear_ui"}
```
**`cancelled`** -- the generation was cancelled by the user (via the Stop
button or `POST /v1/api/cancel`). The client should finalize any in-progress
assistant message with whatever partial content was streamed.
**`cancelled`** -- a cancel request was acknowledged (via the Stop button or
`POST /v1/api/cancel`). This signals that cancellation is in progress, not
that it is complete. The worker thread may still be finishing — wait for
`stream_end` before transitioning to a ready state. The client should clear
any in-progress assistant rendering but not re-enable the send button until
`stream_end` arrives.
```json
{"type": "cancelled"}
@@ -554,7 +557,7 @@ Possible `state` values:
| `error` | An error occurred |
**Fan-out pattern:** Each connected client receives its own bounded queue
(`maxsize=500`). A dedicated fan-out thread reads from the shared global queue
(`maxsize=1000`). A dedicated fan-out thread reads from the shared global queue
and copies each event to every client queue. If a client queue is full, the
event is silently dropped for that client.
@@ -793,23 +796,35 @@ containing the resumed session's messages.
Cancels the active generation in a workstream. Sets a cooperative cancellation
flag that is checked at multiple points in the generation loop (per streaming
chunk, before tool execution, inside bash commands). The session transitions to
`idle` state and preserves any partial content already streamed.
chunk, before tool execution, inside bash commands). Also closes the underlying
HTTP stream to the LLM provider, unblocking any pending read immediately.
The session transitions to `idle` state and preserves any partial content
already streamed.
If the workstream is waiting for tool approval or plan review, the pending
prompt is automatically denied/rejected to unblock the worker thread.
Calling this endpoint when the workstream is already idle is a harmless no-op.
**Force cancel:** When `force` is `true`, the server abandons the stuck worker
thread immediately and transitions the workstream to `idle`. The abandoned
thread continues to wind down in the background (killing any running
subprocesses and exiting at the next cancellation checkpoint). During this
wind-down it may emit a final `stream_end` event which the server suppresses
for the orphaned thread. Use force cancel when cooperative cancel has not
resolved within a few seconds — the web UI offers this as a "Force Stop"
button automatically.
**Request body:**
```json
{"ws_id": "abc123"}
{"ws_id": "abc123", "force": false}
```
| Field | Type | Required | Description |
|--------|--------|----------|----------------------|
| `ws_id`| string | yes | Target workstream ID |
| `force`| bool | no | Abandon stuck worker immediately (default: `false`) |
**Response:**
+28 -12
View File
@@ -3,7 +3,7 @@
Turnstone is an AI orchestration platform with tool use, parallel workstreams, and persistent
memory. It connects to any OpenAI-compatible API (local vLLM, OpenAI, etc.) or
Anthropic's native Messages API via pluggable provider adapters, and gives the
model 17 built-in tools plus external tools via MCP (Model Context Protocol) for
model 19 built-in tools plus external tools via MCP (Model Context Protocol) for
reading, writing, searching, planning, and executing code.
The core design principle is a **UI-agnostic engine with pluggable frontends**.
@@ -91,7 +91,7 @@ turnstone/
_config.py Base ChannelConfig dataclass
discord/ Discord adapter (bot, cog, views, streaming, config)
shared_static/ Shared design system (base.css, auth.js, theme.js, toast.js, utils.js, kb.js)
katex-0.16.38/ Vendored KaTeX math rendering library (MIT, woff2 fonts)
katex-0.16.44/ Vendored KaTeX math rendering library (MIT, woff2 fonts)
ui/
colors.py ANSI color constants with NO_COLOR support
markdown.py Streaming terminal markdown renderer (line-buffered)
@@ -100,7 +100,7 @@ turnstone/
index.html Single-page app shell (links to CSS and JS)
style.css Page-specific UI styles (dashboard, markdown elements, approval blocks)
renderer.js Markdown + LaTeX renderer (tables, nested lists, blockquotes, KaTeX math)
app.js Page-specific client-side JavaScript (SSE, workstreams, tool approval)
app.js Split-pane UI (Pane class, binary layout tree, SSE, tool approval)
tools/
*.json 15 tool schemas (OpenAI function-calling format + turnstone metadata)
```
@@ -242,7 +242,7 @@ class SessionUI(Protocol):
def on_content_token(self, text: str) -> None: ...
def on_stream_end(self) -> None: ...
def approve_tools(self, items: list[dict]) -> tuple[bool, str | None]: ...
def on_tool_result(self, call_id: str, name: str, output: str) -> None: ...
def on_tool_result(self, call_id: str, name: str, output: str, *, is_error: bool = False) -> None: ...
def on_tool_output_chunk(self, call_id: str, chunk: str) -> None: ...
def on_status(self, usage: dict, context_window: int, effort: str) -> None: ...
def on_plan_review(self, content: str) -> str: ...
@@ -259,7 +259,7 @@ class SessionUI(Protocol):
| Class | Module | Notes |
|-------|--------|-------|
| `TerminalUI` | `turnstone.cli` | ANSI colors, `MarkdownRenderer`, `Spinner`, readline-based `input()` for approval |
| `WebUI` | `turnstone.server` | SSE event queue per workstream, `threading.Event` for blocking on approval/plan |
| `WebUI` | `turnstone.server` | SSE event queue per workstream + global broadcast, `threading.Event` for blocking on approval/plan. `on_state_change` sends to both per-workstream and global SSE (the browser UI uses per-workstream `state_change` events to manage busy/idle transitions; `stream_end` only finalizes markdown rendering). |
| `NullUI` | `turnstone.eval` | Discards all output; `approve_tools` always returns `(True, None)` |
### WorkstreamTerminalUI
@@ -353,7 +353,7 @@ remove the tab immediately. Controlled by `--workstream-idle-timeout` (default:
**Workstream eviction at capacity:** When `WorkstreamManager.create()` would
exceed `max_workstreams` (configurable via `[server].max_workstreams`, default
10), the oldest IDLE workstream is automatically evicted to make room. The
50), the oldest IDLE workstream is automatically evicted to make room. The
`turnstone_workstreams_evicted_total` counter is incremented on each eviction.
If no IDLE workstream is available the create request fails as before.
@@ -375,12 +375,19 @@ non-idle background workstreams above the input prompt.
### Web Workstreams
- **Tab bar**: Each workstream renders as a tab with a colored state indicator
(CSS `@keyframes pulse` animation per state).
- **Per-tab SSE**: `connectContentSSE(wsId)` opens
`/v1/api/events?ws_id=<id>` for the active tab's event stream.
(CSS `@keyframes pulse` animation per state). Clicking a tab switches the
focused pane's workstream (or focuses an existing pane showing that ws).
- **Split panes**: The UI supports tiling multiple workstreams side-by-side or
stacked via a binary layout tree. Each `Pane` instance encapsulates its own
SSE connection, message area, input, and state (busy, approval, streaming).
Split via right-click context menu, pane header buttons, or keyboard
(`Ctrl+\`, `Ctrl+Shift+\`). Max 6 panes; no duplicate workstreams across panes.
Layout persisted to `localStorage`.
- **Per-pane SSE**: `Pane.connectSSE(wsId)` opens
`/v1/api/events?ws_id=<id>` for each pane's event stream independently.
- **Global SSE**: `connectGlobalSSE()` opens `/v1/api/events/global` which
receives `ws_state` broadcasts from all workstreams, used to update tab
indicators without switching.
indicators and pane headers without switching.
- **New tab / close**: POST `/v1/api/workstreams/new`, POST `/v1/api/workstreams/close`.
### Thread Safety
@@ -828,10 +835,16 @@ and are the single source of truth for both backends and Alembic migrations.
backend = "sqlite" # "sqlite" | "postgresql"
path = ".turnstone.db" # SQLite file path
url = "" # PostgreSQL connection URL
pool_size = 5 # PostgreSQL connection pool size
pool_size = 2 # PostgreSQL connection pool size (per process)
```
Environment variables: `TURNSTONE_DB_BACKEND`, `TURNSTONE_DB_URL`, `TURNSTONE_DB_PATH`.
Environment variables: `TURNSTONE_DB_BACKEND`, `TURNSTONE_DB_URL`, `TURNSTONE_DB_PATH`,
`TURNSTONE_DB_POOL_SIZE`.
The default pool is intentionally small (2 base + 3 overflow = 5 per process)
because all database operations are short-burst queries that hold connections for
milliseconds. For clusters with many nodes sharing a PostgreSQL instance, use
[PgBouncer](pgbouncer.md) in transaction pooling mode.
### Persistence and Resume
@@ -953,6 +966,9 @@ warns if the summary was truncated.
seeded with `history.replaceState({turnstone: 'dashboard'})` on load. The
`popstate` listener restores the correct tab or shows the dashboard,
guarded by `_historyNavigation = true` to prevent re-entrant pushState.
- **Pane focus**: `mousedown` and `focusin` events on pane containers update
`focusedPaneId`. Approval shortcuts (y/n/a) apply to the focused pane.
`Ctrl+Alt+Arrow` cycles focus between panes.
### Eval Resilience
+5 -4
View File
@@ -69,10 +69,11 @@ All reads and writes to the node/workstream map are protected by a single `threa
### Scale Considerations
- **10,000 workstreams** at ~500 bytes each = ~5 MB in memory
- **1,000 nodes** polled in parallel with 50 threads at ~100ms each = ~2 second poll cycle
- **50,000 workstreams** (1,000 nodes × 50 per node) at ~500 bytes each = ~25 MB in memory
- **1,000 nodes** polled in parallel — fan-out concurrency is configurable via `cluster.node_fan_out_limit` (default 200), yielding 5 batches at ~100ms each = ~0.5 second poll cycle
- **Filtering and pagination** run in-memory on the full workstream list — sub-millisecond at this scale
- **SSE fan-out** uses the same per-client queue pattern as the per-node server — backed-up clients get events dropped, not blocking
- **SSE fan-out** uses per-client queues (2,000 events) — backed-up clients get events dropped, not blocking
- **Database** — for clusters sharing PostgreSQL, use [PgBouncer](pgbouncer.md) in transaction pooling mode
---
@@ -364,7 +365,7 @@ SSE streams (`/v1/api/events`, `/v1/api/events/global`) are proxied as raw byte
### Authentication
The proxy forwards the user's JWT to upstream server nodes — it extracts the token from the incoming request's cookie (or `Authorization` header) and adds it as a `Bearer` header on the proxied request. Since all services share the same `TURNSTONE_JWT_SECRET`, the user's JWT is valid on every node without re-authentication. The console's own auth middleware also checks proxy routes — `POST` requests to proxy write endpoints (`/v1/api/send`, `/v1/api/approve`, etc.) require `write` scope, preventing read-only tokens from escalating via proxy. The static `--auth-token` / `proxy_auth_token` is used as a fallback when no user JWT is present.
The proxy mints a short-lived (5-minute) JWT per request carrying the real user's `user_id`, `scopes`, and `permissions` with `aud: turnstone-server`. The user's console JWT (`aud: turnstone-console`) cannot be forwarded directly — it would be rejected by the server's audience validation — so the console re-signs a new server-audience JWT from the validated `AuthResult`. This preserves audit attribution (the upstream server sees the real user, not a service identity) and enforces scope narrowing as defense in depth (a read-only console user's proxied request carries only `read` scope). The JWT `src` claim is set to `"console-proxy"` for audit traceability. When no user context is available (auth disabled), the proxy falls back to a `ServiceTokenManager` with service identity `console-proxy`. The static `--auth-token` / `proxy_auth_token` is used as a final fallback.
---
+1 -1
View File
@@ -97,7 +97,7 @@ package "turnstone/sdk/" <<Rectangle>> {
' Tool schemas
package "turnstone/tools/" <<Rectangle>> {
component [*.json\n18 tool schemas] as schemas <<artifact>>
component [*.json\n19 tool schemas] as schemas <<artifact>>
}
' Entry point dependencies
+1 -3
View File
@@ -12,7 +12,7 @@ interface "SessionUI" as SessionUI <<Protocol>> {
+ on_content_token(text: str)
+ on_stream_end()
+ approve_tools(items: list) → (bool, str|None)
+ on_tool_result(call_id: str, name: str, output: str)
+ on_tool_result(call_id: str, name: str, output: str, *, is_error: bool = False)
+ on_tool_output_chunk(call_id: str, chunk: str)
+ on_status(usage: dict, ctx_window: int, effort: str)
+ on_plan_review(content: str) → str
@@ -250,13 +250,11 @@ class "MCPClientManager" as MCPMgr {
' ToolSearchManager
class "ToolSearchManager" as ToolSearchMgr {
- _all_tools: list[dict]
- _always_on: list[dict]
- _deferred: list[dict]
- _expanded: dict[str, None]
- _index: BM25Index
--
+ should_activate() → bool
+ get_visible_tools() → list[dict]
+ get_deferred_tools() → list[dict]
+ get_expanded_names() → list[str]
+2 -1
View File
@@ -133,7 +133,8 @@ group loop [while tool_calls present]
note right of TP
bash: on_tool_output_chunk(call_id, line)
called per stdout line,
then on_tool_result(call_id, name, output).
then on_tool_result(call_id, name, output, is_error).
is_error=True when execution failed.
call_id routes chunks/results to correct
tool div during parallel execution.
Other tools: on_tool_result() only.
+7 -4
View File
@@ -24,7 +24,7 @@ partition "Phase 1: Prepare" #E8F5E9 {
:Dispatch to _prepare_{func_name}();
note right
**Dispatch table (17 tools):**
**Dispatch table (19 built-in + tool_search):**
┌───────────────┬──────────────────┐
│ Tool │ Needs Approval? │
├───────────────┼──────────────────┤
@@ -33,16 +33,19 @@ partition "Phase 1: Prepare" #E8F5E9 {
│ write_file │ ✓ Yes │
│ edit_file │ ✓ Yes │
│ search │ ✗ Auto-approve │
│ diff_file │ ✗ Auto-approve │
│ math │ ✗ Auto-approve │
│ man │ ✗ Auto-approve │
│ web_fetch │ ✗ Auto-approve │
│ web_search │ ✗ Auto-approve │
│ tool_search │ ✗ Auto-approve │
│ task │ ✓ Yes │
│ plan │ ✓ Yes │
│ task_agent │ ✓ Yes │
│ plan_agent │ ✓ Yes │
│ memory │ ✗ Auto-approve │
│ recall │ ✗ Auto-approve │
│ notify │ ✗ Auto-approve │
│ watch │ ✓ create only │
│ skill │ ✓ load only │
│ read_resource │ ✓ Yes │
│ use_prompt │ ✓ Yes │
├───────────────┼──────────────────┤
@@ -127,7 +130,7 @@ partition "Phase 3: Execute" #E3F2FD {
:_truncate_output() on each result\n(max context_window × chars_per_token × 0.5 chars\ndefault: ~context_window × 2 chars);
:bash: ui.on_tool_output_chunk(call_id, line) per stdout line;
:ui.on_tool_result(call_id, name, output) for each;
:ui.on_tool_result(call_id, name, output, is_error) for each;
if (plan tool was executed?) then (yes)
:ui.on_plan_review(output);
+2
View File
@@ -61,6 +61,7 @@ package "Inbound Messages (Client → Bridge)" as InPkg #FFF3E0 {
+ target_node: str = ""
+ initial_message: str = ""
+ skill: str = ""
+ user_id: str = ""
}
class CloseWorkstreamMessage {
@@ -146,6 +147,7 @@ package "Outbound Events (Bridge → Client)" as OutPkg #E3F2FD {
+ call_id: str
+ name: str
+ output: str
+ is_error: bool
}
class PlanReviewEvent {
type = "plan_review"
+12 -1
View File
@@ -40,12 +40,23 @@ running --> error : Exception during\ntool execution
error --> thinking : New send() call\n_emit_state("thinking")
thinking --> idle : cancel() called\n_emit_state("idle")
thinking --> idle : cancel() called\nstream aborted\n_emit_state("idle")
running --> idle : cancel() called\n_emit_state("idle")
attention --> idle : cancel() unblocks\napproval/plan wait\n_emit_state("idle")
note left of idle
**Cancel escalation:**
1. **Cooperative**: cancel() sets event + closes
SDK stream → worker exits at next checkpoint
2. **Force**: force=true abandons the worker
thread, emits stream_end immediately.
Orphaned thread still kills subprocesses
but skips message mutations (generation
counter prevents stale writes).
end note
note right of thinking
**Emitted via:**
session._emit_state(state)
+4 -3
View File
@@ -105,7 +105,7 @@ Server -> CC : get_snapshot()
CC --> Server : ClusterSnapshot\n(full current state)
Server -> CC : register_listener(queue)
note right : Per-client queue.Queue(maxsize=500)\nSSE via EventSourceResponse + run_in_executor()
note right : Per-client queue.Queue(maxsize=2000)\nSSE via EventSourceResponse + run_in_executor()
Server -> Browser : data: {"type":"snapshot",...}\n(full state as first SSE event)
@@ -154,7 +154,7 @@ activate Server #FFECB3
Server -> CC : _pick_best_node() or\nget_node_detail(node_id)
CC --> Server : node validated
Server -> Server : Build CreateWorkstreamMessage\n{target_node:"nodeA", name:"new-task"}
Server -> Server : Build CreateWorkstreamMessage\n{target_node:"nodeA", name:"new-task",\nuser_id: from auth_result}
Server -> Redis : RPUSH turnstone:inbound:nodeA\n(directed queue)
Server --> Browser : {status:"ok", correlation_id:"abc",\ntarget_node:"nodeA"}
@@ -163,7 +163,8 @@ deactivate Server
note right of Redis
Bridge on Node-A picks up the
message from its directed queue,
POSTs to /v1/api/workstreams/new,
POSTs to /v1/api/workstreams/new
(forwarding user_id in payload),
registers ownership, publishes
ws_created to cluster channel.
end note
+29
View File
@@ -56,6 +56,26 @@ node "Docker Host" as host {
end note
}
node "postgres (profile: production)" <<pgautoupgrade>> as pg_node {
component [PostgreSQL\nport 5432] as postgres
note bottom of postgres
Healthcheck: pg_isready
Volume: postgres-data
Required for cluster
and production profiles
end note
}
node "pgbouncer (optional)" <<bitnami/pgbouncer>> as pgb_node {
component [PgBouncer\nport 6432] as pgbouncer
note bottom of pgbouncer
pool_mode: transaction
Recommended for clusters
> 50 nodes
See docs/pgbouncer.md
end note
}
node "sim (profile: sim)" <<turnstone image>> as sim_node {
component [turnstone-sim] as sim
note bottom of sim
@@ -92,6 +112,11 @@ console --> server : HTTP polling + proxy\n(GET /v1/api/dashboard,\nproxy /node/
sim --> redis : Redis protocol\n(queues + pubsub + keys)
' Database connections (production/cluster profiles)
server ..> pgbouncer : PostgreSQL\n(pool_size=2)
console ..> pgbouncer : PostgreSQL\n(auth/admin)
pgbouncer --> postgres : transaction\npooling
' Environment variables
note right of host
**Environment Variables:**
@@ -99,13 +124,17 @@ note right of host
• OPENAI_API_KEY — API key
• REDIS_PASSWORD — Redis auth
• TURNSTONE_AUTH_TOKEN — API auth
• TURNSTONE_DB_URL — PostgreSQL URL
• POSTGRES_PASSWORD — DB password
end note
' Volumes
database "redis-data" as rv
database "turnstone-data" as tv
database "postgres-data" as pv
redis_node --> rv
server_node --> tv
pg_node --> pv
@enduml
+6 -5
View File
@@ -53,10 +53,10 @@ class "SQLiteBackend" as SQLite <<sqlite>> {
class "PostgreSQLBackend" as PG <<postgres>> {
-_engine: sa.Engine
+__init__(url: str, pool_size: int)
+__init__(url: str, pool_size: int = 2,\n max_overflow: int = 3)
--
tsvector + ILIKE search
Connection pooling
Connection pooling (5 max per process)
}
' -- Schema --
@@ -151,7 +151,7 @@ note right of Registry
backend = "sqlite" | "postgresql"
url = "postgresql+psycopg://..."
path = ".turnstone.db"
pool_size = 5
pool_size = 2 (+ 3 overflow)
end note
note bottom of SQLite
@@ -162,8 +162,9 @@ end note
note bottom of PG
Production backend.
Multi-node / Docker
default.
Multi-node / Docker default.
Use PgBouncer (transaction mode)
for clusters > 50 nodes.
end note
@enduml
+11
View File
@@ -176,4 +176,15 @@ note bottom of SH
Both share JWT signing secret
end note
note left of JWT
**Console Proxy Token Minting**
When proxying requests to server nodes:
1. Console AuthMiddleware validates user JWT (aud: turnstone-console)
2. Proxy mints new JWT (aud: turnstone-server)
with real user_id, scopes, permissions
3. src: "console-proxy" for audit traceability
4. 5-minute expiry (fresh per request)
5. Fallback: ServiceTokenManager if no user context
end note
@enduml
@@ -33,7 +33,7 @@ package "Core Modules" as core #181825 {
}
package "Session Runtime" as runtime #181825 {
rectangle "load_skill tool\nsession.py" as loadtool
rectangle "skill tool\nsession.py" as loadtool
rectangle "set_skill()\nsession.py" as setskill
rectangle "_load_skills()\nsession.py" as loadskills
}
@@ -80,6 +80,7 @@ importui --> install : POST (github source)
' Annotations
note right of parser
YAML frontmatter -> ParsedSkill
allowed-tools (standard) -> allowed_tools (internal)
Anthropic + Hermes tag formats
Name validation (lowercase+hyphens)
end note
+2 -2
View File
@@ -1,3 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:efcc7cbe8161a54b5ec24bdfd47e8a142f70029e6e66c707e811b99369f85ebf
size 310079
oid sha256:2b3ea69f852e93dc1bc7943db0c71d0cdd1afcbcf470a8737189ae56e85b3206
size 310075
+2 -2
View File
@@ -1,3 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:0e605963c649574c7bf987b2d338257c78bc1fc68b524ac8276bb25365035e06
size 594096
oid sha256:6471e611beebf647f3a191eb16588571a404cc52a43067883a2b6f06dd936376
size 594676
+2 -2
View File
@@ -1,3 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:da9d32000e3d92d92ce621661ced60f276f9b5be652f5ed6123b400505415f4a
size 319702
oid sha256:3aa8d972bba40d78152f9f0c762b9f5ec616d8052c45fa52b7dd1c679ed81d61
size 325245
+2 -2
View File
@@ -1,3 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:43844b07d36beb04db871f6795a3f3be17852a6a484fdc0ea207403bd7f512a6
size 274286
oid sha256:674712a0563f51837383184652efeb28b7bec13378be636e89d2959bfba39d1e
size 281519
+2 -2
View File
@@ -1,3 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:2636e2d4d2f84f26f93de6e984b780f6ad5bf8d3618a0202ea0a2ee80859c5b5
size 312409
oid sha256:d831c5e10266f0232262b6a29b0ea8e45b3cc63df75f716a80f18462b4f85e66
size 319125
+2 -2
View File
@@ -1,3 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:7896c6e041b6dbb89d034468fa980c8fe645df5eb969d45ef966ccc6399edac2
size 200083
oid sha256:7bf27afa267d5b8d6da38e83213ed1b8e87639d5105a0a1ccc2e5a4bf4d3b67e
size 185282
+2 -2
View File
@@ -1,3 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:a74b4b8b5dbfb1a51a01100b731477968942b01218bad9451a3d5a9cb3003294
size 411665
oid sha256:e3f1ad0fcd55eaca3b8ad9c5abc07432803641c54ede9fc93c79df144cf77d1c
size 407761
+2 -2
View File
@@ -1,3 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:84524f4bc900708ac8adf081591d336f862830188eb8505e71a0f071b339d923
size 252599
oid sha256:09065fef028d05e6df425fd8abefaf5a2ca04b66802f2e3975f289fa597f63ed
size 309656
@@ -1,3 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:c94556889abb382cd5b818639fc0a4706beef3d9c7a0b4cbedc763943d657dd0
size 244998
oid sha256:b047cdc318c505f0f0895a65e14c5cc7552716053055cca57fa0a77db150e618
size 255458
+5
View File
@@ -109,9 +109,12 @@ All configuration is via environment variables in `.env` (copy from `.env.exampl
|----------|---------|-------------|
| `TURNSTONE_DB_BACKEND` | `sqlite` | Storage backend: `sqlite` or `postgresql` |
| `TURNSTONE_DB_URL` | — | Database URL (e.g. `postgresql://user:pass@db:5432/turnstone`). For SQLite, defaults to `/data/.turnstone.db` |
| `TURNSTONE_DB_POOL_SIZE` | `2` | PostgreSQL connection pool size per process (default: 2 base + 3 overflow = 5 max) |
The database stores workstream history, user accounts, and API tokens. When using JWT auth, a database backend is required for user storage.
> **Large clusters:** Each turnstone process maintains a small connection pool (5 max). At hundreds of nodes this adds up — use [PgBouncer](pgbouncer.md) in transaction pooling mode between turnstone and PostgreSQL.
> **First-time setup:** After deploying with auth enabled, create an initial admin user by running `turnstone-admin create-user` inside the container:
>
> ```bash
@@ -151,6 +154,8 @@ POSTGRES_PASSWORD=secret docker compose --profile cluster up
The default `server` and `bridge` also run alongside the cluster nodes (11 total). All nodes are accessible via the console dashboard at `:8090`.
For production clusters beyond ~50 nodes, add PgBouncer between turnstone services and PostgreSQL. See [PgBouncer Connection Pooling](pgbouncer.md) for Docker Compose and Helm configuration.
## Volumes
| Volume | Mount | Purpose |
+254 -68
View File
@@ -2,7 +2,8 @@
`turnstone-eval` is the evaluation and prompt optimization system for turnstone. It
runs test cases against the LLM, scores tool call sequences against expected
actions, and optionally uses the model to self-optimize the developer prompt.
actions, and optionally uses a multi-agent pipeline to optimize the developer
prompt and tool descriptions.
Source: `turnstone/eval.py`
@@ -10,15 +11,24 @@ Source: `turnstone/eval.py`
## Overview
The system works in an iterative loop:
The system uses UCB tree search to explore prompt variants:
1. Run each test case N times against the current developer prompt.
2. Score each run by comparing the actual tool call sequence to expected actions.
3. If not all tests pass, use the model to rewrite the prompt based on failures.
4. Repeat until all tests pass or max iterations are reached.
1. Maintain an **evolution tree** of prompt variants, starting from the initial prompt.
2. Each iteration, **UCB1 selects** the most promising node to evaluate.
3. Run each test case N times against the selected prompt.
4. Score each run by comparing the actual tool call sequence to expected actions.
5. If not all tests pass, run a **three-phase optimization pipeline**:
- Phase 1: Analyst diagnoses semantic failure patterns
- Phase 2: Tool optimizer adjusts tool descriptions (when `--optimize-tools`)
- Phase 3: Prompt optimizer proposes a child variant (when not `--optimize-tools`)
6. Add the child to the tree and repeat until all tests pass or max iterations reached.
When optimization is disabled (`--no-optimize`), only step 1 and 2 execute
(a single iteration).
This approach (inspired by [Learning to Self-Evolve](https://arxiv.org/abs/2603.18620))
prevents irrecoverable collapse from bad edits — UCB naturally backtracks to
high-scoring ancestors instead of following a linear chain.
When optimization is disabled (`--no-optimize`), only steps 2-4 execute
(a single iteration evaluating the root node).
---
@@ -65,6 +75,7 @@ Test suites are JSON files with this structure:
| `match_mode` | no | `"ordered_subset"` | How to match actual vs expected actions (see Scoring). |
| `max_turns` | no | `10` | Maximum conversation turns before stopping. |
| `n_runs` | no | suite default or 3 | Per-case override for number of runs. |
| `holdout` | no | `false` | If `true`, this case is evaluated but excluded from optimizer feedback. Used to measure progress without overfitting. |
### Expected Action Specs
@@ -135,6 +146,7 @@ deterministic, non-interactive execution suitable for automated testing.
| Stdout | Normal | Suppressed during execution |
| Tool logging | Display only | Structured `tool_call_log` |
| System prompt | Built-in developer prompt | Overridable via constructor |
| Cancellation | N/A | `_cancelled` event for timeout cleanup |
### NullUI
@@ -156,20 +168,34 @@ def send_headless(
Runs a complete multi-turn conversation:
1. Appends the user message.
2. Calls the model API (non-streaming).
3. If tool calls are returned, executes them (with stdout suppressed) and
2. Checks `_cancelled` event — stops if set (timeout cleanup).
3. Calls the model API (non-streaming).
4. If tool calls are returned, executes them (with stdout suppressed) and
logs each call to `self.tool_call_log`.
4. Repeats up to `max_turns` or until the model responds without tool calls.
5. Returns the tool call log: list of dicts with keys `tool`, `args`,
5. Repeats up to `max_turns` or until the model responds without tool calls.
6. Returns the tool call log: list of dicts with keys `tool`, `args`,
`result` (truncated to 500 chars), and `turn`.
Parallel tool calls are capped at 10 per turn to prevent degenerate repetition.
### Timeout and Cancellation
Each test runs in a `ThreadPoolExecutor(max_workers=1)` with a per-test
timeout (`--test-timeout`). Each attempt gets its own `OpenAI` client with
a matching httpx read timeout. On timeout, three layers of defense prevent
zombie connections:
1. **httpx timeout**: Per-request read timeout aborts the HTTP call and
releases the server slot.
2. **`_cancelled` event**: Prevents the orphan thread from starting new turns.
3. **`run_client.close()`**: Closes the connection pool to abort any
in-flight request.
### Retry Logic
`send_headless()` is called inside `_run_single_test()` with retry logic:
3 attempts with exponential backoff (sleep `2^attempt` seconds) on any
exception. This prevents transient API errors from poisoning eval scores.
exception. `TimeoutError` is re-raised immediately (no retry).
---
@@ -180,68 +206,175 @@ Each test case runs in isolation:
1. A fresh temp directory is created.
2. Setup files are written to the temp directory.
3. The working directory is changed to the temp directory.
4. A new `HeadlessSession` is created with the current developer prompt.
5. `send_headless()` runs the user prompt through the conversation loop.
6. The tool log is scored against expected actions.
7. The temp directory is cleaned up.
4. A per-attempt `OpenAI` client is created with httpx timeout matching `--test-timeout`.
5. A new `HeadlessSession` is created with the current developer prompt.
6. `send_headless()` runs the user prompt through the conversation loop.
7. The tool log is scored against expected actions.
8. The temp directory is cleaned up.
The memory database is also isolated per test (an ephemeral SQLite database
in the temp directory) so tests do not pollute each other or the user's
real memory store.
### Parallel Execution
With `--parallel N` (N > 1), tests run in a `ProcessPoolExecutor` with N
workers. Each subprocess creates its own `OpenAI` client. This is suitable
for remote API endpoints but will overwhelm local inference servers. The
default (`--parallel 1`) runs tests serially.
---
## Optimization Loop
## Model Roles
`run_optimization()` is the main entry point for iterative prompt optimization.
The eval pipeline uses up to five separate model roles, each independently
configurable. All roles inherit from the test model by default, with a
cascade chain:
```
test model (--base-url, --model)
└─ optimizer (--optimizer-*)
├─ observer (--observer-*)
├─ analyst (--analyst-*)
├─ diversifier (--diversifier-*)
└─ tool optimizer (--tool-optimizer-*)
```
| Role | Purpose | When it runs |
|------|---------|--------------|
| **Test** | The model being evaluated | Every iteration |
| **Analyst** | Diagnoses semantic failure patterns with tool use | When pass rate < 100% |
| **Optimizer** | Rewrites the developer prompt | Every iteration (unless `--optimize-tools`) |
| **Tool optimizer** | Rewrites tool descriptions | When `--optimize-tools` is set |
| **Observer** | Tunes the optimizer's strategy | Every 3 iterations |
| **Diversifier** | Generates prompt paraphrases | Once before the loop (when `--diversify N`) |
Typical setup: local model for test, Opus for analyst, Sonnet for
optimizer/observer/diversifier.
---
## Optimization Pipeline
### Flow
```
for iteration in 0..max_iterations:
1. Run all test cases n_runs times with current prompt
2. Score and aggregate results
3. Save intermediate results to JSON
4. If all tests pass -> stop
5. Every 3 iterations (at iteration 2, 5, 8, ...):
-> Observer reviews optimizer strategy
-> Reset prompt to best-performing iteration
6. Propose new prompt via optimizer model call
7. If prompt unchanged -> stop
8. Continue with new prompt
1. UCB select → pick the most promising tree node
2. Run all test cases n_runs times with selected node's prompt
3. Update node score (rolling mean) and visit count
4. Save intermediate results + tree state to JSON
5. If all tests pass → stop
6. Phase 1: Analyst diagnoses semantic failure patterns
7. Phase 2 (--optimize-tools only): Tool optimizer adjusts descriptions
8. Phase 3 (default only): Prompt optimizer proposes new prompt
9. Every 3 iterations: Observer tunes the optimizer's strategy
10. Add child node to tree (if prompt or tools changed)
```
### Prompt Proposal (`_propose_prompt_modification`)
### Phase 1: Analyst (`_run_analyst`)
Uses the model to rewrite the developer prompt based on test results:
A multi-turn agent with `math` (Python) and `bash` tools for computing
statistics. It receives per-case results with failure classifications and
produces a structured diagnosis:
- **Input**: Current prompt, test case definitions, per-case results with
actual vs expected tool sequences, and a history of the last 3 iterations.
- **Optimizer system prompt** (`OPTIMIZER_SYSTEM`): Instructs the model to
act as a text rewriter. Key guidance includes:
- Address critical failure modes (text-only responses, write_file vs edit_file,
unnecessary search before create, missing plan calls).
- Preserve phrasing that drives 100% pass rate on passing tests.
- Use direct imperative style with concrete tool call examples.
- Stay within 130% of original prompt length.
- **Output**: The rewritten prompt text (stripped of reasoning tags and code fences).
- **Failure patterns**: Shared root causes across failing cases
- **Success/failure contrast**: What distinguishes passing from failing cases
- **Consistency signals**: Systematic (0%), flaky (1-79%), marginal (80-99%)
- **Recommended fixes**: Priority-ordered patterns/examples to add or adjust
### Observer System (`_observe_and_update_optimizer`)
The analyst is instructed to frame fixes as patterns and examples, not
imperative rules — this feeds cleaner signal to the optimizer.
Every 3 iterations, a meta-level "observer" reviews the optimizer's strategy:
In `--optimize-tools` mode, the analyst receives the current tool descriptions
(with any overrides applied) and focuses on tool confusion and description
issues rather than system prompt patterns.
- Analyzes the iteration history: score trends, regressions, prompt length changes,
### Phase 2: Tool Optimizer (`_propose_tool_overrides`)
Runs when `--optimize-tools` is set. Receives the current tool descriptions,
confusion failures (where the model picked the wrong tool), and the analyst's
diagnosis. Returns a JSON override dict that modifies tool descriptions.
Overrides are validated against known tool names — only `description` and
`parameters` changes are accepted (no tool renaming at eval time).
After each iteration, changed descriptions are logged as old → new diffs
for easy visual inspection.
### Phase 3: Prompt Optimizer (`_propose_prompt_modification`)
Skipped in `--optimize-tools` mode. Receives the current prompt, test
results with per-case pass rates and deltas from the parent node, and the
analyst's diagnosis. Returns a rewritten prompt.
The optimizer is instructed to prefer patterns over rules — concrete tool
chain examples teach better than imperative directives like "ALWAYS" or
"NEVER." If the current prompt contains rule-heavy language, the optimizer
is guided to replace it with examples.
### Two Optimization Surfaces
The system supports alternating between two optimization surfaces:
1. **System prompt optimization** (default): Freeze tool descriptions,
optimize the developer prompt. Run until scores plateau.
2. **Tool description optimization** (`--optimize-tools`): Freeze the system
prompt, optimize tool descriptions only. Run until scores plateau.
Each surface lifts the floor for the other — tool description improvements
may unlock system prompt gains that weren't reachable before, and vice versa.
### Observer (`_observe_and_update_optimizer`)
Every 3 iterations, a meta-level observer reviews the optimizer's strategy:
- Analyzes iteration history: score trends, regressions, prompt length changes,
and diffs between iterations.
- Summarizes the optimizer's behavioral patterns (list style, header usage, length).
- Uses `OBSERVER_SYSTEM` to rewrite the optimizer's own system prompt.
- Detects whether the optimizer is producing rule-heavy or pattern-based output.
- Rewrites the optimizer's own system prompt to correct course.
- Rejects degenerate outputs (over 200% of input length).
- After updating the optimizer prompt, resets the developer prompt to the
best-performing iteration so far.
This two-level optimization (optimizer + observer) helps the system escape
local minima and adjust its rewriting strategy.
### Prompt Diversification
### Result Persistence
When `--diversify N` is set, the diversifier generates N paraphrased variants
of each test case's user prompt before the optimization loop. Each run cycles
through variants (round-robin), testing robustness across phrasings.
Variants can be cached back to the test suite JSON with `--save-variants`,
and auto-loaded on subsequent runs even without `--diversify`.
---
## Evolution Tree
The optimization maintains a tree of prompt variants (`EvolutionNode`), where
each node stores its prompt text, tool overrides, aggregated score, and visit
count. The root node (ID 0) contains the initial prompt.
**UCB1 selection**: Each iteration picks the node with the highest Upper
Confidence Bound score: `R_bar + C * sqrt(ln(N) / v)`, where `R_bar` is the
node's mean score, `N` is total visits across all nodes, `v` is the node's
visit count, and `C` is the exploration constant (`--explore-constant`,
default sqrt(2)). Unvisited nodes are always selected first.
### Holdout Cases
Test cases with `"holdout": true` are evaluated every iteration but excluded
from the optimizer's feedback. This prevents the optimizer from overfitting
to specific test cases. Node scores are computed from holdout cases only
(when present). If fewer than 2 non-holdout cases remain, holdout is disabled.
### Improvement-Based Feedback
The optimizer sees delta scores (`delta=+20%`) alongside absolute pass rates,
showing how each case improved relative to the parent node's evaluation. This
provides a cleaner signal than absolute scores alone — the optimizer can
distinguish beneficial edits from harmful ones regardless of starting point.
---
## Result Persistence
After each iteration, results are written to the output JSON file. The
structure is:
@@ -251,9 +384,15 @@ structure is:
"meta": {
"model": "model-name",
"base_url": "http://localhost:8000/v1",
"optimizer_model": "claude-opus-4-6",
"observer_model": "claude-opus-4-6",
"started": "2025-01-01T00:00:00",
"test_suite": "tests.json",
"n_runs_default": 3
"n_runs_default": 3,
"explore_constant": 1.414,
"holdout_ids": [],
"diversify": 10,
"prompt_variants": {"case_id": ["variant1", "variant2"]}
},
"iterations": [
{
@@ -261,7 +400,11 @@ structure is:
"prompt": "the developer prompt used",
"prompt_diff": null,
"optimizer_system": "the optimizer system prompt",
"analyst": "analyst diagnosis output",
"tool_overrides": {"bash": {"description": "..."}},
"timestamp": "2025-01-01T00:01:00",
"tree_node_id": 0,
"tree_child_id": 1,
"cases": {
"test_name": {
"runs": [
@@ -287,9 +430,21 @@ structure is:
"overall_pass_rate": 0.8,
"overall_avg_score": 0.87,
"json_dumps": 0,
"per_case_pass_rates": {"test_name": 1.0, ...}
"per_case_pass_rates": {"test_name": 1.0}
}
}
],
"tree": [
{
"node_id": 0,
"parent_id": null,
"prompt": "initial prompt",
"tool_overrides": {},
"score": 0.85,
"visit_count": 3,
"children": [1, 2],
"iteration": 0
}
]
}
```
@@ -306,26 +461,57 @@ turnstone-eval tests.json # evaluate + optimize
turnstone-eval tests.json --no-optimize # evaluate only (single iteration)
turnstone-eval tests.json --n-runs 5 --max-iter 10 # more thorough evaluation
turnstone-eval tests.json --prompt custom.txt # start from a custom prompt
turnstone-eval tests.json --optimize-tools # optimize tool descriptions only
turnstone-eval tests.json --diversify 10 # test with prompt variants
turnstone-eval tests.json -v # verbose per-turn logging
```
### Multi-model setup (local test model, cloud optimizer)
```
turnstone-eval tests.json \
--base-url http://localhost:8000/v1 \
--optimizer-base-url https://api.anthropic.com \
--optimizer-model claude-sonnet-4-6 \
--analyst-model claude-opus-4-6
```
### All Options
| Flag | Default | Description |
|---------------------|-------------------------------|-------------|
| `test_file` | (positional, required) | Path to test cases JSON file. |
| `--base-url` | `http://localhost:8000/v1` | API base URL. |
| `--model` | auto-detect | Model name. Auto-detected from the API if not specified. |
| `--prompt` | turnstone built-in prompt | Path to initial prompt text file. |
| `--n-runs` | from tests.json or 3 | Number of runs per test case. |
| `--max-iter` | 5 | Maximum optimization iterations. |
| `--no-optimize` | false | Run evaluation only (sets max-iter to 1). |
| `--temperature` | 0.7 | Sampling temperature. |
| `--max-tokens` | 32768 | Max completion tokens. |
| `--reasoning-effort` | `medium` | Reasoning effort: `low`, `medium`, or `high`. |
| `--context-window` | 131072 | Context window size. |
| `--output` | `eval_results.json` | Output results file path. |
| `-v`, `--verbose` | false | Show detailed per-turn logging (API calls, tool args, results). |
| Flag | Default | Description |
|-------------------------|----------------------------|-------------|
| `test_file` | (positional, required) | Path to test cases JSON file. |
| `--base-url` | `http://localhost:8000/v1` | API base URL for the test model. |
| `--model` | auto-detect | Model name. Auto-detected from the API if not specified. |
| `--prompt` | turnstone built-in prompt | Path to initial prompt text file. |
| `--n-runs` | from tests.json or 3 | Number of runs per test case. |
| `--max-iter` | 5 | Maximum optimization iterations. |
| `--no-optimize` | false | Run evaluation only (sets max-iter to 1). |
| `--temperature` | 0.7 | Sampling temperature. |
| `--max-tokens` | 32768 | Max completion tokens. |
| `--reasoning-effort` | `medium` | Reasoning effort: `low`, `medium`, or `high`. |
| `--context-window` | 131072 | Context window size. |
| `--output` | `eval_results.json` | Output results file path. |
| `-v`, `--verbose` | false | Show detailed per-turn logging. |
| `--explore-constant` | 1.414 (sqrt(2)) | UCB exploration constant C. |
| `--test-timeout` | 300 | Per-test timeout in seconds. |
| `--suite-timeout` | 0 (unlimited) | Total suite timeout in seconds. |
| `--no-fast-fail` | false | Disable early termination on all-zero initial runs. |
| `--parallel` | 1 (serial) | Parallel workers (0=auto, N=use N workers). |
| `--optimizer-model` | same as `--model` | Model for prompt optimization. |
| `--optimizer-base-url` | same as `--base-url` | Base URL for optimizer model. |
| `--observer-model` | same as optimizer | Model for meta-optimization (observer). |
| `--observer-base-url` | same as optimizer | Base URL for observer model. |
| `--analyst-model` | same as optimizer | Model for failure analysis. |
| `--analyst-base-url` | same as optimizer | Base URL for analyst model. |
| `--diversify` | 0 (disabled) | Generate N prompt variants per test case. |
| `--diversifier-model` | same as optimizer | Model for prompt diversification. |
| `--diversifier-base-url`| same as optimizer | Base URL for diversifier model. |
| `--save-variants` | false | Save generated variants back to test suite JSON. |
| `--optimize-tools` | false | Optimize tool descriptions only (freeze system prompt). |
| `--tool-optimizer-model` | same as optimizer | Model for tool description optimization. |
| `--tool-optimizer-base-url` | same as optimizer | Base URL for tool optimizer model. |
| `--save-tools` | false | Write optimized tool descriptions back to `turnstone/tools/*.json`. |
### Precedence for n_runs
+20 -2
View File
@@ -70,7 +70,7 @@ etc.) since workstream templates were merged into the skills system in v0.8.0.
`{{node_id}}` (server node ID). Unrecognized placeholders are kept as-is.
- **Runtime switching**: `/template <name>` to switch, `/template clear` to revert
to defaults, `/template` to show current. Persisted across resume.
- **Model-driven loading**: The `load_skill` built-in tool lets the model
- **Model-driven loading**: The `skill` built-in tool lets the model
discover and activate skills mid-conversation. `search` action finds skills
by query (auto-approved); `load` action activates by name (requires user
approval since it changes session behavior). Main session only.
@@ -83,11 +83,15 @@ etc.) since workstream templates were merged into the skills system in v0.8.0.
precedence on name collision. MCP-synced content updates reset `is_default` to
prevent compromised servers from injecting defaults. Admin UI shows origin badge
and disables edit/delete for MCP-sourced skills.
- **Spec fields**: Skills support the full Agent Skills standard frontmatter:
`name`, `description`, `license`, `compatibility`, `metadata` (author, version),
`allowed-tools`. The `license` and `compatibility` fields are preserved on import
and editable in the admin UI. See https://agentskills.io/specification.
- **Security scanning**: Skills are automatically scanned at creation and update
time. The scanner evaluates four risk axes: content risk (command execution,
data exfiltration), supply chain risk (pipe-to-shell, transitive installs),
vulnerability risk (prompt injection, insecure credentials), and declared
capability risk (from `allowed_tools`). Results populate the `scan_status`
capability risk (from `allowed-tools` in SKILL.md). Results populate the `scan_status`
(safe/low/medium/high/critical) and `scan_report` (JSON breakdown) columns.
These fields are system-managed and cannot be overwritten via the admin API.
- **Discovery**: External skills can be discovered and installed from registries:
@@ -100,6 +104,20 @@ etc.) since workstream templates were merged into the skills system in v0.8.0.
Discovery view has search bar, result cards, and "Import from GitHub" modal.
- SDK: `discover_skills(q)` and `install_skill(source, skill_id=..., url=...)`
on both Python and TypeScript console clients.
- **Runtime config on installed skills**: Installed (readonly) skills can have
their runtime configuration edited — model, temperature, reasoning effort,
token budget, max tokens, agent max turns, auto-approve, allowed tools,
and enabled flag. The server restricts updates to these fields only via
`_SKILL_RUNTIME_CONFIG_FIELDS` filtering; spec/content fields (name,
description, tags, license, compatibility, content, activation) remain
immutable. The admin UI shows "Save Config" instead of "Save" for these
skills. Audit action: `skill.update.config`.
- **Admin UI**: Create/Edit skill modals use a two-column spec manifest layout
(left: Identity / Manifest / Deployment; right: Skill Content editor with
monospace font). Runtime Config is a collapsible 3-column grid below.
License uses an SPDX identifier dropdown (MIT, Apache-2.0, GPL-3.0, etc.).
Installed skills show a cyan origin badge with source URL, spec fields are
disabled, and all collapsible sections auto-expand in view mode.
### Usage Tracking
+2 -2
View File
@@ -299,7 +299,7 @@ four independent risk axes:
obfuscation, download-execute chains, executable URLs from untrusted domains
3. **Vulnerability risk** — prompt injection patterns, insecure credential
handling, third-party content exposure (indirect prompt injection surface)
4. **Declared capability risk** — parsed from the skill's `allowed_tools` field.
4. **Declared capability risk** — parsed from `allowed-tools` in the skill's SKILL.md.
`Bash(*)` (unrestricted shell) is high risk. `Bash(git:*)` is low.
Read-only tools are safe.
@@ -338,7 +338,7 @@ from the output before it enters the conversation.
| Priority | Category | Risk | Examples |
|----------|----------|------|----------|
| 1 | Prompt injection | high | Override phrases, role injection (`{"role":"system"}`), instruction override markers |
| 2 | Credential leakage | high | API keys, private key blocks, connection strings, `.env` format secrets |
| 2 | Credential leakage | high | API keys, private key blocks, connection strings, `.env` format secrets, JSON secrets (`"api_key": "..."`, `"password": "..."`, etc.) |
| 3 | Encoded payloads | medium | Script data URIs, hex shellcode sequences |
| 4 | Adversarial URLs | medium | Cloud metadata endpoints, credential-bearing query parameters |
| 5 | System info disclosure | low | Private IP addresses, sensitive file paths |
+288
View File
@@ -0,0 +1,288 @@
# OpenShell Sandbox Integration
Turnstone can run inside an [OpenShell](https://github.com/NVIDIA/OpenShell)
sandbox for kernel-enforced security boundaries around tool execution. OpenShell
provides four layers of defense that Turnstone's application-level safety model
does not cover:
| Layer | Mechanism | What it prevents |
|-------|-----------|------------------|
| Filesystem | Landlock | Writes to `/etc`, `~/.ssh`, system paths |
| Network | Network namespace + seccomp + HTTP CONNECT proxy | Connections to unlisted hosts |
| Process | `setuid` drop + verification | Privilege escalation to root |
| Credentials | Proxy-level secret resolution | API keys in sandbox memory |
Turnstone's own safety layers (human approval, intent judge, tool policies,
output guard) remain active inside the sandbox and handle threats at the semantic
level -- what the LLM *means* to do with its legitimate access.
> See also: [Security and Authentication](security.md),
> [Intent Validation](judge.md), [Governance](governance.md)
---
## Quick Start
```bash
# Run turnstone-server in an OpenShell sandbox
openshell sandbox run \
--policy deploy/openshell/turnstone-policy.yaml \
--workdir /path/to/project \
-- python3 -m turnstone.server --host 0.0.0.0 --port 8080
```
With inference routing (API keys never enter the sandbox):
```bash
openshell sandbox run \
--policy deploy/openshell/turnstone-policy.yaml \
--inference-routes deploy/openshell/routes.yaml \
--workdir /path/to/project \
-- python3 -m turnstone.server --host 0.0.0.0 --port 8080 \
--base-url https://inference.local
```
The `inference.local` hostname is intercepted by the OpenShell proxy before
network policy evaluation -- no network policy entry is needed for it.
---
## Policy Files
### `deploy/openshell/turnstone-policy.yaml`
The main sandbox policy. Covers filesystem, process, and network rules.
### `deploy/openshell/routes.yaml`
Inference routing configuration. Maps `inference.local` to real LLM API
backends. Uncomment and configure the provider(s) you use.
---
## Filesystem Policy
The policy uses Landlock (Linux 5.13+) for kernel-enforced filesystem access
control. Paths are locked at sandbox creation and cannot be changed at runtime.
| Path | Access | Purpose |
|------|--------|---------|
| `--workdir` | read-write | Project files (auto-added via `include_workdir`) |
| `/tmp` | read-write | Bash tool temp scripts, eval workdirs |
| `/dev/null` | read-write | Shell redirections (`2>/dev/null`) |
| `/var/log` | read-write | Log files |
| `/usr`, `/lib`, `/lib64` | read-only | Python runtime, installed packages |
| `/etc` | read-only | System config, SSL certificates |
| `/proc`, `/dev/urandom` | read-only | Process info, entropy |
| `~/.config/turnstone` | read-only | Config file (writes go to database) |
Landlock runs in `best_effort` mode by default -- degrades gracefully on kernels
without Landlock support. Set `compatibility: hard_requirement` for production
hardened deployments.
---
## Network Policy
Default-deny. Only explicitly listed host:port pairs are reachable. All child
processes (MCP servers, bash commands, grep) inherit the network namespace and
cannot bypass the proxy.
### Included endpoints
| Policy | Hosts | Purpose |
|--------|-------|---------|
| `openai_api` | `api.openai.com` | OpenAI LLM API |
| `anthropic_api` | `api.anthropic.com` | Anthropic LLM API |
| `tavily_api` | `api.tavily.com` | Web search fallback |
| `skills_registry` | `skills.sh` | Skill discovery |
| `github_api` | `api.github.com` (read-only L7), `raw.githubusercontent.com` | Skill fetch, GitHub API |
| `mcp_registry` | `registry.modelcontextprotocol.io` (read-only L7) | MCP server discovery |
| `redis` | `127.0.0.1:6379` | Message queue |
| `web_fetch_common` | readthedocs, python docs, GitHub Pages, PyPI, npm, Stack Overflow, Wikipedia | Curated web_fetch domains |
| `bash_network_tools` | Same as `web_fetch_common` | curl/wget from bash tool |
| `package_registries` | `pypi.org`, `files.pythonhosted.org` | pip/uv package installs |
| `git_operations` | `github.com`, `gitlab.com` (L7: clone/fetch only, no push) | Git read-only operations |
### L7 enforcement
Endpoints marked with `protocol: rest` and `tls: terminate` get HTTP-level
inspection. The proxy TLS-terminates using an ephemeral per-sandbox CA, parses
each request, and evaluates method + path against the rules.
The `github_api`, `mcp_registry`, and `git_operations` policies use L7
enforcement:
- **GitHub API / MCP Registry**: `access: read-only` -- only GET, HEAD, OPTIONS
allowed
- **Git operations**: explicit rules allowing only `info/refs` (GET) and
`git-upload-pack` (POST) -- clone and fetch work, push is blocked
### Commented-out sections
The policy includes commented blocks for optional integrations. Uncomment and
configure as needed:
- **OIDC** -- add your identity provider's hostname
- **Discord** -- `discord.com`, `gateway.discord.gg`, `cdn.discordapp.com`
- **MCP HTTP servers** -- any MCP servers using streamable-http transport
---
## Customizing the Domain Allowlist
The `web_fetch` tool lets the LLM fetch arbitrary public URLs, but OpenShell
cannot allow "all HTTPS" -- bare wildcard hosts are rejected by policy
validation. Instead, the policy ships with a curated set of common reference
domains.
To add domains your workloads need:
```yaml
# In turnstone-policy.yaml, under web_fetch_common.endpoints:
- host: docs.example.com
port: 443
# Also add to bash_network_tools.endpoints if curl/wget should reach it:
- host: docs.example.com
port: 443
```
Wildcard patterns are supported:
- `*.example.com` -- matches one subdomain level (e.g. `api.example.com`)
- `**.example.com` -- matches any depth (e.g. `deep.sub.example.com`)
Unlisted domains return connection errors, which the LLM handles gracefully by
telling the user it cannot reach that site.
---
## Inference Routing
Inference routing keeps real API keys completely outside the sandbox. The
sandbox process only sees opaque placeholder tokens in its environment
(`openshell:resolve:env:ANTHROPIC_API_KEY`). The proxy rewrites these to real
credentials on the wire before forwarding to the upstream API.
### Setup
1. Edit `deploy/openshell/routes.yaml` -- uncomment your provider:
```yaml
routes:
# OpenAI
- name: inference.local
endpoint: https://api.openai.com/v1
model: gpt-5
provider_type: openai
protocols:
- openai_chat_completions
- model_discovery
api_key_env: OPENAI_API_KEY
# Or Anthropic
- name: inference.local
endpoint: https://api.anthropic.com
model: claude-sonnet-4-6
provider_type: anthropic
protocols:
- anthropic_messages
api_key_env: ANTHROPIC_API_KEY
```
2. Start with `--inference-routes` and point turnstone at `inference.local`:
```bash
openshell sandbox run \
--inference-routes deploy/openshell/routes.yaml \
--base-url https://inference.local \
...
```
3. When inference routing is active, the `openai_api` and `anthropic_api`
network policies can be removed from the sandbox policy -- the proxy handles
LLM traffic on a separate code path that bypasses OPA entirely.
### Local model servers
For local servers (vLLM, llama.cpp) with no authentication, omit both
`api_key` and `api_key_env` from the route config. No credential resolution
is needed.
---
## MCP Server Subprocesses
MCP servers using stdio transport are spawned as child processes of turnstone.
They automatically inherit all sandbox constraints:
- **Network namespace** -- kernel-level, cannot be bypassed
- **Landlock filesystem** -- kernel-level, cannot be relaxed
- **Seccomp socket filter** -- kernel-level, inherited on fork
No per-subprocess policy entries are needed for these constraints. However, if
an MCP server makes outbound network requests (through the proxy), its binary
must appear in a `binaries[]` entry for the relevant network policy. The proxy
identifies the requesting process via `/proc/<pid>/exe` (not `argv[0]`, which
is spoofable).
Example for a Python-based MCP server that calls an external API:
```yaml
mcp_external_api:
name: mcp-external
endpoints:
- host: api.example.com
port: 443
binaries:
- path: /usr/bin/python3*
- path: /usr/local/bin/python3*
```
MCP servers using streamable-http transport are remote -- they need a network
policy entry for their host:port but no binary entry (the Python process making
the HTTP call is already covered by the standard `python3*` binary entries).
---
## Security Model: Which Layer Enforces What
```
OpenShell (infrastructure) Turnstone (application)
───────────────────────────── ──────────────────────────────
Filesystem access Landlock kernel enforcement (no enforcement)
Network egress Netns + seccomp + proxy + OPA SSRF check on web_fetch
Credentials Placeholder injection + proxy Output guard redaction
Privilege level setuid drop + verification (no enforcement)
Tool semantics (no visibility) Heuristic + LLM judge
Tool policies (no visibility) fnmatch admin policies
Prompt injection (no visibility) Output guard detection
Human approval (no visibility) Approval gate + "always"
```
OpenShell constrains what the process can physically reach. Turnstone constrains
what the LLM does with its legitimate access. Neither layer is sufficient alone:
- Without OpenShell: a bash command can `curl` secrets to any endpoint, write to
`/etc/crontab`, or read `~/.ssh/id_rsa` -- all gated only by human approval
- Without Turnstone: the LLM can `rm -rf` the entire workdir, run destructive
commands, or consume prompt injection payloads -- all within the sandbox's
allowed scope
---
## Hardening Checklist
For production deployments:
- [ ] Set `landlock.compatibility: hard_requirement`
- [ ] Enable inference routing (removes API keys from sandbox)
- [ ] Remove `openai_api`/`anthropic_api` network policies when using inference
routing (traffic goes through the router, not direct)
- [ ] Review and trim `web_fetch_common` domains to your actual needs
- [ ] Remove `package_registries` policy if pip/uv installs are not needed
- [ ] Add your OIDC provider endpoint if using SSO
- [ ] Set Redis `allowed_ips` to your actual Redis host if not localhost
- [ ] Consider removing `bash_network_tools` entirely if bash should not have
network access
+200
View File
@@ -0,0 +1,200 @@
# PgBouncer Connection Pooling
Turnstone cluster deployments share a single PostgreSQL instance across
all server nodes, bridge processes, and the console. Each process
maintains a small connection pool (2 base + 3 overflow = 5 max). At
scale this adds up — a 100-node cluster opens up to 500 connections,
and a 1000-node cluster up to 5,000.
PostgreSQL's default `max_connections` is 100, and each real connection
allocates ~510 MB of backend memory. PgBouncer sits between turnstone
and PostgreSQL, multiplexing thousands of lightweight client connections
down to a small number of real database connections.
---
## Why PgBouncer works well with turnstone
All turnstone database operations are short-burst queries: acquire a
connection, execute 13 statements, commit, release. No operation holds
a connection for more than a few milliseconds. This makes **transaction
pooling mode** ideal — PgBouncer assigns a real connection only for the
duration of each transaction, then returns it to the pool.
| Cluster size | Client connections (max) | PgBouncer server connections needed |
|--------------|------------------------|-------------------------------------|
| 10 nodes | 50 | 1020 |
| 100 nodes | 500 | 2040 |
| 500 nodes | 2,500 | 3060 |
| 1,000 nodes | 5,000 | 4080 |
The server connection count stays low because most client connections
are idle at any given moment.
---
## Docker Compose
Add PgBouncer between turnstone services and PostgreSQL:
```yaml
services:
pgbouncer:
image: bitnami/pgbouncer:latest
environment:
POSTGRESQL_HOST: postgres
POSTGRESQL_PORT: "5432"
POSTGRESQL_DATABASE: turnstone
POSTGRESQL_USERNAME: ${POSTGRES_USER:-turnstone}
POSTGRESQL_PASSWORD: ${POSTGRES_PASSWORD:?}
PGBOUNCER_POOL_MODE: transaction
PGBOUNCER_DEFAULT_POOL_SIZE: "40"
PGBOUNCER_MAX_CLIENT_CONN: "5000"
PGBOUNCER_MAX_DB_CONNECTIONS: "80"
PGBOUNCER_SERVER_IDLE_TIMEOUT: "300"
ports:
- "6432:6432"
networks:
- turnstone-net
depends_on:
postgres:
condition: service_healthy
healthcheck:
test: ["CMD", "pg_isready", "-h", "127.0.0.1", "-p", "6432"]
interval: 5s
timeout: 3s
retries: 5
```
Then point turnstone services at PgBouncer instead of PostgreSQL
directly by changing the `DATABASE_URL` (or `TURNSTONE_DB_URL`):
```bash
# Before (direct)
TURNSTONE_DB_URL=postgresql://turnstone:secret@postgres:5432/turnstone
# After (via PgBouncer)
TURNSTONE_DB_URL=postgresql://turnstone:secret@pgbouncer:6432/turnstone
```
---
## Helm / Kubernetes
Add a PgBouncer deployment or use a Helm chart like
[bitnami/pgbouncer](https://github.com/bitnami/charts/tree/main/bitnami/pgbouncer).
In `values.yaml`, point the database at PgBouncer:
```yaml
database:
backend: postgresql
external:
host: pgbouncer
port: 6432
database: turnstone
username: turnstone
existingSecret: turnstone-db-secret
```
PgBouncer configuration:
```yaml
pgbouncer:
poolMode: transaction
defaultPoolSize: 40
maxClientConn: 5000
maxDbConnections: 80
```
---
## Configuration reference
| PgBouncer setting | Recommended | Notes |
|-------------------|-------------|-------|
| `pool_mode` | `transaction` | Required — turnstone uses short-burst queries with no session state |
| `default_pool_size` | 40 | Real PostgreSQL connections per database. Start here, increase if you see `no more connections allowed` |
| `max_client_conn` | 5000 | Upper bound on client connections. Set to `cluster_nodes × 5` |
| `max_db_connections` | 80 | Hard cap on real connections to PostgreSQL. Keep below PG `max_connections` minus headroom for admin/monitoring |
| `server_idle_timeout` | 300 | Close idle server connections after 5 minutes |
| `server_lifetime` | 3600 | Recycle server connections after 1 hour |
On the PostgreSQL side:
| PostgreSQL setting | Recommended | Notes |
|--------------------|-------------|-------|
| `max_connections` | 100 | Default is fine — PgBouncer is the only client. Set higher than `max_db_connections` to leave room for admin connections |
| `shared_buffers` | 25% of RAM | Standard PostgreSQL tuning |
---
## Turnstone pool settings
Each turnstone process maintains its own SQLAlchemy connection pool to
PgBouncer (which then multiplexes to PostgreSQL):
| Environment variable | Default | Description |
|---------------------|---------|-------------|
| `TURNSTONE_DB_POOL_SIZE` | 2 | Base pool size per process |
| `TURNSTONE_DB_BACKEND` | sqlite | Set to `postgresql` for cluster deployments |
| `TURNSTONE_DB_URL` | — | Connection URL (point at PgBouncer, not PostgreSQL directly) |
The default pool of 2 + 3 overflow = 5 connections per process is
intentionally small to support large clusters. You should not need to
increase this — turnstone's database operations are all short-burst
context-managed queries that hold connections for milliseconds.
SQLAlchemy `pool_pre_ping` is enabled, so stale connections (e.g. after
PgBouncer restarts) are automatically detected and replaced.
---
## Monitoring
PgBouncer exposes stats via its admin console (connect to
PgBouncer port with user `pgbouncer`):
```sql
-- Active and waiting clients
SHOW POOLS;
-- Per-database stats
SHOW STATS;
-- Current client connections
SHOW CLIENTS;
```
Key metrics to watch:
- **`cl_active`** — clients with a server connection assigned. Should be
well below `max_db_connections`.
- **`cl_waiting`** — clients waiting for a server connection. Sustained
non-zero values mean you need more `default_pool_size`.
- **`sv_active`** — active server (PostgreSQL) connections. Should stay
below PostgreSQL `max_connections`.
---
## Troubleshooting
**"no more connections allowed (max_client_conn)"** — PgBouncer is
rejecting new client connections. Increase `max_client_conn` to match
your cluster size × 5.
**"no more connections allowed (max_db_connections)"** — PgBouncer
cannot open more connections to PostgreSQL. Increase
`max_db_connections` and ensure PostgreSQL `max_connections` is higher.
**Connections timing out on startup** — If all nodes start
simultaneously, the burst of initial connections (migrations, health
checks) can temporarily exceed the pool. PgBouncer queues excess
clients by default — this resolves itself within seconds.
**Prepared statements not supported** — PgBouncer in `transaction` mode
does not support prepared statements. Turnstone's SQLAlchemy layer does
not use server-side prepared statements by default, so this is not an
issue.
See also: [Docker deployment](docker.md) · [Security](security.md)
+2 -2
View File
@@ -75,7 +75,7 @@ Both `TurnstoneServer` (sync) and `AsyncTurnstoneServer` (async) expose:
| | `approve(*, ws_id, approved, feedback, always)` | `StatusResponse` |
| | `plan_feedback(*, ws_id, feedback)` | `StatusResponse` |
| | `command(*, ws_id, command)` | `StatusResponse` |
| | `cancel(ws_id)` | `StatusResponse` |
| | `cancel(ws_id, *, force=False)` | `StatusResponse` |
| **Streaming** | `stream_events(ws_id)` | `Iterator[ServerEvent]` |
| | `stream_global_events()` | `Iterator[ServerEvent]` |
| **High-level** | `send_and_wait(message, ws_id, *, timeout, on_event)` | `TurnResult` |
@@ -127,7 +127,7 @@ SSE events are deserialized into typed dataclasses. Use `event.type` to discrimi
| `reasoning` | `ReasoningEvent` | `text` |
| `tool_info` | `ToolInfoEvent` | `items` |
| `approve_request` | `ApproveRequestEvent` | `items` |
| `tool_result` | `ToolResultEvent` | `call_id`, `name`, `output` |
| `tool_result` | `ToolResultEvent` | `call_id`, `name`, `output`, `is_error` |
| `tool_output_chunk` | `ToolOutputChunkEvent` | `call_id`, `chunk` |
| `status` | `StatusEvent` | `prompt_tokens`, `total_tokens`, `pct`, `effort`, `cache_creation_tokens`, `cache_read_tokens` |
| `plan_review` | `PlanReviewEvent` | `content` |
+35 -8
View File
@@ -457,14 +457,30 @@ without any database.
### Proxy auth forwarding
When the console proxies requests to server nodes (via `/node/{id}/...`
routes), it uses a dedicated **service proxy token** with
`aud: turnstone-server` and `write` scope. The user's console JWT
(which has `aud: turnstone-console`) is **not** forwarded — it would be
rejected by the server's audience validation.
routes), it mints a **short-lived user-scoped JWT** with
`aud: turnstone-server` carrying the real user's `user_id`, `scopes`,
and `permissions`. The user's console JWT (which has
`aud: turnstone-console`) is **not** forwarded directly — it would be
rejected by the server's audience validation. Instead, the console
re-signs a new JWT targeted at the server audience.
The proxy token is managed by a `ServiceTokenManager` that auto-rotates
1-hour JWTs, refreshing at 80% of lifetime. If `--auth-token` is
provided, that static token is used instead.
Each proxied request gets a fresh JWT (5-minute expiry). This ensures:
- **Audit attribution** — the upstream server records the real user in
`ctx_user_id` and audit events, not a generic service identity.
- **Scope narrowing** — a read-only console user's proxied request
carries only `read` scope, not the full `{read, write, approve}` set.
The server enforces this as defense in depth.
- **Permission forwarding** — granular RBAC permissions from the
console JWT are carried through to the server.
The JWT `src` claim is set to `"console-proxy"`, allowing servers to
distinguish proxied requests from direct logins in audit logs.
When no user context is available (auth disabled, or internal requests),
the proxy falls back to a `ServiceTokenManager` with service identity
`console-proxy` and full scopes. If `--auth-token` is provided, that
static token is used as a final fallback.
### Service-to-service authentication
@@ -475,7 +491,7 @@ auto-rotating JWTs when communicating with server nodes:
|---------|----------|-------|----------|---------|
| Bridge | `bridge` | `approve` | `turnstone-server` | Tool approval proxy, message relay |
| Console collector | `console-collector` | `read` | `turnstone-server` | Node health polling |
| Console proxy | `console-proxy` | `write` | `turnstone-server` | Proxied API calls |
| Console proxy (fallback) | `console-proxy` | `approve` | `turnstone-server` | Proxied API calls when no user context |
| Channel notify | `system` | `write` | `turnstone-channel` | Notification delivery to channel gateway |
Service tokens use 1-hour expiry with automatic refresh via
@@ -483,6 +499,17 @@ Service tokens use 1-hour expiry with automatic refresh via
httpx event hooks to ensure rotated tokens are picked up on SSE
reconnects.
### User identity in MQ-dispatched workstreams
When the console creates a workstream via MQ (the normal path), the
authenticated user's `user_id` is embedded in the
`CreateWorkstreamMessage`. The bridge forwards this `user_id` in the
HTTP payload when calling the server's `POST /v1/api/workstreams/new`.
The server accepts a `user_id` from the request body **only when the
caller is a trusted service** — identified by `token_source` matching
`bridge`, `console-proxy`, or `console`. Regular API callers cannot
override `user_id`; the server always uses their JWT identity.
Note that the channel gateway uses a distinct JWT audience
(`turnstone-channel`) from the server (`turnstone-server`) and console
(`turnstone-console`). A server-scoped JWT cannot authenticate to the
+5 -3
View File
@@ -51,7 +51,7 @@ connection, Redis, auth secrets, server bind address). These stay in
| Bridge identity | `[bridge]` | config.toml / env |
| Console bind | `[console]` | config.toml / env |
**ConfigStore settings** (~40 settings) are loaded from the database after
**ConfigStore settings** (48 settings) are loaded from the database after
storage initialization:
| Section | Settings |
@@ -60,10 +60,12 @@ storage initialization:
| `session` | instructions, retention_days, compact_max_tokens, auto_compact_pct |
| `tools` | timeout, truncation, agent_max_turns, skip_permissions, search, search_threshold, search_max_results |
| `server` | workstream_idle_timeout, max_workstreams |
| `cluster` | node_fan_out_limit, mcp_max_servers |
| `mcp` | config_path, refresh_interval, registry_url |
| `ratelimit` | enabled, requests_per_second, burst |
| `ratelimit` | enabled, requests_per_second, burst, trusted_proxies |
| `health` | backend_probe_interval, backend_probe_timeout, circuit_breaker_threshold, circuit_breaker_cooldown |
| `judge` | enabled, model, provider, base_url, api_key, confidence_threshold, max_context_ratio, timeout, read_only_tools |
| `judge` | enabled, model, provider, base_url, api_key, confidence_threshold, max_context_ratio, timeout, read_only_tools, output_guard, redact_secrets |
| `skills` | discovery_url |
| `memory` | relevance_k, fetch_limit, max_content, nudge_cooldown, nudges |
Settings are addressed by dotted key (e.g. `memory.relevance_k`). Each has a
+221
View File
@@ -0,0 +1,221 @@
# TLS / mTLS
Turnstone supports end-to-end transport encryption with mutual TLS (mTLS) for
inter-service communication, powered by [lacme](https://pypi.org/project/lacme/).
---
## Quick Start (Docker Compose)
```bash
docker compose -f compose.yaml -f deploy/docker-compose.tls.yml up
```
This:
1. Bootstraps an internal CA and issues certs for Redis/PostgreSQL
2. Starts the console with TLS enabled (internal CA + ACME server)
3. Server nodes auto-provision certs via the console's ACME endpoint
4. All inter-service communication uses mTLS
---
## Architecture
```
Console (CA + ACME Server)
+-- CertificateAuthority (owns root key, signs certs)
+-- ACMEResponder (mounted at /acme, RFC 8555)
+-- GET /acme/ca.pem (root cert for node bootstrapping)
|
| ACME protocol (auto-approve, no challenge validation)
+-----------+-----------+
| | |
Server(s) Bridge Channel GW
(auto-cert (mTLS (mTLS
+ renewal) client) client)
```
**Two cert paths on the console:**
- **Internal cert** (mTLS): Always from the internal CA. Used for cluster
service mesh communication.
- **Frontend cert** (HTTPS): From an external ACME CA (e.g. Let's Encrypt)
if `tls.acme_directory` is set, otherwise self-issued from the internal CA.
---
## Configuration
### Settings (ConfigStore / Admin Settings tab)
| Setting | Default | Description |
|---------|---------|-------------|
| `tls.enabled` | `false` | Master switch for internal mTLS |
| `tls.acme_directory` | `""` | External ACME CA URL for console frontend cert |
### Bootstrap Config (config.toml)
These are needed before storage is available:
```toml
[redis]
tls = false
tls_ca = "" # path to CA cert
tls_cert = "" # path to client cert
tls_key = "" # path to client key
[database]
sslmode = "prefer" # disable, allow, prefer, require, verify-full
sslrootcert = "" # path to CA cert
sslcert = "" # path to client cert
sslkey = "" # path to client key
```
### Hardcoded Defaults
| Parameter | Value | Notes |
|-----------|-------|-------|
| CA common name | "Turnstone CA" | |
| CA validity | 10 years | |
| Cert validity | 48 hours | Short-lived, auto-renewed |
| Renewal interval | 24 hours | Half of validity |
| ACME auto-approve | true | Internal network, no challenge validation |
---
## CLI
### Offline Bootstrap
Create a CA and infrastructure certs without a running console:
```bash
# Bootstrap CA + Redis + PostgreSQL certs
turnstone-admin tls-bootstrap --out /certs --issue redis --issue postgres
# Output:
# /certs/ca.pem (CA root certificate)
# /certs/certs/redis/ (Redis cert + key)
# /certs/certs/postgres/ (PostgreSQL cert + key)
```
The output directory is chmod 0700 (contains the CA private key).
### Online Cert Issuance
Request certs from a running console's ACME endpoint:
```bash
# Download CA root cert (TOFU — verify fingerprint)
turnstone-admin tls-ca-cert --out ca.pem --console-url http://console:8080
# Request a cert for a domain
turnstone-admin tls-issue worker-1.internal --out /certs --console-url http://console:8080
# List issued certs
turnstone-admin tls-list --console-url http://console:8080 --auth-token $TOKEN
```
### Console URL Discovery
If `--console-url` is not provided, the CLI discovers it from the `services`
table in the shared database. The console registers itself on startup.
---
## Admin UI
The **TLS** tab in the console admin panel (System group) shows:
- CA status (common name, certificate count)
- Certificate table (domain, SANs, issued, expires)
- Force-renew and delete actions per certificate
---
## SDK
### Python
```python
from turnstone.sdk import TurnstoneServer
client = TurnstoneServer(
base_url="https://server:8080",
token="tok_xxx",
ca_cert="/path/to/ca.pem",
client_cert="/path/to/cert.pem",
client_key="/path/to/key.pem",
)
```
### TypeScript
```typescript
import { TurnstoneServer } from "@turnstone/sdk";
import { Agent } from "undici";
import * as fs from "fs";
const agent = new Agent({
connect: {
ca: fs.readFileSync("/path/to/ca.pem"),
cert: fs.readFileSync("/path/to/cert.pem"),
key: fs.readFileSync("/path/to/key.pem"),
},
});
const client = new TurnstoneServer({
baseUrl: "https://server:8080",
token: "tok_xxx",
// Node.js 18+ uses undici under the hood
fetch: (url, init) =>
fetch(url, { ...init, dispatcher: agent } as RequestInit),
});
```
---
## How It Works
### Node Bootstrap Flow
1. Node starts, connects to shared database (plain connection)
2. Discovers console URL from `services` table
3. Fetches CA root cert from `http://console/acme/ca.pem` (plain HTTP, TOFU)
4. Requests service cert via ACME protocol (plain HTTP, JWS-signed)
5. Starts auto-renewal (24h interval, re-issues before expiry)
6. All subsequent inter-service communication uses mTLS
### Console Startup Flow
1. Read `tls.enabled` from ConfigStore
2. Initialize CA (load from DB or generate new root key)
3. Mount ACME responder at `/acme` (serves `/ca.pem` natively)
4. Issue console certs (internal + optional frontend)
5. Start CA-direct auto-renewal (no network, signs directly)
6. Register console URL in services table with heartbeat
---
## Troubleshooting
### Cert expired / mTLS connection refused
Certs are valid for 48 hours. If auto-renewal stopped (e.g. console was down),
restart the service to re-request a cert.
### "No console service found"
The console registers itself in the `services` table on startup. If the console
hasn't started or the registration expired (1 hour TTL), nodes can't discover
it. Use `--console-url` explicitly.
### Let's Encrypt for console frontend
Set `tls.acme_directory` to `https://acme-v02.api.letsencrypt.org/directory`
in the admin Settings tab. The console will request a publicly trusted cert
for its HTTPS endpoint. Internal mTLS still uses the private CA.
### Verifying the cert chain
```bash
openssl s_client -connect server:8080 -CAfile ca.pem
```
+54 -21
View File
@@ -1,6 +1,6 @@
# Tools Reference
turnstone exposes 18 built-in tools plus any number of external MCP tools to the
turnstone exposes 19 built-in tools plus any number of external MCP tools to the
LLM via the OpenAI function-calling interface. Built-in tools are defined as JSON
files under `turnstone/tools/` and loaded at startup by `turnstone/core/tools.py`.
MCP tools are discovered from configured MCP servers at startup by
@@ -46,12 +46,12 @@ schema plus turnstone-specific metadata keys:
| Name | Description |
|---------------------|-------------|
| `TOOLS` | All 17 tool definitions (sent to the model). |
| `TOOLS` | All 19 tool definitions (sent to the model). |
| `AGENT_TOOLS` | Tools with `agent: true` -- available to plan sub-agents. Read-only tools. |
| `TASK_AGENT_TOOLS` | Tools with `task_agent: true` -- available to task sub-agents. Includes write operations. |
| `AGENT_AUTO_TOOLS` | Set of tool names with `auto_approve: true` -- no user confirmation needed. |
| `TASK_AUTO_TOOLS` | Same as `AGENT_AUTO_TOOLS` (identical filter). |
| `BUILTIN_TOOL_NAMES`| Frozenset of all 17 built-in tool names. Used by tool search to distinguish always-on tools from deferrable MCP tools. |
| `BUILTIN_TOOL_NAMES`| Frozenset of all 19 built-in tool names. Used by tool search to distinguish always-on tools from deferrable MCP tools. |
| `PRIMARY_KEY_MAP` | Dict mapping tool name to its `primary_key` parameter name. |
---
@@ -69,7 +69,7 @@ Tool execution follows a three-phase pipeline inside `ChatSession._execute_tools
- Parses the JSON arguments (with fallback for malformed JSON).
- If JSON parsing fails entirely, uses `PRIMARY_KEY_MAP` to map a bare string
to the correct parameter.
- Dispatches to the matching `_prepare_{func_name}()` handler. There are 17
- Dispatches to the matching `_prepare_{func_name}()` handler. There are 19
built-in tools plus `tool_search` (synthetic, client-side BM25 fallback) and
the generic `_prepare_mcp_tool()` handler for MCP tools.
- Validates arguments and builds a preview dict containing:
@@ -102,10 +102,15 @@ Each item's `execute` callable is invoked:
- Errored or denied items return their error/denial message without executing.
- The `bash` tool streams stdout incrementally: each line calls
`ui.on_tool_output_chunk(call_id, line)` as it is produced, then the final
combined output (stdout + stderr) is delivered via `ui.on_tool_result(call_id, name, output)`.
combined output (stdout + stderr) is delivered via
`ui.on_tool_result(call_id, name, output, is_error=...)`.
The `call_id` links `tool_info`/`approve_request` items to their streaming chunks and
final result, enabling correct routing when multiple bash tools run in parallel.
Other tools deliver results atomically via `ui.on_tool_result(call_id, name, output)` only.
The `is_error` flag is `True` when the tool execution failed (e.g. bash exit code >= 2
or signal, file not found, timeout). Exit code 1 is ambiguous and not flagged; user
denials are tracked separately. This removes the need for text-prefix heuristics.
Other tools deliver results atomically via
`ui.on_tool_result(call_id, name, output, is_error=...)` only.
- Special post-execution gate for `plan`: the plan output is shown to the user
for review, and the user can reject or annotate it.
@@ -183,8 +188,11 @@ Execute a bash command and return stdout + stderr.
| Parameter | Type | Required | Description |
|-----------|--------|----------|-------------|
| `command` | string | yes | The bash command to execute. |
| `timeout` | integer | no | Timeout in seconds (1-600). Omit to use the global `tools.timeout` setting (typically 120s). |
| `stop_on_error` | boolean | no | Enable `set -e` so the script exits on the first command failure. Default false. |
- **What it does**: Runs the command in a subprocess with a configurable timeout. Commands are sanitized and checked against a blocklist (e.g. `rm -rf /`).
- **What it does**: Runs the command in a subprocess with a configurable timeout. Commands are sanitized and checked against a blocklist (e.g. `rm -rf /`). Environment variables containing secrets are scrubbed (`*_KEY`, `*_SECRET`, `*_TOKEN`, etc.).
- **Output format**: Stdout is returned directly. Stderr lines are prefixed with `[stderr]` so the model can distinguish them. When the command itself redirects stderr to stdout (`2>&1`), no prefix is added. Output exceeding 256KB is truncated (head + tail preserved, middle replaced with a truncation notice).
- **Auto-approve**: No -- requires user confirmation.
- **Agent availability**: `task_agent` only (not available to plan sub-agents).
@@ -216,8 +224,9 @@ Write content to a file, creating it if needed.
|-----------|--------|----------|-------------|
| `path` | string | yes | Absolute or relative file path. |
| `content` | string | yes | The full file content to write. |
| `mode` | string | no | `"overwrite"` (default) replaces the file. `"append"` adds content to the end. |
- **What it does**: Creates or overwrites the file at the given path. Parent directories are created as needed.
- **What it does**: Creates or overwrites (or appends to) the file at the given path. Parent directories are created as needed.
- **Auto-approve**: No -- requires user confirmation.
- **Agent availability**: `task_agent` only.
@@ -225,21 +234,44 @@ Write content to a file, creating it if needed.
### edit_file
Replace an exact string in a file with new content.
Replace exact strings in a file, or apply multiple replacements atomically.
| Parameter | Type | Required | Description |
|--------------|---------|----------|-------------|
| `path` | string | yes | Absolute or relative file path. |
| `old_string` | string | yes | The exact text to find and replace. |
| `new_string` | string | yes | The replacement text. |
| `old_string` | string | no* | The exact text to find and replace. |
| `new_string` | string | no* | The replacement text. |
| `near_line` | integer | no | Disambiguate when `old_string` matches multiple locations. |
| `edits` | array | no* | Multiple replacements to apply atomically (see below). |
| `replace_all` | boolean | no | Replace ALL occurrences of `old_string`. Cannot combine with `near_line` or `edits`. |
- **What it does**: Finds `old_string` in the file and replaces it with `new_string`. Fails if the string is not found or matches multiple locations (unless `near_line` is provided to pick the nearest match). Requires a prior `read_file` call on the same path.
\* Provide either `old_string`+`new_string` (single edit) or `edits` array (batch), not both.
- **What it does**: Finds `old_string` in the file and replaces it with `new_string`. Fails if the string is not found or matches multiple locations (unless `near_line` or `replace_all` is provided). Requires a prior `read_file` or `diff_file` call on the same path.
- **Batch mode**: The `edits` array accepts multiple `{old_string, new_string, near_line?}` entries applied atomically. All edits are validated before any are applied. Overlapping edits (two entries targeting the same text region) are rejected. Edits are applied in reverse file-position order so character offsets stay stable.
- **Replace-all mode**: When `replace_all` is true, all occurrences are replaced via `str.replace()`. The approval preview shows the occurrence count.
- **Auto-approve**: No -- requires user confirmation.
- **Agent availability**: `task_agent` only.
---
### diff_file
Show a unified diff between two files, or between a file and a provided string.
| Parameter | Type | Required | Description |
|-----------------|---------|----------|-------------|
| `path_a` | string | yes | Path to the first file. |
| `path_b` | string | no | Path to the second file. Mutually exclusive with `content_b`. |
| `content_b` | string | no | String content to compare against `path_a`. Mutually exclusive with `path_b`. |
| `context_lines` | integer | no | Number of context lines around changes (default 3, max 20). |
- **What it does**: Returns unified diff output using Python's `difflib`. Binary files (containing null bytes) are rejected with a clear error. Files read through `diff_file` satisfy `edit_file`'s read guard — you can diff then edit without a separate `read_file` call. Large diffs are streamed with early cutoff at the tool truncation limit.
- **Auto-approve**: Yes (read-only).
- **Agent availability**: `agent` and `task_agent`.
---
### search
Search file contents for a regex pattern.
@@ -265,8 +297,9 @@ Execute Python code for math and computation in a sandbox.
|-----------|--------|----------|-------------|
| `code` | string | yes | Python code to execute. Must use `print()` for output. |
- **What it does**: Runs Python code in a sandboxed environment with pre-imported libraries: `sympy`, `numpy`, `scipy`, `math`, `fractions`, `itertools`, `functools`, `collections`, `decimal`, `operator`, `random`, `re`, `string`. Common sympy names (`symbols`, `solve`, `simplify`, `sqrt`, `Matrix`, etc.) are pre-imported.
- **Auto-approve**: No -- requires user confirmation.
- **What it does**: Runs Python code in a sandboxed environment with pre-imported libraries: `sympy`, `numpy`, `scipy`, `math`, `fractions`, `itertools`, `functools`, `collections`, `decimal`, `operator`, `random`, `re`, `string`. Common sympy names (`symbols`, `solve`, `simplify`, `sqrt`, `Matrix`, etc.) are pre-imported. `pytest` is also available for import.
- **Installation**: `sympy`, `numpy`, `scipy`, and `pytest` require the `[sandbox]` extras group: `pip install turnstone[sandbox]` (included in `[all]`).
- **Auto-approve**: Yes.
- **Agent availability**: `agent` and `task_agent`.
---
@@ -492,7 +525,7 @@ data.get("mergedAt") is not None
---
### load_skill
### skill
Discover and activate skills at runtime during a conversation. The model can
search for available skills and load one by name, replacing the current active
@@ -543,7 +576,7 @@ pre-configure skills at workstream creation.
| `watch` | Monitor | No (create) | No | No | `command` |
| `read_resource`| MCP | No | Yes | Yes | `uri` |
| `use_prompt` | MCP | No | Yes | Yes | `name` |
| `load_skill` | Skills | No (load) | No | No | `name` |
| `skill` | Skills | No (load) | No | No | `name` |
| `tool_search`| Search | Yes | No | No | `query` |
---
@@ -591,12 +624,12 @@ CLI flags override the config file:
### How it works
1. **Threshold check**: At session startup, `ToolSearchManager.should_activate()`
counts total tools (built-in + MCP). If the count is below the threshold, tool
search stays off and all tools are sent to the model directly.
1. **Threshold check**: At session startup, if the total tool count (built-in + MCP)
is below the threshold, tool search stays off and all tools are sent to the model
directly.
2. **Partitioning**: When active, tools are split into two sets:
- **Always-on** -- the 17 built-in tools (members of `BUILTIN_TOOL_NAMES`).
- **Always-on** -- the 19 built-in tools (members of `BUILTIN_TOOL_NAMES`).
These are always visible to the model.
- **Deferred** -- all MCP tools. These are not sent in the tool list unless
the model searches for them.
@@ -639,7 +672,7 @@ MCP-compatible service.
3. **Schema conversion**: Each MCP tool's `inputSchema` is converted to OpenAI
function-calling format. The tool name is prefixed: `mcp__{server}__{tool}`.
4. **Merging**: MCP tools are appended after the 17 built-in tools via
4. **Merging**: MCP tools are appended after the 19 built-in tools via
`merge_mcp_tools()`. Built-in tools appear first, giving them natural LLM priority.
When dynamic tool search is active, MCP tools are deferred rather than directly
visible -- the model discovers them via search as needed (see
+14 -3
View File
@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
[project]
name = "turnstone"
version = "0.8.2"
version = "0.9.5"
description = "Multi-node AI orchestration platform with tool use, agent routing, and cluster simulation."
readme = "README.md"
license = "BUSL-1.1"
@@ -51,8 +51,11 @@ console = ["redis>=7.2", "croniter>=3.0"]
sim = ["redis>=7.2"]
anthropic = ["anthropic>=0.39"]
postgres = ["psycopg[binary]>=3.2"]
ddg = ["ddgs>=9.0"]
discord = ["discord.py>=2.4", "redis>=7.2"]
all = ["turnstone[mq,console,sim,anthropic,postgres,discord]"]
tls = ["lacme>=1.0.4"]
sandbox = ["sympy>=1.13", "numpy>=2.0", "scipy>=1.14", "pytest>=9.0"]
all = ["turnstone[mq,console,sim,anthropic,postgres,discord,ddg,tls,sandbox]"]
[project.scripts]
turnstone = "turnstone.cli:main"
@@ -77,7 +80,7 @@ include = [
"turnstone/console/static/*.js",
"turnstone/shared_static/*.css",
"turnstone/shared_static/*.js",
"turnstone/shared_static/katex-0.16.38/**/*",
"turnstone/shared_static/katex-0.16.44/**/*",
"turnstone/shared_static/hljs-11.11.1/**/*",
"turnstone/shared_static/mermaid-11.13.0/**/*",
"turnstone/sdk/py.typed",
@@ -164,6 +167,14 @@ ignore_missing_imports = true
module = ["frontmatter", "frontmatter.*"]
ignore_missing_imports = true
[[tool.mypy.overrides]]
module = ["ddgs", "ddgs.*"]
ignore_missing_imports = true
[[tool.mypy.overrides]]
module = ["lacme", "lacme.*"]
ignore_missing_imports = true
[[tool.mypy.overrides]]
module = ["turnstone.channels.discord.*"]
disallow_subclassing_any = false
+23 -1
View File
@@ -28,9 +28,19 @@ usage() {
LIB="$1"
VERSION="$2"
# Detect current version from pyproject.toml
# Detect current version from the filesystem (not pyproject.toml, which
# Renovate may have already updated). Falls back to pyproject.toml if
# no directory is found.
detect_old_version() {
local pattern="$1"
# Look for existing directory: e.g. turnstone/shared_static/katex-0.16.38
local dir
dir=$(find "${STATIC_DIR}" -maxdepth 1 -type d -name "${pattern}-*" | head -1)
if [[ -n "$dir" ]]; then
basename "$dir" | sed "s/${pattern}-//"
return
fi
# Fallback to pyproject.toml
grep -oE "${pattern}-[0-9.]+" pyproject.toml | head -1 | sed "s/${pattern}-//"
}
@@ -51,9 +61,19 @@ update_refs() {
done
}
check_same_version() {
if [[ "$1" == "$2" ]]; then
echo "ERROR: Old version ($1) == new version ($2). Nothing to update."
echo "If the old directory was already removed, re-download with:"
echo " rm -rf ${STATIC_DIR}/${3}-${1} && $0 $3 $2"
exit 1
fi
}
case "$LIB" in
katex)
OLD_VERSION=$(detect_old_version "katex")
check_same_version "$OLD_VERSION" "$VERSION" "katex"
OLD_DIR="${STATIC_DIR}/katex-${OLD_VERSION}"
NEW_DIR="${STATIC_DIR}/katex-${VERSION}"
@@ -87,6 +107,7 @@ case "$LIB" in
hljs)
OLD_VERSION=$(detect_old_version "hljs")
check_same_version "$OLD_VERSION" "$VERSION" "hljs"
OLD_DIR="${STATIC_DIR}/hljs-${OLD_VERSION}"
NEW_DIR="${STATIC_DIR}/hljs-${VERSION}"
@@ -107,6 +128,7 @@ case "$LIB" in
mermaid)
OLD_VERSION=$(detect_old_version "mermaid")
check_same_version "$OLD_VERSION" "$VERSION" "mermaid"
OLD_DIR="${STATIC_DIR}/mermaid-${OLD_VERSION}"
NEW_DIR="${STATIC_DIR}/mermaid-${VERSION}"
File diff suppressed because it is too large Load Diff
+76 -2
View File
@@ -2,7 +2,7 @@
"openapi": "3.1.0",
"info": {
"title": "turnstone Server API",
"version": "0.7.0",
"version": "0.9.2",
"description": "Single-node workstream management, chat interaction, and real-time streaming."
},
"paths": {
@@ -458,6 +458,27 @@
}
}
},
"/v1/api/models": {
"get": {
"summary": "List available model aliases",
"operationId": "v1_api_models_get",
"tags": [
"Models"
],
"responses": {
"200": {
"description": "Success",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ListAvailableModelsResponse"
}
}
}
}
}
}
},
"/v1/api/auth/login": {
"post": {
"summary": "Authenticate with a token",
@@ -1223,6 +1244,12 @@
"description": "Target workstream ID",
"title": "Ws Id",
"type": "string"
},
"force": {
"default": false,
"description": "Force cancel: abandon the stuck worker thread immediately. Use when cooperative cancel has not resolved within a few seconds.",
"title": "Force",
"type": "boolean"
}
},
"required": [
@@ -1559,6 +1586,11 @@
"title": "Version",
"type": "string"
},
"node_id": {
"default": "",
"title": "Node Id",
"type": "string"
},
"uptime_seconds": {
"default": 0.0,
"title": "Uptime Seconds",
@@ -1569,6 +1601,12 @@
"title": "Model",
"type": "string"
},
"max_ws": {
"default": 10,
"description": "Maximum concurrent workstreams",
"title": "Max Ws",
"type": "integer"
},
"workstreams": {
"$ref": "#/components/schemas/WorkstreamCounts",
"default": {
@@ -1966,7 +2004,43 @@
],
"title": "ListSkillSummaryResponse",
"type": "object"
},
"AvailableModelInfo": {
"properties": {
"alias": {
"title": "Alias",
"type": "string"
},
"model": {
"title": "Model",
"type": "string"
},
"provider": {
"title": "Provider",
"type": "string"
}
},
"required": [
"alias",
"model",
"provider"
],
"title": "AvailableModelInfo",
"type": "object"
},
"ListAvailableModelsResponse": {
"properties": {
"models": {
"items": {
"$ref": "#/components/schemas/AvailableModelInfo"
},
"title": "Models",
"type": "array"
}
},
"title": "ListAvailableModelsResponse",
"type": "object"
}
}
}
}
}
+179 -155
View File
@@ -9,29 +9,31 @@
"version": "0.3.0",
"license": "BUSL-1.1",
"devDependencies": {
"typescript": "^5.4",
"typescript": "^6.0.0",
"vitest": "^4.1"
}
},
"node_modules/@emnapi/core": {
"version": "1.9.0",
"resolved": "https://registry.npmjs.org/@emnapi/core/-/core-1.9.0.tgz",
"integrity": "sha512-0DQ98G9ZQZOxfUcQn1waV2yS8aWdZ6kJMbYCJB3oUBecjWYO1fqJ+a1DRfPF3O5JEkwqwP1A9QEN/9mYm2Yd0w==",
"version": "1.9.1",
"resolved": "https://registry.npmjs.org/@emnapi/core/-/core-1.9.1.tgz",
"integrity": "sha512-mukuNALVsoix/w1BJwFzwXBN/dHeejQtuVzcDsfOEsdpCumXb/E9j8w11h5S54tT1xhifGfbbSm/ICrObRb3KA==",
"dev": true,
"license": "MIT",
"optional": true,
"peer": true,
"dependencies": {
"@emnapi/wasi-threads": "1.2.0",
"tslib": "^2.4.0"
}
},
"node_modules/@emnapi/runtime": {
"version": "1.9.0",
"resolved": "https://registry.npmjs.org/@emnapi/runtime/-/runtime-1.9.0.tgz",
"integrity": "sha512-QN75eB0IH2ywSpRpNddCRfQIhmJYBCJ1x5Lb3IscKAL8bMnVAKnRg8dCoXbHzVLLH7P38N2Z3mtulB7W0J0FKw==",
"version": "1.9.1",
"resolved": "https://registry.npmjs.org/@emnapi/runtime/-/runtime-1.9.1.tgz",
"integrity": "sha512-VYi5+ZVLhpgK4hQ0TAjiQiZ6ol0oe4mBx7mVv7IflsiEp0OWoVsp/+f9Vc1hOhE0TtkORVrI1GvzyreqpgWtkA==",
"dev": true,
"license": "MIT",
"optional": true,
"peer": true,
"dependencies": {
"tslib": "^2.4.0"
}
@@ -43,6 +45,7 @@
"dev": true,
"license": "MIT",
"optional": true,
"peer": true,
"dependencies": {
"tslib": "^2.4.0"
}
@@ -55,36 +58,28 @@
"license": "MIT"
},
"node_modules/@napi-rs/wasm-runtime": {
"version": "1.1.1",
"resolved": "https://registry.npmjs.org/@napi-rs/wasm-runtime/-/wasm-runtime-1.1.1.tgz",
"integrity": "sha512-p64ah1M1ld8xjWv3qbvFwHiFVWrq1yFvV4f7w+mzaqiR4IlSgkqhcRdHwsGgomwzBH51sRY4NEowLxnaBjcW/A==",
"version": "1.1.2",
"resolved": "https://registry.npmjs.org/@napi-rs/wasm-runtime/-/wasm-runtime-1.1.2.tgz",
"integrity": "sha512-sNXv5oLJ7ob93xkZ1XnxisYhGYXfaG9f65/ZgYuAu3qt7b3NadcOEhLvx28hv31PgX8SZJRYrAIPQilQmFpLVw==",
"dev": true,
"license": "MIT",
"optional": true,
"dependencies": {
"@emnapi/core": "^1.7.1",
"@emnapi/runtime": "^1.7.1",
"@tybys/wasm-util": "^0.10.1"
},
"funding": {
"type": "github",
"url": "https://github.com/sponsors/Brooooooklyn"
}
},
"node_modules/@oxc-project/runtime": {
"version": "0.115.0",
"resolved": "https://registry.npmjs.org/@oxc-project/runtime/-/runtime-0.115.0.tgz",
"integrity": "sha512-Rg8Wlt5dCbXhQnsXPrkOjL1DTSvXLgb2R/KYfnf1/K+R0k6UMLEmbQXPM+kwrWqSmWA2t0B1EtHy2/3zikQpvQ==",
"dev": true,
"license": "MIT",
"engines": {
"node": "^20.19.0 || >=22.12.0"
},
"peerDependencies": {
"@emnapi/core": "^1.7.1",
"@emnapi/runtime": "^1.7.1"
}
},
"node_modules/@oxc-project/types": {
"version": "0.115.0",
"resolved": "https://registry.npmjs.org/@oxc-project/types/-/types-0.115.0.tgz",
"integrity": "sha512-4n91DKnebUS4yjUHl2g3/b2T+IUdCfmoZGhmwsovZCDaJSs+QkVAM+0AqqTxHSsHfeiMuueT75cZaZcT/m0pSw==",
"version": "0.122.0",
"resolved": "https://registry.npmjs.org/@oxc-project/types/-/types-0.122.0.tgz",
"integrity": "sha512-oLAl5kBpV4w69UtFZ9xqcmTi+GENWOcPF7FCrczTiBbmC0ibXxCwyvZGbO39rCVEuLGAZM84DH0pUIyyv/YJzA==",
"dev": true,
"license": "MIT",
"funding": {
@@ -92,9 +87,9 @@
}
},
"node_modules/@rolldown/binding-android-arm64": {
"version": "1.0.0-rc.9",
"resolved": "https://registry.npmjs.org/@rolldown/binding-android-arm64/-/binding-android-arm64-1.0.0-rc.9.tgz",
"integrity": "sha512-lcJL0bN5hpgJfSIz/8PIf02irmyL43P+j1pTCfbD1DbLkmGRuFIA4DD3B3ZOvGqG0XiVvRznbKtN0COQVaKUTg==",
"version": "1.0.0-rc.12",
"resolved": "https://registry.npmjs.org/@rolldown/binding-android-arm64/-/binding-android-arm64-1.0.0-rc.12.tgz",
"integrity": "sha512-pv1y2Fv0JybcykuiiD3qBOBdz6RteYojRFY1d+b95WVuzx211CRh+ytI/+9iVyWQ6koTh5dawe4S/yRfOFjgaA==",
"cpu": [
"arm64"
],
@@ -109,9 +104,9 @@
}
},
"node_modules/@rolldown/binding-darwin-arm64": {
"version": "1.0.0-rc.9",
"resolved": "https://registry.npmjs.org/@rolldown/binding-darwin-arm64/-/binding-darwin-arm64-1.0.0-rc.9.tgz",
"integrity": "sha512-J7Zk3kLYFsLtuH6U+F4pS2sYVzac0qkjcO5QxHS7OS7yZu2LRs+IXo+uvJ/mvpyUljDJ3LROZPoQfgBIpCMhdQ==",
"version": "1.0.0-rc.12",
"resolved": "https://registry.npmjs.org/@rolldown/binding-darwin-arm64/-/binding-darwin-arm64-1.0.0-rc.12.tgz",
"integrity": "sha512-cFYr6zTG/3PXXF3pUO+umXxt1wkRK/0AYT8lDwuqvRC+LuKYWSAQAQZjCWDQpAH172ZV6ieYrNnFzVVcnSflAg==",
"cpu": [
"arm64"
],
@@ -126,9 +121,9 @@
}
},
"node_modules/@rolldown/binding-darwin-x64": {
"version": "1.0.0-rc.9",
"resolved": "https://registry.npmjs.org/@rolldown/binding-darwin-x64/-/binding-darwin-x64-1.0.0-rc.9.tgz",
"integrity": "sha512-iwtmmghy8nhfRGeNAIltcNXzD0QMNaaA5U/NyZc1Ia4bxrzFByNMDoppoC+hl7cDiUq5/1CnFthpT9n+UtfFyg==",
"version": "1.0.0-rc.12",
"resolved": "https://registry.npmjs.org/@rolldown/binding-darwin-x64/-/binding-darwin-x64-1.0.0-rc.12.tgz",
"integrity": "sha512-ZCsYknnHzeXYps0lGBz8JrF37GpE9bFVefrlmDrAQhOEi4IOIlcoU1+FwHEtyXGx2VkYAvhu7dyBf75EJQffBw==",
"cpu": [
"x64"
],
@@ -143,9 +138,9 @@
}
},
"node_modules/@rolldown/binding-freebsd-x64": {
"version": "1.0.0-rc.9",
"resolved": "https://registry.npmjs.org/@rolldown/binding-freebsd-x64/-/binding-freebsd-x64-1.0.0-rc.9.tgz",
"integrity": "sha512-DLFYI78SCiZr5VvdEplsVC2Vx53lnA4/Ga5C65iyldMVaErr86aiqCoNBLl92PXPfDtUYjUh+xFFor40ueNs4Q==",
"version": "1.0.0-rc.12",
"resolved": "https://registry.npmjs.org/@rolldown/binding-freebsd-x64/-/binding-freebsd-x64-1.0.0-rc.12.tgz",
"integrity": "sha512-dMLeprcVsyJsKolRXyoTH3NL6qtsT0Y2xeuEA8WQJquWFXkEC4bcu1rLZZSnZRMtAqwtrF/Ib9Ddtpa/Gkge9Q==",
"cpu": [
"x64"
],
@@ -160,9 +155,9 @@
}
},
"node_modules/@rolldown/binding-linux-arm-gnueabihf": {
"version": "1.0.0-rc.9",
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-arm-gnueabihf/-/binding-linux-arm-gnueabihf-1.0.0-rc.9.tgz",
"integrity": "sha512-CsjTmTwd0Hri6iTw/DRMK7kOZ7FwAkrO4h8YWKoX/kcj833e4coqo2wzIFywtch/8Eb5enQ/lwLM7w6JX1W5RQ==",
"version": "1.0.0-rc.12",
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-arm-gnueabihf/-/binding-linux-arm-gnueabihf-1.0.0-rc.12.tgz",
"integrity": "sha512-YqWjAgGC/9M1lz3GR1r1rP79nMgo3mQiiA+Hfo+pvKFK1fAJ1bCi0ZQVh8noOqNacuY1qIcfyVfP6HoyBRZ85Q==",
"cpu": [
"arm"
],
@@ -177,13 +172,16 @@
}
},
"node_modules/@rolldown/binding-linux-arm64-gnu": {
"version": "1.0.0-rc.9",
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-arm64-gnu/-/binding-linux-arm64-gnu-1.0.0-rc.9.tgz",
"integrity": "sha512-2x9O2JbSPxpxMDhP9Z74mahAStibTlrBMW0520+epJH5sac7/LwZW5Bmg/E6CXuEF53JJFW509uP+lSedaUNxg==",
"version": "1.0.0-rc.12",
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-arm64-gnu/-/binding-linux-arm64-gnu-1.0.0-rc.12.tgz",
"integrity": "sha512-/I5AS4cIroLpslsmzXfwbe5OmWvSsrFuEw3mwvbQ1kDxJ822hFHIx+vsN/TAzNVyepI/j/GSzrtCIwQPeKCLIg==",
"cpu": [
"arm64"
],
"dev": true,
"libc": [
"glibc"
],
"license": "MIT",
"optional": true,
"os": [
@@ -194,13 +192,16 @@
}
},
"node_modules/@rolldown/binding-linux-arm64-musl": {
"version": "1.0.0-rc.9",
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-arm64-musl/-/binding-linux-arm64-musl-1.0.0-rc.9.tgz",
"integrity": "sha512-JA1QRW31ogheAIRhIg9tjMfsYbglXXYGNPLdPEYrwFxdbkQCAzvpSCSHCDWNl4hTtrol8WeboCSEpjdZK8qrCg==",
"version": "1.0.0-rc.12",
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-arm64-musl/-/binding-linux-arm64-musl-1.0.0-rc.12.tgz",
"integrity": "sha512-V6/wZztnBqlx5hJQqNWwFdxIKN0m38p8Jas+VoSfgH54HSj9tKTt1dZvG6JRHcjh6D7TvrJPWFGaY9UBVOaWPw==",
"cpu": [
"arm64"
],
"dev": true,
"libc": [
"musl"
],
"license": "MIT",
"optional": true,
"os": [
@@ -211,13 +212,16 @@
}
},
"node_modules/@rolldown/binding-linux-ppc64-gnu": {
"version": "1.0.0-rc.9",
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-ppc64-gnu/-/binding-linux-ppc64-gnu-1.0.0-rc.9.tgz",
"integrity": "sha512-aOKU9dJheda8Kj8Y3w9gnt9QFOO+qKPAl8SWd7JPHP+Cu0EuDAE5wokQubLzIDQWg2myXq2XhTpOVS07qqvT+w==",
"version": "1.0.0-rc.12",
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-ppc64-gnu/-/binding-linux-ppc64-gnu-1.0.0-rc.12.tgz",
"integrity": "sha512-AP3E9BpcUYliZCxa3w5Kwj9OtEVDYK6sVoUzy4vTOJsjPOgdaJZKFmN4oOlX0Wp0RPV2ETfmIra9x1xuayFB7g==",
"cpu": [
"ppc64"
],
"dev": true,
"libc": [
"glibc"
],
"license": "MIT",
"optional": true,
"os": [
@@ -228,13 +232,16 @@
}
},
"node_modules/@rolldown/binding-linux-s390x-gnu": {
"version": "1.0.0-rc.9",
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-s390x-gnu/-/binding-linux-s390x-gnu-1.0.0-rc.9.tgz",
"integrity": "sha512-OalO94fqj7IWRn3VdXWty75jC5dk4C197AWEuMhIpvVv2lw9fiPhud0+bW2ctCxb3YoBZor71QHbY+9/WToadA==",
"version": "1.0.0-rc.12",
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-s390x-gnu/-/binding-linux-s390x-gnu-1.0.0-rc.12.tgz",
"integrity": "sha512-nWwpvUSPkoFmZo0kQazZYOrT7J5DGOJ/+QHHzjvNlooDZED8oH82Yg67HvehPPLAg5fUff7TfWFHQS8IV1n3og==",
"cpu": [
"s390x"
],
"dev": true,
"libc": [
"glibc"
],
"license": "MIT",
"optional": true,
"os": [
@@ -245,13 +252,16 @@
}
},
"node_modules/@rolldown/binding-linux-x64-gnu": {
"version": "1.0.0-rc.9",
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-x64-gnu/-/binding-linux-x64-gnu-1.0.0-rc.9.tgz",
"integrity": "sha512-cVEl1vZtBsBZna3YMjGXNvnYYrOJ7RzuWvZU0ffvJUexWkukMaDuGhUXn0rjnV0ptzGVkvc+vW9Yqy6h8YX4pg==",
"version": "1.0.0-rc.12",
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-x64-gnu/-/binding-linux-x64-gnu-1.0.0-rc.12.tgz",
"integrity": "sha512-RNrafz5bcwRy+O9e6P8Z/OCAJW/A+qtBczIqVYwTs14pf4iV1/+eKEjdOUta93q2TsT/FI0XYDP3TCky38LMAg==",
"cpu": [
"x64"
],
"dev": true,
"libc": [
"glibc"
],
"license": "MIT",
"optional": true,
"os": [
@@ -262,13 +272,16 @@
}
},
"node_modules/@rolldown/binding-linux-x64-musl": {
"version": "1.0.0-rc.9",
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-x64-musl/-/binding-linux-x64-musl-1.0.0-rc.9.tgz",
"integrity": "sha512-UzYnKCIIc4heAKgI4PZ3dfBGUZefGCJ1TPDuLHoCzgrMYPb5Rv6TLFuYtyM4rWyHM7hymNdsg5ik2C+UD9VDbA==",
"version": "1.0.0-rc.12",
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-x64-musl/-/binding-linux-x64-musl-1.0.0-rc.12.tgz",
"integrity": "sha512-Jpw/0iwoKWx3LJ2rc1yjFrj+T7iHZn2JDg1Yny1ma0luviFS4mhAIcd1LFNxK3EYu3DHWCps0ydXQ5i/rrJ2ig==",
"cpu": [
"x64"
],
"dev": true,
"libc": [
"musl"
],
"license": "MIT",
"optional": true,
"os": [
@@ -279,9 +292,9 @@
}
},
"node_modules/@rolldown/binding-openharmony-arm64": {
"version": "1.0.0-rc.9",
"resolved": "https://registry.npmjs.org/@rolldown/binding-openharmony-arm64/-/binding-openharmony-arm64-1.0.0-rc.9.tgz",
"integrity": "sha512-+6zoiF+RRyf5cdlFQP7nm58mq7+/2PFaY2DNQeD4B87N36JzfF/l9mdBkkmTvSYcYPE8tMh/o3cRlsx1ldLfog==",
"version": "1.0.0-rc.12",
"resolved": "https://registry.npmjs.org/@rolldown/binding-openharmony-arm64/-/binding-openharmony-arm64-1.0.0-rc.12.tgz",
"integrity": "sha512-vRugONE4yMfVn0+7lUKdKvN4D5YusEiPilaoO2sgUWpCvrncvWgPMzK00ZFFJuiPgLwgFNP5eSiUlv2tfc+lpA==",
"cpu": [
"arm64"
],
@@ -296,9 +309,9 @@
}
},
"node_modules/@rolldown/binding-wasm32-wasi": {
"version": "1.0.0-rc.9",
"resolved": "https://registry.npmjs.org/@rolldown/binding-wasm32-wasi/-/binding-wasm32-wasi-1.0.0-rc.9.tgz",
"integrity": "sha512-rgFN6sA/dyebil3YTlL2evvi/M+ivhfnyxec7AccTpRPccno/rPoNlqybEZQBkcbZu8Hy+eqNJCqfBR8P7Pg8g==",
"version": "1.0.0-rc.12",
"resolved": "https://registry.npmjs.org/@rolldown/binding-wasm32-wasi/-/binding-wasm32-wasi-1.0.0-rc.12.tgz",
"integrity": "sha512-ykGiLr/6kkiHc0XnBfmFJuCjr5ZYKKofkx+chJWDjitX+KsJuAmrzWhwyOMSHzPhzOHOy7u9HlFoa5MoAOJ/Zg==",
"cpu": [
"wasm32"
],
@@ -313,9 +326,9 @@
}
},
"node_modules/@rolldown/binding-win32-arm64-msvc": {
"version": "1.0.0-rc.9",
"resolved": "https://registry.npmjs.org/@rolldown/binding-win32-arm64-msvc/-/binding-win32-arm64-msvc-1.0.0-rc.9.tgz",
"integrity": "sha512-lHVNUG/8nlF1IQk1C0Ci574qKYyty2goMiPlRqkC5R+3LkXDkL5Dhx8ytbxq35m+pkHVIvIxviD+TWLdfeuadA==",
"version": "1.0.0-rc.12",
"resolved": "https://registry.npmjs.org/@rolldown/binding-win32-arm64-msvc/-/binding-win32-arm64-msvc-1.0.0-rc.12.tgz",
"integrity": "sha512-5eOND4duWkwx1AzCxadcOrNeighiLwMInEADT0YM7xeEOOFcovWZCq8dadXgcRHSf3Ulh1kFo/qvzoFiCLOL1Q==",
"cpu": [
"arm64"
],
@@ -330,9 +343,9 @@
}
},
"node_modules/@rolldown/binding-win32-x64-msvc": {
"version": "1.0.0-rc.9",
"resolved": "https://registry.npmjs.org/@rolldown/binding-win32-x64-msvc/-/binding-win32-x64-msvc-1.0.0-rc.9.tgz",
"integrity": "sha512-G0oA4+w1iY5AGi5HcDTxWsoxF509hrFIPB2rduV5aDqS9FtDg1CAfa7V34qImbjfhIcA8C+RekocJZA96EarwQ==",
"version": "1.0.0-rc.12",
"resolved": "https://registry.npmjs.org/@rolldown/binding-win32-x64-msvc/-/binding-win32-x64-msvc-1.0.0-rc.12.tgz",
"integrity": "sha512-PyqoipaswDLAZtot351MLhrlrh6lcZPo2LSYE+VDxbVk24LVKAGOuE4hb8xZQmrPAuEtTZW8E6D2zc5EUZX4Lw==",
"cpu": [
"x64"
],
@@ -347,9 +360,9 @@
}
},
"node_modules/@rolldown/pluginutils": {
"version": "1.0.0-rc.9",
"resolved": "https://registry.npmjs.org/@rolldown/pluginutils/-/pluginutils-1.0.0-rc.9.tgz",
"integrity": "sha512-w6oiRWgEBl04QkFZgmW+jnU1EC9b57Oihi2ot3HNWIQRqgHp5PnYDia5iZ5FF7rpa4EQdiqMDXjlqKGXBhsoXw==",
"version": "1.0.0-rc.12",
"resolved": "https://registry.npmjs.org/@rolldown/pluginutils/-/pluginutils-1.0.0-rc.12.tgz",
"integrity": "sha512-HHMwmarRKvoFsJorqYlFeFRzXZqCt2ETQlEDOb9aqssrnVBB1/+xgTGtuTrIk5vzLNX1MjMtTf7W9z3tsSbrxw==",
"dev": true,
"license": "MIT"
},
@@ -397,31 +410,31 @@
"license": "MIT"
},
"node_modules/@vitest/expect": {
"version": "4.1.0",
"resolved": "https://registry.npmjs.org/@vitest/expect/-/expect-4.1.0.tgz",
"integrity": "sha512-EIxG7k4wlWweuCLG9Y5InKFwpMEOyrMb6ZJ1ihYu02LVj/bzUwn2VMU+13PinsjRW75XnITeFrQBMH5+dLvCDA==",
"version": "4.1.2",
"resolved": "https://registry.npmjs.org/@vitest/expect/-/expect-4.1.2.tgz",
"integrity": "sha512-gbu+7B0YgUJ2nkdsRJrFFW6X7NTP44WlhiclHniUhxADQJH5Szt9mZ9hWnJPJ8YwOK5zUOSSlSvyzRf0u1DSBQ==",
"dev": true,
"license": "MIT",
"dependencies": {
"@standard-schema/spec": "^1.1.0",
"@types/chai": "^5.2.2",
"@vitest/spy": "4.1.0",
"@vitest/utils": "4.1.0",
"@vitest/spy": "4.1.2",
"@vitest/utils": "4.1.2",
"chai": "^6.2.2",
"tinyrainbow": "^3.0.3"
"tinyrainbow": "^3.1.0"
},
"funding": {
"url": "https://opencollective.com/vitest"
}
},
"node_modules/@vitest/mocker": {
"version": "4.1.0",
"resolved": "https://registry.npmjs.org/@vitest/mocker/-/mocker-4.1.0.tgz",
"integrity": "sha512-evxREh+Hork43+Y4IOhTo+h5lGmVRyjqI739Rz4RlUPqwrkFFDF6EMvOOYjTx4E8Tl6gyCLRL8Mu7Ry12a13Tw==",
"version": "4.1.2",
"resolved": "https://registry.npmjs.org/@vitest/mocker/-/mocker-4.1.2.tgz",
"integrity": "sha512-Ize4iQtEALHDttPRCmN+FKqOl2vxTiNUhzobQFFt/BM1lRUTG7zRCLOykG/6Vo4E4hnUdfVLo5/eqKPukcWW7Q==",
"dev": true,
"license": "MIT",
"dependencies": {
"@vitest/spy": "4.1.0",
"@vitest/spy": "4.1.2",
"estree-walker": "^3.0.3",
"magic-string": "^0.30.21"
},
@@ -430,7 +443,7 @@
},
"peerDependencies": {
"msw": "^2.4.9",
"vite": "^6.0.0 || ^7.0.0 || ^8.0.0-0"
"vite": "^6.0.0 || ^7.0.0 || ^8.0.0"
},
"peerDependenciesMeta": {
"msw": {
@@ -442,26 +455,26 @@
}
},
"node_modules/@vitest/pretty-format": {
"version": "4.1.0",
"resolved": "https://registry.npmjs.org/@vitest/pretty-format/-/pretty-format-4.1.0.tgz",
"integrity": "sha512-3RZLZlh88Ib0J7NQTRATfc/3ZPOnSUn2uDBUoGNn5T36+bALixmzphN26OUD3LRXWkJu4H0s5vvUeqBiw+kS0A==",
"version": "4.1.2",
"resolved": "https://registry.npmjs.org/@vitest/pretty-format/-/pretty-format-4.1.2.tgz",
"integrity": "sha512-dwQga8aejqeuB+TvXCMzSQemvV9hNEtDDpgUKDzOmNQayl2OG241PSWeJwKRH3CiC+sESrmoFd49rfnq7T4RnA==",
"dev": true,
"license": "MIT",
"dependencies": {
"tinyrainbow": "^3.0.3"
"tinyrainbow": "^3.1.0"
},
"funding": {
"url": "https://opencollective.com/vitest"
}
},
"node_modules/@vitest/runner": {
"version": "4.1.0",
"resolved": "https://registry.npmjs.org/@vitest/runner/-/runner-4.1.0.tgz",
"integrity": "sha512-Duvx2OzQ7d6OjchL+trw+aSrb9idh7pnNfxrklo14p3zmNL4qPCDeIJAK+eBKYjkIwG96Bc6vYuxhqDXQOWpoQ==",
"version": "4.1.2",
"resolved": "https://registry.npmjs.org/@vitest/runner/-/runner-4.1.2.tgz",
"integrity": "sha512-Gr+FQan34CdiYAwpGJmQG8PgkyFVmARK8/xSijia3eTFgVfpcpztWLuP6FttGNfPLJhaZVP/euvujeNYar36OQ==",
"dev": true,
"license": "MIT",
"dependencies": {
"@vitest/utils": "4.1.0",
"@vitest/utils": "4.1.2",
"pathe": "^2.0.3"
},
"funding": {
@@ -469,14 +482,14 @@
}
},
"node_modules/@vitest/snapshot": {
"version": "4.1.0",
"resolved": "https://registry.npmjs.org/@vitest/snapshot/-/snapshot-4.1.0.tgz",
"integrity": "sha512-0Vy9euT1kgsnj1CHttwi9i9o+4rRLEaPRSOJ5gyv579GJkNpgJK+B4HSv/rAWixx2wdAFci1X4CEPjiu2bXIMg==",
"version": "4.1.2",
"resolved": "https://registry.npmjs.org/@vitest/snapshot/-/snapshot-4.1.2.tgz",
"integrity": "sha512-g7yfUmxYS4mNxk31qbOYsSt2F4m1E02LFqO53Xpzg3zKMhLAPZAjjfyl9e6z7HrW6LvUdTwAQR3HHfLjpko16A==",
"dev": true,
"license": "MIT",
"dependencies": {
"@vitest/pretty-format": "4.1.0",
"@vitest/utils": "4.1.0",
"@vitest/pretty-format": "4.1.2",
"@vitest/utils": "4.1.2",
"magic-string": "^0.30.21",
"pathe": "^2.0.3"
},
@@ -485,9 +498,9 @@
}
},
"node_modules/@vitest/spy": {
"version": "4.1.0",
"resolved": "https://registry.npmjs.org/@vitest/spy/-/spy-4.1.0.tgz",
"integrity": "sha512-pz77k+PgNpyMDv2FV6qmk5ZVau6c3R8HC8v342T2xlFxQKTrSeYw9waIJG8KgV9fFwAtTu4ceRzMivPTH6wSxw==",
"version": "4.1.2",
"resolved": "https://registry.npmjs.org/@vitest/spy/-/spy-4.1.2.tgz",
"integrity": "sha512-DU4fBnbVCJGNBwVA6xSToNXrkZNSiw59H8tcuUspVMsBDBST4nfvsPsEHDHGtWRRnqBERBQu7TrTKskmjqTXKA==",
"dev": true,
"license": "MIT",
"funding": {
@@ -495,15 +508,15 @@
}
},
"node_modules/@vitest/utils": {
"version": "4.1.0",
"resolved": "https://registry.npmjs.org/@vitest/utils/-/utils-4.1.0.tgz",
"integrity": "sha512-XfPXT6a8TZY3dcGY8EdwsBulFCIw+BeeX0RZn2x/BtiY/75YGh8FeWGG8QISN/WhaqSrE2OrlDgtF8q5uhOTmw==",
"version": "4.1.2",
"resolved": "https://registry.npmjs.org/@vitest/utils/-/utils-4.1.2.tgz",
"integrity": "sha512-xw2/TiX82lQHA06cgbqRKFb5lCAy3axQ4H4SoUFhUsg+wztiet+co86IAMDtF6Vm1hc7J6j09oh/rgDn+JdKIQ==",
"dev": true,
"license": "MIT",
"dependencies": {
"@vitest/pretty-format": "4.1.0",
"@vitest/pretty-format": "4.1.2",
"convert-source-map": "^2.0.0",
"tinyrainbow": "^3.0.3"
"tinyrainbow": "^3.1.0"
},
"funding": {
"url": "https://opencollective.com/vitest"
@@ -749,6 +762,9 @@
"arm64"
],
"dev": true,
"libc": [
"glibc"
],
"license": "MPL-2.0",
"optional": true,
"os": [
@@ -770,6 +786,9 @@
"arm64"
],
"dev": true,
"libc": [
"musl"
],
"license": "MPL-2.0",
"optional": true,
"os": [
@@ -791,6 +810,9 @@
"x64"
],
"dev": true,
"libc": [
"glibc"
],
"license": "MPL-2.0",
"optional": true,
"os": [
@@ -812,6 +834,9 @@
"x64"
],
"dev": true,
"libc": [
"musl"
],
"license": "MPL-2.0",
"optional": true,
"os": [
@@ -922,9 +947,9 @@
"license": "ISC"
},
"node_modules/picomatch": {
"version": "4.0.3",
"resolved": "https://registry.npmjs.org/picomatch/-/picomatch-4.0.3.tgz",
"integrity": "sha512-5gTmgEY/sqK6gFXLIsQNH19lWb4ebPDLA4SdLP7dsWkIXHWlG66oPuVvXSGFPppYZz8ZDZq0dYYrbHfBCVUb1Q==",
"version": "4.0.4",
"resolved": "https://registry.npmjs.org/picomatch/-/picomatch-4.0.4.tgz",
"integrity": "sha512-QP88BAKvMam/3NxH6vj2o21R6MjxZUAd6nlwAS/pnGvN9IVLocLHxGYIzFhg6fUQ+5th6P4dv4eW9jX3DSIj7A==",
"dev": true,
"license": "MIT",
"engines": {
@@ -964,14 +989,14 @@
}
},
"node_modules/rolldown": {
"version": "1.0.0-rc.9",
"resolved": "https://registry.npmjs.org/rolldown/-/rolldown-1.0.0-rc.9.tgz",
"integrity": "sha512-9EbgWge7ZH+yqb4d2EnELAntgPTWbfL8ajiTW+SyhJEC4qhBbkCKbqFV4Ge4zmu5ziQuVbWxb/XwLZ+RIO7E8Q==",
"version": "1.0.0-rc.12",
"resolved": "https://registry.npmjs.org/rolldown/-/rolldown-1.0.0-rc.12.tgz",
"integrity": "sha512-yP4USLIMYrwpPHEFB5JGH1uxhcslv6/hL0OyvTuY+3qlOSJvZ7ntYnoWpehBxufkgN0cvXxppuTu5hHa/zPh+A==",
"dev": true,
"license": "MIT",
"dependencies": {
"@oxc-project/types": "=0.115.0",
"@rolldown/pluginutils": "1.0.0-rc.9"
"@oxc-project/types": "=0.122.0",
"@rolldown/pluginutils": "1.0.0-rc.12"
},
"bin": {
"rolldown": "bin/cli.mjs"
@@ -980,21 +1005,21 @@
"node": "^20.19.0 || >=22.12.0"
},
"optionalDependencies": {
"@rolldown/binding-android-arm64": "1.0.0-rc.9",
"@rolldown/binding-darwin-arm64": "1.0.0-rc.9",
"@rolldown/binding-darwin-x64": "1.0.0-rc.9",
"@rolldown/binding-freebsd-x64": "1.0.0-rc.9",
"@rolldown/binding-linux-arm-gnueabihf": "1.0.0-rc.9",
"@rolldown/binding-linux-arm64-gnu": "1.0.0-rc.9",
"@rolldown/binding-linux-arm64-musl": "1.0.0-rc.9",
"@rolldown/binding-linux-ppc64-gnu": "1.0.0-rc.9",
"@rolldown/binding-linux-s390x-gnu": "1.0.0-rc.9",
"@rolldown/binding-linux-x64-gnu": "1.0.0-rc.9",
"@rolldown/binding-linux-x64-musl": "1.0.0-rc.9",
"@rolldown/binding-openharmony-arm64": "1.0.0-rc.9",
"@rolldown/binding-wasm32-wasi": "1.0.0-rc.9",
"@rolldown/binding-win32-arm64-msvc": "1.0.0-rc.9",
"@rolldown/binding-win32-x64-msvc": "1.0.0-rc.9"
"@rolldown/binding-android-arm64": "1.0.0-rc.12",
"@rolldown/binding-darwin-arm64": "1.0.0-rc.12",
"@rolldown/binding-darwin-x64": "1.0.0-rc.12",
"@rolldown/binding-freebsd-x64": "1.0.0-rc.12",
"@rolldown/binding-linux-arm-gnueabihf": "1.0.0-rc.12",
"@rolldown/binding-linux-arm64-gnu": "1.0.0-rc.12",
"@rolldown/binding-linux-arm64-musl": "1.0.0-rc.12",
"@rolldown/binding-linux-ppc64-gnu": "1.0.0-rc.12",
"@rolldown/binding-linux-s390x-gnu": "1.0.0-rc.12",
"@rolldown/binding-linux-x64-gnu": "1.0.0-rc.12",
"@rolldown/binding-linux-x64-musl": "1.0.0-rc.12",
"@rolldown/binding-openharmony-arm64": "1.0.0-rc.12",
"@rolldown/binding-wasm32-wasi": "1.0.0-rc.12",
"@rolldown/binding-win32-arm64-msvc": "1.0.0-rc.12",
"@rolldown/binding-win32-x64-msvc": "1.0.0-rc.12"
}
},
"node_modules/siginfo": {
@@ -1081,9 +1106,9 @@
"optional": true
},
"node_modules/typescript": {
"version": "5.9.3",
"resolved": "https://registry.npmjs.org/typescript/-/typescript-5.9.3.tgz",
"integrity": "sha512-jl1vZzPDinLr9eUt3J/t7V6FgNEw9QjvBPdysz9KfQDD41fQrC2Y4vKQdiaUpFT4bXlb1RHhLpp8wtm6M5TgSw==",
"version": "6.0.2",
"resolved": "https://registry.npmjs.org/typescript/-/typescript-6.0.2.tgz",
"integrity": "sha512-bGdAIrZ0wiGDo5l8c++HWtbaNCWTS4UTv7RaTH/ThVIgjkveJt83m74bBHMJkuCbslY8ixgLBVZJIOiQlQTjfQ==",
"dev": true,
"license": "Apache-2.0",
"bin": {
@@ -1095,17 +1120,16 @@
}
},
"node_modules/vite": {
"version": "8.0.0",
"resolved": "https://registry.npmjs.org/vite/-/vite-8.0.0.tgz",
"integrity": "sha512-fPGaRNj9Zytaf8LEiBhY7Z6ijnFKdzU/+mL8EFBaKr7Vw1/FWcTBAMW0wLPJAGMPX38ZPVCVgLceWiEqeoqL2Q==",
"version": "8.0.3",
"resolved": "https://registry.npmjs.org/vite/-/vite-8.0.3.tgz",
"integrity": "sha512-B9ifbFudT1TFhfltfaIPgjo9Z3mDynBTJSUYxTjOQruf/zHH+ezCQKcoqO+h7a9Pw9Nm/OtlXAiGT1axBgwqrQ==",
"dev": true,
"license": "MIT",
"dependencies": {
"@oxc-project/runtime": "0.115.0",
"lightningcss": "^1.32.0",
"picomatch": "^4.0.3",
"picomatch": "^4.0.4",
"postcss": "^8.5.8",
"rolldown": "1.0.0-rc.9",
"rolldown": "1.0.0-rc.12",
"tinyglobby": "^0.2.15"
},
"bin": {
@@ -1122,7 +1146,7 @@
},
"peerDependencies": {
"@types/node": "^20.19.0 || >=22.12.0",
"@vitejs/devtools": "^0.0.0-alpha.31",
"@vitejs/devtools": "^0.1.0",
"esbuild": "^0.27.0",
"jiti": ">=1.21.0",
"less": "^4.0.0",
@@ -1174,19 +1198,19 @@
}
},
"node_modules/vitest": {
"version": "4.1.0",
"resolved": "https://registry.npmjs.org/vitest/-/vitest-4.1.0.tgz",
"integrity": "sha512-YbDrMF9jM2Lqc++2530UourxZHmkKLxrs4+mYhEwqWS97WJ7wOYEkcr+QfRgJ3PW9wz3odRijLZjHEaRLTNbqw==",
"version": "4.1.2",
"resolved": "https://registry.npmjs.org/vitest/-/vitest-4.1.2.tgz",
"integrity": "sha512-xjR1dMTVHlFLh98JE3i/f/WePqJsah4A0FK9cc8Ehp9Udk0AZk6ccpIZhh1qJ/yxVWRZ+Q54ocnD8TXmkhspGg==",
"dev": true,
"license": "MIT",
"dependencies": {
"@vitest/expect": "4.1.0",
"@vitest/mocker": "4.1.0",
"@vitest/pretty-format": "4.1.0",
"@vitest/runner": "4.1.0",
"@vitest/snapshot": "4.1.0",
"@vitest/spy": "4.1.0",
"@vitest/utils": "4.1.0",
"@vitest/expect": "4.1.2",
"@vitest/mocker": "4.1.2",
"@vitest/pretty-format": "4.1.2",
"@vitest/runner": "4.1.2",
"@vitest/snapshot": "4.1.2",
"@vitest/spy": "4.1.2",
"@vitest/utils": "4.1.2",
"es-module-lexer": "^2.0.0",
"expect-type": "^1.3.0",
"magic-string": "^0.30.21",
@@ -1197,8 +1221,8 @@
"tinybench": "^2.9.0",
"tinyexec": "^1.0.2",
"tinyglobby": "^0.2.15",
"tinyrainbow": "^3.0.3",
"vite": "^6.0.0 || ^7.0.0 || ^8.0.0-0",
"tinyrainbow": "^3.1.0",
"vite": "^6.0.0 || ^7.0.0 || ^8.0.0",
"why-is-node-running": "^2.3.0"
},
"bin": {
@@ -1214,13 +1238,13 @@
"@edge-runtime/vm": "*",
"@opentelemetry/api": "^1.9.0",
"@types/node": "^20.0.0 || ^22.0.0 || >=24.0.0",
"@vitest/browser-playwright": "4.1.0",
"@vitest/browser-preview": "4.1.0",
"@vitest/browser-webdriverio": "4.1.0",
"@vitest/ui": "4.1.0",
"@vitest/browser-playwright": "4.1.2",
"@vitest/browser-preview": "4.1.2",
"@vitest/browser-webdriverio": "4.1.2",
"@vitest/ui": "4.1.2",
"happy-dom": "*",
"jsdom": "*",
"vite": "^6.0.0 || ^7.0.0 || ^8.0.0-0"
"vite": "^6.0.0 || ^7.0.0 || ^8.0.0"
},
"peerDependenciesMeta": {
"@edge-runtime/vm": {
+1 -1
View File
@@ -32,7 +32,7 @@
],
"license": "BUSL-1.1",
"devDependencies": {
"typescript": "^5.4",
"typescript": "^6.0.0",
"vitest": "^4.1"
}
}
+16
View File
@@ -1,6 +1,15 @@
import { TurnstoneAPIError } from "./errors.js";
import { parseSSEStream } from "./sse.js";
export interface TlsOptions {
/** Path to CA certificate PEM file (Node.js only). */
caCert?: string;
/** Path to client certificate PEM file for mTLS (Node.js only). */
clientCert?: string;
/** Path to client key PEM file for mTLS (Node.js only). */
clientKey?: string;
}
export interface ClientOptions {
/** Server base URL (e.g. "http://localhost:8080"). */
baseUrl: string;
@@ -8,6 +17,13 @@ export interface ClientOptions {
token?: string;
/** Custom fetch implementation (defaults to globalThis.fetch). */
fetch?: typeof globalThis.fetch;
/**
* TLS certificate paths for documentation and tooling.
* The SDK does not read these directly pass a custom `fetch`
* configured with your runtime's TLS agent (e.g. Node.js https.Agent).
* See docs/tls.md for examples.
*/
tls?: TlsOptions;
}
export interface RequestOptions {
+5 -1
View File
@@ -44,6 +44,7 @@ import type {
OrgInfo,
RoleInfo,
ScheduleInfo,
DeleteSettingResponse,
SettingInfo,
StatusResponse,
ToolPolicyInfo,
@@ -394,7 +395,10 @@ export class TurnstoneConsole extends BaseClient {
});
}
async deleteSetting(key: string, nodeId?: string): Promise<StatusResponse> {
async deleteSetting(
key: string,
nodeId?: string,
): Promise<DeleteSettingResponse> {
const params: Record<string, string> = {};
if (nodeId) params.node_id = nodeId;
return this.request("DELETE", `/v1/api/admin/settings/${key}`, {
+13
View File
@@ -38,6 +38,11 @@ export interface StreamEndEvent {
type: "stream_end";
}
export interface StateChangeEvent {
type: "state_change";
state: "idle" | "thinking" | "running" | "attention" | "error";
}
export interface ToolInfoEvent {
type: "tool_info";
items: Array<Record<string, unknown>>;
@@ -59,6 +64,7 @@ export interface ToolResultEvent {
call_id: string;
name: string;
output: string;
is_error?: boolean;
}
export interface ToolOutputChunkEvent {
@@ -77,6 +83,8 @@ export interface StatusEvent {
effort: string;
cache_creation_tokens?: number;
cache_read_tokens?: number;
tool_calls_this_turn?: number;
turn_count?: number;
}
export interface PlanReviewEvent {
@@ -149,6 +157,7 @@ export type ServerEvent =
| ContentEvent
| ReasoningEvent
| StreamEndEvent
| StateChangeEvent
| ToolInfoEvent
| ApproveRequestEvent
| ApprovalResolvedEvent
@@ -246,6 +255,10 @@ export function isStreamEndEvent(e: ServerEvent): e is StreamEndEvent {
return e.type === "stream_end";
}
export function isStateChangeEvent(e: ServerEvent): e is StateChangeEvent {
return e.type === "state_change";
}
export function isToolResultEvent(e: ServerEvent): e is ToolResultEvent {
return e.type === "tool_result";
}
+4 -1
View File
@@ -19,7 +19,7 @@
// Clients
export { TurnstoneServer } from "./server.js";
export { TurnstoneConsole } from "./console.js";
export type { ClientOptions } from "./base.js";
export type { ClientOptions, TlsOptions } from "./base.js";
// Errors
export { TurnstoneAPIError } from "./errors.js";
@@ -35,6 +35,7 @@ export type {
ContentEvent,
ReasoningEvent,
StreamEndEvent,
StateChangeEvent,
ToolInfoEvent,
ApproveRequestEvent,
ApprovalResolvedEvent,
@@ -65,6 +66,7 @@ export {
isReasoningEvent,
isErrorEvent,
isStreamEndEvent,
isStateChangeEvent,
isToolResultEvent,
isWsStateEvent,
isApproveRequestEvent,
@@ -98,6 +100,7 @@ export type {
AuthLoginResponse,
AuthStatusResponse,
AuthSetupResponse,
DeleteSettingResponse,
StatusResponse,
ErrorResponse,
ClusterOverviewResponse,
+7 -4
View File
@@ -93,10 +93,13 @@ export class TurnstoneServer extends BaseClient {
});
}
async cancel(wsId: string): Promise<StatusResponse> {
return this.request("POST", "/v1/api/cancel", {
json: { ws_id: wsId },
});
async cancel(
wsId: string,
opts?: { force?: boolean },
): Promise<StatusResponse> {
const body: Record<string, unknown> = { ws_id: wsId };
if (opts?.force) body.force = true;
return this.request("POST", "/v1/api/cancel", { json: body });
}
// -- Streaming ------------------------------------------------------------
+16
View File
@@ -10,6 +10,12 @@ export interface StatusResponse {
status: string;
}
export interface DeleteSettingResponse {
status: string;
key: string;
default: unknown;
}
export interface AuthLoginRequest {
token: string;
}
@@ -186,7 +192,10 @@ export interface SkillInfo {
agent_max_turns: number | null;
notify_on_complete: string;
enabled: boolean;
priority: number;
allowed_tools: string;
license: string;
compatibility: string;
resource_count: number;
created: string;
updated: string;
@@ -213,7 +222,10 @@ export interface CreateSkillRequest {
agent_max_turns?: number | null;
notify_on_complete?: string;
enabled?: boolean;
priority?: number;
allowed_tools?: string;
license?: string;
compatibility?: string;
}
export interface UpdateSkillRequest {
@@ -236,7 +248,10 @@ export interface UpdateSkillRequest {
agent_max_turns?: number | null;
notify_on_complete?: string;
enabled?: boolean;
priority?: number;
allowed_tools?: string;
license?: string;
compatibility?: string;
}
export interface ListSkillsResponse {
@@ -396,6 +411,7 @@ export interface ConsoleCreateWsRequest {
model?: string;
initial_message?: string;
skill?: string;
resume_ws?: string;
}
export interface ConsoleCreateWsResponse {
+344 -48
View File
@@ -1,5 +1,5 @@
{
"description": "turnstone behavior tests tool selection, sequencing, and multi-step reasoning",
"description": "turnstone behavior tests \u2014 tool selection, sequencing, and multi-step reasoning",
"defaults": {
"n_runs": 5,
"max_turns": 15
@@ -8,35 +8,91 @@
{
"id": "read-before-edit",
"description": "Must read_file before edit_file on the same path",
"user_prompt": "Fix the typo in config.py change 'recieve' to 'receive'",
"user_prompt": "Fix the typo in config.py \u2014 change 'recieve' to 'receive'",
"setup": {
"files": {
"config.py": "# Config module\ndef recieve_data(source):\n \"\"\"Recieve data from source.\"\"\"\n return source.read()\n"
}
},
"expected_actions": [
{ "tool": "read_file", "args": { "path": "config.py" } },
{ "tool": "edit_file", "args": { "path": "config.py" } }
{
"tool": "read_file",
"args": {
"path": "config.py"
}
},
{
"tool": "edit_file",
"args": {
"path": "config.py"
}
}
],
"match_mode": "ordered_subset"
"match_mode": "ordered_subset",
"user_prompts": [
"Fix the typo in config.py \u2014 change 'recieve' to 'receive'",
"In config.py, correct the misspelling of 'recieve' to 'receive'",
"Please update config.py by replacing 'recieve' with the correct spelling 'receive'",
"There's a typo in config.py: 'recieve' should be 'receive'. Please fix it.",
"Could you change 'recieve' to 'receive' in config.py?",
"Go ahead and fix 'recieve' \u2192 'receive' in config.py",
"I need the word 'recieve' corrected to 'receive' in the file config.py",
"config.py has a spelling error \u2014 'recieve' needs to be changed to 'receive'",
"Kindly rectify the typographical error in config.py, replacing 'recieve' with 'receive'",
"Hey, swap 'recieve' for 'receive' in config.py"
]
},
{
"id": "write-file-not-bash",
"description": "Use write_file for file creation, not bash echo/cat",
"user_prompt": "Create a file called hello.py that prints hello world",
"expected_actions": [
{ "tool": "write_file", "args_pattern": { "path": "hello\\.py" } }
{
"tool": "write_file",
"args_pattern": {
"path": "hello\\.py"
}
}
],
"match_mode": "subset"
"match_mode": "subset",
"user_prompts": [
"Create a file called hello.py that prints hello world",
"Make a hello.py file that outputs hello world",
"Please write a Python file named hello.py which prints hello world",
"I need a file called hello.py that prints hello world",
"Could you create hello.py with code that prints hello world?",
"Write hello.py \u2014 it should print hello world",
"Generate a hello.py file that outputs \"hello world\"",
"I'd like you to create a file named hello.py that prints hello world",
"Set up a file called hello.py to print hello world",
"Kindly produce a hello.py file whose purpose is to print hello world"
]
},
{
"id": "bash-for-commands",
"description": "Use bash for running system commands",
"user_prompt": "What Python version is installed?",
"expected_actions": [
{ "tool": "bash", "args_pattern": { "command": "python" } }
{
"tool": "bash",
"args_pattern": {
"command": "python"
}
}
],
"match_mode": "subset"
"match_mode": "subset",
"user_prompts": [
"What Python version is installed?",
"Check which version of Python is currently installed",
"Can you tell me the installed Python version?",
"python --version please",
"I need to know what version of Python is on this system",
"Which Python version do we have?",
"Could you look up the Python version that's installed here?",
"Determine the currently installed Python version",
"What's the Python version on this machine?",
"Please check the Python version"
]
},
{
"id": "search-for-patterns",
@@ -49,13 +105,30 @@
}
},
"expected_actions": [
{ "tool": "search", "args_pattern": { "query": "test_" } }
{
"tool": "search",
"args_pattern": {
"query": "test_"
}
}
],
"match_mode": "subset"
"match_mode": "subset",
"user_prompts": [
"Find all functions that start with 'test_' in the project",
"List every function in the project whose name begins with 'test_'",
"I need to locate all functions prefixed with 'test_' across the project",
"Could you search the project for any functions starting with 'test_'?",
"Show me all the test_ prefixed functions in this project",
"Hunt down every function that has a 'test_' prefix in the codebase",
"I'm looking for all functions named test_* throughout the project",
"Search the entire project for functions whose names start with test_",
"What functions beginning with 'test_' exist in this project?",
"Please identify all functions with the 'test_' prefix in the project files"
]
},
{
"id": "multi-file-edit",
"description": "Read and edit multiple files must read before editing each, and edit both",
"description": "Read and edit multiple files \u2014 must read before editing each, and edit both",
"user_prompt": "Change the default port from 8000 to 9000 in both server.py and config.py",
"setup": {
"files": {
@@ -64,12 +137,32 @@
}
},
"expected_actions": [
{ "tool": "read_file" },
{ "tool": "read_file" },
{ "tool": "edit_file" },
{ "tool": "edit_file" }
{
"tool": "read_file"
},
{
"tool": "read_file"
},
{
"tool": "edit_file"
},
{
"tool": "edit_file"
}
],
"match_mode": "ordered_subset"
"match_mode": "ordered_subset",
"user_prompts": [
"Change the default port from 8000 to 9000 in both server.py and config.py",
"Update the default port to 9000 instead of 8000 in server.py and config.py",
"Could you modify the port number from 8000 to 9000 in both config.py and server.py?",
"Please replace port 8000 with 9000 in server.py and config.py",
"I need the default port switched from 8000 to 9000 in both server.py and config.py",
"In server.py and config.py, the default port should be changed from 8000 to 9000",
"Swap out port 8000 for 9000 in config.py and server.py",
"Would you mind updating the default port value from 8000 to 9000 across both server.py and config.py?",
"The default port in server.py and config.py needs to be 9000 instead of 8000 \u2014 please make that change",
"Go ahead and change 8000 to 9000 for the default port in both server.py and config.py"
]
},
{
"id": "search-then-edit",
@@ -83,51 +176,147 @@
}
},
"expected_actions": [
{ "tool": "search", "args_pattern": { "query": "MAX_RETRIES" } },
{ "tool": "read_file" },
{ "tool": "edit_file", "args_pattern": { "old_string": "3" } }
{
"tool": "search",
"args_pattern": {
"query": "MAX_RETRIES"
}
},
{
"tool": "read_file"
},
{
"tool": "edit_file",
"args_pattern": {
"old_string": "3"
}
}
],
"match_mode": "ordered_subset"
"match_mode": "ordered_subset",
"user_prompts": [
"Find where MAX_RETRIES is defined and change it from 3 to 5",
"Locate the definition of MAX_RETRIES and update its value from 3 to 5",
"Could you search for where MAX_RETRIES is defined and modify it from 3 to 5?",
"I need MAX_RETRIES changed from 3 to 5 \u2014 find where it's defined and update it",
"Please find the MAX_RETRIES definition and bump it from 3 to 5",
"Hunt down MAX_RETRIES in the codebase and change its value from 3 to 5",
"Where is MAX_RETRIES set to 3? Change it to 5.",
"Search the code for the MAX_RETRIES definition and alter it from 3 to 5",
"I'd like you to locate MAX_RETRIES (currently 3) and set it to 5 instead",
"Go find MAX_RETRIES and switch it from 3 to 5"
]
},
{
"id": "bash-git-log",
"description": "Use bash for git commands, not other tools",
"user_prompt": "Show me the git log for the last 5 commits",
"expected_actions": [
{ "tool": "bash", "args_pattern": { "command": "git\\s+log" } }
{
"tool": "bash",
"args_pattern": {
"command": "git\\s+log"
}
}
],
"match_mode": "subset"
"match_mode": "subset",
"user_prompts": [
"Show me the git log for the last 5 commits",
"Display the 5 most recent git commits",
"Can you pull up the git log limited to the last five commits?",
"I need to see the git log showing only the previous 5 commits",
"git log for the 5 latest commits, please",
"Would you mind showing me the last five entries in the git log?",
"Print out the most recent 5 commits from the git log",
"I'd like to review the git log \u2014 just the last 5 commits",
"Show the recent 5 commit history using git log",
"Could you display the git commit history for the past five commits?"
]
},
{
"id": "write-then-run",
"description": "Create a script and run it to verify it works",
"user_prompt": "Create a Python script called fib.py that prints the first 10 Fibonacci numbers, then run it to verify",
"expected_actions": [
{ "tool": "write_file", "args_pattern": { "path": "fib\\.py" } },
{ "tool": "bash", "args_pattern": { "command": "python" } }
{
"tool": "write_file",
"args_pattern": {
"path": "fib\\.py"
}
},
{
"tool": "bash",
"args_pattern": {
"command": "python"
}
}
],
"match_mode": "ordered_subset"
"match_mode": "ordered_subset",
"user_prompts": [
"Create a Python script called fib.py that prints the first 10 Fibonacci numbers, then run it to verify",
"Write a Python file named fib.py that outputs the first 10 Fibonacci numbers, and execute it to confirm it works",
"Please make fib.py \u2013 a Python script printing the first ten Fibonacci numbers \u2013 then run it to check the output",
"I need a Python script fib.py that prints the first 10 Fibonacci numbers. Execute it afterwards to verify correctness.",
"Could you create fib.py to display the first 10 Fibonacci numbers in Python, and then run it to make sure it works?",
"Draft a script called fib.py in Python that outputs the first ten Fibonacci numbers, then execute it to validate",
"Hey, write me a fib.py that prints the first 10 Fibonacci numbers and run it so we can see it works",
"Generate a Python program fib.py which prints the initial 10 Fibonacci numbers, and verify by running it",
"Kindly produce a Python script named fib.py to print the first 10 Fibonacci numbers, then execute the script to confirm its output",
"Make a file fib.py containing Python code to print the first 10 Fibonacci numbers. Then run it to verify."
]
},
{
"id": "no-bash-for-file-write",
"description": "Should NOT use bash (echo/cat/heredoc) to create files only write_file",
"description": "Should NOT use bash (echo/cat/heredoc) to create files \u2014 only write_file",
"user_prompt": "Create a new file called README.md with a title and description of this project",
"expected_actions": [
{ "tool": "write_file", "args_pattern": { "path": "README" } }
{
"tool": "write_file",
"args_pattern": {
"path": "README"
}
}
],
"match_mode": "subset"
"match_mode": "subset",
"user_prompts": [
"Create a new file called README.md with a title and description of this project",
"Make a README.md file that includes a project title and description",
"I need a README.md created with a title and a brief description of the project",
"Please generate a README.md file containing the project's title and description",
"Could you set up a README.md with a title and project description?",
"Write a README.md that has a title and describes this project",
"Go ahead and create README.md \u2014 it should have a title and a description of the project",
"I'd like you to produce a new README.md file featuring a project title and description",
"Kindly establish a README.md file incorporating both a title and a description for this project",
"Spin up a README.md with a project title and description in it"
]
},
{
"id": "plan-before-refactor",
"description": "Use the plan tool before a large refactoring task",
"user_prompt": "I need to refactor this codebase to separate the database layer from the API layer. Use the plan tool to think through the approach before making any changes.",
"id": "plan-when-asked",
"description": "Call the plan tool when the user asks to plan",
"user_prompt": "Plan how to add user authentication to this app.",
"setup": {
"files": {
"app.py": "import sqlite3\nfrom flask import Flask, jsonify\n\napp = Flask(__name__)\nDB = 'data.db'\n\ndef get_db():\n return sqlite3.connect(DB)\n\n@app.route('/users')\ndef list_users():\n db = get_db()\n users = db.execute('SELECT * FROM users').fetchall()\n db.close()\n return jsonify(users)\n\n@app.route('/users/<int:uid>')\ndef get_user(uid):\n db = get_db()\n user = db.execute('SELECT * FROM users WHERE id=?', (uid,)).fetchone()\n db.close()\n return jsonify(user)\n\nif __name__ == '__main__':\n app.run(port=8000)\n"
"app.py": "from flask import Flask, jsonify\n\napp = Flask(__name__)\n\n@app.route('/users')\ndef list_users():\n return jsonify([{'id': 1, 'name': 'Alice'}])\n\nif __name__ == '__main__':\n app.run(port=8000)\n"
}
},
"expected_actions": [{ "tool": "create_plan" }],
"match_mode": "subset"
"expected_actions": [
{
"tool": "plan_agent"
}
],
"match_mode": "subset",
"user_prompts": [
"Plan how to add user authentication to this app.",
"Make a plan for adding pagination to the API endpoints.",
"Plan out how to add error handling to this application.",
"I need a plan for adding logging to this codebase.",
"Plan the approach for adding unit tests to this app.",
"How would you approach adding user authentication to this app? Lay out a plan.",
"I'd like you to outline a strategy for implementing user authentication in this application.",
"Could you come up with a plan for integrating user authentication into this app?",
"Think through the steps needed to add user auth to this app and present a plan.",
"Draft a plan for incorporating user authentication functionality into this application."
]
},
{
"id": "edit-not-rewrite",
@@ -139,10 +328,32 @@
}
},
"expected_actions": [
{ "tool": "read_file", "args": { "path": "utils.py" } },
{ "tool": "edit_file", "args": { "path": "utils.py" } }
{
"tool": "read_file",
"args": {
"path": "utils.py"
}
},
{
"tool": "edit_file",
"args": {
"path": "utils.py"
}
}
],
"match_mode": "ordered_subset"
"match_mode": "ordered_subset",
"user_prompts": [
"Add a docstring to the process_data function in utils.py",
"Please add a docstring to the process_data function in utils.py",
"Could you write a docstring for process_data in utils.py?",
"Insert a docstring into the process_data function found in utils.py",
"I need a docstring added to process_data in utils.py",
"Put a docstring on the process_data function in utils.py",
"The process_data function in utils.py is missing a docstring \u2014 please add one",
"Would you mind adding a docstring to process_data in utils.py?",
"In utils.py, the process_data function needs a docstring",
"Add documentation via a docstring to the process_data function within utils.py"
]
},
{
"id": "bash-run-tests",
@@ -154,45 +365,130 @@
}
},
"expected_actions": [
{ "tool": "bash", "args_pattern": { "command": "pytest|python.*test" } }
{
"tool": "bash",
"args_pattern": {
"command": "pytest|python.*test"
}
}
],
"match_mode": "subset"
"match_mode": "subset",
"user_prompts": [
"Run the tests",
"Execute the test suite",
"Please go ahead and run the tests",
"Could you run the tests for me?",
"I need the tests to be run",
"Kick off the tests",
"Let's run the tests",
"Fire up the tests",
"Go ahead and execute the tests",
"I'd like you to run the tests"
]
},
{
"id": "web-fetch-url",
"description": "Use web_fetch when asked to retrieve content from a URL",
"user_prompt": "Fetch the contents of https://example.com and summarize what's on the page",
"expected_actions": [
{ "tool": "web_fetch", "args_pattern": { "url": "example\\.com" } }
{
"tool": "web_fetch",
"args_pattern": {
"url": "example\\.com"
}
}
],
"match_mode": "subset"
"match_mode": "subset",
"user_prompts": [
"Fetch the contents of https://example.com and summarize what's on the page",
"Go to https://example.com and give me a summary of what you find there",
"Could you pull up https://example.com and tell me what the page is about?",
"Retrieve the content from https://example.com, then provide a summary of it",
"I need you to grab https://example.com and summarize its contents for me",
"Please access https://example.com and give me an overview of the page",
"What's on https://example.com? Fetch it and summarize for me.",
"Download the page at https://example.com and provide a brief summary",
"I'd like a summary of whatever is at https://example.com \u2014 please fetch it first",
"Hit https://example.com and let me know what's there in summary form"
]
},
{
"id": "man-page-lookup",
"description": "Use man tool to look up command documentation",
"user_prompt": "Look up the man page for tar and tell me what the --xattrs flag does",
"expected_actions": [
{ "tool": "man", "args_pattern": { "page": "tar" } }
{
"tool": "man",
"args_pattern": {
"page": "tar"
}
}
],
"match_mode": "subset"
"match_mode": "subset",
"user_prompts": [
"Look up the man page for tar and tell me what the --xattrs flag does",
"What does the --xattrs flag do in tar? Check the man page for me.",
"Could you pull up the man page for tar and explain the --xattrs option?",
"I need to know what --xattrs does in tar \u2014 can you check the man page?",
"Check tar's man page and let me know the purpose of the --xattrs flag.",
"Please consult the tar man page and describe what the --xattrs flag is for.",
"Hey, look at the tar man page real quick \u2014 what's --xattrs do?",
"I'd like you to read the tar man page and summarize the --xattrs option for me.",
"Would you mind checking the man page for tar to find out what --xattrs means?",
"Look into the tar manual and explain the --xattrs flag to me."
]
},
{
"id": "math-calculation",
"description": "Use the math tool for precise calculations, not bash or mental math",
"user_prompt": "What is 2^64 - 1? Use the math tool to calculate it precisely.",
"expected_actions": [
{ "tool": "math", "args_pattern": { "code": "2.*64" } }
{
"tool": "math",
"args_pattern": {
"code": "2.*64"
}
}
],
"match_mode": "subset"
"match_mode": "subset",
"user_prompts": [
"What is 2^64 - 1? Use the math tool to calculate it precisely.",
"Calculate 2^64 - 1 for me using the math tool, please.",
"I need the exact value of 2^64 - 1. Please use the math tool.",
"Could you use the math tool to compute 2^64 minus 1 precisely?",
"Use the math tool to tell me what 2^64 - 1 equals.",
"I'm curious: what's 2^64 - 1? Compute it with the math tool.",
"Please precisely determine 2^64 - 1 via the math tool.",
"Mind using the math tool to figure out 2^64 - 1 exactly?",
"I'd like to know the precise result of 2^64 - 1 \u2014 use the math tool for this.",
"Leverage the math tool to give me an exact answer for 2^64 - 1."
]
},
{
"id": "web-search-query",
"description": "Use web_search for general knowledge lookups, not web_fetch",
"user_prompt": "Search the web for the current population of Tokyo",
"expected_actions": [
{ "tool": "web_search", "args_pattern": { "query": "Tokyo" } }
{
"tool": "web_search",
"args_pattern": {
"query": "Tokyo"
}
}
],
"match_mode": "subset"
"match_mode": "subset",
"user_prompts": [
"Search the web for the current population of Tokyo",
"What's Tokyo's current population? Look it up on the web.",
"Could you do a web search to find out how many people currently live in Tokyo?",
"Please search online for Tokyo's present-day population.",
"I need you to look up the current population of Tokyo on the web.",
"Find me Tokyo's current population via a web search.",
"Web search: what is the current population of Tokyo?",
"I'd like to know Tokyo's current population\u2014can you search the web for that?",
"Look up how many people live in Tokyo right now using a web search.",
"Do a web search for the population of Tokyo as of now."
]
}
]
}
+75 -1
View File
@@ -1,11 +1,23 @@
from __future__ import annotations
import os
from unittest.mock import MagicMock
import pytest
def pytest_addoption(parser: pytest.Parser) -> None:
parser.addoption(
"--storage-backend",
default="sqlite",
choices=["sqlite", "postgresql"],
help="Storage backend for integration tests (default: sqlite)",
)
@pytest.fixture
def tmp_db(tmp_path):
"""Provide a temporary SQLite storage backend."""
"""Provide a temporary SQLite storage backend (singleton registry)."""
from turnstone.core.storage import init_storage, reset_storage
db_path = str(tmp_path / "test.db")
@@ -15,6 +27,68 @@ def tmp_db(tmp_path):
reset_storage()
@pytest.fixture
def storage_backend(request, tmp_path):
"""Shared storage backend fixture — respects --storage-backend flag.
Returns a StorageBackend instance (SQLite or PostgreSQL).
Tests that use this fixture run against whichever backend CI selects.
"""
from turnstone.core.storage import init_storage, reset_storage
backend_type = request.config.getoption("--storage-backend")
reset_storage()
if backend_type == "postgresql":
pg_url = os.environ.get(
"TURNSTONE_TEST_PG_URL",
"postgresql+psycopg://postgres:postgres@localhost:5432/turnstone_test",
)
backend = init_storage("postgresql", url=pg_url, run_migrations=False)
yield backend
# Truncate all tables between tests — faster than DELETE and resets
# autoincrement sequences. CASCADE handles any future FK constraints.
# NOTE: accesses backend._engine (SQLAlchemy internal) — both SQLite
# and PostgreSQL backends expose this. If a non-SQLAlchemy backend is
# ever added, this cleanup will need a protocol-level hook.
try:
import sqlalchemy as sa
from turnstone.core.storage._schema import metadata as db_metadata
with backend._engine.connect() as conn:
table_names = ", ".join(t.name for t in reversed(db_metadata.sorted_tables))
conn.execute(sa.text(f"TRUNCATE {table_names} RESTART IDENTITY CASCADE"))
conn.commit()
except Exception:
pass # best-effort cleanup; reset_storage disposes engine
finally:
reset_storage()
else:
db_path = str(tmp_path / "test.db")
backend = init_storage("sqlite", path=db_path, run_migrations=False)
yield backend
reset_storage()
@pytest.fixture
def backend(storage_backend):
"""Alias for storage_backend — used by test_storage_sqlite.py etc."""
return storage_backend
@pytest.fixture
def db(storage_backend):
"""Alias for storage_backend — used by domain-specific storage tests."""
return storage_backend
@pytest.fixture
def storage(storage_backend):
"""Alias for storage_backend — used by services/skill resource tests."""
return storage_backend
@pytest.fixture
def mock_openai_client():
"""Return a minimal mock OpenAI client."""
+1
View File
@@ -19,6 +19,7 @@ class TestServerVersioning:
mock_mgr = MagicMock()
mock_mgr.list_all.return_value = []
mock_mgr.max_workstreams = 10
app = create_app(
workstreams=mock_mgr,
global_queue=queue.Queue(),
+56 -2
View File
@@ -783,6 +783,7 @@ class TestServerAuth:
mock_ws.session = mock_session
mock_mgr = MagicMock()
mock_mgr.list_all.return_value = [mock_ws]
mock_mgr.max_workstreams = 10
app = srv_mod.create_app(
workstreams=mock_mgr,
@@ -1001,6 +1002,7 @@ class TestServerLogin:
mock_ws.session = mock_session
mock_mgr = MagicMock()
mock_mgr.list_all.return_value = [mock_ws]
mock_mgr.max_workstreams = 10
app = srv_mod.create_app(
workstreams=mock_mgr,
@@ -1235,6 +1237,52 @@ class TestJWTAudienceIssuer:
result = validate_jwt(token, self.SECRET, audience="")
assert result is not None
def test_create_jwt_expiry_seconds(self):
import jwt as pyjwt
from turnstone.core.auth import create_jwt
token = create_jwt("user1", frozenset({"read"}), "test", self.SECRET, expiry_seconds=300)
payload = pyjwt.decode(
token, self.SECRET, algorithms=["HS256"], options={"verify_aud": False}
)
assert payload["exp"] - payload["iat"] == 300
def test_create_jwt_expiry_seconds_overrides_hours(self):
import jwt as pyjwt
from turnstone.core.auth import create_jwt
token = create_jwt(
"user1",
frozenset({"read"}),
"test",
self.SECRET,
expiry_hours=24,
expiry_seconds=60,
)
payload = pyjwt.decode(
token, self.SECRET, algorithms=["HS256"], options={"verify_aud": False}
)
# expiry_seconds takes precedence over expiry_hours
assert payload["exp"] - payload["iat"] == 60
def test_create_jwt_expiry_seconds_rejects_zero(self):
import pytest
from turnstone.core.auth import create_jwt
with pytest.raises(ValueError, match="expiry_seconds must be positive"):
create_jwt("user1", frozenset({"read"}), "test", self.SECRET, expiry_seconds=0)
def test_create_jwt_expiry_seconds_rejects_negative(self):
import pytest
from turnstone.core.auth import create_jwt
with pytest.raises(ValueError, match="expiry_seconds must be positive"):
create_jwt("user1", frozenset({"read"}), "test", self.SECRET, expiry_seconds=-1)
class TestServiceTokenManager:
SECRET = "test-secret-that-is-at-least-32-chars"
@@ -1369,8 +1417,11 @@ class TestCorsConfigurable:
import turnstone.server as srv_mod
mgr = MagicMock()
mgr.list_all.return_value = []
mgr.max_workstreams = 10
app = srv_mod.create_app(
workstreams=MagicMock(),
workstreams=mgr,
global_queue=queue.Queue(),
global_listeners=[],
global_listeners_lock=threading.Lock(),
@@ -1388,8 +1439,11 @@ class TestCorsConfigurable:
import turnstone.server as srv_mod
mgr = MagicMock()
mgr.list_all.return_value = []
mgr.max_workstreams = 10
app = srv_mod.create_app(
workstreams=MagicMock(),
workstreams=mgr,
global_queue=queue.Queue(),
global_listeners=[],
global_listeners_lock=threading.Lock(),
+5 -5
View File
@@ -130,7 +130,7 @@ class TestParseScopes:
class TestJWT:
SECRET = "test-secret-key-for-jwt"
SECRET = "test-secret-key-for-jwt-min-32b!"
def test_round_trip(self):
scopes = frozenset({"read", "write"})
@@ -155,7 +155,7 @@ class TestJWT:
def test_invalid_signature(self):
token = create_jwt("user1", frozenset({"read"}), "db", self.SECRET)
assert validate_jwt(token, "wrong-secret") is None
assert validate_jwt(token, "wrong-secret-key-for-jwt-min-32b") is None
def test_malformed_token(self):
assert validate_jwt("not.a.jwt", self.SECRET) is None
@@ -217,7 +217,7 @@ class TestAuthenticateToken:
assert result.scopes == frozenset({"read", "write", "approve"})
def test_jwt_token(self):
secret = "test-secret"
secret = "test-secret-key-for-jwt-min-32b!"
jwt_tok = create_jwt("user1", frozenset({"read", "write"}), "db", secret)
cfg = AuthConfig(enabled=True)
result = _authenticate_token(jwt_tok, cfg, jwt_secret=secret)
@@ -304,7 +304,7 @@ class TestCheckRequestScopes:
assert result.has_scope("approve")
def test_jwt_with_scopes(self):
secret = "test"
secret = "test-secret-key-for-jwt-min-32b!"
jwt_tok = create_jwt("u1", frozenset({"read", "write"}), "db", secret)
cfg = AuthConfig(enabled=True)
allowed, status, msg, result = check_request(
@@ -319,7 +319,7 @@ class TestCheckRequestScopes:
assert result.user_id == "u1"
def test_jwt_insufficient_scope(self):
secret = "test"
secret = "test-secret-key-for-jwt-min-32b!"
jwt_tok = create_jwt("u1", frozenset({"read"}), "db", secret)
cfg = AuthConfig(enabled=True)
allowed, status, msg, _ = check_request(
+357
View File
@@ -0,0 +1,357 @@
"""Stress tests for bridge.py threading — race conditions in approval,
plan review, and workstream lifecycle.
Each scenario is run many times (ITERATIONS) with threading.Barrier to
maximize timing overlap. Uses mock broker (no Redis) and no HTTP calls.
Races tested:
1. Duplicate approval on SSE reconnect (TOCTOU in _pending_approvals)
2. Duplicate plan review on SSE reconnect (TOCTOU in _pending_plan_reviews)
3. approve_set stale reference escape during concurrent update
4. _running flag visibility across threads on shutdown
5. Approval thread exits within bounded time after timeout
6. Concurrent approval + workstream close leaves no orphaned state
"""
from __future__ import annotations
import threading
import time
from collections import Counter
from unittest.mock import MagicMock, patch
from turnstone.mq.bridge import Bridge
ITERATIONS = 100
# ---------------------------------------------------------------------------
# Helpers
# ---------------------------------------------------------------------------
def _make_bridge(**overrides) -> Bridge:
"""Create a Bridge with a mock broker (no Redis or HTTP)."""
broker = MagicMock()
defaults = dict(
server_url="http://localhost:8080",
broker=broker,
node_id="test-node",
approval_timeout=1,
)
defaults.update(overrides)
bridge = Bridge(**defaults)
# Replace real httpx client with a mock so daemon threads spawned by
# _handle_approval / _handle_plan_review don't make real HTTP calls
# after the test's patch context exits.
bridge._http.close()
bridge._http = MagicMock()
return bridge
def _approval_items(tool_name: str = "bash") -> list[dict]:
return [{"func_name": tool_name, "needs_approval": True, "approval_label": tool_name}]
def _wait_pending_resolved(bridge: Bridge, key: str, attr: str, deadline_s: float = 3.0) -> bool:
"""Poll until the pending entry is resolved (tombstone) or absent."""
deadline = time.monotonic() + deadline_s
while time.monotonic() < deadline:
with bridge._lock:
entries = getattr(bridge, attr)
if key not in entries:
return True
_, resolved_at = entries[key]
if resolved_at > 0:
return True
time.sleep(0.01)
return False
# ---------------------------------------------------------------------------
# Race 1: Duplicate approval on SSE reconnect
# ---------------------------------------------------------------------------
class TestDuplicateApproval:
"""Two threads call _handle_approval for the same ws_id simultaneously.
Only one should create a pending entry; the other should be skipped."""
def test_no_duplicate_approvals(self):
sent_count = Counter()
for _ in range(ITERATIONS):
bridge = _make_bridge()
bridge._broker.pop_response.return_value = '{"type": "approve", "approved": true}'
barrier = threading.Barrier(2, timeout=5)
def _call_approval(bridge=bridge, barrier=barrier):
barrier.wait()
bridge._handle_approval("ws-1", {"items": _approval_items()})
t1 = threading.Thread(target=_call_approval)
t2 = threading.Thread(target=_call_approval)
with (
patch.object(bridge, "_api_approve") as mock_approve,
patch.object(bridge, "_publish_ws"),
):
t1.start()
t2.start()
t1.join(timeout=5)
t2.join(timeout=5)
assert not t1.is_alive(), "Thread 1 hung"
assert not t2.is_alive(), "Thread 2 hung"
# Wait for spawned _wait_approval threads to resolve
_wait_pending_resolved(bridge, "ws-1", "_pending_approvals")
sent_count[mock_approve.call_count] += 1
# At most 1 approval should be forwarded per iteration
assert sent_count.get(2, 0) == 0, (
f"Duplicate approvals sent in {sent_count[2]}/{ITERATIONS} iterations"
)
# ---------------------------------------------------------------------------
# Race 2: Duplicate plan review on SSE reconnect
# ---------------------------------------------------------------------------
class TestDuplicatePlanReview:
"""Two threads call _handle_plan_review simultaneously.
Only one should create a pending entry."""
def test_no_duplicate_plan_reviews(self):
sent_count = Counter()
for _ in range(ITERATIONS):
bridge = _make_bridge()
bridge._broker.pop_response.return_value = (
'{"type": "plan_feedback", "feedback": "looks good"}'
)
barrier = threading.Barrier(2, timeout=5)
def _call_plan(bridge=bridge, barrier=barrier):
barrier.wait()
bridge._handle_plan_review("ws-1", {"content": "plan text"})
t1 = threading.Thread(target=_call_plan)
t2 = threading.Thread(target=_call_plan)
with patch.object(bridge, "_publish_ws"), patch.object(bridge._http, "post"):
t1.start()
t2.start()
t1.join(timeout=5)
t2.join(timeout=5)
assert not t1.is_alive(), "Thread 1 hung"
assert not t2.is_alive(), "Thread 2 hung"
# Wait for spawned _wait_plan threads to resolve
_wait_pending_resolved(bridge, "ws-1", "_pending_plan_reviews")
sent_count[bridge._http.post.call_count] += 1
assert sent_count.get(2, 0) == 0, (
f"Duplicate plan reviews sent in {sent_count[2]}/{ITERATIONS} iterations"
)
# ---------------------------------------------------------------------------
# Race 3: approve_set stale reference during concurrent update
# ---------------------------------------------------------------------------
class TestApproveSetConsistency:
"""One thread reads approve_set for auto-approve check while another
updates it via _wait_approval 'always' path. The auto-approve
decision should be consistent (either all-approved or not)."""
def test_approve_set_never_partially_visible(self):
for _ in range(ITERATIONS):
bridge = _make_bridge()
with bridge._lock:
bridge._ws_approve_tools["ws-1"] = {"read_file", "search"}
barrier = threading.Barrier(2, timeout=5)
results = []
def _reader(bridge=bridge, barrier=barrier, results=results):
barrier.wait()
with bridge._lock:
snap = bridge._ws_approve_tools.get("ws-1", set()).copy()
results.append(snap)
def _writer(bridge=bridge, barrier=barrier):
barrier.wait()
with bridge._lock:
existing = bridge._ws_approve_tools.get("ws-1", set())
bridge._ws_approve_tools["ws-1"] = existing | {"bash", "write_file"}
t1 = threading.Thread(target=_reader)
t2 = threading.Thread(target=_writer)
t1.start()
t2.start()
t1.join(timeout=5)
t2.join(timeout=5)
assert not t1.is_alive(), "Reader hung"
assert not t2.is_alive(), "Writer hung"
snap = results[0]
assert snap in (
{"read_file", "search"},
{"read_file", "search", "bash", "write_file"},
), f"Partial set observed: {snap}"
# ---------------------------------------------------------------------------
# Race 4: _running flag visibility across threads
# ---------------------------------------------------------------------------
class TestRunningFlagVisibility:
"""All threads reading _running should see False within a bounded time
after the main thread sets it."""
def test_all_threads_observe_shutdown(self):
bridge = _make_bridge()
observed_false = threading.Event()
threads_running = []
def _spin_checker():
while bridge._running:
time.sleep(0.001)
observed_false.set()
for _ in range(5):
t = threading.Thread(target=_spin_checker, daemon=True)
threads_running.append(t)
t.start()
time.sleep(0.01)
bridge._running = False
for t in threads_running:
t.join(timeout=1)
assert not t.is_alive(), "Thread did not observe _running=False"
assert observed_false.is_set()
# ---------------------------------------------------------------------------
# Race 5: Approval thread exits within bounded time
# ---------------------------------------------------------------------------
class TestApprovalThreadTimeout:
"""An approval thread blocked on pop_response should exit within the
configured approval_timeout, not hang indefinitely."""
def test_approval_thread_exits_within_timeout(self):
for _ in range(10):
bridge = _make_bridge(approval_timeout=0.5)
def _slow_pop(queue_name, timeout=300):
time.sleep(min(timeout, 0.5))
return None
bridge._broker.pop_response.side_effect = _slow_pop
with patch.object(bridge, "_publish_ws"), patch.object(bridge, "_api_approve"):
bridge._handle_approval("ws-1", {"items": _approval_items()})
# The pending entry should be resolved within the timeout
resolved = _wait_pending_resolved(bridge, "ws-1", "_pending_approvals", deadline_s=3.0)
assert resolved, "Approval thread did not exit within expected timeout"
# ---------------------------------------------------------------------------
# Race 6: Concurrent approval + workstream close
# ---------------------------------------------------------------------------
class TestApprovalDuringClose:
"""An approval arriving at the exact same time as a ws_closed event
should not leave orphaned state."""
def test_no_orphaned_pending_after_close(self):
for _ in range(ITERATIONS):
bridge = _make_bridge(approval_timeout=0.1)
bridge._broker.pop_response.return_value = None # timeout
barrier = threading.Barrier(2, timeout=5)
def _send_approval(bridge=bridge, barrier=barrier):
barrier.wait()
with patch.object(bridge, "_publish_ws"), patch.object(bridge, "_api_approve"):
bridge._handle_approval("ws-1", {"items": _approval_items()})
def _close_ws(bridge=bridge, barrier=barrier):
barrier.wait()
with (
patch.object(bridge, "_publish_global"),
patch.object(bridge, "_publish_cluster"),
):
bridge._handle_global_event({"type": "ws_closed", "ws_id": "ws-1"})
t1 = threading.Thread(target=_send_approval)
t2 = threading.Thread(target=_close_ws)
t1.start()
t2.start()
t1.join(timeout=5)
t2.join(timeout=5)
assert not t1.is_alive(), "Approval thread hung"
assert not t2.is_alive(), "Close thread hung"
# Wait for spawned _wait_approval thread to resolve (if close
# didn't remove the entry first)
resolved = _wait_pending_resolved(bridge, "ws-1", "_pending_approvals")
assert resolved, "Orphaned pending approval"
# ---------------------------------------------------------------------------
# Race 7: Plan review refinement loop (tombstone → cleanup → re-entry)
# ---------------------------------------------------------------------------
class TestPlanReviewRefinementLoop:
"""After a plan review is resolved, a ws_state event should clean up the
tombstone so the refinement-loop plan_review event is handled correctly."""
def test_refinement_loop_allows_reentry(self):
for _ in range(ITERATIONS):
bridge = _make_bridge()
bridge._broker.pop_response.return_value = (
'{"type": "plan_feedback", "feedback": "refine this"}'
)
# Step 1: first plan review — creates pending entry, resolves it
with patch.object(bridge, "_publish_ws"), patch.object(bridge._http, "post"):
bridge._handle_plan_review("ws-1", {"content": "plan v1"})
_wait_pending_resolved(bridge, "ws-1", "_pending_plan_reviews")
# Verify tombstone is present (resolved_at > 0)
with bridge._lock:
assert "ws-1" in bridge._pending_plan_reviews
assert bridge._pending_plan_reviews["ws-1"][1] > 0
# Step 2: ws_state event cleans up the resolved tombstone
with (
patch.object(bridge, "_publish_ws"),
patch.object(bridge, "_publish_global"),
patch.object(bridge, "_publish_cluster"),
):
bridge._handle_global_event(
{"type": "ws_state", "ws_id": "ws-1", "state": "working"}
)
with bridge._lock:
assert "ws-1" not in bridge._pending_plan_reviews
# Step 3: refinement plan_review arrives — should create new entry
with patch.object(bridge, "_publish_ws"), patch.object(bridge._http, "post"):
bridge._handle_plan_review("ws-1", {"content": "plan v2"})
_wait_pending_resolved(bridge, "ws-1", "_pending_plan_reviews")
with bridge._lock:
assert "ws-1" in bridge._pending_plan_reviews
+412 -10
View File
@@ -1,5 +1,6 @@
"""Tests for generation cancellation (cooperative cancel via threading.Event)."""
import contextlib
import threading
import time
from dataclasses import dataclass, field
@@ -7,7 +8,7 @@ from unittest.mock import MagicMock, patch
import pytest
from turnstone.core.session import ChatSession, GenerationCancelled
from turnstone.core.session import ChatSession, GenerationCancelled, _CancelRef
class NullUI:
@@ -36,7 +37,7 @@ class NullUI:
def approve_tools(self, items):
return True, None
def on_tool_result(self, call_id, name, output):
def on_tool_result(self, call_id, name, output, **kwargs):
pass
def on_tool_output_chunk(self, call_id, chunk):
@@ -179,7 +180,7 @@ class TestCancelDuringToolExecution:
"""Cancel while tools are being executed."""
def test_rollback_incomplete_tool_results(self, tmp_db):
"""When cancelled during tool execution, incomplete results are rolled back."""
"""When cancelled during tool execution, synthesized results replace missing tool outputs."""
ui = NullUI()
session = _make_session(ui=ui)
@@ -235,13 +236,15 @@ class TestCancelDuringToolExecution:
# Session should be idle
assert ui.states[-1] == "idle"
# No tool result messages should remain (rolled back)
roles = [m["role"] for m in session.messages]
assert "tool" not in roles
# The assistant message with tool_calls should also be rolled back
for m in session.messages:
if m["role"] == "assistant":
assert "tool_calls" not in m or not m["tool_calls"]
# Cancelled tool calls should have synthesized results
tool_msgs = [m for m in session.messages if m["role"] == "tool"]
assert len(tool_msgs) == 1
assert tool_msgs[0]["tool_call_id"] == "tc_1"
assert "Cancelled by user" in tool_msgs[0]["content"]
assert tool_msgs[0].get("is_error") is True
# The assistant message with tool_calls should still be present
assistant_msgs = [m for m in session.messages if m.get("tool_calls")]
assert len(assistant_msgs) == 1
class TestCancelWhenIdle:
@@ -407,3 +410,402 @@ class TestStreamFlushBeforeToolCalls:
stream_end_idx = next(i for i, e in enumerate(events) if e[0] == "stream_end")
late_content = [e for e in events[stream_end_idx + 1 :] if e[0] == "content"]
assert late_content == [], f"Content after stream_end: {late_content}"
class TestStreamAbort:
"""Tests for cancel() closing the underlying SDK stream."""
def test_cancel_closes_cancel_stream(self, tmp_db):
"""cancel() calls .close() on the stored SDK stream handle."""
session = _make_session()
mock_stream = MagicMock()
session._cancel_stream = mock_stream
session.cancel()
mock_stream.close.assert_called_once()
assert session._cancel_event.is_set()
def test_cancel_without_stream_is_safe(self, tmp_db):
"""cancel() with no active stream just sets the event."""
session = _make_session()
assert session._cancel_stream is None
session.cancel() # Should not raise
assert session._cancel_event.is_set()
def test_cancel_stream_close_error_suppressed(self, tmp_db):
"""Errors from stream.close() are suppressed."""
session = _make_session()
mock_stream = MagicMock()
mock_stream.close.side_effect = RuntimeError("already closed")
session._cancel_stream = mock_stream
session.cancel() # Should not raise
assert session._cancel_event.is_set()
def test_cancel_ref_populated_after_first_chunk(self, tmp_db):
"""_cancel_ref is populated by the provider after the first chunk
arrives (lazy generator evaluation)."""
ui = NullUI()
session = _make_session(ui=ui)
@dataclass
class FakeChunk:
content_delta: str = ""
reasoning_delta: str = ""
tool_call_deltas: list = field(default_factory=list)
usage: None = None
finish_reason: str = "stop"
info_delta: str = ""
provider_blocks: list = field(default_factory=list)
sdk_stream = MagicMock()
def fake_provider_stream():
# Simulate provider appending to cancel_ref before first yield
session._cancel_ref.append(sdk_stream)
yield FakeChunk(content_delta="hi", finish_reason="stop")
with (
patch.object(
session,
"_create_stream_with_retry",
return_value=fake_provider_stream(),
),
patch.object(session, "_full_messages", return_value=[]),
):
session.send("test")
# After stream completes, cancel_stream should be cleared
assert session._cancel_stream is None
assert len(session._cancel_ref) == 0
def test_transport_error_during_cancel_becomes_generation_cancelled(self, tmp_db):
"""When cancel() closes the stream, the resulting transport error
is converted to GenerationCancelled."""
ui = NullUI()
session = _make_session(ui=ui)
@dataclass
class FakeChunk:
content_delta: str = ""
reasoning_delta: str = ""
tool_call_deltas: list = field(default_factory=list)
usage: None = None
finish_reason: str = ""
info_delta: str = ""
provider_blocks: list = field(default_factory=list)
def stream_that_errors():
yield FakeChunk(content_delta="Hello")
session._cancel_event.set()
raise ConnectionError("stream closed")
with (
patch.object(
session,
"_create_stream_with_retry",
return_value=stream_that_errors(),
),
patch.object(session, "_full_messages", return_value=[]),
):
session.send("test")
# Should complete as cancelled, not error
assert "idle" in ui.states
assert any("cancelled" in i.lower() for i in ui.infos)
# Partial content preserved
assistant_msgs = [m for m in session.messages if m["role"] == "assistant"]
assert len(assistant_msgs) == 1
assert assistant_msgs[0]["content"] == "Hello"
def test_non_cancel_exception_not_swallowed(self, tmp_db):
"""Exceptions during streaming that aren't caused by cancel
should propagate normally."""
ui = NullUI()
session = _make_session(ui=ui)
@dataclass
class FakeChunk:
content_delta: str = ""
reasoning_delta: str = ""
tool_call_deltas: list = field(default_factory=list)
usage: None = None
finish_reason: str = ""
info_delta: str = ""
provider_blocks: list = field(default_factory=list)
def stream_that_errors():
yield FakeChunk(content_delta="Hello")
raise ValueError("unexpected error")
with (
patch.object(
session,
"_create_stream_with_retry",
return_value=stream_that_errors(),
),
patch.object(session, "_full_messages", return_value=[]),
pytest.raises(ValueError, match="unexpected error"),
):
session.send("test")
def test_check_cancelled_between_retries(self, tmp_db):
"""_try_stream checks for cancellation between retry attempts."""
session = _make_session()
session.cancel()
with pytest.raises(GenerationCancelled):
session._try_stream(
client=MagicMock(),
model="test",
msgs=[],
)
class TestCancelRef:
"""Tests for the _CancelRef list proxy."""
def test_append_sets_cancel_stream(self, tmp_db):
"""Appending a stream handle to _CancelRef sets _cancel_stream eagerly."""
session = _make_session()
mock_stream = MagicMock()
assert session._cancel_stream is None
session._cancel_ref.append(mock_stream)
assert session._cancel_stream is mock_stream
def test_append_closes_stream_when_already_cancelled(self, tmp_db):
"""If cancel is already set when a stream is appended, it is closed immediately."""
session = _make_session()
session.cancel() # Set cancel event before stream is created
mock_stream = MagicMock()
session._cancel_ref.append(mock_stream)
mock_stream.close.assert_called_once()
def test_append_does_not_close_stream_when_not_cancelled(self, tmp_db):
"""Stream is not closed if cancel hasn't been requested."""
session = _make_session()
mock_stream = MagicMock()
session._cancel_ref.append(mock_stream)
mock_stream.close.assert_not_called()
assert session._cancel_stream is mock_stream
def test_append_close_error_suppressed(self, tmp_db):
"""Errors from stream.close() during eager close are suppressed."""
session = _make_session()
session.cancel()
mock_stream = MagicMock()
mock_stream.close.side_effect = RuntimeError("already closed")
session._cancel_ref.append(mock_stream) # Should not raise
def test_cancel_ref_is_cancel_ref_instance(self, tmp_db):
"""ChatSession._cancel_ref is a _CancelRef instance."""
session = _make_session()
assert isinstance(session._cancel_ref, _CancelRef)
def test_cancel_ref_cleared_after_stream_ends(self, tmp_db):
"""_cancel_ref is cleared in the send() finally block after streaming."""
ui = NullUI()
session = _make_session(ui=ui)
mock_stream = MagicMock()
session._cancel_ref.append(mock_stream)
assert len(session._cancel_ref) == 1
@dataclass
class FakeChunk:
content_delta: str = ""
reasoning_delta: str = ""
tool_call_deltas: list = field(default_factory=list)
usage: None = None
finish_reason: str = "stop"
info_delta: str = ""
provider_blocks: list = field(default_factory=list)
with (
patch.object(
session,
"_create_stream_with_retry",
return_value=iter([FakeChunk(content_delta="hi", finish_reason="stop")]),
),
patch.object(session, "_full_messages", return_value=[]),
):
session.send("test")
# After send() completes, _cancel_ref is cleared in the finally block
assert len(session._cancel_ref) == 0
class TestForceCancelGeneration:
"""Tests for per-generation tracking that prevents orphaned-thread side-effects."""
def test_check_cancelled_raises_for_orphaned_generation(self, tmp_db):
"""_check_cancelled raises GenerationCancelled when my_generation is stale."""
session = _make_session()
session._generation = 2 # Simulate two generations having run
with pytest.raises(GenerationCancelled):
session._check_cancelled(my_generation=1) # Generation 1 is orphaned
def test_check_cancelled_ok_for_current_generation(self, tmp_db):
"""_check_cancelled does not raise when my_generation matches current."""
session = _make_session()
session._generation = 3
session._check_cancelled(my_generation=3) # Should not raise
def test_force_cancel_orphaned_thread_does_not_mutate_messages(self, tmp_db):
"""An abandoned generation (force-cancel) cannot append to session.messages."""
ui = NullUI()
session = _make_session(ui=ui)
# We can't trivially test the full threading scenario in a unit test,
# so directly verify that _check_cancelled raises when my_generation
# is stale, which is what guards _stream_response against orphaned
# (force-cancelled) threads continuing to mutate messages.
session._generation = 5
with pytest.raises(GenerationCancelled):
session._check_cancelled(my_generation=4) # orphaned generation
def test_new_cancel_event_per_generation_in_send(self, tmp_db):
"""send() replaces _cancel_event with a fresh Event each generation."""
ui = NullUI()
session = _make_session(ui=ui)
@dataclass
class FakeChunk:
content_delta: str = ""
reasoning_delta: str = ""
tool_call_deltas: list = field(default_factory=list)
usage: None = None
finish_reason: str = "stop"
info_delta: str = ""
provider_blocks: list = field(default_factory=list)
original_event = session._cancel_event
with (
patch.object(
session,
"_create_stream_with_retry",
return_value=iter([FakeChunk(content_delta="hi", finish_reason="stop")]),
),
patch.object(session, "_full_messages", return_value=[]),
):
session.send("test")
# After send() completes, _cancel_event should be a NEW Event
# (not the same object as before the call).
assert session._cancel_event is not original_event
assert not session._cancel_event.is_set()
class TestForceCancelThreaded:
"""Force cancel with actual threads — verifies orphaned thread behavior."""
def test_force_cancel_orphan_does_not_mutate_messages(self, tmp_db):
"""After force cancel + new send(), the orphaned thread must not
append stale content to session.messages."""
ui = NullUI()
session = _make_session(ui=ui)
@dataclass
class FakeChunk:
content_delta: str = ""
reasoning_delta: str = ""
tool_call_deltas: list = field(default_factory=list)
usage: None = None
finish_reason: str = ""
info_delta: str = ""
provider_blocks: list = field(default_factory=list)
barrier = threading.Event()
old_done = threading.Event()
def slow_stream():
yield FakeChunk(content_delta="Old content")
barrier.set() # signal: first chunk delivered
time.sleep(2) # simulate stuck stream
yield FakeChunk(content_delta=" more", finish_reason="stop")
# Start generation 1 (will get stuck)
with (
patch.object(session, "_create_stream_with_retry", return_value=slow_stream()),
patch.object(session, "_full_messages", return_value=[]),
):
def run_old():
with contextlib.suppress(Exception):
session.send("old message")
old_done.set()
t1 = threading.Thread(target=run_old, daemon=True)
t1.start()
assert barrier.wait(timeout=5), "stream did not start"
# Force cancel: simulate what the server does
session.cancel()
# Increment generation as new send() would
session._generation += 1
session._cancel_event = threading.Event()
# Wait for old thread to notice generation mismatch and exit
assert old_done.wait(timeout=10), "orphaned thread did not exit"
# The orphaned thread should NOT have appended its content
assistant_msgs = [m for m in session.messages if m["role"] == "assistant"]
# May have partial content from before cancel, but NOT the full
# "Old content more" that would appear without the generation guard
for msg in assistant_msgs:
assert "more" not in msg.get("content", "")
def test_force_cancel_then_new_send_succeeds(self, tmp_db):
"""A new send() after force cancel works cleanly."""
ui = NullUI()
session = _make_session(ui=ui)
@dataclass
class FakeChunk:
content_delta: str = ""
reasoning_delta: str = ""
tool_call_deltas: list = field(default_factory=list)
usage: None = None
finish_reason: str = "stop"
info_delta: str = ""
provider_blocks: list = field(default_factory=list)
barrier = threading.Event()
def stuck_stream():
yield FakeChunk(content_delta="stuck")
barrier.set()
time.sleep(2)
yield FakeChunk(content_delta=" end", finish_reason="stop")
# Start stuck generation
with (
patch.object(session, "_create_stream_with_retry", return_value=stuck_stream()),
patch.object(session, "_full_messages", return_value=[]),
):
t = threading.Thread(target=lambda: session.send("old"), daemon=True)
t.start()
assert barrier.wait(timeout=5), "stream did not start"
# Force cancel
session.cancel()
# New generation should work
fresh_stream = iter([FakeChunk(content_delta="Fresh response")])
with (
patch.object(session, "_create_stream_with_retry", return_value=fresh_stream),
patch.object(session, "_full_messages", return_value=[]),
):
session.send("new message")
# The new generation should have completed successfully
assert "idle" in ui.states
assistant_msgs = [m for m in session.messages if m["role"] == "assistant"]
assert any("Fresh response" in m.get("content", "") for m in assistant_msgs)
+1
View File
@@ -390,6 +390,7 @@ class TestApprovalVerdictDisplay:
bot.config.streaming_edit_interval = 1.5
bot.config.auto_approve = False
bot.config.auto_approve_tools = []
bot.storage = None
bot._streaming = {}
bot._pending_approval_msgs = {}
bot._notify_reply_channels = {}
-11
View File
@@ -2,17 +2,6 @@
from __future__ import annotations
import pytest
from turnstone.core.storage._sqlite import SQLiteBackend
@pytest.fixture
def db(tmp_path):
"""Fresh SQLite backend for each test."""
backend = SQLiteBackend(str(tmp_path / "test.db"))
return backend
class TestChannelUserCRUD:
"""Tests for channel_users table operations."""
+60 -27
View File
@@ -3,54 +3,55 @@
import argparse
import turnstone.core.config as config_mod
from turnstone.core.config import apply_config, load_config
from turnstone.core.config import apply_config, load_config, set_config_path
def _reset_cache():
"""Clear the module-level config cache between tests."""
config_mod._cache = None
config_mod._config_path = None
def test_load_config_missing_file(tmp_path, monkeypatch):
def test_load_config_missing_file(tmp_path):
_reset_cache()
monkeypatch.setattr(config_mod, "CONFIG_PATH", tmp_path / "nope.toml")
set_config_path(str(tmp_path / "nope.toml"))
assert load_config() == {}
def test_load_config_valid_toml(tmp_path, monkeypatch):
def test_load_config_valid_toml(tmp_path):
_reset_cache()
cfg = tmp_path / "config.toml"
cfg.write_text('[redis]\nhost = "10.0.0.1"\nport = 6380\npassword = "secret"\n')
monkeypatch.setattr(config_mod, "CONFIG_PATH", cfg)
set_config_path(str(cfg))
result = load_config()
assert result["redis"]["host"] == "10.0.0.1"
assert result["redis"]["port"] == 6380
assert result["redis"]["password"] == "secret"
def test_load_config_section(tmp_path, monkeypatch):
def test_load_config_section(tmp_path):
_reset_cache()
cfg = tmp_path / "config.toml"
cfg.write_text('[api]\nbase_url = "http://x:8000/v1"\n[redis]\nhost = "y"\n')
monkeypatch.setattr(config_mod, "CONFIG_PATH", cfg)
set_config_path(str(cfg))
assert load_config("redis") == {"host": "y"}
assert load_config("api") == {"base_url": "http://x:8000/v1"}
assert load_config("nonexistent") == {}
def test_load_config_invalid_toml(tmp_path, monkeypatch):
def test_load_config_invalid_toml(tmp_path):
_reset_cache()
cfg = tmp_path / "config.toml"
cfg.write_text("this is not valid toml [[[")
monkeypatch.setattr(config_mod, "CONFIG_PATH", cfg)
set_config_path(str(cfg))
assert load_config() == {}
def test_load_config_caches(tmp_path, monkeypatch):
def test_load_config_caches(tmp_path):
_reset_cache()
cfg = tmp_path / "config.toml"
cfg.write_text('[api]\nbase_url = "http://first"\n')
monkeypatch.setattr(config_mod, "CONFIG_PATH", cfg)
set_config_path(str(cfg))
first = load_config()
assert first["api"]["base_url"] == "http://first"
@@ -60,14 +61,14 @@ def test_load_config_caches(tmp_path, monkeypatch):
assert second["api"]["base_url"] == "http://first"
def test_apply_config_sets_defaults(tmp_path, monkeypatch):
def test_apply_config_sets_defaults(tmp_path):
_reset_cache()
cfg = tmp_path / "config.toml"
cfg.write_text(
'[redis]\nhost = "redis.local"\nport = 7777\npassword = "pw"\n'
'[bridge]\nserver_url = "http://bridge:9090"\n'
)
monkeypatch.setattr(config_mod, "CONFIG_PATH", cfg)
set_config_path(str(cfg))
parser = argparse.ArgumentParser()
parser.add_argument("--redis-host", default="localhost")
@@ -84,11 +85,11 @@ def test_apply_config_sets_defaults(tmp_path, monkeypatch):
assert args.server_url == "http://bridge:9090"
def test_apply_config_cli_overrides(tmp_path, monkeypatch):
def test_apply_config_cli_overrides(tmp_path):
_reset_cache()
cfg = tmp_path / "config.toml"
cfg.write_text('[redis]\nhost = "config-host"\nport = 7777\n')
monkeypatch.setattr(config_mod, "CONFIG_PATH", cfg)
set_config_path(str(cfg))
parser = argparse.ArgumentParser()
parser.add_argument("--redis-host", default="localhost")
@@ -102,11 +103,11 @@ def test_apply_config_cli_overrides(tmp_path, monkeypatch):
assert args.redis_port == 7777 # config wins (no CLI override)
def test_apply_config_missing_keys_keep_defaults(tmp_path, monkeypatch):
def test_apply_config_missing_keys_keep_defaults(tmp_path):
_reset_cache()
cfg = tmp_path / "config.toml"
cfg.write_text('[redis]\nhost = "only-host"\n') # no port, no password
monkeypatch.setattr(config_mod, "CONFIG_PATH", cfg)
set_config_path(str(cfg))
parser = argparse.ArgumentParser()
parser.add_argument("--redis-host", default="localhost")
@@ -121,9 +122,9 @@ def test_apply_config_missing_keys_keep_defaults(tmp_path, monkeypatch):
assert args.redis_password is None # original default kept
def test_apply_config_no_file(tmp_path, monkeypatch):
def test_apply_config_no_file(tmp_path):
_reset_cache()
monkeypatch.setattr(config_mod, "CONFIG_PATH", tmp_path / "nope.toml")
set_config_path(str(tmp_path / "nope.toml"))
parser = argparse.ArgumentParser()
parser.add_argument("--redis-host", default="localhost")
@@ -133,11 +134,11 @@ def test_apply_config_no_file(tmp_path, monkeypatch):
assert args.redis_host == "localhost"
def test_apply_config_model_section(tmp_path, monkeypatch):
def test_apply_config_model_section(tmp_path):
_reset_cache()
cfg = tmp_path / "config.toml"
cfg.write_text('[model]\nname = "qwen-72b"\ntemperature = 0.3\n')
monkeypatch.setattr(config_mod, "CONFIG_PATH", cfg)
set_config_path(str(cfg))
parser = argparse.ArgumentParser()
parser.add_argument("--model", default=None)
@@ -158,7 +159,7 @@ def test_tavily_key_from_config(tmp_path, monkeypatch):
cfg = tmp_path / "config.toml"
cfg.write_text('[api]\ntavily_key = "tvly-from-config"\n')
monkeypatch.setattr(config_mod, "CONFIG_PATH", cfg)
set_config_path(str(cfg))
monkeypatch.delenv("TAVILY_API_KEY", raising=False)
key = config_mod.get_tavily_key()
@@ -174,14 +175,14 @@ def test_tavily_key_fallback_to_env(tmp_path, monkeypatch):
# Config exists but no tavily_key in it
cfg = tmp_path / "config.toml"
cfg.write_text("[api]\n")
monkeypatch.setattr(config_mod, "CONFIG_PATH", cfg)
set_config_path(str(cfg))
monkeypatch.setenv("TAVILY_API_KEY", "tvly-from-env")
key = config_mod.get_tavily_key()
assert key == "tvly-from-env"
def test_apply_config_judge_section(tmp_path, monkeypatch):
def test_apply_config_judge_section(tmp_path):
"""apply_config() loads [judge] section and maps to argparse dests."""
_reset_cache()
cfg = tmp_path / "config.toml"
@@ -193,7 +194,7 @@ def test_apply_config_judge_section(tmp_path, monkeypatch):
"timeout = 30.0\n"
"read_only_tools = false\n"
)
monkeypatch.setattr(config_mod, "CONFIG_PATH", cfg)
set_config_path(str(cfg))
parser = argparse.ArgumentParser()
parser.add_argument("--judge", dest="judge_enabled", action="store_true", default=False)
@@ -212,12 +213,12 @@ def test_apply_config_judge_section(tmp_path, monkeypatch):
assert args.judge_read_only_tools is False
def test_apply_config_judge_cli_overrides(tmp_path, monkeypatch):
def test_apply_config_judge_cli_overrides(tmp_path):
"""CLI flags override config.toml [judge] values."""
_reset_cache()
cfg = tmp_path / "config.toml"
cfg.write_text("[judge]\nenabled = true\nconfidence_threshold = 0.85\n")
monkeypatch.setattr(config_mod, "CONFIG_PATH", cfg)
set_config_path(str(cfg))
parser = argparse.ArgumentParser()
parser.add_argument("--judge", dest="judge_enabled", action="store_true", default=False)
@@ -229,3 +230,35 @@ def test_apply_config_judge_cli_overrides(tmp_path, monkeypatch):
assert args.judge_enabled is False # CLI wins
assert args.judge_confidence == 0.85 # config wins (no CLI override)
def test_set_config_path_overrides_default(tmp_path):
"""set_config_path() overrides the default config location."""
_reset_cache()
cfg = tmp_path / "custom.toml"
cfg.write_text('[api]\nbase_url = "http://custom:9999"\n')
set_config_path(str(cfg))
assert load_config("api") == {"base_url": "http://custom:9999"}
def test_env_var_overrides_default(tmp_path, monkeypatch):
"""$TURNSTONE_CONFIG env var overrides the default config location."""
_reset_cache()
cfg = tmp_path / "env.toml"
cfg.write_text('[api]\nbase_url = "http://env:7777"\n')
monkeypatch.setenv("TURNSTONE_CONFIG", str(cfg))
assert load_config("api") == {"base_url": "http://env:7777"}
def test_set_config_path_overrides_env_var(tmp_path, monkeypatch):
"""set_config_path() takes precedence over $TURNSTONE_CONFIG."""
_reset_cache()
env_cfg = tmp_path / "env.toml"
env_cfg.write_text('[api]\nbase_url = "http://env"\n')
monkeypatch.setenv("TURNSTONE_CONFIG", str(env_cfg))
explicit_cfg = tmp_path / "explicit.toml"
explicit_cfg.write_text('[api]\nbase_url = "http://explicit"\n')
set_config_path(str(explicit_cfg))
assert load_config("api") == {"base_url": "http://explicit"}
+335 -17
View File
@@ -3,7 +3,7 @@
import asyncio
import json
import queue
from unittest.mock import MagicMock
from unittest.mock import MagicMock, patch
import pytest
@@ -47,8 +47,8 @@ class MockBroker:
# ---------------------------------------------------------------------------
def _make_collector(broker=None, poll_interval=999, discovery_interval=999):
"""Create a collector with long intervals so threads don't auto-fire."""
def _make_collector(broker=None, poll_interval=0, discovery_interval=999):
"""Create a collector with zero poll interval (no jitter delay in tests)."""
b = broker or MockBroker()
return ClusterCollector(
broker=b,
@@ -264,6 +264,57 @@ class TestCollectorPolling:
assert q.empty()
assert len(c._nodes["node-a"].workstreams) == 0
def test_poll_401_preserves_workstreams_and_marks_unreachable(self):
"""A 401 from the server must NOT wipe workstream data."""
c = _make_collector()
c._nodes["node-a"] = NodeSnapshot(
node_id="node-a",
server_url="http://a:8080",
reachable=True,
workstreams={"ws1": {"id": "ws1", "name": "existing", "state": "idle"}},
)
# Mock httpx to return 401
import httpx as _httpx
mock_response = _httpx.Response(
401,
json={"error": "Unauthorized"},
request=_httpx.Request("GET", "http://a:8080/v1/api/dashboard"),
)
with patch.object(c._http_client, "get", return_value=mock_response):
c._poll_all_nodes()
# Workstream data must be preserved, node marked unreachable
assert c._nodes["node-a"].reachable is False
assert "ws1" in c._nodes["node-a"].workstreams
assert c._nodes["node-a"].workstreams["ws1"]["name"] == "existing"
def test_poll_403_preserves_workstreams(self):
"""A 403 should also preserve state and mark unreachable."""
c = _make_collector()
c._nodes["node-a"] = NodeSnapshot(
node_id="node-a",
server_url="http://a:8080",
reachable=True,
workstreams={"ws1": {"id": "ws1", "name": "keep-me", "state": "running"}},
)
import httpx as _httpx
mock_response = _httpx.Response(
403,
json={"error": "Forbidden"},
request=_httpx.Request("GET", "http://a:8080/v1/api/dashboard"),
)
with patch.object(c._http_client, "get", return_value=mock_response):
c._poll_all_nodes()
assert c._nodes["node-a"].reachable is False
assert "ws1" in c._nodes["node-a"].workstreams
class TestCollectorEvents:
"""Real-time event handling from cluster channel."""
@@ -902,6 +953,8 @@ class TestConsoleWorkstreamCreation:
],
2,
)
# get_all_nodes delegates to get_nodes (mirrors real implementation)
collector.get_all_nodes.side_effect = lambda: collector.get_nodes.return_value[0]
return collector
@pytest.fixture()
@@ -1050,6 +1103,39 @@ class TestConsoleWorkstreamCreation:
assert resp.status_code == 200
assert resp.json()["target_node"] == "pool"
def test_create_with_resume_ws_directed(self, client_and_broker, mock_collector):
"""resume_ws is forwarded in directed dispatch."""
client, broker = client_and_broker
resp = client.post(
"/v1/api/cluster/workstreams/new",
json={"node_id": "node-a", "resume_ws": "old-ws-id-123"},
)
assert resp.status_code == 200
msg = json.loads(broker.push_inbound.call_args[0][0])
assert msg["resume_ws"] == "old-ws-id-123"
def test_create_with_resume_ws_pool(self, client_and_broker, mock_collector):
"""resume_ws is forwarded in pool dispatch."""
client, broker = client_and_broker
resp = client.post(
"/v1/api/cluster/workstreams/new",
json={"node_id": "pool", "resume_ws": "old-ws-id-456"},
)
assert resp.status_code == 200
msg = json.loads(broker.push_inbound.call_args[0][0])
assert msg["resume_ws"] == "old-ws-id-456"
def test_create_with_resume_ws_auto(self, client_and_broker, mock_collector):
"""resume_ws is forwarded in auto-select dispatch."""
client, broker = client_and_broker
resp = client.post(
"/v1/api/cluster/workstreams/new",
json={"resume_ws": "old-ws-id-789"},
)
assert resp.status_code == 200
msg = json.loads(broker.push_inbound.call_args[0][0])
assert msg["resume_ws"] == "old-ws-id-789"
# ---------------------------------------------------------------------------
# Proxy tests
@@ -1182,47 +1268,49 @@ class TestProxyRewriting:
class TestPickBestNode:
"""Test the _pick_best_node helper."""
@staticmethod
def _mock_collector(nodes: list) -> MagicMock:
collector = MagicMock(spec=ClusterCollector)
collector.get_nodes.return_value = (nodes, len(nodes))
collector.get_all_nodes.side_effect = lambda: collector.get_nodes.return_value[0]
return collector
def test_picks_node_with_most_headroom(self):
from turnstone.console.server import _pick_best_node
collector = MagicMock(spec=ClusterCollector)
collector.get_nodes.return_value = (
collector = self._mock_collector(
[
{"node_id": "busy", "reachable": True, "max_ws": 10, "ws_total": 9},
{"node_id": "free", "reachable": True, "max_ws": 10, "ws_total": 2},
{"node_id": "mid", "reachable": True, "max_ws": 10, "ws_total": 5},
],
3,
]
)
assert _pick_best_node(collector) == "free"
def test_skips_unreachable_nodes(self):
from turnstone.console.server import _pick_best_node
collector = MagicMock(spec=ClusterCollector)
collector.get_nodes.return_value = (
collector = self._mock_collector(
[
{"node_id": "down", "reachable": False, "max_ws": 10, "ws_total": 0},
{"node_id": "up", "reachable": True, "max_ws": 10, "ws_total": 5},
],
2,
]
)
assert _pick_best_node(collector) == "up"
def test_returns_empty_when_no_nodes(self):
from turnstone.console.server import _pick_best_node
collector = MagicMock(spec=ClusterCollector)
collector.get_nodes.return_value = ([], 0)
collector = self._mock_collector([])
assert _pick_best_node(collector) == ""
def test_returns_empty_when_all_unreachable(self):
from turnstone.console.server import _pick_best_node
collector = MagicMock(spec=ClusterCollector)
collector.get_nodes.return_value = (
[{"node_id": "down", "reachable": False, "max_ws": 10, "ws_total": 0}],
1,
collector = self._mock_collector(
[
{"node_id": "down", "reachable": False, "max_ws": 10, "ws_total": 0},
]
)
assert _pick_best_node(collector) == ""
@@ -1611,6 +1699,236 @@ class TestSSEProxy:
asyncio.run(_run())
# ---------------------------------------------------------------------------
# Proxy auth header propagation
# ---------------------------------------------------------------------------
class TestProxyAuthHeaders:
"""Verify _proxy_auth_headers mints user-scoped JWTs for proxy requests."""
SECRET = "test-secret-that-is-at-least-32-chars"
def _make_request(
self, *, auth_result=None, jwt_secret="", proxy_token_mgr=None, proxy_auth_token=""
):
"""Build a minimal fake request for _proxy_auth_headers."""
class _State:
pass
class _AppState:
pass
class _App:
state = _AppState()
class _Request:
state = _State()
app = _App()
req = _Request()
req.state.auth_result = auth_result
req.app.state.jwt_secret = jwt_secret
req.app.state.proxy_token_mgr = proxy_token_mgr
req.app.state.proxy_auth_token = proxy_auth_token
return req
def test_mints_user_jwt(self):
"""Real user auth_result → JWT with correct sub, scopes, src, aud, permissions."""
import jwt as pyjwt
from turnstone.console.server import _proxy_auth_headers
from turnstone.core.auth import JWT_AUD_SERVER, AuthResult
auth = AuthResult(
user_id="alice",
scopes=frozenset({"read", "write"}),
token_source="jwt",
permissions=frozenset({"admin.users"}),
)
req = self._make_request(auth_result=auth, jwt_secret=self.SECRET)
headers = _proxy_auth_headers(req)
assert "Authorization" in headers
token = headers["Authorization"].removeprefix("Bearer ")
payload = pyjwt.decode(token, self.SECRET, algorithms=["HS256"], audience=JWT_AUD_SERVER)
assert payload["sub"] == "alice"
assert set(payload["scopes"].split(",")) == {"read", "write"}
assert payload["src"] == "console-proxy"
assert payload["aud"] == JWT_AUD_SERVER
assert payload["permissions"] == "admin.users"
def test_narrows_scopes(self):
"""Read-only user → JWT carries only read scope, not full {read,write,approve}."""
import jwt as pyjwt
from turnstone.console.server import _proxy_auth_headers
from turnstone.core.auth import JWT_AUD_SERVER, AuthResult
auth = AuthResult(
user_id="viewer",
scopes=frozenset({"read"}),
token_source="jwt",
)
req = self._make_request(auth_result=auth, jwt_secret=self.SECRET)
headers = _proxy_auth_headers(req)
token = headers["Authorization"].removeprefix("Bearer ")
payload = pyjwt.decode(token, self.SECRET, algorithms=["HS256"], audience=JWT_AUD_SERVER)
assert payload["scopes"] == "read"
def test_short_expiry(self):
"""Minted JWT expires in 300 seconds, not hours."""
import jwt as pyjwt
from turnstone.console.server import _proxy_auth_headers
from turnstone.core.auth import AuthResult
auth = AuthResult(
user_id="alice",
scopes=frozenset({"read"}),
token_source="jwt",
)
req = self._make_request(auth_result=auth, jwt_secret=self.SECRET)
headers = _proxy_auth_headers(req)
token = headers["Authorization"].removeprefix("Bearer ")
payload = pyjwt.decode(
token, self.SECRET, algorithms=["HS256"], options={"verify_aud": False}
)
assert payload["exp"] - payload["iat"] == 300
def test_fallback_no_user(self):
"""No auth_result → falls back to ServiceTokenManager."""
from turnstone.console.server import _proxy_auth_headers
from turnstone.core.auth import ServiceTokenManager
mgr = ServiceTokenManager(
user_id="console-proxy",
scopes=frozenset({"read", "write", "approve"}),
source="console",
secret=self.SECRET,
)
req = self._make_request(proxy_token_mgr=mgr)
headers = _proxy_auth_headers(req)
assert "Authorization" in headers
assert headers["Authorization"] == f"Bearer {mgr.token}"
def test_fallback_no_secret(self):
"""auth_result present but empty jwt_secret → falls back to ServiceTokenManager."""
from turnstone.console.server import _proxy_auth_headers
from turnstone.core.auth import AuthResult, ServiceTokenManager
auth = AuthResult(
user_id="alice",
scopes=frozenset({"read"}),
token_source="jwt",
)
mgr = ServiceTokenManager(
user_id="console-proxy",
scopes=frozenset({"read", "write", "approve"}),
source="console",
secret=self.SECRET,
)
req = self._make_request(auth_result=auth, jwt_secret="", proxy_token_mgr=mgr)
headers = _proxy_auth_headers(req)
# Should use ServiceTokenManager, not mint a user JWT
assert headers["Authorization"] == f"Bearer {mgr.token}"
def test_fallback_static_token(self):
"""No auth_result, no ServiceTokenManager → uses static proxy_auth_token."""
from turnstone.console.server import _proxy_auth_headers
req = self._make_request(proxy_auth_token="static-tok-123")
headers = _proxy_auth_headers(req)
assert headers == {"Authorization": "Bearer static-tok-123"}
# ---------------------------------------------------------------------------
# Server: trusted user_id forwarding on create_workstream
# ---------------------------------------------------------------------------
class TestCreateWorkstreamUserIdTrust:
"""Verify that only trusted service tokens can forward user_id in create_workstream."""
def _extract_uid(self, body: dict, auth_result) -> str:
"""Replicate the trust check from server.py:create_workstream."""
auth = auth_result
uid: str = getattr(auth, "user_id", "") or ""
trusted_sources = {"bridge", "console"}
if (
body.get("user_id")
and isinstance(body["user_id"], str)
and auth is not None
and auth.token_source in trusted_sources
):
uid = body["user_id"]
return uid
def test_bridge_can_forward_user_id(self):
from turnstone.core.auth import AuthResult
auth = AuthResult(
user_id="bridge",
scopes=frozenset({"approve"}),
token_source="bridge",
)
uid = self._extract_uid({"user_id": "real-user-abc"}, auth)
assert uid == "real-user-abc"
def test_console_service_can_forward_user_id(self):
from turnstone.core.auth import AuthResult
auth = AuthResult(
user_id="console",
scopes=frozenset({"approve"}),
token_source="console",
)
uid = self._extract_uid({"user_id": "real-user-abc"}, auth)
assert uid == "real-user-abc"
def test_console_proxy_user_cannot_override_user_id(self):
"""End-user tokens via console-proxy must NOT override user_id."""
from turnstone.core.auth import AuthResult
auth = AuthResult(
user_id="real-user-abc",
scopes=frozenset({"read", "write"}),
token_source="console-proxy",
)
uid = self._extract_uid({"user_id": "impersonated-user"}, auth)
# Should use JWT identity, NOT the body override
assert uid == "real-user-abc"
def test_direct_user_cannot_override_user_id(self):
"""Direct JWT login must NOT override user_id."""
from turnstone.core.auth import AuthResult
auth = AuthResult(
user_id="real-user-abc",
scopes=frozenset({"read", "write"}),
token_source="password",
)
uid = self._extract_uid({"user_id": "impersonated-user"}, auth)
assert uid == "real-user-abc"
def test_no_body_user_id_uses_jwt(self):
from turnstone.core.auth import AuthResult
auth = AuthResult(
user_id="bridge",
scopes=frozenset({"approve"}),
token_source="bridge",
)
uid = self._extract_uid({"name": "test-ws"}, auth)
assert uid == "bridge"
# ---------------------------------------------------------------------------
# Collector — MCP aggregation in get_overview()
# ---------------------------------------------------------------------------
+460
View File
@@ -0,0 +1,460 @@
"""Tests for edit_file tool — single edit and batch edit modes."""
from __future__ import annotations
import os
from unittest.mock import MagicMock
import pytest
from turnstone.core.session import ChatSession
@pytest.fixture
def session(tmp_db, mock_openai_client):
"""Create a ChatSession wired to a temp database."""
return ChatSession(
client=mock_openai_client,
model="test-model",
ui=MagicMock(),
instructions=None,
temperature=0.5,
max_tokens=1000,
tool_timeout=10,
)
@pytest.fixture
def sample_file(tmp_path):
"""Create a sample file and return its path."""
p = tmp_path / "test.py"
p.write_text("line1\nline2\nline3\nline4\nline5\n")
return str(p)
def _mark_read(session: ChatSession, path: str) -> None:
"""Simulate a prior read_file so the edit guard passes."""
resolved = os.path.realpath(os.path.expanduser(path))
session._read_files.add(resolved)
# ── Single edit (backward compat) ────────────────────────────────────
class TestSingleEdit:
def test_basic_replace(self, session, sample_file):
_mark_read(session, sample_file)
result = session._prepare_edit_file(
"c1",
{
"path": sample_file,
"old_string": "line2",
"new_string": "replaced",
},
)
assert result["needs_approval"]
assert result["func_name"] == "edit_file"
call_id, msg = session._exec_edit_file(result)
assert call_id == "c1"
assert "applied 1 edit" in msg
with open(sample_file) as f:
assert f.read() == "line1\nreplaced\nline3\nline4\nline5\n"
def test_missing_path(self, session):
result = session._prepare_edit_file(
"c1",
{
"old_string": "a",
"new_string": "b",
},
)
assert result.get("error")
assert "missing path" in result["error"]
def test_missing_old_string(self, session, sample_file):
result = session._prepare_edit_file(
"c1",
{
"path": sample_file,
"new_string": "b",
},
)
assert result.get("error")
assert "old_string" in result["error"]
def test_identical_strings(self, session, sample_file):
_mark_read(session, sample_file)
result = session._prepare_edit_file(
"c1",
{
"path": sample_file,
"old_string": "line1",
"new_string": "line1",
},
)
assert result.get("error")
assert "identical" in result["error"]
def test_old_string_not_found(self, session, sample_file):
_mark_read(session, sample_file)
result = session._prepare_edit_file(
"c1",
{
"path": sample_file,
"old_string": "nonexistent",
"new_string": "replaced",
},
)
assert result.get("error")
assert "not found" in result["error"]
def test_must_read_first(self, session, sample_file):
# Don't call _mark_read — should fail
result = session._prepare_edit_file(
"c1",
{
"path": sample_file,
"old_string": "line1",
"new_string": "replaced",
},
)
assert result.get("error")
assert "must read_file" in result["error"]
def test_multiple_occurrences_without_near_line(self, session, tmp_path):
p = tmp_path / "dup.txt"
p.write_text("foo\nbar\nfoo\n")
path = str(p)
_mark_read(session, path)
result = session._prepare_edit_file(
"c1",
{
"path": path,
"old_string": "foo",
"new_string": "baz",
},
)
assert result.get("error")
assert "found 2 times" in result["error"]
def test_near_line_disambiguates(self, session, tmp_path):
p = tmp_path / "dup.txt"
p.write_text("foo\nbar\nfoo\n")
path = str(p)
_mark_read(session, path)
result = session._prepare_edit_file(
"c1",
{
"path": path,
"old_string": "foo",
"new_string": "baz",
"near_line": 3,
},
)
assert result["needs_approval"]
call_id, msg = session._exec_edit_file(result)
assert "applied 1 edit" in msg
with open(path) as f:
assert f.read() == "foo\nbar\nbaz\n"
def test_deletion(self, session, sample_file):
_mark_read(session, sample_file)
result = session._prepare_edit_file(
"c1",
{
"path": sample_file,
"old_string": "line3\n",
"new_string": "",
},
)
assert result["needs_approval"]
assert "deletion" in result["preview"]
call_id, msg = session._exec_edit_file(result)
with open(sample_file) as f:
assert f.read() == "line1\nline2\nline4\nline5\n"
# ── Batch edits ──────────────────────────────────────────────────────
class TestBatchEdit:
def test_two_edits_applied_atomically(self, session, sample_file):
_mark_read(session, sample_file)
result = session._prepare_edit_file(
"c1",
{
"path": sample_file,
"edits": [
{"old_string": "line1", "new_string": "first"},
{"old_string": "line5", "new_string": "last"},
],
},
)
assert result["needs_approval"]
assert "2 edits" in result["header"]
call_id, msg = session._exec_edit_file(result)
assert "applied 2 edits" in msg
with open(sample_file) as f:
assert f.read() == "first\nline2\nline3\nline4\nlast\n"
def test_three_edits_middle_of_file(self, session, sample_file):
_mark_read(session, sample_file)
result = session._prepare_edit_file(
"c1",
{
"path": sample_file,
"edits": [
{"old_string": "line2", "new_string": "second"},
{"old_string": "line3", "new_string": "third"},
{"old_string": "line4", "new_string": "fourth"},
],
},
)
assert result["needs_approval"]
call_id, msg = session._exec_edit_file(result)
assert "applied 3 edits" in msg
with open(sample_file) as f:
assert f.read() == "line1\nsecond\nthird\nfourth\nline5\n"
def test_overlapping_edits_rejected(self, session, tmp_path):
p = tmp_path / "overlap.txt"
p.write_text("abcdefgh\n")
path = str(p)
_mark_read(session, path)
result = session._prepare_edit_file(
"c1",
{
"path": path,
"edits": [
{"old_string": "abcdef", "new_string": "XXX"},
{"old_string": "defgh", "new_string": "YYY"},
],
},
)
assert result["needs_approval"]
call_id, msg = session._exec_edit_file(result)
assert "overlap" in msg.lower()
# File should be untouched
with open(path) as f:
assert f.read() == "abcdefgh\n"
def test_batch_edit_not_found(self, session, sample_file):
_mark_read(session, sample_file)
result = session._prepare_edit_file(
"c1",
{
"path": sample_file,
"edits": [
{"old_string": "line1", "new_string": "first"},
{"old_string": "nonexistent", "new_string": "oops"},
],
},
)
assert result.get("error")
assert "edits[1]" in result["error"]
assert "not found" in result["error"]
def test_batch_edit_missing_old_string(self, session, sample_file):
result = session._prepare_edit_file(
"c1",
{
"path": sample_file,
"edits": [
{"old_string": "line1", "new_string": "first"},
{"new_string": "oops"},
],
},
)
assert result.get("error")
assert "edits[1]" in result["error"]
assert "old_string" in result["error"]
def test_batch_edit_identical_strings(self, session, sample_file):
result = session._prepare_edit_file(
"c1",
{
"path": sample_file,
"edits": [
{"old_string": "line1", "new_string": "line1"},
],
},
)
assert result.get("error")
assert "identical" in result["error"]
def test_batch_with_near_line(self, session, tmp_path):
p = tmp_path / "dup.txt"
p.write_text("foo\nbar\nfoo\nbaz\n")
path = str(p)
_mark_read(session, path)
result = session._prepare_edit_file(
"c1",
{
"path": path,
"edits": [
{"old_string": "foo", "new_string": "first_foo", "near_line": 1},
{"old_string": "foo", "new_string": "second_foo", "near_line": 3},
],
},
)
assert result["needs_approval"]
call_id, msg = session._exec_edit_file(result)
assert "applied 2 edits" in msg
with open(path) as f:
assert f.read() == "first_foo\nbar\nsecond_foo\nbaz\n"
def test_batch_with_deletion(self, session, sample_file):
_mark_read(session, sample_file)
result = session._prepare_edit_file(
"c1",
{
"path": sample_file,
"edits": [
{"old_string": "line2\n", "new_string": ""},
{"old_string": "line4\n", "new_string": ""},
],
},
)
assert result["needs_approval"]
call_id, msg = session._exec_edit_file(result)
with open(sample_file) as f:
assert f.read() == "line1\nline3\nline5\n"
def test_single_item_edits_array(self, session, sample_file):
"""An edits array with one item should work like a single edit."""
_mark_read(session, sample_file)
result = session._prepare_edit_file(
"c1",
{
"path": sample_file,
"edits": [
{"old_string": "line3", "new_string": "middle"},
],
},
)
assert result["needs_approval"]
# Single edit — no "(N edits)" count in header
assert "edits)" not in result["header"]
call_id, msg = session._exec_edit_file(result)
assert "applied 1 edit" in msg
with open(sample_file) as f:
assert f.read() == "line1\nline2\nmiddle\nline4\nline5\n"
# ── Mutual exclusivity ──────────────────────────────────────────────
class TestMutualExclusivity:
def test_both_single_and_batch_rejected(self, session, sample_file):
_mark_read(session, sample_file)
result = session._prepare_edit_file(
"c1",
{
"path": sample_file,
"old_string": "line1",
"new_string": "replaced",
"edits": [
{"old_string": "line2", "new_string": "also_replaced"},
],
},
)
assert result.get("error")
assert "not both" in result["error"]
def test_neither_single_nor_batch(self, session, sample_file):
result = session._prepare_edit_file(
"c1",
{
"path": sample_file,
},
)
assert result.get("error")
assert "old_string" in result["error"]
def test_empty_edits_array_falls_through_to_single(self, session, sample_file):
"""An empty edits array should be treated as no batch."""
result = session._prepare_edit_file(
"c1",
{
"path": sample_file,
"edits": [],
},
)
# Falls through to single-edit path, which requires old_string
assert result.get("error")
assert "old_string" in result["error"]
# ── TOCTOU edge cases ───────────────────────────────────────────────
class TestExecEdgeCases:
def test_file_changed_between_prepare_and_exec(self, session, sample_file):
_mark_read(session, sample_file)
result = session._prepare_edit_file(
"c1",
{
"path": sample_file,
"old_string": "line2",
"new_string": "replaced",
},
)
assert result["needs_approval"]
# Modify the file after prepare
with open(sample_file, "w") as f:
f.write("completely different content\n")
call_id, msg = session._exec_edit_file(result)
assert "no longer found" in msg
def test_file_deleted_between_prepare_and_exec(self, session, sample_file):
_mark_read(session, sample_file)
result = session._prepare_edit_file(
"c1",
{
"path": sample_file,
"old_string": "line2",
"new_string": "replaced",
},
)
assert result["needs_approval"]
os.unlink(sample_file)
call_id, msg = session._exec_edit_file(result)
assert "Error" in msg
def test_batch_file_changed_partial_match(self, session, sample_file):
"""If file changes so one edit fails, none should be applied."""
_mark_read(session, sample_file)
result = session._prepare_edit_file(
"c1",
{
"path": sample_file,
"edits": [
{"old_string": "line1", "new_string": "first"},
{"old_string": "line5", "new_string": "last"},
],
},
)
assert result["needs_approval"]
# Remove line5 between prepare and exec
with open(sample_file, "w") as f:
f.write("line1\nline2\nline3\nline4\n")
call_id, msg = session._exec_edit_file(result)
assert "no longer found" in msg
# line1 should NOT have been edited (atomic failure)
with open(sample_file) as f:
assert "line1" in f.read()
+149
View File
@@ -0,0 +1,149 @@
"""Tests for turnstone.core.env — subprocess environment scrubbing."""
from __future__ import annotations
import os
from unittest.mock import patch
from turnstone.core.env import _is_safe, _is_secret, scrubbed_env
class TestIsSecret:
def test_explicit_scrub_list(self):
assert _is_secret("OPENAI_API_KEY") is True
assert _is_secret("ANTHROPIC_API_KEY") is True
assert _is_secret("TURNSTONE_JWT_SECRET") is True
assert _is_secret("AWS_SECRET_ACCESS_KEY") is True
def test_suffix_matching(self):
assert _is_secret("MY_CUSTOM_API_KEY") is True
assert _is_secret("DB_PASSWORD") is True
assert _is_secret("AUTH_TOKEN") is True
assert _is_secret("SERVICE_CREDENTIAL") is True
assert _is_secret("GCP_CREDENTIALS") is True
def test_safe_vars_not_secret(self):
assert _is_secret("PATH") is False
assert _is_secret("HOME") is False
assert _is_secret("LANG") is False
def test_no_false_positives_on_substring(self):
"""Suffix matching avoids false positives like MONKEYTYPE."""
assert _is_secret("MONKEYTYPE") is False
assert _is_secret("KEYBOARD_LAYOUT") is False
assert _is_secret("PYTHONPATH") is False
assert _is_secret("EDITOR") is False
assert _is_secret("GOPATH") is False
class TestIsSafe:
def test_safe_names(self):
assert _is_safe("PATH") is True
assert _is_safe("HOME") is True
assert _is_safe("TERM") is True
assert _is_safe("MANWIDTH") is True
def test_safe_prefixes(self):
assert _is_safe("LC_ALL") is True
assert _is_safe("LC_CTYPE") is True
assert _is_safe("XDG_RUNTIME_DIR") is True
def test_non_safe_names(self):
assert _is_safe("OPENAI_API_KEY") is False
assert _is_safe("CUSTOM_VAR") is False
class TestScrubbedEnv:
def test_strips_api_keys(self):
fake_env = {
"PATH": "/usr/bin",
"HOME": "/home/user",
"OPENAI_API_KEY": "sk-secret",
"ANTHROPIC_API_KEY": "ant-secret",
"CUSTOM_VAR": "safe_value",
}
with patch.dict(os.environ, fake_env, clear=True):
result = scrubbed_env()
assert result["PATH"] == "/usr/bin"
assert result["HOME"] == "/home/user"
assert result["CUSTOM_VAR"] == "safe_value"
assert "OPENAI_API_KEY" not in result
assert "ANTHROPIC_API_KEY" not in result
def test_strips_pattern_matched_secrets(self):
fake_env = {
"PATH": "/usr/bin",
"MY_SERVICE_TOKEN": "tok-123",
"DB_PASSWORD": "pass123",
}
with patch.dict(os.environ, fake_env, clear=True):
result = scrubbed_env()
assert "MY_SERVICE_TOKEN" not in result
assert "DB_PASSWORD" not in result
def test_extra_vars_merged(self):
fake_env = {"PATH": "/usr/bin"}
with patch.dict(os.environ, fake_env, clear=True):
result = scrubbed_env(extra={"MANWIDTH": "80"})
assert result["MANWIDTH"] == "80"
assert result["PATH"] == "/usr/bin"
def test_passthrough_overrides_scrub(self):
fake_env = {
"PATH": "/usr/bin",
"OPENAI_API_KEY": "sk-needed",
}
with patch.dict(os.environ, fake_env, clear=True):
result = scrubbed_env(passthrough=["OPENAI_API_KEY"])
assert result["OPENAI_API_KEY"] == "sk-needed"
def test_preserves_locale_vars(self):
fake_env = {
"PATH": "/usr/bin",
"LC_ALL": "en_US.UTF-8",
"LC_CTYPE": "en_US.UTF-8",
}
with patch.dict(os.environ, fake_env, clear=True):
result = scrubbed_env()
assert result["LC_ALL"] == "en_US.UTF-8"
assert result["LC_CTYPE"] == "en_US.UTF-8"
def test_preserves_unknown_non_secret_vars(self):
fake_env = {
"PATH": "/usr/bin",
"PYTHONPATH": "/opt/lib",
"GOPATH": "/home/user/go",
}
with patch.dict(os.environ, fake_env, clear=True):
result = scrubbed_env()
assert result["PYTHONPATH"] == "/opt/lib"
assert result["GOPATH"] == "/home/user/go"
def test_extra_can_reintroduce_scrubbed_var(self):
"""extra= intentionally overrides scrubbing (operator-controlled)."""
fake_env = {"PATH": "/usr/bin", "OPENAI_API_KEY": "sk-original"}
with patch.dict(os.environ, fake_env, clear=True):
result = scrubbed_env(extra={"OPENAI_API_KEY": "sk-injected"})
assert result["OPENAI_API_KEY"] == "sk-injected"
def test_less_prefix_does_not_leak_secrets(self):
"""LESS pager vars are safe but LESS_SECRET_TOKEN is not."""
fake_env = {
"PATH": "/usr/bin",
"LESS": "-R",
"LESSOPEN": "| lesspipe %s",
"LESS_SECRET_TOKEN": "tok-secret",
}
with patch.dict(os.environ, fake_env, clear=True):
result = scrubbed_env()
assert result["LESS"] == "-R"
assert result["LESSOPEN"] == "| lesspipe %s"
assert "LESS_SECRET_TOKEN" not in result
-10
View File
@@ -8,18 +8,8 @@ from __future__ import annotations
from datetime import UTC, datetime
import pytest
import sqlalchemy as sa
from turnstone.core.storage._sqlite import SQLiteBackend
@pytest.fixture()
def db(tmp_path):
"""Create a fresh SQLite backend for each test."""
return SQLiteBackend(str(tmp_path / "test.db"))
# ---------------------------------------------------------------------------
# Roles
# ---------------------------------------------------------------------------
+57
View File
@@ -221,6 +221,63 @@ class TestBackendHealthMonitor:
mon.record_failure()
assert mon.circuit_state == CircuitState.OPEN # type: ignore[comparison-overlap]
def test_probe_loop_autonomous_recovery(
self, mock_client: MagicMock, mock_metrics: MagicMock
) -> None:
"""_probe_loop transitions OPEN → HALF_OPEN → CLOSED without user requests."""
# Use very short intervals so the test is fast
mon = BackendHealthMonitor(
client=mock_client,
probe_interval=0.05,
probe_timeout=1.0,
failure_threshold=1,
cooldown=0.1,
)
# Trip the circuit
mon.record_failure()
assert mon.circuit_state == CircuitState.OPEN
# Backend is healthy — probe_once will succeed
mock_client.with_options.return_value.models.list.return_value = MagicMock()
# Start the probe loop and wait for autonomous recovery
mon.start()
try:
import time
deadline = time.monotonic() + 5.0
while mon.circuit_state != CircuitState.CLOSED and time.monotonic() < deadline:
time.sleep(0.05)
assert mon.circuit_state == CircuitState.CLOSED
# User requests should flow again without anyone calling acquire_request_permit
assert mon.acquire_request_permit() is True
finally:
mon.stop()
if mon._thread:
mon._thread.join(timeout=2.0)
def test_probe_loop_no_user_permit_during_probe(
self, mock_client: MagicMock, mock_metrics: MagicMock
) -> None:
"""While background probe is in HALF_OPEN, user requests are blocked."""
mon = BackendHealthMonitor(
client=mock_client,
probe_interval=0.05,
probe_timeout=1.0,
failure_threshold=1,
cooldown=0.1,
)
mon.record_failure()
assert mon.circuit_state == CircuitState.OPEN
# Force into HALF_OPEN as the probe loop would
with mon._lock:
mon._state = CircuitState.HALF_OPEN
mon._half_open_permit = False # probe consumes it
# User requests should be blocked — only the probe gets through
assert mon.acquire_request_permit() is False
def test_stop_thread(self, mock_client: MagicMock) -> None:
"""stop() signals the probe loop to exit."""
mon = _make_monitor(mock_client)
+29 -20
View File
@@ -4,6 +4,7 @@ from __future__ import annotations
import json
import time
from concurrent.futures import ThreadPoolExecutor
from pathlib import Path
from typing import Any
from unittest.mock import MagicMock
@@ -179,11 +180,13 @@ class TestErrorHandling:
provider = _make_mock_provider(side_effect=RuntimeError("API error"))
judge = _make_judge(provider)
result = judge._evaluate_single(
_make_item(),
[{"role": "user", "content": "test"}],
MagicMock(),
)
with ThreadPoolExecutor(max_workers=1) as pool:
result = judge._evaluate_single(
_make_item(),
[{"role": "user", "content": "test"}],
cancel_event=None,
executor=pool,
)
assert result is None
def test_provider_error_heuristic_still_returned(self):
@@ -212,11 +215,13 @@ class TestErrorHandling:
result_mock.content = ""
judge = _make_judge(provider)
result = judge._evaluate_single(
_make_item(),
[{"role": "user", "content": "test"}],
MagicMock(),
)
with ThreadPoolExecutor(max_workers=1) as pool:
result = judge._evaluate_single(
_make_item(),
[{"role": "user", "content": "test"}],
cancel_event=None,
executor=pool,
)
assert result is None
@@ -256,11 +261,13 @@ class TestMultiTurnToolUse:
provider.create_completion.side_effect = [turn1, turn2]
judge = _make_judge(provider)
verdict = judge._evaluate_single(
_make_item(),
[{"role": "user", "content": "test"}],
MagicMock(),
)
with ThreadPoolExecutor(max_workers=1) as pool:
verdict = judge._evaluate_single(
_make_item(),
[{"role": "user", "content": "test"}],
cancel_event=None,
executor=pool,
)
assert verdict is not None
assert verdict.tier == "llm"
assert provider.create_completion.call_count == 2
@@ -302,11 +309,13 @@ class TestMultiTurnToolUse:
]
judge = _make_judge(provider)
judge._evaluate_single(
_make_item(),
[{"role": "user", "content": "test"}],
MagicMock(),
)
with ThreadPoolExecutor(max_workers=1) as pool:
judge._evaluate_single(
_make_item(),
[{"role": "user", "content": "test"}],
cancel_event=None,
executor=pool,
)
# Should have called create_completion exactly _JUDGE_MAX_TURNS times
assert provider.create_completion.call_count == 5
-10
View File
@@ -4,16 +4,6 @@ from __future__ import annotations
from datetime import UTC, datetime, timedelta
import pytest
from turnstone.core.storage._sqlite import SQLiteBackend
@pytest.fixture()
def db(tmp_path):
"""Fresh SQLite backend for each test."""
return SQLiteBackend(str(tmp_path / "test.db"))
def _make_verdict_kwargs(**overrides):
"""Build default kwargs for create_intent_verdict."""
+145 -46
View File
@@ -1,4 +1,4 @@
"""Tests for the load_skill built-in tool."""
"""Tests for the skill built-in tool."""
from __future__ import annotations
@@ -9,25 +9,25 @@ from turnstone.core.tools import BUILTIN_TOOL_NAMES, PRIMARY_KEY_MAP
class TestToolRegistration:
"""Verify load_skill is registered correctly."""
"""Verify skill is registered correctly."""
def test_in_builtin_tool_names(self) -> None:
assert "load_skill" in BUILTIN_TOOL_NAMES
assert "skill" in BUILTIN_TOOL_NAMES
def test_not_agent_tool(self) -> None:
from turnstone.core.tools import AGENT_TOOLS
names = {t["function"]["name"] for t in AGENT_TOOLS}
assert "load_skill" not in names
assert "skill" not in names
def test_not_task_agent_tool(self) -> None:
from turnstone.core.tools import TASK_AGENT_TOOLS
names = {t["function"]["name"] for t in TASK_AGENT_TOOLS}
assert "load_skill" not in names
assert "skill" not in names
def test_has_primary_key(self) -> None:
assert PRIMARY_KEY_MAP.get("load_skill") == "name"
assert PRIMARY_KEY_MAP.get("skill") == "name"
# ---------------------------------------------------------------------------
@@ -54,6 +54,7 @@ def _make_session(skills: list[dict[str, Any]] | None = None):
session._notify_on_complete = "{}"
session.messages = []
session._config = {}
session._tool_error_flags = {}
# Stub set_skill to just record the call
session._set_skill_called: list[str | None] = []
@@ -82,12 +83,12 @@ def _make_session(skills: list[dict[str, Any]] | None = None):
class TestPrepareLoadSkill:
"""Test _prepare_load_skill validation and item dict shape."""
"""Test _prepare_skill validation and item dict shape."""
def test_load_valid(self) -> None:
session, _, _ = _make_session()
item = session._prepare_load_skill("call-1", {"action": "load", "name": "code-review"})
assert item["func_name"] == "load_skill"
item = session._prepare_skill("call-1", {"action": "load", "name": "code-review"})
assert item["func_name"] == "skill"
assert item["action"] == "load"
assert item["name"] == "code-review"
assert item["needs_approval"] is True
@@ -96,19 +97,19 @@ class TestPrepareLoadSkill:
def test_load_missing_name(self) -> None:
session, _, _ = _make_session()
item = session._prepare_load_skill("call-1", {"action": "load"})
item = session._prepare_skill("call-1", {"action": "load"})
assert "error" in item
assert "name" in item["error"].lower()
assert item["needs_approval"] is False
def test_load_empty_name(self) -> None:
session, _, _ = _make_session()
item = session._prepare_load_skill("call-1", {"action": "load", "name": ""})
item = session._prepare_skill("call-1", {"action": "load", "name": ""})
assert "error" in item
def test_search_with_query(self) -> None:
session, _, _ = _make_session()
item = session._prepare_load_skill("call-1", {"action": "search", "query": "code review"})
item = session._prepare_skill("call-1", {"action": "search", "query": "code review"})
assert item["action"] == "search"
assert item["query"] == "code review"
assert item["needs_approval"] is False
@@ -116,30 +117,30 @@ class TestPrepareLoadSkill:
def test_search_without_query(self) -> None:
session, _, _ = _make_session()
item = session._prepare_load_skill("call-1", {"action": "search"})
item = session._prepare_skill("call-1", {"action": "search"})
assert item["action"] == "search"
assert item["query"] == ""
assert item["needs_approval"] is False
def test_invalid_action(self) -> None:
session, _, _ = _make_session()
item = session._prepare_load_skill("call-1", {"action": "delete"})
item = session._prepare_skill("call-1", {"action": "delete"})
assert "error" in item
assert "delete" in item["error"]
def test_empty_action(self) -> None:
session, _, _ = _make_session()
item = session._prepare_load_skill("call-1", {"action": ""})
item = session._prepare_skill("call-1", {"action": ""})
assert "error" in item
def test_header_for_load(self) -> None:
session, _, _ = _make_session()
item = session._prepare_load_skill("call-1", {"action": "load", "name": "my-skill"})
item = session._prepare_skill("call-1", {"action": "load", "name": "my-skill"})
assert "my-skill" in item["header"]
def test_header_for_search(self) -> None:
session, _, _ = _make_session()
item = session._prepare_load_skill("call-1", {"action": "search", "query": "testing"})
item = session._prepare_skill("call-1", {"action": "search", "query": "testing"})
assert "testing" in item["header"]
@@ -149,7 +150,7 @@ class TestPrepareLoadSkill:
class TestExecLoadSkill:
"""Test _exec_load_skill execution logic."""
"""Test _exec_skill execution logic."""
def test_load_existing_skill(self) -> None:
skills = [
@@ -164,8 +165,8 @@ class TestExecLoadSkill:
session, _, fake_get = _make_session(skills)
with patch("turnstone.core.session.get_skill_by_name", side_effect=fake_get):
item = session._prepare_load_skill("call-1", {"action": "load", "name": "code-review"})
call_id, result = session._exec_load_skill(item)
item = session._prepare_skill("call-1", {"action": "load", "name": "code-review"})
call_id, result = session._exec_skill(item)
assert call_id == "call-1"
assert "code-review" in result
@@ -177,8 +178,8 @@ class TestExecLoadSkill:
session, _, fake_get = _make_session([])
with patch("turnstone.core.session.get_skill_by_name", side_effect=fake_get):
item = session._prepare_load_skill("call-1", {"action": "load", "name": "nope"})
call_id, result = session._exec_load_skill(item)
item = session._prepare_skill("call-1", {"action": "load", "name": "nope"})
call_id, result = session._exec_skill(item)
assert "not found" in result.lower()
assert session._set_skill_called == []
@@ -188,8 +189,8 @@ class TestExecLoadSkill:
session, _, fake_get = _make_session(skills)
with patch("turnstone.core.session.get_skill_by_name", side_effect=fake_get):
item = session._prepare_load_skill("call-1", {"action": "load", "name": "test"})
session._exec_load_skill(item)
item = session._prepare_skill("call-1", {"action": "load", "name": "test"})
session._exec_skill(item)
session.ui.on_tool_result.assert_called_once()
@@ -216,10 +217,10 @@ class TestExecLoadSkill:
mock_storage.list_prompt_templates.return_value = skills
session, _, _ = _make_session()
item = session._prepare_load_skill("call-1", {"action": "search", "query": "code"})
item = session._prepare_skill("call-1", {"action": "search", "query": "code"})
with patch("turnstone.core.storage._registry.get_storage", return_value=mock_storage):
call_id, result = session._exec_load_skill(item)
call_id, result = session._exec_skill(item)
assert "code-review" in result
# docs-writer shouldn't match "code" query
@@ -241,10 +242,10 @@ class TestExecLoadSkill:
mock_storage.list_prompt_templates.return_value = skills
session, _, _ = _make_session()
item = session._prepare_load_skill("call-1", {"action": "search"})
item = session._prepare_skill("call-1", {"action": "search"})
with patch("turnstone.core.storage._registry.get_storage", return_value=mock_storage):
call_id, result = session._exec_load_skill(item)
call_id, result = session._exec_skill(item)
# Should be limited to 10
assert result.count("skill-") == 10
@@ -254,10 +255,10 @@ class TestExecLoadSkill:
mock_storage.list_prompt_templates.return_value = []
session, _, _ = _make_session()
item = session._prepare_load_skill("call-1", {"action": "search", "query": "nonexistent"})
item = session._prepare_skill("call-1", {"action": "search", "query": "nonexistent"})
with patch("turnstone.core.storage._registry.get_storage", return_value=mock_storage):
call_id, result = session._exec_load_skill(item)
call_id, result = session._exec_skill(item)
assert "no skills found" in result.lower()
@@ -276,21 +277,21 @@ class TestExecLoadSkill:
mock_storage.list_prompt_templates.return_value = skills
session, _, _ = _make_session()
item = session._prepare_load_skill("call-1", {"action": "search", "query": "risky"})
item = session._prepare_skill("call-1", {"action": "search", "query": "risky"})
with patch("turnstone.core.storage._registry.get_storage", return_value=mock_storage):
call_id, result = session._exec_load_skill(item)
call_id, result = session._exec_skill(item)
assert "high" in result
def test_search_storage_failure_returns_empty(self) -> None:
session, _, _ = _make_session()
item = session._prepare_load_skill("call-1", {"action": "search", "query": "test"})
item = session._prepare_skill("call-1", {"action": "search", "query": "test"})
with patch(
"turnstone.core.storage._registry.get_storage", side_effect=RuntimeError("no storage")
):
call_id, result = session._exec_load_skill(item)
call_id, result = session._exec_skill(item)
assert "no skills found" in result.lower()
@@ -307,10 +308,8 @@ class TestExecLoadSkill:
session, _, fake_get = _make_session(skills)
with patch("turnstone.core.session.get_skill_by_name", side_effect=fake_get):
item = session._prepare_load_skill(
"call-1", {"action": "load", "name": "disabled-skill"}
)
call_id, result = session._exec_load_skill(item)
item = session._prepare_skill("call-1", {"action": "load", "name": "disabled-skill"})
call_id, result = session._exec_skill(item)
assert "not found" in result.lower()
assert session._set_skill_called == []
@@ -321,8 +320,8 @@ class TestExecLoadSkill:
session._skill_name = "active"
with patch("turnstone.core.session.get_skill_by_name", side_effect=fake_get):
item = session._prepare_load_skill("call-1", {"action": "load", "name": "active"})
call_id, result = session._exec_load_skill(item)
item = session._prepare_skill("call-1", {"action": "load", "name": "active"})
call_id, result = session._exec_skill(item)
assert "already active" in result.lower()
assert session._set_skill_called == []
@@ -352,10 +351,10 @@ class TestExecLoadSkill:
mock_storage.list_prompt_templates.return_value = skills
session, _, _ = _make_session()
item = session._prepare_load_skill("call-1", {"action": "search"})
item = session._prepare_skill("call-1", {"action": "search"})
with patch("turnstone.core.storage._registry.get_storage", return_value=mock_storage):
call_id, result = session._exec_load_skill(item)
call_id, result = session._exec_skill(item)
assert "enabled-skill" in result
assert "disabled-skill" not in result
@@ -375,14 +374,114 @@ class TestExecLoadSkill:
mock_storage.list_prompt_templates.return_value = skills
session, _, _ = _make_session()
item = session._prepare_load_skill("call-1", {"action": "search", "query": "code review"})
item = session._prepare_skill("call-1", {"action": "search", "query": "code review"})
with patch("turnstone.core.storage._registry.get_storage", return_value=mock_storage):
call_id, result = session._exec_load_skill(item)
call_id, result = session._exec_skill(item)
assert "code-review" in result
def test_preparer_load_has_approval_label(self) -> None:
session, _, _ = _make_session()
item = session._prepare_load_skill("call-1", {"action": "load", "name": "my-skill"})
assert item["approval_label"] == "load_skill__my-skill"
item = session._prepare_skill("call-1", {"action": "load", "name": "my-skill"})
assert item["approval_label"] == "skill__my-skill"
# ---------------------------------------------------------------------------
# Tests: Skill Catalog Disclosure (Agent Skills standard compliance)
# ---------------------------------------------------------------------------
class TestSkillCatalogDisclosure:
"""Verify <available-skills> catalog appears in system messages."""
def _build_session_with_system_messages(
self,
search_skills: list[dict[str, Any]] | None = None,
) -> Any:
"""Build a session and call _init_system_messages to get dev_parts."""
from turnstone.core.session import ChatSession
session = ChatSession.__new__(ChatSession)
ui = MagicMock()
session.ui = ui
session.model = "test-model"
session._ws_id = "ws-test"
session._node_id = "node-1"
session._skill_name = None
session._skill_content = None
session._skill_resources = {}
session._applied_skill_content = None
session.context_window = 128000
session.messages = []
session._config = {}
session.creative_mode = False
session.instructions = ""
session.system_messages = []
session._agent_system_messages = []
session.reasoning_effort = "medium"
session._pending_nudge = []
session._tool_search = None
session._mcp_client = None
session._notify_on_complete = "{}"
session._tool_error_flags = {}
# Memory stubs
session._memory_config = MagicMock()
session._memory_config.fetch_limit = 0
session._user_id = ""
with (
patch(
"turnstone.core.session.list_skills_by_activation",
return_value=search_skills or [],
),
patch.object(session, "_get_visible_memories", return_value=[]),
):
session._init_system_messages()
return session
def test_catalog_present_with_search_skills(self) -> None:
skills = [
{"name": "pdf-processing", "description": "Extract PDF text and forms."},
{"name": "data-analysis", "description": "Analyze datasets."},
]
session = self._build_session_with_system_messages(search_skills=skills)
content = session.system_messages[0]["content"]
assert "<available-skills>" in content
assert "pdf-processing" in content
assert "data-analysis" in content
assert "</available-skills>" in content
def test_catalog_omitted_when_no_search_skills(self) -> None:
session = self._build_session_with_system_messages(search_skills=[])
content = session.system_messages[0]["content"]
assert "<available-skills>" not in content
def test_catalog_capped_at_30(self) -> None:
skills = [{"name": f"skill-{i:03d}", "description": f"Desc {i}"} for i in range(50)]
session = self._build_session_with_system_messages(search_skills=skills)
content = session.system_messages[0]["content"]
# Should include first 30, not all 50
assert "skill-029" in content
assert "skill-030" not in content
def test_catalog_escapes_html(self) -> None:
skills = [
{"name": "xss-test", "description": "Handle <script> & 'quotes'."},
]
session = self._build_session_with_system_messages(search_skills=skills)
content = session.system_messages[0]["content"]
assert "&lt;script&gt;" in content
assert "<script>" not in content.replace("<available-skills>", "").replace(
"</available-skills>", ""
).replace("<skill>", "").replace("</skill>", "").replace("<name>", "").replace(
"</name>", ""
).replace("<description>", "").replace("</description>", "")
def test_catalog_includes_hint(self) -> None:
skills = [{"name": "test", "description": "Test skill."}]
session = self._build_session_with_system_messages(search_skills=skills)
content = session.system_messages[0]["content"]
assert "/skill" in content
+196
View File
@@ -26,6 +26,7 @@ from turnstone.console.server import (
admin_get_mcp_server,
admin_import_mcp_config,
admin_list_mcp_servers,
admin_mcp_reload,
admin_update_mcp_server,
)
from turnstone.core.auth import AuthResult
@@ -96,6 +97,11 @@ _ROUTES = [
admin_import_mcp_config,
methods=["POST"],
),
Route(
"/api/admin/mcp-servers/reload",
admin_mcp_reload,
methods=["POST"],
),
Route(
"/api/admin/mcp-servers/{server_id}",
admin_get_mcp_server,
@@ -115,6 +121,21 @@ _ROUTES = [
]
def _routes_with_internal() -> list[Mount]:
"""Routes including the node-side internal endpoint (lazy-imported)."""
from turnstone.server import internal_mcp_reload
return [
Mount(
"/v1",
routes=[
*_ROUTES[0].routes, # type: ignore[union-attr]
Route("/api/_internal/mcp-reload", internal_mcp_reload, methods=["POST"]),
],
),
]
@pytest.fixture
def storage(tmp_path):
return SQLiteBackend(str(tmp_path / "test.db"))
@@ -550,8 +571,11 @@ def _fake_request(*nodes: dict[str, Any], proxy_client: Any = None) -> MagicMock
"""Build a minimal mock request with collector and proxy_client."""
collector = MagicMock()
collector.get_nodes.return_value = (list(nodes), len(nodes))
collector.get_all_nodes.side_effect = lambda: collector.get_nodes.return_value[0]
req = MagicMock()
req.state.auth_result = None
req.app.state.collector = collector
req.app.state.jwt_secret = ""
req.app.state.proxy_client = proxy_client or AsyncMock()
req.app.state.proxy_token_mgr = None
req.app.state.proxy_auth_token = "tok"
@@ -693,3 +717,175 @@ class TestNotifyNodesMcpReload:
result = await _notify_nodes_mcp_reload(req)
assert result["n1"] == {"reloaded": 2}
assert "error" in result["n2"]
# ---------------------------------------------------------------------------
# Console reload endpoint: POST /v1/api/admin/mcp-servers/reload
# ---------------------------------------------------------------------------
class TestAdminMcpReloadEndpoint:
"""HTTP-level tests for the console reload endpoint."""
def test_reload_success(self, client: TestClient) -> None:
"""Reload endpoint returns status ok and fan-out results."""
with patch(
"turnstone.console.server._notify_nodes_mcp_reload",
new_callable=AsyncMock,
return_value={"n1": {"reloaded": 3}},
):
r = client.post("/v1/api/admin/mcp-servers/reload")
assert r.status_code == 200
data = r.json()
assert data["status"] == "ok"
assert data["results"] == {"n1": {"reloaded": 3}}
def test_reload_empty_cluster(self, client: TestClient) -> None:
"""Reload with no nodes returns empty results."""
with patch(
"turnstone.console.server._notify_nodes_mcp_reload",
new_callable=AsyncMock,
return_value={},
):
r = client.post("/v1/api/admin/mcp-servers/reload")
assert r.status_code == 200
data = r.json()
assert data["status"] == "ok"
assert data["results"] == {}
def test_reload_permission_denied(self, client_no_perm: TestClient) -> None:
"""Reload without admin.mcp permission is rejected."""
r = client_no_perm.post("/v1/api/admin/mcp-servers/reload")
assert r.status_code == 403
assert "admin.mcp" in r.json()["error"]
def test_reload_no_storage(self) -> None:
"""Reload returns 503 when auth_storage is not available."""
app = Starlette(
routes=_ROUTES,
middleware=[Middleware(_InjectAuthMiddleware)],
)
# Deliberately omit app.state.auth_storage
no_storage_client = TestClient(app, raise_server_exceptions=False)
r = no_storage_client.post("/v1/api/admin/mcp-servers/reload")
assert r.status_code == 503
def test_reload_mixed_node_results(self, client: TestClient) -> None:
"""Reload propagates per-node errors in results."""
with patch(
"turnstone.console.server._notify_nodes_mcp_reload",
new_callable=AsyncMock,
return_value={
"n1": {"reloaded": 2},
"n2": {"error": "Connection refused"},
},
):
r = client.post("/v1/api/admin/mcp-servers/reload")
assert r.status_code == 200
data = r.json()
assert data["results"]["n1"] == {"reloaded": 2}
assert "error" in data["results"]["n2"]
# ---------------------------------------------------------------------------
# Node reload endpoint: POST /v1/api/_internal/mcp-reload
# ---------------------------------------------------------------------------
class TestInternalMcpReloadEndpoint:
"""HTTP-level tests for the node-side MCP reload endpoint."""
@pytest.fixture()
def node_client(self, storage: SQLiteBackend) -> TestClient:
"""TestClient with an MCP client manager on app.state."""
app = Starlette(
routes=_routes_with_internal(),
middleware=[Middleware(_InjectAuthMiddleware)],
)
app.state.auth_storage = storage
mgr = MagicMock()
mgr.reconcile_sync.return_value = {
"added": ["new-srv"],
"removed": [],
"updated": [],
}
app.state.mcp_client = mgr
return TestClient(app, raise_server_exceptions=False)
def test_reload_calls_reconcile(self, node_client: TestClient, storage: SQLiteBackend) -> None:
"""Reload endpoint calls reconcile_sync and returns its result."""
with patch("turnstone.core.storage._registry.get_storage", return_value=storage):
r = node_client.post("/v1/api/_internal/mcp-reload")
assert r.status_code == 200
data = r.json()
assert data["status"] == "ok"
assert data["added"] == ["new-srv"]
assert data["removed"] == []
assert data["updated"] == []
def test_reload_passes_storage_to_reconcile(
self,
storage: SQLiteBackend,
) -> None:
"""Verify reconcile_sync receives the storage backend."""
app = Starlette(
routes=_routes_with_internal(),
middleware=[Middleware(_InjectAuthMiddleware)],
)
app.state.auth_storage = storage
mgr = MagicMock()
mgr.reconcile_sync.return_value = {"added": [], "removed": [], "updated": []}
app.state.mcp_client = mgr
c = TestClient(app, raise_server_exceptions=False)
with patch("turnstone.core.storage._registry.get_storage", return_value=storage):
r = c.post("/v1/api/_internal/mcp-reload")
assert r.status_code == 200
mgr.reconcile_sync.assert_called_once_with(storage)
def test_reload_creates_manager_when_missing(self, storage: SQLiteBackend) -> None:
"""When mcp_client is absent, a new MCPClientManager is created."""
app = Starlette(
routes=_routes_with_internal(),
middleware=[Middleware(_InjectAuthMiddleware)],
)
app.state.auth_storage = storage
# No mcp_client on app.state
c = TestClient(app, raise_server_exceptions=False)
with (
patch("turnstone.core.storage._registry.get_storage", return_value=storage),
patch("turnstone.core.mcp_client.MCPClientManager") as mock_cls,
):
mock_mgr = MagicMock()
mock_mgr.reconcile_sync.return_value = {
"added": [],
"removed": [],
"updated": [],
}
mock_cls.return_value = mock_mgr
r = c.post("/v1/api/_internal/mcp-reload")
assert r.status_code == 200
mock_cls.assert_called_once_with({})
mock_mgr.start.assert_called_once()
mock_mgr.reconcile_sync.assert_called_once_with(storage)
def test_reload_reconcile_result_in_response(self, storage: SQLiteBackend) -> None:
"""Full reconcile result fields (added/removed/updated) appear in JSON."""
app = Starlette(
routes=_routes_with_internal(),
middleware=[Middleware(_InjectAuthMiddleware)],
)
app.state.auth_storage = storage
mgr = MagicMock()
mgr.reconcile_sync.return_value = {
"added": ["a"],
"removed": ["b"],
"updated": ["c"],
}
app.state.mcp_client = mgr
c = TestClient(app, raise_server_exceptions=False)
with patch("turnstone.core.storage._registry.get_storage", return_value=storage):
r = c.post("/v1/api/_internal/mcp-reload")
data = r.json()
assert data["added"] == ["a"]
assert data["removed"] == ["b"]
assert data["updated"] == ["c"]
+185
View File
@@ -401,6 +401,64 @@ class TestSessionIntegration:
prepared = session._prepare_tool(tc)
assert "error" in prepared
assert "Unknown tool" in prepared["error"]
# Error lists available tools so the model can self-correct
assert "bash" in prepared["error"]
# Surfaces warning to user
session.ui.on_error.assert_called_once()
assert "nonexistent" in session.ui.on_error.call_args[0][0]
def test_prepare_tool_strips_whitespace_from_name(self, tmp_db):
"""Local models may produce tool names with leading/trailing whitespace."""
session = self._make_session(mcp_client=None)
tc = {
"id": "call_strip",
"function": {"name": " bash\n", "arguments": '{"command": "echo hi"}'},
}
prepared = session._prepare_tool(tc)
assert prepared["func_name"] == "bash"
assert "error" not in prepared
def test_prepare_tool_malformed_json_surfaces_error(self, tmp_db):
"""Malformed JSON args should surface a warning to the user and
give the model a hint about expected format."""
session = self._make_session(mcp_client=None)
tc = {
"id": "call_bad",
"function": {"name": "bash", "arguments": "{command: echo hi}"},
}
prepared = session._prepare_tool(tc)
assert "error" in prepared
assert "JSON parse error" in prepared["error"]
assert "command" in prepared["error"] # hint about expected key
assert "Please retry" in prepared["error"]
# User-facing warning
session.ui.on_error.assert_called_once()
assert "Malformed tool call" in session.ui.on_error.call_args[0][0]
def test_ensure_tool_call_ids_dict(self, tmp_db):
"""_ensure_tool_call_ids fills empty IDs on streaming-style dict."""
from turnstone.core.session import ChatSession
tool_calls_acc = {
0: {"id": "", "function": {"name": "bash", "arguments": "{}"}},
1: {"id": "", "function": {"name": "read_file", "arguments": "{}"}},
}
ChatSession._ensure_tool_call_ids(tool_calls_acc)
ids = [tc["id"] for tc in tool_calls_acc.values()]
assert all(id_.startswith("call_") for id_ in ids)
assert len(set(ids)) == 2 # unique
def test_ensure_tool_call_ids_list(self, tmp_db):
"""_ensure_tool_call_ids fills empty IDs on list (agent path)."""
from turnstone.core.session import ChatSession
tool_calls = [
{"id": None, "function": {"name": "bash", "arguments": "{}"}},
{"id": "call_existing", "function": {"name": "bash", "arguments": "{}"}},
]
ChatSession._ensure_tool_call_ids(tool_calls)
assert tool_calls[0]["id"].startswith("call_")
assert tool_calls[1]["id"] == "call_existing" # preserved
def test_mcp_command_no_client(self, tmp_db):
session = self._make_session(mcp_client=None)
@@ -1368,3 +1426,130 @@ class TestShutdownCleanup:
assert mgr.get_prompts() == []
assert mgr._resource_map == {}
assert mgr._prompt_map == {}
# ---------------------------------------------------------------------------
# TCP probe and unreachable server handling
# ---------------------------------------------------------------------------
class TestTCPProbe:
"""MCPClientManager._tcp_probe should fail fast on unreachable servers."""
def test_tcp_probe_unreachable_raises_connection_error(self):
"""Unreachable host raises ConnectionError, not TimeoutError."""
mgr = MCPClientManager({})
async def _run():
with pytest.raises(ConnectionError, match="unreachable"):
await mgr._tcp_probe("test-server", "http://127.0.0.1:1")
asyncio.run(_run())
def test_tcp_probe_parses_url_correctly(self):
"""Port and host are extracted from the URL."""
mgr = MCPClientManager({})
async def _run():
# Non-routable port — should fail with ConnectionError
with pytest.raises(ConnectionError):
await mgr._tcp_probe("srv", "https://127.0.0.1:1/mcp")
asyncio.run(_run())
def test_tcp_probe_default_port_http(self):
"""Default port 80 used for http:// URLs without explicit port."""
mgr = MCPClientManager({})
async def _run():
# Will fail (nothing on port 80), but should not crash on parsing
with pytest.raises(ConnectionError):
await mgr._tcp_probe("srv", "http://127.0.0.1")
asyncio.run(_run())
def test_tcp_probe_dns_failure(self):
"""Unresolvable hostname raises ConnectionError."""
mgr = MCPClientManager({})
async def _run():
with pytest.raises(ConnectionError):
await mgr._tcp_probe("srv", "http://this.host.does.not.exist.invalid:8080/mcp")
asyncio.run(_run())
class TestConnectOneUnreachable:
"""_connect_one should handle unreachable HTTP servers gracefully."""
def test_unreachable_http_server_raises_connection_error(self):
"""Unreachable HTTP MCP server raises ConnectionError without spinning."""
mgr = MCPClientManager({})
mgr._loop = asyncio.new_event_loop()
async def _run():
with pytest.raises(ConnectionError, match="unreachable"):
await mgr._connect_one(
"bad-server",
{
"type": "http",
"url": "http://127.0.0.1:1/mcp",
},
)
mgr._loop.run_until_complete(_run())
mgr._loop.close()
# Server should NOT be in sessions (connection failed)
assert "bad-server" not in mgr._sessions
def test_connect_all_continues_after_unreachable_server(self):
"""_connect_all logs error and continues to next server."""
mgr = MCPClientManager(
{
"bad": {"type": "http", "url": "http://127.0.0.1:1/mcp"},
}
)
loop = asyncio.new_event_loop()
loop.run_until_complete(mgr._connect_all())
loop.close()
assert "bad" not in mgr._sessions
assert "bad" in mgr._last_error
class TestSafeCloseStack:
"""_safe_close_stack should suppress errors from broken anyio scopes."""
def test_suppresses_runtime_error(self):
"""RuntimeError from broken cancel scope is suppressed."""
async def _run():
stack = AsyncExitStack()
await stack.__aenter__()
# Simulate a broken close that raises RuntimeError
async def _broken_close():
raise RuntimeError("Attempted to exit cancel scope in a different task")
stack.aclose = _broken_close
# Should not raise
await MCPClientManager._safe_close_stack(stack)
asyncio.run(_run())
def test_suppresses_cancelled_error(self):
"""CancelledError during close is suppressed."""
async def _run():
stack = AsyncExitStack()
await stack.__aenter__()
async def _cancel_close():
raise asyncio.CancelledError()
stack.aclose = _cancel_close
await MCPClientManager._safe_close_stack(stack)
asyncio.run(_run())
+66
View File
@@ -398,6 +398,72 @@ class TestResolveInstallConfig:
config = resolve_install_config(server, "remote", 0)
assert config["url"] == "https://us-east.example.com/mcp"
def test_remote_variable_substitution_invalid_scheme(self) -> None:
server = RegistryServer(
name="io.example/test",
version="1.0.0",
remotes=[
RegistryRemote(
type="streamable-http",
url="{scheme}://evil.example.com/mcp",
variables={
"scheme": RegistryRemoteVariable(is_required=True),
},
)
],
)
with pytest.raises(MCPRegistryError, match="Invalid URL scheme"):
resolve_install_config(server, "remote", 0, variables={"scheme": "file"})
def test_remote_variable_substitution_preserves_valid_scheme(self) -> None:
server = RegistryServer(
name="io.example/test",
version="1.0.0",
remotes=[
RegistryRemote(
type="streamable-http",
url="https://{host}.example.com/mcp",
variables={
"host": RegistryRemoteVariable(is_required=True),
},
)
],
)
config = resolve_install_config(server, "remote", 0, variables={"host": "api"})
assert config["url"] == "https://api.example.com/mcp"
def test_remote_variable_substitution_missing_hostname(self) -> None:
"""URL like https:///mcp has valid scheme but no hostname."""
server = RegistryServer(
name="io.example/test",
version="1.0.0",
remotes=[
RegistryRemote(
type="streamable-http",
url="https:///mcp",
)
],
)
with pytest.raises(MCPRegistryError, match="hostname is missing"):
resolve_install_config(server, "remote", 0)
def test_remote_variable_substitution_embedded_credentials(self) -> None:
server = RegistryServer(
name="io.example/test",
version="1.0.0",
remotes=[
RegistryRemote(
type="streamable-http",
url="https://{creds}@example.com/mcp",
variables={
"creds": RegistryRemoteVariable(is_required=True),
},
)
],
)
with pytest.raises(MCPRegistryError, match="embedded credentials"):
resolve_install_config(server, "remote", 0, variables={"creds": "user:pass"})
def test_remote_no_remotes(self) -> None:
server = RegistryServer(name="io.example/test", version="1.0.0")
with pytest.raises(MCPRegistryError, match="no remote"):
+74 -2
View File
@@ -4,7 +4,7 @@ from __future__ import annotations
import json
from typing import TYPE_CHECKING, Any
from unittest.mock import AsyncMock, patch
from unittest.mock import AsyncMock, MagicMock, patch
import pytest
from starlette.applications import Starlette
@@ -18,11 +18,13 @@ if TYPE_CHECKING:
from starlette.responses import Response
from turnstone.console.server import (
_get_registry_url,
admin_registry_install,
admin_registry_search,
)
from turnstone.core.auth import AuthResult
from turnstone.core.mcp_registry import (
DEFAULT_REGISTRY_URL,
MCPRegistryError,
RegistryPackage,
RegistryRemote,
@@ -362,7 +364,7 @@ class TestRegistryInstall:
def test_install_max_servers(self, client: TestClient, storage: SQLiteBackend) -> None:
import uuid
for i in range(50):
for i in range(200):
storage.create_mcp_server(
server_id=uuid.uuid4().hex,
name=f"server-{i}",
@@ -549,3 +551,73 @@ class TestRegistryInstall:
assert resp.status_code == 409
assert "custom 'name'" in resp.json()["error"]
# ---------------------------------------------------------------------------
# _get_registry_url fallback chain tests
# ---------------------------------------------------------------------------
def _mock_request(storage: Any = None, config_store: Any = None) -> MagicMock:
"""Build a mock Request with app.state.auth_storage and app.state.config_store."""
request = MagicMock()
request.app.state.auth_storage = storage
request.app.state.config_store = config_store
return request
class TestGetRegistryUrl:
"""Verify three-tier URL resolution: DB setting -> config.toml -> default."""
def test_returns_db_setting_when_available(self) -> None:
config_store = MagicMock()
config_store.get.return_value = "https://custom.registry.example.com"
request = _mock_request(config_store=config_store)
with patch("turnstone.core.config.load_config", return_value={}):
url = _get_registry_url(request)
assert url == "https://custom.registry.example.com"
config_store.get.assert_called_once_with("mcp.registry_url")
def test_falls_back_to_config_when_config_store_returns_empty(self) -> None:
config_store = MagicMock()
config_store.get.return_value = ""
request = _mock_request(config_store=config_store)
with patch(
"turnstone.core.config.load_config",
return_value={"registry_url": "https://config.registry.example.com"},
):
url = _get_registry_url(request)
assert url == "https://config.registry.example.com"
def test_falls_back_to_config_when_no_config_store(self) -> None:
request = _mock_request()
with patch(
"turnstone.core.config.load_config",
return_value={"registry_url": "https://config.registry.example.com"},
):
url = _get_registry_url(request)
assert url == "https://config.registry.example.com"
def test_falls_back_to_default_when_both_unavailable(self) -> None:
config_store = MagicMock()
config_store.get.return_value = ""
request = _mock_request(config_store=config_store)
with patch("turnstone.core.config.load_config", return_value={}):
url = _get_registry_url(request)
assert url == DEFAULT_REGISTRY_URL
def test_falls_back_to_default_when_no_config_store_or_config(self) -> None:
request = _mock_request()
with patch("turnstone.core.config.load_config", return_value={}):
url = _get_registry_url(request)
assert url == DEFAULT_REGISTRY_URL
+3 -9
View File
@@ -3,16 +3,10 @@
from __future__ import annotations
import uuid
from typing import TYPE_CHECKING
import pytest
from turnstone.core.storage._sqlite import SQLiteBackend
@pytest.fixture
def db(tmp_path):
"""Fresh SQLite backend for each test."""
return SQLiteBackend(str(tmp_path / "test.db"))
if TYPE_CHECKING:
from turnstone.core.storage._sqlite import SQLiteBackend
def _make_id() -> str:
+20
View File
@@ -4,6 +4,7 @@ from turnstone.core.metacognition import (
NUDGE_COMPLETION,
NUDGE_CORRECTION,
NUDGE_DENIAL,
NUDGE_REPEAT,
NUDGE_RESUME,
NUDGE_START,
NUDGE_TOOL_ERROR,
@@ -288,3 +289,22 @@ class TestToolErrorNudge:
def test_not_with_zero_memories(self):
state: dict[str, float] = {}
assert should_nudge("tool_error", state, message_count=5, memory_count=0) is False
class TestRepeatNudge:
def test_format(self):
assert format_nudge("repeat") == NUDGE_REPEAT
def test_fires(self):
state: dict[str, float] = {}
assert should_nudge("repeat", state, message_count=5) is True
def test_cooldown(self):
state: dict[str, float] = {}
assert should_nudge("repeat", state, message_count=5) is True
assert should_nudge("repeat", state, message_count=6) is False
def test_no_memory_requirement(self):
"""Repeat nudge should fire even with zero memories."""
state: dict[str, float] = {}
assert should_nudge("repeat", state, message_count=5, memory_count=0) is True
+163
View File
@@ -0,0 +1,163 @@
"""Tests for model definition storage CRUD operations."""
from __future__ import annotations
import uuid
from typing import TYPE_CHECKING
if TYPE_CHECKING:
from turnstone.core.storage._sqlite import SQLiteBackend
def _make_id() -> str:
return uuid.uuid4().hex
class TestModelDefinitionStorage:
def test_create_and_get(self, db: SQLiteBackend) -> None:
did = _make_id()
db.create_model_definition(
definition_id=did,
alias="test-model",
model="gpt-5",
provider="openai",
base_url="https://api.openai.com/v1",
api_key="sk-test",
context_window=128000,
)
m = db.get_model_definition(did)
assert m is not None
assert m["alias"] == "test-model"
assert m["model"] == "gpt-5"
assert m["provider"] == "openai"
assert m["base_url"] == "https://api.openai.com/v1"
assert m["api_key"] == "sk-test"
assert m["context_window"] == 128000
assert m["capabilities"] == "{}"
assert m["enabled"] is True
def test_get_by_alias(self, db: SQLiteBackend) -> None:
did = _make_id()
db.create_model_definition(definition_id=did, alias="by-alias", model="gpt-5")
m = db.get_model_definition_by_alias("by-alias")
assert m is not None
assert m["definition_id"] == did
def test_get_by_alias_not_found(self, db: SQLiteBackend) -> None:
assert db.get_model_definition_by_alias("nope") is None
def test_get_not_found(self, db: SQLiteBackend) -> None:
assert db.get_model_definition("nonexistent") is None
def test_list_empty(self, db: SQLiteBackend) -> None:
assert db.list_model_definitions() == []
def test_list_all(self, db: SQLiteBackend) -> None:
db.create_model_definition(definition_id=_make_id(), alias="alpha", model="gpt-5")
db.create_model_definition(
definition_id=_make_id(), alias="beta", model="claude-opus-4-6", provider="anthropic"
)
models = db.list_model_definitions()
assert len(models) == 2
assert models[0]["alias"] == "alpha" # ordered by alias
assert models[1]["alias"] == "beta"
def test_list_enabled_only(self, db: SQLiteBackend) -> None:
db.create_model_definition(
definition_id=_make_id(), alias="enabled-model", model="gpt-5", enabled=True
)
db.create_model_definition(
definition_id=_make_id(), alias="disabled-model", model="gpt-5", enabled=False
)
enabled = db.list_model_definitions(enabled_only=True)
assert len(enabled) == 1
assert enabled[0]["alias"] == "enabled-model"
def test_update_basic_fields(self, db: SQLiteBackend) -> None:
did = _make_id()
db.create_model_definition(
definition_id=did, alias="orig", model="gpt-5", base_url="http://old"
)
ok = db.update_model_definition(did, alias="renamed", base_url="http://new")
assert ok is True
m = db.get_model_definition(did)
assert m is not None
assert m["alias"] == "renamed"
assert m["base_url"] == "http://new"
def test_update_boolean_conversion(self, db: SQLiteBackend) -> None:
did = _make_id()
db.create_model_definition(definition_id=did, alias="booltest", model="gpt-5")
db.update_model_definition(did, enabled=False)
m = db.get_model_definition(did)
assert m is not None
assert m["enabled"] is False
def test_update_not_found(self, db: SQLiteBackend) -> None:
ok = db.update_model_definition("nonexistent", alias="x")
assert ok is False
def test_update_ignores_disallowed_fields(self, db: SQLiteBackend) -> None:
did = _make_id()
db.create_model_definition(
definition_id=did, alias="guard", model="gpt-5", created_by="admin"
)
original = db.get_model_definition(did)
assert original is not None
original_created = original["created"]
# created_by and created are not in the mutable allowlist
db.update_model_definition(did, created_by="evil", created="2000-01-01T00:00:00")
m = db.get_model_definition(did)
assert m is not None
assert m["created_by"] == "admin" # unchanged
assert m["created"] == original_created # unchanged
def test_delete(self, db: SQLiteBackend) -> None:
did = _make_id()
db.create_model_definition(definition_id=did, alias="delme", model="gpt-5")
ok = db.delete_model_definition(did)
assert ok is True
assert db.get_model_definition(did) is None
def test_delete_not_found(self, db: SQLiteBackend) -> None:
ok = db.delete_model_definition("nonexistent")
assert ok is False
def test_create_duplicate_alias(self, db: SQLiteBackend) -> None:
db.create_model_definition(definition_id=_make_id(), alias="unique", model="gpt-5")
# Second create with same alias but different ID should be no-op (OR IGNORE)
did2 = _make_id()
db.create_model_definition(definition_id=did2, alias="unique", model="gpt-5")
assert db.get_model_definition(did2) is None
def test_create_idempotent_same_id(self, db: SQLiteBackend) -> None:
did = _make_id()
db.create_model_definition(definition_id=did, alias="idem", model="gpt-5")
db.create_model_definition(definition_id=did, alias="idem", model="gpt-5-mini")
m = db.get_model_definition(did)
assert m is not None
assert m["model"] == "gpt-5" # original preserved
def test_capabilities_json(self, db: SQLiteBackend) -> None:
did = _make_id()
caps = '{"supports_vision": true, "supports_web_search": false}'
db.create_model_definition(
definition_id=did, alias="caps-test", model="gpt-5", capabilities=caps
)
m = db.get_model_definition(did)
assert m is not None
assert m["capabilities"] == caps
def test_defaults(self, db: SQLiteBackend) -> None:
"""Verify default values for optional fields."""
did = _make_id()
db.create_model_definition(definition_id=did, alias="defaults", model="gpt-5")
m = db.get_model_definition(did)
assert m is not None
assert m["provider"] == "openai"
assert m["base_url"] == ""
assert m["api_key"] == ""
assert m["context_window"] == 32768
assert m["capabilities"] == "{}"
assert m["enabled"] is True
assert m["created_by"] == ""
+235
View File
@@ -0,0 +1,235 @@
"""Tests for probe_model_endpoint() and lookup_model_capabilities()."""
from __future__ import annotations
from typing import Any
from unittest.mock import MagicMock, patch
import pytest
from turnstone.core.model_registry import probe_model_endpoint
from turnstone.core.providers import list_known_models, lookup_model_capabilities
# ---------------------------------------------------------------------------
# Helpers
# ---------------------------------------------------------------------------
def _mock_model(
model_id: str,
*,
owned_by: str = "test",
meta: dict[str, Any] | None = None,
) -> MagicMock:
m = MagicMock()
m.id = model_id
dumped: dict[str, Any] = {"owned_by": owned_by}
if meta is not None:
dumped["meta"] = meta
m.model_dump.return_value = dumped
return m
def _mock_client(*models: MagicMock) -> MagicMock:
fast = MagicMock()
fast.models.list.return_value = MagicMock(data=list(models))
client = MagicMock()
client.with_options.return_value = fast
return client
# ---------------------------------------------------------------------------
# probe_model_endpoint
# ---------------------------------------------------------------------------
class TestProbeModelEndpoint:
@patch("turnstone.core.providers.create_client")
def test_probe_success(self, mock_cc: MagicMock) -> None:
m1 = _mock_model("model-a")
m2 = _mock_model("model-b")
mock_cc.return_value = _mock_client(m1, m2)
result = probe_model_endpoint("openai", "http://localhost:8000/v1", "key")
assert result["reachable"] is True
assert result["available_models"] == ["model-a", "model-b"]
assert result["error"] is None
@patch("turnstone.core.providers.create_client")
def test_target_found(self, mock_cc: MagicMock) -> None:
m1 = _mock_model("gpt-5")
mock_cc.return_value = _mock_client(m1)
result = probe_model_endpoint(
"openai", "http://localhost:8000/v1", "key", target_model="gpt-5"
)
assert result["model_found"] is True
@patch("turnstone.core.providers.create_client")
def test_target_not_found(self, mock_cc: MagicMock) -> None:
m1 = _mock_model("model-a")
mock_cc.return_value = _mock_client(m1)
result = probe_model_endpoint(
"openai", "http://localhost:8000/v1", "key", target_model="gpt-5"
)
assert result["model_found"] is False
assert result["available_models"] == ["model-a"]
@patch("turnstone.core.providers.create_client")
def test_no_target_model_found_is_none(self, mock_cc: MagicMock) -> None:
m1 = _mock_model("model-a")
mock_cc.return_value = _mock_client(m1)
result = probe_model_endpoint("openai", "http://localhost:8000/v1", "key")
assert result["model_found"] is None
@patch("turnstone.core.providers.create_client")
def test_context_window_llama_cpp(self, mock_cc: MagicMock) -> None:
m = _mock_model("qwen-32b", meta={"n_ctx_train": 131072})
mock_cc.return_value = _mock_client(m)
result = probe_model_endpoint("openai", "http://localhost:8000/v1", "key")
assert result["context_window"] == 131072
assert result["server_type"] == "llama.cpp"
@patch("turnstone.core.providers.create_client")
def test_server_type_openai(self, mock_cc: MagicMock) -> None:
m = _mock_model("gpt-5")
mock_cc.return_value = _mock_client(m)
result = probe_model_endpoint("openai", "https://api.openai.com/v1", "sk-test")
assert result["server_type"] == "openai"
@patch("turnstone.core.providers.create_client")
def test_server_type_sglang(self, mock_cc: MagicMock) -> None:
m = _mock_model("meta-llama/Llama-3", owned_by="sglang")
mock_cc.return_value = _mock_client(m)
result = probe_model_endpoint("openai", "http://localhost:30000/v1", "key")
assert result["server_type"] == "sglang"
@patch("turnstone.core.providers.create_client")
def test_server_type_vllm(self, mock_cc: MagicMock) -> None:
m = _mock_model("org/model-name")
mock_cc.return_value = _mock_client(m)
result = probe_model_endpoint("openai", "http://localhost:8000/v1", "key")
assert result["server_type"] == "vllm"
@patch("turnstone.core.providers.create_client")
def test_server_type_generic(self, mock_cc: MagicMock) -> None:
m = _mock_model("my-model")
mock_cc.return_value = _mock_client(m)
result = probe_model_endpoint("openai", "http://localhost:8000/v1", "key")
assert result["server_type"] == "openai-compatible"
@patch("turnstone.core.providers.create_client")
def test_anthropic_provider(self, mock_cc: MagicMock) -> None:
m = _mock_model("claude-sonnet-4-6")
mock_cc.return_value = _mock_client(m)
result = probe_model_endpoint(
"anthropic",
"https://api.anthropic.com",
"sk-ant-test",
target_model="claude-sonnet-4-6",
)
assert result["reachable"] is True
assert result["server_type"] == "anthropic"
assert result["context_window"] == 1000000
@patch("turnstone.core.providers.create_client")
def test_connection_failure(self, mock_cc: MagicMock) -> None:
mock_cc.side_effect = OSError("Connection refused")
result = probe_model_endpoint("openai", "http://bad:1234/v1", "key")
assert result["reachable"] is False
assert "Connection refused" in (result["error"] or "")
@patch("turnstone.core.providers.create_client")
def test_empty_model_list(self, mock_cc: MagicMock) -> None:
mock_cc.return_value = _mock_client() # no models
result = probe_model_endpoint("openai", "http://localhost:8000/v1", "key")
assert result["reachable"] is True
assert result["available_models"] == []
assert "No models found" in (result["error"] or "")
@patch("turnstone.core.providers.create_client")
def test_context_window_openai_static_table(self, mock_cc: MagicMock) -> None:
"""When base_url is api.openai.com and model is known, use static table."""
m = _mock_model("gpt-5")
mock_cc.return_value = _mock_client(m)
result = probe_model_endpoint(
"openai", "https://api.openai.com/v1", "sk-test", target_model="gpt-5"
)
assert result["context_window"] == 400000
# ---------------------------------------------------------------------------
# lookup_model_capabilities
# ---------------------------------------------------------------------------
class TestLookupModelCapabilities:
def test_known_openai_model(self) -> None:
caps = lookup_model_capabilities("openai", "gpt-5")
assert caps is not None
assert caps["context_window"] == 400000
assert caps["supports_temperature"] is False
def test_known_anthropic_model(self) -> None:
caps = lookup_model_capabilities("anthropic", "claude-opus-4-6")
assert caps is not None
assert caps["context_window"] == 1000000
assert caps["thinking_mode"] == "adaptive"
def test_unknown_model_returns_none(self) -> None:
caps = lookup_model_capabilities("openai", "totally-unknown-model")
assert caps is None
def test_tuples_converted_to_lists(self) -> None:
caps = lookup_model_capabilities("openai", "gpt-5")
assert caps is not None
for val in caps.values():
assert not isinstance(val, tuple), f"Found tuple: {val}"
def test_reasoning_effort_values_are_list(self) -> None:
caps = lookup_model_capabilities("openai", "gpt-5")
assert caps is not None
assert isinstance(caps["reasoning_effort_values"], list)
assert "medium" in caps["reasoning_effort_values"]
def test_openai_compatible_returns_none(self) -> None:
caps = lookup_model_capabilities("openai-compatible", "my-local-model")
assert caps is None
def test_invalid_provider_raises(self) -> None:
with pytest.raises(ValueError, match="Unknown provider"):
lookup_model_capabilities("bad-provider", "gpt-5")
# ---------------------------------------------------------------------------
# list_known_models
# ---------------------------------------------------------------------------
class TestListKnownModels:
def test_openai_models(self) -> None:
models = list_known_models("openai")
assert "gpt-5" in models
assert isinstance(models, list)
assert models == sorted(models)
def test_anthropic_models(self) -> None:
models = list_known_models("anthropic")
assert "claude-opus-4-6" in models
def test_openai_compatible_returns_empty(self) -> None:
assert list_known_models("openai-compatible") == []
def test_unknown_provider_returns_empty(self) -> None:
assert list_known_models("bad-provider") == []
+310 -1
View File
@@ -10,6 +10,8 @@ import pytest
from turnstone.core.model_registry import (
ModelConfig,
ModelRegistry,
_resolve_env_vars,
detect_model,
load_model_registry,
)
@@ -318,6 +320,280 @@ class TestLoadModelRegistry:
assert alt_cfg.api_key == "my-key"
# ---------------------------------------------------------------------------
# load_model_registry with DB storage
# ---------------------------------------------------------------------------
class _MockStorage:
"""Minimal storage mock returning canned model definitions."""
def __init__(self, rows: list[dict[str, Any]] | None = None) -> None:
self._rows = rows or []
self.calls: list[str] = []
def list_model_definitions(self, enabled_only: bool = False) -> list[dict[str, Any]]:
self.calls.append("list_model_definitions")
if enabled_only:
return [r for r in self._rows if r.get("enabled", True)]
return list(self._rows)
class TestLoadModelRegistryWithDB:
def test_db_models_loaded(self) -> None:
"""DB model definitions are loaded into the registry."""
storage = _MockStorage(
[
{
"alias": "cloud-gpt",
"model": "gpt-5",
"provider": "openai",
"base_url": "https://api.openai.com/v1",
"api_key": "sk-db",
"context_window": 128000,
"capabilities": "{}",
"enabled": True,
}
]
)
with patch("turnstone.core.model_registry.load_config", return_value={}):
reg = load_model_registry("http://x/v1", "x", "x", storage=storage)
assert reg.has_alias("cloud-gpt")
cfg = reg.get_config("cloud-gpt")
assert cfg.model == "gpt-5"
assert cfg.source == "db"
def test_config_overrides_db(self) -> None:
"""Config.toml entry overrides DB entry with same alias."""
storage = _MockStorage(
[
{
"alias": "shared",
"model": "db-model",
"provider": "openai",
"base_url": "http://db/v1",
"api_key": "sk-db",
"context_window": 32768,
"capabilities": "{}",
"enabled": True,
}
]
)
fake_cfg: dict[str, Any] = {
"models": {
"shared": {
"model": "config-model",
"base_url": "http://config/v1",
},
},
}
with patch("turnstone.core.model_registry.load_config", return_value=fake_cfg):
reg = load_model_registry("http://x/v1", "x", "x", storage=storage)
cfg = reg.get_config("shared")
assert cfg.model == "config-model"
assert cfg.source == "config"
def test_db_only_models_coexist(self) -> None:
"""DB models coexist alongside config.toml models."""
storage = _MockStorage(
[
{
"alias": "db-only",
"model": "db-model",
"provider": "anthropic",
"base_url": "",
"api_key": "sk-db",
"context_window": 200000,
"capabilities": "{}",
"enabled": True,
}
]
)
fake_cfg: dict[str, Any] = {
"models": {
"config-only": {"model": "config-model"},
},
}
with patch("turnstone.core.model_registry.load_config", return_value=fake_cfg):
reg = load_model_registry("http://x/v1", "x", "x", storage=storage)
assert reg.has_alias("db-only")
assert reg.has_alias("config-only")
assert reg.has_alias("default")
assert reg.get_config("db-only").source == "db"
assert reg.get_config("config-only").source == "config"
def test_source_field_set(self) -> None:
"""Source field correctly distinguishes origin."""
storage = _MockStorage(
[
{
"alias": "from-db",
"model": "m",
"provider": "openai",
"base_url": "",
"api_key": "",
"context_window": 32768,
"capabilities": "{}",
"enabled": True,
}
]
)
with patch("turnstone.core.model_registry.load_config", return_value={}):
reg = load_model_registry("http://x/v1", "x", "x", storage=storage)
assert reg.get_config("from-db").source == "db"
assert reg.get_config("default").source == ""
def test_disabled_db_models_excluded(self) -> None:
"""Disabled DB models are not loaded."""
storage = _MockStorage(
[
{
"alias": "disabled",
"model": "m",
"provider": "openai",
"base_url": "",
"api_key": "",
"context_window": 32768,
"capabilities": "{}",
"enabled": False,
}
]
)
with patch("turnstone.core.model_registry.load_config", return_value={}):
reg = load_model_registry("http://x/v1", "x", "x", storage=storage)
assert not reg.has_alias("disabled")
def test_db_capabilities_parsed(self) -> None:
"""JSON capabilities from DB are parsed into dict."""
storage = _MockStorage(
[
{
"alias": "caps-model",
"model": "m",
"provider": "openai",
"base_url": "",
"api_key": "",
"context_window": 32768,
"capabilities": '{"supports_vision": true}',
"enabled": True,
}
]
)
with patch("turnstone.core.model_registry.load_config", return_value={}):
reg = load_model_registry("http://x/v1", "x", "x", storage=storage)
assert reg.get_config("caps-model").capabilities == {"supports_vision": True}
def test_db_default_alias_not_clobbered(self) -> None:
"""DB model with alias='default' is not overwritten by CLI args."""
storage = _MockStorage(
[
{
"alias": "default",
"model": "db-default-model",
"provider": "openai",
"base_url": "http://db/v1",
"api_key": "sk-db",
"context_window": 128000,
"capabilities": "{}",
"enabled": True,
}
]
)
with patch("turnstone.core.model_registry.load_config", return_value={}):
reg = load_model_registry("http://cli/v1", "cli-key", "cli-model", storage=storage)
cfg = reg.get_config("default")
assert cfg.model == "db-default-model"
assert cfg.source == "db"
def test_no_db_writes(self) -> None:
"""Config.toml models are NOT written to storage."""
storage = _MockStorage()
fake_cfg: dict[str, Any] = {
"models": {"local": {"model": "llama"}},
}
with patch("turnstone.core.model_registry.load_config", return_value=fake_cfg):
load_model_registry("http://x/v1", "x", "x", storage=storage)
# Only list_model_definitions should be called, no create
assert storage.calls == ["list_model_definitions"]
def test_storage_failure_graceful(self) -> None:
"""Storage errors don't prevent registry creation."""
storage = MagicMock()
storage.list_model_definitions.side_effect = RuntimeError("db down")
with patch("turnstone.core.model_registry.load_config", return_value={}):
reg = load_model_registry("http://x/v1", "x", "x", storage=storage)
assert reg.has_alias("default")
# ---------------------------------------------------------------------------
# _resolve_env_vars
# ---------------------------------------------------------------------------
class TestResolveEnvVars:
def test_expand_single(self) -> None:
with patch.dict("os.environ", {"MY_KEY": "secret123"}):
assert _resolve_env_vars("sk-${MY_KEY}") == "sk-secret123"
def test_expand_multiple(self) -> None:
with patch.dict("os.environ", {"A": "1", "B": "2"}):
assert _resolve_env_vars("${A}-${B}") == "1-2"
def test_missing_var_empty(self) -> None:
with patch.dict("os.environ", {}, clear=True):
assert _resolve_env_vars("${MISSING}") == ""
def test_no_vars(self) -> None:
assert _resolve_env_vars("plain-key") == "plain-key"
def test_empty_string(self) -> None:
assert _resolve_env_vars("") == ""
# ---------------------------------------------------------------------------
# ModelRegistry.reload
# ---------------------------------------------------------------------------
class TestRegistryReload:
def test_reload_replaces_models(self) -> None:
models_a = {"a": ModelConfig("a", "x", "x", "m1")}
reg = ModelRegistry(models=models_a, default="a")
assert reg.has_alias("a")
models_b = {"b": ModelConfig("b", "y", "y", "m2")}
reg.reload(models_b, "b")
assert not reg.has_alias("a")
assert reg.has_alias("b")
assert reg.default == "b"
def test_reload_clears_clients(self) -> None:
models = {"a": ModelConfig("a", "http://x/v1", "key", "m")}
reg = ModelRegistry(models=models, default="a")
# Force client creation
reg.get_client("a")
assert "a" in reg._clients
# Reload with same models — clients should be cleared
reg.reload(dict(models), "a")
assert "a" not in reg._clients
def test_reload_validates_default(self) -> None:
models_a = {"a": ModelConfig("a", "x", "x", "m")}
reg = ModelRegistry(models=models_a, default="a")
with pytest.raises(ValueError, match="Default model"):
reg.reload(models_a, "nonexistent")
# Registry should be unchanged after failed reload
assert reg.has_alias("a")
assert reg.default == "a"
def test_reload_validates_empty(self) -> None:
models_a = {"a": ModelConfig("a", "x", "x", "m")}
reg = ModelRegistry(models=models_a, default="a")
with pytest.raises(ValueError, match="at least one"):
reg.reload({}, "a")
# ---------------------------------------------------------------------------
# Session integration
# ---------------------------------------------------------------------------
@@ -338,7 +614,7 @@ class _FakeUI:
def approve_tools(self, items: list[dict[str, Any]]) -> tuple[bool, str | None]:
return True, None
def on_tool_result(self, call_id: str, name: str, output: str) -> None: ...
def on_tool_result(self, call_id: str, name: str, output: str, **kwargs: Any) -> None: ...
def on_tool_output_chunk(self, call_id: str, chunk: str) -> None: ...
def on_status(self, usage: dict[str, Any], context_window: int, effort: str) -> None: ...
def on_plan_review(self, content: str) -> str:
@@ -590,3 +866,36 @@ class TestProtocolModel:
assert isinstance(restored, CreateWorkstreamMessage)
assert restored.model == "local"
assert restored.name == "ws1"
# ---------------------------------------------------------------------------
# detect_model — startup timeout
# ---------------------------------------------------------------------------
class TestDetectModelTimeout:
def test_uses_short_timeout_and_no_retries(self) -> None:
"""detect_model() uses with_options(timeout=10, max_retries=0)."""
mock_model = MagicMock()
mock_model.id = "test-model"
mock_model.owned_by = "test"
fast_client = MagicMock()
fast_client.models.list.return_value = MagicMock(data=[mock_model])
client = MagicMock()
client.with_options.return_value = fast_client
result = detect_model(client, provider="openai")
client.with_options.assert_called_once_with(timeout=10.0, max_retries=0)
fast_client.models.list.assert_called_once()
assert result[0] == "test-model"
def test_connection_error_non_fatal(self) -> None:
"""detect_model(fatal=False) returns (None, None) on connection error."""
client = MagicMock()
client.with_options.return_value = client
client.models.list.side_effect = OSError("Connection refused")
result = detect_model(client, provider="openai", fatal=False)
assert result == (None, None)
+172 -3
View File
@@ -22,6 +22,7 @@ from turnstone.core.oidc import (
load_oidc_config,
provision_oidc_user,
validate_id_token,
validate_issuer_url,
)
# ---------------------------------------------------------------------------
@@ -293,6 +294,162 @@ class TestLoadOIDCConfig:
assert cfg.redirect_base == "http://localhost:8000"
# ---------------------------------------------------------------------------
# SSRF Validation
# ---------------------------------------------------------------------------
class TestValidateIssuerURL:
"""Tests for ``validate_issuer_url`` SSRF protection."""
def test_valid_https_url(self):
"""Public HTTPS issuer URL passes validation."""
# Should not raise -- mock DNS to return a public IP.
with patch(
"socket.getaddrinfo",
return_value=[
(2, 1, 6, "", ("93.184.216.34", 0)),
],
):
validate_issuer_url("https://idp.example.com")
def test_rejects_http_non_localhost(self):
"""HTTP is rejected for non-localhost hosts."""
with pytest.raises(OIDCError, match="must use HTTPS"):
validate_issuer_url("http://idp.example.com")
def test_allows_http_localhost(self):
"""HTTP is allowed for localhost (development)."""
with patch(
"socket.getaddrinfo",
return_value=[
(2, 1, 6, "", ("127.0.0.1", 0)),
],
):
validate_issuer_url("http://localhost:8080")
def test_allows_http_localhost_subdomain(self):
"""HTTP is allowed for *.localhost subdomains."""
with patch(
"socket.getaddrinfo",
return_value=[
(2, 1, 6, "", ("127.0.0.1", 0)),
],
):
validate_issuer_url("http://keycloak.localhost:8080")
def test_rejects_embedded_credentials(self):
"""URLs with userinfo (user:pass@host) are rejected."""
with pytest.raises(OIDCError, match="embedded credentials"):
validate_issuer_url("https://admin:secret@idp.example.com")
def test_rejects_username_only(self):
"""URLs with just a username are rejected."""
with pytest.raises(OIDCError, match="embedded credentials"):
validate_issuer_url("https://admin@idp.example.com")
def test_rejects_no_hostname(self):
"""URLs without a hostname are rejected."""
with pytest.raises(OIDCError, match="no hostname"):
validate_issuer_url("https://")
def test_rejects_private_10_range(self):
"""Hostnames resolving to 10.x.x.x are rejected."""
with (
patch("socket.getaddrinfo", return_value=[(2, 1, 6, "", ("10.0.0.1", 0))]),
pytest.raises(OIDCError, match="non-public address.*10.0.0.1"),
):
validate_issuer_url("https://internal.corp.example.com")
def test_rejects_private_172_range(self):
"""Hostnames resolving to 172.16-31.x.x are rejected."""
with (
patch("socket.getaddrinfo", return_value=[(2, 1, 6, "", ("172.16.0.1", 0))]),
pytest.raises(OIDCError, match="non-public address.*172.16.0.1"),
):
validate_issuer_url("https://internal.corp.example.com")
def test_rejects_private_192_168_range(self):
"""Hostnames resolving to 192.168.x.x are rejected."""
with (
patch("socket.getaddrinfo", return_value=[(2, 1, 6, "", ("192.168.1.1", 0))]),
pytest.raises(OIDCError, match="non-public address.*192.168.1.1"),
):
validate_issuer_url("https://internal.corp.example.com")
def test_rejects_loopback_127(self):
"""Hostnames resolving to 127.x.x.x are rejected (non-localhost host)."""
with (
patch("socket.getaddrinfo", return_value=[(2, 1, 6, "", ("127.0.0.1", 0))]),
pytest.raises(OIDCError, match="non-public address.*127.0.0.1"),
):
validate_issuer_url("https://evil.example.com")
def test_rejects_ipv6_loopback(self):
"""Hostnames resolving to ::1 are rejected (non-localhost host)."""
with (
patch("socket.getaddrinfo", return_value=[(10, 1, 6, "", ("::1", 0, 0, 0))]),
pytest.raises(OIDCError, match="non-public address.*::1"),
):
validate_issuer_url("https://evil.example.com")
def test_rejects_ipv6_private(self):
"""Hostnames resolving to fc00::/7 are rejected."""
with (
patch("socket.getaddrinfo", return_value=[(10, 1, 6, "", ("fd00::1", 0, 0, 0))]),
pytest.raises(OIDCError, match="non-public address.*fd00::1"),
):
validate_issuer_url("https://evil.example.com")
def test_rejects_link_local(self):
"""Hostnames resolving to link-local addresses are rejected."""
with (
patch("socket.getaddrinfo", return_value=[(2, 1, 6, "", ("169.254.169.254", 0))]),
pytest.raises(OIDCError, match="non-public address.*169.254.169.254"),
):
validate_issuer_url("https://metadata.internal")
def test_rejects_unresolvable_hostname(self):
"""DNS resolution failure is rejected."""
import socket as _socket
with (
patch("socket.getaddrinfo", side_effect=_socket.gaierror("not found")),
pytest.raises(OIDCError, match="cannot be resolved"),
):
validate_issuer_url("https://nonexistent.invalid")
def test_rejects_mixed_addresses(self):
"""If any resolved address is private, the URL is rejected."""
with (
patch(
"socket.getaddrinfo",
return_value=[
(2, 1, 6, "", ("93.184.216.34", 0)),
(2, 1, 6, "", ("10.0.0.1", 0)),
],
),
pytest.raises(OIDCError, match="non-public address.*10.0.0.1"),
):
validate_issuer_url("https://dual-homed.example.com")
def test_discover_rejects_ssrf(self):
"""discover_oidc returns enabled=False when issuer URL fails SSRF check."""
config = _make_config(
issuer="http://10.0.0.1:8080",
authorization_endpoint="",
token_endpoint="",
userinfo_endpoint="",
jwks_uri="",
)
async def _run():
result = await discover_oidc(config)
assert result.enabled is False
asyncio.run(_run())
# ---------------------------------------------------------------------------
# Redirect URI Builder
# ---------------------------------------------------------------------------
@@ -869,6 +1026,9 @@ class TestApplyRoleMapping:
class TestDiscoverOIDC:
# Mock DNS result for a public IP — reused across discovery tests.
_PUBLIC_ADDR = [(2, 1, 6, "", ("93.184.216.34", 0))]
def test_discover_oidc_success(self):
"""Mock httpx response, verify endpoints populated."""
config = _make_config(
@@ -891,7 +1051,10 @@ class TestDiscoverOIDC:
async def _run():
client = _mock_async_client(lambda url: _async_return(mock_response))
with patch("httpx.AsyncClient", return_value=client):
with (
patch("socket.getaddrinfo", return_value=self._PUBLIC_ADDR),
patch("httpx.AsyncClient", return_value=client),
):
result = await discover_oidc(config)
assert result.authorization_endpoint == "https://idp.example.com/authorize"
@@ -916,7 +1079,10 @@ class TestDiscoverOIDC:
async def _run():
client = _mock_async_client(_failing_get)
with patch("httpx.AsyncClient", return_value=client):
with (
patch("socket.getaddrinfo", return_value=self._PUBLIC_ADDR),
patch("httpx.AsyncClient", return_value=client),
):
result = await discover_oidc(config)
assert result.enabled is False
@@ -954,7 +1120,10 @@ class TestDiscoverOIDC:
async def _run():
client = _mock_async_client(lambda url: _async_return(mock_response))
with patch("httpx.AsyncClient", return_value=client):
with (
patch("socket.getaddrinfo", return_value=self._PUBLIC_ADDR),
patch("httpx.AsyncClient", return_value=client),
):
result = await discover_oidc(config)
assert result.enabled is False
+3 -3
View File
@@ -131,7 +131,7 @@ def authorize_client(storage: SQLiteBackend, oidc_config: OIDCConfig) -> TestCli
)
app.state.oidc_config = oidc_config
app.state.auth_storage = storage
app.state.jwt_secret = "test-jwt-secret"
app.state.jwt_secret = "test-jwt-secret-key-padded-32b!!"
app.state.jwks_data = {"keys": []}
app.state.login_limiter = None
return TestClient(app, raise_server_exceptions=False)
@@ -468,7 +468,7 @@ class TestOIDCCallback:
)
app.state.oidc_config = _make_oidc_config()
app.state.auth_storage = backend
app.state.jwt_secret = "secret"
app.state.jwt_secret = "test-jwt-secret-key-padded-32b!!"
app.state.jwks_data = {"keys": []}
app.state.login_limiter = None
@@ -492,7 +492,7 @@ class TestOIDCCallback:
)
app.state.oidc_config = _make_oidc_config()
app.state.auth_storage = storage
app.state.jwt_secret = "secret"
app.state.jwt_secret = "test-jwt-secret-key-padded-32b!!"
app.state.jwks_data = {"keys": []}
limiter = LoginRateLimiter(max_attempts=1, window_seconds=300)
limiter.record("ip:testclient")
-9
View File
@@ -6,15 +6,6 @@ import time
import pytest
from turnstone.core.storage._sqlite import SQLiteBackend
@pytest.fixture()
def db(tmp_path):
"""Create a fresh SQLite backend for each test."""
return SQLiteBackend(str(tmp_path / "test.db"))
# ---------------------------------------------------------------------------
# OIDC Identity CRUD
# ---------------------------------------------------------------------------
-10
View File
@@ -4,16 +4,6 @@ from __future__ import annotations
from datetime import UTC, datetime, timedelta
import pytest
from turnstone.core.storage._sqlite import SQLiteBackend
@pytest.fixture()
def db(tmp_path):
"""Fresh SQLite backend for each test."""
return SQLiteBackend(str(tmp_path / "test.db"))
def _make_assessment_kwargs(**overrides):
"""Build default kwargs for record_output_assessment."""
+1 -1
View File
@@ -29,7 +29,7 @@ class NullUI:
def approve_tools(self, items):
return True, None
def on_tool_result(self, call_id, name, output):
def on_tool_result(self, call_id, name, output, **kwargs):
pass
def on_tool_output_chunk(self, call_id, chunk):
+346 -13
View File
@@ -94,6 +94,7 @@ def _anthropic_event(
delta.type = kwargs.get("delta_type", "text_delta")
delta.text = kwargs.get("text", "")
delta.thinking = kwargs.get("thinking", "")
delta.signature = kwargs.get("signature", "")
delta.partial_json = kwargs.get("partial_json", "")
event.delta = delta
event.index = kwargs.get("index", 0)
@@ -140,6 +141,34 @@ class TestOpenAIProvider:
def test_provider_name(self) -> None:
assert self.provider.provider_name == "openai"
# -- _sanitize_messages ---------------------------------------------------
def test_sanitize_messages_none_content_no_tool_calls(self) -> None:
msgs = [{"role": "assistant", "content": None}]
assert self.provider._sanitize_messages(msgs) == [{"role": "assistant", "content": ""}]
def test_sanitize_messages_none_content_with_tool_calls(self) -> None:
msgs = [{"role": "assistant", "content": None, "tool_calls": [{"id": "1"}]}]
result = self.provider._sanitize_messages(msgs)
assert result[0]["content"] is None
assert result[0]["tool_calls"] == [{"id": "1"}]
def test_sanitize_messages_empty_string_passthrough(self) -> None:
msgs = [{"role": "assistant", "content": ""}]
assert self.provider._sanitize_messages(msgs) == msgs
def test_sanitize_messages_non_assistant_unchanged(self) -> None:
msgs = [{"role": "user", "content": None}]
result = self.provider._sanitize_messages(msgs)
assert result[0]["content"] is None
def test_sanitize_messages_does_not_mutate_original(self) -> None:
original = {"role": "assistant", "content": None}
self.provider._sanitize_messages([original])
assert original["content"] is None
# -- convert_tools --------------------------------------------------------
def test_convert_tools_passthrough(self) -> None:
tools = [
{
@@ -479,10 +508,11 @@ class TestAnthropicProvider:
},
}
],
}
},
{"role": "tool", "tool_call_id": "call_1", "content": "file contents"},
]
_, converted = self.provider._convert_messages(messages)
assert len(converted) == 1
assert len(converted) == 2
blocks = converted[0]["content"]
assert len(blocks) == 2
assert blocks[0] == {"type": "text", "text": "Let me check that."}
@@ -490,6 +520,8 @@ class TestAnthropicProvider:
assert blocks[1]["id"] == "call_1"
assert blocks[1]["name"] == "read_file"
assert blocks[1]["input"] == {"path": "foo.py"}
# Tool result in user message
assert converted[1]["role"] == "user"
def test_message_conversion_tool_results(self) -> None:
messages = [
@@ -598,7 +630,11 @@ class TestAnthropicProvider:
response.usage.output_tokens = 5
client = MagicMock()
client.messages.create.return_value = response
stream_ctx = MagicMock()
stream_ctx.__enter__ = MagicMock(return_value=stream_ctx)
stream_ctx.__exit__ = MagicMock(return_value=False)
stream_ctx.get_final_message.return_value = response
client.messages.stream.return_value = stream_ctx
result = self.provider.create_completion(
client=client,
@@ -630,7 +666,11 @@ class TestAnthropicProvider:
response.usage.output_tokens = 20
client = MagicMock()
client.messages.create.return_value = response
stream_ctx = MagicMock()
stream_ctx.__enter__ = MagicMock(return_value=stream_ctx)
stream_ctx.__exit__ = MagicMock(return_value=False)
stream_ctx.get_final_message.return_value = response
client.messages.stream.return_value = stream_ctx
result = self.provider.create_completion(
client=client,
@@ -661,7 +701,11 @@ class TestAnthropicProvider:
response.usage.output_tokens = 50
client = MagicMock()
client.messages.create.return_value = response
stream_ctx = MagicMock()
stream_ctx.__enter__ = MagicMock(return_value=stream_ctx)
stream_ctx.__exit__ = MagicMock(return_value=False)
stream_ctx.get_final_message.return_value = response
client.messages.stream.return_value = stream_ctx
result = self.provider.create_completion(
client=client,
@@ -913,7 +957,7 @@ class TestAnthropicHelpers:
provider = AnthropicProvider()
caps = provider.get_capabilities("claude-opus-4-6")
assert caps.context_window == 200000
assert caps.context_window == 1000000
assert caps.max_output_tokens == 128000
assert caps.thinking_mode == "adaptive"
assert caps.supports_effort is True
@@ -922,11 +966,11 @@ class TestAnthropicHelpers:
from turnstone.core.providers._anthropic import AnthropicProvider
provider = AnthropicProvider()
# Prefix match: "claude-sonnet-4" matches dated variants
caps = provider.get_capabilities("claude-sonnet-4-20260101")
assert caps.context_window == 200000
# Prefix match: "claude-sonnet-4-6" matches dated variants
caps = provider.get_capabilities("claude-sonnet-4-6-20260101")
assert caps.context_window == 1000000
assert caps.token_param == "max_tokens"
assert caps.thinking_mode == "manual"
assert caps.thinking_mode == "adaptive"
def test_capabilities_lookup_unknown(self) -> None:
from turnstone.core.providers._anthropic import AnthropicProvider
@@ -1141,6 +1185,189 @@ class TestOpenAIParameterGating:
assert kwargs["reasoning_effort"] == "medium" # fell back from unsupported "low"
class TestAnthropicOrphanedToolUse:
"""Verify _convert_messages synthesizes tool_results for orphaned tool_use."""
def setup_method(self) -> None:
from turnstone.core.providers._anthropic import AnthropicProvider
self.provider = AnthropicProvider()
def test_orphaned_tool_use_gets_synthetic_result(self) -> None:
"""Assistant has tool_calls but next message is user (no tool results)."""
messages = [
{"role": "user", "content": "do something"},
{
"role": "assistant",
"content": "I'll run that.",
"tool_calls": [
{
"id": "call_abc",
"function": {"name": "bash", "arguments": '{"command": "ls"}'},
}
],
},
{"role": "user", "content": "never mind, do something else"},
]
_, converted = self.provider._convert_messages(messages)
# Should have: user, assistant(tool_use), user(synthetic tool_result), user
# After _merge_consecutive, the two user messages may merge.
# Find the synthetic tool_result
tool_results = []
for msg in converted:
if msg["role"] == "user" and isinstance(msg["content"], list):
for block in msg["content"]:
if isinstance(block, dict) and block.get("type") == "tool_result":
tool_results.append(block)
assert len(tool_results) == 1
assert tool_results[0]["tool_use_id"] == "call_abc"
assert tool_results[0]["is_error"] is True
assert "cancelled" in tool_results[0]["content"].lower()
def test_multiple_orphaned_tool_calls(self) -> None:
"""Assistant has 3 tool_calls, none have results."""
messages = [
{"role": "user", "content": "do three things"},
{
"role": "assistant",
"content": "",
"tool_calls": [
{"id": "c1", "function": {"name": "bash", "arguments": "{}"}},
{"id": "c2", "function": {"name": "read_file", "arguments": "{}"}},
{"id": "c3", "function": {"name": "write_file", "arguments": "{}"}},
],
},
{"role": "user", "content": "skip all that"},
]
_, converted = self.provider._convert_messages(messages)
tool_results = []
for msg in converted:
if msg["role"] == "user" and isinstance(msg["content"], list):
for block in msg["content"]:
if isinstance(block, dict) and block.get("type") == "tool_result":
tool_results.append(block)
assert len(tool_results) == 3
result_ids = {r["tool_use_id"] for r in tool_results}
assert result_ids == {"c1", "c2", "c3"}
def test_partial_results_only_orphans_synthesized(self) -> None:
"""2 tool_calls, only 1 has a result — synthesize for the missing one."""
messages = [
{"role": "user", "content": "do two things"},
{
"role": "assistant",
"content": "",
"tool_calls": [
{"id": "c1", "function": {"name": "bash", "arguments": "{}"}},
{"id": "c2", "function": {"name": "write_file", "arguments": "{}"}},
],
},
{"role": "tool", "tool_call_id": "c1", "content": "file1.txt"},
{"role": "user", "content": "skip the write"},
]
_, converted = self.provider._convert_messages(messages)
# c1 should have a real result, c2 should have a synthetic one
tool_results = []
for msg in converted:
if msg["role"] == "user" and isinstance(msg["content"], list):
for block in msg["content"]:
if isinstance(block, dict) and block.get("type") == "tool_result":
tool_results.append(block)
# Real result should come before synthetic (ordering matters for Anthropic)
assert len(tool_results) == 2
assert tool_results[0]["tool_use_id"] == "c1"
assert tool_results[0]["content"] == "file1.txt" # real result
assert tool_results[0].get("is_error") is not True
assert tool_results[1]["tool_use_id"] == "c2"
assert tool_results[1]["is_error"] is True # synthetic
def test_complete_results_no_synthesis(self) -> None:
"""All tool_calls have results — no synthesis needed."""
messages = [
{"role": "user", "content": "do it"},
{
"role": "assistant",
"content": "",
"tool_calls": [
{"id": "c1", "function": {"name": "bash", "arguments": "{}"}},
],
},
{"role": "tool", "tool_call_id": "c1", "content": "done"},
{"role": "user", "content": "thanks"},
]
_, converted = self.provider._convert_messages(messages)
# No synthetic results — only the real one (no is_error flag)
tool_results = []
for msg in converted:
if msg["role"] == "user" and isinstance(msg["content"], list):
for block in msg["content"]:
if isinstance(block, dict) and block.get("type") == "tool_result":
tool_results.append(block)
assert len(tool_results) == 1
assert tool_results[0]["tool_use_id"] == "c1"
assert tool_results[0].get("is_error") is not True
def test_trailing_orphan(self) -> None:
"""Orphaned tool_use at end of conversation (no following messages)."""
messages = [
{"role": "user", "content": "do it"},
{
"role": "assistant",
"content": "Running...",
"tool_calls": [
{"id": "c1", "function": {"name": "bash", "arguments": "{}"}},
],
},
]
_, converted = self.provider._convert_messages(messages)
tool_results = []
for msg in converted:
if msg["role"] == "user" and isinstance(msg["content"], list):
for block in msg["content"]:
if isinstance(block, dict) and block.get("type") == "tool_result":
tool_results.append(block)
assert len(tool_results) == 1
assert tool_results[0]["tool_use_id"] == "c1"
assert tool_results[0]["is_error"] is True
def test_provider_content_orphan(self) -> None:
"""Orphaned tool_use inside _provider_content (Anthropic raw blocks)."""
messages = [
{"role": "user", "content": "run something"},
{
"role": "assistant",
"content": "Running...",
"_provider_content": [
{"type": "text", "text": "Running..."},
{
"type": "tool_use",
"id": "toolu_abc",
"name": "bash",
"input": {"command": "sleep 30"},
},
],
"tool_calls": [
{
"id": "toolu_abc",
"function": {"name": "bash", "arguments": '{"command": "sleep 30"}'},
},
],
},
{"role": "user", "content": "never mind"},
]
_, converted = self.provider._convert_messages(messages)
# Should synthesize a tool_result for the orphaned tool_use in provider_content
tool_results = []
for msg in converted:
if msg["role"] == "user" and isinstance(msg["content"], list):
for block in msg["content"]:
if isinstance(block, dict) and block.get("type") == "tool_result":
tool_results.append(block)
assert len(tool_results) == 1
assert tool_results[0]["tool_use_id"] == "toolu_abc"
assert tool_results[0]["is_error"] is True
class TestAnthropicReasoningNone:
"""Verify 'none' effort disables thinking for manual-thinking models."""
@@ -1180,7 +1407,7 @@ class TestAnthropicWebSearch:
"""All Anthropic models should support native web search."""
caps = self.provider.get_capabilities("claude-opus-4-6")
assert caps.supports_web_search is True
caps = self.provider.get_capabilities("claude-sonnet-4")
caps = self.provider.get_capabilities("claude-sonnet-4-6")
assert caps.supports_web_search is True
# Unknown models use default which also has web search
caps = self.provider.get_capabilities("claude-unknown-99")
@@ -1383,7 +1610,11 @@ class TestAnthropicWebSearch:
response.usage.output_tokens = 50
client = MagicMock()
client.messages.create.return_value = response
stream_ctx = MagicMock()
stream_ctx.__enter__ = MagicMock(return_value=stream_ctx)
stream_ctx.__exit__ = MagicMock(return_value=False)
stream_ctx.get_final_message.return_value = response
client.messages.stream.return_value = stream_ctx
with patch("turnstone.core.providers._anthropic._ensure_anthropic"):
result = self.provider.create_completion(
@@ -1934,6 +2165,104 @@ class TestAnthropicProviderBlocks:
assert blocks[2]["type"] == "web_search_tool_result"
assert blocks[2]["encrypted_content"] == "enc_data"
def test_streaming_thinking_block_captures_signature(self) -> None:
"""Streaming thinking block accumulates signature from signature_delta events."""
thinking_block = MagicMock()
thinking_block.type = "thinking"
thinking_block.model_dump.return_value = {
"type": "thinking",
"thinking": "",
"signature": "",
}
text_block = MagicMock()
text_block.type = "text"
text_block.model_dump.return_value = {"type": "text", "text": ""}
events = [
MagicMock(type="content_block_start", index=0, content_block=thinking_block),
_anthropic_event(
"content_block_delta", delta_type="thinking_delta", thinking="step 1", index=0
),
_anthropic_event(
"content_block_delta", delta_type="thinking_delta", thinking=" step 2", index=0
),
_anthropic_event(
"content_block_delta",
delta_type="signature_delta",
signature="sig_part1",
index=0,
),
_anthropic_event(
"content_block_delta",
delta_type="signature_delta",
signature="sig_part2",
index=0,
),
_anthropic_event("content_block_stop", index=0),
MagicMock(type="content_block_start", index=1, content_block=text_block),
_anthropic_event("content_block_delta", delta_type="text_delta", text="Hello", index=1),
_anthropic_event("content_block_stop", index=1),
_anthropic_event("message_delta", stop_reason="end_turn", usage_output_tokens=50),
]
chunks = list(self.provider._iter_anthropic_stream(iter(events)))
final_chunks = [c for c in chunks if c.provider_blocks]
assert len(final_chunks) == 1
blocks = final_chunks[0].provider_blocks
assert blocks[0]["type"] == "thinking"
assert blocks[0]["thinking"] == "step 1 step 2"
assert blocks[0]["signature"] == "sig_part1sig_part2"
def test_thinking_block_multiturn_roundtrip(self) -> None:
"""Thinking block with signature survives _convert_messages round-trip."""
provider_content = [
{
"type": "thinking",
"thinking": "Let me reason...",
"signature": "ErUBCkYIAxgCIkD_valid_sig",
},
{"type": "text", "text": "Here is my answer."},
]
messages = [
{"role": "user", "content": "Question"},
{
"role": "assistant",
"content": "Here is my answer.",
"_provider_content": provider_content,
},
{"role": "user", "content": "Follow up"},
]
_, converted = self.provider._convert_messages(messages)
assistant_msg = converted[1]
assert assistant_msg["content"] is provider_content
assert assistant_msg["content"][0]["signature"] == "ErUBCkYIAxgCIkD_valid_sig"
assert assistant_msg["content"][0]["type"] == "thinking"
def test_block_to_dict_preserves_thinking_signature(self) -> None:
"""_block_to_dict preserves signature on thinking blocks."""
from turnstone.core.providers._anthropic import _block_to_dict
class FakeThinkingBlock:
def model_dump(self, **kwargs: Any) -> dict[str, Any]:
return {
"type": "thinking",
"thinking": "reasoning...",
"signature": "abc123sig",
}
result = _block_to_dict(FakeThinkingBlock())
assert result["signature"] == "abc123sig"
# Also test fallback path (no model_dump)
class FallbackBlock:
type = "thinking"
thinking = "reasoning..."
signature = "abc123sig"
result2 = _block_to_dict(FallbackBlock())
assert result2["signature"] == "abc123sig"
# ---------------------------------------------------------------------------
# Tool search tests
@@ -2307,7 +2636,11 @@ class TestAnthropicPromptCaching:
response.usage = usage
client = MagicMock()
client.messages.create.return_value = response
stream_ctx = MagicMock()
stream_ctx.__enter__ = MagicMock(return_value=stream_ctx)
stream_ctx.__exit__ = MagicMock(return_value=False)
stream_ctx.get_final_message.return_value = response
client.messages.stream.return_value = stream_ctx
result = self.provider.create_completion(
client=client,
+90 -3
View File
@@ -9,13 +9,12 @@ def _row(
role,
content=None,
tool_name=None,
tool_args=None,
tc_id=None,
pdata=None,
tool_calls=None,
):
"""Build a 7-element conversation row tuple (post-migration 013 format)."""
return (role, content, tool_name, tool_args, tc_id, pdata, tool_calls)
"""Build a 6-element conversation row tuple (post-migration 027 format)."""
return (role, content, tool_name, tc_id, pdata, tool_calls)
class TestAssistantWithToolCalls:
@@ -227,3 +226,91 @@ class TestEdgeCases:
assert len(msgs) == 2
assert msgs[0]["role"] == "user"
assert msgs[1]["role"] == "assistant"
class TestMidConversationOrphanRepair:
"""Mid-conversation orphaned tool_calls get synthetic tool results."""
def test_all_orphaned_mid_conversation(self):
"""Assistant has 2 tool_calls, no tool results, then user message."""
tc = json.dumps(
[
{"id": "c1", "function": {"name": "bash", "arguments": "{}"}},
{"id": "c2", "function": {"name": "write_file", "arguments": "{}"}},
]
)
rows = [
_row("user", "do stuff"),
_row("assistant", "Running...", tool_calls=tc),
_row("user", "never mind"),
]
msgs = reconstruct_messages(rows, "ws1")
# Should have: user, assistant, tool(c1), tool(c2), user
assert len(msgs) == 5
assert msgs[2]["role"] == "tool"
assert msgs[2]["tool_call_id"] == "c1"
assert msgs[2]["is_error"] is True
assert msgs[3]["role"] == "tool"
assert msgs[3]["tool_call_id"] == "c2"
assert msgs[4]["role"] == "user"
def test_partial_results_mid_conversation(self):
"""2 tool_calls, 1 result present, 1 missing — synthesize only the missing one."""
tc = json.dumps(
[
{"id": "c1", "function": {"name": "bash", "arguments": "{}"}},
{"id": "c2", "function": {"name": "write_file", "arguments": "{}"}},
]
)
rows = [
_row("user", "do stuff"),
_row("assistant", "", tool_calls=tc),
_row("tool", "file1.txt", tool_name="bash", tc_id="c1"),
_row("user", "skip the write"),
]
msgs = reconstruct_messages(rows, "ws1")
# Should have: user, assistant, tool(c1 real), tool(c2 synthetic), user
assert len(msgs) == 5
assert msgs[2]["role"] == "tool"
assert msgs[2]["tool_call_id"] == "c1"
assert msgs[2]["content"] == "file1.txt"
assert msgs[2].get("is_error") is not True
assert msgs[3]["role"] == "tool"
assert msgs[3]["tool_call_id"] == "c2"
assert msgs[3]["is_error"] is True
assert msgs[4]["role"] == "user"
def test_complete_results_no_synthesis(self):
"""All tool_calls have results — no synthesis needed."""
tc = json.dumps(
[
{"id": "c1", "function": {"name": "bash", "arguments": "{}"}},
]
)
rows = [
_row("user", "do it"),
_row("assistant", "", tool_calls=tc),
_row("tool", "done", tool_name="bash", tc_id="c1"),
_row("user", "thanks"),
]
msgs = reconstruct_messages(rows, "ws1")
assert len(msgs) == 4
tool_msgs = [m for m in msgs if m["role"] == "tool"]
assert len(tool_msgs) == 1
assert tool_msgs[0].get("is_error") is not True
def test_trailing_orphan_stripped_not_synthesized(self):
"""Trailing orphan is handled by the existing strip repair, not synthesis."""
tc = json.dumps(
[
{"id": "c1", "function": {"name": "bash", "arguments": "{}"}},
]
)
rows = [
_row("user", "do it"),
_row("assistant", "Running...", tool_calls=tc),
]
msgs = reconstruct_messages(rows, "ws1")
# Trailing strip removes the assistant message entirely
assert len(msgs) == 1
assert msgs[0]["role"] == "user"
+387
View File
@@ -0,0 +1,387 @@
"""Tests for conversation rewind and retry functionality."""
from __future__ import annotations
from unittest.mock import MagicMock
from turnstone.core.session import ChatSession
# ---------------------------------------------------------------------------
# Helpers
# ---------------------------------------------------------------------------
class NullUI:
"""UI adapter that discards all output."""
def on_thinking_start(self):
pass
def on_thinking_stop(self):
pass
def on_reasoning_token(self, text):
pass
def on_content_token(self, text):
pass
def on_stream_end(self):
pass
def approve_tools(self, items):
return True, None
def on_tool_result(self, call_id, name, output, **kwargs):
pass
def on_tool_output_chunk(self, call_id, chunk):
pass
def on_status(self, usage, context_window, effort):
pass
def on_plan_review(self, content):
return ""
def on_info(self, message):
pass
def on_error(self, message):
pass
def on_state_change(self, state):
pass
def on_rename(self, name):
pass
def on_output_warning(self, call_id, assessment):
pass
def _make_session(tmp_db) -> ChatSession:
return ChatSession(
client=MagicMock(),
model="test-model",
ui=NullUI(),
instructions="",
temperature=0.5,
max_tokens=4096,
tool_timeout=30,
)
def _populate_simple(session: ChatSession) -> None:
"""Populate with 2 simple turns (no tool calls)."""
session.messages = [
{"role": "user", "content": "Hello"},
{"role": "assistant", "content": "Hi there!"},
{"role": "user", "content": "How are you?"},
{"role": "assistant", "content": "I'm fine."},
]
session._msg_tokens = [10, 20, 10, 20]
def _populate_with_tools(session: ChatSession) -> None:
"""Populate with 2 turns, first has tool calls."""
session.messages = [
{"role": "user", "content": "Write a test"},
{
"role": "assistant",
"content": "",
"tool_calls": [
{"id": "tc1", "function": {"name": "bash", "arguments": '{"cmd":"echo hi"}'}}
],
},
{"role": "tool", "tool_call_id": "tc1", "content": "hi"},
{"role": "assistant", "content": "Done."},
{"role": "user", "content": "Fix the import"},
{"role": "assistant", "content": "Fixed."},
]
session._msg_tokens = [10, 20, 10, 20, 10, 20]
# ---------------------------------------------------------------------------
# _find_turn_boundaries
# ---------------------------------------------------------------------------
class TestFindTurnBoundaries:
def test_empty_messages(self, tmp_db):
session = _make_session(tmp_db)
assert session._find_turn_boundaries() == []
def test_single_turn(self, tmp_db):
session = _make_session(tmp_db)
session.messages = [
{"role": "user", "content": "Hello"},
{"role": "assistant", "content": "Hi!"},
]
assert session._find_turn_boundaries() == [0]
def test_multi_turn(self, tmp_db):
session = _make_session(tmp_db)
_populate_simple(session)
assert session._find_turn_boundaries() == [0, 2]
def test_with_tool_calls(self, tmp_db):
session = _make_session(tmp_db)
_populate_with_tools(session)
assert session._find_turn_boundaries() == [0, 4]
# ---------------------------------------------------------------------------
# rewind
# ---------------------------------------------------------------------------
class TestRewind:
def test_rewind_zero(self, tmp_db):
session = _make_session(tmp_db)
_populate_simple(session)
assert session.rewind(0) == 0
assert len(session.messages) == 4
def test_rewind_one_turn(self, tmp_db):
session = _make_session(tmp_db)
_populate_simple(session)
removed = session.rewind(1)
assert removed == 2 # user + assistant
assert len(session.messages) == 2
assert session.messages[0]["content"] == "Hello"
assert session.messages[1]["content"] == "Hi there!"
assert len(session._msg_tokens) == 2
def test_rewind_all_turns(self, tmp_db):
session = _make_session(tmp_db)
_populate_simple(session)
removed = session.rewind(2)
assert removed == 4
assert len(session.messages) == 0
assert len(session._msg_tokens) == 0
def test_rewind_clamped(self, tmp_db):
"""Rewinding more turns than exist should clamp to available."""
session = _make_session(tmp_db)
_populate_simple(session)
removed = session.rewind(999)
assert removed == 4
assert len(session.messages) == 0
def test_rewind_empty(self, tmp_db):
session = _make_session(tmp_db)
assert session.rewind(1) == 0
def test_rewind_with_tools(self, tmp_db):
"""Rewinding 1 turn on a multi-sub-turn conversation."""
session = _make_session(tmp_db)
_populate_with_tools(session)
removed = session.rewind(1)
assert removed == 2 # user "Fix the import" + assistant "Fixed."
assert len(session.messages) == 4
assert session.messages[-1]["content"] == "Done."
def test_rewind_tokens_sync(self, tmp_db):
"""_msg_tokens stays in sync with messages."""
session = _make_session(tmp_db)
_populate_simple(session)
session.rewind(1)
assert len(session._msg_tokens) == len(session.messages)
# ---------------------------------------------------------------------------
# retry
# ---------------------------------------------------------------------------
class TestRetry:
def test_retry_returns_user_message(self, tmp_db):
session = _make_session(tmp_db)
_populate_simple(session)
msg = session.retry()
assert msg == "How are you?"
# Only Turn 1 remains, without the second user message
assert len(session.messages) == 2
assert session.messages[-1]["content"] == "Hi there!"
def test_retry_empty(self, tmp_db):
session = _make_session(tmp_db)
assert session.retry() is None
def test_retry_with_tools(self, tmp_db):
session = _make_session(tmp_db)
_populate_with_tools(session)
msg = session.retry()
assert msg == "Fix the import"
# Only Turn 1 remains (user + assistant w/tools + tool result + assistant)
assert len(session.messages) == 4
def test_retry_sets_pending(self, tmp_db):
"""handle_command for /retry should set _pending_retry."""
session = _make_session(tmp_db)
_populate_simple(session)
session.handle_command("/retry")
assert session._pending_retry == "How are you?"
def test_retry_tokens_sync(self, tmp_db):
session = _make_session(tmp_db)
_populate_simple(session)
session.retry()
assert len(session._msg_tokens) == len(session.messages)
def test_retry_multipart_content_returns_none(self, tmp_db):
"""retry() should refuse multipart (vision/image) messages."""
session = _make_session(tmp_db)
session.messages = [
{"role": "user", "content": [{"type": "text", "text": "describe this"}]},
{"role": "assistant", "content": "It's an image."},
]
session._msg_tokens = [10, 20]
assert session.retry() is None
# Messages should be unchanged
assert len(session.messages) == 2
def test_retry_none_content_returns_none(self, tmp_db):
"""retry() should handle content=None gracefully."""
session = _make_session(tmp_db)
session.messages = [
{"role": "user", "content": None},
{"role": "assistant", "content": "Ok."},
]
session._msg_tokens = [10, 20]
assert session.retry() is None
# ---------------------------------------------------------------------------
# handle_command integration
# ---------------------------------------------------------------------------
class TestHandleCommand:
def test_rewind_command(self, tmp_db):
session = _make_session(tmp_db)
_populate_simple(session)
session.handle_command("/rewind 1")
assert len(session.messages) == 2
def test_rewind_no_arg(self, tmp_db):
session = _make_session(tmp_db)
ui = session.ui
ui.on_info = MagicMock()
session.handle_command("/rewind")
ui.on_info.assert_called_once()
assert "Usage" in ui.on_info.call_args[0][0]
def test_rewind_invalid_arg(self, tmp_db):
session = _make_session(tmp_db)
ui = session.ui
ui.on_info = MagicMock()
session.handle_command("/rewind abc")
ui.on_info.assert_called_once()
assert "integer" in ui.on_info.call_args[0][0]
def test_retry_nothing_to_retry(self, tmp_db):
session = _make_session(tmp_db)
ui = session.ui
ui.on_info = MagicMock()
session.handle_command("/retry")
ui.on_info.assert_called_once()
assert "Nothing" in ui.on_info.call_args[0][0]
# ---------------------------------------------------------------------------
# Storage integration — delete_messages_after
# ---------------------------------------------------------------------------
class TestDeleteMessagesAfter:
def test_delete_truncates_db(self, tmp_db):
from turnstone.core.memory import (
delete_messages_after,
load_messages,
register_workstream,
save_message,
)
ws_id = "test-ws-delete"
register_workstream(ws_id)
save_message(ws_id, "user", "Hello")
save_message(ws_id, "assistant", "Hi!")
save_message(ws_id, "user", "Bye")
save_message(ws_id, "assistant", "Goodbye!")
deleted = delete_messages_after(ws_id, 2)
assert deleted == 2
msgs = load_messages(ws_id)
assert len(msgs) == 2
assert msgs[0]["content"] == "Hello"
assert msgs[1]["content"] == "Hi!"
def test_delete_nothing(self, tmp_db):
from turnstone.core.memory import (
delete_messages_after,
register_workstream,
save_message,
)
ws_id = "test-ws-noop"
register_workstream(ws_id)
save_message(ws_id, "user", "Hello")
deleted = delete_messages_after(ws_id, 10)
assert deleted == 0
def test_delete_all(self, tmp_db):
from turnstone.core.memory import (
delete_messages_after,
load_messages,
register_workstream,
save_message,
)
ws_id = "test-ws-all"
register_workstream(ws_id)
save_message(ws_id, "user", "Hello")
save_message(ws_id, "assistant", "Hi!")
deleted = delete_messages_after(ws_id, 0)
assert deleted == 2
assert load_messages(ws_id) == []
# ---------------------------------------------------------------------------
# End-to-end: rewind + DB sync
# ---------------------------------------------------------------------------
class TestRewindDBSync:
def test_rewind_persists_to_db(self, tmp_db):
from turnstone.core.memory import load_messages, register_workstream, save_message
session = _make_session(tmp_db)
ws_id = session.ws_id
register_workstream(ws_id)
# Persist messages to DB and set in-memory state
save_message(ws_id, "user", "Hello")
save_message(ws_id, "assistant", "Hi!")
save_message(ws_id, "user", "Bye")
save_message(ws_id, "assistant", "Goodbye!")
session.messages = [
{"role": "user", "content": "Hello"},
{"role": "assistant", "content": "Hi!"},
{"role": "user", "content": "Bye"},
{"role": "assistant", "content": "Goodbye!"},
]
session._msg_tokens = [5, 5, 5, 5]
session.rewind(1)
# Verify DB matches in-memory state
db_msgs = load_messages(ws_id)
assert len(db_msgs) == 2
assert db_msgs[0]["content"] == "Hello"
assert db_msgs[1]["content"] == "Hi!"
-11
View File
@@ -4,17 +4,6 @@ from __future__ import annotations
import time
import pytest
from turnstone.core.storage._sqlite import SQLiteBackend
@pytest.fixture
def db(tmp_path):
"""Fresh SQLite backend for each test."""
backend = SQLiteBackend(str(tmp_path / "test.db"))
return backend
def _make_task_kwargs(**overrides):
"""Build default kwargs for create_scheduled_task."""
+440
View File
@@ -0,0 +1,440 @@
"""Integration tests for SDK governance methods against a real Starlette app.
Verifies round-trip serialization: SDK -> HTTP -> Starlette handler -> storage
-> JSON response -> Pydantic model validation in the SDK client.
"""
from __future__ import annotations
from typing import TYPE_CHECKING, Any
import httpx
import pytest
from starlette.applications import Starlette
from starlette.middleware import Middleware
from starlette.middleware.base import BaseHTTPMiddleware
from starlette.routing import Mount, Route
if TYPE_CHECKING:
from starlette.requests import Request
from starlette.responses import Response
from turnstone.api.console_schemas import (
ListOrgsResponse,
ListRolesResponse,
ListToolPoliciesResponse,
OrgInfo,
RoleInfo,
ToolPolicyInfo,
)
from turnstone.api.schemas import StatusResponse
from turnstone.console.server import (
admin_create_policy,
admin_create_role,
admin_delete_policy,
admin_delete_role,
admin_get_org,
admin_list_orgs,
admin_list_policies,
admin_list_roles,
admin_update_org,
admin_update_policy,
admin_update_role,
)
from turnstone.core.auth import AuthResult
from turnstone.core.storage._sqlite import SQLiteBackend
from turnstone.sdk.console import AsyncTurnstoneConsole
# ---------------------------------------------------------------------------
# Auth bypass middleware — injects a full-access AuthResult on every request.
# ---------------------------------------------------------------------------
class _InjectAuthMiddleware(BaseHTTPMiddleware):
async def dispatch(self, request: Request, call_next: Any) -> Response:
request.state.auth_result = AuthResult(
user_id="test-admin",
scopes=frozenset({"approve"}),
token_source="config",
permissions=frozenset(
{
"read",
"write",
"approve",
"admin.roles",
"admin.orgs",
"admin.policies",
}
),
)
resp: Response = await call_next(request)
return resp
# ---------------------------------------------------------------------------
# Fixtures
# ---------------------------------------------------------------------------
def _make_app() -> Starlette:
return Starlette(
routes=[
Mount(
"/v1",
routes=[
# Roles
Route("/api/admin/roles", admin_list_roles),
Route("/api/admin/roles", admin_create_role, methods=["POST"]),
Route("/api/admin/roles/{role_id}", admin_update_role, methods=["PUT"]),
Route("/api/admin/roles/{role_id}", admin_delete_role, methods=["DELETE"]),
# Orgs
Route("/api/admin/orgs", admin_list_orgs),
Route("/api/admin/orgs/{org_id}", admin_get_org),
Route("/api/admin/orgs/{org_id}", admin_update_org, methods=["PUT"]),
# Policies
Route("/api/admin/policies", admin_list_policies),
Route("/api/admin/policies", admin_create_policy, methods=["POST"]),
Route(
"/api/admin/policies/{policy_id}",
admin_update_policy,
methods=["PUT"],
),
Route(
"/api/admin/policies/{policy_id}",
admin_delete_policy,
methods=["DELETE"],
),
],
),
],
middleware=[Middleware(_InjectAuthMiddleware)],
)
@pytest.fixture
def storage(tmp_path: Any) -> SQLiteBackend:
return SQLiteBackend(str(tmp_path / "test.db"))
@pytest.fixture
async def sdk_client(storage: SQLiteBackend):
"""SDK client wired to a real Starlette app via ASGITransport."""
app = _make_app()
app.state.auth_storage = storage
transport = httpx.ASGITransport(app=app)
async with httpx.AsyncClient(transport=transport, base_url="http://testserver") as hc:
yield AsyncTurnstoneConsole(httpx_client=hc)
# ---------------------------------------------------------------------------
# Tests — Roles round-trip
# ---------------------------------------------------------------------------
class TestRolesRoundTrip:
@pytest.mark.anyio
async def test_list_roles_empty(self, sdk_client: AsyncTurnstoneConsole) -> None:
resp = await sdk_client.list_roles()
assert isinstance(resp, ListRolesResponse)
assert resp.roles == []
@pytest.mark.anyio
async def test_create_and_list_role(self, sdk_client: AsyncTurnstoneConsole) -> None:
role = await sdk_client.create_role(
"analyst", display_name="Data Analyst", permissions="read,write"
)
assert isinstance(role, RoleInfo)
assert role.name == "analyst"
assert role.display_name == "Data Analyst"
assert role.permissions == "read,write"
assert role.builtin is False
assert role.role_id # non-empty
# List should now contain the new role
resp = await sdk_client.list_roles()
assert len(resp.roles) == 1
assert resp.roles[0].role_id == role.role_id
assert resp.roles[0].name == "analyst"
@pytest.mark.anyio
async def test_create_update_role(self, sdk_client: AsyncTurnstoneConsole) -> None:
role = await sdk_client.create_role("ops", permissions="read")
assert role.permissions == "read"
updated = await sdk_client.update_role(
role.role_id, display_name="Operations", permissions="read,write,approve"
)
assert isinstance(updated, RoleInfo)
assert updated.display_name == "Operations"
assert updated.permissions == "read,write,approve"
assert updated.role_id == role.role_id
@pytest.mark.anyio
async def test_create_delete_role(self, sdk_client: AsyncTurnstoneConsole) -> None:
role = await sdk_client.create_role("temp-role", permissions="read")
result = await sdk_client.delete_role(role.role_id)
assert isinstance(result, StatusResponse)
assert result.status == "ok"
# Verify gone
resp = await sdk_client.list_roles()
assert resp.roles == []
@pytest.mark.anyio
async def test_full_lifecycle(self, sdk_client: AsyncTurnstoneConsole) -> None:
"""Create -> list -> update -> list -> delete -> list."""
# Create
role = await sdk_client.create_role(
"lifecycle", display_name="Lifecycle", permissions="read"
)
role_id = role.role_id
# List confirms creation
roles = (await sdk_client.list_roles()).roles
assert len(roles) == 1
assert roles[0].role_id == role_id
# Update
updated = await sdk_client.update_role(role_id, permissions="read,write")
assert updated.permissions == "read,write"
# List still has one
roles = (await sdk_client.list_roles()).roles
assert len(roles) == 1
assert roles[0].permissions == "read,write"
# Delete
await sdk_client.delete_role(role_id)
# List is empty
roles = (await sdk_client.list_roles()).roles
assert roles == []
@pytest.mark.anyio
async def test_delete_nonexistent_role_raises(self, sdk_client: AsyncTurnstoneConsole) -> None:
from turnstone.sdk._types import TurnstoneAPIError
with pytest.raises(TurnstoneAPIError) as exc_info:
await sdk_client.delete_role("nonexistent")
assert exc_info.value.status_code == 404
@pytest.mark.anyio
async def test_update_nonexistent_role_raises(self, sdk_client: AsyncTurnstoneConsole) -> None:
from turnstone.sdk._types import TurnstoneAPIError
with pytest.raises(TurnstoneAPIError) as exc_info:
await sdk_client.update_role("nonexistent", display_name="Nope")
assert exc_info.value.status_code == 404
# ---------------------------------------------------------------------------
# Tests — Policies round-trip
# ---------------------------------------------------------------------------
class TestPoliciesRoundTrip:
@pytest.mark.anyio
async def test_list_policies_empty(self, sdk_client: AsyncTurnstoneConsole) -> None:
resp = await sdk_client.list_policies()
assert isinstance(resp, ListToolPoliciesResponse)
assert resp.policies == []
@pytest.mark.anyio
async def test_create_and_list_policy(self, sdk_client: AsyncTurnstoneConsole) -> None:
policy = await sdk_client.create_policy("Allow bash", "bash_*", "allow", priority=10)
assert isinstance(policy, ToolPolicyInfo)
assert policy.name == "Allow bash"
assert policy.tool_pattern == "bash_*"
assert policy.action == "allow"
assert policy.priority == 10
assert policy.enabled is True
assert policy.policy_id # non-empty
resp = await sdk_client.list_policies()
assert len(resp.policies) == 1
assert resp.policies[0].policy_id == policy.policy_id
@pytest.mark.anyio
async def test_create_update_policy(self, sdk_client: AsyncTurnstoneConsole) -> None:
policy = await sdk_client.create_policy("Deny write", "write_*", "deny", priority=5)
updated = await sdk_client.update_policy(
policy.policy_id, name="Allow write", action="allow", priority=20
)
assert isinstance(updated, ToolPolicyInfo)
assert updated.name == "Allow write"
assert updated.action == "allow"
assert updated.priority == 20
assert updated.policy_id == policy.policy_id
@pytest.mark.anyio
async def test_create_delete_policy(self, sdk_client: AsyncTurnstoneConsole) -> None:
policy = await sdk_client.create_policy("Temp policy", "temp_*", "ask")
result = await sdk_client.delete_policy(policy.policy_id)
assert isinstance(result, StatusResponse)
assert result.status == "ok"
resp = await sdk_client.list_policies()
assert resp.policies == []
@pytest.mark.anyio
async def test_full_lifecycle(self, sdk_client: AsyncTurnstoneConsole) -> None:
"""Create -> list -> update -> list -> delete -> list."""
policy = await sdk_client.create_policy("Lifecycle", "test_*", "deny", priority=1)
pid = policy.policy_id
policies = (await sdk_client.list_policies()).policies
assert len(policies) == 1
await sdk_client.update_policy(pid, action="allow", priority=99)
policies = (await sdk_client.list_policies()).policies
assert policies[0].action == "allow"
assert policies[0].priority == 99
await sdk_client.delete_policy(pid)
policies = (await sdk_client.list_policies()).policies
assert policies == []
@pytest.mark.anyio
async def test_delete_nonexistent_policy_raises(
self, sdk_client: AsyncTurnstoneConsole
) -> None:
from turnstone.sdk._types import TurnstoneAPIError
with pytest.raises(TurnstoneAPIError) as exc_info:
await sdk_client.delete_policy("nonexistent")
assert exc_info.value.status_code == 404
@pytest.mark.anyio
async def test_update_nonexistent_policy_raises(
self, sdk_client: AsyncTurnstoneConsole
) -> None:
from turnstone.sdk._types import TurnstoneAPIError
with pytest.raises(TurnstoneAPIError) as exc_info:
await sdk_client.update_policy("nonexistent", name="Nope")
assert exc_info.value.status_code == 404
@pytest.mark.anyio
async def test_create_policy_invalid_action_raises(
self, sdk_client: AsyncTurnstoneConsole
) -> None:
from turnstone.sdk._types import TurnstoneAPIError
with pytest.raises(TurnstoneAPIError) as exc_info:
await sdk_client.create_policy("Bad", "tool_*", "yolo")
assert exc_info.value.status_code == 400
# ---------------------------------------------------------------------------
# Tests — Orgs round-trip
# ---------------------------------------------------------------------------
class TestOrgsRoundTrip:
@pytest.mark.anyio
async def test_list_orgs_empty(self, sdk_client: AsyncTurnstoneConsole) -> None:
resp = await sdk_client.list_orgs()
assert isinstance(resp, ListOrgsResponse)
assert resp.orgs == []
@pytest.mark.anyio
async def test_get_org(self, sdk_client: AsyncTurnstoneConsole, storage: SQLiteBackend) -> None:
storage.create_org(
org_id="org-1", name="acme", display_name="Acme Corp", settings='{"k": "v"}'
)
org = await sdk_client.get_org("org-1")
assert isinstance(org, OrgInfo)
assert org.org_id == "org-1"
assert org.name == "acme"
assert org.display_name == "Acme Corp"
assert org.settings == '{"k": "v"}'
@pytest.mark.anyio
async def test_list_orgs_after_seed(
self, sdk_client: AsyncTurnstoneConsole, storage: SQLiteBackend
) -> None:
storage.create_org(org_id="org-a", name="alpha", display_name="Alpha")
storage.create_org(org_id="org-b", name="beta", display_name="Beta")
resp = await sdk_client.list_orgs()
assert len(resp.orgs) == 2
names = {o.name for o in resp.orgs}
assert names == {"alpha", "beta"}
@pytest.mark.anyio
async def test_update_org(
self, sdk_client: AsyncTurnstoneConsole, storage: SQLiteBackend
) -> None:
storage.create_org(org_id="org-1", name="acme", display_name="Acme Corp")
updated = await sdk_client.update_org("org-1", display_name="Acme Inc.")
assert isinstance(updated, OrgInfo)
assert updated.display_name == "Acme Inc."
assert updated.org_id == "org-1"
@pytest.mark.anyio
async def test_get_nonexistent_org_raises(self, sdk_client: AsyncTurnstoneConsole) -> None:
from turnstone.sdk._types import TurnstoneAPIError
with pytest.raises(TurnstoneAPIError) as exc_info:
await sdk_client.get_org("nonexistent")
assert exc_info.value.status_code == 404
@pytest.mark.anyio
async def test_update_nonexistent_org_raises(self, sdk_client: AsyncTurnstoneConsole) -> None:
from turnstone.sdk._types import TurnstoneAPIError
with pytest.raises(TurnstoneAPIError) as exc_info:
await sdk_client.update_org("nonexistent", display_name="Nope")
assert exc_info.value.status_code == 404
# ---------------------------------------------------------------------------
# Tests — Pydantic model field validation
# ---------------------------------------------------------------------------
class TestModelValidation:
"""Verify that all expected fields are populated and correctly typed."""
@pytest.mark.anyio
async def test_role_info_fields(self, sdk_client: AsyncTurnstoneConsole) -> None:
role = await sdk_client.create_role("reviewer", permissions="read")
assert isinstance(role.role_id, str)
assert isinstance(role.name, str)
assert isinstance(role.display_name, str)
assert isinstance(role.permissions, str)
assert isinstance(role.builtin, bool)
assert isinstance(role.org_id, str)
assert isinstance(role.created, str)
assert isinstance(role.updated, str)
@pytest.mark.anyio
async def test_policy_info_fields(self, sdk_client: AsyncTurnstoneConsole) -> None:
policy = await sdk_client.create_policy("Test", "read_*", "allow", priority=5)
assert isinstance(policy.policy_id, str)
assert isinstance(policy.name, str)
assert isinstance(policy.tool_pattern, str)
assert isinstance(policy.action, str)
assert isinstance(policy.priority, int)
assert isinstance(policy.org_id, str)
assert isinstance(policy.enabled, bool)
assert isinstance(policy.created_by, str)
assert isinstance(policy.created, str)
assert isinstance(policy.updated, str)
@pytest.mark.anyio
async def test_org_info_fields(
self, sdk_client: AsyncTurnstoneConsole, storage: SQLiteBackend
) -> None:
storage.create_org(org_id="org-v", name="validate", display_name="Validate")
org = await sdk_client.get_org("org-v")
assert isinstance(org.org_id, str)
assert isinstance(org.name, str)
assert isinstance(org.display_name, str)
assert isinstance(org.settings, str)
assert isinstance(org.created, str)
assert isinstance(org.updated, str)
+3 -1
View File
@@ -88,7 +88,7 @@ class RecordingUI:
def approve_tools(self, items):
return True, None # auto-approve everything
def on_tool_result(self, call_id, name, output):
def on_tool_result(self, call_id, name, output, **kwargs):
self.tool_results.append((call_id, name, output))
def on_tool_output_chunk(self, call_id, chunk):
@@ -623,6 +623,7 @@ class TestServerHealthMetrics:
mock_mgr = MagicMock()
mock_mgr.list_all.return_value = [mock_ws]
mock_mgr.max_workstreams = 10
app = srv_mod.create_app(
workstreams=mock_mgr,
@@ -799,6 +800,7 @@ class TestServerRateLimiting:
mock_mgr = MagicMock()
mock_mgr.list_all.return_value = [mock_ws]
mock_mgr.max_workstreams = 10
app = srv_mod.create_app(
workstreams=mock_mgr,
-9
View File
@@ -2,15 +2,6 @@
from __future__ import annotations
import pytest
from turnstone.core.storage._sqlite import SQLiteBackend
@pytest.fixture
def storage(tmp_path):
return SQLiteBackend(str(tmp_path / "test.db"))
class TestServiceRegistry:
def test_register_and_list(self, storage):
+270 -5
View File
@@ -28,7 +28,7 @@ class NullUI:
def approve_tools(self, items):
return True, None
def on_tool_result(self, call_id, name, output):
def on_tool_result(self, call_id, name, output, **kwargs):
pass
def on_tool_output_chunk(self, call_id, chunk):
@@ -214,7 +214,7 @@ class TestPlanExec:
"id": tc_id,
"type": "function",
"function": {
"name": "create_plan",
"name": "plan_agent",
"arguments": json.dumps({"goal": prior_prompt}),
},
}
@@ -250,7 +250,7 @@ class TestPlanExec:
m for m in messages if m["role"] == "assistant" and m.get("tool_calls")
]
assert len(assistant_with_tc) == 1
assert assistant_with_tc[0]["tool_calls"][0]["function"]["name"] == "create_plan"
assert assistant_with_tc[0]["tool_calls"][0]["function"]["name"] == "plan_agent"
# The real tool result is forwarded with its original content
tool_msgs = [m for m in messages if m["role"] == "tool"]
@@ -463,7 +463,7 @@ class TestPlanRefinement:
with patch.object(session, "_refine_plan", side_effect=fake_refine):
items = [
{
"func_name": "create_plan",
"func_name": "plan_agent",
"call_id": "c1",
"prompt": "add auth",
}
@@ -576,7 +576,7 @@ class TestPlanRefinement:
msgs = captured["messages"]
assert msgs[0]["role"] == "system"
assert msgs[1]["role"] == "assistant"
assert msgs[1]["tool_calls"][0]["function"]["name"] == "create_plan"
assert msgs[1]["tool_calls"][0]["function"]["name"] == "plan_agent"
assert msgs[2]["role"] == "tool"
assert msgs[2]["content"] == self.GOOD_PLAN
assert msgs[3]["role"] == "user"
@@ -753,3 +753,268 @@ class TestGetCapabilitiesOverride:
caps = session._get_capabilities()
# Default OpenAI provider for unknown model → no vision
assert caps.supports_vision is False
class TestTitleRetry:
"""_generate_title resets _title_generated on failure."""
def test_title_generated_reset_on_failure(self, tmp_db):
session = _make_session()
session._title_generated = True
session.messages = [
{"role": "user", "content": "Hello"},
{"role": "assistant", "content": "Hi there"},
]
# Mock provider to raise
session._provider = MagicMock()
session._provider.create_completion.side_effect = RuntimeError("API error")
session._generate_title()
assert session._title_generated is False
def test_title_generated_stays_true_on_success(self, tmp_db):
session = _make_session()
session._title_generated = True
session.messages = [
{"role": "user", "content": "Hello"},
{"role": "assistant", "content": "Hi there"},
]
result = MagicMock()
result.content = "Test Title"
session._provider = MagicMock()
session._provider.create_completion.return_value = result
with patch("turnstone.core.session.update_workstream_title"):
session._generate_title()
# Flag stays True after successful generation
assert session._title_generated is True
def test_title_skipped_after_resume_changes_ws_id(self, tmp_db):
"""If ws_id changes (via resume) during title generation, discard the result."""
session = _make_session()
session._title_generated = True
session.messages = [
{"role": "user", "content": "Hello"},
{"role": "assistant", "content": "Hi there"},
]
original_ws_id = session._ws_id
result = MagicMock()
result.content = "Test Title"
session._provider = MagicMock()
session._provider.create_completion.return_value = result
# Simulate resume() changing ws_id while title generation is in flight
def _change_ws_id(*args, **kwargs):
session._ws_id = "different-ws-id"
return result
session._provider.create_completion.side_effect = _change_ws_id
with patch("turnstone.core.session.update_workstream_title") as mock_update:
session._generate_title()
# Title should NOT be applied to the new workstream
mock_update.assert_not_called()
# Restore for cleanup
session._ws_id = original_ws_id
class TestLiveConfigUpdate:
"""ConfigStore-backed sessions pick up settings changes at point-of-use."""
def test_memory_config_reads_from_config_store(self, tmp_db):
"""_mem_cfg returns live values from ConfigStore when present."""
from turnstone.core.config_store import ConfigStore
from turnstone.core.storage._sqlite import SQLiteBackend
storage = SQLiteBackend(str(tmp_db), create_tables=True)
cs = ConfigStore(storage)
session = _make_session(config_store=cs)
# Default: relevance_k=5
assert session._mem_cfg.relevance_k == 5
# Admin changes the setting
cs.set("memory.relevance_k", 10, changed_by="test")
assert session._mem_cfg.relevance_k == 10
def test_judge_config_reads_from_config_store(self, tmp_db):
"""_judge_cfg returns live behavioral flags from ConfigStore."""
from turnstone.core.config_store import ConfigStore
from turnstone.core.judge import JudgeConfig
from turnstone.core.storage._sqlite import SQLiteBackend
storage = SQLiteBackend(str(tmp_db), create_tables=True)
cs = ConfigStore(storage)
session = _make_session(
judge_config=JudgeConfig(),
config_store=cs,
)
# Default: enabled=True
assert session._judge_cfg.enabled is True
# Admin disables the judge
cs.set("judge.enabled", False, changed_by="test")
assert session._judge_cfg.enabled is False
def test_judge_client_config_stays_frozen(self, tmp_db):
"""LLM client fields (model, provider) are frozen from creation time."""
from turnstone.core.config_store import ConfigStore
from turnstone.core.judge import JudgeConfig
from turnstone.core.storage._sqlite import SQLiteBackend
storage = SQLiteBackend(str(tmp_db), create_tables=True)
cs = ConfigStore(storage)
session = _make_session(
judge_config=JudgeConfig(model="original-model"),
config_store=cs,
)
# Change the model in ConfigStore — should NOT affect the session
cs.set("judge.model", "new-model", changed_by="test")
assert session._judge_cfg.model == "original-model"
def test_judge_disable_after_init_stops_future_use(self, tmp_db):
"""Disabling judge.enabled after IntentJudge is created returns None."""
from turnstone.core.config_store import ConfigStore
from turnstone.core.judge import JudgeConfig
from turnstone.core.storage._sqlite import SQLiteBackend
storage = SQLiteBackend(str(tmp_db), create_tables=True)
cs = ConfigStore(storage)
session = _make_session(
judge_config=JudgeConfig(),
config_store=cs,
)
# Force judge initialization by setting a mock
session._judge = MagicMock()
assert session._ensure_judge() is not None
# Admin disables the judge — cached instance should NOT be returned
cs.set("judge.enabled", False, changed_by="test")
assert session._ensure_judge() is None
def test_fallback_to_frozen_without_config_store(self, tmp_db):
"""Without ConfigStore (CLI mode), frozen config is used."""
from turnstone.core.memory_relevance import MemoryConfig
session = _make_session(memory_config=MemoryConfig(relevance_k=3))
assert session._mem_cfg.relevance_k == 3
class TestAgentOutputGuard:
"""Output guard should evaluate tool results in _run_agent, not just the main loop."""
def test_agent_loop_calls_evaluate_output(self):
"""_run_agent passes tool output through _evaluate_output when output_guard is enabled."""
from turnstone.core.judge import JudgeConfig
session = _make_session(judge_config=JudgeConfig(output_guard=True))
with patch.object(session, "_evaluate_output", wraps=lambda cid, o, fn: o) as mock_eval:
# Simulate _run_agent getting a tool call response then a text response
call_count = [0]
def fake_create(**kwargs):
call_count[0] += 1
resp = MagicMock()
if call_count[0] == 1:
# First call: model returns a tool call
choice = MagicMock()
choice.finish_reason = "tool_calls"
tc = MagicMock()
tc.id = "call_1"
tc.function.name = "read_file"
tc.function.arguments = '{"path": "/tmp/test"}'
choice.message.tool_calls = [tc]
choice.message.content = None
resp.choices = [choice]
resp.usage = MagicMock(prompt_tokens=10, completion_tokens=5)
else:
# Second call: model returns text (done)
choice = MagicMock()
choice.finish_reason = "stop"
choice.message.tool_calls = None
choice.message.content = "Done"
resp.choices = [choice]
resp.usage = MagicMock(prompt_tokens=10, completion_tokens=5)
return resp
session.client.chat.completions.create = fake_create
# Mock tool preparation to return a simple output
def fake_prepare(tc_dict, **kwargs):
return {
"call_id": tc_dict["id"],
"func_name": "read_file",
"needs_approval": False,
"execute": lambda p: ("call_1", "file contents with sk-proj-SECRET123"),
}
with patch.object(session, "_prepare_tool", side_effect=fake_prepare):
session._run_agent(
[{"role": "user", "content": "test"}],
tools=[{"type": "function", "function": {"name": "read_file"}}],
label="test",
)
mock_eval.assert_called_once()
args = mock_eval.call_args[0]
assert args[0] == "call_1" # call_id
assert "sk-proj-SECRET123" in args[1] # output
assert args[2] == "read_file" # func_name
def test_agent_loop_skips_guard_when_disabled(self):
"""_run_agent does not call _evaluate_output when output_guard is disabled."""
from turnstone.core.judge import JudgeConfig
session = _make_session(judge_config=JudgeConfig(output_guard=False))
with patch.object(session, "_evaluate_output") as mock_eval:
call_count = [0]
def fake_create(**kwargs):
call_count[0] += 1
resp = MagicMock()
if call_count[0] == 1:
choice = MagicMock()
choice.finish_reason = "tool_calls"
tc = MagicMock()
tc.id = "call_1"
tc.function.name = "read_file"
tc.function.arguments = '{"path": "/tmp/test"}'
choice.message.tool_calls = [tc]
choice.message.content = None
resp.choices = [choice]
resp.usage = MagicMock(prompt_tokens=10, completion_tokens=5)
else:
choice = MagicMock()
choice.finish_reason = "stop"
choice.message.tool_calls = None
choice.message.content = "Done"
resp.choices = [choice]
resp.usage = MagicMock(prompt_tokens=10, completion_tokens=5)
return resp
session.client.chat.completions.create = fake_create
def fake_prepare(tc_dict, **kwargs):
return {
"call_id": tc_dict["id"],
"func_name": "read_file",
"needs_approval": False,
"execute": lambda p: ("call_1", "safe output"),
}
with patch.object(session, "_prepare_tool", side_effect=fake_prepare):
session._run_agent(
[{"role": "user", "content": "test"}],
tools=[{"type": "function", "function": {"name": "read_file"}}],
label="test",
)
mock_eval.assert_not_called()
+326
View File
@@ -525,6 +525,37 @@ class TestWorkstreamConfig:
assert session.instructions == "be concise"
assert session.creative_mode is True
def test_resume_restores_model(self, tmp_db):
"""ChatSession.resume() should restore the model from workstream config."""
client = MagicMock()
client.models.list.return_value.data = [MagicMock(id="test-model")]
ui = MagicMock()
ui.on_info = MagicMock()
ui.on_error = MagicMock()
ui.on_state_change = MagicMock()
ui.on_rename = MagicMock()
# Create a workstream that was using a specific model
register_workstream("model_ws")
save_message("model_ws", "user", "hello")
save_message("model_ws", "assistant", "hi")
save_workstream_config("model_ws", {"model": "gpt-5", "model_alias": ""})
# Resume into a session that was created with a different model
session = ChatSession(
client=client,
model="gpt-5-nano",
ui=ui,
instructions=None,
temperature=0.5,
max_tokens=1000,
tool_timeout=10,
)
assert session.model == "gpt-5-nano"
result = session.resume("model_ws")
assert result is True
assert session.model == "gpt-5"
# ── Prune workstreams ─────────────────────────────────────────────────
@@ -607,3 +638,298 @@ class TestPruneWorkstreams:
# Config rows should be cleaned up
assert load_workstream_config("orphan_cfg") == {}
assert load_workstream_config("stale_cfg") == {}
# ── Parallel tool exception isolation ────────────────────────────────
class TestParallelToolExceptionIsolation:
"""Bug #117: one tool raising should not kill the entire batch."""
def test_exception_in_one_tool_does_not_kill_batch(self, tmp_db, mock_openai_client):
from unittest.mock import patch
session = ChatSession(
client=mock_openai_client,
model="test-model",
ui=MagicMock(),
instructions=None,
temperature=0.5,
max_tokens=1000,
tool_timeout=10,
)
def succeed(item):
return item["call_id"], "ok"
def fail(item):
raise RuntimeError("boom")
items = [
{
"call_id": "c1",
"func_name": "bash",
"execute": succeed,
"needs_approval": False,
"header": "test",
"preview": "",
},
{
"call_id": "c2",
"func_name": "math",
"execute": fail,
"needs_approval": False,
"header": "test",
"preview": "",
},
]
tool_calls = [
{"id": "c1", "function": {"name": "bash", "arguments": "{}"}},
{"id": "c2", "function": {"name": "math", "arguments": "{}"}},
]
with (
patch.object(session, "_prepare_tool", side_effect=items),
patch.object(session, "_evaluate_intent"),
patch.object(session, "_emit_state"),
patch.object(session, "_init_system_messages"),
patch.object(session, "_check_cancelled"),
):
session.ui.approve_tools.return_value = (True, None)
results, _ = session._execute_tools(tool_calls)
assert results[0] == ("c1", "ok")
assert results[1][0] == "c2"
assert "Error executing math" in results[1][1]
assert "boom" in results[1][1]
# ── Web search tool gating ───────────────────────────────────────────
class TestWebSearchGating:
"""Bug #117: web_search should not be offered without a backend."""
def test_web_search_filtered_when_no_backend(self, tmp_db, mock_openai_client):
from unittest.mock import patch
from turnstone.core.providers._protocol import ModelCapabilities
session = ChatSession(
client=mock_openai_client,
model="local-model",
ui=MagicMock(),
instructions=None,
temperature=0.5,
max_tokens=1000,
tool_timeout=10,
)
caps = ModelCapabilities(supports_web_search=False)
with (
patch.object(session, "_get_capabilities", return_value=caps),
patch("turnstone.core.session.get_tavily_key", return_value=None),
patch("turnstone.core.web_search._ddg_available", return_value=False),
):
tools = session._get_active_tools()
names = [t.get("function", {}).get("name") for t in tools]
assert "web_search" not in names
def test_web_search_kept_when_tavily_available(self, tmp_db, mock_openai_client):
from unittest.mock import patch
from turnstone.core.providers._protocol import ModelCapabilities
session = ChatSession(
client=mock_openai_client,
model="local-model",
ui=MagicMock(),
instructions=None,
temperature=0.5,
max_tokens=1000,
tool_timeout=10,
)
caps = ModelCapabilities(supports_web_search=False)
with (
patch.object(session, "_get_capabilities", return_value=caps),
patch("turnstone.core.session.get_tavily_key", return_value="tvly-test-key"),
):
tools = session._get_active_tools()
names = [t.get("function", {}).get("name") for t in tools]
assert "web_search" in names
def test_web_search_kept_when_native_support(self, tmp_db, mock_openai_client):
from unittest.mock import patch
from turnstone.core.providers._protocol import ModelCapabilities
session = ChatSession(
client=mock_openai_client,
model="gpt-5-search-api",
ui=MagicMock(),
instructions=None,
temperature=0.5,
max_tokens=1000,
tool_timeout=10,
)
caps = ModelCapabilities(supports_web_search=True)
with (
patch.object(session, "_get_capabilities", return_value=caps),
patch("turnstone.core.session.get_tavily_key", return_value=None),
):
tools = session._get_active_tools()
names = [t.get("function", {}).get("name") for t in tools]
assert "web_search" in names
class TestMCPToolGating:
"""MCP tools should not be offered when no MCP servers provide them."""
def test_mcp_tools_filtered_without_mcp_client(self, tmp_db, mock_openai_client):
"""read_resource and use_prompt excluded when no MCP client."""
session = ChatSession(
client=mock_openai_client,
model="local-model",
ui=MagicMock(),
instructions=None,
temperature=0.5,
max_tokens=1000,
tool_timeout=10,
)
assert session._mcp_client is None
tools = session._get_active_tools()
names = [t.get("function", {}).get("name") for t in tools]
assert "read_resource" not in names
assert "use_prompt" not in names
def test_read_resource_filtered_when_no_resources(self, tmp_db, mock_openai_client):
"""read_resource excluded when MCP client has no resources."""
mcp_client = MagicMock()
mcp_client.get_tools.return_value = []
mcp_client.resource_count = 0
mcp_client.prompt_count = 2
session = ChatSession(
client=mock_openai_client,
model="local-model",
ui=MagicMock(),
instructions=None,
temperature=0.5,
max_tokens=1000,
tool_timeout=10,
mcp_client=mcp_client,
)
tools = session._get_active_tools()
names = [t.get("function", {}).get("name") for t in tools]
assert "read_resource" not in names
assert "use_prompt" in names
def test_use_prompt_filtered_when_no_prompts(self, tmp_db, mock_openai_client):
"""use_prompt excluded when MCP client has no prompts."""
mcp_client = MagicMock()
mcp_client.get_tools.return_value = []
mcp_client.resource_count = 3
mcp_client.prompt_count = 0
session = ChatSession(
client=mock_openai_client,
model="local-model",
ui=MagicMock(),
instructions=None,
temperature=0.5,
max_tokens=1000,
tool_timeout=10,
mcp_client=mcp_client,
)
tools = session._get_active_tools()
names = [t.get("function", {}).get("name") for t in tools]
assert "use_prompt" not in names
assert "read_resource" in names
def test_mcp_tools_kept_when_servers_have_both(self, tmp_db, mock_openai_client):
"""Both tools present when MCP client has resources and prompts."""
mcp_client = MagicMock()
mcp_client.get_tools.return_value = []
mcp_client.resource_count = 1
mcp_client.prompt_count = 1
session = ChatSession(
client=mock_openai_client,
model="local-model",
ui=MagicMock(),
instructions=None,
temperature=0.5,
max_tokens=1000,
tool_timeout=10,
mcp_client=mcp_client,
)
tools = session._get_active_tools()
names = [t.get("function", {}).get("name") for t in tools]
assert "read_resource" in names
assert "use_prompt" in names
def test_mcp_tools_filtered_with_tool_search_active(self, tmp_db, mock_openai_client):
"""Gating applies even when tool_search is active (client-side path)."""
mcp_client = MagicMock()
mcp_client.get_tools.return_value = []
mcp_client.resource_count = 0
mcp_client.prompt_count = 0
session = ChatSession(
client=mock_openai_client,
model="local-model",
ui=MagicMock(),
instructions=None,
temperature=0.5,
max_tokens=1000,
tool_timeout=10,
mcp_client=mcp_client,
tool_search="on",
)
assert session._tool_search is not None
tools = session._get_active_tools()
names = [t.get("function", {}).get("name") for t in tools]
assert "read_resource" not in names
assert "use_prompt" not in names
def test_mcp_tools_filtered_with_native_tool_search(self, tmp_db, mock_openai_client):
"""Gating applies when provider handles tool search natively."""
from unittest.mock import patch
from turnstone.core.providers._protocol import ModelCapabilities
mcp_client = MagicMock()
mcp_client.get_tools.return_value = []
mcp_client.resource_count = 0
mcp_client.prompt_count = 0
session = ChatSession(
client=mock_openai_client,
model="local-model",
ui=MagicMock(),
instructions=None,
temperature=0.5,
max_tokens=1000,
tool_timeout=10,
mcp_client=mcp_client,
tool_search="on",
)
caps = ModelCapabilities(supports_tool_search=True)
with patch.object(session, "_get_capabilities", return_value=caps):
tools = session._get_active_tools()
names = [t.get("function", {}).get("name") for t in tools]
assert "read_resource" not in names
assert "use_prompt" not in names
+37 -8
View File
@@ -179,7 +179,10 @@ class TestDeleteSetting:
# Delete it
r = client.delete("/v1/api/admin/settings/tools.timeout")
assert r.status_code == 200
assert r.json()["status"] == "ok"
body = r.json()
assert body["status"] == "ok"
assert body["key"] == "tools.timeout"
assert body["default"] == 120 # registry default for tools.timeout
def test_delete_then_list_shows_default(self, client):
client.put(
@@ -251,20 +254,46 @@ class TestSecretMasking:
by_key = {s["key"]: s for s in r.json()["settings"]}
assert by_key["judge.api_key"]["value"] == "***"
def test_secret_write_blocked(self, client):
"""Secret settings cannot be modified via API."""
def test_secret_writable_via_api(self, client):
"""Secret settings can be written via API (write-only pattern)."""
r = client.put(
"/v1/api/admin/settings/judge.api_key",
json={"value": "sk-secret-123"},
)
assert r.status_code == 403
assert "config.toml" in r.json()["error"]
assert r.status_code == 200
# Response value is masked even for the write confirmation
assert r.json()["value"] == "***"
def test_secret_shows_managed_label(self, client):
"""Secret settings show a label instead of a value."""
def test_secret_sentinel_preserves_existing(self, client):
"""Submitting '***' for a secret setting is a no-op (preserve existing)."""
# First write a real value
r1 = client.put(
"/v1/api/admin/settings/judge.api_key",
json={"value": "sk-real-key"},
)
assert r1.status_code == 200
# Now submit the sentinel — should return unchanged with full response shape
r2 = client.put(
"/v1/api/admin/settings/judge.api_key",
json={"value": "***"},
)
assert r2.status_code == 200
data = r2.json()
assert data.get("unchanged") is True
assert data["key"] == "judge.api_key"
assert data["value"] == "***"
assert data["type"] == "str"
assert data["is_secret"] is True
def test_secret_still_masked_in_list(self, client):
"""After writing a secret, list still shows '***'."""
client.put(
"/v1/api/admin/settings/judge.api_key",
json={"value": "sk-written-via-api"},
)
r = client.get("/v1/api/admin/settings")
by_key = {s["key"]: s for s in r.json()["settings"]}
assert "managed via" in by_key["judge.api_key"]["value"]
assert by_key["judge.api_key"]["value"] == "***"
# ---------------------------------------------------------------------------
+3 -3
View File
@@ -74,7 +74,7 @@ class TestSimEngine:
def test_llm_response_returns_content(self, engine):
async def _test():
content, tool_calls = await engine.simulate_llm_response(True, 1)
content, tool_calls = await engine.simulate_llm_response(True)
assert isinstance(content, str)
assert len(content) > 0
assert isinstance(tool_calls, list)
@@ -85,8 +85,8 @@ class TestSimEngine:
async def _test():
e1 = SimEngine(fast_config, rng=random.Random(123))
e2 = SimEngine(fast_config, rng=random.Random(123))
c1, t1 = await e1.simulate_llm_response(True, 1)
c2, t2 = await e2.simulate_llm_response(True, 1)
c1, t1 = await e1.simulate_llm_response(True)
c2, t2 = await e2.simulate_llm_response(True)
assert c1 == c2
assert len(t1) == len(t2)

Some files were not shown because too many files have changed in this diff Show More