Compare commits

...

67 Commits

Author SHA1 Message Date
Patrick Buckley bb221f4dab chore: bump v0.8.8, update vendored katex 0.16.40 → 0.16.42 2026-03-25 12:00:26 -07:00
Patrick Buckley 361876b17a feat: eval pipeline improvements + tool description optimization (#174)
Eval pipeline:
- --optimize-tools mode freezes system prompt, optimizes tool descriptions only
- Analyst sees available tool list (prevents hallucinated "tool not in schema")
- Analyst sees current tool descriptions in --optimize-tools mode
- Three-layer timeout defense: httpx timeout + _cancelled event + client.close()
- Filter MCP-only tools (read_resource, use_prompt) from headless eval
- Pattern-over-rules framing in optimizer, analyst, and observer prompts
- Tool description diffs logged after each iteration
- Tool optimizer failure retries instead of stopping the loop

Tool renames (avoid chat template channel collision on local models):
- create_plan -> plan_agent
- task -> task_agent

Tool descriptions (from eval-driven optimization, 79% -> 98%):
- bash: environment question examples, disambiguation from write_file/man
- edit_file: multi-file workflow, prerequisite clarification, docstring example
- man: "questions about flags are tool-use tasks" prefix
- math: simple example up front
- plan_agent: "delegate to sub-agent" framing, negative boundary for direct edits
- read_file: multi-file workflow hint
- search: trigger phrases, prerequisite clarification, disambiguation
- write_file: immediate action framing, placeholder example

System prompt: enriched tool patterns from eval results, added identity opener

Test suite: plan-before-refactor -> plan-when-asked (simplified)

Docs: comprehensive eval.md rewrite covering all current features
2026-03-25 11:58:10 -07:00
renovate[bot] 5d26cd6593 chore(deps): update dependency katex to v0.16.42 (#175)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-03-25 11:57:59 -07:00
renovate[bot] 2d4420e00d chore(deps): lock file maintenance (#177)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-03-25 11:57:47 -07:00
renovate[bot] b20548583d chore(deps): update ghcr.io/astral-sh/uv docker tag to v0.11.1 (#176)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-03-25 11:57:44 -07:00
Patrick Buckley f63b2915cc review: address copilot feedback on user_id trust check
Remove console-proxy from trusted_sources — end-user tokens via the
console proxy already carry the real user_id in the JWT, so they must
not be able to override it via the request body (impersonation risk).
Only bridge and console service identities are trusted to forward
user_id on behalf of users. Add 5 tests covering the trust boundary.
2026-03-24 03:13:23 -07:00
Patrick Buckley bf85bbea94 fix: resolve user_id to username in usage and audit displays
Usage tab grouped by user showed raw hex user_id instead of username.
Audit tab showed truncated hex. Now both API endpoints resolve user_id
to username via list_users() lookup before returning the response.
2026-03-24 03:13:23 -07:00
Patrick Buckley 803d8ee8f9 docs: document user_id propagation through MQ path
Update security.md with trusted service user_id forwarding. Update
MQ protocol diagram to include user_id field on CreateWorkstreamMessage.
Update console data flow diagram to show user_id in message and bridge
forwarding.
2026-03-24 03:13:23 -07:00
Patrick Buckley 17b5961a70 fix: propagate user_id through MQ workstream creation path
Console create_workstream was constructing CreateWorkstreamMessage
without setting user_id, so workstreams created via console→MQ→bridge
→server had empty user_id in usage events. Now the console extracts
user_id from auth_result and passes it through the MQ message. The
bridge forwards it in the HTTP payload, and the server accepts it
from trusted service callers (bridge, console-proxy).
2026-03-24 03:13:23 -07:00
Patrick Buckley 037308f3b1 fix: propagate user identity through console proxy
Console proxy previously used a fixed service identity (console-proxy)
with full {read,write,approve} scopes for all proxied requests, losing
the real user's identity at the proxy boundary. Now mints per-request
short-lived JWTs carrying the authenticated user's actual user_id,
scopes, and permissions so upstream servers record correct audit
attribution and enforce scope narrowing as defense in depth.
2026-03-24 03:13:23 -07:00
Patrick Buckley d7a9895855 feat: tool description optimization + OOM and logging fixes (#172)
* feat: tool description optimization + OOM and logging fixes

Tool description optimization (three-phase pipeline):
- Tool optimizer (phase 2) modifies tool descriptions to resolve
  confusion, gated by --optimize-tools and wrong_tool detection
- _apply_tool_overrides deep-copies modified tools, never mutates TOOLS
- _propose_tool_overrides validates JSON, deep-merges parameter overrides
- tool_overrides on EvolutionNode, plumbed through full eval pipeline
- --save-tools writes best overrides back to turnstone/tools/*.json
- Prompt optimizer informed when tool descriptions have been modified
- TSV tool_changes column

OOM fix (session lifecycle):
- HeadlessSession created inside retry loop, not outside — timed-out
  orphan threads no longer pin old sessions in memory
- Session ref cleared immediately after extracting results
- Previous behavior leaked unbounded memory per timeout (~515GB OOM)

Logging fix:
- Removed _suppress_stdout entirely — redirected fd 1 process-wide,
  causing main thread print() to vanish during slow API calls
- NullUI already discards session output; tools return strings

New CLI: --optimize-tools, --tool-optimizer-model/base-url, --save-tools

* review: address copilot feedback on tool optimization
2026-03-23 22:47:06 -07:00
Patrick Buckley 0ba49b8bb7 review: address copilot feedback on eval pipeline
- Record prompt_variant in parallel execution path (was serial-only)
- Handle "Subprocess error:" and "Skipped (fast-fail)" in failure
  classifier instead of mis-bucketing as missing_tool
- Build case_id->case_def dict once in _build_failure_analysis,
  _run_analyst, _propose_prompt_modification (was O(n²) linear scan)
- Enforce original prompt at slot 0 of cached user_prompts variants
- Use wall clock time for TSV elapsed_s instead of sum of run times
2026-03-23 19:44:32 -07:00
Patrick Buckley 51b5b3ee74 feat: multi-agent eval pipeline with tree search, analyst, diversifier
UCB tree search for prompt optimization (arXiv:2603.18620):
- EvolutionNode dataclass, UCB1 selection, rolling mean scores
- Replaces fragile linear chain with backtracking via tree
- Holdout set separation prevents optimizer overfitting
- Improvement-based delta feedback to optimizer

Multi-agent optimization pipeline:
- Analyst agent (phase 1): multi-turn with math/bash tools, identifies
  semantic failure patterns, computes statistics across test results
- Optimizer (phase 2): uses analyst diagnosis to edit developer prompt
- Observer: tunes optimizer strategy every 3 iterations
- Diversifier: generates paraphrased prompt variants for phrasing
  robustness, with dedup, delta generation, and JSON caching

Failure classification:
- 8 failure mode buckets (no_tool_call, wrong_tool, missing_tool,
  wrong_args, extra_tools, timeout, error, json_dump)
- Consistency signals (systematic, flaky, marginal)
- Rule-based pre-analysis feeds into analyst as structured input

Logging and observability:
- Config summary at startup (models, case count, runs)
- Per-case diversifier progress with dedup stats
- UCB selection reasoning, node score updates, tree growth
- Extended TSV: node_score, elapsed_s, prompt_len, iter_tokens,
  cumul_tokens columns plus 4-decimal precision

Infrastructure:
- Thread-safe fd-level stdout suppression (os.dup2)
- Prompt variants plumbed through parallel execution path
- Cached variants auto-detected from tests.json user_prompts field

New CLI flags: --explore-constant, --analyst-model/base-url,
--diversifier-model/base-url, --diversify N, --save-variants
2026-03-23 19:44:32 -07:00
Patrick Buckley c3b0ddeba7 fix: add ddgs to mypy ignore_missing_imports for CI compat 2026-03-23 19:11:50 -07:00
Patrick Buckley a533e1c783 fix: use fd-level stdout redirect in eval to avoid thread race
_suppress_stdout() was setting sys.stdout = StringIO() which is
process-global. When send_headless runs in a ThreadPoolExecutor and
blocks on an API call inside the suppression context, the main
thread's print() calls silently go to the StringIO and vanish.

Switch to os.dup2 fd-level redirect which is thread-safe.
2026-03-23 18:15:45 -07:00
Patrick Buckley d8bc78556f bump: v0.8.7 2026-03-23 17:27:23 -07:00
Patrick Buckley 4f5854e768 fix: remove stale type: ignore on ddgs import 2026-03-23 17:10:27 -07:00
Patrick Buckley bf2dc04cb3 feat: UCB tree search for eval prompt optimization
Replace linear optimization chain with UCB1 evolution tree. Each
iteration selects the most promising node to extend, preventing
irrecoverable collapse from bad edits. Also adds improvement-based
delta feedback to the optimizer and optional holdout set separation.

Inspired by "Learning to Self-Evolve" (arXiv:2603.18620).
2026-03-23 17:09:22 -07:00
Patrick Buckley bb894d073b fix: SQLite migrations use batch_alter_table for compat
Migrations 014, 021-024 used bare op.add_column/alter_column/drop_column
which fails on SQLite (no ALTER of constraints). Switch to
batch_alter_table and enable render_as_batch in env.py. Migration 014
also moved UniqueConstraint inline into create_table.
2026-03-23 17:09:13 -07:00
Patrick Buckley b3934a2d14 bump: v0.8.6
Hotfix: DDG web search returning empty results.

- Switch dependency from duckduckgo-search (deprecated shim, empty
  results) to ddgs>=9.0 (actively maintained successor)
- Include ddg extra in Docker image for free web search fallback
- Update all user-facing install instructions to reference ddgs
2026-03-23 16:17:28 -07:00
Patrick Buckley 3d02cf66b4 review: update stale duckduckgo-search references to ddgs 2026-03-23 16:09:03 -07:00
Patrick Buckley 7631b88792 fix: switch DDG dependency from duckduckgo-search to ddgs
duckduckgo-search 8.x is a deprecated shim that returns empty results
(upstream temporarily disabled HTML/Lite backends, Bing backend broken).
The package was renamed to ddgs in v9.x which works correctly.
2026-03-23 16:09:03 -07:00
Patrick Buckley a07172b0c0 fix: include ddg extra in Docker image for free web search fallback 2026-03-23 15:23:12 -07:00
Patrick Buckley f3d33bf44a bump: v0.8.5
Features:
- Pluggable web search backends — DDG as free default, Tavily, MCP (#166)
- --config flag and $TURNSTONE_CONFIG env var (#160)
- Live session config via ConfigStore point-of-use reads (#154)
- PostgreSQL CI integration tests (#156)
- Skill priority ordering (#144)
- Raise scaling limits for 1000-node clusters (#129)

Security:
- Output guard wired into agent loops — plan + task agents (#168)
- Tool policy enforcement in CLI, bridge, and channel (#168)
- Subprocess environment scrubbing — API keys stripped (#168)
- OIDC issuer SSRF validation (#140)
- MCP registry URL scheme validation (#133)
- Output guard enabled in CLI mode (#134)

Reliability:
- Bridge approval/plan review TOCTOU races fixed (#167)
- SQLite WAL mode, eviction cancel, title retry (#151)
- Health monitor OPEN → HALF_OPEN autonomous probe (#152)
- ConfigStore spec alignment (#153)
- Critical production readiness fixes (#147)
- Server startup stampede prevention (#132)
- Python 3.14 CancelledError guard (#146)

Performance:
- Conversations index, batch config saves, capabilities cache (#149)

Quality:
- Bridge stress tests (6 scenarios, 100 iterations each) (#157)
- Governance SDK, MCP reload, skill config integration tests
- Structlog standardization across 19 modules (#150)
- Dead code removal (#148)
2026-03-23 15:00:08 -07:00
Patrick Buckley fdb1a189e8 fix: show policy deny reason in CLI approval output
Denied tools now print the error text (e.g. "Blocked by tool policy")
in red below the header, so the user sees why a tool was blocked.
2026-03-23 14:55:14 -07:00
Patrick Buckley 58e2d9348f review: fix mypy, tighten env scrub, bridge storage safety
Address Copilot + code review feedback:
- Fix mypy: rename tool_names → _policy_names in CLI to avoid type clash
- Tighten env scrub from substring to suffix matching (_KEY, _TOKEN, etc.)
  to avoid false positives on MONKEYTYPE, KEYBOARD_LAYOUT
- Add DATABASE_URL/TURNSTONE_DB_URL to explicit scrub list
- Bridge: use _storage directly instead of get_storage() which
  auto-initializes a local SQLite DB with no admin policies
- Discord bot: add self.storage None guard
- Add tests for suffix-only matching and false positive avoidance
2026-03-23 14:55:14 -07:00
Patrick Buckley 771d03b8e6 review: fix LESS prefix leak, move policy before auto-approve, add tests
- Move LESS/LESSOPEN/LESSCLOSE/LESSPIPE/LESSCHARSET to _SAFE_NAMES
  instead of prefix matching (prevents LESS_SECRET_TOKEN leak)
- Move bridge policy evaluation before auto-approve check so deny
  policies override auto-approve
- Add storage None guard in Discord bot
- Clean up _policy_handled pattern in Discord bot
- Add debug logging on policy evaluation exceptions
- Add tests: extra overrides scrub, LESS prefix safety
2026-03-23 14:55:14 -07:00
Patrick Buckley d147aaea36 security: scrub secrets from subprocess environments
New turnstone/core/env.py provides scrubbed_env() that strips API keys,
tokens, passwords, and credentials from os.environ before passing to
subprocesses. Applied to all 5 subprocess call sites: _exec_bash,
_exec_search, _exec_man, watch _run_command, and MCP stdio servers.

Pattern-based scrubbing (KEY, SECRET, TOKEN, PASSWORD, CREDENTIAL
substrings) plus explicit blocklist for known secrets. Safe vars
(PATH, HOME, locale, etc.) always preserved. Passthrough list for
operator overrides.
2026-03-23 14:55:14 -07:00
Patrick Buckley 8b747178e0 security: enforce tool policies in CLI, bridge, and channel
evaluate_tool_policies_batch() was only called in the server WebUI.
CLI, bridge, and channel entry points now evaluate admin-defined tool
policies before auto-approve checks. Deny policies block tools, allow
policies auto-approve. Best-effort: gracefully skipped if storage is
unavailable.
2026-03-23 14:55:14 -07:00
Patrick Buckley d57280d807 security: wire output guard into agent loops
_run_agent (plan + task agents) now passes tool results through
_evaluate_output() before appending to context — same as the main
session loop. Runs before truncation so the guard sees full output.
Catches prompt injection, credential leakage, and encoded payloads
in agent tool results that were previously unscanned.
2026-03-23 14:55:14 -07:00
Patrick Buckley b9870f279c fix: bridge approval & plan review TOCTOU races (#158, #159) (#167)
* fix: bridge approval & plan review TOCTOU races (#158, #159)

Replace "pop on completion" with a tombstone pattern — pending entries
are marked resolved=True instead of being removed, eliminating the
window where stale SSE reconnect events bypass the duplicate guard.
Resolved tombstones are cleaned up on ws_state events and ws_closed.

Stress tests now pass reliably (previously ~12-16% failure rate).

* review: extract _mark_resolved helper, use real ws_closed path in test, add refinement loop test

Address code review suggestions:
- Extract _mark_resolved() helper in _wait_plan to reduce duplication
- Document cross-stream ordering assumption for plan review refinement
- Race 6 test now calls _handle_global_event instead of manual dict pops
- New Race 7 test validates plan review refinement loop (tombstone → cleanup → re-entry)

* fix: add TTL fallback for tombstone cleanup when global SSE lags

If the global SSE stream is temporarily down while per-WS SSE continues,
resolved tombstones would block legitimate new approvals/plan reviews.
Add a 30s TTL so stale tombstones are expired in the duplicate guard
as a fallback to the normal ws_state-based cleanup.

Also changes tombstone type from (request_id, bool) to
(request_id, float) where 0.0 = active, >0 = resolved_at monotonic time.

* fix: use 3x approval_timeout for tombstone TTL instead of hardcoded 30s

Tie the TTL to the configurable approval_timeout (default 300s = 900s TTL)
rather than a short hardcoded value. The TTL is only a fallback for when
the global SSE stream is completely down — a conservative value is safer.
2026-03-23 14:04:12 -07:00
Patrick Buckley 71ee340bc6 feat: pluggable web search backends (DDG, Tavily, MCP) (#166)
* feat: pluggable web search backends (DDG, Tavily, MCP)

web_search is now an abstract capability with swappable backends:

- DuckDuckGoClient — free, no API key, uses duckduckgo-search library
- TavilyClient — existing behavior, requires API key
- MCPSearchClient — delegates to any MCP server tool

New tools.web_search_backend setting (ConfigStore + --web-search-backend
CLI flag): '' (auto), 'tavily', 'ddg', or 'mcp:server:tool'.

Auto-detection (default): Tavily if key present, else DDG if installed,
else disabled. This means users with duckduckgo-search installed get
web search for free with local models — no API key needed.

Closes #131

* fix: address Copilot review on pluggable web search

- Unknown backend values now log warning + return None (not silent
  fallthrough to auto-detect)
- Pass timeout to DDGS constructor
- MCP: use math.ceil for timeout, forward topic kwarg
- Fix agent mode web_search gating to use _resolve_search_client()
  instead of get_tavily_key() (was still using old check)
- Update docstring to reflect new backend resolution
- Fix DDG test to patch DDGS import properly
- Add test for unknown backend rejection
2026-03-23 13:21:04 -07:00
Patrick Buckley c5d5d0b7cd fix: update-vendored-js.sh detects old version from filesystem
The script detected the old version from pyproject.toml, which Renovate
had already updated. This caused OLD_DIR == NEW_DIR, so the script
downloaded files then immediately deleted them.

Fix: detect old version from the actual directory on disk. Add a guard
that errors if old == new version to prevent silent data loss.

Also: run the fixed script to vendor katex 0.16.40 (fonts + css + js).
2026-03-23 13:03:49 -07:00
renovate[bot] 1f9d03c3e0 chore(deps): update dependency katex to v0.16.40 2026-03-23 13:03:49 -07:00
renovate[bot] 24e082df05 chore(deps): update postgres docker tag to v18 2026-03-23 12:55:27 -07:00
renovate[bot] 4a78d20eea chore(deps): update dependency typescript to v6 2026-03-23 12:55:18 -07:00
renovate[bot] e0d17e0f99 chore(deps): update ghcr.io/astral-sh/uv docker tag to v0.10.12 2026-03-23 12:55:08 -07:00
renovate[bot] cd6c49dd01 chore(deps): update dependency vitest to v4.1.1 (#162)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-03-23 19:53:13 +00:00
Patrick Buckley 8454e961ba feat: --config flag and $TURNSTONE_CONFIG env var (#160)
* feat: --config flag and $TURNSTONE_CONFIG env var for config.toml path

Add set_config_path() to config.py with three-tier resolution:
  1. --config CLI flag (via set_config_path)
  2. $TURNSTONE_CONFIG environment variable
  3. ~/.config/turnstone/config.toml (default)

--config added to all 5 entry points that load config.toml: CLI,
server, console, bridge, eval. Uses parse_known_args pre-parse so
the path is resolved before apply_config reads the file.

Closes #130

* fix: centralize --config pre-parse, fix help and docstrings

- Add add_config_arg() helper with separate pre-parser (add_help=False)
  so --help still shows config-derived defaults
- Replace duplicated pre-parse blocks in all 5 entry points
- Fix set_config_path docstring (works after load_config too)
- Fix module docstring precedence description
- Remove redundant import os in get_tavily_key
2026-03-23 12:26:41 -07:00
Patrick Buckley 4ae38bc2ae test: bridge race condition stress tests (#157)
* test: bridge race condition stress tests (5 scenarios)

Repetition-based stress harness (100 iterations per scenario) targeting
threading races in bridge.py:

1. Duplicate approval on SSE reconnect — xfail, confirms known TOCTOU
   race where _wait_approval pops pending entry allowing duplicate
2. Duplicate plan review on SSE reconnect — xfail, same pattern
3. approve_set consistency during concurrent update — passes
4. _running flag visibility across threads — passes
5. Workstream closure during blocked pop_response — passes
6. Concurrent approval + workstream close — passes (no orphaned state)

Two real races confirmed (marked xfail with fix descriptions).

* fix: address Copilot feedback on bridge stress tests

- Fix plan review mock to use correct message type ("plan_feedback")
- Replace fixed sleeps with bounded _wait_pending_clear() polling
- Add assert not t.is_alive() after all thread joins
- Update Race 5 description to reflect timeout validation (not
  closure-unblocks-pop)
- Update plan review xfail reason to mention generation counters
2026-03-23 11:36:42 -07:00
Patrick Buckley ab1a71c86c feat: add PostgreSQL CI integration tests (#156)
* feat: add PostgreSQL CI integration tests

Add --storage-backend pytest option and shared storage_backend fixture
in conftest.py that creates SQLiteBackend or PostgreSQLBackend based
on the flag. Migrate 13 storage test files to use shared fixture
instead of local SQLiteBackend fixtures.

Add test-postgres CI job with PostgreSQL 17 service container that
runs the full test suite against real PostgreSQL.

* fix: use TRUNCATE CASCADE for PG cleanup, wrap in try/finally

TRUNCATE is faster than per-table DELETE and resets autoincrement
sequences. try/except ensures reset_storage() always runs even if
cleanup fails due to a corrupted connection from a failing test.

* fix: document _engine coupling in PG cleanup comment
2026-03-23 11:11:40 -07:00
renovate[bot] ce57df6888 chore(deps): lock file maintenance 2026-03-23 11:04:43 -07:00
Patrick Buckley 275f40eebb feat: live session config via ConfigStore point-of-use reads (#154)
* feat: live session config via ConfigStore point-of-use reads

Existing sessions now pick up admin settings changes without requiring
workstream recreation. ChatSession reads MemoryConfig and JudgeConfig
behavioral flags from ConfigStore at point-of-use via _mem_cfg and
_judge_cfg properties. Judge LLM client config (model, provider,
base_url, api_key) stays frozen from creation time.

Falls back to frozen dataclasses when ConfigStore is absent (CLI mode).

* fix: type config_store param, clarify _ensure_judge guard comment

* fix: re-check live judge.enabled on every _ensure_judge call

Copilot correctly identified that the cached judge was returned without
re-checking the live enabled flag. Move the enabled check before the
cache check so disabling the judge via admin takes immediate effect.
Also defer JudgeConfig import to local scope in _judge_cfg property
to avoid pulling in the full judge module at import time.
Add test for disable-after-init scenario.
2026-03-21 19:16:06 -07:00
Patrick Buckley 2c510f8617 fix: medium reliability — SQLite WAL, eviction cancel, title retry (#151)
* fix: medium reliability — SQLite WAL + timeout, eviction cancel, title retry

M1: Increase SQLite busy timeout to 30s and enable WAL journal mode
    for better concurrent read/write. Prevents OperationalError under
    multi-workstream write contention.

M2: Call session.cancel() during workstream eviction cleanup so
    in-flight worker threads stop promptly instead of running to
    completion on an evicted workstream.

M3: Reset _title_generated flag on exception so title generation
    retries on the next successful exchange instead of permanently
    giving up after one failure.

* fix: address review — WAL pragma error handling, title retry ws_id guard

Wrap WAL pragma in try/except and verify returned mode. Log warning if
WAL is not enabled (e.g., filesystem permissions) instead of aborting
connection.

Guard title retry flag reset with ws_id comparison to prevent
re-enabling titling for a different workstream after /resume.

* fix: address review — add title retry tests

Two tests verifying _title_generated flag behavior: reset on failure
(allows retry on next turn), stays True on success. Covers the new
retry logic added in this PR.

* fix: guard title update success path against ws_id change during resume

Use captured ws_id on success path (not just failure path) so a
concurrent resume() can't cause the background title thread to rename
the wrong workstream. Add test for the race scenario.
2026-03-21 17:02:25 -07:00
Patrick Buckley 2afb9c7f72 fix: health monitor probe loop transitions OPEN → HALF_OPEN autonomously (#152)
The probe loop continued probing while the circuit was OPEN but never
transitioned to HALF_OPEN — that only happened inside
acquire_request_permit() which requires a user request. If no user
sends a message during the cooldown, the circuit never recovers.

Now the probe loop checks cooldown elapsed and transitions to HALF_OPEN
before probing, so recovery happens automatically without user
interaction.
2026-03-21 16:35:47 -07:00
Patrick Buckley 5b8ab94446 fix: align ConfigStore implementation with spec (#153)
* fix: align ConfigStore implementation with spec

- Add cluster + skills sections to admin UI settings order and labels
- Return default value in DELETE /v1/api/admin/settings response per spec
- Document 4 missing settings in docs/settings.md (trusted_proxies,
  output_guard, redact_secrets, discovery_url) and correct count to 48
- Wire ConfigStore into console server replacing 4 raw
  get_system_setting() calls with validated/cached config_store.get()
- Reload console ConfigStore on settings mutations via
  _publish_config_change()
- Update registry URL tests for ConfigStore-based resolution

* fix: address Copilot review feedback on ConfigStore PR

- Move config_store.reload() before collector guard in
  _publish_config_change() so cache refreshes even without collector
- Add DeleteSettingResponse schema and update OpenAPI spec to match
  the actual delete response (status + key + default)
- Add test asserting default field in delete response
- Fix stale docstring in test helper
2026-03-21 16:12:50 -07:00
Patrick Buckley 30828e9f9c perf: conversations index, batch config saves, capabilities cache (#149)
* perf: add conversations.timestamp index, batch config saves, cache capabilities

P1: Add idx_conversations_timestamp index (migration 025) to eliminate
    full table scans on search_history_recent ORDER BY timestamp DESC.

P2: Batch save_workstream_config — replace N separate SQL statements
    with single executemany call. SQLite uses INSERT OR REPLACE,
    PostgreSQL uses INSERT ON CONFLICT DO UPDATE.

P3: Cache _get_capabilities() result on ChatSession — called 4-6x per
    turn but deterministic for session lifetime. Invalidated on model
    switch.

* fix: address review — capabilities cache bypassed for fallback models

Cache only applies to the primary session model. Fallback models
(different provider/model passed to _get_capabilities) resolve fresh
to avoid stale capability flags affecting tool selection and web search.
2026-03-21 04:29:43 -07:00
Patrick Buckley 6d0dc6df94 chore: standardize logging to structlog get_logger across 19 modules (#150)
Replace bare `import logging` / `logging.getLogger(__name__)` with
`from turnstone.core.log import get_logger` / `get_logger(__name__)`
across all core modules and server.py. This enables structured log
context injection (node_id, ws_id, request_id) in modules that
previously used plain stdlib logging.

Also adds _ensure_stdlib_factory() to log.py for pytest caplog
compatibility when configure_logging() hasn't been called.

Renames `logger` to `log` in skill_sources.py for naming consistency.
2026-03-21 04:29:40 -07:00
Patrick Buckley 7e680ee883 chore: remove dead code — chat.py, singular touch, unused vars, inline imports (#148)
* chore: remove dead code — chat.py shim, singular touch method, unused vars

- Delete turnstone/chat.py (backward-compat re-export shim, zero importers)
- Remove touch_structured_memory() singular method from protocol + both
  backends + 6 tests (only plural batch form is used)
- Remove unused _last_err variable in _compact_messages
- Remove redundant _AGENT_AUTO_TOOLS / _TASK_AUTO_TOOLS class aliases,
  use module-level constants directly
- Consolidate ~76 inline schema imports to top-level in both storage
  backends (channel_users, channel_routes, oidc_*, scheduled_tasks,
  watches, services)

Net: -219 lines

* fix: address review — remove stale inline timedelta imports in prune_task_runs

timedelta is already imported at module scope in both backends.
2026-03-21 04:29:37 -07:00
Patrick Buckley e950219246 docs: add beta status warning to README
Mark platform as experimental beta with explicit disclaimers: no
guarantees of determinism, reliability, or backward compatibility.
Advise thorough evaluation before deployment.
2026-03-21 03:43:13 -07:00
Patrick Buckley b3764a8035 fix: critical reliability fixes for production readiness (#147)
* fix: critical reliability fixes for production readiness

C1: Add 1-hour timeout to _approval_event.wait() and _plan_event.wait()
    to prevent permanent worker thread hangs when users disconnect.

C2: Atomically check-and-start worker thread under Workstream._lock to
    prevent race condition where two concurrent send_message requests
    spawn duplicate workers on the same non-thread-safe ChatSession.

C3: Bound _watch_pending queue to maxsize=20 to prevent OOM under
    heavy watch load with busy workstreams.

H1: Add timeout to proc.wait() (10s) and stderr_thread.join() (5s)
    after SIGKILL to prevent indefinite hang on D-state processes.

H2: Protect _pending_verdicts with _ws_lock at all three mutation sites
    (reset in approve_tools, append in on_intent_verdict, swap-and-clear
    in resolve_approval) to prevent lost verdicts from concurrent
    judge daemon and approval threads.

H3: Bound global SSE queue to maxsize=10000 with put_nowait() and
    contextlib.suppress(queue.Full) for backpressure. Prevents
    unbounded memory growth when fanout thread is overloaded.

H4: Bridge SSE threads for closed workstreams now check ws_id membership
    in _ws_threads before reconnecting, preventing thread leak on
    workstream close.

* fix: address review — verdict lock consistency, watch queue non-blocking, SSE drop logging

- Move _last_verdict_decision set inside _ws_lock in resolve_approval()
  so swap+decision is atomic with on_intent_verdict() reads
- Read _last_verdict_decision under _ws_lock in on_intent_verdict()
- Build heuristic_verdicts locally then assign under lock in approve_tools()
- Use resolve_approval() for timeout path so verdicts are updated consistently
- Watch queue producer uses put_nowait with log on Full (prevents WatchRunner hang)
- Global SSE state broadcasts log on queue.Full instead of silent suppress
- Plan event wait also gets 1-hour timeout (same class of bug as approval)
2026-03-21 03:28:01 -07:00
Patrick Buckley 756c4d8929 fix: guard against CancelledError on MCP startup future (Python 3.14) (#146)
On Python 3.14, Future.exception() raises CancelledError on cancelled
futures instead of returning None. Check future.cancelled() before
calling exception() to prevent crash when the MCP event loop shuts down
before _connect_all completes.
2026-03-21 01:06:11 -07:00
Patrick Buckley 04c50568e9 feat: add priority column for skill ordering control (#144)
* feat: add priority column for skill ordering control

Add priority INTEGER DEFAULT 0 column to prompt_templates (migration
024). Skills with activation="default" are now ordered by priority ASC,
name ASC instead of name-only. Admins can set priority via create/update
API. Lower values run first. Priority is editable on readonly/installed
skills. 4 new tests. Python SDK, TypeScript SDK, and Pydantic models
updated.

* fix: address review — apply priority ordering to list_default_templates

list_default_templates() still ordered by name only, so priority had
no effect on default skill execution order. Update both SQLite and
PostgreSQL backends to order by (priority, name).

* fix: address review — regenerate OpenAPI snapshot, add default template ordering test

Regenerate openapi-console.json to include priority field on skill
models. Add test_list_default_templates_ordered_by_priority to verify
the execution path for default skills respects priority ordering.
2026-03-21 00:53:51 -07:00
Patrick Buckley 3bf220c503 fix: use approval_label for per-tool always-approve in CLI and bridge (#143)
* fix: use approval_label for per-tool always-approve in CLI and bridge

The server stores approval_label (e.g. mcp__server__tool) for per-tool
auto-approve, but CLI and bridge extracted only func_name (bare tool
name). This caused always-approve decisions to not carry over across
access paths. Align CLI and bridge to prefer approval_label with
func_name fallback, matching the server's WebUI.approve_tools() pattern.

* fix: address review — exclude errored items from bridge auto-approve check

Filter out items with error set from the auto-approve subset check,
matching the server's WebUI.approve_tools() behavior. Prevents
policy-denied items from affecting auto-approve decisions.
2026-03-21 00:42:33 -07:00
Patrick Buckley 4b853e329e test: verify skill_id/skill_version populated in workstreams table (#145)
The workstreams.skill_id and skill_version columns were already being
populated correctly (wired in the skills unification PR #106). Add two
tests confirming: lineage columns set when skill is applied, and
defaults when no skill is used. Check off the PROGRESS.md item.
2026-03-21 00:40:30 -07:00
Patrick Buckley 29ffdc36d0 test: add governance SDK integration tests against real Starlette app (#142)
* test: add governance SDK integration tests against real Starlette app

24 TestClient-based tests verifying round-trip serialization of SDK
governance methods (roles, policies, orgs) against actual route
handlers with SQLite storage. Covers create/list/update/delete
lifecycles, error cases, and Pydantic model field validation.

* fix: address review — close AsyncClient in sdk_client fixture teardown

Convert sdk_client fixture to async context manager so the httpx
AsyncClient is properly closed after tests, avoiding resource leak
warnings.
2026-03-21 00:31:53 -07:00
Patrick Buckley ada8b80509 test: add MCP reload and reconcile endpoint integration tests (#141)
* test: add MCP reload and reconcile endpoint integration tests

11 new tests covering POST /v1/api/admin/mcp-servers/reload (console)
and POST /v1/api/_internal/mcp-reload (node). Verifies reconcile_sync
invocation, fan-out results, permission checks, missing storage
handling, and mixed node error propagation.

* fix: address review — lazy-import internal_mcp_reload to avoid heavy module load

Move turnstone.server import inside _routes_with_internal() helper so
the full server module (which reads UI static assets) is only loaded
when node-side endpoint tests actually run, not during test collection.
2026-03-21 00:14:50 -07:00
Patrick Buckley 1f47ca62de fix: validate OIDC issuer URLs against SSRF before discovery fetch (#140)
* fix: validate OIDC issuer URLs against SSRF before discovery fetch

Add validate_issuer_url() that rejects private/loopback/link-local IPs,
non-HTTPS (except localhost for dev), embedded credentials, and
unresolvable hostnames. Called before the HTTP fetch in discover_oidc()
so the request is never made for invalid URLs. 17 new tests.

* fix: address review — use is_global, redact userinfo, catch ValueError

Use `not addr.is_global` instead of individual range checks to cover
all non-routable addresses (CGNAT, unspecified, multicast). Redact
credentials from error messages to prevent log leakage. Catch ValueError
from ip_address() for zone-indexed IPv6 addresses.
2026-03-21 00:14:46 -07:00
Patrick Buckley 83d9233304 test: skill session config application to workstreams (#139)
* test: skill session config application to workstreams

13 TestClient-based integration tests verifying that skill session
config fields (model, temperature, token_budget, auto_approve,
allowed_tools, reasoning_effort, agent_max_turns) are correctly applied
to ChatSession and WebUI when creating a workstream with a skill.

Covers: individual fields, combined application, disabled/unknown skill
rejection, zero-value no-ops.

* fix: address review — pass skill kwarg, clarify no-op test assertions

Pass skill=kwargs.get("skill") into ChatSession in test factory to
match production behavior. Clarify zero-value no-op test docstrings
to document they verify the handler's guard conditions, not observable
state changes.
2026-03-20 23:09:59 -07:00
Patrick Buckley bf06102d37 fix: memory access tracking and BM25 context caching (#138)
* fix: memory access tracking and BM25 context caching

Add touch_structured_memory/touch_structured_memories to storage
protocol + SQLite/PostgreSQL backends. Bumps last_accessed and
access_count on memory retrieval (BM25 injection + search results).
9 new storage tests.

Cache the scored BM25 memory context string on ChatSession, invalidated
on memory save/delete. Eliminates ~12 redundant storage queries + index
rebuilds per session lifecycle.

* fix: address review — deduplicate keys in touch facade, clarify contract

Deduplicate keys in the memory.py facade before calling storage so each
distinct memory is touched at most once. Update protocol docstring to
clarify per-call increment semantics. Add deduplication unit test.

* fix: replace unused-import test with real batch duplicate test

Replace facade dedup test (which only tested Python set logic) with a
real storage-level test that verifies duplicate keys each increment
access_count. Fixes ruff F401 lint failure.
2026-03-20 23:08:32 -07:00
Patrick Buckley e015b4512d fix: return typed Pydantic models from SDK skill methods (#137)
Skill methods on TurnstoneConsole and AsyncTurnstoneConsole returned
dict[str, Any] instead of validated Pydantic models. Update list_skills,
create_skill, get_skill, update_skill, list_skill_resources,
create_skill_resource, and install_skill to use response_model= with
ListSkillsResponse, SkillInfo, SkillResourceInfo, and
SkillInstallResponse.
2026-03-20 20:02:06 -07:00
Patrick Buckley 0c1afff7fc test: add _get_registry_url three-tier fallback tests (#136)
8 tests covering the DB setting → config.toml → default URL resolution
chain, including storage errors, empty values, malformed JSON, and
documenting that RuntimeError propagates uncaught through the except
clause.
2026-03-20 20:01:58 -07:00
Patrick Buckley a94051a995 fix: add split pane button to tab bar for discoverability (#135)
* fix: add split pane button to tab bar for discoverability

The split pane feature was only accessible via right-click context menu
or Ctrl+\ keyboard shortcut. Add a subtle split icon (⧉) to the tab
bar that appears at low opacity in single-pane mode. Hidden in
multi-pane mode where pane headers already provide split/close controls.

* fix: address review — change tab-bar from tablist to toolbar role

The tab bar contains both tabs and action buttons (new workstream,
split pane), which is invalid for role=tablist. Change to role=toolbar
which correctly describes a container of mixed interactive controls.

* fix: address design review — WCAG contrast, ARIA structure, mobile

- Drop opacity approach, use border: dashed var(--border) matching
  #new-tab-btn pattern (fixes WCAG contrast failure at 35% opacity)
- Nest tabs in #tab-list[role=tablist] inside toolbar (fixes invalid
  role=tab children inside role=toolbar)
- Hide split button on mobile (<600px) where splits can't work
- Add aria-keyshortcuts to both action buttons
2026-03-20 20:01:35 -07:00
Patrick Buckley 2f906ea1f9 fix: enable output guard in CLI mode (#134)
* fix: enable output guard in CLI mode

The heuristic output guard (credential redaction, prompt injection
detection) only ran in server mode because cli.py never constructed a
JudgeConfig. Additionally, the guard condition in session.py required
enabled=True, coupling the zero-cost heuristic (<5ms) to the full LLM
judge.

Wire JudgeConfig from existing CLI args into the session factory.
Decouple the output_guard condition from the enabled flag so the
heuristic guard runs even when the LLM judge is disabled via --no-judge.

* fix: address review — pass config.toml judge fields to CLI JudgeConfig

apply_config() merges [judge] section from config.toml into args as
judge_base_url and judge_api_key. Pass these through to JudgeConfig so
the CLI respects config.toml judge settings (e.g. separate judge
endpoint).
2026-03-20 20:01:31 -07:00
Patrick Buckley b61bfd1aa6 fix: validate URL scheme after MCP registry template substitution (#133)
* fix: validate URL scheme after MCP registry template substitution

resolve_install_config() substitutes user-provided values into URL
templates via string replacement without validating the resulting URL.
Add urlparse check after substitution to reject non-HTTP(S) schemes,
preventing SSRF-style redirection through crafted template variables.

* fix: address review — reject empty hostname and embedded credentials

Add hostname presence check and userinfo rejection after URL scheme
validation. Prevents URLs like https:///path (no host) and
https://user:pass@host (credential leakage in config). Two new tests.
2026-03-20 20:01:26 -07:00
Patrick Buckley 19c3a48b10 fix: server startup stampede — timeout model detection, non-fatal PG … (#132)
* fix: server startup stampede — timeout model detection, non-fatal PG migrations

detect_model() blocked the main thread for up to 400s when the LLM backend
was unreachable (OpenAI SDK default: 600s read timeout × 2 retries × TCP
retransmit). Cap startup detection at 10s with no retries — the
BackendHealthMonitor handles ongoing availability probing after startup.

PostgreSQL migrations via _run_with_pg_lock() crashed the server on lock
contention when 10 containers stampeded the advisory lock simultaneously.
Wrap in try/except matching the SQLite path — the entrypoint script already
runs migrations before the server process starts.

Health check start_period increased from 15s to 60s to accommodate the
startup sequence under load.

* fix: address review — narrow PG migration except, add detect_model test

Narrow the PG migration except clause to (OSError, EOFError) so DDL
errors still propagate. Add two unit tests for detect_model() verifying
with_options(timeout=10, max_retries=0) is called and that connection
errors in non-fatal mode return (None, None).
2026-03-20 20:01:21 -07:00
Patrick Buckley 414eb52d67 feat: raise scaling limits for 1000-node clusters (#129)
* feat: raise scaling limits for 1000-node clusters

Raise hardcoded limits throughout the codebase so clusters up to 1000
nodes work without configuration changes.

Scaling limits:
- max_workstreams default 10 → 50 (configurable via settings)
- Console fan-out concurrency 50 → 200 (configurable: cluster.node_fan_out_limit)
- MCP max servers 50 → 200 (configurable: cluster.mcp_max_servers)
- Console SSE queue 500 → 2000, server global SSE queue 500 → 1000
- httpx proxy pool: explicit max_connections on both proxy clients
- PostgreSQL pool 5+10 → 2+3 per process (right-sized for short-burst queries)
- Redis pool: explicit max_connections=200 on both sync and async brokers

Performance optimizations:
- Redis list_nodes(): replace N+1 SCAN+GET with SCAN+MGET
- Collector poll: raise thread pool to 200 (matches fan-out limit)
- Server SSE: dedicated ThreadPoolExecutor(200) for queue polling
- Fan-out: new get_all_nodes() removes hardcoded limit=1000 ceiling

Bug fixes:
- Settings reload notification was silently failing (called .get() on tuple)
- Watch fan-out only queried 500 nodes instead of full cluster

New cluster settings (configurable via admin Settings tab):
- cluster.node_fan_out_limit (default 200, range 10-1000)
- cluster.mcp_max_servers (default 200, range 1-2000)

Adds docs/pgbouncer.md for PostgreSQL connection pooling at scale.
Adds ddgStressCluster compose profile (100 nodes, 10 groups of 10).
Updates architecture, console, docker, settings, and API reference docs.

* fix: add image tag to compose anchors to avoid redundant builds

All cluster/stress services inherit `build:` from the anchor, causing
Docker to attempt 200+ separate builds. Adding `image: turnstone:local`
means Docker builds once and all services reuse the cached image.

* fix: address Copilot review feedback on scaling PR

- Remove magic number in get_all_nodes (limit=None instead of 2**31)
- Size httpx proxy pool from fan-out limit setting (not hardcoded 250)
- Cap cluster.node_fan_out_limit max_value to 500, mark restart_required
- Convert _publish_config_change from sync to async (was blocking event loop)
- Use shutdown(wait=True, cancel_futures=True) for SSE executor

* fix: add PostgreSQL env vars to cluster bridge anchor

Bridges initialize storage for auth/migrations but the bridge anchor
was missing TURNSTONE_DB_BACKEND and TURNSTONE_DB_URL, causing all
bridges to fall back to SQLite. With 100 bridges sharing the same
volume, concurrent SQLite migrations corrupt the database.

* fix: address Copilot round 2 + PG connection exhaustion at startup

Copilot feedback:
- Raise cluster.node_fan_out_limit max_value to 1000 (matches target)
- Cache fan-out limit on app.state at startup instead of re-reading DB
  per request (pool and semaphore now use the same value consistently)
- Remove unused params from _publish_config_change

Stress cluster fix:
- Raise PG max_connections to 300 (configurable via POSTGRES_MAX_CONNECTIONS)
  to handle 200 processes connecting simultaneously at startup
- Bump PG shared_buffers to 128MB and memory limit to 1G to match
- Add DB env vars to production bridge service

* fix readme

* fix: startup resilience for large clusters

Server no longer crashes when LLM backend is unreachable at startup.
detect_model() accepts fatal=False, returning (None, None) so the
server starts in degraded mode with circuit breaker open. The health
monitor will detect when the backend becomes available.

Migration runner retries with jittered exponential backoff (up to 10
attempts) when PostgreSQL rejects connections during startup stampedes.

Collector httpx pool sized to match poll workers (was using default of
100 connections with 200 workers).

Also addresses Copilot round 2:
- Raise cluster.node_fan_out_limit max_value to 1000
- Cache fan-out limit on app.state at startup
- Remove unused params from _publish_config_change
- Add DB env vars to production bridge service

* fix: replace silent error suppression with structured logging

Audit and fix 30+ instances of silently swallowed exceptions across 8
files. No-raise contracts are preserved — all changes add logging
while keeping the same return-value behavior.

memory.py (26 changes):
  Every storage operation now logs on failure. Previously the entire
  persistence facade had zero logging — messages, workstream state,
  and structured memories could silently stop being saved.

server.py:
  Usage recording failures now log at warning (was pass).
  Global SSE fan-out errors log at debug (was pass).

console/server.py:
  Config reload notification logs per-node failures at warning.
  Settings read fallbacks log at warning with the default value used.

auth.py:
  User existence check logs at warning (was pass).
  Setup rollback failures log at error (was suppress).
  OIDC state cleanup logs at debug (was suppress).

mcp_client.py:
  DB-managed MCP server list failure logs at warning (was pass).

collector.py:
  Node poll failure upgraded from debug to warning with exc_info.
  Health fetch failure logs at debug with exc_info (was silent).

bridge.py:
  Best-effort plan rejection logs at warning (was suppress).
  Malformed SSE data logs at debug (was suppress).

session.py:
  Tool output UI callback failure logs at debug (was suppress).

* fix: stagger collector poll with deterministic per-node jitter

Each node gets a stable offset within the first half of the poll
interval, derived from hashing the node_id against a Mersenne prime
(2^31 - 1). This spreads HTTP requests across the cycle instead of
firing all 100+ at the same instant.

Also raises poll interval from 10s to 15s and HTTP timeout from 5s
to 30s for large-cluster resilience.

* fix: add startup jitter to bridge heartbeat and health monitor probe

Bridge heartbeat: deterministic per-node jitter (from node_id hash)
spreads initial registration across the first quarter of the heartbeat
TTL. At 100 bridges with 60s TTL, heartbeats spread across 15s instead
of all firing at T=0.

Health monitor probe: deterministic per-process jitter (from PID hash)
spreads initial LLM backend probes across half the probe interval. At
100 servers with 30s interval, probes spread across 15s instead of all
hitting the LLM at T=30.

Both use the same Mersenne prime hashing approach as the collector poll
jitter for consistency.

* fix: split collector httpx timeout and raise keepalive pool

Use separate connect/read/write/pool timeouts instead of a single 30s
for all phases. Raise keepalive connections from 50 to 200 so the
collector reuses TCP connections across poll cycles instead of
constantly tearing down and re-establishing them.

* fix: narrow detect_model return type for CLI and eval callers

detect_model() now returns tuple[str | None, int | None] to support
fatal=False. CLI and eval always use fatal=True (the default), which
guarantees a non-None model or SystemExit. Add assert to narrow the
type for mypy.
2026-03-19 04:53:11 -07:00
190 changed files with 10233 additions and 1466 deletions
+26
View File
@@ -47,6 +47,32 @@ jobs:
name: coverage-${{ matrix.python-version }}
path: coverage.xml
test-postgres:
runs-on: ubuntu-latest
services:
postgres:
image: postgres:18
env:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
POSTGRES_DB: turnstone_test
ports:
- 5432:5432
options: >-
--health-cmd="pg_isready -U postgres"
--health-interval=10s
--health-timeout=5s
--health-retries=5
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
- uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6
with:
python-version: "3.12"
- run: pip install -e ".[test,mq,postgres]"
- run: pytest tests/ -m "not live" --storage-backend=postgresql -q
env:
TURNSTONE_TEST_PG_URL: postgresql+psycopg://postgres:postgres@localhost:5432/turnstone_test
lock-check:
runs-on: ubuntu-latest
steps:
+3 -3
View File
@@ -8,7 +8,7 @@ FROM python:3.14-slim
LABEL org.opencontainers.image.title="turnstone" \
org.opencontainers.image.description="Multi-node AI orchestration platform"
COPY --from=ghcr.io/astral-sh/uv:0.10.11 /uv /usr/local/bin/uv
COPY --from=ghcr.io/astral-sh/uv:0.11.1 /uv /usr/local/bin/uv
# System dependencies for psycopg (PostgreSQL client library)
RUN apt-get update && apt-get upgrade -y && apt-get install -y --no-install-recommends libpq5 \
@@ -25,12 +25,12 @@ ENV UV_COMPILE_BYTECODE=1
# Install dependencies first (cached layer — only re-runs when deps change)
COPY pyproject.toml uv.lock README.md LICENSE ./
RUN uv sync --frozen --no-install-project --no-dev \
--extra mq --extra console --extra sim --extra postgres --extra discord --extra anthropic
--extra mq --extra console --extra sim --extra postgres --extra discord --extra anthropic --extra ddg
# Install the project itself
COPY turnstone/ turnstone/
RUN uv sync --frozen --no-dev \
--extra mq --extra console --extra sim --extra postgres --extra discord --extra anthropic
--extra mq --extra console --extra sim --extra postgres --extra discord --extra anthropic --extra ddg
# Add venv to PATH so entry points are found
ENV PATH="/app/.venv/bin:$PATH"
+8 -6
View File
@@ -5,9 +5,11 @@
[![Python](https://img.shields.io/pypi/pyversions/turnstone)](https://pypi.org/project/turnstone/)
[![License](https://img.shields.io/badge/license-BSL--1.1-blue)](LICENSE)
Multi-node AI orchestration platform. Deploy tool-using AI agents across a cluster of servers, driven by message queues or interactive interfaces.
Experimental multi-node AI orchestration platform. Deploy tool-using AI agents across a cluster of servers, driven by message queues or interactive interfaces.
Named after the [Ruddy Turnstone](https://en.wikipedia.org/wiki/Ruddy_turnstone) — a bird that flips rocks to expose what's hiding underneath.
> **Beta — Use at your own risk.** Turnstone is under active development and has not reached a stable release. APIs, configuration formats, and database schemas may change between versions without migration paths. We make no guarantees of determinism, reliability, or backward compatibility. Evaluate thoroughly before deploying to any environment where these properties matter.
Named after the [Ruddy Turnstone](https://en.wikipedia.org/wiki/Ruddy_turnstone) (*Arenaria interpres*) — a shorebird that flips stones to discover what's hiding underneath.
## What it does
@@ -308,7 +310,7 @@ search_max_results = 5 # max tools returned per search query
[server]
host = "0.0.0.0"
port = 8080
max_workstreams = 10 # auto-evicts oldest idle when full
max_workstreams = 50 # auto-evicts oldest idle when full
[redis]
host = "localhost"
@@ -340,7 +342,7 @@ burst = 20
backend = "sqlite" # "sqlite" (default) or "postgresql"
path = ".turnstone.db" # SQLite file path (relative to working directory)
# url = "postgresql+psycopg://user:pass@host:5432/turnstone" # PostgreSQL
# pool_size = 5 # PostgreSQL connection pool size
# pool_size = 2 # PostgreSQL connection pool size (per process)
[judge]
enabled = true # intent validation for tool approvals (--no-judge to disable)
@@ -394,7 +396,7 @@ Idle workstreams are automatically cleaned up after 2 hours (configurable). In m
- `turnstone_judge_llm_latency_seconds` — LLM judge evaluation latency histogram
- `turnstone_judge_enabled` — whether the intent validation judge is active (0/1)
Per-workstream metrics are labeled by `ws_id` (bounded to 10 max workstreams).
Per-workstream metrics are labeled by `ws_id` (bounded by `[server].max_workstreams`).
### Health & Rate Limiting
@@ -404,7 +406,7 @@ Per-workstream metrics are labeled by `ws_id` (bounded to 10 max workstreams).
**Per-IP rate limiting.** When `[ratelimit].enabled` is true, each client IP is tracked with a token-bucket limiter (`requests_per_second` / `burst`). Rate limiting is applied in `do_GET`/`do_POST` after authentication but before route dispatch. `/health` and `/metrics` are exempt. Requests that exceed the limit receive HTTP 429 with a `Retry-After` header.
**Workstream eviction.** When `WorkstreamManager.create()` would exceed `max_workstreams`, the oldest IDLE workstream is automatically evicted and the `turnstone_workstreams_evicted_total` counter is incremented. Configure via `[server].max_workstreams` (default 10).
**Workstream eviction.** When `WorkstreamManager.create()` would exceed `max_workstreams`, the oldest IDLE workstream is automatically evicted and the `turnstone_workstreams_evicted_total` counter is incremented. Configure via `[server].max_workstreams` (default 50).
## Requirements
+2354 -5
View File
File diff suppressed because it is too large Load Diff
+1 -1
View File
@@ -554,7 +554,7 @@ Possible `state` values:
| `error` | An error occurred |
**Fan-out pattern:** Each connected client receives its own bounded queue
(`maxsize=500`). A dedicated fan-out thread reads from the shared global queue
(`maxsize=1000`). A dedicated fan-out thread reads from the shared global queue
and copies each event to every client queue. If a client queue is full, the
event is silently dropped for that client.
+10 -4
View File
@@ -91,7 +91,7 @@ turnstone/
_config.py Base ChannelConfig dataclass
discord/ Discord adapter (bot, cog, views, streaming, config)
shared_static/ Shared design system (base.css, auth.js, theme.js, toast.js, utils.js, kb.js)
katex-0.16.38/ Vendored KaTeX math rendering library (MIT, woff2 fonts)
katex-0.16.42/ Vendored KaTeX math rendering library (MIT, woff2 fonts)
ui/
colors.py ANSI color constants with NO_COLOR support
markdown.py Streaming terminal markdown renderer (line-buffered)
@@ -353,7 +353,7 @@ remove the tab immediately. Controlled by `--workstream-idle-timeout` (default:
**Workstream eviction at capacity:** When `WorkstreamManager.create()` would
exceed `max_workstreams` (configurable via `[server].max_workstreams`, default
10), the oldest IDLE workstream is automatically evicted to make room. The
50), the oldest IDLE workstream is automatically evicted to make room. The
`turnstone_workstreams_evicted_total` counter is incremented on each eviction.
If no IDLE workstream is available the create request fails as before.
@@ -835,10 +835,16 @@ and are the single source of truth for both backends and Alembic migrations.
backend = "sqlite" # "sqlite" | "postgresql"
path = ".turnstone.db" # SQLite file path
url = "" # PostgreSQL connection URL
pool_size = 5 # PostgreSQL connection pool size
pool_size = 2 # PostgreSQL connection pool size (per process)
```
Environment variables: `TURNSTONE_DB_BACKEND`, `TURNSTONE_DB_URL`, `TURNSTONE_DB_PATH`.
Environment variables: `TURNSTONE_DB_BACKEND`, `TURNSTONE_DB_URL`, `TURNSTONE_DB_PATH`,
`TURNSTONE_DB_POOL_SIZE`.
The default pool is intentionally small (2 base + 3 overflow = 5 per process)
because all database operations are short-burst queries that hold connections for
milliseconds. For clusters with many nodes sharing a PostgreSQL instance, use
[PgBouncer](pgbouncer.md) in transaction pooling mode.
### Persistence and Resume
+5 -4
View File
@@ -69,10 +69,11 @@ All reads and writes to the node/workstream map are protected by a single `threa
### Scale Considerations
- **10,000 workstreams** at ~500 bytes each = ~5 MB in memory
- **1,000 nodes** polled in parallel with 50 threads at ~100ms each = ~2 second poll cycle
- **50,000 workstreams** (1,000 nodes × 50 per node) at ~500 bytes each = ~25 MB in memory
- **1,000 nodes** polled in parallel — fan-out concurrency is configurable via `cluster.node_fan_out_limit` (default 200), yielding 5 batches at ~100ms each = ~0.5 second poll cycle
- **Filtering and pagination** run in-memory on the full workstream list — sub-millisecond at this scale
- **SSE fan-out** uses the same per-client queue pattern as the per-node server — backed-up clients get events dropped, not blocking
- **SSE fan-out** uses per-client queues (2,000 events) — backed-up clients get events dropped, not blocking
- **Database** — for clusters sharing PostgreSQL, use [PgBouncer](pgbouncer.md) in transaction pooling mode
---
@@ -364,7 +365,7 @@ SSE streams (`/v1/api/events`, `/v1/api/events/global`) are proxied as raw byte
### Authentication
The proxy forwards the user's JWT to upstream server nodes — it extracts the token from the incoming request's cookie (or `Authorization` header) and adds it as a `Bearer` header on the proxied request. Since all services share the same `TURNSTONE_JWT_SECRET`, the user's JWT is valid on every node without re-authentication. The console's own auth middleware also checks proxy routes — `POST` requests to proxy write endpoints (`/v1/api/send`, `/v1/api/approve`, etc.) require `write` scope, preventing read-only tokens from escalating via proxy. The static `--auth-token` / `proxy_auth_token` is used as a fallback when no user JWT is present.
The proxy mints a short-lived (5-minute) JWT per request carrying the real user's `user_id`, `scopes`, and `permissions` with `aud: turnstone-server`. The user's console JWT (`aud: turnstone-console`) cannot be forwarded directly — it would be rejected by the server's audience validation — so the console re-signs a new server-audience JWT from the validated `AuthResult`. This preserves audit attribution (the upstream server sees the real user, not a service identity) and enforces scope narrowing as defense in depth (a read-only console user's proxied request carries only `read` scope). The JWT `src` claim is set to `"console-proxy"` for audit traceability. When no user context is available (auth disabled), the proxy falls back to a `ServiceTokenManager` with service identity `console-proxy`. The static `--auth-token` / `proxy_auth_token` is used as a final fallback.
---
+1
View File
@@ -61,6 +61,7 @@ package "Inbound Messages (Client → Bridge)" as InPkg #FFF3E0 {
+ target_node: str = ""
+ initial_message: str = ""
+ skill: str = ""
+ user_id: str = ""
}
class CloseWorkstreamMessage {
+4 -3
View File
@@ -105,7 +105,7 @@ Server -> CC : get_snapshot()
CC --> Server : ClusterSnapshot\n(full current state)
Server -> CC : register_listener(queue)
note right : Per-client queue.Queue(maxsize=500)\nSSE via EventSourceResponse + run_in_executor()
note right : Per-client queue.Queue(maxsize=2000)\nSSE via EventSourceResponse + run_in_executor()
Server -> Browser : data: {"type":"snapshot",...}\n(full state as first SSE event)
@@ -154,7 +154,7 @@ activate Server #FFECB3
Server -> CC : _pick_best_node() or\nget_node_detail(node_id)
CC --> Server : node validated
Server -> Server : Build CreateWorkstreamMessage\n{target_node:"nodeA", name:"new-task"}
Server -> Server : Build CreateWorkstreamMessage\n{target_node:"nodeA", name:"new-task",\nuser_id: from auth_result}
Server -> Redis : RPUSH turnstone:inbound:nodeA\n(directed queue)
Server --> Browser : {status:"ok", correlation_id:"abc",\ntarget_node:"nodeA"}
@@ -163,7 +163,8 @@ deactivate Server
note right of Redis
Bridge on Node-A picks up the
message from its directed queue,
POSTs to /v1/api/workstreams/new,
POSTs to /v1/api/workstreams/new
(forwarding user_id in payload),
registers ownership, publishes
ws_created to cluster channel.
end note
+29
View File
@@ -56,6 +56,26 @@ node "Docker Host" as host {
end note
}
node "postgres (profile: production)" <<pgautoupgrade>> as pg_node {
component [PostgreSQL\nport 5432] as postgres
note bottom of postgres
Healthcheck: pg_isready
Volume: postgres-data
Required for cluster
and production profiles
end note
}
node "pgbouncer (optional)" <<bitnami/pgbouncer>> as pgb_node {
component [PgBouncer\nport 6432] as pgbouncer
note bottom of pgbouncer
pool_mode: transaction
Recommended for clusters
> 50 nodes
See docs/pgbouncer.md
end note
}
node "sim (profile: sim)" <<turnstone image>> as sim_node {
component [turnstone-sim] as sim
note bottom of sim
@@ -92,6 +112,11 @@ console --> server : HTTP polling + proxy\n(GET /v1/api/dashboard,\nproxy /node/
sim --> redis : Redis protocol\n(queues + pubsub + keys)
' Database connections (production/cluster profiles)
server ..> pgbouncer : PostgreSQL\n(pool_size=2)
console ..> pgbouncer : PostgreSQL\n(auth/admin)
pgbouncer --> postgres : transaction\npooling
' Environment variables
note right of host
**Environment Variables:**
@@ -99,13 +124,17 @@ note right of host
• OPENAI_API_KEY — API key
• REDIS_PASSWORD — Redis auth
• TURNSTONE_AUTH_TOKEN — API auth
• TURNSTONE_DB_URL — PostgreSQL URL
• POSTGRES_PASSWORD — DB password
end note
' Volumes
database "redis-data" as rv
database "turnstone-data" as tv
database "postgres-data" as pv
redis_node --> rv
server_node --> tv
pg_node --> pv
@enduml
+6 -5
View File
@@ -53,10 +53,10 @@ class "SQLiteBackend" as SQLite <<sqlite>> {
class "PostgreSQLBackend" as PG <<postgres>> {
-_engine: sa.Engine
+__init__(url: str, pool_size: int)
+__init__(url: str, pool_size: int = 2,\n max_overflow: int = 3)
--
tsvector + ILIKE search
Connection pooling
Connection pooling (5 max per process)
}
' -- Schema --
@@ -151,7 +151,7 @@ note right of Registry
backend = "sqlite" | "postgresql"
url = "postgresql+psycopg://..."
path = ".turnstone.db"
pool_size = 5
pool_size = 2 (+ 3 overflow)
end note
note bottom of SQLite
@@ -162,8 +162,9 @@ end note
note bottom of PG
Production backend.
Multi-node / Docker
default.
Multi-node / Docker default.
Use PgBouncer (transaction mode)
for clusters > 50 nodes.
end note
@enduml
+11
View File
@@ -176,4 +176,15 @@ note bottom of SH
Both share JWT signing secret
end note
note left of JWT
**Console Proxy Token Minting**
When proxying requests to server nodes:
1. Console AuthMiddleware validates user JWT (aud: turnstone-console)
2. Proxy mints new JWT (aud: turnstone-server)
with real user_id, scopes, permissions
3. src: "console-proxy" for audit traceability
4. 5-minute expiry (fresh per request)
5. Fallback: ServiceTokenManager if no user context
end note
@enduml
+2 -2
View File
@@ -1,3 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:a74b4b8b5dbfb1a51a01100b731477968942b01218bad9451a3d5a9cb3003294
size 411665
oid sha256:e3f1ad0fcd55eaca3b8ad9c5abc07432803641c54ede9fc93c79df144cf77d1c
size 407761
+2 -2
View File
@@ -1,3 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:84524f4bc900708ac8adf081591d336f862830188eb8505e71a0f071b339d923
size 252599
oid sha256:09065fef028d05e6df425fd8abefaf5a2ca04b66802f2e3975f289fa597f63ed
size 309656
@@ -1,3 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:c94556889abb382cd5b818639fc0a4706beef3d9c7a0b4cbedc763943d657dd0
size 244998
oid sha256:b047cdc318c505f0f0895a65e14c5cc7552716053055cca57fa0a77db150e618
size 255458
+5
View File
@@ -109,9 +109,12 @@ All configuration is via environment variables in `.env` (copy from `.env.exampl
|----------|---------|-------------|
| `TURNSTONE_DB_BACKEND` | `sqlite` | Storage backend: `sqlite` or `postgresql` |
| `TURNSTONE_DB_URL` | — | Database URL (e.g. `postgresql://user:pass@db:5432/turnstone`). For SQLite, defaults to `/data/.turnstone.db` |
| `TURNSTONE_DB_POOL_SIZE` | `2` | PostgreSQL connection pool size per process (default: 2 base + 3 overflow = 5 max) |
The database stores workstream history, user accounts, and API tokens. When using JWT auth, a database backend is required for user storage.
> **Large clusters:** Each turnstone process maintains a small connection pool (5 max). At hundreds of nodes this adds up — use [PgBouncer](pgbouncer.md) in transaction pooling mode between turnstone and PostgreSQL.
> **First-time setup:** After deploying with auth enabled, create an initial admin user by running `turnstone-admin create-user` inside the container:
>
> ```bash
@@ -151,6 +154,8 @@ POSTGRES_PASSWORD=secret docker compose --profile cluster up
The default `server` and `bridge` also run alongside the cluster nodes (11 total). All nodes are accessible via the console dashboard at `:8090`.
For production clusters beyond ~50 nodes, add PgBouncer between turnstone services and PostgreSQL. See [PgBouncer Connection Pooling](pgbouncer.md) for Docker Compose and Helm configuration.
## Volumes
| Volume | Mount | Purpose |
+254 -68
View File
@@ -2,7 +2,8 @@
`turnstone-eval` is the evaluation and prompt optimization system for turnstone. It
runs test cases against the LLM, scores tool call sequences against expected
actions, and optionally uses the model to self-optimize the developer prompt.
actions, and optionally uses a multi-agent pipeline to optimize the developer
prompt and tool descriptions.
Source: `turnstone/eval.py`
@@ -10,15 +11,24 @@ Source: `turnstone/eval.py`
## Overview
The system works in an iterative loop:
The system uses UCB tree search to explore prompt variants:
1. Run each test case N times against the current developer prompt.
2. Score each run by comparing the actual tool call sequence to expected actions.
3. If not all tests pass, use the model to rewrite the prompt based on failures.
4. Repeat until all tests pass or max iterations are reached.
1. Maintain an **evolution tree** of prompt variants, starting from the initial prompt.
2. Each iteration, **UCB1 selects** the most promising node to evaluate.
3. Run each test case N times against the selected prompt.
4. Score each run by comparing the actual tool call sequence to expected actions.
5. If not all tests pass, run a **three-phase optimization pipeline**:
- Phase 1: Analyst diagnoses semantic failure patterns
- Phase 2: Tool optimizer adjusts tool descriptions (when `--optimize-tools`)
- Phase 3: Prompt optimizer proposes a child variant (when not `--optimize-tools`)
6. Add the child to the tree and repeat until all tests pass or max iterations reached.
When optimization is disabled (`--no-optimize`), only step 1 and 2 execute
(a single iteration).
This approach (inspired by [Learning to Self-Evolve](https://arxiv.org/abs/2603.18620))
prevents irrecoverable collapse from bad edits — UCB naturally backtracks to
high-scoring ancestors instead of following a linear chain.
When optimization is disabled (`--no-optimize`), only steps 2-4 execute
(a single iteration evaluating the root node).
---
@@ -65,6 +75,7 @@ Test suites are JSON files with this structure:
| `match_mode` | no | `"ordered_subset"` | How to match actual vs expected actions (see Scoring). |
| `max_turns` | no | `10` | Maximum conversation turns before stopping. |
| `n_runs` | no | suite default or 3 | Per-case override for number of runs. |
| `holdout` | no | `false` | If `true`, this case is evaluated but excluded from optimizer feedback. Used to measure progress without overfitting. |
### Expected Action Specs
@@ -135,6 +146,7 @@ deterministic, non-interactive execution suitable for automated testing.
| Stdout | Normal | Suppressed during execution |
| Tool logging | Display only | Structured `tool_call_log` |
| System prompt | Built-in developer prompt | Overridable via constructor |
| Cancellation | N/A | `_cancelled` event for timeout cleanup |
### NullUI
@@ -156,20 +168,34 @@ def send_headless(
Runs a complete multi-turn conversation:
1. Appends the user message.
2. Calls the model API (non-streaming).
3. If tool calls are returned, executes them (with stdout suppressed) and
2. Checks `_cancelled` event — stops if set (timeout cleanup).
3. Calls the model API (non-streaming).
4. If tool calls are returned, executes them (with stdout suppressed) and
logs each call to `self.tool_call_log`.
4. Repeats up to `max_turns` or until the model responds without tool calls.
5. Returns the tool call log: list of dicts with keys `tool`, `args`,
5. Repeats up to `max_turns` or until the model responds without tool calls.
6. Returns the tool call log: list of dicts with keys `tool`, `args`,
`result` (truncated to 500 chars), and `turn`.
Parallel tool calls are capped at 10 per turn to prevent degenerate repetition.
### Timeout and Cancellation
Each test runs in a `ThreadPoolExecutor(max_workers=1)` with a per-test
timeout (`--test-timeout`). Each attempt gets its own `OpenAI` client with
a matching httpx read timeout. On timeout, three layers of defense prevent
zombie connections:
1. **httpx timeout**: Per-request read timeout aborts the HTTP call and
releases the server slot.
2. **`_cancelled` event**: Prevents the orphan thread from starting new turns.
3. **`run_client.close()`**: Closes the connection pool to abort any
in-flight request.
### Retry Logic
`send_headless()` is called inside `_run_single_test()` with retry logic:
3 attempts with exponential backoff (sleep `2^attempt` seconds) on any
exception. This prevents transient API errors from poisoning eval scores.
exception. `TimeoutError` is re-raised immediately (no retry).
---
@@ -180,68 +206,175 @@ Each test case runs in isolation:
1. A fresh temp directory is created.
2. Setup files are written to the temp directory.
3. The working directory is changed to the temp directory.
4. A new `HeadlessSession` is created with the current developer prompt.
5. `send_headless()` runs the user prompt through the conversation loop.
6. The tool log is scored against expected actions.
7. The temp directory is cleaned up.
4. A per-attempt `OpenAI` client is created with httpx timeout matching `--test-timeout`.
5. A new `HeadlessSession` is created with the current developer prompt.
6. `send_headless()` runs the user prompt through the conversation loop.
7. The tool log is scored against expected actions.
8. The temp directory is cleaned up.
The memory database is also isolated per test (an ephemeral SQLite database
in the temp directory) so tests do not pollute each other or the user's
real memory store.
### Parallel Execution
With `--parallel N` (N > 1), tests run in a `ProcessPoolExecutor` with N
workers. Each subprocess creates its own `OpenAI` client. This is suitable
for remote API endpoints but will overwhelm local inference servers. The
default (`--parallel 1`) runs tests serially.
---
## Optimization Loop
## Model Roles
`run_optimization()` is the main entry point for iterative prompt optimization.
The eval pipeline uses up to five separate model roles, each independently
configurable. All roles inherit from the test model by default, with a
cascade chain:
```
test model (--base-url, --model)
└─ optimizer (--optimizer-*)
├─ observer (--observer-*)
├─ analyst (--analyst-*)
├─ diversifier (--diversifier-*)
└─ tool optimizer (--tool-optimizer-*)
```
| Role | Purpose | When it runs |
|------|---------|--------------|
| **Test** | The model being evaluated | Every iteration |
| **Analyst** | Diagnoses semantic failure patterns with tool use | When pass rate < 100% |
| **Optimizer** | Rewrites the developer prompt | Every iteration (unless `--optimize-tools`) |
| **Tool optimizer** | Rewrites tool descriptions | When `--optimize-tools` is set |
| **Observer** | Tunes the optimizer's strategy | Every 3 iterations |
| **Diversifier** | Generates prompt paraphrases | Once before the loop (when `--diversify N`) |
Typical setup: local model for test, Opus for analyst, Sonnet for
optimizer/observer/diversifier.
---
## Optimization Pipeline
### Flow
```
for iteration in 0..max_iterations:
1. Run all test cases n_runs times with current prompt
2. Score and aggregate results
3. Save intermediate results to JSON
4. If all tests pass -> stop
5. Every 3 iterations (at iteration 2, 5, 8, ...):
-> Observer reviews optimizer strategy
-> Reset prompt to best-performing iteration
6. Propose new prompt via optimizer model call
7. If prompt unchanged -> stop
8. Continue with new prompt
1. UCB select → pick the most promising tree node
2. Run all test cases n_runs times with selected node's prompt
3. Update node score (rolling mean) and visit count
4. Save intermediate results + tree state to JSON
5. If all tests pass → stop
6. Phase 1: Analyst diagnoses semantic failure patterns
7. Phase 2 (--optimize-tools only): Tool optimizer adjusts descriptions
8. Phase 3 (default only): Prompt optimizer proposes new prompt
9. Every 3 iterations: Observer tunes the optimizer's strategy
10. Add child node to tree (if prompt or tools changed)
```
### Prompt Proposal (`_propose_prompt_modification`)
### Phase 1: Analyst (`_run_analyst`)
Uses the model to rewrite the developer prompt based on test results:
A multi-turn agent with `math` (Python) and `bash` tools for computing
statistics. It receives per-case results with failure classifications and
produces a structured diagnosis:
- **Input**: Current prompt, test case definitions, per-case results with
actual vs expected tool sequences, and a history of the last 3 iterations.
- **Optimizer system prompt** (`OPTIMIZER_SYSTEM`): Instructs the model to
act as a text rewriter. Key guidance includes:
- Address critical failure modes (text-only responses, write_file vs edit_file,
unnecessary search before create, missing plan calls).
- Preserve phrasing that drives 100% pass rate on passing tests.
- Use direct imperative style with concrete tool call examples.
- Stay within 130% of original prompt length.
- **Output**: The rewritten prompt text (stripped of reasoning tags and code fences).
- **Failure patterns**: Shared root causes across failing cases
- **Success/failure contrast**: What distinguishes passing from failing cases
- **Consistency signals**: Systematic (0%), flaky (1-79%), marginal (80-99%)
- **Recommended fixes**: Priority-ordered patterns/examples to add or adjust
### Observer System (`_observe_and_update_optimizer`)
The analyst is instructed to frame fixes as patterns and examples, not
imperative rules — this feeds cleaner signal to the optimizer.
Every 3 iterations, a meta-level "observer" reviews the optimizer's strategy:
In `--optimize-tools` mode, the analyst receives the current tool descriptions
(with any overrides applied) and focuses on tool confusion and description
issues rather than system prompt patterns.
- Analyzes the iteration history: score trends, regressions, prompt length changes,
### Phase 2: Tool Optimizer (`_propose_tool_overrides`)
Runs when `--optimize-tools` is set. Receives the current tool descriptions,
confusion failures (where the model picked the wrong tool), and the analyst's
diagnosis. Returns a JSON override dict that modifies tool descriptions.
Overrides are validated against known tool names — only `description` and
`parameters` changes are accepted (no tool renaming at eval time).
After each iteration, changed descriptions are logged as old → new diffs
for easy visual inspection.
### Phase 3: Prompt Optimizer (`_propose_prompt_modification`)
Skipped in `--optimize-tools` mode. Receives the current prompt, test
results with per-case pass rates and deltas from the parent node, and the
analyst's diagnosis. Returns a rewritten prompt.
The optimizer is instructed to prefer patterns over rules — concrete tool
chain examples teach better than imperative directives like "ALWAYS" or
"NEVER." If the current prompt contains rule-heavy language, the optimizer
is guided to replace it with examples.
### Two Optimization Surfaces
The system supports alternating between two optimization surfaces:
1. **System prompt optimization** (default): Freeze tool descriptions,
optimize the developer prompt. Run until scores plateau.
2. **Tool description optimization** (`--optimize-tools`): Freeze the system
prompt, optimize tool descriptions only. Run until scores plateau.
Each surface lifts the floor for the other — tool description improvements
may unlock system prompt gains that weren't reachable before, and vice versa.
### Observer (`_observe_and_update_optimizer`)
Every 3 iterations, a meta-level observer reviews the optimizer's strategy:
- Analyzes iteration history: score trends, regressions, prompt length changes,
and diffs between iterations.
- Summarizes the optimizer's behavioral patterns (list style, header usage, length).
- Uses `OBSERVER_SYSTEM` to rewrite the optimizer's own system prompt.
- Detects whether the optimizer is producing rule-heavy or pattern-based output.
- Rewrites the optimizer's own system prompt to correct course.
- Rejects degenerate outputs (over 200% of input length).
- After updating the optimizer prompt, resets the developer prompt to the
best-performing iteration so far.
This two-level optimization (optimizer + observer) helps the system escape
local minima and adjust its rewriting strategy.
### Prompt Diversification
### Result Persistence
When `--diversify N` is set, the diversifier generates N paraphrased variants
of each test case's user prompt before the optimization loop. Each run cycles
through variants (round-robin), testing robustness across phrasings.
Variants can be cached back to the test suite JSON with `--save-variants`,
and auto-loaded on subsequent runs even without `--diversify`.
---
## Evolution Tree
The optimization maintains a tree of prompt variants (`EvolutionNode`), where
each node stores its prompt text, tool overrides, aggregated score, and visit
count. The root node (ID 0) contains the initial prompt.
**UCB1 selection**: Each iteration picks the node with the highest Upper
Confidence Bound score: `R_bar + C * sqrt(ln(N) / v)`, where `R_bar` is the
node's mean score, `N` is total visits across all nodes, `v` is the node's
visit count, and `C` is the exploration constant (`--explore-constant`,
default sqrt(2)). Unvisited nodes are always selected first.
### Holdout Cases
Test cases with `"holdout": true` are evaluated every iteration but excluded
from the optimizer's feedback. This prevents the optimizer from overfitting
to specific test cases. Node scores are computed from holdout cases only
(when present). If fewer than 2 non-holdout cases remain, holdout is disabled.
### Improvement-Based Feedback
The optimizer sees delta scores (`delta=+20%`) alongside absolute pass rates,
showing how each case improved relative to the parent node's evaluation. This
provides a cleaner signal than absolute scores alone — the optimizer can
distinguish beneficial edits from harmful ones regardless of starting point.
---
## Result Persistence
After each iteration, results are written to the output JSON file. The
structure is:
@@ -251,9 +384,15 @@ structure is:
"meta": {
"model": "model-name",
"base_url": "http://localhost:8000/v1",
"optimizer_model": "claude-opus-4-6",
"observer_model": "claude-opus-4-6",
"started": "2025-01-01T00:00:00",
"test_suite": "tests.json",
"n_runs_default": 3
"n_runs_default": 3,
"explore_constant": 1.414,
"holdout_ids": [],
"diversify": 10,
"prompt_variants": {"case_id": ["variant1", "variant2"]}
},
"iterations": [
{
@@ -261,7 +400,11 @@ structure is:
"prompt": "the developer prompt used",
"prompt_diff": null,
"optimizer_system": "the optimizer system prompt",
"analyst": "analyst diagnosis output",
"tool_overrides": {"bash": {"description": "..."}},
"timestamp": "2025-01-01T00:01:00",
"tree_node_id": 0,
"tree_child_id": 1,
"cases": {
"test_name": {
"runs": [
@@ -287,9 +430,21 @@ structure is:
"overall_pass_rate": 0.8,
"overall_avg_score": 0.87,
"json_dumps": 0,
"per_case_pass_rates": {"test_name": 1.0, ...}
"per_case_pass_rates": {"test_name": 1.0}
}
}
],
"tree": [
{
"node_id": 0,
"parent_id": null,
"prompt": "initial prompt",
"tool_overrides": {},
"score": 0.85,
"visit_count": 3,
"children": [1, 2],
"iteration": 0
}
]
}
```
@@ -306,26 +461,57 @@ turnstone-eval tests.json # evaluate + optimize
turnstone-eval tests.json --no-optimize # evaluate only (single iteration)
turnstone-eval tests.json --n-runs 5 --max-iter 10 # more thorough evaluation
turnstone-eval tests.json --prompt custom.txt # start from a custom prompt
turnstone-eval tests.json --optimize-tools # optimize tool descriptions only
turnstone-eval tests.json --diversify 10 # test with prompt variants
turnstone-eval tests.json -v # verbose per-turn logging
```
### Multi-model setup (local test model, cloud optimizer)
```
turnstone-eval tests.json \
--base-url http://localhost:8000/v1 \
--optimizer-base-url https://api.anthropic.com \
--optimizer-model claude-sonnet-4-6 \
--analyst-model claude-opus-4-6
```
### All Options
| Flag | Default | Description |
|---------------------|-------------------------------|-------------|
| `test_file` | (positional, required) | Path to test cases JSON file. |
| `--base-url` | `http://localhost:8000/v1` | API base URL. |
| `--model` | auto-detect | Model name. Auto-detected from the API if not specified. |
| `--prompt` | turnstone built-in prompt | Path to initial prompt text file. |
| `--n-runs` | from tests.json or 3 | Number of runs per test case. |
| `--max-iter` | 5 | Maximum optimization iterations. |
| `--no-optimize` | false | Run evaluation only (sets max-iter to 1). |
| `--temperature` | 0.7 | Sampling temperature. |
| `--max-tokens` | 32768 | Max completion tokens. |
| `--reasoning-effort` | `medium` | Reasoning effort: `low`, `medium`, or `high`. |
| `--context-window` | 131072 | Context window size. |
| `--output` | `eval_results.json` | Output results file path. |
| `-v`, `--verbose` | false | Show detailed per-turn logging (API calls, tool args, results). |
| Flag | Default | Description |
|-------------------------|----------------------------|-------------|
| `test_file` | (positional, required) | Path to test cases JSON file. |
| `--base-url` | `http://localhost:8000/v1` | API base URL for the test model. |
| `--model` | auto-detect | Model name. Auto-detected from the API if not specified. |
| `--prompt` | turnstone built-in prompt | Path to initial prompt text file. |
| `--n-runs` | from tests.json or 3 | Number of runs per test case. |
| `--max-iter` | 5 | Maximum optimization iterations. |
| `--no-optimize` | false | Run evaluation only (sets max-iter to 1). |
| `--temperature` | 0.7 | Sampling temperature. |
| `--max-tokens` | 32768 | Max completion tokens. |
| `--reasoning-effort` | `medium` | Reasoning effort: `low`, `medium`, or `high`. |
| `--context-window` | 131072 | Context window size. |
| `--output` | `eval_results.json` | Output results file path. |
| `-v`, `--verbose` | false | Show detailed per-turn logging. |
| `--explore-constant` | 1.414 (sqrt(2)) | UCB exploration constant C. |
| `--test-timeout` | 300 | Per-test timeout in seconds. |
| `--suite-timeout` | 0 (unlimited) | Total suite timeout in seconds. |
| `--no-fast-fail` | false | Disable early termination on all-zero initial runs. |
| `--parallel` | 1 (serial) | Parallel workers (0=auto, N=use N workers). |
| `--optimizer-model` | same as `--model` | Model for prompt optimization. |
| `--optimizer-base-url` | same as `--base-url` | Base URL for optimizer model. |
| `--observer-model` | same as optimizer | Model for meta-optimization (observer). |
| `--observer-base-url` | same as optimizer | Base URL for observer model. |
| `--analyst-model` | same as optimizer | Model for failure analysis. |
| `--analyst-base-url` | same as optimizer | Base URL for analyst model. |
| `--diversify` | 0 (disabled) | Generate N prompt variants per test case. |
| `--diversifier-model` | same as optimizer | Model for prompt diversification. |
| `--diversifier-base-url`| same as optimizer | Base URL for diversifier model. |
| `--save-variants` | false | Save generated variants back to test suite JSON. |
| `--optimize-tools` | false | Optimize tool descriptions only (freeze system prompt). |
| `--tool-optimizer-model` | same as optimizer | Model for tool description optimization. |
| `--tool-optimizer-base-url` | same as optimizer | Base URL for tool optimizer model. |
| `--save-tools` | false | Write optimized tool descriptions back to `turnstone/tools/*.json`. |
### Precedence for n_runs
+200
View File
@@ -0,0 +1,200 @@
# PgBouncer Connection Pooling
Turnstone cluster deployments share a single PostgreSQL instance across
all server nodes, bridge processes, and the console. Each process
maintains a small connection pool (2 base + 3 overflow = 5 max). At
scale this adds up — a 100-node cluster opens up to 500 connections,
and a 1000-node cluster up to 5,000.
PostgreSQL's default `max_connections` is 100, and each real connection
allocates ~510 MB of backend memory. PgBouncer sits between turnstone
and PostgreSQL, multiplexing thousands of lightweight client connections
down to a small number of real database connections.
---
## Why PgBouncer works well with turnstone
All turnstone database operations are short-burst queries: acquire a
connection, execute 13 statements, commit, release. No operation holds
a connection for more than a few milliseconds. This makes **transaction
pooling mode** ideal — PgBouncer assigns a real connection only for the
duration of each transaction, then returns it to the pool.
| Cluster size | Client connections (max) | PgBouncer server connections needed |
|--------------|------------------------|-------------------------------------|
| 10 nodes | 50 | 1020 |
| 100 nodes | 500 | 2040 |
| 500 nodes | 2,500 | 3060 |
| 1,000 nodes | 5,000 | 4080 |
The server connection count stays low because most client connections
are idle at any given moment.
---
## Docker Compose
Add PgBouncer between turnstone services and PostgreSQL:
```yaml
services:
pgbouncer:
image: bitnami/pgbouncer:latest
environment:
POSTGRESQL_HOST: postgres
POSTGRESQL_PORT: "5432"
POSTGRESQL_DATABASE: turnstone
POSTGRESQL_USERNAME: ${POSTGRES_USER:-turnstone}
POSTGRESQL_PASSWORD: ${POSTGRES_PASSWORD:?}
PGBOUNCER_POOL_MODE: transaction
PGBOUNCER_DEFAULT_POOL_SIZE: "40"
PGBOUNCER_MAX_CLIENT_CONN: "5000"
PGBOUNCER_MAX_DB_CONNECTIONS: "80"
PGBOUNCER_SERVER_IDLE_TIMEOUT: "300"
ports:
- "6432:6432"
networks:
- turnstone-net
depends_on:
postgres:
condition: service_healthy
healthcheck:
test: ["CMD", "pg_isready", "-h", "127.0.0.1", "-p", "6432"]
interval: 5s
timeout: 3s
retries: 5
```
Then point turnstone services at PgBouncer instead of PostgreSQL
directly by changing the `DATABASE_URL` (or `TURNSTONE_DB_URL`):
```bash
# Before (direct)
TURNSTONE_DB_URL=postgresql://turnstone:secret@postgres:5432/turnstone
# After (via PgBouncer)
TURNSTONE_DB_URL=postgresql://turnstone:secret@pgbouncer:6432/turnstone
```
---
## Helm / Kubernetes
Add a PgBouncer deployment or use a Helm chart like
[bitnami/pgbouncer](https://github.com/bitnami/charts/tree/main/bitnami/pgbouncer).
In `values.yaml`, point the database at PgBouncer:
```yaml
database:
backend: postgresql
external:
host: pgbouncer
port: 6432
database: turnstone
username: turnstone
existingSecret: turnstone-db-secret
```
PgBouncer configuration:
```yaml
pgbouncer:
poolMode: transaction
defaultPoolSize: 40
maxClientConn: 5000
maxDbConnections: 80
```
---
## Configuration reference
| PgBouncer setting | Recommended | Notes |
|-------------------|-------------|-------|
| `pool_mode` | `transaction` | Required — turnstone uses short-burst queries with no session state |
| `default_pool_size` | 40 | Real PostgreSQL connections per database. Start here, increase if you see `no more connections allowed` |
| `max_client_conn` | 5000 | Upper bound on client connections. Set to `cluster_nodes × 5` |
| `max_db_connections` | 80 | Hard cap on real connections to PostgreSQL. Keep below PG `max_connections` minus headroom for admin/monitoring |
| `server_idle_timeout` | 300 | Close idle server connections after 5 minutes |
| `server_lifetime` | 3600 | Recycle server connections after 1 hour |
On the PostgreSQL side:
| PostgreSQL setting | Recommended | Notes |
|--------------------|-------------|-------|
| `max_connections` | 100 | Default is fine — PgBouncer is the only client. Set higher than `max_db_connections` to leave room for admin connections |
| `shared_buffers` | 25% of RAM | Standard PostgreSQL tuning |
---
## Turnstone pool settings
Each turnstone process maintains its own SQLAlchemy connection pool to
PgBouncer (which then multiplexes to PostgreSQL):
| Environment variable | Default | Description |
|---------------------|---------|-------------|
| `TURNSTONE_DB_POOL_SIZE` | 2 | Base pool size per process |
| `TURNSTONE_DB_BACKEND` | sqlite | Set to `postgresql` for cluster deployments |
| `TURNSTONE_DB_URL` | — | Connection URL (point at PgBouncer, not PostgreSQL directly) |
The default pool of 2 + 3 overflow = 5 connections per process is
intentionally small to support large clusters. You should not need to
increase this — turnstone's database operations are all short-burst
context-managed queries that hold connections for milliseconds.
SQLAlchemy `pool_pre_ping` is enabled, so stale connections (e.g. after
PgBouncer restarts) are automatically detected and replaced.
---
## Monitoring
PgBouncer exposes stats via its admin console (connect to
PgBouncer port with user `pgbouncer`):
```sql
-- Active and waiting clients
SHOW POOLS;
-- Per-database stats
SHOW STATS;
-- Current client connections
SHOW CLIENTS;
```
Key metrics to watch:
- **`cl_active`** — clients with a server connection assigned. Should be
well below `max_db_connections`.
- **`cl_waiting`** — clients waiting for a server connection. Sustained
non-zero values mean you need more `default_pool_size`.
- **`sv_active`** — active server (PostgreSQL) connections. Should stay
below PostgreSQL `max_connections`.
---
## Troubleshooting
**"no more connections allowed (max_client_conn)"** — PgBouncer is
rejecting new client connections. Increase `max_client_conn` to match
your cluster size × 5.
**"no more connections allowed (max_db_connections)"** — PgBouncer
cannot open more connections to PostgreSQL. Increase
`max_db_connections` and ensure PostgreSQL `max_connections` is higher.
**Connections timing out on startup** — If all nodes start
simultaneously, the burst of initial connections (migrations, health
checks) can temporarily exceed the pool. PgBouncer queues excess
clients by default — this resolves itself within seconds.
**Prepared statements not supported** — PgBouncer in `transaction` mode
does not support prepared statements. Turnstone's SQLAlchemy layer does
not use server-side prepared statements by default, so this is not an
issue.
See also: [Docker deployment](docker.md) · [Security](security.md)
+35 -8
View File
@@ -457,14 +457,30 @@ without any database.
### Proxy auth forwarding
When the console proxies requests to server nodes (via `/node/{id}/...`
routes), it uses a dedicated **service proxy token** with
`aud: turnstone-server` and `write` scope. The user's console JWT
(which has `aud: turnstone-console`) is **not** forwarded — it would be
rejected by the server's audience validation.
routes), it mints a **short-lived user-scoped JWT** with
`aud: turnstone-server` carrying the real user's `user_id`, `scopes`,
and `permissions`. The user's console JWT (which has
`aud: turnstone-console`) is **not** forwarded directly — it would be
rejected by the server's audience validation. Instead, the console
re-signs a new JWT targeted at the server audience.
The proxy token is managed by a `ServiceTokenManager` that auto-rotates
1-hour JWTs, refreshing at 80% of lifetime. If `--auth-token` is
provided, that static token is used instead.
Each proxied request gets a fresh JWT (5-minute expiry). This ensures:
- **Audit attribution** — the upstream server records the real user in
`ctx_user_id` and audit events, not a generic service identity.
- **Scope narrowing** — a read-only console user's proxied request
carries only `read` scope, not the full `{read, write, approve}` set.
The server enforces this as defense in depth.
- **Permission forwarding** — granular RBAC permissions from the
console JWT are carried through to the server.
The JWT `src` claim is set to `"console-proxy"`, allowing servers to
distinguish proxied requests from direct logins in audit logs.
When no user context is available (auth disabled, or internal requests),
the proxy falls back to a `ServiceTokenManager` with service identity
`console-proxy` and full scopes. If `--auth-token` is provided, that
static token is used as a final fallback.
### Service-to-service authentication
@@ -475,7 +491,7 @@ auto-rotating JWTs when communicating with server nodes:
|---------|----------|-------|----------|---------|
| Bridge | `bridge` | `approve` | `turnstone-server` | Tool approval proxy, message relay |
| Console collector | `console-collector` | `read` | `turnstone-server` | Node health polling |
| Console proxy | `console-proxy` | `write` | `turnstone-server` | Proxied API calls |
| Console proxy (fallback) | `console-proxy` | `approve` | `turnstone-server` | Proxied API calls when no user context |
| Channel notify | `system` | `write` | `turnstone-channel` | Notification delivery to channel gateway |
Service tokens use 1-hour expiry with automatic refresh via
@@ -483,6 +499,17 @@ Service tokens use 1-hour expiry with automatic refresh via
httpx event hooks to ensure rotated tokens are picked up on SSE
reconnects.
### User identity in MQ-dispatched workstreams
When the console creates a workstream via MQ (the normal path), the
authenticated user's `user_id` is embedded in the
`CreateWorkstreamMessage`. The bridge forwards this `user_id` in the
HTTP payload when calling the server's `POST /v1/api/workstreams/new`.
The server accepts a `user_id` from the request body **only when the
caller is a trusted service** — identified by `token_source` matching
`bridge`, `console-proxy`, or `console`. Regular API callers cannot
override `user_id`; the server always uses their JWT identity.
Note that the channel gateway uses a distinct JWT audience
(`turnstone-channel`) from the server (`turnstone-server`) and console
(`turnstone-console`). A server-scoped JWT cannot authenticate to the
+5 -3
View File
@@ -51,7 +51,7 @@ connection, Redis, auth secrets, server bind address). These stay in
| Bridge identity | `[bridge]` | config.toml / env |
| Console bind | `[console]` | config.toml / env |
**ConfigStore settings** (~40 settings) are loaded from the database after
**ConfigStore settings** (48 settings) are loaded from the database after
storage initialization:
| Section | Settings |
@@ -60,10 +60,12 @@ storage initialization:
| `session` | instructions, retention_days, compact_max_tokens, auto_compact_pct |
| `tools` | timeout, truncation, agent_max_turns, skip_permissions, search, search_threshold, search_max_results |
| `server` | workstream_idle_timeout, max_workstreams |
| `cluster` | node_fan_out_limit, mcp_max_servers |
| `mcp` | config_path, refresh_interval, registry_url |
| `ratelimit` | enabled, requests_per_second, burst |
| `ratelimit` | enabled, requests_per_second, burst, trusted_proxies |
| `health` | backend_probe_interval, backend_probe_timeout, circuit_breaker_threshold, circuit_breaker_cooldown |
| `judge` | enabled, model, provider, base_url, api_key, confidence_threshold, max_context_ratio, timeout, read_only_tools |
| `judge` | enabled, model, provider, base_url, api_key, confidence_threshold, max_context_ratio, timeout, read_only_tools, output_guard, redact_secrets |
| `skills` | discovery_url |
| `memory` | relevance_k, fetch_limit, max_content, nudge_cooldown, nudges |
Settings are addressed by dotted key (e.g. `memory.relevance_k`). Each has a
+8 -3
View File
@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
[project]
name = "turnstone"
version = "0.8.4"
version = "0.8.8"
description = "Multi-node AI orchestration platform with tool use, agent routing, and cluster simulation."
readme = "README.md"
license = "BUSL-1.1"
@@ -51,8 +51,9 @@ console = ["redis>=7.2", "croniter>=3.0"]
sim = ["redis>=7.2"]
anthropic = ["anthropic>=0.39"]
postgres = ["psycopg[binary]>=3.2"]
ddg = ["ddgs>=9.0"]
discord = ["discord.py>=2.4", "redis>=7.2"]
all = ["turnstone[mq,console,sim,anthropic,postgres,discord]"]
all = ["turnstone[mq,console,sim,anthropic,postgres,discord,ddg]"]
[project.scripts]
turnstone = "turnstone.cli:main"
@@ -77,7 +78,7 @@ include = [
"turnstone/console/static/*.js",
"turnstone/shared_static/*.css",
"turnstone/shared_static/*.js",
"turnstone/shared_static/katex-0.16.38/**/*",
"turnstone/shared_static/katex-0.16.42/**/*",
"turnstone/shared_static/hljs-11.11.1/**/*",
"turnstone/shared_static/mermaid-11.13.0/**/*",
"turnstone/sdk/py.typed",
@@ -164,6 +165,10 @@ ignore_missing_imports = true
module = ["frontmatter", "frontmatter.*"]
ignore_missing_imports = true
[[tool.mypy.overrides]]
module = ["ddgs", "ddgs.*"]
ignore_missing_imports = true
[[tool.mypy.overrides]]
module = ["turnstone.channels.discord.*"]
disallow_subclassing_any = false
+23 -1
View File
@@ -28,9 +28,19 @@ usage() {
LIB="$1"
VERSION="$2"
# Detect current version from pyproject.toml
# Detect current version from the filesystem (not pyproject.toml, which
# Renovate may have already updated). Falls back to pyproject.toml if
# no directory is found.
detect_old_version() {
local pattern="$1"
# Look for existing directory: e.g. turnstone/shared_static/katex-0.16.38
local dir
dir=$(find "${STATIC_DIR}" -maxdepth 1 -type d -name "${pattern}-*" | head -1)
if [[ -n "$dir" ]]; then
basename "$dir" | sed "s/${pattern}-//"
return
fi
# Fallback to pyproject.toml
grep -oE "${pattern}-[0-9.]+" pyproject.toml | head -1 | sed "s/${pattern}-//"
}
@@ -51,9 +61,19 @@ update_refs() {
done
}
check_same_version() {
if [[ "$1" == "$2" ]]; then
echo "ERROR: Old version ($1) == new version ($2). Nothing to update."
echo "If the old directory was already removed, re-download with:"
echo " rm -rf ${STATIC_DIR}/${3}-${1} && $0 $3 $2"
exit 1
fi
}
case "$LIB" in
katex)
OLD_VERSION=$(detect_old_version "katex")
check_same_version "$OLD_VERSION" "$VERSION" "katex"
OLD_DIR="${STATIC_DIR}/katex-${OLD_VERSION}"
NEW_DIR="${STATIC_DIR}/katex-${VERSION}"
@@ -87,6 +107,7 @@ case "$LIB" in
hljs)
OLD_VERSION=$(detect_old_version "hljs")
check_same_version "$OLD_VERSION" "$VERSION" "hljs"
OLD_DIR="${STATIC_DIR}/hljs-${OLD_VERSION}"
NEW_DIR="${STATIC_DIR}/hljs-${VERSION}"
@@ -107,6 +128,7 @@ case "$LIB" in
mermaid)
OLD_VERSION=$(detect_old_version "mermaid")
check_same_version "$OLD_VERSION" "$VERSION" "mermaid"
OLD_DIR="${STATIC_DIR}/mermaid-${OLD_VERSION}"
NEW_DIR="${STATIC_DIR}/mermaid-${VERSION}"
+22
View File
@@ -6884,6 +6884,11 @@
"title": "Enabled",
"type": "boolean"
},
"priority": {
"default": 0,
"title": "Priority",
"type": "integer"
},
"allowed_tools": {
"default": "[]",
"title": "Allowed Tools",
@@ -7119,6 +7124,11 @@
"title": "Enabled",
"type": "boolean"
},
"priority": {
"default": 0,
"title": "Priority",
"type": "integer"
},
"allowed_tools": {
"default": "[]",
"title": "Allowed Tools",
@@ -7372,6 +7382,18 @@
"default": null,
"title": "Enabled"
},
"priority": {
"anyOf": [
{
"type": "integer"
},
{
"type": "null"
}
],
"default": null,
"title": "Priority"
},
"allowed_tools": {
"anyOf": [
{
+131 -142
View File
@@ -9,14 +9,14 @@
"version": "0.3.0",
"license": "BUSL-1.1",
"devDependencies": {
"typescript": "^5.4",
"typescript": "^6.0.0",
"vitest": "^4.1"
}
},
"node_modules/@emnapi/core": {
"version": "1.9.0",
"resolved": "https://registry.npmjs.org/@emnapi/core/-/core-1.9.0.tgz",
"integrity": "sha512-0DQ98G9ZQZOxfUcQn1waV2yS8aWdZ6kJMbYCJB3oUBecjWYO1fqJ+a1DRfPF3O5JEkwqwP1A9QEN/9mYm2Yd0w==",
"version": "1.9.1",
"resolved": "https://registry.npmjs.org/@emnapi/core/-/core-1.9.1.tgz",
"integrity": "sha512-mukuNALVsoix/w1BJwFzwXBN/dHeejQtuVzcDsfOEsdpCumXb/E9j8w11h5S54tT1xhifGfbbSm/ICrObRb3KA==",
"dev": true,
"license": "MIT",
"optional": true,
@@ -26,9 +26,9 @@
}
},
"node_modules/@emnapi/runtime": {
"version": "1.9.0",
"resolved": "https://registry.npmjs.org/@emnapi/runtime/-/runtime-1.9.0.tgz",
"integrity": "sha512-QN75eB0IH2ywSpRpNddCRfQIhmJYBCJ1x5Lb3IscKAL8bMnVAKnRg8dCoXbHzVLLH7P38N2Z3mtulB7W0J0FKw==",
"version": "1.9.1",
"resolved": "https://registry.npmjs.org/@emnapi/runtime/-/runtime-1.9.1.tgz",
"integrity": "sha512-VYi5+ZVLhpgK4hQ0TAjiQiZ6ol0oe4mBx7mVv7IflsiEp0OWoVsp/+f9Vc1hOhE0TtkORVrI1GvzyreqpgWtkA==",
"dev": true,
"license": "MIT",
"optional": true,
@@ -71,20 +71,10 @@
"url": "https://github.com/sponsors/Brooooooklyn"
}
},
"node_modules/@oxc-project/runtime": {
"version": "0.115.0",
"resolved": "https://registry.npmjs.org/@oxc-project/runtime/-/runtime-0.115.0.tgz",
"integrity": "sha512-Rg8Wlt5dCbXhQnsXPrkOjL1DTSvXLgb2R/KYfnf1/K+R0k6UMLEmbQXPM+kwrWqSmWA2t0B1EtHy2/3zikQpvQ==",
"dev": true,
"license": "MIT",
"engines": {
"node": "^20.19.0 || >=22.12.0"
}
},
"node_modules/@oxc-project/types": {
"version": "0.115.0",
"resolved": "https://registry.npmjs.org/@oxc-project/types/-/types-0.115.0.tgz",
"integrity": "sha512-4n91DKnebUS4yjUHl2g3/b2T+IUdCfmoZGhmwsovZCDaJSs+QkVAM+0AqqTxHSsHfeiMuueT75cZaZcT/m0pSw==",
"version": "0.120.0",
"resolved": "https://registry.npmjs.org/@oxc-project/types/-/types-0.120.0.tgz",
"integrity": "sha512-k1YNu55DuvAip/MGE1FTsIuU3FUCn6v/ujG9V7Nq5Df/kX2CWb13hhwD0lmJGMGqE+bE1MXvv9SZVnMzEXlWcg==",
"dev": true,
"license": "MIT",
"funding": {
@@ -92,9 +82,9 @@
}
},
"node_modules/@rolldown/binding-android-arm64": {
"version": "1.0.0-rc.9",
"resolved": "https://registry.npmjs.org/@rolldown/binding-android-arm64/-/binding-android-arm64-1.0.0-rc.9.tgz",
"integrity": "sha512-lcJL0bN5hpgJfSIz/8PIf02irmyL43P+j1pTCfbD1DbLkmGRuFIA4DD3B3ZOvGqG0XiVvRznbKtN0COQVaKUTg==",
"version": "1.0.0-rc.10",
"resolved": "https://registry.npmjs.org/@rolldown/binding-android-arm64/-/binding-android-arm64-1.0.0-rc.10.tgz",
"integrity": "sha512-jOHxwXhxmFKuXztiu1ORieJeTbx5vrTkcOkkkn2d35726+iwhrY1w/+nYY/AGgF12thg33qC3R1LMBF5tHTZHg==",
"cpu": [
"arm64"
],
@@ -109,9 +99,9 @@
}
},
"node_modules/@rolldown/binding-darwin-arm64": {
"version": "1.0.0-rc.9",
"resolved": "https://registry.npmjs.org/@rolldown/binding-darwin-arm64/-/binding-darwin-arm64-1.0.0-rc.9.tgz",
"integrity": "sha512-J7Zk3kLYFsLtuH6U+F4pS2sYVzac0qkjcO5QxHS7OS7yZu2LRs+IXo+uvJ/mvpyUljDJ3LROZPoQfgBIpCMhdQ==",
"version": "1.0.0-rc.10",
"resolved": "https://registry.npmjs.org/@rolldown/binding-darwin-arm64/-/binding-darwin-arm64-1.0.0-rc.10.tgz",
"integrity": "sha512-gED05Teg/vtTZbIJBc4VNMAxAFDUPkuO/rAIyyxZjTj1a1/s6z5TII/5yMGZ0uLRCifEtwUQn8OlYzuYc0m70w==",
"cpu": [
"arm64"
],
@@ -126,9 +116,9 @@
}
},
"node_modules/@rolldown/binding-darwin-x64": {
"version": "1.0.0-rc.9",
"resolved": "https://registry.npmjs.org/@rolldown/binding-darwin-x64/-/binding-darwin-x64-1.0.0-rc.9.tgz",
"integrity": "sha512-iwtmmghy8nhfRGeNAIltcNXzD0QMNaaA5U/NyZc1Ia4bxrzFByNMDoppoC+hl7cDiUq5/1CnFthpT9n+UtfFyg==",
"version": "1.0.0-rc.10",
"resolved": "https://registry.npmjs.org/@rolldown/binding-darwin-x64/-/binding-darwin-x64-1.0.0-rc.10.tgz",
"integrity": "sha512-rI15NcM1mA48lqrIxVkHfAqcyFLcQwyXWThy+BQ5+mkKKPvSO26ir+ZDp36AgYoYVkqvMcdS8zOE6SeBsR9e8A==",
"cpu": [
"x64"
],
@@ -143,9 +133,9 @@
}
},
"node_modules/@rolldown/binding-freebsd-x64": {
"version": "1.0.0-rc.9",
"resolved": "https://registry.npmjs.org/@rolldown/binding-freebsd-x64/-/binding-freebsd-x64-1.0.0-rc.9.tgz",
"integrity": "sha512-DLFYI78SCiZr5VvdEplsVC2Vx53lnA4/Ga5C65iyldMVaErr86aiqCoNBLl92PXPfDtUYjUh+xFFor40ueNs4Q==",
"version": "1.0.0-rc.10",
"resolved": "https://registry.npmjs.org/@rolldown/binding-freebsd-x64/-/binding-freebsd-x64-1.0.0-rc.10.tgz",
"integrity": "sha512-XZRXHdTa+4ME1MuDVp021+doQ+z6Ei4CCFmNc5/sKbqb8YmkiJdj8QKlV3rCI0AJtAeSB5n0WGPuJWNL9p/L2w==",
"cpu": [
"x64"
],
@@ -160,9 +150,9 @@
}
},
"node_modules/@rolldown/binding-linux-arm-gnueabihf": {
"version": "1.0.0-rc.9",
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-arm-gnueabihf/-/binding-linux-arm-gnueabihf-1.0.0-rc.9.tgz",
"integrity": "sha512-CsjTmTwd0Hri6iTw/DRMK7kOZ7FwAkrO4h8YWKoX/kcj833e4coqo2wzIFywtch/8Eb5enQ/lwLM7w6JX1W5RQ==",
"version": "1.0.0-rc.10",
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-arm-gnueabihf/-/binding-linux-arm-gnueabihf-1.0.0-rc.10.tgz",
"integrity": "sha512-R0SQMRluISSLzFE20sPWYHVmJdDQnRyc/FzSCN72BqQmh2SOZUFG+N3/vBZpR4C6WpEUVYJLrYUXaj43sJsNLA==",
"cpu": [
"arm"
],
@@ -177,9 +167,9 @@
}
},
"node_modules/@rolldown/binding-linux-arm64-gnu": {
"version": "1.0.0-rc.9",
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-arm64-gnu/-/binding-linux-arm64-gnu-1.0.0-rc.9.tgz",
"integrity": "sha512-2x9O2JbSPxpxMDhP9Z74mahAStibTlrBMW0520+epJH5sac7/LwZW5Bmg/E6CXuEF53JJFW509uP+lSedaUNxg==",
"version": "1.0.0-rc.10",
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-arm64-gnu/-/binding-linux-arm64-gnu-1.0.0-rc.10.tgz",
"integrity": "sha512-Y1reMrV/o+cwpduYhJuOE3OMKx32RMYCidf14y+HssARRmhDuWXJ4yVguDg2R/8SyyGNo+auzz64LnPK9Hq6jg==",
"cpu": [
"arm64"
],
@@ -194,9 +184,9 @@
}
},
"node_modules/@rolldown/binding-linux-arm64-musl": {
"version": "1.0.0-rc.9",
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-arm64-musl/-/binding-linux-arm64-musl-1.0.0-rc.9.tgz",
"integrity": "sha512-JA1QRW31ogheAIRhIg9tjMfsYbglXXYGNPLdPEYrwFxdbkQCAzvpSCSHCDWNl4hTtrol8WeboCSEpjdZK8qrCg==",
"version": "1.0.0-rc.10",
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-arm64-musl/-/binding-linux-arm64-musl-1.0.0-rc.10.tgz",
"integrity": "sha512-vELN+HNb2IzuzSBUOD4NHmP9yrGwl1DVM29wlQvx1OLSclL0NgVWnVDKl/8tEks79EFek/kebQKnNJkIAA4W2g==",
"cpu": [
"arm64"
],
@@ -211,9 +201,9 @@
}
},
"node_modules/@rolldown/binding-linux-ppc64-gnu": {
"version": "1.0.0-rc.9",
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-ppc64-gnu/-/binding-linux-ppc64-gnu-1.0.0-rc.9.tgz",
"integrity": "sha512-aOKU9dJheda8Kj8Y3w9gnt9QFOO+qKPAl8SWd7JPHP+Cu0EuDAE5wokQubLzIDQWg2myXq2XhTpOVS07qqvT+w==",
"version": "1.0.0-rc.10",
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-ppc64-gnu/-/binding-linux-ppc64-gnu-1.0.0-rc.10.tgz",
"integrity": "sha512-ZqrufYTgzxbHwpqOjzSsb0UV/aV2TFIY5rP8HdsiPTv/CuAgCRjM6s9cYFwQ4CNH+hf9Y4erHW1GjZuZ7WoI7w==",
"cpu": [
"ppc64"
],
@@ -228,9 +218,9 @@
}
},
"node_modules/@rolldown/binding-linux-s390x-gnu": {
"version": "1.0.0-rc.9",
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-s390x-gnu/-/binding-linux-s390x-gnu-1.0.0-rc.9.tgz",
"integrity": "sha512-OalO94fqj7IWRn3VdXWty75jC5dk4C197AWEuMhIpvVv2lw9fiPhud0+bW2ctCxb3YoBZor71QHbY+9/WToadA==",
"version": "1.0.0-rc.10",
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-s390x-gnu/-/binding-linux-s390x-gnu-1.0.0-rc.10.tgz",
"integrity": "sha512-gSlmVS1FZJSRicA6IyjoRoKAFK7IIHBs7xJuHRSmjImqk3mPPWbR7RhbnfH2G6bcmMEllCt2vQ/7u9e6bBnByg==",
"cpu": [
"s390x"
],
@@ -245,9 +235,9 @@
}
},
"node_modules/@rolldown/binding-linux-x64-gnu": {
"version": "1.0.0-rc.9",
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-x64-gnu/-/binding-linux-x64-gnu-1.0.0-rc.9.tgz",
"integrity": "sha512-cVEl1vZtBsBZna3YMjGXNvnYYrOJ7RzuWvZU0ffvJUexWkukMaDuGhUXn0rjnV0ptzGVkvc+vW9Yqy6h8YX4pg==",
"version": "1.0.0-rc.10",
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-x64-gnu/-/binding-linux-x64-gnu-1.0.0-rc.10.tgz",
"integrity": "sha512-eOCKUpluKgfObT2pHjztnaWEIbUabWzk3qPZ5PuacuPmr4+JtQG4k2vGTY0H15edaTnicgU428XW/IH6AimcQw==",
"cpu": [
"x64"
],
@@ -262,9 +252,9 @@
}
},
"node_modules/@rolldown/binding-linux-x64-musl": {
"version": "1.0.0-rc.9",
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-x64-musl/-/binding-linux-x64-musl-1.0.0-rc.9.tgz",
"integrity": "sha512-UzYnKCIIc4heAKgI4PZ3dfBGUZefGCJ1TPDuLHoCzgrMYPb5Rv6TLFuYtyM4rWyHM7hymNdsg5ik2C+UD9VDbA==",
"version": "1.0.0-rc.10",
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-x64-musl/-/binding-linux-x64-musl-1.0.0-rc.10.tgz",
"integrity": "sha512-Xdf2jQbfQowJnLcgYfD/m0Uu0Qj5OdxKallD78/IPPfzaiaI4KRAwZzHcKQ4ig1gtg1SuzC7jovNiM2TzQsBXA==",
"cpu": [
"x64"
],
@@ -279,9 +269,9 @@
}
},
"node_modules/@rolldown/binding-openharmony-arm64": {
"version": "1.0.0-rc.9",
"resolved": "https://registry.npmjs.org/@rolldown/binding-openharmony-arm64/-/binding-openharmony-arm64-1.0.0-rc.9.tgz",
"integrity": "sha512-+6zoiF+RRyf5cdlFQP7nm58mq7+/2PFaY2DNQeD4B87N36JzfF/l9mdBkkmTvSYcYPE8tMh/o3cRlsx1ldLfog==",
"version": "1.0.0-rc.10",
"resolved": "https://registry.npmjs.org/@rolldown/binding-openharmony-arm64/-/binding-openharmony-arm64-1.0.0-rc.10.tgz",
"integrity": "sha512-o1hYe8hLi1EY6jgPFyxQgQ1wcycX+qz8eEbVmot2hFkgUzPxy9+kF0u0NIQBeDq+Mko47AkaFFaChcvZa9UX9Q==",
"cpu": [
"arm64"
],
@@ -296,9 +286,9 @@
}
},
"node_modules/@rolldown/binding-wasm32-wasi": {
"version": "1.0.0-rc.9",
"resolved": "https://registry.npmjs.org/@rolldown/binding-wasm32-wasi/-/binding-wasm32-wasi-1.0.0-rc.9.tgz",
"integrity": "sha512-rgFN6sA/dyebil3YTlL2evvi/M+ivhfnyxec7AccTpRPccno/rPoNlqybEZQBkcbZu8Hy+eqNJCqfBR8P7Pg8g==",
"version": "1.0.0-rc.10",
"resolved": "https://registry.npmjs.org/@rolldown/binding-wasm32-wasi/-/binding-wasm32-wasi-1.0.0-rc.10.tgz",
"integrity": "sha512-Ugv9o7qYJudqQO5Y5y2N2SOo6S4WiqiNOpuQyoPInnhVzCY+wi/GHltcLHypG9DEUYMB0iTB/huJrpadiAcNcA==",
"cpu": [
"wasm32"
],
@@ -313,9 +303,9 @@
}
},
"node_modules/@rolldown/binding-win32-arm64-msvc": {
"version": "1.0.0-rc.9",
"resolved": "https://registry.npmjs.org/@rolldown/binding-win32-arm64-msvc/-/binding-win32-arm64-msvc-1.0.0-rc.9.tgz",
"integrity": "sha512-lHVNUG/8nlF1IQk1C0Ci574qKYyty2goMiPlRqkC5R+3LkXDkL5Dhx8ytbxq35m+pkHVIvIxviD+TWLdfeuadA==",
"version": "1.0.0-rc.10",
"resolved": "https://registry.npmjs.org/@rolldown/binding-win32-arm64-msvc/-/binding-win32-arm64-msvc-1.0.0-rc.10.tgz",
"integrity": "sha512-7UODQb4fQUNT/vmgDZBl3XOBAIOutP5R3O/rkxg0aLfEGQ4opbCgU5vOw/scPe4xOqBwL9fw7/RP1vAMZ6QlAQ==",
"cpu": [
"arm64"
],
@@ -330,9 +320,9 @@
}
},
"node_modules/@rolldown/binding-win32-x64-msvc": {
"version": "1.0.0-rc.9",
"resolved": "https://registry.npmjs.org/@rolldown/binding-win32-x64-msvc/-/binding-win32-x64-msvc-1.0.0-rc.9.tgz",
"integrity": "sha512-G0oA4+w1iY5AGi5HcDTxWsoxF509hrFIPB2rduV5aDqS9FtDg1CAfa7V34qImbjfhIcA8C+RekocJZA96EarwQ==",
"version": "1.0.0-rc.10",
"resolved": "https://registry.npmjs.org/@rolldown/binding-win32-x64-msvc/-/binding-win32-x64-msvc-1.0.0-rc.10.tgz",
"integrity": "sha512-PYxKHMVHOb5NJuDL53vBUl1VwUjymDcYI6rzpIni0C9+9mTiJedvUxSk7/RPp7OOAm3v+EjgMu9bIy3N6b408w==",
"cpu": [
"x64"
],
@@ -347,9 +337,9 @@
}
},
"node_modules/@rolldown/pluginutils": {
"version": "1.0.0-rc.9",
"resolved": "https://registry.npmjs.org/@rolldown/pluginutils/-/pluginutils-1.0.0-rc.9.tgz",
"integrity": "sha512-w6oiRWgEBl04QkFZgmW+jnU1EC9b57Oihi2ot3HNWIQRqgHp5PnYDia5iZ5FF7rpa4EQdiqMDXjlqKGXBhsoXw==",
"version": "1.0.0-rc.10",
"resolved": "https://registry.npmjs.org/@rolldown/pluginutils/-/pluginutils-1.0.0-rc.10.tgz",
"integrity": "sha512-UkVDEFk1w3mveXeKgaTuYfKWtPbvgck1dT8TUG3bnccrH0XtLTuAyfCoks4Q/M5ZGToSVJTIQYCzy2g/atAOeg==",
"dev": true,
"license": "MIT"
},
@@ -397,16 +387,16 @@
"license": "MIT"
},
"node_modules/@vitest/expect": {
"version": "4.1.0",
"resolved": "https://registry.npmjs.org/@vitest/expect/-/expect-4.1.0.tgz",
"integrity": "sha512-EIxG7k4wlWweuCLG9Y5InKFwpMEOyrMb6ZJ1ihYu02LVj/bzUwn2VMU+13PinsjRW75XnITeFrQBMH5+dLvCDA==",
"version": "4.1.1",
"resolved": "https://registry.npmjs.org/@vitest/expect/-/expect-4.1.1.tgz",
"integrity": "sha512-xAV0fqBTk44Rn6SjJReEQkHP3RrqbJo6JQ4zZ7/uVOiJZRarBtblzrOfFIZeYUrukp2YD6snZG6IBqhOoHTm+A==",
"dev": true,
"license": "MIT",
"dependencies": {
"@standard-schema/spec": "^1.1.0",
"@types/chai": "^5.2.2",
"@vitest/spy": "4.1.0",
"@vitest/utils": "4.1.0",
"@vitest/spy": "4.1.1",
"@vitest/utils": "4.1.1",
"chai": "^6.2.2",
"tinyrainbow": "^3.0.3"
},
@@ -415,13 +405,13 @@
}
},
"node_modules/@vitest/mocker": {
"version": "4.1.0",
"resolved": "https://registry.npmjs.org/@vitest/mocker/-/mocker-4.1.0.tgz",
"integrity": "sha512-evxREh+Hork43+Y4IOhTo+h5lGmVRyjqI739Rz4RlUPqwrkFFDF6EMvOOYjTx4E8Tl6gyCLRL8Mu7Ry12a13Tw==",
"version": "4.1.1",
"resolved": "https://registry.npmjs.org/@vitest/mocker/-/mocker-4.1.1.tgz",
"integrity": "sha512-h3BOylsfsCLPeceuCPAAJ+BvNwSENgJa4hXoXu4im0bs9Lyp4URc4JYK4pWLZ4pG/UQn7AT92K6IByi6rE6g3A==",
"dev": true,
"license": "MIT",
"dependencies": {
"@vitest/spy": "4.1.0",
"@vitest/spy": "4.1.1",
"estree-walker": "^3.0.3",
"magic-string": "^0.30.21"
},
@@ -430,7 +420,7 @@
},
"peerDependencies": {
"msw": "^2.4.9",
"vite": "^6.0.0 || ^7.0.0 || ^8.0.0-0"
"vite": "^6.0.0 || ^7.0.0 || ^8.0.0"
},
"peerDependenciesMeta": {
"msw": {
@@ -442,9 +432,9 @@
}
},
"node_modules/@vitest/pretty-format": {
"version": "4.1.0",
"resolved": "https://registry.npmjs.org/@vitest/pretty-format/-/pretty-format-4.1.0.tgz",
"integrity": "sha512-3RZLZlh88Ib0J7NQTRATfc/3ZPOnSUn2uDBUoGNn5T36+bALixmzphN26OUD3LRXWkJu4H0s5vvUeqBiw+kS0A==",
"version": "4.1.1",
"resolved": "https://registry.npmjs.org/@vitest/pretty-format/-/pretty-format-4.1.1.tgz",
"integrity": "sha512-GM+TEQN5WhOygr1lp7skeVjdLPqqWMHsfzXrcHAqZJi/lIVh63H0kaRCY8MDhNWikx19zBUK8ceaLB7X5AH9NQ==",
"dev": true,
"license": "MIT",
"dependencies": {
@@ -455,13 +445,13 @@
}
},
"node_modules/@vitest/runner": {
"version": "4.1.0",
"resolved": "https://registry.npmjs.org/@vitest/runner/-/runner-4.1.0.tgz",
"integrity": "sha512-Duvx2OzQ7d6OjchL+trw+aSrb9idh7pnNfxrklo14p3zmNL4qPCDeIJAK+eBKYjkIwG96Bc6vYuxhqDXQOWpoQ==",
"version": "4.1.1",
"resolved": "https://registry.npmjs.org/@vitest/runner/-/runner-4.1.1.tgz",
"integrity": "sha512-f7+FPy75vN91QGWsITueq0gedwUZy1fLtHOCMeQpjs8jTekAHeKP80zfDEnhrleviLHzVSDXIWuCIOFn3D3f8A==",
"dev": true,
"license": "MIT",
"dependencies": {
"@vitest/utils": "4.1.0",
"@vitest/utils": "4.1.1",
"pathe": "^2.0.3"
},
"funding": {
@@ -469,14 +459,14 @@
}
},
"node_modules/@vitest/snapshot": {
"version": "4.1.0",
"resolved": "https://registry.npmjs.org/@vitest/snapshot/-/snapshot-4.1.0.tgz",
"integrity": "sha512-0Vy9euT1kgsnj1CHttwi9i9o+4rRLEaPRSOJ5gyv579GJkNpgJK+B4HSv/rAWixx2wdAFci1X4CEPjiu2bXIMg==",
"version": "4.1.1",
"resolved": "https://registry.npmjs.org/@vitest/snapshot/-/snapshot-4.1.1.tgz",
"integrity": "sha512-kMVSgcegWV2FibXEx9p9WIKgje58lcTbXgnJixfcg15iK8nzCXhmalL0ZLtTWLW9PH1+1NEDShiFFedB3tEgWg==",
"dev": true,
"license": "MIT",
"dependencies": {
"@vitest/pretty-format": "4.1.0",
"@vitest/utils": "4.1.0",
"@vitest/pretty-format": "4.1.1",
"@vitest/utils": "4.1.1",
"magic-string": "^0.30.21",
"pathe": "^2.0.3"
},
@@ -485,9 +475,9 @@
}
},
"node_modules/@vitest/spy": {
"version": "4.1.0",
"resolved": "https://registry.npmjs.org/@vitest/spy/-/spy-4.1.0.tgz",
"integrity": "sha512-pz77k+PgNpyMDv2FV6qmk5ZVau6c3R8HC8v342T2xlFxQKTrSeYw9waIJG8KgV9fFwAtTu4ceRzMivPTH6wSxw==",
"version": "4.1.1",
"resolved": "https://registry.npmjs.org/@vitest/spy/-/spy-4.1.1.tgz",
"integrity": "sha512-6Ti/KT5OVaiupdIZEuZN7l3CZcR0cxnxt70Z0//3CtwgObwA6jZhmVBA3yrXSVN3gmwjgd7oDNLlsXz526gpRA==",
"dev": true,
"license": "MIT",
"funding": {
@@ -495,13 +485,13 @@
}
},
"node_modules/@vitest/utils": {
"version": "4.1.0",
"resolved": "https://registry.npmjs.org/@vitest/utils/-/utils-4.1.0.tgz",
"integrity": "sha512-XfPXT6a8TZY3dcGY8EdwsBulFCIw+BeeX0RZn2x/BtiY/75YGh8FeWGG8QISN/WhaqSrE2OrlDgtF8q5uhOTmw==",
"version": "4.1.1",
"resolved": "https://registry.npmjs.org/@vitest/utils/-/utils-4.1.1.tgz",
"integrity": "sha512-cNxAlaB3sHoCdL6pj6yyUXv9Gry1NHNg0kFTXdvSIZXLHsqKH7chiWOkwJ5s5+d/oMwcoG9T0bKU38JZWKusrQ==",
"dev": true,
"license": "MIT",
"dependencies": {
"@vitest/pretty-format": "4.1.0",
"@vitest/pretty-format": "4.1.1",
"convert-source-map": "^2.0.0",
"tinyrainbow": "^3.0.3"
},
@@ -964,14 +954,14 @@
}
},
"node_modules/rolldown": {
"version": "1.0.0-rc.9",
"resolved": "https://registry.npmjs.org/rolldown/-/rolldown-1.0.0-rc.9.tgz",
"integrity": "sha512-9EbgWge7ZH+yqb4d2EnELAntgPTWbfL8ajiTW+SyhJEC4qhBbkCKbqFV4Ge4zmu5ziQuVbWxb/XwLZ+RIO7E8Q==",
"version": "1.0.0-rc.10",
"resolved": "https://registry.npmjs.org/rolldown/-/rolldown-1.0.0-rc.10.tgz",
"integrity": "sha512-q7j6vvarRFmKpgJUT8HCAUljkgzEp4LAhPlJUvQhA5LA1SUL36s5QCysMutErzL3EbNOZOkoziSx9iZC4FddKA==",
"dev": true,
"license": "MIT",
"dependencies": {
"@oxc-project/types": "=0.115.0",
"@rolldown/pluginutils": "1.0.0-rc.9"
"@oxc-project/types": "=0.120.0",
"@rolldown/pluginutils": "1.0.0-rc.10"
},
"bin": {
"rolldown": "bin/cli.mjs"
@@ -980,21 +970,21 @@
"node": "^20.19.0 || >=22.12.0"
},
"optionalDependencies": {
"@rolldown/binding-android-arm64": "1.0.0-rc.9",
"@rolldown/binding-darwin-arm64": "1.0.0-rc.9",
"@rolldown/binding-darwin-x64": "1.0.0-rc.9",
"@rolldown/binding-freebsd-x64": "1.0.0-rc.9",
"@rolldown/binding-linux-arm-gnueabihf": "1.0.0-rc.9",
"@rolldown/binding-linux-arm64-gnu": "1.0.0-rc.9",
"@rolldown/binding-linux-arm64-musl": "1.0.0-rc.9",
"@rolldown/binding-linux-ppc64-gnu": "1.0.0-rc.9",
"@rolldown/binding-linux-s390x-gnu": "1.0.0-rc.9",
"@rolldown/binding-linux-x64-gnu": "1.0.0-rc.9",
"@rolldown/binding-linux-x64-musl": "1.0.0-rc.9",
"@rolldown/binding-openharmony-arm64": "1.0.0-rc.9",
"@rolldown/binding-wasm32-wasi": "1.0.0-rc.9",
"@rolldown/binding-win32-arm64-msvc": "1.0.0-rc.9",
"@rolldown/binding-win32-x64-msvc": "1.0.0-rc.9"
"@rolldown/binding-android-arm64": "1.0.0-rc.10",
"@rolldown/binding-darwin-arm64": "1.0.0-rc.10",
"@rolldown/binding-darwin-x64": "1.0.0-rc.10",
"@rolldown/binding-freebsd-x64": "1.0.0-rc.10",
"@rolldown/binding-linux-arm-gnueabihf": "1.0.0-rc.10",
"@rolldown/binding-linux-arm64-gnu": "1.0.0-rc.10",
"@rolldown/binding-linux-arm64-musl": "1.0.0-rc.10",
"@rolldown/binding-linux-ppc64-gnu": "1.0.0-rc.10",
"@rolldown/binding-linux-s390x-gnu": "1.0.0-rc.10",
"@rolldown/binding-linux-x64-gnu": "1.0.0-rc.10",
"@rolldown/binding-linux-x64-musl": "1.0.0-rc.10",
"@rolldown/binding-openharmony-arm64": "1.0.0-rc.10",
"@rolldown/binding-wasm32-wasi": "1.0.0-rc.10",
"@rolldown/binding-win32-arm64-msvc": "1.0.0-rc.10",
"@rolldown/binding-win32-x64-msvc": "1.0.0-rc.10"
}
},
"node_modules/siginfo": {
@@ -1081,9 +1071,9 @@
"optional": true
},
"node_modules/typescript": {
"version": "5.9.3",
"resolved": "https://registry.npmjs.org/typescript/-/typescript-5.9.3.tgz",
"integrity": "sha512-jl1vZzPDinLr9eUt3J/t7V6FgNEw9QjvBPdysz9KfQDD41fQrC2Y4vKQdiaUpFT4bXlb1RHhLpp8wtm6M5TgSw==",
"version": "6.0.2",
"resolved": "https://registry.npmjs.org/typescript/-/typescript-6.0.2.tgz",
"integrity": "sha512-bGdAIrZ0wiGDo5l8c++HWtbaNCWTS4UTv7RaTH/ThVIgjkveJt83m74bBHMJkuCbslY8ixgLBVZJIOiQlQTjfQ==",
"dev": true,
"license": "Apache-2.0",
"bin": {
@@ -1095,17 +1085,16 @@
}
},
"node_modules/vite": {
"version": "8.0.0",
"resolved": "https://registry.npmjs.org/vite/-/vite-8.0.0.tgz",
"integrity": "sha512-fPGaRNj9Zytaf8LEiBhY7Z6ijnFKdzU/+mL8EFBaKr7Vw1/FWcTBAMW0wLPJAGMPX38ZPVCVgLceWiEqeoqL2Q==",
"version": "8.0.1",
"resolved": "https://registry.npmjs.org/vite/-/vite-8.0.1.tgz",
"integrity": "sha512-wt+Z2qIhfFt85uiyRt5LPU4oVEJBXj8hZNWKeqFG4gRG/0RaRGJ7njQCwzFVjO+v4+Ipmf5CY7VdmZRAYYBPHw==",
"dev": true,
"license": "MIT",
"dependencies": {
"@oxc-project/runtime": "0.115.0",
"lightningcss": "^1.32.0",
"picomatch": "^4.0.3",
"postcss": "^8.5.8",
"rolldown": "1.0.0-rc.9",
"rolldown": "1.0.0-rc.10",
"tinyglobby": "^0.2.15"
},
"bin": {
@@ -1122,7 +1111,7 @@
},
"peerDependencies": {
"@types/node": "^20.19.0 || >=22.12.0",
"@vitejs/devtools": "^0.0.0-alpha.31",
"@vitejs/devtools": "^0.1.0",
"esbuild": "^0.27.0",
"jiti": ">=1.21.0",
"less": "^4.0.0",
@@ -1174,19 +1163,19 @@
}
},
"node_modules/vitest": {
"version": "4.1.0",
"resolved": "https://registry.npmjs.org/vitest/-/vitest-4.1.0.tgz",
"integrity": "sha512-YbDrMF9jM2Lqc++2530UourxZHmkKLxrs4+mYhEwqWS97WJ7wOYEkcr+QfRgJ3PW9wz3odRijLZjHEaRLTNbqw==",
"version": "4.1.1",
"resolved": "https://registry.npmjs.org/vitest/-/vitest-4.1.1.tgz",
"integrity": "sha512-yF+o4POL41rpAzj5KVILUxm1GCjKnELvaqmU9TLLUbMfDzuN0UpUR9uaDs+mCtjPe+uYPksXDRLQGGPvj1cTmA==",
"dev": true,
"license": "MIT",
"dependencies": {
"@vitest/expect": "4.1.0",
"@vitest/mocker": "4.1.0",
"@vitest/pretty-format": "4.1.0",
"@vitest/runner": "4.1.0",
"@vitest/snapshot": "4.1.0",
"@vitest/spy": "4.1.0",
"@vitest/utils": "4.1.0",
"@vitest/expect": "4.1.1",
"@vitest/mocker": "4.1.1",
"@vitest/pretty-format": "4.1.1",
"@vitest/runner": "4.1.1",
"@vitest/snapshot": "4.1.1",
"@vitest/spy": "4.1.1",
"@vitest/utils": "4.1.1",
"es-module-lexer": "^2.0.0",
"expect-type": "^1.3.0",
"magic-string": "^0.30.21",
@@ -1198,7 +1187,7 @@
"tinyexec": "^1.0.2",
"tinyglobby": "^0.2.15",
"tinyrainbow": "^3.0.3",
"vite": "^6.0.0 || ^7.0.0 || ^8.0.0-0",
"vite": "^6.0.0 || ^7.0.0 || ^8.0.0",
"why-is-node-running": "^2.3.0"
},
"bin": {
@@ -1214,13 +1203,13 @@
"@edge-runtime/vm": "*",
"@opentelemetry/api": "^1.9.0",
"@types/node": "^20.0.0 || ^22.0.0 || >=24.0.0",
"@vitest/browser-playwright": "4.1.0",
"@vitest/browser-preview": "4.1.0",
"@vitest/browser-webdriverio": "4.1.0",
"@vitest/ui": "4.1.0",
"@vitest/browser-playwright": "4.1.1",
"@vitest/browser-preview": "4.1.1",
"@vitest/browser-webdriverio": "4.1.1",
"@vitest/ui": "4.1.1",
"happy-dom": "*",
"jsdom": "*",
"vite": "^6.0.0 || ^7.0.0 || ^8.0.0-0"
"vite": "^6.0.0 || ^7.0.0 || ^8.0.0"
},
"peerDependenciesMeta": {
"@edge-runtime/vm": {
+1 -1
View File
@@ -32,7 +32,7 @@
],
"license": "BUSL-1.1",
"devDependencies": {
"typescript": "^5.4",
"typescript": "^6.0.0",
"vitest": "^4.1"
}
}
+3
View File
@@ -186,6 +186,7 @@ export interface SkillInfo {
agent_max_turns: number | null;
notify_on_complete: string;
enabled: boolean;
priority: number;
allowed_tools: string;
license: string;
compatibility: string;
@@ -215,6 +216,7 @@ export interface CreateSkillRequest {
agent_max_turns?: number | null;
notify_on_complete?: string;
enabled?: boolean;
priority?: number;
allowed_tools?: string;
license?: string;
compatibility?: string;
@@ -240,6 +242,7 @@ export interface UpdateSkillRequest {
agent_max_turns?: number | null;
notify_on_complete?: string;
enabled?: boolean;
priority?: number;
allowed_tools?: string;
license?: string;
compatibility?: string;
+344 -48
View File
@@ -1,5 +1,5 @@
{
"description": "turnstone behavior tests tool selection, sequencing, and multi-step reasoning",
"description": "turnstone behavior tests \u2014 tool selection, sequencing, and multi-step reasoning",
"defaults": {
"n_runs": 5,
"max_turns": 15
@@ -8,35 +8,91 @@
{
"id": "read-before-edit",
"description": "Must read_file before edit_file on the same path",
"user_prompt": "Fix the typo in config.py change 'recieve' to 'receive'",
"user_prompt": "Fix the typo in config.py \u2014 change 'recieve' to 'receive'",
"setup": {
"files": {
"config.py": "# Config module\ndef recieve_data(source):\n \"\"\"Recieve data from source.\"\"\"\n return source.read()\n"
}
},
"expected_actions": [
{ "tool": "read_file", "args": { "path": "config.py" } },
{ "tool": "edit_file", "args": { "path": "config.py" } }
{
"tool": "read_file",
"args": {
"path": "config.py"
}
},
{
"tool": "edit_file",
"args": {
"path": "config.py"
}
}
],
"match_mode": "ordered_subset"
"match_mode": "ordered_subset",
"user_prompts": [
"Fix the typo in config.py \u2014 change 'recieve' to 'receive'",
"In config.py, correct the misspelling of 'recieve' to 'receive'",
"Please update config.py by replacing 'recieve' with the correct spelling 'receive'",
"There's a typo in config.py: 'recieve' should be 'receive'. Please fix it.",
"Could you change 'recieve' to 'receive' in config.py?",
"Go ahead and fix 'recieve' \u2192 'receive' in config.py",
"I need the word 'recieve' corrected to 'receive' in the file config.py",
"config.py has a spelling error \u2014 'recieve' needs to be changed to 'receive'",
"Kindly rectify the typographical error in config.py, replacing 'recieve' with 'receive'",
"Hey, swap 'recieve' for 'receive' in config.py"
]
},
{
"id": "write-file-not-bash",
"description": "Use write_file for file creation, not bash echo/cat",
"user_prompt": "Create a file called hello.py that prints hello world",
"expected_actions": [
{ "tool": "write_file", "args_pattern": { "path": "hello\\.py" } }
{
"tool": "write_file",
"args_pattern": {
"path": "hello\\.py"
}
}
],
"match_mode": "subset"
"match_mode": "subset",
"user_prompts": [
"Create a file called hello.py that prints hello world",
"Make a hello.py file that outputs hello world",
"Please write a Python file named hello.py which prints hello world",
"I need a file called hello.py that prints hello world",
"Could you create hello.py with code that prints hello world?",
"Write hello.py \u2014 it should print hello world",
"Generate a hello.py file that outputs \"hello world\"",
"I'd like you to create a file named hello.py that prints hello world",
"Set up a file called hello.py to print hello world",
"Kindly produce a hello.py file whose purpose is to print hello world"
]
},
{
"id": "bash-for-commands",
"description": "Use bash for running system commands",
"user_prompt": "What Python version is installed?",
"expected_actions": [
{ "tool": "bash", "args_pattern": { "command": "python" } }
{
"tool": "bash",
"args_pattern": {
"command": "python"
}
}
],
"match_mode": "subset"
"match_mode": "subset",
"user_prompts": [
"What Python version is installed?",
"Check which version of Python is currently installed",
"Can you tell me the installed Python version?",
"python --version please",
"I need to know what version of Python is on this system",
"Which Python version do we have?",
"Could you look up the Python version that's installed here?",
"Determine the currently installed Python version",
"What's the Python version on this machine?",
"Please check the Python version"
]
},
{
"id": "search-for-patterns",
@@ -49,13 +105,30 @@
}
},
"expected_actions": [
{ "tool": "search", "args_pattern": { "query": "test_" } }
{
"tool": "search",
"args_pattern": {
"query": "test_"
}
}
],
"match_mode": "subset"
"match_mode": "subset",
"user_prompts": [
"Find all functions that start with 'test_' in the project",
"List every function in the project whose name begins with 'test_'",
"I need to locate all functions prefixed with 'test_' across the project",
"Could you search the project for any functions starting with 'test_'?",
"Show me all the test_ prefixed functions in this project",
"Hunt down every function that has a 'test_' prefix in the codebase",
"I'm looking for all functions named test_* throughout the project",
"Search the entire project for functions whose names start with test_",
"What functions beginning with 'test_' exist in this project?",
"Please identify all functions with the 'test_' prefix in the project files"
]
},
{
"id": "multi-file-edit",
"description": "Read and edit multiple files must read before editing each, and edit both",
"description": "Read and edit multiple files \u2014 must read before editing each, and edit both",
"user_prompt": "Change the default port from 8000 to 9000 in both server.py and config.py",
"setup": {
"files": {
@@ -64,12 +137,32 @@
}
},
"expected_actions": [
{ "tool": "read_file" },
{ "tool": "read_file" },
{ "tool": "edit_file" },
{ "tool": "edit_file" }
{
"tool": "read_file"
},
{
"tool": "read_file"
},
{
"tool": "edit_file"
},
{
"tool": "edit_file"
}
],
"match_mode": "ordered_subset"
"match_mode": "ordered_subset",
"user_prompts": [
"Change the default port from 8000 to 9000 in both server.py and config.py",
"Update the default port to 9000 instead of 8000 in server.py and config.py",
"Could you modify the port number from 8000 to 9000 in both config.py and server.py?",
"Please replace port 8000 with 9000 in server.py and config.py",
"I need the default port switched from 8000 to 9000 in both server.py and config.py",
"In server.py and config.py, the default port should be changed from 8000 to 9000",
"Swap out port 8000 for 9000 in config.py and server.py",
"Would you mind updating the default port value from 8000 to 9000 across both server.py and config.py?",
"The default port in server.py and config.py needs to be 9000 instead of 8000 \u2014 please make that change",
"Go ahead and change 8000 to 9000 for the default port in both server.py and config.py"
]
},
{
"id": "search-then-edit",
@@ -83,51 +176,147 @@
}
},
"expected_actions": [
{ "tool": "search", "args_pattern": { "query": "MAX_RETRIES" } },
{ "tool": "read_file" },
{ "tool": "edit_file", "args_pattern": { "old_string": "3" } }
{
"tool": "search",
"args_pattern": {
"query": "MAX_RETRIES"
}
},
{
"tool": "read_file"
},
{
"tool": "edit_file",
"args_pattern": {
"old_string": "3"
}
}
],
"match_mode": "ordered_subset"
"match_mode": "ordered_subset",
"user_prompts": [
"Find where MAX_RETRIES is defined and change it from 3 to 5",
"Locate the definition of MAX_RETRIES and update its value from 3 to 5",
"Could you search for where MAX_RETRIES is defined and modify it from 3 to 5?",
"I need MAX_RETRIES changed from 3 to 5 \u2014 find where it's defined and update it",
"Please find the MAX_RETRIES definition and bump it from 3 to 5",
"Hunt down MAX_RETRIES in the codebase and change its value from 3 to 5",
"Where is MAX_RETRIES set to 3? Change it to 5.",
"Search the code for the MAX_RETRIES definition and alter it from 3 to 5",
"I'd like you to locate MAX_RETRIES (currently 3) and set it to 5 instead",
"Go find MAX_RETRIES and switch it from 3 to 5"
]
},
{
"id": "bash-git-log",
"description": "Use bash for git commands, not other tools",
"user_prompt": "Show me the git log for the last 5 commits",
"expected_actions": [
{ "tool": "bash", "args_pattern": { "command": "git\\s+log" } }
{
"tool": "bash",
"args_pattern": {
"command": "git\\s+log"
}
}
],
"match_mode": "subset"
"match_mode": "subset",
"user_prompts": [
"Show me the git log for the last 5 commits",
"Display the 5 most recent git commits",
"Can you pull up the git log limited to the last five commits?",
"I need to see the git log showing only the previous 5 commits",
"git log for the 5 latest commits, please",
"Would you mind showing me the last five entries in the git log?",
"Print out the most recent 5 commits from the git log",
"I'd like to review the git log \u2014 just the last 5 commits",
"Show the recent 5 commit history using git log",
"Could you display the git commit history for the past five commits?"
]
},
{
"id": "write-then-run",
"description": "Create a script and run it to verify it works",
"user_prompt": "Create a Python script called fib.py that prints the first 10 Fibonacci numbers, then run it to verify",
"expected_actions": [
{ "tool": "write_file", "args_pattern": { "path": "fib\\.py" } },
{ "tool": "bash", "args_pattern": { "command": "python" } }
{
"tool": "write_file",
"args_pattern": {
"path": "fib\\.py"
}
},
{
"tool": "bash",
"args_pattern": {
"command": "python"
}
}
],
"match_mode": "ordered_subset"
"match_mode": "ordered_subset",
"user_prompts": [
"Create a Python script called fib.py that prints the first 10 Fibonacci numbers, then run it to verify",
"Write a Python file named fib.py that outputs the first 10 Fibonacci numbers, and execute it to confirm it works",
"Please make fib.py \u2013 a Python script printing the first ten Fibonacci numbers \u2013 then run it to check the output",
"I need a Python script fib.py that prints the first 10 Fibonacci numbers. Execute it afterwards to verify correctness.",
"Could you create fib.py to display the first 10 Fibonacci numbers in Python, and then run it to make sure it works?",
"Draft a script called fib.py in Python that outputs the first ten Fibonacci numbers, then execute it to validate",
"Hey, write me a fib.py that prints the first 10 Fibonacci numbers and run it so we can see it works",
"Generate a Python program fib.py which prints the initial 10 Fibonacci numbers, and verify by running it",
"Kindly produce a Python script named fib.py to print the first 10 Fibonacci numbers, then execute the script to confirm its output",
"Make a file fib.py containing Python code to print the first 10 Fibonacci numbers. Then run it to verify."
]
},
{
"id": "no-bash-for-file-write",
"description": "Should NOT use bash (echo/cat/heredoc) to create files only write_file",
"description": "Should NOT use bash (echo/cat/heredoc) to create files \u2014 only write_file",
"user_prompt": "Create a new file called README.md with a title and description of this project",
"expected_actions": [
{ "tool": "write_file", "args_pattern": { "path": "README" } }
{
"tool": "write_file",
"args_pattern": {
"path": "README"
}
}
],
"match_mode": "subset"
"match_mode": "subset",
"user_prompts": [
"Create a new file called README.md with a title and description of this project",
"Make a README.md file that includes a project title and description",
"I need a README.md created with a title and a brief description of the project",
"Please generate a README.md file containing the project's title and description",
"Could you set up a README.md with a title and project description?",
"Write a README.md that has a title and describes this project",
"Go ahead and create README.md \u2014 it should have a title and a description of the project",
"I'd like you to produce a new README.md file featuring a project title and description",
"Kindly establish a README.md file incorporating both a title and a description for this project",
"Spin up a README.md with a project title and description in it"
]
},
{
"id": "plan-before-refactor",
"description": "Use the plan tool before a large refactoring task",
"user_prompt": "I need to refactor this codebase to separate the database layer from the API layer. Use the plan tool to think through the approach before making any changes.",
"id": "plan-when-asked",
"description": "Call the plan tool when the user asks to plan",
"user_prompt": "Plan how to add user authentication to this app.",
"setup": {
"files": {
"app.py": "import sqlite3\nfrom flask import Flask, jsonify\n\napp = Flask(__name__)\nDB = 'data.db'\n\ndef get_db():\n return sqlite3.connect(DB)\n\n@app.route('/users')\ndef list_users():\n db = get_db()\n users = db.execute('SELECT * FROM users').fetchall()\n db.close()\n return jsonify(users)\n\n@app.route('/users/<int:uid>')\ndef get_user(uid):\n db = get_db()\n user = db.execute('SELECT * FROM users WHERE id=?', (uid,)).fetchone()\n db.close()\n return jsonify(user)\n\nif __name__ == '__main__':\n app.run(port=8000)\n"
"app.py": "from flask import Flask, jsonify\n\napp = Flask(__name__)\n\n@app.route('/users')\ndef list_users():\n return jsonify([{'id': 1, 'name': 'Alice'}])\n\nif __name__ == '__main__':\n app.run(port=8000)\n"
}
},
"expected_actions": [{ "tool": "create_plan" }],
"match_mode": "subset"
"expected_actions": [
{
"tool": "plan_agent"
}
],
"match_mode": "subset",
"user_prompts": [
"Plan how to add user authentication to this app.",
"Make a plan for adding pagination to the API endpoints.",
"Plan out how to add error handling to this application.",
"I need a plan for adding logging to this codebase.",
"Plan the approach for adding unit tests to this app.",
"How would you approach adding user authentication to this app? Lay out a plan.",
"I'd like you to outline a strategy for implementing user authentication in this application.",
"Could you come up with a plan for integrating user authentication into this app?",
"Think through the steps needed to add user auth to this app and present a plan.",
"Draft a plan for incorporating user authentication functionality into this application."
]
},
{
"id": "edit-not-rewrite",
@@ -139,10 +328,32 @@
}
},
"expected_actions": [
{ "tool": "read_file", "args": { "path": "utils.py" } },
{ "tool": "edit_file", "args": { "path": "utils.py" } }
{
"tool": "read_file",
"args": {
"path": "utils.py"
}
},
{
"tool": "edit_file",
"args": {
"path": "utils.py"
}
}
],
"match_mode": "ordered_subset"
"match_mode": "ordered_subset",
"user_prompts": [
"Add a docstring to the process_data function in utils.py",
"Please add a docstring to the process_data function in utils.py",
"Could you write a docstring for process_data in utils.py?",
"Insert a docstring into the process_data function found in utils.py",
"I need a docstring added to process_data in utils.py",
"Put a docstring on the process_data function in utils.py",
"The process_data function in utils.py is missing a docstring \u2014 please add one",
"Would you mind adding a docstring to process_data in utils.py?",
"In utils.py, the process_data function needs a docstring",
"Add documentation via a docstring to the process_data function within utils.py"
]
},
{
"id": "bash-run-tests",
@@ -154,45 +365,130 @@
}
},
"expected_actions": [
{ "tool": "bash", "args_pattern": { "command": "pytest|python.*test" } }
{
"tool": "bash",
"args_pattern": {
"command": "pytest|python.*test"
}
}
],
"match_mode": "subset"
"match_mode": "subset",
"user_prompts": [
"Run the tests",
"Execute the test suite",
"Please go ahead and run the tests",
"Could you run the tests for me?",
"I need the tests to be run",
"Kick off the tests",
"Let's run the tests",
"Fire up the tests",
"Go ahead and execute the tests",
"I'd like you to run the tests"
]
},
{
"id": "web-fetch-url",
"description": "Use web_fetch when asked to retrieve content from a URL",
"user_prompt": "Fetch the contents of https://example.com and summarize what's on the page",
"expected_actions": [
{ "tool": "web_fetch", "args_pattern": { "url": "example\\.com" } }
{
"tool": "web_fetch",
"args_pattern": {
"url": "example\\.com"
}
}
],
"match_mode": "subset"
"match_mode": "subset",
"user_prompts": [
"Fetch the contents of https://example.com and summarize what's on the page",
"Go to https://example.com and give me a summary of what you find there",
"Could you pull up https://example.com and tell me what the page is about?",
"Retrieve the content from https://example.com, then provide a summary of it",
"I need you to grab https://example.com and summarize its contents for me",
"Please access https://example.com and give me an overview of the page",
"What's on https://example.com? Fetch it and summarize for me.",
"Download the page at https://example.com and provide a brief summary",
"I'd like a summary of whatever is at https://example.com \u2014 please fetch it first",
"Hit https://example.com and let me know what's there in summary form"
]
},
{
"id": "man-page-lookup",
"description": "Use man tool to look up command documentation",
"user_prompt": "Look up the man page for tar and tell me what the --xattrs flag does",
"expected_actions": [
{ "tool": "man", "args_pattern": { "page": "tar" } }
{
"tool": "man",
"args_pattern": {
"page": "tar"
}
}
],
"match_mode": "subset"
"match_mode": "subset",
"user_prompts": [
"Look up the man page for tar and tell me what the --xattrs flag does",
"What does the --xattrs flag do in tar? Check the man page for me.",
"Could you pull up the man page for tar and explain the --xattrs option?",
"I need to know what --xattrs does in tar \u2014 can you check the man page?",
"Check tar's man page and let me know the purpose of the --xattrs flag.",
"Please consult the tar man page and describe what the --xattrs flag is for.",
"Hey, look at the tar man page real quick \u2014 what's --xattrs do?",
"I'd like you to read the tar man page and summarize the --xattrs option for me.",
"Would you mind checking the man page for tar to find out what --xattrs means?",
"Look into the tar manual and explain the --xattrs flag to me."
]
},
{
"id": "math-calculation",
"description": "Use the math tool for precise calculations, not bash or mental math",
"user_prompt": "What is 2^64 - 1? Use the math tool to calculate it precisely.",
"expected_actions": [
{ "tool": "math", "args_pattern": { "code": "2.*64" } }
{
"tool": "math",
"args_pattern": {
"code": "2.*64"
}
}
],
"match_mode": "subset"
"match_mode": "subset",
"user_prompts": [
"What is 2^64 - 1? Use the math tool to calculate it precisely.",
"Calculate 2^64 - 1 for me using the math tool, please.",
"I need the exact value of 2^64 - 1. Please use the math tool.",
"Could you use the math tool to compute 2^64 minus 1 precisely?",
"Use the math tool to tell me what 2^64 - 1 equals.",
"I'm curious: what's 2^64 - 1? Compute it with the math tool.",
"Please precisely determine 2^64 - 1 via the math tool.",
"Mind using the math tool to figure out 2^64 - 1 exactly?",
"I'd like to know the precise result of 2^64 - 1 \u2014 use the math tool for this.",
"Leverage the math tool to give me an exact answer for 2^64 - 1."
]
},
{
"id": "web-search-query",
"description": "Use web_search for general knowledge lookups, not web_fetch",
"user_prompt": "Search the web for the current population of Tokyo",
"expected_actions": [
{ "tool": "web_search", "args_pattern": { "query": "Tokyo" } }
{
"tool": "web_search",
"args_pattern": {
"query": "Tokyo"
}
}
],
"match_mode": "subset"
"match_mode": "subset",
"user_prompts": [
"Search the web for the current population of Tokyo",
"What's Tokyo's current population? Look it up on the web.",
"Could you do a web search to find out how many people currently live in Tokyo?",
"Please search online for Tokyo's present-day population.",
"I need you to look up the current population of Tokyo on the web.",
"Find me Tokyo's current population via a web search.",
"Web search: what is the current population of Tokyo?",
"I'd like to know Tokyo's current population\u2014can you search the web for that?",
"Look up how many people live in Tokyo right now using a web search.",
"Do a web search for the population of Tokyo as of now."
]
}
]
}
+75 -1
View File
@@ -1,11 +1,23 @@
from __future__ import annotations
import os
from unittest.mock import MagicMock
import pytest
def pytest_addoption(parser: pytest.Parser) -> None:
parser.addoption(
"--storage-backend",
default="sqlite",
choices=["sqlite", "postgresql"],
help="Storage backend for integration tests (default: sqlite)",
)
@pytest.fixture
def tmp_db(tmp_path):
"""Provide a temporary SQLite storage backend."""
"""Provide a temporary SQLite storage backend (singleton registry)."""
from turnstone.core.storage import init_storage, reset_storage
db_path = str(tmp_path / "test.db")
@@ -15,6 +27,68 @@ def tmp_db(tmp_path):
reset_storage()
@pytest.fixture
def storage_backend(request, tmp_path):
"""Shared storage backend fixture — respects --storage-backend flag.
Returns a StorageBackend instance (SQLite or PostgreSQL).
Tests that use this fixture run against whichever backend CI selects.
"""
from turnstone.core.storage import init_storage, reset_storage
backend_type = request.config.getoption("--storage-backend")
reset_storage()
if backend_type == "postgresql":
pg_url = os.environ.get(
"TURNSTONE_TEST_PG_URL",
"postgresql+psycopg://postgres:postgres@localhost:5432/turnstone_test",
)
backend = init_storage("postgresql", url=pg_url, run_migrations=False)
yield backend
# Truncate all tables between tests — faster than DELETE and resets
# autoincrement sequences. CASCADE handles any future FK constraints.
# NOTE: accesses backend._engine (SQLAlchemy internal) — both SQLite
# and PostgreSQL backends expose this. If a non-SQLAlchemy backend is
# ever added, this cleanup will need a protocol-level hook.
try:
import sqlalchemy as sa
from turnstone.core.storage._schema import metadata as db_metadata
with backend._engine.connect() as conn:
table_names = ", ".join(t.name for t in reversed(db_metadata.sorted_tables))
conn.execute(sa.text(f"TRUNCATE {table_names} RESTART IDENTITY CASCADE"))
conn.commit()
except Exception:
pass # best-effort cleanup; reset_storage disposes engine
finally:
reset_storage()
else:
db_path = str(tmp_path / "test.db")
backend = init_storage("sqlite", path=db_path, run_migrations=False)
yield backend
reset_storage()
@pytest.fixture
def backend(storage_backend):
"""Alias for storage_backend — used by test_storage_sqlite.py etc."""
return storage_backend
@pytest.fixture
def db(storage_backend):
"""Alias for storage_backend — used by domain-specific storage tests."""
return storage_backend
@pytest.fixture
def storage(storage_backend):
"""Alias for storage_backend — used by services/skill resource tests."""
return storage_backend
@pytest.fixture
def mock_openai_client():
"""Return a minimal mock OpenAI client."""
+46
View File
@@ -1237,6 +1237,52 @@ class TestJWTAudienceIssuer:
result = validate_jwt(token, self.SECRET, audience="")
assert result is not None
def test_create_jwt_expiry_seconds(self):
import jwt as pyjwt
from turnstone.core.auth import create_jwt
token = create_jwt("user1", frozenset({"read"}), "test", self.SECRET, expiry_seconds=300)
payload = pyjwt.decode(
token, self.SECRET, algorithms=["HS256"], options={"verify_aud": False}
)
assert payload["exp"] - payload["iat"] == 300
def test_create_jwt_expiry_seconds_overrides_hours(self):
import jwt as pyjwt
from turnstone.core.auth import create_jwt
token = create_jwt(
"user1",
frozenset({"read"}),
"test",
self.SECRET,
expiry_hours=24,
expiry_seconds=60,
)
payload = pyjwt.decode(
token, self.SECRET, algorithms=["HS256"], options={"verify_aud": False}
)
# expiry_seconds takes precedence over expiry_hours
assert payload["exp"] - payload["iat"] == 60
def test_create_jwt_expiry_seconds_rejects_zero(self):
import pytest
from turnstone.core.auth import create_jwt
with pytest.raises(ValueError, match="expiry_seconds must be positive"):
create_jwt("user1", frozenset({"read"}), "test", self.SECRET, expiry_seconds=0)
def test_create_jwt_expiry_seconds_rejects_negative(self):
import pytest
from turnstone.core.auth import create_jwt
with pytest.raises(ValueError, match="expiry_seconds must be positive"):
create_jwt("user1", frozenset({"read"}), "test", self.SECRET, expiry_seconds=-1)
class TestServiceTokenManager:
SECRET = "test-secret-that-is-at-least-32-chars"
+351
View File
@@ -0,0 +1,351 @@
"""Stress tests for bridge.py threading — race conditions in approval,
plan review, and workstream lifecycle.
Each scenario is run many times (ITERATIONS) with threading.Barrier to
maximize timing overlap. Uses mock broker (no Redis) and no HTTP calls.
Races tested:
1. Duplicate approval on SSE reconnect (TOCTOU in _pending_approvals)
2. Duplicate plan review on SSE reconnect (TOCTOU in _pending_plan_reviews)
3. approve_set stale reference escape during concurrent update
4. _running flag visibility across threads on shutdown
5. Approval thread exits within bounded time after timeout
6. Concurrent approval + workstream close leaves no orphaned state
"""
from __future__ import annotations
import threading
import time
from collections import Counter
from unittest.mock import MagicMock, patch
from turnstone.mq.bridge import Bridge
ITERATIONS = 100
# ---------------------------------------------------------------------------
# Helpers
# ---------------------------------------------------------------------------
def _make_bridge(**overrides) -> Bridge:
"""Create a Bridge with a mock broker (no Redis or HTTP)."""
broker = MagicMock()
defaults = dict(
server_url="http://localhost:8080",
broker=broker,
node_id="test-node",
approval_timeout=1,
)
defaults.update(overrides)
return Bridge(**defaults)
def _approval_items(tool_name: str = "bash") -> list[dict]:
return [{"func_name": tool_name, "needs_approval": True, "approval_label": tool_name}]
def _wait_pending_resolved(bridge: Bridge, key: str, attr: str, deadline_s: float = 3.0) -> bool:
"""Poll until the pending entry is resolved (tombstone) or absent."""
deadline = time.monotonic() + deadline_s
while time.monotonic() < deadline:
with bridge._lock:
entries = getattr(bridge, attr)
if key not in entries:
return True
_, resolved_at = entries[key]
if resolved_at > 0:
return True
time.sleep(0.01)
return False
# ---------------------------------------------------------------------------
# Race 1: Duplicate approval on SSE reconnect
# ---------------------------------------------------------------------------
class TestDuplicateApproval:
"""Two threads call _handle_approval for the same ws_id simultaneously.
Only one should create a pending entry; the other should be skipped."""
def test_no_duplicate_approvals(self):
sent_count = Counter()
for _ in range(ITERATIONS):
bridge = _make_bridge()
bridge._broker.pop_response.return_value = '{"type": "approve", "approved": true}'
barrier = threading.Barrier(2, timeout=5)
def _call_approval(bridge=bridge, barrier=barrier):
barrier.wait()
bridge._handle_approval("ws-1", {"items": _approval_items()})
t1 = threading.Thread(target=_call_approval)
t2 = threading.Thread(target=_call_approval)
with (
patch.object(bridge, "_api_approve") as mock_approve,
patch.object(bridge, "_publish_ws"),
):
t1.start()
t2.start()
t1.join(timeout=5)
t2.join(timeout=5)
assert not t1.is_alive(), "Thread 1 hung"
assert not t2.is_alive(), "Thread 2 hung"
# Wait for spawned _wait_approval threads to resolve
_wait_pending_resolved(bridge, "ws-1", "_pending_approvals")
sent_count[mock_approve.call_count] += 1
# At most 1 approval should be forwarded per iteration
assert sent_count.get(2, 0) == 0, (
f"Duplicate approvals sent in {sent_count[2]}/{ITERATIONS} iterations"
)
# ---------------------------------------------------------------------------
# Race 2: Duplicate plan review on SSE reconnect
# ---------------------------------------------------------------------------
class TestDuplicatePlanReview:
"""Two threads call _handle_plan_review simultaneously.
Only one should create a pending entry."""
def test_no_duplicate_plan_reviews(self):
sent_count = Counter()
for _ in range(ITERATIONS):
bridge = _make_bridge()
bridge._broker.pop_response.return_value = (
'{"type": "plan_feedback", "feedback": "looks good"}'
)
barrier = threading.Barrier(2, timeout=5)
def _call_plan(bridge=bridge, barrier=barrier):
barrier.wait()
bridge._handle_plan_review("ws-1", {"content": "plan text"})
t1 = threading.Thread(target=_call_plan)
t2 = threading.Thread(target=_call_plan)
with patch.object(bridge, "_publish_ws"), patch.object(bridge._http, "post"):
t1.start()
t2.start()
t1.join(timeout=5)
t2.join(timeout=5)
assert not t1.is_alive(), "Thread 1 hung"
assert not t2.is_alive(), "Thread 2 hung"
# Wait for spawned _wait_plan threads to resolve
_wait_pending_resolved(bridge, "ws-1", "_pending_plan_reviews")
sent_count[bridge._http.post.call_count] += 1
assert sent_count.get(2, 0) == 0, (
f"Duplicate plan reviews sent in {sent_count[2]}/{ITERATIONS} iterations"
)
# ---------------------------------------------------------------------------
# Race 3: approve_set stale reference during concurrent update
# ---------------------------------------------------------------------------
class TestApproveSetConsistency:
"""One thread reads approve_set for auto-approve check while another
updates it via _wait_approval 'always' path. The auto-approve
decision should be consistent (either all-approved or not)."""
def test_approve_set_never_partially_visible(self):
for _ in range(ITERATIONS):
bridge = _make_bridge()
with bridge._lock:
bridge._ws_approve_tools["ws-1"] = {"read_file", "search"}
barrier = threading.Barrier(2, timeout=5)
results = []
def _reader(bridge=bridge, barrier=barrier, results=results):
barrier.wait()
with bridge._lock:
snap = bridge._ws_approve_tools.get("ws-1", set()).copy()
results.append(snap)
def _writer(bridge=bridge, barrier=barrier):
barrier.wait()
with bridge._lock:
existing = bridge._ws_approve_tools.get("ws-1", set())
bridge._ws_approve_tools["ws-1"] = existing | {"bash", "write_file"}
t1 = threading.Thread(target=_reader)
t2 = threading.Thread(target=_writer)
t1.start()
t2.start()
t1.join(timeout=5)
t2.join(timeout=5)
assert not t1.is_alive(), "Reader hung"
assert not t2.is_alive(), "Writer hung"
snap = results[0]
assert snap in (
{"read_file", "search"},
{"read_file", "search", "bash", "write_file"},
), f"Partial set observed: {snap}"
# ---------------------------------------------------------------------------
# Race 4: _running flag visibility across threads
# ---------------------------------------------------------------------------
class TestRunningFlagVisibility:
"""All threads reading _running should see False within a bounded time
after the main thread sets it."""
def test_all_threads_observe_shutdown(self):
bridge = _make_bridge()
observed_false = threading.Event()
threads_running = []
def _spin_checker():
while bridge._running:
time.sleep(0.001)
observed_false.set()
for _ in range(5):
t = threading.Thread(target=_spin_checker, daemon=True)
threads_running.append(t)
t.start()
time.sleep(0.01)
bridge._running = False
for t in threads_running:
t.join(timeout=1)
assert not t.is_alive(), "Thread did not observe _running=False"
assert observed_false.is_set()
# ---------------------------------------------------------------------------
# Race 5: Approval thread exits within bounded time
# ---------------------------------------------------------------------------
class TestApprovalThreadTimeout:
"""An approval thread blocked on pop_response should exit within the
configured approval_timeout, not hang indefinitely."""
def test_approval_thread_exits_within_timeout(self):
for _ in range(10):
bridge = _make_bridge(approval_timeout=0.5)
def _slow_pop(queue_name, timeout=300):
time.sleep(min(timeout, 0.5))
return None
bridge._broker.pop_response.side_effect = _slow_pop
with patch.object(bridge, "_publish_ws"), patch.object(bridge, "_api_approve"):
bridge._handle_approval("ws-1", {"items": _approval_items()})
# The pending entry should be resolved within the timeout
resolved = _wait_pending_resolved(bridge, "ws-1", "_pending_approvals", deadline_s=3.0)
assert resolved, "Approval thread did not exit within expected timeout"
# ---------------------------------------------------------------------------
# Race 6: Concurrent approval + workstream close
# ---------------------------------------------------------------------------
class TestApprovalDuringClose:
"""An approval arriving at the exact same time as a ws_closed event
should not leave orphaned state."""
def test_no_orphaned_pending_after_close(self):
for _ in range(ITERATIONS):
bridge = _make_bridge(approval_timeout=0.1)
bridge._broker.pop_response.return_value = None # timeout
barrier = threading.Barrier(2, timeout=5)
def _send_approval(bridge=bridge, barrier=barrier):
barrier.wait()
with patch.object(bridge, "_publish_ws"), patch.object(bridge, "_api_approve"):
bridge._handle_approval("ws-1", {"items": _approval_items()})
def _close_ws(bridge=bridge, barrier=barrier):
barrier.wait()
with (
patch.object(bridge, "_publish_global"),
patch.object(bridge, "_publish_cluster"),
):
bridge._handle_global_event({"type": "ws_closed", "ws_id": "ws-1"})
t1 = threading.Thread(target=_send_approval)
t2 = threading.Thread(target=_close_ws)
t1.start()
t2.start()
t1.join(timeout=5)
t2.join(timeout=5)
assert not t1.is_alive(), "Approval thread hung"
assert not t2.is_alive(), "Close thread hung"
# Wait for spawned _wait_approval thread to resolve (if close
# didn't remove the entry first)
resolved = _wait_pending_resolved(bridge, "ws-1", "_pending_approvals")
assert resolved, "Orphaned pending approval"
# ---------------------------------------------------------------------------
# Race 7: Plan review refinement loop (tombstone → cleanup → re-entry)
# ---------------------------------------------------------------------------
class TestPlanReviewRefinementLoop:
"""After a plan review is resolved, a ws_state event should clean up the
tombstone so the refinement-loop plan_review event is handled correctly."""
def test_refinement_loop_allows_reentry(self):
for _ in range(ITERATIONS):
bridge = _make_bridge()
bridge._broker.pop_response.return_value = (
'{"type": "plan_feedback", "feedback": "refine this"}'
)
# Step 1: first plan review — creates pending entry, resolves it
with patch.object(bridge, "_publish_ws"), patch.object(bridge._http, "post"):
bridge._handle_plan_review("ws-1", {"content": "plan v1"})
_wait_pending_resolved(bridge, "ws-1", "_pending_plan_reviews")
# Verify tombstone is present (resolved_at > 0)
with bridge._lock:
assert "ws-1" in bridge._pending_plan_reviews
assert bridge._pending_plan_reviews["ws-1"][1] > 0
# Step 2: ws_state event cleans up the resolved tombstone
with (
patch.object(bridge, "_publish_ws"),
patch.object(bridge, "_publish_global"),
patch.object(bridge, "_publish_cluster"),
):
bridge._handle_global_event(
{"type": "ws_state", "ws_id": "ws-1", "state": "working"}
)
with bridge._lock:
assert "ws-1" not in bridge._pending_plan_reviews
# Step 3: refinement plan_review arrives — should create new entry
with patch.object(bridge, "_publish_ws"), patch.object(bridge._http, "post"):
bridge._handle_plan_review("ws-1", {"content": "plan v2"})
_wait_pending_resolved(bridge, "ws-1", "_pending_plan_reviews")
with bridge._lock:
assert "ws-1" in bridge._pending_plan_reviews
-11
View File
@@ -2,17 +2,6 @@
from __future__ import annotations
import pytest
from turnstone.core.storage._sqlite import SQLiteBackend
@pytest.fixture
def db(tmp_path):
"""Fresh SQLite backend for each test."""
backend = SQLiteBackend(str(tmp_path / "test.db"))
return backend
class TestChannelUserCRUD:
"""Tests for channel_users table operations."""
+60 -27
View File
@@ -3,54 +3,55 @@
import argparse
import turnstone.core.config as config_mod
from turnstone.core.config import apply_config, load_config
from turnstone.core.config import apply_config, load_config, set_config_path
def _reset_cache():
"""Clear the module-level config cache between tests."""
config_mod._cache = None
config_mod._config_path = None
def test_load_config_missing_file(tmp_path, monkeypatch):
def test_load_config_missing_file(tmp_path):
_reset_cache()
monkeypatch.setattr(config_mod, "CONFIG_PATH", tmp_path / "nope.toml")
set_config_path(str(tmp_path / "nope.toml"))
assert load_config() == {}
def test_load_config_valid_toml(tmp_path, monkeypatch):
def test_load_config_valid_toml(tmp_path):
_reset_cache()
cfg = tmp_path / "config.toml"
cfg.write_text('[redis]\nhost = "10.0.0.1"\nport = 6380\npassword = "secret"\n')
monkeypatch.setattr(config_mod, "CONFIG_PATH", cfg)
set_config_path(str(cfg))
result = load_config()
assert result["redis"]["host"] == "10.0.0.1"
assert result["redis"]["port"] == 6380
assert result["redis"]["password"] == "secret"
def test_load_config_section(tmp_path, monkeypatch):
def test_load_config_section(tmp_path):
_reset_cache()
cfg = tmp_path / "config.toml"
cfg.write_text('[api]\nbase_url = "http://x:8000/v1"\n[redis]\nhost = "y"\n')
monkeypatch.setattr(config_mod, "CONFIG_PATH", cfg)
set_config_path(str(cfg))
assert load_config("redis") == {"host": "y"}
assert load_config("api") == {"base_url": "http://x:8000/v1"}
assert load_config("nonexistent") == {}
def test_load_config_invalid_toml(tmp_path, monkeypatch):
def test_load_config_invalid_toml(tmp_path):
_reset_cache()
cfg = tmp_path / "config.toml"
cfg.write_text("this is not valid toml [[[")
monkeypatch.setattr(config_mod, "CONFIG_PATH", cfg)
set_config_path(str(cfg))
assert load_config() == {}
def test_load_config_caches(tmp_path, monkeypatch):
def test_load_config_caches(tmp_path):
_reset_cache()
cfg = tmp_path / "config.toml"
cfg.write_text('[api]\nbase_url = "http://first"\n')
monkeypatch.setattr(config_mod, "CONFIG_PATH", cfg)
set_config_path(str(cfg))
first = load_config()
assert first["api"]["base_url"] == "http://first"
@@ -60,14 +61,14 @@ def test_load_config_caches(tmp_path, monkeypatch):
assert second["api"]["base_url"] == "http://first"
def test_apply_config_sets_defaults(tmp_path, monkeypatch):
def test_apply_config_sets_defaults(tmp_path):
_reset_cache()
cfg = tmp_path / "config.toml"
cfg.write_text(
'[redis]\nhost = "redis.local"\nport = 7777\npassword = "pw"\n'
'[bridge]\nserver_url = "http://bridge:9090"\n'
)
monkeypatch.setattr(config_mod, "CONFIG_PATH", cfg)
set_config_path(str(cfg))
parser = argparse.ArgumentParser()
parser.add_argument("--redis-host", default="localhost")
@@ -84,11 +85,11 @@ def test_apply_config_sets_defaults(tmp_path, monkeypatch):
assert args.server_url == "http://bridge:9090"
def test_apply_config_cli_overrides(tmp_path, monkeypatch):
def test_apply_config_cli_overrides(tmp_path):
_reset_cache()
cfg = tmp_path / "config.toml"
cfg.write_text('[redis]\nhost = "config-host"\nport = 7777\n')
monkeypatch.setattr(config_mod, "CONFIG_PATH", cfg)
set_config_path(str(cfg))
parser = argparse.ArgumentParser()
parser.add_argument("--redis-host", default="localhost")
@@ -102,11 +103,11 @@ def test_apply_config_cli_overrides(tmp_path, monkeypatch):
assert args.redis_port == 7777 # config wins (no CLI override)
def test_apply_config_missing_keys_keep_defaults(tmp_path, monkeypatch):
def test_apply_config_missing_keys_keep_defaults(tmp_path):
_reset_cache()
cfg = tmp_path / "config.toml"
cfg.write_text('[redis]\nhost = "only-host"\n') # no port, no password
monkeypatch.setattr(config_mod, "CONFIG_PATH", cfg)
set_config_path(str(cfg))
parser = argparse.ArgumentParser()
parser.add_argument("--redis-host", default="localhost")
@@ -121,9 +122,9 @@ def test_apply_config_missing_keys_keep_defaults(tmp_path, monkeypatch):
assert args.redis_password is None # original default kept
def test_apply_config_no_file(tmp_path, monkeypatch):
def test_apply_config_no_file(tmp_path):
_reset_cache()
monkeypatch.setattr(config_mod, "CONFIG_PATH", tmp_path / "nope.toml")
set_config_path(str(tmp_path / "nope.toml"))
parser = argparse.ArgumentParser()
parser.add_argument("--redis-host", default="localhost")
@@ -133,11 +134,11 @@ def test_apply_config_no_file(tmp_path, monkeypatch):
assert args.redis_host == "localhost"
def test_apply_config_model_section(tmp_path, monkeypatch):
def test_apply_config_model_section(tmp_path):
_reset_cache()
cfg = tmp_path / "config.toml"
cfg.write_text('[model]\nname = "qwen-72b"\ntemperature = 0.3\n')
monkeypatch.setattr(config_mod, "CONFIG_PATH", cfg)
set_config_path(str(cfg))
parser = argparse.ArgumentParser()
parser.add_argument("--model", default=None)
@@ -158,7 +159,7 @@ def test_tavily_key_from_config(tmp_path, monkeypatch):
cfg = tmp_path / "config.toml"
cfg.write_text('[api]\ntavily_key = "tvly-from-config"\n')
monkeypatch.setattr(config_mod, "CONFIG_PATH", cfg)
set_config_path(str(cfg))
monkeypatch.delenv("TAVILY_API_KEY", raising=False)
key = config_mod.get_tavily_key()
@@ -174,14 +175,14 @@ def test_tavily_key_fallback_to_env(tmp_path, monkeypatch):
# Config exists but no tavily_key in it
cfg = tmp_path / "config.toml"
cfg.write_text("[api]\n")
monkeypatch.setattr(config_mod, "CONFIG_PATH", cfg)
set_config_path(str(cfg))
monkeypatch.setenv("TAVILY_API_KEY", "tvly-from-env")
key = config_mod.get_tavily_key()
assert key == "tvly-from-env"
def test_apply_config_judge_section(tmp_path, monkeypatch):
def test_apply_config_judge_section(tmp_path):
"""apply_config() loads [judge] section and maps to argparse dests."""
_reset_cache()
cfg = tmp_path / "config.toml"
@@ -193,7 +194,7 @@ def test_apply_config_judge_section(tmp_path, monkeypatch):
"timeout = 30.0\n"
"read_only_tools = false\n"
)
monkeypatch.setattr(config_mod, "CONFIG_PATH", cfg)
set_config_path(str(cfg))
parser = argparse.ArgumentParser()
parser.add_argument("--judge", dest="judge_enabled", action="store_true", default=False)
@@ -212,12 +213,12 @@ def test_apply_config_judge_section(tmp_path, monkeypatch):
assert args.judge_read_only_tools is False
def test_apply_config_judge_cli_overrides(tmp_path, monkeypatch):
def test_apply_config_judge_cli_overrides(tmp_path):
"""CLI flags override config.toml [judge] values."""
_reset_cache()
cfg = tmp_path / "config.toml"
cfg.write_text("[judge]\nenabled = true\nconfidence_threshold = 0.85\n")
monkeypatch.setattr(config_mod, "CONFIG_PATH", cfg)
set_config_path(str(cfg))
parser = argparse.ArgumentParser()
parser.add_argument("--judge", dest="judge_enabled", action="store_true", default=False)
@@ -229,3 +230,35 @@ def test_apply_config_judge_cli_overrides(tmp_path, monkeypatch):
assert args.judge_enabled is False # CLI wins
assert args.judge_confidence == 0.85 # config wins (no CLI override)
def test_set_config_path_overrides_default(tmp_path):
"""set_config_path() overrides the default config location."""
_reset_cache()
cfg = tmp_path / "custom.toml"
cfg.write_text('[api]\nbase_url = "http://custom:9999"\n')
set_config_path(str(cfg))
assert load_config("api") == {"base_url": "http://custom:9999"}
def test_env_var_overrides_default(tmp_path, monkeypatch):
"""$TURNSTONE_CONFIG env var overrides the default config location."""
_reset_cache()
cfg = tmp_path / "env.toml"
cfg.write_text('[api]\nbase_url = "http://env:7777"\n')
monkeypatch.setenv("TURNSTONE_CONFIG", str(cfg))
assert load_config("api") == {"base_url": "http://env:7777"}
def test_set_config_path_overrides_env_var(tmp_path, monkeypatch):
"""set_config_path() takes precedence over $TURNSTONE_CONFIG."""
_reset_cache()
env_cfg = tmp_path / "env.toml"
env_cfg.write_text('[api]\nbase_url = "http://env"\n')
monkeypatch.setenv("TURNSTONE_CONFIG", str(env_cfg))
explicit_cfg = tmp_path / "explicit.toml"
explicit_cfg.write_text('[api]\nbase_url = "http://explicit"\n')
set_config_path(str(explicit_cfg))
assert load_config("api") == {"base_url": "http://explicit"}
+250 -16
View File
@@ -47,8 +47,8 @@ class MockBroker:
# ---------------------------------------------------------------------------
def _make_collector(broker=None, poll_interval=999, discovery_interval=999):
"""Create a collector with long intervals so threads don't auto-fire."""
def _make_collector(broker=None, poll_interval=0, discovery_interval=999):
"""Create a collector with zero poll interval (no jitter delay in tests)."""
b = broker or MockBroker()
return ClusterCollector(
broker=b,
@@ -953,6 +953,8 @@ class TestConsoleWorkstreamCreation:
],
2,
)
# get_all_nodes delegates to get_nodes (mirrors real implementation)
collector.get_all_nodes.side_effect = lambda: collector.get_nodes.return_value[0]
return collector
@pytest.fixture()
@@ -1266,47 +1268,49 @@ class TestProxyRewriting:
class TestPickBestNode:
"""Test the _pick_best_node helper."""
@staticmethod
def _mock_collector(nodes: list) -> MagicMock:
collector = MagicMock(spec=ClusterCollector)
collector.get_nodes.return_value = (nodes, len(nodes))
collector.get_all_nodes.side_effect = lambda: collector.get_nodes.return_value[0]
return collector
def test_picks_node_with_most_headroom(self):
from turnstone.console.server import _pick_best_node
collector = MagicMock(spec=ClusterCollector)
collector.get_nodes.return_value = (
collector = self._mock_collector(
[
{"node_id": "busy", "reachable": True, "max_ws": 10, "ws_total": 9},
{"node_id": "free", "reachable": True, "max_ws": 10, "ws_total": 2},
{"node_id": "mid", "reachable": True, "max_ws": 10, "ws_total": 5},
],
3,
]
)
assert _pick_best_node(collector) == "free"
def test_skips_unreachable_nodes(self):
from turnstone.console.server import _pick_best_node
collector = MagicMock(spec=ClusterCollector)
collector.get_nodes.return_value = (
collector = self._mock_collector(
[
{"node_id": "down", "reachable": False, "max_ws": 10, "ws_total": 0},
{"node_id": "up", "reachable": True, "max_ws": 10, "ws_total": 5},
],
2,
]
)
assert _pick_best_node(collector) == "up"
def test_returns_empty_when_no_nodes(self):
from turnstone.console.server import _pick_best_node
collector = MagicMock(spec=ClusterCollector)
collector.get_nodes.return_value = ([], 0)
collector = self._mock_collector([])
assert _pick_best_node(collector) == ""
def test_returns_empty_when_all_unreachable(self):
from turnstone.console.server import _pick_best_node
collector = MagicMock(spec=ClusterCollector)
collector.get_nodes.return_value = (
[{"node_id": "down", "reachable": False, "max_ws": 10, "ws_total": 0}],
1,
collector = self._mock_collector(
[
{"node_id": "down", "reachable": False, "max_ws": 10, "ws_total": 0},
]
)
assert _pick_best_node(collector) == ""
@@ -1695,6 +1699,236 @@ class TestSSEProxy:
asyncio.run(_run())
# ---------------------------------------------------------------------------
# Proxy auth header propagation
# ---------------------------------------------------------------------------
class TestProxyAuthHeaders:
"""Verify _proxy_auth_headers mints user-scoped JWTs for proxy requests."""
SECRET = "test-secret-that-is-at-least-32-chars"
def _make_request(
self, *, auth_result=None, jwt_secret="", proxy_token_mgr=None, proxy_auth_token=""
):
"""Build a minimal fake request for _proxy_auth_headers."""
class _State:
pass
class _AppState:
pass
class _App:
state = _AppState()
class _Request:
state = _State()
app = _App()
req = _Request()
req.state.auth_result = auth_result
req.app.state.jwt_secret = jwt_secret
req.app.state.proxy_token_mgr = proxy_token_mgr
req.app.state.proxy_auth_token = proxy_auth_token
return req
def test_mints_user_jwt(self):
"""Real user auth_result → JWT with correct sub, scopes, src, aud, permissions."""
import jwt as pyjwt
from turnstone.console.server import _proxy_auth_headers
from turnstone.core.auth import JWT_AUD_SERVER, AuthResult
auth = AuthResult(
user_id="alice",
scopes=frozenset({"read", "write"}),
token_source="jwt",
permissions=frozenset({"admin.users"}),
)
req = self._make_request(auth_result=auth, jwt_secret=self.SECRET)
headers = _proxy_auth_headers(req)
assert "Authorization" in headers
token = headers["Authorization"].removeprefix("Bearer ")
payload = pyjwt.decode(token, self.SECRET, algorithms=["HS256"], audience=JWT_AUD_SERVER)
assert payload["sub"] == "alice"
assert set(payload["scopes"].split(",")) == {"read", "write"}
assert payload["src"] == "console-proxy"
assert payload["aud"] == JWT_AUD_SERVER
assert payload["permissions"] == "admin.users"
def test_narrows_scopes(self):
"""Read-only user → JWT carries only read scope, not full {read,write,approve}."""
import jwt as pyjwt
from turnstone.console.server import _proxy_auth_headers
from turnstone.core.auth import JWT_AUD_SERVER, AuthResult
auth = AuthResult(
user_id="viewer",
scopes=frozenset({"read"}),
token_source="jwt",
)
req = self._make_request(auth_result=auth, jwt_secret=self.SECRET)
headers = _proxy_auth_headers(req)
token = headers["Authorization"].removeprefix("Bearer ")
payload = pyjwt.decode(token, self.SECRET, algorithms=["HS256"], audience=JWT_AUD_SERVER)
assert payload["scopes"] == "read"
def test_short_expiry(self):
"""Minted JWT expires in 300 seconds, not hours."""
import jwt as pyjwt
from turnstone.console.server import _proxy_auth_headers
from turnstone.core.auth import AuthResult
auth = AuthResult(
user_id="alice",
scopes=frozenset({"read"}),
token_source="jwt",
)
req = self._make_request(auth_result=auth, jwt_secret=self.SECRET)
headers = _proxy_auth_headers(req)
token = headers["Authorization"].removeprefix("Bearer ")
payload = pyjwt.decode(
token, self.SECRET, algorithms=["HS256"], options={"verify_aud": False}
)
assert payload["exp"] - payload["iat"] == 300
def test_fallback_no_user(self):
"""No auth_result → falls back to ServiceTokenManager."""
from turnstone.console.server import _proxy_auth_headers
from turnstone.core.auth import ServiceTokenManager
mgr = ServiceTokenManager(
user_id="console-proxy",
scopes=frozenset({"read", "write", "approve"}),
source="console",
secret=self.SECRET,
)
req = self._make_request(proxy_token_mgr=mgr)
headers = _proxy_auth_headers(req)
assert "Authorization" in headers
assert headers["Authorization"] == f"Bearer {mgr.token}"
def test_fallback_no_secret(self):
"""auth_result present but empty jwt_secret → falls back to ServiceTokenManager."""
from turnstone.console.server import _proxy_auth_headers
from turnstone.core.auth import AuthResult, ServiceTokenManager
auth = AuthResult(
user_id="alice",
scopes=frozenset({"read"}),
token_source="jwt",
)
mgr = ServiceTokenManager(
user_id="console-proxy",
scopes=frozenset({"read", "write", "approve"}),
source="console",
secret=self.SECRET,
)
req = self._make_request(auth_result=auth, jwt_secret="", proxy_token_mgr=mgr)
headers = _proxy_auth_headers(req)
# Should use ServiceTokenManager, not mint a user JWT
assert headers["Authorization"] == f"Bearer {mgr.token}"
def test_fallback_static_token(self):
"""No auth_result, no ServiceTokenManager → uses static proxy_auth_token."""
from turnstone.console.server import _proxy_auth_headers
req = self._make_request(proxy_auth_token="static-tok-123")
headers = _proxy_auth_headers(req)
assert headers == {"Authorization": "Bearer static-tok-123"}
# ---------------------------------------------------------------------------
# Server: trusted user_id forwarding on create_workstream
# ---------------------------------------------------------------------------
class TestCreateWorkstreamUserIdTrust:
"""Verify that only trusted service tokens can forward user_id in create_workstream."""
def _extract_uid(self, body: dict, auth_result) -> str:
"""Replicate the trust check from server.py:create_workstream."""
auth = auth_result
uid: str = getattr(auth, "user_id", "") or ""
trusted_sources = {"bridge", "console"}
if (
body.get("user_id")
and isinstance(body["user_id"], str)
and auth is not None
and auth.token_source in trusted_sources
):
uid = body["user_id"]
return uid
def test_bridge_can_forward_user_id(self):
from turnstone.core.auth import AuthResult
auth = AuthResult(
user_id="bridge",
scopes=frozenset({"approve"}),
token_source="bridge",
)
uid = self._extract_uid({"user_id": "real-user-abc"}, auth)
assert uid == "real-user-abc"
def test_console_service_can_forward_user_id(self):
from turnstone.core.auth import AuthResult
auth = AuthResult(
user_id="console",
scopes=frozenset({"approve"}),
token_source="console",
)
uid = self._extract_uid({"user_id": "real-user-abc"}, auth)
assert uid == "real-user-abc"
def test_console_proxy_user_cannot_override_user_id(self):
"""End-user tokens via console-proxy must NOT override user_id."""
from turnstone.core.auth import AuthResult
auth = AuthResult(
user_id="real-user-abc",
scopes=frozenset({"read", "write"}),
token_source="console-proxy",
)
uid = self._extract_uid({"user_id": "impersonated-user"}, auth)
# Should use JWT identity, NOT the body override
assert uid == "real-user-abc"
def test_direct_user_cannot_override_user_id(self):
"""Direct JWT login must NOT override user_id."""
from turnstone.core.auth import AuthResult
auth = AuthResult(
user_id="real-user-abc",
scopes=frozenset({"read", "write"}),
token_source="password",
)
uid = self._extract_uid({"user_id": "impersonated-user"}, auth)
assert uid == "real-user-abc"
def test_no_body_user_id_uses_jwt(self):
from turnstone.core.auth import AuthResult
auth = AuthResult(
user_id="bridge",
scopes=frozenset({"approve"}),
token_source="bridge",
)
uid = self._extract_uid({"name": "test-ws"}, auth)
assert uid == "bridge"
# ---------------------------------------------------------------------------
# Collector — MCP aggregation in get_overview()
# ---------------------------------------------------------------------------
+149
View File
@@ -0,0 +1,149 @@
"""Tests for turnstone.core.env — subprocess environment scrubbing."""
from __future__ import annotations
import os
from unittest.mock import patch
from turnstone.core.env import _is_safe, _is_secret, scrubbed_env
class TestIsSecret:
def test_explicit_scrub_list(self):
assert _is_secret("OPENAI_API_KEY") is True
assert _is_secret("ANTHROPIC_API_KEY") is True
assert _is_secret("TURNSTONE_JWT_SECRET") is True
assert _is_secret("AWS_SECRET_ACCESS_KEY") is True
def test_suffix_matching(self):
assert _is_secret("MY_CUSTOM_API_KEY") is True
assert _is_secret("DB_PASSWORD") is True
assert _is_secret("AUTH_TOKEN") is True
assert _is_secret("SERVICE_CREDENTIAL") is True
assert _is_secret("GCP_CREDENTIALS") is True
def test_safe_vars_not_secret(self):
assert _is_secret("PATH") is False
assert _is_secret("HOME") is False
assert _is_secret("LANG") is False
def test_no_false_positives_on_substring(self):
"""Suffix matching avoids false positives like MONKEYTYPE."""
assert _is_secret("MONKEYTYPE") is False
assert _is_secret("KEYBOARD_LAYOUT") is False
assert _is_secret("PYTHONPATH") is False
assert _is_secret("EDITOR") is False
assert _is_secret("GOPATH") is False
class TestIsSafe:
def test_safe_names(self):
assert _is_safe("PATH") is True
assert _is_safe("HOME") is True
assert _is_safe("TERM") is True
assert _is_safe("MANWIDTH") is True
def test_safe_prefixes(self):
assert _is_safe("LC_ALL") is True
assert _is_safe("LC_CTYPE") is True
assert _is_safe("XDG_RUNTIME_DIR") is True
def test_non_safe_names(self):
assert _is_safe("OPENAI_API_KEY") is False
assert _is_safe("CUSTOM_VAR") is False
class TestScrubbedEnv:
def test_strips_api_keys(self):
fake_env = {
"PATH": "/usr/bin",
"HOME": "/home/user",
"OPENAI_API_KEY": "sk-secret",
"ANTHROPIC_API_KEY": "ant-secret",
"CUSTOM_VAR": "safe_value",
}
with patch.dict(os.environ, fake_env, clear=True):
result = scrubbed_env()
assert result["PATH"] == "/usr/bin"
assert result["HOME"] == "/home/user"
assert result["CUSTOM_VAR"] == "safe_value"
assert "OPENAI_API_KEY" not in result
assert "ANTHROPIC_API_KEY" not in result
def test_strips_pattern_matched_secrets(self):
fake_env = {
"PATH": "/usr/bin",
"MY_SERVICE_TOKEN": "tok-123",
"DB_PASSWORD": "pass123",
}
with patch.dict(os.environ, fake_env, clear=True):
result = scrubbed_env()
assert "MY_SERVICE_TOKEN" not in result
assert "DB_PASSWORD" not in result
def test_extra_vars_merged(self):
fake_env = {"PATH": "/usr/bin"}
with patch.dict(os.environ, fake_env, clear=True):
result = scrubbed_env(extra={"MANWIDTH": "80"})
assert result["MANWIDTH"] == "80"
assert result["PATH"] == "/usr/bin"
def test_passthrough_overrides_scrub(self):
fake_env = {
"PATH": "/usr/bin",
"OPENAI_API_KEY": "sk-needed",
}
with patch.dict(os.environ, fake_env, clear=True):
result = scrubbed_env(passthrough=["OPENAI_API_KEY"])
assert result["OPENAI_API_KEY"] == "sk-needed"
def test_preserves_locale_vars(self):
fake_env = {
"PATH": "/usr/bin",
"LC_ALL": "en_US.UTF-8",
"LC_CTYPE": "en_US.UTF-8",
}
with patch.dict(os.environ, fake_env, clear=True):
result = scrubbed_env()
assert result["LC_ALL"] == "en_US.UTF-8"
assert result["LC_CTYPE"] == "en_US.UTF-8"
def test_preserves_unknown_non_secret_vars(self):
fake_env = {
"PATH": "/usr/bin",
"PYTHONPATH": "/opt/lib",
"GOPATH": "/home/user/go",
}
with patch.dict(os.environ, fake_env, clear=True):
result = scrubbed_env()
assert result["PYTHONPATH"] == "/opt/lib"
assert result["GOPATH"] == "/home/user/go"
def test_extra_can_reintroduce_scrubbed_var(self):
"""extra= intentionally overrides scrubbing (operator-controlled)."""
fake_env = {"PATH": "/usr/bin", "OPENAI_API_KEY": "sk-original"}
with patch.dict(os.environ, fake_env, clear=True):
result = scrubbed_env(extra={"OPENAI_API_KEY": "sk-injected"})
assert result["OPENAI_API_KEY"] == "sk-injected"
def test_less_prefix_does_not_leak_secrets(self):
"""LESS pager vars are safe but LESS_SECRET_TOKEN is not."""
fake_env = {
"PATH": "/usr/bin",
"LESS": "-R",
"LESSOPEN": "| lesspipe %s",
"LESS_SECRET_TOKEN": "tok-secret",
}
with patch.dict(os.environ, fake_env, clear=True):
result = scrubbed_env()
assert result["LESS"] == "-R"
assert result["LESSOPEN"] == "| lesspipe %s"
assert "LESS_SECRET_TOKEN" not in result
-10
View File
@@ -8,18 +8,8 @@ from __future__ import annotations
from datetime import UTC, datetime
import pytest
import sqlalchemy as sa
from turnstone.core.storage._sqlite import SQLiteBackend
@pytest.fixture()
def db(tmp_path):
"""Create a fresh SQLite backend for each test."""
return SQLiteBackend(str(tmp_path / "test.db"))
# ---------------------------------------------------------------------------
# Roles
# ---------------------------------------------------------------------------
+57
View File
@@ -221,6 +221,63 @@ class TestBackendHealthMonitor:
mon.record_failure()
assert mon.circuit_state == CircuitState.OPEN # type: ignore[comparison-overlap]
def test_probe_loop_autonomous_recovery(
self, mock_client: MagicMock, mock_metrics: MagicMock
) -> None:
"""_probe_loop transitions OPEN → HALF_OPEN → CLOSED without user requests."""
# Use very short intervals so the test is fast
mon = BackendHealthMonitor(
client=mock_client,
probe_interval=0.05,
probe_timeout=1.0,
failure_threshold=1,
cooldown=0.1,
)
# Trip the circuit
mon.record_failure()
assert mon.circuit_state == CircuitState.OPEN
# Backend is healthy — probe_once will succeed
mock_client.with_options.return_value.models.list.return_value = MagicMock()
# Start the probe loop and wait for autonomous recovery
mon.start()
try:
import time
deadline = time.monotonic() + 5.0
while mon.circuit_state != CircuitState.CLOSED and time.monotonic() < deadline:
time.sleep(0.05)
assert mon.circuit_state == CircuitState.CLOSED
# User requests should flow again without anyone calling acquire_request_permit
assert mon.acquire_request_permit() is True
finally:
mon.stop()
if mon._thread:
mon._thread.join(timeout=2.0)
def test_probe_loop_no_user_permit_during_probe(
self, mock_client: MagicMock, mock_metrics: MagicMock
) -> None:
"""While background probe is in HALF_OPEN, user requests are blocked."""
mon = BackendHealthMonitor(
client=mock_client,
probe_interval=0.05,
probe_timeout=1.0,
failure_threshold=1,
cooldown=0.1,
)
mon.record_failure()
assert mon.circuit_state == CircuitState.OPEN
# Force into HALF_OPEN as the probe loop would
with mon._lock:
mon._state = CircuitState.HALF_OPEN
mon._half_open_permit = False # probe consumes it
# User requests should be blocked — only the probe gets through
assert mon.acquire_request_permit() is False
def test_stop_thread(self, mock_client: MagicMock) -> None:
"""stop() signals the probe loop to exit."""
mon = _make_monitor(mock_client)
-10
View File
@@ -4,16 +4,6 @@ from __future__ import annotations
from datetime import UTC, datetime, timedelta
import pytest
from turnstone.core.storage._sqlite import SQLiteBackend
@pytest.fixture()
def db(tmp_path):
"""Fresh SQLite backend for each test."""
return SQLiteBackend(str(tmp_path / "test.db"))
def _make_verdict_kwargs(**overrides):
"""Build default kwargs for create_intent_verdict."""
+196
View File
@@ -26,6 +26,7 @@ from turnstone.console.server import (
admin_get_mcp_server,
admin_import_mcp_config,
admin_list_mcp_servers,
admin_mcp_reload,
admin_update_mcp_server,
)
from turnstone.core.auth import AuthResult
@@ -96,6 +97,11 @@ _ROUTES = [
admin_import_mcp_config,
methods=["POST"],
),
Route(
"/api/admin/mcp-servers/reload",
admin_mcp_reload,
methods=["POST"],
),
Route(
"/api/admin/mcp-servers/{server_id}",
admin_get_mcp_server,
@@ -115,6 +121,21 @@ _ROUTES = [
]
def _routes_with_internal() -> list[Mount]:
"""Routes including the node-side internal endpoint (lazy-imported)."""
from turnstone.server import internal_mcp_reload
return [
Mount(
"/v1",
routes=[
*_ROUTES[0].routes, # type: ignore[union-attr]
Route("/api/_internal/mcp-reload", internal_mcp_reload, methods=["POST"]),
],
),
]
@pytest.fixture
def storage(tmp_path):
return SQLiteBackend(str(tmp_path / "test.db"))
@@ -550,8 +571,11 @@ def _fake_request(*nodes: dict[str, Any], proxy_client: Any = None) -> MagicMock
"""Build a minimal mock request with collector and proxy_client."""
collector = MagicMock()
collector.get_nodes.return_value = (list(nodes), len(nodes))
collector.get_all_nodes.side_effect = lambda: collector.get_nodes.return_value[0]
req = MagicMock()
req.state.auth_result = None
req.app.state.collector = collector
req.app.state.jwt_secret = ""
req.app.state.proxy_client = proxy_client or AsyncMock()
req.app.state.proxy_token_mgr = None
req.app.state.proxy_auth_token = "tok"
@@ -693,3 +717,175 @@ class TestNotifyNodesMcpReload:
result = await _notify_nodes_mcp_reload(req)
assert result["n1"] == {"reloaded": 2}
assert "error" in result["n2"]
# ---------------------------------------------------------------------------
# Console reload endpoint: POST /v1/api/admin/mcp-servers/reload
# ---------------------------------------------------------------------------
class TestAdminMcpReloadEndpoint:
"""HTTP-level tests for the console reload endpoint."""
def test_reload_success(self, client: TestClient) -> None:
"""Reload endpoint returns status ok and fan-out results."""
with patch(
"turnstone.console.server._notify_nodes_mcp_reload",
new_callable=AsyncMock,
return_value={"n1": {"reloaded": 3}},
):
r = client.post("/v1/api/admin/mcp-servers/reload")
assert r.status_code == 200
data = r.json()
assert data["status"] == "ok"
assert data["results"] == {"n1": {"reloaded": 3}}
def test_reload_empty_cluster(self, client: TestClient) -> None:
"""Reload with no nodes returns empty results."""
with patch(
"turnstone.console.server._notify_nodes_mcp_reload",
new_callable=AsyncMock,
return_value={},
):
r = client.post("/v1/api/admin/mcp-servers/reload")
assert r.status_code == 200
data = r.json()
assert data["status"] == "ok"
assert data["results"] == {}
def test_reload_permission_denied(self, client_no_perm: TestClient) -> None:
"""Reload without admin.mcp permission is rejected."""
r = client_no_perm.post("/v1/api/admin/mcp-servers/reload")
assert r.status_code == 403
assert "admin.mcp" in r.json()["error"]
def test_reload_no_storage(self) -> None:
"""Reload returns 503 when auth_storage is not available."""
app = Starlette(
routes=_ROUTES,
middleware=[Middleware(_InjectAuthMiddleware)],
)
# Deliberately omit app.state.auth_storage
no_storage_client = TestClient(app, raise_server_exceptions=False)
r = no_storage_client.post("/v1/api/admin/mcp-servers/reload")
assert r.status_code == 503
def test_reload_mixed_node_results(self, client: TestClient) -> None:
"""Reload propagates per-node errors in results."""
with patch(
"turnstone.console.server._notify_nodes_mcp_reload",
new_callable=AsyncMock,
return_value={
"n1": {"reloaded": 2},
"n2": {"error": "Connection refused"},
},
):
r = client.post("/v1/api/admin/mcp-servers/reload")
assert r.status_code == 200
data = r.json()
assert data["results"]["n1"] == {"reloaded": 2}
assert "error" in data["results"]["n2"]
# ---------------------------------------------------------------------------
# Node reload endpoint: POST /v1/api/_internal/mcp-reload
# ---------------------------------------------------------------------------
class TestInternalMcpReloadEndpoint:
"""HTTP-level tests for the node-side MCP reload endpoint."""
@pytest.fixture()
def node_client(self, storage: SQLiteBackend) -> TestClient:
"""TestClient with an MCP client manager on app.state."""
app = Starlette(
routes=_routes_with_internal(),
middleware=[Middleware(_InjectAuthMiddleware)],
)
app.state.auth_storage = storage
mgr = MagicMock()
mgr.reconcile_sync.return_value = {
"added": ["new-srv"],
"removed": [],
"updated": [],
}
app.state.mcp_client = mgr
return TestClient(app, raise_server_exceptions=False)
def test_reload_calls_reconcile(self, node_client: TestClient, storage: SQLiteBackend) -> None:
"""Reload endpoint calls reconcile_sync and returns its result."""
with patch("turnstone.core.storage._registry.get_storage", return_value=storage):
r = node_client.post("/v1/api/_internal/mcp-reload")
assert r.status_code == 200
data = r.json()
assert data["status"] == "ok"
assert data["added"] == ["new-srv"]
assert data["removed"] == []
assert data["updated"] == []
def test_reload_passes_storage_to_reconcile(
self,
storage: SQLiteBackend,
) -> None:
"""Verify reconcile_sync receives the storage backend."""
app = Starlette(
routes=_routes_with_internal(),
middleware=[Middleware(_InjectAuthMiddleware)],
)
app.state.auth_storage = storage
mgr = MagicMock()
mgr.reconcile_sync.return_value = {"added": [], "removed": [], "updated": []}
app.state.mcp_client = mgr
c = TestClient(app, raise_server_exceptions=False)
with patch("turnstone.core.storage._registry.get_storage", return_value=storage):
r = c.post("/v1/api/_internal/mcp-reload")
assert r.status_code == 200
mgr.reconcile_sync.assert_called_once_with(storage)
def test_reload_creates_manager_when_missing(self, storage: SQLiteBackend) -> None:
"""When mcp_client is absent, a new MCPClientManager is created."""
app = Starlette(
routes=_routes_with_internal(),
middleware=[Middleware(_InjectAuthMiddleware)],
)
app.state.auth_storage = storage
# No mcp_client on app.state
c = TestClient(app, raise_server_exceptions=False)
with (
patch("turnstone.core.storage._registry.get_storage", return_value=storage),
patch("turnstone.core.mcp_client.MCPClientManager") as mock_cls,
):
mock_mgr = MagicMock()
mock_mgr.reconcile_sync.return_value = {
"added": [],
"removed": [],
"updated": [],
}
mock_cls.return_value = mock_mgr
r = c.post("/v1/api/_internal/mcp-reload")
assert r.status_code == 200
mock_cls.assert_called_once_with({})
mock_mgr.start.assert_called_once()
mock_mgr.reconcile_sync.assert_called_once_with(storage)
def test_reload_reconcile_result_in_response(self, storage: SQLiteBackend) -> None:
"""Full reconcile result fields (added/removed/updated) appear in JSON."""
app = Starlette(
routes=_routes_with_internal(),
middleware=[Middleware(_InjectAuthMiddleware)],
)
app.state.auth_storage = storage
mgr = MagicMock()
mgr.reconcile_sync.return_value = {
"added": ["a"],
"removed": ["b"],
"updated": ["c"],
}
app.state.mcp_client = mgr
c = TestClient(app, raise_server_exceptions=False)
with patch("turnstone.core.storage._registry.get_storage", return_value=storage):
r = c.post("/v1/api/_internal/mcp-reload")
data = r.json()
assert data["added"] == ["a"]
assert data["removed"] == ["b"]
assert data["updated"] == ["c"]
+66
View File
@@ -398,6 +398,72 @@ class TestResolveInstallConfig:
config = resolve_install_config(server, "remote", 0)
assert config["url"] == "https://us-east.example.com/mcp"
def test_remote_variable_substitution_invalid_scheme(self) -> None:
server = RegistryServer(
name="io.example/test",
version="1.0.0",
remotes=[
RegistryRemote(
type="streamable-http",
url="{scheme}://evil.example.com/mcp",
variables={
"scheme": RegistryRemoteVariable(is_required=True),
},
)
],
)
with pytest.raises(MCPRegistryError, match="Invalid URL scheme"):
resolve_install_config(server, "remote", 0, variables={"scheme": "file"})
def test_remote_variable_substitution_preserves_valid_scheme(self) -> None:
server = RegistryServer(
name="io.example/test",
version="1.0.0",
remotes=[
RegistryRemote(
type="streamable-http",
url="https://{host}.example.com/mcp",
variables={
"host": RegistryRemoteVariable(is_required=True),
},
)
],
)
config = resolve_install_config(server, "remote", 0, variables={"host": "api"})
assert config["url"] == "https://api.example.com/mcp"
def test_remote_variable_substitution_missing_hostname(self) -> None:
"""URL like https:///mcp has valid scheme but no hostname."""
server = RegistryServer(
name="io.example/test",
version="1.0.0",
remotes=[
RegistryRemote(
type="streamable-http",
url="https:///mcp",
)
],
)
with pytest.raises(MCPRegistryError, match="hostname is missing"):
resolve_install_config(server, "remote", 0)
def test_remote_variable_substitution_embedded_credentials(self) -> None:
server = RegistryServer(
name="io.example/test",
version="1.0.0",
remotes=[
RegistryRemote(
type="streamable-http",
url="https://{creds}@example.com/mcp",
variables={
"creds": RegistryRemoteVariable(is_required=True),
},
)
],
)
with pytest.raises(MCPRegistryError, match="embedded credentials"):
resolve_install_config(server, "remote", 0, variables={"creds": "user:pass"})
def test_remote_no_remotes(self) -> None:
server = RegistryServer(name="io.example/test", version="1.0.0")
with pytest.raises(MCPRegistryError, match="no remote"):
+74 -2
View File
@@ -4,7 +4,7 @@ from __future__ import annotations
import json
from typing import TYPE_CHECKING, Any
from unittest.mock import AsyncMock, patch
from unittest.mock import AsyncMock, MagicMock, patch
import pytest
from starlette.applications import Starlette
@@ -18,11 +18,13 @@ if TYPE_CHECKING:
from starlette.responses import Response
from turnstone.console.server import (
_get_registry_url,
admin_registry_install,
admin_registry_search,
)
from turnstone.core.auth import AuthResult
from turnstone.core.mcp_registry import (
DEFAULT_REGISTRY_URL,
MCPRegistryError,
RegistryPackage,
RegistryRemote,
@@ -362,7 +364,7 @@ class TestRegistryInstall:
def test_install_max_servers(self, client: TestClient, storage: SQLiteBackend) -> None:
import uuid
for i in range(50):
for i in range(200):
storage.create_mcp_server(
server_id=uuid.uuid4().hex,
name=f"server-{i}",
@@ -549,3 +551,73 @@ class TestRegistryInstall:
assert resp.status_code == 409
assert "custom 'name'" in resp.json()["error"]
# ---------------------------------------------------------------------------
# _get_registry_url fallback chain tests
# ---------------------------------------------------------------------------
def _mock_request(storage: Any = None, config_store: Any = None) -> MagicMock:
"""Build a mock Request with app.state.auth_storage and app.state.config_store."""
request = MagicMock()
request.app.state.auth_storage = storage
request.app.state.config_store = config_store
return request
class TestGetRegistryUrl:
"""Verify three-tier URL resolution: DB setting -> config.toml -> default."""
def test_returns_db_setting_when_available(self) -> None:
config_store = MagicMock()
config_store.get.return_value = "https://custom.registry.example.com"
request = _mock_request(config_store=config_store)
with patch("turnstone.core.config.load_config", return_value={}):
url = _get_registry_url(request)
assert url == "https://custom.registry.example.com"
config_store.get.assert_called_once_with("mcp.registry_url")
def test_falls_back_to_config_when_config_store_returns_empty(self) -> None:
config_store = MagicMock()
config_store.get.return_value = ""
request = _mock_request(config_store=config_store)
with patch(
"turnstone.core.config.load_config",
return_value={"registry_url": "https://config.registry.example.com"},
):
url = _get_registry_url(request)
assert url == "https://config.registry.example.com"
def test_falls_back_to_config_when_no_config_store(self) -> None:
request = _mock_request()
with patch(
"turnstone.core.config.load_config",
return_value={"registry_url": "https://config.registry.example.com"},
):
url = _get_registry_url(request)
assert url == "https://config.registry.example.com"
def test_falls_back_to_default_when_both_unavailable(self) -> None:
config_store = MagicMock()
config_store.get.return_value = ""
request = _mock_request(config_store=config_store)
with patch("turnstone.core.config.load_config", return_value={}):
url = _get_registry_url(request)
assert url == DEFAULT_REGISTRY_URL
def test_falls_back_to_default_when_no_config_store_or_config(self) -> None:
request = _mock_request()
with patch("turnstone.core.config.load_config", return_value={}):
url = _get_registry_url(request)
assert url == DEFAULT_REGISTRY_URL
+3 -9
View File
@@ -3,16 +3,10 @@
from __future__ import annotations
import uuid
from typing import TYPE_CHECKING
import pytest
from turnstone.core.storage._sqlite import SQLiteBackend
@pytest.fixture
def db(tmp_path):
"""Fresh SQLite backend for each test."""
return SQLiteBackend(str(tmp_path / "test.db"))
if TYPE_CHECKING:
from turnstone.core.storage._sqlite import SQLiteBackend
def _make_id() -> str:
+34
View File
@@ -10,6 +10,7 @@ import pytest
from turnstone.core.model_registry import (
ModelConfig,
ModelRegistry,
detect_model,
load_model_registry,
)
@@ -590,3 +591,36 @@ class TestProtocolModel:
assert isinstance(restored, CreateWorkstreamMessage)
assert restored.model == "local"
assert restored.name == "ws1"
# ---------------------------------------------------------------------------
# detect_model — startup timeout
# ---------------------------------------------------------------------------
class TestDetectModelTimeout:
def test_uses_short_timeout_and_no_retries(self) -> None:
"""detect_model() uses with_options(timeout=10, max_retries=0)."""
mock_model = MagicMock()
mock_model.id = "test-model"
mock_model.owned_by = "test"
fast_client = MagicMock()
fast_client.models.list.return_value = MagicMock(data=[mock_model])
client = MagicMock()
client.with_options.return_value = fast_client
result = detect_model(client, provider="openai")
client.with_options.assert_called_once_with(timeout=10.0, max_retries=0)
fast_client.models.list.assert_called_once()
assert result[0] == "test-model"
def test_connection_error_non_fatal(self) -> None:
"""detect_model(fatal=False) returns (None, None) on connection error."""
client = MagicMock()
client.with_options.return_value = client
client.models.list.side_effect = OSError("Connection refused")
result = detect_model(client, provider="openai", fatal=False)
assert result == (None, None)
+172 -3
View File
@@ -22,6 +22,7 @@ from turnstone.core.oidc import (
load_oidc_config,
provision_oidc_user,
validate_id_token,
validate_issuer_url,
)
# ---------------------------------------------------------------------------
@@ -293,6 +294,162 @@ class TestLoadOIDCConfig:
assert cfg.redirect_base == "http://localhost:8000"
# ---------------------------------------------------------------------------
# SSRF Validation
# ---------------------------------------------------------------------------
class TestValidateIssuerURL:
"""Tests for ``validate_issuer_url`` SSRF protection."""
def test_valid_https_url(self):
"""Public HTTPS issuer URL passes validation."""
# Should not raise -- mock DNS to return a public IP.
with patch(
"socket.getaddrinfo",
return_value=[
(2, 1, 6, "", ("93.184.216.34", 0)),
],
):
validate_issuer_url("https://idp.example.com")
def test_rejects_http_non_localhost(self):
"""HTTP is rejected for non-localhost hosts."""
with pytest.raises(OIDCError, match="must use HTTPS"):
validate_issuer_url("http://idp.example.com")
def test_allows_http_localhost(self):
"""HTTP is allowed for localhost (development)."""
with patch(
"socket.getaddrinfo",
return_value=[
(2, 1, 6, "", ("127.0.0.1", 0)),
],
):
validate_issuer_url("http://localhost:8080")
def test_allows_http_localhost_subdomain(self):
"""HTTP is allowed for *.localhost subdomains."""
with patch(
"socket.getaddrinfo",
return_value=[
(2, 1, 6, "", ("127.0.0.1", 0)),
],
):
validate_issuer_url("http://keycloak.localhost:8080")
def test_rejects_embedded_credentials(self):
"""URLs with userinfo (user:pass@host) are rejected."""
with pytest.raises(OIDCError, match="embedded credentials"):
validate_issuer_url("https://admin:secret@idp.example.com")
def test_rejects_username_only(self):
"""URLs with just a username are rejected."""
with pytest.raises(OIDCError, match="embedded credentials"):
validate_issuer_url("https://admin@idp.example.com")
def test_rejects_no_hostname(self):
"""URLs without a hostname are rejected."""
with pytest.raises(OIDCError, match="no hostname"):
validate_issuer_url("https://")
def test_rejects_private_10_range(self):
"""Hostnames resolving to 10.x.x.x are rejected."""
with (
patch("socket.getaddrinfo", return_value=[(2, 1, 6, "", ("10.0.0.1", 0))]),
pytest.raises(OIDCError, match="non-public address.*10.0.0.1"),
):
validate_issuer_url("https://internal.corp.example.com")
def test_rejects_private_172_range(self):
"""Hostnames resolving to 172.16-31.x.x are rejected."""
with (
patch("socket.getaddrinfo", return_value=[(2, 1, 6, "", ("172.16.0.1", 0))]),
pytest.raises(OIDCError, match="non-public address.*172.16.0.1"),
):
validate_issuer_url("https://internal.corp.example.com")
def test_rejects_private_192_168_range(self):
"""Hostnames resolving to 192.168.x.x are rejected."""
with (
patch("socket.getaddrinfo", return_value=[(2, 1, 6, "", ("192.168.1.1", 0))]),
pytest.raises(OIDCError, match="non-public address.*192.168.1.1"),
):
validate_issuer_url("https://internal.corp.example.com")
def test_rejects_loopback_127(self):
"""Hostnames resolving to 127.x.x.x are rejected (non-localhost host)."""
with (
patch("socket.getaddrinfo", return_value=[(2, 1, 6, "", ("127.0.0.1", 0))]),
pytest.raises(OIDCError, match="non-public address.*127.0.0.1"),
):
validate_issuer_url("https://evil.example.com")
def test_rejects_ipv6_loopback(self):
"""Hostnames resolving to ::1 are rejected (non-localhost host)."""
with (
patch("socket.getaddrinfo", return_value=[(10, 1, 6, "", ("::1", 0, 0, 0))]),
pytest.raises(OIDCError, match="non-public address.*::1"),
):
validate_issuer_url("https://evil.example.com")
def test_rejects_ipv6_private(self):
"""Hostnames resolving to fc00::/7 are rejected."""
with (
patch("socket.getaddrinfo", return_value=[(10, 1, 6, "", ("fd00::1", 0, 0, 0))]),
pytest.raises(OIDCError, match="non-public address.*fd00::1"),
):
validate_issuer_url("https://evil.example.com")
def test_rejects_link_local(self):
"""Hostnames resolving to link-local addresses are rejected."""
with (
patch("socket.getaddrinfo", return_value=[(2, 1, 6, "", ("169.254.169.254", 0))]),
pytest.raises(OIDCError, match="non-public address.*169.254.169.254"),
):
validate_issuer_url("https://metadata.internal")
def test_rejects_unresolvable_hostname(self):
"""DNS resolution failure is rejected."""
import socket as _socket
with (
patch("socket.getaddrinfo", side_effect=_socket.gaierror("not found")),
pytest.raises(OIDCError, match="cannot be resolved"),
):
validate_issuer_url("https://nonexistent.invalid")
def test_rejects_mixed_addresses(self):
"""If any resolved address is private, the URL is rejected."""
with (
patch(
"socket.getaddrinfo",
return_value=[
(2, 1, 6, "", ("93.184.216.34", 0)),
(2, 1, 6, "", ("10.0.0.1", 0)),
],
),
pytest.raises(OIDCError, match="non-public address.*10.0.0.1"),
):
validate_issuer_url("https://dual-homed.example.com")
def test_discover_rejects_ssrf(self):
"""discover_oidc returns enabled=False when issuer URL fails SSRF check."""
config = _make_config(
issuer="http://10.0.0.1:8080",
authorization_endpoint="",
token_endpoint="",
userinfo_endpoint="",
jwks_uri="",
)
async def _run():
result = await discover_oidc(config)
assert result.enabled is False
asyncio.run(_run())
# ---------------------------------------------------------------------------
# Redirect URI Builder
# ---------------------------------------------------------------------------
@@ -869,6 +1026,9 @@ class TestApplyRoleMapping:
class TestDiscoverOIDC:
# Mock DNS result for a public IP — reused across discovery tests.
_PUBLIC_ADDR = [(2, 1, 6, "", ("93.184.216.34", 0))]
def test_discover_oidc_success(self):
"""Mock httpx response, verify endpoints populated."""
config = _make_config(
@@ -891,7 +1051,10 @@ class TestDiscoverOIDC:
async def _run():
client = _mock_async_client(lambda url: _async_return(mock_response))
with patch("httpx.AsyncClient", return_value=client):
with (
patch("socket.getaddrinfo", return_value=self._PUBLIC_ADDR),
patch("httpx.AsyncClient", return_value=client),
):
result = await discover_oidc(config)
assert result.authorization_endpoint == "https://idp.example.com/authorize"
@@ -916,7 +1079,10 @@ class TestDiscoverOIDC:
async def _run():
client = _mock_async_client(_failing_get)
with patch("httpx.AsyncClient", return_value=client):
with (
patch("socket.getaddrinfo", return_value=self._PUBLIC_ADDR),
patch("httpx.AsyncClient", return_value=client),
):
result = await discover_oidc(config)
assert result.enabled is False
@@ -954,7 +1120,10 @@ class TestDiscoverOIDC:
async def _run():
client = _mock_async_client(lambda url: _async_return(mock_response))
with patch("httpx.AsyncClient", return_value=client):
with (
patch("socket.getaddrinfo", return_value=self._PUBLIC_ADDR),
patch("httpx.AsyncClient", return_value=client),
):
result = await discover_oidc(config)
assert result.enabled is False
-9
View File
@@ -6,15 +6,6 @@ import time
import pytest
from turnstone.core.storage._sqlite import SQLiteBackend
@pytest.fixture()
def db(tmp_path):
"""Create a fresh SQLite backend for each test."""
return SQLiteBackend(str(tmp_path / "test.db"))
# ---------------------------------------------------------------------------
# OIDC Identity CRUD
# ---------------------------------------------------------------------------
-10
View File
@@ -4,16 +4,6 @@ from __future__ import annotations
from datetime import UTC, datetime, timedelta
import pytest
from turnstone.core.storage._sqlite import SQLiteBackend
@pytest.fixture()
def db(tmp_path):
"""Fresh SQLite backend for each test."""
return SQLiteBackend(str(tmp_path / "test.db"))
def _make_assessment_kwargs(**overrides):
"""Build default kwargs for record_output_assessment."""
-11
View File
@@ -4,17 +4,6 @@ from __future__ import annotations
import time
import pytest
from turnstone.core.storage._sqlite import SQLiteBackend
@pytest.fixture
def db(tmp_path):
"""Fresh SQLite backend for each test."""
backend = SQLiteBackend(str(tmp_path / "test.db"))
return backend
def _make_task_kwargs(**overrides):
"""Build default kwargs for create_scheduled_task."""
+440
View File
@@ -0,0 +1,440 @@
"""Integration tests for SDK governance methods against a real Starlette app.
Verifies round-trip serialization: SDK -> HTTP -> Starlette handler -> storage
-> JSON response -> Pydantic model validation in the SDK client.
"""
from __future__ import annotations
from typing import TYPE_CHECKING, Any
import httpx
import pytest
from starlette.applications import Starlette
from starlette.middleware import Middleware
from starlette.middleware.base import BaseHTTPMiddleware
from starlette.routing import Mount, Route
if TYPE_CHECKING:
from starlette.requests import Request
from starlette.responses import Response
from turnstone.api.console_schemas import (
ListOrgsResponse,
ListRolesResponse,
ListToolPoliciesResponse,
OrgInfo,
RoleInfo,
ToolPolicyInfo,
)
from turnstone.api.schemas import StatusResponse
from turnstone.console.server import (
admin_create_policy,
admin_create_role,
admin_delete_policy,
admin_delete_role,
admin_get_org,
admin_list_orgs,
admin_list_policies,
admin_list_roles,
admin_update_org,
admin_update_policy,
admin_update_role,
)
from turnstone.core.auth import AuthResult
from turnstone.core.storage._sqlite import SQLiteBackend
from turnstone.sdk.console import AsyncTurnstoneConsole
# ---------------------------------------------------------------------------
# Auth bypass middleware — injects a full-access AuthResult on every request.
# ---------------------------------------------------------------------------
class _InjectAuthMiddleware(BaseHTTPMiddleware):
async def dispatch(self, request: Request, call_next: Any) -> Response:
request.state.auth_result = AuthResult(
user_id="test-admin",
scopes=frozenset({"approve"}),
token_source="config",
permissions=frozenset(
{
"read",
"write",
"approve",
"admin.roles",
"admin.orgs",
"admin.policies",
}
),
)
resp: Response = await call_next(request)
return resp
# ---------------------------------------------------------------------------
# Fixtures
# ---------------------------------------------------------------------------
def _make_app() -> Starlette:
return Starlette(
routes=[
Mount(
"/v1",
routes=[
# Roles
Route("/api/admin/roles", admin_list_roles),
Route("/api/admin/roles", admin_create_role, methods=["POST"]),
Route("/api/admin/roles/{role_id}", admin_update_role, methods=["PUT"]),
Route("/api/admin/roles/{role_id}", admin_delete_role, methods=["DELETE"]),
# Orgs
Route("/api/admin/orgs", admin_list_orgs),
Route("/api/admin/orgs/{org_id}", admin_get_org),
Route("/api/admin/orgs/{org_id}", admin_update_org, methods=["PUT"]),
# Policies
Route("/api/admin/policies", admin_list_policies),
Route("/api/admin/policies", admin_create_policy, methods=["POST"]),
Route(
"/api/admin/policies/{policy_id}",
admin_update_policy,
methods=["PUT"],
),
Route(
"/api/admin/policies/{policy_id}",
admin_delete_policy,
methods=["DELETE"],
),
],
),
],
middleware=[Middleware(_InjectAuthMiddleware)],
)
@pytest.fixture
def storage(tmp_path: Any) -> SQLiteBackend:
return SQLiteBackend(str(tmp_path / "test.db"))
@pytest.fixture
async def sdk_client(storage: SQLiteBackend):
"""SDK client wired to a real Starlette app via ASGITransport."""
app = _make_app()
app.state.auth_storage = storage
transport = httpx.ASGITransport(app=app)
async with httpx.AsyncClient(transport=transport, base_url="http://testserver") as hc:
yield AsyncTurnstoneConsole(httpx_client=hc)
# ---------------------------------------------------------------------------
# Tests — Roles round-trip
# ---------------------------------------------------------------------------
class TestRolesRoundTrip:
@pytest.mark.anyio
async def test_list_roles_empty(self, sdk_client: AsyncTurnstoneConsole) -> None:
resp = await sdk_client.list_roles()
assert isinstance(resp, ListRolesResponse)
assert resp.roles == []
@pytest.mark.anyio
async def test_create_and_list_role(self, sdk_client: AsyncTurnstoneConsole) -> None:
role = await sdk_client.create_role(
"analyst", display_name="Data Analyst", permissions="read,write"
)
assert isinstance(role, RoleInfo)
assert role.name == "analyst"
assert role.display_name == "Data Analyst"
assert role.permissions == "read,write"
assert role.builtin is False
assert role.role_id # non-empty
# List should now contain the new role
resp = await sdk_client.list_roles()
assert len(resp.roles) == 1
assert resp.roles[0].role_id == role.role_id
assert resp.roles[0].name == "analyst"
@pytest.mark.anyio
async def test_create_update_role(self, sdk_client: AsyncTurnstoneConsole) -> None:
role = await sdk_client.create_role("ops", permissions="read")
assert role.permissions == "read"
updated = await sdk_client.update_role(
role.role_id, display_name="Operations", permissions="read,write,approve"
)
assert isinstance(updated, RoleInfo)
assert updated.display_name == "Operations"
assert updated.permissions == "read,write,approve"
assert updated.role_id == role.role_id
@pytest.mark.anyio
async def test_create_delete_role(self, sdk_client: AsyncTurnstoneConsole) -> None:
role = await sdk_client.create_role("temp-role", permissions="read")
result = await sdk_client.delete_role(role.role_id)
assert isinstance(result, StatusResponse)
assert result.status == "ok"
# Verify gone
resp = await sdk_client.list_roles()
assert resp.roles == []
@pytest.mark.anyio
async def test_full_lifecycle(self, sdk_client: AsyncTurnstoneConsole) -> None:
"""Create -> list -> update -> list -> delete -> list."""
# Create
role = await sdk_client.create_role(
"lifecycle", display_name="Lifecycle", permissions="read"
)
role_id = role.role_id
# List confirms creation
roles = (await sdk_client.list_roles()).roles
assert len(roles) == 1
assert roles[0].role_id == role_id
# Update
updated = await sdk_client.update_role(role_id, permissions="read,write")
assert updated.permissions == "read,write"
# List still has one
roles = (await sdk_client.list_roles()).roles
assert len(roles) == 1
assert roles[0].permissions == "read,write"
# Delete
await sdk_client.delete_role(role_id)
# List is empty
roles = (await sdk_client.list_roles()).roles
assert roles == []
@pytest.mark.anyio
async def test_delete_nonexistent_role_raises(self, sdk_client: AsyncTurnstoneConsole) -> None:
from turnstone.sdk._types import TurnstoneAPIError
with pytest.raises(TurnstoneAPIError) as exc_info:
await sdk_client.delete_role("nonexistent")
assert exc_info.value.status_code == 404
@pytest.mark.anyio
async def test_update_nonexistent_role_raises(self, sdk_client: AsyncTurnstoneConsole) -> None:
from turnstone.sdk._types import TurnstoneAPIError
with pytest.raises(TurnstoneAPIError) as exc_info:
await sdk_client.update_role("nonexistent", display_name="Nope")
assert exc_info.value.status_code == 404
# ---------------------------------------------------------------------------
# Tests — Policies round-trip
# ---------------------------------------------------------------------------
class TestPoliciesRoundTrip:
@pytest.mark.anyio
async def test_list_policies_empty(self, sdk_client: AsyncTurnstoneConsole) -> None:
resp = await sdk_client.list_policies()
assert isinstance(resp, ListToolPoliciesResponse)
assert resp.policies == []
@pytest.mark.anyio
async def test_create_and_list_policy(self, sdk_client: AsyncTurnstoneConsole) -> None:
policy = await sdk_client.create_policy("Allow bash", "bash_*", "allow", priority=10)
assert isinstance(policy, ToolPolicyInfo)
assert policy.name == "Allow bash"
assert policy.tool_pattern == "bash_*"
assert policy.action == "allow"
assert policy.priority == 10
assert policy.enabled is True
assert policy.policy_id # non-empty
resp = await sdk_client.list_policies()
assert len(resp.policies) == 1
assert resp.policies[0].policy_id == policy.policy_id
@pytest.mark.anyio
async def test_create_update_policy(self, sdk_client: AsyncTurnstoneConsole) -> None:
policy = await sdk_client.create_policy("Deny write", "write_*", "deny", priority=5)
updated = await sdk_client.update_policy(
policy.policy_id, name="Allow write", action="allow", priority=20
)
assert isinstance(updated, ToolPolicyInfo)
assert updated.name == "Allow write"
assert updated.action == "allow"
assert updated.priority == 20
assert updated.policy_id == policy.policy_id
@pytest.mark.anyio
async def test_create_delete_policy(self, sdk_client: AsyncTurnstoneConsole) -> None:
policy = await sdk_client.create_policy("Temp policy", "temp_*", "ask")
result = await sdk_client.delete_policy(policy.policy_id)
assert isinstance(result, StatusResponse)
assert result.status == "ok"
resp = await sdk_client.list_policies()
assert resp.policies == []
@pytest.mark.anyio
async def test_full_lifecycle(self, sdk_client: AsyncTurnstoneConsole) -> None:
"""Create -> list -> update -> list -> delete -> list."""
policy = await sdk_client.create_policy("Lifecycle", "test_*", "deny", priority=1)
pid = policy.policy_id
policies = (await sdk_client.list_policies()).policies
assert len(policies) == 1
await sdk_client.update_policy(pid, action="allow", priority=99)
policies = (await sdk_client.list_policies()).policies
assert policies[0].action == "allow"
assert policies[0].priority == 99
await sdk_client.delete_policy(pid)
policies = (await sdk_client.list_policies()).policies
assert policies == []
@pytest.mark.anyio
async def test_delete_nonexistent_policy_raises(
self, sdk_client: AsyncTurnstoneConsole
) -> None:
from turnstone.sdk._types import TurnstoneAPIError
with pytest.raises(TurnstoneAPIError) as exc_info:
await sdk_client.delete_policy("nonexistent")
assert exc_info.value.status_code == 404
@pytest.mark.anyio
async def test_update_nonexistent_policy_raises(
self, sdk_client: AsyncTurnstoneConsole
) -> None:
from turnstone.sdk._types import TurnstoneAPIError
with pytest.raises(TurnstoneAPIError) as exc_info:
await sdk_client.update_policy("nonexistent", name="Nope")
assert exc_info.value.status_code == 404
@pytest.mark.anyio
async def test_create_policy_invalid_action_raises(
self, sdk_client: AsyncTurnstoneConsole
) -> None:
from turnstone.sdk._types import TurnstoneAPIError
with pytest.raises(TurnstoneAPIError) as exc_info:
await sdk_client.create_policy("Bad", "tool_*", "yolo")
assert exc_info.value.status_code == 400
# ---------------------------------------------------------------------------
# Tests — Orgs round-trip
# ---------------------------------------------------------------------------
class TestOrgsRoundTrip:
@pytest.mark.anyio
async def test_list_orgs_empty(self, sdk_client: AsyncTurnstoneConsole) -> None:
resp = await sdk_client.list_orgs()
assert isinstance(resp, ListOrgsResponse)
assert resp.orgs == []
@pytest.mark.anyio
async def test_get_org(self, sdk_client: AsyncTurnstoneConsole, storage: SQLiteBackend) -> None:
storage.create_org(
org_id="org-1", name="acme", display_name="Acme Corp", settings='{"k": "v"}'
)
org = await sdk_client.get_org("org-1")
assert isinstance(org, OrgInfo)
assert org.org_id == "org-1"
assert org.name == "acme"
assert org.display_name == "Acme Corp"
assert org.settings == '{"k": "v"}'
@pytest.mark.anyio
async def test_list_orgs_after_seed(
self, sdk_client: AsyncTurnstoneConsole, storage: SQLiteBackend
) -> None:
storage.create_org(org_id="org-a", name="alpha", display_name="Alpha")
storage.create_org(org_id="org-b", name="beta", display_name="Beta")
resp = await sdk_client.list_orgs()
assert len(resp.orgs) == 2
names = {o.name for o in resp.orgs}
assert names == {"alpha", "beta"}
@pytest.mark.anyio
async def test_update_org(
self, sdk_client: AsyncTurnstoneConsole, storage: SQLiteBackend
) -> None:
storage.create_org(org_id="org-1", name="acme", display_name="Acme Corp")
updated = await sdk_client.update_org("org-1", display_name="Acme Inc.")
assert isinstance(updated, OrgInfo)
assert updated.display_name == "Acme Inc."
assert updated.org_id == "org-1"
@pytest.mark.anyio
async def test_get_nonexistent_org_raises(self, sdk_client: AsyncTurnstoneConsole) -> None:
from turnstone.sdk._types import TurnstoneAPIError
with pytest.raises(TurnstoneAPIError) as exc_info:
await sdk_client.get_org("nonexistent")
assert exc_info.value.status_code == 404
@pytest.mark.anyio
async def test_update_nonexistent_org_raises(self, sdk_client: AsyncTurnstoneConsole) -> None:
from turnstone.sdk._types import TurnstoneAPIError
with pytest.raises(TurnstoneAPIError) as exc_info:
await sdk_client.update_org("nonexistent", display_name="Nope")
assert exc_info.value.status_code == 404
# ---------------------------------------------------------------------------
# Tests — Pydantic model field validation
# ---------------------------------------------------------------------------
class TestModelValidation:
"""Verify that all expected fields are populated and correctly typed."""
@pytest.mark.anyio
async def test_role_info_fields(self, sdk_client: AsyncTurnstoneConsole) -> None:
role = await sdk_client.create_role("reviewer", permissions="read")
assert isinstance(role.role_id, str)
assert isinstance(role.name, str)
assert isinstance(role.display_name, str)
assert isinstance(role.permissions, str)
assert isinstance(role.builtin, bool)
assert isinstance(role.org_id, str)
assert isinstance(role.created, str)
assert isinstance(role.updated, str)
@pytest.mark.anyio
async def test_policy_info_fields(self, sdk_client: AsyncTurnstoneConsole) -> None:
policy = await sdk_client.create_policy("Test", "read_*", "allow", priority=5)
assert isinstance(policy.policy_id, str)
assert isinstance(policy.name, str)
assert isinstance(policy.tool_pattern, str)
assert isinstance(policy.action, str)
assert isinstance(policy.priority, int)
assert isinstance(policy.org_id, str)
assert isinstance(policy.enabled, bool)
assert isinstance(policy.created_by, str)
assert isinstance(policy.created, str)
assert isinstance(policy.updated, str)
@pytest.mark.anyio
async def test_org_info_fields(
self, sdk_client: AsyncTurnstoneConsole, storage: SQLiteBackend
) -> None:
storage.create_org(org_id="org-v", name="validate", display_name="Validate")
org = await sdk_client.get_org("org-v")
assert isinstance(org.org_id, str)
assert isinstance(org.name, str)
assert isinstance(org.display_name, str)
assert isinstance(org.settings, str)
assert isinstance(org.created, str)
assert isinstance(org.updated, str)
-9
View File
@@ -2,15 +2,6 @@
from __future__ import annotations
import pytest
from turnstone.core.storage._sqlite import SQLiteBackend
@pytest.fixture
def storage(tmp_path):
return SQLiteBackend(str(tmp_path / "test.db"))
class TestServiceRegistry:
def test_register_and_list(self, storage):
+269 -4
View File
@@ -214,7 +214,7 @@ class TestPlanExec:
"id": tc_id,
"type": "function",
"function": {
"name": "create_plan",
"name": "plan_agent",
"arguments": json.dumps({"goal": prior_prompt}),
},
}
@@ -250,7 +250,7 @@ class TestPlanExec:
m for m in messages if m["role"] == "assistant" and m.get("tool_calls")
]
assert len(assistant_with_tc) == 1
assert assistant_with_tc[0]["tool_calls"][0]["function"]["name"] == "create_plan"
assert assistant_with_tc[0]["tool_calls"][0]["function"]["name"] == "plan_agent"
# The real tool result is forwarded with its original content
tool_msgs = [m for m in messages if m["role"] == "tool"]
@@ -463,7 +463,7 @@ class TestPlanRefinement:
with patch.object(session, "_refine_plan", side_effect=fake_refine):
items = [
{
"func_name": "create_plan",
"func_name": "plan_agent",
"call_id": "c1",
"prompt": "add auth",
}
@@ -576,7 +576,7 @@ class TestPlanRefinement:
msgs = captured["messages"]
assert msgs[0]["role"] == "system"
assert msgs[1]["role"] == "assistant"
assert msgs[1]["tool_calls"][0]["function"]["name"] == "create_plan"
assert msgs[1]["tool_calls"][0]["function"]["name"] == "plan_agent"
assert msgs[2]["role"] == "tool"
assert msgs[2]["content"] == self.GOOD_PLAN
assert msgs[3]["role"] == "user"
@@ -753,3 +753,268 @@ class TestGetCapabilitiesOverride:
caps = session._get_capabilities()
# Default OpenAI provider for unknown model → no vision
assert caps.supports_vision is False
class TestTitleRetry:
"""_generate_title resets _title_generated on failure."""
def test_title_generated_reset_on_failure(self, tmp_db):
session = _make_session()
session._title_generated = True
session.messages = [
{"role": "user", "content": "Hello"},
{"role": "assistant", "content": "Hi there"},
]
# Mock provider to raise
session._provider = MagicMock()
session._provider.create_completion.side_effect = RuntimeError("API error")
session._generate_title()
assert session._title_generated is False
def test_title_generated_stays_true_on_success(self, tmp_db):
session = _make_session()
session._title_generated = True
session.messages = [
{"role": "user", "content": "Hello"},
{"role": "assistant", "content": "Hi there"},
]
result = MagicMock()
result.content = "Test Title"
session._provider = MagicMock()
session._provider.create_completion.return_value = result
with patch("turnstone.core.session.update_workstream_title"):
session._generate_title()
# Flag stays True after successful generation
assert session._title_generated is True
def test_title_skipped_after_resume_changes_ws_id(self, tmp_db):
"""If ws_id changes (via resume) during title generation, discard the result."""
session = _make_session()
session._title_generated = True
session.messages = [
{"role": "user", "content": "Hello"},
{"role": "assistant", "content": "Hi there"},
]
original_ws_id = session._ws_id
result = MagicMock()
result.content = "Test Title"
session._provider = MagicMock()
session._provider.create_completion.return_value = result
# Simulate resume() changing ws_id while title generation is in flight
def _change_ws_id(*args, **kwargs):
session._ws_id = "different-ws-id"
return result
session._provider.create_completion.side_effect = _change_ws_id
with patch("turnstone.core.session.update_workstream_title") as mock_update:
session._generate_title()
# Title should NOT be applied to the new workstream
mock_update.assert_not_called()
# Restore for cleanup
session._ws_id = original_ws_id
class TestLiveConfigUpdate:
"""ConfigStore-backed sessions pick up settings changes at point-of-use."""
def test_memory_config_reads_from_config_store(self, tmp_db):
"""_mem_cfg returns live values from ConfigStore when present."""
from turnstone.core.config_store import ConfigStore
from turnstone.core.storage._sqlite import SQLiteBackend
storage = SQLiteBackend(str(tmp_db), create_tables=True)
cs = ConfigStore(storage)
session = _make_session(config_store=cs)
# Default: relevance_k=5
assert session._mem_cfg.relevance_k == 5
# Admin changes the setting
cs.set("memory.relevance_k", 10, changed_by="test")
assert session._mem_cfg.relevance_k == 10
def test_judge_config_reads_from_config_store(self, tmp_db):
"""_judge_cfg returns live behavioral flags from ConfigStore."""
from turnstone.core.config_store import ConfigStore
from turnstone.core.judge import JudgeConfig
from turnstone.core.storage._sqlite import SQLiteBackend
storage = SQLiteBackend(str(tmp_db), create_tables=True)
cs = ConfigStore(storage)
session = _make_session(
judge_config=JudgeConfig(),
config_store=cs,
)
# Default: enabled=True
assert session._judge_cfg.enabled is True
# Admin disables the judge
cs.set("judge.enabled", False, changed_by="test")
assert session._judge_cfg.enabled is False
def test_judge_client_config_stays_frozen(self, tmp_db):
"""LLM client fields (model, provider) are frozen from creation time."""
from turnstone.core.config_store import ConfigStore
from turnstone.core.judge import JudgeConfig
from turnstone.core.storage._sqlite import SQLiteBackend
storage = SQLiteBackend(str(tmp_db), create_tables=True)
cs = ConfigStore(storage)
session = _make_session(
judge_config=JudgeConfig(model="original-model"),
config_store=cs,
)
# Change the model in ConfigStore — should NOT affect the session
cs.set("judge.model", "new-model", changed_by="test")
assert session._judge_cfg.model == "original-model"
def test_judge_disable_after_init_stops_future_use(self, tmp_db):
"""Disabling judge.enabled after IntentJudge is created returns None."""
from turnstone.core.config_store import ConfigStore
from turnstone.core.judge import JudgeConfig
from turnstone.core.storage._sqlite import SQLiteBackend
storage = SQLiteBackend(str(tmp_db), create_tables=True)
cs = ConfigStore(storage)
session = _make_session(
judge_config=JudgeConfig(),
config_store=cs,
)
# Force judge initialization by setting a mock
session._judge = MagicMock()
assert session._ensure_judge() is not None
# Admin disables the judge — cached instance should NOT be returned
cs.set("judge.enabled", False, changed_by="test")
assert session._ensure_judge() is None
def test_fallback_to_frozen_without_config_store(self, tmp_db):
"""Without ConfigStore (CLI mode), frozen config is used."""
from turnstone.core.memory_relevance import MemoryConfig
session = _make_session(memory_config=MemoryConfig(relevance_k=3))
assert session._mem_cfg.relevance_k == 3
class TestAgentOutputGuard:
"""Output guard should evaluate tool results in _run_agent, not just the main loop."""
def test_agent_loop_calls_evaluate_output(self):
"""_run_agent passes tool output through _evaluate_output when output_guard is enabled."""
from turnstone.core.judge import JudgeConfig
session = _make_session(judge_config=JudgeConfig(output_guard=True))
with patch.object(session, "_evaluate_output", wraps=lambda cid, o, fn: o) as mock_eval:
# Simulate _run_agent getting a tool call response then a text response
call_count = [0]
def fake_create(**kwargs):
call_count[0] += 1
resp = MagicMock()
if call_count[0] == 1:
# First call: model returns a tool call
choice = MagicMock()
choice.finish_reason = "tool_calls"
tc = MagicMock()
tc.id = "call_1"
tc.function.name = "read_file"
tc.function.arguments = '{"path": "/tmp/test"}'
choice.message.tool_calls = [tc]
choice.message.content = None
resp.choices = [choice]
resp.usage = MagicMock(prompt_tokens=10, completion_tokens=5)
else:
# Second call: model returns text (done)
choice = MagicMock()
choice.finish_reason = "stop"
choice.message.tool_calls = None
choice.message.content = "Done"
resp.choices = [choice]
resp.usage = MagicMock(prompt_tokens=10, completion_tokens=5)
return resp
session.client.chat.completions.create = fake_create
# Mock tool preparation to return a simple output
def fake_prepare(tc_dict, **kwargs):
return {
"call_id": tc_dict["id"],
"func_name": "read_file",
"needs_approval": False,
"execute": lambda p: ("call_1", "file contents with sk-proj-SECRET123"),
}
with patch.object(session, "_prepare_tool", side_effect=fake_prepare):
session._run_agent(
[{"role": "user", "content": "test"}],
tools=[{"type": "function", "function": {"name": "read_file"}}],
label="test",
)
mock_eval.assert_called_once()
args = mock_eval.call_args[0]
assert args[0] == "call_1" # call_id
assert "sk-proj-SECRET123" in args[1] # output
assert args[2] == "read_file" # func_name
def test_agent_loop_skips_guard_when_disabled(self):
"""_run_agent does not call _evaluate_output when output_guard is disabled."""
from turnstone.core.judge import JudgeConfig
session = _make_session(judge_config=JudgeConfig(output_guard=False))
with patch.object(session, "_evaluate_output") as mock_eval:
call_count = [0]
def fake_create(**kwargs):
call_count[0] += 1
resp = MagicMock()
if call_count[0] == 1:
choice = MagicMock()
choice.finish_reason = "tool_calls"
tc = MagicMock()
tc.id = "call_1"
tc.function.name = "read_file"
tc.function.arguments = '{"path": "/tmp/test"}'
choice.message.tool_calls = [tc]
choice.message.content = None
resp.choices = [choice]
resp.usage = MagicMock(prompt_tokens=10, completion_tokens=5)
else:
choice = MagicMock()
choice.finish_reason = "stop"
choice.message.tool_calls = None
choice.message.content = "Done"
resp.choices = [choice]
resp.usage = MagicMock(prompt_tokens=10, completion_tokens=5)
return resp
session.client.chat.completions.create = fake_create
def fake_prepare(tc_dict, **kwargs):
return {
"call_id": tc_dict["id"],
"func_name": "read_file",
"needs_approval": False,
"execute": lambda p: ("call_1", "safe output"),
}
with patch.object(session, "_prepare_tool", side_effect=fake_prepare):
session._run_agent(
[{"role": "user", "content": "test"}],
tools=[{"type": "function", "function": {"name": "read_file"}}],
label="test",
)
mock_eval.assert_not_called()
+4 -1
View File
@@ -179,7 +179,10 @@ class TestDeleteSetting:
# Delete it
r = client.delete("/v1/api/admin/settings/tools.timeout")
assert r.status_code == 200
assert r.json()["status"] == "ok"
body = r.json()
assert body["status"] == "ok"
assert body["key"] == "tools.timeout"
assert body["default"] == 120 # registry default for tools.timeout
def test_delete_then_list_shows_default(self, client):
client.put(
-10
View File
@@ -4,16 +4,6 @@ from __future__ import annotations
import uuid
import pytest
from turnstone.core.storage._sqlite import SQLiteBackend
@pytest.fixture()
def storage(tmp_path):
"""Fresh SQLite backend for each test."""
return SQLiteBackend(str(tmp_path / "test.db"))
class TestDeleteSkillResourceByPath:
def test_delete_existing(self, storage):
+366 -1
View File
@@ -132,6 +132,7 @@ def _create_template(db, template_id, name, content, **kwargs):
notify_on_complete=kwargs.get("notify_on_complete", "{}"),
enabled=kwargs.get("enabled", True),
allowed_tools=kwargs.get("allowed_tools", "[]"),
priority=kwargs.get("priority", 0),
)
@@ -293,7 +294,7 @@ class TestSkillStorage:
assert result == []
def test_list_skills_by_activation_ordered_by_name(self, db):
"""Results are ordered by name ascending."""
"""Results are ordered by name ascending when priority is equal."""
_create_template(db, "s2", "beta-search", "B", activation="search")
_create_template(db, "s1", "alpha-search", "A", activation="search")
results = db.list_skills_by_activation("search")
@@ -301,6 +302,51 @@ class TestSkillStorage:
assert results[0]["name"] == "alpha-search"
assert results[1]["name"] == "beta-search"
def test_list_skills_by_activation_ordered_by_priority(self, db):
"""Results are ordered by priority ascending, then name."""
_create_template(db, "s1", "style", "S", activation="default", priority=20)
_create_template(db, "s2", "safety", "F", activation="default", priority=10)
_create_template(db, "s3", "tone", "T", activation="default", priority=10)
results = db.list_skills_by_activation("default")
assert len(results) == 3
assert results[0]["name"] == "safety"
assert results[1]["name"] == "tone"
assert results[2]["name"] == "style"
def test_priority_default_is_zero(self, db):
"""Priority defaults to 0 when not specified."""
_create_template(db, "s1", "skill", "content")
tpl = db.get_prompt_template("s1")
assert tpl is not None
assert tpl["priority"] == 0
def test_priority_roundtrip(self, db):
"""Priority can be set on create and retrieved."""
_create_template(db, "s1", "skill", "content", priority=42)
tpl = db.get_prompt_template("s1")
assert tpl is not None
assert tpl["priority"] == 42
def test_priority_update(self, db):
"""Priority can be updated."""
_create_template(db, "s1", "skill", "content", priority=10)
db.update_prompt_template("s1", priority=99)
tpl = db.get_prompt_template("s1")
assert tpl is not None
assert tpl["priority"] == 99
def test_list_default_templates_ordered_by_priority(self, db):
"""list_default_templates() respects priority ordering."""
_create_template(db, "s1", "beta", "b", activation="default", priority=10)
_create_template(db, "s2", "alpha", "a", activation="default", priority=5)
_create_template(db, "s3", "gamma", "g", activation="default", priority=1)
results = db.list_default_templates()
assert len(results) == 3
assert results[0]["name"] == "gamma"
assert results[1]["name"] == "alpha"
assert results[2]["name"] == "beta"
# ---------------------------------------------------------------------------
# 1b. Skill resource storage tests
@@ -1555,3 +1601,322 @@ class TestSkillAdminEndpoints:
)
assert resp.status_code == 400
assert "integer" in resp.json()["error"].lower()
# ---------------------------------------------------------------------------
# 9. Skill session config applied to workstream via server handler
# ---------------------------------------------------------------------------
class TestSkillConfigAppliedToWorkstream:
"""Verify that skill session config fields are applied to the ChatSession
when a workstream is created via the server ``create_workstream`` handler.
"""
@pytest.fixture()
def _ws_app(self, tmp_path):
"""Build a minimal Starlette app with the real ``create_workstream``
handler, a real ``WorkstreamManager``, and a temp SQLite storage
backend. Returns ``(TestClient, WorkstreamManager, storage)``.
"""
import queue
import threading
import turnstone.core.storage._registry as _reg
from turnstone.core.workstream import WorkstreamManager
from turnstone.server import create_workstream
storage = SQLiteBackend(str(tmp_path / "ws_test.db"))
# Inject the test storage as the global singleton so that
# get_storage() / get_skill_by_name() resolve against it.
old_storage = _reg._storage
_reg._storage = storage
def _session_factory(
ui: Any, model_alias: Any = None, ws_id: Any = None, **kwargs: Any
) -> ChatSession:
return ChatSession(
client=MagicMock(),
model=model_alias or "test-model",
ui=ui,
instructions=None,
temperature=0.5,
max_tokens=4096,
tool_timeout=30,
ws_id=ws_id,
skill=kwargs.get("skill"),
)
mgr = WorkstreamManager(_session_factory)
routes = [
Mount(
"/v1",
routes=[
Route(
"/api/workstreams/new",
create_workstream,
methods=["POST"],
),
],
),
]
app = Starlette(
routes=routes,
middleware=[Middleware(_InjectAuthMiddleware)],
)
app.state.workstreams = mgr
app.state.skip_permissions = True
app.state.global_queue = queue.Queue()
app.state.global_listeners = []
app.state.global_listeners_lock = threading.Lock()
client = TestClient(app, raise_server_exceptions=False)
yield client, mgr, storage
# Restore original storage singleton.
_reg._storage = old_storage
def test_session_receives_temperature(self, _ws_app):
"""Skill temperature overrides the session default."""
client, mgr, storage = _ws_app
_create_template(storage, "s1", "warm-skill", "Be warm.", temperature=0.9, enabled=True)
resp = client.post("/v1/api/workstreams/new", json={"skill": "warm-skill"})
assert resp.status_code == 200
ws_id = resp.json()["ws_id"]
ws = mgr.get(ws_id)
assert ws is not None and ws.session is not None
assert ws.session.temperature == 0.9
def test_session_receives_max_tokens(self, _ws_app):
"""Skill max_tokens overrides the session default."""
client, mgr, storage = _ws_app
_create_template(storage, "s1", "token-skill", "Be concise.", max_tokens=1024, enabled=True)
resp = client.post("/v1/api/workstreams/new", json={"skill": "token-skill"})
assert resp.status_code == 200
ws = mgr.get(resp.json()["ws_id"])
assert ws is not None and ws.session is not None
assert ws.session.max_tokens == 1024
def test_session_receives_token_budget(self, _ws_app):
"""Skill token_budget is applied to the session."""
client, mgr, storage = _ws_app
_create_template(
storage, "s1", "budget-skill", "Stay on budget.", token_budget=50000, enabled=True
)
resp = client.post("/v1/api/workstreams/new", json={"skill": "budget-skill"})
assert resp.status_code == 200
ws = mgr.get(resp.json()["ws_id"])
assert ws is not None and ws.session is not None
assert ws.session._token_budget == 50000
def test_session_receives_reasoning_effort(self, _ws_app):
"""Skill reasoning_effort is applied to the session."""
client, mgr, storage = _ws_app
_create_template(
storage, "s1", "effort-skill", "Think hard.", reasoning_effort="high", enabled=True
)
resp = client.post("/v1/api/workstreams/new", json={"skill": "effort-skill"})
assert resp.status_code == 200
ws = mgr.get(resp.json()["ws_id"])
assert ws is not None and ws.session is not None
assert ws.session.reasoning_effort == "high"
def test_session_receives_agent_max_turns(self, _ws_app):
"""Skill agent_max_turns is applied to the session."""
client, mgr, storage = _ws_app
_create_template(
storage, "s1", "turns-skill", "Few turns.", agent_max_turns=3, enabled=True
)
resp = client.post("/v1/api/workstreams/new", json={"skill": "turns-skill"})
assert resp.status_code == 200
ws = mgr.get(resp.json()["ws_id"])
assert ws is not None and ws.session is not None
assert ws.session.agent_max_turns == 3
def test_auto_approve_set_on_ui(self, _ws_app):
"""Skill auto_approve=True propagates to the WebUI."""
from turnstone.server import WebUI
client, mgr, storage = _ws_app
_create_template(
storage, "s1", "approve-skill", "Auto approve.", auto_approve=True, enabled=True
)
resp = client.post("/v1/api/workstreams/new", json={"skill": "approve-skill"})
assert resp.status_code == 200
ws = mgr.get(resp.json()["ws_id"])
assert ws is not None
assert isinstance(ws.ui, WebUI)
assert ws.ui.auto_approve is True
def test_allowed_tools_set_on_ui(self, _ws_app):
"""Skill allowed_tools are parsed and set as auto_approve_tools on the UI."""
from turnstone.server import WebUI
client, mgr, storage = _ws_app
_create_template(
storage,
"s1",
"tools-skill",
"Restricted tools.",
allowed_tools='["bash", "read_file"]',
enabled=True,
)
resp = client.post("/v1/api/workstreams/new", json={"skill": "tools-skill"})
assert resp.status_code == 200
ws = mgr.get(resp.json()["ws_id"])
assert ws is not None
assert isinstance(ws.ui, WebUI)
assert ws.ui.auto_approve_tools == {"bash", "read_file"}
def test_skill_model_overrides_resolved_model(self, _ws_app):
"""Skill model field overrides the default session model."""
client, mgr, storage = _ws_app
_create_template(
storage, "s1", "model-skill", "Use specific model.", model="gpt-5", enabled=True
)
resp = client.post("/v1/api/workstreams/new", json={"skill": "model-skill"})
assert resp.status_code == 200
ws = mgr.get(resp.json()["ws_id"])
assert ws is not None and ws.session is not None
assert ws.session.model == "gpt-5"
def test_all_session_config_fields_applied(self, _ws_app):
"""All session config fields from a skill are applied together."""
from turnstone.server import WebUI
client, mgr, storage = _ws_app
_create_template(
storage,
"s1",
"full-skill",
"Full config skill.",
model="gpt-5",
temperature=0.8,
reasoning_effort="high",
max_tokens=2048,
token_budget=100000,
agent_max_turns=10,
auto_approve=True,
allowed_tools='["bash", "write_file", "read_file"]',
notify_on_complete='{"channel": "discord"}',
enabled=True,
)
resp = client.post("/v1/api/workstreams/new", json={"skill": "full-skill"})
assert resp.status_code == 200
ws = mgr.get(resp.json()["ws_id"])
assert ws is not None and ws.session is not None
sess = ws.session
assert sess.model == "gpt-5"
assert sess.temperature == 0.8
assert sess.reasoning_effort == "high"
assert sess.max_tokens == 2048
assert sess._token_budget == 100000
assert sess.agent_max_turns == 10
assert sess._notify_on_complete == '{"channel": "discord"}'
assert sess._applied_skill_id == "s1"
assert sess._applied_skill_content == "Full config skill."
assert isinstance(ws.ui, WebUI)
assert ws.ui.auto_approve is True
assert ws.ui.auto_approve_tools == {"bash", "write_file", "read_file"}
def test_disabled_skill_returns_400(self, _ws_app):
"""Creating a workstream with a disabled skill returns 400."""
client, _mgr, storage = _ws_app
_create_template(storage, "s1", "disabled-skill", "Disabled.", enabled=False)
resp = client.post("/v1/api/workstreams/new", json={"skill": "disabled-skill"})
assert resp.status_code == 400
assert "disabled" in resp.json()["error"].lower()
def test_unknown_skill_returns_400(self, _ws_app):
"""Creating a workstream with a nonexistent skill returns 400."""
client, _mgr, _storage = _ws_app
resp = client.post("/v1/api/workstreams/new", json={"skill": "no-such-skill"})
assert resp.status_code == 400
assert "not found" in resp.json()["error"].lower()
def test_zero_token_budget_is_noop(self, _ws_app):
"""Skill with token_budget=0 — handler skips budget application (> 0 guard)."""
client, mgr, storage = _ws_app
_create_template(
storage, "s1", "no-budget-skill", "No budget.", token_budget=0, enabled=True
)
resp = client.post("/v1/api/workstreams/new", json={"skill": "no-budget-skill"})
assert resp.status_code == 200
ws = mgr.get(resp.json()["ws_id"])
assert ws is not None and ws.session is not None
# Budget stays at default (0) — the handler's > 0 guard prevents application
assert ws.session._token_budget == 0
def test_empty_allowed_tools_is_noop(self, _ws_app):
"""Skill with allowed_tools='[]' — handler skips (empty check)."""
client, mgr, storage = _ws_app
_create_template(
storage, "s1", "no-tools-skill", "No tools.", allowed_tools="[]", enabled=True
)
resp = client.post("/v1/api/workstreams/new", json={"skill": "no-tools-skill"})
assert resp.status_code == 200
ws = mgr.get(resp.json()["ws_id"])
assert ws is not None
# auto_approve_tools stays at default (empty set)
assert ws.ui.auto_approve_tools == set()
def test_skill_lineage_in_workstreams_table(self, _ws_app):
"""skill_id and skill_version columns are populated in the workstreams table."""
import sqlalchemy as sa
from turnstone.core.storage._schema import workstreams
client, _mgr, storage = _ws_app
_create_template(storage, "s1", "lineage-skill", "Track me.", enabled=True)
resp = client.post("/v1/api/workstreams/new", json={"skill": "lineage-skill"})
assert resp.status_code == 200
ws_id = resp.json()["ws_id"]
with storage._engine.connect() as conn:
row = conn.execute(
sa.select(workstreams.c.skill_id, workstreams.c.skill_version).where(
workstreams.c.ws_id == ws_id
)
).fetchone()
assert row is not None
assert row[0] == "s1"
assert row[1] == 1
def test_no_skill_lineage_when_no_skill(self, _ws_app):
"""Workstream without a skill has empty skill_id and zero skill_version."""
import sqlalchemy as sa
from turnstone.core.storage._schema import workstreams
client, _mgr, storage = _ws_app
resp = client.post("/v1/api/workstreams/new", json={})
assert resp.status_code == 200
ws_id = resp.json()["ws_id"]
with storage._engine.connect() as conn:
row = conn.execute(
sa.select(workstreams.c.skill_id, workstreams.c.skill_version).where(
workstreams.c.ws_id == ws_id
)
).fetchone()
assert row is not None
assert row[0] == ""
assert row[1] == 0
+69 -12
View File
@@ -1,18 +1,8 @@
"""Tests for the SQLite storage backend."""
import pytest
from turnstone.core.storage import init_storage, reset_storage
@pytest.fixture
def backend(tmp_path):
"""Create a fresh SQLiteBackend for each test."""
reset_storage()
b = init_storage("sqlite", path=str(tmp_path / "test.db"), run_migrations=False)
yield b
reset_storage()
from __future__ import annotations
from typing import Any
# -- Workstream registration ---------------------------------------------------
@@ -289,6 +279,73 @@ class TestWorkstreams:
assert rows[0][6] == "node-a"
# -- Structured memory touch ---------------------------------------------------
class TestTouchStructuredMemory:
@staticmethod
def _create_memory(
backend: Any, name: str = "m1", scope: str = "global", scope_id: str = ""
) -> None:
import uuid
backend.create_structured_memory(
memory_id=str(uuid.uuid4()),
name=name,
description="test desc",
mem_type="project",
scope=scope,
scope_id=scope_id,
content="test content",
)
def test_batch_touch_multiple(self, backend):
self._create_memory(backend, name="a")
self._create_memory(backend, name="b")
self._create_memory(backend, name="c")
count = backend.touch_structured_memories(
[
("a", "global", ""),
("b", "global", ""),
("c", "global", ""),
]
)
assert count == 3
for name in ("a", "b", "c"):
mem = backend.get_structured_memory_by_name(name, "global", "")
assert int(mem["access_count"]) == 1
def test_batch_touch_empty_list(self, backend):
assert backend.touch_structured_memories([]) == 0
def test_batch_touch_partial_match(self, backend):
self._create_memory(backend, name="exists")
count = backend.touch_structured_memories(
[
("exists", "global", ""),
("missing", "global", ""),
]
)
assert count == 1
mem = backend.get_structured_memory_by_name("exists", "global", "")
assert int(mem["access_count"]) == 1
def test_batch_touch_with_duplicates(self, backend):
"""Duplicate keys in batch should each increment access_count once."""
self._create_memory(backend, name="dup")
# Two identical keys — storage gets called twice for the same row
count = backend.touch_structured_memories([("dup", "global", ""), ("dup", "global", "")])
assert count == 2
mem = backend.get_structured_memory_by_name("dup", "global", "")
assert int(mem["access_count"]) == 2
# -- Lifecycle -----------------------------------------------------------------
-9
View File
@@ -1,14 +1,5 @@
"""Tests for structured memory storage backend operations."""
import pytest
from turnstone.core.storage._sqlite import SQLiteBackend
@pytest.fixture
def backend(tmp_path):
return SQLiteBackend(str(tmp_path / "test.db"))
class TestCreateAndGet:
def test_create_and_get_by_id(self, backend):
+174
View File
@@ -0,0 +1,174 @@
"""Tests for tool policy enforcement across CLI, bridge, and channel entry points."""
from __future__ import annotations
from unittest.mock import MagicMock, patch
from turnstone.cli import TerminalUI
# ---------------------------------------------------------------------------
# CLI
# ---------------------------------------------------------------------------
class TestCLIPolicyEnforcement:
"""Tool policies should be enforced in CLI approve_tools()."""
def _make_items(self, *tool_names: str) -> list[dict]:
return [
{
"call_id": f"call_{i}",
"header": f"Tool: {name}",
"preview": "",
"func_name": name,
"approval_label": name,
"needs_approval": True,
}
for i, name in enumerate(tool_names)
]
def test_deny_policy_blocks_tool(self):
"""A 'deny' policy verdict should block the tool without prompting."""
ui = TerminalUI()
items = self._make_items("bash")
with (
patch(
"turnstone.core.policy.evaluate_tool_policies_batch",
return_value={"bash": "deny"},
),
patch(
"turnstone.core.storage._registry.get_storage",
return_value=MagicMock(),
),
):
approved, _ = ui.approve_tools(items)
assert items[0].get("denied") is True
assert items[0].get("error")
assert "policy" in items[0]["error"].lower()
def test_allow_policy_auto_approves(self):
"""An 'allow' policy verdict should auto-approve without prompting."""
ui = TerminalUI()
items = self._make_items("read_file")
with (
patch(
"turnstone.core.policy.evaluate_tool_policies_batch",
return_value={"read_file": "allow"},
),
patch(
"turnstone.core.storage._registry.get_storage",
return_value=MagicMock(),
),
):
approved, _ = ui.approve_tools(items)
assert approved is True
def test_no_storage_skips_policies(self):
"""When storage is unavailable, policies are skipped (best-effort)."""
ui = TerminalUI()
items = self._make_items("bash")
with (
patch(
"turnstone.core.storage._registry.get_storage",
return_value=None,
),
patch("builtins.input", return_value="y"),
):
approved, _ = ui.approve_tools(items)
# Should fall through to normal prompt (which we answered 'y')
assert approved is True
# ---------------------------------------------------------------------------
# Bridge
# ---------------------------------------------------------------------------
class TestBridgePolicyEnforcement:
"""Tool policies should be enforced in bridge _handle_approval()."""
def _make_bridge(self):
from turnstone.mq.bridge import Bridge
broker = MagicMock()
return Bridge(
server_url="http://localhost:8080",
broker=broker,
node_id="test-node",
approval_timeout=1,
)
def _approval_items(self, *tool_names: str) -> list[dict]:
return [
{"func_name": name, "needs_approval": True, "approval_label": name}
for name in tool_names
]
def test_deny_policy_rejects_approval(self):
"""A 'deny' policy should reject the approval."""
bridge = self._make_bridge()
with (
patch(
"turnstone.core.policy.evaluate_tool_policies_batch",
return_value={"bash": "deny"},
),
patch(
"turnstone.core.storage._registry._storage",
new=MagicMock(),
),
patch.object(bridge, "_api_approve") as mock_approve,
patch.object(bridge, "_publish_ws"),
):
bridge._handle_approval("ws-1", {"items": self._approval_items("bash")})
mock_approve.assert_called_once()
assert mock_approve.call_args.kwargs.get("approved") is False
def test_allow_policy_approves(self):
"""An 'allow' policy should auto-approve."""
bridge = self._make_bridge()
with (
patch(
"turnstone.core.policy.evaluate_tool_policies_batch",
return_value={"read_file": "allow"},
),
patch(
"turnstone.core.storage._registry.get_storage",
return_value=MagicMock(),
),
patch.object(bridge, "_api_approve") as mock_approve,
patch.object(bridge, "_publish_ws"),
):
bridge._handle_approval("ws-1", {"items": self._approval_items("read_file")})
mock_approve.assert_called_once()
assert mock_approve.call_args.kwargs.get("approved") is True
def test_mixed_deny_rejects_batch(self):
"""If any tool is denied, the whole batch is rejected."""
bridge = self._make_bridge()
with (
patch(
"turnstone.core.policy.evaluate_tool_policies_batch",
return_value={"bash": "deny", "read_file": "allow"},
),
patch(
"turnstone.core.storage._registry._storage",
new=MagicMock(),
),
patch.object(bridge, "_api_approve") as mock_approve,
patch.object(bridge, "_publish_ws"),
):
bridge._handle_approval("ws-1", {"items": self._approval_items("bash", "read_file")})
mock_approve.assert_called_once()
assert mock_approve.call_args.kwargs.get("approved") is False
+2 -2
View File
@@ -104,8 +104,8 @@ class TestToolsMetadata:
"man": "page",
"web_fetch": "url",
"web_search": "query",
"task": "prompt",
"create_plan": "goal",
"task_agent": "prompt",
"plan_agent": "goal",
"memory": "name",
"recall": "query",
"notify": "message",
-10
View File
@@ -2,16 +2,6 @@
from __future__ import annotations
import pytest
from turnstone.core.storage._sqlite import SQLiteBackend
@pytest.fixture()
def db(tmp_path):
"""Create a fresh SQLite backend for each test."""
return SQLiteBackend(str(tmp_path / "test.db"))
class TestUserCRUD:
def test_create_and_get(self, db):
-10
View File
@@ -2,16 +2,6 @@
from __future__ import annotations
import pytest
from turnstone.core.storage._sqlite import SQLiteBackend
@pytest.fixture
def db(tmp_path):
"""Fresh SQLite backend for each test."""
return SQLiteBackend(str(tmp_path / "test.db"))
def _make_watch_kwargs(**overrides):
"""Build default kwargs for create_watch."""
+175
View File
@@ -0,0 +1,175 @@
"""Tests for turnstone.core.web_search — pluggable web search backends."""
from __future__ import annotations
from unittest.mock import MagicMock, patch
from turnstone.core.web_search import (
DuckDuckGoClient,
MCPSearchClient,
TavilyClient,
_format_ddg,
_format_tavily,
resolve_web_search_client,
)
# ---------------------------------------------------------------------------
# Formatters
# ---------------------------------------------------------------------------
class TestFormatTavily:
def test_formats_answer_and_results(self):
data = {
"answer": "Python is great",
"results": [
{"title": "Python.org", "url": "https://python.org", "content": "Official site"},
{"title": "PyPI", "url": "https://pypi.org", "content": "Package index"},
],
}
out = _format_tavily(data, "python")
assert "Answer: Python is great" in out
assert "[Python.org](https://python.org)" in out
assert "[PyPI](https://pypi.org)" in out
def test_no_results(self):
out = _format_tavily({"results": []}, "nothing")
assert "No results for 'nothing'" in out
def test_no_answer(self):
data = {
"results": [{"title": "T", "url": "http://t", "content": "C"}],
}
out = _format_tavily(data, "q")
assert "Answer:" not in out
assert "[T](http://t)" in out
class TestFormatDDG:
def test_formats_results(self):
results = [
{"title": "DDG Result", "href": "https://ddg.example.com", "body": "Search body"},
]
out = _format_ddg(results, "test")
assert "[DDG Result](https://ddg.example.com)" in out
assert "Search body" in out
def test_no_results(self):
out = _format_ddg([], "nothing")
assert "No results for 'nothing'" in out
# ---------------------------------------------------------------------------
# Client tests
# ---------------------------------------------------------------------------
class TestTavilyClient:
def test_search_calls_api(self):
mock_resp = MagicMock()
mock_resp.json.return_value = {
"answer": "42",
"results": [{"title": "T", "url": "http://t", "content": "C"}],
}
with patch("turnstone.core.web_search.httpx.post", return_value=mock_resp) as mock_post:
client = TavilyClient("test-key", timeout=10)
result = client.search("meaning of life", max_results=3)
mock_post.assert_called_once()
call_kwargs = mock_post.call_args
assert call_kwargs.kwargs["json"]["query"] == "meaning of life"
assert call_kwargs.kwargs["json"]["max_results"] == 3
assert "Answer: 42" in result
class TestDuckDuckGoClient:
def test_integration_via_mock_ddgs(self):
"""Patch the ddgs import inside DuckDuckGoClient.search."""
mock_ddgs = MagicMock()
mock_ddgs.__enter__ = MagicMock(return_value=mock_ddgs)
mock_ddgs.__exit__ = MagicMock(return_value=False)
mock_ddgs.text.return_value = [
{"title": "DDG Result", "href": "https://ddg.co", "body": "Found it"},
]
mock_module = MagicMock()
mock_module.DDGS.return_value = mock_ddgs
with patch.dict("sys.modules", {"ddgs": mock_module}):
client = DuckDuckGoClient(timeout=10)
result = client.search("test query", max_results=3)
mock_ddgs.text.assert_called_once_with("test query", max_results=3)
assert "[DDG Result](https://ddg.co)" in result
assert "Found it" in result
class TestMCPSearchClient:
def test_delegates_to_mcp(self):
mcp = MagicMock()
mcp.call_tool_sync.return_value = "MCP search results"
client = MCPSearchClient(mcp, "mcp__ddg__search", timeout=30)
result = client.search("test", max_results=3, topic="news")
mcp.call_tool_sync.assert_called_once_with(
"mcp__ddg__search",
{"query": "test", "max_results": 3, "topic": "news"},
timeout=30,
)
assert result == "MCP search results"
# ---------------------------------------------------------------------------
# Resolver
# ---------------------------------------------------------------------------
class TestResolveClient:
def test_auto_tavily_when_key_present(self):
client = resolve_web_search_client("", tavily_key="key")
assert isinstance(client, TavilyClient)
def test_auto_ddg_when_no_tavily(self):
with patch("turnstone.core.web_search._ddg_available", return_value=True):
client = resolve_web_search_client("", tavily_key=None)
assert isinstance(client, DuckDuckGoClient)
def test_auto_none_when_nothing_available(self):
with patch("turnstone.core.web_search._ddg_available", return_value=False):
client = resolve_web_search_client("", tavily_key=None)
assert client is None
def test_explicit_tavily(self):
client = resolve_web_search_client("tavily", tavily_key="key")
assert isinstance(client, TavilyClient)
def test_explicit_tavily_no_key(self):
client = resolve_web_search_client("tavily", tavily_key=None)
assert client is None
def test_explicit_ddg(self):
with patch("turnstone.core.web_search._ddg_available", return_value=True):
client = resolve_web_search_client("ddg", tavily_key=None)
assert isinstance(client, DuckDuckGoClient)
def test_explicit_ddg_not_installed(self):
with patch("turnstone.core.web_search._ddg_available", return_value=False):
client = resolve_web_search_client("ddg", tavily_key=None)
assert client is None
def test_mcp_backend(self):
mcp = MagicMock()
mcp.is_mcp_tool.return_value = True
client = resolve_web_search_client("mcp:ddg:search", tavily_key=None, mcp_client=mcp)
assert isinstance(client, MCPSearchClient)
mcp.is_mcp_tool.assert_called_with("mcp__ddg__search")
def test_mcp_backend_not_connected(self):
mcp = MagicMock()
mcp.is_mcp_tool.return_value = False
client = resolve_web_search_client("mcp:ddg:search", tavily_key=None, mcp_client=mcp)
assert client is None
def test_mcp_backend_no_client(self):
client = resolve_web_search_client("mcp:ddg:search", tavily_key=None, mcp_client=None)
assert client is None
def test_unknown_backend_returns_none(self):
client = resolve_web_search_client("typo_backend", tavily_key="key")
assert client is None
+1 -1
View File
@@ -1,3 +1,3 @@
"""turnstone - Multi-node AI orchestration platform with tool use, agent routing, and cluster simulation."""
__version__ = "0.8.4"
__version__ = "0.8.8"
+3
View File
@@ -309,6 +309,7 @@ class SkillInfo(BaseModel):
agent_max_turns: int | None = None
notify_on_complete: str = "{}"
enabled: bool = True
priority: int = 0
allowed_tools: str = "[]"
license: str = ""
compatibility: str = ""
@@ -341,6 +342,7 @@ class CreateSkillRequest(BaseModel):
agent_max_turns: int | None = None
notify_on_complete: str = "{}"
enabled: bool = True
priority: int = 0
allowed_tools: str = "[]"
license: str = ""
compatibility: str = ""
@@ -366,6 +368,7 @@ class UpdateSkillRequest(BaseModel):
agent_max_turns: int | None = None
notify_on_complete: str | None = None
enabled: bool | None = None
priority: int | None = None
allowed_tools: str | None = None
license: str | None = None
compatibility: str | None = None
+3 -1
View File
@@ -82,6 +82,7 @@ from turnstone.api.schemas import (
CreateTokenRequest,
CreateTokenResponse,
CreateUserRequest,
DeleteSettingResponse,
ErrorResponse,
ListScheduleRunsResponse,
ListSchedulesResponse,
@@ -751,7 +752,7 @@ CONSOLE_ENDPOINTS: list[EndpointSpec] = [
"/v1/api/admin/settings/{key}",
"DELETE",
"Reset a setting to its default value",
response_model=StatusResponse,
response_model=DeleteSettingResponse,
query_params=[
QueryParam("node_id", "Node ID for node-scoped settings"),
],
@@ -855,6 +856,7 @@ CONSOLE_ENDPOINTS: list[EndpointSpec] = [
_ALL_MODELS: list[type[BaseModel]] = [
ErrorResponse,
StatusResponse,
DeleteSettingResponse,
AuthLoginRequest,
AuthLoginResponse,
AuthSetupRequest,
+9
View File
@@ -8,6 +8,7 @@ as the single source of truth for the generated OpenAPI spec.
from __future__ import annotations
from enum import StrEnum
from typing import Any
from pydantic import BaseModel, Field
@@ -34,6 +35,14 @@ class StatusResponse(BaseModel):
status: str = Field(default="ok", examples=["ok"])
class DeleteSettingResponse(BaseModel):
"""DELETE /v1/api/admin/settings/{key} response."""
status: str = Field(default="ok", examples=["ok"])
key: str = Field(description="Dotted setting key that was reset")
default: Any = Field(description="Registry default value the setting reverted to")
class AuthLoginRequest(BaseModel):
"""POST /v1/api/auth/login request body.
+43 -1
View File
@@ -306,7 +306,49 @@ class TurnstoneBot:
await sm.append(event.text)
elif isinstance(event, ApprovalRequestEvent):
if self.config.auto_approve or self._should_auto_approve(event):
# Evaluate admin tool policies before auto-approve.
_policy_handled = False
if self.storage is not None:
try:
from turnstone.core.policy import evaluate_tool_policies_batch
_tool_names = [
it.get("approval_label", "") or it.get("func_name", "")
for it in event.items
if it.get("needs_approval") and it.get("func_name") and not it.get("error")
]
_tool_names = [n for n in _tool_names if n]
if _tool_names:
verdicts = await asyncio.to_thread(
evaluate_tool_policies_batch,
self.storage,
_tool_names,
)
if any(v == "deny" for v in verdicts.values()):
denied = [n for n, v in verdicts.items() if v == "deny"]
await self.router.send_approval(
ws_id,
event.correlation_id,
approved=False,
feedback=f"Blocked by tool policy: {', '.join(denied)}",
)
await thread.send(
f"*Tool blocked by admin policy: {', '.join(denied)}*"
)
_policy_handled = True
elif all(verdicts.get(n) == "allow" for n in _tool_names):
await self.router.send_approval(
ws_id,
event.correlation_id,
approved=True,
)
await thread.send("*Tool approved by policy.*")
_policy_handled = True
except Exception:
log.debug("Tool policy evaluation failed for ws %s", ws_id, exc_info=True)
if not _policy_handled and (
self.config.auto_approve or self._should_auto_approve(event)
):
await self.router.send_approval(ws_id, event.correlation_id, approved=True)
await thread.send("*Tool auto-approved.*")
else:
-42
View File
@@ -1,42 +0,0 @@
"""chat.py — Backward-compatibility shim.
All functionality has been moved to submodules:
- turnstone.core.session: ChatSession, SessionUI
- turnstone.core.tools: TOOLS, AGENT_TOOLS, TASK_AGENT_TOOLS
- turnstone.core.edit: find_occurrences, pick_nearest
- turnstone.core.sandbox: validate_math_code, execute_math_sandboxed
- turnstone.core.safety: is_command_blocked, sanitize_command
- turnstone.core.web: strip_html, check_ssrf
- turnstone.core.memory: save_message, structured memory facade, etc.
- turnstone.ui.colors: ANSI constants and helpers
- turnstone.ui.markdown: MarkdownRenderer
- turnstone.ui.spinner: Spinner
- turnstone.cli: TerminalUI, main, detect_model
"""
# Re-export public API for backward compatibility
from turnstone.cli import detect_model, main # noqa: F401
from turnstone.core.session import ChatSession, SessionUI # noqa: F401
from turnstone.core.tools import AGENT_TOOLS, TASK_AGENT_TOOLS, TOOLS # noqa: F401
from turnstone.core.web import strip_html as _strip_html # noqa: F401
from turnstone.ui.colors import ( # noqa: F401
BLUE,
BOLD,
CYAN,
DIM,
GRAY,
GREEN,
ITALIC,
MAGENTA,
RED,
RESET,
YELLOW,
bold,
cyan,
dim,
green,
red,
yellow,
)
from turnstone.ui.markdown import MarkdownRenderer # noqa: F401
from turnstone.ui.spinner import Spinner # noqa: F401
+74 -6
View File
@@ -14,6 +14,7 @@ import textwrap
import threading
from typing import TYPE_CHECKING, Any
from turnstone.core.judge import JudgeConfig
from turnstone.core.session import ChatSession, SessionUI
from turnstone.core.workstream import Workstream, WorkstreamManager, WorkstreamState
from turnstone.ui.colors import (
@@ -134,11 +135,42 @@ class TerminalUI(SessionUI):
"""
pending = [it for it in items if it.get("needs_approval") and not it.get("error")]
# Evaluate admin tool policies (deny/allow/ask) before prompting.
if pending:
try:
from turnstone.core.policy import evaluate_tool_policies_batch
from turnstone.core.storage._registry import get_storage
storage = get_storage()
if storage is not None:
_policy_names = [
it.get("approval_label", "") or it.get("func_name", "")
for it in pending
if it.get("func_name")
]
if _policy_names:
verdicts = evaluate_tool_policies_batch(storage, _policy_names)
for it in pending:
policy_name = it.get("approval_label", "") or it.get("func_name", "")
verdict = verdicts.get(policy_name)
if verdict == "deny":
it["denied"] = True
it["error"] = f"Blocked by tool policy ('{policy_name}')"
it["needs_approval"] = False
elif verdict == "allow":
it["needs_approval"] = False
pending = [
it for it in items if it.get("needs_approval") and not it.get("error")
]
except Exception:
pass # Best-effort — no policy enforcement on error
with self._print_lock:
# Print all headers, previews, and heuristic verdicts
for item in items:
if item.get("error"):
sys.stdout.write(f" {red(item['header'])}\n")
sys.stdout.write(f" {red(item['error'])}\n")
else:
sys.stdout.write(f" {yellow(item['header'])}\n")
if item.get("preview"):
@@ -162,7 +194,11 @@ class TerminalUI(SessionUI):
# Per-tool auto-approve check
if self.auto_approve_tools:
pending_names = {it.get("func_name", "") for it in pending if it.get("func_name")}
pending_names = {
it.get("approval_label", "") or it.get("func_name", "")
for it in pending
if it.get("func_name")
}
if pending_names and pending_names.issubset(self.auto_approve_tools):
return True, None
@@ -195,7 +231,11 @@ class TerminalUI(SessionUI):
break
if decision in ("a", "always"):
tool_names = {it.get("func_name", "") for it in pending if it.get("func_name")}
tool_names = {
it.get("approval_label", "") or it.get("func_name", "")
for it in pending
if it.get("func_name") and not it.get("error")
}
tool_names.discard("")
tool_names.discard("__budget_override__")
self.auto_approve_tools.update(tool_names)
@@ -752,10 +792,15 @@ def _handle_cluster_command(cmd_line: str, console_url: str | None, auth_token:
def detect_model(client: Any, provider: str = "openai") -> tuple[str, int | None]:
"""Auto-detect model — delegates to :func:`turnstone.core.model_registry.detect_model`."""
"""Auto-detect model — delegates to :func:`turnstone.core.model_registry.detect_model`.
CLI always uses fatal=True, so model is never None.
"""
from turnstone.core.model_registry import detect_model as _detect
return _detect(client, provider=provider)
model, ctx = _detect(client, provider=provider)
assert model is not None # fatal=True guarantees non-None or SystemExit
return model, ctx
# ─── Main ──────────────────────────────────────────────────────────────────
@@ -870,6 +915,12 @@ def main() -> None:
default=5,
help="Max tools returned per tool search query (default: 5)",
)
parser.add_argument(
"--web-search-backend",
default="",
metavar="BACKEND",
help="Web search backend: '' (auto), 'tavily', 'ddg', or 'mcp:server:tool'",
)
parser.add_argument(
"--resume",
default=None,
@@ -959,8 +1010,9 @@ def main() -> None:
default=0.7,
help="Confidence threshold for judge (default: 0.7)",
)
from turnstone.core.config import apply_config
from turnstone.core.config import add_config_arg, apply_config
add_config_arg(parser)
apply_config(
parser,
["api", "model", "session", "tools", "console", "auth", "mcp", "database", "judge"],
@@ -980,7 +1032,7 @@ def main() -> None:
db_url = getattr(args, "db_url", None) or os.environ.get("TURNSTONE_DB_URL", "")
db_path = getattr(args, "db_path", None) or os.environ.get("TURNSTONE_DB_PATH", "")
db_pool_size = int(
getattr(args, "db_pool_size", None) or os.environ.get("TURNSTONE_DB_POOL_SIZE", "5")
getattr(args, "db_pool_size", None) or os.environ.get("TURNSTONE_DB_POOL_SIZE", "2")
)
init_storage(db_backend, path=db_path, url=db_url, pool_size=db_pool_size)
@@ -1040,6 +1092,20 @@ def main() -> None:
storage=_get_storage(),
)
# apply_config() merges [judge] config.toml values into args as
# judge_base_url, judge_api_key, etc. Output_guard and redact_secrets
# default to True, enabling the heuristic guard even when the LLM judge
# is disabled via --no-judge.
judge_config = JudgeConfig(
enabled=args.judge_enabled,
model=args.judge_model,
provider=args.judge_provider,
base_url=getattr(args, "judge_base_url", ""),
api_key=getattr(args, "judge_api_key", ""),
confidence_threshold=args.judge_confidence,
timeout=args.judge_timeout,
)
# ChatSession factory — captures shared config for creating workstreams
def session_factory(
ui: SessionUI | None,
@@ -1070,7 +1136,9 @@ def main() -> None:
tool_search=args.tool_search,
tool_search_threshold=args.tool_search_threshold,
tool_search_max_results=args.tool_search_max_results,
web_search_backend=args.web_search_backend,
skill=skill or args.skill or None,
judge_config=judge_config,
)
# Create workstream manager and initial workstream
+55 -9
View File
@@ -54,10 +54,10 @@ class ClusterCollector:
self,
broker: RedisBroker,
prefix: str = "turnstone",
poll_interval: float = 10.0,
poll_interval: float = 15.0,
discovery_interval: float = 15.0,
max_poll_workers: int = 50,
http_timeout: float = 5.0,
max_poll_workers: int = 200,
http_timeout: float = 30.0,
auth_token: str = "",
token_manager: ServiceTokenManager | None = None,
):
@@ -80,7 +80,13 @@ class ClusterCollector:
self._running = False
self._threads: list[threading.Thread] = []
self._poll_pool = ThreadPoolExecutor(max_workers=max_poll_workers)
self._http_client = httpx.Client(timeout=http_timeout)
self._http_client = httpx.Client(
timeout=httpx.Timeout(connect=10, read=http_timeout, write=5, pool=http_timeout),
limits=httpx.Limits(
max_connections=max_poll_workers + 10,
max_keepalive_connections=min(max_poll_workers, 200),
),
)
# SSE fan-out to browser clients
self._listeners: list[queue.Queue[dict[str, Any]]] = []
@@ -240,8 +246,27 @@ class ClusterCollector:
log.exception("Poll loop error")
time.sleep(self._poll_interval)
@staticmethod
def _node_jitter(node_id: str, window: float) -> float:
"""Deterministic per-node delay within a sliding window.
Uses a Mersenne prime (2^31 - 1) to hash the node_id into a
stable offset so each node is polled at a different point in
the cycle. The offset is consistent across restarts for the
same node_id, giving an even spread without randomness.
"""
h = hash(node_id) & 0x7FFFFFFF # positive 31-bit
return (h % 2147483647) / 2147483647 * window # M31 = 2^31 - 1
def _poll_all_nodes(self) -> None:
"""Fetch dashboard data from all known nodes in parallel."""
"""Fetch dashboard data from all known nodes in parallel.
Submissions are throttled by the thread pool size to avoid a
thundering herd at most ``max_poll_workers`` concurrent HTTP
requests are in flight at any time. Each worker sleeps a
deterministic per-node jitter (derived from its node_id) to
spread requests across the first half of the poll interval.
"""
# Snapshot current auth header for this poll cycle. Per-request
# headers avoid mutating shared client state (thread-safe).
if self._token_manager is not None:
@@ -260,8 +285,18 @@ class ClusterCollector:
if not targets:
return
jitter_window = self._poll_interval / 2
def _jittered_fetch(
nid: str, url: str, headers: dict[str, str] | None
) -> tuple[dict[str, Any], dict[str, Any]]:
delay = self._node_jitter(nid, jitter_window)
if delay > 0.1:
time.sleep(delay)
return self._fetch_node(nid, url, headers)
futures = {
self._poll_pool.submit(self._fetch_node, nid, url, poll_headers): nid
self._poll_pool.submit(_jittered_fetch, nid, url, poll_headers): nid
for nid, url in targets
}
for future in as_completed(futures):
@@ -280,7 +315,7 @@ class ClusterCollector:
if nid in self._nodes:
self._nodes[nid].reachable = False
except Exception:
log.debug("Failed to poll node %s", nid)
log.warning("Failed to poll node %s", nid, exc_info=True)
with self._lock:
if nid in self._nodes:
self._nodes[nid].reachable = False
@@ -300,6 +335,7 @@ class ClusterCollector:
health_resp = self._http_client.get(f"{base}/health", headers=extra_headers)
health_data: dict[str, Any] = health_resp.json()
except Exception:
log.debug("Failed to fetch health from %s", node_id, exc_info=True)
health_data = {}
return dash_data, health_data
@@ -407,9 +443,12 @@ class ClusterCollector:
}
def get_nodes(
self, sort_by: str = "activity", limit: int = 100, offset: int = 0
self, sort_by: str = "activity", limit: int | None = 100, offset: int = 0
) -> tuple[list[dict[str, Any]], int]:
"""Return sorted, paginated node list with per-node counts."""
"""Return sorted, paginated node list with per-node counts.
Pass ``limit=None`` to return all nodes (no pagination).
"""
with self._lock:
items = []
for node in self._nodes.values():
@@ -457,8 +496,15 @@ class ClusterCollector:
elif sort_by == "name":
items.sort(key=lambda n: n["node_id"])
if limit is None:
return items[offset:], total
return items[offset : offset + limit], total
def get_all_nodes(self) -> list[dict[str, Any]]:
"""Return all nodes without pagination (for fan-out operations)."""
nodes, _ = self.get_nodes(sort_by="activity", limit=None)
return nodes
def get_workstreams(
self,
state: str | None = None,
+175 -70
View File
@@ -37,7 +37,7 @@ from starlette.staticfiles import StaticFiles
from turnstone.api.console_spec import build_console_spec
from turnstone.api.docs import make_docs_handler, make_openapi_handler
from turnstone.console.collector import ClusterCollector
from turnstone.core.auth import JWT_AUD_CONSOLE, AuthMiddleware
from turnstone.core.auth import JWT_AUD_CONSOLE, JWT_AUD_SERVER, AuthMiddleware, create_jwt
if TYPE_CHECKING:
from collections.abc import AsyncGenerator
@@ -133,15 +133,36 @@ _CONSOLE_PROXY_STYLE = "<style>.dashboard-overlay{top:32px!important}</style>"
_VALID_NODE_ID = re.compile(r"^[a-zA-Z0-9._-]+$")
_PROXY_JWT_EXPIRY_SECONDS = 300 # 5 min — ample for any request round-trip
def _proxy_auth_headers(request: Request) -> dict[str, str]:
"""Build auth headers for proxied requests to upstream servers.
Uses the service proxy token (``JWT_AUD_SERVER``) so the upstream node
accepts the request. The user's console-audience JWT is *not* forwarded
it would be rejected by the server's audience validation.
Mints a short-lived JWT carrying the real user's identity and scopes
so the upstream server records correct audit attribution and enforces
scope narrowing. Falls back to the ServiceTokenManager when no user
context is available.
"""
# Prefer the auto-rotating ServiceTokenManager when available
auth_result = getattr(getattr(request, "state", None), "auth_result", None)
jwt_secret: str = getattr(request.app.state, "jwt_secret", "")
if auth_result is not None and auth_result.user_id and jwt_secret:
token = create_jwt(
user_id=auth_result.user_id,
scopes=auth_result.scopes,
source="console-proxy",
secret=jwt_secret,
audience=JWT_AUD_SERVER,
permissions=auth_result.permissions,
expiry_seconds=_PROXY_JWT_EXPIRY_SECONDS,
)
return {"Authorization": f"Bearer {token}"}
# Fallback: service identity (no user context).
# When auth is disabled on the console, auth_result is None, so all proxied
# requests use the full-privilege service identity. This is safe only when
# the upstream server also has auth disabled.
mgr = getattr(request.app.state, "proxy_token_mgr", None)
if mgr is not None:
return dict(mgr.bearer_header)
@@ -168,7 +189,7 @@ def _get_server_url(request: Request, node_id: str) -> str | None:
def _pick_best_node(collector: ClusterCollector) -> str:
"""Select the reachable node with the most available capacity."""
nodes, _ = collector.get_nodes(sort_by="activity", limit=1000, offset=0)
nodes = collector.get_all_nodes()
best_id = ""
best_headroom = -1
for n in nodes:
@@ -252,7 +273,7 @@ async def cluster_snapshot(request: Request) -> JSONResponse:
async def cluster_events_sse(request: Request) -> Response:
collector: ClusterCollector = request.app.state.collector
client_queue: queue.Queue[dict[str, Any]] = queue.Queue(maxsize=500)
client_queue: queue.Queue[dict[str, Any]] = queue.Queue(maxsize=2000)
async def event_generator() -> AsyncGenerator[dict[str, str], None]:
loop = asyncio.get_running_loop()
@@ -407,6 +428,9 @@ async def create_workstream(request: Request) -> JSONResponse:
from turnstone.mq.protocol import CreateWorkstreamMessage
auth = getattr(getattr(request, "state", None), "auth_result", None)
uid: str = getattr(auth, "user_id", "") or ""
# General pool — push to shared queue, any bridge picks it up
if node_id == "pool":
msg = CreateWorkstreamMessage(
@@ -415,6 +439,7 @@ async def create_workstream(request: Request) -> JSONResponse:
initial_message=initial_message,
skill=skill,
resume_ws=resume_ws,
user_id=uid,
)
broker.push_inbound(msg.to_json())
log.debug("Pool dispatch: correlation_id=%s name=%r", msg.correlation_id, name)
@@ -444,6 +469,7 @@ async def create_workstream(request: Request) -> JSONResponse:
initial_message=initial_message,
skill=skill,
resume_ws=resume_ws,
user_id=uid,
)
broker.push_inbound(msg.to_json(), node_id=node_id)
@@ -682,10 +708,37 @@ async def _lifespan(app: Starlette) -> AsyncGenerator[None, None]:
# Create async HTTP clients for proxy routes. Auth headers are NOT baked
# in — _proxy_auth_headers() injects a fresh token per-request so JWTs
# auto-rotate via ServiceTokenManager instead of expiring after 1 hour.
app.state.proxy_client = httpx.AsyncClient(timeout=30)
# Size the pool above the fan-out limit to leave headroom for non-fan-out
# proxy traffic (UI proxying, SSE streams, etc.).
#
# Build a ConfigStore so console settings reads get type validation and
# caching instead of raw storage.get_system_setting() calls.
storage = getattr(app.state, "auth_storage", None)
config_store = None
if storage:
try:
from turnstone.core.config_store import ConfigStore
config_store = ConfigStore(storage)
except Exception:
log.warning("Failed to initialise ConfigStore", exc_info=True)
app.state.config_store = config_store
fan_out = (
config_store.get("cluster.node_fan_out_limit") if config_store else _NODE_FAN_OUT_LIMIT
)
app.state.fan_out_limit = fan_out
app.state.proxy_client = httpx.AsyncClient(
timeout=30,
limits=httpx.Limits(
max_connections=fan_out + 50,
max_keepalive_connections=min(fan_out // 4, 100),
),
)
app.state.proxy_sse_client = httpx.AsyncClient(
timeout=httpx.Timeout(connect=5, read=30, write=5, pool=5),
limits=httpx.Limits(keepalive_expiry=30),
limits=httpx.Limits(
max_connections=1100, max_keepalive_connections=100, keepalive_expiry=30
),
)
# Start scheduler if configured
scheduler = getattr(app.state, "scheduler", None)
@@ -1472,10 +1525,10 @@ async def admin_list_watches(request: Request) -> JSONResponse:
if err:
return err
collector: ClusterCollector = request.app.state.collector
nodes, _ = collector.get_nodes(limit=500)
nodes = collector.get_all_nodes()
client: httpx.AsyncClient = request.app.state.proxy_client
headers = _proxy_auth_headers(request)
sem = asyncio.Semaphore(_NODE_FAN_OUT_LIMIT)
sem = asyncio.Semaphore(_get_fan_out_limit(request))
async def _fetch_node(node: dict[str, Any]) -> list[dict[str, Any]]:
server_url = (node.get("server_url") or "").rstrip("/")
@@ -1514,9 +1567,14 @@ async def admin_list_watches(request: Request) -> JSONResponse:
_VALID_WATCH_ID = re.compile(r"^[a-fA-F0-9]+$")
# Max concurrent outbound requests when fanning out to cluster nodes.
# Sized below the default httpx pool limit (100) to leave headroom for
# other proxy traffic (UI proxying, SSE streams, etc.).
_NODE_FAN_OUT_LIMIT = 50
# Must stay below the httpx pool limit (set in _lifespan) to leave
# headroom for non-fan-out proxy traffic (UI proxying, SSE streams).
_NODE_FAN_OUT_LIMIT = 200 # fallback; prefer cluster.node_fan_out_limit from storage
def _get_fan_out_limit(request: Request) -> int:
"""Return the fan-out limit cached at startup on app.state."""
return int(getattr(request.app.state, "fan_out_limit", _NODE_FAN_OUT_LIMIT))
async def admin_cancel_watch(request: Request) -> Response:
@@ -2154,6 +2212,7 @@ _SKILL_RUNTIME_CONFIG_FIELDS = frozenset(
"allowed_tools",
"enabled",
"notify_on_complete",
"priority",
}
)
@@ -2308,6 +2367,7 @@ def _skill_to_response(r: dict[str, Any], resource_count: int = 0) -> dict[str,
"agent_max_turns": r.get("agent_max_turns"),
"notify_on_complete": r.get("notify_on_complete", "{}"),
"enabled": r.get("enabled", True),
"priority": r.get("priority", 0),
"allowed_tools": r.get("allowed_tools", "[]"),
"license": r.get("license", ""),
"compatibility": r.get("compatibility", ""),
@@ -2422,6 +2482,11 @@ async def admin_create_skill(request: Request) -> JSONResponse:
if activation == "default":
is_default = True
try:
priority = max(-1000, min(1000, int(body.get("priority", 0) or 0)))
except (ValueError, TypeError):
priority = 0
if not name:
return JSONResponse({"error": "name is required"}, status_code=400)
if not content:
@@ -2449,6 +2514,7 @@ async def admin_create_skill(request: Request) -> JSONResponse:
compatibility=compatibility,
activation=activation,
token_estimate=token_estimate,
priority=priority,
**session_fields,
)
@@ -2540,6 +2606,11 @@ async def admin_update_skill(request: Request) -> JSONResponse:
except (ValueError, TypeError):
tag_str = "[]"
updates["tags"] = tag_str
if "priority" in body:
try:
updates["priority"] = max(-1000, min(1000, int(body["priority"] or 0)))
except (ValueError, TypeError):
updates["priority"] = 0
# Installed (readonly) skills: restrict updates to runtime config only.
# Spec/content fields are locked to preserve external-source fidelity.
@@ -2703,6 +2774,16 @@ async def admin_usage(request: Request) -> JSONResponse:
group_by=group_by,
)
# Resolve user_id hex → username for display when grouped by user
if group_by == "user" and breakdown:
uid_to_name: dict[str, str] = {}
for u in storage.list_users():
uid_to_name[u["user_id"]] = u.get("username") or u["user_id"]
for row in breakdown:
raw_key = row.get("key", "")
if raw_key and raw_key in uid_to_name:
row["key"] = uid_to_name[raw_key]
return JSONResponse({"summary": summary, "breakdown": breakdown})
@@ -2747,6 +2828,16 @@ async def admin_audit(request: Request) -> JSONResponse:
until=until,
)
# Resolve user_id hex → username for display
if events:
uid_to_name: dict[str, str] = {}
for u in storage.list_users():
uid_to_name[u["user_id"]] = u.get("username") or u["user_id"]
for ev in events:
raw_uid = ev.get("user_id", "")
if raw_uid and raw_uid in uid_to_name:
ev["username"] = uid_to_name[raw_uid]
return JSONResponse({"events": events, "total": total})
@@ -3063,20 +3154,18 @@ async def admin_delete_skill_resource(request: Request) -> JSONResponse:
def _get_discovery_url(request: Request) -> str:
"""Get skills discovery URL from DB settings, config.toml, or default."""
"""Get skills discovery URL via ConfigStore, config.toml, or default."""
from turnstone.core.config import load_config
from turnstone.core.skill_sources import DEFAULT_DISCOVERY_URL
storage = getattr(request.app.state, "auth_storage", None)
if storage:
try:
row = storage.get_system_setting("skills.discovery_url")
if row:
val = json.loads(row["value"])
if val:
return str(val)
except (KeyError, json.JSONDecodeError, TypeError, AttributeError):
pass
# ConfigStore: validated + cached
config_store = getattr(request.app.state, "config_store", None)
if config_store:
val = config_store.get("skills.discovery_url")
if val:
return str(val)
# Fall back to config.toml [skills] section
skills_cfg = load_config("skills")
url = skills_cfg.get("discovery_url", "")
if url:
@@ -3435,31 +3524,40 @@ async def admin_delete_memory(request: Request) -> JSONResponse:
# ---------------------------------------------------------------------------
def _publish_config_change(request: Request, *, key: str, node_id: str, action: str) -> None:
"""Fan out config-reload to all known server nodes (best-effort).
async def _publish_config_change(request: Request) -> None:
"""Fan out config-reload to all known server nodes (best-effort, async).
Uses the collector's node registry and the existing proxy auth
mechanism no MQ dependency.
Uses the collector's node registry, the shared async proxy client,
and bounded concurrency via the fan-out semaphore.
"""
import contextlib
import httpx
# Reload the console's own ConfigStore so cached values stay fresh
# (must happen even when collector is absent — e.g. standalone console)
config_store = getattr(request.app.state, "config_store", None)
if config_store:
config_store.reload()
collector = getattr(request.app.state, "collector", None)
if not collector:
return
client: httpx.AsyncClient = request.app.state.proxy_client
headers = _proxy_auth_headers(request)
with contextlib.suppress(Exception):
nodes = collector.get_nodes()
for node in nodes.get("nodes", []):
url = node.get("url", "")
if url:
with contextlib.suppress(Exception):
httpx.post(
f"{url}/v1/api/_internal/config-reload",
headers=headers,
timeout=5.0,
)
sem = asyncio.Semaphore(_get_fan_out_limit(request))
async def _notify(url: str) -> None:
async with sem:
try:
await client.post(
f"{url.rstrip('/')}/v1/api/_internal/config-reload",
headers=headers,
timeout=5.0,
)
except Exception:
log.warning("Config reload failed for %s", url, exc_info=True)
nodes = collector.get_all_nodes()
tasks = [_notify(n["server_url"]) for n in nodes if n.get("server_url")]
if tasks:
await asyncio.gather(*tasks, return_exceptions=True)
async def admin_list_settings(request: Request) -> JSONResponse:
@@ -3615,7 +3713,7 @@ async def admin_update_setting(request: Request) -> JSONResponse:
ip,
)
_publish_config_change(request, key=key, node_id=node_id, action="set")
await _publish_config_change(request)
return JSONResponse(
{
@@ -3650,7 +3748,7 @@ async def admin_delete_setting(request: Request) -> JSONResponse:
key = request.path_params["key"]
try:
validate_key(key)
defn = validate_key(key)
except ValueError:
return JSONResponse({"error": f"Unknown setting: {key}"}, status_code=400)
@@ -3670,9 +3768,9 @@ async def admin_delete_setting(request: Request) -> JSONResponse:
ip,
)
_publish_config_change(request, key=key, node_id=node_id, action="delete")
await _publish_config_change(request)
return JSONResponse({"status": "ok", "key": key})
return JSONResponse({"status": "ok", "key": key, "default": defn.default})
# ---------------------------------------------------------------------------
@@ -3681,21 +3779,16 @@ async def admin_delete_setting(request: Request) -> JSONResponse:
def _get_registry_url(request: Request) -> str:
"""Get the MCP Registry URL from DB settings, config.toml, or default."""
"""Get the MCP Registry URL via ConfigStore, config.toml, or default."""
from turnstone.core.config import load_config
from turnstone.core.mcp_registry import DEFAULT_REGISTRY_URL
# Check database settings first
storage = getattr(request.app.state, "auth_storage", None)
if storage:
try:
row = storage.get_system_setting("mcp.registry_url")
if row:
val = json.loads(row["value"])
if val:
return str(val)
except (KeyError, json.JSONDecodeError, TypeError, AttributeError):
pass
# ConfigStore: validated + cached
config_store = getattr(request.app.state, "config_store", None)
if config_store:
val = config_store.get("mcp.registry_url")
if val:
return str(val)
# Fall back to config.toml [mcp] section
mcp_cfg = load_config("mcp")
@@ -3846,8 +3939,9 @@ async def admin_registry_install(request: Request) -> JSONResponse:
# Check max servers
current = storage.list_mcp_servers()
if len(current) >= _MCP_MAX_SERVERS:
return JSONResponse({"error": f"Maximum {_MCP_MAX_SERVERS} servers"}, status_code=400)
max_servers = _get_mcp_max_servers(request)
if len(current) >= max_servers:
return JSONResponse({"error": f"Maximum {max_servers} servers"}, status_code=400)
# Fetch the specific server from the registry
registry_url = _get_registry_url(request)
@@ -3943,7 +4037,15 @@ async def admin_registry_install(request: Request) -> JSONResponse:
# ---------------------------------------------------------------------------
_MCP_NAME_RE = re.compile(r"^[a-zA-Z0-9._-]+$")
_MCP_MAX_SERVERS = 50
_MCP_MAX_SERVERS = 200 # fallback; prefer cluster.mcp_max_servers from storage
def _get_mcp_max_servers(request: Request) -> int:
"""Read cluster.mcp_max_servers via ConfigStore (validated + cached)."""
config_store = getattr(request.app.state, "config_store", None)
if config_store:
return int(config_store.get("cluster.mcp_max_servers"))
return _MCP_MAX_SERVERS
def _mask_mcp_secrets(server: dict[str, Any], reveal: bool = False) -> dict[str, Any]:
@@ -3981,10 +4083,10 @@ async def _collect_mcp_status(
) -> dict[str, dict[str, dict[str, Any]]]:
"""Query all nodes for MCP status. Returns {node_id: {server_name: status}}."""
collector: ClusterCollector = request.app.state.collector
nodes, _ = collector.get_nodes(sort_by="activity", limit=1000, offset=0)
nodes = collector.get_all_nodes()
client: httpx.AsyncClient = request.app.state.proxy_client
headers = _proxy_auth_headers(request)
sem = asyncio.Semaphore(_NODE_FAN_OUT_LIMIT)
sem = asyncio.Semaphore(_get_fan_out_limit(request))
async def _fetch(node: dict[str, Any]) -> tuple[str, dict[str, dict[str, Any]] | None]:
node_id = node.get("node_id", "")
@@ -4128,9 +4230,10 @@ async def admin_create_mcp_server(request: Request) -> JSONResponse:
# Check max servers
existing = storage.list_mcp_servers()
if len(existing) >= _MCP_MAX_SERVERS:
max_servers = _get_mcp_max_servers(request)
if len(existing) >= max_servers:
return JSONResponse(
{"error": f"Maximum {_MCP_MAX_SERVERS} servers"},
{"error": f"Maximum {max_servers} servers"},
status_code=400,
)
@@ -4332,10 +4435,10 @@ async def admin_delete_mcp_server(request: Request) -> JSONResponse:
async def _notify_nodes_mcp_reload(request: Request) -> dict[str, Any]:
"""Tell all nodes to re-read the mcp_servers DB table and reconcile."""
collector: ClusterCollector = request.app.state.collector
nodes, _ = collector.get_nodes(sort_by="activity", limit=1000, offset=0)
nodes = collector.get_all_nodes()
client: httpx.AsyncClient = request.app.state.proxy_client
headers = _proxy_auth_headers(request)
sem = asyncio.Semaphore(_NODE_FAN_OUT_LIMIT)
sem = asyncio.Semaphore(_get_fan_out_limit(request))
async def _notify(node: dict[str, Any]) -> tuple[str, Any]:
node_id = node.get("node_id", "")
@@ -4411,6 +4514,7 @@ async def admin_import_mcp_config(request: Request) -> JSONResponse:
errors: list[str] = []
audit_uid, ip = _audit_context(request)
current_count = len(storage.list_mcp_servers())
max_servers = _get_mcp_max_servers(request)
for srv_name, cfg in servers.items():
srv_name = str(srv_name).strip()[:64]
@@ -4420,7 +4524,7 @@ async def admin_import_mcp_config(request: Request) -> JSONResponse:
if storage.get_mcp_server_by_name(srv_name):
skipped.append(srv_name)
continue
if current_count >= _MCP_MAX_SERVERS:
if current_count >= max_servers:
errors.append(f"{srv_name}: max servers reached")
break
@@ -4822,8 +4926,9 @@ def main() -> None:
help="Bearer token for polling turnstone-server nodes (default: $TURNSTONE_AUTH_TOKEN)",
)
from turnstone.core.config import apply_config
from turnstone.core.config import add_config_arg, apply_config
add_config_arg(parser)
apply_config(parser, ["console", "redis", "auth"])
args = parser.parse_args()
+4
View File
@@ -2057,10 +2057,12 @@ var _settingsSectionOrder = [
"session",
"tools",
"server",
"cluster",
"mcp",
"ratelimit",
"health",
"judge",
"skills",
"memory",
];
@@ -2070,10 +2072,12 @@ function _settingsSectionLabel(section) {
session: "Session",
tools: "Tools",
server: "Server",
cluster: "Cluster",
mcp: "MCP",
ratelimit: "Rate Limiting",
health: "Health",
judge: "Judge",
skills: "Skills",
memory: "Memory",
};
return labels[section] || section;
+3 -1
View File
@@ -1838,7 +1838,9 @@ function _renderGovAudit(events, total) {
_relativeTime(ev.timestamp) +
"</span>" +
'<span class="admin-col admin-col-auser">' +
escapeHtml(ev.user_id ? ev.user_id.slice(0, 8) : "\u2014") +
escapeHtml(
ev.username || (ev.user_id ? ev.user_id.slice(0, 8) : "\u2014"),
) +
"</span>" +
'<span class="admin-col admin-col-aaction"><span class="' +
actionCls +
+3 -2
View File
@@ -7,14 +7,15 @@ call after mutations to create a persistent audit trail.
from __future__ import annotations
import json
import logging
import uuid
from typing import TYPE_CHECKING, Any
from turnstone.core.log import get_logger
if TYPE_CHECKING:
from turnstone.core.storage._protocol import StorageBackend
log = logging.getLogger(__name__)
log = get_logger(__name__)
def record_audit(
+18 -8
View File
@@ -18,11 +18,9 @@ always accessible without authentication.
from __future__ import annotations
import contextlib
import hashlib
import hmac
import json
import logging
import os
import re
import secrets
@@ -40,7 +38,9 @@ if TYPE_CHECKING:
from turnstone.core.oidc import OIDCConfig
log = logging.getLogger(__name__)
from turnstone.core.log import get_logger
log = get_logger(__name__)
# ---------------------------------------------------------------------------
# Constants
@@ -329,18 +329,22 @@ def create_jwt(
expiry_hours: int = 24,
audience: str = "",
permissions: frozenset[str] = frozenset(),
expiry_seconds: int | None = None,
) -> str:
"""Create a signed JWT with user identity, scopes, and permissions."""
import jwt
if expiry_seconds is not None and expiry_seconds <= 0:
raise ValueError("expiry_seconds must be positive")
now = int(time.time())
ttl = expiry_seconds if expiry_seconds is not None else expiry_hours * 3600
payload: dict[str, Any] = {
"sub": user_id,
"scopes": ",".join(sorted(scopes)),
"src": source,
"iss": JWT_ISSUER,
"iat": now,
"exp": now + expiry_hours * 3600,
"exp": now + ttl,
}
if audience:
payload["aud"] = audience
@@ -1010,7 +1014,7 @@ async def handle_auth_status(request: Request) -> Response:
users = storage.list_users()
has_users = len(users) > 0
except Exception:
pass
log.warning("Failed to check user existence for auth status", exc_info=True)
# OIDC configuration
oidc_config = getattr(request.app.state, "oidc_config", None)
@@ -1079,8 +1083,10 @@ async def handle_auth_setup(request: Request, audience: str) -> Response:
except Exception:
log.error("Failed to assign admin role to first user %s — aborting setup", user_id)
# Roll back the user creation so setup can be retried
with contextlib.suppress(Exception):
try:
storage.delete_user(user_id)
except Exception:
log.error("Failed to roll back user %s during setup abort", user_id, exc_info=True)
return JSONResponse(
{"error": "Failed to assign admin role. Ensure migrations have run."},
status_code=503,
@@ -1092,8 +1098,10 @@ async def handle_auth_setup(request: Request, audience: str) -> Response:
log.error(
"First user %s has no permissions after role assignment — aborting setup", user_id
)
with contextlib.suppress(Exception):
try:
storage.delete_user(user_id)
except Exception:
log.error("Failed to roll back user %s during setup abort", user_id, exc_info=True)
return JSONResponse(
{"error": "Failed to load permissions. Ensure migrations have run."},
status_code=503,
@@ -1229,8 +1237,10 @@ async def handle_oidc_callback(request: Request, audience: str) -> Response:
return RedirectResponse("/?oidc_error=Too+many+login+attempts", status_code=302)
# Lazy cleanup of expired pending states
with contextlib.suppress(Exception):
try:
storage.cleanup_expired_oidc_states(300)
except Exception:
log.debug("OIDC state cleanup failed", exc_info=True)
def _record_oidc_failure() -> None:
if login_limiter is not None:
+64 -10
View File
@@ -1,28 +1,59 @@
"""Unified configuration for turnstone.
Loads ``~/.config/turnstone/config.toml`` and applies values as argparse defaults.
Precedence: CLI args > env vars > config file > hardcoded defaults.
Loads config.toml and applies values as argparse defaults.
Precedence: CLI args > config file > hardcoded defaults.
Config file resolution:
1. ``--config PATH`` CLI flag (via ``add_config_arg`` pre-parser)
2. ``$TURNSTONE_CONFIG`` environment variable
3. ``~/.config/turnstone/config.toml`` (default)
"""
from __future__ import annotations
import logging
import os
import tomllib
from pathlib import Path
from typing import TYPE_CHECKING, Any
from turnstone.core.log import get_logger
if TYPE_CHECKING:
import argparse
log = logging.getLogger(__name__)
log = get_logger(__name__)
CONFIG_DIR = Path("~/.config/turnstone").expanduser()
CONFIG_PATH = CONFIG_DIR / "config.toml"
_DEFAULT_CONFIG_PATH = CONFIG_DIR / "config.toml"
# Resolved config path — set by set_config_path() or $TURNSTONE_CONFIG
_config_path: Path | None = None
# Cache: None = not loaded yet, {} = loaded but empty/missing
_cache: dict[str, Any] | None = None
def _resolve_config_path() -> Path:
"""Return the effective config file path."""
if _config_path is not None:
return _config_path
env = os.environ.get("TURNSTONE_CONFIG", "").strip()
if env:
return Path(env).expanduser()
return _DEFAULT_CONFIG_PATH
def set_config_path(path: str) -> None:
"""Override the config file path.
Invalidates the cache so subsequent ``load_config()`` calls re-read
from the new path. Typically called from ``add_config_arg()``.
"""
global _config_path, _cache
_config_path = Path(path).expanduser()
_cache = None # invalidate cache so next load_config() re-reads
def load_config(section: str | None = None) -> dict[str, Any]:
"""Load config.toml and return the full dict or a specific section.
@@ -32,11 +63,12 @@ def load_config(section: str | None = None) -> dict[str, Any]:
global _cache
if _cache is None:
_cache = {}
if CONFIG_PATH.is_file():
cfg_path = _resolve_config_path()
if cfg_path.is_file():
try:
_cache = tomllib.loads(CONFIG_PATH.read_text(encoding="utf-8"))
_cache = tomllib.loads(cfg_path.read_text(encoding="utf-8"))
except Exception as exc:
log.warning("Failed to parse %s: %s", CONFIG_PATH, exc)
log.warning("Failed to parse %s: %s", cfg_path, exc)
if section:
result = _cache.get(section, {})
return result if isinstance(result, dict) else {}
@@ -73,6 +105,7 @@ _CONFIG_MAP: dict[str, dict[str, str]] = {
"search": "tool_search",
"search_threshold": "tool_search_threshold",
"search_max_results": "tool_search_max_results",
"web_search_backend": "web_search_backend",
},
"server": {
"host": "host",
@@ -156,8 +189,6 @@ def get_tavily_key() -> str | None:
Precedence: config.toml [api] tavily_key -> $TAVILY_API_KEY
"""
import os
global _tavily_key, _tavily_key_loaded
if _tavily_key_loaded:
return _tavily_key
@@ -202,6 +233,29 @@ def apply_config(parser: argparse.ArgumentParser, sections: list[str]) -> None:
parser.set_defaults(**defaults)
def add_config_arg(parser: argparse.ArgumentParser) -> None:
"""Add ``--config`` to *parser* and resolve the path before returning.
Uses a separate pre-parser (``add_help=False``) so ``--help`` on the
main parser still works and shows config-derived defaults.
"""
import argparse as _ap
import sys
parser.add_argument(
"--config",
default=None,
metavar="PATH",
help="Path to config.toml (default: $TURNSTONE_CONFIG or ~/.config/turnstone/config.toml)",
)
# Pre-parse only --config without intercepting --help
pre = _ap.ArgumentParser(add_help=False)
pre.add_argument("--config", default=None)
pre_args, _ = pre.parse_known_args(sys.argv[1:])
if pre_args.config:
set_config_path(pre_args.config)
def warn_migrated_settings() -> None:
"""Log warnings for config.toml keys that are now managed by ConfigStore.
+2 -2
View File
@@ -18,10 +18,10 @@ ConfigStore) — it is a standalone tool, not a cluster node.
from __future__ import annotations
import logging
import threading
from typing import TYPE_CHECKING, Any
from turnstone.core.log import get_logger
from turnstone.core.settings_registry import (
SETTINGS,
deserialize_value,
@@ -33,7 +33,7 @@ from turnstone.core.settings_registry import (
if TYPE_CHECKING:
from turnstone.core.storage._protocol import StorageBackend
log = logging.getLogger(__name__)
log = get_logger(__name__)
_UNSET: Any = object()
+120
View File
@@ -0,0 +1,120 @@
"""Subprocess environment scrubbing.
Builds a sanitized copy of ``os.environ`` that strips secrets
(API keys, tokens, passwords) while preserving variables needed
for normal tool operation (PATH, HOME, locale, etc.).
"""
from __future__ import annotations
import os
# Env var names that are always preserved regardless of pattern matching.
_SAFE_NAMES: frozenset[str] = frozenset(
{
"PATH",
"HOME",
"USER",
"SHELL",
"LANG",
"TERM",
"TMPDIR",
"TMP",
"TEMP",
"EDITOR",
"VISUAL",
"COLORTERM",
"COLUMNS",
"LINES",
"PWD",
"OLDPWD",
"HOSTNAME",
"LOGNAME",
"DISPLAY",
"WAYLAND_DISPLAY",
"SSH_AUTH_SOCK",
"GPG_AGENT_INFO",
"SHLVL",
"MANWIDTH",
"MAN_KEEP_FORMATTING",
"LESS",
"LESSOPEN",
"LESSCLOSE",
"LESSPIPE",
"LESSCHARSET",
}
)
# Prefixes that are always preserved (locale, XDG, etc.).
_SAFE_PREFIXES: tuple[str, ...] = ("LC_", "XDG_")
# Suffixes that cause a variable to be scrubbed (e.g. *_KEY, *_TOKEN).
# Suffix matching avoids false positives on MONKEYTYPE, KEYBOARD_LAYOUT, etc.
_SECRET_SUFFIXES: tuple[str, ...] = (
"_KEY",
"_SECRET",
"_TOKEN",
"_PASSWORD",
"_CREDENTIAL",
"_CREDENTIALS",
)
# Exact names that are always scrubbed (even if they don't match patterns).
_EXPLICIT_SCRUB: frozenset[str] = frozenset(
{
"OPENAI_API_KEY",
"ANTHROPIC_API_KEY",
"TAVILY_API_KEY",
"TURNSTONE_JWT_SECRET",
"TURNSTONE_AUTH_TOKEN",
"TURNSTONE_DISCORD_TOKEN",
"TURNSTONE_GITHUB_TOKEN",
"TURNSTONE_OIDC_CLIENT_SECRET",
"AWS_SECRET_ACCESS_KEY",
"AZURE_CLIENT_SECRET",
"GCP_SERVICE_ACCOUNT_KEY",
"GOOGLE_APPLICATION_CREDENTIALS",
"DATABASE_URL",
"TURNSTONE_DB_URL",
}
)
def _is_secret(name: str) -> bool:
"""Return True if *name* looks like a secret variable."""
if name in _EXPLICIT_SCRUB:
return True
upper = name.upper()
return any(upper.endswith(sfx) for sfx in _SECRET_SUFFIXES)
def _is_safe(name: str) -> bool:
"""Return True if *name* should always be preserved."""
if name in _SAFE_NAMES:
return True
return any(name.startswith(pfx) for pfx in _SAFE_PREFIXES)
def scrubbed_env(
extra: dict[str, str] | None = None,
passthrough: list[str] | None = None,
) -> dict[str, str]:
"""Return a copy of ``os.environ`` with secrets removed.
Args:
extra: Additional variables to merge on top (e.g. ``MANWIDTH``).
passthrough: Explicit variable names to preserve even if they
match secret patterns (operator override).
"""
passthrough_set = frozenset(passthrough) if passthrough else frozenset()
env: dict[str, str] = {}
for name, value in os.environ.items():
if name in passthrough_set or _is_safe(name):
env[name] = value
elif _is_secret(name):
continue
else:
env[name] = value
if extra:
env.update(extra)
return env
+37 -3
View File
@@ -3,15 +3,16 @@
from __future__ import annotations
import enum
import logging
import threading
import time
from typing import TYPE_CHECKING
from turnstone.core.log import get_logger
if TYPE_CHECKING:
from openai import OpenAI
log = logging.getLogger(__name__)
log = get_logger(__name__)
class CircuitState(enum.Enum):
@@ -149,11 +150,44 @@ class BackendHealthMonitor:
# ------------------------------------------------------------------
def _probe_loop(self) -> None:
"""Background: probe backend every interval."""
"""Background: probe backend every interval.
An initial jitter (derived from the PID) staggers probes across
cluster nodes so they don't all hit the LLM backend at once.
"""
import os
# Deterministic per-process jitter: spread across half the interval
jitter = ((os.getpid() * 2654435761) & 0x7FFFFFFF) / 0x7FFFFFFF * (self._probe_interval / 2)
self._stop_event.wait(jitter)
while not self._stop_event.is_set():
self._stop_event.wait(self._probe_interval)
if self._stop_event.is_set():
break
# When circuit is OPEN, only probe after cooldown expires.
with self._lock:
if self._state == CircuitState.OPEN:
elapsed = time.monotonic() - self._last_state_change
remaining = self._cooldown - elapsed
if remaining > 0:
# Wait precisely for cooldown rather than skipping
# a full probe_interval (which could overshoot).
self._lock.release()
try:
self._stop_event.wait(remaining)
finally:
self._lock.acquire()
if self._stop_event.is_set():
break
# Transition to HALF_OPEN for the probe. The background
# probe itself is the single HALF_OPEN request — keep
# _half_open_permit False so concurrent user requests
# are blocked until the probe completes.
self._state = CircuitState.HALF_OPEN
self._half_open_permit = False
self._last_state_change = time.monotonic()
log.info("Circuit breaker HALF_OPEN: cooldown elapsed, probing")
self._update_metrics()
success = self._probe_once()
if success:
self.record_success()
+3 -2
View File
@@ -9,7 +9,6 @@ from __future__ import annotations
import fnmatch
import json
import logging
import os
import re
import threading
@@ -20,12 +19,14 @@ from dataclasses import dataclass, field
from pathlib import Path
from typing import TYPE_CHECKING, Any
from turnstone.core.log import get_logger
if TYPE_CHECKING:
from collections.abc import Callable
from turnstone.core.providers._protocol import LLMProvider
log = logging.getLogger(__name__)
log = get_logger(__name__)
# ---------------------------------------------------------------------------
# Data structures
+18
View File
@@ -149,6 +149,24 @@ def configure_logging(
logging.getLogger(name).setLevel(logging.WARNING)
def _ensure_stdlib_factory() -> None:
"""Ensure structlog routes through stdlib even before configure_logging().
Without this, ``structlog.get_logger()`` defaults to ``PrintLogger``
which bypasses stdlib handlers (and pytest caplog). Calling
``configure_logging()`` later overwrites this minimal config.
"""
cfg = structlog.get_config()
if not isinstance(cfg.get("logger_factory"), structlog.stdlib.LoggerFactory):
structlog.configure(
logger_factory=structlog.stdlib.LoggerFactory(),
wrapper_class=structlog.stdlib.BoundLogger,
)
_ensure_stdlib_factory()
def get_logger(name: str) -> structlog.stdlib.BoundLogger:
"""Return a structlog bound logger backed by the stdlib."""
result: structlog.stdlib.BoundLogger = structlog.get_logger(name)
+11 -7
View File
@@ -22,7 +22,6 @@ import asyncio
import concurrent.futures
import contextlib
import json
import logging
import os
import random
import threading
@@ -41,8 +40,9 @@ from mcp.client.stdio import stdio_client
from mcp.client.streamable_http import streamablehttp_client
from turnstone.core.config import load_config
from turnstone.core.log import get_logger
log = logging.getLogger("turnstone.mcp")
log = get_logger("turnstone.mcp")
_DEFAULT_REFRESH_INTERVAL: float = 14400 # 4 hours
@@ -162,9 +162,11 @@ class MCPClientManager:
future = asyncio.run_coroutine_threadsafe(self._connect_all(), self._loop)
self._connected.wait(timeout=30)
# Surface any exception from _connect_all (unlikely — per-server errors are caught)
if future.done() and future.exception():
self._error = str(future.exception())
log.error("MCP initialization error: %s", self._error)
if future.done() and not future.cancelled():
exc = future.exception()
if exc:
self._error = str(exc)
log.error("MCP initialization error: %s", self._error)
async def _connect_all(self) -> None:
"""Connect to every configured server (runs on the background loop)."""
@@ -224,7 +226,9 @@ class MCPClientManager:
log.warning("MCP server '%s' has no command configured", name)
await stack.aclose()
return
env = {**os.environ, **cfg.get("env", {})}
from turnstone.core.env import scrubbed_env
env = scrubbed_env(extra=cfg.get("env", {}))
params = StdioServerParameters(
command=command,
args=cfg.get("args", []),
@@ -1415,7 +1419,7 @@ def create_mcp_client(
if rows:
db_names = {r["name"] for r in rows}
except Exception:
pass
log.warning("Failed to load DB-managed MCP servers", exc_info=True)
servers = load_mcp_config(config_path, storage=storage)
if not servers:
+12
View File
@@ -11,6 +11,7 @@ from __future__ import annotations
import re
from dataclasses import dataclass, field
from typing import Any
from urllib.parse import urlparse
import httpx
@@ -400,6 +401,17 @@ def resolve_install_config(
raise MCPRegistryError(f"Required URL variable '{var_name}' not provided")
url = url.replace(placeholder, value)
# Validate URL after substitution to prevent SSRF-style redirection
parsed = urlparse(url)
if parsed.scheme not in ("http", "https"):
raise MCPRegistryError(
f"Invalid URL scheme '{parsed.scheme}' after variable substitution"
)
if not parsed.hostname:
raise MCPRegistryError("Invalid URL (hostname is missing) after variable substitution")
if parsed.username is not None or parsed.password is not None:
raise MCPRegistryError("URLs with embedded credentials are not allowed in MCP remotes")
# Build headers dict (required keys only — values provided by user at install time)
headers: dict[str, str] = {}
for h in remote.headers:
+66 -7
View File
@@ -3,20 +3,26 @@
All functions maintain their existing signatures for consumers (session.py,
server.py, cli.py). The actual storage implementation lives in
``turnstone.core.storage``.
The no-raise contract is preserved callers never see exceptions from this
module. All failures are logged so storage issues are visible in logs
rather than silently swallowed.
"""
from __future__ import annotations
import contextlib
from typing import TYPE_CHECKING, Any
import sqlalchemy as sa
from turnstone.core.log import get_logger
from turnstone.core.storage import get_storage
if TYPE_CHECKING:
from collections.abc import Callable
log = get_logger(__name__)
def normalize_key(key: str) -> str:
"""Normalize a memory key for consistent lookup."""
@@ -37,7 +43,7 @@ def save_message(
tool_calls: str | None = None,
) -> None:
"""Log a message to the conversations table."""
with contextlib.suppress(Exception):
try:
get_storage().save_message(
ws_id,
role,
@@ -48,6 +54,8 @@ def save_message(
provider_data,
tool_calls=tool_calls,
)
except Exception:
log.warning("Failed to save message for ws=%s role=%s", ws_id, role, exc_info=True)
def load_messages(ws_id: str) -> list[dict[str, Any]]:
@@ -55,6 +63,7 @@ def load_messages(ws_id: str) -> list[dict[str, Any]]:
try:
return get_storage().load_messages(ws_id)
except Exception:
log.warning("Failed to load messages for ws=%s", ws_id, exc_info=True)
return []
@@ -70,22 +79,28 @@ def register_workstream(
skill_version: int = 0,
) -> None:
"""Persist a new workstream (no-op if already exists)."""
with contextlib.suppress(Exception):
try:
get_storage().register_workstream(
ws_id, node_id, name, state, skill_id=skill_id, skill_version=skill_version
)
except Exception:
log.warning("Failed to register workstream ws=%s", ws_id, exc_info=True)
def update_workstream_state(ws_id: str, state: str) -> None:
"""Update a workstream's state."""
with contextlib.suppress(Exception):
try:
get_storage().update_workstream_state(ws_id, state)
except Exception:
log.warning("Failed to update workstream state ws=%s state=%s", ws_id, state, exc_info=True)
def update_workstream_name(ws_id: str, name: str) -> None:
"""Update a workstream's display name."""
with contextlib.suppress(Exception):
try:
get_storage().update_workstream_name(ws_id, name)
except Exception:
log.warning("Failed to update workstream name ws=%s", ws_id, exc_info=True)
def list_workstreams(node_id: str | None = None, limit: int = 100) -> list[Any]:
@@ -93,6 +108,7 @@ def list_workstreams(node_id: str | None = None, limit: int = 100) -> list[Any]:
try:
return get_storage().list_workstreams(node_id, limit)
except Exception:
log.warning("Failed to list workstreams", exc_info=True)
return []
@@ -101,6 +117,7 @@ def list_workstreams_with_history(limit: int = 20) -> list[Any]:
try:
return get_storage().list_workstreams_with_history(limit)
except Exception:
log.warning("Failed to list workstreams with history", exc_info=True)
return []
@@ -109,6 +126,7 @@ def delete_workstream(ws_id: str) -> bool:
try:
return get_storage().delete_workstream(ws_id)
except Exception:
log.warning("Failed to delete workstream ws=%s", ws_id, exc_info=True)
return False
@@ -120,6 +138,7 @@ def prune_workstreams(
try:
orphans, stale = get_storage().prune_workstreams(retention_days)
except Exception:
log.warning("Failed to prune workstreams", exc_info=True)
return (0, 0)
if log_fn and (orphans or stale):
@@ -140,6 +159,7 @@ def resolve_workstream(alias_or_id: str) -> str | None:
try:
return get_storage().resolve_workstream(alias_or_id)
except Exception:
log.warning("Failed to resolve workstream alias=%s", alias_or_id, exc_info=True)
return None
@@ -148,8 +168,10 @@ def resolve_workstream(alias_or_id: str) -> str | None:
def save_workstream_config(ws_id: str, config: dict[str, str]) -> None:
"""Persist workstream configuration key/value pairs."""
with contextlib.suppress(Exception):
try:
get_storage().save_workstream_config(ws_id, config)
except Exception:
log.warning("Failed to save workstream config ws=%s", ws_id, exc_info=True)
def load_workstream_config(ws_id: str) -> dict[str, str]:
@@ -157,6 +179,7 @@ def load_workstream_config(ws_id: str) -> dict[str, str]:
try:
return get_storage().load_workstream_config(ws_id)
except Exception:
log.warning("Failed to load workstream config ws=%s", ws_id, exc_info=True)
return {}
@@ -168,6 +191,7 @@ def get_skill_by_name(name: str) -> dict[str, Any] | None:
try:
return get_storage().get_prompt_template_by_name(name)
except Exception:
log.warning("Failed to get skill name=%s", name, exc_info=True)
return None
@@ -176,6 +200,7 @@ def list_default_skills(org_id: str = "") -> list[dict[str, Any]]:
try:
return get_storage().list_default_templates(org_id)
except Exception:
log.warning("Failed to list default skills", exc_info=True)
return []
@@ -191,6 +216,7 @@ def list_skills_by_activation(
activation, enabled_only=enabled_only, limit=limit
)
except Exception:
log.warning("Failed to list skills by activation=%s", activation, exc_info=True)
return []
@@ -202,6 +228,7 @@ def set_workstream_alias(ws_id: str, alias: str) -> bool:
try:
return get_storage().set_workstream_alias(ws_id, alias)
except Exception:
log.warning("Failed to set alias ws=%s alias=%s", ws_id, alias, exc_info=True)
return False
@@ -210,13 +237,16 @@ def get_workstream_display_name(ws_id: str) -> str | None:
try:
return get_storage().get_workstream_display_name(ws_id)
except Exception:
log.warning("Failed to get display name ws=%s", ws_id, exc_info=True)
return None
def update_workstream_title(ws_id: str, title: str) -> None:
"""Set or update the auto-generated title for a workstream."""
with contextlib.suppress(Exception):
try:
get_storage().update_workstream_title(ws_id, title)
except Exception:
log.warning("Failed to update title ws=%s", ws_id, exc_info=True)
# -- Conversation search -------------------------------------------------------
@@ -227,6 +257,7 @@ def search_history(query: str, limit: int = 20) -> list[Any]:
try:
return get_storage().search_history(query, limit)
except Exception:
log.warning("Failed to search history", exc_info=True)
return []
@@ -235,6 +266,7 @@ def search_history_recent(limit: int = 20) -> list[Any]:
try:
return get_storage().search_history_recent(limit)
except Exception:
log.warning("Failed to search recent history", exc_info=True)
return []
@@ -280,6 +312,7 @@ def save_structured_memory(
return existing["memory_id"], old_content
return "", None
except Exception:
log.warning("Failed to save structured memory name=%s", name, exc_info=True)
return "", None
@@ -289,6 +322,7 @@ def delete_structured_memory(name: str, scope: str = "global", scope_id: str = "
try:
return get_storage().delete_structured_memory(name, scope, scope_id)
except Exception:
log.warning("Failed to delete structured memory name=%s", name, exc_info=True)
return False
@@ -297,6 +331,7 @@ def delete_structured_memory_by_id(memory_id: str) -> bool:
try:
return get_storage().delete_structured_memory_by_id(memory_id)
except Exception:
log.warning("Failed to delete structured memory id=%s", memory_id, exc_info=True)
return False
@@ -312,6 +347,7 @@ def list_structured_memories(
mem_type=mem_type, scope=scope, scope_id=scope_id, limit=limit
)
except Exception:
log.warning("Failed to list structured memories", exc_info=True)
return []
@@ -328,9 +364,31 @@ def search_structured_memories(
query, mem_type=mem_type, scope=scope, scope_id=scope_id, limit=limit
)
except Exception:
log.warning("Failed to search structured memories", exc_info=True)
return []
def touch_structured_memories(keys: list[tuple[str, str, str]]) -> int:
"""Batch-touch memories (bump last_accessed, increment access_count).
Each key is ``(name, scope, scope_id)``. Duplicates are removed so each
distinct memory is touched at most once. Returns count of rows updated.
"""
if not keys:
return 0
seen: set[tuple[str, str, str]] = set()
unique: list[tuple[str, str, str]] = []
for k in keys:
if k not in seen:
seen.add(k)
unique.append(k)
try:
return get_storage().touch_structured_memories(unique)
except Exception:
log.warning("Failed to touch structured memories", exc_info=True)
return 0
def count_structured_memories(mem_type: str = "", scope: str = "", scope_id: str = "") -> int:
"""Count structured memories with optional type/scope filter."""
try:
@@ -338,4 +396,5 @@ def count_structured_memories(mem_type: str = "", scope: str = "", scope_id: str
mem_type=mem_type, scope=scope, scope_id=scope_id
)
except Exception:
log.warning("Failed to count structured memories", exc_info=True)
return 0
+28 -10
View File
@@ -7,15 +7,15 @@ resilience when the primary model is unreachable.
from __future__ import annotations
import logging
import threading
from dataclasses import dataclass, field
from typing import Any
from turnstone.core.config import load_config
from turnstone.core.log import get_logger
from turnstone.core.providers import LLMProvider, create_client, create_provider
log = logging.getLogger(__name__)
log = get_logger(__name__)
# ---------------------------------------------------------------------------
@@ -278,7 +278,9 @@ def detect_model(
client: Any,
log_fn: Any = print,
provider: str = "openai",
) -> tuple[str, int | None]:
*,
fatal: bool = True,
) -> tuple[str | None, int | None]:
"""Auto-detect the model and context window from the API's models endpoint.
Returns ``(model_id, context_window)`` where *context_window* is
@@ -289,13 +291,25 @@ def detect_model(
For local single-model servers (vLLM, llama.cpp), uses the first model.
Calls ``log_fn`` for informational messages (defaults to ``print``).
Raises ``SystemExit`` on failure.
When *fatal* is ``True`` (default), raises ``SystemExit`` on failure.
When ``False``, returns ``(None, None)`` so the server can start in
degraded mode (useful for cluster deployments where the LLM backend
may not be available at startup).
"""
try:
models = client.models.list()
# Use a short timeout for startup detection — the default OpenAI client
# timeout is 600s read which blocks the main thread for minutes when the
# backend is unreachable (TCP SYN dropped → kernel retransmit timeout).
# Disable retries (default 2) to avoid compounding the delay.
fast_client = client.with_options(timeout=10.0, max_retries=0)
models = fast_client.models.list()
if not models.data:
log_fn("Error: No models found at server. Use --model to specify.")
raise SystemExit(1)
if fatal:
log_fn("Error: No models found at server. Use --model to specify.")
raise SystemExit(1)
log_fn("Warning: No models found at server — starting in degraded mode.")
return None, None
all_ids = [x.id for x in models.data]
selected_id = _select_best_model(all_ids, provider)
@@ -321,6 +335,10 @@ def detect_model(
except SystemExit:
raise
except Exception as e:
log_fn(f"Error: Could not connect to server: {e}")
log_fn("Is the model server running? Start it or use --base-url to point elsewhere.")
raise SystemExit(1) from e
if fatal:
log_fn(f"Error: Could not connect to server: {e}")
log_fn("Is the model server running? Start it or use --base-url to point elsewhere.")
raise SystemExit(1) from e
log_fn(f"Warning: Could not connect to LLM backend: {e}")
log_fn("Starting in degraded mode — requests will fail until backend is reachable.")
return None, None
+66 -2
View File
@@ -10,10 +10,11 @@ from __future__ import annotations
import base64
import dataclasses
import hashlib
import logging
import ipaddress
import os
import re
import secrets
import socket
import urllib.parse
import uuid
from dataclasses import dataclass, field
@@ -21,7 +22,9 @@ from typing import Any
import httpx
log = logging.getLogger(__name__)
from turnstone.core.log import get_logger
log = get_logger(__name__)
# Sentinel password hash for OIDC-provisioned users.
# Not a valid bcrypt hash -- verify_password() always rejects it.
@@ -212,6 +215,61 @@ def load_oidc_config() -> OIDCConfig:
)
# ---------------------------------------------------------------------------
# SSRF validation
# ---------------------------------------------------------------------------
def _is_localhost(hostname: str) -> bool:
"""Return True if *hostname* refers to the loopback interface."""
return hostname in ("localhost", "127.0.0.1", "::1") or hostname.endswith(".localhost")
def validate_issuer_url(url: str) -> None:
"""Validate an OIDC issuer URL to prevent SSRF.
Rejects:
- Non-HTTPS URLs (except localhost for development)
- URLs with embedded credentials (userinfo)
- Hostnames that resolve to private/internal/loopback IP addresses
Raises :class:`OIDCError` on validation failure.
"""
parsed = urllib.parse.urlparse(url)
# Require a hostname.
hostname = parsed.hostname
if not hostname:
raise OIDCError(f"OIDC issuer URL has no hostname: {url}")
# Reject embedded credentials — redact userinfo from error message.
if parsed.username or parsed.password:
raise OIDCError("OIDC issuer URL must not contain embedded credentials (userinfo)")
# Require HTTPS (allow HTTP only for localhost development).
if parsed.scheme != "https":
if parsed.scheme == "http" and _is_localhost(hostname):
pass # Allow http://localhost for dev
else:
raise OIDCError(f"OIDC issuer URL must use HTTPS (got {parsed.scheme}://): {url}")
# Resolve hostname and reject non-globally-routable addresses.
try:
addr_infos = socket.getaddrinfo(hostname, None, proto=socket.IPPROTO_TCP)
except socket.gaierror as exc:
raise OIDCError(f"OIDC issuer hostname cannot be resolved: {hostname}") from exc
for _family, _type, _proto, _canonname, sockaddr in addr_infos:
try:
addr = ipaddress.ip_address(sockaddr[0])
except ValueError as exc:
raise OIDCError(
f"OIDC issuer hostname resolved to invalid IP {sockaddr[0]!r}: {hostname}"
) from exc
if not addr.is_global and not _is_localhost(hostname):
raise OIDCError(f"OIDC issuer URL resolves to non-public address ({addr}): {url}")
# ---------------------------------------------------------------------------
# Discovery
# ---------------------------------------------------------------------------
@@ -225,6 +283,12 @@ async def discover_oidc(config: OIDCConfig) -> OIDCConfig:
if not config.issuer:
return dataclasses.replace(config, enabled=False)
try:
validate_issuer_url(config.issuer)
except OIDCError as exc:
log.warning("OIDC issuer URL rejected: %s", exc)
return dataclasses.replace(config, enabled=False)
url = config.issuer.rstrip("/") + "/.well-known/openid-configuration"
try:
async with httpx.AsyncClient(timeout=10.0) as client:
+3 -2
View File
@@ -7,13 +7,14 @@ a tool should be auto-allowed, denied, or require human approval.
from __future__ import annotations
import fnmatch
import logging
from typing import TYPE_CHECKING
from turnstone.core.log import get_logger
if TYPE_CHECKING:
from turnstone.core.storage._protocol import StorageBackend
log = logging.getLogger(__name__)
log = get_logger(__name__)
def evaluate_tool_policy(
+3 -2
View File
@@ -7,11 +7,12 @@ Thread-safe. Zero external dependencies.
from __future__ import annotations
import ipaddress
import logging
import threading
import time
log = logging.getLogger(__name__)
from turnstone.core.log import get_logger
log = get_logger(__name__)
_NetworkType = ipaddress.IPv4Network | ipaddress.IPv6Network
+213 -111
View File
@@ -90,6 +90,7 @@ log = get_logger(__name__)
if TYPE_CHECKING:
from collections.abc import Iterator
from turnstone.core.config_store import ConfigStore
from turnstone.core.healthcheck import BackendHealthMonitor
from turnstone.core.judge import IntentJudge, JudgeConfig
from turnstone.core.mcp_client import MCPClientManager
@@ -100,6 +101,7 @@ if TYPE_CHECKING:
ModelCapabilities,
StreamChunk,
)
from turnstone.core.web_search import WebSearchClient
# ---------------------------------------------------------------------------
# Cancellation support
@@ -247,6 +249,8 @@ class ChatSession:
judge_config: JudgeConfig | None = None,
user_id: str = "",
memory_config: MemoryConfig | None = None,
config_store: ConfigStore | None = None,
web_search_backend: str = "",
):
self.client = client
self.model = model
@@ -259,6 +263,7 @@ class ChatSession:
if registry and model_alias
else create_provider("openai")
)
self._cached_capabilities: ModelCapabilities | None = None
self.ui = ui
self.instructions = instructions
self.temperature = temperature
@@ -281,6 +286,7 @@ class ChatSession:
self.auto_approve = False
self._node_id = node_id
self._user_id = user_id
self._config_store = config_store
self._memory_config = memory_config or MemoryConfig()
self._ws_id = ws_id or uuid.uuid4().hex
self._title_generated = False
@@ -302,7 +308,7 @@ class ChatSession:
self._notify_count = 0
# Watch support: server-level runner injected via set_watch_runner()
self._watch_runner: Any = None # WatchRunner | None
self._watch_pending: queue.Queue[dict[str, Any]] = queue.Queue()
self._watch_pending: queue.Queue[dict[str, Any]] = queue.Queue(maxsize=20)
self._watch_dispatch_depth = 0
# Metacognitive nudges: ephemeral prompts for proactive memory use
self._metacog_state: dict[str, float] = {}
@@ -336,6 +342,8 @@ class ChatSession:
self._tools = TOOLS
self._task_tools = TASK_AGENT_TOOLS
self._agent_tools = AGENT_TOOLS
# Web search backend (pluggable: auto/tavily/ddg/mcp:server:tool)
self._web_search_backend = web_search_backend
# Dynamic tool search: defer MCP tools when tool count is high
self._tool_search_setting = tool_search
self._tool_search_threshold = tool_search_threshold
@@ -365,6 +373,70 @@ class ChatSession:
def model_alias(self) -> str | None:
return self._model_alias
@property
def _mem_cfg(self) -> MemoryConfig:
"""Live memory config — reads from ConfigStore when available."""
cs = getattr(self, "_config_store", None)
if cs is None:
return self._memory_config
return MemoryConfig(
relevance_k=cs.get("memory.relevance_k"),
fetch_limit=cs.get("memory.fetch_limit"),
max_content=cs.get("memory.max_content"),
nudge_cooldown=cs.get("memory.nudge_cooldown"),
nudges=cs.get("memory.nudges"),
)
@property
def _judge_cfg(self) -> JudgeConfig | None:
"""Live judge behavioral config — reads from ConfigStore when available.
LLM client fields (model, provider, base_url, api_key) stay frozen
from session creation time since changing them would require tearing
down and rebuilding the IntentJudge instance.
"""
jc = self._judge_config
if jc is None:
return None
cs = getattr(self, "_config_store", None)
if cs is None:
return jc
from turnstone.core.judge import JudgeConfig
return JudgeConfig(
enabled=cs.get("judge.enabled"),
model=jc.model,
provider=jc.provider,
base_url=jc.base_url,
api_key=jc.api_key,
confidence_threshold=cs.get("judge.confidence_threshold"),
max_context_ratio=cs.get("judge.max_context_ratio"),
timeout=cs.get("judge.timeout"),
read_only_tools=cs.get("judge.read_only_tools"),
output_guard=cs.get("judge.output_guard"),
redact_secrets=cs.get("judge.redact_secrets"),
)
def _get_web_search_backend(self) -> str:
"""Effective web search backend — reads from ConfigStore when available."""
cs = getattr(self, "_config_store", None)
if cs is not None:
val = cs.get("tools.web_search_backend")
if val:
return str(val)
return self._web_search_backend
def _resolve_search_client(self) -> WebSearchClient | None:
"""Return a web search client for the configured backend, or None."""
from turnstone.core.web_search import resolve_web_search_client
return resolve_web_search_client(
backend=self._get_web_search_backend(),
tavily_key=get_tavily_key(),
mcp_client=self._mcp_client,
timeout=self.tool_timeout,
)
def _resolve_capabilities(
self,
provider: LLMProvider,
@@ -382,9 +454,16 @@ class ChatSession:
caps = dataclasses.replace(caps, **overrides)
return caps
def _get_capabilities(self) -> ModelCapabilities:
"""Get capabilities for the current model."""
return self._resolve_capabilities(self._provider, self.model, self._model_alias)
def _get_capabilities(self, provider: Any = None, model: str = "") -> ModelCapabilities:
"""Get capabilities for a model. Cached for the primary session model."""
p = provider or self._provider
m = model or self.model
# Only use cache for the primary session model — fallback models bypass.
if p is self._provider and m == self.model:
if self._cached_capabilities is None:
self._cached_capabilities = self._resolve_capabilities(p, m, self._model_alias)
return self._cached_capabilities
return self._resolve_capabilities(p, m, "")
def _save_config(self) -> None:
"""Persist LLM-affecting config so resumed workstreams behave identically."""
@@ -549,7 +628,12 @@ class ChatSession:
pending = self._watch_pending
def _enqueue(msg: str) -> None:
pending.put({"message": msg})
try:
pending.put_nowait({"message": msg})
except queue.Full:
log.warning(
"Watch pending queue full, dropping result for ws_id=%s", self._ws_id
)
runner.set_dispatch_fn(self._ws_id, _enqueue)
@@ -623,6 +707,7 @@ class ChatSession:
def _generate_title(self) -> None:
"""Generate a short title for this session via a background LLM call."""
ws_id = self._ws_id # Capture before async work
try:
# Gather first user message and first assistant reply
user_msg = ""
@@ -669,11 +754,15 @@ class ChatSession:
raw = (result.content or "").strip()
# Take first line, strip quotes
title = raw.split("\n")[0].strip().strip('"').strip("'")
if title:
update_workstream_title(self._ws_id, title[:80])
if title and self._ws_id == ws_id:
update_workstream_title(ws_id, title[:80])
self.ui.on_rename(title[:80])
except Exception:
log.debug("Title generation failed for ws=%s", self._ws_id, exc_info=True)
# Only reset if ws_id hasn't changed (e.g., via /resume) to
# avoid re-enabling titling for a different workstream.
if self._ws_id == ws_id:
self._title_generated = False
log.debug("Title generation failed for ws=%s", ws_id, exc_info=True)
def resume(self, ws_id: str) -> bool:
"""Load messages from a previous workstream and resume it.
@@ -724,12 +813,12 @@ class ChatSession:
self._skill_name = None
if "notify_on_complete" in config:
self._notify_on_complete = config["notify_on_complete"]
if self._memory_config.nudges and should_nudge(
if self._mem_cfg.nudges and should_nudge(
"resume",
self._metacog_state,
message_count=len(self.messages),
memory_count=self._visible_memory_count(),
cooldown_secs=self._memory_config.nudge_cooldown,
cooldown_secs=self._mem_cfg.nudge_cooldown,
):
self._pending_nudge.append(format_nudge("resume"))
self._init_system_messages()
@@ -778,17 +867,34 @@ class ChatSession:
]
else:
dev_parts = [
"You are an expert software engineer. You solve problems "
"by reading code, making targeted edits, and running commands. "
"Always respond with tool calls, not just text.\n\n"
"TOOL PATTERNS:\n\n"
"Modify existing file → read_file then edit_file:\n"
" read_file(path='config.py') → "
"edit_file(path='config.py')\n\n"
"Create new file → write_file:\n"
" write_file(path='hello.py', content='...')\n\n"
"Modify multiple filesread_file then edit_file each:\n"
" read_file(path='a.py') → edit_file(path='a.py')"
"read_file(path='b.py') → edit_file(path='b.py')\n\n"
"Create new file → write_file (generate reasonable "
"content even if the request is vague):\n"
" write_file(path='hello.py', content='...')\n"
" write_file(path='README.md', "
"content='# Project\\nDescription.')\n\n"
"Create a file then run it → write_file then bash:\n"
" write_file(path='fib.py', content='...') → "
"bash(command='python fib.py')\n\n"
"Find something across files → search:\n"
" search(query='test_')\n\n"
"Plan, design, or think through an approach → create_plan:\n"
" create_plan(goal='refactor database from API')\n\n"
"Find and modify → search then read_file then edit_file:\n"
" search(query='MAX_RETRIES') → "
"read_file(path='found.py') → "
"edit_file(path='found.py')\n\n"
"Plan, think through, or strategize → plan_agent:\n"
" plan_agent(goal='refactor database layer "
"from monolith to service')\n"
" plan_agent(goal='restructure auth module')\n\n"
"Run a command, git, or tests → bash:\n"
" bash(command='git log -5')\n"
" bash(command='pytest')\n\n"
@@ -796,8 +902,9 @@ class ChatSession:
" web_fetch(url='https://example.com')\n\n"
"Search the web for information → web_search:\n"
" web_search(query='current population of Tokyo')\n\n"
"Look up documentation → man:\n"
" man(page='tar')",
"Look up command flags or documentation → man:\n"
" man(page='tar')\n"
" man(page='grep')",
]
# Tool search hint (client-side mode only — native mode needs no hint)
if self._tool_search:
@@ -899,10 +1006,10 @@ class ChatSession:
if self.instructions:
dev_parts.append("")
dev_parts.append(self.instructions)
visible_mems = self._get_visible_memories(limit=self._memory_config.fetch_limit)
visible_mems = self._get_visible_memories(limit=self._mem_cfg.fetch_limit)
if visible_mems:
context = extract_recent_context(self.messages)
relevant = score_memories(visible_mems, context, k=self._memory_config.relevance_k)
relevant = score_memories(visible_mems, context, k=self._mem_cfg.relevance_k)
if relevant:
dev_parts.append("")
dev_parts.append(build_memory_context(relevant))
@@ -957,7 +1064,8 @@ class ChatSession:
Without tool search: return self._tools unchanged.
Web search gating: ``web_search`` is removed when the model has
no native search support and no Tavily API key is configured.
no native search support and no search backend is available
(Tavily, DDG, or MCP see ``_resolve_search_client``).
"""
if self.creative_mode:
return None
@@ -974,7 +1082,7 @@ class ChatSession:
tools = visible + [self._tool_search.get_search_tool_definition()]
# Gate web_search: only include when a backend exists
if not caps.supports_web_search and not get_tavily_key():
if not caps.supports_web_search and not self._resolve_search_client():
tools = _without_tool(tools, "web_search")
return tools
@@ -1201,11 +1309,7 @@ class ChatSession:
_tc_names = {c["id"]: c.get("function", {}).get("name", "") for c in tool_calls}
for tc_id, output in results:
# Output guard: evaluate tool result before it enters context
if (
self._judge_config
and self._judge_config.enabled
and self._judge_config.output_guard
):
if self._judge_cfg and self._judge_cfg.output_guard:
if isinstance(output, str):
output = self._evaluate_output(tc_id, output, _tc_names.get(tc_id, ""))
elif isinstance(output, list):
@@ -1264,7 +1368,7 @@ class ChatSession:
)
# Metacognitive nudge: check memories on tool error
if (
self._memory_config.nudges
self._mem_cfg.nudges
and any(
isinstance(out, str)
and (
@@ -1280,7 +1384,7 @@ class ChatSession:
self._metacog_state,
message_count=len(self.messages),
memory_count=self._visible_memory_count(),
cooldown_secs=self._memory_config.nudge_cooldown,
cooldown_secs=self._mem_cfg.nudge_cooldown,
)
):
self._pending_nudge.append(format_nudge("tool_error"))
@@ -1863,7 +1967,6 @@ class ChatSession:
self.ui.on_thinking_start()
try:
_last_err: Exception | None = None
result: CompletionResult | None = None
for attempt in range(self._MAX_RETRIES + 1):
try:
@@ -1884,7 +1987,6 @@ class ChatSession:
or attempt == self._MAX_RETRIES
):
raise
_last_err = e
delay = self._RETRY_BASE_DELAY * (2**attempt)
self.ui.on_info(f"[Compact retrying in {delay:.0f}s: {ename}]")
time.sleep(delay)
@@ -1936,10 +2038,20 @@ class ChatSession:
# -- Intent validation --------------------------------------------------------
def _ensure_judge(self) -> IntentJudge | None:
"""Lazily initialize the intent judge if configured."""
"""Lazily initialize the intent judge if configured.
Re-checks the live ``enabled`` flag every call so disabling the
judge via admin settings takes immediate effect on existing sessions.
"""
if not self._judge_cfg or not self._judge_cfg.enabled:
return None
if self._judge is not None:
return self._judge
if not self._judge_config or not self._judge_config.enabled:
return None
# Frozen config required for IntentJudge init (LLM client fields).
# _judge_cfg already returns None when _judge_config is None, but
# this guard makes the dependency explicit for type narrowing.
if self._judge_config is None:
return None
try:
from turnstone.core.judge import IntentJudge
@@ -1996,8 +2108,10 @@ class ChatSession:
}
elif name == "notify":
it["func_args"] = {"message": it.get("message", "")[:200]}
elif name == "task":
elif name == "task_agent":
it["func_args"] = {"prompt": it.get("prompt", "")[:200]}
elif name == "plan_agent":
it["func_args"] = {"goal": it.get("prompt", "")[:200]}
elif it.get("mcp_args"):
it["func_args"] = it["mcp_args"]
@@ -2046,11 +2160,7 @@ class ChatSession:
except Exception:
log.debug("output_guard.callback_failed", exc_info=True)
if (
assessment.sanitized is not None
and self._judge_config
and self._judge_config.redact_secrets
):
if assessment.sanitized is not None and self._judge_cfg and self._judge_cfg.redact_secrets:
return assessment.sanitized
return output
@@ -2087,12 +2197,12 @@ class ChatSession:
f"Denied by user: {user_feedback}" if user_feedback else "Denied by user"
)
user_feedback = None # feedback is in the denial_msg
if self._memory_config.nudges and should_nudge(
if self._mem_cfg.nudges and should_nudge(
"denial",
self._metacog_state,
message_count=len(self.messages),
memory_count=self._visible_memory_count(),
cooldown_secs=self._memory_config.nudge_cooldown,
cooldown_secs=self._mem_cfg.nudge_cooldown,
):
self._pending_nudge.append(format_nudge("denial"))
self._init_system_messages()
@@ -2129,7 +2239,7 @@ class ChatSession:
# feedback the plan agent re-runs and the revised plan is shown
# again, up to _MAX_PLAN_REFINEMENTS rounds.
for i, item in enumerate(items):
if item.get("func_name") != "create_plan" or item.get("error") or item.get("denied"):
if item.get("func_name") != "plan_agent" or item.get("error") or item.get("denied"):
continue
cid, output = results[i]
@@ -2254,8 +2364,8 @@ class ChatSession:
"web_fetch": self._prepare_web_fetch,
"web_search": self._prepare_web_search,
"tool_search": self._prepare_tool_search,
"task": self._prepare_task,
"create_plan": self._prepare_plan,
"task_agent": self._prepare_task,
"plan_agent": self._prepare_plan,
"memory": self._prepare_memory,
"recall": self._prepare_recall,
"notify": self._prepare_notify,
@@ -2704,17 +2814,17 @@ class ChatSession:
"needs_approval": False,
"error": "Error: no query provided",
}
if not get_tavily_key():
if not self._resolve_search_client():
return {
"call_id": call_id,
"func_name": "web_search",
"header": "\u2717 web_search: no API key",
"header": "\u2717 web_search: no backend available",
"preview": "",
"needs_approval": False,
"error": (
"Error: Tavily API key not configured. "
"Set it in ~/.config/turnstone/tavily_key or $TAVILY_API_KEY. "
"Use web_fetch with a direct URL as an alternative."
"Error: No web search backend available. "
"Install the ddg extra (`pip install turnstone[ddg]`), "
"configure a Tavily API key, or set tools.web_search_backend."
),
}
try:
@@ -2787,8 +2897,8 @@ class ChatSession:
if not prompt:
return {
"call_id": call_id,
"func_name": "task",
"header": "\u2717 task: empty prompt",
"func_name": "task_agent",
"header": "\u2717 task_agent: empty prompt",
"preview": "",
"needs_approval": False,
"error": "Error: empty prompt",
@@ -2796,11 +2906,11 @@ class ChatSession:
preview_text = prompt[:300] + ("..." if len(prompt) > 300 else "")
return {
"call_id": call_id,
"func_name": "task",
"header": "\u2699 task (autonomous agent)",
"func_name": "task_agent",
"header": "\u2699 task_agent (autonomous agent)",
"preview": f" {DIM}{preview_text}{RESET}",
"needs_approval": True,
"approval_label": "task",
"approval_label": "task_agent",
"execute": self._exec_task,
"prompt": prompt,
}
@@ -2811,8 +2921,8 @@ class ChatSession:
if not goal:
return {
"call_id": call_id,
"func_name": "create_plan",
"header": "\u2717 create_plan: empty goal",
"func_name": "plan_agent",
"header": "\u2717 plan_agent: empty goal",
"preview": "",
"needs_approval": False,
"error": "Error: empty goal",
@@ -2820,11 +2930,11 @@ class ChatSession:
preview_text = goal[:300] + ("..." if len(goal) > 300 else "")
return {
"call_id": call_id,
"func_name": "create_plan",
"header": "\u2699 create_plan (planning agent)",
"func_name": "plan_agent",
"header": "\u2699 plan_agent (planning agent)",
"preview": f" {DIM}{preview_text}{RESET}",
"needs_approval": True,
"approval_label": "create_plan",
"approval_label": "plan_agent",
"execute": self._exec_plan,
"prompt": goal,
}
@@ -2871,11 +2981,11 @@ class ChatSession:
def _check_metacognitive_nudge(self, user_message: str) -> str | None:
"""Check if a metacognitive nudge should be injected."""
if not self._memory_config.nudges:
if not self._mem_cfg.nudges:
return None
mem_count = self._visible_memory_count()
msg_count = len(self.messages)
cd = self._memory_config.nudge_cooldown
cd = self._mem_cfg.nudge_cooldown
if should_nudge(
"start",
@@ -2923,14 +3033,14 @@ class ChatSession:
"needs_approval": False,
"error": "Error: both 'name' and 'content' are required for save",
}
if len(content) > self._memory_config.max_content:
if len(content) > self._mem_cfg.max_content:
return {
"call_id": call_id,
"func_name": "memory",
"header": "\u2717 memory save: content too large",
"preview": "",
"needs_approval": False,
"error": f"Error: content exceeds {self._memory_config.max_content} character limit",
"error": f"Error: content exceeds {self._mem_cfg.max_content} character limit",
}
description = (args.get("description") or "").strip()
mem_type = (args.get("type") or "project").strip().lower()
@@ -3458,12 +3568,15 @@ class ChatSession:
f.write(command)
script_path = f.name
try:
from turnstone.core.env import scrubbed_env
proc = subprocess.Popen(
["bash", script_path],
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
text=True,
start_new_session=True,
env=scrubbed_env(),
)
# Drain stderr in background thread to avoid pipe deadlock
stderr_lines: list[str] = []
@@ -3497,8 +3610,10 @@ class ChatSession:
assert proc.stdout is not None
for line in proc.stdout:
stdout_parts.append(line)
with contextlib.suppress(Exception):
try:
self.ui.on_tool_output_chunk(call_id, line)
except Exception:
log.debug("UI callback error during tool output", exc_info=True)
# Check cancellation during long-running commands
if self._cancel_event.is_set():
with contextlib.suppress(OSError, ProcessLookupError):
@@ -3511,8 +3626,11 @@ class ChatSession:
finally:
timer.cancel()
proc.wait()
stderr_thread.join()
try:
proc.wait(timeout=10)
except subprocess.TimeoutExpired:
log.warning("Process did not exit after SIGKILL, pid=%d", proc.pid)
stderr_thread.join(timeout=5)
finally:
os.unlink(script_path)
@@ -3647,6 +3765,8 @@ class ChatSession:
call_id = item["call_id"]
pattern, path = item["pattern"], item["path"]
try:
from turnstone.core.env import scrubbed_env
result = subprocess.run(
[
"grep",
@@ -3663,6 +3783,7 @@ class ChatSession:
capture_output=True,
text=True,
timeout=self.tool_timeout,
env=scrubbed_env(),
)
output = result.stdout.strip()
if result.returncode == 1:
@@ -3692,10 +3813,6 @@ class ChatSession:
self.ui.on_error(msg)
return call_id, msg
# Tools the agent can auto-execute without user approval (read-only).
_AGENT_AUTO_TOOLS = AGENT_AUTO_TOOLS
_TASK_AUTO_TOOLS = TASK_AUTO_TOOLS
def _run_agent(
self,
agent_messages: list[dict[str, Any]],
@@ -3710,7 +3827,7 @@ class ChatSession:
agent_messages: Pre-built message list (system + developer + user).
label: Display prefix for progress lines ("agent" or "plan").
tools: Tool definitions to send to the API. Defaults to AGENT_TOOLS (read-only).
auto_tools: Set of tool names the agent may execute. Defaults to _AGENT_AUTO_TOOLS.
auto_tools: Set of tool names the agent may execute. Defaults to AGENT_AUTO_TOOLS.
reasoning_effort: Override reasoning effort for this agent.
Returns:
@@ -3719,7 +3836,7 @@ class ChatSession:
if tools is None:
tools = self._agent_tools
if auto_tools is None:
auto_tools = self._AGENT_AUTO_TOOLS
auto_tools = AGENT_AUTO_TOOLS
max_tool_turns = self.agent_max_turns
# Resolve agent model and provider: use registry.agent_model if configured
@@ -3733,7 +3850,7 @@ class ChatSession:
# Gate web_search: remove when no backend exists for the agent model
agent_alias = self._registry.agent_model if self._registry else None
agent_caps = self._resolve_capabilities(agent_provider, agent_model, agent_alias)
if not agent_caps.supports_web_search and not get_tavily_key():
if not agent_caps.supports_web_search and not self._resolve_search_client():
tools = _without_tool(tools, "web_search")
# Build extra params for agent calls
@@ -3823,7 +3940,7 @@ class ChatSession:
tool_name = tc_dict["function"]["name"]
# Guard 1: block recursive agent calls.
if tool_name in ("task", "create_plan"):
if tool_name in ("task_agent", "plan_agent"):
output = "Error: agents cannot spawn further agents"
# Guard 2: tool not in this agent's API tool list.
elif tool_name not in tool_names:
@@ -3856,6 +3973,12 @@ class ChatSession:
else:
output = f"Unknown tool: {tool_name}"
# Output guard: evaluate before truncation so the guard
# sees full output (credentials split by truncation would
# evade detection). Agent outputs are always str.
if self._judge_cfg and self._judge_cfg.output_guard and isinstance(output, str):
output = self._evaluate_output(tc_dict["id"], output, tool_name)
# Truncate large tool outputs to avoid blowing context limits.
# Agents operate autonomously; they can refine their queries
# if truncation loses important detail.
@@ -3924,7 +4047,7 @@ class ChatSession:
agent_messages,
label="task",
tools=self._task_tools,
auto_tools=self._TASK_AUTO_TOOLS,
auto_tools=TASK_AUTO_TOOLS,
)
except (KeyboardInterrupt, GenerationCancelled):
return call_id, "(task interrupted by user)"
@@ -4020,7 +4143,7 @@ class ChatSession:
for i, msg in enumerate(self.messages):
if msg.get("role") == "assistant" and msg.get("tool_calls"):
for tc in msg["tool_calls"]:
if tc.get("function", {}).get("name") == "create_plan":
if tc.get("function", {}).get("name") == "plan_agent":
tc_id = tc["id"]
for j in range(i + 1, len(self.messages)):
if (
@@ -4114,7 +4237,7 @@ class ChatSession:
"id": tc_id,
"type": "function",
"function": {
"name": "create_plan",
"name": "plan_agent",
"arguments": json.dumps({"goal": original_goal}),
},
}
@@ -4817,12 +4940,14 @@ class ChatSession:
text = ""
try:
from turnstone.core.env import scrubbed_env
result = subprocess.run(
cmd,
capture_output=True,
text=True,
timeout=10,
env={**os.environ, "MANWIDTH": "80", "MAN_KEEP_FORMATTING": "0"},
env=scrubbed_env(extra={"MANWIDTH": "80", "MAN_KEEP_FORMATTING": "0"}),
)
if result.returncode == 0 and result.stdout.strip():
# Strip formatting: backspace overstrikes and ANSI escapes
@@ -4835,6 +4960,7 @@ class ChatSession:
capture_output=True,
text=True,
timeout=10,
env=scrubbed_env(),
)
if result.returncode == 0 and result.stdout.strip():
text = result.stdout
@@ -4941,51 +5067,26 @@ class ChatSession:
return call_id, answer
def _exec_web_search(self, item: dict[str, Any]) -> tuple[str, str]:
"""Search the web via Tavily API."""
"""Search the web via the configured backend (Tavily, DDG, or MCP)."""
call_id = item["call_id"]
query = item["query"]
max_results = item.get("max_results", 5)
topic = item.get("topic", "general")
api_key = get_tavily_key()
try:
resp = httpx.post(
"https://api.tavily.com/search",
json={
"query": query,
"max_results": max_results,
"topic": topic,
"include_answer": True,
},
headers={"Authorization": f"Bearer {api_key}"},
timeout=self.tool_timeout,
)
resp.raise_for_status()
data = resp.json()
except Exception as e:
msg = f"Tavily search failed: {e}"
client = self._resolve_search_client()
if not client:
msg = "Web search backend not available"
self.ui.on_error(msg)
return call_id, msg
parts: list[str] = []
answer = (data.get("answer") or "").strip()
if answer:
parts.append(f"Answer: {answer}")
results = data.get("results") or []
if results:
lines = []
for i, r in enumerate(results, 1):
title = r.get("title", "")
url = r.get("url", "")
content = (r.get("content") or "")[:500]
lines.append(f"{i}. [{title}]({url})\n {content}")
parts.append("\n".join(lines))
output = "\n\n".join(parts) if parts else f"No results for '{query}'."
try:
output = client.search(query, max_results=max_results, topic=topic)
except Exception as e:
msg = f"Web search failed: {e}"
self.ui.on_error(msg)
return call_id, msg
self.ui.on_tool_result(call_id, "web_search", output)
return call_id, output
def handle_command(self, cmd_line: str) -> bool:
@@ -5153,6 +5254,7 @@ class ChatSession:
self.model = model_name
self._model_alias = arg
self._provider = self._registry.get_provider(arg)
self._cached_capabilities = None
self.context_window = cfg.context_window
if not self._manual_tool_truncation:
self.tool_truncation = int(cfg.context_window * self._chars_per_token * 0.5)
+42 -2
View File
@@ -196,6 +196,17 @@ def _build_registry() -> dict[str, SettingDef]:
min_value=1,
max_value=50,
),
SettingDef(
"tools.web_search_backend",
"str",
"",
"Web search backend: '' (auto), 'tavily', 'ddg', or 'mcp:server:tool'",
"tools",
help="Controls which service handles web_search calls when the model lacks native "
"search support. Empty string auto-detects (Tavily if key present, else DuckDuckGo "
"if installed). 'ddg' uses DuckDuckGo (free, no API key). 'tavily' forces Tavily. "
"'mcp:server:tool' routes to an MCP server (e.g. 'mcp:ddg:search').",
),
# -- server ---------------------------------------------------------
SettingDef(
"server.workstream_idle_timeout",
@@ -212,13 +223,42 @@ def _build_registry() -> dict[str, SettingDef]:
SettingDef(
"server.max_workstreams",
"int",
10,
50,
"Max concurrent workstreams",
"server",
min_value=1,
restart_required=True,
help="Maximum number of active conversation threads on this server node. "
"When the limit is reached, the oldest idle workstream is evicted to make room.",
"When the limit is reached, the oldest idle workstream is evicted to make room. "
"Each workstream uses memory proportional to its conversation history.",
),
# -- cluster --------------------------------------------------------
SettingDef(
"cluster.node_fan_out_limit",
"int",
200,
"Max concurrent outbound requests during cluster-wide operations",
"cluster",
min_value=10,
max_value=1000,
restart_required=True,
help="Controls how many nodes the console queries in parallel during "
"fan-out operations (watch listing, MCP status, reload notifications). "
"Higher values speed up large-cluster admin operations at the cost of "
"more concurrent connections. The httpx proxy pool is sized to match "
"this value (requires console restart to take effect).",
),
SettingDef(
"cluster.mcp_max_servers",
"int",
200,
"Max MCP server definitions in the cluster",
"cluster",
min_value=1,
max_value=2000,
help="Hard cap on the total number of MCP server definitions stored in the "
"database. Each node only connects to the servers it needs, so this "
"limit is on definitions, not active connections.",
),
# -- mcp ------------------------------------------------------------
SettingDef(
+5 -5
View File
@@ -7,7 +7,6 @@ and :func:`fetch_skill_from_github` for fetching SKILL.md from GitHub repos.
from __future__ import annotations
import asyncio
import logging
import os
import re
from dataclasses import dataclass, field
@@ -16,9 +15,10 @@ from urllib.parse import quote
import httpx
from turnstone.core.log import get_logger
from turnstone.core.skill_parser import ParsedSkill, parse_skill_md
logger = logging.getLogger(__name__)
log = get_logger(__name__)
DEFAULT_DISCOVERY_URL = "https://skills.sh"
@@ -176,7 +176,7 @@ def _check_rate_limit(resp: httpx.Response) -> None:
)
remaining = resp.headers.get("x-ratelimit-remaining", "")
if remaining and remaining.isdigit() and int(remaining) < 10:
logger.warning("GitHub API rate limit low: %s remaining", remaining)
log.warning("GitHub API rate limit low: %s remaining", remaining)
_FETCH_CONCURRENCY = 5
@@ -301,7 +301,7 @@ async def fetch_skill_from_github(url: str) -> SkillPackage:
rf = _find_resource_files(tree_data.get("tree", []), skill_md_dir)
resources = await _fetch_resource_contents(client, raw_base, rf)
except httpx.HTTPError:
logger.debug("Failed to fetch resource tree for %s/%s", owner, repo)
log.debug("Failed to fetch resource tree for %s/%s", owner, repo)
# Build a per-skill source URL pointing to the specific subdirectory
if skill_md_dir:
@@ -418,7 +418,7 @@ async def fetch_skills_from_github_repo(url: str) -> list[SkillPackage]:
try:
parsed = parse_skill_md(content)
except ValueError:
logger.debug("Skipping invalid SKILL.md at %s", skill_md_path)
log.debug("Skipping invalid SKILL.md at %s", skill_md_path)
continue
# Collect resources for this skill (concurrent via helper)
+42 -9
View File
@@ -2,11 +2,12 @@
from __future__ import annotations
import logging
from pathlib import Path
from typing import Any
log = logging.getLogger(__name__)
from turnstone.core.log import get_logger
log = get_logger(__name__)
_MIGRATIONS_DIR = str(Path(__file__).parent / "migrations")
@@ -35,7 +36,14 @@ def run_migrations(storage: Any, backend: str) -> None:
_bootstrap_existing_sqlite(engine, cfg)
if backend == "postgresql":
_run_with_pg_lock(engine, cfg)
try:
_run_with_pg_lock(engine, cfg)
except (OSError, EOFError) as exc:
# Non-fatal for connection-class errors only (refused, reset,
# timeout). DDL / migration errors still propagate. The Docker
# entrypoint already runs migrations before the server starts, so
# this second attempt is a safety net for stampede scenarios.
log.warning("PostgreSQL migration failed (non-fatal): %s", exc)
else:
try:
command.upgrade(cfg, "head")
@@ -49,17 +57,42 @@ def _run_with_pg_lock(engine: Any, cfg: Any) -> None:
Advisory lock ID 7_475_283 (arbitrary, derived from 'turnstone').
``pg_advisory_lock`` blocks until the lock is available, so
concurrent containers wait in line rather than racing.
Retries with jittered backoff if PostgreSQL is temporarily at
max_connections (common during large-cluster startup stampedes).
"""
import random
import time
import sqlalchemy as sa
from alembic import command
with engine.connect() as conn:
conn.execute(sa.text("SELECT pg_advisory_lock(7475283)"))
max_retries = 10
for attempt in range(max_retries):
try:
command.upgrade(cfg, "head")
finally:
conn.execute(sa.text("SELECT pg_advisory_unlock(7475283)"))
conn.commit()
with engine.connect() as conn:
conn.execute(sa.text("SELECT pg_advisory_lock(7475283)"))
try:
command.upgrade(cfg, "head")
finally:
conn.execute(sa.text("SELECT pg_advisory_unlock(7475283)"))
conn.commit()
return
except Exception as exc:
err_str = str(exc).lower()
if "too many clients" not in err_str and "connection" not in err_str:
raise
if attempt == max_retries - 1:
raise
delay = min(2**attempt + random.uniform(0, 1), 30) # noqa: S311
log.warning(
"PG connection failed (attempt %d/%d), retrying in %.1fs: %s",
attempt + 1,
max_retries,
delay,
exc,
)
time.sleep(delay)
def _bootstrap_existing_sqlite(engine: Any, cfg: Any) -> None:
+51 -64
View File
@@ -2,23 +2,30 @@
from __future__ import annotations
import logging
from datetime import UTC, datetime, timedelta
from typing import Any
import sqlalchemy as sa
from turnstone.core.log import get_logger
from turnstone.core.storage._schema import (
api_tokens,
audit_events,
channel_routes,
channel_users,
conversations,
intent_verdicts,
mcp_servers,
metadata,
oidc_identities,
oidc_pending_states,
orgs,
output_assessments,
prompt_templates,
roles,
scheduled_task_runs,
scheduled_tasks,
services,
skill_resources,
skill_versions,
structured_memories,
@@ -27,6 +34,7 @@ from turnstone.core.storage._schema import (
usage_events,
user_roles,
users,
watches,
workstream_config,
workstreams,
)
@@ -61,7 +69,7 @@ from turnstone.core.storage._utils import (
scan_skill_content as _scan_skill_content,
)
log = logging.getLogger(__name__)
log = get_logger(__name__)
def _escape_ilike(s: str) -> str:
@@ -73,7 +81,7 @@ class PostgreSQLBackend:
"""PostgreSQL implementation of the StorageBackend protocol."""
def __init__(
self, url: str, pool_size: int = 5, max_overflow: int = 10, *, create_tables: bool = True
self, url: str, pool_size: int = 2, max_overflow: int = 3, *, create_tables: bool = True
) -> None:
self._engine = sa.create_engine(
url,
@@ -229,19 +237,17 @@ class PostgreSQLBackend:
# -- Workstream config -----------------------------------------------------
def save_workstream_config(self, ws_id: str, config: dict[str, str]) -> None:
if not config:
return
with self._engine.connect() as conn:
for key, value in config.items():
# Upsert: delete + insert
conn.execute(
sa.delete(workstream_config).where(
workstream_config.c.ws_id == ws_id,
workstream_config.c.key == key,
)
)
conn.execute(
sa.insert(workstream_config),
{"ws_id": ws_id, "key": key, "value": value},
)
conn.execute(
sa.text(
"INSERT INTO workstream_config (ws_id, key, value) "
"VALUES (:ws_id, :key, :value) "
"ON CONFLICT (ws_id, key) DO UPDATE SET value = EXCLUDED.value"
),
[{"ws_id": ws_id, "key": key, "value": value} for key, value in config.items()],
)
conn.commit()
def load_workstream_config(self, ws_id: str) -> dict[str, str]:
@@ -524,7 +530,6 @@ class PostgreSQLBackend:
]
def delete_user(self, user_id: str) -> bool:
from turnstone.core.storage._schema import channel_users, oidc_identities
with self._engine.connect() as conn:
conn.execute(sa.delete(user_roles).where(user_roles.c.user_id == user_id))
@@ -630,8 +635,6 @@ class PostgreSQLBackend:
def create_channel_user(self, channel_type: str, channel_user_id: str, user_id: str) -> None:
from sqlalchemy.dialects import postgresql
from turnstone.core.storage._schema import channel_users
now = datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%S")
with self._engine.connect() as conn:
conn.execute(
@@ -647,7 +650,6 @@ class PostgreSQLBackend:
conn.commit()
def get_channel_user(self, channel_type: str, channel_user_id: str) -> dict[str, str] | None:
from turnstone.core.storage._schema import channel_users
with self._engine.connect() as conn:
row = conn.execute(
@@ -671,7 +673,6 @@ class PostgreSQLBackend:
return None
def list_channel_users_by_user(self, user_id: str) -> list[dict[str, str]]:
from turnstone.core.storage._schema import channel_users
with self._engine.connect() as conn:
rows = conn.execute(
@@ -695,7 +696,6 @@ class PostgreSQLBackend:
]
def delete_channel_user(self, channel_type: str, channel_user_id: str) -> bool:
from turnstone.core.storage._schema import channel_users
with self._engine.connect() as conn:
result = conn.execute(
@@ -714,8 +714,6 @@ class PostgreSQLBackend:
) -> None:
from sqlalchemy.dialects import postgresql
from turnstone.core.storage._schema import channel_routes
now = datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%S")
with self._engine.connect() as conn:
conn.execute(
@@ -732,7 +730,6 @@ class PostgreSQLBackend:
conn.commit()
def get_channel_route(self, channel_type: str, channel_id: str) -> dict[str, str] | None:
from turnstone.core.storage._schema import channel_routes
with self._engine.connect() as conn:
row = conn.execute(
@@ -758,7 +755,6 @@ class PostgreSQLBackend:
return None
def get_channel_route_by_ws(self, ws_id: str) -> dict[str, str] | None:
from turnstone.core.storage._schema import channel_routes
with self._engine.connect() as conn:
row = conn.execute(
@@ -781,7 +777,6 @@ class PostgreSQLBackend:
return None
def list_channel_routes_by_type(self, channel_type: str) -> list[dict[str, str]]:
from turnstone.core.storage._schema import channel_routes
with self._engine.connect() as conn:
rows = conn.execute(
@@ -807,7 +802,6 @@ class PostgreSQLBackend:
]
def delete_channel_route(self, channel_type: str, channel_id: str) -> bool:
from turnstone.core.storage._schema import channel_routes
with self._engine.connect() as conn:
result = conn.execute(
@@ -840,8 +834,6 @@ class PostgreSQLBackend:
) -> None:
from sqlalchemy.dialects import postgresql
from turnstone.core.storage._schema import scheduled_tasks
now = datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%S")
with self._engine.connect() as conn:
conn.execute(
@@ -870,7 +862,6 @@ class PostgreSQLBackend:
conn.commit()
def get_scheduled_task(self, task_id: str) -> dict[str, Any] | None:
from turnstone.core.storage._schema import scheduled_tasks
with self._engine.connect() as conn:
row = conn.execute(
@@ -881,7 +872,6 @@ class PostgreSQLBackend:
return dict(row._mapping)
def list_scheduled_tasks(self) -> list[dict[str, Any]]:
from turnstone.core.storage._schema import scheduled_tasks
with self._engine.connect() as conn:
rows = conn.execute(
@@ -910,7 +900,6 @@ class PostgreSQLBackend:
)
def update_scheduled_task(self, task_id: str, **fields: Any) -> bool:
from turnstone.core.storage._schema import scheduled_tasks
fields = {k: v for k, v in fields.items() if k in self._UPDATABLE_TASK_FIELDS}
fields["updated"] = datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%S")
@@ -930,7 +919,6 @@ class PostgreSQLBackend:
return result.rowcount > 0
def delete_scheduled_task(self, task_id: str) -> bool:
from turnstone.core.storage._schema import scheduled_task_runs, scheduled_tasks
with self._engine.connect() as conn:
conn.execute(
@@ -943,7 +931,6 @@ class PostgreSQLBackend:
return result.rowcount > 0
def list_due_tasks(self, now: str) -> list[dict[str, Any]]:
from turnstone.core.storage._schema import scheduled_tasks
with self._engine.connect() as conn:
rows = conn.execute(
@@ -969,7 +956,6 @@ class PostgreSQLBackend:
status: str,
error: str,
) -> None:
from turnstone.core.storage._schema import scheduled_task_runs
with self._engine.connect() as conn:
conn.execute(
@@ -988,7 +974,6 @@ class PostgreSQLBackend:
conn.commit()
def list_task_runs(self, task_id: str, limit: int = 50) -> list[dict[str, Any]]:
from turnstone.core.storage._schema import scheduled_task_runs
with self._engine.connect() as conn:
rows = conn.execute(
@@ -1000,10 +985,6 @@ class PostgreSQLBackend:
return [dict(r._mapping) for r in rows]
def prune_task_runs(self, retention_days: int = 90) -> int:
from datetime import timedelta
from turnstone.core.storage._schema import scheduled_task_runs
cutoff = (datetime.now(UTC) - timedelta(days=retention_days)).strftime("%Y-%m-%dT%H:%M:%S")
with self._engine.connect() as conn:
result = conn.execute(
@@ -1029,8 +1010,6 @@ class PostgreSQLBackend:
) -> None:
from sqlalchemy.dialects import postgresql
from turnstone.core.storage._schema import watches
now = datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%S")
with self._engine.connect() as conn:
conn.execute(
@@ -1056,7 +1035,6 @@ class PostgreSQLBackend:
conn.commit()
def get_watch(self, watch_id: str) -> dict[str, Any] | None:
from turnstone.core.storage._schema import watches
with self._engine.connect() as conn:
row = conn.execute(sa.select(watches).where(watches.c.watch_id == watch_id)).fetchone()
@@ -1065,7 +1043,6 @@ class PostgreSQLBackend:
return dict(row._mapping)
def list_watches_for_ws(self, ws_id: str) -> list[dict[str, Any]]:
from turnstone.core.storage._schema import watches
with self._engine.connect() as conn:
rows = conn.execute(
@@ -1076,7 +1053,6 @@ class PostgreSQLBackend:
return [dict(r._mapping) for r in rows]
def list_watches_for_node(self, node_id: str) -> list[dict[str, Any]]:
from turnstone.core.storage._schema import watches
with self._engine.connect() as conn:
rows = conn.execute(
@@ -1087,7 +1063,6 @@ class PostgreSQLBackend:
return [dict(r._mapping) for r in rows]
def list_due_watches(self, now: str) -> list[dict[str, Any]]:
from turnstone.core.storage._schema import watches
with self._engine.connect() as conn:
rows = conn.execute(
@@ -1116,7 +1091,6 @@ class PostgreSQLBackend:
)
def update_watch(self, watch_id: str, **fields: Any) -> bool:
from turnstone.core.storage._schema import watches
fields = {k: v for k, v in fields.items() if k in self._UPDATABLE_WATCH_FIELDS}
fields["updated"] = datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%S")
@@ -1130,7 +1104,6 @@ class PostgreSQLBackend:
return result.rowcount > 0
def delete_watch(self, watch_id: str) -> bool:
from turnstone.core.storage._schema import watches
with self._engine.connect() as conn:
result = conn.execute(sa.delete(watches).where(watches.c.watch_id == watch_id))
@@ -1138,7 +1111,6 @@ class PostgreSQLBackend:
return result.rowcount > 0
def delete_watches_for_ws(self, ws_id: str) -> int:
from turnstone.core.storage._schema import watches
with self._engine.connect() as conn:
result = conn.execute(sa.delete(watches).where(watches.c.ws_id == ws_id))
@@ -1150,7 +1122,6 @@ class PostgreSQLBackend:
def register_service(
self, service_type: str, service_id: str, url: str, metadata: str = "{}"
) -> None:
from turnstone.core.storage._schema import services
now = datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%S")
with self._engine.connect() as conn:
@@ -1172,7 +1143,6 @@ class PostgreSQLBackend:
conn.commit()
def heartbeat_service(self, service_type: str, service_id: str) -> bool:
from turnstone.core.storage._schema import services
now = datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%S")
with self._engine.connect() as conn:
@@ -1188,7 +1158,6 @@ class PostgreSQLBackend:
return result.rowcount > 0
def list_services(self, service_type: str, max_age_seconds: int = 120) -> list[dict[str, str]]:
from turnstone.core.storage._schema import services
cutoff = (datetime.now(UTC) - timedelta(seconds=max_age_seconds)).strftime(
"%Y-%m-%dT%H:%M:%S"
@@ -1205,7 +1174,6 @@ class PostgreSQLBackend:
return [dict(r._mapping) for r in rows]
def deregister_service(self, service_type: str, service_id: str) -> bool:
from turnstone.core.storage._schema import services
with self._engine.connect() as conn:
result = conn.execute(
@@ -1510,6 +1478,7 @@ class PostgreSQLBackend:
allowed_tools: str = "[]",
skill_license: str = "",
compatibility: str = "",
priority: int = 0,
) -> None:
# Sync is_default from activation when activation is explicitly set
if activation == "default":
@@ -1556,6 +1525,7 @@ class PostgreSQLBackend:
"agent_max_turns": agent_max_turns,
"notify_on_complete": notify_on_complete,
"enabled": 1 if enabled else 0,
"priority": priority,
"created": now,
"updated": now,
},
@@ -1609,7 +1579,7 @@ class PostgreSQLBackend:
sa.select(prompt_templates)
.where(prompt_templates.c.is_default == 1)
.where(prompt_templates.c.enabled == 1)
.order_by(prompt_templates.c.name)
.order_by(prompt_templates.c.priority, prompt_templates.c.name)
)
if org_id:
q = q.where(prompt_templates.c.org_id == org_id)
@@ -1694,7 +1664,7 @@ class PostgreSQLBackend:
q = (
sa.select(prompt_templates)
.where(prompt_templates.c.activation == activation)
.order_by(prompt_templates.c.name)
.order_by(prompt_templates.c.priority, prompt_templates.c.name)
)
if enabled_only:
q = q.where(prompt_templates.c.enabled == 1)
@@ -2426,6 +2396,32 @@ class PostgreSQLBackend:
).fetchall()
return [dict(r._mapping) for r in rows]
def touch_structured_memories(self, keys: list[tuple[str, str, str]]) -> int:
"""Batch-touch multiple memories by (name, scope, scope_id)."""
if not keys:
return 0
now = datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%S")
total = 0
with self._engine.connect() as conn:
for name, scope, scope_id in keys:
result = conn.execute(
sa.update(structured_memories)
.where(
sa.and_(
structured_memories.c.name == name,
structured_memories.c.scope == scope,
structured_memories.c.scope_id == scope_id,
)
)
.values(
last_accessed=now,
access_count=structured_memories.c.access_count + 1,
)
)
total += result.rowcount
conn.commit()
return total
def count_structured_memories(
self, mem_type: str = "", scope: str = "", scope_id: str = ""
) -> int:
@@ -2650,8 +2646,6 @@ class PostgreSQLBackend:
def create_oidc_identity(self, issuer: str, subject: str, user_id: str, email: str) -> None:
from sqlalchemy.dialects import postgresql
from turnstone.core.storage._schema import oidc_identities
now = datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%S")
with self._engine.connect() as conn:
conn.execute(
@@ -2669,7 +2663,6 @@ class PostgreSQLBackend:
conn.commit()
def get_oidc_identity(self, issuer: str, subject: str) -> dict[str, str] | None:
from turnstone.core.storage._schema import oidc_identities
with self._engine.connect() as conn:
row = conn.execute(
@@ -2696,7 +2689,6 @@ class PostgreSQLBackend:
return None
def update_oidc_identity_login(self, issuer: str, subject: str) -> bool:
from turnstone.core.storage._schema import oidc_identities
now = datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%S")
with self._engine.connect() as conn:
@@ -2711,7 +2703,6 @@ class PostgreSQLBackend:
return result.rowcount > 0
def list_oidc_identities_for_user(self, user_id: str) -> list[dict[str, str]]:
from turnstone.core.storage._schema import oidc_identities
with self._engine.connect() as conn:
rows = conn.execute(
@@ -2739,7 +2730,6 @@ class PostgreSQLBackend:
]
def delete_oidc_identity(self, issuer: str, subject: str) -> bool:
from turnstone.core.storage._schema import oidc_identities
with self._engine.connect() as conn:
result = conn.execute(
@@ -2755,7 +2745,6 @@ class PostgreSQLBackend:
def create_oidc_pending_state(
self, state: str, nonce: str, code_verifier: str, audience: str
) -> None:
from turnstone.core.storage._schema import oidc_pending_states
now = datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%S")
with self._engine.connect() as conn:
@@ -2774,7 +2763,6 @@ class PostgreSQLBackend:
def pop_oidc_pending_state(
self, state: str, max_age_seconds: int = 300
) -> dict[str, str] | None:
from turnstone.core.storage._schema import oidc_pending_states
cutoff = (datetime.now(UTC) - timedelta(seconds=max_age_seconds)).strftime(
"%Y-%m-%dT%H:%M:%S"
@@ -2806,7 +2794,6 @@ class PostgreSQLBackend:
}
def cleanup_expired_oidc_states(self, max_age_seconds: int = 300) -> int:
from turnstone.core.storage._schema import oidc_pending_states
cutoff = (datetime.now(UTC) - timedelta(seconds=max_age_seconds)).strftime(
"%Y-%m-%dT%H:%M:%S"
+11 -1
View File
@@ -131,6 +131,15 @@ class StorageBackend(Protocol):
"""Search structured memories by query. Returns matching memory dicts."""
...
def touch_structured_memories(self, keys: list[tuple[str, str, str]]) -> int:
"""Batch-touch multiple memories.
Each key is ``(name, scope, scope_id)``. Callers should deduplicate
before calling; each key increments ``access_count`` once per call.
Returns count of rows found and updated.
"""
...
def count_structured_memories(
self, mem_type: str = "", scope: str = "", scope_id: str = ""
) -> int:
@@ -581,6 +590,7 @@ class StorageBackend(Protocol):
allowed_tools: str = "[]",
skill_license: str = "",
compatibility: str = "",
priority: int = 0,
) -> None:
"""Create a prompt template (skill)."""
...
@@ -626,7 +636,7 @@ class StorageBackend(Protocol):
enabled_only: bool = False,
limit: int = 0,
) -> list[dict[str, Any]]:
"""Return prompt templates filtered by activation value, ordered by name."""
"""Return prompt templates filtered by activation value, ordered by priority then name."""
...
def get_skill_by_name(self, name: str) -> dict[str, Any] | None:
+4 -3
View File
@@ -2,14 +2,15 @@
from __future__ import annotations
import logging
import os
from typing import TYPE_CHECKING
from turnstone.core.log import get_logger
if TYPE_CHECKING:
from turnstone.core.storage._protocol import StorageBackend
log = logging.getLogger(__name__)
log = get_logger(__name__)
_storage: StorageBackend | None = None
@@ -19,7 +20,7 @@ def init_storage(
*,
path: str = "",
url: str = "",
pool_size: int = 5,
pool_size: int = 2,
run_migrations: bool = True,
) -> StorageBackend:
"""Initialize the storage backend singleton.
+3
View File
@@ -41,6 +41,8 @@ conversations = sa.Table(
sa.Column("tool_calls", sa.Text),
)
sa.Index("idx_conversations_timestamp", conversations.c.timestamp)
workstreams = sa.Table(
"workstreams",
metadata,
@@ -328,6 +330,7 @@ prompt_templates = sa.Table(
sa.Column("agent_max_turns", sa.Integer, nullable=True),
sa.Column("notify_on_complete", sa.Text, nullable=False, server_default="{}"),
sa.Column("enabled", sa.Integer, nullable=False, server_default="1"),
sa.Column("priority", sa.Integer, nullable=False, server_default="0"),
sa.Column("created", sa.Text, nullable=False),
sa.Column("updated", sa.Text, nullable=False),
)
+63 -56
View File
@@ -2,23 +2,30 @@
from __future__ import annotations
import logging
from datetime import UTC, datetime, timedelta
from typing import Any
import sqlalchemy as sa
from turnstone.core.log import get_logger
from turnstone.core.storage._schema import (
api_tokens,
audit_events,
channel_routes,
channel_users,
conversations,
intent_verdicts,
mcp_servers,
metadata,
oidc_identities,
oidc_pending_states,
orgs,
output_assessments,
prompt_templates,
roles,
scheduled_task_runs,
scheduled_tasks,
services,
skill_resources,
skill_versions,
structured_memories,
@@ -27,6 +34,7 @@ from turnstone.core.storage._schema import (
usage_events,
user_roles,
users,
watches,
workstream_config,
workstreams,
)
@@ -61,7 +69,7 @@ from turnstone.core.storage._utils import (
scan_skill_content as _scan_skill_content,
)
log = logging.getLogger(__name__)
log = get_logger(__name__)
def _escape_like(s: str) -> str:
@@ -87,8 +95,21 @@ class SQLiteBackend:
self._engine = sa.create_engine(
f"sqlite:///{path}",
pool_pre_ping=True,
connect_args={"check_same_thread": False},
connect_args={"check_same_thread": False, "timeout": 30},
)
# Enable WAL mode for better concurrent read/write performance.
@sa.event.listens_for(self._engine, "connect")
def _set_wal(dbapi_conn: Any, _rec: Any) -> None:
try:
cursor = dbapi_conn.execute("PRAGMA journal_mode=WAL")
mode = cursor.fetchone()
cursor.close()
if mode and mode[0] != "wal":
log.warning("SQLite WAL mode not enabled (got %s)", mode[0])
except Exception:
log.warning("Failed to set SQLite WAL mode", exc_info=True)
self._fts5_available = False
if create_tables:
self._init_schema()
@@ -295,15 +316,16 @@ class SQLiteBackend:
# -- Workstream config -----------------------------------------------------
def save_workstream_config(self, ws_id: str, config: dict[str, str]) -> None:
if not config:
return
with self._engine.connect() as conn:
for key, value in config.items():
conn.execute(
sa.text(
"INSERT OR REPLACE INTO workstream_config "
"(ws_id, key, value) VALUES (:wid, :key, :value)"
),
{"wid": ws_id, "key": key, "value": value},
)
conn.execute(
sa.text(
"INSERT OR REPLACE INTO workstream_config "
"(ws_id, key, value) VALUES (:wid, :key, :value)"
),
[{"wid": ws_id, "key": key, "value": value} for key, value in config.items()],
)
conn.commit()
def load_workstream_config(self, ws_id: str) -> dict[str, str]:
@@ -573,7 +595,6 @@ class SQLiteBackend:
]
def delete_user(self, user_id: str) -> bool:
from turnstone.core.storage._schema import channel_users, oidc_identities
with self._engine.connect() as conn:
conn.execute(sa.delete(user_roles).where(user_roles.c.user_id == user_id))
@@ -677,7 +698,6 @@ class SQLiteBackend:
# -- Channel user mapping ---------------------------------------------------
def create_channel_user(self, channel_type: str, channel_user_id: str, user_id: str) -> None:
from turnstone.core.storage._schema import channel_users
now = datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%S")
with self._engine.connect() as conn:
@@ -693,7 +713,6 @@ class SQLiteBackend:
conn.commit()
def get_channel_user(self, channel_type: str, channel_user_id: str) -> dict[str, str] | None:
from turnstone.core.storage._schema import channel_users
with self._engine.connect() as conn:
row = conn.execute(
@@ -717,7 +736,6 @@ class SQLiteBackend:
return None
def list_channel_users_by_user(self, user_id: str) -> list[dict[str, str]]:
from turnstone.core.storage._schema import channel_users
with self._engine.connect() as conn:
rows = conn.execute(
@@ -741,7 +759,6 @@ class SQLiteBackend:
]
def delete_channel_user(self, channel_type: str, channel_user_id: str) -> bool:
from turnstone.core.storage._schema import channel_users
with self._engine.connect() as conn:
result = conn.execute(
@@ -758,7 +775,6 @@ class SQLiteBackend:
def create_channel_route(
self, channel_type: str, channel_id: str, ws_id: str, node_id: str = ""
) -> None:
from turnstone.core.storage._schema import channel_routes
now = datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%S")
with self._engine.connect() as conn:
@@ -775,7 +791,6 @@ class SQLiteBackend:
conn.commit()
def get_channel_route(self, channel_type: str, channel_id: str) -> dict[str, str] | None:
from turnstone.core.storage._schema import channel_routes
with self._engine.connect() as conn:
row = conn.execute(
@@ -801,7 +816,6 @@ class SQLiteBackend:
return None
def get_channel_route_by_ws(self, ws_id: str) -> dict[str, str] | None:
from turnstone.core.storage._schema import channel_routes
with self._engine.connect() as conn:
row = conn.execute(
@@ -824,7 +838,6 @@ class SQLiteBackend:
return None
def list_channel_routes_by_type(self, channel_type: str) -> list[dict[str, str]]:
from turnstone.core.storage._schema import channel_routes
with self._engine.connect() as conn:
rows = conn.execute(
@@ -850,7 +863,6 @@ class SQLiteBackend:
]
def delete_channel_route(self, channel_type: str, channel_id: str) -> bool:
from turnstone.core.storage._schema import channel_routes
with self._engine.connect() as conn:
result = conn.execute(
@@ -881,7 +893,6 @@ class SQLiteBackend:
next_run: str,
skill: str = "",
) -> None:
from turnstone.core.storage._schema import scheduled_tasks
now = datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%S")
with self._engine.connect() as conn:
@@ -910,7 +921,6 @@ class SQLiteBackend:
conn.commit()
def get_scheduled_task(self, task_id: str) -> dict[str, Any] | None:
from turnstone.core.storage._schema import scheduled_tasks
with self._engine.connect() as conn:
row = conn.execute(
@@ -921,7 +931,6 @@ class SQLiteBackend:
return dict(row._mapping)
def list_scheduled_tasks(self) -> list[dict[str, Any]]:
from turnstone.core.storage._schema import scheduled_tasks
with self._engine.connect() as conn:
rows = conn.execute(
@@ -950,7 +959,6 @@ class SQLiteBackend:
)
def update_scheduled_task(self, task_id: str, **fields: Any) -> bool:
from turnstone.core.storage._schema import scheduled_tasks
fields = {k: v for k, v in fields.items() if k in self._UPDATABLE_TASK_FIELDS}
fields["updated"] = datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%S")
@@ -971,7 +979,6 @@ class SQLiteBackend:
return result.rowcount > 0
def delete_scheduled_task(self, task_id: str) -> bool:
from turnstone.core.storage._schema import scheduled_task_runs, scheduled_tasks
with self._engine.connect() as conn:
conn.execute(
@@ -984,7 +991,6 @@ class SQLiteBackend:
return result.rowcount > 0
def list_due_tasks(self, now: str) -> list[dict[str, Any]]:
from turnstone.core.storage._schema import scheduled_tasks
with self._engine.connect() as conn:
rows = conn.execute(
@@ -1010,7 +1016,6 @@ class SQLiteBackend:
status: str,
error: str,
) -> None:
from turnstone.core.storage._schema import scheduled_task_runs
with self._engine.connect() as conn:
conn.execute(
@@ -1029,7 +1034,6 @@ class SQLiteBackend:
conn.commit()
def list_task_runs(self, task_id: str, limit: int = 50) -> list[dict[str, Any]]:
from turnstone.core.storage._schema import scheduled_task_runs
with self._engine.connect() as conn:
rows = conn.execute(
@@ -1041,10 +1045,6 @@ class SQLiteBackend:
return [dict(r._mapping) for r in rows]
def prune_task_runs(self, retention_days: int = 90) -> int:
from datetime import timedelta
from turnstone.core.storage._schema import scheduled_task_runs
cutoff = (datetime.now(UTC) - timedelta(days=retention_days)).strftime("%Y-%m-%dT%H:%M:%S")
with self._engine.connect() as conn:
result = conn.execute(
@@ -1068,7 +1068,6 @@ class SQLiteBackend:
created_by: str,
next_poll: str,
) -> None:
from turnstone.core.storage._schema import watches
now = datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%S")
with self._engine.connect() as conn:
@@ -1094,7 +1093,6 @@ class SQLiteBackend:
conn.commit()
def get_watch(self, watch_id: str) -> dict[str, Any] | None:
from turnstone.core.storage._schema import watches
with self._engine.connect() as conn:
row = conn.execute(sa.select(watches).where(watches.c.watch_id == watch_id)).fetchone()
@@ -1103,7 +1101,6 @@ class SQLiteBackend:
return dict(row._mapping)
def list_watches_for_ws(self, ws_id: str) -> list[dict[str, Any]]:
from turnstone.core.storage._schema import watches
with self._engine.connect() as conn:
rows = conn.execute(
@@ -1114,7 +1111,6 @@ class SQLiteBackend:
return [dict(r._mapping) for r in rows]
def list_watches_for_node(self, node_id: str) -> list[dict[str, Any]]:
from turnstone.core.storage._schema import watches
with self._engine.connect() as conn:
rows = conn.execute(
@@ -1125,7 +1121,6 @@ class SQLiteBackend:
return [dict(r._mapping) for r in rows]
def list_due_watches(self, now: str) -> list[dict[str, Any]]:
from turnstone.core.storage._schema import watches
with self._engine.connect() as conn:
rows = conn.execute(
@@ -1154,7 +1149,6 @@ class SQLiteBackend:
)
def update_watch(self, watch_id: str, **fields: Any) -> bool:
from turnstone.core.storage._schema import watches
fields = {k: v for k, v in fields.items() if k in self._UPDATABLE_WATCH_FIELDS}
fields["updated"] = datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%S")
@@ -1168,7 +1162,6 @@ class SQLiteBackend:
return result.rowcount > 0
def delete_watch(self, watch_id: str) -> bool:
from turnstone.core.storage._schema import watches
with self._engine.connect() as conn:
result = conn.execute(sa.delete(watches).where(watches.c.watch_id == watch_id))
@@ -1176,7 +1169,6 @@ class SQLiteBackend:
return result.rowcount > 0
def delete_watches_for_ws(self, ws_id: str) -> int:
from turnstone.core.storage._schema import watches
with self._engine.connect() as conn:
result = conn.execute(sa.delete(watches).where(watches.c.ws_id == ws_id))
@@ -1190,8 +1182,6 @@ class SQLiteBackend:
) -> None:
from sqlalchemy.dialects.sqlite import insert as sqlite_insert
from turnstone.core.storage._schema import services
now = datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%S")
stmt = sqlite_insert(services).values(
service_type=service_type,
@@ -1210,7 +1200,6 @@ class SQLiteBackend:
conn.commit()
def heartbeat_service(self, service_type: str, service_id: str) -> bool:
from turnstone.core.storage._schema import services
now = datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%S")
with self._engine.connect() as conn:
@@ -1226,7 +1215,6 @@ class SQLiteBackend:
return result.rowcount > 0
def list_services(self, service_type: str, max_age_seconds: int = 120) -> list[dict[str, str]]:
from turnstone.core.storage._schema import services
cutoff = (datetime.now(UTC) - timedelta(seconds=max_age_seconds)).strftime(
"%Y-%m-%dT%H:%M:%S"
@@ -1243,7 +1231,6 @@ class SQLiteBackend:
return [dict(r._mapping) for r in rows]
def deregister_service(self, service_type: str, service_id: str) -> bool:
from turnstone.core.storage._schema import services
with self._engine.connect() as conn:
result = conn.execute(
@@ -1534,6 +1521,7 @@ class SQLiteBackend:
allowed_tools: str = "[]",
skill_license: str = "",
compatibility: str = "",
priority: int = 0,
) -> None:
# Sync is_default from activation when activation is explicitly set
if activation == "default":
@@ -1580,6 +1568,7 @@ class SQLiteBackend:
"agent_max_turns": agent_max_turns,
"notify_on_complete": notify_on_complete,
"enabled": 1 if enabled else 0,
"priority": priority,
"created": now,
"updated": now,
},
@@ -1633,7 +1622,7 @@ class SQLiteBackend:
sa.select(prompt_templates)
.where(prompt_templates.c.is_default == 1)
.where(prompt_templates.c.enabled == 1)
.order_by(prompt_templates.c.name)
.order_by(prompt_templates.c.priority, prompt_templates.c.name)
)
if org_id:
q = q.where(prompt_templates.c.org_id == org_id)
@@ -1718,7 +1707,7 @@ class SQLiteBackend:
q = (
sa.select(prompt_templates)
.where(prompt_templates.c.activation == activation)
.order_by(prompt_templates.c.name)
.order_by(prompt_templates.c.priority, prompt_templates.c.name)
)
if enabled_only:
q = q.where(prompt_templates.c.enabled == 1)
@@ -2457,6 +2446,32 @@ class SQLiteBackend:
).fetchall()
return [dict(r._mapping) for r in rows]
def touch_structured_memories(self, keys: list[tuple[str, str, str]]) -> int:
"""Batch-touch multiple memories by (name, scope, scope_id)."""
if not keys:
return 0
now = datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%S")
total = 0
with self._engine.connect() as conn:
for name, scope, scope_id in keys:
result = conn.execute(
sa.update(structured_memories)
.where(
sa.and_(
structured_memories.c.name == name,
structured_memories.c.scope == scope,
structured_memories.c.scope_id == scope_id,
)
)
.values(
last_accessed=now,
access_count=structured_memories.c.access_count + 1,
)
)
total += result.rowcount
conn.commit()
return total
def count_structured_memories(
self, mem_type: str = "", scope: str = "", scope_id: str = ""
) -> int:
@@ -2678,7 +2693,6 @@ class SQLiteBackend:
# -- OIDC identity ---------------------------------------------------------
def create_oidc_identity(self, issuer: str, subject: str, user_id: str, email: str) -> None:
from turnstone.core.storage._schema import oidc_identities
now = datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%S")
with self._engine.connect() as conn:
@@ -2696,7 +2710,6 @@ class SQLiteBackend:
conn.commit()
def get_oidc_identity(self, issuer: str, subject: str) -> dict[str, str] | None:
from turnstone.core.storage._schema import oidc_identities
with self._engine.connect() as conn:
row = conn.execute(
@@ -2723,7 +2736,6 @@ class SQLiteBackend:
return None
def update_oidc_identity_login(self, issuer: str, subject: str) -> bool:
from turnstone.core.storage._schema import oidc_identities
now = datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%S")
with self._engine.connect() as conn:
@@ -2738,7 +2750,6 @@ class SQLiteBackend:
return result.rowcount > 0
def list_oidc_identities_for_user(self, user_id: str) -> list[dict[str, str]]:
from turnstone.core.storage._schema import oidc_identities
with self._engine.connect() as conn:
rows = conn.execute(
@@ -2766,7 +2777,6 @@ class SQLiteBackend:
]
def delete_oidc_identity(self, issuer: str, subject: str) -> bool:
from turnstone.core.storage._schema import oidc_identities
with self._engine.connect() as conn:
result = conn.execute(
@@ -2782,7 +2792,6 @@ class SQLiteBackend:
def create_oidc_pending_state(
self, state: str, nonce: str, code_verifier: str, audience: str
) -> None:
from turnstone.core.storage._schema import oidc_pending_states
now = datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%S")
with self._engine.connect() as conn:
@@ -2801,7 +2810,6 @@ class SQLiteBackend:
def pop_oidc_pending_state(
self, state: str, max_age_seconds: int = 300
) -> dict[str, str] | None:
from turnstone.core.storage._schema import oidc_pending_states
cutoff = (datetime.now(UTC) - timedelta(seconds=max_age_seconds)).strftime(
"%Y-%m-%dT%H:%M:%S"
@@ -2835,7 +2843,6 @@ class SQLiteBackend:
}
def cleanup_expired_oidc_states(self, max_age_seconds: int = 300) -> int:
from turnstone.core.storage._schema import oidc_pending_states
cutoff = (datetime.now(UTC) - timedelta(seconds=max_age_seconds)).strftime(
"%Y-%m-%dT%H:%M:%S"
+4 -2
View File
@@ -4,10 +4,11 @@ from __future__ import annotations
import contextlib
import json
import logging
from typing import Any
log = logging.getLogger(__name__)
from turnstone.core.log import get_logger
log = get_logger(__name__)
# ---------------------------------------------------------------------------
# Row helper
@@ -59,6 +60,7 @@ SKILL_MUTABLE = frozenset(
"scan_version",
"scan_status",
"scan_report",
"priority",
}
)
STRUCTURED_MEMORY_MUTABLE = frozenset({"content", "description", "type"})
+5 -1
View File
@@ -19,7 +19,11 @@ def run_migrations_online() -> None:
)
with connectable.connect() as connection:
context.configure(connection=connection, target_metadata=target_metadata)
context.configure(
connection=connection,
target_metadata=target_metadata,
render_as_batch=True,
)
with context.begin_transaction():
context.run_migrations()
@@ -28,9 +28,7 @@ def upgrade() -> None:
sa.Column("updated", sa.Text, nullable=False),
sa.Column("last_accessed", sa.Text, nullable=False, server_default=""),
sa.Column("access_count", sa.Integer, nullable=False, server_default="0"),
)
op.create_unique_constraint(
"uq_smem_name_scope", "structured_memories", ["name", "scope", "scope_id"]
sa.UniqueConstraint("name", "scope", "scope_id", name="uq_smem_name_scope"),
)
op.create_index("idx_smem_type", "structured_memories", ["type"])
op.create_index("idx_smem_scope", "structured_memories", ["scope", "scope_id"])
@@ -27,56 +27,32 @@ depends_on = None
def upgrade() -> None:
# Phase 1: Skills evolution columns
op.add_column(
"prompt_templates",
sa.Column("description", sa.Text, nullable=False, server_default=""),
)
op.add_column(
"prompt_templates",
sa.Column("tags", sa.Text, nullable=False, server_default="[]"),
)
op.add_column(
"prompt_templates",
sa.Column("source_url", sa.Text, nullable=False, server_default=""),
)
op.add_column(
"prompt_templates",
sa.Column("version", sa.Text, nullable=False, server_default="1.0.0"),
)
op.add_column(
"prompt_templates",
sa.Column("author", sa.Text, nullable=False, server_default=""),
)
op.add_column(
"prompt_templates",
sa.Column("activation", sa.Text, nullable=False, server_default="named"),
)
op.add_column(
"prompt_templates",
sa.Column("token_estimate", sa.Integer, nullable=False, server_default="0"),
)
# Phase 2: Security scanning + install provenance
op.add_column(
"prompt_templates",
sa.Column("allowed_tools", sa.Text, nullable=False, server_default="[]"),
)
op.add_column(
"prompt_templates",
sa.Column("scan_status", sa.Text, nullable=False, server_default=""),
)
op.add_column(
"prompt_templates",
sa.Column("scan_report", sa.Text, nullable=False, server_default="{}"),
)
op.add_column(
"prompt_templates",
sa.Column("installed_at", sa.Text, nullable=False, server_default=""),
)
op.add_column(
"prompt_templates",
sa.Column("installed_by", sa.Text, nullable=False, server_default=""),
)
# Phase 1+2: Skills evolution columns + security scanning (batch for SQLite)
with op.batch_alter_table("prompt_templates") as batch_op:
batch_op.add_column(sa.Column("description", sa.Text, nullable=False, server_default=""))
batch_op.add_column(sa.Column("tags", sa.Text, nullable=False, server_default="[]"))
batch_op.add_column(sa.Column("source_url", sa.Text, nullable=False, server_default=""))
batch_op.add_column(sa.Column("version", sa.Text, nullable=False, server_default="1.0.0"))
batch_op.add_column(sa.Column("author", sa.Text, nullable=False, server_default=""))
batch_op.add_column(
sa.Column("activation", sa.Text, nullable=False, server_default="named"),
)
batch_op.add_column(
sa.Column("token_estimate", sa.Integer, nullable=False, server_default="0"),
)
batch_op.add_column(
sa.Column("allowed_tools", sa.Text, nullable=False, server_default="[]"),
)
batch_op.add_column(sa.Column("scan_status", sa.Text, nullable=False, server_default=""))
batch_op.add_column(
sa.Column("scan_report", sa.Text, nullable=False, server_default="{}"),
)
batch_op.add_column(
sa.Column("installed_at", sa.Text, nullable=False, server_default=""),
)
batch_op.add_column(
sa.Column("installed_by", sa.Text, nullable=False, server_default=""),
)
# Backfill activation from is_default
op.execute("UPDATE prompt_templates SET activation = 'default' WHERE is_default = 1")
@@ -101,42 +77,26 @@ def upgrade() -> None:
)
# Phase 3: Session config columns (from workstream templates)
op.add_column(
"prompt_templates",
sa.Column("model", sa.Text, nullable=False, server_default=""),
)
op.add_column(
"prompt_templates",
sa.Column("auto_approve", sa.Integer, nullable=False, server_default="0"),
)
op.add_column(
"prompt_templates",
sa.Column("temperature", sa.Float, nullable=True),
)
op.add_column(
"prompt_templates",
sa.Column("reasoning_effort", sa.Text, nullable=False, server_default=""),
)
op.add_column(
"prompt_templates",
sa.Column("max_tokens", sa.Integer, nullable=True),
)
op.add_column(
"prompt_templates",
sa.Column("token_budget", sa.Integer, nullable=False, server_default="0"),
)
op.add_column(
"prompt_templates",
sa.Column("agent_max_turns", sa.Integer, nullable=True),
)
op.add_column(
"prompt_templates",
sa.Column("notify_on_complete", sa.Text, nullable=False, server_default="{}"),
)
op.add_column(
"prompt_templates",
sa.Column("enabled", sa.Integer, nullable=False, server_default="1"),
)
with op.batch_alter_table("prompt_templates") as batch_op:
batch_op.add_column(sa.Column("model", sa.Text, nullable=False, server_default=""))
batch_op.add_column(
sa.Column("auto_approve", sa.Integer, nullable=False, server_default="0"),
)
batch_op.add_column(sa.Column("temperature", sa.Float, nullable=True))
batch_op.add_column(
sa.Column("reasoning_effort", sa.Text, nullable=False, server_default=""),
)
batch_op.add_column(sa.Column("max_tokens", sa.Integer, nullable=True))
batch_op.add_column(
sa.Column("token_budget", sa.Integer, nullable=False, server_default="0"),
)
batch_op.add_column(sa.Column("agent_max_turns", sa.Integer, nullable=True))
batch_op.add_column(
sa.Column("notify_on_complete", sa.Text, nullable=False, server_default="{}"),
)
batch_op.add_column(
sa.Column("enabled", sa.Integer, nullable=False, server_default="1"),
)
# Skill versions — version history for skills
op.create_table(
@@ -253,24 +213,25 @@ def upgrade() -> None:
# Rename workstreams table columns: ws_template_id → skill_id,
# ws_template_version → skill_version
op.alter_column("workstreams", "ws_template_id", new_column_name="skill_id")
op.alter_column("workstreams", "ws_template_version", new_column_name="skill_version")
with op.batch_alter_table("workstreams") as batch_op:
batch_op.alter_column("ws_template_id", new_column_name="skill_id")
batch_op.alter_column("ws_template_version", new_column_name="skill_version")
# Rename scheduled_tasks.template → skill
op.alter_column("scheduled_tasks", "template", new_column_name="skill")
# Rename scheduled_tasks.template → skill, drop ws_template
with op.batch_alter_table("scheduled_tasks") as batch_op:
batch_op.alter_column("template", new_column_name="skill")
batch_op.drop_column("ws_template")
# Drop old tables
op.drop_table("workstream_template_versions")
op.drop_table("workstream_templates")
# Drop ws_template column from scheduled_tasks
op.drop_column("scheduled_tasks", "ws_template")
def downgrade() -> None:
# Reverse workstreams column renames
op.alter_column("workstreams", "skill_id", new_column_name="ws_template_id")
op.alter_column("workstreams", "skill_version", new_column_name="ws_template_version")
with op.batch_alter_table("workstreams") as batch_op:
batch_op.alter_column("skill_id", new_column_name="ws_template_id")
batch_op.alter_column("skill_version", new_column_name="ws_template_version")
# Reverse workstream_config key renames
op.execute("UPDATE workstream_config SET key = 'ws_template_id' WHERE key = 'applied_skill_id'")
@@ -283,14 +244,10 @@ def downgrade() -> None:
"WHERE key = 'applied_skill_content'"
)
# Reverse scheduled_tasks column rename: skill → template
op.alter_column("scheduled_tasks", "skill", new_column_name="template")
# Re-add ws_template column to scheduled_tasks
op.add_column(
"scheduled_tasks",
sa.Column("ws_template", sa.Text, nullable=False, server_default=""),
)
# Reverse scheduled_tasks column rename + re-add ws_template
with op.batch_alter_table("scheduled_tasks") as batch_op:
batch_op.alter_column("skill", new_column_name="template")
batch_op.add_column(sa.Column("ws_template", sa.Text, nullable=False, server_default=""))
# Recreate workstream_templates (empty — destructive migration)
op.create_table(
@@ -333,32 +290,31 @@ def downgrade() -> None:
op.drop_index("idx_skill_versions_skill_id", table_name="skill_versions")
op.drop_table("skill_versions")
# Drop session config columns from prompt_templates
op.drop_column("prompt_templates", "enabled")
op.drop_column("prompt_templates", "notify_on_complete")
op.drop_column("prompt_templates", "agent_max_turns")
op.drop_column("prompt_templates", "token_budget")
op.drop_column("prompt_templates", "max_tokens")
op.drop_column("prompt_templates", "reasoning_effort")
op.drop_column("prompt_templates", "temperature")
op.drop_column("prompt_templates", "auto_approve")
op.drop_column("prompt_templates", "model")
# Drop session config + skills evolution columns from prompt_templates
with op.batch_alter_table("prompt_templates") as batch_op:
batch_op.drop_column("enabled")
batch_op.drop_column("notify_on_complete")
batch_op.drop_column("agent_max_turns")
batch_op.drop_column("token_budget")
batch_op.drop_column("max_tokens")
batch_op.drop_column("reasoning_effort")
batch_op.drop_column("temperature")
batch_op.drop_column("auto_approve")
batch_op.drop_column("model")
batch_op.drop_column("installed_by")
batch_op.drop_column("installed_at")
batch_op.drop_column("scan_report")
batch_op.drop_column("scan_status")
batch_op.drop_column("allowed_tools")
batch_op.drop_column("token_estimate")
batch_op.drop_column("activation")
batch_op.drop_column("author")
batch_op.drop_column("version")
batch_op.drop_column("source_url")
batch_op.drop_column("tags")
batch_op.drop_column("description")
# Drop skill resources
op.drop_index("idx_skill_resources_skill_path", table_name="skill_resources")
op.drop_index("idx_skill_resources_skill_id", table_name="skill_resources")
op.drop_table("skill_resources")
# Drop skills evolution columns
op.drop_column("prompt_templates", "installed_by")
op.drop_column("prompt_templates", "installed_at")
op.drop_column("prompt_templates", "scan_report")
op.drop_column("prompt_templates", "scan_status")
op.drop_column("prompt_templates", "allowed_tools")
op.drop_column("prompt_templates", "token_estimate")
op.drop_column("prompt_templates", "activation")
op.drop_column("prompt_templates", "author")
op.drop_column("prompt_templates", "version")
op.drop_column("prompt_templates", "source_url")
op.drop_column("prompt_templates", "tags")
op.drop_column("prompt_templates", "description")
@@ -33,14 +33,13 @@ def upgrade() -> None:
op.create_index("ix_oa_created", "output_assessments", ["created"])
op.create_index("ix_oa_risk", "output_assessments", ["risk_level"])
op.add_column(
"prompt_templates",
sa.Column("scan_version", sa.Text, nullable=False, server_default=""),
)
with op.batch_alter_table("prompt_templates") as batch_op:
batch_op.add_column(sa.Column("scan_version", sa.Text, nullable=False, server_default=""))
def downgrade() -> None:
op.drop_column("prompt_templates", "scan_version")
with op.batch_alter_table("prompt_templates") as batch_op:
batch_op.drop_column("scan_version")
op.drop_index("ix_oa_risk", table_name="output_assessments")
op.drop_index("ix_oa_created", table_name="output_assessments")
op.drop_index("ix_oa_ws_id", table_name="output_assessments")
@@ -19,16 +19,14 @@ depends_on = None
def upgrade() -> None:
op.add_column(
"prompt_templates",
sa.Column("license", sa.Text, nullable=False, server_default=""),
)
op.add_column(
"prompt_templates",
sa.Column("compatibility", sa.Text, nullable=False, server_default=""),
)
with op.batch_alter_table("prompt_templates") as batch_op:
batch_op.add_column(sa.Column("license", sa.Text, nullable=False, server_default=""))
batch_op.add_column(
sa.Column("compatibility", sa.Text, nullable=False, server_default=""),
)
def downgrade() -> None:
op.drop_column("prompt_templates", "compatibility")
op.drop_column("prompt_templates", "license")
with op.batch_alter_table("prompt_templates") as batch_op:
batch_op.drop_column("compatibility")
batch_op.drop_column("license")

Some files were not shown because too many files have changed in this diff Show More