Compare commits

...

37 Commits

Author SHA1 Message Date
Patrick Buckley 1cded9b430 chore: bump version to 0.9.0 2026-03-27 21:24:49 -07:00
Patrick Buckley 62c741eb8a fix: prevent assistant messages with content=None from reaching OpenAI API (#195)
Replayed or cancelled conversations could produce assistant messages
with content=None and no tool_calls, which OpenAI-compatible APIs
reject with a 400.  Fix at three layers for defense in depth:

- session.py: use empty string instead of None when building assistant
  messages (streaming + cancellation paths)
- _utils.py: normalise content on DB load in reconstruct_messages()
- _openai.py: add _sanitize_messages() catch-all at provider boundary

Closes #194
2026-03-27 21:22:39 -07:00
Patrick Buckley 3362917e1e chore(deps): update vendored KaTeX 0.16.42 → 0.16.43 (#193) 2026-03-27 10:54:21 -07:00
renovate[bot] 698cbbf988 chore(deps): lock file maintenance (#192)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-03-26 20:55:46 -07:00
renovate[bot] e47a08b7bc chore(deps): update ghcr.io/astral-sh/uv docker tag to v0.11.2 (#191)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-03-26 20:55:00 -07:00
renovate[bot] aaa427debd chore(deps): update dependency vitest to v4.1.2 (#190)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-03-26 20:54:58 -07:00
renovate[bot] 611af76971 chore(deps): pin dependencies (#188)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-03-26 20:54:49 -07:00
Patrick Buckley bbe28ecab3 fix: cancel LLM judge daemon when user approves/denies tools (#187)
When the LLM judge is enabled and the user makes rapid approval
decisions, judge daemon threads pile up competing for the inference
server, causing timeouts. Pass a threading.Event from session to the
judge — set on approval — so the daemon abandons remaining work
within ~1s, including shutting down the executor to kill in-flight
API calls.
2026-03-26 18:10:23 -07:00
Patrick Buckley 93a9fd3c28 bump: v0.8.9 — mTLS + ACME integration via lacme 2026-03-26 14:33:11 -07:00
Patrick Buckley 62ff3217d0 fix: TLS Docker end-to-end testing fixes (#185)
* fix: TLS Docker end-to-end testing fixes

Fixes discovered during Docker Compose TLS integration testing:

- Dockerfile: use --extra all (prevents missing optional deps)
- lacme 1.0.3: fixes CACertificateIssued event logging crash
- chmod PermissionError: guard for Docker volume mounts
- socket import: moved to top of main() (was inside TLS conditional,
  caused NameError in _default_node_id)
- redis.SSLConnection: ConnectionPool needs explicit connection_class,
  not ssl=True (which only works on Redis() directly)
- Empty redis password: pass None instead of "" to avoid AUTH error
- TURNSTONE_CONSOLE_URL: env var for Docker service discovery
  (0.0.0.0 bind address isn't reachable from other containers)
- HTTP01Handler: ACME client needs a challenge handler even when
  server auto-approves
- Docker overlay: tls-init as root with chmod, Redis conditional
  password, console Redis TLS flags, TURNSTONE_CONSOLE_URL

* feat: full mTLS end-to-end with lacme 1.0.4

Completes the mTLS chain across all services:

lacme 1.0.4:
- Dual EKU certs (serverAuth + clientAuth) — fixes mTLS rejection
- Configurable CA name (name="turnstone") — consistent store key

Bootstrap CA import:
- Console imports bootstrap CA from /certs volume on first boot
- Single trust root: bootstrap CA → console → all service certs

Bridge mTLS:
- TLSClient init when TURNSTONE_TLS_ENABLED set
- Auto-upgrades server URL from http:// to https://
- SSLContext passed to all 3 httpx clients via verify=

Console collector mTLS:
- upgrade_tls() method replaces httpx client with mTLS context
- Called in lifespan after cert issuance alongside proxy upgrade
- Fixes "Failed to poll node" when server serves HTTPS

Docker overlay:
- TURNSTONE_TLS_SANS on all services (Docker service names as SANs)
- TURNSTONE_TLS_ENABLED on bridge
- Channel service with Redis TLS flags
- TURNSTONE_CONSOLE_URL for service discovery
- Server healthcheck disabled (mTLS healthcheck deferred)
- Redis conditional password from env

Verified end-to-end: bootstrap → console CA → server HTTPS →
bridge mTLS → Redis TLS → channel Redis TLS → console collector
polls server over mTLS → workstream creation works through bridge

* fix: lint + copilot feedback on TLS Docker e2e

- SIM105: contextlib.suppress(PermissionError) for chmod
- F401: remove unused get_storage import in bridge
- Redis healthcheck: pass password when REDIS_PASSWORD is set

* fix: sort imports in admin.py and bridge.py

* fix: tls-init key permissions, healthcheck env, collector race

- tls-init: add set -e, chown to turnstone:turnstone with restrictive
  perms (keys 0600, certs 0640, dirs 0750) instead of world-readable
- Redis healthcheck: use container runtime $$REDIS_PASSWORD instead of
  Compose-time interpolation for consistency with --requirepass block
- collector upgrade_tls(): don't close old httpx client while concurrent
  poll threads may still be using it — let GC handle cleanup
2026-03-26 14:24:44 -07:00
Patrick Buckley b086390558 fix: TLS deferred work — wiring, security, tests, Docker overlay (#184)
* fix: TLS deferred work — wiring, security, tests, Docker overlay

Security fixes:
- PostgreSQL SSL: validate sslmode against known values, urlencode
  all params to prevent URL injection
- ConfigStore env seeding: removed redundant type coercion, delegate
  to validate_value() which handles all coercion correctly

Functional wiring:
- Database SSL: init_storage() passes SSL params to PostgreSQL URL
- Server: env var fallbacks for DB SSL (TURNSTONE_DB_SSLMODE etc.)
- Proxy mTLS: re-create proxy clients after TLS cert issuance
- Channel gateway: --ssl-certfile/keyfile/ca-certs CLI args, HTTPS
  advertise URL when SSL configured
- ConfigStore env seeding: TURNSTONE_{SECTION}_{KEY} seeds on first boot
- Console deregistration on shutdown (with debug logging)

Specs, tests, Docker:
- OpenAPI: 5 TLS admin endpoints in console_spec.py
- Auth enforcement test (401 without auth)
- SDK ValueError test (mismatched cert/key)
- Docker overlay: TURNSTONE_TLS_ENABLED, bridge --redis-tls, Redis
  healthcheck with client cert
- Removed stale type:ignore comments (lacme 1.0.2 type stubs)

* review: address copilot feedback on TLS deferred work

- ConfigStore env seeding: use config_store.set() instead of
  storage.set_system_setting() (correct API, updates cache)
- Remove unused defn variable (iterate SETTINGS keys only)
- Fix structlog call-arg error (positional args, not kwargs)
- Channel gateway: validate cert+key provided together
- Restore type:ignore[no-any-return] for CI mypy (lacme 1.0.2
  type stubs not in CI's mypy overrides yet)

* fix: rename _VALID_SSLMODES to lowercase (N806)
2026-03-25 22:43:35 -07:00
Patrick Buckley d08a57dfc2 feat: SDK TLS support, Docker Compose overlay, TLS docs (#183)
Python SDK:
- ca_cert, client_cert, client_key on all 4 client classes
- ValueError if only one of client_cert/client_key provided
- Passed to httpx verify=/cert=

TypeScript SDK:
- TlsOptions type exported (zero runtime code)
- Fix picomatch vulnerability (npm audit fix)

Docker Compose:
- deploy/docker-compose.tls.yml overlay with tls-init bootstrap
- Notes it's an overlay requiring a base compose file

Documentation:
- docs/tls.md: architecture, config, CLI, SDK examples, troubleshooting
- Fixed package name (@turnstone/sdk), Node.js 18+ note
2026-03-25 20:43:34 -07:00
Patrick Buckley 9fdf51ff3d feat: TLS admin UI, CLI cert management, Redis/PG TLS (#182)
Admin API (require admin.settings):
- GET /v1/api/admin/tls/certs, POST .../renew, DELETE .../certs/{domain}
- renew_cert() updates in-memory bundles immediately

Admin UI (instrument panel grid pattern):
- TLS tab with CA status bar, cert grid, renew/delete actions
- Uses admin-row/admin-colheaders grid system (consistent with 12+ tabs)
- showConfirmModal for destructive actions, aria-labels on buttons
- Expired certs show "EXPIRED" text prefix + red color (WCAG 1.4.1)
- Loading state, empty state, error state

CLI (turnstone-admin):
- tls-bootstrap: offline CA + cert issuance (dir perms 0700)
- tls-issue: ACME cert request with key perms 0600
- tls-ca-cert: SHA-256 fingerprint for TOFU verification
- tls-list: auth via --auth-token or config token

Redis/PG TLS:
- RedisBroker + AsyncRedisBroker: ssl params wired through
- broker_from_args + async_broker_from_args: forward TLS kwargs
- add_redis_args: --redis-tls CLI flags
- Config map: [redis] and [database] TLS passthrough
2026-03-25 18:55:11 -07:00
Patrick Buckley 45471894da refactor: adopt lacme 1.0.2 — eliminate loopback client + temp file boilerplate (#181)
lacme 1.0.2 ships four features that simplify turnstone's TLS code:

- RenewalManager CA-direct mode: console renewal now uses ca= param
  instead of a loopback ACME client. No network, no startup ordering
  dependency, no client lifecycle management.
- ACMEResponder serves /ca.pem natively: removed custom route handler
  and route ordering workaround.
- write_pem_files_persistent: replaced manual temp file creation,
  chmod, and atexit cleanup with lacme's secure PEM file helper.
- Removed port param from TLSManager (was only for loopback URL).

Net: ~50 lines removed, two tech debt items resolved.
2026-03-25 18:02:26 -07:00
Patrick Buckley c0b5952573 feat: mTLS service clients with ACME auto-provisioning (#180)
TLSClient class for service nodes — discovers console via services
table, fetches CA cert, requests cert via ACME, provides SSL contexts:

- Console self-registers in services table for discovery
- Services auto-discover console URL from DB (no extra config)
- Initial cert request over plain HTTP (ACME provides integrity)
- Auto-renewal via RenewalManager in server lifespan
- Unauthenticated /acme/ca.pem endpoint for node bootstrapping

RenewalManager fix (was passing client=None):
- Console creates loopback ACME client for self-renewal
- Proper async lifecycle (aenter/aexit) with clean shutdown

Integration points wired:
- Server: TLS init before uvicorn, temp PEM files (0o600, atexit
  cleanup), auto-renewal in lifespan
- Bridge: tls_verify + tls_cert params on all 3 httpx clients
- Console collector: tls_verify + tls_cert params on httpx client
- Console proxy: mTLS context from TLSManager on proxy clients
- Channel gateway: optional SSL params on uvicorn.Config
- Console main(): reads tls.enabled, creates TLSManager, passes
  to create_app with console_url

6 tests for TLSClient (discovery, defaults, backward compat)
2026-03-25 17:48:34 -07:00
Patrick Buckley b9d5b5b671 feat: console CA + ACME server via lacme (#179)
TLSManager class owns the internal CA, ACME responder, and cert
lifecycle:
- CertificateAuthority with DB-backed storage (StorageStore adapter)
- ACMEResponder mounted at /acme (auto_approve for internal network)
- Dual cert issuance: internal CA for mTLS, optional external ACME CA
  for frontend HTTPS (Let's Encrypt via acme_directory setting)
- Auto-renewal via RenewalManager with clean async shutdown
- Expired cert detection on reload (re-issues instead of loading stale)
- EventDispatcher wired to structlog + lacme Prometheus metrics
- GET /v1/api/admin/tls/ca.pem — root cert download
- GET /v1/api/admin/tls/ca — CA status + cert inventory
- Console lifespan: init CA, issue certs, start renewal, stop on shutdown
- 11 async tests covering CA lifecycle, cert persistence, SSL contexts,
  endpoints, and event wiring
2026-03-25 16:35:55 -07:00
Patrick Buckley 274c97135e feat: TLS storage backend + config for lacme integration (#178)
Storage layer for mTLS certificate management via lacme:

- Migration 026: tls_account_keys, tls_ca, tls_certificates tables
- 8 protocol methods on StorageBackend (save/load account keys, CA,
  certs; list/delete certs)
- SQLite and PostgreSQL implementations with dialect-specific upserts
- StorageStore adapter bridging lacme's Store protocol to turnstone
  storage (bytes↔str PEM conversion, CertBundle↔dict mapping)
- Settings registry: tls.enabled (bool), tls.acme_directory (string)
- Config.toml: [redis] TLS and [database] SSL passthrough params
- lacme>=1.0.1 as optional [tls] dependency
- 20 unit tests covering storage CRUD + adapter + crypto roundtrip
2026-03-25 16:07:05 -07:00
Patrick Buckley e87f8e19c2 feat: adopt eval-optimized system prompt for plan_agent pattern
Update plan_agent tool pattern to show codebase exploration before
delegating to the planning agent. This pattern achieved 100% pass
rate (160/160 runs) on the eval suite — up from 98% on the previous
prompt.
2026-03-25 13:32:01 -07:00
Patrick Buckley bb221f4dab chore: bump v0.8.8, update vendored katex 0.16.40 → 0.16.42 2026-03-25 12:00:26 -07:00
Patrick Buckley 361876b17a feat: eval pipeline improvements + tool description optimization (#174)
Eval pipeline:
- --optimize-tools mode freezes system prompt, optimizes tool descriptions only
- Analyst sees available tool list (prevents hallucinated "tool not in schema")
- Analyst sees current tool descriptions in --optimize-tools mode
- Three-layer timeout defense: httpx timeout + _cancelled event + client.close()
- Filter MCP-only tools (read_resource, use_prompt) from headless eval
- Pattern-over-rules framing in optimizer, analyst, and observer prompts
- Tool description diffs logged after each iteration
- Tool optimizer failure retries instead of stopping the loop

Tool renames (avoid chat template channel collision on local models):
- create_plan -> plan_agent
- task -> task_agent

Tool descriptions (from eval-driven optimization, 79% -> 98%):
- bash: environment question examples, disambiguation from write_file/man
- edit_file: multi-file workflow, prerequisite clarification, docstring example
- man: "questions about flags are tool-use tasks" prefix
- math: simple example up front
- plan_agent: "delegate to sub-agent" framing, negative boundary for direct edits
- read_file: multi-file workflow hint
- search: trigger phrases, prerequisite clarification, disambiguation
- write_file: immediate action framing, placeholder example

System prompt: enriched tool patterns from eval results, added identity opener

Test suite: plan-before-refactor -> plan-when-asked (simplified)

Docs: comprehensive eval.md rewrite covering all current features
2026-03-25 11:58:10 -07:00
renovate[bot] 5d26cd6593 chore(deps): update dependency katex to v0.16.42 (#175)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-03-25 11:57:59 -07:00
renovate[bot] 2d4420e00d chore(deps): lock file maintenance (#177)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-03-25 11:57:47 -07:00
renovate[bot] b20548583d chore(deps): update ghcr.io/astral-sh/uv docker tag to v0.11.1 (#176)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-03-25 11:57:44 -07:00
Patrick Buckley f63b2915cc review: address copilot feedback on user_id trust check
Remove console-proxy from trusted_sources — end-user tokens via the
console proxy already carry the real user_id in the JWT, so they must
not be able to override it via the request body (impersonation risk).
Only bridge and console service identities are trusted to forward
user_id on behalf of users. Add 5 tests covering the trust boundary.
2026-03-24 03:13:23 -07:00
Patrick Buckley bf85bbea94 fix: resolve user_id to username in usage and audit displays
Usage tab grouped by user showed raw hex user_id instead of username.
Audit tab showed truncated hex. Now both API endpoints resolve user_id
to username via list_users() lookup before returning the response.
2026-03-24 03:13:23 -07:00
Patrick Buckley 803d8ee8f9 docs: document user_id propagation through MQ path
Update security.md with trusted service user_id forwarding. Update
MQ protocol diagram to include user_id field on CreateWorkstreamMessage.
Update console data flow diagram to show user_id in message and bridge
forwarding.
2026-03-24 03:13:23 -07:00
Patrick Buckley 17b5961a70 fix: propagate user_id through MQ workstream creation path
Console create_workstream was constructing CreateWorkstreamMessage
without setting user_id, so workstreams created via console→MQ→bridge
→server had empty user_id in usage events. Now the console extracts
user_id from auth_result and passes it through the MQ message. The
bridge forwards it in the HTTP payload, and the server accepts it
from trusted service callers (bridge, console-proxy).
2026-03-24 03:13:23 -07:00
Patrick Buckley 037308f3b1 fix: propagate user identity through console proxy
Console proxy previously used a fixed service identity (console-proxy)
with full {read,write,approve} scopes for all proxied requests, losing
the real user's identity at the proxy boundary. Now mints per-request
short-lived JWTs carrying the authenticated user's actual user_id,
scopes, and permissions so upstream servers record correct audit
attribution and enforce scope narrowing as defense in depth.
2026-03-24 03:13:23 -07:00
Patrick Buckley d7a9895855 feat: tool description optimization + OOM and logging fixes (#172)
* feat: tool description optimization + OOM and logging fixes

Tool description optimization (three-phase pipeline):
- Tool optimizer (phase 2) modifies tool descriptions to resolve
  confusion, gated by --optimize-tools and wrong_tool detection
- _apply_tool_overrides deep-copies modified tools, never mutates TOOLS
- _propose_tool_overrides validates JSON, deep-merges parameter overrides
- tool_overrides on EvolutionNode, plumbed through full eval pipeline
- --save-tools writes best overrides back to turnstone/tools/*.json
- Prompt optimizer informed when tool descriptions have been modified
- TSV tool_changes column

OOM fix (session lifecycle):
- HeadlessSession created inside retry loop, not outside — timed-out
  orphan threads no longer pin old sessions in memory
- Session ref cleared immediately after extracting results
- Previous behavior leaked unbounded memory per timeout (~515GB OOM)

Logging fix:
- Removed _suppress_stdout entirely — redirected fd 1 process-wide,
  causing main thread print() to vanish during slow API calls
- NullUI already discards session output; tools return strings

New CLI: --optimize-tools, --tool-optimizer-model/base-url, --save-tools

* review: address copilot feedback on tool optimization
2026-03-23 22:47:06 -07:00
Patrick Buckley 0ba49b8bb7 review: address copilot feedback on eval pipeline
- Record prompt_variant in parallel execution path (was serial-only)
- Handle "Subprocess error:" and "Skipped (fast-fail)" in failure
  classifier instead of mis-bucketing as missing_tool
- Build case_id->case_def dict once in _build_failure_analysis,
  _run_analyst, _propose_prompt_modification (was O(n²) linear scan)
- Enforce original prompt at slot 0 of cached user_prompts variants
- Use wall clock time for TSV elapsed_s instead of sum of run times
2026-03-23 19:44:32 -07:00
Patrick Buckley 51b5b3ee74 feat: multi-agent eval pipeline with tree search, analyst, diversifier
UCB tree search for prompt optimization (arXiv:2603.18620):
- EvolutionNode dataclass, UCB1 selection, rolling mean scores
- Replaces fragile linear chain with backtracking via tree
- Holdout set separation prevents optimizer overfitting
- Improvement-based delta feedback to optimizer

Multi-agent optimization pipeline:
- Analyst agent (phase 1): multi-turn with math/bash tools, identifies
  semantic failure patterns, computes statistics across test results
- Optimizer (phase 2): uses analyst diagnosis to edit developer prompt
- Observer: tunes optimizer strategy every 3 iterations
- Diversifier: generates paraphrased prompt variants for phrasing
  robustness, with dedup, delta generation, and JSON caching

Failure classification:
- 8 failure mode buckets (no_tool_call, wrong_tool, missing_tool,
  wrong_args, extra_tools, timeout, error, json_dump)
- Consistency signals (systematic, flaky, marginal)
- Rule-based pre-analysis feeds into analyst as structured input

Logging and observability:
- Config summary at startup (models, case count, runs)
- Per-case diversifier progress with dedup stats
- UCB selection reasoning, node score updates, tree growth
- Extended TSV: node_score, elapsed_s, prompt_len, iter_tokens,
  cumul_tokens columns plus 4-decimal precision

Infrastructure:
- Thread-safe fd-level stdout suppression (os.dup2)
- Prompt variants plumbed through parallel execution path
- Cached variants auto-detected from tests.json user_prompts field

New CLI flags: --explore-constant, --analyst-model/base-url,
--diversifier-model/base-url, --diversify N, --save-variants
2026-03-23 19:44:32 -07:00
Patrick Buckley c3b0ddeba7 fix: add ddgs to mypy ignore_missing_imports for CI compat 2026-03-23 19:11:50 -07:00
Patrick Buckley a533e1c783 fix: use fd-level stdout redirect in eval to avoid thread race
_suppress_stdout() was setting sys.stdout = StringIO() which is
process-global. When send_headless runs in a ThreadPoolExecutor and
blocks on an API call inside the suppression context, the main
thread's print() calls silently go to the StringIO and vanish.

Switch to os.dup2 fd-level redirect which is thread-safe.
2026-03-23 18:15:45 -07:00
Patrick Buckley d8bc78556f bump: v0.8.7 2026-03-23 17:27:23 -07:00
Patrick Buckley 4f5854e768 fix: remove stale type: ignore on ddgs import 2026-03-23 17:10:27 -07:00
Patrick Buckley bf2dc04cb3 feat: UCB tree search for eval prompt optimization
Replace linear optimization chain with UCB1 evolution tree. Each
iteration selects the most promising node to extend, preventing
irrecoverable collapse from bad edits. Also adds improvement-based
delta feedback to the optimizer and optional holdout set separation.

Inspired by "Learning to Self-Evolve" (arXiv:2603.18620).
2026-03-23 17:09:22 -07:00
Patrick Buckley bb894d073b fix: SQLite migrations use batch_alter_table for compat
Migrations 014, 021-024 used bare op.add_column/alter_column/drop_column
which fails on SQLite (no ALTER of constraints). Switch to
batch_alter_table and enable render_as_batch in env.py. Migration 014
also moved UniqueConstraint inline into create_table.
2026-03-23 17:09:13 -07:00
146 changed files with 6724 additions and 881 deletions
+1 -1
View File
@@ -67,7 +67,7 @@ jobs:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
- uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6
with:
python-version: "3.12"
python-version: "3.14"
- run: pip install -e ".[test,mq,postgres]"
- run: pytest tests/ -m "not live" --storage-backend=postgresql -q
env:
+1 -1
View File
@@ -19,7 +19,7 @@ jobs:
python-version: "3.14"
- run: pip install build
- run: python -m build
- uses: pypa/gh-action-pypi-publish@release/v1
- uses: pypa/gh-action-pypi-publish@ed0c53931b1dc9bd32cbe73a98c7f6766f8a527e # release/v1
- name: Create GitHub Release
uses: softprops/action-gh-release@153bb8e04406b158c6c84fc1615b65b24149a1fe # v2
+3 -3
View File
@@ -8,7 +8,7 @@ FROM python:3.14-slim
LABEL org.opencontainers.image.title="turnstone" \
org.opencontainers.image.description="Multi-node AI orchestration platform"
COPY --from=ghcr.io/astral-sh/uv:0.10.12 /uv /usr/local/bin/uv
COPY --from=ghcr.io/astral-sh/uv:0.11.2 /uv /usr/local/bin/uv
# System dependencies for psycopg (PostgreSQL client library)
RUN apt-get update && apt-get upgrade -y && apt-get install -y --no-install-recommends libpq5 \
@@ -25,12 +25,12 @@ ENV UV_COMPILE_BYTECODE=1
# Install dependencies first (cached layer — only re-runs when deps change)
COPY pyproject.toml uv.lock README.md LICENSE ./
RUN uv sync --frozen --no-install-project --no-dev \
--extra mq --extra console --extra sim --extra postgres --extra discord --extra anthropic --extra ddg
--extra all
# Install the project itself
COPY turnstone/ turnstone/
RUN uv sync --frozen --no-dev \
--extra mq --extra console --extra sim --extra postgres --extra discord --extra anthropic --extra ddg
--extra all
# Add venv to PATH so entry points are found
ENV PATH="/app/.venv/bin:$PATH"
+146
View File
@@ -0,0 +1,146 @@
# TLS overlay — enables mTLS across the turnstone cluster.
#
# Usage (requires base compose.yaml with production profile):
# docker compose -f compose.yaml -f deploy/docker-compose.tls.yml --profile production up
#
# The tls-init service bootstraps a CA and issues a cert for Redis.
# All turnstone services auto-provision their own certs via the
# console's ACME endpoint.
services:
# Bootstrap: create CA + Redis cert before anything starts.
# Runs as root to create directories in the volume, then chowns
# to turnstone:turnstone with restrictive perms (keys 0600).
tls-init:
build: .
user: root
command:
- sh
- -c
- |
set -e
turnstone-admin tls-bootstrap --out /certs --issue redis
chown -R turnstone:turnstone /certs
find /certs -type d -exec chmod 750 {} +
find /certs -type f -name '*key.pem' -exec chmod 600 {} +
find /certs -type f ! -name '*key.pem' -exec chmod 640 {} +
volumes:
- tls-certs:/certs
networks:
- turnstone-net
restart: "no"
# Console: runs the internal CA + ACME server
console:
depends_on:
tls-init:
condition: service_completed_successfully
volumes:
- tls-certs:/certs:ro
environment:
TURNSTONE_TLS_ENABLED: "true"
TURNSTONE_TLS_SANS: "console"
TURNSTONE_CONSOLE_URL: "http://console:8090"
command:
- turnstone-console
- --host=0.0.0.0
- --port=8090
- --redis-host=redis
- --redis-port=6379
- --poll-interval=${CONSOLE_POLL_INTERVAL:-10}
- --redis-tls
- --redis-tls-ca=/certs/ca.pem
# Server: auto-provisions certs via console ACME, serves HTTPS
server:
depends_on:
console:
condition: service_healthy
volumes:
- tls-certs:/certs:ro
environment:
TURNSTONE_TLS_ENABLED: "true"
TURNSTONE_TLS_SANS: "server"
# Disable healthcheck — server serves HTTPS with mTLS which the
# stdlib healthcheck script can't satisfy. The base compose
# healthcheck uses plain HTTP which won't work on an HTTPS listener.
# TODO: wire healthcheck with client cert from /certs volume
healthcheck:
disable: true
# Bridge: mTLS to server + Redis TLS
bridge:
depends_on:
console:
condition: service_healthy
server:
condition: service_started
redis:
condition: service_healthy
volumes:
- tls-certs:/certs:ro
environment:
TURNSTONE_TLS_ENABLED: "true"
TURNSTONE_TLS_SANS: "bridge"
command:
- turnstone-bridge
- --server-url=http://server:8080
- --redis-host=redis
- --redis-port=6379
- --heartbeat-ttl=${HEARTBEAT_TTL:-60}
- --approval-timeout=${APPROVAL_TIMEOUT:-3600}
- --redis-tls
- --redis-tls-ca=/certs/ca.pem
# Channel: Redis TLS
channel:
depends_on:
console:
condition: service_healthy
redis:
condition: service_healthy
volumes:
- tls-certs:/certs:ro
environment:
TURNSTONE_TLS_ENABLED: "true"
TURNSTONE_TLS_SANS: "channel"
command:
- sh
- -c
- >-
turnstone-channel
--redis-host=redis
--redis-port=6379
--redis-tls
--redis-tls-ca=/certs/ca.pem
--http-host=0.0.0.0
$${TURNSTONE_DISCORD_GUILD:+--discord-guild $$TURNSTONE_DISCORD_GUILD}
# Redis: TLS with certs from bootstrap
redis:
depends_on:
tls-init:
condition: service_completed_successfully
volumes:
- tls-certs:/certs:ro
command:
- sh
- -c
- |
ARGS="--tls-port 6379 --port 0 \
--tls-cert-file /certs/certs/redis/cert.pem \
--tls-key-file /certs/certs/redis/key.pem \
--tls-ca-cert-file /certs/ca.pem \
--tls-auth-clients no"
if [ -n "$$REDIS_PASSWORD" ]; then
ARGS="$$ARGS --requirepass $$REDIS_PASSWORD"
fi
exec redis-server $$ARGS
healthcheck:
test: ["CMD-SHELL", "if [ -n \"$$REDIS_PASSWORD\" ]; then redis-cli --tls --cacert /certs/ca.pem -a $$REDIS_PASSWORD ping; else redis-cli --tls --cacert /certs/ca.pem ping; fi"]
interval: 5s
timeout: 3s
retries: 5
volumes:
tls-certs:
+1 -1
View File
@@ -91,7 +91,7 @@ turnstone/
_config.py Base ChannelConfig dataclass
discord/ Discord adapter (bot, cog, views, streaming, config)
shared_static/ Shared design system (base.css, auth.js, theme.js, toast.js, utils.js, kb.js)
katex-0.16.40/ Vendored KaTeX math rendering library (MIT, woff2 fonts)
katex-0.16.43/ Vendored KaTeX math rendering library (MIT, woff2 fonts)
ui/
colors.py ANSI color constants with NO_COLOR support
markdown.py Streaming terminal markdown renderer (line-buffered)
+1 -1
View File
@@ -365,7 +365,7 @@ SSE streams (`/v1/api/events`, `/v1/api/events/global`) are proxied as raw byte
### Authentication
The proxy forwards the user's JWT to upstream server nodes — it extracts the token from the incoming request's cookie (or `Authorization` header) and adds it as a `Bearer` header on the proxied request. Since all services share the same `TURNSTONE_JWT_SECRET`, the user's JWT is valid on every node without re-authentication. The console's own auth middleware also checks proxy routes — `POST` requests to proxy write endpoints (`/v1/api/send`, `/v1/api/approve`, etc.) require `write` scope, preventing read-only tokens from escalating via proxy. The static `--auth-token` / `proxy_auth_token` is used as a fallback when no user JWT is present.
The proxy mints a short-lived (5-minute) JWT per request carrying the real user's `user_id`, `scopes`, and `permissions` with `aud: turnstone-server`. The user's console JWT (`aud: turnstone-console`) cannot be forwarded directly — it would be rejected by the server's audience validation — so the console re-signs a new server-audience JWT from the validated `AuthResult`. This preserves audit attribution (the upstream server sees the real user, not a service identity) and enforces scope narrowing as defense in depth (a read-only console user's proxied request carries only `read` scope). The JWT `src` claim is set to `"console-proxy"` for audit traceability. When no user context is available (auth disabled), the proxy falls back to a `ServiceTokenManager` with service identity `console-proxy`. The static `--auth-token` / `proxy_auth_token` is used as a final fallback.
---
+1
View File
@@ -61,6 +61,7 @@ package "Inbound Messages (Client → Bridge)" as InPkg #FFF3E0 {
+ target_node: str = ""
+ initial_message: str = ""
+ skill: str = ""
+ user_id: str = ""
}
class CloseWorkstreamMessage {
+3 -2
View File
@@ -154,7 +154,7 @@ activate Server #FFECB3
Server -> CC : _pick_best_node() or\nget_node_detail(node_id)
CC --> Server : node validated
Server -> Server : Build CreateWorkstreamMessage\n{target_node:"nodeA", name:"new-task"}
Server -> Server : Build CreateWorkstreamMessage\n{target_node:"nodeA", name:"new-task",\nuser_id: from auth_result}
Server -> Redis : RPUSH turnstone:inbound:nodeA\n(directed queue)
Server --> Browser : {status:"ok", correlation_id:"abc",\ntarget_node:"nodeA"}
@@ -163,7 +163,8 @@ deactivate Server
note right of Redis
Bridge on Node-A picks up the
message from its directed queue,
POSTs to /v1/api/workstreams/new,
POSTs to /v1/api/workstreams/new
(forwarding user_id in payload),
registers ownership, publishes
ws_created to cluster channel.
end note
+11
View File
@@ -176,4 +176,15 @@ note bottom of SH
Both share JWT signing secret
end note
note left of JWT
**Console Proxy Token Minting**
When proxying requests to server nodes:
1. Console AuthMiddleware validates user JWT (aud: turnstone-console)
2. Proxy mints new JWT (aud: turnstone-server)
with real user_id, scopes, permissions
3. src: "console-proxy" for audit traceability
4. 5-minute expiry (fresh per request)
5. Fallback: ServiceTokenManager if no user context
end note
@enduml
+254 -68
View File
@@ -2,7 +2,8 @@
`turnstone-eval` is the evaluation and prompt optimization system for turnstone. It
runs test cases against the LLM, scores tool call sequences against expected
actions, and optionally uses the model to self-optimize the developer prompt.
actions, and optionally uses a multi-agent pipeline to optimize the developer
prompt and tool descriptions.
Source: `turnstone/eval.py`
@@ -10,15 +11,24 @@ Source: `turnstone/eval.py`
## Overview
The system works in an iterative loop:
The system uses UCB tree search to explore prompt variants:
1. Run each test case N times against the current developer prompt.
2. Score each run by comparing the actual tool call sequence to expected actions.
3. If not all tests pass, use the model to rewrite the prompt based on failures.
4. Repeat until all tests pass or max iterations are reached.
1. Maintain an **evolution tree** of prompt variants, starting from the initial prompt.
2. Each iteration, **UCB1 selects** the most promising node to evaluate.
3. Run each test case N times against the selected prompt.
4. Score each run by comparing the actual tool call sequence to expected actions.
5. If not all tests pass, run a **three-phase optimization pipeline**:
- Phase 1: Analyst diagnoses semantic failure patterns
- Phase 2: Tool optimizer adjusts tool descriptions (when `--optimize-tools`)
- Phase 3: Prompt optimizer proposes a child variant (when not `--optimize-tools`)
6. Add the child to the tree and repeat until all tests pass or max iterations reached.
When optimization is disabled (`--no-optimize`), only step 1 and 2 execute
(a single iteration).
This approach (inspired by [Learning to Self-Evolve](https://arxiv.org/abs/2603.18620))
prevents irrecoverable collapse from bad edits — UCB naturally backtracks to
high-scoring ancestors instead of following a linear chain.
When optimization is disabled (`--no-optimize`), only steps 2-4 execute
(a single iteration evaluating the root node).
---
@@ -65,6 +75,7 @@ Test suites are JSON files with this structure:
| `match_mode` | no | `"ordered_subset"` | How to match actual vs expected actions (see Scoring). |
| `max_turns` | no | `10` | Maximum conversation turns before stopping. |
| `n_runs` | no | suite default or 3 | Per-case override for number of runs. |
| `holdout` | no | `false` | If `true`, this case is evaluated but excluded from optimizer feedback. Used to measure progress without overfitting. |
### Expected Action Specs
@@ -135,6 +146,7 @@ deterministic, non-interactive execution suitable for automated testing.
| Stdout | Normal | Suppressed during execution |
| Tool logging | Display only | Structured `tool_call_log` |
| System prompt | Built-in developer prompt | Overridable via constructor |
| Cancellation | N/A | `_cancelled` event for timeout cleanup |
### NullUI
@@ -156,20 +168,34 @@ def send_headless(
Runs a complete multi-turn conversation:
1. Appends the user message.
2. Calls the model API (non-streaming).
3. If tool calls are returned, executes them (with stdout suppressed) and
2. Checks `_cancelled` event — stops if set (timeout cleanup).
3. Calls the model API (non-streaming).
4. If tool calls are returned, executes them (with stdout suppressed) and
logs each call to `self.tool_call_log`.
4. Repeats up to `max_turns` or until the model responds without tool calls.
5. Returns the tool call log: list of dicts with keys `tool`, `args`,
5. Repeats up to `max_turns` or until the model responds without tool calls.
6. Returns the tool call log: list of dicts with keys `tool`, `args`,
`result` (truncated to 500 chars), and `turn`.
Parallel tool calls are capped at 10 per turn to prevent degenerate repetition.
### Timeout and Cancellation
Each test runs in a `ThreadPoolExecutor(max_workers=1)` with a per-test
timeout (`--test-timeout`). Each attempt gets its own `OpenAI` client with
a matching httpx read timeout. On timeout, three layers of defense prevent
zombie connections:
1. **httpx timeout**: Per-request read timeout aborts the HTTP call and
releases the server slot.
2. **`_cancelled` event**: Prevents the orphan thread from starting new turns.
3. **`run_client.close()`**: Closes the connection pool to abort any
in-flight request.
### Retry Logic
`send_headless()` is called inside `_run_single_test()` with retry logic:
3 attempts with exponential backoff (sleep `2^attempt` seconds) on any
exception. This prevents transient API errors from poisoning eval scores.
exception. `TimeoutError` is re-raised immediately (no retry).
---
@@ -180,68 +206,175 @@ Each test case runs in isolation:
1. A fresh temp directory is created.
2. Setup files are written to the temp directory.
3. The working directory is changed to the temp directory.
4. A new `HeadlessSession` is created with the current developer prompt.
5. `send_headless()` runs the user prompt through the conversation loop.
6. The tool log is scored against expected actions.
7. The temp directory is cleaned up.
4. A per-attempt `OpenAI` client is created with httpx timeout matching `--test-timeout`.
5. A new `HeadlessSession` is created with the current developer prompt.
6. `send_headless()` runs the user prompt through the conversation loop.
7. The tool log is scored against expected actions.
8. The temp directory is cleaned up.
The memory database is also isolated per test (an ephemeral SQLite database
in the temp directory) so tests do not pollute each other or the user's
real memory store.
### Parallel Execution
With `--parallel N` (N > 1), tests run in a `ProcessPoolExecutor` with N
workers. Each subprocess creates its own `OpenAI` client. This is suitable
for remote API endpoints but will overwhelm local inference servers. The
default (`--parallel 1`) runs tests serially.
---
## Optimization Loop
## Model Roles
`run_optimization()` is the main entry point for iterative prompt optimization.
The eval pipeline uses up to five separate model roles, each independently
configurable. All roles inherit from the test model by default, with a
cascade chain:
```
test model (--base-url, --model)
└─ optimizer (--optimizer-*)
├─ observer (--observer-*)
├─ analyst (--analyst-*)
├─ diversifier (--diversifier-*)
└─ tool optimizer (--tool-optimizer-*)
```
| Role | Purpose | When it runs |
|------|---------|--------------|
| **Test** | The model being evaluated | Every iteration |
| **Analyst** | Diagnoses semantic failure patterns with tool use | When pass rate < 100% |
| **Optimizer** | Rewrites the developer prompt | Every iteration (unless `--optimize-tools`) |
| **Tool optimizer** | Rewrites tool descriptions | When `--optimize-tools` is set |
| **Observer** | Tunes the optimizer's strategy | Every 3 iterations |
| **Diversifier** | Generates prompt paraphrases | Once before the loop (when `--diversify N`) |
Typical setup: local model for test, Opus for analyst, Sonnet for
optimizer/observer/diversifier.
---
## Optimization Pipeline
### Flow
```
for iteration in 0..max_iterations:
1. Run all test cases n_runs times with current prompt
2. Score and aggregate results
3. Save intermediate results to JSON
4. If all tests pass -> stop
5. Every 3 iterations (at iteration 2, 5, 8, ...):
-> Observer reviews optimizer strategy
-> Reset prompt to best-performing iteration
6. Propose new prompt via optimizer model call
7. If prompt unchanged -> stop
8. Continue with new prompt
1. UCB select → pick the most promising tree node
2. Run all test cases n_runs times with selected node's prompt
3. Update node score (rolling mean) and visit count
4. Save intermediate results + tree state to JSON
5. If all tests pass → stop
6. Phase 1: Analyst diagnoses semantic failure patterns
7. Phase 2 (--optimize-tools only): Tool optimizer adjusts descriptions
8. Phase 3 (default only): Prompt optimizer proposes new prompt
9. Every 3 iterations: Observer tunes the optimizer's strategy
10. Add child node to tree (if prompt or tools changed)
```
### Prompt Proposal (`_propose_prompt_modification`)
### Phase 1: Analyst (`_run_analyst`)
Uses the model to rewrite the developer prompt based on test results:
A multi-turn agent with `math` (Python) and `bash` tools for computing
statistics. It receives per-case results with failure classifications and
produces a structured diagnosis:
- **Input**: Current prompt, test case definitions, per-case results with
actual vs expected tool sequences, and a history of the last 3 iterations.
- **Optimizer system prompt** (`OPTIMIZER_SYSTEM`): Instructs the model to
act as a text rewriter. Key guidance includes:
- Address critical failure modes (text-only responses, write_file vs edit_file,
unnecessary search before create, missing plan calls).
- Preserve phrasing that drives 100% pass rate on passing tests.
- Use direct imperative style with concrete tool call examples.
- Stay within 130% of original prompt length.
- **Output**: The rewritten prompt text (stripped of reasoning tags and code fences).
- **Failure patterns**: Shared root causes across failing cases
- **Success/failure contrast**: What distinguishes passing from failing cases
- **Consistency signals**: Systematic (0%), flaky (1-79%), marginal (80-99%)
- **Recommended fixes**: Priority-ordered patterns/examples to add or adjust
### Observer System (`_observe_and_update_optimizer`)
The analyst is instructed to frame fixes as patterns and examples, not
imperative rules — this feeds cleaner signal to the optimizer.
Every 3 iterations, a meta-level "observer" reviews the optimizer's strategy:
In `--optimize-tools` mode, the analyst receives the current tool descriptions
(with any overrides applied) and focuses on tool confusion and description
issues rather than system prompt patterns.
- Analyzes the iteration history: score trends, regressions, prompt length changes,
### Phase 2: Tool Optimizer (`_propose_tool_overrides`)
Runs when `--optimize-tools` is set. Receives the current tool descriptions,
confusion failures (where the model picked the wrong tool), and the analyst's
diagnosis. Returns a JSON override dict that modifies tool descriptions.
Overrides are validated against known tool names — only `description` and
`parameters` changes are accepted (no tool renaming at eval time).
After each iteration, changed descriptions are logged as old → new diffs
for easy visual inspection.
### Phase 3: Prompt Optimizer (`_propose_prompt_modification`)
Skipped in `--optimize-tools` mode. Receives the current prompt, test
results with per-case pass rates and deltas from the parent node, and the
analyst's diagnosis. Returns a rewritten prompt.
The optimizer is instructed to prefer patterns over rules — concrete tool
chain examples teach better than imperative directives like "ALWAYS" or
"NEVER." If the current prompt contains rule-heavy language, the optimizer
is guided to replace it with examples.
### Two Optimization Surfaces
The system supports alternating between two optimization surfaces:
1. **System prompt optimization** (default): Freeze tool descriptions,
optimize the developer prompt. Run until scores plateau.
2. **Tool description optimization** (`--optimize-tools`): Freeze the system
prompt, optimize tool descriptions only. Run until scores plateau.
Each surface lifts the floor for the other — tool description improvements
may unlock system prompt gains that weren't reachable before, and vice versa.
### Observer (`_observe_and_update_optimizer`)
Every 3 iterations, a meta-level observer reviews the optimizer's strategy:
- Analyzes iteration history: score trends, regressions, prompt length changes,
and diffs between iterations.
- Summarizes the optimizer's behavioral patterns (list style, header usage, length).
- Uses `OBSERVER_SYSTEM` to rewrite the optimizer's own system prompt.
- Detects whether the optimizer is producing rule-heavy or pattern-based output.
- Rewrites the optimizer's own system prompt to correct course.
- Rejects degenerate outputs (over 200% of input length).
- After updating the optimizer prompt, resets the developer prompt to the
best-performing iteration so far.
This two-level optimization (optimizer + observer) helps the system escape
local minima and adjust its rewriting strategy.
### Prompt Diversification
### Result Persistence
When `--diversify N` is set, the diversifier generates N paraphrased variants
of each test case's user prompt before the optimization loop. Each run cycles
through variants (round-robin), testing robustness across phrasings.
Variants can be cached back to the test suite JSON with `--save-variants`,
and auto-loaded on subsequent runs even without `--diversify`.
---
## Evolution Tree
The optimization maintains a tree of prompt variants (`EvolutionNode`), where
each node stores its prompt text, tool overrides, aggregated score, and visit
count. The root node (ID 0) contains the initial prompt.
**UCB1 selection**: Each iteration picks the node with the highest Upper
Confidence Bound score: `R_bar + C * sqrt(ln(N) / v)`, where `R_bar` is the
node's mean score, `N` is total visits across all nodes, `v` is the node's
visit count, and `C` is the exploration constant (`--explore-constant`,
default sqrt(2)). Unvisited nodes are always selected first.
### Holdout Cases
Test cases with `"holdout": true` are evaluated every iteration but excluded
from the optimizer's feedback. This prevents the optimizer from overfitting
to specific test cases. Node scores are computed from holdout cases only
(when present). If fewer than 2 non-holdout cases remain, holdout is disabled.
### Improvement-Based Feedback
The optimizer sees delta scores (`delta=+20%`) alongside absolute pass rates,
showing how each case improved relative to the parent node's evaluation. This
provides a cleaner signal than absolute scores alone — the optimizer can
distinguish beneficial edits from harmful ones regardless of starting point.
---
## Result Persistence
After each iteration, results are written to the output JSON file. The
structure is:
@@ -251,9 +384,15 @@ structure is:
"meta": {
"model": "model-name",
"base_url": "http://localhost:8000/v1",
"optimizer_model": "claude-opus-4-6",
"observer_model": "claude-opus-4-6",
"started": "2025-01-01T00:00:00",
"test_suite": "tests.json",
"n_runs_default": 3
"n_runs_default": 3,
"explore_constant": 1.414,
"holdout_ids": [],
"diversify": 10,
"prompt_variants": {"case_id": ["variant1", "variant2"]}
},
"iterations": [
{
@@ -261,7 +400,11 @@ structure is:
"prompt": "the developer prompt used",
"prompt_diff": null,
"optimizer_system": "the optimizer system prompt",
"analyst": "analyst diagnosis output",
"tool_overrides": {"bash": {"description": "..."}},
"timestamp": "2025-01-01T00:01:00",
"tree_node_id": 0,
"tree_child_id": 1,
"cases": {
"test_name": {
"runs": [
@@ -287,9 +430,21 @@ structure is:
"overall_pass_rate": 0.8,
"overall_avg_score": 0.87,
"json_dumps": 0,
"per_case_pass_rates": {"test_name": 1.0, ...}
"per_case_pass_rates": {"test_name": 1.0}
}
}
],
"tree": [
{
"node_id": 0,
"parent_id": null,
"prompt": "initial prompt",
"tool_overrides": {},
"score": 0.85,
"visit_count": 3,
"children": [1, 2],
"iteration": 0
}
]
}
```
@@ -306,26 +461,57 @@ turnstone-eval tests.json # evaluate + optimize
turnstone-eval tests.json --no-optimize # evaluate only (single iteration)
turnstone-eval tests.json --n-runs 5 --max-iter 10 # more thorough evaluation
turnstone-eval tests.json --prompt custom.txt # start from a custom prompt
turnstone-eval tests.json --optimize-tools # optimize tool descriptions only
turnstone-eval tests.json --diversify 10 # test with prompt variants
turnstone-eval tests.json -v # verbose per-turn logging
```
### Multi-model setup (local test model, cloud optimizer)
```
turnstone-eval tests.json \
--base-url http://localhost:8000/v1 \
--optimizer-base-url https://api.anthropic.com \
--optimizer-model claude-sonnet-4-6 \
--analyst-model claude-opus-4-6
```
### All Options
| Flag | Default | Description |
|---------------------|-------------------------------|-------------|
| `test_file` | (positional, required) | Path to test cases JSON file. |
| `--base-url` | `http://localhost:8000/v1` | API base URL. |
| `--model` | auto-detect | Model name. Auto-detected from the API if not specified. |
| `--prompt` | turnstone built-in prompt | Path to initial prompt text file. |
| `--n-runs` | from tests.json or 3 | Number of runs per test case. |
| `--max-iter` | 5 | Maximum optimization iterations. |
| `--no-optimize` | false | Run evaluation only (sets max-iter to 1). |
| `--temperature` | 0.7 | Sampling temperature. |
| `--max-tokens` | 32768 | Max completion tokens. |
| `--reasoning-effort` | `medium` | Reasoning effort: `low`, `medium`, or `high`. |
| `--context-window` | 131072 | Context window size. |
| `--output` | `eval_results.json` | Output results file path. |
| `-v`, `--verbose` | false | Show detailed per-turn logging (API calls, tool args, results). |
| Flag | Default | Description |
|-------------------------|----------------------------|-------------|
| `test_file` | (positional, required) | Path to test cases JSON file. |
| `--base-url` | `http://localhost:8000/v1` | API base URL for the test model. |
| `--model` | auto-detect | Model name. Auto-detected from the API if not specified. |
| `--prompt` | turnstone built-in prompt | Path to initial prompt text file. |
| `--n-runs` | from tests.json or 3 | Number of runs per test case. |
| `--max-iter` | 5 | Maximum optimization iterations. |
| `--no-optimize` | false | Run evaluation only (sets max-iter to 1). |
| `--temperature` | 0.7 | Sampling temperature. |
| `--max-tokens` | 32768 | Max completion tokens. |
| `--reasoning-effort` | `medium` | Reasoning effort: `low`, `medium`, or `high`. |
| `--context-window` | 131072 | Context window size. |
| `--output` | `eval_results.json` | Output results file path. |
| `-v`, `--verbose` | false | Show detailed per-turn logging. |
| `--explore-constant` | 1.414 (sqrt(2)) | UCB exploration constant C. |
| `--test-timeout` | 300 | Per-test timeout in seconds. |
| `--suite-timeout` | 0 (unlimited) | Total suite timeout in seconds. |
| `--no-fast-fail` | false | Disable early termination on all-zero initial runs. |
| `--parallel` | 1 (serial) | Parallel workers (0=auto, N=use N workers). |
| `--optimizer-model` | same as `--model` | Model for prompt optimization. |
| `--optimizer-base-url` | same as `--base-url` | Base URL for optimizer model. |
| `--observer-model` | same as optimizer | Model for meta-optimization (observer). |
| `--observer-base-url` | same as optimizer | Base URL for observer model. |
| `--analyst-model` | same as optimizer | Model for failure analysis. |
| `--analyst-base-url` | same as optimizer | Base URL for analyst model. |
| `--diversify` | 0 (disabled) | Generate N prompt variants per test case. |
| `--diversifier-model` | same as optimizer | Model for prompt diversification. |
| `--diversifier-base-url`| same as optimizer | Base URL for diversifier model. |
| `--save-variants` | false | Save generated variants back to test suite JSON. |
| `--optimize-tools` | false | Optimize tool descriptions only (freeze system prompt). |
| `--tool-optimizer-model` | same as optimizer | Model for tool description optimization. |
| `--tool-optimizer-base-url` | same as optimizer | Base URL for tool optimizer model. |
| `--save-tools` | false | Write optimized tool descriptions back to `turnstone/tools/*.json`. |
### Precedence for n_runs
+35 -8
View File
@@ -457,14 +457,30 @@ without any database.
### Proxy auth forwarding
When the console proxies requests to server nodes (via `/node/{id}/...`
routes), it uses a dedicated **service proxy token** with
`aud: turnstone-server` and `write` scope. The user's console JWT
(which has `aud: turnstone-console`) is **not** forwarded — it would be
rejected by the server's audience validation.
routes), it mints a **short-lived user-scoped JWT** with
`aud: turnstone-server` carrying the real user's `user_id`, `scopes`,
and `permissions`. The user's console JWT (which has
`aud: turnstone-console`) is **not** forwarded directly — it would be
rejected by the server's audience validation. Instead, the console
re-signs a new JWT targeted at the server audience.
The proxy token is managed by a `ServiceTokenManager` that auto-rotates
1-hour JWTs, refreshing at 80% of lifetime. If `--auth-token` is
provided, that static token is used instead.
Each proxied request gets a fresh JWT (5-minute expiry). This ensures:
- **Audit attribution** — the upstream server records the real user in
`ctx_user_id` and audit events, not a generic service identity.
- **Scope narrowing** — a read-only console user's proxied request
carries only `read` scope, not the full `{read, write, approve}` set.
The server enforces this as defense in depth.
- **Permission forwarding** — granular RBAC permissions from the
console JWT are carried through to the server.
The JWT `src` claim is set to `"console-proxy"`, allowing servers to
distinguish proxied requests from direct logins in audit logs.
When no user context is available (auth disabled, or internal requests),
the proxy falls back to a `ServiceTokenManager` with service identity
`console-proxy` and full scopes. If `--auth-token` is provided, that
static token is used as a final fallback.
### Service-to-service authentication
@@ -475,7 +491,7 @@ auto-rotating JWTs when communicating with server nodes:
|---------|----------|-------|----------|---------|
| Bridge | `bridge` | `approve` | `turnstone-server` | Tool approval proxy, message relay |
| Console collector | `console-collector` | `read` | `turnstone-server` | Node health polling |
| Console proxy | `console-proxy` | `write` | `turnstone-server` | Proxied API calls |
| Console proxy (fallback) | `console-proxy` | `approve` | `turnstone-server` | Proxied API calls when no user context |
| Channel notify | `system` | `write` | `turnstone-channel` | Notification delivery to channel gateway |
Service tokens use 1-hour expiry with automatic refresh via
@@ -483,6 +499,17 @@ Service tokens use 1-hour expiry with automatic refresh via
httpx event hooks to ensure rotated tokens are picked up on SSE
reconnects.
### User identity in MQ-dispatched workstreams
When the console creates a workstream via MQ (the normal path), the
authenticated user's `user_id` is embedded in the
`CreateWorkstreamMessage`. The bridge forwards this `user_id` in the
HTTP payload when calling the server's `POST /v1/api/workstreams/new`.
The server accepts a `user_id` from the request body **only when the
caller is a trusted service** — identified by `token_source` matching
`bridge`, `console-proxy`, or `console`. Regular API callers cannot
override `user_id`; the server always uses their JWT identity.
Note that the channel gateway uses a distinct JWT audience
(`turnstone-channel`) from the server (`turnstone-server`) and console
(`turnstone-console`). A server-scoped JWT cannot authenticate to the
+221
View File
@@ -0,0 +1,221 @@
# TLS / mTLS
Turnstone supports end-to-end transport encryption with mutual TLS (mTLS) for
inter-service communication, powered by [lacme](https://pypi.org/project/lacme/).
---
## Quick Start (Docker Compose)
```bash
docker compose -f compose.yaml -f deploy/docker-compose.tls.yml up
```
This:
1. Bootstraps an internal CA and issues certs for Redis/PostgreSQL
2. Starts the console with TLS enabled (internal CA + ACME server)
3. Server nodes auto-provision certs via the console's ACME endpoint
4. All inter-service communication uses mTLS
---
## Architecture
```
Console (CA + ACME Server)
+-- CertificateAuthority (owns root key, signs certs)
+-- ACMEResponder (mounted at /acme, RFC 8555)
+-- GET /acme/ca.pem (root cert for node bootstrapping)
|
| ACME protocol (auto-approve, no challenge validation)
+-----------+-----------+
| | |
Server(s) Bridge Channel GW
(auto-cert (mTLS (mTLS
+ renewal) client) client)
```
**Two cert paths on the console:**
- **Internal cert** (mTLS): Always from the internal CA. Used for cluster
service mesh communication.
- **Frontend cert** (HTTPS): From an external ACME CA (e.g. Let's Encrypt)
if `tls.acme_directory` is set, otherwise self-issued from the internal CA.
---
## Configuration
### Settings (ConfigStore / Admin Settings tab)
| Setting | Default | Description |
|---------|---------|-------------|
| `tls.enabled` | `false` | Master switch for internal mTLS |
| `tls.acme_directory` | `""` | External ACME CA URL for console frontend cert |
### Bootstrap Config (config.toml)
These are needed before storage is available:
```toml
[redis]
tls = false
tls_ca = "" # path to CA cert
tls_cert = "" # path to client cert
tls_key = "" # path to client key
[database]
sslmode = "prefer" # disable, allow, prefer, require, verify-full
sslrootcert = "" # path to CA cert
sslcert = "" # path to client cert
sslkey = "" # path to client key
```
### Hardcoded Defaults
| Parameter | Value | Notes |
|-----------|-------|-------|
| CA common name | "Turnstone CA" | |
| CA validity | 10 years | |
| Cert validity | 48 hours | Short-lived, auto-renewed |
| Renewal interval | 24 hours | Half of validity |
| ACME auto-approve | true | Internal network, no challenge validation |
---
## CLI
### Offline Bootstrap
Create a CA and infrastructure certs without a running console:
```bash
# Bootstrap CA + Redis + PostgreSQL certs
turnstone-admin tls-bootstrap --out /certs --issue redis --issue postgres
# Output:
# /certs/ca.pem (CA root certificate)
# /certs/certs/redis/ (Redis cert + key)
# /certs/certs/postgres/ (PostgreSQL cert + key)
```
The output directory is chmod 0700 (contains the CA private key).
### Online Cert Issuance
Request certs from a running console's ACME endpoint:
```bash
# Download CA root cert (TOFU — verify fingerprint)
turnstone-admin tls-ca-cert --out ca.pem --console-url http://console:8080
# Request a cert for a domain
turnstone-admin tls-issue worker-1.internal --out /certs --console-url http://console:8080
# List issued certs
turnstone-admin tls-list --console-url http://console:8080 --auth-token $TOKEN
```
### Console URL Discovery
If `--console-url` is not provided, the CLI discovers it from the `services`
table in the shared database. The console registers itself on startup.
---
## Admin UI
The **TLS** tab in the console admin panel (System group) shows:
- CA status (common name, certificate count)
- Certificate table (domain, SANs, issued, expires)
- Force-renew and delete actions per certificate
---
## SDK
### Python
```python
from turnstone.sdk import TurnstoneServer
client = TurnstoneServer(
base_url="https://server:8080",
token="tok_xxx",
ca_cert="/path/to/ca.pem",
client_cert="/path/to/cert.pem",
client_key="/path/to/key.pem",
)
```
### TypeScript
```typescript
import { TurnstoneServer } from "@turnstone/sdk";
import { Agent } from "undici";
import * as fs from "fs";
const agent = new Agent({
connect: {
ca: fs.readFileSync("/path/to/ca.pem"),
cert: fs.readFileSync("/path/to/cert.pem"),
key: fs.readFileSync("/path/to/key.pem"),
},
});
const client = new TurnstoneServer({
baseUrl: "https://server:8080",
token: "tok_xxx",
// Node.js 18+ uses undici under the hood
fetch: (url, init) =>
fetch(url, { ...init, dispatcher: agent } as RequestInit),
});
```
---
## How It Works
### Node Bootstrap Flow
1. Node starts, connects to shared database (plain connection)
2. Discovers console URL from `services` table
3. Fetches CA root cert from `http://console/acme/ca.pem` (plain HTTP, TOFU)
4. Requests service cert via ACME protocol (plain HTTP, JWS-signed)
5. Starts auto-renewal (24h interval, re-issues before expiry)
6. All subsequent inter-service communication uses mTLS
### Console Startup Flow
1. Read `tls.enabled` from ConfigStore
2. Initialize CA (load from DB or generate new root key)
3. Mount ACME responder at `/acme` (serves `/ca.pem` natively)
4. Issue console certs (internal + optional frontend)
5. Start CA-direct auto-renewal (no network, signs directly)
6. Register console URL in services table with heartbeat
---
## Troubleshooting
### Cert expired / mTLS connection refused
Certs are valid for 48 hours. If auto-renewal stopped (e.g. console was down),
restart the service to re-request a cert.
### "No console service found"
The console registers itself in the `services` table on startup. If the console
hasn't started or the registration expired (1 hour TTL), nodes can't discover
it. Use `--console-url` explicitly.
### Let's Encrypt for console frontend
Set `tls.acme_directory` to `https://acme-v02.api.letsencrypt.org/directory`
in the admin Settings tab. The console will request a publicly trusted cert
for its HTTPS endpoint. Internal mTLS still uses the private CA.
### Verifying the cert chain
```bash
openssl s_client -connect server:8080 -CAfile ca.pem
```
+12 -3
View File
@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
[project]
name = "turnstone"
version = "0.8.6"
version = "0.9.0"
description = "Multi-node AI orchestration platform with tool use, agent routing, and cluster simulation."
readme = "README.md"
license = "BUSL-1.1"
@@ -53,7 +53,8 @@ anthropic = ["anthropic>=0.39"]
postgres = ["psycopg[binary]>=3.2"]
ddg = ["ddgs>=9.0"]
discord = ["discord.py>=2.4", "redis>=7.2"]
all = ["turnstone[mq,console,sim,anthropic,postgres,discord,ddg]"]
tls = ["lacme>=1.0.4"]
all = ["turnstone[mq,console,sim,anthropic,postgres,discord,ddg,tls]"]
[project.scripts]
turnstone = "turnstone.cli:main"
@@ -78,7 +79,7 @@ include = [
"turnstone/console/static/*.js",
"turnstone/shared_static/*.css",
"turnstone/shared_static/*.js",
"turnstone/shared_static/katex-0.16.40/**/*",
"turnstone/shared_static/katex-0.16.43/**/*",
"turnstone/shared_static/hljs-11.11.1/**/*",
"turnstone/shared_static/mermaid-11.13.0/**/*",
"turnstone/sdk/py.typed",
@@ -165,6 +166,14 @@ ignore_missing_imports = true
module = ["frontmatter", "frontmatter.*"]
ignore_missing_imports = true
[[tool.mypy.overrides]]
module = ["ddgs", "ddgs.*"]
ignore_missing_imports = true
[[tool.mypy.overrides]]
module = ["lacme", "lacme.*"]
ignore_missing_imports = true
[[tool.mypy.overrides]]
module = ["turnstone.channels.discord.*"]
disallow_subclassing_any = false
+155 -125
View File
@@ -72,9 +72,9 @@
}
},
"node_modules/@oxc-project/types": {
"version": "0.120.0",
"resolved": "https://registry.npmjs.org/@oxc-project/types/-/types-0.120.0.tgz",
"integrity": "sha512-k1YNu55DuvAip/MGE1FTsIuU3FUCn6v/ujG9V7Nq5Df/kX2CWb13hhwD0lmJGMGqE+bE1MXvv9SZVnMzEXlWcg==",
"version": "0.122.0",
"resolved": "https://registry.npmjs.org/@oxc-project/types/-/types-0.122.0.tgz",
"integrity": "sha512-oLAl5kBpV4w69UtFZ9xqcmTi+GENWOcPF7FCrczTiBbmC0ibXxCwyvZGbO39rCVEuLGAZM84DH0pUIyyv/YJzA==",
"dev": true,
"license": "MIT",
"funding": {
@@ -82,9 +82,9 @@
}
},
"node_modules/@rolldown/binding-android-arm64": {
"version": "1.0.0-rc.10",
"resolved": "https://registry.npmjs.org/@rolldown/binding-android-arm64/-/binding-android-arm64-1.0.0-rc.10.tgz",
"integrity": "sha512-jOHxwXhxmFKuXztiu1ORieJeTbx5vrTkcOkkkn2d35726+iwhrY1w/+nYY/AGgF12thg33qC3R1LMBF5tHTZHg==",
"version": "1.0.0-rc.12",
"resolved": "https://registry.npmjs.org/@rolldown/binding-android-arm64/-/binding-android-arm64-1.0.0-rc.12.tgz",
"integrity": "sha512-pv1y2Fv0JybcykuiiD3qBOBdz6RteYojRFY1d+b95WVuzx211CRh+ytI/+9iVyWQ6koTh5dawe4S/yRfOFjgaA==",
"cpu": [
"arm64"
],
@@ -99,9 +99,9 @@
}
},
"node_modules/@rolldown/binding-darwin-arm64": {
"version": "1.0.0-rc.10",
"resolved": "https://registry.npmjs.org/@rolldown/binding-darwin-arm64/-/binding-darwin-arm64-1.0.0-rc.10.tgz",
"integrity": "sha512-gED05Teg/vtTZbIJBc4VNMAxAFDUPkuO/rAIyyxZjTj1a1/s6z5TII/5yMGZ0uLRCifEtwUQn8OlYzuYc0m70w==",
"version": "1.0.0-rc.12",
"resolved": "https://registry.npmjs.org/@rolldown/binding-darwin-arm64/-/binding-darwin-arm64-1.0.0-rc.12.tgz",
"integrity": "sha512-cFYr6zTG/3PXXF3pUO+umXxt1wkRK/0AYT8lDwuqvRC+LuKYWSAQAQZjCWDQpAH172ZV6ieYrNnFzVVcnSflAg==",
"cpu": [
"arm64"
],
@@ -116,9 +116,9 @@
}
},
"node_modules/@rolldown/binding-darwin-x64": {
"version": "1.0.0-rc.10",
"resolved": "https://registry.npmjs.org/@rolldown/binding-darwin-x64/-/binding-darwin-x64-1.0.0-rc.10.tgz",
"integrity": "sha512-rI15NcM1mA48lqrIxVkHfAqcyFLcQwyXWThy+BQ5+mkKKPvSO26ir+ZDp36AgYoYVkqvMcdS8zOE6SeBsR9e8A==",
"version": "1.0.0-rc.12",
"resolved": "https://registry.npmjs.org/@rolldown/binding-darwin-x64/-/binding-darwin-x64-1.0.0-rc.12.tgz",
"integrity": "sha512-ZCsYknnHzeXYps0lGBz8JrF37GpE9bFVefrlmDrAQhOEi4IOIlcoU1+FwHEtyXGx2VkYAvhu7dyBf75EJQffBw==",
"cpu": [
"x64"
],
@@ -133,9 +133,9 @@
}
},
"node_modules/@rolldown/binding-freebsd-x64": {
"version": "1.0.0-rc.10",
"resolved": "https://registry.npmjs.org/@rolldown/binding-freebsd-x64/-/binding-freebsd-x64-1.0.0-rc.10.tgz",
"integrity": "sha512-XZRXHdTa+4ME1MuDVp021+doQ+z6Ei4CCFmNc5/sKbqb8YmkiJdj8QKlV3rCI0AJtAeSB5n0WGPuJWNL9p/L2w==",
"version": "1.0.0-rc.12",
"resolved": "https://registry.npmjs.org/@rolldown/binding-freebsd-x64/-/binding-freebsd-x64-1.0.0-rc.12.tgz",
"integrity": "sha512-dMLeprcVsyJsKolRXyoTH3NL6qtsT0Y2xeuEA8WQJquWFXkEC4bcu1rLZZSnZRMtAqwtrF/Ib9Ddtpa/Gkge9Q==",
"cpu": [
"x64"
],
@@ -150,9 +150,9 @@
}
},
"node_modules/@rolldown/binding-linux-arm-gnueabihf": {
"version": "1.0.0-rc.10",
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-arm-gnueabihf/-/binding-linux-arm-gnueabihf-1.0.0-rc.10.tgz",
"integrity": "sha512-R0SQMRluISSLzFE20sPWYHVmJdDQnRyc/FzSCN72BqQmh2SOZUFG+N3/vBZpR4C6WpEUVYJLrYUXaj43sJsNLA==",
"version": "1.0.0-rc.12",
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-arm-gnueabihf/-/binding-linux-arm-gnueabihf-1.0.0-rc.12.tgz",
"integrity": "sha512-YqWjAgGC/9M1lz3GR1r1rP79nMgo3mQiiA+Hfo+pvKFK1fAJ1bCi0ZQVh8noOqNacuY1qIcfyVfP6HoyBRZ85Q==",
"cpu": [
"arm"
],
@@ -167,13 +167,16 @@
}
},
"node_modules/@rolldown/binding-linux-arm64-gnu": {
"version": "1.0.0-rc.10",
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-arm64-gnu/-/binding-linux-arm64-gnu-1.0.0-rc.10.tgz",
"integrity": "sha512-Y1reMrV/o+cwpduYhJuOE3OMKx32RMYCidf14y+HssARRmhDuWXJ4yVguDg2R/8SyyGNo+auzz64LnPK9Hq6jg==",
"version": "1.0.0-rc.12",
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-arm64-gnu/-/binding-linux-arm64-gnu-1.0.0-rc.12.tgz",
"integrity": "sha512-/I5AS4cIroLpslsmzXfwbe5OmWvSsrFuEw3mwvbQ1kDxJ822hFHIx+vsN/TAzNVyepI/j/GSzrtCIwQPeKCLIg==",
"cpu": [
"arm64"
],
"dev": true,
"libc": [
"glibc"
],
"license": "MIT",
"optional": true,
"os": [
@@ -184,13 +187,16 @@
}
},
"node_modules/@rolldown/binding-linux-arm64-musl": {
"version": "1.0.0-rc.10",
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-arm64-musl/-/binding-linux-arm64-musl-1.0.0-rc.10.tgz",
"integrity": "sha512-vELN+HNb2IzuzSBUOD4NHmP9yrGwl1DVM29wlQvx1OLSclL0NgVWnVDKl/8tEks79EFek/kebQKnNJkIAA4W2g==",
"version": "1.0.0-rc.12",
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-arm64-musl/-/binding-linux-arm64-musl-1.0.0-rc.12.tgz",
"integrity": "sha512-V6/wZztnBqlx5hJQqNWwFdxIKN0m38p8Jas+VoSfgH54HSj9tKTt1dZvG6JRHcjh6D7TvrJPWFGaY9UBVOaWPw==",
"cpu": [
"arm64"
],
"dev": true,
"libc": [
"musl"
],
"license": "MIT",
"optional": true,
"os": [
@@ -201,13 +207,16 @@
}
},
"node_modules/@rolldown/binding-linux-ppc64-gnu": {
"version": "1.0.0-rc.10",
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-ppc64-gnu/-/binding-linux-ppc64-gnu-1.0.0-rc.10.tgz",
"integrity": "sha512-ZqrufYTgzxbHwpqOjzSsb0UV/aV2TFIY5rP8HdsiPTv/CuAgCRjM6s9cYFwQ4CNH+hf9Y4erHW1GjZuZ7WoI7w==",
"version": "1.0.0-rc.12",
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-ppc64-gnu/-/binding-linux-ppc64-gnu-1.0.0-rc.12.tgz",
"integrity": "sha512-AP3E9BpcUYliZCxa3w5Kwj9OtEVDYK6sVoUzy4vTOJsjPOgdaJZKFmN4oOlX0Wp0RPV2ETfmIra9x1xuayFB7g==",
"cpu": [
"ppc64"
],
"dev": true,
"libc": [
"glibc"
],
"license": "MIT",
"optional": true,
"os": [
@@ -218,13 +227,16 @@
}
},
"node_modules/@rolldown/binding-linux-s390x-gnu": {
"version": "1.0.0-rc.10",
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-s390x-gnu/-/binding-linux-s390x-gnu-1.0.0-rc.10.tgz",
"integrity": "sha512-gSlmVS1FZJSRicA6IyjoRoKAFK7IIHBs7xJuHRSmjImqk3mPPWbR7RhbnfH2G6bcmMEllCt2vQ/7u9e6bBnByg==",
"version": "1.0.0-rc.12",
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-s390x-gnu/-/binding-linux-s390x-gnu-1.0.0-rc.12.tgz",
"integrity": "sha512-nWwpvUSPkoFmZo0kQazZYOrT7J5DGOJ/+QHHzjvNlooDZED8oH82Yg67HvehPPLAg5fUff7TfWFHQS8IV1n3og==",
"cpu": [
"s390x"
],
"dev": true,
"libc": [
"glibc"
],
"license": "MIT",
"optional": true,
"os": [
@@ -235,13 +247,16 @@
}
},
"node_modules/@rolldown/binding-linux-x64-gnu": {
"version": "1.0.0-rc.10",
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-x64-gnu/-/binding-linux-x64-gnu-1.0.0-rc.10.tgz",
"integrity": "sha512-eOCKUpluKgfObT2pHjztnaWEIbUabWzk3qPZ5PuacuPmr4+JtQG4k2vGTY0H15edaTnicgU428XW/IH6AimcQw==",
"version": "1.0.0-rc.12",
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-x64-gnu/-/binding-linux-x64-gnu-1.0.0-rc.12.tgz",
"integrity": "sha512-RNrafz5bcwRy+O9e6P8Z/OCAJW/A+qtBczIqVYwTs14pf4iV1/+eKEjdOUta93q2TsT/FI0XYDP3TCky38LMAg==",
"cpu": [
"x64"
],
"dev": true,
"libc": [
"glibc"
],
"license": "MIT",
"optional": true,
"os": [
@@ -252,13 +267,16 @@
}
},
"node_modules/@rolldown/binding-linux-x64-musl": {
"version": "1.0.0-rc.10",
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-x64-musl/-/binding-linux-x64-musl-1.0.0-rc.10.tgz",
"integrity": "sha512-Xdf2jQbfQowJnLcgYfD/m0Uu0Qj5OdxKallD78/IPPfzaiaI4KRAwZzHcKQ4ig1gtg1SuzC7jovNiM2TzQsBXA==",
"version": "1.0.0-rc.12",
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-x64-musl/-/binding-linux-x64-musl-1.0.0-rc.12.tgz",
"integrity": "sha512-Jpw/0iwoKWx3LJ2rc1yjFrj+T7iHZn2JDg1Yny1ma0luviFS4mhAIcd1LFNxK3EYu3DHWCps0ydXQ5i/rrJ2ig==",
"cpu": [
"x64"
],
"dev": true,
"libc": [
"musl"
],
"license": "MIT",
"optional": true,
"os": [
@@ -269,9 +287,9 @@
}
},
"node_modules/@rolldown/binding-openharmony-arm64": {
"version": "1.0.0-rc.10",
"resolved": "https://registry.npmjs.org/@rolldown/binding-openharmony-arm64/-/binding-openharmony-arm64-1.0.0-rc.10.tgz",
"integrity": "sha512-o1hYe8hLi1EY6jgPFyxQgQ1wcycX+qz8eEbVmot2hFkgUzPxy9+kF0u0NIQBeDq+Mko47AkaFFaChcvZa9UX9Q==",
"version": "1.0.0-rc.12",
"resolved": "https://registry.npmjs.org/@rolldown/binding-openharmony-arm64/-/binding-openharmony-arm64-1.0.0-rc.12.tgz",
"integrity": "sha512-vRugONE4yMfVn0+7lUKdKvN4D5YusEiPilaoO2sgUWpCvrncvWgPMzK00ZFFJuiPgLwgFNP5eSiUlv2tfc+lpA==",
"cpu": [
"arm64"
],
@@ -286,9 +304,9 @@
}
},
"node_modules/@rolldown/binding-wasm32-wasi": {
"version": "1.0.0-rc.10",
"resolved": "https://registry.npmjs.org/@rolldown/binding-wasm32-wasi/-/binding-wasm32-wasi-1.0.0-rc.10.tgz",
"integrity": "sha512-Ugv9o7qYJudqQO5Y5y2N2SOo6S4WiqiNOpuQyoPInnhVzCY+wi/GHltcLHypG9DEUYMB0iTB/huJrpadiAcNcA==",
"version": "1.0.0-rc.12",
"resolved": "https://registry.npmjs.org/@rolldown/binding-wasm32-wasi/-/binding-wasm32-wasi-1.0.0-rc.12.tgz",
"integrity": "sha512-ykGiLr/6kkiHc0XnBfmFJuCjr5ZYKKofkx+chJWDjitX+KsJuAmrzWhwyOMSHzPhzOHOy7u9HlFoa5MoAOJ/Zg==",
"cpu": [
"wasm32"
],
@@ -303,9 +321,9 @@
}
},
"node_modules/@rolldown/binding-win32-arm64-msvc": {
"version": "1.0.0-rc.10",
"resolved": "https://registry.npmjs.org/@rolldown/binding-win32-arm64-msvc/-/binding-win32-arm64-msvc-1.0.0-rc.10.tgz",
"integrity": "sha512-7UODQb4fQUNT/vmgDZBl3XOBAIOutP5R3O/rkxg0aLfEGQ4opbCgU5vOw/scPe4xOqBwL9fw7/RP1vAMZ6QlAQ==",
"version": "1.0.0-rc.12",
"resolved": "https://registry.npmjs.org/@rolldown/binding-win32-arm64-msvc/-/binding-win32-arm64-msvc-1.0.0-rc.12.tgz",
"integrity": "sha512-5eOND4duWkwx1AzCxadcOrNeighiLwMInEADT0YM7xeEOOFcovWZCq8dadXgcRHSf3Ulh1kFo/qvzoFiCLOL1Q==",
"cpu": [
"arm64"
],
@@ -320,9 +338,9 @@
}
},
"node_modules/@rolldown/binding-win32-x64-msvc": {
"version": "1.0.0-rc.10",
"resolved": "https://registry.npmjs.org/@rolldown/binding-win32-x64-msvc/-/binding-win32-x64-msvc-1.0.0-rc.10.tgz",
"integrity": "sha512-PYxKHMVHOb5NJuDL53vBUl1VwUjymDcYI6rzpIni0C9+9mTiJedvUxSk7/RPp7OOAm3v+EjgMu9bIy3N6b408w==",
"version": "1.0.0-rc.12",
"resolved": "https://registry.npmjs.org/@rolldown/binding-win32-x64-msvc/-/binding-win32-x64-msvc-1.0.0-rc.12.tgz",
"integrity": "sha512-PyqoipaswDLAZtot351MLhrlrh6lcZPo2LSYE+VDxbVk24LVKAGOuE4hb8xZQmrPAuEtTZW8E6D2zc5EUZX4Lw==",
"cpu": [
"x64"
],
@@ -337,9 +355,9 @@
}
},
"node_modules/@rolldown/pluginutils": {
"version": "1.0.0-rc.10",
"resolved": "https://registry.npmjs.org/@rolldown/pluginutils/-/pluginutils-1.0.0-rc.10.tgz",
"integrity": "sha512-UkVDEFk1w3mveXeKgaTuYfKWtPbvgck1dT8TUG3bnccrH0XtLTuAyfCoks4Q/M5ZGToSVJTIQYCzy2g/atAOeg==",
"version": "1.0.0-rc.12",
"resolved": "https://registry.npmjs.org/@rolldown/pluginutils/-/pluginutils-1.0.0-rc.12.tgz",
"integrity": "sha512-HHMwmarRKvoFsJorqYlFeFRzXZqCt2ETQlEDOb9aqssrnVBB1/+xgTGtuTrIk5vzLNX1MjMtTf7W9z3tsSbrxw==",
"dev": true,
"license": "MIT"
},
@@ -387,31 +405,31 @@
"license": "MIT"
},
"node_modules/@vitest/expect": {
"version": "4.1.1",
"resolved": "https://registry.npmjs.org/@vitest/expect/-/expect-4.1.1.tgz",
"integrity": "sha512-xAV0fqBTk44Rn6SjJReEQkHP3RrqbJo6JQ4zZ7/uVOiJZRarBtblzrOfFIZeYUrukp2YD6snZG6IBqhOoHTm+A==",
"version": "4.1.2",
"resolved": "https://registry.npmjs.org/@vitest/expect/-/expect-4.1.2.tgz",
"integrity": "sha512-gbu+7B0YgUJ2nkdsRJrFFW6X7NTP44WlhiclHniUhxADQJH5Szt9mZ9hWnJPJ8YwOK5zUOSSlSvyzRf0u1DSBQ==",
"dev": true,
"license": "MIT",
"dependencies": {
"@standard-schema/spec": "^1.1.0",
"@types/chai": "^5.2.2",
"@vitest/spy": "4.1.1",
"@vitest/utils": "4.1.1",
"@vitest/spy": "4.1.2",
"@vitest/utils": "4.1.2",
"chai": "^6.2.2",
"tinyrainbow": "^3.0.3"
"tinyrainbow": "^3.1.0"
},
"funding": {
"url": "https://opencollective.com/vitest"
}
},
"node_modules/@vitest/mocker": {
"version": "4.1.1",
"resolved": "https://registry.npmjs.org/@vitest/mocker/-/mocker-4.1.1.tgz",
"integrity": "sha512-h3BOylsfsCLPeceuCPAAJ+BvNwSENgJa4hXoXu4im0bs9Lyp4URc4JYK4pWLZ4pG/UQn7AT92K6IByi6rE6g3A==",
"version": "4.1.2",
"resolved": "https://registry.npmjs.org/@vitest/mocker/-/mocker-4.1.2.tgz",
"integrity": "sha512-Ize4iQtEALHDttPRCmN+FKqOl2vxTiNUhzobQFFt/BM1lRUTG7zRCLOykG/6Vo4E4hnUdfVLo5/eqKPukcWW7Q==",
"dev": true,
"license": "MIT",
"dependencies": {
"@vitest/spy": "4.1.1",
"@vitest/spy": "4.1.2",
"estree-walker": "^3.0.3",
"magic-string": "^0.30.21"
},
@@ -432,26 +450,26 @@
}
},
"node_modules/@vitest/pretty-format": {
"version": "4.1.1",
"resolved": "https://registry.npmjs.org/@vitest/pretty-format/-/pretty-format-4.1.1.tgz",
"integrity": "sha512-GM+TEQN5WhOygr1lp7skeVjdLPqqWMHsfzXrcHAqZJi/lIVh63H0kaRCY8MDhNWikx19zBUK8ceaLB7X5AH9NQ==",
"version": "4.1.2",
"resolved": "https://registry.npmjs.org/@vitest/pretty-format/-/pretty-format-4.1.2.tgz",
"integrity": "sha512-dwQga8aejqeuB+TvXCMzSQemvV9hNEtDDpgUKDzOmNQayl2OG241PSWeJwKRH3CiC+sESrmoFd49rfnq7T4RnA==",
"dev": true,
"license": "MIT",
"dependencies": {
"tinyrainbow": "^3.0.3"
"tinyrainbow": "^3.1.0"
},
"funding": {
"url": "https://opencollective.com/vitest"
}
},
"node_modules/@vitest/runner": {
"version": "4.1.1",
"resolved": "https://registry.npmjs.org/@vitest/runner/-/runner-4.1.1.tgz",
"integrity": "sha512-f7+FPy75vN91QGWsITueq0gedwUZy1fLtHOCMeQpjs8jTekAHeKP80zfDEnhrleviLHzVSDXIWuCIOFn3D3f8A==",
"version": "4.1.2",
"resolved": "https://registry.npmjs.org/@vitest/runner/-/runner-4.1.2.tgz",
"integrity": "sha512-Gr+FQan34CdiYAwpGJmQG8PgkyFVmARK8/xSijia3eTFgVfpcpztWLuP6FttGNfPLJhaZVP/euvujeNYar36OQ==",
"dev": true,
"license": "MIT",
"dependencies": {
"@vitest/utils": "4.1.1",
"@vitest/utils": "4.1.2",
"pathe": "^2.0.3"
},
"funding": {
@@ -459,14 +477,14 @@
}
},
"node_modules/@vitest/snapshot": {
"version": "4.1.1",
"resolved": "https://registry.npmjs.org/@vitest/snapshot/-/snapshot-4.1.1.tgz",
"integrity": "sha512-kMVSgcegWV2FibXEx9p9WIKgje58lcTbXgnJixfcg15iK8nzCXhmalL0ZLtTWLW9PH1+1NEDShiFFedB3tEgWg==",
"version": "4.1.2",
"resolved": "https://registry.npmjs.org/@vitest/snapshot/-/snapshot-4.1.2.tgz",
"integrity": "sha512-g7yfUmxYS4mNxk31qbOYsSt2F4m1E02LFqO53Xpzg3zKMhLAPZAjjfyl9e6z7HrW6LvUdTwAQR3HHfLjpko16A==",
"dev": true,
"license": "MIT",
"dependencies": {
"@vitest/pretty-format": "4.1.1",
"@vitest/utils": "4.1.1",
"@vitest/pretty-format": "4.1.2",
"@vitest/utils": "4.1.2",
"magic-string": "^0.30.21",
"pathe": "^2.0.3"
},
@@ -475,9 +493,9 @@
}
},
"node_modules/@vitest/spy": {
"version": "4.1.1",
"resolved": "https://registry.npmjs.org/@vitest/spy/-/spy-4.1.1.tgz",
"integrity": "sha512-6Ti/KT5OVaiupdIZEuZN7l3CZcR0cxnxt70Z0//3CtwgObwA6jZhmVBA3yrXSVN3gmwjgd7oDNLlsXz526gpRA==",
"version": "4.1.2",
"resolved": "https://registry.npmjs.org/@vitest/spy/-/spy-4.1.2.tgz",
"integrity": "sha512-DU4fBnbVCJGNBwVA6xSToNXrkZNSiw59H8tcuUspVMsBDBST4nfvsPsEHDHGtWRRnqBERBQu7TrTKskmjqTXKA==",
"dev": true,
"license": "MIT",
"funding": {
@@ -485,15 +503,15 @@
}
},
"node_modules/@vitest/utils": {
"version": "4.1.1",
"resolved": "https://registry.npmjs.org/@vitest/utils/-/utils-4.1.1.tgz",
"integrity": "sha512-cNxAlaB3sHoCdL6pj6yyUXv9Gry1NHNg0kFTXdvSIZXLHsqKH7chiWOkwJ5s5+d/oMwcoG9T0bKU38JZWKusrQ==",
"version": "4.1.2",
"resolved": "https://registry.npmjs.org/@vitest/utils/-/utils-4.1.2.tgz",
"integrity": "sha512-xw2/TiX82lQHA06cgbqRKFb5lCAy3axQ4H4SoUFhUsg+wztiet+co86IAMDtF6Vm1hc7J6j09oh/rgDn+JdKIQ==",
"dev": true,
"license": "MIT",
"dependencies": {
"@vitest/pretty-format": "4.1.1",
"@vitest/pretty-format": "4.1.2",
"convert-source-map": "^2.0.0",
"tinyrainbow": "^3.0.3"
"tinyrainbow": "^3.1.0"
},
"funding": {
"url": "https://opencollective.com/vitest"
@@ -739,6 +757,9 @@
"arm64"
],
"dev": true,
"libc": [
"glibc"
],
"license": "MPL-2.0",
"optional": true,
"os": [
@@ -760,6 +781,9 @@
"arm64"
],
"dev": true,
"libc": [
"musl"
],
"license": "MPL-2.0",
"optional": true,
"os": [
@@ -781,6 +805,9 @@
"x64"
],
"dev": true,
"libc": [
"glibc"
],
"license": "MPL-2.0",
"optional": true,
"os": [
@@ -802,6 +829,9 @@
"x64"
],
"dev": true,
"libc": [
"musl"
],
"license": "MPL-2.0",
"optional": true,
"os": [
@@ -912,9 +942,9 @@
"license": "ISC"
},
"node_modules/picomatch": {
"version": "4.0.3",
"resolved": "https://registry.npmjs.org/picomatch/-/picomatch-4.0.3.tgz",
"integrity": "sha512-5gTmgEY/sqK6gFXLIsQNH19lWb4ebPDLA4SdLP7dsWkIXHWlG66oPuVvXSGFPppYZz8ZDZq0dYYrbHfBCVUb1Q==",
"version": "4.0.4",
"resolved": "https://registry.npmjs.org/picomatch/-/picomatch-4.0.4.tgz",
"integrity": "sha512-QP88BAKvMam/3NxH6vj2o21R6MjxZUAd6nlwAS/pnGvN9IVLocLHxGYIzFhg6fUQ+5th6P4dv4eW9jX3DSIj7A==",
"dev": true,
"license": "MIT",
"engines": {
@@ -954,14 +984,14 @@
}
},
"node_modules/rolldown": {
"version": "1.0.0-rc.10",
"resolved": "https://registry.npmjs.org/rolldown/-/rolldown-1.0.0-rc.10.tgz",
"integrity": "sha512-q7j6vvarRFmKpgJUT8HCAUljkgzEp4LAhPlJUvQhA5LA1SUL36s5QCysMutErzL3EbNOZOkoziSx9iZC4FddKA==",
"version": "1.0.0-rc.12",
"resolved": "https://registry.npmjs.org/rolldown/-/rolldown-1.0.0-rc.12.tgz",
"integrity": "sha512-yP4USLIMYrwpPHEFB5JGH1uxhcslv6/hL0OyvTuY+3qlOSJvZ7ntYnoWpehBxufkgN0cvXxppuTu5hHa/zPh+A==",
"dev": true,
"license": "MIT",
"dependencies": {
"@oxc-project/types": "=0.120.0",
"@rolldown/pluginutils": "1.0.0-rc.10"
"@oxc-project/types": "=0.122.0",
"@rolldown/pluginutils": "1.0.0-rc.12"
},
"bin": {
"rolldown": "bin/cli.mjs"
@@ -970,21 +1000,21 @@
"node": "^20.19.0 || >=22.12.0"
},
"optionalDependencies": {
"@rolldown/binding-android-arm64": "1.0.0-rc.10",
"@rolldown/binding-darwin-arm64": "1.0.0-rc.10",
"@rolldown/binding-darwin-x64": "1.0.0-rc.10",
"@rolldown/binding-freebsd-x64": "1.0.0-rc.10",
"@rolldown/binding-linux-arm-gnueabihf": "1.0.0-rc.10",
"@rolldown/binding-linux-arm64-gnu": "1.0.0-rc.10",
"@rolldown/binding-linux-arm64-musl": "1.0.0-rc.10",
"@rolldown/binding-linux-ppc64-gnu": "1.0.0-rc.10",
"@rolldown/binding-linux-s390x-gnu": "1.0.0-rc.10",
"@rolldown/binding-linux-x64-gnu": "1.0.0-rc.10",
"@rolldown/binding-linux-x64-musl": "1.0.0-rc.10",
"@rolldown/binding-openharmony-arm64": "1.0.0-rc.10",
"@rolldown/binding-wasm32-wasi": "1.0.0-rc.10",
"@rolldown/binding-win32-arm64-msvc": "1.0.0-rc.10",
"@rolldown/binding-win32-x64-msvc": "1.0.0-rc.10"
"@rolldown/binding-android-arm64": "1.0.0-rc.12",
"@rolldown/binding-darwin-arm64": "1.0.0-rc.12",
"@rolldown/binding-darwin-x64": "1.0.0-rc.12",
"@rolldown/binding-freebsd-x64": "1.0.0-rc.12",
"@rolldown/binding-linux-arm-gnueabihf": "1.0.0-rc.12",
"@rolldown/binding-linux-arm64-gnu": "1.0.0-rc.12",
"@rolldown/binding-linux-arm64-musl": "1.0.0-rc.12",
"@rolldown/binding-linux-ppc64-gnu": "1.0.0-rc.12",
"@rolldown/binding-linux-s390x-gnu": "1.0.0-rc.12",
"@rolldown/binding-linux-x64-gnu": "1.0.0-rc.12",
"@rolldown/binding-linux-x64-musl": "1.0.0-rc.12",
"@rolldown/binding-openharmony-arm64": "1.0.0-rc.12",
"@rolldown/binding-wasm32-wasi": "1.0.0-rc.12",
"@rolldown/binding-win32-arm64-msvc": "1.0.0-rc.12",
"@rolldown/binding-win32-x64-msvc": "1.0.0-rc.12"
}
},
"node_modules/siginfo": {
@@ -1085,16 +1115,16 @@
}
},
"node_modules/vite": {
"version": "8.0.1",
"resolved": "https://registry.npmjs.org/vite/-/vite-8.0.1.tgz",
"integrity": "sha512-wt+Z2qIhfFt85uiyRt5LPU4oVEJBXj8hZNWKeqFG4gRG/0RaRGJ7njQCwzFVjO+v4+Ipmf5CY7VdmZRAYYBPHw==",
"version": "8.0.3",
"resolved": "https://registry.npmjs.org/vite/-/vite-8.0.3.tgz",
"integrity": "sha512-B9ifbFudT1TFhfltfaIPgjo9Z3mDynBTJSUYxTjOQruf/zHH+ezCQKcoqO+h7a9Pw9Nm/OtlXAiGT1axBgwqrQ==",
"dev": true,
"license": "MIT",
"dependencies": {
"lightningcss": "^1.32.0",
"picomatch": "^4.0.3",
"picomatch": "^4.0.4",
"postcss": "^8.5.8",
"rolldown": "1.0.0-rc.10",
"rolldown": "1.0.0-rc.12",
"tinyglobby": "^0.2.15"
},
"bin": {
@@ -1163,19 +1193,19 @@
}
},
"node_modules/vitest": {
"version": "4.1.1",
"resolved": "https://registry.npmjs.org/vitest/-/vitest-4.1.1.tgz",
"integrity": "sha512-yF+o4POL41rpAzj5KVILUxm1GCjKnELvaqmU9TLLUbMfDzuN0UpUR9uaDs+mCtjPe+uYPksXDRLQGGPvj1cTmA==",
"version": "4.1.2",
"resolved": "https://registry.npmjs.org/vitest/-/vitest-4.1.2.tgz",
"integrity": "sha512-xjR1dMTVHlFLh98JE3i/f/WePqJsah4A0FK9cc8Ehp9Udk0AZk6ccpIZhh1qJ/yxVWRZ+Q54ocnD8TXmkhspGg==",
"dev": true,
"license": "MIT",
"dependencies": {
"@vitest/expect": "4.1.1",
"@vitest/mocker": "4.1.1",
"@vitest/pretty-format": "4.1.1",
"@vitest/runner": "4.1.1",
"@vitest/snapshot": "4.1.1",
"@vitest/spy": "4.1.1",
"@vitest/utils": "4.1.1",
"@vitest/expect": "4.1.2",
"@vitest/mocker": "4.1.2",
"@vitest/pretty-format": "4.1.2",
"@vitest/runner": "4.1.2",
"@vitest/snapshot": "4.1.2",
"@vitest/spy": "4.1.2",
"@vitest/utils": "4.1.2",
"es-module-lexer": "^2.0.0",
"expect-type": "^1.3.0",
"magic-string": "^0.30.21",
@@ -1186,7 +1216,7 @@
"tinybench": "^2.9.0",
"tinyexec": "^1.0.2",
"tinyglobby": "^0.2.15",
"tinyrainbow": "^3.0.3",
"tinyrainbow": "^3.1.0",
"vite": "^6.0.0 || ^7.0.0 || ^8.0.0",
"why-is-node-running": "^2.3.0"
},
@@ -1203,10 +1233,10 @@
"@edge-runtime/vm": "*",
"@opentelemetry/api": "^1.9.0",
"@types/node": "^20.0.0 || ^22.0.0 || >=24.0.0",
"@vitest/browser-playwright": "4.1.1",
"@vitest/browser-preview": "4.1.1",
"@vitest/browser-webdriverio": "4.1.1",
"@vitest/ui": "4.1.1",
"@vitest/browser-playwright": "4.1.2",
"@vitest/browser-preview": "4.1.2",
"@vitest/browser-webdriverio": "4.1.2",
"@vitest/ui": "4.1.2",
"happy-dom": "*",
"jsdom": "*",
"vite": "^6.0.0 || ^7.0.0 || ^8.0.0"
+16
View File
@@ -1,6 +1,15 @@
import { TurnstoneAPIError } from "./errors.js";
import { parseSSEStream } from "./sse.js";
export interface TlsOptions {
/** Path to CA certificate PEM file (Node.js only). */
caCert?: string;
/** Path to client certificate PEM file for mTLS (Node.js only). */
clientCert?: string;
/** Path to client key PEM file for mTLS (Node.js only). */
clientKey?: string;
}
export interface ClientOptions {
/** Server base URL (e.g. "http://localhost:8080"). */
baseUrl: string;
@@ -8,6 +17,13 @@ export interface ClientOptions {
token?: string;
/** Custom fetch implementation (defaults to globalThis.fetch). */
fetch?: typeof globalThis.fetch;
/**
* TLS certificate paths for documentation and tooling.
* The SDK does not read these directly pass a custom `fetch`
* configured with your runtime's TLS agent (e.g. Node.js https.Agent).
* See docs/tls.md for examples.
*/
tls?: TlsOptions;
}
export interface RequestOptions {
+1 -1
View File
@@ -19,7 +19,7 @@
// Clients
export { TurnstoneServer } from "./server.js";
export { TurnstoneConsole } from "./console.js";
export type { ClientOptions } from "./base.js";
export type { ClientOptions, TlsOptions } from "./base.js";
// Errors
export { TurnstoneAPIError } from "./errors.js";
+344 -48
View File
@@ -1,5 +1,5 @@
{
"description": "turnstone behavior tests tool selection, sequencing, and multi-step reasoning",
"description": "turnstone behavior tests \u2014 tool selection, sequencing, and multi-step reasoning",
"defaults": {
"n_runs": 5,
"max_turns": 15
@@ -8,35 +8,91 @@
{
"id": "read-before-edit",
"description": "Must read_file before edit_file on the same path",
"user_prompt": "Fix the typo in config.py change 'recieve' to 'receive'",
"user_prompt": "Fix the typo in config.py \u2014 change 'recieve' to 'receive'",
"setup": {
"files": {
"config.py": "# Config module\ndef recieve_data(source):\n \"\"\"Recieve data from source.\"\"\"\n return source.read()\n"
}
},
"expected_actions": [
{ "tool": "read_file", "args": { "path": "config.py" } },
{ "tool": "edit_file", "args": { "path": "config.py" } }
{
"tool": "read_file",
"args": {
"path": "config.py"
}
},
{
"tool": "edit_file",
"args": {
"path": "config.py"
}
}
],
"match_mode": "ordered_subset"
"match_mode": "ordered_subset",
"user_prompts": [
"Fix the typo in config.py \u2014 change 'recieve' to 'receive'",
"In config.py, correct the misspelling of 'recieve' to 'receive'",
"Please update config.py by replacing 'recieve' with the correct spelling 'receive'",
"There's a typo in config.py: 'recieve' should be 'receive'. Please fix it.",
"Could you change 'recieve' to 'receive' in config.py?",
"Go ahead and fix 'recieve' \u2192 'receive' in config.py",
"I need the word 'recieve' corrected to 'receive' in the file config.py",
"config.py has a spelling error \u2014 'recieve' needs to be changed to 'receive'",
"Kindly rectify the typographical error in config.py, replacing 'recieve' with 'receive'",
"Hey, swap 'recieve' for 'receive' in config.py"
]
},
{
"id": "write-file-not-bash",
"description": "Use write_file for file creation, not bash echo/cat",
"user_prompt": "Create a file called hello.py that prints hello world",
"expected_actions": [
{ "tool": "write_file", "args_pattern": { "path": "hello\\.py" } }
{
"tool": "write_file",
"args_pattern": {
"path": "hello\\.py"
}
}
],
"match_mode": "subset"
"match_mode": "subset",
"user_prompts": [
"Create a file called hello.py that prints hello world",
"Make a hello.py file that outputs hello world",
"Please write a Python file named hello.py which prints hello world",
"I need a file called hello.py that prints hello world",
"Could you create hello.py with code that prints hello world?",
"Write hello.py \u2014 it should print hello world",
"Generate a hello.py file that outputs \"hello world\"",
"I'd like you to create a file named hello.py that prints hello world",
"Set up a file called hello.py to print hello world",
"Kindly produce a hello.py file whose purpose is to print hello world"
]
},
{
"id": "bash-for-commands",
"description": "Use bash for running system commands",
"user_prompt": "What Python version is installed?",
"expected_actions": [
{ "tool": "bash", "args_pattern": { "command": "python" } }
{
"tool": "bash",
"args_pattern": {
"command": "python"
}
}
],
"match_mode": "subset"
"match_mode": "subset",
"user_prompts": [
"What Python version is installed?",
"Check which version of Python is currently installed",
"Can you tell me the installed Python version?",
"python --version please",
"I need to know what version of Python is on this system",
"Which Python version do we have?",
"Could you look up the Python version that's installed here?",
"Determine the currently installed Python version",
"What's the Python version on this machine?",
"Please check the Python version"
]
},
{
"id": "search-for-patterns",
@@ -49,13 +105,30 @@
}
},
"expected_actions": [
{ "tool": "search", "args_pattern": { "query": "test_" } }
{
"tool": "search",
"args_pattern": {
"query": "test_"
}
}
],
"match_mode": "subset"
"match_mode": "subset",
"user_prompts": [
"Find all functions that start with 'test_' in the project",
"List every function in the project whose name begins with 'test_'",
"I need to locate all functions prefixed with 'test_' across the project",
"Could you search the project for any functions starting with 'test_'?",
"Show me all the test_ prefixed functions in this project",
"Hunt down every function that has a 'test_' prefix in the codebase",
"I'm looking for all functions named test_* throughout the project",
"Search the entire project for functions whose names start with test_",
"What functions beginning with 'test_' exist in this project?",
"Please identify all functions with the 'test_' prefix in the project files"
]
},
{
"id": "multi-file-edit",
"description": "Read and edit multiple files must read before editing each, and edit both",
"description": "Read and edit multiple files \u2014 must read before editing each, and edit both",
"user_prompt": "Change the default port from 8000 to 9000 in both server.py and config.py",
"setup": {
"files": {
@@ -64,12 +137,32 @@
}
},
"expected_actions": [
{ "tool": "read_file" },
{ "tool": "read_file" },
{ "tool": "edit_file" },
{ "tool": "edit_file" }
{
"tool": "read_file"
},
{
"tool": "read_file"
},
{
"tool": "edit_file"
},
{
"tool": "edit_file"
}
],
"match_mode": "ordered_subset"
"match_mode": "ordered_subset",
"user_prompts": [
"Change the default port from 8000 to 9000 in both server.py and config.py",
"Update the default port to 9000 instead of 8000 in server.py and config.py",
"Could you modify the port number from 8000 to 9000 in both config.py and server.py?",
"Please replace port 8000 with 9000 in server.py and config.py",
"I need the default port switched from 8000 to 9000 in both server.py and config.py",
"In server.py and config.py, the default port should be changed from 8000 to 9000",
"Swap out port 8000 for 9000 in config.py and server.py",
"Would you mind updating the default port value from 8000 to 9000 across both server.py and config.py?",
"The default port in server.py and config.py needs to be 9000 instead of 8000 \u2014 please make that change",
"Go ahead and change 8000 to 9000 for the default port in both server.py and config.py"
]
},
{
"id": "search-then-edit",
@@ -83,51 +176,147 @@
}
},
"expected_actions": [
{ "tool": "search", "args_pattern": { "query": "MAX_RETRIES" } },
{ "tool": "read_file" },
{ "tool": "edit_file", "args_pattern": { "old_string": "3" } }
{
"tool": "search",
"args_pattern": {
"query": "MAX_RETRIES"
}
},
{
"tool": "read_file"
},
{
"tool": "edit_file",
"args_pattern": {
"old_string": "3"
}
}
],
"match_mode": "ordered_subset"
"match_mode": "ordered_subset",
"user_prompts": [
"Find where MAX_RETRIES is defined and change it from 3 to 5",
"Locate the definition of MAX_RETRIES and update its value from 3 to 5",
"Could you search for where MAX_RETRIES is defined and modify it from 3 to 5?",
"I need MAX_RETRIES changed from 3 to 5 \u2014 find where it's defined and update it",
"Please find the MAX_RETRIES definition and bump it from 3 to 5",
"Hunt down MAX_RETRIES in the codebase and change its value from 3 to 5",
"Where is MAX_RETRIES set to 3? Change it to 5.",
"Search the code for the MAX_RETRIES definition and alter it from 3 to 5",
"I'd like you to locate MAX_RETRIES (currently 3) and set it to 5 instead",
"Go find MAX_RETRIES and switch it from 3 to 5"
]
},
{
"id": "bash-git-log",
"description": "Use bash for git commands, not other tools",
"user_prompt": "Show me the git log for the last 5 commits",
"expected_actions": [
{ "tool": "bash", "args_pattern": { "command": "git\\s+log" } }
{
"tool": "bash",
"args_pattern": {
"command": "git\\s+log"
}
}
],
"match_mode": "subset"
"match_mode": "subset",
"user_prompts": [
"Show me the git log for the last 5 commits",
"Display the 5 most recent git commits",
"Can you pull up the git log limited to the last five commits?",
"I need to see the git log showing only the previous 5 commits",
"git log for the 5 latest commits, please",
"Would you mind showing me the last five entries in the git log?",
"Print out the most recent 5 commits from the git log",
"I'd like to review the git log \u2014 just the last 5 commits",
"Show the recent 5 commit history using git log",
"Could you display the git commit history for the past five commits?"
]
},
{
"id": "write-then-run",
"description": "Create a script and run it to verify it works",
"user_prompt": "Create a Python script called fib.py that prints the first 10 Fibonacci numbers, then run it to verify",
"expected_actions": [
{ "tool": "write_file", "args_pattern": { "path": "fib\\.py" } },
{ "tool": "bash", "args_pattern": { "command": "python" } }
{
"tool": "write_file",
"args_pattern": {
"path": "fib\\.py"
}
},
{
"tool": "bash",
"args_pattern": {
"command": "python"
}
}
],
"match_mode": "ordered_subset"
"match_mode": "ordered_subset",
"user_prompts": [
"Create a Python script called fib.py that prints the first 10 Fibonacci numbers, then run it to verify",
"Write a Python file named fib.py that outputs the first 10 Fibonacci numbers, and execute it to confirm it works",
"Please make fib.py \u2013 a Python script printing the first ten Fibonacci numbers \u2013 then run it to check the output",
"I need a Python script fib.py that prints the first 10 Fibonacci numbers. Execute it afterwards to verify correctness.",
"Could you create fib.py to display the first 10 Fibonacci numbers in Python, and then run it to make sure it works?",
"Draft a script called fib.py in Python that outputs the first ten Fibonacci numbers, then execute it to validate",
"Hey, write me a fib.py that prints the first 10 Fibonacci numbers and run it so we can see it works",
"Generate a Python program fib.py which prints the initial 10 Fibonacci numbers, and verify by running it",
"Kindly produce a Python script named fib.py to print the first 10 Fibonacci numbers, then execute the script to confirm its output",
"Make a file fib.py containing Python code to print the first 10 Fibonacci numbers. Then run it to verify."
]
},
{
"id": "no-bash-for-file-write",
"description": "Should NOT use bash (echo/cat/heredoc) to create files only write_file",
"description": "Should NOT use bash (echo/cat/heredoc) to create files \u2014 only write_file",
"user_prompt": "Create a new file called README.md with a title and description of this project",
"expected_actions": [
{ "tool": "write_file", "args_pattern": { "path": "README" } }
{
"tool": "write_file",
"args_pattern": {
"path": "README"
}
}
],
"match_mode": "subset"
"match_mode": "subset",
"user_prompts": [
"Create a new file called README.md with a title and description of this project",
"Make a README.md file that includes a project title and description",
"I need a README.md created with a title and a brief description of the project",
"Please generate a README.md file containing the project's title and description",
"Could you set up a README.md with a title and project description?",
"Write a README.md that has a title and describes this project",
"Go ahead and create README.md \u2014 it should have a title and a description of the project",
"I'd like you to produce a new README.md file featuring a project title and description",
"Kindly establish a README.md file incorporating both a title and a description for this project",
"Spin up a README.md with a project title and description in it"
]
},
{
"id": "plan-before-refactor",
"description": "Use the plan tool before a large refactoring task",
"user_prompt": "I need to refactor this codebase to separate the database layer from the API layer. Use the plan tool to think through the approach before making any changes.",
"id": "plan-when-asked",
"description": "Call the plan tool when the user asks to plan",
"user_prompt": "Plan how to add user authentication to this app.",
"setup": {
"files": {
"app.py": "import sqlite3\nfrom flask import Flask, jsonify\n\napp = Flask(__name__)\nDB = 'data.db'\n\ndef get_db():\n return sqlite3.connect(DB)\n\n@app.route('/users')\ndef list_users():\n db = get_db()\n users = db.execute('SELECT * FROM users').fetchall()\n db.close()\n return jsonify(users)\n\n@app.route('/users/<int:uid>')\ndef get_user(uid):\n db = get_db()\n user = db.execute('SELECT * FROM users WHERE id=?', (uid,)).fetchone()\n db.close()\n return jsonify(user)\n\nif __name__ == '__main__':\n app.run(port=8000)\n"
"app.py": "from flask import Flask, jsonify\n\napp = Flask(__name__)\n\n@app.route('/users')\ndef list_users():\n return jsonify([{'id': 1, 'name': 'Alice'}])\n\nif __name__ == '__main__':\n app.run(port=8000)\n"
}
},
"expected_actions": [{ "tool": "create_plan" }],
"match_mode": "subset"
"expected_actions": [
{
"tool": "plan_agent"
}
],
"match_mode": "subset",
"user_prompts": [
"Plan how to add user authentication to this app.",
"Make a plan for adding pagination to the API endpoints.",
"Plan out how to add error handling to this application.",
"I need a plan for adding logging to this codebase.",
"Plan the approach for adding unit tests to this app.",
"How would you approach adding user authentication to this app? Lay out a plan.",
"I'd like you to outline a strategy for implementing user authentication in this application.",
"Could you come up with a plan for integrating user authentication into this app?",
"Think through the steps needed to add user auth to this app and present a plan.",
"Draft a plan for incorporating user authentication functionality into this application."
]
},
{
"id": "edit-not-rewrite",
@@ -139,10 +328,32 @@
}
},
"expected_actions": [
{ "tool": "read_file", "args": { "path": "utils.py" } },
{ "tool": "edit_file", "args": { "path": "utils.py" } }
{
"tool": "read_file",
"args": {
"path": "utils.py"
}
},
{
"tool": "edit_file",
"args": {
"path": "utils.py"
}
}
],
"match_mode": "ordered_subset"
"match_mode": "ordered_subset",
"user_prompts": [
"Add a docstring to the process_data function in utils.py",
"Please add a docstring to the process_data function in utils.py",
"Could you write a docstring for process_data in utils.py?",
"Insert a docstring into the process_data function found in utils.py",
"I need a docstring added to process_data in utils.py",
"Put a docstring on the process_data function in utils.py",
"The process_data function in utils.py is missing a docstring \u2014 please add one",
"Would you mind adding a docstring to process_data in utils.py?",
"In utils.py, the process_data function needs a docstring",
"Add documentation via a docstring to the process_data function within utils.py"
]
},
{
"id": "bash-run-tests",
@@ -154,45 +365,130 @@
}
},
"expected_actions": [
{ "tool": "bash", "args_pattern": { "command": "pytest|python.*test" } }
{
"tool": "bash",
"args_pattern": {
"command": "pytest|python.*test"
}
}
],
"match_mode": "subset"
"match_mode": "subset",
"user_prompts": [
"Run the tests",
"Execute the test suite",
"Please go ahead and run the tests",
"Could you run the tests for me?",
"I need the tests to be run",
"Kick off the tests",
"Let's run the tests",
"Fire up the tests",
"Go ahead and execute the tests",
"I'd like you to run the tests"
]
},
{
"id": "web-fetch-url",
"description": "Use web_fetch when asked to retrieve content from a URL",
"user_prompt": "Fetch the contents of https://example.com and summarize what's on the page",
"expected_actions": [
{ "tool": "web_fetch", "args_pattern": { "url": "example\\.com" } }
{
"tool": "web_fetch",
"args_pattern": {
"url": "example\\.com"
}
}
],
"match_mode": "subset"
"match_mode": "subset",
"user_prompts": [
"Fetch the contents of https://example.com and summarize what's on the page",
"Go to https://example.com and give me a summary of what you find there",
"Could you pull up https://example.com and tell me what the page is about?",
"Retrieve the content from https://example.com, then provide a summary of it",
"I need you to grab https://example.com and summarize its contents for me",
"Please access https://example.com and give me an overview of the page",
"What's on https://example.com? Fetch it and summarize for me.",
"Download the page at https://example.com and provide a brief summary",
"I'd like a summary of whatever is at https://example.com \u2014 please fetch it first",
"Hit https://example.com and let me know what's there in summary form"
]
},
{
"id": "man-page-lookup",
"description": "Use man tool to look up command documentation",
"user_prompt": "Look up the man page for tar and tell me what the --xattrs flag does",
"expected_actions": [
{ "tool": "man", "args_pattern": { "page": "tar" } }
{
"tool": "man",
"args_pattern": {
"page": "tar"
}
}
],
"match_mode": "subset"
"match_mode": "subset",
"user_prompts": [
"Look up the man page for tar and tell me what the --xattrs flag does",
"What does the --xattrs flag do in tar? Check the man page for me.",
"Could you pull up the man page for tar and explain the --xattrs option?",
"I need to know what --xattrs does in tar \u2014 can you check the man page?",
"Check tar's man page and let me know the purpose of the --xattrs flag.",
"Please consult the tar man page and describe what the --xattrs flag is for.",
"Hey, look at the tar man page real quick \u2014 what's --xattrs do?",
"I'd like you to read the tar man page and summarize the --xattrs option for me.",
"Would you mind checking the man page for tar to find out what --xattrs means?",
"Look into the tar manual and explain the --xattrs flag to me."
]
},
{
"id": "math-calculation",
"description": "Use the math tool for precise calculations, not bash or mental math",
"user_prompt": "What is 2^64 - 1? Use the math tool to calculate it precisely.",
"expected_actions": [
{ "tool": "math", "args_pattern": { "code": "2.*64" } }
{
"tool": "math",
"args_pattern": {
"code": "2.*64"
}
}
],
"match_mode": "subset"
"match_mode": "subset",
"user_prompts": [
"What is 2^64 - 1? Use the math tool to calculate it precisely.",
"Calculate 2^64 - 1 for me using the math tool, please.",
"I need the exact value of 2^64 - 1. Please use the math tool.",
"Could you use the math tool to compute 2^64 minus 1 precisely?",
"Use the math tool to tell me what 2^64 - 1 equals.",
"I'm curious: what's 2^64 - 1? Compute it with the math tool.",
"Please precisely determine 2^64 - 1 via the math tool.",
"Mind using the math tool to figure out 2^64 - 1 exactly?",
"I'd like to know the precise result of 2^64 - 1 \u2014 use the math tool for this.",
"Leverage the math tool to give me an exact answer for 2^64 - 1."
]
},
{
"id": "web-search-query",
"description": "Use web_search for general knowledge lookups, not web_fetch",
"user_prompt": "Search the web for the current population of Tokyo",
"expected_actions": [
{ "tool": "web_search", "args_pattern": { "query": "Tokyo" } }
{
"tool": "web_search",
"args_pattern": {
"query": "Tokyo"
}
}
],
"match_mode": "subset"
"match_mode": "subset",
"user_prompts": [
"Search the web for the current population of Tokyo",
"What's Tokyo's current population? Look it up on the web.",
"Could you do a web search to find out how many people currently live in Tokyo?",
"Please search online for Tokyo's present-day population.",
"I need you to look up the current population of Tokyo on the web.",
"Find me Tokyo's current population via a web search.",
"Web search: what is the current population of Tokyo?",
"I'd like to know Tokyo's current population\u2014can you search the web for that?",
"Look up how many people live in Tokyo right now using a web search.",
"Do a web search for the population of Tokyo as of now."
]
}
]
}
+46
View File
@@ -1237,6 +1237,52 @@ class TestJWTAudienceIssuer:
result = validate_jwt(token, self.SECRET, audience="")
assert result is not None
def test_create_jwt_expiry_seconds(self):
import jwt as pyjwt
from turnstone.core.auth import create_jwt
token = create_jwt("user1", frozenset({"read"}), "test", self.SECRET, expiry_seconds=300)
payload = pyjwt.decode(
token, self.SECRET, algorithms=["HS256"], options={"verify_aud": False}
)
assert payload["exp"] - payload["iat"] == 300
def test_create_jwt_expiry_seconds_overrides_hours(self):
import jwt as pyjwt
from turnstone.core.auth import create_jwt
token = create_jwt(
"user1",
frozenset({"read"}),
"test",
self.SECRET,
expiry_hours=24,
expiry_seconds=60,
)
payload = pyjwt.decode(
token, self.SECRET, algorithms=["HS256"], options={"verify_aud": False}
)
# expiry_seconds takes precedence over expiry_hours
assert payload["exp"] - payload["iat"] == 60
def test_create_jwt_expiry_seconds_rejects_zero(self):
import pytest
from turnstone.core.auth import create_jwt
with pytest.raises(ValueError, match="expiry_seconds must be positive"):
create_jwt("user1", frozenset({"read"}), "test", self.SECRET, expiry_seconds=0)
def test_create_jwt_expiry_seconds_rejects_negative(self):
import pytest
from turnstone.core.auth import create_jwt
with pytest.raises(ValueError, match="expiry_seconds must be positive"):
create_jwt("user1", frozenset({"read"}), "test", self.SECRET, expiry_seconds=-1)
class TestServiceTokenManager:
SECRET = "test-secret-that-is-at-least-32-chars"
+230
View File
@@ -1699,6 +1699,236 @@ class TestSSEProxy:
asyncio.run(_run())
# ---------------------------------------------------------------------------
# Proxy auth header propagation
# ---------------------------------------------------------------------------
class TestProxyAuthHeaders:
"""Verify _proxy_auth_headers mints user-scoped JWTs for proxy requests."""
SECRET = "test-secret-that-is-at-least-32-chars"
def _make_request(
self, *, auth_result=None, jwt_secret="", proxy_token_mgr=None, proxy_auth_token=""
):
"""Build a minimal fake request for _proxy_auth_headers."""
class _State:
pass
class _AppState:
pass
class _App:
state = _AppState()
class _Request:
state = _State()
app = _App()
req = _Request()
req.state.auth_result = auth_result
req.app.state.jwt_secret = jwt_secret
req.app.state.proxy_token_mgr = proxy_token_mgr
req.app.state.proxy_auth_token = proxy_auth_token
return req
def test_mints_user_jwt(self):
"""Real user auth_result → JWT with correct sub, scopes, src, aud, permissions."""
import jwt as pyjwt
from turnstone.console.server import _proxy_auth_headers
from turnstone.core.auth import JWT_AUD_SERVER, AuthResult
auth = AuthResult(
user_id="alice",
scopes=frozenset({"read", "write"}),
token_source="jwt",
permissions=frozenset({"admin.users"}),
)
req = self._make_request(auth_result=auth, jwt_secret=self.SECRET)
headers = _proxy_auth_headers(req)
assert "Authorization" in headers
token = headers["Authorization"].removeprefix("Bearer ")
payload = pyjwt.decode(token, self.SECRET, algorithms=["HS256"], audience=JWT_AUD_SERVER)
assert payload["sub"] == "alice"
assert set(payload["scopes"].split(",")) == {"read", "write"}
assert payload["src"] == "console-proxy"
assert payload["aud"] == JWT_AUD_SERVER
assert payload["permissions"] == "admin.users"
def test_narrows_scopes(self):
"""Read-only user → JWT carries only read scope, not full {read,write,approve}."""
import jwt as pyjwt
from turnstone.console.server import _proxy_auth_headers
from turnstone.core.auth import JWT_AUD_SERVER, AuthResult
auth = AuthResult(
user_id="viewer",
scopes=frozenset({"read"}),
token_source="jwt",
)
req = self._make_request(auth_result=auth, jwt_secret=self.SECRET)
headers = _proxy_auth_headers(req)
token = headers["Authorization"].removeprefix("Bearer ")
payload = pyjwt.decode(token, self.SECRET, algorithms=["HS256"], audience=JWT_AUD_SERVER)
assert payload["scopes"] == "read"
def test_short_expiry(self):
"""Minted JWT expires in 300 seconds, not hours."""
import jwt as pyjwt
from turnstone.console.server import _proxy_auth_headers
from turnstone.core.auth import AuthResult
auth = AuthResult(
user_id="alice",
scopes=frozenset({"read"}),
token_source="jwt",
)
req = self._make_request(auth_result=auth, jwt_secret=self.SECRET)
headers = _proxy_auth_headers(req)
token = headers["Authorization"].removeprefix("Bearer ")
payload = pyjwt.decode(
token, self.SECRET, algorithms=["HS256"], options={"verify_aud": False}
)
assert payload["exp"] - payload["iat"] == 300
def test_fallback_no_user(self):
"""No auth_result → falls back to ServiceTokenManager."""
from turnstone.console.server import _proxy_auth_headers
from turnstone.core.auth import ServiceTokenManager
mgr = ServiceTokenManager(
user_id="console-proxy",
scopes=frozenset({"read", "write", "approve"}),
source="console",
secret=self.SECRET,
)
req = self._make_request(proxy_token_mgr=mgr)
headers = _proxy_auth_headers(req)
assert "Authorization" in headers
assert headers["Authorization"] == f"Bearer {mgr.token}"
def test_fallback_no_secret(self):
"""auth_result present but empty jwt_secret → falls back to ServiceTokenManager."""
from turnstone.console.server import _proxy_auth_headers
from turnstone.core.auth import AuthResult, ServiceTokenManager
auth = AuthResult(
user_id="alice",
scopes=frozenset({"read"}),
token_source="jwt",
)
mgr = ServiceTokenManager(
user_id="console-proxy",
scopes=frozenset({"read", "write", "approve"}),
source="console",
secret=self.SECRET,
)
req = self._make_request(auth_result=auth, jwt_secret="", proxy_token_mgr=mgr)
headers = _proxy_auth_headers(req)
# Should use ServiceTokenManager, not mint a user JWT
assert headers["Authorization"] == f"Bearer {mgr.token}"
def test_fallback_static_token(self):
"""No auth_result, no ServiceTokenManager → uses static proxy_auth_token."""
from turnstone.console.server import _proxy_auth_headers
req = self._make_request(proxy_auth_token="static-tok-123")
headers = _proxy_auth_headers(req)
assert headers == {"Authorization": "Bearer static-tok-123"}
# ---------------------------------------------------------------------------
# Server: trusted user_id forwarding on create_workstream
# ---------------------------------------------------------------------------
class TestCreateWorkstreamUserIdTrust:
"""Verify that only trusted service tokens can forward user_id in create_workstream."""
def _extract_uid(self, body: dict, auth_result) -> str:
"""Replicate the trust check from server.py:create_workstream."""
auth = auth_result
uid: str = getattr(auth, "user_id", "") or ""
trusted_sources = {"bridge", "console"}
if (
body.get("user_id")
and isinstance(body["user_id"], str)
and auth is not None
and auth.token_source in trusted_sources
):
uid = body["user_id"]
return uid
def test_bridge_can_forward_user_id(self):
from turnstone.core.auth import AuthResult
auth = AuthResult(
user_id="bridge",
scopes=frozenset({"approve"}),
token_source="bridge",
)
uid = self._extract_uid({"user_id": "real-user-abc"}, auth)
assert uid == "real-user-abc"
def test_console_service_can_forward_user_id(self):
from turnstone.core.auth import AuthResult
auth = AuthResult(
user_id="console",
scopes=frozenset({"approve"}),
token_source="console",
)
uid = self._extract_uid({"user_id": "real-user-abc"}, auth)
assert uid == "real-user-abc"
def test_console_proxy_user_cannot_override_user_id(self):
"""End-user tokens via console-proxy must NOT override user_id."""
from turnstone.core.auth import AuthResult
auth = AuthResult(
user_id="real-user-abc",
scopes=frozenset({"read", "write"}),
token_source="console-proxy",
)
uid = self._extract_uid({"user_id": "impersonated-user"}, auth)
# Should use JWT identity, NOT the body override
assert uid == "real-user-abc"
def test_direct_user_cannot_override_user_id(self):
"""Direct JWT login must NOT override user_id."""
from turnstone.core.auth import AuthResult
auth = AuthResult(
user_id="real-user-abc",
scopes=frozenset({"read", "write"}),
token_source="password",
)
uid = self._extract_uid({"user_id": "impersonated-user"}, auth)
assert uid == "real-user-abc"
def test_no_body_user_id_uses_jwt(self):
from turnstone.core.auth import AuthResult
auth = AuthResult(
user_id="bridge",
scopes=frozenset({"approve"}),
token_source="bridge",
)
uid = self._extract_uid({"name": "test-ws"}, auth)
assert uid == "bridge"
# ---------------------------------------------------------------------------
# Collector — MCP aggregation in get_overview()
# ---------------------------------------------------------------------------
+29 -16
View File
@@ -4,6 +4,7 @@ from __future__ import annotations
import json
import time
from concurrent.futures import ThreadPoolExecutor
from pathlib import Path
from typing import Any
from unittest.mock import MagicMock
@@ -179,10 +180,13 @@ class TestErrorHandling:
provider = _make_mock_provider(side_effect=RuntimeError("API error"))
judge = _make_judge(provider)
result = judge._evaluate_single(
_make_item(),
[{"role": "user", "content": "test"}],
)
with ThreadPoolExecutor(max_workers=1) as pool:
result = judge._evaluate_single(
_make_item(),
[{"role": "user", "content": "test"}],
cancel_event=None,
executor=pool,
)
assert result is None
def test_provider_error_heuristic_still_returned(self):
@@ -211,10 +215,13 @@ class TestErrorHandling:
result_mock.content = ""
judge = _make_judge(provider)
result = judge._evaluate_single(
_make_item(),
[{"role": "user", "content": "test"}],
)
with ThreadPoolExecutor(max_workers=1) as pool:
result = judge._evaluate_single(
_make_item(),
[{"role": "user", "content": "test"}],
cancel_event=None,
executor=pool,
)
assert result is None
@@ -254,10 +261,13 @@ class TestMultiTurnToolUse:
provider.create_completion.side_effect = [turn1, turn2]
judge = _make_judge(provider)
verdict = judge._evaluate_single(
_make_item(),
[{"role": "user", "content": "test"}],
)
with ThreadPoolExecutor(max_workers=1) as pool:
verdict = judge._evaluate_single(
_make_item(),
[{"role": "user", "content": "test"}],
cancel_event=None,
executor=pool,
)
assert verdict is not None
assert verdict.tier == "llm"
assert provider.create_completion.call_count == 2
@@ -299,10 +309,13 @@ class TestMultiTurnToolUse:
]
judge = _make_judge(provider)
judge._evaluate_single(
_make_item(),
[{"role": "user", "content": "test"}],
)
with ThreadPoolExecutor(max_workers=1) as pool:
judge._evaluate_single(
_make_item(),
[{"role": "user", "content": "test"}],
cancel_event=None,
executor=pool,
)
# Should have called create_completion exactly _JUDGE_MAX_TURNS times
assert provider.create_completion.call_count == 5
+2
View File
@@ -573,7 +573,9 @@ def _fake_request(*nodes: dict[str, Any], proxy_client: Any = None) -> MagicMock
collector.get_nodes.return_value = (list(nodes), len(nodes))
collector.get_all_nodes.side_effect = lambda: collector.get_nodes.return_value[0]
req = MagicMock()
req.state.auth_result = None
req.app.state.collector = collector
req.app.state.jwt_secret = ""
req.app.state.proxy_client = proxy_client or AsyncMock()
req.app.state.proxy_token_mgr = None
req.app.state.proxy_auth_token = "tok"
+28
View File
@@ -140,6 +140,34 @@ class TestOpenAIProvider:
def test_provider_name(self) -> None:
assert self.provider.provider_name == "openai"
# -- _sanitize_messages ---------------------------------------------------
def test_sanitize_messages_none_content_no_tool_calls(self) -> None:
msgs = [{"role": "assistant", "content": None}]
assert self.provider._sanitize_messages(msgs) == [{"role": "assistant", "content": ""}]
def test_sanitize_messages_none_content_with_tool_calls(self) -> None:
msgs = [{"role": "assistant", "content": None, "tool_calls": [{"id": "1"}]}]
result = self.provider._sanitize_messages(msgs)
assert result[0]["content"] is None
assert result[0]["tool_calls"] == [{"id": "1"}]
def test_sanitize_messages_empty_string_passthrough(self) -> None:
msgs = [{"role": "assistant", "content": ""}]
assert self.provider._sanitize_messages(msgs) == msgs
def test_sanitize_messages_non_assistant_unchanged(self) -> None:
msgs = [{"role": "user", "content": None}]
result = self.provider._sanitize_messages(msgs)
assert result[0]["content"] is None
def test_sanitize_messages_does_not_mutate_original(self) -> None:
original = {"role": "assistant", "content": None}
self.provider._sanitize_messages([original])
assert original["content"] is None
# -- convert_tools --------------------------------------------------------
def test_convert_tools_passthrough(self) -> None:
tools = [
{
+4 -4
View File
@@ -214,7 +214,7 @@ class TestPlanExec:
"id": tc_id,
"type": "function",
"function": {
"name": "create_plan",
"name": "plan_agent",
"arguments": json.dumps({"goal": prior_prompt}),
},
}
@@ -250,7 +250,7 @@ class TestPlanExec:
m for m in messages if m["role"] == "assistant" and m.get("tool_calls")
]
assert len(assistant_with_tc) == 1
assert assistant_with_tc[0]["tool_calls"][0]["function"]["name"] == "create_plan"
assert assistant_with_tc[0]["tool_calls"][0]["function"]["name"] == "plan_agent"
# The real tool result is forwarded with its original content
tool_msgs = [m for m in messages if m["role"] == "tool"]
@@ -463,7 +463,7 @@ class TestPlanRefinement:
with patch.object(session, "_refine_plan", side_effect=fake_refine):
items = [
{
"func_name": "create_plan",
"func_name": "plan_agent",
"call_id": "c1",
"prompt": "add auth",
}
@@ -576,7 +576,7 @@ class TestPlanRefinement:
msgs = captured["messages"]
assert msgs[0]["role"] == "system"
assert msgs[1]["role"] == "assistant"
assert msgs[1]["tool_calls"][0]["function"]["name"] == "create_plan"
assert msgs[1]["tool_calls"][0]["function"]["name"] == "plan_agent"
assert msgs[2]["role"] == "tool"
assert msgs[2]["content"] == self.GOOD_PLAN
assert msgs[3]["role"] == "user"
+234
View File
@@ -0,0 +1,234 @@
"""Tests for TLS admin API endpoints and CLI commands."""
from __future__ import annotations
import pytest
from turnstone.core.storage import get_storage, init_storage, reset_storage
lacme = pytest.importorskip("lacme")
@pytest.fixture(autouse=True)
def _storage(tmp_path):
"""Initialize ephemeral SQLite storage for each test."""
reset_storage()
db = str(tmp_path / "test.db")
init_storage("sqlite", path=db)
yield
reset_storage()
@pytest.fixture
def tls_manager():
"""Create an initialized TLSManager."""
import asyncio
from turnstone.console.tls import TLSManager
mgr = TLSManager(get_storage())
asyncio.run(mgr.init_ca())
# Issue a test cert
asyncio.run(mgr.issue_console_certs(["test.internal", "localhost"]))
return mgr
# ── Admin API endpoints ───────────────────────────────────────────────────────
def _make_app(tls_manager):
"""Create a minimal Starlette app with TLS endpoints."""
from starlette.applications import Starlette
from starlette.middleware import Middleware
from starlette.middleware.base import BaseHTTPMiddleware
from starlette.routing import Route
from turnstone.console.server import (
tls_ca_cert,
tls_ca_status,
tls_delete_cert,
tls_list_certs,
tls_renew_cert,
)
from turnstone.core.auth import AuthResult
async def _grant_access(request, call_next): # type: ignore[no-untyped-def]
request.state.auth_result = AuthResult(
user_id="",
scopes=frozenset({"approve"}),
token_source="config",
)
return await call_next(request)
app = Starlette(
routes=[
Route("/ca", tls_ca_status),
Route("/ca.pem", tls_ca_cert),
Route("/certs", tls_list_certs),
Route("/certs/{domain}/renew", tls_renew_cert, methods=["POST"]),
Route("/certs/{domain}", tls_delete_cert, methods=["DELETE"]),
],
middleware=[Middleware(BaseHTTPMiddleware, dispatch=_grant_access)],
)
app.state.tls_manager = tls_manager
return app
def test_list_certs(tls_manager):
from starlette.testclient import TestClient
client = TestClient(_make_app(tls_manager))
resp = client.get("/certs")
assert resp.status_code == 200
data = resp.json()
assert len(data["certs"]) >= 1
assert data["certs"][0]["domain"] == "test.internal"
def test_renew_cert(tls_manager):
from starlette.testclient import TestClient
client = TestClient(_make_app(tls_manager))
resp = client.post("/certs/test.internal/renew")
assert resp.status_code == 200
data = resp.json()
assert data["domain"] == "test.internal"
def test_renew_cert_not_found(tls_manager):
from starlette.testclient import TestClient
client = TestClient(_make_app(tls_manager))
resp = client.post("/certs/nonexistent.internal/renew")
assert resp.status_code == 404
def test_delete_cert(tls_manager):
from starlette.testclient import TestClient
client = TestClient(_make_app(tls_manager))
resp = client.delete("/certs/test.internal")
assert resp.status_code == 200
assert resp.json()["deleted"] == "test.internal"
# Verify it's gone
resp = client.get("/certs")
domains = [c["domain"] for c in resp.json()["certs"]]
assert "test.internal" not in domains
def test_delete_cert_not_found(tls_manager):
from starlette.testclient import TestClient
client = TestClient(_make_app(tls_manager))
resp = client.delete("/certs/nonexistent.internal")
assert resp.status_code == 404
# ── CLI bootstrap ─────────────────────────────────────────────────────────────
def test_cli_bootstrap(tmp_path):
"""Test offline CA bootstrap."""
import argparse
from turnstone.admin import _cmd_tls_bootstrap
out = tmp_path / "certs"
args = argparse.Namespace(out=str(out), issue=["redis.internal", "pg.internal"])
_cmd_tls_bootstrap(args)
assert (out / "ca.pem").exists()
assert b"BEGIN CERTIFICATE" in (out / "ca.pem").read_bytes()
# Check certs were issued
assert (out / "certs" / "redis.internal").exists()
assert (out / "certs" / "pg.internal").exists()
def test_cli_bootstrap_no_issue(tmp_path):
"""Bootstrap with no --issue creates CA only."""
import argparse
from turnstone.admin import _cmd_tls_bootstrap
out = tmp_path / "certs"
args = argparse.Namespace(out=str(out), issue=[])
_cmd_tls_bootstrap(args)
assert (out / "ca.pem").exists()
# No certs dir
certs_dir = out / "certs"
if certs_dir.exists():
assert len(list(certs_dir.iterdir())) == 0
# ── Config parsing ────────────────────────────────────────────────────────────
def test_redis_tls_config_map():
"""Redis TLS keys are in the config map."""
from turnstone.core.config import _CONFIG_MAP
redis_map = _CONFIG_MAP["redis"]
assert "tls" in redis_map
assert "tls_ca" in redis_map
assert "tls_cert" in redis_map
assert "tls_key" in redis_map
def test_database_ssl_config_map():
"""Database SSL keys are in the config map."""
from turnstone.core.config import _CONFIG_MAP
db_map = _CONFIG_MAP["database"]
assert "sslmode" in db_map
assert "sslrootcert" in db_map
assert "sslcert" in db_map
assert "sslkey" in db_map
# ── Auth enforcement ──────────────────────────────────────────────────────────
def test_tls_endpoints_require_auth(tls_manager):
"""TLS admin endpoints return 401 without auth."""
from starlette.applications import Starlette
from starlette.routing import Route
from starlette.testclient import TestClient
from turnstone.console.server import tls_ca_status, tls_list_certs
# No auth middleware — request.state.auth_result will be missing
app = Starlette(
routes=[
Route("/ca", tls_ca_status),
Route("/certs", tls_list_certs),
]
)
app.state.tls_manager = tls_manager
client = TestClient(app)
resp = client.get("/ca")
assert resp.status_code == 401
resp = client.get("/certs")
assert resp.status_code == 401
# ── SDK TLS params ────────────────────────────────────────────────────────────
def test_sdk_client_cert_requires_both():
"""SDK raises ValueError if only one of client_cert/client_key provided."""
from turnstone.sdk._base import _BaseClient
with pytest.raises(ValueError, match="Both client_cert and client_key"):
_BaseClient(
base_url="http://localhost:8080",
client_cert="/path/to/cert.pem",
)
with pytest.raises(ValueError, match="Both client_cert and client_key"):
_BaseClient(
base_url="http://localhost:8080",
client_key="/path/to/key.pem",
)
+98
View File
@@ -0,0 +1,98 @@
"""Tests for TLSClient — service node certificate provisioning."""
from __future__ import annotations
from unittest.mock import MagicMock
import pytest
from turnstone.core.storage import get_storage, init_storage, reset_storage
lacme = pytest.importorskip("lacme")
@pytest.fixture(autouse=True)
def _storage(tmp_path):
"""Initialize ephemeral SQLite storage for each test."""
reset_storage()
db = str(tmp_path / "test.db")
init_storage("sqlite", path=db)
yield
reset_storage()
# ── Console URL discovery ─────────────────────────────────────────────────────
def test_discover_console_url():
"""TLSClient discovers console URL from services table."""
from turnstone.core.tls import TLSClient
storage = get_storage()
storage.register_service("console", "console", "http://console:8080")
client = TLSClient(storage=storage, hostnames=["node-1"])
url = client._discover_console_url()
assert url == "http://console:8080"
def test_discover_console_url_missing():
"""TLSClient raises if no console registered."""
from turnstone.core.tls import TLSClient
client = TLSClient(storage=get_storage(), hostnames=["node-1"])
with pytest.raises(RuntimeError, match="No console service found"):
client._discover_console_url()
def test_explicit_console_url_skips_discovery():
"""When console_url is provided, discovery is skipped."""
from turnstone.core.tls import TLSClient
client = TLSClient(
storage=get_storage(),
console_url="http://explicit:9090",
hostnames=["node-1"],
)
assert client._console_url == "http://explicit:9090"
# ── SSL context construction ─────────────────────────────────────────────────
@pytest.mark.anyio
async def test_ssl_contexts_none_before_init():
"""SSL contexts are None before init()."""
from turnstone.core.tls import TLSClient
client = TLSClient(
storage=get_storage(),
console_url="http://localhost:8080",
hostnames=["node-1"],
)
assert client.get_server_ssl_context() is None
assert client.get_client_ssl_context() is None
assert not client.initialized
# ── Backward compatibility ───────────────────────────────────────────────────
def test_bridge_tls_defaults():
"""Bridge with default TLS params works without changes."""
from turnstone.mq.bridge import Bridge
# Default: tls_verify=True, tls_cert=None — no mTLS
bridge = Bridge(server_url="http://localhost:8080")
assert bridge._tls_verify is True
assert bridge._tls_cert is None
def test_collector_tls_defaults():
"""Collector with default TLS params works without changes."""
from turnstone.console.collector import ClusterCollector
broker_mock = MagicMock()
collector = ClusterCollector(broker=broker_mock)
# Should create httpx client without errors
assert collector._http_client is not None
+226
View File
@@ -0,0 +1,226 @@
"""Tests for TLSManager — console CA and ACME server."""
from __future__ import annotations
import pytest
from turnstone.core.storage import get_storage, init_storage, reset_storage
lacme = pytest.importorskip("lacme")
@pytest.fixture(autouse=True)
def _storage(tmp_path):
"""Initialize ephemeral SQLite storage for each test."""
reset_storage()
db = str(tmp_path / "test.db")
init_storage("sqlite", path=db)
yield
reset_storage()
@pytest.fixture
def tls_manager():
"""Create a TLSManager backed by test storage."""
from turnstone.console.tls import TLSManager
return TLSManager(get_storage())
# ── CA initialization ─────────────────────────────────────────────────────────
@pytest.mark.anyio
async def test_init_ca(tls_manager):
await tls_manager.init_ca()
assert tls_manager.ca_initialized
root_pem = tls_manager.get_root_cert_pem()
assert b"BEGIN CERTIFICATE" in root_pem
@pytest.mark.anyio
async def test_init_ca_persists(tls_manager):
"""CA root survives re-initialization (loaded from storage)."""
await tls_manager.init_ca()
pem1 = tls_manager.get_root_cert_pem()
# Create a new manager on the same storage
from turnstone.console.tls import TLSManager
mgr2 = TLSManager(get_storage())
await mgr2.init_ca()
pem2 = mgr2.get_root_cert_pem()
assert pem1 == pem2 # Same CA loaded from DB
@pytest.mark.anyio
async def test_get_responder_before_init(tls_manager):
with pytest.raises(RuntimeError, match="CA not initialized"):
tls_manager.get_responder()
@pytest.mark.anyio
async def test_get_responder(tls_manager):
await tls_manager.init_ca()
responder = tls_manager.get_responder()
assert responder is not None
# Should be an ASGI app (callable)
assert callable(responder)
# ── Cert issuance ─────────────────────────────────────────────────────────────
@pytest.mark.anyio
async def test_issue_console_certs_internal(tls_manager):
"""Console certs issued from internal CA when no external directory."""
await tls_manager.init_ca()
await tls_manager.issue_console_certs(["console.internal", "localhost"])
assert tls_manager.internal_bundle is not None
assert tls_manager.frontend_bundle is not None
assert tls_manager.internal_bundle.domain == "console.internal"
assert b"BEGIN CERTIFICATE" in tls_manager.internal_bundle.cert_pem
@pytest.mark.anyio
async def test_issue_console_certs_persists(tls_manager):
"""Certs loaded from storage on re-issue."""
await tls_manager.init_ca()
await tls_manager.issue_console_certs(["console.internal"])
bundle1 = tls_manager.internal_bundle
# New manager, same storage
from turnstone.console.tls import TLSManager
mgr2 = TLSManager(get_storage())
await mgr2.init_ca()
await mgr2.issue_console_certs(["console.internal"])
bundle2 = mgr2.internal_bundle
assert bundle1.cert_pem == bundle2.cert_pem
# ── SSL contexts ──────────────────────────────────────────────────────────────
@pytest.mark.anyio
async def test_ssl_contexts_none_before_certs(tls_manager):
await tls_manager.init_ca()
assert tls_manager.get_server_ssl_context() is None
assert tls_manager.get_client_ssl_context() is None
@pytest.mark.anyio
async def test_ssl_contexts_after_certs(tls_manager):
await tls_manager.init_ca()
await tls_manager.issue_console_certs(["console.internal"])
server_ctx = tls_manager.get_server_ssl_context()
client_ctx = tls_manager.get_client_ssl_context()
assert server_ctx is not None
assert client_ctx is not None
import ssl
assert isinstance(server_ctx, ssl.SSLContext)
assert isinstance(client_ctx, ssl.SSLContext)
# ── Root cert endpoint ────────────────────────────────────────────────────────
@pytest.mark.anyio
async def test_tls_ca_cert_endpoint(tls_manager):
"""Test the CA cert download endpoint via test client."""
await tls_manager.init_ca()
from starlette.applications import Starlette
from starlette.middleware import Middleware
from starlette.routing import Route
from starlette.testclient import TestClient
from turnstone.console.server import tls_ca_cert, tls_ca_status
# Middleware that grants full access (config-token style: no user_id)
from turnstone.core.auth import AuthResult
async def _grant_access(request, call_next): # type: ignore[no-untyped-def]
request.state.auth_result = AuthResult(
user_id="", scopes=frozenset({"approve"}), token_source="config"
)
return await call_next(request)
from starlette.middleware.base import BaseHTTPMiddleware
app = Starlette(
routes=[
Route("/ca.pem", tls_ca_cert),
Route("/ca", tls_ca_status),
],
middleware=[Middleware(BaseHTTPMiddleware, dispatch=_grant_access)],
)
app.state.tls_manager = tls_manager
client = TestClient(app)
# CA cert download
resp = client.get("/ca.pem")
assert resp.status_code == 200
assert b"BEGIN CERTIFICATE" in resp.content
assert resp.headers["content-type"] == "application/x-pem-file"
# CA status
resp = client.get("/ca")
assert resp.status_code == 200
data = resp.json()
assert data["enabled"] is True
assert data["ca_cn"] == "Turnstone CA"
@pytest.mark.anyio
async def test_tls_endpoints_disabled():
"""Endpoints return 404/disabled when TLS not enabled."""
from starlette.applications import Starlette
from starlette.middleware import Middleware
from starlette.middleware.base import BaseHTTPMiddleware
from starlette.routing import Route
from starlette.testclient import TestClient
from turnstone.console.server import tls_ca_cert, tls_ca_status
from turnstone.core.auth import AuthResult
async def _grant_access(request, call_next): # type: ignore[no-untyped-def]
request.state.auth_result = AuthResult(
user_id="", scopes=frozenset({"approve"}), token_source="config"
)
return await call_next(request)
app = Starlette(
routes=[
Route("/ca.pem", tls_ca_cert),
Route("/ca", tls_ca_status),
],
middleware=[Middleware(BaseHTTPMiddleware, dispatch=_grant_access)],
)
# No tls_manager on state
client = TestClient(app)
resp = client.get("/ca.pem")
assert resp.status_code == 404
resp = client.get("/ca")
data = resp.json()
assert data["enabled"] is False
# ── Events ────────────────────────────────────────────────────────────────────
@pytest.mark.anyio
async def test_event_dispatcher_wired(tls_manager):
"""Verify the event dispatcher has subscribers."""
assert tls_manager._event_dispatcher is not None
# Should have at least 4 subscriptions (issued, renewed, expiring, failed)
# The exact check depends on lacme's EventDispatcher internals,
# so just verify the dispatcher exists and the manager initializes cleanly
await tls_manager.init_ca()
+226
View File
@@ -0,0 +1,226 @@
"""Tests for TLS storage backend and lacme Store adapter."""
from __future__ import annotations
import json
from datetime import UTC, datetime
import pytest
from turnstone.core.storage import get_storage, init_storage, reset_storage
@pytest.fixture(autouse=True)
def _storage(tmp_path):
"""Initialize ephemeral SQLite storage for each test."""
reset_storage()
db = str(tmp_path / "test.db")
init_storage("sqlite", path=db)
yield
reset_storage()
# ── Account keys ──────────────────────────────────────────────────────────────
def test_save_and_load_account_key():
s = get_storage()
s.save_tls_account_key(
"default",
"-----BEGIN EC PRIVATE KEY-----\nfake\n-----END EC PRIVATE KEY-----",
)
result = s.load_tls_account_key("default")
assert result is not None
assert "EC PRIVATE KEY" in result
def test_load_account_key_missing():
s = get_storage()
assert s.load_tls_account_key("nonexistent") is None
def test_save_account_key_upsert():
s = get_storage()
s.save_tls_account_key("default", "key-v1")
s.save_tls_account_key("default", "key-v2")
assert s.load_tls_account_key("default") == "key-v2"
# ── CA ────────────────────────────────────────────────────────────────────────
def test_save_and_load_ca():
s = get_storage()
s.save_tls_ca("Turnstone CA", "cert-pem-data", "key-pem-data")
result = s.load_tls_ca("Turnstone CA")
assert result is not None
assert result["cert_pem"] == "cert-pem-data"
assert result["key_pem"] == "key-pem-data"
assert result["name"] == "Turnstone CA"
def test_load_ca_missing():
s = get_storage()
assert s.load_tls_ca("nonexistent") is None
def test_save_ca_upsert():
s = get_storage()
s.save_tls_ca("CA", "cert-v1", "key-v1")
s.save_tls_ca("CA", "cert-v2", "key-v2")
result = s.load_tls_ca("CA")
assert result["cert_pem"] == "cert-v2"
assert result["key_pem"] == "key-v2"
# ── Certificates ──────────────────────────────────────────────────────────────
def test_save_and_load_cert():
s = get_storage()
s.save_tls_cert(
domain="node-1.internal",
cert_pem="cert-data",
fullchain_pem="fullchain-data",
key_pem="key-data",
issued_at="2026-03-25T00:00:00",
expires_at="2026-03-27T00:00:00",
meta=json.dumps({"domains": ["node-1.internal", "10.0.1.5"]}),
)
result = s.load_tls_cert("node-1.internal")
assert result is not None
assert result["domain"] == "node-1.internal"
assert result["cert_pem"] == "cert-data"
assert result["fullchain_pem"] == "fullchain-data"
assert result["key_pem"] == "key-data"
assert result["issued_at"] == "2026-03-25T00:00:00"
assert result["expires_at"] == "2026-03-27T00:00:00"
meta = json.loads(result["meta"])
assert meta["domains"] == ["node-1.internal", "10.0.1.5"]
def test_load_cert_missing():
s = get_storage()
assert s.load_tls_cert("nonexistent") is None
def test_save_cert_upsert():
s = get_storage()
s.save_tls_cert("d", "c1", "f1", "k1", "2026-01-01", "2026-01-02")
s.save_tls_cert("d", "c2", "f2", "k2", "2026-02-01", "2026-02-02")
result = s.load_tls_cert("d")
assert result["cert_pem"] == "c2"
assert result["issued_at"] == "2026-02-01"
def test_list_certs_empty():
s = get_storage()
assert s.list_tls_certs() == []
def test_list_certs():
s = get_storage()
s.save_tls_cert("alpha.internal", "c", "f", "k", "2026-01-01", "2026-01-02")
s.save_tls_cert("beta.internal", "c", "f", "k", "2026-01-01", "2026-01-02")
certs = s.list_tls_certs()
assert len(certs) == 2
assert certs[0]["domain"] == "alpha.internal" # sorted by domain
assert certs[1]["domain"] == "beta.internal"
def test_delete_cert():
s = get_storage()
s.save_tls_cert("d", "c", "f", "k", "2026-01-01", "2026-01-02")
assert s.delete_tls_cert("d") is True
assert s.load_tls_cert("d") is None
def test_delete_cert_missing():
s = get_storage()
assert s.delete_tls_cert("nonexistent") is False
# ── StorageStore adapter ──────────────────────────────────────────────────────
@pytest.fixture
def store_adapter():
"""Create a StorageStore backed by the test database."""
from turnstone.core.tls_store import StorageStore
return StorageStore(get_storage())
def test_adapter_save_load_ca(store_adapter):
store_adapter.save_ca("test-ca", b"cert-pem", b"key-pem")
result = store_adapter.load_ca("test-ca")
assert result is not None
cert_pem, key_pem = result
assert cert_pem == b"cert-pem"
assert key_pem == b"key-pem"
def test_adapter_load_ca_missing(store_adapter):
assert store_adapter.load_ca("missing") is None
def test_adapter_save_load_cert(store_adapter):
lacme = pytest.importorskip("lacme")
now = datetime.now(UTC)
bundle = lacme.CertBundle(
domain="test.internal",
domains=("test.internal", "10.0.1.1"),
cert_pem=b"cert",
fullchain_pem=b"fullchain",
key_pem=b"key",
issued_at=now,
expires_at=now,
)
store_adapter.save_cert(bundle)
loaded = store_adapter.load_cert("test.internal")
assert loaded is not None
assert loaded.domain == "test.internal"
assert loaded.domains == ("test.internal", "10.0.1.1")
assert loaded.cert_pem == b"cert"
assert loaded.fullchain_pem == b"fullchain"
assert loaded.key_pem == b"key"
def test_adapter_list_certs(store_adapter):
lacme = pytest.importorskip("lacme")
now = datetime.now(UTC)
for name in ["alpha", "beta"]:
bundle = lacme.CertBundle(
domain=f"{name}.internal",
domains=(f"{name}.internal",),
cert_pem=b"c",
fullchain_pem=b"f",
key_pem=b"k",
issued_at=now,
expires_at=now,
)
store_adapter.save_cert(bundle)
certs = store_adapter.list_certs()
assert len(certs) == 2
assert certs[0].domain == "alpha.internal"
def test_adapter_load_cert_missing(store_adapter):
assert store_adapter.load_cert("missing") is None
def test_adapter_account_key_roundtrip(store_adapter):
"""Test account key save/load with real cryptography objects."""
pytest.importorskip("lacme")
from cryptography.hazmat.primitives.asymmetric import ec
key = ec.generate_private_key(ec.SECP256R1())
store_adapter.save_account_key(key)
loaded = store_adapter.load_account_key()
assert loaded is not None
# Verify it's a usable EC key
assert loaded.key_size == key.key_size
def test_adapter_account_key_missing(store_adapter):
assert store_adapter.load_account_key() is None
+2 -2
View File
@@ -104,8 +104,8 @@ class TestToolsMetadata:
"man": "page",
"web_fetch": "url",
"web_search": "query",
"task": "prompt",
"create_plan": "goal",
"task_agent": "prompt",
"plan_agent": "goal",
"memory": "name",
"recall": "query",
"notify": "message",
+124
View File
@@ -0,0 +1,124 @@
# turnstone.toml — shared bootstrap configuration
#
# This file is read once at startup. Values here are overridden by
# environment variables, which are in turn overridden by CLI flags.
#
# All sections are optional. Missing sections use binary defaults.
# Config file location precedence:
# 1. --config flag
# 2. $TURNSTONE_CONFIG env var
# 3. ~/.config/turnstone/config.toml
# --- LLM API (turnstone, node, eval) ---
[api]
# base_url = "" # API endpoint; empty = binary default
# api_key = "" # env: OPENAI_API_KEY or ANTHROPIC_API_KEY
# --- Default Model (turnstone, node, eval) ---
[model]
# name = "" # Model ID; empty = provider default (gpt-5 / claude-sonnet-4)
# temperature = 0.0 # 0 = provider default
# reasoning_effort = "" # "low", "medium", "high", "max"
# context_window = 0 # 0 = auto-detect from provider capabilities
# max_tokens = 0 # 0 = provider default
# --- Named Models (turnstone, node, eval) ---
# Define model aliases with per-model overrides. Useful for local model
# servers or mixing providers. Reference by name with --model flag.
#
# [models.local]
# name = "llama-3-70b"
# provider = "openai"
# base_url = "http://localhost:8000/v1"
# context_window = 8192
#
# [models.local.capabilities]
# supports_vision = false
# supports_web_search = false
#
# [models.claude]
# name = "claude-opus-4-6"
# provider = "anthropic"
# --- Database (turnstone, node, console) ---
[database]
# url = "" # postgres://user:pass@host/db or /path/to.db
# env: TURNSTONE_DB_URL
# SSL params (passed through to SQLAlchemy connection):
# sslmode = "prefer" # disable, allow, prefer, require, verify-ca, verify-full
# sslrootcert = "" # path to CA cert for verify-ca/verify-full
# sslcert = "" # path to client cert (mTLS)
# sslkey = "" # path to client key (mTLS)
# --- Redis (bridge, console, channel) ---
[redis]
# url = "" # redis://host:6379/0 or rediss://host:6380/0
# env: TURNSTONE_REDIS_URL
# TLS params (passed through to Redis connection):
# tls = false # enable TLS (also auto-enabled by rediss:// scheme)
# tls_ca = "" # path to CA cert
# tls_cert = "" # path to client cert (mTLS)
# tls_key = "" # path to client key (mTLS)
# --- Auth (node, console) ---
[auth]
# enabled = true # env: TURNSTONE_AUTH_ENABLED
# jwt_secret = "" # HS256 signing secret (min 32 bytes recommended)
# env: TURNSTONE_JWT_SECRET
# token = "" # Static config token for full access
# env: TURNSTONE_AUTH_TOKEN
# --- Logging (turnstone, node, console) ---
[log]
# level = "" # "debug", "info", "warn", "error"
# empty = binary default (warn for CLI, info for servers)
# env: TURNSTONE_LOG_LEVEL
# json = false # JSON output; auto-enabled when stderr is not a TTY
# --- Session (turnstone, node) ---
[session]
# instructions = "" # Default system message
# compact_max_tokens = 32768 # Max tokens for context compaction summary
# auto_compact_pct = 0.8 # Trigger compaction at this % of context window
# --- Tools (turnstone, node) ---
[tools]
# timeout = 120 # Tool execution timeout in seconds
# skip_permissions = false # Auto-approve all tool calls
# --- Judge (turnstone, node) ---
[judge]
# enabled = true # Enable intent validation
# confidence_threshold = 0.7 # Minimum confidence for heuristic verdicts
# output_guard = true # Scan tool output for security signals
# redact_secrets = true # Redact detected credentials in output
# --- Memory (turnstone, node) ---
[memory]
# relevance_k = 5 # Top-K memories for context injection
# fetch_limit = 50 # Max memories to fetch for ranking
# max_content = 32768 # Max memory content size in chars
# nudge_cooldown = 300 # Min seconds between metacognitive nudges
# nudges = true # Enable memory nudges
# --- MCP (turnstone, node) ---
[mcp]
# config_path = "" # Path to MCP servers config file (JSON)
# refresh_interval = 14400 # Refresh interval in seconds (default: 4h)
# --- Server (node, console) ---
[server]
# max_workstreams = 50 # Maximum concurrent workstreams per node
# env: TURNSTONE_MAX_WORKSTREAMS
+1 -1
View File
@@ -1,3 +1,3 @@
"""turnstone - Multi-node AI orchestration platform with tool use, agent routing, and cluster simulation."""
__version__ = "0.8.6"
__version__ = "0.9.0"
+216
View File
@@ -142,6 +142,189 @@ def _cmd_revoke_token(args: argparse.Namespace) -> None:
sys.exit(1)
# ---------------------------------------------------------------------------
# TLS commands
# ---------------------------------------------------------------------------
def _cmd_tls_bootstrap(args: argparse.Namespace) -> None:
"""Initialize CA and issue certs offline."""
try:
from lacme import CertificateAuthority, FileStore
except ImportError:
print("lacme not installed. Run: pip install turnstone[tls]", file=sys.stderr)
sys.exit(1)
import contextlib
import os
from pathlib import Path
out_dir = Path(args.out)
out_dir.mkdir(parents=True, exist_ok=True)
with contextlib.suppress(PermissionError):
os.chmod(out_dir, 0o700) # Restrict access — contains CA private key
store = FileStore(str(out_dir))
ca = CertificateAuthority(store, name="turnstone")
ca.init(cn="Turnstone CA", validity_days=3650)
print(f"CA initialized in {out_dir} (permissions: 0700)")
# Write CA cert to a well-known location
ca_cert_path = out_dir / "ca.pem"
ca_cert_path.write_bytes(ca.root_cert_pem)
with contextlib.suppress(PermissionError):
os.chmod(ca_cert_path, 0o644)
print(f"CA cert: {ca_cert_path}")
# Issue certs for requested domains
for domain in args.issue:
bundle = ca.issue([domain], validity_hours=48)
store.save_cert(bundle)
cert_dir = out_dir / "certs" / domain
print(f"Issued: {domain} -> {cert_dir}")
print(f"\nBootstrap complete. {len(args.issue)} cert(s) issued.")
print(f"CA and certs written to: {out_dir}")
def _cmd_tls_issue(args: argparse.Namespace) -> None:
"""Request a cert from the console's ACME endpoint."""
try:
from lacme import SyncClient
except ImportError:
print("lacme not installed. Run: pip install turnstone[tls]", file=sys.stderr)
sys.exit(1)
import os
from pathlib import Path
console_url = args.console_url
if not console_url:
console_url = _discover_console_url()
domains = [args.domain] + args.san
directory_url = f"{console_url}/acme/directory"
print(f"Requesting cert for {domains} from {directory_url}")
client = SyncClient(
directory_url=directory_url,
allow_insecure=True,
)
bundle = client.issue(domains)
out_dir = Path(args.out)
out_dir.mkdir(parents=True, exist_ok=True)
(out_dir / "cert.pem").write_bytes(bundle.cert_pem)
(out_dir / "fullchain.pem").write_bytes(bundle.fullchain_pem)
(out_dir / "key.pem").write_bytes(bundle.key_pem)
os.chmod(out_dir / "key.pem", 0o600)
print(f"Certificate written to {out_dir}/")
print(" cert.pem (leaf certificate)")
print(" fullchain.pem (cert + chain)")
print(" key.pem (private key, 0600)")
def _cmd_tls_ca_cert(args: argparse.Namespace) -> None:
"""Download the CA root certificate from the console."""
import httpx
console_url = args.console_url
if not console_url:
console_url = _discover_console_url()
# Use plain HTTP for bootstrap (node may not have CA cert yet)
# WARNING: This is trust-on-first-use (TOFU) — verify the fingerprint
base = console_url.replace("https://", "http://")
url = f"{base}/acme/ca.pem"
print(f"Fetching CA cert from {url}")
print("WARNING: Fetching over plain HTTP — verify the fingerprint below")
resp = httpx.get(url)
resp.raise_for_status()
# Show fingerprint for out-of-band verification
import hashlib
fingerprint = hashlib.sha256(resp.content).hexdigest()
print(f"CA cert SHA-256: {fingerprint}")
from pathlib import Path
Path(args.out).write_bytes(resp.content)
print(f"CA cert written to {args.out}")
def _cmd_tls_list(args: argparse.Namespace) -> None:
"""List certificates from the console."""
import httpx
console_url = args.console_url
if not console_url:
console_url = _discover_console_url()
url = f"{console_url}/v1/api/admin/tls/certs"
headers = {}
token = getattr(args, "auth_token", "") or _get_config_token()
if token:
headers["Authorization"] = f"Bearer {token}"
resp = httpx.get(url, headers=headers)
resp.raise_for_status()
data = resp.json()
certs = data.get("certs", [])
if not certs:
print("No certificates issued.")
return
print(f"{'DOMAIN':<30s} {'ISSUED':<22s} {'EXPIRES':<22s}")
print("-" * 74)
for c in certs:
print(f"{c['domain']:<30s} {c['issued_at']:<22s} {c['expires_at']:<22s}")
def _get_config_token() -> str:
"""Try to load auth token from config.toml or environment."""
token = os.environ.get("TURNSTONE_AUTH_TOKEN", "")
if token:
return token
try:
from turnstone.core.config import load_config
cfg = load_config("auth")
return str(cfg.get("token", ""))
except Exception:
return ""
def _discover_console_url() -> str:
"""Discover console URL from the services table."""
from turnstone.core.storage import get_storage
try:
storage = get_storage()
except Exception:
print(
"No storage configured. Use --console-url or run from a "
"directory with a turnstone database.",
file=sys.stderr,
)
sys.exit(1)
consoles = storage.list_services("console", max_age_seconds=3600)
if not consoles:
print(
"No console found in services table. Use --console-url explicitly.",
file=sys.stderr,
)
sys.exit(1)
return consoles[0]["url"]
# ---------------------------------------------------------------------------
# Entry point
# ---------------------------------------------------------------------------
def main() -> None:
"""Entry point for turnstone-admin CLI."""
parser = argparse.ArgumentParser(
@@ -171,6 +354,35 @@ def main() -> None:
p_rt = sub.add_parser("revoke-token", help="Revoke an API token")
p_rt.add_argument("--token-id", required=True, help="Token ID to revoke")
# TLS subcommands
p_bootstrap = sub.add_parser(
"tls-bootstrap",
help="Initialize CA and issue certs offline (no running console needed)",
)
p_bootstrap.add_argument("--out", required=True, help="Output directory for PEM files")
p_bootstrap.add_argument(
"--issue",
action="append",
default=[],
help="Domain to issue cert for (repeatable)",
)
p_issue = sub.add_parser("tls-issue", help="Request cert from console ACME")
p_issue.add_argument("domain", help="Primary domain for the certificate")
p_issue.add_argument("--san", action="append", default=[], help="Additional SAN (repeatable)")
p_issue.add_argument("--out", default=".", help="Output directory for PEM files")
p_issue.add_argument(
"--console-url", default="", help="Console URL (discovered from DB if empty)"
)
p_cacert = sub.add_parser("tls-ca-cert", help="Download CA root certificate")
p_cacert.add_argument("--out", default="ca.pem", help="Output file path")
p_cacert.add_argument("--console-url", default="", help="Console URL")
p_tlslist = sub.add_parser("tls-list", help="List issued certificates")
p_tlslist.add_argument("--console-url", default="", help="Console URL")
p_tlslist.add_argument("--auth-token", default="", help="Auth token for admin API")
args = parser.parse_args()
if not args.command:
parser.print_help()
@@ -182,5 +394,9 @@ def main() -> None:
"list-users": _cmd_list_users,
"list-tokens": _cmd_list_tokens,
"revoke-token": _cmd_revoke_token,
"tls-bootstrap": _cmd_tls_bootstrap,
"tls-issue": _cmd_tls_issue,
"tls-ca-cert": _cmd_tls_ca_cert,
"tls-list": _cmd_tls_list,
}
dispatch[args.command](args)
+33
View File
@@ -843,6 +843,39 @@ CONSOLE_ENDPOINTS: list[EndpointSpec] = [
error_codes=[400],
tags=["Admin"],
),
# --- Admin: TLS / ACME ---
EndpointSpec(
"/v1/api/admin/tls/ca",
"GET",
"CA status: initialization state, CN, cert count, cert inventory",
tags=["Admin"],
),
EndpointSpec(
"/v1/api/admin/tls/ca.pem",
"GET",
"Download CA root certificate (PEM format)",
tags=["Admin"],
),
EndpointSpec(
"/v1/api/admin/tls/certs",
"GET",
"List all issued TLS certificates",
tags=["Admin"],
),
EndpointSpec(
"/v1/api/admin/tls/certs/{domain}/renew",
"POST",
"Force-renew a certificate by domain",
error_codes=[404, 500],
tags=["Admin"],
),
EndpointSpec(
"/v1/api/admin/tls/certs/{domain}",
"DELETE",
"Delete a certificate by domain",
error_codes=[404],
tags=["Admin"],
),
# --- Observability ---
EndpointSpec(
"/health",
+21 -1
View File
@@ -58,6 +58,11 @@ def main() -> None:
help="HTTP server port (default: $TURNSTONE_CHANNEL_PORT or 8091)",
)
# -- TLS -----------------------------------------------------------------
parser.add_argument("--ssl-certfile", default=None, help="SSL certificate file for HTTPS")
parser.add_argument("--ssl-keyfile", default=None, help="SSL private key file")
parser.add_argument("--ssl-ca-certs", default=None, help="SSL CA certs for client verification")
# -- Auth ----------------------------------------------------------------
parser.add_argument(
"--auth-token",
@@ -189,7 +194,8 @@ def main() -> None:
advertise_host = socket.gethostname()
else:
advertise_host = args.http_host
advertise_url = f"http://{advertise_host}:{args.http_port}"
scheme = "https" if args.ssl_certfile else "http"
advertise_url = f"{scheme}://{advertise_host}:{args.http_port}"
service_url = advertise_url
# Register in service registry
@@ -209,11 +215,25 @@ def main() -> None:
except Exception:
log.exception("channel.heartbeat_failed")
# TLS: use cert files if available (from bootstrap or TLSClient)
ssl_certfile = getattr(args, "ssl_certfile", None)
ssl_keyfile = getattr(args, "ssl_keyfile", None)
ssl_ca_certs = getattr(args, "ssl_ca_certs", None)
if bool(ssl_certfile) != bool(ssl_keyfile):
print(
"Both --ssl-certfile and --ssl-keyfile are required for TLS",
file=sys.stderr,
)
sys.exit(1)
uv_config = uvicorn.Config(
channel_app,
host=args.http_host,
port=args.http_port,
log_level="warning",
ssl_certfile=ssl_certfile,
ssl_keyfile=ssl_keyfile,
ssl_ca_certs=ssl_ca_certs,
)
server = uvicorn.Server(uv_config)
+23
View File
@@ -60,6 +60,8 @@ class ClusterCollector:
http_timeout: float = 30.0,
auth_token: str = "",
token_manager: ServiceTokenManager | None = None,
tls_verify: Any = True,
tls_cert: tuple[str, str] | None = None,
):
self._broker = broker
self._prefix = prefix
@@ -86,12 +88,33 @@ class ClusterCollector:
max_connections=max_poll_workers + 10,
max_keepalive_connections=min(max_poll_workers, 200),
),
verify=tls_verify,
cert=tls_cert,
)
# SSE fan-out to browser clients
self._listeners: list[queue.Queue[dict[str, Any]]] = []
self._listeners_lock = threading.Lock()
def upgrade_tls(self, tls_verify: Any = True, tls_cert: tuple[str, str] | None = None) -> None:
"""Replace the httpx client with one using mTLS context."""
old = self._http_client
self._http_client = httpx.Client(
timeout=httpx.Timeout(
connect=10, read=self._http_timeout, write=5, pool=self._http_timeout
),
limits=httpx.Limits(
max_connections=self._max_poll_workers + 10,
max_keepalive_connections=min(self._max_poll_workers, 200),
),
verify=tls_verify,
cert=tls_cert,
)
# Don't close old client — concurrent _fetch_node() threads may still
# be using it. It will be GC'd once all references are released, and
# the current client is closed in stop().
del old
# -- lifecycle -----------------------------------------------------------
def start(self) -> None:
+365 -7
View File
@@ -37,7 +37,7 @@ from starlette.staticfiles import StaticFiles
from turnstone.api.console_spec import build_console_spec
from turnstone.api.docs import make_docs_handler, make_openapi_handler
from turnstone.console.collector import ClusterCollector
from turnstone.core.auth import JWT_AUD_CONSOLE, AuthMiddleware
from turnstone.core.auth import JWT_AUD_CONSOLE, JWT_AUD_SERVER, AuthMiddleware, create_jwt
if TYPE_CHECKING:
from collections.abc import AsyncGenerator
@@ -133,15 +133,36 @@ _CONSOLE_PROXY_STYLE = "<style>.dashboard-overlay{top:32px!important}</style>"
_VALID_NODE_ID = re.compile(r"^[a-zA-Z0-9._-]+$")
_PROXY_JWT_EXPIRY_SECONDS = 300 # 5 min — ample for any request round-trip
def _proxy_auth_headers(request: Request) -> dict[str, str]:
"""Build auth headers for proxied requests to upstream servers.
Uses the service proxy token (``JWT_AUD_SERVER``) so the upstream node
accepts the request. The user's console-audience JWT is *not* forwarded
it would be rejected by the server's audience validation.
Mints a short-lived JWT carrying the real user's identity and scopes
so the upstream server records correct audit attribution and enforces
scope narrowing. Falls back to the ServiceTokenManager when no user
context is available.
"""
# Prefer the auto-rotating ServiceTokenManager when available
auth_result = getattr(getattr(request, "state", None), "auth_result", None)
jwt_secret: str = getattr(request.app.state, "jwt_secret", "")
if auth_result is not None and auth_result.user_id and jwt_secret:
token = create_jwt(
user_id=auth_result.user_id,
scopes=auth_result.scopes,
source="console-proxy",
secret=jwt_secret,
audience=JWT_AUD_SERVER,
permissions=auth_result.permissions,
expiry_seconds=_PROXY_JWT_EXPIRY_SECONDS,
)
return {"Authorization": f"Bearer {token}"}
# Fallback: service identity (no user context).
# When auth is disabled on the console, auth_result is None, so all proxied
# requests use the full-privilege service identity. This is safe only when
# the upstream server also has auth disabled.
mgr = getattr(request.app.state, "proxy_token_mgr", None)
if mgr is not None:
return dict(mgr.bearer_header)
@@ -407,6 +428,9 @@ async def create_workstream(request: Request) -> JSONResponse:
from turnstone.mq.protocol import CreateWorkstreamMessage
auth = getattr(getattr(request, "state", None), "auth_result", None)
uid: str = getattr(auth, "user_id", "") or ""
# General pool — push to shared queue, any bridge picks it up
if node_id == "pool":
msg = CreateWorkstreamMessage(
@@ -415,6 +439,7 @@ async def create_workstream(request: Request) -> JSONResponse:
initial_message=initial_message,
skill=skill,
resume_ws=resume_ws,
user_id=uid,
)
broker.push_inbound(msg.to_json())
log.debug("Pool dispatch: correlation_id=%s name=%r", msg.correlation_id, name)
@@ -444,6 +469,7 @@ async def create_workstream(request: Request) -> JSONResponse:
initial_message=initial_message,
skill=skill,
resume_ws=resume_ws,
user_id=uid,
)
broker.push_inbound(msg.to_json(), node_id=node_id)
@@ -701,18 +727,25 @@ async def _lifespan(app: Starlette) -> AsyncGenerator[None, None]:
config_store.get("cluster.node_fan_out_limit") if config_store else _NODE_FAN_OUT_LIMIT
)
app.state.fan_out_limit = fan_out
# Build mTLS context for proxy clients if TLS is enabled
_tls_mgr = getattr(app.state, "tls_manager", None)
_proxy_ssl = _tls_mgr.get_client_ssl_context() if _tls_mgr and _tls_mgr.ca_initialized else None
_proxy_verify: Any = _proxy_ssl if _proxy_ssl else True
app.state.proxy_client = httpx.AsyncClient(
timeout=30,
limits=httpx.Limits(
max_connections=fan_out + 50,
max_keepalive_connections=min(fan_out // 4, 100),
),
verify=_proxy_verify,
)
app.state.proxy_sse_client = httpx.AsyncClient(
timeout=httpx.Timeout(connect=5, read=30, write=5, pool=5),
limits=httpx.Limits(
max_connections=1100, max_keepalive_connections=100, keepalive_expiry=30
),
verify=_proxy_verify,
)
# Start scheduler if configured
scheduler = getattr(app.state, "scheduler", None)
@@ -743,8 +776,84 @@ async def _lifespan(app: Starlette) -> AsyncGenerator[None, None]:
"OIDC JWKS prefetch failed — will retry on first login",
exc_info=True,
)
# Register console in service registry so other services can discover it
console_url = getattr(app.state, "console_url", "")
_console_heartbeat_task: Any = None
if console_url and storage:
try:
storage.register_service("console", "console", console_url)
# Periodic heartbeat to keep the registration alive
import asyncio
async def _console_heartbeat() -> None:
while True:
await asyncio.sleep(30)
try:
storage.heartbeat_service("console", "console")
except Exception:
log.warning("console.heartbeat_failed", exc_info=True)
_console_heartbeat_task = asyncio.create_task(_console_heartbeat())
except Exception:
log.warning("Failed to register console service", exc_info=True)
# TLS: init CA, issue console certs, start renewal
tls_mgr = getattr(app.state, "tls_manager", None)
if tls_mgr is not None:
import socket
try:
if not tls_mgr.ca_initialized:
await tls_mgr.init_ca()
hostname = socket.getfqdn()
cert_hostnames = [hostname, "localhost", "127.0.0.1"]
extra_sans = os.environ.get("TURNSTONE_TLS_SANS", "")
if extra_sans:
cert_hostnames.extend(s.strip() for s in extra_sans.split(",") if s.strip())
await tls_mgr.issue_console_certs(cert_hostnames)
await tls_mgr.start_renewal()
# Re-create proxy clients with mTLS context now that certs are ready
client_ctx = tls_mgr.get_client_ssl_context()
if client_ctx:
await app.state.proxy_client.aclose()
await app.state.proxy_sse_client.aclose()
app.state.proxy_client = httpx.AsyncClient(
timeout=30,
limits=httpx.Limits(
max_connections=app.state.fan_out_limit + 50,
max_keepalive_connections=min(app.state.fan_out_limit // 4, 100),
),
verify=client_ctx,
)
app.state.proxy_sse_client = httpx.AsyncClient(
timeout=httpx.Timeout(connect=5, read=30, write=5, pool=5),
limits=httpx.Limits(
max_connections=1100,
max_keepalive_connections=100,
keepalive_expiry=30,
),
verify=client_ctx,
)
# Upgrade collector httpx client for mTLS node polling
app.state.collector.upgrade_tls(tls_verify=client_ctx)
log.info("tls.proxy_clients.upgraded")
except Exception:
log.warning("TLS initialization failed — continuing without TLS", exc_info=True)
yield
# Shutdown
if _console_heartbeat_task is not None:
_console_heartbeat_task.cancel()
# Deregister console from services table
if console_url and storage:
try:
storage.deregister_service("console", "console")
except Exception:
log.debug("console.deregister_failed", exc_info=True)
tls_mgr = getattr(app.state, "tls_manager", None)
if tls_mgr is not None:
await tls_mgr.stop_renewal()
if scheduler is not None:
scheduler.stop()
await app.state.proxy_sse_client.aclose()
@@ -2748,6 +2857,16 @@ async def admin_usage(request: Request) -> JSONResponse:
group_by=group_by,
)
# Resolve user_id hex → username for display when grouped by user
if group_by == "user" and breakdown:
uid_to_name: dict[str, str] = {}
for u in storage.list_users():
uid_to_name[u["user_id"]] = u.get("username") or u["user_id"]
for row in breakdown:
raw_key = row.get("key", "")
if raw_key and raw_key in uid_to_name:
row["key"] = uid_to_name[raw_key]
return JSONResponse({"summary": summary, "breakdown": breakdown})
@@ -2792,6 +2911,16 @@ async def admin_audit(request: Request) -> JSONResponse:
until=until,
)
# Resolve user_id hex → username for display
if events:
uid_to_name: dict[str, str] = {}
for u in storage.list_users():
uid_to_name[u["user_id"]] = u.get("username") or u["user_id"]
for ev in events:
raw_uid = ev.get("user_id", "")
if raw_uid and raw_uid in uid_to_name:
ev["username"] = uid_to_name[raw_uid]
return JSONResponse({"events": events, "total": total})
@@ -4534,6 +4663,161 @@ async def admin_import_mcp_config(request: Request) -> JSONResponse:
return JSONResponse({"imported": imported, "skipped": skipped, "errors": errors})
# ---------------------------------------------------------------------------
# TLS endpoints
# ---------------------------------------------------------------------------
async def tls_ca_cert(request: Request) -> Response:
"""GET /v1/api/admin/tls/ca.pem — Download CA root certificate."""
from turnstone.core.auth import require_permission
err = require_permission(request, "admin.settings")
if err:
return err
mgr = getattr(request.app.state, "tls_manager", None)
if mgr is None or not mgr.ca_initialized:
return JSONResponse({"error": "TLS not enabled"}, status_code=404)
return Response(
content=mgr.get_root_cert_pem(),
media_type="application/x-pem-file",
headers={"Content-Disposition": "attachment; filename=turnstone-ca.pem"},
)
async def tls_ca_status(request: Request) -> JSONResponse:
"""GET /v1/api/admin/tls/ca — CA status."""
from turnstone.core.auth import require_permission
err = require_permission(request, "admin.settings")
if err:
return err
mgr = getattr(request.app.state, "tls_manager", None)
if mgr is None or not mgr.ca_initialized:
return JSONResponse({"enabled": False})
from turnstone.console.tls import _CA_CN
certs = mgr.list_certs()
return JSONResponse(
{
"enabled": True,
"ca_cn": _CA_CN,
"cert_count": len(certs),
"certs": [
{
"domain": c.domain,
"issued_at": c.issued_at.isoformat(),
"expires_at": c.expires_at.isoformat(),
}
for c in certs
],
},
)
async def tls_list_certs(request: Request) -> JSONResponse:
"""GET /v1/api/admin/tls/certs — List issued certificates."""
from turnstone.core.auth import require_permission
err = require_permission(request, "admin.settings")
if err:
return err
mgr = getattr(request.app.state, "tls_manager", None)
if mgr is None or not mgr.ca_initialized:
return JSONResponse({"certs": []})
certs = mgr.list_certs()
return JSONResponse(
{
"certs": [
{
"domain": c.domain,
"domains": list(c.domains),
"issued_at": c.issued_at.isoformat(),
"expires_at": c.expires_at.isoformat(),
}
for c in certs
],
},
)
async def tls_renew_cert(request: Request) -> JSONResponse:
"""POST /v1/api/admin/tls/certs/{domain}/renew — Force cert renewal."""
from turnstone.core.auth import require_permission
err = require_permission(request, "admin.settings")
if err:
return err
mgr = getattr(request.app.state, "tls_manager", None)
if mgr is None or not mgr.ca_initialized:
return JSONResponse({"error": "TLS not enabled"}, status_code=404)
domain = request.path_params["domain"]
try:
bundle = mgr.renew_cert(domain)
return JSONResponse(
{
"domain": bundle.domain,
"issued_at": bundle.issued_at.isoformat(),
"expires_at": bundle.expires_at.isoformat(),
},
)
except ValueError as e:
return JSONResponse({"error": str(e)}, status_code=404)
except Exception as e:
return JSONResponse({"error": str(e)}, status_code=500)
async def tls_delete_cert(request: Request) -> JSONResponse:
"""DELETE /v1/api/admin/tls/certs/{domain} — Delete a certificate."""
from turnstone.core.auth import require_permission
err = require_permission(request, "admin.settings")
if err:
return err
mgr = getattr(request.app.state, "tls_manager", None)
if mgr is None or not mgr.ca_initialized:
return JSONResponse({"error": "TLS not enabled"}, status_code=404)
domain = request.path_params["domain"]
if not mgr.delete_cert(domain):
return JSONResponse({"error": f"No cert for {domain}"}, status_code=404)
return JSONResponse({"deleted": domain})
# ---------------------------------------------------------------------------
# ConfigStore env seeding
# ---------------------------------------------------------------------------
def _seed_config_from_env(config_store: Any, storage: Any) -> None:
"""Seed ConfigStore settings from environment variables.
Checks for ``TURNSTONE_{SECTION}_{KEY}`` env vars and writes them
to ConfigStore if they aren't already set. This allows container
deployments to configure settings before the admin UI is available.
Only seeds known settings from the registry to avoid storing garbage.
Uses config_store.set() for proper validation, serialization, and
cache invalidation.
"""
from turnstone.core.settings_registry import SETTINGS
for key in SETTINGS:
env_name = "TURNSTONE_" + key.replace(".", "_").upper()
env_val = os.environ.get(env_name)
if env_val is None:
continue
# Only seed if not already stored (check raw storage to avoid
# config_store cache, which may not reflect DB state yet)
existing = storage.get_system_setting(key)
if existing is not None:
continue
try:
config_store.set(key, env_val, changed_by="env")
log.info("config.seeded_from_env: %s from %s", key, env_name)
except Exception:
log.warning("config.seed_failed: %s from %s", key, env_name, exc_info=True)
# ---------------------------------------------------------------------------
# App factory
# ---------------------------------------------------------------------------
@@ -4549,6 +4833,8 @@ def create_app(
proxy_auth_token: str = "",
proxy_token_mgr: Any = None,
cors_origins: list[str] | None = None,
tls_manager: Any = None,
console_url: str = "",
) -> Starlette:
"""Build the Starlette ASGI application for the console dashboard."""
_spec = build_console_spec()
@@ -4771,6 +5057,20 @@ def create_app(
admin_rescan_skill,
methods=["POST"],
),
# TLS / ACME
Route("/api/admin/tls/ca", tls_ca_status),
Route("/api/admin/tls/ca.pem", tls_ca_cert),
Route("/api/admin/tls/certs", tls_list_certs),
Route(
"/api/admin/tls/certs/{domain}/renew",
tls_renew_cert,
methods=["POST"],
),
Route(
"/api/admin/tls/certs/{domain}",
tls_delete_cert,
methods=["DELETE"],
),
],
),
Route("/health", health),
@@ -4796,6 +5096,15 @@ def create_app(
app.state.auth_storage = auth_storage
app.state.proxy_auth_token = proxy_auth_token
app.state.proxy_token_mgr = proxy_token_mgr
app.state.console_url = console_url
app.state.tls_manager = tls_manager
# Mount ACME responder whenever a TLS manager is configured.
# ACMEResponder (lacme 1.0.2+) serves /ca.pem natively.
if tls_manager is not None:
from starlette.routing import Mount as RouteMount
app.routes.insert(0, RouteMount("/acme", app=tls_manager.get_responder()))
from turnstone.core.auth import LoginRateLimiter
@@ -4943,7 +5252,15 @@ def main() -> None:
db_backend = os.environ.get("TURNSTONE_DB_BACKEND", "sqlite")
db_url = os.environ.get("TURNSTONE_DB_URL", "")
db_path = os.environ.get("TURNSTONE_DB_PATH", "")
auth_storage = init_storage(db_backend, path=db_path, url=db_url)
auth_storage = init_storage(
db_backend,
path=db_path,
url=db_url,
sslmode=os.environ.get("TURNSTONE_DB_SSLMODE", ""),
sslrootcert=os.environ.get("TURNSTONE_DB_SSLROOTCERT", ""),
sslcert=os.environ.get("TURNSTONE_DB_SSLCERT", ""),
sslkey=os.environ.get("TURNSTONE_DB_SSLKEY", ""),
)
except Exception:
log.info("Console storage not available — admin API disabled, JWT-only auth")
@@ -4968,6 +5285,45 @@ def main() -> None:
cors_origins = parse_cors_origins()
# TLS: initialize manager if enabled
tls_mgr = None
# Console URL for service registration — other services use this to discover the console.
# Precedence: TURNSTONE_CONSOLE_URL env > auto-detect from bind address.
# In Docker Compose, set TURNSTONE_CONSOLE_URL to the service name (e.g. http://console:8090).
import socket as _socket
_console_url_env = os.environ.get("TURNSTONE_CONSOLE_URL", "")
if _console_url_env:
console_url = _console_url_env
else:
_advertise_host = args.host
if _advertise_host in ("0.0.0.0", "::", ""):
_advertise_host = _socket.getfqdn()
console_url = f"http://{_advertise_host}:{args.port}"
if auth_storage:
try:
from turnstone.core.config_store import ConfigStore
_cs = ConfigStore(auth_storage)
# Seed ConfigStore from env vars (TURNSTONE_{SECTION}_{KEY})
_seed_config_from_env(_cs, auth_storage)
if _cs.get("tls.enabled"):
from turnstone.console.tls import TLSManager
tls_mgr = TLSManager(auth_storage, config_store=_cs)
# Init CA before create_app so ACME responder can be mounted
import asyncio
asyncio.run(tls_mgr.init_ca())
# Upgrade scheme to https if no explicit URL was provided
if not _console_url_env:
console_url = console_url.replace("http://", "https://")
log.info("TLS enabled")
except ImportError:
log.warning("TLS enabled but lacme not installed — pip install turnstone[tls]")
except Exception:
log.warning("TLS initialization failed", exc_info=True)
app = create_app(
collector=collector,
broker=broker,
@@ -4977,9 +5333,11 @@ def main() -> None:
proxy_auth_token=proxy_token if proxy_token_mgr is None else "",
proxy_token_mgr=proxy_token_mgr,
cors_origins=cors_origins,
tls_manager=tls_mgr,
console_url=console_url,
)
log.info("Console starting on http://%s:%s", args.host, args.port)
log.info("Console starting on %s", console_url)
if auth_config.enabled:
log.info("Auth: enabled (%d config token(s))", len(auth_config.tokens))
print("Press Ctrl+C to stop.")
+177
View File
@@ -64,6 +64,7 @@ function showAdmin() {
audit: "admin.audit",
memories: "admin.memories",
settings: "admin.settings",
tls: "admin.settings",
mcp: "admin.mcp",
};
if (perms) {
@@ -193,6 +194,7 @@ function switchAdminTab(tab) {
"audit",
"memories",
"settings",
"tls",
"mcp",
];
for (var p = 0; p < panels.length; p++) {
@@ -215,6 +217,7 @@ function switchAdminTab(tab) {
}
if (tab === "memories") loadAdminMemories();
if (tab === "settings") loadSettings();
if (tab === "tls") loadTlsCerts();
if (tab === "mcp") loadAdminMcp();
// Update breadcrumb with active tab label
@@ -2083,6 +2086,180 @@ function _settingsSectionLabel(section) {
return labels[section] || section;
}
// ---------------------------------------------------------------------------
// TLS tab
// ---------------------------------------------------------------------------
function loadTlsCerts() {
var statusEl = document.getElementById("tls-ca-status");
var listEl = document.getElementById("tls-cert-list");
if (!statusEl || !listEl) return;
// Fetch CA status and cert list in parallel
Promise.all([
authFetch("/v1/api/admin/tls/ca").then(function (r) {
if (!r.ok) throw new Error("Failed");
return r.json();
}),
authFetch("/v1/api/admin/tls/certs").then(function (r) {
if (!r.ok) return { certs: [] };
return r.json();
}),
])
.then(function (results) {
var data = results[0];
var certData = results[1];
while (statusEl.firstChild) statusEl.removeChild(statusEl.firstChild);
while (listEl.firstChild) listEl.removeChild(listEl.firstChild);
if (!data.enabled) {
var msg = document.createElement("div");
msg.className = "dashboard-empty";
msg.textContent =
"TLS is not enabled. Set tls.enabled = true in Settings.";
statusEl.appendChild(msg);
return;
}
// CA status bar
var bar = document.createElement("div");
bar.className = "tls-ca-bar";
var caLabel = document.createElement("span");
caLabel.textContent = "CA: " + data.ca_cn;
var countLabel = document.createElement("span");
countLabel.textContent = "Certificates: " + data.cert_count;
bar.appendChild(caLabel);
bar.appendChild(countLabel);
statusEl.appendChild(bar);
var certs = certData.certs || [];
if (certs.length === 0) {
var empty = document.createElement("div");
empty.className = "dashboard-empty";
empty.textContent = "No certificates issued yet.";
listEl.appendChild(empty);
return;
}
// Cert rows
certs.forEach(function (c) {
var row = document.createElement("div");
row.className = "admin-row";
row.setAttribute("role", "listitem");
var colDomain = document.createElement("span");
colDomain.className = "admin-col";
colDomain.textContent = c.domain;
var colSans = document.createElement("span");
colSans.className = "admin-col";
colSans.textContent = (c.domains || [c.domain]).join(", ");
var colIssued = document.createElement("span");
colIssued.className = "admin-col";
colIssued.textContent = (c.issued_at || "")
.slice(0, 16)
.replace("T", " ");
var colExpires = document.createElement("span");
colExpires.className = "admin-col";
var expires = new Date(c.expires_at);
var isExpired = expires < new Date();
colExpires.textContent =
(isExpired ? "EXPIRED " : "") +
(c.expires_at || "").slice(0, 16).replace("T", " ");
if (isExpired) colExpires.style.color = "var(--red)";
var colActions = document.createElement("span");
colActions.className = "admin-col admin-col-actions";
var renewBtn = document.createElement("button");
renewBtn.className = "admin-btn-action";
renewBtn.textContent = "Renew";
renewBtn.setAttribute(
"aria-label",
"Renew certificate for " + c.domain,
);
renewBtn.onclick = function () {
tlsRenewCert(c.domain);
};
var deleteBtn = document.createElement("button");
deleteBtn.className = "admin-btn-danger";
deleteBtn.textContent = "Delete";
deleteBtn.setAttribute(
"aria-label",
"Delete certificate for " + c.domain,
);
deleteBtn.onclick = function () {
tlsDeleteCert(c.domain);
};
colActions.appendChild(renewBtn);
colActions.appendChild(deleteBtn);
row.appendChild(colDomain);
row.appendChild(colSans);
row.appendChild(colIssued);
row.appendChild(colExpires);
row.appendChild(colActions);
listEl.appendChild(row);
});
})
.catch(function () {
while (statusEl.firstChild) statusEl.removeChild(statusEl.firstChild);
while (listEl.firstChild) listEl.removeChild(listEl.firstChild);
var errMsg = document.createElement("div");
errMsg.className = "dashboard-empty";
errMsg.textContent = "Failed to load TLS status";
statusEl.appendChild(errMsg);
});
}
function tlsRenewCert(domain) {
showConfirmModal(
"Renew Certificate",
"Force renew certificate for \u2018" + domain + "\u2019?",
"Renew",
function () {
authFetch(
"/v1/api/admin/tls/certs/" + encodeURIComponent(domain) + "/renew",
{ method: "POST" },
)
.then(function (r) {
if (!r.ok) throw new Error("Renew failed");
showToast("Certificate renewed for " + domain);
loadTlsCerts();
})
.catch(function () {
showToast("Failed to renew certificate", "error");
});
},
);
}
function tlsDeleteCert(domain) {
showConfirmModal(
"Delete Certificate",
"Delete certificate for \u2018" + domain + "\u2019? This cannot be undone.",
"Delete",
function () {
authFetch("/v1/api/admin/tls/certs/" + encodeURIComponent(domain), {
method: "DELETE",
})
.then(function (r) {
if (!r.ok) throw new Error("Delete failed");
showToast("Certificate deleted for " + domain);
loadTlsCerts();
})
.catch(function () {
showToast("Failed to delete certificate", "error");
});
},
);
}
// ---------------------------------------------------------------------------
// Settings tab
// ---------------------------------------------------------------------------
function loadSettings() {
var el = document.getElementById("admin-settings-content");
if (!el) return;
+3 -1
View File
@@ -1838,7 +1838,9 @@ function _renderGovAudit(events, total) {
_relativeTime(ev.timestamp) +
"</span>" +
'<span class="admin-col admin-col-auser">' +
escapeHtml(ev.user_id ? ev.user_id.slice(0, 8) : "\u2014") +
escapeHtml(
ev.username || (ev.user_id ? ev.user_id.slice(0, 8) : "\u2014"),
) +
"</span>" +
'<span class="admin-col admin-col-aaction"><span class="' +
actionCls +
+19
View File
@@ -110,6 +110,7 @@
<div class="admin-sidebar-group" data-group="system" role="group" aria-label="System">
<div class="admin-sidebar-group-label" aria-hidden="true">System</div>
<button id="tab-settings" class="admin-nav" data-tab="settings" role="tab" aria-selected="false" aria-controls="admin-settings" tabindex="-1" onclick="switchAdminTab('settings')">Settings</button>
<button id="tab-tls" class="admin-nav" data-tab="tls" role="tab" aria-selected="false" aria-controls="admin-tls" tabindex="-1" onclick="switchAdminTab('tls')">TLS</button>
</div>
</nav>
<div id="admin-sidebar-backdrop" class="admin-sidebar-backdrop" aria-hidden="true"></div>
@@ -404,6 +405,24 @@
</div>
</div>
<!-- TLS Tab -->
<div id="admin-tls" class="admin-panel" role="tabpanel" aria-labelledby="tab-tls" style="display:none">
<div class="admin-toolbar">
<span class="section-header" style="margin:0">TLS / CERTIFICATES</span>
</div>
<div id="tls-ca-status"></div>
<div class="admin-colheaders admin-colheaders-tls" aria-hidden="true">
<span class="admin-col">DOMAIN</span>
<span class="admin-col">SANS</span>
<span class="admin-col">ISSUED</span>
<span class="admin-col">EXPIRES</span>
<span class="admin-col admin-col-actions">ACTIONS</span>
</div>
<div id="tls-cert-list" role="list" aria-label="TLS certificates" aria-live="polite">
<div class="dashboard-empty">Loading&hellip;</div>
</div>
</div>
<div id="admin-mcp" class="admin-panel" role="tabpanel" aria-labelledby="tab-mcp" style="display:none">
<div class="admin-toolbar">
<span class="section-header">MCP</span>
+13
View File
@@ -948,6 +948,19 @@
grid-template-columns: 100px 1fr 100px 80px;
}
/* TLS grid: DOMAIN | SANS | ISSUED | EXPIRES | ACTIONS */
#admin-tls .admin-colheaders,
#admin-tls .admin-row {
grid-template-columns: 160px 1fr 130px 150px 120px;
}
.tls-ca-bar {
display: flex;
gap: 2rem;
margin-bottom: 0.75rem;
font-size: 12px;
color: var(--fg-dim);
}
/* Scope badges */
.scope-badge {
display: inline-block;
+386
View File
@@ -0,0 +1,386 @@
"""TLS Manager — Certificate Authority and ACME server for the console.
Owns the lacme CertificateAuthority, ACMEResponder, and RenewalManager
lifecycle. When TLS is enabled, the console acts as the cluster's internal
CA and ACME server, issuing short-lived mTLS certificates to all services.
"""
from __future__ import annotations
from typing import TYPE_CHECKING, Any
import structlog
if TYPE_CHECKING:
import ssl
from starlette.types import ASGIApp
from turnstone.core.config_store import ConfigStore
from turnstone.core.storage._protocol import StorageBackend
log = structlog.get_logger(__name__)
# Hardcoded defaults — no operator config needed
_CA_CN = "Turnstone CA"
_CA_NAME = "turnstone" # Store key for save_ca/load_ca
_CA_VALIDITY_DAYS = 3650 # 10 years
_CERT_VALIDITY_HOURS = 48
_RENEW_INTERVAL_HOURS = 24
_RENEW_BEFORE_EXPIRY_DAYS = 1
def _require_lacme() -> Any:
try:
import lacme
except ImportError:
raise ImportError(
"lacme is required for TLS support. Install with: pip install turnstone[tls]",
) from None
return lacme
class TLSManager:
"""Manages the internal CA, ACME responder, and certificate lifecycle.
Typical usage::
mgr = TLSManager(storage, config_store)
await mgr.init_ca()
responder = mgr.get_responder() # Mount at /acme
await mgr.issue_console_certs() # Self-issue for this node
mgr.start_renewal() # Background auto-renewal
"""
def __init__(
self,
storage: StorageBackend,
config_store: ConfigStore | None = None,
) -> None:
lacme = _require_lacme()
from turnstone.core.tls_store import StorageStore
self._store = StorageStore(storage)
self._config_store = config_store
self._event_dispatcher = lacme.EventDispatcher()
self._ca: Any | None = None
self._responder: Any | None = None
self._renewal_task: Any | None = None
self._renewal_manager: Any | None = None
self._internal_bundle: Any | None = None
self._frontend_bundle: Any | None = None
# Wire structlog to lacme events
self._subscribe_events()
# Wire Prometheus metrics (if prometheus_client available)
try:
from lacme.metrics import setup_metrics
setup_metrics(self._event_dispatcher)
except ImportError:
pass # prometheus_client not installed
def _subscribe_events(self) -> None:
"""Subscribe structlog handlers to lacme lifecycle events."""
_require_lacme()
from lacme.events import (
CertificateExpiring,
CertificateIssued,
CertificateRenewed,
ChallengeFailed,
)
def _on_issued(event: Any) -> None:
if isinstance(event, CertificateIssued):
log.info("tls.cert.issued", domain=event.domain)
def _on_renewed(event: Any) -> None:
if isinstance(event, CertificateRenewed):
log.info("tls.cert.renewed", domain=event.domain)
def _on_expiring(event: Any) -> None:
if isinstance(event, CertificateExpiring):
log.warning("tls.cert.expiring", domain=event.domain)
def _on_failed(event: Any) -> None:
if isinstance(event, ChallengeFailed):
log.error("tls.challenge.failed", domain=getattr(event, "domain", "unknown"))
self._event_dispatcher.subscribe(_on_issued, event_type=CertificateIssued)
self._event_dispatcher.subscribe(_on_renewed, event_type=CertificateRenewed)
self._event_dispatcher.subscribe(_on_expiring, event_type=CertificateExpiring)
self._event_dispatcher.subscribe(_on_failed, event_type=ChallengeFailed)
# -- CA lifecycle ----------------------------------------------------------
async def init_ca(self) -> None:
"""Initialize the internal Certificate Authority.
If a bootstrap CA exists on disk (from tls-bootstrap), imports it
into the database store first so the console uses the same CA that
signed the infrastructure certs.
"""
lacme = _require_lacme()
# Import bootstrap CA from well-known volume path if not already in DB
self._import_bootstrap_ca()
self._ca = lacme.CertificateAuthority(
self._store,
name=_CA_NAME,
event_dispatcher=self._event_dispatcher,
)
self._ca.init(cn=_CA_CN, validity_days=_CA_VALIDITY_DAYS)
log.info("tls.ca.initialized", cn=_CA_CN)
def _import_bootstrap_ca(self) -> None:
"""Import a bootstrap CA from /certs into the database store.
The tls-bootstrap CLI writes the CA to a FileStore at /certs.
On first boot, the console imports it so all services share
the same trust root.
"""
import os
from pathlib import Path
# Check if bootstrap CA exists and DB CA doesn't
bootstrap_dir = Path(os.environ.get("TURNSTONE_TLS_BOOTSTRAP_DIR", "/certs"))
ca_dir = bootstrap_dir / "ca" / _CA_NAME
ca_cert_file = ca_dir / "cert.pem"
ca_key_file = ca_dir / "key.pem"
if not ca_cert_file.exists() or not ca_key_file.exists():
return # No bootstrap CA found
existing = self._store.load_ca(_CA_NAME)
if existing is not None:
return # Already imported
cert_pem = ca_cert_file.read_bytes()
key_pem = ca_key_file.read_bytes()
self._store.save_ca(_CA_NAME, cert_pem, key_pem)
log.info("tls.ca.imported_from_bootstrap", path=str(ca_dir))
def get_responder(self) -> ASGIApp:
"""Return the ACME responder ASGI app for mounting."""
if self._ca is None:
raise RuntimeError("CA not initialized — call init_ca() first")
lacme = _require_lacme()
if self._responder is None:
self._responder = lacme.ACMEResponder(
ca=self._ca,
auto_approve=True,
)
return self._responder # type: ignore[no-any-return]
def get_root_cert_pem(self) -> bytes:
"""Return the CA root certificate in PEM format."""
if self._ca is None:
raise RuntimeError("CA not initialized — call init_ca() first")
return self._ca.root_cert_pem # type: ignore[no-any-return]
# -- Cert issuance ---------------------------------------------------------
async def issue_console_certs(self, hostnames: list[str]) -> None:
"""Issue certificates for the console node.
Raises ValueError if hostnames is empty.
Issues two certificates:
- Internal cert: always from the internal CA (for mTLS with cluster)
- Frontend cert: from external ACME CA if configured, else internal CA
"""
if not hostnames:
raise ValueError("issue_console_certs requires at least one hostname")
# Internal cert — always from our own CA
await self._issue_internal_cert(hostnames)
# Frontend cert — external CA if configured
acme_directory = ""
if self._config_store:
acme_directory = self._config_store.get("tls.acme_directory") or ""
if acme_directory:
await self._issue_frontend_cert(hostnames, acme_directory)
else:
# Self-issue from internal CA (behind reverse proxy or internal only)
self._frontend_bundle = self._internal_bundle
log.info("tls.frontend.self_issued", hostnames=hostnames)
async def _issue_internal_cert(self, hostnames: list[str]) -> None:
"""Issue an internal mTLS cert from the internal CA."""
if self._ca is None:
raise RuntimeError("CA not initialized")
# Check for existing cert in store (skip if expired)
existing = self._store.load_cert(hostnames[0])
if existing is not None:
from datetime import UTC, datetime
if existing.expires_at > datetime.now(UTC):
self._internal_bundle = existing
log.info("tls.internal.loaded", domain=hostnames[0])
return
log.info("tls.internal.expired", domain=hostnames[0])
self._store.delete_cert(hostnames[0])
# Issue new cert
bundle = self._ca.issue(
hostnames,
validity_hours=_CERT_VALIDITY_HOURS,
)
self._store.save_cert(bundle)
self._internal_bundle = bundle
log.info("tls.internal.issued", domain=hostnames[0])
async def _issue_frontend_cert(
self,
hostnames: list[str],
acme_directory: str,
) -> None:
"""Issue a frontend cert from an external ACME CA."""
lacme = _require_lacme()
from lacme.challenges.http01 import HTTP01Handler
handler = HTTP01Handler()
async with lacme.Client(
directory_url=acme_directory,
store=self._store,
challenge_handler=handler,
event_dispatcher=self._event_dispatcher,
) as client:
self._frontend_bundle = await client.issue(hostnames)
self._store.save_cert(self._frontend_bundle)
log.info(
"tls.frontend.issued",
domain=hostnames[0],
ca=acme_directory,
)
# -- Auto-renewal ----------------------------------------------------------
async def start_renewal(self) -> None:
"""Start background auto-renewal for all stored certificates.
Uses CA-direct mode (lacme 1.0.2+) signs directly via the CA
without going through ACME. No loopback client, no network,
no startup ordering dependency.
"""
if self._ca is None:
raise RuntimeError("CA not initialized")
lacme = _require_lacme()
def _on_renewed(bundle: Any) -> None:
# Update our cached bundles if the renewed domain matches
if self._internal_bundle and bundle.domain == self._internal_bundle.domain:
self._internal_bundle = bundle
if self._frontend_bundle and bundle.domain == self._frontend_bundle.domain:
self._frontend_bundle = bundle
self._renewal_manager = lacme.RenewalManager(
ca=self._ca,
store=self._store,
interval_hours=_RENEW_INTERVAL_HOURS,
days_before_expiry=_RENEW_BEFORE_EXPIRY_DAYS,
on_renewed=_on_renewed,
event_dispatcher=self._event_dispatcher,
)
self._renewal_task = self._renewal_manager.start()
log.info(
"tls.renewal.started",
interval_hours=_RENEW_INTERVAL_HOURS,
)
async def stop_renewal(self) -> None:
"""Stop the background renewal task."""
if self._renewal_task is not None:
import asyncio
import contextlib
self._renewal_task.cancel()
try:
with contextlib.suppress(asyncio.CancelledError):
await self._renewal_task
except Exception:
log.exception("tls.renewal.stop_error")
self._renewal_task = None
# -- SSL contexts ----------------------------------------------------------
def get_server_ssl_context(self) -> ssl.SSLContext | None:
"""Build an SSL context for the uvicorn HTTPS listener.
Uses the frontend cert (external CA or self-issued).
Returns None if no certs are available.
"""
if self._frontend_bundle is None:
return None
_require_lacme()
from lacme.mtls import server_ssl_context
return server_ssl_context( # type: ignore[no-any-return]
cert_pem=self._frontend_bundle.fullchain_pem,
key_pem=self._frontend_bundle.key_pem,
ca_cert_pem=self.get_root_cert_pem(),
)
def get_client_ssl_context(self) -> ssl.SSLContext | None:
"""Build an mTLS client context for connecting to cluster services.
Uses the internal cert for mutual authentication.
Returns None if no certs are available.
"""
if self._internal_bundle is None:
return None
_require_lacme()
from lacme.mtls import client_ssl_context
return client_ssl_context( # type: ignore[no-any-return]
cert_pem=self._internal_bundle.cert_pem,
key_pem=self._internal_bundle.key_pem,
ca_cert_pem=self.get_root_cert_pem(),
)
# -- Properties ------------------------------------------------------------
def list_certs(self) -> list[Any]:
"""List all stored certificate bundles."""
return self._store.list_certs()
def renew_cert(self, domain: str) -> Any:
"""Force-renew a certificate by domain. Returns the new bundle."""
if self._ca is None:
raise RuntimeError("CA not initialized")
existing = self._store.load_cert(domain)
if existing is None:
raise ValueError(f"No certificate for {domain}")
# Issue new cert first, then delete old (safe if issuance fails)
bundle = self._ca.issue(list(existing.domains))
self._store.delete_cert(domain)
self._store.save_cert(bundle)
# Update in-memory bundles if this is the console's own cert
if self._internal_bundle and bundle.domain == self._internal_bundle.domain:
self._internal_bundle = bundle
if self._frontend_bundle and bundle.domain == self._frontend_bundle.domain:
self._frontend_bundle = bundle
return bundle
def delete_cert(self, domain: str) -> bool:
"""Delete a certificate by domain."""
return self._store.delete_cert(domain)
@property
def ca_initialized(self) -> bool:
return self._ca is not None
@property
def internal_bundle(self) -> Any | None:
return self._internal_bundle
@property
def frontend_bundle(self) -> Any | None:
return self._frontend_bundle
+6 -2
View File
@@ -157,7 +157,7 @@ PUBLIC_PATHS: frozenset[str] = frozenset(
"/api/auth/oidc/callback",
}
)
PUBLIC_PREFIXES: tuple[str, ...] = ("/static/", "/shared/")
PUBLIC_PREFIXES: tuple[str, ...] = ("/static/", "/shared/", "/acme/")
WRITE_PATHS: frozenset[str] = frozenset(
{
@@ -329,18 +329,22 @@ def create_jwt(
expiry_hours: int = 24,
audience: str = "",
permissions: frozenset[str] = frozenset(),
expiry_seconds: int | None = None,
) -> str:
"""Create a signed JWT with user identity, scopes, and permissions."""
import jwt
if expiry_seconds is not None and expiry_seconds <= 0:
raise ValueError("expiry_seconds must be positive")
now = int(time.time())
ttl = expiry_seconds if expiry_seconds is not None else expiry_hours * 3600
payload: dict[str, Any] = {
"sub": user_id,
"scopes": ",".join(sorted(scopes)),
"src": source,
"iss": JWT_ISSUER,
"iat": now,
"exp": now + expiry_hours * 3600,
"exp": now + ttl,
}
if audience:
payload["aud"] = audience
+8
View File
@@ -125,6 +125,10 @@ _CONFIG_MAP: dict[str, dict[str, str]] = {
"port": "redis_port",
"password": "redis_password",
"db": "redis_db",
"tls": "redis_tls",
"tls_ca": "redis_tls_ca",
"tls_cert": "redis_tls_cert",
"tls_key": "redis_tls_key",
},
"console": {
"host": "host",
@@ -157,6 +161,10 @@ _CONFIG_MAP: dict[str, dict[str, str]] = {
"url": "db_url",
"path": "db_path",
"pool_size": "db_pool_size",
"sslmode": "db_sslmode",
"sslrootcert": "db_sslrootcert",
"sslcert": "db_sslcert",
"sslkey": "db_sslkey",
},
"judge": {
"enabled": "judge_enabled",
+61 -30
View File
@@ -853,6 +853,10 @@ If you used read_file to check a target, cite what you found."""
# ---------------------------------------------------------------------------
class _ExecutorPoisonedError(Exception):
"""Raised when a timeout leaves the executor's worker thread stuck."""
class IntentJudge:
"""Session-scoped LLM judge for intent validation.
@@ -912,18 +916,12 @@ class IntentJudge:
self._model = session_model
self._judge_context_window = context_window
# Executor for timeout-guarded API calls (1 thread — judge is serial)
self._executor = ThreadPoolExecutor(max_workers=1, thread_name_prefix="judge-api")
def shutdown(self) -> None:
"""Release the executor thread pool."""
self._executor.shutdown(wait=False, cancel_futures=True)
def evaluate(
self,
items: list[dict[str, Any]],
messages: list[dict[str, Any]],
callback: Callable[[IntentVerdict], None],
cancel_event: threading.Event | None = None,
) -> list[IntentVerdict]:
"""Evaluate tool calls. Returns heuristic verdicts immediately.
@@ -936,6 +934,10 @@ class IntentJudge:
``func_args``, ``approval_label``, ``call_id``).
messages: Conversation history (OpenAI message format).
callback: Called with each LLM verdict (or timeout/error fallback).
cancel_event: When set, the daemon judge thread abandons
remaining work. Callers should set this after the user
has already made an approval decision so the judge does
not keep consuming inference resources.
Returns:
List of heuristic verdicts (one per item), available immediately.
@@ -958,7 +960,7 @@ class IntentJudge:
# Spawn daemon thread for LLM judge
thread = threading.Thread(
target=self._run_judge,
args=(items, messages, heuristic_verdicts, callback),
args=(items, messages, heuristic_verdicts, callback, cancel_event),
daemon=True,
name="intent-judge",
)
@@ -972,25 +974,44 @@ class IntentJudge:
messages: list[dict[str, Any]],
heuristic_verdicts: list[IntentVerdict],
callback: Callable[[IntentVerdict], None],
cancel_event: threading.Event | None = None,
) -> None:
"""Daemon thread: run LLM judge for each item and invoke callback."""
for item, h_verdict in zip(items, heuristic_verdicts, strict=True):
try:
llm_verdict = self._evaluate_single(item, messages)
# Arbitrate: only callback when LLM upgrades the heuristic
if llm_verdict and llm_verdict.confidence > h_verdict.confidence:
callback(llm_verdict)
# else: heuristic already delivered, no duplicate callback
except Exception:
log.exception(
"Judge evaluation failed for %s",
item.get("func_name", "?"),
)
# Evaluation-scoped executor — avoids sharing mutable state with
# other daemon threads from concurrent evaluate() calls.
executor = ThreadPoolExecutor(max_workers=1, thread_name_prefix="judge-api")
try:
for idx, (item, h_verdict) in enumerate(zip(items, heuristic_verdicts, strict=True)):
if cancel_event and cancel_event.is_set():
log.debug("judge.cancelled", remaining=len(items) - idx)
return
try:
llm_verdict = self._evaluate_single(item, messages, cancel_event, executor)
if cancel_event and cancel_event.is_set():
return
# Arbitrate: only callback when LLM upgrades the heuristic
if llm_verdict and llm_verdict.confidence > h_verdict.confidence:
callback(llm_verdict)
# else: heuristic already delivered, no duplicate callback
except _ExecutorPoisonedError:
# Timeout left the worker stuck — replace the executor
# so subsequent items don't queue behind it.
executor.shutdown(wait=False, cancel_futures=True)
executor = ThreadPoolExecutor(max_workers=1, thread_name_prefix="judge-api")
except Exception:
log.exception(
"Judge evaluation failed for %s",
item.get("func_name", "?"),
)
finally:
executor.shutdown(wait=False, cancel_futures=True)
def _evaluate_single(
self,
item: dict[str, Any],
messages: list[dict[str, Any]],
cancel_event: threading.Event | None,
executor: ThreadPoolExecutor,
) -> IntentVerdict | None:
"""Run LLM judge for a single tool call. Returns verdict or None."""
start = time.monotonic()
@@ -1020,6 +1041,9 @@ class IntentJudge:
result = None # will hold the last CompletionResult
for turn in range(_JUDGE_MAX_TURNS):
if cancel_event and cancel_event.is_set():
return None
turn_start = time.monotonic()
is_last_turn = turn == _JUDGE_MAX_TURNS - 1
@@ -1043,7 +1067,7 @@ class IntentJudge:
# 10 minutes — far too long for an advisory judge on local models.
per_call_timeout = max(timeout_budget, 5.0) # at least 5s
try:
future = self._executor.submit(
future = executor.submit(
self._provider.create_completion,
client=self._client,
model=self._model,
@@ -1053,17 +1077,24 @@ class IntentJudge:
temperature=0.0,
reasoning_effort="medium",
)
result = future.result(timeout=per_call_timeout)
# Poll in 1s increments so we notice cancellation promptly
# instead of blocking for the full per_call_timeout.
deadline = time.monotonic() + per_call_timeout
while True:
remaining = deadline - time.monotonic()
if cancel_event and cancel_event.is_set():
future.cancel()
return None
if remaining <= 0:
raise TimeoutError
try:
result = future.result(timeout=min(remaining, 1.0))
break
except TimeoutError:
pass # loop back to check remaining/cancel
except TimeoutError:
log.warning("Judge LLM call timed out on turn %d (%.0fs)", turn, per_call_timeout)
# Abandon the lingering API call and replace the executor so
# subsequent items in the batch don't queue behind it.
self._executor.shutdown(wait=False, cancel_futures=True)
self._executor = ThreadPoolExecutor(
max_workers=1,
thread_name_prefix="judge-api",
)
return None
raise _ExecutorPoisonedError from None
except Exception:
log.exception("Judge LLM call failed on turn %d", turn)
return None
+25
View File
@@ -273,6 +273,29 @@ class OpenAIProvider:
result.append(tool)
return result
# -- message sanitisation ------------------------------------------------
@staticmethod
def _sanitize_messages(
messages: list[dict[str, Any]],
) -> list[dict[str, Any]]:
"""Ensure assistant messages always have ``content`` or ``tool_calls``.
OpenAI-compatible APIs reject assistant messages that have neither.
This is a defensive catch-all; the upstream layers should already
guarantee well-formed messages.
"""
out: list[dict[str, Any]] = []
for msg in messages:
if (
msg.get("role") == "assistant"
and msg.get("content") is None
and not msg.get("tool_calls")
):
msg = {**msg, "content": ""}
out.append(msg)
return out
# -- streaming -----------------------------------------------------------
def create_streaming(
@@ -289,6 +312,7 @@ class OpenAIProvider:
deferred_names: frozenset[str] | None = None,
) -> Iterator[StreamChunk]:
caps = self.get_capabilities(model)
messages = self._sanitize_messages(messages)
kwargs: dict[str, Any] = {
"model": model,
"messages": messages,
@@ -402,6 +426,7 @@ class OpenAIProvider:
deferred_names: frozenset[str] | None = None,
) -> CompletionResult:
caps = self.get_capabilities(model)
messages = self._sanitize_messages(messages)
kwargs: dict[str, Any] = {
"model": model,
"messages": messages,
+74 -37
View File
@@ -319,6 +319,7 @@ class ChatSession:
# Intent validation judge (lazy-initialized)
self._judge_config: JudgeConfig | None = judge_config
self._judge: IntentJudge | None = None
self._judge_cancel_event: threading.Event | None = None
# MCP tool integration: merge external tools with built-in
self._mcp_client = mcp_client
self._mcp_refresh_cb: Any = None # Callable | None (avoid import)
@@ -639,8 +640,8 @@ class ChatSession:
def close(self) -> None:
"""Release resources (listener registrations, etc.)."""
if self._judge is not None:
self._judge.shutdown()
if self._judge_cancel_event is not None:
self._judge_cancel_event.set()
if self._mcp_client and self._mcp_refresh_cb:
self._mcp_client.remove_listener(self._mcp_refresh_cb)
self._mcp_refresh_cb = None
@@ -867,17 +868,37 @@ class ChatSession:
]
else:
dev_parts = [
"You are an expert software engineer. You solve problems "
"by reading code, making targeted edits, and running commands. "
"Always respond with tool calls, not just text.\n\n"
"TOOL PATTERNS:\n\n"
"Modify existing file → read_file then edit_file:\n"
" read_file(path='config.py') → "
"edit_file(path='config.py')\n\n"
"Create new file → write_file:\n"
" write_file(path='hello.py', content='...')\n\n"
"Modify multiple filesread_file then edit_file each:\n"
" read_file(path='a.py') → edit_file(path='a.py')"
"read_file(path='b.py') → edit_file(path='b.py')\n\n"
"Create new file → write_file (generate reasonable "
"content even if the request is vague):\n"
" write_file(path='hello.py', content='...')\n"
" write_file(path='README.md', "
"content='# Project\\nDescription.')\n\n"
"Create a file then run it → write_file then bash:\n"
" write_file(path='fib.py', content='...') → "
"bash(command='python fib.py')\n\n"
"Find something across files → search:\n"
" search(query='test_')\n\n"
"Plan, design, or think through an approach → create_plan:\n"
" create_plan(goal='refactor database from API')\n\n"
"Find and modify → search then read_file then edit_file:\n"
" search(query='MAX_RETRIES') → "
"read_file(path='found.py') → "
"edit_file(path='found.py')\n\n"
"Plan, design, or architect something → "
"explore codebase then plan_agent:\n"
" bash(command='ls') → read_file(path='app.py') → "
"plan_agent(goal='add caching to the application')\n"
" plan_agent(goal='refactor database layer "
"from monolith to service')\n"
" plan_agent(goal='restructure auth module')\n\n"
"Run a command, git, or tests → bash:\n"
" bash(command='git log -5')\n"
" bash(command='pytest')\n\n"
@@ -885,8 +906,9 @@ class ChatSession:
" web_fetch(url='https://example.com')\n\n"
"Search the web for information → web_search:\n"
" web_search(query='current population of Tokyo')\n\n"
"Look up documentation → man:\n"
" man(page='tar')",
"Look up command flags or documentation → man:\n"
" man(page='tar')\n"
" man(page='grep')",
]
# Tool search hint (client-side mode only — native mode needs no hint)
if self._tool_search:
@@ -1648,7 +1670,7 @@ class ChatSession:
self.ui.on_stream_end()
partial: dict[str, Any] = {"role": "assistant"}
partial_content = "".join(content_parts)
partial["content"] = partial_content or None
partial["content"] = partial_content or ""
# Deliberately omit tool_calls — they are incomplete
if provider_blocks:
partial["_provider_content"] = provider_blocks
@@ -1679,10 +1701,7 @@ class ChatSession:
msg: dict[str, Any] = {"role": "assistant"}
content = "".join(content_parts)
if content:
msg["content"] = content
else:
msg["content"] = None
msg["content"] = content or ""
if tool_calls_acc:
msg["tool_calls"] = [tool_calls_acc[i] for i in sorted(tool_calls_acc)]
@@ -2053,20 +2072,24 @@ class ChatSession:
def _evaluate_intent(
self,
items: list[dict[str, Any]],
) -> None:
) -> threading.Event | None:
"""Run intent validation on pending approval items.
Attaches heuristic verdicts to items immediately. Spawns the
async LLM judge that delivers final verdicts via UI callback.
Returns a cancel event that, when set, tells the daemon judge
thread to abandon remaining work. Callers should set this
after the user has made an approval decision.
"""
judge = self._ensure_judge()
if not judge:
return
return None
# Only evaluate items that need approval and aren't errors
pending = [it for it in items if it.get("needs_approval") and not it.get("error")]
if not pending:
return
return None
# Build func_args from tool-specific item keys so the heuristic
# engine can pattern-match on argument content.
@@ -2090,8 +2113,10 @@ class ChatSession:
}
elif name == "notify":
it["func_args"] = {"message": it.get("message", "")[:200]}
elif name == "task":
elif name == "task_agent":
it["func_args"] = {"prompt": it.get("prompt", "")[:200]}
elif name == "plan_agent":
it["func_args"] = {"goal": it.get("prompt", "")[:200]}
elif it.get("mcp_args"):
it["func_args"] = it["mcp_args"]
@@ -2102,16 +2127,20 @@ class ChatSession:
except Exception:
log.debug("judge.verdict_delivery_failed", exc_info=True)
cancel_event = threading.Event()
heuristic_verdicts = judge.evaluate(
pending,
list(self.messages), # snapshot — daemon thread must not see mutations
callback=_on_verdict,
cancel_event=cancel_event,
)
# Attach heuristic verdicts to items for the approval UI
for item, verdict in zip(pending, heuristic_verdicts, strict=True):
item["_heuristic_verdict"] = verdict.to_dict()
return cancel_event
def _evaluate_output(self, call_id: str, output: str, func_name: str) -> str:
"""Run the output guard on tool result text.
@@ -2161,12 +2190,20 @@ class ChatSession:
# Phase 1: prepare all tool calls
items = [self._prepare_tool(tc) for tc in tool_calls]
# Intent validation (advisory, non-blocking)
self._evaluate_intent(items)
# Intent validation (advisory, non-blocking).
# Cancel any prior judge thread before spawning a new one.
if self._judge_cancel_event is not None:
self._judge_cancel_event.set()
judge_cancel = self._evaluate_intent(items)
self._judge_cancel_event = judge_cancel # track for close()
# Phase 2: approve via UI
self._emit_state("attention")
approved, user_feedback = self.ui.approve_tools(items)
try:
approved, user_feedback = self.ui.approve_tools(items)
finally:
if judge_cancel:
judge_cancel.set() # user decided (or disconnected) — stop judge
self._emit_state("running")
if not approved:
# Mark all pending items as denied
@@ -2219,7 +2256,7 @@ class ChatSession:
# feedback the plan agent re-runs and the revised plan is shown
# again, up to _MAX_PLAN_REFINEMENTS rounds.
for i, item in enumerate(items):
if item.get("func_name") != "create_plan" or item.get("error") or item.get("denied"):
if item.get("func_name") != "plan_agent" or item.get("error") or item.get("denied"):
continue
cid, output = results[i]
@@ -2344,8 +2381,8 @@ class ChatSession:
"web_fetch": self._prepare_web_fetch,
"web_search": self._prepare_web_search,
"tool_search": self._prepare_tool_search,
"task": self._prepare_task,
"create_plan": self._prepare_plan,
"task_agent": self._prepare_task,
"plan_agent": self._prepare_plan,
"memory": self._prepare_memory,
"recall": self._prepare_recall,
"notify": self._prepare_notify,
@@ -2877,8 +2914,8 @@ class ChatSession:
if not prompt:
return {
"call_id": call_id,
"func_name": "task",
"header": "\u2717 task: empty prompt",
"func_name": "task_agent",
"header": "\u2717 task_agent: empty prompt",
"preview": "",
"needs_approval": False,
"error": "Error: empty prompt",
@@ -2886,11 +2923,11 @@ class ChatSession:
preview_text = prompt[:300] + ("..." if len(prompt) > 300 else "")
return {
"call_id": call_id,
"func_name": "task",
"header": "\u2699 task (autonomous agent)",
"func_name": "task_agent",
"header": "\u2699 task_agent (autonomous agent)",
"preview": f" {DIM}{preview_text}{RESET}",
"needs_approval": True,
"approval_label": "task",
"approval_label": "task_agent",
"execute": self._exec_task,
"prompt": prompt,
}
@@ -2901,8 +2938,8 @@ class ChatSession:
if not goal:
return {
"call_id": call_id,
"func_name": "create_plan",
"header": "\u2717 create_plan: empty goal",
"func_name": "plan_agent",
"header": "\u2717 plan_agent: empty goal",
"preview": "",
"needs_approval": False,
"error": "Error: empty goal",
@@ -2910,11 +2947,11 @@ class ChatSession:
preview_text = goal[:300] + ("..." if len(goal) > 300 else "")
return {
"call_id": call_id,
"func_name": "create_plan",
"header": "\u2699 create_plan (planning agent)",
"func_name": "plan_agent",
"header": "\u2699 plan_agent (planning agent)",
"preview": f" {DIM}{preview_text}{RESET}",
"needs_approval": True,
"approval_label": "create_plan",
"approval_label": "plan_agent",
"execute": self._exec_plan,
"prompt": goal,
}
@@ -3920,7 +3957,7 @@ class ChatSession:
tool_name = tc_dict["function"]["name"]
# Guard 1: block recursive agent calls.
if tool_name in ("task", "create_plan"):
if tool_name in ("task_agent", "plan_agent"):
output = "Error: agents cannot spawn further agents"
# Guard 2: tool not in this agent's API tool list.
elif tool_name not in tool_names:
@@ -4123,7 +4160,7 @@ class ChatSession:
for i, msg in enumerate(self.messages):
if msg.get("role") == "assistant" and msg.get("tool_calls"):
for tc in msg["tool_calls"]:
if tc.get("function", {}).get("name") == "create_plan":
if tc.get("function", {}).get("name") == "plan_agent":
tc_id = tc["id"]
for j in range(i + 1, len(self.messages)):
if (
@@ -4211,13 +4248,13 @@ class ChatSession:
{"role": "system", "content": self._plan_system_content()},
{
"role": "assistant",
"content": None,
"content": "",
"tool_calls": [
{
"id": tc_id,
"type": "function",
"function": {
"name": "create_plan",
"name": "plan_agent",
"arguments": json.dumps({"goal": original_goal}),
},
}
+24 -1
View File
@@ -528,6 +528,29 @@ def _build_registry() -> dict[str, SettingDef]:
"information from conversations into long-term memory. This helps the AI "
"remember context across separate conversations.",
),
# -- tls ----------------------------------------------------------------
SettingDef(
"tls.enabled",
"bool",
False,
"Enable mTLS for inter-service communication",
"tls",
restart_required=True,
help="When enabled, the console runs an internal Certificate Authority and "
"ACME server. All cluster services (servers, bridge, channels) auto-provision "
"short-lived certificates for mutual TLS. Requires lacme: pip install turnstone[tls]",
),
SettingDef(
"tls.acme_directory",
"str",
"",
"External ACME CA URL for the console's frontend HTTPS cert",
"tls",
restart_required=True,
help="Set to a public ACME directory URL (e.g. https://acme-v02.api.letsencrypt.org/"
"directory) to get a publicly trusted certificate for the console's HTTPS endpoint. "
"Leave empty to self-issue from the internal CA (use when behind a reverse proxy).",
),
]
return {d.key: d for d in defs}
@@ -543,7 +566,7 @@ BOOTSTRAP_SECTIONS: frozenset[str] = frozenset(
"auth",
"bridge",
"console",
}
},
)
+105
View File
@@ -30,6 +30,9 @@ from turnstone.core.storage._schema import (
skill_versions,
structured_memories,
system_settings,
tls_account_keys,
tls_ca,
tls_certificates,
tool_policies,
usage_events,
user_roles,
@@ -2805,6 +2808,108 @@ class PostgreSQLBackend:
conn.commit()
return result.rowcount
# -- TLS / ACME ------------------------------------------------------------
def save_tls_account_key(self, key_id: str, key_pem: str) -> None:
from sqlalchemy.dialects.postgresql import insert as pg_insert
now = datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%S")
stmt = pg_insert(tls_account_keys).values(id=key_id, key_pem=key_pem, created=now)
stmt = stmt.on_conflict_do_update(
index_elements=["id"],
set_={"key_pem": key_pem},
)
with self._engine.connect() as conn:
conn.execute(stmt)
conn.commit()
def load_tls_account_key(self, key_id: str) -> str | None:
with self._engine.connect() as conn:
row = conn.execute(
sa.select(tls_account_keys.c.key_pem).where(tls_account_keys.c.id == key_id)
).first()
return row[0] if row else None
def save_tls_ca(self, name: str, cert_pem: str, key_pem: str) -> None:
from sqlalchemy.dialects.postgresql import insert as pg_insert
now = datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%S")
stmt = pg_insert(tls_ca).values(name=name, cert_pem=cert_pem, key_pem=key_pem, created=now)
stmt = stmt.on_conflict_do_update(
index_elements=["name"],
set_={"cert_pem": cert_pem, "key_pem": key_pem},
)
with self._engine.connect() as conn:
conn.execute(stmt)
conn.commit()
def load_tls_ca(self, name: str) -> dict[str, Any] | None:
with self._engine.connect() as conn:
row = conn.execute(sa.select(tls_ca).where(tls_ca.c.name == name)).first()
if not row:
return None
return _row_to_dict(row)
def save_tls_cert(
self,
domain: str,
cert_pem: str,
fullchain_pem: str,
key_pem: str,
issued_at: str,
expires_at: str,
meta: str | None = None,
) -> None:
from sqlalchemy.dialects.postgresql import insert as pg_insert
stmt = pg_insert(tls_certificates).values(
domain=domain,
cert_pem=cert_pem,
fullchain_pem=fullchain_pem,
key_pem=key_pem,
issued_at=issued_at,
expires_at=expires_at,
meta=meta,
)
stmt = stmt.on_conflict_do_update(
index_elements=["domain"],
set_={
"cert_pem": cert_pem,
"fullchain_pem": fullchain_pem,
"key_pem": key_pem,
"issued_at": issued_at,
"expires_at": expires_at,
"meta": meta,
},
)
with self._engine.connect() as conn:
conn.execute(stmt)
conn.commit()
def load_tls_cert(self, domain: str) -> dict[str, Any] | None:
with self._engine.connect() as conn:
row = conn.execute(
sa.select(tls_certificates).where(tls_certificates.c.domain == domain)
).first()
if not row:
return None
return _row_to_dict(row)
def list_tls_certs(self) -> list[dict[str, Any]]:
with self._engine.connect() as conn:
rows = conn.execute(
sa.select(tls_certificates).order_by(tls_certificates.c.domain)
).fetchall()
return [_row_to_dict(r) for r in rows]
def delete_tls_cert(self, domain: str) -> bool:
with self._engine.connect() as conn:
result = conn.execute(
sa.delete(tls_certificates).where(tls_certificates.c.domain == domain)
)
conn.commit()
return result.rowcount > 0
# -- Lifecycle -------------------------------------------------------------
def close(self) -> None:
+43
View File
@@ -952,6 +952,49 @@ class StorageBackend(Protocol):
"""Delete an MCP server definition. Returns True if existed."""
...
# -- TLS / ACME (lacme Store) ----------------------------------------------
def save_tls_account_key(self, key_id: str, key_pem: str) -> None:
"""Persist an ACME account private key."""
...
def load_tls_account_key(self, key_id: str) -> str | None:
"""Load an ACME account key PEM by ID. Returns None if not found."""
...
def save_tls_ca(self, name: str, cert_pem: str, key_pem: str) -> None:
"""Persist a CA root certificate and key."""
...
def load_tls_ca(self, name: str) -> dict[str, Any] | None:
"""Load CA cert+key by name. Returns dict with cert_pem, key_pem or None."""
...
def save_tls_cert(
self,
domain: str,
cert_pem: str,
fullchain_pem: str,
key_pem: str,
issued_at: str,
expires_at: str,
meta: str | None = None,
) -> None:
"""Persist an issued certificate (upsert by domain)."""
...
def load_tls_cert(self, domain: str) -> dict[str, Any] | None:
"""Load certificate by domain. Returns dict or None."""
...
def list_tls_certs(self) -> list[dict[str, Any]]:
"""List all stored certificates."""
...
def delete_tls_cert(self, domain: str) -> bool:
"""Delete a certificate by domain. Returns True if existed."""
...
# -- Lifecycle -------------------------------------------------------------
def close(self) -> None:
+24
View File
@@ -22,6 +22,10 @@ def init_storage(
url: str = "",
pool_size: int = 2,
run_migrations: bool = True,
sslmode: str = "",
sslrootcert: str = "",
sslcert: str = "",
sslkey: str = "",
) -> StorageBackend:
"""Initialize the storage backend singleton.
@@ -52,6 +56,26 @@ def init_storage(
if not url:
msg = "PostgreSQL backend requires a connection URL (db_url)"
raise ValueError(msg)
# Append SSL params to URL if provided (validated + encoded)
valid_sslmodes = {"disable", "allow", "prefer", "require", "verify-ca", "verify-full"}
if sslmode and sslmode not in valid_sslmodes:
msg = f"Invalid sslmode: {sslmode!r} (expected one of {sorted(valid_sslmodes)})"
raise ValueError(msg)
ssl_params = {
k: v
for k, v in {
"sslmode": sslmode,
"sslrootcert": sslrootcert,
"sslcert": sslcert,
"sslkey": sslkey,
}.items()
if v
}
if ssl_params:
from urllib.parse import urlencode
sep = "&" if "?" in url else "?"
url += sep + urlencode(ssl_params)
_storage = PostgreSQLBackend(url, pool_size=pool_size, create_tables=create_tables)
log.info("Storage initialized: PostgreSQL")
+31
View File
@@ -547,3 +547,34 @@ oidc_pending_states = sa.Table(
sa.Column("audience", sa.Text, nullable=False),
sa.Column("created_at", sa.Text, nullable=False),
)
# ── TLS / ACME (lacme integration) ──────────────────────────────────────────
tls_account_keys = sa.Table(
"tls_account_keys",
metadata,
sa.Column("id", sa.Text, primary_key=True),
sa.Column("key_pem", sa.Text, nullable=False),
sa.Column("created", sa.Text, nullable=False),
)
tls_ca = sa.Table(
"tls_ca",
metadata,
sa.Column("name", sa.Text, primary_key=True),
sa.Column("cert_pem", sa.Text, nullable=False),
sa.Column("key_pem", sa.Text, nullable=False),
sa.Column("created", sa.Text, nullable=False),
)
tls_certificates = sa.Table(
"tls_certificates",
metadata,
sa.Column("domain", sa.Text, primary_key=True),
sa.Column("cert_pem", sa.Text, nullable=False),
sa.Column("fullchain_pem", sa.Text, nullable=False),
sa.Column("key_pem", sa.Text, nullable=False),
sa.Column("issued_at", sa.Text, nullable=False),
sa.Column("expires_at", sa.Text, nullable=False),
sa.Column("meta", sa.Text, nullable=True),
)
+107
View File
@@ -30,6 +30,9 @@ from turnstone.core.storage._schema import (
skill_versions,
structured_memories,
system_settings,
tls_account_keys,
tls_ca,
tls_certificates,
tool_policies,
usage_events,
user_roles,
@@ -2854,6 +2857,110 @@ class SQLiteBackend:
conn.commit()
return result.rowcount
# -- TLS / ACME ------------------------------------------------------------
def save_tls_account_key(self, key_id: str, key_pem: str) -> None:
from sqlalchemy.dialects.sqlite import insert as sqlite_insert
now = datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%S")
stmt = sqlite_insert(tls_account_keys).values(id=key_id, key_pem=key_pem, created=now)
stmt = stmt.on_conflict_do_update(
index_elements=["id"],
set_={"key_pem": key_pem},
)
with self._engine.connect() as conn:
conn.execute(stmt)
conn.commit()
def load_tls_account_key(self, key_id: str) -> str | None:
with self._engine.connect() as conn:
row = conn.execute(
sa.select(tls_account_keys.c.key_pem).where(tls_account_keys.c.id == key_id)
).first()
return row[0] if row else None
def save_tls_ca(self, name: str, cert_pem: str, key_pem: str) -> None:
from sqlalchemy.dialects.sqlite import insert as sqlite_insert
now = datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%S")
stmt = sqlite_insert(tls_ca).values(
name=name, cert_pem=cert_pem, key_pem=key_pem, created=now
)
stmt = stmt.on_conflict_do_update(
index_elements=["name"],
set_={"cert_pem": cert_pem, "key_pem": key_pem},
)
with self._engine.connect() as conn:
conn.execute(stmt)
conn.commit()
def load_tls_ca(self, name: str) -> dict[str, Any] | None:
with self._engine.connect() as conn:
row = conn.execute(sa.select(tls_ca).where(tls_ca.c.name == name)).first()
if not row:
return None
return _row_to_dict(row)
def save_tls_cert(
self,
domain: str,
cert_pem: str,
fullchain_pem: str,
key_pem: str,
issued_at: str,
expires_at: str,
meta: str | None = None,
) -> None:
from sqlalchemy.dialects.sqlite import insert as sqlite_insert
stmt = sqlite_insert(tls_certificates).values(
domain=domain,
cert_pem=cert_pem,
fullchain_pem=fullchain_pem,
key_pem=key_pem,
issued_at=issued_at,
expires_at=expires_at,
meta=meta,
)
stmt = stmt.on_conflict_do_update(
index_elements=["domain"],
set_={
"cert_pem": cert_pem,
"fullchain_pem": fullchain_pem,
"key_pem": key_pem,
"issued_at": issued_at,
"expires_at": expires_at,
"meta": meta,
},
)
with self._engine.connect() as conn:
conn.execute(stmt)
conn.commit()
def load_tls_cert(self, domain: str) -> dict[str, Any] | None:
with self._engine.connect() as conn:
row = conn.execute(
sa.select(tls_certificates).where(tls_certificates.c.domain == domain)
).first()
if not row:
return None
return _row_to_dict(row)
def list_tls_certs(self) -> list[dict[str, Any]]:
with self._engine.connect() as conn:
rows = conn.execute(
sa.select(tls_certificates).order_by(tls_certificates.c.domain)
).fetchall()
return [_row_to_dict(r) for r in rows]
def delete_tls_cert(self, domain: str) -> bool:
with self._engine.connect() as conn:
result = conn.execute(
sa.delete(tls_certificates).where(tls_certificates.c.domain == domain)
)
conn.commit()
return result.rowcount > 0
# -- Lifecycle -------------------------------------------------------------
def close(self) -> None:
+1 -1
View File
@@ -152,7 +152,7 @@ def reconstruct_messages(rows: list[Any], ws_id: str) -> list[dict[str, Any]]:
messages.append({"role": "user", "content": content or ""})
elif role == "assistant":
msg: dict[str, Any] = {"role": "assistant", "content": content}
msg: dict[str, Any] = {"role": "assistant", "content": content or ""}
if provider_data:
with contextlib.suppress(json.JSONDecodeError, TypeError):
msg["_provider_content"] = json.loads(provider_data)
+5 -1
View File
@@ -19,7 +19,11 @@ def run_migrations_online() -> None:
)
with connectable.connect() as connection:
context.configure(connection=connection, target_metadata=target_metadata)
context.configure(
connection=connection,
target_metadata=target_metadata,
render_as_batch=True,
)
with context.begin_transaction():
context.run_migrations()
@@ -28,9 +28,7 @@ def upgrade() -> None:
sa.Column("updated", sa.Text, nullable=False),
sa.Column("last_accessed", sa.Text, nullable=False, server_default=""),
sa.Column("access_count", sa.Integer, nullable=False, server_default="0"),
)
op.create_unique_constraint(
"uq_smem_name_scope", "structured_memories", ["name", "scope", "scope_id"]
sa.UniqueConstraint("name", "scope", "scope_id", name="uq_smem_name_scope"),
)
op.create_index("idx_smem_type", "structured_memories", ["type"])
op.create_index("idx_smem_scope", "structured_memories", ["scope", "scope_id"])
@@ -27,56 +27,32 @@ depends_on = None
def upgrade() -> None:
# Phase 1: Skills evolution columns
op.add_column(
"prompt_templates",
sa.Column("description", sa.Text, nullable=False, server_default=""),
)
op.add_column(
"prompt_templates",
sa.Column("tags", sa.Text, nullable=False, server_default="[]"),
)
op.add_column(
"prompt_templates",
sa.Column("source_url", sa.Text, nullable=False, server_default=""),
)
op.add_column(
"prompt_templates",
sa.Column("version", sa.Text, nullable=False, server_default="1.0.0"),
)
op.add_column(
"prompt_templates",
sa.Column("author", sa.Text, nullable=False, server_default=""),
)
op.add_column(
"prompt_templates",
sa.Column("activation", sa.Text, nullable=False, server_default="named"),
)
op.add_column(
"prompt_templates",
sa.Column("token_estimate", sa.Integer, nullable=False, server_default="0"),
)
# Phase 2: Security scanning + install provenance
op.add_column(
"prompt_templates",
sa.Column("allowed_tools", sa.Text, nullable=False, server_default="[]"),
)
op.add_column(
"prompt_templates",
sa.Column("scan_status", sa.Text, nullable=False, server_default=""),
)
op.add_column(
"prompt_templates",
sa.Column("scan_report", sa.Text, nullable=False, server_default="{}"),
)
op.add_column(
"prompt_templates",
sa.Column("installed_at", sa.Text, nullable=False, server_default=""),
)
op.add_column(
"prompt_templates",
sa.Column("installed_by", sa.Text, nullable=False, server_default=""),
)
# Phase 1+2: Skills evolution columns + security scanning (batch for SQLite)
with op.batch_alter_table("prompt_templates") as batch_op:
batch_op.add_column(sa.Column("description", sa.Text, nullable=False, server_default=""))
batch_op.add_column(sa.Column("tags", sa.Text, nullable=False, server_default="[]"))
batch_op.add_column(sa.Column("source_url", sa.Text, nullable=False, server_default=""))
batch_op.add_column(sa.Column("version", sa.Text, nullable=False, server_default="1.0.0"))
batch_op.add_column(sa.Column("author", sa.Text, nullable=False, server_default=""))
batch_op.add_column(
sa.Column("activation", sa.Text, nullable=False, server_default="named"),
)
batch_op.add_column(
sa.Column("token_estimate", sa.Integer, nullable=False, server_default="0"),
)
batch_op.add_column(
sa.Column("allowed_tools", sa.Text, nullable=False, server_default="[]"),
)
batch_op.add_column(sa.Column("scan_status", sa.Text, nullable=False, server_default=""))
batch_op.add_column(
sa.Column("scan_report", sa.Text, nullable=False, server_default="{}"),
)
batch_op.add_column(
sa.Column("installed_at", sa.Text, nullable=False, server_default=""),
)
batch_op.add_column(
sa.Column("installed_by", sa.Text, nullable=False, server_default=""),
)
# Backfill activation from is_default
op.execute("UPDATE prompt_templates SET activation = 'default' WHERE is_default = 1")
@@ -101,42 +77,26 @@ def upgrade() -> None:
)
# Phase 3: Session config columns (from workstream templates)
op.add_column(
"prompt_templates",
sa.Column("model", sa.Text, nullable=False, server_default=""),
)
op.add_column(
"prompt_templates",
sa.Column("auto_approve", sa.Integer, nullable=False, server_default="0"),
)
op.add_column(
"prompt_templates",
sa.Column("temperature", sa.Float, nullable=True),
)
op.add_column(
"prompt_templates",
sa.Column("reasoning_effort", sa.Text, nullable=False, server_default=""),
)
op.add_column(
"prompt_templates",
sa.Column("max_tokens", sa.Integer, nullable=True),
)
op.add_column(
"prompt_templates",
sa.Column("token_budget", sa.Integer, nullable=False, server_default="0"),
)
op.add_column(
"prompt_templates",
sa.Column("agent_max_turns", sa.Integer, nullable=True),
)
op.add_column(
"prompt_templates",
sa.Column("notify_on_complete", sa.Text, nullable=False, server_default="{}"),
)
op.add_column(
"prompt_templates",
sa.Column("enabled", sa.Integer, nullable=False, server_default="1"),
)
with op.batch_alter_table("prompt_templates") as batch_op:
batch_op.add_column(sa.Column("model", sa.Text, nullable=False, server_default=""))
batch_op.add_column(
sa.Column("auto_approve", sa.Integer, nullable=False, server_default="0"),
)
batch_op.add_column(sa.Column("temperature", sa.Float, nullable=True))
batch_op.add_column(
sa.Column("reasoning_effort", sa.Text, nullable=False, server_default=""),
)
batch_op.add_column(sa.Column("max_tokens", sa.Integer, nullable=True))
batch_op.add_column(
sa.Column("token_budget", sa.Integer, nullable=False, server_default="0"),
)
batch_op.add_column(sa.Column("agent_max_turns", sa.Integer, nullable=True))
batch_op.add_column(
sa.Column("notify_on_complete", sa.Text, nullable=False, server_default="{}"),
)
batch_op.add_column(
sa.Column("enabled", sa.Integer, nullable=False, server_default="1"),
)
# Skill versions — version history for skills
op.create_table(
@@ -253,24 +213,25 @@ def upgrade() -> None:
# Rename workstreams table columns: ws_template_id → skill_id,
# ws_template_version → skill_version
op.alter_column("workstreams", "ws_template_id", new_column_name="skill_id")
op.alter_column("workstreams", "ws_template_version", new_column_name="skill_version")
with op.batch_alter_table("workstreams") as batch_op:
batch_op.alter_column("ws_template_id", new_column_name="skill_id")
batch_op.alter_column("ws_template_version", new_column_name="skill_version")
# Rename scheduled_tasks.template → skill
op.alter_column("scheduled_tasks", "template", new_column_name="skill")
# Rename scheduled_tasks.template → skill, drop ws_template
with op.batch_alter_table("scheduled_tasks") as batch_op:
batch_op.alter_column("template", new_column_name="skill")
batch_op.drop_column("ws_template")
# Drop old tables
op.drop_table("workstream_template_versions")
op.drop_table("workstream_templates")
# Drop ws_template column from scheduled_tasks
op.drop_column("scheduled_tasks", "ws_template")
def downgrade() -> None:
# Reverse workstreams column renames
op.alter_column("workstreams", "skill_id", new_column_name="ws_template_id")
op.alter_column("workstreams", "skill_version", new_column_name="ws_template_version")
with op.batch_alter_table("workstreams") as batch_op:
batch_op.alter_column("skill_id", new_column_name="ws_template_id")
batch_op.alter_column("skill_version", new_column_name="ws_template_version")
# Reverse workstream_config key renames
op.execute("UPDATE workstream_config SET key = 'ws_template_id' WHERE key = 'applied_skill_id'")
@@ -283,14 +244,10 @@ def downgrade() -> None:
"WHERE key = 'applied_skill_content'"
)
# Reverse scheduled_tasks column rename: skill → template
op.alter_column("scheduled_tasks", "skill", new_column_name="template")
# Re-add ws_template column to scheduled_tasks
op.add_column(
"scheduled_tasks",
sa.Column("ws_template", sa.Text, nullable=False, server_default=""),
)
# Reverse scheduled_tasks column rename + re-add ws_template
with op.batch_alter_table("scheduled_tasks") as batch_op:
batch_op.alter_column("skill", new_column_name="template")
batch_op.add_column(sa.Column("ws_template", sa.Text, nullable=False, server_default=""))
# Recreate workstream_templates (empty — destructive migration)
op.create_table(
@@ -333,32 +290,31 @@ def downgrade() -> None:
op.drop_index("idx_skill_versions_skill_id", table_name="skill_versions")
op.drop_table("skill_versions")
# Drop session config columns from prompt_templates
op.drop_column("prompt_templates", "enabled")
op.drop_column("prompt_templates", "notify_on_complete")
op.drop_column("prompt_templates", "agent_max_turns")
op.drop_column("prompt_templates", "token_budget")
op.drop_column("prompt_templates", "max_tokens")
op.drop_column("prompt_templates", "reasoning_effort")
op.drop_column("prompt_templates", "temperature")
op.drop_column("prompt_templates", "auto_approve")
op.drop_column("prompt_templates", "model")
# Drop session config + skills evolution columns from prompt_templates
with op.batch_alter_table("prompt_templates") as batch_op:
batch_op.drop_column("enabled")
batch_op.drop_column("notify_on_complete")
batch_op.drop_column("agent_max_turns")
batch_op.drop_column("token_budget")
batch_op.drop_column("max_tokens")
batch_op.drop_column("reasoning_effort")
batch_op.drop_column("temperature")
batch_op.drop_column("auto_approve")
batch_op.drop_column("model")
batch_op.drop_column("installed_by")
batch_op.drop_column("installed_at")
batch_op.drop_column("scan_report")
batch_op.drop_column("scan_status")
batch_op.drop_column("allowed_tools")
batch_op.drop_column("token_estimate")
batch_op.drop_column("activation")
batch_op.drop_column("author")
batch_op.drop_column("version")
batch_op.drop_column("source_url")
batch_op.drop_column("tags")
batch_op.drop_column("description")
# Drop skill resources
op.drop_index("idx_skill_resources_skill_path", table_name="skill_resources")
op.drop_index("idx_skill_resources_skill_id", table_name="skill_resources")
op.drop_table("skill_resources")
# Drop skills evolution columns
op.drop_column("prompt_templates", "installed_by")
op.drop_column("prompt_templates", "installed_at")
op.drop_column("prompt_templates", "scan_report")
op.drop_column("prompt_templates", "scan_status")
op.drop_column("prompt_templates", "allowed_tools")
op.drop_column("prompt_templates", "token_estimate")
op.drop_column("prompt_templates", "activation")
op.drop_column("prompt_templates", "author")
op.drop_column("prompt_templates", "version")
op.drop_column("prompt_templates", "source_url")
op.drop_column("prompt_templates", "tags")
op.drop_column("prompt_templates", "description")
@@ -33,14 +33,13 @@ def upgrade() -> None:
op.create_index("ix_oa_created", "output_assessments", ["created"])
op.create_index("ix_oa_risk", "output_assessments", ["risk_level"])
op.add_column(
"prompt_templates",
sa.Column("scan_version", sa.Text, nullable=False, server_default=""),
)
with op.batch_alter_table("prompt_templates") as batch_op:
batch_op.add_column(sa.Column("scan_version", sa.Text, nullable=False, server_default=""))
def downgrade() -> None:
op.drop_column("prompt_templates", "scan_version")
with op.batch_alter_table("prompt_templates") as batch_op:
batch_op.drop_column("scan_version")
op.drop_index("ix_oa_risk", table_name="output_assessments")
op.drop_index("ix_oa_created", table_name="output_assessments")
op.drop_index("ix_oa_ws_id", table_name="output_assessments")
@@ -19,16 +19,14 @@ depends_on = None
def upgrade() -> None:
op.add_column(
"prompt_templates",
sa.Column("license", sa.Text, nullable=False, server_default=""),
)
op.add_column(
"prompt_templates",
sa.Column("compatibility", sa.Text, nullable=False, server_default=""),
)
with op.batch_alter_table("prompt_templates") as batch_op:
batch_op.add_column(sa.Column("license", sa.Text, nullable=False, server_default=""))
batch_op.add_column(
sa.Column("compatibility", sa.Text, nullable=False, server_default=""),
)
def downgrade() -> None:
op.drop_column("prompt_templates", "compatibility")
op.drop_column("prompt_templates", "license")
with op.batch_alter_table("prompt_templates") as batch_op:
batch_op.drop_column("compatibility")
batch_op.drop_column("license")
@@ -19,11 +19,10 @@ depends_on = None
def upgrade() -> None:
op.add_column(
"prompt_templates",
sa.Column("priority", sa.Integer, nullable=False, server_default="0"),
)
with op.batch_alter_table("prompt_templates") as batch_op:
batch_op.add_column(sa.Column("priority", sa.Integer, nullable=False, server_default="0"))
def downgrade() -> None:
op.drop_column("prompt_templates", "priority")
with op.batch_alter_table("prompt_templates") as batch_op:
batch_op.drop_column("priority")
@@ -0,0 +1,53 @@
"""TLS certificate storage for lacme ACME integration.
Three tables for the lacme Store protocol:
- tls_account_keys: ACME account private keys
- tls_ca: CA root certificate and key
- tls_certificates: Issued service certificates
Revision ID: 026
Revises: 025
Create Date: 2026-03-25
"""
import sqlalchemy as sa
from alembic import op
revision = "026"
down_revision = "025"
branch_labels = None
depends_on = None
def upgrade() -> None:
op.create_table(
"tls_account_keys",
sa.Column("id", sa.Text, primary_key=True),
sa.Column("key_pem", sa.Text, nullable=False),
sa.Column("created", sa.Text, nullable=False),
)
op.create_table(
"tls_ca",
sa.Column("name", sa.Text, primary_key=True),
sa.Column("cert_pem", sa.Text, nullable=False),
sa.Column("key_pem", sa.Text, nullable=False),
sa.Column("created", sa.Text, nullable=False),
)
op.create_table(
"tls_certificates",
sa.Column("domain", sa.Text, primary_key=True),
sa.Column("cert_pem", sa.Text, nullable=False),
sa.Column("fullchain_pem", sa.Text, nullable=False),
sa.Column("key_pem", sa.Text, nullable=False),
sa.Column("issued_at", sa.Text, nullable=False),
sa.Column("expires_at", sa.Text, nullable=False),
sa.Column("meta", sa.Text, nullable=True),
)
def downgrade() -> None:
op.drop_table("tls_certificates")
op.drop_table("tls_ca")
op.drop_table("tls_account_keys")
+254
View File
@@ -0,0 +1,254 @@
"""TLS Client — certificate provisioning for service nodes.
Non-console services (server, bridge, channel gateway) use this to
request certificates from the console's ACME endpoint and build
SSL contexts for mTLS communication.
Flow:
1. Fetch CA root cert from console (plain HTTP, first boot)
2. Request service cert via ACME (plain HTTP, first boot)
3. Build SSL contexts for uvicorn (server) and httpx (client)
4. Start auto-renewal (uses existing cert for mTLS to console)
"""
from __future__ import annotations
from typing import TYPE_CHECKING, Any
if TYPE_CHECKING:
import ssl
from turnstone.core.storage._protocol import StorageBackend
from turnstone.core.log import get_logger
log = get_logger(__name__)
_RENEW_INTERVAL_HOURS = 24
_RENEW_BEFORE_EXPIRY_DAYS = 1
def _require_lacme() -> Any:
try:
import lacme
except ImportError:
raise ImportError(
"lacme is required for TLS support. Install with: pip install turnstone[tls]",
) from None
return lacme
class TLSClient:
"""TLS client for service nodes.
Requests certificates from the console's ACME endpoint and provides
SSL contexts for server (uvicorn) and client (httpx) use.
Typical usage::
client = TLSClient(storage, console_url="http://console:8080")
await client.init() # Fetch CA, request cert
server_ctx = client.get_server_ssl_context() # For uvicorn
client_ctx = client.get_client_ssl_context() # For httpx
await client.start_renewal() # Background auto-renewal
"""
def __init__(
self,
storage: StorageBackend,
console_url: str = "",
hostnames: list[str] | None = None,
) -> None:
lacme = _require_lacme()
from turnstone.core.tls_store import StorageStore
self._storage = storage
self._store = StorageStore(storage)
self._console_url = console_url.rstrip("/") if console_url else ""
self._hostnames = hostnames or []
self._event_dispatcher = lacme.EventDispatcher()
self._ca_pem: bytes | None = None
self._bundle: Any | None = None
self._renewal_task: Any | None = None
self._renewal_client: Any | None = None
# Wire Prometheus metrics
try:
from lacme.metrics import setup_metrics
setup_metrics(self._event_dispatcher)
except ImportError:
pass
async def init(self) -> None:
"""Fetch CA root cert and request a service certificate.
If no console_url was provided, discovers it from the services
table. Performs initial cert provisioning over plain HTTP (ACME
protocol provides integrity via JWS).
"""
if not self._console_url:
self._console_url = self._discover_console_url()
await self._fetch_ca_cert()
await self._request_cert()
def _discover_console_url(self) -> str:
"""Look up the console URL from the services table."""
consoles = self._storage.list_services("console", max_age_seconds=3600)
if not consoles:
raise RuntimeError(
"No console service found in services table. "
"Ensure the console is running and has registered, "
"or provide console_url explicitly."
)
url = consoles[0]["url"]
log.info("tls.console.discovered", url=url)
return url
async def _fetch_ca_cert(self) -> None:
"""Fetch the CA root cert from the console.
Always uses plain HTTP for bootstrapping the node doesn't have
the CA cert yet, so it can't verify HTTPS.
"""
import httpx
# Force HTTP for bootstrap (can't verify HTTPS without CA cert)
base = self._console_url.replace("https://", "http://")
url = f"{base}/acme/ca.pem"
try:
async with httpx.AsyncClient() as client:
resp = await client.get(url)
resp.raise_for_status()
self._ca_pem = resp.content
log.info("tls.ca.fetched", url=url)
except Exception:
log.error("tls.ca.fetch_failed", url=url, exc_info=True)
raise
async def _request_cert(self) -> None:
"""Request a certificate from the console's ACME endpoint."""
if not self._hostnames:
raise ValueError("No hostnames configured for TLS cert request")
# Check for existing valid cert
from datetime import UTC, datetime
existing = self._store.load_cert(self._hostnames[0])
if existing is not None and existing.expires_at > datetime.now(UTC):
self._bundle = existing
log.info("tls.cert.loaded", domain=self._hostnames[0])
return
# Request new cert via ACME (plain HTTP for initial request)
lacme = _require_lacme()
from lacme.challenges.http01 import HTTP01Handler
directory_url = f"{self._console_url}/acme/directory"
async with lacme.Client(
directory_url=directory_url,
store=self._store,
event_dispatcher=self._event_dispatcher,
challenge_handler=HTTP01Handler(),
allow_insecure=True,
) as client:
self._bundle = await client.issue(self._hostnames)
self._store.save_cert(self._bundle)
log.info("tls.cert.issued", domain=self._hostnames[0])
# -- Auto-renewal ----------------------------------------------------------
async def start_renewal(self) -> None:
"""Start background auto-renewal via the console's ACME endpoint."""
lacme = _require_lacme()
def _on_renewed(bundle: Any) -> None:
self._bundle = bundle
log.info("tls.cert.renewed", domain=bundle.domain)
from lacme.challenges.http01 import HTTP01Handler
directory_url = f"{self._console_url}/acme/directory"
client = lacme.Client(
directory_url=directory_url,
store=self._store,
event_dispatcher=self._event_dispatcher,
challenge_handler=HTTP01Handler(),
allow_insecure=True,
)
await client.__aenter__()
manager = lacme.RenewalManager(
client=client,
store=self._store,
interval_hours=_RENEW_INTERVAL_HOURS,
days_before_expiry=_RENEW_BEFORE_EXPIRY_DAYS,
on_renewed=_on_renewed,
event_dispatcher=self._event_dispatcher,
)
self._renewal_task = manager.start()
self._renewal_client = client
log.info("tls.renewal.started", directory=directory_url)
async def stop_renewal(self) -> None:
"""Stop background renewal and close the ACME client."""
import contextlib
if self._renewal_task is not None:
import asyncio
self._renewal_task.cancel()
try:
with contextlib.suppress(asyncio.CancelledError):
await self._renewal_task
except Exception:
log.exception("tls.renewal.stop_error")
self._renewal_task = None
if self._renewal_client is not None:
with contextlib.suppress(Exception):
await self._renewal_client.__aexit__(None, None, None)
self._renewal_client = None
# -- SSL contexts ----------------------------------------------------------
def get_server_ssl_context(self) -> ssl.SSLContext | None:
"""Build SSL context for uvicorn HTTPS listener."""
if self._bundle is None or self._ca_pem is None:
return None
_require_lacme()
from lacme.mtls import server_ssl_context
return server_ssl_context( # type: ignore[no-any-return]
cert_pem=self._bundle.fullchain_pem,
key_pem=self._bundle.key_pem,
ca_cert_pem=self._ca_pem,
)
def get_client_ssl_context(self) -> ssl.SSLContext | None:
"""Build mTLS client context for httpx connections."""
if self._bundle is None or self._ca_pem is None:
return None
_require_lacme()
from lacme.mtls import client_ssl_context
return client_ssl_context( # type: ignore[no-any-return]
cert_pem=self._bundle.cert_pem,
key_pem=self._bundle.key_pem,
ca_cert_pem=self._ca_pem,
)
# -- Properties ------------------------------------------------------------
@property
def ca_pem(self) -> bytes | None:
return self._ca_pem
@property
def bundle(self) -> Any | None:
return self._bundle
@property
def initialized(self) -> bool:
return self._bundle is not None and self._ca_pem is not None
+127
View File
@@ -0,0 +1,127 @@
"""lacme Store adapter backed by turnstone's storage backend.
Bridges lacme's Store protocol to turnstone's StorageBackend, keeping
all TLS state (account keys, CA, certificates) in the shared database
rather than the filesystem.
"""
from __future__ import annotations
import json
from datetime import UTC, datetime
from typing import TYPE_CHECKING, Any
if TYPE_CHECKING:
from turnstone.core.storage._protocol import StorageBackend
def _parse_utc(iso: str) -> datetime:
"""Parse an ISO timestamp, assuming UTC if naive."""
dt = datetime.fromisoformat(iso)
if dt.tzinfo is None:
dt = dt.replace(tzinfo=UTC)
return dt
def _ensure_lacme() -> Any:
"""Import lacme, raising a clear error if not installed."""
try:
import lacme
except ImportError:
raise ImportError(
"lacme is required for TLS support. Install with: pip install turnstone[tls]",
) from None
return lacme
class StorageStore:
"""lacme Store implementation backed by turnstone's database.
Implements the 7-method Store protocol that lacme's CertificateAuthority
and Client use for persistence.
"""
def __init__(self, storage: StorageBackend) -> None:
self._storage = storage
# -- Account key -----------------------------------------------------------
def save_account_key(self, key: Any) -> None:
"""Persist the ACME account private key."""
from cryptography.hazmat.primitives.serialization import (
Encoding,
NoEncryption,
PrivateFormat,
)
key_pem = key.private_bytes(Encoding.PEM, PrivateFormat.PKCS8, NoEncryption()).decode()
self._storage.save_tls_account_key("default", key_pem)
def load_account_key(self) -> Any | None:
"""Load the ACME account private key, or None."""
from cryptography.hazmat.primitives.serialization import load_pem_private_key
pem = self._storage.load_tls_account_key("default")
if pem is None:
return None
return load_pem_private_key(pem.encode(), password=None)
# -- CA --------------------------------------------------------------------
def save_ca(self, name: str, cert_pem: bytes, key_pem: bytes) -> None:
"""Persist a CA root certificate and key."""
self._storage.save_tls_ca(name, cert_pem.decode(), key_pem.decode())
def load_ca(self, name: str) -> tuple[bytes, bytes] | None:
"""Load CA cert+key by name. Returns (cert_pem, key_pem) or None."""
row = self._storage.load_tls_ca(name)
if row is None:
return None
return row["cert_pem"].encode(), row["key_pem"].encode()
# -- Certificates ----------------------------------------------------------
def save_cert(self, bundle: Any) -> Any:
"""Persist an issued certificate bundle."""
meta = json.dumps({"domains": list(bundle.domains)})
self._storage.save_tls_cert(
domain=bundle.domain,
cert_pem=bundle.cert_pem.decode(),
fullchain_pem=bundle.fullchain_pem.decode(),
key_pem=bundle.key_pem.decode(),
issued_at=bundle.issued_at.isoformat(),
expires_at=bundle.expires_at.isoformat(),
meta=meta,
)
return bundle
def load_cert(self, domain: str) -> Any | None:
"""Load a certificate bundle by domain."""
row = self._storage.load_tls_cert(domain)
if row is None:
return None
return self._row_to_bundle(row)
def list_certs(self) -> list[Any]:
"""List all stored certificate bundles."""
rows = self._storage.list_tls_certs()
return [self._row_to_bundle(r) for r in rows]
def delete_cert(self, domain: str) -> bool:
"""Delete a stored certificate bundle by domain."""
return self._storage.delete_tls_cert(domain)
def _row_to_bundle(self, row: dict[str, Any]) -> Any:
"""Convert a storage row dict to a lacme CertBundle."""
lacme = _ensure_lacme()
meta = json.loads(row.get("meta") or "{}")
domains = tuple(meta.get("domains", [row["domain"]]))
return lacme.CertBundle(
domain=row["domain"],
domains=domains,
cert_pem=row["cert_pem"].encode(),
fullchain_pem=row["fullchain_pem"].encode(),
key_pem=row["key_pem"].encode(),
issued_at=_parse_utc(row["issued_at"]),
expires_at=_parse_utc(row["expires_at"]),
)
+1 -1
View File
@@ -63,7 +63,7 @@ class DuckDuckGoClient:
self._timeout = timeout
def search(self, query: str, max_results: int = 5, **kwargs: Any) -> str:
from ddgs import DDGS # type: ignore[import-not-found]
from ddgs import DDGS
with DDGS(timeout=int(self._timeout)) as ddgs:
raw = list(ddgs.text(query, max_results=max_results))
+1530 -193
View File
File diff suppressed because it is too large Load Diff
+14
View File
@@ -47,6 +47,10 @@ class AsyncRedisBroker:
prefix: str = "turnstone",
password: str | None = None,
response_ttl: int = 600,
ssl: bool = False,
ssl_ca_certs: str | None = None,
ssl_certfile: str | None = None,
ssl_keyfile: str | None = None,
) -> None:
self._host = host
self._port = port
@@ -54,6 +58,15 @@ class AsyncRedisBroker:
self._password = password
self._prefix = prefix
self._response_ttl = response_ttl
self._ssl_kwargs: dict[str, Any] = {}
if ssl:
self._ssl_kwargs["ssl"] = True
if ssl_ca_certs:
self._ssl_kwargs["ssl_ca_certs"] = ssl_ca_certs
if ssl_certfile:
self._ssl_kwargs["ssl_certfile"] = ssl_certfile
if ssl_keyfile:
self._ssl_kwargs["ssl_keyfile"] = ssl_keyfile
self._redis: _aredis_t.Redis[str] | None = None
self._pubsub: _aredis_t.client.PubSub | None = None
self._tasks: dict[str, asyncio.Task[None]] = {}
@@ -82,6 +95,7 @@ class AsyncRedisBroker:
password=self._password,
decode_responses=True,
retry_on_timeout=True,
**self._ssl_kwargs,
max_connections=200,
)
self._pubsub = self._redis.pubsub(ignore_subscribe_messages=True)
+55
View File
@@ -78,6 +78,8 @@ class Bridge:
heartbeat_ttl: int = 60,
auth_token: str = "",
token_manager: Any = None,
tls_verify: Any = True,
tls_cert: tuple[str, str] | None = None,
) -> None:
self._server_url = server_url.rstrip("/")
self._broker = broker or RedisBroker()
@@ -89,6 +91,8 @@ class Bridge:
self._started_at = time.time()
self._auth_token = auth_token
self._token_manager = token_manager # ServiceTokenManager (auto-rotating)
self._tls_verify = tls_verify # CA cert path or ssl.SSLContext or True
self._tls_cert = tls_cert # (cert_path, key_path) for mTLS
# Shared httpx client for short-lived POST requests (main thread only).
# Auth headers refreshed per-request via event hook so auto-rotating
@@ -97,6 +101,8 @@ class Bridge:
base_url=self._server_url,
timeout=30,
event_hooks={"request": [self._inject_auth]},
verify=self._tls_verify,
cert=self._tls_cert,
)
# Protected by _lock — accessed from main, global SSE, and per-ws SSE threads
@@ -423,6 +429,7 @@ class Bridge:
model=model,
resume_ws=resume_ws,
skill=skill,
user_id=user_id,
)
# Send initial_message only when no workstream was actually resumed.
# Use the server's `resumed` response (not just the intent) so that
@@ -491,6 +498,7 @@ class Bridge:
model: str = "",
resume_ws: str = "",
skill: str = "",
user_id: str = "",
) -> tuple[str, bool]:
"""Create a workstream on the server. Returns (ws_id, resumed)."""
try:
@@ -501,6 +509,8 @@ class Bridge:
payload["resume_ws"] = resume_ws
if skill:
payload["skill"] = skill
if user_id:
payload["user_id"] = user_id
resp = self._http.post(
"/v1/api/workstreams/new",
json=payload,
@@ -588,6 +598,8 @@ class Bridge:
base_url=self._server_url,
timeout=None,
event_hooks={"request": [self._inject_auth]},
verify=self._tls_verify,
cert=self._tls_cert,
) as sse_client:
while self._running:
# Stop if workstream was closed (thread removed from registry)
@@ -899,6 +911,8 @@ class Bridge:
base_url=self._server_url,
timeout=None,
event_hooks={"request": [self._inject_auth]},
verify=self._tls_verify,
cert=self._tls_cert,
) as sse_client:
while self._running:
try:
@@ -1121,6 +1135,45 @@ def main() -> None:
)
log.info("bridge.jwt_minted")
# TLS: request cert from console ACME if enabled
tls_verify: Any = True
tls_cert: tuple[str, str] | None = None
if os.environ.get("TURNSTONE_TLS_ENABLED", "").lower() in ("true", "1", "yes"):
try:
import asyncio
import socket
from turnstone.core.storage import init_storage
from turnstone.core.tls import TLSClient
db_backend = os.environ.get("TURNSTONE_DB_BACKEND", "sqlite")
db_url = os.environ.get("TURNSTONE_DB_URL", "")
db_path = os.environ.get("TURNSTONE_DB_PATH", "")
storage = init_storage(db_backend, path=db_path, url=db_url)
hostname = socket.getfqdn()
hostnames = [hostname, "localhost", "127.0.0.1"]
extra_sans = os.environ.get("TURNSTONE_TLS_SANS", "")
if extra_sans:
hostnames.extend(s.strip() for s in extra_sans.split(",") if s.strip())
tls_client = TLSClient(
storage=storage,
hostnames=hostnames,
)
asyncio.run(tls_client.init())
ssl_ctx = tls_client.get_client_ssl_context()
if ssl_ctx:
# SSLContext has both CA (verify server) and client cert
# (present to server) loaded — full mTLS in one object
tls_verify = ssl_ctx
if args.server_url.startswith("http://"):
args.server_url = args.server_url.replace("http://", "https://")
log.info("bridge.tls.enabled: %s", args.server_url)
except ImportError:
log.warning("TLS enabled but lacme not installed")
except Exception:
log.warning("bridge.tls.init_failed", exc_info=True)
bridge = Bridge(
server_url=args.server_url,
broker=broker,
@@ -1129,6 +1182,8 @@ def main() -> None:
heartbeat_ttl=args.heartbeat_ttl,
auth_token=auth_token,
token_manager=token_manager,
tls_verify=tls_verify,
tls_cert=tls_cert,
)
bridge.run()
+40 -3
View File
@@ -122,11 +122,24 @@ class RedisBroker:
prefix: str = "turnstone",
password: str | None = None,
response_ttl: int = 600,
ssl: bool = False,
ssl_ca_certs: str | None = None,
ssl_certfile: str | None = None,
ssl_keyfile: str | None = None,
) -> None:
import redis
self._prefix = prefix
self._response_ttl = response_ttl
pool_kwargs: dict[str, Any] = {}
if ssl:
pool_kwargs["connection_class"] = redis.SSLConnection
if ssl_ca_certs:
pool_kwargs["ssl_ca_certs"] = ssl_ca_certs
if ssl_certfile:
pool_kwargs["ssl_certfile"] = ssl_certfile
if ssl_keyfile:
pool_kwargs["ssl_keyfile"] = ssl_keyfile
self._pool: _redis_t.ConnectionPool = redis.ConnectionPool(
host=host,
port=port,
@@ -135,6 +148,7 @@ class RedisBroker:
decode_responses=True,
retry_on_timeout=True,
max_connections=200,
**pool_kwargs,
)
self._redis: _redis_t.Redis[str] = cast(
"_redis_t.Redis[str]",
@@ -256,7 +270,7 @@ class RedisBroker:
def add_redis_args(parser: Any) -> None:
"""Add ``--redis-host``, ``--redis-port``, ``--redis-password``, ``--redis-db``."""
"""Add Redis CLI arguments including TLS options."""
import os
parser.add_argument(
@@ -281,6 +295,27 @@ def add_redis_args(parser: Any) -> None:
default=0,
help="Redis DB number (default: %(default)s)",
)
parser.add_argument("--redis-tls", action="store_true", help="Enable Redis TLS")
parser.add_argument("--redis-tls-ca", default=None, help="Redis CA cert path")
parser.add_argument("--redis-tls-cert", default=None, help="Redis client cert path")
parser.add_argument("--redis-tls-key", default=None, help="Redis client key path")
def _redis_tls_kwargs(args: Any) -> dict[str, Any]:
"""Extract Redis TLS kwargs from parsed args."""
kwargs: dict[str, Any] = {}
if getattr(args, "redis_tls", False):
kwargs["ssl"] = True
ca = getattr(args, "redis_tls_ca", None)
if ca:
kwargs["ssl_ca_certs"] = ca
cert = getattr(args, "redis_tls_cert", None)
if cert:
kwargs["ssl_certfile"] = cert
key = getattr(args, "redis_tls_key", None)
if key:
kwargs["ssl_keyfile"] = key
return kwargs
def broker_from_args(args: Any) -> RedisBroker:
@@ -289,7 +324,8 @@ def broker_from_args(args: Any) -> RedisBroker:
host=args.redis_host,
port=args.redis_port,
db=args.redis_db,
password=args.redis_password,
password=args.redis_password or None,
**_redis_tls_kwargs(args),
)
@@ -301,5 +337,6 @@ def async_broker_from_args(args: Any) -> Any:
host=args.redis_host,
port=args.redis_port,
db=args.redis_db,
password=args.redis_password,
password=args.redis_password or None,
**_redis_tls_kwargs(args),
)
+21 -4
View File
@@ -25,12 +25,18 @@ class _BaseClient:
token: str = "",
timeout: float = 30.0,
httpx_client: httpx.AsyncClient | None = None,
ca_cert: str | None = None,
client_cert: str | None = None,
client_key: str | None = None,
) -> None:
"""Initialise the client.
When *httpx_client* is provided it is used directly and *base_url*,
*token*, and *timeout* are ignored configure headers and base URL
on the injected client instead.
When *httpx_client* is provided it is used directly and all other
params are ignored configure headers, base URL, and TLS on the
injected client instead.
For mTLS, pass *ca_cert* (CA bundle path), *client_cert* and
*client_key* (client certificate + key paths).
"""
headers: dict[str, str] = {}
if token:
@@ -39,8 +45,19 @@ class _BaseClient:
self._client = httpx_client
self._owns_client = False
else:
tls_kwargs: dict[str, Any] = {}
if ca_cert:
tls_kwargs["verify"] = ca_cert
if client_cert or client_key:
if not (client_cert and client_key):
raise ValueError("Both client_cert and client_key must be provided for mTLS")
tls_kwargs["cert"] = (client_cert, client_key)
self._client = httpx.AsyncClient(
base_url=base_url, timeout=timeout, headers=headers, follow_redirects=True
base_url=base_url,
timeout=timeout,
headers=headers,
follow_redirects=True,
**tls_kwargs,
)
self._owns_client = True
+23 -2
View File
@@ -76,8 +76,19 @@ class AsyncTurnstoneConsole(_BaseClient):
token: str = "",
timeout: float = 30.0,
httpx_client: httpx.AsyncClient | None = None,
ca_cert: str | None = None,
client_cert: str | None = None,
client_key: str | None = None,
) -> None:
super().__init__(base_url=base_url, token=token, timeout=timeout, httpx_client=httpx_client)
super().__init__(
base_url=base_url,
token=token,
timeout=timeout,
httpx_client=httpx_client,
ca_cert=ca_cert,
client_cert=client_cert,
client_key=client_key,
)
# -- cluster overview ----------------------------------------------------
@@ -847,9 +858,19 @@ class TurnstoneConsole:
base_url: str = "http://localhost:8081",
token: str = "",
timeout: float = 30.0,
ca_cert: str | None = None,
client_cert: str | None = None,
client_key: str | None = None,
) -> None:
self._runner = _SyncRunner()
self._async = AsyncTurnstoneConsole(base_url=base_url, token=token, timeout=timeout)
self._async = AsyncTurnstoneConsole(
base_url=base_url,
token=token,
timeout=timeout,
ca_cert=ca_cert,
client_cert=client_cert,
client_key=client_key,
)
# -- cluster overview ----------------------------------------------------
+23 -2
View File
@@ -60,8 +60,19 @@ class AsyncTurnstoneServer(_BaseClient):
token: str = "",
timeout: float = 30.0,
httpx_client: httpx.AsyncClient | None = None,
ca_cert: str | None = None,
client_cert: str | None = None,
client_key: str | None = None,
) -> None:
super().__init__(base_url=base_url, token=token, timeout=timeout, httpx_client=httpx_client)
super().__init__(
base_url=base_url,
token=token,
timeout=timeout,
httpx_client=httpx_client,
ca_cert=ca_cert,
client_cert=client_cert,
client_key=client_key,
)
# -- workstream management -----------------------------------------------
@@ -395,9 +406,19 @@ class TurnstoneServer:
base_url: str = "http://localhost:8080",
token: str = "",
timeout: float = 30.0,
ca_cert: str | None = None,
client_cert: str | None = None,
client_key: str | None = None,
) -> None:
self._runner = _SyncRunner()
self._async = AsyncTurnstoneServer(base_url=base_url, token=token, timeout=timeout)
self._async = AsyncTurnstoneServer(
base_url=base_url,
token=token,
timeout=timeout,
ca_cert=ca_cert,
client_cert=client_cert,
client_key=client_key,
)
# -- workstream management -----------------------------------------------
+82 -3
View File
@@ -18,7 +18,6 @@ import functools
import json
import os
import queue
import socket
import sys
import textwrap
import threading
@@ -1280,6 +1279,18 @@ async def create_workstream(request: Request) -> JSONResponse:
skip: bool = request.app.state.skip_permissions
auth = getattr(getattr(request, "state", None), "auth_result", None)
uid: str = getattr(auth, "user_id", "") or ""
# Trusted services (bridge, console) may forward the real user_id in the
# request body when creating workstreams on behalf of a user. Only service
# identities are trusted — end-user tokens (including console-proxy tokens
# that carry the real user's identity) must not override user_id.
trusted_sources = {"bridge", "console"}
if (
body.get("user_id")
and isinstance(body["user_id"], str)
and auth is not None
and auth.token_source in trusted_sources
):
uid = body["user_id"]
body_skill = body.get("skill", "")
resume_ws_id = body.get("resume_ws", "")
# Resolve skill — applies content + session config (model, temperature, etc.)
@@ -1839,8 +1850,19 @@ async def _lifespan(app: Starlette) -> AsyncGenerator[None, None]:
"OIDC JWKS prefetch failed — will retry on first login",
exc_info=True,
)
# TLS: start auto-renewal if client was initialized
tls_client = getattr(app.state, "tls_client", None)
if tls_client is not None:
try:
await tls_client.start_renewal()
except Exception:
log.warning("TLS auto-renewal startup failed", exc_info=True)
yield
# Shutdown
tls_client = getattr(app.state, "tls_client", None)
if tls_client is not None:
await tls_client.stop_renewal()
if app.state.watch_runner:
app.state.watch_runner.stop()
if app.state.health_monitor:
@@ -2063,6 +2085,8 @@ def main() -> None:
configure_logging_from_args(args, "server")
import socket
# Initialize storage backend
from turnstone.core.storage import init_storage
@@ -2074,7 +2098,17 @@ def main() -> None:
db_pool_size = int(
getattr(args, "db_pool_size", None) or os.environ.get("TURNSTONE_DB_POOL_SIZE", "2")
)
init_storage(db_backend, path=db_path, url=db_url, pool_size=db_pool_size)
init_storage(
db_backend,
path=db_path,
url=db_url,
pool_size=db_pool_size,
sslmode=getattr(args, "db_sslmode", None) or os.environ.get("TURNSTONE_DB_SSLMODE", ""),
sslrootcert=getattr(args, "db_sslrootcert", None)
or os.environ.get("TURNSTONE_DB_SSLROOTCERT", ""),
sslcert=getattr(args, "db_sslcert", None) or os.environ.get("TURNSTONE_DB_SSLCERT", ""),
sslkey=getattr(args, "db_sslkey", None) or os.environ.get("TURNSTONE_DB_SSLKEY", ""),
)
# Server-owned node identity (needed before ConfigStore for node_id scoping)
def _default_node_id() -> str:
@@ -2416,11 +2450,56 @@ def main() -> None:
)
log.info("Max workstreams: %s", config_store.get("server.max_workstreams"))
log.info("Node ID: %s", _node_id)
# TLS: request cert from console ACME if enabled
ssl_kwargs: dict[str, Any] = {}
if config_store.get("tls.enabled"):
try:
import asyncio
from turnstone.core.tls import TLSClient
hostname = socket.getfqdn()
hostnames = [hostname, "localhost", "127.0.0.1"]
# Only add bind host if it's a concrete address
if args.host not in ("0.0.0.0", "::", ""):
hostnames.append(args.host)
# Additional SANs from env (e.g. Docker service name)
extra_sans = os.environ.get("TURNSTONE_TLS_SANS", "")
if extra_sans:
hostnames.extend(s.strip() for s in extra_sans.split(",") if s.strip())
tls_client = TLSClient(
storage=get_storage(),
hostnames=hostnames,
)
asyncio.run(tls_client.init())
bundle = tls_client.bundle
if bundle:
from lacme.mtls import write_pem_files_persistent
pem_paths = write_pem_files_persistent(
bundle,
ca_pem=tls_client.ca_pem,
)
ssl_kwargs.update(pem_paths.as_uvicorn_kwargs())
if tls_client.ca_pem:
import ssl as _ssl
ssl_kwargs["ssl_cert_reqs"] = _ssl.CERT_REQUIRED
# Store client on app state for lifespan renewal
app.state.tls_client = tls_client
log.info("TLS enabled — serving HTTPS")
else:
log.warning("TLS enabled but no cert available")
except Exception:
log.warning("TLS initialization failed — serving plain HTTP", exc_info=True)
print("Press Ctrl+C to stop.")
import uvicorn
uvicorn.run(app, host=args.host, port=args.port, log_level="warning")
uvicorn.run(app, host=args.host, port=args.port, log_level="warning", **ssl_kwargs)
if __name__ == "__main__":
File diff suppressed because one or more lines are too long

Some files were not shown because too many files have changed in this diff Show More