mirror of
https://github.com/turnstonelabs/turnstone.git
synced 2026-08-13 15:32:24 -06:00
Compare commits
33 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 087f5b49f6 | |||
| fd507c6a3c | |||
| 562c3c8ab7 | |||
| 4773535bb8 | |||
| 7492816ab2 | |||
| d6ba1d5e25 | |||
| 41d1b27d34 | |||
| 8bc284c60e | |||
| a322d6b1d1 | |||
| 70d495aa5b | |||
| de64535221 | |||
| 187d004033 | |||
| 7ea150fa71 | |||
| 4d665a5f62 | |||
| 3bc3250869 | |||
| db937486cf | |||
| 554257ac4d | |||
| 5f0004dc91 | |||
| 6cc1b3a5bd | |||
| cc9afe94cd | |||
| 136b75fdef | |||
| 4d1107839b | |||
| 7d66bc2159 | |||
| 165cbb2d29 | |||
| c79c47b940 | |||
| 660c273e8e | |||
| c7586abd0a | |||
| 14d57176ce | |||
| 96084ca5f3 | |||
| 8b92302247 | |||
| e195ca54a6 | |||
| 50277cd4de | |||
| fb190f8977 |
@@ -5,6 +5,7 @@ on:
|
||||
tags: ["v*"]
|
||||
|
||||
permissions:
|
||||
contents: write
|
||||
id-token: write
|
||||
|
||||
jobs:
|
||||
@@ -19,3 +20,10 @@ jobs:
|
||||
- run: pip install build
|
||||
- run: python -m build
|
||||
- uses: pypa/gh-action-pypi-publish@release/v1
|
||||
|
||||
- name: Create GitHub Release
|
||||
uses: softprops/action-gh-release@v2
|
||||
with:
|
||||
generate_release_notes: true
|
||||
draft: false
|
||||
prerelease: ${{ contains(github.ref, '-') }}
|
||||
|
||||
@@ -11,61 +11,18 @@ Named after the [Ruddy Turnstone](https://en.wikipedia.org/wiki/Ruddy_turnstone)
|
||||
|
||||
## What it does
|
||||
|
||||
Turnstone gives LLMs tools — shell, files, search, web, planning — and orchestrates multi-turn conversations where the model investigates, acts, and reports. It runs as:
|
||||
Turnstone gives LLMs tools — shell, files, search, web, planning — and orchestrates multi-turn conversations where the model investigates, acts, and reports. Native deferred tool loading for Anthropic and OpenAI APIs reduces token overhead and improves tool selection accuracy when MCP servers expose many tools; local models (vLLM, llama.cpp) get a transparent client-side BM25 fallback. It runs as:
|
||||
|
||||
- **Interactive sessions** — terminal CLI or browser UI with parallel workstreams
|
||||
- **Queue-driven agents** — trigger workstreams via message queue, stream progress, approve or auto-approve tool use
|
||||
- **Multi-node clusters** — generic work load-balances across nodes, directed work routes to a specific server
|
||||
- **Cluster dashboard** — real-time view of all nodes and workstreams, workstream creation with node targeting, reverse proxy for server UIs (only the console port needs network access)
|
||||
- **Governance & compliance** — role-based access control, tool policies, usage tracking, and append-only audit logs
|
||||
- **Cluster simulator** — test the stack at scale (up to 1000 nodes) without an LLM backend
|
||||
|
||||
```mermaid
|
||||
graph LR
|
||||
subgraph Clients
|
||||
CLI[turnstone CLI]
|
||||
UI[Browser UI]
|
||||
SDK[SDK / API]
|
||||
Discord[Discord / Slack]
|
||||
end
|
||||
|
||||
Console[turnstone-console<br/><i>dashboard + proxy</i>]
|
||||
Channel[turnstone-channel<br/><i>platform gateway</i>]
|
||||
|
||||
subgraph Cluster
|
||||
Redis[(Redis MQ)]
|
||||
DB[(PostgreSQL / SQLite)]
|
||||
|
||||
subgraph Node A
|
||||
BridgeA[bridge]
|
||||
ServerA[server]
|
||||
end
|
||||
subgraph Node B
|
||||
BridgeB[bridge]
|
||||
ServerB[server]
|
||||
end
|
||||
end
|
||||
|
||||
LLM[LLM Provider<br/><i>OpenAI · Anthropic · local</i>]
|
||||
|
||||
CLI --> ServerA
|
||||
UI --> ServerB
|
||||
SDK --> Redis
|
||||
Discord --> Channel
|
||||
|
||||
Channel <--> Redis
|
||||
Console --> Redis
|
||||
Redis --> BridgeA & BridgeB
|
||||
BridgeA --> ServerA
|
||||
BridgeB --> ServerB
|
||||
ServerA & ServerB --> LLM
|
||||
|
||||
ServerA & ServerB --> DB
|
||||
Console --> DB
|
||||
Channel --> DB
|
||||
ServerA -.->|notify| Channel
|
||||
BridgeA & BridgeB -.->|events| Redis
|
||||
Console -.->|proxy| ServerA & ServerB
|
||||
```
|
||||
<p align="center">
|
||||
<img src="docs/diagrams/architecture-overview.svg" alt="Turnstone system architecture — data flow from clients through gateways, Redis MQ, cluster nodes, to LLM providers" width="960"/>
|
||||
</p>
|
||||
|
||||
## Quickstart
|
||||
|
||||
@@ -171,6 +128,23 @@ Detailed UML diagrams are available in [`docs/diagrams/`](docs/diagrams/):
|
||||
| [Deployment](docs/diagrams/png/12-deployment.png) | Docker Compose service topology |
|
||||
| [SDK Architecture](docs/diagrams/png/13-sdk-architecture.png) | Python + TypeScript client libraries |
|
||||
| [Storage Architecture](docs/diagrams/png/14-storage-architecture.png) | Pluggable database backends (SQLite + PostgreSQL) |
|
||||
| [Auth Architecture](docs/diagrams/png/15-auth-architecture.png) | JWT, scopes, token types, login flows |
|
||||
| [Channel Architecture](docs/diagrams/png/16-channel-architecture.png) | Discord/Slack adapter protocol and routing |
|
||||
| [Notify Flow](docs/diagrams/png/17-notify-flow.png) | Channel notification dispatch |
|
||||
| [Watch Architecture](docs/diagrams/png/18-watch-architecture.png) | Periodic command polling daemon |
|
||||
| [Governance Architecture](docs/diagrams/png/19-governance-architecture.png) | RBAC, policies, audit, usage enforcement flow |
|
||||
|
||||
### Governance
|
||||
|
||||
Turnstone includes a built-in governance layer for enterprise deployments — manage who can do what, which tools run unattended, and where every token goes.
|
||||
|
||||
- **RBAC** — 15 granular permissions, 3 built-in roles (admin / operator / viewer), custom roles, privilege escalation prevention
|
||||
- **Tool policies** — glob-pattern rules (`allow` / `deny` / `ask`) with priority ordering; automate approvals or lock down dangerous tools
|
||||
- **Prompt templates** — reusable system messages with `{{variable}}` substitution and categories
|
||||
- **Usage tracking** — per-request token and tool metrics, aggregation by day / model / user, automatic 90-day pruning
|
||||
- **Audit logging** — append-only event trail for all admin mutations, IP-aware, 365-day retention
|
||||
|
||||
All governance features are managed through the console admin panel (10 tabs) and the full REST API. See [docs/governance.md](docs/governance.md) for setup and configuration.
|
||||
|
||||
## Multi-node routing
|
||||
|
||||
@@ -195,12 +169,12 @@ Bridges BLPOP from their per-node queue (priority) then the shared queue. Direct
|
||||
|
||||
## Tools
|
||||
|
||||
14 built-in tools, 2 agent tools, plus external tools via MCP:
|
||||
16 built-in tools, 2 agent tools, plus external tools via MCP:
|
||||
|
||||
| Tool | Description | Auto-approved |
|
||||
|------|-------------|:---:|
|
||||
| `bash` | Execute shell commands | |
|
||||
| `read_file` | Read file contents | yes |
|
||||
| `read_file` | Read file contents (text or images with vision models) | yes |
|
||||
| `write_file` | Write/create files | |
|
||||
| `edit_file` | Fuzzy-match file editing | |
|
||||
| `search` | Search files by name/content | yes |
|
||||
@@ -211,13 +185,17 @@ Bridges BLPOP from their per-node queue (priority) then the shared queue. Direct
|
||||
| `remember` | Save persistent facts | yes |
|
||||
| `recall` | Search memories and history | yes |
|
||||
| `forget` | Remove a memory | yes |
|
||||
| `notify` | Send notifications to linked channels | yes |
|
||||
| `watch` | Periodic command polling with conditions | |
|
||||
| `task` | Spawn autonomous sub-agent | |
|
||||
| `plan` | Explore codebase, write .plan.md | |
|
||||
| `mcp__*` | External tools from MCP servers | |
|
||||
|
||||
When the total tool count exceeds a configurable threshold (default 20), MCP tools are automatically deferred using native `defer_loading` on Anthropic and OpenAI APIs, or a transparent client-side BM25 search for local models. The LLM discovers deferred tools on demand via a `tool_search` capability — no configuration needed beyond `--tool-search auto` (the default).
|
||||
|
||||
### MCP Tool Servers
|
||||
|
||||
Turnstone supports the [Model Context Protocol](https://modelcontextprotocol.io/) (MCP) for connecting external tool servers. MCP tools are discovered at startup, converted to OpenAI function-calling format, and merged with built-in tools. Each MCP tool is prefixed with `mcp__{server}__{tool}` to avoid name collisions.
|
||||
Turnstone supports the [Model Context Protocol](https://modelcontextprotocol.io/) (MCP) for connecting external tool servers. MCP tools are discovered at startup, converted to OpenAI function-calling format, and merged with built-in tools. Each MCP tool is prefixed with `mcp__{server}__{tool}` to avoid name collisions. Tool lists stay fresh via push notifications (`tools.listChanged`), periodic polling for servers without push, and manual `/mcp refresh`.
|
||||
|
||||
Configure via `config.toml` or `--mcp-config`:
|
||||
|
||||
@@ -237,7 +215,7 @@ turnstone --mcp-config ~/.config/turnstone/mcp.json
|
||||
turnstone-server --mcp-config ~/.config/turnstone/mcp.json
|
||||
```
|
||||
|
||||
Use `/mcp` in the REPL to list connected tools. MCP tools require user approval by default (overridden by `--skip-permissions` or UI auto-approve).
|
||||
Use `/mcp` in the REPL to list connected tools, `/mcp refresh` to re-fetch tool lists from servers. MCP tools require user approval by default (overridden by `--skip-permissions` or UI auto-approve).
|
||||
|
||||
### Multi-Model and Multi-Provider Support
|
||||
|
||||
@@ -292,6 +270,9 @@ agent_model = "" # model alias for plan/task sub-agents
|
||||
[tools]
|
||||
timeout = 30
|
||||
skip_permissions = false
|
||||
search = "auto" # "auto" (enable when >threshold tools), "on", "off"
|
||||
search_threshold = 20 # min tools before tool search activates
|
||||
search_max_results = 5 # max tools returned per search query
|
||||
|
||||
[server]
|
||||
host = "0.0.0.0"
|
||||
@@ -332,6 +313,7 @@ path = ".turnstone.db" # SQLite file path (relative to working directory)
|
||||
|
||||
[mcp]
|
||||
config_path = "" # path to MCP JSON config file (alternative to TOML sections)
|
||||
refresh_interval = 14400 # periodic refresh for servers without push notifications (seconds, 0 to disable)
|
||||
|
||||
[mcp.servers.example] # one section per MCP server
|
||||
command = "npx"
|
||||
|
||||
+219
-1
@@ -5,8 +5,8 @@
|
||||
# Default (SQLite): docker compose up
|
||||
# Production (PG): DB_BACKEND=postgresql docker compose --profile production up
|
||||
# (or set DB_BACKEND=postgresql in .env)
|
||||
# 10-node cluster: docker compose --profile cluster up
|
||||
# With simulator: docker compose --profile sim up
|
||||
# Scale bridges: docker compose up --scale bridge=3
|
||||
# =============================================================================
|
||||
|
||||
name: turnstone
|
||||
@@ -28,6 +28,7 @@ services:
|
||||
image: postgres:17-alpine
|
||||
profiles:
|
||||
- production
|
||||
- cluster
|
||||
environment:
|
||||
POSTGRES_DB: turnstone
|
||||
POSTGRES_USER: ${POSTGRES_USER:-turnstone}
|
||||
@@ -206,6 +207,7 @@ services:
|
||||
dockerfile: Dockerfile
|
||||
profiles:
|
||||
- production
|
||||
- cluster
|
||||
command:
|
||||
- sh
|
||||
- -c
|
||||
@@ -272,3 +274,219 @@ services:
|
||||
redis:
|
||||
condition: service_healthy
|
||||
restart: "no"
|
||||
|
||||
# ===================================================================
|
||||
# 10-node cluster (profile: cluster)
|
||||
#
|
||||
# Each node is a server + bridge pair. All share the same PostgreSQL
|
||||
# and Redis instances. Access via console at :8090.
|
||||
#
|
||||
# Start: docker compose --profile cluster up
|
||||
# ===================================================================
|
||||
|
||||
# -- cluster servers ------------------------------------------------
|
||||
|
||||
server-1: &cluster-server
|
||||
build: { context: ., dockerfile: Dockerfile }
|
||||
profiles: [cluster]
|
||||
command: &cluster-server-cmd
|
||||
- sh
|
||||
- -c
|
||||
- >-
|
||||
turnstone-server
|
||||
--host 0.0.0.0
|
||||
--port 8080
|
||||
--base-url "$${LLM_BASE_URL}"
|
||||
--api-key "$${OPENAI_API_KEY}"
|
||||
$${MODEL:+--model $$MODEL}
|
||||
$${SKIP_PERMISSIONS:+--skip-permissions}
|
||||
volumes: [turnstone-data:/data]
|
||||
environment: &cluster-server-env
|
||||
LLM_BASE_URL: ${LLM_BASE_URL:-http://host.docker.internal:8000/v1}
|
||||
OPENAI_API_KEY: ${OPENAI_API_KEY:-dummy}
|
||||
TAVILY_API_KEY: ${TAVILY_API_KEY:-}
|
||||
SKIP_PERMISSIONS: ${SKIP_PERMISSIONS:-}
|
||||
TURNSTONE_AUTH_ENABLED: ${TURNSTONE_AUTH_ENABLED:-}
|
||||
TURNSTONE_AUTH_TOKEN: ${TURNSTONE_AUTH_TOKEN:-}
|
||||
TURNSTONE_JWT_SECRET: ${TURNSTONE_JWT_SECRET:-}
|
||||
MODEL: ${MODEL:-}
|
||||
TURNSTONE_DB_BACKEND: ${DB_BACKEND:-postgresql}
|
||||
TURNSTONE_DB_URL: ${DATABASE_URL:-postgresql://${POSTGRES_USER:-turnstone}:${POSTGRES_PASSWORD:?}@postgres:5432/turnstone}
|
||||
TURNSTONE_NODE_ID: node-1
|
||||
extra_hosts: ["host.docker.internal:host-gateway"]
|
||||
networks: [turnstone-net]
|
||||
depends_on:
|
||||
redis: { condition: service_healthy }
|
||||
postgres: { condition: service_healthy }
|
||||
healthcheck:
|
||||
test: ["CMD", "python", "/usr/local/bin/healthcheck.py", "http://127.0.0.1:8080/health"]
|
||||
interval: 10s
|
||||
timeout: 5s
|
||||
retries: 3
|
||||
start_period: 15s
|
||||
deploy:
|
||||
resources:
|
||||
limits: { memory: 384M, cpus: '0.5' }
|
||||
restart: unless-stopped
|
||||
|
||||
server-2:
|
||||
<<: *cluster-server
|
||||
environment: { <<: *cluster-server-env, TURNSTONE_NODE_ID: node-2 }
|
||||
server-3:
|
||||
<<: *cluster-server
|
||||
environment: { <<: *cluster-server-env, TURNSTONE_NODE_ID: node-3 }
|
||||
server-4:
|
||||
<<: *cluster-server
|
||||
environment: { <<: *cluster-server-env, TURNSTONE_NODE_ID: node-4 }
|
||||
server-5:
|
||||
<<: *cluster-server
|
||||
environment: { <<: *cluster-server-env, TURNSTONE_NODE_ID: node-5 }
|
||||
server-6:
|
||||
<<: *cluster-server
|
||||
environment: { <<: *cluster-server-env, TURNSTONE_NODE_ID: node-6 }
|
||||
server-7:
|
||||
<<: *cluster-server
|
||||
environment: { <<: *cluster-server-env, TURNSTONE_NODE_ID: node-7 }
|
||||
server-8:
|
||||
<<: *cluster-server
|
||||
environment: { <<: *cluster-server-env, TURNSTONE_NODE_ID: node-8 }
|
||||
server-9:
|
||||
<<: *cluster-server
|
||||
environment: { <<: *cluster-server-env, TURNSTONE_NODE_ID: node-9 }
|
||||
server-10:
|
||||
<<: *cluster-server
|
||||
environment: { <<: *cluster-server-env, TURNSTONE_NODE_ID: node-10 }
|
||||
|
||||
# -- cluster bridges ------------------------------------------------
|
||||
|
||||
bridge-1: &cluster-bridge
|
||||
build: { context: ., dockerfile: Dockerfile }
|
||||
profiles: [cluster]
|
||||
command:
|
||||
- turnstone-bridge
|
||||
- --server-url=http://server-1:8080
|
||||
- --redis-host=redis
|
||||
- --redis-port=6379
|
||||
- --heartbeat-ttl=${HEARTBEAT_TTL:-60}
|
||||
- --approval-timeout=${APPROVAL_TIMEOUT:-3600}
|
||||
environment: &cluster-bridge-env
|
||||
REDIS_PASSWORD: ${REDIS_PASSWORD:-}
|
||||
TURNSTONE_AUTH_TOKEN: ${TURNSTONE_AUTH_TOKEN:-}
|
||||
TURNSTONE_JWT_SECRET: ${TURNSTONE_JWT_SECRET:-}
|
||||
networks: [turnstone-net]
|
||||
depends_on:
|
||||
server-1: { condition: service_healthy }
|
||||
redis: { condition: service_healthy }
|
||||
deploy:
|
||||
resources:
|
||||
limits: { memory: 256M, cpus: '0.25' }
|
||||
restart: unless-stopped
|
||||
|
||||
bridge-2:
|
||||
<<: *cluster-bridge
|
||||
command:
|
||||
- turnstone-bridge
|
||||
- --server-url=http://server-2:8080
|
||||
- --redis-host=redis
|
||||
- --redis-port=6379
|
||||
- --heartbeat-ttl=${HEARTBEAT_TTL:-60}
|
||||
- --approval-timeout=${APPROVAL_TIMEOUT:-3600}
|
||||
depends_on:
|
||||
server-2: { condition: service_healthy }
|
||||
redis: { condition: service_healthy }
|
||||
bridge-3:
|
||||
<<: *cluster-bridge
|
||||
command:
|
||||
- turnstone-bridge
|
||||
- --server-url=http://server-3:8080
|
||||
- --redis-host=redis
|
||||
- --redis-port=6379
|
||||
- --heartbeat-ttl=${HEARTBEAT_TTL:-60}
|
||||
- --approval-timeout=${APPROVAL_TIMEOUT:-3600}
|
||||
depends_on:
|
||||
server-3: { condition: service_healthy }
|
||||
redis: { condition: service_healthy }
|
||||
bridge-4:
|
||||
<<: *cluster-bridge
|
||||
command:
|
||||
- turnstone-bridge
|
||||
- --server-url=http://server-4:8080
|
||||
- --redis-host=redis
|
||||
- --redis-port=6379
|
||||
- --heartbeat-ttl=${HEARTBEAT_TTL:-60}
|
||||
- --approval-timeout=${APPROVAL_TIMEOUT:-3600}
|
||||
depends_on:
|
||||
server-4: { condition: service_healthy }
|
||||
redis: { condition: service_healthy }
|
||||
bridge-5:
|
||||
<<: *cluster-bridge
|
||||
command:
|
||||
- turnstone-bridge
|
||||
- --server-url=http://server-5:8080
|
||||
- --redis-host=redis
|
||||
- --redis-port=6379
|
||||
- --heartbeat-ttl=${HEARTBEAT_TTL:-60}
|
||||
- --approval-timeout=${APPROVAL_TIMEOUT:-3600}
|
||||
depends_on:
|
||||
server-5: { condition: service_healthy }
|
||||
redis: { condition: service_healthy }
|
||||
bridge-6:
|
||||
<<: *cluster-bridge
|
||||
command:
|
||||
- turnstone-bridge
|
||||
- --server-url=http://server-6:8080
|
||||
- --redis-host=redis
|
||||
- --redis-port=6379
|
||||
- --heartbeat-ttl=${HEARTBEAT_TTL:-60}
|
||||
- --approval-timeout=${APPROVAL_TIMEOUT:-3600}
|
||||
depends_on:
|
||||
server-6: { condition: service_healthy }
|
||||
redis: { condition: service_healthy }
|
||||
bridge-7:
|
||||
<<: *cluster-bridge
|
||||
command:
|
||||
- turnstone-bridge
|
||||
- --server-url=http://server-7:8080
|
||||
- --redis-host=redis
|
||||
- --redis-port=6379
|
||||
- --heartbeat-ttl=${HEARTBEAT_TTL:-60}
|
||||
- --approval-timeout=${APPROVAL_TIMEOUT:-3600}
|
||||
depends_on:
|
||||
server-7: { condition: service_healthy }
|
||||
redis: { condition: service_healthy }
|
||||
bridge-8:
|
||||
<<: *cluster-bridge
|
||||
command:
|
||||
- turnstone-bridge
|
||||
- --server-url=http://server-8:8080
|
||||
- --redis-host=redis
|
||||
- --redis-port=6379
|
||||
- --heartbeat-ttl=${HEARTBEAT_TTL:-60}
|
||||
- --approval-timeout=${APPROVAL_TIMEOUT:-3600}
|
||||
depends_on:
|
||||
server-8: { condition: service_healthy }
|
||||
redis: { condition: service_healthy }
|
||||
bridge-9:
|
||||
<<: *cluster-bridge
|
||||
command:
|
||||
- turnstone-bridge
|
||||
- --server-url=http://server-9:8080
|
||||
- --redis-host=redis
|
||||
- --redis-port=6379
|
||||
- --heartbeat-ttl=${HEARTBEAT_TTL:-60}
|
||||
- --approval-timeout=${APPROVAL_TIMEOUT:-3600}
|
||||
depends_on:
|
||||
server-9: { condition: service_healthy }
|
||||
redis: { condition: service_healthy }
|
||||
bridge-10:
|
||||
<<: *cluster-bridge
|
||||
command:
|
||||
- turnstone-bridge
|
||||
- --server-url=http://server-10:8080
|
||||
- --redis-host=redis
|
||||
- --redis-port=6379
|
||||
- --heartbeat-ttl=${HEARTBEAT_TTL:-60}
|
||||
- --approval-timeout=${APPROVAL_TIMEOUT:-3600}
|
||||
depends_on:
|
||||
server-10: { condition: service_healthy }
|
||||
redis: { condition: service_healthy }
|
||||
|
||||
@@ -1,221 +0,0 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 860 520" font-family="ui-monospace,SFMono-Regular,Menlo,Monaco,Consolas,monospace" font-size="13">
|
||||
<style>
|
||||
@keyframes pulse-green { 0%,100% { opacity:0.5 } 50% { opacity:1 } }
|
||||
@keyframes pulse-yellow { 0%,100% { opacity:0.4 } 50% { opacity:1 } }
|
||||
@keyframes pulse-blue { 0%,100% { opacity:0.3 } 50% { opacity:1 } }
|
||||
@keyframes fadein { from { opacity:0 } to { opacity:1 } }
|
||||
.pg { animation: pulse-green 2s infinite }
|
||||
.py { animation: pulse-yellow 1.8s infinite }
|
||||
.pb { animation: pulse-blue 2.2s infinite }
|
||||
.f1 { animation: fadein 0.4s 0.2s both }
|
||||
.f2 { animation: fadein 0.4s 0.4s both }
|
||||
.f3 { animation: fadein 0.4s 0.6s both }
|
||||
.f4 { animation: fadein 0.4s 0.8s both }
|
||||
.f5 { animation: fadein 0.4s 1.0s both }
|
||||
.f6 { animation: fadein 0.4s 1.3s both }
|
||||
.f7 { animation: fadein 0.4s 1.5s both }
|
||||
.f8 { animation: fadein 0.4s 1.7s both }
|
||||
.f9 { animation: fadein 0.4s 1.9s both }
|
||||
.f10 { animation: fadein 0.4s 2.1s both }
|
||||
.f11 { animation: fadein 0.4s 2.3s both }
|
||||
.f12 { animation: fadein 0.4s 2.5s both }
|
||||
</style>
|
||||
|
||||
<!-- Window chrome -->
|
||||
<rect rx="10" width="860" height="520" fill="#1a1b26"/>
|
||||
<rect width="860" height="36" rx="10" fill="#16161e"/>
|
||||
<rect y="26" width="860" height="10" fill="#16161e"/>
|
||||
<circle cx="20" cy="18" r="6" fill="#f7768e"/>
|
||||
<circle cx="40" cy="18" r="6" fill="#e0af68"/>
|
||||
<circle cx="60" cy="18" r="6" fill="#9ece6a"/>
|
||||
<text x="430" y="22" text-anchor="middle" fill="#565f89" font-size="12">turnstone — console</text>
|
||||
|
||||
<!-- Header -->
|
||||
<rect y="36" width="860" height="30" fill="#24283b"/>
|
||||
<rect y="66" width="860" height="1" fill="#3b4261"/>
|
||||
<text x="16" y="56" fill="#7aa2f7" font-size="14" font-weight="bold">turnstone console</text>
|
||||
<text x="200" y="56" fill="#565f89" font-size="12">6 nodes · 10 workstreams</text>
|
||||
|
||||
<!-- ====== State cards ====== -->
|
||||
<g transform="translate(16, 78)" class="f1" opacity="0">
|
||||
<!-- RUN card -->
|
||||
<rect x="0" y="0" width="156" height="64" rx="6" fill="#24283b" stroke="#3b4261"/>
|
||||
<rect x="0" y="0" width="156" height="3" rx="6" fill="#9ece6a"/>
|
||||
<text x="78" y="30" text-anchor="middle" fill="#a9b1d6" font-size="22" font-weight="bold">3</text>
|
||||
<text x="78" y="50" text-anchor="middle" fill="#565f89" font-size="10">▸ RUN</text>
|
||||
|
||||
<!-- THINK card -->
|
||||
<rect x="168" y="0" width="156" height="64" rx="6" fill="#24283b" stroke="#3b4261"/>
|
||||
<rect x="168" y="0" width="156" height="3" rx="6" fill="#7aa2f7"/>
|
||||
<text x="246" y="30" text-anchor="middle" fill="#a9b1d6" font-size="22" font-weight="bold">2</text>
|
||||
<text x="246" y="50" text-anchor="middle" fill="#565f89" font-size="10">◌ THINK</text>
|
||||
|
||||
<!-- ATTN card -->
|
||||
<rect x="336" y="0" width="156" height="64" rx="6" fill="#24283b" stroke="#3b4261"/>
|
||||
<rect x="336" y="0" width="156" height="3" rx="6" fill="#e0af68"/>
|
||||
<text x="414" y="30" text-anchor="middle" fill="#a9b1d6" font-size="22" font-weight="bold">1</text>
|
||||
<text x="414" y="50" text-anchor="middle" fill="#565f89" font-size="10">◆ ATTN</text>
|
||||
|
||||
<!-- ERR card -->
|
||||
<rect x="504" y="0" width="156" height="64" rx="6" fill="#24283b" stroke="#3b4261"/>
|
||||
<rect x="504" y="0" width="156" height="3" rx="6" fill="#f7768e"/>
|
||||
<text x="582" y="30" text-anchor="middle" fill="#a9b1d6" font-size="22" font-weight="bold">0</text>
|
||||
<text x="582" y="50" text-anchor="middle" fill="#565f89" font-size="10">✖ ERR</text>
|
||||
|
||||
<!-- IDLE card -->
|
||||
<rect x="672" y="0" width="156" height="64" rx="6" fill="#24283b" stroke="#3b4261"/>
|
||||
<rect x="672" y="0" width="156" height="3" rx="6" fill="#565f89"/>
|
||||
<text x="750" y="30" text-anchor="middle" fill="#a9b1d6" font-size="22" font-weight="bold">4</text>
|
||||
<text x="750" y="50" text-anchor="middle" fill="#565f89" font-size="10">· IDLE</text>
|
||||
</g>
|
||||
|
||||
<!-- Aggregate bar -->
|
||||
<text x="16" y="160" fill="#565f89" font-size="11" class="f2" opacity="0">197k tokens · 42 tool calls</text>
|
||||
|
||||
<!-- ====== NODES section ====== -->
|
||||
<text x="16" y="182" fill="#7aa2f7" font-size="12" font-weight="bold" class="f3" opacity="0">NODES</text>
|
||||
|
||||
<!-- Node column headers -->
|
||||
<g transform="translate(0, 190)" class="f4" opacity="0">
|
||||
<rect width="860" height="20" fill="#24283b"/>
|
||||
<rect y="20" width="860" height="1" fill="#3b4261"/>
|
||||
<text y="14" fill="#565f89" font-size="10" letter-spacing="0.5">
|
||||
<tspan x="36">NODE</tspan>
|
||||
<tspan x="560">WS</tspan>
|
||||
<tspan x="610">RUN</tspan>
|
||||
<tspan x="660">ATTN</tspan>
|
||||
<tspan x="710">TOKENS</tspan>
|
||||
<tspan x="790">LOAD</tspan>
|
||||
</text>
|
||||
</g>
|
||||
|
||||
<!-- Node rows -->
|
||||
<g transform="translate(0, 214)">
|
||||
|
||||
<!-- Node 1: db-west-04 — 3 ws, 1 running, has-running bar -->
|
||||
<g class="f5" opacity="0">
|
||||
<rect y="0" width="860" height="38" fill="#1a1b26"/>
|
||||
<rect y="0" width="3" height="38" fill="#9ece6a"/>
|
||||
<circle cx="22" cy="19" r="4" fill="#9ece6a"/>
|
||||
<text x="36" y="23" fill="#a9b1d6" font-size="12" font-weight="bold">db-west-04</text>
|
||||
<text x="566" y="23" fill="#a9b1d6" font-size="11">3</text>
|
||||
<text x="616" y="23" fill="#a9b1d6" font-size="11">1</text>
|
||||
<text x="666" y="23" fill="#565f89" font-size="11">0</text>
|
||||
<text x="710" y="23" fill="#565f89" font-size="11">57.6k</text>
|
||||
<!-- Load bar: 3/10 = 30% -->
|
||||
<rect x="770" y="15" width="60" height="6" rx="3" fill="#292e42"/>
|
||||
<rect x="770" y="15" width="18" height="6" rx="3" fill="#9ece6a"/>
|
||||
<text x="838" y="23" fill="#565f89" font-size="11">30%</text>
|
||||
</g>
|
||||
|
||||
<!-- Node 2: api-east-01 — 3 ws, 1 attention, has-attention bar -->
|
||||
<g class="f6" opacity="0">
|
||||
<rect y="40" width="860" height="38" fill="#24283b"/>
|
||||
<rect y="40" width="3" height="38" fill="#e0af68"/>
|
||||
<circle cx="22" cy="59" r="4" fill="#9ece6a"/>
|
||||
<text x="36" y="63" fill="#a9b1d6" font-size="12" font-weight="bold">api-east-01</text>
|
||||
<text x="566" y="63" fill="#a9b1d6" font-size="11">3</text>
|
||||
<text x="616" y="63" fill="#565f89" font-size="11">0</text>
|
||||
<text x="666" y="63" fill="#a9b1d6" font-size="11">1</text>
|
||||
<text x="710" y="63" fill="#565f89" font-size="11">109k</text>
|
||||
<!-- Load bar: 3/10 = 30% -->
|
||||
<rect x="770" y="55" width="60" height="6" rx="3" fill="#292e42"/>
|
||||
<rect x="770" y="55" width="18" height="6" rx="3" fill="#9ece6a"/>
|
||||
<text x="838" y="63" fill="#565f89" font-size="11">30%</text>
|
||||
</g>
|
||||
|
||||
<!-- Node 3: sre-node-03 — 2 ws, 1 running, has-running bar -->
|
||||
<g class="f7" opacity="0">
|
||||
<rect y="80" width="860" height="38" fill="#1a1b26"/>
|
||||
<rect y="80" width="3" height="38" fill="#9ece6a"/>
|
||||
<circle cx="22" cy="99" r="4" fill="#9ece6a"/>
|
||||
<text x="36" y="103" fill="#a9b1d6" font-size="12" font-weight="bold">sre-node-03</text>
|
||||
<text x="566" y="103" fill="#a9b1d6" font-size="11">2</text>
|
||||
<text x="616" y="103" fill="#a9b1d6" font-size="11">1</text>
|
||||
<text x="666" y="103" fill="#565f89" font-size="11">0</text>
|
||||
<text x="710" y="103" fill="#565f89" font-size="11">64.4k</text>
|
||||
<!-- Load bar: 2/10 = 20% -->
|
||||
<rect x="770" y="95" width="60" height="6" rx="3" fill="#292e42"/>
|
||||
<rect x="770" y="95" width="12" height="6" rx="3" fill="#9ece6a"/>
|
||||
<text x="838" y="103" fill="#565f89" font-size="11">20%</text>
|
||||
</g>
|
||||
|
||||
<!-- Node 4: analytics-02 — 1 ws, thinking, has-thinking bar -->
|
||||
<g class="f8" opacity="0">
|
||||
<rect y="120" width="860" height="38" fill="#24283b"/>
|
||||
<rect y="120" width="3" height="38" fill="#7aa2f7"/>
|
||||
<circle cx="22" cy="139" r="4" fill="#9ece6a"/>
|
||||
<text x="36" y="143" fill="#a9b1d6" font-size="12" font-weight="bold">analytics-02</text>
|
||||
<text x="566" y="143" fill="#a9b1d6" font-size="11">1</text>
|
||||
<text x="616" y="143" fill="#565f89" font-size="11">0</text>
|
||||
<text x="666" y="143" fill="#565f89" font-size="11">0</text>
|
||||
<text x="710" y="143" fill="#565f89" font-size="11">18.3k</text>
|
||||
<!-- Load bar: 1/10 = 10% -->
|
||||
<rect x="770" y="135" width="60" height="6" rx="3" fill="#292e42"/>
|
||||
<rect x="770" y="135" width="6" height="6" rx="3" fill="#9ece6a"/>
|
||||
<text x="838" y="143" fill="#565f89" font-size="11">10%</text>
|
||||
</g>
|
||||
|
||||
<!-- Node 5: data-ops-05 — 1 ws, thinking, has-thinking bar -->
|
||||
<g class="f9" opacity="0">
|
||||
<rect y="160" width="860" height="38" fill="#1a1b26"/>
|
||||
<rect y="160" width="3" height="38" fill="#7aa2f7"/>
|
||||
<circle cx="22" cy="179" r="4" fill="#9ece6a"/>
|
||||
<text x="36" y="183" fill="#a9b1d6" font-size="12" font-weight="bold">data-ops-05</text>
|
||||
<text x="566" y="183" fill="#a9b1d6" font-size="11">1</text>
|
||||
<text x="616" y="183" fill="#565f89" font-size="11">0</text>
|
||||
<text x="666" y="183" fill="#565f89" font-size="11">0</text>
|
||||
<text x="710" y="183" fill="#565f89" font-size="11">8.7k</text>
|
||||
<!-- Load bar: 1/10 = 10% -->
|
||||
<rect x="770" y="175" width="60" height="6" rx="3" fill="#292e42"/>
|
||||
<rect x="770" y="175" width="6" height="6" rx="3" fill="#9ece6a"/>
|
||||
<text x="838" y="183" fill="#565f89" font-size="11">10%</text>
|
||||
</g>
|
||||
|
||||
<!-- Node 6: ml-gpu-07 — 0 ws, empty, no bar -->
|
||||
<g class="f10" opacity="0">
|
||||
<rect y="200" width="860" height="38" fill="#24283b"/>
|
||||
<rect y="200" width="3" height="38" fill="transparent"/>
|
||||
<circle cx="22" cy="219" r="4" fill="#9ece6a"/>
|
||||
<text x="36" y="223" fill="#a9b1d6" font-size="12" font-weight="bold">ml-gpu-07</text>
|
||||
<text x="566" y="223" fill="#565f89" font-size="11">0</text>
|
||||
<text x="616" y="223" fill="#565f89" font-size="11">0</text>
|
||||
<text x="666" y="223" fill="#565f89" font-size="11">0</text>
|
||||
<text x="710" y="223" fill="#565f89" font-size="11">0</text>
|
||||
<!-- Load bar: 0/10 = 0% (empty track) -->
|
||||
<rect x="770" y="215" width="60" height="6" rx="3" fill="#292e42"/>
|
||||
<text x="842" y="223" fill="#565f89" font-size="11">0%</text>
|
||||
</g>
|
||||
|
||||
</g>
|
||||
|
||||
<!-- ====== Footer ====== -->
|
||||
<g transform="translate(0, 468)" class="f12" opacity="0">
|
||||
<rect width="860" height="1" fill="#3b4261"/>
|
||||
<rect y="1" width="860" height="24" fill="#16161e"/>
|
||||
|
||||
<circle cx="20" cy="14" r="3" fill="#9ece6a"/>
|
||||
<text x="28" y="18" fill="#565f89" font-size="10">db-west-04</text>
|
||||
|
||||
<circle cx="120" cy="14" r="3" fill="#9ece6a"/>
|
||||
<text x="128" y="18" fill="#565f89" font-size="10">api-east-01</text>
|
||||
|
||||
<circle cx="225" cy="14" r="3" fill="#9ece6a"/>
|
||||
<text x="233" y="18" fill="#565f89" font-size="10">sre-node-03</text>
|
||||
|
||||
<circle cx="335" cy="14" r="3" fill="#9ece6a"/>
|
||||
<text x="343" y="18" fill="#565f89" font-size="10">analytics-02</text>
|
||||
|
||||
<circle cx="450" cy="14" r="3" fill="#9ece6a"/>
|
||||
<text x="458" y="18" fill="#565f89" font-size="10">data-ops-05</text>
|
||||
|
||||
<circle cx="560" cy="14" r="3" fill="#9ece6a"/>
|
||||
<text x="568" y="18" fill="#565f89" font-size="10">ml-gpu-07</text>
|
||||
|
||||
<text x="680" y="18" fill="#3b4261" font-size="10">258k tokens · 42 calls · 12m</text>
|
||||
</g>
|
||||
|
||||
<!-- Bottom edge -->
|
||||
<rect y="493" width="860" height="27" fill="#16161e"/>
|
||||
<rect y="510" width="860" height="10" rx="10" fill="#16161e"/>
|
||||
</svg>
|
||||
|
Before Width: | Height: | Size: 11 KiB |
+135
-20
@@ -448,6 +448,14 @@ after `/clear` or `/new` commands).
|
||||
{"type": "clear_ui"}
|
||||
```
|
||||
|
||||
**`cancelled`** -- the generation was cancelled by the user (via the Stop
|
||||
button or `POST /v1/api/cancel`). The client should finalize any in-progress
|
||||
assistant message with whatever partial content was streamed.
|
||||
|
||||
```json
|
||||
{"type": "cancelled"}
|
||||
```
|
||||
|
||||
#### Keepalive
|
||||
|
||||
The server sends an SSE comment every 5 seconds when no events are pending:
|
||||
@@ -460,13 +468,13 @@ The server sends an SSE comment every 5 seconds when no events are pending:
|
||||
This prevents proxies and browsers from closing the connection due to
|
||||
inactivity.
|
||||
|
||||
#### Generation mechanism
|
||||
#### Multi-consumer fan-out
|
||||
|
||||
Each new SSE connection to a workstream increments an internal
|
||||
`_sse_generation` counter. The previous SSE handler detects the generation
|
||||
mismatch and exits its event loop, ensuring only one active SSE connection per
|
||||
workstream at a time. The event queue is drained of stale events before the new
|
||||
connection begins streaming.
|
||||
Each SSE connection to a workstream receives its own delivery queue. Events
|
||||
produced by the worker thread are fanned out to all registered listener queues,
|
||||
so multiple consumers (browser, bridge, console proxy, SDK) can connect
|
||||
simultaneously and each receives every event. On reconnect the client receives
|
||||
a full history replay, so no catch-up mechanism is needed.
|
||||
|
||||
---
|
||||
|
||||
@@ -515,8 +523,8 @@ Returns a list of all active workstreams.
|
||||
```json
|
||||
{
|
||||
"workstreams": [
|
||||
{"id": "abc123", "name": "default", "state": "idle", "session_id": "a1b2c3d4e5f6"},
|
||||
{"id": "def456", "name": "hacker-news", "state": "thinking", "session_id": "c5d6e7f8a9b0"}
|
||||
{"id": "abc123", "name": "default", "state": "idle"},
|
||||
{"id": "def456", "name": "hacker-news", "state": "thinking"}
|
||||
]
|
||||
}
|
||||
```
|
||||
@@ -528,22 +536,21 @@ Each workstream object:
|
||||
| `id` | string | Unique workstream routing identifier |
|
||||
| `name` | string | Display name (alias if set, otherwise `ws-xxxx`) |
|
||||
| `state` | string | Current state (see state values above) |
|
||||
| `session_id` | string/null | Session ID of the workstream's `ChatSession`, used for deduplication against `/v1/api/sessions` |
|
||||
|
||||
---
|
||||
|
||||
### `GET /v1/api/sessions`
|
||||
### `GET /v1/api/workstreams/saved`
|
||||
|
||||
Returns a list of saved sessions from the database, ordered by most recently
|
||||
Returns a list of saved workstreams from the database, ordered by most recently
|
||||
updated.
|
||||
|
||||
**Response:**
|
||||
|
||||
```json
|
||||
{
|
||||
"sessions": [
|
||||
"workstreams": [
|
||||
{
|
||||
"session_id": "a1b2c3d4e5f6",
|
||||
"ws_id": "a1b2c3d4e5f6",
|
||||
"alias": "refactor",
|
||||
"title": "JWT Authentication Refactor",
|
||||
"created": "2026-03-01 10:00:00",
|
||||
@@ -554,18 +561,16 @@ updated.
|
||||
}
|
||||
```
|
||||
|
||||
Each session object:
|
||||
Each saved workstream object:
|
||||
|
||||
| Field | Type | Description |
|
||||
|-----------------|-------------|--------------------------------------------|
|
||||
| `session_id` | string | Unique 32-char hex UUID session identifier |
|
||||
| `ws_id` | string | Unique workstream identifier |
|
||||
| `alias` | string/null | User-assigned short name |
|
||||
| `title` | string/null | LLM-generated title |
|
||||
| `created` | string | ISO timestamp of session creation |
|
||||
| `created` | string | ISO timestamp of workstream creation |
|
||||
| `updated` | string | ISO timestamp of last message |
|
||||
| `message_count` | int | Number of messages in the session |
|
||||
| `node_id` | string/null | Server node that created the session |
|
||||
| `ws_id` | string/null | Workstream the session belongs to |
|
||||
| `message_count` | int | Number of messages in the workstream |
|
||||
|
||||
---
|
||||
|
||||
@@ -704,6 +709,43 @@ containing the resumed session's messages.
|
||||
|
||||
---
|
||||
|
||||
### `POST /v1/api/cancel`
|
||||
|
||||
Cancels the active generation in a workstream. Sets a cooperative cancellation
|
||||
flag that is checked at multiple points in the generation loop (per streaming
|
||||
chunk, before tool execution, inside bash commands). The session transitions to
|
||||
`idle` state and preserves any partial content already streamed.
|
||||
|
||||
If the workstream is waiting for tool approval or plan review, the pending
|
||||
prompt is automatically denied/rejected to unblock the worker thread.
|
||||
|
||||
Calling this endpoint when the workstream is already idle is a harmless no-op.
|
||||
|
||||
**Request body:**
|
||||
|
||||
```json
|
||||
{"ws_id": "abc123"}
|
||||
```
|
||||
|
||||
| Field | Type | Required | Description |
|
||||
|--------|--------|----------|----------------------|
|
||||
| `ws_id`| string | yes | Target workstream ID |
|
||||
|
||||
**Response:**
|
||||
|
||||
```json
|
||||
{"status": "ok"}
|
||||
```
|
||||
|
||||
**Error responses:**
|
||||
|
||||
| Status | Body | Condition |
|
||||
|--------|------------------------------------|------------------------|
|
||||
| 400 | `{"error": "No session"}` | Session not initialized|
|
||||
| 404 | `{"error": "Unknown workstream"}` | `ws_id` not found |
|
||||
|
||||
---
|
||||
|
||||
### `POST /v1/api/workstreams/new`
|
||||
|
||||
Creates a new workstream. The server supports up to 10 concurrent workstreams.
|
||||
@@ -721,7 +763,7 @@ All fields are optional. The body can be empty or an empty JSON object.
|
||||
| `name` | string | auto | Workstream display name |
|
||||
| `model` | string | default | Model alias from the registry (`[models.*]`) |
|
||||
| `auto_approve` | bool | false | Auto-approve all tool calls for this workstream |
|
||||
| `resume_session` | string | "" | Session ID to resume atomically during creation (empty = fresh)|
|
||||
| `resume_ws` | string | "" | Workstream ID to resume atomically during creation (empty = fresh)|
|
||||
|
||||
**Response (success):**
|
||||
|
||||
@@ -777,6 +819,79 @@ Status code: `400`
|
||||
|
||||
---
|
||||
|
||||
### `GET /v1/api/watches`
|
||||
|
||||
List active watches on this server node. Optionally filter by workstream.
|
||||
Requires `write` scope.
|
||||
|
||||
**Query parameters:**
|
||||
|
||||
| Parameter | Type | Required | Description |
|
||||
|-----------|--------|----------|------------------------------------|
|
||||
| `ws_id` | string | no | Filter to watches for this workstream. If omitted, returns all watches on the node. |
|
||||
|
||||
**Response:**
|
||||
|
||||
```json
|
||||
{
|
||||
"watches": [
|
||||
{
|
||||
"watch_id": "abc123def456...",
|
||||
"ws_id": "ws-1",
|
||||
"node_id": "host_a1b2",
|
||||
"name": "pr-review",
|
||||
"command": "gh pr view --json state",
|
||||
"interval_secs": 300.0,
|
||||
"stop_on": "data[\"state\"] == \"MERGED\"",
|
||||
"max_polls": 100,
|
||||
"poll_count": 5,
|
||||
"last_output": "{\"state\": \"OPEN\"}",
|
||||
"last_poll": "2026-03-09T12:00:00",
|
||||
"next_poll": "2026-03-09T12:05:00",
|
||||
"active": 1,
|
||||
"created": "2026-03-09T11:30:00"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### `POST /v1/api/watches/{watch_id}/cancel`
|
||||
|
||||
Cancel an active watch. Sets `active=0` and clears `next_poll`.
|
||||
Requires `write` scope. Verifies node ownership in multi-node deployments.
|
||||
|
||||
**Path parameters:**
|
||||
|
||||
| Parameter | Type | Description |
|
||||
|------------|--------|-----------------|
|
||||
| `watch_id` | string | Watch ID to cancel |
|
||||
|
||||
**Response (success):**
|
||||
|
||||
```json
|
||||
{"status": "ok", "watch_id": "abc123def456..."}
|
||||
```
|
||||
|
||||
**Error (not found):**
|
||||
|
||||
```json
|
||||
{"error": "Watch not found"}
|
||||
```
|
||||
|
||||
Status code: `404`
|
||||
|
||||
**Error (wrong node):**
|
||||
|
||||
```json
|
||||
{"error": "Watch belongs to another node"}
|
||||
```
|
||||
|
||||
Status code: `403`
|
||||
|
||||
---
|
||||
|
||||
### `OPTIONS` (any path)
|
||||
|
||||
Handles CORS preflight requests.
|
||||
|
||||
+145
-70
@@ -42,7 +42,9 @@ turnstone/
|
||||
__init__.py create_provider() + create_client() factory functions
|
||||
workstream.py Parallel workstream manager (WorkstreamState, Workstream, WorkstreamManager)
|
||||
tools.py Tool schema loader (JSON -> OpenAI function-calling format)
|
||||
mcp_client.py MCPClientManager — MCP server connections, tool discovery, async-sync bridge
|
||||
mcp_client.py MCPClientManager — MCP server connections, tool discovery, dynamic refresh, async-sync bridge
|
||||
tool_search.py Dynamic tool search — BM25 index, session-scoped tool visibility
|
||||
watch.py WatchRunner daemon — periodic command polling, condition DSL, result dispatch
|
||||
model_registry.py ModelRegistry — named model configs, lazy client creation, fallback routing
|
||||
memory.py Persistence facade (delegates to storage backend)
|
||||
storage/ Pluggable storage: StorageBackend protocol, SQLite + PostgreSQL
|
||||
@@ -94,7 +96,7 @@ turnstone/
|
||||
style.css Page-specific UI styles (dashboard layout, approval blocks)
|
||||
app.js Page-specific client-side JavaScript (SSE, workstreams, markdown)
|
||||
tools/
|
||||
*.json 14 tool schemas (OpenAI function-calling format + turnstone metadata)
|
||||
*.json 15 tool schemas (OpenAI function-calling format + turnstone metadata)
|
||||
```
|
||||
|
||||
Both UIs share a common design system extracted into `turnstone/shared_static/`: design tokens, login overlay, toast notifications, theme toggle, keyboard shortcuts, and utility functions. Each UI imports `base.css` and the shared JS modules at `/shared/`, then adds only page-specific code at `/static/`.
|
||||
@@ -127,6 +129,7 @@ A user message flows through the system as follows:
|
||||
| on_reasoning_token() / on_content_token()
|
||||
| accumulate tool_calls from deltas
|
||||
| track finish_reason
|
||||
| _check_cancelled() per chunk (cooperative cancel)
|
||||
v
|
||||
finish_reason check:
|
||||
+--- "length" --> warn, discard partial tool_calls
|
||||
@@ -172,11 +175,13 @@ Phase 2: APPROVE (serial, blocking)
|
||||
_emit_state("running")
|
||||
|
||||
Phase 3: EXECUTE (parallel)
|
||||
_check_cancelled() <-- cancellation checkpoint before execution starts
|
||||
if len(items) == 1:
|
||||
run_one(items[0])
|
||||
else:
|
||||
ThreadPoolExecutor(max_workers=4).map(run_one, items)
|
||||
Bash tool streams stdout line-by-line via ui.on_tool_output_chunk(call_id, line)
|
||||
(cancel_event also checked per line — kills process group on cancel)
|
||||
Final output (stdout + stderr) delivered via ui.on_tool_result(call_id, name, output)
|
||||
call_id links tool_info items → streaming chunks → final result
|
||||
For plan tool: post-execution gate via ui.on_plan_review()
|
||||
@@ -207,6 +212,11 @@ The engine emits state changes via `_emit_state()` which calls
|
||||
"idle" ---> no more tool calls, turn complete
|
||||
|
|
||||
(or "error" ---> exception or KeyboardInterrupt)
|
||||
|
||||
cancel() may be called from any state. It sets a cooperative flag
|
||||
checked at each streaming chunk, before tool execution, and inside
|
||||
bash commands. The session transitions to "idle" with partial
|
||||
content preserved, emitting on_info("[Generation cancelled]").
|
||||
```
|
||||
|
||||
---
|
||||
@@ -472,7 +482,7 @@ independently, then returns the final content as the tool result.
|
||||
|
||||
- **task**: uses `self._task_tools` (`TASK_AGENT_TOOLS` + MCP tools)
|
||||
- **plan**: uses `self._agent_tools` (`AGENT_TOOLS` + MCP tools). Writes output
|
||||
to `.plan-<session_id>.md` — unique per `ChatSession` so concurrent workstreams
|
||||
to `.plan-<ws_id>.md` — unique per `ChatSession` so concurrent workstreams
|
||||
don't collide. On repeat invocations the prior `plan` tool call and its result
|
||||
are forwarded from `self.messages` so the agent refines the existing plan rather
|
||||
than starting over. Planning instructions are injected as a developer message
|
||||
@@ -497,17 +507,32 @@ bridges this with a background asyncio event loop in a daemon thread.
|
||||
1. `create_mcp_client()` reads server configs from TOML or JSON
|
||||
2. `MCPClientManager.start()` launches the background event loop thread
|
||||
3. `_connect_all()` connects to each server (stdio subprocess or HTTP), runs
|
||||
`initialize()` + `list_tools()`, converts schemas to OpenAI format
|
||||
4. `ChatSession.__init__` receives the manager and builds `self._tools` (built-in + MCP)
|
||||
`initialize()` + `list_tools()`, converts schemas to OpenAI format, detects
|
||||
`tools.listChanged` capability for push notification support
|
||||
4. `ChatSession.__init__` receives the manager, builds `self._tools` (built-in + MCP),
|
||||
and registers a listener callback for tool-change notifications
|
||||
5. `_prepare_tool()` routes MCP tools to `_prepare_mcp_tool()` / `_exec_mcp_tool()`
|
||||
6. `_exec_mcp_tool()` calls `call_tool_sync()` which dispatches to the async loop
|
||||
via `asyncio.run_coroutine_threadsafe()`
|
||||
|
||||
**Tool refresh:** Three mechanisms keep tools up-to-date without restart:
|
||||
- **Push:** Servers declaring `tools.listChanged` send `ToolListChangedNotification`;
|
||||
the registered `message_handler` triggers immediate single-server refresh.
|
||||
- **Periodic:** Servers without push support are polled on a staggered interval
|
||||
(default 4 h, configurable via `[mcp] refresh_interval` or `--mcp-refresh-interval`).
|
||||
- **Manual:** `/mcp refresh [server]` calls `refresh_sync()` for on-demand refresh
|
||||
(also attempts reconnection for disconnected servers).
|
||||
|
||||
When tools change, `_rebuild_tools()` creates new `_tools`/`_tool_map` objects
|
||||
(copy-on-write for thread safety) and notifies listener callbacks. Each `ChatSession`
|
||||
rebuilds its merged tool lists and reconstructs `ToolSearchManager` (preserving
|
||||
expanded tools).
|
||||
|
||||
**Tool naming:** `mcp__{server}__{tool}` — double underscore delimiter, validated
|
||||
at connection time (server names with `__` are rejected).
|
||||
|
||||
**Error isolation:** Per-server connection failures are caught and logged; other
|
||||
servers still connect. Tool execution errors return error strings to the LLM
|
||||
**Error isolation:** Per-server connection/refresh failures are caught and logged; other
|
||||
servers are unaffected. Tool execution errors return error strings to the LLM
|
||||
rather than crashing the session.
|
||||
|
||||
### Provider Adapter Layer
|
||||
@@ -544,21 +569,23 @@ LLMProvider (protocol)
|
||||
|------|--------|
|
||||
| `StreamChunk` | `content_delta`, `reasoning_delta`, `tool_call_deltas`, `info_delta`, `usage`, `finish_reason` |
|
||||
| `CompletionResult` | `content`, `tool_calls`, `finish_reason`, `usage` |
|
||||
| `ModelCapabilities` | `context_window`, `max_output_tokens`, `supports_temperature`, `token_param`, `thinking_mode`, `supports_effort`, `supports_web_search` |
|
||||
| `ModelCapabilities` | `context_window`, `max_output_tokens`, `supports_temperature`, `token_param`, `thinking_mode`, `supports_effort`, `supports_web_search`, `supports_tool_search`, `supports_vision` |
|
||||
| `UsageInfo` | `prompt_tokens`, `completion_tokens`, `total_tokens` |
|
||||
|
||||
**OpenAIProvider** (`_openai.py`): passes messages through unchanged (they are
|
||||
already in OpenAI format). Model capability lookup table covers
|
||||
GPT-5/5.1/5.2, O-series, and search models (`gpt-5-search-api`).
|
||||
already in OpenAI format), including multi-part content blocks (text + images)
|
||||
in tool results. Model capability lookup table covers GPT-5/5.1/5.2/5.3/5.4,
|
||||
O-series, and search models (`gpt-5-search-api`) — all with `supports_vision`.
|
||||
For search models, injects `web_search_options` and removes the `web_search`
|
||||
function tool (the model always searches). Citations from `url_citation`
|
||||
annotations are formatted as footnotes. Unknown models (local servers) get
|
||||
permissive defaults and use Tavily for web search.
|
||||
permissive defaults with `supports_vision=False` and use Tavily for web search.
|
||||
|
||||
**AnthropicProvider** (`_anthropic.py`): converts OpenAI-format messages to
|
||||
Anthropic content blocks, maps `system`/`developer` roles to the `system`
|
||||
parameter, groups consecutive `tool` result messages into user-role content
|
||||
blocks, and translates tool schemas from OpenAI function-calling format to
|
||||
blocks (converting `image_url` parts to Anthropic's `image` source format),
|
||||
and translates tool schemas from OpenAI function-calling format to
|
||||
Anthropic's `input_schema` format. Supports both manual and adaptive thinking
|
||||
modes, with effort parameter support for models like Claude Opus 4.6 and
|
||||
Sonnet 4.6. Replaces the `web_search` function tool with Anthropic's native
|
||||
@@ -604,6 +631,18 @@ agent_model = "claude"
|
||||
|
||||
Each `[models.*]` entry produces a `ModelConfig` with a `provider` field
|
||||
(default: `"openai"`). Supported values: `"openai"` and `"anthropic"`.
|
||||
An optional `[models.*.capabilities]` sub-table overrides per-model
|
||||
`ModelCapabilities` flags (useful for local models whose capabilities
|
||||
cannot be detected programmatically):
|
||||
|
||||
```toml
|
||||
[models.qwen-vl]
|
||||
base_url = "http://localhost:8000/v1"
|
||||
model = "qwen-3.5-vl"
|
||||
|
||||
[models.qwen-vl.capabilities]
|
||||
supports_vision = true
|
||||
```
|
||||
|
||||
**Lifecycle:**
|
||||
1. `load_model_registry()` reads `[models.*]` sections from config.toml and
|
||||
@@ -688,8 +727,11 @@ memories
|
||||
created TEXT NOT NULL
|
||||
updated TEXT NOT NULL
|
||||
|
||||
sessions
|
||||
session_id TEXT PRIMARY KEY
|
||||
workstreams
|
||||
ws_id TEXT PRIMARY KEY
|
||||
node_id TEXT NOT NULL
|
||||
name TEXT NOT NULL
|
||||
state TEXT NOT NULL DEFAULT 'idle'
|
||||
alias TEXT UNIQUE -- user-assigned short name (nullable)
|
||||
title TEXT -- LLM-generated title (nullable)
|
||||
created TEXT NOT NULL
|
||||
@@ -697,7 +739,7 @@ sessions
|
||||
|
||||
conversations
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT
|
||||
session_id TEXT NOT NULL
|
||||
ws_id TEXT NOT NULL
|
||||
timestamp TEXT NOT NULL
|
||||
role TEXT NOT NULL -- user | assistant | tool_call | tool_result
|
||||
content TEXT
|
||||
@@ -706,8 +748,8 @@ conversations
|
||||
tool_call_id TEXT -- links tool_call ↔ tool_result for resume
|
||||
provider_data TEXT -- raw provider content (e.g. Anthropic encrypted)
|
||||
|
||||
session_config
|
||||
session_id TEXT NOT NULL -- composite PK with key
|
||||
workstream_config
|
||||
ws_id TEXT NOT NULL -- composite PK with key
|
||||
key TEXT NOT NULL
|
||||
value TEXT
|
||||
|
||||
@@ -722,22 +764,20 @@ and are the single source of truth for both backends and Alembic migrations.
|
||||
|
||||
| Method | Purpose |
|
||||
|--------|---------|
|
||||
| `register_session(session_id, title, node_id, ws_id)` | Create a sessions row (no-op if exists) |
|
||||
| `save_message(session_id, role, content, ...)` | Log a message to conversations |
|
||||
| `load_session_messages(session_id)` | Reconstruct OpenAI message format from DB rows |
|
||||
| `list_sessions(limit)` | List sessions with >=1 message, ordered by updated DESC |
|
||||
| `delete_session(session_id)` | Delete session and all its messages |
|
||||
| `prune_sessions(retention_days)` | Remove empty sessions and old unnamed sessions |
|
||||
| `resolve_session(alias_or_id)` | Resolve alias, exact id, or id prefix to full session_id |
|
||||
| `save_session_config(session_id, config)` | Persist session configuration key/value pairs |
|
||||
| `load_session_config(session_id)` | Retrieve session configuration |
|
||||
| `set_session_alias(session_id, alias)` | Set user-friendly alias (returns False if taken) |
|
||||
| `get_session_name(session_id)` | Return alias if set, else title, else None |
|
||||
| `update_session_title(session_id, title)` | Set/update LLM-generated title |
|
||||
| `register_workstream(ws_id, node_id, name, state)` | Create a workstreams row (no-op if exists) |
|
||||
| `save_message(ws_id, role, content, ...)` | Log a message to conversations |
|
||||
| `load_messages(ws_id)` | Reconstruct OpenAI message format from DB rows |
|
||||
| `list_workstreams_with_history(limit)` | List workstreams with >=1 message, ordered by updated DESC |
|
||||
| `delete_workstream(ws_id)` | Delete workstream and cascade conversations + config |
|
||||
| `prune_workstreams(retention_days)` | Remove empty workstreams and old unnamed workstreams |
|
||||
| `resolve_workstream(alias_or_id)` | Resolve alias, exact id, or id prefix to full ws_id |
|
||||
| `save_workstream_config(ws_id, config)` | Persist workstream configuration key/value pairs |
|
||||
| `load_workstream_config(ws_id)` | Retrieve workstream configuration |
|
||||
| `set_workstream_alias(ws_id, alias)` | Set user-friendly alias (returns False if taken) |
|
||||
| `get_workstream_display_name(ws_id)` | Return alias if set, else title, else None |
|
||||
| `update_workstream_title(ws_id, title)` | Set/update LLM-generated title |
|
||||
| `update_workstream_state(ws_id, state)` | Update workstream state and bump timestamp |
|
||||
| `update_workstream_name(ws_id, name)` | Update workstream display name |
|
||||
| `delete_workstream(ws_id)` | Delete a workstream row |
|
||||
| `list_workstreams(node_id, limit)` | List workstreams, optionally by node |
|
||||
| `kv_get(key)` / `kv_set(key, value)` / `kv_delete(key)` | Generic key-value store (backs memories table) |
|
||||
| `kv_list()` / `kv_search(query)` | List or search key-value pairs |
|
||||
@@ -757,59 +797,59 @@ pool_size = 5 # PostgreSQL connection pool size
|
||||
|
||||
Environment variables: `TURNSTONE_DB_BACKEND`, `TURNSTONE_DB_URL`, `TURNSTONE_DB_PATH`.
|
||||
|
||||
### Session Persistence and Resume
|
||||
### Persistence and Resume
|
||||
|
||||
Each `ChatSession` generates a full 32-char hex UUID `_session_id` on creation
|
||||
and registers it in the `sessions` table with the server's `node_id` and the
|
||||
owning `ws_id`. Messages are saved to `conversations` as they happen via
|
||||
`save_message()`. Workstreams are persisted to the `workstreams` table on
|
||||
creation, with state changes tracked via `update_workstream_state()`.
|
||||
`ws_id` is the sole persistent identity for both routing and conversation
|
||||
history. There is no separate `session_id` — the `workstreams` table holds
|
||||
alias, title, and state alongside the routing fields (`node_id`, `name`).
|
||||
Messages are saved to `conversations` (keyed by `ws_id`) as they happen
|
||||
via `save_message()`. Workstream state changes are tracked via
|
||||
`update_workstream_state()`.
|
||||
|
||||
**Auto-titling:** After the first complete exchange (user message + assistant
|
||||
response), a background thread calls the LLM with a title-generation prompt
|
||||
(`reasoning_effort: "low"`, `max_completion_tokens: 200`). The generated
|
||||
title (3-8 words) is stored in `sessions.title`.
|
||||
title (3-8 words) is stored in `workstreams.title`.
|
||||
|
||||
**Resume flow:** `ChatSession.resume_session(session_id)` calls
|
||||
`load_session_messages()` which reconstructs the OpenAI message format from
|
||||
database rows:
|
||||
**Resume flow:** `ChatSession.resume(ws_id)` calls `load_messages()` which
|
||||
reconstructs the OpenAI message format from database rows:
|
||||
|
||||
- `user` and `assistant` rows map directly
|
||||
- Consecutive `tool_call` rows are grouped into one assistant message's
|
||||
`tool_calls` array, paired with subsequent `tool_result` rows via
|
||||
`tool_call_id` (or positional matching for legacy data)
|
||||
- **Interrupted session repair:** If the last assistant message has
|
||||
`tool_calls` but fewer tool results than expected (session was
|
||||
- **Interrupted conversation repair:** If the last assistant message has
|
||||
`tool_calls` but fewer tool results than expected (conversation was
|
||||
interrupted mid-execution), the incomplete turn is stripped so the
|
||||
LLM can re-generate cleanly
|
||||
- The session adopts the old `_session_id`, so new messages continue in
|
||||
the same session
|
||||
- The `ChatSession` adopts the resumed `_ws_id`, so new messages continue
|
||||
in the same workstream
|
||||
|
||||
**Config persistence:** LLM-affecting parameters (`temperature`,
|
||||
`reasoning_effort`, `max_tokens`, `instructions`, `creative_mode`) are
|
||||
persisted to the `session_config` table on creation and whenever changed
|
||||
via slash commands. `resume_session()` restores these values so resumed
|
||||
sessions behave identically to the original.
|
||||
persisted to the `workstream_config` table on creation and whenever changed
|
||||
via slash commands. `resume()` restores these values so resumed workstreams
|
||||
behave identically to the original.
|
||||
|
||||
**`/clear` vs `/new`:** `/clear` wipes in-memory context but preserves
|
||||
messages in the database for future resume. `/new` starts a fresh session
|
||||
(new `_session_id`), leaving the old session resumable.
|
||||
messages in the database for future resume. `/new` starts a fresh workstream
|
||||
(new `_ws_id`), leaving the old workstream resumable.
|
||||
|
||||
**Resolution:** `resolve_session()` accepts aliases, exact session IDs, or
|
||||
session ID prefixes, enabling `turnstone --resume refactor` or `/resume abc12`.
|
||||
**Resolution:** `resolve_workstream()` accepts aliases, exact workstream IDs,
|
||||
or ID prefixes, enabling `turnstone --resume refactor` or `/resume abc12`.
|
||||
|
||||
**Session listing:** `list_sessions()` only returns sessions that have at
|
||||
least one saved message (`WHERE EXISTS` on `conversations`). Sessions
|
||||
registered but never used (e.g., from process startup) are invisible until
|
||||
a message is sent.
|
||||
**Workstream listing:** `list_workstreams_with_history()` only returns
|
||||
workstreams that have at least one saved message (`WHERE EXISTS` on
|
||||
`conversations`). Workstreams registered but never used (e.g., from process
|
||||
startup) are invisible until a message is sent.
|
||||
|
||||
**Session pruning:** `prune_sessions(retention_days, log_fn)` runs once at
|
||||
startup (CLI and server). It removes:
|
||||
- Sessions with no messages (orphaned registrations)
|
||||
- Unnamed sessions (`alias IS NULL`) older than `retention_days` days (default 90)
|
||||
**Workstream pruning:** `prune_workstreams(retention_days, log_fn)` runs once
|
||||
at startup (CLI and server). It removes:
|
||||
- Workstreams with no messages (orphaned registrations)
|
||||
- Unnamed workstreams (`alias IS NULL`) older than `retention_days` days (default 90)
|
||||
|
||||
Named (aliased) sessions are never age-pruned. Configure with
|
||||
`--session-retention-days N` (0 = disable age pruning).
|
||||
Named (aliased) workstreams are never age-pruned. Configure with
|
||||
`--retention-days N` (0 = disable age pruning).
|
||||
|
||||
---
|
||||
|
||||
@@ -947,7 +987,7 @@ Three hierarchical scopes control endpoint access:
|
||||
|
||||
| Scope | Grants | Endpoints |
|
||||
|-------|--------|-----------|
|
||||
| `read` | SSE streams, workstream listing, sessions | GET endpoints |
|
||||
| `read` | SSE streams, workstream listing, history | GET endpoints |
|
||||
| `write` | `read` + send, command, workstream create/close | POST to `/api/send`, `/api/command`, etc. |
|
||||
| `approve` | `write` + tool approval, admin operations | POST to `/api/approve`, `/api/admin/*` |
|
||||
|
||||
@@ -1081,7 +1121,7 @@ context manager handles startup/shutdown (health monitor, MCP client,
|
||||
registry).
|
||||
|
||||
Each workstream's `WebUI` has:
|
||||
- `_event_queue` (per-workstream SSE events, `queue.Queue`)
|
||||
- `_listeners` (per-client SSE queues, fan-out on `_enqueue()`)
|
||||
- `_approval_event` / `_plan_event` (`threading.Event` for blocking)
|
||||
- `_global_queue` (class variable, shared, for state broadcasts)
|
||||
|
||||
@@ -1141,6 +1181,10 @@ bridge auto-approves via `POST /v1/api/approve`. Otherwise, it publishes an
|
||||
`BLPOP` of a Redis response queue (`turnstone:resp:{request_id}`) until the client pushes
|
||||
a response or the approval timeout (default 3600s / 1 hour) expires.
|
||||
|
||||
**Cancellation:** The `CancelMessage` (type `"cancel"`) is a routed inbound message.
|
||||
The bridge dispatches it to `POST /v1/api/cancel` on the server owning the workstream,
|
||||
which sets the cooperative cancel flag and unblocks any pending approval/plan waits.
|
||||
|
||||
**Completion detection:** The bridge tracks which `correlation_id` maps to which
|
||||
`ws_id` for active sends. When the global SSE reports `ws_state → idle` for a tracked
|
||||
workstream, the bridge emits a synthetic `TurnCompleteEvent` with the correlation ID.
|
||||
@@ -1155,6 +1199,9 @@ for existing workstreams are auto-routed via `turnstone:ws:{ws_id}` ownership ke
|
||||
If a bridge picks up a shared-queue message for a workstream owned by another node, it
|
||||
re-routes to that node's queue (1 extra hop). Bridges publish heartbeats to
|
||||
`turnstone:node:{node_id}` with configurable TTL for node discovery.
|
||||
On startup, `_recover_workstreams` re-registers ownership of existing
|
||||
workstreams and publishes `WorkstreamCreatedEvent` to the cluster channel
|
||||
so the console collector picks them up immediately.
|
||||
|
||||
### Cluster Console
|
||||
|
||||
@@ -1180,7 +1227,10 @@ The console HTTP layer is a Starlette/ASGI app served by uvicorn. The SSE
|
||||
endpoint uses `EventSourceResponse` with the same listener queue pattern as
|
||||
the main server. `ClusterCollector`'s background threads (event subscriber,
|
||||
node discovery, poll loop) use sync Redis clients and `ThreadPoolExecutor`
|
||||
for parallel HTTP polling.
|
||||
for parallel HTTP polling. The poll loop diffs workstream IDs between poll
|
||||
cycles and fans out synthetic `ws_created`/`ws_closed` SSE events for any
|
||||
changes, ensuring browser clients stay in sync even when real-time cluster
|
||||
events are missed (e.g. bridge startup recovery).
|
||||
|
||||
The console has two write-path capabilities:
|
||||
|
||||
@@ -1239,7 +1289,7 @@ typed event dataclasses.
|
||||
|
||||
**Two client pairs** (sync + async):
|
||||
|
||||
- `TurnstoneServer` / `AsyncTurnstoneServer` — server API (workstreams, chat, streaming, sessions)
|
||||
- `TurnstoneServer` / `AsyncTurnstoneServer` — server API (workstreams, chat, streaming)
|
||||
- `TurnstoneConsole` / `AsyncTurnstoneConsole` — console API (cluster overview, nodes, workstreams)
|
||||
|
||||
**Design**: async-first with thin sync wrappers. `_BaseClient` provides httpx
|
||||
@@ -1279,11 +1329,11 @@ The `ChannelRouter` manages bidirectional routing: it maps platform
|
||||
channel/thread IDs to turnstone workstream IDs, handles workstream
|
||||
creation and stale-route recovery, and resolves platform users to
|
||||
turnstone identities via the `channel_users` table. When an evicted
|
||||
workstream is reactivated, the router uses atomic session resume via the
|
||||
`resume_session` field on `CreateWorkstreamMessage` — the server resumes
|
||||
the old session during workstream creation in a single HTTP request,
|
||||
eliminating ordering fragility. The bridge emits a `SessionResumedEvent`
|
||||
to confirm success.
|
||||
workstream is reactivated, the router uses atomic resume via the
|
||||
`resume_ws` field on `CreateWorkstreamMessage` — the server resumes
|
||||
the old workstream's conversation during creation in a single HTTP
|
||||
request, eliminating ordering fragility. The bridge emits a
|
||||
`WorkstreamResumedEvent` to confirm success.
|
||||
|
||||
Discord ships as the first adapter. See [channels.md](channels.md) for
|
||||
setup instructions, configuration reference, and the adapter development
|
||||
@@ -1302,3 +1352,28 @@ gateway validates the JWT, resolves the target (username lookup via
|
||||
the appropriate `ChannelAdapter.send()`. Delivery retries up to 3 times
|
||||
with backoff, re-querying the service registry on each attempt. See
|
||||
[Notification Flow diagram](diagrams/png/17-notify-flow.png).
|
||||
|
||||
---
|
||||
|
||||
## Governance
|
||||
|
||||
> See also: [Governance documentation](governance.md) | [Governance Architecture diagram](diagrams/19-governance-architecture.puml)
|
||||
|
||||
Turnstone governance extends the Phase 1 auth system with role-based access
|
||||
control (RBAC), tool execution policies, prompt templates, usage tracking,
|
||||
and audit logging. The permission model has two layers: legacy scopes
|
||||
(`read`, `write`, `approve`) checked by `AuthMiddleware`, and 15 granular
|
||||
permissions checked per-endpoint by `require_permission()`. Three built-in
|
||||
roles (admin, operator, viewer) are seeded by migration 008; custom roles
|
||||
can be created with any permission subset. JWTs carry both `scopes` and
|
||||
`permissions` claims for backward compatibility.
|
||||
|
||||
Tool policies use glob pattern matching (`fnmatch`) with priority-ordered
|
||||
first-match-wins evaluation to control tool execution (allow/deny/ask).
|
||||
Prompt templates provide reusable system messages with `{{variable}}`
|
||||
substitution. Usage events are recorded per-LLM-request for token
|
||||
accounting. An append-only audit log captures all admin mutations.
|
||||
|
||||
The console admin panel adds 5 governance tabs (Roles, Policies, Templates,
|
||||
Usage, Audit) for a total of 10 tabs, all permission-gated. Both Python
|
||||
and TypeScript SDKs expose governance methods on the console client.
|
||||
|
||||
+11
-13
@@ -136,11 +136,11 @@ An admin can also force-link or unlink users via the console admin panel
|
||||
1.5 seconds.
|
||||
- If the workstream is evicted for capacity, the next message in the
|
||||
thread auto-creates a new workstream and atomically resumes the
|
||||
previous session via the `resume_session` field on
|
||||
`CreateWorkstreamMessage`. The server resumes the session during
|
||||
workstream creation (same HTTP request), and the bridge emits a
|
||||
`SessionResumedEvent` back to the channel. The thread receives a
|
||||
*"Session resumed: {name} ({count} messages restored)"* confirmation.
|
||||
previous workstream via the `resume_ws` field on
|
||||
`CreateWorkstreamMessage`. The server resumes the workstream during
|
||||
creation (same HTTP request), and the bridge emits a
|
||||
`WorkstreamResumedEvent` back to the channel. The thread receives a
|
||||
*"Resumed: {name} ({count} messages restored)"* confirmation.
|
||||
|
||||
### Slash Commands
|
||||
|
||||
@@ -232,14 +232,12 @@ See [Security: Database Schema](security.md#database-schema) for the
|
||||
3. **Eviction** — the server evicts an idle workstream for capacity. The
|
||||
route is preserved and the thread stays open.
|
||||
4. **Reactivation** — the next message in the thread detects the stale
|
||||
route (no MQ owner), looks up the old session via
|
||||
`get_session_id_by_ws()`, and creates a new workstream with
|
||||
`resume_session` set atomically on the `CreateWorkstreamMessage`. The
|
||||
server resumes the session during creation (no separate command
|
||||
needed). The bridge emits a `SessionResumedEvent` to the channel, and
|
||||
the thread displays *"Session resumed: {name} ({count} messages
|
||||
restored)"*. If the old session was pruned, the workstream starts
|
||||
fresh with no error.
|
||||
route (no MQ owner) and creates a new workstream with the old `ws_id`
|
||||
as `resume_ws` on the `CreateWorkstreamMessage`. The server resumes
|
||||
the workstream during creation (no separate command or reverse lookup
|
||||
needed). The bridge emits a `WorkstreamResumedEvent` to the channel, and
|
||||
the thread displays *"Resumed: {name} ({count} messages restored)"*.
|
||||
If the old workstream was pruned, a fresh one starts with no error.
|
||||
5. **Close** — `/close` command closes the workstream via MQ, deletes the
|
||||
route, unsubscribes from events, and archives the Discord thread.
|
||||
|
||||
|
||||
+38
-2
@@ -61,6 +61,8 @@ The collector (`turnstone/console/collector.py`) maintains an in-memory snapshot
|
||||
|
||||
3. **Poll loop** — fetches `GET /v1/api/dashboard` and `GET /health` from each known node every 10 seconds. Uses `ThreadPoolExecutor(max_workers=50)` for parallelism. Each poll replaces the node's workstream list with the authoritative server data.
|
||||
|
||||
A `get_snapshot()` method builds the full cluster state under a single lock acquisition — overview aggregates and per-node workstream lists in one atomic read. This is served both as a REST endpoint and as the initial SSE event on client connect.
|
||||
|
||||
### Thread Safety
|
||||
|
||||
All reads and writes to the node/workstream map are protected by a single `threading.Lock`. Query methods acquire the lock, copy data, and release before returning.
|
||||
@@ -146,6 +148,38 @@ Single node detail with all its workstreams.
|
||||
}
|
||||
```
|
||||
|
||||
### `GET /v1/api/cluster/snapshot`
|
||||
|
||||
Full cluster state in a single response — all nodes with their workstreams plus overview aggregates. Built under a single lock for internal consistency. Used by the browser on initial load and SSE reconnect.
|
||||
|
||||
```json
|
||||
{
|
||||
"nodes": [
|
||||
{
|
||||
"node_id": "db-west-04",
|
||||
"server_url": "http://10.0.3.4:8080",
|
||||
"max_ws": 10,
|
||||
"reachable": true,
|
||||
"version": "0.3.0",
|
||||
"health": {"status": "ok", "version": "0.3.0"},
|
||||
"aggregate": {"total_tokens": 48200, "total_tool_calls": 156},
|
||||
"workstreams": [
|
||||
{"id": "a1b2c3d4", "name": "perf-db-west", "state": "running", ...}
|
||||
]
|
||||
}
|
||||
],
|
||||
"overview": {
|
||||
"nodes": 847,
|
||||
"workstreams": 4219,
|
||||
"states": {"running": 1847, "thinking": 312, "attention": 89, "idle": 1940, "error": 31},
|
||||
"aggregate": {"total_tokens": 12400000, "total_tool_calls": 34200},
|
||||
"version_drift": false,
|
||||
"versions": ["0.3.0"]
|
||||
},
|
||||
"timestamp": 1709294400.0
|
||||
}
|
||||
```
|
||||
|
||||
### `POST /v1/api/cluster/workstreams/new`
|
||||
|
||||
Create a new workstream on a target node. Dispatches a `CreateWorkstreamMessage` through the Redis MQ pipeline — the bridge on the target node picks it up and creates the workstream on the server. Requires `write` scope.
|
||||
@@ -182,7 +216,7 @@ Creation is asynchronous — the response confirms the MQ message was dispatched
|
||||
|
||||
### `GET /v1/api/cluster/events`
|
||||
|
||||
Server-Sent Events stream for real-time cluster updates.
|
||||
Server-Sent Events stream for real-time cluster updates. The first event is always a `snapshot` containing the full cluster state (same shape as `GET /v1/api/cluster/snapshot` with an added `type: "snapshot"` field), followed by incremental events:
|
||||
|
||||
```
|
||||
data: {"type":"cluster_state","ws_id":"a1b2","node_id":"db-west-04","state":"running"}
|
||||
@@ -326,7 +360,7 @@ The server UI uses root-relative URLs (`/v1/api/send`, `/static/app.js`, `/share
|
||||
|
||||
### SSE Proxy
|
||||
|
||||
SSE streams (`/v1/api/events`, `/v1/api/events/global`) are proxied by creating a per-connection `httpx.AsyncClient(timeout=None)`, streaming the upstream response via `aiter_text()`, parsing SSE framing (`\n\n` delimiters), and re-emitting events through `EventSourceResponse`. Each proxied SSE stream requires its own httpx client since the shared client's 30-second timeout would kill long-lived connections.
|
||||
SSE streams (`/v1/api/events`, `/v1/api/events/global`) are proxied as raw byte passthrough — the console opens an `httpx.AsyncClient.stream()` to the upstream server (with `read=None` and `pool=None` timeouts since SSE connections are long-lived) and relays every byte via `StreamingResponse`. This preserves server-side ping comments, event framing, and keepalives verbatim without parsing or re-encoding.
|
||||
|
||||
### Authentication
|
||||
|
||||
@@ -368,6 +402,8 @@ On submit, `POST /v1/api/cluster/workstreams/new` dispatches the creation reques
|
||||
|
||||
All five views receive live updates via SSE — state cards update counts, node rows update metrics, workstream rows update state indicators.
|
||||
|
||||
The browser maintains a local `clusterState` object that mirrors the cluster snapshot. It is initialized from the SSE `snapshot` event on connect (or via `GET /v1/api/cluster/snapshot` on initial page load) and updated incrementally by SSE events. View navigation reads from local state — no API round-trips needed after the initial snapshot.
|
||||
|
||||
### 5. Admin Panel
|
||||
|
||||
Accessed via the "admin" button in the header (visible when authenticated
|
||||
|
||||
@@ -40,7 +40,8 @@ package "turnstone/core/" <<Rectangle>> {
|
||||
component [auth.py\nAuthentication] as auth <<core>>
|
||||
component [healthcheck.py\nBackendHealthMonitor] as healthcheck <<core>>
|
||||
component [ratelimit.py\nRateLimiter] as ratelimit <<core>>
|
||||
component [mcp_client.py\nMCPClientManager] as mcp <<core>>
|
||||
component [mcp_client.py\nMCPClientManager\n(push + periodic refresh)] as mcp <<core>>
|
||||
component [tool_search.py\nToolSearchManager, BM25] as toolsearch <<core>>
|
||||
component [model_registry.py\nModelRegistry] as registry <<core>>
|
||||
}
|
||||
|
||||
@@ -95,7 +96,7 @@ package "turnstone/sdk/" <<Rectangle>> {
|
||||
|
||||
' Tool schemas
|
||||
package "turnstone/tools/" <<Rectangle>> {
|
||||
component [*.json\n14 tool schemas] as schemas <<artifact>>
|
||||
component [*.json\n15 tool schemas] as schemas <<artifact>>
|
||||
}
|
||||
|
||||
' Entry point dependencies
|
||||
@@ -136,6 +137,7 @@ session --> edit
|
||||
session --> web
|
||||
session --> healthcheck
|
||||
session --> mcp : optional
|
||||
session --> toolsearch : optional
|
||||
session --> registry : optional
|
||||
registry --> providers
|
||||
healthcheck --> metrics
|
||||
|
||||
@@ -41,7 +41,7 @@ class "WorkstreamTerminalUI" as WsTermUI {
|
||||
}
|
||||
|
||||
class "WebUI" as WebUI {
|
||||
- _event_queue: Queue
|
||||
- _listeners: list[Queue]
|
||||
- _approval_event: Event
|
||||
- _plan_event: Event
|
||||
- _ws_prompt_tokens: int
|
||||
@@ -108,6 +108,8 @@ class "ModelCapabilities" as ModelCaps <<frozen>> {
|
||||
+ thinking_mode: str
|
||||
+ supports_effort: bool
|
||||
+ supports_web_search: bool
|
||||
+ supports_tool_search: bool
|
||||
+ supports_vision: bool
|
||||
}
|
||||
|
||||
' ChatSession
|
||||
@@ -118,8 +120,9 @@ class "ChatSession" as ChatSession {
|
||||
- ui: SessionUI
|
||||
- messages: list[dict]
|
||||
- _msg_tokens: list[int]
|
||||
- _session_id: str
|
||||
- _ws_id: str
|
||||
- _mcp_client: MCPClientManager | None
|
||||
- _tool_search: ToolSearchManager | None
|
||||
- _registry: ModelRegistry | None
|
||||
+ model_alias: str | None {property}
|
||||
- _tools: list[dict]
|
||||
@@ -130,7 +133,7 @@ class "ChatSession" as ChatSession {
|
||||
--
|
||||
+ send(user_input: str)
|
||||
+ handle_command(command: str)
|
||||
+ resume_session(session_id: str)
|
||||
+ resume(ws_id: str)
|
||||
- _save_config()
|
||||
- _stream_response(stream) → dict
|
||||
- _create_stream_with_retry(msgs) → Stream (+ fallback)
|
||||
@@ -139,6 +142,12 @@ class "ChatSession" as ChatSession {
|
||||
- _prepare_tool(tc) → item dict
|
||||
- _prepare_mcp_tool(call_id, name, args) → item dict
|
||||
- _exec_mcp_tool(item) → (call_id, output)
|
||||
- _get_active_tools() → list[dict]
|
||||
- _prepare_tool_search() → None
|
||||
- _exec_tool_search(item) → (call_id, output)
|
||||
- _on_mcp_tools_changed()
|
||||
- _rebuild_tool_search()
|
||||
+ close()
|
||||
- _run_agent(messages, tools, ...) → str
|
||||
- _compact_messages(auto: bool)
|
||||
- _full_messages() → list[dict]
|
||||
@@ -201,22 +210,48 @@ enum "WorkstreamState" as WsState {
|
||||
' MCPClientManager
|
||||
class "MCPClientManager" as MCPMgr {
|
||||
- _sessions: dict[str, ClientSession]
|
||||
- _per_server_tools: dict[str, list[dict]]
|
||||
- _tools: list[dict]
|
||||
- _tool_map: dict[str, tuple]
|
||||
- _supports_list_changed: dict[str, bool]
|
||||
- _listeners: list[Callable]
|
||||
--
|
||||
+ start()
|
||||
+ get_tools() → list[dict]
|
||||
+ is_mcp_tool(name) → bool
|
||||
+ call_tool_sync(name, args) → str
|
||||
+ refresh_sync(server?) → dict
|
||||
+ add_listener(callback)
|
||||
+ remove_listener(callback)
|
||||
+ server_names: list[str] {property}
|
||||
+ shutdown()
|
||||
--
|
||||
Background asyncio event loop
|
||||
bridges async MCP SDK to
|
||||
sync ChatSession dispatch.
|
||||
Push + periodic + manual refresh.
|
||||
--
|
||||
core/mcp_client.py
|
||||
}
|
||||
|
||||
' ToolSearchManager
|
||||
class "ToolSearchManager" as ToolSearchMgr {
|
||||
- _all_tools: list[dict]
|
||||
- _always_on: list[dict]
|
||||
- _deferred: list[dict]
|
||||
- _expanded: dict[str, None]
|
||||
- _index: BM25Index
|
||||
--
|
||||
+ should_activate() → bool
|
||||
+ get_visible_tools() → list[dict]
|
||||
+ get_deferred_tools() → list[dict]
|
||||
+ get_expanded_names() → list[str]
|
||||
+ search(query, k) → list[dict]
|
||||
+ expand_visible(names) → list[dict]
|
||||
+ get_search_tool_definition() → dict
|
||||
+ format_search_results(tools) → str
|
||||
}
|
||||
|
||||
' ModelRegistry
|
||||
class "ModelRegistry" as ModelReg {
|
||||
- _models: dict[str, ModelConfig]
|
||||
@@ -317,6 +352,7 @@ LLMProvider <|.. AnthropicProv
|
||||
ChatSession --> SessionUI : uses
|
||||
ChatSession --> LLMProvider : delegates LLM calls
|
||||
ChatSession --> MCPMgr : optional
|
||||
ChatSession --o ToolSearchMgr : _tool_search
|
||||
ChatSession --> ModelReg : optional
|
||||
ChatSession <|-- HeadlessSession
|
||||
|
||||
|
||||
@@ -18,7 +18,7 @@ User -> CS : send(user_input)
|
||||
activate CS
|
||||
|
||||
CS -> CS : messages.append({role: "user", content: input})
|
||||
CS -> DB : save_message(session_id, "user", input)
|
||||
CS -> DB : save_message(ws_id, "user", input)
|
||||
|
||||
== LLM Call Loop ==
|
||||
|
||||
@@ -57,6 +57,14 @@ group loop [while tool_calls present]
|
||||
end
|
||||
end
|
||||
|
||||
note right of CS
|
||||
**Cancellation checkpoint:**
|
||||
_check_cancelled() runs per chunk.
|
||||
If cancel_event is set, raises
|
||||
GenerationCancelled — preserves
|
||||
partial content, emits idle state.
|
||||
end note
|
||||
|
||||
LLM --> CS : stream complete (usage stats)
|
||||
deactivate LLM
|
||||
|
||||
@@ -65,8 +73,8 @@ group loop [while tool_calls present]
|
||||
|
||||
CS -> CS : _update_token_table()\ncalibrate chars_per_token ratio
|
||||
CS -> CS : messages.append(assistant_msg)
|
||||
CS -> DB : save_message(session_id, "assistant", content)
|
||||
CS -> DB : save_message(session_id, "tool_call", ...) ×N
|
||||
CS -> DB : save_message(ws_id, "assistant", content)
|
||||
CS -> DB : save_message(ws_id, "tool_call", ...) ×N
|
||||
|
||||
== Tool Dispatch (if tool_calls) ==
|
||||
|
||||
@@ -112,7 +120,7 @@ group loop [while tool_calls present]
|
||||
note right of TP
|
||||
Parallel execution:
|
||||
bash → Popen + line-by-line streaming
|
||||
read_file → open().read()
|
||||
read_file → open().read() or base64 image
|
||||
search → grep subprocess
|
||||
edit_file → string replace
|
||||
task/plan → _run_agent() sub-loop
|
||||
@@ -136,7 +144,7 @@ group loop [while tool_calls present]
|
||||
|
||||
loop for each result
|
||||
CS -> CS : messages.append({role: "tool", ...})
|
||||
CS -> DB : save_message(session_id, "tool_result", ...)
|
||||
CS -> DB : save_message(ws_id, "tool_result", ...)
|
||||
end
|
||||
|
||||
opt user_feedback from approval
|
||||
@@ -144,6 +152,12 @@ group loop [while tool_calls present]
|
||||
end
|
||||
|
||||
note right of CS : Loop back for next LLM call
|
||||
|
||||
else GenerationCancelled
|
||||
CS -> CS : Preserve partial content\nor roll back incomplete tools
|
||||
CS -> UI : on_info("[Generation cancelled]")
|
||||
CS -> UI : on_state_change("idle")
|
||||
CS --> User : return (no re-raise)
|
||||
end
|
||||
|
||||
end
|
||||
|
||||
@@ -24,27 +24,29 @@ partition "Phase 1: Prepare" #E8F5E9 {
|
||||
:Dispatch to _prepare_{func_name}();
|
||||
|
||||
note right
|
||||
**Dispatch table (14 tools):**
|
||||
┌─────────────┬──────────────────┐
|
||||
│ Tool │ Needs Approval? │
|
||||
├─────────────┼──────────────────┤
|
||||
│ bash │ ✓ Yes │
|
||||
│ read_file │ ✗ Auto-approve │
|
||||
│ write_file │ ✓ Yes │
|
||||
│ edit_file │ ✓ Yes │
|
||||
│ search │ ✗ Auto-approve │
|
||||
│ math │ ✓ Yes │
|
||||
│ man │ ✗ Auto-approve │
|
||||
│ web_fetch │ ✓ Yes │
|
||||
│ web_search │ ✓ Yes │
|
||||
│ task │ ✓ Yes │
|
||||
│ plan │ ✓ Yes │
|
||||
│ remember │ ✗ Auto-approve │
|
||||
│ recall │ ✗ Auto-approve │
|
||||
│ forget │ ✗ Auto-approve │
|
||||
├─────────────┼──────────────────┤
|
||||
│ mcp__* │ ✓ Yes (external) │
|
||||
└─────────────┴──────────────────┘
|
||||
**Dispatch table (16 tools):**
|
||||
┌──────────────┬──────────────────┐
|
||||
│ Tool │ Needs Approval? │
|
||||
├──────────────┼──────────────────┤
|
||||
│ bash │ ✓ Yes │
|
||||
│ read_file │ ✗ Auto-approve │
|
||||
│ write_file │ ✓ Yes │
|
||||
│ edit_file │ ✓ Yes │
|
||||
│ search │ ✗ Auto-approve │
|
||||
│ math │ ✓ Yes │
|
||||
│ man │ ✗ Auto-approve │
|
||||
│ web_fetch │ ✓ Yes │
|
||||
│ web_search │ ✓ Yes │
|
||||
│ tool_search │ ✗ Auto-approve │
|
||||
│ task │ ✓ Yes │
|
||||
│ plan │ ✓ Yes │
|
||||
│ remember │ ✗ Auto-approve │
|
||||
│ recall │ ✗ Auto-approve │
|
||||
│ forget │ ✗ Auto-approve │
|
||||
│ notify │ ✗ Auto-approve │
|
||||
├──────────────┼──────────────────┤
|
||||
│ mcp__* │ ✓ Yes (external) │
|
||||
└──────────────┴──────────────────┘
|
||||
end note
|
||||
|
||||
:Build item dict:
|
||||
@@ -86,6 +88,8 @@ partition "Phase 2: Approve" #FFF3E0 {
|
||||
}
|
||||
|
||||
partition "Phase 3: Execute" #E3F2FD {
|
||||
:_check_cancelled();
|
||||
note right: Cancellation checkpoint:\nraises GenerationCancelled if\ncancel event is set
|
||||
if (single tool call?) then (yes)
|
||||
:Execute sequentially:\nrun_one(items[0]);
|
||||
else (multiple)
|
||||
@@ -98,7 +102,7 @@ partition "Phase 3: Execute" #E3F2FD {
|
||||
if item.denied → return denial message
|
||||
else → item["execute"](item)
|
||||
├─ _exec_bash: subprocess.run(["bash", script.sh])
|
||||
├─ _exec_read_file: open().readlines()
|
||||
├─ _exec_read_file: open().readlines() or _exec_read_image (base64)
|
||||
├─ _exec_write_file: makedirs + write
|
||||
├─ _exec_edit_file: find_occurrences + replace
|
||||
├─ _exec_search: grep subprocess
|
||||
@@ -106,8 +110,10 @@ partition "Phase 3: Execute" #E3F2FD {
|
||||
├─ _exec_man: man/info subprocess
|
||||
├─ _exec_web_fetch: httpx.get + LLM summary
|
||||
├─ _exec_web_search: Tavily API POST (fallback for local models)
|
||||
├─ _exec_tool_search: BM25 search + expand_visible()
|
||||
├─ _exec_task: _run_agent(TASK_AGENT_TOOLS)
|
||||
├─ _exec_plan: _run_agent(AGENT_TOOLS, read-only)
|
||||
├─ _exec_notify: HTTP POST to channel gateway
|
||||
├─ _exec_remember: SQLite INSERT OR REPLACE
|
||||
├─ _exec_recall: SQLite FTS5/LIKE search
|
||||
├─ _exec_forget: SQLite DELETE
|
||||
|
||||
@@ -79,6 +79,12 @@ package "Inbound Messages (Client → Bridge)" #FFF3E0 {
|
||||
type = "list_nodes"
|
||||
}
|
||||
|
||||
class CancelMessage {
|
||||
type = "cancel"
|
||||
--
|
||||
+ ws_id: str
|
||||
}
|
||||
|
||||
IM <|-- SendMessage
|
||||
IM <|-- ApproveMessage
|
||||
IM <|-- PlanFeedbackMessage
|
||||
@@ -88,6 +94,7 @@ package "Inbound Messages (Client → Bridge)" #FFF3E0 {
|
||||
IM <|-- ListWorkstreamsMessage
|
||||
IM <|-- HealthMessage
|
||||
IM <|-- ListNodesMessage
|
||||
IM <|-- CancelMessage
|
||||
}
|
||||
|
||||
package "Outbound Events (Bridge → Client)" #E3F2FD {
|
||||
|
||||
@@ -40,6 +40,12 @@ running --> error : Exception during\ntool execution
|
||||
|
||||
error --> thinking : New send() call\n_emit_state("thinking")
|
||||
|
||||
thinking --> idle : cancel() called\n_emit_state("idle")
|
||||
|
||||
running --> idle : cancel() called\n_emit_state("idle")
|
||||
|
||||
attention --> idle : cancel() unblocks\napproval/plan wait\n_emit_state("idle")
|
||||
|
||||
note right of thinking
|
||||
**Emitted via:**
|
||||
session._emit_state(state)
|
||||
|
||||
@@ -78,7 +78,19 @@ activate NodeA
|
||||
NodeA --> CC : {status:"ok", version:"0.3.0",\nmodel:"...", workstreams:{...}}
|
||||
deactivate NodeA
|
||||
|
||||
CC -> CC : Diff old vs new workstream IDs
|
||||
CC -> CC : Replace NodeSnapshot["nodeA"]\n.workstreams, .health, .aggregate
|
||||
CC -> CC : _fanout(ws_created) for\nnewly appeared workstreams
|
||||
CC -> CC : _fanout(ws_closed) for\nremoved workstreams
|
||||
|
||||
note right of CC
|
||||
Poll-diff fanout ensures
|
||||
browser SSE clients learn
|
||||
about workstreams that
|
||||
appeared without a real-time
|
||||
cluster event (e.g. bridge
|
||||
startup recovery).
|
||||
end note
|
||||
|
||||
CC -x NodeB : (SKIPPED: sim:// URL)
|
||||
|
||||
@@ -89,10 +101,15 @@ deactivate CC
|
||||
Browser -> Server : GET /v1/api/cluster/events
|
||||
activate Server
|
||||
|
||||
Server -> CC : get_snapshot()
|
||||
CC --> Server : ClusterSnapshot\n(full current state)
|
||||
|
||||
Server -> CC : register_listener(queue)
|
||||
note right : Per-client queue.Queue(maxsize=500)\nSSE via EventSourceResponse + run_in_executor()
|
||||
|
||||
loop continuous
|
||||
Server -> Browser : data: {"type":"snapshot",...}\n(full state as first SSE event)
|
||||
|
||||
loop continuous (incremental updates)
|
||||
CC -> Server : event via listener queue\n(from any of the 3 threads)
|
||||
Server -> Browser : data: {"type":"cluster_state",...}\n\n
|
||||
end
|
||||
@@ -105,6 +122,13 @@ Browser -> Server : connection closed
|
||||
Server -> CC : unregister_listener(queue)
|
||||
deactivate Server
|
||||
|
||||
== Browser REST: Snapshot ==
|
||||
|
||||
Browser -> Server : GET /v1/api/cluster/snapshot
|
||||
Server -> CC : get_snapshot()
|
||||
CC --> Server : ClusterSnapshot\n(full current state)
|
||||
Server --> Browser : JSON response
|
||||
|
||||
== Browser REST Requests ==
|
||||
|
||||
Browser -> Server : GET /v1/api/cluster/overview
|
||||
|
||||
@@ -32,10 +32,11 @@ package "turnstone/sdk/ (Python)" {
|
||||
+ approve()
|
||||
+ plan_feedback()
|
||||
+ command()
|
||||
+ cancel(ws_id)
|
||||
+ stream_events(ws_id)
|
||||
+ stream_global_events()
|
||||
+ send_and_wait()
|
||||
+ list_sessions()
|
||||
+ list_saved_workstreams()
|
||||
+ login() / logout()
|
||||
+ health()
|
||||
}
|
||||
@@ -45,6 +46,7 @@ package "turnstone/sdk/ (Python)" {
|
||||
+ nodes()
|
||||
+ workstreams()
|
||||
+ node_detail()
|
||||
+ snapshot()
|
||||
+ create_workstream()
|
||||
+ stream_cluster_events()
|
||||
+ login() / logout()
|
||||
@@ -129,6 +131,7 @@ package "sdk/typescript/ (TypeScript)" {
|
||||
class "TurnstoneConsole" as TSConsole <<ts>> {
|
||||
+ overview()
|
||||
+ nodes()
|
||||
+ snapshot()
|
||||
+ clusterEvents()
|
||||
...
|
||||
}
|
||||
|
||||
@@ -13,23 +13,19 @@ skinparam class {
|
||||
|
||||
' -- Protocol --
|
||||
interface "StorageBackend" as SB <<protocol>> {
|
||||
+register_session(session_id, title, node_id, ws_id)
|
||||
+save_message(session_id, role, content, ...)
|
||||
+load_session_messages(session_id) → list[dict]
|
||||
+list_sessions(limit) → list
|
||||
+delete_session(session_id) → bool
|
||||
+prune_sessions(retention_days) → (int, int)
|
||||
+resolve_session(alias_or_id) → str | None
|
||||
+save_session_config(session_id, config)
|
||||
+load_session_config(session_id) → dict
|
||||
+set_session_alias(session_id, alias) → bool
|
||||
+get_session_name(session_id) → str | None
|
||||
+update_session_title(session_id, title)
|
||||
+save_message(ws_id, role, content, ...)
|
||||
+load_messages(ws_id) → list[dict]
|
||||
+register_workstream(ws_id, node_id, name, state)
|
||||
+update_workstream_state(ws_id, state)
|
||||
+update_workstream_name(ws_id, name)
|
||||
+set_workstream_alias(ws_id, alias) → bool
|
||||
+update_workstream_title(ws_id, title)
|
||||
+resolve_workstream(alias_or_id) → str | None
|
||||
+delete_workstream(ws_id) → bool
|
||||
+prune_workstreams(retention_days) → (int, int)
|
||||
+list_workstreams(node_id, limit) → list
|
||||
+save_workstream_config(ws_id, config)
|
||||
+load_workstream_config(ws_id) → dict
|
||||
+kv_get(key) → str | None
|
||||
+kv_set(key, value) → str | None
|
||||
+kv_delete(key) → bool
|
||||
@@ -68,9 +64,8 @@ class "_schema.py" as Schema <<schema>> {
|
||||
+metadata: MetaData
|
||||
+memories: Table
|
||||
+conversations: Table
|
||||
+sessions: Table (node_id, ws_id, user_id)
|
||||
+workstreams: Table (node_id, user_id, state)
|
||||
+session_config: Table
|
||||
+workstreams: Table (node_id, alias, title, state)
|
||||
+workstream_config: Table
|
||||
+users: Table (username, password_hash)
|
||||
+api_tokens: Table (token_hash, scopes)
|
||||
+channel_users: Table (channel_type)
|
||||
@@ -106,14 +101,14 @@ class "_registry.py" as Registry {
|
||||
|
||||
' -- Facade --
|
||||
class "memory.py" as Facade <<facade>> {
|
||||
+register_session()
|
||||
+save_message()
|
||||
+load_session_messages()
|
||||
+load_messages()
|
||||
+register_workstream()
|
||||
+update_workstream_state()
|
||||
+save_workstream_config()
|
||||
+save_memory() / delete_memory()
|
||||
+search_memories()
|
||||
+... (all 22 functions)
|
||||
+... (all delegated functions)
|
||||
--
|
||||
Thin delegation to
|
||||
get_storage()
|
||||
|
||||
@@ -195,13 +195,13 @@ note right of Bot
|
||||
5. Broker.push_inbound(SendMessage)
|
||||
6. Bridge pops from Redis, drives server
|
||||
|
||||
**Session Resume (evicted workstreams)**
|
||||
**Workstream Resume (evicted workstreams)**
|
||||
1. Stale route detected (no MQ owner)
|
||||
2. Old session looked up via get_session_id_by_ws()
|
||||
2. Existing ws_id reused directly from route
|
||||
3. CreateWorkstreamMessage sent with
|
||||
resume_session=<old_session_id>
|
||||
resume_ws=<ws_id>
|
||||
4. Server resumes atomically during creation
|
||||
5. Bridge emits SessionResumedEvent → thread
|
||||
5. Bridge emits WorkstreamResumedEvent → thread
|
||||
end note
|
||||
|
||||
note right of Broker
|
||||
|
||||
@@ -0,0 +1,166 @@
|
||||
@startuml
|
||||
!theme plain
|
||||
title Turnstone — Watch Tool Architecture
|
||||
|
||||
skinparam participant {
|
||||
BackgroundColor<<server>> #FFE0B2
|
||||
BackgroundColor<<storage>> #B3E5FC
|
||||
BackgroundColor<<session>> #C8E6C9
|
||||
BackgroundColor<<ui>> #E8EAF6
|
||||
}
|
||||
|
||||
participant "ChatSession\n(session.py)" as Session <<session>>
|
||||
participant "WatchRunner\n(watch.py)" as Runner <<server>>
|
||||
participant "StorageBackend\n(SQLite)" as Storage <<storage>>
|
||||
participant "WebUI / SSE\n(server.py)" as UI <<ui>>
|
||||
|
||||
== Create Phase ==
|
||||
|
||||
Session -> Session : _prepare_watch(action="create")
|
||||
note right
|
||||
Validates:
|
||||
- command via is_command_blocked()
|
||||
- poll_every → parse_duration()
|
||||
- stop_on → validate_condition()
|
||||
- max watches limit (5)
|
||||
- duplicate name check
|
||||
needs_approval = True
|
||||
end note
|
||||
|
||||
Session -> Storage : create_watch(watch_id, ws_id,\nnode_id, command, interval,\nstop_on, max_polls, next_poll)
|
||||
|
||||
Session --> UI : tool_result:\n"Watch 'pr-review' created"
|
||||
|
||||
== Poll Phase (WatchRunner daemon, every 15s) ==
|
||||
|
||||
Runner -> Storage : list_due_watches(now)
|
||||
Storage --> Runner : due_watches[]
|
||||
note right
|
||||
Filters:
|
||||
active=1 AND
|
||||
next_poll <= now AND
|
||||
node_id matches
|
||||
end note
|
||||
|
||||
loop for each due watch
|
||||
|
||||
Runner -> Runner : is_command_blocked()?
|
||||
alt blocked
|
||||
Runner -> Storage : update_watch(active=False)
|
||||
else safe
|
||||
|
||||
Runner -> Runner : subprocess.run(command)
|
||||
note right
|
||||
timeout = tool_timeout
|
||||
start_new_session = True
|
||||
output truncated at 64KB
|
||||
end note
|
||||
|
||||
Runner -> Runner : evaluate_condition(\nstop_on, output,\nexit_code, prev_output)
|
||||
note right
|
||||
**Variables:**
|
||||
output, data, exit_code,
|
||||
prev_output, changed
|
||||
|
||||
**Safe builtins only:**
|
||||
len, str, int, sorted, ...
|
||||
No import/open/exec/eval
|
||||
|
||||
**stop_on=None:**
|
||||
fires on change (skip 1st poll)
|
||||
end note
|
||||
|
||||
alt condition fired OR max_polls reached
|
||||
Runner -> Storage : update_watch(\npoll_count++,\nlast_output, active=False)
|
||||
Runner -> Runner : format_watch_message()
|
||||
Runner -> Runner : _dispatch_result(ws_id, msg)
|
||||
else not fired
|
||||
Runner -> Storage : update_watch(\npoll_count++,\nlast_output, next_poll)
|
||||
end
|
||||
|
||||
end
|
||||
end
|
||||
|
||||
== Dispatch Phase ==
|
||||
|
||||
note over Runner, Session
|
||||
**Three dispatch paths:**
|
||||
end note
|
||||
|
||||
alt Path A: workstream active + idle
|
||||
Runner -> Session : dispatch_fn(message)\n→ _watch_pending.put()
|
||||
Session -> Session : _dispatch_pending_watch()\n→ self.send(message)
|
||||
Session -> UI : SSE: thinking, content,\ntool calls...
|
||||
note right
|
||||
Watch result appears as
|
||||
synthetic user message.
|
||||
Model sees it and responds.
|
||||
Depth guard: max 5 chains.
|
||||
end note
|
||||
|
||||
else Path B: workstream active + busy
|
||||
Runner -> Session : dispatch_fn(message)\n→ _watch_pending.put()
|
||||
note right
|
||||
Queued. Dispatched when
|
||||
current send() reaches IDLE.
|
||||
end note
|
||||
|
||||
else Path C: workstream evicted
|
||||
Runner -> Runner : restore_fn(ws_id)
|
||||
note right
|
||||
1. mgr.create() — may evict
|
||||
another idle workstream
|
||||
2. session.resume(ws_id)
|
||||
3. set_watch_runner()
|
||||
4. register new dispatch_fn
|
||||
end note
|
||||
Runner -> Session : restored dispatch_fn(message)
|
||||
end
|
||||
|
||||
== Cancel / List ==
|
||||
|
||||
Session -> Storage : list_watches_for_ws(ws_id)
|
||||
note right : action="list" (auto-approve)
|
||||
|
||||
Session -> Storage : update_watch(active=False)
|
||||
note right : action="cancel" (auto-approve)
|
||||
|
||||
== Server Lifecycle ==
|
||||
|
||||
note over Runner, Storage
|
||||
**Startup:**
|
||||
1. WatchRunner created in main() with storage + node_id
|
||||
2. restore_fn closure captures WorkstreamManager
|
||||
3. Initial workstream: session.set_watch_runner(runner)
|
||||
4. _lifespan(): runner.start() — daemon thread begins
|
||||
|
||||
**New workstream:**
|
||||
session.set_watch_runner(runner) in create_workstream()
|
||||
→ registers dispatch_fn for ws_id
|
||||
|
||||
**Eviction / close:**
|
||||
session.close() → runner.remove_dispatch_fn(ws_id)
|
||||
Watches remain active in DB — WatchRunner uses restore_fn
|
||||
|
||||
**Restart recovery:**
|
||||
Overdue watches fire ONE immediate poll
|
||||
next_poll updated to now + interval
|
||||
Normal cadence resumes
|
||||
|
||||
**Shutdown:**
|
||||
_lifespan(): runner.stop() — joins thread
|
||||
end note
|
||||
|
||||
== REST API ==
|
||||
|
||||
note over UI, Storage
|
||||
**GET /v1/api/watches[?ws_id=X]**
|
||||
List active watches (for node or workstream)
|
||||
|
||||
**POST /v1/api/watches/{watch_id}/cancel**
|
||||
Cancel a watch (sets active=False)
|
||||
|
||||
Both require write scope
|
||||
end note
|
||||
|
||||
@enduml
|
||||
@@ -0,0 +1,69 @@
|
||||
@startuml
|
||||
!theme plain
|
||||
skinparam backgroundColor #FFFFFF
|
||||
skinparam defaultFontName "IBM Plex Mono"
|
||||
skinparam componentStyle rectangle
|
||||
|
||||
title Turnstone Governance Architecture
|
||||
|
||||
package "Auth Flow" {
|
||||
[Login/Token Auth] as auth
|
||||
[_load_user_permissions()] as perms
|
||||
[_permissions_to_scopes()] as scopes
|
||||
[create_jwt()] as jwt
|
||||
}
|
||||
|
||||
package "Middleware" {
|
||||
[AuthMiddleware\n(scope check)] as mw
|
||||
[require_permission()\n(granular check)] as rp
|
||||
}
|
||||
|
||||
package "Governance Storage" {
|
||||
database "roles" as roles_db
|
||||
database "user_roles" as ur_db
|
||||
database "orgs" as orgs_db
|
||||
database "tool_policies" as tp_db
|
||||
database "prompt_templates" as pt_db
|
||||
database "usage_events" as ue_db
|
||||
database "audit_events" as ae_db
|
||||
}
|
||||
|
||||
package "Runtime Enforcement" {
|
||||
[evaluate_tool_policies_batch()] as eval
|
||||
[WebUI.approve_tools()] as approve
|
||||
[record_usage_event()] as usage
|
||||
[record_audit()] as audit
|
||||
}
|
||||
|
||||
package "Console UI" {
|
||||
[Admin Panel\n10 tabs] as ui
|
||||
[governance.js] as govjs
|
||||
[sessionStorage\npermissions] as ss
|
||||
}
|
||||
|
||||
auth --> perms : user_id
|
||||
perms --> roles_db : JOIN user_roles + roles
|
||||
perms --> scopes : permission set
|
||||
scopes --> jwt : scopes + permissions
|
||||
|
||||
jwt --> mw : JWT in cookie/header
|
||||
mw --> rp : scope OK → check permission
|
||||
|
||||
rp --> ui : 403 or allow
|
||||
|
||||
eval --> tp_db : list_tool_policies()
|
||||
approve --> eval : tool names
|
||||
approve --> ae_db : (via audit)
|
||||
|
||||
usage --> ue_db : on_status()
|
||||
audit --> ae_db : admin handlers
|
||||
|
||||
govjs --> roles_db : /v1/api/admin/roles
|
||||
govjs --> tp_db : /v1/api/admin/policies
|
||||
govjs --> pt_db : /v1/api/admin/templates
|
||||
govjs --> ue_db : /v1/api/admin/usage
|
||||
govjs --> ae_db : /v1/api/admin/audit
|
||||
|
||||
auth -[hidden]-> mw
|
||||
mw -[hidden]-> approve
|
||||
@enduml
|
||||
@@ -0,0 +1,286 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 1200 540" font-family="-apple-system, BlinkMacSystemFont, 'Segoe UI', Helvetica, Arial, sans-serif">
|
||||
<defs>
|
||||
<!-- Arrowhead markers -->
|
||||
<marker id="arrow" viewBox="0 0 10 7" refX="10" refY="3.5" markerWidth="8" markerHeight="6" orient="auto-start-auto">
|
||||
<path d="M 0 0 L 10 3.5 L 0 7 z" fill="#484f58"/>
|
||||
</marker>
|
||||
<marker id="arrow-blue" viewBox="0 0 10 7" refX="10" refY="3.5" markerWidth="8" markerHeight="6" orient="auto-start-auto">
|
||||
<path d="M 0 0 L 10 3.5 L 0 7 z" fill="#58a6ff"/>
|
||||
</marker>
|
||||
<marker id="arrow-green" viewBox="0 0 10 7" refX="10" refY="3.5" markerWidth="8" markerHeight="6" orient="auto-start-auto">
|
||||
<path d="M 0 0 L 10 3.5 L 0 7 z" fill="#3fb950"/>
|
||||
</marker>
|
||||
<marker id="arrow-orange" viewBox="0 0 10 7" refX="10" refY="3.5" markerWidth="8" markerHeight="6" orient="auto-start-auto">
|
||||
<path d="M 0 0 L 10 3.5 L 0 7 z" fill="#f0883e"/>
|
||||
</marker>
|
||||
<marker id="arrow-coral" viewBox="0 0 10 7" refX="10" refY="3.5" markerWidth="8" markerHeight="6" orient="auto-start-auto">
|
||||
<path d="M 0 0 L 10 3.5 L 0 7 z" fill="#f47067"/>
|
||||
</marker>
|
||||
<marker id="arrow-muted" viewBox="0 0 10 7" refX="10" refY="3.5" markerWidth="8" markerHeight="6" orient="auto-start-auto">
|
||||
<path d="M 0 0 L 10 3.5 L 0 7 z" fill="#8b949e"/>
|
||||
</marker>
|
||||
|
||||
<!-- Card shadow filter -->
|
||||
<filter id="shadow" x="-4%" y="-4%" width="108%" height="112%">
|
||||
<feDropShadow dx="0" dy="1" stdDeviation="2" flood-color="#000" flood-opacity="0.4"/>
|
||||
</filter>
|
||||
</defs>
|
||||
|
||||
<!-- Background -->
|
||||
<rect width="1200" height="540" rx="8" fill="#0d1117"/>
|
||||
|
||||
<!-- Title -->
|
||||
<text x="600" y="36" text-anchor="middle" fill="#e6edf3" font-size="15" font-weight="700" letter-spacing="3">TURNSTONE</text>
|
||||
<text x="600" y="54" text-anchor="middle" fill="#8b949e" font-size="11" letter-spacing="1">SYSTEM ARCHITECTURE</text>
|
||||
|
||||
<!-- ==================== COLUMN HEADERS ==================== -->
|
||||
<text x="90" y="86" text-anchor="middle" fill="#58a6ff" font-size="9" font-weight="600" letter-spacing="2">CLIENTS</text>
|
||||
<text x="276" y="86" text-anchor="middle" fill="#3fb950" font-size="9" font-weight="600" letter-spacing="2">GATEWAYS</text>
|
||||
<text x="480" y="86" text-anchor="middle" fill="#f0883e" font-size="9" font-weight="600" letter-spacing="2">MESSAGE QUEUE</text>
|
||||
<text x="700" y="86" text-anchor="middle" fill="#f47067" font-size="9" font-weight="600" letter-spacing="2">CLUSTER NODES</text>
|
||||
<text x="940" y="86" text-anchor="middle" fill="#f778ba" font-size="9" font-weight="600" letter-spacing="2">LLM PROVIDERS</text>
|
||||
|
||||
<!-- ==================== CLIENT BOXES ==================== -->
|
||||
<!-- CLI -->
|
||||
<g filter="url(#shadow)">
|
||||
<rect x="30" y="108" width="120" height="46" rx="5" fill="#161b22" stroke="#30363d" stroke-width="1"/>
|
||||
<rect x="30" y="108" width="120" height="5" rx="5" fill="#58a6ff"/>
|
||||
<rect x="30" y="108" width="120" height="5" fill="#58a6ff"/>
|
||||
<rect x="30" y="111" width="120" height="2" fill="#161b22"/>
|
||||
<text x="90" y="130" text-anchor="middle" fill="#e6edf3" font-size="11" font-weight="600">CLI</text>
|
||||
<text x="90" y="145" text-anchor="middle" fill="#8b949e" font-size="9">terminal REPL</text>
|
||||
</g>
|
||||
|
||||
<!-- Browser UI -->
|
||||
<g filter="url(#shadow)">
|
||||
<rect x="30" y="174" width="120" height="46" rx="5" fill="#161b22" stroke="#30363d" stroke-width="1"/>
|
||||
<rect x="30" y="174" width="120" height="5" rx="5" fill="#58a6ff"/>
|
||||
<rect x="30" y="174" width="120" height="5" fill="#58a6ff"/>
|
||||
<rect x="30" y="177" width="120" height="2" fill="#161b22"/>
|
||||
<text x="90" y="196" text-anchor="middle" fill="#e6edf3" font-size="11" font-weight="600">Browser UI</text>
|
||||
<text x="90" y="211" text-anchor="middle" fill="#8b949e" font-size="9">HTTP + SSE</text>
|
||||
</g>
|
||||
|
||||
<!-- SDK / API -->
|
||||
<g filter="url(#shadow)">
|
||||
<rect x="30" y="244" width="120" height="46" rx="5" fill="#161b22" stroke="#30363d" stroke-width="1"/>
|
||||
<rect x="30" y="244" width="120" height="5" rx="5" fill="#58a6ff"/>
|
||||
<rect x="30" y="244" width="120" height="5" fill="#58a6ff"/>
|
||||
<rect x="30" y="247" width="120" height="2" fill="#161b22"/>
|
||||
<text x="90" y="266" text-anchor="middle" fill="#e6edf3" font-size="11" font-weight="600">SDK / API</text>
|
||||
<text x="90" y="281" text-anchor="middle" fill="#8b949e" font-size="9">programmatic</text>
|
||||
</g>
|
||||
|
||||
<!-- Discord / Slack -->
|
||||
<g filter="url(#shadow)">
|
||||
<rect x="30" y="314" width="120" height="46" rx="5" fill="#161b22" stroke="#30363d" stroke-width="1"/>
|
||||
<rect x="30" y="314" width="120" height="5" rx="5" fill="#58a6ff"/>
|
||||
<rect x="30" y="314" width="120" height="5" fill="#58a6ff"/>
|
||||
<rect x="30" y="317" width="120" height="2" fill="#161b22"/>
|
||||
<text x="90" y="336" text-anchor="middle" fill="#e6edf3" font-size="11" font-weight="600">Discord / Slack</text>
|
||||
<text x="90" y="351" text-anchor="middle" fill="#8b949e" font-size="9">chat platforms</text>
|
||||
</g>
|
||||
|
||||
<!-- ==================== GATEWAY BOXES ==================== -->
|
||||
<!-- Console -->
|
||||
<g filter="url(#shadow)">
|
||||
<rect x="216" y="118" width="120" height="58" rx="5" fill="#161b22" stroke="#30363d" stroke-width="1"/>
|
||||
<rect x="216" y="118" width="120" height="5" rx="5" fill="#3fb950"/>
|
||||
<rect x="216" y="118" width="120" height="5" fill="#3fb950"/>
|
||||
<rect x="216" y="121" width="120" height="2" fill="#161b22"/>
|
||||
<text x="276" y="142" text-anchor="middle" fill="#e6edf3" font-size="11" font-weight="600">Console</text>
|
||||
<text x="276" y="157" text-anchor="middle" fill="#8b949e" font-size="9">dashboard + proxy</text>
|
||||
<text x="276" y="169" text-anchor="middle" fill="#8b949e" font-size="9">cluster management</text>
|
||||
</g>
|
||||
|
||||
<!-- Channel Gateway -->
|
||||
<g filter="url(#shadow)">
|
||||
<rect x="216" y="292" width="120" height="58" rx="5" fill="#161b22" stroke="#30363d" stroke-width="1"/>
|
||||
<rect x="216" y="292" width="120" height="5" rx="5" fill="#3fb950"/>
|
||||
<rect x="216" y="292" width="120" height="5" fill="#3fb950"/>
|
||||
<rect x="216" y="295" width="120" height="2" fill="#161b22"/>
|
||||
<text x="276" y="316" text-anchor="middle" fill="#e6edf3" font-size="11" font-weight="600">Channel Gateway</text>
|
||||
<text x="276" y="331" text-anchor="middle" fill="#8b949e" font-size="9">platform adapter</text>
|
||||
<text x="276" y="343" text-anchor="middle" fill="#8b949e" font-size="9">Discord, Slack, ...</text>
|
||||
</g>
|
||||
|
||||
<!-- ==================== REDIS MQ ==================== -->
|
||||
<g filter="url(#shadow)">
|
||||
<rect x="420" y="168" width="120" height="132" rx="5" fill="#161b22" stroke="#30363d" stroke-width="1"/>
|
||||
<rect x="420" y="168" width="120" height="5" rx="5" fill="#f0883e"/>
|
||||
<rect x="420" y="168" width="120" height="5" fill="#f0883e"/>
|
||||
<rect x="420" y="171" width="120" height="2" fill="#161b22"/>
|
||||
<text x="480" y="198" text-anchor="middle" fill="#e6edf3" font-size="11" font-weight="600">Redis MQ</text>
|
||||
<line x1="438" y1="210" x2="522" y2="210" stroke="#30363d" stroke-width="1"/>
|
||||
<text x="480" y="228" text-anchor="middle" fill="#8b949e" font-size="9">inbound queues</text>
|
||||
<text x="480" y="243" text-anchor="middle" fill="#8b949e" font-size="9">event pub/sub</text>
|
||||
<text x="480" y="258" text-anchor="middle" fill="#8b949e" font-size="9">node heartbeats</text>
|
||||
<text x="480" y="273" text-anchor="middle" fill="#8b949e" font-size="9">workstream routing</text>
|
||||
<text x="480" y="288" text-anchor="middle" fill="#8b949e" font-size="9">cluster state</text>
|
||||
</g>
|
||||
|
||||
<!-- ==================== CLUSTER NODES ==================== -->
|
||||
<!-- Cluster outline -->
|
||||
<rect x="598" y="100" width="204" height="310" rx="8" fill="none" stroke="#30363d" stroke-width="1" stroke-dasharray="4,3"/>
|
||||
<text x="700" y="422" text-anchor="middle" fill="#30363d" font-size="9" letter-spacing="1">CLUSTER</text>
|
||||
|
||||
<!-- Node A -->
|
||||
<g filter="url(#shadow)">
|
||||
<rect x="614" y="118" width="170" height="100" rx="5" fill="#161b22" stroke="#30363d" stroke-width="1"/>
|
||||
<rect x="614" y="118" width="170" height="5" rx="5" fill="#f47067"/>
|
||||
<rect x="614" y="118" width="170" height="5" fill="#f47067"/>
|
||||
<rect x="614" y="121" width="170" height="2" fill="#161b22"/>
|
||||
<text x="699" y="142" text-anchor="middle" fill="#e6edf3" font-size="11" font-weight="600">Node A</text>
|
||||
<line x1="632" y1="152" x2="766" y2="152" stroke="#30363d" stroke-width="1"/>
|
||||
<!-- Bridge -->
|
||||
<rect x="626" y="162" width="70" height="28" rx="3" fill="#1c2128" stroke="#30363d" stroke-width="1"/>
|
||||
<text x="661" y="180" text-anchor="middle" fill="#8b949e" font-size="9">bridge</text>
|
||||
<!-- Server -->
|
||||
<rect x="704" y="162" width="70" height="28" rx="3" fill="#1c2128" stroke="#30363d" stroke-width="1"/>
|
||||
<text x="739" y="180" text-anchor="middle" fill="#8b949e" font-size="9">server</text>
|
||||
<!-- Arrow bridge to server -->
|
||||
<line x1="696" y1="176" x2="702" y2="176" stroke="#484f58" stroke-width="1" marker-end="url(#arrow)"/>
|
||||
<!-- Tools label -->
|
||||
<text x="699" y="206" text-anchor="middle" fill="#484f58" font-size="8">14 tools + MCP</text>
|
||||
</g>
|
||||
|
||||
<!-- Node B -->
|
||||
<g filter="url(#shadow)">
|
||||
<rect x="614" y="238" width="170" height="100" rx="5" fill="#161b22" stroke="#30363d" stroke-width="1"/>
|
||||
<rect x="614" y="238" width="170" height="5" rx="5" fill="#f47067"/>
|
||||
<rect x="614" y="238" width="170" height="5" fill="#f47067"/>
|
||||
<rect x="614" y="241" width="170" height="2" fill="#161b22"/>
|
||||
<text x="699" y="262" text-anchor="middle" fill="#e6edf3" font-size="11" font-weight="600">Node B</text>
|
||||
<line x1="632" y1="272" x2="766" y2="272" stroke="#30363d" stroke-width="1"/>
|
||||
<!-- Bridge -->
|
||||
<rect x="626" y="282" width="70" height="28" rx="3" fill="#1c2128" stroke="#30363d" stroke-width="1"/>
|
||||
<text x="661" y="300" text-anchor="middle" fill="#8b949e" font-size="9">bridge</text>
|
||||
<!-- Server -->
|
||||
<rect x="704" y="282" width="70" height="28" rx="3" fill="#1c2128" stroke="#30363d" stroke-width="1"/>
|
||||
<text x="739" y="300" text-anchor="middle" fill="#8b949e" font-size="9">server</text>
|
||||
<!-- Arrow bridge to server -->
|
||||
<line x1="696" y1="296" x2="702" y2="296" stroke="#484f58" stroke-width="1" marker-end="url(#arrow)"/>
|
||||
<!-- Tools label -->
|
||||
<text x="699" y="326" text-anchor="middle" fill="#484f58" font-size="8">14 tools + MCP</text>
|
||||
</g>
|
||||
|
||||
<!-- ==================== LLM PROVIDERS ==================== -->
|
||||
<!-- OpenAI -->
|
||||
<g filter="url(#shadow)">
|
||||
<rect x="870" y="130" width="140" height="46" rx="5" fill="#161b22" stroke="#30363d" stroke-width="1"/>
|
||||
<rect x="870" y="130" width="140" height="5" rx="5" fill="#f778ba"/>
|
||||
<rect x="870" y="130" width="140" height="5" fill="#f778ba"/>
|
||||
<rect x="870" y="133" width="140" height="2" fill="#161b22"/>
|
||||
<text x="940" y="153" text-anchor="middle" fill="#e6edf3" font-size="11" font-weight="600">OpenAI</text>
|
||||
<text x="940" y="167" text-anchor="middle" fill="#8b949e" font-size="9">GPT-5, o-series</text>
|
||||
</g>
|
||||
|
||||
<!-- Anthropic -->
|
||||
<g filter="url(#shadow)">
|
||||
<rect x="870" y="196" width="140" height="46" rx="5" fill="#161b22" stroke="#30363d" stroke-width="1"/>
|
||||
<rect x="870" y="196" width="140" height="5" rx="5" fill="#f778ba"/>
|
||||
<rect x="870" y="196" width="140" height="5" fill="#f778ba"/>
|
||||
<rect x="870" y="199" width="140" height="2" fill="#161b22"/>
|
||||
<text x="940" y="219" text-anchor="middle" fill="#e6edf3" font-size="11" font-weight="600">Anthropic</text>
|
||||
<text x="940" y="233" text-anchor="middle" fill="#8b949e" font-size="9">Claude 4.5 / 4.6</text>
|
||||
</g>
|
||||
|
||||
<!-- Local / vLLM -->
|
||||
<g filter="url(#shadow)">
|
||||
<rect x="870" y="262" width="140" height="46" rx="5" fill="#161b22" stroke="#30363d" stroke-width="1"/>
|
||||
<rect x="870" y="262" width="140" height="5" rx="5" fill="#f778ba"/>
|
||||
<rect x="870" y="262" width="140" height="5" fill="#f778ba"/>
|
||||
<rect x="870" y="265" width="140" height="2" fill="#161b22"/>
|
||||
<text x="940" y="285" text-anchor="middle" fill="#e6edf3" font-size="11" font-weight="600">Local / vLLM</text>
|
||||
<text x="940" y="299" text-anchor="middle" fill="#8b949e" font-size="9">llama.cpp, NIM</text>
|
||||
</g>
|
||||
|
||||
<!-- ==================== STORAGE ==================== -->
|
||||
<g filter="url(#shadow)">
|
||||
<rect x="614" y="450" width="170" height="52" rx="5" fill="#161b22" stroke="#30363d" stroke-width="1"/>
|
||||
<rect x="614" y="450" width="170" height="5" rx="5" fill="#bc8cff"/>
|
||||
<rect x="614" y="450" width="170" height="5" fill="#bc8cff"/>
|
||||
<rect x="614" y="453" width="170" height="2" fill="#161b22"/>
|
||||
<text x="699" y="476" text-anchor="middle" fill="#e6edf3" font-size="11" font-weight="600">PostgreSQL / SQLite</text>
|
||||
<text x="699" y="492" text-anchor="middle" fill="#8b949e" font-size="9">conversations, memory, auth</text>
|
||||
</g>
|
||||
<text x="699" y="444" text-anchor="middle" fill="#bc8cff" font-size="9" font-weight="600" letter-spacing="2">STORAGE</text>
|
||||
|
||||
<!-- ==================== CONNECTION LINES ==================== -->
|
||||
|
||||
<!-- CLIENT -> GATEWAY connections -->
|
||||
<!-- Browser -> Console -->
|
||||
<line x1="150" y1="197" x2="214" y2="155" stroke="#58a6ff" stroke-width="1.2" stroke-opacity="0.6" marker-end="url(#arrow-blue)"/>
|
||||
<!-- Discord -> Channel -->
|
||||
<line x1="150" y1="337" x2="214" y2="325" stroke="#58a6ff" stroke-width="1.2" stroke-opacity="0.6" marker-end="url(#arrow-blue)"/>
|
||||
|
||||
<!-- CLI -> direct to Node A server (top path, curved) -->
|
||||
<path d="M 150 131 C 200 131, 200 100, 400 100 L 400 100 C 500 100, 570 140, 612 168" stroke="#58a6ff" stroke-width="1.2" stroke-opacity="0.5" fill="none" stroke-dasharray="6,3" marker-end="url(#arrow-blue)"/>
|
||||
<text x="370" y="96" fill="#484f58" font-size="8" text-anchor="middle">direct</text>
|
||||
|
||||
<!-- SDK -> Redis (direct push) -->
|
||||
<line x1="150" y1="267" x2="418" y2="240" stroke="#58a6ff" stroke-width="1.2" stroke-opacity="0.6" marker-end="url(#arrow-blue)"/>
|
||||
|
||||
<!-- GATEWAY -> REDIS connections -->
|
||||
<!-- Console -> Redis -->
|
||||
<line x1="336" y1="160" x2="418" y2="200" stroke="#3fb950" stroke-width="1.2" stroke-opacity="0.6" marker-end="url(#arrow-green)"/>
|
||||
<!-- Channel -> Redis -->
|
||||
<line x1="336" y1="318" x2="418" y2="272" stroke="#3fb950" stroke-width="1.2" stroke-opacity="0.6" marker-end="url(#arrow-green)"/>
|
||||
|
||||
<!-- REDIS -> NODE connections -->
|
||||
<!-- Redis -> Node A bridge -->
|
||||
<line x1="540" y1="210" x2="624" y2="176" stroke="#f0883e" stroke-width="1.2" stroke-opacity="0.6" marker-end="url(#arrow-orange)"/>
|
||||
<!-- Redis -> Node B bridge -->
|
||||
<line x1="540" y1="260" x2="624" y2="296" stroke="#f0883e" stroke-width="1.2" stroke-opacity="0.6" marker-end="url(#arrow-orange)"/>
|
||||
|
||||
<!-- Console -> Node (proxy, dashed) -->
|
||||
<path d="M 336 147 C 380 130, 500 108, 612 145" stroke="#3fb950" stroke-width="1" stroke-opacity="0.4" fill="none" stroke-dasharray="4,3" marker-end="url(#arrow-green)"/>
|
||||
<text x="468" y="120" fill="#484f58" font-size="8" text-anchor="middle">proxy</text>
|
||||
|
||||
<!-- NODE -> LLM connections -->
|
||||
<!-- Node A -> LLM providers -->
|
||||
<line x1="784" y1="168" x2="868" y2="155" stroke="#f47067" stroke-width="1.2" stroke-opacity="0.5" marker-end="url(#arrow-coral)"/>
|
||||
<line x1="784" y1="176" x2="868" y2="219" stroke="#f47067" stroke-width="1.2" stroke-opacity="0.3"/>
|
||||
<line x1="784" y1="180" x2="868" y2="282" stroke="#f47067" stroke-width="1.2" stroke-opacity="0.2"/>
|
||||
|
||||
<!-- Node B -> LLM providers -->
|
||||
<line x1="784" y1="288" x2="868" y2="163" stroke="#f47067" stroke-width="1.2" stroke-opacity="0.2"/>
|
||||
<line x1="784" y1="296" x2="868" y2="222" stroke="#f47067" stroke-width="1.2" stroke-opacity="0.3"/>
|
||||
<line x1="784" y1="300" x2="868" y2="288" stroke="#f47067" stroke-width="1.2" stroke-opacity="0.5" marker-end="url(#arrow-coral)"/>
|
||||
|
||||
<!-- NODE -> STORAGE connections -->
|
||||
<line x1="680" y1="338" x2="680" y2="448" stroke="#bc8cff" stroke-width="1.2" stroke-opacity="0.4" stroke-dasharray="4,3" marker-end="url(#arrow-muted)"/>
|
||||
<line x1="718" y1="218" x2="718" y2="236" stroke="#484f58" stroke-width="1" stroke-opacity="0.3" stroke-dasharray="2,2"/>
|
||||
|
||||
<!-- Extensibility hint -->
|
||||
<text x="699" y="392" text-anchor="middle" fill="#30363d" font-size="10">...</text>
|
||||
|
||||
<!-- Event flow: Bridges -> Redis (dashed, bidirectional feel) -->
|
||||
<line x1="624" y1="186" x2="542" y2="220" stroke="#f0883e" stroke-width="1" stroke-opacity="0.3" stroke-dasharray="3,3"/>
|
||||
<line x1="624" y1="286" x2="542" y2="250" stroke="#f0883e" stroke-width="1" stroke-opacity="0.3" stroke-dasharray="3,3"/>
|
||||
<text x="574" y="242" fill="#484f58" font-size="7" text-anchor="middle">events</text>
|
||||
|
||||
<!-- ==================== FLOW LABELS ==================== -->
|
||||
<!-- Interactive flow label -->
|
||||
<rect x="30" y="395" width="10" height="10" rx="2" fill="none" stroke="#58a6ff" stroke-width="1.5" stroke-dasharray="3,2"/>
|
||||
<text x="46" y="404" fill="#8b949e" font-size="9">interactive (direct)</text>
|
||||
|
||||
<!-- Queue flow label -->
|
||||
<rect x="160" y="395" width="10" height="10" rx="2" fill="none" stroke="#f0883e" stroke-width="1.5"/>
|
||||
<text x="176" y="404" fill="#8b949e" font-size="9">queue-driven</text>
|
||||
|
||||
<!-- Proxy/event label -->
|
||||
<rect x="275" y="395" width="10" height="10" rx="2" fill="none" stroke="#3fb950" stroke-width="1.5" stroke-dasharray="3,2"/>
|
||||
<text x="291" y="404" fill="#8b949e" font-size="9">proxy / events</text>
|
||||
|
||||
<!-- ==================== BOTTOM DETAILS ==================== -->
|
||||
<line x1="30" y1="430" x2="1170" y2="430" stroke="#21262d" stroke-width="1"/>
|
||||
|
||||
<!-- Routing rules at bottom, left-aligned -->
|
||||
<text x="44" y="456" fill="#30363d" font-size="9" font-weight="600" letter-spacing="1">ROUTING</text>
|
||||
<circle cx="44" cy="474" r="3" fill="#f47067" opacity="0.6"/>
|
||||
<text x="54" y="477" fill="#484f58" font-size="9">target_node set → route to specific node queue</text>
|
||||
<circle cx="44" cy="494" r="3" fill="#f0883e" opacity="0.6"/>
|
||||
<text x="54" y="497" fill="#484f58" font-size="9">ws_id set → route to owning node</text>
|
||||
<circle cx="44" cy="514" r="3" fill="#58a6ff" opacity="0.6"/>
|
||||
<text x="54" y="517" fill="#484f58" font-size="9">neither → shared queue, any node picks up</text></svg>
|
||||
|
After Width: | Height: | Size: 18 KiB |
@@ -1,3 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:9a1b0361c466327d0011a488847ea3c0365983713537d4a7c27cd7f5538ba33c
|
||||
size 164829
|
||||
oid sha256:d8ce6d2a43a991655c3f64a20b6e810fdb2f78eb767acc3d3d1b8d2c9f443181
|
||||
size 165011
|
||||
|
||||
@@ -1,3 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:7d75da92a657525bcbb7a425dc6c8d3cafe074c3ac3bff3cf4b1d44aea607b50
|
||||
size 330156
|
||||
oid sha256:0ee0a9391bd19d92e9271bf6bd531e9c2e18baf8c5a11ead49b3c10db4d8939b
|
||||
size 329625
|
||||
|
||||
@@ -1,3 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:637458e0d78df82752746e519cd7300a830c8ce211f21625694ad0c162ca316d
|
||||
size 481637
|
||||
oid sha256:760c37e67736588dadee21d500419a48e9fc50f8bdc5667e686c580022bd40e2
|
||||
size 554869
|
||||
|
||||
@@ -1,3 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:dc3b64c9e48153641af62ed43fbc1d89a31d1a8a61e7e71cfc550c805000310d
|
||||
size 288290
|
||||
oid sha256:da9d32000e3d92d92ce621661ced60f276f9b5be652f5ed6123b400505415f4a
|
||||
size 319702
|
||||
|
||||
@@ -1,3 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:b842683d238664a3e35d04358fecfc56cefd013f7dca5f13357b0376f881e1b3
|
||||
size 245043
|
||||
oid sha256:a3ffd93ccb634f76560f1dd65242b89cd34443b37e355f25f29a1c63a22001be
|
||||
size 265259
|
||||
|
||||
@@ -1,3 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:b22d5980fe5cc4b8466ba0797113dc8fa83fab8df24b5dacceaf97e62e2e25b0
|
||||
size 187649
|
||||
oid sha256:d17f3feacf7bc9f64dfea19464143bc9b6ef0da5d55e6d57c0bc5a73d5724eba
|
||||
size 184466
|
||||
|
||||
@@ -1,3 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:90e4f74be795b530e711faa87bc6eb2b3bf6abb68d8fac8ebff7aaf30c6fbe53
|
||||
oid sha256:09535722ba975e47cf0557a40b6c481f125ff2022c396f79715c3bba9f715871
|
||||
size 222032
|
||||
|
||||
@@ -1,3 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:d33b9b3affcdb07086b5aebca8a3b9c2b009cdfc6f360950a0e72e65fbcb8f17
|
||||
size 201602
|
||||
oid sha256:ed457b10b534b5fc2a5e190b281d7ded4dd1615da2229d67a373cf5dddccd059
|
||||
size 201601
|
||||
|
||||
@@ -1,3 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:adac93a0bb062d7199b819a600a0983ff011a75d16928fb80322cbb41f9284ea
|
||||
size 158866
|
||||
oid sha256:7896c6e041b6dbb89d034468fa980c8fe645df5eb969d45ef966ccc6399edac2
|
||||
size 200083
|
||||
|
||||
@@ -1,3 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:69f201cff948cb0a19810b7c4ad26d346f869ee2dd3141eba4f353332efa2e21
|
||||
size 373649
|
||||
oid sha256:35cf3a6942f62dabcbbe012ac2f9e6f155332c894692981b076de5a25c1f3330
|
||||
size 374055
|
||||
|
||||
@@ -1,3 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:97e7210cd8f1ad195f4d5e25e778d82df3c08c5c6e0f09722e84a7a453714867
|
||||
size 411664
|
||||
oid sha256:a74b4b8b5dbfb1a51a01100b731477968942b01218bad9451a3d5a9cb3003294
|
||||
size 411665
|
||||
|
||||
@@ -1,3 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:4c3214ef416c1dfe4fa17834c2b6f4071a8093cfdb2b862848ca79938f726a13
|
||||
oid sha256:84524f4bc900708ac8adf081591d336f862830188eb8505e71a0f071b339d923
|
||||
size 252599
|
||||
|
||||
@@ -1,3 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:c9823a41e09611c5c0530d9fc12ad4139cfcc3ae238dc665b2888ec94d7d6781
|
||||
size 195708
|
||||
oid sha256:e7c3e40c10425d721f833390ae3531c09af501157fd3142531ba4eba86ff719d
|
||||
size 197112
|
||||
|
||||
@@ -1,3 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:733aa17cbfdab60a601cac6adf439c657dd3535e3d6c33c69c2ef93ba8ec5989
|
||||
size 251042
|
||||
oid sha256:5faa5335152685cf1c8bf77ed93847d751cde59e1afed651e5991113f2f0f31b
|
||||
size 242670
|
||||
|
||||
@@ -1,3 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:f4b2a2010335f986511c8dabaf49ec046ac02e577f9bc9924897e045f860bb13
|
||||
size 248808
|
||||
oid sha256:af5ab3126bf685afe68e24bc4b0ed97371d0ebdb77bf4d76c0331ab120580cc0
|
||||
size 248809
|
||||
|
||||
@@ -1,3 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:6049cc0b07480df88d0d93aa977a1e97f64b41588325ff41d98be0e39431fc5c
|
||||
size 431712
|
||||
oid sha256:1380065cbb5f95b5ea7dc6b2a00986c455b82888af60784980dffbd936460dcf
|
||||
size 431129
|
||||
|
||||
@@ -1,3 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:f55e177e0838a16d9bc4f07b162b4b6a966cc596c9d0a022d35a3c84f23e7b02
|
||||
oid sha256:f0f6097840fccdbfe16cd5e4c9f5d063b2c36942460a944df68b8ec947e63ea3
|
||||
size 221452
|
||||
|
||||
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:96176a09e65e90dadc32d5e9ed778423842be89204d2cf382225f53a90cfaf01
|
||||
size 258547
|
||||
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:b68663599922f72d7ca21be820523a5b472c268897d194e5237bba2441c004ec
|
||||
size 124497
|
||||
+11
-3
@@ -28,6 +28,8 @@ Console dashboard: http://localhost:8090
|
||||
| `bridge` | — | default | Redis-to-HTTP bridge (multi-node routing) |
|
||||
| `console` | 8090 | default | Cluster dashboard |
|
||||
| `channel` | — | production | Channel gateway (Discord, Slack, etc.) |
|
||||
| `server-1`…`server-10` | — | cluster | 10-node server fleet (PostgreSQL required) |
|
||||
| `bridge-1`…`bridge-10` | — | cluster | Matching bridge fleet |
|
||||
| `sim` | — | sim | Multi-node cluster simulator |
|
||||
|
||||
## Profiles
|
||||
@@ -44,6 +46,12 @@ docker compose up
|
||||
docker compose --profile production up
|
||||
```
|
||||
|
||||
**Cluster** — 10-node server/bridge fleet sharing PostgreSQL and Redis. Access all nodes via the console at `:8090`. Requires `POSTGRES_PASSWORD`:
|
||||
|
||||
```bash
|
||||
docker compose --profile cluster up
|
||||
```
|
||||
|
||||
**Sim** — adds the simulator. Can run alongside the full stack or standalone with just Redis and the console:
|
||||
|
||||
```bash
|
||||
@@ -135,13 +143,13 @@ The channel service runs in the `production` profile. When `TURNSTONE_DISCORD_TO
|
||||
|
||||
## Scaling
|
||||
|
||||
Scale to multiple server/bridge pairs:
|
||||
For multi-node testing, use the `cluster` profile which provides 10 dedicated server+bridge pairs with unique node IDs (`node-1` through `node-10`), resource limits, and shared PostgreSQL:
|
||||
|
||||
```bash
|
||||
docker compose up --scale server=3 --scale bridge=3
|
||||
POSTGRES_PASSWORD=secret docker compose --profile cluster up
|
||||
```
|
||||
|
||||
Each bridge auto-generates a unique node ID from its container hostname. When scaling `server`, remove the host port mapping (or use a reverse proxy) to avoid port conflicts.
|
||||
The default `server` and `bridge` also run alongside the cluster nodes (11 total). All nodes are accessible via the console dashboard at `:8090`.
|
||||
|
||||
## Volumes
|
||||
|
||||
|
||||
@@ -0,0 +1,158 @@
|
||||
# Governance
|
||||
|
||||
Turnstone governance provides role-based access control (RBAC), tool execution
|
||||
policies, prompt templates, usage tracking, and audit logging for the admin
|
||||
console.
|
||||
|
||||
## Architecture
|
||||
|
||||
See [diagram: 19-governance-architecture.puml](diagrams/19-governance-architecture.puml).
|
||||
|
||||
### RBAC (Roles & Permissions)
|
||||
|
||||
The permission model has two layers:
|
||||
|
||||
1. **Scopes** (legacy) — `read`, `write`, `approve`. Checked by `AuthMiddleware`
|
||||
on every request based on URL path classification.
|
||||
2. **Permissions** (granular) — 15 permission strings checked per-endpoint by
|
||||
`require_permission()`.
|
||||
|
||||
**Built-in roles** (seeded by migration 008):
|
||||
|
||||
| Role | Permissions |
|
||||
|------|-------------|
|
||||
| admin | read, write, approve, admin.users, admin.roles, admin.orgs, admin.policies, admin.templates, admin.audit, admin.usage, admin.schedules, admin.watches, tools.approve, workstreams.create, workstreams.close |
|
||||
| operator | read, write, workstreams.create, workstreams.close |
|
||||
| viewer | read |
|
||||
|
||||
Custom roles can be created with any subset of the 15 valid permissions.
|
||||
|
||||
**Auth flow:**
|
||||
1. User logs in (password or API token) → `_load_user_permissions()` aggregates
|
||||
permissions from all assigned roles
|
||||
2. `_permissions_to_scopes()` derives legacy scopes (any `admin.*` → `approve`)
|
||||
3. JWT created with both `scopes` and `permissions` claims
|
||||
4. Middleware checks scope → handler checks permission via `require_permission()`
|
||||
|
||||
### Tool Policies
|
||||
|
||||
Admin-defined rules that control tool execution:
|
||||
|
||||
- **Pattern matching**: Glob syntax via `fnmatch` (e.g., `bash*`, `file_write`, `*`)
|
||||
- **Actions**: `allow` (auto-approve), `deny` (block), `ask` (normal approval flow)
|
||||
- **Priority**: Higher priority evaluated first, first match wins
|
||||
- **Enforcement**: `evaluate_tool_policies_batch()` called in `WebUI.approve_tools()`
|
||||
before the `auto_approve` check
|
||||
|
||||
### Prompt Templates
|
||||
|
||||
Reusable system message templates with variable substitution:
|
||||
|
||||
- **Variables**: `{{variable_name}}` placeholders in content
|
||||
- **Categories**: general, engineering, support, custom
|
||||
- **Default flag**: `is_default=true` templates intended for new workstreams
|
||||
- **Storage**: `prompt_templates` table with JSON `variables` array
|
||||
|
||||
### Usage Tracking
|
||||
|
||||
Per-LLM-request token and tool call metrics:
|
||||
|
||||
- **Recording**: `on_status()` in `WebUI` records a `usage_event` after each
|
||||
LLM response with prompt/completion tokens, tool call count, model, ws_id
|
||||
- **Querying**: `GET /v1/api/admin/usage` with `group_by` (day/hour/model/user)
|
||||
and time range filtering
|
||||
- **Pruning**: `prune_usage_events(retention_days=90)` and
|
||||
`prune_audit_events(retention_days=365)` run automatically via the
|
||||
console scheduler's periodic cleanup cycle
|
||||
|
||||
### Audit Logging
|
||||
|
||||
Append-only trail of admin actions:
|
||||
|
||||
- **Recording**: `record_audit()` helper called from all admin mutation handlers
|
||||
- **Events captured**: user.create, user.delete, token.create, token.revoke,
|
||||
channel.link, channel.unlink, role.create, role.update, role.delete,
|
||||
role.assign, role.unassign, policy.create, policy.update, policy.delete,
|
||||
template.create, template.update, template.delete, org.update
|
||||
- **Querying**: `GET /v1/api/admin/audit` with action/user/time filters + pagination
|
||||
|
||||
## Database Schema
|
||||
|
||||
Migration 008 adds 7 tables:
|
||||
|
||||
| Table | Purpose |
|
||||
|-------|---------|
|
||||
| `orgs` | Organizations (single default org for now) |
|
||||
| `roles` | Named permission bundles (3 builtin + custom) |
|
||||
| `user_roles` | User-to-role assignments (composite PK) |
|
||||
| `tool_policies` | Per-tool approve/deny/ask rules |
|
||||
| `prompt_templates` | Reusable system message templates |
|
||||
| `usage_events` | Per-request token/tool metrics |
|
||||
| `audit_events` | Admin action log |
|
||||
|
||||
Also adds `org_id` column to `users` table.
|
||||
|
||||
## API Endpoints
|
||||
|
||||
All under `/v1/api/admin/` (requires `approve` scope + granular permission).
|
||||
|
||||
| Group | Endpoints | Permission |
|
||||
|-------|-----------|------------|
|
||||
| Users / Tokens / Channels | 9 (CRUD) | `admin.users` |
|
||||
| Roles | 7 (CRUD + assignment) | `admin.roles` / `admin.users` |
|
||||
| Orgs | 3 (list, get, update) | `admin.orgs` |
|
||||
| Tool Policies | 4 (CRUD) | `admin.policies` |
|
||||
| Prompt Templates | 4 (CRUD) | `admin.templates` |
|
||||
| Schedules | 6 (CRUD + runs) | `admin.schedules` |
|
||||
| Watches | 3 (list, create, cancel) | `admin.watches` |
|
||||
| Usage | 1 (aggregated query) | `admin.usage` |
|
||||
| Audit | 1 (paginated, filtered) | `admin.audit` |
|
||||
|
||||
Full OpenAPI spec at `/openapi.json` and Swagger UI at `/docs`.
|
||||
|
||||
## Admin Console UI
|
||||
|
||||
5 new tabs added to the admin panel (10 total):
|
||||
|
||||
- **Roles** — CRUD roles, permission checkbox grid, user role assignment modal
|
||||
- **Policies** — CRUD tool policies with colored action badges (green/red/amber)
|
||||
- **Templates** — CRUD prompt templates with wide modal, textarea editor
|
||||
- **Usage** — Summary readouts + CSS bar chart, time range + group-by selectors
|
||||
- **Audit** — Filterable log with relative timestamps, load-more pagination
|
||||
|
||||
Tabs are permission-gated: hidden if the user lacks the required permission.
|
||||
|
||||
## SDK
|
||||
|
||||
Both Python and TypeScript console SDKs expose governance methods:
|
||||
|
||||
**Python** (`TurnstoneConsole` / `AsyncTurnstoneConsole`):
|
||||
- `list_roles()`, `create_role()`, `update_role()`, `delete_role()`
|
||||
- `list_user_roles()`, `assign_role()`, `unassign_role()`
|
||||
- `list_orgs()`, `get_org()`, `update_org()`
|
||||
- `list_policies()`, `create_policy()`, `update_policy()`, `delete_policy()`
|
||||
- `list_templates()`, `create_template()`, `update_template()`, `delete_template()`
|
||||
- `get_usage(since, group_by=...)`, `get_audit(action=..., limit=...)`
|
||||
|
||||
**TypeScript** (`TurnstoneConsole`):
|
||||
- Same methods with camelCase naming and typed interfaces
|
||||
|
||||
## Security Considerations
|
||||
|
||||
- **Privilege escalation prevented**: `admin_assign_role` blocks self-assignment
|
||||
and requires caller to hold a superset of the target role's permissions
|
||||
- **Permission validation**: Role create/update validates permissions against
|
||||
a 15-item allowlist (`_VALID_PERMISSIONS`)
|
||||
- **Self-deletion blocked**: `admin_delete_user` rejects attempts to delete
|
||||
your own account (matching the self-assignment guard on role endpoints)
|
||||
- **Field allowlists**: Storage `update_*` methods filter fields against
|
||||
allowlists (`_ROLE_MUTABLE`, `_POLICY_MUTABLE`, etc.) — handler bugs
|
||||
cannot overwrite `role_id`, `builtin`, `created`, or other protected columns
|
||||
- **Bootstrap safety**: `handle_auth_setup` fails and rolls back if admin role
|
||||
assignment fails, preventing locked-out first user
|
||||
- **API token RBAC**: `_authenticate_api_token` loads permissions from user's
|
||||
roles, ensuring API tokens are subject to RBAC enforcement
|
||||
- **Policy evaluation is fail-open**: If storage is unavailable, tool policies
|
||||
degrade to the existing approval flow (not auto-approve)
|
||||
- **Audit IP resolution**: `_audit_context()` prefers `X-Forwarded-For` for
|
||||
client IP when behind a reverse proxy, falling back to `request.client.host`
|
||||
+7
-1
@@ -75,10 +75,11 @@ Both `TurnstoneServer` (sync) and `AsyncTurnstoneServer` (async) expose:
|
||||
| | `approve(*, ws_id, approved, feedback, always)` | `StatusResponse` |
|
||||
| | `plan_feedback(*, ws_id, feedback)` | `StatusResponse` |
|
||||
| | `command(*, ws_id, command)` | `StatusResponse` |
|
||||
| | `cancel(ws_id)` | `StatusResponse` |
|
||||
| **Streaming** | `stream_events(ws_id)` | `Iterator[ServerEvent]` |
|
||||
| | `stream_global_events()` | `Iterator[ServerEvent]` |
|
||||
| **High-level** | `send_and_wait(message, ws_id, *, timeout, on_event)` | `TurnResult` |
|
||||
| **Sessions** | `list_sessions()` | `ListSessionsResponse` |
|
||||
| **Saved** | `list_saved_workstreams()` | `ListSavedWorkstreamsResponse` |
|
||||
| **Auth** | `login(username=..., password=...)` | `AuthLoginResponse` |
|
||||
| | `login(token="ts_xxx")` | `AuthLoginResponse` |
|
||||
| | `logout()` | `StatusResponse` |
|
||||
@@ -95,6 +96,7 @@ Both `TurnstoneConsole` (sync) and `AsyncTurnstoneConsole` (async) expose:
|
||||
| | `nodes(*, sort, limit, offset)` | `ClusterNodesResponse` |
|
||||
| | `workstreams(*, state, node, search, sort, page, per_page)` | `ClusterWorkstreamsResponse` |
|
||||
| | `node_detail(node_id)` | `NodeDetailResponse` |
|
||||
| | `snapshot()` | `ClusterSnapshotResponse` |
|
||||
| | `create_workstream(*, node_id, name, model, initial_message)` | `ConsoleCreateWsResponse` |
|
||||
| **Schedules** | `list_schedules()` | `ListSchedulesResponse` |
|
||||
| | `create_schedule(*, name, schedule_type, initial_message, ...)` | `ScheduleInfo` |
|
||||
@@ -128,6 +130,7 @@ SSE events are deserialized into typed dataclasses. Use `event.type` to discrimi
|
||||
| `error` | `ErrorEvent` | `message` |
|
||||
| `info` | `InfoEvent` | `message` |
|
||||
| `stream_end` | `StreamEndEvent` | — |
|
||||
| `cancelled` | `CancelledEvent` | — |
|
||||
|
||||
**Global events** (from `stream_global_events()`):
|
||||
|
||||
@@ -146,6 +149,9 @@ SSE events are deserialized into typed dataclasses. Use `event.type` to discrimi
|
||||
| `node_lost` | `NodeLostEvent` | `node_id` |
|
||||
| `cluster_state` | `ClusterStateEvent` | `ws_id`, `node_id`, `state`, `tokens` |
|
||||
| `ws_created` | `ClusterWsCreatedEvent` | `ws_id`, `node_id`, `name` |
|
||||
| `ws_closed` | `ClusterWsClosedEvent` | `ws_id` |
|
||||
| `ws_rename` | `ClusterWsRenameEvent` | `ws_id`, `name` |
|
||||
| `snapshot` | `ClusterSnapshotEvent` | `nodes`, `overview`, `timestamp` |
|
||||
|
||||
### TurnResult
|
||||
|
||||
|
||||
+29
-1
@@ -73,7 +73,7 @@ Scopes are hierarchical — higher scopes imply all lower ones.
|
||||
|
||||
| Scope | Grants | Implies |
|
||||
|-------|--------|---------|
|
||||
| `read` | View workstreams, sessions, history | — |
|
||||
| `read` | View workstreams, saved workstreams, history | — |
|
||||
| `write` | Send messages, create/close workstreams | `read` |
|
||||
| `approve` | Approve tool calls, admin endpoints | `read`, `write` |
|
||||
|
||||
@@ -92,6 +92,34 @@ Public paths bypass authentication entirely: `/`, `/health`, `/metrics`,
|
||||
`/static/*`, `/shared/*`, `/docs`, `/openapi.json`, `/api/auth/login`,
|
||||
`/api/auth/logout`, `/api/auth/status`, `/api/auth/setup`.
|
||||
|
||||
### RBAC (Granular Permissions)
|
||||
|
||||
> See also: [Governance documentation](governance.md)
|
||||
|
||||
Scopes provide coarse endpoint-level access control. For finer-grained
|
||||
enforcement, the governance layer adds 15 named permissions checked
|
||||
per-endpoint by `require_permission()`. Permissions are bundled into
|
||||
roles; users are assigned roles via the `user_roles` join table.
|
||||
|
||||
At login, `_load_user_permissions()` aggregates all permissions from
|
||||
the user's assigned roles. `_permissions_to_scopes()` derives legacy
|
||||
scopes for backward compatibility (e.g., any `admin.*` permission
|
||||
implies the `approve` scope). The JWT carries both `scopes` and
|
||||
`permissions` claims.
|
||||
|
||||
Three built-in roles are seeded by migration 008:
|
||||
|
||||
| Role | Permissions |
|
||||
|------|-------------|
|
||||
| admin | All 15 permissions |
|
||||
| operator | read, write, workstreams.create, workstreams.close |
|
||||
| viewer | read |
|
||||
|
||||
Custom roles can be created with any subset of the valid permissions.
|
||||
Role creation and update validate permissions against a static allowlist.
|
||||
Self-assignment is blocked, and assigning a role requires the caller to
|
||||
hold a superset of the target role's permissions.
|
||||
|
||||
---
|
||||
|
||||
## Login Flows
|
||||
|
||||
+208
-9
@@ -1,6 +1,6 @@
|
||||
# Tools Reference
|
||||
|
||||
turnstone exposes 15 built-in tools plus any number of external MCP tools to the
|
||||
turnstone exposes 16 built-in tools plus any number of external MCP tools to the
|
||||
LLM via the OpenAI function-calling interface. Built-in tools are defined as JSON
|
||||
files under `turnstone/tools/` and loaded at startup by `turnstone/core/tools.py`.
|
||||
MCP tools are discovered from configured MCP servers at startup by
|
||||
@@ -46,11 +46,12 @@ schema plus turnstone-specific metadata keys:
|
||||
|
||||
| Name | Description |
|
||||
|---------------------|-------------|
|
||||
| `TOOLS` | All 15 tool definitions (sent to the model). |
|
||||
| `TOOLS` | All 16 tool definitions (sent to the model). |
|
||||
| `AGENT_TOOLS` | Tools with `agent: true` -- available to plan sub-agents. Read-only tools. |
|
||||
| `TASK_AGENT_TOOLS` | Tools with `task_agent: true` -- available to task sub-agents. Includes write operations. |
|
||||
| `AGENT_AUTO_TOOLS` | Set of tool names with `auto_approve: true` -- no user confirmation needed. |
|
||||
| `TASK_AUTO_TOOLS` | Same as `AGENT_AUTO_TOOLS` (identical filter). |
|
||||
| `BUILTIN_TOOL_NAMES`| Frozenset of all 16 built-in tool names. Used by tool search to distinguish always-on tools from deferrable MCP tools. |
|
||||
| `PRIMARY_KEY_MAP` | Dict mapping tool name to its `primary_key` parameter name. |
|
||||
|
||||
---
|
||||
@@ -68,6 +69,9 @@ Tool execution follows a three-phase pipeline inside `ChatSession._execute_tools
|
||||
- Parses the JSON arguments (with fallback for malformed JSON).
|
||||
- If JSON parsing fails entirely, uses `PRIMARY_KEY_MAP` to map a bare string
|
||||
to the correct parameter.
|
||||
- Dispatches to the matching `_prepare_{func_name}()` handler. There are 15
|
||||
built-in tools plus `tool_search` (synthetic, client-side BM25 fallback) and
|
||||
the generic `_prepare_mcp_tool()` handler for MCP tools.
|
||||
- Validates arguments and builds a preview dict containing:
|
||||
- `call_id`, `func_name`, `header`, `preview` (for display)
|
||||
- `needs_approval` (bool)
|
||||
@@ -185,15 +189,17 @@ Execute a bash command and return stdout + stderr.
|
||||
|
||||
### read_file
|
||||
|
||||
Read the contents of a file, returning numbered lines.
|
||||
Read the contents of a file, returning numbered lines for text files or
|
||||
base64-encoded image data for supported image formats.
|
||||
|
||||
| Parameter | Type | Required | Description |
|
||||
|-----------|---------|----------|-------------|
|
||||
| `path` | string | yes | Absolute or relative file path. |
|
||||
| `offset` | integer | no | Line number to start from (1-based, default: 1). |
|
||||
| `limit` | integer | no | Maximum number of lines to read. Omit for full file. |
|
||||
| `offset` | integer | no | Line number to start from (1-based, default: 1). Text files only. |
|
||||
| `limit` | integer | no | Maximum number of lines to read. Omit for full file. Text files only. |
|
||||
|
||||
- **What it does**: Reads the file and returns content with line numbers. Must be called before `edit_file` on the same path (the session tracks which files have been read).
|
||||
- **What it does**: For text files, reads and returns content with line numbers. For image files (PNG, JPEG, GIF, WebP, BMP, TIFF, ICO), returns image data as multi-part content when the model supports vision, or a text description when it does not. SVG files are read as text. Images larger than 4 MB are rejected. Must be called before `edit_file` on the same path (the session tracks which files have been read).
|
||||
- **Vision support**: Controlled by `ModelCapabilities.supports_vision`. All commercial OpenAI and Anthropic models have vision enabled. Local models (vLLM, llama.cpp, NIM) default to off — enable via `[models.*.capabilities] supports_vision = true` in config.toml.
|
||||
- **Auto-approve**: Yes.
|
||||
- **Agent availability**: `agent` and `task_agent`.
|
||||
|
||||
@@ -337,7 +343,7 @@ Plan before implementing -- an autonomous agent explores the codebase and writes
|
||||
|-----------|--------|----------|-------------|
|
||||
| `prompt` | string | yes | What to plan -- the goal, constraints, and scope. |
|
||||
|
||||
- **What it does**: Spawns a planning sub-agent with `AGENT_TOOLS` (read-only tools: `read_file`, `search`, `math`, `man`, `web_fetch`, `web_search`). The agent explores the codebase and writes a structured plan to `.plan-<session_id>.md` (unique per session, so concurrent workstreams never collide). If the `plan` tool has been called before in the same session, the prior plan is passed to the agent as context so it refines rather than restarts. After completion, the user is prompted to review and can accept, reject, or annotate the plan.
|
||||
- **What it does**: Spawns a planning sub-agent with `AGENT_TOOLS` (read-only tools: `read_file`, `search`, `math`, `man`, `web_fetch`, `web_search`). The agent explores the codebase and writes a structured plan to `.plan-<ws_id>.md` (unique per workstream, so concurrent workstreams never collide). If the `plan` tool has been called before in the same session, the prior plan is passed to the agent as context so it refines rather than restarts. After completion, the user is prompted to review and can accept, reject, or annotate the plan.
|
||||
- **Auto-approve**: No -- requires user confirmation, plus post-execution review gate.
|
||||
- **Agent availability**: Not available to sub-agents (top-level only).
|
||||
|
||||
@@ -416,6 +422,81 @@ Provide either `username` for user-based targeting or `channel_type` +
|
||||
|
||||
---
|
||||
|
||||
### watch
|
||||
|
||||
Set up periodic polling of a shell command within the current workstream.
|
||||
Results are injected back into the conversation as synthetic user messages,
|
||||
triggering the model to respond and act. Use for monitoring CI/CD pipelines,
|
||||
PR reviews, deployments, file changes, etc.
|
||||
|
||||
| Parameter | Type | Required | Description |
|
||||
|-------------|---------|----------|-------------|
|
||||
| `action` | string | yes | `create`, `list`, or `cancel`. |
|
||||
| `command` | string | create | Shell command to poll periodically. |
|
||||
| `poll_every`| string | no | Poll interval as duration (`30s`, `5m`, `1h`). Default: `5m`. |
|
||||
| `stop_on` | string | no | Python expression for stop condition (see below). Omit for change detection. |
|
||||
| `name` | string | create | Human-readable watch name (e.g. `pr-review`). Used as identifier for cancel. |
|
||||
| `max_polls` | integer | no | Max poll cycles before auto-cancel. Default: 100. |
|
||||
|
||||
**Actions:**
|
||||
|
||||
- `create` — Start a new watch. Requires approval (same as bash — runs shell
|
||||
commands). Persists to the `watches` table; the server-level `WatchRunner`
|
||||
daemon polls every 15 seconds for due watches.
|
||||
- `list` — Show all active watches in this workstream. Auto-approved.
|
||||
- `cancel` — Stop a watch by name or ID prefix. Auto-approved.
|
||||
|
||||
**Stop condition DSL** — The `stop_on` parameter accepts a Python expression
|
||||
evaluated after each poll. Available variables:
|
||||
|
||||
| Variable | Type | Description |
|
||||
|---------------|------------|-------------|
|
||||
| `output` | `str` | stdout (+stderr) of the command. |
|
||||
| `data` | `Any` | `json.loads(output)`, or `None` if not valid JSON. |
|
||||
| `exit_code` | `int` | Process exit code. |
|
||||
| `prev_output` | `str|None` | Previous poll's stdout (`None` on first poll). |
|
||||
| `changed` | `bool` | `True` if output differs from previous poll. |
|
||||
|
||||
Safe builtins: `len`, `str`, `int`, `float`, `bool`, `abs`, `min`, `max`,
|
||||
`any`, `all`, `isinstance`, `sorted`. No `import`, `open`, `exec`, or
|
||||
`eval`. Security model: equivalent to `bash` — the model already has shell
|
||||
access.
|
||||
|
||||
**Examples:**
|
||||
```
|
||||
data["state"] == "MERGED"
|
||||
"error" in output
|
||||
exit_code != 0
|
||||
changed and "ready" in output.lower()
|
||||
data.get("mergedAt") is not None
|
||||
```
|
||||
|
||||
**Lifecycle:**
|
||||
|
||||
1. Model calls `watch(action="create", ...)` — persisted to SQLite.
|
||||
2. `WatchRunner` daemon polls for due watches every 15s.
|
||||
3. Each poll runs the command, evaluates the condition.
|
||||
4. When the condition fires (or max polls reached), the result is injected
|
||||
as a synthetic user message and the watch auto-cancels.
|
||||
5. If the workstream was evicted, it is restored before injection.
|
||||
6. Watches survive server restart (overdue watches fire once on recovery).
|
||||
|
||||
**Constraints:**
|
||||
|
||||
- Max 5 active watches per workstream.
|
||||
- Poll interval: 10s–24h.
|
||||
- Output truncated at 64 KB.
|
||||
- Max 5 consecutive watch dispatches per worker thread (depth guard).
|
||||
- Duplicate names rejected within the same workstream.
|
||||
|
||||
- **Auto-approve**: `create` requires approval; `list` and `cancel` are auto-approved.
|
||||
- **Agent availability**: Main session only — not available to plan/task sub-agents.
|
||||
|
||||
> See [Watch Architecture](diagrams/png/18-watch-architecture.png) for the
|
||||
> full poll → evaluate → dispatch flow.
|
||||
|
||||
---
|
||||
|
||||
## Summary Table
|
||||
|
||||
| Tool | Category | Auto-approve | agent | task_agent | primary_key |
|
||||
@@ -435,6 +516,78 @@ Provide either `username` for user-based targeting or `channel_type` +
|
||||
| `recall` | Memory | Yes | No | No | `query` |
|
||||
| `forget` | Memory | Yes | No | No | `key` |
|
||||
| `notify` | Notify | Yes | Yes | Yes | `message` |
|
||||
| `watch` | Monitor | No (create) | No | No | `command` |
|
||||
| `tool_search`| Search | Yes | No | No | `query` |
|
||||
|
||||
---
|
||||
|
||||
## Dynamic Tool Search
|
||||
|
||||
When many MCP tools are connected, the total tool count can grow large enough to
|
||||
consume significant context window tokens and reduce model accuracy. Dynamic tool
|
||||
search addresses this by deferring tools the model is unlikely to need on the
|
||||
current turn and letting it search for them on demand.
|
||||
|
||||
### Three-tier approach
|
||||
|
||||
Tool search uses the best available mechanism for each provider:
|
||||
|
||||
1. **Anthropic (native)** -- Models that support it receive `defer_loading: true`
|
||||
on deferred tool definitions plus the `tool_search_tool_bm25_20251119` server-side
|
||||
search tool. Anthropic's API handles search and expansion transparently.
|
||||
|
||||
2. **OpenAI GPT-5.4+ (native)** -- Models with hosted tool search receive
|
||||
`defer_loading: true` on deferred definitions. The API handles search internally.
|
||||
|
||||
3. **vLLM / llama.cpp / NIM (client-side BM25)** -- A synthetic `tool_search`
|
||||
function tool is injected into the tool list. When the model calls it,
|
||||
`_exec_tool_search()` runs a pure-Python BM25 index over tool names and
|
||||
descriptions, then expands the matched tools into the visible set.
|
||||
|
||||
### Configuration
|
||||
|
||||
Tool search is configured in `config.toml` under the `[tools]` section:
|
||||
|
||||
```toml
|
||||
[tools]
|
||||
search = "auto" # "auto", "on", or "off"
|
||||
search_threshold = 20 # minimum total tool count to activate
|
||||
search_max_results = 5 # max tools returned per search call
|
||||
```
|
||||
|
||||
CLI flags override the config file:
|
||||
|
||||
- `--tool-search {auto,on,off}` -- force tool search on or off, or let turnstone
|
||||
decide based on threshold (default: `auto`).
|
||||
- `--tool-search-threshold N` -- minimum tool count to activate (default: 20).
|
||||
- `--tool-search-max-results N` -- max results per search (default: 5).
|
||||
|
||||
### How it works
|
||||
|
||||
1. **Threshold check**: At session startup, `ToolSearchManager.should_activate()`
|
||||
counts total tools (built-in + MCP). If the count is below the threshold, tool
|
||||
search stays off and all tools are sent to the model directly.
|
||||
|
||||
2. **Partitioning**: When active, tools are split into two sets:
|
||||
- **Always-on** -- the 15 built-in tools (members of `BUILTIN_TOOL_NAMES`).
|
||||
These are always visible to the model.
|
||||
- **Deferred** -- all MCP tools. These are not sent in the tool list unless
|
||||
the model searches for them.
|
||||
|
||||
3. **Search and expand**: When the model calls `tool_search` (client-side) or the
|
||||
provider's native search returns results, the matched tools are added to the
|
||||
visible set via `expand_visible()`. Once expanded, a tool stays visible for
|
||||
the remainder of the session.
|
||||
|
||||
4. **Multi-turn persistence**: Expanded tools are never removed. This avoids
|
||||
confusing the model when it references a tool it discovered in an earlier turn.
|
||||
|
||||
### Agent exemption
|
||||
|
||||
Plan and task sub-agents do not use tool search. They operate on scoped tool
|
||||
sets (`AGENT_TOOLS` for plan agents, `TASK_AGENT_TOOLS` for task agents) with
|
||||
MCP tools merged in. Tool search is only active for the top-level session,
|
||||
where the model can interactively search for tools it needs.
|
||||
|
||||
---
|
||||
|
||||
@@ -451,13 +604,17 @@ MCP-compatible service.
|
||||
|
||||
2. **Discovery**: At startup, `MCPClientManager` connects to each configured server
|
||||
(via stdio subprocess or HTTP), performs the MCP `initialize` handshake, and calls
|
||||
`tools/list` to discover available tools.
|
||||
`tools/list` to discover available tools. During the handshake, the manager checks
|
||||
each server's capabilities for `tools.listChanged` support (push notifications).
|
||||
|
||||
3. **Schema conversion**: Each MCP tool's `inputSchema` is converted to OpenAI
|
||||
function-calling format. The tool name is prefixed: `mcp__{server}__{tool}`.
|
||||
|
||||
4. **Merging**: MCP tools are appended after the 14 built-in tools via
|
||||
4. **Merging**: MCP tools are appended after the 15 built-in tools via
|
||||
`merge_mcp_tools()`. Built-in tools appear first, giving them natural LLM priority.
|
||||
When dynamic tool search is active, MCP tools are deferred rather than directly
|
||||
visible -- the model discovers them via search as needed (see
|
||||
[Dynamic Tool Search](#dynamic-tool-search) above).
|
||||
|
||||
5. **Dispatch**: When the LLM calls an MCP tool, `_prepare_mcp_tool()` builds a
|
||||
generic approval preview and `_exec_mcp_tool()` calls `MCPClientManager.call_tool_sync()`,
|
||||
@@ -530,3 +687,45 @@ MCP tools (3):
|
||||
mcp__github__create_issue [MCP: github] Create a GitHub issue
|
||||
mcp__postgres__query [MCP: postgres] Run a SQL query
|
||||
```
|
||||
|
||||
### Dynamic tool refresh
|
||||
|
||||
MCP tool lists stay up-to-date without restart through three mechanisms:
|
||||
|
||||
1. **Push notifications** -- MCP servers that declare `tools.listChanged: true` in
|
||||
their capabilities send `notifications/tools/list_changed` when their tool list
|
||||
changes. `MCPClientManager` registers a `message_handler` on each `ClientSession`
|
||||
that triggers an immediate refresh for that server.
|
||||
|
||||
2. **Periodic timer** -- Servers that do *not* support push notifications are polled
|
||||
on a configurable interval (default 4 hours). The timer is staggered using a
|
||||
launch-time seed (`monotonic_ns ^ pid`) so cluster nodes don't all hit MCP
|
||||
servers simultaneously. Configure via `[mcp] refresh_interval` in `config.toml`
|
||||
or `--mcp-refresh-interval SECONDS` on the CLI. Set to `0` to disable.
|
||||
|
||||
3. **Manual** -- `/mcp refresh` re-fetches tools from all servers immediately.
|
||||
`/mcp refresh <server>` targets a single server. If a server has disconnected,
|
||||
manual refresh attempts reconnection.
|
||||
|
||||
When tools change, `MCPClientManager` rebuilds its merged tool list using copy-on-write
|
||||
(new list/dict objects assigned atomically) and notifies all active `ChatSession`
|
||||
instances via registered listener callbacks. Each session rebuilds its `_tools`,
|
||||
`_task_tools`, `_agent_tools`, and reconstructs its `ToolSearchManager` (if active),
|
||||
preserving the set of previously expanded (discovered) tools.
|
||||
|
||||
```toml
|
||||
[mcp]
|
||||
refresh_interval = 14400 # seconds (default 4h), 0 to disable
|
||||
```
|
||||
|
||||
```
|
||||
/mcp refresh
|
||||
MCP refresh complete:
|
||||
github: +1 added
|
||||
+ mcp__github__create_pr
|
||||
postgres: no changes
|
||||
|
||||
/mcp refresh github
|
||||
MCP refresh complete:
|
||||
github: no changes
|
||||
```
|
||||
|
||||
+1
-1
@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
|
||||
|
||||
[project]
|
||||
name = "turnstone"
|
||||
version = "0.4.2"
|
||||
version = "0.5.4"
|
||||
description = "Multi-node AI orchestration platform with tool use, agent routing, and cluster simulation."
|
||||
readme = "README.md"
|
||||
license = "BUSL-1.1"
|
||||
|
||||
+323
-175
@@ -2,7 +2,7 @@
|
||||
"openapi": "3.1.0",
|
||||
"info": {
|
||||
"title": "turnstone Server API",
|
||||
"version": "0.3.0",
|
||||
"version": "0.4.2",
|
||||
"description": "Single-node workstream management, chat interaction, and real-time streaming."
|
||||
},
|
||||
"paths": {
|
||||
@@ -10,9 +10,7 @@
|
||||
"get": {
|
||||
"summary": "List active workstreams",
|
||||
"operationId": "v1_api_workstreams_get",
|
||||
"tags": [
|
||||
"Workstreams"
|
||||
],
|
||||
"tags": ["Workstreams"],
|
||||
"responses": {
|
||||
"200": {
|
||||
"description": "Success",
|
||||
@@ -31,9 +29,7 @@
|
||||
"get": {
|
||||
"summary": "Dashboard with workstream details and aggregates",
|
||||
"operationId": "v1_api_dashboard_get",
|
||||
"tags": [
|
||||
"Workstreams"
|
||||
],
|
||||
"tags": ["Workstreams"],
|
||||
"responses": {
|
||||
"200": {
|
||||
"description": "Success",
|
||||
@@ -52,9 +48,7 @@
|
||||
"post": {
|
||||
"summary": "Create a new workstream",
|
||||
"operationId": "v1_api_workstreams_new_post",
|
||||
"tags": [
|
||||
"Workstreams"
|
||||
],
|
||||
"tags": ["Workstreams"],
|
||||
"requestBody": {
|
||||
"required": true,
|
||||
"content": {
|
||||
@@ -93,9 +87,7 @@
|
||||
"post": {
|
||||
"summary": "Close a workstream",
|
||||
"operationId": "v1_api_workstreams_close_post",
|
||||
"tags": [
|
||||
"Workstreams"
|
||||
],
|
||||
"tags": ["Workstreams"],
|
||||
"requestBody": {
|
||||
"required": true,
|
||||
"content": {
|
||||
@@ -134,9 +126,7 @@
|
||||
"post": {
|
||||
"summary": "Send a user message",
|
||||
"operationId": "v1_api_send_post",
|
||||
"tags": [
|
||||
"Chat"
|
||||
],
|
||||
"tags": ["Chat"],
|
||||
"requestBody": {
|
||||
"required": true,
|
||||
"content": {
|
||||
@@ -185,9 +175,7 @@
|
||||
"post": {
|
||||
"summary": "Approve or deny a tool call",
|
||||
"operationId": "v1_api_approve_post",
|
||||
"tags": [
|
||||
"Chat"
|
||||
],
|
||||
"tags": ["Chat"],
|
||||
"requestBody": {
|
||||
"required": true,
|
||||
"content": {
|
||||
@@ -226,9 +214,7 @@
|
||||
"post": {
|
||||
"summary": "Respond to a plan review",
|
||||
"operationId": "v1_api_plan_post",
|
||||
"tags": [
|
||||
"Chat"
|
||||
],
|
||||
"tags": ["Chat"],
|
||||
"requestBody": {
|
||||
"required": true,
|
||||
"content": {
|
||||
@@ -267,9 +253,7 @@
|
||||
"post": {
|
||||
"summary": "Execute a slash command",
|
||||
"operationId": "v1_api_command_post",
|
||||
"tags": [
|
||||
"Chat"
|
||||
],
|
||||
"tags": ["Chat"],
|
||||
"requestBody": {
|
||||
"required": true,
|
||||
"content": {
|
||||
@@ -314,13 +298,60 @@
|
||||
}
|
||||
}
|
||||
},
|
||||
"/v1/api/cancel": {
|
||||
"post": {
|
||||
"summary": "Cancel the active generation in a workstream",
|
||||
"operationId": "v1_api_cancel_post",
|
||||
"tags": ["Chat"],
|
||||
"requestBody": {
|
||||
"required": true,
|
||||
"content": {
|
||||
"application/json": {
|
||||
"schema": {
|
||||
"$ref": "#/components/schemas/CancelRequest"
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"responses": {
|
||||
"200": {
|
||||
"description": "Success",
|
||||
"content": {
|
||||
"application/json": {
|
||||
"schema": {
|
||||
"$ref": "#/components/schemas/StatusResponse"
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"400": {
|
||||
"description": "Error 400",
|
||||
"content": {
|
||||
"application/json": {
|
||||
"schema": {
|
||||
"$ref": "#/components/schemas/ErrorResponse"
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"404": {
|
||||
"description": "Error 404",
|
||||
"content": {
|
||||
"application/json": {
|
||||
"schema": {
|
||||
"$ref": "#/components/schemas/ErrorResponse"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"/v1/api/events": {
|
||||
"get": {
|
||||
"summary": "Per-workstream SSE event stream",
|
||||
"operationId": "v1_api_events_get",
|
||||
"tags": [
|
||||
"Streaming"
|
||||
],
|
||||
"tags": ["Streaming"],
|
||||
"description": "Opens a Server-Sent Events stream scoped to a single workstream. Returns text/event-stream. See API reference for event types.",
|
||||
"parameters": [
|
||||
{
|
||||
@@ -354,9 +385,7 @@
|
||||
"get": {
|
||||
"summary": "Global SSE event stream",
|
||||
"operationId": "v1_api_events_global_get",
|
||||
"tags": [
|
||||
"Streaming"
|
||||
],
|
||||
"tags": ["Streaming"],
|
||||
"description": "Global Server-Sent Events stream for state-change broadcasts across all workstreams. Returns text/event-stream.",
|
||||
"responses": {
|
||||
"200": {
|
||||
@@ -365,20 +394,18 @@
|
||||
}
|
||||
}
|
||||
},
|
||||
"/v1/api/sessions": {
|
||||
"/v1/api/workstreams/saved": {
|
||||
"get": {
|
||||
"summary": "List saved sessions",
|
||||
"operationId": "v1_api_sessions_get",
|
||||
"tags": [
|
||||
"Sessions"
|
||||
],
|
||||
"summary": "List saved workstreams",
|
||||
"operationId": "v1_api_workstreams_saved_get",
|
||||
"tags": ["Workstreams"],
|
||||
"responses": {
|
||||
"200": {
|
||||
"description": "Success",
|
||||
"content": {
|
||||
"application/json": {
|
||||
"schema": {
|
||||
"$ref": "#/components/schemas/ListSessionsResponse"
|
||||
"$ref": "#/components/schemas/ListSavedWorkstreamsResponse"
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -390,9 +417,7 @@
|
||||
"post": {
|
||||
"summary": "Authenticate with a token",
|
||||
"operationId": "v1_api_auth_login_post",
|
||||
"tags": [
|
||||
"Auth"
|
||||
],
|
||||
"tags": ["Auth"],
|
||||
"requestBody": {
|
||||
"required": true,
|
||||
"content": {
|
||||
@@ -427,13 +452,89 @@
|
||||
}
|
||||
}
|
||||
},
|
||||
"/v1/api/auth/setup": {
|
||||
"post": {
|
||||
"summary": "Create first admin user",
|
||||
"operationId": "v1_api_auth_setup_post",
|
||||
"tags": ["Auth"],
|
||||
"requestBody": {
|
||||
"required": true,
|
||||
"content": {
|
||||
"application/json": {
|
||||
"schema": {
|
||||
"$ref": "#/components/schemas/AuthSetupRequest"
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"responses": {
|
||||
"200": {
|
||||
"description": "Success",
|
||||
"content": {
|
||||
"application/json": {
|
||||
"schema": {
|
||||
"$ref": "#/components/schemas/AuthSetupResponse"
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"400": {
|
||||
"description": "Error 400",
|
||||
"content": {
|
||||
"application/json": {
|
||||
"schema": {
|
||||
"$ref": "#/components/schemas/ErrorResponse"
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"409": {
|
||||
"description": "Error 409",
|
||||
"content": {
|
||||
"application/json": {
|
||||
"schema": {
|
||||
"$ref": "#/components/schemas/ErrorResponse"
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"503": {
|
||||
"description": "Error 503",
|
||||
"content": {
|
||||
"application/json": {
|
||||
"schema": {
|
||||
"$ref": "#/components/schemas/ErrorResponse"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"/v1/api/auth/status": {
|
||||
"get": {
|
||||
"summary": "Return auth state",
|
||||
"operationId": "v1_api_auth_status_get",
|
||||
"tags": ["Auth"],
|
||||
"responses": {
|
||||
"200": {
|
||||
"description": "Success",
|
||||
"content": {
|
||||
"application/json": {
|
||||
"schema": {
|
||||
"$ref": "#/components/schemas/AuthStatusResponse"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"/v1/api/auth/logout": {
|
||||
"post": {
|
||||
"summary": "Clear auth cookie",
|
||||
"operationId": "v1_api_auth_logout_post",
|
||||
"tags": [
|
||||
"Auth"
|
||||
],
|
||||
"tags": ["Auth"],
|
||||
"responses": {
|
||||
"200": {
|
||||
"description": "Success",
|
||||
@@ -452,9 +553,7 @@
|
||||
"get": {
|
||||
"summary": "Server health check",
|
||||
"operationId": "health_get",
|
||||
"tags": [
|
||||
"Observability"
|
||||
],
|
||||
"tags": ["Observability"],
|
||||
"responses": {
|
||||
"200": {
|
||||
"description": "Success",
|
||||
@@ -481,9 +580,7 @@
|
||||
"type": "string"
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
"error"
|
||||
],
|
||||
"required": ["error"],
|
||||
"title": "ErrorResponse",
|
||||
"type": "object"
|
||||
},
|
||||
@@ -492,9 +589,7 @@
|
||||
"properties": {
|
||||
"status": {
|
||||
"default": "ok",
|
||||
"examples": [
|
||||
"ok"
|
||||
],
|
||||
"examples": ["ok"],
|
||||
"title": "Status",
|
||||
"type": "string"
|
||||
}
|
||||
@@ -503,17 +598,27 @@
|
||||
"type": "object"
|
||||
},
|
||||
"AuthLoginRequest": {
|
||||
"description": "POST /v1/api/auth/login request body.",
|
||||
"description": "POST /v1/api/auth/login request body.\n\nEither username+password or token must be provided.",
|
||||
"properties": {
|
||||
"username": {
|
||||
"default": "",
|
||||
"description": "Login username",
|
||||
"title": "Username",
|
||||
"type": "string"
|
||||
},
|
||||
"password": {
|
||||
"default": "",
|
||||
"description": "Login password",
|
||||
"title": "Password",
|
||||
"type": "string"
|
||||
},
|
||||
"token": {
|
||||
"description": "Bearer token to authenticate",
|
||||
"default": "",
|
||||
"description": "Legacy: bearer token to authenticate",
|
||||
"title": "Token",
|
||||
"type": "string"
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
"token"
|
||||
],
|
||||
"title": "AuthLoginRequest",
|
||||
"type": "object"
|
||||
},
|
||||
@@ -525,22 +630,116 @@
|
||||
"title": "Status",
|
||||
"type": "string"
|
||||
},
|
||||
"user_id": {
|
||||
"default": "",
|
||||
"description": "Authenticated user ID",
|
||||
"title": "User Id",
|
||||
"type": "string"
|
||||
},
|
||||
"role": {
|
||||
"description": "Assigned role",
|
||||
"examples": [
|
||||
"full",
|
||||
"read"
|
||||
],
|
||||
"description": "Legacy role",
|
||||
"examples": ["full", "read"],
|
||||
"title": "Role",
|
||||
"type": "string"
|
||||
},
|
||||
"scopes": {
|
||||
"default": "",
|
||||
"description": "Comma-separated scopes",
|
||||
"examples": ["read,write,approve"],
|
||||
"title": "Scopes",
|
||||
"type": "string"
|
||||
},
|
||||
"jwt": {
|
||||
"default": "",
|
||||
"description": "JWT session token (if JWT auth is configured)",
|
||||
"title": "Jwt",
|
||||
"type": "string"
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
"role"
|
||||
],
|
||||
"required": ["role"],
|
||||
"title": "AuthLoginResponse",
|
||||
"type": "object"
|
||||
},
|
||||
"AuthSetupRequest": {
|
||||
"description": "POST /v1/api/auth/setup request body.",
|
||||
"properties": {
|
||||
"username": {
|
||||
"description": "Login username (1-64 ASCII characters)",
|
||||
"title": "Username",
|
||||
"type": "string"
|
||||
},
|
||||
"display_name": {
|
||||
"description": "Display name",
|
||||
"title": "Display Name",
|
||||
"type": "string"
|
||||
},
|
||||
"password": {
|
||||
"description": "Password (minimum 8 characters)",
|
||||
"title": "Password",
|
||||
"type": "string"
|
||||
}
|
||||
},
|
||||
"required": ["username", "display_name", "password"],
|
||||
"title": "AuthSetupRequest",
|
||||
"type": "object"
|
||||
},
|
||||
"AuthSetupResponse": {
|
||||
"description": "POST /v1/api/auth/setup success response.",
|
||||
"properties": {
|
||||
"status": {
|
||||
"default": "ok",
|
||||
"title": "Status",
|
||||
"type": "string"
|
||||
},
|
||||
"user_id": {
|
||||
"title": "User Id",
|
||||
"type": "string"
|
||||
},
|
||||
"username": {
|
||||
"title": "Username",
|
||||
"type": "string"
|
||||
},
|
||||
"role": {
|
||||
"default": "full",
|
||||
"title": "Role",
|
||||
"type": "string"
|
||||
},
|
||||
"scopes": {
|
||||
"default": "approve,read,write",
|
||||
"title": "Scopes",
|
||||
"type": "string"
|
||||
},
|
||||
"jwt": {
|
||||
"default": "",
|
||||
"description": "JWT session token",
|
||||
"title": "Jwt",
|
||||
"type": "string"
|
||||
}
|
||||
},
|
||||
"required": ["user_id", "username"],
|
||||
"title": "AuthSetupResponse",
|
||||
"type": "object"
|
||||
},
|
||||
"AuthStatusResponse": {
|
||||
"description": "GET /v1/api/auth/status response.",
|
||||
"properties": {
|
||||
"auth_enabled": {
|
||||
"title": "Auth Enabled",
|
||||
"type": "boolean"
|
||||
},
|
||||
"has_users": {
|
||||
"title": "Has Users",
|
||||
"type": "boolean"
|
||||
},
|
||||
"setup_required": {
|
||||
"title": "Setup Required",
|
||||
"type": "boolean"
|
||||
}
|
||||
},
|
||||
"required": ["auth_enabled", "has_users", "setup_required"],
|
||||
"title": "AuthStatusResponse",
|
||||
"type": "object"
|
||||
},
|
||||
"SendRequest": {
|
||||
"properties": {
|
||||
"message": {
|
||||
@@ -554,10 +753,7 @@
|
||||
"type": "string"
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
"message",
|
||||
"ws_id"
|
||||
],
|
||||
"required": ["message", "ws_id"],
|
||||
"title": "SendRequest",
|
||||
"type": "object"
|
||||
},
|
||||
@@ -565,17 +761,12 @@
|
||||
"properties": {
|
||||
"status": {
|
||||
"description": "'ok' or 'busy'",
|
||||
"examples": [
|
||||
"ok",
|
||||
"busy"
|
||||
],
|
||||
"examples": ["ok", "busy"],
|
||||
"title": "Status",
|
||||
"type": "string"
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
"status"
|
||||
],
|
||||
"required": ["status"],
|
||||
"title": "SendResponse",
|
||||
"type": "object"
|
||||
},
|
||||
@@ -611,10 +802,7 @@
|
||||
"type": "string"
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
"approved",
|
||||
"ws_id"
|
||||
],
|
||||
"required": ["approved", "ws_id"],
|
||||
"title": "ApproveRequest",
|
||||
"type": "object"
|
||||
},
|
||||
@@ -631,10 +819,7 @@
|
||||
"type": "string"
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
"feedback",
|
||||
"ws_id"
|
||||
],
|
||||
"required": ["feedback", "ws_id"],
|
||||
"title": "PlanFeedbackRequest",
|
||||
"type": "object"
|
||||
},
|
||||
@@ -651,13 +836,22 @@
|
||||
"type": "string"
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
"command",
|
||||
"ws_id"
|
||||
],
|
||||
"required": ["command", "ws_id"],
|
||||
"title": "CommandRequest",
|
||||
"type": "object"
|
||||
},
|
||||
"CancelRequest": {
|
||||
"properties": {
|
||||
"ws_id": {
|
||||
"description": "Target workstream ID",
|
||||
"title": "Ws Id",
|
||||
"type": "string"
|
||||
}
|
||||
},
|
||||
"required": ["ws_id"],
|
||||
"title": "CancelRequest",
|
||||
"type": "object"
|
||||
},
|
||||
"CreateWorkstreamRequest": {
|
||||
"properties": {
|
||||
"name": {
|
||||
@@ -677,6 +871,12 @@
|
||||
"description": "Auto-approve all tool calls",
|
||||
"title": "Auto Approve",
|
||||
"type": "boolean"
|
||||
},
|
||||
"resume_ws": {
|
||||
"default": "",
|
||||
"description": "Workstream ID to resume atomically during creation (empty = fresh start)",
|
||||
"title": "Resume Ws",
|
||||
"type": "string"
|
||||
}
|
||||
},
|
||||
"title": "CreateWorkstreamRequest",
|
||||
@@ -693,12 +893,21 @@
|
||||
"description": "Assigned workstream name",
|
||||
"title": "Name",
|
||||
"type": "string"
|
||||
},
|
||||
"resumed": {
|
||||
"default": false,
|
||||
"description": "Whether a previous workstream was resumed",
|
||||
"title": "Resumed",
|
||||
"type": "boolean"
|
||||
},
|
||||
"message_count": {
|
||||
"default": 0,
|
||||
"description": "Number of messages in the resumed workstream",
|
||||
"title": "Message Count",
|
||||
"type": "integer"
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
"ws_id",
|
||||
"name"
|
||||
],
|
||||
"required": ["ws_id", "name"],
|
||||
"title": "CreateWorkstreamResponse",
|
||||
"type": "object"
|
||||
},
|
||||
@@ -710,9 +919,7 @@
|
||||
"type": "string"
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
"ws_id"
|
||||
],
|
||||
"required": ["ws_id"],
|
||||
"title": "CloseWorkstreamRequest",
|
||||
"type": "object"
|
||||
},
|
||||
@@ -726,9 +933,7 @@
|
||||
"type": "array"
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
"workstreams"
|
||||
],
|
||||
"required": ["workstreams"],
|
||||
"title": "ListWorkstreamsResponse",
|
||||
"type": "object"
|
||||
},
|
||||
@@ -745,25 +950,9 @@
|
||||
"state": {
|
||||
"title": "State",
|
||||
"type": "string"
|
||||
},
|
||||
"session_id": {
|
||||
"anyOf": [
|
||||
{
|
||||
"type": "string"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
],
|
||||
"default": null,
|
||||
"title": "Session Id"
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
"id",
|
||||
"name",
|
||||
"state"
|
||||
],
|
||||
"required": ["id", "name", "state"],
|
||||
"title": "WorkstreamInfo",
|
||||
"type": "object"
|
||||
},
|
||||
@@ -780,10 +969,7 @@
|
||||
"$ref": "#/components/schemas/DashboardAggregate"
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
"workstreams",
|
||||
"aggregate"
|
||||
],
|
||||
"required": ["workstreams", "aggregate"],
|
||||
"title": "DashboardResponse",
|
||||
"type": "object"
|
||||
},
|
||||
@@ -837,18 +1023,6 @@
|
||||
"title": "State",
|
||||
"type": "string"
|
||||
},
|
||||
"session_id": {
|
||||
"anyOf": [
|
||||
{
|
||||
"type": "string"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
],
|
||||
"default": null,
|
||||
"title": "Session Id"
|
||||
},
|
||||
"title": {
|
||||
"default": "",
|
||||
"title": "Title",
|
||||
@@ -895,34 +1069,28 @@
|
||||
"type": "string"
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
"id",
|
||||
"name",
|
||||
"state"
|
||||
],
|
||||
"required": ["id", "name", "state"],
|
||||
"title": "DashboardWorkstream",
|
||||
"type": "object"
|
||||
},
|
||||
"ListSessionsResponse": {
|
||||
"ListSavedWorkstreamsResponse": {
|
||||
"properties": {
|
||||
"sessions": {
|
||||
"workstreams": {
|
||||
"items": {
|
||||
"$ref": "#/components/schemas/SessionInfo"
|
||||
"$ref": "#/components/schemas/SavedWorkstreamInfo"
|
||||
},
|
||||
"title": "Sessions",
|
||||
"title": "Workstreams",
|
||||
"type": "array"
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
"sessions"
|
||||
],
|
||||
"title": "ListSessionsResponse",
|
||||
"required": ["workstreams"],
|
||||
"title": "ListSavedWorkstreamsResponse",
|
||||
"type": "object"
|
||||
},
|
||||
"SessionInfo": {
|
||||
"SavedWorkstreamInfo": {
|
||||
"properties": {
|
||||
"session_id": {
|
||||
"title": "Session Id",
|
||||
"ws_id": {
|
||||
"title": "Ws Id",
|
||||
"type": "string"
|
||||
},
|
||||
"alias": {
|
||||
@@ -962,22 +1130,14 @@
|
||||
"type": "integer"
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
"session_id",
|
||||
"created",
|
||||
"updated",
|
||||
"message_count"
|
||||
],
|
||||
"title": "SessionInfo",
|
||||
"required": ["ws_id", "created", "updated", "message_count"],
|
||||
"title": "SavedWorkstreamInfo",
|
||||
"type": "object"
|
||||
},
|
||||
"HealthResponse": {
|
||||
"properties": {
|
||||
"status": {
|
||||
"examples": [
|
||||
"ok",
|
||||
"degraded"
|
||||
],
|
||||
"examples": ["ok", "degraded"],
|
||||
"title": "Status",
|
||||
"type": "string"
|
||||
},
|
||||
@@ -1019,36 +1179,24 @@
|
||||
"default": null
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
"status"
|
||||
],
|
||||
"required": ["status"],
|
||||
"title": "HealthResponse",
|
||||
"type": "object"
|
||||
},
|
||||
"BackendStatus": {
|
||||
"properties": {
|
||||
"status": {
|
||||
"examples": [
|
||||
"up",
|
||||
"down"
|
||||
],
|
||||
"examples": ["up", "down"],
|
||||
"title": "Status",
|
||||
"type": "string"
|
||||
},
|
||||
"circuit_state": {
|
||||
"examples": [
|
||||
"closed",
|
||||
"open",
|
||||
"half_open"
|
||||
],
|
||||
"examples": ["closed", "open", "half_open"],
|
||||
"title": "Circuit State",
|
||||
"type": "string"
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
"status",
|
||||
"circuit_state"
|
||||
],
|
||||
"required": ["status", "circuit_state"],
|
||||
"title": "BackendStatus",
|
||||
"type": "object"
|
||||
},
|
||||
|
||||
@@ -1,23 +1,40 @@
|
||||
import { BaseClient, type ClientOptions } from "./base.js";
|
||||
import type { ClusterEvent } from "./events.js";
|
||||
import type {
|
||||
AuditQueryOptions,
|
||||
AuditResponse,
|
||||
AuthLoginResponse,
|
||||
AuthSetupResponse,
|
||||
AuthStatusResponse,
|
||||
ClusterNodesResponse,
|
||||
ClusterOverviewResponse,
|
||||
ClusterSnapshotResponse,
|
||||
ClusterWorkstreamsResponse,
|
||||
ConsoleCreateWsRequest,
|
||||
ConsoleCreateWsResponse,
|
||||
ConsoleHealthResponse,
|
||||
CreatePolicyOptions,
|
||||
CreateRoleOptions,
|
||||
CreateScheduleRequest,
|
||||
CreateTemplateOptions,
|
||||
ListScheduleRunsResponse,
|
||||
ListSchedulesResponse,
|
||||
NodeDetailResponse,
|
||||
NodesOptions,
|
||||
OrgInfo,
|
||||
PromptTemplateInfo,
|
||||
RoleInfo,
|
||||
ScheduleInfo,
|
||||
StatusResponse,
|
||||
ToolPolicyInfo,
|
||||
UpdateOrgOptions,
|
||||
UpdatePolicyOptions,
|
||||
UpdateRoleOptions,
|
||||
UpdateScheduleRequest,
|
||||
UpdateTemplateOptions,
|
||||
UsageQueryOptions,
|
||||
UsageResponse,
|
||||
UserRoleInfo,
|
||||
WorkstreamsOptions,
|
||||
} from "./types.js";
|
||||
|
||||
@@ -33,6 +50,10 @@ export class TurnstoneConsole extends BaseClient {
|
||||
return this.request("GET", "/v1/api/cluster/overview");
|
||||
}
|
||||
|
||||
async snapshot(): Promise<ClusterSnapshotResponse> {
|
||||
return this.request("GET", "/v1/api/cluster/snapshot");
|
||||
}
|
||||
|
||||
async nodes(opts?: NodesOptions): Promise<ClusterNodesResponse> {
|
||||
return this.request("GET", "/v1/api/cluster/nodes", {
|
||||
params: {
|
||||
@@ -152,4 +173,125 @@ export class TurnstoneConsole extends BaseClient {
|
||||
params: { limit: opts?.limit ?? 50 },
|
||||
});
|
||||
}
|
||||
|
||||
// -- Governance: Roles ------------------------------------------------------
|
||||
|
||||
async listRoles(): Promise<{ roles: RoleInfo[] }> {
|
||||
return this.request("GET", "/v1/api/admin/roles");
|
||||
}
|
||||
|
||||
async createRole(opts: CreateRoleOptions): Promise<RoleInfo> {
|
||||
return this.request("POST", "/v1/api/admin/roles", { json: opts });
|
||||
}
|
||||
|
||||
async updateRole(roleId: string, opts: UpdateRoleOptions): Promise<RoleInfo> {
|
||||
return this.request("PUT", `/v1/api/admin/roles/${roleId}`, {
|
||||
json: opts,
|
||||
});
|
||||
}
|
||||
|
||||
async deleteRole(roleId: string): Promise<StatusResponse> {
|
||||
return this.request("DELETE", `/v1/api/admin/roles/${roleId}`);
|
||||
}
|
||||
|
||||
async listUserRoles(userId: string): Promise<{ roles: UserRoleInfo[] }> {
|
||||
return this.request("GET", `/v1/api/admin/users/${userId}/roles`);
|
||||
}
|
||||
|
||||
async assignRole(userId: string, roleId: string): Promise<StatusResponse> {
|
||||
return this.request("POST", `/v1/api/admin/users/${userId}/roles`, {
|
||||
json: { role_id: roleId },
|
||||
});
|
||||
}
|
||||
|
||||
async unassignRole(userId: string, roleId: string): Promise<StatusResponse> {
|
||||
return this.request(
|
||||
"DELETE",
|
||||
`/v1/api/admin/users/${userId}/roles/${roleId}`,
|
||||
);
|
||||
}
|
||||
|
||||
// -- Governance: Organizations ----------------------------------------------
|
||||
|
||||
async listOrgs(): Promise<{ orgs: OrgInfo[] }> {
|
||||
return this.request("GET", "/v1/api/admin/orgs");
|
||||
}
|
||||
|
||||
async getOrg(orgId: string): Promise<OrgInfo> {
|
||||
return this.request("GET", `/v1/api/admin/orgs/${orgId}`);
|
||||
}
|
||||
|
||||
async updateOrg(orgId: string, opts: UpdateOrgOptions): Promise<OrgInfo> {
|
||||
return this.request("PUT", `/v1/api/admin/orgs/${orgId}`, { json: opts });
|
||||
}
|
||||
|
||||
// -- Governance: Tool Policies ----------------------------------------------
|
||||
|
||||
async listPolicies(): Promise<{ policies: ToolPolicyInfo[] }> {
|
||||
return this.request("GET", "/v1/api/admin/policies");
|
||||
}
|
||||
|
||||
async createPolicy(opts: CreatePolicyOptions): Promise<ToolPolicyInfo> {
|
||||
return this.request("POST", "/v1/api/admin/policies", { json: opts });
|
||||
}
|
||||
|
||||
async updatePolicy(
|
||||
policyId: string,
|
||||
opts: UpdatePolicyOptions,
|
||||
): Promise<ToolPolicyInfo> {
|
||||
return this.request("PUT", `/v1/api/admin/policies/${policyId}`, {
|
||||
json: opts,
|
||||
});
|
||||
}
|
||||
|
||||
async deletePolicy(policyId: string): Promise<StatusResponse> {
|
||||
return this.request("DELETE", `/v1/api/admin/policies/${policyId}`);
|
||||
}
|
||||
|
||||
// -- Governance: Prompt Templates -------------------------------------------
|
||||
|
||||
async listTemplates(): Promise<{ templates: PromptTemplateInfo[] }> {
|
||||
return this.request("GET", "/v1/api/admin/templates");
|
||||
}
|
||||
|
||||
async createTemplate(
|
||||
opts: CreateTemplateOptions,
|
||||
): Promise<PromptTemplateInfo> {
|
||||
return this.request("POST", "/v1/api/admin/templates", { json: opts });
|
||||
}
|
||||
|
||||
async updateTemplate(
|
||||
templateId: string,
|
||||
opts: UpdateTemplateOptions,
|
||||
): Promise<PromptTemplateInfo> {
|
||||
return this.request("PUT", `/v1/api/admin/templates/${templateId}`, {
|
||||
json: opts,
|
||||
});
|
||||
}
|
||||
|
||||
async deleteTemplate(templateId: string): Promise<StatusResponse> {
|
||||
return this.request("DELETE", `/v1/api/admin/templates/${templateId}`);
|
||||
}
|
||||
|
||||
// -- Governance: Usage & Audit ----------------------------------------------
|
||||
|
||||
async getUsage(opts: UsageQueryOptions): Promise<UsageResponse> {
|
||||
const params: Record<string, string> = { since: opts.since };
|
||||
if (opts.until) params.until = opts.until;
|
||||
if (opts.user_id) params.user_id = opts.user_id;
|
||||
if (opts.model) params.model = opts.model;
|
||||
if (opts.group_by) params.group_by = opts.group_by;
|
||||
return this.request("GET", "/v1/api/admin/usage", { params });
|
||||
}
|
||||
|
||||
async getAudit(opts?: AuditQueryOptions): Promise<AuditResponse> {
|
||||
const params: Record<string, string> = {};
|
||||
if (opts?.action) params.action = opts.action;
|
||||
if (opts?.user_id) params.user_id = opts.user_id;
|
||||
if (opts?.since) params.since = opts.since;
|
||||
if (opts?.until) params.until = opts.until;
|
||||
if (opts?.limit !== undefined) params.limit = String(opts.limit);
|
||||
if (opts?.offset !== undefined) params.offset = String(opts.offset);
|
||||
return this.request("GET", "/v1/api/admin/audit", { params });
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,3 +1,5 @@
|
||||
import type { ClusterOverviewResponse, ClusterSnapshotNode } from "./types.js";
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Server SSE events
|
||||
// ---------------------------------------------------------------------------
|
||||
@@ -93,6 +95,10 @@ export interface ClearUiEvent {
|
||||
type: "clear_ui";
|
||||
}
|
||||
|
||||
export interface CancelledEvent {
|
||||
type: "cancelled";
|
||||
}
|
||||
|
||||
// Global events
|
||||
|
||||
export interface WsStateEvent {
|
||||
@@ -143,6 +149,7 @@ export type ServerEvent =
|
||||
| ErrorEvent
|
||||
| BusyErrorEvent
|
||||
| ClearUiEvent
|
||||
| CancelledEvent
|
||||
| WsStateEvent
|
||||
| WsActivityEvent
|
||||
| WsRenameEvent
|
||||
@@ -191,6 +198,13 @@ export interface ClusterWsRenameEvent {
|
||||
name: string;
|
||||
}
|
||||
|
||||
export interface ClusterSnapshotEvent {
|
||||
type: "snapshot";
|
||||
nodes: ClusterSnapshotNode[];
|
||||
overview: ClusterOverviewResponse;
|
||||
timestamp: number;
|
||||
}
|
||||
|
||||
/** Discriminated union of all console cluster SSE event types. */
|
||||
export type ClusterEvent =
|
||||
| NodeJoinedEvent
|
||||
@@ -198,7 +212,8 @@ export type ClusterEvent =
|
||||
| ClusterStateEvent
|
||||
| ClusterWsCreatedEvent
|
||||
| ClusterWsClosedEvent
|
||||
| ClusterWsRenameEvent;
|
||||
| ClusterWsRenameEvent
|
||||
| ClusterSnapshotEvent;
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Type guards
|
||||
@@ -237,3 +252,7 @@ export function isApproveRequestEvent(
|
||||
export function isPlanReviewEvent(e: ServerEvent): e is PlanReviewEvent {
|
||||
return e.type === "plan_review";
|
||||
}
|
||||
|
||||
export function isCancelledEvent(e: ServerEvent): e is CancelledEvent {
|
||||
return e.type === "cancelled";
|
||||
}
|
||||
|
||||
@@ -45,6 +45,7 @@ export type {
|
||||
ErrorEvent,
|
||||
BusyErrorEvent,
|
||||
ClearUiEvent,
|
||||
CancelledEvent,
|
||||
WsStateEvent,
|
||||
WsActivityEvent,
|
||||
WsRenameEvent,
|
||||
@@ -55,6 +56,7 @@ export type {
|
||||
ClusterWsCreatedEvent,
|
||||
ClusterWsClosedEvent,
|
||||
ClusterWsRenameEvent,
|
||||
ClusterSnapshotEvent,
|
||||
} from "./events.js";
|
||||
|
||||
export {
|
||||
@@ -66,6 +68,7 @@ export {
|
||||
isWsStateEvent,
|
||||
isApproveRequestEvent,
|
||||
isPlanReviewEvent,
|
||||
isCancelledEvent,
|
||||
} from "./events.js";
|
||||
|
||||
// Request/response types
|
||||
@@ -83,8 +86,8 @@ export type {
|
||||
DashboardWorkstream,
|
||||
DashboardAggregate,
|
||||
DashboardResponse,
|
||||
SessionInfo,
|
||||
ListSessionsResponse,
|
||||
SavedWorkstreamInfo,
|
||||
ListSavedWorkstreamsResponse,
|
||||
BackendStatus,
|
||||
WorkstreamCounts,
|
||||
HealthResponse,
|
||||
@@ -97,6 +100,8 @@ export type {
|
||||
ClusterOverviewResponse,
|
||||
ClusterNodeInfo,
|
||||
ClusterNodesResponse,
|
||||
ClusterSnapshotNode,
|
||||
ClusterSnapshotResponse,
|
||||
ClusterWorkstreamInfo,
|
||||
ClusterWorkstreamsResponse,
|
||||
NodeDetailResponse,
|
||||
@@ -109,6 +114,24 @@ export type {
|
||||
ScheduleRunInfo,
|
||||
ListSchedulesResponse,
|
||||
ListScheduleRunsResponse,
|
||||
RoleInfo,
|
||||
CreateRoleOptions,
|
||||
UpdateRoleOptions,
|
||||
UserRoleInfo,
|
||||
OrgInfo,
|
||||
UpdateOrgOptions,
|
||||
ToolPolicyInfo,
|
||||
CreatePolicyOptions,
|
||||
UpdatePolicyOptions,
|
||||
PromptTemplateInfo,
|
||||
CreateTemplateOptions,
|
||||
UpdateTemplateOptions,
|
||||
UsageBreakdownItem,
|
||||
UsageResponse,
|
||||
UsageQueryOptions,
|
||||
AuditEventInfo,
|
||||
AuditQueryOptions,
|
||||
AuditResponse,
|
||||
TurnResult,
|
||||
SendAndWaitOptions,
|
||||
NodesOptions,
|
||||
|
||||
@@ -8,7 +8,7 @@ import type {
|
||||
CreateWorkstreamResponse,
|
||||
DashboardResponse,
|
||||
HealthResponse,
|
||||
ListSessionsResponse,
|
||||
ListSavedWorkstreamsResponse,
|
||||
ListWorkstreamsResponse,
|
||||
SendAndWaitOptions,
|
||||
SendResponse,
|
||||
@@ -86,6 +86,12 @@ export class TurnstoneServer extends BaseClient {
|
||||
});
|
||||
}
|
||||
|
||||
async cancel(wsId: string): Promise<StatusResponse> {
|
||||
return this.request("POST", "/v1/api/cancel", {
|
||||
json: { ws_id: wsId },
|
||||
});
|
||||
}
|
||||
|
||||
// -- Streaming ------------------------------------------------------------
|
||||
|
||||
async *streamEvents(wsId: string): AsyncIterableIterator<ServerEvent> {
|
||||
@@ -178,10 +184,10 @@ export class TurnstoneServer extends BaseClient {
|
||||
return result;
|
||||
}
|
||||
|
||||
// -- Sessions -------------------------------------------------------------
|
||||
// -- Saved workstreams ----------------------------------------------------
|
||||
|
||||
async listSessions(): Promise<ListSessionsResponse> {
|
||||
return this.request("GET", "/v1/api/sessions");
|
||||
async listSavedWorkstreams(): Promise<ListSavedWorkstreamsResponse> {
|
||||
return this.request("GET", "/v1/api/workstreams/saved");
|
||||
}
|
||||
|
||||
// -- Auth -----------------------------------------------------------------
|
||||
|
||||
+190
-9
@@ -71,14 +71,13 @@ export interface CreateWorkstreamRequest {
|
||||
name?: string;
|
||||
model?: string;
|
||||
auto_approve?: boolean;
|
||||
resume_session?: string;
|
||||
resume_ws?: string;
|
||||
}
|
||||
|
||||
export interface CreateWorkstreamResponse {
|
||||
ws_id: string;
|
||||
name: string;
|
||||
resumed?: boolean;
|
||||
session_id?: string;
|
||||
message_count?: number;
|
||||
}
|
||||
|
||||
@@ -90,7 +89,6 @@ export interface WorkstreamInfo {
|
||||
id: string;
|
||||
name: string;
|
||||
state: string;
|
||||
session_id?: string | null;
|
||||
}
|
||||
|
||||
export interface ListWorkstreamsResponse {
|
||||
@@ -101,7 +99,6 @@ export interface DashboardWorkstream {
|
||||
id: string;
|
||||
name: string;
|
||||
state: string;
|
||||
session_id?: string | null;
|
||||
title?: string;
|
||||
tokens?: number;
|
||||
context_ratio?: number;
|
||||
@@ -128,11 +125,11 @@ export interface DashboardResponse {
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Server API — Sessions
|
||||
// Server API — Saved workstreams
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export interface SessionInfo {
|
||||
session_id: string;
|
||||
export interface SavedWorkstreamInfo {
|
||||
ws_id: string;
|
||||
alias?: string | null;
|
||||
title?: string | null;
|
||||
created: string;
|
||||
@@ -140,8 +137,8 @@ export interface SessionInfo {
|
||||
message_count: number;
|
||||
}
|
||||
|
||||
export interface ListSessionsResponse {
|
||||
sessions: SessionInfo[];
|
||||
export interface ListSavedWorkstreamsResponse {
|
||||
workstreams: SavedWorkstreamInfo[];
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
@@ -247,6 +244,23 @@ export interface NodeDetailResponse {
|
||||
aggregate: ClusterAggregate;
|
||||
}
|
||||
|
||||
export interface ClusterSnapshotNode {
|
||||
node_id: string;
|
||||
server_url: string;
|
||||
max_ws: number;
|
||||
reachable: boolean;
|
||||
version: string;
|
||||
health: Record<string, string>;
|
||||
aggregate: Record<string, number>;
|
||||
workstreams: ClusterWorkstreamInfo[];
|
||||
}
|
||||
|
||||
export interface ClusterSnapshotResponse {
|
||||
nodes: ClusterSnapshotNode[];
|
||||
overview: ClusterOverviewResponse;
|
||||
timestamp: number;
|
||||
}
|
||||
|
||||
export interface ConsoleCreateWsRequest {
|
||||
node_id?: string;
|
||||
name?: string;
|
||||
@@ -340,6 +354,173 @@ export interface ListScheduleRunsResponse {
|
||||
runs: ScheduleRunInfo[];
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Console API — Governance: Roles
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export interface RoleInfo {
|
||||
role_id: string;
|
||||
name: string;
|
||||
display_name: string;
|
||||
permissions: string;
|
||||
builtin: boolean;
|
||||
org_id: string;
|
||||
created: string;
|
||||
updated: string;
|
||||
}
|
||||
|
||||
export interface CreateRoleOptions {
|
||||
name: string;
|
||||
display_name?: string;
|
||||
permissions?: string;
|
||||
}
|
||||
|
||||
export interface UpdateRoleOptions {
|
||||
display_name?: string;
|
||||
permissions?: string;
|
||||
}
|
||||
|
||||
export interface UserRoleInfo extends RoleInfo {
|
||||
assigned_by: string;
|
||||
assignment_created: string;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Console API — Governance: Orgs
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export interface OrgInfo {
|
||||
org_id: string;
|
||||
name: string;
|
||||
display_name: string;
|
||||
settings: string;
|
||||
created: string;
|
||||
updated: string;
|
||||
}
|
||||
|
||||
export interface UpdateOrgOptions {
|
||||
display_name?: string;
|
||||
settings?: string;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Console API — Governance: Tool Policies
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export interface ToolPolicyInfo {
|
||||
policy_id: string;
|
||||
name: string;
|
||||
tool_pattern: string;
|
||||
action: string;
|
||||
priority: number;
|
||||
org_id: string;
|
||||
enabled: boolean;
|
||||
created_by: string;
|
||||
created: string;
|
||||
updated: string;
|
||||
}
|
||||
|
||||
export interface CreatePolicyOptions {
|
||||
name: string;
|
||||
tool_pattern: string;
|
||||
action: string;
|
||||
priority?: number;
|
||||
org_id?: string;
|
||||
enabled?: boolean;
|
||||
}
|
||||
|
||||
export interface UpdatePolicyOptions {
|
||||
name?: string;
|
||||
tool_pattern?: string;
|
||||
action?: string;
|
||||
priority?: number;
|
||||
enabled?: boolean;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Console API — Governance: Prompt Templates
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export interface PromptTemplateInfo {
|
||||
template_id: string;
|
||||
name: string;
|
||||
category: string;
|
||||
content: string;
|
||||
variables: string;
|
||||
is_default: boolean;
|
||||
org_id: string;
|
||||
created_by: string;
|
||||
created: string;
|
||||
updated: string;
|
||||
}
|
||||
|
||||
export interface CreateTemplateOptions {
|
||||
name: string;
|
||||
content: string;
|
||||
category?: string;
|
||||
variables?: string;
|
||||
is_default?: boolean;
|
||||
org_id?: string;
|
||||
}
|
||||
|
||||
export interface UpdateTemplateOptions {
|
||||
name?: string;
|
||||
content?: string;
|
||||
category?: string;
|
||||
variables?: string;
|
||||
is_default?: boolean;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Console API — Governance: Usage & Audit
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export interface UsageBreakdownItem {
|
||||
key?: string;
|
||||
prompt_tokens: number;
|
||||
completion_tokens: number;
|
||||
tool_calls_count: number;
|
||||
}
|
||||
|
||||
export interface UsageResponse {
|
||||
summary: UsageBreakdownItem[];
|
||||
breakdown: UsageBreakdownItem[];
|
||||
}
|
||||
|
||||
export interface UsageQueryOptions {
|
||||
since: string;
|
||||
until?: string;
|
||||
user_id?: string;
|
||||
model?: string;
|
||||
group_by?: string;
|
||||
}
|
||||
|
||||
export interface AuditEventInfo {
|
||||
event_id: string;
|
||||
timestamp: string;
|
||||
user_id: string;
|
||||
action: string;
|
||||
resource_type: string;
|
||||
resource_id: string;
|
||||
detail: string;
|
||||
ip_address: string;
|
||||
created: string;
|
||||
}
|
||||
|
||||
export interface AuditQueryOptions {
|
||||
action?: string;
|
||||
user_id?: string;
|
||||
since?: string;
|
||||
until?: string;
|
||||
limit?: number;
|
||||
offset?: number;
|
||||
}
|
||||
|
||||
export interface AuditResponse {
|
||||
events: AuditEventInfo[];
|
||||
total: number;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// SDK-specific types
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
+20
-2
@@ -59,7 +59,7 @@
|
||||
"user_prompt": "Change the default port from 8000 to 9000 in both server.py and config.py",
|
||||
"setup": {
|
||||
"files": {
|
||||
"server.py": "from config import PORT\n\ndef run():\n print(f'Listening on port {PORT}')\n",
|
||||
"server.py": "import socket\n\ndef run():\n sock = socket.socket()\n sock.bind(('localhost', 8000))\n print('Server running on port 8000')\n",
|
||||
"config.py": "PORT = 8000\nHOST = 'localhost'\n"
|
||||
}
|
||||
},
|
||||
@@ -126,7 +126,7 @@
|
||||
"app.py": "import sqlite3\nfrom flask import Flask, jsonify\n\napp = Flask(__name__)\nDB = 'data.db'\n\ndef get_db():\n return sqlite3.connect(DB)\n\n@app.route('/users')\ndef list_users():\n db = get_db()\n users = db.execute('SELECT * FROM users').fetchall()\n db.close()\n return jsonify(users)\n\n@app.route('/users/<int:uid>')\ndef get_user(uid):\n db = get_db()\n user = db.execute('SELECT * FROM users WHERE id=?', (uid,)).fetchone()\n db.close()\n return jsonify(user)\n\nif __name__ == '__main__':\n app.run(port=8000)\n"
|
||||
}
|
||||
},
|
||||
"expected_actions": [{ "tool": "plan" }],
|
||||
"expected_actions": [{ "tool": "create_plan" }],
|
||||
"match_mode": "subset"
|
||||
},
|
||||
{
|
||||
@@ -175,6 +175,24 @@
|
||||
{ "tool": "man", "args_pattern": { "page": "tar" } }
|
||||
],
|
||||
"match_mode": "subset"
|
||||
},
|
||||
{
|
||||
"id": "math-calculation",
|
||||
"description": "Use the math tool for precise calculations, not bash or mental math",
|
||||
"user_prompt": "What is 2^64 - 1? Use the math tool to calculate it precisely.",
|
||||
"expected_actions": [
|
||||
{ "tool": "math", "args_pattern": { "code": "2.*64" } }
|
||||
],
|
||||
"match_mode": "subset"
|
||||
},
|
||||
{
|
||||
"id": "web-search-query",
|
||||
"description": "Use web_search for general knowledge lookups, not web_fetch",
|
||||
"user_prompt": "Search the web for the current population of Tokyo",
|
||||
"expected_actions": [
|
||||
{ "tool": "web_search", "args_pattern": { "query": "Tokyo" } }
|
||||
],
|
||||
"match_mode": "subset"
|
||||
}
|
||||
]
|
||||
}
|
||||
|
||||
@@ -0,0 +1,58 @@
|
||||
"""Tests for turnstone.core.audit."""
|
||||
|
||||
import json
|
||||
|
||||
import pytest
|
||||
|
||||
from turnstone.core.audit import record_audit
|
||||
from turnstone.core.storage._sqlite import SQLiteBackend
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def storage(tmp_path):
|
||||
path = str(tmp_path / "test.db")
|
||||
backend = SQLiteBackend(path)
|
||||
yield backend
|
||||
backend.close()
|
||||
|
||||
|
||||
def test_record_audit_basic(storage):
|
||||
record_audit(
|
||||
storage, "user-1", "user.create", "user", "u123", {"username": "alice"}, "127.0.0.1"
|
||||
)
|
||||
events = storage.list_audit_events()
|
||||
assert len(events) == 1
|
||||
ev = events[0]
|
||||
assert ev["user_id"] == "user-1"
|
||||
assert ev["action"] == "user.create"
|
||||
assert ev["resource_type"] == "user"
|
||||
assert ev["resource_id"] == "u123"
|
||||
assert ev["ip_address"] == "127.0.0.1"
|
||||
detail = json.loads(ev["detail"])
|
||||
assert detail["username"] == "alice"
|
||||
|
||||
|
||||
def test_record_audit_no_detail(storage):
|
||||
record_audit(storage, "user-1", "token.revoke", "token", "t456")
|
||||
events = storage.list_audit_events()
|
||||
assert len(events) == 1
|
||||
assert events[0]["detail"] == "{}"
|
||||
|
||||
|
||||
def test_record_audit_silent_on_failure():
|
||||
"""record_audit should not raise even if storage is broken."""
|
||||
|
||||
class BrokenStorage:
|
||||
def record_audit_event(self, **kw):
|
||||
raise RuntimeError("boom")
|
||||
|
||||
# Should not raise
|
||||
record_audit(BrokenStorage(), "u1", "test.action")
|
||||
|
||||
|
||||
def test_record_audit_generates_unique_ids(storage):
|
||||
record_audit(storage, "u1", "a.one")
|
||||
record_audit(storage, "u1", "a.two")
|
||||
events = storage.list_audit_events()
|
||||
assert len(events) == 2
|
||||
assert events[0]["event_id"] != events[1]["event_id"]
|
||||
+2
-2
@@ -719,7 +719,7 @@ class TestServerAuth:
|
||||
srv_mod._metrics.model = "test-model"
|
||||
|
||||
mock_session = MagicMock()
|
||||
mock_session.session_id = "test-session-id"
|
||||
mock_session.ws_id = "test-session-id"
|
||||
|
||||
mock_ws = MagicMock()
|
||||
mock_ws.id = "test-ws"
|
||||
@@ -937,7 +937,7 @@ class TestServerLogin:
|
||||
srv_mod._metrics.model = "test-model"
|
||||
|
||||
mock_session = MagicMock()
|
||||
mock_session.session_id = "test-session-id"
|
||||
mock_session.ws_id = "test-session-id"
|
||||
|
||||
mock_ws = MagicMock()
|
||||
mock_ws.id = "test-ws"
|
||||
|
||||
@@ -0,0 +1,337 @@
|
||||
"""Tests for generation cancellation (cooperative cancel via threading.Event)."""
|
||||
|
||||
import threading
|
||||
import time
|
||||
from dataclasses import dataclass, field
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
import pytest
|
||||
|
||||
from turnstone.core.session import ChatSession, GenerationCancelled
|
||||
|
||||
|
||||
class NullUI:
|
||||
"""UI adapter that records state changes and discards other output."""
|
||||
|
||||
def __init__(self):
|
||||
self.states = []
|
||||
self.infos = []
|
||||
self.stream_ends = 0
|
||||
|
||||
def on_thinking_start(self):
|
||||
pass
|
||||
|
||||
def on_thinking_stop(self):
|
||||
pass
|
||||
|
||||
def on_reasoning_token(self, text):
|
||||
pass
|
||||
|
||||
def on_content_token(self, text):
|
||||
pass
|
||||
|
||||
def on_stream_end(self):
|
||||
self.stream_ends += 1
|
||||
|
||||
def approve_tools(self, items):
|
||||
return True, None
|
||||
|
||||
def on_tool_result(self, call_id, name, output):
|
||||
pass
|
||||
|
||||
def on_tool_output_chunk(self, call_id, chunk):
|
||||
pass
|
||||
|
||||
def on_status(self, usage, context_window, effort):
|
||||
pass
|
||||
|
||||
def on_plan_review(self, content):
|
||||
return ""
|
||||
|
||||
def on_info(self, message):
|
||||
self.infos.append(message)
|
||||
|
||||
def on_error(self, message):
|
||||
pass
|
||||
|
||||
def on_state_change(self, state):
|
||||
self.states.append(state)
|
||||
|
||||
def on_rename(self, name):
|
||||
pass
|
||||
|
||||
|
||||
def _make_session(ui=None, **kwargs):
|
||||
"""Helper to construct a ChatSession with minimal setup."""
|
||||
defaults = dict(
|
||||
client=MagicMock(),
|
||||
model="test-model",
|
||||
ui=ui or NullUI(),
|
||||
instructions=None,
|
||||
temperature=0.5,
|
||||
max_tokens=4096,
|
||||
tool_timeout=30,
|
||||
)
|
||||
defaults.update(kwargs)
|
||||
return ChatSession(**defaults)
|
||||
|
||||
|
||||
class TestCancelEvent:
|
||||
"""Basic cancel event mechanics."""
|
||||
|
||||
def test_cancel_sets_event(self, tmp_db):
|
||||
session = _make_session()
|
||||
assert not session._cancel_event.is_set()
|
||||
session.cancel()
|
||||
assert session._cancel_event.is_set()
|
||||
|
||||
def test_check_cancelled_raises_when_set(self, tmp_db):
|
||||
session = _make_session()
|
||||
session.cancel()
|
||||
with pytest.raises(GenerationCancelled):
|
||||
session._check_cancelled()
|
||||
|
||||
def test_check_cancelled_noop_when_clear(self, tmp_db):
|
||||
session = _make_session()
|
||||
session._check_cancelled() # Should not raise
|
||||
|
||||
def test_cancel_is_idempotent(self, tmp_db):
|
||||
session = _make_session()
|
||||
session.cancel()
|
||||
session.cancel() # Double call is harmless
|
||||
assert session._cancel_event.is_set()
|
||||
|
||||
def test_cancel_event_cleared_on_send_start(self, tmp_db):
|
||||
"""send() clears a stale cancel flag before starting."""
|
||||
ui = NullUI()
|
||||
session = _make_session(ui=ui)
|
||||
session.cancel() # Set stale flag
|
||||
|
||||
@dataclass
|
||||
class FakeChunk:
|
||||
content_delta: str = ""
|
||||
reasoning_delta: str = ""
|
||||
tool_call_deltas: list = field(default_factory=list)
|
||||
usage: None = None
|
||||
finish_reason: str = "stop"
|
||||
info_delta: str = ""
|
||||
provider_blocks: list = field(default_factory=list)
|
||||
|
||||
fake_stream = iter([FakeChunk(content_delta="Hello", finish_reason="stop")])
|
||||
|
||||
with (
|
||||
patch.object(session, "_create_stream_with_retry", return_value=fake_stream),
|
||||
patch.object(session, "_full_messages", return_value=[]),
|
||||
):
|
||||
session.send("test")
|
||||
|
||||
# Should complete normally — cancel flag was cleared
|
||||
assert "idle" in ui.states
|
||||
|
||||
|
||||
class TestCancelDuringStreaming:
|
||||
"""Cancel while _stream_response is iterating chunks."""
|
||||
|
||||
def test_preserves_partial_content(self, tmp_db):
|
||||
"""Partial content already streamed should be preserved in messages."""
|
||||
ui = NullUI()
|
||||
session = _make_session(ui=ui)
|
||||
|
||||
@dataclass
|
||||
class FakeChunk:
|
||||
content_delta: str = ""
|
||||
reasoning_delta: str = ""
|
||||
tool_call_deltas: list = field(default_factory=list)
|
||||
usage: None = None
|
||||
finish_reason: str = ""
|
||||
info_delta: str = ""
|
||||
provider_blocks: list = field(default_factory=list)
|
||||
|
||||
def cancelling_stream():
|
||||
"""Yield a few chunks then cancel."""
|
||||
yield FakeChunk(content_delta="Hello ")
|
||||
yield FakeChunk(content_delta="world")
|
||||
session.cancel()
|
||||
yield FakeChunk(content_delta=" — this should not appear")
|
||||
|
||||
with (
|
||||
patch.object(session, "_create_stream_with_retry", return_value=cancelling_stream()),
|
||||
patch.object(session, "_full_messages", return_value=[]),
|
||||
):
|
||||
session.send("test")
|
||||
|
||||
# Session should be idle (not error)
|
||||
assert ui.states[-1] == "idle"
|
||||
# Check that "[Generation cancelled]" was emitted
|
||||
assert any("cancelled" in i.lower() for i in ui.infos)
|
||||
# The partial content should be preserved as an assistant message
|
||||
assistant_msgs = [m for m in session.messages if m["role"] == "assistant"]
|
||||
assert len(assistant_msgs) == 1
|
||||
assert assistant_msgs[0]["content"] == "Hello world"
|
||||
# No tool_calls in the partial message
|
||||
assert "tool_calls" not in assistant_msgs[0]
|
||||
|
||||
|
||||
class TestCancelDuringToolExecution:
|
||||
"""Cancel while tools are being executed."""
|
||||
|
||||
def test_rollback_incomplete_tool_results(self, tmp_db):
|
||||
"""When cancelled during tool execution, incomplete results are rolled back."""
|
||||
ui = NullUI()
|
||||
session = _make_session(ui=ui)
|
||||
|
||||
@dataclass
|
||||
class FakeChunk:
|
||||
content_delta: str = ""
|
||||
reasoning_delta: str = ""
|
||||
tool_call_deltas: list = field(default_factory=list)
|
||||
usage: None = None
|
||||
finish_reason: str = ""
|
||||
info_delta: str = ""
|
||||
provider_blocks: list = field(default_factory=list)
|
||||
|
||||
@dataclass
|
||||
class FakeToolDelta:
|
||||
index: int = 0
|
||||
id: str = ""
|
||||
name: str = ""
|
||||
arguments_delta: str = ""
|
||||
|
||||
# First call: return content with a tool call
|
||||
def stream_with_tool():
|
||||
yield FakeChunk(
|
||||
tool_call_deltas=[FakeToolDelta(index=0, id="tc_1", name="bash")],
|
||||
finish_reason="",
|
||||
)
|
||||
yield FakeChunk(
|
||||
tool_call_deltas=[FakeToolDelta(index=0, arguments_delta='{"command":"echo hi"}')],
|
||||
finish_reason="tool_calls",
|
||||
)
|
||||
|
||||
call_count = 0
|
||||
|
||||
def fake_create_stream(msgs):
|
||||
nonlocal call_count
|
||||
call_count += 1
|
||||
if call_count == 1:
|
||||
return stream_with_tool()
|
||||
# Should not be called a second time since cancel happens before phase 3
|
||||
raise AssertionError("Should not stream again after cancel")
|
||||
|
||||
def cancel_before_execute(tool_calls):
|
||||
"""Simulate cancel happening before tool execution."""
|
||||
session.cancel()
|
||||
raise GenerationCancelled()
|
||||
|
||||
with (
|
||||
patch.object(session, "_create_stream_with_retry", side_effect=fake_create_stream),
|
||||
patch.object(session, "_full_messages", return_value=[]),
|
||||
patch.object(session, "_execute_tools", side_effect=cancel_before_execute),
|
||||
):
|
||||
session.send("run something")
|
||||
|
||||
# Session should be idle
|
||||
assert ui.states[-1] == "idle"
|
||||
# No tool result messages should remain (rolled back)
|
||||
roles = [m["role"] for m in session.messages]
|
||||
assert "tool" not in roles
|
||||
# The assistant message with tool_calls should also be rolled back
|
||||
for m in session.messages:
|
||||
if m["role"] == "assistant":
|
||||
assert "tool_calls" not in m or not m["tool_calls"]
|
||||
|
||||
|
||||
class TestCancelWhenIdle:
|
||||
"""Cancelling when no generation is active is harmless."""
|
||||
|
||||
def test_cancel_when_idle_is_noop(self, tmp_db):
|
||||
session = _make_session()
|
||||
session.cancel()
|
||||
# Next send should work normally (cancel cleared at start)
|
||||
|
||||
@dataclass
|
||||
class FakeChunk:
|
||||
content_delta: str = ""
|
||||
reasoning_delta: str = ""
|
||||
tool_call_deltas: list = field(default_factory=list)
|
||||
usage: None = None
|
||||
finish_reason: str = "stop"
|
||||
info_delta: str = ""
|
||||
provider_blocks: list = field(default_factory=list)
|
||||
|
||||
fake_stream = iter([FakeChunk(content_delta="ok", finish_reason="stop")])
|
||||
with (
|
||||
patch.object(session, "_create_stream_with_retry", return_value=fake_stream),
|
||||
patch.object(session, "_full_messages", return_value=[]),
|
||||
):
|
||||
session.send("hello")
|
||||
|
||||
# Should complete normally
|
||||
assistant_msgs = [m for m in session.messages if m["role"] == "assistant"]
|
||||
assert len(assistant_msgs) == 1
|
||||
assert assistant_msgs[0]["content"] == "ok"
|
||||
|
||||
|
||||
class TestCancelThreadSafety:
|
||||
"""Cancel from a different thread while generation is running."""
|
||||
|
||||
def test_cancel_from_another_thread(self, tmp_db):
|
||||
ui = NullUI()
|
||||
session = _make_session(ui=ui)
|
||||
|
||||
@dataclass
|
||||
class FakeChunk:
|
||||
content_delta: str = ""
|
||||
reasoning_delta: str = ""
|
||||
tool_call_deltas: list = field(default_factory=list)
|
||||
usage: None = None
|
||||
finish_reason: str = ""
|
||||
info_delta: str = ""
|
||||
provider_blocks: list = field(default_factory=list)
|
||||
|
||||
barrier = threading.Event()
|
||||
|
||||
def slow_stream():
|
||||
yield FakeChunk(content_delta="Start")
|
||||
barrier.set() # Signal that streaming has started
|
||||
time.sleep(2) # Simulate slow streaming
|
||||
yield FakeChunk(content_delta=" end", finish_reason="stop")
|
||||
|
||||
with (
|
||||
patch.object(session, "_create_stream_with_retry", return_value=slow_stream()),
|
||||
patch.object(session, "_full_messages", return_value=[]),
|
||||
):
|
||||
# Run send() in a thread
|
||||
error = []
|
||||
|
||||
def run():
|
||||
try:
|
||||
session.send("test")
|
||||
except Exception as e:
|
||||
error.append(e)
|
||||
|
||||
t = threading.Thread(target=run)
|
||||
t.start()
|
||||
barrier.wait(timeout=5)
|
||||
# Cancel from main thread
|
||||
session.cancel()
|
||||
t.join(timeout=5)
|
||||
|
||||
assert not error
|
||||
assert ui.states[-1] == "idle"
|
||||
assert any("cancelled" in i.lower() for i in ui.infos)
|
||||
|
||||
|
||||
class TestGenerationCancelledException:
|
||||
"""GenerationCancelled is a BaseException, not Exception."""
|
||||
|
||||
def test_is_base_exception(self):
|
||||
assert issubclass(GenerationCancelled, BaseException)
|
||||
|
||||
def test_not_caught_by_except_exception(self):
|
||||
"""Verify GenerationCancelled is NOT caught by except Exception."""
|
||||
with pytest.raises(GenerationCancelled):
|
||||
try:
|
||||
raise GenerationCancelled()
|
||||
except Exception:
|
||||
pytest.fail("GenerationCancelled was caught by except Exception")
|
||||
@@ -1,5 +1,6 @@
|
||||
"""Tests for turnstone.console — collector and HTTP server."""
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
import queue
|
||||
from unittest.mock import MagicMock
|
||||
@@ -201,6 +202,68 @@ class TestCollectorPolling:
|
||||
# Should not raise
|
||||
c._apply_poll("unknown", _dashboard_response(), {})
|
||||
|
||||
def test_apply_poll_emits_ws_created_for_new_workstream(self):
|
||||
c = _make_collector()
|
||||
c._nodes["node-a"] = NodeSnapshot(node_id="node-a", server_url="http://a:8080")
|
||||
q: queue.Queue[dict] = queue.Queue()
|
||||
c.register_listener(q)
|
||||
|
||||
dashboard = _dashboard_response(
|
||||
workstreams=[{"id": "ws1", "name": "new-task", "state": "idle"}]
|
||||
)
|
||||
c._apply_poll("node-a", dashboard, {})
|
||||
|
||||
event = q.get_nowait()
|
||||
assert event["type"] == "ws_created"
|
||||
assert event["ws_id"] == "ws1"
|
||||
assert event["name"] == "new-task"
|
||||
assert event["node_id"] == "node-a"
|
||||
|
||||
def test_apply_poll_emits_ws_closed_for_removed_workstream(self):
|
||||
c = _make_collector()
|
||||
c._nodes["node-a"] = NodeSnapshot(
|
||||
node_id="node-a",
|
||||
server_url="http://a:8080",
|
||||
workstreams={"ws1": {"id": "ws1", "name": "old", "state": "idle"}},
|
||||
)
|
||||
q: queue.Queue[dict] = queue.Queue()
|
||||
c.register_listener(q)
|
||||
|
||||
c._apply_poll("node-a", _dashboard_response(), {})
|
||||
|
||||
event = q.get_nowait()
|
||||
assert event["type"] == "ws_closed"
|
||||
assert event["ws_id"] == "ws1"
|
||||
|
||||
def test_apply_poll_no_events_when_unchanged(self):
|
||||
c = _make_collector()
|
||||
c._nodes["node-a"] = NodeSnapshot(
|
||||
node_id="node-a",
|
||||
server_url="http://a:8080",
|
||||
workstreams={"ws1": {"id": "ws1", "name": "same", "state": "idle"}},
|
||||
)
|
||||
q: queue.Queue[dict] = queue.Queue()
|
||||
c.register_listener(q)
|
||||
|
||||
dashboard = _dashboard_response(
|
||||
workstreams=[{"id": "ws1", "name": "same", "state": "running"}]
|
||||
)
|
||||
c._apply_poll("node-a", dashboard, {})
|
||||
|
||||
assert q.empty()
|
||||
|
||||
def test_apply_poll_skips_empty_id_workstream(self):
|
||||
c = _make_collector()
|
||||
c._nodes["node-a"] = NodeSnapshot(node_id="node-a", server_url="http://a:8080")
|
||||
q: queue.Queue[dict] = queue.Queue()
|
||||
c.register_listener(q)
|
||||
|
||||
dashboard = _dashboard_response(workstreams=[{"name": "no-id", "state": "idle"}])
|
||||
c._apply_poll("node-a", dashboard, {})
|
||||
|
||||
assert q.empty()
|
||||
assert len(c._nodes["node-a"].workstreams) == 0
|
||||
|
||||
|
||||
class TestCollectorEvents:
|
||||
"""Real-time event handling from cluster channel."""
|
||||
@@ -445,6 +508,44 @@ class TestCollectorQueries:
|
||||
def test_get_node_detail_not_found(self, populated_collector):
|
||||
assert populated_collector.get_node_detail("nonexistent") is None
|
||||
|
||||
def test_get_snapshot_empty(self):
|
||||
c = _make_collector()
|
||||
snap = c.get_snapshot()
|
||||
assert snap["nodes"] == []
|
||||
assert snap["overview"]["nodes"] == 0
|
||||
assert snap["overview"]["workstreams"] == 0
|
||||
assert snap["overview"]["states"]["running"] == 0
|
||||
assert "timestamp" in snap
|
||||
|
||||
def test_get_snapshot_with_nodes(self, populated_collector):
|
||||
snap = populated_collector.get_snapshot()
|
||||
assert len(snap["nodes"]) == 2
|
||||
assert snap["overview"]["nodes"] == 2
|
||||
assert snap["overview"]["workstreams"] == 3
|
||||
assert snap["overview"]["states"]["running"] == 1
|
||||
assert snap["overview"]["states"]["attention"] == 1
|
||||
assert snap["overview"]["states"]["idle"] == 1
|
||||
assert snap["overview"]["aggregate"]["total_tokens"] == 17000
|
||||
assert snap["timestamp"] > 0
|
||||
# Each node should embed its workstreams
|
||||
node_ids = {n["node_id"] for n in snap["nodes"]}
|
||||
assert node_ids == {"node-a", "node-b"}
|
||||
for n in snap["nodes"]:
|
||||
if n["node_id"] == "node-a":
|
||||
assert len(n["workstreams"]) == 2
|
||||
elif n["node_id"] == "node-b":
|
||||
assert len(n["workstreams"]) == 1
|
||||
|
||||
def test_get_snapshot_consistency(self, populated_collector):
|
||||
"""Snapshot overview should match get_overview()."""
|
||||
snap = populated_collector.get_snapshot()
|
||||
overview = populated_collector.get_overview()
|
||||
assert snap["overview"]["nodes"] == overview["nodes"]
|
||||
assert snap["overview"]["workstreams"] == overview["workstreams"]
|
||||
assert snap["overview"]["states"] == overview["states"]
|
||||
assert snap["overview"]["aggregate"] == overview["aggregate"]
|
||||
assert snap["overview"]["version_drift"] == overview["version_drift"]
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# ClusterStateEvent protocol tests
|
||||
@@ -535,6 +636,31 @@ class TestConsoleHTTPEndpoints:
|
||||
"workstreams": [],
|
||||
"aggregate": {},
|
||||
}
|
||||
collector.get_snapshot.return_value = {
|
||||
"nodes": [
|
||||
{
|
||||
"node_id": "node-a",
|
||||
"server_url": "http://a:8080",
|
||||
"max_ws": 10,
|
||||
"reachable": True,
|
||||
"version": "0.5.0",
|
||||
"health": {},
|
||||
"aggregate": {"total_tokens": 50000, "total_tool_calls": 200},
|
||||
"workstreams": [
|
||||
{"id": "ws1", "name": "test", "state": "running", "node": "node-a"},
|
||||
],
|
||||
},
|
||||
],
|
||||
"overview": {
|
||||
"nodes": 3,
|
||||
"workstreams": 15,
|
||||
"states": {"running": 5, "thinking": 2, "attention": 1, "idle": 6, "error": 1},
|
||||
"aggregate": {"total_tokens": 50000, "total_tool_calls": 200},
|
||||
"version_drift": False,
|
||||
"versions": ["0.5.0"],
|
||||
},
|
||||
"timestamp": 1234567890.0,
|
||||
}
|
||||
return collector
|
||||
|
||||
@pytest.fixture()
|
||||
@@ -614,6 +740,16 @@ class TestConsoleHTTPEndpoints:
|
||||
assert status == 404
|
||||
assert "error" in data
|
||||
|
||||
def test_get_snapshot(self, client, mock_collector):
|
||||
status, data = self._get(client, "/v1/api/cluster/snapshot")
|
||||
assert status == 200
|
||||
assert len(data["nodes"]) == 1
|
||||
assert data["nodes"][0]["node_id"] == "node-a"
|
||||
assert data["overview"]["nodes"] == 3
|
||||
assert data["overview"]["workstreams"] == 15
|
||||
assert data["timestamp"] == 1234567890.0
|
||||
mock_collector.get_snapshot.assert_called_once()
|
||||
|
||||
def test_health_endpoint(self, client, mock_collector):
|
||||
status, data = self._get(client, "/health")
|
||||
assert status == 200
|
||||
@@ -1299,3 +1435,177 @@ class TestProxySharedStatic:
|
||||
resp = client.get("/node/unknown/shared/base.css")
|
||||
assert resp.status_code == 404
|
||||
client.close()
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# SSE proxy — raw byte passthrough
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestSSEProxy:
|
||||
"""Verify _proxy_sse forwards raw bytes including ping comments."""
|
||||
|
||||
def test_proxy_sse_preserves_pings_and_events(self):
|
||||
"""SSE proxy should forward ping comments and events verbatim."""
|
||||
from turnstone.console.server import _proxy_sse
|
||||
|
||||
# Simulate an upstream SSE response with a ping comment and a real event
|
||||
sse_payload = b': ping - 2026-03-08T12:00:00Z\n\nevent: message\ndata: {"type": "test"}\n\n'
|
||||
|
||||
class FakeResponse:
|
||||
status_code = 200
|
||||
headers = {"content-type": "text/event-stream"}
|
||||
|
||||
async def aiter_bytes(self):
|
||||
yield sse_payload
|
||||
|
||||
async def aclose(self):
|
||||
pass
|
||||
|
||||
async def __aenter__(self):
|
||||
return self
|
||||
|
||||
async def __aexit__(self, *args):
|
||||
pass
|
||||
|
||||
class FakeClient:
|
||||
def stream(self, method, url, **kwargs):
|
||||
return FakeResponse()
|
||||
|
||||
class FakeRequest:
|
||||
class url: # noqa: N801
|
||||
query = "ws_id=test123"
|
||||
|
||||
class app: # noqa: N801
|
||||
class state: # noqa: N801
|
||||
proxy_sse_client = FakeClient()
|
||||
proxy_auth_token = ""
|
||||
|
||||
headers = {}
|
||||
|
||||
async def is_disconnected(self):
|
||||
return False
|
||||
|
||||
async def _run():
|
||||
response = await _proxy_sse(
|
||||
FakeRequest(), "http://fake:8080", "events", api_prefix="v1/api"
|
||||
)
|
||||
assert response.media_type == "text/event-stream"
|
||||
# Collect the streamed bytes
|
||||
chunks: list[bytes] = []
|
||||
async for chunk in response.body_iterator:
|
||||
chunks.append(chunk if isinstance(chunk, bytes) else chunk.encode())
|
||||
body = b"".join(chunks)
|
||||
# Ping comment must be preserved (not filtered)
|
||||
assert b": ping" in body
|
||||
# Real event must be preserved
|
||||
assert b"event: message" in body
|
||||
assert b'"type": "test"' in body
|
||||
|
||||
asyncio.run(_run())
|
||||
|
||||
def test_proxy_sse_upstream_error_status(self):
|
||||
"""Non-200 upstream status should yield an error event."""
|
||||
|
||||
from turnstone.console.server import _proxy_sse
|
||||
|
||||
class FakeResponse:
|
||||
status_code = 502
|
||||
|
||||
async def aiter_bytes(self):
|
||||
return
|
||||
yield # make it an async generator
|
||||
|
||||
async def aclose(self):
|
||||
pass
|
||||
|
||||
async def __aenter__(self):
|
||||
return self
|
||||
|
||||
async def __aexit__(self, *args):
|
||||
pass
|
||||
|
||||
class FakeClient:
|
||||
def stream(self, method, url, **kwargs):
|
||||
return FakeResponse()
|
||||
|
||||
class FakeRequest:
|
||||
class url: # noqa: N801
|
||||
query = ""
|
||||
|
||||
class app: # noqa: N801
|
||||
class state: # noqa: N801
|
||||
proxy_sse_client = FakeClient()
|
||||
proxy_auth_token = ""
|
||||
|
||||
headers = {}
|
||||
|
||||
async def is_disconnected(self):
|
||||
return False
|
||||
|
||||
async def _run():
|
||||
response = await _proxy_sse(FakeRequest(), "http://fake:8080", "events")
|
||||
chunks: list[bytes] = []
|
||||
async for chunk in response.body_iterator:
|
||||
chunks.append(chunk if isinstance(chunk, bytes) else chunk.encode())
|
||||
body = b"".join(chunks)
|
||||
assert b"event: error" in body
|
||||
assert b"502" in body
|
||||
|
||||
asyncio.run(_run())
|
||||
|
||||
def test_proxy_sse_disconnect_handling(self):
|
||||
"""Proxy should stop when browser disconnects."""
|
||||
|
||||
from turnstone.console.server import _proxy_sse
|
||||
|
||||
class FakeResponse:
|
||||
status_code = 200
|
||||
|
||||
async def aiter_bytes(self):
|
||||
yield b"data: chunk1\n\n"
|
||||
yield b"data: chunk2\n\n" # should not be reached
|
||||
yield b"data: chunk3\n\n"
|
||||
|
||||
async def aclose(self):
|
||||
pass
|
||||
|
||||
async def __aenter__(self):
|
||||
return self
|
||||
|
||||
async def __aexit__(self, *args):
|
||||
pass
|
||||
|
||||
class FakeClient:
|
||||
def stream(self, method, url, **kwargs):
|
||||
return FakeResponse()
|
||||
|
||||
call_count = 0
|
||||
|
||||
class FakeRequest:
|
||||
class url: # noqa: N801
|
||||
query = ""
|
||||
|
||||
class app: # noqa: N801
|
||||
class state: # noqa: N801
|
||||
proxy_sse_client = FakeClient()
|
||||
proxy_auth_token = ""
|
||||
|
||||
headers = {}
|
||||
|
||||
async def is_disconnected(self):
|
||||
nonlocal call_count
|
||||
call_count += 1
|
||||
return call_count > 1 # disconnect after first chunk
|
||||
|
||||
async def _run():
|
||||
response = await _proxy_sse(FakeRequest(), "http://fake:8080", "events")
|
||||
chunks: list[bytes] = []
|
||||
async for chunk in response.body_iterator:
|
||||
chunks.append(chunk if isinstance(chunk, bytes) else chunk.encode())
|
||||
body = b"".join(chunks)
|
||||
assert b"chunk1" in body
|
||||
# Should have stopped before chunk3
|
||||
assert b"chunk3" not in body
|
||||
|
||||
asyncio.run(_run())
|
||||
|
||||
@@ -0,0 +1,761 @@
|
||||
"""Tests for governance admin API endpoints (roles, orgs, policies, templates, usage, audit)."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
import pytest
|
||||
from starlette.applications import Starlette
|
||||
from starlette.middleware import Middleware
|
||||
from starlette.middleware.base import BaseHTTPMiddleware
|
||||
from starlette.routing import Mount, Route
|
||||
from starlette.testclient import TestClient
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from starlette.requests import Request
|
||||
from starlette.responses import Response
|
||||
|
||||
from turnstone.console.server import (
|
||||
admin_assign_role,
|
||||
admin_audit,
|
||||
admin_create_policy,
|
||||
admin_create_role,
|
||||
admin_create_template,
|
||||
admin_delete_policy,
|
||||
admin_delete_role,
|
||||
admin_delete_template,
|
||||
admin_delete_user,
|
||||
admin_get_org,
|
||||
admin_list_orgs,
|
||||
admin_list_policies,
|
||||
admin_list_roles,
|
||||
admin_list_templates,
|
||||
admin_list_user_roles,
|
||||
admin_unassign_role,
|
||||
admin_update_org,
|
||||
admin_update_policy,
|
||||
admin_update_role,
|
||||
admin_update_template,
|
||||
admin_usage,
|
||||
)
|
||||
from turnstone.core.auth import AuthResult
|
||||
from turnstone.core.storage._sqlite import SQLiteBackend
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Auth bypass middleware — injects a full-access AuthResult on every request.
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class _InjectAuthMiddleware(BaseHTTPMiddleware):
|
||||
async def dispatch(self, request: Request, call_next: Any) -> Response:
|
||||
request.state.auth_result = AuthResult(
|
||||
user_id="test-admin",
|
||||
scopes=frozenset({"approve"}),
|
||||
token_source="config",
|
||||
permissions=frozenset(
|
||||
{
|
||||
"read",
|
||||
"write",
|
||||
"approve",
|
||||
"admin.roles",
|
||||
"admin.users",
|
||||
"admin.orgs",
|
||||
"admin.policies",
|
||||
"admin.templates",
|
||||
"admin.usage",
|
||||
"admin.audit",
|
||||
"admin.schedules",
|
||||
"admin.watches",
|
||||
"tools.approve",
|
||||
"workstreams.create",
|
||||
"workstreams.close",
|
||||
}
|
||||
),
|
||||
)
|
||||
return await call_next(request)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Fixtures
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def storage(tmp_path):
|
||||
"""Fresh SQLite backend for each test, seeded with test users."""
|
||||
backend = SQLiteBackend(str(tmp_path / "test.db"))
|
||||
# Seed users required by role assignment tests
|
||||
backend.create_user("test-admin", "testadmin", "Test Admin", "hash")
|
||||
backend.create_user("user-1", "user1", "User One", "hash")
|
||||
return backend
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def client(storage):
|
||||
"""TestClient with storage and auth bypassed."""
|
||||
app = Starlette(
|
||||
routes=[
|
||||
Mount(
|
||||
"/v1",
|
||||
routes=[
|
||||
# Roles
|
||||
Route("/api/admin/roles", admin_list_roles),
|
||||
Route("/api/admin/roles", admin_create_role, methods=["POST"]),
|
||||
Route("/api/admin/roles/{role_id}", admin_update_role, methods=["PUT"]),
|
||||
Route("/api/admin/roles/{role_id}", admin_delete_role, methods=["DELETE"]),
|
||||
# Users
|
||||
Route(
|
||||
"/api/admin/users/{user_id}",
|
||||
admin_delete_user,
|
||||
methods=["DELETE"],
|
||||
),
|
||||
# User-role assignments
|
||||
Route("/api/admin/users/{user_id}/roles", admin_list_user_roles),
|
||||
Route(
|
||||
"/api/admin/users/{user_id}/roles",
|
||||
admin_assign_role,
|
||||
methods=["POST"],
|
||||
),
|
||||
Route(
|
||||
"/api/admin/users/{user_id}/roles/{role_id}",
|
||||
admin_unassign_role,
|
||||
methods=["DELETE"],
|
||||
),
|
||||
# Orgs
|
||||
Route("/api/admin/orgs", admin_list_orgs),
|
||||
Route("/api/admin/orgs/{org_id}", admin_get_org),
|
||||
Route("/api/admin/orgs/{org_id}", admin_update_org, methods=["PUT"]),
|
||||
# Policies
|
||||
Route("/api/admin/policies", admin_list_policies),
|
||||
Route("/api/admin/policies", admin_create_policy, methods=["POST"]),
|
||||
Route(
|
||||
"/api/admin/policies/{policy_id}",
|
||||
admin_update_policy,
|
||||
methods=["PUT"],
|
||||
),
|
||||
Route(
|
||||
"/api/admin/policies/{policy_id}",
|
||||
admin_delete_policy,
|
||||
methods=["DELETE"],
|
||||
),
|
||||
# Templates
|
||||
Route("/api/admin/templates", admin_list_templates),
|
||||
Route("/api/admin/templates", admin_create_template, methods=["POST"]),
|
||||
Route(
|
||||
"/api/admin/templates/{template_id}",
|
||||
admin_update_template,
|
||||
methods=["PUT"],
|
||||
),
|
||||
Route(
|
||||
"/api/admin/templates/{template_id}",
|
||||
admin_delete_template,
|
||||
methods=["DELETE"],
|
||||
),
|
||||
# Usage & Audit
|
||||
Route("/api/admin/usage", admin_usage),
|
||||
Route("/api/admin/audit", admin_audit),
|
||||
],
|
||||
),
|
||||
],
|
||||
middleware=[Middleware(_InjectAuthMiddleware)],
|
||||
)
|
||||
app.state.auth_storage = storage
|
||||
return TestClient(app)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Helpers
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _role_payload(**overrides: Any) -> dict[str, Any]:
|
||||
defaults: dict[str, Any] = {
|
||||
"name": "analyst",
|
||||
"display_name": "Data Analyst",
|
||||
"permissions": "read,write",
|
||||
}
|
||||
defaults.update(overrides)
|
||||
return defaults
|
||||
|
||||
|
||||
def _policy_payload(**overrides: Any) -> dict[str, Any]:
|
||||
defaults: dict[str, Any] = {
|
||||
"name": "Allow bash",
|
||||
"tool_pattern": "bash_*",
|
||||
"action": "allow",
|
||||
"priority": 10,
|
||||
}
|
||||
defaults.update(overrides)
|
||||
return defaults
|
||||
|
||||
|
||||
def _template_payload(**overrides: Any) -> dict[str, Any]:
|
||||
defaults: dict[str, Any] = {
|
||||
"name": "Greeting",
|
||||
"content": "Hello {{user}}, how can I help?",
|
||||
"category": "system",
|
||||
}
|
||||
defaults.update(overrides)
|
||||
return defaults
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Tests — Roles
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestRoles:
|
||||
def test_list_empty(self, client):
|
||||
resp = client.get("/v1/api/admin/roles")
|
||||
assert resp.status_code == 200
|
||||
assert resp.json()["roles"] == []
|
||||
|
||||
def test_create_role(self, client):
|
||||
resp = client.post("/v1/api/admin/roles", json=_role_payload())
|
||||
assert resp.status_code == 200
|
||||
role = resp.json()
|
||||
assert role["name"] == "analyst"
|
||||
assert role["display_name"] == "Data Analyst"
|
||||
assert role["permissions"] == "read,write"
|
||||
assert role["builtin"] is False
|
||||
assert "role_id" in role
|
||||
assert "created" in role
|
||||
|
||||
def test_create_role_missing_name(self, client):
|
||||
resp = client.post("/v1/api/admin/roles", json=_role_payload(name=""))
|
||||
assert resp.status_code == 400
|
||||
assert "name" in resp.json()["error"].lower()
|
||||
|
||||
def test_create_role_invalid_name(self, client):
|
||||
resp = client.post("/v1/api/admin/roles", json=_role_payload(name="bad name!@#"))
|
||||
assert resp.status_code == 400
|
||||
assert "name" in resp.json()["error"].lower()
|
||||
|
||||
def test_create_role_default_display_name(self, client):
|
||||
resp = client.post(
|
||||
"/v1/api/admin/roles",
|
||||
json={"name": "ops", "permissions": ""},
|
||||
)
|
||||
assert resp.status_code == 200
|
||||
role = resp.json()
|
||||
# display_name defaults to name when not provided
|
||||
assert role["display_name"] == "ops"
|
||||
|
||||
def test_list_after_create(self, client):
|
||||
client.post("/v1/api/admin/roles", json=_role_payload())
|
||||
resp = client.get("/v1/api/admin/roles")
|
||||
assert resp.status_code == 200
|
||||
roles = resp.json()["roles"]
|
||||
assert len(roles) == 1
|
||||
assert roles[0]["name"] == "analyst"
|
||||
|
||||
def test_update_role(self, client):
|
||||
create_resp = client.post("/v1/api/admin/roles", json=_role_payload())
|
||||
role_id = create_resp.json()["role_id"]
|
||||
|
||||
resp = client.put(
|
||||
f"/v1/api/admin/roles/{role_id}",
|
||||
json={"display_name": "Senior Analyst", "permissions": "read,write,approve"},
|
||||
)
|
||||
assert resp.status_code == 200
|
||||
role = resp.json()
|
||||
assert role["display_name"] == "Senior Analyst"
|
||||
assert role["permissions"] == "read,write,approve"
|
||||
|
||||
def test_update_nonexistent_role(self, client):
|
||||
resp = client.put(
|
||||
"/v1/api/admin/roles/nonexistent",
|
||||
json={"display_name": "Nope"},
|
||||
)
|
||||
assert resp.status_code == 404
|
||||
|
||||
def test_update_builtin_role_rejected(self, client, storage):
|
||||
# Seed a builtin role directly via storage
|
||||
storage.create_role(
|
||||
role_id="builtin-admin",
|
||||
name="admin",
|
||||
display_name="Administrator",
|
||||
permissions="*",
|
||||
builtin=True,
|
||||
)
|
||||
resp = client.put(
|
||||
"/v1/api/admin/roles/builtin-admin",
|
||||
json={"display_name": "Hacked"},
|
||||
)
|
||||
assert resp.status_code == 400
|
||||
assert "builtin" in resp.json()["error"].lower()
|
||||
|
||||
def test_delete_role(self, client):
|
||||
create_resp = client.post("/v1/api/admin/roles", json=_role_payload())
|
||||
role_id = create_resp.json()["role_id"]
|
||||
|
||||
resp = client.delete(f"/v1/api/admin/roles/{role_id}")
|
||||
assert resp.status_code == 200
|
||||
assert resp.json()["status"] == "ok"
|
||||
|
||||
# Verify gone from listing
|
||||
list_resp = client.get("/v1/api/admin/roles")
|
||||
assert list_resp.json()["roles"] == []
|
||||
|
||||
def test_delete_nonexistent_role(self, client):
|
||||
resp = client.delete("/v1/api/admin/roles/nonexistent")
|
||||
assert resp.status_code == 404
|
||||
|
||||
def test_delete_builtin_role_rejected(self, client, storage):
|
||||
storage.create_role(
|
||||
role_id="builtin-viewer",
|
||||
name="viewer",
|
||||
display_name="Viewer",
|
||||
permissions="read",
|
||||
builtin=True,
|
||||
)
|
||||
resp = client.delete("/v1/api/admin/roles/builtin-viewer")
|
||||
assert resp.status_code == 400
|
||||
assert "builtin" in resp.json()["error"].lower()
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Tests — Role assignments
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestRoleAssignments:
|
||||
def test_list_user_roles_empty(self, client):
|
||||
resp = client.get("/v1/api/admin/users/user-1/roles")
|
||||
assert resp.status_code == 200
|
||||
assert resp.json()["roles"] == []
|
||||
|
||||
def test_assign_role(self, client):
|
||||
create_resp = client.post("/v1/api/admin/roles", json=_role_payload())
|
||||
role_id = create_resp.json()["role_id"]
|
||||
|
||||
resp = client.post(
|
||||
"/v1/api/admin/users/user-1/roles",
|
||||
json={"role_id": role_id},
|
||||
)
|
||||
assert resp.status_code == 200
|
||||
assert resp.json()["status"] == "ok"
|
||||
|
||||
# Verify listed
|
||||
list_resp = client.get("/v1/api/admin/users/user-1/roles")
|
||||
roles = list_resp.json()["roles"]
|
||||
assert len(roles) >= 1
|
||||
|
||||
def test_assign_role_missing_role_id(self, client):
|
||||
resp = client.post(
|
||||
"/v1/api/admin/users/user-1/roles",
|
||||
json={},
|
||||
)
|
||||
assert resp.status_code == 400
|
||||
assert "role_id" in resp.json()["error"].lower()
|
||||
|
||||
def test_unassign_role(self, client):
|
||||
create_resp = client.post("/v1/api/admin/roles", json=_role_payload())
|
||||
role_id = create_resp.json()["role_id"]
|
||||
|
||||
# Assign first
|
||||
client.post(
|
||||
"/v1/api/admin/users/user-1/roles",
|
||||
json={"role_id": role_id},
|
||||
)
|
||||
|
||||
# Now unassign
|
||||
resp = client.delete(f"/v1/api/admin/users/user-1/roles/{role_id}")
|
||||
assert resp.status_code == 200
|
||||
assert resp.json()["status"] == "ok"
|
||||
|
||||
# Verify removed
|
||||
list_resp = client.get("/v1/api/admin/users/user-1/roles")
|
||||
assert list_resp.json()["roles"] == []
|
||||
|
||||
def test_unassign_nonexistent(self, client):
|
||||
resp = client.delete("/v1/api/admin/users/user-1/roles/nonexistent")
|
||||
assert resp.status_code == 404
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Tests — Orgs
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestOrgs:
|
||||
def test_list_empty(self, client):
|
||||
resp = client.get("/v1/api/admin/orgs")
|
||||
assert resp.status_code == 200
|
||||
assert resp.json()["orgs"] == []
|
||||
|
||||
def test_get_org(self, client, storage):
|
||||
storage.create_org(
|
||||
org_id="org-1",
|
||||
name="acme",
|
||||
display_name="Acme Corp",
|
||||
settings='{"theme": "dark"}',
|
||||
)
|
||||
resp = client.get("/v1/api/admin/orgs/org-1")
|
||||
assert resp.status_code == 200
|
||||
org = resp.json()
|
||||
assert org["org_id"] == "org-1"
|
||||
assert org["name"] == "acme"
|
||||
assert org["display_name"] == "Acme Corp"
|
||||
|
||||
def test_get_org_not_found(self, client):
|
||||
resp = client.get("/v1/api/admin/orgs/nonexistent")
|
||||
assert resp.status_code == 404
|
||||
|
||||
def test_update_org(self, client, storage):
|
||||
storage.create_org(org_id="org-1", name="acme", display_name="Acme Corp")
|
||||
|
||||
resp = client.put(
|
||||
"/v1/api/admin/orgs/org-1",
|
||||
json={"display_name": "Acme Inc.", "settings": '{"theme": "light"}'},
|
||||
)
|
||||
assert resp.status_code == 200
|
||||
org = resp.json()
|
||||
assert org["display_name"] == "Acme Inc."
|
||||
assert org["settings"] == '{"theme": "light"}'
|
||||
|
||||
def test_update_org_not_found(self, client):
|
||||
resp = client.put(
|
||||
"/v1/api/admin/orgs/nonexistent",
|
||||
json={"display_name": "Nope"},
|
||||
)
|
||||
assert resp.status_code == 404
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Tests — Tool policies
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestPolicies:
|
||||
def test_list_empty(self, client):
|
||||
resp = client.get("/v1/api/admin/policies")
|
||||
assert resp.status_code == 200
|
||||
assert resp.json()["policies"] == []
|
||||
|
||||
def test_create_policy(self, client):
|
||||
resp = client.post("/v1/api/admin/policies", json=_policy_payload())
|
||||
assert resp.status_code == 200
|
||||
policy = resp.json()
|
||||
assert policy["name"] == "Allow bash"
|
||||
assert policy["tool_pattern"] == "bash_*"
|
||||
assert policy["action"] == "allow"
|
||||
assert policy["priority"] == 10
|
||||
assert "policy_id" in policy
|
||||
assert "created" in policy
|
||||
|
||||
def test_create_policy_missing_name(self, client):
|
||||
resp = client.post("/v1/api/admin/policies", json=_policy_payload(name=""))
|
||||
assert resp.status_code == 400
|
||||
assert "name" in resp.json()["error"].lower()
|
||||
|
||||
def test_create_policy_missing_tool_pattern(self, client):
|
||||
resp = client.post(
|
||||
"/v1/api/admin/policies",
|
||||
json=_policy_payload(tool_pattern=""),
|
||||
)
|
||||
assert resp.status_code == 400
|
||||
assert "tool_pattern" in resp.json()["error"].lower()
|
||||
|
||||
def test_create_policy_invalid_action(self, client):
|
||||
resp = client.post(
|
||||
"/v1/api/admin/policies",
|
||||
json=_policy_payload(action="yolo"),
|
||||
)
|
||||
assert resp.status_code == 400
|
||||
assert "action" in resp.json()["error"].lower()
|
||||
|
||||
def test_list_after_create(self, client):
|
||||
client.post("/v1/api/admin/policies", json=_policy_payload())
|
||||
resp = client.get("/v1/api/admin/policies")
|
||||
assert resp.status_code == 200
|
||||
policies = resp.json()["policies"]
|
||||
assert len(policies) == 1
|
||||
assert policies[0]["name"] == "Allow bash"
|
||||
|
||||
def test_update_policy(self, client):
|
||||
create_resp = client.post("/v1/api/admin/policies", json=_policy_payload())
|
||||
policy_id = create_resp.json()["policy_id"]
|
||||
|
||||
resp = client.put(
|
||||
f"/v1/api/admin/policies/{policy_id}",
|
||||
json={"name": "Deny bash", "action": "deny", "priority": 20},
|
||||
)
|
||||
assert resp.status_code == 200
|
||||
policy = resp.json()
|
||||
assert policy["name"] == "Deny bash"
|
||||
assert policy["action"] == "deny"
|
||||
assert policy["priority"] == 20
|
||||
|
||||
def test_update_policy_invalid_action(self, client):
|
||||
create_resp = client.post("/v1/api/admin/policies", json=_policy_payload())
|
||||
policy_id = create_resp.json()["policy_id"]
|
||||
|
||||
resp = client.put(
|
||||
f"/v1/api/admin/policies/{policy_id}",
|
||||
json={"action": "nope"},
|
||||
)
|
||||
assert resp.status_code == 400
|
||||
assert "action" in resp.json()["error"].lower()
|
||||
|
||||
def test_update_policy_not_found(self, client):
|
||||
resp = client.put(
|
||||
"/v1/api/admin/policies/nonexistent",
|
||||
json={"name": "Nope"},
|
||||
)
|
||||
assert resp.status_code == 404
|
||||
|
||||
def test_delete_policy(self, client):
|
||||
create_resp = client.post("/v1/api/admin/policies", json=_policy_payload())
|
||||
policy_id = create_resp.json()["policy_id"]
|
||||
|
||||
resp = client.delete(f"/v1/api/admin/policies/{policy_id}")
|
||||
assert resp.status_code == 200
|
||||
assert resp.json()["status"] == "ok"
|
||||
|
||||
# Verify gone
|
||||
list_resp = client.get("/v1/api/admin/policies")
|
||||
assert list_resp.json()["policies"] == []
|
||||
|
||||
def test_delete_policy_not_found(self, client):
|
||||
resp = client.delete("/v1/api/admin/policies/nonexistent")
|
||||
assert resp.status_code == 404
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Tests — Prompt templates
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestTemplates:
|
||||
def test_list_empty(self, client):
|
||||
resp = client.get("/v1/api/admin/templates")
|
||||
assert resp.status_code == 200
|
||||
assert resp.json()["templates"] == []
|
||||
|
||||
def test_create_template(self, client):
|
||||
resp = client.post("/v1/api/admin/templates", json=_template_payload())
|
||||
assert resp.status_code == 200
|
||||
tmpl = resp.json()
|
||||
assert tmpl["name"] == "Greeting"
|
||||
assert "{{user}}" in tmpl["content"]
|
||||
assert tmpl["category"] == "system"
|
||||
assert "template_id" in tmpl
|
||||
assert "created" in tmpl
|
||||
|
||||
def test_create_template_missing_name(self, client):
|
||||
resp = client.post(
|
||||
"/v1/api/admin/templates",
|
||||
json=_template_payload(name=""),
|
||||
)
|
||||
assert resp.status_code == 400
|
||||
assert "name" in resp.json()["error"].lower()
|
||||
|
||||
def test_create_template_missing_content(self, client):
|
||||
resp = client.post(
|
||||
"/v1/api/admin/templates",
|
||||
json=_template_payload(content=""),
|
||||
)
|
||||
assert resp.status_code == 400
|
||||
assert "content" in resp.json()["error"].lower()
|
||||
|
||||
def test_list_after_create(self, client):
|
||||
client.post("/v1/api/admin/templates", json=_template_payload())
|
||||
resp = client.get("/v1/api/admin/templates")
|
||||
assert resp.status_code == 200
|
||||
templates = resp.json()["templates"]
|
||||
assert len(templates) == 1
|
||||
assert templates[0]["name"] == "Greeting"
|
||||
|
||||
def test_update_template(self, client):
|
||||
create_resp = client.post("/v1/api/admin/templates", json=_template_payload())
|
||||
template_id = create_resp.json()["template_id"]
|
||||
|
||||
resp = client.put(
|
||||
f"/v1/api/admin/templates/{template_id}",
|
||||
json={"name": "Welcome", "content": "Welcome, {{user}}!", "is_default": True},
|
||||
)
|
||||
assert resp.status_code == 200
|
||||
tmpl = resp.json()
|
||||
assert tmpl["name"] == "Welcome"
|
||||
assert tmpl["content"] == "Welcome, {{user}}!"
|
||||
assert tmpl["is_default"] is True
|
||||
|
||||
def test_update_template_not_found(self, client):
|
||||
resp = client.put(
|
||||
"/v1/api/admin/templates/nonexistent",
|
||||
json={"name": "Nope"},
|
||||
)
|
||||
assert resp.status_code == 404
|
||||
|
||||
def test_delete_template(self, client):
|
||||
create_resp = client.post("/v1/api/admin/templates", json=_template_payload())
|
||||
template_id = create_resp.json()["template_id"]
|
||||
|
||||
resp = client.delete(f"/v1/api/admin/templates/{template_id}")
|
||||
assert resp.status_code == 200
|
||||
assert resp.json()["status"] == "ok"
|
||||
|
||||
# Verify gone
|
||||
list_resp = client.get("/v1/api/admin/templates")
|
||||
assert list_resp.json()["templates"] == []
|
||||
|
||||
def test_delete_template_not_found(self, client):
|
||||
resp = client.delete("/v1/api/admin/templates/nonexistent")
|
||||
assert resp.status_code == 404
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Tests — Usage
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestUsage:
|
||||
def test_usage_defaults(self, client):
|
||||
"""Query usage with no params — should return summary and breakdown."""
|
||||
resp = client.get("/v1/api/admin/usage")
|
||||
assert resp.status_code == 200
|
||||
data = resp.json()
|
||||
assert "summary" in data
|
||||
assert "breakdown" in data
|
||||
# Summary is a list with at least one row
|
||||
assert isinstance(data["summary"], list)
|
||||
assert len(data["summary"]) >= 1
|
||||
# All-zeros when no data
|
||||
assert data["summary"][0]["prompt_tokens"] == 0
|
||||
|
||||
def test_usage_with_data(self, client, storage):
|
||||
"""Seed usage events and verify they appear in the query."""
|
||||
storage.record_usage_event(
|
||||
event_id="evt-1",
|
||||
user_id="user-1",
|
||||
model="gpt-5",
|
||||
prompt_tokens=100,
|
||||
completion_tokens=50,
|
||||
tool_calls_count=2,
|
||||
)
|
||||
storage.record_usage_event(
|
||||
event_id="evt-2",
|
||||
user_id="user-1",
|
||||
model="gpt-5",
|
||||
prompt_tokens=200,
|
||||
completion_tokens=75,
|
||||
tool_calls_count=1,
|
||||
)
|
||||
resp = client.get("/v1/api/admin/usage")
|
||||
assert resp.status_code == 200
|
||||
summary = resp.json()["summary"]
|
||||
assert summary[0]["prompt_tokens"] == 300
|
||||
assert summary[0]["completion_tokens"] == 125
|
||||
assert summary[0]["tool_calls_count"] == 3
|
||||
|
||||
def test_usage_with_filters(self, client, storage):
|
||||
storage.record_usage_event(
|
||||
event_id="evt-f1",
|
||||
user_id="user-a",
|
||||
model="gpt-5",
|
||||
prompt_tokens=100,
|
||||
completion_tokens=10,
|
||||
)
|
||||
storage.record_usage_event(
|
||||
event_id="evt-f2",
|
||||
user_id="user-b",
|
||||
model="claude-4",
|
||||
prompt_tokens=200,
|
||||
completion_tokens=20,
|
||||
)
|
||||
resp = client.get("/v1/api/admin/usage?user_id=user-a")
|
||||
assert resp.status_code == 200
|
||||
summary = resp.json()["summary"]
|
||||
assert summary[0]["prompt_tokens"] == 100
|
||||
|
||||
resp2 = client.get("/v1/api/admin/usage?model=claude-4")
|
||||
assert resp2.status_code == 200
|
||||
summary2 = resp2.json()["summary"]
|
||||
assert summary2[0]["prompt_tokens"] == 200
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Tests — Audit
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestAudit:
|
||||
def test_audit_empty(self, client):
|
||||
resp = client.get("/v1/api/admin/audit")
|
||||
assert resp.status_code == 200
|
||||
data = resp.json()
|
||||
assert data["events"] == []
|
||||
assert data["total"] == 0
|
||||
|
||||
def test_audit_populated_by_mutations(self, client):
|
||||
"""Creating a role should produce an audit event."""
|
||||
client.post("/v1/api/admin/roles", json=_role_payload())
|
||||
|
||||
resp = client.get("/v1/api/admin/audit")
|
||||
assert resp.status_code == 200
|
||||
data = resp.json()
|
||||
assert data["total"] >= 1
|
||||
actions = [e["action"] for e in data["events"]]
|
||||
assert "role.create" in actions
|
||||
|
||||
def test_audit_filter_by_action(self, client):
|
||||
# Create a role and a policy to produce different audit actions
|
||||
client.post("/v1/api/admin/roles", json=_role_payload())
|
||||
client.post("/v1/api/admin/policies", json=_policy_payload())
|
||||
|
||||
resp = client.get("/v1/api/admin/audit?action=policy.create")
|
||||
assert resp.status_code == 200
|
||||
data = resp.json()
|
||||
assert data["total"] >= 1
|
||||
assert all(e["action"] == "policy.create" for e in data["events"])
|
||||
|
||||
def test_audit_filter_by_user_id(self, client):
|
||||
client.post("/v1/api/admin/roles", json=_role_payload())
|
||||
|
||||
resp = client.get("/v1/api/admin/audit?user_id=test-admin")
|
||||
assert resp.status_code == 200
|
||||
data = resp.json()
|
||||
assert data["total"] >= 1
|
||||
assert all(e["user_id"] == "test-admin" for e in data["events"])
|
||||
|
||||
def test_audit_pagination(self, client):
|
||||
# Create several resources to produce multiple audit events
|
||||
for i in range(5):
|
||||
client.post(
|
||||
"/v1/api/admin/roles",
|
||||
json=_role_payload(name=f"role-{i}"),
|
||||
)
|
||||
|
||||
resp = client.get("/v1/api/admin/audit?limit=2&offset=0")
|
||||
assert resp.status_code == 200
|
||||
data = resp.json()
|
||||
assert len(data["events"]) == 2
|
||||
assert data["total"] >= 5
|
||||
|
||||
resp2 = client.get("/v1/api/admin/audit?limit=2&offset=2")
|
||||
assert resp2.status_code == 200
|
||||
data2 = resp2.json()
|
||||
assert len(data2["events"]) == 2
|
||||
# The two pages should not overlap
|
||||
ids_page1 = {e["event_id"] for e in data["events"]}
|
||||
ids_page2 = {e["event_id"] for e in data2["events"]}
|
||||
assert ids_page1.isdisjoint(ids_page2)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Tests — User self-deletion guard
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestUserSelfDeletion:
|
||||
def test_cannot_delete_self(self, client):
|
||||
"""Admin should not be able to delete their own account."""
|
||||
resp = client.delete("/v1/api/admin/users/test-admin")
|
||||
assert resp.status_code == 400
|
||||
assert "own account" in resp.json()["error"].lower()
|
||||
|
||||
def test_can_delete_other_user(self, client):
|
||||
resp = client.delete("/v1/api/admin/users/user-1")
|
||||
assert resp.status_code == 200
|
||||
assert resp.json()["status"] == "ok"
|
||||
@@ -0,0 +1,746 @@
|
||||
"""Tests for governance storage operations (SQLite backend).
|
||||
|
||||
Covers RBAC roles, organizations, tool policies, prompt templates,
|
||||
usage events, and audit events.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import UTC, datetime
|
||||
|
||||
import pytest
|
||||
import sqlalchemy as sa
|
||||
|
||||
from turnstone.core.storage._sqlite import SQLiteBackend
|
||||
|
||||
|
||||
@pytest.fixture()
|
||||
def db(tmp_path):
|
||||
"""Create a fresh SQLite backend for each test."""
|
||||
return SQLiteBackend(str(tmp_path / "test.db"))
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Roles
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestRoleCRUD:
|
||||
def test_create_role(self, db):
|
||||
db.create_role("r1", "editor", "Editor", "read,write", builtin=False, org_id="")
|
||||
role = db.get_role("r1")
|
||||
assert role is not None
|
||||
assert role["role_id"] == "r1"
|
||||
assert role["name"] == "editor"
|
||||
assert role["display_name"] == "Editor"
|
||||
assert role["permissions"] == "read,write"
|
||||
assert role["builtin"] is False
|
||||
assert role["org_id"] == ""
|
||||
assert "created" in role
|
||||
assert "updated" in role
|
||||
|
||||
def test_create_role_idempotent(self, db):
|
||||
db.create_role("r1", "editor", "Editor", "read,write", builtin=False, org_id="")
|
||||
# Second insert with same role_id should be silently ignored.
|
||||
db.create_role("r1", "editor2", "Editor 2", "read", builtin=True, org_id="org1")
|
||||
role = db.get_role("r1")
|
||||
assert role is not None
|
||||
# Original values preserved.
|
||||
assert role["name"] == "editor"
|
||||
assert role["display_name"] == "Editor"
|
||||
|
||||
def test_get_role_by_name(self, db):
|
||||
db.create_role("r1", "editor", "Editor", "read,write", builtin=False, org_id="")
|
||||
role = db.get_role_by_name("editor")
|
||||
assert role is not None
|
||||
assert role["role_id"] == "r1"
|
||||
|
||||
def test_get_role_by_name_nonexistent(self, db):
|
||||
assert db.get_role_by_name("nope") is None
|
||||
|
||||
def test_list_roles(self, db):
|
||||
db.create_role("r2", "beta", "Beta Role", "read", builtin=False, org_id="")
|
||||
db.create_role("r1", "alpha", "Alpha Role", "write", builtin=False, org_id="")
|
||||
roles = db.list_roles()
|
||||
assert len(roles) == 2
|
||||
# Ordered by name ascending.
|
||||
assert roles[0]["name"] == "alpha"
|
||||
assert roles[1]["name"] == "beta"
|
||||
|
||||
def test_list_roles_filter_org(self, db):
|
||||
db.create_role("r1", "role_a", "A", "read", builtin=False, org_id="org1")
|
||||
db.create_role("r2", "role_b", "B", "read", builtin=False, org_id="org2")
|
||||
db.create_role("r3", "role_c", "C", "read", builtin=False, org_id="org1")
|
||||
result = db.list_roles(org_id="org1")
|
||||
assert len(result) == 2
|
||||
assert {r["role_id"] for r in result} == {"r1", "r3"}
|
||||
|
||||
def test_update_role(self, db):
|
||||
db.create_role("r1", "editor", "Editor", "read,write", builtin=False, org_id="")
|
||||
ok = db.update_role("r1", permissions="read,write,approve", display_name="Senior Editor")
|
||||
assert ok is True
|
||||
role = db.get_role("r1")
|
||||
assert role is not None
|
||||
assert role["permissions"] == "read,write,approve"
|
||||
assert role["display_name"] == "Senior Editor"
|
||||
|
||||
def test_update_role_nonexistent(self, db):
|
||||
assert db.update_role("missing", permissions="read") is False
|
||||
|
||||
def test_delete_role(self, db):
|
||||
db.create_role("r1", "editor", "Editor", "read", builtin=False, org_id="")
|
||||
db.create_user("u1", "alice", "Alice", "$2b$hash")
|
||||
db.assign_role("u1", "r1")
|
||||
# Verify assignment exists.
|
||||
assert len(db.list_user_roles("u1")) == 1
|
||||
ok = db.delete_role("r1")
|
||||
assert ok is True
|
||||
assert db.get_role("r1") is None
|
||||
# Cascade: user_roles for this role should be gone.
|
||||
assert len(db.list_user_roles("u1")) == 0
|
||||
|
||||
def test_delete_role_nonexistent(self, db):
|
||||
assert db.delete_role("missing") is False
|
||||
|
||||
def test_assign_role(self, db):
|
||||
db.create_role("r1", "editor", "Editor", "read,write", builtin=False, org_id="")
|
||||
db.create_user("u1", "alice", "Alice", "$2b$hash")
|
||||
db.assign_role("u1", "r1", assigned_by="admin")
|
||||
roles = db.list_user_roles("u1")
|
||||
assert len(roles) == 1
|
||||
assert roles[0]["role_id"] == "r1"
|
||||
assert roles[0]["assigned_by"] == "admin"
|
||||
|
||||
def test_assign_role_idempotent(self, db):
|
||||
db.create_role("r1", "editor", "Editor", "read,write", builtin=False, org_id="")
|
||||
db.create_user("u1", "alice", "Alice", "$2b$hash")
|
||||
db.assign_role("u1", "r1")
|
||||
# Second assign should not raise.
|
||||
db.assign_role("u1", "r1")
|
||||
roles = db.list_user_roles("u1")
|
||||
assert len(roles) == 1
|
||||
|
||||
def test_unassign_role(self, db):
|
||||
db.create_role("r1", "editor", "Editor", "read,write", builtin=False, org_id="")
|
||||
db.create_user("u1", "alice", "Alice", "$2b$hash")
|
||||
db.assign_role("u1", "r1")
|
||||
ok = db.unassign_role("u1", "r1")
|
||||
assert ok is True
|
||||
assert len(db.list_user_roles("u1")) == 0
|
||||
|
||||
def test_unassign_role_nonexistent(self, db):
|
||||
assert db.unassign_role("u1", "r1") is False
|
||||
|
||||
def test_list_user_roles(self, db):
|
||||
db.create_role("r1", "editor", "Editor", "read,write", builtin=False, org_id="")
|
||||
db.create_role("r2", "viewer", "Viewer", "read", builtin=True, org_id="")
|
||||
db.create_user("u1", "alice", "Alice", "$2b$hash")
|
||||
db.assign_role("u1", "r1", assigned_by="admin")
|
||||
db.assign_role("u1", "r2", assigned_by="system")
|
||||
roles = db.list_user_roles("u1")
|
||||
assert len(roles) == 2
|
||||
# Each entry should have joined role fields plus assignment metadata.
|
||||
for r in roles:
|
||||
assert "role_id" in r
|
||||
assert "name" in r
|
||||
assert "permissions" in r
|
||||
assert "assigned_by" in r
|
||||
assert "assignment_created" in r
|
||||
|
||||
def test_get_user_permissions(self, db):
|
||||
db.create_role("r1", "editor", "Editor", "read,write", builtin=False, org_id="")
|
||||
db.create_role("r2", "approver", "Approver", "approve,read", builtin=False, org_id="")
|
||||
db.create_user("u1", "alice", "Alice", "$2b$hash")
|
||||
db.assign_role("u1", "r1")
|
||||
db.assign_role("u1", "r2")
|
||||
perms = db.get_user_permissions("u1")
|
||||
assert perms == {"read", "write", "approve"}
|
||||
|
||||
def test_get_user_permissions_no_roles(self, db):
|
||||
db.create_user("u1", "alice", "Alice", "$2b$hash")
|
||||
assert db.get_user_permissions("u1") == set()
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Organizations
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestOrgCRUD:
|
||||
def test_create_org(self, db):
|
||||
db.create_org("org1", "acme", "Acme Corp", '{"plan":"pro"}')
|
||||
org = db.get_org("org1")
|
||||
assert org is not None
|
||||
assert org["org_id"] == "org1"
|
||||
assert org["name"] == "acme"
|
||||
assert org["display_name"] == "Acme Corp"
|
||||
assert org["settings"] == '{"plan":"pro"}'
|
||||
assert "created" in org
|
||||
assert "updated" in org
|
||||
|
||||
def test_get_org_nonexistent(self, db):
|
||||
assert db.get_org("nope") is None
|
||||
|
||||
def test_create_org_idempotent(self, db):
|
||||
db.create_org("org1", "acme", "Acme Corp")
|
||||
db.create_org("org1", "acme2", "Acme 2")
|
||||
org = db.get_org("org1")
|
||||
assert org is not None
|
||||
assert org["name"] == "acme"
|
||||
|
||||
def test_list_orgs(self, db):
|
||||
db.create_org("o2", "beta", "Beta Inc")
|
||||
db.create_org("o1", "alpha", "Alpha LLC")
|
||||
orgs = db.list_orgs()
|
||||
assert len(orgs) == 2
|
||||
# Ordered by name ascending.
|
||||
assert orgs[0]["name"] == "alpha"
|
||||
assert orgs[1]["name"] == "beta"
|
||||
|
||||
def test_update_org(self, db):
|
||||
db.create_org("org1", "acme", "Acme Corp")
|
||||
ok = db.update_org(
|
||||
"org1", display_name="Acme Corp Global", settings='{"plan":"enterprise"}'
|
||||
)
|
||||
assert ok is True
|
||||
org = db.get_org("org1")
|
||||
assert org is not None
|
||||
assert org["display_name"] == "Acme Corp Global"
|
||||
assert org["settings"] == '{"plan":"enterprise"}'
|
||||
|
||||
def test_update_org_nonexistent(self, db):
|
||||
assert db.update_org("missing", display_name="X") is False
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Tool Policies
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestToolPolicyCRUD:
|
||||
def test_create_tool_policy(self, db):
|
||||
db.create_tool_policy(
|
||||
"p1",
|
||||
"deny-bash",
|
||||
"bash*",
|
||||
"deny",
|
||||
priority=100,
|
||||
org_id="org1",
|
||||
enabled=True,
|
||||
created_by="admin",
|
||||
)
|
||||
pol = db.get_tool_policy("p1")
|
||||
assert pol is not None
|
||||
assert pol["policy_id"] == "p1"
|
||||
assert pol["name"] == "deny-bash"
|
||||
assert pol["tool_pattern"] == "bash*"
|
||||
assert pol["action"] == "deny"
|
||||
assert pol["priority"] == 100
|
||||
assert pol["org_id"] == "org1"
|
||||
assert pol["enabled"] is True
|
||||
assert pol["created_by"] == "admin"
|
||||
|
||||
def test_get_tool_policy_nonexistent(self, db):
|
||||
assert db.get_tool_policy("missing") is None
|
||||
|
||||
def test_list_tool_policies_ordered_by_priority(self, db):
|
||||
db.create_tool_policy("p1", "low", "*", "allow", priority=10)
|
||||
db.create_tool_policy("p2", "high", "*", "deny", priority=100)
|
||||
db.create_tool_policy("p3", "mid", "*", "ask", priority=50)
|
||||
policies = db.list_tool_policies()
|
||||
assert len(policies) == 3
|
||||
# DESC priority order.
|
||||
assert policies[0]["priority"] == 100
|
||||
assert policies[1]["priority"] == 50
|
||||
assert policies[2]["priority"] == 10
|
||||
|
||||
def test_update_tool_policy(self, db):
|
||||
db.create_tool_policy("p1", "deny-bash", "bash*", "deny", priority=100)
|
||||
ok = db.update_tool_policy("p1", action="allow", priority=50)
|
||||
assert ok is True
|
||||
pol = db.get_tool_policy("p1")
|
||||
assert pol is not None
|
||||
assert pol["action"] == "allow"
|
||||
assert pol["priority"] == 50
|
||||
|
||||
def test_update_tool_policy_nonexistent(self, db):
|
||||
assert db.update_tool_policy("missing", action="deny") is False
|
||||
|
||||
def test_delete_tool_policy(self, db):
|
||||
db.create_tool_policy("p1", "deny-bash", "bash*", "deny", priority=100)
|
||||
ok = db.delete_tool_policy("p1")
|
||||
assert ok is True
|
||||
assert db.get_tool_policy("p1") is None
|
||||
|
||||
def test_delete_tool_policy_nonexistent(self, db):
|
||||
assert db.delete_tool_policy("missing") is False
|
||||
|
||||
def test_enabled_as_bool(self, db):
|
||||
db.create_tool_policy("p1", "on", "*", "allow", priority=0, enabled=True)
|
||||
db.create_tool_policy("p2", "off", "*", "deny", priority=0, enabled=False)
|
||||
p1 = db.get_tool_policy("p1")
|
||||
p2 = db.get_tool_policy("p2")
|
||||
assert p1 is not None
|
||||
assert p2 is not None
|
||||
assert p1["enabled"] is True
|
||||
assert isinstance(p1["enabled"], bool)
|
||||
assert p2["enabled"] is False
|
||||
assert isinstance(p2["enabled"], bool)
|
||||
|
||||
def test_list_policies_filter_org(self, db):
|
||||
db.create_tool_policy("p1", "a", "*", "allow", priority=0, org_id="org1")
|
||||
db.create_tool_policy("p2", "b", "*", "deny", priority=0, org_id="org2")
|
||||
db.create_tool_policy("p3", "c", "*", "ask", priority=0, org_id="org1")
|
||||
result = db.list_tool_policies(org_id="org1")
|
||||
assert len(result) == 2
|
||||
assert {r["policy_id"] for r in result} == {"p1", "p3"}
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Prompt Templates
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestPromptTemplateCRUD:
|
||||
def test_create_prompt_template(self, db):
|
||||
db.create_prompt_template(
|
||||
"t1",
|
||||
"greeting",
|
||||
"general",
|
||||
"Hello {{name}}!",
|
||||
variables='["name"]',
|
||||
is_default=True,
|
||||
org_id="org1",
|
||||
created_by="admin",
|
||||
)
|
||||
tpl = db.get_prompt_template("t1")
|
||||
assert tpl is not None
|
||||
assert tpl["template_id"] == "t1"
|
||||
assert tpl["name"] == "greeting"
|
||||
assert tpl["category"] == "general"
|
||||
assert tpl["content"] == "Hello {{name}}!"
|
||||
assert tpl["variables"] == '["name"]'
|
||||
assert tpl["is_default"] is True
|
||||
assert tpl["org_id"] == "org1"
|
||||
assert tpl["created_by"] == "admin"
|
||||
|
||||
def test_get_prompt_template_nonexistent(self, db):
|
||||
assert db.get_prompt_template("missing") is None
|
||||
|
||||
def test_list_prompt_templates_ordered_by_name(self, db):
|
||||
db.create_prompt_template("t2", "beta", "general", "B")
|
||||
db.create_prompt_template("t1", "alpha", "general", "A")
|
||||
templates = db.list_prompt_templates()
|
||||
assert len(templates) == 2
|
||||
assert templates[0]["name"] == "alpha"
|
||||
assert templates[1]["name"] == "beta"
|
||||
|
||||
def test_list_prompt_templates_filter_org(self, db):
|
||||
db.create_prompt_template("t1", "a", "general", "A", org_id="org1")
|
||||
db.create_prompt_template("t2", "b", "general", "B", org_id="org2")
|
||||
result = db.list_prompt_templates(org_id="org1")
|
||||
assert len(result) == 1
|
||||
assert result[0]["template_id"] == "t1"
|
||||
|
||||
def test_update_prompt_template(self, db):
|
||||
db.create_prompt_template("t1", "greeting", "general", "Hello!")
|
||||
ok = db.update_prompt_template("t1", content="Hi there!", category="custom")
|
||||
assert ok is True
|
||||
tpl = db.get_prompt_template("t1")
|
||||
assert tpl is not None
|
||||
assert tpl["content"] == "Hi there!"
|
||||
assert tpl["category"] == "custom"
|
||||
|
||||
def test_update_prompt_template_nonexistent(self, db):
|
||||
assert db.update_prompt_template("missing", content="x") is False
|
||||
|
||||
def test_delete_prompt_template(self, db):
|
||||
db.create_prompt_template("t1", "greeting", "general", "Hello!")
|
||||
ok = db.delete_prompt_template("t1")
|
||||
assert ok is True
|
||||
assert db.get_prompt_template("t1") is None
|
||||
|
||||
def test_delete_prompt_template_nonexistent(self, db):
|
||||
assert db.delete_prompt_template("missing") is False
|
||||
|
||||
def test_is_default_as_bool(self, db):
|
||||
db.create_prompt_template("t1", "default_one", "general", "D", is_default=True)
|
||||
db.create_prompt_template("t2", "not_default", "general", "N", is_default=False)
|
||||
t1 = db.get_prompt_template("t1")
|
||||
t2 = db.get_prompt_template("t2")
|
||||
assert t1 is not None
|
||||
assert t2 is not None
|
||||
assert t1["is_default"] is True
|
||||
assert isinstance(t1["is_default"], bool)
|
||||
assert t2["is_default"] is False
|
||||
assert isinstance(t2["is_default"], bool)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Usage Events
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestUsageEvents:
|
||||
def test_record_usage_event(self, db):
|
||||
db.record_usage_event(
|
||||
"ev1",
|
||||
user_id="u1",
|
||||
ws_id="ws1",
|
||||
node_id="n1",
|
||||
model="gpt-5",
|
||||
prompt_tokens=100,
|
||||
completion_tokens=50,
|
||||
tool_calls_count=2,
|
||||
)
|
||||
# Verify via query_usage (no group_by returns summary).
|
||||
result = db.query_usage(since="2000-01-01T00:00:00")
|
||||
assert len(result) == 1
|
||||
assert result[0]["prompt_tokens"] == 100
|
||||
assert result[0]["completion_tokens"] == 50
|
||||
assert result[0]["tool_calls_count"] == 2
|
||||
|
||||
def test_query_usage_summary(self, db):
|
||||
db.record_usage_event("ev1", model="gpt-5", prompt_tokens=100, completion_tokens=50)
|
||||
db.record_usage_event("ev2", model="gpt-5", prompt_tokens=200, completion_tokens=75)
|
||||
result = db.query_usage(since="2000-01-01T00:00:00")
|
||||
assert len(result) == 1
|
||||
assert result[0]["prompt_tokens"] == 300
|
||||
assert result[0]["completion_tokens"] == 125
|
||||
|
||||
def test_query_usage_by_day(self, db):
|
||||
# Insert events with known timestamps by directly inserting rows.
|
||||
from turnstone.core.storage._schema import usage_events
|
||||
|
||||
with db._engine.connect() as conn:
|
||||
conn.execute(
|
||||
sa.insert(usage_events),
|
||||
[
|
||||
{
|
||||
"event_id": "e1",
|
||||
"timestamp": "2026-03-01T10:00:00",
|
||||
"user_id": "",
|
||||
"ws_id": "",
|
||||
"node_id": "",
|
||||
"model": "gpt-5",
|
||||
"prompt_tokens": 100,
|
||||
"completion_tokens": 50,
|
||||
"tool_calls_count": 0,
|
||||
"created": "2026-03-01T10:00:00",
|
||||
},
|
||||
{
|
||||
"event_id": "e2",
|
||||
"timestamp": "2026-03-01T14:00:00",
|
||||
"user_id": "",
|
||||
"ws_id": "",
|
||||
"node_id": "",
|
||||
"model": "gpt-5",
|
||||
"prompt_tokens": 50,
|
||||
"completion_tokens": 25,
|
||||
"tool_calls_count": 0,
|
||||
"created": "2026-03-01T14:00:00",
|
||||
},
|
||||
{
|
||||
"event_id": "e3",
|
||||
"timestamp": "2026-03-02T08:00:00",
|
||||
"user_id": "",
|
||||
"ws_id": "",
|
||||
"node_id": "",
|
||||
"model": "gpt-5",
|
||||
"prompt_tokens": 200,
|
||||
"completion_tokens": 100,
|
||||
"tool_calls_count": 0,
|
||||
"created": "2026-03-02T08:00:00",
|
||||
},
|
||||
],
|
||||
)
|
||||
conn.commit()
|
||||
|
||||
result = db.query_usage(since="2026-03-01T00:00:00", group_by="day")
|
||||
assert len(result) == 2
|
||||
assert result[0]["key"] == "2026-03-01"
|
||||
assert result[0]["prompt_tokens"] == 150
|
||||
assert result[1]["key"] == "2026-03-02"
|
||||
assert result[1]["prompt_tokens"] == 200
|
||||
|
||||
def test_query_usage_by_model(self, db):
|
||||
from turnstone.core.storage._schema import usage_events
|
||||
|
||||
with db._engine.connect() as conn:
|
||||
conn.execute(
|
||||
sa.insert(usage_events),
|
||||
[
|
||||
{
|
||||
"event_id": "e1",
|
||||
"timestamp": "2026-03-01T10:00:00",
|
||||
"user_id": "",
|
||||
"ws_id": "",
|
||||
"node_id": "",
|
||||
"model": "gpt-5",
|
||||
"prompt_tokens": 100,
|
||||
"completion_tokens": 50,
|
||||
"tool_calls_count": 0,
|
||||
"created": "2026-03-01T10:00:00",
|
||||
},
|
||||
{
|
||||
"event_id": "e2",
|
||||
"timestamp": "2026-03-01T10:00:00",
|
||||
"user_id": "",
|
||||
"ws_id": "",
|
||||
"node_id": "",
|
||||
"model": "claude-4",
|
||||
"prompt_tokens": 200,
|
||||
"completion_tokens": 100,
|
||||
"tool_calls_count": 1,
|
||||
"created": "2026-03-01T10:00:00",
|
||||
},
|
||||
],
|
||||
)
|
||||
conn.commit()
|
||||
|
||||
result = db.query_usage(since="2026-03-01T00:00:00", group_by="model")
|
||||
assert len(result) == 2
|
||||
keys = [r["key"] for r in result]
|
||||
assert "gpt-5" in keys
|
||||
assert "claude-4" in keys
|
||||
|
||||
def test_query_usage_by_user(self, db):
|
||||
from turnstone.core.storage._schema import usage_events
|
||||
|
||||
with db._engine.connect() as conn:
|
||||
conn.execute(
|
||||
sa.insert(usage_events),
|
||||
[
|
||||
{
|
||||
"event_id": "e1",
|
||||
"timestamp": "2026-03-01T10:00:00",
|
||||
"user_id": "u1",
|
||||
"ws_id": "",
|
||||
"node_id": "",
|
||||
"model": "",
|
||||
"prompt_tokens": 100,
|
||||
"completion_tokens": 50,
|
||||
"tool_calls_count": 0,
|
||||
"created": "2026-03-01T10:00:00",
|
||||
},
|
||||
{
|
||||
"event_id": "e2",
|
||||
"timestamp": "2026-03-01T10:00:00",
|
||||
"user_id": "u2",
|
||||
"ws_id": "",
|
||||
"node_id": "",
|
||||
"model": "",
|
||||
"prompt_tokens": 300,
|
||||
"completion_tokens": 150,
|
||||
"tool_calls_count": 2,
|
||||
"created": "2026-03-01T10:00:00",
|
||||
},
|
||||
],
|
||||
)
|
||||
conn.commit()
|
||||
|
||||
result = db.query_usage(since="2026-03-01T00:00:00", group_by="user")
|
||||
assert len(result) == 2
|
||||
by_key = {r["key"]: r for r in result}
|
||||
assert by_key["u1"]["prompt_tokens"] == 100
|
||||
assert by_key["u2"]["prompt_tokens"] == 300
|
||||
|
||||
def test_query_usage_filter_model(self, db):
|
||||
from turnstone.core.storage._schema import usage_events
|
||||
|
||||
with db._engine.connect() as conn:
|
||||
conn.execute(
|
||||
sa.insert(usage_events),
|
||||
[
|
||||
{
|
||||
"event_id": "e1",
|
||||
"timestamp": "2026-03-01T10:00:00",
|
||||
"user_id": "",
|
||||
"ws_id": "",
|
||||
"node_id": "",
|
||||
"model": "gpt-5",
|
||||
"prompt_tokens": 100,
|
||||
"completion_tokens": 50,
|
||||
"tool_calls_count": 0,
|
||||
"created": "2026-03-01T10:00:00",
|
||||
},
|
||||
{
|
||||
"event_id": "e2",
|
||||
"timestamp": "2026-03-01T10:00:00",
|
||||
"user_id": "",
|
||||
"ws_id": "",
|
||||
"node_id": "",
|
||||
"model": "claude-4",
|
||||
"prompt_tokens": 200,
|
||||
"completion_tokens": 100,
|
||||
"tool_calls_count": 0,
|
||||
"created": "2026-03-01T10:00:00",
|
||||
},
|
||||
],
|
||||
)
|
||||
conn.commit()
|
||||
|
||||
result = db.query_usage(since="2026-03-01T00:00:00", model="gpt-5")
|
||||
assert len(result) == 1
|
||||
assert result[0]["prompt_tokens"] == 100
|
||||
|
||||
def test_prune_usage_events(self, db):
|
||||
from turnstone.core.storage._schema import usage_events
|
||||
|
||||
old_ts = "2020-01-01T00:00:00"
|
||||
now_ts = datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%S")
|
||||
with db._engine.connect() as conn:
|
||||
conn.execute(
|
||||
sa.insert(usage_events),
|
||||
[
|
||||
{
|
||||
"event_id": "old",
|
||||
"timestamp": old_ts,
|
||||
"user_id": "",
|
||||
"ws_id": "",
|
||||
"node_id": "",
|
||||
"model": "",
|
||||
"prompt_tokens": 10,
|
||||
"completion_tokens": 5,
|
||||
"tool_calls_count": 0,
|
||||
"created": old_ts,
|
||||
},
|
||||
{
|
||||
"event_id": "new",
|
||||
"timestamp": now_ts,
|
||||
"user_id": "",
|
||||
"ws_id": "",
|
||||
"node_id": "",
|
||||
"model": "",
|
||||
"prompt_tokens": 20,
|
||||
"completion_tokens": 10,
|
||||
"tool_calls_count": 0,
|
||||
"created": now_ts,
|
||||
},
|
||||
],
|
||||
)
|
||||
conn.commit()
|
||||
|
||||
pruned = db.prune_usage_events(retention_days=30)
|
||||
assert pruned == 1
|
||||
# Only the recent event should remain.
|
||||
result = db.query_usage(since="2000-01-01T00:00:00")
|
||||
assert result[0]["prompt_tokens"] == 20
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Audit Events
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestAuditEvents:
|
||||
def test_record_audit_event(self, db):
|
||||
db.record_audit_event(
|
||||
"a1",
|
||||
user_id="u1",
|
||||
action="role.create",
|
||||
resource_type="role",
|
||||
resource_id="r1",
|
||||
detail='{"name":"editor"}',
|
||||
ip_address="127.0.0.1",
|
||||
)
|
||||
events = db.list_audit_events()
|
||||
assert len(events) == 1
|
||||
ev = events[0]
|
||||
assert ev["event_id"] == "a1"
|
||||
assert ev["user_id"] == "u1"
|
||||
assert ev["action"] == "role.create"
|
||||
assert ev["resource_type"] == "role"
|
||||
assert ev["resource_id"] == "r1"
|
||||
assert ev["detail"] == '{"name":"editor"}'
|
||||
assert ev["ip_address"] == "127.0.0.1"
|
||||
|
||||
def test_list_audit_events(self, db):
|
||||
db.record_audit_event("a1", action="login")
|
||||
db.record_audit_event("a2", action="logout")
|
||||
events = db.list_audit_events()
|
||||
assert len(events) == 2
|
||||
# Ordered by timestamp DESC — most recent first.
|
||||
# Both created in quick succession with same-second granularity,
|
||||
# but the order should still be deterministic (DESC).
|
||||
assert {e["event_id"] for e in events} == {"a1", "a2"}
|
||||
|
||||
def test_list_audit_events_filter_action(self, db):
|
||||
db.record_audit_event("a1", action="login")
|
||||
db.record_audit_event("a2", action="logout")
|
||||
db.record_audit_event("a3", action="login")
|
||||
events = db.list_audit_events(action="login")
|
||||
assert len(events) == 2
|
||||
assert all(e["action"] == "login" for e in events)
|
||||
|
||||
def test_list_audit_events_filter_user(self, db):
|
||||
db.record_audit_event("a1", user_id="u1", action="login")
|
||||
db.record_audit_event("a2", user_id="u2", action="login")
|
||||
events = db.list_audit_events(user_id="u1")
|
||||
assert len(events) == 1
|
||||
assert events[0]["user_id"] == "u1"
|
||||
|
||||
def test_list_audit_events_pagination(self, db):
|
||||
for i in range(5):
|
||||
db.record_audit_event(f"a{i}", action="test")
|
||||
page1 = db.list_audit_events(limit=2, offset=0)
|
||||
page2 = db.list_audit_events(limit=2, offset=2)
|
||||
page3 = db.list_audit_events(limit=2, offset=4)
|
||||
assert len(page1) == 2
|
||||
assert len(page2) == 2
|
||||
assert len(page3) == 1
|
||||
# No overlap.
|
||||
ids = [e["event_id"] for e in page1 + page2 + page3]
|
||||
assert len(set(ids)) == 5
|
||||
|
||||
def test_count_audit_events(self, db):
|
||||
db.record_audit_event("a1", action="login")
|
||||
db.record_audit_event("a2", action="logout")
|
||||
db.record_audit_event("a3", action="login")
|
||||
assert db.count_audit_events() == 3
|
||||
assert db.count_audit_events(action="login") == 2
|
||||
assert db.count_audit_events(action="logout") == 1
|
||||
|
||||
def test_count_audit_events_filter_user(self, db):
|
||||
db.record_audit_event("a1", user_id="u1", action="login")
|
||||
db.record_audit_event("a2", user_id="u2", action="login")
|
||||
assert db.count_audit_events(user_id="u1") == 1
|
||||
|
||||
def test_prune_audit_events(self, db):
|
||||
from turnstone.core.storage._schema import audit_events
|
||||
|
||||
old_ts = "2020-01-01T00:00:00"
|
||||
now_ts = datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%S")
|
||||
with db._engine.connect() as conn:
|
||||
conn.execute(
|
||||
sa.insert(audit_events),
|
||||
[
|
||||
{
|
||||
"event_id": "old",
|
||||
"timestamp": old_ts,
|
||||
"user_id": "",
|
||||
"action": "test",
|
||||
"resource_type": "",
|
||||
"resource_id": "",
|
||||
"detail": "{}",
|
||||
"ip_address": "",
|
||||
"created": old_ts,
|
||||
},
|
||||
{
|
||||
"event_id": "new",
|
||||
"timestamp": now_ts,
|
||||
"user_id": "",
|
||||
"action": "test",
|
||||
"resource_type": "",
|
||||
"resource_id": "",
|
||||
"detail": "{}",
|
||||
"ip_address": "",
|
||||
"created": now_ts,
|
||||
},
|
||||
],
|
||||
)
|
||||
conn.commit()
|
||||
|
||||
pruned = db.prune_audit_events(retention_days=30)
|
||||
assert pruned == 1
|
||||
assert db.count_audit_events() == 1
|
||||
+299
-1
@@ -2,10 +2,11 @@
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
from contextlib import AsyncExitStack
|
||||
from typing import Any
|
||||
from unittest.mock import MagicMock, patch
|
||||
from unittest.mock import AsyncMock, MagicMock, patch
|
||||
|
||||
import pytest
|
||||
|
||||
@@ -411,3 +412,300 @@ class TestCreateMcpClient:
|
||||
|
||||
result = create_mcp_client()
|
||||
assert result is None
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Tool refresh — _rebuild_tools, _refresh_server, listeners
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestRebuildTools:
|
||||
def test_rebuild_from_per_server(self):
|
||||
mgr = MCPClientManager({})
|
||||
mgr._per_server_tools = {
|
||||
"github": [_fake_openai_tool("mcp__github__search")],
|
||||
"slack": [_fake_openai_tool("mcp__slack__send")],
|
||||
}
|
||||
mgr._rebuild_tools()
|
||||
assert len(mgr._tools) == 2
|
||||
names = {t["function"]["name"] for t in mgr._tools}
|
||||
assert names == {"mcp__github__search", "mcp__slack__send"}
|
||||
assert mgr._tool_map["mcp__github__search"] == ("github", "search")
|
||||
assert mgr._tool_map["mcp__slack__send"] == ("slack", "send")
|
||||
|
||||
def test_rebuild_copy_on_write(self):
|
||||
mgr = MCPClientManager({})
|
||||
mgr._per_server_tools = {"a": [_fake_openai_tool("mcp__a__x")]}
|
||||
mgr._rebuild_tools()
|
||||
old_tools = mgr._tools
|
||||
old_map = mgr._tool_map
|
||||
mgr._per_server_tools["b"] = [_fake_openai_tool("mcp__b__y")]
|
||||
mgr._rebuild_tools()
|
||||
assert mgr._tools is not old_tools
|
||||
assert mgr._tool_map is not old_map
|
||||
|
||||
def test_rebuild_empty(self):
|
||||
mgr = MCPClientManager({})
|
||||
mgr._per_server_tools = {}
|
||||
mgr._rebuild_tools()
|
||||
assert mgr._tools == []
|
||||
assert mgr._tool_map == {}
|
||||
|
||||
|
||||
class TestRefreshServer:
|
||||
def test_refresh_detects_added_tools(self):
|
||||
async def _run() -> None:
|
||||
mgr = MCPClientManager({})
|
||||
mock_session = MagicMock()
|
||||
mock_result = MagicMock()
|
||||
mock_result.tools = [
|
||||
_fake_mcp_tool("search"),
|
||||
_fake_mcp_tool("create"), # new tool
|
||||
]
|
||||
mock_session.list_tools = AsyncMock(return_value=mock_result)
|
||||
mgr._sessions["github"] = mock_session
|
||||
mgr._per_server_tools["github"] = [_fake_openai_tool("mcp__github__search")]
|
||||
mgr._rebuild_tools()
|
||||
|
||||
added, removed = await mgr._refresh_server("github")
|
||||
assert "mcp__github__create" in added
|
||||
assert removed == []
|
||||
assert len(mgr._tools) == 2
|
||||
|
||||
asyncio.run(_run())
|
||||
|
||||
def test_refresh_detects_removed_tools(self):
|
||||
async def _run() -> None:
|
||||
mgr = MCPClientManager({})
|
||||
mock_session = MagicMock()
|
||||
mock_result = MagicMock()
|
||||
mock_result.tools = [] # all tools removed
|
||||
mock_session.list_tools = AsyncMock(return_value=mock_result)
|
||||
mgr._sessions["github"] = mock_session
|
||||
mgr._per_server_tools["github"] = [_fake_openai_tool("mcp__github__search")]
|
||||
mgr._rebuild_tools()
|
||||
|
||||
added, removed = await mgr._refresh_server("github")
|
||||
assert added == []
|
||||
assert "mcp__github__search" in removed
|
||||
assert mgr._tools == []
|
||||
|
||||
asyncio.run(_run())
|
||||
|
||||
def test_refresh_no_changes(self):
|
||||
async def _run() -> None:
|
||||
mgr = MCPClientManager({})
|
||||
mock_session = MagicMock()
|
||||
mock_result = MagicMock()
|
||||
mock_result.tools = [_fake_mcp_tool("search")]
|
||||
mock_session.list_tools = AsyncMock(return_value=mock_result)
|
||||
mgr._sessions["github"] = mock_session
|
||||
mgr._per_server_tools["github"] = [_fake_openai_tool("mcp__github__search")]
|
||||
mgr._rebuild_tools()
|
||||
|
||||
added, removed = await mgr._refresh_server("github")
|
||||
assert added == []
|
||||
assert removed == []
|
||||
|
||||
asyncio.run(_run())
|
||||
|
||||
def test_refresh_disconnected_raises(self):
|
||||
async def _run() -> None:
|
||||
mgr = MCPClientManager({})
|
||||
with pytest.raises(RuntimeError, match="not connected"):
|
||||
await mgr._refresh_server("ghost")
|
||||
|
||||
asyncio.run(_run())
|
||||
|
||||
|
||||
class TestListeners:
|
||||
def test_add_and_notify(self):
|
||||
mgr = MCPClientManager({})
|
||||
calls: list[int] = []
|
||||
mgr.add_listener(lambda: calls.append(1))
|
||||
mgr._per_server_tools = {"a": [_fake_openai_tool("mcp__a__x")]}
|
||||
mgr._rebuild_tools()
|
||||
assert len(calls) == 1
|
||||
|
||||
def test_remove_listener(self):
|
||||
mgr = MCPClientManager({})
|
||||
calls: list[int] = []
|
||||
cb = lambda: calls.append(1) # noqa: E731
|
||||
mgr.add_listener(cb)
|
||||
mgr.remove_listener(cb)
|
||||
mgr._rebuild_tools()
|
||||
assert calls == []
|
||||
|
||||
def test_remove_nonexistent_listener(self):
|
||||
mgr = MCPClientManager({})
|
||||
mgr.remove_listener(lambda: None) # should not raise
|
||||
|
||||
def test_listener_error_does_not_propagate(self):
|
||||
mgr = MCPClientManager({})
|
||||
mgr.add_listener(lambda: 1 / 0) # will raise ZeroDivisionError
|
||||
mgr._rebuild_tools() # should not raise
|
||||
|
||||
|
||||
class TestServerNames:
|
||||
def test_server_names_property(self):
|
||||
mgr = MCPClientManager({"github": {}, "slack": {}})
|
||||
assert sorted(mgr.server_names) == ["github", "slack"]
|
||||
|
||||
def test_server_names_empty(self):
|
||||
mgr = MCPClientManager({})
|
||||
assert mgr.server_names == []
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Session integration — tool refresh propagation
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestSessionRefresh:
|
||||
@pytest.fixture()
|
||||
def tmp_db(self, tmp_path):
|
||||
from turnstone.core.storage import init_storage, reset_storage
|
||||
|
||||
reset_storage()
|
||||
init_storage("sqlite", path=str(tmp_path / "test.db"), run_migrations=False)
|
||||
yield
|
||||
reset_storage()
|
||||
|
||||
def _make_session(self, mcp_client=None, **kwargs):
|
||||
from turnstone.core.session import ChatSession
|
||||
|
||||
defaults: dict[str, Any] = dict(
|
||||
client=MagicMock(),
|
||||
model="test-model",
|
||||
ui=MagicMock(),
|
||||
instructions=None,
|
||||
temperature=0.5,
|
||||
max_tokens=4096,
|
||||
tool_timeout=30,
|
||||
mcp_client=mcp_client,
|
||||
)
|
||||
defaults.update(kwargs)
|
||||
return ChatSession(**defaults)
|
||||
|
||||
def test_listener_registered_on_init(self, tmp_db):
|
||||
mock_mcp = MagicMock()
|
||||
mock_mcp.get_tools.return_value = []
|
||||
session = self._make_session(mcp_client=mock_mcp)
|
||||
mock_mcp.add_listener.assert_called_once()
|
||||
assert session._mcp_refresh_cb is not None
|
||||
|
||||
def test_no_listener_without_mcp(self, tmp_db):
|
||||
session = self._make_session(mcp_client=None)
|
||||
assert session._mcp_refresh_cb is None
|
||||
|
||||
def test_close_removes_listener(self, tmp_db):
|
||||
mock_mcp = MagicMock()
|
||||
mock_mcp.get_tools.return_value = []
|
||||
session = self._make_session(mcp_client=mock_mcp)
|
||||
session.close()
|
||||
mock_mcp.remove_listener.assert_called_once()
|
||||
assert session._mcp_refresh_cb is None
|
||||
|
||||
def test_close_idempotent(self, tmp_db):
|
||||
mock_mcp = MagicMock()
|
||||
mock_mcp.get_tools.return_value = []
|
||||
session = self._make_session(mcp_client=mock_mcp)
|
||||
session.close()
|
||||
session.close() # should not raise
|
||||
assert mock_mcp.remove_listener.call_count == 1
|
||||
|
||||
def test_on_mcp_tools_changed_rebuilds_tools(self, tmp_db):
|
||||
mock_mcp = MagicMock()
|
||||
mock_mcp.get_tools.return_value = [_fake_openai_tool("mcp__test__a")]
|
||||
session = self._make_session(mcp_client=mock_mcp)
|
||||
initial_count = len(session._tools)
|
||||
|
||||
# Simulate a tool refresh — MCP now has 2 tools
|
||||
mock_mcp.get_tools.return_value = [
|
||||
_fake_openai_tool("mcp__test__a"),
|
||||
_fake_openai_tool("mcp__test__b"),
|
||||
]
|
||||
session._on_mcp_tools_changed()
|
||||
assert len(session._tools) == initial_count + 1
|
||||
|
||||
def test_tool_search_preserved_across_refresh(self, tmp_db):
|
||||
# Create enough MCP tools to trigger tool search
|
||||
mcp_tools = [_fake_openai_tool(f"mcp__srv__tool{i}") for i in range(25)]
|
||||
mock_mcp = MagicMock()
|
||||
mock_mcp.get_tools.return_value = mcp_tools
|
||||
session = self._make_session(
|
||||
mcp_client=mock_mcp,
|
||||
tool_search="auto",
|
||||
tool_search_threshold=20,
|
||||
)
|
||||
assert session._tool_search is not None
|
||||
|
||||
# Expand a tool
|
||||
session._tool_search.expand_visible(["mcp__srv__tool0"])
|
||||
assert "mcp__srv__tool0" in session._tool_search.get_expanded_names()
|
||||
|
||||
# Refresh with same tools
|
||||
session._on_mcp_tools_changed()
|
||||
assert session._tool_search is not None
|
||||
assert "mcp__srv__tool0" in session._tool_search.get_expanded_names()
|
||||
|
||||
def test_tool_search_prunes_removed_from_expanded(self, tmp_db):
|
||||
mcp_tools = [_fake_openai_tool(f"mcp__srv__tool{i}") for i in range(25)]
|
||||
mock_mcp = MagicMock()
|
||||
mock_mcp.get_tools.return_value = mcp_tools
|
||||
session = self._make_session(
|
||||
mcp_client=mock_mcp,
|
||||
tool_search="auto",
|
||||
tool_search_threshold=20,
|
||||
)
|
||||
session._tool_search.expand_visible(["mcp__srv__tool0"])
|
||||
|
||||
# Refresh with tool0 removed
|
||||
new_tools = [_fake_openai_tool(f"mcp__srv__tool{i}") for i in range(1, 25)]
|
||||
mock_mcp.get_tools.return_value = new_tools
|
||||
session._on_mcp_tools_changed()
|
||||
# tool0 was removed, so it should no longer be in expanded
|
||||
expanded = session._tool_search.get_expanded_names()
|
||||
assert "mcp__srv__tool0" not in expanded
|
||||
|
||||
def test_mcp_refresh_command(self, tmp_db):
|
||||
mock_mcp = MagicMock()
|
||||
mock_mcp.get_tools.return_value = [_fake_openai_tool()]
|
||||
mock_mcp.server_names = ["test"]
|
||||
mock_mcp.refresh_sync.return_value = {"test": (["mcp__test__new"], [])}
|
||||
session = self._make_session(mcp_client=mock_mcp)
|
||||
|
||||
session.handle_command("/mcp refresh")
|
||||
mock_mcp.refresh_sync.assert_called_once_with(None)
|
||||
session.ui.on_info.assert_called()
|
||||
|
||||
def test_mcp_refresh_specific_server(self, tmp_db):
|
||||
mock_mcp = MagicMock()
|
||||
mock_mcp.get_tools.return_value = [_fake_openai_tool()]
|
||||
mock_mcp.server_names = ["github", "slack"]
|
||||
mock_mcp.refresh_sync.return_value = {"github": ([], [])}
|
||||
session = self._make_session(mcp_client=mock_mcp)
|
||||
|
||||
session.handle_command("/mcp refresh github")
|
||||
mock_mcp.refresh_sync.assert_called_once_with("github")
|
||||
|
||||
def test_mcp_refresh_unknown_server(self, tmp_db):
|
||||
mock_mcp = MagicMock()
|
||||
mock_mcp.get_tools.return_value = [_fake_openai_tool()]
|
||||
mock_mcp.server_names = ["github"]
|
||||
session = self._make_session(mcp_client=mock_mcp)
|
||||
|
||||
session.handle_command("/mcp refresh nonexistent")
|
||||
session.ui.on_error.assert_called_once()
|
||||
assert "Unknown MCP server" in session.ui.on_error.call_args[0][0]
|
||||
|
||||
def test_mcp_refresh_error_handling(self, tmp_db):
|
||||
mock_mcp = MagicMock()
|
||||
mock_mcp.get_tools.return_value = [_fake_openai_tool()]
|
||||
mock_mcp.server_names = ["test"]
|
||||
mock_mcp.refresh_sync.side_effect = TimeoutError("timed out")
|
||||
session = self._make_session(mcp_client=mock_mcp)
|
||||
|
||||
session.handle_command("/mcp refresh")
|
||||
session.ui.on_error.assert_called_once()
|
||||
assert "MCP refresh failed" in session.ui.on_error.call_args[0][0]
|
||||
|
||||
@@ -534,7 +534,7 @@ class TestWorkstreamModelParam:
|
||||
nonlocal captured_alias
|
||||
captured_alias = model_alias
|
||||
mock_session = MagicMock()
|
||||
mock_session.session_id = "test123"
|
||||
mock_session.ws_id = "test123"
|
||||
return mock_session
|
||||
|
||||
mgr = WorkstreamManager(factory)
|
||||
@@ -548,7 +548,7 @@ class TestWorkstreamModelParam:
|
||||
nonlocal captured_alias
|
||||
captured_alias = model_alias
|
||||
mock_session = MagicMock()
|
||||
mock_session.session_id = "test123"
|
||||
mock_session.ws_id = "test123"
|
||||
return mock_session
|
||||
|
||||
from turnstone.core.workstream import WorkstreamManager
|
||||
|
||||
@@ -14,7 +14,7 @@ def _make_session() -> ChatSession:
|
||||
from unittest.mock import patch
|
||||
|
||||
with (
|
||||
patch("turnstone.core.session.register_session"),
|
||||
patch("turnstone.core.memory.register_workstream"),
|
||||
patch("turnstone.core.session.save_message"),
|
||||
):
|
||||
from turnstone.core.session import ChatSession
|
||||
|
||||
@@ -27,7 +27,7 @@ class TestServerSpec:
|
||||
expected = {
|
||||
"/v1/api/workstreams",
|
||||
"/v1/api/dashboard",
|
||||
"/v1/api/sessions",
|
||||
"/v1/api/workstreams/saved",
|
||||
"/v1/api/send",
|
||||
"/v1/api/approve",
|
||||
"/v1/api/plan",
|
||||
|
||||
@@ -8,6 +8,7 @@ from turnstone.mq.protocol import (
|
||||
AckEvent,
|
||||
ApprovalRequestEvent,
|
||||
ApproveMessage,
|
||||
CancelMessage,
|
||||
CloseWorkstreamMessage,
|
||||
CommandMessage,
|
||||
ContentEvent,
|
||||
@@ -68,6 +69,7 @@ INBOUND_TYPES = [
|
||||
(ListWorkstreamsMessage, {}),
|
||||
(HealthMessage, {}),
|
||||
(ListNodesMessage, {}),
|
||||
(CancelMessage, {"ws_id": "abc"}),
|
||||
]
|
||||
|
||||
|
||||
|
||||
@@ -1933,3 +1933,254 @@ class TestAnthropicProviderBlocks:
|
||||
assert blocks[1]["input"] == {"query": "test"} # parsed from accumulated JSON
|
||||
assert blocks[2]["type"] == "web_search_tool_result"
|
||||
assert blocks[2]["encrypted_content"] == "enc_data"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Tool search tests
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestAnthropicToolSearch:
|
||||
"""Test Anthropic provider tool search injection."""
|
||||
|
||||
@pytest.fixture()
|
||||
def provider(self):
|
||||
from turnstone.core.providers._anthropic import AnthropicProvider
|
||||
|
||||
return AnthropicProvider()
|
||||
|
||||
def test_tool_search_capability_flag(self, provider):
|
||||
caps = provider.get_capabilities("claude-opus-4-6-20260101")
|
||||
assert caps.supports_tool_search is True
|
||||
|
||||
def test_tool_search_not_supported_on_haiku(self, provider):
|
||||
caps = provider.get_capabilities("claude-haiku-4-5-20251001")
|
||||
assert caps.supports_tool_search is False
|
||||
|
||||
def test_inject_tool_search_marks_deferred(self, provider):
|
||||
caps = provider.get_capabilities("claude-opus-4-6-20260101")
|
||||
tools = [
|
||||
{"name": "bash", "description": "Run commands", "input_schema": {}},
|
||||
{
|
||||
"name": "mcp__github__create_issue",
|
||||
"description": "Create issue",
|
||||
"input_schema": {},
|
||||
},
|
||||
]
|
||||
deferred = frozenset(["mcp__github__create_issue"])
|
||||
result = provider._inject_tool_search(tools, caps, deferred)
|
||||
# bash should not be deferred
|
||||
assert result[0].get("defer_loading") is None or result[0].get("defer_loading") is False
|
||||
# MCP tool should be deferred
|
||||
assert result[1]["defer_loading"] is True
|
||||
# Search tool should be appended
|
||||
assert result[-1]["type"] == "tool_search_tool_bm25_20251119"
|
||||
assert result[-1]["name"] == "tool_search"
|
||||
|
||||
def test_inject_tool_search_no_op_without_deferred(self, provider):
|
||||
caps = provider.get_capabilities("claude-opus-4-6-20260101")
|
||||
tools = [{"name": "bash", "description": "Run commands", "input_schema": {}}]
|
||||
result = provider._inject_tool_search(tools, caps, None)
|
||||
assert result == tools
|
||||
|
||||
def test_inject_tool_search_no_op_on_unsupported_model(self, provider):
|
||||
caps = provider.get_capabilities("claude-haiku-4-5-20251001")
|
||||
tools = [{"name": "bash", "description": "Run commands", "input_schema": {}}]
|
||||
deferred = frozenset(["some_tool"])
|
||||
result = provider._inject_tool_search(tools, caps, deferred)
|
||||
assert result == tools
|
||||
|
||||
|
||||
class TestOpenAIToolSearch:
|
||||
"""Test OpenAI provider tool search injection."""
|
||||
|
||||
@pytest.fixture()
|
||||
def provider(self):
|
||||
return OpenAIProvider()
|
||||
|
||||
def test_tool_search_capability_on_gpt54(self, provider):
|
||||
caps = provider.get_capabilities("gpt-5.4")
|
||||
assert caps.supports_tool_search is True
|
||||
|
||||
def test_tool_search_not_supported_on_gpt5(self, provider):
|
||||
caps = provider.get_capabilities("gpt-5")
|
||||
assert caps.supports_tool_search is False
|
||||
|
||||
def test_apply_tool_search_marks_deferred(self, provider):
|
||||
caps = provider.get_capabilities("gpt-5.4")
|
||||
tools = [
|
||||
{"type": "function", "function": {"name": "bash", "description": "Run commands"}},
|
||||
{
|
||||
"type": "function",
|
||||
"function": {"name": "mcp__slack__send", "description": "Send message"},
|
||||
},
|
||||
]
|
||||
deferred = frozenset(["mcp__slack__send"])
|
||||
result = provider._apply_tool_search(caps, tools, deferred)
|
||||
assert result is not None
|
||||
# bash not deferred
|
||||
assert result[0].get("defer_loading") is None or result[0].get("defer_loading") is False
|
||||
# slack tool deferred
|
||||
assert result[1]["defer_loading"] is True
|
||||
|
||||
def test_apply_tool_search_no_op_without_deferred(self, provider):
|
||||
caps = provider.get_capabilities("gpt-5.4")
|
||||
tools = [
|
||||
{"type": "function", "function": {"name": "bash", "description": "Run commands"}},
|
||||
]
|
||||
result = provider._apply_tool_search(caps, tools, None)
|
||||
assert result == tools
|
||||
|
||||
def test_apply_tool_search_no_op_on_unsupported_model(self, provider):
|
||||
caps = provider.get_capabilities("gpt-5")
|
||||
tools = [
|
||||
{"type": "function", "function": {"name": "bash", "description": "Run commands"}},
|
||||
]
|
||||
deferred = frozenset(["some_tool"])
|
||||
result = provider._apply_tool_search(caps, tools, deferred)
|
||||
assert result == tools
|
||||
|
||||
|
||||
class TestModelCapabilitiesToolSearch:
|
||||
"""Test supports_tool_search defaults and values."""
|
||||
|
||||
def test_default_is_false(self):
|
||||
from turnstone.core.providers._protocol import ModelCapabilities
|
||||
|
||||
caps = ModelCapabilities()
|
||||
assert caps.supports_tool_search is False
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Vision support
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestVisionCapabilities:
|
||||
"""Test supports_vision flag across providers."""
|
||||
|
||||
def test_default_is_false(self) -> None:
|
||||
from turnstone.core.providers._protocol import ModelCapabilities
|
||||
|
||||
caps = ModelCapabilities()
|
||||
assert caps.supports_vision is False
|
||||
|
||||
def test_openai_commercial_supports_vision(self) -> None:
|
||||
provider = OpenAIProvider()
|
||||
for model in ("gpt-5", "gpt-5-mini", "gpt-5.4", "o3", "o4-mini"):
|
||||
caps = provider.get_capabilities(model)
|
||||
assert caps.supports_vision is True, f"{model} should support vision"
|
||||
|
||||
def test_openai_default_no_vision(self) -> None:
|
||||
"""Unknown models (local servers) default to no vision."""
|
||||
provider = OpenAIProvider()
|
||||
caps = provider.get_capabilities("some-local-model")
|
||||
assert caps.supports_vision is False
|
||||
|
||||
def test_anthropic_supports_vision(self) -> None:
|
||||
from turnstone.core.providers._anthropic import AnthropicProvider
|
||||
|
||||
provider = AnthropicProvider()
|
||||
for model in ("claude-opus-4-6", "claude-sonnet-4-6", "claude-haiku-4-5"):
|
||||
caps = provider.get_capabilities(model)
|
||||
assert caps.supports_vision is True, f"{model} should support vision"
|
||||
|
||||
def test_anthropic_default_supports_vision(self) -> None:
|
||||
"""Anthropic default (unknown Claude model) supports vision."""
|
||||
from turnstone.core.providers._anthropic import AnthropicProvider
|
||||
|
||||
provider = AnthropicProvider()
|
||||
caps = provider.get_capabilities("claude-unknown-9")
|
||||
assert caps.supports_vision is True
|
||||
|
||||
|
||||
class TestAnthropicVisionConversion:
|
||||
"""Test image content conversion in _convert_messages."""
|
||||
|
||||
def setup_method(self) -> None:
|
||||
from turnstone.core.providers._anthropic import AnthropicProvider
|
||||
|
||||
self.provider = AnthropicProvider()
|
||||
|
||||
def test_tool_result_with_image_content(self) -> None:
|
||||
"""Tool result with list content converts image_url to Anthropic image."""
|
||||
messages = [
|
||||
{"role": "user", "content": "Read this image"},
|
||||
{
|
||||
"role": "assistant",
|
||||
"content": "",
|
||||
"tool_calls": [
|
||||
{
|
||||
"id": "call_1",
|
||||
"function": {"name": "read_file", "arguments": '{"path": "img.png"}'},
|
||||
}
|
||||
],
|
||||
},
|
||||
{
|
||||
"role": "tool",
|
||||
"tool_call_id": "call_1",
|
||||
"content": [
|
||||
{"type": "text", "text": "Image file: img.png (1024 bytes)"},
|
||||
{
|
||||
"type": "image_url",
|
||||
"image_url": {"url": "data:image/png;base64,iVBORw0KGgo="},
|
||||
},
|
||||
],
|
||||
},
|
||||
]
|
||||
_, converted = self.provider._convert_messages(messages)
|
||||
# Tool result should be in a user message
|
||||
tool_user_msg = converted[2]
|
||||
assert tool_user_msg["role"] == "user"
|
||||
tool_result = tool_user_msg["content"][0]
|
||||
assert tool_result["type"] == "tool_result"
|
||||
assert tool_result["tool_use_id"] == "call_1"
|
||||
# Content should be a list with converted image block
|
||||
content = tool_result["content"]
|
||||
assert isinstance(content, list)
|
||||
assert content[0] == {"type": "text", "text": "Image file: img.png (1024 bytes)"}
|
||||
assert content[1]["type"] == "image"
|
||||
assert content[1]["source"]["type"] == "base64"
|
||||
assert content[1]["source"]["media_type"] == "image/png"
|
||||
assert content[1]["source"]["data"] == "iVBORw0KGgo="
|
||||
|
||||
def test_tool_result_with_string_content_unchanged(self) -> None:
|
||||
"""Tool result with plain string content is unchanged."""
|
||||
messages = [
|
||||
{"role": "user", "content": "Read file"},
|
||||
{
|
||||
"role": "assistant",
|
||||
"content": "",
|
||||
"tool_calls": [
|
||||
{
|
||||
"id": "call_2",
|
||||
"function": {"name": "read_file", "arguments": '{"path": "f.py"}'},
|
||||
}
|
||||
],
|
||||
},
|
||||
{
|
||||
"role": "tool",
|
||||
"tool_call_id": "call_2",
|
||||
"content": " 1\tprint('hello')",
|
||||
},
|
||||
]
|
||||
_, converted = self.provider._convert_messages(messages)
|
||||
tool_result = converted[2]["content"][0]
|
||||
assert tool_result["content"] == " 1\tprint('hello')"
|
||||
|
||||
def test_convert_content_parts_static_method(self) -> None:
|
||||
"""_convert_content_parts handles both image_url and text."""
|
||||
from turnstone.core.providers._anthropic import AnthropicProvider
|
||||
|
||||
parts = [
|
||||
{"type": "text", "text": "description"},
|
||||
{
|
||||
"type": "image_url",
|
||||
"image_url": {"url": "data:image/jpeg;base64,/9j/4AAQ"},
|
||||
},
|
||||
]
|
||||
result = AnthropicProvider._convert_content_parts(parts)
|
||||
assert result[0] == {"type": "text", "text": "description"}
|
||||
assert result[1]["type"] == "image"
|
||||
assert result[1]["source"]["media_type"] == "image/jpeg"
|
||||
assert result[1]["source"]["data"] == "/9j/4AAQ"
|
||||
|
||||
+24
-39
@@ -1,6 +1,6 @@
|
||||
"""Tests for the atomic workstream resumption flow.
|
||||
|
||||
Covers CreateWorkstreamMessage resume_session field, SessionResumedEvent,
|
||||
Covers CreateWorkstreamMessage resume_ws field, WorkstreamResumedEvent,
|
||||
WorkstreamCreatedEvent resumed fields, and server endpoint handling.
|
||||
"""
|
||||
|
||||
@@ -10,8 +10,8 @@ import json
|
||||
|
||||
from turnstone.mq.protocol import (
|
||||
CreateWorkstreamMessage,
|
||||
SessionResumedEvent,
|
||||
WorkstreamCreatedEvent,
|
||||
WorkstreamResumedEvent,
|
||||
)
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
@@ -20,93 +20,78 @@ from turnstone.mq.protocol import (
|
||||
|
||||
|
||||
class TestCreateWorkstreamMessageResumeField:
|
||||
def test_resume_session_defaults_empty(self) -> None:
|
||||
def test_resume_ws_defaults_empty(self) -> None:
|
||||
msg = CreateWorkstreamMessage(name="test")
|
||||
assert msg.resume_session == ""
|
||||
assert msg.resume_ws == ""
|
||||
|
||||
def test_resume_session_set(self) -> None:
|
||||
msg = CreateWorkstreamMessage(name="test", resume_session="sess-abc")
|
||||
assert msg.resume_session == "sess-abc"
|
||||
def test_resume_ws_set(self) -> None:
|
||||
msg = CreateWorkstreamMessage(name="test", resume_ws="ws-abc")
|
||||
assert msg.resume_ws == "ws-abc"
|
||||
|
||||
def test_resume_session_serializes(self) -> None:
|
||||
msg = CreateWorkstreamMessage(resume_session="sess-xyz")
|
||||
def test_resume_ws_serializes(self) -> None:
|
||||
msg = CreateWorkstreamMessage(resume_ws="ws-xyz")
|
||||
data = json.loads(msg.to_json())
|
||||
assert data["resume_session"] == "sess-xyz"
|
||||
assert data["resume_ws"] == "ws-xyz"
|
||||
|
||||
def test_resume_session_deserializes(self) -> None:
|
||||
msg = CreateWorkstreamMessage(resume_session="sess-123")
|
||||
def test_resume_ws_deserializes(self) -> None:
|
||||
msg = CreateWorkstreamMessage(resume_ws="ws-123")
|
||||
raw = msg.to_json()
|
||||
from turnstone.mq.protocol import InboundMessage
|
||||
|
||||
restored = InboundMessage.from_json(raw)
|
||||
assert getattr(restored, "resume_session", "") == "sess-123"
|
||||
assert getattr(restored, "resume_ws", "") == "ws-123"
|
||||
|
||||
|
||||
class TestWorkstreamCreatedEventResumeFields:
|
||||
def test_default_not_resumed(self) -> None:
|
||||
event = WorkstreamCreatedEvent(ws_id="ws-1", name="test")
|
||||
assert event.resumed is False
|
||||
assert event.session_id == ""
|
||||
assert event.message_count == 0
|
||||
|
||||
def test_resumed_fields(self) -> None:
|
||||
event = WorkstreamCreatedEvent(
|
||||
ws_id="ws-1", name="test", resumed=True, session_id="s-1", message_count=42
|
||||
)
|
||||
event = WorkstreamCreatedEvent(ws_id="ws-1", name="test", resumed=True, message_count=42)
|
||||
assert event.resumed is True
|
||||
assert event.session_id == "s-1"
|
||||
assert event.message_count == 42
|
||||
|
||||
def test_serializes_resumed_fields(self) -> None:
|
||||
event = WorkstreamCreatedEvent(
|
||||
ws_id="ws-1", resumed=True, session_id="s-1", message_count=10
|
||||
)
|
||||
event = WorkstreamCreatedEvent(ws_id="ws-1", resumed=True, message_count=10)
|
||||
data = json.loads(event.to_json())
|
||||
assert data["resumed"] is True
|
||||
assert data["session_id"] == "s-1"
|
||||
assert data["message_count"] == 10
|
||||
|
||||
def test_deserializes_resumed_fields(self) -> None:
|
||||
event = WorkstreamCreatedEvent(
|
||||
ws_id="ws-1", resumed=True, session_id="s-1", message_count=5
|
||||
)
|
||||
event = WorkstreamCreatedEvent(ws_id="ws-1", resumed=True, message_count=5)
|
||||
from turnstone.mq.protocol import OutboundEvent
|
||||
|
||||
restored = OutboundEvent.from_json(event.to_json())
|
||||
assert isinstance(restored, WorkstreamCreatedEvent)
|
||||
assert restored.resumed is True
|
||||
assert restored.session_id == "s-1"
|
||||
assert restored.message_count == 5
|
||||
|
||||
|
||||
class TestSessionResumedEvent:
|
||||
class TestWorkstreamResumedEvent:
|
||||
def test_defaults(self) -> None:
|
||||
event = SessionResumedEvent(ws_id="ws-1")
|
||||
assert event.type == "session_resumed"
|
||||
assert event.session_id == ""
|
||||
event = WorkstreamResumedEvent(ws_id="ws-1")
|
||||
assert event.type == "ws_resumed"
|
||||
assert event.message_count == 0
|
||||
assert event.name == ""
|
||||
|
||||
def test_with_values(self) -> None:
|
||||
event = SessionResumedEvent(
|
||||
ws_id="ws-1", session_id="s-abc", message_count=25, name="My Chat"
|
||||
)
|
||||
assert event.session_id == "s-abc"
|
||||
event = WorkstreamResumedEvent(ws_id="ws-1", message_count=25, name="My Chat")
|
||||
assert event.message_count == 25
|
||||
assert event.name == "My Chat"
|
||||
|
||||
def test_round_trip(self) -> None:
|
||||
event = SessionResumedEvent(ws_id="ws-1", session_id="s-abc", message_count=10, name="Chat")
|
||||
event = WorkstreamResumedEvent(ws_id="ws-1", message_count=10, name="Chat")
|
||||
from turnstone.mq.protocol import OutboundEvent
|
||||
|
||||
restored = OutboundEvent.from_json(event.to_json())
|
||||
assert isinstance(restored, SessionResumedEvent)
|
||||
assert restored.session_id == "s-abc"
|
||||
assert isinstance(restored, WorkstreamResumedEvent)
|
||||
assert restored.message_count == 10
|
||||
assert restored.name == "Chat"
|
||||
|
||||
def test_registered_in_outbound_registry(self) -> None:
|
||||
from turnstone.mq.protocol import _OUTBOUND_REGISTRY
|
||||
|
||||
assert "session_resumed" in _OUTBOUND_REGISTRY
|
||||
assert _OUTBOUND_REGISTRY["session_resumed"] is SessionResumedEvent
|
||||
assert "ws_resumed" in _OUTBOUND_REGISTRY
|
||||
assert _OUTBOUND_REGISTRY["ws_resumed"] is WorkstreamResumedEvent
|
||||
|
||||
@@ -2,11 +2,19 @@
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
import pytest
|
||||
from starlette.applications import Starlette
|
||||
from starlette.middleware import Middleware
|
||||
from starlette.middleware.base import BaseHTTPMiddleware
|
||||
from starlette.routing import Mount, Route
|
||||
from starlette.testclient import TestClient
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from starlette.requests import Request
|
||||
from starlette.responses import Response
|
||||
|
||||
from turnstone.console.server import (
|
||||
admin_create_schedule,
|
||||
admin_delete_schedule,
|
||||
@@ -15,9 +23,21 @@ from turnstone.console.server import (
|
||||
admin_list_schedules,
|
||||
admin_update_schedule,
|
||||
)
|
||||
from turnstone.core.auth import AuthResult
|
||||
from turnstone.core.storage._sqlite import SQLiteBackend
|
||||
|
||||
|
||||
class _InjectAuthMiddleware(BaseHTTPMiddleware):
|
||||
async def dispatch(self, request: Request, call_next: Any) -> Response:
|
||||
request.state.auth_result = AuthResult(
|
||||
user_id="test-admin",
|
||||
scopes=frozenset({"approve"}),
|
||||
token_source="config",
|
||||
permissions=frozenset({"admin.schedules"}),
|
||||
)
|
||||
return await call_next(request)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def storage(tmp_path):
|
||||
"""Fresh SQLite backend for each test."""
|
||||
@@ -52,6 +72,7 @@ def client(storage):
|
||||
],
|
||||
),
|
||||
],
|
||||
middleware=[Middleware(_InjectAuthMiddleware)],
|
||||
)
|
||||
app.state.auth_storage = storage
|
||||
return TestClient(app)
|
||||
|
||||
@@ -148,19 +148,19 @@ async def test_command():
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Sessions
|
||||
# History
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@pytest.mark.anyio
|
||||
async def test_list_sessions():
|
||||
async def test_list_saved_workstreams():
|
||||
transport = _mock_transport(
|
||||
{
|
||||
"GET /v1/api/sessions": _json_response(
|
||||
"GET /v1/api/workstreams/saved": _json_response(
|
||||
{
|
||||
"sessions": [
|
||||
"workstreams": [
|
||||
{
|
||||
"session_id": "s1",
|
||||
"ws_id": "s1",
|
||||
"title": "test",
|
||||
"created": "2024-01-01",
|
||||
"updated": "2024-01-02",
|
||||
@@ -173,8 +173,8 @@ async def test_list_sessions():
|
||||
)
|
||||
async with httpx.AsyncClient(transport=transport, base_url="http://test") as hc:
|
||||
client = AsyncTurnstoneServer(httpx_client=hc)
|
||||
resp = await client.list_sessions()
|
||||
assert len(resp.sessions) == 1
|
||||
resp = await client.list_saved_workstreams()
|
||||
assert len(resp.workstreams) == 1
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
@@ -609,7 +609,7 @@ class TestServerHealthMetrics:
|
||||
mock_ui._ws_context_ratio = 0.0
|
||||
|
||||
mock_session = MagicMock()
|
||||
mock_session.session_id = "test-session-id"
|
||||
mock_session.ws_id = "test-session-id"
|
||||
|
||||
mock_ws = MagicMock()
|
||||
mock_ws.id = "test-ws"
|
||||
@@ -785,7 +785,7 @@ class TestServerRateLimiting:
|
||||
mock_ui._ws_context_ratio = 0.0
|
||||
|
||||
mock_session = MagicMock()
|
||||
mock_session.session_id = "test-session-id"
|
||||
mock_session.ws_id = "test-session-id"
|
||||
|
||||
mock_ws = MagicMock()
|
||||
mock_ws.id = "test-ws"
|
||||
|
||||
+161
-9
@@ -1,9 +1,10 @@
|
||||
"""Tests for turnstone.core.session — ChatSession construction."""
|
||||
|
||||
import base64
|
||||
import json
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
from turnstone.core.session import ChatSession
|
||||
from turnstone.core.session import _IMAGE_EXTENSIONS, _IMAGE_SIZE_CAP, ChatSession
|
||||
|
||||
|
||||
class NullUI:
|
||||
@@ -161,12 +162,12 @@ class TestPlanExec:
|
||||
|
||||
return call_id, content, captured.get("messages", [])
|
||||
|
||||
def test_plan_file_uses_session_id(self, tmp_db, tmp_path, monkeypatch):
|
||||
"""Plan file is named .plan-<session_id>.md, not .plan.md."""
|
||||
def test_plan_file_uses_ws_id(self, tmp_db, tmp_path, monkeypatch):
|
||||
"""Plan file is named .plan-<ws_id>.md, not .plan.md."""
|
||||
monkeypatch.chdir(tmp_path)
|
||||
session = _make_session()
|
||||
self._run_plan(session, "add feature")
|
||||
expected = tmp_path / f".plan-{session._session_id}.md"
|
||||
expected = tmp_path / f".plan-{session._ws_id}.md"
|
||||
assert expected.exists(), f"Expected {expected} to be created"
|
||||
assert not (tmp_path / ".plan.md").exists()
|
||||
|
||||
@@ -176,7 +177,7 @@ class TestPlanExec:
|
||||
session = _make_session()
|
||||
plan_content = "## Goal\n\nAdd a new endpoint."
|
||||
self._run_plan(session, "add endpoint", agent_return=plan_content)
|
||||
plan_file = tmp_path / f".plan-{session._session_id}.md"
|
||||
plan_file = tmp_path / f".plan-{session._ws_id}.md"
|
||||
assert plan_file.read_text() == plan_content
|
||||
|
||||
def test_two_sessions_produce_different_files(self, tmp_db, tmp_path, monkeypatch):
|
||||
@@ -184,7 +185,7 @@ class TestPlanExec:
|
||||
monkeypatch.chdir(tmp_path)
|
||||
s1 = _make_session()
|
||||
s2 = _make_session()
|
||||
assert s1._session_id != s2._session_id
|
||||
assert s1._ws_id != s2._ws_id
|
||||
self._run_plan(s1, "feature A")
|
||||
self._run_plan(s2, "feature B")
|
||||
files = list(tmp_path.glob(".plan-*.md"))
|
||||
@@ -202,8 +203,8 @@ class TestPlanExec:
|
||||
"id": tc_id,
|
||||
"type": "function",
|
||||
"function": {
|
||||
"name": "plan",
|
||||
"arguments": json.dumps({"prompt": prior_prompt}),
|
||||
"name": "create_plan",
|
||||
"arguments": json.dumps({"goal": prior_prompt}),
|
||||
},
|
||||
}
|
||||
],
|
||||
@@ -238,7 +239,7 @@ class TestPlanExec:
|
||||
m for m in messages if m["role"] == "assistant" and m.get("tool_calls")
|
||||
]
|
||||
assert len(assistant_with_tc) == 1
|
||||
assert assistant_with_tc[0]["tool_calls"][0]["function"]["name"] == "plan"
|
||||
assert assistant_with_tc[0]["tool_calls"][0]["function"]["name"] == "create_plan"
|
||||
|
||||
# The real tool result is forwarded with its original content
|
||||
tool_msgs = [m for m in messages if m["role"] == "tool"]
|
||||
@@ -265,3 +266,154 @@ class TestPlanExec:
|
||||
call_id, content, _ = self._run_plan(session, "do stuff", agent_return=agent_output)
|
||||
assert call_id == "test-call-1"
|
||||
assert content == agent_output
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Vision / image support
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestImageExtensions:
|
||||
"""Test _IMAGE_EXTENSIONS constant and detection logic."""
|
||||
|
||||
def test_common_image_extensions(self):
|
||||
for ext in (".png", ".jpg", ".jpeg", ".gif", ".webp", ".bmp", ".tiff", ".tif", ".ico"):
|
||||
assert ext in _IMAGE_EXTENSIONS, f"{ext} should be in _IMAGE_EXTENSIONS"
|
||||
|
||||
def test_svg_excluded(self):
|
||||
assert ".svg" not in _IMAGE_EXTENSIONS
|
||||
|
||||
def test_text_extensions_excluded(self):
|
||||
for ext in (".py", ".txt", ".json", ".md", ".rs", ".go"):
|
||||
assert ext not in _IMAGE_EXTENSIONS
|
||||
|
||||
|
||||
class TestExecReadImage:
|
||||
"""Test _exec_read_image method."""
|
||||
|
||||
def _make_png(self, path: str, size: int = 100) -> None:
|
||||
"""Write a minimal valid-ish PNG header to a file."""
|
||||
# 8-byte PNG signature + enough bytes to reach target size
|
||||
header = b"\x89PNG\r\n\x1a\n"
|
||||
with open(path, "wb") as f:
|
||||
f.write(header + b"\x00" * max(0, size - len(header)))
|
||||
|
||||
def test_image_returns_content_parts(self, tmp_db, tmp_path):
|
||||
"""read_file on a PNG with vision support returns content parts."""
|
||||
img = tmp_path / "test.png"
|
||||
self._make_png(str(img))
|
||||
|
||||
session = _make_session()
|
||||
# Mock provider to report vision support
|
||||
mock_caps = MagicMock()
|
||||
mock_caps.supports_vision = True
|
||||
session._provider.get_capabilities = MagicMock(return_value=mock_caps)
|
||||
|
||||
item = {"call_id": "c1", "path": str(img), "offset": None, "limit": None}
|
||||
call_id, output = session._exec_read_file(item)
|
||||
|
||||
assert call_id == "c1"
|
||||
assert isinstance(output, list)
|
||||
assert len(output) == 2
|
||||
assert output[0]["type"] == "text"
|
||||
assert "test.png" in output[0]["text"]
|
||||
assert output[1]["type"] == "image_url"
|
||||
url = output[1]["image_url"]["url"]
|
||||
assert url.startswith("data:image/png;base64,")
|
||||
# Verify base64 round-trip
|
||||
b64part = url.split(",", 1)[1]
|
||||
decoded = base64.b64decode(b64part)
|
||||
assert decoded == img.read_bytes()
|
||||
|
||||
def test_no_vision_returns_text(self, tmp_db, tmp_path):
|
||||
"""read_file on image with non-vision model returns text description."""
|
||||
img = tmp_path / "photo.jpg"
|
||||
self._make_png(str(img), size=2048)
|
||||
|
||||
session = _make_session()
|
||||
mock_caps = MagicMock()
|
||||
mock_caps.supports_vision = False
|
||||
session._provider.get_capabilities = MagicMock(return_value=mock_caps)
|
||||
|
||||
item = {"call_id": "c2", "path": str(img), "offset": None, "limit": None}
|
||||
call_id, output = session._exec_read_file(item)
|
||||
|
||||
assert call_id == "c2"
|
||||
assert isinstance(output, str)
|
||||
assert "does not support vision" in output
|
||||
assert "photo.jpg" in output
|
||||
|
||||
def test_oversized_image_returns_error(self, tmp_db, tmp_path):
|
||||
"""Images exceeding _IMAGE_SIZE_CAP return an error string."""
|
||||
img = tmp_path / "huge.png"
|
||||
# Write slightly over the cap
|
||||
with open(img, "wb") as f:
|
||||
f.write(b"\x89PNG\r\n\x1a\n" + b"\x00" * _IMAGE_SIZE_CAP)
|
||||
|
||||
session = _make_session()
|
||||
mock_caps = MagicMock()
|
||||
mock_caps.supports_vision = True
|
||||
session._provider.get_capabilities = MagicMock(return_value=mock_caps)
|
||||
|
||||
item = {"call_id": "c3", "path": str(img), "offset": None, "limit": None}
|
||||
call_id, output = session._exec_read_file(item)
|
||||
|
||||
assert call_id == "c3"
|
||||
assert isinstance(output, str)
|
||||
assert "exceeds" in output
|
||||
|
||||
def test_missing_image_returns_error(self, tmp_db, tmp_path):
|
||||
"""read_file on non-existent image returns error."""
|
||||
session = _make_session()
|
||||
mock_caps = MagicMock()
|
||||
mock_caps.supports_vision = True
|
||||
session._provider.get_capabilities = MagicMock(return_value=mock_caps)
|
||||
|
||||
item = {"call_id": "c4", "path": str(tmp_path / "nope.png"), "offset": None, "limit": None}
|
||||
call_id, output = session._exec_read_file(item)
|
||||
assert isinstance(output, str)
|
||||
assert "not found" in output
|
||||
|
||||
def test_svg_read_as_text(self, tmp_db, tmp_path):
|
||||
"""SVG files are read as text, not as images."""
|
||||
svg = tmp_path / "icon.svg"
|
||||
svg.write_text('<svg xmlns="http://www.w3.org/2000/svg"><circle r="10"/></svg>')
|
||||
|
||||
session = _make_session()
|
||||
item = {"call_id": "c5", "path": str(svg), "offset": None, "limit": None}
|
||||
call_id, output = session._exec_read_file(item)
|
||||
assert isinstance(output, str)
|
||||
assert "<svg" in output # Read as text
|
||||
|
||||
|
||||
class TestGetCapabilitiesOverride:
|
||||
"""Test _get_capabilities with config.toml overrides."""
|
||||
|
||||
def test_config_override_applies(self, tmp_db):
|
||||
"""capabilities dict from ModelConfig is merged onto provider caps."""
|
||||
from turnstone.core.model_registry import ModelConfig, ModelRegistry
|
||||
from turnstone.core.providers._protocol import ModelCapabilities
|
||||
|
||||
cfg = ModelConfig(
|
||||
alias="qwen-vl",
|
||||
base_url="http://localhost:8000/v1",
|
||||
api_key="dummy",
|
||||
model="qwen-3.5-vl",
|
||||
capabilities={"supports_vision": True},
|
||||
)
|
||||
registry = ModelRegistry(
|
||||
models={"qwen-vl": cfg},
|
||||
default="qwen-vl",
|
||||
)
|
||||
session = _make_session(registry=registry, model_alias="qwen-vl")
|
||||
# Ensure provider returns a real ModelCapabilities (not MagicMock)
|
||||
session._provider.get_capabilities = MagicMock(return_value=ModelCapabilities())
|
||||
caps = session._get_capabilities()
|
||||
assert caps.supports_vision is True
|
||||
|
||||
def test_no_override_uses_provider_default(self, tmp_db):
|
||||
"""Without config override, provider defaults are used."""
|
||||
session = _make_session()
|
||||
caps = session._get_capabilities()
|
||||
# Default OpenAI provider for unknown model → no vision
|
||||
assert caps.supports_vision is False
|
||||
|
||||
+178
-179
@@ -1,155 +1,155 @@
|
||||
"""Tests for session persistence and resume functionality."""
|
||||
"""Tests for workstream persistence and resume functionality."""
|
||||
|
||||
from unittest.mock import MagicMock
|
||||
|
||||
import sqlalchemy as sa
|
||||
|
||||
from turnstone.core.memory import (
|
||||
delete_session,
|
||||
list_sessions,
|
||||
load_session_config,
|
||||
load_session_messages,
|
||||
prune_sessions,
|
||||
register_session,
|
||||
resolve_session,
|
||||
delete_workstream,
|
||||
list_workstreams_with_history,
|
||||
load_messages,
|
||||
load_workstream_config,
|
||||
prune_workstreams,
|
||||
register_workstream,
|
||||
resolve_workstream,
|
||||
save_message,
|
||||
save_session_config,
|
||||
set_session_alias,
|
||||
update_session_title,
|
||||
save_workstream_config,
|
||||
set_workstream_alias,
|
||||
update_workstream_title,
|
||||
)
|
||||
from turnstone.core.session import ChatSession
|
||||
from turnstone.core.storage import get_storage
|
||||
|
||||
# ── Session registration ──────────────────────────────────────────────
|
||||
# ── Workstream registration ───────────────────────────────────────────
|
||||
|
||||
|
||||
class TestRegisterSession:
|
||||
class TestRegisterWorkstream:
|
||||
def test_register_creates_row(self, tmp_db):
|
||||
register_session("abc123")
|
||||
# Session exists in DB (resolve works) even without messages
|
||||
assert resolve_session("abc123") == "abc123"
|
||||
register_workstream("abc123")
|
||||
# Workstream exists in DB (resolve works) even without messages
|
||||
assert resolve_workstream("abc123") == "abc123"
|
||||
|
||||
def test_register_with_title(self, tmp_db):
|
||||
register_session("abc123", title="My Session")
|
||||
register_workstream("abc123", name="My Workstream")
|
||||
save_message("abc123", "user", "hello")
|
||||
rows = list_sessions()
|
||||
assert rows[0][2] == "My Session" # title
|
||||
rows = list_workstreams_with_history()
|
||||
assert rows[0][2] is None # title column (name is separate)
|
||||
|
||||
def test_register_idempotent(self, tmp_db):
|
||||
register_session("abc123", title="First")
|
||||
register_session("abc123", title="Second") # should be ignored
|
||||
register_workstream("abc123")
|
||||
update_workstream_title("abc123", "First")
|
||||
register_workstream("abc123") # should be ignored
|
||||
update_workstream_title("abc123", "First") # title is set via update
|
||||
save_message("abc123", "user", "hello")
|
||||
rows = list_sessions()
|
||||
rows = list_workstreams_with_history()
|
||||
assert len(rows) == 1
|
||||
assert rows[0][2] == "First" # original title preserved
|
||||
assert rows[0][2] == "First" # title preserved
|
||||
|
||||
def test_update_title(self, tmp_db):
|
||||
register_session("abc123")
|
||||
update_session_title("abc123", "New Title")
|
||||
register_workstream("abc123")
|
||||
update_workstream_title("abc123", "New Title")
|
||||
save_message("abc123", "user", "hello")
|
||||
rows = list_sessions()
|
||||
rows = list_workstreams_with_history()
|
||||
assert rows[0][2] == "New Title"
|
||||
|
||||
|
||||
# ── Session alias ─────────────────────────────────────────────────────
|
||||
# ── Workstream alias ──────────────────────────────────────────────────
|
||||
|
||||
|
||||
class TestSessionAlias:
|
||||
class TestWorkstreamAlias:
|
||||
def test_set_alias(self, tmp_db):
|
||||
register_session("abc123")
|
||||
assert set_session_alias("abc123", "my-session") is True
|
||||
register_workstream("abc123")
|
||||
assert set_workstream_alias("abc123", "my-session") is True
|
||||
save_message("abc123", "user", "hello")
|
||||
rows = list_sessions()
|
||||
rows = list_workstreams_with_history()
|
||||
assert rows[0][1] == "my-session" # alias
|
||||
|
||||
def test_alias_conflict(self, tmp_db):
|
||||
register_session("abc123")
|
||||
register_session("def456")
|
||||
set_session_alias("abc123", "taken")
|
||||
assert set_session_alias("def456", "taken") is False
|
||||
register_workstream("abc123")
|
||||
register_workstream("def456")
|
||||
set_workstream_alias("abc123", "taken")
|
||||
assert set_workstream_alias("def456", "taken") is False
|
||||
|
||||
def test_alias_same_session_ok(self, tmp_db):
|
||||
register_session("abc123")
|
||||
set_session_alias("abc123", "mine")
|
||||
assert set_session_alias("abc123", "mine") is True # no-op, same session
|
||||
def test_alias_same_workstream_ok(self, tmp_db):
|
||||
register_workstream("abc123")
|
||||
set_workstream_alias("abc123", "mine")
|
||||
assert set_workstream_alias("abc123", "mine") is True # no-op, same workstream
|
||||
|
||||
|
||||
# ── Session resolution ────────────────────────────────────────────────
|
||||
# ── Workstream resolution ─────────────────────────────────────────────
|
||||
|
||||
|
||||
class TestResolveSession:
|
||||
class TestResolveWorkstream:
|
||||
def test_resolve_by_alias(self, tmp_db):
|
||||
register_session("abc123")
|
||||
set_session_alias("abc123", "my-alias")
|
||||
assert resolve_session("my-alias") == "abc123"
|
||||
register_workstream("abc123")
|
||||
set_workstream_alias("abc123", "my-alias")
|
||||
assert resolve_workstream("my-alias") == "abc123"
|
||||
|
||||
def test_resolve_by_exact_id(self, tmp_db):
|
||||
register_session("abc123def456")
|
||||
assert resolve_session("abc123def456") == "abc123def456"
|
||||
register_workstream("abc123def456")
|
||||
assert resolve_workstream("abc123def456") == "abc123def456"
|
||||
|
||||
def test_resolve_by_prefix(self, tmp_db):
|
||||
register_session("abc123def456")
|
||||
assert resolve_session("abc123") == "abc123def456"
|
||||
register_workstream("abc123def456")
|
||||
assert resolve_workstream("abc123") == "abc123def456"
|
||||
|
||||
def test_resolve_prefix_ambiguous(self, tmp_db):
|
||||
register_session("abc123aaaaaa")
|
||||
register_session("abc123bbbbbb")
|
||||
register_workstream("abc123aaaaaa")
|
||||
register_workstream("abc123bbbbbb")
|
||||
# Ambiguous prefix should return None
|
||||
assert resolve_session("abc123") is None
|
||||
assert resolve_workstream("abc123") is None
|
||||
|
||||
def test_resolve_not_found(self, tmp_db):
|
||||
assert resolve_session("nonexistent") is None
|
||||
|
||||
def test_resolve_legacy_session(self, tmp_db):
|
||||
"""Sessions that exist only in conversations (pre-migration) should auto-register."""
|
||||
save_message("legacy123456", "user", "old message")
|
||||
result = resolve_session("legacy123456")
|
||||
assert result == "legacy123456"
|
||||
# Should now appear in sessions list
|
||||
rows = list_sessions()
|
||||
assert any(r[0] == "legacy123456" for r in rows)
|
||||
assert resolve_workstream("nonexistent") is None
|
||||
|
||||
|
||||
# ── List sessions ─────────────────────────────────────────────────────
|
||||
# ── List workstreams with history ──────────────────────────────────────
|
||||
|
||||
|
||||
class TestListSessions:
|
||||
class TestListWorkstreamsWithHistory:
|
||||
def test_empty(self, tmp_db):
|
||||
assert list_sessions() == []
|
||||
assert list_workstreams_with_history() == []
|
||||
|
||||
def test_ordered_by_updated(self, tmp_db):
|
||||
register_session("first")
|
||||
register_workstream("first")
|
||||
save_message("first", "user", "hello")
|
||||
register_session("second")
|
||||
# Force an older timestamp so ordering is deterministic
|
||||
engine = get_storage()._engine # noqa: SLF001
|
||||
with engine.connect() as conn:
|
||||
conn.execute(
|
||||
sa.text("UPDATE workstreams SET updated = '2020-01-01' WHERE ws_id = 'first'")
|
||||
)
|
||||
conn.commit()
|
||||
register_workstream("second")
|
||||
save_message("second", "user", "hello")
|
||||
# second is more recent
|
||||
rows = list_sessions()
|
||||
rows = list_workstreams_with_history()
|
||||
assert rows[0][0] == "second"
|
||||
assert rows[1][0] == "first"
|
||||
|
||||
def test_includes_message_count(self, tmp_db):
|
||||
register_session("sess1")
|
||||
register_workstream("sess1")
|
||||
save_message("sess1", "user", "hello")
|
||||
save_message("sess1", "assistant", "hi")
|
||||
rows = list_sessions()
|
||||
rows = list_workstreams_with_history()
|
||||
assert rows[0][5] == 2 # msg_count
|
||||
|
||||
def test_respects_limit(self, tmp_db):
|
||||
for i in range(5):
|
||||
register_session(f"sess{i}")
|
||||
register_workstream(f"sess{i}")
|
||||
save_message(f"sess{i}", "user", "hello")
|
||||
rows = list_sessions(limit=3)
|
||||
rows = list_workstreams_with_history(limit=3)
|
||||
assert len(rows) == 3
|
||||
|
||||
|
||||
# ── Load session messages ─────────────────────────────────────────────
|
||||
# ── Load messages ─────────────────────────────────────────────────────
|
||||
|
||||
|
||||
class TestLoadSessionMessages:
|
||||
class TestLoadMessages:
|
||||
def test_simple_user_assistant(self, tmp_db):
|
||||
save_message("s1", "user", "hello")
|
||||
save_message("s1", "assistant", "hi there")
|
||||
msgs = load_session_messages("s1")
|
||||
msgs = load_messages("s1")
|
||||
assert len(msgs) == 2
|
||||
assert msgs[0] == {"role": "user", "content": "hello"}
|
||||
assert msgs[1] == {"role": "assistant", "content": "hi there"}
|
||||
@@ -159,7 +159,7 @@ class TestLoadSessionMessages:
|
||||
save_message("s1", "assistant", "Let me check.")
|
||||
save_message("s1", "tool_call", None, "bash", '{"command":"ls"}', tool_call_id="call_abc")
|
||||
save_message("s1", "tool_result", "file1.txt\nfile2.txt", "bash", tool_call_id="call_abc")
|
||||
msgs = load_session_messages("s1")
|
||||
msgs = load_messages("s1")
|
||||
assert len(msgs) == 3 # user, assistant+tool_calls, tool
|
||||
# Assistant should have content merged with tool_calls
|
||||
assert msgs[1]["role"] == "assistant"
|
||||
@@ -177,7 +177,7 @@ class TestLoadSessionMessages:
|
||||
save_message("s1", "user", "do stuff")
|
||||
save_message("s1", "tool_call", None, "bash", '{"command":"ls"}')
|
||||
save_message("s1", "tool_result", "output", "bash")
|
||||
msgs = load_session_messages("s1")
|
||||
msgs = load_messages("s1")
|
||||
assert len(msgs) == 3
|
||||
# Synthetic IDs should match
|
||||
tc_id = msgs[1]["tool_calls"][0]["id"]
|
||||
@@ -189,36 +189,36 @@ class TestLoadSessionMessages:
|
||||
save_message("s1", "tool_call", None, "search", '{"query":"b"}', tool_call_id="call_2")
|
||||
save_message("s1", "tool_result", "result a", "search", tool_call_id="call_1")
|
||||
save_message("s1", "tool_result", "result b", "search", tool_call_id="call_2")
|
||||
msgs = load_session_messages("s1")
|
||||
msgs = load_messages("s1")
|
||||
assert len(msgs) == 4 # user, assistant+2 tool_calls, 2 tool results
|
||||
assert len(msgs[1]["tool_calls"]) == 2
|
||||
assert msgs[2]["tool_call_id"] == "call_1"
|
||||
assert msgs[3]["tool_call_id"] == "call_2"
|
||||
|
||||
def test_empty_session(self, tmp_db):
|
||||
assert load_session_messages("nonexistent") == []
|
||||
def test_empty_workstream(self, tmp_db):
|
||||
assert load_messages("nonexistent") == []
|
||||
|
||||
def test_orphaned_tool_result_skipped(self, tmp_db):
|
||||
save_message("s1", "user", "hello")
|
||||
save_message("s1", "tool_result", "orphan", "bash")
|
||||
msgs = load_session_messages("s1")
|
||||
msgs = load_messages("s1")
|
||||
assert len(msgs) == 1 # only the user message
|
||||
|
||||
|
||||
# ── Delete session ────────────────────────────────────────────────────
|
||||
# ── Delete workstream ─────────────────────────────────────────────────
|
||||
|
||||
|
||||
class TestDeleteSession:
|
||||
def test_delete_removes_session_and_messages(self, tmp_db):
|
||||
register_session("abc123")
|
||||
class TestDeleteWorkstream:
|
||||
def test_delete_removes_workstream_and_messages(self, tmp_db):
|
||||
register_workstream("abc123")
|
||||
save_message("abc123", "user", "hello")
|
||||
save_message("abc123", "assistant", "hi")
|
||||
assert delete_session("abc123") is True
|
||||
assert list_sessions() == []
|
||||
assert load_session_messages("abc123") == []
|
||||
assert delete_workstream("abc123") is True
|
||||
assert list_workstreams_with_history() == []
|
||||
assert load_messages("abc123") == []
|
||||
|
||||
def test_delete_nonexistent(self, tmp_db):
|
||||
assert delete_session("nonexistent") is True # no-op, still returns True
|
||||
assert delete_workstream("nonexistent") is False
|
||||
|
||||
|
||||
# ── save_message with tool_call_id ────────────────────────────────────
|
||||
@@ -230,7 +230,7 @@ class TestSaveMessageToolCallId:
|
||||
engine = get_storage()._engine # noqa: SLF001
|
||||
with engine.connect() as conn:
|
||||
row = conn.execute(
|
||||
sa.text("SELECT tool_call_id FROM conversations WHERE session_id = 's1'")
|
||||
sa.text("SELECT tool_call_id FROM conversations WHERE ws_id = 's1'")
|
||||
).fetchone()
|
||||
assert row[0] == "call_xyz"
|
||||
|
||||
@@ -239,20 +239,20 @@ class TestSaveMessageToolCallId:
|
||||
engine = get_storage()._engine # noqa: SLF001
|
||||
with engine.connect() as conn:
|
||||
row = conn.execute(
|
||||
sa.text("SELECT tool_call_id FROM conversations WHERE session_id = 's1'")
|
||||
sa.text("SELECT tool_call_id FROM conversations WHERE ws_id = 's1'")
|
||||
).fetchone()
|
||||
assert row[0] is None
|
||||
|
||||
|
||||
# ── Sessions table creation ───────────────────────────────────────────
|
||||
# ── Workstreams table creation ────────────────────────────────────────
|
||||
|
||||
|
||||
class TestSessionsTable:
|
||||
def test_sessions_table_exists(self, tmp_db):
|
||||
class TestWorkstreamsTable:
|
||||
def test_workstreams_table_exists(self, tmp_db):
|
||||
engine = get_storage()._engine # noqa: SLF001
|
||||
with engine.connect() as conn:
|
||||
rows = conn.execute(
|
||||
sa.text("SELECT name FROM sqlite_master WHERE type='table' AND name='sessions'")
|
||||
sa.text("SELECT name FROM sqlite_master WHERE type='table' AND name='workstreams'")
|
||||
).fetchall()
|
||||
assert len(rows) == 1
|
||||
|
||||
@@ -263,15 +263,15 @@ class TestSessionsTable:
|
||||
conn.execute(sa.text("SELECT tool_call_id FROM conversations LIMIT 0"))
|
||||
|
||||
|
||||
# ── ChatSession.resume_session ────────────────────────────────────────
|
||||
# ── ChatSession.resume ────────────────────────────────────────────────
|
||||
|
||||
|
||||
class TestResumeSession:
|
||||
class TestResumeWorkstream:
|
||||
def test_resume_loads_messages(self, tmp_db, mock_openai_client):
|
||||
# Set up a session with messages in DB
|
||||
register_session("old_sess_123")
|
||||
save_message("old_sess_123", "user", "hello world")
|
||||
save_message("old_sess_123", "assistant", "hi there")
|
||||
# Set up a workstream with messages in DB
|
||||
register_workstream("old_ws_123")
|
||||
save_message("old_ws_123", "user", "hello world")
|
||||
save_message("old_ws_123", "assistant", "hi there")
|
||||
|
||||
# Create a new session and resume
|
||||
session = ChatSession(
|
||||
@@ -283,12 +283,12 @@ class TestResumeSession:
|
||||
max_tokens=1000,
|
||||
tool_timeout=10,
|
||||
)
|
||||
original_id = session._session_id
|
||||
assert original_id != "old_sess_123"
|
||||
original_id = session._ws_id
|
||||
assert original_id != "old_ws_123"
|
||||
|
||||
result = session.resume_session("old_sess_123")
|
||||
result = session.resume("old_ws_123")
|
||||
assert result is True
|
||||
assert session._session_id == "old_sess_123"
|
||||
assert session._ws_id == "old_ws_123"
|
||||
assert len(session.messages) == 2
|
||||
assert session.messages[0]["content"] == "hello world"
|
||||
assert session._title_generated is True
|
||||
@@ -303,9 +303,9 @@ class TestResumeSession:
|
||||
max_tokens=1000,
|
||||
tool_timeout=10,
|
||||
)
|
||||
assert session.resume_session("nonexistent") is False
|
||||
assert session.resume("nonexistent") is False
|
||||
|
||||
def test_session_registered_on_init(self, tmp_db, mock_openai_client):
|
||||
def test_workstream_not_registered_until_message(self, tmp_db, mock_openai_client):
|
||||
session = ChatSession(
|
||||
client=mock_openai_client,
|
||||
model="test-model",
|
||||
@@ -315,20 +315,19 @@ class TestResumeSession:
|
||||
max_tokens=1000,
|
||||
tool_timeout=10,
|
||||
)
|
||||
# Session is registered in DB (resolvable) even before any messages
|
||||
assert resolve_session(session._session_id) == session._session_id
|
||||
# But does not appear in list_sessions until a message is saved
|
||||
assert not any(r[0] == session._session_id for r in list_sessions())
|
||||
# Workstream is not auto-registered on init — only on /new or server creation
|
||||
assert resolve_workstream(session._ws_id) is None
|
||||
assert not any(r[0] == session._ws_id for r in list_workstreams_with_history())
|
||||
|
||||
|
||||
# ── save_message updates sessions.updated ─────────────────────────────
|
||||
# ── save_message updates workstreams.updated ──────────────────────────
|
||||
|
||||
|
||||
class TestSaveMessageUpdatesSession:
|
||||
class TestSaveMessageUpdatesWorkstream:
|
||||
def test_updated_timestamp_bumped(self, tmp_db):
|
||||
register_session("s1")
|
||||
register_workstream("s1")
|
||||
save_message("s1", "user", "first")
|
||||
rows = list_sessions()
|
||||
rows = list_workstreams_with_history()
|
||||
_original_updated = rows[0][4]
|
||||
|
||||
import time
|
||||
@@ -336,18 +335,18 @@ class TestSaveMessageUpdatesSession:
|
||||
time.sleep(0.01) # ensure different timestamp
|
||||
save_message("s1", "user", "hello")
|
||||
|
||||
rows = list_sessions()
|
||||
rows = list_workstreams_with_history()
|
||||
new_updated = rows[0][4]
|
||||
# updated should be same or later (sqlite datetime resolution is seconds,
|
||||
# so they may be equal in fast tests — just verify no error)
|
||||
assert new_updated is not None
|
||||
|
||||
|
||||
# ── Interrupted session repair ───────────────────────────────────────
|
||||
# ── Interrupted workstream repair ─────────────────────────────────────
|
||||
|
||||
|
||||
class TestInterruptedSessionRepair:
|
||||
"""load_session_messages() should strip trailing incomplete tool call turns."""
|
||||
class TestInterruptedWorkstreamRepair:
|
||||
"""load_messages() should strip trailing incomplete tool call turns."""
|
||||
|
||||
def test_complete_tool_turn_preserved(self, tmp_db):
|
||||
"""2 tool_calls + 2 tool_results = complete, no stripping."""
|
||||
@@ -356,7 +355,7 @@ class TestInterruptedSessionRepair:
|
||||
save_message("s1", "tool_call", None, "bash", '{"command":"pwd"}', "call_2")
|
||||
save_message("s1", "tool_result", "file.txt", tool_call_id="call_1")
|
||||
save_message("s1", "tool_result", "/home", tool_call_id="call_2")
|
||||
msgs = load_session_messages("s1")
|
||||
msgs = load_messages("s1")
|
||||
assert len(msgs) == 4 # user + assistant(2 calls) + 2 tool results
|
||||
|
||||
def test_partial_tool_results_stripped(self, tmp_db):
|
||||
@@ -365,7 +364,7 @@ class TestInterruptedSessionRepair:
|
||||
save_message("s1", "tool_call", None, "bash", '{"command":"ls"}', "call_1")
|
||||
save_message("s1", "tool_call", None, "bash", '{"command":"pwd"}', "call_2")
|
||||
save_message("s1", "tool_result", "file.txt", tool_call_id="call_1")
|
||||
msgs = load_session_messages("s1")
|
||||
msgs = load_messages("s1")
|
||||
assert len(msgs) == 1 # only user message remains
|
||||
assert msgs[0]["role"] == "user"
|
||||
|
||||
@@ -375,7 +374,7 @@ class TestInterruptedSessionRepair:
|
||||
save_message("s1", "assistant", "Let me check")
|
||||
save_message("s1", "tool_call", None, "bash", '{"command":"ls"}', "call_1")
|
||||
save_message("s1", "tool_call", None, "bash", '{"command":"pwd"}', "call_2")
|
||||
msgs = load_session_messages("s1")
|
||||
msgs = load_messages("s1")
|
||||
# assistant with content was merged into tool_call assistant, so stripped
|
||||
assert len(msgs) == 1
|
||||
assert msgs[0]["role"] == "user"
|
||||
@@ -386,42 +385,42 @@ class TestInterruptedSessionRepair:
|
||||
save_message("s1", "assistant", "response")
|
||||
save_message("s1", "user", "second")
|
||||
save_message("s1", "tool_call", None, "bash", '{"command":"ls"}', "call_1")
|
||||
msgs = load_session_messages("s1")
|
||||
msgs = load_messages("s1")
|
||||
assert len(msgs) == 3 # user + assistant + user (incomplete turn stripped)
|
||||
assert msgs[0]["role"] == "user"
|
||||
assert msgs[1]["role"] == "assistant"
|
||||
assert msgs[2]["role"] == "user"
|
||||
|
||||
|
||||
# ── Session config persistence ───────────────────────────────────────
|
||||
# ── Workstream config persistence ─────────────────────────────────────
|
||||
|
||||
|
||||
class TestSessionConfig:
|
||||
class TestWorkstreamConfig:
|
||||
def test_save_load_roundtrip(self, tmp_db):
|
||||
config = {"temperature": "0.3", "reasoning_effort": "high", "creative_mode": "False"}
|
||||
save_session_config("s1", config)
|
||||
loaded = load_session_config("s1")
|
||||
save_workstream_config("s1", config)
|
||||
loaded = load_workstream_config("s1")
|
||||
assert loaded == config
|
||||
|
||||
def test_update_existing_key(self, tmp_db):
|
||||
save_session_config("s1", {"temperature": "0.3"})
|
||||
save_session_config("s1", {"temperature": "0.7"})
|
||||
loaded = load_session_config("s1")
|
||||
save_workstream_config("s1", {"temperature": "0.3"})
|
||||
save_workstream_config("s1", {"temperature": "0.7"})
|
||||
loaded = load_workstream_config("s1")
|
||||
assert loaded["temperature"] == "0.7"
|
||||
|
||||
def test_missing_session_returns_empty(self, tmp_db):
|
||||
loaded = load_session_config("nonexistent")
|
||||
def test_missing_workstream_returns_empty(self, tmp_db):
|
||||
loaded = load_workstream_config("nonexistent")
|
||||
assert loaded == {}
|
||||
|
||||
def test_delete_session_removes_config(self, tmp_db):
|
||||
register_session("s1")
|
||||
def test_delete_workstream_removes_config(self, tmp_db):
|
||||
register_workstream("s1")
|
||||
save_message("s1", "user", "hi")
|
||||
save_session_config("s1", {"temperature": "0.5"})
|
||||
delete_session("s1")
|
||||
assert load_session_config("s1") == {}
|
||||
save_workstream_config("s1", {"temperature": "0.5"})
|
||||
delete_workstream("s1")
|
||||
assert load_workstream_config("s1") == {}
|
||||
|
||||
def test_resume_restores_config(self, tmp_db):
|
||||
"""ChatSession.resume_session() should restore persisted config."""
|
||||
"""ChatSession.resume() should restore persisted config."""
|
||||
client = MagicMock()
|
||||
client.models.list.return_value.data = [MagicMock(id="test-model")]
|
||||
ui = MagicMock()
|
||||
@@ -430,11 +429,11 @@ class TestSessionConfig:
|
||||
ui.on_state_change = MagicMock()
|
||||
ui.on_rename = MagicMock()
|
||||
|
||||
# Create a session with specific config
|
||||
register_session("orig")
|
||||
# Create a workstream with specific config
|
||||
register_workstream("orig")
|
||||
save_message("orig", "user", "hello")
|
||||
save_message("orig", "assistant", "hi there")
|
||||
save_session_config(
|
||||
save_workstream_config(
|
||||
"orig",
|
||||
{
|
||||
"temperature": "0.3",
|
||||
@@ -456,7 +455,7 @@ class TestSessionConfig:
|
||||
tool_timeout=30,
|
||||
)
|
||||
assert session.temperature == 0.7 # default
|
||||
result = session.resume_session("orig")
|
||||
result = session.resume("orig")
|
||||
assert result is True
|
||||
assert session.temperature == 0.3
|
||||
assert session.reasoning_effort == "high"
|
||||
@@ -465,84 +464,84 @@ class TestSessionConfig:
|
||||
assert session.creative_mode is True
|
||||
|
||||
|
||||
# ── Prune sessions ───────────────────────────────────────────────────
|
||||
# ── Prune workstreams ─────────────────────────────────────────────────
|
||||
|
||||
|
||||
class TestPruneSessions:
|
||||
class TestPruneWorkstreams:
|
||||
def test_orphan_removed(self, tmp_db):
|
||||
"""Session registered with no messages should be pruned."""
|
||||
register_session("orphan")
|
||||
orphans, stale = prune_sessions()
|
||||
"""Workstream registered with no messages should be pruned."""
|
||||
register_workstream("orphan")
|
||||
orphans, stale = prune_workstreams()
|
||||
assert orphans == 1
|
||||
assert list_sessions() == []
|
||||
assert list_workstreams_with_history() == []
|
||||
|
||||
def test_session_with_messages_kept(self, tmp_db):
|
||||
"""Session with messages should not be pruned."""
|
||||
register_session("active")
|
||||
def test_workstream_with_messages_kept(self, tmp_db):
|
||||
"""Workstream with messages should not be pruned."""
|
||||
register_workstream("active")
|
||||
save_message("active", "user", "hello")
|
||||
orphans, _stale = prune_sessions()
|
||||
orphans, _stale = prune_workstreams()
|
||||
assert orphans == 0
|
||||
assert len(list_sessions()) == 1
|
||||
assert len(list_workstreams_with_history()) == 1
|
||||
|
||||
def test_stale_unnamed_removed(self, tmp_db):
|
||||
"""Old unnamed session should be pruned by retention policy."""
|
||||
register_session("old1")
|
||||
"""Old unnamed workstream should be pruned by retention policy."""
|
||||
register_workstream("old1")
|
||||
save_message("old1", "user", "ancient message")
|
||||
# Force the updated timestamp to the past so it looks stale
|
||||
engine = get_storage()._engine # noqa: SLF001
|
||||
with engine.connect() as conn:
|
||||
conn.execute(
|
||||
sa.text("UPDATE sessions SET updated = '2020-01-01' WHERE session_id = 'old1'")
|
||||
sa.text("UPDATE workstreams SET updated = '2020-01-01' WHERE ws_id = 'old1'")
|
||||
)
|
||||
conn.commit()
|
||||
_orphans, stale = prune_sessions(retention_days=30)
|
||||
_orphans, stale = prune_workstreams(retention_days=30)
|
||||
assert stale == 1
|
||||
|
||||
def test_named_session_preserved(self, tmp_db):
|
||||
"""Session with alias should be kept regardless of age."""
|
||||
register_session("old2")
|
||||
set_session_alias("old2", "important")
|
||||
def test_named_workstream_preserved(self, tmp_db):
|
||||
"""Workstream with alias should be kept regardless of age."""
|
||||
register_workstream("old2")
|
||||
set_workstream_alias("old2", "important")
|
||||
save_message("old2", "user", "old but named")
|
||||
# Force old timestamp
|
||||
engine = get_storage()._engine # noqa: SLF001
|
||||
with engine.connect() as conn:
|
||||
conn.execute(
|
||||
sa.text("UPDATE sessions SET updated = '2020-01-01' WHERE session_id = 'old2'")
|
||||
sa.text("UPDATE workstreams SET updated = '2020-01-01' WHERE ws_id = 'old2'")
|
||||
)
|
||||
conn.commit()
|
||||
_orphans, stale = prune_sessions(retention_days=30)
|
||||
_orphans, stale = prune_workstreams(retention_days=30)
|
||||
assert stale == 0
|
||||
assert len(list_sessions()) == 1
|
||||
assert len(list_workstreams_with_history()) == 1
|
||||
|
||||
def test_fresh_unnamed_preserved(self, tmp_db):
|
||||
"""Recent unnamed session should not be pruned."""
|
||||
register_session("fresh")
|
||||
"""Recent unnamed workstream should not be pruned."""
|
||||
register_workstream("fresh")
|
||||
save_message("fresh", "user", "just now")
|
||||
_orphans, stale = prune_sessions(retention_days=30)
|
||||
_orphans, stale = prune_workstreams(retention_days=30)
|
||||
assert stale == 0
|
||||
assert len(list_sessions()) == 1
|
||||
assert len(list_workstreams_with_history()) == 1
|
||||
|
||||
def test_prune_removes_session_config(self, tmp_db):
|
||||
"""Pruning orphan/stale sessions should also remove their config rows."""
|
||||
register_session("orphan_cfg")
|
||||
save_session_config("orphan_cfg", {"temperature": "0.5"})
|
||||
def test_prune_removes_workstream_config(self, tmp_db):
|
||||
"""Pruning orphan/stale workstreams should also remove their config rows."""
|
||||
register_workstream("orphan_cfg")
|
||||
save_workstream_config("orphan_cfg", {"temperature": "0.5"})
|
||||
|
||||
register_session("stale_cfg")
|
||||
register_workstream("stale_cfg")
|
||||
save_message("stale_cfg", "user", "old")
|
||||
save_session_config("stale_cfg", {"temperature": "0.9"})
|
||||
save_workstream_config("stale_cfg", {"temperature": "0.9"})
|
||||
engine = get_storage()._engine # noqa: SLF001
|
||||
with engine.connect() as conn:
|
||||
conn.execute(
|
||||
sa.text("UPDATE sessions SET updated = '2020-01-01' WHERE session_id = 'stale_cfg'")
|
||||
sa.text("UPDATE workstreams SET updated = '2020-01-01' WHERE ws_id = 'stale_cfg'")
|
||||
)
|
||||
conn.commit()
|
||||
|
||||
# Both should have config before prune
|
||||
assert load_session_config("orphan_cfg") == {"temperature": "0.5"}
|
||||
assert load_session_config("stale_cfg") == {"temperature": "0.9"}
|
||||
assert load_workstream_config("orphan_cfg") == {"temperature": "0.5"}
|
||||
assert load_workstream_config("stale_cfg") == {"temperature": "0.9"}
|
||||
|
||||
prune_sessions(retention_days=30)
|
||||
prune_workstreams(retention_days=30)
|
||||
|
||||
# Config rows should be cleaned up
|
||||
assert load_session_config("orphan_cfg") == {}
|
||||
assert load_session_config("stale_cfg") == {}
|
||||
assert load_workstream_config("orphan_cfg") == {}
|
||||
assert load_workstream_config("stale_cfg") == {}
|
||||
|
||||
@@ -14,28 +14,28 @@ def backend(tmp_path):
|
||||
reset_storage()
|
||||
|
||||
|
||||
# -- Session operations --------------------------------------------------------
|
||||
# -- Workstream registration ---------------------------------------------------
|
||||
|
||||
|
||||
class TestRegisterSession:
|
||||
def test_register_creates_session(self, backend):
|
||||
backend.register_session("s1", title="Test")
|
||||
name = backend.get_session_name("s1")
|
||||
class TestRegisterWorkstream:
|
||||
def test_register_creates_workstream(self, backend):
|
||||
backend.register_workstream("s1", title="Test")
|
||||
name = backend.get_workstream_display_name("s1")
|
||||
assert name == "Test"
|
||||
|
||||
def test_register_idempotent(self, backend):
|
||||
backend.register_session("s1", title="First")
|
||||
backend.register_session("s1", title="Second")
|
||||
name = backend.get_session_name("s1")
|
||||
backend.register_workstream("s1", title="First")
|
||||
backend.register_workstream("s1", title="Second")
|
||||
name = backend.get_workstream_display_name("s1")
|
||||
assert name == "First" # INSERT OR IGNORE preserves first
|
||||
|
||||
|
||||
class TestSaveAndLoadMessages:
|
||||
def test_roundtrip(self, backend):
|
||||
backend.register_session("s1")
|
||||
backend.register_workstream("s1")
|
||||
backend.save_message("s1", "user", "hello")
|
||||
backend.save_message("s1", "assistant", "world")
|
||||
msgs = backend.load_session_messages("s1")
|
||||
msgs = backend.load_messages("s1")
|
||||
assert len(msgs) == 2
|
||||
assert msgs[0]["role"] == "user"
|
||||
assert msgs[0]["content"] == "hello"
|
||||
@@ -43,12 +43,12 @@ class TestSaveAndLoadMessages:
|
||||
assert msgs[1]["content"] == "world"
|
||||
|
||||
def test_tool_call_grouping(self, backend):
|
||||
backend.register_session("s1")
|
||||
backend.register_workstream("s1")
|
||||
backend.save_message("s1", "user", "do something")
|
||||
backend.save_message("s1", "tool_call", None, "bash", '{"cmd":"ls"}', tool_call_id="c1")
|
||||
backend.save_message("s1", "tool_result", "file.txt", tool_call_id="c1")
|
||||
backend.save_message("s1", "assistant", "done")
|
||||
msgs = backend.load_session_messages("s1")
|
||||
msgs = backend.load_messages("s1")
|
||||
assert len(msgs) == 4
|
||||
assert msgs[1]["role"] == "assistant"
|
||||
assert len(msgs[1]["tool_calls"]) == 1
|
||||
@@ -57,136 +57,136 @@ class TestSaveAndLoadMessages:
|
||||
assert msgs[2]["content"] == "file.txt"
|
||||
|
||||
def test_incomplete_turn_repair(self, backend):
|
||||
backend.register_session("s1")
|
||||
backend.register_workstream("s1")
|
||||
backend.save_message("s1", "user", "do something")
|
||||
backend.save_message("s1", "tool_call", None, "bash", '{"cmd":"ls"}', tool_call_id="c1")
|
||||
backend.save_message("s1", "tool_call", None, "read", '{"path":"a"}', tool_call_id="c2")
|
||||
# Only 1 result for 2 calls — incomplete turn
|
||||
backend.save_message("s1", "tool_result", "ok", tool_call_id="c1")
|
||||
msgs = backend.load_session_messages("s1")
|
||||
msgs = backend.load_messages("s1")
|
||||
# Incomplete turn should be stripped
|
||||
assert len(msgs) == 1 # only the user message remains
|
||||
|
||||
def test_provider_data_preserved(self, backend):
|
||||
import json
|
||||
|
||||
backend.register_session("s1")
|
||||
backend.register_workstream("s1")
|
||||
pd = json.dumps({"encrypted": True})
|
||||
backend.save_message("s1", "assistant", "hi", provider_data=pd)
|
||||
msgs = backend.load_session_messages("s1")
|
||||
msgs = backend.load_messages("s1")
|
||||
assert msgs[0].get("_provider_content") == {"encrypted": True}
|
||||
|
||||
def test_empty_session_returns_empty(self, backend):
|
||||
assert backend.load_session_messages("nonexistent") == []
|
||||
def test_empty_workstream_returns_empty(self, backend):
|
||||
assert backend.load_messages("nonexistent") == []
|
||||
|
||||
|
||||
class TestListSessions:
|
||||
def test_lists_sessions_with_messages(self, backend):
|
||||
backend.register_session("s1")
|
||||
class TestListWorkstreamsWithHistory:
|
||||
def test_lists_workstreams_with_messages(self, backend):
|
||||
backend.register_workstream("s1")
|
||||
backend.save_message("s1", "user", "hi")
|
||||
backend.register_session("s2") # no messages
|
||||
rows = backend.list_sessions()
|
||||
backend.register_workstream("s2") # no messages
|
||||
rows = backend.list_workstreams_with_history()
|
||||
assert len(rows) == 1
|
||||
assert rows[0][0] == "s1"
|
||||
|
||||
def test_respects_limit(self, backend):
|
||||
for i in range(5):
|
||||
sid = f"s{i}"
|
||||
backend.register_session(sid)
|
||||
backend.register_workstream(sid)
|
||||
backend.save_message(sid, "user", f"msg {i}")
|
||||
rows = backend.list_sessions(limit=3)
|
||||
rows = backend.list_workstreams_with_history(limit=3)
|
||||
assert len(rows) == 3
|
||||
|
||||
|
||||
class TestDeleteSession:
|
||||
class TestDeleteWorkstream:
|
||||
def test_deletes_all_data(self, backend):
|
||||
backend.register_session("s1")
|
||||
backend.register_workstream("s1")
|
||||
backend.save_message("s1", "user", "hi")
|
||||
backend.save_session_config("s1", {"temp": "0.5"})
|
||||
assert backend.delete_session("s1")
|
||||
assert backend.load_session_messages("s1") == []
|
||||
assert backend.load_session_config("s1") == {}
|
||||
assert backend.get_session_name("s1") is None
|
||||
backend.save_workstream_config("s1", {"temp": "0.5"})
|
||||
assert backend.delete_workstream("s1")
|
||||
assert backend.load_messages("s1") == []
|
||||
assert backend.load_workstream_config("s1") == {}
|
||||
assert backend.get_workstream_display_name("s1") is None
|
||||
|
||||
|
||||
class TestPruneSessions:
|
||||
class TestPruneWorkstreams:
|
||||
def test_orphan_removed(self, backend):
|
||||
backend.register_session("orphan")
|
||||
orphans, stale = backend.prune_sessions()
|
||||
backend.register_workstream("orphan")
|
||||
orphans, stale = backend.prune_workstreams()
|
||||
assert orphans == 1
|
||||
|
||||
def test_stale_removed(self, backend):
|
||||
import sqlalchemy as sa
|
||||
|
||||
backend.register_session("old")
|
||||
backend.register_workstream("old")
|
||||
backend.save_message("old", "user", "hi")
|
||||
# Force old timestamp
|
||||
with backend._engine.connect() as conn:
|
||||
conn.execute(
|
||||
sa.text("UPDATE sessions SET updated = '2020-01-01' WHERE session_id = 'old'")
|
||||
sa.text("UPDATE workstreams SET updated = '2020-01-01' WHERE ws_id = 'old'")
|
||||
)
|
||||
conn.commit()
|
||||
_, stale = backend.prune_sessions(retention_days=30)
|
||||
_, stale = backend.prune_workstreams(retention_days=30)
|
||||
assert stale == 1
|
||||
|
||||
|
||||
class TestResolveSession:
|
||||
class TestResolveWorkstream:
|
||||
def test_exact_alias(self, backend):
|
||||
backend.register_session("s1")
|
||||
backend.set_session_alias("s1", "myalias")
|
||||
assert backend.resolve_session("myalias") == "s1"
|
||||
backend.register_workstream("s1")
|
||||
backend.set_workstream_alias("s1", "myalias")
|
||||
assert backend.resolve_workstream("myalias") == "s1"
|
||||
|
||||
def test_exact_id(self, backend):
|
||||
backend.register_session("abc-123-def")
|
||||
assert backend.resolve_session("abc-123-def") == "abc-123-def"
|
||||
backend.register_workstream("abc-123-def")
|
||||
assert backend.resolve_workstream("abc-123-def") == "abc-123-def"
|
||||
|
||||
def test_prefix_match(self, backend):
|
||||
backend.register_session("abc-123-def")
|
||||
assert backend.resolve_session("abc") == "abc-123-def"
|
||||
backend.register_workstream("abc-123-def")
|
||||
assert backend.resolve_workstream("abc") == "abc-123-def"
|
||||
|
||||
def test_not_found(self, backend):
|
||||
assert backend.resolve_session("nonexistent") is None
|
||||
assert backend.resolve_workstream("nonexistent") is None
|
||||
|
||||
|
||||
# -- Session config ------------------------------------------------------------
|
||||
# -- Workstream config ---------------------------------------------------------
|
||||
|
||||
|
||||
class TestSessionConfig:
|
||||
class TestWorkstreamConfig:
|
||||
def test_roundtrip(self, backend):
|
||||
backend.register_session("s1")
|
||||
backend.save_session_config("s1", {"temperature": "0.7", "effort": "high"})
|
||||
cfg = backend.load_session_config("s1")
|
||||
backend.register_workstream("s1")
|
||||
backend.save_workstream_config("s1", {"temperature": "0.7", "effort": "high"})
|
||||
cfg = backend.load_workstream_config("s1")
|
||||
assert cfg == {"temperature": "0.7", "effort": "high"}
|
||||
|
||||
def test_empty_config(self, backend):
|
||||
assert backend.load_session_config("nonexistent") == {}
|
||||
assert backend.load_workstream_config("nonexistent") == {}
|
||||
|
||||
|
||||
# -- Session metadata ----------------------------------------------------------
|
||||
# -- Workstream metadata ------------------------------------------------------
|
||||
|
||||
|
||||
class TestSessionMetadata:
|
||||
class TestWorkstreamMetadata:
|
||||
def test_alias(self, backend):
|
||||
backend.register_session("s1")
|
||||
assert backend.set_session_alias("s1", "my-session")
|
||||
assert backend.get_session_name("s1") == "my-session"
|
||||
backend.register_workstream("s1")
|
||||
assert backend.set_workstream_alias("s1", "my-session")
|
||||
assert backend.get_workstream_display_name("s1") == "my-session"
|
||||
|
||||
def test_alias_conflict(self, backend):
|
||||
backend.register_session("s1")
|
||||
backend.register_session("s2")
|
||||
backend.set_session_alias("s1", "taken")
|
||||
assert not backend.set_session_alias("s2", "taken")
|
||||
backend.register_workstream("s1")
|
||||
backend.register_workstream("s2")
|
||||
backend.set_workstream_alias("s1", "taken")
|
||||
assert not backend.set_workstream_alias("s2", "taken")
|
||||
|
||||
def test_title(self, backend):
|
||||
backend.register_session("s1")
|
||||
backend.update_session_title("s1", "My Title")
|
||||
assert backend.get_session_name("s1") == "My Title"
|
||||
backend.register_workstream("s1")
|
||||
backend.update_workstream_title("s1", "My Title")
|
||||
assert backend.get_workstream_display_name("s1") == "My Title"
|
||||
|
||||
def test_alias_preferred_over_title(self, backend):
|
||||
backend.register_session("s1")
|
||||
backend.update_session_title("s1", "Title")
|
||||
backend.set_session_alias("s1", "Alias")
|
||||
assert backend.get_session_name("s1") == "Alias"
|
||||
backend.register_workstream("s1")
|
||||
backend.update_workstream_title("s1", "Title")
|
||||
backend.set_workstream_alias("s1", "Alias")
|
||||
assert backend.get_workstream_display_name("s1") == "Alias"
|
||||
|
||||
|
||||
# -- Key-value store -----------------------------------------------------------
|
||||
@@ -234,7 +234,7 @@ class TestKVStore:
|
||||
|
||||
class TestSearch:
|
||||
def test_search_history(self, backend):
|
||||
backend.register_session("s1")
|
||||
backend.register_workstream("s1")
|
||||
backend.save_message("s1", "user", "hello world")
|
||||
backend.save_message("s1", "user", "goodbye world")
|
||||
results = backend.search_history("hello")
|
||||
@@ -242,7 +242,7 @@ class TestSearch:
|
||||
assert any("hello" in str(r[3]) for r in results)
|
||||
|
||||
def test_search_recent(self, backend):
|
||||
backend.register_session("s1")
|
||||
backend.register_workstream("s1")
|
||||
backend.save_message("s1", "user", "msg1")
|
||||
backend.save_message("s1", "user", "msg2")
|
||||
results = backend.search_history_recent(limit=1)
|
||||
@@ -293,15 +293,14 @@ class TestWorkstreams:
|
||||
assert len(rows) == 1
|
||||
assert rows[0][0] == "ws1"
|
||||
|
||||
def test_session_with_ws_id(self, backend):
|
||||
def test_workstream_with_messages_in_history(self, backend):
|
||||
backend.register_workstream("ws1", node_id="node-a")
|
||||
backend.register_session("s1", node_id="node-a", ws_id="ws1")
|
||||
backend.save_message("s1", "user", "hello")
|
||||
rows = backend.list_sessions()
|
||||
backend.save_message("ws1", "user", "hello")
|
||||
rows = backend.list_workstreams_with_history()
|
||||
assert len(rows) == 1
|
||||
# Columns: sid, alias, title, created, updated, count, node_id, ws_id
|
||||
# Columns: ws_id, alias, title, created, updated, count, node_id
|
||||
assert rows[0][0] == "ws1"
|
||||
assert rows[0][6] == "node-a"
|
||||
assert rows[0][7] == "ws1"
|
||||
|
||||
|
||||
# -- Lifecycle -----------------------------------------------------------------
|
||||
|
||||
@@ -0,0 +1,86 @@
|
||||
"""Tests for turnstone.core.policy."""
|
||||
|
||||
import pytest
|
||||
|
||||
from turnstone.core.policy import evaluate_tool_policies_batch, evaluate_tool_policy
|
||||
from turnstone.core.storage._sqlite import SQLiteBackend
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def storage(tmp_path):
|
||||
path = str(tmp_path / "test.db")
|
||||
backend = SQLiteBackend(path)
|
||||
yield backend
|
||||
backend.close()
|
||||
|
||||
|
||||
def test_no_policies_returns_none(storage):
|
||||
result = evaluate_tool_policy(storage, "bash")
|
||||
assert result is None
|
||||
|
||||
|
||||
def test_exact_match_allow(storage):
|
||||
storage.create_tool_policy("p1", "allow-read", "read_file", "allow", 0)
|
||||
assert evaluate_tool_policy(storage, "read_file") == "allow"
|
||||
assert evaluate_tool_policy(storage, "write_file") is None
|
||||
|
||||
|
||||
def test_glob_match_deny(storage):
|
||||
storage.create_tool_policy("p1", "block-bash", "bash*", "deny", 0)
|
||||
assert evaluate_tool_policy(storage, "bash") == "deny"
|
||||
assert evaluate_tool_policy(storage, "bash_exec") == "deny"
|
||||
assert evaluate_tool_policy(storage, "read_file") is None
|
||||
|
||||
|
||||
def test_wildcard_match(storage):
|
||||
storage.create_tool_policy("p1", "ask-all", "*", "ask", 0)
|
||||
assert evaluate_tool_policy(storage, "anything") == "ask"
|
||||
|
||||
|
||||
def test_priority_ordering(storage):
|
||||
# Higher priority wins
|
||||
storage.create_tool_policy("p1", "allow-all", "*", "allow", 0)
|
||||
storage.create_tool_policy("p2", "deny-bash", "bash*", "deny", 100)
|
||||
assert evaluate_tool_policy(storage, "bash") == "deny" # p2 matches first (higher priority)
|
||||
assert evaluate_tool_policy(storage, "read_file") == "allow" # p1 matches
|
||||
|
||||
|
||||
def test_disabled_policy_skipped(storage):
|
||||
storage.create_tool_policy("p1", "block-bash", "bash*", "deny", 100, enabled=False)
|
||||
storage.create_tool_policy("p2", "allow-all", "*", "allow", 0)
|
||||
assert evaluate_tool_policy(storage, "bash") == "allow" # p1 disabled, falls through to p2
|
||||
|
||||
|
||||
def test_batch_evaluation(storage):
|
||||
storage.create_tool_policy("p1", "block-bash", "bash*", "deny", 100)
|
||||
storage.create_tool_policy("p2", "allow-read", "read_*", "allow", 50)
|
||||
results = evaluate_tool_policies_batch(storage, ["bash", "read_file", "write_file"])
|
||||
assert results["bash"] == "deny"
|
||||
assert results["read_file"] == "allow"
|
||||
assert results["write_file"] is None
|
||||
|
||||
|
||||
def test_storage_failure_returns_none():
|
||||
"""Graceful degradation on storage failure."""
|
||||
|
||||
class BrokenStorage:
|
||||
def list_tool_policies(self, org_id=""):
|
||||
raise RuntimeError("boom")
|
||||
|
||||
assert evaluate_tool_policy(BrokenStorage(), "bash") is None
|
||||
|
||||
|
||||
def test_batch_storage_failure():
|
||||
class BrokenStorage:
|
||||
def list_tool_policies(self, org_id=""):
|
||||
raise RuntimeError("boom")
|
||||
|
||||
results = evaluate_tool_policies_batch(BrokenStorage(), ["a", "b"])
|
||||
assert results == {"a": None, "b": None}
|
||||
|
||||
|
||||
def test_first_match_wins(storage):
|
||||
# Two policies match, first by priority wins
|
||||
storage.create_tool_policy("p1", "deny-bash", "bash*", "deny", 100)
|
||||
storage.create_tool_policy("p2", "allow-bash", "bash*", "allow", 50)
|
||||
assert evaluate_tool_policy(storage, "bash_exec") == "deny"
|
||||
@@ -0,0 +1,254 @@
|
||||
"""Tests for turnstone.core.tool_search — BM25 index and tool search manager."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import pytest
|
||||
|
||||
from turnstone.core.tool_search import (
|
||||
BM25Index,
|
||||
ToolSearchManager,
|
||||
_mcp_server_summary,
|
||||
_tokenize,
|
||||
_tool_name,
|
||||
)
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Helpers
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _make_tool(name: str, description: str = "") -> dict:
|
||||
"""Create a minimal OpenAI-format tool dict for testing."""
|
||||
return {
|
||||
"type": "function",
|
||||
"function": {
|
||||
"name": name,
|
||||
"description": description or f"Tool {name}",
|
||||
"parameters": {"type": "object", "properties": {}},
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# BM25Index tests
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestTokenize:
|
||||
def test_basic_split(self):
|
||||
assert _tokenize("hello world") == ["hello", "world"]
|
||||
|
||||
def test_underscore_split(self):
|
||||
assert _tokenize("create_issue") == ["create", "issue"]
|
||||
|
||||
def test_mixed_delimiters(self):
|
||||
assert _tokenize("mcp__github__create-issue") == ["mcp", "github", "create", "issue"]
|
||||
|
||||
def test_empty_string(self):
|
||||
assert _tokenize("") == []
|
||||
|
||||
def test_lowercased(self):
|
||||
assert _tokenize("GitHub Create") == ["github", "create"]
|
||||
|
||||
|
||||
class TestBM25Index:
|
||||
def test_empty_corpus(self):
|
||||
idx = BM25Index([])
|
||||
assert idx.search("test") == []
|
||||
|
||||
def test_empty_query(self):
|
||||
idx = BM25Index(["hello world", "foo bar"])
|
||||
assert idx.search("") == []
|
||||
|
||||
def test_single_document(self):
|
||||
idx = BM25Index(["create github issue"])
|
||||
assert idx.search("github") == [0]
|
||||
|
||||
def test_ranking_order(self):
|
||||
docs = [
|
||||
"list_repos List all repositories",
|
||||
"create_issue Create a new GitHub issue",
|
||||
"get_issue Get details of a GitHub issue",
|
||||
]
|
||||
idx = BM25Index(docs)
|
||||
results = idx.search("github issue")
|
||||
# Both issue-related docs should rank above list_repos
|
||||
assert 1 in results[:2]
|
||||
assert 2 in results[:2]
|
||||
|
||||
def test_top_k_limit(self):
|
||||
docs = [f"tool_{i} description {i}" for i in range(20)]
|
||||
idx = BM25Index(docs)
|
||||
results = idx.search("tool description", k=3)
|
||||
assert len(results) <= 3
|
||||
|
||||
def test_no_match(self):
|
||||
idx = BM25Index(["alpha beta gamma"])
|
||||
assert idx.search("zzzzz") == []
|
||||
|
||||
def test_exact_name_match_ranks_high(self):
|
||||
docs = [
|
||||
"send_email Send an email message",
|
||||
"send_slack Send a Slack message",
|
||||
"read_email Read email inbox",
|
||||
]
|
||||
idx = BM25Index(docs)
|
||||
results = idx.search("send email")
|
||||
assert results[0] == 0 # send_email should rank first
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# ToolSearchManager tests
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestToolSearchManager:
|
||||
@pytest.fixture()
|
||||
def builtin_tools(self):
|
||||
return [
|
||||
_make_tool("bash", "Execute shell commands"),
|
||||
_make_tool("read_file", "Read a file"),
|
||||
_make_tool("edit_file", "Edit a file"),
|
||||
]
|
||||
|
||||
@pytest.fixture()
|
||||
def mcp_tools(self):
|
||||
return [
|
||||
_make_tool("mcp__github__create_issue", "Create a new GitHub issue"),
|
||||
_make_tool("mcp__github__list_issues", "List GitHub issues"),
|
||||
_make_tool("mcp__github__get_repo", "Get repository details"),
|
||||
_make_tool("mcp__slack__send_message", "Send a Slack message"),
|
||||
_make_tool("mcp__slack__list_channels", "List Slack channels"),
|
||||
_make_tool("mcp__jira__create_ticket", "Create a Jira ticket"),
|
||||
]
|
||||
|
||||
@pytest.fixture()
|
||||
def manager(self, builtin_tools, mcp_tools):
|
||||
all_tools = builtin_tools + mcp_tools
|
||||
return ToolSearchManager(
|
||||
all_tools,
|
||||
always_on_names={"bash", "read_file", "edit_file"},
|
||||
threshold=5,
|
||||
max_results=3,
|
||||
)
|
||||
|
||||
def test_should_activate_above_threshold(self, manager):
|
||||
assert manager.should_activate()
|
||||
|
||||
def test_should_not_activate_below_threshold(self, builtin_tools):
|
||||
mgr = ToolSearchManager(builtin_tools, always_on_names={"bash", "read_file", "edit_file"})
|
||||
assert not mgr.should_activate()
|
||||
|
||||
def test_visible_tools_initially_builtin_only(self, manager):
|
||||
visible = manager.get_visible_tools()
|
||||
names = {_tool_name(t) for t in visible}
|
||||
assert names == {"bash", "read_file", "edit_file"}
|
||||
|
||||
def test_deferred_tools_excludes_builtin(self, manager):
|
||||
deferred = manager.get_deferred_tools()
|
||||
names = {_tool_name(t) for t in deferred}
|
||||
assert "bash" not in names
|
||||
assert "mcp__github__create_issue" in names
|
||||
|
||||
def test_search_returns_relevant_tools(self, manager):
|
||||
results = manager.search("github issue")
|
||||
names = {_tool_name(t) for t in results}
|
||||
assert "mcp__github__create_issue" in names or "mcp__github__list_issues" in names
|
||||
|
||||
def test_search_respects_max_results(self, manager):
|
||||
results = manager.search("tool")
|
||||
assert len(results) <= 3
|
||||
|
||||
def test_search_excludes_already_expanded(self, manager):
|
||||
# Expand a github tool, then search for github — expanded tool should not appear
|
||||
manager.expand_visible(["mcp__github__create_issue"])
|
||||
results = manager.search("github issue")
|
||||
names = {_tool_name(t) for t in results}
|
||||
assert "mcp__github__create_issue" not in names
|
||||
|
||||
def test_expand_visible_adds_tools(self, manager):
|
||||
manager.expand_visible(["mcp__github__create_issue"])
|
||||
visible = manager.get_visible_tools()
|
||||
names = {_tool_name(t) for t in visible}
|
||||
assert "mcp__github__create_issue" in names
|
||||
|
||||
def test_expand_visible_returns_newly_added(self, manager):
|
||||
added = manager.expand_visible(["mcp__github__create_issue", "mcp__slack__send_message"])
|
||||
assert len(added) == 2
|
||||
names = {_tool_name(t) for t in added}
|
||||
assert names == {"mcp__github__create_issue", "mcp__slack__send_message"}
|
||||
|
||||
def test_expand_visible_idempotent(self, manager):
|
||||
manager.expand_visible(["mcp__github__create_issue"])
|
||||
added = manager.expand_visible(["mcp__github__create_issue"])
|
||||
assert added == []
|
||||
|
||||
def test_expand_visible_ignores_unknown(self, manager):
|
||||
added = manager.expand_visible(["nonexistent_tool"])
|
||||
assert added == []
|
||||
|
||||
def test_get_expanded_names_empty(self, manager):
|
||||
assert manager.get_expanded_names() == []
|
||||
|
||||
def test_get_expanded_names_after_expand(self, manager):
|
||||
manager.expand_visible(["mcp__github__create_issue", "mcp__slack__send_message"])
|
||||
names = manager.get_expanded_names()
|
||||
assert names == ["mcp__github__create_issue", "mcp__slack__send_message"]
|
||||
|
||||
def test_deferred_excludes_expanded(self, manager):
|
||||
manager.expand_visible(["mcp__github__create_issue"])
|
||||
deferred = manager.get_deferred_tools()
|
||||
names = {_tool_name(t) for t in deferred}
|
||||
assert "mcp__github__create_issue" not in names
|
||||
|
||||
def test_get_all_tools_returns_everything(self, manager, builtin_tools, mcp_tools):
|
||||
assert len(manager.get_all_tools()) == len(builtin_tools) + len(mcp_tools)
|
||||
|
||||
def test_search_tool_definition_format(self, manager):
|
||||
defn = manager.get_search_tool_definition()
|
||||
assert defn["type"] == "function"
|
||||
fn = defn["function"]
|
||||
assert fn["name"] == "tool_search"
|
||||
assert "query" in fn["parameters"]["properties"]
|
||||
assert "query" in fn["parameters"]["required"]
|
||||
|
||||
def test_search_tool_description_has_server_hint(self, manager):
|
||||
defn = manager.get_search_tool_definition()
|
||||
desc = defn["function"]["description"]
|
||||
assert "github" in desc
|
||||
assert "slack" in desc
|
||||
assert "jira" in desc
|
||||
|
||||
def test_format_search_results_empty(self, manager):
|
||||
text = manager.format_search_results([])
|
||||
assert "No matching tools found" in text
|
||||
|
||||
def test_format_search_results_with_tools(self, manager, mcp_tools):
|
||||
text = manager.format_search_results(mcp_tools[:2])
|
||||
assert "Found 2" in text
|
||||
assert "mcp__github__create_issue" in text
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Helper function tests
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestMCPServerSummary:
|
||||
def test_groups_by_server(self):
|
||||
tools = [
|
||||
_make_tool("mcp__github__a"),
|
||||
_make_tool("mcp__github__b"),
|
||||
_make_tool("mcp__slack__c"),
|
||||
]
|
||||
summary = _mcp_server_summary(tools)
|
||||
assert "github (2 tools)" in summary
|
||||
assert "slack (1 tool)" in summary
|
||||
|
||||
def test_non_mcp_tools_counted_as_other(self):
|
||||
tools = [_make_tool("custom_tool")]
|
||||
summary = _mcp_server_summary(tools)
|
||||
assert "other (1 tool)" in summary
|
||||
|
||||
def test_empty_list(self):
|
||||
assert _mcp_server_summary([]) == ""
|
||||
@@ -72,7 +72,7 @@ class TestToolsMetadata:
|
||||
"""Validate the metadata extracted from JSON files."""
|
||||
|
||||
def test_tool_count(self):
|
||||
assert len(TOOLS) == 15
|
||||
assert len(TOOLS) == 16
|
||||
|
||||
def test_agent_tools_count(self):
|
||||
assert len(AGENT_TOOLS) == 7
|
||||
@@ -97,11 +97,12 @@ class TestToolsMetadata:
|
||||
"web_fetch": "url",
|
||||
"web_search": "query",
|
||||
"task": "prompt",
|
||||
"plan": "prompt",
|
||||
"create_plan": "goal",
|
||||
"remember": "key",
|
||||
"recall": "query",
|
||||
"forget": "key",
|
||||
"notify": "message",
|
||||
"watch": "command",
|
||||
}
|
||||
assert expected == PRIMARY_KEY_MAP
|
||||
|
||||
|
||||
+13
-19
@@ -65,6 +65,14 @@ class TestUserCRUD:
|
||||
db.delete_user("u1")
|
||||
assert len(db.list_api_tokens("u1")) == 0
|
||||
|
||||
def test_delete_cascades_user_roles(self, db):
|
||||
db.create_user("u1", "admin", "Admin", "$2b$hash")
|
||||
db.create_role("r1", "editor", "Editor", "read,write", builtin=False, org_id="")
|
||||
db.assign_role("u1", "r1")
|
||||
assert len(db.list_user_roles("u1")) == 1
|
||||
db.delete_user("u1")
|
||||
assert len(db.list_user_roles("u1")) == 0
|
||||
|
||||
|
||||
class TestApiTokenCRUD:
|
||||
def test_create_and_lookup_by_hash(self, db):
|
||||
@@ -127,21 +135,7 @@ class TestApiTokenCRUD:
|
||||
assert "expires" not in tok
|
||||
|
||||
|
||||
class TestSessionWorkstreamUserId:
|
||||
def test_register_session_with_user_id(self, db):
|
||||
db.register_session("s1", user_id="u1")
|
||||
# Verify via raw SQL that user_id is stored
|
||||
import sqlalchemy as sa
|
||||
|
||||
from turnstone.core.storage._schema import sessions
|
||||
|
||||
with db._engine.connect() as conn:
|
||||
row = conn.execute(
|
||||
sa.select(sessions.c.user_id).where(sessions.c.session_id == "s1")
|
||||
).fetchone()
|
||||
assert row is not None
|
||||
assert row[0] == "u1"
|
||||
|
||||
class TestWorkstreamUserId:
|
||||
def test_register_workstream_with_user_id(self, db):
|
||||
db.register_workstream("ws1", user_id="u1")
|
||||
import sqlalchemy as sa
|
||||
@@ -155,15 +149,15 @@ class TestSessionWorkstreamUserId:
|
||||
assert row is not None
|
||||
assert row[0] == "u1"
|
||||
|
||||
def test_register_session_without_user_id(self, db):
|
||||
db.register_session("s1")
|
||||
def test_register_workstream_without_user_id(self, db):
|
||||
db.register_workstream("ws1")
|
||||
import sqlalchemy as sa
|
||||
|
||||
from turnstone.core.storage._schema import sessions
|
||||
from turnstone.core.storage._schema import workstreams
|
||||
|
||||
with db._engine.connect() as conn:
|
||||
row = conn.execute(
|
||||
sa.select(sessions.c.user_id).where(sessions.c.session_id == "s1")
|
||||
sa.select(workstreams.c.user_id).where(workstreams.c.ws_id == "ws1")
|
||||
).fetchone()
|
||||
assert row is not None
|
||||
assert row[0] is None
|
||||
|
||||
@@ -0,0 +1,487 @@
|
||||
"""Tests for the watch module — duration parsing, condition evaluation, WatchRunner."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import UTC, datetime
|
||||
from unittest.mock import MagicMock
|
||||
|
||||
import pytest
|
||||
|
||||
from turnstone.core.watch import (
|
||||
WatchRunner,
|
||||
evaluate_condition,
|
||||
format_interval,
|
||||
format_watch_message,
|
||||
parse_duration,
|
||||
validate_condition,
|
||||
)
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# parse_duration
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestParseDuration:
|
||||
def test_seconds(self):
|
||||
assert parse_duration("30s") == 30.0
|
||||
|
||||
def test_minutes(self):
|
||||
assert parse_duration("5m") == 300.0
|
||||
|
||||
def test_hours(self):
|
||||
assert parse_duration("1h") == 3600.0
|
||||
|
||||
def test_compound(self):
|
||||
assert parse_duration("2h30m") == 9000.0
|
||||
|
||||
def test_bare_number(self):
|
||||
assert parse_duration("90") == 90.0
|
||||
|
||||
def test_bare_float(self):
|
||||
assert parse_duration("10.5") == 10.5
|
||||
|
||||
def test_whitespace(self):
|
||||
assert parse_duration(" 5m ") == 300.0
|
||||
|
||||
def test_case_insensitive(self):
|
||||
assert parse_duration("1H30M") == 5400.0
|
||||
|
||||
def test_empty_raises(self):
|
||||
with pytest.raises(ValueError, match="empty"):
|
||||
parse_duration("")
|
||||
|
||||
def test_invalid_raises(self):
|
||||
with pytest.raises(ValueError, match="invalid duration"):
|
||||
parse_duration("abc")
|
||||
|
||||
def test_negative_raises(self):
|
||||
with pytest.raises(ValueError, match="positive"):
|
||||
parse_duration("-5")
|
||||
|
||||
def test_zero_raises(self):
|
||||
with pytest.raises(ValueError, match="positive"):
|
||||
parse_duration("0")
|
||||
|
||||
def test_zero_duration_raises(self):
|
||||
with pytest.raises(ValueError, match="positive"):
|
||||
parse_duration("0s")
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# validate_condition
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestValidateCondition:
|
||||
def test_valid_expression(self):
|
||||
assert validate_condition('data["state"] == "MERGED"') is None
|
||||
|
||||
def test_valid_simple(self):
|
||||
assert validate_condition('"error" in output') is None
|
||||
|
||||
def test_valid_compound(self):
|
||||
assert validate_condition('changed and "ready" in output.lower()') is None
|
||||
|
||||
def test_syntax_error(self):
|
||||
result = validate_condition("if True:")
|
||||
assert result is not None
|
||||
assert "syntax" in result.lower()
|
||||
|
||||
def test_incomplete_expression(self):
|
||||
result = validate_condition("==")
|
||||
assert result is not None
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# evaluate_condition
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestEvaluateCondition:
|
||||
def test_none_first_poll_no_fire(self):
|
||||
"""With stop_on=None, first poll (prev_output=None) should not fire."""
|
||||
fired, reason = evaluate_condition(None, "hello", 0, None)
|
||||
assert not fired
|
||||
|
||||
def test_none_change_detected(self):
|
||||
fired, reason = evaluate_condition(None, "world", 0, "hello")
|
||||
assert fired
|
||||
assert "changed" in reason
|
||||
|
||||
def test_none_no_change(self):
|
||||
fired, reason = evaluate_condition(None, "same", 0, "same")
|
||||
assert not fired
|
||||
|
||||
def test_string_match(self):
|
||||
fired, reason = evaluate_condition('"error" in output', "has error here", 0, None)
|
||||
assert fired
|
||||
|
||||
def test_string_no_match(self):
|
||||
fired, reason = evaluate_condition('"error" in output', "all good", 0, None)
|
||||
assert not fired
|
||||
|
||||
def test_exit_code(self):
|
||||
fired, reason = evaluate_condition("exit_code != 0", "fail", 1, None)
|
||||
assert fired
|
||||
|
||||
def test_exit_code_zero(self):
|
||||
fired, reason = evaluate_condition("exit_code != 0", "ok", 0, None)
|
||||
assert not fired
|
||||
|
||||
def test_json_data(self):
|
||||
output = '{"state": "MERGED"}'
|
||||
fired, reason = evaluate_condition('data["state"] == "MERGED"', output, 0, None)
|
||||
assert fired
|
||||
|
||||
def test_json_data_no_match(self):
|
||||
output = '{"state": "OPEN"}'
|
||||
fired, reason = evaluate_condition('data["state"] == "MERGED"', output, 0, None)
|
||||
assert not fired
|
||||
|
||||
def test_json_data_none_for_non_json(self):
|
||||
"""Non-JSON output should have data=None."""
|
||||
fired, reason = evaluate_condition("data is None", "plain text", 0, None)
|
||||
assert fired
|
||||
|
||||
def test_changed_variable(self):
|
||||
fired, reason = evaluate_condition("changed", "new", 0, "old")
|
||||
assert fired
|
||||
|
||||
def test_changed_false(self):
|
||||
fired, reason = evaluate_condition("changed", "same", 0, "same")
|
||||
assert not fired
|
||||
|
||||
def test_compound_condition(self):
|
||||
fired, reason = evaluate_condition(
|
||||
'changed and "ready" in output.lower()',
|
||||
"System Ready",
|
||||
0,
|
||||
"System Starting",
|
||||
)
|
||||
assert fired
|
||||
|
||||
def test_invalid_expression_no_crash(self):
|
||||
fired, reason = evaluate_condition("1/0", "hello", 0, None)
|
||||
assert not fired
|
||||
assert "error" in reason.lower()
|
||||
|
||||
def test_no_import_builtin(self):
|
||||
"""__import__ should not be accessible."""
|
||||
fired, reason = evaluate_condition("__import__('os')", "hello", 0, None)
|
||||
assert not fired
|
||||
assert "error" in reason.lower()
|
||||
|
||||
def test_no_open_builtin(self):
|
||||
fired, reason = evaluate_condition("open('/etc/passwd')", "hello", 0, None)
|
||||
assert not fired
|
||||
assert "error" in reason.lower()
|
||||
|
||||
def test_no_exec_builtin(self):
|
||||
fired, reason = evaluate_condition("exec('print(1)')", "hello", 0, None)
|
||||
assert not fired
|
||||
assert "error" in reason.lower()
|
||||
|
||||
def test_no_eval_builtin(self):
|
||||
fired, reason = evaluate_condition("eval('1+1')", "hello", 0, None)
|
||||
assert not fired
|
||||
assert "error" in reason.lower()
|
||||
|
||||
def test_no_compile_builtin(self):
|
||||
fired, reason = evaluate_condition("compile('1','','eval')", "hello", 0, None)
|
||||
assert not fired
|
||||
assert "error" in reason.lower()
|
||||
|
||||
def test_safe_len(self):
|
||||
fired, reason = evaluate_condition("len(output) > 0", "hello", 0, None)
|
||||
assert fired
|
||||
|
||||
def test_safe_sorted(self):
|
||||
fired, reason = evaluate_condition("sorted([3,1,2]) == [1,2,3]", "x", 0, None)
|
||||
assert fired
|
||||
|
||||
def test_data_get_method(self):
|
||||
output = '{"mergedAt": "2024-01-15"}'
|
||||
fired, reason = evaluate_condition('data.get("mergedAt") is not None', output, 0, None)
|
||||
assert fired
|
||||
|
||||
def test_prev_output_available(self):
|
||||
fired, reason = evaluate_condition(
|
||||
"prev_output is not None and output != prev_output",
|
||||
"new",
|
||||
0,
|
||||
"old",
|
||||
)
|
||||
assert fired
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# format_interval
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestFormatInterval:
|
||||
def test_seconds(self):
|
||||
assert format_interval(30) == "30s"
|
||||
|
||||
def test_exactly_60(self):
|
||||
assert format_interval(60) == "1m"
|
||||
|
||||
def test_minutes(self):
|
||||
assert format_interval(300) == "5m"
|
||||
|
||||
def test_exactly_3600(self):
|
||||
assert format_interval(3600) == "1h"
|
||||
|
||||
def test_hours_and_minutes(self):
|
||||
assert format_interval(5400) == "1h30m"
|
||||
|
||||
def test_hours_only(self):
|
||||
assert format_interval(7200) == "2h"
|
||||
|
||||
def test_large_value(self):
|
||||
assert format_interval(86400) == "24h"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# format_watch_message
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestFormatWatchMessage:
|
||||
def test_basic(self):
|
||||
msg = format_watch_message(
|
||||
name="pr-review",
|
||||
command="gh pr view --json state",
|
||||
output='{"state": "MERGED"}',
|
||||
poll_count=5,
|
||||
max_polls=100,
|
||||
elapsed_secs=1500,
|
||||
stop_on='data["state"] == "MERGED"',
|
||||
is_final=True,
|
||||
reason='condition met: data["state"] == "MERGED"',
|
||||
)
|
||||
assert "pr-review" in msg
|
||||
assert "poll #5/100" in msg
|
||||
assert "25m" in msg
|
||||
assert "gh pr view --json state" in msg
|
||||
assert "MERGED" in msg
|
||||
assert "auto-cancelled" in msg.lower()
|
||||
# Model should see the condition it was waiting for
|
||||
assert "condition:" in msg.lower()
|
||||
|
||||
def test_non_final(self):
|
||||
msg = format_watch_message(
|
||||
name="deploy",
|
||||
command="curl -s http://localhost/health",
|
||||
output="ok",
|
||||
poll_count=3,
|
||||
max_polls=50,
|
||||
elapsed_secs=90,
|
||||
stop_on=None,
|
||||
is_final=False,
|
||||
reason="",
|
||||
)
|
||||
assert "deploy" in msg
|
||||
assert "auto-cancelled" not in msg.lower()
|
||||
# Change-detection mode should be indicated
|
||||
assert "output change" in msg.lower()
|
||||
|
||||
def test_max_polls_final(self):
|
||||
msg = format_watch_message(
|
||||
name="test",
|
||||
command="echo hello",
|
||||
output="hello",
|
||||
poll_count=100,
|
||||
max_polls=100,
|
||||
elapsed_secs=6000,
|
||||
stop_on=None,
|
||||
is_final=True,
|
||||
reason="",
|
||||
)
|
||||
assert "max polls" in msg.lower()
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# WatchRunner
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestWatchRunner:
|
||||
def _make_runner(self, storage=None, **kwargs):
|
||||
if storage is None:
|
||||
storage = MagicMock()
|
||||
storage.list_due_watches.return_value = []
|
||||
return WatchRunner(
|
||||
storage=storage,
|
||||
node_id="test-node",
|
||||
check_interval=0.1,
|
||||
tool_timeout=5,
|
||||
**kwargs,
|
||||
)
|
||||
|
||||
def test_start_stop(self):
|
||||
runner = self._make_runner()
|
||||
runner.start()
|
||||
assert runner._thread is not None
|
||||
assert runner._thread.is_alive()
|
||||
runner.stop()
|
||||
assert runner._thread is None
|
||||
|
||||
def test_tick_calls_list_due(self):
|
||||
storage = MagicMock()
|
||||
storage.list_due_watches.return_value = []
|
||||
runner = self._make_runner(storage=storage)
|
||||
runner._tick()
|
||||
storage.list_due_watches.assert_called_once()
|
||||
|
||||
def test_poll_watch_runs_command(self):
|
||||
storage = MagicMock()
|
||||
storage.update_watch.return_value = True
|
||||
runner = self._make_runner(storage=storage)
|
||||
dispatch_fn = MagicMock()
|
||||
runner.set_dispatch_fn("ws-1", dispatch_fn)
|
||||
|
||||
watch_row = {
|
||||
"watch_id": "abc123",
|
||||
"ws_id": "ws-1",
|
||||
"name": "test-watch",
|
||||
"command": "echo hello",
|
||||
"stop_on": '"hello" in output',
|
||||
"max_polls": 100,
|
||||
"poll_count": 0,
|
||||
"last_output": None,
|
||||
"interval_secs": 60,
|
||||
"created": datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%S"),
|
||||
}
|
||||
runner._poll_watch(watch_row)
|
||||
|
||||
# Should update the watch in storage
|
||||
storage.update_watch.assert_called_once()
|
||||
call_kwargs = storage.update_watch.call_args
|
||||
assert call_kwargs[0][0] == "abc123" # watch_id
|
||||
assert call_kwargs[1]["poll_count"] == 1
|
||||
# Condition should fire (output contains "hello")
|
||||
assert call_kwargs[1]["active"] is False # deactivated
|
||||
# Should dispatch result
|
||||
dispatch_fn.assert_called_once()
|
||||
|
||||
def test_poll_watch_no_fire_on_first_change_detection(self):
|
||||
storage = MagicMock()
|
||||
storage.update_watch.return_value = True
|
||||
runner = self._make_runner(storage=storage)
|
||||
dispatch_fn = MagicMock()
|
||||
runner.set_dispatch_fn("ws-1", dispatch_fn)
|
||||
|
||||
watch_row = {
|
||||
"watch_id": "abc123",
|
||||
"ws_id": "ws-1",
|
||||
"name": "test-watch",
|
||||
"command": "echo hello",
|
||||
"stop_on": None, # change detection
|
||||
"max_polls": 100,
|
||||
"poll_count": 0,
|
||||
"last_output": None, # first poll
|
||||
"interval_secs": 60,
|
||||
"created": datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%S"),
|
||||
}
|
||||
runner._poll_watch(watch_row)
|
||||
|
||||
# First poll with change detection should not fire
|
||||
dispatch_fn.assert_not_called()
|
||||
call_kwargs = storage.update_watch.call_args
|
||||
# Watch should remain active
|
||||
assert "active" not in call_kwargs[1] or call_kwargs[1].get("active") is not False
|
||||
|
||||
def test_max_polls_deactivates(self):
|
||||
storage = MagicMock()
|
||||
storage.update_watch.return_value = True
|
||||
runner = self._make_runner(storage=storage)
|
||||
dispatch_fn = MagicMock()
|
||||
runner.set_dispatch_fn("ws-1", dispatch_fn)
|
||||
|
||||
watch_row = {
|
||||
"watch_id": "abc123",
|
||||
"ws_id": "ws-1",
|
||||
"name": "test-watch",
|
||||
"command": "echo hello",
|
||||
"stop_on": '"never" in output', # won't fire
|
||||
"max_polls": 5,
|
||||
"poll_count": 4, # next is #5 = max
|
||||
"last_output": "hello\n",
|
||||
"interval_secs": 60,
|
||||
"created": datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%S"),
|
||||
}
|
||||
runner._poll_watch(watch_row)
|
||||
|
||||
call_kwargs = storage.update_watch.call_args
|
||||
assert call_kwargs[1]["active"] is False
|
||||
assert call_kwargs[1]["poll_count"] == 5
|
||||
dispatch_fn.assert_called_once()
|
||||
|
||||
def test_blocked_command_deactivates(self):
|
||||
storage = MagicMock()
|
||||
storage.update_watch.return_value = True
|
||||
runner = self._make_runner(storage=storage)
|
||||
|
||||
watch_row = {
|
||||
"watch_id": "abc123",
|
||||
"ws_id": "ws-1",
|
||||
"name": "test-watch",
|
||||
"command": "rm -rf /",
|
||||
"stop_on": None,
|
||||
"max_polls": 100,
|
||||
"poll_count": 0,
|
||||
"last_output": None,
|
||||
"interval_secs": 60,
|
||||
"created": datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%S"),
|
||||
}
|
||||
runner._poll_watch(watch_row)
|
||||
|
||||
storage.update_watch.assert_called_once()
|
||||
call_kwargs = storage.update_watch.call_args
|
||||
assert call_kwargs[0][0] == "abc123"
|
||||
assert call_kwargs[1]["active"] is False
|
||||
|
||||
def test_dispatch_fn_registry(self):
|
||||
runner = self._make_runner()
|
||||
fn1 = MagicMock()
|
||||
fn2 = MagicMock()
|
||||
|
||||
runner.set_dispatch_fn("ws-1", fn1)
|
||||
runner.set_dispatch_fn("ws-2", fn2)
|
||||
|
||||
runner._dispatch_result("ws-1", "msg1")
|
||||
fn1.assert_called_once_with("msg1")
|
||||
fn2.assert_not_called()
|
||||
|
||||
runner.remove_dispatch_fn("ws-1")
|
||||
# After removal, dispatch should try restore_fn
|
||||
runner._dispatch_result("ws-1", "msg2")
|
||||
fn1.assert_called_once() # still just the one call
|
||||
|
||||
def test_restore_fn_called_for_evicted(self):
|
||||
restored_fn = MagicMock()
|
||||
restore_fn = MagicMock(return_value=restored_fn)
|
||||
runner = self._make_runner(restore_fn=restore_fn)
|
||||
|
||||
runner._dispatch_result("ws-evicted", "hello")
|
||||
restore_fn.assert_called_once_with("ws-evicted")
|
||||
restored_fn.assert_called_once_with("hello")
|
||||
|
||||
def test_run_command_success(self):
|
||||
runner = self._make_runner()
|
||||
output, code = runner._run_command("echo hello")
|
||||
assert "hello" in output
|
||||
assert code == 0
|
||||
|
||||
def test_run_command_failure(self):
|
||||
runner = self._make_runner()
|
||||
output, code = runner._run_command("exit 42")
|
||||
assert code == 42
|
||||
|
||||
def test_run_command_timeout(self):
|
||||
runner = self._make_runner()
|
||||
runner._tool_timeout = 1
|
||||
output, code = runner._run_command("sleep 30")
|
||||
assert "timed out" in output.lower()
|
||||
assert code == -1
|
||||
@@ -0,0 +1,130 @@
|
||||
"""Tests for watches storage CRUD."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import pytest
|
||||
|
||||
from turnstone.core.storage._sqlite import SQLiteBackend
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def db(tmp_path):
|
||||
"""Fresh SQLite backend for each test."""
|
||||
return SQLiteBackend(str(tmp_path / "test.db"))
|
||||
|
||||
|
||||
def _make_watch_kwargs(**overrides):
|
||||
"""Build default kwargs for create_watch."""
|
||||
defaults = {
|
||||
"watch_id": "watch_001",
|
||||
"ws_id": "ws-abc",
|
||||
"node_id": "node-1",
|
||||
"name": "pr-review",
|
||||
"command": "gh pr view --json state",
|
||||
"interval_secs": 300.0,
|
||||
"stop_on": 'data["state"] == "MERGED"',
|
||||
"max_polls": 100,
|
||||
"created_by": "model",
|
||||
"next_poll": "2099-01-01T00:05:00",
|
||||
}
|
||||
defaults.update(overrides)
|
||||
return defaults
|
||||
|
||||
|
||||
class TestWatchCRUD:
|
||||
def test_create_and_get(self, db):
|
||||
db.create_watch(**_make_watch_kwargs())
|
||||
w = db.get_watch("watch_001")
|
||||
assert w is not None
|
||||
assert w["name"] == "pr-review"
|
||||
assert w["command"] == "gh pr view --json state"
|
||||
assert w["interval_secs"] == 300.0
|
||||
assert w["active"] == 1
|
||||
assert w["poll_count"] == 0
|
||||
|
||||
def test_get_nonexistent(self, db):
|
||||
assert db.get_watch("nope") is None
|
||||
|
||||
def test_create_idempotent(self, db):
|
||||
db.create_watch(**_make_watch_kwargs())
|
||||
db.create_watch(**_make_watch_kwargs()) # OR IGNORE
|
||||
assert db.get_watch("watch_001") is not None
|
||||
|
||||
def test_update(self, db):
|
||||
db.create_watch(**_make_watch_kwargs())
|
||||
updated = db.update_watch(
|
||||
"watch_001",
|
||||
poll_count=5,
|
||||
last_output="hello",
|
||||
last_exit_code=0,
|
||||
)
|
||||
assert updated is True
|
||||
w = db.get_watch("watch_001")
|
||||
assert w["poll_count"] == 5
|
||||
assert w["last_output"] == "hello"
|
||||
assert w["last_exit_code"] == 0
|
||||
|
||||
def test_update_nonexistent(self, db):
|
||||
assert db.update_watch("nope", poll_count=1) is False
|
||||
|
||||
def test_update_active_flag(self, db):
|
||||
db.create_watch(**_make_watch_kwargs())
|
||||
db.update_watch("watch_001", active=False)
|
||||
w = db.get_watch("watch_001")
|
||||
assert w["active"] == 0
|
||||
|
||||
def test_delete(self, db):
|
||||
db.create_watch(**_make_watch_kwargs())
|
||||
assert db.delete_watch("watch_001") is True
|
||||
assert db.get_watch("watch_001") is None
|
||||
|
||||
def test_delete_nonexistent(self, db):
|
||||
assert db.delete_watch("nope") is False
|
||||
|
||||
|
||||
class TestWatchListQueries:
|
||||
def test_list_for_ws(self, db):
|
||||
db.create_watch(**_make_watch_kwargs(watch_id="w1", ws_id="ws-1", name="a"))
|
||||
db.create_watch(**_make_watch_kwargs(watch_id="w2", ws_id="ws-1", name="b"))
|
||||
db.create_watch(**_make_watch_kwargs(watch_id="w3", ws_id="ws-2", name="c"))
|
||||
|
||||
ws1 = db.list_watches_for_ws("ws-1")
|
||||
assert len(ws1) == 2
|
||||
assert {w["name"] for w in ws1} == {"a", "b"}
|
||||
|
||||
def test_list_for_ws_excludes_inactive(self, db):
|
||||
db.create_watch(**_make_watch_kwargs(watch_id="w1", ws_id="ws-1"))
|
||||
db.update_watch("w1", active=False)
|
||||
assert db.list_watches_for_ws("ws-1") == []
|
||||
|
||||
def test_list_for_node(self, db):
|
||||
db.create_watch(**_make_watch_kwargs(watch_id="w1", node_id="n1"))
|
||||
db.create_watch(**_make_watch_kwargs(watch_id="w2", node_id="n1"))
|
||||
db.create_watch(**_make_watch_kwargs(watch_id="w3", node_id="n2"))
|
||||
|
||||
n1 = db.list_watches_for_node("n1")
|
||||
assert len(n1) == 2
|
||||
|
||||
def test_list_due(self, db):
|
||||
# Due
|
||||
db.create_watch(**_make_watch_kwargs(watch_id="w1", next_poll="2020-01-01T00:00:00"))
|
||||
# Not due (far future)
|
||||
db.create_watch(**_make_watch_kwargs(watch_id="w2", next_poll="2099-01-01T00:00:00"))
|
||||
# Due but inactive
|
||||
db.create_watch(**_make_watch_kwargs(watch_id="w3", next_poll="2020-01-01T00:00:00"))
|
||||
db.update_watch("w3", active=False)
|
||||
|
||||
due = db.list_due_watches("2025-01-01T00:00:00")
|
||||
assert len(due) == 1
|
||||
assert due[0]["watch_id"] == "w1"
|
||||
|
||||
def test_delete_for_ws(self, db):
|
||||
db.create_watch(**_make_watch_kwargs(watch_id="w1", ws_id="ws-1"))
|
||||
db.create_watch(**_make_watch_kwargs(watch_id="w2", ws_id="ws-1"))
|
||||
db.create_watch(**_make_watch_kwargs(watch_id="w3", ws_id="ws-2"))
|
||||
|
||||
count = db.delete_watches_for_ws("ws-1")
|
||||
assert count == 2
|
||||
assert db.get_watch("w1") is None
|
||||
assert db.get_watch("w2") is None
|
||||
assert db.get_watch("w3") is not None
|
||||
@@ -683,6 +683,117 @@ class TestWebUI:
|
||||
t.join()
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# WebUI SSE fan-out
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestWebUIFanOut:
|
||||
"""Verify per-client SSE fan-out on WebUI._enqueue / _register_listener."""
|
||||
|
||||
def test_enqueue_no_listeners(self):
|
||||
"""Events silently dropped when no listeners are registered."""
|
||||
from turnstone.server import WebUI
|
||||
|
||||
ui = WebUI(ws_id="test")
|
||||
ui._enqueue({"type": "content", "text": "hello"}) # should not raise
|
||||
|
||||
def test_enqueue_single_listener(self):
|
||||
"""Single listener receives the event."""
|
||||
from turnstone.server import WebUI
|
||||
|
||||
ui = WebUI(ws_id="test")
|
||||
q = ui._register_listener()
|
||||
ui._enqueue({"type": "content", "text": "hello"})
|
||||
assert q.get_nowait() == {"type": "content", "text": "hello"}
|
||||
|
||||
def test_enqueue_multiple_listeners(self):
|
||||
"""All registered listeners receive an identical copy."""
|
||||
from turnstone.server import WebUI
|
||||
|
||||
ui = WebUI(ws_id="test")
|
||||
q1 = ui._register_listener()
|
||||
q2 = ui._register_listener()
|
||||
q3 = ui._register_listener()
|
||||
|
||||
event = {"type": "content", "text": "world"}
|
||||
ui._enqueue(event)
|
||||
|
||||
assert q1.get_nowait() == event
|
||||
assert q2.get_nowait() == event
|
||||
assert q3.get_nowait() == event
|
||||
|
||||
def test_unregister_stops_delivery(self):
|
||||
"""After unregister, the queue receives no further events."""
|
||||
import queue as queue_mod
|
||||
|
||||
from turnstone.server import WebUI
|
||||
|
||||
ui = WebUI(ws_id="test")
|
||||
q = ui._register_listener()
|
||||
ui._unregister_listener(q)
|
||||
ui._enqueue({"type": "content", "text": "gone"})
|
||||
|
||||
with pytest.raises(queue_mod.Empty):
|
||||
q.get_nowait()
|
||||
|
||||
def test_slow_consumer_does_not_block(self):
|
||||
"""A full queue doesn't block the producer or starve other listeners."""
|
||||
from turnstone.server import WebUI
|
||||
|
||||
ui = WebUI(ws_id="test")
|
||||
slow = ui._register_listener()
|
||||
fast = ui._register_listener()
|
||||
|
||||
# Fill only the slow consumer's queue directly to capacity
|
||||
for i in range(500):
|
||||
slow.put_nowait({"type": "content", "text": f"fill-{i}"})
|
||||
|
||||
assert slow.qsize() == 500
|
||||
assert fast.qsize() == 0
|
||||
|
||||
# Enqueue via fan-out — slow drops (full), fast receives
|
||||
event = {"type": "content", "text": "overflow"}
|
||||
ui._enqueue(event)
|
||||
assert slow.qsize() == 500 # still full, overflow dropped
|
||||
assert fast.qsize() == 1
|
||||
assert fast.get_nowait() == event
|
||||
|
||||
def test_unregister_idempotent(self):
|
||||
"""Double unregister does not raise."""
|
||||
|
||||
from turnstone.server import WebUI
|
||||
|
||||
ui = WebUI(ws_id="test")
|
||||
q = ui._register_listener()
|
||||
ui._unregister_listener(q)
|
||||
ui._unregister_listener(q) # should not raise
|
||||
|
||||
def test_concurrent_enqueue_and_register(self):
|
||||
"""Concurrent register/unregister and enqueue should not crash."""
|
||||
from turnstone.server import WebUI
|
||||
|
||||
ui = WebUI(ws_id="test")
|
||||
stop = threading.Event()
|
||||
|
||||
def register_loop():
|
||||
while not stop.is_set():
|
||||
q = ui._register_listener()
|
||||
ui._unregister_listener(q)
|
||||
|
||||
def enqueue_loop():
|
||||
for i in range(500):
|
||||
ui._enqueue({"type": "content", "text": f"tok-{i}"})
|
||||
|
||||
t1 = threading.Thread(target=register_loop)
|
||||
t2 = threading.Thread(target=enqueue_loop)
|
||||
t1.start()
|
||||
t2.start()
|
||||
t2.join()
|
||||
stop.set()
|
||||
t1.join()
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Integration: WorkstreamManager + session state transitions
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
@@ -1,3 +1,3 @@
|
||||
"""turnstone - Multi-node AI orchestration platform with tool use, agent routing, and cluster simulation."""
|
||||
|
||||
__version__ = "0.4.2"
|
||||
__version__ = "0.5.4"
|
||||
|
||||
@@ -2,6 +2,8 @@
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Any
|
||||
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
@@ -48,7 +50,7 @@ class ClusterNodeInfo(BaseModel):
|
||||
total_tokens: int = 0
|
||||
started: float = 0.0
|
||||
reachable: bool = True
|
||||
health: dict[str, str] = Field(default_factory=dict)
|
||||
health: dict[str, Any] = Field(default_factory=dict)
|
||||
version: str = ""
|
||||
|
||||
|
||||
@@ -91,12 +93,34 @@ class ClusterWorkstreamsResponse(BaseModel):
|
||||
class NodeDetailResponse(BaseModel):
|
||||
node_id: str
|
||||
server_url: str = ""
|
||||
health: dict[str, str] = Field(default_factory=dict)
|
||||
health: dict[str, Any] = Field(default_factory=dict)
|
||||
workstreams: list[ClusterWorkstreamInfo] = []
|
||||
aggregate: dict[str, int] = Field(default_factory=dict)
|
||||
reachable: bool = True
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Cluster snapshot
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class ClusterSnapshotNode(BaseModel):
|
||||
node_id: str
|
||||
server_url: str = ""
|
||||
max_ws: int = 10
|
||||
reachable: bool = True
|
||||
version: str = ""
|
||||
health: dict[str, Any] = Field(default_factory=dict)
|
||||
aggregate: dict[str, int] = Field(default_factory=dict)
|
||||
workstreams: list[ClusterWorkstreamInfo] = []
|
||||
|
||||
|
||||
class ClusterSnapshotResponse(BaseModel):
|
||||
nodes: list[ClusterSnapshotNode]
|
||||
overview: ClusterOverviewResponse
|
||||
timestamp: float = 0.0
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Workstream creation
|
||||
# ---------------------------------------------------------------------------
|
||||
@@ -132,3 +156,195 @@ class ConsoleHealthResponse(BaseModel):
|
||||
workstreams: int = 0
|
||||
version_drift: bool = False
|
||||
versions: list[str] = []
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Governance: Roles
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class RoleInfo(BaseModel):
|
||||
role_id: str
|
||||
name: str
|
||||
display_name: str
|
||||
permissions: str
|
||||
builtin: bool
|
||||
org_id: str
|
||||
created: str
|
||||
updated: str
|
||||
|
||||
|
||||
class CreateRoleRequest(BaseModel):
|
||||
name: str
|
||||
display_name: str = ""
|
||||
permissions: str = "read"
|
||||
|
||||
|
||||
class UpdateRoleRequest(BaseModel):
|
||||
display_name: str | None = None
|
||||
permissions: str | None = None
|
||||
|
||||
|
||||
class ListRolesResponse(BaseModel):
|
||||
roles: list[RoleInfo]
|
||||
|
||||
|
||||
class AssignRoleRequest(BaseModel):
|
||||
role_id: str
|
||||
|
||||
|
||||
class UserRoleInfo(BaseModel):
|
||||
role_id: str
|
||||
name: str
|
||||
display_name: str
|
||||
permissions: str
|
||||
builtin: bool
|
||||
org_id: str
|
||||
created: str
|
||||
updated: str
|
||||
assigned_by: str
|
||||
assignment_created: str
|
||||
|
||||
|
||||
class ListUserRolesResponse(BaseModel):
|
||||
roles: list[UserRoleInfo]
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Governance: Orgs
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class OrgInfo(BaseModel):
|
||||
org_id: str
|
||||
name: str
|
||||
display_name: str
|
||||
settings: str
|
||||
created: str
|
||||
updated: str
|
||||
|
||||
|
||||
class UpdateOrgRequest(BaseModel):
|
||||
display_name: str | None = None
|
||||
settings: str | None = None
|
||||
|
||||
|
||||
class ListOrgsResponse(BaseModel):
|
||||
orgs: list[OrgInfo]
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Governance: Tool Policies
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class ToolPolicyInfo(BaseModel):
|
||||
policy_id: str
|
||||
name: str
|
||||
tool_pattern: str
|
||||
action: str
|
||||
priority: int
|
||||
org_id: str
|
||||
enabled: bool
|
||||
created_by: str
|
||||
created: str
|
||||
updated: str
|
||||
|
||||
|
||||
class CreateToolPolicyRequest(BaseModel):
|
||||
name: str
|
||||
tool_pattern: str
|
||||
action: str # allow, deny, ask
|
||||
priority: int = 0
|
||||
org_id: str = ""
|
||||
enabled: bool = True
|
||||
|
||||
|
||||
class UpdateToolPolicyRequest(BaseModel):
|
||||
name: str | None = None
|
||||
tool_pattern: str | None = None
|
||||
action: str | None = None
|
||||
priority: int | None = None
|
||||
enabled: bool | None = None
|
||||
|
||||
|
||||
class ListToolPoliciesResponse(BaseModel):
|
||||
policies: list[ToolPolicyInfo]
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Governance: Prompt Templates
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class PromptTemplateInfo(BaseModel):
|
||||
template_id: str
|
||||
name: str
|
||||
category: str
|
||||
content: str
|
||||
variables: str
|
||||
is_default: bool
|
||||
org_id: str
|
||||
created_by: str
|
||||
created: str
|
||||
updated: str
|
||||
|
||||
|
||||
class CreatePromptTemplateRequest(BaseModel):
|
||||
name: str
|
||||
content: str
|
||||
category: str = "general"
|
||||
variables: str = "[]"
|
||||
is_default: bool = False
|
||||
org_id: str = ""
|
||||
|
||||
|
||||
class UpdatePromptTemplateRequest(BaseModel):
|
||||
name: str | None = None
|
||||
content: str | None = None
|
||||
category: str | None = None
|
||||
variables: str | None = None
|
||||
is_default: bool | None = None
|
||||
|
||||
|
||||
class ListPromptTemplatesResponse(BaseModel):
|
||||
templates: list[PromptTemplateInfo]
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Governance: Usage
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class UsageBreakdownItem(BaseModel):
|
||||
key: str = ""
|
||||
prompt_tokens: int = 0
|
||||
completion_tokens: int = 0
|
||||
tool_calls_count: int = 0
|
||||
|
||||
|
||||
class UsageResponse(BaseModel):
|
||||
summary: list[UsageBreakdownItem]
|
||||
breakdown: list[UsageBreakdownItem]
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Governance: Audit
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class AuditEventInfo(BaseModel):
|
||||
event_id: str
|
||||
timestamp: str
|
||||
user_id: str
|
||||
action: str
|
||||
resource_type: str
|
||||
resource_id: str
|
||||
detail: str
|
||||
ip_address: str
|
||||
created: str
|
||||
|
||||
|
||||
class ListAuditEventsResponse(BaseModel):
|
||||
events: list[AuditEventInfo]
|
||||
total: int
|
||||
|
||||
@@ -8,13 +8,36 @@ if TYPE_CHECKING:
|
||||
from pydantic import BaseModel
|
||||
|
||||
from turnstone.api.console_schemas import (
|
||||
AssignRoleRequest,
|
||||
AuditEventInfo,
|
||||
ClusterNodesResponse,
|
||||
ClusterOverviewResponse,
|
||||
ClusterSnapshotResponse,
|
||||
ClusterWorkstreamsResponse,
|
||||
ConsoleCreateWsRequest,
|
||||
ConsoleCreateWsResponse,
|
||||
ConsoleHealthResponse,
|
||||
CreatePromptTemplateRequest,
|
||||
CreateRoleRequest,
|
||||
CreateToolPolicyRequest,
|
||||
ListAuditEventsResponse,
|
||||
ListOrgsResponse,
|
||||
ListPromptTemplatesResponse,
|
||||
ListRolesResponse,
|
||||
ListToolPoliciesResponse,
|
||||
ListUserRolesResponse,
|
||||
NodeDetailResponse,
|
||||
OrgInfo,
|
||||
PromptTemplateInfo,
|
||||
RoleInfo,
|
||||
ToolPolicyInfo,
|
||||
UpdateOrgRequest,
|
||||
UpdatePromptTemplateRequest,
|
||||
UpdateRoleRequest,
|
||||
UpdateToolPolicyRequest,
|
||||
UsageBreakdownItem,
|
||||
UsageResponse,
|
||||
UserRoleInfo,
|
||||
)
|
||||
from turnstone.api.openapi import EndpointSpec, QueryParam, build_openapi
|
||||
from turnstone.api.schemas import (
|
||||
@@ -97,14 +120,23 @@ CONSOLE_ENDPOINTS: list[EndpointSpec] = [
|
||||
error_codes=[400, 404, 503],
|
||||
tags=["Cluster"],
|
||||
),
|
||||
EndpointSpec(
|
||||
"/v1/api/cluster/snapshot",
|
||||
"GET",
|
||||
"Full cluster state snapshot",
|
||||
description="Returns the complete cluster state: all nodes with their workstreams "
|
||||
"and overview aggregates. Used for initial load and reconnection.",
|
||||
response_model=ClusterSnapshotResponse,
|
||||
tags=["Cluster"],
|
||||
),
|
||||
# --- Streaming ---
|
||||
EndpointSpec(
|
||||
"/v1/api/cluster/events",
|
||||
"GET",
|
||||
"Cluster SSE event stream",
|
||||
description="Server-Sent Events stream for real-time cluster updates. "
|
||||
"Returns text/event-stream with node_joined, node_lost, cluster_state, "
|
||||
"ws_created, ws_closed, ws_rename events.",
|
||||
"First event is a 'snapshot' with full cluster state, followed by "
|
||||
"node_joined, node_lost, cluster_state, ws_created, ws_closed, ws_rename events.",
|
||||
tags=["Streaming"],
|
||||
),
|
||||
# --- Auth ---
|
||||
@@ -242,6 +274,191 @@ CONSOLE_ENDPOINTS: list[EndpointSpec] = [
|
||||
error_codes=[404],
|
||||
tags=["Schedules"],
|
||||
),
|
||||
# --- Governance: Roles ---
|
||||
EndpointSpec(
|
||||
"/v1/api/admin/roles",
|
||||
"GET",
|
||||
"List all roles",
|
||||
response_model=ListRolesResponse,
|
||||
tags=["Admin"],
|
||||
),
|
||||
EndpointSpec(
|
||||
"/v1/api/admin/roles",
|
||||
"POST",
|
||||
"Create a custom role",
|
||||
request_model=CreateRoleRequest,
|
||||
response_model=RoleInfo,
|
||||
error_codes=[400],
|
||||
tags=["Admin"],
|
||||
),
|
||||
EndpointSpec(
|
||||
"/v1/api/admin/roles/{role_id}",
|
||||
"PUT",
|
||||
"Update a role",
|
||||
request_model=UpdateRoleRequest,
|
||||
response_model=RoleInfo,
|
||||
error_codes=[400, 404],
|
||||
tags=["Admin"],
|
||||
),
|
||||
EndpointSpec(
|
||||
"/v1/api/admin/roles/{role_id}",
|
||||
"DELETE",
|
||||
"Delete a custom role",
|
||||
response_model=StatusResponse,
|
||||
error_codes=[400, 404],
|
||||
tags=["Admin"],
|
||||
),
|
||||
EndpointSpec(
|
||||
"/v1/api/admin/users/{user_id}/roles",
|
||||
"GET",
|
||||
"List roles assigned to a user",
|
||||
response_model=ListUserRolesResponse,
|
||||
tags=["Admin"],
|
||||
),
|
||||
EndpointSpec(
|
||||
"/v1/api/admin/users/{user_id}/roles",
|
||||
"POST",
|
||||
"Assign a role to a user",
|
||||
request_model=AssignRoleRequest,
|
||||
response_model=StatusResponse,
|
||||
error_codes=[400, 404],
|
||||
tags=["Admin"],
|
||||
),
|
||||
EndpointSpec(
|
||||
"/v1/api/admin/users/{user_id}/roles/{role_id}",
|
||||
"DELETE",
|
||||
"Unassign a role from a user",
|
||||
response_model=StatusResponse,
|
||||
error_codes=[404],
|
||||
tags=["Admin"],
|
||||
),
|
||||
# --- Governance: Orgs ---
|
||||
EndpointSpec(
|
||||
"/v1/api/admin/orgs",
|
||||
"GET",
|
||||
"List organizations",
|
||||
response_model=ListOrgsResponse,
|
||||
tags=["Admin"],
|
||||
),
|
||||
EndpointSpec(
|
||||
"/v1/api/admin/orgs/{org_id}",
|
||||
"GET",
|
||||
"Get organization details",
|
||||
response_model=OrgInfo,
|
||||
error_codes=[404],
|
||||
tags=["Admin"],
|
||||
),
|
||||
EndpointSpec(
|
||||
"/v1/api/admin/orgs/{org_id}",
|
||||
"PUT",
|
||||
"Update organization settings",
|
||||
request_model=UpdateOrgRequest,
|
||||
response_model=OrgInfo,
|
||||
error_codes=[404],
|
||||
tags=["Admin"],
|
||||
),
|
||||
# --- Governance: Tool Policies ---
|
||||
EndpointSpec(
|
||||
"/v1/api/admin/policies",
|
||||
"GET",
|
||||
"List tool policies",
|
||||
response_model=ListToolPoliciesResponse,
|
||||
tags=["Admin"],
|
||||
),
|
||||
EndpointSpec(
|
||||
"/v1/api/admin/policies",
|
||||
"POST",
|
||||
"Create a tool policy",
|
||||
request_model=CreateToolPolicyRequest,
|
||||
response_model=ToolPolicyInfo,
|
||||
error_codes=[400],
|
||||
tags=["Admin"],
|
||||
),
|
||||
EndpointSpec(
|
||||
"/v1/api/admin/policies/{policy_id}",
|
||||
"PUT",
|
||||
"Update a tool policy",
|
||||
request_model=UpdateToolPolicyRequest,
|
||||
response_model=ToolPolicyInfo,
|
||||
error_codes=[404],
|
||||
tags=["Admin"],
|
||||
),
|
||||
EndpointSpec(
|
||||
"/v1/api/admin/policies/{policy_id}",
|
||||
"DELETE",
|
||||
"Delete a tool policy",
|
||||
response_model=StatusResponse,
|
||||
error_codes=[404],
|
||||
tags=["Admin"],
|
||||
),
|
||||
# --- Governance: Prompt Templates ---
|
||||
EndpointSpec(
|
||||
"/v1/api/admin/templates",
|
||||
"GET",
|
||||
"List prompt templates",
|
||||
response_model=ListPromptTemplatesResponse,
|
||||
tags=["Admin"],
|
||||
),
|
||||
EndpointSpec(
|
||||
"/v1/api/admin/templates",
|
||||
"POST",
|
||||
"Create a prompt template",
|
||||
request_model=CreatePromptTemplateRequest,
|
||||
response_model=PromptTemplateInfo,
|
||||
error_codes=[400],
|
||||
tags=["Admin"],
|
||||
),
|
||||
EndpointSpec(
|
||||
"/v1/api/admin/templates/{template_id}",
|
||||
"PUT",
|
||||
"Update a prompt template",
|
||||
request_model=UpdatePromptTemplateRequest,
|
||||
response_model=PromptTemplateInfo,
|
||||
error_codes=[404],
|
||||
tags=["Admin"],
|
||||
),
|
||||
EndpointSpec(
|
||||
"/v1/api/admin/templates/{template_id}",
|
||||
"DELETE",
|
||||
"Delete a prompt template",
|
||||
response_model=StatusResponse,
|
||||
error_codes=[404],
|
||||
tags=["Admin"],
|
||||
),
|
||||
# --- Governance: Usage & Audit ---
|
||||
EndpointSpec(
|
||||
"/v1/api/admin/usage",
|
||||
"GET",
|
||||
"Aggregated usage data",
|
||||
response_model=UsageResponse,
|
||||
query_params=[
|
||||
QueryParam("since", "Start timestamp (ISO8601, defaults to last 7 days)"),
|
||||
QueryParam("until", "End timestamp (ISO8601)"),
|
||||
QueryParam("user_id", "Filter by user"),
|
||||
QueryParam("model", "Filter by model"),
|
||||
QueryParam(
|
||||
"group_by",
|
||||
"Group results",
|
||||
enum=["day", "hour", "model", "user"],
|
||||
),
|
||||
],
|
||||
tags=["Admin"],
|
||||
),
|
||||
EndpointSpec(
|
||||
"/v1/api/admin/audit",
|
||||
"GET",
|
||||
"Paginated audit events",
|
||||
response_model=ListAuditEventsResponse,
|
||||
query_params=[
|
||||
QueryParam("action", "Filter by action type"),
|
||||
QueryParam("user_id", "Filter by user"),
|
||||
QueryParam("since", "Start timestamp (ISO8601)"),
|
||||
QueryParam("until", "End timestamp (ISO8601)"),
|
||||
QueryParam("limit", "Page size", schema_type="integer", default=50),
|
||||
QueryParam("offset", "Pagination offset", schema_type="integer", default=0),
|
||||
],
|
||||
tags=["Admin"],
|
||||
),
|
||||
# --- Observability ---
|
||||
EndpointSpec(
|
||||
"/health",
|
||||
@@ -270,6 +487,7 @@ _ALL_MODELS: list[type[BaseModel]] = [
|
||||
ClusterNodesResponse,
|
||||
ClusterWorkstreamsResponse,
|
||||
NodeDetailResponse,
|
||||
ClusterSnapshotResponse,
|
||||
ConsoleCreateWsRequest,
|
||||
ConsoleCreateWsResponse,
|
||||
ConsoleHealthResponse,
|
||||
@@ -278,6 +496,28 @@ _ALL_MODELS: list[type[BaseModel]] = [
|
||||
ScheduleInfo,
|
||||
ListSchedulesResponse,
|
||||
ListScheduleRunsResponse,
|
||||
RoleInfo,
|
||||
CreateRoleRequest,
|
||||
UpdateRoleRequest,
|
||||
ListRolesResponse,
|
||||
AssignRoleRequest,
|
||||
UserRoleInfo,
|
||||
ListUserRolesResponse,
|
||||
OrgInfo,
|
||||
UpdateOrgRequest,
|
||||
ListOrgsResponse,
|
||||
ToolPolicyInfo,
|
||||
CreateToolPolicyRequest,
|
||||
UpdateToolPolicyRequest,
|
||||
ListToolPoliciesResponse,
|
||||
PromptTemplateInfo,
|
||||
CreatePromptTemplateRequest,
|
||||
UpdatePromptTemplateRequest,
|
||||
ListPromptTemplatesResponse,
|
||||
UsageBreakdownItem,
|
||||
UsageResponse,
|
||||
AuditEventInfo,
|
||||
ListAuditEventsResponse,
|
||||
]
|
||||
|
||||
|
||||
|
||||
@@ -35,22 +35,27 @@ class CommandRequest(BaseModel):
|
||||
ws_id: str = Field(description="Target workstream ID")
|
||||
|
||||
|
||||
class CancelRequest(BaseModel):
|
||||
ws_id: str = Field(description="Target workstream ID")
|
||||
|
||||
|
||||
class CreateWorkstreamRequest(BaseModel):
|
||||
name: str = Field(default="", description="Workstream display name (auto-generated if empty)")
|
||||
model: str = Field(default="", description="Model alias from registry")
|
||||
auto_approve: bool = Field(default=False, description="Auto-approve all tool calls")
|
||||
resume_session: str = Field(
|
||||
resume_ws: str = Field(
|
||||
default="",
|
||||
description="Session ID to resume atomically during creation (empty = fresh start)",
|
||||
description="Workstream ID to resume atomically during creation (empty = fresh start)",
|
||||
)
|
||||
|
||||
|
||||
class CreateWorkstreamResponse(BaseModel):
|
||||
ws_id: str = Field(description="Unique ID of the new workstream")
|
||||
name: str = Field(description="Assigned workstream name")
|
||||
resumed: bool = Field(default=False, description="Whether a previous session was resumed")
|
||||
session_id: str = Field(default="", description="Resolved session ID (set when resumed)")
|
||||
message_count: int = Field(default=0, description="Number of messages in the resumed session")
|
||||
resumed: bool = Field(default=False, description="Whether a previous workstream was resumed")
|
||||
message_count: int = Field(
|
||||
default=0, description="Number of messages in the resumed workstream"
|
||||
)
|
||||
|
||||
|
||||
class CloseWorkstreamRequest(BaseModel):
|
||||
@@ -66,7 +71,6 @@ class WorkstreamInfo(BaseModel):
|
||||
id: str
|
||||
name: str
|
||||
state: str
|
||||
session_id: str | None = None
|
||||
|
||||
|
||||
class ListWorkstreamsResponse(BaseModel):
|
||||
@@ -77,7 +81,6 @@ class DashboardWorkstream(BaseModel):
|
||||
id: str
|
||||
name: str
|
||||
state: str
|
||||
session_id: str | None = None
|
||||
title: str = ""
|
||||
tokens: int = 0
|
||||
context_ratio: float = 0.0
|
||||
@@ -104,12 +107,12 @@ class DashboardResponse(BaseModel):
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Sessions
|
||||
# Saved workstreams
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class SessionInfo(BaseModel):
|
||||
session_id: str
|
||||
class SavedWorkstreamInfo(BaseModel):
|
||||
ws_id: str
|
||||
alias: str | None = None
|
||||
title: str | None = None
|
||||
created: str
|
||||
@@ -117,8 +120,8 @@ class SessionInfo(BaseModel):
|
||||
message_count: int
|
||||
|
||||
|
||||
class ListSessionsResponse(BaseModel):
|
||||
sessions: list[SessionInfo]
|
||||
class ListSavedWorkstreamsResponse(BaseModel):
|
||||
workstreams: list[SavedWorkstreamInfo]
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
@@ -19,13 +19,14 @@ from turnstone.api.schemas import (
|
||||
)
|
||||
from turnstone.api.server_schemas import (
|
||||
ApproveRequest,
|
||||
CancelRequest,
|
||||
CloseWorkstreamRequest,
|
||||
CommandRequest,
|
||||
CreateWorkstreamRequest,
|
||||
CreateWorkstreamResponse,
|
||||
DashboardResponse,
|
||||
HealthResponse,
|
||||
ListSessionsResponse,
|
||||
ListSavedWorkstreamsResponse,
|
||||
ListWorkstreamsResponse,
|
||||
PlanFeedbackRequest,
|
||||
SendRequest,
|
||||
@@ -103,6 +104,15 @@ SERVER_ENDPOINTS: list[EndpointSpec] = [
|
||||
error_codes=[400, 404],
|
||||
tags=["Chat"],
|
||||
),
|
||||
EndpointSpec(
|
||||
"/v1/api/cancel",
|
||||
"POST",
|
||||
"Cancel the active generation in a workstream",
|
||||
request_model=CancelRequest,
|
||||
response_model=StatusResponse,
|
||||
error_codes=[400, 404],
|
||||
tags=["Chat"],
|
||||
),
|
||||
# --- Streaming ---
|
||||
EndpointSpec(
|
||||
"/v1/api/events",
|
||||
@@ -122,13 +132,13 @@ SERVER_ENDPOINTS: list[EndpointSpec] = [
|
||||
"across all workstreams. Returns text/event-stream.",
|
||||
tags=["Streaming"],
|
||||
),
|
||||
# --- Sessions ---
|
||||
# --- Saved workstreams ---
|
||||
EndpointSpec(
|
||||
"/v1/api/sessions",
|
||||
"/v1/api/workstreams/saved",
|
||||
"GET",
|
||||
"List saved sessions",
|
||||
response_model=ListSessionsResponse,
|
||||
tags=["Sessions"],
|
||||
"List saved workstreams",
|
||||
response_model=ListSavedWorkstreamsResponse,
|
||||
tags=["Workstreams"],
|
||||
),
|
||||
# --- Auth ---
|
||||
EndpointSpec(
|
||||
@@ -186,12 +196,13 @@ _ALL_MODELS: list[type[BaseModel]] = [
|
||||
ApproveRequest,
|
||||
PlanFeedbackRequest,
|
||||
CommandRequest,
|
||||
CancelRequest,
|
||||
CreateWorkstreamRequest,
|
||||
CreateWorkstreamResponse,
|
||||
CloseWorkstreamRequest,
|
||||
ListWorkstreamsResponse,
|
||||
DashboardResponse,
|
||||
ListSessionsResponse,
|
||||
ListSavedWorkstreamsResponse,
|
||||
HealthResponse,
|
||||
]
|
||||
|
||||
|
||||
@@ -150,7 +150,7 @@ class ChannelRouter:
|
||||
owner = await self._broker.get_ws_owner(route["ws_id"])
|
||||
if owner:
|
||||
return route["ws_id"], False
|
||||
# Workstream was evicted/closed — capture old ws_id for session
|
||||
# Workstream was evicted/closed — capture old ws_id for
|
||||
# resume, then remove the stale route.
|
||||
old_ws_id = route["ws_id"]
|
||||
await asyncio.to_thread(
|
||||
@@ -163,20 +163,13 @@ class ChannelRouter:
|
||||
channel_id=channel_id,
|
||||
)
|
||||
|
||||
# 2. Look up old session for atomic resume (if stale route).
|
||||
resume_session = ""
|
||||
if old_ws_id:
|
||||
old_sid: str | None = await asyncio.to_thread(
|
||||
self._storage.get_session_id_by_ws, old_ws_id
|
||||
)
|
||||
resume_session = old_sid or ""
|
||||
|
||||
# 3. Create via MQ with atomic resume.
|
||||
# 2. Create via MQ with atomic resume (reuse old ws_id directly).
|
||||
resume_ws = old_ws_id or ""
|
||||
msg = CreateWorkstreamMessage(
|
||||
name=name,
|
||||
model=model,
|
||||
initial_message="" if resume_session else initial_message,
|
||||
resume_session=resume_session,
|
||||
initial_message="" if resume_ws else initial_message,
|
||||
resume_ws=resume_ws,
|
||||
auto_approve=self._auto_approve,
|
||||
auto_approve_tools=list(self._auto_approve_tools),
|
||||
)
|
||||
@@ -190,7 +183,7 @@ class ChannelRouter:
|
||||
correlation_id=cid,
|
||||
channel_type=channel_type,
|
||||
channel_id=channel_id,
|
||||
resume_session=resume_session or None,
|
||||
resume_ws=resume_ws or None,
|
||||
)
|
||||
|
||||
try:
|
||||
|
||||
@@ -21,8 +21,8 @@ from turnstone.mq.protocol import (
|
||||
ErrorEvent,
|
||||
OutboundEvent,
|
||||
PlanReviewEvent,
|
||||
SessionResumedEvent,
|
||||
TurnCompleteEvent,
|
||||
WorkstreamResumedEvent,
|
||||
)
|
||||
|
||||
if TYPE_CHECKING:
|
||||
@@ -307,10 +307,10 @@ class TurnstoneBot:
|
||||
if sm is not None:
|
||||
await sm.finalize()
|
||||
|
||||
elif isinstance(event, SessionResumedEvent):
|
||||
name = event.name or "previous session"
|
||||
elif isinstance(event, WorkstreamResumedEvent):
|
||||
name = event.name or "previous workstream"
|
||||
count = event.message_count
|
||||
await thread.send(f"*Session resumed: {name} ({count} messages restored)*")
|
||||
await thread.send(f"*Resumed: {name} ({count} messages restored)*")
|
||||
|
||||
elif isinstance(event, ErrorEvent):
|
||||
safe_msg = event.message[:500] if event.message else "An error occurred"
|
||||
|
||||
+53
-16
@@ -41,7 +41,7 @@ SLASH_COMMANDS = [
|
||||
"/instructions",
|
||||
"/clear",
|
||||
"/new",
|
||||
"/sessions",
|
||||
"/workstreams",
|
||||
"/resume",
|
||||
"/name",
|
||||
"/delete",
|
||||
@@ -784,11 +784,29 @@ def main() -> None:
|
||||
default=0,
|
||||
help="Tool output truncation limit in chars, 0 for auto (50%% of context window) (default: 0)",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--tool-search",
|
||||
choices=["auto", "on", "off"],
|
||||
default="auto",
|
||||
help="Dynamic tool search: auto (enable when tool count exceeds threshold), on, off (default: auto)",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--tool-search-threshold",
|
||||
type=int,
|
||||
default=20,
|
||||
help="Min tools before tool search activates (default: 20)",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--tool-search-max-results",
|
||||
type=int,
|
||||
default=5,
|
||||
help="Max tools returned per tool search query (default: 5)",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--resume",
|
||||
default=None,
|
||||
metavar="SESSION",
|
||||
help="Resume a previous session by alias or session_id",
|
||||
metavar="WS",
|
||||
help="Resume a previous workstream by alias or ws_id",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--skip-permissions",
|
||||
@@ -801,11 +819,11 @@ def main() -> None:
|
||||
help="API key (default: $OPENAI_API_KEY, or 'dummy' for local servers)",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--session-retention-days",
|
||||
"--retention-days",
|
||||
type=int,
|
||||
default=90,
|
||||
metavar="DAYS",
|
||||
help="Delete unnamed sessions older than DAYS days on startup, 0 to disable (default: 90)",
|
||||
help="Delete unnamed workstreams older than DAYS days on startup, 0 to disable (default: 90)",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--console-url",
|
||||
@@ -823,6 +841,16 @@ def main() -> None:
|
||||
metavar="PATH",
|
||||
help="Path to MCP server config file (standard mcpServers JSON format)",
|
||||
)
|
||||
|
||||
from turnstone.core.config import nonneg_float
|
||||
|
||||
parser.add_argument(
|
||||
"--mcp-refresh-interval",
|
||||
type=nonneg_float,
|
||||
default=14400,
|
||||
metavar="SECONDS",
|
||||
help="Periodic MCP tool refresh interval for servers without push notifications (default: 14400 = 4h, 0 to disable)",
|
||||
)
|
||||
from turnstone.core.config import apply_config
|
||||
|
||||
apply_config(parser, ["api", "model", "session", "tools", "console", "auth", "mcp", "database"])
|
||||
@@ -845,10 +873,10 @@ def main() -> None:
|
||||
)
|
||||
init_storage(db_backend, path=db_path, url=db_url, pool_size=db_pool_size)
|
||||
|
||||
# Prune stale / empty sessions on startup
|
||||
from turnstone.core.memory import prune_sessions
|
||||
# Prune stale / empty workstreams on startup
|
||||
from turnstone.core.memory import prune_workstreams
|
||||
|
||||
prune_sessions(retention_days=args.session_retention_days, log_fn=print)
|
||||
prune_workstreams(retention_days=args.retention_days, log_fn=print)
|
||||
|
||||
# Set up readline
|
||||
setup_readline()
|
||||
@@ -892,9 +920,12 @@ def main() -> None:
|
||||
# Initialize MCP client (connects to configured MCP servers, if any)
|
||||
from turnstone.core.mcp_client import create_mcp_client
|
||||
|
||||
mcp_client = create_mcp_client(getattr(args, "mcp_config", None))
|
||||
mcp_client = create_mcp_client(
|
||||
getattr(args, "mcp_config", None),
|
||||
refresh_interval=getattr(args, "mcp_refresh_interval", 14400),
|
||||
)
|
||||
|
||||
# Session factory — captures shared config for creating workstream sessions
|
||||
# ChatSession factory — captures shared config for creating workstreams
|
||||
def session_factory(
|
||||
ui: SessionUI | None, model_alias: str | None = None, ws_id: str | None = None
|
||||
) -> ChatSession:
|
||||
@@ -917,6 +948,9 @@ def main() -> None:
|
||||
mcp_client=mcp_client,
|
||||
registry=registry,
|
||||
model_alias=model_alias or registry.default,
|
||||
tool_search=args.tool_search,
|
||||
tool_search_threshold=args.tool_search_threshold,
|
||||
tool_search_max_results=args.tool_search_max_results,
|
||||
)
|
||||
|
||||
# Create workstream manager and initial workstream
|
||||
@@ -929,19 +963,19 @@ def main() -> None:
|
||||
|
||||
# Handle --resume
|
||||
if args.resume:
|
||||
from turnstone.core.memory import resolve_session
|
||||
from turnstone.core.memory import resolve_workstream
|
||||
|
||||
target_id = resolve_session(args.resume)
|
||||
target_id = resolve_workstream(args.resume)
|
||||
if not target_id:
|
||||
print(red(f"Session not found: {args.resume}"))
|
||||
print(red(f"Workstream not found: {args.resume}"))
|
||||
sys.exit(1)
|
||||
if ws.session is None:
|
||||
print(red("No session available."))
|
||||
sys.exit(1)
|
||||
if not ws.session.resume_session(target_id):
|
||||
print(red(f"Session '{args.resume}' has no messages."))
|
||||
if not ws.session.resume(target_id):
|
||||
print(red(f"Workstream '{args.resume}' has no messages."))
|
||||
sys.exit(1)
|
||||
print(f"Resumed session {bold(target_id)} ({len(ws.session.messages)} messages)")
|
||||
print(f"Resumed workstream {bold(target_id)} ({len(ws.session.messages)} messages)")
|
||||
|
||||
# Background attention notification — write to stderr while user types
|
||||
def _bg_attention_notify(ws_id: str, state: WorkstreamState) -> None:
|
||||
@@ -1020,6 +1054,9 @@ def main() -> None:
|
||||
except Exception as e:
|
||||
print(f"\n{red(f'Error: {e}')}")
|
||||
|
||||
# Close active session (removes MCP listener) before shutting down MCP
|
||||
if active and active.session:
|
||||
active.session.close()
|
||||
if mcp_client:
|
||||
mcp_client.shutdown()
|
||||
registry.shutdown()
|
||||
|
||||
@@ -150,7 +150,7 @@ class ClusterCollector:
|
||||
"state": "idle",
|
||||
"node": node_id,
|
||||
"server_url": node.server_url,
|
||||
"title": "",
|
||||
"title": data.get("title", ""),
|
||||
"tokens": 0,
|
||||
"context_ratio": 0.0,
|
||||
"activity": "",
|
||||
@@ -273,6 +273,7 @@ class ClusterCollector:
|
||||
"""Apply polled data to the in-memory node snapshot."""
|
||||
ws_list = dashboard.get("workstreams", [])
|
||||
aggregate = dashboard.get("aggregate", {})
|
||||
pending_events: list[dict[str, Any]] = []
|
||||
with self._lock:
|
||||
node = self._nodes.get(node_id)
|
||||
if not node:
|
||||
@@ -281,12 +282,35 @@ class ClusterCollector:
|
||||
node.reachable = True
|
||||
node.health = health
|
||||
node.aggregate = aggregate
|
||||
# Replace workstreams entirely from the authoritative poll
|
||||
node.workstreams = {}
|
||||
# Build new workstream map
|
||||
old_ids = {k for k in node.workstreams if k}
|
||||
new_ws: dict[str, dict[str, Any]] = {}
|
||||
for ws in ws_list:
|
||||
ws_id = ws.get("id", "")
|
||||
if not ws_id:
|
||||
continue
|
||||
ws["node"] = node_id
|
||||
ws["server_url"] = node.server_url
|
||||
node.workstreams[ws.get("id", "")] = ws
|
||||
new_ws[ws_id] = ws
|
||||
new_ids = set(new_ws.keys())
|
||||
# Detect additions not yet known to SSE clients
|
||||
for ws_id in sorted(new_ids - old_ids):
|
||||
ws = new_ws[ws_id]
|
||||
pending_events.append(
|
||||
{
|
||||
"type": "ws_created",
|
||||
"ws_id": ws_id,
|
||||
"name": ws.get("name", ""),
|
||||
"node_id": node_id,
|
||||
}
|
||||
)
|
||||
# Detect removals
|
||||
for ws_id in sorted(old_ids - new_ids):
|
||||
pending_events.append({"type": "ws_closed", "ws_id": ws_id})
|
||||
node.workstreams = new_ws
|
||||
# Fan out diffs to SSE listeners outside the lock
|
||||
for event in pending_events:
|
||||
self._fanout(event)
|
||||
|
||||
# -- query methods (thread-safe) -----------------------------------------
|
||||
|
||||
@@ -379,11 +403,11 @@ class ClusterCollector:
|
||||
)
|
||||
total = len(items)
|
||||
|
||||
# Sort
|
||||
# Sort (secondary key: node_id for stable ordering)
|
||||
if sort_by == "activity":
|
||||
items.sort(key=lambda n: n["ws_running"] + n["ws_attention"], reverse=True)
|
||||
items.sort(key=lambda n: (-(n["ws_running"] + n["ws_attention"]), n["node_id"]))
|
||||
elif sort_by == "tokens":
|
||||
items.sort(key=lambda n: n["total_tokens"], reverse=True)
|
||||
items.sort(key=lambda n: (-n["total_tokens"], n["node_id"]))
|
||||
elif sort_by == "name":
|
||||
items.sort(key=lambda n: n["node_id"])
|
||||
|
||||
@@ -455,6 +479,89 @@ class ClusterCollector:
|
||||
"reachable": node.reachable,
|
||||
}
|
||||
|
||||
def get_snapshot(self) -> dict[str, Any]:
|
||||
"""Build a complete cluster snapshot under a single lock.
|
||||
|
||||
Returns everything the UI needs to render the full dashboard:
|
||||
all nodes with their workstreams plus pre-computed overview aggregates.
|
||||
"""
|
||||
with self._lock:
|
||||
return self._build_snapshot_locked()
|
||||
|
||||
def get_snapshot_and_register(self, q: queue.Queue[dict[str, Any]]) -> dict[str, Any]:
|
||||
"""Build snapshot and register listener atomically.
|
||||
|
||||
Acquiring both locks ensures no event can be published between
|
||||
the snapshot read and the listener registration — the client
|
||||
receives the snapshot followed by every subsequent event with
|
||||
no gap.
|
||||
"""
|
||||
with self._lock:
|
||||
snap = self._build_snapshot_locked()
|
||||
with self._listeners_lock:
|
||||
self._listeners.append(q)
|
||||
return snap
|
||||
|
||||
def _build_snapshot_locked(self) -> dict[str, Any]:
|
||||
"""Build snapshot data — caller must hold ``_lock``."""
|
||||
nodes_out = []
|
||||
states: dict[str, int] = {
|
||||
"running": 0,
|
||||
"thinking": 0,
|
||||
"attention": 0,
|
||||
"idle": 0,
|
||||
"error": 0,
|
||||
}
|
||||
total_tokens = 0
|
||||
total_tool_calls = 0
|
||||
total_ws = 0
|
||||
versions: set[str] = set()
|
||||
|
||||
for node in self._nodes.values():
|
||||
ws_list = []
|
||||
for ws in node.workstreams.values():
|
||||
ws_list.append(dict(ws))
|
||||
s = ws.get("state", "idle")
|
||||
states[s] = states.get(s, 0) + 1
|
||||
total_ws += 1
|
||||
|
||||
total_tokens += node.aggregate.get("total_tokens", 0)
|
||||
total_tool_calls += node.aggregate.get("total_tool_calls", 0)
|
||||
ver = node.health.get("version", "")
|
||||
if ver:
|
||||
versions.add(ver)
|
||||
|
||||
nodes_out.append(
|
||||
{
|
||||
"node_id": node.node_id,
|
||||
"server_url": node.server_url,
|
||||
"max_ws": node.max_ws,
|
||||
"reachable": node.reachable,
|
||||
"version": ver,
|
||||
"health": dict(node.health),
|
||||
"aggregate": dict(node.aggregate),
|
||||
"workstreams": ws_list,
|
||||
}
|
||||
)
|
||||
|
||||
node_count = len(self._nodes)
|
||||
|
||||
return {
|
||||
"nodes": nodes_out,
|
||||
"overview": {
|
||||
"nodes": node_count,
|
||||
"workstreams": total_ws,
|
||||
"states": states,
|
||||
"aggregate": {
|
||||
"total_tokens": total_tokens,
|
||||
"total_tool_calls": total_tool_calls,
|
||||
},
|
||||
"version_drift": len(versions) > 1,
|
||||
"versions": sorted(versions),
|
||||
},
|
||||
"timestamp": time.time(),
|
||||
}
|
||||
|
||||
# -- SSE listener management ---------------------------------------------
|
||||
|
||||
def register_listener(self, q: queue.Queue[dict[str, Any]]) -> None:
|
||||
|
||||
@@ -112,6 +112,18 @@ class TaskScheduler:
|
||||
pruned = self._storage.prune_task_runs(retention_days=90)
|
||||
if pruned:
|
||||
log.info("scheduler.pruned_runs", count=pruned)
|
||||
try:
|
||||
usage_pruned = self._storage.prune_usage_events(retention_days=90)
|
||||
if usage_pruned:
|
||||
log.info("scheduler.pruned_usage", count=usage_pruned)
|
||||
except Exception:
|
||||
log.warning("scheduler.prune_usage_error", exc_info=True)
|
||||
try:
|
||||
audit_pruned = self._storage.prune_audit_events(retention_days=365)
|
||||
if audit_pruned:
|
||||
log.info("scheduler.pruned_audit", count=audit_pruned)
|
||||
except Exception:
|
||||
log.warning("scheduler.prune_audit_error", exc_info=True)
|
||||
finally:
|
||||
# Only release our own lock (safe even if TTL expired and another took it)
|
||||
self._broker._redis.eval( # type: ignore[no-untyped-call]
|
||||
|
||||
+1152
-18
File diff suppressed because it is too large
Load Diff
@@ -10,6 +10,7 @@ var _ctTrapHandler = null;
|
||||
var _tcTrapHandler = null;
|
||||
var _ccTrapHandler = null;
|
||||
var _cfTrapHandler = null;
|
||||
var _adminWatches = [];
|
||||
var _confirmCallbackFn = null;
|
||||
var _confirmTriggerEl = null;
|
||||
|
||||
@@ -28,11 +29,62 @@ function showAdmin() {
|
||||
document.getElementById("breadcrumb-label").textContent = "Admin";
|
||||
document.getElementById("main").scrollTop = 0;
|
||||
history.pushState({ view: "admin" }, "");
|
||||
loadAdminUsers();
|
||||
|
||||
// Permission gating: hide tabs the user cannot access
|
||||
var perms = sessionStorage.getItem("turnstone_permissions") || "";
|
||||
var tabPerms = {
|
||||
users: "admin.users",
|
||||
tokens: "admin.users",
|
||||
channels: "admin.users",
|
||||
schedules: "admin.schedules",
|
||||
watches: "admin.watches",
|
||||
roles: "admin.roles",
|
||||
policies: "admin.policies",
|
||||
templates: "admin.templates",
|
||||
usage: "admin.usage",
|
||||
audit: "admin.audit",
|
||||
};
|
||||
if (perms) {
|
||||
var permSet = perms.split(",");
|
||||
var tabs = document.querySelectorAll(".admin-tab");
|
||||
for (var i = 0; i < tabs.length; i++) {
|
||||
var tabName = tabs[i].getAttribute("data-tab");
|
||||
var needed = tabPerms[tabName];
|
||||
if (needed && permSet.indexOf(needed) < 0) {
|
||||
tabs[i].style.display = "none";
|
||||
} else {
|
||||
tabs[i].style.display = "";
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Switch to the first visible tab
|
||||
var visibleTabs = document.querySelectorAll(
|
||||
'.admin-tab:not([style*="display: none"])',
|
||||
);
|
||||
if (visibleTabs.length > 0) {
|
||||
switchAdminTab(visibleTabs[0].getAttribute("data-tab"));
|
||||
} else {
|
||||
// No tabs visible — show empty state instead of loading an inaccessible tab
|
||||
var panels = document.querySelectorAll(".admin-panel");
|
||||
for (var j = 0; j < panels.length; j++) panels[j].style.display = "none";
|
||||
var empty = document.getElementById("admin-no-permissions");
|
||||
if (!empty) {
|
||||
empty = document.createElement("div");
|
||||
empty.id = "admin-no-permissions";
|
||||
empty.className = "dashboard-empty";
|
||||
empty.textContent = "You do not have permissions to view any admin tabs.";
|
||||
document.getElementById("view-admin").appendChild(empty);
|
||||
}
|
||||
empty.style.display = "";
|
||||
}
|
||||
}
|
||||
|
||||
function switchAdminTab(tab) {
|
||||
_adminTab = tab;
|
||||
// Hide no-permissions empty state if it was showing
|
||||
var noPerms = document.getElementById("admin-no-permissions");
|
||||
if (noPerms) noPerms.style.display = "none";
|
||||
var tabs = document.querySelectorAll(".admin-tab");
|
||||
for (var i = 0; i < tabs.length; i++) {
|
||||
var isActive = tabs[i].getAttribute("data-tab") === tab;
|
||||
@@ -40,19 +92,36 @@ function switchAdminTab(tab) {
|
||||
tabs[i].setAttribute("aria-selected", isActive ? "true" : "false");
|
||||
tabs[i].setAttribute("tabindex", isActive ? "0" : "-1");
|
||||
}
|
||||
document.getElementById("admin-users").style.display =
|
||||
tab === "users" ? "" : "none";
|
||||
document.getElementById("admin-tokens").style.display =
|
||||
tab === "tokens" ? "" : "none";
|
||||
document.getElementById("admin-channels").style.display =
|
||||
tab === "channels" ? "" : "none";
|
||||
document.getElementById("admin-schedules").style.display =
|
||||
tab === "schedules" ? "" : "none";
|
||||
var panels = [
|
||||
"users",
|
||||
"tokens",
|
||||
"channels",
|
||||
"schedules",
|
||||
"watches",
|
||||
"roles",
|
||||
"policies",
|
||||
"templates",
|
||||
"usage",
|
||||
"audit",
|
||||
];
|
||||
for (var p = 0; p < panels.length; p++) {
|
||||
var el = document.getElementById("admin-" + panels[p]);
|
||||
if (el) el.style.display = panels[p] === tab ? "" : "none";
|
||||
}
|
||||
|
||||
if (tab === "users") loadAdminUsers();
|
||||
if (tab === "tokens") _populateTokenUserSelect();
|
||||
if (tab === "channels") _populateChannelUserSelect();
|
||||
if (tab === "schedules") loadAdminSchedules();
|
||||
if (tab === "watches") loadAdminWatches();
|
||||
if (tab === "roles") loadGovRoles();
|
||||
if (tab === "policies") loadGovPolicies();
|
||||
if (tab === "templates") loadGovTemplates();
|
||||
if (tab === "usage") loadGovUsage();
|
||||
if (tab === "audit") {
|
||||
_populateAuditUserFilter();
|
||||
loadGovAudit();
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
@@ -98,6 +167,9 @@ function _renderUsers(users) {
|
||||
escapeHtml(u.created || "").slice(0, 10) +
|
||||
"</span>" +
|
||||
'<span class="admin-col admin-col-actions">' +
|
||||
'<button class="admin-btn-action" data-user-roles="' +
|
||||
escapeHtml(u.user_id) +
|
||||
'" title="Manage roles">roles</button>' +
|
||||
'<button class="admin-btn-danger" data-delete-user="' +
|
||||
escapeHtml(u.user_id) +
|
||||
'" data-username="' +
|
||||
@@ -107,6 +179,13 @@ function _renderUsers(users) {
|
||||
"</div>";
|
||||
}
|
||||
container.innerHTML = html;
|
||||
// Bind roles buttons
|
||||
var roleBtns = container.querySelectorAll("[data-user-roles]");
|
||||
for (var rj = 0; rj < roleBtns.length; rj++) {
|
||||
roleBtns[rj].addEventListener("click", function () {
|
||||
showUserRolesModal(this.getAttribute("data-user-roles"));
|
||||
});
|
||||
}
|
||||
// Bind delete buttons via delegation (avoids inline JS injection)
|
||||
var btns = container.querySelectorAll("[data-delete-user]");
|
||||
for (var j = 0; j < btns.length; j++) {
|
||||
@@ -887,6 +966,167 @@ function hideScheduleRunsModal() {
|
||||
_runsScheduleTriggerEl = null;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Watches
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
function _populateWatchNodeSelect() {
|
||||
var sel = document.getElementById("admin-watch-node");
|
||||
var current = sel.value;
|
||||
var seen = {};
|
||||
sel.innerHTML = '<option value="">All nodes</option>';
|
||||
for (var i = 0; i < _adminWatches.length; i++) {
|
||||
var nid = _adminWatches[i].node_id || "";
|
||||
if (nid && !seen[nid]) {
|
||||
seen[nid] = true;
|
||||
var opt = document.createElement("option");
|
||||
opt.value = nid;
|
||||
opt.textContent = nid;
|
||||
sel.appendChild(opt);
|
||||
}
|
||||
}
|
||||
if (current) sel.value = current;
|
||||
}
|
||||
|
||||
function loadAdminWatches() {
|
||||
authFetch("/v1/api/admin/watches")
|
||||
.then(function (r) {
|
||||
if (!r.ok) throw new Error("Failed to load watches");
|
||||
return r.json();
|
||||
})
|
||||
.then(function (data) {
|
||||
_adminWatches = data.watches || [];
|
||||
_populateWatchNodeSelect();
|
||||
var nodeFilter = document.getElementById("admin-watch-node").value;
|
||||
var filtered = _adminWatches;
|
||||
if (nodeFilter) {
|
||||
filtered = _adminWatches.filter(function (w) {
|
||||
return w.node_id === nodeFilter;
|
||||
});
|
||||
}
|
||||
_renderWatches(filtered);
|
||||
})
|
||||
.catch(function () {
|
||||
document.getElementById("admin-watches-table").innerHTML =
|
||||
'<div class="dashboard-empty">Failed to load watches</div>';
|
||||
});
|
||||
}
|
||||
|
||||
function _formatInterval(secs) {
|
||||
if (!secs || secs <= 0) return "\u2014";
|
||||
if (secs >= 3600) return Math.round(secs / 3600) + "h";
|
||||
if (secs >= 60) return Math.round(secs / 60) + "m";
|
||||
return secs + "s";
|
||||
}
|
||||
|
||||
function _renderWatches(watches) {
|
||||
var container = document.getElementById("admin-watches-table");
|
||||
if (!watches.length) {
|
||||
container.innerHTML =
|
||||
'<div class="dashboard-empty">No active watches. Watches are created when workstreams use the watch tool.</div>';
|
||||
return;
|
||||
}
|
||||
var html = "";
|
||||
for (var i = 0; i < watches.length; i++) {
|
||||
var w = watches[i];
|
||||
var name = w.name || w.watch_id || "\u2014";
|
||||
var nodeShort = (w.node_id || "").slice(0, 8);
|
||||
var cmd = w.command || "";
|
||||
var cmdTrunc = cmd.length > 40 ? cmd.slice(0, 40) + "\u2026" : cmd;
|
||||
var interval = _formatInterval(w.interval_secs);
|
||||
var pollMax = w.max_polls ? w.max_polls : "\u221e";
|
||||
var pollLabel = (w.poll_count || 0) + "/" + pollMax;
|
||||
var cond = w.stop_on || "on change";
|
||||
var condTrunc = cond.length > 30 ? cond.slice(0, 30) + "\u2026" : cond;
|
||||
var active = w.active;
|
||||
var statusCls = active ? "watch-active" : "watch-completed";
|
||||
var statusLabel = active ? "active" : "done";
|
||||
var statusDot = active ? "\u25cf " : "\u25cb ";
|
||||
var cancelBtn = active
|
||||
? '<button class="admin-btn-danger" data-cancel-watch="' +
|
||||
escapeHtml(w.watch_id) +
|
||||
'" data-watch-node="' +
|
||||
escapeHtml(w.node_id || "") +
|
||||
'" data-watch-name="' +
|
||||
escapeHtml(name) +
|
||||
'" title="Cancel watch">cancel</button>'
|
||||
: "";
|
||||
html +=
|
||||
'<div class="admin-row" role="listitem">' +
|
||||
'<span class="admin-col admin-col-wname">' +
|
||||
escapeHtml(name) +
|
||||
"</span>" +
|
||||
'<span class="admin-col admin-col-wnode" title="' +
|
||||
escapeHtml(w.node_id || "") +
|
||||
'"><code>' +
|
||||
escapeHtml(nodeShort) +
|
||||
"</code></span>" +
|
||||
'<span class="admin-col admin-col-wcmd" title="' +
|
||||
escapeHtml(cmd) +
|
||||
'"><code>' +
|
||||
escapeHtml(cmdTrunc) +
|
||||
"</code></span>" +
|
||||
'<span class="admin-col admin-col-winterval">' +
|
||||
escapeHtml(interval) +
|
||||
"</span>" +
|
||||
'<span class="admin-col admin-col-wpoll"><code>' +
|
||||
escapeHtml(pollLabel) +
|
||||
"</code></span>" +
|
||||
'<span class="admin-col admin-col-wcond" title="' +
|
||||
escapeHtml(cond) +
|
||||
'">' +
|
||||
escapeHtml(condTrunc) +
|
||||
"</span>" +
|
||||
'<span class="admin-col admin-col-wstatus"><span class="' +
|
||||
statusCls +
|
||||
'">' +
|
||||
statusDot +
|
||||
statusLabel +
|
||||
"</span></span>" +
|
||||
'<span class="admin-col admin-col-actions">' +
|
||||
cancelBtn +
|
||||
"</span></div>";
|
||||
}
|
||||
container.innerHTML = html;
|
||||
// Bind cancel buttons
|
||||
var btns = container.querySelectorAll("[data-cancel-watch]");
|
||||
for (var j = 0; j < btns.length; j++) {
|
||||
btns[j].addEventListener("click", function () {
|
||||
_cancelWatch(
|
||||
this.getAttribute("data-cancel-watch"),
|
||||
this.getAttribute("data-watch-node"),
|
||||
this.getAttribute("data-watch-name"),
|
||||
);
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
function _cancelWatch(watchId, nodeId, name) {
|
||||
showConfirmModal(
|
||||
"Cancel Watch",
|
||||
"Cancel watch \u2018" + name + "\u2019? This will stop future polling.",
|
||||
"Cancel watch",
|
||||
function () {
|
||||
authFetch(
|
||||
"/v1/api/admin/watches/" + encodeURIComponent(watchId) + "/cancel",
|
||||
{
|
||||
method: "POST",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify({ node_id: nodeId }),
|
||||
},
|
||||
)
|
||||
.then(function (r) {
|
||||
if (!r.ok) throw new Error("Cancel failed");
|
||||
showToast("Watch '" + name + "' cancelled");
|
||||
loadAdminWatches();
|
||||
})
|
||||
.catch(function () {
|
||||
showToast("Failed to cancel watch");
|
||||
});
|
||||
},
|
||||
);
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Create Channel Link Modal
|
||||
// ---------------------------------------------------------------------------
|
||||
@@ -1197,6 +1437,14 @@ function _installTrap(overlayId, boxId, trapRef) {
|
||||
else if (overlayId === "edit-schedule-overlay") hideEditScheduleModal();
|
||||
else if (overlayId === "schedule-runs-overlay") hideScheduleRunsModal();
|
||||
else if (overlayId === "confirm-overlay") hideConfirmModal();
|
||||
else if (overlayId === "create-role-overlay") hideCreateRoleModal();
|
||||
else if (overlayId === "edit-role-overlay") hideEditRoleModal();
|
||||
else if (overlayId === "user-roles-overlay") hideUserRolesModal();
|
||||
else if (overlayId === "create-policy-overlay") hideCreatePolicyModal();
|
||||
else if (overlayId === "edit-policy-overlay") hideEditPolicyModal();
|
||||
else if (overlayId === "create-template-overlay")
|
||||
hideCreateTemplateModal();
|
||||
else if (overlayId === "edit-template-overlay") hideEditTemplateModal();
|
||||
}
|
||||
};
|
||||
}
|
||||
@@ -1263,6 +1511,24 @@ document.addEventListener("keydown", function (e) {
|
||||
hideConfirmModal();
|
||||
return;
|
||||
}
|
||||
// Governance modals
|
||||
var govOverlays = [
|
||||
["create-role-overlay", hideCreateRoleModal],
|
||||
["edit-role-overlay", hideEditRoleModal],
|
||||
["user-roles-overlay", hideUserRolesModal],
|
||||
["create-policy-overlay", hideCreatePolicyModal],
|
||||
["edit-policy-overlay", hideEditPolicyModal],
|
||||
["create-template-overlay", hideCreateTemplateModal],
|
||||
["edit-template-overlay", hideEditTemplateModal],
|
||||
];
|
||||
for (var gi = 0; gi < govOverlays.length; gi++) {
|
||||
var govEl = document.getElementById(govOverlays[gi][0]);
|
||||
if (govEl && govEl.style.display !== "none") {
|
||||
e.preventDefault();
|
||||
govOverlays[gi][1]();
|
||||
return;
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
// Tab arrow key navigation
|
||||
@@ -1271,7 +1537,14 @@ document.addEventListener("keydown", function (e) {
|
||||
if (!tablist) return;
|
||||
tablist.addEventListener("keydown", function (e) {
|
||||
if (e.key !== "ArrowLeft" && e.key !== "ArrowRight") return;
|
||||
var tabOrder = ["users", "tokens", "channels", "schedules"];
|
||||
var allTabs = document.querySelectorAll(
|
||||
'.admin-tab:not([style*="display: none"])',
|
||||
);
|
||||
var tabOrder = [];
|
||||
for (var ti = 0; ti < allTabs.length; ti++) {
|
||||
tabOrder.push(allTabs[ti].getAttribute("data-tab"));
|
||||
}
|
||||
if (tabOrder.length === 0) return;
|
||||
var idx = tabOrder.indexOf(_adminTab);
|
||||
if (e.key === "ArrowRight") idx = (idx + 1) % tabOrder.length;
|
||||
else idx = (idx - 1 + tabOrder.length) % tabOrder.length;
|
||||
|
||||
+308
-116
@@ -1,9 +1,6 @@
|
||||
// --- Shared hooks ---
|
||||
window.onLoginSuccess = function () {
|
||||
connectSSE();
|
||||
if (currentView === "overview") loadOverview();
|
||||
else if (currentView === "node") drillDownToNode(currentNodeId);
|
||||
else if (currentView === "filtered") loadFilteredWorkstreams();
|
||||
};
|
||||
window.onLogout = function () {
|
||||
if (evtSource) {
|
||||
@@ -33,6 +30,8 @@ var _lastOverviewJson = "";
|
||||
var _lastNodesJson = "";
|
||||
var evtSource = null;
|
||||
var retryDelay = 1000;
|
||||
var clusterState = null;
|
||||
var _navigatingFromPopstate = false;
|
||||
|
||||
// --- Constants ---
|
||||
var STATE_DISPLAY = {
|
||||
@@ -44,6 +43,236 @@ var STATE_DISPLAY = {
|
||||
};
|
||||
var STATE_ORDER = ["running", "thinking", "attention", "error", "idle"];
|
||||
|
||||
// --- Cluster State Model ---
|
||||
function applySnapshot(data) {
|
||||
clusterState = {
|
||||
nodes: {},
|
||||
overview: data.overview || {},
|
||||
timestamp: data.timestamp || 0,
|
||||
};
|
||||
(data.nodes || []).forEach(function (n) {
|
||||
clusterState.nodes[n.node_id] = n;
|
||||
});
|
||||
renderFromState();
|
||||
}
|
||||
|
||||
function patchClusterState(data) {
|
||||
if (!clusterState) return;
|
||||
var t = data.type;
|
||||
if (t === "cluster_state") {
|
||||
var node = clusterState.nodes[data.node_id];
|
||||
if (node) {
|
||||
(node.workstreams || []).forEach(function (ws) {
|
||||
if (ws.id === data.ws_id) {
|
||||
if ("state" in data) ws.state = data.state;
|
||||
if ("tokens" in data) ws.tokens = data.tokens;
|
||||
if ("context_ratio" in data) ws.context_ratio = data.context_ratio;
|
||||
if ("activity" in data) ws.activity = data.activity;
|
||||
if ("activity_state" in data) ws.activity_state = data.activity_state;
|
||||
}
|
||||
});
|
||||
}
|
||||
} else if (t === "ws_created") {
|
||||
var targetNode = clusterState.nodes[data.node_id];
|
||||
if (targetNode) {
|
||||
targetNode.workstreams = targetNode.workstreams || [];
|
||||
targetNode.workstreams.push({
|
||||
id: data.ws_id,
|
||||
name: data.name || "",
|
||||
state: "idle",
|
||||
node: data.node_id,
|
||||
server_url: targetNode.server_url || "",
|
||||
title: data.title || "",
|
||||
tokens: 0,
|
||||
context_ratio: 0.0,
|
||||
activity: "",
|
||||
activity_state: "",
|
||||
tool_calls: 0,
|
||||
});
|
||||
}
|
||||
} else if (t === "ws_closed") {
|
||||
Object.keys(clusterState.nodes).forEach(function (nid) {
|
||||
var n = clusterState.nodes[nid];
|
||||
n.workstreams = (n.workstreams || []).filter(function (ws) {
|
||||
return ws.id !== data.ws_id;
|
||||
});
|
||||
});
|
||||
} else if (t === "ws_rename") {
|
||||
Object.keys(clusterState.nodes).forEach(function (nid) {
|
||||
(clusterState.nodes[nid].workstreams || []).forEach(function (ws) {
|
||||
if (ws.id === data.ws_id) ws.name = data.name || "";
|
||||
});
|
||||
});
|
||||
} else if (t === "node_joined") {
|
||||
if (!clusterState.nodes[data.node_id]) {
|
||||
clusterState.nodes[data.node_id] = {
|
||||
node_id: data.node_id,
|
||||
server_url: "",
|
||||
max_ws: 10,
|
||||
reachable: true,
|
||||
version: "",
|
||||
health: {},
|
||||
aggregate: {},
|
||||
workstreams: [],
|
||||
};
|
||||
}
|
||||
} else if (t === "node_lost") {
|
||||
delete clusterState.nodes[data.node_id];
|
||||
} else {
|
||||
return;
|
||||
}
|
||||
scheduleRender();
|
||||
}
|
||||
|
||||
var _renderTimer = null;
|
||||
function scheduleRender() {
|
||||
if (_renderTimer) return;
|
||||
_renderTimer = requestAnimationFrame(function () {
|
||||
_renderTimer = null;
|
||||
recomputeOverview();
|
||||
renderFromState();
|
||||
});
|
||||
}
|
||||
|
||||
function recomputeOverview() {
|
||||
if (!clusterState) return;
|
||||
var states = { running: 0, thinking: 0, attention: 0, idle: 0, error: 0 };
|
||||
var totalTokens = 0,
|
||||
totalToolCalls = 0,
|
||||
totalWs = 0;
|
||||
var versions = {};
|
||||
Object.keys(clusterState.nodes).forEach(function (nid) {
|
||||
var node = clusterState.nodes[nid];
|
||||
var nodeWsTokens = 0;
|
||||
(node.workstreams || []).forEach(function (ws) {
|
||||
var s = ws.state || "idle";
|
||||
states[s] = (states[s] || 0) + 1;
|
||||
totalWs++;
|
||||
nodeWsTokens += ws.tokens || 0;
|
||||
});
|
||||
var aggTokens = (node.aggregate || {}).total_tokens || 0;
|
||||
totalTokens += aggTokens || nodeWsTokens;
|
||||
totalToolCalls += (node.aggregate || {}).total_tool_calls || 0;
|
||||
if (node.version) versions[node.version] = true;
|
||||
});
|
||||
var versionList = Object.keys(versions).sort();
|
||||
clusterState.overview = {
|
||||
nodes: Object.keys(clusterState.nodes).length,
|
||||
workstreams: totalWs,
|
||||
states: states,
|
||||
aggregate: {
|
||||
total_tokens: totalTokens,
|
||||
total_tool_calls: totalToolCalls,
|
||||
},
|
||||
version_drift: versionList.length > 1,
|
||||
versions: versionList,
|
||||
};
|
||||
}
|
||||
|
||||
function buildNodeInfoFromSnapshot(node) {
|
||||
var states = { running: 0, thinking: 0, attention: 0, idle: 0, error: 0 };
|
||||
var ws = node.workstreams || [];
|
||||
ws.forEach(function (w) {
|
||||
var s = w.state || "idle";
|
||||
states[s] = (states[s] || 0) + 1;
|
||||
});
|
||||
var aggTokens = (node.aggregate || {}).total_tokens || 0;
|
||||
if (!aggTokens) {
|
||||
ws.forEach(function (w) {
|
||||
aggTokens += w.tokens || 0;
|
||||
});
|
||||
}
|
||||
return {
|
||||
node_id: node.node_id,
|
||||
server_url: node.server_url || "",
|
||||
ws_total: ws.length,
|
||||
ws_running: states.running,
|
||||
ws_thinking: states.thinking,
|
||||
ws_attention: states.attention,
|
||||
ws_idle: states.idle,
|
||||
ws_error: states.error,
|
||||
total_tokens: aggTokens,
|
||||
ws_tokens: aggTokens,
|
||||
max_ws: node.max_ws || 10,
|
||||
started: node.started || 0,
|
||||
reachable: node.reachable !== false,
|
||||
health: node.health || {},
|
||||
version: node.version || "",
|
||||
};
|
||||
}
|
||||
|
||||
function renderFromState() {
|
||||
if (!clusterState) return;
|
||||
renderStatusBar(clusterState.overview);
|
||||
if (currentView === "overview") {
|
||||
var nodesList = Object.keys(clusterState.nodes).map(function (nid) {
|
||||
return buildNodeInfoFromSnapshot(clusterState.nodes[nid]);
|
||||
});
|
||||
nodesList.sort(function (a, b) {
|
||||
var d = b.ws_running + b.ws_attention - (a.ws_running + a.ws_attention);
|
||||
return d !== 0 ? d : a.node_id.localeCompare(b.node_id);
|
||||
});
|
||||
renderNodeGroups(nodesList, nodesList.length);
|
||||
document.getElementById("cluster-summary").textContent =
|
||||
clusterState.overview.nodes +
|
||||
" nodes \u00b7 " +
|
||||
formatCount(clusterState.overview.workstreams) +
|
||||
" workstreams";
|
||||
} else if (currentView === "node" && currentNodeId) {
|
||||
var snapNode = clusterState.nodes[currentNodeId];
|
||||
if (snapNode) {
|
||||
var wsList = snapNode.workstreams || [];
|
||||
var active = wsList.filter(function (w) {
|
||||
return w.state !== "idle";
|
||||
}).length;
|
||||
document.getElementById("node-ws-summary").textContent =
|
||||
active + " active \u00b7 " + wsList.length + " total";
|
||||
renderWsTable(document.getElementById("node-ws-table"), wsList);
|
||||
}
|
||||
} else if (currentView === "filtered") {
|
||||
var allWs = [];
|
||||
Object.keys(clusterState.nodes).forEach(function (nid) {
|
||||
(clusterState.nodes[nid].workstreams || []).forEach(function (ws) {
|
||||
allWs.push(ws);
|
||||
});
|
||||
});
|
||||
if (currentFilter.state) {
|
||||
allWs = allWs.filter(function (ws) {
|
||||
return ws.state === currentFilter.state;
|
||||
});
|
||||
}
|
||||
if (currentFilter.node) {
|
||||
allWs = allWs.filter(function (ws) {
|
||||
return ws.node === currentFilter.node;
|
||||
});
|
||||
}
|
||||
var stateOrder = {
|
||||
running: 0,
|
||||
thinking: 1,
|
||||
attention: 2,
|
||||
error: 3,
|
||||
idle: 4,
|
||||
};
|
||||
allWs.sort(function (a, b) {
|
||||
return (stateOrder[a.state] || 9) - (stateOrder[b.state] || 9);
|
||||
});
|
||||
var total = allWs.length;
|
||||
var perPage = currentFilter.per_page || 50;
|
||||
var pages = Math.max(1, Math.ceil(total / perPage));
|
||||
var page = Math.min(currentFilter.page || 1, pages);
|
||||
var start = (page - 1) * perPage;
|
||||
var pageWs = allWs.slice(start, start + perPage);
|
||||
document.getElementById("filtered-summary").textContent =
|
||||
"Page " + page + " of " + pages + " (" + total + " total)";
|
||||
renderWsTable(document.getElementById("filtered-ws-table"), pageWs);
|
||||
renderPagination(
|
||||
document.getElementById("filtered-pagination"),
|
||||
page,
|
||||
pages,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
// --- SSE Connection ---
|
||||
function connectSSE() {
|
||||
if (evtSource) {
|
||||
@@ -94,19 +323,11 @@ function connectSSE() {
|
||||
};
|
||||
}
|
||||
|
||||
var _refreshTimer = null;
|
||||
function scheduleRefresh() {
|
||||
if (_refreshTimer) return;
|
||||
_refreshTimer = setTimeout(function () {
|
||||
_refreshTimer = null;
|
||||
if (currentView === "overview") loadOverview();
|
||||
else if (currentView === "node" && currentNodeId)
|
||||
loadNodeDetail(currentNodeId);
|
||||
else if (currentView === "filtered") loadFilteredWorkstreams();
|
||||
}, 250);
|
||||
}
|
||||
|
||||
function handleClusterEvent(data) {
|
||||
if (data.type === "snapshot") {
|
||||
applySnapshot(data);
|
||||
return;
|
||||
}
|
||||
if (
|
||||
data.type === "cluster_state" ||
|
||||
data.type === "ws_created" ||
|
||||
@@ -115,7 +336,7 @@ function handleClusterEvent(data) {
|
||||
data.type === "node_joined" ||
|
||||
data.type === "node_lost"
|
||||
) {
|
||||
scheduleRefresh();
|
||||
patchClusterState(data);
|
||||
}
|
||||
if (data.type === "ws_closed" && data.reason === "evicted") {
|
||||
showToast("Evicted" + (data.name ? ": " + data.name : "") + " (capacity)");
|
||||
@@ -135,28 +356,18 @@ function showOverview() {
|
||||
if (adminView) adminView.style.display = "none";
|
||||
document.getElementById("breadcrumb").style.display = "none";
|
||||
document.getElementById("main").scrollTop = 0;
|
||||
loadOverview();
|
||||
history.pushState({ view: "overview" }, "");
|
||||
if (clusterState) renderFromState();
|
||||
else loadOverview();
|
||||
if (!_navigatingFromPopstate) history.pushState({ view: "overview" }, "");
|
||||
}
|
||||
|
||||
function loadOverview() {
|
||||
var overviewP = authFetch("/v1/api/cluster/overview").then(function (r) {
|
||||
return r.json();
|
||||
});
|
||||
var nodesP = authFetch("/v1/api/cluster/nodes?sort=activity&limit=1000").then(
|
||||
function (r) {
|
||||
authFetch("/v1/api/cluster/snapshot")
|
||||
.then(function (r) {
|
||||
return r.json();
|
||||
},
|
||||
);
|
||||
Promise.all([overviewP, nodesP])
|
||||
.then(function (res) {
|
||||
renderStatusBar(res[0]);
|
||||
renderNodeGroups(res[1].nodes, res[1].total);
|
||||
document.getElementById("cluster-summary").textContent =
|
||||
res[0].nodes +
|
||||
" nodes \u00b7 " +
|
||||
formatCount(res[0].workstreams) +
|
||||
" workstreams";
|
||||
})
|
||||
.then(function (data) {
|
||||
applySnapshot(data);
|
||||
})
|
||||
.catch(function () {
|
||||
document.getElementById("node-table").innerHTML =
|
||||
@@ -310,7 +521,8 @@ function groupNodes(nodes) {
|
||||
});
|
||||
groupOrder.forEach(function (prefix) {
|
||||
groupMap[prefix].nodes.sort(function (a, b) {
|
||||
return b.ws_running + b.ws_attention - (a.ws_running + a.ws_attention);
|
||||
var d = b.ws_running + b.ws_attention - (a.ws_running + a.ws_attention);
|
||||
return d !== 0 ? d : a.node_id.localeCompare(b.node_id);
|
||||
});
|
||||
});
|
||||
var groups = groupOrder.map(function (p) {
|
||||
@@ -653,38 +865,37 @@ function drillDownToNode(nodeId, serverUrl) {
|
||||
link.href = "/node/" + encodeURIComponent(nodeId) + "/";
|
||||
link.style.display = "";
|
||||
document.getElementById("main").scrollTop = 0;
|
||||
document.getElementById("node-ws-table").innerHTML =
|
||||
'<div class="dashboard-empty">Loading workstreams...</div>';
|
||||
loadNodeDetail(nodeId);
|
||||
if (clusterState && clusterState.nodes[nodeId]) {
|
||||
renderFromState();
|
||||
} else {
|
||||
document.getElementById("node-ws-table").innerHTML =
|
||||
'<div class="dashboard-empty">Loading workstreams...</div>';
|
||||
loadNodeDetail(nodeId);
|
||||
}
|
||||
document.getElementById("breadcrumb-home").focus();
|
||||
history.pushState({ view: "node", nodeId: nodeId, serverUrl: serverUrl }, "");
|
||||
if (!_navigatingFromPopstate)
|
||||
history.pushState(
|
||||
{ view: "node", nodeId: nodeId, serverUrl: serverUrl },
|
||||
"",
|
||||
);
|
||||
}
|
||||
|
||||
function loadNodeDetail(nodeId) {
|
||||
var detailP = authFetch(
|
||||
"/v1/api/cluster/node/" + encodeURIComponent(nodeId),
|
||||
).then(function (r) {
|
||||
return r.json();
|
||||
});
|
||||
var overviewP = authFetch("/v1/api/cluster/overview").then(function (r) {
|
||||
return r.json();
|
||||
});
|
||||
Promise.all([detailP, overviewP]).then(function (res) {
|
||||
var data = res[0];
|
||||
renderStatusBar(res[1]);
|
||||
if (data.error) {
|
||||
authFetch("/v1/api/cluster/snapshot")
|
||||
.then(function (r) {
|
||||
return r.json();
|
||||
})
|
||||
.then(function (data) {
|
||||
applySnapshot(data);
|
||||
if (!clusterState || !clusterState.nodes[nodeId]) {
|
||||
document.getElementById("node-ws-table").innerHTML =
|
||||
'<div class="dashboard-empty">Node not found</div>';
|
||||
}
|
||||
})
|
||||
.catch(function () {
|
||||
document.getElementById("node-ws-table").innerHTML =
|
||||
'<div class="dashboard-empty">' + escapeHtml(data.error) + "</div>";
|
||||
return;
|
||||
}
|
||||
var ws = data.workstreams || [];
|
||||
var active = ws.filter(function (w) {
|
||||
return w.state !== "idle";
|
||||
}).length;
|
||||
document.getElementById("node-ws-summary").textContent =
|
||||
active + " active \u00b7 " + ws.length + " total";
|
||||
renderWsTable(document.getElementById("node-ws-table"), ws);
|
||||
});
|
||||
'<div class="dashboard-empty">Failed to load</div>';
|
||||
});
|
||||
}
|
||||
|
||||
// --- Drill-down: Filtered ---
|
||||
@@ -703,9 +914,11 @@ function drillDownByState(state) {
|
||||
document.getElementById("filtered-title").textContent =
|
||||
"WORKSTREAMS — " + sd.label.toUpperCase();
|
||||
document.getElementById("main").scrollTop = 0;
|
||||
loadFilteredWorkstreams();
|
||||
if (clusterState) renderFromState();
|
||||
else loadFilteredWorkstreams();
|
||||
document.getElementById("breadcrumb-home").focus();
|
||||
history.pushState({ view: "filtered", filter: currentFilter }, "");
|
||||
if (!_navigatingFromPopstate)
|
||||
history.pushState({ view: "filtered", filter: currentFilter }, "");
|
||||
}
|
||||
|
||||
function drillDownByNode(nodeId) {
|
||||
@@ -721,48 +934,20 @@ function drillDownByNode(nodeId) {
|
||||
document.getElementById("filtered-title").textContent =
|
||||
"WORKSTREAMS — " + nodeId;
|
||||
document.getElementById("main").scrollTop = 0;
|
||||
loadFilteredWorkstreams();
|
||||
if (clusterState) renderFromState();
|
||||
else loadFilteredWorkstreams();
|
||||
document.getElementById("breadcrumb-home").focus();
|
||||
history.pushState({ view: "filtered", filter: currentFilter }, "");
|
||||
if (!_navigatingFromPopstate)
|
||||
history.pushState({ view: "filtered", filter: currentFilter }, "");
|
||||
}
|
||||
|
||||
function loadFilteredWorkstreams() {
|
||||
var params =
|
||||
"page=" + currentFilter.page + "&per_page=" + currentFilter.per_page;
|
||||
if (currentFilter.state)
|
||||
params += "&state=" + encodeURIComponent(currentFilter.state);
|
||||
if (currentFilter.node)
|
||||
params += "&node=" + encodeURIComponent(currentFilter.node);
|
||||
var wsP = authFetch("/v1/api/cluster/workstreams?" + params).then(
|
||||
function (r) {
|
||||
authFetch("/v1/api/cluster/snapshot")
|
||||
.then(function (r) {
|
||||
return r.json();
|
||||
},
|
||||
);
|
||||
var overviewP = authFetch("/v1/api/cluster/overview").then(function (r) {
|
||||
return r.json();
|
||||
});
|
||||
Promise.all([wsP, overviewP])
|
||||
.then(function (res) {
|
||||
var data = res[0];
|
||||
renderStatusBar(res[1]);
|
||||
document.getElementById("main").scrollTop = 0;
|
||||
document.getElementById("filtered-summary").textContent =
|
||||
"Page " +
|
||||
data.page +
|
||||
" of " +
|
||||
data.pages +
|
||||
" (" +
|
||||
data.total +
|
||||
" total)";
|
||||
renderWsTable(
|
||||
document.getElementById("filtered-ws-table"),
|
||||
data.workstreams,
|
||||
);
|
||||
renderPagination(
|
||||
document.getElementById("filtered-pagination"),
|
||||
data.page,
|
||||
data.pages,
|
||||
);
|
||||
})
|
||||
.then(function (data) {
|
||||
applySnapshot(data);
|
||||
})
|
||||
.catch(function () {
|
||||
document.getElementById("filtered-ws-table").innerHTML =
|
||||
@@ -778,7 +963,8 @@ function renderPagination(container, page, pages) {
|
||||
prev.disabled = page <= 1;
|
||||
prev.onclick = function () {
|
||||
currentFilter.page--;
|
||||
loadFilteredWorkstreams();
|
||||
if (clusterState) renderFromState();
|
||||
else loadFilteredWorkstreams();
|
||||
};
|
||||
container.appendChild(prev);
|
||||
var info = document.createElement("span");
|
||||
@@ -789,7 +975,8 @@ function renderPagination(container, page, pages) {
|
||||
next.disabled = page >= pages;
|
||||
next.onclick = function () {
|
||||
currentFilter.page++;
|
||||
loadFilteredWorkstreams();
|
||||
if (clusterState) renderFromState();
|
||||
else loadFilteredWorkstreams();
|
||||
};
|
||||
container.appendChild(next);
|
||||
}
|
||||
@@ -923,19 +1110,24 @@ function renderWsTable(container, wsList) {
|
||||
window.addEventListener("popstate", function (e) {
|
||||
var overlay = document.getElementById("login-overlay");
|
||||
if (overlay && overlay.style.display !== "none") return;
|
||||
if (!e.state) {
|
||||
showOverview();
|
||||
return;
|
||||
}
|
||||
if (e.state.view === "overview") showOverview();
|
||||
else if (e.state.view === "admin" && typeof showAdmin === "function")
|
||||
showAdmin();
|
||||
else if (e.state.view === "node" && e.state.nodeId)
|
||||
drillDownToNode(e.state.nodeId, e.state.serverUrl);
|
||||
else if (e.state.view === "filtered" && e.state.filter) {
|
||||
currentFilter = e.state.filter;
|
||||
if (currentFilter.state) drillDownByState(currentFilter.state);
|
||||
else if (currentFilter.node) drillDownByNode(currentFilter.node);
|
||||
_navigatingFromPopstate = true;
|
||||
try {
|
||||
if (!e.state) {
|
||||
showOverview();
|
||||
return;
|
||||
}
|
||||
if (e.state.view === "overview") showOverview();
|
||||
else if (e.state.view === "admin" && typeof showAdmin === "function")
|
||||
showAdmin();
|
||||
else if (e.state.view === "node" && e.state.nodeId)
|
||||
drillDownToNode(e.state.nodeId, e.state.serverUrl);
|
||||
else if (e.state.view === "filtered" && e.state.filter) {
|
||||
currentFilter = e.state.filter;
|
||||
if (currentFilter.state) drillDownByState(currentFilter.state);
|
||||
else if (currentFilter.node) drillDownByNode(currentFilter.node);
|
||||
}
|
||||
} finally {
|
||||
_navigatingFromPopstate = false;
|
||||
}
|
||||
});
|
||||
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -81,6 +81,12 @@
|
||||
<button id="tab-tokens" class="admin-tab" data-tab="tokens" role="tab" aria-selected="false" aria-controls="admin-tokens" tabindex="-1" onclick="switchAdminTab('tokens')">Tokens</button>
|
||||
<button id="tab-channels" class="admin-tab" data-tab="channels" role="tab" aria-selected="false" aria-controls="admin-channels" tabindex="-1" onclick="switchAdminTab('channels')">Channels</button>
|
||||
<button id="tab-schedules" class="admin-tab" data-tab="schedules" role="tab" aria-selected="false" aria-controls="admin-schedules" tabindex="-1" onclick="switchAdminTab('schedules')">Schedules</button>
|
||||
<button id="tab-watches" class="admin-tab" data-tab="watches" role="tab" aria-selected="false" aria-controls="admin-watches" tabindex="-1" onclick="switchAdminTab('watches')">Watches</button>
|
||||
<button id="tab-roles" class="admin-tab" data-tab="roles" role="tab" aria-selected="false" aria-controls="admin-roles" tabindex="-1" onclick="switchAdminTab('roles')">Roles</button>
|
||||
<button id="tab-policies" class="admin-tab" data-tab="policies" role="tab" aria-selected="false" aria-controls="admin-policies" tabindex="-1" onclick="switchAdminTab('policies')">Policies</button>
|
||||
<button id="tab-templates" class="admin-tab" data-tab="templates" role="tab" aria-selected="false" aria-controls="admin-templates" tabindex="-1" onclick="switchAdminTab('templates')">Templates</button>
|
||||
<button id="tab-usage" class="admin-tab" data-tab="usage" role="tab" aria-selected="false" aria-controls="admin-usage" tabindex="-1" onclick="switchAdminTab('usage')">Usage</button>
|
||||
<button id="tab-audit" class="admin-tab" data-tab="audit" role="tab" aria-selected="false" aria-controls="admin-audit" tabindex="-1" onclick="switchAdminTab('audit')">Audit</button>
|
||||
</div>
|
||||
|
||||
<!-- Users Tab -->
|
||||
@@ -163,6 +169,142 @@
|
||||
<div class="dashboard-empty">Loading schedules...</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Watches Tab -->
|
||||
<div id="admin-watches" class="admin-panel" role="tabpanel" aria-labelledby="tab-watches" style="display:none">
|
||||
<div class="admin-toolbar">
|
||||
<span class="section-header" style="margin:0">WATCHES</span>
|
||||
<label for="admin-watch-node" class="sr-only">Filter watches by node</label>
|
||||
<select id="admin-watch-node" onchange="loadAdminWatches()">
|
||||
<option value="">All nodes</option>
|
||||
</select>
|
||||
</div>
|
||||
<div class="admin-colheaders" aria-hidden="true">
|
||||
<span class="admin-col admin-col-wname">NAME</span>
|
||||
<span class="admin-col admin-col-wnode">NODE</span>
|
||||
<span class="admin-col admin-col-wcmd">COMMAND</span>
|
||||
<span class="admin-col admin-col-winterval">INTERVAL</span>
|
||||
<span class="admin-col admin-col-wpoll">POLL</span>
|
||||
<span class="admin-col admin-col-wcond">CONDITION</span>
|
||||
<span class="admin-col admin-col-wstatus">STATUS</span>
|
||||
<span class="admin-col admin-col-actions">ACTIONS</span>
|
||||
</div>
|
||||
<div id="admin-watches-table" role="list" aria-label="Watches" aria-live="polite">
|
||||
<div class="dashboard-empty">Loading watches...</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Roles Tab -->
|
||||
<div id="admin-roles" class="admin-panel" role="tabpanel" aria-labelledby="tab-roles" style="display:none">
|
||||
<div class="admin-toolbar">
|
||||
<span class="section-header" style="margin:0">ROLES</span>
|
||||
<button class="admin-action-btn" onclick="showCreateRoleModal()">+ Create role</button>
|
||||
</div>
|
||||
<div class="admin-colheaders" aria-hidden="true">
|
||||
<span class="admin-col admin-col-rname">NAME</span>
|
||||
<span class="admin-col admin-col-rperms">PERMISSIONS</span>
|
||||
<span class="admin-col admin-col-actions">ACTIONS</span>
|
||||
</div>
|
||||
<div id="admin-roles-table" role="list" aria-label="Roles" aria-live="polite">
|
||||
<div class="dashboard-empty">Loading roles...</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Policies Tab -->
|
||||
<div id="admin-policies" class="admin-panel" role="tabpanel" aria-labelledby="tab-policies" style="display:none">
|
||||
<div class="admin-toolbar">
|
||||
<span class="section-header" style="margin:0">TOOL POLICIES</span>
|
||||
<button class="admin-action-btn" onclick="showCreatePolicyModal()">+ Create policy</button>
|
||||
</div>
|
||||
<div class="admin-colheaders" aria-hidden="true">
|
||||
<span class="admin-col admin-col-pname">NAME</span>
|
||||
<span class="admin-col admin-col-ppattern">PATTERN</span>
|
||||
<span class="admin-col admin-col-paction">ACTION</span>
|
||||
<span class="admin-col admin-col-ppriority">PRI</span>
|
||||
<span class="admin-col admin-col-pstatus">STATUS</span>
|
||||
<span class="admin-col admin-col-actions">ACTIONS</span>
|
||||
</div>
|
||||
<div id="admin-policies-table" role="list" aria-label="Tool policies" aria-live="polite">
|
||||
<div class="dashboard-empty">Loading policies...</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Templates Tab -->
|
||||
<div id="admin-templates" class="admin-panel" role="tabpanel" aria-labelledby="tab-templates" style="display:none">
|
||||
<div class="admin-toolbar">
|
||||
<span class="section-header" style="margin:0">PROMPT TEMPLATES</span>
|
||||
<button class="admin-action-btn" onclick="showCreateTemplateModal()">+ Create template</button>
|
||||
</div>
|
||||
<div class="admin-colheaders" aria-hidden="true">
|
||||
<span class="admin-col admin-col-tmname">NAME</span>
|
||||
<span class="admin-col admin-col-tmcat">CATEGORY</span>
|
||||
<span class="admin-col admin-col-tmvars">VARIABLES</span>
|
||||
<span class="admin-col admin-col-actions">ACTIONS</span>
|
||||
</div>
|
||||
<div id="admin-templates-table" role="list" aria-label="Prompt templates" aria-live="polite">
|
||||
<div class="dashboard-empty">Loading templates...</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Usage Tab -->
|
||||
<div id="admin-usage" class="admin-panel" role="tabpanel" aria-labelledby="tab-usage" style="display:none">
|
||||
<div class="admin-toolbar">
|
||||
<span class="section-header" style="margin:0">USAGE</span>
|
||||
<div class="usage-range-group" role="group" aria-label="Time range">
|
||||
<button class="usage-range-btn" data-range="24h" aria-pressed="false" onclick="setUsageRange('24h')">24h</button>
|
||||
<button class="usage-range-btn active" data-range="7d" aria-pressed="true" onclick="setUsageRange('7d')">7d</button>
|
||||
<button class="usage-range-btn" data-range="30d" aria-pressed="false" onclick="setUsageRange('30d')">30d</button>
|
||||
</div>
|
||||
<div class="usage-range-group" role="group" aria-label="Group by">
|
||||
<button class="usage-group-btn active" data-group="day" aria-pressed="true" onclick="setUsageGroupBy('day')">day</button>
|
||||
<button class="usage-group-btn" data-group="model" aria-pressed="false" onclick="setUsageGroupBy('model')">model</button>
|
||||
<button class="usage-group-btn" data-group="user" aria-pressed="false" onclick="setUsageGroupBy('user')">user</button>
|
||||
</div>
|
||||
</div>
|
||||
<div id="admin-usage-content">
|
||||
<div class="dashboard-empty">Loading usage data...</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Audit Tab -->
|
||||
<div id="admin-audit" class="admin-panel" role="tabpanel" aria-labelledby="tab-audit" style="display:none">
|
||||
<div class="admin-toolbar">
|
||||
<span class="section-header" style="margin:0">AUDIT LOG</span>
|
||||
<label for="audit-action-filter" class="sr-only">Filter by action</label>
|
||||
<select id="audit-action-filter" onchange="loadGovAudit()">
|
||||
<option value="">All actions</option>
|
||||
<option value="user.create">user.create</option>
|
||||
<option value="user.delete">user.delete</option>
|
||||
<option value="token.create">token.create</option>
|
||||
<option value="token.revoke">token.revoke</option>
|
||||
<option value="role.create">role.create</option>
|
||||
<option value="role.update">role.update</option>
|
||||
<option value="role.delete">role.delete</option>
|
||||
<option value="role.assign">role.assign</option>
|
||||
<option value="role.unassign">role.unassign</option>
|
||||
<option value="policy.create">policy.create</option>
|
||||
<option value="policy.update">policy.update</option>
|
||||
<option value="policy.delete">policy.delete</option>
|
||||
<option value="template.create">template.create</option>
|
||||
<option value="template.update">template.update</option>
|
||||
<option value="template.delete">template.delete</option>
|
||||
</select>
|
||||
<label for="audit-user-filter" class="sr-only">Filter by user</label>
|
||||
<select id="audit-user-filter" onchange="loadGovAudit()">
|
||||
<option value="">All users</option>
|
||||
</select>
|
||||
</div>
|
||||
<div class="admin-colheaders" aria-hidden="true">
|
||||
<span class="admin-col admin-col-atime">TIME</span>
|
||||
<span class="admin-col admin-col-auser">USER</span>
|
||||
<span class="admin-col admin-col-aaction">ACTION</span>
|
||||
<span class="admin-col admin-col-aresource">RESOURCE</span>
|
||||
<span class="admin-col admin-col-adetail">DETAIL</span>
|
||||
</div>
|
||||
<div id="admin-audit-table" role="list" aria-label="Audit events" aria-live="polite">
|
||||
<div class="dashboard-empty">Loading audit log...</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
@@ -409,7 +551,162 @@ window.TURNSTONE_KB_SHORTCUTS = [
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Create Role Modal -->
|
||||
<div id="create-role-overlay" style="display:none" role="dialog" aria-modal="true" aria-labelledby="create-role-title">
|
||||
<div id="create-role-box" class="admin-modal admin-modal-wide">
|
||||
<h2 id="create-role-title">Create Role</h2>
|
||||
<div id="create-role-error" role="alert" aria-live="assertive"></div>
|
||||
<label for="cr-name">Name</label>
|
||||
<input id="cr-name" type="text" placeholder="e.g. security-reviewer" autocomplete="off" spellcheck="false">
|
||||
<label for="cr-displayname">Display name</label>
|
||||
<input id="cr-displayname" type="text" placeholder="Security Reviewer" autocomplete="off">
|
||||
<fieldset class="perm-fieldset"><legend>Permissions</legend>
|
||||
<div id="cr-perms-container" role="group" aria-label="Permissions"></div>
|
||||
</fieldset>
|
||||
<div class="modal-buttons">
|
||||
<button class="modal-cancel" onclick="hideCreateRoleModal()">Cancel</button>
|
||||
<button id="cr-submit" class="modal-submit" onclick="submitCreateRole()">Create</button>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Edit Role Modal -->
|
||||
<div id="edit-role-overlay" style="display:none" role="dialog" aria-modal="true" aria-labelledby="edit-role-title">
|
||||
<div id="edit-role-box" class="admin-modal admin-modal-wide">
|
||||
<h2 id="edit-role-title">Edit Role</h2>
|
||||
<div id="edit-role-error" role="alert" aria-live="assertive"></div>
|
||||
<input id="er-id" type="hidden">
|
||||
<label for="er-name">Display name</label>
|
||||
<input id="er-name" type="text" autocomplete="off">
|
||||
<fieldset class="perm-fieldset"><legend>Permissions</legend>
|
||||
<div id="er-perms-container" role="group" aria-label="Permissions"></div>
|
||||
</fieldset>
|
||||
<div class="modal-buttons">
|
||||
<button class="modal-cancel" onclick="hideEditRoleModal()">Cancel</button>
|
||||
<button id="er-submit" class="modal-submit" onclick="submitEditRole()">Save</button>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- User Roles Modal -->
|
||||
<div id="user-roles-overlay" style="display:none" role="dialog" aria-modal="true" aria-labelledby="user-roles-title">
|
||||
<div id="user-roles-box" class="admin-modal">
|
||||
<h2 id="user-roles-title">Assign Roles</h2>
|
||||
<div id="user-roles-error" role="alert" aria-live="assertive"></div>
|
||||
<input id="ur-user-id" type="hidden">
|
||||
<div id="ur-roles-container"></div>
|
||||
<div class="modal-buttons">
|
||||
<button class="modal-cancel" onclick="hideUserRolesModal()">Cancel</button>
|
||||
<button class="modal-submit" onclick="submitUserRoles()">Save</button>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Create Policy Modal -->
|
||||
<div id="create-policy-overlay" style="display:none" role="dialog" aria-modal="true" aria-labelledby="create-policy-title">
|
||||
<div id="create-policy-box" class="admin-modal">
|
||||
<h2 id="create-policy-title">Create Tool Policy</h2>
|
||||
<div id="create-policy-error" role="alert" aria-live="assertive"></div>
|
||||
<label for="cp-name">Name</label>
|
||||
<input id="cp-name" type="text" placeholder="e.g. Block shell access" autocomplete="off">
|
||||
<label for="cp-pattern">Tool pattern <span class="label-hint">glob syntax: bash*, file_write, *</span></label>
|
||||
<input id="cp-pattern" type="text" placeholder="bash*" autocomplete="off" spellcheck="false">
|
||||
<label for="cp-action">Action</label>
|
||||
<select id="cp-action">
|
||||
<option value="ask">Ask (require approval)</option>
|
||||
<option value="allow">Allow (auto-approve)</option>
|
||||
<option value="deny">Deny (block)</option>
|
||||
</select>
|
||||
<label for="cp-priority">Priority <span class="label-hint">higher = evaluated first</span></label>
|
||||
<input id="cp-priority" type="number" value="0" min="0" max="9999">
|
||||
<div class="modal-buttons">
|
||||
<button class="modal-cancel" onclick="hideCreatePolicyModal()">Cancel</button>
|
||||
<button id="cp-submit" class="modal-submit" onclick="submitCreatePolicy()">Create</button>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Edit Policy Modal -->
|
||||
<div id="edit-policy-overlay" style="display:none" role="dialog" aria-modal="true" aria-labelledby="edit-policy-title">
|
||||
<div id="edit-policy-box" class="admin-modal">
|
||||
<h2 id="edit-policy-title">Edit Tool Policy</h2>
|
||||
<div id="edit-policy-error" role="alert" aria-live="assertive"></div>
|
||||
<input id="ep-id" type="hidden">
|
||||
<label for="ep-name">Name</label>
|
||||
<input id="ep-name" type="text" autocomplete="off">
|
||||
<label for="ep-pattern">Tool pattern</label>
|
||||
<input id="ep-pattern" type="text" autocomplete="off" spellcheck="false">
|
||||
<label for="ep-action">Action</label>
|
||||
<select id="ep-action">
|
||||
<option value="ask">Ask (require approval)</option>
|
||||
<option value="allow">Allow (auto-approve)</option>
|
||||
<option value="deny">Deny (block)</option>
|
||||
</select>
|
||||
<label for="ep-priority">Priority</label>
|
||||
<input id="ep-priority" type="number" value="0" min="0" max="9999">
|
||||
<label class="admin-checkbox"><input id="ep-enabled" type="checkbox" checked> Enabled</label>
|
||||
<div class="modal-buttons">
|
||||
<button class="modal-cancel" onclick="hideEditPolicyModal()">Cancel</button>
|
||||
<button id="ep-submit" class="modal-submit" onclick="submitEditPolicy()">Save</button>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Create Template Modal -->
|
||||
<div id="create-template-overlay" style="display:none" role="dialog" aria-modal="true" aria-labelledby="create-template-title">
|
||||
<div id="create-template-box" class="admin-modal admin-modal-wide">
|
||||
<h2 id="create-template-title">Create Prompt Template</h2>
|
||||
<div id="create-template-error" role="alert" aria-live="assertive"></div>
|
||||
<label for="ctm-name">Name</label>
|
||||
<input id="ctm-name" type="text" placeholder="e.g. Code Review Agent" autocomplete="off">
|
||||
<label for="ctm-category">Category</label>
|
||||
<select id="ctm-category">
|
||||
<option value="general">General</option>
|
||||
<option value="engineering">Engineering</option>
|
||||
<option value="support">Support</option>
|
||||
<option value="custom">Custom</option>
|
||||
</select>
|
||||
<label for="ctm-content">Content <span class="label-hint">system message text, use {{variable}} for placeholders</span></label>
|
||||
<textarea id="ctm-content" rows="6" placeholder="You are a helpful assistant for {{project_name}}..."></textarea>
|
||||
<label for="ctm-variables">Variables <span class="label-hint">comma-separated list</span></label>
|
||||
<input id="ctm-variables" type="text" placeholder="project_name, review_focus" autocomplete="off">
|
||||
<label class="admin-checkbox"><input id="ctm-default" type="checkbox"> Set as default for new workstreams</label>
|
||||
<div class="modal-buttons">
|
||||
<button class="modal-cancel" onclick="hideCreateTemplateModal()">Cancel</button>
|
||||
<button id="ctm-submit" class="modal-submit" onclick="submitCreateTemplate()">Create</button>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Edit Template Modal -->
|
||||
<div id="edit-template-overlay" style="display:none" role="dialog" aria-modal="true" aria-labelledby="edit-template-title">
|
||||
<div id="edit-template-box" class="admin-modal admin-modal-wide">
|
||||
<h2 id="edit-template-title">Edit Prompt Template</h2>
|
||||
<div id="edit-template-error" role="alert" aria-live="assertive"></div>
|
||||
<input id="etm-id" type="hidden">
|
||||
<label for="etm-name">Name</label>
|
||||
<input id="etm-name" type="text" autocomplete="off">
|
||||
<label for="etm-category">Category</label>
|
||||
<select id="etm-category">
|
||||
<option value="general">General</option>
|
||||
<option value="engineering">Engineering</option>
|
||||
<option value="support">Support</option>
|
||||
<option value="custom">Custom</option>
|
||||
</select>
|
||||
<label for="etm-content">Content</label>
|
||||
<textarea id="etm-content" rows="6"></textarea>
|
||||
<label for="etm-variables">Variables</label>
|
||||
<input id="etm-variables" type="text" autocomplete="off">
|
||||
<label class="admin-checkbox"><input id="etm-default" type="checkbox"> Set as default</label>
|
||||
<div class="modal-buttons">
|
||||
<button class="modal-cancel" onclick="hideEditTemplateModal()">Cancel</button>
|
||||
<button id="etm-submit" class="modal-submit" onclick="submitEditTemplate()">Save</button>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<script src="/static/admin.js"></script>
|
||||
<script src="/static/governance.js"></script>
|
||||
<script src="/static/app.js"></script>
|
||||
</body>
|
||||
</html>
|
||||
|
||||
@@ -709,6 +709,11 @@
|
||||
color: var(--accent);
|
||||
border-bottom-color: var(--accent);
|
||||
}
|
||||
.admin-tab:focus-visible {
|
||||
outline: 2px solid var(--accent);
|
||||
outline-offset: -2px;
|
||||
border-radius: var(--radius-sm);
|
||||
}
|
||||
|
||||
.admin-toolbar {
|
||||
display: flex;
|
||||
@@ -863,6 +868,16 @@
|
||||
.sched-disabled { color: var(--fg-dim); }
|
||||
.sched-expired { color: var(--accent); }
|
||||
|
||||
/* Watches grid: NAME | NODE | COMMAND | INTERVAL | POLL | CONDITION | STATUS | ACTIONS */
|
||||
#admin-watches .admin-colheaders,
|
||||
#admin-watches .admin-row {
|
||||
grid-template-columns: 1.2fr 80px 1.5fr 60px 70px 1fr 70px 70px;
|
||||
}
|
||||
|
||||
/* Watch status indicators */
|
||||
.watch-active { color: var(--green); font-weight: 500; }
|
||||
.watch-completed { color: var(--accent); }
|
||||
|
||||
/* Wide modal variant for schedule forms */
|
||||
.admin-modal-wide { width: 480px; }
|
||||
|
||||
@@ -979,7 +994,10 @@
|
||||
.modal-submit:disabled { opacity: 0.4; cursor: not-allowed; filter: none; }
|
||||
|
||||
#create-user-overlay, #create-token-overlay, #token-created-overlay, #create-channel-overlay, #confirm-overlay,
|
||||
#create-schedule-overlay, #edit-schedule-overlay, #schedule-runs-overlay {
|
||||
#create-schedule-overlay, #edit-schedule-overlay, #schedule-runs-overlay,
|
||||
#create-role-overlay, #edit-role-overlay, #user-roles-overlay,
|
||||
#create-policy-overlay, #edit-policy-overlay,
|
||||
#create-template-overlay, #edit-template-overlay {
|
||||
position: fixed;
|
||||
inset: 0;
|
||||
background: rgba(0, 0, 0, 0.7);
|
||||
@@ -1027,6 +1045,267 @@
|
||||
grid-template-columns: 1fr 60px 80px 130px;
|
||||
}
|
||||
.admin-col-sschedule, .admin-col-starget, .admin-col-snext { display: none; }
|
||||
#admin-watches .admin-colheaders, #admin-watches .admin-row {
|
||||
grid-template-columns: 1.2fr 80px 70px 70px 70px;
|
||||
}
|
||||
.admin-col-wcmd, .admin-col-wcond, .admin-col-winterval { display: none; }
|
||||
}
|
||||
|
||||
/* ==========================================================================
|
||||
Admin tabs — horizontal scroll for 10+ tabs
|
||||
========================================================================== */
|
||||
.admin-tabs {
|
||||
overflow-x: auto;
|
||||
-webkit-overflow-scrolling: touch;
|
||||
flex-wrap: nowrap;
|
||||
scrollbar-width: thin;
|
||||
}
|
||||
|
||||
/* ==========================================================================
|
||||
Governance: Roles grid
|
||||
========================================================================== */
|
||||
#admin-roles .admin-colheaders,
|
||||
#admin-roles .admin-row {
|
||||
grid-template-columns: 160px 1fr 110px;
|
||||
}
|
||||
|
||||
/* ==========================================================================
|
||||
Governance: Tool Policies grid
|
||||
========================================================================== */
|
||||
#admin-policies .admin-colheaders,
|
||||
#admin-policies .admin-row {
|
||||
grid-template-columns: 1.2fr 1fr 70px 50px 80px 140px;
|
||||
}
|
||||
|
||||
/* Policy action badges */
|
||||
.policy-badge {
|
||||
display: inline-block;
|
||||
font-family: var(--font-display);
|
||||
font-size: 9px;
|
||||
font-weight: 600;
|
||||
text-transform: uppercase;
|
||||
letter-spacing: 0.06em;
|
||||
padding: 2px 8px;
|
||||
border-radius: 2px;
|
||||
}
|
||||
.policy-allow {
|
||||
color: var(--green);
|
||||
background: var(--green-glow);
|
||||
border: 1px solid var(--green-glow);
|
||||
}
|
||||
.policy-deny {
|
||||
color: var(--red);
|
||||
background: var(--red-glow);
|
||||
border: 1px solid var(--red-glow);
|
||||
}
|
||||
.policy-ask {
|
||||
color: var(--yellow);
|
||||
background: var(--yellow-glow);
|
||||
border: 1px solid var(--yellow-glow);
|
||||
}
|
||||
|
||||
/* ==========================================================================
|
||||
Governance: Prompt Templates grid
|
||||
========================================================================== */
|
||||
#admin-templates .admin-colheaders,
|
||||
#admin-templates .admin-row {
|
||||
grid-template-columns: 1.5fr 100px 1fr 140px;
|
||||
}
|
||||
|
||||
/* ==========================================================================
|
||||
Governance: Audit grid
|
||||
========================================================================== */
|
||||
#admin-audit .admin-colheaders,
|
||||
#admin-audit .admin-row {
|
||||
grid-template-columns: 80px 80px 1fr 120px 1.5fr;
|
||||
}
|
||||
|
||||
/* Audit action badges */
|
||||
.audit-badge {
|
||||
display: inline-block;
|
||||
font-family: var(--font-display);
|
||||
font-size: 9px;
|
||||
font-weight: 600;
|
||||
text-transform: none;
|
||||
letter-spacing: 0.02em;
|
||||
padding: 1px 6px;
|
||||
border-radius: 2px;
|
||||
background: var(--bg-highlight);
|
||||
color: var(--fg-dim);
|
||||
border: 1px solid var(--border);
|
||||
}
|
||||
.audit-danger { color: var(--red); border-color: var(--red-glow); }
|
||||
.audit-success { color: var(--green); border-color: var(--green-glow); }
|
||||
|
||||
/* ==========================================================================
|
||||
Governance: Usage dashboard
|
||||
========================================================================== */
|
||||
.usage-summary {
|
||||
display: flex;
|
||||
gap: 24px;
|
||||
padding: 16px 0 20px;
|
||||
flex-wrap: wrap;
|
||||
}
|
||||
.usage-readout {
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
gap: 2px;
|
||||
}
|
||||
.usage-readout-value {
|
||||
font-size: 22px;
|
||||
font-weight: 600;
|
||||
color: var(--fg-bright);
|
||||
font-variant-numeric: tabular-nums;
|
||||
font-family: var(--font-mono);
|
||||
letter-spacing: -0.02em;
|
||||
}
|
||||
.usage-readout-label {
|
||||
font-size: 10px;
|
||||
font-family: var(--font-display);
|
||||
font-weight: 600;
|
||||
text-transform: uppercase;
|
||||
letter-spacing: 0.08em;
|
||||
color: var(--fg-dim);
|
||||
}
|
||||
|
||||
/* Usage bar chart */
|
||||
.usage-chart { padding-top: 4px; }
|
||||
.usage-bar-row {
|
||||
display: grid;
|
||||
grid-template-columns: 90px 1fr 60px;
|
||||
align-items: center;
|
||||
gap: 10px;
|
||||
padding: 4px 0;
|
||||
}
|
||||
.usage-bar-label {
|
||||
font-size: 11px;
|
||||
color: var(--fg-dim);
|
||||
text-align: right;
|
||||
font-variant-numeric: tabular-nums;
|
||||
overflow: hidden;
|
||||
text-overflow: ellipsis;
|
||||
white-space: nowrap;
|
||||
}
|
||||
.usage-bar-track {
|
||||
height: 16px;
|
||||
background: var(--bg-highlight);
|
||||
border-radius: 2px;
|
||||
overflow: hidden;
|
||||
}
|
||||
.usage-bar-fill {
|
||||
height: 100%;
|
||||
background: var(--accent);
|
||||
border-radius: 2px;
|
||||
min-width: 2px;
|
||||
transition: width 0.3s ease;
|
||||
box-shadow: 0 0 6px var(--accent-glow);
|
||||
}
|
||||
.usage-bar-value {
|
||||
font-size: 11px;
|
||||
color: var(--fg-dim);
|
||||
font-variant-numeric: tabular-nums;
|
||||
text-align: right;
|
||||
}
|
||||
|
||||
/* Usage range/group buttons */
|
||||
.usage-range-group {
|
||||
display: flex;
|
||||
gap: 2px;
|
||||
border: 1px solid var(--border-strong);
|
||||
border-radius: var(--radius-sm);
|
||||
overflow: hidden;
|
||||
}
|
||||
.usage-range-btn, .usage-group-btn {
|
||||
background: var(--bg);
|
||||
color: var(--fg-dim);
|
||||
border: none;
|
||||
font-family: var(--font-display);
|
||||
font-size: 10px;
|
||||
font-weight: 600;
|
||||
text-transform: uppercase;
|
||||
letter-spacing: 0.06em;
|
||||
padding: 5px 10px;
|
||||
cursor: pointer;
|
||||
transition: background 0.15s, color 0.15s;
|
||||
}
|
||||
.usage-range-btn:hover, .usage-group-btn:hover {
|
||||
background: var(--bg-highlight);
|
||||
color: var(--fg);
|
||||
}
|
||||
.usage-range-btn.active, .usage-group-btn.active {
|
||||
background: var(--accent-dim);
|
||||
color: var(--accent);
|
||||
}
|
||||
.usage-range-btn:focus-visible, .usage-group-btn:focus-visible {
|
||||
outline: 2px solid var(--accent);
|
||||
outline-offset: -2px;
|
||||
}
|
||||
|
||||
/* ==========================================================================
|
||||
Governance: Permission grid (modal checkboxes)
|
||||
========================================================================== */
|
||||
.perm-fieldset {
|
||||
border: none;
|
||||
padding: 0;
|
||||
margin: 12px 0 0;
|
||||
}
|
||||
.perm-fieldset legend {
|
||||
font-family: var(--font-display);
|
||||
font-size: 10px;
|
||||
font-weight: 600;
|
||||
text-transform: uppercase;
|
||||
letter-spacing: 0.08em;
|
||||
color: var(--fg-dim);
|
||||
padding: 0;
|
||||
margin-bottom: 4px;
|
||||
}
|
||||
.perm-grid {
|
||||
display: grid;
|
||||
grid-template-columns: 1fr 1fr;
|
||||
gap: 4px 16px;
|
||||
padding: 8px 0;
|
||||
}
|
||||
.perm-checkbox {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 6px;
|
||||
font-size: 11px;
|
||||
font-family: var(--font-mono);
|
||||
color: var(--fg);
|
||||
padding: 3px 0;
|
||||
cursor: pointer;
|
||||
text-transform: none;
|
||||
letter-spacing: normal;
|
||||
}
|
||||
.perm-checkbox input[type="checkbox"] {
|
||||
width: auto;
|
||||
margin: 0;
|
||||
accent-color: var(--accent);
|
||||
}
|
||||
|
||||
/* ==========================================================================
|
||||
Governance: Responsive
|
||||
========================================================================== */
|
||||
@media (max-width: 700px) {
|
||||
#admin-roles .admin-colheaders, #admin-roles .admin-row {
|
||||
grid-template-columns: 1fr 100px;
|
||||
}
|
||||
.admin-col-rperms { display: none; }
|
||||
#admin-policies .admin-colheaders, #admin-policies .admin-row {
|
||||
grid-template-columns: 1fr 70px 50px 100px;
|
||||
}
|
||||
.admin-col-pstatus, .admin-col-ppriority { display: none; }
|
||||
#admin-templates .admin-colheaders, #admin-templates .admin-row {
|
||||
grid-template-columns: 1fr 100px;
|
||||
}
|
||||
.admin-col-tmcat, .admin-col-tmvars { display: none; }
|
||||
#admin-audit .admin-colheaders, #admin-audit .admin-row {
|
||||
grid-template-columns: 60px 1fr 100px;
|
||||
}
|
||||
.admin-col-auser, .admin-col-adetail { display: none; }
|
||||
.usage-readout-value { font-size: 18px; }
|
||||
.usage-bar-row { grid-template-columns: 70px 1fr 50px; }
|
||||
.perm-grid { grid-template-columns: 1fr; }
|
||||
}
|
||||
|
||||
/* ==========================================================================
|
||||
|
||||
@@ -0,0 +1,41 @@
|
||||
"""Audit event recording helper.
|
||||
|
||||
Provides a fire-and-forget ``record_audit`` function that admin handlers
|
||||
call after mutations to create a persistent audit trail.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import logging
|
||||
import uuid
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from turnstone.core.storage._protocol import StorageBackend
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def record_audit(
|
||||
storage: StorageBackend,
|
||||
user_id: str,
|
||||
action: str,
|
||||
resource_type: str = "",
|
||||
resource_id: str = "",
|
||||
detail: dict[str, Any] | None = None,
|
||||
ip_address: str = "",
|
||||
) -> None:
|
||||
"""Record an audit event. Silently logs on failure (never raises)."""
|
||||
try:
|
||||
storage.record_audit_event(
|
||||
event_id=uuid.uuid4().hex,
|
||||
user_id=user_id,
|
||||
action=action,
|
||||
resource_type=resource_type,
|
||||
resource_id=resource_id,
|
||||
detail=json.dumps(detail) if detail else "{}",
|
||||
ip_address=ip_address,
|
||||
)
|
||||
except Exception:
|
||||
log.warning("Failed to record audit event: %s %s", action, resource_id, exc_info=True)
|
||||
+117
-4
@@ -18,6 +18,7 @@ always accessible without authentication.
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import contextlib
|
||||
import hashlib
|
||||
import hmac
|
||||
import json
|
||||
@@ -33,7 +34,7 @@ from typing import TYPE_CHECKING, Any
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from starlette.requests import Request
|
||||
from starlette.responses import Response
|
||||
from starlette.responses import JSONResponse, Response
|
||||
from starlette.types import ASGIApp, Receive, Scope, Send
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
@@ -80,6 +81,60 @@ _ROLE_TO_SCOPES: dict[str, frozenset[str]] = {
|
||||
"full": frozenset({"read", "write", "approve"}),
|
||||
}
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# RBAC helpers
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _load_user_permissions(storage: Any, user_id: str) -> set[str]:
|
||||
"""Load the union of all permissions from a user's assigned roles."""
|
||||
try:
|
||||
result: set[str] = storage.get_user_permissions(user_id)
|
||||
return result
|
||||
except Exception:
|
||||
log.warning("Failed to load permissions for user %s", user_id)
|
||||
return set()
|
||||
|
||||
|
||||
def _permissions_to_scopes(permissions: set[str]) -> frozenset[str]:
|
||||
"""Derive legacy scopes from a granular permission set."""
|
||||
scopes: set[str] = set()
|
||||
if not permissions:
|
||||
scopes.add("read")
|
||||
return frozenset(scopes)
|
||||
for perm in permissions:
|
||||
if perm in VALID_SCOPES:
|
||||
scopes.update(SCOPE_HIERARCHY.get(perm, {perm}))
|
||||
# Any admin.* permission requires access to admin endpoints → approve scope
|
||||
if any(p.startswith("admin.") for p in permissions):
|
||||
scopes.update(SCOPE_HIERARCHY["approve"])
|
||||
if not scopes:
|
||||
scopes.add("read")
|
||||
return frozenset(scopes)
|
||||
|
||||
|
||||
def require_permission(request: Request, permission: str) -> JSONResponse | None:
|
||||
"""Return a 403 JSONResponse if the user lacks *permission*, else None.
|
||||
|
||||
Call from admin handlers after the middleware scope check passes.
|
||||
Config-file tokens (no user_id) are treated as full-access.
|
||||
"""
|
||||
from starlette.responses import JSONResponse
|
||||
|
||||
auth_result: AuthResult | None = getattr(getattr(request, "state", None), "auth_result", None)
|
||||
if auth_result is None:
|
||||
return JSONResponse({"error": "Unauthorized"}, status_code=401)
|
||||
# Config-file tokens (no user_id) are treated as full-access
|
||||
if not auth_result.user_id:
|
||||
return None
|
||||
if auth_result.has_permission(permission):
|
||||
return None
|
||||
return JSONResponse(
|
||||
{"error": f"Forbidden: missing '{permission}' permission"},
|
||||
status_code=403,
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Path classification
|
||||
# ---------------------------------------------------------------------------
|
||||
@@ -104,6 +159,7 @@ WRITE_PATHS: frozenset[str] = frozenset(
|
||||
"/api/send",
|
||||
"/api/plan",
|
||||
"/api/command",
|
||||
"/api/cancel",
|
||||
"/api/workstreams/new",
|
||||
"/api/workstreams/close",
|
||||
"/api/cluster/workstreams/new",
|
||||
@@ -133,11 +189,16 @@ class AuthResult:
|
||||
user_id: str # empty string for config-file tokens
|
||||
scopes: frozenset[str]
|
||||
token_source: str # "config", "jwt", "database"
|
||||
permissions: frozenset[str] = frozenset()
|
||||
|
||||
def has_scope(self, scope: str) -> bool:
|
||||
"""Return True if this result includes *scope*."""
|
||||
return scope in self.scopes
|
||||
|
||||
def has_permission(self, permission: str) -> bool:
|
||||
"""Return True if this result includes *permission*."""
|
||||
return permission in self.permissions
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# AuthConfig (unchanged from before — static config-file tokens)
|
||||
@@ -249,8 +310,9 @@ def create_jwt(
|
||||
secret: str,
|
||||
expiry_hours: int = 24,
|
||||
audience: str = "",
|
||||
permissions: frozenset[str] = frozenset(),
|
||||
) -> str:
|
||||
"""Create a signed JWT with user identity and scopes."""
|
||||
"""Create a signed JWT with user identity, scopes, and permissions."""
|
||||
import jwt
|
||||
|
||||
now = int(time.time())
|
||||
@@ -264,6 +326,8 @@ def create_jwt(
|
||||
}
|
||||
if audience:
|
||||
payload["aud"] = audience
|
||||
if permissions:
|
||||
payload["permissions"] = ",".join(sorted(permissions))
|
||||
return jwt.encode(payload, secret, algorithm="HS256")
|
||||
|
||||
|
||||
@@ -293,11 +357,15 @@ def validate_jwt(token: str, secret: str, audience: str = "") -> AuthResult | No
|
||||
user_id = payload.get("sub", "")
|
||||
scopes_str = payload.get("scopes", "")
|
||||
source = payload.get("src", "jwt")
|
||||
perms_str = payload.get("permissions", "")
|
||||
|
||||
perms = frozenset(p for p in perms_str.split(",") if p) if perms_str else frozenset()
|
||||
|
||||
return AuthResult(
|
||||
user_id=user_id,
|
||||
scopes=parse_scopes(scopes_str),
|
||||
token_source=source,
|
||||
permissions=perms,
|
||||
)
|
||||
|
||||
|
||||
@@ -390,6 +458,13 @@ def required_scope(method: str, path: str) -> str:
|
||||
# Write endpoints
|
||||
if method == "POST" and normalized in WRITE_PATHS:
|
||||
return "write"
|
||||
# Watch cancel has a path parameter: /api/watches/{id}/cancel
|
||||
if (
|
||||
method == "POST"
|
||||
and normalized.startswith("/api/watches/")
|
||||
and normalized.endswith("/cancel")
|
||||
):
|
||||
return "write"
|
||||
|
||||
# Console proxy routes: /node/{node_id}/api/{tail} or /node/{node_id}/v1/api/{tail}
|
||||
if method == "POST" and normalized.startswith("/node/"):
|
||||
@@ -524,10 +599,12 @@ def _authenticate_api_token(token: str, storage: Any) -> AuthResult | None:
|
||||
if exp_dt < now:
|
||||
return None
|
||||
|
||||
perms = _load_user_permissions(storage, row["user_id"]) if storage else set()
|
||||
return AuthResult(
|
||||
user_id=row["user_id"],
|
||||
scopes=parse_scopes(row["scopes"]),
|
||||
token_source="database",
|
||||
permissions=frozenset(perms),
|
||||
)
|
||||
|
||||
|
||||
@@ -828,10 +905,14 @@ async def handle_auth_login(request: Request, audience: str) -> Response:
|
||||
if username and password and storage is not None:
|
||||
user = storage.get_user_by_username(username)
|
||||
if user and verify_password(password, user["password_hash"]):
|
||||
# Derive scopes and permissions from assigned roles
|
||||
perms = _load_user_permissions(storage, user["user_id"])
|
||||
scopes = _permissions_to_scopes(perms)
|
||||
result = AuthResult(
|
||||
user_id=user["user_id"],
|
||||
scopes=frozenset({"read", "write", "approve"}),
|
||||
scopes=scopes,
|
||||
token_source="password",
|
||||
permissions=frozenset(perms),
|
||||
)
|
||||
elif body.get("token"):
|
||||
result = _authenticate_token(
|
||||
@@ -858,11 +939,14 @@ async def handle_auth_login(request: Request, audience: str) -> Response:
|
||||
source=result.token_source,
|
||||
secret=jwt_secret,
|
||||
audience=audience,
|
||||
permissions=result.permissions,
|
||||
)
|
||||
|
||||
role = "full" if result.has_scope("write") else "read"
|
||||
scopes_str = ",".join(sorted(result.scopes))
|
||||
resp_body: dict[str, str] = {"status": "ok", "role": role, "scopes": scopes_str}
|
||||
if result.permissions:
|
||||
resp_body["permissions"] = ",".join(sorted(result.permissions))
|
||||
if jwt_token:
|
||||
resp_body["jwt"] = jwt_token
|
||||
if result.user_id:
|
||||
@@ -952,7 +1036,33 @@ async def handle_auth_setup(request: Request, audience: str) -> Response:
|
||||
if not created:
|
||||
return JSONResponse({"error": "Setup already completed"}, status_code=409)
|
||||
|
||||
scopes = frozenset({"read", "write", "approve"})
|
||||
# Assign admin role to the first user — fail setup if this breaks,
|
||||
# otherwise the admin is created with read-only access and locked out.
|
||||
try:
|
||||
storage.assign_role(user_id, "builtin-admin", "")
|
||||
except Exception:
|
||||
log.error("Failed to assign admin role to first user %s — aborting setup", user_id)
|
||||
# Roll back the user creation so setup can be retried
|
||||
with contextlib.suppress(Exception):
|
||||
storage.delete_user(user_id)
|
||||
return JSONResponse(
|
||||
{"error": "Failed to assign admin role. Ensure migrations have run."},
|
||||
status_code=503,
|
||||
)
|
||||
|
||||
# Derive permissions from roles
|
||||
perms = _load_user_permissions(storage, user_id)
|
||||
if not perms:
|
||||
log.error(
|
||||
"First user %s has no permissions after role assignment — aborting setup", user_id
|
||||
)
|
||||
with contextlib.suppress(Exception):
|
||||
storage.delete_user(user_id)
|
||||
return JSONResponse(
|
||||
{"error": "Failed to load permissions. Ensure migrations have run."},
|
||||
status_code=503,
|
||||
)
|
||||
scopes = _permissions_to_scopes(perms)
|
||||
jwt_token = ""
|
||||
if jwt_secret:
|
||||
jwt_token = create_jwt(
|
||||
@@ -961,6 +1071,7 @@ async def handle_auth_setup(request: Request, audience: str) -> Response:
|
||||
source="password",
|
||||
secret=jwt_secret,
|
||||
audience=audience,
|
||||
permissions=frozenset(perms),
|
||||
)
|
||||
|
||||
resp_body: dict[str, str] = {
|
||||
@@ -970,6 +1081,8 @@ async def handle_auth_setup(request: Request, audience: str) -> Response:
|
||||
"role": "full",
|
||||
"scopes": ",".join(sorted(scopes)),
|
||||
}
|
||||
if perms:
|
||||
resp_body["permissions"] = ",".join(sorted(perms))
|
||||
if jwt_token:
|
||||
resp_body["jwt"] = jwt_token
|
||||
|
||||
|
||||
@@ -61,7 +61,7 @@ _CONFIG_MAP: dict[str, dict[str, str]] = {
|
||||
},
|
||||
"session": {
|
||||
"instructions": "instructions",
|
||||
"retention_days": "session_retention_days",
|
||||
"retention_days": "retention_days",
|
||||
"compact_max_tokens": "compact_max_tokens",
|
||||
"auto_compact_pct": "auto_compact_pct",
|
||||
},
|
||||
@@ -70,6 +70,9 @@ _CONFIG_MAP: dict[str, dict[str, str]] = {
|
||||
"truncation": "tool_truncation",
|
||||
"agent_max_turns": "agent_max_turns",
|
||||
"skip_permissions": "skip_permissions",
|
||||
"search": "tool_search",
|
||||
"search_threshold": "tool_search_threshold",
|
||||
"search_max_results": "tool_search_max_results",
|
||||
},
|
||||
"server": {
|
||||
"host": "host",
|
||||
@@ -102,6 +105,7 @@ _CONFIG_MAP: dict[str, dict[str, str]] = {
|
||||
},
|
||||
"mcp": {
|
||||
"config_path": "mcp_config",
|
||||
"refresh_interval": "mcp_refresh_interval",
|
||||
},
|
||||
"ratelimit": {
|
||||
"enabled": "ratelimit_enabled",
|
||||
@@ -150,6 +154,16 @@ def get_tavily_key() -> str | None:
|
||||
return _tavily_key
|
||||
|
||||
|
||||
def nonneg_float(val: str) -> float:
|
||||
"""Argparse type for non-negative floats (``>= 0``)."""
|
||||
f = float(val)
|
||||
if f < 0:
|
||||
import argparse
|
||||
|
||||
raise argparse.ArgumentTypeError("must be >= 0")
|
||||
return f
|
||||
|
||||
|
||||
def apply_config(parser: argparse.ArgumentParser, sections: list[str]) -> None:
|
||||
"""Set argparse defaults from config file.
|
||||
|
||||
|
||||
+223
-10
@@ -7,19 +7,33 @@ Architecture: the MCP SDK is fully async, but turnstone's ChatSession is
|
||||
synchronous. We bridge the two by running a dedicated asyncio event loop
|
||||
in a daemon thread. ``call_tool_sync`` dispatches coroutines onto that loop
|
||||
via ``asyncio.run_coroutine_threadsafe``.
|
||||
|
||||
Tool refresh: three mechanisms keep tool lists up-to-date without restart:
|
||||
1. Push notifications — servers declaring ``tools.listChanged`` trigger
|
||||
immediate refresh via ``ToolListChangedNotification``.
|
||||
2. Periodic timer — servers *without* push support are polled on a
|
||||
staggered interval (configurable, default 4 h, seeded at launch).
|
||||
3. Manual — ``/mcp refresh [server]`` triggers ``refresh_sync()``.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import contextlib
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import random
|
||||
import threading
|
||||
import time
|
||||
from contextlib import AsyncExitStack
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from collections.abc import Callable
|
||||
|
||||
import mcp.types as mcp_types
|
||||
from mcp import ClientSession, StdioServerParameters
|
||||
from mcp.client.stdio import stdio_client
|
||||
from mcp.client.streamable_http import streamablehttp_client
|
||||
@@ -28,6 +42,8 @@ from turnstone.core.config import load_config
|
||||
|
||||
log = logging.getLogger("turnstone.mcp")
|
||||
|
||||
_DEFAULT_REFRESH_INTERVAL: float = 14400 # 4 hours
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# MCP ↔ OpenAI schema conversion
|
||||
@@ -67,8 +83,15 @@ class MCPClientManager:
|
||||
synchronous methods for tool discovery and invocation.
|
||||
"""
|
||||
|
||||
def __init__(self, server_configs: dict[str, dict[str, Any]]) -> None:
|
||||
def __init__(
|
||||
self,
|
||||
server_configs: dict[str, dict[str, Any]],
|
||||
*,
|
||||
refresh_interval: float = _DEFAULT_REFRESH_INTERVAL,
|
||||
) -> None:
|
||||
self._server_configs = server_configs
|
||||
if refresh_interval < 0:
|
||||
refresh_interval = 0.0
|
||||
self._loop: asyncio.AbstractEventLoop | None = None
|
||||
self._thread: threading.Thread | None = None
|
||||
self._exit_stack: AsyncExitStack | None = None
|
||||
@@ -80,6 +103,19 @@ class MCPClientManager:
|
||||
self._connected = threading.Event()
|
||||
self._error: str | None = None
|
||||
|
||||
# Per-server tool storage for surgical refresh
|
||||
self._per_server_tools: dict[str, list[dict[str, Any]]] = {}
|
||||
# Tracks which servers support push notifications
|
||||
self._supports_list_changed: dict[str, bool] = {}
|
||||
|
||||
# Listener infrastructure (tool-change callbacks for ChatSession)
|
||||
self._listeners: list[Callable[[], None]] = []
|
||||
self._listeners_lock = threading.Lock()
|
||||
|
||||
# Periodic refresh for servers without push notifications
|
||||
self._refresh_interval = refresh_interval
|
||||
self._refresh_task: asyncio.Task[None] | None = None
|
||||
|
||||
# -- lifecycle -----------------------------------------------------------
|
||||
|
||||
def start(self) -> None:
|
||||
@@ -108,6 +144,13 @@ class MCPClientManager:
|
||||
|
||||
self._connected.set()
|
||||
|
||||
# Start periodic refresh for servers without push notifications
|
||||
needs_periodic = any(
|
||||
not self._supports_list_changed.get(name, False) for name in self._sessions
|
||||
)
|
||||
if needs_periodic and self._refresh_interval > 0:
|
||||
self._refresh_task = asyncio.get_running_loop().create_task(self._periodic_refresh())
|
||||
|
||||
async def _connect_one(self, name: str, cfg: dict[str, Any]) -> None:
|
||||
"""Connect to a single MCP server and discover its tools."""
|
||||
assert self._exit_stack is not None
|
||||
@@ -135,26 +178,187 @@ class MCPClientManager:
|
||||
)
|
||||
read, write = await self._exit_stack.enter_async_context(stdio_client(params))
|
||||
|
||||
session = await self._exit_stack.enter_async_context(ClientSession(read, write))
|
||||
# Register notification handler — lightweight; only acts on
|
||||
# ToolListChangedNotification, which is a no-op if the server
|
||||
# never sends it.
|
||||
async def _on_notification(
|
||||
msg: Any, # RequestResponder | ServerNotification | Exception
|
||||
) -> None:
|
||||
if isinstance(msg, mcp_types.ServerNotification) and isinstance(
|
||||
msg.root, mcp_types.ToolListChangedNotification
|
||||
):
|
||||
log.info("Received tools/list_changed from '%s'", name)
|
||||
try:
|
||||
await self._refresh_server(name)
|
||||
except Exception:
|
||||
log.warning("Refresh after notification failed for '%s'", name, exc_info=True)
|
||||
|
||||
session = await self._exit_stack.enter_async_context(
|
||||
ClientSession(read, write, message_handler=_on_notification) # type: ignore[arg-type]
|
||||
)
|
||||
await session.initialize()
|
||||
self._sessions[name] = session
|
||||
|
||||
# Check push notification support
|
||||
caps = session.get_server_capabilities()
|
||||
tools_cap = getattr(caps, "tools", None) if caps else None
|
||||
self._supports_list_changed[name] = bool(getattr(tools_cap, "listChanged", False))
|
||||
|
||||
# Discover tools
|
||||
result = await session.list_tools()
|
||||
server_tools: list[dict[str, Any]] = []
|
||||
for tool in result.tools:
|
||||
openai_def = _mcp_to_openai(name, tool)
|
||||
prefixed = openai_def["function"]["name"]
|
||||
self._tools.append(openai_def)
|
||||
self._tool_map[prefixed] = (name, tool.name)
|
||||
server_tools.append(_mcp_to_openai(name, tool))
|
||||
|
||||
self._per_server_tools[name] = server_tools
|
||||
self._rebuild_tools()
|
||||
|
||||
push_status = " (push)" if self._supports_list_changed[name] else ""
|
||||
log.info(
|
||||
"Connected MCP server '%s' — %d tool(s)",
|
||||
"Connected MCP server '%s' — %d tool(s)%s",
|
||||
name,
|
||||
len(result.tools),
|
||||
push_status,
|
||||
)
|
||||
|
||||
# -- tool refresh --------------------------------------------------------
|
||||
|
||||
def _rebuild_tools(self) -> None:
|
||||
"""Rebuild merged ``_tools`` and ``_tool_map`` from per-server state.
|
||||
|
||||
Uses copy-on-write: builds new objects, then assigns atomically.
|
||||
Concurrent readers see either the old or new snapshot — both valid.
|
||||
"""
|
||||
new_tools: list[dict[str, Any]] = []
|
||||
new_map: dict[str, tuple[str, str]] = {}
|
||||
for srv_name, srv_tools in self._per_server_tools.items():
|
||||
for tool in srv_tools:
|
||||
prefixed: str = tool["function"]["name"]
|
||||
new_tools.append(tool)
|
||||
# Extract original name from the mcp__server__original pattern
|
||||
original = prefixed.split("__", 2)[2] if prefixed.count("__") >= 2 else prefixed
|
||||
new_map[prefixed] = (srv_name, original)
|
||||
self._tools = new_tools
|
||||
self._tool_map = new_map
|
||||
self._notify_listeners()
|
||||
|
||||
async def _refresh_server(self, name: str) -> tuple[list[str], list[str]]:
|
||||
"""Re-fetch tools for one server. Returns ``(added, removed)`` names."""
|
||||
session = self._sessions.get(name)
|
||||
if session is None:
|
||||
raise RuntimeError(f"MCP server '{name}' is not connected")
|
||||
|
||||
old_names = {t["function"]["name"] for t in self._per_server_tools.get(name, [])}
|
||||
|
||||
result = await session.list_tools()
|
||||
server_tools = [_mcp_to_openai(name, tool) for tool in result.tools]
|
||||
new_names = {t["function"]["name"] for t in server_tools}
|
||||
|
||||
self._per_server_tools[name] = server_tools
|
||||
self._rebuild_tools()
|
||||
|
||||
added = sorted(new_names - old_names)
|
||||
removed = sorted(old_names - new_names)
|
||||
if added or removed:
|
||||
log.info(
|
||||
"Refreshed MCP server '%s': +%d/-%d tool(s)",
|
||||
name,
|
||||
len(added),
|
||||
len(removed),
|
||||
)
|
||||
return added, removed
|
||||
|
||||
async def _refresh_all(
|
||||
self, server_name: str | None = None
|
||||
) -> dict[str, tuple[list[str], list[str]]]:
|
||||
"""Refresh tools for one or all servers.
|
||||
|
||||
For disconnected servers (in config but not connected), attempts
|
||||
reconnect. Returns ``{server: (added, removed)}`` per server.
|
||||
"""
|
||||
results: dict[str, tuple[list[str], list[str]]] = {}
|
||||
targets = [server_name] if server_name else list(self._server_configs.keys())
|
||||
|
||||
for name in targets:
|
||||
try:
|
||||
if name not in self._sessions:
|
||||
# Attempt reconnect
|
||||
cfg = self._server_configs.get(name)
|
||||
if cfg:
|
||||
log.info("Reconnecting MCP server '%s'", name)
|
||||
await self._connect_one(name, cfg)
|
||||
new_names = [
|
||||
t["function"]["name"] for t in self._per_server_tools.get(name, [])
|
||||
]
|
||||
results[name] = (new_names, [])
|
||||
continue
|
||||
added, removed = await self._refresh_server(name)
|
||||
results[name] = (added, removed)
|
||||
except Exception:
|
||||
log.warning("Refresh failed for MCP server '%s'", name, exc_info=True)
|
||||
results[name] = ([], [])
|
||||
return results
|
||||
|
||||
def refresh_sync(
|
||||
self, server_name: str | None = None, timeout: int = 30
|
||||
) -> dict[str, tuple[list[str], list[str]]]:
|
||||
"""Refresh tools synchronously (blocks the calling thread).
|
||||
|
||||
Returns ``{server: (added_names, removed_names)}`` per server.
|
||||
"""
|
||||
assert self._loop is not None
|
||||
future = asyncio.run_coroutine_threadsafe(self._refresh_all(server_name), self._loop)
|
||||
return future.result(timeout=timeout)
|
||||
|
||||
async def _periodic_refresh(self) -> None:
|
||||
"""Periodically refresh servers that lack push notifications."""
|
||||
# Stagger start using a launch-time seed so cluster nodes don't
|
||||
# all hit MCP servers simultaneously.
|
||||
seed = random.Random(time.monotonic_ns() ^ os.getpid()).random()
|
||||
initial_delay = seed * self._refresh_interval
|
||||
await asyncio.sleep(initial_delay)
|
||||
while True:
|
||||
for name in list(self._server_configs):
|
||||
if self._supports_list_changed.get(name, False):
|
||||
continue # has push — skip
|
||||
if name not in self._sessions:
|
||||
continue # not connected — skip (reconnect on manual refresh)
|
||||
try:
|
||||
await self._refresh_server(name)
|
||||
except Exception:
|
||||
log.warning("Periodic refresh failed for '%s'", name, exc_info=True)
|
||||
await asyncio.sleep(self._refresh_interval)
|
||||
|
||||
# -- listener infrastructure ---------------------------------------------
|
||||
|
||||
def add_listener(self, callback: Callable[[], None]) -> None:
|
||||
"""Register a callback invoked when the tool list changes."""
|
||||
with self._listeners_lock:
|
||||
self._listeners.append(callback)
|
||||
|
||||
def remove_listener(self, callback: Callable[[], None]) -> None:
|
||||
"""Unregister a tool-change callback."""
|
||||
with self._listeners_lock, contextlib.suppress(ValueError):
|
||||
self._listeners.remove(callback)
|
||||
|
||||
def _notify_listeners(self) -> None:
|
||||
"""Invoke all registered listeners (runs on MCP background thread)."""
|
||||
with self._listeners_lock:
|
||||
listeners = list(self._listeners)
|
||||
for cb in listeners:
|
||||
try:
|
||||
cb()
|
||||
except Exception:
|
||||
log.warning("Tool-change listener raised", exc_info=True)
|
||||
|
||||
# -- lifecycle (shutdown) ------------------------------------------------
|
||||
|
||||
def shutdown(self) -> None:
|
||||
"""Close all MCP sessions and stop the background loop."""
|
||||
# Cancel periodic refresh
|
||||
if self._refresh_task and self._loop:
|
||||
self._loop.call_soon_threadsafe(self._refresh_task.cancel)
|
||||
|
||||
if self._loop and self._exit_stack:
|
||||
future = asyncio.run_coroutine_threadsafe(self._exit_stack.aclose(), self._loop)
|
||||
try:
|
||||
@@ -183,6 +387,11 @@ class MCPClientManager:
|
||||
def server_count(self) -> int:
|
||||
return len(self._sessions)
|
||||
|
||||
@property
|
||||
def server_names(self) -> list[str]:
|
||||
"""Return configured server names."""
|
||||
return list(self._server_configs.keys())
|
||||
|
||||
# -- tool invocation -----------------------------------------------------
|
||||
|
||||
def call_tool_sync(
|
||||
@@ -273,7 +482,11 @@ def load_mcp_config(config_path: str | None = None) -> dict[str, dict[str, Any]]
|
||||
return {}
|
||||
|
||||
|
||||
def create_mcp_client(config_path: str | None = None) -> MCPClientManager | None:
|
||||
def create_mcp_client(
|
||||
config_path: str | None = None,
|
||||
*,
|
||||
refresh_interval: float = _DEFAULT_REFRESH_INTERVAL,
|
||||
) -> MCPClientManager | None:
|
||||
"""Create and start an MCP client manager.
|
||||
|
||||
Returns *None* if no servers are configured.
|
||||
@@ -282,6 +495,6 @@ def create_mcp_client(config_path: str | None = None) -> MCPClientManager | None
|
||||
if not servers:
|
||||
return None
|
||||
|
||||
mgr = MCPClientManager(servers)
|
||||
mgr = MCPClientManager(servers, refresh_interval=refresh_interval)
|
||||
mgr.start()
|
||||
return mgr
|
||||
|
||||
+96
-110
@@ -21,22 +21,11 @@ def normalize_key(key: str) -> str:
|
||||
return key.lower().replace("-", "_").replace(" ", "_")
|
||||
|
||||
|
||||
# -- Core session operations ---------------------------------------------------
|
||||
|
||||
|
||||
def register_session(
|
||||
session_id: str,
|
||||
title: str | None = None,
|
||||
node_id: str | None = None,
|
||||
ws_id: str | None = None,
|
||||
) -> None:
|
||||
"""Create a sessions row for a new session (no-op if already exists)."""
|
||||
with contextlib.suppress(Exception):
|
||||
get_storage().register_session(session_id, title, node_id=node_id, ws_id=ws_id)
|
||||
# -- Core conversation operations ---------------------------------------------
|
||||
|
||||
|
||||
def save_message(
|
||||
session_id: str,
|
||||
ws_id: str,
|
||||
role: str,
|
||||
content: str | None,
|
||||
tool_name: str | None = None,
|
||||
@@ -47,111 +36,19 @@ def save_message(
|
||||
"""Log a message to the conversations table."""
|
||||
with contextlib.suppress(Exception):
|
||||
get_storage().save_message(
|
||||
session_id, role, content, tool_name, tool_args, tool_call_id, provider_data
|
||||
ws_id, role, content, tool_name, tool_args, tool_call_id, provider_data
|
||||
)
|
||||
|
||||
|
||||
def load_session_messages(session_id: str) -> list[dict[str, Any]]:
|
||||
"""Load messages for a session and reconstruct OpenAI message format."""
|
||||
def load_messages(ws_id: str) -> list[dict[str, Any]]:
|
||||
"""Load messages for a workstream and reconstruct OpenAI message format."""
|
||||
try:
|
||||
return get_storage().load_session_messages(session_id)
|
||||
return get_storage().load_messages(ws_id)
|
||||
except Exception:
|
||||
return []
|
||||
|
||||
|
||||
# -- Session management --------------------------------------------------------
|
||||
|
||||
|
||||
def list_sessions(limit: int = 20) -> list[Any]:
|
||||
"""List recent sessions with message counts."""
|
||||
try:
|
||||
return get_storage().list_sessions(limit)
|
||||
except Exception:
|
||||
return []
|
||||
|
||||
|
||||
def delete_session(session_id: str) -> bool:
|
||||
"""Delete a session and all its messages."""
|
||||
try:
|
||||
return get_storage().delete_session(session_id)
|
||||
except Exception:
|
||||
return False
|
||||
|
||||
|
||||
def prune_sessions(
|
||||
retention_days: int = 90,
|
||||
log_fn: Callable[[str], None] | None = None,
|
||||
) -> tuple[int, int]:
|
||||
"""Prune orphaned and stale sessions."""
|
||||
try:
|
||||
orphans, stale = get_storage().prune_sessions(retention_days)
|
||||
except Exception:
|
||||
return (0, 0)
|
||||
|
||||
if log_fn and (orphans or stale):
|
||||
parts = []
|
||||
if orphans:
|
||||
parts.append(f"{orphans} empty session{'s' if orphans != 1 else ''}")
|
||||
if stale:
|
||||
parts.append(
|
||||
f"{stale} session{'s' if stale != 1 else ''} older than {retention_days} days"
|
||||
)
|
||||
log_fn(f"[turnstone] Session cleanup: removed {', '.join(parts)}.")
|
||||
|
||||
return (orphans, stale)
|
||||
|
||||
|
||||
def resolve_session(alias_or_id: str) -> str | None:
|
||||
"""Resolve an alias or session_id (or prefix) to a full session_id."""
|
||||
try:
|
||||
return get_storage().resolve_session(alias_or_id)
|
||||
except Exception:
|
||||
return None
|
||||
|
||||
|
||||
# -- Session config ------------------------------------------------------------
|
||||
|
||||
|
||||
def save_session_config(session_id: str, config: dict[str, str]) -> None:
|
||||
"""Persist session configuration key/value pairs."""
|
||||
with contextlib.suppress(Exception):
|
||||
get_storage().save_session_config(session_id, config)
|
||||
|
||||
|
||||
def load_session_config(session_id: str) -> dict[str, str]:
|
||||
"""Load session configuration."""
|
||||
try:
|
||||
return get_storage().load_session_config(session_id)
|
||||
except Exception:
|
||||
return {}
|
||||
|
||||
|
||||
# -- Session metadata ----------------------------------------------------------
|
||||
|
||||
|
||||
def set_session_alias(session_id: str, alias: str) -> bool:
|
||||
"""Set a human-friendly alias. Returns False if alias is taken."""
|
||||
try:
|
||||
return get_storage().set_session_alias(session_id, alias)
|
||||
except Exception:
|
||||
return False
|
||||
|
||||
|
||||
def get_session_name(session_id: str) -> str | None:
|
||||
"""Return the alias (or title) for a session, or None if unset."""
|
||||
try:
|
||||
return get_storage().get_session_name(session_id)
|
||||
except Exception:
|
||||
return None
|
||||
|
||||
|
||||
def update_session_title(session_id: str, title: str) -> None:
|
||||
"""Set or update the auto-generated title for a session."""
|
||||
with contextlib.suppress(Exception):
|
||||
get_storage().update_session_title(session_id, title)
|
||||
|
||||
|
||||
# -- Workstream operations -----------------------------------------------------
|
||||
# -- Workstream management ----------------------------------------------------
|
||||
|
||||
|
||||
def register_workstream(
|
||||
@@ -182,6 +79,95 @@ def list_workstreams(node_id: str | None = None, limit: int = 100) -> list[Any]:
|
||||
return []
|
||||
|
||||
|
||||
def list_workstreams_with_history(limit: int = 20) -> list[Any]:
|
||||
"""List workstreams that have conversation messages."""
|
||||
try:
|
||||
return get_storage().list_workstreams_with_history(limit)
|
||||
except Exception:
|
||||
return []
|
||||
|
||||
|
||||
def delete_workstream(ws_id: str) -> bool:
|
||||
"""Delete a workstream and all its conversations + config."""
|
||||
try:
|
||||
return get_storage().delete_workstream(ws_id)
|
||||
except Exception:
|
||||
return False
|
||||
|
||||
|
||||
def prune_workstreams(
|
||||
retention_days: int = 90,
|
||||
log_fn: Callable[[str], None] | None = None,
|
||||
) -> tuple[int, int]:
|
||||
"""Prune orphaned and stale workstreams."""
|
||||
try:
|
||||
orphans, stale = get_storage().prune_workstreams(retention_days)
|
||||
except Exception:
|
||||
return (0, 0)
|
||||
|
||||
if log_fn and (orphans or stale):
|
||||
parts = []
|
||||
if orphans:
|
||||
parts.append(f"{orphans} empty workstream{'s' if orphans != 1 else ''}")
|
||||
if stale:
|
||||
parts.append(
|
||||
f"{stale} workstream{'s' if stale != 1 else ''} older than {retention_days} days"
|
||||
)
|
||||
log_fn(f"[turnstone] Cleanup: removed {', '.join(parts)}.")
|
||||
|
||||
return (orphans, stale)
|
||||
|
||||
|
||||
def resolve_workstream(alias_or_id: str) -> str | None:
|
||||
"""Resolve an alias or ws_id (or prefix) to a full ws_id."""
|
||||
try:
|
||||
return get_storage().resolve_workstream(alias_or_id)
|
||||
except Exception:
|
||||
return None
|
||||
|
||||
|
||||
# -- Workstream config --------------------------------------------------------
|
||||
|
||||
|
||||
def save_workstream_config(ws_id: str, config: dict[str, str]) -> None:
|
||||
"""Persist workstream configuration key/value pairs."""
|
||||
with contextlib.suppress(Exception):
|
||||
get_storage().save_workstream_config(ws_id, config)
|
||||
|
||||
|
||||
def load_workstream_config(ws_id: str) -> dict[str, str]:
|
||||
"""Load workstream configuration."""
|
||||
try:
|
||||
return get_storage().load_workstream_config(ws_id)
|
||||
except Exception:
|
||||
return {}
|
||||
|
||||
|
||||
# -- Workstream metadata ------------------------------------------------------
|
||||
|
||||
|
||||
def set_workstream_alias(ws_id: str, alias: str) -> bool:
|
||||
"""Set a human-friendly alias. Returns False if alias is taken."""
|
||||
try:
|
||||
return get_storage().set_workstream_alias(ws_id, alias)
|
||||
except Exception:
|
||||
return False
|
||||
|
||||
|
||||
def get_workstream_display_name(ws_id: str) -> str | None:
|
||||
"""Return the alias (or title) for a workstream, or None if unset."""
|
||||
try:
|
||||
return get_storage().get_workstream_display_name(ws_id)
|
||||
except Exception:
|
||||
return None
|
||||
|
||||
|
||||
def update_workstream_title(ws_id: str, title: str) -> None:
|
||||
"""Set or update the auto-generated title for a workstream."""
|
||||
with contextlib.suppress(Exception):
|
||||
get_storage().update_workstream_title(ws_id, title)
|
||||
|
||||
|
||||
# -- Key-value store (memories) ------------------------------------------------
|
||||
|
||||
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user