Compare commits

...

14 Commits

Author SHA1 Message Date
Patrick Buckley 41d1b27d34 Bump version to 0.5.3 2026-03-10 13:46:50 -07:00
Patrick Buckley 8bc284c60e fix: agent context overflow — truncate tool output, catch context errors
Agent tool outputs are now truncated to 16k chars to prevent search
results (14M+ chars observed) from blowing past the model's context
limit. On context-exceeded API errors, the agent returns its last
content instead of crashing.
2026-03-10 13:43:01 -07:00
Patrick Buckley a322d6b1d1 fix: sub-agent context — clean plan, merged task, no Qwen template error
Plan agent: own identity only (no base system prompt needed).
Task agent: base system prompt merged with task identity into a single
system message (needs tool patterns for tool execution).
Neither agent receives conversation history.

Fixes Jinja template error on Qwen models that reject system messages
appearing after non-system messages.
2026-03-10 13:36:23 -07:00
Patrick Buckley 70d495aa5b fix: per-workstream SSE fan-out — multiple consumers no longer steal … (#38)
* fix: per-workstream SSE fan-out — multiple consumers no longer steal each other's tokens

After 4d665a5 removed the single-consumer SSE lock, the shared
_event_queue let concurrent consumers (browser, bridge, console proxy)
race on Queue.get(), each receiving ~1/N of content tokens and producing
garbled streaming text.

Replace the single queue with per-client fan-out: each SSE connection
registers its own bounded queue (maxsize=500) on WebUI._listeners, and
_enqueue() copies every event to all registered queues. On eviction or
close, a ws_closed sentinel is injected so SSE generators exit promptly.

* fix: address CI failures and Copilot review feedback

- Handle ws_closed sentinel in events_sse generator (break on close)
- Guarantee sentinel delivery by evicting one item when queue is full
- Clear listeners list after injecting sentinels on cleanup
- Fix test_slow_consumer to fill only slow queue directly
- Fix ruff SIM117 (nested with), unused import, mypy unused-ignore
2026-03-10 13:25:16 -07:00
Patrick Buckley de64535221 Feat/eval improvements (#37)
* ci: add GitHub Release creation on tag push

* refactor: rename plan tool to create_plan

Rename plan → create_plan to resolve cross-provider tool selection
failures. Models consistently treated "plan" as a reasoning concept
rather than a callable tool. The new name is an unambiguous verb+noun
action. Also rename the parameter from prompt → goal for clarity,
add web_search to the default system prompt tool patterns

* feat: eval harness improvements inspired by autoresearch patterns

Major enhancements to turnstone-eval:

- Per-test timeout (--test-timeout, default 300s) and suite timeout
  (--suite-timeout) prevent stuck runs from blocking the suite
- Fast-fail skips remaining runs after ceil(n/2) consecutive zeros
- Summary table with colored PASS/WEAK/FAIL and append-only TSV output
- Progress reporting with running pass rate, token count, and ETA
- Parallel test execution via ProcessPoolExecutor (--parallel N)
- Per-role model assignment: test/optimizer/observer can use different
  models and providers (--optimizer-model, --observer-model, etc.)
  with auto-detection from base URL
- Improved optimizer and observer system prompts with structured
  failure-mode diagnosis, keep/discard rules, and trend analysis
- Fixed token counting (prompt tokens use last-turn value, not sum)
- Added math-calculation and web-search-query test cases
- Fixed multi-file-edit test (both files now contain the target string)

* fix: address Copilot review feedback

- Revert prompt token counting to sum (reflects billed usage)
- Add tool_args to fast-fail skipped run dicts for schema consistency
- Align approval_label with func_name ("create_plan")
- Add timeout to future.result() in parallel path (test_timeout + 30s)
- Document thread-leak trade-off on serial timeout path
2026-03-10 13:06:39 -07:00
Patrick Buckley 187d004033 feat: watch tool — periodic command polling within workstreams (#36)
* feat: watch tool — periodic command polling within workstreams

Add a new `watch` tool that lets the model (or user) set up periodic
polling of a shell command. Results inject as synthetic user messages
that trigger LLM turns, enabling reactive workflows like PR monitoring,
CI/CD status tracking, and deployment health checks.

Key design:
- Single tool with create/list/cancel actions
- Python expression DSL for stop conditions (restricted eval)
- Server-owned WatchRunner daemon (DB-persisted, survives eviction + restart)
- Three dispatch paths: idle, busy, and evicted workstream restore
- REST API for console visibility (GET /v1/api/watches, POST cancel)
- Migration 007, 8 storage CRUD methods, 75 new tests (1383 total)

* fix: address Copilot review — condition errors, restore deadlock, docs

- Condition eval errors now deactivate the watch immediately instead
  of silently looping until max_polls
- Restored (evicted) workstreams set auto_approve=True to prevent
  approval deadlocks with no connected user
- Tool description clarifies first-poll baseline behavior for change
  detection mode
- Diagram updated: DELETE → POST /v1/api/watches/{id}/cancel
2026-03-10 08:18:28 -07:00
Patrick Buckley 7ea150fa71 Bump version to 0.5.2 2026-03-09 13:42:28 -07:00
Patrick Buckley 4d665a5f62 fix: SSE reconnect loop — remove _sse_generation single-consumer lock
The _sse_generation mechanism assumed one SSE consumer per workstream,
but the bridge also maintains an SSE connection to each workstream.
When a new client connected (browser, proxy, or test), it incremented
the generation counter, killing the bridge's connection. The bridge
reconnected, killing the new client's connection — creating a
mutual-kill cascade that closed every SSE connection after one ping
cycle (5s).

Fix: remove _sse_generation entirely. sse-starlette handles disconnect
detection via its own ASGI task. Also remove the redundant
request.is_disconnected() check which raced with sse-starlette's
disconnect listener in Starlette 0.52.

Root cause confirmed via raw socket test: the server was sending
a zero-length chunked terminator (0\r\n\r\n) at exactly 5s,
cleanly ending the HTTP response body.
2026-03-09 13:40:36 -07:00
Patrick Buckley 3bc3250869 fix: recovered workstreams invisible in console UI (#35)
* fix: recovered workstreams invisible in console UI

Bridge startup recovery (_recover_workstreams) re-registered workstream
ownership but never published WorkstreamCreatedEvent to the cluster
channel. The collector's poll loop would pick up the workstream in its
internal state, but _apply_poll never fanned out SSE events to connected
browsers. Combined, this made channel-resumed workstreams invisible in
the console while remaining accessible through the proxied node UI.

- Bridge: emit WorkstreamCreatedEvent for each recovered workstream
- Collector: diff poll results and fan out synthetic ws_created/ws_closed
  events for workstream additions and removals
- Skip workstreams with empty IDs in poll processing
- Add 4 tests for poll-diff fanout behavior
- Update console data-flow diagram and architecture docs

* fix: address PR review — filter empty ws IDs, stable event ordering

- Filter empty-string keys from old_ids to avoid phantom ws_closed
  events if a previous poll inserted a workstream under key "".
- Sort set diffs before iterating so ws_created/ws_closed fanout
  order is deterministic across poll cycles.
2026-03-09 13:39:47 -07:00
Patrick Buckley db937486cf Bump version to 0.5.1 2026-03-09 01:48:18 -07:00
Patrick Buckley 554257ac4d fix: SSE proxy Firefox reconnect — Connection: keep-alive header 2026-03-09 01:46:35 -07:00
Patrick Buckley 5f0004dc91 feat: add ClusterSnapshot for instant console UI state rebuild (#34)
* feat: add ClusterSnapshot for instant console UI state rebuild

The console web UI was SSE-driven with no initial state — reloads and
navigation caused blank/loading gaps while waiting for API re-fetches.

Server-side: GET /v1/api/cluster/snapshot returns the full cluster state
(all nodes with workstreams + overview aggregates) built under a single
lock. The SSE stream now emits this snapshot as the first event on
connect (snapshot taken before listener registration to avoid race).

Frontend: local clusterState object mirrors the snapshot, patched
incrementally by SSE events. View navigation renders from local state
with no API round-trips. Fixes popstate/pushState history corruption
on Back/Forward navigation (pre-existing bug). Stable node sorting
with node_id tie-breaker on both server and client.

SDK: snapshot() method on Python (sync + async) and TypeScript console
clients. ClusterSnapshotEvent in event registries.

* fix: address review feedback and SSE proxy reconnect bug

Copilot review fixes:
- Atomic snapshot+register: new get_snapshot_and_register() acquires
  both state and listener locks, eliminating the event gap between
  snapshot read and listener registration.
- Debounce patch renders: patchClusterState uses requestAnimationFrame
  to batch rapid SSE events into a single recompute+render cycle.
- Fix health type: dict[str, str] → dict[str, Any] on all three
  console schema models (ClusterNodeInfo, NodeDetailResponse,
  ClusterSnapshotNode) since /health payloads contain nested objects.
- TypeScript ClusterSnapshotEvent: use concrete ClusterSnapshotNode[]
  and ClusterOverviewResponse types instead of Record<string, unknown>.

SSE proxy reconnect fix:
- _proxy_sse raw_stream now emits `: proxy-ping` comments every 3s
  when no upstream data arrives, preventing the browser EventSource
  from dropping idle connections. The raw byte passthrough refactor
  (4d11078) removed the proxy's independent keepalive — this restores
  it without reverting to EventSourceResponse.
2026-03-09 01:13:35 -07:00
Patrick Buckley 6cc1b3a5bd feat: add vision/image support to read_file tool (#33)
* feat: add vision/image support to read_file tool

read_file now detects image files (PNG, JPEG, GIF, WebP, BMP, TIFF, ICO)
and returns base64-encoded content parts for vision-capable models.
Non-vision models receive a text description instead. A new
supports_vision flag on ModelCapabilities gates the feature, with
config.toml [models.*.capabilities] overrides for local models
(vLLM, llama.cpp, NIM).

* fix: address PR review feedback

- Discard _read_files on no-vision OSError path, include exception detail
- Discard _read_files on oversized image error (not a successful read)
- Validate capabilities type from config.toml (reject non-dict)
- Clarify tool description re: vision behavior and offset/limit scope
- Remove unused os import in tests, fix import sort order
- Handle list content (image tool results) in eval.py tool result loop
2026-03-08 23:43:42 -07:00
Patrick Buckley cc9afe94cd get title in collector for console 2026-03-08 22:32:52 -07:00
62 changed files with 4838 additions and 340 deletions
+8
View File
@@ -5,6 +5,7 @@ on:
tags: ["v*"]
permissions:
contents: write
id-token: write
jobs:
@@ -19,3 +20,10 @@ jobs:
- run: pip install build
- run: python -m build
- uses: pypa/gh-action-pypi-publish@release/v1
- name: Create GitHub Release
uses: softprops/action-gh-release@v2
with:
generate_release_notes: true
draft: false
prerelease: ${{ contains(github.ref, '-') }}
+1 -1
View File
@@ -156,7 +156,7 @@ Bridges BLPOP from their per-node queue (priority) then the shared queue. Direct
| Tool | Description | Auto-approved |
|------|-------------|:---:|
| `bash` | Execute shell commands | |
| `read_file` | Read file contents | yes |
| `read_file` | Read file contents (text or images with vision models) | yes |
| `write_file` | Write/create files | |
| `edit_file` | Fuzzy-match file editing | |
| `search` | Search files by name/content | yes |
+79 -6
View File
@@ -460,13 +460,13 @@ The server sends an SSE comment every 5 seconds when no events are pending:
This prevents proxies and browsers from closing the connection due to
inactivity.
#### Generation mechanism
#### Multi-consumer fan-out
Each new SSE connection to a workstream increments an internal
`_sse_generation` counter. The previous SSE handler detects the generation
mismatch and exits its event loop, ensuring only one active SSE connection per
workstream at a time. The event queue is drained of stale events before the new
connection begins streaming.
Each SSE connection to a workstream receives its own delivery queue. Events
produced by the worker thread are fanned out to all registered listener queues,
so multiple consumers (browser, bridge, console proxy, SDK) can connect
simultaneously and each receives every event. On reconnect the client receives
a full history replay, so no catch-up mechanism is needed.
---
@@ -774,6 +774,79 @@ Status code: `400`
---
### `GET /v1/api/watches`
List active watches on this server node. Optionally filter by workstream.
Requires `write` scope.
**Query parameters:**
| Parameter | Type | Required | Description |
|-----------|--------|----------|------------------------------------|
| `ws_id` | string | no | Filter to watches for this workstream. If omitted, returns all watches on the node. |
**Response:**
```json
{
"watches": [
{
"watch_id": "abc123def456...",
"ws_id": "ws-1",
"node_id": "host_a1b2",
"name": "pr-review",
"command": "gh pr view --json state",
"interval_secs": 300.0,
"stop_on": "data[\"state\"] == \"MERGED\"",
"max_polls": 100,
"poll_count": 5,
"last_output": "{\"state\": \"OPEN\"}",
"last_poll": "2026-03-09T12:00:00",
"next_poll": "2026-03-09T12:05:00",
"active": 1,
"created": "2026-03-09T11:30:00"
}
]
}
```
---
### `POST /v1/api/watches/{watch_id}/cancel`
Cancel an active watch. Sets `active=0` and clears `next_poll`.
Requires `write` scope. Verifies node ownership in multi-node deployments.
**Path parameters:**
| Parameter | Type | Description |
|------------|--------|-----------------|
| `watch_id` | string | Watch ID to cancel |
**Response (success):**
```json
{"status": "ok", "watch_id": "abc123def456..."}
```
**Error (not found):**
```json
{"error": "Watch not found"}
```
Status code: `404`
**Error (wrong node):**
```json
{"error": "Watch belongs to another node"}
```
Status code: `403`
---
### `OPTIONS` (any path)
Handles CORS preflight requests.
+28 -7
View File
@@ -44,6 +44,7 @@ turnstone/
tools.py Tool schema loader (JSON -> OpenAI function-calling format)
mcp_client.py MCPClientManager — MCP server connections, tool discovery, dynamic refresh, async-sync bridge
tool_search.py Dynamic tool search — BM25 index, session-scoped tool visibility
watch.py WatchRunner daemon — periodic command polling, condition DSL, result dispatch
model_registry.py ModelRegistry — named model configs, lazy client creation, fallback routing
memory.py Persistence facade (delegates to storage backend)
storage/ Pluggable storage: StorageBackend protocol, SQLite + PostgreSQL
@@ -560,21 +561,23 @@ LLMProvider (protocol)
|------|--------|
| `StreamChunk` | `content_delta`, `reasoning_delta`, `tool_call_deltas`, `info_delta`, `usage`, `finish_reason` |
| `CompletionResult` | `content`, `tool_calls`, `finish_reason`, `usage` |
| `ModelCapabilities` | `context_window`, `max_output_tokens`, `supports_temperature`, `token_param`, `thinking_mode`, `supports_effort`, `supports_web_search` |
| `ModelCapabilities` | `context_window`, `max_output_tokens`, `supports_temperature`, `token_param`, `thinking_mode`, `supports_effort`, `supports_web_search`, `supports_tool_search`, `supports_vision` |
| `UsageInfo` | `prompt_tokens`, `completion_tokens`, `total_tokens` |
**OpenAIProvider** (`_openai.py`): passes messages through unchanged (they are
already in OpenAI format). Model capability lookup table covers
GPT-5/5.1/5.2, O-series, and search models (`gpt-5-search-api`).
already in OpenAI format), including multi-part content blocks (text + images)
in tool results. Model capability lookup table covers GPT-5/5.1/5.2/5.3/5.4,
O-series, and search models (`gpt-5-search-api`) — all with `supports_vision`.
For search models, injects `web_search_options` and removes the `web_search`
function tool (the model always searches). Citations from `url_citation`
annotations are formatted as footnotes. Unknown models (local servers) get
permissive defaults and use Tavily for web search.
permissive defaults with `supports_vision=False` and use Tavily for web search.
**AnthropicProvider** (`_anthropic.py`): converts OpenAI-format messages to
Anthropic content blocks, maps `system`/`developer` roles to the `system`
parameter, groups consecutive `tool` result messages into user-role content
blocks, and translates tool schemas from OpenAI function-calling format to
blocks (converting `image_url` parts to Anthropic's `image` source format),
and translates tool schemas from OpenAI function-calling format to
Anthropic's `input_schema` format. Supports both manual and adaptive thinking
modes, with effort parameter support for models like Claude Opus 4.6 and
Sonnet 4.6. Replaces the `web_search` function tool with Anthropic's native
@@ -620,6 +623,18 @@ agent_model = "claude"
Each `[models.*]` entry produces a `ModelConfig` with a `provider` field
(default: `"openai"`). Supported values: `"openai"` and `"anthropic"`.
An optional `[models.*.capabilities]` sub-table overrides per-model
`ModelCapabilities` flags (useful for local models whose capabilities
cannot be detected programmatically):
```toml
[models.qwen-vl]
base_url = "http://localhost:8000/v1"
model = "qwen-3.5-vl"
[models.qwen-vl.capabilities]
supports_vision = true
```
**Lifecycle:**
1. `load_model_registry()` reads `[models.*]` sections from config.toml and
@@ -1098,7 +1113,7 @@ context manager handles startup/shutdown (health monitor, MCP client,
registry).
Each workstream's `WebUI` has:
- `_event_queue` (per-workstream SSE events, `queue.Queue`)
- `_listeners` (per-client SSE queues, fan-out on `_enqueue()`)
- `_approval_event` / `_plan_event` (`threading.Event` for blocking)
- `_global_queue` (class variable, shared, for state broadcasts)
@@ -1172,6 +1187,9 @@ for existing workstreams are auto-routed via `turnstone:ws:{ws_id}` ownership ke
If a bridge picks up a shared-queue message for a workstream owned by another node, it
re-routes to that node's queue (1 extra hop). Bridges publish heartbeats to
`turnstone:node:{node_id}` with configurable TTL for node discovery.
On startup, `_recover_workstreams` re-registers ownership of existing
workstreams and publishes `WorkstreamCreatedEvent` to the cluster channel
so the console collector picks them up immediately.
### Cluster Console
@@ -1197,7 +1215,10 @@ The console HTTP layer is a Starlette/ASGI app served by uvicorn. The SSE
endpoint uses `EventSourceResponse` with the same listener queue pattern as
the main server. `ClusterCollector`'s background threads (event subscriber,
node discovery, poll loop) use sync Redis clients and `ThreadPoolExecutor`
for parallel HTTP polling.
for parallel HTTP polling. The poll loop diffs workstream IDs between poll
cycles and fans out synthetic `ws_created`/`ws_closed` SSE events for any
changes, ensuring browser clients stay in sync even when real-time cluster
events are missed (e.g. bridge startup recovery).
The console has two write-path capabilities:
+37 -1
View File
@@ -61,6 +61,8 @@ The collector (`turnstone/console/collector.py`) maintains an in-memory snapshot
3. **Poll loop** — fetches `GET /v1/api/dashboard` and `GET /health` from each known node every 10 seconds. Uses `ThreadPoolExecutor(max_workers=50)` for parallelism. Each poll replaces the node's workstream list with the authoritative server data.
A `get_snapshot()` method builds the full cluster state under a single lock acquisition — overview aggregates and per-node workstream lists in one atomic read. This is served both as a REST endpoint and as the initial SSE event on client connect.
### Thread Safety
All reads and writes to the node/workstream map are protected by a single `threading.Lock`. Query methods acquire the lock, copy data, and release before returning.
@@ -146,6 +148,38 @@ Single node detail with all its workstreams.
}
```
### `GET /v1/api/cluster/snapshot`
Full cluster state in a single response — all nodes with their workstreams plus overview aggregates. Built under a single lock for internal consistency. Used by the browser on initial load and SSE reconnect.
```json
{
"nodes": [
{
"node_id": "db-west-04",
"server_url": "http://10.0.3.4:8080",
"max_ws": 10,
"reachable": true,
"version": "0.3.0",
"health": {"status": "ok", "version": "0.3.0"},
"aggregate": {"total_tokens": 48200, "total_tool_calls": 156},
"workstreams": [
{"id": "a1b2c3d4", "name": "perf-db-west", "state": "running", ...}
]
}
],
"overview": {
"nodes": 847,
"workstreams": 4219,
"states": {"running": 1847, "thinking": 312, "attention": 89, "idle": 1940, "error": 31},
"aggregate": {"total_tokens": 12400000, "total_tool_calls": 34200},
"version_drift": false,
"versions": ["0.3.0"]
},
"timestamp": 1709294400.0
}
```
### `POST /v1/api/cluster/workstreams/new`
Create a new workstream on a target node. Dispatches a `CreateWorkstreamMessage` through the Redis MQ pipeline — the bridge on the target node picks it up and creates the workstream on the server. Requires `write` scope.
@@ -182,7 +216,7 @@ Creation is asynchronous — the response confirms the MQ message was dispatched
### `GET /v1/api/cluster/events`
Server-Sent Events stream for real-time cluster updates.
Server-Sent Events stream for real-time cluster updates. The first event is always a `snapshot` containing the full cluster state (same shape as `GET /v1/api/cluster/snapshot` with an added `type: "snapshot"` field), followed by incremental events:
```
data: {"type":"cluster_state","ws_id":"a1b2","node_id":"db-west-04","state":"running"}
@@ -368,6 +402,8 @@ On submit, `POST /v1/api/cluster/workstreams/new` dispatches the creation reques
All five views receive live updates via SSE — state cards update counts, node rows update metrics, workstream rows update state indicators.
The browser maintains a local `clusterState` object that mirrors the cluster snapshot. It is initialized from the SSE `snapshot` event on connect (or via `GET /v1/api/cluster/snapshot` on initial page load) and updated incrementally by SSE events. View navigation reads from local state — no API round-trips needed after the initial snapshot.
### 5. Admin Panel
Accessed via the "admin" button in the header (visible when authenticated
+2 -1
View File
@@ -41,7 +41,7 @@ class "WorkstreamTerminalUI" as WsTermUI {
}
class "WebUI" as WebUI {
- _event_queue: Queue
- _listeners: list[Queue]
- _approval_event: Event
- _plan_event: Event
- _ws_prompt_tokens: int
@@ -109,6 +109,7 @@ class "ModelCapabilities" as ModelCaps <<frozen>> {
+ supports_effort: bool
+ supports_web_search: bool
+ supports_tool_search: bool
+ supports_vision: bool
}
' ChatSession
+1 -1
View File
@@ -112,7 +112,7 @@ group loop [while tool_calls present]
note right of TP
Parallel execution:
bash → Popen + line-by-line streaming
read_file → open().read()
read_file → open().read() or base64 image
search → grep subprocess
edit_file → string replace
task/plan → _run_agent() sub-loop
+1 -1
View File
@@ -100,7 +100,7 @@ partition "Phase 3: Execute" #E3F2FD {
if item.denied → return denial message
else → item["execute"](item)
├─ _exec_bash: subprocess.run(["bash", script.sh])
├─ _exec_read_file: open().readlines()
├─ _exec_read_file: open().readlines() or _exec_read_image (base64)
├─ _exec_write_file: makedirs + write
├─ _exec_edit_file: find_occurrences + replace
├─ _exec_search: grep subprocess
+25 -1
View File
@@ -78,7 +78,19 @@ activate NodeA
NodeA --> CC : {status:"ok", version:"0.3.0",\nmodel:"...", workstreams:{...}}
deactivate NodeA
CC -> CC : Diff old vs new workstream IDs
CC -> CC : Replace NodeSnapshot["nodeA"]\n.workstreams, .health, .aggregate
CC -> CC : _fanout(ws_created) for\nnewly appeared workstreams
CC -> CC : _fanout(ws_closed) for\nremoved workstreams
note right of CC
Poll-diff fanout ensures
browser SSE clients learn
about workstreams that
appeared without a real-time
cluster event (e.g. bridge
startup recovery).
end note
CC -x NodeB : (SKIPPED: sim:// URL)
@@ -89,10 +101,15 @@ deactivate CC
Browser -> Server : GET /v1/api/cluster/events
activate Server
Server -> CC : get_snapshot()
CC --> Server : ClusterSnapshot\n(full current state)
Server -> CC : register_listener(queue)
note right : Per-client queue.Queue(maxsize=500)\nSSE via EventSourceResponse + run_in_executor()
loop continuous
Server -> Browser : data: {"type":"snapshot",...}\n(full state as first SSE event)
loop continuous (incremental updates)
CC -> Server : event via listener queue\n(from any of the 3 threads)
Server -> Browser : data: {"type":"cluster_state",...}\n\n
end
@@ -105,6 +122,13 @@ Browser -> Server : connection closed
Server -> CC : unregister_listener(queue)
deactivate Server
== Browser REST: Snapshot ==
Browser -> Server : GET /v1/api/cluster/snapshot
Server -> CC : get_snapshot()
CC --> Server : ClusterSnapshot\n(full current state)
Server --> Browser : JSON response
== Browser REST Requests ==
Browser -> Server : GET /v1/api/cluster/overview
+2
View File
@@ -45,6 +45,7 @@ package "turnstone/sdk/ (Python)" {
+ nodes()
+ workstreams()
+ node_detail()
+ snapshot()
+ create_workstream()
+ stream_cluster_events()
+ login() / logout()
@@ -129,6 +130,7 @@ package "sdk/typescript/ (TypeScript)" {
class "TurnstoneConsole" as TSConsole <<ts>> {
+ overview()
+ nodes()
+ snapshot()
+ clusterEvents()
...
}
+166
View File
@@ -0,0 +1,166 @@
@startuml
!theme plain
title Turnstone — Watch Tool Architecture
skinparam participant {
BackgroundColor<<server>> #FFE0B2
BackgroundColor<<storage>> #B3E5FC
BackgroundColor<<session>> #C8E6C9
BackgroundColor<<ui>> #E8EAF6
}
participant "ChatSession\n(session.py)" as Session <<session>>
participant "WatchRunner\n(watch.py)" as Runner <<server>>
participant "StorageBackend\n(SQLite)" as Storage <<storage>>
participant "WebUI / SSE\n(server.py)" as UI <<ui>>
== Create Phase ==
Session -> Session : _prepare_watch(action="create")
note right
Validates:
- command via is_command_blocked()
- poll_every → parse_duration()
- stop_on → validate_condition()
- max watches limit (5)
- duplicate name check
needs_approval = True
end note
Session -> Storage : create_watch(watch_id, ws_id,\nnode_id, command, interval,\nstop_on, max_polls, next_poll)
Session --> UI : tool_result:\n"Watch 'pr-review' created"
== Poll Phase (WatchRunner daemon, every 15s) ==
Runner -> Storage : list_due_watches(now)
Storage --> Runner : due_watches[]
note right
Filters:
active=1 AND
next_poll <= now AND
node_id matches
end note
loop for each due watch
Runner -> Runner : is_command_blocked()?
alt blocked
Runner -> Storage : update_watch(active=False)
else safe
Runner -> Runner : subprocess.run(command)
note right
timeout = tool_timeout
start_new_session = True
output truncated at 64KB
end note
Runner -> Runner : evaluate_condition(\nstop_on, output,\nexit_code, prev_output)
note right
**Variables:**
output, data, exit_code,
prev_output, changed
**Safe builtins only:**
len, str, int, sorted, ...
No import/open/exec/eval
**stop_on=None:**
fires on change (skip 1st poll)
end note
alt condition fired OR max_polls reached
Runner -> Storage : update_watch(\npoll_count++,\nlast_output, active=False)
Runner -> Runner : format_watch_message()
Runner -> Runner : _dispatch_result(ws_id, msg)
else not fired
Runner -> Storage : update_watch(\npoll_count++,\nlast_output, next_poll)
end
end
end
== Dispatch Phase ==
note over Runner, Session
**Three dispatch paths:**
end note
alt Path A: workstream active + idle
Runner -> Session : dispatch_fn(message)\n→ _watch_pending.put()
Session -> Session : _dispatch_pending_watch()\n→ self.send(message)
Session -> UI : SSE: thinking, content,\ntool calls...
note right
Watch result appears as
synthetic user message.
Model sees it and responds.
Depth guard: max 5 chains.
end note
else Path B: workstream active + busy
Runner -> Session : dispatch_fn(message)\n→ _watch_pending.put()
note right
Queued. Dispatched when
current send() reaches IDLE.
end note
else Path C: workstream evicted
Runner -> Runner : restore_fn(ws_id)
note right
1. mgr.create() — may evict
another idle workstream
2. session.resume(ws_id)
3. set_watch_runner()
4. register new dispatch_fn
end note
Runner -> Session : restored dispatch_fn(message)
end
== Cancel / List ==
Session -> Storage : list_watches_for_ws(ws_id)
note right : action="list" (auto-approve)
Session -> Storage : update_watch(active=False)
note right : action="cancel" (auto-approve)
== Server Lifecycle ==
note over Runner, Storage
**Startup:**
1. WatchRunner created in main() with storage + node_id
2. restore_fn closure captures WorkstreamManager
3. Initial workstream: session.set_watch_runner(runner)
4. _lifespan(): runner.start() — daemon thread begins
**New workstream:**
session.set_watch_runner(runner) in create_workstream()
→ registers dispatch_fn for ws_id
**Eviction / close:**
session.close() → runner.remove_dispatch_fn(ws_id)
Watches remain active in DB — WatchRunner uses restore_fn
**Restart recovery:**
Overdue watches fire ONE immediate poll
next_poll updated to now + interval
Normal cadence resumes
**Shutdown:**
_lifespan(): runner.stop() — joins thread
end note
== REST API ==
note over UI, Storage
**GET /v1/api/watches[?ws_id=X]**
List active watches (for node or workstream)
**POST /v1/api/watches/{watch_id}/cancel**
Cancel a watch (sets active=False)
Both require write scope
end note
@enduml
+2 -2
View File
@@ -1,3 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:4512be48a51f7cd1136ea8e3344489a8d225c611cb1b961643b869351de76812
size 549668
oid sha256:760c37e67736588dadee21d500419a48e9fc50f8bdc5667e686c580022bd40e2
size 554869
+2 -2
View File
@@ -1,3 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:24bdc6a83259e4db6aaa24f581bed59d351c7a83e1c282aac123c52b32d9f80d
size 288250
oid sha256:e3044c738d6d6853aab5c4990e6c67bab0165eba991a4f5bebdfc4d4a0b305ee
size 289165
+2 -2
View File
@@ -1,3 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:027ad99469d69f1d6b2e73ee802b50a75617cd286392375aa69133c13d3683dc
size 256347
oid sha256:282820fe416961e735d050f86ecdc079e29824d2b3c4d5c8c174d0533d41f211
size 258045
@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:96176a09e65e90dadc32d5e9ed778423842be89204d2cf382225f53a90cfaf01
size 258547
+4
View File
@@ -95,6 +95,7 @@ Both `TurnstoneConsole` (sync) and `AsyncTurnstoneConsole` (async) expose:
| | `nodes(*, sort, limit, offset)` | `ClusterNodesResponse` |
| | `workstreams(*, state, node, search, sort, page, per_page)` | `ClusterWorkstreamsResponse` |
| | `node_detail(node_id)` | `NodeDetailResponse` |
| | `snapshot()` | `ClusterSnapshotResponse` |
| | `create_workstream(*, node_id, name, model, initial_message)` | `ConsoleCreateWsResponse` |
| **Schedules** | `list_schedules()` | `ListSchedulesResponse` |
| | `create_schedule(*, name, schedule_type, initial_message, ...)` | `ScheduleInfo` |
@@ -146,6 +147,9 @@ SSE events are deserialized into typed dataclasses. Use `event.type` to discrimi
| `node_lost` | `NodeLostEvent` | `node_id` |
| `cluster_state` | `ClusterStateEvent` | `ws_id`, `node_id`, `state`, `tokens` |
| `ws_created` | `ClusterWsCreatedEvent` | `ws_id`, `node_id`, `name` |
| `ws_closed` | `ClusterWsClosedEvent` | `ws_id` |
| `ws_rename` | `ClusterWsRenameEvent` | `ws_id`, `name` |
| `snapshot` | `ClusterSnapshotEvent` | `nodes`, `overview`, `timestamp` |
### TurnResult
+85 -7
View File
@@ -1,6 +1,6 @@
# Tools Reference
turnstone exposes 15 built-in tools plus any number of external MCP tools to the
turnstone exposes 16 built-in tools plus any number of external MCP tools to the
LLM via the OpenAI function-calling interface. Built-in tools are defined as JSON
files under `turnstone/tools/` and loaded at startup by `turnstone/core/tools.py`.
MCP tools are discovered from configured MCP servers at startup by
@@ -46,12 +46,12 @@ schema plus turnstone-specific metadata keys:
| Name | Description |
|---------------------|-------------|
| `TOOLS` | All 15 tool definitions (sent to the model). |
| `TOOLS` | All 16 tool definitions (sent to the model). |
| `AGENT_TOOLS` | Tools with `agent: true` -- available to plan sub-agents. Read-only tools. |
| `TASK_AGENT_TOOLS` | Tools with `task_agent: true` -- available to task sub-agents. Includes write operations. |
| `AGENT_AUTO_TOOLS` | Set of tool names with `auto_approve: true` -- no user confirmation needed. |
| `TASK_AUTO_TOOLS` | Same as `AGENT_AUTO_TOOLS` (identical filter). |
| `BUILTIN_TOOL_NAMES`| Frozenset of all 15 built-in tool names. Used by tool search to distinguish always-on tools from deferrable MCP tools. |
| `BUILTIN_TOOL_NAMES`| Frozenset of all 16 built-in tool names. Used by tool search to distinguish always-on tools from deferrable MCP tools. |
| `PRIMARY_KEY_MAP` | Dict mapping tool name to its `primary_key` parameter name. |
---
@@ -189,15 +189,17 @@ Execute a bash command and return stdout + stderr.
### read_file
Read the contents of a file, returning numbered lines.
Read the contents of a file, returning numbered lines for text files or
base64-encoded image data for supported image formats.
| Parameter | Type | Required | Description |
|-----------|---------|----------|-------------|
| `path` | string | yes | Absolute or relative file path. |
| `offset` | integer | no | Line number to start from (1-based, default: 1). |
| `limit` | integer | no | Maximum number of lines to read. Omit for full file. |
| `offset` | integer | no | Line number to start from (1-based, default: 1). Text files only. |
| `limit` | integer | no | Maximum number of lines to read. Omit for full file. Text files only. |
- **What it does**: Reads the file and returns content with line numbers. Must be called before `edit_file` on the same path (the session tracks which files have been read).
- **What it does**: For text files, reads and returns content with line numbers. For image files (PNG, JPEG, GIF, WebP, BMP, TIFF, ICO), returns image data as multi-part content when the model supports vision, or a text description when it does not. SVG files are read as text. Images larger than 4 MB are rejected. Must be called before `edit_file` on the same path (the session tracks which files have been read).
- **Vision support**: Controlled by `ModelCapabilities.supports_vision`. All commercial OpenAI and Anthropic models have vision enabled. Local models (vLLM, llama.cpp, NIM) default to off — enable via `[models.*.capabilities] supports_vision = true` in config.toml.
- **Auto-approve**: Yes.
- **Agent availability**: `agent` and `task_agent`.
@@ -420,6 +422,81 @@ Provide either `username` for user-based targeting or `channel_type` +
---
### watch
Set up periodic polling of a shell command within the current workstream.
Results are injected back into the conversation as synthetic user messages,
triggering the model to respond and act. Use for monitoring CI/CD pipelines,
PR reviews, deployments, file changes, etc.
| Parameter | Type | Required | Description |
|-------------|---------|----------|-------------|
| `action` | string | yes | `create`, `list`, or `cancel`. |
| `command` | string | create | Shell command to poll periodically. |
| `poll_every`| string | no | Poll interval as duration (`30s`, `5m`, `1h`). Default: `5m`. |
| `stop_on` | string | no | Python expression for stop condition (see below). Omit for change detection. |
| `name` | string | create | Human-readable watch name (e.g. `pr-review`). Used as identifier for cancel. |
| `max_polls` | integer | no | Max poll cycles before auto-cancel. Default: 100. |
**Actions:**
- `create` — Start a new watch. Requires approval (same as bash — runs shell
commands). Persists to the `watches` table; the server-level `WatchRunner`
daemon polls every 15 seconds for due watches.
- `list` — Show all active watches in this workstream. Auto-approved.
- `cancel` — Stop a watch by name or ID prefix. Auto-approved.
**Stop condition DSL** — The `stop_on` parameter accepts a Python expression
evaluated after each poll. Available variables:
| Variable | Type | Description |
|---------------|------------|-------------|
| `output` | `str` | stdout (+stderr) of the command. |
| `data` | `Any` | `json.loads(output)`, or `None` if not valid JSON. |
| `exit_code` | `int` | Process exit code. |
| `prev_output` | `str|None` | Previous poll's stdout (`None` on first poll). |
| `changed` | `bool` | `True` if output differs from previous poll. |
Safe builtins: `len`, `str`, `int`, `float`, `bool`, `abs`, `min`, `max`,
`any`, `all`, `isinstance`, `sorted`. No `import`, `open`, `exec`, or
`eval`. Security model: equivalent to `bash` — the model already has shell
access.
**Examples:**
```
data["state"] == "MERGED"
"error" in output
exit_code != 0
changed and "ready" in output.lower()
data.get("mergedAt") is not None
```
**Lifecycle:**
1. Model calls `watch(action="create", ...)` — persisted to SQLite.
2. `WatchRunner` daemon polls for due watches every 15s.
3. Each poll runs the command, evaluates the condition.
4. When the condition fires (or max polls reached), the result is injected
as a synthetic user message and the watch auto-cancels.
5. If the workstream was evicted, it is restored before injection.
6. Watches survive server restart (overdue watches fire once on recovery).
**Constraints:**
- Max 5 active watches per workstream.
- Poll interval: 10s24h.
- Output truncated at 64 KB.
- Max 5 consecutive watch dispatches per worker thread (depth guard).
- Duplicate names rejected within the same workstream.
- **Auto-approve**: `create` requires approval; `list` and `cancel` are auto-approved.
- **Agent availability**: Main session only — not available to plan/task sub-agents.
> See [Watch Architecture](diagrams/png/18-watch-architecture.png) for the
> full poll → evaluate → dispatch flow.
---
## Summary Table
| Tool | Category | Auto-approve | agent | task_agent | primary_key |
@@ -439,6 +516,7 @@ Provide either `username` for user-based targeting or `channel_type` +
| `recall` | Memory | Yes | No | No | `query` |
| `forget` | Memory | Yes | No | No | `key` |
| `notify` | Notify | Yes | Yes | Yes | `message` |
| `watch` | Monitor | No (create) | No | No | `command` |
| `tool_search`| Search | Yes | No | No | `query` |
---
+1 -1
View File
@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
[project]
name = "turnstone"
version = "0.5.0"
version = "0.5.3"
description = "Multi-node AI orchestration platform with tool use, agent routing, and cluster simulation."
readme = "README.md"
license = "BUSL-1.1"
+5
View File
@@ -6,6 +6,7 @@ import type {
AuthStatusResponse,
ClusterNodesResponse,
ClusterOverviewResponse,
ClusterSnapshotResponse,
ClusterWorkstreamsResponse,
ConsoleCreateWsRequest,
ConsoleCreateWsResponse,
@@ -33,6 +34,10 @@ export class TurnstoneConsole extends BaseClient {
return this.request("GET", "/v1/api/cluster/overview");
}
async snapshot(): Promise<ClusterSnapshotResponse> {
return this.request("GET", "/v1/api/cluster/snapshot");
}
async nodes(opts?: NodesOptions): Promise<ClusterNodesResponse> {
return this.request("GET", "/v1/api/cluster/nodes", {
params: {
+11 -1
View File
@@ -1,3 +1,5 @@
import type { ClusterOverviewResponse, ClusterSnapshotNode } from "./types.js";
// ---------------------------------------------------------------------------
// Server SSE events
// ---------------------------------------------------------------------------
@@ -191,6 +193,13 @@ export interface ClusterWsRenameEvent {
name: string;
}
export interface ClusterSnapshotEvent {
type: "snapshot";
nodes: ClusterSnapshotNode[];
overview: ClusterOverviewResponse;
timestamp: number;
}
/** Discriminated union of all console cluster SSE event types. */
export type ClusterEvent =
| NodeJoinedEvent
@@ -198,7 +207,8 @@ export type ClusterEvent =
| ClusterStateEvent
| ClusterWsCreatedEvent
| ClusterWsClosedEvent
| ClusterWsRenameEvent;
| ClusterWsRenameEvent
| ClusterSnapshotEvent;
// ---------------------------------------------------------------------------
// Type guards
+3
View File
@@ -55,6 +55,7 @@ export type {
ClusterWsCreatedEvent,
ClusterWsClosedEvent,
ClusterWsRenameEvent,
ClusterSnapshotEvent,
} from "./events.js";
export {
@@ -97,6 +98,8 @@ export type {
ClusterOverviewResponse,
ClusterNodeInfo,
ClusterNodesResponse,
ClusterSnapshotNode,
ClusterSnapshotResponse,
ClusterWorkstreamInfo,
ClusterWorkstreamsResponse,
NodeDetailResponse,
+17
View File
@@ -244,6 +244,23 @@ export interface NodeDetailResponse {
aggregate: ClusterAggregate;
}
export interface ClusterSnapshotNode {
node_id: string;
server_url: string;
max_ws: number;
reachable: boolean;
version: string;
health: Record<string, string>;
aggregate: Record<string, number>;
workstreams: ClusterWorkstreamInfo[];
}
export interface ClusterSnapshotResponse {
nodes: ClusterSnapshotNode[];
overview: ClusterOverviewResponse;
timestamp: number;
}
export interface ConsoleCreateWsRequest {
node_id?: string;
name?: string;
+20 -2
View File
@@ -59,7 +59,7 @@
"user_prompt": "Change the default port from 8000 to 9000 in both server.py and config.py",
"setup": {
"files": {
"server.py": "from config import PORT\n\ndef run():\n print(f'Listening on port {PORT}')\n",
"server.py": "import socket\n\ndef run():\n sock = socket.socket()\n sock.bind(('localhost', 8000))\n print('Server running on port 8000')\n",
"config.py": "PORT = 8000\nHOST = 'localhost'\n"
}
},
@@ -126,7 +126,7 @@
"app.py": "import sqlite3\nfrom flask import Flask, jsonify\n\napp = Flask(__name__)\nDB = 'data.db'\n\ndef get_db():\n return sqlite3.connect(DB)\n\n@app.route('/users')\ndef list_users():\n db = get_db()\n users = db.execute('SELECT * FROM users').fetchall()\n db.close()\n return jsonify(users)\n\n@app.route('/users/<int:uid>')\ndef get_user(uid):\n db = get_db()\n user = db.execute('SELECT * FROM users WHERE id=?', (uid,)).fetchone()\n db.close()\n return jsonify(user)\n\nif __name__ == '__main__':\n app.run(port=8000)\n"
}
},
"expected_actions": [{ "tool": "plan" }],
"expected_actions": [{ "tool": "create_plan" }],
"match_mode": "subset"
},
{
@@ -175,6 +175,24 @@
{ "tool": "man", "args_pattern": { "page": "tar" } }
],
"match_mode": "subset"
},
{
"id": "math-calculation",
"description": "Use the math tool for precise calculations, not bash or mental math",
"user_prompt": "What is 2^64 - 1? Use the math tool to calculate it precisely.",
"expected_actions": [
{ "tool": "math", "args_pattern": { "code": "2.*64" } }
],
"match_mode": "subset"
},
{
"id": "web-search-query",
"description": "Use web_search for general knowledge lookups, not web_fetch",
"user_prompt": "Search the web for the current population of Tokyo",
"expected_actions": [
{ "tool": "web_search", "args_pattern": { "query": "Tokyo" } }
],
"match_mode": "subset"
}
]
}
+135
View File
@@ -202,6 +202,68 @@ class TestCollectorPolling:
# Should not raise
c._apply_poll("unknown", _dashboard_response(), {})
def test_apply_poll_emits_ws_created_for_new_workstream(self):
c = _make_collector()
c._nodes["node-a"] = NodeSnapshot(node_id="node-a", server_url="http://a:8080")
q: queue.Queue[dict] = queue.Queue()
c.register_listener(q)
dashboard = _dashboard_response(
workstreams=[{"id": "ws1", "name": "new-task", "state": "idle"}]
)
c._apply_poll("node-a", dashboard, {})
event = q.get_nowait()
assert event["type"] == "ws_created"
assert event["ws_id"] == "ws1"
assert event["name"] == "new-task"
assert event["node_id"] == "node-a"
def test_apply_poll_emits_ws_closed_for_removed_workstream(self):
c = _make_collector()
c._nodes["node-a"] = NodeSnapshot(
node_id="node-a",
server_url="http://a:8080",
workstreams={"ws1": {"id": "ws1", "name": "old", "state": "idle"}},
)
q: queue.Queue[dict] = queue.Queue()
c.register_listener(q)
c._apply_poll("node-a", _dashboard_response(), {})
event = q.get_nowait()
assert event["type"] == "ws_closed"
assert event["ws_id"] == "ws1"
def test_apply_poll_no_events_when_unchanged(self):
c = _make_collector()
c._nodes["node-a"] = NodeSnapshot(
node_id="node-a",
server_url="http://a:8080",
workstreams={"ws1": {"id": "ws1", "name": "same", "state": "idle"}},
)
q: queue.Queue[dict] = queue.Queue()
c.register_listener(q)
dashboard = _dashboard_response(
workstreams=[{"id": "ws1", "name": "same", "state": "running"}]
)
c._apply_poll("node-a", dashboard, {})
assert q.empty()
def test_apply_poll_skips_empty_id_workstream(self):
c = _make_collector()
c._nodes["node-a"] = NodeSnapshot(node_id="node-a", server_url="http://a:8080")
q: queue.Queue[dict] = queue.Queue()
c.register_listener(q)
dashboard = _dashboard_response(workstreams=[{"name": "no-id", "state": "idle"}])
c._apply_poll("node-a", dashboard, {})
assert q.empty()
assert len(c._nodes["node-a"].workstreams) == 0
class TestCollectorEvents:
"""Real-time event handling from cluster channel."""
@@ -446,6 +508,44 @@ class TestCollectorQueries:
def test_get_node_detail_not_found(self, populated_collector):
assert populated_collector.get_node_detail("nonexistent") is None
def test_get_snapshot_empty(self):
c = _make_collector()
snap = c.get_snapshot()
assert snap["nodes"] == []
assert snap["overview"]["nodes"] == 0
assert snap["overview"]["workstreams"] == 0
assert snap["overview"]["states"]["running"] == 0
assert "timestamp" in snap
def test_get_snapshot_with_nodes(self, populated_collector):
snap = populated_collector.get_snapshot()
assert len(snap["nodes"]) == 2
assert snap["overview"]["nodes"] == 2
assert snap["overview"]["workstreams"] == 3
assert snap["overview"]["states"]["running"] == 1
assert snap["overview"]["states"]["attention"] == 1
assert snap["overview"]["states"]["idle"] == 1
assert snap["overview"]["aggregate"]["total_tokens"] == 17000
assert snap["timestamp"] > 0
# Each node should embed its workstreams
node_ids = {n["node_id"] for n in snap["nodes"]}
assert node_ids == {"node-a", "node-b"}
for n in snap["nodes"]:
if n["node_id"] == "node-a":
assert len(n["workstreams"]) == 2
elif n["node_id"] == "node-b":
assert len(n["workstreams"]) == 1
def test_get_snapshot_consistency(self, populated_collector):
"""Snapshot overview should match get_overview()."""
snap = populated_collector.get_snapshot()
overview = populated_collector.get_overview()
assert snap["overview"]["nodes"] == overview["nodes"]
assert snap["overview"]["workstreams"] == overview["workstreams"]
assert snap["overview"]["states"] == overview["states"]
assert snap["overview"]["aggregate"] == overview["aggregate"]
assert snap["overview"]["version_drift"] == overview["version_drift"]
# ---------------------------------------------------------------------------
# ClusterStateEvent protocol tests
@@ -536,6 +636,31 @@ class TestConsoleHTTPEndpoints:
"workstreams": [],
"aggregate": {},
}
collector.get_snapshot.return_value = {
"nodes": [
{
"node_id": "node-a",
"server_url": "http://a:8080",
"max_ws": 10,
"reachable": True,
"version": "0.5.0",
"health": {},
"aggregate": {"total_tokens": 50000, "total_tool_calls": 200},
"workstreams": [
{"id": "ws1", "name": "test", "state": "running", "node": "node-a"},
],
},
],
"overview": {
"nodes": 3,
"workstreams": 15,
"states": {"running": 5, "thinking": 2, "attention": 1, "idle": 6, "error": 1},
"aggregate": {"total_tokens": 50000, "total_tool_calls": 200},
"version_drift": False,
"versions": ["0.5.0"],
},
"timestamp": 1234567890.0,
}
return collector
@pytest.fixture()
@@ -615,6 +740,16 @@ class TestConsoleHTTPEndpoints:
assert status == 404
assert "error" in data
def test_get_snapshot(self, client, mock_collector):
status, data = self._get(client, "/v1/api/cluster/snapshot")
assert status == 200
assert len(data["nodes"]) == 1
assert data["nodes"][0]["node_id"] == "node-a"
assert data["overview"]["nodes"] == 3
assert data["overview"]["workstreams"] == 15
assert data["timestamp"] == 1234567890.0
mock_collector.get_snapshot.assert_called_once()
def test_health_endpoint(self, client, mock_collector):
status, data = self._get(client, "/health")
assert status == 200
+135
View File
@@ -2049,3 +2049,138 @@ class TestModelCapabilitiesToolSearch:
caps = ModelCapabilities()
assert caps.supports_tool_search is False
# ---------------------------------------------------------------------------
# Vision support
# ---------------------------------------------------------------------------
class TestVisionCapabilities:
"""Test supports_vision flag across providers."""
def test_default_is_false(self) -> None:
from turnstone.core.providers._protocol import ModelCapabilities
caps = ModelCapabilities()
assert caps.supports_vision is False
def test_openai_commercial_supports_vision(self) -> None:
provider = OpenAIProvider()
for model in ("gpt-5", "gpt-5-mini", "gpt-5.4", "o3", "o4-mini"):
caps = provider.get_capabilities(model)
assert caps.supports_vision is True, f"{model} should support vision"
def test_openai_default_no_vision(self) -> None:
"""Unknown models (local servers) default to no vision."""
provider = OpenAIProvider()
caps = provider.get_capabilities("some-local-model")
assert caps.supports_vision is False
def test_anthropic_supports_vision(self) -> None:
from turnstone.core.providers._anthropic import AnthropicProvider
provider = AnthropicProvider()
for model in ("claude-opus-4-6", "claude-sonnet-4-6", "claude-haiku-4-5"):
caps = provider.get_capabilities(model)
assert caps.supports_vision is True, f"{model} should support vision"
def test_anthropic_default_supports_vision(self) -> None:
"""Anthropic default (unknown Claude model) supports vision."""
from turnstone.core.providers._anthropic import AnthropicProvider
provider = AnthropicProvider()
caps = provider.get_capabilities("claude-unknown-9")
assert caps.supports_vision is True
class TestAnthropicVisionConversion:
"""Test image content conversion in _convert_messages."""
def setup_method(self) -> None:
from turnstone.core.providers._anthropic import AnthropicProvider
self.provider = AnthropicProvider()
def test_tool_result_with_image_content(self) -> None:
"""Tool result with list content converts image_url to Anthropic image."""
messages = [
{"role": "user", "content": "Read this image"},
{
"role": "assistant",
"content": "",
"tool_calls": [
{
"id": "call_1",
"function": {"name": "read_file", "arguments": '{"path": "img.png"}'},
}
],
},
{
"role": "tool",
"tool_call_id": "call_1",
"content": [
{"type": "text", "text": "Image file: img.png (1024 bytes)"},
{
"type": "image_url",
"image_url": {"url": "data:image/png;base64,iVBORw0KGgo="},
},
],
},
]
_, converted = self.provider._convert_messages(messages)
# Tool result should be in a user message
tool_user_msg = converted[2]
assert tool_user_msg["role"] == "user"
tool_result = tool_user_msg["content"][0]
assert tool_result["type"] == "tool_result"
assert tool_result["tool_use_id"] == "call_1"
# Content should be a list with converted image block
content = tool_result["content"]
assert isinstance(content, list)
assert content[0] == {"type": "text", "text": "Image file: img.png (1024 bytes)"}
assert content[1]["type"] == "image"
assert content[1]["source"]["type"] == "base64"
assert content[1]["source"]["media_type"] == "image/png"
assert content[1]["source"]["data"] == "iVBORw0KGgo="
def test_tool_result_with_string_content_unchanged(self) -> None:
"""Tool result with plain string content is unchanged."""
messages = [
{"role": "user", "content": "Read file"},
{
"role": "assistant",
"content": "",
"tool_calls": [
{
"id": "call_2",
"function": {"name": "read_file", "arguments": '{"path": "f.py"}'},
}
],
},
{
"role": "tool",
"tool_call_id": "call_2",
"content": " 1\tprint('hello')",
},
]
_, converted = self.provider._convert_messages(messages)
tool_result = converted[2]["content"][0]
assert tool_result["content"] == " 1\tprint('hello')"
def test_convert_content_parts_static_method(self) -> None:
"""_convert_content_parts handles both image_url and text."""
from turnstone.core.providers._anthropic import AnthropicProvider
parts = [
{"type": "text", "text": "description"},
{
"type": "image_url",
"image_url": {"url": "data:image/jpeg;base64,/9j/4AAQ"},
},
]
result = AnthropicProvider._convert_content_parts(parts)
assert result[0] == {"type": "text", "text": "description"}
assert result[1]["type"] == "image"
assert result[1]["source"]["media_type"] == "image/jpeg"
assert result[1]["source"]["data"] == "/9j/4AAQ"
+156 -4
View File
@@ -1,9 +1,10 @@
"""Tests for turnstone.core.session — ChatSession construction."""
import base64
import json
from unittest.mock import MagicMock, patch
from turnstone.core.session import ChatSession
from turnstone.core.session import _IMAGE_EXTENSIONS, _IMAGE_SIZE_CAP, ChatSession
class NullUI:
@@ -202,8 +203,8 @@ class TestPlanExec:
"id": tc_id,
"type": "function",
"function": {
"name": "plan",
"arguments": json.dumps({"prompt": prior_prompt}),
"name": "create_plan",
"arguments": json.dumps({"goal": prior_prompt}),
},
}
],
@@ -238,7 +239,7 @@ class TestPlanExec:
m for m in messages if m["role"] == "assistant" and m.get("tool_calls")
]
assert len(assistant_with_tc) == 1
assert assistant_with_tc[0]["tool_calls"][0]["function"]["name"] == "plan"
assert assistant_with_tc[0]["tool_calls"][0]["function"]["name"] == "create_plan"
# The real tool result is forwarded with its original content
tool_msgs = [m for m in messages if m["role"] == "tool"]
@@ -265,3 +266,154 @@ class TestPlanExec:
call_id, content, _ = self._run_plan(session, "do stuff", agent_return=agent_output)
assert call_id == "test-call-1"
assert content == agent_output
# ---------------------------------------------------------------------------
# Vision / image support
# ---------------------------------------------------------------------------
class TestImageExtensions:
"""Test _IMAGE_EXTENSIONS constant and detection logic."""
def test_common_image_extensions(self):
for ext in (".png", ".jpg", ".jpeg", ".gif", ".webp", ".bmp", ".tiff", ".tif", ".ico"):
assert ext in _IMAGE_EXTENSIONS, f"{ext} should be in _IMAGE_EXTENSIONS"
def test_svg_excluded(self):
assert ".svg" not in _IMAGE_EXTENSIONS
def test_text_extensions_excluded(self):
for ext in (".py", ".txt", ".json", ".md", ".rs", ".go"):
assert ext not in _IMAGE_EXTENSIONS
class TestExecReadImage:
"""Test _exec_read_image method."""
def _make_png(self, path: str, size: int = 100) -> None:
"""Write a minimal valid-ish PNG header to a file."""
# 8-byte PNG signature + enough bytes to reach target size
header = b"\x89PNG\r\n\x1a\n"
with open(path, "wb") as f:
f.write(header + b"\x00" * max(0, size - len(header)))
def test_image_returns_content_parts(self, tmp_db, tmp_path):
"""read_file on a PNG with vision support returns content parts."""
img = tmp_path / "test.png"
self._make_png(str(img))
session = _make_session()
# Mock provider to report vision support
mock_caps = MagicMock()
mock_caps.supports_vision = True
session._provider.get_capabilities = MagicMock(return_value=mock_caps)
item = {"call_id": "c1", "path": str(img), "offset": None, "limit": None}
call_id, output = session._exec_read_file(item)
assert call_id == "c1"
assert isinstance(output, list)
assert len(output) == 2
assert output[0]["type"] == "text"
assert "test.png" in output[0]["text"]
assert output[1]["type"] == "image_url"
url = output[1]["image_url"]["url"]
assert url.startswith("data:image/png;base64,")
# Verify base64 round-trip
b64part = url.split(",", 1)[1]
decoded = base64.b64decode(b64part)
assert decoded == img.read_bytes()
def test_no_vision_returns_text(self, tmp_db, tmp_path):
"""read_file on image with non-vision model returns text description."""
img = tmp_path / "photo.jpg"
self._make_png(str(img), size=2048)
session = _make_session()
mock_caps = MagicMock()
mock_caps.supports_vision = False
session._provider.get_capabilities = MagicMock(return_value=mock_caps)
item = {"call_id": "c2", "path": str(img), "offset": None, "limit": None}
call_id, output = session._exec_read_file(item)
assert call_id == "c2"
assert isinstance(output, str)
assert "does not support vision" in output
assert "photo.jpg" in output
def test_oversized_image_returns_error(self, tmp_db, tmp_path):
"""Images exceeding _IMAGE_SIZE_CAP return an error string."""
img = tmp_path / "huge.png"
# Write slightly over the cap
with open(img, "wb") as f:
f.write(b"\x89PNG\r\n\x1a\n" + b"\x00" * _IMAGE_SIZE_CAP)
session = _make_session()
mock_caps = MagicMock()
mock_caps.supports_vision = True
session._provider.get_capabilities = MagicMock(return_value=mock_caps)
item = {"call_id": "c3", "path": str(img), "offset": None, "limit": None}
call_id, output = session._exec_read_file(item)
assert call_id == "c3"
assert isinstance(output, str)
assert "exceeds" in output
def test_missing_image_returns_error(self, tmp_db, tmp_path):
"""read_file on non-existent image returns error."""
session = _make_session()
mock_caps = MagicMock()
mock_caps.supports_vision = True
session._provider.get_capabilities = MagicMock(return_value=mock_caps)
item = {"call_id": "c4", "path": str(tmp_path / "nope.png"), "offset": None, "limit": None}
call_id, output = session._exec_read_file(item)
assert isinstance(output, str)
assert "not found" in output
def test_svg_read_as_text(self, tmp_db, tmp_path):
"""SVG files are read as text, not as images."""
svg = tmp_path / "icon.svg"
svg.write_text('<svg xmlns="http://www.w3.org/2000/svg"><circle r="10"/></svg>')
session = _make_session()
item = {"call_id": "c5", "path": str(svg), "offset": None, "limit": None}
call_id, output = session._exec_read_file(item)
assert isinstance(output, str)
assert "<svg" in output # Read as text
class TestGetCapabilitiesOverride:
"""Test _get_capabilities with config.toml overrides."""
def test_config_override_applies(self, tmp_db):
"""capabilities dict from ModelConfig is merged onto provider caps."""
from turnstone.core.model_registry import ModelConfig, ModelRegistry
from turnstone.core.providers._protocol import ModelCapabilities
cfg = ModelConfig(
alias="qwen-vl",
base_url="http://localhost:8000/v1",
api_key="dummy",
model="qwen-3.5-vl",
capabilities={"supports_vision": True},
)
registry = ModelRegistry(
models={"qwen-vl": cfg},
default="qwen-vl",
)
session = _make_session(registry=registry, model_alias="qwen-vl")
# Ensure provider returns a real ModelCapabilities (not MagicMock)
session._provider.get_capabilities = MagicMock(return_value=ModelCapabilities())
caps = session._get_capabilities()
assert caps.supports_vision is True
def test_no_override_uses_provider_default(self, tmp_db):
"""Without config override, provider defaults are used."""
session = _make_session()
caps = session._get_capabilities()
# Default OpenAI provider for unknown model → no vision
assert caps.supports_vision is False
+3 -2
View File
@@ -72,7 +72,7 @@ class TestToolsMetadata:
"""Validate the metadata extracted from JSON files."""
def test_tool_count(self):
assert len(TOOLS) == 15
assert len(TOOLS) == 16
def test_agent_tools_count(self):
assert len(AGENT_TOOLS) == 7
@@ -97,11 +97,12 @@ class TestToolsMetadata:
"web_fetch": "url",
"web_search": "query",
"task": "prompt",
"plan": "prompt",
"create_plan": "goal",
"remember": "key",
"recall": "query",
"forget": "key",
"notify": "message",
"watch": "command",
}
assert expected == PRIMARY_KEY_MAP
+487
View File
@@ -0,0 +1,487 @@
"""Tests for the watch module — duration parsing, condition evaluation, WatchRunner."""
from __future__ import annotations
from datetime import UTC, datetime
from unittest.mock import MagicMock
import pytest
from turnstone.core.watch import (
WatchRunner,
evaluate_condition,
format_interval,
format_watch_message,
parse_duration,
validate_condition,
)
# ---------------------------------------------------------------------------
# parse_duration
# ---------------------------------------------------------------------------
class TestParseDuration:
def test_seconds(self):
assert parse_duration("30s") == 30.0
def test_minutes(self):
assert parse_duration("5m") == 300.0
def test_hours(self):
assert parse_duration("1h") == 3600.0
def test_compound(self):
assert parse_duration("2h30m") == 9000.0
def test_bare_number(self):
assert parse_duration("90") == 90.0
def test_bare_float(self):
assert parse_duration("10.5") == 10.5
def test_whitespace(self):
assert parse_duration(" 5m ") == 300.0
def test_case_insensitive(self):
assert parse_duration("1H30M") == 5400.0
def test_empty_raises(self):
with pytest.raises(ValueError, match="empty"):
parse_duration("")
def test_invalid_raises(self):
with pytest.raises(ValueError, match="invalid duration"):
parse_duration("abc")
def test_negative_raises(self):
with pytest.raises(ValueError, match="positive"):
parse_duration("-5")
def test_zero_raises(self):
with pytest.raises(ValueError, match="positive"):
parse_duration("0")
def test_zero_duration_raises(self):
with pytest.raises(ValueError, match="positive"):
parse_duration("0s")
# ---------------------------------------------------------------------------
# validate_condition
# ---------------------------------------------------------------------------
class TestValidateCondition:
def test_valid_expression(self):
assert validate_condition('data["state"] == "MERGED"') is None
def test_valid_simple(self):
assert validate_condition('"error" in output') is None
def test_valid_compound(self):
assert validate_condition('changed and "ready" in output.lower()') is None
def test_syntax_error(self):
result = validate_condition("if True:")
assert result is not None
assert "syntax" in result.lower()
def test_incomplete_expression(self):
result = validate_condition("==")
assert result is not None
# ---------------------------------------------------------------------------
# evaluate_condition
# ---------------------------------------------------------------------------
class TestEvaluateCondition:
def test_none_first_poll_no_fire(self):
"""With stop_on=None, first poll (prev_output=None) should not fire."""
fired, reason = evaluate_condition(None, "hello", 0, None)
assert not fired
def test_none_change_detected(self):
fired, reason = evaluate_condition(None, "world", 0, "hello")
assert fired
assert "changed" in reason
def test_none_no_change(self):
fired, reason = evaluate_condition(None, "same", 0, "same")
assert not fired
def test_string_match(self):
fired, reason = evaluate_condition('"error" in output', "has error here", 0, None)
assert fired
def test_string_no_match(self):
fired, reason = evaluate_condition('"error" in output', "all good", 0, None)
assert not fired
def test_exit_code(self):
fired, reason = evaluate_condition("exit_code != 0", "fail", 1, None)
assert fired
def test_exit_code_zero(self):
fired, reason = evaluate_condition("exit_code != 0", "ok", 0, None)
assert not fired
def test_json_data(self):
output = '{"state": "MERGED"}'
fired, reason = evaluate_condition('data["state"] == "MERGED"', output, 0, None)
assert fired
def test_json_data_no_match(self):
output = '{"state": "OPEN"}'
fired, reason = evaluate_condition('data["state"] == "MERGED"', output, 0, None)
assert not fired
def test_json_data_none_for_non_json(self):
"""Non-JSON output should have data=None."""
fired, reason = evaluate_condition("data is None", "plain text", 0, None)
assert fired
def test_changed_variable(self):
fired, reason = evaluate_condition("changed", "new", 0, "old")
assert fired
def test_changed_false(self):
fired, reason = evaluate_condition("changed", "same", 0, "same")
assert not fired
def test_compound_condition(self):
fired, reason = evaluate_condition(
'changed and "ready" in output.lower()',
"System Ready",
0,
"System Starting",
)
assert fired
def test_invalid_expression_no_crash(self):
fired, reason = evaluate_condition("1/0", "hello", 0, None)
assert not fired
assert "error" in reason.lower()
def test_no_import_builtin(self):
"""__import__ should not be accessible."""
fired, reason = evaluate_condition("__import__('os')", "hello", 0, None)
assert not fired
assert "error" in reason.lower()
def test_no_open_builtin(self):
fired, reason = evaluate_condition("open('/etc/passwd')", "hello", 0, None)
assert not fired
assert "error" in reason.lower()
def test_no_exec_builtin(self):
fired, reason = evaluate_condition("exec('print(1)')", "hello", 0, None)
assert not fired
assert "error" in reason.lower()
def test_no_eval_builtin(self):
fired, reason = evaluate_condition("eval('1+1')", "hello", 0, None)
assert not fired
assert "error" in reason.lower()
def test_no_compile_builtin(self):
fired, reason = evaluate_condition("compile('1','','eval')", "hello", 0, None)
assert not fired
assert "error" in reason.lower()
def test_safe_len(self):
fired, reason = evaluate_condition("len(output) > 0", "hello", 0, None)
assert fired
def test_safe_sorted(self):
fired, reason = evaluate_condition("sorted([3,1,2]) == [1,2,3]", "x", 0, None)
assert fired
def test_data_get_method(self):
output = '{"mergedAt": "2024-01-15"}'
fired, reason = evaluate_condition('data.get("mergedAt") is not None', output, 0, None)
assert fired
def test_prev_output_available(self):
fired, reason = evaluate_condition(
"prev_output is not None and output != prev_output",
"new",
0,
"old",
)
assert fired
# ---------------------------------------------------------------------------
# format_interval
# ---------------------------------------------------------------------------
class TestFormatInterval:
def test_seconds(self):
assert format_interval(30) == "30s"
def test_exactly_60(self):
assert format_interval(60) == "1m"
def test_minutes(self):
assert format_interval(300) == "5m"
def test_exactly_3600(self):
assert format_interval(3600) == "1h"
def test_hours_and_minutes(self):
assert format_interval(5400) == "1h30m"
def test_hours_only(self):
assert format_interval(7200) == "2h"
def test_large_value(self):
assert format_interval(86400) == "24h"
# ---------------------------------------------------------------------------
# format_watch_message
# ---------------------------------------------------------------------------
class TestFormatWatchMessage:
def test_basic(self):
msg = format_watch_message(
name="pr-review",
command="gh pr view --json state",
output='{"state": "MERGED"}',
poll_count=5,
max_polls=100,
elapsed_secs=1500,
stop_on='data["state"] == "MERGED"',
is_final=True,
reason='condition met: data["state"] == "MERGED"',
)
assert "pr-review" in msg
assert "poll #5/100" in msg
assert "25m" in msg
assert "gh pr view --json state" in msg
assert "MERGED" in msg
assert "auto-cancelled" in msg.lower()
# Model should see the condition it was waiting for
assert "condition:" in msg.lower()
def test_non_final(self):
msg = format_watch_message(
name="deploy",
command="curl -s http://localhost/health",
output="ok",
poll_count=3,
max_polls=50,
elapsed_secs=90,
stop_on=None,
is_final=False,
reason="",
)
assert "deploy" in msg
assert "auto-cancelled" not in msg.lower()
# Change-detection mode should be indicated
assert "output change" in msg.lower()
def test_max_polls_final(self):
msg = format_watch_message(
name="test",
command="echo hello",
output="hello",
poll_count=100,
max_polls=100,
elapsed_secs=6000,
stop_on=None,
is_final=True,
reason="",
)
assert "max polls" in msg.lower()
# ---------------------------------------------------------------------------
# WatchRunner
# ---------------------------------------------------------------------------
class TestWatchRunner:
def _make_runner(self, storage=None, **kwargs):
if storage is None:
storage = MagicMock()
storage.list_due_watches.return_value = []
return WatchRunner(
storage=storage,
node_id="test-node",
check_interval=0.1,
tool_timeout=5,
**kwargs,
)
def test_start_stop(self):
runner = self._make_runner()
runner.start()
assert runner._thread is not None
assert runner._thread.is_alive()
runner.stop()
assert runner._thread is None
def test_tick_calls_list_due(self):
storage = MagicMock()
storage.list_due_watches.return_value = []
runner = self._make_runner(storage=storage)
runner._tick()
storage.list_due_watches.assert_called_once()
def test_poll_watch_runs_command(self):
storage = MagicMock()
storage.update_watch.return_value = True
runner = self._make_runner(storage=storage)
dispatch_fn = MagicMock()
runner.set_dispatch_fn("ws-1", dispatch_fn)
watch_row = {
"watch_id": "abc123",
"ws_id": "ws-1",
"name": "test-watch",
"command": "echo hello",
"stop_on": '"hello" in output',
"max_polls": 100,
"poll_count": 0,
"last_output": None,
"interval_secs": 60,
"created": datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%S"),
}
runner._poll_watch(watch_row)
# Should update the watch in storage
storage.update_watch.assert_called_once()
call_kwargs = storage.update_watch.call_args
assert call_kwargs[0][0] == "abc123" # watch_id
assert call_kwargs[1]["poll_count"] == 1
# Condition should fire (output contains "hello")
assert call_kwargs[1]["active"] is False # deactivated
# Should dispatch result
dispatch_fn.assert_called_once()
def test_poll_watch_no_fire_on_first_change_detection(self):
storage = MagicMock()
storage.update_watch.return_value = True
runner = self._make_runner(storage=storage)
dispatch_fn = MagicMock()
runner.set_dispatch_fn("ws-1", dispatch_fn)
watch_row = {
"watch_id": "abc123",
"ws_id": "ws-1",
"name": "test-watch",
"command": "echo hello",
"stop_on": None, # change detection
"max_polls": 100,
"poll_count": 0,
"last_output": None, # first poll
"interval_secs": 60,
"created": datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%S"),
}
runner._poll_watch(watch_row)
# First poll with change detection should not fire
dispatch_fn.assert_not_called()
call_kwargs = storage.update_watch.call_args
# Watch should remain active
assert "active" not in call_kwargs[1] or call_kwargs[1].get("active") is not False
def test_max_polls_deactivates(self):
storage = MagicMock()
storage.update_watch.return_value = True
runner = self._make_runner(storage=storage)
dispatch_fn = MagicMock()
runner.set_dispatch_fn("ws-1", dispatch_fn)
watch_row = {
"watch_id": "abc123",
"ws_id": "ws-1",
"name": "test-watch",
"command": "echo hello",
"stop_on": '"never" in output', # won't fire
"max_polls": 5,
"poll_count": 4, # next is #5 = max
"last_output": "hello\n",
"interval_secs": 60,
"created": datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%S"),
}
runner._poll_watch(watch_row)
call_kwargs = storage.update_watch.call_args
assert call_kwargs[1]["active"] is False
assert call_kwargs[1]["poll_count"] == 5
dispatch_fn.assert_called_once()
def test_blocked_command_deactivates(self):
storage = MagicMock()
storage.update_watch.return_value = True
runner = self._make_runner(storage=storage)
watch_row = {
"watch_id": "abc123",
"ws_id": "ws-1",
"name": "test-watch",
"command": "rm -rf /",
"stop_on": None,
"max_polls": 100,
"poll_count": 0,
"last_output": None,
"interval_secs": 60,
"created": datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%S"),
}
runner._poll_watch(watch_row)
storage.update_watch.assert_called_once()
call_kwargs = storage.update_watch.call_args
assert call_kwargs[0][0] == "abc123"
assert call_kwargs[1]["active"] is False
def test_dispatch_fn_registry(self):
runner = self._make_runner()
fn1 = MagicMock()
fn2 = MagicMock()
runner.set_dispatch_fn("ws-1", fn1)
runner.set_dispatch_fn("ws-2", fn2)
runner._dispatch_result("ws-1", "msg1")
fn1.assert_called_once_with("msg1")
fn2.assert_not_called()
runner.remove_dispatch_fn("ws-1")
# After removal, dispatch should try restore_fn
runner._dispatch_result("ws-1", "msg2")
fn1.assert_called_once() # still just the one call
def test_restore_fn_called_for_evicted(self):
restored_fn = MagicMock()
restore_fn = MagicMock(return_value=restored_fn)
runner = self._make_runner(restore_fn=restore_fn)
runner._dispatch_result("ws-evicted", "hello")
restore_fn.assert_called_once_with("ws-evicted")
restored_fn.assert_called_once_with("hello")
def test_run_command_success(self):
runner = self._make_runner()
output, code = runner._run_command("echo hello")
assert "hello" in output
assert code == 0
def test_run_command_failure(self):
runner = self._make_runner()
output, code = runner._run_command("exit 42")
assert code == 42
def test_run_command_timeout(self):
runner = self._make_runner()
runner._tool_timeout = 1
output, code = runner._run_command("sleep 30")
assert "timed out" in output.lower()
assert code == -1
+130
View File
@@ -0,0 +1,130 @@
"""Tests for watches storage CRUD."""
from __future__ import annotations
import pytest
from turnstone.core.storage._sqlite import SQLiteBackend
@pytest.fixture
def db(tmp_path):
"""Fresh SQLite backend for each test."""
return SQLiteBackend(str(tmp_path / "test.db"))
def _make_watch_kwargs(**overrides):
"""Build default kwargs for create_watch."""
defaults = {
"watch_id": "watch_001",
"ws_id": "ws-abc",
"node_id": "node-1",
"name": "pr-review",
"command": "gh pr view --json state",
"interval_secs": 300.0,
"stop_on": 'data["state"] == "MERGED"',
"max_polls": 100,
"created_by": "model",
"next_poll": "2099-01-01T00:05:00",
}
defaults.update(overrides)
return defaults
class TestWatchCRUD:
def test_create_and_get(self, db):
db.create_watch(**_make_watch_kwargs())
w = db.get_watch("watch_001")
assert w is not None
assert w["name"] == "pr-review"
assert w["command"] == "gh pr view --json state"
assert w["interval_secs"] == 300.0
assert w["active"] == 1
assert w["poll_count"] == 0
def test_get_nonexistent(self, db):
assert db.get_watch("nope") is None
def test_create_idempotent(self, db):
db.create_watch(**_make_watch_kwargs())
db.create_watch(**_make_watch_kwargs()) # OR IGNORE
assert db.get_watch("watch_001") is not None
def test_update(self, db):
db.create_watch(**_make_watch_kwargs())
updated = db.update_watch(
"watch_001",
poll_count=5,
last_output="hello",
last_exit_code=0,
)
assert updated is True
w = db.get_watch("watch_001")
assert w["poll_count"] == 5
assert w["last_output"] == "hello"
assert w["last_exit_code"] == 0
def test_update_nonexistent(self, db):
assert db.update_watch("nope", poll_count=1) is False
def test_update_active_flag(self, db):
db.create_watch(**_make_watch_kwargs())
db.update_watch("watch_001", active=False)
w = db.get_watch("watch_001")
assert w["active"] == 0
def test_delete(self, db):
db.create_watch(**_make_watch_kwargs())
assert db.delete_watch("watch_001") is True
assert db.get_watch("watch_001") is None
def test_delete_nonexistent(self, db):
assert db.delete_watch("nope") is False
class TestWatchListQueries:
def test_list_for_ws(self, db):
db.create_watch(**_make_watch_kwargs(watch_id="w1", ws_id="ws-1", name="a"))
db.create_watch(**_make_watch_kwargs(watch_id="w2", ws_id="ws-1", name="b"))
db.create_watch(**_make_watch_kwargs(watch_id="w3", ws_id="ws-2", name="c"))
ws1 = db.list_watches_for_ws("ws-1")
assert len(ws1) == 2
assert {w["name"] for w in ws1} == {"a", "b"}
def test_list_for_ws_excludes_inactive(self, db):
db.create_watch(**_make_watch_kwargs(watch_id="w1", ws_id="ws-1"))
db.update_watch("w1", active=False)
assert db.list_watches_for_ws("ws-1") == []
def test_list_for_node(self, db):
db.create_watch(**_make_watch_kwargs(watch_id="w1", node_id="n1"))
db.create_watch(**_make_watch_kwargs(watch_id="w2", node_id="n1"))
db.create_watch(**_make_watch_kwargs(watch_id="w3", node_id="n2"))
n1 = db.list_watches_for_node("n1")
assert len(n1) == 2
def test_list_due(self, db):
# Due
db.create_watch(**_make_watch_kwargs(watch_id="w1", next_poll="2020-01-01T00:00:00"))
# Not due (far future)
db.create_watch(**_make_watch_kwargs(watch_id="w2", next_poll="2099-01-01T00:00:00"))
# Due but inactive
db.create_watch(**_make_watch_kwargs(watch_id="w3", next_poll="2020-01-01T00:00:00"))
db.update_watch("w3", active=False)
due = db.list_due_watches("2025-01-01T00:00:00")
assert len(due) == 1
assert due[0]["watch_id"] == "w1"
def test_delete_for_ws(self, db):
db.create_watch(**_make_watch_kwargs(watch_id="w1", ws_id="ws-1"))
db.create_watch(**_make_watch_kwargs(watch_id="w2", ws_id="ws-1"))
db.create_watch(**_make_watch_kwargs(watch_id="w3", ws_id="ws-2"))
count = db.delete_watches_for_ws("ws-1")
assert count == 2
assert db.get_watch("w1") is None
assert db.get_watch("w2") is None
assert db.get_watch("w3") is not None
+111
View File
@@ -683,6 +683,117 @@ class TestWebUI:
t.join()
# ---------------------------------------------------------------------------
# WebUI SSE fan-out
# ---------------------------------------------------------------------------
class TestWebUIFanOut:
"""Verify per-client SSE fan-out on WebUI._enqueue / _register_listener."""
def test_enqueue_no_listeners(self):
"""Events silently dropped when no listeners are registered."""
from turnstone.server import WebUI
ui = WebUI(ws_id="test")
ui._enqueue({"type": "content", "text": "hello"}) # should not raise
def test_enqueue_single_listener(self):
"""Single listener receives the event."""
from turnstone.server import WebUI
ui = WebUI(ws_id="test")
q = ui._register_listener()
ui._enqueue({"type": "content", "text": "hello"})
assert q.get_nowait() == {"type": "content", "text": "hello"}
def test_enqueue_multiple_listeners(self):
"""All registered listeners receive an identical copy."""
from turnstone.server import WebUI
ui = WebUI(ws_id="test")
q1 = ui._register_listener()
q2 = ui._register_listener()
q3 = ui._register_listener()
event = {"type": "content", "text": "world"}
ui._enqueue(event)
assert q1.get_nowait() == event
assert q2.get_nowait() == event
assert q3.get_nowait() == event
def test_unregister_stops_delivery(self):
"""After unregister, the queue receives no further events."""
import queue as queue_mod
from turnstone.server import WebUI
ui = WebUI(ws_id="test")
q = ui._register_listener()
ui._unregister_listener(q)
ui._enqueue({"type": "content", "text": "gone"})
with pytest.raises(queue_mod.Empty):
q.get_nowait()
def test_slow_consumer_does_not_block(self):
"""A full queue doesn't block the producer or starve other listeners."""
from turnstone.server import WebUI
ui = WebUI(ws_id="test")
slow = ui._register_listener()
fast = ui._register_listener()
# Fill only the slow consumer's queue directly to capacity
for i in range(500):
slow.put_nowait({"type": "content", "text": f"fill-{i}"})
assert slow.qsize() == 500
assert fast.qsize() == 0
# Enqueue via fan-out — slow drops (full), fast receives
event = {"type": "content", "text": "overflow"}
ui._enqueue(event)
assert slow.qsize() == 500 # still full, overflow dropped
assert fast.qsize() == 1
assert fast.get_nowait() == event
def test_unregister_idempotent(self):
"""Double unregister does not raise."""
from turnstone.server import WebUI
ui = WebUI(ws_id="test")
q = ui._register_listener()
ui._unregister_listener(q)
ui._unregister_listener(q) # should not raise
def test_concurrent_enqueue_and_register(self):
"""Concurrent register/unregister and enqueue should not crash."""
from turnstone.server import WebUI
ui = WebUI(ws_id="test")
stop = threading.Event()
def register_loop():
while not stop.is_set():
q = ui._register_listener()
ui._unregister_listener(q)
def enqueue_loop():
for i in range(500):
ui._enqueue({"type": "content", "text": f"tok-{i}"})
t1 = threading.Thread(target=register_loop)
t2 = threading.Thread(target=enqueue_loop)
t1.start()
t2.start()
t2.join()
stop.set()
t1.join()
# ---------------------------------------------------------------------------
# Integration: WorkstreamManager + session state transitions
# ---------------------------------------------------------------------------
+1 -1
View File
@@ -1,3 +1,3 @@
"""turnstone - Multi-node AI orchestration platform with tool use, agent routing, and cluster simulation."""
__version__ = "0.5.0"
__version__ = "0.5.3"
+26 -2
View File
@@ -2,6 +2,8 @@
from __future__ import annotations
from typing import Any
from pydantic import BaseModel, Field
# ---------------------------------------------------------------------------
@@ -48,7 +50,7 @@ class ClusterNodeInfo(BaseModel):
total_tokens: int = 0
started: float = 0.0
reachable: bool = True
health: dict[str, str] = Field(default_factory=dict)
health: dict[str, Any] = Field(default_factory=dict)
version: str = ""
@@ -91,12 +93,34 @@ class ClusterWorkstreamsResponse(BaseModel):
class NodeDetailResponse(BaseModel):
node_id: str
server_url: str = ""
health: dict[str, str] = Field(default_factory=dict)
health: dict[str, Any] = Field(default_factory=dict)
workstreams: list[ClusterWorkstreamInfo] = []
aggregate: dict[str, int] = Field(default_factory=dict)
reachable: bool = True
# ---------------------------------------------------------------------------
# Cluster snapshot
# ---------------------------------------------------------------------------
class ClusterSnapshotNode(BaseModel):
node_id: str
server_url: str = ""
max_ws: int = 10
reachable: bool = True
version: str = ""
health: dict[str, Any] = Field(default_factory=dict)
aggregate: dict[str, int] = Field(default_factory=dict)
workstreams: list[ClusterWorkstreamInfo] = []
class ClusterSnapshotResponse(BaseModel):
nodes: list[ClusterSnapshotNode]
overview: ClusterOverviewResponse
timestamp: float = 0.0
# ---------------------------------------------------------------------------
# Workstream creation
# ---------------------------------------------------------------------------
+13 -2
View File
@@ -10,6 +10,7 @@ if TYPE_CHECKING:
from turnstone.api.console_schemas import (
ClusterNodesResponse,
ClusterOverviewResponse,
ClusterSnapshotResponse,
ClusterWorkstreamsResponse,
ConsoleCreateWsRequest,
ConsoleCreateWsResponse,
@@ -97,14 +98,23 @@ CONSOLE_ENDPOINTS: list[EndpointSpec] = [
error_codes=[400, 404, 503],
tags=["Cluster"],
),
EndpointSpec(
"/v1/api/cluster/snapshot",
"GET",
"Full cluster state snapshot",
description="Returns the complete cluster state: all nodes with their workstreams "
"and overview aggregates. Used for initial load and reconnection.",
response_model=ClusterSnapshotResponse,
tags=["Cluster"],
),
# --- Streaming ---
EndpointSpec(
"/v1/api/cluster/events",
"GET",
"Cluster SSE event stream",
description="Server-Sent Events stream for real-time cluster updates. "
"Returns text/event-stream with node_joined, node_lost, cluster_state, "
"ws_created, ws_closed, ws_rename events.",
"First event is a 'snapshot' with full cluster state, followed by "
"node_joined, node_lost, cluster_state, ws_created, ws_closed, ws_rename events.",
tags=["Streaming"],
),
# --- Auth ---
@@ -270,6 +280,7 @@ _ALL_MODELS: list[type[BaseModel]] = [
ClusterNodesResponse,
ClusterWorkstreamsResponse,
NodeDetailResponse,
ClusterSnapshotResponse,
ConsoleCreateWsRequest,
ConsoleCreateWsResponse,
ConsoleHealthResponse,
+114 -7
View File
@@ -150,7 +150,7 @@ class ClusterCollector:
"state": "idle",
"node": node_id,
"server_url": node.server_url,
"title": "",
"title": data.get("title", ""),
"tokens": 0,
"context_ratio": 0.0,
"activity": "",
@@ -273,6 +273,7 @@ class ClusterCollector:
"""Apply polled data to the in-memory node snapshot."""
ws_list = dashboard.get("workstreams", [])
aggregate = dashboard.get("aggregate", {})
pending_events: list[dict[str, Any]] = []
with self._lock:
node = self._nodes.get(node_id)
if not node:
@@ -281,12 +282,35 @@ class ClusterCollector:
node.reachable = True
node.health = health
node.aggregate = aggregate
# Replace workstreams entirely from the authoritative poll
node.workstreams = {}
# Build new workstream map
old_ids = {k for k in node.workstreams if k}
new_ws: dict[str, dict[str, Any]] = {}
for ws in ws_list:
ws_id = ws.get("id", "")
if not ws_id:
continue
ws["node"] = node_id
ws["server_url"] = node.server_url
node.workstreams[ws.get("id", "")] = ws
new_ws[ws_id] = ws
new_ids = set(new_ws.keys())
# Detect additions not yet known to SSE clients
for ws_id in sorted(new_ids - old_ids):
ws = new_ws[ws_id]
pending_events.append(
{
"type": "ws_created",
"ws_id": ws_id,
"name": ws.get("name", ""),
"node_id": node_id,
}
)
# Detect removals
for ws_id in sorted(old_ids - new_ids):
pending_events.append({"type": "ws_closed", "ws_id": ws_id})
node.workstreams = new_ws
# Fan out diffs to SSE listeners outside the lock
for event in pending_events:
self._fanout(event)
# -- query methods (thread-safe) -----------------------------------------
@@ -379,11 +403,11 @@ class ClusterCollector:
)
total = len(items)
# Sort
# Sort (secondary key: node_id for stable ordering)
if sort_by == "activity":
items.sort(key=lambda n: n["ws_running"] + n["ws_attention"], reverse=True)
items.sort(key=lambda n: (-(n["ws_running"] + n["ws_attention"]), n["node_id"]))
elif sort_by == "tokens":
items.sort(key=lambda n: n["total_tokens"], reverse=True)
items.sort(key=lambda n: (-n["total_tokens"], n["node_id"]))
elif sort_by == "name":
items.sort(key=lambda n: n["node_id"])
@@ -455,6 +479,89 @@ class ClusterCollector:
"reachable": node.reachable,
}
def get_snapshot(self) -> dict[str, Any]:
"""Build a complete cluster snapshot under a single lock.
Returns everything the UI needs to render the full dashboard:
all nodes with their workstreams plus pre-computed overview aggregates.
"""
with self._lock:
return self._build_snapshot_locked()
def get_snapshot_and_register(self, q: queue.Queue[dict[str, Any]]) -> dict[str, Any]:
"""Build snapshot and register listener atomically.
Acquiring both locks ensures no event can be published between
the snapshot read and the listener registration the client
receives the snapshot followed by every subsequent event with
no gap.
"""
with self._lock:
snap = self._build_snapshot_locked()
with self._listeners_lock:
self._listeners.append(q)
return snap
def _build_snapshot_locked(self) -> dict[str, Any]:
"""Build snapshot data — caller must hold ``_lock``."""
nodes_out = []
states: dict[str, int] = {
"running": 0,
"thinking": 0,
"attention": 0,
"idle": 0,
"error": 0,
}
total_tokens = 0
total_tool_calls = 0
total_ws = 0
versions: set[str] = set()
for node in self._nodes.values():
ws_list = []
for ws in node.workstreams.values():
ws_list.append(dict(ws))
s = ws.get("state", "idle")
states[s] = states.get(s, 0) + 1
total_ws += 1
total_tokens += node.aggregate.get("total_tokens", 0)
total_tool_calls += node.aggregate.get("total_tool_calls", 0)
ver = node.health.get("version", "")
if ver:
versions.add(ver)
nodes_out.append(
{
"node_id": node.node_id,
"server_url": node.server_url,
"max_ws": node.max_ws,
"reachable": node.reachable,
"version": ver,
"health": dict(node.health),
"aggregate": dict(node.aggregate),
"workstreams": ws_list,
}
)
node_count = len(self._nodes)
return {
"nodes": nodes_out,
"overview": {
"nodes": node_count,
"workstreams": total_ws,
"states": states,
"aggregate": {
"total_tokens": total_tokens,
"total_tool_calls": total_tool_calls,
},
"version_drift": len(versions) > 1,
"versions": sorted(versions),
},
"timestamp": time.time(),
}
# -- SSE listener management ---------------------------------------------
def register_listener(self, q: queue.Queue[dict[str, Any]]) -> None:
+107 -2
View File
@@ -245,14 +245,25 @@ async def cluster_node_detail(request: Request) -> JSONResponse:
return JSONResponse({"error": "Node not found"}, status_code=404)
async def cluster_snapshot(request: Request) -> JSONResponse:
collector: ClusterCollector = request.app.state.collector
return JSONResponse(collector.get_snapshot())
async def cluster_events_sse(request: Request) -> Response:
collector: ClusterCollector = request.app.state.collector
client_queue: queue.Queue[dict[str, Any]] = queue.Queue(maxsize=500)
collector.register_listener(client_queue)
async def event_generator() -> AsyncGenerator[dict[str, str], None]:
loop = asyncio.get_running_loop()
try:
# Atomic snapshot+register — no event gap possible.
snap = await loop.run_in_executor(
None, collector.get_snapshot_and_register, client_queue
)
snap["type"] = "snapshot"
yield {"data": json.dumps(snap)}
while True:
try:
event = await loop.run_in_executor(
@@ -606,7 +617,11 @@ async def _proxy_sse(
return StreamingResponse(
raw_stream(),
media_type="text/event-stream",
headers={"Cache-Control": "no-store", "X-Accel-Buffering": "no"},
headers={
"Cache-Control": "no-store",
"Connection": "keep-alive",
"X-Accel-Buffering": "no",
},
)
@@ -626,6 +641,7 @@ async def _lifespan(app: Starlette) -> AsyncGenerator[None, None]:
# Separate client for SSE streams — longer read timeout, shared connection pool
app.state.proxy_sse_client = httpx.AsyncClient(
timeout=httpx.Timeout(connect=5, read=30, write=5, pool=5),
limits=httpx.Limits(keepalive_expiry=30),
headers=headers,
)
# Start scheduler if configured
@@ -1163,6 +1179,88 @@ async def admin_list_schedule_runs(request: Request) -> JSONResponse:
return JSONResponse({"runs": runs})
# ---------------------------------------------------------------------------
# Admin API endpoints — watches (aggregated from nodes)
# ---------------------------------------------------------------------------
async def admin_list_watches(request: Request) -> JSONResponse:
"""GET /v1/api/admin/watches — aggregate watches from all nodes."""
collector: ClusterCollector = request.app.state.collector
nodes, _ = collector.get_nodes(limit=500)
client: httpx.AsyncClient = request.app.state.proxy_client
headers = _proxy_auth_headers(request)
async def _fetch_node(node: dict[str, Any]) -> list[dict[str, Any]]:
server_url = (node.get("server_url") or "").rstrip("/")
if not server_url:
return []
try:
resp = await client.get(f"{server_url}/v1/api/watches", headers=headers)
if resp.status_code == 200:
data = resp.json()
watches: list[dict[str, Any]] = data.get("watches", [])
# Tag each watch with node_id in case the server omits it
for w in watches:
if not w.get("node_id"):
w["node_id"] = node["node_id"]
return watches
except Exception:
log.debug("Failed to fetch watches from node %s", node.get("node_id"))
return []
tasks = [_fetch_node(n) for n in nodes]
results = await asyncio.gather(*tasks)
all_watches: list[dict[str, Any]] = []
for batch in results:
all_watches.extend(batch)
# Sort: active first, then by created descending (stable sort trick)
all_watches.sort(key=lambda w: w.get("created", ""), reverse=True)
all_watches.sort(key=lambda w: not w.get("active", False))
return JSONResponse({"watches": all_watches})
_VALID_WATCH_ID = re.compile(r"^[a-fA-F0-9]+$")
async def admin_cancel_watch(request: Request) -> Response:
"""POST /v1/api/admin/watches/{watch_id}/cancel — proxy cancel to the owning node."""
from turnstone.core.web_helpers import read_json_or_400
watch_id = request.path_params["watch_id"]
if not watch_id or not _VALID_WATCH_ID.match(watch_id) or len(watch_id) > 128:
return JSONResponse({"error": "Invalid watch_id"}, status_code=400)
body = await read_json_or_400(request)
if isinstance(body, JSONResponse):
return body
node_id = str(body.get("node_id", "") or request.query_params.get("node_id", "")).strip()
if not node_id:
return JSONResponse({"error": "node_id is required"}, status_code=400)
server_url = _get_server_url(request, node_id)
if not server_url:
return JSONResponse({"error": "Node not found"}, status_code=404)
client: httpx.AsyncClient = request.app.state.proxy_client
headers = {"Content-Type": "application/json"}
headers.update(_proxy_auth_headers(request))
try:
resp = await client.post(
f"{server_url}/v1/api/watches/{watch_id}/cancel",
content=b"{}",
headers=headers,
)
return Response(
content=resp.content,
status_code=resp.status_code,
media_type=resp.headers.get("content-type", "application/json"),
)
except httpx.HTTPError:
return JSONResponse({"error": "Node unreachable"}, status_code=502)
# ---------------------------------------------------------------------------
# App factory
# ---------------------------------------------------------------------------
@@ -1195,6 +1293,7 @@ def create_app(
Route("/api/cluster/workstreams", cluster_workstreams),
Route("/api/cluster/workstreams/new", create_workstream, methods=["POST"]),
Route("/api/cluster/node/{node_id}", cluster_node_detail),
Route("/api/cluster/snapshot", cluster_snapshot),
Route("/api/cluster/events", cluster_events_sse),
Route("/api/auth/login", auth_login, methods=["POST"]),
Route("/api/auth/logout", auth_logout, methods=["POST"]),
@@ -1232,6 +1331,12 @@ def create_app(
methods=["DELETE"],
),
Route("/api/admin/schedules/{task_id}/runs", admin_list_schedule_runs),
Route("/api/admin/watches", admin_list_watches),
Route(
"/api/admin/watches/{watch_id}/cancel",
admin_cancel_watch,
methods=["POST"],
),
],
),
Route("/health", health),
+166 -1
View File
@@ -10,6 +10,7 @@ var _ctTrapHandler = null;
var _tcTrapHandler = null;
var _ccTrapHandler = null;
var _cfTrapHandler = null;
var _adminWatches = [];
var _confirmCallbackFn = null;
var _confirmTriggerEl = null;
@@ -48,11 +49,14 @@ function switchAdminTab(tab) {
tab === "channels" ? "" : "none";
document.getElementById("admin-schedules").style.display =
tab === "schedules" ? "" : "none";
document.getElementById("admin-watches").style.display =
tab === "watches" ? "" : "none";
if (tab === "users") loadAdminUsers();
if (tab === "tokens") _populateTokenUserSelect();
if (tab === "channels") _populateChannelUserSelect();
if (tab === "schedules") loadAdminSchedules();
if (tab === "watches") loadAdminWatches();
}
// ---------------------------------------------------------------------------
@@ -887,6 +891,167 @@ function hideScheduleRunsModal() {
_runsScheduleTriggerEl = null;
}
// ---------------------------------------------------------------------------
// Watches
// ---------------------------------------------------------------------------
function _populateWatchNodeSelect() {
var sel = document.getElementById("admin-watch-node");
var current = sel.value;
var seen = {};
sel.innerHTML = '<option value="">All nodes</option>';
for (var i = 0; i < _adminWatches.length; i++) {
var nid = _adminWatches[i].node_id || "";
if (nid && !seen[nid]) {
seen[nid] = true;
var opt = document.createElement("option");
opt.value = nid;
opt.textContent = nid;
sel.appendChild(opt);
}
}
if (current) sel.value = current;
}
function loadAdminWatches() {
authFetch("/v1/api/admin/watches")
.then(function (r) {
if (!r.ok) throw new Error("Failed to load watches");
return r.json();
})
.then(function (data) {
_adminWatches = data.watches || [];
_populateWatchNodeSelect();
var nodeFilter = document.getElementById("admin-watch-node").value;
var filtered = _adminWatches;
if (nodeFilter) {
filtered = _adminWatches.filter(function (w) {
return w.node_id === nodeFilter;
});
}
_renderWatches(filtered);
})
.catch(function () {
document.getElementById("admin-watches-table").innerHTML =
'<div class="dashboard-empty">Failed to load watches</div>';
});
}
function _formatInterval(secs) {
if (!secs || secs <= 0) return "\u2014";
if (secs >= 3600) return Math.round(secs / 3600) + "h";
if (secs >= 60) return Math.round(secs / 60) + "m";
return secs + "s";
}
function _renderWatches(watches) {
var container = document.getElementById("admin-watches-table");
if (!watches.length) {
container.innerHTML =
'<div class="dashboard-empty">No active watches. Watches are created when workstreams use the watch tool.</div>';
return;
}
var html = "";
for (var i = 0; i < watches.length; i++) {
var w = watches[i];
var name = w.name || w.watch_id || "\u2014";
var nodeShort = (w.node_id || "").slice(0, 8);
var cmd = w.command || "";
var cmdTrunc = cmd.length > 40 ? cmd.slice(0, 40) + "\u2026" : cmd;
var interval = _formatInterval(w.interval_secs);
var pollMax = w.max_polls ? w.max_polls : "\u221e";
var pollLabel = (w.poll_count || 0) + "/" + pollMax;
var cond = w.stop_on || "on change";
var condTrunc = cond.length > 30 ? cond.slice(0, 30) + "\u2026" : cond;
var active = w.active;
var statusCls = active ? "watch-active" : "watch-completed";
var statusLabel = active ? "active" : "done";
var statusDot = active ? "\u25cf " : "\u25cb ";
var cancelBtn = active
? '<button class="admin-btn-danger" data-cancel-watch="' +
escapeHtml(w.watch_id) +
'" data-watch-node="' +
escapeHtml(w.node_id || "") +
'" data-watch-name="' +
escapeHtml(name) +
'" title="Cancel watch">cancel</button>'
: "";
html +=
'<div class="admin-row" role="listitem">' +
'<span class="admin-col admin-col-wname">' +
escapeHtml(name) +
"</span>" +
'<span class="admin-col admin-col-wnode" title="' +
escapeHtml(w.node_id || "") +
'"><code>' +
escapeHtml(nodeShort) +
"</code></span>" +
'<span class="admin-col admin-col-wcmd" title="' +
escapeHtml(cmd) +
'"><code>' +
escapeHtml(cmdTrunc) +
"</code></span>" +
'<span class="admin-col admin-col-winterval">' +
escapeHtml(interval) +
"</span>" +
'<span class="admin-col admin-col-wpoll"><code>' +
escapeHtml(pollLabel) +
"</code></span>" +
'<span class="admin-col admin-col-wcond" title="' +
escapeHtml(cond) +
'">' +
escapeHtml(condTrunc) +
"</span>" +
'<span class="admin-col admin-col-wstatus"><span class="' +
statusCls +
'">' +
statusDot +
statusLabel +
"</span></span>" +
'<span class="admin-col admin-col-actions">' +
cancelBtn +
"</span></div>";
}
container.innerHTML = html;
// Bind cancel buttons
var btns = container.querySelectorAll("[data-cancel-watch]");
for (var j = 0; j < btns.length; j++) {
btns[j].addEventListener("click", function () {
_cancelWatch(
this.getAttribute("data-cancel-watch"),
this.getAttribute("data-watch-node"),
this.getAttribute("data-watch-name"),
);
});
}
}
function _cancelWatch(watchId, nodeId, name) {
showConfirmModal(
"Cancel Watch",
"Cancel watch \u2018" + name + "\u2019? This will stop future polling.",
"Cancel watch",
function () {
authFetch(
"/v1/api/admin/watches/" + encodeURIComponent(watchId) + "/cancel",
{
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({ node_id: nodeId }),
},
)
.then(function (r) {
if (!r.ok) throw new Error("Cancel failed");
showToast("Watch '" + name + "' cancelled");
loadAdminWatches();
})
.catch(function () {
showToast("Failed to cancel watch");
});
},
);
}
// ---------------------------------------------------------------------------
// Create Channel Link Modal
// ---------------------------------------------------------------------------
@@ -1271,7 +1436,7 @@ document.addEventListener("keydown", function (e) {
if (!tablist) return;
tablist.addEventListener("keydown", function (e) {
if (e.key !== "ArrowLeft" && e.key !== "ArrowRight") return;
var tabOrder = ["users", "tokens", "channels", "schedules"];
var tabOrder = ["users", "tokens", "channels", "schedules", "watches"];
var idx = tabOrder.indexOf(_adminTab);
if (e.key === "ArrowRight") idx = (idx + 1) % tabOrder.length;
else idx = (idx - 1 + tabOrder.length) % tabOrder.length;
+308 -116
View File
@@ -1,9 +1,6 @@
// --- Shared hooks ---
window.onLoginSuccess = function () {
connectSSE();
if (currentView === "overview") loadOverview();
else if (currentView === "node") drillDownToNode(currentNodeId);
else if (currentView === "filtered") loadFilteredWorkstreams();
};
window.onLogout = function () {
if (evtSource) {
@@ -33,6 +30,8 @@ var _lastOverviewJson = "";
var _lastNodesJson = "";
var evtSource = null;
var retryDelay = 1000;
var clusterState = null;
var _navigatingFromPopstate = false;
// --- Constants ---
var STATE_DISPLAY = {
@@ -44,6 +43,236 @@ var STATE_DISPLAY = {
};
var STATE_ORDER = ["running", "thinking", "attention", "error", "idle"];
// --- Cluster State Model ---
function applySnapshot(data) {
clusterState = {
nodes: {},
overview: data.overview || {},
timestamp: data.timestamp || 0,
};
(data.nodes || []).forEach(function (n) {
clusterState.nodes[n.node_id] = n;
});
renderFromState();
}
function patchClusterState(data) {
if (!clusterState) return;
var t = data.type;
if (t === "cluster_state") {
var node = clusterState.nodes[data.node_id];
if (node) {
(node.workstreams || []).forEach(function (ws) {
if (ws.id === data.ws_id) {
if ("state" in data) ws.state = data.state;
if ("tokens" in data) ws.tokens = data.tokens;
if ("context_ratio" in data) ws.context_ratio = data.context_ratio;
if ("activity" in data) ws.activity = data.activity;
if ("activity_state" in data) ws.activity_state = data.activity_state;
}
});
}
} else if (t === "ws_created") {
var targetNode = clusterState.nodes[data.node_id];
if (targetNode) {
targetNode.workstreams = targetNode.workstreams || [];
targetNode.workstreams.push({
id: data.ws_id,
name: data.name || "",
state: "idle",
node: data.node_id,
server_url: targetNode.server_url || "",
title: data.title || "",
tokens: 0,
context_ratio: 0.0,
activity: "",
activity_state: "",
tool_calls: 0,
});
}
} else if (t === "ws_closed") {
Object.keys(clusterState.nodes).forEach(function (nid) {
var n = clusterState.nodes[nid];
n.workstreams = (n.workstreams || []).filter(function (ws) {
return ws.id !== data.ws_id;
});
});
} else if (t === "ws_rename") {
Object.keys(clusterState.nodes).forEach(function (nid) {
(clusterState.nodes[nid].workstreams || []).forEach(function (ws) {
if (ws.id === data.ws_id) ws.name = data.name || "";
});
});
} else if (t === "node_joined") {
if (!clusterState.nodes[data.node_id]) {
clusterState.nodes[data.node_id] = {
node_id: data.node_id,
server_url: "",
max_ws: 10,
reachable: true,
version: "",
health: {},
aggregate: {},
workstreams: [],
};
}
} else if (t === "node_lost") {
delete clusterState.nodes[data.node_id];
} else {
return;
}
scheduleRender();
}
var _renderTimer = null;
function scheduleRender() {
if (_renderTimer) return;
_renderTimer = requestAnimationFrame(function () {
_renderTimer = null;
recomputeOverview();
renderFromState();
});
}
function recomputeOverview() {
if (!clusterState) return;
var states = { running: 0, thinking: 0, attention: 0, idle: 0, error: 0 };
var totalTokens = 0,
totalToolCalls = 0,
totalWs = 0;
var versions = {};
Object.keys(clusterState.nodes).forEach(function (nid) {
var node = clusterState.nodes[nid];
var nodeWsTokens = 0;
(node.workstreams || []).forEach(function (ws) {
var s = ws.state || "idle";
states[s] = (states[s] || 0) + 1;
totalWs++;
nodeWsTokens += ws.tokens || 0;
});
var aggTokens = (node.aggregate || {}).total_tokens || 0;
totalTokens += aggTokens || nodeWsTokens;
totalToolCalls += (node.aggregate || {}).total_tool_calls || 0;
if (node.version) versions[node.version] = true;
});
var versionList = Object.keys(versions).sort();
clusterState.overview = {
nodes: Object.keys(clusterState.nodes).length,
workstreams: totalWs,
states: states,
aggregate: {
total_tokens: totalTokens,
total_tool_calls: totalToolCalls,
},
version_drift: versionList.length > 1,
versions: versionList,
};
}
function buildNodeInfoFromSnapshot(node) {
var states = { running: 0, thinking: 0, attention: 0, idle: 0, error: 0 };
var ws = node.workstreams || [];
ws.forEach(function (w) {
var s = w.state || "idle";
states[s] = (states[s] || 0) + 1;
});
var aggTokens = (node.aggregate || {}).total_tokens || 0;
if (!aggTokens) {
ws.forEach(function (w) {
aggTokens += w.tokens || 0;
});
}
return {
node_id: node.node_id,
server_url: node.server_url || "",
ws_total: ws.length,
ws_running: states.running,
ws_thinking: states.thinking,
ws_attention: states.attention,
ws_idle: states.idle,
ws_error: states.error,
total_tokens: aggTokens,
ws_tokens: aggTokens,
max_ws: node.max_ws || 10,
started: node.started || 0,
reachable: node.reachable !== false,
health: node.health || {},
version: node.version || "",
};
}
function renderFromState() {
if (!clusterState) return;
renderStatusBar(clusterState.overview);
if (currentView === "overview") {
var nodesList = Object.keys(clusterState.nodes).map(function (nid) {
return buildNodeInfoFromSnapshot(clusterState.nodes[nid]);
});
nodesList.sort(function (a, b) {
var d = b.ws_running + b.ws_attention - (a.ws_running + a.ws_attention);
return d !== 0 ? d : a.node_id.localeCompare(b.node_id);
});
renderNodeGroups(nodesList, nodesList.length);
document.getElementById("cluster-summary").textContent =
clusterState.overview.nodes +
" nodes \u00b7 " +
formatCount(clusterState.overview.workstreams) +
" workstreams";
} else if (currentView === "node" && currentNodeId) {
var snapNode = clusterState.nodes[currentNodeId];
if (snapNode) {
var wsList = snapNode.workstreams || [];
var active = wsList.filter(function (w) {
return w.state !== "idle";
}).length;
document.getElementById("node-ws-summary").textContent =
active + " active \u00b7 " + wsList.length + " total";
renderWsTable(document.getElementById("node-ws-table"), wsList);
}
} else if (currentView === "filtered") {
var allWs = [];
Object.keys(clusterState.nodes).forEach(function (nid) {
(clusterState.nodes[nid].workstreams || []).forEach(function (ws) {
allWs.push(ws);
});
});
if (currentFilter.state) {
allWs = allWs.filter(function (ws) {
return ws.state === currentFilter.state;
});
}
if (currentFilter.node) {
allWs = allWs.filter(function (ws) {
return ws.node === currentFilter.node;
});
}
var stateOrder = {
running: 0,
thinking: 1,
attention: 2,
error: 3,
idle: 4,
};
allWs.sort(function (a, b) {
return (stateOrder[a.state] || 9) - (stateOrder[b.state] || 9);
});
var total = allWs.length;
var perPage = currentFilter.per_page || 50;
var pages = Math.max(1, Math.ceil(total / perPage));
var page = Math.min(currentFilter.page || 1, pages);
var start = (page - 1) * perPage;
var pageWs = allWs.slice(start, start + perPage);
document.getElementById("filtered-summary").textContent =
"Page " + page + " of " + pages + " (" + total + " total)";
renderWsTable(document.getElementById("filtered-ws-table"), pageWs);
renderPagination(
document.getElementById("filtered-pagination"),
page,
pages,
);
}
}
// --- SSE Connection ---
function connectSSE() {
if (evtSource) {
@@ -94,19 +323,11 @@ function connectSSE() {
};
}
var _refreshTimer = null;
function scheduleRefresh() {
if (_refreshTimer) return;
_refreshTimer = setTimeout(function () {
_refreshTimer = null;
if (currentView === "overview") loadOverview();
else if (currentView === "node" && currentNodeId)
loadNodeDetail(currentNodeId);
else if (currentView === "filtered") loadFilteredWorkstreams();
}, 250);
}
function handleClusterEvent(data) {
if (data.type === "snapshot") {
applySnapshot(data);
return;
}
if (
data.type === "cluster_state" ||
data.type === "ws_created" ||
@@ -115,7 +336,7 @@ function handleClusterEvent(data) {
data.type === "node_joined" ||
data.type === "node_lost"
) {
scheduleRefresh();
patchClusterState(data);
}
if (data.type === "ws_closed" && data.reason === "evicted") {
showToast("Evicted" + (data.name ? ": " + data.name : "") + " (capacity)");
@@ -135,28 +356,18 @@ function showOverview() {
if (adminView) adminView.style.display = "none";
document.getElementById("breadcrumb").style.display = "none";
document.getElementById("main").scrollTop = 0;
loadOverview();
history.pushState({ view: "overview" }, "");
if (clusterState) renderFromState();
else loadOverview();
if (!_navigatingFromPopstate) history.pushState({ view: "overview" }, "");
}
function loadOverview() {
var overviewP = authFetch("/v1/api/cluster/overview").then(function (r) {
return r.json();
});
var nodesP = authFetch("/v1/api/cluster/nodes?sort=activity&limit=1000").then(
function (r) {
authFetch("/v1/api/cluster/snapshot")
.then(function (r) {
return r.json();
},
);
Promise.all([overviewP, nodesP])
.then(function (res) {
renderStatusBar(res[0]);
renderNodeGroups(res[1].nodes, res[1].total);
document.getElementById("cluster-summary").textContent =
res[0].nodes +
" nodes \u00b7 " +
formatCount(res[0].workstreams) +
" workstreams";
})
.then(function (data) {
applySnapshot(data);
})
.catch(function () {
document.getElementById("node-table").innerHTML =
@@ -310,7 +521,8 @@ function groupNodes(nodes) {
});
groupOrder.forEach(function (prefix) {
groupMap[prefix].nodes.sort(function (a, b) {
return b.ws_running + b.ws_attention - (a.ws_running + a.ws_attention);
var d = b.ws_running + b.ws_attention - (a.ws_running + a.ws_attention);
return d !== 0 ? d : a.node_id.localeCompare(b.node_id);
});
});
var groups = groupOrder.map(function (p) {
@@ -653,38 +865,37 @@ function drillDownToNode(nodeId, serverUrl) {
link.href = "/node/" + encodeURIComponent(nodeId) + "/";
link.style.display = "";
document.getElementById("main").scrollTop = 0;
document.getElementById("node-ws-table").innerHTML =
'<div class="dashboard-empty">Loading workstreams...</div>';
loadNodeDetail(nodeId);
if (clusterState && clusterState.nodes[nodeId]) {
renderFromState();
} else {
document.getElementById("node-ws-table").innerHTML =
'<div class="dashboard-empty">Loading workstreams...</div>';
loadNodeDetail(nodeId);
}
document.getElementById("breadcrumb-home").focus();
history.pushState({ view: "node", nodeId: nodeId, serverUrl: serverUrl }, "");
if (!_navigatingFromPopstate)
history.pushState(
{ view: "node", nodeId: nodeId, serverUrl: serverUrl },
"",
);
}
function loadNodeDetail(nodeId) {
var detailP = authFetch(
"/v1/api/cluster/node/" + encodeURIComponent(nodeId),
).then(function (r) {
return r.json();
});
var overviewP = authFetch("/v1/api/cluster/overview").then(function (r) {
return r.json();
});
Promise.all([detailP, overviewP]).then(function (res) {
var data = res[0];
renderStatusBar(res[1]);
if (data.error) {
authFetch("/v1/api/cluster/snapshot")
.then(function (r) {
return r.json();
})
.then(function (data) {
applySnapshot(data);
if (!clusterState || !clusterState.nodes[nodeId]) {
document.getElementById("node-ws-table").innerHTML =
'<div class="dashboard-empty">Node not found</div>';
}
})
.catch(function () {
document.getElementById("node-ws-table").innerHTML =
'<div class="dashboard-empty">' + escapeHtml(data.error) + "</div>";
return;
}
var ws = data.workstreams || [];
var active = ws.filter(function (w) {
return w.state !== "idle";
}).length;
document.getElementById("node-ws-summary").textContent =
active + " active \u00b7 " + ws.length + " total";
renderWsTable(document.getElementById("node-ws-table"), ws);
});
'<div class="dashboard-empty">Failed to load</div>';
});
}
// --- Drill-down: Filtered ---
@@ -703,9 +914,11 @@ function drillDownByState(state) {
document.getElementById("filtered-title").textContent =
"WORKSTREAMS — " + sd.label.toUpperCase();
document.getElementById("main").scrollTop = 0;
loadFilteredWorkstreams();
if (clusterState) renderFromState();
else loadFilteredWorkstreams();
document.getElementById("breadcrumb-home").focus();
history.pushState({ view: "filtered", filter: currentFilter }, "");
if (!_navigatingFromPopstate)
history.pushState({ view: "filtered", filter: currentFilter }, "");
}
function drillDownByNode(nodeId) {
@@ -721,48 +934,20 @@ function drillDownByNode(nodeId) {
document.getElementById("filtered-title").textContent =
"WORKSTREAMS — " + nodeId;
document.getElementById("main").scrollTop = 0;
loadFilteredWorkstreams();
if (clusterState) renderFromState();
else loadFilteredWorkstreams();
document.getElementById("breadcrumb-home").focus();
history.pushState({ view: "filtered", filter: currentFilter }, "");
if (!_navigatingFromPopstate)
history.pushState({ view: "filtered", filter: currentFilter }, "");
}
function loadFilteredWorkstreams() {
var params =
"page=" + currentFilter.page + "&per_page=" + currentFilter.per_page;
if (currentFilter.state)
params += "&state=" + encodeURIComponent(currentFilter.state);
if (currentFilter.node)
params += "&node=" + encodeURIComponent(currentFilter.node);
var wsP = authFetch("/v1/api/cluster/workstreams?" + params).then(
function (r) {
authFetch("/v1/api/cluster/snapshot")
.then(function (r) {
return r.json();
},
);
var overviewP = authFetch("/v1/api/cluster/overview").then(function (r) {
return r.json();
});
Promise.all([wsP, overviewP])
.then(function (res) {
var data = res[0];
renderStatusBar(res[1]);
document.getElementById("main").scrollTop = 0;
document.getElementById("filtered-summary").textContent =
"Page " +
data.page +
" of " +
data.pages +
" (" +
data.total +
" total)";
renderWsTable(
document.getElementById("filtered-ws-table"),
data.workstreams,
);
renderPagination(
document.getElementById("filtered-pagination"),
data.page,
data.pages,
);
})
.then(function (data) {
applySnapshot(data);
})
.catch(function () {
document.getElementById("filtered-ws-table").innerHTML =
@@ -778,7 +963,8 @@ function renderPagination(container, page, pages) {
prev.disabled = page <= 1;
prev.onclick = function () {
currentFilter.page--;
loadFilteredWorkstreams();
if (clusterState) renderFromState();
else loadFilteredWorkstreams();
};
container.appendChild(prev);
var info = document.createElement("span");
@@ -789,7 +975,8 @@ function renderPagination(container, page, pages) {
next.disabled = page >= pages;
next.onclick = function () {
currentFilter.page++;
loadFilteredWorkstreams();
if (clusterState) renderFromState();
else loadFilteredWorkstreams();
};
container.appendChild(next);
}
@@ -923,19 +1110,24 @@ function renderWsTable(container, wsList) {
window.addEventListener("popstate", function (e) {
var overlay = document.getElementById("login-overlay");
if (overlay && overlay.style.display !== "none") return;
if (!e.state) {
showOverview();
return;
}
if (e.state.view === "overview") showOverview();
else if (e.state.view === "admin" && typeof showAdmin === "function")
showAdmin();
else if (e.state.view === "node" && e.state.nodeId)
drillDownToNode(e.state.nodeId, e.state.serverUrl);
else if (e.state.view === "filtered" && e.state.filter) {
currentFilter = e.state.filter;
if (currentFilter.state) drillDownByState(currentFilter.state);
else if (currentFilter.node) drillDownByNode(currentFilter.node);
_navigatingFromPopstate = true;
try {
if (!e.state) {
showOverview();
return;
}
if (e.state.view === "overview") showOverview();
else if (e.state.view === "admin" && typeof showAdmin === "function")
showAdmin();
else if (e.state.view === "node" && e.state.nodeId)
drillDownToNode(e.state.nodeId, e.state.serverUrl);
else if (e.state.view === "filtered" && e.state.filter) {
currentFilter = e.state.filter;
if (currentFilter.state) drillDownByState(currentFilter.state);
else if (currentFilter.node) drillDownByNode(currentFilter.node);
}
} finally {
_navigatingFromPopstate = false;
}
});
+25
View File
@@ -81,6 +81,7 @@
<button id="tab-tokens" class="admin-tab" data-tab="tokens" role="tab" aria-selected="false" aria-controls="admin-tokens" tabindex="-1" onclick="switchAdminTab('tokens')">Tokens</button>
<button id="tab-channels" class="admin-tab" data-tab="channels" role="tab" aria-selected="false" aria-controls="admin-channels" tabindex="-1" onclick="switchAdminTab('channels')">Channels</button>
<button id="tab-schedules" class="admin-tab" data-tab="schedules" role="tab" aria-selected="false" aria-controls="admin-schedules" tabindex="-1" onclick="switchAdminTab('schedules')">Schedules</button>
<button id="tab-watches" class="admin-tab" data-tab="watches" role="tab" aria-selected="false" aria-controls="admin-watches" tabindex="-1" onclick="switchAdminTab('watches')">Watches</button>
</div>
<!-- Users Tab -->
@@ -163,6 +164,30 @@
<div class="dashboard-empty">Loading schedules...</div>
</div>
</div>
<!-- Watches Tab -->
<div id="admin-watches" class="admin-panel" role="tabpanel" aria-labelledby="tab-watches" style="display:none">
<div class="admin-toolbar">
<span class="section-header" style="margin:0">WATCHES</span>
<label for="admin-watch-node" class="sr-only">Filter watches by node</label>
<select id="admin-watch-node" onchange="loadAdminWatches()">
<option value="">All nodes</option>
</select>
</div>
<div class="admin-colheaders" aria-hidden="true">
<span class="admin-col admin-col-wname">NAME</span>
<span class="admin-col admin-col-wnode">NODE</span>
<span class="admin-col admin-col-wcmd">COMMAND</span>
<span class="admin-col admin-col-winterval">INTERVAL</span>
<span class="admin-col admin-col-wpoll">POLL</span>
<span class="admin-col admin-col-wcond">CONDITION</span>
<span class="admin-col admin-col-wstatus">STATUS</span>
<span class="admin-col admin-col-actions">ACTIONS</span>
</div>
<div id="admin-watches-table" role="list" aria-label="Watches" aria-live="polite">
<div class="dashboard-empty">Loading watches...</div>
</div>
</div>
</div>
</div>
+14
View File
@@ -863,6 +863,16 @@
.sched-disabled { color: var(--fg-dim); }
.sched-expired { color: var(--accent); }
/* Watches grid: NAME | NODE | COMMAND | INTERVAL | POLL | CONDITION | STATUS | ACTIONS */
#admin-watches .admin-colheaders,
#admin-watches .admin-row {
grid-template-columns: 1.2fr 80px 1.5fr 60px 70px 1fr 70px 70px;
}
/* Watch status indicators */
.watch-active { color: var(--green); font-weight: 500; }
.watch-completed { color: var(--accent); }
/* Wide modal variant for schedule forms */
.admin-modal-wide { width: 480px; }
@@ -1027,6 +1037,10 @@
grid-template-columns: 1fr 60px 80px 130px;
}
.admin-col-sschedule, .admin-col-starget, .admin-col-snext { display: none; }
#admin-watches .admin-colheaders, #admin-watches .admin-row {
grid-template-columns: 1.2fr 80px 70px 70px 70px;
}
.admin-col-wcmd, .admin-col-wcond, .admin-col-winterval { display: none; }
}
/* ==========================================================================
+7
View File
@@ -390,6 +390,13 @@ def required_scope(method: str, path: str) -> str:
# Write endpoints
if method == "POST" and normalized in WRITE_PATHS:
return "write"
# Watch cancel has a path parameter: /api/watches/{id}/cancel
if (
method == "POST"
and normalized.startswith("/api/watches/")
and normalized.endswith("/cancel")
):
return "write"
# Console proxy routes: /node/{node_id}/api/{tail} or /node/{node_id}/v1/api/{tail}
if method == "POST" and normalized.startswith("/node/"):
+4
View File
@@ -33,6 +33,7 @@ class ModelConfig:
model: str
context_window: int = 131072
provider: str = "openai"
capabilities: dict[str, Any] = field(default_factory=dict)
# ---------------------------------------------------------------------------
@@ -185,6 +186,9 @@ def load_model_registry(
model=model_name,
context_window=entry.get("context_window", context_window),
provider=entry.get("provider", "openai"),
capabilities=entry.get("capabilities", {})
if isinstance(entry.get("capabilities"), dict)
else {},
)
# Ensure a "default" entry from CLI args
+50 -1
View File
@@ -75,6 +75,7 @@ _ANTHROPIC_DEFAULT = ModelCapabilities(
token_param="max_tokens",
thinking_mode="manual",
supports_web_search=True,
supports_vision=True,
)
_ANTHROPIC_CAPABILITIES: dict[str, ModelCapabilities] = {
@@ -87,6 +88,7 @@ _ANTHROPIC_CAPABILITIES: dict[str, ModelCapabilities] = {
effort_levels=("low", "medium", "high", "max"),
supports_web_search=True,
supports_tool_search=True,
supports_vision=True,
),
"claude-sonnet-4-6": ModelCapabilities(
context_window=200000,
@@ -97,6 +99,7 @@ _ANTHROPIC_CAPABILITIES: dict[str, ModelCapabilities] = {
effort_levels=("low", "medium", "high"),
supports_web_search=True,
supports_tool_search=True,
supports_vision=True,
),
"claude-haiku-4-5": ModelCapabilities(
context_window=200000,
@@ -104,6 +107,7 @@ _ANTHROPIC_CAPABILITIES: dict[str, ModelCapabilities] = {
token_param="max_tokens",
thinking_mode="manual",
supports_web_search=True,
supports_vision=True,
),
"claude-sonnet-4-5": ModelCapabilities(
context_window=200000,
@@ -111,6 +115,7 @@ _ANTHROPIC_CAPABILITIES: dict[str, ModelCapabilities] = {
token_param="max_tokens",
thinking_mode="manual",
supports_web_search=True,
supports_vision=True,
),
"claude-opus-4-5": ModelCapabilities(
context_window=200000,
@@ -120,6 +125,7 @@ _ANTHROPIC_CAPABILITIES: dict[str, ModelCapabilities] = {
supports_effort=True,
effort_levels=("low", "medium", "high"),
supports_web_search=True,
supports_vision=True,
),
"claude-opus-4": ModelCapabilities(
context_window=200000,
@@ -128,6 +134,7 @@ _ANTHROPIC_CAPABILITIES: dict[str, ModelCapabilities] = {
thinking_mode="manual",
supports_web_search=True,
supports_tool_search=True,
supports_vision=True,
),
"claude-sonnet-4": ModelCapabilities(
context_window=200000,
@@ -136,6 +143,7 @@ _ANTHROPIC_CAPABILITIES: dict[str, ModelCapabilities] = {
thinking_mode="manual",
supports_web_search=True,
supports_tool_search=True,
supports_vision=True,
),
}
@@ -324,11 +332,15 @@ class AnthropicProvider:
tool_results: list[dict[str, Any]] = []
while i < len(messages) and messages[i]["role"] == "tool":
tool_msg = messages[i]
content = tool_msg.get("content", "")
# Convert image_url parts to Anthropic image format
if isinstance(content, list):
content = self._convert_content_parts(content)
tool_results.append(
{
"type": "tool_result",
"tool_use_id": tool_msg.get("tool_call_id", ""),
"content": tool_msg.get("content", ""),
"content": content,
}
)
i += 1
@@ -346,6 +358,43 @@ class AnthropicProvider:
return "\n\n".join(system_parts), _merge_consecutive(converted)
@staticmethod
def _convert_content_parts(parts: list[dict[str, Any]]) -> list[dict[str, Any]]:
"""Convert OpenAI-format content parts to Anthropic format.
Transforms ``image_url`` parts (with ``data:`` URIs) to Anthropic's
``image`` source blocks. Text parts pass through unchanged.
"""
converted: list[dict[str, Any]] = []
for part in parts:
if part.get("type") == "image_url":
url = part.get("image_url", {}).get("url", "")
if url.startswith("data:") and "," in url:
# Parse "data:image/png;base64,<data>"
header, _, b64data = url.partition(",")
media_type = header.split(":", 1)[1].split(";", 1)[0]
converted.append(
{
"type": "image",
"source": {
"type": "base64",
"media_type": media_type,
"data": b64data,
},
}
)
else:
# URL-based image — pass as Anthropic URL source
converted.append(
{
"type": "image",
"source": {"type": "url", "url": url},
}
)
else:
converted.append(part)
return converted
# -- tool conversion -----------------------------------------------------
def convert_tools(
+17
View File
@@ -30,6 +30,7 @@ _OPENAI_CAPABILITIES: dict[str, ModelCapabilities] = {
supports_temperature=False,
reasoning_effort_values=("minimal", "low", "medium", "high"),
default_reasoning_effort="medium",
supports_vision=True,
),
"gpt-5-mini": ModelCapabilities(
context_window=400000,
@@ -37,6 +38,7 @@ _OPENAI_CAPABILITIES: dict[str, ModelCapabilities] = {
supports_temperature=False,
reasoning_effort_values=("minimal", "low", "medium", "high"),
default_reasoning_effort="medium",
supports_vision=True,
),
"gpt-5-nano": ModelCapabilities(
context_window=400000,
@@ -44,6 +46,7 @@ _OPENAI_CAPABILITIES: dict[str, ModelCapabilities] = {
supports_temperature=False,
reasoning_effort_values=("minimal", "low", "medium", "high"),
default_reasoning_effort="medium",
supports_vision=True,
),
# GPT-5 pro — high reasoning only, extended output
"gpt-5-pro": ModelCapabilities(
@@ -52,6 +55,7 @@ _OPENAI_CAPABILITIES: dict[str, ModelCapabilities] = {
supports_temperature=False,
reasoning_effort_values=("high",),
default_reasoning_effort="high",
supports_vision=True,
),
# GPT-5.1 — temperature OK when reasoning_effort=none (default)
"gpt-5.1": ModelCapabilities(
@@ -59,6 +63,7 @@ _OPENAI_CAPABILITIES: dict[str, ModelCapabilities] = {
max_output_tokens=128000,
reasoning_effort_values=("none", "low", "medium", "high"),
default_reasoning_effort="none",
supports_vision=True,
),
# GPT-5.2 — adds xhigh
"gpt-5.2": ModelCapabilities(
@@ -66,6 +71,7 @@ _OPENAI_CAPABILITIES: dict[str, ModelCapabilities] = {
max_output_tokens=128000,
reasoning_effort_values=("none", "low", "medium", "high", "xhigh"),
default_reasoning_effort="none",
supports_vision=True,
),
# GPT-5.2 pro — always-reasoning variant
"gpt-5.2-pro": ModelCapabilities(
@@ -74,6 +80,7 @@ _OPENAI_CAPABILITIES: dict[str, ModelCapabilities] = {
supports_temperature=False,
reasoning_effort_values=("medium", "high", "xhigh"),
default_reasoning_effort="medium",
supports_vision=True,
),
# GPT-5.3 — same capabilities as 5.2 (matches gpt-5.3-chat-latest, codex)
"gpt-5.3": ModelCapabilities(
@@ -81,6 +88,7 @@ _OPENAI_CAPABILITIES: dict[str, ModelCapabilities] = {
max_output_tokens=128000,
reasoning_effort_values=("none", "low", "medium", "high", "xhigh"),
default_reasoning_effort="none",
supports_vision=True,
),
# GPT-5.4 — 1M context window, native tool search
"gpt-5.4": ModelCapabilities(
@@ -89,6 +97,7 @@ _OPENAI_CAPABILITIES: dict[str, ModelCapabilities] = {
reasoning_effort_values=("none", "low", "medium", "high", "xhigh"),
default_reasoning_effort="none",
supports_tool_search=True,
supports_vision=True,
),
# GPT-5.4 pro — always-reasoning, 1M context, native tool search
"gpt-5.4-pro": ModelCapabilities(
@@ -98,6 +107,7 @@ _OPENAI_CAPABILITIES: dict[str, ModelCapabilities] = {
reasoning_effort_values=("medium", "high", "xhigh"),
default_reasoning_effort="medium",
supports_tool_search=True,
supports_vision=True,
),
# O-series reasoning models
"o1": ModelCapabilities(
@@ -105,33 +115,39 @@ _OPENAI_CAPABILITIES: dict[str, ModelCapabilities] = {
max_output_tokens=100000,
supports_temperature=False,
supports_streaming=False,
supports_vision=True,
),
"o1-mini": ModelCapabilities(
context_window=128000,
max_output_tokens=65536,
supports_temperature=False,
supports_streaming=False,
supports_vision=True,
),
"o3": ModelCapabilities(
context_window=200000,
max_output_tokens=100000,
supports_temperature=False,
supports_vision=True,
),
"o3-mini": ModelCapabilities(
context_window=200000,
max_output_tokens=100000,
supports_temperature=False,
supports_vision=True,
),
"o3-pro": ModelCapabilities(
context_window=200000,
max_output_tokens=100000,
supports_temperature=False,
supports_streaming=False,
supports_vision=True,
),
"o4-mini": ModelCapabilities(
context_window=200000,
max_output_tokens=100000,
supports_temperature=False,
supports_vision=True,
),
# Search models — always search on every request, no reasoning_effort
"gpt-5-search-api": ModelCapabilities(
@@ -140,6 +156,7 @@ _OPENAI_CAPABILITIES: dict[str, ModelCapabilities] = {
supports_temperature=False,
supports_web_search=True,
reasoning_effort_values=(),
supports_vision=True,
),
}
+1
View File
@@ -77,6 +77,7 @@ class ModelCapabilities:
default_reasoning_effort: str = "medium"
supports_web_search: bool = False
supports_tool_search: bool = False
supports_vision: bool = False
def _lookup_capabilities(
+513 -35
View File
@@ -8,10 +8,14 @@ to receive events and handle approval prompts.
from __future__ import annotations
import base64
import concurrent.futures
import contextlib
import dataclasses
import json
import mimetypes
import os
import queue
import re
import signal
import subprocess
@@ -71,8 +75,21 @@ if TYPE_CHECKING:
from turnstone.core.healthcheck import BackendHealthMonitor
from turnstone.core.mcp_client import MCPClientManager
from turnstone.core.model_registry import ModelRegistry
from turnstone.core.providers import CompletionResult, LLMProvider, StreamChunk
from turnstone.core.model_registry import ModelConfig, ModelRegistry
from turnstone.core.providers import (
CompletionResult,
LLMProvider,
ModelCapabilities,
StreamChunk,
)
# Image extensions handled as vision content (SVG excluded — it's XML text)
_IMAGE_EXTENSIONS: frozenset[str] = frozenset(
{".png", ".jpg", ".jpeg", ".gif", ".webp", ".bmp", ".tiff", ".tif", ".ico"}
)
# 4 MB raw → ~5.3 MB base64, safely under Anthropic's per-block limit
_IMAGE_SIZE_CAP: int = 4 * 1024 * 1024
# ---------------------------------------------------------------------------
# SessionUI protocol — the contract every frontend must implement
@@ -206,6 +223,10 @@ class ChatSession:
self._assistant_pending_tokens = 0
self.creative_mode = False
self._notify_count = 0
# Watch support: server-level runner injected via set_watch_runner()
self._watch_runner: Any = None # WatchRunner | None
self._watch_pending: queue.Queue[dict[str, Any]] = queue.Queue()
self._watch_dispatch_depth = 0
# MCP tool integration: merge external tools with built-in
self._mcp_client = mcp_client
self._mcp_refresh_cb: Any = None # Callable | None (avoid import)
@@ -246,6 +267,18 @@ class ChatSession:
def model_alias(self) -> str | None:
return self._model_alias
def _get_capabilities(self) -> ModelCapabilities:
"""Get model capabilities, applying config.toml overrides if present."""
caps = self._provider.get_capabilities(self.model)
if self._registry and self._model_alias:
cfg: ModelConfig = self._registry.get_config(self._model_alias)
if cfg.capabilities:
fields = {f.name for f in dataclasses.fields(type(caps))}
overrides = {k: v for k, v in cfg.capabilities.items() if k in fields}
if overrides:
caps = dataclasses.replace(caps, **overrides)
return caps
def _save_config(self) -> None:
"""Persist LLM-affecting config so resumed workstreams behave identically."""
save_workstream_config(
@@ -302,11 +335,32 @@ class ChatSession:
else:
self._tool_search = None
def set_watch_runner(self, runner: Any, dispatch_fn: Any = None) -> None:
"""Inject the server-level WatchRunner (called after workstream setup).
If *dispatch_fn* is provided (the server passes one that can start
worker threads), it is registered directly. Otherwise a simple
enqueue fallback is used suitable only when ``send()`` is already
active (Path A).
"""
self._watch_runner = runner
if dispatch_fn is not None:
runner.set_dispatch_fn(self._ws_id, dispatch_fn)
else:
pending = self._watch_pending
def _enqueue(msg: str) -> None:
pending.put({"message": msg})
runner.set_dispatch_fn(self._ws_id, _enqueue)
def close(self) -> None:
"""Release resources (listener registrations, etc.)."""
if self._mcp_client and self._mcp_refresh_cb:
self._mcp_client.remove_listener(self._mcp_refresh_cb)
self._mcp_refresh_cb = None
if self._watch_runner:
self._watch_runner.remove_dispatch_fn(self._ws_id)
def _handle_mcp_refresh(self, arg: str) -> None:
"""Handle ``/mcp refresh [server]``."""
@@ -493,19 +547,21 @@ class ChatSession:
" write_file(path='hello.py', content='...')\n\n"
"Find something across files → search:\n"
" search(query='test_')\n\n"
"Complex or multi-step task → plan first:\n"
" plan(prompt='refactor database from API')\n\n"
"Plan, design, or think through an approach → create_plan:\n"
" create_plan(goal='refactor database from API')\n\n"
"Run a command, git, or tests → bash:\n"
" bash(command='git log -5')\n"
" bash(command='pytest')\n\n"
"Retrieve a URL → web_fetch:\n"
" web_fetch(url='https://example.com')\n\n"
"Search the web for information → web_search:\n"
" web_search(query='current population of Tokyo')\n\n"
"Look up documentation → man:\n"
" man(page='tar')",
]
# Tool search hint (client-side mode only — native mode needs no hint)
if self._tool_search:
caps = self._provider.get_capabilities(self.model)
caps = self._get_capabilities()
if not caps.supports_tool_search:
dev_parts.append(
"\n\nAdditional tools are available via tool_search. "
@@ -564,7 +620,7 @@ class ChatSession:
if not self._tool_search:
return self._tools
# Check if provider supports native tool search
caps = self._provider.get_capabilities(self.model)
caps = self._get_capabilities()
if caps.supports_tool_search:
# Provider handles defer_loading — send all tools
return self._tools
@@ -576,7 +632,7 @@ class ChatSession:
"""Return names of deferred tools for native provider search, or None."""
if not self._tool_search:
return None
caps = self._provider.get_capabilities(self.model)
caps = self._get_capabilities()
if not caps.supports_tool_search:
return None # Client-side mode — no deferred names for provider
deferred = self._tool_search.get_deferred_tools()
@@ -738,6 +794,9 @@ class ChatSession:
self._title_generated = True
threading.Thread(target=self._generate_title, daemon=True).start()
self._emit_state("idle")
# Dispatch any pending watch results (chains into
# a new send() within the same worker thread).
self._dispatch_pending_watch(self._watch_dispatch_depth)
break
# Execute tool calls (potentially in parallel)
@@ -746,13 +805,27 @@ class ChatSession:
# Map tool_call_id → tool name for logging
_tc_names = {c["id"]: c.get("function", {}).get("name", "") for c in tool_calls}
for tc_id, output in results:
tool_msg = {
tool_msg: dict[str, Any] = {
"role": "tool",
"tool_call_id": tc_id,
"content": output,
}
self.messages.append(tool_msg)
self._msg_tokens.append(max(1, int(len(output) / self._chars_per_token)))
# Token estimation — image content uses a fixed heuristic
if isinstance(output, list):
text_chars = sum(
len(p.get("text", "")) for p in output if p.get("type") == "text"
)
image_count = sum(1 for p in output if p.get("type") == "image_url")
tok_est = max(
1,
int(text_chars / self._chars_per_token) + image_count * 1000,
)
else:
tok_est = max(1, int(len(output) / self._chars_per_token))
self._msg_tokens.append(tok_est)
# Log tool result (skip memory tools to avoid noise)
_tname = _tc_names.get(tc_id, "")
if _tname not in (
@@ -760,10 +833,17 @@ class ChatSession:
"forget",
"recall",
):
# For image content, store text description only
if isinstance(output, list):
store_text = " ".join(
p.get("text", "") for p in output if p.get("type") == "text"
)[:2000]
else:
store_text = output[:2000]
save_message(
self._ws_id,
"tool_result",
output[:2000],
store_text,
_tname,
tool_call_id=tc_id,
)
@@ -1047,6 +1127,16 @@ class ChatSession:
tool_calls = m.get("tool_calls")
tc_id = m.get("tool_call_id")
# Flatten list content (image tool results) for display
if isinstance(content, list):
parts = []
for p in content:
if p.get("type") == "text":
parts.append(p.get("text", ""))
elif p.get("type") == "image_url":
parts.append("[image]")
content = " ".join(parts)
# Truncate long content for readability
if len(content) > 300:
display = content[:200] + f"...({len(content)} chars)..." + content[-50:]
@@ -1076,7 +1166,11 @@ class ChatSession:
def _msg_char_count(self, msg: dict[str, Any]) -> int:
"""Count characters in a message, including tool call arguments."""
n = len(msg.get("content") or "")
content = msg.get("content")
if isinstance(content, list):
n = sum(len(p.get("text", "")) for p in content if p.get("type") == "text")
else:
n = len(content or "")
for tc in msg.get("tool_calls", []):
n += len(tc.get("function", {}).get("name", ""))
n += len(tc.get("function", {}).get("arguments", ""))
@@ -1134,6 +1228,16 @@ class ChatSession:
role = m["role"].upper()
content = m.get("content") or ""
# Flatten list content (image tool results) to text for summary
if isinstance(content, list):
text_parts = []
for p in content:
if p.get("type") == "text":
text_parts.append(p["text"])
elif p.get("type") == "image_url":
text_parts.append("[image]")
content = " ".join(text_parts)
if m.get("tool_calls"):
calls = []
for tc in m["tool_calls"]:
@@ -1321,7 +1425,7 @@ class ChatSession:
def _execute_tools(
self, tool_calls: list[dict[str, Any]]
) -> tuple[list[tuple[str, str]], str | None]:
) -> tuple[list[tuple[str, str | list[dict[str, Any]]]], str | None]:
"""Execute tool calls with batch preview and approval.
Returns (results, user_feedback) where user_feedback is an optional
@@ -1343,12 +1447,14 @@ class ChatSession:
user_feedback = None # feedback is in the denial_msg
# Phase 3: execute
def run_one(item: dict[str, Any]) -> tuple[str, str]:
def run_one(
item: dict[str, Any],
) -> tuple[str, str | list[dict[str, Any]]]:
if item.get("error"):
return item["call_id"], item["error"]
if item.get("denied"):
return item["call_id"], item.get("denial_msg", "Denied by user")
result: tuple[str, str] = item["execute"](item)
result: tuple[str, str | list[dict[str, Any]]] = item["execute"](item)
return result
if len(items) == 1:
@@ -1360,12 +1466,13 @@ class ChatSession:
# Post-plan gate: prompt user on main thread after plan completes
for i, item in enumerate(items):
if (
item.get("func_name") == "plan"
item.get("func_name") == "create_plan"
and not item.get("error")
and not item.get("denied")
and not self.auto_approve
):
cid, output = results[i]
assert isinstance(output, str) # plan always returns text
# Let the UI present the plan for review
self._emit_state("attention")
resp = self.ui.on_plan_review(output)
@@ -1439,11 +1546,12 @@ class ChatSession:
"web_search": self._prepare_web_search,
"tool_search": self._prepare_tool_search,
"task": self._prepare_task,
"plan": self._prepare_plan,
"create_plan": self._prepare_plan,
"remember": self._prepare_remember,
"recall": self._prepare_recall,
"forget": self._prepare_forget,
"notify": self._prepare_notify,
"watch": self._prepare_watch,
}
preparer = preparers.get(func_name)
if not preparer:
@@ -1988,26 +2096,26 @@ class ChatSession:
def _prepare_plan(self, call_id: str, args: dict[str, Any]) -> dict[str, Any]:
"""Prepare a planning agent for approval."""
prompt = (args.get("prompt") or "").strip()
if not prompt:
goal = args.get("goal", "").strip()
if not goal:
return {
"call_id": call_id,
"func_name": "plan",
"header": "\u2717 plan: empty prompt",
"func_name": "create_plan",
"header": "\u2717 create_plan: empty goal",
"preview": "",
"needs_approval": False,
"error": "Error: empty prompt",
"error": "Error: empty goal",
}
preview_text = prompt[:300] + ("..." if len(prompt) > 300 else "")
preview_text = goal[:300] + ("..." if len(goal) > 300 else "")
return {
"call_id": call_id,
"func_name": "plan",
"header": "\u2699 plan (planning agent)",
"func_name": "create_plan",
"header": "\u2699 create_plan (planning agent)",
"preview": f" {DIM}{preview_text}{RESET}",
"needs_approval": True,
"approval_label": "plan",
"approval_label": "create_plan",
"execute": self._exec_plan,
"prompt": prompt,
"prompt": goal,
}
def _prepare_remember(self, call_id: str, args: dict[str, Any]) -> dict[str, Any]:
@@ -2210,13 +2318,18 @@ class ChatSession:
self.ui.on_error(msg)
return call_id, msg
def _exec_read_file(self, item: dict[str, Any]) -> tuple[str, str]:
"""Read a file and return numbered lines, optionally sliced."""
def _exec_read_file(self, item: dict[str, Any]) -> tuple[str, str | list[dict[str, Any]]]:
"""Read a file and return numbered lines, or image content parts."""
call_id, path = item["call_id"], item["path"]
offset = item.get("offset") # 1-based, or None
limit = item.get("limit") # max lines, or None
resolved = os.path.realpath(path)
# Image file detection (branch before text open)
ext = os.path.splitext(path)[1].lower()
if ext in _IMAGE_EXTENSIONS:
return self._exec_read_image(call_id, path, resolved)
try:
with open(path) as f:
all_lines = f.readlines()
@@ -2251,6 +2364,61 @@ class ChatSession:
return call_id, output if output else "(empty file)"
def _exec_read_image(
self, call_id: str, path: str, resolved: str
) -> tuple[str, str | list[dict[str, Any]]]:
"""Read an image file and return as base64 content parts for vision."""
caps = self._get_capabilities()
if not caps.supports_vision:
try:
size = os.path.getsize(path)
except OSError as e:
self._read_files.discard(resolved)
return call_id, f"Error: {path}: {e}"
self._read_files.add(resolved)
desc = f"image (no vision, {size:,} bytes)"
self.ui.on_tool_result(call_id, "read_file", desc)
return call_id, (
f"Binary image file: {path} ({size:,} bytes). "
"Current model does not support vision."
)
try:
with open(path, "rb") as f:
raw = f.read()
except FileNotFoundError:
self._read_files.discard(resolved)
return call_id, f"Error: {path} not found"
except Exception as e:
self._read_files.discard(resolved)
return call_id, f"Error reading {path}: {e}"
if len(raw) > _IMAGE_SIZE_CAP:
self._read_files.discard(resolved)
size_mb = len(raw) / (1024 * 1024)
cap_mb = _IMAGE_SIZE_CAP / (1024 * 1024)
return call_id, (
f"Error: image {path} is {size_mb:.1f} MB, "
f"exceeds {cap_mb:.0f} MB limit for vision."
)
self._read_files.add(resolved)
b64data = base64.b64encode(raw).decode("ascii")
mime, _ = mimetypes.guess_type(path)
if not mime:
mime = "image/png"
content_parts: list[dict[str, Any]] = [
{"type": "text", "text": f"Image file: {path} ({len(raw):,} bytes)"},
{
"type": "image_url",
"image_url": {"url": f"data:{mime};base64,{b64data}"},
},
]
self.ui.on_tool_result(call_id, "read_file", f"image ({len(raw):,} bytes)")
return call_id, content_parts
def _exec_search(self, item: dict[str, Any]) -> tuple[str, str]:
"""Search file contents for a regex pattern using grep."""
call_id = item["call_id"]
@@ -2380,7 +2548,20 @@ class ChatSession:
turn = 0
while max_tool_turns < 0 or turn < max_tool_turns:
result = _api_call(agent_messages)
try:
result = _api_call(agent_messages)
except Exception as e:
# Context-exceeded or other non-retryable API error.
# Return what we have so far rather than crashing.
err_str = str(e).lower()
if "context" in err_str or "token" in err_str:
self.ui.on_info(f"[{label}] context limit reached, stopping early")
# Find the last assistant content we have
for msg in reversed(agent_messages):
if msg.get("role") == "assistant" and msg.get("content"):
return msg["content"]
return f"({label} stopped: context limit exceeded)"
raise
# Handle truncation or content filter — stop agent early
if result.finish_reason == "length":
@@ -2411,7 +2592,7 @@ class ChatSession:
tool_name = tc_dict["function"]["name"]
# Guard 1: block recursive agent calls.
if tool_name in ("task", "plan"):
if tool_name in ("task", "create_plan"):
output = "Error: agents cannot spawn further agents"
# Guard 2: tool not in this agent's API tool list.
elif tool_name not in tool_names:
@@ -2444,6 +2625,12 @@ class ChatSession:
else:
output = f"Unknown tool: {tool_name}"
# Truncate large tool outputs to avoid blowing context limits.
# Agents operate autonomously; they can refine their queries
# if truncation loses important detail.
if isinstance(output, str) and len(output) > 16000:
output = output[:16000] + f"\n\n... (truncated from {len(output)} chars)"
agent_messages.append(
{
"role": "tool",
@@ -2492,8 +2679,13 @@ class ChatSession:
"questions — execute the work as described in the prompt."
),
}
agent_messages = list(self._agent_system_messages) + [
task_instruction,
# Task agent gets the base system prompt (tool patterns) merged
# with its own identity in a single system message. No conversation
# history — it's an autonomous sub-agent. Merged to avoid
# multi-system-message errors on models like Qwen.
base = self._agent_system_messages[0]["content"] if self._agent_system_messages else ""
agent_messages = [
{"role": "system", "content": base + "\n\n" + task_instruction["content"]},
{"role": "user", "content": prompt},
]
try:
@@ -2532,7 +2724,7 @@ class ChatSession:
for i, msg in enumerate(self.messages):
if msg.get("role") == "assistant" and msg.get("tool_calls"):
for tc in msg["tool_calls"]:
if tc.get("function", {}).get("name") == "plan":
if tc.get("function", {}).get("name") == "create_plan":
tc_id = tc["id"]
for j in range(i + 1, len(self.messages)):
if (
@@ -2542,8 +2734,11 @@ class ChatSession:
prior_plan_msgs = [msg, self.messages[j]]
break
agent_messages = list(self._agent_system_messages)
agent_messages.append({"role": "system", "content": self._PLAN_IDENTITY})
# Plan agent gets its own identity only — no main session system
# prompt or conversation history. It's an autonomous sub-agent.
agent_messages: list[dict[str, Any]] = [
{"role": "system", "content": self._PLAN_IDENTITY},
]
agent_messages.extend(prior_plan_msgs)
agent_messages.append({"role": "user", "content": prompt})
@@ -2821,6 +3016,289 @@ class ChatSession:
self.ui.on_tool_result(call_id, "notify", msg)
return call_id, msg
# -- Watch tool ----------------------------------------------------------
def _prepare_watch(self, call_id: str, args: dict[str, Any]) -> dict[str, Any]:
from turnstone.core.watch import (
MAX_INTERVAL,
MAX_WATCHES_PER_WS,
MIN_INTERVAL,
parse_duration,
validate_condition,
)
action = args.get("action", "")
if action == "list":
return {
"call_id": call_id,
"func_name": "watch",
"header": "\u23f1 watch: list",
"preview": "",
"needs_approval": False,
"execute": self._exec_watch,
"action": "list",
}
if action == "cancel":
name = args.get("name", "")
if not name:
return {
"call_id": call_id,
"func_name": "watch",
"header": "\u2717 watch cancel: missing name",
"preview": "",
"needs_approval": False,
"error": "Error: 'name' is required for cancel",
}
return {
"call_id": call_id,
"func_name": "watch",
"header": f'\u23f1 watch: cancel "{name}"',
"preview": "",
"needs_approval": False,
"execute": self._exec_watch,
"action": "cancel",
"watch_name": name,
}
if action != "create":
return {
"call_id": call_id,
"func_name": "watch",
"header": f"\u2717 watch: unknown action '{action}'",
"preview": "",
"needs_approval": False,
"error": f"Error: unknown action '{action}'. Use create, list, or cancel.",
}
# --- action=create ---
command = sanitize_command(args.get("command", ""))
if not command:
return {
"call_id": call_id,
"func_name": "watch",
"header": "\u2717 watch create: missing command",
"preview": "",
"needs_approval": False,
"error": "Error: 'command' is required for create",
}
blocked = is_command_blocked(command)
if blocked:
return {
"call_id": call_id,
"func_name": "watch",
"header": f"\u2717 {blocked}",
"preview": "",
"needs_approval": False,
"error": blocked,
}
# Parse poll interval
poll_every_str = args.get("poll_every", "5m")
try:
interval_secs = parse_duration(poll_every_str)
except ValueError as exc:
return {
"call_id": call_id,
"func_name": "watch",
"header": f"\u2717 watch: invalid poll_every: {exc}",
"preview": "",
"needs_approval": False,
"error": f"Error: invalid poll_every: {exc}",
}
if interval_secs < MIN_INTERVAL:
return {
"call_id": call_id,
"func_name": "watch",
"header": f"\u2717 watch: interval too short (min {MIN_INTERVAL}s)",
"preview": "",
"needs_approval": False,
"error": f"Error: minimum poll interval is {MIN_INTERVAL}s",
}
if interval_secs > MAX_INTERVAL:
return {
"call_id": call_id,
"func_name": "watch",
"header": f"\u2717 watch: interval too long (max {MAX_INTERVAL}s)",
"preview": "",
"needs_approval": False,
"error": f"Error: maximum poll interval is {MAX_INTERVAL}s",
}
# Validate stop condition
stop_on = args.get("stop_on")
if stop_on is not None:
err = validate_condition(stop_on)
if err:
return {
"call_id": call_id,
"func_name": "watch",
"header": f"\u2717 watch: {err}",
"preview": "",
"needs_approval": False,
"error": f"Error: {err}",
}
# Check max watches limit and duplicate names
storage = get_storage()
existing: list[dict[str, Any]] = []
if storage:
existing = storage.list_watches_for_ws(self._ws_id)
if len(existing) >= MAX_WATCHES_PER_WS:
return {
"call_id": call_id,
"func_name": "watch",
"header": f"\u2717 watch: limit reached ({MAX_WATCHES_PER_WS})",
"preview": "",
"needs_approval": False,
"error": f"Error: maximum {MAX_WATCHES_PER_WS} active watches per workstream",
}
name = args.get("name", "")
if not name:
name = f"watch-{uuid.uuid4().hex[:4]}"
elif storage and any(w["name"] == name for w in existing):
return {
"call_id": call_id,
"func_name": "watch",
"header": f'\u2717 watch: name "{name}" already in use',
"preview": "",
"needs_approval": False,
"error": f'Error: a watch named "{name}" already exists in this workstream',
}
max_polls = args.get("max_polls", 100)
try:
max_polls = int(max_polls)
except (ValueError, TypeError):
max_polls = 100
display_cmd = command.split("\n")[0]
condition_display = f", stop_on={stop_on}" if stop_on else ", on change"
return {
"call_id": call_id,
"func_name": "watch",
"header": f'\u23f1 watch: "{name}" every {poll_every_str}',
"preview": f" {display_cmd}{condition_display}",
"needs_approval": True,
"approval_label": "watch",
"execute": self._exec_watch,
"action": "create",
"command": command,
"interval_secs": interval_secs,
"stop_on": stop_on,
"watch_name": name,
"max_polls": max_polls,
}
def _exec_watch(self, item: dict[str, Any]) -> tuple[str, str]:
from datetime import UTC, datetime, timedelta
call_id = item["call_id"]
action = item["action"]
storage = get_storage()
if action == "list":
if not storage:
msg = "No watches (storage unavailable)"
self.ui.on_tool_result(call_id, "watch", msg)
return call_id, msg
watches = storage.list_watches_for_ws(self._ws_id)
if not watches:
msg = "No active watches."
self.ui.on_tool_result(call_id, "watch", msg)
return call_id, msg
from turnstone.core.watch import format_interval
lines = []
for w in watches:
condition = w.get("stop_on") or "on change"
lines.append(
f" {w['name']} ({w['watch_id'][:8]}): "
f"every {format_interval(w['interval_secs'])}, "
f"poll #{w['poll_count']}/{w['max_polls']}, "
f"condition: {condition}, "
f"cmd: {w['command'][:60]}"
)
msg = "Active watches:\n" + "\n".join(lines)
self.ui.on_tool_result(call_id, "watch", msg)
return call_id, msg
if action == "cancel":
name = item.get("watch_name", "")
if not storage:
msg = "Error: storage unavailable"
self.ui.on_tool_result(call_id, "watch", msg)
return call_id, msg
watches = storage.list_watches_for_ws(self._ws_id)
target = None
for w in watches:
if w["name"] == name or w["watch_id"].startswith(name):
target = w
break
if target is None:
msg = f'Watch "{name}" not found.'
self.ui.on_tool_result(call_id, "watch", msg)
return call_id, msg
storage.update_watch(target["watch_id"], active=False, next_poll="")
msg = f'Watch "{target["name"]}" cancelled.'
self.ui.on_tool_result(call_id, "watch", msg)
return call_id, msg
# action == "create"
if not storage:
msg = "Error: storage unavailable"
self.ui.on_tool_result(call_id, "watch", msg)
return call_id, msg
watch_id = uuid.uuid4().hex
now = datetime.now(UTC)
next_poll = now + timedelta(seconds=item["interval_secs"])
storage.create_watch(
watch_id=watch_id,
ws_id=self._ws_id,
node_id=self._node_id or "",
name=item["watch_name"],
command=item["command"],
interval_secs=item["interval_secs"],
stop_on=item.get("stop_on"),
max_polls=item["max_polls"],
created_by="model",
next_poll=next_poll.strftime("%Y-%m-%dT%H:%M:%S"),
)
from turnstone.core.watch import format_interval
stop_desc = f"stop_on: {item['stop_on']}" if item.get("stop_on") else "on output change"
msg = (
f'Watch "{item["watch_name"]}" created.\n'
f" Polling every {format_interval(item['interval_secs'])}, "
f"max {item['max_polls']} polls\n"
f" Command: {item['command']}\n"
f" Condition: {stop_desc}"
)
self.ui.on_tool_result(call_id, "watch", msg)
return call_id, msg
_MAX_WATCH_CHAIN = 5 # max consecutive watch dispatches per worker thread
def _dispatch_pending_watch(self, depth: int = 0) -> None:
"""Dispatch one pending watch result as a new send() turn.
Each ``send()`` chains back here on IDLE, so multiple queued results
are processed sequentially. Depth is capped to prevent unbounded
stack growth.
"""
if depth >= self._MAX_WATCH_CHAIN:
self._watch_dispatch_depth = 0
return
try:
result = self._watch_pending.get_nowait()
except queue.Empty:
self._watch_dispatch_depth = 0 # chain ended — reset for next user turn
return
message = result.get("message", "")
if message:
self._watch_dispatch_depth = depth + 1
self.send(message)
def _exec_write_file(self, item: dict[str, Any]) -> tuple[str, str]:
"""Write content to a file, creating parent directories as needed."""
call_id = item["call_id"]
+133
View File
@@ -1011,6 +1011,139 @@ class PostgreSQLBackend:
conn.commit()
return result.rowcount
# -- Watches ---------------------------------------------------------------
def create_watch(
self,
watch_id: str,
ws_id: str,
node_id: str,
name: str,
command: str,
interval_secs: float,
stop_on: str | None,
max_polls: int,
created_by: str,
next_poll: str,
) -> None:
from sqlalchemy.dialects import postgresql
from turnstone.core.storage._schema import watches
now = datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%S")
with self._engine.connect() as conn:
conn.execute(
postgresql.insert(watches)
.values(
watch_id=watch_id,
ws_id=ws_id,
node_id=node_id,
name=name,
command=command,
interval_secs=interval_secs,
stop_on=stop_on,
max_polls=max_polls,
poll_count=0,
active=1,
created_by=created_by,
next_poll=next_poll,
created=now,
updated=now,
)
.on_conflict_do_nothing()
)
conn.commit()
def get_watch(self, watch_id: str) -> dict[str, Any] | None:
from turnstone.core.storage._schema import watches
with self._engine.connect() as conn:
row = conn.execute(sa.select(watches).where(watches.c.watch_id == watch_id)).fetchone()
if row is None:
return None
return dict(row._mapping)
def list_watches_for_ws(self, ws_id: str) -> list[dict[str, Any]]:
from turnstone.core.storage._schema import watches
with self._engine.connect() as conn:
rows = conn.execute(
sa.select(watches)
.where((watches.c.ws_id == ws_id) & (watches.c.active == 1))
.order_by(watches.c.created.desc())
).fetchall()
return [dict(r._mapping) for r in rows]
def list_watches_for_node(self, node_id: str) -> list[dict[str, Any]]:
from turnstone.core.storage._schema import watches
with self._engine.connect() as conn:
rows = conn.execute(
sa.select(watches)
.where((watches.c.node_id == node_id) & (watches.c.active == 1))
.order_by(watches.c.created.desc())
).fetchall()
return [dict(r._mapping) for r in rows]
def list_due_watches(self, now: str) -> list[dict[str, Any]]:
from turnstone.core.storage._schema import watches
with self._engine.connect() as conn:
rows = conn.execute(
sa.select(watches)
.where(
(watches.c.active == 1)
& (watches.c.next_poll <= now)
& (watches.c.next_poll != "")
)
.order_by(watches.c.next_poll)
.limit(100)
).fetchall()
return [dict(r._mapping) for r in rows]
_UPDATABLE_WATCH_FIELDS = frozenset(
{
"name",
"poll_count",
"last_output",
"last_exit_code",
"last_poll",
"next_poll",
"active",
"updated",
}
)
def update_watch(self, watch_id: str, **fields: Any) -> bool:
from turnstone.core.storage._schema import watches
fields = {k: v for k, v in fields.items() if k in self._UPDATABLE_WATCH_FIELDS}
fields["updated"] = datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%S")
if "active" in fields:
fields["active"] = 1 if fields["active"] else 0
with self._engine.connect() as conn:
result = conn.execute(
sa.update(watches).where(watches.c.watch_id == watch_id).values(**fields)
)
conn.commit()
return result.rowcount > 0
def delete_watch(self, watch_id: str) -> bool:
from turnstone.core.storage._schema import watches
with self._engine.connect() as conn:
result = conn.execute(sa.delete(watches).where(watches.c.watch_id == watch_id))
conn.commit()
return result.rowcount > 0
def delete_watches_for_ws(self, ws_id: str) -> int:
from turnstone.core.storage._schema import watches
with self._engine.connect() as conn:
result = conn.execute(sa.delete(watches).where(watches.c.ws_id == ws_id))
conn.commit()
return result.rowcount
# -- Service registry ------------------------------------------------------
def register_service(
+46
View File
@@ -293,6 +293,52 @@ class StorageBackend(Protocol):
"""Delete task runs older than retention_days. Returns count deleted."""
...
# -- Watches ---------------------------------------------------------------
def create_watch(
self,
watch_id: str,
ws_id: str,
node_id: str,
name: str,
command: str,
interval_secs: float,
stop_on: str | None,
max_polls: int,
created_by: str,
next_poll: str,
) -> None:
"""Create a watch. No-op if watch_id already exists."""
...
def get_watch(self, watch_id: str) -> dict[str, Any] | None:
"""Return watch dict or None."""
...
def list_watches_for_ws(self, ws_id: str) -> list[dict[str, Any]]:
"""Return active watches for a workstream, ordered by created DESC."""
...
def list_watches_for_node(self, node_id: str) -> list[dict[str, Any]]:
"""Return all active watches on a node, ordered by created DESC."""
...
def list_due_watches(self, now: str) -> list[dict[str, Any]]:
"""Return active watches whose next_poll <= now, ordered by next_poll."""
...
def update_watch(self, watch_id: str, **fields: Any) -> bool:
"""Update specified fields on a watch. Returns True if found."""
...
def delete_watch(self, watch_id: str) -> bool:
"""Delete a watch. Returns True if found."""
...
def delete_watches_for_ws(self, ws_id: str) -> int:
"""Delete all watches for a workstream. Returns count deleted."""
...
# -- Service registry ------------------------------------------------------
def register_service(
+34
View File
@@ -170,6 +170,40 @@ sa.Index("idx_scheduled_task_runs_started", scheduled_task_runs.c.started)
# Service registry
# ---------------------------------------------------------------------------
# ---------------------------------------------------------------------------
# Watches — in-session periodic command polling
# ---------------------------------------------------------------------------
watches = sa.Table(
"watches",
metadata,
sa.Column("watch_id", sa.Text, primary_key=True),
sa.Column("ws_id", sa.Text, nullable=False),
sa.Column("node_id", sa.Text, nullable=False, server_default=""),
sa.Column("name", sa.Text, nullable=False),
sa.Column("command", sa.Text, nullable=False),
sa.Column("interval_secs", sa.Float, nullable=False),
sa.Column("stop_on", sa.Text), # Python expression, NULL = change detection
sa.Column("max_polls", sa.Integer, nullable=False, server_default="100"),
sa.Column("poll_count", sa.Integer, nullable=False, server_default="0"),
sa.Column("last_output", sa.Text),
sa.Column("last_exit_code", sa.Integer),
sa.Column("last_poll", sa.Text), # ISO8601
sa.Column("next_poll", sa.Text), # ISO8601
sa.Column("active", sa.Integer, nullable=False, server_default="1"),
sa.Column("created_by", sa.Text, nullable=False, server_default=""),
sa.Column("created", sa.Text, nullable=False),
sa.Column("updated", sa.Text, nullable=False),
)
sa.Index("idx_watches_active_next", watches.c.active, watches.c.next_poll)
sa.Index("idx_watches_ws_id", watches.c.ws_id)
sa.Index("idx_watches_node_id", watches.c.node_id)
# ---------------------------------------------------------------------------
# Service registry
# ---------------------------------------------------------------------------
services = sa.Table(
"services",
metadata,
+130
View File
@@ -1062,6 +1062,136 @@ class SQLiteBackend:
conn.commit()
return result.rowcount
# -- Watches ---------------------------------------------------------------
def create_watch(
self,
watch_id: str,
ws_id: str,
node_id: str,
name: str,
command: str,
interval_secs: float,
stop_on: str | None,
max_polls: int,
created_by: str,
next_poll: str,
) -> None:
from turnstone.core.storage._schema import watches
now = datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%S")
with self._engine.connect() as conn:
conn.execute(
sa.insert(watches).prefix_with("OR IGNORE"),
{
"watch_id": watch_id,
"ws_id": ws_id,
"node_id": node_id,
"name": name,
"command": command,
"interval_secs": interval_secs,
"stop_on": stop_on,
"max_polls": max_polls,
"poll_count": 0,
"active": 1,
"created_by": created_by,
"next_poll": next_poll,
"created": now,
"updated": now,
},
)
conn.commit()
def get_watch(self, watch_id: str) -> dict[str, Any] | None:
from turnstone.core.storage._schema import watches
with self._engine.connect() as conn:
row = conn.execute(sa.select(watches).where(watches.c.watch_id == watch_id)).fetchone()
if row is None:
return None
return dict(row._mapping)
def list_watches_for_ws(self, ws_id: str) -> list[dict[str, Any]]:
from turnstone.core.storage._schema import watches
with self._engine.connect() as conn:
rows = conn.execute(
sa.select(watches)
.where((watches.c.ws_id == ws_id) & (watches.c.active == 1))
.order_by(watches.c.created.desc())
).fetchall()
return [dict(r._mapping) for r in rows]
def list_watches_for_node(self, node_id: str) -> list[dict[str, Any]]:
from turnstone.core.storage._schema import watches
with self._engine.connect() as conn:
rows = conn.execute(
sa.select(watches)
.where((watches.c.node_id == node_id) & (watches.c.active == 1))
.order_by(watches.c.created.desc())
).fetchall()
return [dict(r._mapping) for r in rows]
def list_due_watches(self, now: str) -> list[dict[str, Any]]:
from turnstone.core.storage._schema import watches
with self._engine.connect() as conn:
rows = conn.execute(
sa.select(watches)
.where(
(watches.c.active == 1)
& (watches.c.next_poll <= now)
& (watches.c.next_poll != "")
)
.order_by(watches.c.next_poll)
.limit(100)
).fetchall()
return [dict(r._mapping) for r in rows]
_UPDATABLE_WATCH_FIELDS = frozenset(
{
"name",
"poll_count",
"last_output",
"last_exit_code",
"last_poll",
"next_poll",
"active",
"updated",
}
)
def update_watch(self, watch_id: str, **fields: Any) -> bool:
from turnstone.core.storage._schema import watches
fields = {k: v for k, v in fields.items() if k in self._UPDATABLE_WATCH_FIELDS}
fields["updated"] = datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%S")
if "active" in fields:
fields["active"] = 1 if fields["active"] else 0
with self._engine.connect() as conn:
result = conn.execute(
sa.update(watches).where(watches.c.watch_id == watch_id).values(**fields)
)
conn.commit()
return result.rowcount > 0
def delete_watch(self, watch_id: str) -> bool:
from turnstone.core.storage._schema import watches
with self._engine.connect() as conn:
result = conn.execute(sa.delete(watches).where(watches.c.watch_id == watch_id))
conn.commit()
return result.rowcount > 0
def delete_watches_for_ws(self, ws_id: str) -> int:
from turnstone.core.storage._schema import watches
with self._engine.connect() as conn:
result = conn.execute(sa.delete(watches).where(watches.c.ws_id == ws_id))
conn.commit()
return result.rowcount
# -- Service registry ------------------------------------------------------
def register_service(
@@ -0,0 +1,47 @@
"""Watches table for in-session periodic command polling.
Revision ID: 007
Revises: 006
Create Date: 2026-03-09
"""
import sqlalchemy as sa
from alembic import op
revision = "007"
down_revision = "006"
branch_labels = None
depends_on = None
def upgrade() -> None:
op.create_table(
"watches",
sa.Column("watch_id", sa.Text, primary_key=True),
sa.Column("ws_id", sa.Text, nullable=False),
sa.Column("node_id", sa.Text, nullable=False, server_default=""),
sa.Column("name", sa.Text, nullable=False),
sa.Column("command", sa.Text, nullable=False),
sa.Column("interval_secs", sa.Float, nullable=False),
sa.Column("stop_on", sa.Text),
sa.Column("max_polls", sa.Integer, nullable=False, server_default="100"),
sa.Column("poll_count", sa.Integer, nullable=False, server_default="0"),
sa.Column("last_output", sa.Text),
sa.Column("last_exit_code", sa.Integer),
sa.Column("last_poll", sa.Text),
sa.Column("next_poll", sa.Text),
sa.Column("active", sa.Integer, nullable=False, server_default="1"),
sa.Column("created_by", sa.Text, nullable=False, server_default=""),
sa.Column("created", sa.Text, nullable=False),
sa.Column("updated", sa.Text, nullable=False),
)
op.create_index("idx_watches_active_next", "watches", ["active", "next_poll"])
op.create_index("idx_watches_ws_id", "watches", ["ws_id"])
op.create_index("idx_watches_node_id", "watches", ["node_id"])
def downgrade() -> None:
op.drop_index("idx_watches_node_id", "watches")
op.drop_index("idx_watches_ws_id", "watches")
op.drop_index("idx_watches_active_next", "watches")
op.drop_table("watches")
+442
View File
@@ -0,0 +1,442 @@
"""Watch — periodic command polling within a workstream.
A watch periodically runs a shell command and injects results back into the
conversation when a stop condition is met or the output changes. The
``WatchRunner`` is a server-level daemon thread that polls the database for
due watches, runs their commands, and dispatches results.
"""
from __future__ import annotations
import contextlib
import json
import logging
import re
import subprocess
import threading
from datetime import UTC, datetime, timedelta
from typing import TYPE_CHECKING, Any
from turnstone.core.safety import is_command_blocked, sanitize_command
if TYPE_CHECKING:
from collections.abc import Callable
log = logging.getLogger(__name__)
# ---------------------------------------------------------------------------
# Constants
# ---------------------------------------------------------------------------
MAX_WATCHES_PER_WS = 5
MIN_INTERVAL = 10 # seconds
MAX_INTERVAL = 86_400 # 24 hours
DEFAULT_MAX_POLLS = 100
DEFAULT_INTERVAL = 300 # 5 minutes
MAX_OUTPUT_SIZE = 65_536 # truncate stored/dispatched output at 64 KB
# Safe builtins exposed to condition expressions.
_SAFE_BUILTINS: dict[str, Any] = {
"len": len,
"str": str,
"int": int,
"float": float,
"bool": bool,
"abs": abs,
"min": min,
"max": max,
"any": any,
"all": all,
"isinstance": isinstance,
"sorted": sorted,
"True": True,
"False": False,
"None": None,
}
# ---------------------------------------------------------------------------
# Duration parsing
# ---------------------------------------------------------------------------
_DURATION_RE = re.compile(r"(?:(\d+)\s*h)?\s*(?:(\d+)\s*m)?\s*(?:(\d+)\s*s)?$", re.IGNORECASE)
def parse_duration(s: str) -> float:
"""Convert a duration string to seconds.
Supported formats: ``"30s"``, ``"5m"``, ``"1h"``, ``"2h30m"``,
``"90"`` (bare number = seconds).
Raises ``ValueError`` on invalid input.
"""
s = s.strip()
if not s:
raise ValueError("empty duration string")
# Bare number → seconds
try:
val = float(s)
except ValueError:
val = None
if val is not None:
if val <= 0:
raise ValueError(f"duration must be positive, got {val}")
return val
m = _DURATION_RE.match(s)
if not m or not any(m.groups()):
raise ValueError(f"invalid duration format: {s!r}")
hours = int(m.group(1) or 0)
minutes = int(m.group(2) or 0)
seconds = int(m.group(3) or 0)
total = hours * 3600 + minutes * 60 + seconds
if total <= 0:
raise ValueError(f"duration must be positive, got {total}s")
return float(total)
# ---------------------------------------------------------------------------
# Condition evaluation
# ---------------------------------------------------------------------------
def validate_condition(expr: str) -> str | None:
"""Syntax-check a condition expression.
Returns an error message string, or ``None`` if the expression is valid.
"""
try:
compile(expr, "<watch>", "eval")
except SyntaxError as exc:
return f"invalid condition syntax: {exc}"
return None
def evaluate_condition(
expr: str | None,
output: str,
exit_code: int,
prev_output: str | None,
) -> tuple[bool, str]:
"""Evaluate a stop condition.
Returns ``(fired, reason)`` where *fired* is ``True`` when the watch
should report a result and *reason* is a human-readable explanation.
"""
changed = output != prev_output
if expr is None:
# Default: fire on any change (skip first poll where prev is None)
if prev_output is None:
return False, ""
return changed, "output changed" if changed else ""
# Build data context
data: Any = None
with contextlib.suppress(json.JSONDecodeError, ValueError):
data = json.loads(output)
context = {
"output": output,
"data": data,
"exit_code": exit_code,
"prev_output": prev_output,
"changed": changed,
}
try:
result = eval(expr, {"__builtins__": _SAFE_BUILTINS}, context) # noqa: S307
if result:
return True, f"condition met: {expr}"
return False, ""
except Exception as exc:
log.warning("watch.condition_error", extra={"expr": expr, "error": str(exc)})
return False, f"condition error: {exc}"
# ---------------------------------------------------------------------------
# Message formatting
# ---------------------------------------------------------------------------
def format_watch_message(
name: str,
command: str,
output: str,
poll_count: int,
max_polls: int,
elapsed_secs: float,
stop_on: str | None,
is_final: bool,
reason: str,
) -> str:
"""Format a watch result as a synthetic user message."""
elapsed = format_interval(elapsed_secs)
lines = [f'[Watch "{name}" \u2014 poll #{poll_count}/{max_polls}, {elapsed} elapsed]']
# Show the condition so the model knows what this watch was waiting for
if stop_on:
lines.append(f"[condition: {stop_on}]")
else:
lines.append("[mode: fire on output change]")
lines.append("")
lines.append(f"$ {command}")
lines.append(output)
if is_final:
if reason:
lines.append("")
lines.append(f"[{reason} \u2014 watch auto-cancelled]")
else:
lines.append("")
lines.append("[max polls reached \u2014 watch auto-cancelled]")
return "\n".join(lines)
def format_interval(secs: float) -> str:
"""Human-readable duration (e.g. ``'5m'``, ``'1h30m'``)."""
if secs < 60:
return f"{secs:.0f}s"
if secs < 3600:
return f"{secs / 60:.0f}m"
hours = int(secs // 3600)
mins = int((secs % 3600) // 60)
if mins:
return f"{hours}h{mins}m"
return f"{hours}h"
# ---------------------------------------------------------------------------
# WatchRunner — server-level daemon thread
# ---------------------------------------------------------------------------
class WatchRunner:
"""Polls the database for due watches and dispatches results.
Runs as a daemon thread in the server process, analogous to
``TaskScheduler`` in the console.
"""
def __init__(
self,
storage: Any,
node_id: str,
*,
check_interval: float = 15.0,
tool_timeout: float = 30.0,
restore_fn: Callable[[str], Callable[[str], None] | None] | None = None,
) -> None:
self._storage = storage
self._node_id = node_id
self._check_interval = check_interval
self._tool_timeout = tool_timeout
self._restore_fn = restore_fn
self._dispatch_fns: dict[str, Callable[[str], None]] = {}
self._dispatch_lock = threading.Lock()
self._stop_event = threading.Event()
self._thread: threading.Thread | None = None
# -- Lifecycle -----------------------------------------------------------
def start(self) -> None:
if self._thread is not None:
return
self._stop_event.clear()
self._thread = threading.Thread(target=self._run, daemon=True, name="watch-runner")
self._thread.start()
log.info("watch_runner.started", extra={"node_id": self._node_id})
def stop(self) -> None:
self._stop_event.set()
if self._thread is not None:
self._thread.join(timeout=self._check_interval + 5)
self._thread = None
log.info("watch_runner.stopped")
# -- Dispatch function registry ------------------------------------------
def set_dispatch_fn(self, ws_id: str, fn: Callable[[str], None]) -> None:
with self._dispatch_lock:
self._dispatch_fns[ws_id] = fn
def remove_dispatch_fn(self, ws_id: str) -> None:
with self._dispatch_lock:
self._dispatch_fns.pop(ws_id, None)
# -- Main loop -----------------------------------------------------------
def _run(self) -> None:
while not self._stop_event.is_set():
try:
self._tick()
except Exception:
log.exception("watch_runner.tick_error")
self._stop_event.wait(self._check_interval)
def _tick(self) -> None:
if self._storage is None:
return
now = datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%S")
due = self._storage.list_due_watches(now)
for watch_row in due:
if self._stop_event.is_set():
break
# Only poll watches owned by this node
row_node = watch_row.get("node_id", "")
if row_node and row_node != self._node_id:
continue
try:
self._poll_watch(watch_row)
except Exception:
log.exception(
"watch_runner.poll_error",
extra={"watch_id": watch_row.get("watch_id")},
)
def _poll_watch(self, watch_row: dict[str, Any]) -> None:
watch_id = watch_row["watch_id"]
ws_id = watch_row["ws_id"]
command = watch_row["command"]
stop_on = watch_row.get("stop_on")
max_polls = watch_row.get("max_polls", DEFAULT_MAX_POLLS)
poll_count = watch_row.get("poll_count", 0) + 1
prev_output = watch_row.get("last_output")
created = watch_row.get("created", "")
# Safety check
blocked = is_command_blocked(command)
if blocked:
log.warning(
"watch_runner.blocked_command", extra={"watch_id": watch_id, "reason": blocked}
)
self._deactivate_watch(watch_id)
return
# Run command
output, exit_code = self._run_command(sanitize_command(command))
# Truncate to avoid unbounded storage / context window usage
if len(output) > MAX_OUTPUT_SIZE:
output = output[:MAX_OUTPUT_SIZE] + f"\n[truncated at {MAX_OUTPUT_SIZE} bytes]"
# Evaluate condition
fired, reason = evaluate_condition(stop_on, output, exit_code, prev_output)
# Treat condition evaluation errors as terminal — don't silently
# loop until max_polls while the user/model never sees the problem.
if not fired and reason.startswith("condition error:"):
fired = True
# Check max polls
is_final = fired or poll_count >= max_polls
if not fired and poll_count >= max_polls:
reason = "max polls reached"
is_final = True
now = datetime.now(UTC)
now_str = now.strftime("%Y-%m-%dT%H:%M:%S")
# Update DB
update_fields: dict[str, Any] = {
"poll_count": poll_count,
"last_output": output,
"last_exit_code": exit_code,
"last_poll": now_str,
}
if is_final:
update_fields["active"] = False
update_fields["next_poll"] = ""
else:
next_poll = now + timedelta(seconds=watch_row["interval_secs"])
update_fields["next_poll"] = next_poll.strftime("%Y-%m-%dT%H:%M:%S")
self._storage.update_watch(watch_id, **update_fields)
# Dispatch result if condition fired or final
if fired or is_final:
# Compute elapsed from created time
elapsed_secs = 0.0
if created:
try:
created_dt = datetime.fromisoformat(created).replace(tzinfo=UTC)
elapsed_secs = (now - created_dt).total_seconds()
except (ValueError, TypeError):
pass
message = format_watch_message(
name=watch_row["name"],
command=command,
output=output,
poll_count=poll_count,
max_polls=max_polls,
elapsed_secs=elapsed_secs,
stop_on=stop_on,
is_final=is_final,
reason=reason,
)
self._dispatch_result(ws_id, message)
log.debug(
"watch_runner.polled",
extra={
"watch_id": watch_id,
"poll_count": poll_count,
"fired": fired,
"is_final": is_final,
},
)
def _run_command(self, command: str) -> tuple[str, int]:
"""Run a shell command and return (stdout, exit_code)."""
try:
proc = subprocess.run(
command,
shell=True,
capture_output=True,
text=True,
timeout=self._tool_timeout,
start_new_session=True,
)
output = proc.stdout
if proc.stderr:
output = output + "\n[stderr]\n" + proc.stderr if output else proc.stderr
return output, proc.returncode
except subprocess.TimeoutExpired:
return f"[command timed out after {self._tool_timeout}s]", -1
except Exception as exc:
return f"[command failed: {exc}]", -1
def _dispatch_result(self, ws_id: str, message: str) -> None:
"""Deliver a watch result to the owning workstream."""
with self._dispatch_lock:
fn = self._dispatch_fns.get(ws_id)
if fn is not None:
try:
fn(message)
return
except Exception:
log.exception("watch_runner.dispatch_error", extra={"ws_id": ws_id})
# Workstream may be evicted — try to restore
if self._restore_fn is not None:
try:
restored_fn = self._restore_fn(ws_id)
if restored_fn is not None:
restored_fn(message)
return
except Exception:
log.exception("watch_runner.restore_error", extra={"ws_id": ws_id})
log.warning(
"watch_runner.dispatch_failed",
extra={"ws_id": ws_id, "reason": "no dispatch function and restore failed"},
)
def _deactivate_watch(self, watch_id: str) -> None:
self._storage.update_watch(watch_id, active=False, next_poll="")
+15
View File
@@ -213,6 +213,21 @@ class WorkstreamManager:
ws.ui._plan_event.set()
if hasattr(ws.ui, "_fg_event"):
ws.ui._fg_event.set()
# Notify SSE listeners so generators exit promptly
if hasattr(ws.ui, "_listeners_lock"):
import contextlib
import queue as _queue
with ws.ui._listeners_lock:
for lq in ws.ui._listeners: # type: ignore[attr-defined]
try:
lq.put_nowait({"type": "ws_closed"})
except _queue.Full:
with contextlib.suppress(_queue.Empty):
lq.get_nowait()
with contextlib.suppress(_queue.Full):
lq.put_nowait({"type": "ws_closed"})
ws.ui._listeners.clear() # type: ignore[attr-defined]
# Release MCP listener registration
if ws.session and hasattr(ws.session, "close"):
ws.session.close()
+736 -93
View File
File diff suppressed because it is too large Load Diff
+9 -1
View File
@@ -206,9 +206,17 @@ class Bridge:
data = resp.json()
for ws in data.get("workstreams", []):
ws_id = ws["id"]
log.info("Recovered workstream %s (%s)", ws_id, ws.get("name", ""))
ws_name = ws.get("name", "")
log.info("Recovered workstream %s (%s)", ws_id, ws_name)
self._broker.set_ws_owner(ws_id, self._node_id)
self._start_ws_sse(ws_id)
self._publish_cluster(
WorkstreamCreatedEvent(
ws_id=ws_id,
name=ws_name,
node_id=self._node_id,
)
)
except Exception as exc:
log.warning("Could not recover workstreams: %s", exc)
+9
View File
@@ -16,6 +16,7 @@ from typing import TYPE_CHECKING, Any
from turnstone.api.console_schemas import (
ClusterNodesResponse,
ClusterOverviewResponse,
ClusterSnapshotResponse,
ClusterWorkstreamsResponse,
ConsoleCreateWsResponse,
ConsoleHealthResponse,
@@ -102,6 +103,11 @@ class AsyncTurnstoneConsole(_BaseClient):
"GET", f"/v1/api/cluster/node/{node_id}", response_model=NodeDetailResponse
)
async def snapshot(self) -> ClusterSnapshotResponse:
return await self._request(
"GET", "/v1/api/cluster/snapshot", response_model=ClusterSnapshotResponse
)
async def create_workstream(
self,
*,
@@ -342,6 +348,9 @@ class TurnstoneConsole:
def node_detail(self, node_id: str) -> NodeDetailResponse:
return self._runner.run(self._async.node_detail(node_id))
def snapshot(self) -> ClusterSnapshotResponse:
return self._runner.run(self._async.snapshot())
def create_workstream(
self,
*,
+9
View File
@@ -247,6 +247,14 @@ class ClusterWsRenameEvent(ClusterEvent):
name: str = ""
@dataclass
class ClusterSnapshotEvent(ClusterEvent):
type: str = "snapshot"
nodes: list[dict[str, Any]] = field(default_factory=list)
overview: dict[str, Any] = field(default_factory=dict)
timestamp: float = 0.0
# ---------------------------------------------------------------------------
# Type registries (built after all classes are defined)
# ---------------------------------------------------------------------------
@@ -296,5 +304,6 @@ _CLUSTER_REGISTRY: dict[str, type[ClusterEvent]] = {
ClusterWsCreatedEvent,
ClusterWsClosedEvent,
ClusterWsRenameEvent,
ClusterSnapshotEvent,
]
}
+147 -20
View File
@@ -82,8 +82,8 @@ class WebUI:
def __init__(self, ws_id: str = "") -> None:
self.ws_id = ws_id
self._event_queue: queue.Queue[dict[str, Any]] = queue.Queue()
self._sse_generation = 0 # incremented on each new SSE connection
self._listeners: list[queue.Queue[dict[str, Any]]] = []
self._listeners_lock = threading.Lock()
self._approval_event = threading.Event()
self._approval_result: tuple[bool, str | None] = (False, None)
self._pending_approval: dict[str, Any] | None = None # re-sent on SSE reconnect
@@ -102,7 +102,23 @@ class WebUI:
self._ws_activity_state: str = "" # "tool" | "approval" | "thinking" | ""
def _enqueue(self, data: dict[str, Any]) -> None:
self._event_queue.put(data)
with self._listeners_lock:
snapshot = list(self._listeners)
for lq in snapshot:
with contextlib.suppress(queue.Full):
lq.put_nowait(data)
def _register_listener(self) -> queue.Queue[dict[str, Any]]:
"""Create a per-client queue and register it as a listener."""
client_queue: queue.Queue[dict[str, Any]] = queue.Queue(maxsize=500)
with self._listeners_lock:
self._listeners.append(client_queue)
return client_queue
def _unregister_listener(self, client_queue: queue.Queue[dict[str, Any]]) -> None:
"""Remove a client queue from the listeners list."""
with self._listeners_lock, contextlib.suppress(ValueError):
self._listeners.remove(client_queue)
def _broadcast_state(self, state: str) -> None:
"""Send a state-change event to the global SSE channel."""
@@ -473,16 +489,8 @@ async def events_sse(request: Request) -> Response:
if not ws or not ui:
return JSONResponse({"error": "Unknown workstream"}, status_code=404)
ui._sse_generation += 1
my_gen = ui._sse_generation
# Drain stale events. A race with the worker thread is acceptable:
# worst case we discard one fresh event, and the client catches up
# via the history replay above.
while not ui._event_queue.empty():
try:
ui._event_queue.get_nowait()
except queue.Empty:
break
# Each client gets its own queue — no drain needed.
client_queue = ui._register_listener()
async def event_generator() -> AsyncGenerator[dict[str, str], None]:
assert ws.session is not None
@@ -509,18 +517,19 @@ async def events_sse(request: Request) -> Response:
_metrics.record_sse_connect()
try:
loop = asyncio.get_running_loop()
while my_gen == ui._sse_generation:
while True:
try:
event = await loop.run_in_executor(
None, functools.partial(ui._event_queue.get, timeout=5)
None, functools.partial(client_queue.get, timeout=5)
)
if event.get("type") == "ws_closed":
return
yield {"data": json.dumps(event)}
except queue.Empty:
pass
if await request.is_disconnected():
break
finally:
_metrics.record_sse_disconnect()
ui._unregister_listener(client_queue)
return EventSourceResponse(event_generator(), ping=5)
@@ -545,8 +554,6 @@ async def global_events_sse(request: Request) -> Response:
yield {"data": json.dumps(event)}
except queue.Empty:
pass
if await request.is_disconnected():
break
finally:
_metrics.record_sse_disconnect()
with listeners_lock:
@@ -705,6 +712,35 @@ async def metrics_endpoint(request: Request) -> Response:
return Response(content, media_type="text/plain; version=0.0.4; charset=utf-8")
def _make_watch_dispatch(ws: Workstream, session: ChatSession, ui: Any) -> Any:
"""Create a dispatch function for watch results on a workstream.
Handles both idle (start worker thread) and busy (enqueue for IDLE drain)
cases. Mirrors the ``send_message`` worker-thread pattern.
"""
pending = session._watch_pending
def dispatch(msg: str) -> None:
if ws.worker_thread and ws.worker_thread.is_alive():
# Workstream is busy — queue for drain at IDLE (Path A)
pending.put({"message": msg})
return
# Workstream is idle — start a worker thread (Path B)
def run() -> None:
try:
session.send(msg)
except Exception as exc:
if ui:
ui.on_error(f"Watch error: {exc}")
t = threading.Thread(target=run, daemon=True)
ws.worker_thread = t
t.start()
return dispatch
async def send_message(request: Request) -> JSONResponse:
"""POST /v1/api/send — send a user message to the workstream."""
from turnstone.core.web_helpers import read_json_or_400
@@ -850,6 +886,12 @@ async def create_workstream(request: Request) -> JSONResponse:
assert isinstance(ws.ui, WebUI)
if skip or body.get("auto_approve", False):
ws.ui.auto_approve = True
# Register watch runner for this workstream
runner = getattr(request.app.state, "watch_runner", None)
if runner and ws.session:
ws.session.set_watch_runner(
runner, dispatch_fn=_make_watch_dispatch(ws, ws.session, ws.ui)
)
# Emit eviction event if a workstream was evicted to make room
evicted = mgr.last_evicted
if evicted is not None:
@@ -908,6 +950,42 @@ async def close_workstream(request: Request) -> JSONResponse:
return JSONResponse({"error": "Cannot close last workstream"}, status_code=400)
async def list_watches(request: Request) -> JSONResponse:
"""GET /v1/api/watches — list active watches, optionally filtered by ws_id."""
from turnstone.core.storage._registry import get_storage
storage = get_storage()
if not storage:
return JSONResponse({"watches": []})
ws_id = request.query_params.get("ws_id")
if ws_id:
watches = storage.list_watches_for_ws(ws_id)
else:
node_id = getattr(request.app.state, "node_id", "")
watches = storage.list_watches_for_node(node_id) if node_id else []
return JSONResponse({"watches": watches})
async def cancel_watch(request: Request) -> JSONResponse:
"""POST /v1/api/watches/{watch_id}/cancel — cancel an active watch."""
from turnstone.core.storage._registry import get_storage
watch_id = request.path_params["watch_id"]
storage = get_storage()
if not storage:
return JSONResponse({"error": "Storage unavailable"}, status_code=500)
watch = storage.get_watch(watch_id)
if not watch:
return JSONResponse({"error": "Watch not found"}, status_code=404)
# Verify node ownership in multi-node deployments
node_id = getattr(request.app.state, "node_id", "")
watch_node = watch.get("node_id", "")
if watch_node and node_id and watch_node != node_id:
return JSONResponse({"error": "Watch belongs to another node"}, status_code=403)
storage.update_watch(watch_id, active=False, next_poll="")
return JSONResponse({"status": "ok", "watch_id": watch_id})
async def auth_login(request: Request) -> Response:
"""POST /v1/api/auth/login — authenticate and return JWT."""
from turnstone.core.auth import handle_auth_login
@@ -1009,8 +1087,13 @@ async def _lifespan(app: Starlette) -> AsyncGenerator[None, None]:
daemon=True,
)
cleanup.start()
# Start watch runner (periodic command polling)
if app.state.watch_runner:
app.state.watch_runner.start()
yield
# Shutdown
if app.state.watch_runner:
app.state.watch_runner.stop()
if app.state.health_monitor:
app.state.health_monitor.stop()
if app.state.mcp_client:
@@ -1060,6 +1143,7 @@ def create_app(
idle_timeout: int = 0,
node_id: str = "",
cors_origins: list[str] | None = None,
watch_runner: Any = None,
) -> Starlette:
"""Create and configure the Starlette ASGI application."""
_spec = build_server_spec()
@@ -1083,6 +1167,8 @@ def create_app(
Route("/api/command", command, methods=["POST"]),
Route("/api/workstreams/new", create_workstream, methods=["POST"]),
Route("/api/workstreams/close", close_workstream, methods=["POST"]),
Route("/api/watches", list_watches),
Route("/api/watches/{watch_id}/cancel", cancel_watch, methods=["POST"]),
Route("/api/auth/login", auth_login, methods=["POST"]),
Route("/api/auth/logout", auth_logout, methods=["POST"]),
Route("/api/auth/status", auth_status),
@@ -1113,6 +1199,7 @@ def create_app(
app.state.registry = registry
app.state.idle_timeout = idle_timeout
app.state.node_id = node_id
app.state.watch_runner = watch_runner
from turnstone.core.auth import LoginRateLimiter
@@ -1498,11 +1585,47 @@ def main() -> None:
tool_search_max_results=args.tool_search_max_results,
)
# Create workstream manager and initial workstream
# Create WatchRunner (periodic command polling, server-level)
from turnstone.core.storage import get_storage as _get_storage
from turnstone.core.watch import WatchRunner
# Create workstream manager first (watch restore_fn captures it)
manager = WorkstreamManager(
session_factory, max_workstreams=args.max_workstreams, node_id=_node_id
)
WebUI._workstream_mgr = manager
def _watch_restore_fn(ws_id: str) -> Any:
"""Restore an evicted workstream so a watch can deliver results.
Returns a callable that starts a worker thread to send() the watch
result. Unlike the normal dispatch path (which enqueues for IDLE
drain), the restored workstream has no active send() loop, so we
must start a worker thread directly same pattern as send_message().
"""
try:
ws = manager.create(
ui_factory=lambda wid: WebUI(ws_id=wid),
)
# Restored workstreams run unattended — auto-approve tool calls
# to avoid blocking forever on approval with no connected user.
if isinstance(ws.ui, WebUI):
ws.ui.auto_approve = True
if ws.session:
ws.session.resume(ws_id)
dispatch_fn = _make_watch_dispatch(ws, ws.session, ws.ui)
ws.session.set_watch_runner(_watch_runner, dispatch_fn=dispatch_fn)
return dispatch_fn
except RuntimeError:
log.warning("watch_restore: cannot restore ws %s (all slots active)", ws_id)
return None
_watch_runner = WatchRunner(
storage=_get_storage(),
node_id=_node_id,
tool_timeout=args.tool_timeout,
restore_fn=_watch_restore_fn,
)
ws = manager.create(
name="default",
ui_factory=lambda wid: WebUI(ws_id=wid),
@@ -1513,6 +1636,9 @@ def main() -> None:
# Handle --resume
assert ws.session is not None
ws.session.set_watch_runner(
_watch_runner, dispatch_fn=_make_watch_dispatch(ws, ws.session, ws.ui)
)
if args.resume:
from turnstone.core.memory import resolve_workstream
@@ -1558,6 +1684,7 @@ def main() -> None:
idle_timeout=args.workstream_idle_timeout,
node_id=_node_id,
cors_origins=cors_origins,
watch_runner=_watch_runner,
)
log.info("Server starting on http://%s:%s", args.host, args.port)
+15
View File
@@ -0,0 +1,15 @@
{
"name": "create_plan",
"description": "Create a structured plan before taking action. An autonomous agent explores the available context, identifies what needs to change, and writes a step-by-step plan. Call this tool when the user asks to plan, design, or think through an approach, or when a task is complex, touches multiple areas, or has unclear scope.",
"parameters": {
"type": "object",
"properties": {
"goal": {
"type": "string",
"description": "The goal and scope of the plan, including any constraints."
}
},
"required": ["goal"]
},
"primary_key": "goal"
}
-15
View File
@@ -1,15 +0,0 @@
{
"name": "plan",
"description": "Plan before implementing. An autonomous agent explores the codebase and writes a structured plan to .plan-<ws_id>.md (unique per workstream to avoid collisions). If a plan for this workstream already exists it is re-read and refined rather than overwritten from scratch. Use plan BEFORE writing code — when the user asks to build, add, refactor, or change something that touches multiple files or has unclear scope. The plan identifies files to modify, existing patterns to reuse, and risks to consider.",
"parameters": {
"type": "object",
"properties": {
"prompt": {
"type": "string",
"description": "What to plan — the goal, constraints, and scope."
}
},
"required": ["prompt"]
},
"primary_key": "prompt"
}
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "read_file",
"description": "Read the contents of a file. Returns numbered lines. Must be called before edit_file on the same path.",
"description": "Read the contents of a file. Returns numbered lines for text files. For image files (PNG, JPEG, GIF, WebP, BMP, TIFF, ICO), returns the image content if the model supports vision, or a text description otherwise. The offset and limit parameters apply to text files only.",
"parameters": {
"type": "object",
"properties": {
+36
View File
@@ -0,0 +1,36 @@
{
"name": "watch",
"description": "Set up periodic polling of a shell command within this workstream. Actions: 'create' starts a new watch, 'list' shows active watches, 'cancel' stops a watch. Watch results are injected into the conversation when the stop condition is met or output changes. Use for monitoring CI/CD, PR status, deployments, file changes, etc.",
"parameters": {
"type": "object",
"properties": {
"action": {
"type": "string",
"enum": ["create", "list", "cancel"],
"description": "Action to perform."
},
"command": {
"type": "string",
"description": "Shell command to poll (required for 'create')."
},
"poll_every": {
"type": "string",
"description": "Poll interval as duration (e.g., '30s', '5m', '1h'). Default: '5m'."
},
"stop_on": {
"type": "string",
"description": "Python expression evaluated after each poll. Variables: output (str), data (parsed JSON or None), exit_code (int), prev_output (str|None), changed (bool). Truthy result fires the watch and auto-cancels. Omit for change-detection mode (first poll establishes a baseline, subsequent polls fire when output differs). Examples: 'data[\"state\"] == \"MERGED\"', '\"error\" in output', 'exit_code != 0'."
},
"name": {
"type": "string",
"description": "Human-readable name (e.g., 'pr-review'). Required for 'create', used as identifier for 'cancel'."
},
"max_polls": {
"type": "integer",
"description": "Max poll cycles before auto-cancel. Default: 100."
}
},
"required": ["action"]
},
"primary_key": "command"
}
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "web_fetch",
"description": "Fetch a URL and extract specific information from it. You must provide a question or extraction guidance — the page is fetched, analyzed, and only relevant information is returned (not raw page content).",
"description": "Fetch a URL and extract specific information from it. You must provide a question or extraction guidance. The page is fetched, analyzed, and only relevant information is returned (not raw page content).",
"parameters": {
"type": "object",
"properties": {