Compare commits

...

10 Commits

Author SHA1 Message Date
Patrick Buckley 25b5e32089 Bump version to 0.4.2 2026-03-05 20:18:46 -08:00
Patrick Buckley 339981a258 Add GPT-5.3, GPT-5.4, and pro model capabilities (#28)
* Add GPT-5.3, GPT-5.4, and pro model capabilities

Add capability entries for gpt-5-pro (272k output, high-only reasoning),
gpt-5.2-pro, gpt-5.3, gpt-5.4 (1.05M context), and gpt-5.4-pro.

* Validate reasoning_effort against model capabilities

_apply_model_params now falls back to caps.default_reasoning_effort when
the requested value is not in caps.reasoning_effort_values. Prevents
sending unsupported effort levels to models like gpt-5-pro (high only).
2026-03-05 20:17:03 -08:00
Patrick Buckley 06de9ff83b Bump version to 0.4.1 2026-03-05 18:13:42 -08:00
Patrick Buckley 924b976f1f Add scheduled task docs and SDK client methods
Add documentation and SDK support for the scheduled task system
(cron/at scheduling via console API). Includes Python SDK methods
(async + sync), TypeScript SDK methods, console.md API reference,
sdk.md table update, and architecture.md module map entry.
2026-03-05 18:11:36 -08:00
Patrick Buckley d5db817391 Fix channel gateway Docker networking and console proxy approval scope
Two runtime bugs:

1. Channel gateway advertised http://127.0.0.1:8091 which is
   unreachable from other Docker containers. Add
   TURNSTONE_CHANNEL_ADVERTISE_URL env var override for Docker/K8s
   environments, set to http://channel:8091 in compose.yaml, and
   pass --http-host=0.0.0.0 so the gateway listens on all interfaces.

2. Console proxy service JWT had only "write" scope but the approval
   endpoint requires "approve". Tool approval buttons in the server
   web UI silently failed when accessed through the console proxy.
   Changed proxy token scopes to read+write+approve.
2026-03-05 18:01:52 -08:00
Patrick Buckley fc8ceb4c72 Update README: 3-node cluster diagram, remove directory tree
Replace single-node Mermaid diagram with a 3-node cluster layout
showing bridge+server pairs per node, shared Redis MQ, console, and
channel gateway. Remove the verbose directory tree listing.
2026-03-05 17:44:33 -08:00
Patrick Buckley 07234dec4d Replace ASCII architecture diagram with Mermaid in README
GitHub renders Mermaid natively as an interactive SVG. The new diagram
shows all client entry points (CLI, browser, SDK, Discord), the full
cluster topology including the channel gateway and notify path, and
the LLM provider layer.
2026-03-05 17:34:42 -08:00
Patrick Buckley dd4cc0b30d Add PROGRESS.md and .coverage to .gitignore 2026-03-05 17:31:55 -08:00
Patrick Buckley e7fe8fca9d Add channel notification tool with security hardening (#27)
* Add channel notification tool with security hardening

Implements the `notify` tool allowing the LLM to send notifications to
Discord channels/users via the channel gateway. Includes fixes for 11
review findings: JWT auth on the gateway endpoint, first-healthy gateway
delivery with retry+backoff, rate limiting only on success, SSRF URL
scheme validation, Discord mention sanitization, SQLite ON CONFLICT
upsert preserving created timestamps, advertise URL resolution for
0.0.0.0 bind, randomized service IDs, generic error messages to prevent
internal state leakage, and partial direct-target validation.

Service registry with heartbeat-based health filtering (migration 005).
Channel gateway registers on startup, heartbeats every 30s, deregisters
on shutdown. 70 new tests covering tool prepare/execute, HTTP endpoint
auth (static + JWT), storage CRUD, and retry behavior.

* Add notify documentation, diagrams, and review fixes

Documentation:
- New sequence diagram 17-notify-flow.puml showing end-to-end delivery
- Updated 16-channel-architecture.puml with services table, notify HTTP
  path, and Notification Flow note
- channels.md: Notifications section (targeting, delivery flow, service
  registry, security) and new config table entries
- tools.md: notify tool reference, updated counts/tables (14→15 tools)
- security.md: channel gateway row in service-to-service auth table
- architecture.md: notification subsystem paragraph

Review fixes (copilot):
- _http.py: fail closed when auth unconfigured (401 instead of pass-
  through), strip whitespace on message/title, generic error messages
  for user-not-found vs no-linked-channels
- session.py: parse gateway response JSON and require at least one
  result with status=="sent" before counting as success
- _postgresql.py: use index_elements instead of constraint for upsert
2026-03-05 17:24:00 -08:00
Patrick Buckley 42b9f89988 Add scheduled task system with cron/at scheduling (#26)
* Add scheduled task system with cron/at scheduling, admin API, and console UI

Console-integrated background scheduler dispatches workstreams on recurring
cron expressions or one-shot ISO8601 timestamps. Four target modes: auto
(best node by headroom), pool (shared queue), all (fan-out), or specific
node. Redis distributed lock with unique owner + Lua conditional release
prevents duplicate dispatch in multi-console deployments.

Storage: scheduled_tasks + scheduled_task_runs tables (migration 004),
9 protocol methods on both SQLite and PostgreSQL backends, field allowlist
on updates, run history auto-pruned at 90 days.

API: 6 CRUD endpoints under /v1/api/admin/schedules with croniter
validation, ISO8601 future-time checks, field length bounds, schedule
count cap (200), and OpenAPI spec entries with Pydantic models.

UI: Schedules tab in admin panel with create/edit/delete modals, run
history modal, cron/at type toggle, target mode select, status dots for
accessibility, responsive grid, keyboard navigation, and focus management.

Security: fan-out capped at 20 nodes/task/tick, auto-approve dispatches
logged at WARNING with created_by attribution, user_id propagated in
CreateWorkstreamMessage for audit trail.

46 tests across storage, scheduler engine, and API endpoints.

* Add croniter to test extras for CI compatibility

CI installs [test] extras but not [console], so croniter was missing
when schedule API tests import console/server.py validation functions.

* Address Copilot review: timezone validation, focus trap, enabled flag

- Reject naive at_time timestamps — require timezone offset (e.g. +00:00 or Z)
- UI appends +00:00 to datetime-local values for explicit UTC
- Fix datetime-local normalization: check length before appending seconds
- Add textarea to modal focus trap selector (prevents focus escape)
- Fix _normalize_task_dict not called in update response
- Persist enabled=false on create (storage defaults to enabled=1)
- Validate at_time is still in future when re-enabling a one-shot task
- broker._redis coupling acknowledged as tracked tech debt
2026-03-05 16:05:00 -08:00
50 changed files with 5671 additions and 103 deletions
+2
View File
@@ -17,3 +17,5 @@ venv/
.plan.md
.plan-*.md
.hypothesis/
PROGRESS.md
.coverage
+47 -69
View File
@@ -19,12 +19,52 @@ Turnstone gives LLMs tools — shell, files, search, web, planning — and orche
- **Cluster dashboard** — real-time view of all nodes and workstreams, workstream creation with node targeting, reverse proxy for server UIs (only the console port needs network access)
- **Cluster simulator** — test the stack at scale (up to 1000 nodes) without an LLM backend
```
External System → Message Queue → Bridge (per node) → Turnstone Server → LLM + Tools
Pub/Sub → Progress Events → External System
turnstone-console → Cluster Dashboard (browser)
```mermaid
graph LR
subgraph Clients
CLI[turnstone CLI]
UI[Browser UI]
SDK[SDK / API]
Discord[Discord / Slack]
end
Console[turnstone-console<br/><i>dashboard + proxy</i>]
Channel[turnstone-channel<br/><i>platform gateway</i>]
subgraph Cluster
Redis[(Redis MQ)]
DB[(PostgreSQL / SQLite)]
subgraph Node A
BridgeA[bridge]
ServerA[server]
end
subgraph Node B
BridgeB[bridge]
ServerB[server]
end
end
LLM[LLM Provider<br/><i>OpenAI · Anthropic · local</i>]
CLI --> ServerA
UI --> ServerB
SDK --> Redis
Discord --> Channel
Channel <--> Redis
Console --> Redis
Redis --> BridgeA & BridgeB
BridgeA --> ServerA
BridgeB --> ServerB
ServerA & ServerB --> LLM
ServerA & ServerB --> DB
Console --> DB
Channel --> DB
ServerA -.->|notify| Channel
BridgeA & BridgeB -.->|events| Redis
Console -.->|proxy| ServerA & ServerB
```
## Quickstart
@@ -111,69 +151,7 @@ All frontends connect to any OpenAI-compatible API (vLLM, NVIDIA NIM/NGC, llama.
## Architecture
```
turnstone/
├── core/ # UI-agnostic engine
│ ├── session.py # ChatSession — multi-turn loop, tool dispatch, agents
│ ├── providers/ # LLM provider adapters (OpenAI, Anthropic)
│ │ ├── _protocol.py # LLMProvider protocol, ModelCapabilities, StreamChunk
│ │ ├── _openai.py # OpenAI-compatible (OpenAI, vLLM, llama.cpp)
│ │ └── _anthropic.py # Anthropic Messages API (native streaming, thinking)
│ ├── tools.py # Tool definitions (auto-loaded from JSON)
│ ├── workstream.py # WorkstreamManager — parallel independent sessions
│ ├── mcp_client.py # MCP client manager (external tool servers)
│ ├── model_registry.py # ModelRegistry — named models, fallback routing, per-workstream selection
│ ├── config.py # Unified TOML config (~/.config/turnstone/config.toml)
│ ├── memory.py # Persistence facade (delegates to storage/)
│ ├── storage/ # Pluggable storage backend (SQLite + PostgreSQL)
│ ├── metrics.py # Prometheus-compatible metrics collector
│ ├── healthcheck.py # Backend health monitor + circuit breaker
│ ├── ratelimit.py # Per-IP token-bucket rate limiter
│ ├── edit.py # File editing (fuzzy match, indentation)
│ ├── safety.py # Path validation, sandbox checks
│ ├── sandbox.py # Command sandboxing
│ └── web.py # Web fetch/search helpers
├── mq/ # Message queue integration
│ ├── protocol.py # Typed message dataclasses (JSON serialization)
│ ├── broker.py # Abstract MessageBroker + RedisBroker
│ ├── bridge.py # Bridge service (queue ↔ HTTP API, multi-node routing)
│ └── client.py # TurnstoneClient — Python API for external systems
├── console/ # Cluster dashboard
│ ├── collector.py # ClusterCollector — aggregates all nodes via Redis + HTTP
│ ├── server.py # Dashboard Starlette/ASGI server + SSE
│ └── static/ # Cluster dashboard web UI
├── tools/ # Tool schemas (one JSON file per tool)
├── ui/ # Frontend assets and terminal rendering
│ └── static/ # Web UI (HTML, CSS, JS)
├── sim/ # Cluster simulator
│ ├── cluster.py # SimCluster — orchestrates N nodes + dispatchers
│ ├── node.py # SimNode + SimWorkstream — protocol-compatible node
│ ├── engine.py # LLM + tool execution simulation
│ ├── scenario.py # 5 workload scenarios (steady, burst, node_failure, …)
│ ├── metrics.py # Latency, throughput, utilization collection
│ └── cli.py # CLI entry point (turnstone-sim)
├── cli.py # Terminal frontend (+ /cluster commands for console)
├── server.py # Web frontend (Starlette/ASGI + SSE)
└── eval.py # Evaluation and prompt optimization harness
├── api/ # OpenAPI spec generation (Pydantic v2 models)
├── sdk/ # Client SDKs (sync + async, Python)
docs/
├── architecture.md # System architecture and threading model
├── api-reference.md # Web server API and SSE event reference
├── sdk.md # Client SDK reference (Python + TypeScript)
├── console.md # Cluster dashboard service (turnstone-console)
├── docker.md # Docker Compose deployment and configuration
├── simulator.md # Cluster simulator usage and scenarios
├── tools.md # Tool schemas, execution pipeline, approval flow
├── eval.md # Evaluation harness internals
└── diagrams/ # UML architecture diagrams (PlantUML sources + PNGs)
└── png/ # Pre-rendered diagram images
deploy/
├── helm/turnstone/ # Helm chart for Kubernetes
└── terraform/ # Terraform modules (AWS ECS/Fargate)
```
### Architecture Diagrams
### Diagrams
Detailed UML diagrams are available in [`docs/diagrams/`](docs/diagrams/):
+2
View File
@@ -213,6 +213,7 @@ services:
turnstone-channel
--redis-host=redis
--redis-port=6379
--http-host=0.0.0.0
$${TURNSTONE_DISCORD_GUILD:+--discord-guild $$TURNSTONE_DISCORD_GUILD}
environment:
- TURNSTONE_DISCORD_TOKEN=${TURNSTONE_DISCORD_TOKEN:-}
@@ -221,6 +222,7 @@ services:
- TURNSTONE_JWT_SECRET=${TURNSTONE_JWT_SECRET:-}
- TURNSTONE_DB_BACKEND=${DB_BACKEND:-sqlite}
- TURNSTONE_DB_URL=${DATABASE_URL:-}
- TURNSTONE_CHANNEL_ADVERTISE_URL=http://channel:8091
networks:
- turnstone-net
depends_on:
+15
View File
@@ -75,6 +75,7 @@ turnstone/
client.py TurnstoneClient library + TurnResult for MQ-based access
console/
collector.py ClusterCollector — aggregates state from all nodes via Redis + HTTP
scheduler.py TaskScheduler — background cron/at scheduler, dispatches via MQ
server.py Cluster dashboard HTTP server + SSE + CLI entry point
static/ Cluster dashboard web UI (page-specific HTML, CSS, JS)
channels/
@@ -1287,3 +1288,17 @@ to confirm success.
Discord ships as the first adapter. See [channels.md](channels.md) for
setup instructions, configuration reference, and the adapter development
guide.
### Notification Subsystem
The `notify` tool enables the LLM to send notifications to users or
channels without going through MQ. The server calls the channel gateway
directly over HTTP for lower latency: `_exec_notify()` queries the
`services` database table for healthy channel gateways (heartbeat within
120 seconds), authenticates with a service JWT (`aud: turnstone-channel`),
and POSTs to `POST /v1/api/notify` on the first healthy gateway. The
gateway validates the JWT, resolves the target (username lookup via
`channel_users` or direct `channel_type`+`channel_id`), and delegates to
the appropriate `ChannelAdapter.send()`. Delivery retries up to 3 times
with backoff, re-querying the service registry on each attempt. See
[Notification Flow diagram](diagrams/png/17-notify-flow.png).
+76
View File
@@ -198,6 +198,9 @@ Plan review requests are displayed as a blue embed with:
| `--redis-db` | — | `0` | Redis DB number |
| `--model` | — | server default | Default model for new workstreams |
| `--auto-approve` | — | `false` | Auto-approve ALL tool calls (skips approval buttons entirely) |
| `--http-host` | — | `127.0.0.1` | HTTP server bind address for notify endpoint |
| `--http-port` | `TURNSTONE_CHANNEL_PORT` | `8091` | HTTP server port |
| `--auth-token` | `TURNSTONE_CHANNEL_AUTH_TOKEN` | — | Static auth token for `/v1/api/notify` (alternative to JWT) |
| `--log-level` | `TURNSTONE_LOG_LEVEL` | `INFO` | Log level |
| `--log-format` | `TURNSTONE_LOG_FORMAT` | `auto` | Log format (`auto`/`json`/`text`) |
@@ -242,6 +245,79 @@ See [Security: Database Schema](security.md#database-schema) for the
---
## Notifications
> See also: [Notification Flow diagram](diagrams/png/17-notify-flow.png)
The `notify` tool allows the LLM to proactively send notifications to
users or channels on external platforms. This is useful for alerting
people about task completion, errors, or important updates without
waiting for them to check in.
### Targeting
Two modes:
- **Username** — provide a turnstone `username`. The gateway resolves
it via the `channel_users` table and sends to all linked channels
(e.g. Discord + future Slack).
- **Direct** — provide `channel_type` + `channel_id` to target a
specific platform channel or user DM.
### Delivery Flow
Notifications bypass MQ for lower latency. The server calls the channel
gateway directly over HTTP:
1. The LLM calls the `notify` tool with a message and target
2. `_exec_notify()` queries the `services` table for healthy channel
gateways (heartbeat within the last 120 seconds)
3. The server mints a service JWT (`aud: turnstone-channel`) via
`ServiceTokenManager` and POSTs to the first healthy gateway
4. The gateway validates the JWT, resolves the target, and calls
`adapter.send()` on the appropriate platform adapter
5. On failure, the server tries the next gateway. If all fail, it
retries up to 2 more times (delays: 1s, 3s), re-querying the
service registry on each attempt
### Service Registry
The channel gateway registers itself in the `services` database table
on startup and sends a heartbeat every 30 seconds. On shutdown it
deregisters. Services are considered stale after 120 seconds (4 missed
heartbeats) and are excluded from `list_services()` queries.
The `services` table schema:
| Column | Description |
|--------|-------------|
| `service_type` | Service category (e.g. `"channel"`) |
| `service_id` | Unique instance ID (`channel-<hostname>-<random>`) |
| `url` | HTTP base URL for the service |
| `last_heartbeat` | ISO 8601 timestamp of last heartbeat |
| `created` | ISO 8601 timestamp of initial registration |
### Security
- **Authentication** — the gateway's `POST /v1/api/notify` endpoint
requires authentication. Configure either `TURNSTONE_JWT_SECRET`
(the server mints JWTs with `aud: turnstone-channel` automatically)
or a static token via `--auth-token`. If neither is set, the
gateway fails closed and rejects all requests with 401. Server JWTs
(`aud: turnstone-server`) are rejected.
- **Rate limit** — maximum 5 notifications per turn. The counter only
increments on successful delivery, so failures don't consume the
budget.
- **SSRF protection** — only `http://` and `https://` service URLs
are allowed. Other schemes are silently skipped.
- **Mention sanitization** — `discord.utils.escape_mentions()` is
applied before sending, preventing `@everyone` / `@here` abuse.
- **Error redaction** — generic error messages are returned to the
LLM. Internal details (service IDs, URLs, exception messages) are
logged server-side only.
---
## Adding New Adapters
The `ChannelAdapter` protocol defines the interface any platform adapter
+144
View File
@@ -423,6 +423,150 @@ to create the initial admin user and receive a JWT in one step. See
---
## Scheduled Tasks
The console includes a background **TaskScheduler** daemon that creates workstreams on a timed basis via the MQ broker. It supports cron-based recurring schedules and one-shot `at` schedules.
### Architecture
The scheduler runs as a daemon thread inside the console process. Every `check_interval` seconds (default 15) it:
1. Acquires a distributed lock via Redis `SET NX EX` (prevents duplicate dispatch in multi-console deployments)
2. Queries the storage backend for tasks whose `next_run <= now` and `enabled = true`
3. Dispatches each due task as one or more `CreateWorkstreamMessage` via MQ
4. Updates `last_run` and computes the next `next_run` (or disables one-shot `at` tasks)
5. Releases the lock via Lua script (safe conditional delete)
Run history is automatically pruned (runs older than 90 days) approximately once per hour.
### Schedule Types
| Type | Field | Behavior |
|------|-------|----------|
| `cron` | `cron_expr` | Recurring schedule using standard 5-field cron syntax. Requires `croniter`. |
| `at` | `at_time` | One-shot: fires once at the given ISO 8601 timestamp (must include timezone), then auto-disables. |
### Target Modes
| Mode | Behavior |
|------|----------|
| `auto` | Picks the reachable node with the most available capacity |
| `pool` | Pushes to the shared inbound queue (any bridge picks it up) |
| `all` | Fan-out to all reachable nodes (capped at `max_fan_out`, default 20) |
| `<node_id>` | Targets a specific node by ID |
### Configuration
| Parameter | Default | Description |
|-----------|---------|-------------|
| `check_interval` | `15.0` | Seconds between scheduler ticks |
| `lock_ttl` | `60` | Distributed lock TTL in seconds |
| `max_fan_out` | `20` | Maximum nodes for `all` target mode |
Dependency: `croniter` (installed with turnstone).
### Schedule API
All schedule endpoints require `approve` scope. Maximum 200 schedules.
#### `GET /v1/api/admin/schedules`
List all scheduled tasks.
```json
{
"schedules": [
{
"task_id": "a1b2c3d4",
"name": "nightly-checks",
"description": "Run nightly health checks",
"schedule_type": "cron",
"cron_expr": "0 2 * * *",
"at_time": "",
"target_mode": "auto",
"model": "",
"initial_message": "Run the nightly health check suite.",
"auto_approve": false,
"auto_approve_tools": [],
"enabled": true,
"created_by": "u_admin",
"last_run": "2026-03-05T02:00:00Z",
"next_run": "2026-03-06T02:00:00Z",
"created": "2026-03-01T12:00:00Z",
"updated": "2026-03-05T02:00:01Z"
}
]
}
```
#### `POST /v1/api/admin/schedules`
Create a scheduled task.
Request:
```json
{
"name": "nightly-checks",
"description": "Run nightly health checks",
"schedule_type": "cron",
"cron_expr": "0 2 * * *",
"target_mode": "auto",
"initial_message": "Run the nightly health check suite.",
"auto_approve": false,
"enabled": true
}
```
Required fields: `name`, `schedule_type`, `initial_message`. For `cron` schedules provide `cron_expr`; for `at` schedules provide `at_time` (ISO 8601 with timezone, must be in the future).
Response: `ScheduleInfo` (same shape as list items above). Returns `400` for invalid cron syntax, naive timestamps, or past `at_time`. Returns `409` if the 200-schedule cap is reached.
#### `GET /v1/api/admin/schedules/{task_id}`
Get a single scheduled task. Returns `ScheduleInfo` or `404`.
#### `PUT /v1/api/admin/schedules/{task_id}`
Partial update — only include fields to change. If `schedule_type`, `cron_expr`, or `at_time` change, `next_run` is recomputed automatically.
```json
{
"enabled": false
}
```
Response: updated `ScheduleInfo`. Returns `400` for validation errors, `404` if not found.
#### `DELETE /v1/api/admin/schedules/{task_id}`
Delete a scheduled task and all its run history. Returns `{"status": "ok"}` or `404`.
#### `GET /v1/api/admin/schedules/{task_id}/runs?limit=50`
List execution history for a task (most recent first). `limit` defaults to 50, max 200.
```json
{
"runs": [
{
"run_id": "r_abc123",
"task_id": "a1b2c3d4",
"node_id": "db-west-04",
"ws_id": "ws_xyz",
"correlation_id": "corr_789",
"started": "2026-03-05T02:00:00Z",
"status": "dispatched",
"error": ""
}
]
}
```
Status is `dispatched` on success or `failed` with an `error` message (e.g. no reachable nodes). Failed runs do not advance `next_run`.
---
## CLI Commands
The `/cluster` command in the turnstone CLI queries the console's HTTP API. Requires `--console-url` or `[console] url` in config.toml.
@@ -44,11 +44,15 @@ class "turnstone-channel" as ChannelService <<service>> {
asyncio event loop
Structured logging (structlog)
--log-level, --log-format
--
POST /v1/api/notify (HTTP)
GET /health
}
class "DiscordBot" as Bot <<service>> {
+on_message(msg)
+on_interaction(interaction)
+send(channel_id, content)
+run(token)
--
discord.py Client
@@ -56,6 +60,7 @@ class "DiscordBot" as Bot <<service>> {
Sends replies + embeds
Creates threads for workstreams
Renders approval buttons
escape_mentions() on send
}
class "ChannelRouter" as Router <<service>> {
@@ -109,6 +114,9 @@ class "turnstone-server" as Server <<server>> {
--
LLM execution + tool use
SSE event stream
--
notify tool: _exec_notify()
ServiceTokenManager (JWT)
}
' -- Storage --
@@ -134,6 +142,18 @@ class "channel_routes" as CR <<storage>> {
to turnstone workstreams
}
class "services" as SVC <<storage>> {
service_type (PK)
service_id (PK)
url
last_heartbeat
created
--
Heartbeat every 30s
Stale after 120s
ON CONFLICT DO UPDATE
}
' -- Relationships --
Discord --> Bot : gateway\nevents
Bot --> Router : on_message\non_interaction
@@ -157,6 +177,11 @@ Teams .[hidden]. Slack
ChannelService --> Bot : creates + runs
ChannelService --> Router : creates
ChannelService --> Broker : creates
ChannelService --> SVC : register / heartbeat /\nderegister
' -- Notification path (direct HTTP, bypasses MQ) --
Server --> ChannelService : POST /v1/api/notify\n(JWT: aud=turnstone-channel)
Server --> SVC : list_services("channel",\nmax_age_seconds=120)
' -- Notes --
note right of Bot
@@ -208,4 +233,18 @@ note bottom of CU
7. AuthResult scopes applied by server
end note
note bottom of SVC
**Notification Flow** (direct HTTP, bypasses MQ)
1. LLM calls notify tool → _prepare_notify()
2. _exec_notify() checks rate limit (5/turn)
3. Queries services table for healthy gateways
4. Mints JWT (aud: turnstone-channel) via
ServiceTokenManager
5. POSTs to first healthy gateway
6. Gateway validates JWT, resolves target
7. adapter.send() → Discord API
8. On failure: retry up to 3× (1s, 3s backoff)
9. SSRF: only http(s) URLs allowed
end note
@enduml
+116
View File
@@ -0,0 +1,116 @@
@startuml
!theme plain
title Turnstone — Notification Delivery Flow
skinparam participant {
BackgroundColor<<server>> #FFE0B2
BackgroundColor<<storage>> #B3E5FC
BackgroundColor<<service>> #E8EAF6
BackgroundColor<<platform>> #E1BEE7
}
participant "ChatSession\n(turnstone-server)" as Session <<server>>
participant "StorageBackend" as Storage <<storage>>
participant "ServiceTokenManager" as STM <<server>>
participant "Channel Gateway\n(_http.py)" as Gateway <<service>>
participant "ChannelAdapter\n(Discord bot)" as Adapter <<service>>
participant "Discord API" as Discord <<platform>>
== Prepare Phase ==
Session -> Session : _prepare_notify(call_id, args)
note right
Validates:
- message (required, ≤2000 chars)
- target: username OR channel_type+channel_id
- no ambiguous targeting (both set)
- partial targeting errors
end note
== Execute Phase ==
Session -> Session : _exec_notify(item)
Session -> Session : check rate limit\n(≥5 per turn?)
alt rate limit exceeded
Session --> Session : "Error: rate limit exceeded"
end
loop up to 3 attempts (retry delays: 1s, 3s)
Session -> Storage : list_services("channel",\nmax_age_seconds=120)
Storage --> Session : services[] (sorted by\nlast_heartbeat DESC)
alt no healthy services
Session -> Session : log.warning("notify.no_services")
Session -> Session : sleep(delay)
else services available
Session -> STM : bearer_header
note right
Lazy-init ServiceTokenManager
aud: turnstone-channel
scope: write
Auto-rotates 1h JWTs
end note
STM --> Session : Authorization: Bearer <jwt>
loop for each gateway (first-healthy)
Session -> Session : SSRF check:\nurl.startswith("http://"|"https://")
Session -> Gateway : POST /v1/api/notify\n+ Authorization header
Gateway -> Gateway : _check_auth()\nvalidate JWT (aud=turnstone-channel)\nor static token
alt auth failed
Gateway --> Session : 401 Unauthorized
else auth ok
alt username target
Gateway -> Storage : get_user_by_username()
Storage --> Gateway : user
Gateway -> Storage : list_channel_users_by_user()
Storage --> Gateway : linked channels
else direct target
Gateway -> Gateway : use channel_type + channel_id
end
Gateway -> Adapter : send(channel_id, content)
note right
escape_mentions() applied
Chunked for 2000-char limit
end note
Adapter -> Discord : POST message
Discord --> Adapter : message_id
Adapter --> Gateway : message_id
Gateway --> Session : 200 {results: [{status: "sent"}]}
Session -> Session : _notify_count += 1
Session --> Session : "Notification sent successfully"
note right : Return — no further\ngateways tried
end
end
alt all gateways failed
Session -> Session : log.warning(\n"notify.all_gateways_failed")
Session -> Session : sleep(delay)
end
end
end
alt all retries exhausted
Session -> Session : log.warning("notify.delivery_failed")
Session --> Session : "Error: notification delivery failed"
end
== Service Registry (Background) ==
note over Gateway, Storage
**Heartbeat Lifecycle**
1. Gateway startup: register_service("channel", id, url)
2. Every 30s: heartbeat_service("channel", id)
3. Shutdown: deregister_service("channel", id)
4. Stale after 120s (4 missed heartbeats)
end note
@enduml
@@ -1,3 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:47b106bbcb1041fe4122007065c6cc85605348b42fc7140287194cf25e42e095
size 318036
oid sha256:6049cc0b07480df88d0d93aa977a1e97f64b41588325ff41d98be0e39431fc5c
size 431712
+3
View File
@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:f55e177e0838a16d9bc4f07b162b4b6a966cc596c9d0a022d35a3c84f23e7b02
size 221452
+6
View File
@@ -96,6 +96,12 @@ Both `TurnstoneConsole` (sync) and `AsyncTurnstoneConsole` (async) expose:
| | `workstreams(*, state, node, search, sort, page, per_page)` | `ClusterWorkstreamsResponse` |
| | `node_detail(node_id)` | `NodeDetailResponse` |
| | `create_workstream(*, node_id, name, model, initial_message)` | `ConsoleCreateWsResponse` |
| **Schedules** | `list_schedules()` | `ListSchedulesResponse` |
| | `create_schedule(*, name, schedule_type, initial_message, ...)` | `ScheduleInfo` |
| | `get_schedule(task_id)` | `ScheduleInfo` |
| | `update_schedule(task_id, *, name=..., enabled=..., ...)` | `ScheduleInfo` |
| | `delete_schedule(task_id)` | `StatusResponse` |
| | `list_schedule_runs(task_id, *, limit=50)` | `ListScheduleRunsResponse` |
| **Streaming** | `stream_cluster_events()` | `Iterator[ClusterEvent]` |
| **Auth** | `login(username=..., password=...)` / `login(token="ts_xxx")` | `AuthLoginResponse` |
| | `logout()` | `StatusResponse` |
+15 -8
View File
@@ -354,15 +354,22 @@ provided, that static token is used instead.
The bridge and console collector use `ServiceTokenManager` for
auto-rotating JWTs when communicating with server nodes:
| Service | Identity | Scope | Purpose |
|---------|----------|-------|---------|
| Bridge | `bridge` | `approve` | Tool approval proxy, message relay |
| Console collector | `console-collector` | `read` | Node health polling |
| Console proxy | `console-proxy` | `write` | Proxied API calls |
| Service | Identity | Scope | Audience | Purpose |
|---------|----------|-------|----------|---------|
| Bridge | `bridge` | `approve` | `turnstone-server` | Tool approval proxy, message relay |
| Console collector | `console-collector` | `read` | `turnstone-server` | Node health polling |
| Console proxy | `console-proxy` | `write` | `turnstone-server` | Proxied API calls |
| Channel notify | `system` | `write` | `turnstone-channel` | Notification delivery to channel gateway |
All service tokens use `aud: turnstone-server` and 1-hour expiry with
automatic refresh. The bridge injects auth headers per-request via httpx
event hooks to ensure rotated tokens are picked up on SSE reconnects.
Service tokens use 1-hour expiry with automatic refresh via
`ServiceTokenManager`. The bridge injects auth headers per-request via
httpx event hooks to ensure rotated tokens are picked up on SSE
reconnects.
Note that the channel gateway uses a distinct JWT audience
(`turnstone-channel`) from the server (`turnstone-server`) and console
(`turnstone-console`). A server-scoped JWT cannot authenticate to the
channel gateway endpoint, and vice versa.
---
+32 -2
View File
@@ -1,6 +1,6 @@
# Tools Reference
turnstone exposes 14 built-in tools plus any number of external MCP tools to the
turnstone exposes 15 built-in tools plus any number of external MCP tools to the
LLM via the OpenAI function-calling interface. Built-in tools are defined as JSON
files under `turnstone/tools/` and loaded at startup by `turnstone/core/tools.py`.
MCP tools are discovered from configured MCP servers at startup by
@@ -46,7 +46,7 @@ schema plus turnstone-specific metadata keys:
| Name | Description |
|---------------------|-------------|
| `TOOLS` | All 14 tool definitions (sent to the model). |
| `TOOLS` | All 15 tool definitions (sent to the model). |
| `AGENT_TOOLS` | Tools with `agent: true` -- available to plan sub-agents. Read-only tools. |
| `TASK_AGENT_TOOLS` | Tools with `task_agent: true` -- available to task sub-agents. Includes write operations. |
| `AGENT_AUTO_TOOLS` | Set of tool names with `auto_approve: true` -- no user confirmation needed. |
@@ -113,6 +113,7 @@ Each item's `execute` callable is invoked:
- `remember` -- writes to persistent memory database (lightweight, always auto-approved)
- `recall` -- reads from persistent memory database
- `forget` -- deletes from persistent memory database (lightweight, always auto-approved)
- `notify` -- sends notifications to linked channels (time-sensitive, auto-approved for urgency)
**Requires user confirmation** (write operations, network access, side effects):
- `bash` -- arbitrary command execution
@@ -162,6 +163,7 @@ Every tool defines a `primary_key`. The mapping is:
| `remember` | `key` |
| `recall` | `query` |
| `forget` | `key` |
| `notify` | `message` |
---
@@ -387,6 +389,33 @@ Remove a persistent memory by key.
---
## Notifications
### notify
Send a notification to a user or channel on an external platform.
| Parameter | Type | Required | Description |
|----------------|--------|----------|-------------|
| `message` | string | yes | Notification content (plain text, max 2000 chars). |
| `username` | string | no | Turnstone username — sends to all linked channels. |
| `channel_type` | string | no | Platform for direct targeting (`discord`). |
| `channel_id` | string | no | Platform-specific channel or user ID for direct targeting. |
| `title` | string | no | Optional short title (rendered as bold prefix). |
Provide either `username` for user-based targeting or `channel_type` +
`channel_id` for direct targeting. Do not combine both.
- **What it does**: Sends a notification via the channel gateway's HTTP endpoint (`POST /v1/api/notify`). The server queries the `services` table for healthy channel gateways, authenticates with a service JWT (`aud: turnstone-channel`), and delivers to the first healthy gateway. On failure, retries up to 2 additional times with backoff (1s, 3s). Rate-limited to 5 notifications per turn (counter only increments on success).
- **Auto-approve**: Yes — notifications are time-sensitive and auto-approved so the model can alert users urgently.
- **Agent availability**: `agent` and `task_agent`.
> See [Channel Integrations: Notifications](channels.md#notifications)
> for the full delivery flow, service registry details, and security
> measures.
---
## Summary Table
| Tool | Category | Auto-approve | agent | task_agent | primary_key |
@@ -405,6 +434,7 @@ Remove a persistent memory by key.
| `remember` | Memory | Yes | No | No | `key` |
| `recall` | Memory | Yes | No | No | `query` |
| `forget` | Memory | Yes | No | No | `key` |
| `notify` | Notify | Yes | Yes | Yes | `message` |
---
+7 -3
View File
@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
[project]
name = "turnstone"
version = "0.4.0"
version = "0.4.2"
description = "Multi-node AI orchestration platform with tool use, agent routing, and cluster simulation."
readme = "README.md"
license = "BUSL-1.1"
@@ -43,10 +43,10 @@ Repository = "https://github.com/turnstonelabs/turnstone"
Issues = "https://github.com/turnstonelabs/turnstone/issues"
[project.optional-dependencies]
test = ["pytest>=9.0", "pytest-cov>=6.0"]
test = ["pytest>=9.0", "pytest-cov>=6.0", "croniter>=3.0"]
dev = ["ruff>=0.9", "mypy>=1.14", "types-redis>=4.6"]
mq = ["redis>=7.2"]
console = ["redis>=7.2"]
console = ["redis>=7.2", "croniter>=3.0"]
sim = ["redis>=7.2"]
anthropic = ["anthropic>=0.39"]
postgres = ["psycopg[binary]>=3.2"]
@@ -150,6 +150,10 @@ ignore_missing_imports = true
module = ["discord", "discord.*"]
ignore_missing_imports = true
[[tool.mypy.overrides]]
module = ["croniter", "croniter.*"]
ignore_missing_imports = true
[[tool.mypy.overrides]]
module = ["turnstone.channels.discord.*"]
disallow_subclassing_any = false
+41
View File
@@ -10,9 +10,14 @@ import type {
ConsoleCreateWsRequest,
ConsoleCreateWsResponse,
ConsoleHealthResponse,
CreateScheduleRequest,
ListScheduleRunsResponse,
ListSchedulesResponse,
NodeDetailResponse,
NodesOptions,
ScheduleInfo,
StatusResponse,
UpdateScheduleRequest,
WorkstreamsOptions,
} from "./types.js";
@@ -111,4 +116,40 @@ export class TurnstoneConsole extends BaseClient {
async health(): Promise<ConsoleHealthResponse> {
return this.request("GET", "/health");
}
// -- Schedules ------------------------------------------------------------
async listSchedules(): Promise<ListSchedulesResponse> {
return this.request("GET", "/v1/api/admin/schedules");
}
async createSchedule(opts: CreateScheduleRequest): Promise<ScheduleInfo> {
return this.request("POST", "/v1/api/admin/schedules", { json: opts });
}
async getSchedule(taskId: string): Promise<ScheduleInfo> {
return this.request("GET", `/v1/api/admin/schedules/${taskId}`);
}
async updateSchedule(
taskId: string,
opts: UpdateScheduleRequest,
): Promise<ScheduleInfo> {
return this.request("PUT", `/v1/api/admin/schedules/${taskId}`, {
json: opts,
});
}
async deleteSchedule(taskId: string): Promise<StatusResponse> {
return this.request("DELETE", `/v1/api/admin/schedules/${taskId}`);
}
async listScheduleRuns(
taskId: string,
opts?: { limit?: number },
): Promise<ListScheduleRunsResponse> {
return this.request("GET", `/v1/api/admin/schedules/${taskId}/runs`, {
params: { limit: opts?.limit ?? 50 },
});
}
}
+6
View File
@@ -103,6 +103,12 @@ export type {
ConsoleCreateWsRequest,
ConsoleCreateWsResponse,
ConsoleHealthResponse,
CreateScheduleRequest,
UpdateScheduleRequest,
ScheduleInfo,
ScheduleRunInfo,
ListSchedulesResponse,
ListScheduleRunsResponse,
TurnResult,
SendAndWaitOptions,
NodesOptions,
+71
View File
@@ -269,6 +269,77 @@ export interface ConsoleHealthResponse {
versions: string[];
}
// ---------------------------------------------------------------------------
// Console API — Schedules
// ---------------------------------------------------------------------------
export interface CreateScheduleRequest {
name: string;
schedule_type: string;
initial_message: string;
description?: string;
cron_expr?: string;
at_time?: string;
target_mode?: string;
model?: string;
auto_approve?: boolean;
auto_approve_tools?: string[];
enabled?: boolean;
}
export interface UpdateScheduleRequest {
name?: string;
description?: string;
schedule_type?: string;
cron_expr?: string;
at_time?: string;
target_mode?: string;
model?: string;
initial_message?: string;
auto_approve?: boolean;
auto_approve_tools?: string[];
enabled?: boolean;
}
export interface ScheduleInfo {
task_id: string;
name: string;
description: string;
schedule_type: string;
cron_expr: string;
at_time: string;
target_mode: string;
model: string;
initial_message: string;
auto_approve: boolean;
auto_approve_tools: string[];
enabled: boolean;
created_by: string;
last_run: string | null;
next_run: string | null;
created: string;
updated: string;
}
export interface ListSchedulesResponse {
schedules: ScheduleInfo[];
}
export interface ScheduleRunInfo {
run_id: string;
task_id: string;
node_id: string;
ws_id: string;
correlation_id: string;
started: string;
status: string;
error: string;
}
export interface ListScheduleRunsResponse {
runs: ScheduleRunInfo[];
}
// ---------------------------------------------------------------------------
// SDK-specific types
// ---------------------------------------------------------------------------
+338
View File
@@ -0,0 +1,338 @@
"""Tests for the channel gateway HTTP notify endpoint."""
from __future__ import annotations
from unittest.mock import AsyncMock
import pytest
from starlette.testclient import TestClient
from turnstone.channels._http import create_channel_app
from turnstone.core.storage._sqlite import SQLiteBackend
@pytest.fixture
def storage(tmp_path):
return SQLiteBackend(str(tmp_path / "test.db"))
@pytest.fixture
def mock_adapter():
adapter = AsyncMock()
adapter.channel_type = "discord"
adapter.send = AsyncMock(return_value="msg_001")
return adapter
@pytest.fixture
def no_auth_client(storage, mock_adapter):
"""Client with no auth configured (for fail-closed tests)."""
app = create_channel_app({"discord": mock_adapter}, storage)
return TestClient(app)
@pytest.fixture
def client(storage, mock_adapter):
"""Default client with static auth token configured."""
app = create_channel_app({"discord": mock_adapter}, storage, auth_token="test-secret-token")
return TestClient(app)
@pytest.fixture
def authed_client(storage, mock_adapter):
"""Alias — same as client, for auth-specific test clarity."""
app = create_channel_app({"discord": mock_adapter}, storage, auth_token="test-secret-token")
return TestClient(app)
@pytest.fixture
def jwt_client(storage, mock_adapter):
"""Client with JWT auth configured."""
app = create_channel_app({"discord": mock_adapter}, storage, jwt_secret="a" * 32)
return TestClient(app)
class TestNotifyEndpoint:
def test_health(self, client):
resp = client.get("/health")
assert resp.status_code == 200
assert resp.json()["status"] == "ok"
def _headers(self) -> dict[str, str]:
return {"Authorization": "Bearer test-secret-token"}
def test_direct_discord_target(self, client, mock_adapter):
resp = client.post(
"/v1/api/notify",
json={
"target": {"channel_type": "discord", "channel_id": "123456"},
"message": "Hello!",
},
headers=self._headers(),
)
assert resp.status_code == 200
results = resp.json()["results"]
assert len(results) == 1
assert results[0]["status"] == "sent"
assert results[0]["message_id"] == "msg_001"
mock_adapter.send.assert_called_once_with("123456", "Hello!")
def test_with_title(self, client, mock_adapter):
resp = client.post(
"/v1/api/notify",
json={
"target": {"channel_type": "discord", "channel_id": "123456"},
"message": "Hello!",
"title": "Alert",
},
headers=self._headers(),
)
assert resp.status_code == 200
mock_adapter.send.assert_called_once_with("123456", "**Alert**\nHello!")
def test_username_resolution(self, client, storage, mock_adapter):
# Create a user and link a channel
storage.create_user("u1", "testuser", "Test User", "hash")
storage.create_channel_user("discord", "disc_123", "u1")
resp = client.post(
"/v1/api/notify",
json={
"target": {"username": "testuser"},
"message": "Hello!",
},
headers=self._headers(),
)
assert resp.status_code == 200
results = resp.json()["results"]
assert len(results) == 1
assert results[0]["status"] == "sent"
mock_adapter.send.assert_called_once_with("disc_123", "Hello!")
def test_unknown_username(self, client):
resp = client.post(
"/v1/api/notify",
json={
"target": {"username": "nobody"},
"message": "Hello!",
},
headers=self._headers(),
)
assert resp.status_code == 404
error = resp.json()["error"]
assert "nobody" not in error
assert "not found or has no linked channels" in error
def test_user_no_channels(self, authed_client, storage):
storage.create_user("u1", "testuser", "Test User", "hash")
resp = authed_client.post(
"/v1/api/notify",
json={
"target": {"username": "testuser"},
"message": "Hello!",
},
headers={"Authorization": "Bearer test-secret-token"},
)
assert resp.status_code == 404
# Generic message — must not differentiate "not found" vs "no channels"
error = resp.json()["error"]
assert "testuser" not in error
assert "not found or has no linked channels" in error
def test_missing_fields(self, client):
resp = client.post(
"/v1/api/notify",
json={"target": {"username": "x"}},
headers=self._headers(),
)
assert resp.status_code == 400
def test_missing_target(self, client):
resp = client.post(
"/v1/api/notify",
json={"message": "Hello!"},
headers=self._headers(),
)
assert resp.status_code == 400
def test_invalid_target(self, client):
resp = client.post(
"/v1/api/notify",
json={
"target": {"invalid": "field"},
"message": "Hello!",
},
headers=self._headers(),
)
assert resp.status_code == 400
def test_no_adapter(self, client, storage):
# App has discord adapter, try email target
resp = client.post(
"/v1/api/notify",
json={
"target": {"channel_type": "email", "channel_id": "test@example.com"},
"message": "Hello!",
},
headers=self._headers(),
)
assert resp.status_code == 200
results = resp.json()["results"]
assert results[0]["status"] == "no_adapter"
def test_adapter_failure(self, client, mock_adapter):
mock_adapter.send.side_effect = RuntimeError("Discord API error")
resp = client.post(
"/v1/api/notify",
json={
"target": {"channel_type": "discord", "channel_id": "123456"},
"message": "Hello!",
},
headers=self._headers(),
)
assert resp.status_code == 200
results = resp.json()["results"]
assert results[0]["status"] == "failed"
def test_invalid_json(self, client):
resp = client.post(
"/v1/api/notify",
content=b"not json",
headers={
"content-type": "application/json",
"Authorization": "Bearer test-secret-token",
},
)
assert resp.status_code == 400
def test_whitespace_only_message(self, client):
"""Whitespace-only messages should be rejected."""
resp = client.post(
"/v1/api/notify",
json={
"target": {"channel_type": "discord", "channel_id": "123"},
"message": " ",
},
headers=self._headers(),
)
assert resp.status_code == 400
class TestNotifyAuth:
"""Tests for authentication on the /v1/api/notify endpoint."""
def test_reject_when_unconfigured(self, no_auth_client):
"""Requests are rejected (fail closed) when no auth is configured."""
resp = no_auth_client.post(
"/v1/api/notify",
json={
"target": {"channel_type": "discord", "channel_id": "123"},
"message": "Hello!",
},
)
assert resp.status_code == 401
def test_reject_without_token(self, authed_client):
"""Requests without Authorization header are rejected when auth is configured."""
resp = authed_client.post(
"/v1/api/notify",
json={
"target": {"channel_type": "discord", "channel_id": "123"},
"message": "Hello!",
},
)
assert resp.status_code == 401
def test_reject_wrong_token(self, authed_client):
"""Requests with wrong token are rejected."""
resp = authed_client.post(
"/v1/api/notify",
json={
"target": {"channel_type": "discord", "channel_id": "123"},
"message": "Hello!",
},
headers={"Authorization": "Bearer wrong-token"},
)
assert resp.status_code == 401
def test_accept_valid_static_token(self, authed_client, mock_adapter):
"""Requests with correct static token are accepted."""
resp = authed_client.post(
"/v1/api/notify",
json={
"target": {"channel_type": "discord", "channel_id": "123"},
"message": "Hello!",
},
headers={"Authorization": "Bearer test-secret-token"},
)
assert resp.status_code == 200
assert resp.json()["results"][0]["status"] == "sent"
def test_accept_valid_jwt(self, jwt_client, mock_adapter):
"""Requests with a valid JWT for the channel audience are accepted."""
from turnstone.core.auth import JWT_AUD_CHANNEL, create_jwt
token = create_jwt(
user_id="system",
scopes=frozenset({"write"}),
source="service",
secret="a" * 32,
audience=JWT_AUD_CHANNEL,
)
resp = jwt_client.post(
"/v1/api/notify",
json={
"target": {"channel_type": "discord", "channel_id": "123"},
"message": "Hello!",
},
headers={"Authorization": f"Bearer {token}"},
)
assert resp.status_code == 200
def test_reject_jwt_wrong_audience(self, jwt_client):
"""JWTs with wrong audience are rejected."""
from turnstone.core.auth import create_jwt
token = create_jwt(
user_id="system",
scopes=frozenset({"write"}),
source="service",
secret="a" * 32,
audience="turnstone-server", # wrong audience
)
resp = jwt_client.post(
"/v1/api/notify",
json={
"target": {"channel_type": "discord", "channel_id": "123"},
"message": "Hello!",
},
headers={"Authorization": f"Bearer {token}"},
)
assert resp.status_code == 401
def test_reject_jwt_wrong_secret(self, jwt_client):
"""JWTs signed with wrong secret are rejected."""
from turnstone.core.auth import JWT_AUD_CHANNEL, create_jwt
token = create_jwt(
user_id="system",
scopes=frozenset({"write"}),
source="service",
secret="b" * 32, # wrong secret
audience=JWT_AUD_CHANNEL,
)
resp = jwt_client.post(
"/v1/api/notify",
json={
"target": {"channel_type": "discord", "channel_id": "123"},
"message": "Hello!",
},
headers={"Authorization": f"Bearer {token}"},
)
assert resp.status_code == 401
def test_health_bypasses_auth(self, authed_client):
"""Health endpoint is always accessible regardless of auth config."""
resp = authed_client.get("/health")
assert resp.status_code == 200
+618
View File
@@ -0,0 +1,618 @@
"""Tests for the notify tool (prepare + execute) in ChatSession."""
from __future__ import annotations
from typing import TYPE_CHECKING
from unittest.mock import MagicMock
if TYPE_CHECKING:
from turnstone.core.session import ChatSession
def _make_session() -> ChatSession:
"""Create a minimal ChatSession with mocked dependencies."""
from unittest.mock import patch
with (
patch("turnstone.core.session.register_session"),
patch("turnstone.core.session.save_message"),
):
from turnstone.core.session import ChatSession
ui = MagicMock()
session = ChatSession(
client=MagicMock(),
model="test-model",
ui=ui,
instructions=None,
temperature=0.7,
max_tokens=1000,
tool_timeout=30,
)
return session
class TestPrepareNotify:
def test_valid_username_target(self):
session = _make_session()
result = session._prepare_notify(
"call_1",
{
"message": "Hello!",
"username": "admin",
},
)
assert "execute" in result
assert result["func_name"] == "notify"
assert result["needs_approval"] is False
assert "@admin" in result["header"]
assert result["username"] == "admin"
assert result["message"] == "Hello!"
def test_valid_direct_target(self):
session = _make_session()
result = session._prepare_notify(
"call_1",
{
"message": "Hello!",
"channel_type": "discord",
"channel_id": "123456",
},
)
assert "execute" in result
assert result["channel_type"] == "discord"
assert result["channel_id"] == "123456"
assert "discord:123456" in result["header"]
def test_missing_message(self):
session = _make_session()
result = session._prepare_notify("call_1", {"username": "admin"})
assert "error" in result
assert "message" in result["error"].lower()
def test_empty_message(self):
session = _make_session()
result = session._prepare_notify(
"call_1",
{
"message": "",
"username": "admin",
},
)
assert "error" in result
def test_message_too_long(self):
session = _make_session()
result = session._prepare_notify(
"call_1",
{
"message": "x" * 2001,
"username": "admin",
},
)
assert "error" in result
assert "2000" in result["error"]
def test_both_username_and_direct(self):
session = _make_session()
result = session._prepare_notify(
"call_1",
{
"message": "Hello!",
"username": "admin",
"channel_type": "discord",
"channel_id": "123",
},
)
assert "error" in result
assert "both" in result["error"].lower() or "ambiguous" in result["error"].lower()
def test_no_target(self):
session = _make_session()
result = session._prepare_notify("call_1", {"message": "Hello!"})
assert "error" in result
def test_channel_type_without_id(self):
session = _make_session()
result = session._prepare_notify(
"call_1",
{
"message": "Hello!",
"channel_type": "discord",
},
)
assert "error" in result
assert "channel_id" in result["error"]
def test_channel_id_without_type(self):
session = _make_session()
result = session._prepare_notify(
"call_1",
{
"message": "Hello!",
"channel_id": "123456",
},
)
assert "error" in result
assert "channel_type" in result["error"]
def test_preview_truncated(self):
session = _make_session()
result = session._prepare_notify(
"call_1",
{
"message": "a" * 200,
"username": "admin",
},
)
assert result["preview"].endswith("...")
assert len(result["preview"]) <= 123 # 120 chars + "..."
def test_title_passed_through(self):
session = _make_session()
result = session._prepare_notify(
"call_1",
{
"message": "Hello!",
"username": "admin",
"title": "Alert",
},
)
assert result["title"] == "Alert"
class TestExecNotify:
def test_sends_http_to_channel_gateway(self, tmp_path):
from turnstone.core.storage._sqlite import SQLiteBackend
storage = SQLiteBackend(str(tmp_path / "test.db"))
storage.register_service("channel", "ch-1", "http://localhost:8091")
session = _make_session()
item = {
"call_id": "call_1",
"func_name": "notify",
"message": "Hello!",
"username": "admin",
"channel_type": "",
"channel_id": "",
"title": "Alert",
}
mock_resp = MagicMock()
mock_resp.status_code = 200
mock_resp.json.return_value = {
"results": [{"channel_type": "discord", "channel_id": "123", "status": "sent"}]
}
from unittest.mock import patch
with (
patch("turnstone.core.session.get_storage", return_value=storage),
patch("turnstone.core.session.httpx.post", return_value=mock_resp) as mock_post,
patch.dict("os.environ", {}, clear=False),
):
call_id, msg = session._exec_notify(item)
assert call_id == "call_1"
assert "sent successfully" in msg.lower()
mock_post.assert_called_once()
post_kwargs = mock_post.call_args
assert post_kwargs.kwargs["json"]["target"] == {"username": "admin"}
assert post_kwargs.kwargs["json"]["message"] == "Hello!"
def test_no_services_available(self, tmp_path):
from turnstone.core.storage._sqlite import SQLiteBackend
storage = SQLiteBackend(str(tmp_path / "test.db"))
# No services registered
session = _make_session()
item = {
"call_id": "call_1",
"func_name": "notify",
"message": "Hello!",
"username": "admin",
"channel_type": "",
"channel_id": "",
"title": "",
}
from unittest.mock import patch
with (
patch("turnstone.core.session.get_storage", return_value=storage),
patch("turnstone.core.session.time.sleep"),
):
call_id, msg = session._exec_notify(item)
assert "no channel gateway" in msg.lower()
def test_rate_limit(self, tmp_path):
from turnstone.core.storage._sqlite import SQLiteBackend
storage = SQLiteBackend(str(tmp_path / "test.db"))
storage.register_service("channel", "ch-1", "http://localhost:8091")
session = _make_session()
item = {
"call_id": "call_1",
"func_name": "notify",
"message": "Hello!",
"username": "admin",
"channel_type": "",
"channel_id": "",
"title": "",
}
mock_resp = MagicMock()
mock_resp.status_code = 200
mock_resp.json.return_value = {
"results": [{"channel_type": "discord", "channel_id": "123", "status": "sent"}]
}
from unittest.mock import patch
with (
patch("turnstone.core.session.get_storage", return_value=storage),
patch("turnstone.core.session.httpx.post", return_value=mock_resp),
patch.dict("os.environ", {}, clear=False),
):
for _i in range(5):
call_id, msg = session._exec_notify(item)
assert "sent successfully" in msg.lower()
# 6th should fail
call_id, msg = session._exec_notify(item)
assert "rate limit" in msg.lower()
def test_rate_limit_not_consumed_on_failure(self, tmp_path):
"""Failed delivery should not consume rate limit slots."""
from turnstone.core.storage._sqlite import SQLiteBackend
storage = SQLiteBackend(str(tmp_path / "test.db"))
storage.register_service("channel", "ch-1", "http://localhost:8091")
session = _make_session()
item = {
"call_id": "call_1",
"func_name": "notify",
"message": "Hello!",
"username": "admin",
"channel_type": "",
"channel_id": "",
"title": "",
}
from unittest.mock import patch
with (
patch("turnstone.core.session.get_storage", return_value=storage),
patch(
"turnstone.core.session.httpx.post",
side_effect=ConnectionError("refused"),
),
patch.dict("os.environ", {}, clear=False),
patch("turnstone.core.session.time.sleep"),
):
# All fail — counter should stay at 0
for _i in range(3):
session._exec_notify(item)
assert session._notify_count == 0
def test_counter_on_init(self):
session = _make_session()
assert session._notify_count == 0
def test_http_failure_reported(self, tmp_path):
from turnstone.core.storage._sqlite import SQLiteBackend
storage = SQLiteBackend(str(tmp_path / "test.db"))
storage.register_service("channel", "ch-1", "http://localhost:8091")
session = _make_session()
item = {
"call_id": "call_1",
"func_name": "notify",
"message": "Hello!",
"username": "",
"channel_type": "discord",
"channel_id": "999",
"title": "",
}
from unittest.mock import patch
with (
patch("turnstone.core.session.get_storage", return_value=storage),
patch(
"turnstone.core.session.httpx.post",
side_effect=ConnectionError("refused"),
),
patch.dict("os.environ", {}, clear=False),
patch("turnstone.core.session.time.sleep"),
):
call_id, msg = session._exec_notify(item)
# Error message should be generic (no internal details)
assert "delivery failed" in msg.lower()
assert "refused" not in msg
assert "ch-1" not in msg
def test_first_healthy_only(self, tmp_path):
"""Only the first healthy gateway should receive the request."""
from turnstone.core.storage._sqlite import SQLiteBackend
storage = SQLiteBackend(str(tmp_path / "test.db"))
storage.register_service("channel", "ch-1", "http://localhost:8091")
storage.register_service("channel", "ch-2", "http://localhost:8092")
session = _make_session()
item = {
"call_id": "call_1",
"func_name": "notify",
"message": "Hello!",
"username": "admin",
"channel_type": "",
"channel_id": "",
"title": "",
}
mock_resp = MagicMock()
mock_resp.status_code = 200
mock_resp.json.return_value = {
"results": [{"channel_type": "discord", "channel_id": "123", "status": "sent"}]
}
from unittest.mock import patch
with (
patch("turnstone.core.session.get_storage", return_value=storage),
patch("turnstone.core.session.httpx.post", return_value=mock_resp) as mock_post,
patch.dict("os.environ", {}, clear=False),
):
session._exec_notify(item)
# Should only have been called once (first healthy)
assert mock_post.call_count == 1
def test_ssrf_protection(self, tmp_path):
"""URLs with non-http(s) schemes should be skipped."""
from turnstone.core.storage._sqlite import SQLiteBackend
storage = SQLiteBackend(str(tmp_path / "test.db"))
# Register a service with an invalid scheme
storage.register_service("channel", "ch-bad", "ftp://evil.example.com")
session = _make_session()
item = {
"call_id": "call_1",
"func_name": "notify",
"message": "Hello!",
"username": "admin",
"channel_type": "",
"channel_id": "",
"title": "",
}
from unittest.mock import patch
with (
patch("turnstone.core.session.get_storage", return_value=storage),
patch("turnstone.core.session.httpx.post") as mock_post,
patch.dict("os.environ", {}, clear=False),
patch("turnstone.core.session.time.sleep"),
):
call_id, msg = session._exec_notify(item)
# httpx.post should never be called for ftp:// URL
mock_post.assert_not_called()
assert "delivery failed" in msg.lower()
def test_retry_on_no_services(self, tmp_path):
"""Retries service lookup when no gateways are initially available."""
session = _make_session()
item = {
"call_id": "call_1",
"func_name": "notify",
"message": "Hello!",
"username": "admin",
"channel_type": "",
"channel_id": "",
"title": "",
}
# First two calls return empty, third returns a service
call_count = 0
def _list_services(stype: str, max_age_seconds: int = 120) -> list[dict[str, str]]:
nonlocal call_count
call_count += 1
if call_count <= 2:
return []
return [
{
"service_type": "channel",
"service_id": "ch-1",
"url": "http://localhost:8091",
"metadata": "{}",
"last_heartbeat": "",
"created": "",
}
]
mock_resp = MagicMock()
mock_resp.status_code = 200
mock_resp.json.return_value = {
"results": [{"channel_type": "discord", "channel_id": "123", "status": "sent"}]
}
from unittest.mock import patch
mock_storage = MagicMock()
mock_storage.list_services = _list_services
with (
patch("turnstone.core.session.get_storage", return_value=mock_storage),
patch("turnstone.core.session.httpx.post", return_value=mock_resp),
patch.dict("os.environ", {}, clear=False),
patch("turnstone.core.session.time.sleep") as mock_sleep,
):
call_id, msg = session._exec_notify(item)
assert "sent successfully" in msg.lower()
# Should have slept twice (retry delays)
assert mock_sleep.call_count == 2
def test_retry_on_all_gateways_failed(self, tmp_path):
"""Retries when all gateways fail on first attempt but succeed on retry."""
from turnstone.core.storage._sqlite import SQLiteBackend
storage = SQLiteBackend(str(tmp_path / "test.db"))
storage.register_service("channel", "ch-1", "http://localhost:8091")
session = _make_session()
item = {
"call_id": "call_1",
"func_name": "notify",
"message": "Hello!",
"username": "admin",
"channel_type": "",
"channel_id": "",
"title": "",
}
call_count = 0
def _post(*args, **kwargs):
nonlocal call_count
call_count += 1
if call_count <= 1:
raise ConnectionError("refused")
resp = MagicMock()
resp.status_code = 200
resp.json.return_value = {
"results": [{"channel_type": "discord", "channel_id": "123", "status": "sent"}]
}
return resp
from unittest.mock import patch
with (
patch("turnstone.core.session.get_storage", return_value=storage),
patch("turnstone.core.session.httpx.post", side_effect=_post),
patch.dict("os.environ", {}, clear=False),
patch("turnstone.core.session.time.sleep") as mock_sleep,
):
call_id, msg = session._exec_notify(item)
assert "sent successfully" in msg.lower()
assert mock_sleep.call_count == 1
def test_no_services_logs_warning(self, tmp_path):
"""Server-side warning is logged when no services are available."""
from turnstone.core.storage._sqlite import SQLiteBackend
storage = SQLiteBackend(str(tmp_path / "test.db"))
session = _make_session()
item = {
"call_id": "call_1",
"func_name": "notify",
"message": "Hello!",
"username": "admin",
"channel_type": "",
"channel_id": "",
"title": "",
}
from unittest.mock import patch
with (
patch("turnstone.core.session.get_storage", return_value=storage),
patch("turnstone.core.session.time.sleep"),
patch("turnstone.core.session.log") as mock_log,
):
session._exec_notify(item)
# Should have logged warnings for retries + final exhaustion
warning_calls = [c for c in mock_log.warning.call_args_list]
assert len(warning_calls) >= 3 # 2 retry warnings + 1 exhaustion
events = [c.args[0] for c in warning_calls]
assert "notify.no_services" in events
assert "notify.no_services_exhausted" in events
def test_all_gateways_failed_logs_warning(self, tmp_path):
"""Server-side warning is logged when all gateways fail."""
from turnstone.core.storage._sqlite import SQLiteBackend
storage = SQLiteBackend(str(tmp_path / "test.db"))
storage.register_service("channel", "ch-1", "http://localhost:8091")
session = _make_session()
item = {
"call_id": "call_1",
"func_name": "notify",
"message": "Hello!",
"username": "admin",
"channel_type": "",
"channel_id": "",
"title": "",
}
from unittest.mock import patch
with (
patch("turnstone.core.session.get_storage", return_value=storage),
patch(
"turnstone.core.session.httpx.post",
side_effect=ConnectionError("refused"),
),
patch.dict("os.environ", {}, clear=False),
patch("turnstone.core.session.time.sleep"),
patch("turnstone.core.session.log") as mock_log,
):
session._exec_notify(item)
warning_calls = [c for c in mock_log.warning.call_args_list]
events = [c.args[0] for c in warning_calls]
# 2 retry warnings + 1 final failure
assert "notify.all_gateways_failed" in events
assert "notify.delivery_failed" in events
def test_gateway_200_but_no_delivery(self, tmp_path):
"""HTTP 200 with all results failed should not count as success."""
from turnstone.core.storage._sqlite import SQLiteBackend
storage = SQLiteBackend(str(tmp_path / "test.db"))
storage.register_service("channel", "ch-1", "http://localhost:8091")
session = _make_session()
item = {
"call_id": "call_1",
"func_name": "notify",
"message": "Hello!",
"username": "admin",
"channel_type": "",
"channel_id": "",
"title": "",
}
mock_resp = MagicMock()
mock_resp.status_code = 200
mock_resp.json.return_value = {
"results": [{"channel_type": "discord", "channel_id": "123", "status": "no_adapter"}]
}
from unittest.mock import patch
with (
patch("turnstone.core.session.get_storage", return_value=storage),
patch("turnstone.core.session.httpx.post", return_value=mock_resp),
patch.dict("os.environ", {}, clear=False),
patch("turnstone.core.session.time.sleep"),
):
call_id, msg = session._exec_notify(item)
assert "delivery failed" in msg.lower()
assert session._notify_count == 0
+37
View File
@@ -1103,6 +1103,43 @@ class TestOpenAIParameterGating:
assert "temperature" not in kwargs
assert "reasoning_effort" not in kwargs
def test_gpt5_pro_unsupported_effort_falls_back(self) -> None:
"""GPT-5 pro only supports 'high'; unsupported values fall back to default."""
caps = self.provider.get_capabilities("gpt-5-pro")
kwargs: dict[str, Any] = {}
self.provider._apply_model_params(kwargs, caps, temperature=0.7, reasoning_effort="medium")
assert "temperature" not in kwargs
assert kwargs["reasoning_effort"] == "high" # fell back to default
def test_gpt5_pro_supported_effort_passes_through(self) -> None:
"""GPT-5 pro accepts 'high' directly."""
caps = self.provider.get_capabilities("gpt-5-pro")
kwargs: dict[str, Any] = {}
self.provider._apply_model_params(kwargs, caps, temperature=0.7, reasoning_effort="high")
assert kwargs["reasoning_effort"] == "high"
def test_gpt54_1m_context_and_effort(self) -> None:
"""GPT-5.4: 1M context, temperature when effort=none, xhigh supported."""
caps = self.provider.get_capabilities("gpt-5.4")
assert caps.context_window == 1050000
kwargs: dict[str, Any] = {}
self.provider._apply_model_params(kwargs, caps, temperature=0.7, reasoning_effort="none")
assert kwargs["temperature"] == 0.7
assert "reasoning_effort" not in kwargs
kwargs2: dict[str, Any] = {}
self.provider._apply_model_params(kwargs2, caps, temperature=0.7, reasoning_effort="xhigh")
assert "temperature" not in kwargs2
assert kwargs2["reasoning_effort"] == "xhigh"
def test_gpt54_pro_no_temperature_always_reasoning(self) -> None:
"""GPT-5.4 pro: no temperature, medium/high/xhigh only."""
caps = self.provider.get_capabilities("gpt-5.4-pro")
assert caps.context_window == 1050000
kwargs: dict[str, Any] = {}
self.provider._apply_model_params(kwargs, caps, temperature=0.7, reasoning_effort="low")
assert "temperature" not in kwargs
assert kwargs["reasoning_effort"] == "medium" # fell back from unsupported "low"
class TestAnthropicReasoningNone:
"""Verify 'none' effort disables thinking for manual-thinking models."""
+264
View File
@@ -0,0 +1,264 @@
"""Tests for scheduled task admin API endpoints."""
from __future__ import annotations
import pytest
from starlette.applications import Starlette
from starlette.routing import Mount, Route
from starlette.testclient import TestClient
from turnstone.console.server import (
admin_create_schedule,
admin_delete_schedule,
admin_get_schedule,
admin_list_schedule_runs,
admin_list_schedules,
admin_update_schedule,
)
from turnstone.core.storage._sqlite import SQLiteBackend
@pytest.fixture
def storage(tmp_path):
"""Fresh SQLite backend for each test."""
return SQLiteBackend(str(tmp_path / "test.db"))
@pytest.fixture
def client(storage):
"""TestClient with storage and auth bypassed."""
app = Starlette(
routes=[
Mount(
"/v1",
routes=[
Route("/api/admin/schedules", admin_list_schedules),
Route("/api/admin/schedules", admin_create_schedule, methods=["POST"]),
Route("/api/admin/schedules/{task_id}", admin_get_schedule),
Route(
"/api/admin/schedules/{task_id}",
admin_update_schedule,
methods=["PUT"],
),
Route(
"/api/admin/schedules/{task_id}",
admin_delete_schedule,
methods=["DELETE"],
),
Route(
"/api/admin/schedules/{task_id}/runs",
admin_list_schedule_runs,
),
],
),
],
)
app.state.auth_storage = storage
return TestClient(app)
def _cron_payload(**overrides):
"""Build default cron schedule creation payload."""
defaults = {
"name": "Daily report",
"description": "Generate the summary",
"schedule_type": "cron",
"cron_expr": "0 9 * * *",
"target_mode": "auto",
"model": "gpt-5",
"initial_message": "Generate the daily report",
}
defaults.update(overrides)
return defaults
def _at_payload(**overrides):
"""Build default at-time schedule creation payload."""
defaults = {
"name": "One-shot task",
"description": "Run once",
"schedule_type": "at",
"at_time": "2099-01-01T00:00:00+00:00",
"target_mode": "auto",
"model": "gpt-5",
"initial_message": "Do the thing",
}
defaults.update(overrides)
return defaults
class TestScheduleAPI:
"""Tests for the 6 admin schedule endpoints."""
def test_list_empty(self, client):
resp = client.get("/v1/api/admin/schedules")
assert resp.status_code == 200
data = resp.json()
assert data["schedules"] == []
def test_create_cron(self, client):
resp = client.post("/v1/api/admin/schedules", json=_cron_payload())
assert resp.status_code == 200
task = resp.json()
assert task["name"] == "Daily report"
assert task["schedule_type"] == "cron"
assert task["cron_expr"] == "0 9 * * *"
assert task["enabled"] is True
assert "task_id" in task
assert "created" in task
assert "next_run" in task
assert task["next_run"] != ""
def test_create_at(self, client):
resp = client.post("/v1/api/admin/schedules", json=_at_payload())
assert resp.status_code == 200
task = resp.json()
assert task["schedule_type"] == "at"
assert task["at_time"] == "2099-01-01T00:00:00+00:00"
assert task["next_run"] == "2099-01-01T00:00:00+00:00"
def test_create_missing_name(self, client):
payload = _cron_payload()
del payload["name"]
resp = client.post("/v1/api/admin/schedules", json=payload)
assert resp.status_code == 400
assert "name" in resp.json()["error"].lower()
def test_create_invalid_cron(self, client):
resp = client.post(
"/v1/api/admin/schedules",
json=_cron_payload(cron_expr="not a cron"),
)
assert resp.status_code == 400
assert "cron" in resp.json()["error"].lower()
def test_create_naive_at_time(self, client):
"""Naive timestamps (no timezone) should be rejected."""
resp = client.post(
"/v1/api/admin/schedules",
json=_at_payload(at_time="2099-01-01T00:00:00"),
)
assert resp.status_code == 400
assert "timezone" in resp.json()["error"].lower()
def test_create_past_at_time(self, client):
resp = client.post(
"/v1/api/admin/schedules",
json=_at_payload(at_time="2000-01-01T00:00:00+00:00"),
)
assert resp.status_code == 400
assert "future" in resp.json()["error"].lower()
def test_get_schedule(self, client):
create_resp = client.post("/v1/api/admin/schedules", json=_cron_payload())
task_id = create_resp.json()["task_id"]
resp = client.get(f"/v1/api/admin/schedules/{task_id}")
assert resp.status_code == 200
assert resp.json()["task_id"] == task_id
assert resp.json()["name"] == "Daily report"
def test_get_nonexistent(self, client):
resp = client.get("/v1/api/admin/schedules/nonexistent_id")
assert resp.status_code == 404
def test_update_schedule(self, client):
create_resp = client.post("/v1/api/admin/schedules", json=_cron_payload())
task_id = create_resp.json()["task_id"]
resp = client.put(
f"/v1/api/admin/schedules/{task_id}",
json={"name": "Weekly report"},
)
assert resp.status_code == 200
assert resp.json()["name"] == "Weekly report"
# Verify via GET
get_resp = client.get(f"/v1/api/admin/schedules/{task_id}")
assert get_resp.json()["name"] == "Weekly report"
def test_update_nonexistent(self, client):
resp = client.put(
"/v1/api/admin/schedules/nonexistent_id",
json={"name": "Nope"},
)
assert resp.status_code == 404
def test_delete_schedule(self, client):
create_resp = client.post("/v1/api/admin/schedules", json=_cron_payload())
task_id = create_resp.json()["task_id"]
resp = client.delete(f"/v1/api/admin/schedules/{task_id}")
assert resp.status_code == 200
assert resp.json()["status"] == "ok"
# Verify gone
get_resp = client.get(f"/v1/api/admin/schedules/{task_id}")
assert get_resp.status_code == 404
def test_delete_nonexistent(self, client):
resp = client.delete("/v1/api/admin/schedules/nonexistent_id")
assert resp.status_code == 404
def test_list_runs_empty(self, client):
create_resp = client.post("/v1/api/admin/schedules", json=_cron_payload())
task_id = create_resp.json()["task_id"]
resp = client.get(f"/v1/api/admin/schedules/{task_id}/runs")
assert resp.status_code == 200
assert resp.json()["runs"] == []
def test_list_runs_nonexistent(self, client):
resp = client.get("/v1/api/admin/schedules/nonexistent_id/runs")
assert resp.status_code == 404
def test_create_specific_node_target(self, client):
payload = _cron_payload(target_mode="node-custom-001")
resp = client.post("/v1/api/admin/schedules", json=payload)
assert resp.status_code == 200
data = resp.json()
assert data["target_mode"] == "node-custom-001"
def test_list_runs_with_data(self, client, storage):
create_resp = client.post("/v1/api/admin/schedules", json=_cron_payload())
task_id = create_resp.json()["task_id"]
# Record runs directly in storage
storage.record_task_run(
run_id="run_001",
task_id=task_id,
node_id="node-1",
ws_id="ws_abc",
correlation_id="corr_001",
started="2025-06-01T09:00:00",
status="dispatched",
error="",
)
storage.record_task_run(
run_id="run_002",
task_id=task_id,
node_id="node-2",
ws_id="",
correlation_id="corr_002",
started="2025-06-01T09:01:00",
status="failed",
error="No reachable nodes",
)
resp = client.get(f"/v1/api/admin/schedules/{task_id}/runs")
assert resp.status_code == 200
runs = resp.json()["runs"]
assert len(runs) == 2
# Most recent first
assert runs[0]["run_id"] == "run_002"
assert runs[0]["status"] == "failed"
assert runs[1]["run_id"] == "run_001"
def test_list_runs_invalid_limit(self, client):
create_resp = client.post("/v1/api/admin/schedules", json=_cron_payload())
task_id = create_resp.json()["task_id"]
# Invalid limit should not crash — falls back to 50
resp = client.get(f"/v1/api/admin/schedules/{task_id}/runs?limit=abc")
assert resp.status_code == 200
assert resp.json()["runs"] == []
+259
View File
@@ -0,0 +1,259 @@
"""Tests for scheduled_tasks and scheduled_task_runs storage CRUD."""
from __future__ import annotations
import time
import pytest
from turnstone.core.storage._sqlite import SQLiteBackend
@pytest.fixture
def db(tmp_path):
"""Fresh SQLite backend for each test."""
backend = SQLiteBackend(str(tmp_path / "test.db"))
return backend
def _make_task_kwargs(**overrides):
"""Build default kwargs for create_scheduled_task."""
defaults = {
"task_id": "task_001",
"name": "Daily report",
"description": "Generate the daily summary",
"schedule_type": "cron",
"cron_expr": "0 9 * * *",
"at_time": "",
"target_mode": "auto",
"model": "gpt-5",
"initial_message": "Generate the daily report",
"auto_approve": False,
"auto_approve_tools": [],
"created_by": "u_admin",
"next_run": "2099-01-01T09:00:00",
}
defaults.update(overrides)
return defaults
class TestScheduledTaskCRUD:
"""Tests for scheduled_tasks table operations."""
def test_create_and_get(self, db):
db.create_scheduled_task(**_make_task_kwargs())
result = db.get_scheduled_task("task_001")
assert result is not None
assert result["task_id"] == "task_001"
assert result["name"] == "Daily report"
assert result["description"] == "Generate the daily summary"
assert result["schedule_type"] == "cron"
assert result["cron_expr"] == "0 9 * * *"
assert result["at_time"] == ""
assert result["target_mode"] == "auto"
assert result["model"] == "gpt-5"
assert result["initial_message"] == "Generate the daily report"
assert result["auto_approve"] == 0
assert result["auto_approve_tools"] == ""
assert result["enabled"] == 1
assert result["created_by"] == "u_admin"
assert result["next_run"] == "2099-01-01T09:00:00"
assert "created" in result
assert "updated" in result
def test_get_nonexistent(self, db):
assert db.get_scheduled_task("no_such_task") is None
def test_create_duplicate_noop(self, db):
db.create_scheduled_task(**_make_task_kwargs(name="First"))
db.create_scheduled_task(**_make_task_kwargs(name="Second"))
result = db.get_scheduled_task("task_001")
assert result is not None
assert result["name"] == "First" # first write wins
def test_list_tasks(self, db):
db.create_scheduled_task(**_make_task_kwargs(task_id="task_a", name="Alpha"))
# Ensure different created timestamps (resolution is 1 second)
time.sleep(1.1)
db.create_scheduled_task(**_make_task_kwargs(task_id="task_b", name="Beta"))
tasks = db.list_scheduled_tasks()
assert len(tasks) == 2
# Ordered by created DESC — most recent first
assert tasks[0]["task_id"] == "task_b"
assert tasks[1]["task_id"] == "task_a"
def test_update_task(self, db):
db.create_scheduled_task(**_make_task_kwargs())
original = db.get_scheduled_task("task_001")
assert original is not None
original_updated = original["updated"]
time.sleep(0.05)
result = db.update_scheduled_task("task_001", name="Weekly report")
assert result is True
updated = db.get_scheduled_task("task_001")
assert updated is not None
assert updated["name"] == "Weekly report"
assert updated["updated"] >= original_updated
def test_update_enable_disable(self, db):
db.create_scheduled_task(**_make_task_kwargs())
task = db.get_scheduled_task("task_001")
assert task is not None
assert task["enabled"] == 1
db.update_scheduled_task("task_001", enabled=False)
task = db.get_scheduled_task("task_001")
assert task is not None
assert task["enabled"] == 0
db.update_scheduled_task("task_001", enabled=True)
task = db.get_scheduled_task("task_001")
assert task is not None
assert task["enabled"] == 1
def test_delete_task(self, db):
db.create_scheduled_task(**_make_task_kwargs())
assert db.delete_scheduled_task("task_001") is True
assert db.get_scheduled_task("task_001") is None
# Deleting again returns False
assert db.delete_scheduled_task("task_001") is False
def test_delete_cascades_runs(self, db):
db.create_scheduled_task(**_make_task_kwargs())
db.record_task_run(
run_id="run_001",
task_id="task_001",
node_id="node_1",
ws_id="ws_abc",
correlation_id="corr_001",
started="2025-01-01T09:00:00",
status="dispatched",
error="",
)
assert len(db.list_task_runs("task_001")) == 1
db.delete_scheduled_task("task_001")
assert db.list_task_runs("task_001") == []
def test_list_due_tasks(self, db):
db.create_scheduled_task(
**_make_task_kwargs(task_id="past", next_run="2020-01-01T00:00:00")
)
db.create_scheduled_task(
**_make_task_kwargs(task_id="future", next_run="2099-12-31T23:59:59")
)
now = "2025-06-01T12:00:00"
due = db.list_due_tasks(now)
assert len(due) == 1
assert due[0]["task_id"] == "past"
def test_list_due_tasks_skips_disabled(self, db):
db.create_scheduled_task(
**_make_task_kwargs(task_id="disabled_task", next_run="2020-01-01T00:00:00")
)
db.update_scheduled_task("disabled_task", enabled=False)
due = db.list_due_tasks("2025-06-01T12:00:00")
assert len(due) == 0
def test_list_due_tasks_empty_next_run(self, db):
db.create_scheduled_task(**_make_task_kwargs(task_id="empty_next", next_run=""))
due = db.list_due_tasks("2099-12-31T23:59:59")
assert len(due) == 0
def test_at_task_fields(self, db):
db.create_scheduled_task(
**_make_task_kwargs(
task_id="at_task",
schedule_type="at",
cron_expr="",
at_time="2099-06-15T14:00:00",
next_run="2099-06-15T14:00:00",
)
)
result = db.get_scheduled_task("at_task")
assert result is not None
assert result["schedule_type"] == "at"
assert result["at_time"] == "2099-06-15T14:00:00"
class TestScheduledTaskRuns:
"""Tests for scheduled_task_runs table operations."""
def test_record_and_list(self, db):
db.create_scheduled_task(**_make_task_kwargs())
db.record_task_run(
run_id="run_a",
task_id="task_001",
node_id="node_1",
ws_id="ws_1",
correlation_id="corr_a",
started="2025-01-01T09:00:00",
status="dispatched",
error="",
)
db.record_task_run(
run_id="run_b",
task_id="task_001",
node_id="node_2",
ws_id="ws_2",
correlation_id="corr_b",
started="2025-01-02T09:00:00",
status="dispatched",
error="",
)
runs = db.list_task_runs("task_001")
assert len(runs) == 2
# Ordered by started DESC — most recent first
assert runs[0]["run_id"] == "run_b"
assert runs[1]["run_id"] == "run_a"
def test_list_runs_respects_limit(self, db):
db.create_scheduled_task(**_make_task_kwargs())
for i in range(3):
db.record_task_run(
run_id=f"run_{i}",
task_id="task_001",
node_id="node_1",
ws_id="",
correlation_id=f"corr_{i}",
started=f"2025-01-0{i + 1}T09:00:00",
status="dispatched",
error="",
)
runs = db.list_task_runs("task_001", limit=2)
assert len(runs) == 2
def test_list_runs_empty(self, db):
assert db.list_task_runs("no_such_task") == []
def test_prune_task_runs(self, db):
db.create_scheduled_task(**_make_task_kwargs())
# Old run (should be pruned)
db.record_task_run(
run_id="old_run",
task_id="task_001",
node_id="node_1",
ws_id="",
correlation_id="c_old",
started="2020-01-01T00:00:00",
status="dispatched",
error="",
)
# Recent run (should survive)
db.record_task_run(
run_id="new_run",
task_id="task_001",
node_id="node_1",
ws_id="",
correlation_id="c_new",
started="2099-01-01T00:00:00",
status="dispatched",
error="",
)
pruned = db.prune_task_runs(retention_days=90)
assert pruned == 1
runs = db.list_task_runs("task_001")
assert len(runs) == 1
assert runs[0]["run_id"] == "new_run"
+272
View File
@@ -0,0 +1,272 @@
"""Tests for turnstone.console.scheduler — TaskScheduler tick and dispatch."""
from __future__ import annotations
from unittest.mock import MagicMock
import pytest
from turnstone.console.scheduler import TaskScheduler
@pytest.fixture
def mocks():
"""Broker, collector, and storage mocks for scheduler tests."""
broker = MagicMock()
broker._redis = MagicMock()
collector = MagicMock()
storage = MagicMock()
return broker, collector, storage
def _make_task(**overrides):
"""Build a minimal task dict matching storage row format."""
defaults = {
"task_id": "task_001",
"name": "Test task",
"description": "",
"schedule_type": "cron",
"cron_expr": "0 9 * * *",
"at_time": "",
"target_mode": "auto",
"model": "gpt-5",
"initial_message": "Run the tests",
"auto_approve": 0,
"auto_approve_tools": "",
"enabled": 1,
"created_by": "u_admin",
"next_run": "2020-01-01T09:00:00",
"last_run": "",
"created": "2020-01-01T00:00:00",
"updated": "2020-01-01T00:00:00",
}
defaults.update(overrides)
return defaults
def _make_node(node_id="node-001", reachable=True, ws_total=2, max_ws=10):
"""Build a minimal node dict matching collector output."""
return {
"node_id": node_id,
"reachable": reachable,
"ws_total": ws_total,
"max_ws": max_ws,
}
class TestSchedulerTick:
"""Tests for _tick() lock acquisition and dispatch logic."""
def test_tick_acquires_lock(self, mocks):
broker, collector, storage = mocks
broker._redis.set.return_value = True
storage.list_due_tasks.return_value = []
scheduler = TaskScheduler(broker, collector, storage)
scheduler._tick()
broker._redis.set.assert_called_once()
storage.list_due_tasks.assert_called_once()
# Lock released via Lua eval (conditional delete)
broker._redis.eval.assert_called_once()
def test_tick_skips_when_locked(self, mocks):
broker, collector, storage = mocks
broker._redis.set.return_value = None # lock held by another console
scheduler = TaskScheduler(broker, collector, storage)
scheduler._tick()
storage.list_due_tasks.assert_not_called()
def test_dispatch_auto_mode(self, mocks):
broker, collector, storage = mocks
broker._redis.set.return_value = True
task = _make_task(target_mode="auto")
storage.list_due_tasks.return_value = [task]
collector.get_nodes.return_value = ([_make_node()], 1)
scheduler = TaskScheduler(broker, collector, storage)
scheduler._tick()
broker.push_inbound.assert_called_once()
_, kwargs = broker.push_inbound.call_args
assert (
kwargs.get("node_id") == "node-001"
or broker.push_inbound.call_args[1].get("node_id") == "node-001"
)
storage.record_task_run.assert_called_once()
run_kwargs = storage.record_task_run.call_args[1]
assert run_kwargs["node_id"] == "node-001"
assert run_kwargs["status"] == "dispatched"
def test_dispatch_pool_mode(self, mocks):
broker, collector, storage = mocks
broker._redis.set.return_value = True
task = _make_task(target_mode="pool")
storage.list_due_tasks.return_value = [task]
scheduler = TaskScheduler(broker, collector, storage)
scheduler._tick()
broker.push_inbound.assert_called_once()
# Pool dispatch calls push_inbound without node_id kwarg
args, kwargs = broker.push_inbound.call_args
assert kwargs.get("node_id") is None or "node_id" not in kwargs
storage.record_task_run.assert_called_once()
run_kwargs = storage.record_task_run.call_args[1]
assert run_kwargs["node_id"] == "pool"
def test_dispatch_all_mode(self, mocks):
broker, collector, storage = mocks
broker._redis.set.return_value = True
task = _make_task(target_mode="all")
storage.list_due_tasks.return_value = [task]
collector.get_nodes.return_value = (
[_make_node("node-001"), _make_node("node-002")],
2,
)
scheduler = TaskScheduler(broker, collector, storage)
scheduler._tick()
assert broker.push_inbound.call_count == 2
assert storage.record_task_run.call_count == 2
def test_dispatch_specific_node(self, mocks):
broker, collector, storage = mocks
broker._redis.set.return_value = True
task = _make_task(target_mode="node-001")
storage.list_due_tasks.return_value = [task]
scheduler = TaskScheduler(broker, collector, storage)
scheduler._tick()
broker.push_inbound.assert_called_once()
_, kwargs = broker.push_inbound.call_args
assert kwargs["node_id"] == "node-001"
def test_at_task_disables_after_dispatch(self, mocks):
broker, collector, storage = mocks
broker._redis.set.return_value = True
task = _make_task(schedule_type="at", cron_expr="", at_time="2099-01-01T00:00:00")
storage.list_due_tasks.return_value = [task]
collector.get_nodes.return_value = ([_make_node()], 1)
scheduler = TaskScheduler(broker, collector, storage)
scheduler._tick()
# At-task should be disabled after dispatch
update_calls = storage.update_scheduled_task.call_args_list
assert len(update_calls) == 1
args, kwargs = update_calls[0]
assert args[0] == "task_001"
assert kwargs["enabled"] is False
assert kwargs["next_run"] == ""
def test_cron_task_updates_next_run(self, mocks):
broker, collector, storage = mocks
broker._redis.set.return_value = True
task = _make_task(schedule_type="cron", cron_expr="0 9 * * *")
storage.list_due_tasks.return_value = [task]
collector.get_nodes.return_value = ([_make_node()], 1)
scheduler = TaskScheduler(broker, collector, storage)
scheduler._tick()
update_calls = storage.update_scheduled_task.call_args_list
assert len(update_calls) == 1
_, kwargs = update_calls[0]
assert kwargs["next_run"] != ""
assert "enabled" not in kwargs # cron tasks stay enabled
def test_no_reachable_nodes_records_failure(self, mocks):
broker, collector, storage = mocks
broker._redis.set.return_value = True
task = _make_task(target_mode="auto")
storage.list_due_tasks.return_value = [task]
# No reachable nodes
collector.get_nodes.return_value = (
[_make_node("node-001", reachable=False)],
1,
)
scheduler = TaskScheduler(broker, collector, storage)
scheduler._tick()
broker.push_inbound.assert_not_called()
storage.record_task_run.assert_called_once()
run_kwargs = storage.record_task_run.call_args[1]
assert run_kwargs["status"] == "failed"
assert run_kwargs["error"] != ""
def test_failure_does_not_advance_schedule(self, mocks):
"""When dispatch fails, last_run/next_run should not be updated."""
broker, collector, storage = mocks
broker._redis.set.return_value = True
task = _make_task(target_mode="auto")
storage.list_due_tasks.return_value = [task]
collector.get_nodes.return_value = ([], 0) # no nodes at all
scheduler = TaskScheduler(broker, collector, storage)
scheduler._tick()
# update_scheduled_task should NOT be called (no last_run/next_run advance)
storage.update_scheduled_task.assert_not_called()
def test_fan_out_capped(self, mocks):
"""Fan-out 'all' mode should respect max_fan_out limit."""
broker, collector, storage = mocks
broker._redis.set.return_value = True
task = _make_task(target_mode="all")
storage.list_due_tasks.return_value = [task]
# 10 reachable nodes but max_fan_out=3
nodes = [_make_node(f"node-{i:03d}") for i in range(10)]
collector.get_nodes.return_value = (nodes, 10)
scheduler = TaskScheduler(broker, collector, storage, max_fan_out=3)
scheduler._tick()
assert broker.push_inbound.call_count == 3
assert storage.record_task_run.call_count == 3
def test_specific_node_target(self, mocks):
"""Non-enum target_mode is treated as a specific node_id."""
broker, collector, storage = mocks
broker._redis.set.return_value = True
task = _make_task(target_mode="node-custom-123")
storage.list_due_tasks.return_value = [task]
scheduler = TaskScheduler(broker, collector, storage)
scheduler._tick()
broker.push_inbound.assert_called_once()
call_kwargs = broker.push_inbound.call_args
assert call_kwargs[1]["node_id"] == "node-custom-123"
def test_user_id_in_dispatched_message(self, mocks):
"""Dispatched message should include created_by as user_id."""
import json
broker, collector, storage = mocks
broker._redis.set.return_value = True
task = _make_task(target_mode="pool", created_by="u_scheduler_admin")
storage.list_due_tasks.return_value = [task]
scheduler = TaskScheduler(broker, collector, storage)
scheduler._tick()
msg_json = broker.push_inbound.call_args[0][0]
msg_data = json.loads(msg_json)
assert msg_data["user_id"] == "u_scheduler_admin"
+139
View File
@@ -2,6 +2,8 @@
from __future__ import annotations
import json
import httpx
import pytest
@@ -240,3 +242,140 @@ async def test_query_params_passed():
assert "state=running" in captured_url[0]
assert "page=2" in captured_url[0]
assert "per_page=25" in captured_url[0]
# ---------------------------------------------------------------------------
# Schedules
# ---------------------------------------------------------------------------
_SCHEDULE_FIXTURE = {
"task_id": "t1",
"name": "nightly",
"description": "",
"schedule_type": "cron",
"cron_expr": "0 2 * * *",
"at_time": "",
"target_mode": "auto",
"model": "",
"initial_message": "Run nightly checks",
"auto_approve": False,
"auto_approve_tools": [],
"enabled": True,
"created_by": "u1",
"last_run": None,
"next_run": "2026-03-06T02:00:00Z",
"created": "2026-03-05T12:00:00Z",
"updated": "2026-03-05T12:00:00Z",
}
@pytest.mark.anyio
async def test_list_schedules():
transport = _mock_transport(
{"GET /v1/api/admin/schedules": _json_response({"schedules": [_SCHEDULE_FIXTURE]})}
)
async with httpx.AsyncClient(transport=transport, base_url="http://test") as hc:
client = AsyncTurnstoneConsole(httpx_client=hc)
resp = await client.list_schedules()
assert len(resp.schedules) == 1
assert resp.schedules[0].task_id == "t1"
assert resp.schedules[0].name == "nightly"
@pytest.mark.anyio
async def test_create_schedule():
captured_body: list[dict] = []
def handler(request: httpx.Request) -> httpx.Response:
captured_body.append(json.loads(request.content))
return _json_response(_SCHEDULE_FIXTURE)
transport = httpx.MockTransport(handler)
async with httpx.AsyncClient(transport=transport, base_url="http://test") as hc:
client = AsyncTurnstoneConsole(httpx_client=hc)
resp = await client.create_schedule(
name="nightly",
schedule_type="cron",
initial_message="Run nightly checks",
cron_expr="0 2 * * *",
)
assert resp.task_id == "t1"
body = captured_body[0]
assert body["name"] == "nightly"
assert body["schedule_type"] == "cron"
assert body["cron_expr"] == "0 2 * * *"
assert body["initial_message"] == "Run nightly checks"
# Optional fields with defaults should not appear when not set
assert "description" not in body
assert "model" not in body
@pytest.mark.anyio
async def test_get_schedule():
transport = _mock_transport(
{"GET /v1/api/admin/schedules/t1": _json_response(_SCHEDULE_FIXTURE)}
)
async with httpx.AsyncClient(transport=transport, base_url="http://test") as hc:
client = AsyncTurnstoneConsole(httpx_client=hc)
resp = await client.get_schedule("t1")
assert resp.task_id == "t1"
assert resp.schedule_type == "cron"
@pytest.mark.anyio
async def test_update_schedule_partial():
"""Only explicitly-passed fields should appear in the request body."""
captured_body: list[dict] = []
def handler(request: httpx.Request) -> httpx.Response:
captured_body.append(json.loads(request.content))
return _json_response({**_SCHEDULE_FIXTURE, "enabled": False})
transport = httpx.MockTransport(handler)
async with httpx.AsyncClient(transport=transport, base_url="http://test") as hc:
client = AsyncTurnstoneConsole(httpx_client=hc)
resp = await client.update_schedule("t1", enabled=False)
assert resp.enabled is False
body = captured_body[0]
assert body == {"enabled": False}
@pytest.mark.anyio
async def test_delete_schedule():
transport = _mock_transport(
{"DELETE /v1/api/admin/schedules/t1": _json_response({"status": "ok"})}
)
async with httpx.AsyncClient(transport=transport, base_url="http://test") as hc:
client = AsyncTurnstoneConsole(httpx_client=hc)
resp = await client.delete_schedule("t1")
assert resp.status == "ok"
@pytest.mark.anyio
async def test_list_schedule_runs():
transport = _mock_transport(
{
"GET /v1/api/admin/schedules/t1/runs": _json_response(
{
"runs": [
{
"run_id": "r1",
"task_id": "t1",
"node_id": "n1",
"ws_id": "ws1",
"correlation_id": "c1",
"started": "2026-03-05T02:00:00Z",
"status": "dispatched",
"error": "",
}
]
}
)
}
)
async with httpx.AsyncClient(transport=transport, base_url="http://test") as hc:
client = AsyncTurnstoneConsole(httpx_client=hc)
resp = await client.list_schedule_runs("t1", limit=10)
assert len(resp.runs) == 1
assert resp.runs[0].run_id == "r1"
assert resp.runs[0].status == "dispatched"
+86
View File
@@ -0,0 +1,86 @@
"""Tests for the services registry storage methods."""
from __future__ import annotations
import pytest
from turnstone.core.storage._sqlite import SQLiteBackend
@pytest.fixture
def storage(tmp_path):
return SQLiteBackend(str(tmp_path / "test.db"))
class TestServiceRegistry:
def test_register_and_list(self, storage):
storage.register_service("channel", "ch-1", "http://localhost:8091")
services = storage.list_services("channel", max_age_seconds=120)
assert len(services) == 1
assert services[0]["service_type"] == "channel"
assert services[0]["service_id"] == "ch-1"
assert services[0]["url"] == "http://localhost:8091"
def test_register_upsert(self, storage):
storage.register_service("channel", "ch-1", "http://old:8091")
storage.register_service("channel", "ch-1", "http://new:8091")
services = storage.list_services("channel", max_age_seconds=120)
assert len(services) == 1
assert services[0]["url"] == "http://new:8091"
def test_heartbeat(self, storage):
storage.register_service("channel", "ch-1", "http://localhost:8091")
result = storage.heartbeat_service("channel", "ch-1")
assert result is True
def test_heartbeat_nonexistent(self, storage):
result = storage.heartbeat_service("channel", "nonexistent")
assert result is False
def test_list_filters_stale(self, storage):
storage.register_service("channel", "ch-1", "http://localhost:8091")
# Manually set heartbeat to the past so it's stale
from datetime import UTC, datetime, timedelta
import sqlalchemy as sa
from turnstone.core.storage._schema import services
old_time = (datetime.now(UTC) - timedelta(seconds=300)).strftime("%Y-%m-%dT%H:%M:%S")
with storage._engine.connect() as conn:
conn.execute(sa.update(services).values(last_heartbeat=old_time))
conn.commit()
# Should be excluded with 120s max age
result = storage.list_services("channel", max_age_seconds=120)
assert len(result) == 0
def test_list_empty(self, storage):
services = storage.list_services("channel", max_age_seconds=120)
assert services == []
def test_list_filters_by_type(self, storage):
storage.register_service("channel", "ch-1", "http://localhost:8091")
storage.register_service("bridge", "br-1", "http://localhost:8080")
channels = storage.list_services("channel", max_age_seconds=120)
bridges = storage.list_services("bridge", max_age_seconds=120)
assert len(channels) == 1
assert len(bridges) == 1
def test_deregister(self, storage):
storage.register_service("channel", "ch-1", "http://localhost:8091")
result = storage.deregister_service("channel", "ch-1")
assert result is True
services = storage.list_services("channel", max_age_seconds=120)
assert services == []
def test_deregister_nonexistent(self, storage):
result = storage.deregister_service("channel", "nonexistent")
assert result is False
def test_metadata(self, storage):
storage.register_service(
"channel", "ch-1", "http://localhost:8091", metadata='{"adapter": "discord"}'
)
services = storage.list_services("channel", max_age_seconds=120)
assert services[0]["metadata"] == '{"adapter": "discord"}'
+5 -4
View File
@@ -72,16 +72,16 @@ class TestToolsMetadata:
"""Validate the metadata extracted from JSON files."""
def test_tool_count(self):
assert len(TOOLS) == 14
assert len(TOOLS) == 15
def test_agent_tools_count(self):
assert len(AGENT_TOOLS) == 6
assert len(AGENT_TOOLS) == 7
def test_task_agent_tools_count(self):
assert len(TASK_AGENT_TOOLS) == 9
assert len(TASK_AGENT_TOOLS) == 10
def test_auto_approve_sets_match(self):
expected = {"read_file", "search", "math", "man", "web_fetch", "web_search"}
expected = {"read_file", "search", "math", "man", "web_fetch", "web_search", "notify"}
assert expected == AGENT_AUTO_TOOLS
assert expected == TASK_AUTO_TOOLS
@@ -101,6 +101,7 @@ class TestToolsMetadata:
"remember": "key",
"recall": "query",
"forget": "key",
"notify": "message",
}
assert expected == PRIMARY_KEY_MAP
+1 -1
View File
@@ -1,3 +1,3 @@
"""turnstone - Multi-node AI orchestration platform with tool use, agent routing, and cluster simulation."""
__version__ = "0.4.0"
__version__ = "0.4.2"
+65
View File
@@ -23,13 +23,18 @@ from turnstone.api.schemas import (
AuthSetupRequest,
AuthSetupResponse,
AuthStatusResponse,
CreateScheduleRequest,
CreateTokenRequest,
CreateTokenResponse,
CreateUserRequest,
ErrorResponse,
ListScheduleRunsResponse,
ListSchedulesResponse,
ListTokensResponse,
ListUsersResponse,
ScheduleInfo,
StatusResponse,
UpdateScheduleRequest,
UserInfo,
)
@@ -182,6 +187,61 @@ CONSOLE_ENDPOINTS: list[EndpointSpec] = [
error_codes=[404],
tags=["Admin"],
),
# --- Schedules ---
EndpointSpec(
"/v1/api/admin/schedules",
"GET",
"List all scheduled tasks",
response_model=ListSchedulesResponse,
tags=["Schedules"],
),
EndpointSpec(
"/v1/api/admin/schedules",
"POST",
"Create a scheduled task",
request_model=CreateScheduleRequest,
response_model=ScheduleInfo,
error_codes=[400],
tags=["Schedules"],
),
EndpointSpec(
"/v1/api/admin/schedules/{task_id}",
"GET",
"Get a scheduled task",
response_model=ScheduleInfo,
error_codes=[404],
tags=["Schedules"],
),
EndpointSpec(
"/v1/api/admin/schedules/{task_id}",
"PUT",
"Update a scheduled task",
request_model=UpdateScheduleRequest,
response_model=ScheduleInfo,
error_codes=[400, 404],
tags=["Schedules"],
),
EndpointSpec(
"/v1/api/admin/schedules/{task_id}",
"DELETE",
"Delete a scheduled task",
response_model=StatusResponse,
error_codes=[404],
tags=["Schedules"],
),
EndpointSpec(
"/v1/api/admin/schedules/{task_id}/runs",
"GET",
"List run history for a scheduled task",
response_model=ListScheduleRunsResponse,
query_params=[
QueryParam(
"limit", "Max results (default 50, max 200)", schema_type="integer", default=50
),
],
error_codes=[404],
tags=["Schedules"],
),
# --- Observability ---
EndpointSpec(
"/health",
@@ -213,6 +273,11 @@ _ALL_MODELS: list[type[BaseModel]] = [
ConsoleCreateWsRequest,
ConsoleCreateWsResponse,
ConsoleHealthResponse,
CreateScheduleRequest,
UpdateScheduleRequest,
ScheduleInfo,
ListSchedulesResponse,
ListScheduleRunsResponse,
]
+84
View File
@@ -155,3 +155,87 @@ class AuthStatusResponse(BaseModel):
auth_enabled: bool
has_users: bool
setup_required: bool
# ---------------------------------------------------------------------------
# Schedules
# ---------------------------------------------------------------------------
class CreateScheduleRequest(BaseModel):
"""POST /v1/api/admin/schedules request body."""
name: str = Field(description="Human-readable schedule name")
description: str = Field(default="", description="Optional description")
schedule_type: str = Field(description="'cron' or 'at'")
cron_expr: str = Field(default="", description="Cron expression (when schedule_type='cron')")
at_time: str = Field(default="", description="ISO8601 timestamp (when schedule_type='at')")
target_mode: str = Field(default="auto", description="auto, pool, all, or specific node_id")
model: str = Field(default="", description="Model alias for the workstream")
initial_message: str = Field(description="Message sent to the new workstream")
auto_approve: bool = Field(default=False)
auto_approve_tools: list[str] = Field(default_factory=list)
enabled: bool = Field(default=True)
class UpdateScheduleRequest(BaseModel):
"""PUT /v1/api/admin/schedules/{task_id} request body (partial update)."""
name: str | None = None
description: str | None = None
schedule_type: str | None = None
cron_expr: str | None = None
at_time: str | None = None
target_mode: str | None = None
model: str | None = None
initial_message: str | None = None
auto_approve: bool | None = None
auto_approve_tools: list[str] | None = None
enabled: bool | None = None
class ScheduleInfo(BaseModel):
"""Scheduled task details."""
task_id: str
name: str
description: str = ""
schedule_type: str
cron_expr: str = ""
at_time: str = ""
target_mode: str = "auto"
model: str = ""
initial_message: str
auto_approve: bool = False
auto_approve_tools: list[str] = Field(default_factory=list)
enabled: bool = True
created_by: str = ""
last_run: str | None = None
next_run: str | None = None
created: str = ""
updated: str = ""
class ListSchedulesResponse(BaseModel):
"""GET /v1/api/admin/schedules response."""
schedules: list[ScheduleInfo]
class ScheduleRunInfo(BaseModel):
"""Single execution record for a scheduled task."""
run_id: str
task_id: str
node_id: str = ""
ws_id: str = ""
correlation_id: str = ""
started: str
status: str = "dispatched"
error: str = ""
class ListScheduleRunsResponse(BaseModel):
"""GET /v1/api/admin/schedules/{task_id}/runs response."""
runs: list[ScheduleRunInfo]
+195
View File
@@ -0,0 +1,195 @@
"""Lightweight HTTP server for the channel gateway.
Runs alongside the channel adapters (Discord, etc.) to receive notification
requests from the bridge. Exposes ``POST /v1/api/notify`` and ``GET /health``.
"""
from __future__ import annotations
import asyncio
import json
import socket
import uuid
from typing import TYPE_CHECKING, Any
from starlette.applications import Starlette
from starlette.responses import JSONResponse
from starlette.routing import Mount, Route
from turnstone.core.log import get_logger
if TYPE_CHECKING:
from starlette.requests import Request
from turnstone.channels._protocol import ChannelAdapter
from turnstone.core.storage._protocol import StorageBackend
log = get_logger(__name__)
async def _handle_health(request: Request) -> JSONResponse:
return JSONResponse({"status": "ok", "service": "channel"})
def _check_auth(request: Request) -> JSONResponse | None:
"""Validate the request's Authorization header. Returns an error response or None."""
auth_token: str = getattr(request.app.state, "auth_token", "")
jwt_secret: str = getattr(request.app.state, "jwt_secret", "")
if not auth_token and not jwt_secret:
log.warning("notify.auth_not_configured")
return JSONResponse({"error": "authentication not configured"}, status_code=401)
header = request.headers.get("Authorization", "")
if not header.startswith("Bearer "):
return JSONResponse({"error": "Unauthorized"}, status_code=401)
token = header[7:]
# Static token check
if auth_token:
import hmac
if hmac.compare_digest(token, auth_token):
return None
# JWT check
if jwt_secret and "." in token:
from turnstone.core.auth import JWT_AUD_CHANNEL, validate_jwt
result = validate_jwt(token, jwt_secret, audience=JWT_AUD_CHANNEL)
if result is not None:
return None
return JSONResponse({"error": "Unauthorized"}, status_code=401)
async def _handle_notify(request: Request) -> JSONResponse:
"""Deliver a notification to one or more channel adapters."""
auth_err = _check_auth(request)
if auth_err is not None:
return auth_err
adapters: dict[str, ChannelAdapter] = request.app.state.adapters
storage: StorageBackend = request.app.state.storage
try:
body: dict[str, Any] = await request.json()
except (json.JSONDecodeError, ValueError):
return JSONResponse({"error": "invalid JSON"}, status_code=400)
target = body.get("target")
message = body.get("message", "").strip() if isinstance(body.get("message"), str) else ""
title = body.get("title", "").strip() if isinstance(body.get("title"), str) else ""
if not target or not message:
return JSONResponse({"error": "target and message are required"}, status_code=400)
content = f"**{title}**\n{message}" if title else message
# Resolve targets
targets: list[tuple[str, str]] = []
if "username" in target:
user = await asyncio.to_thread(storage.get_user_by_username, target["username"])
if user is None:
log.warning("notify.user_not_found", username=target["username"])
return JSONResponse(
{"error": "target not found or has no linked channels"},
status_code=404,
)
links = await asyncio.to_thread(storage.list_channel_users_by_user, user["user_id"])
for link in links:
targets.append((link["channel_type"], link["channel_user_id"]))
if not targets:
log.warning("notify.user_no_linked_channels", username=target["username"])
return JSONResponse(
{"error": "target not found or has no linked channels"},
status_code=404,
)
elif "channel_type" in target and "channel_id" in target:
targets.append((target["channel_type"], target["channel_id"]))
else:
return JSONResponse(
{"error": "target must have username or channel_type+channel_id"},
status_code=400,
)
results: list[dict[str, str]] = []
for channel_type, channel_id in targets:
adapter = adapters.get(channel_type)
if adapter is None:
results.append(
{
"channel_type": channel_type,
"channel_id": channel_id,
"status": "no_adapter",
}
)
log.warning(
"notify.no_adapter",
channel_type=channel_type,
channel_id=channel_id,
)
continue
try:
msg_id = await adapter.send(channel_id, content)
results.append(
{
"channel_type": channel_type,
"channel_id": channel_id,
"status": "sent",
"message_id": msg_id,
}
)
log.info(
"notify.delivered",
channel_type=channel_type,
channel_id=channel_id,
message_id=msg_id,
)
except Exception:
log.exception(
"notify.delivery_failed",
channel_type=channel_type,
channel_id=channel_id,
)
results.append(
{
"channel_type": channel_type,
"channel_id": channel_id,
"status": "failed",
}
)
return JSONResponse({"results": results})
def create_channel_app(
adapters: dict[str, ChannelAdapter],
storage: StorageBackend,
*,
auth_token: str = "",
jwt_secret: str = "",
) -> Starlette:
"""Create the channel gateway HTTP application."""
app = Starlette(
routes=[
Route("/health", _handle_health),
Mount(
"/v1",
routes=[
Route("/api/notify", _handle_notify, methods=["POST"]),
],
),
],
)
app.state.adapters = adapters
app.state.storage = storage
app.state.auth_token = auth_token
app.state.jwt_secret = jwt_secret
return app
def _get_service_id() -> str:
"""Generate a unique service ID from hostname + random suffix."""
return f"channel-{socket.gethostname()}-{uuid.uuid4().hex[:8]}"
+106 -5
View File
@@ -1,8 +1,8 @@
"""Unified channel gateway entry point.
Launches one or more channel adapters (Discord, Slack, etc.) connected to
the turnstone cluster via Redis MQ. Currently supports Discord; future
adapters will be added as additional ``--*-token`` flags.
the turnstone cluster via Redis MQ. An HTTP server runs alongside for
inbound notification delivery from the server.
Run as: ``turnstone-channel --discord-token $TURNSTONE_DISCORD_TOKEN``
"""
@@ -10,6 +10,7 @@ Run as: ``turnstone-channel --discord-token $TURNSTONE_DISCORD_TOKEN``
from __future__ import annotations
import os
import socket
import sys
@@ -44,6 +45,26 @@ def main() -> None:
help="Comma-separated list of allowed Discord channel IDs (default: all)",
)
# -- HTTP server ---------------------------------------------------------
parser.add_argument(
"--http-host",
default="127.0.0.1",
help="HTTP server bind address (default: %(default)s)",
)
parser.add_argument(
"--http-port",
type=int,
default=int(os.environ.get("TURNSTONE_CHANNEL_PORT", "8091")),
help="HTTP server port (default: $TURNSTONE_CHANNEL_PORT or 8091)",
)
# -- Auth ----------------------------------------------------------------
parser.add_argument(
"--auth-token",
default=os.environ.get("TURNSTONE_CHANNEL_AUTH_TOKEN", ""),
help="Static auth token for /v1/api/notify (default: $TURNSTONE_CHANNEL_AUTH_TOKEN)",
)
# -- Workstream defaults -------------------------------------------------
parser.add_argument(
"--model",
@@ -85,6 +106,10 @@ def main() -> None:
path=db_path,
)
# -- Auth config ---------------------------------------------------------
auth_token = args.auth_token
jwt_secret = os.environ.get("TURNSTONE_JWT_SECRET", "").strip()
# -- Broker --------------------------------------------------------------
from turnstone.mq.broker import async_broker_from_args
@@ -106,6 +131,9 @@ def main() -> None:
# -- Run -----------------------------------------------------------------
if args.discord_token:
import asyncio
from turnstone.channels._http import _get_service_id, create_channel_app
from turnstone.channels.discord.bot import TurnstoneBot
from turnstone.channels.discord.config import DiscordConfig
from turnstone.core.storage._registry import get_storage
@@ -128,9 +156,82 @@ def main() -> None:
allowed_channels=allowed_channels,
)
bot = TurnstoneBot(config, broker, get_storage())
log.info("channel.starting", adapter="discord", guild_id=config.guild_id)
bot.run()
storage = get_storage()
bot = TurnstoneBot(config, broker, storage)
adapters = {"discord": bot}
# Create HTTP app for notification delivery
channel_app = create_channel_app(
adapters, # type: ignore[arg-type]
storage,
auth_token=auth_token,
jwt_secret=jwt_secret,
)
log.info(
"channel.starting",
adapter="discord",
guild_id=config.guild_id,
http_port=args.http_port,
)
async def _run_all() -> None:
"""Run Discord bot + HTTP server + service heartbeat concurrently."""
import uvicorn
service_id = _get_service_id()
# Resolve advertise URL — env override for Docker/K8s,
# otherwise derive from bind address.
advertise_url = os.environ.get("TURNSTONE_CHANNEL_ADVERTISE_URL", "").strip()
if not advertise_url:
if args.http_host in ("0.0.0.0", "::"):
advertise_host = socket.gethostname()
else:
advertise_host = args.http_host
advertise_url = f"http://{advertise_host}:{args.http_port}"
service_url = advertise_url
# Register in service registry
storage.register_service("channel", service_id, service_url)
log.info(
"channel.service_registered",
service_id=service_id,
url=service_url,
)
async def _heartbeat_loop() -> None:
"""Periodically update service heartbeat."""
while True:
await asyncio.sleep(30)
try:
await asyncio.to_thread(storage.heartbeat_service, "channel", service_id)
except Exception:
log.exception("channel.heartbeat_failed")
uv_config = uvicorn.Config(
channel_app,
host=args.http_host,
port=args.http_port,
log_level="warning",
)
server = uvicorn.Server(uv_config)
heartbeat_task = asyncio.create_task(_heartbeat_loop())
try:
await asyncio.gather(
bot.start(),
server.serve(),
)
finally:
heartbeat_task.cancel()
await asyncio.to_thread(storage.deregister_service, "channel", service_id)
log.info("channel.service_deregistered", service_id=service_id)
import contextlib
with contextlib.suppress(KeyboardInterrupt):
asyncio.run(_run_all())
if __name__ == "__main__":
+26
View File
@@ -348,6 +348,32 @@ class TurnstoneBot:
"""Start the bot (async). Use this for multi-adapter ``asyncio.gather``."""
await self._bot.start(self.config.bot_token, reconnect=True)
async def send(self, channel_id: str, content: str) -> str:
"""Send a message to a Discord channel or user DM.
Implements the :class:`ChannelAdapter` protocol. Tries the ID as a
channel first; if not found, attempts a user DM. Long messages are
chunked via :func:`chunk_message`.
"""
import discord
int_id = int(channel_id)
target: discord.abc.Messageable | None = self._bot.get_channel(int_id) # type: ignore[assignment]
if target is None:
try:
user = await self._bot.fetch_user(int_id)
target = await user.create_dm()
except discord.NotFound as exc:
raise ValueError(f"Discord channel/user {channel_id} not found") from exc
content = discord.utils.escape_mentions(content)
chunks = chunk_message(content, self.config.max_message_length)
msg: discord.Message | None = None
for chunk in chunks:
msg = await target.send(chunk) # type: ignore[union-attr]
return str(msg.id) if msg else ""
async def stop(self) -> None:
"""Disconnect the bot and clean up subscriptions."""
for ws_id in list(self._subscribed_ws):
+259
View File
@@ -0,0 +1,259 @@
"""Background task scheduler for timed workstream dispatch.
Runs as a daemon thread inside the console process. Checks for due tasks
every ``check_interval`` seconds and dispatches them as
``CreateWorkstreamMessage`` via the MQ broker.
Uses Redis ``SET NX EX`` for distributed locking in multi-console deployments.
"""
from __future__ import annotations
import threading
import uuid
from datetime import UTC, datetime
from typing import TYPE_CHECKING, Any
import structlog
if TYPE_CHECKING:
from turnstone.console.collector import ClusterCollector
from turnstone.core.storage._protocol import StorageBackend
from turnstone.mq.broker import RedisBroker
log = structlog.get_logger(__name__)
def _pick_best_node(collector: ClusterCollector) -> str:
"""Select the reachable node with the most available capacity."""
nodes, _ = collector.get_nodes(sort_by="activity", limit=1000, offset=0)
best_id = ""
best_headroom = -1
for n in nodes:
if not n.get("reachable", False):
continue
headroom = n.get("max_ws", 10) - n.get("ws_total", 0)
if headroom > best_headroom:
best_headroom = headroom
best_id = n["node_id"]
return best_id
class TaskScheduler:
"""Background scheduler for dispatching timed workstreams."""
def __init__(
self,
broker: RedisBroker,
collector: ClusterCollector,
storage: StorageBackend,
prefix: str = "turnstone",
check_interval: float = 15.0,
lock_ttl: int = 60,
max_fan_out: int = 20,
) -> None:
self._broker = broker
self._collector = collector
self._storage = storage
self._prefix = prefix
self._check_interval = check_interval
self._lock_ttl = lock_ttl
self._max_fan_out = max_fan_out
self._stop_event = threading.Event()
self._thread: threading.Thread | None = None
self._tick_count = 0
self._prune_every = 240 # ~1 hour at 15s intervals
def start(self) -> None:
"""Start the scheduler daemon thread."""
self._stop_event.clear()
self._thread = threading.Thread(target=self._loop, daemon=True, name="scheduler")
self._thread.start()
log.info("scheduler.started", check_interval=self._check_interval)
def stop(self) -> None:
"""Stop the scheduler and wait for the thread to finish."""
self._stop_event.set()
if self._thread is not None:
self._thread.join(timeout=5)
log.info("scheduler.stopped")
def _loop(self) -> None:
"""Main scheduler loop — tick then sleep."""
while not self._stop_event.is_set():
try:
self._tick()
except Exception:
log.exception("scheduler.tick_error")
self._stop_event.wait(self._check_interval)
# Lua script for safe lock release — only delete if we still own the lock
_UNLOCK_SCRIPT = "if redis.call('get',KEYS[1])==ARGV[1] then return redis.call('del',KEYS[1]) else return 0 end"
def _tick(self) -> None:
"""Single scheduler iteration: acquire lock, query due tasks, dispatch."""
now = datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%S")
# Distributed lock with unique owner — prevents releasing another instance's lock
lock_key = f"{self._prefix}:scheduler:lock"
lock_value = uuid.uuid4().hex
acquired = self._broker._redis.set(lock_key, lock_value, nx=True, ex=self._lock_ttl)
if not acquired:
return
try:
due_tasks = self._storage.list_due_tasks(now)
for task in due_tasks:
self._dispatch_task(task, now)
# Periodic run history pruning (~once per hour)
self._tick_count += 1
if self._tick_count % self._prune_every == 0:
pruned = self._storage.prune_task_runs(retention_days=90)
if pruned:
log.info("scheduler.pruned_runs", count=pruned)
finally:
# Only release our own lock (safe even if TTL expired and another took it)
self._broker._redis.eval( # type: ignore[no-untyped-call]
self._UNLOCK_SCRIPT, 1, lock_key, lock_value
)
def _dispatch_task(self, task: dict[str, Any], now: str) -> None:
"""Dispatch a single task as one or more CreateWorkstreamMessages."""
target_mode = task["target_mode"]
task_id = task["task_id"]
dispatched = False
if target_mode == "all":
nodes, _ = self._collector.get_nodes(sort_by="activity", limit=1000, offset=0)
fan_count = 0
for n in nodes:
if n.get("reachable", False):
if fan_count >= self._max_fan_out:
log.warning(
"scheduler.fan_out_capped",
task_id=task_id,
max_fan_out=self._max_fan_out,
)
break
self._dispatch_to_node(task, n["node_id"], now)
fan_count += 1
dispatched = True
if not dispatched:
self._record_failure(task, now, "No reachable nodes for fan-out")
elif target_mode == "pool":
self._dispatch_to_pool(task, now)
dispatched = True
elif target_mode == "auto":
node_id = _pick_best_node(self._collector)
if node_id:
self._dispatch_to_node(task, node_id, now)
dispatched = True
else:
self._record_failure(task, now, "No reachable nodes")
else:
# Specific node_id
self._dispatch_to_node(task, target_mode, now)
dispatched = True
if not dispatched:
return # Don't advance schedule on failure
# Update last_run and compute next_run
next_run = self._compute_next_run(task)
if task["schedule_type"] == "at":
self._storage.update_scheduled_task(task_id, last_run=now, next_run="", enabled=False)
else:
self._storage.update_scheduled_task(task_id, last_run=now, next_run=next_run)
log_kw: dict[str, Any] = {
"task_id": task_id,
"target_mode": target_mode,
"schedule_type": task["schedule_type"],
"created_by": task.get("created_by", ""),
}
if task.get("auto_approve", 0):
log_kw["auto_approve"] = True
log_kw["auto_approve_tools"] = task.get("auto_approve_tools", "")
log.warning("scheduler.task_dispatched_auto_approve", **log_kw)
else:
log.info("scheduler.task_dispatched", **log_kw)
@staticmethod
def _parse_tools(task: dict[str, Any]) -> list[str]:
raw = task.get("auto_approve_tools", "")
return [t.strip() for t in raw.split(",") if t.strip()]
def _dispatch_to_node(self, task: dict[str, Any], node_id: str, now: str) -> None:
"""Send a CreateWorkstreamMessage to a specific node."""
from turnstone.mq.protocol import CreateWorkstreamMessage
msg = CreateWorkstreamMessage(
name=task["name"],
model=task.get("model", ""),
target_node=node_id,
initial_message=task["initial_message"],
auto_approve=bool(task.get("auto_approve", 0)),
auto_approve_tools=self._parse_tools(task),
user_id=task.get("created_by", ""),
)
self._broker.push_inbound(msg.to_json(), node_id=node_id)
self._storage.record_task_run(
run_id=uuid.uuid4().hex,
task_id=task["task_id"],
node_id=node_id,
ws_id="",
correlation_id=msg.correlation_id,
started=now,
status="dispatched",
error="",
)
def _dispatch_to_pool(self, task: dict[str, Any], now: str) -> None:
"""Send a CreateWorkstreamMessage to the shared pool queue."""
from turnstone.mq.protocol import CreateWorkstreamMessage
msg = CreateWorkstreamMessage(
name=task["name"],
model=task.get("model", ""),
initial_message=task["initial_message"],
auto_approve=bool(task.get("auto_approve", 0)),
auto_approve_tools=self._parse_tools(task),
user_id=task.get("created_by", ""),
)
self._broker.push_inbound(msg.to_json())
self._storage.record_task_run(
run_id=uuid.uuid4().hex,
task_id=task["task_id"],
node_id="pool",
ws_id="",
correlation_id=msg.correlation_id,
started=now,
status="dispatched",
error="",
)
def _record_failure(self, task: dict[str, Any], now: str, error: str) -> None:
"""Record a failed dispatch attempt."""
self._storage.record_task_run(
run_id=uuid.uuid4().hex,
task_id=task["task_id"],
node_id="",
ws_id="",
correlation_id="",
started=now,
status="failed",
error=error,
)
log.warning("scheduler.dispatch_failed", task_id=task["task_id"], error=error)
@staticmethod
def _compute_next_run(task: dict[str, Any]) -> str:
"""Compute the next run time. Returns empty string for one-shot tasks."""
from turnstone.console.server import _compute_next_run
return _compute_next_run(
task["schedule_type"], task.get("cron_expr", ""), task.get("at_time", "")
)
+302 -1
View File
@@ -620,8 +620,14 @@ async def _lifespan(app: Starlette) -> AsyncGenerator[None, None]:
timeout=httpx.Timeout(connect=5, read=30, write=5, pool=5),
headers=headers,
)
# Start scheduler if configured
scheduler = getattr(app.state, "scheduler", None)
if scheduler is not None:
scheduler.start()
yield
# Shutdown
if scheduler is not None:
scheduler.stop()
await app.state.proxy_sse_client.aclose()
await app.state.proxy_client.aclose()
app.state.collector.stop()
@@ -878,6 +884,277 @@ async def admin_delete_channel(request: Request) -> JSONResponse:
return JSONResponse({"error": "Channel link not found"}, status_code=404)
# ---------------------------------------------------------------------------
# Admin API endpoints — scheduled tasks
# ---------------------------------------------------------------------------
def _normalize_task_dict(task: dict[str, Any]) -> dict[str, Any]:
"""Convert DB row ints/csv to JSON-friendly bools/lists."""
tools_str = task.get("auto_approve_tools", "")
task["auto_approve_tools"] = [s.strip() for s in tools_str.split(",") if s.strip()]
task["auto_approve"] = bool(task.get("auto_approve", 0))
task["enabled"] = bool(task.get("enabled", 1))
return task
def _compute_next_run(schedule_type: str, cron_expr: str, at_time: str) -> str:
"""Compute the next run time for a schedule. Empty string if invalid."""
if schedule_type == "at":
return at_time
if schedule_type == "cron" and cron_expr:
from datetime import UTC, datetime
from croniter import croniter
cron = croniter(cron_expr, datetime.now(UTC))
next_dt = cron.get_next(datetime)
return str(next_dt.strftime("%Y-%m-%dT%H:%M:%S"))
return ""
def _validate_schedule_fields(schedule_type: str, cron_expr: str, at_time: str) -> str | None:
"""Validate schedule type/expression. Returns error string or None."""
if schedule_type not in ("cron", "at"):
return "schedule_type must be 'cron' or 'at'"
if schedule_type == "cron":
if not cron_expr:
return "cron_expr is required when schedule_type is 'cron'"
from croniter import croniter
if not croniter.is_valid(cron_expr):
return f"Invalid cron expression: {cron_expr}"
if schedule_type == "at":
if not at_time:
return "at_time is required when schedule_type is 'at'"
from datetime import UTC, datetime
try:
dt = datetime.fromisoformat(at_time)
if dt.tzinfo is None:
return (
"at_time must include a timezone offset (e.g. 2024-01-01T12:00:00Z or +00:00)"
)
if dt <= datetime.now(UTC):
return "at_time must be in the future"
except ValueError:
return "at_time must be a valid ISO8601 timestamp with timezone"
return None
async def admin_list_schedules(request: Request) -> JSONResponse:
"""GET /v1/api/admin/schedules — list all scheduled tasks."""
from turnstone.core.web_helpers import require_storage_or_503
storage, err = require_storage_or_503(request)
if err:
return err
tasks = storage.list_scheduled_tasks()
for t in tasks:
_normalize_task_dict(t)
return JSONResponse({"schedules": tasks})
async def admin_create_schedule(request: Request) -> JSONResponse:
"""POST /v1/api/admin/schedules — create a scheduled task."""
import uuid
from turnstone.core.web_helpers import read_json_or_400, require_storage_or_503
storage, err = require_storage_or_503(request)
if err:
return err
body = await read_json_or_400(request)
if isinstance(body, JSONResponse):
return body
name = str(body.get("name", "")).strip()[:256]
description = str(body.get("description", "")).strip()[:1024]
schedule_type = str(body.get("schedule_type", "")).strip()
cron_expr = str(body.get("cron_expr", "")).strip()[:256]
at_time = str(body.get("at_time", "")).strip()[:64]
target_mode = str(body.get("target_mode", "auto")).strip()[:256]
model = str(body.get("model", "")).strip()[:128]
initial_message = str(body.get("initial_message", "")).strip()[:4096]
auto_approve = bool(body.get("auto_approve", False))
raw_tools = body.get("auto_approve_tools", [])
auto_approve_tools = raw_tools if isinstance(raw_tools, list) else []
enabled = bool(body.get("enabled", True))
if not name:
return JSONResponse({"error": "name is required"}, status_code=400)
if not initial_message:
return JSONResponse({"error": "initial_message is required"}, status_code=400)
validation_err = _validate_schedule_fields(schedule_type, cron_expr, at_time)
if validation_err:
return JSONResponse({"error": validation_err}, status_code=400)
if not target_mode:
return JSONResponse({"error": "target_mode is required"}, status_code=400)
# Cap total schedule count to prevent unbounded growth
max_schedules = 200
existing = storage.list_scheduled_tasks()
if len(existing) >= max_schedules:
return JSONResponse(
{"error": f"Maximum of {max_schedules} schedules reached"}, status_code=409
)
next_run = _compute_next_run(schedule_type, cron_expr, at_time)
task_id = uuid.uuid4().hex
created_by = getattr(getattr(request, "state", None), "user_id", "")
storage.create_scheduled_task(
task_id=task_id,
name=name,
description=description,
schedule_type=schedule_type,
cron_expr=cron_expr,
at_time=at_time,
target_mode=target_mode,
model=model,
initial_message=initial_message,
auto_approve=auto_approve,
auto_approve_tools=auto_approve_tools,
created_by=created_by,
next_run=next_run if enabled else "",
)
if not enabled:
# Storage backends default enabled=1 on create; persist user's choice
storage.update_scheduled_task(task_id, enabled=False)
task = storage.get_scheduled_task(task_id)
if task:
_normalize_task_dict(task)
return JSONResponse(task)
async def admin_get_schedule(request: Request) -> JSONResponse:
"""GET /v1/api/admin/schedules/{task_id} — get single task."""
from turnstone.core.web_helpers import require_storage_or_503
storage, err = require_storage_or_503(request)
if err:
return err
task_id = request.path_params["task_id"]
task = storage.get_scheduled_task(task_id)
if task is None:
return JSONResponse({"error": "Schedule not found"}, status_code=404)
_normalize_task_dict(task)
return JSONResponse(task)
async def admin_update_schedule(request: Request) -> JSONResponse:
"""PUT /v1/api/admin/schedules/{task_id} — partial update."""
from turnstone.core.web_helpers import read_json_or_400, require_storage_or_503
storage, err = require_storage_or_503(request)
if err:
return err
task_id = request.path_params["task_id"]
existing = storage.get_scheduled_task(task_id)
if existing is None:
return JSONResponse({"error": "Schedule not found"}, status_code=404)
body = await read_json_or_400(request)
if isinstance(body, JSONResponse):
return body
updates: dict[str, Any] = {}
if "name" in body:
updates["name"] = str(body["name"]).strip()[:256]
if "description" in body:
updates["description"] = str(body["description"]).strip()[:1024]
if "schedule_type" in body:
updates["schedule_type"] = str(body["schedule_type"]).strip()
if "cron_expr" in body:
updates["cron_expr"] = str(body["cron_expr"]).strip()[:256]
if "at_time" in body:
updates["at_time"] = str(body["at_time"]).strip()[:64]
if "target_mode" in body:
updates["target_mode"] = str(body["target_mode"]).strip()[:256]
if "model" in body:
updates["model"] = str(body["model"]).strip()[:128]
if "initial_message" in body:
updates["initial_message"] = str(body["initial_message"]).strip()[:4096]
if "auto_approve" in body:
updates["auto_approve"] = bool(body["auto_approve"])
if "auto_approve_tools" in body:
raw = body["auto_approve_tools"]
updates["auto_approve_tools"] = raw if isinstance(raw, list) else []
if "enabled" in body:
updates["enabled"] = bool(body["enabled"])
# Validate schedule fields if changed
stype = updates.get("schedule_type", existing["schedule_type"])
cexpr = updates.get("cron_expr", existing["cron_expr"])
atime = updates.get("at_time", existing["at_time"])
schedule_fields_changed = (
"schedule_type" in updates or "cron_expr" in updates or "at_time" in updates
)
if schedule_fields_changed:
validation_err = _validate_schedule_fields(stype, cexpr, atime)
if validation_err:
return JSONResponse({"error": validation_err}, status_code=400)
# Recompute next_run if schedule changed or enabled toggled
if schedule_fields_changed or "enabled" in updates:
enabled = updates.get("enabled", bool(existing.get("enabled", 1)))
if enabled:
# Re-validate at_time when re-enabling a one-shot task
if stype == "at" and not schedule_fields_changed:
validation_err = _validate_schedule_fields(stype, cexpr, atime)
if validation_err:
return JSONResponse({"error": validation_err}, status_code=400)
updates["next_run"] = _compute_next_run(stype, cexpr, atime)
else:
updates["next_run"] = ""
storage.update_scheduled_task(task_id, **updates)
task = storage.get_scheduled_task(task_id)
if task:
_normalize_task_dict(task)
return JSONResponse(task)
async def admin_delete_schedule(request: Request) -> JSONResponse:
"""DELETE /v1/api/admin/schedules/{task_id} — delete task + runs."""
from turnstone.core.web_helpers import require_storage_or_503
storage, err = require_storage_or_503(request)
if err:
return err
task_id = request.path_params["task_id"]
if storage.delete_scheduled_task(task_id):
return JSONResponse({"status": "ok"})
return JSONResponse({"error": "Schedule not found"}, status_code=404)
async def admin_list_schedule_runs(request: Request) -> JSONResponse:
"""GET /v1/api/admin/schedules/{task_id}/runs — run history."""
from turnstone.core.web_helpers import require_storage_or_503
storage, err = require_storage_or_503(request)
if err:
return err
task_id = request.path_params["task_id"]
# Verify task exists
if storage.get_scheduled_task(task_id) is None:
return JSONResponse({"error": "Schedule not found"}, status_code=404)
try:
limit = min(int(request.query_params.get("limit", "50")), 200)
except (ValueError, TypeError):
limit = 50
runs = storage.list_task_runs(task_id, limit=limit)
return JSONResponse({"runs": runs})
# ---------------------------------------------------------------------------
# App factory
# ---------------------------------------------------------------------------
@@ -937,6 +1214,16 @@ def create_app(
admin_delete_channel,
methods=["DELETE"],
),
Route("/api/admin/schedules", admin_list_schedules),
Route("/api/admin/schedules", admin_create_schedule, methods=["POST"]),
Route("/api/admin/schedules/{task_id}", admin_get_schedule),
Route("/api/admin/schedules/{task_id}", admin_update_schedule, methods=["PUT"]),
Route(
"/api/admin/schedules/{task_id}",
admin_delete_schedule,
methods=["DELETE"],
),
Route("/api/admin/schedules/{task_id}/runs", admin_list_schedule_runs),
],
),
Route("/health", health),
@@ -966,6 +1253,20 @@ def create_app(
from turnstone.core.auth import LoginRateLimiter
app.state.login_limiter = LoginRateLimiter()
# Scheduler — start background thread if storage is available
if auth_storage is not None:
from turnstone.console.scheduler import TaskScheduler
scheduler = TaskScheduler(
broker=broker,
collector=collector,
storage=auth_storage,
)
app.state.scheduler = scheduler
else:
app.state.scheduler = None
return app
@@ -1100,7 +1401,7 @@ def main() -> None:
proxy_token_mgr = ServiceTokenManager(
user_id="console-proxy",
scopes=frozenset({"write"}),
scopes=frozenset({"read", "write", "approve"}),
source="console",
secret=jwt_secret,
audience=JWT_AUD_SERVER,
+538 -2
View File
@@ -46,10 +46,13 @@ function switchAdminTab(tab) {
tab === "tokens" ? "" : "none";
document.getElementById("admin-channels").style.display =
tab === "channels" ? "" : "none";
document.getElementById("admin-schedules").style.display =
tab === "schedules" ? "" : "none";
if (tab === "users") loadAdminUsers();
if (tab === "tokens") _populateTokenUserSelect();
if (tab === "channels") _populateChannelUserSelect();
if (tab === "schedules") loadAdminSchedules();
}
// ---------------------------------------------------------------------------
@@ -373,6 +376,517 @@ function confirmUnlinkChannel(channelType, channelUserId) {
);
}
// ---------------------------------------------------------------------------
// Schedules
// ---------------------------------------------------------------------------
var _csTrapHandler = null;
var _esTrapHandler = null;
var _srTrapHandler = null;
var _editScheduleTriggerEl = null;
var _runsScheduleTriggerEl = null;
function loadAdminSchedules() {
authFetch("/v1/api/admin/schedules")
.then(function (r) {
if (!r.ok) throw new Error("Failed to load schedules");
return r.json();
})
.then(function (data) {
_renderSchedules(data.schedules || []);
})
.catch(function () {
document.getElementById("admin-schedules-table").innerHTML =
'<div class="dashboard-empty">Failed to load schedules</div>';
});
}
function _renderSchedules(schedules) {
var container = document.getElementById("admin-schedules-table");
if (!schedules.length) {
container.innerHTML =
'<div class="dashboard-empty">No scheduled tasks. Create one to get started.</div>';
return;
}
var html = "";
for (var i = 0; i < schedules.length; i++) {
var s = schedules[i];
var typeLabel = s.schedule_type === "cron" ? "cron" : "at";
var typeCls = s.schedule_type === "cron" ? "scope-write" : "scope-approve";
var schedule =
s.schedule_type === "cron"
? s.cron_expr
: (s.at_time || "").slice(0, 16).replace("T", " ");
var target = s.target_mode;
var nextRun = s.next_run
? escapeHtml(s.next_run).slice(0, 16).replace("T", " ")
: "\u2014";
var enabled = s.enabled;
var statusCls = enabled ? "sched-active" : "sched-disabled";
var statusLabel = enabled ? "active" : "disabled";
var statusDot = enabled ? "\u25cf " : "\u25cb ";
if (s.schedule_type === "at" && !enabled && s.last_run) {
statusCls = "sched-expired";
statusLabel = "completed";
statusDot = "\u25c9 ";
}
html +=
'<div class="admin-row" role="listitem">' +
'<span class="admin-col admin-col-sname">' +
escapeHtml(s.name) +
"</span>" +
'<span class="admin-col admin-col-stype"><span class="scope-badge ' +
typeCls +
'">' +
typeLabel +
"</span></span>" +
'<span class="admin-col admin-col-sschedule"><code>' +
escapeHtml(schedule) +
"</code></span>" +
'<span class="admin-col admin-col-starget">' +
escapeHtml(target) +
"</span>" +
'<span class="admin-col admin-col-snext">' +
nextRun +
"</span>" +
'<span class="admin-col admin-col-sstatus"><span class="' +
statusCls +
'">' +
statusDot +
statusLabel +
"</span></span>" +
'<span class="admin-col admin-col-actions">' +
'<button class="admin-btn-action" data-edit-sched="' +
escapeHtml(s.task_id) +
'" title="Edit">edit</button>' +
'<button class="admin-btn-action" data-runs-sched="' +
escapeHtml(s.task_id) +
'" title="Run history">runs</button>' +
'<button class="admin-btn-action" data-toggle-sched="' +
escapeHtml(s.task_id) +
'" data-enabled="' +
(enabled ? "1" : "0") +
'" title="' +
(enabled ? "Disable" : "Enable") +
'">' +
(enabled ? "disable" : "enable") +
"</button>" +
'<button class="admin-btn-danger" data-delete-sched="' +
escapeHtml(s.task_id) +
'" data-sname="' +
escapeHtml(s.name) +
'" title="Delete">delete</button>' +
"</span></div>";
}
container.innerHTML = html;
// Bind buttons
var editBtns = container.querySelectorAll("[data-edit-sched]");
for (var j = 0; j < editBtns.length; j++) {
editBtns[j].addEventListener("click", function () {
showEditScheduleModal(this.getAttribute("data-edit-sched"));
});
}
var runsBtns = container.querySelectorAll("[data-runs-sched]");
for (var k = 0; k < runsBtns.length; k++) {
runsBtns[k].addEventListener("click", function () {
showScheduleRuns(this.getAttribute("data-runs-sched"));
});
}
var toggleBtns = container.querySelectorAll("[data-toggle-sched]");
for (var m = 0; m < toggleBtns.length; m++) {
toggleBtns[m].addEventListener("click", function () {
toggleSchedule(
this.getAttribute("data-toggle-sched"),
this.getAttribute("data-enabled") === "1",
);
});
}
var delBtns = container.querySelectorAll("[data-delete-sched]");
for (var n = 0; n < delBtns.length; n++) {
delBtns[n].addEventListener("click", function () {
confirmDeleteSchedule(
this.getAttribute("data-delete-sched"),
this.getAttribute("data-sname"),
);
});
}
}
function toggleSchedule(taskId, currentlyEnabled) {
authFetch("/v1/api/admin/schedules/" + encodeURIComponent(taskId), {
method: "PUT",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({ enabled: !currentlyEnabled }),
})
.then(function (r) {
if (!r.ok) throw new Error("Toggle failed");
showToast(currentlyEnabled ? "Schedule disabled" : "Schedule enabled");
loadAdminSchedules();
})
.catch(function () {
showToast("Failed to toggle schedule");
});
}
function confirmDeleteSchedule(taskId, name) {
showConfirmModal(
"Delete Schedule",
"Delete schedule \u2018" +
name +
"\u2019 and its run history? This cannot be undone.",
"Delete",
function () {
authFetch("/v1/api/admin/schedules/" + encodeURIComponent(taskId), {
method: "DELETE",
})
.then(function (r) {
if (!r.ok) throw new Error("Delete failed");
showToast("Schedule deleted");
loadAdminSchedules();
})
.catch(function () {
showToast("Failed to delete schedule");
});
},
);
}
// --- Create Schedule Modal ---
function toggleScheduleTypeFields() {
var t = document.getElementById("cs-type").value;
document.getElementById("cs-cron-group").style.display =
t === "cron" ? "" : "none";
document.getElementById("cs-at-group").style.display =
t === "at" ? "" : "none";
if (t === "cron") document.getElementById("cs-cron").focus();
else document.getElementById("cs-at").focus();
}
function toggleScheduleNodeField() {
var v = document.getElementById("cs-target").value;
document.getElementById("cs-node-group").style.display =
v === "node" ? "" : "none";
if (v === "node") document.getElementById("cs-node").focus();
}
function showCreateScheduleModal() {
var overlay = document.getElementById("create-schedule-overlay");
overlay.style.display = "flex";
document.getElementById("create-schedule-error").style.display = "none";
document.getElementById("cs-name").value = "";
document.getElementById("cs-desc").value = "";
document.getElementById("cs-type").value = "cron";
document.getElementById("cs-cron").value = "";
document.getElementById("cs-at").value = "";
document.getElementById("cs-target").value = "auto";
document.getElementById("cs-node").value = "";
document.getElementById("cs-model").value = "";
document.getElementById("cs-message").value = "";
document.getElementById("cs-autoapprove").checked = false;
toggleScheduleTypeFields();
toggleScheduleNodeField();
document.getElementById("cs-submit").disabled = false;
document.getElementById("cs-submit").textContent = "Create";
_csTrapHandler = _installTrap(
"create-schedule-overlay",
"create-schedule-box",
);
setTimeout(function () {
document.getElementById("cs-name").focus();
}, 50);
}
function hideCreateScheduleModal() {
document.getElementById("create-schedule-overlay").style.display = "none";
_csTrapHandler = _removeTrap(_csTrapHandler);
var trigger = document.querySelector("#admin-schedules .admin-action-btn");
if (trigger) trigger.focus();
}
function submitCreateSchedule() {
var name = (document.getElementById("cs-name").value || "").trim();
var desc = (document.getElementById("cs-desc").value || "").trim();
var schedType = document.getElementById("cs-type").value;
var cronExpr = (document.getElementById("cs-cron").value || "").trim();
var atTime = document.getElementById("cs-at").value || "";
var targetMode = document.getElementById("cs-target").value;
var nodeId = (document.getElementById("cs-node").value || "").trim();
var model = (document.getElementById("cs-model").value || "").trim();
var message = (document.getElementById("cs-message").value || "").trim();
var autoApprove = document.getElementById("cs-autoapprove").checked;
var errEl = document.getElementById("create-schedule-error");
if (!name) return _showModalError(errEl, "Name is required");
if (!message) return _showModalError(errEl, "Initial message is required");
if (schedType === "cron" && !cronExpr)
return _showModalError(errEl, "Cron expression is required");
if (schedType === "at" && !atTime)
return _showModalError(errEl, "Run time is required");
// Normalize datetime-local to "YYYY-MM-DDTHH:MM:SS+00:00" (UTC)
if (schedType === "at" && atTime) {
if (atTime.length === 16) atTime += ":00";
else if (atTime.length > 19) atTime = atTime.slice(0, 19);
atTime += "+00:00";
}
if (targetMode === "node") targetMode = nodeId;
var btn = document.getElementById("cs-submit");
btn.disabled = true;
btn.textContent = "Creating\u2026";
authFetch("/v1/api/admin/schedules", {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({
name: name,
description: desc,
schedule_type: schedType,
cron_expr: cronExpr,
at_time: atTime,
target_mode: targetMode,
model: model,
initial_message: message,
auto_approve: autoApprove,
}),
})
.then(function (r) {
if (!r.ok)
return r.json().then(function (d) {
throw new Error(d.error || "Failed");
});
return r.json();
})
.then(function () {
hideCreateScheduleModal();
showToast("Schedule '" + name + "' created");
loadAdminSchedules();
})
.catch(function (err) {
btn.disabled = false;
btn.textContent = "Create";
_showModalError(errEl, err.message || "Failed to create schedule");
});
}
// --- Edit Schedule Modal ---
function toggleEditScheduleTypeFields() {
var t = document.getElementById("es-type").value;
document.getElementById("es-cron-group").style.display =
t === "cron" ? "" : "none";
document.getElementById("es-at-group").style.display =
t === "at" ? "" : "none";
if (t === "cron") document.getElementById("es-cron").focus();
else document.getElementById("es-at").focus();
}
function toggleEditScheduleNodeField() {
var v = document.getElementById("es-target").value;
document.getElementById("es-node-group").style.display =
v === "node" ? "" : "none";
if (v === "node") document.getElementById("es-node").focus();
}
function showEditScheduleModal(taskId) {
_editScheduleTriggerEl = document.activeElement;
authFetch("/v1/api/admin/schedules/" + encodeURIComponent(taskId))
.then(function (r) {
if (!r.ok) throw new Error("Not found");
return r.json();
})
.then(function (s) {
document.getElementById("es-id").value = s.task_id;
document.getElementById("es-name").value = s.name || "";
document.getElementById("es-desc").value = s.description || "";
document.getElementById("es-type").value = s.schedule_type;
document.getElementById("es-cron").value = s.cron_expr || "";
document.getElementById("es-at").value = (s.at_time || "").slice(0, 16);
var isSpecificNode =
s.target_mode &&
s.target_mode !== "auto" &&
s.target_mode !== "pool" &&
s.target_mode !== "all";
document.getElementById("es-target").value = isSpecificNode
? "node"
: s.target_mode;
document.getElementById("es-node").value = isSpecificNode
? s.target_mode
: "";
document.getElementById("es-model").value = s.model || "";
document.getElementById("es-message").value = s.initial_message || "";
document.getElementById("es-autoapprove").checked = !!s.auto_approve;
document.getElementById("es-enabled").checked = !!s.enabled;
toggleEditScheduleTypeFields();
toggleEditScheduleNodeField();
document.getElementById("edit-schedule-error").style.display = "none";
document.getElementById("es-submit").disabled = false;
document.getElementById("es-submit").textContent = "Save";
var overlay = document.getElementById("edit-schedule-overlay");
overlay.style.display = "flex";
_esTrapHandler = _installTrap(
"edit-schedule-overlay",
"edit-schedule-box",
);
setTimeout(function () {
document.getElementById("es-name").focus();
}, 50);
})
.catch(function () {
showToast("Failed to load schedule");
});
}
function hideEditScheduleModal() {
document.getElementById("edit-schedule-overlay").style.display = "none";
_esTrapHandler = _removeTrap(_esTrapHandler);
if (_editScheduleTriggerEl && _editScheduleTriggerEl.isConnected) {
_editScheduleTriggerEl.focus();
}
_editScheduleTriggerEl = null;
}
function submitEditSchedule() {
var taskId = document.getElementById("es-id").value;
var name = (document.getElementById("es-name").value || "").trim();
var message = (document.getElementById("es-message").value || "").trim();
var schedType = document.getElementById("es-type").value;
var cronExpr = (document.getElementById("es-cron").value || "").trim();
var targetMode = document.getElementById("es-target").value;
if (targetMode === "node")
targetMode = (document.getElementById("es-node").value || "").trim();
var atTime = document.getElementById("es-at").value || "";
if (atTime) {
if (atTime.length === 16) atTime += ":00";
else if (atTime.length > 19) atTime = atTime.slice(0, 19);
atTime += "+00:00";
}
var errEl = document.getElementById("edit-schedule-error");
if (!name) return _showModalError(errEl, "Name is required");
if (!message) return _showModalError(errEl, "Initial message is required");
if (schedType === "cron" && !cronExpr)
return _showModalError(errEl, "Cron expression is required");
if (schedType === "at" && !atTime)
return _showModalError(errEl, "Run time is required");
var btn = document.getElementById("es-submit");
btn.disabled = true;
btn.textContent = "Saving\u2026";
authFetch("/v1/api/admin/schedules/" + encodeURIComponent(taskId), {
method: "PUT",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({
name: (document.getElementById("es-name").value || "").trim(),
description: (document.getElementById("es-desc").value || "").trim(),
schedule_type: document.getElementById("es-type").value,
cron_expr: (document.getElementById("es-cron").value || "").trim(),
at_time: atTime,
target_mode: targetMode,
model: (document.getElementById("es-model").value || "").trim(),
initial_message: (
document.getElementById("es-message").value || ""
).trim(),
auto_approve: document.getElementById("es-autoapprove").checked,
enabled: document.getElementById("es-enabled").checked,
}),
})
.then(function (r) {
if (!r.ok)
return r.json().then(function (d) {
throw new Error(d.error || "Failed");
});
return r.json();
})
.then(function () {
hideEditScheduleModal();
showToast("Schedule updated");
loadAdminSchedules();
})
.catch(function (err) {
btn.disabled = false;
btn.textContent = "Save";
_showModalError(errEl, err.message || "Failed to update schedule");
});
}
// --- Schedule Runs Modal ---
function showScheduleRuns(taskId) {
_runsScheduleTriggerEl = document.activeElement;
authFetch(
"/v1/api/admin/schedules/" + encodeURIComponent(taskId) + "/runs?limit=50",
)
.then(function (r) {
if (!r.ok) throw new Error("Not found");
return r.json();
})
.then(function (data) {
var runs = data.runs || [];
var container = document.getElementById("schedule-runs-table");
if (!runs.length) {
container.innerHTML = '<div class="dashboard-empty">No runs yet</div>';
} else {
var html =
'<div class="admin-colheaders sched-runs-grid" aria-hidden="true">' +
'<span class="admin-col">STARTED</span>' +
'<span class="admin-col">NODE</span>' +
'<span class="admin-col">STATUS</span>' +
'<span class="admin-col">ERROR</span></div>';
for (var i = 0; i < runs.length; i++) {
var r = runs[i];
var statusCls =
r.status === "dispatched"
? "sched-active"
: r.status === "failed"
? "sched-expired"
: "";
html +=
'<div class="admin-row sched-runs-grid">' +
'<span class="admin-col">' +
escapeHtml(r.started || "")
.slice(0, 19)
.replace("T", " ") +
"</span>" +
'<span class="admin-col">' +
escapeHtml(r.node_id || "\u2014") +
"</span>" +
'<span class="admin-col"><span class="' +
statusCls +
'">' +
escapeHtml(r.status) +
"</span></span>" +
'<span class="admin-col">' +
escapeHtml(r.error || "\u2014") +
"</span></div>";
}
container.innerHTML = html;
}
var overlay = document.getElementById("schedule-runs-overlay");
overlay.style.display = "flex";
_srTrapHandler = _installTrap(
"schedule-runs-overlay",
"schedule-runs-box",
);
})
.catch(function () {
showToast("Failed to load run history");
});
}
function hideScheduleRunsModal() {
document.getElementById("schedule-runs-overlay").style.display = "none";
_srTrapHandler = _removeTrap(_srTrapHandler);
if (_runsScheduleTriggerEl && _runsScheduleTriggerEl.isConnected) {
_runsScheduleTriggerEl.focus();
}
_runsScheduleTriggerEl = null;
}
// ---------------------------------------------------------------------------
// Create Channel Link Modal
// ---------------------------------------------------------------------------
@@ -644,7 +1158,7 @@ function _modalFocusTrap(boxId) {
var box = document.getElementById(boxId);
if (!box) return;
var focusable = box.querySelectorAll(
"input:not([disabled]), select:not([disabled]), button:not([disabled])",
"input:not([disabled]):not([type='hidden']), select:not([disabled]), textarea:not([disabled]), button:not([disabled])",
);
var visible = [];
for (var i = 0; i < focusable.length; i++) {
@@ -678,6 +1192,10 @@ function _installTrap(overlayId, boxId, trapRef) {
else if (overlayId === "token-created-overlay") hideTokenCreatedModal();
else if (overlayId === "create-channel-overlay")
hideCreateChannelModal();
else if (overlayId === "create-schedule-overlay")
hideCreateScheduleModal();
else if (overlayId === "edit-schedule-overlay") hideEditScheduleModal();
else if (overlayId === "schedule-runs-overlay") hideScheduleRunsModal();
else if (overlayId === "confirm-overlay") hideConfirmModal();
}
};
@@ -721,6 +1239,24 @@ document.addEventListener("keydown", function (e) {
hideCreateChannelModal();
return;
}
var cso = document.getElementById("create-schedule-overlay");
if (cso && cso.style.display !== "none") {
e.preventDefault();
hideCreateScheduleModal();
return;
}
var eso = document.getElementById("edit-schedule-overlay");
if (eso && eso.style.display !== "none") {
e.preventDefault();
hideEditScheduleModal();
return;
}
var sro = document.getElementById("schedule-runs-overlay");
if (sro && sro.style.display !== "none") {
e.preventDefault();
hideScheduleRunsModal();
return;
}
var cf = document.getElementById("confirm-overlay");
if (cf && cf.style.display !== "none") {
e.preventDefault();
@@ -735,7 +1271,7 @@ document.addEventListener("keydown", function (e) {
if (!tablist) return;
tablist.addEventListener("keydown", function (e) {
if (e.key !== "ArrowLeft" && e.key !== "ArrowRight") return;
var tabOrder = ["users", "tokens", "channels"];
var tabOrder = ["users", "tokens", "channels", "schedules"];
var idx = tabOrder.indexOf(_adminTab);
if (e.key === "ArrowRight") idx = (idx + 1) % tabOrder.length;
else idx = (idx - 1 + tabOrder.length) % tabOrder.length;
+125
View File
@@ -80,6 +80,7 @@
<button id="tab-users" class="admin-tab active" data-tab="users" role="tab" aria-selected="true" aria-controls="admin-users" tabindex="0" onclick="switchAdminTab('users')">Users</button>
<button id="tab-tokens" class="admin-tab" data-tab="tokens" role="tab" aria-selected="false" aria-controls="admin-tokens" tabindex="-1" onclick="switchAdminTab('tokens')">Tokens</button>
<button id="tab-channels" class="admin-tab" data-tab="channels" role="tab" aria-selected="false" aria-controls="admin-channels" tabindex="-1" onclick="switchAdminTab('channels')">Channels</button>
<button id="tab-schedules" class="admin-tab" data-tab="schedules" role="tab" aria-selected="false" aria-controls="admin-schedules" tabindex="-1" onclick="switchAdminTab('schedules')">Schedules</button>
</div>
<!-- Users Tab -->
@@ -142,6 +143,26 @@
<div class="dashboard-empty">Select a user to view channel links</div>
</div>
</div>
<!-- Schedules Tab -->
<div id="admin-schedules" class="admin-panel" role="tabpanel" aria-labelledby="tab-schedules" style="display:none">
<div class="admin-toolbar">
<span class="section-header" style="margin:0">SCHEDULED TASKS</span>
<button class="admin-action-btn" onclick="showCreateScheduleModal()">+ New schedule</button>
</div>
<div class="admin-colheaders" aria-hidden="true">
<span class="admin-col admin-col-sname">NAME</span>
<span class="admin-col admin-col-stype">TYPE</span>
<span class="admin-col admin-col-sschedule">SCHEDULE</span>
<span class="admin-col admin-col-starget">TARGET</span>
<span class="admin-col admin-col-snext">NEXT RUN</span>
<span class="admin-col admin-col-sstatus">STATUS</span>
<span class="admin-col admin-col-actions">ACTIONS</span>
</div>
<div id="admin-schedules-table" role="list" aria-label="Scheduled tasks" aria-live="polite">
<div class="dashboard-empty">Loading schedules...</div>
</div>
</div>
</div>
</div>
@@ -284,6 +305,110 @@ window.TURNSTONE_KB_SHORTCUTS = [
</div>
</div>
<!-- Create Schedule Modal -->
<div id="create-schedule-overlay" style="display:none" role="dialog" aria-modal="true" aria-labelledby="create-schedule-title">
<div id="create-schedule-box" class="admin-modal admin-modal-wide">
<h2 id="create-schedule-title">New Schedule</h2>
<div id="create-schedule-error" role="alert" aria-live="assertive"></div>
<label for="cs-name">Name</label>
<input id="cs-name" type="text" placeholder="Daily health check" autocomplete="off">
<label for="cs-desc">Description <span class="label-hint">optional</span></label>
<input id="cs-desc" type="text" placeholder="" autocomplete="off">
<label for="cs-type">Schedule type</label>
<select id="cs-type" onchange="toggleScheduleTypeFields()">
<option value="cron">Cron (recurring)</option>
<option value="at">At (one-shot)</option>
</select>
<div id="cs-cron-group">
<label for="cs-cron">Cron expression</label>
<input id="cs-cron" type="text" placeholder="0 9 * * MON-FRI" autocomplete="off" spellcheck="false" aria-describedby="cs-cron-hint">
<span id="cs-cron-hint" class="label-hint" style="display:block;margin-top:3px">min hour day month weekday</span>
</div>
<div id="cs-at-group" style="display:none">
<label for="cs-at">Run at</label>
<input id="cs-at" type="datetime-local">
</div>
<label for="cs-target">Target</label>
<select id="cs-target" onchange="toggleScheduleNodeField()">
<option value="auto">Auto (best available)</option>
<option value="pool">Pool (any bridge)</option>
<option value="all">All nodes</option>
<option value="node">Specific node...</option>
</select>
<div id="cs-node-group" style="display:none">
<label for="cs-node">Node ID</label>
<input id="cs-node" type="text" placeholder="node-001" autocomplete="off" spellcheck="false">
</div>
<label for="cs-model">Model <span class="label-hint">optional</span></label>
<input id="cs-model" type="text" placeholder="Default model" autocomplete="off">
<label for="cs-message">Initial message</label>
<textarea id="cs-message" rows="3" placeholder="What should the workstream do?"></textarea>
<label class="admin-checkbox"><input id="cs-autoapprove" type="checkbox"> Auto-approve tool calls</label>
<div class="modal-buttons">
<button class="modal-cancel" onclick="hideCreateScheduleModal()">Cancel</button>
<button id="cs-submit" class="modal-submit" onclick="submitCreateSchedule()">Create</button>
</div>
</div>
</div>
<!-- Edit Schedule Modal -->
<div id="edit-schedule-overlay" style="display:none" role="dialog" aria-modal="true" aria-labelledby="edit-schedule-title">
<div id="edit-schedule-box" class="admin-modal admin-modal-wide">
<h2 id="edit-schedule-title">Edit Schedule</h2>
<div id="edit-schedule-error" role="alert" aria-live="assertive"></div>
<input id="es-id" type="hidden">
<label for="es-name">Name</label>
<input id="es-name" type="text" autocomplete="off">
<label for="es-desc">Description</label>
<input id="es-desc" type="text" autocomplete="off">
<label for="es-type">Schedule type</label>
<select id="es-type" onchange="toggleEditScheduleTypeFields()">
<option value="cron">Cron (recurring)</option>
<option value="at">At (one-shot)</option>
</select>
<div id="es-cron-group">
<label for="es-cron">Cron expression</label>
<input id="es-cron" type="text" autocomplete="off" spellcheck="false">
</div>
<div id="es-at-group" style="display:none">
<label for="es-at">Run at</label>
<input id="es-at" type="datetime-local">
</div>
<label for="es-target">Target</label>
<select id="es-target" onchange="toggleEditScheduleNodeField()">
<option value="auto">Auto (best available)</option>
<option value="pool">Pool (any bridge)</option>
<option value="all">All nodes</option>
<option value="node">Specific node...</option>
</select>
<div id="es-node-group" style="display:none">
<label for="es-node">Node ID</label>
<input id="es-node" type="text" autocomplete="off" spellcheck="false">
</div>
<label for="es-model">Model</label>
<input id="es-model" type="text" autocomplete="off">
<label for="es-message">Initial message</label>
<textarea id="es-message" rows="3"></textarea>
<label class="admin-checkbox"><input id="es-autoapprove" type="checkbox"> Auto-approve tool calls</label>
<label class="admin-checkbox"><input id="es-enabled" type="checkbox"> Enabled</label>
<div class="modal-buttons">
<button class="modal-cancel" onclick="hideEditScheduleModal()">Cancel</button>
<button id="es-submit" class="modal-submit" onclick="submitEditSchedule()">Save</button>
</div>
</div>
</div>
<!-- Schedule Runs Modal -->
<div id="schedule-runs-overlay" style="display:none" role="dialog" aria-modal="true" aria-labelledby="schedule-runs-title">
<div id="schedule-runs-box" class="admin-modal admin-modal-wide">
<h2 id="schedule-runs-title">Run History</h2>
<div id="schedule-runs-table"></div>
<div class="modal-buttons">
<button class="modal-cancel" onclick="hideScheduleRunsModal()">Close</button>
</div>
</div>
</div>
<script src="/static/admin.js"></script>
<script src="/static/app.js"></script>
</body>
+63 -5
View File
@@ -832,6 +832,58 @@
.admin-btn-danger:hover { opacity: 1; background: rgba(248, 113, 113, 0.1); }
.admin-btn-danger:focus-visible { outline: 2px solid var(--red); outline-offset: 2px; }
.admin-btn-action {
background: none;
border: 1px solid var(--border-strong);
color: var(--fg-dim);
font-family: var(--font-display);
font-size: 10px;
font-weight: 500;
padding: 2px 8px;
border-radius: var(--radius-sm);
cursor: pointer;
opacity: 0.8;
transition: opacity 0.15s, background 0.15s;
margin-right: 4px;
}
.admin-btn-action:hover { opacity: 1; background: rgba(255, 255, 255, 0.05); color: var(--fg); }
.admin-btn-action:focus-visible { outline: 2px solid var(--accent); outline-offset: 2px; }
/* Schedules grid: NAME | TYPE | SCHEDULE | TARGET | NEXT RUN | STATUS | ACTIONS */
#admin-schedules .admin-colheaders,
#admin-schedules .admin-row {
grid-template-columns: 1.5fr 60px 1.2fr 80px 130px 70px 170px;
}
/* Schedule runs grid: STARTED | NODE | STATUS | ERROR */
.sched-runs-grid { grid-template-columns: 2fr 1fr 1fr 2fr; }
/* Schedule status indicators */
.sched-active { color: var(--green); font-weight: 500; }
.sched-disabled { color: var(--fg-dim); }
.sched-expired { color: var(--accent); }
/* Wide modal variant for schedule forms */
.admin-modal-wide { width: 480px; }
/* Checkbox labels inside admin modals */
.admin-modal label.admin-checkbox {
display: flex;
align-items: center;
gap: 8px;
font-size: 12px;
font-weight: 500;
text-transform: none;
letter-spacing: 0;
color: var(--fg);
cursor: pointer;
margin-top: 14px;
}
.admin-modal label.admin-checkbox input[type="checkbox"] {
width: auto;
margin: 0;
}
/* Admin modals (reuse new-ws-overlay pattern) */
.admin-modal {
background: var(--bg-surface);
@@ -871,7 +923,7 @@
margin-top: 12px;
}
.admin-modal label:first-of-type { margin-top: 0; }
.admin-modal input, .admin-modal select {
.admin-modal input:not([type="hidden"]), .admin-modal select, .admin-modal textarea {
width: 100%;
padding: 9px 12px;
background: var(--bg);
@@ -882,12 +934,13 @@
font-size: 13px;
transition: border-color 0.15s, box-shadow 0.15s;
}
.admin-modal input:focus, .admin-modal select:focus {
.admin-modal input:focus, .admin-modal select:focus, .admin-modal textarea:focus {
border-color: var(--accent);
outline: none;
box-shadow: 0 0 0 3px var(--accent-dim);
}
.admin-modal input::placeholder { color: var(--fg-dim); opacity: 0.6; }
.admin-modal input::placeholder, .admin-modal textarea::placeholder { color: var(--fg-dim); opacity: 0.6; }
.admin-modal textarea { resize: vertical; min-height: 40px; }
.admin-modal [role="alert"] { color: var(--red); font-size: 12px; margin-bottom: 8px; display: none; }
.modal-buttons { display: flex; gap: 10px; margin-top: 20px; }
@@ -925,7 +978,8 @@
.modal-submit:focus-visible { outline: 2px solid var(--fg-bright); outline-offset: 2px; }
.modal-submit:disabled { opacity: 0.4; cursor: not-allowed; filter: none; }
#create-user-overlay, #create-token-overlay, #token-created-overlay, #create-channel-overlay, #confirm-overlay {
#create-user-overlay, #create-token-overlay, #token-created-overlay, #create-channel-overlay, #confirm-overlay,
#create-schedule-overlay, #edit-schedule-overlay, #schedule-runs-overlay {
position: fixed;
inset: 0;
background: rgba(0, 0, 0, 0.7);
@@ -969,6 +1023,10 @@
grid-template-columns: 80px 1fr 80px;
}
#admin-channels .admin-col-created { display: none; }
#admin-schedules .admin-colheaders, #admin-schedules .admin-row {
grid-template-columns: 1fr 60px 80px 130px;
}
.admin-col-sschedule, .admin-col-starget, .admin-col-snext { display: none; }
}
/* ==========================================================================
@@ -981,7 +1039,7 @@
.node-link, .dash-cell-node, .pagination button { transition: none; }
.dash-row.has-link::after, .node-group-header::before { transition: none; }
#new-ws-box select, #new-ws-box input, #new-ws-buttons button { transition: none; }
.admin-tab, .admin-row, .admin-btn-danger { transition: none; }
.admin-tab, .admin-row, .admin-btn-danger, .admin-btn-action { transition: none; }
.admin-action-btn, .modal-cancel, .modal-submit { transition: none; }
.admin-modal input, .admin-modal select { transition: none; }
}
+1
View File
@@ -49,6 +49,7 @@ TOKEN_BYTES = 32 # 64 hex chars after prefix
JWT_ISSUER = "turnstone"
JWT_AUD_SERVER = "turnstone-server"
JWT_AUD_CONSOLE = "turnstone-console"
JWT_AUD_CHANNEL = "turnstone-channel"
_MIN_SECRET_LENGTH = 32 # 256 bits minimum for HMAC-SHA256
VALID_SCOPES: frozenset[str] = frozenset({"read", "write", "approve"})
+43 -1
View File
@@ -45,6 +45,14 @@ _OPENAI_CAPABILITIES: dict[str, ModelCapabilities] = {
reasoning_effort_values=("minimal", "low", "medium", "high"),
default_reasoning_effort="medium",
),
# GPT-5 pro — high reasoning only, extended output
"gpt-5-pro": ModelCapabilities(
context_window=400000,
max_output_tokens=272000,
supports_temperature=False,
reasoning_effort_values=("high",),
default_reasoning_effort="high",
),
# GPT-5.1 — temperature OK when reasoning_effort=none (default)
"gpt-5.1": ModelCapabilities(
context_window=400000,
@@ -59,6 +67,36 @@ _OPENAI_CAPABILITIES: dict[str, ModelCapabilities] = {
reasoning_effort_values=("none", "low", "medium", "high", "xhigh"),
default_reasoning_effort="none",
),
# GPT-5.2 pro — always-reasoning variant
"gpt-5.2-pro": ModelCapabilities(
context_window=400000,
max_output_tokens=128000,
supports_temperature=False,
reasoning_effort_values=("medium", "high", "xhigh"),
default_reasoning_effort="medium",
),
# GPT-5.3 — same capabilities as 5.2 (matches gpt-5.3-chat-latest, codex)
"gpt-5.3": ModelCapabilities(
context_window=400000,
max_output_tokens=128000,
reasoning_effort_values=("none", "low", "medium", "high", "xhigh"),
default_reasoning_effort="none",
),
# GPT-5.4 — 1M context window
"gpt-5.4": ModelCapabilities(
context_window=1050000,
max_output_tokens=128000,
reasoning_effort_values=("none", "low", "medium", "high", "xhigh"),
default_reasoning_effort="none",
),
# GPT-5.4 pro — always-reasoning, 1M context
"gpt-5.4-pro": ModelCapabilities(
context_window=1050000,
max_output_tokens=128000,
supports_temperature=False,
reasoning_effort_values=("medium", "high", "xhigh"),
default_reasoning_effort="medium",
),
# O-series reasoning models
"o1": ModelCapabilities(
context_window=200000,
@@ -145,7 +183,11 @@ class OpenAIProvider:
else:
kwargs["temperature"] = temperature
if caps.reasoning_effort_values and reasoning_effort and reasoning_effort != "none":
kwargs["reasoning_effort"] = reasoning_effort
# Validate against supported values; fall back to model default
if reasoning_effort in caps.reasoning_effort_values:
kwargs["reasoning_effort"] = reasoning_effort
elif caps.default_reasoning_effort and caps.default_reasoning_effort != "none":
kwargs["reasoning_effort"] = caps.default_reasoning_effort
# -- web search ----------------------------------------------------------
+233
View File
@@ -26,6 +26,7 @@ import httpx
from turnstone.core.config import get_tavily_key
from turnstone.core.edit import find_occurrences, pick_nearest
from turnstone.core.log import get_logger
from turnstone.core.memory import (
delete_memory,
delete_session,
@@ -49,6 +50,7 @@ from turnstone.core.memory import (
from turnstone.core.providers import create_provider
from turnstone.core.safety import is_command_blocked, sanitize_command
from turnstone.core.sandbox import execute_math_sandboxed
from turnstone.core.storage._registry import get_storage
from turnstone.core.tools import (
AGENT_AUTO_TOOLS,
AGENT_TOOLS,
@@ -61,6 +63,8 @@ from turnstone.core.tools import (
from turnstone.core.web import check_ssrf, strip_html
from turnstone.ui.colors import DIM, GRAY, GREEN, RED, RESET, YELLOW, bold, cyan, dim
log = get_logger(__name__)
if TYPE_CHECKING:
from collections.abc import Iterator
@@ -91,6 +95,43 @@ class SessionUI(Protocol):
def on_rename(self, name: str) -> None: ...
# ---------------------------------------------------------------------------
# Notify auth helper (module-level, lazy-init)
# ---------------------------------------------------------------------------
_notify_token_manager: Any = None
_notify_token_lock = threading.Lock()
def _notify_auth_headers() -> dict[str, str]:
"""Return Authorization headers for outbound notify requests."""
global _notify_token_manager
# Static token from env takes precedence
static_token = os.environ.get("TURNSTONE_CHANNEL_AUTH_TOKEN", "").strip()
if static_token:
return {"Authorization": f"Bearer {static_token}"}
# JWT via ServiceTokenManager
jwt_secret = os.environ.get("TURNSTONE_JWT_SECRET", "").strip()
if not jwt_secret:
return {}
with _notify_token_lock:
if _notify_token_manager is None:
from turnstone.core.auth import JWT_AUD_CHANNEL, ServiceTokenManager
_notify_token_manager = ServiceTokenManager(
user_id="system",
scopes=frozenset({"write"}),
source="service",
secret=jwt_secret,
audience=JWT_AUD_CHANNEL,
)
header: dict[str, str] = _notify_token_manager.bearer_header
return header
# ---------------------------------------------------------------------------
# ChatSession — the core engine
# ---------------------------------------------------------------------------
@@ -162,6 +203,7 @@ class ChatSession:
self._system_tokens = 0 # tokens for system_messages
self._assistant_pending_tokens = 0
self.creative_mode = False
self._notify_count = 0
# MCP tool integration: merge external tools with built-in
self._mcp_client = mcp_client
if mcp_client:
@@ -468,6 +510,7 @@ class ChatSession:
def send(self, user_input: str) -> None:
"""Send user input and handle the response loop (including tool calls)."""
self._notify_count = 0
self.messages.append({"role": "user", "content": user_input})
self._msg_tokens.append(max(1, int(len(user_input) / self._chars_per_token)))
save_message(self._session_id, "user", user_input)
@@ -1248,6 +1291,7 @@ class ChatSession:
"remember": self._prepare_remember,
"recall": self._prepare_recall,
"forget": self._prepare_forget,
"notify": self._prepare_notify,
}
preparer = preparers.get(func_name)
if not preparer:
@@ -2394,6 +2438,195 @@ class ChatSession:
self.ui.on_tool_result(call_id, "recall", output)
return call_id, output
# -- Notify tool -----------------------------------------------------------
def _prepare_notify(self, call_id: str, args: dict[str, Any]) -> dict[str, Any]:
"""Prepare a channel notification."""
message = (args.get("message") or "").strip()
if not message:
return {
"call_id": call_id,
"func_name": "notify",
"header": "\u2717 notify: empty message",
"preview": "",
"needs_approval": False,
"error": "Error: message is required",
}
if len(message) > 2000:
return {
"call_id": call_id,
"func_name": "notify",
"header": "\u2717 notify: message too long",
"preview": "",
"needs_approval": False,
"error": "Error: message exceeds 2000 character limit",
}
username = (args.get("username") or "").strip()
channel_type = (args.get("channel_type") or "").strip()
channel_id = (args.get("channel_id") or "").strip()
title = (args.get("title") or "").strip()
has_username = bool(username)
has_direct = bool(channel_type and channel_id)
if has_username and has_direct:
return {
"call_id": call_id,
"func_name": "notify",
"header": "\u2717 notify: ambiguous target",
"preview": "",
"needs_approval": False,
"error": "Error: provide either username or channel_type+channel_id, not both",
}
if channel_type and not channel_id:
return {
"call_id": call_id,
"func_name": "notify",
"header": "\u2717 notify: incomplete target",
"preview": "",
"needs_approval": False,
"error": "Error: channel_id is required when channel_type is provided",
}
if channel_id and not channel_type:
return {
"call_id": call_id,
"func_name": "notify",
"header": "\u2717 notify: incomplete target",
"preview": "",
"needs_approval": False,
"error": "Error: channel_type is required when channel_id is provided",
}
if not has_username and not has_direct:
return {
"call_id": call_id,
"func_name": "notify",
"header": "\u2717 notify: no target",
"preview": "",
"needs_approval": False,
"error": "Error: provide username or channel_type+channel_id",
}
target_desc = f"@{username}" if has_username else f"{channel_type}:{channel_id}"
preview = message[:120] + ("..." if len(message) > 120 else "")
return {
"call_id": call_id,
"func_name": "notify",
"header": f"\u2709 notify \u2192 {target_desc}",
"preview": preview,
"needs_approval": False,
"execute": self._exec_notify,
"message": message,
"username": username,
"channel_type": channel_type,
"channel_id": channel_id,
"title": title,
}
_NOTIFY_MAX_RETRIES = 2
_NOTIFY_RETRY_DELAYS = (1.0, 3.0)
def _exec_notify(self, item: dict[str, Any]) -> tuple[str, str]:
"""Send a notification directly to the channel gateway via HTTP."""
call_id = item["call_id"]
if self._notify_count >= 5:
msg = "Error: notification rate limit exceeded (max 5 per turn)"
self.ui.on_tool_result(call_id, "notify", msg)
return call_id, msg
target: dict[str, str] = {}
if item.get("username"):
target["username"] = item["username"]
else:
target["channel_type"] = item["channel_type"]
target["channel_id"] = item["channel_id"]
payload = {
"target": target,
"message": item["message"],
"title": item.get("title", ""),
}
# Build auth headers for service-to-service call
auth_headers = _notify_auth_headers()
# Retry loop: attempt delivery, re-query services on each retry
# in case a gateway comes back online between attempts.
for attempt in range(1 + self._NOTIFY_MAX_RETRIES):
storage = get_storage()
services = storage.list_services("channel", max_age_seconds=120)
if not services:
if attempt < self._NOTIFY_MAX_RETRIES:
delay = self._NOTIFY_RETRY_DELAYS[attempt]
log.warning(
"notify.no_services",
attempt=attempt + 1,
max_retries=self._NOTIFY_MAX_RETRIES,
retry_delay=delay,
)
time.sleep(delay)
continue
log.warning("notify.no_services_exhausted")
msg = "Error: no channel gateway services available"
self.ui.on_tool_result(call_id, "notify", msg)
return call_id, msg
# Try first healthy gateway, fall back to next
last_error: str = ""
for svc in services:
url = svc["url"].rstrip("/") + "/v1/api/notify"
# SSRF guard: only allow http(s) URLs
if not url.startswith(("http://", "https://")):
continue
try:
resp = httpx.post(url, json=payload, timeout=10, headers=auth_headers)
if resp.status_code < 300:
# Check that at least one target was actually delivered
try:
data = resp.json()
except Exception:
last_error = "invalid gateway response"
continue
results = data.get("results") if isinstance(data, dict) else None
if isinstance(results, list) and any(
isinstance(r, dict) and r.get("status") == "sent" for r in results
):
self._notify_count += 1
msg = "Notification sent successfully"
self.ui.on_tool_result(call_id, "notify", msg)
return call_id, msg
last_error = "no successful deliveries"
continue
last_error = f"HTTP {resp.status_code}"
except Exception as exc:
last_error = type(exc).__name__
continue # try next gateway
# All gateways failed this attempt — retry if we have attempts left
if attempt < self._NOTIFY_MAX_RETRIES:
delay = self._NOTIFY_RETRY_DELAYS[attempt]
log.warning(
"notify.all_gateways_failed",
attempt=attempt + 1,
max_retries=self._NOTIFY_MAX_RETRIES,
last_error=last_error,
gateway_count=len(services),
retry_delay=delay,
)
time.sleep(delay)
else:
log.warning(
"notify.delivery_failed",
last_error=last_error,
gateway_count=len(services),
)
msg = "Error: notification delivery failed"
self.ui.on_tool_result(call_id, "notify", msg)
return call_id, msg
def _exec_write_file(self, item: dict[str, Any]) -> tuple[str, str]:
"""Write content to a file, creating parent directories as needed."""
call_id = item["call_id"]
+262
View File
@@ -879,6 +879,268 @@ class PostgreSQLBackend:
conn.commit()
return result.rowcount > 0
# -- Scheduled tasks -------------------------------------------------------
def create_scheduled_task(
self,
task_id: str,
name: str,
description: str,
schedule_type: str,
cron_expr: str,
at_time: str,
target_mode: str,
model: str,
initial_message: str,
auto_approve: bool,
auto_approve_tools: list[str],
created_by: str,
next_run: str,
) -> None:
from sqlalchemy.dialects import postgresql
from turnstone.core.storage._schema import scheduled_tasks
now = datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%S")
with self._engine.connect() as conn:
conn.execute(
postgresql.insert(scheduled_tasks)
.values(
task_id=task_id,
name=name,
description=description,
schedule_type=schedule_type,
cron_expr=cron_expr,
at_time=at_time,
target_mode=target_mode,
model=model,
initial_message=initial_message,
auto_approve=1 if auto_approve else 0,
auto_approve_tools=",".join(auto_approve_tools),
enabled=1,
created_by=created_by,
next_run=next_run,
created=now,
updated=now,
)
.on_conflict_do_nothing()
)
conn.commit()
def get_scheduled_task(self, task_id: str) -> dict[str, Any] | None:
from turnstone.core.storage._schema import scheduled_tasks
with self._engine.connect() as conn:
row = conn.execute(
sa.select(scheduled_tasks).where(scheduled_tasks.c.task_id == task_id)
).fetchone()
if row is None:
return None
return dict(row._mapping)
def list_scheduled_tasks(self) -> list[dict[str, Any]]:
from turnstone.core.storage._schema import scheduled_tasks
with self._engine.connect() as conn:
rows = conn.execute(
sa.select(scheduled_tasks).order_by(scheduled_tasks.c.created.desc())
).fetchall()
return [dict(r._mapping) for r in rows]
_UPDATABLE_TASK_FIELDS = frozenset(
{
"name",
"description",
"schedule_type",
"cron_expr",
"at_time",
"target_mode",
"model",
"initial_message",
"auto_approve",
"auto_approve_tools",
"enabled",
"last_run",
"next_run",
"updated",
}
)
def update_scheduled_task(self, task_id: str, **fields: Any) -> bool:
from turnstone.core.storage._schema import scheduled_tasks
fields = {k: v for k, v in fields.items() if k in self._UPDATABLE_TASK_FIELDS}
fields["updated"] = datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%S")
if "auto_approve" in fields:
fields["auto_approve"] = 1 if fields["auto_approve"] else 0
if "auto_approve_tools" in fields and isinstance(fields["auto_approve_tools"], list):
fields["auto_approve_tools"] = ",".join(fields["auto_approve_tools"])
if "enabled" in fields:
fields["enabled"] = 1 if fields["enabled"] else 0
with self._engine.connect() as conn:
result = conn.execute(
sa.update(scheduled_tasks)
.where(scheduled_tasks.c.task_id == task_id)
.values(**fields)
)
conn.commit()
return result.rowcount > 0
def delete_scheduled_task(self, task_id: str) -> bool:
from turnstone.core.storage._schema import scheduled_task_runs, scheduled_tasks
with self._engine.connect() as conn:
conn.execute(
sa.delete(scheduled_task_runs).where(scheduled_task_runs.c.task_id == task_id)
)
result = conn.execute(
sa.delete(scheduled_tasks).where(scheduled_tasks.c.task_id == task_id)
)
conn.commit()
return result.rowcount > 0
def list_due_tasks(self, now: str) -> list[dict[str, Any]]:
from turnstone.core.storage._schema import scheduled_tasks
with self._engine.connect() as conn:
rows = conn.execute(
sa.select(scheduled_tasks)
.where(
(scheduled_tasks.c.enabled == 1)
& (scheduled_tasks.c.next_run <= now)
& (scheduled_tasks.c.next_run != "")
)
.order_by(scheduled_tasks.c.next_run)
.limit(100)
).fetchall()
return [dict(r._mapping) for r in rows]
def record_task_run(
self,
run_id: str,
task_id: str,
node_id: str,
ws_id: str,
correlation_id: str,
started: str,
status: str,
error: str,
) -> None:
from turnstone.core.storage._schema import scheduled_task_runs
with self._engine.connect() as conn:
conn.execute(
sa.insert(scheduled_task_runs),
{
"run_id": run_id,
"task_id": task_id,
"node_id": node_id,
"ws_id": ws_id,
"correlation_id": correlation_id,
"started": started,
"status": status,
"error": error,
},
)
conn.commit()
def list_task_runs(self, task_id: str, limit: int = 50) -> list[dict[str, Any]]:
from turnstone.core.storage._schema import scheduled_task_runs
with self._engine.connect() as conn:
rows = conn.execute(
sa.select(scheduled_task_runs)
.where(scheduled_task_runs.c.task_id == task_id)
.order_by(scheduled_task_runs.c.started.desc())
.limit(limit)
).fetchall()
return [dict(r._mapping) for r in rows]
def prune_task_runs(self, retention_days: int = 90) -> int:
from datetime import timedelta
from turnstone.core.storage._schema import scheduled_task_runs
cutoff = (datetime.now(UTC) - timedelta(days=retention_days)).strftime("%Y-%m-%dT%H:%M:%S")
with self._engine.connect() as conn:
result = conn.execute(
sa.delete(scheduled_task_runs).where(scheduled_task_runs.c.started < cutoff)
)
conn.commit()
return result.rowcount
# -- Service registry ------------------------------------------------------
def register_service(
self, service_type: str, service_id: str, url: str, metadata: str = "{}"
) -> None:
from turnstone.core.storage._schema import services
now = datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%S")
with self._engine.connect() as conn:
from sqlalchemy.dialects.postgresql import insert as pg_insert
stmt = pg_insert(services).values(
service_type=service_type,
service_id=service_id,
url=url,
metadata=metadata,
last_heartbeat=now,
created=now,
)
stmt = stmt.on_conflict_do_update(
index_elements=[services.c.service_type, services.c.service_id],
set_={"url": url, "metadata": metadata, "last_heartbeat": now},
)
conn.execute(stmt)
conn.commit()
def heartbeat_service(self, service_type: str, service_id: str) -> bool:
from turnstone.core.storage._schema import services
now = datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%S")
with self._engine.connect() as conn:
result = conn.execute(
sa.update(services)
.where(
(services.c.service_type == service_type)
& (services.c.service_id == service_id)
)
.values(last_heartbeat=now)
)
conn.commit()
return result.rowcount > 0
def list_services(self, service_type: str, max_age_seconds: int = 120) -> list[dict[str, str]]:
from turnstone.core.storage._schema import services
cutoff = (datetime.now(UTC) - timedelta(seconds=max_age_seconds)).strftime(
"%Y-%m-%dT%H:%M:%S"
)
with self._engine.connect() as conn:
rows = conn.execute(
sa.select(services)
.where(
(services.c.service_type == service_type)
& (services.c.last_heartbeat >= cutoff)
)
.order_by(services.c.last_heartbeat.desc())
).fetchall()
return [dict(r._mapping) for r in rows]
def deregister_service(self, service_type: str, service_id: str) -> bool:
from turnstone.core.storage._schema import services
with self._engine.connect() as conn:
result = conn.execute(
sa.delete(services).where(
(services.c.service_type == service_type)
& (services.c.service_id == service_id)
)
)
conn.commit()
return result.rowcount > 0
# -- Lifecycle -------------------------------------------------------------
def close(self) -> None:
+83
View File
@@ -249,6 +249,89 @@ class StorageBackend(Protocol):
"""Remove a channel route. Returns True if existed."""
...
# -- Scheduled tasks -------------------------------------------------------
def create_scheduled_task(
self,
task_id: str,
name: str,
description: str,
schedule_type: str,
cron_expr: str,
at_time: str,
target_mode: str,
model: str,
initial_message: str,
auto_approve: bool,
auto_approve_tools: list[str],
created_by: str,
next_run: str,
) -> None:
"""Create a scheduled task. No-op if task_id already exists."""
...
def get_scheduled_task(self, task_id: str) -> dict[str, Any] | None:
"""Return scheduled task dict or None."""
...
def list_scheduled_tasks(self) -> list[dict[str, Any]]:
"""Return all scheduled tasks ordered by created DESC."""
...
def update_scheduled_task(self, task_id: str, **fields: Any) -> bool:
"""Update specified fields on a scheduled task. Returns True if found."""
...
def delete_scheduled_task(self, task_id: str) -> bool:
"""Delete a scheduled task and its run history. Returns True if found."""
...
def list_due_tasks(self, now: str) -> list[dict[str, Any]]:
"""Return enabled tasks whose next_run <= now, ordered by next_run."""
...
def record_task_run(
self,
run_id: str,
task_id: str,
node_id: str,
ws_id: str,
correlation_id: str,
started: str,
status: str,
error: str,
) -> None:
"""Record a scheduled task execution."""
...
def list_task_runs(self, task_id: str, limit: int = 50) -> list[dict[str, Any]]:
"""List run history for a task, ordered by started DESC."""
...
def prune_task_runs(self, retention_days: int = 90) -> int:
"""Delete task runs older than retention_days. Returns count deleted."""
...
# -- Service registry ------------------------------------------------------
def register_service(
self, service_type: str, service_id: str, url: str, metadata: str = "{}"
) -> None:
"""Register or update a service instance. Upserts by (service_type, service_id)."""
...
def heartbeat_service(self, service_type: str, service_id: str) -> bool:
"""Update last_heartbeat for a registered service. Returns False if not found."""
...
def list_services(self, service_type: str, max_age_seconds: int = 120) -> list[dict[str, str]]:
"""Return healthy services of a given type (heartbeat within max_age_seconds)."""
...
def deregister_service(self, service_type: str, service_id: str) -> bool:
"""Remove a service registration. Returns True if existed."""
...
# -- Lifecycle -------------------------------------------------------------
def close(self) -> None:
+63
View File
@@ -137,3 +137,66 @@ channel_routes = sa.Table(
)
sa.Index("idx_channel_routes_ws", channel_routes.c.ws_id)
# ---------------------------------------------------------------------------
# Scheduled task tables
# ---------------------------------------------------------------------------
scheduled_tasks = sa.Table(
"scheduled_tasks",
metadata,
sa.Column("task_id", sa.Text, primary_key=True),
sa.Column("name", sa.Text, nullable=False),
sa.Column("description", sa.Text, nullable=False, server_default=""),
sa.Column("schedule_type", sa.Text, nullable=False), # "cron" or "at"
sa.Column("cron_expr", sa.Text, nullable=False, server_default=""),
sa.Column("at_time", sa.Text, nullable=False, server_default=""), # ISO8601
sa.Column("target_mode", sa.Text, nullable=False, server_default="auto"),
sa.Column("model", sa.Text, nullable=False, server_default=""),
sa.Column("initial_message", sa.Text, nullable=False),
sa.Column("auto_approve", sa.Integer, nullable=False, server_default="0"),
sa.Column("auto_approve_tools", sa.Text, nullable=False, server_default=""),
sa.Column("enabled", sa.Integer, nullable=False, server_default="1"),
sa.Column("created_by", sa.Text, nullable=False, server_default=""),
sa.Column("last_run", sa.Text),
sa.Column("next_run", sa.Text),
sa.Column("created", sa.Text, nullable=False),
sa.Column("updated", sa.Text, nullable=False),
)
sa.Index("idx_scheduled_tasks_enabled", scheduled_tasks.c.enabled)
sa.Index("idx_scheduled_tasks_next_run", scheduled_tasks.c.next_run)
scheduled_task_runs = sa.Table(
"scheduled_task_runs",
metadata,
sa.Column("run_id", sa.Text, primary_key=True),
sa.Column("task_id", sa.Text, nullable=False),
sa.Column("node_id", sa.Text, nullable=False, server_default=""),
sa.Column("ws_id", sa.Text, nullable=False, server_default=""),
sa.Column("correlation_id", sa.Text, nullable=False, server_default=""),
sa.Column("started", sa.Text, nullable=False),
sa.Column("status", sa.Text, nullable=False, server_default="dispatched"),
sa.Column("error", sa.Text, nullable=False, server_default=""),
)
sa.Index("idx_scheduled_task_runs_task_id", scheduled_task_runs.c.task_id)
sa.Index("idx_scheduled_task_runs_started", scheduled_task_runs.c.started)
# ---------------------------------------------------------------------------
# Service registry
# ---------------------------------------------------------------------------
services = sa.Table(
"services",
metadata,
sa.Column("service_type", sa.Text, nullable=False),
sa.Column("service_id", sa.Text, nullable=False),
sa.Column("url", sa.Text, nullable=False),
sa.Column("metadata", sa.Text, nullable=False, server_default="{}"),
sa.Column("last_heartbeat", sa.Text, nullable=False),
sa.Column("created", sa.Text, nullable=False),
sa.PrimaryKeyConstraint("service_type", "service_id"),
)
sa.Index("idx_services_type_heartbeat", services.c.service_type, services.c.last_heartbeat)
+260
View File
@@ -929,6 +929,266 @@ class SQLiteBackend:
conn.commit()
return result.rowcount > 0
# -- Scheduled tasks -------------------------------------------------------
def create_scheduled_task(
self,
task_id: str,
name: str,
description: str,
schedule_type: str,
cron_expr: str,
at_time: str,
target_mode: str,
model: str,
initial_message: str,
auto_approve: bool,
auto_approve_tools: list[str],
created_by: str,
next_run: str,
) -> None:
from turnstone.core.storage._schema import scheduled_tasks
now = datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%S")
with self._engine.connect() as conn:
conn.execute(
sa.insert(scheduled_tasks).prefix_with("OR IGNORE"),
{
"task_id": task_id,
"name": name,
"description": description,
"schedule_type": schedule_type,
"cron_expr": cron_expr,
"at_time": at_time,
"target_mode": target_mode,
"model": model,
"initial_message": initial_message,
"auto_approve": 1 if auto_approve else 0,
"auto_approve_tools": ",".join(auto_approve_tools),
"enabled": 1,
"created_by": created_by,
"next_run": next_run,
"created": now,
"updated": now,
},
)
conn.commit()
def get_scheduled_task(self, task_id: str) -> dict[str, Any] | None:
from turnstone.core.storage._schema import scheduled_tasks
with self._engine.connect() as conn:
row = conn.execute(
sa.select(scheduled_tasks).where(scheduled_tasks.c.task_id == task_id)
).fetchone()
if row is None:
return None
return dict(row._mapping)
def list_scheduled_tasks(self) -> list[dict[str, Any]]:
from turnstone.core.storage._schema import scheduled_tasks
with self._engine.connect() as conn:
rows = conn.execute(
sa.select(scheduled_tasks).order_by(scheduled_tasks.c.created.desc())
).fetchall()
return [dict(r._mapping) for r in rows]
_UPDATABLE_TASK_FIELDS = frozenset(
{
"name",
"description",
"schedule_type",
"cron_expr",
"at_time",
"target_mode",
"model",
"initial_message",
"auto_approve",
"auto_approve_tools",
"enabled",
"last_run",
"next_run",
"updated",
}
)
def update_scheduled_task(self, task_id: str, **fields: Any) -> bool:
from turnstone.core.storage._schema import scheduled_tasks
fields = {k: v for k, v in fields.items() if k in self._UPDATABLE_TASK_FIELDS}
fields["updated"] = datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%S")
# Normalize boolean → int for auto_approve
if "auto_approve" in fields:
fields["auto_approve"] = 1 if fields["auto_approve"] else 0
if "auto_approve_tools" in fields and isinstance(fields["auto_approve_tools"], list):
fields["auto_approve_tools"] = ",".join(fields["auto_approve_tools"])
if "enabled" in fields:
fields["enabled"] = 1 if fields["enabled"] else 0
with self._engine.connect() as conn:
result = conn.execute(
sa.update(scheduled_tasks)
.where(scheduled_tasks.c.task_id == task_id)
.values(**fields)
)
conn.commit()
return result.rowcount > 0
def delete_scheduled_task(self, task_id: str) -> bool:
from turnstone.core.storage._schema import scheduled_task_runs, scheduled_tasks
with self._engine.connect() as conn:
conn.execute(
sa.delete(scheduled_task_runs).where(scheduled_task_runs.c.task_id == task_id)
)
result = conn.execute(
sa.delete(scheduled_tasks).where(scheduled_tasks.c.task_id == task_id)
)
conn.commit()
return result.rowcount > 0
def list_due_tasks(self, now: str) -> list[dict[str, Any]]:
from turnstone.core.storage._schema import scheduled_tasks
with self._engine.connect() as conn:
rows = conn.execute(
sa.select(scheduled_tasks)
.where(
(scheduled_tasks.c.enabled == 1)
& (scheduled_tasks.c.next_run <= now)
& (scheduled_tasks.c.next_run != "")
)
.order_by(scheduled_tasks.c.next_run)
.limit(100)
).fetchall()
return [dict(r._mapping) for r in rows]
def record_task_run(
self,
run_id: str,
task_id: str,
node_id: str,
ws_id: str,
correlation_id: str,
started: str,
status: str,
error: str,
) -> None:
from turnstone.core.storage._schema import scheduled_task_runs
with self._engine.connect() as conn:
conn.execute(
sa.insert(scheduled_task_runs),
{
"run_id": run_id,
"task_id": task_id,
"node_id": node_id,
"ws_id": ws_id,
"correlation_id": correlation_id,
"started": started,
"status": status,
"error": error,
},
)
conn.commit()
def list_task_runs(self, task_id: str, limit: int = 50) -> list[dict[str, Any]]:
from turnstone.core.storage._schema import scheduled_task_runs
with self._engine.connect() as conn:
rows = conn.execute(
sa.select(scheduled_task_runs)
.where(scheduled_task_runs.c.task_id == task_id)
.order_by(scheduled_task_runs.c.started.desc())
.limit(limit)
).fetchall()
return [dict(r._mapping) for r in rows]
def prune_task_runs(self, retention_days: int = 90) -> int:
from datetime import timedelta
from turnstone.core.storage._schema import scheduled_task_runs
cutoff = (datetime.now(UTC) - timedelta(days=retention_days)).strftime("%Y-%m-%dT%H:%M:%S")
with self._engine.connect() as conn:
result = conn.execute(
sa.delete(scheduled_task_runs).where(scheduled_task_runs.c.started < cutoff)
)
conn.commit()
return result.rowcount
# -- Service registry ------------------------------------------------------
def register_service(
self, service_type: str, service_id: str, url: str, metadata: str = "{}"
) -> None:
from sqlalchemy.dialects.sqlite import insert as sqlite_insert
from turnstone.core.storage._schema import services
now = datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%S")
stmt = sqlite_insert(services).values(
service_type=service_type,
service_id=service_id,
url=url,
metadata=metadata,
last_heartbeat=now,
created=now,
)
stmt = stmt.on_conflict_do_update(
index_elements=["service_type", "service_id"],
set_={"url": url, "metadata": metadata, "last_heartbeat": now},
)
with self._engine.connect() as conn:
conn.execute(stmt)
conn.commit()
def heartbeat_service(self, service_type: str, service_id: str) -> bool:
from turnstone.core.storage._schema import services
now = datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%S")
with self._engine.connect() as conn:
result = conn.execute(
sa.update(services)
.where(
(services.c.service_type == service_type)
& (services.c.service_id == service_id)
)
.values(last_heartbeat=now)
)
conn.commit()
return result.rowcount > 0
def list_services(self, service_type: str, max_age_seconds: int = 120) -> list[dict[str, str]]:
from turnstone.core.storage._schema import services
cutoff = (datetime.now(UTC) - timedelta(seconds=max_age_seconds)).strftime(
"%Y-%m-%dT%H:%M:%S"
)
with self._engine.connect() as conn:
rows = conn.execute(
sa.select(services)
.where(
(services.c.service_type == service_type)
& (services.c.last_heartbeat >= cutoff)
)
.order_by(services.c.last_heartbeat.desc())
).fetchall()
return [dict(r._mapping) for r in rows]
def deregister_service(self, service_type: str, service_id: str) -> bool:
from turnstone.core.storage._schema import services
with self._engine.connect() as conn:
result = conn.execute(
sa.delete(services).where(
(services.c.service_type == service_type)
& (services.c.service_id == service_id)
)
)
conn.commit()
return result.rowcount > 0
# -- Lifecycle -------------------------------------------------------------
def close(self) -> None:
@@ -0,0 +1,62 @@
"""Scheduled tasks and run history tables.
Revision ID: 004
Revises: 003
Create Date: 2026-03-05
"""
import sqlalchemy as sa
from alembic import op
revision = "004"
down_revision = "003"
branch_labels = None
depends_on = None
def upgrade() -> None:
op.create_table(
"scheduled_tasks",
sa.Column("task_id", sa.Text, primary_key=True),
sa.Column("name", sa.Text, nullable=False),
sa.Column("description", sa.Text, nullable=False, server_default=""),
sa.Column("schedule_type", sa.Text, nullable=False),
sa.Column("cron_expr", sa.Text, nullable=False, server_default=""),
sa.Column("at_time", sa.Text, nullable=False, server_default=""),
sa.Column("target_mode", sa.Text, nullable=False, server_default="auto"),
sa.Column("model", sa.Text, nullable=False, server_default=""),
sa.Column("initial_message", sa.Text, nullable=False),
sa.Column("auto_approve", sa.Integer, nullable=False, server_default="0"),
sa.Column("auto_approve_tools", sa.Text, nullable=False, server_default=""),
sa.Column("enabled", sa.Integer, nullable=False, server_default="1"),
sa.Column("created_by", sa.Text, nullable=False, server_default=""),
sa.Column("last_run", sa.Text),
sa.Column("next_run", sa.Text),
sa.Column("created", sa.Text, nullable=False),
sa.Column("updated", sa.Text, nullable=False),
)
op.create_index("idx_scheduled_tasks_enabled", "scheduled_tasks", ["enabled"])
op.create_index("idx_scheduled_tasks_next_run", "scheduled_tasks", ["next_run"])
op.create_table(
"scheduled_task_runs",
sa.Column("run_id", sa.Text, primary_key=True),
sa.Column("task_id", sa.Text, nullable=False),
sa.Column("node_id", sa.Text, nullable=False, server_default=""),
sa.Column("ws_id", sa.Text, nullable=False, server_default=""),
sa.Column("correlation_id", sa.Text, nullable=False, server_default=""),
sa.Column("started", sa.Text, nullable=False),
sa.Column("status", sa.Text, nullable=False, server_default="dispatched"),
sa.Column("error", sa.Text, nullable=False, server_default=""),
)
op.create_index("idx_scheduled_task_runs_task_id", "scheduled_task_runs", ["task_id"])
op.create_index("idx_scheduled_task_runs_started", "scheduled_task_runs", ["started"])
def downgrade() -> None:
op.drop_index("idx_scheduled_task_runs_started", "scheduled_task_runs")
op.drop_index("idx_scheduled_task_runs_task_id", "scheduled_task_runs")
op.drop_table("scheduled_task_runs")
op.drop_index("idx_scheduled_tasks_next_run", "scheduled_tasks")
op.drop_index("idx_scheduled_tasks_enabled", "scheduled_tasks")
op.drop_table("scheduled_tasks")
@@ -0,0 +1,33 @@
"""Service registry table.
Revision ID: 005
Revises: 004
Create Date: 2026-03-05
"""
import sqlalchemy as sa
from alembic import op
revision = "005"
down_revision = "004"
branch_labels = None
depends_on = None
def upgrade() -> None:
op.create_table(
"services",
sa.Column("service_type", sa.Text, nullable=False),
sa.Column("service_id", sa.Text, nullable=False),
sa.Column("url", sa.Text, nullable=False),
sa.Column("metadata", sa.Text, nullable=False, server_default="{}"),
sa.Column("last_heartbeat", sa.Text, nullable=False),
sa.Column("created", sa.Text, nullable=False),
sa.PrimaryKeyConstraint("service_type", "service_id"),
)
op.create_index("idx_services_type_heartbeat", "services", ["service_type", "last_heartbeat"])
def downgrade() -> None:
op.drop_index("idx_services_type_heartbeat", "services")
op.drop_table("services")
+3
View File
@@ -362,6 +362,9 @@ class Bridge:
model = getattr(msg, "model", "")
initial_message = getattr(msg, "initial_message", "")
resume_session = getattr(msg, "resume_session", "")
user_id = getattr(msg, "user_id", "")
if user_id:
log.info("bridge.create_ws user_id=%s name=%s model=%s", user_id, name, model)
ws_id, resumed = self._create_ws_on_server(
name=name,
auto_approve=auto_approve,
+1
View File
@@ -96,6 +96,7 @@ class CreateWorkstreamMessage(InboundMessage):
model: str = ""
initial_message: str = ""
resume_session: str = ""
user_id: str = ""
@dataclass
+186
View File
@@ -25,12 +25,17 @@ from turnstone.api.schemas import (
AuthLoginResponse,
AuthSetupResponse,
AuthStatusResponse,
ListScheduleRunsResponse,
ListSchedulesResponse,
ScheduleInfo,
StatusResponse,
)
from turnstone.sdk._base import _BaseClient
from turnstone.sdk._sync import _SyncRunner
from turnstone.sdk.events import ClusterEvent
_UNSET: Any = object()
if TYPE_CHECKING:
from collections.abc import AsyncIterator, Iterator
@@ -179,6 +184,109 @@ class AsyncTurnstoneConsole(_BaseClient):
async def health(self) -> ConsoleHealthResponse:
return await self._request("GET", "/health", response_model=ConsoleHealthResponse)
# -- schedules -----------------------------------------------------------
async def list_schedules(self) -> ListSchedulesResponse:
return await self._request(
"GET", "/v1/api/admin/schedules", response_model=ListSchedulesResponse
)
async def create_schedule(
self,
*,
name: str,
schedule_type: str,
initial_message: str,
description: str = "",
cron_expr: str = "",
at_time: str = "",
target_mode: str = "auto",
model: str = "",
auto_approve: bool = False,
auto_approve_tools: list[str] | None = None,
enabled: bool = True,
) -> ScheduleInfo:
body: dict[str, Any] = {
"name": name,
"schedule_type": schedule_type,
"initial_message": initial_message,
"target_mode": target_mode,
"auto_approve": auto_approve,
"enabled": enabled,
}
if description:
body["description"] = description
if cron_expr:
body["cron_expr"] = cron_expr
if at_time:
body["at_time"] = at_time
if model:
body["model"] = model
if auto_approve_tools:
body["auto_approve_tools"] = auto_approve_tools
return await self._request(
"POST", "/v1/api/admin/schedules", json_body=body, response_model=ScheduleInfo
)
async def get_schedule(self, task_id: str) -> ScheduleInfo:
return await self._request(
"GET", f"/v1/api/admin/schedules/{task_id}", response_model=ScheduleInfo
)
async def update_schedule(
self,
task_id: str,
*,
name: Any = _UNSET,
description: Any = _UNSET,
schedule_type: Any = _UNSET,
cron_expr: Any = _UNSET,
at_time: Any = _UNSET,
target_mode: Any = _UNSET,
model: Any = _UNSET,
initial_message: Any = _UNSET,
auto_approve: Any = _UNSET,
auto_approve_tools: Any = _UNSET,
enabled: Any = _UNSET,
) -> ScheduleInfo:
body: dict[str, Any] = {}
for key, val in [
("name", name),
("description", description),
("schedule_type", schedule_type),
("cron_expr", cron_expr),
("at_time", at_time),
("target_mode", target_mode),
("model", model),
("initial_message", initial_message),
("auto_approve", auto_approve),
("auto_approve_tools", auto_approve_tools),
("enabled", enabled),
]:
if val is not _UNSET:
body[key] = val
return await self._request(
"PUT",
f"/v1/api/admin/schedules/{task_id}",
json_body=body,
response_model=ScheduleInfo,
)
async def delete_schedule(self, task_id: str) -> StatusResponse:
return await self._request(
"DELETE", f"/v1/api/admin/schedules/{task_id}", response_model=StatusResponse
)
async def list_schedule_runs(
self, task_id: str, *, limit: int = 50
) -> ListScheduleRunsResponse:
return await self._request(
"GET",
f"/v1/api/admin/schedules/{task_id}/runs",
params={"limit": limit},
response_model=ListScheduleRunsResponse,
)
class TurnstoneConsole:
"""Synchronous client for the turnstone console API.
@@ -274,6 +382,84 @@ class TurnstoneConsole:
def health(self) -> ConsoleHealthResponse:
return self._runner.run(self._async.health())
# -- schedules -----------------------------------------------------------
def list_schedules(self) -> ListSchedulesResponse:
return self._runner.run(self._async.list_schedules())
def create_schedule(
self,
*,
name: str,
schedule_type: str,
initial_message: str,
description: str = "",
cron_expr: str = "",
at_time: str = "",
target_mode: str = "auto",
model: str = "",
auto_approve: bool = False,
auto_approve_tools: list[str] | None = None,
enabled: bool = True,
) -> ScheduleInfo:
return self._runner.run(
self._async.create_schedule(
name=name,
schedule_type=schedule_type,
initial_message=initial_message,
description=description,
cron_expr=cron_expr,
at_time=at_time,
target_mode=target_mode,
model=model,
auto_approve=auto_approve,
auto_approve_tools=auto_approve_tools,
enabled=enabled,
)
)
def get_schedule(self, task_id: str) -> ScheduleInfo:
return self._runner.run(self._async.get_schedule(task_id))
def update_schedule(
self,
task_id: str,
*,
name: Any = _UNSET,
description: Any = _UNSET,
schedule_type: Any = _UNSET,
cron_expr: Any = _UNSET,
at_time: Any = _UNSET,
target_mode: Any = _UNSET,
model: Any = _UNSET,
initial_message: Any = _UNSET,
auto_approve: Any = _UNSET,
auto_approve_tools: Any = _UNSET,
enabled: Any = _UNSET,
) -> ScheduleInfo:
return self._runner.run(
self._async.update_schedule(
task_id,
name=name,
description=description,
schedule_type=schedule_type,
cron_expr=cron_expr,
at_time=at_time,
target_mode=target_mode,
model=model,
initial_message=initial_message,
auto_approve=auto_approve,
auto_approve_tools=auto_approve_tools,
enabled=enabled,
)
)
def delete_schedule(self, task_id: str) -> StatusResponse:
return self._runner.run(self._async.delete_schedule(task_id))
def list_schedule_runs(self, task_id: str, *, limit: int = 50) -> ListScheduleRunsResponse:
return self._runner.run(self._async.list_schedule_runs(task_id, limit=limit))
# -- lifecycle -----------------------------------------------------------
def close(self) -> None:
+35
View File
@@ -0,0 +1,35 @@
{
"name": "notify",
"description": "Send a notification to a user or channel on an external platform (Discord, etc.). Use this to proactively alert people about task completion, errors, or important updates. Provide either 'username' for user-based targeting (sends to all linked channels) or 'channel_type' + 'channel_id' for direct targeting. Do not combine both.",
"parameters": {
"type": "object",
"properties": {
"message": {
"type": "string",
"description": "The notification content (plain text, max 2000 chars)."
},
"username": {
"type": "string",
"description": "Turnstone username to notify. Sends to all linked channels for this user."
},
"channel_type": {
"type": "string",
"description": "Channel platform for direct targeting.",
"enum": ["discord"]
},
"channel_id": {
"type": "string",
"description": "Platform-specific channel or user ID for direct targeting."
},
"title": {
"type": "string",
"description": "Optional short title for the notification."
}
},
"required": ["message"]
},
"agent": true,
"task_agent": true,
"auto_approve": true,
"primary_key": "message"
}