diff --git a/docs/api-reference.md b/docs/api-reference.md index 4fc48079..ceea754e 100644 --- a/docs/api-reference.md +++ b/docs/api-reference.md @@ -765,6 +765,9 @@ All fields are optional. The body can be empty or an empty JSON object. | `auto_approve` | bool | false | Auto-approve all tool calls for this workstream | | `resume_ws` | string | "" | Workstream ID to resume atomically during creation (empty = fresh)| | `template` | string | "" | Prompt template name (replaces default templates; 400 if not found)| +| `ws_template` | string | "" | Workstream template name. Applies model, temperature, reasoning effort, max tokens, auto-approve policy, and token budget. Returns 400 if not found or disabled. | + +> **Template precedence:** When `ws_template` is specified, its model override takes effect before workstream creation. Both `template` (prompt template) and `ws_template` (workstream template) can be used together — `ws_template` controls the behavioral profile while `template` sets the system message text. If `ws_template` defines its own system prompt or prompt template reference, that takes precedence over the `template` parameter. **Response (success):** diff --git a/docs/architecture.md b/docs/architecture.md index 39720f34..f7216c54 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -663,7 +663,8 @@ supports_vision = true **Per-workstream selection:** `POST /v1/api/workstreams/new` accepts an optional `"model"` field. The bridge `CreateWorkstreamMessage` carries the same field -through the MQ protocol. +through the MQ protocol, along with `ws_template` (workstream template name) +which can override the model before workstream creation. ### Tool Output Truncation @@ -1237,7 +1238,11 @@ The console has two write-path capabilities: 1. **Workstream creation** — pushes `CreateWorkstreamMessage` to Redis inbound queues targeting specific nodes. The bridge on each node picks up the message and creates the workstream on the local server. Auto-selects the node with - the most available capacity if no target is specified. + the most available capacity if no target is specified. When a `ws_template` + field is present, the server resolves the template BEFORE `mgr.create()` + (applying the model override to the creation request) and snapshot-applies + remaining settings (auto-approve, token budget, temperature, etc.) to the + workstream config AFTER creation. 2. **Reverse proxy** — serves each node's server UI through the console port at `/node/{node_id}/`. Uses `httpx.AsyncClient` to proxy HTTP and SSE traffic. @@ -1374,6 +1379,17 @@ Prompt templates provide reusable system messages with `{{variable}}` substitution. Usage events are recorded per-LLM-request for token accounting. An append-only audit log captures all admin mutations. -The console admin panel adds 5 governance tabs (Roles, Policies, Templates, -Usage, Audit) for a total of 10 tabs, all permission-gated. Both Python -and TypeScript SDKs expose governance methods on the console client. +Workstream templates build on top of prompt templates as complete behavioral +profiles applied at workstream creation. While prompt templates inject system +message text, workstream templates define model, temperature, reasoning effort, +max tokens, auto-approve policy, token budget, and agent max turns. Templates +are snapshot-applied once at creation — not a live binding. The +`workstream_templates` table (migration 011) supports auto-versioning, and +workstreams record which template and version spawned them. Token budget +enforcement tracks consumption in `session.send()` with 80% warning and +100% approval gate via the `__budget_override__` synthetic tool name. + +The console admin panel adds 6 governance tabs (Roles, Policies, Templates, +WS Templates, Usage, Audit) for a total of 11 tabs, all permission-gated. +Both Python and TypeScript SDKs expose governance methods on the console +client. diff --git a/docs/console.md b/docs/console.md index 32917bf6..c326ff3c 100644 --- a/docs/console.md +++ b/docs/console.md @@ -306,6 +306,18 @@ Revoke a specific API token. These endpoints manage the `channel_users` table mappings that connect external platform identities (e.g. Discord user IDs) to turnstone users. See [Channel Integrations](channels.md) for details on the linking flow. +### Workstream Templates + +| Method | Path | Description | +|--------|------|-------------| +| GET | `/v1/api/admin/ws-templates` | List all workstream templates | +| POST | `/v1/api/admin/ws-templates` | Create a workstream template | +| GET | `/v1/api/admin/ws-templates/{id}` | Get a single workstream template | +| PUT | `/v1/api/admin/ws-templates/{id}` | Update (auto-versions, audit logged) | +| DELETE | `/v1/api/admin/ws-templates/{id}` | Delete + cascade versions (audit logged) | +| GET | `/v1/api/admin/ws-templates/{id}/versions` | Version history | +| GET | `/v1/api/ws-templates` | Enabled templates summary (name, description, model) — requires write scope, not admin | + #### `GET /v1/api/auth/status` Public endpoint for login UI state detection. Returns auth configuration, not @@ -395,6 +407,7 @@ Breadcrumb: `Cluster > Running` or `Cluster > db-west-04`. Server-side paginated Triggered by the "+ new" header button. A modal dialog with: - **Node selector** — dropdown with three targeting modes: "Auto (best available)" picks the node with the most headroom, "General pool (any node)" pushes to the shared queue for any bridge to pick up, or a specific node from the list (showing capacity). +- **Profile** — optional dropdown listing enabled workstream templates. Applies the template's model, auto-approve policy, token budget, and other behavioral settings at creation time. - **Name** — optional text input. Auto-generated if left empty. - **Model** — optional text input for a model alias from the target node's registry. @@ -407,8 +420,9 @@ The browser maintains a local `clusterState` object that mirrors the cluster sna ### 5. Admin Panel Accessed via the "admin" button in the header (visible when authenticated -with `approve` scope). Provides user, API token, and channel link management -with three tabs: +with `approve` scope). Provides user, API token, channel link, and workstream +template management with 11 tabs (see also [Governance](governance.md) for +the Roles, Policies, Templates, WS Templates, Usage, and Audit tabs): **Users tab:** diff --git a/docs/diagrams/06-mq-protocol.puml b/docs/diagrams/06-mq-protocol.puml index 7da63c65..3c896582 100644 --- a/docs/diagrams/06-mq-protocol.puml +++ b/docs/diagrams/06-mq-protocol.puml @@ -59,6 +59,8 @@ package "Inbound Messages (Client → Bridge)" #FFF3E0 { + auto_approve_tools: list[str] = [] + target_node: str = "" + initial_message: str = "" + + template: str = "" + + ws_template: str = "" } class CloseWorkstreamMessage { diff --git a/docs/diagrams/14-storage-architecture.puml b/docs/diagrams/14-storage-architecture.puml index 6a0a48af..57250693 100644 --- a/docs/diagrams/14-storage-architecture.puml +++ b/docs/diagrams/14-storage-architecture.puml @@ -64,11 +64,14 @@ class "_schema.py" as Schema <> { +metadata: MetaData +memories: Table +conversations: Table - +workstreams: Table (node_id, alias, title, state) + +workstreams: Table (node_id, alias, title,\n state, ws_template_id, ws_template_version) +workstream_config: Table +users: Table (username, password_hash) +api_tokens: Table (token_hash, scopes) +channel_users: Table (channel_type) + +workstream_templates: Table (name, model,\n system_prompt, token_budget, version) + +workstream_template_versions: Table\n (template_id, version, snapshot) + +scheduled_tasks: Table (..., ws_template) -- SQLAlchemy Core Single source of truth diff --git a/docs/diagrams/19-governance-architecture.puml b/docs/diagrams/19-governance-architecture.puml index 0bc9cfd8..74191d76 100644 --- a/docs/diagrams/19-governance-architecture.puml +++ b/docs/diagrams/19-governance-architecture.puml @@ -26,6 +26,8 @@ package "Governance Storage" { database "prompt_templates" as pt_db database "usage_events" as ue_db database "audit_events" as ae_db + database "workstream_templates" as wt_db + database "workstream_template_versions" as wtv_db } package "Runtime Enforcement" { @@ -42,6 +44,13 @@ package "Template Runtime" { [set_template() / /template] as tset } +package "WS Template Runtime" { + [resolve_ws_template()] as wtr + [apply settings\n(model, budget, prompt)] as wta + [drift detection\n(prompt_template_hash)] as wtd + [budget gate\n(session.send)] as wtb +} + package "Console UI" { [Admin Panel\n10 tabs] as ui [governance.js] as govjs @@ -76,6 +85,14 @@ tload --> trender : template content trender --> tsys : rendered content tset --> tload : name or None +govjs --> wt_db : /v1/api/admin/ws-templates +wtr --> wt_db : get_ws_template_by_name() +wtr --> wta : template settings +wta --> pt_db : prompt_template lookup +wtd --> wt_db : compare hash +wtb --> approve : __budget_override__ +wtv_db <.. wt_db : version snapshots + auth -[hidden]-> mw mw -[hidden]-> approve @enduml diff --git a/docs/diagrams/21-ws-template-architecture.puml b/docs/diagrams/21-ws-template-architecture.puml new file mode 100644 index 00000000..eb454fb1 --- /dev/null +++ b/docs/diagrams/21-ws-template-architecture.puml @@ -0,0 +1,161 @@ +@startuml +!theme plain +title Turnstone — Workstream Template Architecture + +skinparam participant { + BackgroundColor<> #E8EAF6 + BackgroundColor<> #FFE0B2 + BackgroundColor<> #C8E6C9 + BackgroundColor<> #B3E5FC + BackgroundColor<> #F3E5F5 +} + +participant "Admin / Console UI\n(governance.js)" as Admin <> +participant "Server\n(server.py)" as Server <> +participant "ChatSession\n(session.py)" as Session <> +participant "StorageBackend\n(SQLite)" as Storage <> +participant "Integration Points\n(scheduler, channel,\nbridge, MQ)" as Integrations <> + +== Admin CRUD == + +Admin -> Server : POST /v1/api/admin/ws-templates +note right + **Payload:** + name, model, system_prompt, + temperature, reasoning_effort, + max_tokens, agent_max_turns, + auto_approve, auto_approve_tools, + token_budget, prompt_template, + prompt_template_hash, notify_on_complete +end note + +Server -> Storage : create_ws_template() +Storage --> Server : ws_template_id + +Admin -> Server : PUT /v1/api/admin/ws-templates/{id} +Server -> Storage : get_ws_template(id)\n(snapshot pre-update state) +Storage --> Server : existing template +Server -> Storage : create_ws_template_version()\n(version snapshot) +Server -> Storage : update_ws_template(id, ...) +note right + **Versioning:** + Each update snapshots + pre-update state into + workstream_template_versions. + version counter increments. +end note + +Admin -> Server : GET /v1/api/admin/ws-templates +Server -> Storage : list_ws_templates() + +Admin -> Server : DELETE /v1/api/admin/ws-templates/{id} +Server -> Storage : delete_ws_template(id) + +== Workstream Creation Flow == + +Integrations -> Server : CreateWorkstreamMessage\n(ws_template="production-agent") +note right + **Sources:** + - Console UI (Profile dropdown) + - Scheduler (ws_template field) + - Channel Router (ws_template) + - Bridge (ws_template forwarding) + - MQ Client (ws_template) +end note + +Server -> Storage : get_ws_template_by_name("production-agent") +Storage --> Server : template dict + +Server -> Server : resolve_ws_template()\napply model override +note right + **Settings applied:** + - model (overrides default) + - system_prompt + - temperature + - reasoning_effort + - max_tokens + - agent_max_turns + - auto_approve / auto_approve_tools + - token_budget + - tool_search config +end note + +Server -> Session : mgr.create(model=template.model, ...) +Session -> Session : _init_system_messages() + +alt template has prompt_template + Session -> Storage : get_prompt_template_by_name() + Session -> Session : _render_template()\n{{model}}, {{ws_id}}, {{node_id}} +end + +Session -> Storage : _save_config()\n+ ws_template_id, ws_template_version + +== Drift Detection == + +Server -> Server : compute prompt_template_hash\n(at creation time) +note right + **Hash stored:** + SHA-256 of prompt_template + content at ws creation time. + Compared at next creation + to detect upstream changes. +end note + +Server -> Storage : update_workstream()\n(store prompt_template_hash) + +... later, new workstream created ... + +Server -> Storage : get_ws_template() +Server -> Server : compare hash vs\ncurrent prompt_template content +alt hash mismatch + Server -> Server : log.warning(\n"prompt template drift detected") +end + +== Token Budget Enforcement == + +Session -> Session : send(message) +Session -> Session : _check_budget_gate() +note right + **Budget gate:** + if token_budget set: + total = prompt_tokens + completion_tokens + if total >= token_budget: + block further sends +end note + +alt budget exceeded + Session -> Session : approve_tools(\n__budget_override__) + note right + Model can request + budget override via + special approval label. + User must approve. + end note +else within budget + Session -> Session : continue normal flow +end + +== Storage Schema == + +note over Storage + **workstream_templates** + id, name (unique), model, system_prompt, + temperature, reasoning_effort, max_tokens, + agent_max_turns, auto_approve, auto_approve_tools, + token_budget, prompt_template, prompt_template_hash, + tool_search, tool_search_threshold, tool_search_max_results, + version, created_at, updated_at + + **workstream_template_versions** + id, template_id (FK), version, snapshot (JSON), + created_at + + **workstreams** (updated columns) + + ws_template_id: str | None + + ws_template_version: int | None + + **scheduled_tasks** (updated column) + + ws_template: str | None +end note + +@enduml diff --git a/docs/diagrams/png/06-mq-protocol.png b/docs/diagrams/png/06-mq-protocol.png index 47b5b886..6d294cde 100644 --- a/docs/diagrams/png/06-mq-protocol.png +++ b/docs/diagrams/png/06-mq-protocol.png @@ -1,3 +1,3 @@ version https://git-lfs.github.com/spec/v1 -oid sha256:d17f3feacf7bc9f64dfea19464143bc9b6ef0da5d55e6d57c0bc5a73d5724eba -size 184466 +oid sha256:2229801220548e4794baa67e27a0a39dc7968c826a28fa8144763e678c8ed733 +size 192556 diff --git a/docs/diagrams/png/14-storage-architecture.png b/docs/diagrams/png/14-storage-architecture.png index 00b0de54..2681398b 100644 --- a/docs/diagrams/png/14-storage-architecture.png +++ b/docs/diagrams/png/14-storage-architecture.png @@ -1,3 +1,3 @@ version https://git-lfs.github.com/spec/v1 -oid sha256:5faa5335152685cf1c8bf77ed93847d751cde59e1afed651e5991113f2f0f31b -size 242670 +oid sha256:fb5e7c221f6b1ee1082b37da32e65c45b5e468014cf6881a210e5b4d8a8dca8b +size 255736 diff --git a/docs/diagrams/png/19-governance-architecture.png b/docs/diagrams/png/19-governance-architecture.png index ffce2170..fe48bbe3 100644 --- a/docs/diagrams/png/19-governance-architecture.png +++ b/docs/diagrams/png/19-governance-architecture.png @@ -1,3 +1,3 @@ version https://git-lfs.github.com/spec/v1 -oid sha256:a889d4bb84c4afa3c822c3acb7021395a9463462f7aeae5382e583b783412814 -size 144960 +oid sha256:f4dac4948d928b4705936d73b4d159aa1e89315ec0397616ca914bbf19e7a1ce +size 206479 diff --git a/docs/diagrams/png/21-ws-template-architecture.png b/docs/diagrams/png/21-ws-template-architecture.png new file mode 100644 index 00000000..f032b7e1 --- /dev/null +++ b/docs/diagrams/png/21-ws-template-architecture.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c06d7086d7965eb9fe333396f027133d42507cf120bfe8dc851c009a8768ec48 +size 284926 diff --git a/docs/governance.md b/docs/governance.md index 4394080a..209d9dd6 100644 --- a/docs/governance.md +++ b/docs/governance.md @@ -79,6 +79,32 @@ Admin-curated system message templates injected at workstream startup: prevent compromised servers from injecting defaults. Admin UI shows origin badge and disables edit/delete for MCP-sourced templates. +### Workstream Templates + +Workstream templates are behavioral profiles applied at workstream creation — the next level beyond prompt templates. While prompt templates inject system message text, workstream templates define the complete workstream configuration. + +**What they define:** +- System prompt (inline text OR reference to a prompt template by name) +- Model override (empty = server default) +- Temperature, reasoning effort, max tokens, agent max turns +- Auto-approve policy (blanket and/or per-tool list) +- Token budget (0 = unlimited; warns at 80%, requires approval at 100%) +- Completion notification config (stored for v2 dispatch) + +**Storage:** `workstream_templates` table (migration 011) with auto-versioning. Edits snapshot the pre-update state into `workstream_template_versions`. Workstreams record which template and version spawned them via `ws_template_id` + `ws_template_version` columns. + +**Applied once at creation:** Template settings are snapshot-applied to the workstream's config. Not a live binding — template updates don't affect running workstreams. + +**Prompt template drift detection:** When a workstream template references a prompt template, a SHA-256 hash of the prompt content is stored at ws_template create/update time. At workstream creation, the server compares the stored hash against current content and logs a warning on mismatch. + +**Admin API:** 7 endpoints under `/v1/api/admin/ws-templates` (list, create, get, update, delete, version history) plus a read-only summary at `/v1/api/ws-templates`. Permission: `admin.ws_templates`. + +**Console UI:** "WS Templates" tab (11th admin tab) with CRUD table, create/edit modals (name, description, system prompt source toggle, model, auto-approve, per-tool auto-approve, temperature, reasoning effort, max tokens, agent max turns, token budget, enabled), and version history modal. "Profile" dropdown on workstream creation modal. "WS Template" dropdown on scheduler create/edit modals. + +**Token budget enforcement:** Tracked in `session.send()`. At 80% consumption, emits an info message. At 100%, the next turn requires explicit approval via the `__budget_override__` synthetic tool name (reuses existing approval UI — inline in browser, Discord buttons, bridge auto-approve). The synthetic name can be targeted by tool policies (e.g. `__budget_override__` → `allow` for admins). + +**SDK:** Python (`list_ws_templates`, `create_ws_template`, `get_ws_template`, `update_ws_template`, `delete_ws_template`, `list_ws_template_versions`) and TypeScript (`listWsTemplates`, `createWsTemplate`, etc.) on both sync and async console clients. `ws_template` parameter on `create_workstream()` for both server and console SDKs. + ### Usage Tracking Per-LLM-request token and tool call metrics: @@ -99,7 +125,8 @@ Append-only trail of admin actions: - **Events captured**: user.create, user.delete, token.create, token.revoke, channel.link, channel.unlink, role.create, role.update, role.delete, role.assign, role.unassign, policy.create, policy.update, policy.delete, - template.create, template.update, template.delete, org.update + template.create, template.update, template.delete, + ws_template.create, ws_template.update, ws_template.delete, org.update - **Querying**: `GET /v1/api/admin/audit` with action/user/time filters + pagination ## Database Schema @@ -130,6 +157,7 @@ All under `/v1/api/admin/` (requires `approve` scope + granular permission). | Tool Policies | 4 (CRUD) | `admin.policies` | | Prompt Templates | 4 (CRUD) | `admin.templates` | | Schedules | 6 (CRUD + runs) | `admin.schedules` | +| WS Templates | 7 (CRUD + versions + summary) | `admin.ws_templates` | | Watches | 3 (list, create, cancel) | `admin.watches` | | Usage | 1 (aggregated query) | `admin.usage` | | Audit | 1 (paginated, filtered) | `admin.audit` | @@ -138,11 +166,12 @@ Full OpenAPI spec at `/openapi.json` and Swagger UI at `/docs`. ## Admin Console UI -5 new tabs added to the admin panel (10 total): +6 new tabs added to the admin panel (11 total): - **Roles** — CRUD roles, permission checkbox grid, user role assignment modal - **Policies** — CRUD tool policies with colored action badges (green/red/amber) - **Templates** — CRUD prompt templates with wide modal, textarea editor +- **WS Templates** — CRUD workstream templates with create/edit modals, version history - **Usage** — Summary readouts + CSS bar chart, time range + group-by selectors - **Audit** — Filterable log with relative timestamps, load-more pagination @@ -158,6 +187,7 @@ Both Python and TypeScript console SDKs expose governance methods: - `list_orgs()`, `get_org()`, `update_org()` - `list_policies()`, `create_policy()`, `update_policy()`, `delete_policy()` - `list_templates()`, `create_template()`, `update_template()`, `delete_template()` +- `list_ws_templates()`, `create_ws_template()`, `get_ws_template()`, `update_ws_template()`, `delete_ws_template()`, `list_ws_template_versions()` - `get_usage(since, group_by=...)`, `get_audit(action=..., limit=...)` **TypeScript** (`TurnstoneConsole`): diff --git a/docs/sdk.md b/docs/sdk.md index 6d9f3a67..78a5caa2 100644 --- a/docs/sdk.md +++ b/docs/sdk.md @@ -69,7 +69,7 @@ Both `TurnstoneServer` (sync) and `AsyncTurnstoneServer` (async) expose: |----------|--------|---------| | **Workstreams** | `list_workstreams()` | `ListWorkstreamsResponse` | | | `dashboard()` | `DashboardResponse` | -| | `create_workstream(*, name, model, auto_approve)` | `CreateWorkstreamResponse` | +| | `create_workstream(*, name, model, auto_approve, ws_template)` | `CreateWorkstreamResponse` | | | `close_workstream(ws_id)` | `StatusResponse` | | **Chat** | `send(message, ws_id)` | `SendResponse` | | | `approve(*, ws_id, approved, feedback, always)` | `StatusResponse` | @@ -97,13 +97,19 @@ Both `TurnstoneConsole` (sync) and `AsyncTurnstoneConsole` (async) expose: | | `workstreams(*, state, node, search, sort, page, per_page)` | `ClusterWorkstreamsResponse` | | | `node_detail(node_id)` | `NodeDetailResponse` | | | `snapshot()` | `ClusterSnapshotResponse` | -| | `create_workstream(*, node_id, name, model, initial_message)` | `ConsoleCreateWsResponse` | +| | `create_workstream(*, node_id, name, model, initial_message, ws_template)` | `ConsoleCreateWsResponse` | | **Schedules** | `list_schedules()` | `ListSchedulesResponse` | | | `create_schedule(*, name, schedule_type, initial_message, ...)` | `ScheduleInfo` | | | `get_schedule(task_id)` | `ScheduleInfo` | | | `update_schedule(task_id, *, name=..., enabled=..., ...)` | `ScheduleInfo` | | | `delete_schedule(task_id)` | `StatusResponse` | | | `list_schedule_runs(task_id, *, limit=50)` | `ListScheduleRunsResponse` | +| **WS Templates** | `list_ws_templates()` | `ListWsTemplatesResponse` | +| | `create_ws_template(*, name, description, ...)` | `WsTemplateInfo` | +| | `get_ws_template(template_id)` | `WsTemplateInfo` | +| | `update_ws_template(template_id, *, name=..., enabled=..., ...)` | `WsTemplateInfo` | +| | `delete_ws_template(template_id)` | `StatusResponse` | +| | `list_ws_template_versions(template_id)` | `ListWsTemplateVersionsResponse` | | **Streaming** | `stream_cluster_events()` | `Iterator[ClusterEvent]` | | **Auth** | `login(username=..., password=...)` / `login(token="ts_xxx")` | `AuthLoginResponse` | | | `logout()` | `StatusResponse` | diff --git a/sdk/typescript/openapi-console.json b/sdk/typescript/openapi-console.json index 90137432..ce413d0c 100644 --- a/sdk/typescript/openapi-console.json +++ b/sdk/typescript/openapi-console.json @@ -755,6 +755,12 @@ "description": "Prompt template name (replaces default templates)", "title": "Template", "type": "string" + }, + "ws_template": { + "default": "", + "description": "Workstream template name (behavioral profile applied at creation)", + "title": "Ws Template", + "type": "string" } }, "title": "ConsoleCreateWsRequest", diff --git a/sdk/typescript/openapi-server.json b/sdk/typescript/openapi-server.json index 2dba8b73..5d61367b 100644 --- a/sdk/typescript/openapi-server.json +++ b/sdk/typescript/openapi-server.json @@ -883,6 +883,12 @@ "description": "Prompt template name (replaces default templates)", "title": "Template", "type": "string" + }, + "ws_template": { + "default": "", + "description": "Workstream template name (behavioral profile applied at creation)", + "title": "Ws Template", + "type": "string" } }, "title": "CreateWorkstreamRequest", diff --git a/sdk/typescript/src/console.ts b/sdk/typescript/src/console.ts index 656c49aa..b087878f 100644 --- a/sdk/typescript/src/console.ts +++ b/sdk/typescript/src/console.ts @@ -17,6 +17,7 @@ import type { CreateRoleOptions, CreateScheduleRequest, CreateTemplateOptions, + CreateWsTemplateOptions, ListScheduleRunsResponse, ListSchedulesResponse, NodeDetailResponse, @@ -32,10 +33,13 @@ import type { UpdateRoleOptions, UpdateScheduleRequest, UpdateTemplateOptions, + UpdateWsTemplateOptions, UsageQueryOptions, UsageResponse, UserRoleInfo, WorkstreamsOptions, + WsTemplateInfo, + WsTemplateVersionInfo, } from "./types.js"; /** Async client for the turnstone console API. */ @@ -273,6 +277,51 @@ export class TurnstoneConsole extends BaseClient { return this.request("DELETE", `/v1/api/admin/templates/${templateId}`); } + // -- Governance: Workstream Templates ---------------------------------------- + + async listWsTemplates(): Promise { + const data = await this.request<{ ws_templates: WsTemplateInfo[] }>( + "GET", + "/v1/api/admin/ws-templates", + ); + return data.ws_templates || []; + } + + async createWsTemplate( + opts: CreateWsTemplateOptions, + ): Promise { + return this.request("POST", "/v1/api/admin/ws-templates", { + json: opts, + }); + } + + async getWsTemplate(wsTemplateId: string): Promise { + return this.request("GET", `/v1/api/admin/ws-templates/${wsTemplateId}`); + } + + async updateWsTemplate( + wsTemplateId: string, + opts: UpdateWsTemplateOptions, + ): Promise { + return this.request("PUT", `/v1/api/admin/ws-templates/${wsTemplateId}`, { + json: opts, + }); + } + + async deleteWsTemplate(wsTemplateId: string): Promise { + await this.request("DELETE", `/v1/api/admin/ws-templates/${wsTemplateId}`); + } + + async listWsTemplateVersions( + wsTemplateId: string, + ): Promise { + const data = await this.request<{ versions: WsTemplateVersionInfo[] }>( + "GET", + `/v1/api/admin/ws-templates/${wsTemplateId}/versions`, + ); + return data.versions || []; + } + // -- Governance: Usage & Audit ---------------------------------------------- async getUsage(opts: UsageQueryOptions): Promise { diff --git a/sdk/typescript/src/index.ts b/sdk/typescript/src/index.ts index 7fa83c73..11a5fedb 100644 --- a/sdk/typescript/src/index.ts +++ b/sdk/typescript/src/index.ts @@ -129,6 +129,10 @@ export type { PromptTemplateInfo, CreateTemplateOptions, UpdateTemplateOptions, + WsTemplateInfo, + CreateWsTemplateOptions, + UpdateWsTemplateOptions, + WsTemplateVersionInfo, UsageBreakdownItem, UsageResponse, UsageQueryOptions, diff --git a/sdk/typescript/src/types.ts b/sdk/typescript/src/types.ts index 816971d0..3c023605 100644 --- a/sdk/typescript/src/types.ts +++ b/sdk/typescript/src/types.ts @@ -73,6 +73,7 @@ export interface CreateWorkstreamRequest { auto_approve?: boolean; resume_ws?: string; template?: string; + ws_template?: string; } export interface CreateWorkstreamResponse { @@ -275,6 +276,7 @@ export interface ConsoleCreateWsRequest { model?: string; initial_message?: string; template?: string; + ws_template?: string; } export interface ConsoleCreateWsResponse { @@ -483,6 +485,78 @@ export interface UpdateTemplateOptions { is_default?: boolean; } +// --------------------------------------------------------------------------- +// Console API — Governance: Workstream Templates +// --------------------------------------------------------------------------- + +export interface WsTemplateInfo { + ws_template_id: string; + name: string; + description: string; + system_prompt: string; + prompt_template: string; + prompt_template_hash: string; + model: string; + auto_approve: boolean; + auto_approve_tools: string; + temperature: number | null; + reasoning_effort: string; + max_tokens: number | null; + token_budget: number; + agent_max_turns: number | null; + notify_on_complete: string; + org_id: string; + created_by: string; + enabled: boolean; + version: number; + created: string; + updated: string; +} + +export interface CreateWsTemplateOptions { + name: string; + description?: string; + system_prompt?: string; + prompt_template?: string; + model?: string; + auto_approve?: boolean; + auto_approve_tools?: string; + temperature?: number | null; + reasoning_effort?: string; + max_tokens?: number | null; + token_budget?: number; + agent_max_turns?: number | null; + notify_on_complete?: string; + org_id?: string; + enabled?: boolean; +} + +export interface UpdateWsTemplateOptions { + name?: string; + description?: string; + system_prompt?: string; + prompt_template?: string; + model?: string; + auto_approve?: boolean; + auto_approve_tools?: string; + temperature?: number | null; + reasoning_effort?: string; + max_tokens?: number | null; + token_budget?: number; + agent_max_turns?: number | null; + notify_on_complete?: string; + enabled?: boolean; +} + +export interface WsTemplateVersionInfo { + id: number; + ws_template_id: string; + version: number; + snapshot: string; + changed_by: string; + created: string; +} + // --------------------------------------------------------------------------- // Console API — Governance: Usage & Audit // --------------------------------------------------------------------------- diff --git a/tests/test_ws_template_runtime.py b/tests/test_ws_template_runtime.py new file mode 100644 index 00000000..21ee5a2f --- /dev/null +++ b/tests/test_ws_template_runtime.py @@ -0,0 +1,374 @@ +"""Tests for workstream template runtime — template application, token budget, config persistence.""" + +from __future__ import annotations + +from unittest.mock import MagicMock, patch + +from turnstone.core.session import ChatSession +from turnstone.mq.protocol import CreateWorkstreamMessage +from turnstone.server import WebUI + +# --------------------------------------------------------------------------- +# Helpers +# --------------------------------------------------------------------------- + + +class NullUI: + """UI adapter that discards all output.""" + + def on_thinking_start(self): + pass + + def on_thinking_stop(self): + pass + + def on_reasoning_token(self, text): + pass + + def on_content_token(self, text): + pass + + def on_stream_end(self): + pass + + def approve_tools(self, items): + return True, None + + def on_tool_result(self, call_id, name, output): + pass + + def on_tool_output_chunk(self, call_id, chunk): + pass + + def on_status(self, usage, context_window, effort): + pass + + def on_plan_review(self, content): + return "" + + def on_info(self, message): + pass + + def on_error(self, message): + pass + + def on_state_change(self, state): + pass + + def on_rename(self, name): + pass + + +def _make_session(ui=None, **kwargs): + defaults = dict( + client=MagicMock(), + model="test-model", + ui=ui or NullUI(), + instructions=None, + temperature=0.5, + max_tokens=4096, + tool_timeout=30, + ) + defaults.update(kwargs) + return ChatSession(**defaults) + + +# --------------------------------------------------------------------------- +# Template application — defaults and constructor +# --------------------------------------------------------------------------- + + +def test_session_token_budget_default_zero(tmp_db): + session = _make_session() + assert session._token_budget == 0 + + +def test_session_save_config_includes_ws_template_fields(tmp_db): + session = _make_session() + session._token_budget = 50000 + session._ws_template_id = "tpl-abc" + session._ws_template_version = 3 + session._notify_on_complete = '{"url": "http://example.com"}' + session._save_config() + + from turnstone.core.memory import load_workstream_config + + config = load_workstream_config(session._ws_id) + assert config["token_budget"] == "50000" + assert config["ws_template_id"] == "tpl-abc" + assert config["ws_template_version"] == "3" + assert config["notify_on_complete"] == '{"url": "http://example.com"}' + + +def test_session_resume_restores_token_budget(tmp_db): + s1 = _make_session() + s1._token_budget = 100000 + s1._save_config() + # Seed at least one message so resume can load the workstream + s1.messages.append({"role": "user", "content": "hello"}) + from turnstone.core.memory import save_message + + save_message(s1._ws_id, "user", "hello") + + s2 = _make_session() + assert s2.resume(s1._ws_id) + assert s2._token_budget == 100000 + + +def test_session_resume_restores_ws_template_id(tmp_db): + s1 = _make_session() + s1._ws_template_id = "tpl-xyz" + s1._save_config() + from turnstone.core.memory import save_message + + save_message(s1._ws_id, "user", "ping") + + s2 = _make_session() + assert s2.resume(s1._ws_id) + assert s2._ws_template_id == "tpl-xyz" + + +def test_session_resume_restores_ws_template_version(tmp_db): + s1 = _make_session() + s1._ws_template_version = 7 + s1._save_config() + from turnstone.core.memory import save_message + + save_message(s1._ws_id, "user", "ping") + + s2 = _make_session() + assert s2.resume(s1._ws_id) + assert s2._ws_template_version == 7 + + +def test_session_resume_restores_notify_on_complete(tmp_db): + s1 = _make_session() + s1._notify_on_complete = '{"channel": "#ops"}' + s1._save_config() + from turnstone.core.memory import save_message + + save_message(s1._ws_id, "user", "ping") + + s2 = _make_session() + assert s2.resume(s1._ws_id) + assert s2._notify_on_complete == '{"channel": "#ops"}' + + +# --------------------------------------------------------------------------- +# Token budget tracking +# --------------------------------------------------------------------------- + + +def test_budget_warning_at_80_percent(tmp_db): + ui = MagicMock(spec_set=NullUI) + ui.approve_tools.return_value = (True, None) + session = _make_session(ui=ui) + session._token_budget = 10000 + # Simulate usage at 80% of budget + session._last_usage = {"prompt_tokens": 7500, "completion_tokens": 500} + session._update_token_table({"role": "assistant", "content": "hi"}) + assert session._budget_warned is True + ui.on_info.assert_called_once() + assert "80%" in ui.on_info.call_args[0][0] + + +def test_budget_exhausted_at_100_percent(tmp_db): + ui = MagicMock(spec_set=NullUI) + ui.approve_tools.return_value = (True, None) + session = _make_session(ui=ui) + session._token_budget = 10000 + session._last_usage = {"prompt_tokens": 9000, "completion_tokens": 1500} + session._update_token_table({"role": "assistant", "content": "hi"}) + assert session._budget_exhausted is True + + +def test_budget_zero_no_tracking(tmp_db): + ui = MagicMock(spec_set=NullUI) + ui.approve_tools.return_value = (True, None) + session = _make_session(ui=ui) + assert session._token_budget == 0 + session._last_usage = {"prompt_tokens": 999999, "completion_tokens": 999999} + session._update_token_table({"role": "assistant", "content": "hi"}) + assert session._budget_warned is False + assert session._budget_exhausted is False + ui.on_info.assert_not_called() + + +def test_budget_warning_only_once(tmp_db): + ui = MagicMock(spec_set=NullUI) + ui.approve_tools.return_value = (True, None) + session = _make_session(ui=ui) + session._token_budget = 10000 + # First call at 80% + session._last_usage = {"prompt_tokens": 7500, "completion_tokens": 500} + session._update_token_table({"role": "assistant", "content": "a"}) + assert session._budget_warned is True + assert ui.on_info.call_count == 1 + # Second call still above 80% — should not warn again + session._last_usage = {"prompt_tokens": 8500, "completion_tokens": 500} + session._update_token_table({"role": "assistant", "content": "b"}) + assert session._budget_warned is True + assert ui.on_info.call_count == 1 + + +# --------------------------------------------------------------------------- +# Token budget approval gate in send() +# --------------------------------------------------------------------------- + + +def test_send_blocked_when_budget_exhausted(tmp_db): + ui = MagicMock(spec_set=NullUI) + ui.approve_tools.return_value = (False, None) + session = _make_session(ui=ui) + session._budget_exhausted = True + session._token_budget = 5000 + session.send("hello") + # approve_tools should have been called with __budget_override__ + ui.approve_tools.assert_called_once() + items = ui.approve_tools.call_args[0][0] + assert len(items) == 1 + assert items[0]["func_name"] == "__budget_override__" + assert "5,000" in items[0]["preview"] + # on_error should have been called since approval was denied + ui.on_error.assert_called_once() + assert "budget" in ui.on_error.call_args[0][0].lower() + + +def test_send_continues_after_budget_approval(tmp_db): + ui = MagicMock(spec_set=NullUI) + ui.approve_tools.return_value = (True, None) + session = _make_session(ui=ui) + session._budget_exhausted = True + session._budget_warned = True + session._token_budget = 5000 + + # Patch _create_stream_with_retry to avoid actual LLM call + with ( + patch.object(session, "_create_stream_with_retry"), + patch.object(session, "_stream_response") as mock_resp, + patch.object(session, "_update_token_table"), + patch.object(session, "_print_status_line"), + ): + mock_resp.return_value = {"role": "assistant", "content": "ok", "tool_calls": []} + session.send("hello") + + # Budget flags should be reset + assert session._budget_exhausted is False + assert session._budget_warned is False + # approve_tools was called for budget gate + ui.approve_tools.assert_called_once() + + +def test_send_returns_when_budget_denied(tmp_db): + ui = MagicMock(spec_set=NullUI) + ui.approve_tools.return_value = (False, None) + session = _make_session(ui=ui) + session._budget_exhausted = True + session._token_budget = 5000 + + # Patch to detect if _create_stream_with_retry is called (it shouldn't be) + with patch.object(session, "_create_stream_with_retry") as mock_stream: + session.send("hello") + mock_stream.assert_not_called() + + # Message should NOT have been appended + assert len(session.messages) == 0 + + +# --------------------------------------------------------------------------- +# WebUI auto_approve_tools +# --------------------------------------------------------------------------- + + +def test_webui_auto_approve_tools_default_empty(): + webui = WebUI(ws_id="ws-1") + assert webui.auto_approve_tools == set() + + +def test_webui_auto_approve_tools_subset_approves(): + webui = WebUI(ws_id="ws-1") + webui.auto_approve_tools = {"bash", "read_file", "write_file"} + items = [ + {"func_name": "bash", "preview": "ls", "needs_approval": True}, + {"func_name": "read_file", "preview": "/tmp/x", "needs_approval": True}, + ] + # Patch out policy evaluation and global queue to isolate auto_approve_tools + with patch("turnstone.server.WebUI._global_queue", None): + approved, _ = webui.approve_tools(items) + assert approved is True + + +def test_webui_auto_approve_tools_partial_no_approve(): + webui = WebUI(ws_id="ws-1") + webui.auto_approve_tools = {"bash"} + items = [ + {"func_name": "bash", "preview": "ls", "needs_approval": True}, + {"func_name": "write_file", "preview": "/tmp/x", "needs_approval": True}, + ] + # write_file is NOT in auto_approve_tools, so it won't auto-approve. + # The method will block on _approval_event, so we set it immediately. + webui._approval_event = MagicMock() + webui._approval_event.wait.return_value = None + webui._approval_result = (False, None) + with patch("turnstone.server.WebUI._global_queue", None): + approved, _ = webui.approve_tools(items) + assert approved is False + + +def test_webui_auto_approve_tools_empty_no_effect(): + webui = WebUI(ws_id="ws-1") + webui.auto_approve_tools = set() + items = [ + {"func_name": "bash", "preview": "ls", "needs_approval": True}, + ] + # Empty set should not auto-approve; must wait for manual approval. + webui._approval_event = MagicMock() + webui._approval_event.wait.return_value = None + webui._approval_result = (True, None) + with patch("turnstone.server.WebUI._global_queue", None): + approved, _ = webui.approve_tools(items) + # Approval comes from the manual path (we set _approval_result to True) + assert approved is True + # The approval event wait should have been called (manual approval path) + webui._approval_event.wait.assert_called_once() + + +# --------------------------------------------------------------------------- +# Protocol round-trip — CreateWorkstreamMessage +# --------------------------------------------------------------------------- + + +def test_create_workstream_message_ws_template(): + msg = CreateWorkstreamMessage(ws_template="deploy-v2") + assert msg.ws_template == "deploy-v2" + assert msg.type == "create_workstream" + + +def test_create_workstream_message_ws_template_default(): + msg = CreateWorkstreamMessage() + assert msg.ws_template == "" + + +# --------------------------------------------------------------------------- +# Config persistence round-trip +# --------------------------------------------------------------------------- + + +def test_save_config_round_trip(tmp_db): + s1 = _make_session() + s1._token_budget = 75000 + s1._ws_template_id = "tpl-roundtrip" + s1._ws_template_version = 12 + s1._notify_on_complete = '{"webhook": "https://hooks.example.com/done"}' + s1._save_config() + + from turnstone.core.memory import save_message + + save_message(s1._ws_id, "user", "test") + + s2 = _make_session() + assert s2.resume(s1._ws_id) + assert s2._token_budget == 75000 + assert s2._ws_template_id == "tpl-roundtrip" + assert s2._ws_template_version == 12 + assert s2._notify_on_complete == '{"webhook": "https://hooks.example.com/done"}' diff --git a/tests/test_ws_template_storage.py b/tests/test_ws_template_storage.py new file mode 100644 index 00000000..456939bb --- /dev/null +++ b/tests/test_ws_template_storage.py @@ -0,0 +1,329 @@ +"""Tests for workstream template storage CRUD operations.""" + +from __future__ import annotations + +import json + +import pytest +import sqlalchemy as sa +from sqlalchemy.exc import IntegrityError + +from turnstone.core.storage._schema import workstreams +from turnstone.core.storage._sqlite import SQLiteBackend + + +@pytest.fixture() +def db(tmp_path): + return SQLiteBackend(str(tmp_path / "test.db")) + + +def _make_template_kwargs(**overrides): + defaults = { + "ws_template_id": "tpl_001", + "name": "research-agent", + "description": "Deep research profile", + "system_prompt": "You are a research assistant.", + "prompt_template": "tpl-greeting", + "model": "gpt-5", + "auto_approve": False, + "auto_approve_tools": "read_file,write_file", + "temperature": 0.7, + "reasoning_effort": "medium", + "max_tokens": 4096, + "token_budget": 100000, + "agent_max_turns": 10, + "notify_on_complete": '{"webhook":"https://example.com"}', + "org_id": "org1", + "created_by": "admin", + "enabled": True, + } + defaults.update(overrides) + return defaults + + +# --------------------------------------------------------------------------- +# CRUD Operations +# --------------------------------------------------------------------------- + + +class TestWsTemplateCRUD: + def test_create_ws_template(self, db): + db.create_ws_template(**_make_template_kwargs()) + tpl = db.get_ws_template("tpl_001") + assert tpl is not None + assert tpl["ws_template_id"] == "tpl_001" + assert tpl["name"] == "research-agent" + + def test_create_ws_template_fields(self, db): + db.create_ws_template(**_make_template_kwargs()) + tpl = db.get_ws_template("tpl_001") + assert tpl is not None + assert tpl["description"] == "Deep research profile" + assert tpl["system_prompt"] == "You are a research assistant." + assert tpl["prompt_template"] == "tpl-greeting" + assert tpl["model"] == "gpt-5" + assert tpl["auto_approve"] is False + assert isinstance(tpl["auto_approve"], bool) + assert tpl["auto_approve_tools"] == "read_file,write_file" + assert tpl["temperature"] == 0.7 + assert tpl["reasoning_effort"] == "medium" + assert tpl["max_tokens"] == 4096 + assert tpl["token_budget"] == 100000 + assert tpl["agent_max_turns"] == 10 + assert tpl["notify_on_complete"] == '{"webhook":"https://example.com"}' + assert tpl["org_id"] == "org1" + assert tpl["created_by"] == "admin" + assert tpl["enabled"] is True + assert isinstance(tpl["enabled"], bool) + assert tpl["version"] == 1 + assert "created" in tpl + assert "updated" in tpl + + def test_get_ws_template_not_found(self, db): + assert db.get_ws_template("nonexistent") is None + + def test_get_ws_template_by_name(self, db): + db.create_ws_template(**_make_template_kwargs()) + tpl = db.get_ws_template_by_name("research-agent") + assert tpl is not None + assert tpl["ws_template_id"] == "tpl_001" + assert tpl["auto_approve"] is False + assert isinstance(tpl["auto_approve"], bool) + assert tpl["enabled"] is True + assert isinstance(tpl["enabled"], bool) + + def test_get_ws_template_by_name_not_found(self, db): + assert db.get_ws_template_by_name("nope") is None + + def test_list_ws_templates(self, db): + db.create_ws_template(**_make_template_kwargs(ws_template_id="t2", name="beta")) + db.create_ws_template(**_make_template_kwargs(ws_template_id="t1", name="alpha")) + templates = db.list_ws_templates() + assert len(templates) == 2 + assert templates[0]["name"] == "alpha" + assert templates[1]["name"] == "beta" + + def test_list_ws_templates_empty(self, db): + assert db.list_ws_templates() == [] + + def test_list_ws_templates_enabled_only(self, db): + db.create_ws_template( + **_make_template_kwargs(ws_template_id="t1", name="active", enabled=True) + ) + db.create_ws_template( + **_make_template_kwargs(ws_template_id="t2", name="disabled", enabled=False) + ) + result = db.list_ws_templates(enabled_only=True) + assert len(result) == 1 + assert result[0]["name"] == "active" + + def test_list_ws_templates_org_filter(self, db): + db.create_ws_template(**_make_template_kwargs(ws_template_id="t1", name="a", org_id="org1")) + db.create_ws_template(**_make_template_kwargs(ws_template_id="t2", name="b", org_id="org2")) + db.create_ws_template(**_make_template_kwargs(ws_template_id="t3", name="c", org_id="org1")) + result = db.list_ws_templates(org_id="org1") + assert len(result) == 2 + assert {r["ws_template_id"] for r in result} == {"t1", "t3"} + + def test_update_ws_template(self, db): + db.create_ws_template(**_make_template_kwargs()) + ok = db.update_ws_template("tpl_001", name="updated-agent", description="New desc") + assert ok is True + tpl = db.get_ws_template("tpl_001") + assert tpl is not None + assert tpl["name"] == "updated-agent" + assert tpl["description"] == "New desc" + + def test_update_ws_template_not_found(self, db): + assert db.update_ws_template("missing", name="x") is False + + def test_update_ws_template_ignores_unknown_fields(self, db): + db.create_ws_template(**_make_template_kwargs()) + ok = db.update_ws_template("tpl_001", name="new-name", org_id="hack", created_by="hack") + assert ok is True + tpl = db.get_ws_template("tpl_001") + assert tpl is not None + assert tpl["name"] == "new-name" + # Non-mutable fields unchanged. + assert tpl["org_id"] == "org1" + assert tpl["created_by"] == "admin" + + def test_update_ws_template_boolean_normalization(self, db): + db.create_ws_template(**_make_template_kwargs()) + db.update_ws_template("tpl_001", auto_approve=True, enabled=False) + tpl = db.get_ws_template("tpl_001") + assert tpl is not None + assert tpl["auto_approve"] is True + assert isinstance(tpl["auto_approve"], bool) + assert tpl["enabled"] is False + assert isinstance(tpl["enabled"], bool) + + def test_delete_ws_template(self, db): + db.create_ws_template(**_make_template_kwargs()) + ok = db.delete_ws_template("tpl_001") + assert ok is True + assert db.get_ws_template("tpl_001") is None + + def test_delete_ws_template_not_found(self, db): + assert db.delete_ws_template("missing") is False + + def test_delete_ws_template_cascades_versions(self, db): + db.create_ws_template(**_make_template_kwargs()) + # Create a version snapshot via update. + db.update_ws_template("tpl_001", name="v2-name") + versions = db.list_ws_template_versions("tpl_001") + assert len(versions) == 1 + # Delete template — versions should be gone too. + db.delete_ws_template("tpl_001") + assert db.list_ws_template_versions("tpl_001") == [] + + def test_create_ws_template_with_hash(self, db): + db.create_ws_template( + ws_template_id="tpl_hash", + name="hashed-template", + prompt_template="my-prompt", + prompt_template_hash="abc123hash", + ) + tpl = db.get_ws_template("tpl_hash") + assert tpl["prompt_template_hash"] == "abc123hash" + + def test_update_ws_template_hash(self, db): + db.create_ws_template(**_make_template_kwargs()) + db.update_ws_template("tpl_001", prompt_template_hash="newhash456") + tpl = db.get_ws_template("tpl_001") + assert tpl["prompt_template_hash"] == "newhash456" + + def test_create_duplicate_name(self, db): + db.create_ws_template(**_make_template_kwargs(ws_template_id="t1", name="unique")) + with pytest.raises(IntegrityError): + db.create_ws_template(**_make_template_kwargs(ws_template_id="t2", name="unique")) + + +# --------------------------------------------------------------------------- +# Versioning +# --------------------------------------------------------------------------- + + +class TestWsTemplateVersioning: + def test_update_creates_version_snapshot(self, db): + db.create_ws_template(**_make_template_kwargs()) + db.update_ws_template("tpl_001", description="Changed") + versions = db.list_ws_template_versions("tpl_001") + assert len(versions) == 1 + assert versions[0]["ws_template_id"] == "tpl_001" + assert versions[0]["version"] == 1 + + def test_version_increments_on_update(self, db): + db.create_ws_template(**_make_template_kwargs()) + tpl = db.get_ws_template("tpl_001") + assert tpl is not None + assert tpl["version"] == 1 + db.update_ws_template("tpl_001", description="v2") + tpl = db.get_ws_template("tpl_001") + assert tpl is not None + assert tpl["version"] == 2 + db.update_ws_template("tpl_001", description="v3") + tpl = db.get_ws_template("tpl_001") + assert tpl is not None + assert tpl["version"] == 3 + + def test_version_snapshot_contains_json(self, db): + db.create_ws_template(**_make_template_kwargs()) + db.update_ws_template("tpl_001", description="Changed") + versions = db.list_ws_template_versions("tpl_001") + snapshot = json.loads(versions[0]["snapshot"]) + # Snapshot should contain the pre-update state. + assert snapshot["description"] == "Deep research profile" + assert snapshot["name"] == "research-agent" + assert snapshot["version"] == 1 + + def test_multiple_updates_create_versions(self, db): + db.create_ws_template(**_make_template_kwargs()) + db.update_ws_template("tpl_001", description="Second") + db.update_ws_template("tpl_001", description="Third") + db.update_ws_template("tpl_001", description="Fourth") + versions = db.list_ws_template_versions("tpl_001") + assert len(versions) == 3 + # Ordered by version DESC. + assert versions[0]["version"] == 3 + assert versions[1]["version"] == 2 + assert versions[2]["version"] == 1 + + def test_list_ws_template_versions(self, db): + db.create_ws_template(**_make_template_kwargs()) + db.update_ws_template("tpl_001", description="v2") + db.update_ws_template("tpl_001", description="v3") + versions = db.list_ws_template_versions("tpl_001") + assert len(versions) == 2 + # Ordered by version DESC. + assert versions[0]["version"] == 2 + assert versions[1]["version"] == 1 + for v in versions: + assert "created" in v + assert "snapshot" in v + assert "changed_by" in v + + def test_list_ws_template_versions_empty(self, db): + assert db.list_ws_template_versions("nonexistent") == [] + + def test_create_ws_template_version_direct(self, db): + db.create_ws_template(**_make_template_kwargs()) + snapshot_data = json.dumps({"name": "manual-snapshot", "version": 99}) + db.create_ws_template_version( + "tpl_001", version=99, snapshot=snapshot_data, changed_by="admin" + ) + versions = db.list_ws_template_versions("tpl_001") + assert len(versions) == 1 + assert versions[0]["version"] == 99 + assert versions[0]["changed_by"] == "admin" + parsed = json.loads(versions[0]["snapshot"]) + assert parsed["name"] == "manual-snapshot" + + +# --------------------------------------------------------------------------- +# Workstream Integration +# --------------------------------------------------------------------------- + + +class TestWsTemplateWorkstreamIntegration: + def test_register_workstream_with_template(self, db): + db.create_ws_template(**_make_template_kwargs()) + db.register_workstream( + ws_id="ws-001", + node_id="node-1", + name="test-ws", + ws_template_id="tpl_001", + ws_template_version=1, + ) + # Verify via direct query — list_workstreams doesn't select template fields. + with db._engine.connect() as conn: + row = conn.execute( + sa.select(workstreams.c.ws_template_id, workstreams.c.ws_template_version).where( + workstreams.c.ws_id == "ws-001" + ) + ).fetchone() + assert row is not None + assert row[0] == "tpl_001" + assert row[1] == 1 + + def test_update_workstream_template(self, db): + db.register_workstream(ws_id="ws-002", node_id="node-1", name="test-ws") + # Initially defaults + with db._engine.connect() as conn: + row = conn.execute( + sa.select(workstreams.c.ws_template_id, workstreams.c.ws_template_version).where( + workstreams.c.ws_id == "ws-002" + ) + ).fetchone() + assert row[0] == "" + assert row[1] == 0 + # Update template lineage + db.update_workstream_template("ws-002", "tpl_abc", 3) + with db._engine.connect() as conn: + row = conn.execute( + sa.select(workstreams.c.ws_template_id, workstreams.c.ws_template_version).where( + workstreams.c.ws_id == "ws-002" + ) + ).fetchone() + assert row[0] == "tpl_abc" + assert row[1] == 3 diff --git a/turnstone/api/console_schemas.py b/turnstone/api/console_schemas.py index 229e3097..81d45fb8 100644 --- a/turnstone/api/console_schemas.py +++ b/turnstone/api/console_schemas.py @@ -139,6 +139,9 @@ class ConsoleCreateWsRequest(BaseModel): template: str = Field( default="", description="Prompt template name (replaces default templates)" ) + ws_template: str = Field( + default="", description="Workstream template name (behavioral profile)" + ) class ConsoleCreateWsResponse(BaseModel): @@ -317,6 +320,97 @@ class ListPromptTemplatesResponse(BaseModel): templates: list[PromptTemplateInfo] +# --------------------------------------------------------------------------- +# Governance: Workstream Templates +# --------------------------------------------------------------------------- + + +class WsTemplateInfo(BaseModel): + ws_template_id: str + name: str + description: str + system_prompt: str + prompt_template: str + prompt_template_hash: str = "" + model: str + auto_approve: bool + auto_approve_tools: str + temperature: float | None = None + reasoning_effort: str + max_tokens: int | None = None + token_budget: int + agent_max_turns: int | None = None + notify_on_complete: str + org_id: str + created_by: str + enabled: bool + version: int + created: str + updated: str + + +class CreateWsTemplateRequest(BaseModel): + name: str + description: str = "" + system_prompt: str = "" + prompt_template: str = "" + model: str = "" + auto_approve: bool = False + auto_approve_tools: str = "" + temperature: float | None = None + reasoning_effort: str = "" + max_tokens: int | None = None + token_budget: int = 0 + agent_max_turns: int | None = None + notify_on_complete: str = "{}" + org_id: str = "" + enabled: bool = True + + +class UpdateWsTemplateRequest(BaseModel): + name: str | None = None + description: str | None = None + system_prompt: str | None = None + prompt_template: str | None = None + model: str | None = None + auto_approve: bool | None = None + auto_approve_tools: str | None = None + temperature: float | None = None + reasoning_effort: str | None = None + max_tokens: int | None = None + token_budget: int | None = None + agent_max_turns: int | None = None + notify_on_complete: str | None = None + enabled: bool | None = None + + +class ListWsTemplatesResponse(BaseModel): + ws_templates: list[WsTemplateInfo] + + +class WsTemplateVersionInfo(BaseModel): + id: int + ws_template_id: str + version: int + snapshot: str + changed_by: str + created: str + + +class ListWsTemplateVersionsResponse(BaseModel): + versions: list[WsTemplateVersionInfo] + + +class WsTemplateSummary(BaseModel): + name: str + description: str + model: str + + +class ListWsTemplateSummaryResponse(BaseModel): + ws_templates: list[WsTemplateSummary] + + # --------------------------------------------------------------------------- # Governance: Usage # --------------------------------------------------------------------------- diff --git a/turnstone/api/console_spec.py b/turnstone/api/console_spec.py index 49eaec7a..dc994d46 100644 --- a/turnstone/api/console_spec.py +++ b/turnstone/api/console_spec.py @@ -22,6 +22,7 @@ from turnstone.api.console_schemas import ( CreatePromptTemplateRequest, CreateRoleRequest, CreateToolPolicyRequest, + CreateWsTemplateRequest, ListAuditEventsResponse, ListChannelUsersResponse, ListOrgsResponse, @@ -29,6 +30,9 @@ from turnstone.api.console_schemas import ( ListRolesResponse, ListToolPoliciesResponse, ListUserRolesResponse, + ListWsTemplatesResponse, + ListWsTemplateSummaryResponse, + ListWsTemplateVersionsResponse, NodeDetailResponse, OrgInfo, PromptTemplateInfo, @@ -38,9 +42,11 @@ from turnstone.api.console_schemas import ( UpdatePromptTemplateRequest, UpdateRoleRequest, UpdateToolPolicyRequest, + UpdateWsTemplateRequest, UsageBreakdownItem, UsageResponse, UserRoleInfo, + WsTemplateInfo, ) from turnstone.api.openapi import EndpointSpec, QueryParam, build_openapi from turnstone.api.schemas import ( @@ -453,6 +459,63 @@ CONSOLE_ENDPOINTS: list[EndpointSpec] = [ error_codes=[404], tags=["Admin"], ), + # --- Governance: Workstream Templates --- + EndpointSpec( + "/v1/api/admin/ws-templates", + "GET", + "List workstream templates", + response_model=ListWsTemplatesResponse, + tags=["Admin"], + ), + EndpointSpec( + "/v1/api/admin/ws-templates", + "POST", + "Create a workstream template", + request_model=CreateWsTemplateRequest, + response_model=WsTemplateInfo, + error_codes=[400, 409], + tags=["Admin"], + ), + EndpointSpec( + "/v1/api/admin/ws-templates/{ws_template_id}", + "GET", + "Get a workstream template", + response_model=WsTemplateInfo, + error_codes=[404], + tags=["Admin"], + ), + EndpointSpec( + "/v1/api/admin/ws-templates/{ws_template_id}", + "PUT", + "Update a workstream template", + request_model=UpdateWsTemplateRequest, + response_model=WsTemplateInfo, + error_codes=[404, 409], + tags=["Admin"], + ), + EndpointSpec( + "/v1/api/admin/ws-templates/{ws_template_id}", + "DELETE", + "Delete a workstream template", + response_model=StatusResponse, + error_codes=[404], + tags=["Admin"], + ), + EndpointSpec( + "/v1/api/admin/ws-templates/{ws_template_id}/versions", + "GET", + "List workstream template version history", + response_model=ListWsTemplateVersionsResponse, + error_codes=[404], + tags=["Admin"], + ), + EndpointSpec( + "/v1/api/ws-templates", + "GET", + "List enabled workstream templates (summary)", + response_model=ListWsTemplateSummaryResponse, + tags=["Workstreams"], + ), # --- Governance: Usage & Audit --- EndpointSpec( "/v1/api/admin/usage", diff --git a/turnstone/api/schemas.py b/turnstone/api/schemas.py index 45fa68e5..136b8694 100644 --- a/turnstone/api/schemas.py +++ b/turnstone/api/schemas.py @@ -176,6 +176,7 @@ class CreateScheduleRequest(BaseModel): auto_approve: bool = Field(default=False) auto_approve_tools: list[str] = Field(default_factory=list) template: str = Field(default="", description="Prompt template name") + ws_template: str = Field(default="", description="Workstream template name") enabled: bool = Field(default=True) @@ -193,6 +194,7 @@ class UpdateScheduleRequest(BaseModel): auto_approve: bool | None = None auto_approve_tools: list[str] | None = None template: str | None = None + ws_template: str | None = None enabled: bool | None = None @@ -211,6 +213,7 @@ class ScheduleInfo(BaseModel): auto_approve: bool = False auto_approve_tools: list[str] = Field(default_factory=list) template: str = "" + ws_template: str = "" enabled: bool = True created_by: str = "" last_run: str | None = None diff --git a/turnstone/api/server_schemas.py b/turnstone/api/server_schemas.py index f9e88dbf..1d7b532b 100644 --- a/turnstone/api/server_schemas.py +++ b/turnstone/api/server_schemas.py @@ -50,6 +50,9 @@ class CreateWorkstreamRequest(BaseModel): template: str = Field( default="", description="Prompt template name (replaces default templates)" ) + ws_template: str = Field( + default="", description="Workstream template name to apply defaults from" + ) class CreateWorkstreamResponse(BaseModel): diff --git a/turnstone/channels/_routing.py b/turnstone/channels/_routing.py index 97660b88..67d9f0ed 100644 --- a/turnstone/channels/_routing.py +++ b/turnstone/channels/_routing.py @@ -50,12 +50,14 @@ class ChannelRouter: auto_approve: bool = False, auto_approve_tools: list[str] | None = None, template: str = "", + ws_template: str = "", ) -> None: self._broker = broker self._storage = storage self._auto_approve = auto_approve self._auto_approve_tools: list[str] = auto_approve_tools or [] self._template = template + self._ws_template = ws_template self._pending: dict[str, asyncio.Event] = {} self._pending_results: dict[str, str] = {} self._global_task: asyncio.Task[None] | None = None @@ -175,6 +177,7 @@ class ChannelRouter: auto_approve=self._auto_approve, auto_approve_tools=list(self._auto_approve_tools), template=self._template, + ws_template=self._ws_template, ) cid = msg.correlation_id waiter = asyncio.Event() diff --git a/turnstone/console/scheduler.py b/turnstone/console/scheduler.py index 934cb173..64fe11f8 100644 --- a/turnstone/console/scheduler.py +++ b/turnstone/console/scheduler.py @@ -209,6 +209,7 @@ class TaskScheduler: auto_approve_tools=self._parse_tools(task), user_id=task.get("created_by", ""), template=task.get("template", ""), + ws_template=task.get("ws_template", ""), ) self._broker.push_inbound(msg.to_json(), node_id=node_id) @@ -235,6 +236,7 @@ class TaskScheduler: auto_approve_tools=self._parse_tools(task), user_id=task.get("created_by", ""), template=task.get("template", ""), + ws_template=task.get("ws_template", ""), ) self._broker.push_inbound(msg.to_json()) diff --git a/turnstone/console/server.py b/turnstone/console/server.py index 37adaf92..9230fad2 100644 --- a/turnstone/console/server.py +++ b/turnstone/console/server.py @@ -350,6 +350,7 @@ async def create_workstream(request: Request) -> JSONResponse: raw_model = body.get("model", "") raw_initial_message = body.get("initial_message", "") raw_template = body.get("template", "") + raw_ws_template = body.get("ws_template", "") if not isinstance(raw_node_id, str): raw_node_id = "" if raw_node_id is None else None if not isinstance(raw_name, str): @@ -360,15 +361,20 @@ async def create_workstream(request: Request) -> JSONResponse: raw_initial_message = "" if raw_initial_message is None else None if not isinstance(raw_template, str): raw_template = "" if raw_template is None else None + if not isinstance(raw_ws_template, str): + raw_ws_template = "" if raw_ws_template is None else None if ( raw_node_id is None or raw_name is None or raw_model is None or raw_initial_message is None or raw_template is None + or raw_ws_template is None ): return JSONResponse( - {"error": "node_id, name, model, initial_message, and template must be strings"}, + { + "error": "node_id, name, model, initial_message, template, and ws_template must be strings" + }, status_code=400, ) node_id = raw_node_id @@ -376,13 +382,18 @@ async def create_workstream(request: Request) -> JSONResponse: model = raw_model[:128] initial_message = raw_initial_message[:4096] template = raw_template[:256] + ws_template = raw_ws_template[:256] from turnstone.mq.protocol import CreateWorkstreamMessage # General pool — push to shared queue, any bridge picks it up if node_id == "pool": msg = CreateWorkstreamMessage( - name=name, model=model, initial_message=initial_message, template=template + name=name, + model=model, + initial_message=initial_message, + template=template, + ws_template=ws_template, ) broker.push_inbound(msg.to_json()) log.debug("Pool dispatch: correlation_id=%s name=%r", msg.correlation_id, name) @@ -411,6 +422,7 @@ async def create_workstream(request: Request) -> JSONResponse: target_node=node_id, initial_message=initial_message, template=template, + ws_template=ws_template, ) broker.push_inbound(msg.to_json(), node_id=node_id) @@ -1145,6 +1157,7 @@ async def admin_create_schedule(request: Request) -> JSONResponse: raw_tools = body.get("auto_approve_tools", []) auto_approve_tools = raw_tools if isinstance(raw_tools, list) else [] template = str(body.get("template", "")).strip()[:256] + ws_template = str(body.get("ws_template", "")).strip()[:256] enabled = bool(body.get("enabled", True)) if not name: @@ -1153,6 +1166,10 @@ async def admin_create_schedule(request: Request) -> JSONResponse: return JSONResponse({"error": "initial_message is required"}, status_code=400) if template and not storage.get_prompt_template_by_name(template): return JSONResponse({"error": f"Template not found: {template}"}, status_code=400) + if ws_template and not storage.get_ws_template_by_name(ws_template): + return JSONResponse( + {"error": f"Workstream template not found: {ws_template}"}, status_code=400 + ) validation_err = _validate_schedule_fields(schedule_type, cron_expr, at_time) if validation_err: @@ -1188,6 +1205,7 @@ async def admin_create_schedule(request: Request) -> JSONResponse: created_by=created_by, next_run=next_run if enabled else "", template=template, + ws_template=ws_template, ) if not enabled: @@ -1267,6 +1285,13 @@ async def admin_update_schedule(request: Request) -> JSONResponse: if tpl_name and not storage.get_prompt_template_by_name(tpl_name): return JSONResponse({"error": f"Template not found: {tpl_name}"}, status_code=400) updates["template"] = tpl_name + if "ws_template" in body: + ws_tpl_name = str(body["ws_template"]).strip()[:256] + if ws_tpl_name and not storage.get_ws_template_by_name(ws_tpl_name): + return JSONResponse( + {"error": f"Workstream template not found: {ws_tpl_name}"}, status_code=400 + ) + updates["ws_template"] = ws_tpl_name if "enabled" in body: updates["enabled"] = bool(body["enabled"]) @@ -1440,6 +1465,13 @@ async def admin_cancel_watch(request: Request) -> Response: # --------------------------------------------------------------------------- +def _hash_content(content: str) -> str: + """SHA-256 hash of content for drift detection.""" + import hashlib + + return hashlib.sha256(content.encode()).hexdigest() + + def _audit_context(request: Request) -> tuple[str, str]: """Extract (user_id, ip_address) from request for audit logging. @@ -1477,6 +1509,7 @@ _VALID_PERMISSIONS = frozenset( "admin.usage", "admin.schedules", "admin.watches", + "admin.ws_templates", "tools.approve", "workstreams.create", "workstreams.close", @@ -2174,6 +2207,275 @@ async def admin_delete_template(request: Request) -> JSONResponse: return JSONResponse({"status": "ok"}) +# --------------------------------------------------------------------------- +# Admin: Workstream Templates +# --------------------------------------------------------------------------- + + +async def admin_list_ws_templates(request: Request) -> JSONResponse: + """GET /v1/api/admin/ws-templates — list all workstream templates.""" + from turnstone.core.auth import require_permission + from turnstone.core.web_helpers import require_storage_or_503 + + storage, err = require_storage_or_503(request) + if err: + return err + err = require_permission(request, "admin.ws_templates") + if err: + return err + return JSONResponse({"ws_templates": storage.list_ws_templates()}) + + +async def admin_create_ws_template(request: Request) -> JSONResponse: + """POST /v1/api/admin/ws-templates — create a workstream template.""" + import uuid + + from turnstone.core.audit import record_audit + from turnstone.core.auth import require_permission + from turnstone.core.web_helpers import read_json_or_400, require_storage_or_503 + + storage, err = require_storage_or_503(request) + if err: + return err + err = require_permission(request, "admin.ws_templates") + if err: + return err + body = await read_json_or_400(request) + if isinstance(body, JSONResponse): + return body + + name = str(body.get("name", "")).strip()[:256] + if not name: + return JSONResponse({"error": "name is required"}, status_code=400) + if storage.get_ws_template_by_name(name) is not None: + return JSONResponse({"error": "Name already exists"}, status_code=409) + + prompt_template_ref = str(body.get("prompt_template", ""))[:256] + prompt_template_hash = "" + if prompt_template_ref: + pt = storage.get_prompt_template_by_name(prompt_template_ref) + if not pt: + return JSONResponse( + {"error": f"Prompt template not found: {prompt_template_ref}"}, status_code=400 + ) + prompt_template_hash = _hash_content(pt.get("content", "")) + + try: + temperature = float(body["temperature"]) if body.get("temperature") is not None else None + max_tokens = int(body["max_tokens"]) if body.get("max_tokens") is not None else None + token_budget = int(body.get("token_budget", 0)) + agent_max_turns = ( + int(body["agent_max_turns"]) if body.get("agent_max_turns") is not None else None + ) + except (ValueError, TypeError) as exc: + return JSONResponse({"error": f"Invalid numeric field: {exc}"}, status_code=400) + + ws_template_id = uuid.uuid4().hex + storage.create_ws_template( + ws_template_id=ws_template_id, + name=name, + description=str(body.get("description", ""))[:1024], + system_prompt=str(body.get("system_prompt", ""))[:32768], + prompt_template=prompt_template_ref, + prompt_template_hash=prompt_template_hash, + model=str(body.get("model", ""))[:128], + auto_approve=bool(body.get("auto_approve", False)), + auto_approve_tools=str(body.get("auto_approve_tools", ""))[:2048], + temperature=temperature, + reasoning_effort=str(body.get("reasoning_effort", ""))[:32], + max_tokens=max_tokens, + token_budget=token_budget, + agent_max_turns=agent_max_turns, + notify_on_complete=str(body.get("notify_on_complete", "{}"))[:4096], + org_id=str(body.get("org_id", ""))[:128], + created_by=getattr(getattr(request.state, "auth_result", None), "user_id", ""), + enabled=bool(body.get("enabled", True)), + ) + + audit_uid, ip = _audit_context(request) + record_audit( + storage, + audit_uid, + "ws_template.create", + "ws_template", + ws_template_id, + {"name": name}, + ip, + ) + + tpl = storage.get_ws_template(ws_template_id) + return JSONResponse(tpl) + + +async def admin_get_ws_template(request: Request) -> JSONResponse: + """GET /v1/api/admin/ws-templates/{ws_template_id} — get a single workstream template.""" + from turnstone.core.auth import require_permission + from turnstone.core.web_helpers import require_storage_or_503 + + storage, err = require_storage_or_503(request) + if err: + return err + err = require_permission(request, "admin.ws_templates") + if err: + return err + ws_template_id = request.path_params["ws_template_id"] + tpl = storage.get_ws_template(ws_template_id) + if not tpl: + return JSONResponse({"error": "Not found"}, status_code=404) + return JSONResponse(tpl) + + +async def admin_update_ws_template(request: Request) -> JSONResponse: + """PUT /v1/api/admin/ws-templates/{ws_template_id} — update a workstream template.""" + from turnstone.core.audit import record_audit + from turnstone.core.auth import require_permission + from turnstone.core.web_helpers import read_json_or_400, require_storage_or_503 + + storage, err = require_storage_or_503(request) + if err: + return err + err = require_permission(request, "admin.ws_templates") + if err: + return err + ws_template_id = request.path_params["ws_template_id"] + existing = storage.get_ws_template(ws_template_id) + if not existing: + return JSONResponse({"error": "Not found"}, status_code=404) + + body = await read_json_or_400(request) + if isinstance(body, JSONResponse): + return body + + updates: dict[str, Any] = {} + if "name" in body: + new_name = str(body["name"]).strip()[:256] + if new_name != existing["name"] and storage.get_ws_template_by_name(new_name) is not None: + return JSONResponse({"error": "Name already exists"}, status_code=409) + updates["name"] = new_name + if "description" in body: + updates["description"] = str(body["description"])[:1024] + if "system_prompt" in body: + updates["system_prompt"] = str(body["system_prompt"])[:32768] + if "prompt_template" in body: + pt_ref = str(body["prompt_template"])[:256] + pt_obj = storage.get_prompt_template_by_name(pt_ref) if pt_ref else None + if pt_ref and not pt_obj: + return JSONResponse({"error": f"Prompt template not found: {pt_ref}"}, status_code=400) + updates["prompt_template"] = pt_ref + updates["prompt_template_hash"] = _hash_content(pt_obj.get("content", "")) if pt_obj else "" + if "model" in body: + updates["model"] = str(body["model"])[:128] + if "auto_approve" in body: + updates["auto_approve"] = bool(body["auto_approve"]) + if "auto_approve_tools" in body: + updates["auto_approve_tools"] = str(body["auto_approve_tools"])[:2048] + try: + if "temperature" in body: + updates["temperature"] = ( + float(body["temperature"]) if body["temperature"] is not None else None + ) + if "max_tokens" in body: + updates["max_tokens"] = ( + int(body["max_tokens"]) if body["max_tokens"] is not None else None + ) + if "token_budget" in body: + updates["token_budget"] = int(body["token_budget"]) + if "agent_max_turns" in body: + updates["agent_max_turns"] = ( + int(body["agent_max_turns"]) if body["agent_max_turns"] is not None else None + ) + except (ValueError, TypeError) as exc: + return JSONResponse({"error": f"Invalid numeric field: {exc}"}, status_code=400) + if "reasoning_effort" in body: + updates["reasoning_effort"] = str(body["reasoning_effort"])[:32] + if "notify_on_complete" in body: + updates["notify_on_complete"] = str(body["notify_on_complete"])[:4096] + if "enabled" in body: + updates["enabled"] = bool(body["enabled"]) + + changed_by = getattr(getattr(request.state, "auth_result", None), "user_id", "") + storage.update_ws_template(ws_template_id, changed_by=changed_by, **updates) + + audit_uid, ip = _audit_context(request) + record_audit( + storage, + audit_uid, + "ws_template.update", + "ws_template", + ws_template_id, + updates, + ip, + ) + + tpl = storage.get_ws_template(ws_template_id) + return JSONResponse(tpl) + + +async def admin_delete_ws_template(request: Request) -> JSONResponse: + """DELETE /v1/api/admin/ws-templates/{ws_template_id} — delete a workstream template.""" + from turnstone.core.audit import record_audit + from turnstone.core.auth import require_permission + from turnstone.core.web_helpers import require_storage_or_503 + + storage, err = require_storage_or_503(request) + if err: + return err + err = require_permission(request, "admin.ws_templates") + if err: + return err + ws_template_id = request.path_params["ws_template_id"] + existing = storage.get_ws_template(ws_template_id) + if not existing: + return JSONResponse({"error": "Not found"}, status_code=404) + + storage.delete_ws_template(ws_template_id) + + audit_uid, ip = _audit_context(request) + record_audit( + storage, + audit_uid, + "ws_template.delete", + "ws_template", + ws_template_id, + {"name": existing["name"]}, + ip, + ) + return JSONResponse({"status": "ok"}) + + +async def admin_list_ws_template_versions(request: Request) -> JSONResponse: + """GET /v1/api/admin/ws-templates/{ws_template_id}/versions — version history.""" + from turnstone.core.auth import require_permission + from turnstone.core.web_helpers import require_storage_or_503 + + storage, err = require_storage_or_503(request) + if err: + return err + err = require_permission(request, "admin.ws_templates") + if err: + return err + ws_template_id = request.path_params["ws_template_id"] + if not storage.get_ws_template(ws_template_id): + return JSONResponse({"error": "Not found"}, status_code=404) + versions = storage.list_ws_template_versions(ws_template_id) + return JSONResponse({"versions": versions}) + + +async def list_ws_templates_summary(request: Request) -> JSONResponse: + """GET /v1/api/ws-templates — enabled workstream templates summary.""" + from turnstone.core.web_helpers import require_storage_or_503 + + storage, err = require_storage_or_503(request) + if err: + return err + templates = storage.list_ws_templates(enabled_only=True) + summary = [ + {"name": t["name"], "description": t.get("description", ""), "model": t.get("model", "")} + for t in templates + ] + return JSONResponse({"ws_templates": summary}) + + async def admin_usage(request: Request) -> JSONResponse: """GET /v1/api/admin/usage — query usage data.""" from datetime import UTC, datetime, timedelta @@ -2294,6 +2596,7 @@ def create_app( Route("/api/cluster/node/{node_id}", cluster_node_detail), Route("/api/cluster/snapshot", cluster_snapshot), Route("/api/cluster/events", cluster_events_sse), + Route("/api/ws-templates", list_ws_templates_summary), Route("/api/auth/login", auth_login, methods=["POST"]), Route("/api/auth/logout", auth_logout, methods=["POST"]), Route("/api/auth/status", auth_status), @@ -2382,6 +2685,24 @@ def create_app( admin_delete_template, methods=["DELETE"], ), + # Governance: Workstream templates + Route("/api/admin/ws-templates", admin_list_ws_templates), + Route("/api/admin/ws-templates", admin_create_ws_template, methods=["POST"]), + Route("/api/admin/ws-templates/{ws_template_id}", admin_get_ws_template), + Route( + "/api/admin/ws-templates/{ws_template_id}", + admin_update_ws_template, + methods=["PUT"], + ), + Route( + "/api/admin/ws-templates/{ws_template_id}", + admin_delete_ws_template, + methods=["DELETE"], + ), + Route( + "/api/admin/ws-templates/{ws_template_id}/versions", + admin_list_ws_template_versions, + ), # Governance: Usage & Audit Route("/api/admin/usage", admin_usage), Route("/api/admin/audit", admin_audit), diff --git a/turnstone/console/static/admin.js b/turnstone/console/static/admin.js index 6a36112f..16aa76ef 100644 --- a/turnstone/console/static/admin.js +++ b/turnstone/console/static/admin.js @@ -41,6 +41,7 @@ function showAdmin() { roles: "admin.roles", policies: "admin.policies", templates: "admin.templates", + "ws-templates": "admin.ws_templates", usage: "admin.usage", audit: "admin.audit", }; @@ -101,6 +102,7 @@ function switchAdminTab(tab) { "roles", "policies", "templates", + "ws-templates", "usage", "audit", ]; @@ -117,6 +119,7 @@ function switchAdminTab(tab) { if (tab === "roles") loadGovRoles(); if (tab === "policies") loadGovPolicies(); if (tab === "templates") loadGovTemplates(); + if (tab === "ws-templates") loadGovWsTemplates(); if (tab === "usage") loadGovUsage(); if (tab === "audit") { _populateAuditUserFilter(); @@ -465,6 +468,28 @@ var _srTrapHandler = null; var _editScheduleTriggerEl = null; var _runsScheduleTriggerEl = null; +function _populateWsTemplateSelect(selectId) { + var sel = document.getElementById(selectId); + sel.innerHTML = ''; + return authFetch("/v1/api/ws-templates") + .then(function (r) { + return r.json(); + }) + .then(function (data) { + (data.ws_templates || []).forEach(function (t) { + var opt = document.createElement("option"); + opt.value = t.name; + var label = t.name; + if (t.model) label += " (" + t.model + ")"; + opt.textContent = label; + sel.appendChild(opt); + }); + }) + .catch(function () { + /* ignore — dropdown stays with "None" */ + }); +} + function loadAdminSchedules() { authFetch("/v1/api/admin/schedules") .then(function (r) { @@ -662,6 +687,7 @@ function showCreateScheduleModal() { document.getElementById("cs-node").value = ""; document.getElementById("cs-model").value = ""; document.getElementById("cs-template").value = ""; + _populateWsTemplateSelect("cs-ws-template"); document.getElementById("cs-message").value = ""; document.getElementById("cs-autoapprove").checked = false; toggleScheduleTypeFields(); @@ -695,6 +721,7 @@ function submitCreateSchedule() { var model = (document.getElementById("cs-model").value || "").trim(); var message = (document.getElementById("cs-message").value || "").trim(); var template = (document.getElementById("cs-template").value || "").trim(); + var wsTemplate = document.getElementById("cs-ws-template").value; var autoApprove = document.getElementById("cs-autoapprove").checked; var errEl = document.getElementById("create-schedule-error"); @@ -732,6 +759,7 @@ function submitCreateSchedule() { initial_message: message, auto_approve: autoApprove, template: template, + ws_template: wsTemplate, }), }) .then(function (r) { @@ -799,6 +827,10 @@ function showEditScheduleModal(taskId) { : ""; document.getElementById("es-model").value = s.model || ""; document.getElementById("es-template").value = s.template || ""; + var _wsTemplateVal = s.ws_template || ""; + _populateWsTemplateSelect("es-ws-template").then(function () { + document.getElementById("es-ws-template").value = _wsTemplateVal; + }); document.getElementById("es-message").value = s.initial_message || ""; document.getElementById("es-autoapprove").checked = !!s.auto_approve; document.getElementById("es-enabled").checked = !!s.enabled; @@ -872,6 +904,7 @@ function submitEditSchedule() { target_mode: targetMode, model: (document.getElementById("es-model").value || "").trim(), template: (document.getElementById("es-template").value || "").trim(), + ws_template: document.getElementById("es-ws-template").value, initial_message: ( document.getElementById("es-message").value || "" ).trim(), @@ -1450,6 +1483,10 @@ function _installTrap(overlayId, boxId, trapRef) { else if (overlayId === "create-template-overlay") hideCreateTemplateModal(); else if (overlayId === "edit-template-overlay") hideEditTemplateModal(); + else if (overlayId === "create-wst-overlay") + hideCreateWsTemplateModal(); + else if (overlayId === "edit-wst-overlay") hideEditWsTemplateModal(); + else if (overlayId === "wst-history-overlay") hideWstHistoryModal(); } }; } @@ -1525,6 +1562,9 @@ document.addEventListener("keydown", function (e) { ["edit-policy-overlay", hideEditPolicyModal], ["create-template-overlay", hideCreateTemplateModal], ["edit-template-overlay", hideEditTemplateModal], + ["create-wst-overlay", hideCreateWsTemplateModal], + ["edit-wst-overlay", hideEditWsTemplateModal], + ["wst-history-overlay", hideWstHistoryModal], ]; for (var gi = 0; gi < govOverlays.length; gi++) { var govEl = document.getElementById(govOverlays[gi][0]); diff --git a/turnstone/console/static/app.js b/turnstone/console/static/app.js index 15a767de..4fd5b6d8 100644 --- a/turnstone/console/static/app.js +++ b/turnstone/console/static/app.js @@ -1261,6 +1261,26 @@ function showNewWsModal() { .catch(function () { /* ignore — defaults still work */ }); + // Populate profile (WS template) dropdown + var profSelect = document.getElementById("new-ws-profile"); + profSelect.innerHTML = ''; + authFetch("/v1/api/ws-templates") + .then(function (r) { + return r.json(); + }) + .then(function (data) { + (data.ws_templates || []).forEach(function (t) { + var opt = document.createElement("option"); + opt.value = t.name; + var label = t.name; + if (t.model) label += " (" + t.model + ")"; + opt.textContent = label; + profSelect.appendChild(opt); + }); + }) + .catch(function () { + /* ignore — profiles optional */ + }); document.getElementById("new-ws-name").value = ""; document.getElementById("new-ws-model").value = ""; document.getElementById("new-ws-task").value = ""; @@ -1330,6 +1350,8 @@ function submitNewWs() { if (model) body.model = model; if (task) body.initial_message = task; if (template) body.template = template; + var profile = document.getElementById("new-ws-profile").value; + if (profile) body.ws_template = profile; authFetch("/v1/api/cluster/workstreams/new", { method: "POST", diff --git a/turnstone/console/static/governance.js b/turnstone/console/static/governance.js index db0bd19d..3e47deab 100644 --- a/turnstone/console/static/governance.js +++ b/turnstone/console/static/governance.js @@ -6,6 +6,7 @@ var _govRoles = []; var _govPolicies = []; var _govTemplates = []; +var _govWsTemplates = []; var _govUsageRange = "7d"; var _govUsageGroupBy = "day"; var _govAuditEvents = []; @@ -20,6 +21,8 @@ var _cpTrapHandler = null; // create policy var _epTrapHandler = null; // edit policy var _ctmTrapHandler = null; // create template var _etmTrapHandler = null; // edit template +var _cwstTrapHandler = null; // create ws template +var _ewstTrapHandler = null; // edit ws template // Trigger element refs for focus restoration var _crTriggerEl = null; @@ -29,6 +32,8 @@ var _cpTriggerEl = null; var _epTriggerEl = null; var _ctmTriggerEl = null; var _etmTriggerEl = null; +var _cwstTriggerEl = null; +var _ewstTriggerEl = null; // --------------------------------------------------------------------------- // Roles @@ -928,6 +933,445 @@ function submitEditTemplate() { }); } +// --------------------------------------------------------------------------- +// WS Templates +// --------------------------------------------------------------------------- + +function loadGovWsTemplates() { + authFetch("/v1/api/admin/ws-templates") + .then(function (r) { + if (!r.ok) throw new Error("Failed"); + return r.json(); + }) + .then(function (data) { + _govWsTemplates = data.ws_templates || []; + _renderGovWsTemplates(_govWsTemplates); + }) + .catch(function () { + document.getElementById("admin-ws-templates-table").innerHTML = + '
Failed to load WS templates
'; + }); +} + +function _renderGovWsTemplates(items) { + var el = document.getElementById("admin-ws-templates-table"); + if (!items.length) { + el.innerHTML = + '
No workstream templates defined
'; + return; + } + var html = ""; + for (var i = 0; i < items.length; i++) { + var t = items[i]; + var modelBadge = t.model + ? '' + escapeHtml(t.model) + "" + : 'default'; + var approveBadge = t.auto_approve + ? 'auto' + : ""; + var budgetBadge = + t.token_budget > 0 + ? '' + + t.token_budget.toLocaleString() + + "" + : ""; + var enabledBadge = !t.enabled + ? ' disabled' + : ""; + html += + '
' + + '' + + escapeHtml(t.name) + + enabledBadge + + "" + + '' + + modelBadge + + "" + + '' + + approveBadge + + " " + + budgetBadge + + "" + + '' + + "v" + + t.version + + " " + + ' ' + + '' + + '' + + "
"; + } + el.innerHTML = html; + el.querySelectorAll("[data-edit-wst]").forEach(function (btn) { + btn.addEventListener("click", function () { + showEditWsTemplateModal(this.getAttribute("data-edit-wst")); + }); + }); + el.querySelectorAll("[data-delete-wst]").forEach(function (btn) { + btn.addEventListener("click", function () { + var tid = this.getAttribute("data-delete-wst"); + var tname = this.getAttribute("data-wst-name"); + showConfirmModal( + "Delete WS Template", + 'Delete workstream template "' + tname + '"?', + "Delete", + function () { + authFetch("/v1/api/admin/ws-templates/" + tid, { + method: "DELETE", + }) + .then(function (r) { + if (!r.ok) throw new Error(); + return r.json(); + }) + .then(function () { + showToast("WS template deleted"); + loadGovWsTemplates(); + }) + .catch(function () { + showToast("Failed to delete WS template"); + }); + }, + ); + }); + }); + el.querySelectorAll("[data-history-wst]").forEach(function (btn) { + btn.addEventListener("click", function () { + showWstHistoryModal(this.getAttribute("data-history-wst")); + }); + }); +} + +function toggleWstPromptSource() { + var inline = document.getElementById("cwst-src-inline").checked; + document.getElementById("cwst-inline-section").style.display = inline + ? "" + : "none"; + document.getElementById("cwst-ref-section").style.display = inline + ? "none" + : ""; +} + +function toggleEditWstPromptSource() { + var inline = document.getElementById("ewst-src-inline").checked; + document.getElementById("ewst-inline-section").style.display = inline + ? "" + : "none"; + document.getElementById("ewst-ref-section").style.display = inline + ? "none" + : ""; +} + +function _populateWstPromptTemplates(selectId) { + var sel = document.getElementById(selectId); + sel.innerHTML = ''; + return authFetch("/v1/api/admin/templates") + .then(function (r) { + return r.json(); + }) + .then(function (data) { + (data.templates || []).forEach(function (t) { + var opt = document.createElement("option"); + opt.value = t.name; + opt.textContent = t.name; + sel.appendChild(opt); + }); + }) + .catch(function () { + /* ignore */ + }); +} + +function showCreateWsTemplateModal() { + _cwstTriggerEl = document.activeElement; + var ov = document.getElementById("create-wst-overlay"); + ov.style.display = "flex"; + document.getElementById("cwst-name").value = ""; + document.getElementById("cwst-description").value = ""; + document.getElementById("cwst-system-prompt").value = ""; + document.getElementById("cwst-src-inline").checked = true; + toggleWstPromptSource(); + _populateWstPromptTemplates("cwst-prompt-template"); + document.getElementById("cwst-model").value = ""; + document.getElementById("cwst-auto-approve").checked = false; + document.getElementById("cwst-auto-approve-tools").value = ""; + document.getElementById("cwst-token-budget").value = "0"; + document.getElementById("cwst-temperature").value = ""; + document.getElementById("cwst-reasoning-effort").value = ""; + document.getElementById("cwst-max-tokens").value = ""; + document.getElementById("cwst-agent-max-turns").value = ""; + document.getElementById("cwst-enabled").checked = true; + document.getElementById("create-wst-error").style.display = "none"; + document.getElementById("cwst-name").focus(); + _cwstTrapHandler = _installTrap("create-wst-overlay", "create-wst-box"); +} + +function hideCreateWsTemplateModal() { + document.getElementById("create-wst-overlay").style.display = "none"; + _cwstTrapHandler = _removeTrap(_cwstTrapHandler); + if (_cwstTriggerEl && _cwstTriggerEl.focus) _cwstTriggerEl.focus(); + _cwstTriggerEl = null; +} + +function submitCreateWsTemplate() { + var name = document.getElementById("cwst-name").value.trim(); + if (!name) { + var e = document.getElementById("create-wst-error"); + e.textContent = "Name is required"; + e.style.display = ""; + return; + } + var isInline = document.getElementById("cwst-src-inline").checked; + document.getElementById("cwst-submit").disabled = true; + authFetch("/v1/api/admin/ws-templates", { + method: "POST", + headers: { "Content-Type": "application/json" }, + body: JSON.stringify({ + name: name, + description: document.getElementById("cwst-description").value, + system_prompt: isInline + ? document.getElementById("cwst-system-prompt").value + : "", + prompt_template: isInline + ? "" + : document.getElementById("cwst-prompt-template").value, + model: document.getElementById("cwst-model").value.trim(), + auto_approve: document.getElementById("cwst-auto-approve").checked, + auto_approve_tools: document + .getElementById("cwst-auto-approve-tools") + .value.trim(), + token_budget: parseInt( + document.getElementById("cwst-token-budget").value || "0", + 10, + ), + temperature: document.getElementById("cwst-temperature").value + ? parseFloat(document.getElementById("cwst-temperature").value) + : null, + reasoning_effort: document.getElementById("cwst-reasoning-effort").value, + max_tokens: document.getElementById("cwst-max-tokens").value + ? parseInt(document.getElementById("cwst-max-tokens").value, 10) + : null, + agent_max_turns: document.getElementById("cwst-agent-max-turns").value + ? parseInt(document.getElementById("cwst-agent-max-turns").value, 10) + : null, + enabled: document.getElementById("cwst-enabled").checked, + }), + }) + .then(function (r) { + if (!r.ok) + return r.json().then(function (d) { + throw new Error(d.error || "Failed"); + }); + return r.json(); + }) + .then(function () { + hideCreateWsTemplateModal(); + showToast("WS template created"); + loadGovWsTemplates(); + }) + .catch(function (e) { + var el = document.getElementById("create-wst-error"); + el.textContent = e.message; + el.style.display = ""; + }) + .finally(function () { + document.getElementById("cwst-submit").disabled = false; + }); +} + +function showEditWsTemplateModal(wstId) { + _ewstTriggerEl = document.activeElement; + var tpl = null; + for (var i = 0; i < _govWsTemplates.length; i++) { + if (_govWsTemplates[i].ws_template_id === wstId) { + tpl = _govWsTemplates[i]; + break; + } + } + if (!tpl) return; + var ov = document.getElementById("edit-wst-overlay"); + ov.style.display = "flex"; + document.getElementById("ewst-id").value = wstId; + document.getElementById("ewst-name").value = tpl.name; + document.getElementById("ewst-description").value = tpl.description || ""; + document.getElementById("ewst-system-prompt").value = tpl.system_prompt || ""; + // Set radio based on which field has content + if (tpl.prompt_template && !tpl.system_prompt) { + document.getElementById("ewst-src-ref").checked = true; + } else { + document.getElementById("ewst-src-inline").checked = true; + } + toggleEditWstPromptSource(); + _populateWstPromptTemplates("ewst-prompt-template").then(function () { + if (tpl.prompt_template) { + document.getElementById("ewst-prompt-template").value = + tpl.prompt_template; + } + }); + document.getElementById("ewst-model").value = tpl.model || ""; + document.getElementById("ewst-auto-approve").checked = tpl.auto_approve; + document.getElementById("ewst-auto-approve-tools").value = + tpl.auto_approve_tools || ""; + document.getElementById("ewst-token-budget").value = tpl.token_budget || 0; + document.getElementById("ewst-temperature").value = + tpl.temperature != null ? tpl.temperature : ""; + document.getElementById("ewst-reasoning-effort").value = + tpl.reasoning_effort || ""; + document.getElementById("ewst-max-tokens").value = + tpl.max_tokens != null ? tpl.max_tokens : ""; + document.getElementById("ewst-agent-max-turns").value = + tpl.agent_max_turns != null ? tpl.agent_max_turns : ""; + document.getElementById("ewst-enabled").checked = tpl.enabled; + document.getElementById("edit-wst-error").style.display = "none"; + _ewstTrapHandler = _installTrap("edit-wst-overlay", "edit-wst-box"); +} + +function hideEditWsTemplateModal() { + document.getElementById("edit-wst-overlay").style.display = "none"; + _ewstTrapHandler = _removeTrap(_ewstTrapHandler); + if (_ewstTriggerEl && _ewstTriggerEl.focus) _ewstTriggerEl.focus(); + _ewstTriggerEl = null; +} + +function submitEditWsTemplate() { + var id = document.getElementById("ewst-id").value; + var name = document.getElementById("ewst-name").value.trim(); + if (!name) { + var e = document.getElementById("edit-wst-error"); + e.textContent = "Name is required"; + e.style.display = ""; + return; + } + var isInline = document.getElementById("ewst-src-inline").checked; + document.getElementById("ewst-submit").disabled = true; + authFetch("/v1/api/admin/ws-templates/" + id, { + method: "PUT", + headers: { "Content-Type": "application/json" }, + body: JSON.stringify({ + name: document.getElementById("ewst-name").value.trim(), + description: document.getElementById("ewst-description").value, + system_prompt: isInline + ? document.getElementById("ewst-system-prompt").value + : "", + prompt_template: isInline + ? "" + : document.getElementById("ewst-prompt-template").value, + model: document.getElementById("ewst-model").value.trim(), + auto_approve: document.getElementById("ewst-auto-approve").checked, + auto_approve_tools: document + .getElementById("ewst-auto-approve-tools") + .value.trim(), + token_budget: parseInt( + document.getElementById("ewst-token-budget").value || "0", + 10, + ), + temperature: document.getElementById("ewst-temperature").value + ? parseFloat(document.getElementById("ewst-temperature").value) + : null, + reasoning_effort: document.getElementById("ewst-reasoning-effort").value, + max_tokens: document.getElementById("ewst-max-tokens").value + ? parseInt(document.getElementById("ewst-max-tokens").value, 10) + : null, + agent_max_turns: document.getElementById("ewst-agent-max-turns").value + ? parseInt(document.getElementById("ewst-agent-max-turns").value, 10) + : null, + enabled: document.getElementById("ewst-enabled").checked, + }), + }) + .then(function (r) { + if (!r.ok) + return r.json().then(function (d) { + throw new Error(d.error || "Failed"); + }); + return r.json(); + }) + .then(function () { + hideEditWsTemplateModal(); + showToast("WS template updated"); + loadGovWsTemplates(); + }) + .catch(function (e) { + var el = document.getElementById("edit-wst-error"); + el.textContent = e.message; + el.style.display = ""; + }) + .finally(function () { + document.getElementById("ewst-submit").disabled = false; + }); +} + +// --------------------------------------------------------------------------- +// WS Template Version History +// --------------------------------------------------------------------------- + +var _whTrapHandler = null; +var _whTriggerEl = null; + +function showWstHistoryModal(wstId) { + _whTriggerEl = document.activeElement; + var ov = document.getElementById("wst-history-overlay"); + ov.style.display = "flex"; + document.getElementById("wst-history-content").innerHTML = + '
Loading...
'; + _whTrapHandler = _installTrap("wst-history-overlay", "wst-history-box"); + authFetch("/v1/api/admin/ws-templates/" + wstId + "/versions") + .then(function (r) { + if (!r.ok) throw new Error("Failed"); + return r.json(); + }) + .then(function (data) { + var versions = data.versions || []; + if (!versions.length) { + document.getElementById("wst-history-content").innerHTML = + '
No version history yet
'; + return; + } + var html = ""; + for (var i = 0; i < versions.length; i++) { + var v = versions[i]; + var snapshot = "{}"; + try { + snapshot = JSON.stringify(JSON.parse(v.snapshot), null, 2); + } catch (e) { + snapshot = v.snapshot; + } + html += + '
' + + '
' + + "v" + + v.version + + "" + + '' + + escapeHtml(v.changed_by || "unknown") + + " — " + + escapeHtml(v.created) + + "
" + + '
' +
+          escapeHtml(snapshot) +
+          "
"; + } + document.getElementById("wst-history-content").innerHTML = html; + }) + .catch(function () { + document.getElementById("wst-history-content").innerHTML = + '
Failed to load version history
'; + }); +} + +function hideWstHistoryModal() { + document.getElementById("wst-history-overlay").style.display = "none"; + _whTrapHandler = _removeTrap(_whTrapHandler); + if (_whTriggerEl && _whTriggerEl.focus) _whTriggerEl.focus(); + _whTriggerEl = null; +} + // --------------------------------------------------------------------------- // Usage // --------------------------------------------------------------------------- diff --git a/turnstone/console/static/index.html b/turnstone/console/static/index.html index 625dff54..434be854 100644 --- a/turnstone/console/static/index.html +++ b/turnstone/console/static/index.html @@ -86,6 +86,7 @@ + @@ -247,6 +248,23 @@ + + +