mirror of
https://github.com/turnstonelabs/turnstone.git
synced 2026-08-12 23:12:23 -06:00
e60c19befd5e31376bb606cd380c3564ac4e27df
20 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Patrick Buckley
|
b65e5cae0e |
docs(personas): accuracy sweep — spec models, protocol contracts, page corrections
Spec models now describe what the endpoints do: ListPersonasResponse declares the tool_inventory the shelf depends on, both console create models declare persona, CreatePersonaRequest declares org_id, and UpdatePersonaRequest documents the null-vs-absent split (null clears base_prompt/tool_allowlist, null on flags/kinds is ignored). Console OpenAPI regenerated. Protocol contracts match the implementations: update_persona's return covers the no-op case, create_persona's raises-list is complete, and both extended row-shape docstrings gain their tail columns plus the append-only rule. The workstreams.persona comments say slug, not display name. Page corrections from the docs review: personas.md documents the creative_mode-to-writer migration conversion, the mid-session /resume MCP-lever behavior, visibility-based nudge gating, the soft-set prompt-cache cost, and the executive tool list — and drops internal jargon. The changelog entry moves under [Unreleased] with the house breaking-marker style and the auto-conversion note. coordinator-skills and the API tour stop using persona to mean framing; governance, api-reference, sdk, console, tools, and memory pick up the new permission family, endpoints, kwargs, picker, and lever caveats. |
||
|
Patrick Buckley
|
110d44b07e |
refactor(tools): remove man, math, and plan_agent built-in tools
`man` and `math` duplicated capabilities already reachable through `bash`; `plan_agent` is better expressed as a `task_agent` running a planning skill, and carried a large amount of special-case machinery (plan-review gate, refinement loop, per-kind model routing). Removing all three shrinks the tool surface and cuts per-call token cost. Also removed, as dead-once-the-tools-are-gone: - the `math` sandbox executor (`turnstone.core.sandbox`) and its `[sandbox]` extra; the eval analyst now runs bash-only - the read-only `AGENT_TOOLS` sub-agent tool set and the `agent` tool-metadata key (`task_agent`/`TASK_AGENT_TOOLS` retained) - the plan-review protocol end to end: the `on_plan_review` UI hook, `resolve_plan`, `POST /v1/api/plan` + `POST /v1/api/route/plan`, the `plan_review`/`plan_resolved` SSE events, and their Python SDK / TypeScript SDK / OpenAPI / frontend / Discord+Slack bindings - the `model.plan_alias` / `model.plan_effort` settings and the registry `plan_model` / `plan_effort` routing fields TOOLS 31->28, TASK_AGENT_TOOLS 13->11; COORDINATOR_TOOLS unchanged. BREAKING CHANGE: removes the `man`, `math`, `plan_agent` tools, the plan-review SSE/HTTP/SDK surface, and the plan_* model-routing settings from the experimental 1.6 line. |
||
|
Patrick Buckley
|
57cb09c871 |
docs(sse): document state_change + in_progress_snapshot events
Updates the docs that describe the per-workstream SSE event stream and the SessionUI lifecycle to match the refresh-resume changes: - api-reference.md: documented the `state_change` event (previously undocumented despite already being a live event) and the new `in_progress_snapshot` event; rewrote the multi-consumer fan-out paragraph to mention the kind-specific replay tail (state_change + optional in_progress_snapshot) so the "no catch-up needed" claim is no longer misleading. - architecture.md: bumped the SessionUI Protocol stub to 16 methods (added `on_turn_start` / `on_turn_committed`) and pointed at the in_progress_snapshot section in the API reference. - sdk.md: added rows for `state_change`, `in_progress_snapshot`, and `approval_resolved` (preexisting gap) to the per-workstream event table. - coordinator-api-tour.md: added an `in_progress_snapshot` row to the event table and rewrote the reconnection-contract paragraph to cover mid-stream content/reasoning restoration. - diagrams/04-conversation-turn.puml: added `on_turn_start()` before the thinking-start emit and `on_turn_committed()` immediately after `messages.append(assistant_msg)`, with notes explaining the inflight- buffer reset semantics. PNG regenerated. |
||
|
Patrick Buckley
|
a917bf2690 |
docs: apply Copilot review feedback on PR #367
All eight suggestions verified against source before applying: - docs/settings.md — ConfigStore key names are `model.plan_alias` / `model.task_alias` (not `plan_model` / `task_model`); updated in both the overview list and the plan/task overrides table. - docs/security.md — `src` claim values now reflect what actually gets minted: `password`, `database` (from API-token exchange), `oidc`, plus service origins `console`, `cli`, `channel`. - docs/sdk.md — `upload_attachment(ws_id, filename, data, *, mime_type=...)` matches the real SDK signature; `bytes`-returning helper is `get_attachment_content` (not `download_attachment`); code example reordered so it doesn't collide on `filename=` kwarg. - docs/architecture.md — "prior `plan` tool call" → "prior `plan_agent` tool call" so wording stays consistent with the renamed tool. - docs/tools.md — `plan_agent` `primary_key` is `goal`, not `prompt`, in both the primary-key table and the summary table (matches the JSON schema in turnstone/tools/plan_agent.json). |
||
|
Patrick Buckley
|
471d1a3311 |
docs: audit documentation for 1.4 / 1.5 state
Systematic pass over every doc under docs/, the root-level README /
QUICKSTART / CONTRIBUTING, and the PlantUML diagrams. Memory and docs
had drifted against the code since 1.2 — this catches them up to the
1.4.0 release and the 1.5.0a1 experimental line.
User-facing fixes
- README: fix broken docs/mcp.md link (→ mcp-registry.md); channel
gateway entry reflects shipped Discord + Slack adapters instead of
"Slack/Teams planned"; diagrams table mentions both.
- QUICKSTART: docs/*.md relative links were wrong from the repo root;
wizard version bumped from 0.5.4.
- CONTRIBUTING: add dev extra plus the ruff / mypy / pytest commands
we actually expect before push.
Reference docs
- architecture.md: 19 tool schemas (was 15), 18 admin tabs (was 14),
turnstone-bootstrap added to entry-points table, OpenAI provider
file split (chat/responses/common) documented, 38 SDK event
dataclasses (was 27 and referenced deleted mq/protocol.py), Slack
adapter + multi-adapter gateway, plan_agent/task_agent naming,
governance admin-panel rewrite.
- api-reference.md: full attachment endpoints (POST/GET/content/
DELETE on /v1/api/workstreams/{ws_id}/attachments) plus the
multipart mode on POST /v1/api/workstreams/new.
- channels.md: Slack Setup section (Socket Mode app creation, OAuth
scopes, tokens), Slack CLI/env reference in config table, combined-
adapter architecture diagram.
- console.md: 18-tab listing (was 13) with Channels/Models/Nodes/TLS
descriptions and ConfigStore live-edit note.
- docker.md: Slack env vars block; image entry-point list now
includes turnstone / turnstone-bootstrap.
- sdk.md: attachments methods on the server client, attachments
example (upload-then-send and at-creation), event count fixed.
- releasing.md: four-track table (stable/1.0, 1.3, 1.4 + main 1.5);
promotion workflow uses 1.5 / 1.6 numbering.
- settings.md: plan_model / task_model / plan_effort / task_effort
overrides section.
- governance.md: skill naming (/skill, `skill` field — not /template),
Prompts/Judge tabs called out.
- security.md: two-token-types wording; src claim values match the
AuthResult source strings actually emitted.
- mcp-registry.md: SDK package name is @turnstone/sdk.
- tools.md: plan / task renamed to plan_agent / task_agent in the
section headings and summary table; primary-key table matched.
- design/consistent-hash-ring.md: dead direct-http-transport.md
pointer redirected to architecture.md.
Diagrams
- 02-package-structure: drop phantom chat.py entry point, add admin
and bootstrap, add slack/bot.py, rename channels/gateway.py →
channels/cli.py.
- 16-channel-architecture: Slack is no longer "(future)", add a
SlackBot class and the slack-bolt Socket Mode edges; wire the new
bot into ChannelService. PNGs regenerated from both puml sources.
|
||
|
Patrick Buckley
|
62d2a0fe6a |
fix: remove non-auth support from bootstrap wizard (#274)
* fix: remove non-auth support from bootstrap wizard Auth is now mandatory for all deployments. Remove the TURNSTONE_AUTH_ENABLED toggle and make JWT_SECRET and AUTH_TOKEN required in the wizard's system prompt. * fix: remove auth disable support from runtime and infra Remove AuthConfig.enabled field — auth is always on. Drop TURNSTONE_AUTH_ENABLED env var, config toggle, and the check_request bypass. Update compose.yaml, Helm chart, Terraform, docs, and tests to match. * feat: deprecate config tokens, require JWT secret, prefer JWT auth Phase 1 of config-token removal: - load_jwt_secret() now exits with error if no secret is configured (was: silently auto-generated ephemeral secret) - _authenticate_token() logs deprecation warning on config token use - CLI /cluster commands use ServiceTokenManager when JWT secret is set - turnstone-admin tls-list uses ServiceTokenManager when JWT secret is set - Update bootstrap wizard, docker.md, security.md to mark TURNSTONE_AUTH_TOKEN as deprecated and JWT_SECRET as required - Console test fixtures use auth token + headers (auth always enforced) * feat: add service scope for inter-service JWT auth Add "service" to VALID_SCOPES and SCOPE_HIERARCHY. Service tokens bypass require_permission() RBAC checks, replacing the old empty-user-id bypass that config tokens relied on. All ServiceTokenManager instances that need admin access now include "service" in their scopes (console proxy, channel gateway, CLI, admin CLI). Read-only services (collector, notification) unchanged. * feat: phase 2 config token deprecation - SDK doc examples now show API tokens (ts_) instead of config tokens - Remove _get_config_token() from admin CLI (dead code) - Block config token exchange in handle_auth_login — only password and API token login allowed - Update login tests to use password-based auth instead of config token exchange * feat: phase 3 — remove config tokens entirely Complete removal of config-file token authentication: - Delete AuthConfig.tokens, check(), _ROLE_TO_SCOPES, hmac dispatch branch, and config token loading from load_auth_config() - Remove auth_config parameter from _authenticate_token() and check_request() — callers updated throughout - Remove TURNSTONE_AUTH_TOKEN from compose.yaml, Helm charts, Terraform, turnstone.example.toml - Remove --auth-token CLI flags from turnstone, turnstone-admin, and turnstone-console - Simplify console main() — always use ServiceTokenManager (no fallback to static tokens) - Delete config-token-specific tests, rewrite check_request and integration tests to use JWT auth with proper audience claims - Remove all config token references from docs (security.md, docker.md, sdk.md, console.md, architecture.md, bootstrap prompt) * fix: address code review findings - Fix 33 broken tests: add JWT auth to test_api_versioning, test_console_routing_proxy, test_tls_admin, test_tls_manager, test_server_live (jwt_secret + audience-scoped auth headers) - Add TestRequirePermissionServiceScope: 4 tests covering the service scope RBAC bypass path - Remove stale comments referencing config tokens in auth.py and console/server.py - Remove dead proxy_auth_token parameter from console create_app() and static token fallback in _proxy_auth_headers() - Remove TURNSTONE_AUTH_TOKEN from env.py scrub list * fix: address Copilot review — JWT audience, compose require secret - CLI /cluster: add audience=JWT_AUD_CONSOLE to ServiceTokenManager (console validates audience, JWTs without it were rejected) - Admin CLI tls-list: same audience fix - compose.yaml: TURNSTONE_JWT_SECRET now uses :? to fail fast if unset - SDK console: fix default port from 8081 to 8090 * test: add auth enforcement tests for TLS admin endpoints 5 new tests: unauthenticated requests return 401 (list, renew, delete), read-only-scoped requests return 403 (renew, delete). Closes the TLS auth enforcement test gap noted in PROGRESS.md. * fix: address remaining Copilot review feedback - Fix token_source="config" → "test" in TLS test fixtures - Fix AuthResult.token_source docstring to include service origins - Require TURNSTONE_JWT_SECRET in cluster compose profile (:?) - Helm: add auth.jwtSecret + auth.existingSecret values, wire TURNSTONE_JWT_SECRET into secret.yaml and both deployments - Terraform: replace auth_token with jwt_secret variable + secret, remove orphaned auth_token resources and IAM reference - Remove [[auth.tokens]] from security.md config example * fix: address full code review — 10 findings Critical: - Terraform: replace concat(common_env, auth_env) with common_env (auth_env local was removed but still referenced) - Channel gateway: remove hmac static token auth from _check_auth(), use JWT-only validation. Remove --auth-token CLI arg from channel - Rebalancer: add token_manager support so migration requests carry JWT auth (was sending unauthenticated POST to /internal/migrate) Major: - Guard _permissions_to_scopes() against "service" privilege escalation from DB role permissions - Remove dead AuthConfig class, load_auth_config(), and all auth_config parameters from create_app() signatures - Helm: inject JWT secret for both inline and existingSecret paths Minor: - Remove dead auth_token param from ClusterCollector - Remove empty TestLoadAuthConfig class - Short JWT secret now exits instead of warning - Compose: add generation command comment above JWT_SECRET - Clean stale config token references from 6 doc files - Clean stale AUTH_TOKEN reference from bootstrap wizard prompt * fix: remove remaining stale config token references from docs - channels.md: remove --auth-token from options table - oidc.md: remove "config-file tokens still work" claim - security.md: remove config token section, fix JWT secret docs (now required/exits, no ephemeral fallback), remove hmac from ASCII diagram, remove --auth-token reference |
||
|
Patrick Buckley
|
f74aa2264e |
refactor: add is_error to on_tool_result protocol, remove text heuris… (#207)
* refactor: add is_error to on_tool_result protocol, remove text heuristics Add is_error keyword arg to SessionUI.on_tool_result() so tools report errors structurally. Server and JS client no longer guess from output text prefixes — each tool sets the flag at the source. Bash tool: exit code >= 2 is error, exit code 1 is ambiguous (grep no-match). History reconstruction keeps text heuristic as fallback for pre-migration data. Update SDKs (Python + TypeScript), test mocks, docs, and diagrams. * fix: infinite recursion in _report_tool_result, signal exits, stale docs * fix: add _tool_error_flags to test_load_skill ChatSession stubs |
||
|
Patrick Buckley
|
4f6ef13ce9 |
fix: cancel button race condition with stream abort and force cancel (#202)
The cancel endpoint emitted a 'cancelled' SSE event before the worker thread terminated. The frontend transitioned to "send" mode prematurely, so the next send got rejected with "Already processing a request." Backend: - Providers expose SDK stream handle via cancel_ref parameter so cancel() can close the HTTP connection and unblock iteration - Generation counter prevents orphaned threads from mutating messages or clearing cancel state after force cancel - _check_cancelled() added between retry attempts in _try_stream - Server polls (async, non-blocking) for cancelled worker to exit - Force cancel (force:true) abandons stuck worker, keeps cancel event set so subprocesses are killed, guards against spurious SSE events Frontend: - 'cancelled' shows "Cancelling..." then escalates to "Force Stop" after 2s for a harder cancel that abandons the worker immediately - 10s safety timeout auto-recovers if stream_end never arrives - busy_error re-enables stop button instead of showing send - Timeout cleanup in disconnectSSE, stream_end, and force .then() - Layout shift prevention (min-width, white-space: nowrap) - aria-label updates for accessibility Tests: - 7 new tests: stream close, error suppression, cancel_ref population, transport error conversion, non-cancel exception propagation, retry cancellation check |
||
|
Patrick Buckley
|
c28bfc1e58 |
feat: skill discovery — search and install skills from external sources (#111)
* feat: skill discovery — search and install skills from external sources Add discovery UI and API for finding and installing skills from skills.sh registries and GitHub repositories with one-click install, SKILL.md frontmatter parsing, and security scan integration. Core modules: - skill_parser.py: ParsedSkill dataclass, parse_skill_md() with YAML frontmatter support (Anthropic + Hermes tag formats), name validation - skill_sources.py: SkillsShClient (async search + resolve), fetch_skill_from_github (SKILL.md + bundled resource fetching with 256KB cap, text extension filter, GitHub API tree traversal) API: - GET /v1/api/admin/skills/discover — search with installed annotation and scan_status for installed skills - POST /v1/api/admin/skills/install — fetch, parse, duplicate check, create with origin="source" readonly=true, store resources, audit Also fixes pre-existing bug where _skill_to_response omitted scan_status, scan_report, scan_version fields — scan tier badges in the installed skills table were silently empty despite data existing in storage. Admin UI: pill toggle (Installed/Discover), discovery cards with scan tier badges, GitHub import modal with proper focus trap/Escape/backdrop, scoped selectors preventing MCP↔Skills cross-tab state corruption. SDK: discover_skills() + install_skill() on Python (async+sync) and TypeScript console clients. 48 new tests across 3 test files. All 2632 tests pass. * fix: address copilot review — 404 vs 502, O(n) lookups, branch fallback - SkillNotFoundError subclass: install returns 404 when SKILL.md is missing, 502 only for connectivity/upstream errors - get_skill_by_source_url() + list_installed_skill_urls(): indexed storage lookups replace O(n) full-table scans with content blobs - Default branch fallback: tries main then master when URL doesn't specify a branch - Path normalization: strip trailing slash once, remove redundant candidate - SDK install_skill() returns typed SkillInfo with response_model - Tree size guard: skip resource tree if response >2MB |
||
|
Patrick Buckley
|
75eda9a096 |
feat: unified skills system — merge prompt templates + workstream tem… (#106)
* feat: unified skills system — merge prompt templates + workstream templates Evolves prompt_templates into a first-class skills entity and merges workstream templates into the same model, collapsing two concepts into one. Migration 021: 21 new columns on prompt_templates (skills metadata, security scan fields, session config from WS templates), skill_resources table for bundled files, skill_versions table for auto-snapshot version history. Data migration converts existing WS templates into skills with name collision handling, migrates version history, renames workstreams and scheduled_tasks columns, cleans orphaned permissions, drops old tables. Key changes: - All public interfaces renamed: templates → skills (API, CLI, SDK, UI) - Session config (model, temperature, token_budget, auto_approve, etc.) now lives on the skill and is applied at workstream creation - /skill slash command, set_skill() API, --skill CLI flag - BM25 skill search via SkillSearchManager for activation="search" skills - Admin UI: Skills tab with collapsible Session Config section, description subtitles, activation/origin/MCP badges, pagination - Shared validation helper (_parse_skill_session_config) for DRY CRUD - Version history with auto-snapshot on every edit + API endpoint - Cascade delete (resources + versions) on skill removal - Security: range validation, activation allowlist, fail-closed enabled check, duplicate name 409, readonly guard, JSON validation - 77 new tests across storage, runtime, search, API integration, and migration behavior verification (2521 total) * fix: address Copilot review + rename admin.templates → admin.skills - Skip skill lookup when resume_ws is set (avoids spurious 400) - Fix _applied_skill_version mismatch (1 in both workstreams table and session) - Remove stale template field from MQ protocol diagram - Rename admin.templates permission to admin.skills everywhere (runtime, frontend, tests, docs) with migration step for persisted role data - Fix stale /api/templates references in docs and diagrams - Update docstrings/comments for skills terminology * fix: address Copilot round 2 — skill version lineage + stale doc refs - Compute actual skill version from skill_versions count (not hardcoded 1) - Use same version in both workstreams table and session metadata - Fix response payload example: "templates" → "skills" key - Fix "Each template summary" → "Each skill summary" |
||
|
Patrick Buckley
|
80e1924d7f |
feat: enable prompt caching for Anthropic and OpenAI providers (#104)
* feat: enable prompt caching for Anthropic and OpenAI providers
Activate automatic prompt caching on both LLM providers to reduce input
token costs on multi-turn conversations. Anthropic gets cache_control:
ephemeral (90% savings on cache hits), OpenAI GPT-5.x gets 24h extended
cache retention (free). Cache metrics flow end-to-end through the entire
data pipeline: provider → session → server SSE → MQ protocol → storage →
Prometheus metrics → admin Usage tab.
- AnthropicProvider: top-level cache_control on all requests, extract
cache_creation_input_tokens and cache_read_input_tokens from streaming
and non-streaming responses
- OpenAIProvider: prompt_cache_retention=24h for GPT-5.x, extract
cached_tokens from usage.prompt_tokens_details
- UsageInfo: new cache_creation_tokens and cache_read_tokens fields
- Migration 020: add cache columns to usage_events table
- Storage: record_usage_event and query_usage updated (sqlite + pg)
- Metrics: turnstone_tokens_total{type="cache_creation|cache_read"}
- Server: on_status passes cache tokens to SSE, storage, and metrics
- MQ: StatusEvent carries cache fields through bridge
- SDKs: Python and TypeScript StatusEvent types updated
- OpenAPI: UsageBreakdownItem schema includes cache fields
- Console UI: Usage tab shows cache write/read as secondary readouts
with visual separator, dimmed when zero
- 16 new tests, docs and 3 diagrams updated
* fix: address Copilot review feedback
- Fix MQ protocol diagram clipping by switching to vertical package
layout (inbound on top, outbound below) with package aliases
- Regenerate OpenAPI snapshots to include cache_creation_tokens and
cache_read_tokens on UsageBreakdownItem
- Replace fragile MagicMock(spec=[]) + del pattern with
types.SimpleNamespace in cache metrics missing-attributes test
|
||
|
Patrick Buckley
|
50544c0d1b |
feat: MCP Registry integration — discover and install servers from the official registry
Backend: standalone MCPRegistryClient (httpx async) queries the official MCP Registry API (registry.modelcontextprotocol.io, v0.1). Two new console admin endpoints: GET /v1/api/admin/mcp-registry/search (proxy with installed-status annotation, dedup, uninstallable server filtering) and POST /v1/api/admin/mcp-registry/install (auto-reloads all cluster nodes). Migration 019 adds registry_name/version/meta columns to mcp_servers with partial unique index. Configurable registry URL via mcp.registry_url setting for enterprise/private registries. resolve_install_config() handles both remote (streamable-http) and package (npm→npx, pypi→uvx) installs. Pydantic models, OpenAPI spec, Python + TypeScript SDK methods. Frontend: unified MCP admin tab with Servers/Registry pill toggle (ARIA tablist). Servers view: tri-state source badges (CONFIG/MANUAL/REGISTRY). Registry view: search bar with type filter (remote/npm/pypi), auto-browse on tab switch, result cards with source-type badges and repo links, one-click install for zero-config remotes, install modal with dynamic form for servers needing env vars/headers/URL variables. Package install warning banner. Post-install status polling with connection/error feedback toasts. Trust notice banner linking to the official registry. Safety: 30s connect timeout on streamablehttp_client and session.initialize() prevents hung connections from blocking the MCP event loop indefinitely. Required-only headers in install config prevents empty auth headers from causing silent 401s. 71 new tests (registry client, API endpoints, storage columns). Docs: dedicated docs/mcp-registry.md, updated api-reference, architecture, console, sdk, settings docs. Updated MCP architecture diagram. |
||
|
Patrick Buckley
|
02d9c5c797 |
feat: workstream templates — behavioral profiles for workstream creation (#49)
* feat: workstream templates — behavioral profiles for workstream creation Workstream templates define the complete configuration for workstream creation: system prompt, model, auto-approve policy, per-tool auto-approve, temperature, reasoning effort, max tokens, agent max turns, token budget, and completion notifications. Applied once at creation time (snapshot, not live binding). Auto-versioning captures pre-update state on every edit. Schema & storage: - workstream_templates + workstream_template_versions tables (migration 011) - ws_template_id/ws_template_version columns on workstreams table - ws_template column on scheduled_tasks table - Full CRUD + versioning on SQLite and PostgreSQL backends - prompt_template_hash (SHA-256) for drift detection Runtime: - Template resolution before mgr.create() for model override - Post-creation settings application (prompt, temperature, approval, budget) - Token budget enforcement in session.send() — 80% warning, approval gate at 100% via __budget_override__ synthetic tool - WebUI.auto_approve_tools server-side per-tool auto-approve - Prompt template drift detection (hash comparison, log warning on mismatch) Integration: - ws_template field on CreateWorkstreamMessage, bridge, channel router, scheduler dispatch, MQ client - Console admin "WS Templates" tab (11th) with CRUD, version history modal - Profile dropdown on workstream creation modal - WS template dropdown on scheduler create/edit modals - Prompt template name validation on ws_template create/update - 7 console admin API endpoints + read-only summary endpoint - Full OpenAPI spec entries in console_spec.py - Python SDK (sync + async) and TypeScript SDK methods - Pydantic schemas for all request/response models Docs & diagrams: - New 21-ws-template-architecture.puml sequence diagram - Updated governance, storage, MQ protocol diagrams + PNGs - Updated architecture.md, governance.md, api-reference.md, console.md, sdk.md 48 new tests (1788 total). mypy clean. ruff clean. * fix: address PR #49 review feedback - auto_approve_tools uses approval_label (not just func_name) for consistency with tool policy evaluation - inline system_prompt from ws_template persisted as _ws_template_system_prompt in workstream_config, restored on resume (previously lost because _template_content wasn't persisted) - budget gate (__budget_override__) no longer bypassed by blanket auto_approve — requires explicit approval or tool policy allow - diagram 21 field list corrected (removed tool_search/threshold, added prompt_template_hash/notify_on_complete) * fix: address PR #49 review feedback (round 2) - Grant admin.ws_templates permission in migration 011 (tab was hidden) - Center WS template modals and fix radio button alignment - Skip template validation when ws_template overrides prompt - Guard against empty version snapshots on no-op updates - Replace setTimeout race with Promise chain in schedule ws_template select - Validate numeric fields in admin create/update handlers (400 not 500) - Add ws_template to TypeScript OpenAPI specs - Use typed Pydantic response models in SDK ws_template methods |
||
|
Patrick Buckley
|
fd507c6a3c |
feat: generation cancellation — stop button, cancel API, cooperative … (#40)
* feat: generation cancellation — stop button, cancel API, cooperative cancel Add cooperative cancellation via threading.Event on ChatSession. The cancel signal is set from outside the worker thread (HTTP handler, MQ bridge, or Escape key) and checked at defined checkpoints: per streaming chunk, before tool execution, inside bash commands, and at each sub-agent turn. Core: GenerationCancelled(BaseException) exception, cancel()/_check_cancelled() methods, partial content preservation in _stream_response, clean rollback in send() with idle state emission (no re-raise). Server: POST /v1/api/cancel endpoint, CancelledEvent SSE emission, worker thread safety net. Frontend: Stop button (■ Stop) with send/stop swap via setBusy(), Escape key shortcut, cancelled event handler. Accessible: aria-label, focus-visible override, light theme contrast, non-color differentiation. MQ: CancelMessage inbound type, bridge _handle_cancel routed handler. SDK: cancel() on Python async+sync clients, CancelledEvent in Python+TypeScript event registries, isCancelledEvent type guard. OpenAPI: CancelRequest schema + endpoint spec. Docs: API reference, architecture, SDK docs updated. Diagrams: conversation turn, tool pipeline, MQ protocol, workstream states, SDK architecture. * fix: address PR #40 review feedback - setBusy() now resets stopBtn.disabled so stop button is re-enabled on next generation after a successful cancel - Gate cancel side effects (resolve_approval, resolve_plan, cancelled SSE event) on worker_thread.is_alive() to avoid spurious events when idle - Add /v1/api/cancel endpoint and CancelRequest schema to TypeScript openapi-server.json to keep it in sync with Python-generated spec |
||
|
Patrick Buckley
|
5f0004dc91 |
feat: add ClusterSnapshot for instant console UI state rebuild (#34)
* feat: add ClusterSnapshot for instant console UI state rebuild
The console web UI was SSE-driven with no initial state — reloads and
navigation caused blank/loading gaps while waiting for API re-fetches.
Server-side: GET /v1/api/cluster/snapshot returns the full cluster state
(all nodes with workstreams + overview aggregates) built under a single
lock. The SSE stream now emits this snapshot as the first event on
connect (snapshot taken before listener registration to avoid race).
Frontend: local clusterState object mirrors the snapshot, patched
incrementally by SSE events. View navigation renders from local state
with no API round-trips. Fixes popstate/pushState history corruption
on Back/Forward navigation (pre-existing bug). Stable node sorting
with node_id tie-breaker on both server and client.
SDK: snapshot() method on Python (sync + async) and TypeScript console
clients. ClusterSnapshotEvent in event registries.
* fix: address review feedback and SSE proxy reconnect bug
Copilot review fixes:
- Atomic snapshot+register: new get_snapshot_and_register() acquires
both state and listener locks, eliminating the event gap between
snapshot read and listener registration.
- Debounce patch renders: patchClusterState uses requestAnimationFrame
to batch rapid SSE events into a single recompute+render cycle.
- Fix health type: dict[str, str] → dict[str, Any] on all three
console schema models (ClusterNodeInfo, NodeDetailResponse,
ClusterSnapshotNode) since /health payloads contain nested objects.
- TypeScript ClusterSnapshotEvent: use concrete ClusterSnapshotNode[]
and ClusterOverviewResponse types instead of Record<string, unknown>.
SSE proxy reconnect fix:
- _proxy_sse raw_stream now emits `: proxy-ping` comments every 3s
when no upstream data arrives, preventing the browser EventSource
from dropping idle connections. The raw byte passthrough refactor
(
|
||
|
Patrick Buckley
|
fb190f8977 |
Normalize session_id into ws_id as sole persistent identity (#29)
* Normalize session_id into ws_id as sole persistent identity Eliminate the separate session_id concept. The workstream ID (ws_id) is now the single identity used for both real-time routing and conversation persistence, removing a layer of indirection that was 1:1 in practice and buggy on resume (stale pointers, orphaned rows). Schema changes (migration 006): - Drop sessions table; add alias/title columns to workstreams - Rename conversations.session_id → ws_id - Rename session_config table → workstream_config (ws_id column) - Data migration remaps existing conversations to ws_id Storage/API renames: - register_session → register_workstream (already existed, merged) - save_message/load_messages now keyed by ws_id - resolve_session → resolve_workstream - ChatSession.session_id property → ws_id - ChatSession.resume_session() → resume() - resume_session field → resume_ws - SessionResumedEvent → WorkstreamResumedEvent - /api/sessions → /api/workstreams/saved - /sessions slash command → /workstreams - --session-retention-days → --retention-days Channel eviction recovery simplified: reuses old ws_id directly instead of get_session_id_by_ws() reverse lookup. * Fix Copilot review feedback: stale session wording in docs, regenerate OpenAPI spec - docs/channels.md: "resumes the session" → "resumes the workstream", "Session resumed:" → "Resumed:", "old session was pruned" → "old workstream was pruned" - docs/api-reference.md: "Each session object" → "Each saved workstream object", field descriptions updated, removed stale node_id field - sdk/typescript/openapi-server.json: fully regenerated from Python models — removes all stale session_id properties from WorkstreamInfo, DashboardWorkstream, CreateWorkstreamResponse schemas |
||
|
Patrick Buckley
|
924b976f1f |
Add scheduled task docs and SDK client methods
Add documentation and SDK support for the scheduled task system (cron/at scheduling via console API). Includes Python SDK methods (async + sync), TypeScript SDK methods, console.md API reference, sdk.md table update, and architecture.md module map entry. |
||
|
Patrick Buckley
|
047680d669 |
Add user identity, JWT auth, and admin console UI (#23)
* Add user identity, JWT auth, and admin console UI (#23) JWT-based authentication with three token types: config-file (hmac, backward-compat), API tokens (ts_ prefix, SHA-256 hashed), and JWTs (HS256, 24h expiry). Username:password login via bcrypt. Hierarchical scopes: read < write < approve. New tables: users (username, password_hash), api_tokens (token_hash, scopes, expires), channel_users (future channel integrations). user_id column added to sessions and workstreams for attribution. Console owns admin CRUD (6 endpoints under /api/admin/). Server validates JWTs locally with shared signing secret. Public /api/auth/setup endpoint for first-time admin creation (atomic, only works with zero users). turnstone-admin CLI for user/token management. Admin console UI: Users and Tokens tabs with full CRUD modals, scope badges, token show-once with clipboard copy, keyboard accessibility (focus traps, Escape, arrow key tabs, ARIA roles). Login UI redesigned: username:password primary, token toggle for legacy, setup wizard auto-detected via /api/auth/status. Python + TypeScript SDKs updated with login(username, password), authStatus(), setup(). New docs/security.md + diagram 15-auth-architecture.puml. All existing docs updated. OpenAPI specs include all new endpoints. 64 new tests (1023 total). Dependencies: PyJWT, bcrypt. * Fix auth bugs, XSS vector, and doc inaccuracies from PR #23 review Address Copilot review feedback: escape double quotes in escapeHtml() to prevent XSS in HTML attributes, add JWT validation fallback so config tokens containing dots still work, add user_id to AuthLoginResponse schema, return created field from admin_create_user, and correct five documentation files to match actual API behavior. |
||
|
Patrick Buckley
|
e057c364b8 |
Fix console proxy regressions and add workstream task field (#21) (#21)
* Fix console proxy regressions and add workstream task field (#21) Bug fixes: - Fix collector polling unversioned /api/dashboard (404 after API versioning PR) — nodes showed red/unreachable, no workstreams - Fix SSE proxy dropping all data events — upstream sends \r\n line endings but proxy split on \n\n only; normalize before parsing - Fix workstream state stuck on idle — on_state_change() only broadcasted via SSE but never updated ws.state on the Workstream object; dashboard polling now sees correct attention/running states - Fix deep-link switchTab early return — when ?ws_id matched the only workstream, switchTab bailed (wsId === currentWsId) before establishing SSE connection; inline init instead of delegating - Fix console banner covering dashboard overlay — inject <style> offsetting .dashboard-overlay below the 32px banner Enhancements: - Add turnstone branding to console proxy banner (turnstone │ Console │ node-id) - Add initial_message field to CreateWorkstreamMessage protocol and console "New Workstream" modal (Task textarea, sent as first message) - Refactor SSE proxy to use shared httpx client with 30s read timeout instead of per-request client creation - Increase approval timeout default from 300s to 3600s (1 hour) Updated: Python SDK, TypeScript SDK, OpenAPI specs, MQ client, API schemas, MQ protocol diagram, SDK docs. * Address PR #21 review feedback (4 items) - Log unknown state strings in on_state_change instead of silently swallowing; remove unnecessary KeyError catch - Wrap initial_message POST in _handle_create_ws with error handling so workstream creation success isn't masked by send failure - Strip all \r from SSE chunks instead of replacing \r\n, fixing chunk-boundary split edge case - Add tests for initial_message wiring in directed and pool targeting * Refactor SSE proxy to use httpx-sse aconnect_sse Replace manual SSE chunk buffering/parsing with httpx_sse.aconnect_sse() which handles line endings, event types, and all SSE spec edge cases. Eliminates the \r\n chunk-boundary bug class entirely. Event types are now always forwarded (sse.event defaults to "message" per spec). |
||
|
Patrick Buckley
|
5ee539c983 |
Add Python and TypeScript client SDKs for server and console APIs (#19)
* Add Python and TypeScript client SDKs for server and console APIs Python SDK (turnstone/sdk/) with sync + async clients for both server and console APIs. Returns Pydantic models directly, streams SSE events as typed dataclasses. 27 event types with registry-based deserialization. High-level send_and_wait() for request-response patterns. TypeScript SDK (sdk/typescript/) with zero browser dependencies. Uses fetch + ReadableStream for SSE parsing. Discriminated union event types with type guards. Same API surface as Python SDK. 63 Python tests, 21 TypeScript tests (vitest). Comprehensive docs at docs/sdk.md with SDK architecture diagram. * Address PR #19 review feedback + fix lint - Fix consume_task leak in send_and_wait when send() raises (try/finally) - Fix TS sendAndWait: open SSE before send, plumb AbortSignal for timeout - Add signal param to TS streamSSE for cancellation support - Fix SSE parser: join multi-line data: fields with \n per spec, handle CRLF - Fix generate-types.py sys.path (parents[3] not parents[2]) - Document token ignored when httpx_client provided - Document TS timeout units as milliseconds - Fix stale docstring in test_sdk_sse.py - Fix import sorting (ruff I001) |