* feat: admin Settings tab — form-based editor replacing "coming soon" stub
Section-grouped layout with collapsible headers for all ~40 ConfigStore
settings (model, session, tools, server, mcp, ratelimit, health, judge,
memory). Type-appropriate inputs: CSS toggle for bools, number with
min/max/step, select for choices, text for strings. Secret fields shown
read-only. Source badge (storage/default), amber restart indicator.
Inline save per field with dirty detection, row flash on success, reset
to default via styled confirm modal. Full WCAG keyboard accessibility
(Enter/Space on section headers, aria-labels, focus-visible). Mobile
responsive single-column at <700px. Reduced-motion safe.
* fix: Settings tab polish — help tooltips, context_window auto-detect, UX fixes
Settings UI:
- Help tooltips: ? button on ~25 settings with plain-English explanations
and optional reference links (arXiv, Fowler, MCP spec). Click to toggle
popover, Escape to dismiss, aria-expanded for accessibility.
- Sections start collapsed for scannable overview.
- Restart badge: hidden by default, shows when dirty, persists after save
with amber glow. Positioned left of source badge.
- Secret row alignment fixed (transparent border matches input box model).
- Docs link in toolbar → Swagger UI Settings section.
- Number inputs: spin buttons hidden (Firefox/WebKit), empty value guard,
numeric dirty detection (0.1 vs 0.10 no longer false positive).
- Secret reset button enabled when source=storage (clear legacy overrides).
- Space key repeat guard on section headers.
- Sidebar: sticky + max-height:100vh, no longer stretches with content.
Backend:
- context_window default changed from 131072 to 0 (auto-detect). Fallback
lowered from 131K to 32K (realistic for local models when detection fails).
Session normalizes 0→32768 defensively.
- Settings registry: help + reference_url fields on SettingDef, richer
descriptions for model/session/tools/judge/memory settings.
- Schema API includes help + reference_url.
- Bootstrap system prompt: added Runtime Settings section.
Docs: tab counts updated to 13 across README, architecture, console, governance.
* feat: database-backed settings (ConfigStore) with admin API
Replace config.toml for non-bootstrap settings on the server with a
database-backed ConfigStore. ~40 settings across model, session,
tools, server, mcp, ratelimit, health, judge, and memory sections are
now managed via the admin Settings API. CLI flags for these settings
removed from the server entry point (CLI standalone tool unchanged).
Storage: system_settings table (migration 015) with composite PK
(key, node_id) for per-node overrides. ON CONFLICT upsert in both
SQLite and PostgreSQL. admin.settings permission granted to
builtin-admin role.
Settings registry (settings_registry.py): code-defined catalog of
all known settings with types, defaults, validation, descriptions.
Registry defaults aligned with previous argparse defaults.
ConfigStore (config_store.py): thread-safe in-memory cache loaded
from storage on init. Lock-free reads via dict snapshot swap.
reload() for hot-reload via internal endpoint.
Secret settings (judge.api_key) blocked from write via admin API
(403) — must be configured via config.toml or env vars.
warn_migrated_settings() logs warnings for config.toml keys that
overlap with ConfigStore-managed settings.
Console admin API: GET /v1/api/admin/settings (list with effective
values), GET .../schema (registry catalog), PUT .../{key} (update),
DELETE .../{key} (reset to default). Audit trail on mutations.
MQ: ConfigChangeEvent for cross-node cache invalidation (emission
from console deferred to bridge integration).
Python + TypeScript SDK methods. 63 new tests. Feature docs at
docs/settings.md, PlantUML diagram 24-settings-architecture.
* fix: address PR review — config-reload scope, registry defaults, doc alignment
- config-reload endpoint requires approve scope (was write)
- config-reload handler is sync def (avoids blocking event loop)
- reasoning_effort passes empty string through (removes `or "medium"`)
- ratelimit.requests_per_second changed to float (matches RateLimiter)
- session.retention_days allows 0 (disable pruning)
- ratelimit.trusted_proxies added to registry + wired in server
- admin_list_settings filters to global settings only (no node_id ambiguity)
- admin_update_setting validates "value" key presence (400 if missing)
- Console config change fans out reload to nodes directly (no MQ dep)
- Docs aligned with actual API response shapes and masking ("***")
* feat: [memory] admin panel Memories tab — browse, search, inspect, delete
Add 13th admin tab in the Observe group for cluster-wide memory
management. List view with type/scope filter dropdowns and debounced
search input. Detail modal shows full metadata grid and scrollable
content block. Delete from both list row and detail modal with
confirmation and audit trail.
Permission-gated behind admin.memories. Escape key, backdrop click,
and focus trap wired for the detail modal. Mobile responsive: hides
description and updated columns below 700px.
* fix: memory detail modal — focus, delete safety, CSS shorthand order
Address Copilot review feedback: move focus to close button on modal
open for keyboard accessibility, disable delete button and clear stale
handler during loading/error states to prevent wrong-memory deletion,
and fix font shorthand/font-size ordering in toolbar filter styles.
* feat: [memory] REST API endpoints + SDK methods + docs
Server API (4 endpoints):
- GET /v1/api/memories — list with type/scope/scope_id/limit filters
- POST /v1/api/memories — save (upsert) with validation
- POST /v1/api/memories/search — search by query (read scope)
- DELETE /v1/api/memories/{name} — delete by name+scope
Console admin API (4 endpoints):
- GET /v1/api/admin/memories — list all memories
- GET /v1/api/admin/memories/search — search with ?q= param
- GET /v1/api/admin/memories/{memory_id} — get by ID
- DELETE /v1/api/admin/memories/{memory_id} — delete by ID with audit
Storage: add delete_structured_memory_by_id, add mem_type filter to
count_structured_memories. Auth: memory DELETE requires write scope,
admin.memories permission added to valid set + builtin-admin role.
Python SDK: list_memories, save_memory, search_memories, delete_memory
on both server (async+sync) and console (async+sync) clients.
TypeScript SDK: matching methods + types on both clients.
Pydantic schemas with Literal type/scope validation, OpenAPI endpoint
specs on both servers. 33 endpoint tests + 8 auth scope tests.
Docs: docs/memory.md feature guide, api-reference.md endpoint docs,
23-memory-architecture.puml diagram.
Also fixes stray `total: int` on CreateChannelUserRequest.
* fix: [memory] address PR review — cross-user scope, schema types, snapshots
Security: user-scoped memory endpoints now bind scope_id to the
authenticated user's identity. Providing a mismatched scope_id
returns 403, preventing cross-user memory access on all 4 server
endpoints.
Schema: MemoryInfo response uses MemoryType/MemoryScope Literals.
SearchMemoriesRequest uses filter Literals (empty string allowed).
Limit query params declare schema_type="integer" for correct OpenAPI.
Regenerate sdk/typescript/openapi-{server,console}.json snapshots.
Update count_structured_memories docstring for mem_type param.
Fix fallback response to use normalized name after save.
6 new security tests for user-scope access control.
* feat: MCP cluster-ops example — reference MCP server + SDK implementation
Standalone MCP server under examples/mcp-cluster-ops/ that exposes
tools for executing commands across a Turnstone cluster via the MQ
client SDK. Serves as a reference implementation for both MCP server
patterns (FastMCP, lifespan, tool handlers) and TurnstoneClient usage.
4 tools: list_nodes, run_on_node, run_on_nodes, run_on_all_nodes.
Parallel dispatch via asyncio.gather, raw ToolResultEvent output
capture, UTF-8 safe truncation, input validation, concurrency caps.
35 tests, ruff clean, mypy --strict clean.
* fix: address review feedback on MCP cluster-ops example
- Remove REDIS_SSL support (RedisBroker doesn't accept ssl kwarg)
- Move max-nodes check from _dispatch_parallel into tool handlers
for consistent error shape (always returns {"error": ...} object)
- Propagate KeyboardInterrupt/SystemExit from asyncio.gather instead
of swallowing them as per-node failures
- Fix _truncate omitted bytes count to reflect actual bytes dropped
after multi-byte boundary adjustment
- Apply strip/dedup to node IDs in run_on_all_nodes (matching
run_on_nodes behavior)
- Add __name__ guard to __main__.py
- Fix misleading UTF-8 byte count comment in tests
* feat: structured memory system — typed/scoped memories with BM25 relevance and metacognitive prompting
Replace flat key-value memories table with structured_memories (migration 014).
Four memory types (user/project/feedback/reference), three scopes
(global/workstream/user). Consolidate remember/recall/forget into two tools:
memory (action-based: save/search/delete/list) and recall (conversation
history only).
BM25 relevance scoring (extracted to turnstone/core/bm25.py) selects top-5
memories for system message injection based on conversation context.
Metacognitive prompting injects ephemeral nudges after corrections, tool
denials, workstream resume, and completion signals.
Scope isolation enforced: system message injection and nudge counts filtered
to visible memories only (global + current workstream + authenticated user).
User scope requires authentication. Content capped at 32KB. ILIKE/LIKE
metacharacters escaped in both backends.
113 new tests (2053 total).
* fix: CI failure + copilot review feedback
- Fix time.monotonic() cooldown: use None sentinel instead of 0.0
default (monotonic clock starts at boot, not epoch — fresh CI
runners have uptime < 300s so cooldown check always triggered)
- Catch sa.exc.IntegrityError specifically in upsert instead of
broad Exception (copilot review)
- Preserve existing description/type on upsert when caller doesn't
explicitly set them (copilot review)
- Add last_accessed + access_count columns to schema/migration for
future LRU/LFU eviction support
* feat: admin panel — right-aligned sidebar navigation with two-column modals
Replace the horizontal tab bar (11 tabs, overflowing on standard monitors)
with a grouped sidebar on the right side, matching the admin button's
position in the header for natural spatial flow.
Sidebar: 5 groups (Identity, Automation, Governance, Observe, System) with
12 nav items including new Settings stub. Always visible on desktop (180px),
off-canvas drawer on mobile (<700px) sliding from right with backdrop.
Admin button: toggle behavior (click again to return to overview), active
state with amber highlight + top accent line, aria-expanded management.
Breadcrumb: shows active tab ("Admin / Users", "Admin / Audit", etc).
Modals: WS Template and Schedule create/edit forms restructured into
two-column grid (820px) with "Identity"/"Model Config" and
"Schedule"/"Execution" column headings. All modals gain max-height: 85vh
+ overflow-y: auto safety net. Modal z-index bumped to 600 (above sidebar).
Also: "Tokens" renamed to "API Tokens", redundant "Server default"
placeholders removed from model config fields, view fade-in transition,
comprehensive ARIA (grouped sidebar, aria-hidden on mobile, focus return
on drawer close), reduced-motion support.
* fix: address Copilot review — aria-orientation, settings permission gate, inert sidebar
- Add aria-orientation="vertical" to sidebar tablist for assistive tech
- Gate Settings tab behind admin.users permission so empty-state logic
works correctly when user has no admin permissions
- Use inert attribute on mobile sidebar when closed to prevent keyboard
focus from reaching off-canvas controls
- Add resize listener to sync aria-hidden/inert when crossing the
700px mobile breakpoint
* fix: simplify conversation storage — atomic assistant rows with tool_calls JSON
Replace the denormalized storage model (separate rows for assistant
content, tool_call, tool_result) with atomic assistant rows carrying
tool_calls as a JSON column. Eliminates the 100-line heuristic
reconstruct_messages function and its cross-turn merge bug.
Schema: add tool_calls TEXT column to conversations (migration 013).
Migration backfills existing data — merges tool_call rows into their
parent assistant row as JSON, renames tool_result to tool, deletes
consumed tool_call rows.
Session save path: assistant content + tool_calls saved in one
save_message call before tool execution (crash resilient). Tool
results saved as role="tool".
Extract shared storage utilities to _utils.py: row_to_dict, mutable
field frozensets, reconstruct_messages. Both backends import from
_utils — PostgreSQL no longer depends on _sqlite.py.
Includes denied/blocked tool call badge fix on resume: _build_history
detects denied results and propagates flag to parent assistant entry.
Frontend uses flag for correct badge-denied rendering. Denied tools
visually muted. role="status" on badges for accessibility.
Net -45 lines. 8 new tests for reconstruction, all 1914 tests pass.
* fix: migration 013 uses parameterized deletes and ordered downgrade
- DELETE of consumed tool_call rows now uses parameterized batches
(chunks of 500) instead of string interpolation
- Downgrade rebuilds via temp table to preserve chronological id
ordering when re-inserting tool_call rows
* feat: intent validation v1 — advisory LLM judge for tool approvals (#50)
Two-tier evaluation pipeline for non-auto-approved tool calls:
- Heuristic tier (instant): 23 pattern-based rules across 4 severity
levels (critical/high/medium/low) with first-match-wins priority
- LLM judge tier (async): multi-turn evaluation with read_file/
list_directory tool access, security-hardened path blocking, forcing
message on final turn, four-stage JSON parsing with retry nudge
Progressive UI: heuristic verdict badge + judge spinner, LLM verdict
upgrade via intent_verdict SSE event, glow on action buttons. Verdict
persisted to intent_verdicts table for audit. Prometheus metrics for
verdict counts and LLM latency. Enabled by default (--no-judge to opt
out). 132 new tests (1938 total).
Integration: session, server/WebUI, CLI, MQ bridge, console admin API,
Discord channel adapter. Config via [judge] in config.toml or CLI flags.
* fix: address PR #50 Copilot review feedback
- Fix double JSON encoding of func_args in both heuristic and LLM
verdict persistence paths — use pre-serialized string from verdict
- Fix confidence 0.0 treated as falsy in channel verdict formatter
- Fix timestamp format inconsistency in storage backends (isoformat
vs strftime) — now uses strftime consistently
- Add on_intent_verdict to eval.py NullUI (mypy fix)
- Fix late verdict after approval resolved — store last decision and
apply immediately to late-arriving verdicts
- Add permission rollback to migration 012 downgrade
- Update docs to reflect judge enabled by default
- Document confidence_threshold as reserved for v2
* fix: judge per-call timeout and credential recon heuristic
- Wrap create_completion() in ThreadPoolExecutor with per-call timeout
to prevent indefinite hangs on slow local models. On timeout, replace
the executor so subsequent batch items don't queue behind lingering
API calls
- Add IntentJudge.shutdown() and wire into session.close() for cleanup
- Add credential-recon heuristic rule: /etc/passwd, /etc/shadow,
/etc/master.passwd access flagged as HIGH/review (reconnaissance
pattern even though the command itself is read-only)
- 3 new tests for credential file access patterns
* fix: denied/blocked tool calls show correct badge on resume
- _build_history() detects denied results ("Denied by user") and
blocked results ("Blocked") and propagates denied flag to parent
assistant entry for frontend consumption
- Frontend history replay uses denied flag for badge-denied class
instead of hardcoding badge-approved for all historical tool calls
- Denial feedback always prefixed with "Denied by user:" so content
detection works with custom user feedback
- Denied tools visually muted (opacity 0.55, muted tool name)
- role="status" on all approval badge elements (accessibility)
- Broadened "Blocked" prefix match (catches "Blocked by tool policy")
* feat: wire prompt templates into session startup with full creation-path support
Prompt templates (prompt_templates table) now have runtime effect:
- is_default=true templates auto-apply as system message content,
concatenated in name order before user instructions
- Per-workstream template selection via --template CLI flag, template
field on POST /v1/api/workstreams/new, console creation modal dropdown,
scheduled task config, and channel adapter config
- {{model}}, {{ws_id}}, {{node_id}} variable substitution via single-pass
regex (prevents cross-variable injection)
- /template slash command for runtime switching, persisted across resume
- set_template() public API on ChatSession
Security hardening:
- MCP sync resets is_default=False on content update (prevents compromised
server from injecting defaults)
- 32KB content cap on template create/update + defensive truncation
- Template existence validation returns 400 before workstream creation
- Single-pass regex eliminates cross-variable expansion
Template field plumbed through all creation paths: CLI, server API, MQ
protocol/bridge, console backend, scheduler dispatch, channel router,
MQ client. Migration 010 adds template column to scheduled_tasks.
Frontend: console workstream modal template dropdown, scheduler
create/edit template field, governance template UI variables auto-detected
from content (read-only display replaces editable input). Focus trap and
Enter-key accessibility fixes in workstream modal.
Docs: governance.md template runtime section, api-reference.md template
field, governance + MCP architecture diagrams updated.
Python + TypeScript SDKs, Pydantic schemas all updated. 29 new tests.
* fix: address PR #47 review feedback
- Defer template validation until after resume_ws — a bad template name
no longer 400s when resume would have ignored it anyway
- Add template field to OpenAPI JSON specs (openapi-server.json,
openapi-console.json) for SDK/docs consistency
- Validate template existence in schedule create and update endpoints —
reject unknown template names with 400 instead of allowing schedules
that would silently fail at dispatch time
Add ddgCluster profile extending the 10-node cluster with a DuckDuckGo
Search MCP sidecar. All cluster nodes connect via streamable-http and
gain duckduckgo_web_search + duckduckgo_fetch_content tools. No API
key required.
Key implementation details learned during testing:
- MCP SDK DNS rebinding protection must be disabled for Docker
internal networking (Host header uses container names)
- FastMCP server binds to 127.0.0.1 by default; must set
mcp.settings.host='0.0.0.0' for cross-container access
- DDG CLI lacks --host/--port flags; settings configured via Python
entry point that patches FastMCP.settings directly
- Safe search disabled by default
Also adds MCP_CONFIG env var support to all server commands (shell
conditional, no-op when empty) and moves default server/bridge to
production profile for cleaner profile separation.
* fix: MCP resource template URI expansion via prefix matching
Resource templates (RFC 6570 URI patterns like `db://tables/{table}/rows/{id}`)
were discovered from MCP servers but non-functional — `read_resource_sync()`
only accepted exact URIs from `_resource_map`, which excludes templates.
Add prefix-based fallback: extract the static prefix from each template
(everything before the first `{`), store a prefix→server mapping, and
fall back to longest-prefix matching when exact URI lookup fails. MCP
servers handle URI routing internally so we just need to route the
expanded URI to the correct server.
Also surface templates in the system message catalog and `/mcp` command
so the model knows they exist and can construct expanded URIs.
* fix: address PR #46 review feedback
- Template prefix collision now keeps more specific (longer) template
URI instead of blindly overriding
- Fix _match_template docstring to accurately describe startswith
matching on static prefixes (not full template matching)
- Add missing loop.close() in integration test finally block
- Rewrite test_template_longest_prefix_wins with genuinely different
prefix lengths to avoid brittle collision-order dependency
* feat: MCP resource and prompt discovery with read_resource tool
Extends MCPClientManager with resource and prompt discovery alongside
existing tool support. Resources and prompts are discovered on connect,
cached per-server with copy-on-write rebuilds, and refreshed via push
notifications, periodic polling, or manual /mcp refresh.
New read_resource built-in tool reads MCP resources by URI. Requires
user approval (same as MCP tool calls) since resources are served by
external MCP servers. Resource catalog injected into system message
with XML delimiters. Error messages sanitized to prevent leaking
server internals to the model.
Prompt discovery stores prefixed names (mcp__server__prompt) and
exposes get_prompt_sync() for future use_prompt tool (Chunk D).
/mcp command now shows tools, resources, and prompts. Docs and
diagrams updated.
* feat: MCP prompt governance sync with origin tracking and readonly guards
Migration 009 adds origin, mcp_server, and readonly columns to
prompt_templates. MCP prompts discovered by MCPClientManager are
automatically synced into the governance table as read-only templates
with origin="mcp".
Sync engine handles: create on connect, update on prompt refresh,
delete when prompts are removed from server. Manual templates take
precedence on name collision (MCP prompt skipped with warning).
Admin API returns 403 on update/delete of readonly templates. Console
UI shows MCP origin badge and disables edit/delete buttons. Storage
backends gain get_prompt_template_by_name, list_prompt_templates_by_origin,
and delete_prompt_templates_by_server methods.
Also addresses PR #44 review feedback: concurrent.futures.TimeoutError
handling in sync dispatch, XML-escape resource catalog descriptions,
resource template entries excluded from _resource_map, URI collision
warnings, needs_periodic capability-aware computation, malformed JSON
primary key fallback for read_resource.
* feat: use_prompt tool, prompt catalog, and PR review hardening
New use_prompt built-in tool invokes MCP prompt templates by name,
expanding them into messages. Requires user approval (external MCP
servers). Prompt catalog injected into system message with XML
delimiters (up to 30 prompts, HTML-escaped).
Prompt listener registered in session for catalog rebuild on changes.
Addresses PR #44 review feedback:
- _init_system_messages() now uses copy-on-write (build locally,
assign atomically) so background thread callbacks never see
partial system messages
- sync_prompts_to_storage() serialized behind _sync_lock to prevent
races between set_storage() (main thread) and MCP background thread
- shutdown() clears listener lists to release callback references
Docs and diagrams updated for 18 built-in tools.
* feat: granular tool policies for MCP resources, prompts, and tools
Policy evaluation now uses approval_label (falling back to func_name)
for fnmatch pattern matching, enabling fine-grained per-URI and
per-server policies:
- read_resource: mcp_resource__{normalized_uri}
- use_prompt: mcp__{server}__{prompt} (prefixed name)
- MCP tools: mcp__{server}__{tool} (was static "mcp_tool")
URI normalization resolves .. path segments to prevent traversal
bypasses in policy matching. Resource templates filtered from system
message catalog (not directly readable). use_prompt arguments
validated as dict with string coercion.
TypeScript SDK PromptTemplateInfo gains origin, mcp_server, readonly
fields. Governance docs updated with MCP policy patterns.
* feat: MCP visibility in server and console UIs
Server health endpoint includes mcp.servers, mcp.resources, mcp.prompts
counts. Server UI status bar shows magenta MCP indicator with tooltip.
Console cluster status bar shows MCP metrics with magenta LED dot.
Console node detail view shows per-node MCP summary. Console collector
aggregates MCP counts across nodes in overview.
Uses var(--magenta) design token with new --magenta-glow for theme
adaptation. ARIA roles on MCP status elements. Tooltips on console
MCP metric labels. Node MCP summary hidden on mobile (< 700px).
New diagram: 20-mcp-architecture.puml covering full MCP lifecycle
(connection, discovery, refresh, governance sync, policy, UI).
* fix: McpStatus in health schema, count properties, catalog name fidelity
Adds McpStatus model to HealthResponse (Python + TypeScript SDKs) so
typed clients see the mcp field from /health.
Addresses Copilot review feedback:
- resource_count/prompt_count properties avoid list allocation on
/health and /metrics polls
- get_tools/resources/prompts return shallow-copied dicts to prevent
callers from mutating internal cache
- Prompt names and arg names in system message catalog are NOT
HTML-escaped (model must use exact strings in use_prompt calls);
only descriptions are escaped
* fix: OpenAPI spec McpStatus + diagram approval column accuracy
Adds McpStatus schema and optional mcp field to HealthResponse in
openapi-server.json, matching the Python schema and TypeScript types.
Fixes tool pipeline diagram: math, web_fetch, web_search correctly
shown as auto-approve (not "Yes" for approval).
* fix: channel bidirectional routing — emit TurnCompleteEvent on all idle transitions
Bridge previously only emitted TurnCompleteEvent for MQ-initiated turns
(those with a correlation_id in _active_sends). Server-UI-initiated turns
went idle without emitting TurnCompleteEvent, so the Discord bot's
StreamingMessage never finalized — content accumulated in the buffer and
collided with the next Discord-triggered response.
Now TurnCompleteEvent is emitted unconditionally on every idle transition.
correlation_id is empty for non-MQ turns; SDK client filters by
correlation_id so existing consumers are unaffected.
* fix: remove unused variable flagged by ruff
* fix: approval timeout UI state and content flush before tool calls
Two bug fixes:
1. Approval timeout now shows denied state in UI — resolve_approval()
emits an approval_resolved SSE event so the browser transitions
from pending to denied (red border + badge). Also fixes the cancel-
during-approval path. Frontend resolveInlineApproval() gains a
skipPost parameter to avoid redundant POST when server-initiated.
ApprovalResolvedEvent added to Python and TypeScript SDKs.
2. Content streaming flushes pending buffer before tool call deltas —
_stream_response() held up to 13 trailing chars in the pending
buffer (for <think> tag detection) when transitioning to tool calls.
Now flushed eagerly when tool_call_deltas arrive, before clearing
in_think so reasoning text is correctly categorized.
* fix: address Copilot review feedback on PR #42
Patch _execute_tools in stream flush test to prevent real bash execution,
simplify confusing nested comprehension, and update resolve_approval()
docstring to reflect cancel/timeout call paths.
* feat: robust plan quality gate, iterative refinement, and amend UX
Plan agent output from weak models often produced garbage (11-char plans
that echo the prompt). Two fixes:
1. Quality validation (_validate_plan) checks length, section structure,
echo detection, and refusal patterns. Fails trigger one automatic
retry with a coaching message injected into the agent's existing
conversation, preserving all prior exploration context.
2. Iterative feedback loop — user feedback at plan review re-runs the
plan agent via _refine_plan() instead of appending text to the tool
result. Up to 5 refinement rounds. The plan file path is always
included in the tool result so the outer model knows where it lives.
UI improvements:
- Web: Reject button dynamically becomes "Amend" (amber) when feedback
is typed. Key hint badges (Esc/Enter) on plan buttons. Main input
disabled during review. Light-theme contrast fix via --on-color var.
- CLI: Prompt shows all three actions (approve/amend/reject).
- Bridge: Race condition fix — clear pending entry before HTTP POST so
sequential plan reviews from the refinement loop aren't skipped.
15 new tests covering validation, retry, and refinement.
* fix: address PR 41 review feedback
- Escape key in plan dialog now mirrors the Amend button: if feedback is
typed, Esc sends the feedback (amend); if empty, Esc rejects. Previously
Esc always hard-coded "reject", discarding typed feedback.
- Coaching message for plan retry now says "should include at least two of"
instead of "MUST include these", matching the actual validation rule
(_MIN_PLAN_SECTIONS = 2).
* feat: render plan inline in chat after approval
After the plan review dialog closes, the plan content is now rendered
as a collapsible inline block in the chat stream — styled with a
status header (approved/rejected/amending), markdown-rendered body,
and feedback note when amending. Uses the same makeCollapsible pattern
as tool output blocks.
* fix: prevent plan approval hang when inline render fails
The authFetch call that unblocks the server must fire before the
cosmetic inline plan rendering. Previously _addInlinePlan ran first
and any JS error (e.g. from renderMarkdown) prevented the API call,
leaving the session thread blocked forever.
- Move authFetch before _addInlinePlan
- Wrap _addInlinePlan in try-catch
- Guard against empty content
- Only auto-collapse plans longer than 12 lines
* fix: address PR 41 review feedback (round 2)
- Max refinement rounds no longer implicitly approve: the loop now
shows the final plan for explicit approve/reject before proceeding.
Previously exhausting 5 rounds silently accepted the last revision.
- Plan inline block: correct aria-label from "Tool output" to
"Plan content" when makeCollapsible is applied.
- XSS concern (not applicable): renderMarkdown is used for all
assistant messages — plan content follows the same trust model.
- Test loop concern (acknowledged): refinement tests verify component
logic; full _execute_tools integration would require extensive
mocking for marginal coverage gain.
* feat: thinking spinner + inline plan hardening
* fix lint
Add `turnstone-bootstrap`, a new entry point that uses any LLM (OpenAI,
Anthropic, or local/vLLM) to conversationally walk users through
configuring a Turnstone deployment. Generates .env files, setup.sh
scripts, and optional docker-compose overrides.
- Fully interactive startup (zero CLI args) with provider/model selection
- Auto-detects available models on local OpenAI-compatible endpoints
- 7 tools: read_file, write_file, generate_secret, check_port,
validate_api_key, check_docker, finish
- Path traversal protection on file read/write
- Duplicate write detection (skips identical content)
- Bounded retry loop (3 attempts) on LLM errors
- Anthropic message conversion with consecutive-role merging
* feat: generation cancellation — stop button, cancel API, cooperative cancel
Add cooperative cancellation via threading.Event on ChatSession. The cancel
signal is set from outside the worker thread (HTTP handler, MQ bridge, or
Escape key) and checked at defined checkpoints: per streaming chunk, before
tool execution, inside bash commands, and at each sub-agent turn.
Core: GenerationCancelled(BaseException) exception, cancel()/_check_cancelled()
methods, partial content preservation in _stream_response, clean rollback in
send() with idle state emission (no re-raise).
Server: POST /v1/api/cancel endpoint, CancelledEvent SSE emission, worker
thread safety net.
Frontend: Stop button (■ Stop) with send/stop swap via setBusy(), Escape key
shortcut, cancelled event handler. Accessible: aria-label, focus-visible
override, light theme contrast, non-color differentiation.
MQ: CancelMessage inbound type, bridge _handle_cancel routed handler.
SDK: cancel() on Python async+sync clients, CancelledEvent in Python+TypeScript
event registries, isCancelledEvent type guard.
OpenAPI: CancelRequest schema + endpoint spec.
Docs: API reference, architecture, SDK docs updated. Diagrams: conversation
turn, tool pipeline, MQ protocol, workstream states, SDK architecture.
* fix: address PR #40 review feedback
- setBusy() now resets stopBtn.disabled so stop button is re-enabled on
next generation after a successful cancel
- Gate cancel side effects (resolve_approval, resolve_plan, cancelled SSE
event) on worker_thread.is_alive() to avoid spurious events when idle
- Add /v1/api/cancel endpoint and CancelRequest schema to TypeScript
openapi-server.json to keep it in sync with Python-generated spec
* feat: governance — RBAC, tool policies, prompt templates, usage tracking, audit logging
Add comprehensive governance layer for the admin console:
- RBAC with 15 granular permissions, 3 builtin roles (admin, operator, viewer),
custom role CRUD, user-role assignment with privilege escalation prevention
- Tool policies with glob pattern matching, priority-ordered evaluation
(allow/deny/ask), enforced before auto-approve in WebUI.approve_tools()
- Prompt templates with variable substitution, categories, default flag
- Usage tracking: per-LLM-request token/tool metrics, aggregated queries
(group by day/model/user), automatic 90-day pruning via scheduler
- Audit logging: append-only event trail for all admin mutations,
filterable/paginated queries, automatic 365-day pruning, X-Forwarded-For
aware IP extraction
- require_permission() enforced on all 35+ admin endpoints (users, tokens,
channels, schedules, watches, roles, orgs, policies, templates, usage, audit)
- Field allowlists on storage update methods prevent mass-assignment bugs
- Self-deletion guard on admin_delete_user, delete_user cascades user_roles
- _row_to_dict helper eliminates ~400 lines of fragile positional row mapping
- _audit_context helper deduplicates 18 instances of audit boilerplate
- Migration 008: 7 new tables, 3 builtin roles, org_id on users
- Console admin panel: 5 new tabs (Roles, Policies, Templates, Usage, Audit)
with permission-gated visibility, 7 modal dialogs, full keyboard accessibility
- Python + TypeScript SDK methods for all governance endpoints
- 120+ new tests (1554 total)
* fix: address PR #39 review feedback
- Rebuild serialized items after policy evaluation so denied/allowed
verdicts are reflected in tool_info/approve_request SSE payloads
- Make `since` query param optional in usage OpenAPI spec (handler
already defaults to last 7 days)
- Add response_model=StatusResponse to DELETE role/policy/template
and POST/DELETE role assignment endpoints in OpenAPI spec
- Add missing org_id/created/updated fields to UserRoleInfo schema
- Add missing created field to AuditEventInfo schema
- Show "no permissions" empty state instead of loading inaccessible
tab when all admin tabs are permission-gated
- Fix "13 permissions" → "15 permissions" in architecture.md and
security.md
- Fix import sorting in test_audit.py and test_tool_policy.py
* fix: address PR #39 round 2 review feedback
- Clear stale permissions from sessionStorage on config-token login
(auth.js _storePermissions)
- Only trust X-Forwarded-For when behind a proxy that sets
X-Forwarded-Proto (conditional on is_secure_request trust model)
- Thread user_id from auth into WebUI.on_status for usage events
- Add created field to TS AuditEventInfo type
- Return typed Pydantic models from all SDK governance methods instead
of dict[str, Any] — both async and sync clients
- Validate group_by param against allowed enum in admin_usage handler
- Add deterministic secondary sort (event_id DESC) to
list_audit_events in both SQLite and PostgreSQL backends
Agent tool outputs are now truncated to 16k chars to prevent search
results (14M+ chars observed) from blowing past the model's context
limit. On context-exceeded API errors, the agent returns its last
content instead of crashing.
Plan agent: own identity only (no base system prompt needed).
Task agent: base system prompt merged with task identity into a single
system message (needs tool patterns for tool execution).
Neither agent receives conversation history.
Fixes Jinja template error on Qwen models that reject system messages
appearing after non-system messages.
* fix: per-workstream SSE fan-out — multiple consumers no longer steal each other's tokens
After 4d665a5 removed the single-consumer SSE lock, the shared
_event_queue let concurrent consumers (browser, bridge, console proxy)
race on Queue.get(), each receiving ~1/N of content tokens and producing
garbled streaming text.
Replace the single queue with per-client fan-out: each SSE connection
registers its own bounded queue (maxsize=500) on WebUI._listeners, and
_enqueue() copies every event to all registered queues. On eviction or
close, a ws_closed sentinel is injected so SSE generators exit promptly.
* fix: address CI failures and Copilot review feedback
- Handle ws_closed sentinel in events_sse generator (break on close)
- Guarantee sentinel delivery by evicting one item when queue is full
- Clear listeners list after injecting sentinels on cleanup
- Fix test_slow_consumer to fill only slow queue directly
- Fix ruff SIM117 (nested with), unused import, mypy unused-ignore
* ci: add GitHub Release creation on tag push
* refactor: rename plan tool to create_plan
Rename plan → create_plan to resolve cross-provider tool selection
failures. Models consistently treated "plan" as a reasoning concept
rather than a callable tool. The new name is an unambiguous verb+noun
action. Also rename the parameter from prompt → goal for clarity,
add web_search to the default system prompt tool patterns
* feat: eval harness improvements inspired by autoresearch patterns
Major enhancements to turnstone-eval:
- Per-test timeout (--test-timeout, default 300s) and suite timeout
(--suite-timeout) prevent stuck runs from blocking the suite
- Fast-fail skips remaining runs after ceil(n/2) consecutive zeros
- Summary table with colored PASS/WEAK/FAIL and append-only TSV output
- Progress reporting with running pass rate, token count, and ETA
- Parallel test execution via ProcessPoolExecutor (--parallel N)
- Per-role model assignment: test/optimizer/observer can use different
models and providers (--optimizer-model, --observer-model, etc.)
with auto-detection from base URL
- Improved optimizer and observer system prompts with structured
failure-mode diagnosis, keep/discard rules, and trend analysis
- Fixed token counting (prompt tokens use last-turn value, not sum)
- Added math-calculation and web-search-query test cases
- Fixed multi-file-edit test (both files now contain the target string)
* fix: address Copilot review feedback
- Revert prompt token counting to sum (reflects billed usage)
- Add tool_args to fast-fail skipped run dicts for schema consistency
- Align approval_label with func_name ("create_plan")
- Add timeout to future.result() in parallel path (test_timeout + 30s)
- Document thread-leak trade-off on serial timeout path
* feat: watch tool — periodic command polling within workstreams
Add a new `watch` tool that lets the model (or user) set up periodic
polling of a shell command. Results inject as synthetic user messages
that trigger LLM turns, enabling reactive workflows like PR monitoring,
CI/CD status tracking, and deployment health checks.
Key design:
- Single tool with create/list/cancel actions
- Python expression DSL for stop conditions (restricted eval)
- Server-owned WatchRunner daemon (DB-persisted, survives eviction + restart)
- Three dispatch paths: idle, busy, and evicted workstream restore
- REST API for console visibility (GET /v1/api/watches, POST cancel)
- Migration 007, 8 storage CRUD methods, 75 new tests (1383 total)
* fix: address Copilot review — condition errors, restore deadlock, docs
- Condition eval errors now deactivate the watch immediately instead
of silently looping until max_polls
- Restored (evicted) workstreams set auto_approve=True to prevent
approval deadlocks with no connected user
- Tool description clarifies first-poll baseline behavior for change
detection mode
- Diagram updated: DELETE → POST /v1/api/watches/{id}/cancel
The _sse_generation mechanism assumed one SSE consumer per workstream,
but the bridge also maintains an SSE connection to each workstream.
When a new client connected (browser, proxy, or test), it incremented
the generation counter, killing the bridge's connection. The bridge
reconnected, killing the new client's connection — creating a
mutual-kill cascade that closed every SSE connection after one ping
cycle (5s).
Fix: remove _sse_generation entirely. sse-starlette handles disconnect
detection via its own ASGI task. Also remove the redundant
request.is_disconnected() check which raced with sse-starlette's
disconnect listener in Starlette 0.52.
Root cause confirmed via raw socket test: the server was sending
a zero-length chunked terminator (0\r\n\r\n) at exactly 5s,
cleanly ending the HTTP response body.
* fix: recovered workstreams invisible in console UI
Bridge startup recovery (_recover_workstreams) re-registered workstream
ownership but never published WorkstreamCreatedEvent to the cluster
channel. The collector's poll loop would pick up the workstream in its
internal state, but _apply_poll never fanned out SSE events to connected
browsers. Combined, this made channel-resumed workstreams invisible in
the console while remaining accessible through the proxied node UI.
- Bridge: emit WorkstreamCreatedEvent for each recovered workstream
- Collector: diff poll results and fan out synthetic ws_created/ws_closed
events for workstream additions and removals
- Skip workstreams with empty IDs in poll processing
- Add 4 tests for poll-diff fanout behavior
- Update console data-flow diagram and architecture docs
* fix: address PR review — filter empty ws IDs, stable event ordering
- Filter empty-string keys from old_ids to avoid phantom ws_closed
events if a previous poll inserted a workstream under key "".
- Sort set diffs before iterating so ws_created/ws_closed fanout
order is deterministic across poll cycles.
* feat: add ClusterSnapshot for instant console UI state rebuild
The console web UI was SSE-driven with no initial state — reloads and
navigation caused blank/loading gaps while waiting for API re-fetches.
Server-side: GET /v1/api/cluster/snapshot returns the full cluster state
(all nodes with workstreams + overview aggregates) built under a single
lock. The SSE stream now emits this snapshot as the first event on
connect (snapshot taken before listener registration to avoid race).
Frontend: local clusterState object mirrors the snapshot, patched
incrementally by SSE events. View navigation renders from local state
with no API round-trips. Fixes popstate/pushState history corruption
on Back/Forward navigation (pre-existing bug). Stable node sorting
with node_id tie-breaker on both server and client.
SDK: snapshot() method on Python (sync + async) and TypeScript console
clients. ClusterSnapshotEvent in event registries.
* fix: address review feedback and SSE proxy reconnect bug
Copilot review fixes:
- Atomic snapshot+register: new get_snapshot_and_register() acquires
both state and listener locks, eliminating the event gap between
snapshot read and listener registration.
- Debounce patch renders: patchClusterState uses requestAnimationFrame
to batch rapid SSE events into a single recompute+render cycle.
- Fix health type: dict[str, str] → dict[str, Any] on all three
console schema models (ClusterNodeInfo, NodeDetailResponse,
ClusterSnapshotNode) since /health payloads contain nested objects.
- TypeScript ClusterSnapshotEvent: use concrete ClusterSnapshotNode[]
and ClusterOverviewResponse types instead of Record<string, unknown>.
SSE proxy reconnect fix:
- _proxy_sse raw_stream now emits `: proxy-ping` comments every 3s
when no upstream data arrives, preventing the browser EventSource
from dropping idle connections. The raw byte passthrough refactor
(4d11078) removed the proxy's independent keepalive — this restores
it without reverting to EventSourceResponse.
* feat: add vision/image support to read_file tool
read_file now detects image files (PNG, JPEG, GIF, WebP, BMP, TIFF, ICO)
and returns base64-encoded content parts for vision-capable models.
Non-vision models receive a text description instead. A new
supports_vision flag on ModelCapabilities gates the feature, with
config.toml [models.*.capabilities] overrides for local models
(vLLM, llama.cpp, NIM).
* fix: address PR review feedback
- Discard _read_files on no-vision OSError path, include exception detail
- Discard _read_files on oversized image error (not a successful read)
- Validate capabilities type from config.toml (reject non-dict)
- Clarify tool description re: vision behavior and offset/limit scope
- Remove unused os import in tests, fix import sort order
- Handle list content (image tool results) in eval.py tool result loop
Patrick Buckley