* fix: drop orphaned tool_results with no matching tool_use in _convert_messages
The context window increase from 200K to 1M for Claude 4.6 means
conversations that previously triggered auto-compaction now send their
full history. Older messages with orphaned tool_results (from
pre-fix cancels or compaction boundaries) are now visible to the API,
causing "unexpected tool_use_id in tool_result blocks" errors.
The existing repair code handles orphaned tool_use (synthesizes
missing results), but not the reverse. Now validates each
tool_result against the preceding assistant message's tool_use IDs
and silently drops results with no match.
* fix: filter empty IDs from prev_tool_use_ids, document pass-through
Code review: empty-ID tool_use blocks were added to the filter set,
and the intentional pass-through when prev_tool_use_ids is empty
needed documentation.
* fix: block math sandbox escape via getattr/setattr/type reflection
getattr() with runtime-constructed strings bypassed the AST validator,
allowing full os/subprocess access from the sandboxed math tool via
module.__builtins__['__import__']('os').
Three-layer fix:
- Block getattr, setattr, delattr, type, __import__ in
_MATH_BLOCKED_BUILTINS (prevents direct calls)
- Add AST validation for getattr/setattr/delattr call nodes
(catches them even if builtins dict is bypassed)
- Strip __builtins__ from all pre-imported modules in the execution
namespace (runtime defense — even if AST is somehow bypassed,
module.__builtins__ returns empty dict)
Normal math, sympy, numpy, scipy operations unaffected.
* fix: harden _safe_import to strip __builtins__ from runtime imports
Copilot review: modules imported at runtime via _safe_import still
had their original __builtins__ dict, accessible via
operator.attrgetter('__builtins__'). Now _safe_import strips
__builtins__ from every module it returns. Also blocks
operator.attrgetter/itemgetter at the AST level, and removes the
redundant duplicate getattr check in visit_Call.
* fix: add type ignore for module __builtins__ assignment
* fix: block /proc/*/environ access in bash filter and judge heuristic
/proc/1/environ leaks the full server environment including DB
credentials, API keys, and JWT secrets. Env scrubbing in env.py
only affects subprocess calls, not procfs reads.
- Add /proc/1/environ and /proc/self/environ to BLOCKED_PATTERNS
in safety.py (hard block)
- Add proc-environ-exfil heuristic rule at critical severity with
deny recommendation (catches /proc/<pid>/environ patterns)
* fix: move proc-environ-exfil rule to _CRITICAL_RULES list
Copilot review: rule had risk_level=critical but was placed in
_HIGH_RULES. Move to _CRITICAL_RULES for consistency with the
first-match-wins severity ordering.
The intent judge was receiving up to 50% of the context window in
conversation history (FIFO from end), which grows linearly with
conversation length and causes increasing latency. The judge only
needs the immediate request context to evaluate a tool call's safety.
Now trims to messages from the last user message onward before
applying the FIFO budget cap. Keeps the user's request, the
assistant's response with tool calls, and any recent tool results
while discarding earlier conversation that isn't relevant to the
current intent evaluation.
Claude 4.6 (Opus + Sonnet) unified on 1M token context windows.
Update capabilities table from 200K to 1M for both models. Remove
claude-opus-4 and claude-sonnet-4 entries (end of life). 4.5 models
remain at 200K. Default fallback stays at 200K for unknown models.
* fix: distinguish user cancel from crash in bash tool results
When a user cancels a running bash command, the process is killed
with SIGKILL (exit code -9). Previously this showed as an error,
causing the model to retry. Now checks cancel.is_set() after proc
exit and returns "Cancelled by user." as a non-error result so the
model knows to stop rather than retry.
* fix: use -signal.SIGKILL instead of magic -9
Copilot review: replace hard-coded -9 with -signal.SIGKILL for
clarity. Popen.returncode is negative of signal number when killed.
* feat: add stop_on_error param to bash tool for set -e behavior
New boolean parameter enables 'set -e' in the bash preamble so
multi-step scripts exit on the first command failure instead of
silently continuing. Default false (existing behavior preserved).
pipefail remains always-on.
* fix: strict bool parsing for stop_on_error, treat exit 1 as error with set -e
Copilot review: bool("false") is True — use `is True` for strict
JSON boolean parsing. Also, with stop_on_error enabled, any non-zero
exit code is now treated as an error (set -e means the script halted
on failure), whereas without it exit code 1 remains benign.
* fix: synthesize cancelled tool results instead of stripping turns
When a user cancels during tool execution, the model previously lost
all context about what was attempted (assistant message + tool_calls
stripped entirely). Now synthesizes tool_result messages with
is_error=true and "Cancelled by user." content for any tool_calls
that lack matching results. This keeps the conversation valid for
both providers while preserving the full tool call structure so the
model knows what was tried.
Also applies to KeyboardInterrupt with "Interrupted by user." text.
* fix: persist synthesized cancel results to DB, assert is_error in test
Copilot review: synthesized tool messages were in-memory only,
creating a mismatch with DB that could break rewind/retry. Now
calls save_message() for each synthesized result. Also adds
is_error=True assertion to the cancel test.
* feat: add pagination and longer content to recall tool
- New offset parameter for paginating through recall results
- Content preview increased from 500 to 2000 chars per match with
total length indicator when truncated
- Output passed through _truncate_output for consistency
- OFFSET clause added to SQLite (FTS5 + LIKE) and PostgreSQL
(tsvector + ILIKE) search queries
* fix: defensive int coercion for recall offset/limit
Copilot review: offset/limit could arrive as null, float, or other
non-int types from JSON. Coerce with int() + try/except in prepare,
and int() at the storage layer before binding into SQL OFFSET/LIMIT.
* feat: add diff_file tool for comparing files and content
New read-only tool that shows unified diffs between two files or
between a file and provided content. Useful for verifying edit_file
changes and comparing file versions. Auto-approved (no side effects).
Configurable context lines (default 3). Available to task agents.
* refactor: extract _read_text_lines helper, share across read_file and diff_file
Copilot review: diff_file duplicated file-loading and lacked binary
detection. Extract _read_text_lines() that handles realpath
resolution, null-byte binary detection, and error handling. Used by
both _exec_read_file and _exec_diff for consistent behavior.
* fix: address code review — agent flag, resolved shadowing, read_files
- Add agent: true to diff_file schema so plan agents can use it
- Fix resolved variable shadowing in _exec_read_file (use _ for
unused return from _read_text_lines)
- Register diffed files in _read_files so edit_file read guard
is satisfied after diff_file
- Move difflib import to module level (stdlib, no lazy-load needed)
- Fix description wording ("provided string" not "previous version")
* fix: stream diff with early cutoff, expand paths before header
- Stream difflib output and stop collecting after tool_truncation
chars to avoid large intermediate allocations on big diffs
- Expand paths with expanduser before building the approval header
so display matches actual execution paths
* docs: tool descriptions, bash timeout param, multi-line preview
- task_agent/plan_agent: document the tool subset limitation (no
memory, recall, watch, skill, or further delegation)
- bash: add per-call timeout parameter (1-600s, defaults to 120s),
shown in approval header when specified
- bash: show full command in preview for multi-line scripts so the
approval flow displays the complete command, not just the first line
- bash: document 256KB output cap and stderr prefix in description
* fix: address Copilot review on tool descriptions
- bash: say "truncated" not "256KB" (limit is configurable), document
timeout clamping range (1-600) and global fallback
- bash preview: fix "1 more lines" → "1 more line" singular
- plan_agent: remove bash from listed tools (not in AGENT_TOOLS)
* fix: improve memory save error message, narrow dd command filter
Two minor fixes from harness shakedown:
- memory save: split "both name and content required" into separate
errors for missing name vs empty content
- bash safety: replace blanket "dd if=" block with targeted patterns
for writes to block devices (of=/dev/sd*, /dev/nvme*, /dev/disk/,
etc.) and redirects to the same. Legitimate dd use like generating
test data or benchmarking reads is no longer blocked.
* fix: generalize > /dev/sda redirect pattern to > /dev/sd
Copilot review: only /dev/sda was blocked for redirects while
/dev/sdb, /dev/sdc etc were not. Generalize to match any /dev/sd*
device, consistent with the of= patterns.
* feat: edit_file replace_all, write_file append mode, search match count
Three tool enhancements from harness shakedown feedback:
- edit_file: new replace_all parameter replaces all occurrences of
old_string instead of requiring a unique match. Cannot combine with
near_line or edits array.
- write_file: new mode parameter with "append" option. Appends
content to end of file instead of truncating.
- search: output now includes a summary footer showing total match
count and file count (e.g. "47 matches across 12 files").
* fix: address Copilot review on tool enhancements
- replace_all: skip multi-occurrence rejection in pre-validation so
the feature actually works; show occurrence count in preview
- write_file mode: coerce non-string types safely via str()
- search footer: append before truncation to respect output limits
- edit_file error: mention replace_all as alternative to near_line
read_file silently converted null bytes to spaces, showing corrupted
content with no warning. Now samples the first 8KB for null bytes and
returns a clear error directing the user to bash for binary inspection.
* fix: memory delete searches all scopes when scope not specified
Previously delete defaulted to scope=global, so deleting a
workstream-scoped memory without explicitly passing scope=workstream
silently failed. Now tries narrowest scope first (workstream → user
→ global) and deletes the first match. Explicit scope still honored
when provided.
* fix: reject invalid scope on memory delete instead of silent fallback
Copilot review: invalid scope values were silently treated as
unspecified, which could cause accidental deletion from the wrong
scope. Now returns a clear error listing valid scopes.
* fix: exclude build/vendor/VCS directories from search tool
grep -rn recursed into .git, node_modules, target, __pycache__, etc.
producing hundreds of noise hits from generated content. Add
--exclude-dir flags for common directories that should never appear
in search results.
* fix: glob egg-info pattern and add vendor exclude
Copilot review: .egg-info misses turnstone.egg-info (named dirs),
use *.egg-info glob. Also add vendor to the exclude list.
Agent workflows need git for version control, curl for raw HTTP
requests, jq for JSON processing, and man/info for documentation
lookup. All were missing from the slim base image, leaving the man
tool non-functional and standard dev workflows broken.
* fix: block IPv6 loopback/link-local/private in SSRF filter
check_ssrf used gethostbyname which only resolves IPv4. IPv6 addresses
like ::1, fe80::, fd00:: bypassed the filter entirely. Switch to
getaddrinfo which resolves both address families and check all results.
* fix: handle IPv4-mapped IPv6 and zone IDs in SSRF filter
Copilot review caught two bypasses: ::ffff:127.0.0.1 (IPv4-mapped
IPv6) wasn't normalized before private/loopback checks, and fe80::1%lo0
(zone ID suffix) caused a ValueError that was silently swallowed.
Now normalizes IPv4-mapped addresses and strips zone IDs before parsing.
* fix: resolve symlinks before file I/O to prevent path-based bypass
write_file and edit_file followed symlinks silently — a symlink at
/data/link → /etc/passwd would show the /data path in the approval
header while writing to the real target. Three changes:
- open() calls in _exec_write_file, _exec_edit_file, _exec_read_file
now use the resolved (realpath) path instead of the raw symlink
- Approval headers show both paths when a symlink is detected
(e.g. "⚙ write_file: /data/link → /etc/passwd")
- Judge _get_arg_text includes the resolved path so heuristic rules
like write-system-path fire even through symlinks
* fix: address Copilot review — expanduser in fallback, pre-read, image paths
- edit_file exec fallback: add expanduser before realpath (tilde bypass)
- judge _get_arg_text: compare resolved against abspath(expanduser(path))
so ~/ paths don't false-positive as symlinks
- edit_file pre-read: use resolved path instead of raw symlink path
- _exec_read_image: use resolved path for getsize and binary open
* fix: clear dedup sigs after write tools to avoid false repeat warnings
The read→edit→read workflow triggered "identical repeat" warnings
because the dedup tracker compared (tool_name, args) without
considering intervening state changes. Now clears the signature set
when write_file, edit_file, or bash executes successfully, so
subsequent reads of the same file are not flagged.
* fix: use shared error prefixes for write-success detection in dedup
Copilot review: the error detection for write tools only checked
"Error" prefix, missing "Command timed out", "Blocked:", "Denied",
etc. Now shares the same _error_prefixes tuple used by the repeat
detection below, ensuring consistent classification.
The judge pre-converted tool schemas via convert_tools() before
passing them to create_completion(), which internally calls
convert_tools() again. The second conversion tried to extract
function.name from already-converted Anthropic-format tools,
producing empty tool names that the API rejected with
"tools.0.custom.name: String should have at least 1 character".
Fix: pass raw OpenAI-format schemas directly — create_completion
handles the provider-specific conversion.
* feat: add /retry and /rewind commands for conversation history navigation
Allow users to re-send the last message for a new response (/retry) or
drop the last N turns to restore an earlier conversation state (/rewind N).
Both operations sync in-memory state with the persistent database.
Server path includes conversation.modify permission gate, audit trail
(conversation.rewind / conversation.retry events), and thread-safe retry
dispatch. Migration 029 grants the permission to admin and operator roles.
* feat: add message action controls for retry, edit, and rewind in web UI
Hover toolbar on messages with CSS-only icons matching instrument panel
aesthetic. User messages get edit (pencil) and rewind (chevrons) buttons;
last assistant message gets retry (circular arrow). Edit flow uses
event-driven coordination — rewind completes via SSE history event before
send fires. Includes ARIA labels, keyboard nav, touch device support,
reduced motion, and busy-state gating.
Addresses Copilot review feedback on #219:
1. Anthropic _convert_messages: collect tool_use IDs in order (list
not set), filter empty IDs, defer synthetic results until after
real tool results so _merge_consecutive produces correct ordering.
2. Universal repair in reconstruct_messages: synthesize tool results
for mid-conversation orphaned tool calls on DB load. Benefits all
providers (OpenAI is lenient today but may tighten).
3. Test improvements: assert on is_error flag instead of "cancelled"
substring, verify real-before-synthetic ordering in partial results.
When a cancel interrupts tool execution, the assistant message with
tool_use blocks is saved to DB before tools run, but
GenerationCancelled prevents tool results from being created. The
in-memory rollback removes the orphaned message, but the DB row
persists. On resume, Anthropic rejects the conversation with
"tool_use ids were found without tool_result blocks".
Fix: _convert_messages now peeks ahead after each assistant message
with tool_use blocks. If any tool_use IDs lack matching tool_result
messages, synthetic error results are injected (is_error: true,
"Tool execution was cancelled."). Transparent to all callers,
provider-specific (OpenAI is lenient about this).
5 new tests covering: single orphan, multiple orphans, partial
results, complete results (no synthesis), and trailing orphan.
plan_agent and task_agent fail on Anthropic models with "Streaming is
required for operations that may take longer than 10 minutes" from
the SDK. The non-streaming create_completion path used
client.messages.create() which the SDK rejects for thinking-enabled
models.
Fix: use client.messages.stream() internally and call
get_final_message() to get the same Message object. Transparent to
all callers — fixes sub-agents, title generation, summarization,
web fetch, and judge create_completion calls.
Five improvements from Opus self-evaluation of the turnstone harness:
1. Batch edit_file: edits array parameter for atomic multi-edit in a
single tool call. Overlap detection, reverse-order application,
mutual exclusivity with single-edit params.
2. Sandbox packages: new [sandbox] extras group with sympy, numpy,
scipy, pytest — the sandbox already had graceful ImportError
fallbacks, now the packages are actually installed.
3. Stderr labeling: bash tool output prefixes stderr lines with
[stderr] so the model can distinguish errors from stdout.
4. JSON secret redaction: output guard now detects and redacts secrets
in JSON format ("api_key": "...", "password": "...", etc.) with
18 key patterns and 8-char minimum value length.
5. Model persisted on resume: workstream config now saves model and
model_alias. Resume restores the original model via registry
(same path as /model command), falling back to raw model name
if the alias is no longer available.
24 new tests (23 in test_edit_file.py, 1 in test_sessions.py).
stream_end fires per-segment (between tool calls), not per-turn.
The UI was using stream_end to transition to idle, causing a window
where the Send button appeared but the server worker thread was still
alive. User messages submitted during this window were silently
dropped. No Stop button was visible, so the user had no cancel path.
Root cause: state_change events (idle/thinking/running/error) were
only broadcast to the global SSE stream (console dashboard), never
to the per-workstream SSE that the browser UI listens to.
Fix: (1) server.py: on_state_change now also enqueues to the
per-workstream SSE listeners. (2) app.js: stream_end no longer
calls setBusy(false) — it only finalizes markdown rendering.
New state_change handler manages busy transitions: idle/error
set busy=false, thinking/running set busy=true.
* feat: model detect button, capabilities API, and model dropdowns
Admin Models tab: add Detect button that probes a model endpoint to
verify reachability, list available models, detect context_window, and
identify server type (llama.cpp/vLLM/SGLang/OpenAI/Anthropic). Add
static capability lookup endpoint for auto-filling form fields when
a known model name is entered. Add known-models endpoint for datalist
autocomplete suggestions.
Add "openai-compatible" as a third provider option for local servers,
keeping the OpenAI SDK under the hood but suppressing capability
auto-fill and known-model suggestions.
Replace free-text model input with a select dropdown in both console
and server new-workstream modals, populated from a new lightweight
GET /v1/api/models endpoint.
New endpoints:
- POST /v1/api/admin/model-definitions/detect
- GET /v1/api/admin/model-capabilities
- GET /v1/api/admin/model-capabilities/known
- GET /v1/api/models (both console and server)
* fix: accumulate signature_delta for Anthropic thinking blocks (#214)
The streaming path captured thinking_delta events but not
signature_delta, leaving the signature empty on round-trip and
causing 400 errors on multi-turn conversations with thinking enabled.
* fix: address PR review — empty base_url, capability leak, response schemas
- Don't pass empty base_url to OpenAI client (falls back to SDK default)
- Return None from lookup_model_capabilities for openai-compatible provider
- Only use static capability table for known models in _detect_openai_compat,
avoiding misleading 200k default for unknown local models
- Add AvailableModelInfo + ListAvailableModelsResponse schemas to both
console_spec and server_spec
- Regenerate TypeScript SDK OpenAPI snapshots
* fix: apply same known-model guard to Anthropic context_window detection
Only report context_window from the static capability table when the
Anthropic model is actually known, matching the OpenAI path fix.
* ui: add autocomplete hint to Model ID label in admin modal
* fix: use explicit kwargs for OpenAI() to satisfy strict mypy
The streaming path captured thinking_delta events but not
signature_delta, leaving the signature empty on round-trip and
causing 400 errors on multi-turn conversations with thinking enabled.
When a local model generates tool calls that are silently dropped
(truncation, missing tool-call-parser), there was zero server-side
logging — making it impossible to diagnose from docker compose logs.
- OpenAI provider: log request params and response summary (debug)
- Session: log stream completion, tool call presence (info), and
tool call discard with names when truncated (warning)
CLI unaffected — log level is WARNING there.
Add model_definitions table (migration 028) enabling model management
via the admin console without SSH access or server restarts. Models
defined in the database coexist with config.toml models through a
per-node merge strategy — config.toml overrides DB for the same alias,
DB-only models coexist alongside, no cross-node contamination.
Storage layer:
- model_definitions table with CRUD (SQLite + PostgreSQL)
- MODEL_DEFINITION_MUTABLE allowlist, admin.models permission
ModelRegistry integration:
- load_model_registry() merges DB + config.toml + CLI models
- context_window=0 auto-detects from provider capability table
or inherits CLI-detected value (same as config.toml behavior)
- ModelRegistry.reload() with validation, TOCTOU-safe accessors
- internal_model_reload + internal_model_status server endpoints
Admin API + UI:
- 6 console endpoints (list, create, get, update, delete, reload)
with admin.models permission, audit trail, provider validation
- Models tab in System group with sky blue (--blue) accent color
- Provider badges (openai/anthropic), source badges (config/db)
- Write-only API keys (never readable, "***" sentinel on update)
- Sync-pending indicator, mobile responsive, focus-trapped modal
Also changes is_secret settings from write-blocked (403) to write-only
across all settings, making judge.api_key configurable via admin UI.
* fix: watch dispatch error handler missing stream_end and state cleanup
The watch dispatch run() closure was missing GenerationCancelled
handling, stream_end emission, on_state_change calls, and the
worker_thread identity guard that the send_message path has. This
left the web UI in a stale state when watch-dispatched sends failed.
* fix: address review feedback - on_stream_end, put_nowait, ws._lock, tests
* fix: ruff lint (unused pytest import)
* fix: send_message() use on_stream_end() instead of raw _enqueue
* refactor: add is_error to on_tool_result protocol, remove text heuristics
Add is_error keyword arg to SessionUI.on_tool_result() so tools
report errors structurally. Server and JS client no longer guess
from output text prefixes — each tool sets the flag at the source.
Bash tool: exit code >= 2 is error, exit code 1 is ambiguous (grep
no-match). History reconstruction keeps text heuristic as fallback
for pre-migration data.
Update SDKs (Python + TypeScript), test mocks, docs, and diagrams.
* fix: infinite recursion in _report_tool_result, signal exits, stale docs
* fix: add _tool_error_flags to test_load_skill ChatSession stubs
Add vendor-js workflow that triggers on Renovate PRs touching
pyproject.toml — detects version changes, runs update-vendored-js.sh,
and commits the actual files back to the PR branch. Supports manual
dispatch via pr_number input for one-off runs.
Providers now register the SDK stream handle eagerly (before returning
the iterator) instead of lazily inside a generator body. This closes
the window where cancel() couldn't abort a blocked HTTP read because
the stream handle wasn't populated yet.
- OpenAI: yield from → return (HTTP call + cancel_ref happen eagerly)
- Anthropic: split into eager __enter__ + _iter_with_cleanup generator
with defensive __exit__ on __enter__ failure
- Console SDK: delete_setting() return type fixed from StatusResponse
to DeleteSettingResponse (matching actual endpoint response)
- 2 new threaded force-cancel integration tests verifying orphaned
threads don't mutate messages and new generations succeed
- Updated _CancelRef docstring, test robustness (assert on wait)
The cancel endpoint emitted a 'cancelled' SSE event before the worker
thread terminated. The frontend transitioned to "send" mode prematurely,
so the next send got rejected with "Already processing a request."
Backend:
- Providers expose SDK stream handle via cancel_ref parameter so
cancel() can close the HTTP connection and unblock iteration
- Generation counter prevents orphaned threads from mutating messages
or clearing cancel state after force cancel
- _check_cancelled() added between retry attempts in _try_stream
- Server polls (async, non-blocking) for cancelled worker to exit
- Force cancel (force:true) abandons stuck worker, keeps cancel event
set so subprocesses are killed, guards against spurious SSE events
Frontend:
- 'cancelled' shows "Cancelling..." then escalates to "Force Stop"
after 2s for a harder cancel that abandons the worker immediately
- 10s safety timeout auto-recovers if stream_end never arrives
- busy_error re-enables stop button instead of showing send
- Timeout cleanup in disconnectSSE, stream_end, and force .then()
- Layout shift prevention (min-width, white-space: nowrap)
- aria-label updates for accessibility
Tests:
- 7 new tests: stream close, error suppression, cancel_ref population,
transport error conversion, non-cancel exception propagation, retry
cancellation check
When a model calls the same tool with identical arguments as a previous
call, append a warning to the tool result and inject a metacognitive
nudge. This breaks loops where small local models get stuck repeating
the same action (e.g. running the same bash command 3+ times).
The repeat signature set is cleared after a warning fires, giving the
model a clean slate. Also cleared on conversation compaction.
Ref: #186
* fix: harden tool call handling for local model servers
Local models (Qwen 3.5 9B, etc.) via llama.cpp produce tool calls with
empty IDs, whitespace-padded names, and malformed JSON arguments. These
defensive gaps caused cascading conversation corruption and silent
failures.
- Strip whitespace from tool names in both main and agent paths
- Generate synthetic UUIDs when tool call IDs are empty/null
- Surface malformed tool call errors to the user via on_error
- Give the model actionable hints (expected JSON format, available tools)
so it can self-correct on retry
- Surface metacognition nudge types to UI via on_info
Ref: #186, #117
* refactor: extract _ensure_tool_call_ids helper, include MCP tools in error
Address Copilot review feedback on PR #200:
- Extract duplicated ID fixup into _ensure_tool_call_ids() static method
- Tests now exercise the actual helper instead of reimplementing the logic
- Unknown tool error now includes MCP tool names alongside builtins
Patrick Buckley