12 Commits

Author SHA1 Message Date
Patrick Buckley c006be25de Bump version to 0.3.0 v0.3.0 2026-03-02 22:16:34 -08:00
Patrick Buckley 167d63b385 Add operational features: health degradation, rate limiting, workstre… (#10)
* Add operational features: health degradation, rate limiting, workstream eviction

Backend health monitor with circuit breaker (CLOSED/OPEN/HALF_OPEN) probes
LLM backend periodically; /health returns "degraded" when unreachable.
Token-bucket per-IP rate limiter with 429 + Retry-After responses;
/health and /metrics exempt. Workstream auto-eviction of oldest idle
when at configurable max_workstreams capacity.

New modules: healthcheck.py (BackendHealthMonitor, CircuitState),
ratelimit.py (TokenBucket, RateLimiter). 5 new Prometheus metrics.
Both UIs: health indicator, 429 retry with toast, eviction notifications,
node degradation badges (console), circuit state in dashboard footer.

Config: [health] and [ratelimit] TOML sections, max_workstreams in [server].
Docs: README, architecture, API reference, PlantUML diagrams updated.
616 tests pass (35 new), mypy clean, ruff clean.

* Fix Copilot PR #10 review: version import, capacity check order, validations, docs

- Use turnstone.__version__ instead of hard-coded "0.2.1" in /health and
  /metrics endpoints
- Move capacity check/eviction before session creation in
  WorkstreamManager.create() to avoid wasted work when at capacity
- Validate rate > 0 and burst >= 1 in RateLimiter when enabled
- Validate max_workstreams >= 1 in WorkstreamManager.__init__
- Parse do_POST path with urlparse for consistent rate limit exemptions
  and metrics labeling
- Fix should_allow_request docstring: HALF_OPEN allows requests through
  (not just one probe)
- Fix /health docstring: degraded when circuit is not CLOSED (includes
  HALF_OPEN)
- Add class="health-ok" to health indicator HTML to prevent visible
  empty pill before first poll
- Update PlantUML: remove stale MAX_WORKSTREAMS constant, fix
  RateLimiter.check and TokenBucket signatures; regenerate PNG
2026-03-02 22:14:03 -08:00
Patrick Buckley 2c48f694db Add multi-model support with ModelRegistry, fallback routing, and per… (#9)
* Add multi-model support with ModelRegistry, fallback routing, and per-workstream selection

Introduces a ModelRegistry that holds named model configurations loaded from
[models.*] sections in config.toml. Each workstream can select its model at
creation time or switch mid-session via /model <alias>. When the primary model
is unreachable, a configurable fallback chain tries alternative models. Sub-agents
(plan/task) can optionally use a cheaper model via the agent_model setting.

Core changes:
- New turnstone/core/model_registry.py: ModelConfig (frozen, api_key redacted from
  repr), ModelRegistry (thread-safe lazy client creation, resolve, fallback chain),
  load_model_registry() with backwards-compatible config loading
- session.py: registry/model_alias params, /model show+switch command, fallback in
  _create_stream_with_retry (extracted _try_stream), agent model override in _run_agent
- workstream.py: factory signature accepts optional model_alias, create() gains model param
- cli.py + server.py: build registry, updated session factories, banner, shutdown
- protocol.py: model field on CreateWorkstreamMessage
- bridge.py: pass model through workstream creation chain

Frontend:
- MODEL column added to dashboard tables in both server and console UIs
- Responsive: hidden alongside NODE at narrow viewports
- ARIA labels include model info, title attributes for truncated text
- SSE connected event includes model_alias

Documentation:
- README: architecture tree, Multi-Model Support section, config keys
- docs/architecture.md: module map, Multi-Model Registry subsection
- docs/api-reference.md: model field in workstream creation, model_alias in SSE
- PlantUML diagrams 02 + 03 updated with ModelRegistry

Tests: 43 new tests (576 total), mypy clean, ruff clean.

* Fix Copilot PR #9 review: model_alias property, preserve manual tool_truncation

- Expose model_alias as a public @property on ChatSession instead of
  accessing the private _model_alias from server.py and tests
- Track _manual_tool_truncation flag so /model switch only recomputes
  tool_truncation when it was auto-derived, preserving --tool-truncation
  overrides
- Update PlantUML diagram to reflect the public property
2026-03-02 20:58:35 -08:00
Patrick Buckley 5118808f24 Add MCP client support for external tool servers (#8)
* Add MCP client support for external tool servers

MCPClientManager connects to stdio and HTTP MCP servers via a background
asyncio event loop, discovers tools at startup, and converts schemas to
OpenAI function-calling format with mcp__{server}__{tool} prefixing.

- New turnstone/core/mcp_client.py: async-sync bridge, config loader
  (TOML [mcp.servers.*] + standard mcpServers JSON), tool discovery
- session.py: mcp_client param, self._tools/_task_tools/_agent_tools,
  _prepare_mcp_tool/_exec_mcp_tool, /mcp introspection command
- tools.py: merge_mcp_tools() helper
- cli.py + server.py: --mcp-config arg, client lifecycle, banner info
- pyproject.toml: mcp>=1.6 required dependency, mypy override
- 30 new tests (config, schema conversion, session integration, errors)
- Docs: README MCP section, tools.md MCP reference, architecture.md
  MCP subsection, 3 updated PlantUML diagrams + PNGs

* Fix Copilot PR #8 review: hermetic MCP config tests, approval docs wording

- Patch load_config in test_json_file_not_found and test_invalid_json so
  a developer's local config.toml doesn't leak into test results
- Clarify MCP approval docs: tools require approval by default, but
  --skip-permissions and UI auto-approve override this
2026-03-02 19:54:23 -08:00
Patrick Buckley b6c1c3a676 Add console-to-server deep linking via ?ws_id= query parameter (#7)
* Add console-to-server deep linking via ?ws_id= query parameter

Server UI parses ?ws_id= on load (both direct init and post-login) and
auto-selects the matching workstream instead of showing the dashboard.
URL is cleaned from the address bar via history.replaceState after
navigation. Defers initial SSE connection to avoid redundant connect
when deep-linking switches tabs immediately.

Console workstream rows are now clickable — opens the node's server UI
in a new tab with ?ws_id= targeting that workstream. Uses URL constructor
for safe URL building. External-link indicator (↗) appears on hover.
Rows without server_url have role/tabindex removed to avoid broken
affordance. currentServerUrl reset on showOverview() to prevent stale
fallback across views.

Collector injects server_url into workstream dicts in both the poll path
and ws_created event path so deep links work immediately.

* Fix Copilot PR #7 review: deep-link duplicate history entry, server_url test coverage

- Suppress history.pushState in switchTab() during deep-link navigation by
  setting _historyNavigation=true around both call sites (post-login and
  direct init). Fixes Back button appearing to do nothing on first press.
- Add server_url assertions to poll and ws_created collector tests to
  prevent regressions of deep-link functionality.
2026-03-02 19:46:35 -08:00
Patrick Buckley 7fbcb70ec1 Add call_id routing for streaming tool output during parallel execution (#6)
* Add call_id routing for streaming tool output during parallel execution

Thread call_id through tool_info, approve_request, and tool_result SSE
events so the browser can route streaming output chunks and final results
to the correct tool div when multiple bash tools run in parallel.

Server: include call_id in serialized approval items and tool_result events.
Protocol: add call_id to on_tool_result signature (session, cli, eval, server)
         and ToolResultEvent dataclass; pass through MQ bridge.
Client: set data-call-id on tool divs, match by call_id in appendToolOutputChunk
        and appendToolOutput with func_name fallback; extract makeCollapsible
        helper; use CSS.escape for querySelector safety; fix replayHistory
        \\n typo and missing keyboard accessibility on collapsed output.
Bridge: fix pre-existing bug using "name" instead of "func_name" for
        auto-approval matching; include call_id in _build_history for replay.
Also adds on_tool_result calls to write_file and edit_file exec methods.

* Update docs/tools.md
2026-03-02 18:49:24 -08:00
Patrick Buckley 7ebc04d1cc Fix Copilot PR #3 review: timeout race, ANSI stripping, streaming perf (#5)
- Fix timeout race in _exec_bash: check proc.poll() before setting
  timed_out flag; fall back to proc.kill() if process group kill fails
- Remove stderr_thread.join(timeout=5) — after proc.wait() the pipe
  is closed so join completes promptly without risk of truncation
- Fix CSI final byte range in stripAnsi: [A-Za-z] → [@-~] to handle
  sequences like \x1b[1~ (Home key) that end in non-letter bytes
- Use appendChild(createTextNode()) instead of textContent += for O(1)
  chunk appending in streaming output (avoids O(n²) on large output)
- Use var(--accent-dim) in stream-pulse keyframe instead of hardcoded
  rgba() so pulse color adapts to light/dark theme
- Record tool_output_chunk calls in RecordingUI for test assertions
2026-03-02 18:17:05 -08:00
Patrick Buckley 5c452a1239 Harden session persistence: config storage, interrupted repair, prune… (#4)
* Harden session persistence: config storage, interrupted repair, prune tests

Session config persistence:
- Add session_config table to SQLite schema for persisting LLM-affecting
  parameters (temperature, reasoning_effort, max_tokens, instructions,
  creative_mode) across resume
- Add save_session_config() and load_session_config() to memory.py
- ChatSession._save_config() called on init and when /instructions,
  /effort, /creative slash commands change config
- resume_session() restores persisted config and rebuilds system messages

Interrupted session repair:
- load_session_messages() now strips trailing incomplete tool call turns
  where tool_calls exist but fewer tool results than expected (session
  was interrupted mid-execution via Ctrl+C or crash)

Cleanup:
- delete_session() now also removes session_config rows
- 14 new tests: interrupted repair (4), config persistence (5),
  prune_sessions (5)

Docs & diagrams:
- Document config persistence and interrupted repair in architecture.md
- Add _save_config() to ChatSession in 03-core-engine-classes.puml

* Fix Copilot PR #4 review: prune config cleanup, /new config persist, resume instructions

- prune_sessions() now deletes session_config rows for orphaned/stale sessions
- /new command calls _save_config() so config persists immediately
- resume_session() uses key presence check for instructions, fixing cross-session leak
- Add test_prune_removes_session_config covering both orphan and stale paths
2026-03-02 18:14:33 -08:00
Patrick Buckley 14a9ff9513 Stream bash tool output incrementally via SSE
Replace subprocess.run() with Popen for bash tool execution, streaming
stdout line-by-line through a new on_tool_output_chunk callback. Web UI
renders chunks incrementally with a pulsing amber border indicator.

Core:
- Add on_tool_output_chunk(call_id, chunk) to SessionUI protocol
- Rewrite _exec_bash() with Popen, process-group kill via
  start_new_session + os.killpg, background stderr drain thread,
  threading.Event-based timeout detection
- Guard UI callback with contextlib.suppress so errors don't
  interrupt output collection

Server/CLI/eval:
- Add tool_output_chunk SSE event type in WebUI
- No-op implementations in TerminalUI, BackgroundTerminalUI, SilentUI

MQ:
- Add ToolOutputChunkEvent to mq/protocol.py and _OUTBOUND_REGISTRY
- Handle tool_output_chunk in bridge._handle_ws_event

Web UI:
- Add appendToolOutputChunk() with call_id-keyed DOM elements,
  inner auto-scroll, ARIA attributes, and empty chunk guards
- Fix appendToolOutput() streaming cleanup using adjacency matching
- Make collapsed output keyboard-accessible (tabindex, role, keydown)
- Improve stripAnsi() to handle CSI, OSC, and two-byte escapes;
  use it consistently in replayHistory, addInfoMessage, addErrorMessage
- Add .tool-output-stream CSS with soft pulse animation, mobile
  max-height cap, and consolidated prefers-reduced-motion support

Docs & diagrams:
- Document tool_output_chunk SSE event in api-reference.md
- Update SessionUI protocol (14 methods) in architecture.md
- Update Phase 3 execution flow in tools.md
- Add on_tool_output_chunk to 03-core-engine-classes.puml
- Update 04-conversation-turn.puml, 05-tool-pipeline.puml
- Add ToolOutputChunkEvent to 06-mq-protocol.puml
- Add to event list in 07-message-routing.puml
- Regenerate all 5 affected PNG diagrams
2026-03-02 17:52:09 -08:00
Patrick Buckley c39affbf6b Add test coverage reporting and pre-commit hooks (#2)
* Add test coverage reporting and pre-commit hooks

- Add pytest-cov to test dependencies, configure coverage in pyproject.toml
  (branch coverage, static asset omission, standard exclusion patterns)
- CI test job now runs with --cov and uploads coverage XML as artifact
- Add .pre-commit-config.yaml with ruff (check + format) and mypy hooks

Baseline coverage: 41% (482 tests, Python 3.13)

* Fix Copilot review: add redis dep for pre-commit mypy, explicit --cov target

- Add redis>=7.2 to mypy pre-commit hook additional_dependencies so
  mypy can resolve redis imports in isolated pre-commit environments
- Use --cov=turnstone instead of bare --cov to explicitly scope coverage
2026-03-02 17:28:50 -08:00
Patrick Buckley 9be155b97a Quality overhaul: code tooling, CI/CD, architecture diagrams, UI rede… (#1)
* Quality overhaul: code tooling, CI/CD, architecture diagrams, UI redesign, and legacy cleanup

- Add ruff (lint+format) and mypy (strict) with zero errors across 37 source files
- Add GitHub Actions CI (lint, typecheck, test matrix 3.11/3.12/3.13) and PyPI publish workflow
- Create 12 PlantUML architecture diagrams with PNG renders covering all subsystems
- Refresh README and docs with badges, diagram links, and current descriptions
- Refactor test_server_live.py with mock streaming helpers for deterministic CI testing
- Update dependencies to current versions (openai>=2.24, httpx>=0.28, redis>=7.2)

Console dashboard:
- Move state indicators from top cards to fixed bottom status bar with cluster metrics
- Replace flat 50-node list with hostname-prefix grouped nodes (expand/collapse, up to 1000)
- Apply "Instrument Panel" visual redesign: IBM Plex Mono + Outfit fonts, warm amber accent,
  LED glow state indicators, deep charcoal surfaces, WCAG AA contrast compliance
- Add render cache, stale indicator, active filter highlight, loading states

Server web UI:
- Apply matching Instrument Panel aesthetic for visual consistency with console
- Fix branding (pcode → turnstone), extract inline styles to CSS classes
- Rename pcode localStorage keys and history state to turnstone

Legacy cleanup:
- Remove persona-model-specific --persona flag and /persona slash command
- Remove model_identity from chat_template_kwargs (vLLM-specific mechanism)
- Refactor plan agent to use standard developer message instead of model_identity
- Remove dead code (unused date/has_tools variables, noqa suppressions)

* Fix CI typecheck: add mypy overrides for optional sympy/numpy imports

The math sandbox optionally imports sympy and numpy at runtime (try/except
ImportError). In CI these packages are not installed, so mypy raises
import-not-found rather than import-untyped. Add mypy overrides to
ignore missing imports for these optional dependencies.

* Fix Copilot review findings: ARIA role, status bar cache, and pulse opacity

- Change #node-table from role="tree" to role="list" and group elements
  from role="treeitem" to role="listitem" (proper ARIA semantics)
- Include currentView and currentFilter.state in renderStatusBar cache key
  so active pill highlight updates when switching views
- Align pulse animation to 0.35 opacity (already applied in CSS)
2026-03-02 16:55:12 -08:00
Patrick Buckley 0d6252dd7d Initial commit — turnstone multi-node AI orchestration platform. 2026-03-02 00:33:37 -08:00