2 Commits

Author	SHA1	Message	Date
Patrick Buckley	09ea3d164d	feat: intent validation v1 — advisory LLM judge for tool approvals (#50) (#50) ... * feat: intent validation v1 — advisory LLM judge for tool approvals (#50) Two-tier evaluation pipeline for non-auto-approved tool calls: - Heuristic tier (instant): 23 pattern-based rules across 4 severity levels (critical/high/medium/low) with first-match-wins priority - LLM judge tier (async): multi-turn evaluation with read_file/ list_directory tool access, security-hardened path blocking, forcing message on final turn, four-stage JSON parsing with retry nudge Progressive UI: heuristic verdict badge + judge spinner, LLM verdict upgrade via intent_verdict SSE event, glow on action buttons. Verdict persisted to intent_verdicts table for audit. Prometheus metrics for verdict counts and LLM latency. Enabled by default (--no-judge to opt out). 132 new tests (1938 total). Integration: session, server/WebUI, CLI, MQ bridge, console admin API, Discord channel adapter. Config via [judge] in config.toml or CLI flags. * fix: address PR #50 Copilot review feedback - Fix double JSON encoding of func_args in both heuristic and LLM verdict persistence paths — use pre-serialized string from verdict - Fix confidence 0.0 treated as falsy in channel verdict formatter - Fix timestamp format inconsistency in storage backends (isoformat vs strftime) — now uses strftime consistently - Add on_intent_verdict to eval.py NullUI (mypy fix) - Fix late verdict after approval resolved — store last decision and apply immediately to late-arriving verdicts - Add permission rollback to migration 012 downgrade - Update docs to reflect judge enabled by default - Document confidence_threshold as reserved for v2 * fix: judge per-call timeout and credential recon heuristic - Wrap create_completion() in ThreadPoolExecutor with per-call timeout to prevent indefinite hangs on slow local models. On timeout, replace the executor so subsequent batch items don't queue behind lingering API calls - Add IntentJudge.shutdown() and wire into session.close() for cleanup - Add credential-recon heuristic rule: /etc/passwd, /etc/shadow, /etc/master.passwd access flagged as HIGH/review (reconnaissance pattern even though the command itself is read-only) - 3 new tests for credential file access patterns * fix: denied/blocked tool calls show correct badge on resume - _build_history() detects denied results ("Denied by user") and blocked results ("Blocked") and propagates denied flag to parent assistant entry for frontend consumption - Frontend history replay uses denied flag for badge-denied class instead of hardcoding badge-approved for all historical tool calls - Denial feedback always prefixed with "Denied by user:" so content detection works with custom user feedback - Denied tools visually muted (opacity 0.55, muted tool name) - role="status" on all approval badge elements (accessibility) - Broadened "Blocked" prefix match (catches "Blocked by tool policy")	2026-03-13 04:12:46 -07:00
Patrick Buckley	a6e929b0a0	Add channel integrations with Discord adapter and atomic session resu… (#24) ... * Add channel integrations with Discord adapter and atomic session resume (#24) Bidirectional channel adapter framework connecting external messaging platforms to turnstone workstreams via Redis MQ. Discord ships as the first adapter; the protocol supports future Slack/Teams integrations. Channel framework: - ChannelAdapter protocol and ChannelRouter for channel↔workstream mapping - AsyncRedisBroker with single dispatch loop and per-channel ordered workers - channel_routes table (migration 003) for persistent route storage - 9 new StorageBackend methods (4 channel_user + 5 channel_route CRUD) - Unified turnstone-channel gateway entry point, loads adapters by config - Message chunking, approval formatting, plan review formatting Discord adapter: - discord.py v2.4+ bot with thread-per-@mention model - Slash commands: /link (modal), /unlink, /ask, /status, /close - Persistent button views for tool approval and plan review - Streaming responses via edit-in-place (1.5s interval) - Stale route detection and atomic session resume via resume_session field - SessionResumedEvent confirmation back to channel - Auto-approve support (blanket + per-tool list) Atomic session resume: - resume_session field on CreateWorkstreamMessage for single-request resume - Server resumes session during POST /v1/api/workstreams/new atomically - Bridge emits SessionResumedEvent to per-workstream channel - WorkstreamCreatedEvent extended with resumed/session_id/message_count - Server UI dashboardResumeSession simplified to single request - Pruned sessions fall back gracefully to fresh start Service auth: - Bridge and console auto-mint service JWTs from TURNSTONE_JWT_SECRET - Bridge: approve scope (1 week). Console collector: read. Proxy: write. Console admin: - Channels tab with per-user view, force-link modal, unlink - 3 admin API endpoints for channel user management - Styled confirm modals replacing browser confirm() dialogs Bug fixes: - AsyncRedisBroker: replaced per-channel listener tasks with single dispatch loop + per-channel queue workers (fixes message stealing race) - Bridge: approval/plan review dedup guard prevents SSE reconnect duplicates - Bridge: _active_sends tracked for initial messages (fixes missing TurnCompleteEvent and unfinalized streaming messages) - Bridge: HTTP calls moved outside lock scope in approval handlers - Bridge: _handle_send cleans up _active_sends on HTTP/server errors - Formatter: reads server SSE format (func_name/preview) with fallback Docs, SDK, tests: - docs/channels.md setup guide, architecture diagram 16 - Updated api-reference.md, architecture.md, console.md, docker.md - Python SDK: resume_session param on create_workstream (async + sync) - TypeScript SDK: updated CreateWorkstreamRequest/Response interfaces - OpenAPI schema: resume_session request, resumed/message_count response - 91 new tests (19 storage, 15 broker, 22 protocol, 6 routing, 18 discord, 12 resume flow) — 1120 total passing * Fix CI lint/typecheck failures and address Copilot review feedback (#24) Lint: fix import ordering, remove unused imports, use contextlib.suppress. Mypy: explicit postgresql dialect import, add discord module overrides for optional-dependency CI environments. Copilot: fix double-escaping in admin confirm modals, return resolved session_id from server resume response, fix channel_routes diagram schema, use atomic setdefault for routing locks, add post-insert race guard in admin channel create, support SSE format in auto-approve check, update identity linking note in architecture diagram. * Fix remaining mypy call-arg errors for discord.py optional dependency Add type: ignore[call-arg] on Modal(title=) and Cog(name=) class definitions that fail when discord.py is not installed in CI.	2026-03-04 13:02:58 -08:00