Commit Graph

6 Commits

Author SHA1 Message Date
Patrick Buckley 766223e774 feat(judge): parallelize batch evaluations (#991) 2026-08-08 23:56:15 -07:00
Patrick Buckley 7a06f5e8bc refactor(session): make ModelLane the provider boundary (#979) (#989)
* refactor(session): make ModelLane the provider boundary (#979)

## Summary

This closes the model-lane ownership gap left by #832: `ChatSession` no longer stores raw provider/client handles. `ResolvedModelBinding` now carries the provider, client, model, capabilities, registry generation, and backend-auth configuration as one coherent snapshot.

- Atomically rebind existing sessions after model-registry changes while pinning each in-flight send, fallback, judge, output guard, task agent, title, compaction, perception, and voice operation to its initiating principal and binding.
- Fence UI publication, canonical trajectory folds, durable writes, streams, retries, child scopes, and judge work by generation. Stop can hand off to a successor without accepting late state; cancelled tools retain typed effect receipts, and concurrent approval batches resolve by exact cycle or call.
- Make create, fork, open, close, and delete race-safe with hidden `creating` reservations, incarnation-aware state tails, and an ACL-rechecked transaction that clones checkpoint-bounded history, configuration, project/persona state, and attachment references.
- Extend REST/OpenAPI and Python/TypeScript SDK contracts for create/fork inputs, routed-create metadata, live-workstream probes, targeted approvals, and structured cancellation results.
- Update architecture, storage, authentication, judge, channel, console, API, and SDK documentation, including regenerated architecture diagrams and OpenAPI artifacts.

## Validation

- SQLite suite: 11,188 passed, 9 skipped, 10 deselected
- PostgreSQL suite: 11,195 passed, 2 skipped, 10 deselected
- Live backend: 3 passed
- SSE recovery: 6 passed; browser recovery harness passed all scenarios
- Ruff: clean; 595 files correctly formatted
- mypy: 243 source files clean
- TypeScript: typecheck/build and 35 tests passed
- OpenAPI artifacts fresh; all 14 changed diagrams reproduce byte-for-byte
- `git diff --check` and Git LFS integrity clean

Closes #979.

* fix(deps): update nanoid for GHSA-2v37-7h3g-55p8

Refresh the transitive lock entry admitted by PostCSS so the TypeScript security gate no longer resolves the vulnerable custom-generator implementation.

Validation:
- npm ci
- npm audit --audit-level=moderate: 0 vulnerabilities
- TypeScript typecheck and build
- TypeScript tests: 35 passed

* fix(test): assert canonical model registry URLs

Replace prefix checks with exact canonical base URL assertions so the tests do not model incomplete URL validation.

Validation: tests/test_model_registry.py (185 passed); Ruff check/format; mypy.
2026-08-08 16:13:35 -07:00
Patrick Buckley e71ea38953 feat: output guard data pipeline — persist assessments, SSE events, a… (#110)
* feat: output guard data pipeline — persist assessments, SSE events, admin UI

Complete the output guard pipeline: persist assessments for v2 calibration,
surface warnings in every UI layer, and add scan badges to admin skills tab.

Storage: migration 022 adds output_assessments table (flags, risk_level,
annotations, output_length, redacted — raw output never stored) and
scan_version column on prompt_templates. Three new protocol methods with
SQLite + PostgreSQL implementations.

Server/CLI: on_output_warning now persists assessments fire-and-forget.
CLI shows flags, annotations, and redaction notice. Session emits on_info
warning when high/critical scan_status skill is loaded.

MQ: OutputWarningEvent dataclass + bridge SSE forwarding.

Web UI: output_warning SSE handler with inline warning rendering
(role="alert" for accessibility), semantic risk colors.

Console admin: scan badges on skills list (dedicated scope-scan-* CSS with
green/yellow/red risk vocabulary), scan report breakdown in edit modal with
4-axis scores, POST /admin/skills/{id}/rescan endpoint, GET
/admin/output-assessments endpoint with date-filtered pagination.

Security fixes: ReDoS in connection string regex ([^@\s]+ → [^:@\s]+),
negative limit bypass in all admin endpoints (max(1, ...)), to_dict()
excludes sanitized output by default.

False-positive fixes: credentials pattern anchored to path context,
env secret key check restricted to key portion only.

* fix: address PR #110 review — list redaction, test fixture, OpenAPI snapshot

Per-part redaction: evaluate each text part independently in structured
output instead of joining all parts and replacing only the first one.

Fix test annotations default from "{}" to "[]" matching schema.

Regenerate TypeScript OpenAPI snapshots for new admin endpoints.
2026-03-16 17:39:24 -07:00
Patrick Buckley 5378b33641 feat: output guard — evaluate tool results before they enter context (#109)
* feat: output guard — evaluate tool results before they enter context

Add turnstone/core/output_guard.py — a time-budgeted heuristic that
evaluates tool execution results after execution but before they
enter the conversation context window.

Priority-ordered detection (5s budget, highest priority first):
1. Prompt injection: override phrases, role injection, instruction
   override markers, meta-injection patterns
2. Credential leakage: API keys (OpenAI/GitHub/AWS/Google), PEM
   private key blocks, connection strings, .env secret format
3. Encoded payloads: script data URIs, hex shellcode sequences
4. Adversarial URLs: cloud metadata endpoints, credential query params
5. System info disclosure: private IPs, sensitive file paths

Annotates and optionally redacts (credentials → [REDACTED:<type>]).
Does NOT gate — surfaces warnings via on_output_warning callback.

Integration:
- Wired into session.py tool result loop via _evaluate_output()
- JudgeConfig gains output_guard + redact_secrets fields (both default true)
- SessionUI protocol gains on_output_warning callback
- 25 compiled regex patterns, pure function, no I/O

29 tests covering all detection categories, benign output false
positive checks, credential redaction, and time budget behavior.

* fix: address PR #109 review — protocol, config, and guard fixes

Copilot review feedback:
- Replace _CLEAN singleton with _clean() factory to prevent mutable
  shared state (OutputAssessment has list fields)
- Remove redundant second _CREDENTIAL_PATTERNS loop in _check_credentials
- Evaluate text parts of list outputs (images) not just string outputs
- Wire output_guard + redact_secrets through ConfigStore settings
  registry and _build_judge_config() so operators can configure via
  admin Settings tab
- Remove --no-output-guard CLI flag claim from docs (use Settings tab)

Typecheck fix:
- Add on_output_warning to all SessionUI implementations: NullUI
  (eval, 5 test files), WebUI (server — emits SSE event), TerminalUI
  (CLI — ANSI colored warning), RecordingUI, FakeUI
2026-03-16 16:22:10 -07:00
Patrick Buckley f05e6bddad feat(judge): enrich heuristic rules from 23 to 36 (#107)
* feat(judge): enrich heuristic rules from 23 to 36

Add 13 new pattern-based rules to the intent validation heuristic,
calibrated from analysis of 25K public agent skill security audits
across three independent auditors.

New critical: download-then-execute chains.
New high: browser+data export, transitive installs from untrusted
sources, control plane mutations (crontab, systemctl).
New medium: content ingestion pipelines (curl|python3), interpreter
execution (python3 script.py), cloud CLI mutations (az/gcloud/aws/
kubectl/terraform create/delete/destroy).
New low: tool_search, read_resource, web_search.

Fixes: crontab -l no longer false-positives, systemctl stop/disable
now flagged, az/gcloud subcommand patterns work correctly.

* fix(judge): address PR #107 review feedback

- content-ingestion: narrow second pattern to specific interpreters/
  processors (python3, node, ruby, perl, php, jq) instead of any word.
  Prevents false positives on read-only downstream (wget -O - | head).
- cloud-infra-mutation: split kubectl into its own pattern with specific
  verbs (apply, create, delete, scale, rollout, drain, cordon) to avoid
  false positive on resource types (kubectl get deploy).
- cloud-infra-mutation: split terraform/pulumi to specific verbs only
  (apply, destroy, import) — terraform plan no longer matches.
- control-plane-mutation: exclude -h and -V flags from crontab pattern
  alongside existing -l exclusion.
- Add 35 heuristic rule tests covering all 13 new rules with positive
  matches and negative (false-positive prevention) cases.
2026-03-16 15:09:36 -07:00
Patrick Buckley 09ea3d164d feat: intent validation v1 — advisory LLM judge for tool approvals (#50) (#50)
* feat: intent validation v1 — advisory LLM judge for tool approvals (#50)

Two-tier evaluation pipeline for non-auto-approved tool calls:
- Heuristic tier (instant): 23 pattern-based rules across 4 severity
  levels (critical/high/medium/low) with first-match-wins priority
- LLM judge tier (async): multi-turn evaluation with read_file/
  list_directory tool access, security-hardened path blocking, forcing
  message on final turn, four-stage JSON parsing with retry nudge

Progressive UI: heuristic verdict badge + judge spinner, LLM verdict
upgrade via intent_verdict SSE event, glow on action buttons. Verdict
persisted to intent_verdicts table for audit. Prometheus metrics for
verdict counts and LLM latency. Enabled by default (--no-judge to opt
out). 132 new tests (1938 total).

Integration: session, server/WebUI, CLI, MQ bridge, console admin API,
Discord channel adapter. Config via [judge] in config.toml or CLI flags.

* fix: address PR #50 Copilot review feedback

- Fix double JSON encoding of func_args in both heuristic and LLM
  verdict persistence paths — use pre-serialized string from verdict
- Fix confidence 0.0 treated as falsy in channel verdict formatter
- Fix timestamp format inconsistency in storage backends (isoformat
  vs strftime) — now uses strftime consistently
- Add on_intent_verdict to eval.py NullUI (mypy fix)
- Fix late verdict after approval resolved — store last decision and
  apply immediately to late-arriving verdicts
- Add permission rollback to migration 012 downgrade
- Update docs to reflect judge enabled by default
- Document confidence_threshold as reserved for v2

* fix: judge per-call timeout and credential recon heuristic

- Wrap create_completion() in ThreadPoolExecutor with per-call timeout
  to prevent indefinite hangs on slow local models. On timeout, replace
  the executor so subsequent batch items don't queue behind lingering
  API calls
- Add IntentJudge.shutdown() and wire into session.close() for cleanup
- Add credential-recon heuristic rule: /etc/passwd, /etc/shadow,
  /etc/master.passwd access flagged as HIGH/review (reconnaissance
  pattern even though the command itself is read-only)
- 3 new tests for credential file access patterns

* fix: denied/blocked tool calls show correct badge on resume

- _build_history() detects denied results ("Denied by user") and
  blocked results ("Blocked") and propagates denied flag to parent
  assistant entry for frontend consumption
- Frontend history replay uses denied flag for badge-denied class
  instead of hardcoding badge-approved for all historical tool calls
- Denial feedback always prefixed with "Denied by user:" so content
  detection works with custom user feedback
- Denied tools visually muted (opacity 0.55, muted tool name)
- role="status" on all approval badge elements (accessibility)
- Broadened "Blocked" prefix match (catches "Blocked by tool policy")
2026-03-13 04:12:46 -07:00