mirror of
https://github.com/turnstonelabs/turnstone.git
synced 2026-08-12 23:12:23 -06:00
feat: output guard data pipeline — persist assessments, SSE events, a… (#110)
* feat: output guard data pipeline — persist assessments, SSE events, admin UI
Complete the output guard pipeline: persist assessments for v2 calibration,
surface warnings in every UI layer, and add scan badges to admin skills tab.
Storage: migration 022 adds output_assessments table (flags, risk_level,
annotations, output_length, redacted — raw output never stored) and
scan_version column on prompt_templates. Three new protocol methods with
SQLite + PostgreSQL implementations.
Server/CLI: on_output_warning now persists assessments fire-and-forget.
CLI shows flags, annotations, and redaction notice. Session emits on_info
warning when high/critical scan_status skill is loaded.
MQ: OutputWarningEvent dataclass + bridge SSE forwarding.
Web UI: output_warning SSE handler with inline warning rendering
(role="alert" for accessibility), semantic risk colors.
Console admin: scan badges on skills list (dedicated scope-scan-* CSS with
green/yellow/red risk vocabulary), scan report breakdown in edit modal with
4-axis scores, POST /admin/skills/{id}/rescan endpoint, GET
/admin/output-assessments endpoint with date-filtered pagination.
Security fixes: ReDoS in connection string regex ([^@\s]+ → [^:@\s]+),
negative limit bypass in all admin endpoints (max(1, ...)), to_dict()
excludes sanitized output by default.
False-positive fixes: credentials pattern anchored to path context,
env secret key check restricted to key portion only.
* fix: address PR #110 review — list redaction, test fixture, OpenAPI snapshot
Per-part redaction: evaluate each text part independently in structured
output instead of joining all parts and replacing only the first one.
Fix test annotations default from "{}" to "[]" matching schema.
Regenerate TypeScript OpenAPI snapshots for new admin endpoints.
This commit is contained in:
@@ -1285,6 +1285,71 @@ is on the **console** server and requires the `admin.judge` permission.
|
||||
|
||||
---
|
||||
|
||||
### `GET /v1/api/admin/output-assessments` (Console)
|
||||
|
||||
List output guard assessments from the `output_assessments` table. This endpoint
|
||||
is on the **console** server and requires the `admin.judge` permission.
|
||||
|
||||
**Query parameters:**
|
||||
|
||||
| Parameter | Type | Required | Description |
|
||||
|--------------|--------|----------|----------------------------------------------------|
|
||||
| `ws_id` | string | no | Filter by workstream ID |
|
||||
| `risk_level` | string | no | Filter by risk level (`low`/`medium`/`high`) |
|
||||
| `since` | string | no | ISO timestamp lower bound |
|
||||
| `until` | string | no | ISO timestamp upper bound |
|
||||
| `limit` | int | no | Max results (default 100, max 500) |
|
||||
| `offset` | int | no | Pagination offset (default 0) |
|
||||
|
||||
**Response:**
|
||||
|
||||
```json
|
||||
{
|
||||
"assessments": [
|
||||
{
|
||||
"assessment_id": "a1b2c3d4e5f6",
|
||||
"ws_id": "ws-1",
|
||||
"call_id": "call_abc123",
|
||||
"func_name": "bash",
|
||||
"flags": "[\"credential_leak\"]",
|
||||
"risk_level": "high",
|
||||
"annotations": "[\"API key detected (sk-proj-...)\"]",
|
||||
"output_length": 1024,
|
||||
"redacted": 1,
|
||||
"created": "2026-03-16T10:00:00"
|
||||
}
|
||||
],
|
||||
"total": 7
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### `POST /v1/api/admin/skills/{skill_id}/rescan` (Console)
|
||||
|
||||
Re-scan a skill's content for security signals using the current scanner
|
||||
version. Requires the `admin.skills` permission.
|
||||
|
||||
**Path parameters:**
|
||||
|
||||
| Parameter | Type | Description |
|
||||
|------------|--------|-------------|
|
||||
| `skill_id` | string | Skill (prompt template) ID |
|
||||
|
||||
**Response:**
|
||||
|
||||
```json
|
||||
{
|
||||
"scan_status": "medium",
|
||||
"scan_report": "{\"composite\": 1.75, \"details\": {...}}",
|
||||
"scan_version": "1"
|
||||
}
|
||||
```
|
||||
|
||||
**Error:** `404` if skill not found.
|
||||
|
||||
---
|
||||
|
||||
### `GET /v1/api/admin/settings` (Console)
|
||||
|
||||
List all settings with their effective values, defaults, and metadata. Requires
|
||||
|
||||
Reference in New Issue
Block a user