mirror of
https://github.com/turnstonelabs/turnstone.git
synced 2026-08-12 23:12:23 -06:00
dc464ac3135b21cfe5680e205cd4e2a303a568a2
11 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Patrick Buckley
|
dc464ac313 |
feat: Agent Skills standard compliance + frontend spec fields
Brings skills implementation into full compliance with agentskills.io: Parser: - Read `allowed-tools` (hyphenated, standard) only; stored as `allowed_tools` internally — no underscore fallback - Reject consecutive hyphens in skill names - Extract author/version from standard `metadata:` map with top-level fallback; null-safe (no "None" string for bare YAML keys) - Truncate description at 1024 chars, compatibility at 500 chars (spec caps) with log warnings - Lenient parsing mode (lenient=True) for cross-client import: sanitizes names, returns None on skip, malformed-YAML colon-value retry - Type overloads: strict mode returns ParsedSkill, lenient returns ParsedSkill | None Session: - `<available-skills>` XML catalog in system messages for activation="search" skills (disabled ones filtered out, capped at 30) Tool rename: - `load_skill` tool → `skill` (JSON, session preparers/executors, approval labels, tests, docs) Storage (migration 023): - Add `license` and `compatibility` columns to prompt_templates - skill_license / compatibility params on create_prompt_template across protocol, SQLite, PostgreSQL backends - Add to SKILL_MUTABLE for update_prompt_template API + server: - SkillInfo, CreateSkillRequest, UpdateSkillRequest: license + compatibility fields - Create/update/install endpoints extract and persist both fields - Install endpoint maps parsed.license + parsed.compatibility from imported SKILL.md (previously discarded) - _skill_to_response() includes both fields Admin UI: - Create + edit modals: version, license, compatibility fields - Readonly (imported) skills: "edit" → "view" button, modal title "View Skill", all fields disabled, Save hidden, Cancel → "Close", collapsibles auto-expand, focus on Close button - :disabled CSS for dark-theme modal inputs (bg-highlight, cursor not-allowed, dimmed text) - Fix addEventListener stacking on auto-approve checkboxes → .onchange SDK: license + compatibility on SkillInfo, CreateSkillRequest, UpdateSkillRequest TypeScript interfaces Docs: governance.md, judge.md, tools.md, README, diagram updated |
||
|
Patrick Buckley
|
1010f163f0 |
feat: load_skill built-in tool — model-driven skill discovery and act… (#112)
* feat: load_skill built-in tool — model-driven skill discovery and activation Two-action tool: 'search' finds skills by multi-word query with substring matching on name/description/tags/category (auto-approved, read-only); 'load' activates a skill by name via set_skill() (requires approval). Guards: filters disabled skills from search + load; short-circuits when skill is already active; approval_label includes skill name for granular tool policies (load_skill__<name>); main session only (excluded from sub-agents). Logs storage errors in search path. 25 tests covering registration, preparer validation, executor logic, disabled/already-active edge cases, multi-word queries, approval labels. * refactor: use BM25 relevance ranking for load_skill search Replace substring matching with BM25Index from turnstone/core/bm25.py, matching the pattern used by memory relevance and tool search. Handles multi-word queries, term frequency, and document length normalization. * fix: address copilot review — BM25 tags parsing, primary_key, test cleanup - Parse JSON tags into space-separated text before BM25 indexing so individual tag terms match queries (was passing raw '["foo","bar"]') - Add primary_key: "name" to load_skill.json for PRIMARY_KEY_MAP - Remove dead resolve_workstream patch from test helper - Update diagram: "substring match" → "BM25 ranking" |
||
|
Patrick Buckley
|
c28bfc1e58 |
feat: skill discovery — search and install skills from external sources (#111)
* feat: skill discovery — search and install skills from external sources Add discovery UI and API for finding and installing skills from skills.sh registries and GitHub repositories with one-click install, SKILL.md frontmatter parsing, and security scan integration. Core modules: - skill_parser.py: ParsedSkill dataclass, parse_skill_md() with YAML frontmatter support (Anthropic + Hermes tag formats), name validation - skill_sources.py: SkillsShClient (async search + resolve), fetch_skill_from_github (SKILL.md + bundled resource fetching with 256KB cap, text extension filter, GitHub API tree traversal) API: - GET /v1/api/admin/skills/discover — search with installed annotation and scan_status for installed skills - POST /v1/api/admin/skills/install — fetch, parse, duplicate check, create with origin="source" readonly=true, store resources, audit Also fixes pre-existing bug where _skill_to_response omitted scan_status, scan_report, scan_version fields — scan tier badges in the installed skills table were silently empty despite data existing in storage. Admin UI: pill toggle (Installed/Discover), discovery cards with scan tier badges, GitHub import modal with proper focus trap/Escape/backdrop, scoped selectors preventing MCP↔Skills cross-tab state corruption. SDK: discover_skills() + install_skill() on Python (async+sync) and TypeScript console clients. 48 new tests across 3 test files. All 2632 tests pass. * fix: address copilot review — 404 vs 502, O(n) lookups, branch fallback - SkillNotFoundError subclass: install returns 404 when SKILL.md is missing, 502 only for connectivity/upstream errors - get_skill_by_source_url() + list_installed_skill_urls(): indexed storage lookups replace O(n) full-table scans with content blobs - Default branch fallback: tries main then master when URL doesn't specify a branch - Path normalization: strip trailing slash once, remove redundant candidate - SDK install_skill() returns typed SkillInfo with response_model - Tree size guard: skip resource tree if response >2MB |
||
|
Patrick Buckley
|
9b605f81a3 |
feat: skill scanner — evaluate SKILL.md content at install time
Add turnstone/core/skill_scanner.py — a production content scanner that evaluates skill risk across four axes: 1. Content risk: command execution, external downloads, credential handling, data exfiltration, eval/exec, sudo, browser automation 2. Supply chain risk: pipe-to-shell, transitive installs, obfuscation, download-exec chains, executable URLs from untrusted domains 3. Vulnerability risk: prompt injection (E004), insecure credential handling (W007), third-party content exposure (W011) 4. Declared capability risk: parsed from allowed_tools field — Bash(*) is high, Bash(git:*) is low, read-only tools are safe Composite score with equal 25% weights per axis. Floor rule: any single axis at critical forces composite to at least medium tier. Wired into both SQLite and PostgreSQL storage backends: - scan_skill() runs at create_prompt_template time - Re-scan triggers on update when content or allowed_tools change - Results populate the existing scan_status and scan_report columns - Silent failure on scanner errors (never blocks skill creation) Scanner helper factored into _utils.py (shared across backends). 23 unit tests covering tier classification, capability scoring, negation filtering, floor rule, serialization, and trusted domains. |
||
|
Patrick Buckley
|
75eda9a096 |
feat: unified skills system — merge prompt templates + workstream tem… (#106)
* feat: unified skills system — merge prompt templates + workstream templates Evolves prompt_templates into a first-class skills entity and merges workstream templates into the same model, collapsing two concepts into one. Migration 021: 21 new columns on prompt_templates (skills metadata, security scan fields, session config from WS templates), skill_resources table for bundled files, skill_versions table for auto-snapshot version history. Data migration converts existing WS templates into skills with name collision handling, migrates version history, renames workstreams and scheduled_tasks columns, cleans orphaned permissions, drops old tables. Key changes: - All public interfaces renamed: templates → skills (API, CLI, SDK, UI) - Session config (model, temperature, token_budget, auto_approve, etc.) now lives on the skill and is applied at workstream creation - /skill slash command, set_skill() API, --skill CLI flag - BM25 skill search via SkillSearchManager for activation="search" skills - Admin UI: Skills tab with collapsible Session Config section, description subtitles, activation/origin/MCP badges, pagination - Shared validation helper (_parse_skill_session_config) for DRY CRUD - Version history with auto-snapshot on every edit + API endpoint - Cascade delete (resources + versions) on skill removal - Security: range validation, activation allowlist, fail-closed enabled check, duplicate name 409, readonly guard, JSON validation - 77 new tests across storage, runtime, search, API integration, and migration behavior verification (2521 total) * fix: address Copilot review + rename admin.templates → admin.skills - Skip skill lookup when resume_ws is set (avoids spurious 400) - Fix _applied_skill_version mismatch (1 in both workstreams table and session) - Remove stale template field from MQ protocol diagram - Rename admin.templates permission to admin.skills everywhere (runtime, frontend, tests, docs) with migration step for persisted role data - Fix stale /api/templates references in docs and diagrams - Update docstrings/comments for skills terminology * fix: address Copilot round 2 — skill version lineage + stale doc refs - Compute actual skill version from skill_versions count (not hardcoded 1) - Use same version in both workstreams table and session metadata - Fix response payload example: "templates" → "skills" key - Fix "Each template summary" → "Each skill summary" |
||
|
Patrick Buckley
|
80e1924d7f |
feat: enable prompt caching for Anthropic and OpenAI providers (#104)
* feat: enable prompt caching for Anthropic and OpenAI providers
Activate automatic prompt caching on both LLM providers to reduce input
token costs on multi-turn conversations. Anthropic gets cache_control:
ephemeral (90% savings on cache hits), OpenAI GPT-5.x gets 24h extended
cache retention (free). Cache metrics flow end-to-end through the entire
data pipeline: provider → session → server SSE → MQ protocol → storage →
Prometheus metrics → admin Usage tab.
- AnthropicProvider: top-level cache_control on all requests, extract
cache_creation_input_tokens and cache_read_input_tokens from streaming
and non-streaming responses
- OpenAIProvider: prompt_cache_retention=24h for GPT-5.x, extract
cached_tokens from usage.prompt_tokens_details
- UsageInfo: new cache_creation_tokens and cache_read_tokens fields
- Migration 020: add cache columns to usage_events table
- Storage: record_usage_event and query_usage updated (sqlite + pg)
- Metrics: turnstone_tokens_total{type="cache_creation|cache_read"}
- Server: on_status passes cache tokens to SSE, storage, and metrics
- MQ: StatusEvent carries cache fields through bridge
- SDKs: Python and TypeScript StatusEvent types updated
- OpenAPI: UsageBreakdownItem schema includes cache fields
- Console UI: Usage tab shows cache write/read as secondary readouts
with visual separator, dimmed when zero
- 16 new tests, docs and 3 diagrams updated
* fix: address Copilot review feedback
- Fix MQ protocol diagram clipping by switching to vertical package
layout (inbound on top, outbound below) with package aliases
- Regenerate OpenAPI snapshots to include cache_creation_tokens and
cache_read_tokens on UsageBreakdownItem
- Replace fragile MagicMock(spec=[]) + del pattern with
types.SimpleNamespace in cache metrics missing-attributes test
|
||
|
Patrick Buckley
|
8895bf07eb |
feat: admin Settings tab — form-based editor replacing "coming soon" … (#60)
* feat: admin Settings tab — form-based editor replacing "coming soon" stub Section-grouped layout with collapsible headers for all ~40 ConfigStore settings (model, session, tools, server, mcp, ratelimit, health, judge, memory). Type-appropriate inputs: CSS toggle for bools, number with min/max/step, select for choices, text for strings. Secret fields shown read-only. Source badge (storage/default), amber restart indicator. Inline save per field with dirty detection, row flash on success, reset to default via styled confirm modal. Full WCAG keyboard accessibility (Enter/Space on section headers, aria-labels, focus-visible). Mobile responsive single-column at <700px. Reduced-motion safe. * fix: Settings tab polish — help tooltips, context_window auto-detect, UX fixes Settings UI: - Help tooltips: ? button on ~25 settings with plain-English explanations and optional reference links (arXiv, Fowler, MCP spec). Click to toggle popover, Escape to dismiss, aria-expanded for accessibility. - Sections start collapsed for scannable overview. - Restart badge: hidden by default, shows when dirty, persists after save with amber glow. Positioned left of source badge. - Secret row alignment fixed (transparent border matches input box model). - Docs link in toolbar → Swagger UI Settings section. - Number inputs: spin buttons hidden (Firefox/WebKit), empty value guard, numeric dirty detection (0.1 vs 0.10 no longer false positive). - Secret reset button enabled when source=storage (clear legacy overrides). - Space key repeat guard on section headers. - Sidebar: sticky + max-height:100vh, no longer stretches with content. Backend: - context_window default changed from 131072 to 0 (auto-detect). Fallback lowered from 131K to 32K (realistic for local models when detection fails). Session normalizes 0→32768 defensively. - Settings registry: help + reference_url fields on SettingDef, richer descriptions for model/session/tools/judge/memory settings. - Schema API includes help + reference_url. - Bootstrap system prompt: added Runtime Settings section. Docs: tab counts updated to 13 across README, architecture, console, governance. |
||
|
Patrick Buckley
|
02d9c5c797 |
feat: workstream templates — behavioral profiles for workstream creation (#49)
* feat: workstream templates — behavioral profiles for workstream creation Workstream templates define the complete configuration for workstream creation: system prompt, model, auto-approve policy, per-tool auto-approve, temperature, reasoning effort, max tokens, agent max turns, token budget, and completion notifications. Applied once at creation time (snapshot, not live binding). Auto-versioning captures pre-update state on every edit. Schema & storage: - workstream_templates + workstream_template_versions tables (migration 011) - ws_template_id/ws_template_version columns on workstreams table - ws_template column on scheduled_tasks table - Full CRUD + versioning on SQLite and PostgreSQL backends - prompt_template_hash (SHA-256) for drift detection Runtime: - Template resolution before mgr.create() for model override - Post-creation settings application (prompt, temperature, approval, budget) - Token budget enforcement in session.send() — 80% warning, approval gate at 100% via __budget_override__ synthetic tool - WebUI.auto_approve_tools server-side per-tool auto-approve - Prompt template drift detection (hash comparison, log warning on mismatch) Integration: - ws_template field on CreateWorkstreamMessage, bridge, channel router, scheduler dispatch, MQ client - Console admin "WS Templates" tab (11th) with CRUD, version history modal - Profile dropdown on workstream creation modal - WS template dropdown on scheduler create/edit modals - Prompt template name validation on ws_template create/update - 7 console admin API endpoints + read-only summary endpoint - Full OpenAPI spec entries in console_spec.py - Python SDK (sync + async) and TypeScript SDK methods - Pydantic schemas for all request/response models Docs & diagrams: - New 21-ws-template-architecture.puml sequence diagram - Updated governance, storage, MQ protocol diagrams + PNGs - Updated architecture.md, governance.md, api-reference.md, console.md, sdk.md 48 new tests (1788 total). mypy clean. ruff clean. * fix: address PR #49 review feedback - auto_approve_tools uses approval_label (not just func_name) for consistency with tool policy evaluation - inline system_prompt from ws_template persisted as _ws_template_system_prompt in workstream_config, restored on resume (previously lost because _template_content wasn't persisted) - budget gate (__budget_override__) no longer bypassed by blanket auto_approve — requires explicit approval or tool policy allow - diagram 21 field list corrected (removed tool_search/threshold, added prompt_template_hash/notify_on_complete) * fix: address PR #49 review feedback (round 2) - Grant admin.ws_templates permission in migration 011 (tab was hidden) - Center WS template modals and fix radio button alignment - Skip template validation when ws_template overrides prompt - Guard against empty version snapshots on no-op updates - Replace setTimeout race with Promise chain in schedule ws_template select - Validate numeric fields in admin create/update handlers (400 not 500) - Add ws_template to TypeScript OpenAPI specs - Use typed Pydantic response models in SDK ws_template methods |
||
|
Patrick Buckley
|
2f7f70825b |
feat: wire prompt templates into session startup with full creation-p… (#47)
* feat: wire prompt templates into session startup with full creation-path support
Prompt templates (prompt_templates table) now have runtime effect:
- is_default=true templates auto-apply as system message content,
concatenated in name order before user instructions
- Per-workstream template selection via --template CLI flag, template
field on POST /v1/api/workstreams/new, console creation modal dropdown,
scheduled task config, and channel adapter config
- {{model}}, {{ws_id}}, {{node_id}} variable substitution via single-pass
regex (prevents cross-variable injection)
- /template slash command for runtime switching, persisted across resume
- set_template() public API on ChatSession
Security hardening:
- MCP sync resets is_default=False on content update (prevents compromised
server from injecting defaults)
- 32KB content cap on template create/update + defensive truncation
- Template existence validation returns 400 before workstream creation
- Single-pass regex eliminates cross-variable expansion
Template field plumbed through all creation paths: CLI, server API, MQ
protocol/bridge, console backend, scheduler dispatch, channel router,
MQ client. Migration 010 adds template column to scheduled_tasks.
Frontend: console workstream modal template dropdown, scheduler
create/edit template field, governance template UI variables auto-detected
from content (read-only display replaces editable input). Focus trap and
Enter-key accessibility fixes in workstream modal.
Docs: governance.md template runtime section, api-reference.md template
field, governance + MCP architecture diagrams updated.
Python + TypeScript SDKs, Pydantic schemas all updated. 29 new tests.
* fix: address PR #47 review feedback
- Defer template validation until after resume_ws — a bad template name
no longer 400s when resume would have ignored it anyway
- Add template field to OpenAPI JSON specs (openapi-server.json,
openapi-console.json) for SDK/docs consistency
- Validate template existence in schedule create and update endpoints —
reject unknown template names with 400 instead of allowing schedules
that would silently fail at dispatch time
|
||
|
Patrick Buckley
|
be165c1971 |
feat: MCP resource and prompt discovery with read_resource tool (#44)
* feat: MCP resource and prompt discovery with read_resource tool Extends MCPClientManager with resource and prompt discovery alongside existing tool support. Resources and prompts are discovered on connect, cached per-server with copy-on-write rebuilds, and refreshed via push notifications, periodic polling, or manual /mcp refresh. New read_resource built-in tool reads MCP resources by URI. Requires user approval (same as MCP tool calls) since resources are served by external MCP servers. Resource catalog injected into system message with XML delimiters. Error messages sanitized to prevent leaking server internals to the model. Prompt discovery stores prefixed names (mcp__server__prompt) and exposes get_prompt_sync() for future use_prompt tool (Chunk D). /mcp command now shows tools, resources, and prompts. Docs and diagrams updated. * feat: MCP prompt governance sync with origin tracking and readonly guards Migration 009 adds origin, mcp_server, and readonly columns to prompt_templates. MCP prompts discovered by MCPClientManager are automatically synced into the governance table as read-only templates with origin="mcp". Sync engine handles: create on connect, update on prompt refresh, delete when prompts are removed from server. Manual templates take precedence on name collision (MCP prompt skipped with warning). Admin API returns 403 on update/delete of readonly templates. Console UI shows MCP origin badge and disables edit/delete buttons. Storage backends gain get_prompt_template_by_name, list_prompt_templates_by_origin, and delete_prompt_templates_by_server methods. Also addresses PR #44 review feedback: concurrent.futures.TimeoutError handling in sync dispatch, XML-escape resource catalog descriptions, resource template entries excluded from _resource_map, URI collision warnings, needs_periodic capability-aware computation, malformed JSON primary key fallback for read_resource. * feat: use_prompt tool, prompt catalog, and PR review hardening New use_prompt built-in tool invokes MCP prompt templates by name, expanding them into messages. Requires user approval (external MCP servers). Prompt catalog injected into system message with XML delimiters (up to 30 prompts, HTML-escaped). Prompt listener registered in session for catalog rebuild on changes. Addresses PR #44 review feedback: - _init_system_messages() now uses copy-on-write (build locally, assign atomically) so background thread callbacks never see partial system messages - sync_prompts_to_storage() serialized behind _sync_lock to prevent races between set_storage() (main thread) and MCP background thread - shutdown() clears listener lists to release callback references Docs and diagrams updated for 18 built-in tools. * feat: granular tool policies for MCP resources, prompts, and tools Policy evaluation now uses approval_label (falling back to func_name) for fnmatch pattern matching, enabling fine-grained per-URI and per-server policies: - read_resource: mcp_resource__{normalized_uri} - use_prompt: mcp__{server}__{prompt} (prefixed name) - MCP tools: mcp__{server}__{tool} (was static "mcp_tool") URI normalization resolves .. path segments to prevent traversal bypasses in policy matching. Resource templates filtered from system message catalog (not directly readable). use_prompt arguments validated as dict with string coercion. TypeScript SDK PromptTemplateInfo gains origin, mcp_server, readonly fields. Governance docs updated with MCP policy patterns. * feat: MCP visibility in server and console UIs Server health endpoint includes mcp.servers, mcp.resources, mcp.prompts counts. Server UI status bar shows magenta MCP indicator with tooltip. Console cluster status bar shows MCP metrics with magenta LED dot. Console node detail view shows per-node MCP summary. Console collector aggregates MCP counts across nodes in overview. Uses var(--magenta) design token with new --magenta-glow for theme adaptation. ARIA roles on MCP status elements. Tooltips on console MCP metric labels. Node MCP summary hidden on mobile (< 700px). New diagram: 20-mcp-architecture.puml covering full MCP lifecycle (connection, discovery, refresh, governance sync, policy, UI). * fix: McpStatus in health schema, count properties, catalog name fidelity Adds McpStatus model to HealthResponse (Python + TypeScript SDKs) so typed clients see the mcp field from /health. Addresses Copilot review feedback: - resource_count/prompt_count properties avoid list allocation on /health and /metrics polls - get_tools/resources/prompts return shallow-copied dicts to prevent callers from mutating internal cache - Prompt names and arg names in system message catalog are NOT HTML-escaped (model must use exact strings in use_prompt calls); only descriptions are escaped * fix: OpenAPI spec McpStatus + diagram approval column accuracy Adds McpStatus schema and optional mcp field to HealthResponse in openapi-server.json, matching the Python schema and TypeScript types. Fixes tool pipeline diagram: math, web_fetch, web_search correctly shown as auto-approve (not "Yes" for approval). |
||
|
Patrick Buckley
|
7492816ab2 |
feat: governance — RBAC, tool policies, prompt templates, usage track… (#39)
* feat: governance — RBAC, tool policies, prompt templates, usage tracking, audit logging Add comprehensive governance layer for the admin console: - RBAC with 15 granular permissions, 3 builtin roles (admin, operator, viewer), custom role CRUD, user-role assignment with privilege escalation prevention - Tool policies with glob pattern matching, priority-ordered evaluation (allow/deny/ask), enforced before auto-approve in WebUI.approve_tools() - Prompt templates with variable substitution, categories, default flag - Usage tracking: per-LLM-request token/tool metrics, aggregated queries (group by day/model/user), automatic 90-day pruning via scheduler - Audit logging: append-only event trail for all admin mutations, filterable/paginated queries, automatic 365-day pruning, X-Forwarded-For aware IP extraction - require_permission() enforced on all 35+ admin endpoints (users, tokens, channels, schedules, watches, roles, orgs, policies, templates, usage, audit) - Field allowlists on storage update methods prevent mass-assignment bugs - Self-deletion guard on admin_delete_user, delete_user cascades user_roles - _row_to_dict helper eliminates ~400 lines of fragile positional row mapping - _audit_context helper deduplicates 18 instances of audit boilerplate - Migration 008: 7 new tables, 3 builtin roles, org_id on users - Console admin panel: 5 new tabs (Roles, Policies, Templates, Usage, Audit) with permission-gated visibility, 7 modal dialogs, full keyboard accessibility - Python + TypeScript SDK methods for all governance endpoints - 120+ new tests (1554 total) * fix: address PR #39 review feedback - Rebuild serialized items after policy evaluation so denied/allowed verdicts are reflected in tool_info/approve_request SSE payloads - Make `since` query param optional in usage OpenAPI spec (handler already defaults to last 7 days) - Add response_model=StatusResponse to DELETE role/policy/template and POST/DELETE role assignment endpoints in OpenAPI spec - Add missing org_id/created/updated fields to UserRoleInfo schema - Add missing created field to AuditEventInfo schema - Show "no permissions" empty state instead of loading inaccessible tab when all admin tabs are permission-gated - Fix "13 permissions" → "15 permissions" in architecture.md and security.md - Fix import sorting in test_audit.py and test_tool_policy.py * fix: address PR #39 round 2 review feedback - Clear stale permissions from sessionStorage on config-token login (auth.js _storePermissions) - Only trust X-Forwarded-For when behind a proxy that sets X-Forwarded-Proto (conditional on is_secure_request trust model) - Thread user_id from auth into WebUI.on_status for usage events - Add created field to TS AuditEventInfo type - Return typed Pydantic models from all SDK governance methods instead of dict[str, Any] — both async and sync clients - Validate group_by param against allowed enum in admin_usage handler - Add deterministic secondary sort (event_id DESC) to list_audit_events in both SQLite and PostgreSQL backends |