mirror of
https://github.com/turnstonelabs/turnstone.git
synced 2026-08-12 23:12:23 -06:00
Patrick Buckley
be165c1971
* feat: MCP resource and prompt discovery with read_resource tool Extends MCPClientManager with resource and prompt discovery alongside existing tool support. Resources and prompts are discovered on connect, cached per-server with copy-on-write rebuilds, and refreshed via push notifications, periodic polling, or manual /mcp refresh. New read_resource built-in tool reads MCP resources by URI. Requires user approval (same as MCP tool calls) since resources are served by external MCP servers. Resource catalog injected into system message with XML delimiters. Error messages sanitized to prevent leaking server internals to the model. Prompt discovery stores prefixed names (mcp__server__prompt) and exposes get_prompt_sync() for future use_prompt tool (Chunk D). /mcp command now shows tools, resources, and prompts. Docs and diagrams updated. * feat: MCP prompt governance sync with origin tracking and readonly guards Migration 009 adds origin, mcp_server, and readonly columns to prompt_templates. MCP prompts discovered by MCPClientManager are automatically synced into the governance table as read-only templates with origin="mcp". Sync engine handles: create on connect, update on prompt refresh, delete when prompts are removed from server. Manual templates take precedence on name collision (MCP prompt skipped with warning). Admin API returns 403 on update/delete of readonly templates. Console UI shows MCP origin badge and disables edit/delete buttons. Storage backends gain get_prompt_template_by_name, list_prompt_templates_by_origin, and delete_prompt_templates_by_server methods. Also addresses PR #44 review feedback: concurrent.futures.TimeoutError handling in sync dispatch, XML-escape resource catalog descriptions, resource template entries excluded from _resource_map, URI collision warnings, needs_periodic capability-aware computation, malformed JSON primary key fallback for read_resource. * feat: use_prompt tool, prompt catalog, and PR review hardening New use_prompt built-in tool invokes MCP prompt templates by name, expanding them into messages. Requires user approval (external MCP servers). Prompt catalog injected into system message with XML delimiters (up to 30 prompts, HTML-escaped). Prompt listener registered in session for catalog rebuild on changes. Addresses PR #44 review feedback: - _init_system_messages() now uses copy-on-write (build locally, assign atomically) so background thread callbacks never see partial system messages - sync_prompts_to_storage() serialized behind _sync_lock to prevent races between set_storage() (main thread) and MCP background thread - shutdown() clears listener lists to release callback references Docs and diagrams updated for 18 built-in tools. * feat: granular tool policies for MCP resources, prompts, and tools Policy evaluation now uses approval_label (falling back to func_name) for fnmatch pattern matching, enabling fine-grained per-URI and per-server policies: - read_resource: mcp_resource__{normalized_uri} - use_prompt: mcp__{server}__{prompt} (prefixed name) - MCP tools: mcp__{server}__{tool} (was static "mcp_tool") URI normalization resolves .. path segments to prevent traversal bypasses in policy matching. Resource templates filtered from system message catalog (not directly readable). use_prompt arguments validated as dict with string coercion. TypeScript SDK PromptTemplateInfo gains origin, mcp_server, readonly fields. Governance docs updated with MCP policy patterns. * feat: MCP visibility in server and console UIs Server health endpoint includes mcp.servers, mcp.resources, mcp.prompts counts. Server UI status bar shows magenta MCP indicator with tooltip. Console cluster status bar shows MCP metrics with magenta LED dot. Console node detail view shows per-node MCP summary. Console collector aggregates MCP counts across nodes in overview. Uses var(--magenta) design token with new --magenta-glow for theme adaptation. ARIA roles on MCP status elements. Tooltips on console MCP metric labels. Node MCP summary hidden on mobile (< 700px). New diagram: 20-mcp-architecture.puml covering full MCP lifecycle (connection, discovery, refresh, governance sync, policy, UI). * fix: McpStatus in health schema, count properties, catalog name fidelity Adds McpStatus model to HealthResponse (Python + TypeScript SDKs) so typed clients see the mcp field from /health. Addresses Copilot review feedback: - resource_count/prompt_count properties avoid list allocation on /health and /metrics polls - get_tools/resources/prompts return shallow-copied dicts to prevent callers from mutating internal cache - Prompt names and arg names in system message catalog are NOT HTML-escaped (model must use exact strings in use_prompt calls); only descriptions are escaped * fix: OpenAPI spec McpStatus + diagram approval column accuracy Adds McpStatus schema and optional mcp field to HealthResponse in openapi-server.json, matching the Python schema and TypeScript types. Fixes tool pipeline diagram: math, web_fetch, web_search correctly shown as auto-approve (not "Yes" for approval).
168 lines
7.0 KiB
Python
168 lines
7.0 KiB
Python
"""Tests for turnstone.core.policy."""
|
|
|
|
import pytest
|
|
|
|
from turnstone.core.policy import evaluate_tool_policies_batch, evaluate_tool_policy
|
|
from turnstone.core.storage._sqlite import SQLiteBackend
|
|
|
|
|
|
@pytest.fixture
|
|
def storage(tmp_path):
|
|
path = str(tmp_path / "test.db")
|
|
backend = SQLiteBackend(path)
|
|
yield backend
|
|
backend.close()
|
|
|
|
|
|
def test_no_policies_returns_none(storage):
|
|
result = evaluate_tool_policy(storage, "bash")
|
|
assert result is None
|
|
|
|
|
|
def test_exact_match_allow(storage):
|
|
storage.create_tool_policy("p1", "allow-read", "read_file", "allow", 0)
|
|
assert evaluate_tool_policy(storage, "read_file") == "allow"
|
|
assert evaluate_tool_policy(storage, "write_file") is None
|
|
|
|
|
|
def test_glob_match_deny(storage):
|
|
storage.create_tool_policy("p1", "block-bash", "bash*", "deny", 0)
|
|
assert evaluate_tool_policy(storage, "bash") == "deny"
|
|
assert evaluate_tool_policy(storage, "bash_exec") == "deny"
|
|
assert evaluate_tool_policy(storage, "read_file") is None
|
|
|
|
|
|
def test_wildcard_match(storage):
|
|
storage.create_tool_policy("p1", "ask-all", "*", "ask", 0)
|
|
assert evaluate_tool_policy(storage, "anything") == "ask"
|
|
|
|
|
|
def test_priority_ordering(storage):
|
|
# Higher priority wins
|
|
storage.create_tool_policy("p1", "allow-all", "*", "allow", 0)
|
|
storage.create_tool_policy("p2", "deny-bash", "bash*", "deny", 100)
|
|
assert evaluate_tool_policy(storage, "bash") == "deny" # p2 matches first (higher priority)
|
|
assert evaluate_tool_policy(storage, "read_file") == "allow" # p1 matches
|
|
|
|
|
|
def test_disabled_policy_skipped(storage):
|
|
storage.create_tool_policy("p1", "block-bash", "bash*", "deny", 100, enabled=False)
|
|
storage.create_tool_policy("p2", "allow-all", "*", "allow", 0)
|
|
assert evaluate_tool_policy(storage, "bash") == "allow" # p1 disabled, falls through to p2
|
|
|
|
|
|
def test_batch_evaluation(storage):
|
|
storage.create_tool_policy("p1", "block-bash", "bash*", "deny", 100)
|
|
storage.create_tool_policy("p2", "allow-read", "read_*", "allow", 50)
|
|
results = evaluate_tool_policies_batch(storage, ["bash", "read_file", "write_file"])
|
|
assert results["bash"] == "deny"
|
|
assert results["read_file"] == "allow"
|
|
assert results["write_file"] is None
|
|
|
|
|
|
def test_storage_failure_returns_none():
|
|
"""Graceful degradation on storage failure."""
|
|
|
|
class BrokenStorage:
|
|
def list_tool_policies(self, org_id=""):
|
|
raise RuntimeError("boom")
|
|
|
|
assert evaluate_tool_policy(BrokenStorage(), "bash") is None
|
|
|
|
|
|
def test_batch_storage_failure():
|
|
class BrokenStorage:
|
|
def list_tool_policies(self, org_id=""):
|
|
raise RuntimeError("boom")
|
|
|
|
results = evaluate_tool_policies_batch(BrokenStorage(), ["a", "b"])
|
|
assert results == {"a": None, "b": None}
|
|
|
|
|
|
def test_first_match_wins(storage):
|
|
# Two policies match, first by priority wins
|
|
storage.create_tool_policy("p1", "deny-bash", "bash*", "deny", 100)
|
|
storage.create_tool_policy("p2", "allow-bash", "bash*", "allow", 50)
|
|
assert evaluate_tool_policy(storage, "bash_exec") == "deny"
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# MCP resource and prompt policy patterns
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
def test_mcp_resource_wildcard_deny(storage):
|
|
"""Deny all MCP resource reads via glob pattern."""
|
|
storage.create_tool_policy("p1", "block-resources", "mcp_resource__*", "deny", 100)
|
|
assert evaluate_tool_policy(storage, "mcp_resource__file:///secret.txt") == "deny"
|
|
assert evaluate_tool_policy(storage, "mcp_resource__db://users") == "deny"
|
|
assert evaluate_tool_policy(storage, "read_file") is None # unrelated tool
|
|
|
|
|
|
def test_mcp_resource_per_server_pattern(storage):
|
|
"""Allow resources from a specific server, deny others."""
|
|
storage.create_tool_policy("p1", "block-all-resources", "mcp_resource__*", "deny", 50)
|
|
storage.create_tool_policy("p2", "allow-docs", "mcp_resource__file:///docs/*", "allow", 100)
|
|
assert evaluate_tool_policy(storage, "mcp_resource__file:///docs/readme.md") == "allow"
|
|
assert evaluate_tool_policy(storage, "mcp_resource__file:///etc/passwd") == "deny"
|
|
|
|
|
|
def test_mcp_prompt_wildcard_ask(storage):
|
|
"""Require approval for all MCP prompt invocations."""
|
|
storage.create_tool_policy("p1", "ask-prompts", "mcp__*", "ask", 100)
|
|
assert evaluate_tool_policy(storage, "mcp__github__code_review") == "ask"
|
|
assert evaluate_tool_policy(storage, "mcp__templates__greeting") == "ask"
|
|
assert evaluate_tool_policy(storage, "bash") is None
|
|
|
|
|
|
def test_mcp_prompt_per_server_allow(storage):
|
|
"""Auto-approve prompts from a trusted server."""
|
|
storage.create_tool_policy("p1", "ask-all-mcp", "mcp__*", "ask", 50)
|
|
storage.create_tool_policy("p2", "allow-trusted", "mcp__trusted__*", "allow", 100)
|
|
assert evaluate_tool_policy(storage, "mcp__trusted__greeting") == "allow"
|
|
assert evaluate_tool_policy(storage, "mcp__untrusted__evil") == "ask"
|
|
|
|
|
|
def test_mcp_batch_mixed(storage):
|
|
"""Batch evaluation with mixed MCP and built-in tools."""
|
|
storage.create_tool_policy("p1", "block-resources", "mcp_resource__*", "deny", 100)
|
|
storage.create_tool_policy("p2", "allow-prompts", "mcp__trusted__*", "allow", 100)
|
|
results = evaluate_tool_policies_batch(
|
|
storage,
|
|
["mcp_resource__file:///x", "mcp__trusted__greeting", "bash", "mcp__other__y"],
|
|
)
|
|
assert results["mcp_resource__file:///x"] == "deny"
|
|
assert results["mcp__trusted__greeting"] == "allow"
|
|
assert results["bash"] is None
|
|
assert results["mcp__other__y"] is None
|
|
|
|
|
|
def test_normalize_resource_uri_prevents_traversal():
|
|
"""URI normalization resolves .. segments to prevent policy traversal bypass."""
|
|
from turnstone.core.session import ChatSession
|
|
|
|
# Normal URI unchanged
|
|
assert ChatSession._normalize_resource_uri("file:///docs/readme.md") == "file:///docs/readme.md"
|
|
# Traversal resolved
|
|
assert ChatSession._normalize_resource_uri("file:///docs/../etc/passwd") == "file:///etc/passwd"
|
|
# Double traversal
|
|
assert ChatSession._normalize_resource_uri("file:///a/b/../../c") == "file:///c"
|
|
# Non-file scheme (netloc preserved, path normalized)
|
|
assert ChatSession._normalize_resource_uri("db://host/tables/../secrets") == "db://host/secrets"
|
|
# Percent-encoded traversal decoded before normalization
|
|
assert (
|
|
ChatSession._normalize_resource_uri("file:///docs/%2e%2e/etc/passwd")
|
|
== "file:///etc/passwd"
|
|
)
|
|
# Mixed percent-encoded and literal traversal
|
|
assert ChatSession._normalize_resource_uri("file:///a/%2e%2e/b/../c") == "file:///c"
|
|
|
|
|
|
def test_mcp_tool_granular_policy(storage):
|
|
"""MCP tool calls use their prefixed func_name for granular policy matching."""
|
|
storage.create_tool_policy("p1", "ask-all-mcp", "mcp__*", "ask", 50)
|
|
storage.create_tool_policy("p2", "allow-github", "mcp__github__*", "allow", 100)
|
|
# MCP tools now use func_name as approval_label
|
|
assert evaluate_tool_policy(storage, "mcp__github__search") == "allow"
|
|
assert evaluate_tool_policy(storage, "mcp__untrusted__exec") == "ask"
|