mirror of
https://github.com/turnstonelabs/turnstone.git
synced 2026-08-12 23:12:23 -06:00
feat: MCP resource and prompt discovery with read_resource tool (#44)
* feat: MCP resource and prompt discovery with read_resource tool Extends MCPClientManager with resource and prompt discovery alongside existing tool support. Resources and prompts are discovered on connect, cached per-server with copy-on-write rebuilds, and refreshed via push notifications, periodic polling, or manual /mcp refresh. New read_resource built-in tool reads MCP resources by URI. Requires user approval (same as MCP tool calls) since resources are served by external MCP servers. Resource catalog injected into system message with XML delimiters. Error messages sanitized to prevent leaking server internals to the model. Prompt discovery stores prefixed names (mcp__server__prompt) and exposes get_prompt_sync() for future use_prompt tool (Chunk D). /mcp command now shows tools, resources, and prompts. Docs and diagrams updated. * feat: MCP prompt governance sync with origin tracking and readonly guards Migration 009 adds origin, mcp_server, and readonly columns to prompt_templates. MCP prompts discovered by MCPClientManager are automatically synced into the governance table as read-only templates with origin="mcp". Sync engine handles: create on connect, update on prompt refresh, delete when prompts are removed from server. Manual templates take precedence on name collision (MCP prompt skipped with warning). Admin API returns 403 on update/delete of readonly templates. Console UI shows MCP origin badge and disables edit/delete buttons. Storage backends gain get_prompt_template_by_name, list_prompt_templates_by_origin, and delete_prompt_templates_by_server methods. Also addresses PR #44 review feedback: concurrent.futures.TimeoutError handling in sync dispatch, XML-escape resource catalog descriptions, resource template entries excluded from _resource_map, URI collision warnings, needs_periodic capability-aware computation, malformed JSON primary key fallback for read_resource. * feat: use_prompt tool, prompt catalog, and PR review hardening New use_prompt built-in tool invokes MCP prompt templates by name, expanding them into messages. Requires user approval (external MCP servers). Prompt catalog injected into system message with XML delimiters (up to 30 prompts, HTML-escaped). Prompt listener registered in session for catalog rebuild on changes. Addresses PR #44 review feedback: - _init_system_messages() now uses copy-on-write (build locally, assign atomically) so background thread callbacks never see partial system messages - sync_prompts_to_storage() serialized behind _sync_lock to prevent races between set_storage() (main thread) and MCP background thread - shutdown() clears listener lists to release callback references Docs and diagrams updated for 18 built-in tools. * feat: granular tool policies for MCP resources, prompts, and tools Policy evaluation now uses approval_label (falling back to func_name) for fnmatch pattern matching, enabling fine-grained per-URI and per-server policies: - read_resource: mcp_resource__{normalized_uri} - use_prompt: mcp__{server}__{prompt} (prefixed name) - MCP tools: mcp__{server}__{tool} (was static "mcp_tool") URI normalization resolves .. path segments to prevent traversal bypasses in policy matching. Resource templates filtered from system message catalog (not directly readable). use_prompt arguments validated as dict with string coercion. TypeScript SDK PromptTemplateInfo gains origin, mcp_server, readonly fields. Governance docs updated with MCP policy patterns. * feat: MCP visibility in server and console UIs Server health endpoint includes mcp.servers, mcp.resources, mcp.prompts counts. Server UI status bar shows magenta MCP indicator with tooltip. Console cluster status bar shows MCP metrics with magenta LED dot. Console node detail view shows per-node MCP summary. Console collector aggregates MCP counts across nodes in overview. Uses var(--magenta) design token with new --magenta-glow for theme adaptation. ARIA roles on MCP status elements. Tooltips on console MCP metric labels. Node MCP summary hidden on mobile (< 700px). New diagram: 20-mcp-architecture.puml covering full MCP lifecycle (connection, discovery, refresh, governance sync, policy, UI). * fix: McpStatus in health schema, count properties, catalog name fidelity Adds McpStatus model to HealthResponse (Python + TypeScript SDKs) so typed clients see the mcp field from /health. Addresses Copilot review feedback: - resource_count/prompt_count properties avoid list allocation on /health and /metrics polls - get_tools/resources/prompts return shallow-copied dicts to prevent callers from mutating internal cache - Prompt names and arg names in system message catalog are NOT HTML-escaped (model must use exact strings in use_prompt calls); only descriptions are escaped * fix: OpenAPI spec McpStatus + diagram approval column accuracy Adds McpStatus schema and optional mcp field to HealthResponse in openapi-server.json, matching the Python schema and TypeScript types. Fixes tool pipeline diagram: math, web_fetch, web_search correctly shown as auto-approve (not "Yes" for approval).
This commit is contained in:
@@ -3,7 +3,7 @@
|
||||
Turnstone is an AI orchestration platform with tool use, parallel workstreams, and persistent
|
||||
memory. It connects to any OpenAI-compatible API (local vLLM, OpenAI, etc.) or
|
||||
Anthropic's native Messages API via pluggable provider adapters, and gives the
|
||||
model 14 built-in tools plus external tools via MCP (Model Context Protocol) for
|
||||
model 18 built-in tools plus external tools via MCP (Model Context Protocol) for
|
||||
reading, writing, searching, planning, and executing code.
|
||||
|
||||
The core design principle is a **UI-agnostic engine with pluggable frontends**.
|
||||
|
||||
@@ -96,7 +96,7 @@ package "turnstone/sdk/" <<Rectangle>> {
|
||||
|
||||
' Tool schemas
|
||||
package "turnstone/tools/" <<Rectangle>> {
|
||||
component [*.json\n15 tool schemas] as schemas <<artifact>>
|
||||
component [*.json\n18 tool schemas] as schemas <<artifact>>
|
||||
}
|
||||
|
||||
' Entry point dependencies
|
||||
|
||||
@@ -211,15 +211,23 @@ enum "WorkstreamState" as WsState {
|
||||
class "MCPClientManager" as MCPMgr {
|
||||
- _sessions: dict[str, ClientSession]
|
||||
- _per_server_tools: dict[str, list[dict]]
|
||||
- _per_server_resources: dict[str, list[dict]]
|
||||
- _per_server_prompts: dict[str, list[dict]]
|
||||
- _tools: list[dict]
|
||||
- _tool_map: dict[str, tuple]
|
||||
- _resource_map: dict[str, tuple]
|
||||
- _prompt_map: dict[str, tuple]
|
||||
- _supports_list_changed: dict[str, bool]
|
||||
- _listeners: list[Callable]
|
||||
--
|
||||
+ start()
|
||||
+ get_tools() → list[dict]
|
||||
+ get_resources() → list[dict]
|
||||
+ get_prompts() → list[dict]
|
||||
+ is_mcp_tool(name) → bool
|
||||
+ call_tool_sync(name, args) → str
|
||||
+ read_resource_sync(uri) → str
|
||||
+ get_prompt_sync(name, args?) → list[dict]
|
||||
+ refresh_sync(server?) → dict
|
||||
+ add_listener(callback)
|
||||
+ remove_listener(callback)
|
||||
@@ -230,6 +238,8 @@ class "MCPClientManager" as MCPMgr {
|
||||
bridges async MCP SDK to
|
||||
sync ChatSession dispatch.
|
||||
Push + periodic + manual refresh.
|
||||
Resources + prompts discovered
|
||||
alongside tools at startup.
|
||||
--
|
||||
core/mcp_client.py
|
||||
}
|
||||
|
||||
@@ -24,29 +24,31 @@ partition "Phase 1: Prepare" #E8F5E9 {
|
||||
:Dispatch to _prepare_{func_name}();
|
||||
|
||||
note right
|
||||
**Dispatch table (16 tools):**
|
||||
┌──────────────┬──────────────────┐
|
||||
│ Tool │ Needs Approval? │
|
||||
├──────────────┼──────────────────┤
|
||||
│ bash │ ✓ Yes │
|
||||
│ read_file │ ✗ Auto-approve │
|
||||
│ write_file │ ✓ Yes │
|
||||
│ edit_file │ ✓ Yes │
|
||||
│ search │ ✗ Auto-approve │
|
||||
│ math │ ✓ Yes │
|
||||
│ man │ ✗ Auto-approve │
|
||||
│ web_fetch │ ✓ Yes │
|
||||
│ web_search │ ✓ Yes │
|
||||
│ tool_search │ ✗ Auto-approve │
|
||||
│ task │ ✓ Yes │
|
||||
│ plan │ ✓ Yes │
|
||||
│ remember │ ✗ Auto-approve │
|
||||
│ recall │ ✗ Auto-approve │
|
||||
│ forget │ ✗ Auto-approve │
|
||||
│ notify │ ✗ Auto-approve │
|
||||
├──────────────┼──────────────────┤
|
||||
│ mcp__* │ ✓ Yes (external) │
|
||||
└──────────────┴──────────────────┘
|
||||
**Dispatch table (18 tools):**
|
||||
┌───────────────┬──────────────────┐
|
||||
│ Tool │ Needs Approval? │
|
||||
├───────────────┼──────────────────┤
|
||||
│ bash │ ✓ Yes │
|
||||
│ read_file │ ✗ Auto-approve │
|
||||
│ write_file │ ✓ Yes │
|
||||
│ edit_file │ ✓ Yes │
|
||||
│ search │ ✗ Auto-approve │
|
||||
│ math │ ✗ Auto-approve │
|
||||
│ man │ ✗ Auto-approve │
|
||||
│ web_fetch │ ✗ Auto-approve │
|
||||
│ web_search │ ✗ Auto-approve │
|
||||
│ tool_search │ ✗ Auto-approve │
|
||||
│ task │ ✓ Yes │
|
||||
│ plan │ ✓ Yes │
|
||||
│ remember │ ✗ Auto-approve │
|
||||
│ recall │ ✗ Auto-approve │
|
||||
│ forget │ ✗ Auto-approve │
|
||||
│ notify │ ✗ Auto-approve │
|
||||
│ read_resource │ ✓ Yes │
|
||||
│ use_prompt │ ✓ Yes │
|
||||
├───────────────┼──────────────────┤
|
||||
│ mcp__* │ ✓ Yes (external) │
|
||||
└───────────────┴──────────────────┘
|
||||
end note
|
||||
|
||||
:Build item dict:
|
||||
@@ -117,6 +119,8 @@ partition "Phase 3: Execute" #E3F2FD {
|
||||
├─ _exec_remember: SQLite INSERT OR REPLACE
|
||||
├─ _exec_recall: SQLite FTS5/LIKE search
|
||||
├─ _exec_forget: SQLite DELETE
|
||||
├─ _exec_read_resource: MCPClientManager.read_resource_sync()
|
||||
├─ _exec_use_prompt: MCPClientManager.get_prompt_sync()
|
||||
└─ _exec_mcp_tool: MCPClientManager.call_tool_sync()
|
||||
end note
|
||||
|
||||
|
||||
@@ -0,0 +1,157 @@
|
||||
@startuml
|
||||
!theme plain
|
||||
title Turnstone — MCP Architecture (Resources, Prompts, Tools)
|
||||
|
||||
skinparam participant {
|
||||
BackgroundColor<<mcp>> #E1BEE7
|
||||
BackgroundColor<<session>> #C8E6C9
|
||||
BackgroundColor<<storage>> #B3E5FC
|
||||
BackgroundColor<<server>> #FFE0B2
|
||||
BackgroundColor<<ui>> #E8EAF6
|
||||
}
|
||||
|
||||
participant "MCP Server\n(external)" as MCPSrv <<mcp>>
|
||||
participant "MCPClientManager\n(mcp_client.py)" as MCPMgr <<mcp>>
|
||||
participant "ChatSession\n(session.py)" as Session <<session>>
|
||||
participant "StorageBackend\n(governance)" as Storage <<storage>>
|
||||
participant "Server / Console\n(health + UI)" as UI <<server>>
|
||||
|
||||
== Startup: Connection & Discovery ==
|
||||
|
||||
MCPMgr -> MCPSrv : initialize (stdio or HTTP)
|
||||
MCPSrv --> MCPMgr : capabilities\n(tools, resources, prompts)
|
||||
|
||||
MCPMgr -> MCPSrv : tools/list
|
||||
MCPSrv --> MCPMgr : Tool[]
|
||||
|
||||
opt resources capability
|
||||
MCPMgr -> MCPSrv : resources/list
|
||||
MCPSrv --> MCPMgr : Resource[]
|
||||
MCPMgr -> MCPSrv : resources/templates/list
|
||||
MCPSrv --> MCPMgr : ResourceTemplate[]
|
||||
end
|
||||
|
||||
opt prompts capability
|
||||
MCPMgr -> MCPSrv : prompts/list
|
||||
MCPSrv --> MCPMgr : Prompt[]
|
||||
end
|
||||
|
||||
note over MCPMgr
|
||||
Per-server storage:
|
||||
_per_server_tools, _per_server_resources, _per_server_prompts
|
||||
Copy-on-write rebuild into _tools, _resources, _prompts
|
||||
Prefix: mcp__{server}__{name}
|
||||
end note
|
||||
|
||||
MCPMgr -> Session : notify tool listeners
|
||||
MCPMgr -> Session : notify resource listeners
|
||||
|
||||
== Governance Sync (on connect & refresh) ==
|
||||
|
||||
MCPMgr -> Storage : sync_prompts_to_storage()
|
||||
note right
|
||||
For each MCP prompt:
|
||||
- Manual template exists? → skip
|
||||
- MCP template exists? → update
|
||||
- New? → create (origin="mcp",
|
||||
readonly=True)
|
||||
Removed prompts → delete
|
||||
Protected by _sync_lock
|
||||
end note
|
||||
|
||||
== set_storage() from entry point ==
|
||||
|
||||
UI -> MCPMgr : set_storage(backend)
|
||||
note right
|
||||
If servers already connected,
|
||||
triggers immediate sync
|
||||
end note
|
||||
|
||||
== Runtime: Tool Execution ==
|
||||
|
||||
Session -> Session : _prepare_mcp_tool(func_name, args)
|
||||
note right
|
||||
approval_label = func_name
|
||||
(e.g. mcp__github__search)
|
||||
needs_approval = True
|
||||
end note
|
||||
Session -> MCPMgr : call_tool_sync(name, args)
|
||||
MCPMgr -> MCPSrv : tools/call
|
||||
MCPSrv --> MCPMgr : ToolResult
|
||||
MCPMgr --> Session : output (text)
|
||||
|
||||
== Runtime: Resource Read ==
|
||||
|
||||
Session -> Session : _prepare_read_resource(uri)
|
||||
note right
|
||||
approval_label = mcp_resource__{normalized_uri}
|
||||
URI normalized (.. resolved)
|
||||
needs_approval = True
|
||||
end note
|
||||
Session -> MCPMgr : read_resource_sync(uri)
|
||||
MCPMgr -> MCPSrv : resources/read
|
||||
MCPSrv --> MCPMgr : ReadResourceResult
|
||||
MCPMgr --> Session : content (text/blob)
|
||||
|
||||
== Runtime: Prompt Invocation ==
|
||||
|
||||
Session -> Session : _prepare_use_prompt(name, arguments)
|
||||
note right
|
||||
approval_label = mcp__srv__prompt
|
||||
Validated via is_mcp_prompt()
|
||||
needs_approval = True
|
||||
end note
|
||||
Session -> MCPMgr : get_prompt_sync(name, args)
|
||||
MCPMgr -> MCPSrv : prompts/get
|
||||
MCPSrv --> MCPMgr : GetPromptResult
|
||||
MCPMgr --> Session : messages [{role, content}]
|
||||
|
||||
== Three-Tier Refresh ==
|
||||
|
||||
group Push Notifications
|
||||
MCPSrv -> MCPMgr : ToolListChangedNotification
|
||||
MCPMgr -> MCPMgr : _refresh_server_tools()
|
||||
|
||||
MCPSrv -> MCPMgr : ResourceListChangedNotification
|
||||
MCPMgr -> MCPMgr : _refresh_server_resources()
|
||||
|
||||
MCPSrv -> MCPMgr : PromptListChangedNotification
|
||||
MCPMgr -> MCPMgr : _refresh_server_prompts()
|
||||
MCPMgr -> Storage : sync_prompts_to_storage()
|
||||
end
|
||||
|
||||
group Periodic Polling (default 4h)
|
||||
MCPMgr -> MCPMgr : _periodic_refresh()
|
||||
note right
|
||||
Only polls capabilities
|
||||
without push support.
|
||||
Staggered per-server.
|
||||
end note
|
||||
end
|
||||
|
||||
group Manual Refresh
|
||||
Session -> MCPMgr : refresh_sync()
|
||||
note right: /mcp refresh [server]
|
||||
end
|
||||
|
||||
== Policy Evaluation ==
|
||||
|
||||
note over Session
|
||||
Tool policies use fnmatch on approval_label:
|
||||
- mcp__github__* → allow (all GitHub tools/prompts)
|
||||
- mcp_resource__file:///docs/* → allow
|
||||
- mcp_resource__* → deny (block all resource reads)
|
||||
- mcp__untrusted__* → ask
|
||||
end note
|
||||
|
||||
== UI Visibility ==
|
||||
|
||||
UI -> MCPMgr : server_count, get_resources(), get_prompts()
|
||||
note over UI
|
||||
/health → mcp.servers, mcp.resources, mcp.prompts
|
||||
Server UI: magenta status badge
|
||||
Console: cluster status bar + node detail
|
||||
System message: <mcp-resources> + <mcp-prompts> catalogs
|
||||
end note
|
||||
|
||||
@enduml
|
||||
@@ -1,3 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:0ee0a9391bd19d92e9271bf6bd531e9c2e18baf8c5a11ead49b3c10db4d8939b
|
||||
size 329625
|
||||
oid sha256:c9daca81971ba7a8ed6736d23d5373c69435158fa6240b9880d14fc4759ab580
|
||||
size 329673
|
||||
|
||||
@@ -1,3 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:760c37e67736588dadee21d500419a48e9fc50f8bdc5667e686c580022bd40e2
|
||||
size 554869
|
||||
oid sha256:01fbb3338df6426cefc2811541a865f268673b4febf32f524c264d120bc068fa
|
||||
size 589546
|
||||
|
||||
@@ -1,3 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:a3ffd93ccb634f76560f1dd65242b89cd34443b37e355f25f29a1c63a22001be
|
||||
size 265259
|
||||
oid sha256:6dd3c923d1e1c49b5f91d8d342fb4b0d49a46d432460379ad146a9e3b075a05a
|
||||
size 277234
|
||||
|
||||
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:2c19dfae7606de8277d44d11226bdfba608d830382e92ebf0bec284b901fb807
|
||||
size 248194
|
||||
+12
-1
@@ -43,15 +43,26 @@ Admin-defined rules that control tool execution:
|
||||
- **Priority**: Higher priority evaluated first, first match wins
|
||||
- **Enforcement**: `evaluate_tool_policies_batch()` called in `WebUI.approve_tools()`
|
||||
before the `auto_approve` check
|
||||
- **MCP granular policies**: MCP resources and prompts are evaluated using their
|
||||
`approval_label` for fine-grained control:
|
||||
- Resource reads: `mcp_resource__{uri}` (e.g., `mcp_resource__file:///docs/*` to allow,
|
||||
`mcp_resource__*` to deny all)
|
||||
- Prompt invocations: `mcp__{server}__{prompt}` (e.g., `mcp__trusted__*` to allow,
|
||||
`mcp__*` to require approval for all)
|
||||
- Built-in tools continue to use `func_name` for backward compatibility
|
||||
|
||||
### Prompt Templates
|
||||
|
||||
Reusable system message templates with variable substitution:
|
||||
|
||||
- **Variables**: `{{variable_name}}` placeholders in content
|
||||
- **Categories**: general, engineering, support, custom
|
||||
- **Categories**: general, engineering, support, custom, mcp
|
||||
- **Default flag**: `is_default=true` templates intended for new workstreams
|
||||
- **Storage**: `prompt_templates` table with JSON `variables` array
|
||||
- **MCP sync**: MCP server prompts are auto-synced into prompt_templates with
|
||||
`origin="mcp"`, `mcp_server` set, and `readonly=True`. Manual templates take
|
||||
precedence on name collision. Admin UI shows origin badge and disables
|
||||
edit/delete for MCP-sourced templates. See `docs/tools.md` MCP Prompts section
|
||||
|
||||
### Usage Tracking
|
||||
|
||||
|
||||
+149
-6
@@ -1,6 +1,6 @@
|
||||
# Tools Reference
|
||||
|
||||
turnstone exposes 16 built-in tools plus any number of external MCP tools to the
|
||||
turnstone exposes 18 built-in tools plus any number of external MCP tools to the
|
||||
LLM via the OpenAI function-calling interface. Built-in tools are defined as JSON
|
||||
files under `turnstone/tools/` and loaded at startup by `turnstone/core/tools.py`.
|
||||
MCP tools are discovered from configured MCP servers at startup by
|
||||
@@ -46,12 +46,12 @@ schema plus turnstone-specific metadata keys:
|
||||
|
||||
| Name | Description |
|
||||
|---------------------|-------------|
|
||||
| `TOOLS` | All 16 tool definitions (sent to the model). |
|
||||
| `TOOLS` | All 18 tool definitions (sent to the model). |
|
||||
| `AGENT_TOOLS` | Tools with `agent: true` -- available to plan sub-agents. Read-only tools. |
|
||||
| `TASK_AGENT_TOOLS` | Tools with `task_agent: true` -- available to task sub-agents. Includes write operations. |
|
||||
| `AGENT_AUTO_TOOLS` | Set of tool names with `auto_approve: true` -- no user confirmation needed. |
|
||||
| `TASK_AUTO_TOOLS` | Same as `AGENT_AUTO_TOOLS` (identical filter). |
|
||||
| `BUILTIN_TOOL_NAMES`| Frozenset of all 16 built-in tool names. Used by tool search to distinguish always-on tools from deferrable MCP tools. |
|
||||
| `BUILTIN_TOOL_NAMES`| Frozenset of all 18 built-in tool names. Used by tool search to distinguish always-on tools from deferrable MCP tools. |
|
||||
| `PRIMARY_KEY_MAP` | Dict mapping tool name to its `primary_key` parameter name. |
|
||||
|
||||
---
|
||||
@@ -69,7 +69,7 @@ Tool execution follows a three-phase pipeline inside `ChatSession._execute_tools
|
||||
- Parses the JSON arguments (with fallback for malformed JSON).
|
||||
- If JSON parsing fails entirely, uses `PRIMARY_KEY_MAP` to map a bare string
|
||||
to the correct parameter.
|
||||
- Dispatches to the matching `_prepare_{func_name}()` handler. There are 15
|
||||
- Dispatches to the matching `_prepare_{func_name}()` handler. There are 18
|
||||
built-in tools plus `tool_search` (synthetic, client-side BM25 fallback) and
|
||||
the generic `_prepare_mcp_tool()` handler for MCP tools.
|
||||
- Validates arguments and builds a preview dict containing:
|
||||
@@ -168,6 +168,8 @@ Every tool defines a `primary_key`. The mapping is:
|
||||
| `recall` | `query` |
|
||||
| `forget` | `key` |
|
||||
| `notify` | `message` |
|
||||
| `read_resource` | `uri` |
|
||||
| `use_prompt` | `name` |
|
||||
|
||||
---
|
||||
|
||||
@@ -517,6 +519,8 @@ data.get("mergedAt") is not None
|
||||
| `forget` | Memory | Yes | No | No | `key` |
|
||||
| `notify` | Notify | Yes | Yes | Yes | `message` |
|
||||
| `watch` | Monitor | No (create) | No | No | `command` |
|
||||
| `read_resource`| MCP | No | Yes | Yes | `uri` |
|
||||
| `use_prompt` | MCP | No | Yes | Yes | `name` |
|
||||
| `tool_search`| Search | Yes | No | No | `query` |
|
||||
|
||||
---
|
||||
@@ -569,7 +573,7 @@ CLI flags override the config file:
|
||||
search stays off and all tools are sent to the model directly.
|
||||
|
||||
2. **Partitioning**: When active, tools are split into two sets:
|
||||
- **Always-on** -- the 15 built-in tools (members of `BUILTIN_TOOL_NAMES`).
|
||||
- **Always-on** -- the 18 built-in tools (members of `BUILTIN_TOOL_NAMES`).
|
||||
These are always visible to the model.
|
||||
- **Deferred** -- all MCP tools. These are not sent in the tool list unless
|
||||
the model searches for them.
|
||||
@@ -593,6 +597,8 @@ where the model can interactively search for tools it needs.
|
||||
|
||||
## MCP Tools (External)
|
||||
|
||||
> See also: [MCP Architecture diagram](diagrams/png/20-mcp-architecture.png)
|
||||
|
||||
Turnstone supports the [Model Context Protocol](https://modelcontextprotocol.io/)
|
||||
(MCP) for connecting external tool servers — GitHub, databases, filesystems, or any
|
||||
MCP-compatible service.
|
||||
@@ -610,7 +616,7 @@ MCP-compatible service.
|
||||
3. **Schema conversion**: Each MCP tool's `inputSchema` is converted to OpenAI
|
||||
function-calling format. The tool name is prefixed: `mcp__{server}__{tool}`.
|
||||
|
||||
4. **Merging**: MCP tools are appended after the 15 built-in tools via
|
||||
4. **Merging**: MCP tools are appended after the 18 built-in tools via
|
||||
`merge_mcp_tools()`. Built-in tools appear first, giving them natural LLM priority.
|
||||
When dynamic tool search is active, MCP tools are deferred rather than directly
|
||||
visible -- the model discovers them via search as needed (see
|
||||
@@ -729,3 +735,140 @@ MCP refresh complete:
|
||||
MCP refresh complete:
|
||||
github: no changes
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## MCP Resources
|
||||
|
||||
MCP servers can expose **resources** -- named data items (files, database rows,
|
||||
API responses) addressable by URI. turnstone discovers resources at startup and
|
||||
makes them available to the model via the `read_resource` built-in tool.
|
||||
|
||||
### Discovery
|
||||
|
||||
During the MCP `initialize` handshake, `MCPClientManager` checks each server's
|
||||
capabilities for the `resources` capability. For servers that declare it:
|
||||
|
||||
1. `list_resources` fetches static resources (fixed URIs).
|
||||
2. `list_resource_templates` fetches URI templates (parameterized patterns like
|
||||
`db://tables/{table}/rows/{id}`).
|
||||
|
||||
Both are stored as `{uri, name, description, mimeType, server}` dicts and
|
||||
merged into a unified catalog.
|
||||
|
||||
### Resource catalog in system message
|
||||
|
||||
The first 50 resources are injected into the system message as an XML-delimited
|
||||
block so the model knows what URIs are available:
|
||||
|
||||
```xml
|
||||
<mcp-resources>
|
||||
file:///project/README.md Project readme
|
||||
db://users/schema User table schema
|
||||
</mcp-resources>
|
||||
Use read_resource(uri='...') to access the resources listed above.
|
||||
```
|
||||
|
||||
### read_resource tool
|
||||
|
||||
| Parameter | Type | Required | Description |
|
||||
|-----------|--------|----------|-------------|
|
||||
| `uri` | string | yes | The resource URI to read. |
|
||||
|
||||
- **What it does**: Reads the resource from its MCP server via `MCPClientManager.read_resource_sync()`. Returns text content for text resources or base64-encoded data for binary resources. Output is truncated by the standard tool output limiter.
|
||||
- **Auto-approve**: No -- requires user confirmation (reads external data).
|
||||
- **Agent availability**: `agent` and `task_agent`.
|
||||
|
||||
### Capability guards
|
||||
|
||||
The `read_resource` tool schema is always loaded (it is a built-in JSON schema),
|
||||
but resource discovery only runs for servers that declare the `resources`
|
||||
capability. Servers without the capability contribute zero resources to the
|
||||
catalog.
|
||||
|
||||
### Refresh
|
||||
|
||||
Resource lists stay current through the same three-tier mechanism as tool lists:
|
||||
|
||||
1. **Push** -- Servers declaring `resources.listChanged: true` send
|
||||
`notifications/resources/list_changed`, triggering an immediate refresh.
|
||||
2. **Periodic** -- Servers without push are polled on the configured refresh
|
||||
interval (default 4 hours, same timer as tools).
|
||||
3. **Manual** -- `/mcp refresh` re-fetches resources alongside tools.
|
||||
|
||||
---
|
||||
|
||||
## MCP Prompts
|
||||
|
||||
MCP servers can also expose **prompts** -- reusable message templates with
|
||||
optional arguments. turnstone discovers prompts at startup for servers that
|
||||
declare the `prompts` capability.
|
||||
|
||||
### Discovery
|
||||
|
||||
Prompt discovery mirrors resource discovery: `list_prompts` is called during
|
||||
the `initialize` handshake. Each prompt is stored with its prefixed name
|
||||
(`mcp__{server}__{prompt}`), description, and argument schema.
|
||||
|
||||
### use_prompt tool
|
||||
|
||||
| Parameter | Type | Required | Description |
|
||||
|-------------|--------|----------|-------------|
|
||||
| `name` | string | yes | The prompt name (e.g. `mcp__server__prompt_name`). |
|
||||
| `arguments` | object | no | Key-value argument pairs for the prompt. Values must be strings. |
|
||||
|
||||
- **What it does**: Invokes an MCP prompt template by name via `MCPClientManager.get_prompt_sync()`, expanding it into messages. Returns the expanded prompt content formatted as `[role]: content` blocks joined with blank lines. The prompt catalog is listed in the system message so the model knows which prompts are available. Output is truncated by the standard tool output limiter.
|
||||
- **Auto-approve**: No -- requires user confirmation (invokes external prompt servers).
|
||||
- **Agent availability**: `agent` and `task_agent`.
|
||||
|
||||
### Invocation
|
||||
|
||||
`MCPClientManager.get_prompt_sync()` calls the server's `get_prompt` method
|
||||
with the provided arguments and returns the expanded messages. The `use_prompt`
|
||||
built-in tool exposes this to the model as a function call.
|
||||
|
||||
### Governance Sync
|
||||
|
||||
Discovered MCP prompts are automatically synced into the `prompt_templates`
|
||||
governance table as first-class governed templates:
|
||||
|
||||
- **Origin tracking**: MCP-sourced templates have `origin="mcp"` and
|
||||
`mcp_server` set to the server name. Manual templates have
|
||||
`origin="manual"`.
|
||||
- **Read-only**: MCP-sourced templates are `readonly=True`. The admin API
|
||||
returns 403 on update/delete attempts. The admin UI disables edit/delete
|
||||
buttons and shows an origin badge.
|
||||
- **Precedence**: If a manual template and MCP prompt share the same name,
|
||||
the manual template wins and the MCP prompt is skipped (with a log
|
||||
warning).
|
||||
- **Lifecycle**: Templates are created on connect, updated on prompt list
|
||||
refresh, and removed when the MCP server no longer exposes the prompt.
|
||||
The sync runs automatically on connect, on `PromptListChangedNotification`,
|
||||
and on manual `/mcp refresh`.
|
||||
- **Schema**: Migration 009 adds `origin`, `mcp_server`, and `readonly`
|
||||
columns to the `prompt_templates` table.
|
||||
|
||||
The `use_prompt` tool allows the model to invoke any discovered MCP prompt at
|
||||
runtime. A catalog of up to 30 prompts is injected into the system message
|
||||
inside `<mcp-prompts>` XML tags so the model can discover available prompts.
|
||||
|
||||
---
|
||||
|
||||
## MCP UI Visibility
|
||||
|
||||
MCP server, resource, and prompt counts are surfaced across the UI:
|
||||
|
||||
- **Server `/health` endpoint**: Returns `mcp.servers`, `mcp.resources`,
|
||||
`mcp.prompts` when MCP is configured
|
||||
- **Server UI**: Magenta status badge in the header showing server count,
|
||||
with resource/prompt counts in tooltip
|
||||
- **Console cluster status bar**: MCP metrics (servers/resources/prompts)
|
||||
with magenta LED dot indicator, shown after a divider from workstream
|
||||
metrics
|
||||
- **Console node detail**: Per-node MCP summary showing server, resource,
|
||||
and prompt counts
|
||||
- **Console collector**: Aggregates MCP counts across all nodes in the
|
||||
cluster overview
|
||||
|
||||
MCP indicators use the `--magenta` design token for consistent theming
|
||||
across light and dark modes.
|
||||
|
||||
@@ -1177,12 +1177,44 @@
|
||||
}
|
||||
],
|
||||
"default": null
|
||||
},
|
||||
"mcp": {
|
||||
"anyOf": [
|
||||
{
|
||||
"$ref": "#/components/schemas/McpStatus"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
],
|
||||
"default": null
|
||||
}
|
||||
},
|
||||
"required": ["status"],
|
||||
"title": "HealthResponse",
|
||||
"type": "object"
|
||||
},
|
||||
"McpStatus": {
|
||||
"properties": {
|
||||
"servers": {
|
||||
"default": 0,
|
||||
"title": "Servers",
|
||||
"type": "integer"
|
||||
},
|
||||
"resources": {
|
||||
"default": 0,
|
||||
"title": "Resources",
|
||||
"type": "integer"
|
||||
},
|
||||
"prompts": {
|
||||
"default": 0,
|
||||
"title": "Prompts",
|
||||
"type": "integer"
|
||||
}
|
||||
},
|
||||
"title": "McpStatus",
|
||||
"type": "object"
|
||||
},
|
||||
"BackendStatus": {
|
||||
"properties": {
|
||||
"status": {
|
||||
|
||||
@@ -91,6 +91,7 @@ export type {
|
||||
SavedWorkstreamInfo,
|
||||
ListSavedWorkstreamsResponse,
|
||||
BackendStatus,
|
||||
McpStatus,
|
||||
WorkstreamCounts,
|
||||
HealthResponse,
|
||||
AuthLoginRequest,
|
||||
|
||||
@@ -159,6 +159,12 @@ export interface WorkstreamCounts {
|
||||
error?: number;
|
||||
}
|
||||
|
||||
export interface McpStatus {
|
||||
servers: number;
|
||||
resources: number;
|
||||
prompts: number;
|
||||
}
|
||||
|
||||
export interface HealthResponse {
|
||||
status: string;
|
||||
version?: string;
|
||||
@@ -166,6 +172,7 @@ export interface HealthResponse {
|
||||
model?: string;
|
||||
workstreams?: WorkstreamCounts;
|
||||
backend?: BackendStatus | null;
|
||||
mcp?: McpStatus | null;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
@@ -452,6 +459,9 @@ export interface PromptTemplateInfo {
|
||||
created_by: string;
|
||||
created: string;
|
||||
updated: string;
|
||||
origin: string;
|
||||
mcp_server: string;
|
||||
readonly: boolean;
|
||||
}
|
||||
|
||||
export interface CreateTemplateOptions {
|
||||
|
||||
@@ -375,6 +375,66 @@ class TestPromptTemplateCRUD:
|
||||
assert t2["is_default"] is False
|
||||
assert isinstance(t2["is_default"], bool)
|
||||
|
||||
def test_create_with_mcp_origin(self, db):
|
||||
db.create_prompt_template(
|
||||
"t1",
|
||||
"mcp__srv__prompt",
|
||||
"mcp",
|
||||
"content",
|
||||
variables="[]",
|
||||
is_default=False,
|
||||
org_id="",
|
||||
created_by="",
|
||||
origin="mcp",
|
||||
mcp_server="srv",
|
||||
readonly=True,
|
||||
)
|
||||
tpl = db.get_prompt_template("t1")
|
||||
assert tpl is not None
|
||||
assert tpl["origin"] == "mcp"
|
||||
assert tpl["mcp_server"] == "srv"
|
||||
assert tpl["readonly"] is True
|
||||
assert isinstance(tpl["readonly"], bool)
|
||||
|
||||
def test_default_origin_values(self, db):
|
||||
db.create_prompt_template("t1", "basic", "general", "Hello")
|
||||
tpl = db.get_prompt_template("t1")
|
||||
assert tpl is not None
|
||||
assert tpl["origin"] == "manual"
|
||||
assert tpl["mcp_server"] == ""
|
||||
assert tpl["readonly"] is False
|
||||
|
||||
def test_get_prompt_template_by_name(self, db):
|
||||
db.create_prompt_template("t1", "greeting", "general", "Hello!")
|
||||
tpl = db.get_prompt_template_by_name("greeting")
|
||||
assert tpl is not None
|
||||
assert tpl["template_id"] == "t1"
|
||||
assert tpl["name"] == "greeting"
|
||||
|
||||
def test_get_prompt_template_by_name_nonexistent(self, db):
|
||||
assert db.get_prompt_template_by_name("nope") is None
|
||||
|
||||
def test_list_prompt_templates_by_origin(self, db):
|
||||
db.create_prompt_template("t1", "manual_one", "general", "A", origin="manual")
|
||||
db.create_prompt_template("t2", "mcp_one", "mcp", "B", origin="mcp", mcp_server="srv1")
|
||||
db.create_prompt_template("t3", "mcp_two", "mcp", "C", origin="mcp", mcp_server="srv2")
|
||||
result = db.list_prompt_templates_by_origin("mcp")
|
||||
assert len(result) == 2
|
||||
names = [r["name"] for r in result]
|
||||
assert "mcp_one" in names
|
||||
assert "mcp_two" in names
|
||||
|
||||
def test_delete_prompt_templates_by_server(self, db):
|
||||
db.create_prompt_template("t1", "a", "mcp", "A", origin="mcp", mcp_server="srv1")
|
||||
db.create_prompt_template("t2", "b", "mcp", "B", origin="mcp", mcp_server="srv1")
|
||||
db.create_prompt_template("t3", "c", "mcp", "C", origin="mcp", mcp_server="srv2")
|
||||
deleted = db.delete_prompt_templates_by_server("srv1")
|
||||
assert deleted == 2
|
||||
# Only srv2 template remains.
|
||||
remaining = db.list_prompt_templates()
|
||||
assert len(remaining) == 1
|
||||
assert remaining[0]["mcp_server"] == "srv2"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Usage Events
|
||||
|
||||
+535
-1
@@ -51,6 +51,83 @@ def _fake_openai_tool(name: str = "mcp__test__search") -> dict[str, Any]:
|
||||
}
|
||||
|
||||
|
||||
def _fake_mcp_resource(
|
||||
uri: str = "file:///README.md",
|
||||
name: str = "readme",
|
||||
description: str = "Project readme",
|
||||
mime_type: str = "text/plain",
|
||||
) -> MagicMock:
|
||||
"""Create a mock MCP Resource object matching the SDK's Resource type."""
|
||||
res = MagicMock()
|
||||
res.uri = uri
|
||||
res.name = name
|
||||
res.description = description
|
||||
res.mimeType = mime_type
|
||||
return res
|
||||
|
||||
|
||||
def _fake_resource_dict(
|
||||
uri: str = "file:///README.md",
|
||||
name: str = "readme",
|
||||
description: str = "Project readme",
|
||||
mime_type: str = "text/plain",
|
||||
server: str = "test",
|
||||
) -> dict[str, Any]:
|
||||
"""Create a fake resource dict as stored in per-server state."""
|
||||
return {
|
||||
"uri": uri,
|
||||
"name": name,
|
||||
"description": description,
|
||||
"mimeType": mime_type,
|
||||
"server": server,
|
||||
}
|
||||
|
||||
|
||||
def _fake_mcp_prompt(
|
||||
name: str = "code_review",
|
||||
description: str = "Generate a code review",
|
||||
arguments: list[dict[str, Any]] | None = None,
|
||||
) -> MagicMock:
|
||||
"""Create a mock MCP Prompt object matching the SDK's Prompt type."""
|
||||
prompt = MagicMock()
|
||||
prompt.name = name
|
||||
prompt.description = description
|
||||
if arguments is None:
|
||||
arg = MagicMock()
|
||||
arg.name = "language"
|
||||
arg.description = "Programming language"
|
||||
arg.required = True
|
||||
prompt.arguments = [arg]
|
||||
else:
|
||||
mock_args = []
|
||||
for a in arguments:
|
||||
arg = MagicMock()
|
||||
arg.name = a["name"]
|
||||
arg.description = a.get("description", "")
|
||||
arg.required = a.get("required", False)
|
||||
mock_args.append(arg)
|
||||
prompt.arguments = mock_args
|
||||
return prompt
|
||||
|
||||
|
||||
def _fake_prompt_dict(
|
||||
name: str = "mcp__test__code_review",
|
||||
original_name: str = "code_review",
|
||||
server: str = "test",
|
||||
description: str = "Generate a code review",
|
||||
) -> dict[str, Any]:
|
||||
"""Create a fake prompt dict as stored in per-server state."""
|
||||
return {
|
||||
"name": name,
|
||||
"original_name": original_name,
|
||||
"server": server,
|
||||
"description": description,
|
||||
"arguments": [
|
||||
{"name": "language", "description": "Programming language", "required": True}
|
||||
],
|
||||
}
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Schema conversion
|
||||
# ---------------------------------------------------------------------------
|
||||
@@ -453,6 +530,23 @@ class TestRebuildTools:
|
||||
|
||||
|
||||
class TestRefreshServer:
|
||||
@staticmethod
|
||||
def _add_empty_resource_prompt_mocks(
|
||||
mgr: MCPClientManager, server_name: str, mock_session: MagicMock
|
||||
) -> None:
|
||||
"""Add empty list_resources/list_prompts mocks so _refresh_server works."""
|
||||
mgr._supports_resources[server_name] = True
|
||||
mgr._supports_prompts[server_name] = True
|
||||
empty_res = MagicMock()
|
||||
empty_res.resources = []
|
||||
mock_session.list_resources = AsyncMock(return_value=empty_res)
|
||||
empty_tmpl = MagicMock()
|
||||
empty_tmpl.resourceTemplates = []
|
||||
mock_session.list_resource_templates = AsyncMock(return_value=empty_tmpl)
|
||||
empty_prompts = MagicMock()
|
||||
empty_prompts.prompts = []
|
||||
mock_session.list_prompts = AsyncMock(return_value=empty_prompts)
|
||||
|
||||
def test_refresh_detects_added_tools(self):
|
||||
async def _run() -> None:
|
||||
mgr = MCPClientManager({})
|
||||
@@ -463,6 +557,7 @@ class TestRefreshServer:
|
||||
_fake_mcp_tool("create"), # new tool
|
||||
]
|
||||
mock_session.list_tools = AsyncMock(return_value=mock_result)
|
||||
self._add_empty_resource_prompt_mocks(mgr, "github", mock_session)
|
||||
mgr._sessions["github"] = mock_session
|
||||
mgr._per_server_tools["github"] = [_fake_openai_tool("mcp__github__search")]
|
||||
mgr._rebuild_tools()
|
||||
@@ -481,6 +576,7 @@ class TestRefreshServer:
|
||||
mock_result = MagicMock()
|
||||
mock_result.tools = [] # all tools removed
|
||||
mock_session.list_tools = AsyncMock(return_value=mock_result)
|
||||
self._add_empty_resource_prompt_mocks(mgr, "github", mock_session)
|
||||
mgr._sessions["github"] = mock_session
|
||||
mgr._per_server_tools["github"] = [_fake_openai_tool("mcp__github__search")]
|
||||
mgr._rebuild_tools()
|
||||
@@ -499,6 +595,7 @@ class TestRefreshServer:
|
||||
mock_result = MagicMock()
|
||||
mock_result.tools = [_fake_mcp_tool("search")]
|
||||
mock_session.list_tools = AsyncMock(return_value=mock_result)
|
||||
self._add_empty_resource_prompt_mocks(mgr, "github", mock_session)
|
||||
mgr._sessions["github"] = mock_session
|
||||
mgr._per_server_tools["github"] = [_fake_openai_tool("mcp__github__search")]
|
||||
mgr._rebuild_tools()
|
||||
@@ -513,7 +610,7 @@ class TestRefreshServer:
|
||||
async def _run() -> None:
|
||||
mgr = MCPClientManager({})
|
||||
with pytest.raises(RuntimeError, match="not connected"):
|
||||
await mgr._refresh_server("ghost")
|
||||
await mgr._refresh_server_tools("ghost")
|
||||
|
||||
asyncio.run(_run())
|
||||
|
||||
@@ -709,3 +806,440 @@ class TestSessionRefresh:
|
||||
session.handle_command("/mcp refresh")
|
||||
session.ui.on_error.assert_called_once()
|
||||
assert "MCP refresh failed" in session.ui.on_error.call_args[0][0]
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# MCP Resources
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestMCPResources:
|
||||
def test_resource_discovery(self):
|
||||
"""Mock list_resources() returning 2 resources, verify get_resources()."""
|
||||
mgr = MCPClientManager({})
|
||||
mgr._per_server_resources = {
|
||||
"fs": [
|
||||
_fake_resource_dict("file:///a.txt", "a", "File A", "text/plain", "fs"),
|
||||
_fake_resource_dict("file:///b.txt", "b", "File B", "text/plain", "fs"),
|
||||
],
|
||||
}
|
||||
mgr._rebuild_resources()
|
||||
resources = mgr.get_resources()
|
||||
assert len(resources) == 2
|
||||
uris = {r["uri"] for r in resources}
|
||||
assert uris == {"file:///a.txt", "file:///b.txt"}
|
||||
assert all(r["server"] == "fs" for r in resources)
|
||||
|
||||
def test_rebuild_resources_copy_on_write(self):
|
||||
"""Verify mutation safety — get_resources() returns independent copy."""
|
||||
mgr = MCPClientManager({})
|
||||
mgr._per_server_resources = {
|
||||
"a": [_fake_resource_dict("file:///x", "x", "", "", "a")],
|
||||
}
|
||||
mgr._rebuild_resources()
|
||||
old_resources = mgr._resources
|
||||
old_map = mgr._resource_map
|
||||
mgr._per_server_resources["b"] = [_fake_resource_dict("file:///y", "y", "", "", "b")]
|
||||
mgr._rebuild_resources()
|
||||
assert mgr._resources is not old_resources
|
||||
assert mgr._resource_map is not old_map
|
||||
|
||||
def test_get_resources_returns_copy(self):
|
||||
mgr = MCPClientManager({})
|
||||
mgr._per_server_resources = {
|
||||
"a": [_fake_resource_dict("file:///x", "x", "", "", "a")],
|
||||
}
|
||||
mgr._rebuild_resources()
|
||||
resources = mgr.get_resources()
|
||||
assert len(resources) == 1
|
||||
resources.clear()
|
||||
assert len(mgr.get_resources()) == 1
|
||||
|
||||
def test_read_resource_sync(self):
|
||||
"""Mock session.read_resource(), verify text extraction."""
|
||||
mgr = MCPClientManager({})
|
||||
mgr._resource_map = {"file:///readme": ("fs", "file:///readme")}
|
||||
mock_session = MagicMock()
|
||||
mgr._sessions["fs"] = mock_session
|
||||
mgr._loop = asyncio.new_event_loop()
|
||||
|
||||
# Mock the read_resource result
|
||||
text_content = MagicMock(spec=["text"])
|
||||
text_content.text = "Hello, world!"
|
||||
mock_result = MagicMock()
|
||||
mock_result.contents = [text_content]
|
||||
mock_session.read_resource = AsyncMock(return_value=mock_result)
|
||||
|
||||
thread = None
|
||||
try:
|
||||
thread = __import__("threading").Thread(target=mgr._loop.run_forever, daemon=True)
|
||||
thread.start()
|
||||
output = mgr.read_resource_sync("file:///readme", timeout=5)
|
||||
assert output == "Hello, world!"
|
||||
mock_session.read_resource.assert_awaited_once_with("file:///readme")
|
||||
finally:
|
||||
mgr._loop.call_soon_threadsafe(mgr._loop.stop)
|
||||
if thread:
|
||||
thread.join(timeout=5)
|
||||
mgr._loop.close()
|
||||
|
||||
def test_read_resource_sync_blob(self):
|
||||
"""Verify base64 blob extraction."""
|
||||
mgr = MCPClientManager({})
|
||||
mgr._resource_map = {"file:///img.png": ("fs", "file:///img.png")}
|
||||
mock_session = MagicMock()
|
||||
mgr._sessions["fs"] = mock_session
|
||||
mgr._loop = asyncio.new_event_loop()
|
||||
|
||||
blob_content = MagicMock(spec=["blob"])
|
||||
blob_content.blob = "aGVsbG8="
|
||||
mock_result = MagicMock()
|
||||
mock_result.contents = [blob_content]
|
||||
mock_session.read_resource = AsyncMock(return_value=mock_result)
|
||||
|
||||
thread = None
|
||||
try:
|
||||
thread = __import__("threading").Thread(target=mgr._loop.run_forever, daemon=True)
|
||||
thread.start()
|
||||
output = mgr.read_resource_sync("file:///img.png", timeout=5)
|
||||
assert output == "aGVsbG8="
|
||||
finally:
|
||||
mgr._loop.call_soon_threadsafe(mgr._loop.stop)
|
||||
if thread:
|
||||
thread.join(timeout=5)
|
||||
mgr._loop.close()
|
||||
|
||||
def test_read_resource_sync_unknown_uri(self):
|
||||
mgr = MCPClientManager({})
|
||||
with pytest.raises(ValueError, match="Unknown MCP resource"):
|
||||
mgr.read_resource_sync("file:///nonexistent")
|
||||
|
||||
def test_read_resource_sync_disconnected(self):
|
||||
mgr = MCPClientManager({})
|
||||
mgr._resource_map = {"file:///x": ("dead", "file:///x")}
|
||||
with pytest.raises(RuntimeError, match="not connected"):
|
||||
mgr.read_resource_sync("file:///x")
|
||||
|
||||
def test_read_resource_sync_timeout(self):
|
||||
"""Verify timeout handling."""
|
||||
mgr = MCPClientManager({})
|
||||
mgr._resource_map = {"file:///x": ("fs", "file:///x")}
|
||||
mock_session = MagicMock()
|
||||
mgr._sessions["fs"] = mock_session
|
||||
mgr._loop = asyncio.new_event_loop()
|
||||
|
||||
async def _slow_read(_uri: str) -> None:
|
||||
await asyncio.sleep(10)
|
||||
|
||||
mock_session.read_resource = _slow_read
|
||||
|
||||
thread = None
|
||||
try:
|
||||
thread = __import__("threading").Thread(target=mgr._loop.run_forever, daemon=True)
|
||||
thread.start()
|
||||
with pytest.raises(TimeoutError):
|
||||
mgr.read_resource_sync("file:///x", timeout=1)
|
||||
finally:
|
||||
mgr._loop.call_soon_threadsafe(mgr._loop.stop)
|
||||
if thread:
|
||||
thread.join(timeout=5)
|
||||
mgr._loop.close()
|
||||
|
||||
def test_resource_listener_notification(self):
|
||||
"""Verify callback fires on rebuild."""
|
||||
mgr = MCPClientManager({})
|
||||
calls: list[int] = []
|
||||
mgr.add_resource_listener(lambda: calls.append(1))
|
||||
mgr._per_server_resources = {"a": [_fake_resource_dict()]}
|
||||
mgr._rebuild_resources()
|
||||
assert len(calls) == 1
|
||||
|
||||
def test_resource_listener_remove(self):
|
||||
mgr = MCPClientManager({})
|
||||
calls: list[int] = []
|
||||
cb = lambda: calls.append(1) # noqa: E731
|
||||
mgr.add_resource_listener(cb)
|
||||
mgr.remove_resource_listener(cb)
|
||||
mgr._rebuild_resources()
|
||||
assert calls == []
|
||||
|
||||
def test_resource_listener_error_does_not_propagate(self):
|
||||
mgr = MCPClientManager({})
|
||||
mgr.add_resource_listener(lambda: 1 / 0)
|
||||
mgr._rebuild_resources() # should not raise
|
||||
|
||||
def test_resource_refresh_on_notification(self):
|
||||
"""Mock notification, verify re-fetch of resources."""
|
||||
|
||||
async def _run() -> None:
|
||||
mgr = MCPClientManager({})
|
||||
mock_session = MagicMock()
|
||||
mgr._sessions["fs"] = mock_session
|
||||
mgr._supports_resources["fs"] = True
|
||||
|
||||
# Initial state
|
||||
mgr._per_server_resources["fs"] = [
|
||||
_fake_resource_dict("file:///old", server="fs"),
|
||||
]
|
||||
mgr._rebuild_resources()
|
||||
assert len(mgr.get_resources()) == 1
|
||||
|
||||
# Mock the re-fetch returning a new resource
|
||||
new_res = _fake_mcp_resource("file:///new", "new")
|
||||
mock_res_result = MagicMock()
|
||||
mock_res_result.resources = [new_res]
|
||||
mock_session.list_resources = AsyncMock(return_value=mock_res_result)
|
||||
mock_tmpl_result = MagicMock()
|
||||
mock_tmpl_result.resourceTemplates = []
|
||||
mock_session.list_resource_templates = AsyncMock(return_value=mock_tmpl_result)
|
||||
|
||||
await mgr._refresh_server_resources("fs")
|
||||
resources = mgr.get_resources()
|
||||
assert len(resources) == 1
|
||||
assert resources[0]["uri"] == "file:///new"
|
||||
|
||||
asyncio.run(_run())
|
||||
|
||||
def test_rebuild_resources_empty(self):
|
||||
mgr = MCPClientManager({})
|
||||
mgr._per_server_resources = {}
|
||||
mgr._rebuild_resources()
|
||||
assert mgr._resources == []
|
||||
assert mgr._resource_map == {}
|
||||
|
||||
def test_rebuild_resources_multi_server(self):
|
||||
mgr = MCPClientManager({})
|
||||
mgr._per_server_resources = {
|
||||
"fs": [_fake_resource_dict("file:///a", server="fs")],
|
||||
"db": [_fake_resource_dict("db://table", name="table", server="db")],
|
||||
}
|
||||
mgr._rebuild_resources()
|
||||
assert len(mgr._resources) == 2
|
||||
assert mgr._resource_map["file:///a"] == ("fs", "file:///a")
|
||||
assert mgr._resource_map["db://table"] == ("db", "db://table")
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# MCP Prompts
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestMCPPrompts:
|
||||
def test_prompt_discovery(self):
|
||||
"""Mock list_prompts(), verify get_prompts() with correct prefixed names."""
|
||||
mgr = MCPClientManager({})
|
||||
mgr._per_server_prompts = {
|
||||
"tmpl": [
|
||||
_fake_prompt_dict("mcp__tmpl__code_review", "code_review", "tmpl"),
|
||||
_fake_prompt_dict("mcp__tmpl__summarize", "summarize", "tmpl"),
|
||||
],
|
||||
}
|
||||
mgr._rebuild_prompts()
|
||||
prompts = mgr.get_prompts()
|
||||
assert len(prompts) == 2
|
||||
names = {p["name"] for p in prompts}
|
||||
assert names == {"mcp__tmpl__code_review", "mcp__tmpl__summarize"}
|
||||
# Verify map entries
|
||||
assert mgr._prompt_map["mcp__tmpl__code_review"] == ("tmpl", "code_review")
|
||||
assert mgr._prompt_map["mcp__tmpl__summarize"] == ("tmpl", "summarize")
|
||||
|
||||
def test_rebuild_prompts_copy_on_write(self):
|
||||
"""Verify mutation safety."""
|
||||
mgr = MCPClientManager({})
|
||||
mgr._per_server_prompts = {
|
||||
"a": [_fake_prompt_dict("mcp__a__p1", "p1", "a")],
|
||||
}
|
||||
mgr._rebuild_prompts()
|
||||
old_prompts = mgr._prompts
|
||||
old_map = mgr._prompt_map
|
||||
mgr._per_server_prompts["b"] = [_fake_prompt_dict("mcp__b__p2", "p2", "b")]
|
||||
mgr._rebuild_prompts()
|
||||
assert mgr._prompts is not old_prompts
|
||||
assert mgr._prompt_map is not old_map
|
||||
|
||||
def test_get_prompts_returns_copy(self):
|
||||
mgr = MCPClientManager({})
|
||||
mgr._per_server_prompts = {
|
||||
"a": [_fake_prompt_dict("mcp__a__p1", "p1", "a")],
|
||||
}
|
||||
mgr._rebuild_prompts()
|
||||
prompts = mgr.get_prompts()
|
||||
assert len(prompts) == 1
|
||||
prompts.clear()
|
||||
assert len(mgr.get_prompts()) == 1
|
||||
|
||||
def test_get_prompt_sync(self):
|
||||
"""Mock session.get_prompt(), verify message conversion."""
|
||||
mgr = MCPClientManager({})
|
||||
mgr._prompt_map = {"mcp__tmpl__review": ("tmpl", "review")}
|
||||
mock_session = MagicMock()
|
||||
mgr._sessions["tmpl"] = mock_session
|
||||
mgr._loop = asyncio.new_event_loop()
|
||||
|
||||
# Build mock PromptMessage
|
||||
msg1 = MagicMock()
|
||||
msg1.role = "user"
|
||||
msg1.content = MagicMock()
|
||||
msg1.content.text = "Review this code"
|
||||
msg2 = MagicMock()
|
||||
msg2.role = "assistant"
|
||||
msg2.content = MagicMock()
|
||||
msg2.content.text = "Looks good!"
|
||||
mock_result = MagicMock()
|
||||
mock_result.messages = [msg1, msg2]
|
||||
mock_session.get_prompt = AsyncMock(return_value=mock_result)
|
||||
|
||||
thread = None
|
||||
try:
|
||||
thread = __import__("threading").Thread(target=mgr._loop.run_forever, daemon=True)
|
||||
thread.start()
|
||||
messages = mgr.get_prompt_sync(
|
||||
"mcp__tmpl__review", arguments={"language": "python"}, timeout=5
|
||||
)
|
||||
assert len(messages) == 2
|
||||
assert messages[0] == {"role": "user", "content": "Review this code"}
|
||||
assert messages[1] == {"role": "assistant", "content": "Looks good!"}
|
||||
mock_session.get_prompt.assert_awaited_once_with(
|
||||
"review", arguments={"language": "python"}
|
||||
)
|
||||
finally:
|
||||
mgr._loop.call_soon_threadsafe(mgr._loop.stop)
|
||||
if thread:
|
||||
thread.join(timeout=5)
|
||||
mgr._loop.close()
|
||||
|
||||
def test_get_prompt_sync_unknown(self):
|
||||
mgr = MCPClientManager({})
|
||||
with pytest.raises(ValueError, match="Unknown MCP prompt"):
|
||||
mgr.get_prompt_sync("mcp__no__such")
|
||||
|
||||
def test_get_prompt_sync_disconnected(self):
|
||||
mgr = MCPClientManager({})
|
||||
mgr._prompt_map = {"mcp__dead__p": ("dead", "p")}
|
||||
with pytest.raises(RuntimeError, match="not connected"):
|
||||
mgr.get_prompt_sync("mcp__dead__p")
|
||||
|
||||
def test_get_prompt_sync_timeout(self):
|
||||
"""Verify timeout handling."""
|
||||
mgr = MCPClientManager({})
|
||||
mgr._prompt_map = {"mcp__tmpl__slow": ("tmpl", "slow")}
|
||||
mock_session = MagicMock()
|
||||
mgr._sessions["tmpl"] = mock_session
|
||||
mgr._loop = asyncio.new_event_loop()
|
||||
|
||||
async def _slow_prompt(_name: str, *, arguments: dict[str, str] | None = None) -> None:
|
||||
await asyncio.sleep(10)
|
||||
|
||||
mock_session.get_prompt = _slow_prompt
|
||||
|
||||
thread = None
|
||||
try:
|
||||
thread = __import__("threading").Thread(target=mgr._loop.run_forever, daemon=True)
|
||||
thread.start()
|
||||
with pytest.raises(TimeoutError):
|
||||
mgr.get_prompt_sync("mcp__tmpl__slow", timeout=1)
|
||||
finally:
|
||||
mgr._loop.call_soon_threadsafe(mgr._loop.stop)
|
||||
if thread:
|
||||
thread.join(timeout=5)
|
||||
mgr._loop.close()
|
||||
|
||||
def test_prompt_listener_notification(self):
|
||||
"""Verify callback fires on rebuild."""
|
||||
mgr = MCPClientManager({})
|
||||
calls: list[int] = []
|
||||
mgr.add_prompt_listener(lambda: calls.append(1))
|
||||
mgr._per_server_prompts = {"a": [_fake_prompt_dict()]}
|
||||
mgr._rebuild_prompts()
|
||||
assert len(calls) == 1
|
||||
|
||||
def test_prompt_listener_remove(self):
|
||||
mgr = MCPClientManager({})
|
||||
calls: list[int] = []
|
||||
cb = lambda: calls.append(1) # noqa: E731
|
||||
mgr.add_prompt_listener(cb)
|
||||
mgr.remove_prompt_listener(cb)
|
||||
mgr._rebuild_prompts()
|
||||
assert calls == []
|
||||
|
||||
def test_prompt_listener_error_does_not_propagate(self):
|
||||
mgr = MCPClientManager({})
|
||||
mgr.add_prompt_listener(lambda: 1 / 0)
|
||||
mgr._rebuild_prompts() # should not raise
|
||||
|
||||
def test_is_mcp_prompt(self):
|
||||
"""Verify name lookup."""
|
||||
mgr = MCPClientManager({})
|
||||
mgr._prompt_map["mcp__tmpl__review"] = ("tmpl", "review")
|
||||
assert mgr.is_mcp_prompt("mcp__tmpl__review") is True
|
||||
assert mgr.is_mcp_prompt("nonexistent") is False
|
||||
|
||||
def test_prompt_refresh_on_notification(self):
|
||||
"""Mock notification, verify re-fetch of prompts."""
|
||||
|
||||
async def _run() -> None:
|
||||
mgr = MCPClientManager({})
|
||||
mock_session = MagicMock()
|
||||
mgr._sessions["tmpl"] = mock_session
|
||||
mgr._supports_prompts["tmpl"] = True
|
||||
|
||||
# Initial state
|
||||
mgr._per_server_prompts["tmpl"] = [
|
||||
_fake_prompt_dict("mcp__tmpl__old", "old", "tmpl"),
|
||||
]
|
||||
mgr._rebuild_prompts()
|
||||
assert len(mgr.get_prompts()) == 1
|
||||
|
||||
# Mock re-fetch returning a new prompt
|
||||
new_prompt = _fake_mcp_prompt("new_prompt", "A new prompt")
|
||||
mock_prompt_result = MagicMock()
|
||||
mock_prompt_result.prompts = [new_prompt]
|
||||
mock_session.list_prompts = AsyncMock(return_value=mock_prompt_result)
|
||||
|
||||
await mgr._refresh_server_prompts("tmpl")
|
||||
prompts = mgr.get_prompts()
|
||||
assert len(prompts) == 1
|
||||
assert prompts[0]["name"] == "mcp__tmpl__new_prompt"
|
||||
assert prompts[0]["original_name"] == "new_prompt"
|
||||
|
||||
asyncio.run(_run())
|
||||
|
||||
def test_rebuild_prompts_empty(self):
|
||||
mgr = MCPClientManager({})
|
||||
mgr._per_server_prompts = {}
|
||||
mgr._rebuild_prompts()
|
||||
assert mgr._prompts == []
|
||||
assert mgr._prompt_map == {}
|
||||
|
||||
def test_rebuild_prompts_multi_server(self):
|
||||
mgr = MCPClientManager({})
|
||||
mgr._per_server_prompts = {
|
||||
"a": [_fake_prompt_dict("mcp__a__p1", "p1", "a")],
|
||||
"b": [_fake_prompt_dict("mcp__b__p2", "p2", "b")],
|
||||
}
|
||||
mgr._rebuild_prompts()
|
||||
assert len(mgr._prompts) == 2
|
||||
assert mgr._prompt_map["mcp__a__p1"] == ("a", "p1")
|
||||
assert mgr._prompt_map["mcp__b__p2"] == ("b", "p2")
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Shutdown cleans up new state
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestShutdownCleanup:
|
||||
def test_shutdown_clears_resources_and_prompts(self):
|
||||
mgr = MCPClientManager({})
|
||||
mgr._per_server_resources = {"a": [_fake_resource_dict()]}
|
||||
mgr._rebuild_resources()
|
||||
mgr._per_server_prompts = {"a": [_fake_prompt_dict()]}
|
||||
mgr._rebuild_prompts()
|
||||
assert mgr.get_resources() != []
|
||||
assert mgr.get_prompts() != []
|
||||
|
||||
mgr.shutdown()
|
||||
assert mgr.get_resources() == []
|
||||
assert mgr.get_prompts() == []
|
||||
assert mgr._resource_map == {}
|
||||
assert mgr._prompt_map == {}
|
||||
|
||||
@@ -0,0 +1,241 @@
|
||||
"""Tests for MCP prompt → governance template sync and readonly API guards."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from unittest.mock import MagicMock
|
||||
|
||||
import pytest
|
||||
|
||||
from turnstone.core.mcp_client import MCPClientManager
|
||||
|
||||
|
||||
@pytest.fixture()
|
||||
def mgr() -> MCPClientManager:
|
||||
"""Create an MCPClientManager with no real servers (no start())."""
|
||||
return MCPClientManager({})
|
||||
|
||||
|
||||
def _make_storage() -> MagicMock:
|
||||
"""Create a mock storage backend with prompt template methods."""
|
||||
storage = MagicMock()
|
||||
storage.get_prompt_template_by_name.return_value = None
|
||||
storage.list_prompt_templates_by_origin.return_value = []
|
||||
storage.create_prompt_template.return_value = None
|
||||
storage.update_prompt_template.return_value = True
|
||||
storage.delete_prompt_template.return_value = True
|
||||
return storage
|
||||
|
||||
|
||||
class TestSyncPromptsToStorage:
|
||||
def test_sync_no_storage(self, mgr: MCPClientManager) -> None:
|
||||
"""Without storage set, sync returns empty stats."""
|
||||
result = mgr.sync_prompts_to_storage()
|
||||
assert result == {"added": [], "removed": [], "skipped": []}
|
||||
|
||||
def test_sync_creates_mcp_templates(self, mgr: MCPClientManager) -> None:
|
||||
"""New MCP prompts are created as templates."""
|
||||
storage = _make_storage()
|
||||
mgr.set_storage(storage)
|
||||
|
||||
# Populate internal prompts list directly
|
||||
mgr._prompts = [
|
||||
{
|
||||
"name": "mcp__test__greeting",
|
||||
"original_name": "greeting",
|
||||
"server": "test",
|
||||
"description": "Say hello",
|
||||
"arguments": [
|
||||
{"name": "name", "description": "Who to greet", "required": True},
|
||||
],
|
||||
},
|
||||
]
|
||||
|
||||
result = mgr.sync_prompts_to_storage()
|
||||
|
||||
assert result["added"] == ["mcp__test__greeting"]
|
||||
assert result["removed"] == []
|
||||
assert result["skipped"] == []
|
||||
storage.create_prompt_template.assert_called_once()
|
||||
call_kwargs = storage.create_prompt_template.call_args
|
||||
assert call_kwargs[1]["name"] == "mcp__test__greeting"
|
||||
assert call_kwargs[1]["origin"] == "mcp"
|
||||
assert call_kwargs[1]["mcp_server"] == "test"
|
||||
assert call_kwargs[1]["readonly"] is True
|
||||
assert call_kwargs[1]["category"] == "mcp"
|
||||
assert '"name"' in call_kwargs[1]["variables"]
|
||||
|
||||
def test_sync_skips_manual_overrides(self, mgr: MCPClientManager) -> None:
|
||||
"""A manual template with the same name is not overwritten."""
|
||||
storage = _make_storage()
|
||||
storage.get_prompt_template_by_name.return_value = {
|
||||
"template_id": "existing-id",
|
||||
"name": "mcp__test__greeting",
|
||||
"origin": "manual",
|
||||
"readonly": False,
|
||||
}
|
||||
mgr.set_storage(storage)
|
||||
|
||||
mgr._prompts = [
|
||||
{
|
||||
"name": "mcp__test__greeting",
|
||||
"original_name": "greeting",
|
||||
"server": "test",
|
||||
"description": "Say hello",
|
||||
"arguments": [],
|
||||
},
|
||||
]
|
||||
|
||||
result = mgr.sync_prompts_to_storage()
|
||||
|
||||
assert result["skipped"] == ["mcp__test__greeting"]
|
||||
assert result["added"] == []
|
||||
storage.create_prompt_template.assert_not_called()
|
||||
storage.update_prompt_template.assert_not_called()
|
||||
|
||||
def test_sync_updates_existing_mcp_template(self, mgr: MCPClientManager) -> None:
|
||||
"""An existing MCP template gets its content/variables updated."""
|
||||
storage = _make_storage()
|
||||
storage.get_prompt_template_by_name.return_value = {
|
||||
"template_id": "existing-id",
|
||||
"name": "mcp__test__greeting",
|
||||
"origin": "mcp",
|
||||
"mcp_server": "test",
|
||||
"readonly": True,
|
||||
}
|
||||
mgr.set_storage(storage)
|
||||
|
||||
mgr._prompts = [
|
||||
{
|
||||
"name": "mcp__test__greeting",
|
||||
"original_name": "greeting",
|
||||
"server": "test",
|
||||
"description": "Updated description",
|
||||
"arguments": [
|
||||
{"name": "user", "description": "The user", "required": False},
|
||||
],
|
||||
},
|
||||
]
|
||||
|
||||
result = mgr.sync_prompts_to_storage()
|
||||
|
||||
assert result["added"] == []
|
||||
assert result["skipped"] == []
|
||||
storage.create_prompt_template.assert_not_called()
|
||||
storage.update_prompt_template.assert_called_once()
|
||||
call_args = storage.update_prompt_template.call_args
|
||||
assert call_args[0][0] == "existing-id"
|
||||
assert "Updated description" in call_args[1]["content"]
|
||||
assert "user" in call_args[1]["variables"]
|
||||
|
||||
def test_sync_removes_deleted_prompts(self, mgr: MCPClientManager) -> None:
|
||||
"""MCP templates in storage with no matching prompt are deleted."""
|
||||
storage = _make_storage()
|
||||
storage.list_prompt_templates_by_origin.return_value = [
|
||||
{
|
||||
"template_id": "old-id",
|
||||
"name": "mcp__test__old_prompt",
|
||||
"origin": "mcp",
|
||||
"mcp_server": "test",
|
||||
},
|
||||
]
|
||||
mgr.set_storage(storage)
|
||||
mgr._prompts = [] # No prompts at all
|
||||
|
||||
result = mgr.sync_prompts_to_storage()
|
||||
|
||||
assert result["removed"] == ["mcp__test__old_prompt"]
|
||||
storage.delete_prompt_template.assert_called_once_with("old-id")
|
||||
|
||||
|
||||
class TestSetStorageAutoSync:
|
||||
"""set_storage() triggers an immediate sync when servers are already connected."""
|
||||
|
||||
def test_set_storage_syncs_when_connected(self, mgr) -> None:
|
||||
storage = _make_storage()
|
||||
mgr._prompts = [
|
||||
{
|
||||
"name": "mcp__srv__p1",
|
||||
"original_name": "p1",
|
||||
"server": "srv",
|
||||
"description": "A prompt",
|
||||
"arguments": [],
|
||||
}
|
||||
]
|
||||
mgr._connected.set()
|
||||
|
||||
mgr.set_storage(storage)
|
||||
|
||||
# Should have called create_prompt_template for the discovered prompt
|
||||
storage.create_prompt_template.assert_called_once()
|
||||
call_kwargs = storage.create_prompt_template.call_args
|
||||
assert call_kwargs[1]["name"] == "mcp__srv__p1"
|
||||
assert call_kwargs[1]["origin"] == "mcp"
|
||||
|
||||
def test_set_storage_no_sync_when_not_connected(self, mgr) -> None:
|
||||
storage = _make_storage()
|
||||
mgr._prompts = [
|
||||
{
|
||||
"name": "mcp__srv__p1",
|
||||
"original_name": "p1",
|
||||
"server": "srv",
|
||||
"description": "A prompt",
|
||||
"arguments": [],
|
||||
}
|
||||
]
|
||||
# _connected is NOT set
|
||||
|
||||
mgr.set_storage(storage)
|
||||
|
||||
# Should not have synced
|
||||
storage.create_prompt_template.assert_not_called()
|
||||
|
||||
|
||||
class TestReadonlyAPIGuards:
|
||||
"""Test that the console server API guards reject edits to readonly templates."""
|
||||
|
||||
@pytest.fixture()
|
||||
def db(self, tmp_path):
|
||||
"""Create a fresh SQLite backend for each test."""
|
||||
from turnstone.core.storage._sqlite import SQLiteBackend
|
||||
|
||||
return SQLiteBackend(str(tmp_path / "test.db"))
|
||||
|
||||
def test_readonly_guard_update(self, db) -> None:
|
||||
"""Readonly templates cannot be updated via storage guard logic."""
|
||||
db.create_prompt_template(
|
||||
"t1",
|
||||
"mcp__srv__prompt",
|
||||
"mcp",
|
||||
"content",
|
||||
variables="[]",
|
||||
is_default=False,
|
||||
org_id="",
|
||||
created_by="",
|
||||
origin="mcp",
|
||||
mcp_server="srv",
|
||||
readonly=True,
|
||||
)
|
||||
tpl = db.get_prompt_template("t1")
|
||||
assert tpl is not None
|
||||
assert tpl["readonly"] is True
|
||||
# Simulate API guard check
|
||||
assert tpl.get("readonly") is True
|
||||
|
||||
def test_readonly_guard_delete(self, db) -> None:
|
||||
"""Readonly templates are flagged for API-level rejection."""
|
||||
db.create_prompt_template(
|
||||
"t1",
|
||||
"mcp__srv__prompt",
|
||||
"mcp",
|
||||
"content",
|
||||
variables="[]",
|
||||
is_default=False,
|
||||
org_id="",
|
||||
created_by="",
|
||||
origin="mcp",
|
||||
mcp_server="srv",
|
||||
readonly=True,
|
||||
)
|
||||
existing = db.get_prompt_template("t1")
|
||||
assert existing is not None
|
||||
assert existing.get("readonly") is True
|
||||
@@ -84,3 +84,84 @@ def test_first_match_wins(storage):
|
||||
storage.create_tool_policy("p1", "deny-bash", "bash*", "deny", 100)
|
||||
storage.create_tool_policy("p2", "allow-bash", "bash*", "allow", 50)
|
||||
assert evaluate_tool_policy(storage, "bash_exec") == "deny"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# MCP resource and prompt policy patterns
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_mcp_resource_wildcard_deny(storage):
|
||||
"""Deny all MCP resource reads via glob pattern."""
|
||||
storage.create_tool_policy("p1", "block-resources", "mcp_resource__*", "deny", 100)
|
||||
assert evaluate_tool_policy(storage, "mcp_resource__file:///secret.txt") == "deny"
|
||||
assert evaluate_tool_policy(storage, "mcp_resource__db://users") == "deny"
|
||||
assert evaluate_tool_policy(storage, "read_file") is None # unrelated tool
|
||||
|
||||
|
||||
def test_mcp_resource_per_server_pattern(storage):
|
||||
"""Allow resources from a specific server, deny others."""
|
||||
storage.create_tool_policy("p1", "block-all-resources", "mcp_resource__*", "deny", 50)
|
||||
storage.create_tool_policy("p2", "allow-docs", "mcp_resource__file:///docs/*", "allow", 100)
|
||||
assert evaluate_tool_policy(storage, "mcp_resource__file:///docs/readme.md") == "allow"
|
||||
assert evaluate_tool_policy(storage, "mcp_resource__file:///etc/passwd") == "deny"
|
||||
|
||||
|
||||
def test_mcp_prompt_wildcard_ask(storage):
|
||||
"""Require approval for all MCP prompt invocations."""
|
||||
storage.create_tool_policy("p1", "ask-prompts", "mcp__*", "ask", 100)
|
||||
assert evaluate_tool_policy(storage, "mcp__github__code_review") == "ask"
|
||||
assert evaluate_tool_policy(storage, "mcp__templates__greeting") == "ask"
|
||||
assert evaluate_tool_policy(storage, "bash") is None
|
||||
|
||||
|
||||
def test_mcp_prompt_per_server_allow(storage):
|
||||
"""Auto-approve prompts from a trusted server."""
|
||||
storage.create_tool_policy("p1", "ask-all-mcp", "mcp__*", "ask", 50)
|
||||
storage.create_tool_policy("p2", "allow-trusted", "mcp__trusted__*", "allow", 100)
|
||||
assert evaluate_tool_policy(storage, "mcp__trusted__greeting") == "allow"
|
||||
assert evaluate_tool_policy(storage, "mcp__untrusted__evil") == "ask"
|
||||
|
||||
|
||||
def test_mcp_batch_mixed(storage):
|
||||
"""Batch evaluation with mixed MCP and built-in tools."""
|
||||
storage.create_tool_policy("p1", "block-resources", "mcp_resource__*", "deny", 100)
|
||||
storage.create_tool_policy("p2", "allow-prompts", "mcp__trusted__*", "allow", 100)
|
||||
results = evaluate_tool_policies_batch(
|
||||
storage,
|
||||
["mcp_resource__file:///x", "mcp__trusted__greeting", "bash", "mcp__other__y"],
|
||||
)
|
||||
assert results["mcp_resource__file:///x"] == "deny"
|
||||
assert results["mcp__trusted__greeting"] == "allow"
|
||||
assert results["bash"] is None
|
||||
assert results["mcp__other__y"] is None
|
||||
|
||||
|
||||
def test_normalize_resource_uri_prevents_traversal():
|
||||
"""URI normalization resolves .. segments to prevent policy traversal bypass."""
|
||||
from turnstone.core.session import ChatSession
|
||||
|
||||
# Normal URI unchanged
|
||||
assert ChatSession._normalize_resource_uri("file:///docs/readme.md") == "file:///docs/readme.md"
|
||||
# Traversal resolved
|
||||
assert ChatSession._normalize_resource_uri("file:///docs/../etc/passwd") == "file:///etc/passwd"
|
||||
# Double traversal
|
||||
assert ChatSession._normalize_resource_uri("file:///a/b/../../c") == "file:///c"
|
||||
# Non-file scheme (netloc preserved, path normalized)
|
||||
assert ChatSession._normalize_resource_uri("db://host/tables/../secrets") == "db://host/secrets"
|
||||
# Percent-encoded traversal decoded before normalization
|
||||
assert (
|
||||
ChatSession._normalize_resource_uri("file:///docs/%2e%2e/etc/passwd")
|
||||
== "file:///etc/passwd"
|
||||
)
|
||||
# Mixed percent-encoded and literal traversal
|
||||
assert ChatSession._normalize_resource_uri("file:///a/%2e%2e/b/../c") == "file:///c"
|
||||
|
||||
|
||||
def test_mcp_tool_granular_policy(storage):
|
||||
"""MCP tool calls use their prefixed func_name for granular policy matching."""
|
||||
storage.create_tool_policy("p1", "ask-all-mcp", "mcp__*", "ask", 50)
|
||||
storage.create_tool_policy("p2", "allow-github", "mcp__github__*", "allow", 100)
|
||||
# MCP tools now use func_name as approval_label
|
||||
assert evaluate_tool_policy(storage, "mcp__github__search") == "allow"
|
||||
assert evaluate_tool_policy(storage, "mcp__untrusted__exec") == "ask"
|
||||
|
||||
@@ -72,16 +72,24 @@ class TestToolsMetadata:
|
||||
"""Validate the metadata extracted from JSON files."""
|
||||
|
||||
def test_tool_count(self):
|
||||
assert len(TOOLS) == 16
|
||||
assert len(TOOLS) == 18
|
||||
|
||||
def test_agent_tools_count(self):
|
||||
assert len(AGENT_TOOLS) == 7
|
||||
assert len(AGENT_TOOLS) == 9
|
||||
|
||||
def test_task_agent_tools_count(self):
|
||||
assert len(TASK_AGENT_TOOLS) == 10
|
||||
assert len(TASK_AGENT_TOOLS) == 12
|
||||
|
||||
def test_auto_approve_sets_match(self):
|
||||
expected = {"read_file", "search", "math", "man", "web_fetch", "web_search", "notify"}
|
||||
expected = {
|
||||
"read_file",
|
||||
"search",
|
||||
"math",
|
||||
"man",
|
||||
"web_fetch",
|
||||
"web_search",
|
||||
"notify",
|
||||
}
|
||||
assert expected == AGENT_AUTO_TOOLS
|
||||
assert expected == TASK_AUTO_TOOLS
|
||||
|
||||
@@ -103,6 +111,8 @@ class TestToolsMetadata:
|
||||
"forget": "key",
|
||||
"notify": "message",
|
||||
"watch": "command",
|
||||
"read_resource": "uri",
|
||||
"use_prompt": "name",
|
||||
}
|
||||
assert expected == PRIMARY_KEY_MAP
|
||||
|
||||
|
||||
@@ -286,6 +286,9 @@ class PromptTemplateInfo(BaseModel):
|
||||
is_default: bool
|
||||
org_id: str
|
||||
created_by: str
|
||||
origin: str = "manual"
|
||||
mcp_server: str = ""
|
||||
readonly: bool = False
|
||||
created: str
|
||||
updated: str
|
||||
|
||||
|
||||
@@ -143,6 +143,12 @@ class WorkstreamCounts(BaseModel):
|
||||
error: int = 0
|
||||
|
||||
|
||||
class McpStatus(BaseModel):
|
||||
servers: int = 0
|
||||
resources: int = 0
|
||||
prompts: int = 0
|
||||
|
||||
|
||||
class HealthResponse(BaseModel):
|
||||
status: str = Field(examples=["ok", "degraded"])
|
||||
version: str = ""
|
||||
@@ -150,3 +156,4 @@ class HealthResponse(BaseModel):
|
||||
model: str = ""
|
||||
workstreams: WorkstreamCounts = WorkstreamCounts()
|
||||
backend: BackendStatus | None = None
|
||||
mcp: McpStatus | None = None
|
||||
|
||||
@@ -1002,6 +1002,9 @@ def main() -> None:
|
||||
mcp_tools = mcp_client.get_tools()
|
||||
if mcp_tools:
|
||||
print(f"MCP tools: {len(mcp_tools)} from {mcp_client.server_count} server(s)")
|
||||
from turnstone.core.storage import get_storage as _cli_get_storage
|
||||
|
||||
mcp_client.set_storage(_cli_get_storage())
|
||||
print("Type /help for commands, /ws for workstreams, /exit or Ctrl+D to quit.\n")
|
||||
|
||||
# Prompt string -- use a short display name
|
||||
|
||||
@@ -320,6 +320,9 @@ class ClusterCollector:
|
||||
total_tokens = 0
|
||||
total_tool_calls = 0
|
||||
total_ws = 0
|
||||
mcp_servers = 0
|
||||
mcp_resources = 0
|
||||
mcp_prompts = 0
|
||||
versions: set[str] = set()
|
||||
with self._lock:
|
||||
for node in self._nodes.values():
|
||||
@@ -332,8 +335,12 @@ class ClusterCollector:
|
||||
ver = node.health.get("version", "")
|
||||
if ver:
|
||||
versions.add(ver)
|
||||
mcp = node.health.get("mcp", {})
|
||||
mcp_servers += mcp.get("servers", 0)
|
||||
mcp_resources += mcp.get("resources", 0)
|
||||
mcp_prompts += mcp.get("prompts", 0)
|
||||
node_count = len(self._nodes)
|
||||
return {
|
||||
result: dict[str, Any] = {
|
||||
"nodes": node_count,
|
||||
"workstreams": total_ws,
|
||||
"states": states,
|
||||
@@ -344,6 +351,11 @@ class ClusterCollector:
|
||||
"version_drift": len(versions) > 1,
|
||||
"versions": sorted(versions),
|
||||
}
|
||||
if mcp_servers:
|
||||
result["mcp_servers"] = mcp_servers
|
||||
result["mcp_resources"] = mcp_resources
|
||||
result["mcp_prompts"] = mcp_prompts
|
||||
return result
|
||||
|
||||
def get_version_info(self) -> dict[str, Any]:
|
||||
"""Return per-node version map and drift flag."""
|
||||
@@ -515,6 +527,9 @@ class ClusterCollector:
|
||||
total_tokens = 0
|
||||
total_tool_calls = 0
|
||||
total_ws = 0
|
||||
mcp_servers = 0
|
||||
mcp_resources = 0
|
||||
mcp_prompts = 0
|
||||
versions: set[str] = set()
|
||||
|
||||
for node in self._nodes.values():
|
||||
@@ -530,6 +545,10 @@ class ClusterCollector:
|
||||
ver = node.health.get("version", "")
|
||||
if ver:
|
||||
versions.add(ver)
|
||||
mcp = node.health.get("mcp", {})
|
||||
mcp_servers += mcp.get("servers", 0)
|
||||
mcp_resources += mcp.get("resources", 0)
|
||||
mcp_prompts += mcp.get("prompts", 0)
|
||||
|
||||
nodes_out.append(
|
||||
{
|
||||
@@ -546,19 +565,25 @@ class ClusterCollector:
|
||||
|
||||
node_count = len(self._nodes)
|
||||
|
||||
overview: dict[str, Any] = {
|
||||
"nodes": node_count,
|
||||
"workstreams": total_ws,
|
||||
"states": states,
|
||||
"aggregate": {
|
||||
"total_tokens": total_tokens,
|
||||
"total_tool_calls": total_tool_calls,
|
||||
},
|
||||
"version_drift": len(versions) > 1,
|
||||
"versions": sorted(versions),
|
||||
}
|
||||
if mcp_servers:
|
||||
overview["mcp_servers"] = mcp_servers
|
||||
overview["mcp_resources"] = mcp_resources
|
||||
overview["mcp_prompts"] = mcp_prompts
|
||||
|
||||
return {
|
||||
"nodes": nodes_out,
|
||||
"overview": {
|
||||
"nodes": node_count,
|
||||
"workstreams": total_ws,
|
||||
"states": states,
|
||||
"aggregate": {
|
||||
"total_tokens": total_tokens,
|
||||
"total_tool_calls": total_tool_calls,
|
||||
},
|
||||
"version_drift": len(versions) > 1,
|
||||
"versions": sorted(versions),
|
||||
},
|
||||
"overview": overview,
|
||||
"timestamp": time.time(),
|
||||
}
|
||||
|
||||
|
||||
@@ -2074,6 +2074,8 @@ async def admin_update_template(request: Request) -> JSONResponse:
|
||||
existing = storage.get_prompt_template(template_id)
|
||||
if existing is None:
|
||||
return JSONResponse({"error": "Template not found"}, status_code=404)
|
||||
if existing.get("readonly"):
|
||||
return JSONResponse({"error": "MCP-sourced templates are read-only"}, status_code=403)
|
||||
|
||||
body = await read_json_or_400(request)
|
||||
if isinstance(body, JSONResponse):
|
||||
@@ -2130,6 +2132,8 @@ async def admin_delete_template(request: Request) -> JSONResponse:
|
||||
existing = storage.get_prompt_template(template_id)
|
||||
if existing is None:
|
||||
return JSONResponse({"error": "Template not found"}, status_code=404)
|
||||
if existing.get("readonly"):
|
||||
return JSONResponse({"error": "MCP-sourced templates are read-only"}, status_code=403)
|
||||
|
||||
storage.delete_prompt_template(template_id)
|
||||
|
||||
|
||||
@@ -140,6 +140,9 @@ function recomputeOverview() {
|
||||
var totalTokens = 0,
|
||||
totalToolCalls = 0,
|
||||
totalWs = 0;
|
||||
var mcpServers = 0,
|
||||
mcpResources = 0,
|
||||
mcpPrompts = 0;
|
||||
var versions = {};
|
||||
Object.keys(clusterState.nodes).forEach(function (nid) {
|
||||
var node = clusterState.nodes[nid];
|
||||
@@ -154,6 +157,10 @@ function recomputeOverview() {
|
||||
totalTokens += aggTokens || nodeWsTokens;
|
||||
totalToolCalls += (node.aggregate || {}).total_tool_calls || 0;
|
||||
if (node.version) versions[node.version] = true;
|
||||
var mcp = (node.health || {}).mcp || {};
|
||||
mcpServers += mcp.servers || 0;
|
||||
mcpResources += mcp.resources || 0;
|
||||
mcpPrompts += mcp.prompts || 0;
|
||||
});
|
||||
var versionList = Object.keys(versions).sort();
|
||||
clusterState.overview = {
|
||||
@@ -167,6 +174,11 @@ function recomputeOverview() {
|
||||
version_drift: versionList.length > 1,
|
||||
versions: versionList,
|
||||
};
|
||||
if (mcpServers > 0) {
|
||||
clusterState.overview.mcp_servers = mcpServers;
|
||||
clusterState.overview.mcp_resources = mcpResources;
|
||||
clusterState.overview.mcp_prompts = mcpPrompts;
|
||||
}
|
||||
}
|
||||
|
||||
function buildNodeInfoFromSnapshot(node) {
|
||||
@@ -227,6 +239,23 @@ function renderFromState() {
|
||||
}).length;
|
||||
document.getElementById("node-ws-summary").textContent =
|
||||
active + " active \u00b7 " + wsList.length + " total";
|
||||
var mcpSumEl = document.getElementById("node-mcp-summary");
|
||||
if (mcpSumEl) {
|
||||
var mcpInfo = snapNode.health && snapNode.health.mcp;
|
||||
if (mcpInfo && mcpInfo.servers > 0) {
|
||||
mcpSumEl.textContent =
|
||||
mcpInfo.servers +
|
||||
" MCP server" +
|
||||
(mcpInfo.servers !== 1 ? "s" : "") +
|
||||
" \u00b7 " +
|
||||
mcpInfo.resources +
|
||||
" resources \u00b7 " +
|
||||
mcpInfo.prompts +
|
||||
" prompts";
|
||||
} else {
|
||||
mcpSumEl.textContent = "";
|
||||
}
|
||||
}
|
||||
renderWsTable(document.getElementById("node-ws-table"), wsList);
|
||||
}
|
||||
} else if (currentView === "filtered") {
|
||||
@@ -470,6 +499,43 @@ function renderStatusBar(overview) {
|
||||
verEl.appendChild(verLbl);
|
||||
metricsContainer.appendChild(verEl);
|
||||
}
|
||||
// MCP aggregate metrics
|
||||
if (overview.mcp_servers && overview.mcp_servers > 0) {
|
||||
var mcpDivider = document.createElement("span");
|
||||
mcpDivider.className = "csb-divider";
|
||||
mcpDivider.setAttribute("aria-hidden", "true");
|
||||
metricsContainer.appendChild(mcpDivider);
|
||||
var mcpTitles = {
|
||||
mcp: "MCP servers",
|
||||
rsrc: "MCP resources",
|
||||
pmpt: "MCP prompts",
|
||||
};
|
||||
var mcpMetrics = [
|
||||
{ value: overview.mcp_servers, label: "mcp" },
|
||||
{ value: overview.mcp_resources, label: "rsrc" },
|
||||
{ value: overview.mcp_prompts, label: "pmpt" },
|
||||
];
|
||||
mcpMetrics.forEach(function (m) {
|
||||
var el = document.createElement("span");
|
||||
el.className = "csb-metric";
|
||||
el.title = mcpTitles[m.label] || "";
|
||||
if (m.label === "mcp") {
|
||||
var dot = document.createElement("span");
|
||||
dot.className = "csb-mcp-dot";
|
||||
dot.setAttribute("aria-hidden", "true");
|
||||
el.appendChild(dot);
|
||||
}
|
||||
var valSpan = document.createElement("span");
|
||||
valSpan.className = "csb-metric-value";
|
||||
valSpan.textContent = formatCount(m.value);
|
||||
var labelSpan = document.createElement("span");
|
||||
labelSpan.className = "csb-metric-label";
|
||||
labelSpan.textContent = m.label;
|
||||
el.appendChild(valSpan);
|
||||
el.appendChild(labelSpan);
|
||||
metricsContainer.appendChild(el);
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
// --- Node Grouping ---
|
||||
|
||||
@@ -689,14 +689,23 @@ function _renderGovTemplates(items) {
|
||||
var defBadge = t.is_default
|
||||
? '<span class="scope-badge scope-approve">default</span>'
|
||||
: "";
|
||||
var originBadge =
|
||||
t.origin === "mcp"
|
||||
? ' <span class="scope-badge scope-deny">mcp:' +
|
||||
escapeHtml(t.mcp_server) +
|
||||
"</span>"
|
||||
: "";
|
||||
var catBadge =
|
||||
'<span class="scope-badge">' + escapeHtml(t.category) + "</span>";
|
||||
var editDisabled = t.readonly ? " disabled" : "";
|
||||
var deleteDisabled = t.readonly ? " disabled" : "";
|
||||
html +=
|
||||
'<div class="admin-row" role="listitem">' +
|
||||
'<span class="admin-col admin-col-tmname">' +
|
||||
escapeHtml(t.name) +
|
||||
" " +
|
||||
defBadge +
|
||||
originBadge +
|
||||
"</span>" +
|
||||
'<span class="admin-col admin-col-tmcat">' +
|
||||
catBadge +
|
||||
@@ -707,12 +716,16 @@ function _renderGovTemplates(items) {
|
||||
'<span class="admin-col admin-col-actions">' +
|
||||
'<button class="admin-btn-action" data-edit-tmpl="' +
|
||||
escapeHtml(t.template_id) +
|
||||
'">edit</button>' +
|
||||
'"' +
|
||||
editDisabled +
|
||||
">edit</button>" +
|
||||
'<button class="admin-btn-danger" data-delete-tmpl="' +
|
||||
escapeHtml(t.template_id) +
|
||||
'" data-tmpl-name="' +
|
||||
escapeHtml(t.name) +
|
||||
'">delete</button>' +
|
||||
'"' +
|
||||
deleteDisabled +
|
||||
">delete</button>" +
|
||||
"</span></div>";
|
||||
}
|
||||
el.innerHTML = html;
|
||||
|
||||
@@ -41,6 +41,7 @@
|
||||
<div class="dash-header">
|
||||
<span class="dash-header-title">WORKSTREAMS</span>
|
||||
<span class="dash-header-summary" id="node-ws-summary"></span>
|
||||
<span id="node-mcp-summary" aria-label="MCP status"></span>
|
||||
</div>
|
||||
<div class="dash-colheaders" aria-hidden="true">
|
||||
<span class="dash-col dash-col-state">STATE</span>
|
||||
|
||||
@@ -166,6 +166,17 @@
|
||||
opacity: 0.7;
|
||||
}
|
||||
|
||||
/* MCP indicator dot — LED effect with magenta glow */
|
||||
.csb-mcp-dot {
|
||||
width: 7px;
|
||||
height: 7px;
|
||||
border-radius: 50%;
|
||||
background: var(--magenta);
|
||||
box-shadow: 0 0 4px var(--magenta-glow);
|
||||
display: inline-block;
|
||||
flex-shrink: 0;
|
||||
}
|
||||
|
||||
.csb-loading { color: var(--fg-dim); font-size: 11px; font-style: italic; opacity: 0.8; }
|
||||
|
||||
#cluster-status-bar.stale { border-top-color: var(--yellow); }
|
||||
@@ -454,6 +465,16 @@
|
||||
}
|
||||
.dash-cell-node:hover { text-decoration: underline; color: var(--fg-bright); }
|
||||
|
||||
/* ==========================================================================
|
||||
MCP summary in node detail
|
||||
========================================================================== */
|
||||
#node-mcp-summary {
|
||||
color: var(--magenta);
|
||||
font-size: 11px;
|
||||
font-family: var(--font-mono);
|
||||
margin-left: 12px;
|
||||
}
|
||||
|
||||
/* ==========================================================================
|
||||
Node link
|
||||
========================================================================== */
|
||||
@@ -665,6 +686,7 @@
|
||||
.node-group-header .node-group-cell:last-child { display: none; }
|
||||
.ncol-version, .node-cell-version { display: none; }
|
||||
.ncol-health, .node-cell-health { display: none; }
|
||||
#node-mcp-summary { display: none; }
|
||||
#main { padding: 16px; padding-bottom: 60px; }
|
||||
}
|
||||
@media (max-width: 480px) {
|
||||
|
||||
+551
-28
@@ -1,16 +1,16 @@
|
||||
"""MCP (Model Context Protocol) client manager.
|
||||
|
||||
Connects to external MCP tool servers and exposes their tools alongside
|
||||
turnstone's built-in tools.
|
||||
Connects to external MCP tool servers and exposes their tools, resources,
|
||||
and prompts alongside turnstone's built-in capabilities.
|
||||
|
||||
Architecture: the MCP SDK is fully async, but turnstone's ChatSession is
|
||||
synchronous. We bridge the two by running a dedicated asyncio event loop
|
||||
in a daemon thread. ``call_tool_sync`` dispatches coroutines onto that loop
|
||||
via ``asyncio.run_coroutine_threadsafe``.
|
||||
|
||||
Tool refresh: three mechanisms keep tool lists up-to-date without restart:
|
||||
1. Push notifications — servers declaring ``tools.listChanged`` trigger
|
||||
immediate refresh via ``ToolListChangedNotification``.
|
||||
Refresh: three mechanisms keep tool/resource/prompt lists up-to-date:
|
||||
1. Push notifications — servers declaring ``listChanged`` on the
|
||||
respective capability trigger immediate refresh.
|
||||
2. Periodic timer — servers *without* push support are polled on a
|
||||
staggered interval (configurable, default 4 h, seeded at launch).
|
||||
3. Manual — ``/mcp refresh [server]`` triggers ``refresh_sync()``.
|
||||
@@ -19,6 +19,7 @@ Tool refresh: three mechanisms keep tool lists up-to-date without restart:
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import concurrent.futures
|
||||
import contextlib
|
||||
import json
|
||||
import logging
|
||||
@@ -26,6 +27,7 @@ import os
|
||||
import random
|
||||
import threading
|
||||
import time
|
||||
import uuid
|
||||
from contextlib import AsyncExitStack
|
||||
from pathlib import Path
|
||||
from typing import TYPE_CHECKING, Any
|
||||
@@ -112,6 +114,28 @@ class MCPClientManager:
|
||||
self._listeners: list[Callable[[], None]] = []
|
||||
self._listeners_lock = threading.Lock()
|
||||
|
||||
# Resources — parallel to tools
|
||||
self._per_server_resources: dict[str, list[dict[str, Any]]] = {}
|
||||
self._resources: list[dict[str, Any]] = []
|
||||
self._resource_map: dict[str, tuple[str, str]] = {} # uri → (server, uri)
|
||||
self._supports_resources: dict[str, bool] = {} # server has resources capability
|
||||
self._supports_resource_list_changed: dict[str, bool] = {}
|
||||
self._resource_listeners: list[Callable[[], None]] = []
|
||||
self._resource_listeners_lock = threading.Lock()
|
||||
|
||||
# Prompts — parallel to tools
|
||||
self._per_server_prompts: dict[str, list[dict[str, Any]]] = {}
|
||||
self._prompts: list[dict[str, Any]] = []
|
||||
self._prompt_map: dict[str, tuple[str, str]] = {} # prefixed → (server, original)
|
||||
self._supports_prompts: dict[str, bool] = {} # server has prompts capability
|
||||
self._supports_prompt_list_changed: dict[str, bool] = {}
|
||||
self._prompt_listeners: list[Callable[[], None]] = []
|
||||
self._prompt_listeners_lock = threading.Lock()
|
||||
|
||||
# Governance storage (optional — set via set_storage())
|
||||
self._storage: Any = None
|
||||
self._sync_lock = threading.Lock()
|
||||
|
||||
# Periodic refresh for servers without push notifications
|
||||
self._refresh_interval = refresh_interval
|
||||
self._refresh_task: asyncio.Task[None] | None = None
|
||||
@@ -146,7 +170,16 @@ class MCPClientManager:
|
||||
|
||||
# Start periodic refresh for servers without push notifications
|
||||
needs_periodic = any(
|
||||
not self._supports_list_changed.get(name, False) for name in self._sessions
|
||||
not self._supports_list_changed.get(name, False)
|
||||
or (
|
||||
self._supports_resources.get(name, False)
|
||||
and not self._supports_resource_list_changed.get(name, False)
|
||||
)
|
||||
or (
|
||||
self._supports_prompts.get(name, False)
|
||||
and not self._supports_prompt_list_changed.get(name, False)
|
||||
)
|
||||
for name in self._sessions
|
||||
)
|
||||
if needs_periodic and self._refresh_interval > 0:
|
||||
self._refresh_task = asyncio.get_running_loop().create_task(self._periodic_refresh())
|
||||
@@ -178,20 +211,26 @@ class MCPClientManager:
|
||||
)
|
||||
read, write = await self._exit_stack.enter_async_context(stdio_client(params))
|
||||
|
||||
# Register notification handler — lightweight; only acts on
|
||||
# ToolListChangedNotification, which is a no-op if the server
|
||||
# never sends it.
|
||||
# Register notification handler — dispatches tool, resource, and
|
||||
# prompt list-change notifications to the appropriate refresh method.
|
||||
async def _on_notification(
|
||||
msg: Any, # RequestResponder | ServerNotification | Exception
|
||||
) -> None:
|
||||
if isinstance(msg, mcp_types.ServerNotification) and isinstance(
|
||||
msg.root, mcp_types.ToolListChangedNotification
|
||||
):
|
||||
log.info("Received tools/list_changed from '%s'", name)
|
||||
try:
|
||||
await self._refresh_server(name)
|
||||
except Exception:
|
||||
log.warning("Refresh after notification failed for '%s'", name, exc_info=True)
|
||||
if not isinstance(msg, mcp_types.ServerNotification):
|
||||
return
|
||||
root = msg.root
|
||||
try:
|
||||
if isinstance(root, mcp_types.ToolListChangedNotification):
|
||||
log.info("Received tools/list_changed from '%s'", name)
|
||||
await self._refresh_server_tools(name)
|
||||
elif isinstance(root, mcp_types.ResourceListChangedNotification):
|
||||
log.info("Received resources/list_changed from '%s'", name)
|
||||
await self._refresh_server_resources(name)
|
||||
elif isinstance(root, mcp_types.PromptListChangedNotification):
|
||||
log.info("Received prompts/list_changed from '%s'", name)
|
||||
await self._refresh_server_prompts(name)
|
||||
except Exception:
|
||||
log.warning("Refresh after notification failed for '%s'", name, exc_info=True)
|
||||
|
||||
session = await self._exit_stack.enter_async_context(
|
||||
ClientSession(read, write, message_handler=_on_notification) # type: ignore[arg-type]
|
||||
@@ -199,11 +238,22 @@ class MCPClientManager:
|
||||
await session.initialize()
|
||||
self._sessions[name] = session
|
||||
|
||||
# Check push notification support
|
||||
# Check push notification support for each capability
|
||||
caps = session.get_server_capabilities()
|
||||
|
||||
tools_cap = getattr(caps, "tools", None) if caps else None
|
||||
self._supports_list_changed[name] = bool(getattr(tools_cap, "listChanged", False))
|
||||
|
||||
resources_cap = getattr(caps, "resources", None) if caps else None
|
||||
self._supports_resources[name] = resources_cap is not None
|
||||
self._supports_resource_list_changed[name] = bool(
|
||||
getattr(resources_cap, "listChanged", False)
|
||||
)
|
||||
|
||||
prompts_cap = getattr(caps, "prompts", None) if caps else None
|
||||
self._supports_prompts[name] = prompts_cap is not None
|
||||
self._supports_prompt_list_changed[name] = bool(getattr(prompts_cap, "listChanged", False))
|
||||
|
||||
# Discover tools
|
||||
result = await session.list_tools()
|
||||
server_tools: list[dict[str, Any]] = []
|
||||
@@ -213,14 +263,88 @@ class MCPClientManager:
|
||||
self._per_server_tools[name] = server_tools
|
||||
self._rebuild_tools()
|
||||
|
||||
push_status = " (push)" if self._supports_list_changed[name] else ""
|
||||
# Discover resources
|
||||
resource_count = 0
|
||||
if resources_cap is not None:
|
||||
server_resources: list[dict[str, Any]] = []
|
||||
res_result = await session.list_resources()
|
||||
for r in res_result.resources:
|
||||
server_resources.append(
|
||||
{
|
||||
"uri": str(r.uri),
|
||||
"name": r.name or "",
|
||||
"description": r.description or "",
|
||||
"mimeType": r.mimeType or "",
|
||||
"server": name,
|
||||
}
|
||||
)
|
||||
# Also include resource templates (catalog-only — not directly
|
||||
# readable via read_resource since they contain URI placeholders)
|
||||
tmpl_result = await session.list_resource_templates()
|
||||
for t in tmpl_result.resourceTemplates:
|
||||
server_resources.append(
|
||||
{
|
||||
"uri": str(t.uriTemplate),
|
||||
"name": t.name or "",
|
||||
"description": t.description or "",
|
||||
"mimeType": t.mimeType or "",
|
||||
"server": name,
|
||||
"template": True,
|
||||
}
|
||||
)
|
||||
resource_count = len(server_resources)
|
||||
self._per_server_resources[name] = server_resources
|
||||
self._rebuild_resources()
|
||||
|
||||
# Discover prompts
|
||||
prompt_count = 0
|
||||
if prompts_cap is not None:
|
||||
server_prompts: list[dict[str, Any]] = []
|
||||
prompt_result = await session.list_prompts()
|
||||
for p in prompt_result.prompts:
|
||||
server_prompts.append(
|
||||
{
|
||||
"name": f"mcp__{name}__{p.name}",
|
||||
"original_name": p.name,
|
||||
"server": name,
|
||||
"description": p.description or "",
|
||||
"arguments": [
|
||||
{
|
||||
"name": a.name,
|
||||
"description": a.description or "",
|
||||
"required": a.required or False,
|
||||
}
|
||||
for a in (p.arguments or [])
|
||||
],
|
||||
}
|
||||
)
|
||||
prompt_count = len(server_prompts)
|
||||
self._per_server_prompts[name] = server_prompts
|
||||
self._rebuild_prompts()
|
||||
|
||||
push_parts: list[str] = []
|
||||
if self._supports_list_changed[name]:
|
||||
push_parts.append("tools")
|
||||
if self._supports_resource_list_changed[name]:
|
||||
push_parts.append("resources")
|
||||
if self._supports_prompt_list_changed[name]:
|
||||
push_parts.append("prompts")
|
||||
push_status = f" (push: {','.join(push_parts)})" if push_parts else ""
|
||||
log.info(
|
||||
"Connected MCP server '%s' — %d tool(s)%s",
|
||||
"Connected MCP server '%s' — %d tool(s), %d resource(s), %d prompt(s)%s",
|
||||
name,
|
||||
len(result.tools),
|
||||
resource_count,
|
||||
prompt_count,
|
||||
push_status,
|
||||
)
|
||||
|
||||
# Sync discovered prompts into governance storage
|
||||
try:
|
||||
self.sync_prompts_to_storage()
|
||||
except Exception:
|
||||
log.warning("Prompt sync after connect failed for '%s'", name, exc_info=True)
|
||||
|
||||
# -- tool refresh --------------------------------------------------------
|
||||
|
||||
def _rebuild_tools(self) -> None:
|
||||
@@ -242,7 +366,7 @@ class MCPClientManager:
|
||||
self._tool_map = new_map
|
||||
self._notify_listeners()
|
||||
|
||||
async def _refresh_server(self, name: str) -> tuple[list[str], list[str]]:
|
||||
async def _refresh_server_tools(self, name: str) -> tuple[list[str], list[str]]:
|
||||
"""Re-fetch tools for one server. Returns ``(added, removed)`` names."""
|
||||
session = self._sessions.get(name)
|
||||
if session is None:
|
||||
@@ -268,10 +392,21 @@ class MCPClientManager:
|
||||
)
|
||||
return added, removed
|
||||
|
||||
async def _refresh_server(self, name: str) -> tuple[list[str], list[str]]:
|
||||
"""Re-fetch tools, resources, and prompts for one server.
|
||||
|
||||
Returns ``(added_tools, removed_tools)`` names (tool diff only,
|
||||
for backward compatibility with ``/mcp refresh`` output).
|
||||
"""
|
||||
added, removed = await self._refresh_server_tools(name)
|
||||
await self._refresh_server_resources(name)
|
||||
await self._refresh_server_prompts(name)
|
||||
return added, removed
|
||||
|
||||
async def _refresh_all(
|
||||
self, server_name: str | None = None
|
||||
) -> dict[str, tuple[list[str], list[str]]]:
|
||||
"""Refresh tools for one or all servers.
|
||||
"""Refresh tools, resources, and prompts for one or all servers.
|
||||
|
||||
For disconnected servers (in config but not connected), attempts
|
||||
reconnect. Returns ``{server: (added, removed)}`` per server.
|
||||
@@ -297,6 +432,13 @@ class MCPClientManager:
|
||||
except Exception:
|
||||
log.warning("Refresh failed for MCP server '%s'", name, exc_info=True)
|
||||
results[name] = ([], [])
|
||||
|
||||
# Final sync to clean up templates from servers that are no longer connected
|
||||
try:
|
||||
self.sync_prompts_to_storage()
|
||||
except Exception:
|
||||
log.warning("Prompt sync after refresh_all failed", exc_info=True)
|
||||
|
||||
return results
|
||||
|
||||
def refresh_sync(
|
||||
@@ -319,16 +461,137 @@ class MCPClientManager:
|
||||
await asyncio.sleep(initial_delay)
|
||||
while True:
|
||||
for name in list(self._server_configs):
|
||||
if self._supports_list_changed.get(name, False):
|
||||
continue # has push — skip
|
||||
if name not in self._sessions:
|
||||
continue # not connected — skip (reconnect on manual refresh)
|
||||
try:
|
||||
await self._refresh_server(name)
|
||||
if not self._supports_list_changed.get(name, False):
|
||||
await self._refresh_server_tools(name)
|
||||
if not self._supports_resource_list_changed.get(name, False):
|
||||
await self._refresh_server_resources(name)
|
||||
if not self._supports_prompt_list_changed.get(name, False):
|
||||
await self._refresh_server_prompts(name)
|
||||
except Exception:
|
||||
log.warning("Periodic refresh failed for '%s'", name, exc_info=True)
|
||||
await asyncio.sleep(self._refresh_interval)
|
||||
|
||||
# -- resource refresh ----------------------------------------------------
|
||||
|
||||
def _rebuild_resources(self) -> None:
|
||||
"""Rebuild merged ``_resources`` and ``_resource_map`` from per-server state.
|
||||
|
||||
Uses copy-on-write: builds new objects, then assigns atomically.
|
||||
"""
|
||||
new_resources: list[dict[str, Any]] = []
|
||||
new_map: dict[str, tuple[str, str]] = {}
|
||||
for srv_name, srv_resources in self._per_server_resources.items():
|
||||
for res in srv_resources:
|
||||
uri: str = res["uri"]
|
||||
new_resources.append(res)
|
||||
if res.get("template"):
|
||||
continue # templates are catalog-only, not directly readable
|
||||
if uri in new_map:
|
||||
log.warning(
|
||||
"Resource URI collision: '%s' from '%s' overrides '%s'",
|
||||
uri,
|
||||
srv_name,
|
||||
new_map[uri][0],
|
||||
)
|
||||
new_map[uri] = (srv_name, uri)
|
||||
self._resources = new_resources
|
||||
self._resource_map = new_map
|
||||
self._notify_resource_listeners()
|
||||
|
||||
async def _refresh_server_resources(self, name: str) -> None:
|
||||
"""Re-fetch resources for one server."""
|
||||
if not self._supports_resources.get(name, False):
|
||||
return
|
||||
session = self._sessions.get(name)
|
||||
if session is None:
|
||||
return
|
||||
|
||||
server_resources: list[dict[str, Any]] = []
|
||||
res_result = await session.list_resources()
|
||||
for r in res_result.resources:
|
||||
server_resources.append(
|
||||
{
|
||||
"uri": str(r.uri),
|
||||
"name": r.name or "",
|
||||
"description": r.description or "",
|
||||
"mimeType": r.mimeType or "",
|
||||
"server": name,
|
||||
}
|
||||
)
|
||||
tmpl_result = await session.list_resource_templates()
|
||||
for t in tmpl_result.resourceTemplates:
|
||||
server_resources.append(
|
||||
{
|
||||
"uri": str(t.uriTemplate),
|
||||
"name": t.name or "",
|
||||
"description": t.description or "",
|
||||
"mimeType": t.mimeType or "",
|
||||
"server": name,
|
||||
"template": True,
|
||||
}
|
||||
)
|
||||
|
||||
self._per_server_resources[name] = server_resources
|
||||
self._rebuild_resources()
|
||||
|
||||
# -- prompt refresh ------------------------------------------------------
|
||||
|
||||
def _rebuild_prompts(self) -> None:
|
||||
"""Rebuild merged ``_prompts`` and ``_prompt_map`` from per-server state.
|
||||
|
||||
Uses copy-on-write: builds new objects, then assigns atomically.
|
||||
"""
|
||||
new_prompts: list[dict[str, Any]] = []
|
||||
new_map: dict[str, tuple[str, str]] = {}
|
||||
for srv_name, srv_prompts in self._per_server_prompts.items():
|
||||
for prompt in srv_prompts:
|
||||
prefixed: str = prompt["name"]
|
||||
new_prompts.append(prompt)
|
||||
new_map[prefixed] = (srv_name, prompt["original_name"])
|
||||
self._prompts = new_prompts
|
||||
self._prompt_map = new_map
|
||||
self._notify_prompt_listeners()
|
||||
|
||||
async def _refresh_server_prompts(self, name: str) -> None:
|
||||
"""Re-fetch prompts for one server."""
|
||||
if not self._supports_prompts.get(name, False):
|
||||
return
|
||||
session = self._sessions.get(name)
|
||||
if session is None:
|
||||
return
|
||||
|
||||
server_prompts: list[dict[str, Any]] = []
|
||||
prompt_result = await session.list_prompts()
|
||||
for p in prompt_result.prompts:
|
||||
server_prompts.append(
|
||||
{
|
||||
"name": f"mcp__{name}__{p.name}",
|
||||
"original_name": p.name,
|
||||
"server": name,
|
||||
"description": p.description or "",
|
||||
"arguments": [
|
||||
{
|
||||
"name": a.name,
|
||||
"description": a.description or "",
|
||||
"required": a.required or False,
|
||||
}
|
||||
for a in (p.arguments or [])
|
||||
],
|
||||
}
|
||||
)
|
||||
|
||||
self._per_server_prompts[name] = server_prompts
|
||||
self._rebuild_prompts()
|
||||
|
||||
# Sync discovered prompts into governance storage
|
||||
try:
|
||||
self.sync_prompts_to_storage()
|
||||
except Exception:
|
||||
log.warning("Prompt sync after refresh failed for '%s'", name, exc_info=True)
|
||||
|
||||
# -- listener infrastructure ---------------------------------------------
|
||||
|
||||
def add_listener(self, callback: Callable[[], None]) -> None:
|
||||
@@ -342,7 +605,7 @@ class MCPClientManager:
|
||||
self._listeners.remove(callback)
|
||||
|
||||
def _notify_listeners(self) -> None:
|
||||
"""Invoke all registered listeners (runs on MCP background thread)."""
|
||||
"""Invoke all registered tool-change listeners."""
|
||||
with self._listeners_lock:
|
||||
listeners = list(self._listeners)
|
||||
for cb in listeners:
|
||||
@@ -351,6 +614,151 @@ class MCPClientManager:
|
||||
except Exception:
|
||||
log.warning("Tool-change listener raised", exc_info=True)
|
||||
|
||||
def add_resource_listener(self, callback: Callable[[], None]) -> None:
|
||||
"""Register a callback invoked when the resource list changes."""
|
||||
with self._resource_listeners_lock:
|
||||
self._resource_listeners.append(callback)
|
||||
|
||||
def remove_resource_listener(self, callback: Callable[[], None]) -> None:
|
||||
"""Unregister a resource-change callback."""
|
||||
with self._resource_listeners_lock, contextlib.suppress(ValueError):
|
||||
self._resource_listeners.remove(callback)
|
||||
|
||||
def _notify_resource_listeners(self) -> None:
|
||||
"""Invoke all registered resource-change listeners."""
|
||||
with self._resource_listeners_lock:
|
||||
listeners = list(self._resource_listeners)
|
||||
for cb in listeners:
|
||||
try:
|
||||
cb()
|
||||
except Exception:
|
||||
log.warning("Resource-change listener raised", exc_info=True)
|
||||
|
||||
def add_prompt_listener(self, callback: Callable[[], None]) -> None:
|
||||
"""Register a callback invoked when the prompt list changes."""
|
||||
with self._prompt_listeners_lock:
|
||||
self._prompt_listeners.append(callback)
|
||||
|
||||
def remove_prompt_listener(self, callback: Callable[[], None]) -> None:
|
||||
"""Unregister a prompt-change callback."""
|
||||
with self._prompt_listeners_lock, contextlib.suppress(ValueError):
|
||||
self._prompt_listeners.remove(callback)
|
||||
|
||||
def _notify_prompt_listeners(self) -> None:
|
||||
"""Invoke all registered prompt-change listeners."""
|
||||
with self._prompt_listeners_lock:
|
||||
listeners = list(self._prompt_listeners)
|
||||
for cb in listeners:
|
||||
try:
|
||||
cb()
|
||||
except Exception:
|
||||
log.warning("Prompt-change listener raised", exc_info=True)
|
||||
|
||||
# -- governance storage sync ---------------------------------------------
|
||||
|
||||
def set_storage(self, storage: Any) -> None:
|
||||
"""Inject governance storage backend for prompt template sync.
|
||||
|
||||
If MCP servers are already connected, triggers an immediate sync
|
||||
so prompts discovered during startup appear in governance storage
|
||||
(``start()`` completes before ``set_storage()`` is called).
|
||||
"""
|
||||
self._storage = storage
|
||||
if self._connected.is_set():
|
||||
try:
|
||||
self.sync_prompts_to_storage()
|
||||
except Exception:
|
||||
log.warning("Prompt sync after set_storage failed", exc_info=True)
|
||||
|
||||
def sync_prompts_to_storage(self) -> dict[str, Any]:
|
||||
"""Sync discovered MCP prompts into the prompt_templates governance table.
|
||||
|
||||
Returns ``{"added": [...], "removed": [...], "skipped": [...]}``.
|
||||
Thread-safe: serialized via ``_sync_lock`` to prevent races
|
||||
between ``set_storage()`` (main thread) and MCP background thread.
|
||||
"""
|
||||
if self._storage is None:
|
||||
return {"added": [], "removed": [], "skipped": []}
|
||||
|
||||
with self._sync_lock:
|
||||
return self._sync_prompts_locked()
|
||||
|
||||
def _sync_prompts_locked(self) -> dict[str, Any]:
|
||||
"""Inner sync logic — must be called under ``_sync_lock``."""
|
||||
storage = self._storage
|
||||
added: list[str] = []
|
||||
removed: list[str] = []
|
||||
skipped: list[str] = []
|
||||
|
||||
# Current MCP prompt names (the prefixed names used as template names)
|
||||
current_names: set[str] = set()
|
||||
|
||||
for prompt in list(self._prompts):
|
||||
name: str = prompt["name"][:256]
|
||||
server: str = prompt["server"][:128]
|
||||
current_names.add(name)
|
||||
|
||||
# Build content from description + argument schema
|
||||
desc = prompt.get("description", "")[:4096]
|
||||
args_list = prompt.get("arguments", [])
|
||||
content_parts = [desc] if desc else []
|
||||
if args_list:
|
||||
content_parts.append("\nArguments:")
|
||||
for arg in args_list:
|
||||
req = " (required)" if arg.get("required") else ""
|
||||
arg_desc = arg.get("description", "")[:512]
|
||||
content_parts.append(f" - {arg['name'][:128]}{req}: {arg_desc}")
|
||||
content = "\n".join(content_parts) if content_parts else name
|
||||
|
||||
# Variables = JSON list of argument names
|
||||
variables = json.dumps([a["name"] for a in args_list])
|
||||
|
||||
existing = storage.get_prompt_template_by_name(name)
|
||||
if existing is not None:
|
||||
if existing.get("origin") == "manual":
|
||||
log.info(
|
||||
"Skipping MCP prompt '%s' — manual template with same name exists", name
|
||||
)
|
||||
skipped.append(name)
|
||||
continue
|
||||
# Existing MCP template — update content/variables
|
||||
storage.update_prompt_template(
|
||||
existing["template_id"], content=content, variables=variables
|
||||
)
|
||||
else:
|
||||
# Create new MCP-sourced template
|
||||
template_id = str(uuid.uuid4())
|
||||
storage.create_prompt_template(
|
||||
template_id=template_id,
|
||||
name=name,
|
||||
category="mcp",
|
||||
content=content,
|
||||
variables=variables,
|
||||
is_default=False,
|
||||
org_id="",
|
||||
created_by="",
|
||||
origin="mcp",
|
||||
mcp_server=server,
|
||||
readonly=True,
|
||||
)
|
||||
added.append(name)
|
||||
|
||||
# Remove MCP templates whose prompts no longer exist
|
||||
existing_mcp = storage.list_prompt_templates_by_origin("mcp")
|
||||
for tpl in existing_mcp:
|
||||
if tpl["name"] not in current_names:
|
||||
storage.delete_prompt_template(tpl["template_id"])
|
||||
removed.append(tpl["name"])
|
||||
|
||||
if added or removed:
|
||||
log.info(
|
||||
"MCP prompt sync: +%d added, -%d removed, %d skipped",
|
||||
len(added),
|
||||
len(removed),
|
||||
len(skipped),
|
||||
)
|
||||
return {"added": added, "removed": removed, "skipped": skipped}
|
||||
|
||||
# -- lifecycle (shutdown) ------------------------------------------------
|
||||
|
||||
def shutdown(self) -> None:
|
||||
@@ -371,18 +779,61 @@ class MCPClientManager:
|
||||
if self._thread:
|
||||
self._thread.join(timeout=5)
|
||||
|
||||
# Clear all state
|
||||
self._sessions.clear()
|
||||
self._tools = []
|
||||
self._tool_map = {}
|
||||
self._per_server_tools.clear()
|
||||
self._supports_list_changed.clear()
|
||||
self._resources = []
|
||||
self._resource_map = {}
|
||||
self._per_server_resources.clear()
|
||||
self._supports_resources.clear()
|
||||
self._supports_resource_list_changed.clear()
|
||||
self._prompts = []
|
||||
self._prompt_map = {}
|
||||
self._per_server_prompts.clear()
|
||||
self._supports_prompts.clear()
|
||||
self._supports_prompt_list_changed.clear()
|
||||
# Clear listener lists to release callback references
|
||||
self._listeners.clear()
|
||||
self._resource_listeners.clear()
|
||||
self._prompt_listeners.clear()
|
||||
|
||||
log.info("MCP client shut down")
|
||||
|
||||
# -- query methods -------------------------------------------------------
|
||||
|
||||
def get_tools(self) -> list[dict[str, Any]]:
|
||||
"""Return MCP tools in OpenAI function-calling format."""
|
||||
return list(self._tools)
|
||||
return [dict(t) for t in self._tools]
|
||||
|
||||
def get_resources(self) -> list[dict[str, Any]]:
|
||||
"""Return discovered MCP resources (shallow-copied dicts)."""
|
||||
return [dict(r) for r in self._resources]
|
||||
|
||||
def get_prompts(self) -> list[dict[str, Any]]:
|
||||
"""Return discovered MCP prompts (shallow-copied dicts)."""
|
||||
return [dict(p) for p in self._prompts]
|
||||
|
||||
@property
|
||||
def resource_count(self) -> int:
|
||||
"""Number of discovered resources (no allocation)."""
|
||||
return len(self._resources)
|
||||
|
||||
@property
|
||||
def prompt_count(self) -> int:
|
||||
"""Number of discovered prompts (no allocation)."""
|
||||
return len(self._prompts)
|
||||
|
||||
def is_mcp_tool(self, func_name: str) -> bool:
|
||||
"""Check whether *func_name* belongs to an MCP server."""
|
||||
return func_name in self._tool_map
|
||||
|
||||
def is_mcp_prompt(self, name: str) -> bool:
|
||||
"""Check whether *name* is a known MCP prompt."""
|
||||
return name in self._prompt_map
|
||||
|
||||
@property
|
||||
def server_count(self) -> int:
|
||||
return len(self._sessions)
|
||||
@@ -417,7 +868,10 @@ class MCPClientManager:
|
||||
future = asyncio.run_coroutine_threadsafe(
|
||||
session.call_tool(original_name, arguments), self._loop
|
||||
)
|
||||
result = future.result(timeout=timeout)
|
||||
try:
|
||||
result = future.result(timeout=timeout)
|
||||
except concurrent.futures.TimeoutError:
|
||||
raise TimeoutError(f"MCP tool call timed out after {timeout}s") from None
|
||||
|
||||
# Extract text from the content array
|
||||
texts: list[str] = []
|
||||
@@ -435,6 +889,75 @@ class MCPClientManager:
|
||||
output = f"Error: {output}"
|
||||
return output
|
||||
|
||||
# -- resource read -------------------------------------------------------
|
||||
|
||||
def read_resource_sync(self, uri: str, timeout: int = 120) -> str:
|
||||
"""Read a resource by URI synchronously (blocks the calling thread).
|
||||
|
||||
Returns text content for ``TextResourceContents``, or base64 data
|
||||
for ``BlobResourceContents``.
|
||||
"""
|
||||
mapping = self._resource_map.get(uri)
|
||||
if mapping is None:
|
||||
raise ValueError(f"Unknown MCP resource: {uri}")
|
||||
server_name, _ = mapping
|
||||
session = self._sessions.get(server_name)
|
||||
if session is None:
|
||||
raise RuntimeError(f"MCP server '{server_name}' is not connected")
|
||||
assert self._loop is not None
|
||||
|
||||
future = asyncio.run_coroutine_threadsafe(session.read_resource(uri), self._loop)
|
||||
try:
|
||||
result = future.result(timeout=timeout)
|
||||
except concurrent.futures.TimeoutError:
|
||||
raise TimeoutError(f"MCP resource read timed out after {timeout}s") from None
|
||||
|
||||
parts: list[str] = []
|
||||
for item in result.contents:
|
||||
if hasattr(item, "text"):
|
||||
parts.append(item.text)
|
||||
elif hasattr(item, "blob"):
|
||||
parts.append(item.blob)
|
||||
else:
|
||||
parts.append(str(item))
|
||||
return "\n".join(parts) if parts else "(empty resource)"
|
||||
|
||||
# -- prompt invocation ---------------------------------------------------
|
||||
|
||||
def get_prompt_sync(
|
||||
self,
|
||||
prefixed_name: str,
|
||||
arguments: dict[str, str] | None = None,
|
||||
timeout: int = 30,
|
||||
) -> list[dict[str, Any]]:
|
||||
"""Invoke an MCP prompt synchronously and return expanded messages.
|
||||
|
||||
Returns a list of ``{role: str, content: str}`` dicts.
|
||||
"""
|
||||
mapping = self._prompt_map.get(prefixed_name)
|
||||
if mapping is None:
|
||||
raise ValueError(f"Unknown MCP prompt: {prefixed_name}")
|
||||
server_name, original_name = mapping
|
||||
session = self._sessions.get(server_name)
|
||||
if session is None:
|
||||
raise RuntimeError(f"MCP server '{server_name}' is not connected")
|
||||
assert self._loop is not None
|
||||
|
||||
future = asyncio.run_coroutine_threadsafe(
|
||||
session.get_prompt(original_name, arguments=arguments), self._loop
|
||||
)
|
||||
try:
|
||||
result = future.result(timeout=timeout)
|
||||
except concurrent.futures.TimeoutError:
|
||||
raise TimeoutError(f"MCP prompt retrieval timed out after {timeout}s") from None
|
||||
|
||||
messages: list[dict[str, Any]] = []
|
||||
for msg in result.messages:
|
||||
content = msg.content
|
||||
text = content.text if hasattr(content, "text") else str(content)
|
||||
messages.append({"role": msg.role, "content": text})
|
||||
return messages
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Config loading
|
||||
|
||||
@@ -102,6 +102,7 @@ class MetricsCollector:
|
||||
workstream_states: dict[str, int],
|
||||
total_workstreams: int,
|
||||
workstream_metrics: list[dict[str, Any]] | None = None,
|
||||
mcp_info: dict[str, int] | None = None,
|
||||
) -> str:
|
||||
"""Return Prometheus text exposition format (v0.0.4)."""
|
||||
lines: list[str] = []
|
||||
@@ -322,6 +323,24 @@ class MetricsCollector:
|
||||
f"turnstone_workstream_context_ratio{lstr} {_fmt_value(wm['context_ratio'])}"
|
||||
)
|
||||
|
||||
# MCP gauges (optional)
|
||||
if mcp_info:
|
||||
gauge(
|
||||
"turnstone_mcp_servers",
|
||||
"Number of connected MCP servers",
|
||||
mcp_info.get("servers", 0),
|
||||
)
|
||||
gauge(
|
||||
"turnstone_mcp_resources",
|
||||
"Number of MCP resources available",
|
||||
mcp_info.get("resources", 0),
|
||||
)
|
||||
gauge(
|
||||
"turnstone_mcp_prompts",
|
||||
"Number of MCP prompts available",
|
||||
mcp_info.get("prompts", 0),
|
||||
)
|
||||
|
||||
lines.append("") # trailing newline
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
+259
-11
@@ -24,6 +24,7 @@ import textwrap
|
||||
import threading
|
||||
import time
|
||||
import uuid
|
||||
from html import escape as _html_escape
|
||||
from typing import TYPE_CHECKING, Any, Protocol
|
||||
|
||||
import httpx
|
||||
@@ -246,6 +247,8 @@ class ChatSession:
|
||||
# MCP tool integration: merge external tools with built-in
|
||||
self._mcp_client = mcp_client
|
||||
self._mcp_refresh_cb: Any = None # Callable | None (avoid import)
|
||||
self._mcp_resource_cb: Any = None
|
||||
self._mcp_prompt_cb: Any = None
|
||||
if mcp_client:
|
||||
mcp_tools = mcp_client.get_tools()
|
||||
self._tools = merge_mcp_tools(TOOLS, mcp_tools)
|
||||
@@ -254,6 +257,12 @@ class ChatSession:
|
||||
# Register for tool-change notifications from MCP servers
|
||||
self._mcp_refresh_cb = self._on_mcp_tools_changed
|
||||
mcp_client.add_listener(self._mcp_refresh_cb)
|
||||
# Register for resource-change notifications
|
||||
self._mcp_resource_cb = self._on_mcp_resources_changed
|
||||
mcp_client.add_resource_listener(self._mcp_resource_cb)
|
||||
# Register for prompt-change notifications
|
||||
self._mcp_prompt_cb = self._on_mcp_prompts_changed
|
||||
mcp_client.add_prompt_listener(self._mcp_prompt_cb)
|
||||
else:
|
||||
self._tools = TOOLS
|
||||
self._task_tools = TASK_AGENT_TOOLS
|
||||
@@ -333,6 +342,22 @@ class ChatSession:
|
||||
self._agent_tools = merge_mcp_tools(AGENT_TOOLS, mcp_tools)
|
||||
self._rebuild_tool_search()
|
||||
|
||||
def _on_mcp_resources_changed(self) -> None:
|
||||
"""Callback from MCPClientManager when the resource list changes.
|
||||
|
||||
Rebuilds the system message to update the resource catalog.
|
||||
Called on the MCP background thread.
|
||||
"""
|
||||
self._init_system_messages()
|
||||
|
||||
def _on_mcp_prompts_changed(self) -> None:
|
||||
"""Callback from MCPClientManager when the prompt list changes.
|
||||
|
||||
Rebuilds the system message to update the prompt catalog.
|
||||
Called on the MCP background thread.
|
||||
"""
|
||||
self._init_system_messages()
|
||||
|
||||
def _rebuild_tool_search(self) -> None:
|
||||
"""Reconstruct ToolSearchManager, preserving expanded tools."""
|
||||
old_expanded = self._tool_search.get_expanded_names() if self._tool_search else []
|
||||
@@ -375,6 +400,12 @@ class ChatSession:
|
||||
if self._mcp_client and self._mcp_refresh_cb:
|
||||
self._mcp_client.remove_listener(self._mcp_refresh_cb)
|
||||
self._mcp_refresh_cb = None
|
||||
if self._mcp_client and self._mcp_resource_cb:
|
||||
self._mcp_client.remove_resource_listener(self._mcp_resource_cb)
|
||||
self._mcp_resource_cb = None
|
||||
if self._mcp_client and self._mcp_prompt_cb:
|
||||
self._mcp_client.remove_prompt_listener(self._mcp_prompt_cb)
|
||||
self._mcp_prompt_cb = None
|
||||
if self._watch_runner:
|
||||
self._watch_runner.remove_dispatch_fn(self._ws_id)
|
||||
|
||||
@@ -521,8 +552,12 @@ class ChatSession:
|
||||
Developer message contains tool patterns (or creative writing
|
||||
instructions when creative_mode is on), plus any user-supplied
|
||||
instructions and memory reminders.
|
||||
|
||||
Uses copy-on-write: builds new lists locally, then assigns
|
||||
atomically so concurrent readers (e.g. background thread
|
||||
callbacks) never see a partially-built system message.
|
||||
"""
|
||||
self.system_messages: list[dict[str, Any]] = []
|
||||
new_system_messages: list[dict[str, Any]] = []
|
||||
|
||||
# -- Chat template kwargs --
|
||||
self._chat_template_kwargs_base: dict[str, Any] = {
|
||||
@@ -583,6 +618,38 @@ class ChatSession:
|
||||
"\n\nAdditional tools are available via tool_search. "
|
||||
"Use it when you need a capability not in your current tool set."
|
||||
)
|
||||
# MCP resource catalog (lets the model know what's available for read_resource)
|
||||
# Only concrete resources — templates are not directly readable.
|
||||
if self._mcp_client:
|
||||
concrete = [r for r in self._mcp_client.get_resources() if not r.get("template")]
|
||||
if concrete:
|
||||
lines = ["\n<mcp-resources>"]
|
||||
for r in concrete[:50]:
|
||||
safe_uri = _html_escape(r["uri"])
|
||||
desc = r.get("description", "")
|
||||
if desc:
|
||||
desc = f" {_html_escape(desc[:100])}"
|
||||
lines.append(f" {safe_uri}{desc}")
|
||||
lines.append("</mcp-resources>")
|
||||
lines.append("Use read_resource(uri='...') to access the resources listed above.")
|
||||
dev_parts.append("\n".join(lines))
|
||||
# MCP prompt catalog (lets the model know what's available for use_prompt)
|
||||
if self._mcp_client:
|
||||
prompts = self._mcp_client.get_prompts()
|
||||
if prompts:
|
||||
lines = ["<mcp-prompts>"]
|
||||
for p in prompts[:30]:
|
||||
# Names/args are NOT escaped — model must use exact strings
|
||||
# in use_prompt(). Only description (display-only) is escaped.
|
||||
arg_names = ", ".join(a["name"] for a in p.get("arguments", []))
|
||||
desc = _html_escape(p.get("description", "")[:100])
|
||||
lines.append(f" {p['name']}({arg_names}) {desc}")
|
||||
lines.append("</mcp-prompts>")
|
||||
lines.append(
|
||||
"Use use_prompt(name='...', arguments={...}) "
|
||||
"to invoke the prompts listed above."
|
||||
)
|
||||
dev_parts.append("\n".join(lines))
|
||||
if self.instructions:
|
||||
dev_parts.append("")
|
||||
dev_parts.append(self.instructions)
|
||||
@@ -593,9 +660,11 @@ class ChatSession:
|
||||
f"REMINDER: You currently have {len(memories)} memories stored. "
|
||||
"Use recall to see them."
|
||||
)
|
||||
self.system_messages.append({"role": "system", "content": "\n".join(dev_parts)})
|
||||
new_system_messages.append({"role": "system", "content": "\n".join(dev_parts)})
|
||||
# Atomic swap — readers see either old or new, never partial
|
||||
self.system_messages = new_system_messages
|
||||
# Agent prefix: system + developer only (no memories)
|
||||
self._agent_system_messages = list(self.system_messages)
|
||||
self._agent_system_messages = list(new_system_messages)
|
||||
|
||||
def _full_messages(self) -> list[dict[str, Any]]:
|
||||
"""System messages + conversation history."""
|
||||
@@ -1641,11 +1710,13 @@ class ChatSession:
|
||||
"command",
|
||||
"code",
|
||||
"content",
|
||||
"name",
|
||||
"page",
|
||||
"path",
|
||||
"pattern",
|
||||
"prompt",
|
||||
"query",
|
||||
"uri",
|
||||
"url",
|
||||
):
|
||||
m = re.search(rf'"{key}"\s*:\s*"((?:[^"\\]|\\.)*)"', raw_args)
|
||||
@@ -1690,6 +1761,8 @@ class ChatSession:
|
||||
"forget": self._prepare_forget,
|
||||
"notify": self._prepare_notify,
|
||||
"watch": self._prepare_watch,
|
||||
"read_resource": self._prepare_read_resource,
|
||||
"use_prompt": self._prepare_use_prompt,
|
||||
}
|
||||
preparer = preparers.get(func_name)
|
||||
if not preparer:
|
||||
@@ -2346,7 +2419,7 @@ class ChatSession:
|
||||
"header": f"\u2699 mcp:{display}",
|
||||
"preview": f"{DIM}{preview}{RESET}",
|
||||
"needs_approval": True,
|
||||
"approval_label": "mcp_tool",
|
||||
"approval_label": func_name,
|
||||
"execute": self._exec_mcp_tool,
|
||||
"mcp_func_name": func_name,
|
||||
"mcp_args": args,
|
||||
@@ -2372,6 +2445,160 @@ class ChatSession:
|
||||
self.ui.on_tool_result(call_id, func_name, output)
|
||||
return call_id, output
|
||||
|
||||
@staticmethod
|
||||
def _normalize_resource_uri(uri: str) -> str:
|
||||
"""Normalize a resource URI for policy matching.
|
||||
|
||||
Decodes percent-encoded path segments (e.g. ``%2e%2e`` → ``..``)
|
||||
then resolves ``..`` to prevent traversal bypasses where
|
||||
``file:///docs/%2e%2e/etc/passwd`` would match a policy
|
||||
allowing ``mcp_resource__file:///docs/*``.
|
||||
"""
|
||||
import posixpath
|
||||
from urllib.parse import quote, unquote, urlparse, urlunparse
|
||||
|
||||
parsed = urlparse(uri)
|
||||
if parsed.path:
|
||||
decoded = unquote(parsed.path)
|
||||
normalized = posixpath.normpath(decoded)
|
||||
if parsed.path.startswith("/") and not normalized.startswith("/"):
|
||||
normalized = "/" + normalized
|
||||
parsed = parsed._replace(path=quote(normalized, safe="/"))
|
||||
return urlunparse(parsed)
|
||||
|
||||
def _prepare_read_resource(self, call_id: str, args: dict[str, Any]) -> dict[str, Any]:
|
||||
"""Prepare an MCP resource read."""
|
||||
uri = args.get("uri", "")
|
||||
if not uri:
|
||||
return {
|
||||
"call_id": call_id,
|
||||
"func_name": "read_resource",
|
||||
"header": "\u2717 read_resource: missing uri",
|
||||
"preview": "",
|
||||
"needs_approval": False,
|
||||
"error": "Missing required parameter: uri",
|
||||
}
|
||||
if not self._mcp_client:
|
||||
return {
|
||||
"call_id": call_id,
|
||||
"func_name": "read_resource",
|
||||
"header": "\u2717 read_resource: no MCP servers",
|
||||
"preview": "",
|
||||
"needs_approval": False,
|
||||
"error": "No MCP servers configured",
|
||||
}
|
||||
return {
|
||||
"call_id": call_id,
|
||||
"func_name": "read_resource",
|
||||
"header": "\u2699 read_resource",
|
||||
"preview": f"{DIM} uri: {uri}{RESET}",
|
||||
"needs_approval": True,
|
||||
"approval_label": f"mcp_resource__{self._normalize_resource_uri(uri)}",
|
||||
"execute": self._exec_read_resource,
|
||||
"resource_uri": uri,
|
||||
}
|
||||
|
||||
def _exec_read_resource(self, item: dict[str, Any]) -> tuple[str, str]:
|
||||
"""Read an MCP resource by URI."""
|
||||
call_id: str = item["call_id"]
|
||||
uri: str = item["resource_uri"]
|
||||
|
||||
assert self._mcp_client is not None
|
||||
try:
|
||||
output = self._mcp_client.read_resource_sync(uri, timeout=self.tool_timeout)
|
||||
except TimeoutError:
|
||||
output = f"MCP resource read timed out after {self.tool_timeout}s"
|
||||
self.ui.on_error(output)
|
||||
except Exception:
|
||||
log.warning("MCP resource read failed for %s", uri, exc_info=True)
|
||||
output = "MCP resource error: failed to read resource"
|
||||
self.ui.on_error(output)
|
||||
|
||||
output = self._truncate_output(output)
|
||||
self.ui.on_tool_result(call_id, "read_resource", output)
|
||||
return call_id, output
|
||||
|
||||
def _prepare_use_prompt(self, call_id: str, args: dict[str, Any]) -> dict[str, Any]:
|
||||
"""Prepare an MCP prompt invocation."""
|
||||
name = args.get("name", "")
|
||||
if not name:
|
||||
return {
|
||||
"call_id": call_id,
|
||||
"func_name": "use_prompt",
|
||||
"header": "\u2717 use_prompt: missing name",
|
||||
"preview": "",
|
||||
"needs_approval": False,
|
||||
"error": "Missing required parameter: name",
|
||||
}
|
||||
if not self._mcp_client:
|
||||
return {
|
||||
"call_id": call_id,
|
||||
"func_name": "use_prompt",
|
||||
"header": "\u2717 use_prompt: no MCP servers",
|
||||
"preview": "",
|
||||
"needs_approval": False,
|
||||
"error": "No MCP servers configured",
|
||||
}
|
||||
if not self._mcp_client.is_mcp_prompt(name):
|
||||
return {
|
||||
"call_id": call_id,
|
||||
"func_name": "use_prompt",
|
||||
"header": f"\u2717 use_prompt: unknown prompt '{name}'",
|
||||
"preview": "",
|
||||
"needs_approval": False,
|
||||
"error": f"Unknown MCP prompt: {name}",
|
||||
}
|
||||
raw_arguments = args.get("arguments") or {}
|
||||
if not isinstance(raw_arguments, dict):
|
||||
return {
|
||||
"call_id": call_id,
|
||||
"func_name": "use_prompt",
|
||||
"header": "\u2717 use_prompt: arguments must be an object",
|
||||
"preview": "",
|
||||
"needs_approval": False,
|
||||
"error": "arguments must be a JSON object with string values",
|
||||
}
|
||||
arguments = {str(k): str(v) for k, v in raw_arguments.items()}
|
||||
preview_parts = [f" {DIM}name: {name}"]
|
||||
if arguments:
|
||||
preview_parts.append(f" arguments: {arguments}")
|
||||
preview_parts.append(RESET)
|
||||
return {
|
||||
"call_id": call_id,
|
||||
"func_name": "use_prompt",
|
||||
"header": "\u2699 use_prompt",
|
||||
"preview": "\n".join(preview_parts),
|
||||
"needs_approval": True,
|
||||
"approval_label": name,
|
||||
"execute": self._exec_use_prompt,
|
||||
"prompt_name": name,
|
||||
"prompt_arguments": arguments,
|
||||
}
|
||||
|
||||
def _exec_use_prompt(self, item: dict[str, Any]) -> tuple[str, str]:
|
||||
"""Invoke an MCP prompt and return expanded messages."""
|
||||
call_id: str = item["call_id"]
|
||||
name: str = item["prompt_name"]
|
||||
arguments: dict[str, str] = item["prompt_arguments"]
|
||||
|
||||
assert self._mcp_client is not None
|
||||
try:
|
||||
messages = self._mcp_client.get_prompt_sync(
|
||||
name, arguments or None, timeout=self.tool_timeout
|
||||
)
|
||||
output = "\n\n".join(f"[{m['role']}]: {m['content']}" for m in messages)
|
||||
except TimeoutError:
|
||||
output = f"MCP prompt timed out after {self.tool_timeout}s"
|
||||
self.ui.on_error(output)
|
||||
except Exception:
|
||||
log.warning("MCP prompt invocation failed for %s", name, exc_info=True)
|
||||
output = "MCP prompt error: failed to invoke prompt"
|
||||
self.ui.on_error(output)
|
||||
|
||||
output = self._truncate_output(output)
|
||||
self.ui.on_tool_result(call_id, "use_prompt", output)
|
||||
return call_id, output
|
||||
|
||||
# -- Execute methods (do the work, report output via UI) -------------------
|
||||
|
||||
def _exec_bash(self, item: dict[str, Any]) -> tuple[str, str]:
|
||||
@@ -4062,15 +4289,36 @@ class ChatSession:
|
||||
self._handle_mcp_refresh(arg)
|
||||
else:
|
||||
tools = self._mcp_client.get_tools()
|
||||
if not tools:
|
||||
self.ui.on_info("MCP client connected but no tools available.")
|
||||
else:
|
||||
lines = [f"MCP tools ({len(tools)}):"]
|
||||
resources = self._mcp_client.get_resources()
|
||||
prompts = self._mcp_client.get_prompts()
|
||||
mcp_lines = []
|
||||
if tools:
|
||||
mcp_lines.append(f"MCP tools ({len(tools)}):")
|
||||
for t in tools:
|
||||
name = t["function"]["name"]
|
||||
desc = t["function"].get("description", "")[:80]
|
||||
lines.append(f" {name} {dim(desc)}")
|
||||
self.ui.on_info("\n".join(lines))
|
||||
mcp_lines.append(f" {name} {dim(desc)}")
|
||||
if resources:
|
||||
if mcp_lines:
|
||||
mcp_lines.append("")
|
||||
mcp_lines.append(f"MCP resources ({len(resources)}):")
|
||||
for r in resources:
|
||||
desc = r.get("description", "")[:80]
|
||||
mcp_lines.append(f" {r['uri']} {dim(desc)}")
|
||||
if prompts:
|
||||
if mcp_lines:
|
||||
mcp_lines.append("")
|
||||
mcp_lines.append(f"MCP prompts ({len(prompts)}):")
|
||||
for p in prompts:
|
||||
arg_names = ", ".join(a["name"] for a in p.get("arguments", []))
|
||||
desc = p.get("description", "")[:60]
|
||||
mcp_lines.append(f" {p['name']}({arg_names}) {dim(desc)}")
|
||||
if not mcp_lines:
|
||||
self.ui.on_info(
|
||||
"MCP client connected but no tools, resources, or prompts available."
|
||||
)
|
||||
else:
|
||||
self.ui.on_info("\n".join(mcp_lines))
|
||||
|
||||
elif cmd == "/help":
|
||||
self.ui.on_info(
|
||||
@@ -4094,7 +4342,7 @@ class ChatSession:
|
||||
" /reason [low|med|high] Set/show reasoning effort",
|
||||
" /creative Toggle creative writing mode (no tools)",
|
||||
" /debug Toggle raw SSE delta logging",
|
||||
" /mcp [refresh [server]] List or refresh MCP tools",
|
||||
" /mcp [refresh [server]] List or refresh MCP tools, resources, and prompts",
|
||||
" /help Show this help",
|
||||
" /exit Exit (also: Ctrl+D)",
|
||||
"────────────────────────────────────────────────────────",
|
||||
|
||||
@@ -1512,6 +1512,9 @@ class PostgreSQLBackend:
|
||||
is_default: bool = False,
|
||||
org_id: str = "",
|
||||
created_by: str = "",
|
||||
origin: str = "manual",
|
||||
mcp_server: str = "",
|
||||
readonly: bool = False,
|
||||
) -> None:
|
||||
now = datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%S")
|
||||
with self._engine.connect() as conn:
|
||||
@@ -1526,6 +1529,9 @@ class PostgreSQLBackend:
|
||||
"is_default": 1 if is_default else 0,
|
||||
"org_id": org_id,
|
||||
"created_by": created_by,
|
||||
"origin": origin,
|
||||
"mcp_server": mcp_server,
|
||||
"readonly": 1 if readonly else 0,
|
||||
"created": now,
|
||||
"updated": now,
|
||||
},
|
||||
@@ -1538,7 +1544,16 @@ class PostgreSQLBackend:
|
||||
sa.select(prompt_templates).where(prompt_templates.c.template_id == template_id)
|
||||
).fetchone()
|
||||
if row:
|
||||
return _row_to_dict(row, "is_default")
|
||||
return _row_to_dict(row, "is_default", "readonly")
|
||||
return None
|
||||
|
||||
def get_prompt_template_by_name(self, name: str) -> dict[str, Any] | None:
|
||||
with self._engine.connect() as conn:
|
||||
row = conn.execute(
|
||||
sa.select(prompt_templates).where(prompt_templates.c.name == name)
|
||||
).fetchone()
|
||||
if row:
|
||||
return _row_to_dict(row, "is_default", "readonly")
|
||||
return None
|
||||
|
||||
def list_prompt_templates(self, org_id: str = "") -> list[dict[str, Any]]:
|
||||
@@ -1547,7 +1562,16 @@ class PostgreSQLBackend:
|
||||
if org_id:
|
||||
q = q.where(prompt_templates.c.org_id == org_id)
|
||||
rows = conn.execute(q).fetchall()
|
||||
return [_row_to_dict(r, "is_default") for r in rows]
|
||||
return [_row_to_dict(r, "is_default", "readonly") for r in rows]
|
||||
|
||||
def list_prompt_templates_by_origin(self, origin: str) -> list[dict[str, Any]]:
|
||||
with self._engine.connect() as conn:
|
||||
rows = conn.execute(
|
||||
sa.select(prompt_templates)
|
||||
.where(prompt_templates.c.origin == origin)
|
||||
.order_by(prompt_templates.c.name)
|
||||
).fetchall()
|
||||
return [_row_to_dict(r, "is_default", "readonly") for r in rows]
|
||||
|
||||
def update_prompt_template(self, template_id: str, **fields: Any) -> bool:
|
||||
dropped = set(fields) - _TEMPLATE_MUTABLE
|
||||
@@ -1574,6 +1598,14 @@ class PostgreSQLBackend:
|
||||
conn.commit()
|
||||
return result.rowcount > 0
|
||||
|
||||
def delete_prompt_templates_by_server(self, mcp_server: str) -> int:
|
||||
with self._engine.connect() as conn:
|
||||
result = conn.execute(
|
||||
sa.delete(prompt_templates).where(prompt_templates.c.mcp_server == mcp_server)
|
||||
)
|
||||
conn.commit()
|
||||
return result.rowcount
|
||||
|
||||
# -- Usage events ----------------------------------------------------------
|
||||
|
||||
def record_usage_event(
|
||||
|
||||
@@ -471,6 +471,9 @@ class StorageBackend(Protocol):
|
||||
is_default: bool,
|
||||
org_id: str,
|
||||
created_by: str,
|
||||
origin: str = "manual",
|
||||
mcp_server: str = "",
|
||||
readonly: bool = False,
|
||||
) -> None:
|
||||
"""Create a prompt template."""
|
||||
...
|
||||
@@ -479,10 +482,18 @@ class StorageBackend(Protocol):
|
||||
"""Return prompt template dict or None."""
|
||||
...
|
||||
|
||||
def get_prompt_template_by_name(self, name: str) -> dict[str, Any] | None:
|
||||
"""Lookup prompt template by name. Returns same dict as get_prompt_template or None."""
|
||||
...
|
||||
|
||||
def list_prompt_templates(self, org_id: str = "") -> list[dict[str, Any]]:
|
||||
"""Return all prompt templates ordered by name."""
|
||||
...
|
||||
|
||||
def list_prompt_templates_by_origin(self, origin: str) -> list[dict[str, Any]]:
|
||||
"""Return all prompt templates with the given origin, ordered by name."""
|
||||
...
|
||||
|
||||
def update_prompt_template(self, template_id: str, **fields: Any) -> bool:
|
||||
"""Update specified fields on a prompt template. Returns True if found."""
|
||||
...
|
||||
@@ -491,6 +502,10 @@ class StorageBackend(Protocol):
|
||||
"""Delete a prompt template. Returns True if found."""
|
||||
...
|
||||
|
||||
def delete_prompt_templates_by_server(self, mcp_server: str) -> int:
|
||||
"""Delete all prompt templates from a given MCP server. Returns count deleted."""
|
||||
...
|
||||
|
||||
# -- Usage events ----------------------------------------------------------
|
||||
|
||||
def record_usage_event(
|
||||
|
||||
@@ -288,6 +288,9 @@ prompt_templates = sa.Table(
|
||||
sa.Column("is_default", sa.Integer, nullable=False, server_default="0"),
|
||||
sa.Column("org_id", sa.Text, nullable=False, server_default=""),
|
||||
sa.Column("created_by", sa.Text, nullable=False, server_default=""),
|
||||
sa.Column("origin", sa.Text, nullable=False, server_default="manual"),
|
||||
sa.Column("mcp_server", sa.Text, nullable=False, server_default=""),
|
||||
sa.Column("readonly", sa.Integer, nullable=False, server_default="0"),
|
||||
sa.Column("created", sa.Text, nullable=False),
|
||||
sa.Column("updated", sa.Text, nullable=False),
|
||||
)
|
||||
|
||||
@@ -1546,6 +1546,9 @@ class SQLiteBackend:
|
||||
is_default: bool = False,
|
||||
org_id: str = "",
|
||||
created_by: str = "",
|
||||
origin: str = "manual",
|
||||
mcp_server: str = "",
|
||||
readonly: bool = False,
|
||||
) -> None:
|
||||
now = datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%S")
|
||||
with self._engine.connect() as conn:
|
||||
@@ -1560,6 +1563,9 @@ class SQLiteBackend:
|
||||
"is_default": 1 if is_default else 0,
|
||||
"org_id": org_id,
|
||||
"created_by": created_by,
|
||||
"origin": origin,
|
||||
"mcp_server": mcp_server,
|
||||
"readonly": 1 if readonly else 0,
|
||||
"created": now,
|
||||
"updated": now,
|
||||
},
|
||||
@@ -1572,7 +1578,16 @@ class SQLiteBackend:
|
||||
sa.select(prompt_templates).where(prompt_templates.c.template_id == template_id)
|
||||
).fetchone()
|
||||
if row:
|
||||
return _row_to_dict(row, "is_default")
|
||||
return _row_to_dict(row, "is_default", "readonly")
|
||||
return None
|
||||
|
||||
def get_prompt_template_by_name(self, name: str) -> dict[str, Any] | None:
|
||||
with self._engine.connect() as conn:
|
||||
row = conn.execute(
|
||||
sa.select(prompt_templates).where(prompt_templates.c.name == name)
|
||||
).fetchone()
|
||||
if row:
|
||||
return _row_to_dict(row, "is_default", "readonly")
|
||||
return None
|
||||
|
||||
def list_prompt_templates(self, org_id: str = "") -> list[dict[str, Any]]:
|
||||
@@ -1581,7 +1596,16 @@ class SQLiteBackend:
|
||||
if org_id:
|
||||
q = q.where(prompt_templates.c.org_id == org_id)
|
||||
rows = conn.execute(q).fetchall()
|
||||
return [_row_to_dict(r, "is_default") for r in rows]
|
||||
return [_row_to_dict(r, "is_default", "readonly") for r in rows]
|
||||
|
||||
def list_prompt_templates_by_origin(self, origin: str) -> list[dict[str, Any]]:
|
||||
with self._engine.connect() as conn:
|
||||
rows = conn.execute(
|
||||
sa.select(prompt_templates)
|
||||
.where(prompt_templates.c.origin == origin)
|
||||
.order_by(prompt_templates.c.name)
|
||||
).fetchall()
|
||||
return [_row_to_dict(r, "is_default", "readonly") for r in rows]
|
||||
|
||||
def update_prompt_template(self, template_id: str, **fields: Any) -> bool:
|
||||
dropped = set(fields) - _TEMPLATE_MUTABLE
|
||||
@@ -1608,6 +1632,14 @@ class SQLiteBackend:
|
||||
conn.commit()
|
||||
return result.rowcount > 0
|
||||
|
||||
def delete_prompt_templates_by_server(self, mcp_server: str) -> int:
|
||||
with self._engine.connect() as conn:
|
||||
result = conn.execute(
|
||||
sa.delete(prompt_templates).where(prompt_templates.c.mcp_server == mcp_server)
|
||||
)
|
||||
conn.commit()
|
||||
return result.rowcount
|
||||
|
||||
# -- Usage events ----------------------------------------------------------
|
||||
|
||||
def record_usage_event(
|
||||
|
||||
@@ -0,0 +1,28 @@
|
||||
"""Add MCP origin tracking columns to prompt_templates.
|
||||
|
||||
Revision ID: 009
|
||||
Revises: 008
|
||||
Create Date: 2026-03-12
|
||||
"""
|
||||
|
||||
import sqlalchemy as sa
|
||||
from alembic import op
|
||||
|
||||
revision = "009"
|
||||
down_revision = "008"
|
||||
branch_labels = None
|
||||
depends_on = None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
with op.batch_alter_table("prompt_templates") as batch_op:
|
||||
batch_op.add_column(sa.Column("origin", sa.Text, nullable=False, server_default="manual"))
|
||||
batch_op.add_column(sa.Column("mcp_server", sa.Text, nullable=False, server_default=""))
|
||||
batch_op.add_column(sa.Column("readonly", sa.Integer, nullable=False, server_default="0"))
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
with op.batch_alter_table("prompt_templates") as batch_op:
|
||||
batch_op.drop_column("readonly")
|
||||
batch_op.drop_column("mcp_server")
|
||||
batch_op.drop_column("origin")
|
||||
+25
-4
@@ -209,17 +209,21 @@ class WebUI:
|
||||
|
||||
storage = get_storage()
|
||||
if storage is not None:
|
||||
tool_names = [it.get("func_name", "") for it in pending if it.get("func_name")]
|
||||
tool_names = [
|
||||
it.get("approval_label", "") or it.get("func_name", "")
|
||||
for it in pending
|
||||
if it.get("func_name")
|
||||
]
|
||||
if tool_names:
|
||||
verdicts = evaluate_tool_policies_batch(storage, tool_names)
|
||||
still_pending = []
|
||||
for it in pending:
|
||||
fname = it.get("func_name", "")
|
||||
verdict = verdicts.get(fname)
|
||||
policy_name = it.get("approval_label", "") or it.get("func_name", "")
|
||||
verdict = verdicts.get(policy_name)
|
||||
if verdict == "deny":
|
||||
it["denied"] = True
|
||||
it["denial_msg"] = (
|
||||
f"Blocked by tool policy (pattern match for '{fname}')"
|
||||
f"Blocked by tool policy (pattern match for '{policy_name}')"
|
||||
)
|
||||
elif verdict == "allow":
|
||||
it["needs_approval"] = False
|
||||
@@ -764,6 +768,13 @@ async def health(request: Request) -> JSONResponse:
|
||||
"circuit_state": monitor.circuit_state.value if monitor else "closed",
|
||||
},
|
||||
}
|
||||
mc = getattr(request.app.state, "mcp_client", None)
|
||||
if mc:
|
||||
data["mcp"] = {
|
||||
"servers": mc.server_count,
|
||||
"resources": mc.resource_count,
|
||||
"prompts": mc.prompt_count,
|
||||
}
|
||||
return JSONResponse(data)
|
||||
|
||||
|
||||
@@ -787,10 +798,19 @@ async def metrics_endpoint(request: Request) -> Response:
|
||||
"context_ratio": ui._ws_context_ratio,
|
||||
}
|
||||
)
|
||||
mcp_info = None
|
||||
mc = getattr(request.app.state, "mcp_client", None)
|
||||
if mc:
|
||||
mcp_info = {
|
||||
"servers": mc.server_count,
|
||||
"resources": mc.resource_count,
|
||||
"prompts": mc.prompt_count,
|
||||
}
|
||||
content = _metrics.generate_text(
|
||||
workstream_states=states,
|
||||
total_workstreams=len(wss),
|
||||
workstream_metrics=ws_data,
|
||||
mcp_info=mcp_info,
|
||||
)
|
||||
return Response(content, media_type="text/plain; version=0.0.4; charset=utf-8")
|
||||
|
||||
@@ -1813,6 +1833,7 @@ def main() -> None:
|
||||
mcp_tools = mcp_client.get_tools()
|
||||
if mcp_tools:
|
||||
log.info("MCP tools: %d from %d server(s)", len(mcp_tools), mcp_client.server_count)
|
||||
mcp_client.set_storage(get_storage())
|
||||
log.info(
|
||||
"Health monitor: probe every %ss, circuit breaker threshold=%s",
|
||||
args.health_probe_interval,
|
||||
|
||||
@@ -36,6 +36,7 @@
|
||||
--yellow-glow: rgba(251, 191, 36, 0.25);
|
||||
--accent-glow-strong: rgba(229, 160, 66, 0.3);
|
||||
--cyan-glow: rgba(103, 232, 249, 0.2);
|
||||
--magenta-glow: rgba(192, 132, 252, 0.25);
|
||||
|
||||
/* Structure */
|
||||
--border: rgba(255, 255, 255, 0.06);
|
||||
@@ -73,6 +74,7 @@
|
||||
--yellow-glow: rgba(180, 83, 9, 0.25);
|
||||
--accent-glow-strong: rgba(140, 94, 27, 0.15);
|
||||
--cyan-glow: rgba(14, 116, 144, 0.2);
|
||||
--magenta-glow: rgba(124, 58, 237, 0.2);
|
||||
--border: rgba(0, 0, 0, 0.08);
|
||||
--border-strong: rgba(0, 0, 0, 0.12);
|
||||
--code-bg: #f0f1f5;
|
||||
|
||||
@@ -0,0 +1,18 @@
|
||||
{
|
||||
"name": "read_resource",
|
||||
"description": "Read a resource from a connected MCP server by URI. Returns the resource content (text or base64-encoded binary). Use this to access data, files, or content exposed by connected MCP servers. Available resource URIs are listed in your system context.",
|
||||
"parameters": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"uri": {
|
||||
"type": "string",
|
||||
"description": "The resource URI to read (e.g. 'file:///path', 'db://table/row')."
|
||||
}
|
||||
},
|
||||
"required": ["uri"]
|
||||
},
|
||||
"agent": true,
|
||||
"task_agent": true,
|
||||
"auto_approve": false,
|
||||
"primary_key": "uri"
|
||||
}
|
||||
@@ -0,0 +1,23 @@
|
||||
{
|
||||
"name": "use_prompt",
|
||||
"description": "Invoke an MCP prompt template by name, expanding it into messages. Returns the expanded prompt content that can be used as context or instructions.",
|
||||
"parameters": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"name": {
|
||||
"type": "string",
|
||||
"description": "The prompt name (e.g. 'mcp__server__prompt_name')."
|
||||
},
|
||||
"arguments": {
|
||||
"type": "object",
|
||||
"description": "Key-value argument pairs for the prompt.",
|
||||
"additionalProperties": { "type": "string" }
|
||||
}
|
||||
},
|
||||
"required": ["name"]
|
||||
},
|
||||
"agent": true,
|
||||
"task_agent": true,
|
||||
"auto_approve": false,
|
||||
"primary_key": "name"
|
||||
}
|
||||
@@ -42,6 +42,26 @@ function pollHealth() {
|
||||
.then(function (data) {
|
||||
pollHealth._failCount = 0;
|
||||
_lastHealth = data;
|
||||
var mcpEl = document.getElementById("mcp-status");
|
||||
if (mcpEl) {
|
||||
if (data.mcp && data.mcp.servers > 0) {
|
||||
mcpEl.textContent =
|
||||
"MCP: " +
|
||||
data.mcp.servers +
|
||||
" server" +
|
||||
(data.mcp.servers !== 1 ? "s" : "");
|
||||
mcpEl.title =
|
||||
data.mcp.resources +
|
||||
" resources \u00b7 " +
|
||||
data.mcp.prompts +
|
||||
" prompts";
|
||||
mcpEl.style.opacity = "1";
|
||||
} else {
|
||||
mcpEl.textContent = "";
|
||||
mcpEl.title = "";
|
||||
mcpEl.style.opacity = "0";
|
||||
}
|
||||
}
|
||||
var el = document.getElementById("health-indicator");
|
||||
if (!el) return;
|
||||
if (data.status === "degraded") {
|
||||
|
||||
@@ -28,6 +28,7 @@
|
||||
</div>
|
||||
<h1>turnstone</h1>
|
||||
<span id="model-name"></span>
|
||||
<span id="mcp-status" role="status" aria-live="polite"></span>
|
||||
<span id="status-bar"></span>
|
||||
<span id="health-indicator" class="health-ok" role="status" aria-live="polite" aria-atomic="true"></span>
|
||||
<button id="logout-btn" class="header-btn" onclick="logout()" style="display:none">logout</button>
|
||||
|
||||
@@ -12,6 +12,14 @@
|
||||
font-family: var(--font-display);
|
||||
letter-spacing: 0.02em;
|
||||
}
|
||||
#mcp-status {
|
||||
color: var(--magenta);
|
||||
font-size: 11px;
|
||||
font-family: var(--font-mono);
|
||||
opacity: 0;
|
||||
transition: opacity 0.3s;
|
||||
cursor: default;
|
||||
}
|
||||
#health-indicator {
|
||||
font-size: 11px;
|
||||
padding: 2px 8px;
|
||||
@@ -699,5 +707,6 @@
|
||||
.approval-btn, .approval-feedback-input,
|
||||
#plan-buttons button, #input-area button,
|
||||
.dashboard-new-btn, .dashboard-input,
|
||||
#health-indicator, #hamburger-btn { transition: none; }
|
||||
#health-indicator, #hamburger-btn,
|
||||
#mcp-status { transition: none; }
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user