mirror of
https://github.com/turnstonelabs/turnstone.git
synced 2026-08-12 23:12:23 -06:00
5a3f46a1fa
Wires the structured-error envelopes produced by Phase 7b's pool
dispatcher (mcp_consent_required / mcp_insufficient_scope /
mcp_*_forbidden / mcp_token_undecryptable_key_unknown /
mcp_oauth_url_insecure) through to the user-facing dashboard, and
adds a per-user settings panel for managing MCP server consents.
Changes
- ``_dispatch_pool_sync`` and ``_dispatch_pool_resource_sync`` wrap
structured-error string returns as ``RuntimeError(json_str)`` via
``_is_structured_error()`` so the session-layer ``except Exception``
branch fires uniformly across tool / resource / prompt dispatchers
(the prompt path's ``isinstance(result, str)`` shortcut works only
because prompts return ``list[dict]`` on success). Without this,
the consent UX silently does not render for tool / resource calls.
- ``_structured_error`` extended with an optional ``consent_url``
field; ``_build_consent_url`` produces ``/v1/api/mcp/oauth/start``
query strings (path-relative; the dashboard appends ``return_url``
at click time). Wired to all 12 ``mcp_consent_required`` and the
``mcp_insufficient_scope`` emit sites.
- New endpoints ``GET /v1/api/mcp/oauth/connections`` and
``DELETE /v1/api/mcp/oauth/connections/{server_name}`` registered
on both ``turnstone-server`` and ``turnstone-console``. The DELETE
handler runs local delete + audit + 204 first, then schedules the
RFC 7009 upstream revoke as a fire-and-forget ``asyncio.create_task``
with strong-ref tracking via ``_revoke_upstream_tasks`` (mirrors
the ``_pg_refresh_drain_tasks`` pattern). Soft cap of 256 concurrent
in-flight revokes prevents pile-up under coordinated mass-revoke;
the audit detail records ``upstream_revoke_outcome`` as
``scheduled | no_refresh_token | no_http_client | shed_by_cap``.
- ``ASMetadata`` extended with ``revocation_endpoint`` parsed from
RFC 8414 metadata. ``revoke_token_at_as`` helper posts the form
body under ``asyncio.timeout`` (not ``asyncio.wait_for``) and
never raises; ``_attempt_upstream_revoke`` is wrapped in an outer
``try/except Exception`` so unhandled exceptions don't surface as
``Task exception was never retrieved``.
- ``/v1/api/mcp/oauth/start`` accepts an optional ``scopes=`` query
param; tokens are validated against RFC 6749 §3.3 grammar via
``is_valid_scope_token`` (promoted to ``mcp_http_parsers``),
capped at ``_MAX_INSUFFICIENT_SCOPE_REPORTED`` (32), and unioned
with the configured server scopes for the step-up consent flow.
- Storage primitive ``list_mcp_user_token_metadata_by_user`` projects
the metadata columns at the SQL boundary so ciphertext blobs never
cross the wire on the settings-list path. New
``MCPUserTokenMetadataRow`` TypedDict in ``_protocol.py``;
``MCPTokenStore.list_user_token_metadata`` re-types to the existing
``MCPUserTokenMetadata`` shape.
- Dashboard renderer (``app.js``): ``tryParseMcpError`` detects the
envelope shape on ``tool_result`` SSE events with ``is_error=True``
and ``buildMcpErrorEmbed`` renders an action card mirroring the
existing ``buildMediaEmbed`` pattern. Three categories: actionable
(consent_required / insufficient_scope) with a ``Connect`` button
that opens ``/v1/api/mcp/oauth/start`` in a popup with a scheme
guard, forbidden (mcp_*_forbidden) with a static notice, operator
(key-mismatch / url-insecure) with an operator-action notice.
- New gear button in the appbar opens an MCP-connections settings
modal driven by ``loadMcpConnections`` / ``confirmRevokeMcp``
(two-step revoke confirmation matching the existing delete-ws
pattern). Pending-consent badge tracks unresolved consent prompts
in this tab; cleared after the connections list returns. Console
proxy collision-checked: the IIFE only prepends a node-id pill to
``header.firstChild``, so the right-anchored gear button is safe.
Bearer-leak invariant
- No ``exc_info=True`` on any new path that can carry a chained
``httpx.Request`` (revoke handler, dispatch sites, exec sites).
The two pre-existing ``exc_info=True`` calls in
``_exec_read_resource`` / ``_exec_use_prompt`` were replaced with
structured-field logs as a Phase 8 sibling fix.
Tests
- 440 pytest passes on both Python 3.13 (.venv) and 3.11
(/tmp/venv311); ruff + mypy clean.
- 5 new test files: ``test_mcp_consent_url_sibling_audit`` (structural
gate that every ``code="mcp_consent_required"`` / ``mcp_insufficient_scope``
site carries ``consent_url=``), ``test_mcp_oauth_connections``,
``test_mcp_oauth_revoke``, ``test_mcp_token_store_metadata``,
``test_session_mcp_dispatch_error``.
- End-to-end regression coverage for the bug-1 sibling pattern:
``test_call_tool_sync_raises_on_structured_error_envelope``,
``test_read_resource_sync_raises_on_structured_error_envelope``,
``test_get_prompt_sync_raises_on_structured_error_envelope``, plus
``test_call_tool_sync_does_not_wrap_non_structured_string`` as the
defensive gate (only ``mcp_*`` envelopes are wrapped).
Hard invariants honored
- Static path byte-identical for ``auth_type ∈ {none, static}``: the
wrap fires only when the dispatcher returns a structured-mcp-error
string, which only happens on the oauth_user pool path.
- ``asyncio.timeout`` (not ``asyncio.wait_for``) on every new
AS / SDK / pool-loop await per Python 3.11 anyio cancel-scope
hazard.
- Scope cap ``_MAX_INSUFFICIENT_SCOPE_REPORTED = 32`` enforced at
every output / merge site.
- Cross-user isolation on the revoke endpoint: a non-owner DELETE
returns 404 with the same body shape as a never-existed row;
``http_client_mock.post.assert_not_called()`` pins this in 3 tests.
Deferred (not Phase 8 blockers)
- perf-2 (``asyncio.gather`` parallelisation in revoke handler) —
superseded by perf-1's fire-and-forget pattern.
- q-4 (prompt-path ``isinstance(str)`` vs sibling ``_is_structured_error``
asymmetry) — already documented in the function docstring.
- q-9 (``_pendingConsentServers`` → ``_serversNeedingConsent``
rename) — pure naming taste.
213 lines
8.1 KiB
Python
213 lines
8.1 KiB
Python
"""Storage CRUD tests for the per-(user, server) MCP OAuth pending-state table.
|
|
|
|
Validates the storage-protocol additions for the per-(user, server)
|
|
OAuth flow:
|
|
|
|
- ``create_mcp_oauth_pending_state``
|
|
- ``pop_mcp_oauth_pending_state`` (atomic, with TTL)
|
|
- ``cleanup_expired_mcp_oauth_pending_states``
|
|
- ``get_mcp_oauth_client_secret_ct``
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
import sqlalchemy as sa
|
|
|
|
|
|
class TestCreateAndPop:
|
|
def test_round_trip(self, backend) -> None:
|
|
backend.create_mcp_oauth_pending_state(
|
|
"state-1",
|
|
"user-a",
|
|
"srv-x",
|
|
"verifier-blob",
|
|
"/admin/mcp-servers",
|
|
)
|
|
row = backend.pop_mcp_oauth_pending_state("state-1", max_age_seconds=600)
|
|
assert row is not None
|
|
assert row["state"] == "state-1"
|
|
assert row["user_id"] == "user-a"
|
|
assert row["server_name"] == "srv-x"
|
|
assert row["code_verifier"] == "verifier-blob"
|
|
assert row["return_url"] == "/admin/mcp-servers"
|
|
|
|
def test_pop_consumes_row(self, backend) -> None:
|
|
backend.create_mcp_oauth_pending_state("s2", "u", "s", "v", "/r")
|
|
first = backend.pop_mcp_oauth_pending_state("s2")
|
|
assert first is not None
|
|
# Second pop must miss — row was consumed.
|
|
second = backend.pop_mcp_oauth_pending_state("s2")
|
|
assert second is None
|
|
|
|
def test_pop_missing_returns_none(self, backend) -> None:
|
|
assert backend.pop_mcp_oauth_pending_state("never-existed") is None
|
|
|
|
|
|
class TestTTL:
|
|
def test_pop_rejects_expired_row(self, backend) -> None:
|
|
backend.create_mcp_oauth_pending_state("old-state", "u", "s", "v", "/r")
|
|
# Backdate it so it's older than the TTL window.
|
|
with backend._engine.connect() as conn:
|
|
conn.execute(
|
|
sa.text(
|
|
"UPDATE mcp_oauth_pending SET created_at = '2020-01-01T00:00:00' "
|
|
"WHERE state = 'old-state'"
|
|
)
|
|
)
|
|
conn.commit()
|
|
|
|
# Default TTL is 600s — the row is decades old.
|
|
row = backend.pop_mcp_oauth_pending_state("old-state")
|
|
assert row is None
|
|
|
|
# Even though pop returned None, the row must have been wiped — a
|
|
# second pop with a giant TTL must still see nothing.
|
|
again = backend.pop_mcp_oauth_pending_state("old-state", max_age_seconds=10**9)
|
|
assert again is None
|
|
|
|
def test_pop_accepts_fresh_row(self, backend) -> None:
|
|
backend.create_mcp_oauth_pending_state("fresh", "u", "s", "v", "/r")
|
|
row = backend.pop_mcp_oauth_pending_state("fresh", max_age_seconds=600)
|
|
assert row is not None
|
|
assert row["state"] == "fresh"
|
|
|
|
|
|
class TestCleanup:
|
|
def test_cleanup_deletes_only_expired(self, backend) -> None:
|
|
backend.create_mcp_oauth_pending_state("old", "u", "s", "v", "/r")
|
|
backend.create_mcp_oauth_pending_state("new", "u", "s", "v", "/r")
|
|
with backend._engine.connect() as conn:
|
|
conn.execute(
|
|
sa.text(
|
|
"UPDATE mcp_oauth_pending SET created_at = '2020-01-01T00:00:00' "
|
|
"WHERE state = 'old'"
|
|
)
|
|
)
|
|
conn.commit()
|
|
|
|
deleted = backend.cleanup_expired_mcp_oauth_pending_states(max_age_seconds=600)
|
|
assert deleted == 1
|
|
# Old gone, new still around.
|
|
assert backend.pop_mcp_oauth_pending_state("old") is None
|
|
survivor = backend.pop_mcp_oauth_pending_state("new")
|
|
assert survivor is not None
|
|
|
|
def test_cleanup_no_rows(self, backend) -> None:
|
|
assert backend.cleanup_expired_mcp_oauth_pending_states() == 0
|
|
|
|
|
|
class TestGetOAuthClientSecretCt:
|
|
def test_returns_none_when_unset(self, backend) -> None:
|
|
backend.create_mcp_server(
|
|
server_id="srv-id",
|
|
name="srv-x",
|
|
transport="streamable-http",
|
|
url="https://mcp.example.com/sse",
|
|
auth_type="oauth_user",
|
|
)
|
|
assert backend.get_mcp_oauth_client_secret_ct("srv-id") is None
|
|
|
|
def test_returns_ciphertext_after_set(self, backend) -> None:
|
|
backend.create_mcp_server(
|
|
server_id="srv-id",
|
|
name="srv-x",
|
|
transport="streamable-http",
|
|
url="https://mcp.example.com/sse",
|
|
auth_type="oauth_user",
|
|
)
|
|
ct = b"\x00\xff\x42encrypted-blob"
|
|
ok = backend.set_mcp_oauth_client_secret_ct("srv-id", ct)
|
|
assert ok is True
|
|
out = backend.get_mcp_oauth_client_secret_ct("srv-id")
|
|
assert out == ct
|
|
|
|
def test_returns_none_for_missing_server(self, backend) -> None:
|
|
assert backend.get_mcp_oauth_client_secret_ct("does-not-exist") is None
|
|
|
|
|
|
def _create_user_token_row(
|
|
backend,
|
|
*,
|
|
user_id: str,
|
|
server_name: str,
|
|
created: str,
|
|
) -> None:
|
|
"""Insert a token row + backdate ``created`` so ordering is deterministic.
|
|
|
|
The storage helper stamps ``created`` from ``datetime.now(UTC)``; for
|
|
multi-row ordering tests we backdate via raw SQL so the inserts stay
|
|
independent of clock resolution.
|
|
"""
|
|
backend.create_mcp_user_token(
|
|
user_id,
|
|
server_name,
|
|
access_token_ct=b"ct-access",
|
|
refresh_token_ct=b"ct-refresh",
|
|
expires_at="2026-05-04T12:00:00",
|
|
scopes="openid",
|
|
as_issuer="https://auth.example.com",
|
|
audience="https://mcp.example.com",
|
|
)
|
|
with backend._engine.connect() as conn:
|
|
conn.execute(
|
|
sa.text(
|
|
"UPDATE mcp_user_tokens SET created = :created "
|
|
"WHERE user_id = :uid AND server_name = :sn"
|
|
),
|
|
{"created": created, "uid": user_id, "sn": server_name},
|
|
)
|
|
conn.commit()
|
|
|
|
|
|
class TestListMCPUserTokenMetadataByUser:
|
|
def test_list_mcp_user_token_metadata_by_user_empty(self, backend) -> None:
|
|
assert backend.list_mcp_user_token_metadata_by_user("nobody") == []
|
|
|
|
def test_list_mcp_user_token_metadata_by_user_single_server(self, backend) -> None:
|
|
_create_user_token_row(
|
|
backend, user_id="u1", server_name="srv-a", created="2026-05-01T00:00:00"
|
|
)
|
|
rows = backend.list_mcp_user_token_metadata_by_user("u1")
|
|
assert len(rows) == 1
|
|
assert rows[0]["user_id"] == "u1"
|
|
assert rows[0]["server_name"] == "srv-a"
|
|
assert rows[0]["as_issuer"] == "https://auth.example.com"
|
|
assert rows[0]["audience"] == "https://mcp.example.com"
|
|
assert rows[0]["scopes"] == "openid"
|
|
# Projection MUST omit ciphertext columns — the SQL no longer
|
|
# selects them, so the TypedDict has no key.
|
|
assert "access_token_ct" not in rows[0]
|
|
assert "refresh_token_ct" not in rows[0]
|
|
|
|
def test_list_mcp_user_token_metadata_by_user_multiple_servers(self, backend) -> None:
|
|
_create_user_token_row(
|
|
backend, user_id="u1", server_name="srv-c", created="2026-05-03T00:00:00"
|
|
)
|
|
_create_user_token_row(
|
|
backend, user_id="u1", server_name="srv-a", created="2026-05-01T00:00:00"
|
|
)
|
|
_create_user_token_row(
|
|
backend, user_id="u1", server_name="srv-b", created="2026-05-02T00:00:00"
|
|
)
|
|
rows = backend.list_mcp_user_token_metadata_by_user("u1")
|
|
assert [r["server_name"] for r in rows] == ["srv-a", "srv-b", "srv-c"]
|
|
|
|
def test_list_mcp_user_token_metadata_by_user_isolates_by_user(self, backend) -> None:
|
|
_create_user_token_row(
|
|
backend, user_id="user-a", server_name="srv-a", created="2026-05-01T00:00:00"
|
|
)
|
|
_create_user_token_row(
|
|
backend, user_id="user-a", server_name="srv-b", created="2026-05-02T00:00:00"
|
|
)
|
|
_create_user_token_row(
|
|
backend, user_id="user-b", server_name="srv-a", created="2026-05-03T00:00:00"
|
|
)
|
|
rows_a = backend.list_mcp_user_token_metadata_by_user("user-a")
|
|
assert {r["server_name"] for r in rows_a} == {"srv-a", "srv-b"}
|
|
assert all(r["user_id"] == "user-a" for r in rows_a)
|
|
|
|
rows_b = backend.list_mcp_user_token_metadata_by_user("user-b")
|
|
assert len(rows_b) == 1
|
|
assert rows_b[0]["user_id"] == "user-b"
|
|
assert rows_b[0]["server_name"] == "srv-a"
|