mirror of
https://github.com/turnstonelabs/turnstone.git
synced 2026-08-13 15:32:24 -06:00
Compare commits
32 Commits
v1.5.13
...
stable/1.5
| Author | SHA1 | Date | |
|---|---|---|---|
| 4a38b835f5 | |||
| 415be00149 | |||
| 346ad2a6aa | |||
| 8003fcbebe | |||
| d4f3711d63 | |||
| cea229206b | |||
| b7b4dcc0df | |||
| 97080e1df9 | |||
| af0cbaaec3 | |||
| b9ce0d388e | |||
| d07d2242aa | |||
| 70514cc406 | |||
| 687a3367c0 | |||
| bfd99c6a81 | |||
| 1a813c8130 | |||
| d6aa85db6d | |||
| c0c7fda7f9 | |||
| 7fae2698d7 | |||
| 4bbe64755e | |||
| c1281b9721 | |||
| 3124dbe52f | |||
| a8a1e738ca | |||
| 27768abecc | |||
| 1e5017293a | |||
| b2c688f3b1 | |||
| 09b1f07e18 | |||
| 1de8f6b4c7 | |||
| 4f89c1c3f3 | |||
| 6898860f18 | |||
| 3d07cad272 | |||
| 3737ddf89f | |||
| 464450b9e2 |
+164
@@ -14,6 +14,170 @@ Three release tracks are maintained:
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
## [1.5.18]
|
||||
|
||||
Backports the `turnstone-admin` config-loading alignment from `main`
|
||||
plus the accompanying `load_config` permission-warning hardening. No
|
||||
schema changes.
|
||||
|
||||
### Added
|
||||
|
||||
- **`turnstone-admin` reads `config.toml`** — the admin CLI now honors
|
||||
the same `[database]` section that `turnstone-server` does, with the
|
||||
same precedence (`CLI / config.toml > TURNSTONE_DB_* env > defaults`).
|
||||
Operators with DB credentials in `config.toml` no longer need to
|
||||
re-export `TURNSTONE_DB_URL` before every admin invocation. Newly
|
||||
plumbed through to `init_storage`: `pool_size`, `sslmode`,
|
||||
`sslrootcert`, `sslcert`, `sslkey` — previously the admin CLI
|
||||
silently dropped these. A new `--config PATH` flag mirrors the
|
||||
one already on `turnstone-server`.
|
||||
|
||||
### Security
|
||||
|
||||
- **Permissive `config.toml` now warns** — `turnstone.core.config.load_config`
|
||||
logs a single warning when the resolved config file is group- or
|
||||
world-readable (any bit in `0o077`). DB password and TLS key paths
|
||||
live in `[database]`; operators usually want the file at `0600`.
|
||||
|
||||
## [1.5.17]
|
||||
|
||||
Backports a clutch of coordinator-tool clarity fixes plus a watch-delivery
|
||||
correctness fix from `main` to the `stable/1.5` track, plus a previously-
|
||||
latent intent-verdicts persistence bug exposed by the new heuristic-verdict
|
||||
INSERT paths. No schema changes.
|
||||
|
||||
### Fixed
|
||||
|
||||
- **`intent_verdicts` PK collisions on every llm_fallback delivery** —
|
||||
async LLM-tier "llm_fallback" verdicts (`turnstone/core/judge.py` —
|
||||
`_deliver_fallbacks` and the in-loop fallback path) deliberately
|
||||
reuse the heuristic verdict's `verdict_id` so the row gets
|
||||
"upgraded in place" from `tier="heuristic"` → `tier="llm_fallback"`
|
||||
when the LLM judge times out, is cancelled, or returns no content.
|
||||
The consumer `_persist_intent_verdict` was doing a plain INSERT,
|
||||
hitting the `intent_verdicts_pkey` constraint on every fallback
|
||||
delivery; Postgres logged the duplicate-key error, the application
|
||||
try/except swallowed it at `log.debug`, and the row never actually
|
||||
got upgraded — the LLM judge's annotation
|
||||
(`"(LLM judge did not return a verdict)"`) was lost. The collision
|
||||
rate exploded on this release because the new heuristic-INSERT
|
||||
paths in the auto-approve early-return branches of `approve_tools`
|
||||
(introduced below) leave no gap for the fallback to land cleanly
|
||||
into. Fix: new `upsert_intent_verdict` storage method using
|
||||
`ON CONFLICT (verdict_id) DO UPDATE` that updates only `tier`,
|
||||
`reasoning`, `judge_model` — the three fields that genuinely
|
||||
change between heuristic and llm_fallback. Every other column
|
||||
(identity, carried-verbatim, and `user_decision`) is excluded;
|
||||
`user_decision` in particular would otherwise be clobbered back
|
||||
to `"pending"` when a fallback arrives after the operator has
|
||||
already resolved the approval. The bulk-INSERT path stays as
|
||||
plain INSERT — fresh UUIDs in `judge.evaluate` make in-turn dups
|
||||
impossible; the inverse race (fallback wins before bulk lands) is
|
||||
reachable but unchanged in observable behavior by this fix,
|
||||
documented at the bulk site for a future hardening pass.
|
||||
- **Coordinator LLM re-spawn loops on large fan-outs** — the spawn-tool
|
||||
return JSON used `ws_id` as its key, which primed the model's recency
|
||||
bias to feed the spawn result straight back into another
|
||||
`spawn_workstream(ws_id=...)` call instead of progressing to
|
||||
`wait_for_workstream(ws_ids=[...])`. On 10+ child fan-outs this cascaded
|
||||
into self-inflicted re-spawn loops. The LLM-facing tool result now emits
|
||||
`child_ws_id` (the storage column / HTTP API contract is unchanged); the
|
||||
field name is already an existing project term so the rename aligns
|
||||
rather than introduces new vocabulary. Also handles the silent
|
||||
upstream-omits-ws_id success-shape edge that previously emitted
|
||||
`{"child_ws_id": null}` to the LLM — now surfaces a tool error so the
|
||||
model retries rather than chasing a null id.
|
||||
- **`inspect_workstream` blowing the coordinator context budget** — a
|
||||
coord doing a fan-out wave against tool-heavy children could land
|
||||
>100 KB of raw output per inspect call, and the previous safety net
|
||||
(`_truncate_output`'s head+tail strategy) silently dropped *middle*
|
||||
messages — exactly the wrong shape for understanding a child's
|
||||
trajectory (the FIRST sets the brief, the LAST shows the conclusion,
|
||||
the middle is the connective tissue). Output now goes through a
|
||||
three-tier degradation ladder mirroring the search tool's
|
||||
`_format_search_results`: `_tier="full"` (every message verbatim) →
|
||||
`_tier="compact"` (per-message head/tail-snipped content + snipped
|
||||
`tool_calls.arguments`, falling through a `(20,30)` / `(10,20)` /
|
||||
`(5,10)` message-list trim ladder) → `_tier="skeleton"` (counts, role
|
||||
distribution, last-assistant preview). Budget 32 KiB matches the
|
||||
search tool's; the chosen tier is annotated on the response so the
|
||||
model can recall with a tighter `message_limit` if signal was lost.
|
||||
- **Auto-approved verdicts indistinguishable from pending review** —
|
||||
`intent_verdict` rows for auto-approved tool calls landed with
|
||||
`user_decision=""`, which read identically to "still waiting for the
|
||||
operator" in the audit trail and led to a real misdiagnosis incident.
|
||||
The column now carries an explicit vocabulary at insert: `pending` /
|
||||
`approved` / `denied` / `timeout` / `policy` / `blanket` / `skill` /
|
||||
`always` / `auto_approve_tools`. The auto-approve early-return
|
||||
branches in `approve_tools` now persist heuristic verdicts stamped
|
||||
with their reason (previously dropped on the floor), and late LLM-tier
|
||||
verdicts that arrive for an already-auto-approved call_id are stamped
|
||||
via a TTL-pruned lookup map — so the audit row carries the
|
||||
auto-approve reason even when the LLM judge daemon completes after
|
||||
the synchronous approval cycle finished. `resolve_approval` gains a
|
||||
`timeout` kwarg writing `"timeout"` (the previous shape collapsed
|
||||
passive timeouts and active denials into the same column).
|
||||
- **`list_skills` empty `allowed_tools` misread as "no tool access"** —
|
||||
the response previously emitted `"allowed_tools": []` for every skill
|
||||
that hadn't declared an auto-approve allowlist, which a coordinator
|
||||
model read as "this skill can't use any tools" (real misdiagnosis: a
|
||||
code-review child appeared to have been spawned with zero tool
|
||||
access). The field is now omitted entirely when empty — absence
|
||||
carries the unambiguous meaning "no tool is pre-approved for this
|
||||
skill", presence (non-empty list) keeps the standard Claude Code
|
||||
skill-spec shape. The tool description rewrite makes the
|
||||
auto-approve-allowlist semantics explicit so a future reader doesn't
|
||||
re-derive the gating misread.
|
||||
- **Watch terminal-fires silently dropped on backpressure** —
|
||||
delivery now routes terminal events through the same path as
|
||||
normal fires instead of being filtered out when the consumer was
|
||||
saturated.
|
||||
|
||||
### Documentation
|
||||
|
||||
- **Storage `LIKE_ESCAPE` contract** — clarify that callers passing
|
||||
`.like(escape=...)` must use the same escape character that the
|
||||
storage helper assumes; previous wording let a reader pass a
|
||||
different escape and silently produce no matches.
|
||||
|
||||
## [1.5.15]
|
||||
|
||||
### Fixed
|
||||
|
||||
- **Admin console blank-page on MCP server rows with consented users** — a
|
||||
Phase 9 (1.5.14) regression in `admin.js` used double-quote string
|
||||
delimiters on the bulk-revoke button HTML literal, but the literal embeds
|
||||
a `"` mid-attribute. JS closed the string early, turned `bulk-revoke (`
|
||||
into bare tokens, and the resulting `SyntaxError` wiped out every global
|
||||
in `admin.js` — `showAdmin` and all other admin entry points became
|
||||
undefined, so the console UI was non-functional whenever the rendered MCP
|
||||
server list contained at least one row with `consented_users_count > 0`.
|
||||
Switch the literal to single-quote delimiters to match the surrounding
|
||||
block.
|
||||
|
||||
## [1.5.14]
|
||||
|
||||
Backports OAuth-MCP Phase 9 from `main` to the `stable/1.5` track.
|
||||
|
||||
### Added
|
||||
|
||||
- **OAuth-MCP Phase 9 — admin status, deferred-consent persistence, operator
|
||||
docs** — completes the per-(user, server) OAuth-MCP build-out. The sync pool
|
||||
dispatchers now upsert into a new `mcp_pending_consent` table on
|
||||
`mcp_consent_required` / `mcp_insufficient_scope`, so a non-interactive run
|
||||
(scheduled / channel) that hits an unconsented server surfaces the deferred
|
||||
prompt to the user on their next dashboard load via the gear-icon badge —
|
||||
rows are cleared automatically by the OAuth callback handler on consent
|
||||
completion, or via new DELETE endpoints for manual dismiss. The MCP Servers
|
||||
admin row gains a `consented_users_count` pill and a two-step-confirm
|
||||
bulk-revoke button for `auth_type=oauth_user` servers (upstream RFC 7009
|
||||
revoke is intentionally not attempted in bulk to avoid N synchronous
|
||||
round-trips against the provider). Operator-facing docs land at
|
||||
`docs/mcp-oauth.md` and `docs/operations/mcp-oauth-headless.md`.
|
||||
|
||||
Introduces forward-only migrations `054_mcp_pending_consent` and
|
||||
`055_mcp_user_tokens_server_index`.
|
||||
|
||||
## [1.5.13]
|
||||
|
||||
This release introduces one forward-only schema migration:
|
||||
|
||||
+1
-1
@@ -8,7 +8,7 @@ FROM python:3.14-slim
|
||||
LABEL org.opencontainers.image.title="turnstone" \
|
||||
org.opencontainers.image.description="Multi-node AI orchestration platform"
|
||||
|
||||
COPY --from=ghcr.io/astral-sh/uv:0.11.8 /uv /usr/local/bin/uv
|
||||
COPY --from=ghcr.io/astral-sh/uv:0.11.14 /uv /usr/local/bin/uv
|
||||
|
||||
# Remove the slim image's man page exclusion so man-db has actual content
|
||||
RUN rm -f /etc/dpkg/dpkg.cfg.d/docker
|
||||
|
||||
@@ -91,7 +91,7 @@ turnstone/
|
||||
discord/ Discord adapter (bot, cog, views, streaming, config)
|
||||
slack/ Slack adapter (Socket Mode bot, DM routing, approval buttons)
|
||||
shared_static/ Shared design system (base.css, auth.js, theme.js, toast.js, utils.js, kb.js)
|
||||
katex-0.16.45/ Vendored KaTeX math rendering library (MIT, woff2 fonts)
|
||||
katex-0.16.47/ Vendored KaTeX math rendering library (MIT, woff2 fonts)
|
||||
ui/
|
||||
colors.py ANSI color constants with NO_COLOR support
|
||||
markdown.py Streaming terminal markdown renderer (line-buffered)
|
||||
|
||||
@@ -110,11 +110,18 @@ owns it; the node is just currently unreachable.
|
||||
|
||||
### Example — `spawn_batch`
|
||||
|
||||
This is the coordinator-tool result shape (the JSON the LLM receives),
|
||||
not an HTTP API response — the table above keys it under "model tool"
|
||||
to distinguish it from the `/v1/api/...` endpoints in the same table.
|
||||
The underlying HTTP spawn endpoint still returns `ws_id`; the tool
|
||||
result re-keys it to `child_ws_id` to defuse a coordinator-LLM recency
|
||||
bias (see `docs/coordinator-skills.md`).
|
||||
|
||||
```json
|
||||
{
|
||||
"results": {
|
||||
"0": {"ws_id": "d4e5f6...", "name": "csrf-audit", "node_id": "gpu-3"},
|
||||
"2": {"ws_id": "f1a2b3...", "name": "xss-audit", "node_id": "gpu-1"}
|
||||
"0": {"child_ws_id": "d4e5f6...", "name": "csrf-audit", "node_id": "gpu-3"},
|
||||
"2": {"child_ws_id": "f1a2b3...", "name": "xss-audit", "node_id": "gpu-1"}
|
||||
},
|
||||
"denied": [
|
||||
{"idx": 1, "reason": "skill not found: nonexistent-skill"}
|
||||
|
||||
@@ -169,14 +169,21 @@ validates ws_id against `parent_ws_id=coord_ws_id` AND
|
||||
the wait into reporting "complete".
|
||||
|
||||
Pattern: capture each spawn result in the next tool call's input.
|
||||
The JSON tool-result carries `{"ws_id": "...", "name": "...",
|
||||
The JSON tool-result carries `{"child_ws_id": "...", "name": "...",
|
||||
"node_id": "...", "routing_strategy": "..."}`; the model should
|
||||
extract the ws_id and pass it to `inspect_workstream` /
|
||||
`wait_for_workstream` / `send_to_workstream` / `close_workstream`
|
||||
verbatim.
|
||||
extract the `child_ws_id` and pass it as `ws_id` (or in the `ws_ids`
|
||||
list) to `inspect_workstream` / `wait_for_workstream` /
|
||||
`send_to_workstream` / `close_workstream` verbatim. The asymmetry
|
||||
— spawn returns `child_ws_id` but the other tools accept `ws_id` /
|
||||
`ws_ids` — is intentional: it defuses a coordinator-LLM recency
|
||||
bias where seeing `ws_id` in a spawn return primed re-spawn loops
|
||||
instead of progression to the wait phase.
|
||||
|
||||
A UI that wants human-readable identifiers should render the `name`
|
||||
field and keep the ws_id as the click-through key.
|
||||
field and keep the workstream id as the click-through key — note
|
||||
that the id *value* is the same regardless of whether it arrived
|
||||
under the `child_ws_id` key (spawn return) or the `ws_id` key
|
||||
(every other tool's input/output); only the field name differs.
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -0,0 +1,117 @@
|
||||
# MCP OAuth — per-user authorization for MCP servers
|
||||
|
||||
Turnstone supports **per-(user, MCP server) OAuth 2.1 + PKCE** delegation so each Turnstone user authorizes a remote MCP server with their own identity, rather than sharing a single bearer token across the deployment. This is the right shape for MCP servers that expose user-specific data (a personal CRM, an email inbox, a calendar) and for MCP servers that want per-user audit attribution.
|
||||
|
||||
Per-user OAuth is opt-in per `mcp_servers` row. Local-auth Turnstone installs with no `oauth_user` rows exercise zero new code paths — the entire feature is dark by default.
|
||||
|
||||
> **Note**: This is a separate authorization layer from Turnstone's own user authentication. A user who logs into Turnstone with a local username + password can still authorize a per-server OAuth MCP server. OIDC SSO and per-server OAuth are orthogonal.
|
||||
|
||||
---
|
||||
|
||||
## When to use which `auth_type`
|
||||
|
||||
The MCP server admin form exposes three authorization modes ("Multitenant Authorization"):
|
||||
|
||||
| `auth_type` | What it means | When to use |
|
||||
|---|---|---|
|
||||
| `none` | No headers attached. Open MCP server (or one gated by network policy only). | Internal MCP servers on a trusted network. |
|
||||
| `static` | One static bearer token, configured per server, sent on every request from every user. | Service-to-service MCP servers where per-user attribution doesn't matter, or single-tenant deployments. |
|
||||
| `oauth_user` *(recommended for user-data servers)* | Each user authorizes separately via OAuth 2.1 + PKCE; Turnstone stores per-user tokens encrypted at rest. | MCP servers that expose user-specific data or that want per-user audit attribution. |
|
||||
|
||||
Switching `auth_type` away from `oauth_user` orphans existing per-user tokens. Use the admin **bulk-revoke** affordance on the server row (Phase 9) to clear them, or let them expire naturally — they're inert without the matching `auth_type` value.
|
||||
|
||||
---
|
||||
|
||||
## Prerequisites for `auth_type=oauth_user`
|
||||
|
||||
1. **Encryption key**. Tokens are stored encrypted with Fernet. Set `[security] mcp_token_encryption_key` in `config.toml` (Turnstone won't start with an `oauth_user` row configured but no key installed). Rotate via `MultiFernet` — add the new key first, then later remove the old one once all rows have been re-encrypted.
|
||||
|
||||
2. **MCP server publishes RFC 9728 PRM and RFC 8414 AS metadata** *or* you configure the AS URL override on the server row. PKCE S256 is mandatory; Turnstone refuses to connect to authorization servers that don't advertise `code_challenge_methods_supported: ["S256"]`.
|
||||
|
||||
3. **OAuth client registration**. Two paths:
|
||||
- **Pre-registered** (most common): you create an OAuth client at the authorization server (manually, via admin console, or via Terraform), then paste the `client_id` / `client_secret` into the Turnstone admin form.
|
||||
- **Dynamic client registration** (RFC 7591): if the AS supports it and you select that mode in the admin form, Turnstone registers a client at first use and persists the `client_id` automatically.
|
||||
|
||||
4. **Redirect URI** registered at the authorization server: `https://your-turnstone-host/v1/api/mcp/oauth/callback`.
|
||||
|
||||
---
|
||||
|
||||
## Configuration
|
||||
|
||||
### Per-server fields (admin UI)
|
||||
|
||||
| Field | Required | Description |
|
||||
|---|---|---|
|
||||
| Server URL | Yes | The MCP server's `streamable-http` base URL. |
|
||||
| Multitenant Authorization | Yes | `none` / `static` / `oauth_user` (recommended). |
|
||||
| Authorization Server URL | No | Override for RFC 9728 PRM discovery. Set when your AS endpoint differs from the MCP server URL (e.g., corporate AS protecting a third-party MCP). When unset, Turnstone falls back to PRM discovery against the MCP server itself. |
|
||||
| Client Registration | Yes (oauth_user) | `preregistered` or `dynamic`. |
|
||||
| Client ID | Yes (preregistered) | OAuth 2.0 client ID. Stored unencrypted. |
|
||||
| Client Secret | Optional (write-only) | OAuth 2.0 client secret (confidential client). Encrypted at rest. Written but never re-read by the API; field stays masked. |
|
||||
| Scopes | No | Space-separated default scope set requested at the authorize endpoint. Per-tool step-up may union additional scopes from a server's `insufficient_scope` response. |
|
||||
| Audience | No | RFC 8707 `resource=` parameter sent on every authorize and token request. Defaults to the MCP server URL when unset. Validate against the `aud` claim in returned JWT tokens. |
|
||||
|
||||
### Encryption key
|
||||
|
||||
```toml
|
||||
[security]
|
||||
mcp_token_encryption_key = "base64-fernet-key"
|
||||
# For rotation, list the keys in priority order — first is used for new
|
||||
# writes, all are tried for reads.
|
||||
# mcp_token_encryption_keys = ["new-key", "old-key"]
|
||||
```
|
||||
|
||||
Keep this in `config.toml` rather than environment variables. An in-process LLM with shell-tool access can read the server's environment via `env` / `os.environ` and exfiltrate any secret stored there; secrets in `config.toml` are only loaded into the server at startup and never re-read on a tool-driven path, so a prompt-injection attack against the agent cannot reach them.
|
||||
|
||||
---
|
||||
|
||||
## Lifecycle
|
||||
|
||||
1. **First tool call** for a user against an `oauth_user` MCP server: pool dispatch finds no stored token, returns `mcp_consent_required` to the agent. Dashboard renders an inline "Connect" action card.
|
||||
|
||||
2. **User clicks Connect**: opens `/v1/api/mcp/oauth/start?server=<name>` in a popup. Browser redirects through the AS authorize endpoint, user grants consent, AS redirects back to `/v1/api/mcp/oauth/callback`. Turnstone exchanges code → tokens via PKCE, validates audience, encrypts, persists in `mcp_user_tokens`, redirects user back to the originating URL.
|
||||
|
||||
3. **Subsequent tool calls** by the same user against the same server reuse the persisted token via the per-(user, server) session pool. Tokens auto-refresh via the refresh-token grant when expired; failed refresh emits `mcp_consent_required` to drive re-consent.
|
||||
|
||||
4. **Step-up scope**: when a tool call hits `403` with `WWW-Authenticate: error="insufficient_scope"`, Turnstone emits `mcp_insufficient_scope` with the parsed scope set; the dashboard offers a "Connect with additional scopes" affordance that opens `/v1/api/mcp/oauth/start?server=<name>&scopes=<extra>` so the union of original + new scopes flows into the AS authorize request.
|
||||
|
||||
5. **User revoke** (settings modal): `DELETE /v1/api/mcp/oauth/connections/{server_name}` runs the authoritative local delete + best-effort RFC 7009 upstream revoke (fire-and-forget, capped at 256 concurrent in-flight tasks).
|
||||
|
||||
6. **Admin bulk-revoke** (Phase 9): `POST /v1/api/admin/mcp-servers/{name}/bulk-revoke` drops every user's token for the server. Upstream RFC 7009 revoke is intentionally **not** attempted in bulk (avoids N upstream HTTP calls per admin click); tokens at the AS expire naturally. Use the per-user revoke endpoint if you need guaranteed upstream invalidation.
|
||||
|
||||
---
|
||||
|
||||
## Admin status indicators
|
||||
|
||||
The MCP Servers admin tab shows per-server status pills (Phase 9):
|
||||
|
||||
- **Consented users count** — distinct users with a non-expired token for this server. Surfaced as a `bulk-revoke (N)` button when ≥1; clicking it opens a confirmation dialog. Hidden when 0.
|
||||
- **Last refresh** — timestamp + outcome (`ok` / `error:ClassName`) of the most recent manual or auto-reconnect refresh. Per node. Absent until at least one refresh has occurred (renders as "never" in the admin UI).
|
||||
|
||||
Additional indicators (circuit-breaker state, encryption-key mismatch) are exposed via `get_server_status` on the API but do not yet have a dedicated admin pill — operators see them today via the per-server status text + error tooltip and in audit logs. A future phase may surface these as discrete pills.
|
||||
|
||||
---
|
||||
|
||||
## Auth-type transitions
|
||||
|
||||
| From | To | What happens |
|
||||
|---|---|---|
|
||||
| `none` / `static` → `oauth_user` | — | New code path activates for this server. Existing static headers (if any) are no longer sent. Users must authorize on first use. |
|
||||
| `oauth_user` → `none` / `static` | — | Existing `mcp_user_tokens` rows are **orphaned** — inert without a matching `auth_type`. Use admin bulk-revoke to drop them, or let them expire. Switching back to `oauth_user` later re-activates the orphaned rows if they haven't been deleted. |
|
||||
| OAuth `client_id` or `client_secret` rotated | — | Existing tokens may stop refreshing if the AS treats them as bound to the previous client. Bulk-revoke after rotation. |
|
||||
|
||||
The orphan-by-default behavior is chosen so switching back to `oauth_user` is non-destructive. Bulk-revoke is the explicit cleanup path.
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
| Symptom | Likely cause | Action |
|
||||
|---|---|---|
|
||||
| `mcp_consent_required` even after consenting | Token persistence failed, or refresh-token rejected by AS | Check audit log for `mcp_server.oauth.persist_failed` or `mcp_server.oauth.token_revoked`. Re-consent via settings modal. |
|
||||
| `mcp_token_undecryptable_key_unknown` | Encryption key rotated without keeping the previous key in the keyring | Add the previous key back to `mcp_token_encryption_keys` until all rows have been re-encrypted, then drop. |
|
||||
| `mcp_oauth_url_insecure` | MCP server URL is `http://` (not `https://`) on a non-loopback host | Use `https://`. Per-user bearers must not transit cleartext. |
|
||||
| Tools fail in scheduled / Discord / Slack runs | OAuth-MCP requires browser-based consent | Users must pre-consent via the web UI. Phase 9 dashboard badge surfaces deferred consents from these runs on next login. |
|
||||
| Circuit breaker open repeatedly | Transport-level errors on the MCP server (DNS, TLS, 5xx) | Check the per-server error pill; auth errors do not trip the breaker. |
|
||||
|
||||
See also: `docs/operations/mcp-oauth-headless.md` for the cron / channel-driven run caveat.
|
||||
@@ -0,0 +1,29 @@
|
||||
# MCP OAuth in headless / scheduled / channel-driven runs
|
||||
|
||||
**Constraint**: OAuth-MCP servers (`auth_type=oauth_user`) require browser-based user consent. Users must pre-consent via the web UI before any run that cannot drive a browser redirect.
|
||||
|
||||
**Affected surfaces**:
|
||||
|
||||
- Scheduled workstreams (`turnstone-console` task scheduler).
|
||||
- Discord adapter runs.
|
||||
- Slack adapter runs.
|
||||
- Any future channel adapter without an interactive browser session.
|
||||
|
||||
**What happens when consent is missing**:
|
||||
|
||||
A tool call against an `oauth_user` server returns a structured `mcp_consent_required` error to the agent. The agent surfaces the deferred work in its output. Turnstone persists a record to `mcp_pending_consent` so the dashboard badge surfaces the deferred consent need to the user on next login.
|
||||
|
||||
**Recovery**:
|
||||
|
||||
The user opens the dashboard, sees the gear-icon badge counting pending consents, opens the settings modal, clicks Connect for each affected server, and completes the OAuth dance. The pending-consent record is cleared by the OAuth callback handler on success. Subsequent scheduled / channel runs use the freshly-stored token.
|
||||
|
||||
**Pre-consent recipe**:
|
||||
|
||||
Before scheduling a workstream that depends on an `oauth_user` MCP server, the user should:
|
||||
|
||||
1. Open the dashboard.
|
||||
2. Open the settings modal (gear icon).
|
||||
3. Click Connect on each MCP server the schedule will use.
|
||||
4. Confirm consent in the popup.
|
||||
|
||||
This stores tokens that the scheduled run will reuse. Refresh-token rotation is handled transparently on the run side; only the first consent requires browser interaction.
|
||||
+3
-3
@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
|
||||
|
||||
[project]
|
||||
name = "turnstone"
|
||||
version = "1.5.13"
|
||||
version = "1.5.18"
|
||||
description = "Multi-node AI orchestration platform with tool use, agent routing, and cluster simulation."
|
||||
readme = "README.md"
|
||||
license = "BUSL-1.1"
|
||||
@@ -82,9 +82,9 @@ include = [
|
||||
"turnstone/console/static/coordinator/*.js",
|
||||
"turnstone/shared_static/*.css",
|
||||
"turnstone/shared_static/*.js",
|
||||
"turnstone/shared_static/katex-0.16.45/**/*",
|
||||
"turnstone/shared_static/katex-0.16.47/**/*",
|
||||
"turnstone/shared_static/hljs-11.11.1/**/*",
|
||||
"turnstone/shared_static/mermaid-11.14.0/**/*",
|
||||
"turnstone/shared_static/mermaid-11.15.0/**/*",
|
||||
"turnstone/shared_static/hls-1.6.16/**/*",
|
||||
"turnstone/sdk/py.typed",
|
||||
"turnstone/deploy/*.yaml",
|
||||
|
||||
@@ -50,11 +50,14 @@ update_refs() {
|
||||
local old_pattern="$1" # e.g. katex-0.16.38
|
||||
local new_pattern="$2" # e.g. katex-0.16.39
|
||||
|
||||
# Find all files with version references (excludes vendored JS and worktrees)
|
||||
# Find all files with version references. Excludes the old versioned vendor
|
||||
# directory itself (about to be rm -rf'd anyway) so we don't bother rewriting
|
||||
# self-references inside it — but does NOT exclude all of shared_static/,
|
||||
# because shared_static/renderer.js loads the vendored libs and needs the bump.
|
||||
local files
|
||||
files=$(grep -rl --include='*.toml' --include='*.html' --include='*.js' --include='*.md' \
|
||||
files=$(grep -rl --include='*.toml' --include='*.html' --include='*.js' --include='*.md' --include='*.py' \
|
||||
-F "$old_pattern" . \
|
||||
--exclude-dir='.claude' --exclude-dir='node_modules' --exclude-dir='shared_static' \
|
||||
--exclude-dir='.claude' --exclude-dir='node_modules' --exclude-dir="$old_pattern" \
|
||||
2>/dev/null || true)
|
||||
for f in $files; do
|
||||
sed -i "s|${old_pattern}|${new_pattern}|g" "$f"
|
||||
|
||||
Generated
+119
-119
@@ -74,9 +74,9 @@
|
||||
}
|
||||
},
|
||||
"node_modules/@oxc-project/types": {
|
||||
"version": "0.129.0",
|
||||
"resolved": "https://registry.npmjs.org/@oxc-project/types/-/types-0.129.0.tgz",
|
||||
"integrity": "sha512-3oz8m3FGdr2nDXVqmFUw7jolKliC4MoyXYIG2c7gpjBnzUWQpUGIYcXYKxTdTi+N2jusvt610ckTMkxdwHkYEg==",
|
||||
"version": "0.130.0",
|
||||
"resolved": "https://registry.npmjs.org/@oxc-project/types/-/types-0.130.0.tgz",
|
||||
"integrity": "sha512-ibD2usx9JRu7f5pu2tMKMI4cpA4NgXJQoYRP4pQ7Pxmn1l6k/53qWtQWZayhYy3X4QZkt90Ot+mJEaeXouio6Q==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"funding": {
|
||||
@@ -84,9 +84,9 @@
|
||||
}
|
||||
},
|
||||
"node_modules/@rolldown/binding-android-arm64": {
|
||||
"version": "1.0.0",
|
||||
"resolved": "https://registry.npmjs.org/@rolldown/binding-android-arm64/-/binding-android-arm64-1.0.0.tgz",
|
||||
"integrity": "sha512-TWMZnRLMe63C2Lhyicviu7ZHaU4kxa6PS3rofvc9GmcvptzNN11BcfQ4Sl7MwTOsisQoa2keB/EBdNCAnUo8vA==",
|
||||
"version": "1.0.1",
|
||||
"resolved": "https://registry.npmjs.org/@rolldown/binding-android-arm64/-/binding-android-arm64-1.0.1.tgz",
|
||||
"integrity": "sha512-fJI3I0r3C3Oj/zdBCpaCmBRZYf07xpaq4yCfDDoSFm+beWNzbIl26puW8RraUdugoJw/95zerNOn6jasAhzSmg==",
|
||||
"cpu": [
|
||||
"arm64"
|
||||
],
|
||||
@@ -101,9 +101,9 @@
|
||||
}
|
||||
},
|
||||
"node_modules/@rolldown/binding-darwin-arm64": {
|
||||
"version": "1.0.0",
|
||||
"resolved": "https://registry.npmjs.org/@rolldown/binding-darwin-arm64/-/binding-darwin-arm64-1.0.0.tgz",
|
||||
"integrity": "sha512-6XcD+8k0gPVItNagEw78/qqcBDwKcwDYS8V2hRmVsfUSIrd8cWe/CBvRDI5toqFyPfj+FJr6t8U6Xj2P2prEew==",
|
||||
"version": "1.0.1",
|
||||
"resolved": "https://registry.npmjs.org/@rolldown/binding-darwin-arm64/-/binding-darwin-arm64-1.0.1.tgz",
|
||||
"integrity": "sha512-cKnAhWEsV7TPcA/5EAteDp6KcJZBQ2G+BqE7zayMMi7kMvwRsbv7WT9aOnn0WNl4SKEIf43vjS31iUPu80nzXg==",
|
||||
"cpu": [
|
||||
"arm64"
|
||||
],
|
||||
@@ -118,9 +118,9 @@
|
||||
}
|
||||
},
|
||||
"node_modules/@rolldown/binding-darwin-x64": {
|
||||
"version": "1.0.0",
|
||||
"resolved": "https://registry.npmjs.org/@rolldown/binding-darwin-x64/-/binding-darwin-x64-1.0.0.tgz",
|
||||
"integrity": "sha512-iN/tWVXRQDWvmZlKdceP1Dwug9GDpEymhb9p4xnEe6zvCg5lFmzVljl+1qR1NVx3yfGpr2Na+CuLmv5IU8uzfQ==",
|
||||
"version": "1.0.1",
|
||||
"resolved": "https://registry.npmjs.org/@rolldown/binding-darwin-x64/-/binding-darwin-x64-1.0.1.tgz",
|
||||
"integrity": "sha512-YKrVwQjIRBPo+5G/u03wGjbdy4q7pyzCe93DK9VJ7zkVmeg8LJ7GbgsiHWdR4xSoe4CAXRD7Bcjgbtr64bkXNg==",
|
||||
"cpu": [
|
||||
"x64"
|
||||
],
|
||||
@@ -135,9 +135,9 @@
|
||||
}
|
||||
},
|
||||
"node_modules/@rolldown/binding-freebsd-x64": {
|
||||
"version": "1.0.0",
|
||||
"resolved": "https://registry.npmjs.org/@rolldown/binding-freebsd-x64/-/binding-freebsd-x64-1.0.0.tgz",
|
||||
"integrity": "sha512-jjQMDvvwSOuhOwMszD/klSOjyWMM3zI64hWTj9KT5x4MxRbZAf+7vLQ6qouRhtsLVFHr3f0ILaJAfgENPiQdAQ==",
|
||||
"version": "1.0.1",
|
||||
"resolved": "https://registry.npmjs.org/@rolldown/binding-freebsd-x64/-/binding-freebsd-x64-1.0.1.tgz",
|
||||
"integrity": "sha512-z/oBsREo46SsFqBwYtFe0kpJeBijAT48O/WXLI4suiCLBkr03RTtTJMCzSdDd2znlh8VJizL09XVkQgk8IZonw==",
|
||||
"cpu": [
|
||||
"x64"
|
||||
],
|
||||
@@ -152,9 +152,9 @@
|
||||
}
|
||||
},
|
||||
"node_modules/@rolldown/binding-linux-arm-gnueabihf": {
|
||||
"version": "1.0.0",
|
||||
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-arm-gnueabihf/-/binding-linux-arm-gnueabihf-1.0.0.tgz",
|
||||
"integrity": "sha512-d//Dtg2x6/m3mbV64yUGNnDGNZaDGRpDLLNGerHQUVObuNaIQaaDp25yUiqGXtHEXX+NP2d0wAlmKgpYgIAJ2A==",
|
||||
"version": "1.0.1",
|
||||
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-arm-gnueabihf/-/binding-linux-arm-gnueabihf-1.0.1.tgz",
|
||||
"integrity": "sha512-ik8q7GM11zxvYxFc2PeDcT6TBvhCQMaUxfph/M5l9sKuTs/Sjg3L+Byw0F7w0ZVLBZmx30P+gG0ECzzN+MFcmQ==",
|
||||
"cpu": [
|
||||
"arm"
|
||||
],
|
||||
@@ -169,9 +169,9 @@
|
||||
}
|
||||
},
|
||||
"node_modules/@rolldown/binding-linux-arm64-gnu": {
|
||||
"version": "1.0.0",
|
||||
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-arm64-gnu/-/binding-linux-arm64-gnu-1.0.0.tgz",
|
||||
"integrity": "sha512-n7Ofp0mx+aB2cC+Sdy5YtMnXtY9lchnHbY+3Yt0uq9JsWQExf4f5Whu0tK0R8Jdc9S6RchTHjIFY7uc92puOVQ==",
|
||||
"version": "1.0.1",
|
||||
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-arm64-gnu/-/binding-linux-arm64-gnu-1.0.1.tgz",
|
||||
"integrity": "sha512-QoSx2EkyrrdZ6kcyE8stqZ62t0Yra8Fs5ia9lOxJrh6TMQJK7gQKmscdTHf7pOXKREKrVwOtJcQG3qVSfc866A==",
|
||||
"cpu": [
|
||||
"arm64"
|
||||
],
|
||||
@@ -189,9 +189,9 @@
|
||||
}
|
||||
},
|
||||
"node_modules/@rolldown/binding-linux-arm64-musl": {
|
||||
"version": "1.0.0",
|
||||
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-arm64-musl/-/binding-linux-arm64-musl-1.0.0.tgz",
|
||||
"integrity": "sha512-EIVjy2cgd7uuMMo94FVkBp7F6DhcZAUwNURkSG3RwUmvAXR6s0ISxM81U+IydcZByPG0pZIHsf1b6kTxoFDgJA==",
|
||||
"version": "1.0.1",
|
||||
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-arm64-musl/-/binding-linux-arm64-musl-1.0.1.tgz",
|
||||
"integrity": "sha512-uwNwFpwKeNiZawfAWBgg0VIztPTV3ihhh1vV334h9ivnNLorxnQMU6Fz8wG1Zb4Qh9LC1/MkcyT3YlDXG3Rsgg==",
|
||||
"cpu": [
|
||||
"arm64"
|
||||
],
|
||||
@@ -209,9 +209,9 @@
|
||||
}
|
||||
},
|
||||
"node_modules/@rolldown/binding-linux-ppc64-gnu": {
|
||||
"version": "1.0.0",
|
||||
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-ppc64-gnu/-/binding-linux-ppc64-gnu-1.0.0.tgz",
|
||||
"integrity": "sha512-JEwwOPcwTLAcpDQlqSmjEmfs63xJnSiUNIGvLcDLUHCWK4XowpS/7c7tUsUH6uT/ct6bMUTdXKfI8967FYj6mg==",
|
||||
"version": "1.0.1",
|
||||
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-ppc64-gnu/-/binding-linux-ppc64-gnu-1.0.1.tgz",
|
||||
"integrity": "sha512-zY1bul7OWr7DFBiJ++wofXvnr8B45ce3QsQUhKrIhXsygAh7bTkwyeM1bi1a2g5C/yC/N8TZyGDEoMfm/l9mpg==",
|
||||
"cpu": [
|
||||
"ppc64"
|
||||
],
|
||||
@@ -229,9 +229,9 @@
|
||||
}
|
||||
},
|
||||
"node_modules/@rolldown/binding-linux-s390x-gnu": {
|
||||
"version": "1.0.0",
|
||||
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-s390x-gnu/-/binding-linux-s390x-gnu-1.0.0.tgz",
|
||||
"integrity": "sha512-0wjCFhLrihtAubnT9iA0N++0pSV0z5Hg7tNGdNJ4RFaINceHadoF+kiFGyY1qSSNVIAZtLotG8Ju1bgDPkjnFA==",
|
||||
"version": "1.0.1",
|
||||
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-s390x-gnu/-/binding-linux-s390x-gnu-1.0.1.tgz",
|
||||
"integrity": "sha512-0frlsT/f4Ft6I7SMESTKnF3cZsdicQn1dCMkF/jT9wDLE+gGoiQfv1nmT9e+s7s/fekvvy6tZM2jHvI2tkbJDQ==",
|
||||
"cpu": [
|
||||
"s390x"
|
||||
],
|
||||
@@ -249,9 +249,9 @@
|
||||
}
|
||||
},
|
||||
"node_modules/@rolldown/binding-linux-x64-gnu": {
|
||||
"version": "1.0.0",
|
||||
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-x64-gnu/-/binding-linux-x64-gnu-1.0.0.tgz",
|
||||
"integrity": "sha512-Dfn7iak9BcMMePxcoJfpSbWqnEyrp/dRF63/8qW/eHBdOZov6x5aShLLEYGYdIeSJ6vMLK/XCVB+lGIxm41bQA==",
|
||||
"version": "1.0.1",
|
||||
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-x64-gnu/-/binding-linux-x64-gnu-1.0.1.tgz",
|
||||
"integrity": "sha512-XABVmGp9Tg0WspTVvwduTc4fpqy6JnAUrSQe6OuyqD/03nI7r0O9OWUkMIwFrjKAIqolvqoA4ZrJppgwE0Gxmw==",
|
||||
"cpu": [
|
||||
"x64"
|
||||
],
|
||||
@@ -269,9 +269,9 @@
|
||||
}
|
||||
},
|
||||
"node_modules/@rolldown/binding-linux-x64-musl": {
|
||||
"version": "1.0.0",
|
||||
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-x64-musl/-/binding-linux-x64-musl-1.0.0.tgz",
|
||||
"integrity": "sha512-5/utzzDmD/pD/bmuaUcbTf/sZYy0aztwIVlfpoW1fTjCZ0BaPOMVWGZL1zvgxyi7ZIVYWlxKONHmSbHuiOh8Jw==",
|
||||
"version": "1.0.1",
|
||||
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-x64-musl/-/binding-linux-x64-musl-1.0.1.tgz",
|
||||
"integrity": "sha512-bV4fzswuzVcKD90o/VM6QqKxnxlDq0g2BISDLNVmxrnhpv1DDbyPhCIjYfvzYLV+MvkKKnQt2Q6AO86SEBULUQ==",
|
||||
"cpu": [
|
||||
"x64"
|
||||
],
|
||||
@@ -289,9 +289,9 @@
|
||||
}
|
||||
},
|
||||
"node_modules/@rolldown/binding-openharmony-arm64": {
|
||||
"version": "1.0.0",
|
||||
"resolved": "https://registry.npmjs.org/@rolldown/binding-openharmony-arm64/-/binding-openharmony-arm64-1.0.0.tgz",
|
||||
"integrity": "sha512-ouJs8VcUomfLfpbUECqFMRqdV4x6aeAK3MA4m6vTrJJjKyWTV5KnxZx7Jd9G+GlDaQQxubcba00x16OyJ1meig==",
|
||||
"version": "1.0.1",
|
||||
"resolved": "https://registry.npmjs.org/@rolldown/binding-openharmony-arm64/-/binding-openharmony-arm64-1.0.1.tgz",
|
||||
"integrity": "sha512-/Mh0Zhq3OP7fVs0kcQHZP6lZEthMGTaSf8UBQYSFEZDWGXXlEC+nJ6EqenaK2t4LBXMe3A+K/G2BVXXdtOr4PQ==",
|
||||
"cpu": [
|
||||
"arm64"
|
||||
],
|
||||
@@ -306,9 +306,9 @@
|
||||
}
|
||||
},
|
||||
"node_modules/@rolldown/binding-wasm32-wasi": {
|
||||
"version": "1.0.0",
|
||||
"resolved": "https://registry.npmjs.org/@rolldown/binding-wasm32-wasi/-/binding-wasm32-wasi-1.0.0.tgz",
|
||||
"integrity": "sha512-E+oHKGiDA+lsKMmFtffDDw91EryDT7uJocrIuCHqhm6bCTM6xFK+3gaCkYOHfPwQr0cCNarSM2xaELoQDz9jJg==",
|
||||
"version": "1.0.1",
|
||||
"resolved": "https://registry.npmjs.org/@rolldown/binding-wasm32-wasi/-/binding-wasm32-wasi-1.0.1.tgz",
|
||||
"integrity": "sha512-+1xc9X45l8ufsBAm6Gjvx2qDRIY9lTVt0cgWNcJ+1gdhXvkbxePA60yRTwSTuXL09CMhyJmjpV7E3NoyxbqFQQ==",
|
||||
"cpu": [
|
||||
"wasm32"
|
||||
],
|
||||
@@ -325,9 +325,9 @@
|
||||
}
|
||||
},
|
||||
"node_modules/@rolldown/binding-win32-arm64-msvc": {
|
||||
"version": "1.0.0",
|
||||
"resolved": "https://registry.npmjs.org/@rolldown/binding-win32-arm64-msvc/-/binding-win32-arm64-msvc-1.0.0.tgz",
|
||||
"integrity": "sha512-yYK02n8Rngo+gbm1y6G0+7jk1sJ/2Wt7K0me0Y7k/ErBpyf+LJ2gFpqWVTcRV1rUepBlQRmpgWkTQCiiwrK0Ow==",
|
||||
"version": "1.0.1",
|
||||
"resolved": "https://registry.npmjs.org/@rolldown/binding-win32-arm64-msvc/-/binding-win32-arm64-msvc-1.0.1.tgz",
|
||||
"integrity": "sha512-1D+UqZdfnuR+Jy1GgMJwi85bD40H21uNmOPRWQhw4oRSuolZ/B5rixZ45DK2KXOTCvmVCecauWgEhbw8bI7tOw==",
|
||||
"cpu": [
|
||||
"arm64"
|
||||
],
|
||||
@@ -342,9 +342,9 @@
|
||||
}
|
||||
},
|
||||
"node_modules/@rolldown/binding-win32-x64-msvc": {
|
||||
"version": "1.0.0",
|
||||
"resolved": "https://registry.npmjs.org/@rolldown/binding-win32-x64-msvc/-/binding-win32-x64-msvc-1.0.0.tgz",
|
||||
"integrity": "sha512-14bpChMahXRRXiTwahSl+zzHPW6qQTXtkMuJBFlbo+pqSAews2d4BdCSHfrJ/MBsCZtpmTafsY+1QhBzitcmdg==",
|
||||
"version": "1.0.1",
|
||||
"resolved": "https://registry.npmjs.org/@rolldown/binding-win32-x64-msvc/-/binding-win32-x64-msvc-1.0.1.tgz",
|
||||
"integrity": "sha512-INAycaWuhlOK3wk4mRHGsdgwYWmd9cChdPdE9bwWmy6rn9VqVNYNFGhOdXrofXUxwHIncSiPNb8tNm8knDVIeQ==",
|
||||
"cpu": [
|
||||
"x64"
|
||||
],
|
||||
@@ -359,9 +359,9 @@
|
||||
}
|
||||
},
|
||||
"node_modules/@rolldown/pluginutils": {
|
||||
"version": "1.0.0",
|
||||
"resolved": "https://registry.npmjs.org/@rolldown/pluginutils/-/pluginutils-1.0.0.tgz",
|
||||
"integrity": "sha512-aKs/3GSWyV0mrhNmt/96/Z3yczC3yvrzYATCiCXQebBsGyYzjNdUphRVLeJQ67ySKVXRfMxt2lm12pmXvbPFQQ==",
|
||||
"version": "1.0.1",
|
||||
"resolved": "https://registry.npmjs.org/@rolldown/pluginutils/-/pluginutils-1.0.1.tgz",
|
||||
"integrity": "sha512-2j9bGt5Jh8hj+vPtgzPtl72j0yRxHAyumoo6TNfAjsLB04UtpSvPbPcDcBMxz7n+9CYB0c1GxQFxYRg2jimqGw==",
|
||||
"dev": true,
|
||||
"license": "MIT"
|
||||
},
|
||||
@@ -409,16 +409,16 @@
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/@vitest/expect": {
|
||||
"version": "4.1.5",
|
||||
"resolved": "https://registry.npmjs.org/@vitest/expect/-/expect-4.1.5.tgz",
|
||||
"integrity": "sha512-PWBaRY5JoKuRnHlUHfpV/KohFylaDZTupcXN1H9vYryNLOnitSw60Mw9IAE2r67NbwwzBw/Cc/8q9BK3kIX8Kw==",
|
||||
"version": "4.1.6",
|
||||
"resolved": "https://registry.npmjs.org/@vitest/expect/-/expect-4.1.6.tgz",
|
||||
"integrity": "sha512-7EHDquPthALSV0jhhjgEW8FXaviMx7rSqu8W6oqCoAuOhKov814P99QDV1pxMA3QPv21YudvJngIhjrNI4opLg==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"@standard-schema/spec": "^1.1.0",
|
||||
"@types/chai": "^5.2.2",
|
||||
"@vitest/spy": "4.1.5",
|
||||
"@vitest/utils": "4.1.5",
|
||||
"@vitest/spy": "4.1.6",
|
||||
"@vitest/utils": "4.1.6",
|
||||
"chai": "^6.2.2",
|
||||
"tinyrainbow": "^3.1.0"
|
||||
},
|
||||
@@ -427,13 +427,13 @@
|
||||
}
|
||||
},
|
||||
"node_modules/@vitest/mocker": {
|
||||
"version": "4.1.5",
|
||||
"resolved": "https://registry.npmjs.org/@vitest/mocker/-/mocker-4.1.5.tgz",
|
||||
"integrity": "sha512-/x2EmFC4mT4NNzqvC3fmesuV97w5FC903KPmey4gsnJiMQ3Be1IlDKVaDaG8iqaLFHqJ2FVEkxZk5VmeLjIItw==",
|
||||
"version": "4.1.6",
|
||||
"resolved": "https://registry.npmjs.org/@vitest/mocker/-/mocker-4.1.6.tgz",
|
||||
"integrity": "sha512-MCFc63czMjEInOlcY2cpQCvCN+KgbAn+60xu9cMgP4sKaLC5JNAKw7JH8QdAnoAC88hW1IiSNZ+GgVXlN1UcMQ==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"@vitest/spy": "4.1.5",
|
||||
"@vitest/spy": "4.1.6",
|
||||
"estree-walker": "^3.0.3",
|
||||
"magic-string": "^0.30.21"
|
||||
},
|
||||
@@ -454,9 +454,9 @@
|
||||
}
|
||||
},
|
||||
"node_modules/@vitest/pretty-format": {
|
||||
"version": "4.1.5",
|
||||
"resolved": "https://registry.npmjs.org/@vitest/pretty-format/-/pretty-format-4.1.5.tgz",
|
||||
"integrity": "sha512-7I3q6l5qr03dVfMX2wCo9FxwSJbPdwKjy2uu/YPpU3wfHvIL4QHwVRp57OfGrDFeUJ8/8QdfBKIV12FTtLn00g==",
|
||||
"version": "4.1.6",
|
||||
"resolved": "https://registry.npmjs.org/@vitest/pretty-format/-/pretty-format-4.1.6.tgz",
|
||||
"integrity": "sha512-h5SxD/IzNhZYnrSZRsUZQIC+vD0GY8cUvq0iwsmkFKixRCKLLWqCXa/FIQ4S1R+sI+PGoojkHsdNrbZiM9Qpgw==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
@@ -467,13 +467,13 @@
|
||||
}
|
||||
},
|
||||
"node_modules/@vitest/runner": {
|
||||
"version": "4.1.5",
|
||||
"resolved": "https://registry.npmjs.org/@vitest/runner/-/runner-4.1.5.tgz",
|
||||
"integrity": "sha512-2D+o7Pr82IEO46YPpoA/YU0neeyr6FTerQb5Ro7BUnBuv6NQtT/kmVnczngiMEBhzgqz2UZYl5gArejsyERDSQ==",
|
||||
"version": "4.1.6",
|
||||
"resolved": "https://registry.npmjs.org/@vitest/runner/-/runner-4.1.6.tgz",
|
||||
"integrity": "sha512-nOPCmn2+yD0ZNmKdsXGv/UxMMWbMuKeD6GyYncNwdkYDxpQvrPSKYj2rWuDjC2Y4b6w6hjip5dBKFzEUuZe3vA==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"@vitest/utils": "4.1.5",
|
||||
"@vitest/utils": "4.1.6",
|
||||
"pathe": "^2.0.3"
|
||||
},
|
||||
"funding": {
|
||||
@@ -481,14 +481,14 @@
|
||||
}
|
||||
},
|
||||
"node_modules/@vitest/snapshot": {
|
||||
"version": "4.1.5",
|
||||
"resolved": "https://registry.npmjs.org/@vitest/snapshot/-/snapshot-4.1.5.tgz",
|
||||
"integrity": "sha512-zypXEt4KH/XgKGPUz4eC2AvErYx0My5hfL8oDb1HzGFpEk1P62bxSohdyOmvz+d9UJwanI68MKwr2EquOaOgMQ==",
|
||||
"version": "4.1.6",
|
||||
"resolved": "https://registry.npmjs.org/@vitest/snapshot/-/snapshot-4.1.6.tgz",
|
||||
"integrity": "sha512-YhsdE6xAVfTDmzjxL2ZDUvjj+ZsgyOKe+TdQzqkD72wIOmHka8NuGQ6NpTNZv9D2Z63fbwWKJPeVpEw4EQgYxw==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"@vitest/pretty-format": "4.1.5",
|
||||
"@vitest/utils": "4.1.5",
|
||||
"@vitest/pretty-format": "4.1.6",
|
||||
"@vitest/utils": "4.1.6",
|
||||
"magic-string": "^0.30.21",
|
||||
"pathe": "^2.0.3"
|
||||
},
|
||||
@@ -497,9 +497,9 @@
|
||||
}
|
||||
},
|
||||
"node_modules/@vitest/spy": {
|
||||
"version": "4.1.5",
|
||||
"resolved": "https://registry.npmjs.org/@vitest/spy/-/spy-4.1.5.tgz",
|
||||
"integrity": "sha512-2lNOsh6+R2Idnf1TCZqSwYlKN2E/iDlD8sgU59kYVl+OMDmvldO1VDk39smRfpUNwYpNRVn3w4YfuC7KfbBnkQ==",
|
||||
"version": "4.1.6",
|
||||
"resolved": "https://registry.npmjs.org/@vitest/spy/-/spy-4.1.6.tgz",
|
||||
"integrity": "sha512-JFKxMx6udhwKh/Ldo270e17QX710vgunMkuPAvXjHSvC6oqLWAHhVhjg/I71q0u0CBSErIODV1Kjv0FQNSWjdg==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"funding": {
|
||||
@@ -507,13 +507,13 @@
|
||||
}
|
||||
},
|
||||
"node_modules/@vitest/utils": {
|
||||
"version": "4.1.5",
|
||||
"resolved": "https://registry.npmjs.org/@vitest/utils/-/utils-4.1.5.tgz",
|
||||
"integrity": "sha512-76wdkrmfXfqGjueGgnb45ITPyUi1ycZ4IHgC2bhPDUfWHklY/q3MdLOAB+TF1e6xfl8NxNY0ZYaPCFNWSsw3Ug==",
|
||||
"version": "4.1.6",
|
||||
"resolved": "https://registry.npmjs.org/@vitest/utils/-/utils-4.1.6.tgz",
|
||||
"integrity": "sha512-FxIY+U81R3LGKCxaHHFRQ5+g6/iRgGLmeHWdp2Amj4ljQRrEIWHmZyDfDYBRZlpyqA7qKxtS9DD1dhk8RnRIVQ==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"@vitest/pretty-format": "4.1.5",
|
||||
"@vitest/pretty-format": "4.1.6",
|
||||
"convert-source-map": "^2.0.0",
|
||||
"tinyrainbow": "^3.1.0"
|
||||
},
|
||||
@@ -988,14 +988,14 @@
|
||||
}
|
||||
},
|
||||
"node_modules/rolldown": {
|
||||
"version": "1.0.0",
|
||||
"resolved": "https://registry.npmjs.org/rolldown/-/rolldown-1.0.0.tgz",
|
||||
"integrity": "sha512-yD986aXDESFGS95spT1LAv0jssywP4npMEjmMHyN2/5+eE8qQJUype2AaKkRiLgBgyD0LFlubwAht7VmY8rGoA==",
|
||||
"version": "1.0.1",
|
||||
"resolved": "https://registry.npmjs.org/rolldown/-/rolldown-1.0.1.tgz",
|
||||
"integrity": "sha512-X0KQHljNnEkWNqqiz9zJrGunh1B0HgOxLXvnFpCOcadzcy5qohZ3tqMEUg00vncoRovXuK3ZqCT9KnnKzoInFQ==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"@oxc-project/types": "=0.129.0",
|
||||
"@rolldown/pluginutils": "1.0.0"
|
||||
"@oxc-project/types": "=0.130.0",
|
||||
"@rolldown/pluginutils": "^1.0.0"
|
||||
},
|
||||
"bin": {
|
||||
"rolldown": "bin/cli.mjs"
|
||||
@@ -1004,21 +1004,21 @@
|
||||
"node": "^20.19.0 || >=22.12.0"
|
||||
},
|
||||
"optionalDependencies": {
|
||||
"@rolldown/binding-android-arm64": "1.0.0",
|
||||
"@rolldown/binding-darwin-arm64": "1.0.0",
|
||||
"@rolldown/binding-darwin-x64": "1.0.0",
|
||||
"@rolldown/binding-freebsd-x64": "1.0.0",
|
||||
"@rolldown/binding-linux-arm-gnueabihf": "1.0.0",
|
||||
"@rolldown/binding-linux-arm64-gnu": "1.0.0",
|
||||
"@rolldown/binding-linux-arm64-musl": "1.0.0",
|
||||
"@rolldown/binding-linux-ppc64-gnu": "1.0.0",
|
||||
"@rolldown/binding-linux-s390x-gnu": "1.0.0",
|
||||
"@rolldown/binding-linux-x64-gnu": "1.0.0",
|
||||
"@rolldown/binding-linux-x64-musl": "1.0.0",
|
||||
"@rolldown/binding-openharmony-arm64": "1.0.0",
|
||||
"@rolldown/binding-wasm32-wasi": "1.0.0",
|
||||
"@rolldown/binding-win32-arm64-msvc": "1.0.0",
|
||||
"@rolldown/binding-win32-x64-msvc": "1.0.0"
|
||||
"@rolldown/binding-android-arm64": "1.0.1",
|
||||
"@rolldown/binding-darwin-arm64": "1.0.1",
|
||||
"@rolldown/binding-darwin-x64": "1.0.1",
|
||||
"@rolldown/binding-freebsd-x64": "1.0.1",
|
||||
"@rolldown/binding-linux-arm-gnueabihf": "1.0.1",
|
||||
"@rolldown/binding-linux-arm64-gnu": "1.0.1",
|
||||
"@rolldown/binding-linux-arm64-musl": "1.0.1",
|
||||
"@rolldown/binding-linux-ppc64-gnu": "1.0.1",
|
||||
"@rolldown/binding-linux-s390x-gnu": "1.0.1",
|
||||
"@rolldown/binding-linux-x64-gnu": "1.0.1",
|
||||
"@rolldown/binding-linux-x64-musl": "1.0.1",
|
||||
"@rolldown/binding-openharmony-arm64": "1.0.1",
|
||||
"@rolldown/binding-wasm32-wasi": "1.0.1",
|
||||
"@rolldown/binding-win32-arm64-msvc": "1.0.1",
|
||||
"@rolldown/binding-win32-x64-msvc": "1.0.1"
|
||||
}
|
||||
},
|
||||
"node_modules/siginfo": {
|
||||
@@ -1119,16 +1119,16 @@
|
||||
}
|
||||
},
|
||||
"node_modules/vite": {
|
||||
"version": "8.0.12",
|
||||
"resolved": "https://registry.npmjs.org/vite/-/vite-8.0.12.tgz",
|
||||
"integrity": "sha512-w2dDofOWv2QB09ZITZBsvKTVAlYvPR4IAmrY/v0ir9KvLs0xybR7i48wxhM1/oyBWO34wPns+bPGw5ZrZqDpZg==",
|
||||
"version": "8.0.13",
|
||||
"resolved": "https://registry.npmjs.org/vite/-/vite-8.0.13.tgz",
|
||||
"integrity": "sha512-MFtjBYgzmSxmgA4RAfjIyXWpGe1oALnjgUTzzV7QLx/TKxCzjtMH6Fd9/eVK+5Fg1qNoz5VAwsmMs/NofrmJvw==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"lightningcss": "^1.32.0",
|
||||
"picomatch": "^4.0.4",
|
||||
"postcss": "^8.5.14",
|
||||
"rolldown": "1.0.0",
|
||||
"rolldown": "1.0.1",
|
||||
"tinyglobby": "^0.2.16"
|
||||
},
|
||||
"bin": {
|
||||
@@ -1197,19 +1197,19 @@
|
||||
}
|
||||
},
|
||||
"node_modules/vitest": {
|
||||
"version": "4.1.5",
|
||||
"resolved": "https://registry.npmjs.org/vitest/-/vitest-4.1.5.tgz",
|
||||
"integrity": "sha512-9Xx1v3/ih3m9hN+SbfkUyy0JAs72ap3r7joc87XL6jwF0jGg6mFBvQ1SrwaX+h8BlkX6Hz9shdd1uo6AF+ZGpg==",
|
||||
"version": "4.1.6",
|
||||
"resolved": "https://registry.npmjs.org/vitest/-/vitest-4.1.6.tgz",
|
||||
"integrity": "sha512-6lvjbS3p9b4CrdCmguzbh2/4uoXhGE2q71R4OX5sqF9R1bo9Xd6fGrMAfvp5wnCzlBnFVdCOp6onuTQVbo8iUQ==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"@vitest/expect": "4.1.5",
|
||||
"@vitest/mocker": "4.1.5",
|
||||
"@vitest/pretty-format": "4.1.5",
|
||||
"@vitest/runner": "4.1.5",
|
||||
"@vitest/snapshot": "4.1.5",
|
||||
"@vitest/spy": "4.1.5",
|
||||
"@vitest/utils": "4.1.5",
|
||||
"@vitest/expect": "4.1.6",
|
||||
"@vitest/mocker": "4.1.6",
|
||||
"@vitest/pretty-format": "4.1.6",
|
||||
"@vitest/runner": "4.1.6",
|
||||
"@vitest/snapshot": "4.1.6",
|
||||
"@vitest/spy": "4.1.6",
|
||||
"@vitest/utils": "4.1.6",
|
||||
"es-module-lexer": "^2.0.0",
|
||||
"expect-type": "^1.3.0",
|
||||
"magic-string": "^0.30.21",
|
||||
@@ -1237,12 +1237,12 @@
|
||||
"@edge-runtime/vm": "*",
|
||||
"@opentelemetry/api": "^1.9.0",
|
||||
"@types/node": "^20.0.0 || ^22.0.0 || >=24.0.0",
|
||||
"@vitest/browser-playwright": "4.1.5",
|
||||
"@vitest/browser-preview": "4.1.5",
|
||||
"@vitest/browser-webdriverio": "4.1.5",
|
||||
"@vitest/coverage-istanbul": "4.1.5",
|
||||
"@vitest/coverage-v8": "4.1.5",
|
||||
"@vitest/ui": "4.1.5",
|
||||
"@vitest/browser-playwright": "4.1.6",
|
||||
"@vitest/browser-preview": "4.1.6",
|
||||
"@vitest/browser-webdriverio": "4.1.6",
|
||||
"@vitest/coverage-istanbul": "4.1.6",
|
||||
"@vitest/coverage-v8": "4.1.6",
|
||||
"@vitest/ui": "4.1.6",
|
||||
"happy-dom": "*",
|
||||
"jsdom": "*",
|
||||
"vite": "^6.0.0 || ^7.0.0 || ^8.0.0"
|
||||
|
||||
@@ -0,0 +1,220 @@
|
||||
"""Tests for turnstone-admin DB configuration precedence.
|
||||
|
||||
Locks in the alignment with turnstone-server:
|
||||
CLI / config.toml [database] > TURNSTONE_DB_* env > hardcoded default
|
||||
|
||||
The motivation is to keep DB secrets in config.toml (see
|
||||
feedback_secrets_not_in_env) rather than forcing operators to export
|
||||
TURNSTONE_DB_URL before every admin invocation.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
from typing import TYPE_CHECKING
|
||||
from unittest.mock import patch
|
||||
|
||||
import pytest
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from collections.abc import Iterator
|
||||
from pathlib import Path
|
||||
|
||||
import turnstone.core.config as config_mod
|
||||
from turnstone.admin import _get_storage
|
||||
|
||||
|
||||
def _reset_cache() -> None:
|
||||
config_mod._cache = None
|
||||
config_mod._config_path = None
|
||||
|
||||
|
||||
def _build_args(config_path: str | None) -> argparse.Namespace:
|
||||
"""Build an args namespace the way admin.main() does.
|
||||
|
||||
Skips ``add_config_arg`` (which reads ``sys.argv``) — the test
|
||||
constructs the args programmatically instead.
|
||||
"""
|
||||
config_mod.set_config_path(config_path or "/nonexistent/turnstone-admin-test.toml")
|
||||
parser = argparse.ArgumentParser()
|
||||
config_mod.apply_config(parser, ["database"])
|
||||
sub = parser.add_subparsers(dest="command")
|
||||
sub.add_parser("list-users")
|
||||
return parser.parse_args(["list-users"])
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def _clear_db_env(monkeypatch: pytest.MonkeyPatch) -> Iterator[None]:
|
||||
"""Clean slate: no TURNSTONE_DB_* env vars unless a test sets them."""
|
||||
for var in (
|
||||
"TURNSTONE_DB_BACKEND",
|
||||
"TURNSTONE_DB_URL",
|
||||
"TURNSTONE_DB_PATH",
|
||||
"TURNSTONE_DB_POOL_SIZE",
|
||||
"TURNSTONE_DB_SSLMODE",
|
||||
"TURNSTONE_DB_SSLROOTCERT",
|
||||
"TURNSTONE_DB_SSLCERT",
|
||||
"TURNSTONE_DB_SSLKEY",
|
||||
"TURNSTONE_CONFIG",
|
||||
):
|
||||
monkeypatch.delenv(var, raising=False)
|
||||
_reset_cache()
|
||||
yield
|
||||
_reset_cache()
|
||||
|
||||
|
||||
def test_defaults_to_sqlite_when_neither_config_nor_env_set() -> None:
|
||||
args = _build_args(None)
|
||||
with patch("turnstone.core.storage.init_storage") as init:
|
||||
_get_storage(args)
|
||||
assert init.call_args.args == ("sqlite",)
|
||||
assert init.call_args.kwargs["url"] == ""
|
||||
assert init.call_args.kwargs["path"] == ""
|
||||
assert init.call_args.kwargs["pool_size"] == 2
|
||||
|
||||
|
||||
def test_config_toml_database_section_drives_init_storage(tmp_path: Path) -> None:
|
||||
cfg = tmp_path / "config.toml"
|
||||
cfg.write_text(
|
||||
"[database]\n"
|
||||
'backend = "postgresql"\n'
|
||||
'url = "postgresql+psycopg://fromconfig:x@host/db"\n'
|
||||
"pool_size = 5\n"
|
||||
'sslmode = "verify-full"\n'
|
||||
'sslrootcert = "/etc/ssl/ca.pem"\n'
|
||||
'sslcert = "/etc/ssl/client.pem"\n'
|
||||
'sslkey = "/etc/ssl/client.key"\n'
|
||||
)
|
||||
args = _build_args(str(cfg))
|
||||
with patch("turnstone.core.storage.init_storage") as init:
|
||||
_get_storage(args)
|
||||
assert init.call_args.args == ("postgresql",)
|
||||
kw = init.call_args.kwargs
|
||||
assert kw["url"] == "postgresql+psycopg://fromconfig:x@host/db"
|
||||
assert kw["pool_size"] == 5
|
||||
assert kw["sslmode"] == "verify-full"
|
||||
assert kw["sslrootcert"] == "/etc/ssl/ca.pem"
|
||||
assert kw["sslcert"] == "/etc/ssl/client.pem"
|
||||
assert kw["sslkey"] == "/etc/ssl/client.key"
|
||||
|
||||
|
||||
def test_env_used_as_fallback_when_config_absent(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
monkeypatch.setenv("TURNSTONE_DB_BACKEND", "postgresql")
|
||||
monkeypatch.setenv("TURNSTONE_DB_URL", "postgresql+psycopg://fromenv:x@host/db")
|
||||
monkeypatch.setenv("TURNSTONE_DB_POOL_SIZE", "7")
|
||||
monkeypatch.setenv("TURNSTONE_DB_SSLMODE", "require")
|
||||
|
||||
args = _build_args(None)
|
||||
with patch("turnstone.core.storage.init_storage") as init:
|
||||
_get_storage(args)
|
||||
assert init.call_args.args == ("postgresql",)
|
||||
kw = init.call_args.kwargs
|
||||
assert kw["url"] == "postgresql+psycopg://fromenv:x@host/db"
|
||||
assert kw["pool_size"] == 7
|
||||
assert kw["sslmode"] == "require"
|
||||
|
||||
|
||||
def test_config_toml_wins_over_env(monkeypatch: pytest.MonkeyPatch, tmp_path: Path) -> None:
|
||||
"""config.toml beats env — operators should put secrets in TOML."""
|
||||
monkeypatch.setenv("TURNSTONE_DB_BACKEND", "sqlite")
|
||||
monkeypatch.setenv("TURNSTONE_DB_URL", "postgresql+psycopg://fromenv:x@host/db")
|
||||
monkeypatch.setenv("TURNSTONE_DB_SSLMODE", "require")
|
||||
|
||||
cfg = tmp_path / "config.toml"
|
||||
cfg.write_text(
|
||||
"[database]\n"
|
||||
'backend = "postgresql"\n'
|
||||
'url = "postgresql+psycopg://fromconfig:x@host/db"\n'
|
||||
'sslmode = "verify-full"\n'
|
||||
)
|
||||
args = _build_args(str(cfg))
|
||||
with patch("turnstone.core.storage.init_storage") as init:
|
||||
_get_storage(args)
|
||||
assert init.call_args.args == ("postgresql",)
|
||||
kw = init.call_args.kwargs
|
||||
assert kw["url"] == "postgresql+psycopg://fromconfig:x@host/db"
|
||||
assert kw["sslmode"] == "verify-full"
|
||||
|
||||
|
||||
def test_partial_config_falls_through_to_env_per_key(
|
||||
monkeypatch: pytest.MonkeyPatch, tmp_path: Path
|
||||
) -> None:
|
||||
"""A key missing from [database] should fall back to its env var."""
|
||||
monkeypatch.setenv("TURNSTONE_DB_SSLMODE", "require")
|
||||
monkeypatch.setenv("TURNSTONE_DB_POOL_SIZE", "9")
|
||||
|
||||
cfg = tmp_path / "config.toml"
|
||||
cfg.write_text(
|
||||
'[database]\nbackend = "postgresql"\nurl = "postgresql+psycopg://fromconfig:x@host/db"\n'
|
||||
)
|
||||
args = _build_args(str(cfg))
|
||||
with patch("turnstone.core.storage.init_storage") as init:
|
||||
_get_storage(args)
|
||||
kw = init.call_args.kwargs
|
||||
assert kw["url"] == "postgresql+psycopg://fromconfig:x@host/db"
|
||||
assert kw["sslmode"] == "require"
|
||||
assert kw["pool_size"] == 9
|
||||
|
||||
|
||||
def test_empty_string_in_config_beats_env(monkeypatch: pytest.MonkeyPatch, tmp_path: Path) -> None:
|
||||
"""`url = ""` in config.toml beats an env var.
|
||||
|
||||
Locks in the `is not None` guard — a falsy-but-present TOML value
|
||||
should NOT silently fall through to the env fallback.
|
||||
"""
|
||||
monkeypatch.setenv("TURNSTONE_DB_URL", "postgresql+psycopg://fromenv:x@host/db")
|
||||
cfg = tmp_path / "config.toml"
|
||||
cfg.write_text('[database]\nbackend = "sqlite"\nurl = ""\n')
|
||||
args = _build_args(str(cfg))
|
||||
with patch("turnstone.core.storage.init_storage") as init:
|
||||
_get_storage(args)
|
||||
assert init.call_args.kwargs["url"] == ""
|
||||
|
||||
|
||||
def test_main_threads_config_toml_through_real_argv(
|
||||
monkeypatch: pytest.MonkeyPatch, tmp_path: Path
|
||||
) -> None:
|
||||
"""End-to-end: ``turnstone-admin --config <toml> list-users`` honors TOML.
|
||||
|
||||
Covers the ``add_config_arg`` -> ``apply_config`` -> ``_get_storage``
|
||||
chain that the programmatic ``_build_args`` helper skips.
|
||||
"""
|
||||
cfg = tmp_path / "config.toml"
|
||||
cfg.write_text(
|
||||
'[database]\nbackend = "postgresql"\nurl = "postgresql+psycopg://fromcli:x@host/db"\n'
|
||||
)
|
||||
monkeypatch.setattr("sys.argv", ["turnstone-admin", "--config", str(cfg), "list-users"])
|
||||
|
||||
fake_storage = patch("turnstone.core.storage.init_storage").start()
|
||||
fake_storage.return_value.list_users.return_value = []
|
||||
try:
|
||||
from turnstone.admin import main
|
||||
|
||||
main()
|
||||
finally:
|
||||
patch.stopall()
|
||||
|
||||
assert fake_storage.call_args.args == ("postgresql",)
|
||||
assert fake_storage.call_args.kwargs["url"] == "postgresql+psycopg://fromcli:x@host/db"
|
||||
|
||||
|
||||
def test_get_storage_initializes_real_sqlite_backend(tmp_path: Path) -> None:
|
||||
"""Drives the real ``init_storage`` boundary on a fresh sqlite file.
|
||||
|
||||
Mock-only tests would miss a kwarg-name typo (sslmode -> ssl_mode).
|
||||
This test trips on any such drift because Alembic + the backend
|
||||
actually run.
|
||||
"""
|
||||
from turnstone.core.storage import reset_storage
|
||||
|
||||
db_file = tmp_path / "admin.db"
|
||||
cfg = tmp_path / "config.toml"
|
||||
cfg.write_text(f'[database]\nbackend = "sqlite"\npath = "{db_file}"\n')
|
||||
args = _build_args(str(cfg))
|
||||
|
||||
reset_storage()
|
||||
try:
|
||||
storage = _get_storage(args)
|
||||
assert storage.list_users() == []
|
||||
finally:
|
||||
reset_storage()
|
||||
@@ -0,0 +1,66 @@
|
||||
"""ChatSession interactivity flag tests (Phase 9).
|
||||
|
||||
Validates that ``ChatSession._is_interactive_for_consent`` is computed
|
||||
correctly from ``client_type`` on construction. This is the front of
|
||||
the Phase 9 plumb-through: the flag flows from here to
|
||||
``_dispatch_pool_sync`` to the structured-error → pending-consent
|
||||
write path.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from tests._session_helpers import make_session
|
||||
from turnstone.prompts import INTERACTIVE_CONSENT_CLIENT_TYPES, ClientType
|
||||
|
||||
|
||||
def test_web_is_interactive() -> None:
|
||||
s = make_session(client_type=ClientType.WEB)
|
||||
assert s._is_interactive_for_consent is True
|
||||
|
||||
|
||||
def test_cli_is_interactive() -> None:
|
||||
s = make_session(client_type=ClientType.CLI)
|
||||
assert s._is_interactive_for_consent is True
|
||||
|
||||
|
||||
def test_chat_is_not_interactive() -> None:
|
||||
# Discord / Slack adapters cannot drive a browser redirect from
|
||||
# inside the channel — consent prompts must be deferred to the
|
||||
# dashboard badge.
|
||||
s = make_session(client_type=ClientType.CHAT)
|
||||
assert s._is_interactive_for_consent is False
|
||||
|
||||
|
||||
def test_scheduled_is_not_interactive() -> None:
|
||||
# The scheduler runs autonomously; the user isn't online to
|
||||
# complete the OAuth redirect.
|
||||
s = make_session(client_type=ClientType.SCHEDULED)
|
||||
assert s._is_interactive_for_consent is False
|
||||
|
||||
|
||||
def test_interactive_set_matches_module_constant() -> None:
|
||||
# Pin the module-level frozenset against the flag computation —
|
||||
# a future reorganisation that drifts the set vs the per-session
|
||||
# logic would silently break the gating.
|
||||
for ct in ClientType:
|
||||
s = make_session(client_type=ct)
|
||||
assert s._is_interactive_for_consent == (ct in INTERACTIVE_CONSENT_CLIENT_TYPES), ct
|
||||
|
||||
|
||||
def test_default_client_type_is_cli_interactive() -> None:
|
||||
# Defaults preserved — make_session uses ChatSession's default
|
||||
# which is CLI. Sanity check that the default user experience
|
||||
# stays interactive-for-consent.
|
||||
s = make_session()
|
||||
assert s._client_type == ClientType.CLI
|
||||
assert s._is_interactive_for_consent is True
|
||||
|
||||
|
||||
def test_scheduled_env_file_exists() -> None:
|
||||
"""The SCHEDULED env module must exist; otherwise
|
||||
``compose_system_message`` for a scheduled session would 500."""
|
||||
from turnstone.prompts import _load
|
||||
|
||||
text = _load("env/scheduled.md")
|
||||
assert "Output Environment" in text
|
||||
assert "consent" in text.lower()
|
||||
@@ -50,6 +50,41 @@ def test_load_config_invalid_toml(tmp_path):
|
||||
assert load_config() == {}
|
||||
|
||||
|
||||
def test_load_config_warns_when_world_readable(tmp_path, caplog):
|
||||
"""Secrets in config.toml — warn if anyone but the owner can read it."""
|
||||
import logging
|
||||
import os
|
||||
|
||||
_reset_cache()
|
||||
cfg = tmp_path / "config.toml"
|
||||
cfg.write_text('[database]\nurl = "postgresql+psycopg://u:secret@h/d"\n')
|
||||
os.chmod(cfg, 0o644)
|
||||
set_config_path(str(cfg))
|
||||
|
||||
with caplog.at_level(logging.WARNING, logger="turnstone.core.config"):
|
||||
load_config()
|
||||
|
||||
messages = [r.getMessage() for r in caplog.records]
|
||||
assert any("group/world-readable" in m for m in messages)
|
||||
|
||||
|
||||
def test_load_config_quiet_when_mode_0600(tmp_path, caplog):
|
||||
import logging
|
||||
import os
|
||||
|
||||
_reset_cache()
|
||||
cfg = tmp_path / "config.toml"
|
||||
cfg.write_text('[database]\nurl = "postgresql+psycopg://u:secret@h/d"\n')
|
||||
os.chmod(cfg, 0o600)
|
||||
set_config_path(str(cfg))
|
||||
|
||||
with caplog.at_level(logging.WARNING, logger="turnstone.core.config"):
|
||||
load_config()
|
||||
|
||||
messages = [r.getMessage() for r in caplog.records]
|
||||
assert not any("group/world-readable" in m for m in messages)
|
||||
|
||||
|
||||
def test_load_config_caches(tmp_path):
|
||||
_reset_cache()
|
||||
cfg = tmp_path / "config.toml"
|
||||
|
||||
@@ -1228,6 +1228,49 @@ def test_list_skills_hides_interactive_only_skills(tmp_path):
|
||||
assert skill["kind"] in {"coordinator", "any"}
|
||||
|
||||
|
||||
def test_list_skills_omits_allowed_tools_when_empty(tmp_path):
|
||||
"""``allowed_tools`` is the auto-approve allowlist (tools exempt
|
||||
from the operator approval gate), NOT the set of tools the skill
|
||||
can use. An empty list reads as "no tool access" to a model
|
||||
that doesn't know the semantics — real misdiagnosis source: a
|
||||
code-review skill with no auto-approve allowlist looked like it
|
||||
had been spawned with zero tools. Dropping the key when empty
|
||||
removes the ambiguity at the source; absence of the field carries
|
||||
the unambiguous meaning "no tool is pre-approved for this skill"
|
||||
while a tool list reads as "these specific tools bypass the prompt".
|
||||
"""
|
||||
st = SQLiteBackend(str(tmp_path / "skills_empty.db"))
|
||||
st.create_prompt_template(
|
||||
template_id="s-empty",
|
||||
name="empty-skill",
|
||||
category="ops",
|
||||
content="",
|
||||
variables="[]",
|
||||
is_default=False,
|
||||
org_id="",
|
||||
created_by="test",
|
||||
tags="[]",
|
||||
allowed_tools="[]",
|
||||
)
|
||||
st.create_prompt_template(
|
||||
template_id="s-nonempty",
|
||||
name="nonempty-skill",
|
||||
category="ops",
|
||||
content="",
|
||||
variables="[]",
|
||||
is_default=False,
|
||||
org_id="",
|
||||
created_by="test",
|
||||
tags="[]",
|
||||
allowed_tools='["read_file"]',
|
||||
)
|
||||
client = _make_read_client(st)
|
||||
result = client.list_skills()
|
||||
by_name = {s["name"]: s for s in result["skills"]}
|
||||
assert "allowed_tools" not in by_name["empty-skill"]
|
||||
assert by_name["nonempty-skill"]["allowed_tools"] == ["read_file"]
|
||||
|
||||
|
||||
def test_list_skills_projects_allowed_tools_capped_with_sentinel(tmp_path):
|
||||
"""Each row carries the skill's allowed_tools (capped at the projection
|
||||
cap with a +N more sentinel) so coordinators can pick a skill without
|
||||
@@ -2616,3 +2659,362 @@ def test_cleanup_dead_task_child_refs_storage_batch_failure_swallows(populated_s
|
||||
|
||||
populated_storage.get_workstreams_batch = _boom # type: ignore[method-assign]
|
||||
assert client.cleanup_dead_task_child_refs("coord-1") == 0
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# inspect_workstream — three-tier output compression
|
||||
# ---------------------------------------------------------------------------
|
||||
#
|
||||
# A coord doing a fan-out wave against tool-heavy children would
|
||||
# otherwise blow the context budget on raw output alone. Mirrors the
|
||||
# search tool's Tier-1/Tier-2/Tier-3 ladder.
|
||||
|
||||
|
||||
def _make_inspect_result(
|
||||
*, ws_id: str = "ws-test", state: str = "running", n_messages: int = 5
|
||||
) -> dict[str, Any]:
|
||||
"""Build an inspect-result dict shaped like ``coordinator_client.inspect()``.
|
||||
|
||||
Production output keys (``ws_id``, ``skill_id``) mirror the storage
|
||||
row that ``inspect()`` spreads from ``get_workstream``. Tests that
|
||||
synthesize an inspect result must match these keys — otherwise a
|
||||
formatter that looks at the production keys silently emits null
|
||||
values against a fixture that uses different ones (real bug-1
|
||||
regression source: skeleton tier read ``skill`` from a fixture
|
||||
that wrote ``skill`` while production wrote ``skill_id``).
|
||||
"""
|
||||
return {
|
||||
"ws_id": ws_id,
|
||||
"state": state,
|
||||
"title": "test workstream",
|
||||
"skill_id": "researcher",
|
||||
"messages": [
|
||||
{"role": "user" if i % 2 == 0 else "assistant", "content": f"msg {i} content"}
|
||||
for i in range(n_messages)
|
||||
],
|
||||
"verdicts": [],
|
||||
}
|
||||
|
||||
|
||||
def test_format_inspect_tiered_full_fits_returns_full_tier():
|
||||
"""Small payloads pass through with `_tier='full'` — no compression."""
|
||||
from turnstone.console.coordinator_client import _format_inspect_tiered
|
||||
|
||||
result = _make_inspect_result(n_messages=3)
|
||||
out = _format_inspect_tiered(result)
|
||||
parsed = json.loads(out)
|
||||
assert parsed["_tier"] == "full"
|
||||
# Every message verbatim.
|
||||
assert len(parsed["messages"]) == 3
|
||||
assert parsed["messages"][0]["content"] == "msg 0 content"
|
||||
|
||||
|
||||
def test_format_inspect_tiered_compact_when_full_exceeds_budget():
|
||||
"""Large messages trigger the compact tier — head/tail-snipped
|
||||
content with the rest of the row intact."""
|
||||
from turnstone.console.coordinator_client import (
|
||||
_INSPECT_MSG_CONTENT_HEAD,
|
||||
_INSPECT_MSG_CONTENT_TAIL,
|
||||
_INSPECT_OUTPUT_BUDGET,
|
||||
_format_inspect_tiered,
|
||||
)
|
||||
|
||||
# Each message ~5KB; with 20 messages, full tier blows the 32KB budget.
|
||||
fat = "X" * 5000
|
||||
result = {
|
||||
"id": "ws-fat",
|
||||
"state": "running",
|
||||
"messages": [{"role": "assistant", "content": fat} for _ in range(20)],
|
||||
"verdicts": [],
|
||||
}
|
||||
out = _format_inspect_tiered(result)
|
||||
parsed = json.loads(out)
|
||||
assert parsed["_tier"] == "compact"
|
||||
# Every message preserved (compact keeps the count, just snips content).
|
||||
assert len(parsed["messages"]) == 20
|
||||
# Head/tail snip kicked in.
|
||||
msg_content = parsed["messages"][0]["content"]
|
||||
assert msg_content.startswith("X" * _INSPECT_MSG_CONTENT_HEAD)
|
||||
assert msg_content.endswith("X" * _INSPECT_MSG_CONTENT_TAIL)
|
||||
assert "chars elided" in msg_content
|
||||
# Budget invariant — the load-bearing contract of the formatter.
|
||||
# Without this assertion, a future change to ``_tier_note`` or
|
||||
# ``_compact_message`` could push the output over budget and the
|
||||
# ``_truncate_output`` head+tail safety net would silently mask
|
||||
# the regression, re-introducing the middle-message-drop pathology.
|
||||
assert len(out) <= _INSPECT_OUTPUT_BUDGET
|
||||
|
||||
|
||||
def test_format_inspect_tiered_compact_when_content_below_snip_threshold():
|
||||
"""When per-message content is below the snip threshold but the
|
||||
message COUNT alone overflows the budget, compact tier must still
|
||||
stay within budget — by trimming the message list (head + tail of
|
||||
messages) rather than degrading straight to skeleton. Bug-3
|
||||
regression cover: with 400 × 100-char messages, the original
|
||||
formatter fell through to skeleton because adding ``_tier_note``
|
||||
to an un-snipped tier-2 produced output strictly larger than
|
||||
tier-1 (both over budget). The fix preserves messages from both
|
||||
ends of the list and inserts an ``_omitted`` sentinel."""
|
||||
from turnstone.console.coordinator_client import (
|
||||
_INSPECT_OUTPUT_BUDGET,
|
||||
_format_inspect_tiered,
|
||||
)
|
||||
|
||||
# 400 × ~100 chars → Tier-1 ~53 KB (over budget), per-message
|
||||
# content under the 964-char snip threshold so content-snipping
|
||||
# saves nothing. Without the list-trim rung the formatter would
|
||||
# fall to skeleton and drop all 400 messages.
|
||||
smallish = "S" * 100
|
||||
result = {
|
||||
"ws_id": "ws-many-small",
|
||||
"state": "running",
|
||||
"messages": [
|
||||
{"role": "assistant" if i % 2 == 0 else "user", "content": smallish} for i in range(400)
|
||||
],
|
||||
"verdicts": [],
|
||||
}
|
||||
out = _format_inspect_tiered(result)
|
||||
parsed = json.loads(out)
|
||||
# Should NOT fall through to skeleton — message-list trim preserves
|
||||
# head + tail of the conversation.
|
||||
assert parsed["_tier"] == "compact"
|
||||
assert "messages" in parsed
|
||||
# Some messages must survive; the trim shape is head + tail with an
|
||||
# ``_omitted`` sentinel between them.
|
||||
assert len(parsed["messages"]) > 0
|
||||
assert len(parsed["messages"]) < 400
|
||||
# Budget invariant.
|
||||
assert len(out) <= _INSPECT_OUTPUT_BUDGET
|
||||
|
||||
|
||||
def test_format_inspect_tiered_skeleton_when_compact_also_exceeds_budget():
|
||||
"""Tier 3 fallback: counts + last assistant preview only. Trigger by
|
||||
flooding with messages whose content is a multi-block list — the
|
||||
snipper correctly leaves non-string content unchanged (mirrors
|
||||
Anthropic/OpenAI multi-block content shape), so even after the
|
||||
(5, 10) message-list trim the surviving 15 messages don't fit in
|
||||
the 32 KB budget."""
|
||||
from turnstone.console.coordinator_client import (
|
||||
_INSPECT_OUTPUT_BUDGET,
|
||||
_format_inspect_tiered,
|
||||
)
|
||||
|
||||
# 50 messages × multi-block content (~30 KB each — list-shape
|
||||
# content bypasses the head/tail string snipper because lists
|
||||
# aren't strings). Even (5, 10) trim leaves 15 × 30 KB which
|
||||
# blows the 32 KB budget — forces skeleton.
|
||||
fat_block = {"type": "text", "text": "Y" * 3000}
|
||||
result = {
|
||||
"ws_id": "ws-flood",
|
||||
"state": "running",
|
||||
"title": "flood",
|
||||
"skill_id": "researcher",
|
||||
"messages": [
|
||||
{
|
||||
"role": "assistant" if i % 2 == 0 else "user",
|
||||
"content": [fat_block] * 10,
|
||||
}
|
||||
for i in range(50)
|
||||
],
|
||||
"verdicts": [],
|
||||
}
|
||||
out = _format_inspect_tiered(result)
|
||||
parsed = json.loads(out)
|
||||
assert parsed["_tier"] == "skeleton"
|
||||
assert parsed["message_count"] == 50
|
||||
# Role distribution surfaces — the "what shape of activity" signal.
|
||||
assert parsed["roles"]["assistant"] == 25
|
||||
assert parsed["roles"]["user"] == 25
|
||||
# No `messages` field at skeleton tier — only the aggregate signal.
|
||||
assert "messages" not in parsed
|
||||
# Budget invariant.
|
||||
assert len(out) <= _INSPECT_OUTPUT_BUDGET
|
||||
|
||||
|
||||
def test_format_inspect_tiered_skeleton_keeps_terminal_state_fields():
|
||||
"""``close_reason`` / ``last_error`` survive the skeleton fall — they're
|
||||
small, load-bearing, and the operator needs them to understand WHY
|
||||
a terminal child landed in its state."""
|
||||
from turnstone.console.coordinator_client import (
|
||||
_INSPECT_OUTPUT_BUDGET,
|
||||
_format_inspect_tiered,
|
||||
)
|
||||
|
||||
# Same flood pattern as the bare-skeleton test (multi-block content
|
||||
# bypasses the string snipper) — paired with terminal-state fields
|
||||
# that must survive the skeleton fall.
|
||||
fat_block = {"type": "text", "text": "Z" * 3000}
|
||||
result = {
|
||||
"ws_id": "ws-closed",
|
||||
"state": "closed",
|
||||
"title": "done",
|
||||
"skill_id": "researcher",
|
||||
"messages": [{"role": "user", "content": [fat_block] * 10} for _ in range(50)],
|
||||
"verdicts": [],
|
||||
"close_reason": "task complete: report attached",
|
||||
"live": None, # filtered by truthy check
|
||||
}
|
||||
out = _format_inspect_tiered(result)
|
||||
parsed = json.loads(out)
|
||||
assert parsed["_tier"] == "skeleton"
|
||||
assert parsed["close_reason"] == "task complete: report attached"
|
||||
# Falsy ``live`` doesn't bleed through.
|
||||
assert "live" not in parsed
|
||||
assert len(out) <= _INSPECT_OUTPUT_BUDGET
|
||||
|
||||
|
||||
def test_format_inspect_tiered_error_shapes_bypass_tiering():
|
||||
"""Cross-tenant / not-found responses keep their original shape — they
|
||||
carry no messages, are already tiny, and changing them would break
|
||||
callers that key on the ``error`` field."""
|
||||
from turnstone.console.coordinator_client import _format_inspect_tiered
|
||||
|
||||
result = {"error": "workstream not found", "ws_id": "ws-foreign"}
|
||||
out = _format_inspect_tiered(result)
|
||||
parsed = json.loads(out)
|
||||
assert parsed == {"error": "workstream not found", "ws_id": "ws-foreign"}
|
||||
# No `_tier` annotation — error shapes are self-describing.
|
||||
assert "_tier" not in parsed
|
||||
|
||||
|
||||
def test_format_inspect_tiered_compact_preserves_tool_call_linkage():
|
||||
"""Compact tier keeps ``tool_name`` / ``tool_call_id`` / ``name`` so a
|
||||
model reading the snipped trace can still pair a tool call to its
|
||||
response — the linkage is load-bearing for "what happened" signal."""
|
||||
from turnstone.console.coordinator_client import _format_inspect_tiered
|
||||
|
||||
fat = "Q" * 5000
|
||||
result = {
|
||||
"ws_id": "ws-tools",
|
||||
"state": "running",
|
||||
"messages": [
|
||||
{
|
||||
"role": "assistant",
|
||||
"content": fat,
|
||||
"tool_name": "bash",
|
||||
"tool_call_id": "call-1",
|
||||
}
|
||||
for _ in range(20)
|
||||
],
|
||||
"verdicts": [],
|
||||
}
|
||||
out = _format_inspect_tiered(result)
|
||||
parsed = json.loads(out)
|
||||
assert parsed["_tier"] == "compact"
|
||||
first = parsed["messages"][0]
|
||||
assert first["tool_name"] == "bash"
|
||||
assert first["tool_call_id"] == "call-1"
|
||||
|
||||
|
||||
def test_format_inspect_tiered_compact_preserves_assistant_tool_calls():
|
||||
"""Compact tier must preserve the assistant-side ``tool_calls`` list
|
||||
(OpenAI shape: ``[{id, type, function: {name, arguments}}]``) so a
|
||||
model reading the snipped trace can see WHICH tool was called and
|
||||
pair it with the corresponding result row via ``id`` ↔ ``tool_call_id``.
|
||||
Bug-2 regression cover: the pre-fix compactor stripped ``tool_calls``,
|
||||
leaving the audit reader with a tool-result orphan against an
|
||||
invisible call.
|
||||
|
||||
``function.arguments`` strings are snipped head/tail (analogous to
|
||||
content) because they can be multi-KB JSON; ``id`` and
|
||||
``function.name`` are preserved verbatim — they're the linkage."""
|
||||
from turnstone.console.coordinator_client import (
|
||||
_INSPECT_TOOL_ARG_HEAD,
|
||||
_INSPECT_TOOL_ARG_TAIL,
|
||||
_format_inspect_tiered,
|
||||
)
|
||||
|
||||
fat_content = "C" * 5000 # forces compact tier
|
||||
fat_args = "A" * 5000 # forces argument snipping
|
||||
tool_calls = [
|
||||
{
|
||||
"id": "call-abc-123",
|
||||
"type": "function",
|
||||
"function": {"name": "bash", "arguments": fat_args},
|
||||
},
|
||||
{
|
||||
"id": "call-def-456",
|
||||
"type": "function",
|
||||
"function": {"name": "read_file", "arguments": fat_args},
|
||||
},
|
||||
]
|
||||
result = {
|
||||
"ws_id": "ws-tool-calls",
|
||||
"state": "running",
|
||||
"messages": [
|
||||
{"role": "assistant", "content": fat_content, "tool_calls": tool_calls}
|
||||
for _ in range(20)
|
||||
],
|
||||
"verdicts": [],
|
||||
}
|
||||
out = _format_inspect_tiered(result)
|
||||
parsed = json.loads(out)
|
||||
assert parsed["_tier"] == "compact"
|
||||
first = parsed["messages"][0]
|
||||
# tool_calls survives compaction.
|
||||
assert "tool_calls" in first
|
||||
assert len(first["tool_calls"]) == 2
|
||||
# Linkage fields verbatim.
|
||||
assert first["tool_calls"][0]["id"] == "call-abc-123"
|
||||
assert first["tool_calls"][0]["function"]["name"] == "bash"
|
||||
assert first["tool_calls"][1]["id"] == "call-def-456"
|
||||
assert first["tool_calls"][1]["function"]["name"] == "read_file"
|
||||
# arguments snipped head/tail — both prefix and suffix preserved.
|
||||
snipped_args = first["tool_calls"][0]["function"]["arguments"]
|
||||
assert snipped_args.startswith("A" * _INSPECT_TOOL_ARG_HEAD)
|
||||
assert snipped_args.endswith("A" * _INSPECT_TOOL_ARG_TAIL)
|
||||
assert "chars elided" in snipped_args
|
||||
|
||||
|
||||
def test_format_inspect_tiered_compact_passes_small_messages_through_unsnipped():
|
||||
"""Messages under the snip threshold pass through verbatim at compact
|
||||
tier — snipping a 100-byte message costs more bytes (the elision
|
||||
marker) than it saves."""
|
||||
from turnstone.console.coordinator_client import _format_inspect_tiered
|
||||
|
||||
# Mix: a few large messages force compact tier; small messages must
|
||||
# not be snipped.
|
||||
big = "B" * 5000
|
||||
small = "S" * 50
|
||||
result = {
|
||||
"id": "ws-mixed",
|
||||
"state": "running",
|
||||
"messages": [{"role": "assistant", "content": big} for _ in range(15)]
|
||||
+ [{"role": "user", "content": small}],
|
||||
"verdicts": [],
|
||||
}
|
||||
out = _format_inspect_tiered(result)
|
||||
parsed = json.loads(out)
|
||||
assert parsed["_tier"] == "compact"
|
||||
# The trailing small message is exact, not snipped.
|
||||
assert parsed["messages"][-1]["content"] == small
|
||||
|
||||
|
||||
def test_format_inspect_tiered_emits_tier_note_when_compressed():
|
||||
"""The ``_tier_note`` advisory tells the LLM how to ask for a tighter
|
||||
or fuller view next time — actionable feedback rather than a bare
|
||||
"we compressed your output" signal."""
|
||||
from turnstone.console.coordinator_client import _format_inspect_tiered
|
||||
|
||||
fat = "F" * 5000
|
||||
result = {
|
||||
"id": "ws-noted",
|
||||
"state": "running",
|
||||
"messages": [{"role": "assistant", "content": fat} for _ in range(20)],
|
||||
"verdicts": [],
|
||||
}
|
||||
out = _format_inspect_tiered(result)
|
||||
parsed = json.loads(out)
|
||||
assert "_tier_note" in parsed
|
||||
assert "message_limit" in parsed["_tier_note"]
|
||||
|
||||
|
||||
def test_format_inspect_tiered_full_tier_omits_tier_note():
|
||||
"""When the full tier fits, no note is emitted — the absence of a
|
||||
note is the signal that nothing was compressed."""
|
||||
from turnstone.console.coordinator_client import _format_inspect_tiered
|
||||
|
||||
out = _format_inspect_tiered(_make_inspect_result(n_messages=2))
|
||||
parsed = json.loads(out)
|
||||
assert parsed["_tier"] == "full"
|
||||
assert "_tier_note" not in parsed
|
||||
|
||||
@@ -215,7 +215,12 @@ def test_spawn_exec_does_not_surface_misleading_status_field(coord_session):
|
||||
summary tempted callers to write ``if result["status"] == "idle"``
|
||||
which silently never matched. The summary now omits the field
|
||||
entirely; lifecycle state lives on the workstream row and is read
|
||||
via inspect_workstream."""
|
||||
via inspect_workstream.
|
||||
|
||||
Also asserts the return key is ``child_ws_id`` (not ``ws_id``) so
|
||||
the coordinator LLM doesn't recency-bias toward feeding the spawn
|
||||
output back into another ``spawn_workstream(ws_id=...)`` call.
|
||||
"""
|
||||
sess, coord, _ui = coord_session
|
||||
coord.spawn.return_value = {
|
||||
"ws_id": "child-7",
|
||||
@@ -227,8 +232,9 @@ def test_spawn_exec_does_not_surface_misleading_status_field(coord_session):
|
||||
_call_id, output = sess._exec_spawn_workstream(item)
|
||||
body = json.loads(output)
|
||||
assert "status" not in body
|
||||
assert "ws_id" not in body
|
||||
# The substantive fields are still here.
|
||||
assert body["ws_id"] == "child-7"
|
||||
assert body["child_ws_id"] == "child-7"
|
||||
assert body["node_id"] == "node-1"
|
||||
|
||||
|
||||
@@ -248,6 +254,10 @@ def test_spawn_batch_exec_does_not_surface_misleading_status_field(coord_session
|
||||
body = json.loads(output)
|
||||
assert "0" in body["results"]
|
||||
assert "status" not in body["results"]["0"]
|
||||
# Per-result entries surface ``child_ws_id``, not ``ws_id`` — same
|
||||
# recency-bias rationale as the spawn_workstream test above.
|
||||
assert body["results"]["0"]["child_ws_id"] == "c-x"
|
||||
assert "ws_id" not in body["results"]["0"]
|
||||
|
||||
|
||||
def test_spawn_exec_surfaces_client_error(coord_session):
|
||||
@@ -260,6 +270,21 @@ def test_spawn_exec_surfaces_client_error(coord_session):
|
||||
assert ui.tool_results[-1][3] is True # is_error
|
||||
|
||||
|
||||
def test_spawn_exec_treats_missing_ws_id_on_success_path_as_error(coord_session):
|
||||
"""A malformed upstream response (200-success-shape with no
|
||||
``ws_id``) used to emit ``{"child_ws_id": null}`` to the LLM,
|
||||
which then chased a null id through follow-up tools. Now matches
|
||||
the matching guard in ``_exec_spawn_batch``: surface as a tool
|
||||
error so the model retries instead of acting on garbage."""
|
||||
sess, coord, ui = coord_session
|
||||
# No ``error`` field, but ``ws_id`` is missing — the silent-null path.
|
||||
coord.spawn.return_value = {"name": "c", "node_id": "node-1", "status": 200}
|
||||
item = sess._prepare_tool(_tc("spawn_workstream", {"initial_message": "hi"}))
|
||||
_call_id, output = sess._exec_spawn_workstream(item)
|
||||
assert "no ws_id" in output
|
||||
assert ui.tool_results[-1][3] is True # is_error
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# inspect_workstream
|
||||
# ---------------------------------------------------------------------------
|
||||
@@ -1410,9 +1435,13 @@ def test_spawn_batch_exec_serialises_spawns_and_returns_results(coord_session):
|
||||
assert body["denied"] == []
|
||||
# Keyed by input index (stringified).
|
||||
assert set(body["results"].keys()) == {"0", "1", "2"}
|
||||
assert body["results"]["0"]["ws_id"] == "child-0"
|
||||
assert body["results"]["0"]["child_ws_id"] == "child-0"
|
||||
assert body["results"]["1"]["node_id"] == "n-1"
|
||||
assert body["results"]["2"]["ws_id"] == "child-2"
|
||||
assert body["results"]["2"]["child_ws_id"] == "child-2"
|
||||
# Confirm we don't leak the old ``ws_id`` key alongside the new
|
||||
# ``child_ws_id`` — see test_spawn_exec_does_not_surface_misleading_status_field
|
||||
# for the rationale on the rename.
|
||||
assert "ws_id" not in body["results"]["0"]
|
||||
|
||||
|
||||
def test_spawn_batch_exec_surfaces_per_item_errors_in_denied(coord_session):
|
||||
|
||||
+122
-1
@@ -51,7 +51,11 @@ class TestIntentVerdictCRUD:
|
||||
assert v["tier"] == "heuristic"
|
||||
assert v["judge_model"] == ""
|
||||
assert v["latency_ms"] == 2
|
||||
assert v["user_decision"] == ""
|
||||
# ``user_decision`` defaults to ``"pending"`` (not the empty
|
||||
# string) so an audit reader can distinguish in-flight rows
|
||||
# from pre-convention legacy rows that carry the column's
|
||||
# server_default of ``""``.
|
||||
assert v["user_decision"] == "pending"
|
||||
assert "created" in v
|
||||
|
||||
def test_get_nonexistent(self, db):
|
||||
@@ -114,6 +118,123 @@ class TestIntentVerdictCRUD:
|
||||
assert ok is False
|
||||
|
||||
|
||||
class TestIntentVerdictUpsert:
|
||||
"""``upsert_intent_verdict`` — the LLM-tier-aware persistence path.
|
||||
|
||||
Backs the heuristic → llm_fallback "upgrade in place" pattern.
|
||||
The async judge's fallback verdicts deliberately reuse the
|
||||
heuristic ``verdict_id``; a plain INSERT would collide on the
|
||||
PK and the upgrade would be lost to a silently-swallowed
|
||||
exception (Postgres logged ``intent_verdicts_pkey`` violations
|
||||
for every fallback delivery on stable/1.5 smoke tests).
|
||||
"""
|
||||
|
||||
def test_upsert_on_fresh_id_inserts(self, db):
|
||||
"""No conflict — behaves like a regular INSERT."""
|
||||
db.upsert_intent_verdict(**_make_verdict_kwargs())
|
||||
v = db.get_intent_verdict("v_001")
|
||||
assert v is not None
|
||||
assert v["tier"] == "heuristic"
|
||||
assert v["user_decision"] == "pending"
|
||||
|
||||
def test_upsert_on_conflict_upgrades_tier_reasoning_judge_model(self, db):
|
||||
"""On PK conflict: tier, reasoning, judge_model update — every
|
||||
other field is preserved. Mirrors what the judge emits when
|
||||
promoting heuristic → llm_fallback."""
|
||||
db.upsert_intent_verdict(
|
||||
**_make_verdict_kwargs(
|
||||
tier="heuristic",
|
||||
reasoning="initial heuristic reasoning",
|
||||
judge_model="",
|
||||
)
|
||||
)
|
||||
db.upsert_intent_verdict(
|
||||
**_make_verdict_kwargs(
|
||||
tier="llm_fallback",
|
||||
reasoning="initial heuristic reasoning (LLM judge did not return a verdict)",
|
||||
judge_model="gpt-5-judge",
|
||||
)
|
||||
)
|
||||
v = db.get_intent_verdict("v_001")
|
||||
assert v is not None
|
||||
# The three fields that should change.
|
||||
assert v["tier"] == "llm_fallback"
|
||||
assert "LLM judge did not return" in v["reasoning"]
|
||||
assert v["judge_model"] == "gpt-5-judge"
|
||||
|
||||
def test_upsert_on_conflict_preserves_user_decision(self, db):
|
||||
"""LOAD-BEARING: a manually-resolved approval (user_decision=
|
||||
``"approved"``) or auto-approve-stamped row (user_decision=
|
||||
``"policy"``/``"blanket"``/etc.) must NOT be clobbered back to
|
||||
``"pending"`` when the late LLM-fallback verdict lands.
|
||||
``IntentVerdict.to_dict()`` doesn't project user_decision, so
|
||||
the upsert's defaulted ``"pending"`` would silently overwrite
|
||||
the real value if user_decision were in the on-conflict
|
||||
SET clause."""
|
||||
db.upsert_intent_verdict(**_make_verdict_kwargs())
|
||||
ok = db.update_intent_verdict("v_001", user_decision="approved")
|
||||
assert ok is True
|
||||
# Simulate the late LLM-fallback delivery — same verdict_id,
|
||||
# default user_decision (the IntentVerdict.to_dict() shape).
|
||||
db.upsert_intent_verdict(
|
||||
**_make_verdict_kwargs(
|
||||
tier="llm_fallback",
|
||||
reasoning="extended (LLM judge did not return a verdict)",
|
||||
judge_model="gpt-5-judge",
|
||||
)
|
||||
)
|
||||
v = db.get_intent_verdict("v_001")
|
||||
assert v is not None
|
||||
assert v["user_decision"] == "approved" # NOT clobbered to "pending"
|
||||
assert v["tier"] == "llm_fallback" # but the upgrade did land
|
||||
|
||||
def test_upsert_on_conflict_preserves_identity_and_carried_fields(self, db):
|
||||
"""Identity columns (ws_id, call_id, func_name, func_args) and
|
||||
carried-verbatim columns (intent_summary, risk_level,
|
||||
confidence, recommendation, evidence, latency_ms) are
|
||||
excluded from the on-conflict SET — verify they aren't
|
||||
changed even when the second upsert passes different values
|
||||
(defensive against a future judge bug that ships divergent
|
||||
carried fields)."""
|
||||
db.upsert_intent_verdict(**_make_verdict_kwargs())
|
||||
db.upsert_intent_verdict(
|
||||
**_make_verdict_kwargs(
|
||||
# Same verdict_id (conflict trigger), divergent everything else.
|
||||
ws_id="ws-different",
|
||||
call_id="tc_different",
|
||||
func_name="bash_v2",
|
||||
func_args='{"command":"rm -rf /"}',
|
||||
intent_summary="totally different summary",
|
||||
risk_level="critical",
|
||||
confidence=0.0,
|
||||
recommendation="deny",
|
||||
evidence='["dangerous"]',
|
||||
latency_ms=99999,
|
||||
# The three fields that DO update.
|
||||
tier="llm_fallback",
|
||||
reasoning="upgraded reasoning",
|
||||
judge_model="judge-v2",
|
||||
)
|
||||
)
|
||||
v = db.get_intent_verdict("v_001")
|
||||
assert v is not None
|
||||
# All preserved from the first upsert (identity + carried).
|
||||
assert v["ws_id"] == "ws-abc"
|
||||
assert v["call_id"] == "tc_001"
|
||||
assert v["func_name"] == "bash"
|
||||
assert v["func_args"] == '{"command":"echo hello"}'
|
||||
assert v["intent_summary"] == "Echo a greeting to stdout"
|
||||
assert v["risk_level"] == "low"
|
||||
assert v["confidence"] == 0.85
|
||||
assert v["recommendation"] == "approve"
|
||||
assert v["evidence"] == '["The command only prints text."]'
|
||||
assert v["latency_ms"] == 2
|
||||
# Only the three updated.
|
||||
assert v["tier"] == "llm_fallback"
|
||||
assert v["reasoning"] == "upgraded reasoning"
|
||||
assert v["judge_model"] == "judge-v2"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Bulk insert
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
@@ -0,0 +1,211 @@
|
||||
"""Integration tests for the Phase 9 admin bulk-revoke endpoint.
|
||||
|
||||
POST /v1/api/admin/mcp-servers/{name}/bulk-revoke clears every user's
|
||||
OAuth token for a server (admin-side counterpart to the per-user
|
||||
DELETE /v1/api/mcp/oauth/connections/{server_name} that shipped in
|
||||
Phase 8).
|
||||
|
||||
Coverage:
|
||||
- requires ``admin.mcp`` permission (401/403 without).
|
||||
- 404 when the named server is missing.
|
||||
- 400 when the server's ``auth_type`` is not ``oauth_user``.
|
||||
- 200 + ``rows_deleted`` + ``consented_users_before`` on success.
|
||||
- Audit row written with
|
||||
``upstream_revoke_outcome="bulk_admin_no_upstream"``.
|
||||
- Token rows are gone from ``mcp_user_tokens`` post-call.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
import pytest
|
||||
from starlette.applications import Starlette
|
||||
from starlette.middleware import Middleware
|
||||
from starlette.middleware.base import BaseHTTPMiddleware
|
||||
from starlette.routing import Mount, Route
|
||||
from starlette.testclient import TestClient
|
||||
|
||||
from turnstone.console.server import admin_mcp_bulk_revoke
|
||||
from turnstone.core.auth import AuthResult
|
||||
from turnstone.core.storage._sqlite import SQLiteBackend
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from starlette.requests import Request
|
||||
from starlette.responses import Response
|
||||
|
||||
|
||||
class _InjectAdminMcp(BaseHTTPMiddleware):
|
||||
async def dispatch(self, request: Request, call_next: Any) -> Response:
|
||||
request.state.auth_result = AuthResult(
|
||||
user_id="admin-user",
|
||||
scopes=frozenset({"approve"}),
|
||||
token_source="config",
|
||||
permissions=frozenset({"read", "write", "approve", "admin.mcp"}),
|
||||
)
|
||||
return await call_next(request)
|
||||
|
||||
|
||||
class _InjectNoAdminMcp(BaseHTTPMiddleware):
|
||||
async def dispatch(self, request: Request, call_next: Any) -> Response:
|
||||
request.state.auth_result = AuthResult(
|
||||
user_id="regular-user",
|
||||
scopes=frozenset({"approve"}),
|
||||
token_source="jwt",
|
||||
permissions=frozenset({"read", "write", "approve"}),
|
||||
)
|
||||
return await call_next(request)
|
||||
|
||||
|
||||
def _build_app(storage: SQLiteBackend, *, with_admin_mcp: bool = True) -> Starlette:
|
||||
mw = _InjectAdminMcp if with_admin_mcp else _InjectNoAdminMcp
|
||||
app = Starlette(
|
||||
routes=[
|
||||
Mount(
|
||||
"/v1",
|
||||
routes=[
|
||||
Route(
|
||||
"/api/admin/mcp-servers/{name}/bulk-revoke",
|
||||
admin_mcp_bulk_revoke,
|
||||
methods=["POST"],
|
||||
),
|
||||
],
|
||||
),
|
||||
],
|
||||
middleware=[Middleware(mw)],
|
||||
)
|
||||
app.state.auth_storage = storage
|
||||
return app
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def storage(tmp_path: Any) -> SQLiteBackend:
|
||||
return SQLiteBackend(str(tmp_path / "test.db"))
|
||||
|
||||
|
||||
def _seed_oauth_server(
|
||||
backend: SQLiteBackend,
|
||||
*,
|
||||
name: str = "srv-oauth",
|
||||
server_id: str = "srv-oauth-id",
|
||||
) -> None:
|
||||
backend.create_mcp_server(
|
||||
server_id=server_id,
|
||||
name=name,
|
||||
transport="streamable-http",
|
||||
url="https://example.com/mcp",
|
||||
auth_type="oauth_user",
|
||||
)
|
||||
|
||||
|
||||
def _seed_static_server(
|
||||
backend: SQLiteBackend,
|
||||
*,
|
||||
name: str = "srv-static",
|
||||
server_id: str = "srv-static-id",
|
||||
) -> None:
|
||||
backend.create_mcp_server(
|
||||
server_id=server_id,
|
||||
name=name,
|
||||
transport="streamable-http",
|
||||
url="https://example.com/mcp",
|
||||
auth_type="static",
|
||||
)
|
||||
|
||||
|
||||
def _seed_user_tokens(backend: SQLiteBackend, server_name: str, users: int) -> None:
|
||||
for i in range(users):
|
||||
backend.create_mcp_user_token(
|
||||
f"user-{i}",
|
||||
server_name,
|
||||
access_token_ct=b"ct",
|
||||
refresh_token_ct=None,
|
||||
expires_at=None,
|
||||
scopes=None,
|
||||
as_issuer="https://as.example.com",
|
||||
audience="https://example.com/mcp",
|
||||
)
|
||||
|
||||
|
||||
def test_requires_admin_mcp_permission(storage: SQLiteBackend) -> None:
|
||||
_seed_oauth_server(storage)
|
||||
client = TestClient(_build_app(storage, with_admin_mcp=False))
|
||||
resp = client.post("/v1/api/admin/mcp-servers/srv-oauth/bulk-revoke")
|
||||
assert resp.status_code == 403
|
||||
|
||||
|
||||
def test_404_on_missing_server(storage: SQLiteBackend) -> None:
|
||||
client = TestClient(_build_app(storage))
|
||||
resp = client.post("/v1/api/admin/mcp-servers/never-existed/bulk-revoke")
|
||||
assert resp.status_code == 404
|
||||
assert resp.json() == {"error": "No such server"}
|
||||
|
||||
|
||||
def test_400_on_static_server(storage: SQLiteBackend) -> None:
|
||||
_seed_static_server(storage)
|
||||
client = TestClient(_build_app(storage))
|
||||
resp = client.post("/v1/api/admin/mcp-servers/srv-static/bulk-revoke")
|
||||
assert resp.status_code == 400
|
||||
body = resp.json()
|
||||
assert "oauth_user" in body["error"]
|
||||
|
||||
|
||||
def test_400_on_invalid_server_name(storage: SQLiteBackend) -> None:
|
||||
# double-underscore is reserved for the prefixed-tool-name encoding.
|
||||
client = TestClient(_build_app(storage))
|
||||
resp = client.post("/v1/api/admin/mcp-servers/bad__name/bulk-revoke")
|
||||
assert resp.status_code == 400
|
||||
|
||||
|
||||
def test_200_on_success_with_no_consented_users(storage: SQLiteBackend) -> None:
|
||||
_seed_oauth_server(storage)
|
||||
client = TestClient(_build_app(storage))
|
||||
resp = client.post("/v1/api/admin/mcp-servers/srv-oauth/bulk-revoke")
|
||||
assert resp.status_code == 200
|
||||
body = resp.json()
|
||||
assert body["status"] == "ok"
|
||||
assert body["rows_deleted"] == 0
|
||||
assert body["consented_users_before"] == 0
|
||||
|
||||
|
||||
def test_200_clears_all_user_tokens(storage: SQLiteBackend) -> None:
|
||||
_seed_oauth_server(storage)
|
||||
_seed_user_tokens(storage, "srv-oauth", users=3)
|
||||
# Token for another server must survive the bulk-revoke.
|
||||
_seed_oauth_server(storage, name="srv-other", server_id="srv-other-id")
|
||||
_seed_user_tokens(storage, "srv-other", users=2)
|
||||
|
||||
client = TestClient(_build_app(storage))
|
||||
resp = client.post("/v1/api/admin/mcp-servers/srv-oauth/bulk-revoke")
|
||||
assert resp.status_code == 200
|
||||
body = resp.json()
|
||||
assert body["status"] == "ok"
|
||||
assert body["rows_deleted"] == 3
|
||||
assert body["consented_users_before"] == 3
|
||||
|
||||
# Target server's tokens are gone; bystander's tokens survive.
|
||||
assert storage.count_mcp_consented_users_by_server("srv-oauth") == 0
|
||||
assert storage.count_mcp_consented_users_by_server("srv-other") == 2
|
||||
|
||||
|
||||
def test_audits_with_bulk_admin_no_upstream(storage: SQLiteBackend) -> None:
|
||||
_seed_oauth_server(storage)
|
||||
_seed_user_tokens(storage, "srv-oauth", users=2)
|
||||
client = TestClient(_build_app(storage))
|
||||
resp = client.post("/v1/api/admin/mcp-servers/srv-oauth/bulk-revoke")
|
||||
assert resp.status_code == 200
|
||||
|
||||
# Pull the most-recent audit row for the bulk_revoked action and
|
||||
# verify it carries the deferral marker.
|
||||
events = storage.list_audit_events(limit=10)
|
||||
bulk_rows = [e for e in events if e.get("action") == "mcp_server.oauth.bulk_revoked"]
|
||||
assert len(bulk_rows) == 1
|
||||
detail = bulk_rows[0].get("detail")
|
||||
if isinstance(detail, str):
|
||||
import json as _json
|
||||
|
||||
detail = _json.loads(detail)
|
||||
assert detail.get("upstream_revoke_outcome") == "bulk_admin_no_upstream"
|
||||
assert detail.get("rows_deleted") == 2
|
||||
assert detail.get("consented_users_before") == 2
|
||||
assert detail.get("name") == "srv-oauth"
|
||||
+146
-1
@@ -788,7 +788,11 @@ class TestSessionIntegration:
|
||||
assert call_id == "call_789"
|
||||
assert output == "result text"
|
||||
mock_mcp.call_tool_sync.assert_called_once_with(
|
||||
"mcp__test__search", {"query": "hello"}, user_id=None, timeout=30
|
||||
"mcp__test__search",
|
||||
{"query": "hello"},
|
||||
user_id=None,
|
||||
timeout=30,
|
||||
is_interactive_for_consent=True,
|
||||
)
|
||||
|
||||
def test_exec_mcp_tool_error(self, tmp_db):
|
||||
@@ -1056,6 +1060,147 @@ class TestRefreshServer:
|
||||
asyncio.run(_run())
|
||||
|
||||
|
||||
class TestLastRefreshTracking:
|
||||
"""Phase 9 admin status pill — ``_last_refresh`` is written on every
|
||||
refresh path so the admin UI reflects manual-refresh AND auto-
|
||||
reconnect outcomes uniformly. This test class pins the contract.
|
||||
"""
|
||||
|
||||
@staticmethod
|
||||
def _seed_minimal(mgr: MCPClientManager, name: str = "srv") -> MagicMock:
|
||||
mock_session = MagicMock()
|
||||
mock_session.list_tools = AsyncMock(return_value=MagicMock(tools=[]))
|
||||
mock_session.list_resources = AsyncMock(return_value=MagicMock(resources=[]))
|
||||
mock_session.list_resource_templates = AsyncMock(
|
||||
return_value=MagicMock(resourceTemplates=[])
|
||||
)
|
||||
mock_session.list_prompts = AsyncMock(return_value=MagicMock(prompts=[]))
|
||||
_seed_static_state(
|
||||
mgr,
|
||||
name,
|
||||
session=mock_session,
|
||||
tools=[],
|
||||
supports_resources=True,
|
||||
supports_prompts=True,
|
||||
)
|
||||
return mock_session
|
||||
|
||||
def test_last_refresh_written_on_success(self) -> None:
|
||||
async def _run() -> None:
|
||||
mgr = MCPClientManager({})
|
||||
self._seed_minimal(mgr)
|
||||
assert "srv" not in mgr._last_refresh
|
||||
|
||||
await mgr._refresh_server("srv")
|
||||
|
||||
entry = mgr._last_refresh.get("srv")
|
||||
assert entry is not None
|
||||
ts, outcome = entry
|
||||
assert outcome == "ok"
|
||||
assert isinstance(ts, float) and ts > 0
|
||||
|
||||
asyncio.run(_run())
|
||||
|
||||
def test_last_refresh_written_on_tool_refresh_failure(self) -> None:
|
||||
"""When ``_refresh_server_tools`` raises, the outcome reflects
|
||||
the exception class and the exception still propagates."""
|
||||
|
||||
async def _run() -> None:
|
||||
mgr = MCPClientManager({})
|
||||
mock_session = self._seed_minimal(mgr)
|
||||
mock_session.list_tools = AsyncMock(side_effect=RuntimeError("upstream down"))
|
||||
|
||||
with pytest.raises(RuntimeError, match="upstream down"):
|
||||
await mgr._refresh_server("srv")
|
||||
|
||||
entry = mgr._last_refresh.get("srv")
|
||||
assert entry is not None
|
||||
_, outcome = entry
|
||||
assert outcome == "error:RuntimeError"
|
||||
|
||||
asyncio.run(_run())
|
||||
|
||||
def test_last_refresh_records_first_exception_when_multiple_fail(
|
||||
self,
|
||||
) -> None:
|
||||
"""``return_exceptions=True`` lets sibling tasks complete; the
|
||||
outcome reflects the FIRST exception encountered."""
|
||||
|
||||
async def _run() -> None:
|
||||
mgr = MCPClientManager({})
|
||||
mock_session = self._seed_minimal(mgr)
|
||||
# Tools succeeds; resources raises first (gather preserves
|
||||
# argument order in its results list, so resources is the
|
||||
# first failure regardless of which awaitable finished first
|
||||
# in wall-clock terms).
|
||||
mock_session.list_resources = AsyncMock(side_effect=ValueError("res boom"))
|
||||
mock_session.list_prompts = AsyncMock(side_effect=KeyError("prompts boom"))
|
||||
|
||||
with pytest.raises((ValueError, KeyError)):
|
||||
await mgr._refresh_server("srv")
|
||||
|
||||
entry = mgr._last_refresh.get("srv")
|
||||
assert entry is not None
|
||||
_, outcome = entry
|
||||
# Either of the two failing tasks could be "first" in
|
||||
# gather's results list ordering — the order is positional
|
||||
# so resources (arg #2) comes before prompts (arg #3).
|
||||
assert outcome == "error:ValueError"
|
||||
|
||||
asyncio.run(_run())
|
||||
|
||||
def test_refresh_all_overwrites_stale_ok_on_reconnect_failure(
|
||||
self,
|
||||
) -> None:
|
||||
"""The chokepoint bug-1 fix: a prior successful refresh's ``'ok'``
|
||||
entry MUST be overwritten when a subsequent reconnect fails —
|
||||
otherwise the admin pill shows misleading "ok" while the server
|
||||
is in fact broken."""
|
||||
|
||||
async def _run() -> None:
|
||||
mgr = MCPClientManager({})
|
||||
# Server is configured but has no live session — _refresh_all
|
||||
# routes to the reconnect branch.
|
||||
mgr._server_configs["srv"] = {"type": "stdio", "command": "x"}
|
||||
# Pre-seed a stale "ok" from an earlier successful refresh.
|
||||
mgr._last_refresh["srv"] = (1000.0, "ok")
|
||||
|
||||
async def _raise(*_a: object, **_kw: object) -> None:
|
||||
raise ConnectionError("reconnect failed")
|
||||
|
||||
mgr._connect_one = _raise # type: ignore[assignment]
|
||||
|
||||
await mgr._refresh_all("srv")
|
||||
|
||||
entry = mgr._last_refresh.get("srv")
|
||||
assert entry is not None
|
||||
ts, outcome = entry
|
||||
# Outcome reflects the new failure, not the stale ok.
|
||||
assert outcome == "error:ConnectionError"
|
||||
assert ts > 1000.0
|
||||
|
||||
asyncio.run(_run())
|
||||
|
||||
def test_get_server_status_surfaces_last_refresh_fields(self) -> None:
|
||||
"""``get_server_status`` surfaces ``last_refresh_at`` and
|
||||
``last_refresh_outcome`` for the admin pill — null when no
|
||||
refresh has occurred yet, populated after one."""
|
||||
mgr = MCPClientManager({})
|
||||
mgr._server_configs["srv"] = {"type": "stdio", "command": "x"}
|
||||
|
||||
# No refresh yet — fields must be present and null so the JS
|
||||
# renderer can branch on absence cleanly.
|
||||
status = mgr.get_server_status("srv")
|
||||
assert status["last_refresh_at"] is None
|
||||
assert status["last_refresh_outcome"] is None
|
||||
|
||||
# Populate the tuple directly and re-read.
|
||||
mgr._last_refresh["srv"] = (12345.5, "ok")
|
||||
status = mgr.get_server_status("srv")
|
||||
assert status["last_refresh_at"] == 12345.5
|
||||
assert status["last_refresh_outcome"] == "ok"
|
||||
|
||||
|
||||
class TestListeners:
|
||||
def test_add_and_notify(self):
|
||||
mgr = MCPClientManager({})
|
||||
|
||||
@@ -612,6 +612,97 @@ class TestCallback:
|
||||
assert plain is not None
|
||||
assert plain["refresh_token"] is None
|
||||
|
||||
def test_callback_clears_pending_consent_on_success(
|
||||
self, storage: SQLiteBackend, http_client_mock: MagicMock
|
||||
) -> None:
|
||||
"""Successful callback must drop any ``mcp_pending_consent`` rows
|
||||
for the just-consented ``(user, server)`` (Phase 9 lifecycle
|
||||
contract). Regression guard for the dashboard-stays-stale-after-
|
||||
consent invariant.
|
||||
"""
|
||||
_seed_oauth_user_server(storage)
|
||||
self._seed_pending(storage)
|
||||
# Seed a deferred-consent record that a prior non-interactive run
|
||||
# would have left behind. Plus a cross-tenant record that must
|
||||
# NOT be touched.
|
||||
storage.upsert_mcp_pending_consent(
|
||||
user_id="user-1",
|
||||
server_name="srv-oauth",
|
||||
error_code="mcp_consent_required",
|
||||
scopes_required=None,
|
||||
last_ws_id="ws-1",
|
||||
last_tool_call_id="tool-1",
|
||||
now_iso="2026-05-11T12:00:00",
|
||||
)
|
||||
storage.upsert_mcp_pending_consent(
|
||||
user_id="other-user",
|
||||
server_name="srv-oauth",
|
||||
error_code="mcp_consent_required",
|
||||
scopes_required=None,
|
||||
last_ws_id=None,
|
||||
last_tool_call_id=None,
|
||||
now_iso="2026-05-11T12:00:00",
|
||||
)
|
||||
token_store = _make_token_store(storage)
|
||||
http_client_mock.get.return_value = _mk_response(200, _good_as_metadata_doc())
|
||||
http_client_mock.post.return_value = _mk_response(
|
||||
200,
|
||||
{"access_token": "opaque-aaa", "expires_in": 3600},
|
||||
)
|
||||
app = _build_app(storage=storage, http_client=http_client_mock, token_store=token_store)
|
||||
client = TestClient(app, raise_server_exceptions=False)
|
||||
|
||||
with _public_addr_patch():
|
||||
resp = client.get(
|
||||
"/v1/api/mcp/oauth/callback?code=c&state=valid-state",
|
||||
follow_redirects=False,
|
||||
)
|
||||
assert resp.status_code == 302
|
||||
# Callback completed → user-1's deferred-consent row was cleared.
|
||||
assert storage.list_mcp_pending_consent_by_user("user-1") == []
|
||||
# Cross-tenant row survives — clear is per-(user, server).
|
||||
other = storage.list_mcp_pending_consent_by_user("other-user")
|
||||
assert len(other) == 1
|
||||
assert other[0]["server_name"] == "srv-oauth"
|
||||
|
||||
def test_callback_storage_failure_does_not_block_redirect(
|
||||
self, storage: SQLiteBackend, http_client_mock: MagicMock
|
||||
) -> None:
|
||||
"""If the post-persist ``delete_mcp_pending_consent`` raises, the
|
||||
callback's redirect still completes (best-effort contract). The
|
||||
stale badge is preferred over a broken consent flow.
|
||||
"""
|
||||
_seed_oauth_user_server(storage)
|
||||
self._seed_pending(storage)
|
||||
token_store = _make_token_store(storage)
|
||||
http_client_mock.get.return_value = _mk_response(200, _good_as_metadata_doc())
|
||||
http_client_mock.post.return_value = _mk_response(
|
||||
200,
|
||||
{"access_token": "opaque-aaa", "expires_in": 3600},
|
||||
)
|
||||
app = _build_app(storage=storage, http_client=http_client_mock, token_store=token_store)
|
||||
client = TestClient(app, raise_server_exceptions=False)
|
||||
|
||||
original_delete = storage.delete_mcp_pending_consent
|
||||
|
||||
def _raise(*_a: Any, **_kw: Any) -> bool:
|
||||
raise RuntimeError("storage offline")
|
||||
|
||||
storage.delete_mcp_pending_consent = _raise # type: ignore[method-assign]
|
||||
try:
|
||||
with _public_addr_patch():
|
||||
resp = client.get(
|
||||
"/v1/api/mcp/oauth/callback?code=c&state=valid-state",
|
||||
follow_redirects=False,
|
||||
)
|
||||
finally:
|
||||
storage.delete_mcp_pending_consent = original_delete # type: ignore[method-assign]
|
||||
|
||||
assert resp.status_code == 302
|
||||
# Token persistence still succeeded — the user-visible contract.
|
||||
plain = token_store.get_user_token("user-1", "srv-oauth")
|
||||
assert plain is not None
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 503 paths when mcp_token_store is None
|
||||
|
||||
@@ -0,0 +1,304 @@
|
||||
"""Boundary tests for the Phase 9 pending-consent write path.
|
||||
|
||||
Drives ``MCPClientManager._dispatch_pool_sync`` (and the helper it
|
||||
calls, ``_record_pending_consent_best_effort``) and asserts that
|
||||
deferred-consent records reach storage only on non-interactive callers.
|
||||
|
||||
Per ``feedback_tests_through_boundaries.md``, at least one test must
|
||||
drive the real sync dispatcher → real ``_is_structured_error`` →
|
||||
real ``_record_pending_consent_best_effort`` plumb-through; the
|
||||
``_helpers`` unit tests below cover the classifier in isolation, but
|
||||
the end-to-end test is the structural gate that catches
|
||||
plumb-through regressions.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import contextlib
|
||||
import json
|
||||
import threading
|
||||
from typing import Any
|
||||
from unittest.mock import patch
|
||||
|
||||
import pytest
|
||||
|
||||
from tests.conftest import make_mcp_token_cipher
|
||||
from turnstone.core.mcp_client import (
|
||||
_PENDING_CONSENT_PERSIST_CODES,
|
||||
MCPClientManager,
|
||||
_parse_pending_consent_envelope,
|
||||
)
|
||||
from turnstone.core.mcp_crypto import MCPTokenStore
|
||||
from turnstone.core.mcp_oauth import TokenLookupResult
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Helper-level unit tests (cheap, no event loop)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestParseEnvelope:
|
||||
def test_consent_required_no_scopes(self) -> None:
|
||||
env = json.dumps({"error": {"code": "mcp_consent_required", "server": "x", "detail": "d"}})
|
||||
assert _parse_pending_consent_envelope(env) == ("mcp_consent_required", None)
|
||||
|
||||
def test_insufficient_scope_with_scopes(self) -> None:
|
||||
env = json.dumps(
|
||||
{
|
||||
"error": {
|
||||
"code": "mcp_insufficient_scope",
|
||||
"server": "x",
|
||||
"detail": "d",
|
||||
"scopes_required": ["read", "write"],
|
||||
}
|
||||
}
|
||||
)
|
||||
assert _parse_pending_consent_envelope(env) == (
|
||||
"mcp_insufficient_scope",
|
||||
["read", "write"],
|
||||
)
|
||||
|
||||
def test_operator_codes_filtered(self) -> None:
|
||||
# Key-unknown / url-insecure / *_forbidden are operator-actionable,
|
||||
# NOT user-consent-shaped. They must not produce pending-consent
|
||||
# rows, regardless of whether the caller is interactive.
|
||||
for code in (
|
||||
"mcp_token_undecryptable_key_unknown",
|
||||
"mcp_oauth_url_insecure",
|
||||
"mcp_tool_call_forbidden",
|
||||
"mcp_resource_read_forbidden",
|
||||
"mcp_prompt_get_forbidden",
|
||||
):
|
||||
env = json.dumps({"error": {"code": code, "server": "x", "detail": "d"}})
|
||||
assert _parse_pending_consent_envelope(env) is None, code
|
||||
|
||||
def test_malformed_json_returns_none(self) -> None:
|
||||
assert _parse_pending_consent_envelope("not json") is None
|
||||
assert _parse_pending_consent_envelope("") is None
|
||||
|
||||
def test_persist_codes_set_is_expected(self) -> None:
|
||||
# Pin the contract — adding a new persistable code here is a
|
||||
# deliberate design decision and should require a test update.
|
||||
assert {
|
||||
"mcp_consent_required",
|
||||
"mcp_insufficient_scope",
|
||||
} == _PENDING_CONSENT_PERSIST_CODES
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# End-to-end plumb-through (drives _dispatch_pool_sync)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _seed_oauth_server(backend: Any, *, name: str = "pool-srv") -> None:
|
||||
backend.create_mcp_server(
|
||||
server_id="srv-" + name,
|
||||
name=name,
|
||||
transport="streamable-http",
|
||||
command="",
|
||||
args="[]",
|
||||
url="https://example.com/mcp",
|
||||
headers="{}",
|
||||
env="{}",
|
||||
auto_approve=False,
|
||||
enabled=True,
|
||||
created_by="admin",
|
||||
)
|
||||
backend.update_mcp_server("srv-" + name, auth_type="oauth_user")
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def running_loop_mgr():
|
||||
cfg: dict[str, Any] = {}
|
||||
mgr = MCPClientManager(cfg)
|
||||
loop = asyncio.new_event_loop()
|
||||
thread = threading.Thread(target=loop.run_forever, daemon=True, name="phase9-test-loop")
|
||||
thread.start()
|
||||
mgr._loop = loop
|
||||
try:
|
||||
yield mgr, loop, thread
|
||||
finally:
|
||||
|
||||
async def _drain(m: MCPClientManager) -> None:
|
||||
task = m._user_pool_eviction_task
|
||||
if task is not None:
|
||||
task.cancel()
|
||||
with contextlib.suppress(BaseException):
|
||||
await task
|
||||
m._user_pool_eviction_task = None
|
||||
|
||||
with contextlib.suppress(Exception):
|
||||
asyncio.run_coroutine_threadsafe(_drain(mgr), loop).result(timeout=2)
|
||||
loop.call_soon_threadsafe(loop.stop)
|
||||
thread.join(timeout=2)
|
||||
|
||||
|
||||
def _wire_mgr(mgr: MCPClientManager, backend: Any) -> None:
|
||||
cipher = make_mcp_token_cipher()
|
||||
from types import SimpleNamespace
|
||||
from unittest.mock import MagicMock
|
||||
|
||||
app_state = SimpleNamespace(
|
||||
auth_storage=backend,
|
||||
mcp_token_store=MCPTokenStore(backend, cipher, node_id="test"),
|
||||
mcp_oauth_http_client=MagicMock(),
|
||||
mcp_oauth_refresh_locks={},
|
||||
mcp_oauth_metadata_cache={},
|
||||
)
|
||||
mgr.set_storage(backend)
|
||||
mgr.set_app_state(app_state)
|
||||
|
||||
|
||||
def test_dispatch_persists_pending_for_non_interactive_caller(
|
||||
running_loop_mgr: Any, backend: Any
|
||||
) -> None:
|
||||
"""Non-interactive caller hits ``mcp_consent_required`` → a
|
||||
``mcp_pending_consent`` row appears for ``(user_id, server_name)``."""
|
||||
mgr, _loop, _ = running_loop_mgr
|
||||
_seed_oauth_server(backend)
|
||||
_wire_mgr(mgr, backend)
|
||||
|
||||
async def _missing_token(**kwargs: Any) -> TokenLookupResult:
|
||||
return TokenLookupResult(kind="missing")
|
||||
|
||||
with (
|
||||
patch(
|
||||
"turnstone.core.mcp_client.get_user_access_token_classified",
|
||||
side_effect=_missing_token,
|
||||
),
|
||||
pytest.raises(RuntimeError) as exc_info,
|
||||
):
|
||||
mgr.call_tool_sync(
|
||||
"mcp__pool-srv__echo",
|
||||
{"payload": "hi"},
|
||||
user_id="user-a",
|
||||
timeout=10,
|
||||
is_interactive_for_consent=False,
|
||||
)
|
||||
|
||||
# Structured error envelope surfaces as RuntimeError to the caller.
|
||||
payload = json.loads(str(exc_info.value)).get("error", {})
|
||||
assert payload.get("code") == "mcp_consent_required"
|
||||
|
||||
# Persistent row written for the dashboard badge.
|
||||
rows = backend.list_mcp_pending_consent_by_user("user-a")
|
||||
assert len(rows) == 1
|
||||
r = rows[0]
|
||||
assert r["user_id"] == "user-a"
|
||||
assert r["server_name"] == "pool-srv"
|
||||
assert r["error_code"] == "mcp_consent_required"
|
||||
assert r["occurrence_count"] == 1
|
||||
|
||||
|
||||
def test_dispatch_does_not_persist_for_interactive_caller(
|
||||
running_loop_mgr: Any, backend: Any
|
||||
) -> None:
|
||||
"""Interactive caller hits the same error path → NO row written.
|
||||
|
||||
Interactive (WEB / CLI) sessions surface the consent prompt in-flight
|
||||
via the Phase 8 SSE renderer; persisting would just produce
|
||||
immediately-stale dashboard badges.
|
||||
"""
|
||||
mgr, _loop, _ = running_loop_mgr
|
||||
_seed_oauth_server(backend)
|
||||
_wire_mgr(mgr, backend)
|
||||
|
||||
async def _missing_token(**kwargs: Any) -> TokenLookupResult:
|
||||
return TokenLookupResult(kind="missing")
|
||||
|
||||
with (
|
||||
patch(
|
||||
"turnstone.core.mcp_client.get_user_access_token_classified",
|
||||
side_effect=_missing_token,
|
||||
),
|
||||
pytest.raises(RuntimeError),
|
||||
):
|
||||
mgr.call_tool_sync(
|
||||
"mcp__pool-srv__echo",
|
||||
{"payload": "hi"},
|
||||
user_id="user-a",
|
||||
timeout=10,
|
||||
is_interactive_for_consent=True,
|
||||
)
|
||||
|
||||
assert backend.list_mcp_pending_consent_by_user("user-a") == []
|
||||
|
||||
|
||||
def test_dispatch_returns_envelope_unchanged_on_storage_failure(
|
||||
running_loop_mgr: Any, backend: Any
|
||||
) -> None:
|
||||
"""When ``upsert_mcp_pending_consent`` raises, the agent-observable
|
||||
contract is unchanged: the structured-error ``RuntimeError`` still
|
||||
surfaces with the original ``mcp_consent_required`` code. The doc-
|
||||
string promises best-effort persistence; this test pins that
|
||||
promise so a regression that propagates the storage exception would
|
||||
fail visibly.
|
||||
"""
|
||||
mgr, _loop, _ = running_loop_mgr
|
||||
_seed_oauth_server(backend)
|
||||
_wire_mgr(mgr, backend)
|
||||
|
||||
async def _missing_token(**kwargs: Any) -> TokenLookupResult:
|
||||
return TokenLookupResult(kind="missing")
|
||||
|
||||
original_upsert = backend.upsert_mcp_pending_consent
|
||||
|
||||
def _raise(*_a: Any, **_kw: Any) -> None:
|
||||
raise RuntimeError("storage offline")
|
||||
|
||||
backend.upsert_mcp_pending_consent = _raise # type: ignore[method-assign]
|
||||
try:
|
||||
with (
|
||||
patch(
|
||||
"turnstone.core.mcp_client.get_user_access_token_classified",
|
||||
side_effect=_missing_token,
|
||||
),
|
||||
pytest.raises(RuntimeError) as exc_info,
|
||||
):
|
||||
mgr.call_tool_sync(
|
||||
"mcp__pool-srv__echo",
|
||||
{"payload": "hi"},
|
||||
user_id="user-a",
|
||||
timeout=10,
|
||||
is_interactive_for_consent=False,
|
||||
)
|
||||
finally:
|
||||
backend.upsert_mcp_pending_consent = original_upsert # type: ignore[method-assign]
|
||||
|
||||
payload = json.loads(str(exc_info.value)).get("error", {})
|
||||
assert payload.get("code") == "mcp_consent_required"
|
||||
|
||||
|
||||
def test_dispatch_does_not_persist_for_operator_actionable_code(
|
||||
running_loop_mgr: Any, backend: Any
|
||||
) -> None:
|
||||
"""Decrypt-failure → operator-actionable; even non-interactive callers
|
||||
must NOT produce a user-facing pending-consent record (the user can't
|
||||
resolve this by re-consenting).
|
||||
"""
|
||||
mgr, _loop, _ = running_loop_mgr
|
||||
_seed_oauth_server(backend)
|
||||
_wire_mgr(mgr, backend)
|
||||
|
||||
async def _decrypt_failure(**kwargs: Any) -> TokenLookupResult:
|
||||
return TokenLookupResult(kind="decrypt_failure")
|
||||
|
||||
with (
|
||||
patch(
|
||||
"turnstone.core.mcp_client.get_user_access_token_classified",
|
||||
side_effect=_decrypt_failure,
|
||||
),
|
||||
pytest.raises(RuntimeError) as exc_info,
|
||||
):
|
||||
mgr.call_tool_sync(
|
||||
"mcp__pool-srv__echo",
|
||||
{"payload": "hi"},
|
||||
user_id="user-a",
|
||||
timeout=10,
|
||||
is_interactive_for_consent=False,
|
||||
)
|
||||
|
||||
payload = json.loads(str(exc_info.value)).get("error", {})
|
||||
assert payload.get("code") == "mcp_token_undecryptable_key_unknown"
|
||||
# The operator-actionable code does NOT produce a pending-consent row.
|
||||
assert backend.list_mcp_pending_consent_by_user("user-a") == []
|
||||
@@ -0,0 +1,259 @@
|
||||
"""HTTP tests for the Phase 9 pending-consent endpoints.
|
||||
|
||||
Covers:
|
||||
- ``GET /v1/api/mcp/oauth/pending`` (install gate + read path)
|
||||
- ``DELETE /v1/api/mcp/oauth/pending/{server_name}`` (single clear)
|
||||
- ``DELETE /v1/api/mcp/oauth/pending`` (bulk clear)
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
import pytest
|
||||
from starlette.applications import Starlette
|
||||
from starlette.middleware import Middleware
|
||||
from starlette.middleware.base import BaseHTTPMiddleware
|
||||
from starlette.routing import Mount, Route
|
||||
from starlette.testclient import TestClient
|
||||
|
||||
from turnstone.core.auth import AuthResult
|
||||
from turnstone.core.mcp_oauth import (
|
||||
handle_mcp_oauth_clear_all_pending,
|
||||
handle_mcp_oauth_clear_pending,
|
||||
handle_mcp_oauth_list_pending,
|
||||
)
|
||||
from turnstone.core.storage._sqlite import SQLiteBackend
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from starlette.requests import Request
|
||||
from starlette.responses import Response
|
||||
|
||||
|
||||
class _InjectAuthMiddleware(BaseHTTPMiddleware):
|
||||
"""Stamp a fixed authenticated user on every request."""
|
||||
|
||||
def __init__(self, app: Any, user_id: str = "user-1") -> None:
|
||||
super().__init__(app)
|
||||
self._user_id = user_id
|
||||
|
||||
async def dispatch(self, request: Request, call_next: Any) -> Response:
|
||||
request.state.auth_result = AuthResult(
|
||||
user_id=self._user_id,
|
||||
scopes=frozenset({"write"}),
|
||||
token_source="config",
|
||||
permissions=frozenset({"read", "write"}),
|
||||
)
|
||||
return await call_next(request)
|
||||
|
||||
|
||||
def _build_app(storage: SQLiteBackend, *, user_id: str = "user-1") -> Starlette:
|
||||
class _Mw(_InjectAuthMiddleware):
|
||||
def __init__(self, app: Any) -> None:
|
||||
super().__init__(app, user_id=user_id)
|
||||
|
||||
app = Starlette(
|
||||
routes=[
|
||||
Mount(
|
||||
"/v1",
|
||||
routes=[
|
||||
Route("/api/mcp/oauth/pending", handle_mcp_oauth_list_pending),
|
||||
Route(
|
||||
"/api/mcp/oauth/pending",
|
||||
handle_mcp_oauth_clear_all_pending,
|
||||
methods=["DELETE"],
|
||||
),
|
||||
Route(
|
||||
"/api/mcp/oauth/pending/{server_name}",
|
||||
handle_mcp_oauth_clear_pending,
|
||||
methods=["DELETE"],
|
||||
),
|
||||
],
|
||||
),
|
||||
],
|
||||
middleware=[Middleware(_Mw)],
|
||||
)
|
||||
app.state.auth_storage = storage
|
||||
return app
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def storage(tmp_path: Any) -> SQLiteBackend:
|
||||
backend = SQLiteBackend(str(tmp_path / "test.db"))
|
||||
backend.create_user("user-1", "user1", "User One", "hash")
|
||||
backend.create_user("user-2", "user2", "User Two", "hash")
|
||||
return backend
|
||||
|
||||
|
||||
def _seed_oauth_server(backend: SQLiteBackend, *, name: str = "srv-x") -> None:
|
||||
backend.create_mcp_server(
|
||||
server_id="srv-id-" + name,
|
||||
name=name,
|
||||
transport="streamable-http",
|
||||
url="https://example.com/mcp",
|
||||
auth_type="oauth_user",
|
||||
)
|
||||
|
||||
|
||||
def _seed_pending(
|
||||
backend: SQLiteBackend,
|
||||
*,
|
||||
user_id: str = "user-1",
|
||||
server_name: str = "srv-x",
|
||||
error_code: str = "mcp_consent_required",
|
||||
now_iso: str = "2026-05-11T12:00:00",
|
||||
) -> None:
|
||||
backend.upsert_mcp_pending_consent(
|
||||
user_id=user_id,
|
||||
server_name=server_name,
|
||||
error_code=error_code,
|
||||
scopes_required=None,
|
||||
last_ws_id=None,
|
||||
last_tool_call_id=None,
|
||||
now_iso=now_iso,
|
||||
)
|
||||
|
||||
|
||||
class TestListPending:
|
||||
def test_install_gate_short_circuits_on_no_oauth_servers(self, storage: SQLiteBackend) -> None:
|
||||
# Seed a pending row but NO oauth_user MCP server — the gate
|
||||
# must short-circuit to {pending: 0} regardless.
|
||||
_seed_pending(storage)
|
||||
client = TestClient(_build_app(storage))
|
||||
resp = client.get("/v1/api/mcp/oauth/pending")
|
||||
assert resp.status_code == 200
|
||||
assert resp.json() == {"pending": 0, "servers": []}
|
||||
|
||||
def test_lists_pending_records_for_authenticated_user(self, storage: SQLiteBackend) -> None:
|
||||
_seed_oauth_server(storage)
|
||||
_seed_pending(storage)
|
||||
client = TestClient(_build_app(storage))
|
||||
resp = client.get("/v1/api/mcp/oauth/pending")
|
||||
assert resp.status_code == 200
|
||||
body = resp.json()
|
||||
assert body["pending"] == 1
|
||||
assert len(body["servers"]) == 1
|
||||
assert body["servers"][0]["server_name"] == "srv-x"
|
||||
assert body["servers"][0]["error_code"] == "mcp_consent_required"
|
||||
|
||||
def test_does_not_leak_cross_user_records(self, storage: SQLiteBackend) -> None:
|
||||
_seed_oauth_server(storage)
|
||||
_seed_pending(storage, user_id="user-2")
|
||||
client = TestClient(_build_app(storage, user_id="user-1"))
|
||||
resp = client.get("/v1/api/mcp/oauth/pending")
|
||||
assert resp.status_code == 200
|
||||
assert resp.json() == {"pending": 0, "servers": []}
|
||||
|
||||
|
||||
class TestClearPending:
|
||||
def test_delete_single(self, storage: SQLiteBackend) -> None:
|
||||
_seed_oauth_server(storage)
|
||||
_seed_pending(storage)
|
||||
client = TestClient(_build_app(storage))
|
||||
resp = client.delete("/v1/api/mcp/oauth/pending/srv-x")
|
||||
assert resp.status_code == 204
|
||||
assert storage.list_mcp_pending_consent_by_user("user-1") == []
|
||||
|
||||
def test_delete_missing_still_returns_204(self, storage: SQLiteBackend) -> None:
|
||||
# Idempotent — must not leak cross-user existence info via 404.
|
||||
_seed_oauth_server(storage)
|
||||
client = TestClient(_build_app(storage))
|
||||
resp = client.delete("/v1/api/mcp/oauth/pending/never-existed")
|
||||
assert resp.status_code == 204
|
||||
|
||||
def test_delete_does_not_touch_cross_user_rows(self, storage: SQLiteBackend) -> None:
|
||||
_seed_oauth_server(storage)
|
||||
_seed_pending(storage, user_id="user-1")
|
||||
_seed_pending(storage, user_id="user-2")
|
||||
client = TestClient(_build_app(storage, user_id="user-1"))
|
||||
resp = client.delete("/v1/api/mcp/oauth/pending/srv-x")
|
||||
assert resp.status_code == 204
|
||||
# User-2's row survives.
|
||||
assert len(storage.list_mcp_pending_consent_by_user("user-2")) == 1
|
||||
|
||||
|
||||
class TestAuditTrail:
|
||||
def test_single_dismiss_audits(self, storage: SQLiteBackend) -> None:
|
||||
_seed_oauth_server(storage)
|
||||
_seed_pending(storage)
|
||||
client = TestClient(_build_app(storage))
|
||||
resp = client.delete("/v1/api/mcp/oauth/pending/srv-x")
|
||||
assert resp.status_code == 204
|
||||
|
||||
events = storage.list_audit_events(limit=10)
|
||||
rows = [
|
||||
e for e in events if e.get("action") == "mcp_server.oauth.pending_consent_dismissed"
|
||||
]
|
||||
assert len(rows) == 1
|
||||
detail = rows[0].get("detail")
|
||||
if isinstance(detail, str):
|
||||
import json as _json
|
||||
|
||||
detail = _json.loads(detail)
|
||||
assert detail.get("mode") == "single"
|
||||
assert detail.get("cleared") == 1
|
||||
|
||||
def test_single_dismiss_audits_even_when_no_row_existed(self, storage: SQLiteBackend) -> None:
|
||||
# Cross-tenant non-observability requires a 204 in the never-existed
|
||||
# case — the audit row distinguishes a real dismiss from a stuffed
|
||||
# attempt by recording ``cleared=0``.
|
||||
_seed_oauth_server(storage)
|
||||
client = TestClient(_build_app(storage))
|
||||
resp = client.delete("/v1/api/mcp/oauth/pending/never-existed")
|
||||
assert resp.status_code == 204
|
||||
|
||||
events = storage.list_audit_events(limit=10)
|
||||
rows = [
|
||||
e for e in events if e.get("action") == "mcp_server.oauth.pending_consent_dismissed"
|
||||
]
|
||||
assert len(rows) == 1
|
||||
detail = rows[0].get("detail")
|
||||
if isinstance(detail, str):
|
||||
import json as _json
|
||||
|
||||
detail = _json.loads(detail)
|
||||
assert detail.get("mode") == "single"
|
||||
assert detail.get("cleared") == 0
|
||||
|
||||
def test_bulk_dismiss_audits(self, storage: SQLiteBackend) -> None:
|
||||
_seed_oauth_server(storage)
|
||||
_seed_oauth_server(storage, name="srv-y")
|
||||
_seed_pending(storage, server_name="srv-x")
|
||||
_seed_pending(storage, server_name="srv-y")
|
||||
client = TestClient(_build_app(storage))
|
||||
resp = client.delete("/v1/api/mcp/oauth/pending")
|
||||
assert resp.status_code == 200
|
||||
assert resp.json() == {"cleared": 2}
|
||||
|
||||
events = storage.list_audit_events(limit=10)
|
||||
rows = [
|
||||
e for e in events if e.get("action") == "mcp_server.oauth.pending_consent_dismissed"
|
||||
]
|
||||
assert len(rows) == 1
|
||||
detail = rows[0].get("detail")
|
||||
if isinstance(detail, str):
|
||||
import json as _json
|
||||
|
||||
detail = _json.loads(detail)
|
||||
assert detail.get("mode") == "bulk"
|
||||
assert detail.get("cleared") == 2
|
||||
|
||||
|
||||
class TestClearAllPending:
|
||||
def test_bulk_clear(self, storage: SQLiteBackend) -> None:
|
||||
_seed_oauth_server(storage)
|
||||
_seed_oauth_server(storage, name="srv-y")
|
||||
_seed_pending(storage, server_name="srv-x")
|
||||
_seed_pending(storage, server_name="srv-y")
|
||||
client = TestClient(_build_app(storage))
|
||||
resp = client.delete("/v1/api/mcp/oauth/pending")
|
||||
assert resp.status_code == 200
|
||||
assert resp.json() == {"cleared": 2}
|
||||
assert storage.list_mcp_pending_consent_by_user("user-1") == []
|
||||
|
||||
def test_bulk_clear_zero_when_empty(self, storage: SQLiteBackend) -> None:
|
||||
_seed_oauth_server(storage)
|
||||
client = TestClient(_build_app(storage))
|
||||
resp = client.delete("/v1/api/mcp/oauth/pending")
|
||||
assert resp.status_code == 200
|
||||
assert resp.json() == {"cleared": 0}
|
||||
@@ -0,0 +1,263 @@
|
||||
"""Storage CRUD tests for the Phase 9 ``mcp_pending_consent`` table.
|
||||
|
||||
Validates protocol additions backing the dashboard pending-consent badge:
|
||||
|
||||
- ``upsert_mcp_pending_consent`` — insert + on-conflict refresh
|
||||
- ``list_mcp_pending_consent_by_user`` — read path
|
||||
- ``delete_mcp_pending_consent`` — single-row clear
|
||||
- ``delete_all_mcp_pending_consent_by_user`` — bulk clear
|
||||
- ``count_mcp_consented_users_by_server`` — admin status pill
|
||||
- ``any_oauth_user_mcp_servers`` — install-level gate
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
|
||||
def _iso(ts: str = "2026-05-11T12:00:00") -> str:
|
||||
return ts
|
||||
|
||||
|
||||
class TestUpsertAndList:
|
||||
def test_insert_round_trip(self, backend) -> None:
|
||||
backend.upsert_mcp_pending_consent(
|
||||
user_id="user-a",
|
||||
server_name="srv-x",
|
||||
error_code="mcp_consent_required",
|
||||
scopes_required="read write",
|
||||
last_ws_id="ws-1",
|
||||
last_tool_call_id="tool-1",
|
||||
now_iso=_iso(),
|
||||
)
|
||||
rows = backend.list_mcp_pending_consent_by_user("user-a")
|
||||
assert len(rows) == 1
|
||||
r = rows[0]
|
||||
assert r["user_id"] == "user-a"
|
||||
assert r["server_name"] == "srv-x"
|
||||
assert r["error_code"] == "mcp_consent_required"
|
||||
assert r["scopes_required"] == "read write"
|
||||
assert r["last_ws_id"] == "ws-1"
|
||||
assert r["last_tool_call_id"] == "tool-1"
|
||||
assert r["occurrence_count"] == 1
|
||||
assert r["first_seen_at"] == r["last_seen_at"]
|
||||
|
||||
def test_upsert_bumps_count_and_refreshes_recency(self, backend) -> None:
|
||||
backend.upsert_mcp_pending_consent(
|
||||
user_id="user-a",
|
||||
server_name="srv-x",
|
||||
error_code="mcp_consent_required",
|
||||
scopes_required=None,
|
||||
last_ws_id=None,
|
||||
last_tool_call_id=None,
|
||||
now_iso="2026-05-11T12:00:00",
|
||||
)
|
||||
backend.upsert_mcp_pending_consent(
|
||||
user_id="user-a",
|
||||
server_name="srv-x",
|
||||
error_code="mcp_insufficient_scope",
|
||||
scopes_required="read",
|
||||
last_ws_id="ws-2",
|
||||
last_tool_call_id="tool-2",
|
||||
now_iso="2026-05-11T13:00:00",
|
||||
)
|
||||
rows = backend.list_mcp_pending_consent_by_user("user-a")
|
||||
assert len(rows) == 1
|
||||
r = rows[0]
|
||||
# Recency fields refreshed to the second call's values; count bumped.
|
||||
assert r["occurrence_count"] == 2
|
||||
assert r["error_code"] == "mcp_insufficient_scope"
|
||||
assert r["scopes_required"] == "read"
|
||||
assert r["last_ws_id"] == "ws-2"
|
||||
assert r["last_tool_call_id"] == "tool-2"
|
||||
assert r["last_seen_at"] == "2026-05-11T13:00:00"
|
||||
# first_seen_at preserved — that's the load-bearing audit value.
|
||||
assert r["first_seen_at"] == "2026-05-11T12:00:00"
|
||||
|
||||
def test_list_orders_by_last_seen_desc(self, backend) -> None:
|
||||
backend.upsert_mcp_pending_consent(
|
||||
user_id="user-a",
|
||||
server_name="srv-old",
|
||||
error_code="mcp_consent_required",
|
||||
scopes_required=None,
|
||||
last_ws_id=None,
|
||||
last_tool_call_id=None,
|
||||
now_iso="2026-05-11T10:00:00",
|
||||
)
|
||||
backend.upsert_mcp_pending_consent(
|
||||
user_id="user-a",
|
||||
server_name="srv-new",
|
||||
error_code="mcp_consent_required",
|
||||
scopes_required=None,
|
||||
last_ws_id=None,
|
||||
last_tool_call_id=None,
|
||||
now_iso="2026-05-11T11:00:00",
|
||||
)
|
||||
rows = backend.list_mcp_pending_consent_by_user("user-a")
|
||||
assert [r["server_name"] for r in rows] == ["srv-new", "srv-old"]
|
||||
|
||||
def test_per_user_isolation(self, backend) -> None:
|
||||
backend.upsert_mcp_pending_consent(
|
||||
user_id="user-a",
|
||||
server_name="srv",
|
||||
error_code="mcp_consent_required",
|
||||
scopes_required=None,
|
||||
last_ws_id=None,
|
||||
last_tool_call_id=None,
|
||||
now_iso=_iso(),
|
||||
)
|
||||
assert backend.list_mcp_pending_consent_by_user("user-b") == []
|
||||
|
||||
|
||||
class TestDelete:
|
||||
def test_delete_single(self, backend) -> None:
|
||||
backend.upsert_mcp_pending_consent(
|
||||
user_id="user-a",
|
||||
server_name="srv-x",
|
||||
error_code="mcp_consent_required",
|
||||
scopes_required=None,
|
||||
last_ws_id=None,
|
||||
last_tool_call_id=None,
|
||||
now_iso=_iso(),
|
||||
)
|
||||
assert backend.delete_mcp_pending_consent("user-a", "srv-x") is True
|
||||
assert backend.list_mcp_pending_consent_by_user("user-a") == []
|
||||
# Second delete returns False (no row).
|
||||
assert backend.delete_mcp_pending_consent("user-a", "srv-x") is False
|
||||
|
||||
def test_delete_missing_returns_false(self, backend) -> None:
|
||||
assert backend.delete_mcp_pending_consent("never", "missing") is False
|
||||
|
||||
def test_delete_all_by_user(self, backend) -> None:
|
||||
for name in ("srv-a", "srv-b", "srv-c"):
|
||||
backend.upsert_mcp_pending_consent(
|
||||
user_id="user-a",
|
||||
server_name=name,
|
||||
error_code="mcp_consent_required",
|
||||
scopes_required=None,
|
||||
last_ws_id=None,
|
||||
last_tool_call_id=None,
|
||||
now_iso=_iso(),
|
||||
)
|
||||
# Cross-user row that must NOT be touched.
|
||||
backend.upsert_mcp_pending_consent(
|
||||
user_id="user-b",
|
||||
server_name="srv-z",
|
||||
error_code="mcp_consent_required",
|
||||
scopes_required=None,
|
||||
last_ws_id=None,
|
||||
last_tool_call_id=None,
|
||||
now_iso=_iso(),
|
||||
)
|
||||
assert backend.delete_all_mcp_pending_consent_by_user("user-a") == 3
|
||||
assert backend.list_mcp_pending_consent_by_user("user-a") == []
|
||||
assert len(backend.list_mcp_pending_consent_by_user("user-b")) == 1
|
||||
|
||||
|
||||
class TestCountConsentedUsersByServer:
|
||||
def _seed_server(self, backend, name: str = "srv-x") -> None:
|
||||
backend.create_mcp_server(
|
||||
server_id="srv-id-" + name,
|
||||
name=name,
|
||||
transport="streamable-http",
|
||||
command="",
|
||||
args="[]",
|
||||
url="https://example.com/mcp",
|
||||
headers="{}",
|
||||
env="{}",
|
||||
auto_approve=False,
|
||||
enabled=True,
|
||||
created_by="admin",
|
||||
)
|
||||
backend.update_mcp_server("srv-id-" + name, auth_type="oauth_user")
|
||||
|
||||
def test_counts_distinct_non_expired_users(self, backend) -> None:
|
||||
self._seed_server(backend)
|
||||
future = "2099-01-01T00:00:00"
|
||||
backend.create_mcp_user_token(
|
||||
"alice",
|
||||
"srv-x",
|
||||
access_token_ct=b"ct",
|
||||
refresh_token_ct=None,
|
||||
expires_at=future,
|
||||
scopes=None,
|
||||
as_issuer="https://as.example.com",
|
||||
audience="https://example.com/mcp",
|
||||
)
|
||||
backend.create_mcp_user_token(
|
||||
"bob",
|
||||
"srv-x",
|
||||
access_token_ct=b"ct",
|
||||
refresh_token_ct=None,
|
||||
expires_at=None, # null treated as non-expired
|
||||
scopes=None,
|
||||
as_issuer="https://as.example.com",
|
||||
audience="https://example.com/mcp",
|
||||
)
|
||||
# Different server — must not count.
|
||||
self._seed_server(backend, name="srv-y")
|
||||
backend.create_mcp_user_token(
|
||||
"carol",
|
||||
"srv-y",
|
||||
access_token_ct=b"ct",
|
||||
refresh_token_ct=None,
|
||||
expires_at=future,
|
||||
scopes=None,
|
||||
as_issuer="https://as.example.com",
|
||||
audience="https://example.com/mcp",
|
||||
)
|
||||
assert backend.count_mcp_consented_users_by_server("srv-x") == 2
|
||||
assert backend.count_mcp_consented_users_by_server("srv-y") == 1
|
||||
|
||||
def test_excludes_expired(self, backend) -> None:
|
||||
self._seed_server(backend)
|
||||
backend.create_mcp_user_token(
|
||||
"alice",
|
||||
"srv-x",
|
||||
access_token_ct=b"ct",
|
||||
refresh_token_ct=None,
|
||||
expires_at="2020-01-01T00:00:00", # well in the past
|
||||
scopes=None,
|
||||
as_issuer="https://as.example.com",
|
||||
audience="https://example.com/mcp",
|
||||
)
|
||||
assert backend.count_mcp_consented_users_by_server("srv-x") == 0
|
||||
|
||||
def test_zero_when_no_rows(self, backend) -> None:
|
||||
assert backend.count_mcp_consented_users_by_server("missing") == 0
|
||||
|
||||
|
||||
class TestInstallGate:
|
||||
def test_any_oauth_user_returns_false_on_empty(self, backend) -> None:
|
||||
assert backend.any_oauth_user_mcp_servers() is False
|
||||
|
||||
def test_any_oauth_user_ignores_static_rows(self, backend) -> None:
|
||||
backend.create_mcp_server(
|
||||
server_id="srv-1",
|
||||
name="static-only",
|
||||
transport="streamable-http",
|
||||
command="",
|
||||
args="[]",
|
||||
url="https://example.com",
|
||||
headers='{"Authorization": "Bearer x"}',
|
||||
env="{}",
|
||||
auto_approve=False,
|
||||
enabled=True,
|
||||
created_by="admin",
|
||||
)
|
||||
assert backend.any_oauth_user_mcp_servers() is False
|
||||
|
||||
def test_any_oauth_user_returns_true_when_one_exists(self, backend) -> None:
|
||||
backend.create_mcp_server(
|
||||
server_id="srv-2",
|
||||
name="oauth-srv",
|
||||
transport="streamable-http",
|
||||
command="",
|
||||
args="[]",
|
||||
url="https://example.com",
|
||||
headers="{}",
|
||||
env="{}",
|
||||
auto_approve=False,
|
||||
enabled=True,
|
||||
created_by="admin",
|
||||
)
|
||||
backend.update_mcp_server("srv-2", auth_type="oauth_user")
|
||||
assert backend.any_oauth_user_mcp_servers() is True
|
||||
@@ -2,6 +2,7 @@
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import threading
|
||||
|
||||
import pytest
|
||||
@@ -401,8 +402,10 @@ class TestValidation:
|
||||
|
||||
|
||||
class TestValidUntil:
|
||||
"""``valid_until`` predicate: drain re-checks freshness; falsy /
|
||||
raising predicates drop the entry without delivery.
|
||||
"""``valid_until`` predicate: drain re-checks freshness. Falsy
|
||||
predicates drop the entry without delivery and log at ``info``
|
||||
(normal lifecycle outcome); raising predicates drop the entry and
|
||||
log at ``warning`` with ``exc_info`` (misbehaving predicate).
|
||||
"""
|
||||
|
||||
def test_valid_until_true_delivers(self):
|
||||
@@ -411,26 +414,52 @@ class TestValidUntil:
|
||||
out = q.drain({"any"})
|
||||
assert out == [("a", "1", None)]
|
||||
|
||||
def test_valid_until_false_drops_silently(self):
|
||||
def test_valid_until_false_drops_with_info_log(self, caplog: pytest.LogCaptureFixture):
|
||||
q = NudgeQueue()
|
||||
q.enqueue("a", "1", "any", valid_until=lambda: False)
|
||||
out = q.drain({"any"})
|
||||
with caplog.at_level(logging.INFO, logger="turnstone.core.nudge_queue"):
|
||||
out = q.drain({"any"})
|
||||
assert out == []
|
||||
# Already removed from queue (drain partition removes BEFORE
|
||||
# predicate check — falsy doesn't return to queue).
|
||||
assert len(q) == 0
|
||||
# The drop emits a structured info record so a wiring
|
||||
# regression (a predicate that always returns False) is still
|
||||
# observable, without spamming ``warning`` for the routine
|
||||
# lifecycle case where ``valid_until`` is doing its job.
|
||||
# structlog renders the event name + extras into ``msg`` as a
|
||||
# single rendered string, so substring-match like the
|
||||
# ``watch_dispatch.queue_full`` assertion in
|
||||
# tests/test_watch_dispatch.py.
|
||||
drops = [r for r in caplog.records if "nudge_queue.predicate_dropped" in r.getMessage()]
|
||||
assert len(drops) == 1
|
||||
assert drops[0].levelno == logging.INFO
|
||||
assert "predicate_false" in drops[0].getMessage()
|
||||
assert "'nudge_type': 'a'" in drops[0].getMessage()
|
||||
assert "'channel': 'any'" in drops[0].getMessage()
|
||||
assert "'text_len': 1" in drops[0].getMessage()
|
||||
|
||||
def test_valid_until_exception_drops_silently(self):
|
||||
def test_valid_until_exception_drops_with_warning(self, caplog: pytest.LogCaptureFixture):
|
||||
q = NudgeQueue()
|
||||
|
||||
def boom() -> bool:
|
||||
raise RuntimeError("predicate crash")
|
||||
|
||||
q.enqueue("a", "1", "any", valid_until=boom)
|
||||
out = q.drain({"any"})
|
||||
with caplog.at_level(logging.WARNING, logger="turnstone.core.nudge_queue"):
|
||||
out = q.drain({"any"})
|
||||
assert out == []
|
||||
# Crash-on-predicate is treated as "no longer valid" — drop, not propagate.
|
||||
assert len(q) == 0
|
||||
# Stays at ``warning`` (with ``exc_info``) because a raising
|
||||
# predicate is a bug, not a normal lifecycle outcome.
|
||||
drops = [r for r in caplog.records if "nudge_queue.predicate_dropped" in r.getMessage()]
|
||||
assert len(drops) == 1
|
||||
assert drops[0].levelno == logging.WARNING
|
||||
rendered = drops[0].getMessage()
|
||||
assert "predicate_raised" in rendered
|
||||
assert "RuntimeError" in rendered
|
||||
assert "predicate crash" in rendered
|
||||
|
||||
def test_valid_until_evaluated_outside_lock(self):
|
||||
"""The predicate may do non-trivial work (e.g. storage I/O)
|
||||
|
||||
@@ -167,8 +167,8 @@ def test_on_intent_verdict_persists_verdict_row() -> None:
|
||||
}
|
||||
with _patch_get_storage(storage):
|
||||
ui.on_intent_verdict(verdict)
|
||||
storage.create_intent_verdict.assert_called_once()
|
||||
kwargs = storage.create_intent_verdict.call_args.kwargs
|
||||
storage.upsert_intent_verdict.assert_called_once()
|
||||
kwargs = storage.upsert_intent_verdict.call_args.kwargs
|
||||
assert kwargs["verdict_id"] == "v1"
|
||||
assert kwargs["ws_id"] == "ws-1"
|
||||
assert kwargs["call_id"] == "c1"
|
||||
@@ -352,6 +352,192 @@ def test_resolve_approval_stamps_all_pending_verdicts() -> None:
|
||||
assert ui._last_verdict_decision == "denied"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# user_decision value space — pending / approved / denied / timeout
|
||||
# / auto-approve reasons (policy / blanket / skill / always / auto_approve_tools).
|
||||
# Guards the "user_decision is never empty for new rows" invariant.
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_resolve_approval_timeout_kwarg_writes_timeout_value() -> None:
|
||||
"""``resolve_approval(False, ..., timeout=True)`` writes
|
||||
``user_decision="timeout"`` so the audit trail can distinguish a
|
||||
passive timeout expiry from an active user denial — the feedback
|
||||
string used to carry this distinction but the column alone could not."""
|
||||
storage = MagicMock()
|
||||
ui = _make_ui()
|
||||
with _patch_get_storage(storage):
|
||||
ui.on_intent_verdict({"verdict_id": "v1", "call_id": "c1"})
|
||||
with _patch_get_storage(storage):
|
||||
ui.resolve_approval(False, "expired", timeout=True)
|
||||
storage.update_intent_verdict.assert_any_call("v1", user_decision="timeout")
|
||||
assert ui._last_verdict_decision == "timeout"
|
||||
|
||||
|
||||
def test_resolve_approval_timeout_with_approved_raises() -> None:
|
||||
"""``timeout=True`` is mutually exclusive with ``approved=True`` —
|
||||
the combination would land a row whose audit column says
|
||||
``"timeout"`` while the SSE event reports ``approved=True``. Fail
|
||||
loud so the inconsistency can't ship silently."""
|
||||
import pytest
|
||||
|
||||
ui = _make_ui()
|
||||
with pytest.raises(ValueError, match="timeout"):
|
||||
ui.resolve_approval(True, timeout=True)
|
||||
|
||||
|
||||
def test_record_auto_approves_populates_reason_lookup() -> None:
|
||||
"""``_record_auto_approves`` must seed
|
||||
``_auto_approve_reasons[call_id]`` with the per-item reason so a
|
||||
late-arriving LLM judge verdict can recover the auto-approve
|
||||
reason via ``on_intent_verdict``."""
|
||||
storage = MagicMock()
|
||||
ui = _make_ui()
|
||||
items = [
|
||||
{
|
||||
"call_id": "c-policy",
|
||||
"func_name": "bash",
|
||||
"auto_approved": True,
|
||||
"auto_approve_reason": "policy",
|
||||
},
|
||||
{
|
||||
"call_id": "c-blanket",
|
||||
"func_name": "list_workstreams",
|
||||
"auto_approved": True,
|
||||
"auto_approve_reason": "blanket",
|
||||
},
|
||||
]
|
||||
with _patch_get_storage(storage):
|
||||
ui._record_auto_approves(items)
|
||||
assert "c-policy" in ui._auto_approve_reasons
|
||||
assert "c-blanket" in ui._auto_approve_reasons
|
||||
assert ui._auto_approve_reasons["c-policy"][0] == "policy"
|
||||
assert ui._auto_approve_reasons["c-blanket"][0] == "blanket"
|
||||
|
||||
|
||||
def test_on_intent_verdict_consumes_auto_approve_reason() -> None:
|
||||
"""A late LLM verdict for a previously auto-approved call_id picks
|
||||
up the reason from ``_auto_approve_reasons``, stamps it on the
|
||||
verdict before persist, and pops the entry so re-use isn't
|
||||
possible. Closes the misdiagnosis bug where auto-approved tools
|
||||
landed verdict rows with ``user_decision=""``."""
|
||||
storage = MagicMock()
|
||||
ui = _make_ui()
|
||||
ui._auto_approve_reasons["c-x"] = ("auto_approve_tools", 0.0)
|
||||
with _patch_get_storage(storage):
|
||||
ui.on_intent_verdict({"verdict_id": "v-x", "call_id": "c-x"})
|
||||
storage.upsert_intent_verdict.assert_called_once()
|
||||
kwargs = storage.upsert_intent_verdict.call_args.kwargs
|
||||
assert kwargs["user_decision"] == "auto_approve_tools"
|
||||
# Consumed on read so the same call_id can't double-stamp later.
|
||||
assert "c-x" not in ui._auto_approve_reasons
|
||||
# Auto-stamped verdicts must NOT join _pending_verdicts — the
|
||||
# row's final decision is already set; appending would let a
|
||||
# later resolve_approval overwrite the auto-reason with the
|
||||
# manual decision (real audit-trail clobber bug).
|
||||
assert ui._pending_verdicts == []
|
||||
|
||||
|
||||
def test_on_intent_verdict_auto_reason_survives_resolve_cycle() -> None:
|
||||
"""Mixed-batch case: one tool was auto-approved (policy), another
|
||||
needs manual approval. The LLM judge fires for the auto-approved
|
||||
sibling DURING the manual-approval wait. The verdict must land
|
||||
with ``user_decision="policy"`` and stay that way even after
|
||||
``resolve_approval`` fires for the pending sibling — the prior
|
||||
bug was that the auto-stamped row got overwritten with
|
||||
``"approved"``/``"denied"`` by the resolve path."""
|
||||
storage = MagicMock()
|
||||
ui = _make_ui()
|
||||
ui._auto_approve_reasons["c-auto"] = ("policy", 0.0)
|
||||
with _patch_get_storage(storage):
|
||||
# LLM verdict fires for the auto-approved sibling.
|
||||
ui.on_intent_verdict({"verdict_id": "v-auto", "call_id": "c-auto"})
|
||||
# Now the pending sibling gets a verdict + manual resolve.
|
||||
ui.on_intent_verdict({"verdict_id": "v-pending", "call_id": "c-pending"})
|
||||
ui.resolve_approval(True, "looks good")
|
||||
# Only the pending verdict should be UPDATEd to "approved" — the
|
||||
# auto-stamped one stays "policy" via its INSERT.
|
||||
update_calls = {
|
||||
c.args[0]: c.kwargs.get("user_decision")
|
||||
for c in storage.update_intent_verdict.call_args_list
|
||||
}
|
||||
assert update_calls == {"v-pending": "approved"}
|
||||
# The auto verdict's INSERT carried the policy reason.
|
||||
insert_calls = {
|
||||
c.kwargs["verdict_id"]: c.kwargs["user_decision"]
|
||||
for c in storage.upsert_intent_verdict.call_args_list
|
||||
}
|
||||
assert insert_calls["v-auto"] == "policy"
|
||||
assert insert_calls["v-pending"] == "pending"
|
||||
|
||||
|
||||
def test_persist_auto_approved_heuristic_verdicts_stamps_reason() -> None:
|
||||
"""The auto-approve early-return branches in ``approve_tools`` used
|
||||
to drop heuristic verdicts on the floor — auditors couldn't tell
|
||||
whether the judge ran or the call was simply silently auto-approved.
|
||||
``_persist_auto_approved_heuristic_verdicts`` closes that gap and
|
||||
stamps each verdict with the item's reason."""
|
||||
storage = MagicMock()
|
||||
ui = _make_ui()
|
||||
items = [
|
||||
{
|
||||
"call_id": "c-1",
|
||||
"auto_approved": True,
|
||||
"auto_approve_reason": "blanket",
|
||||
"_heuristic_verdict": {
|
||||
"verdict_id": "v-1",
|
||||
"call_id": "c-1",
|
||||
"risk_level": "low",
|
||||
"recommendation": "review",
|
||||
},
|
||||
},
|
||||
# No _heuristic_verdict — skipped (judge didn't run for this item).
|
||||
{"call_id": "c-2", "auto_approved": True, "auto_approve_reason": "blanket"},
|
||||
# Not auto_approved — skipped (this helper only handles auto-approved).
|
||||
{
|
||||
"call_id": "c-3",
|
||||
"_heuristic_verdict": {"verdict_id": "v-3", "call_id": "c-3"},
|
||||
},
|
||||
]
|
||||
with _patch_get_storage(storage):
|
||||
ui._persist_auto_approved_heuristic_verdicts(items)
|
||||
storage.create_intent_verdicts_bulk.assert_called_once()
|
||||
rows = storage.create_intent_verdicts_bulk.call_args.args[0]
|
||||
assert len(rows) == 1
|
||||
assert rows[0]["verdict_id"] == "v-1"
|
||||
assert rows[0]["user_decision"] == "blanket"
|
||||
|
||||
|
||||
def test_auto_approve_reasons_ttl_prune_drops_stale_entries() -> None:
|
||||
"""Lazy TTL eviction at write time: entries older than
|
||||
``_AUTO_APPROVE_REASON_TTL`` are pruned on the next
|
||||
``_record_auto_approves`` call. Without this, a session with the
|
||||
LLM judge disabled would accumulate entries that never get
|
||||
consumed."""
|
||||
import time as time_module
|
||||
|
||||
storage = MagicMock()
|
||||
ui = _make_ui()
|
||||
# Seed two stale entries (well past the TTL).
|
||||
stale_ts = time_module.time() - ui._AUTO_APPROVE_REASON_TTL - 30.0
|
||||
ui._auto_approve_reasons["c-stale-1"] = ("policy", stale_ts)
|
||||
ui._auto_approve_reasons["c-stale-2"] = ("blanket", stale_ts)
|
||||
items = [
|
||||
{
|
||||
"call_id": "c-fresh",
|
||||
"auto_approved": True,
|
||||
"auto_approve_reason": "skill",
|
||||
"func_name": "bash",
|
||||
}
|
||||
]
|
||||
with _patch_get_storage(storage):
|
||||
ui._record_auto_approves(items)
|
||||
# Stale entries pruned; only the fresh one remains.
|
||||
assert "c-stale-1" not in ui._auto_approve_reasons
|
||||
assert "c-stale-2" not in ui._auto_approve_reasons
|
||||
assert "c-fresh" in ui._auto_approve_reasons
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Output guard persistence
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
@@ -232,59 +232,52 @@ class TestSoftCap:
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# valid_until predicate
|
||||
# Predicate independence
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestValidUntil:
|
||||
"""The ``valid_until`` predicate captured at dispatch time re-checks
|
||||
the watch's ``active`` flag at drain time, so a cancelled watch's
|
||||
last splat doesn't ride out a future wake.
|
||||
class TestPredicateIndependence:
|
||||
"""The watch closure does NOT wire a ``valid_until`` predicate.
|
||||
|
||||
Earlier the closure wired ``_still_active`` (re-reading
|
||||
``is_watch_active`` at drain time). That predicate raced
|
||||
``WatchRunner._poll_watch``'s commit of ``active=False`` and silently
|
||||
dropped every terminal fire. The closure now enqueues without a
|
||||
predicate; entries survive drain regardless of the row's ``active``
|
||||
column state.
|
||||
"""
|
||||
|
||||
def test_valid_until_drops_when_watch_inactive(self, tmp_db, monkeypatch):
|
||||
def test_drain_delivers_even_when_storage_reports_inactive(self, tmp_db, monkeypatch):
|
||||
session = _make_session_for_dispatch()
|
||||
_runner, dispatch = _register_runner(session)
|
||||
|
||||
# Storage stub returns False at drain time.
|
||||
is_active_calls = patch_session_storage(monkeypatch, active=False)
|
||||
|
||||
dispatch(_reminder("body"), "watch-1")
|
||||
# Drain fires the predicate; entry should NOT be delivered.
|
||||
out = session._nudge_queue.drain({"any"})
|
||||
assert out == []
|
||||
# Predicate ran once with the dispatched watch_id.
|
||||
assert is_active_calls == ["watch-1"]
|
||||
|
||||
def test_valid_until_drops_when_storage_raises(self, tmp_db, monkeypatch):
|
||||
"""The closure's broad-except in the predicate translates a
|
||||
storage-layer exception to ``False`` so the drain doesn't
|
||||
propagate; the predicate captured ``watch_id`` correctly
|
||||
(otherwise storage wouldn't even be touched).
|
||||
"""
|
||||
session = _make_session_for_dispatch()
|
||||
_runner, dispatch = _register_runner(session)
|
||||
|
||||
patch_session_storage(monkeypatch, raise_on_is_active=True)
|
||||
|
||||
dispatch(_reminder("body"), "watch-bound-id")
|
||||
out = session._nudge_queue.drain({"any"})
|
||||
assert out == []
|
||||
|
||||
def test_valid_until_delivers_when_watch_active(self, tmp_db, monkeypatch):
|
||||
"""Happy-path counter-test for the predicate above: the entry
|
||||
DOES drain when the watch is still active.
|
||||
"""
|
||||
session = _make_session_for_dispatch()
|
||||
_runner, dispatch = _register_runner(session)
|
||||
|
||||
patch_session_storage(monkeypatch, active=True)
|
||||
# Even if storage reports active=False, the entry should still
|
||||
# drain — no predicate to drop it.
|
||||
patch_session_storage(monkeypatch, active=False)
|
||||
|
||||
dispatch(_reminder("body"), "watch-1")
|
||||
out = session._nudge_queue.drain({"any"})
|
||||
assert len(out) == 1
|
||||
assert out[0][0] == "watch_triggered"
|
||||
|
||||
def test_dispatch_never_calls_is_watch_active(self, tmp_db, monkeypatch):
|
||||
"""Pin the invariant directly: the closure must NOT consult
|
||||
``storage.is_watch_active`` anywhere along the enqueue + drain
|
||||
path. Without this assertion, a future change that re-wires
|
||||
an ``is_watch_active`` predicate would silently bring back the
|
||||
bug that motivates this whole module.
|
||||
"""
|
||||
session = _make_session_for_dispatch()
|
||||
_runner, dispatch = _register_runner(session)
|
||||
|
||||
is_active_calls = patch_session_storage(monkeypatch, active=True)
|
||||
|
||||
dispatch(_reminder("body"), "watch-bound-id")
|
||||
session._nudge_queue.drain({"any"})
|
||||
assert is_active_calls == [], (
|
||||
f"watch closure must not call is_watch_active; got {is_active_calls!r}"
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Concurrency
|
||||
|
||||
@@ -24,11 +24,15 @@ the structural integration gate for the watch switchover.
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import contextlib
|
||||
from typing import Any
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
import pytest
|
||||
|
||||
from tests._helpers import patch_session_storage
|
||||
from turnstone.core.session import ChatSession
|
||||
from turnstone.core.storage import get_storage
|
||||
from turnstone.core.watch import WatchRunner
|
||||
|
||||
|
||||
@@ -272,3 +276,283 @@ def test_watch_dispatch_through_restore_fn_lands_on_rehydrated_session(tmp_db, m
|
||||
# Original session's queue stays empty — the dispatch did NOT
|
||||
# accidentally route back to it.
|
||||
assert len(original._nudge_queue) == 0
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("stop_on", "max_polls", "label"),
|
||||
[
|
||||
('"HIT" in output', 100, "stop_on_fired"),
|
||||
(None, 1, "max_polls_reached"),
|
||||
],
|
||||
)
|
||||
def test_poll_watch_terminal_fire_survives_drain(
|
||||
tmp_db: str,
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
stop_on: str | None,
|
||||
max_polls: int,
|
||||
label: str,
|
||||
) -> None:
|
||||
"""Regression for the dispatch-ordering bug.
|
||||
|
||||
With the broken ordering (``update_watch(active=False)`` before
|
||||
``_dispatch_result``) plus the ``_still_active`` ``valid_until``
|
||||
predicate that re-reads ``is_watch_active`` at drain time, every
|
||||
terminal watch fire was silently dropped — the closure enqueued
|
||||
the entry but the predicate immediately invalidated it because
|
||||
the row's ``active`` flag had already been flipped to ``0`` in
|
||||
the same poll. The model never saw the fire.
|
||||
|
||||
This test drives a REAL ``WatchRunner._poll_watch`` against a real
|
||||
``tmp_db`` watch row (no ``patch_session_storage(active=True)``
|
||||
stub — that stub is exactly what masked the bug in earlier tests).
|
||||
Covers both terminal paths: ``stop_on`` condition matched and
|
||||
``poll_count >= max_polls`` reached.
|
||||
"""
|
||||
session = _make_session()
|
||||
storage = get_storage()
|
||||
|
||||
runner = WatchRunner(storage=storage, node_id="test-node")
|
||||
session.set_watch_runner(runner)
|
||||
|
||||
storage.create_watch(
|
||||
watch_id=f"w-regression-{label}",
|
||||
ws_id=session._ws_id,
|
||||
node_id="test-node",
|
||||
name=f"regression-{label}",
|
||||
command="echo HIT",
|
||||
interval_secs=10.0,
|
||||
stop_on=stop_on,
|
||||
max_polls=max_polls,
|
||||
created_by="model",
|
||||
next_poll="1970-01-01T00:00:00",
|
||||
)
|
||||
|
||||
# Spy ``enqueue`` so the assertion can distinguish "dispatch never
|
||||
# called" (a different bug class) from "dispatch enqueued but the
|
||||
# predicate dropped it at drain" (this bug).
|
||||
enqueue_calls: list[tuple[str, str, str]] = []
|
||||
real_enqueue = session._nudge_queue.enqueue
|
||||
|
||||
def _spy_enqueue(*args: Any, **kwargs: Any) -> None:
|
||||
enqueue_calls.append((args[0], args[1][:40], args[2]))
|
||||
return real_enqueue(*args, **kwargs)
|
||||
|
||||
monkeypatch.setattr(session._nudge_queue, "enqueue", _spy_enqueue)
|
||||
|
||||
# For the max_polls=1 case the first poll has prev_output=None and
|
||||
# would not normally fire on output change; the max_polls branch
|
||||
# at watch.py:412-414 still marks is_final=True so dispatch runs.
|
||||
due = storage.list_due_watches("2099-01-01T00:00:00")
|
||||
matching = [r for r in due if r["watch_id"] == f"w-regression-{label}"]
|
||||
assert len(matching) == 1, f"watch row not picked up by list_due_watches: {due!r}"
|
||||
runner._poll_watch(matching[0])
|
||||
|
||||
assert len(enqueue_calls) == 1, (
|
||||
f"_poll_watch did not enqueue exactly one fire (got {enqueue_calls!r}); "
|
||||
"this is a different bug from the predicate-drop regression"
|
||||
)
|
||||
assert enqueue_calls[0][0] == "watch_triggered"
|
||||
|
||||
assert storage.is_watch_active(f"w-regression-{label}") is False, (
|
||||
"terminal fire should have committed active=False on the row"
|
||||
)
|
||||
|
||||
# The key assertion: drain delivers the entry. Pre-fix this
|
||||
# returned ``[]`` because the ``_still_active`` predicate re-read
|
||||
# ``active=0``. Post-fix the watch closure no longer wires a
|
||||
# predicate and the entry survives.
|
||||
out = session._nudge_queue.drain({"any"})
|
||||
assert len(out) == 1, (
|
||||
"Watch fire was enqueued but never reached drain — dispatch-ordering "
|
||||
"regression. Check that WatchRunner._poll_watch dispatches BEFORE "
|
||||
"committing active=False, and that the watch closure in "
|
||||
"ChatSession.set_watch_runner does not wire an is_watch_active "
|
||||
"predicate."
|
||||
)
|
||||
nt, text, _meta = out[0]
|
||||
assert nt == "watch_triggered"
|
||||
assert "HIT" in text
|
||||
|
||||
|
||||
def test_cancel_reports_already_completed_for_auto_cancelled_watch(tmp_db: str) -> None:
|
||||
"""After a watch fires and auto-cancels, the cancel-by-name path
|
||||
should report 'already completed' rather than 'not found'.
|
||||
|
||||
Pre-fix, ``_exec_watch`` cancel looked the watch up via
|
||||
``list_watches_for_ws`` which filters ``active==1``, so a recently-
|
||||
auto-cancelled row was invisible and the model got the same
|
||||
'not found' message it would for a typo'd name. Post-fix the
|
||||
cancel path uses ``find_watch_by_name`` (no active filter) and
|
||||
branches on ``row["active"]``.
|
||||
"""
|
||||
session = _make_session()
|
||||
storage = get_storage()
|
||||
|
||||
storage.create_watch(
|
||||
watch_id="w-completed-1",
|
||||
ws_id=session._ws_id,
|
||||
node_id="test-node",
|
||||
name="completed-watch",
|
||||
command="echo x",
|
||||
interval_secs=10.0,
|
||||
stop_on=None,
|
||||
max_polls=100,
|
||||
created_by="model",
|
||||
next_poll="",
|
||||
)
|
||||
# Simulate the post-fire state.
|
||||
storage.update_watch("w-completed-1", active=False, next_poll="")
|
||||
|
||||
_call_id, msg = session._exec_watch(
|
||||
{"call_id": "c1", "action": "cancel", "watch_name": "completed-watch"}
|
||||
)
|
||||
|
||||
assert "not found" not in msg.lower()
|
||||
assert "completed" in msg.lower()
|
||||
|
||||
|
||||
def test_cancel_reports_not_found_for_unknown_watch(tmp_db: str) -> None:
|
||||
"""The 'not found' message still applies when the watch genuinely
|
||||
does not exist — make sure the new ``find_watch_by_name`` path
|
||||
didn't accidentally turn every cancel into 'already completed'.
|
||||
"""
|
||||
session = _make_session()
|
||||
|
||||
_call_id, msg = session._exec_watch(
|
||||
{"call_id": "c1", "action": "cancel", "watch_name": "ghost-watch"}
|
||||
)
|
||||
|
||||
assert "not found" in msg.lower()
|
||||
|
||||
|
||||
def test_poll_watch_retry_deactivate_after_update_watch_failure(
|
||||
tmp_db: str, monkeypatch: pytest.MonkeyPatch
|
||||
) -> None:
|
||||
"""``_terminal_dispatched`` lifecycle: if ``update_watch`` raises
|
||||
AFTER ``_dispatch_result`` shipped the reminder for a terminal
|
||||
fire, the next ``_poll_watch`` tick MUST retry the row write
|
||||
(so the row stops appearing in ``list_due_watches``) and MUST NOT
|
||||
re-dispatch the reminder the model already saw.
|
||||
|
||||
This is the keystone path that prevents duplicate-fire under
|
||||
transient storage failure. Pre-this-test, the entire branch was
|
||||
unexercised.
|
||||
"""
|
||||
session = _make_session()
|
||||
storage = get_storage()
|
||||
runner = WatchRunner(storage=storage, node_id="test-node")
|
||||
session.set_watch_runner(runner)
|
||||
|
||||
watch_id = "w-retry-1"
|
||||
storage.create_watch(
|
||||
watch_id=watch_id,
|
||||
ws_id=session._ws_id,
|
||||
node_id="test-node",
|
||||
name="retry-watch",
|
||||
command="echo HIT",
|
||||
interval_secs=10.0,
|
||||
stop_on='"HIT" in output',
|
||||
max_polls=100,
|
||||
created_by="model",
|
||||
next_poll="1970-01-01T00:00:00",
|
||||
)
|
||||
|
||||
enqueue_calls: list[tuple[str, str]] = []
|
||||
real_enqueue = session._nudge_queue.enqueue
|
||||
|
||||
def _spy_enqueue(*args: Any, **kwargs: Any) -> None:
|
||||
enqueue_calls.append((args[0], args[1][:32]))
|
||||
return real_enqueue(*args, **kwargs)
|
||||
|
||||
monkeypatch.setattr(session._nudge_queue, "enqueue", _spy_enqueue)
|
||||
|
||||
# Stage 1 — first poll. ``update_watch`` raises AFTER dispatch.
|
||||
real_update = storage.update_watch
|
||||
update_raise = {"armed": True}
|
||||
|
||||
def _failing_update(wid: str, **fields: Any) -> bool:
|
||||
if update_raise["armed"]:
|
||||
raise RuntimeError("simulated transient storage failure")
|
||||
return real_update(wid, **fields)
|
||||
|
||||
monkeypatch.setattr(storage, "update_watch", _failing_update)
|
||||
|
||||
due = storage.list_due_watches("2099-01-01T00:00:00")
|
||||
matching = [r for r in due if r["watch_id"] == watch_id]
|
||||
assert len(matching) == 1
|
||||
# ``_poll_watch`` doesn't catch the storage error; the outer
|
||||
# ``_tick`` would log it. Suppress here so the test owns the
|
||||
# boundary and continues to its assertions.
|
||||
with contextlib.suppress(RuntimeError):
|
||||
runner._poll_watch(matching[0])
|
||||
|
||||
# Dispatch ran exactly once and the watch_id sits in the
|
||||
# terminal-dispatched set awaiting retry.
|
||||
assert len(enqueue_calls) == 1
|
||||
assert enqueue_calls[0][0] == "watch_triggered"
|
||||
assert watch_id in runner._terminal_dispatched
|
||||
|
||||
# The row is still active=1 because update_watch raised. It
|
||||
# would re-appear in list_due_watches on the next tick.
|
||||
assert storage.is_watch_active(watch_id) is True
|
||||
|
||||
# Stage 2 — second poll. Storage now succeeds; retry-deactivate
|
||||
# branch must commit active=False WITHOUT re-dispatching.
|
||||
update_raise["armed"] = False
|
||||
|
||||
due = storage.list_due_watches("2099-01-01T00:00:00")
|
||||
matching = [r for r in due if r["watch_id"] == watch_id]
|
||||
assert len(matching) == 1
|
||||
runner._poll_watch(matching[0])
|
||||
|
||||
# Exactly one dispatch in total — the retry path took the
|
||||
# short-circuit return at the top of _poll_watch.
|
||||
assert len(enqueue_calls) == 1, f"retry-deactivate must not re-dispatch; got {enqueue_calls!r}"
|
||||
# Row is now inactive (the retry path's update_watch landed).
|
||||
assert storage.is_watch_active(watch_id) is False
|
||||
# Set is cleared so future watches with the same id (unlikely) /
|
||||
# process memory doesn't accumulate.
|
||||
assert watch_id not in runner._terminal_dispatched
|
||||
|
||||
|
||||
def test_cancel_clears_pending_terminal_dispatched_entry(
|
||||
tmp_db: str, monkeypatch: pytest.MonkeyPatch
|
||||
) -> None:
|
||||
"""If ``update_watch`` raised after dispatch, leaving a pending
|
||||
entry in ``_terminal_dispatched``, and the user then cancels the
|
||||
watch out-of-band, the retry-deactivate branch never gets to run
|
||||
(the cancel sets ``next_poll=""`` which removes the row from
|
||||
``list_due_watches``). The cancel path itself must discard the
|
||||
pending entry; otherwise the runner leaks ``watch_id``s for the
|
||||
process lifetime.
|
||||
"""
|
||||
session = _make_session()
|
||||
storage = get_storage()
|
||||
runner = WatchRunner(storage=storage, node_id="test-node")
|
||||
session.set_watch_runner(runner)
|
||||
|
||||
watch_id = "w-leak-1"
|
||||
storage.create_watch(
|
||||
watch_id=watch_id,
|
||||
ws_id=session._ws_id,
|
||||
node_id="test-node",
|
||||
name="leak-watch",
|
||||
command="echo x",
|
||||
interval_secs=10.0,
|
||||
stop_on=None,
|
||||
max_polls=100,
|
||||
created_by="model",
|
||||
next_poll="",
|
||||
)
|
||||
# Simulate: dispatch shipped, update_watch raised, watch_id sits
|
||||
# in the runner's pending set.
|
||||
with runner._terminal_dispatched_lock:
|
||||
runner._terminal_dispatched.add(watch_id)
|
||||
|
||||
# User cancels. Because the cancel writes active=False, next_poll="",
|
||||
# the row leaves list_due_watches and the runner's retry-deactivate
|
||||
# branch never executes for it. The cancel must discard the entry.
|
||||
storage.update_watch(watch_id, active=False, next_poll="")
|
||||
session._exec_watch({"call_id": "c1", "action": "cancel", "watch_name": "leak-watch"})
|
||||
|
||||
assert watch_id not in runner._terminal_dispatched
|
||||
|
||||
@@ -2,6 +2,10 @@
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import sqlalchemy as sa
|
||||
|
||||
from turnstone.core.storage._schema import watches as watches_table
|
||||
|
||||
|
||||
def _make_watch_kwargs(**overrides):
|
||||
"""Build default kwargs for create_watch."""
|
||||
@@ -101,6 +105,91 @@ class TestWatchListQueries:
|
||||
db.update_watch("w1", active=False)
|
||||
assert db.list_watches_for_ws("ws-1") == []
|
||||
|
||||
def test_find_by_name_returns_inactive(self, db):
|
||||
"""``find_watch_by_name`` ignores the active filter — that is
|
||||
what lets the cancel-by-name UX distinguish 'already completed'
|
||||
from 'no such watch.'
|
||||
"""
|
||||
db.create_watch(**_make_watch_kwargs(watch_id="w1", ws_id="ws-1", name="completed"))
|
||||
db.update_watch("w1", active=False)
|
||||
|
||||
row = db.find_watch_by_name("ws-1", "completed")
|
||||
assert row is not None
|
||||
assert row["watch_id"] == "w1"
|
||||
assert not row["active"]
|
||||
|
||||
def test_find_by_name_matches_watch_id_prefix(self, db):
|
||||
db.create_watch(**_make_watch_kwargs(watch_id="abcdef123", ws_id="ws-1", name="x"))
|
||||
row = db.find_watch_by_name("ws-1", "abc")
|
||||
assert row is not None
|
||||
assert row["watch_id"] == "abcdef123"
|
||||
|
||||
def test_find_by_name_scoped_to_ws(self, db):
|
||||
db.create_watch(**_make_watch_kwargs(watch_id="w1", ws_id="ws-1", name="shared"))
|
||||
db.create_watch(**_make_watch_kwargs(watch_id="w2", ws_id="ws-2", name="shared"))
|
||||
|
||||
row = db.find_watch_by_name("ws-1", "shared")
|
||||
assert row is not None
|
||||
assert row["watch_id"] == "w1"
|
||||
|
||||
def test_find_by_name_returns_none_when_missing(self, db):
|
||||
assert db.find_watch_by_name("ws-1", "ghost") is None
|
||||
|
||||
def test_find_by_name_empty_input_returns_none(self, db):
|
||||
db.create_watch(**_make_watch_kwargs(watch_id="w1", ws_id="ws-1", name="x"))
|
||||
assert db.find_watch_by_name("ws-1", "") is None
|
||||
|
||||
def test_find_by_name_treats_percent_as_literal(self, db):
|
||||
"""A model-supplied '%' must NOT match arbitrary watch_ids.
|
||||
|
||||
Pre-escape, ``watch_id.like(f"{name_or_prefix}%")`` would
|
||||
interpret '%' as 'match anything' and pick up the first row in
|
||||
the workstream regardless of name.
|
||||
"""
|
||||
db.create_watch(**_make_watch_kwargs(watch_id="w1", ws_id="ws-1", name="real-watch"))
|
||||
assert db.find_watch_by_name("ws-1", "%") is None
|
||||
|
||||
def test_find_by_name_treats_underscore_as_literal(self, db):
|
||||
"""Same as the '%' case for the single-char LIKE wildcard."""
|
||||
db.create_watch(**_make_watch_kwargs(watch_id="abcd", ws_id="ws-1", name="real-watch"))
|
||||
# '_' would otherwise match any single char, picking up
|
||||
# watch_ids beginning with 'a', 'b', etc.
|
||||
assert db.find_watch_by_name("ws-1", "_") is None
|
||||
|
||||
def test_find_by_name_prefers_active_over_newer_inactive(self, db):
|
||||
"""If a same-name pair exists where the inactive row is NEWER
|
||||
than the active row, find_watch_by_name must still return the
|
||||
active row. Pre-fix the query was ``ORDER BY created DESC
|
||||
LIMIT 1`` — which would return the newer inactive row and
|
||||
cause the cancel UX to report 'already completed' for a name
|
||||
whose live row is still polling.
|
||||
|
||||
Reachable in practice because storage allows out-of-band
|
||||
writes (e.g. ``delete_watches_for_ws`` cleanup followed by
|
||||
re-create, an admin manually flipping ``active``, or test
|
||||
scaffolding) that bypass the create-time duplicate-name
|
||||
guard.
|
||||
"""
|
||||
# Older active watch.
|
||||
db.create_watch(**_make_watch_kwargs(watch_id="w-active", ws_id="ws-1", name="recurring"))
|
||||
# Newer inactive watch with the same name. ``create_watch``
|
||||
# stamps ``created`` to ``now`` at second resolution, so we
|
||||
# bypass the API to give the inactive row a deterministically
|
||||
# later timestamp.
|
||||
db.create_watch(**_make_watch_kwargs(watch_id="w-inactive", ws_id="ws-1", name="recurring"))
|
||||
with db._conn() as conn:
|
||||
conn.execute(
|
||||
sa.update(watches_table)
|
||||
.where(watches_table.c.watch_id == "w-inactive")
|
||||
.values(active=0, next_poll="", created="2099-01-01T00:00:00")
|
||||
)
|
||||
conn.commit()
|
||||
|
||||
row = db.find_watch_by_name("ws-1", "recurring")
|
||||
assert row is not None
|
||||
assert row["watch_id"] == "w-active"
|
||||
assert row["active"]
|
||||
|
||||
def test_list_for_node(self, db):
|
||||
db.create_watch(**_make_watch_kwargs(watch_id="w1", node_id="n1"))
|
||||
db.create_watch(**_make_watch_kwargs(watch_id="w2", node_id="n1"))
|
||||
|
||||
@@ -29,7 +29,7 @@ class TestVersionHtml:
|
||||
def test_vendored_katex_skipped(self):
|
||||
from turnstone.core.web_helpers import version_html
|
||||
|
||||
html = '<link rel="stylesheet" href="/shared/katex-0.16.44/katex.min.css">'
|
||||
html = '<link rel="stylesheet" href="/shared/katex-0.16.47/katex.min.css">'
|
||||
result = version_html(html)
|
||||
assert result == html # unchanged
|
||||
|
||||
@@ -43,14 +43,14 @@ class TestVersionHtml:
|
||||
def test_vendored_mermaid_skipped(self):
|
||||
from turnstone.core.web_helpers import version_html
|
||||
|
||||
html = '<script src="/shared/mermaid-11.14.0/mermaid.min.js"></script>'
|
||||
html = '<script src="/shared/mermaid-11.15.0/mermaid.min.js"></script>'
|
||||
result = version_html(html)
|
||||
assert result == html # unchanged
|
||||
|
||||
def test_vendored_hls_skipped(self):
|
||||
from turnstone.core.web_helpers import version_html
|
||||
|
||||
html = '<script src="/shared/hls-1.6.15/hls.min.js"></script>'
|
||||
html = '<script src="/shared/hls-1.6.16/hls.min.js"></script>'
|
||||
result = version_html(html)
|
||||
assert result == html # unchanged
|
||||
|
||||
@@ -76,7 +76,7 @@ class TestVersionHtml:
|
||||
|
||||
html = (
|
||||
'<link rel="stylesheet" href="/shared/base.css">\n'
|
||||
'<link rel="stylesheet" href="/shared/katex-0.16.44/katex.min.css">\n'
|
||||
'<link rel="stylesheet" href="/shared/katex-0.16.47/katex.min.css">\n'
|
||||
'<link rel="stylesheet" href="/static/style.css">\n'
|
||||
'<script src="/shared/utils.js"></script>\n'
|
||||
'<script src="/shared/hljs-11.11.1/highlight.min.js"></script>\n'
|
||||
@@ -88,7 +88,7 @@ class TestVersionHtml:
|
||||
assert f'/shared/utils.js?v={__version__}"' in result
|
||||
assert f'/static/app.js?v={__version__}"' in result
|
||||
# Vendored libs unchanged
|
||||
assert '/shared/katex-0.16.44/katex.min.css"' in result
|
||||
assert '/shared/katex-0.16.47/katex.min.css"' in result
|
||||
assert '/shared/hljs-11.11.1/highlight.min.js"' in result
|
||||
|
||||
def test_version_matches_package(self):
|
||||
|
||||
@@ -1,3 +1,3 @@
|
||||
"""turnstone - Multi-node AI orchestration platform with tool use, agent routing, and cluster simulation."""
|
||||
|
||||
__version__ = "1.5.13"
|
||||
__version__ = "1.5.18"
|
||||
|
||||
+40
-14
@@ -12,14 +12,36 @@ import uuid
|
||||
from typing import Any
|
||||
|
||||
|
||||
def _get_storage() -> Any:
|
||||
"""Initialize and return the storage backend."""
|
||||
def _get_storage(args: argparse.Namespace) -> Any:
|
||||
"""Initialize and return the storage backend.
|
||||
|
||||
Precedence (matches turnstone-server): CLI / config.toml ``[database]``
|
||||
> ``TURNSTONE_DB_*`` env vars > hardcoded defaults.
|
||||
"""
|
||||
from turnstone.core.storage import init_storage
|
||||
|
||||
db_backend = os.environ.get("TURNSTONE_DB_BACKEND", "sqlite")
|
||||
db_url = os.environ.get("TURNSTONE_DB_URL", "")
|
||||
db_path = os.environ.get("TURNSTONE_DB_PATH", "")
|
||||
return init_storage(db_backend, path=db_path, url=db_url)
|
||||
def _pick(arg_name: str, env_name: str, default: str = "") -> Any:
|
||||
# `is not None` (not truthy) so a legitimate falsy TOML value
|
||||
# like `pool_size = 0` or `url = ""` still beats the env fallback.
|
||||
val = getattr(args, arg_name, None)
|
||||
if val is not None:
|
||||
return val
|
||||
return os.environ.get(env_name, default)
|
||||
|
||||
db_backend = str(_pick("db_backend", "TURNSTONE_DB_BACKEND", "sqlite"))
|
||||
db_url = str(_pick("db_url", "TURNSTONE_DB_URL"))
|
||||
db_path = str(_pick("db_path", "TURNSTONE_DB_PATH"))
|
||||
db_pool_size = int(_pick("db_pool_size", "TURNSTONE_DB_POOL_SIZE", "2"))
|
||||
return init_storage(
|
||||
db_backend,
|
||||
path=db_path,
|
||||
url=db_url,
|
||||
pool_size=db_pool_size,
|
||||
sslmode=str(_pick("db_sslmode", "TURNSTONE_DB_SSLMODE")),
|
||||
sslrootcert=str(_pick("db_sslrootcert", "TURNSTONE_DB_SSLROOTCERT")),
|
||||
sslcert=str(_pick("db_sslcert", "TURNSTONE_DB_SSLCERT")),
|
||||
sslkey=str(_pick("db_sslkey", "TURNSTONE_DB_SSLKEY")),
|
||||
)
|
||||
|
||||
|
||||
def _cmd_create_user(args: argparse.Namespace) -> None:
|
||||
@@ -37,7 +59,7 @@ def _cmd_create_user(args: argparse.Namespace) -> None:
|
||||
print("Error: invalid username (1-64 chars: letters, digits, . _ -)", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
storage = _get_storage()
|
||||
storage = _get_storage(args)
|
||||
user_id = uuid.uuid4().hex
|
||||
|
||||
# Prompt for password
|
||||
@@ -76,7 +98,7 @@ def _cmd_create_user(args: argparse.Namespace) -> None:
|
||||
def _cmd_create_token(args: argparse.Namespace) -> None:
|
||||
from turnstone.core.auth import generate_token, hash_token, token_prefix
|
||||
|
||||
storage = _get_storage()
|
||||
storage = _get_storage(args)
|
||||
|
||||
if storage.get_user(args.user) is None:
|
||||
print(f"Error: user {args.user} not found", file=sys.stderr)
|
||||
@@ -110,7 +132,7 @@ def _cmd_create_token(args: argparse.Namespace) -> None:
|
||||
|
||||
|
||||
def _cmd_list_users(args: argparse.Namespace) -> None:
|
||||
storage = _get_storage()
|
||||
storage = _get_storage(args)
|
||||
users = storage.list_users()
|
||||
if not users:
|
||||
print("No users found.")
|
||||
@@ -120,7 +142,7 @@ def _cmd_list_users(args: argparse.Namespace) -> None:
|
||||
|
||||
|
||||
def _cmd_list_tokens(args: argparse.Namespace) -> None:
|
||||
storage = _get_storage()
|
||||
storage = _get_storage(args)
|
||||
tokens = storage.list_api_tokens(args.user)
|
||||
if not tokens:
|
||||
print(f"No tokens found for user {args.user}.")
|
||||
@@ -134,7 +156,7 @@ def _cmd_list_tokens(args: argparse.Namespace) -> None:
|
||||
|
||||
|
||||
def _cmd_revoke_token(args: argparse.Namespace) -> None:
|
||||
storage = _get_storage()
|
||||
storage = _get_storage(args)
|
||||
if storage.delete_api_token(args.token_id):
|
||||
print(f"Revoked token {args.token_id}")
|
||||
else:
|
||||
@@ -297,7 +319,7 @@ def _cmd_list_node_metadata(args: argparse.Namespace) -> None:
|
||||
"""List metadata for a node."""
|
||||
import json
|
||||
|
||||
storage = _get_storage()
|
||||
storage = _get_storage(args)
|
||||
rows = storage.get_node_metadata(args.node_id)
|
||||
if not rows:
|
||||
print(f"No metadata for node: {args.node_id}")
|
||||
@@ -324,7 +346,7 @@ def _cmd_set_node_metadata(args: argparse.Namespace) -> None:
|
||||
"""Set a metadata key on a node."""
|
||||
import json
|
||||
|
||||
storage = _get_storage()
|
||||
storage = _get_storage(args)
|
||||
|
||||
# Check for auto-source conflict
|
||||
existing = storage.get_node_metadata(args.node_id)
|
||||
@@ -345,7 +367,7 @@ def _cmd_set_node_metadata(args: argparse.Namespace) -> None:
|
||||
|
||||
def _cmd_delete_node_metadata(args: argparse.Namespace) -> None:
|
||||
"""Delete a metadata key from a node."""
|
||||
storage = _get_storage()
|
||||
storage = _get_storage(args)
|
||||
|
||||
existing = storage.get_node_metadata(args.node_id)
|
||||
for r in existing:
|
||||
@@ -395,6 +417,10 @@ def main() -> None:
|
||||
prog="turnstone-admin",
|
||||
description="Turnstone user and token administration",
|
||||
)
|
||||
from turnstone.core.config import add_config_arg, apply_config
|
||||
|
||||
add_config_arg(parser)
|
||||
apply_config(parser, ["database"])
|
||||
sub = parser.add_subparsers(dest="command")
|
||||
|
||||
p_cu = sub.add_parser("create-user", help="Create a new user")
|
||||
|
||||
@@ -1278,21 +1278,29 @@ class CoordinatorClient:
|
||||
allowed_tools: list[str] = [str(t) for t in allowed_full[:_SKILL_TOOLS_PROJECTION_CAP]]
|
||||
if len(allowed_full) > _SKILL_TOOLS_PROJECTION_CAP:
|
||||
allowed_tools.append(f"+{len(allowed_full) - _SKILL_TOOLS_PROJECTION_CAP} more")
|
||||
skills.append(
|
||||
{
|
||||
"name": r.get("name") or "",
|
||||
"category": r.get("category") or "",
|
||||
"tags": tags,
|
||||
"version": r.get("version") or "",
|
||||
"description": r.get("description") or "",
|
||||
"model": r.get("model") or "",
|
||||
"enabled": bool(r.get("enabled")),
|
||||
"risk_level": r.get("risk_level") or "",
|
||||
"activation": r.get("activation") or "",
|
||||
"kind": r["kind"],
|
||||
"allowed_tools": allowed_tools,
|
||||
}
|
||||
)
|
||||
skill_row: dict[str, Any] = {
|
||||
"name": r.get("name") or "",
|
||||
"category": r.get("category") or "",
|
||||
"tags": tags,
|
||||
"version": r.get("version") or "",
|
||||
"description": r.get("description") or "",
|
||||
"model": r.get("model") or "",
|
||||
"enabled": bool(r.get("enabled")),
|
||||
"risk_level": r.get("risk_level") or "",
|
||||
"activation": r.get("activation") or "",
|
||||
"kind": r["kind"],
|
||||
}
|
||||
# Omit ``allowed_tools`` when empty: an empty list reads as
|
||||
# "no tools are usable by this skill" to a model that doesn't
|
||||
# know the semantics, but the actual meaning is "no tools are
|
||||
# pre-approved (auto-approve exemption list)". Real
|
||||
# misdiagnosis happened in testing when a code-review skill
|
||||
# with no auto-approve allowlist looked like it had been
|
||||
# spawned with zero tool access. Dropping the key altogether
|
||||
# when empty removes the ambiguity at the source.
|
||||
if allowed_tools:
|
||||
skill_row["allowed_tools"] = allowed_tools
|
||||
skills.append(skill_row)
|
||||
return {"skills": skills, "truncated": truncated}
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
@@ -1791,6 +1799,293 @@ def _serialize_verdicts(rows: list[Any]) -> list[dict[str, Any]]:
|
||||
return out
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# inspect_workstream — tiered output compression
|
||||
# ---------------------------------------------------------------------------
|
||||
#
|
||||
# A coord doing a fan-out wave of inspect_workstream calls against
|
||||
# tool-heavy children can blow the context budget on raw output alone
|
||||
# (one child with a 100 KB bash result × N children). The previous
|
||||
# safety net was ``_truncate_output``'s head+tail strategy, which
|
||||
# silently drops *middle* messages — exactly the wrong shape for a
|
||||
# coordinator trying to understand a child's trajectory (the LAST
|
||||
# message tells the model what the child concluded; the FIRST sets
|
||||
# the brief; the middle is the connective tissue).
|
||||
#
|
||||
# The three-tier degradation pattern matches the ``search`` tool's
|
||||
# Tier-1/Tier-2/Tier-3 ladder at ``session.py:_format_search_results``.
|
||||
# First tier whose serialized size fits the budget wins; the LLM
|
||||
# learns which tier it got via the ``_tier`` field in the response
|
||||
# (no API change to the coordinator tool).
|
||||
#
|
||||
# Budget chosen well under ``tool_truncation`` (typically 256 KB+) so
|
||||
# the head+tail safety net never fires for inspect_workstream — that
|
||||
# strategy silently drops middle messages, which is exactly the
|
||||
# pathology this formatter exists to avoid.
|
||||
|
||||
_INSPECT_OUTPUT_BUDGET: int = 32_768
|
||||
# Per-message head/tail snip when Tier 2 needs to compress content.
|
||||
# Head dominates because the first ~600 chars of an assistant message
|
||||
# usually contains the conclusion / direction; the tail is the
|
||||
# follow-through. Tool results compress similarly: head shows what
|
||||
# the tool was asked / what it found at the top; tail shows the final
|
||||
# state / error suffix.
|
||||
_INSPECT_MSG_CONTENT_HEAD: int = 600
|
||||
_INSPECT_MSG_CONTENT_TAIL: int = 300
|
||||
# Skeleton-tier preview length on the last assistant message. Single
|
||||
# value because the skeleton wants ONE meaningful signal ("what did
|
||||
# the child last say"), not a head/tail snip.
|
||||
_INSPECT_SKELETON_LAST_PREVIEW: int = 400
|
||||
|
||||
# Snip lengths for tool-call ``function.arguments`` strings on
|
||||
# assistant turns. Tighter than content snipping because tool calls
|
||||
# often appear in clusters (10+ per turn for a fan-out) and the
|
||||
# arguments JSON is dense — keep just enough to see what was invoked
|
||||
# and the head of the args structure.
|
||||
_INSPECT_TOOL_ARG_HEAD: int = 300
|
||||
_INSPECT_TOOL_ARG_TAIL: int = 100
|
||||
|
||||
# Bytes ``_snip_head_tail`` reserves for the elision marker itself
|
||||
# (``\n...[N chars elided]...\n``). A text shorter than
|
||||
# ``head + tail + this margin`` passes through unsnipped — snipping
|
||||
# would cost more bytes (the marker) than it saves.
|
||||
_INSPECT_ELISION_MARGIN: int = 64
|
||||
|
||||
# Message-list trim ladder for the compact tier when per-message
|
||||
# content snipping alone doesn't free enough budget. Each rung is
|
||||
# ``(head_count, tail_count)`` — keep the first N + last M messages,
|
||||
# elide the middle as ``{"_omitted": K}``. Tail-weighted because the
|
||||
# last assistant turn carries the load-bearing "what did the child
|
||||
# conclude" signal (same rationale as ``_inspect_skeleton``'s
|
||||
# last-assistant preview). Tried in order; first rung whose
|
||||
# serialized emission fits the budget wins. Mirrors the per-file
|
||||
# sample ladder in ``_format_search_results`` at session.py:254.
|
||||
_INSPECT_LIST_TRIM_LADDER: tuple[tuple[int, int], ...] = ((20, 30), (10, 20), (5, 10))
|
||||
|
||||
|
||||
def _snip_head_tail(text: str, head: int, tail: int) -> str:
|
||||
"""Head/tail snip with elision marker; passthrough when shorter than threshold."""
|
||||
if not isinstance(text, str) or len(text) <= head + tail + _INSPECT_ELISION_MARGIN:
|
||||
return text
|
||||
elided = len(text) - head - tail
|
||||
return text[:head] + f"\n...[{elided} chars elided]...\n" + text[-tail:]
|
||||
|
||||
|
||||
def _compact_tool_calls(tool_calls: Any) -> Any:
|
||||
"""Snip ``function.arguments`` on each tool-call entry; keep ``id`` and
|
||||
``function.name`` verbatim.
|
||||
|
||||
OpenAI shape: ``[{"id": ..., "type": "function", "function":
|
||||
{"name": ..., "arguments": "<json-string>"}}, ...]``. The
|
||||
arguments string is the dominant size term on a fan-out turn that
|
||||
issued many tool calls with multi-KB JSON arguments each;
|
||||
preserving them verbatim re-opens the same size pressure the
|
||||
compact tier is trying to relieve. Non-list / non-dict entries
|
||||
pass through so a future shape change doesn't crash the formatter.
|
||||
"""
|
||||
if not isinstance(tool_calls, list):
|
||||
return tool_calls
|
||||
out: list[Any] = []
|
||||
for call in tool_calls:
|
||||
if not isinstance(call, dict):
|
||||
out.append(call)
|
||||
continue
|
||||
compact_call: dict[str, Any] = {}
|
||||
for k in ("id", "type"):
|
||||
v = call.get(k)
|
||||
if v:
|
||||
compact_call[k] = v
|
||||
func = call.get("function")
|
||||
if isinstance(func, dict):
|
||||
compact_func: dict[str, Any] = {}
|
||||
name = func.get("name")
|
||||
if name:
|
||||
compact_func["name"] = name
|
||||
args = func.get("arguments", "")
|
||||
if args:
|
||||
compact_func["arguments"] = _snip_head_tail(
|
||||
args, _INSPECT_TOOL_ARG_HEAD, _INSPECT_TOOL_ARG_TAIL
|
||||
)
|
||||
compact_call["function"] = compact_func
|
||||
out.append(compact_call)
|
||||
return out
|
||||
|
||||
|
||||
def _compact_message(msg: dict[str, Any]) -> dict[str, Any]:
|
||||
"""Tier-2 per-message projection: keep role + identifier keys, snip content + tool_calls.
|
||||
|
||||
Tool-call linkage is the load-bearing "what happened" signal:
|
||||
``tool_call_id`` on the result side matches an ``id`` in
|
||||
``tool_calls`` on the issuing assistant turn. Stripping
|
||||
``tool_calls`` (the pre-fix shape) left tool results dangling
|
||||
against an invisible call — the audit reader could see "bash
|
||||
returned X" but not "the assistant asked for ``ls /tmp``". The
|
||||
``arguments`` string is the size offender, so we snip it head/tail
|
||||
rather than dropping the call entirely.
|
||||
"""
|
||||
content = msg.get("content", "")
|
||||
snipped = _snip_head_tail(content, _INSPECT_MSG_CONTENT_HEAD, _INSPECT_MSG_CONTENT_TAIL)
|
||||
compact: dict[str, Any] = {"role": msg.get("role"), "content": snipped}
|
||||
# Tool-result linkage (result-side keys).
|
||||
for k in ("tool_name", "tool_call_id", "name"):
|
||||
v = msg.get(k)
|
||||
if v:
|
||||
compact[k] = v
|
||||
# Tool-call request linkage (issuing-side list), snipped per-call.
|
||||
tool_calls = msg.get("tool_calls")
|
||||
if tool_calls:
|
||||
compact["tool_calls"] = _compact_tool_calls(tool_calls)
|
||||
return compact
|
||||
|
||||
|
||||
def _inspect_skeleton(result: dict[str, Any]) -> dict[str, Any]:
|
||||
"""Tier-3 fallback: state + counts + last assistant preview + terminal info.
|
||||
|
||||
Drops every message, keeping only aggregate signal: state, message
|
||||
count, role distribution, verdict count + risk distribution, and a
|
||||
short preview of the most recent assistant turn (the "what did this
|
||||
child last say" signal). Terminal-state fields (``close_reason``,
|
||||
``last_error``) and the ``live`` block pass through unchanged
|
||||
because they're already small and load-bearing.
|
||||
"""
|
||||
messages = result.get("messages") or []
|
||||
verdicts = result.get("verdicts") or []
|
||||
role_counts: dict[str, int] = {}
|
||||
for m in messages:
|
||||
role = m.get("role") if isinstance(m, dict) else None
|
||||
if role:
|
||||
role_counts[role] = role_counts.get(role, 0) + 1
|
||||
verdicts_by_risk: dict[str, int] = {}
|
||||
for v in verdicts:
|
||||
if isinstance(v, dict):
|
||||
risk = v.get("risk_level") or "unknown"
|
||||
verdicts_by_risk[risk] = verdicts_by_risk.get(risk, 0) + 1
|
||||
last_preview = ""
|
||||
for m in reversed(messages):
|
||||
if not isinstance(m, dict) or m.get("role") != "assistant":
|
||||
continue
|
||||
c = m.get("content", "")
|
||||
if isinstance(c, str) and c:
|
||||
last_preview = c[:_INSPECT_SKELETON_LAST_PREVIEW]
|
||||
if len(c) > _INSPECT_SKELETON_LAST_PREVIEW:
|
||||
last_preview += "..."
|
||||
break
|
||||
skeleton: dict[str, Any] = {
|
||||
# Storage row keys verbatim from ``get_workstreams_batch``
|
||||
# (the projection backing ``get_workstream`` → ``inspect()``):
|
||||
# ``ws_id``, ``skill_id``. No fallback to ``id`` / ``skill``
|
||||
# — fail loud on storage column drift rather than silently
|
||||
# emitting null.
|
||||
"ws_id": result["ws_id"],
|
||||
"state": result.get("state"),
|
||||
"title": result.get("title"),
|
||||
"skill": result["skill_id"],
|
||||
"message_count": len(messages),
|
||||
"roles": role_counts,
|
||||
"verdict_count": len(verdicts),
|
||||
"verdicts_by_risk": verdicts_by_risk,
|
||||
"last_assistant_preview": last_preview,
|
||||
"_tier": "skeleton",
|
||||
"_tier_note": (
|
||||
"Output exceeded the inspect_workstream budget at both full and compact "
|
||||
"tiers; skeleton-only. Re-call with a smaller ``message_limit`` to fit "
|
||||
"the compact tier, or read individual messages via the storage admin path."
|
||||
),
|
||||
}
|
||||
for k in ("close_reason", "last_error", "live"):
|
||||
v = result.get(k)
|
||||
if v:
|
||||
skeleton[k] = v
|
||||
return skeleton
|
||||
|
||||
|
||||
def _format_inspect_tiered(result: dict[str, Any], *, budget: int = _INSPECT_OUTPUT_BUDGET) -> str:
|
||||
"""Serialize an ``inspect_workstream`` result with tiered degradation.
|
||||
|
||||
Tier 1 (full): every message verbatim — used when the size fits.
|
||||
Tier 2 (compact): per-message ``{role, head/tail-snipped content,
|
||||
tool linkage, snipped tool_calls.arguments}`` for
|
||||
every message, then a head+tail message-list trim
|
||||
ladder when content snipping alone doesn't free
|
||||
enough budget.
|
||||
Tier 3 (skeleton): no messages — counts + last assistant preview only.
|
||||
|
||||
First emission whose JSON serialization fits ``budget`` wins.
|
||||
``_tier`` appears on every non-error emission so the coordinator
|
||||
LLM (and any audit reader) can see which compression rung the
|
||||
output landed on without inferring from length. Error-shape
|
||||
results (missing or cross-tenant ws_id) bypass tiering entirely —
|
||||
they're already small and the ``error`` key signals the shape.
|
||||
|
||||
The intermediate Tier-2 list-trim rungs exist because content
|
||||
snipping alone fails on workloads where many small messages
|
||||
overflow the budget by sheer count (``message_limit=200`` × a few
|
||||
hundred chars each). In that regime, dropping content-snipping
|
||||
saves zero bytes per message, so without the list-trim ladder
|
||||
Tier-2 produces output strictly larger than Tier-1 (added
|
||||
``_tier_note``) and the formatter fell through to skeleton —
|
||||
losing every message when a head+tail message-list trim would
|
||||
have preserved dozens. Mirrors the per-file sample ladder in
|
||||
``_format_search_results`` (session.py:_SEARCH_TIER2_SAMPLE_LADDER).
|
||||
"""
|
||||
if "error" in result:
|
||||
# Cross-tenant guard / not-found responses — pass through.
|
||||
return json.dumps(result, default=str, separators=(",", ":"))
|
||||
tier1 = {**result, "_tier": "full"}
|
||||
out1 = json.dumps(tier1, default=str, separators=(",", ":"))
|
||||
if len(out1) <= budget:
|
||||
return out1
|
||||
messages = result.get("messages") or []
|
||||
compact_msgs = [_compact_message(m) if isinstance(m, dict) else m for m in messages]
|
||||
tier2_note_full = (
|
||||
"Output exceeded the inspect_workstream budget at the full tier; messages "
|
||||
"are head/tail-snipped at "
|
||||
f"{_INSPECT_MSG_CONTENT_HEAD}/{_INSPECT_MSG_CONTENT_TAIL} chars. Re-call "
|
||||
"with a smaller ``message_limit`` for a tighter tail, or include_provider_"
|
||||
"content=False if it was on."
|
||||
)
|
||||
tier2 = {
|
||||
**result,
|
||||
"messages": compact_msgs,
|
||||
"_tier": "compact",
|
||||
"_tier_note": tier2_note_full,
|
||||
}
|
||||
out2 = json.dumps(tier2, default=str, separators=(",", ":"))
|
||||
if len(out2) <= budget:
|
||||
return out2
|
||||
# Tier-2 list-trim ladder: keep head N + tail M, elide the middle.
|
||||
# Tail-weighted because the recent turns carry the load-bearing
|
||||
# signal ("what did the child conclude") — same reason
|
||||
# ``_inspect_skeleton`` keeps a last-assistant preview rather than
|
||||
# a first-user preview.
|
||||
total = len(compact_msgs)
|
||||
for head_n, tail_n in _INSPECT_LIST_TRIM_LADDER:
|
||||
if head_n + tail_n >= total:
|
||||
# Rung doesn't actually trim — would re-emit Tier-2 verbatim.
|
||||
continue
|
||||
omitted = total - head_n - tail_n
|
||||
trimmed: list[Any] = (
|
||||
compact_msgs[:head_n] + [{"_omitted": omitted}] + compact_msgs[-tail_n:]
|
||||
)
|
||||
tier2_trim_note = (
|
||||
f"Output exceeded the inspect_workstream budget at the compact tier; "
|
||||
f"keeping first {head_n} + last {tail_n} of {total} messages, eliding "
|
||||
f"{omitted} middle messages. Re-call with a smaller ``message_limit`` "
|
||||
"to fit the full compact tier."
|
||||
)
|
||||
tier2_trim = {
|
||||
**result,
|
||||
"messages": trimmed,
|
||||
"_tier": "compact",
|
||||
"_tier_note": tier2_trim_note,
|
||||
}
|
||||
out2_trim = json.dumps(tier2_trim, default=str, separators=(",", ":"))
|
||||
if len(out2_trim) <= budget:
|
||||
return out2_trim
|
||||
skeleton = _inspect_skeleton(result)
|
||||
return json.dumps(skeleton, default=str, separators=(",", ":"))
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# wait_for_workstream — last-message extraction
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
@@ -319,6 +319,12 @@ class TaskScheduler:
|
||||
user_id=task.get("created_by", ""),
|
||||
skill=task.get("skill", ""),
|
||||
notify_targets=task.get("notify_targets", "[]"),
|
||||
# Mark the resulting ChatSession as non-interactive-for-
|
||||
# consent so OAuth-MCP errors get persisted to
|
||||
# ``mcp_pending_consent`` for later dashboard surfacing,
|
||||
# rather than relying on an in-flight SSE redirect the
|
||||
# absent user can't complete.
|
||||
client_type="scheduled",
|
||||
)
|
||||
ws_id = resp.ws_id
|
||||
except Exception:
|
||||
|
||||
+165
-4
@@ -1652,6 +1652,27 @@ async def mcp_oauth_revoke_connection(request: Request) -> Response:
|
||||
return await handle_mcp_oauth_revoke_connection(request)
|
||||
|
||||
|
||||
async def mcp_oauth_list_pending(request: Request) -> Response:
|
||||
"""GET /v1/api/mcp/oauth/pending — list deferred-consent records (Phase 9)."""
|
||||
from turnstone.core.mcp_oauth import handle_mcp_oauth_list_pending
|
||||
|
||||
return await handle_mcp_oauth_list_pending(request)
|
||||
|
||||
|
||||
async def mcp_oauth_clear_pending(request: Request) -> Response:
|
||||
"""DELETE /v1/api/mcp/oauth/pending/{server_name} — dismiss a deferred-consent record."""
|
||||
from turnstone.core.mcp_oauth import handle_mcp_oauth_clear_pending
|
||||
|
||||
return await handle_mcp_oauth_clear_pending(request)
|
||||
|
||||
|
||||
async def mcp_oauth_clear_all_pending(request: Request) -> Response:
|
||||
"""DELETE /v1/api/mcp/oauth/pending — bulk-dismiss deferred-consent records."""
|
||||
from turnstone.core.mcp_oauth import handle_mcp_oauth_clear_all_pending
|
||||
|
||||
return await handle_mcp_oauth_clear_all_pending(request)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Route handlers — available models (lightweight, no admin permission)
|
||||
# ---------------------------------------------------------------------------
|
||||
@@ -8818,10 +8839,19 @@ def _mask_mcp_secrets(server: dict[str, Any], reveal: bool = False) -> dict[str,
|
||||
def _mcp_server_to_detail(
|
||||
server: dict[str, Any],
|
||||
node_statuses: dict[str, dict[str, Any]] | None = None,
|
||||
consented_users_count: int | None = None,
|
||||
) -> dict[str, Any]:
|
||||
"""Convert a storage dict to a McpServerDetail-shaped dict."""
|
||||
"""Convert a storage dict to a McpServerDetail-shaped dict.
|
||||
|
||||
*consented_users_count* is the Phase 9 admin pill data — distinct
|
||||
non-expired tokens issued for this ``(server_name)``. Omitted
|
||||
(``None``) when the row's ``auth_type`` is not ``oauth_user``, so
|
||||
static / none rows don't carry an irrelevant ``0``.
|
||||
"""
|
||||
d = dict(server)
|
||||
d["status"] = node_statuses or {}
|
||||
if consented_users_count is not None:
|
||||
d["consented_users_count"] = consented_users_count
|
||||
return d
|
||||
|
||||
|
||||
@@ -8872,8 +8902,31 @@ async def admin_list_mcp_servers(request: Request) -> JSONResponse:
|
||||
reveal = str(request.query_params.get("reveal", "")).lower() in ("true", "1")
|
||||
servers = storage.list_mcp_servers()
|
||||
|
||||
# Collect live status from all nodes
|
||||
node_statuses = await _collect_mcp_status(request)
|
||||
# Phase 9: bulk-aggregate consented-users-count across all oauth_user
|
||||
# rows in a single GROUP BY query (rather than N per-row sync DB
|
||||
# round-trips inside this async handler). Run in parallel with the
|
||||
# cross-node HTTP status fan-out below — neither has a data
|
||||
# dependency on the other, so awaiting them sequentially would
|
||||
# stack the DB latency on top of the fan-out latency. Skipped
|
||||
# entirely when no row is oauth_user so static-only installs
|
||||
# exercise zero new storage queries.
|
||||
has_oauth_user = any(s.get("auth_type") == "oauth_user" for s in servers)
|
||||
status_task: asyncio.Task[dict[str, dict[str, dict[str, Any]]]] = asyncio.create_task(
|
||||
_collect_mcp_status(request)
|
||||
)
|
||||
count_task: asyncio.Task[dict[str, int]] | None = (
|
||||
asyncio.create_task(asyncio.to_thread(storage.count_mcp_consented_users_grouped_by_server))
|
||||
if has_oauth_user
|
||||
else None
|
||||
)
|
||||
|
||||
node_statuses = await status_task
|
||||
consent_counts: dict[str, int] = {}
|
||||
if count_task is not None:
|
||||
try:
|
||||
consent_counts = await count_task
|
||||
except Exception:
|
||||
log.debug("admin.mcp_consented_users_bulk_count_failed", exc_info=True)
|
||||
|
||||
db_names: set[str] = set()
|
||||
result = []
|
||||
@@ -8885,8 +8938,14 @@ async def admin_list_mcp_servers(request: Request) -> JSONResponse:
|
||||
status = node_servers.get(s["name"])
|
||||
if status:
|
||||
per_node[node_id] = status
|
||||
# Phase 9: surface the consented-users-count pill for
|
||||
# oauth_user rows. Aggregate was pre-computed above with a
|
||||
# single bulk GROUP BY query; we just look up here.
|
||||
consent_count: int | None = None
|
||||
if s.get("auth_type") == "oauth_user":
|
||||
consent_count = consent_counts.get(s["name"], 0)
|
||||
s = _mask_mcp_secrets(s, reveal)
|
||||
result.append(_mcp_server_to_detail(s, per_node))
|
||||
result.append(_mcp_server_to_detail(s, per_node, consent_count))
|
||||
|
||||
# Merge config-sourced servers visible on nodes but not in DB
|
||||
config_names: set[str] = set()
|
||||
@@ -9527,6 +9586,92 @@ async def admin_mcp_reconnect_one(request: Request) -> JSONResponse:
|
||||
return await _admin_mcp_action(request, "reconnect")
|
||||
|
||||
|
||||
async def admin_mcp_bulk_revoke(request: Request) -> JSONResponse:
|
||||
"""POST /v1/api/admin/mcp-servers/{name}/bulk-revoke — clear every user's token (Phase 9).
|
||||
|
||||
Admin-side counterpart to the per-user
|
||||
``DELETE /v1/api/mcp/oauth/connections/{server_name}`` revoke that
|
||||
shipped in Phase 8. Used to drop orphaned tokens after an
|
||||
``auth_type`` transition (oauth_user → static) or after rotating
|
||||
the configured OAuth client.
|
||||
|
||||
Authoritative local delete via
|
||||
:meth:`StorageBackend.delete_mcp_oauth_rows_by_server_name` —
|
||||
purges both ``mcp_user_tokens`` and ``mcp_oauth_pending`` rows for
|
||||
the named server. Upstream RFC 7009 revoke is intentionally NOT
|
||||
attempted in bulk (would require per-row decrypt + N upstream HTTP
|
||||
calls); operators who need upstream cleanup should use the per-
|
||||
user revoke endpoint or let tokens expire naturally. The audit
|
||||
detail records ``upstream_revoke_outcome="bulk_admin_no_upstream"``
|
||||
so the deferral is visible.
|
||||
|
||||
Pool eviction is NOT performed by this handler. Per-user revoke
|
||||
has a per-(user, server) eviction primitive
|
||||
(``MCPClientManager.evict_user_session``); bulk-revoke would need a
|
||||
per-server iteration over consented users that no current primitive
|
||||
supports. Stale in-flight sessions surface as a per-user 401 on
|
||||
the next dispatch, which refreshes through the now-empty token row
|
||||
and emits ``mcp_consent_required`` — the documented v1 fallback.
|
||||
See :func:`turnstone.core.mcp_client._dispatch_pool` retry path.
|
||||
"""
|
||||
from turnstone.core.audit import record_audit
|
||||
from turnstone.core.auth import require_permission
|
||||
from turnstone.core.web_helpers import require_storage_or_503
|
||||
|
||||
storage, err = require_storage_or_503(request)
|
||||
if err:
|
||||
return err
|
||||
err = require_permission(request, "admin.mcp")
|
||||
if err:
|
||||
return err
|
||||
|
||||
name = request.path_params.get("name", "").strip()
|
||||
if not name or "__" in name:
|
||||
return JSONResponse({"error": "invalid server name"}, status_code=400)
|
||||
|
||||
existing = storage.get_mcp_server_by_name(name)
|
||||
if existing is None:
|
||||
return JSONResponse({"error": "No such server"}, status_code=404)
|
||||
if existing.get("auth_type") != "oauth_user":
|
||||
return JSONResponse(
|
||||
{"error": "bulk-revoke is only valid for auth_type=oauth_user servers"},
|
||||
status_code=400,
|
||||
)
|
||||
|
||||
target_id = existing.get("server_id", name)
|
||||
consented_before = 0
|
||||
try:
|
||||
consented_before = storage.count_mcp_consented_users_by_server(name)
|
||||
except Exception:
|
||||
log.debug("admin.mcp_bulk_revoke_pre_count_failed server=%s", name, exc_info=True)
|
||||
|
||||
deleted = storage.delete_mcp_oauth_rows_by_server_name(name)
|
||||
|
||||
audit_uid, ip = _audit_context(request)
|
||||
record_audit(
|
||||
storage,
|
||||
audit_uid,
|
||||
"mcp_server.oauth.bulk_revoked",
|
||||
"mcp_server",
|
||||
target_id,
|
||||
{
|
||||
"name": name,
|
||||
"rows_deleted": deleted,
|
||||
"consented_users_before": consented_before,
|
||||
"upstream_revoke_outcome": "bulk_admin_no_upstream",
|
||||
},
|
||||
ip,
|
||||
)
|
||||
|
||||
return JSONResponse(
|
||||
{
|
||||
"status": "ok",
|
||||
"rows_deleted": deleted,
|
||||
"consented_users_before": consented_before,
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
async def admin_import_mcp_config(request: Request) -> JSONResponse:
|
||||
"""POST /v1/api/admin/mcp-servers/import — import from pasted JSON config."""
|
||||
import uuid
|
||||
@@ -12247,6 +12392,17 @@ def create_app(
|
||||
mcp_oauth_revoke_connection,
|
||||
methods=["DELETE"],
|
||||
),
|
||||
Route("/api/mcp/oauth/pending", mcp_oauth_list_pending),
|
||||
Route(
|
||||
"/api/mcp/oauth/pending",
|
||||
mcp_oauth_clear_all_pending,
|
||||
methods=["DELETE"],
|
||||
),
|
||||
Route(
|
||||
"/api/mcp/oauth/pending/{server_name}",
|
||||
mcp_oauth_clear_pending,
|
||||
methods=["DELETE"],
|
||||
),
|
||||
Route("/api/admin/users", admin_list_users),
|
||||
Route("/api/admin/users", admin_create_user, methods=["POST"]),
|
||||
Route("/api/admin/users/{user_id}", admin_delete_user, methods=["DELETE"]),
|
||||
@@ -12432,6 +12588,11 @@ def create_app(
|
||||
admin_mcp_reconnect_one,
|
||||
methods=["POST"],
|
||||
),
|
||||
Route(
|
||||
"/api/admin/mcp-servers/{name}/bulk-revoke",
|
||||
admin_mcp_bulk_revoke,
|
||||
methods=["POST"],
|
||||
),
|
||||
Route(
|
||||
"/api/admin/mcp-servers/{server_id}",
|
||||
admin_get_mcp_server,
|
||||
|
||||
@@ -3314,6 +3314,11 @@ function _renderMcpServers(items) {
|
||||
var totalTools = 0,
|
||||
totalRes = 0,
|
||||
totalPrompts = 0;
|
||||
// Phase 9: aggregate the most-recent refresh entry across nodes
|
||||
// so the admin pill reflects "the freshest known state" rather
|
||||
// than picking an arbitrary node.
|
||||
var newestRefreshAt = null;
|
||||
var newestRefreshOutcome = null;
|
||||
for (var j = 0; j < nodeIds.length; j++) {
|
||||
var ns = statusEntries[nodeIds[j]];
|
||||
if (ns.connected) {
|
||||
@@ -3326,6 +3331,13 @@ function _renderMcpServers(items) {
|
||||
anyError = true;
|
||||
if (!firstError) firstError = ns.error;
|
||||
}
|
||||
if (
|
||||
typeof ns.last_refresh_at === "number" &&
|
||||
(newestRefreshAt === null || ns.last_refresh_at > newestRefreshAt)
|
||||
) {
|
||||
newestRefreshAt = ns.last_refresh_at;
|
||||
newestRefreshOutcome = ns.last_refresh_outcome || null;
|
||||
}
|
||||
}
|
||||
|
||||
var dotClass = "mcp-status-dot disabled";
|
||||
@@ -3351,6 +3363,42 @@ function _renderMcpServers(items) {
|
||||
statusText = "idle";
|
||||
}
|
||||
|
||||
// Phase 9: refresh pill shows the short-relative age (e.g. "12m") with
|
||||
// outcome-tinted color (ok vs err) and the full ISO timestamp + outcome
|
||||
// in the tooltip. Pill is omitted (and the cell stays unchanged from
|
||||
// its pre-Phase-9 shape) when no node has yet recorded a refresh
|
||||
// outcome for this server.
|
||||
var refreshPill = "";
|
||||
if (newestRefreshAt !== null) {
|
||||
var ageSeconds = Math.max(
|
||||
0,
|
||||
Math.floor(Date.now() / 1000 - newestRefreshAt),
|
||||
);
|
||||
var ageShort;
|
||||
if (ageSeconds < 60) ageShort = ageSeconds + "s";
|
||||
else if (ageSeconds < 3600) ageShort = Math.floor(ageSeconds / 60) + "m";
|
||||
else if (ageSeconds < 86400)
|
||||
ageShort = Math.floor(ageSeconds / 3600) + "h";
|
||||
else ageShort = Math.floor(ageSeconds / 86400) + "d";
|
||||
var outcomeText = newestRefreshOutcome || "unknown";
|
||||
var pillCls =
|
||||
outcomeText === "ok" ? "mcp-refresh-pill-ok" : "mcp-refresh-pill-err";
|
||||
var pillTitle =
|
||||
"Last refresh " +
|
||||
new Date(newestRefreshAt * 1000).toISOString() +
|
||||
" (" +
|
||||
outcomeText +
|
||||
")";
|
||||
refreshPill =
|
||||
' <span class="mcp-refresh-pill ' +
|
||||
pillCls +
|
||||
'" title="' +
|
||||
escapeHtml(pillTitle) +
|
||||
'">' +
|
||||
escapeHtml(ageShort) +
|
||||
"</span>";
|
||||
}
|
||||
|
||||
var transportCls =
|
||||
s.transport === "stdio" ? "mcp-transport-stdio" : "mcp-transport-http";
|
||||
var toolsVal = anyConnected
|
||||
@@ -3385,6 +3433,24 @@ function _renderMcpServers(items) {
|
||||
'<button class="admin-btn-action" data-mcp-oauth-connect="' +
|
||||
escapeHtml(s.name) +
|
||||
'">connect</button>';
|
||||
// Phase 9: surface the consented-users count + bulk-revoke
|
||||
// affordance only when at least one user has consented.
|
||||
var consentCount =
|
||||
typeof s.consented_users_count === "number"
|
||||
? s.consented_users_count
|
||||
: 0;
|
||||
if (consentCount > 0) {
|
||||
actionBtns +=
|
||||
'<button class="admin-btn-danger" data-mcp-bulk-revoke="' +
|
||||
escapeHtml(s.name) +
|
||||
'" data-mcp-consent-count="' +
|
||||
consentCount +
|
||||
'" title="Drop all ' +
|
||||
consentCount +
|
||||
' user consents for this server">bulk-revoke (' +
|
||||
consentCount +
|
||||
")</button>";
|
||||
}
|
||||
}
|
||||
var actions = isConfig
|
||||
? actionBtns
|
||||
@@ -3429,6 +3495,7 @@ function _renderMcpServers(items) {
|
||||
dotClass +
|
||||
'" aria-hidden="true"></span>' +
|
||||
escapeHtml(statusText) +
|
||||
refreshPill +
|
||||
"</span>" +
|
||||
'<span class="admin-col admin-col-mactions">' +
|
||||
actions +
|
||||
@@ -3507,6 +3574,42 @@ function _renderMcpServers(items) {
|
||||
window.open(url, "_blank", "noopener");
|
||||
});
|
||||
});
|
||||
el.querySelectorAll("[data-mcp-bulk-revoke]").forEach(function (btn) {
|
||||
btn.addEventListener("click", function () {
|
||||
var name = this.getAttribute("data-mcp-bulk-revoke");
|
||||
var count = this.getAttribute("data-mcp-consent-count") || "?";
|
||||
showConfirmModal(
|
||||
"Bulk-revoke MCP consents",
|
||||
"Drop all " +
|
||||
count +
|
||||
' user consents for server "' +
|
||||
name +
|
||||
'"? Users will need to re-consent on next use. Upstream revoke is not attempted in bulk; tokens at the authorization server will expire naturally.',
|
||||
"Bulk-revoke",
|
||||
function () {
|
||||
authFetch(
|
||||
"/v1/api/admin/mcp-servers/" +
|
||||
encodeURIComponent(name) +
|
||||
"/bulk-revoke",
|
||||
{ method: "POST" },
|
||||
)
|
||||
.then(function (r) {
|
||||
if (!r.ok) throw new Error();
|
||||
return r.json();
|
||||
})
|
||||
.then(function (j) {
|
||||
showToast(
|
||||
"Bulk-revoked " + (j.rows_deleted || 0) + " row(s) for " + name,
|
||||
);
|
||||
loadAdminMcp();
|
||||
})
|
||||
.catch(function () {
|
||||
showToast("Failed to bulk-revoke " + name);
|
||||
});
|
||||
},
|
||||
);
|
||||
});
|
||||
});
|
||||
el.querySelectorAll("[data-mcp-delete]").forEach(function (btn) {
|
||||
btn.addEventListener("click", function () {
|
||||
var sid = this.getAttribute("data-mcp-delete");
|
||||
|
||||
@@ -10,7 +10,7 @@
|
||||
<link rel="stylesheet" href="/shared/base.css">
|
||||
<link rel="stylesheet" href="/shared/ui-base.css">
|
||||
<link rel="stylesheet" href="/shared/chat.css">
|
||||
<link rel="stylesheet" href="/shared/katex-0.16.45/katex.min.css">
|
||||
<link rel="stylesheet" href="/shared/katex-0.16.47/katex.min.css">
|
||||
<link rel="stylesheet" href="/static/style.css">
|
||||
<link rel="stylesheet" href="/static/coordinator/coordinator.css">
|
||||
<style>
|
||||
@@ -634,7 +634,7 @@
|
||||
<script src="/shared/composer_attachments.js"></script>
|
||||
<script src="/shared/composer_queue.js"></script>
|
||||
<script src="/shared/status_bar.js"></script>
|
||||
<script src="/shared/katex-0.16.45/katex.min.js"></script>
|
||||
<script src="/shared/katex-0.16.47/katex.min.js"></script>
|
||||
<script src="/shared/hljs-11.11.1/highlight.min.js"></script>
|
||||
<script src="/shared/renderer.js"></script>
|
||||
<script src="/static/coordinator/coordinator.js"></script>
|
||||
|
||||
@@ -3602,6 +3602,28 @@ textarea.skill-content-area {
|
||||
opacity: 0.4;
|
||||
}
|
||||
|
||||
/* Phase 9 — last-refresh pill in the MCP status cell. Compact age +
|
||||
outcome indicator inline with the existing status text. Uses the
|
||||
semantic theme tokens (--bg-highlight, --fg-dim, --warn) defined in
|
||||
shared_static/base.css so the pill follows dark/light theme swaps. */
|
||||
.mcp-refresh-pill {
|
||||
display: inline-block;
|
||||
margin-left: 6px;
|
||||
padding: 0 4px;
|
||||
border-radius: var(--radius-sm);
|
||||
font-size: 0.8em;
|
||||
font-variant-numeric: tabular-nums;
|
||||
opacity: 0.85;
|
||||
}
|
||||
.mcp-refresh-pill-ok {
|
||||
background: var(--bg-highlight);
|
||||
color: var(--fg-dim);
|
||||
}
|
||||
.mcp-refresh-pill-err {
|
||||
background: color-mix(in srgb, var(--warn) 15%, transparent);
|
||||
color: var(--warn);
|
||||
}
|
||||
|
||||
.mcp-detail-modal::before {
|
||||
background: linear-gradient(
|
||||
90deg,
|
||||
|
||||
@@ -54,6 +54,27 @@ def set_config_path(path: str) -> None:
|
||||
_cache = None # invalidate cache so next load_config() re-reads
|
||||
|
||||
|
||||
def _warn_if_world_readable(cfg_path: Path) -> None:
|
||||
"""Warn once if config.toml is group- or world-readable.
|
||||
|
||||
DB passwords, OIDC client secrets, and TLS key paths live in this
|
||||
file — operators usually want it at 0600. POSIX-only; no-ops where
|
||||
``stat()`` modes are meaningless (Windows).
|
||||
"""
|
||||
try:
|
||||
mode = cfg_path.stat().st_mode & 0o777
|
||||
except OSError:
|
||||
return
|
||||
if mode & 0o077:
|
||||
log.warning(
|
||||
"%s is mode %04o (group/world-readable); secrets live here — "
|
||||
"run `chmod 0600 %s` to restrict access",
|
||||
cfg_path,
|
||||
mode,
|
||||
cfg_path,
|
||||
)
|
||||
|
||||
|
||||
def load_config(section: str | None = None) -> dict[str, Any]:
|
||||
"""Load config.toml and return the full dict or a specific section.
|
||||
|
||||
@@ -66,6 +87,7 @@ def load_config(section: str | None = None) -> dict[str, Any]:
|
||||
cfg_path = _resolve_config_path()
|
||||
if cfg_path.is_file():
|
||||
try:
|
||||
_warn_if_world_readable(cfg_path)
|
||||
_cache = tomllib.loads(cfg_path.read_text(encoding="utf-8"))
|
||||
except Exception as exc:
|
||||
log.warning("Failed to parse %s: %s", cfg_path, exc)
|
||||
|
||||
@@ -27,6 +27,7 @@ import urllib.parse
|
||||
import uuid
|
||||
from contextlib import AsyncExitStack
|
||||
from dataclasses import dataclass, field
|
||||
from datetime import UTC, datetime
|
||||
from pathlib import Path
|
||||
from typing import TYPE_CHECKING, Any, Literal
|
||||
|
||||
@@ -45,6 +46,7 @@ from turnstone.core.config import load_config
|
||||
from turnstone.core.log import get_logger
|
||||
from turnstone.core.mcp_http_parsers import (
|
||||
MAX_INSUFFICIENT_SCOPE_REPORTED,
|
||||
is_valid_scope_token,
|
||||
parse_www_authenticate_error,
|
||||
parse_www_authenticate_scope,
|
||||
)
|
||||
@@ -510,6 +512,20 @@ class MCPClientManager:
|
||||
# Notification debounce (per-server)
|
||||
self._last_notification_refresh: dict[str, float] = {}
|
||||
|
||||
# Last refresh outcome (Phase 9 — admin status indicator). Per-
|
||||
# server tuple of ``(unix_ts, outcome)`` where outcome is one of
|
||||
# ``ok`` or ``error:<ExceptionClassName>``. Populated by
|
||||
# ``_refresh_server`` on every call (success and failure paths),
|
||||
# which means manual operator-driven refresh (``refresh_sync``)
|
||||
# AND the ``_cb_auto_reconnect`` follow-up that schedules
|
||||
# ``_refresh_server`` directly both populate the field — adding
|
||||
# a future schedule site only needs to call ``_refresh_server``
|
||||
# to participate. Read by the admin status endpoint to render
|
||||
# the per-server "last refresh" pill. No initial entry is
|
||||
# created at server-register time — absence surfaces as ``null``
|
||||
# in the admin JSON, which the UI renders as "never".
|
||||
self._last_refresh: dict[str, tuple[float, str]] = {}
|
||||
|
||||
# Per-(user, server) state for auth_type=oauth_user. Loop-bound:
|
||||
# mutated only on the mcp-loop. Sync threads interact via
|
||||
# ``asyncio.run_coroutine_threadsafe``.
|
||||
@@ -2131,14 +2147,51 @@ class MCPClientManager:
|
||||
|
||||
Returns ``(added_tools, removed_tools)`` names (tool diff only,
|
||||
for backward compatibility with ``/mcp refresh`` output).
|
||||
|
||||
Writes the ``_last_refresh`` entry on every call so the Phase 9
|
||||
admin status pill reflects every refresh path — manual
|
||||
operator-driven ``refresh_sync`` AND the ``_cb_auto_reconnect``
|
||||
follow-up that schedules ``_refresh_server`` directly.
|
||||
Centralising the write here means future schedule sites
|
||||
automatically populate the field.
|
||||
|
||||
Uses ``asyncio.gather(return_exceptions=True)`` so that a failure
|
||||
in one of the three concurrent sub-refreshes does NOT orphan the
|
||||
others mid-mutation: every sibling reaches completion (success
|
||||
or per-task failure) before the outcome is computed. The
|
||||
``_last_refresh`` write is ``"ok"`` iff all three succeeded; on
|
||||
any failure the outcome is ``f"error:{type(first_exc).__name__}"``
|
||||
and the first exception is re-raised so the outer caller's error
|
||||
path (``_refresh_all``'s except, or the manual-refresh sync
|
||||
wrapper) sees the same shape it did before this rework.
|
||||
Partial-success mutations of ``state.tools`` / ``state.resources``
|
||||
/ ``state.prompts`` are bounded to whichever sub-refresh
|
||||
succeeded — the documented trade-off vs leaving orphan tasks
|
||||
running after the error is observed.
|
||||
"""
|
||||
tool_diff, _, _ = await asyncio.gather(
|
||||
results = await asyncio.gather(
|
||||
self._refresh_server_tools(name),
|
||||
self._refresh_server_resources(name),
|
||||
self._refresh_server_prompts(name),
|
||||
return_exceptions=True,
|
||||
)
|
||||
first_exc: BaseException | None = next(
|
||||
(r for r in results if isinstance(r, BaseException)), None
|
||||
)
|
||||
if first_exc is not None:
|
||||
self._last_refresh[name] = (
|
||||
time.time(),
|
||||
f"error:{type(first_exc).__name__}",
|
||||
)
|
||||
raise first_exc
|
||||
tool_diff = results[0]
|
||||
# ``return_exceptions=True`` widens the static type; on the all-
|
||||
# success path each entry is the awaited result. We narrow the
|
||||
# tool-diff entry to the documented ``(added, removed)`` shape.
|
||||
assert isinstance(tool_diff, tuple)
|
||||
added, removed = tool_diff
|
||||
self._last_error.pop(name, None)
|
||||
self._last_refresh[name] = (time.time(), "ok")
|
||||
return added, removed
|
||||
|
||||
async def _refresh_all(
|
||||
@@ -2167,6 +2220,7 @@ class MCPClientManager:
|
||||
[t["function"]["name"] for t in post.tools] if post is not None else []
|
||||
)
|
||||
results[name] = (new_names, [])
|
||||
self._last_refresh[name] = (time.time(), "ok")
|
||||
continue
|
||||
added, removed = await self._refresh_server(name)
|
||||
self._cb_record_success(name)
|
||||
@@ -2175,6 +2229,20 @@ class MCPClientManager:
|
||||
log.warning("Refresh failed for MCP server '%s'", name, exc_info=True)
|
||||
self._set_error(name, f"Refresh failed: {exc}")
|
||||
results[name] = ([], [])
|
||||
# Overwrite unconditionally with the freshest observed
|
||||
# outcome. Two cases produce the write:
|
||||
# (1) Reconnect branch: ``_connect_one`` raised before
|
||||
# ``_refresh_server`` could write — no prior entry
|
||||
# from this iteration exists yet, so the write is
|
||||
# the only fresh signal.
|
||||
# (2) ``_refresh_server`` branch: it already wrote a
|
||||
# fresh ``error:<ClassName>`` before re-raising, so
|
||||
# the outer overwrite is a no-op for the value.
|
||||
# Using ``setdefault`` here would preserve a stale prior
|
||||
# ``"ok"`` from the previous successful refresh when the
|
||||
# current attempt fails — the admin pill would show
|
||||
# "ok" for a broken server.
|
||||
self._last_refresh[name] = (time.time(), f"error:{type(exc).__name__}")
|
||||
|
||||
# Final sync to clean up templates from servers that are no longer connected
|
||||
try:
|
||||
@@ -2952,6 +3020,7 @@ class MCPClientManager:
|
||||
transport = cfg.get("type", "stdio")
|
||||
cb_deadline = self._circuit_open_until.get(name)
|
||||
cb_open = cb_deadline is not None and time.monotonic() < cb_deadline
|
||||
last_refresh = self._last_refresh.get(name)
|
||||
# Inline predicate (instead of reusing ``connected``) so mypy narrows
|
||||
# ``state`` for the attribute reads — a separate boolean wouldn't.
|
||||
return {
|
||||
@@ -2967,6 +3036,10 @@ class MCPClientManager:
|
||||
"url": cfg.get("url", "") if transport != "stdio" else "",
|
||||
"circuit_open": cb_open,
|
||||
"consecutive_failures": self._consecutive_failures.get(name, 0),
|
||||
# Phase 9 admin status: last manual / auto-reconnect refresh.
|
||||
# ``null`` when no refresh has occurred since process start.
|
||||
"last_refresh_at": last_refresh[0] if last_refresh is not None else None,
|
||||
"last_refresh_outcome": last_refresh[1] if last_refresh is not None else None,
|
||||
}
|
||||
|
||||
def get_all_server_status(self) -> dict[str, dict[str, Any]]:
|
||||
@@ -3303,6 +3376,7 @@ class MCPClientManager:
|
||||
*,
|
||||
user_id: str | None = None,
|
||||
timeout: int = 120,
|
||||
is_interactive_for_consent: bool = True,
|
||||
) -> str:
|
||||
"""Execute an MCP tool call synchronously (blocks the calling thread).
|
||||
|
||||
@@ -3348,6 +3422,7 @@ class MCPClientManager:
|
||||
arguments=arguments,
|
||||
server_row=pool_target[2],
|
||||
timeout=timeout,
|
||||
is_interactive_for_consent=is_interactive_for_consent,
|
||||
)
|
||||
|
||||
if mapping is None or server_name is None or original_name is None:
|
||||
@@ -3500,6 +3575,54 @@ class MCPClientManager:
|
||||
return None
|
||||
return server_name, original, row
|
||||
|
||||
def _record_pending_consent_best_effort(
|
||||
self,
|
||||
*,
|
||||
user_id: str,
|
||||
server_name: str,
|
||||
result: str,
|
||||
) -> None:
|
||||
"""Persist a deferred-consent row for non-interactive callers.
|
||||
|
||||
Called from the three sync dispatchers when the dispatch returns
|
||||
a structured-error envelope AND the caller is not interactive
|
||||
(CHAT / SCHEDULED). Filters out structured-error codes that
|
||||
aren't user-consent-shaped (key-unknown, url-insecure,
|
||||
*_forbidden) — those are operator-actionable and outside the
|
||||
scope of the dashboard pending-consent badge.
|
||||
|
||||
Best-effort. A storage exception is logged but never raised:
|
||||
the structured-error envelope must reach the agent unchanged
|
||||
regardless of whether the pending-consent row was persisted, so
|
||||
a transient DB failure doesn't change the agent-observable
|
||||
contract.
|
||||
"""
|
||||
if self._storage is None:
|
||||
return
|
||||
parsed = _parse_pending_consent_envelope(result)
|
||||
if parsed is None:
|
||||
return
|
||||
code, scopes = parsed
|
||||
scopes_str = " ".join(scopes) if scopes else None
|
||||
try:
|
||||
self._storage.upsert_mcp_pending_consent(
|
||||
user_id=user_id,
|
||||
server_name=server_name,
|
||||
error_code=code,
|
||||
scopes_required=scopes_str,
|
||||
last_ws_id=None,
|
||||
last_tool_call_id=None,
|
||||
now_iso=datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%S"),
|
||||
)
|
||||
except Exception:
|
||||
log.warning(
|
||||
"mcp_pool.pending_consent_persist_failed user=%s server=%s code=%s",
|
||||
user_id,
|
||||
server_name,
|
||||
code,
|
||||
exc_info=True,
|
||||
)
|
||||
|
||||
def _dispatch_pool_sync(
|
||||
self,
|
||||
*,
|
||||
@@ -3509,6 +3632,7 @@ class MCPClientManager:
|
||||
arguments: dict[str, Any],
|
||||
server_row: dict[str, Any],
|
||||
timeout: int,
|
||||
is_interactive_for_consent: bool = True,
|
||||
) -> str:
|
||||
"""Synchronous wrapper for pool dispatch.
|
||||
|
||||
@@ -3581,6 +3705,10 @@ class MCPClientManager:
|
||||
server_row=server_row,
|
||||
)
|
||||
if _is_structured_error(result):
|
||||
if not is_interactive_for_consent:
|
||||
self._record_pending_consent_best_effort(
|
||||
user_id=user_id, server_name=server_name, result=result
|
||||
)
|
||||
raise RuntimeError(result)
|
||||
return result
|
||||
|
||||
@@ -3636,6 +3764,7 @@ class MCPClientManager:
|
||||
uri: str,
|
||||
server_row: dict[str, Any],
|
||||
timeout: int,
|
||||
is_interactive_for_consent: bool = True,
|
||||
) -> str:
|
||||
"""Synchronous wrapper for pool resource read.
|
||||
|
||||
@@ -3679,6 +3808,10 @@ class MCPClientManager:
|
||||
server_row=server_row,
|
||||
)
|
||||
if _is_structured_error(result):
|
||||
if not is_interactive_for_consent:
|
||||
self._record_pending_consent_best_effort(
|
||||
user_id=user_id, server_name=server_name, result=result
|
||||
)
|
||||
raise RuntimeError(result)
|
||||
return result
|
||||
|
||||
@@ -3722,6 +3855,7 @@ class MCPClientManager:
|
||||
arguments: dict[str, str] | None,
|
||||
server_row: dict[str, Any],
|
||||
timeout: int,
|
||||
is_interactive_for_consent: bool = True,
|
||||
) -> list[dict[str, Any]]:
|
||||
"""Synchronous wrapper for pool prompt invocation.
|
||||
|
||||
@@ -3765,6 +3899,10 @@ class MCPClientManager:
|
||||
# Structured-error path — surface as RuntimeError so the
|
||||
# agent-loop renders the JSON via its except-Exception
|
||||
# handler.
|
||||
if not is_interactive_for_consent:
|
||||
self._record_pending_consent_best_effort(
|
||||
user_id=user_id, server_name=server_name, result=result
|
||||
)
|
||||
raise RuntimeError(result)
|
||||
return result
|
||||
|
||||
@@ -4604,7 +4742,12 @@ class MCPClientManager:
|
||||
return best
|
||||
|
||||
def read_resource_sync(
|
||||
self, uri: str, *, user_id: str | None = None, timeout: int = 120
|
||||
self,
|
||||
uri: str,
|
||||
*,
|
||||
user_id: str | None = None,
|
||||
timeout: int = 120,
|
||||
is_interactive_for_consent: bool = True,
|
||||
) -> str:
|
||||
"""Read a resource by URI synchronously (blocks the calling thread).
|
||||
|
||||
@@ -4627,6 +4770,7 @@ class MCPClientManager:
|
||||
uri=pool_target[1],
|
||||
server_row=pool_target[2],
|
||||
timeout=timeout,
|
||||
is_interactive_for_consent=is_interactive_for_consent,
|
||||
)
|
||||
|
||||
mapping = self._resource_map.get(uri)
|
||||
@@ -4689,6 +4833,7 @@ class MCPClientManager:
|
||||
*,
|
||||
user_id: str | None = None,
|
||||
timeout: int = 30,
|
||||
is_interactive_for_consent: bool = True,
|
||||
) -> list[dict[str, Any]]:
|
||||
"""Invoke an MCP prompt synchronously and return expanded messages.
|
||||
|
||||
@@ -4727,6 +4872,7 @@ class MCPClientManager:
|
||||
arguments=arguments,
|
||||
server_row=pool_target[2],
|
||||
timeout=timeout,
|
||||
is_interactive_for_consent=is_interactive_for_consent,
|
||||
)
|
||||
|
||||
if static_mapping is None:
|
||||
@@ -4912,6 +5058,64 @@ def _structured_error(
|
||||
return json.dumps({"error": err})
|
||||
|
||||
|
||||
# Structured-error codes that represent a deferred-consent need. When
|
||||
# encountered on a non-interactive call (chat / scheduled), the sync
|
||||
# dispatcher persists a row to ``mcp_pending_consent`` so the dashboard
|
||||
# badge can surface the deferred work later. Operator-actionable codes
|
||||
# (key-unknown, url-insecure, *_forbidden) are intentionally excluded —
|
||||
# the user cannot resolve them by completing a consent flow.
|
||||
_PENDING_CONSENT_PERSIST_CODES: frozenset[str] = frozenset(
|
||||
{"mcp_consent_required", "mcp_insufficient_scope"}
|
||||
)
|
||||
|
||||
|
||||
def _parse_pending_consent_envelope(
|
||||
result: str,
|
||||
) -> tuple[str, list[str] | None] | None:
|
||||
"""Extract ``(error_code, scopes_required)`` from a structured-error JSON.
|
||||
|
||||
Returns ``None`` when the envelope's ``code`` is not in
|
||||
:data:`_PENDING_CONSENT_PERSIST_CODES`. Callers should already have
|
||||
gated on :func:`_is_structured_error`; this helper deliberately
|
||||
re-parses (cheap on the failure path) rather than threading the
|
||||
decoded dict through the sync-dispatcher hot path.
|
||||
|
||||
Defends against non-dict JSON values (``null``, strings, numbers)
|
||||
via the same ``isinstance(decoded, dict)`` guard
|
||||
:func:`_is_structured_error` uses, so a misuse from a future caller
|
||||
that bypasses the structured-error contract surfaces as a clean
|
||||
``None`` rather than an ``AttributeError`` propagating out of the
|
||||
sync dispatcher's hot path.
|
||||
"""
|
||||
try:
|
||||
decoded = json.loads(result)
|
||||
except (json.JSONDecodeError, ValueError):
|
||||
return None
|
||||
if not isinstance(decoded, dict):
|
||||
return None
|
||||
err = decoded.get("error")
|
||||
if not isinstance(err, dict):
|
||||
return None
|
||||
code = err.get("code", "")
|
||||
if code not in _PENDING_CONSENT_PERSIST_CODES:
|
||||
return None
|
||||
scopes = err.get("scopes_required")
|
||||
if isinstance(scopes, list):
|
||||
# Defense-in-depth scope filter — production paths construct
|
||||
# this list via ``parse_www_authenticate_scope`` which already
|
||||
# validates and caps, but the helper is reusable; re-applying
|
||||
# the predicate here forecloses any future caller that bypasses
|
||||
# the upstream filter from landing attacker-controlled bytes in
|
||||
# ``mcp_pending_consent.scopes_required``. Type-filter BEFORE
|
||||
# ``is_valid_scope_token`` so non-string entries (``None``,
|
||||
# ints) don't slip through as their ``str()`` repr (e.g.
|
||||
# ``None`` → ``"None"`` passes the ASCII grammar). Cap mirrors
|
||||
# ``MAX_INSUFFICIENT_SCOPE_REPORTED`` semantics.
|
||||
cleaned = [s for s in scopes if isinstance(s, str) and is_valid_scope_token(s)]
|
||||
return code, cleaned[:MAX_INSUFFICIENT_SCOPE_REPORTED]
|
||||
return code, None
|
||||
|
||||
|
||||
def _is_structured_error(result: str) -> bool:
|
||||
"""Return True if *result* parses as a :func:`_structured_error` envelope.
|
||||
|
||||
|
||||
@@ -2350,6 +2350,22 @@ async def _handle_mcp_oauth_callback_inner(request: Request) -> Response:
|
||||
},
|
||||
)
|
||||
|
||||
# Phase 9 — clear any deferred-consent records for this (user,
|
||||
# server) now that consent has completed. Best-effort: a storage
|
||||
# failure here doesn't change the user-observable callback success;
|
||||
# the worst case is a stale badge that the user can dismiss
|
||||
# manually. ``delete_mcp_pending_consent`` returns False on
|
||||
# no-such-row (the common case for interactive consent flows that
|
||||
# never deferred), which is fine.
|
||||
try:
|
||||
await asyncio.to_thread(storage.delete_mcp_pending_consent, user_id, server_name)
|
||||
except Exception:
|
||||
log.debug(
|
||||
"mcp_server.oauth.pending_consent_clear_failed",
|
||||
server_name=server_name,
|
||||
exc_info=True,
|
||||
)
|
||||
|
||||
return RedirectResponse(pending["return_url"] or "/", status_code=302)
|
||||
|
||||
|
||||
@@ -2618,6 +2634,153 @@ async def _handle_mcp_oauth_revoke_connection_inner(request: Request) -> Respons
|
||||
return Response(status_code=204)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Pending-consent endpoints (Phase 9)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
async def handle_mcp_oauth_list_pending(request: Request) -> Response:
|
||||
"""``GET /v1/api/mcp/oauth/pending``.
|
||||
|
||||
Returns the authenticated user's deferred-consent records — populated
|
||||
by the pool dispatchers when a non-interactive run (scheduled /
|
||||
channel) hits ``mcp_consent_required`` or ``mcp_insufficient_scope``.
|
||||
Used by the dashboard badge to surface deferred consent needs on
|
||||
next login.
|
||||
|
||||
Install-level gate: when no ``mcp_servers`` row has
|
||||
``auth_type='oauth_user'``, the entire feature is dark — we
|
||||
short-circuit to ``{pending: 0, servers: []}`` without querying the
|
||||
pending table at all. This keeps local-auth installs on a
|
||||
zero-new-storage-query path.
|
||||
"""
|
||||
return _apply_security_headers(await _handle_mcp_oauth_list_pending_inner(request))
|
||||
|
||||
|
||||
_INSTALL_GATE_CACHE_TTL_S = 60.0
|
||||
|
||||
|
||||
async def _install_gate_passes(app_state: Any, storage: Any) -> bool:
|
||||
"""Cached install-level gate for OAuth-MCP features.
|
||||
|
||||
Returns True iff at least one ``mcp_servers`` row has
|
||||
``auth_type='oauth_user'``. Result is cached on ``app_state`` for
|
||||
:data:`_INSTALL_GATE_CACHE_TTL_S` seconds — admin-rare transitions
|
||||
don't justify a per-request DB round-trip on every dashboard load.
|
||||
|
||||
Reset semantics: cache is invalidated by time only. Operators who
|
||||
just enabled an ``oauth_user`` row see the gate flip within the TTL
|
||||
window. False positives (cache says True but the row was just
|
||||
deleted) are bounded by the same window — the downstream list
|
||||
query already filters by user, so the cost is at most one cheap
|
||||
user-scoped read.
|
||||
"""
|
||||
now = time.monotonic()
|
||||
cached = getattr(app_state, "_mcp_install_gate_cache", None)
|
||||
if cached is not None:
|
||||
cached_value, cached_at = cached
|
||||
if (now - cached_at) < _INSTALL_GATE_CACHE_TTL_S:
|
||||
return bool(cached_value)
|
||||
value = bool(await asyncio.to_thread(storage.any_oauth_user_mcp_servers))
|
||||
app_state._mcp_install_gate_cache = (value, now)
|
||||
return value
|
||||
|
||||
|
||||
async def _handle_mcp_oauth_list_pending_inner(request: Request) -> Response:
|
||||
from starlette.responses import JSONResponse
|
||||
|
||||
user_id = _require_user_id(request)
|
||||
if user_id is None:
|
||||
return JSONResponse({"error": "Authentication required"}, status_code=401)
|
||||
|
||||
storage = _get_storage(request.app.state)
|
||||
if storage is None:
|
||||
return JSONResponse({"pending": 0, "servers": []})
|
||||
|
||||
if not await _install_gate_passes(request.app.state, storage):
|
||||
return JSONResponse({"pending": 0, "servers": []})
|
||||
|
||||
rows = await asyncio.to_thread(storage.list_mcp_pending_consent_by_user, user_id)
|
||||
return JSONResponse({"pending": len(rows), "servers": list(rows)})
|
||||
|
||||
|
||||
async def handle_mcp_oauth_clear_pending(request: Request) -> Response:
|
||||
"""``DELETE /v1/api/mcp/oauth/pending/{server_name}``.
|
||||
|
||||
Manual user-initiated dismissal of a single deferred-consent record.
|
||||
Called from the dashboard settings modal when the user opts to clear
|
||||
the entry without completing consent (e.g., the underlying
|
||||
auth_type was changed and the deferred record is now stale).
|
||||
|
||||
Returns 204 in both the existed-and-deleted and never-existed cases
|
||||
to keep cross-tenant existence non-observable.
|
||||
"""
|
||||
return _apply_security_headers(await _handle_mcp_oauth_clear_pending_inner(request))
|
||||
|
||||
|
||||
async def _handle_mcp_oauth_clear_pending_inner(request: Request) -> Response:
|
||||
from starlette.responses import JSONResponse, Response
|
||||
|
||||
user_id = _require_user_id(request)
|
||||
if user_id is None:
|
||||
return JSONResponse({"error": "Authentication required"}, status_code=401)
|
||||
|
||||
server_name = request.path_params.get("server_name", "").strip()
|
||||
if not server_name:
|
||||
return JSONResponse({"error": "Missing server_name"}, status_code=400)
|
||||
|
||||
storage = _get_storage(request.app.state)
|
||||
if storage is None:
|
||||
return JSONResponse({"error": "Storage unavailable"}, status_code=503)
|
||||
|
||||
cleared = bool(
|
||||
await asyncio.to_thread(storage.delete_mcp_pending_consent, user_id, server_name)
|
||||
)
|
||||
# Audit even on no-op deletes (returns 204 either way for cross-tenant
|
||||
# non-observability) so an attacker who tries to scrub deferred-consent
|
||||
# breadcrumbs leaves an audit trail of the attempts.
|
||||
await _audit_event(
|
||||
request.app.state,
|
||||
user_id=user_id,
|
||||
action="mcp_server.oauth.pending_consent_dismissed",
|
||||
server_name=server_name,
|
||||
detail={"mode": "single", "cleared": 1 if cleared else 0},
|
||||
)
|
||||
return Response(status_code=204)
|
||||
|
||||
|
||||
async def handle_mcp_oauth_clear_all_pending(request: Request) -> Response:
|
||||
"""``DELETE /v1/api/mcp/oauth/pending``.
|
||||
|
||||
Bulk dismiss of every deferred-consent record for the authenticated
|
||||
user. Returns the count cleared so the dashboard can update its
|
||||
badge in one round-trip.
|
||||
"""
|
||||
return _apply_security_headers(await _handle_mcp_oauth_clear_all_pending_inner(request))
|
||||
|
||||
|
||||
async def _handle_mcp_oauth_clear_all_pending_inner(request: Request) -> Response:
|
||||
from starlette.responses import JSONResponse
|
||||
|
||||
user_id = _require_user_id(request)
|
||||
if user_id is None:
|
||||
return JSONResponse({"error": "Authentication required"}, status_code=401)
|
||||
|
||||
storage = _get_storage(request.app.state)
|
||||
if storage is None:
|
||||
return JSONResponse({"error": "Storage unavailable"}, status_code=503)
|
||||
|
||||
cleared = await asyncio.to_thread(storage.delete_all_mcp_pending_consent_by_user, user_id)
|
||||
await _audit_event(
|
||||
request.app.state,
|
||||
user_id=user_id,
|
||||
action="mcp_server.oauth.pending_consent_dismissed",
|
||||
server_name="(bulk)",
|
||||
detail={"mode": "bulk", "cleared": cleared},
|
||||
)
|
||||
return JSONResponse({"cleared": cleared})
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Lifespan integration
|
||||
# ---------------------------------------------------------------------------
|
||||
@@ -2674,7 +2837,10 @@ __all__ = [
|
||||
"get_user_access_token_classified",
|
||||
"handle_mcp_oauth_authorize",
|
||||
"handle_mcp_oauth_callback",
|
||||
"handle_mcp_oauth_clear_all_pending",
|
||||
"handle_mcp_oauth_clear_pending",
|
||||
"handle_mcp_oauth_list_connections",
|
||||
"handle_mcp_oauth_list_pending",
|
||||
"handle_mcp_oauth_revoke_connection",
|
||||
"initialize_mcp_oauth_state",
|
||||
"pop_pending_state",
|
||||
|
||||
@@ -19,11 +19,11 @@ Channels:
|
||||
Drain preserves FIFO order; non-matching entries stay queued. Each
|
||||
entry can carry an optional ``valid_until`` predicate that drain
|
||||
evaluates outside the queue lock; entries whose predicate returns
|
||||
``False`` (or raises) are silently dropped without delivery — used by
|
||||
producers whose payload becomes stale if the underlying state changes
|
||||
between enqueue and drain (e.g. ``idle_children`` re-checks the active
|
||||
child set, dropping the nudge if every child finished while the queue
|
||||
sat). Operations are atomic under an internal :class:`threading.Lock`.
|
||||
``False`` are dropped (logged at ``info`` — normal lifecycle outcome,
|
||||
e.g. ``idle_children`` after every child closed) and entries whose
|
||||
predicate raises are dropped (logged at ``warning`` with ``exc_info``
|
||||
— a misbehaving predicate). Operations are atomic under an internal
|
||||
:class:`threading.Lock`.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
@@ -32,9 +32,13 @@ import threading
|
||||
from collections import deque
|
||||
from typing import TYPE_CHECKING, Any, Literal, NamedTuple
|
||||
|
||||
from turnstone.core.log import get_logger
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from collections.abc import Callable
|
||||
|
||||
log = get_logger(__name__)
|
||||
|
||||
Channel = Literal["user", "tool", "any"]
|
||||
_VALID_CHANNELS: frozenset[str] = frozenset({"user", "tool", "any"})
|
||||
|
||||
@@ -139,7 +143,12 @@ class NudgeQueue:
|
||||
self._items = kept
|
||||
# Predicates evaluate outside the lock — they may do storage
|
||||
# I/O or other work that shouldn't block other producers /
|
||||
# the drain consumer's other queues.
|
||||
# the drain consumer's other queues. Drop-level distinction:
|
||||
# a ``False`` return is a normal lifecycle outcome (the
|
||||
# producer's snapshot is stale — e.g. ``idle_children`` after
|
||||
# every child closed) and logs at ``info``; a raised exception
|
||||
# is a wiring bug (predicate is misbehaving) and stays at
|
||||
# ``warning`` with ``exc_info`` so the traceback surfaces.
|
||||
out: list[tuple[str, str, dict[str, Any] | None]] = []
|
||||
for entry in candidates:
|
||||
if entry.valid_until is None:
|
||||
@@ -148,11 +157,27 @@ class NudgeQueue:
|
||||
try:
|
||||
if entry.valid_until():
|
||||
out.append((entry.nudge_type, entry.text, entry.metadata))
|
||||
continue
|
||||
log.info(
|
||||
"nudge_queue.predicate_dropped",
|
||||
extra={
|
||||
"nudge_type": entry.nudge_type,
|
||||
"channel": entry.channel,
|
||||
"reason": "predicate_false",
|
||||
"text_len": len(entry.text),
|
||||
},
|
||||
)
|
||||
except Exception:
|
||||
# Predicate raising is treated as "no longer valid" —
|
||||
# drop silently rather than letting one bad predicate
|
||||
# poison the whole drain batch.
|
||||
pass
|
||||
log.warning(
|
||||
"nudge_queue.predicate_dropped",
|
||||
extra={
|
||||
"nudge_type": entry.nudge_type,
|
||||
"channel": entry.channel,
|
||||
"reason": "predicate_raised",
|
||||
"text_len": len(entry.text),
|
||||
},
|
||||
exc_info=True,
|
||||
)
|
||||
return out
|
||||
|
||||
def __len__(self) -> int:
|
||||
|
||||
+74
-32
@@ -112,7 +112,12 @@ from turnstone.core.tools import (
|
||||
from turnstone.core.watch import WATCH_REMINDER_OPTIONAL_KEYS
|
||||
from turnstone.core.web import check_ssrf, strip_html
|
||||
from turnstone.core.workstream import WorkstreamKind
|
||||
from turnstone.prompts import ClientType, SessionContext, compose_system_message
|
||||
from turnstone.prompts import (
|
||||
INTERACTIVE_CONSENT_CLIENT_TYPES,
|
||||
ClientType,
|
||||
SessionContext,
|
||||
compose_system_message,
|
||||
)
|
||||
from turnstone.ui.colors import DIM, GRAY, GREEN, RED, RESET, YELLOW, bold, cyan, dim
|
||||
|
||||
log = get_logger(__name__)
|
||||
@@ -851,6 +856,15 @@ class ChatSession:
|
||||
self._mcp_user_id: str | None = user_id or None
|
||||
self._username = username
|
||||
self._client_type = client_type
|
||||
# Whether the user is online to complete an in-flight OAuth
|
||||
# consent redirect. WEB and CLI users are; CHAT (Discord /
|
||||
# Slack) and SCHEDULED (autonomous runs) are not — their
|
||||
# consent-required errors must be persisted to
|
||||
# ``mcp_pending_consent`` by the pool dispatchers for later
|
||||
# surfacing on the dashboard badge, rather than relying on the
|
||||
# in-flight SSE rendering path that Phase 8 ships for
|
||||
# interactive surfaces.
|
||||
self._is_interactive_for_consent: bool = client_type in INTERACTIVE_CONSENT_CLIENT_TYPES
|
||||
self._config_store = config_store
|
||||
# Initialize rule registry for configurable judge rules
|
||||
self._rule_registry = None
|
||||
@@ -1656,13 +1670,17 @@ class ChatSession:
|
||||
The closure carries:
|
||||
- a soft cap on per-session ``"watch_triggered"`` depth via
|
||||
:data:`_WATCH_QUEUE_SOFT_CAP` + drop-oldest-on-saturation.
|
||||
- a ``valid_until`` predicate that re-checks
|
||||
``storage.is_watch_active(watch_id)`` at drain time so a
|
||||
cancelled watch's last splat doesn't ride out a future wake.
|
||||
- producer-side :func:`sanitize_payload` over the whole
|
||||
formatted message so steering-vector / control-char payloads
|
||||
sourced from arbitrary shell output can't tamper with the
|
||||
envelope at interpolation time.
|
||||
|
||||
No ``valid_until`` predicate is wired: ``WatchRunner._poll_watch``
|
||||
commits ``active=False`` for terminal fires right after dispatch
|
||||
returns, and an ``is_watch_active`` predicate would race that
|
||||
write at drain time and drop the fire the model was meant to see.
|
||||
A user-cancelled watch's last splat is informative (the reminder
|
||||
carries ``is_final=True``), not stale-noise to suppress.
|
||||
"""
|
||||
self._watch_runner = runner
|
||||
nudge_queue = self._nudge_queue
|
||||
@@ -1693,18 +1711,6 @@ class ChatSession:
|
||||
_WATCH_QUEUE_SOFT_CAP,
|
||||
)
|
||||
|
||||
def _still_active() -> bool:
|
||||
# Re-checked at drain time outside the queue lock — if
|
||||
# the watch was cancelled between fire and drain, the
|
||||
# entry gets dropped silently rather than splicing a
|
||||
# stale result onto the user's next turn. Single-column
|
||||
# ``is_watch_active`` avoids the full-row marshal of
|
||||
# ``get_watch`` on this hot path.
|
||||
try:
|
||||
return get_storage().is_watch_active(watch_id)
|
||||
except Exception:
|
||||
return False
|
||||
|
||||
def _maybe_sanitize(v: Any) -> Any:
|
||||
return sanitize_payload(v) if isinstance(v, str) else v
|
||||
|
||||
@@ -1717,7 +1723,6 @@ class ChatSession:
|
||||
"watch_triggered",
|
||||
sanitized,
|
||||
"any",
|
||||
valid_until=_still_active,
|
||||
metadata=metadata or None,
|
||||
)
|
||||
|
||||
@@ -7000,9 +7005,20 @@ class ChatSession:
|
||||
msg = f"Error: {result['error']}"
|
||||
self._report_tool_result(call_id, "spawn_workstream", msg, is_error=True)
|
||||
return call_id, msg
|
||||
# Successful spawn — surface ws_id + node_id + name + routing
|
||||
# strategy so the coordinator can follow up with inspect / send
|
||||
# and explain why a given node was chosen. ``status`` was
|
||||
# Defensive: absence of ``error`` is the success signal, but
|
||||
# a malformed upstream response could land here with no
|
||||
# ``ws_id``. Without this check the LLM gets
|
||||
# ``{"child_ws_id": null}`` and chases a null id through
|
||||
# follow-up tools. Mirrors the matching guard in
|
||||
# ``_exec_spawn_batch`` (denied row on empty ws_id).
|
||||
child_ws_id = str(result.get("ws_id") or "")
|
||||
if not child_ws_id:
|
||||
msg = "Error: spawn returned no ws_id"
|
||||
self._report_tool_result(call_id, "spawn_workstream", msg, is_error=True)
|
||||
return call_id, msg
|
||||
# Successful spawn — surface child_ws_id + node_id + name +
|
||||
# routing strategy so the coordinator can follow up with inspect
|
||||
# / send and explain why a given node was chosen. ``status`` was
|
||||
# historically included but it was the routing-proxy's HTTP
|
||||
# code (always 200 on this branch); the absence of an
|
||||
# ``error`` field is the success signal. Dropped here to
|
||||
@@ -7013,14 +7029,19 @@ class ChatSession:
|
||||
# ``inspect_workstream``.
|
||||
summary = json.dumps(
|
||||
{
|
||||
"ws_id": result.get("ws_id"),
|
||||
# Key is ``child_ws_id`` (not ``ws_id``) so the coordinator
|
||||
# LLM doesn't recency-bias toward feeding the spawn-return
|
||||
# straight back into another ``spawn_workstream(ws_id=...)``
|
||||
# call. On large fan-outs this cascaded into self-inflicted
|
||||
# re-spawn loops instead of progressing to ``wait_for_workstream``.
|
||||
"child_ws_id": child_ws_id,
|
||||
"name": result.get("name"),
|
||||
"node_id": result.get("node_id"),
|
||||
"routing_strategy": result.get("routing_strategy"),
|
||||
},
|
||||
separators=(",", ":"),
|
||||
)
|
||||
self._report_tool_result(call_id, "spawn_workstream", f"spawned {result.get('ws_id', '?')}")
|
||||
self._report_tool_result(call_id, "spawn_workstream", f"spawned {child_ws_id}")
|
||||
return call_id, summary
|
||||
|
||||
# Cap per batch call. Matches the ``wait_for_workstream`` ws_ids
|
||||
@@ -7151,7 +7172,9 @@ class ChatSession:
|
||||
denied.append({"idx": idx, "reason": "spawn returned no ws_id"})
|
||||
continue
|
||||
results[str(idx)] = {
|
||||
"ws_id": ws_id,
|
||||
# ``child_ws_id`` (not ``ws_id``) — see the matching
|
||||
# comment in ``_exec_spawn_workstream``.
|
||||
"child_ws_id": ws_id,
|
||||
"name": result.get("name", ""),
|
||||
"node_id": result.get("node_id", ""),
|
||||
# ``status`` deliberately omitted — see the matching
|
||||
@@ -7229,6 +7252,8 @@ class ChatSession:
|
||||
}
|
||||
|
||||
def _exec_inspect_workstream(self, item: dict[str, Any]) -> tuple[str, str]:
|
||||
from turnstone.console.coordinator_client import _format_inspect_tiered
|
||||
|
||||
call_id = item["call_id"]
|
||||
ws_id = item["ws_id"]
|
||||
try:
|
||||
@@ -7241,8 +7266,13 @@ class ChatSession:
|
||||
msg = f"Error: inspect_workstream failed: {e}"
|
||||
self._report_tool_result(call_id, "inspect_workstream", msg, is_error=True)
|
||||
return call_id, msg
|
||||
output = json.dumps(result, default=str, separators=(",", ":"))
|
||||
# Summary for UI: state + message count
|
||||
# Tiered output: full → compact (head/tail-snipped messages) →
|
||||
# skeleton (counts + last-assistant preview). First tier that
|
||||
# fits the budget wins; the LLM sees a ``_tier`` field on every
|
||||
# non-error response. ``_truncate_output`` remains the safety
|
||||
# net for the (rare) skeleton-exceeds-budget case — guarding
|
||||
# against a single-field blowup we didn't anticipate.
|
||||
output = _format_inspect_tiered(result)
|
||||
desc = f"{result.get('state', '?')} ({len(result.get('messages', []))} msgs)"
|
||||
self._report_tool_result(call_id, "inspect_workstream", desc)
|
||||
return call_id, self._truncate_output(output)
|
||||
@@ -8595,6 +8625,7 @@ class ChatSession:
|
||||
args,
|
||||
user_id=self._mcp_user_id,
|
||||
timeout=self.tool_timeout,
|
||||
is_interactive_for_consent=self._is_interactive_for_consent,
|
||||
)
|
||||
except TimeoutError:
|
||||
output = f"MCP tool timed out after {self.tool_timeout}s"
|
||||
@@ -8677,7 +8708,10 @@ class ChatSession:
|
||||
# 401 / 403 / consent-required handling. Otherwise the
|
||||
# static path runs byte-identical (invariant 1).
|
||||
output = self._mcp_client.read_resource_sync(
|
||||
uri, user_id=self._mcp_user_id, timeout=self.tool_timeout
|
||||
uri,
|
||||
user_id=self._mcp_user_id,
|
||||
timeout=self.tool_timeout,
|
||||
is_interactive_for_consent=self._is_interactive_for_consent,
|
||||
)
|
||||
except TimeoutError:
|
||||
output = f"MCP resource read timed out after {self.tool_timeout}s"
|
||||
@@ -8774,6 +8808,7 @@ class ChatSession:
|
||||
arguments or None,
|
||||
user_id=self._mcp_user_id,
|
||||
timeout=self.tool_timeout,
|
||||
is_interactive_for_consent=self._is_interactive_for_consent,
|
||||
)
|
||||
output = "\n\n".join(f"[{m['role']}]: {m['content']}" for m in messages)
|
||||
except TimeoutError:
|
||||
@@ -10454,16 +10489,23 @@ class ChatSession:
|
||||
msg = "Error: storage unavailable"
|
||||
self._report_tool_result(call_id, "watch", msg, is_error=True)
|
||||
return call_id, msg
|
||||
watches = storage.list_watches_for_ws(self._ws_id)
|
||||
target = None
|
||||
for w in watches:
|
||||
if w["name"] == name or w["watch_id"].startswith(name):
|
||||
target = w
|
||||
break
|
||||
target = storage.find_watch_by_name(self._ws_id, name)
|
||||
if target is None:
|
||||
msg = f'Watch "{name}" not found.'
|
||||
self._report_tool_result(call_id, "watch", msg, is_error=True)
|
||||
return call_id, msg
|
||||
# In either branch below the row leaves ``list_due_watches``
|
||||
# view (already-inactive or just-cancelled with empty
|
||||
# next_poll), so the runner's retry-deactivate branch will
|
||||
# never reclaim a pending ``_terminal_dispatched`` entry.
|
||||
# Clear it here to bound the lifetime of any leftover from
|
||||
# a previous dispatch-then-failed-row-write.
|
||||
if self._watch_runner is not None:
|
||||
self._watch_runner.forget_terminal_dispatched(target["watch_id"])
|
||||
if not target["active"]:
|
||||
msg = f'Watch "{target["name"]}" already completed (auto-cancelled).'
|
||||
self._report_tool_result(call_id, "watch", msg)
|
||||
return call_id, msg
|
||||
storage.update_watch(target["watch_id"], active=False, next_poll="")
|
||||
msg = f'Watch "{target["name"]}" cancelled.'
|
||||
self._report_tool_result(call_id, "watch", msg)
|
||||
|
||||
@@ -179,6 +179,24 @@ class SessionUIBase:
|
||||
# ``/dashboard`` payload. Capped so a long-running skill
|
||||
# workstream can't fill the live block with stale rows.
|
||||
self._recent_auto_approvals: list[dict[str, Any]] = []
|
||||
# Maps ``call_id`` → ``(auto_approve_reason, inserted_ts)`` for
|
||||
# verdicts that arrive AFTER ``approve_tools`` already returned.
|
||||
# The LLM judge tier is asynchronous: ``on_intent_verdict`` can
|
||||
# fire seconds later for a tool that ``approve_tools``
|
||||
# short-circuited via one of the auto-approve branches. Without
|
||||
# this lookup the late-arriving LLM verdict lands with
|
||||
# ``user_decision="pending"`` and stays that way forever (no
|
||||
# ``resolve_approval`` cycle on the auto-approve path).
|
||||
#
|
||||
# Lifetime is bounded by ``_AUTO_APPROVE_REASON_TTL`` rather
|
||||
# than by a count-cap or by session lifetime: a fixed cap
|
||||
# would silently break the fix on the (N+1)th in-flight
|
||||
# auto-approve; "evict on consume" alone would leak entries
|
||||
# whenever the LLM judge is disabled (no ``on_intent_verdict``
|
||||
# ever fires to drain them). TTL means entries clear lazily
|
||||
# on the next ``_record_auto_approves`` write whether or not
|
||||
# the LLM judge tier is active. Guarded by ``_ws_lock``.
|
||||
self._auto_approve_reasons: dict[str, tuple[str, float]] = {}
|
||||
# Foreground gate — used by the CLI's WorkstreamTerminalUI to
|
||||
# block output when the workstream is in the background.
|
||||
# Starts set so non-CLI UIs can skip any explicit management.
|
||||
@@ -367,6 +385,7 @@ class SessionUIBase:
|
||||
feedback: str | None = None,
|
||||
*,
|
||||
always: bool = False,
|
||||
timeout: bool = False,
|
||||
) -> None:
|
||||
"""Unblock a pending approval with the caller's decision.
|
||||
|
||||
@@ -383,8 +402,22 @@ class SessionUIBase:
|
||||
can label their resolved-status pill correctly). Keyword-only
|
||||
+ default ``False`` so the four pre-existing callers (cancel,
|
||||
timeout, channel adapters) compile unchanged.
|
||||
|
||||
``timeout`` flips the persisted ``user_decision`` from
|
||||
``"denied"`` to ``"timeout"`` so the audit trail can
|
||||
distinguish an active user denial from a passive
|
||||
approval-timeout expiry — the feedback string carries the
|
||||
same information today but operators querying on the
|
||||
``user_decision`` column alone could not tell them apart.
|
||||
Mutually exclusive with ``approved=True`` (a timeout is a
|
||||
passive denial); the combination raises ``ValueError`` so a
|
||||
future caller can't accidentally ship a row whose audit
|
||||
column says ``"timeout"`` while the SSE event reports
|
||||
``approved=True``.
|
||||
"""
|
||||
decision_str = "approved" if approved else "denied"
|
||||
if timeout and approved:
|
||||
raise ValueError("resolve_approval: timeout=True is incompatible with approved=True")
|
||||
decision_str = "timeout" if timeout else ("approved" if approved else "denied")
|
||||
# Swap-and-clear + set decision under lock to avoid racing
|
||||
# with the daemon judge thread's ``on_intent_verdict`` appends.
|
||||
with self._ws_lock:
|
||||
@@ -544,8 +577,15 @@ class SessionUIBase:
|
||||
# the early return — the fall-through
|
||||
# branch never runs on this path, so without
|
||||
# this the policy bypass is invisible to
|
||||
# /dashboard + audit.
|
||||
# /dashboard + audit. ``_record_auto_approves``
|
||||
# MUST run before ``_persist_auto_approved_*``
|
||||
# so the call_id → reason lookup map is
|
||||
# populated before the heuristic INSERTs go
|
||||
# in: otherwise an LLM judge verdict firing
|
||||
# in the gap lands with ``user_decision=
|
||||
# "pending"`` and stays that way.
|
||||
self._record_auto_approves(items)
|
||||
self._persist_auto_approved_heuristic_verdicts(items)
|
||||
self._enqueue(
|
||||
{
|
||||
"type": "tool_info",
|
||||
@@ -608,7 +648,12 @@ class SessionUIBase:
|
||||
self._ws_current_activity = f"⚙ {label}: {preview}" if label else ""
|
||||
self._ws_activity_state = "tool" if label else ""
|
||||
self._broadcast_activity()
|
||||
# ``_record_auto_approves`` runs FIRST so the call_id → reason
|
||||
# lookup is populated before the heuristic INSERT can race
|
||||
# against a concurrent LLM judge verdict — see the matching
|
||||
# comment on the policy-deny branch above.
|
||||
self._record_auto_approves(items)
|
||||
self._persist_auto_approved_heuristic_verdicts(items)
|
||||
self._enqueue({"type": "tool_info", "items": self._serialize_approval_items(items)})
|
||||
return True, None
|
||||
|
||||
@@ -628,19 +673,34 @@ class SessionUIBase:
|
||||
# commit instead of N (was visible as time-to-render-prompt
|
||||
# latency for fan-out turns); the per-item Prometheus call stays
|
||||
# in the loop because it's a lock+increment, not a DB round-trip.
|
||||
#
|
||||
# ``user_decision`` is stamped per-verdict here so the row lands
|
||||
# with a meaningful value at insert: auto-approved items
|
||||
# (mixed-path case: policy allowed some, others still prompt)
|
||||
# carry their auto_approve_reason directly; items still pending
|
||||
# operator decision carry ``"pending"`` and get updated by
|
||||
# ``resolve_approval`` on close. ``_pending_verdicts`` only
|
||||
# tracks the latter — auto-approved verdicts are already final.
|
||||
heuristic_verdicts: list[dict[str, Any]] = []
|
||||
pending_verdicts: list[dict[str, Any]] = []
|
||||
for item in items:
|
||||
hv = item.get("_heuristic_verdict")
|
||||
if hv:
|
||||
heuristic_verdicts.append(hv)
|
||||
# Subclass-overridden Prometheus surface: WebUI feeds
|
||||
# the per-node /metrics endpoint, ConsoleCoordinatorUI
|
||||
# feeds the console's /metrics endpoint via ConsoleMetrics.
|
||||
self._record_judge_metric(hv)
|
||||
if not hv:
|
||||
continue
|
||||
if item.get("auto_approved"):
|
||||
hv["user_decision"] = item.get("auto_approve_reason", "") or "pending"
|
||||
else:
|
||||
hv["user_decision"] = "pending"
|
||||
pending_verdicts.append(hv)
|
||||
heuristic_verdicts.append(hv)
|
||||
# Subclass-overridden Prometheus surface: WebUI feeds
|
||||
# the per-node /metrics endpoint, ConsoleCoordinatorUI
|
||||
# feeds the console's /metrics endpoint via ConsoleMetrics.
|
||||
self._record_judge_metric(hv)
|
||||
self._persist_intent_verdicts_bulk(heuristic_verdicts, default_tier="heuristic")
|
||||
|
||||
with self._ws_lock:
|
||||
self._pending_verdicts = heuristic_verdicts
|
||||
self._pending_verdicts = pending_verdicts
|
||||
|
||||
# Record any items the policy block already auto-approved
|
||||
# before falling through to the prompt — without this the
|
||||
@@ -676,8 +736,14 @@ class SessionUIBase:
|
||||
if not self._approval_event.wait(timeout=self._APPROVAL_WAIT_TIMEOUT):
|
||||
# Approval timed out (e.g., user disconnected). Deny via
|
||||
# resolve_approval so verdicts and state are updated consistently.
|
||||
# Feedback string derives from ``_APPROVAL_WAIT_TIMEOUT`` so the
|
||||
# text follows the constant if the timeout knob moves.
|
||||
log.warning("Approval timed out for ws_id=%s", self.ws_id)
|
||||
self.resolve_approval(False, "Approval timed out after 1 hour")
|
||||
self.resolve_approval(
|
||||
False,
|
||||
f"Approval timed out after {self._APPROVAL_WAIT_TIMEOUT}s",
|
||||
timeout=True,
|
||||
)
|
||||
self._pending_approval = None
|
||||
approved, feedback = self._approval_result
|
||||
|
||||
@@ -714,8 +780,17 @@ class SessionUIBase:
|
||||
``user_decision`` immediately (if the approval already
|
||||
resolved) or parks the verdict in ``_pending_verdicts`` for
|
||||
``resolve_approval`` to stamp on close.
|
||||
|
||||
When the verdict arrives for a call_id that ``approve_tools``
|
||||
already auto-approved (the LLM judge is async and can fire
|
||||
seconds after the auto-approve path returned), stamp the
|
||||
``auto_approve_reason`` onto the verdict before persist so
|
||||
the row lands with a meaningful ``user_decision`` instead of
|
||||
the default ``"pending"`` (which would never be updated for
|
||||
this code path).
|
||||
"""
|
||||
call_id = verdict.get("call_id", "")
|
||||
auto_reason = ""
|
||||
if call_id:
|
||||
with self._ws_lock:
|
||||
if (
|
||||
@@ -725,6 +800,14 @@ class SessionUIBase:
|
||||
oldest_key = next(iter(self._llm_verdicts))
|
||||
del self._llm_verdicts[oldest_key]
|
||||
self._llm_verdicts[call_id] = verdict
|
||||
# Pop (not get) — once consumed the entry isn't useful
|
||||
# again; TTL pruning at the writer side keeps the
|
||||
# never-consumed case bounded too.
|
||||
entry = self._auto_approve_reasons.pop(call_id, None)
|
||||
if entry is not None:
|
||||
auto_reason = entry[0]
|
||||
if auto_reason:
|
||||
verdict["user_decision"] = auto_reason
|
||||
self._enqueue({"type": "intent_verdict", **verdict})
|
||||
# Kind-specific cross-stream broadcast — ConsoleCoordinatorUI
|
||||
# overrides to push onto the cluster bus so a coord parent's
|
||||
@@ -743,6 +826,17 @@ class SessionUIBase:
|
||||
# WRONG decision. Storage UPDATE happens outside the lock
|
||||
# on the already-resolved path — no contention with other
|
||||
# ws-scoped work.
|
||||
# If ``auto_reason`` was stamped above, the verdict already
|
||||
# carries the final ``user_decision`` for this row. Neither
|
||||
# path below applies: appending to ``_pending_verdicts`` would
|
||||
# cause ``resolve_approval`` (on the manual-approval sibling
|
||||
# in a mixed batch) to overwrite the auto-reason with
|
||||
# ``"approved"``/``"denied"``/``"timeout"``; the
|
||||
# ``_persist_verdict_decisions`` immediate-stamp path would
|
||||
# overwrite it the same way from a prior cycle's decision.
|
||||
# Skip both so the audit trail keeps the auto-approve reason.
|
||||
if auto_reason:
|
||||
return
|
||||
with self._ws_lock:
|
||||
decision = self._last_verdict_decision
|
||||
if not decision:
|
||||
@@ -811,9 +905,29 @@ class SessionUIBase:
|
||||
"tier": v.get("tier", default_tier),
|
||||
"judge_model": v.get("judge_model", ""),
|
||||
"latency_ms": v.get("latency_ms", 0),
|
||||
"user_decision": v.get("user_decision", "pending"),
|
||||
}
|
||||
for v in verdicts
|
||||
]
|
||||
# Plain INSERT (not UPSERT) at the bulk site. The race
|
||||
# where a daemon-judge verdict lands BEFORE this bulk
|
||||
# write IS reachable today: ``_evaluate_intent``
|
||||
# (session.py) spawns the daemon thread before
|
||||
# ``approve_tools`` is called, and the daemon's first
|
||||
# emission (heuristic-only short batch, fast LLM response,
|
||||
# or cancel-event ``_deliver_fallbacks`` from judge.py)
|
||||
# can fire ``_persist_intent_verdict`` before this bulk
|
||||
# INSERT runs. Outcome of that race is unchanged by the
|
||||
# per-row UPSERT switch: the bulk INSERT statement aborts
|
||||
# on PK collision regardless of whether the colliding row
|
||||
# was planted by INSERT or UPSERT, and the wrapping
|
||||
# ``try/except`` swallows it. Race A (daemon fires AFTER
|
||||
# bulk) IS improved by the fix: heuristic→llm_fallback
|
||||
# upgrade-in-place now lands. Future bulk-side hardening
|
||||
# (``ON CONFLICT DO NOTHING``) would preserve the OTHER
|
||||
# rows in the batch when one collides, but would keep the
|
||||
# daemon's ``tier`` ("llm"/"llm_fallback") for the
|
||||
# colliding row instead of the bulk's heuristic stamp.
|
||||
storage.create_intent_verdicts_bulk(rows)
|
||||
except Exception:
|
||||
log.debug("Failed to bulk-persist intent verdicts", exc_info=True)
|
||||
@@ -824,15 +938,21 @@ class SessionUIBase:
|
||||
*,
|
||||
default_tier: str = "llm",
|
||||
) -> None:
|
||||
"""Persist an intent-judge verdict row.
|
||||
"""Persist an intent-judge verdict row via UPSERT.
|
||||
|
||||
Used by both the async LLM-tier path (``on_intent_verdict``,
|
||||
default tier ``"llm"``) and the synchronous heuristic-tier
|
||||
path (``approve_tools``, caller passes ``default_tier="heuristic"``).
|
||||
Used by the async LLM-tier path (``on_intent_verdict``,
|
||||
default tier ``"llm"``). Routes through ``upsert_intent_verdict``
|
||||
because ``tier="llm_fallback"`` verdicts deliberately reuse the
|
||||
heuristic verdict's ``verdict_id`` (see ``judge.py`` —
|
||||
``_deliver_fallbacks`` and the in-loop fallback path)
|
||||
so the row gets "upgraded in place" from heuristic →
|
||||
llm_fallback. A plain INSERT would collide on the PK and the
|
||||
upgrade would be lost to a silently-swallowed exception.
|
||||
``default_tier`` only matters when the verdict dict doesn't
|
||||
already carry a ``tier`` key — both real producers always set it,
|
||||
but the fallback is the right call-site label so a malformed
|
||||
verdict still lands on the correct row classification.
|
||||
already carry a ``tier`` key — both real producers always set
|
||||
it, but the fallback is the right call-site label so a
|
||||
malformed verdict still lands on the correct row
|
||||
classification.
|
||||
"""
|
||||
try:
|
||||
from turnstone.core.storage._registry import get_storage
|
||||
@@ -840,7 +960,7 @@ class SessionUIBase:
|
||||
storage = get_storage()
|
||||
if storage is None:
|
||||
return
|
||||
storage.create_intent_verdict(
|
||||
storage.upsert_intent_verdict(
|
||||
verdict_id=verdict.get("verdict_id", ""),
|
||||
ws_id=self.ws_id,
|
||||
call_id=verdict.get("call_id", ""),
|
||||
@@ -855,6 +975,7 @@ class SessionUIBase:
|
||||
tier=verdict.get("tier", default_tier),
|
||||
judge_model=verdict.get("judge_model", ""),
|
||||
latency_ms=verdict.get("latency_ms", 0),
|
||||
user_decision=verdict.get("user_decision", "pending"),
|
||||
)
|
||||
except Exception:
|
||||
log.debug("Failed to persist intent verdict", exc_info=True)
|
||||
@@ -955,6 +1076,14 @@ class SessionUIBase:
|
||||
# workstreams that auto-approve dozens of tool calls per turn.
|
||||
_RECENT_AUTO_APPROVALS_MAX = 10
|
||||
|
||||
# TTL on the call_id → auto_approve_reason map. Sized to comfortably
|
||||
# cover the LLM judge's worst-case latency (cold start + a slow model
|
||||
# + queue depth). Pruning happens lazily at write time so the cost
|
||||
# is paid only on the next auto-approve event; a session that goes
|
||||
# quiet after auto-approving never pays the prune cost at all but
|
||||
# the resident-set is also tiny.
|
||||
_AUTO_APPROVE_REASON_TTL = 60.0
|
||||
|
||||
@staticmethod
|
||||
def _serialize_approval_items(items: list[dict[str, Any]]) -> list[dict[str, Any]]:
|
||||
"""Project each item to the wire shape the SSE event payload uses.
|
||||
@@ -1026,6 +1155,38 @@ class SessionUIBase:
|
||||
else:
|
||||
it["auto_approve_reason"] = reason
|
||||
|
||||
def _persist_auto_approved_heuristic_verdicts(self, items: list[dict[str, Any]]) -> None:
|
||||
"""Persist heuristic verdicts for items the auto-approve path resolved.
|
||||
|
||||
The manual-approval block at the bottom of ``approve_tools``
|
||||
handles its own verdict persistence (and stamps
|
||||
``user_decision`` per item — auto-approved items in a
|
||||
mixed-path batch carry their reason, pending items carry
|
||||
``"pending"``). The auto-approve early-return branches
|
||||
(policy-allow-with-deny, blanket flag, auto_approve_tools
|
||||
match) used to drop heuristic verdicts on the floor — an
|
||||
operator querying ``user_decision`` for the auto-approve
|
||||
reason would find no row at all, conflating "we auto-approved
|
||||
silently" with "the judge didn't run". This helper closes
|
||||
that gap: walk ``items``, persist each auto-approved verdict
|
||||
with its reason stamped, and fan a metric row per verdict.
|
||||
Safe to call with empty items.
|
||||
"""
|
||||
if not items:
|
||||
return
|
||||
verdicts: list[dict[str, Any]] = []
|
||||
for it in items:
|
||||
if not it.get("auto_approved"):
|
||||
continue
|
||||
hv = it.get("_heuristic_verdict")
|
||||
if not hv:
|
||||
continue
|
||||
hv["user_decision"] = it.get("auto_approve_reason", "") or "pending"
|
||||
verdicts.append(hv)
|
||||
self._record_judge_metric(hv)
|
||||
if verdicts:
|
||||
self._persist_intent_verdicts_bulk(verdicts, default_tier="heuristic")
|
||||
|
||||
def _record_auto_approves(self, items: list[dict[str, Any]]) -> None:
|
||||
"""Append auto-approved items to the per-ws ring buffer + audit log.
|
||||
|
||||
@@ -1062,6 +1223,32 @@ class SessionUIBase:
|
||||
overflow = len(self._recent_auto_approvals) - self._RECENT_AUTO_APPROVALS_MAX
|
||||
if overflow > 0:
|
||||
self._recent_auto_approvals = self._recent_auto_approvals[overflow:]
|
||||
# Mirror call_id → reason into the lookup map so a late
|
||||
# ``on_intent_verdict`` (LLM judge tier) can stamp the
|
||||
# right ``user_decision`` instead of leaving the verdict
|
||||
# stuck as ``"pending"`` forever. Prune expired entries
|
||||
# first (lazy TTL eviction) so a session with the LLM
|
||||
# judge disabled doesn't accumulate entries that will
|
||||
# never be consumed. Skip the rebuild when the map is
|
||||
# empty or all entries are still fresh — the common case
|
||||
# on a healthy LLM-judge-enabled session where entries
|
||||
# drain via ``on_intent_verdict.pop`` within the TTL.
|
||||
cutoff = ts - self._AUTO_APPROVE_REASON_TTL
|
||||
if self._auto_approve_reasons and any(
|
||||
ins_ts < cutoff for _, ins_ts in self._auto_approve_reasons.values()
|
||||
):
|
||||
self._auto_approve_reasons = {
|
||||
cid: (reason, ins_ts)
|
||||
for cid, (reason, ins_ts) in self._auto_approve_reasons.items()
|
||||
if ins_ts >= cutoff
|
||||
}
|
||||
for entry in appended:
|
||||
cid = entry["call_id"]
|
||||
if cid:
|
||||
self._auto_approve_reasons[cid] = (
|
||||
entry["auto_approve_reason"],
|
||||
ts,
|
||||
)
|
||||
# Audit emission — one row per ``approve_tools`` call (not one
|
||||
# per item) keeps the audit table from blowing up on
|
||||
# tool-heavy turns while still capturing every tool name +
|
||||
|
||||
@@ -19,6 +19,7 @@ import sqlalchemy as sa
|
||||
from turnstone.core.log import get_logger
|
||||
from turnstone.core.storage._protocol import (
|
||||
MCPOAuthPendingState,
|
||||
MCPPendingConsentRow,
|
||||
MCPUserToken,
|
||||
MCPUserTokenMetadataRow,
|
||||
OIDCIdentity,
|
||||
@@ -33,6 +34,7 @@ from turnstone.core.storage._schema import (
|
||||
heuristic_rules,
|
||||
intent_verdicts,
|
||||
mcp_oauth_pending,
|
||||
mcp_pending_consent,
|
||||
mcp_servers,
|
||||
mcp_user_tokens,
|
||||
metadata,
|
||||
@@ -70,6 +72,9 @@ from turnstone.core.storage._schema import (
|
||||
from turnstone.core.storage._utils import (
|
||||
HEURISTIC_RULE_MUTABLE as _HEURISTIC_RULE_MUTABLE,
|
||||
)
|
||||
from turnstone.core.storage._utils import (
|
||||
LIKE_ESCAPE as _LIKE_ESCAPE,
|
||||
)
|
||||
from turnstone.core.storage._utils import (
|
||||
MCP_SERVER_MUTABLE as _MCP_SERVER_MUTABLE,
|
||||
)
|
||||
@@ -100,6 +105,9 @@ from turnstone.core.storage._utils import (
|
||||
from turnstone.core.storage._utils import (
|
||||
VERDICT_MUTABLE as _VERDICT_MUTABLE,
|
||||
)
|
||||
from turnstone.core.storage._utils import (
|
||||
escape_like as _escape_like,
|
||||
)
|
||||
from turnstone.core.storage._utils import (
|
||||
normalize_search_terms as _normalize_search_terms,
|
||||
)
|
||||
@@ -118,11 +126,6 @@ from turnstone.core.workstream import BULK_CLOSE_STATE_VALUES, WorkstreamKind
|
||||
log = get_logger(__name__)
|
||||
|
||||
|
||||
def _escape_ilike(s: str) -> str:
|
||||
"""Escape ILIKE metacharacters for use with ESCAPE '\\\\'."""
|
||||
return s.replace("\\", "\\\\").replace("%", "\\%").replace("_", "\\_")
|
||||
|
||||
|
||||
def _resolve_pg_listen_url(override: str, sqlalchemy_url: str) -> str:
|
||||
"""Resolve the URL used by the dedicated LISTEN connection.
|
||||
|
||||
@@ -1833,6 +1836,33 @@ class PostgreSQLBackend:
|
||||
).fetchall()
|
||||
return [dict(r._mapping) for r in rows]
|
||||
|
||||
def find_watch_by_name(self, ws_id: str, name_or_prefix: str) -> dict[str, Any] | None:
|
||||
|
||||
if not name_or_prefix:
|
||||
return None
|
||||
like_pattern = _escape_like(name_or_prefix) + "%"
|
||||
with self._conn() as conn:
|
||||
row = conn.execute(
|
||||
sa.select(watches)
|
||||
.where(
|
||||
(watches.c.ws_id == ws_id)
|
||||
& (
|
||||
(watches.c.name == name_or_prefix)
|
||||
| watches.c.watch_id.like(like_pattern, escape=_LIKE_ESCAPE)
|
||||
)
|
||||
)
|
||||
# Active rows win over inactive ones with the same name.
|
||||
# _prepare_watch's duplicate-name guard filters active=1,
|
||||
# so a model can recreate a name after the previous one
|
||||
# auto-cancelled; a cancel-by-name request on the live
|
||||
# row must not be shadowed by the older completed row.
|
||||
.order_by(watches.c.active.desc(), watches.c.created.desc())
|
||||
.limit(1)
|
||||
).fetchone()
|
||||
if row is None:
|
||||
return None
|
||||
return dict(row._mapping)
|
||||
|
||||
def list_watches_for_node(self, node_id: str) -> list[dict[str, Any]]:
|
||||
|
||||
with self._conn() as conn:
|
||||
@@ -3332,6 +3362,7 @@ class PostgreSQLBackend:
|
||||
tier: str,
|
||||
judge_model: str,
|
||||
latency_ms: int,
|
||||
user_decision: str = "pending",
|
||||
) -> None:
|
||||
now = datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%S")
|
||||
with self._conn() as conn:
|
||||
@@ -3352,11 +3383,67 @@ class PostgreSQLBackend:
|
||||
"tier": tier,
|
||||
"judge_model": judge_model,
|
||||
"latency_ms": latency_ms,
|
||||
"user_decision": user_decision,
|
||||
"created": now,
|
||||
},
|
||||
)
|
||||
conn.commit()
|
||||
|
||||
def upsert_intent_verdict(
|
||||
self,
|
||||
verdict_id: str,
|
||||
ws_id: str,
|
||||
call_id: str,
|
||||
func_name: str,
|
||||
func_args: str,
|
||||
intent_summary: str,
|
||||
risk_level: str,
|
||||
confidence: float,
|
||||
recommendation: str,
|
||||
reasoning: str,
|
||||
evidence: str,
|
||||
tier: str,
|
||||
judge_model: str,
|
||||
latency_ms: int,
|
||||
user_decision: str = "pending",
|
||||
) -> None:
|
||||
from sqlalchemy.dialects.postgresql import insert as pg_insert
|
||||
|
||||
now = datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%S")
|
||||
stmt = pg_insert(intent_verdicts).values(
|
||||
verdict_id=verdict_id,
|
||||
ws_id=ws_id,
|
||||
call_id=call_id,
|
||||
func_name=func_name,
|
||||
func_args=func_args,
|
||||
intent_summary=intent_summary,
|
||||
risk_level=risk_level,
|
||||
confidence=confidence,
|
||||
recommendation=recommendation,
|
||||
reasoning=reasoning,
|
||||
evidence=evidence,
|
||||
tier=tier,
|
||||
judge_model=judge_model,
|
||||
latency_ms=latency_ms,
|
||||
user_decision=user_decision,
|
||||
created=now,
|
||||
)
|
||||
# On verdict_id conflict, update only the three fields that
|
||||
# genuinely change between heuristic and llm_fallback. See the
|
||||
# protocol docstring for the full exclusion rationale —
|
||||
# ``user_decision`` exclusion in particular is load-bearing.
|
||||
stmt = stmt.on_conflict_do_update(
|
||||
index_elements=[intent_verdicts.c.verdict_id],
|
||||
set_={
|
||||
"tier": tier,
|
||||
"reasoning": reasoning,
|
||||
"judge_model": judge_model,
|
||||
},
|
||||
)
|
||||
with self._conn() as conn:
|
||||
conn.execute(stmt)
|
||||
conn.commit()
|
||||
|
||||
def create_intent_verdicts_bulk(self, verdicts: list[dict[str, Any]]) -> None:
|
||||
if not verdicts:
|
||||
return
|
||||
@@ -3377,6 +3464,7 @@ class PostgreSQLBackend:
|
||||
"tier": v.get("tier", "heuristic"),
|
||||
"judge_model": v.get("judge_model", ""),
|
||||
"latency_ms": v.get("latency_ms", 0),
|
||||
"user_decision": v.get("user_decision", "pending"),
|
||||
"created": now,
|
||||
}
|
||||
for v in verdicts
|
||||
@@ -3669,7 +3757,7 @@ class PostgreSQLBackend:
|
||||
clauses = []
|
||||
params: dict[str, str] = {}
|
||||
for i, t in enumerate(terms):
|
||||
escaped = _escape_ilike(t)
|
||||
escaped = _escape_like(t)
|
||||
clauses.append(
|
||||
f"(name ILIKE :n{i} ESCAPE '\\' "
|
||||
f"OR description ILIKE :d{i} ESCAPE '\\' "
|
||||
@@ -3748,7 +3836,7 @@ class PostgreSQLBackend:
|
||||
scope_clauses, params = self._build_scope_or_clause(scopes)
|
||||
term_clauses = []
|
||||
for i, t in enumerate(terms):
|
||||
escaped = _escape_ilike(t)
|
||||
escaped = _escape_like(t)
|
||||
term_clauses.append(
|
||||
f"(name ILIKE :n{i} ESCAPE '\\' "
|
||||
f"OR description ILIKE :d{i} ESCAPE '\\' "
|
||||
@@ -4308,6 +4396,136 @@ class PostgreSQLBackend:
|
||||
conn.commit()
|
||||
return result.rowcount
|
||||
|
||||
# -- MCP pending-consent (Phase 9) ----------------------------------------
|
||||
|
||||
def upsert_mcp_pending_consent(
|
||||
self,
|
||||
user_id: str,
|
||||
server_name: str,
|
||||
error_code: str,
|
||||
scopes_required: str | None,
|
||||
last_ws_id: str | None,
|
||||
last_tool_call_id: str | None,
|
||||
now_iso: str,
|
||||
) -> None:
|
||||
from sqlalchemy.dialects import postgresql
|
||||
|
||||
stmt = postgresql.insert(mcp_pending_consent).values(
|
||||
user_id=user_id,
|
||||
server_name=server_name,
|
||||
error_code=error_code,
|
||||
scopes_required=scopes_required,
|
||||
last_ws_id=last_ws_id,
|
||||
last_tool_call_id=last_tool_call_id,
|
||||
first_seen_at=now_iso,
|
||||
last_seen_at=now_iso,
|
||||
occurrence_count=1,
|
||||
)
|
||||
stmt = stmt.on_conflict_do_update(
|
||||
index_elements=["user_id", "server_name"],
|
||||
set_={
|
||||
"error_code": stmt.excluded.error_code,
|
||||
"scopes_required": stmt.excluded.scopes_required,
|
||||
"last_ws_id": stmt.excluded.last_ws_id,
|
||||
"last_tool_call_id": stmt.excluded.last_tool_call_id,
|
||||
"last_seen_at": stmt.excluded.last_seen_at,
|
||||
"occurrence_count": mcp_pending_consent.c.occurrence_count + 1,
|
||||
},
|
||||
)
|
||||
with self._conn() as conn:
|
||||
conn.execute(stmt)
|
||||
conn.commit()
|
||||
|
||||
def list_mcp_pending_consent_by_user(self, user_id: str) -> list[MCPPendingConsentRow]:
|
||||
with self._conn() as conn:
|
||||
rows = conn.execute(
|
||||
sa.select(mcp_pending_consent)
|
||||
.where(mcp_pending_consent.c.user_id == user_id)
|
||||
.order_by(mcp_pending_consent.c.last_seen_at.desc())
|
||||
).fetchall()
|
||||
out: list[MCPPendingConsentRow] = []
|
||||
for r in rows:
|
||||
m = r._mapping
|
||||
out.append(
|
||||
MCPPendingConsentRow(
|
||||
user_id=m["user_id"],
|
||||
server_name=m["server_name"],
|
||||
error_code=m["error_code"],
|
||||
scopes_required=m["scopes_required"],
|
||||
last_ws_id=m["last_ws_id"],
|
||||
last_tool_call_id=m["last_tool_call_id"],
|
||||
first_seen_at=m["first_seen_at"],
|
||||
last_seen_at=m["last_seen_at"],
|
||||
occurrence_count=m["occurrence_count"],
|
||||
)
|
||||
)
|
||||
return out
|
||||
|
||||
def delete_mcp_pending_consent(self, user_id: str, server_name: str) -> bool:
|
||||
with self._conn() as conn:
|
||||
result = conn.execute(
|
||||
sa.delete(mcp_pending_consent).where(
|
||||
(mcp_pending_consent.c.user_id == user_id)
|
||||
& (mcp_pending_consent.c.server_name == server_name)
|
||||
)
|
||||
)
|
||||
conn.commit()
|
||||
return bool(result.rowcount)
|
||||
|
||||
def delete_all_mcp_pending_consent_by_user(self, user_id: str) -> int:
|
||||
with self._conn() as conn:
|
||||
result = conn.execute(
|
||||
sa.delete(mcp_pending_consent).where(mcp_pending_consent.c.user_id == user_id)
|
||||
)
|
||||
conn.commit()
|
||||
return int(result.rowcount or 0)
|
||||
|
||||
def count_mcp_consented_users_by_server(self, server_name: str) -> int:
|
||||
# ``expires_at IS NULL`` => non-expired (refresh-only tokens with no
|
||||
# advertised expiry). Compare lexically against ISO-8601 strings,
|
||||
# mirroring the convention in ``mcp_user_tokens.expires_at``.
|
||||
now_iso = datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%S")
|
||||
with self._conn() as conn:
|
||||
result = conn.execute(
|
||||
sa.select(sa.func.count(sa.distinct(mcp_user_tokens.c.user_id)))
|
||||
.where(mcp_user_tokens.c.server_name == server_name)
|
||||
.where(
|
||||
sa.or_(
|
||||
mcp_user_tokens.c.expires_at.is_(None),
|
||||
mcp_user_tokens.c.expires_at > now_iso,
|
||||
)
|
||||
)
|
||||
).scalar()
|
||||
return int(result or 0)
|
||||
|
||||
def count_mcp_consented_users_grouped_by_server(self) -> dict[str, int]:
|
||||
now_iso = datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%S")
|
||||
with self._conn() as conn:
|
||||
rows = conn.execute(
|
||||
sa.select(
|
||||
mcp_user_tokens.c.server_name,
|
||||
sa.func.count(sa.distinct(mcp_user_tokens.c.user_id)),
|
||||
)
|
||||
.where(
|
||||
sa.or_(
|
||||
mcp_user_tokens.c.expires_at.is_(None),
|
||||
mcp_user_tokens.c.expires_at > now_iso,
|
||||
)
|
||||
)
|
||||
.group_by(mcp_user_tokens.c.server_name)
|
||||
).fetchall()
|
||||
return {row[0]: int(row[1] or 0) for row in rows}
|
||||
|
||||
def any_oauth_user_mcp_servers(self) -> bool:
|
||||
with self._conn() as conn:
|
||||
result = conn.execute(
|
||||
sa.select(sa.literal(1))
|
||||
.select_from(mcp_servers)
|
||||
.where(mcp_servers.c.auth_type == "oauth_user")
|
||||
.limit(1)
|
||||
).scalar()
|
||||
return result is not None
|
||||
|
||||
# -- Model definitions -----------------------------------------------------
|
||||
|
||||
def create_model_definition(
|
||||
|
||||
@@ -93,6 +93,27 @@ class MCPOAuthPendingState(TypedDict):
|
||||
created_at: str
|
||||
|
||||
|
||||
class MCPPendingConsentRow(TypedDict):
|
||||
"""Row shape for deferred-consent records.
|
||||
|
||||
Emitted by the pool dispatchers (Phase 5+) when a non-interactive
|
||||
run (scheduled / channel) hits ``mcp_consent_required`` or
|
||||
``mcp_insufficient_scope`` and the user can't be prompted in the
|
||||
moment. Composite PK ``(user_id, server_name)`` collapses repeat
|
||||
occurrences for the same server into one row.
|
||||
"""
|
||||
|
||||
user_id: str
|
||||
server_name: str
|
||||
error_code: str
|
||||
scopes_required: str | None
|
||||
last_ws_id: str | None
|
||||
last_tool_call_id: str | None
|
||||
first_seen_at: str
|
||||
last_seen_at: str
|
||||
occurrence_count: int
|
||||
|
||||
|
||||
@runtime_checkable
|
||||
class StorageBackend(Protocol):
|
||||
"""Protocol that every storage backend adapter must implement.
|
||||
@@ -982,6 +1003,23 @@ class StorageBackend(Protocol):
|
||||
"""Return active watches for a workstream, ordered by created DESC."""
|
||||
...
|
||||
|
||||
def find_watch_by_name(self, ws_id: str, name_or_prefix: str) -> dict[str, Any] | None:
|
||||
"""Return a watch in ``ws_id`` whose ``name`` matches
|
||||
``name_or_prefix`` exactly, or whose ``watch_id`` starts with it.
|
||||
|
||||
Unlike :meth:`list_watches_for_ws` this DOES NOT filter on the
|
||||
``active`` flag — callers can inspect ``row["active"]`` to
|
||||
distinguish a still-running watch from one that fired and
|
||||
auto-cancelled. Returns ``None`` if no match.
|
||||
|
||||
When multiple rows match, prefers active rows over inactive
|
||||
ones, then most-recently-created. Without the active
|
||||
preference, a recreated-after-completion name would let the
|
||||
older inactive row shadow the new active one in the cancel
|
||||
path.
|
||||
"""
|
||||
...
|
||||
|
||||
def list_watches_for_node(self, node_id: str) -> list[dict[str, Any]]:
|
||||
"""Return all active watches on a node, ordered by created DESC."""
|
||||
...
|
||||
@@ -1569,8 +1607,77 @@ class StorageBackend(Protocol):
|
||||
tier: str,
|
||||
judge_model: str,
|
||||
latency_ms: int,
|
||||
user_decision: str = "pending",
|
||||
) -> None:
|
||||
"""Record an intent validation verdict."""
|
||||
"""Record an intent validation verdict.
|
||||
|
||||
``user_decision`` defaults to ``"pending"`` rather than ``""``
|
||||
so an audit reader can distinguish "in-flight" rows from
|
||||
legacy pre-fix rows (which carry ``""`` from the column's
|
||||
server_default and indicate "convention not yet established
|
||||
when this row was written"). Resolution writers
|
||||
(:meth:`update_intent_verdict`) later overwrite the field with
|
||||
``"approved"`` / ``"denied"`` / ``"timeout"`` (user-driven) or
|
||||
``"policy"`` / ``"blanket"`` / ``"auto_approve_tools"``
|
||||
(auto-approve reason, mirroring :class:`AutoApproveReason`).
|
||||
"""
|
||||
...
|
||||
|
||||
def upsert_intent_verdict(
|
||||
self,
|
||||
verdict_id: str,
|
||||
ws_id: str,
|
||||
call_id: str,
|
||||
func_name: str,
|
||||
func_args: str,
|
||||
intent_summary: str,
|
||||
risk_level: str,
|
||||
confidence: float,
|
||||
recommendation: str,
|
||||
reasoning: str,
|
||||
evidence: str,
|
||||
tier: str,
|
||||
judge_model: str,
|
||||
latency_ms: int,
|
||||
user_decision: str = "pending",
|
||||
) -> None:
|
||||
"""INSERT a verdict row, or UPDATE the judge-output fields on conflict.
|
||||
|
||||
Async LLM judge verdicts with ``tier="llm_fallback"`` deliberately
|
||||
reuse the heuristic verdict's ``verdict_id`` so the row gets
|
||||
"upgraded in place" from heuristic → fallback when the LLM tier
|
||||
doesn't return a real verdict (timeout / cancelled / no-content).
|
||||
A plain INSERT collides on ``intent_verdicts_pkey``; this method
|
||||
``ON CONFLICT (verdict_id) DO UPDATE`` updates only the columns
|
||||
that genuinely change between the two tiers:
|
||||
|
||||
- ``tier`` (the upgrade itself)
|
||||
- ``reasoning`` (gets " (LLM judge did not return a verdict)" appended)
|
||||
- ``judge_model`` (heuristic carries "", fallback carries the model)
|
||||
|
||||
Every other column is EXCLUDED from the on-conflict SET clause:
|
||||
|
||||
- Identity columns (``verdict_id``, ``ws_id``, ``call_id``,
|
||||
``func_name``, ``func_args``) — already the same row.
|
||||
- Carried-verbatim columns (``intent_summary``, ``risk_level``,
|
||||
``confidence``, ``recommendation``, ``evidence``, ``latency_ms``) —
|
||||
the fallback copies them from the heuristic verdict; updating
|
||||
would be a no-op.
|
||||
- ``user_decision`` — LOAD-BEARING exclusion. ``IntentVerdict.to_dict()``
|
||||
doesn't project it, so a fallback verdict reaching this layer
|
||||
defaults the kwarg to ``"pending"``. If the operator already
|
||||
resolved the approval between heuristic INSERT and fallback
|
||||
fire, the row's ``user_decision`` was already updated to
|
||||
``"approved"``/``"denied"``/``"timeout"`` (or stamped to an
|
||||
auto-approve reason at heuristic-INSERT time). Clobbering it
|
||||
back to ``"pending"`` would undo that.
|
||||
- ``created`` — preserve the original timestamp.
|
||||
|
||||
Used by :meth:`SessionUIBase._persist_intent_verdict` for every
|
||||
async LLM-tier delivery; the synchronous heuristic-bulk path
|
||||
(:meth:`create_intent_verdicts_bulk`) stays as plain INSERT
|
||||
since each heuristic UUID is freshly generated per turn.
|
||||
"""
|
||||
...
|
||||
|
||||
def create_intent_verdicts_bulk(self, verdicts: list[dict[str, Any]]) -> None:
|
||||
@@ -1580,10 +1687,12 @@ class StorageBackend(Protocol):
|
||||
(``verdict_id`` / ``ws_id`` / ``call_id`` / ``func_name`` /
|
||||
``func_args`` / ``intent_summary`` / ``risk_level`` /
|
||||
``confidence`` / ``recommendation`` / ``reasoning`` / ``evidence`` /
|
||||
``tier`` / ``judge_model`` / ``latency_ms``). Used by the
|
||||
synchronous heuristic-verdict persistence loop in
|
||||
``approve_tools`` so a tool-heavy turn doesn't pay N×commit
|
||||
latency before the approval prompt renders.
|
||||
``tier`` / ``judge_model`` / ``latency_ms`` /
|
||||
``user_decision``). ``user_decision`` defaults to ``"pending"``
|
||||
when absent — see :meth:`create_intent_verdict` for the
|
||||
vocabulary. Used by the synchronous heuristic-verdict
|
||||
persistence loop in ``approve_tools`` so a tool-heavy turn
|
||||
doesn't pay N×commit latency before the approval prompt renders.
|
||||
"""
|
||||
...
|
||||
|
||||
@@ -1865,6 +1974,89 @@ class StorageBackend(Protocol):
|
||||
"""Bulk-delete expired pending MCP OAuth rows. Returns count deleted."""
|
||||
...
|
||||
|
||||
# -- MCP pending-consent (Phase 9; deferred-consent persistence) ----------
|
||||
|
||||
def upsert_mcp_pending_consent(
|
||||
self,
|
||||
user_id: str,
|
||||
server_name: str,
|
||||
error_code: str,
|
||||
scopes_required: str | None,
|
||||
last_ws_id: str | None,
|
||||
last_tool_call_id: str | None,
|
||||
now_iso: str,
|
||||
) -> None:
|
||||
"""Insert or refresh a deferred-consent record for ``(user, server)``.
|
||||
|
||||
On insert: ``first_seen_at = last_seen_at = now_iso``,
|
||||
``occurrence_count = 1``. On conflict (existing row for the
|
||||
same composite PK): rewrites ``error_code``, ``scopes_required``,
|
||||
``last_ws_id``, ``last_tool_call_id``, ``last_seen_at`` to the
|
||||
current values; bumps ``occurrence_count`` by 1. Preserves
|
||||
``first_seen_at`` so the dashboard can show how long the
|
||||
deferred-consent need has been pending.
|
||||
"""
|
||||
...
|
||||
|
||||
def list_mcp_pending_consent_by_user(self, user_id: str) -> list[MCPPendingConsentRow]:
|
||||
"""Return all deferred-consent records for ``user_id``.
|
||||
|
||||
Ordered by ``last_seen_at`` DESC. Empty list when the user has
|
||||
none. Used by the dashboard badge endpoint to render the
|
||||
servers-need-consent list.
|
||||
"""
|
||||
...
|
||||
|
||||
def delete_mcp_pending_consent(self, user_id: str, server_name: str) -> bool:
|
||||
"""Delete the pending-consent row for ``(user, server)``. Returns True if existed.
|
||||
|
||||
Called automatically by the OAuth callback handler when consent
|
||||
completes, and manually via the user-facing DELETE endpoint.
|
||||
"""
|
||||
...
|
||||
|
||||
def delete_all_mcp_pending_consent_by_user(self, user_id: str) -> int:
|
||||
"""Bulk-delete every pending-consent row for ``user_id``. Returns count.
|
||||
|
||||
Used by the manual "dismiss all" endpoint from the settings
|
||||
modal.
|
||||
"""
|
||||
...
|
||||
|
||||
def count_mcp_consented_users_by_server(self, server_name: str) -> int:
|
||||
"""Distinct-user count of non-expired tokens for ``server_name``.
|
||||
|
||||
``expires_at IS NULL`` is treated as non-expired (refresh-only
|
||||
tokens with no advertised expiry). Used by the admin status
|
||||
indicator to show "N users consented" per MCP server row.
|
||||
"""
|
||||
...
|
||||
|
||||
def count_mcp_consented_users_grouped_by_server(self) -> dict[str, int]:
|
||||
"""Bulk distinct-user count of non-expired tokens, grouped by server.
|
||||
|
||||
Single round-trip variant of
|
||||
:meth:`count_mcp_consented_users_by_server` for the admin list
|
||||
handler — replaces the N-call loop that issued one query per
|
||||
server with one ``GROUP BY`` query returning ``{server_name:
|
||||
count}`` for every server that has at least one non-expired
|
||||
token. Servers with zero consented users are absent from the
|
||||
result; callers should ``dict.get(name, 0)`` rather than
|
||||
indexing.
|
||||
"""
|
||||
...
|
||||
|
||||
def any_oauth_user_mcp_servers(self) -> bool:
|
||||
"""Install-level gate for OAuth-MCP features.
|
||||
|
||||
Returns True iff at least one ``mcp_servers`` row has
|
||||
``auth_type='oauth_user'``. Used to short-circuit the pending-
|
||||
consent badge endpoint to ``{pending: 0}`` on local-auth installs
|
||||
with no OAuth MCP servers, so those code paths exercise zero new
|
||||
storage queries.
|
||||
"""
|
||||
...
|
||||
|
||||
# -- Model definitions -----------------------------------------------------
|
||||
|
||||
def create_model_definition(
|
||||
|
||||
@@ -807,6 +807,15 @@ mcp_user_tokens = sa.Table(
|
||||
sa.Column("last_refreshed", sa.Text, nullable=True),
|
||||
sa.PrimaryKeyConstraint("user_id", "server_name"),
|
||||
)
|
||||
# Phase 9: covers the ``WHERE server_name = ? AND (expires_at IS NULL
|
||||
# OR expires_at > now)`` shape used by ``count_mcp_consented_users_*``
|
||||
# for the admin status pill. The composite PK can't satisfy filters
|
||||
# that don't lead with ``user_id``.
|
||||
sa.Index(
|
||||
"idx_mcp_user_tokens_server",
|
||||
mcp_user_tokens.c.server_name,
|
||||
mcp_user_tokens.c.expires_at,
|
||||
)
|
||||
|
||||
mcp_oauth_pending = sa.Table(
|
||||
"mcp_oauth_pending",
|
||||
@@ -821,6 +830,32 @@ mcp_oauth_pending = sa.Table(
|
||||
|
||||
sa.Index("idx_mcp_pending_created", mcp_oauth_pending.c.created_at)
|
||||
|
||||
# Per-(user, server) pending-consent state for non-interactive contexts.
|
||||
# Populated by the pool dispatchers when a scheduled / channel-driven run
|
||||
# hits ``mcp_consent_required`` or ``mcp_insufficient_scope`` and the user
|
||||
# can't be prompted in the moment. Read on dashboard load to render the
|
||||
# "N MCP servers need consent" badge. Cleared by the OAuth callback when
|
||||
# the matching ``(user, server)`` completes consent.
|
||||
#
|
||||
# Composite PK ``(user_id, server_name)`` collapses repeat occurrences for
|
||||
# the same server into one row; ``occurrence_count`` + ``last_*`` fields
|
||||
# carry recency metadata for the dashboard without inflating row count.
|
||||
mcp_pending_consent = sa.Table(
|
||||
"mcp_pending_consent",
|
||||
metadata,
|
||||
sa.Column("user_id", sa.Text, nullable=False),
|
||||
sa.Column("server_name", sa.Text, nullable=False),
|
||||
sa.Column("error_code", sa.Text, nullable=False),
|
||||
sa.Column("scopes_required", sa.Text, nullable=True),
|
||||
sa.Column("last_ws_id", sa.Text, nullable=True),
|
||||
sa.Column("last_tool_call_id", sa.Text, nullable=True),
|
||||
sa.Column("first_seen_at", sa.Text, nullable=False),
|
||||
sa.Column("last_seen_at", sa.Text, nullable=False),
|
||||
sa.Column("occurrence_count", sa.Integer, nullable=False, server_default="1"),
|
||||
sa.PrimaryKeyConstraint("user_id", "server_name"),
|
||||
)
|
||||
sa.Index("idx_mcp_pending_consent_user", mcp_pending_consent.c.user_id)
|
||||
|
||||
# ── TLS / ACME (lacme integration) ──────────────────────────────────────────
|
||||
|
||||
tls_account_keys = sa.Table(
|
||||
|
||||
@@ -19,6 +19,7 @@ if TYPE_CHECKING:
|
||||
from turnstone.core.log import get_logger
|
||||
from turnstone.core.storage._protocol import (
|
||||
MCPOAuthPendingState,
|
||||
MCPPendingConsentRow,
|
||||
MCPUserToken,
|
||||
MCPUserTokenMetadataRow,
|
||||
OIDCIdentity,
|
||||
@@ -33,6 +34,7 @@ from turnstone.core.storage._schema import (
|
||||
heuristic_rules,
|
||||
intent_verdicts,
|
||||
mcp_oauth_pending,
|
||||
mcp_pending_consent,
|
||||
mcp_servers,
|
||||
mcp_user_tokens,
|
||||
metadata,
|
||||
@@ -70,6 +72,9 @@ from turnstone.core.storage._schema import (
|
||||
from turnstone.core.storage._utils import (
|
||||
HEURISTIC_RULE_MUTABLE as _HEURISTIC_RULE_MUTABLE,
|
||||
)
|
||||
from turnstone.core.storage._utils import (
|
||||
LIKE_ESCAPE as _LIKE_ESCAPE,
|
||||
)
|
||||
from turnstone.core.storage._utils import (
|
||||
MCP_SERVER_MUTABLE as _MCP_SERVER_MUTABLE,
|
||||
)
|
||||
@@ -100,6 +105,9 @@ from turnstone.core.storage._utils import (
|
||||
from turnstone.core.storage._utils import (
|
||||
VERDICT_MUTABLE as _VERDICT_MUTABLE,
|
||||
)
|
||||
from turnstone.core.storage._utils import (
|
||||
escape_like as _escape_like,
|
||||
)
|
||||
from turnstone.core.storage._utils import (
|
||||
normalize_search_terms as _normalize_search_terms,
|
||||
)
|
||||
@@ -118,11 +126,6 @@ from turnstone.core.workstream import BULK_CLOSE_STATE_VALUES, WorkstreamKind
|
||||
log = get_logger(__name__)
|
||||
|
||||
|
||||
def _escape_like(s: str) -> str:
|
||||
"""Escape LIKE metacharacters for use with ESCAPE '\\\\'."""
|
||||
return s.replace("\\", "\\\\").replace("%", "\\%").replace("_", "\\_")
|
||||
|
||||
|
||||
def _fts5_query(query: str) -> str:
|
||||
"""Convert a plain search string into a safe FTS5 query."""
|
||||
terms = query.split()
|
||||
@@ -1974,6 +1977,33 @@ class SQLiteBackend:
|
||||
).fetchall()
|
||||
return [dict(r._mapping) for r in rows]
|
||||
|
||||
def find_watch_by_name(self, ws_id: str, name_or_prefix: str) -> dict[str, Any] | None:
|
||||
|
||||
if not name_or_prefix:
|
||||
return None
|
||||
like_pattern = _escape_like(name_or_prefix) + "%"
|
||||
with self._conn() as conn:
|
||||
row = conn.execute(
|
||||
sa.select(watches)
|
||||
.where(
|
||||
(watches.c.ws_id == ws_id)
|
||||
& (
|
||||
(watches.c.name == name_or_prefix)
|
||||
| watches.c.watch_id.like(like_pattern, escape=_LIKE_ESCAPE)
|
||||
)
|
||||
)
|
||||
# Active rows win over inactive ones with the same name.
|
||||
# _prepare_watch's duplicate-name guard filters active=1,
|
||||
# so a model can recreate a name after the previous one
|
||||
# auto-cancelled; a cancel-by-name request on the live
|
||||
# row must not be shadowed by the older completed row.
|
||||
.order_by(watches.c.active.desc(), watches.c.created.desc())
|
||||
.limit(1)
|
||||
).fetchone()
|
||||
if row is None:
|
||||
return None
|
||||
return dict(row._mapping)
|
||||
|
||||
def list_watches_for_node(self, node_id: str) -> list[dict[str, Any]]:
|
||||
|
||||
with self._conn() as conn:
|
||||
@@ -3494,6 +3524,7 @@ class SQLiteBackend:
|
||||
tier: str,
|
||||
judge_model: str,
|
||||
latency_ms: int,
|
||||
user_decision: str = "pending",
|
||||
) -> None:
|
||||
now = datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%S")
|
||||
with self._conn() as conn:
|
||||
@@ -3514,11 +3545,67 @@ class SQLiteBackend:
|
||||
"tier": tier,
|
||||
"judge_model": judge_model,
|
||||
"latency_ms": latency_ms,
|
||||
"user_decision": user_decision,
|
||||
"created": now,
|
||||
},
|
||||
)
|
||||
conn.commit()
|
||||
|
||||
def upsert_intent_verdict(
|
||||
self,
|
||||
verdict_id: str,
|
||||
ws_id: str,
|
||||
call_id: str,
|
||||
func_name: str,
|
||||
func_args: str,
|
||||
intent_summary: str,
|
||||
risk_level: str,
|
||||
confidence: float,
|
||||
recommendation: str,
|
||||
reasoning: str,
|
||||
evidence: str,
|
||||
tier: str,
|
||||
judge_model: str,
|
||||
latency_ms: int,
|
||||
user_decision: str = "pending",
|
||||
) -> None:
|
||||
from sqlalchemy.dialects.sqlite import insert as sqlite_insert
|
||||
|
||||
now = datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%S")
|
||||
stmt = sqlite_insert(intent_verdicts).values(
|
||||
verdict_id=verdict_id,
|
||||
ws_id=ws_id,
|
||||
call_id=call_id,
|
||||
func_name=func_name,
|
||||
func_args=func_args,
|
||||
intent_summary=intent_summary,
|
||||
risk_level=risk_level,
|
||||
confidence=confidence,
|
||||
recommendation=recommendation,
|
||||
reasoning=reasoning,
|
||||
evidence=evidence,
|
||||
tier=tier,
|
||||
judge_model=judge_model,
|
||||
latency_ms=latency_ms,
|
||||
user_decision=user_decision,
|
||||
created=now,
|
||||
)
|
||||
# On verdict_id conflict, update only the three fields that
|
||||
# genuinely change between heuristic and llm_fallback. See the
|
||||
# protocol docstring for the full exclusion rationale —
|
||||
# ``user_decision`` exclusion in particular is load-bearing.
|
||||
stmt = stmt.on_conflict_do_update(
|
||||
index_elements=["verdict_id"],
|
||||
set_={
|
||||
"tier": tier,
|
||||
"reasoning": reasoning,
|
||||
"judge_model": judge_model,
|
||||
},
|
||||
)
|
||||
with self._conn() as conn:
|
||||
conn.execute(stmt)
|
||||
conn.commit()
|
||||
|
||||
def create_intent_verdicts_bulk(self, verdicts: list[dict[str, Any]]) -> None:
|
||||
if not verdicts:
|
||||
return
|
||||
@@ -3539,6 +3626,7 @@ class SQLiteBackend:
|
||||
"tier": v.get("tier", "heuristic"),
|
||||
"judge_model": v.get("judge_model", ""),
|
||||
"latency_ms": v.get("latency_ms", 0),
|
||||
"user_decision": v.get("user_decision", "pending"),
|
||||
"created": now,
|
||||
}
|
||||
for v in verdicts
|
||||
@@ -4462,6 +4550,133 @@ class SQLiteBackend:
|
||||
conn.commit()
|
||||
return result.rowcount
|
||||
|
||||
# -- MCP pending-consent (Phase 9) ----------------------------------------
|
||||
|
||||
def upsert_mcp_pending_consent(
|
||||
self,
|
||||
user_id: str,
|
||||
server_name: str,
|
||||
error_code: str,
|
||||
scopes_required: str | None,
|
||||
last_ws_id: str | None,
|
||||
last_tool_call_id: str | None,
|
||||
now_iso: str,
|
||||
) -> None:
|
||||
from sqlalchemy.dialects import sqlite as sa_sqlite
|
||||
|
||||
stmt = sa_sqlite.insert(mcp_pending_consent).values(
|
||||
user_id=user_id,
|
||||
server_name=server_name,
|
||||
error_code=error_code,
|
||||
scopes_required=scopes_required,
|
||||
last_ws_id=last_ws_id,
|
||||
last_tool_call_id=last_tool_call_id,
|
||||
first_seen_at=now_iso,
|
||||
last_seen_at=now_iso,
|
||||
occurrence_count=1,
|
||||
)
|
||||
stmt = stmt.on_conflict_do_update(
|
||||
index_elements=["user_id", "server_name"],
|
||||
set_={
|
||||
"error_code": stmt.excluded.error_code,
|
||||
"scopes_required": stmt.excluded.scopes_required,
|
||||
"last_ws_id": stmt.excluded.last_ws_id,
|
||||
"last_tool_call_id": stmt.excluded.last_tool_call_id,
|
||||
"last_seen_at": stmt.excluded.last_seen_at,
|
||||
"occurrence_count": mcp_pending_consent.c.occurrence_count + 1,
|
||||
},
|
||||
)
|
||||
with self._conn() as conn:
|
||||
conn.execute(stmt)
|
||||
conn.commit()
|
||||
|
||||
def list_mcp_pending_consent_by_user(self, user_id: str) -> list[MCPPendingConsentRow]:
|
||||
with self._conn() as conn:
|
||||
rows = conn.execute(
|
||||
sa.select(mcp_pending_consent)
|
||||
.where(mcp_pending_consent.c.user_id == user_id)
|
||||
.order_by(mcp_pending_consent.c.last_seen_at.desc())
|
||||
).fetchall()
|
||||
out: list[MCPPendingConsentRow] = []
|
||||
for r in rows:
|
||||
m = r._mapping
|
||||
out.append(
|
||||
MCPPendingConsentRow(
|
||||
user_id=m["user_id"],
|
||||
server_name=m["server_name"],
|
||||
error_code=m["error_code"],
|
||||
scopes_required=m["scopes_required"],
|
||||
last_ws_id=m["last_ws_id"],
|
||||
last_tool_call_id=m["last_tool_call_id"],
|
||||
first_seen_at=m["first_seen_at"],
|
||||
last_seen_at=m["last_seen_at"],
|
||||
occurrence_count=m["occurrence_count"],
|
||||
)
|
||||
)
|
||||
return out
|
||||
|
||||
def delete_mcp_pending_consent(self, user_id: str, server_name: str) -> bool:
|
||||
with self._conn() as conn:
|
||||
result = conn.execute(
|
||||
sa.delete(mcp_pending_consent).where(
|
||||
(mcp_pending_consent.c.user_id == user_id)
|
||||
& (mcp_pending_consent.c.server_name == server_name)
|
||||
)
|
||||
)
|
||||
conn.commit()
|
||||
return bool(result.rowcount)
|
||||
|
||||
def delete_all_mcp_pending_consent_by_user(self, user_id: str) -> int:
|
||||
with self._conn() as conn:
|
||||
result = conn.execute(
|
||||
sa.delete(mcp_pending_consent).where(mcp_pending_consent.c.user_id == user_id)
|
||||
)
|
||||
conn.commit()
|
||||
return int(result.rowcount or 0)
|
||||
|
||||
def count_mcp_consented_users_by_server(self, server_name: str) -> int:
|
||||
now_iso = datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%S")
|
||||
with self._conn() as conn:
|
||||
result = conn.execute(
|
||||
sa.select(sa.func.count(sa.distinct(mcp_user_tokens.c.user_id)))
|
||||
.where(mcp_user_tokens.c.server_name == server_name)
|
||||
.where(
|
||||
sa.or_(
|
||||
mcp_user_tokens.c.expires_at.is_(None),
|
||||
mcp_user_tokens.c.expires_at > now_iso,
|
||||
)
|
||||
)
|
||||
).scalar()
|
||||
return int(result or 0)
|
||||
|
||||
def count_mcp_consented_users_grouped_by_server(self) -> dict[str, int]:
|
||||
now_iso = datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%S")
|
||||
with self._conn() as conn:
|
||||
rows = conn.execute(
|
||||
sa.select(
|
||||
mcp_user_tokens.c.server_name,
|
||||
sa.func.count(sa.distinct(mcp_user_tokens.c.user_id)),
|
||||
)
|
||||
.where(
|
||||
sa.or_(
|
||||
mcp_user_tokens.c.expires_at.is_(None),
|
||||
mcp_user_tokens.c.expires_at > now_iso,
|
||||
)
|
||||
)
|
||||
.group_by(mcp_user_tokens.c.server_name)
|
||||
).fetchall()
|
||||
return {row[0]: int(row[1] or 0) for row in rows}
|
||||
|
||||
def any_oauth_user_mcp_servers(self) -> bool:
|
||||
with self._conn() as conn:
|
||||
result = conn.execute(
|
||||
sa.select(sa.literal(1))
|
||||
.select_from(mcp_servers)
|
||||
.where(mcp_servers.c.auth_type == "oauth_user")
|
||||
.limit(1)
|
||||
).scalar()
|
||||
return result is not None
|
||||
|
||||
# -- Model definitions -----------------------------------------------------
|
||||
|
||||
def create_model_definition(
|
||||
|
||||
@@ -98,6 +98,37 @@ def sanitize_text(value: str | None) -> str | None:
|
||||
return value
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# SQL LIKE escaping
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
# The escape character paired with :func:`escape_like`. Callers MUST
|
||||
# pass ``escape=LIKE_ESCAPE`` to SQLAlchemy's ``.like()`` — without
|
||||
# that kwarg, ``.like()`` uses no escape character at all and the
|
||||
# ``\%`` / ``\_`` sequences produced by :func:`escape_like` would be
|
||||
# interpreted as a literal backslash followed by a wildcard. ``\``
|
||||
# is the SQL standard escape character and works identically on SQLite
|
||||
# and PostgreSQL when passed explicitly.
|
||||
LIKE_ESCAPE = "\\"
|
||||
|
||||
|
||||
def escape_like(value: str) -> str:
|
||||
"""Escape ``%`` and ``_`` (and the escape character itself) so the
|
||||
string can be safely embedded in a SQL ``LIKE`` pattern.
|
||||
|
||||
Pair with ``column.like(escape_like(prefix) + "%", escape=LIKE_ESCAPE)``
|
||||
to do a true prefix match against caller-supplied input. Without
|
||||
this, untrusted text containing ``%`` or ``_`` is interpreted as a
|
||||
wildcard — e.g. a model-supplied watch name of ``"%"`` would match
|
||||
every row in the queried partition.
|
||||
"""
|
||||
return (
|
||||
value.replace(LIKE_ESCAPE, LIKE_ESCAPE * 2)
|
||||
.replace("%", LIKE_ESCAPE + "%")
|
||||
.replace("_", LIKE_ESCAPE + "_")
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Row helper
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
@@ -0,0 +1,50 @@
|
||||
"""Add mcp_pending_consent table.
|
||||
|
||||
Stores per-(user, server) deferred-consent records emitted by the pool
|
||||
dispatchers when a non-interactive run (scheduled / channel) hits
|
||||
``mcp_consent_required`` or ``mcp_insufficient_scope``. Read on
|
||||
dashboard load to render the "N MCP servers need consent" badge; cleared
|
||||
by the OAuth callback handler when consent completes.
|
||||
|
||||
Composite PK ``(user_id, server_name)`` collapses repeat occurrences for
|
||||
the same server into one row. No FKs (matches the rest of the
|
||||
oauth_user schema in migration 049).
|
||||
|
||||
Revision ID: 054
|
||||
Revises: 053
|
||||
Create Date: 2026-05-11
|
||||
"""
|
||||
|
||||
import sqlalchemy as sa
|
||||
from alembic import op
|
||||
|
||||
revision = "054"
|
||||
down_revision = "053"
|
||||
branch_labels = None
|
||||
depends_on = None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
op.create_table(
|
||||
"mcp_pending_consent",
|
||||
sa.Column("user_id", sa.Text, nullable=False),
|
||||
sa.Column("server_name", sa.Text, nullable=False),
|
||||
sa.Column("error_code", sa.Text, nullable=False),
|
||||
sa.Column("scopes_required", sa.Text, nullable=True),
|
||||
sa.Column("last_ws_id", sa.Text, nullable=True),
|
||||
sa.Column("last_tool_call_id", sa.Text, nullable=True),
|
||||
sa.Column("first_seen_at", sa.Text, nullable=False),
|
||||
sa.Column("last_seen_at", sa.Text, nullable=False),
|
||||
sa.Column("occurrence_count", sa.Integer, nullable=False, server_default="1"),
|
||||
sa.PrimaryKeyConstraint("user_id", "server_name"),
|
||||
)
|
||||
op.create_index(
|
||||
"idx_mcp_pending_consent_user",
|
||||
"mcp_pending_consent",
|
||||
["user_id"],
|
||||
)
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
op.drop_index("idx_mcp_pending_consent_user", table_name="mcp_pending_consent")
|
||||
op.drop_table("mcp_pending_consent")
|
||||
@@ -0,0 +1,59 @@
|
||||
"""Index mcp_user_tokens by (server_name, expires_at).
|
||||
|
||||
Phase 9 admin pill (``count_mcp_consented_users_*``) filters
|
||||
``mcp_user_tokens`` by ``server_name`` and ``expires_at``. The table's
|
||||
only existing index is the composite PK ``(user_id, server_name)`` —
|
||||
``user_id`` is the leading column, so a filter on ``server_name`` alone
|
||||
must full-scan the table. The bulk ``GROUP BY server_name`` variant in
|
||||
the admin list handler benefits from the same index.
|
||||
|
||||
PostgreSQL uses ``CREATE INDEX CONCURRENTLY`` inside an
|
||||
``autocommit_block`` so the build is non-blocking on a live system —
|
||||
``mcp_user_tokens`` is on the token-refresh hot path and an
|
||||
ACCESS EXCLUSIVE lock during build would stall refresh writers on
|
||||
installs with non-trivial row counts. SQLite has no concurrent build
|
||||
concept and the table-level write lock already serializes, so a plain
|
||||
``op.create_index`` is fine. Pattern mirrors migration 048
|
||||
(``idx_workstreams_reaper``).
|
||||
|
||||
Revision ID: 055
|
||||
Revises: 054
|
||||
Create Date: 2026-05-11
|
||||
"""
|
||||
|
||||
from alembic import op
|
||||
|
||||
revision = "055"
|
||||
down_revision = "054"
|
||||
branch_labels = None
|
||||
depends_on = None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
bind = op.get_bind()
|
||||
dialect = bind.dialect.name
|
||||
|
||||
if dialect == "postgresql":
|
||||
with op.get_context().autocommit_block():
|
||||
op.execute(
|
||||
"CREATE INDEX CONCURRENTLY IF NOT EXISTS "
|
||||
"idx_mcp_user_tokens_server ON mcp_user_tokens "
|
||||
"(server_name, expires_at)"
|
||||
)
|
||||
else:
|
||||
op.create_index(
|
||||
"idx_mcp_user_tokens_server",
|
||||
"mcp_user_tokens",
|
||||
["server_name", "expires_at"],
|
||||
)
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
bind = op.get_bind()
|
||||
dialect = bind.dialect.name
|
||||
|
||||
if dialect == "postgresql":
|
||||
with op.get_context().autocommit_block():
|
||||
op.execute("DROP INDEX CONCURRENTLY IF EXISTS idx_mcp_user_tokens_server")
|
||||
else:
|
||||
op.drop_index("idx_mcp_user_tokens_server", table_name="mcp_user_tokens")
|
||||
+92
-24
@@ -289,6 +289,17 @@ class WatchRunner:
|
||||
self._dispatch_fns: dict[str, Callable[[dict[str, Any], str], None]] = {}
|
||||
self._dispatch_lock = threading.Lock()
|
||||
|
||||
# Watch ids whose terminal reminder has already been dispatched
|
||||
# but whose row write has not yet been confirmed. Populated
|
||||
# between ``_dispatch_result`` and ``update_watch`` in
|
||||
# :meth:`_poll_watch`; on a subsequent tick the same row will
|
||||
# still appear in ``list_due_watches`` (active=1, next_poll
|
||||
# unchanged) — the guard at the top of ``_poll_watch`` retries
|
||||
# the row write WITHOUT re-dispatching. Bounded by transient
|
||||
# storage failure depth (~MAX_WATCHES_PER_WS × num_ws).
|
||||
self._terminal_dispatched: set[str] = set()
|
||||
self._terminal_dispatched_lock = threading.Lock()
|
||||
|
||||
self._stop_event = threading.Event()
|
||||
self._thread: threading.Thread | None = None
|
||||
|
||||
@@ -314,14 +325,18 @@ class WatchRunner:
|
||||
def set_dispatch_fn(self, ws_id: str, fn: Callable[[dict[str, Any], str], None]) -> None:
|
||||
"""Register a per-workstream dispatch fn.
|
||||
|
||||
The fn signature is ``(reminder, watch_id)`` — the runner passes
|
||||
the originating ``watch_id`` so dispatch closures can capture
|
||||
per-watch metadata (e.g. a ``valid_until`` predicate that
|
||||
re-checks ``storage.is_watch_active(watch_id)`` before
|
||||
delivering a stale entry). ``reminder`` is the structured dict
|
||||
returned by :func:`build_watch_reminder` — ``text`` carries the
|
||||
formatted body, the remaining fields ride as queue-entry
|
||||
metadata so the frontend can render a ``.msg.watch-result`` card.
|
||||
The fn signature is ``(reminder, watch_id)``. ``reminder`` is
|
||||
the structured dict returned by :func:`build_watch_reminder` —
|
||||
``text`` carries the formatted body, the remaining fields ride
|
||||
as queue-entry metadata so the frontend can render a
|
||||
``.msg.watch-result`` card. ``watch_id`` is passed for
|
||||
closures that need per-watch metadata in their queue plumbing
|
||||
(e.g. correlating a fire back to the originating row in logs);
|
||||
do NOT use it to gate delivery against
|
||||
``storage.is_watch_active(watch_id)`` — see
|
||||
:meth:`ChatSession.set_watch_runner` for why that pattern
|
||||
races :meth:`_poll_watch`'s commit of ``active=False`` and
|
||||
drops fires the model was meant to see.
|
||||
"""
|
||||
with self._dispatch_lock:
|
||||
self._dispatch_fns[ws_id] = fn
|
||||
@@ -339,6 +354,22 @@ class WatchRunner:
|
||||
with self._dispatch_lock:
|
||||
return self._dispatch_fns.get(ws_id)
|
||||
|
||||
def forget_terminal_dispatched(self, watch_id: str) -> None:
|
||||
"""Discard ``watch_id`` from the pending-terminal-dispatched
|
||||
set if present. Called by paths that take a watch out of
|
||||
:meth:`StorageBackend.list_due_watches` view independent of
|
||||
the runner's own poll (most importantly the user-cancel path
|
||||
in :meth:`ChatSession._exec_watch`). Without this, a
|
||||
``_poll_watch`` whose row write failed AFTER dispatch would
|
||||
leak ``watch_id`` in ``_terminal_dispatched`` indefinitely —
|
||||
the user-cancel writes ``next_poll=''`` which excludes the
|
||||
row from ``list_due_watches``, so the retry-deactivate branch
|
||||
at the top of :meth:`_poll_watch` never fires to clear the
|
||||
entry.
|
||||
"""
|
||||
with self._terminal_dispatched_lock:
|
||||
self._terminal_dispatched.discard(watch_id)
|
||||
|
||||
# -- Main loop -----------------------------------------------------------
|
||||
|
||||
def _run(self) -> None:
|
||||
@@ -383,6 +414,21 @@ class WatchRunner:
|
||||
prev_output = watch_row.get("last_output")
|
||||
created = watch_row.get("created", "")
|
||||
|
||||
# Re-poll of a row whose terminal reminder already shipped but
|
||||
# whose ``active=False`` write didn't land — retry just the row
|
||||
# write so the row stops appearing in ``list_due_watches``; do
|
||||
# NOT re-dispatch the reminder, which the model already saw.
|
||||
with self._terminal_dispatched_lock:
|
||||
already_dispatched = watch_id in self._terminal_dispatched
|
||||
if already_dispatched:
|
||||
try:
|
||||
self._storage.update_watch(watch_id, active=False, next_poll="")
|
||||
with self._terminal_dispatched_lock:
|
||||
self._terminal_dispatched.discard(watch_id)
|
||||
except Exception:
|
||||
log.exception("watch_runner.retry_deactivate_failed", extra={"watch_id": watch_id})
|
||||
return
|
||||
|
||||
# Safety check
|
||||
blocked = is_command_blocked(command)
|
||||
if blocked:
|
||||
@@ -416,22 +462,17 @@ class WatchRunner:
|
||||
now = datetime.now(UTC)
|
||||
now_str = now.strftime("%Y-%m-%dT%H:%M:%S")
|
||||
|
||||
# Update DB
|
||||
update_fields: dict[str, Any] = {
|
||||
"poll_count": poll_count,
|
||||
"last_output": output,
|
||||
"last_exit_code": exit_code,
|
||||
"last_poll": now_str,
|
||||
}
|
||||
if is_final:
|
||||
update_fields["active"] = False
|
||||
update_fields["next_poll"] = ""
|
||||
else:
|
||||
next_poll = now + timedelta(seconds=watch_row["interval_secs"])
|
||||
update_fields["next_poll"] = next_poll.strftime("%Y-%m-%dT%H:%M:%S")
|
||||
self._storage.update_watch(watch_id, **update_fields)
|
||||
|
||||
# Dispatch result if condition fired or final
|
||||
# Dispatch before committing the row update. Belt-and-braces
|
||||
# given the rest of the fix (closure no longer wires a
|
||||
# ``valid_until`` predicate, cancel-by-name uses
|
||||
# :meth:`find_watch_by_name` which ignores the ``active``
|
||||
# filter): either order would deliver the reminder today, but
|
||||
# this ordering preserves the invariant against re-wiring an
|
||||
# ``is_watch_active`` predicate or adding a new
|
||||
# ``active``-filtered read on this hot path. Combined with the
|
||||
# ``_terminal_dispatched`` guard above it also bounds the
|
||||
# duplicate-fire blast radius if the row write fails after the
|
||||
# reminder shipped.
|
||||
if fired or is_final:
|
||||
# Compute elapsed from created time
|
||||
elapsed_secs = 0.0
|
||||
@@ -454,6 +495,33 @@ class WatchRunner:
|
||||
reason=reason,
|
||||
)
|
||||
self._dispatch_result(ws_id, reminder, watch_id)
|
||||
if is_final:
|
||||
# Mark BEFORE the row write so a raise below routes the
|
||||
# next tick into the retry-deactivate branch instead of
|
||||
# re-firing the reminder.
|
||||
with self._terminal_dispatched_lock:
|
||||
self._terminal_dispatched.add(watch_id)
|
||||
|
||||
# Update DB
|
||||
update_fields: dict[str, Any] = {
|
||||
"poll_count": poll_count,
|
||||
"last_output": output,
|
||||
"last_exit_code": exit_code,
|
||||
"last_poll": now_str,
|
||||
}
|
||||
if is_final:
|
||||
update_fields["active"] = False
|
||||
update_fields["next_poll"] = ""
|
||||
else:
|
||||
next_poll = now + timedelta(seconds=watch_row["interval_secs"])
|
||||
update_fields["next_poll"] = next_poll.strftime("%Y-%m-%dT%H:%M:%S")
|
||||
self._storage.update_watch(watch_id, **update_fields)
|
||||
|
||||
if is_final:
|
||||
# Row write committed; the retry-deactivate branch will
|
||||
# never be reached for this watch_id.
|
||||
with self._terminal_dispatched_lock:
|
||||
self._terminal_dispatched.discard(watch_id)
|
||||
|
||||
log.debug(
|
||||
"watch_runner.polled",
|
||||
|
||||
@@ -37,6 +37,19 @@ class ClientType(enum.StrEnum):
|
||||
WEB = "web"
|
||||
CLI = "cli"
|
||||
CHAT = "chat"
|
||||
SCHEDULED = "scheduled"
|
||||
|
||||
|
||||
# Subset of ``ClientType`` values where the user is present to complete
|
||||
# an in-flight OAuth consent flow (browser redirect + return). CHAT and
|
||||
# SCHEDULED users cannot drive a browser redirect from inside their
|
||||
# delivery surface, so consent-required errors must be persisted to
|
||||
# ``mcp_pending_consent`` for later surfacing rather than relying on the
|
||||
# in-flight SSE rendering path. Used by ``ChatSession`` to set
|
||||
# ``_is_interactive_for_consent`` at construction time.
|
||||
INTERACTIVE_CONSENT_CLIENT_TYPES: frozenset[ClientType] = frozenset(
|
||||
{ClientType.WEB, ClientType.CLI}
|
||||
)
|
||||
|
||||
|
||||
@dataclasses.dataclass
|
||||
@@ -56,6 +69,7 @@ _ENV_MAP: dict[ClientType, str] = {
|
||||
ClientType.WEB: "env/web.md",
|
||||
ClientType.CLI: "env/cli.md",
|
||||
ClientType.CHAT: "env/chat.md",
|
||||
ClientType.SCHEDULED: "env/scheduled.md",
|
||||
}
|
||||
|
||||
|
||||
|
||||
Vendored
+20
@@ -0,0 +1,20 @@
|
||||
## Output Environment
|
||||
|
||||
Your response is generated by a scheduled or autonomous run — no human is watching the output as it streams. The result is delivered to the user later (Discord notification, dashboard badge, or persisted workstream history) where they will see it as a static block of markdown.
|
||||
|
||||
**Implications:**
|
||||
|
||||
- The user is not online to answer mid-task clarifying questions. Make the reasonable judgment and continue; mention the assumption you made in the final output so the user can correct course on the next run if needed.
|
||||
- Tool calls that require interactive user consent (e.g., MCP servers gated on OAuth user authorization that the user has not yet completed) will return a deferred-consent error rather than block the run. Surface the deferred work in your final summary so the user knows what was skipped.
|
||||
- Optimize for a clear, scannable final summary over conversational back-and-forth — the user reads the whole transcript at once, not turn-by-turn.
|
||||
|
||||
**Available rendering:**
|
||||
|
||||
- Standard GitHub-flavored markdown is supported in the dashboard surface. Discord delivery uses the same constraints as `chat.md` (no tables, no headings beyond bold text, no Mermaid/KaTeX).
|
||||
- Default to chat-portable formatting (bold/italic/inline-code/bullets/code-blocks) unless you know the destination is the web dashboard.
|
||||
|
||||
**Formatting principles:**
|
||||
|
||||
- Lead with the outcome in one line: what was accomplished, what was skipped, and why.
|
||||
- For multi-step work, end with a concise checklist of what ran and what remains.
|
||||
- Cite specific identifiers (workstream IDs, tool names, MCP server names) so the user can resume the work without re-reading the trace.
|
||||
@@ -2822,6 +2822,27 @@ async def mcp_oauth_revoke_connection(request: Request) -> Response:
|
||||
return await handle_mcp_oauth_revoke_connection(request)
|
||||
|
||||
|
||||
async def mcp_oauth_list_pending(request: Request) -> Response:
|
||||
"""GET /v1/api/mcp/oauth/pending — list deferred-consent records (Phase 9)."""
|
||||
from turnstone.core.mcp_oauth import handle_mcp_oauth_list_pending
|
||||
|
||||
return await handle_mcp_oauth_list_pending(request)
|
||||
|
||||
|
||||
async def mcp_oauth_clear_pending(request: Request) -> Response:
|
||||
"""DELETE /v1/api/mcp/oauth/pending/{server_name} — dismiss a deferred-consent record."""
|
||||
from turnstone.core.mcp_oauth import handle_mcp_oauth_clear_pending
|
||||
|
||||
return await handle_mcp_oauth_clear_pending(request)
|
||||
|
||||
|
||||
async def mcp_oauth_clear_all_pending(request: Request) -> Response:
|
||||
"""DELETE /v1/api/mcp/oauth/pending — bulk-dismiss deferred-consent records."""
|
||||
from turnstone.core.mcp_oauth import handle_mcp_oauth_clear_all_pending
|
||||
|
||||
return await handle_mcp_oauth_clear_all_pending(request)
|
||||
|
||||
|
||||
def list_interface_settings(request: Request) -> JSONResponse:
|
||||
"""GET /v1/api/admin/settings — return interface settings from ConfigStore.
|
||||
|
||||
@@ -4064,6 +4085,17 @@ def create_app(
|
||||
mcp_oauth_revoke_connection,
|
||||
methods=["DELETE"],
|
||||
),
|
||||
Route("/api/mcp/oauth/pending", mcp_oauth_list_pending),
|
||||
Route(
|
||||
"/api/mcp/oauth/pending",
|
||||
mcp_oauth_clear_all_pending,
|
||||
methods=["DELETE"],
|
||||
),
|
||||
Route(
|
||||
"/api/mcp/oauth/pending/{server_name}",
|
||||
mcp_oauth_clear_pending,
|
||||
methods=["DELETE"],
|
||||
),
|
||||
Route("/api/admin/settings", list_interface_settings),
|
||||
Route(
|
||||
"/api/admin/settings/{key:path}",
|
||||
|
||||
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user