mirror of
https://github.com/turnstonelabs/turnstone.git
synced 2026-08-12 23:12:23 -06:00
main
13 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Patrick Buckley
|
33ace975d2 |
feat(models): default-deny governance and admin UI for per-alias backend auth
Follow-up to the per-alias Entra OBO/app-identity backend auth: the console write path now applies default-deny field classification, the admin shelf gains full backend-auth support, and the session/registry rebind machinery is hardened for config changes landing under live sessions. Console write gate: - Default-deny classification: any non-neutral change to a row that is or becomes dynamic requires admin.mcp plus validation; the provably auth-neutral columns are enumerated (MODEL_AUTH_NEUTRAL_FIELDS) and a live-schema classification test forces every future column to be classified. The derivation is a pure function (_derive_auth_gate) with unit-pinned exclusivity invariants. - Two-tier validation mirroring the MCP oauth_obo validator: the row tier (audience allow-list) runs on every gated write; the posture tier (OIDC configured, token store present) runs on pair changes and on enable-arming. - Pure-disable carve-out: disabling a dynamic row is de-escalation and is never blocked — admin.models suffices and validation is skipped, including for rows with corrupt or skewed stored values. - Capabilities are compared canonically (key order, integral floats), the audience compare normalizes both sides, and staging an audience on a static row is refused on both write twins. - Calibrate writes the capabilities column under an enforced confinement invariant with a compare-and-swap persist. Admin shelf: - Backend-auth section with a per-open constraints fetch (GET /model-definitions/auth-constraints: audience allow-list, grant profile, dynamic modes), datalist audience suggestions, server-defined modes preserved on round-trip, and permission-aware visibility built on cache-skew-safe helpers shared through auth.js. - Refused live-registry swaps surface as an amber registry_warning on the write, delete, reload, and calibrate responses; audit rows carry auth_gated / auth_disarmed markers visible in the audit view. Registry and sessions: - The encryption-key requirement for dynamic auth is enforced inside ModelRegistry.reload() itself — nodes refuse with 503 and the console records coord_registry_error — and reload bumps the generation before the map swap so a racing reader can never pair a stale generation with new maps. - resolve()/resolve_binding() return the generation from inside the registry lock; sessions rebind per send on generation change with atomic client/provider/config commits, fallback-first handling of removed or unconstructable aliases, and judge/limiter resets only when the binding actually changed. - Mint refusals record per-user causes surfaced in the per-turn heartbeat logs; misconfiguration warnings are deduplicated with bounded state. Verification: 10417 tests (99 added on this branch), a 71-scenario browser harness over the real admin shelf, and a live rfc8693 token-exchange e2e run (MCP legs verified end to end; the model-leg scope gap is tracked as #955 under a narrow known-gap signature). Closes #950. |
||
|
Patrick Buckley
|
d1de602b78 |
docs(mcp): align token-encryption + mint docstrings with oauth_obo
Startup key-enforcement counts ALL user-scoped auth types (oauth_user and oauth_obo, per is_user_scoped_auth), and the entra mint leg always carries scope=<audience>/.default (per-server oauth_scopes is ignored on that leg). The docstrings named only oauth_user / left the scope behavior ambiguous. Comment-only; no behavior change. |
||
|
Patrick Buckley
|
8a1efaf55e |
perf(mcp): skip redundant priming credential read; dedup sweep clear bookkeeping
Round-11 review follow-up — no correctness findings; efficiency/DRY cleanups. - Session-start priming already confirms the captured credential exists once for all of a user's obo servers, but each per-server get_obo_access_token_classified re-read it pre-lock (N+1 reads). The priming path now passes credential_present=True so the per-server existence read is skipped; other callers keep their own read. - _clear_pending_consent_best_effort (the sweep clear path) now routes through _mark_pending_consent_cleared instead of inlining the prune-then-stamp step, matching the helper's documented contract so the two DB-confirmed clear sites can't drift. The per-dispatch pending-consent clear's DELETE volume and the removed interactive.js no-consent-URL fallback are left as-is: the former is the deliberate, TTL-bounded cost of cross-node badge self-heal, and the latter is unreachable for oauth_user (which always carries a consent_url) and intended for oauth_obo (which has no per-server consent flow). |
||
|
Patrick Buckley
|
08d765a74a |
fix(mcp): record effective obo scope so entra .default isn't cached as narrow
Round-10 review follow-up.
- A server scoped under obo_grant_profile=rfc8693 that survives a switch
to the entra profile mints <audience>/.default (the entra leg cannot
honor per-server oauth_scopes), but the cache row recorded the
configured narrow scope — so _is_fresh_obo_cache_row kept serving the
broad .default bearer believing it was narrow, and a scope change that
can't apply under entra looked like it had. The freshness gate and the
cache row now record the EFFECTIVE scope the leg actually mints ('' for
entra, the configured scope for rfc8693); the raw scope is still passed
to the mint so the entra leg's "oauth_scopes ignored" warning still
surfaces the misconfigured leftover.
Cleanup: the R9-5 single-per-mint client made every token-POST caller pass
a non-None client, so the transient-client fallback in _hardened_token_post
was dead and two doc/comment blocks described the opposite of the real
behavior. Removed the dead branch, tightened the http_client typing across
the mint chain, and corrected the docs.
|
||
|
Patrick Buckley
|
50d0ac9833 |
fix(mcp): classify oversized token error by status; dedup transition/obo-scan
Round-6 review follow-up — no CONFIRMED correctness bugs; one plausible edge case and four DRY/drift cleanups. - The shared hardened token-POST raised its 64KB body-size guard with the default TRANSIENT class before the non-200 was classified, so a permanent dead-grant whose error body exceeded the cap looped "please retry" forever and never escalated. An over-sized client-error response is now classified AMBIGUOUS by status (without reading the over-sized body), so it still escalates to the honest re-login/admin remedy after the streak. - The admin update handler re-derived the is_flip predicate inline in the three token-purge guards (and computed target_auth / auth_type_now as two names for the same effective auth type). Both now reuse the single is_flip / target_auth derivations, so the purge guards and the column scrub can't desync on what counts as a flip. - The oauth_obo server-name scan was hand-rolled in two places (the connections-list filter and the identity-delete cache purge) with divergent null handling. Extracted obo_server_names(storage) so a change to how sign-in-passthrough is recognised can't leave one path silently missing servers. - Inlined the two single-use _*_detail wrappers into direct _pool_error_detail calls, keeping named wrappers only for the multi-caller situations. |
||
|
Patrick Buckley
|
09aa50b7a1 |
fix(mcp): close obo auth-column leak, capture gate, and cooldown classification
Round-5 review follow-up — three CONFIRMED (one security) plus two correctness issues, all traceable to earlier fixes in this branch. SECURITY: the round-2 redesign gated the "scrub OAuth columns this auth_type doesn't use" on is_flip, replacing the old unconditional scrub. A same-type static/none/obo edit could then inject an oauth_authorization_server_url that survived a later flip to oauth_user (which uses that column) and redirected every consenting user's OAuth traffic to an attacker AS. The scrub is now applied on EVERY write, and a flip into oauth_user recomputes the oauth_user-only columns from the request so a stale value can't carry in — the persisted OAuth columns are once again a pure function of the target auth_type. - The oauth_obo write gate now also requires capture_user_credential to be enabled: without it, login persists no credential and every dispatch returns "missing" with a remedy that can never succeed — the permanent misconfig the gate exists to reject. - A permanent obo mint failure arms the cooldown (its shared credential survives the per-server revoke), but the in-cooldown short-circuit reported it as a retryable transient for the whole window, flapping against the honest re-login/admin affordance. The backoff state now records whether the arming failure was permanent, and the short-circuit surfaces the matching classification. - The ambiguous-escalation revoke cleared the cooldown without re-arming; for obo (surviving credential) that let the next dispatch immediately re-mint against the still-failing IdP. It now re-arms the same terminal backstop the permanent branch has. - The force-refresh reuse gate keyed on the cache row's 1-second `created` time, which couldn't tell a concurrent peer's fresh mint from the caller's own just-rejected token minted in the same second — so a retry could re-serve the rejected bearer. It now decides by token identity (the under-lock row differs from the pre-lock one), preserving the single-flight reuse while never re-serving a rejected token. Also: guard _pool_error_detail's str.format so placeholder-free copy can't raise inside the error renderer, and note why the connections-list classifies obo rows by authoritative auth_type on that cold path. |
||
|
Patrick Buckley
|
6d80051925 |
fix(mcp): decouple capture key guard from OIDC discovery; bound obo token TTL
Round-3 review follow-up. - The startup guard that refuses to boot without a token-encryption key when capture_user_credential is enabled was gated on oidc_config.enabled. Enabled reflects whether OIDC *discovery* succeeded, which is transient: a node that boots while the IdP is unreachable comes up enabled=False, so the guard was silently skipped exactly when it was needed, and runtime rediscovery would later re-enable OIDC with the first login persisting a refresh token and no key. Gate on the operator's capture opt-in alone (a static config value), independent of discovery state. - An obo mint response omitting the RFC 8693-optional expires_in cached expires_at=NULL, which the freshness gate reads as never-expiring — fine for opaque oauth_user tokens, wrong for a short-lived minted token, which would then be served indefinitely and defeat audience/scope narrowing that relies on TTL turnover. Fall back to a bounded default expiry. - The empty-token fallback in the shared pool-lookup error mapping now uses the auth-model-aware consent detail like its sibling missing branch, so an obo row never shows per-server-consent copy with a null consent URL. - Documented the _build_consent_url invariant at the chat error-card render gate: oauth_user rows always carry a consent URL, so gating the Connect button on its presence never hides a needed button for them; the button's absence for sign-in passthrough is intended (the detail text is the affordance). |
||
|
Patrick Buckley
|
d2e69ca527 |
fix(mcp): coherent obo auth-type carry-over + honest error affordances
Round-2 review follow-up. The headline is a redesign of the OAuth column carry-over so scopes/audience can no longer leak or vanish across an auth-type flip: - oauth_audience and oauth_scopes keep their meaning only WITHIN an auth type (a resource indicator vs. an IdP app id; AS-consent scopes vs. an rfc8693 exchange scope). On any oauth_user<->oauth_obo flip they are now recomputed from the request (present -> value, absent -> NULL) and never carried from the old row. A shared _oauth_columns_to_clear policy drives both the create and update handlers. No-op normalization of a re-sent equal value applies only to same-type edits. - The console form clears both semantic fields when the auth type changes and always submits the visible values; the previous "omit unchanged scopes" logic collided with the backend's flip handling and could silently drop or carry scopes. Write-time validation now rejects oauth_obo rows that can never mint — OIDC disabled/unconfigured, or an invalid obo_grant_profile — instead of letting them surface per-dispatch as a retryable transient that never heals. Honest failure affordances for sign-in passthrough (no per-server consent flow exists): - the token_revoked audit fires only when a row was actually deleted, so a permanent mint rejection against a surviving credential no longer appends a bogus revocation on every post-cooldown dispatch/prime; - the 403 insufficient-scope detail and the chat error card's action button are now auth-model-aware — obo errors point at the administrator rather than a dead-end re-consent, and the Connect button renders only when a real consent URL is present; - the read-side freshness gate now enforces scopes as well as audience, so an rfc8693 scope narrowing takes effect on the next dispatch even if the best-effort admin cache purge failed. Cleanups: the five decrypt-failure result constructions collapse into _decrypt_failure_result; the cleared-pairs TTL bookkeeping into _mark_pending_consent_cleared; drop the dead USER_SCOPED_AUTH_TYPES re-export from mcp_oauth; correct the now-bidirectional oidc<->mcp_oauth lazy-import note. Docs updated for the flip semantics and the OIDC prerequisite. |
||
|
Patrick Buckley
|
32c76499fa |
fix(mcp): harden obo mint path and admin lifecycle after review
Mint engine: guard the credential-rotation persist so a storage blip cannot escape the classified-result contract mid-mint (and cannot brick the user's other obo servers on strict-rotation IdPs); stop borrowing the login flow's httpx client across event loops — mints use a transient per-request client (obo_http_client remains as a test seam); retry OIDC discovery at runtime (cooldown-gated, single-flight) so a node that booted during an IdP outage can mint again without a restart; key the under-lock force-refresh reuse gate on created, which delete+create makes the mint time (obo rows never set last_refreshed, so the copied oauth_user gate never fired and serialized waiters each re-redeemed). Cross-node consent badges: the cleared-pairs set becomes a TTL map with bounded growth, so a badge written by another node after this node's last clear self-heals within one TTL window instead of surviving until a restart. Admin lifecycle: purge the mint cache when oauth_scopes changes on an obo row (an rfc8693 privilege reduction now applies immediately, like audience changes); normalize no-op scope/audience re-sends out of updates — the admin form re-submits pre-filled fields on every save, which both re-triggered purges and made entra-profile rows with legacy scopes un-editable; make flip-into-obo scope handling grant-profile aware (entra clears the carry-over, rfc8693 honors the request); clear obo-era audience/scopes when flipping back to oauth_user (the IdP-side app identifier is not a resource indicator); mirror the same column policy in the create handler. Revocation honesty: hide obo mint-cache rows from the user connections list and refuse the per-server disconnect with 409 — deleting the row returned 204, audited token_revoked, and then session-start priming silently re-minted from the surviving captured credential. Console form: keep the audience-from-URL autofill off for sign-in passthrough (the audience there is an IdP application identifier, and the prefilled URL passed every validation layer then failed every mint); clear the autofill artifact when switching modes; omit unchanged scopes from submissions. Dispatchers: route tool/resource/prompt through one shared lookup-error mapping and an auth-model-aware 401-exhausted detail (obo users are no longer pointed at a consent flow that does not exist). The consent-url audit count drops 13 → 7: the three per-dispatcher mapping copies collapsed into _pool_lookup_error. Priming: skip all obo servers for users with no captured credential via one existence SELECT (previously three reads per server per session). Also: USER_SCOPED_AUTH_TYPES now lives in storage._protocol so the backend SQL predicates share the application layer's set; docs describe the actual purge-on-transition behavior (the orphan-and-reactivate claims were wrong); the entra e2e setup script no longer aborts silently under set -e with suppressed stderr. |
||
|
Patrick Buckley
|
44e9d46e40 |
fix(mcp): address pre-push review — obo scope/audience/priming defects
Frontend↔backend interaction bugs the backend-only rounds couldn't see: - flip oauth_user->oauth_obo: the admin form re-submits the pre-filled oauth_user scopes, so the flip-clear (gated on 'oauth_scopes' not in body) was skipped -> rfc8693 mints broke permanently. Clear now compares to the existing value, robust to the re-send. - entra edit-lockout: update validated the MERGED scopes, so a pre-existing scoped obo row under the entra profile became un-editable (every PUT 400'd). Reject only when the request actually SETS scopes. - flush-cache button never rendered: consented_users_count is now populated for oauth_obo rows too, not just oauth_user. Mint engine + priming: - audience guard: a cached token minted for a since-narrowed audience is no longer served (extracted _is_fresh_obo_cache_row, used pre/post-lock, checks refresh-less + audience-match + fresh). _persist_obo_cache_row now delete+creates so the row's audience column tracks the mint (a plain update kept the stale audience -> re-mint loop). - obo session priming passes revoke_ambiguous_escalation=False (new param threaded through get_obo_...), so an IdP wobble during a bulk prime can't escalate-revoke obo cache rows cluster-wide. Cross-node + lifecycle: - pending-consent success-clear now clears once-per-failure-cycle via a _pending_consent_cleared set (was gated on 'we wrote it' -> never fired cross-node/after-restart -> stale badge). Still no per-call SQL. - identity-unlink cache purge: per-server try/except so one failure doesn't leave other servers' bearers un-purged. - entra ignored-scopes: warn once per audience (was per-mint flood -> downgraded to debug -> no signal on a profile switch). - entra_setup.sh writes single-quoted .env values (secret may contain $). +6 regression tests. 1892 mcp/oidc/console tests green; mypy clean. Refs #551. |
||
|
Patrick Buckley
|
e5f8453e1a |
fix(mcp): complete oauth_obo revocation lifecycle + fix hot-path regression (follow-up review)
Addresses the high follow-up review of the first fix round: Revocation lifecycle (the review's dominant theme): - identity-unlink now purges the user's minted obo cache rows in addition to revoking the credential, and the response/audit report the actual effect (credential + N cache rows) instead of a blanket revoked=true; warmed-session residual (bounded by token TTL) documented - bulk-revoke on obo is now an honest cache-FLUSH: distinct audit event (obo_cache_flushed) + response effect=cache_flush_remints, since the shared credential survives and the next dispatch re-mints (oauth_user keeps its durable revoke semantics) - changing oauth_audience on a pool-backed row now purges cached tokens (audience is the token binding), like URL/name/auth_type changes - flipping oauth_user->oauth_obo now clears the stale AS-consent scopes (else rfc8693 sends them -> invalid_scope loop); write path rejects oauth_scopes under the entra profile (it mints <audience>/.default) - a cache row bearing a refresh token is never served as an obo token (guards the cross-node purge-vs-refresh race) Self-inflicted regression: - _clear_pending_consent_sync is now gated on an in-memory _pending_consent_written hint, so the common successful-dispatch path issues ZERO SQL (was an unconditional per-dispatch DELETE) Observability + cleanups: - restore the obo_mint_rejected log carrying the IdP error text (the shared-helper unification dropped it); event names passed as whole literals so alerting can grep them - persist_rotation typed Callable[[str], Awaitable[None]] (was Any) - _prime_one branches on _obo_server_names (no pre-lookup SQL for oauth_user) - removed now-dead any_oauth_user_mcp_servers (3 impls + tests) +13 regression tests. Full mcp/oidc/console suite 1888 green; mypy clean. Refs #551. |
||
|
Patrick Buckley
|
17c44305d6 |
fix(mcp): harden oauth_obo mint engine per max review (P0 security core)
Addresses the review's B/D/F classes + single-sourcing: - B (credential corruption): the rfc8693 refresh-leg rotation is now persisted the instant it is obtained, BEFORE the exchange leg, via a persist_rotation callback under the held credential lock. A rotated RT survives an exchange-leg failure (no more cascade lockout), and the exchange response's own audience-scoped RT is never written to the shared credential. - D (wrong-audience bearer): the entra leg ALWAYS pins scope=<audience>/ .default (scope is Entra's only audience carrier); per-server oauth_scopes no longer replaces it (that dropped the audience and leaked a Graph-audience token to the MCP server). oauth_scopes stays a rfc8693-only knob. - F (state-machine divergence): extracted _handle_refresh_failure, called by BOTH oauth_user and oauth_obo — oauth_user behaviour byte-identical (1304 tests green). Fixes: obo cooldown now gated on needs-mint so a force_refresh 401-retry falls through (2063); credential decrypt errors classified not raised (2099); permanent-rejection arms the cooldown as a terminal backstop so it stops re-minting + re-auditing every dispatch (2156); malformed-200 resets the ambiguous streak (2196); misconfig arms the cooldown to dampen the log/SQL flood (2089); server_row threaded from the dispatch caller to drop a hot-path SQL round-trip (2069). - messaging (5993): obo refresh_failed now points at re-login/admin, not a nonexistent per-server consent flow. - single-source (580/9732/217/1830): USER_SCOPED_AUTH_TYPES + is_user_scoped_auth live in mcp_crypto (leaf), re-exported; OBO_GRANT_ PROFILES derives from _OBO_MINT_LEGS and drives oidc validation (was dead-exported). +5 obo regression tests (rotation-survives-exchange-fail, exchange-RT- ignored, terminal cooldown, cooldown fall-through, decrypt classified). Refs #551. |
||
|
Patrick Buckley
|
fd60450700 |
test(mcp): pin oauth_obo mint engine wire shapes and custody semantics
14 cases with exact request-body assertions per the spike-verified shapes: entra default-scope + per-server override, rfc8693 two-call chain with subject-token threading and rotation write-back, cache-hit zero-call fast path, permanent-rejection cache-drop-credential-kept (with the token_revoked audit row), transient cooldown short-circuit, and loud-but-retryable misconfiguration. Refs #551. |