mirror of
https://github.com/turnstonelabs/turnstone.git
synced 2026-08-12 23:12:23 -06:00
2b43b8dd9035ab372202cd9e66c0604989dd2fdc
10 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Patrick Buckley
|
33ace975d2 |
feat(models): default-deny governance and admin UI for per-alias backend auth
Follow-up to the per-alias Entra OBO/app-identity backend auth: the console write path now applies default-deny field classification, the admin shelf gains full backend-auth support, and the session/registry rebind machinery is hardened for config changes landing under live sessions. Console write gate: - Default-deny classification: any non-neutral change to a row that is or becomes dynamic requires admin.mcp plus validation; the provably auth-neutral columns are enumerated (MODEL_AUTH_NEUTRAL_FIELDS) and a live-schema classification test forces every future column to be classified. The derivation is a pure function (_derive_auth_gate) with unit-pinned exclusivity invariants. - Two-tier validation mirroring the MCP oauth_obo validator: the row tier (audience allow-list) runs on every gated write; the posture tier (OIDC configured, token store present) runs on pair changes and on enable-arming. - Pure-disable carve-out: disabling a dynamic row is de-escalation and is never blocked — admin.models suffices and validation is skipped, including for rows with corrupt or skewed stored values. - Capabilities are compared canonically (key order, integral floats), the audience compare normalizes both sides, and staging an audience on a static row is refused on both write twins. - Calibrate writes the capabilities column under an enforced confinement invariant with a compare-and-swap persist. Admin shelf: - Backend-auth section with a per-open constraints fetch (GET /model-definitions/auth-constraints: audience allow-list, grant profile, dynamic modes), datalist audience suggestions, server-defined modes preserved on round-trip, and permission-aware visibility built on cache-skew-safe helpers shared through auth.js. - Refused live-registry swaps surface as an amber registry_warning on the write, delete, reload, and calibrate responses; audit rows carry auth_gated / auth_disarmed markers visible in the audit view. Registry and sessions: - The encryption-key requirement for dynamic auth is enforced inside ModelRegistry.reload() itself — nodes refuse with 503 and the console records coord_registry_error — and reload bumps the generation before the map swap so a racing reader can never pair a stale generation with new maps. - resolve()/resolve_binding() return the generation from inside the registry lock; sessions rebind per send on generation change with atomic client/provider/config commits, fallback-first handling of removed or unconstructable aliases, and judge/limiter resets only when the binding actually changed. - Mint refusals record per-user causes surfaced in the per-turn heartbeat logs; misconfiguration warnings are deduplicated with bounded state. Verification: 10417 tests (99 added on this branch), a 71-scenario browser harness over the real admin shelf, and a live rfc8693 token-exchange e2e run (MCP legs verified end to end; the model-leg scope gap is tracked as #955 under a narrow known-gap signature). Closes #950. |
||
|
metaclassing
|
9adde920d4 |
feat(models): per-alias backend auth via Entra OBO and app identity (#898)
Adds a per-alias `auth_mode` on model definitions so a model backend can authenticate to an Entra-fronted gateway with a per-request minted token instead of one shared static API key, letting the gateway attribute calls to the actual user or to the app as a machine identity. - `static` (default, unchanged) sends the stored `api_key`. - `entra_obo` mints a per-user On-Behalf-Of token for `obo_audience` from the caller's captured refresh credential. - `entra_app` mints an app-identity token via the client-credentials grant, and covers userless turns that OBO cannot. Reuses the existing OBO grant legs, refresh-token rotation CAS, cluster advisory lock and the `mcp_user_tokens` mint-cache, keyed under synthetic `__model_obo__:<audience>` / `__model_app__:<audience>` rows. The token binds at the call site through `client.with_options(api_key=...)` so each SDK emits it on its own auth path rather than through header injection. Migration 068 adds `auth_mode` and `obo_audience`. Both are additive and existing rows default to `static`, so behaviour is unchanged unless an alias opts in. Operator controls: `model.auth_audience_allowlist` is an exact-match allow-list that gates which audiences may be configured and denies all by default, and changing a mode or audience requires `admin.mcp`. `model.auth_fail_closed` decides whether a failed mint may fall back to an explicitly configured static key. A delegated call with no user, or a dynamic alias with no real static key, always refuses. Two changes here apply regardless of whether any alias opts in: - Storage and app state are now wired into the console MCP client manager. This fixes per-user `oauth_user` / `oauth_obo` dispatch for coordinator-hosted sessions, which previously raised `RuntimeError` on first call because `set_app_state` was only ever called on the node. - Unattended watch restores and `--resume` resolve the persisted workstream owner instead of constructing the session under an empty principal. A workstream with no owner is now a permanent refusal rather than an anonymous, auto-approved run. |
||
|
Patrick Buckley
|
9391509e85 |
fix(oidc): trust Entra's graph.microsoft.com userinfo out of the box
Microsoft Entra's discovery document advertises userinfo_endpoint on graph.microsoft.com — a host distinct from the login.microsoftonline.com issuer — so discover_oidc's cross-host guard rejected it and disabled OIDC unless the operator set trusted_endpoint_hosts. Add login.microsoftonline.com to the built-in KNOWN_TRUSTED_OAUTH_ENDPOINT_HOSTS allow-list (mirroring the Google entry) so Azure AD OIDC works with no extra configuration. Surfaced by the live obo integration test. |
||
|
Patrick Buckley
|
9c74673fd4 |
feat(oidc): allow_private_network opt-in for self-hosted IdPs
The SSRF guard on OIDC endpoint URLs hard-refused any hostname resolving to a non-public address, which made it impossible to use a self-hosted IdP (Keycloak, Authentik, Dex) on an internal network — even though the login-flow issuer is operator-configured, i.e. trusted input. Add [oidc] allow_private_network in config.toml (or TURNSTONE_OIDC_ALLOW_PRIVATE_NETWORK), default off. When set, the issuer and its discovered endpoints may resolve to private-range, unique-local, CGNAT, and loopback addresses. Link-local, multicast, unspecified, and reserved ranges stay refused regardless — cloud metadata services live on link-local and no legitimate IdP does. The HTTPS requirement and same-origin endpoint checks are unchanged. The private-address refusal now raises OAuthSSRFPrivateAddressError, and the OIDC wrapper appends the remediation hint to the error message so the failure is self-service. mcp_oauth call sites — where endpoint URLs come from untrusted remote-server metadata — do not get the knob and keep the strict public-address rule. |
||
|
Patrick Buckley
|
3cf87628d2 |
docs(oidc): document TRUSTED_ENDPOINT_HOSTS + fix three-vs-four required drift (cumulative q-1, q-2)
The 8-commit OIDC stack added TURNSTONE_OIDC_TRUSTED_ENDPOINT_HOSTS (operator allow-list for cross-host IdP discovery endpoints) and promoted TURNSTONE_OIDC_REDIRECT_BASE to required, but the docs drifted in two places: q-1 — Troubleshooting > "OIDC not configured" still listed three required env vars. An operator hitting the missing-redirect-base startup error landed on a debugging entry that didn't mention the variable they were missing. Fixed; added a separate troubleshooting entry naming the exact log message produced by initialize_oidc_state when redirect_base is unset. q-2 — TURNSTONE_OIDC_TRUSTED_ENDPOINT_HOSTS was undocumented entirely. Added a row to the env-var table and a new "Cross-host endpoints" section explaining when the knob is needed (Google is the canonical multi-origin IdP, but it's auto-handled; the env var is for any other IdP whose discovery doc legitimately references hosts beyond the issuer's origin). Added a troubleshooting entry pointing at the new section. |
||
|
Patrick Buckley
|
bae4adca12 |
refactor(oidc): quality cleanup (bug-3, q-1/3/4/6/7/9/10/11/12/13)
Eleven small maintenance fixes; no behavior change beyond bug-3.
bug-3: pending.get('audience', audience) couldn't fall back because
pop_oidc_pending_state always returns a dict with the audience key
set verbatim from a non-null TEXT column. Replaced with
pending.get('audience') or audience to cover the empty-string case
defensively. Comment explains the security rationale.
q-1: extract _env_or_cfg_str / _env_or_cfg_bool helpers in oidc.py;
load_oidc_config's six near-identical env-or-config blocks collapse
to one-liners. role_map / trusted_endpoint_hosts / redirect_base
retain bespoke parsing.
q-3: discover_oidc narrows except (httpx.HTTPError, ValueError, KeyError)
with exc_info=True.
q-4: OIDC_STATE_TTL_SECONDS = 300 constant in oidc.py; auth.py imports
and passes it explicitly. Storage signatures keep the literal default
(storage layer doesn't know OIDC TTL semantics).
q-6: hoist runtime imports (OIDCError, OIDCKeyNotFoundError, exchange_code,
fetch_jwks, provision_oidc_user, validate_id_token, build_authorize_url,
generate_pkce_verifier) to module scope in auth.py. The genuine cycle
is only oidc._derive_username -> auth.is_valid_username, kept
function-scoped. test_oidc_handlers.py mock targets repointed to
turnstone.core.auth.X to match the new binding.
q-7: comment + docs explain the 'oidc' vs 'oidc-default' assigned_by
marker distinction.
q-9: OIDCIdentity / OIDCPendingState TypedDicts in storage protocol.
Implementations construct via TypedDict syntax so mypy structurally
verifies all required fields.
q-10: fetch_jwks narrows except (httpx.HTTPError, ValueError); docstring
matches.
q-11: rename generate_pkce_pair -> generate_pkce_verifier; return only
the verifier (build_authorize_url already recomputes the challenge).
q-12: extract _buildOidcRow helper in admin.js so future field additions
go in one place.
q-13: OIDCConfig docstring lists startup-config vs discovery-derived
field groups.
|
||
|
Patrick Buckley
|
52aba17740 |
fix(oidc): require TURNSTONE_OIDC_REDIRECT_BASE; drop Host-header fallback (sec-2)
_build_oidc_redirect_uri previously fell back to the request Host
header when redirect_base was unset. With a permissive reverse proxy
or direct backend access, a spoofed Host minted an authorize URL
pointing to attacker-controlled host — combined with a permissive
IdP redirect_uri allowlist this enables auth-code interception.
There is no production scenario where a Host-derived redirect_uri is
correct, so this fails closed:
- initialize_oidc_state checks redirect_base after discovery succeeds
and disables OIDC (with an explicit error log naming the env var)
if it's empty. Runs before fetch_jwks so a misconfigured deploy
doesn't make a wasted JWKS call.
- _build_oidc_redirect_uri simplifies to f"{redirect_base}/v1/api/auth/oidc/callback".
request parameter dropped; both call sites (handle_oidc_authorize,
handle_oidc_callback) updated.
- docs/oidc.md promotes TURNSTONE_OIDC_REDIRECT_BASE from "Recommended"
to "Required" with the security rationale.
|
||
|
Patrick Buckley
|
62d2a0fe6a |
fix: remove non-auth support from bootstrap wizard (#274)
* fix: remove non-auth support from bootstrap wizard Auth is now mandatory for all deployments. Remove the TURNSTONE_AUTH_ENABLED toggle and make JWT_SECRET and AUTH_TOKEN required in the wizard's system prompt. * fix: remove auth disable support from runtime and infra Remove AuthConfig.enabled field — auth is always on. Drop TURNSTONE_AUTH_ENABLED env var, config toggle, and the check_request bypass. Update compose.yaml, Helm chart, Terraform, docs, and tests to match. * feat: deprecate config tokens, require JWT secret, prefer JWT auth Phase 1 of config-token removal: - load_jwt_secret() now exits with error if no secret is configured (was: silently auto-generated ephemeral secret) - _authenticate_token() logs deprecation warning on config token use - CLI /cluster commands use ServiceTokenManager when JWT secret is set - turnstone-admin tls-list uses ServiceTokenManager when JWT secret is set - Update bootstrap wizard, docker.md, security.md to mark TURNSTONE_AUTH_TOKEN as deprecated and JWT_SECRET as required - Console test fixtures use auth token + headers (auth always enforced) * feat: add service scope for inter-service JWT auth Add "service" to VALID_SCOPES and SCOPE_HIERARCHY. Service tokens bypass require_permission() RBAC checks, replacing the old empty-user-id bypass that config tokens relied on. All ServiceTokenManager instances that need admin access now include "service" in their scopes (console proxy, channel gateway, CLI, admin CLI). Read-only services (collector, notification) unchanged. * feat: phase 2 config token deprecation - SDK doc examples now show API tokens (ts_) instead of config tokens - Remove _get_config_token() from admin CLI (dead code) - Block config token exchange in handle_auth_login — only password and API token login allowed - Update login tests to use password-based auth instead of config token exchange * feat: phase 3 — remove config tokens entirely Complete removal of config-file token authentication: - Delete AuthConfig.tokens, check(), _ROLE_TO_SCOPES, hmac dispatch branch, and config token loading from load_auth_config() - Remove auth_config parameter from _authenticate_token() and check_request() — callers updated throughout - Remove TURNSTONE_AUTH_TOKEN from compose.yaml, Helm charts, Terraform, turnstone.example.toml - Remove --auth-token CLI flags from turnstone, turnstone-admin, and turnstone-console - Simplify console main() — always use ServiceTokenManager (no fallback to static tokens) - Delete config-token-specific tests, rewrite check_request and integration tests to use JWT auth with proper audience claims - Remove all config token references from docs (security.md, docker.md, sdk.md, console.md, architecture.md, bootstrap prompt) * fix: address code review findings - Fix 33 broken tests: add JWT auth to test_api_versioning, test_console_routing_proxy, test_tls_admin, test_tls_manager, test_server_live (jwt_secret + audience-scoped auth headers) - Add TestRequirePermissionServiceScope: 4 tests covering the service scope RBAC bypass path - Remove stale comments referencing config tokens in auth.py and console/server.py - Remove dead proxy_auth_token parameter from console create_app() and static token fallback in _proxy_auth_headers() - Remove TURNSTONE_AUTH_TOKEN from env.py scrub list * fix: address Copilot review — JWT audience, compose require secret - CLI /cluster: add audience=JWT_AUD_CONSOLE to ServiceTokenManager (console validates audience, JWTs without it were rejected) - Admin CLI tls-list: same audience fix - compose.yaml: TURNSTONE_JWT_SECRET now uses :? to fail fast if unset - SDK console: fix default port from 8081 to 8090 * test: add auth enforcement tests for TLS admin endpoints 5 new tests: unauthenticated requests return 401 (list, renew, delete), read-only-scoped requests return 403 (renew, delete). Closes the TLS auth enforcement test gap noted in PROGRESS.md. * fix: address remaining Copilot review feedback - Fix token_source="config" → "test" in TLS test fixtures - Fix AuthResult.token_source docstring to include service origins - Require TURNSTONE_JWT_SECRET in cluster compose profile (:?) - Helm: add auth.jwtSecret + auth.existingSecret values, wire TURNSTONE_JWT_SECRET into secret.yaml and both deployments - Terraform: replace auth_token with jwt_secret variable + secret, remove orphaned auth_token resources and IAM reference - Remove [[auth.tokens]] from security.md config example * fix: address full code review — 10 findings Critical: - Terraform: replace concat(common_env, auth_env) with common_env (auth_env local was removed but still referenced) - Channel gateway: remove hmac static token auth from _check_auth(), use JWT-only validation. Remove --auth-token CLI arg from channel - Rebalancer: add token_manager support so migration requests carry JWT auth (was sending unauthenticated POST to /internal/migrate) Major: - Guard _permissions_to_scopes() against "service" privilege escalation from DB role permissions - Remove dead AuthConfig class, load_auth_config(), and all auth_config parameters from create_app() signatures - Helm: inject JWT secret for both inline and existingSecret paths Minor: - Remove dead auth_token param from ClusterCollector - Remove empty TestLoadAuthConfig class - Short JWT secret now exits instead of warning - Compose: add generation command comment above JWT_SECRET - Clean stale config token references from 6 doc files - Clean stale AUTH_TOKEN reference from bootstrap wizard prompt * fix: remove remaining stale config token references from docs - channels.md: remove --auth-token from options table - oidc.md: remove "config-file tokens still work" claim - security.md: remove config token section, fix JWT secret docs (now required/exits, no ephemeral fallback), remove hmac from ASCII diagram, remove --auth-token reference |
||
|
Patrick Buckley
|
e603a6a7d1 |
fix(oidc): pin redirect URI via TURNSTONE_OIDC_REDIRECT_BASE env var (#74)
* fix(oidc): pin redirect URI via TURNSTONE_OIDC_REDIRECT_BASE env var OIDC redirect_uri was derived from the request Host header, which is unreliable behind reverse proxies. Add TURNSTONE_OIDC_REDIRECT_BASE (env var / config.toml) to pin the externally-reachable origin. Extract _build_oidc_redirect_uri() helper to deduplicate the authorize and callback handlers. Validate redirect_base at load time (must be scheme://host[:port], rejects paths/query strings/invalid schemes). * fix(oidc): reject redirect_base with missing hostname Addresses Copilot review: values like `https://` or `https://:443` passed validation but would produce invalid redirect URIs. * fix(oidc): reject redirect_base with userinfo or invalid port Addresses Copilot round 2: urlparse silently accepts user:pass@host and non-numeric ports. Now explicitly rejects both. |
||
|
Patrick Buckley
|
20df7b3034 |
feat: OIDC SSO authentication with PKCE, auto-provisioning, and role … (#71)
* feat: OIDC SSO authentication with PKCE, auto-provisioning, and role mapping Add OpenID Connect as a fourth authentication method, enabling single sign-on via any OIDC provider (Okta, Azure AD, Google, Keycloak). Opt-in via env vars (TURNSTONE_OIDC_ISSUER, CLIENT_ID, CLIENT_SECRET). Security: - Authorization Code Flow with PKCE (S256) - State/nonce parameters with database-backed pending store (multi-node safe) - JWKS signature validation with async fetch + key rotation retry - Algorithm allowlist from JWKS key (not token header) prevents confusion - Identity matching exclusively by (issuer, sub) — prevents account takeover - password_enabled=false enforced server-side, not just UI - Rate limiting on both authorize and callback endpoints - OIDC users get "!oidc" password sentinel (bcrypt rejects naturally) - ID token validated for iss, aud, exp, nonce Features: - Auto-provisioning with username deduplication on first login - Claim-based role mapping with IdP demotion propagation (revokes stale roles) - "Continue with [Provider]" SSO button on login page - OIDC-only mode hides password form - Setup wizard required before OIDC login (admin bootstrap) Storage: migration 018 (oidc_identities + oidc_pending_states tables), 8 new protocol methods on both SQLite and PostgreSQL backends. 66 new tests (2273 total). * fix: address PR #71 review feedback (18 items) Bugs fixed: - OIDC success redirect now fetches permissions via new /auth/whoami endpoint before completing login (fixes permission-gating in UI) - Remove double decodeURIComponent on oidc_error (URLSearchParams already decodes; extra call throws on stray %) - Authorize rate limiter returns redirect instead of JSON 429 (endpoint reached via browser navigation, not fetch) - Lazy JWKS fetch in callback when startup discovery failed (IdP recovery without restart) - Startup exception handlers now log with exc_info=True - PostgreSQL pop_oidc_pending_state uses DELETE...RETURNING for true atomicity (eliminates TOCTOU) Behavior: - New OIDC users without role mapping get builtin-viewer by default (assigned_by="oidc-default", not revoked by role sync) Documentation fixes: - Role mapping: sync semantics (add + revoke stale), not "additive only" - PASSWORD_ENABLED=false blocks ALL password logins including admin - Algorithm: asymmetric allowlist, not per-key derivation - PlantUML diagram updated for role revocation API spec fixes: - Removed error_codes=[302] from callback (302 is success redirect) - Added /auth/whoami to both server + console specs - Regenerated TypeScript SDK OpenAPI snapshots (23 + 51 paths) * fix: address PR #71 round 2 review feedback (10 items) Rate limiting: - Authorize endpoint now calls record() after check() so the rate limiter actually counts attempts (was a no-op before) OIDC resilience: - Split startup try/except: discovery failure disables OIDC, JWKS prefetch failure leaves OIDC enabled for lazy retry on first login - JWKS unavailable message changed to "temporarily unavailable" (was misleadingly "not configured") - create_oidc_pending_state raises on collision instead of OR IGNORE (prevents silent insert drop on state collision) - SQLite pop_oidc_pending_state uses BEGIN IMMEDIATE for write lock (eliminates TOCTOU race) Frontend: - OIDC error display deferred 300ms so showLogin()'s async status fetch doesn't clear it via _switchMode → _clearError API spec: - OIDC authorize/callback endpoints now declare response_code=302 - Added AuthWhoamiResponse Pydantic model for /auth/whoami - Regenerated TypeScript SDK OpenAPI snapshots Documentation: - Diagram: JWKS "cached at startup, refreshed on-demand" (was "hourly") - Added TODO(tech-debt) comments on Host header redirect_uri sites |