turnstone

ewilliams28/turnstone

Fork 0

mirror of https://github.com/turnstonelabs/turnstone.git synced 2026-08-12 23:12:23 -06:00

Commit Graph

Author	SHA1	Message	Date
Patrick Buckley	f4fd7e1f67	fix(security): classify outbound addresses by what they reach (GHSA-wm4f-79pw-pfr9) (#1003 ) * fix(security): classify outbound addresses by what they reach (GHSA-wm4f-79pw-pfr9) Five guards screened outbound URLs and each hand-rolled its own address normalization and policy tests, so each had a different hole. An IPv6 transition address carries an IPv4 destination in its low bits and `ipaddress` classifies the wrapper, not the destination: 64:ff9b::a9fe:a9fe reports is_global because 64:ff9b::/96 is global unicast, while a NAT64 gateway routes it to the cloud metadata endpoint. CGNAT (100.64.0.0/10) is neither is_private nor is_global, so a denylist built on is_private missed it with no gateway involved at all. Add turnstone/core/ip_classify.py as the single classifier. One function returns exactly one policy lane — PUBLIC, PRIVATE (operator-approvable) or NEVER — and every guard branches on the lane rather than re-deriving it. Two overlapping booleans would make a verdict depend on which one a caller tested first; several addresses are simultaneously globally routable and metadata-reaching. - Decode transition addresses per RFC 6052 §2.2 (NAT64 well-known and local-use prefixes, 6to4, Teredo, IPv4-mapped, IPv4-compatible) and judge them by the IPv4 they reach. The local-use prefix does not say which layout its gateway uses, so every length it can carry is decoded and the worst result classified. - Share hostname resolution too. The five copies had already drifted on which failures they caught, and getaddrinfo raises UnicodeError — not an OSError — from the IDNA encoder. - Resolution failure is a refusal, not a pass: the fetch resolves again, so an authority answering the guard with SERVFAIL and the fetch with an internal address would otherwise switch the guard off for that hop. - Screen every redirect hop in every mode. allow_private_origin widens which lanes are acceptable rather than turning screening off, and the permission is revoked after any hop that is not wholly private. - Cleartext http is allowed only for a hostname that RESOLVES to loopback. .localhost is ordinary DNS, and trusting the name put an OIDC token exchange on the wire in the clear. - Screen doctor and console-probe URLs through the classifier. Both used a host.startswith("169.254.") string test that never resolved, so any DNS name pointing at the metadata service passed and its body was returned to the model. - Add known vendor metadata prefixes the stdlib does not flag, and place deprecated IPv6 site-local outside the public lane. The operator's private-network opt-in still admits the whole home lab, including IPv6 loopback, CGNAT and split-horizon hosts. Metadata, link-local, multicast, unspecified and reserved addresses stay refused regardless of the opt-in, including as a redirect target from an approved private origin — the settings help and docs now say so. Reported by @tonghuaroot. fix(security): close Azure/Oracle metadata gap and restore dual-stack origins Review follow-ups on the address-classification rework. Azure's host-agent endpoint (168.63.129.16) and Oracle Cloud's metadata endpoint (192.0.0.192) sit in ordinary unicast space, so the stdlib reported them as globally routable and both classified PUBLIC — reachable with no opt-in at all, a worse position than the RFC 1918 host beside them, and directly contradicting the "metadata stays refused even with the opt-in" guarantee the settings help and docs now advertise. Both join the shared vendor list. Revoking the private-hop permission on the ORIGIN hop broke the case `_screen_tool_url` deliberately admits: a dual-stack or split-horizon home-lab host answering with both a LAN and a public record was approved, then refused on its own `302 /login` — one hop was all it ever got. Track the approved HOST instead, so redirects that stay on it remain covered while a redirect to any other private host is still refused once the chain is no longer wholly private. Also: - Try several registry candidates for the collector-scope probe instead of abandoning it when the first is unresolvable, which also stopped a healthy registry from logging as malformed. - Bound the probe's name resolution with an explicit timeout matching the 2s the httpx connect deadline used to provide; it runs before the console lifespan yields and getaddrinfo has no timeout of its own. - Route doctor and the console probe through `web.screen_url` rather than keeping a third and fourth copy of parse/resolve/classify/fold, which had already diverged on default port and empty-hostname wording. An empty hostname no longer reports as a cloud-metadata refusal. - Give `screen_url` a scheme-aware default port. - Stop doubling the word "hostname" in the OAuth resolution refusal. - Correct the `_screen_tool_url` docstring: it described `private_origin` as requiring every record to be private, which the mixed-record decision reversed, and `private_block` as a property of a refusal when it reports the lane on the success path too. - Make the preview tests' screening stub opt-in rather than autouse — as a module-wide fixture it also stubbed the tests whose subject IS the screen, so one of them would have passed even if screening refused everything. Verified the module now passes with all name resolution blocked. * fix(security): refuse mixed-record private origins instead of exempting them The previous commit let an approved private origin redirect to itself by exempting its hostname from the chain-wide revocation. That exemption was wrong three ways: it was captured once and never cleared, so a public hop could steer the fetcher back into the approved host at a path of its choosing — reopening the private -> public -> private bypass; it was re-entrant across same-host redirects with fresh DNS each time, so a self-redirecting host could walk arbitrary internal addresses; and it matched on bare hostname, so it spanned every port on the approved box. All three were reproduced against the parent commit, which refuses them. Delete the exemption rather than repair it. The case it existed for — a dual-stack host answering with both a LAN and a public record — is now refused where it is actually decidable, in `_screen_tool_url`, with the remedy in the message: point the tool at the LAN address directly. A granted chain therefore always starts wholly private, so the fetch guard needs no notion of an approved host and stays one unconditional rule. That the accommodation could not be expressed safely in the guard is the signal: the connection may land on either record, so approving such a host never described where the fetch would go. Also from the same review: - Walk the whole service registry for a collector-scope probe candidate instead of the first three, and split the outcome into three log lines, so entries that are merely unreachable stop raising the malformed-registry alarm and skipping the boot check cluster-wide. - Stop the candidate walk on a resolver timeout. `asyncio.timeout` bounds the await, not the work, so continuing left one parked thread per timed-out candidate on the shared executor. - Move the metadata-hostname denylist into `ip_classify` and enforce it in `screen_url`, so doctor and the console probe inherit it instead of each keeping a copy. - Drop the scheme-aware default port: a numeric service does not change which addresses resolution returns, and classification reads only those. `parsed.port` is still touched so an out-of-range value refuses. - Correct the vendor-metadata comment, which generalized a claim true of Azure's and Oracle's addresses to Alibaba's CGNAT one. - Rename a test class that was still named for the rule it no longer tests.	2026-08-11 02:18:03 -07:00
Patrick Buckley	33ace975d2	feat(models): default-deny governance and admin UI for per-alias backend auth Follow-up to the per-alias Entra OBO/app-identity backend auth: the console write path now applies default-deny field classification, the admin shelf gains full backend-auth support, and the session/registry rebind machinery is hardened for config changes landing under live sessions. Console write gate: - Default-deny classification: any non-neutral change to a row that is or becomes dynamic requires admin.mcp plus validation; the provably auth-neutral columns are enumerated (MODEL_AUTH_NEUTRAL_FIELDS) and a live-schema classification test forces every future column to be classified. The derivation is a pure function (_derive_auth_gate) with unit-pinned exclusivity invariants. - Two-tier validation mirroring the MCP oauth_obo validator: the row tier (audience allow-list) runs on every gated write; the posture tier (OIDC configured, token store present) runs on pair changes and on enable-arming. - Pure-disable carve-out: disabling a dynamic row is de-escalation and is never blocked — admin.models suffices and validation is skipped, including for rows with corrupt or skewed stored values. - Capabilities are compared canonically (key order, integral floats), the audience compare normalizes both sides, and staging an audience on a static row is refused on both write twins. - Calibrate writes the capabilities column under an enforced confinement invariant with a compare-and-swap persist. Admin shelf: - Backend-auth section with a per-open constraints fetch (GET /model-definitions/auth-constraints: audience allow-list, grant profile, dynamic modes), datalist audience suggestions, server-defined modes preserved on round-trip, and permission-aware visibility built on cache-skew-safe helpers shared through auth.js. - Refused live-registry swaps surface as an amber registry_warning on the write, delete, reload, and calibrate responses; audit rows carry auth_gated / auth_disarmed markers visible in the audit view. Registry and sessions: - The encryption-key requirement for dynamic auth is enforced inside ModelRegistry.reload() itself — nodes refuse with 503 and the console records coord_registry_error — and reload bumps the generation before the map swap so a racing reader can never pair a stale generation with new maps. - resolve()/resolve_binding() return the generation from inside the registry lock; sessions rebind per send on generation change with atomic client/provider/config commits, fallback-first handling of removed or unconstructable aliases, and judge/limiter resets only when the binding actually changed. - Mint refusals record per-user causes surfaced in the per-turn heartbeat logs; misconfiguration warnings are deduplicated with bounded state. Verification: 10417 tests (99 added on this branch), a 71-scenario browser harness over the real admin shelf, and a live rfc8693 token-exchange e2e run (MCP legs verified end to end; the model-leg scope gap is tracked as #955 under a narrow known-gap signature). Closes #950.	2026-08-03 20:11:28 -07:00
Patrick Buckley	c0be383f99	refactor(doctor): replace turnstone-bootstrap with turnstone-doctor (#718 ) * refactor(doctor): replace turnstone-bootstrap with turnstone-doctor turnstone-bootstrap was an LLM setup wizard for Day-0; run.sh now owns install. Repurpose its LLM/conversation plumbing into turnstone-doctor — a diagnose-only tool for a running cluster. - Preflight detects the install kind (docker-compose/systemd/pip/source) from config.toml + TURNSTONE_* env, with secret redaction. - Self-configuring brain resolves the cluster's own model from config/env/storage read-only (no migrations, no create_all), falling back to interactive selection; the attempt itself is the LLM-backend health check. - Deterministic version check: installed version, cluster drift via the console's authoritative /health, and latest upstream stable/experimental (offline-safe). - Read-only diagnostic tools (read_file, compose/systemd/journal, http_health, check_llm_backend, node_health, finish) behind one secret-scrubbing chokepoint; no generic shell, so read-only is structural. - node_health reaches a node the right way for the detected install kind (exec-into-container for compose, direct HTTP otherwise), overridable per node for mixed clusters. - mTLS-aware: forwards [database] SSL params and reports node-mesh mTLS instead of mislabelling healthy nodes "unreachable". init_storage gains a backward-compatible create_tables override for read-only opens. Entry point turnstone-bootstrap -> turnstone-doctor; README/QUICKSTART/ architecture/docker docs, the bundled compose header, run.sh, and the CI smoke updated. CHANGELOG deferred. * fix(doctor): address Copilot + CodeQL review findings on #718 Validated all seven review findings (none false positives) and fixed: - check_llm_backend now applies the same scheme / metadata-host guard as http_health (extracted to _assert_safe_http_url), so a model-supplied base_url can't be steered at the cloud metadata endpoint or a file:// URL. - node_health no longer double-appends the default port when the operator passes host:port (regression: 10.0.0.5:8081 -> http://10.0.0.5:8081:8080). - node_health install_type enum uses "git-source" to match the label the rest of the module and the prompt/report show the model (a schema-strict provider would otherwise reject the value the model is told to use). - _read_api_creds takes base_url + api_key as a unit from the first config source that defines either field, then env-fills, instead of splicing the two across different config files into a pair that exists in no real config. - _mask_secrets masks assignment-shaped content inside comment lines, so a commented-out real secret can't leak through read_file / the report; prose comments (no KEY=value shape) still pass through untouched. - drop the mixed import styles CodeQL flagged in doctor.py and test_doctor.py. Adds 5 tests; ruff + mypy clean; full doctor suite passes (129).	2026-06-26 04:57:20 -07:00

Author

SHA1

Message

Date

Patrick Buckley

f4fd7e1f67

fix(security): classify outbound addresses by what they reach (GHSA-wm4f-79pw-pfr9) (#1003 )

* fix(security): classify outbound addresses by what they reach (GHSA-wm4f-79pw-pfr9)

Five guards screened outbound URLs and each hand-rolled its own address
normalization and policy tests, so each had a different hole. An IPv6
transition address carries an IPv4 destination in its low bits and
`ipaddress` classifies the wrapper, not the destination: 64:ff9b::a9fe:a9fe
reports is_global because 64:ff9b::/96 is global unicast, while a NAT64
gateway routes it to the cloud metadata endpoint. CGNAT (100.64.0.0/10) is
neither is_private nor is_global, so a denylist built on is_private missed
it with no gateway involved at all.

Add turnstone/core/ip_classify.py as the single classifier. One function
returns exactly one policy lane — PUBLIC, PRIVATE (operator-approvable) or
NEVER — and every guard branches on the lane rather than re-deriving it.
Two overlapping booleans would make a verdict depend on which one a caller
tested first; several addresses are simultaneously globally routable and
metadata-reaching.

- Decode transition addresses per RFC 6052 §2.2 (NAT64 well-known and
  local-use prefixes, 6to4, Teredo, IPv4-mapped, IPv4-compatible) and judge
  them by the IPv4 they reach. The local-use prefix does not say which
  layout its gateway uses, so every length it can carry is decoded and the
  worst result classified.
- Share hostname resolution too. The five copies had already drifted on
  which failures they caught, and getaddrinfo raises UnicodeError — not an
  OSError — from the IDNA encoder.
- Resolution failure is a refusal, not a pass: the fetch resolves again, so
  an authority answering the guard with SERVFAIL and the fetch with an
  internal address would otherwise switch the guard off for that hop.
- Screen every redirect hop in every mode. allow_private_origin widens which
  lanes are acceptable rather than turning screening off, and the permission
  is revoked after any hop that is not wholly private.
- Cleartext http is allowed only for a hostname that RESOLVES to loopback.
  *.localhost is ordinary DNS, and trusting the name put an OIDC token
  exchange on the wire in the clear.
- Screen doctor and console-probe URLs through the classifier. Both used a
  host.startswith("169.254.") string test that never resolved, so any DNS
  name pointing at the metadata service passed and its body was returned to
  the model.
- Add known vendor metadata prefixes the stdlib does not flag, and place
  deprecated IPv6 site-local outside the public lane.

The operator's private-network opt-in still admits the whole home lab,
including IPv6 loopback, CGNAT and split-horizon hosts. Metadata,
link-local, multicast, unspecified and reserved addresses stay refused
regardless of the opt-in, including as a redirect target from an approved
private origin — the settings help and docs now say so.

Reported by @tonghuaroot.

* fix(security): close Azure/Oracle metadata gap and restore dual-stack origins

Review follow-ups on the address-classification rework.

Azure's host-agent endpoint (168.63.129.16) and Oracle Cloud's metadata
endpoint (192.0.0.192) sit in ordinary unicast space, so the stdlib reported
them as globally routable and both classified PUBLIC — reachable with no
opt-in at all, a worse position than the RFC 1918 host beside them, and
directly contradicting the "metadata stays refused even with the opt-in"
guarantee the settings help and docs now advertise. Both join the shared
vendor list.

Revoking the private-hop permission on the ORIGIN hop broke the case
`_screen_tool_url` deliberately admits: a dual-stack or split-horizon
home-lab host answering with both a LAN and a public record was approved,
then refused on its own `302 /login` — one hop was all it ever got. Track
the approved HOST instead, so redirects that stay on it remain covered while
a redirect to any other private host is still refused once the chain is no
longer wholly private.

Also:

- Try several registry candidates for the collector-scope probe instead of
  abandoning it when the first is unresolvable, which also stopped a healthy
  registry from logging as malformed.
- Bound the probe's name resolution with an explicit timeout matching the
  2s the httpx connect deadline used to provide; it runs before the console
  lifespan yields and getaddrinfo has no timeout of its own.
- Route doctor and the console probe through `web.screen_url` rather than
  keeping a third and fourth copy of parse/resolve/classify/fold, which had
  already diverged on default port and empty-hostname wording. An empty
  hostname no longer reports as a cloud-metadata refusal.
- Give `screen_url` a scheme-aware default port.
- Stop doubling the word "hostname" in the OAuth resolution refusal.
- Correct the `_screen_tool_url` docstring: it described `private_origin` as
  requiring every record to be private, which the mixed-record decision
  reversed, and `private_block` as a property of a refusal when it reports
  the lane on the success path too.
- Make the preview tests' screening stub opt-in rather than autouse — as a
  module-wide fixture it also stubbed the tests whose subject IS the screen,
  so one of them would have passed even if screening refused everything.
  Verified the module now passes with all name resolution blocked.

* fix(security): refuse mixed-record private origins instead of exempting them

The previous commit let an approved private origin redirect to itself by
exempting its hostname from the chain-wide revocation. That exemption was
wrong three ways: it was captured once and never cleared, so a public hop
could steer the fetcher back into the approved host at a path of its
choosing — reopening the private -> public -> private bypass; it was
re-entrant across same-host redirects with fresh DNS each time, so a
self-redirecting host could walk arbitrary internal addresses; and it
matched on bare hostname, so it spanned every port on the approved box.
All three were reproduced against the parent commit, which refuses them.

Delete the exemption rather than repair it. The case it existed for — a
dual-stack host answering with both a LAN and a public record — is now
refused where it is actually decidable, in `_screen_tool_url`, with the
remedy in the message: point the tool at the LAN address directly. A
granted chain therefore always starts wholly private, so the fetch guard
needs no notion of an approved host and stays one unconditional rule.

That the accommodation could not be expressed safely in the guard is the
signal: the connection may land on either record, so approving such a host
never described where the fetch would go.

Also from the same review:

- Walk the whole service registry for a collector-scope probe candidate
  instead of the first three, and split the outcome into three log lines,
  so entries that are merely unreachable stop raising the malformed-registry
  alarm and skipping the boot check cluster-wide.
- Stop the candidate walk on a resolver timeout. `asyncio.timeout` bounds
  the await, not the work, so continuing left one parked thread per timed-out
  candidate on the shared executor.
- Move the metadata-hostname denylist into `ip_classify` and enforce it in
  `screen_url`, so doctor and the console probe inherit it instead of each
  keeping a copy.
- Drop the scheme-aware default port: a numeric service does not change
  which addresses resolution returns, and classification reads only those.
  `parsed.port` is still touched so an out-of-range value refuses.
- Correct the vendor-metadata comment, which generalized a claim true of
  Azure's and Oracle's addresses to Alibaba's CGNAT one.
- Rename a test class that was still named for the rule it no longer tests.

2026-08-11 02:18:03 -07:00

Patrick Buckley

33ace975d2

feat(models): default-deny governance and admin UI for per-alias backend auth

Follow-up to the per-alias Entra OBO/app-identity backend auth: the
console write path now applies default-deny field classification, the
admin shelf gains full backend-auth support, and the session/registry
rebind machinery is hardened for config changes landing under live
sessions.

Console write gate:
- Default-deny classification: any non-neutral change to a row that is
  or becomes dynamic requires admin.mcp plus validation; the provably
  auth-neutral columns are enumerated (MODEL_AUTH_NEUTRAL_FIELDS) and a
  live-schema classification test forces every future column to be
  classified. The derivation is a pure function (_derive_auth_gate)
  with unit-pinned exclusivity invariants.
- Two-tier validation mirroring the MCP oauth_obo validator: the row
  tier (audience allow-list) runs on every gated write; the posture
  tier (OIDC configured, token store present) runs on pair changes and
  on enable-arming.
- Pure-disable carve-out: disabling a dynamic row is de-escalation and
  is never blocked — admin.models suffices and validation is skipped,
  including for rows with corrupt or skewed stored values.
- Capabilities are compared canonically (key order, integral floats),
  the audience compare normalizes both sides, and staging an audience
  on a static row is refused on both write twins.
- Calibrate writes the capabilities column under an enforced
  confinement invariant with a compare-and-swap persist.

Admin shelf:
- Backend-auth section with a per-open constraints fetch
  (GET /model-definitions/auth-constraints: audience allow-list, grant
  profile, dynamic modes), datalist audience suggestions,
  server-defined modes preserved on round-trip, and permission-aware
  visibility built on cache-skew-safe helpers shared through auth.js.
- Refused live-registry swaps surface as an amber registry_warning on
  the write, delete, reload, and calibrate responses; audit rows carry
  auth_gated / auth_disarmed markers visible in the audit view.

Registry and sessions:
- The encryption-key requirement for dynamic auth is enforced inside
  ModelRegistry.reload() itself — nodes refuse with 503 and the
  console records coord_registry_error — and reload bumps the
  generation before the map swap so a racing reader can never pair a
  stale generation with new maps.
- resolve()/resolve_binding() return the generation from inside the
  registry lock; sessions rebind per send on generation change with
  atomic client/provider/config commits, fallback-first handling of
  removed or unconstructable aliases, and judge/limiter resets only
  when the binding actually changed.
- Mint refusals record per-user causes surfaced in the per-turn
  heartbeat logs; misconfiguration warnings are deduplicated with
  bounded state.

Verification: 10417 tests (99 added on this branch), a 71-scenario
browser harness over the real admin shelf, and a live rfc8693
token-exchange e2e run (MCP legs verified end to end; the model-leg
scope gap is tracked as #955 under a narrow known-gap signature).

Closes #950.

2026-08-03 20:11:28 -07:00

Patrick Buckley

c0be383f99

refactor(doctor): replace turnstone-bootstrap with turnstone-doctor (#718 )

* refactor(doctor): replace turnstone-bootstrap with turnstone-doctor

turnstone-bootstrap was an LLM setup wizard for Day-0; run.sh now owns install.
Repurpose its LLM/conversation plumbing into turnstone-doctor — a diagnose-only
tool for a running cluster.

- Preflight detects the install kind (docker-compose/systemd/pip/source) from
  config.toml + TURNSTONE_* env, with secret redaction.
- Self-configuring brain resolves the cluster's own model from config/env/storage
  read-only (no migrations, no create_all), falling back to interactive
  selection; the attempt itself is the LLM-backend health check.
- Deterministic version check: installed version, cluster drift via the console's
  authoritative /health, and latest upstream stable/experimental (offline-safe).
- Read-only diagnostic tools (read_file, compose/systemd/journal, http_health,
  check_llm_backend, node_health, finish) behind one secret-scrubbing chokepoint;
  no generic shell, so read-only is structural.
- node_health reaches a node the right way for the detected install kind
  (exec-into-container for compose, direct HTTP otherwise), overridable per node
  for mixed clusters.
- mTLS-aware: forwards [database] SSL params and reports node-mesh mTLS instead of
  mislabelling healthy nodes "unreachable".

init_storage gains a backward-compatible create_tables override for read-only
opens. Entry point turnstone-bootstrap -> turnstone-doctor; README/QUICKSTART/
architecture/docker docs, the bundled compose header, run.sh, and the CI smoke
updated. CHANGELOG deferred.

* fix(doctor): address Copilot + CodeQL review findings on #718

Validated all seven review findings (none false positives) and fixed:

- check_llm_backend now applies the same scheme / metadata-host guard as
  http_health (extracted to _assert_safe_http_url), so a model-supplied
  base_url can't be steered at the cloud metadata endpoint or a file:// URL.
- node_health no longer double-appends the default port when the operator
  passes host:port (regression: 10.0.0.5:8081 -> http://10.0.0.5:8081:8080).
- node_health install_type enum uses "git-source" to match the label the
  rest of the module and the prompt/report show the model (a schema-strict
  provider would otherwise reject the value the model is told to use).
- _read_api_creds takes base_url + api_key as a unit from the first config
  source that defines either field, then env-fills, instead of splicing the
  two across different config files into a pair that exists in no real config.
- _mask_secrets masks assignment-shaped content inside comment lines, so a
  commented-out real secret can't leak through read_file / the report; prose
  comments (no KEY=value shape) still pass through untouched.
- drop the mixed import styles CodeQL flagged in doctor.py and test_doctor.py.

Adds 5 tests; ruff + mypy clean; full doctor suite passes (129).

2026-06-26 04:57:20 -07:00

3 Commits