mirror of
https://github.com/turnstonelabs/turnstone.git
synced 2026-08-24 12:54:48 -06:00
9826ea15c5
* feat(coordinator): phase 7 — governance + skill metadata + cross-cutting invariants
Combines three stacked sub-PRs into a single coordinator phase-7
shipment against the phase-7 plan doc. The sub-PR structure (0 / A /
B) preserved on individual branches for reviewer drill-down; this
branch is the one reviewers should merge.
## Sub-PR 0 — service-auth boundary invariants
Shared helpers and contracts that lock the console ↔ node service-auth
boundary so later authz surfaces use them by construction.
- ``_effective_user_filter(request)`` in both ``turnstone.console.server``
and ``turnstone.server`` with a shared ``DENY_EMPTY_SUB`` sentinel
on ``turnstone.core.auth``. Three-way return — admin/service
bypass, scoped caller uid, or fail-closed sentinel on blank sub.
Four callsite migrations (``_coordinator_rows``,
``coordinator_children``, ``coordinator_metrics``,
``cluster_ws_live_bulk``).
- ``StorageBackend`` class docstring codifies the tenancy contract
(every list/count/aggregate method must accept ``user_id: str |
None = None`` and push ``WHERE user_id = :user_id`` into SQL) and
the ``_mapping`` row-access contract. New
``turnstone.testing.row_contract`` ships ``assert_row_like()``.
- ``_verify_collector_service_scope`` probes an upstream node at boot
with ``expected_node_id=_scope-probe_``; a 409 proves the scope
gate was passed, a 403/401 sets ``collector_scope_error`` and
causes ``cluster_snapshot`` / ``cluster_events_sse`` to return 503
with a remediation hint. Probe URL allowlist rejects non-http(s)
schemes and 169.254.0.0/16 hosts.
- 4xx log-level floor on ``_NodeDashboardCache.get``,
``_fetch_live_block``, and ``_proxy_sse`` — dotted-hierarchy
prefixes with bounded body previews. ``_bounded_body_preview`` and
``_bounded_stream_preview`` strip control chars.
## Sub-PR A — coordinator governance core
Mid-session governance surface for coordinator workstreams.
- **Trusted-session mode.** New ``coordinator.trust.send``
permission (migration 042). ``ChatSession.set_trust_send`` /
``revoke_tools`` methods with a ``_governance_lock``. ``POST
/v1/api/coordinator/{ws_id}/trust {send: bool}`` double-gated on
``admin.coordinator`` AND ``coordinator.trust.send`` with
``allow_service_bypass=False`` so service tokens can't escalate.
``_prepare_send_to_workstream`` auto-approves sends whose target is
in the coordinator's own subtree; foreign ws_ids still require
approval. ``_is_own_subtree`` checks both ``parent_ws_id`` AND
``user_id`` to defend against cross-tenant row corruption.
- **Audit-layer credential redaction.** ``record_audit`` walks
``detail`` (dicts, lists, tuples, sets, frozensets; keys too)
and routes every string through ``redact_credentials`` + a C0
control-char scrub. New kw-only ``raw_detail=True`` opt-out.
``_has_any_string`` fast-path. Audit action registry extended
with the four new governance sub-prefixes.
- **Mid-session revocation + cascading stop.** ``POST
/v1/api/coordinator/{ws_id}/restrict {revoke: [...]}`` caps 256
entries / 128 chars; ``_prepare_tool`` short-circuits with a
tool-error. ``POST /v1/api/coordinator/{ws_id}/stop_cascade``
cancels the coord's in-flight generation then dispatches
``cancel_workstream`` for every direct child in parallel via
``asyncio.gather`` bounded by ``Semaphore(16)``. Per-child
outcomes split into ``cancelled`` / ``failed`` / ``skipped``
(404 = already-gone rather than dispatch-broken). Both endpoints
apply ``allow_service_bypass=False`` on the admin gate.
- **Shared plumbing.** ``_resolve_coord_session`` helper collapses
the handler prelude three endpoints shared. ``_emit_coord_audit``
wraps ``record_audit`` in a dedicated ``ThreadPoolExecutor``
(``app.state.audit_executor``) so audit bursts don't starve cancel
dispatches. ``_require_json_object`` guards body parsing so non-
object JSON returns 400 instead of 500.
## Sub-PR B — skill metadata governance
- **Description validator (migration 043).** ``prompt_templates``
rows now require a non-empty ``description``. Existing empty rows
get backfilled with a ``"Skill: <name>"`` placeholder on upgrade.
The installer (``admin_skill_discover``) and MCP prompt sync both
synthesise a placeholder when the upstream description is blank
so non-admin write paths satisfy the invariant.
- **Skill kind classifier (migration 044).** New
``prompt_templates.kind`` column (``interactive`` / ``coordinator``
/ ``any``; defaults to ``any``). New
``turnstone.core.skill_kind.SkillKind`` StrEnum is the single
source of truth; Pydantic schemas type ``kind`` as ``SkillKind``
(OpenAPI advertises the enum) and the handler validator catches
the ValueError. ``list_skills_filtered`` gains a
``kinds: list[str] | None = None`` SQL filter.
``CoordinatorClient.list_skills`` defaults to
``kinds=["coordinator", "any"]`` so interactive-only skills are
hidden from the orchestrator.
- **``scan_status`` → ``risk_level`` rename (migration 045).**
Lossless column rename to align with ``IntentVerdict.risk_level``
terminology. Swept storage (both backends + schema + protocol),
handlers, API schemas, tool JSON, generated OpenAPI specs,
TypeScript SDK types, frontend (``governance.js``), tests, and
English prose in ``docs/judge.md`` + ``docs/tools.md``. The
user-facing on-load warning now reads ``has risk level:
{risk_tier}``. Tool JSON's ``risk_level`` enum corrected to the
scanner's actual taxonomy (``safe / low / medium / high /
critical``; was the never-shipped ``clean / flagged / unscanned /
pending``). Historical migration 021 left untouched.
## Migrations
042 (``coordinator.trust.send`` perm — PR A)
043 (description backfill — PR B)
044 (``kind`` column add — PR B)
045 (``scan_status`` → ``risk_level`` rename — PR B)
All four use position-anchored permission strings / host-side
parse-filter-rejoin on downgrade where SQL ``REPLACE`` could
corrupt prefix-overlapping values.
## Verification
- ``ruff check turnstone tests`` clean.
- ``mypy turnstone`` clean on 165 source files.
- ``pytest -m "not live"``: 4431 passed (+85 over the phase-6
baseline). Includes +32 tests in ``tests/test_service_auth_boundary.py``
and +38 in ``tests/test_coordinator_governance.py``; shared fixtures
extracted to ``tests/_coord_test_helpers.py``.
- Generated OpenAPI JSON (``sdk/typescript/openapi-{console,server}.json``)
regenerated via ``sdk/typescript/scripts/generate-types.py``; zero
``scan_status`` occurrences remaining outside the historical
migration 021 and the rename migration 045.
## Security reviews
Both reviews flagged by the phase-7 plan (items 1 + 5, plus 0a's
refuse-to-serve gate) ran through the multi-stage ``/review``
pipeline twice per sub-PR; all confirmed findings landed in-branch.
* fixup(phase-7): CI lint + PR #383 review fixups
Addresses the lint CI failure (ruff format) plus 12 findings from the
two automated PR reviewers.
Copilot:
- ``_sqlite.list_installed_skill_urls`` / ``_postgresql.list_installed_skill_urls``
used positional row indexing (``r[0]``/``r[1]``/``r[2]``) while this
same PR's ``StorageBackend`` class docstring forbids it. Switched
both to ``r._mapping["..."]`` access.
- ``list_skills.json`` previously advertised ``risk_level=""`` as a
filter for unscanned skills, but the implementation treats empty
strings as "no filter". Clarified the tool description to say
omit the filter entirely to include unscanned rows, and added an
explicit ``enum`` on the parameter restricting it to the scanner
tiers. ``_prepare_list_skills`` keeps the ``strip() or None``
normalisation — unscanned filtering now has an unambiguous contract.
- ``test_storage_skills_filtered.test_risk_level_filter`` used the
legacy ``clean`` / ``flagged`` values from the pre-rename column.
Rewritten with the scanner's actual taxonomy (``safe`` / ``high``).
github-code-quality (CodeQL):
- ``test_deny_sentinel_is_singleton`` previously asserted
``cs.DENY_EMPTY_SUB is cs.DENY_EMPTY_SUB`` — an identical-expression
comparison. Rewritten as two separate ``from ... import ... as`` aliases
(``FIRST_READ`` / ``SECOND_READ``) so the identity check is between
distinct bindings.
- ``test_restrict_empty_revoke_is_noop_but_audits`` unpacked ``state``
without using it. Renamed to ``_state``.
- Mixed import styles in ``test_service_auth_boundary.py`` — the
file previously used both ``import turnstone.console.server as cs``
and ``from turnstone.console.server import ...`` for the same
module (same story for ``turnstone.core.auth`` and
``turnstone.server``). Consolidated to the ``from X import Y`` style
used elsewhere in the file; the ``_fetch_live_block`` test now
patches via pytest's ``monkeypatch`` fixture instead of a manual
rebind through a module alias.
CI:
- ``ruff format`` reformatted one line in
``tests/test_coordinator_endpoints.py``.
Verification: ruff check + mypy clean (166 files); 4459 non-live
pytest pass.
* fix(tests): swap asyncio marker for anyio in service-auth boundary tests
PR #383 CI caught that the 13 ``@pytest.mark.asyncio`` decorators I
added in ``test_service_auth_boundary.py`` are an off-convention
choice — the rest of the repo uses ``@pytest.mark.anyio`` (148 sites
vs my 13). The CI environment pulls in ``anyio`` but not
``pytest-asyncio``, so every async test in this one file was failing
with "async def functions are not natively supported". It passed
locally by accident — my dev venv happens to have pytest-asyncio
installed ambiently.
Swapped all 13 marker sites to ``@pytest.mark.anyio``. No functional
change; the tests run under the same default asyncio backend anyio
provides.
Verification: ruff + mypy clean (166 files); 4459 non-live pytest
pass.
211 lines
6.7 KiB
Python
211 lines
6.7 KiB
Python
"""Tests for turnstone.core.audit."""
|
|
|
|
import json
|
|
|
|
import pytest
|
|
|
|
from turnstone.core.audit import record_audit
|
|
from turnstone.core.storage._sqlite import SQLiteBackend
|
|
|
|
|
|
@pytest.fixture
|
|
def storage(tmp_path):
|
|
path = str(tmp_path / "test.db")
|
|
backend = SQLiteBackend(path)
|
|
yield backend
|
|
backend.close()
|
|
|
|
|
|
def test_record_audit_basic(storage):
|
|
record_audit(
|
|
storage, "user-1", "user.create", "user", "u123", {"username": "alice"}, "127.0.0.1"
|
|
)
|
|
events = storage.list_audit_events()
|
|
assert len(events) == 1
|
|
ev = events[0]
|
|
assert ev["user_id"] == "user-1"
|
|
assert ev["action"] == "user.create"
|
|
assert ev["resource_type"] == "user"
|
|
assert ev["resource_id"] == "u123"
|
|
assert ev["ip_address"] == "127.0.0.1"
|
|
detail = json.loads(ev["detail"])
|
|
assert detail["username"] == "alice"
|
|
|
|
|
|
def test_record_audit_no_detail(storage):
|
|
record_audit(storage, "user-1", "token.revoke", "token", "t456")
|
|
events = storage.list_audit_events()
|
|
assert len(events) == 1
|
|
assert events[0]["detail"] == "{}"
|
|
|
|
|
|
def test_record_audit_silent_on_failure():
|
|
"""record_audit should not raise even if storage is broken."""
|
|
|
|
class BrokenStorage:
|
|
def record_audit_event(self, **kw):
|
|
raise RuntimeError("boom")
|
|
|
|
# Should not raise
|
|
record_audit(BrokenStorage(), "u1", "test.action")
|
|
|
|
|
|
def test_record_audit_generates_unique_ids(storage):
|
|
record_audit(storage, "u1", "a.one")
|
|
record_audit(storage, "u1", "a.two")
|
|
events = storage.list_audit_events()
|
|
assert len(events) == 2
|
|
assert events[0]["event_id"] != events[1]["event_id"]
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Credential redaction at the audit boundary
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
def test_record_audit_redacts_passwords_by_default(storage):
|
|
"""Detail strings go through redact_credentials by default."""
|
|
record_audit(
|
|
storage,
|
|
"u1",
|
|
"coordinator.spawn",
|
|
detail={
|
|
"initial_message": "connect via postgresql://alice:s3cret@db.example.com/app",
|
|
},
|
|
)
|
|
event = storage.list_audit_events()[0]
|
|
detail = json.loads(event["detail"])
|
|
# Exact redaction text comes from output_guard._redact_credentials —
|
|
# assert the token is stripped rather than the exact marker so
|
|
# this test doesn't break if the marker format evolves.
|
|
assert "s3cret" not in detail["initial_message"]
|
|
assert "REDACTED" in detail["initial_message"]
|
|
|
|
|
|
def test_record_audit_redacts_nested_strings(storage):
|
|
"""Walker descends into lists / nested dicts."""
|
|
record_audit(
|
|
storage,
|
|
"u1",
|
|
"task_list.update",
|
|
detail={
|
|
"tasks": [
|
|
{"title": "normal task"},
|
|
{"title": "pull secret from AWS_SECRET_ACCESS_KEY=AKIAEXAMPLE123"},
|
|
],
|
|
},
|
|
)
|
|
event = storage.list_audit_events()[0]
|
|
detail = json.loads(event["detail"])
|
|
assert detail["tasks"][0]["title"] == "normal task"
|
|
assert "AKIAEXAMPLE123" not in detail["tasks"][1]["title"]
|
|
|
|
|
|
def test_record_audit_raw_detail_preserves_payload(storage):
|
|
"""`raw_detail=True` bypasses the scrub — operator-originated detail only."""
|
|
secret_like = "postgresql://alice:s3cret@db.example.com/app"
|
|
record_audit(
|
|
storage,
|
|
"admin-1",
|
|
"investigation.note",
|
|
detail={"note": secret_like},
|
|
raw_detail=True,
|
|
)
|
|
event = storage.list_audit_events()[0]
|
|
detail = json.loads(event["detail"])
|
|
assert detail["note"] == secret_like
|
|
|
|
|
|
def test_record_audit_strips_control_chars(storage):
|
|
"""CR/LF/NUL/DEL and C0 controls are replaced with spaces so a
|
|
downstream exporter that prints raw detail strings can't re-surface
|
|
log-injection. Tab/newline are deliberately preserved."""
|
|
record_audit(
|
|
storage,
|
|
"u1",
|
|
"coordinator.note",
|
|
detail={
|
|
"msg": "hello\r\nInjected: bad\x00 escape \x1b[31mred\x1b[0m\x7f",
|
|
"ok_tab": "a\tb\nc",
|
|
},
|
|
)
|
|
event = storage.list_audit_events()[0]
|
|
detail = json.loads(event["detail"])
|
|
# CR / NUL / ESC / DEL scrubbed to spaces; tab + newline kept.
|
|
assert "\r" not in detail["msg"]
|
|
assert "\x00" not in detail["msg"]
|
|
assert "\x1b" not in detail["msg"]
|
|
assert "\x7f" not in detail["msg"]
|
|
assert "hello" in detail["msg"]
|
|
assert detail["ok_tab"] == "a\tb\nc"
|
|
|
|
|
|
def test_record_audit_clean_strings_roundtrip_unchanged(storage):
|
|
"""Detail strings with no credential patterns and no control chars
|
|
pass through unchanged — the fast-path / scrub must not corrupt the
|
|
common case."""
|
|
clean = {"note": "hello world", "code": "import foo", "state": "ok"}
|
|
record_audit(storage, "u1", "coordinator.note", detail=clean)
|
|
event = storage.list_audit_events()[0]
|
|
assert json.loads(event["detail"]) == clean
|
|
|
|
|
|
def test_record_audit_fast_path_skips_no_string_detail(storage):
|
|
"""A detail carrying only scalars (no strings anywhere) must persist
|
|
identically — exercises the ``_has_any_string`` fast path."""
|
|
record_audit(
|
|
storage,
|
|
"u1",
|
|
"coordinator.metric",
|
|
detail={"spawned": 5, "ok": True, "parent": None, "tail": [1, 2, 3]},
|
|
)
|
|
event = storage.list_audit_events()[0]
|
|
assert json.loads(event["detail"]) == {
|
|
"spawned": 5,
|
|
"ok": True,
|
|
"parent": None,
|
|
"tail": [1, 2, 3],
|
|
}
|
|
|
|
|
|
def test_record_audit_redacts_dict_keys(storage):
|
|
"""Walker descends into dict keys too — a caller using
|
|
model-controlled text as a key can't leak it verbatim."""
|
|
record_audit(
|
|
storage,
|
|
"u1",
|
|
"coordinator.note",
|
|
detail={"postgresql://alice:s3cret@db.example.com/app": True},
|
|
)
|
|
event = storage.list_audit_events()[0]
|
|
detail = json.loads(event["detail"])
|
|
assert all("s3cret" not in k for k in detail)
|
|
|
|
|
|
def test_record_audit_walks_set_and_frozenset(storage):
|
|
"""Walker handles set/frozenset values (docstring promise)."""
|
|
record_audit(
|
|
storage,
|
|
"u1",
|
|
"coordinator.note",
|
|
detail={"tags": frozenset({"ak_" + "x" * 40, "plain"})},
|
|
)
|
|
event = storage.list_audit_events()[0]
|
|
detail = json.loads(event["detail"])
|
|
# The credential-looking AK token gets scrubbed; the plain one survives.
|
|
tags = detail["tags"]
|
|
assert "plain" in tags
|
|
|
|
|
|
def test_record_audit_leaves_non_string_scalars_alone(storage):
|
|
"""Non-string scalars (int / bool / None) pass through unchanged."""
|
|
record_audit(
|
|
storage,
|
|
"u1",
|
|
"coordinator.spawn",
|
|
detail={"budget_ok": True, "spawned": 5, "parent": None},
|
|
)
|
|
event = storage.list_audit_events()[0]
|
|
detail = json.loads(event["detail"])
|
|
assert detail == {"budget_ok": True, "spawned": 5, "parent": None}
|