mirror of
https://github.com/turnstonelabs/turnstone.git
synced 2026-08-28 06:44:51 -06:00
d068366a61
* feat(rbac): editable builtin role permissions via overlay layer
Adds a ``role_permission_overrides`` table that stores per-(role_id,
permission) grant/revoke deltas, applied on top of the immutable
``roles.permissions`` baseline at permission-load time. Builtin roles
(``builtin-admin/operator/viewer``) become customizable through the
admin Roles UI without losing the "reset to default" guarantee — every
override is auditable and reversible.
Motivating case: ``model.skills.write`` is deliberately default-ungranted
on every role so operators must consciously opt in before a coordinator
session can mutate the skill catalog. Until now there was no UX path to
do that opt-in — the only options were dropping into SQL or running a
fresh migration. The overrides editor closes that gap.
Backend
- Migration 057 + storage methods on both sqlite + postgresql backends
- ``get_user_permissions`` merges baseline ∪ grants − revokes for builtin
rows; custom rows pass through unchanged
- ``GET /v1/api/admin/roles/{id}/effective`` for inspect
- ``PUT /v1/api/admin/roles/{id}/overrides`` for write — admin.roles gated,
audited, validates against ``_VALID_PERMISSIONS``, refuses non-builtin
targets, strips no-op grants/revokes before persisting
- Lockout guard: cannot revoke ``admin.roles`` if doing so would leave
zero users with the permission (returns 409)
- ``coordinator.trust.send`` added to ``_VALID_PERMISSIONS`` — was
seeded into builtin-admin by migration 042 but never registered with
the validator, so the very first round-trip through the editor 400'd
on it. Drift-detection test guards future migrations from recreating
the same gap
Frontend
- Roles tab redesign: chevron + permission-count chip replace the
"..." truncation; expand-on-click drawer groups perms by namespace
with baseline / grant (green +) / revoke (red −) chip variants
- Edit modal opens for builtin rows ("Customize Built-in Role" title);
toggles show baseline-default vs override state; submit diffs against
the rendered toggle universe (not raw baseline) so future taxonomy
drift can't silently strip unknown perms
- "Modified +N/-N" pill on rows with active overrides; "Reset to default"
drawer action clears the override set
- ``_PERMISSION_SECTIONS`` brought up to date with all currently-seeded
perms (admin.coordinator, admin.cluster.inspect, admin.models,
admin.nodes, admin.prompt_policies, conversation.modify,
coordinator.trust.send were missing)
Tests
- 7 storage tests covering set/list/clear/effective + overlay merge into
``get_user_permissions`` for both builtin and custom roles
- 11 endpoint tests covering effective/overrides happy paths, validation,
lockout guard, builtin-only restriction, no-op normalization, list
enrichment
* feat(rbac): enforce workstreams.{create,close} + tools.approve gates
These three permissions were declared in ``_VALID_PERMISSIONS``, seeded
into ``builtin-operator``'s baseline by migration 008/017, surfaced in
the admin Roles UI as toggles, and documented in ``bootstrap.py`` as
the operator role's capabilities — and never enforced anywhere. The
audit that ran out of the overlay PR found zero ``require_permission``
sites for any of them; any authenticated user could create workstreams,
close any workstream, or approve any pending tool regardless of role.
Behaviour change for callers without the perms:
- ``POST /v1/api/workstreams/new`` (node + console proxy variants)
now 403 without ``workstreams.create``
- ``POST /v1/api/workstreams/{ws_id}/close`` (and ``/route/`` proxy)
now 403 without ``workstreams.close``
- ``POST /v1/api/workstreams/{ws_id}/approve`` (and ``/route/`` proxy)
now 403 without ``tools.approve``
The OR-fallback to ``admin.coordinator`` keeps coord sessions spawning
interactive children unblocked without needing operator-style perms.
Service-scoped inter-cluster calls bypass via the existing
``allow_service_bypass`` path on the new ``require_any_permission``
helper. Builtin admin and operator both already carry these perms;
viewer correctly loses workstream create/close/approve (it already
couldn't do those in spirit).
Implementation
- ``require_any_permission`` (core/auth.py) — OR-semantics variant of
``require_permission`` with per-conditional comments documenting the
security policy at the choke point. 403 body names every accepted
perm so operators get an actionable remediation
- ``make_{create,close,approve}_handler`` (core/session_routes.py)
accept ``fallback_permissions: tuple[str, ...]`` — checked only when
``cfg.permission_gate is None`` (interactive case). Coord's
``permission_gate=_require_admin_coordinator`` continues to take
precedence on the coord-config side
- Console-side ``create_workstream`` and ``route_create`` inline the
same OR check before proxying — fail fast on a forbidden request
without burning a cluster round-trip
- ``route_proxy`` adds a verb-scoped gate on ``approve`` and ``close``
only; ``send``/``cancel``/``dequeue``/``command``/``plan`` remain
authenticated-only (pre-existing, out of scope for this audit)
Tests
- New ``TestPermissionGatesOnLifecycle`` (4 tests) in test_server_authz
pinning 403-without-perm + non-403-with-perm at the node lift sites
- New ``TestRouteProxyPermissionGates`` (5 tests) in
test_console_routing_proxy covering 403 paths, OR fallback via
``admin.coordinator``, and that ``send`` remains ungated
- ``_make_jwt`` helpers in test_server_authz, test_close_reason_
persistence, test_server_attachments_on_create updated to embed
operator-shaped perms by default so existing tests continue to
exercise the post-gate logic rather than 403'ing on the new check
Docs
- ``bootstrap.py`` operator role line corrected to list every perm
it actually carries (was missing ``tools.approve`` and
``conversation.modify``)
* fix(rbac): close lockout + escalation gaps in role-overrides editor
Three issues surfaced by /review of the overlay layer and gate uplift —
all in the RBAC/auth surface, treated as zero-days.
**F-1: lockout guard misses the grant-removal path.** PUT-replace
semantics on ``set_role_overrides`` mean an existing grant of
``admin.roles`` (added via override to e.g. builtin-operator) is
silently dropped when the new payload omits it. The previous guard
short-circuited on ``"admin.roles" not in revokes`` and never noticed.
Concrete cluster-bricking scenario: grant admin.roles to operator via
override, unassign builtin-admin, click "Reset to default" on operator
→ all users lose admin.roles, recoverable only via SQL.
The rewritten guard simulates the post-PUT effective set on the target
role directly: if ``(baseline | new_grants) - new_revokes`` lacks
admin.roles AND nobody holds it via another role, refuse the change.
The "via another role" question is answered by one bulk query rather
than the prior O(users × roles) round-trip loop.
**F-3: lockout check blocked the event loop on moderate deployments.**
The prior check called ``storage.list_user_roles`` per user and
``storage.effective_role_permissions`` per (user, role) pair —
synchronous SQL inside an async handler. 200 users × 5 roles = 1000
connection cycles long enough to trip reverse-proxy timeouts on a
permission revoke.
Replaced with ``storage.users_with_permission(perm, *,
exclude_role_id)`` — one join over ``user_roles ⋈ roles`` plus one IN
fetch on overrides for the builtin role ids in the result, folded
in-process. Two queries total, independent of cluster size. The whole
check now runs under ``asyncio.to_thread`` so even the bulk read
doesn't stall the loop.
**F-2 reframed: admin_assign_role's subset check ignored the overlay.**
The check at lines 6321-6328 reads ``target_role.get("permissions",
"")`` (baseline column) when computing the perms it requires the
caller to hold. After this branch, an admin.roles holder can grant
e.g. ``model.skills.write`` to builtin-operator via override; an
admin.users holder (who happens to NOT hold that perm) could then
assign operator to a new user, silently escalating the assignee. The
existing two-person-rule by perm split (admin.roles for catalog edits,
admin.users for assignments) only holds if the assignment-time check
considers the overlay. Switched ``target_perms`` to
``storage.effective_role_permissions(role_id)["effective"]``.
Note: this PR retains the existing model where admin.roles is the
catalog-edit superuser (admin_create_role, admin_update_role, and now
admin_role_overrides all skip the caller-holds-grants check). The
two-person rule against escalation lives at the assignment gate, which
this fix reinforces.
**F-7: delete_role left orphaned override rows.** No FK on
``role_permission_overrides.role_id`` (migration 057 omitted FKs to
match the rest of the governance schema). Added explicit cleanup in
both sqlite + postgresql ``delete_role`` implementations so a
re-seeded role_id (deterministic for builtins on schema reseed) can't
silently inherit stale overrides from the prior occupant.
Tests
- storage: ``test_users_with_permission_bulk`` exercises the new bulk
helper including ``exclude_role_id`` and overlay folding
- storage: ``test_delete_role_cleans_up_overrides`` pins the F-7 fix
- endpoint: ``test_overrides_lockout_guard_blocks_grant_removal`` is
the F-1 reproduction — operator-overlay grants admin.roles, builtin-
admin has it removed, attempting to reset operator's overrides 409s
- endpoint: ``test_assign_role_blocks_escalation_via_overlay_grant``
pins the F-2 reframed fix — overlay-poisoned operator can't be
assigned by a caller missing the overlay perms
* refactor(rbac): cleanup batch from /review (#584)
Five non-security findings folded into one commit so the security
batch stays focused. All consistent with the existing intent of
``feat/builtin-role-overrides``.
**F-4: presence check on ``_effectivePerms``.** ``governance.js`` was
guarding on ``Array.isArray(role.effective) && role.effective.length > 0``,
falling through to splitting ``role.permissions`` (the baseline) when
the array was empty. For a builtin role whose overrides legitimately
revoke every baseline perm, that path silently rendered the baseline
chips with no override indicators — the inspector lied about what the
role can do. ``_enrich_role`` always sets ``effective: []``, so
presence is the right sentinel.
**F-5: JS-side drift detector.** Commit 1 added a Python-side test
asserting ``_VALID_PERMISSIONS`` covers every baseline perm; the
mirror invariant on the frontend went uncaught. A new perm added to
``_VALID_PERMISSIONS`` without a matching entry in
``_PERMISSION_SECTIONS`` becomes silently un-customizable through the
admin UI (the only documented grant/revoke path). Test parses the
JS const out via regex and asserts set-equality both directions —
detects "missing in UI" and "extra in UI" so the toggle catalog and
validator can't fork.
**F-6: bulk enrich for ``admin_list_roles``.** Was ``1 +
2*builtin_count + 1*custom_count`` SELECTs per admin-tab open;
collapsed to one ``IN``-filtered query via new
``storage.effective_role_permissions_bulk(role_ids)``. Implemented
on both sqlite + postgresql backends following the existing
``effective_role_permissions`` shape.
**F-8: rename ``fallback_permissions`` → ``accepted_permissions``.**
The lift body uses ``if cfg.permission_gate / elif accepted_permissions``
— mutually exclusive — so when ``permission_gate`` is None this IS
the primary gate, not a fallback to anything. The "fallback" name
suggested a tier-2-after-tier-1 semantic that didn't exist. Renamed
across ``make_{approve,close,create}_handler`` factories, the three
call sites in ``turnstone/server.py``, and the docstrings.
**F-9: positive lift-level tests for ``admin.coordinator``-only.**
``TestPermissionGatesOnLifecycle`` previously had a single positive
test for ``workstreams.create`` alone, plus negative-403 tests for
each verb without perms. The OR-fallback to ``admin.coordinator``
(which keeps coord sessions spawning interactive children unblocked)
had no positive coverage at the lift code path — only at the proxy,
which exercises a different verb-dict gate. Added three tests
(create / close / approve) that pass ``admin.coordinator`` alone and
assert non-403, so a future tightening of the accepted_permissions
tuple can't silently regress coord-driven child workstreams.
Out of scope: nit perf-4 (event-delegation refactor on
``_renderGovRoles``). ``setSafeHtml`` rebuild is the existing
pattern across every admin tab; rewriting one tab's render path on
this branch would be drive-by inconsistent with the surrounding
codebase. Filed as a separate concern if the Roles tab grows past
the scale where it bites.
* fix(rbac-ui): aria-expanded + row-click on Roles drawer (#585)
Two Copilot review findings on governance.js:
- Expand button was missing aria-expanded — screen readers couldn't
announce drawer state. Now reflects the row's expanded flag.
- Comment said "row + chevron both work" but only the chevron was
wired. Added data-expand-role to the row element too so the
existing handler loop (querySelectorAll on the attribute) picks up
both — clicking anywhere in the role row toggles the drawer.
Edit/Delete handlers already stopPropagation so they aren't
triggered by the row-level click.
* fix(migrations): rebase role_permission_overrides to 058
PR #560 mitigation #1 landed 057_output_assessments_llm_judge.py on
main in parallel; my migration claimed the same number, forking
alembic's head and breaking postgres. Renumbered to 058 and
re-pointed down_revision at 057 so the chain stays linear.
No behaviour change — same DDL. Full sweep clean (6730 passed).
* fix(migrations): update 058 revision strings to match filename
Previous commit (ea86aefc) renamed 057_role_permission_overrides.py to
058_* but the in-file revision = "057" / down_revision = "056"
strings stayed — leftover from when the file shipped as 057. Tests
pass because alembic walks the chain by revision string, and the
strings now correctly read revision = "058" / down_revision = "057"
to make the chain linear with main's 057_output_assessments_llm_judge.
Caught locally before re-running CI; my prior `git mv` + content edit
landed as a staged rename + unstaged modification on the previous
push.
1008 lines
39 KiB
Python
1008 lines
39 KiB
Python
"""Tests for governance admin API endpoints (roles, orgs, policies, templates, usage, audit)."""
|
|
|
|
from __future__ import annotations
|
|
|
|
from typing import TYPE_CHECKING, Any
|
|
|
|
import pytest
|
|
from starlette.applications import Starlette
|
|
from starlette.middleware import Middleware
|
|
from starlette.middleware.base import BaseHTTPMiddleware
|
|
from starlette.routing import Mount, Route
|
|
from starlette.testclient import TestClient
|
|
|
|
if TYPE_CHECKING:
|
|
from starlette.requests import Request
|
|
from starlette.responses import Response
|
|
|
|
from turnstone.console.server import (
|
|
admin_assign_role,
|
|
admin_audit,
|
|
admin_create_policy,
|
|
admin_create_role,
|
|
admin_delete_policy,
|
|
admin_delete_role,
|
|
admin_delete_user,
|
|
admin_get_org,
|
|
admin_list_orgs,
|
|
admin_list_policies,
|
|
admin_list_roles,
|
|
admin_list_user_roles,
|
|
admin_role_effective,
|
|
admin_role_overrides,
|
|
admin_unassign_role,
|
|
admin_update_org,
|
|
admin_update_policy,
|
|
admin_update_role,
|
|
admin_usage,
|
|
)
|
|
from turnstone.core.auth import AuthResult
|
|
from turnstone.core.storage._sqlite import SQLiteBackend
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Auth bypass middleware — injects a full-access AuthResult on every request.
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
class _InjectAuthMiddleware(BaseHTTPMiddleware):
|
|
async def dispatch(self, request: Request, call_next: Any) -> Response:
|
|
request.state.auth_result = AuthResult(
|
|
user_id="test-admin",
|
|
scopes=frozenset({"approve"}),
|
|
token_source="config",
|
|
permissions=frozenset(
|
|
{
|
|
"read",
|
|
"write",
|
|
"approve",
|
|
"admin.roles",
|
|
"admin.users",
|
|
"admin.orgs",
|
|
"admin.policies",
|
|
"admin.prompt_policies",
|
|
"admin.skills",
|
|
"admin.usage",
|
|
"admin.audit",
|
|
"admin.schedules",
|
|
"admin.watches",
|
|
"tools.approve",
|
|
"workstreams.create",
|
|
"workstreams.close",
|
|
}
|
|
),
|
|
)
|
|
return await call_next(request)
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Fixtures
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
@pytest.fixture
|
|
def storage(tmp_path):
|
|
"""Fresh SQLite backend for each test, seeded with test users."""
|
|
backend = SQLiteBackend(str(tmp_path / "test.db"))
|
|
# Seed users required by role assignment tests
|
|
backend.create_user("test-admin", "testadmin", "Test Admin", "hash")
|
|
backend.create_user("user-1", "user1", "User One", "hash")
|
|
return backend
|
|
|
|
|
|
@pytest.fixture
|
|
def client(storage):
|
|
"""TestClient with storage and auth bypassed."""
|
|
app = Starlette(
|
|
routes=[
|
|
Mount(
|
|
"/v1",
|
|
routes=[
|
|
# Roles
|
|
Route("/api/admin/roles", admin_list_roles),
|
|
Route("/api/admin/roles", admin_create_role, methods=["POST"]),
|
|
Route("/api/admin/roles/{role_id}", admin_update_role, methods=["PUT"]),
|
|
Route("/api/admin/roles/{role_id}", admin_delete_role, methods=["DELETE"]),
|
|
Route("/api/admin/roles/{role_id}/effective", admin_role_effective),
|
|
Route(
|
|
"/api/admin/roles/{role_id}/overrides",
|
|
admin_role_overrides,
|
|
methods=["PUT"],
|
|
),
|
|
# Users
|
|
Route(
|
|
"/api/admin/users/{user_id}",
|
|
admin_delete_user,
|
|
methods=["DELETE"],
|
|
),
|
|
# User-role assignments
|
|
Route("/api/admin/users/{user_id}/roles", admin_list_user_roles),
|
|
Route(
|
|
"/api/admin/users/{user_id}/roles",
|
|
admin_assign_role,
|
|
methods=["POST"],
|
|
),
|
|
Route(
|
|
"/api/admin/users/{user_id}/roles/{role_id}",
|
|
admin_unassign_role,
|
|
methods=["DELETE"],
|
|
),
|
|
# Orgs
|
|
Route("/api/admin/orgs", admin_list_orgs),
|
|
Route("/api/admin/orgs/{org_id}", admin_get_org),
|
|
Route("/api/admin/orgs/{org_id}", admin_update_org, methods=["PUT"]),
|
|
# Policies
|
|
Route("/api/admin/policies", admin_list_policies),
|
|
Route("/api/admin/policies", admin_create_policy, methods=["POST"]),
|
|
Route(
|
|
"/api/admin/policies/{policy_id}",
|
|
admin_update_policy,
|
|
methods=["PUT"],
|
|
),
|
|
Route(
|
|
"/api/admin/policies/{policy_id}",
|
|
admin_delete_policy,
|
|
methods=["DELETE"],
|
|
),
|
|
# Usage & Audit
|
|
Route("/api/admin/usage", admin_usage),
|
|
Route("/api/admin/audit", admin_audit),
|
|
],
|
|
),
|
|
],
|
|
middleware=[Middleware(_InjectAuthMiddleware)],
|
|
)
|
|
app.state.auth_storage = storage
|
|
return TestClient(app)
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Helpers
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
def _role_payload(**overrides: Any) -> dict[str, Any]:
|
|
defaults: dict[str, Any] = {
|
|
"name": "analyst",
|
|
"display_name": "Data Analyst",
|
|
"permissions": "read,write",
|
|
}
|
|
defaults.update(overrides)
|
|
return defaults
|
|
|
|
|
|
def _policy_payload(**overrides: Any) -> dict[str, Any]:
|
|
defaults: dict[str, Any] = {
|
|
"name": "Allow bash",
|
|
"tool_pattern": "bash_*",
|
|
"action": "allow",
|
|
"priority": 10,
|
|
}
|
|
defaults.update(overrides)
|
|
return defaults
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Tests — Roles
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
class TestRoles:
|
|
def test_list_empty(self, client):
|
|
resp = client.get("/v1/api/admin/roles")
|
|
assert resp.status_code == 200
|
|
assert resp.json()["roles"] == []
|
|
|
|
def test_create_role(self, client):
|
|
resp = client.post("/v1/api/admin/roles", json=_role_payload())
|
|
assert resp.status_code == 200
|
|
role = resp.json()
|
|
assert role["name"] == "analyst"
|
|
assert role["display_name"] == "Data Analyst"
|
|
assert role["permissions"] == "read,write"
|
|
assert role["builtin"] is False
|
|
assert "role_id" in role
|
|
assert "created" in role
|
|
|
|
def test_create_role_missing_name(self, client):
|
|
resp = client.post("/v1/api/admin/roles", json=_role_payload(name=""))
|
|
assert resp.status_code == 400
|
|
assert "name" in resp.json()["error"].lower()
|
|
|
|
def test_create_role_invalid_name(self, client):
|
|
resp = client.post("/v1/api/admin/roles", json=_role_payload(name="bad name!@#"))
|
|
assert resp.status_code == 400
|
|
assert "name" in resp.json()["error"].lower()
|
|
|
|
def test_create_role_with_model_skills_write_permission(self, client):
|
|
"""``model.skills.write`` is enumerated in ``_VALID_PERMISSIONS`` and
|
|
passes role-create validation. Catches the case where the constant
|
|
is added on the server but missed by the validator or the constant
|
|
list."""
|
|
resp = client.post(
|
|
"/v1/api/admin/roles",
|
|
json=_role_payload(name="skillwriter", permissions="read,model.skills.write"),
|
|
)
|
|
assert resp.status_code == 200, resp.json()
|
|
assert "model.skills.write" in resp.json()["permissions"]
|
|
|
|
def test_permission_sections_js_covers_valid_permissions(self):
|
|
"""F-5: ``_PERMISSION_SECTIONS`` in governance.js mirrors
|
|
``_VALID_PERMISSIONS`` in console/server.py. A new perm added
|
|
to the Python validator without a matching JS toggle becomes
|
|
silently un-customizable through the admin Roles UI — the only
|
|
documented path for granting/revoking perms on a builtin.
|
|
Catches the same shape that surfaced ``coordinator.trust.send``
|
|
missing from the validator during manual verification of the
|
|
overlay editor (a similar drift, in the opposite direction)."""
|
|
import re
|
|
from pathlib import Path
|
|
|
|
from turnstone.console.server import _VALID_PERMISSIONS
|
|
|
|
src = Path("turnstone/console/static/governance.js").read_text()
|
|
# _PERMISSION_SECTIONS is a `const X = [...]` containing nested
|
|
# `permissions: ["a", "b", ...]` arrays. Pull every quoted
|
|
# string out of every permissions: [...] block; we don't need
|
|
# a full JS parser to enumerate the perm names.
|
|
m = re.search(
|
|
r"const _PERMISSION_SECTIONS\s*=\s*\[(.*?)\];",
|
|
src,
|
|
re.DOTALL,
|
|
)
|
|
assert m, "could not locate _PERMISSION_SECTIONS in governance.js"
|
|
body = m.group(1)
|
|
in_ui = set(re.findall(r'"([a-z][a-z._]*)"', body))
|
|
# Exclude the section labels themselves (they're sentence-case
|
|
# like "Scopes", "Admin"; the regex above already excludes them
|
|
# by anchoring on lowercase, but be explicit about intent).
|
|
missing_in_ui = sorted(_VALID_PERMISSIONS - in_ui)
|
|
extra_in_ui = sorted(in_ui - _VALID_PERMISSIONS)
|
|
assert not missing_in_ui, (
|
|
f"perms in _VALID_PERMISSIONS but not _PERMISSION_SECTIONS "
|
|
f"(silently un-customizable in admin UI): {missing_in_ui}"
|
|
)
|
|
assert not extra_in_ui, (
|
|
f"perms in _PERMISSION_SECTIONS but not _VALID_PERMISSIONS "
|
|
f"(toggle would 400 on save): {extra_in_ui}"
|
|
)
|
|
|
|
def test_valid_permissions_covers_all_seeded_builtin_perms(self):
|
|
"""Every permission migration 008/011/014/015/029/032/033/035/040/042
|
|
adds to a builtin role must be in ``_VALID_PERMISSIONS`` — otherwise
|
|
the overrides editor cannot round-trip the baseline (a perm dropped
|
|
from the toggle universe gets stripped to satisfy the validator,
|
|
producing a silent capability loss). Caught by the manual
|
|
verification run of feat/builtin-role-overrides:
|
|
``coordinator.trust.send`` was in the baseline but not the
|
|
validator, so the very first Save through the overrides editor
|
|
400'd."""
|
|
from turnstone.console.server import _VALID_PERMISSIONS
|
|
|
|
# Mirror the union the bootstrap migrations write into the baseline
|
|
# ``permissions`` column for builtin-admin. Keep this in sync with
|
|
# 017_catchup_admin_permissions.py and every subsequent migration
|
|
# that touches builtin-admin.
|
|
seeded = {
|
|
"read",
|
|
"write",
|
|
"approve",
|
|
"admin.users",
|
|
"admin.roles",
|
|
"admin.orgs",
|
|
"admin.policies",
|
|
"admin.prompt_policies",
|
|
"admin.skills",
|
|
"admin.audit",
|
|
"admin.usage",
|
|
"admin.schedules",
|
|
"admin.watches",
|
|
"admin.judge",
|
|
"admin.memories",
|
|
"admin.settings",
|
|
"admin.mcp",
|
|
"admin.models",
|
|
"admin.nodes",
|
|
"admin.coordinator",
|
|
"admin.cluster.inspect",
|
|
"tools.approve",
|
|
"workstreams.create",
|
|
"workstreams.close",
|
|
"conversation.modify",
|
|
"coordinator.trust.send",
|
|
}
|
|
missing = sorted(seeded - _VALID_PERMISSIONS)
|
|
assert not missing, f"perms in baseline but not _VALID_PERMISSIONS: {missing}"
|
|
|
|
def test_create_role_rejects_unknown_permission(self, client):
|
|
"""Unknown permission strings are rejected — guards the validator
|
|
against typos in the constant list and would-be capability inflation
|
|
via the admin API."""
|
|
resp = client.post(
|
|
"/v1/api/admin/roles",
|
|
json=_role_payload(name="bogus", permissions="read,model.does.not.exist"),
|
|
)
|
|
assert resp.status_code == 400
|
|
assert "invalid" in resp.json()["error"].lower()
|
|
|
|
def test_create_role_default_display_name(self, client):
|
|
resp = client.post(
|
|
"/v1/api/admin/roles",
|
|
json={"name": "ops", "permissions": ""},
|
|
)
|
|
assert resp.status_code == 200
|
|
role = resp.json()
|
|
# display_name defaults to name when not provided
|
|
assert role["display_name"] == "ops"
|
|
|
|
def test_list_after_create(self, client):
|
|
client.post("/v1/api/admin/roles", json=_role_payload())
|
|
resp = client.get("/v1/api/admin/roles")
|
|
assert resp.status_code == 200
|
|
roles = resp.json()["roles"]
|
|
assert len(roles) == 1
|
|
assert roles[0]["name"] == "analyst"
|
|
|
|
def test_update_role(self, client):
|
|
create_resp = client.post("/v1/api/admin/roles", json=_role_payload())
|
|
role_id = create_resp.json()["role_id"]
|
|
|
|
resp = client.put(
|
|
f"/v1/api/admin/roles/{role_id}",
|
|
json={"display_name": "Senior Analyst", "permissions": "read,write,approve"},
|
|
)
|
|
assert resp.status_code == 200
|
|
role = resp.json()
|
|
assert role["display_name"] == "Senior Analyst"
|
|
assert role["permissions"] == "read,write,approve"
|
|
|
|
def test_update_nonexistent_role(self, client):
|
|
resp = client.put(
|
|
"/v1/api/admin/roles/nonexistent",
|
|
json={"display_name": "Nope"},
|
|
)
|
|
assert resp.status_code == 404
|
|
|
|
def test_update_builtin_role_rejected(self, client, storage):
|
|
# Seed a builtin role directly via storage
|
|
storage.create_role(
|
|
role_id="builtin-admin",
|
|
name="admin",
|
|
display_name="Administrator",
|
|
permissions="*",
|
|
builtin=True,
|
|
)
|
|
resp = client.put(
|
|
"/v1/api/admin/roles/builtin-admin",
|
|
json={"display_name": "Hacked"},
|
|
)
|
|
assert resp.status_code == 400
|
|
assert "builtin" in resp.json()["error"].lower()
|
|
|
|
def test_delete_role(self, client):
|
|
create_resp = client.post("/v1/api/admin/roles", json=_role_payload())
|
|
role_id = create_resp.json()["role_id"]
|
|
|
|
resp = client.delete(f"/v1/api/admin/roles/{role_id}")
|
|
assert resp.status_code == 200
|
|
assert resp.json()["status"] == "ok"
|
|
|
|
# Verify gone from listing
|
|
list_resp = client.get("/v1/api/admin/roles")
|
|
assert list_resp.json()["roles"] == []
|
|
|
|
def test_delete_nonexistent_role(self, client):
|
|
resp = client.delete("/v1/api/admin/roles/nonexistent")
|
|
assert resp.status_code == 404
|
|
|
|
def test_delete_builtin_role_rejected(self, client, storage):
|
|
storage.create_role(
|
|
role_id="builtin-viewer",
|
|
name="viewer",
|
|
display_name="Viewer",
|
|
permissions="read",
|
|
builtin=True,
|
|
)
|
|
resp = client.delete("/v1/api/admin/roles/builtin-viewer")
|
|
assert resp.status_code == 400
|
|
assert "builtin" in resp.json()["error"].lower()
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Tests — Role permission overrides (builtin customization)
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
def _seed_builtin_admin(storage: Any, perms: str = "read,write,admin.roles") -> None:
|
|
storage.create_role(
|
|
role_id="builtin-admin",
|
|
name="admin",
|
|
display_name="Admin",
|
|
permissions=perms,
|
|
builtin=True,
|
|
)
|
|
storage.assign_role("test-admin", "builtin-admin")
|
|
|
|
|
|
class TestRoleOverrides:
|
|
def test_effective_returns_baseline_when_no_overrides(self, client, storage):
|
|
_seed_builtin_admin(storage, "read,admin.roles")
|
|
resp = client.get("/v1/api/admin/roles/builtin-admin/effective")
|
|
assert resp.status_code == 200
|
|
body = resp.json()
|
|
assert body["baseline"] == ["admin.roles", "read"]
|
|
assert body["grants"] == []
|
|
assert body["revokes"] == []
|
|
assert body["effective"] == ["admin.roles", "read"]
|
|
|
|
def test_effective_404_unknown_role(self, client):
|
|
resp = client.get("/v1/api/admin/roles/nope/effective")
|
|
assert resp.status_code == 404
|
|
|
|
def test_overrides_grant_skills_write(self, client, storage):
|
|
# The motivating case: model.skills.write is default-ungranted,
|
|
# operator opts in via the overrides endpoint.
|
|
_seed_builtin_admin(storage, "read,write,admin.roles")
|
|
resp = client.put(
|
|
"/v1/api/admin/roles/builtin-admin/overrides",
|
|
json={"grant": ["model.skills.write"], "revoke": []},
|
|
)
|
|
assert resp.status_code == 200, resp.json()
|
|
body = resp.json()
|
|
assert "model.skills.write" in body["effective"]
|
|
assert body["grants"] == ["model.skills.write"]
|
|
|
|
def test_overrides_replace_semantics(self, client, storage):
|
|
_seed_builtin_admin(storage, "read,write,admin.roles")
|
|
client.put(
|
|
"/v1/api/admin/roles/builtin-admin/overrides",
|
|
json={"grant": ["model.skills.write"], "revoke": []},
|
|
)
|
|
# PUT replaces — the prior grant should be gone after sending an
|
|
# empty body, leaving only the new revoke (which IS in baseline).
|
|
resp = client.put(
|
|
"/v1/api/admin/roles/builtin-admin/overrides",
|
|
json={"grant": [], "revoke": ["write"]},
|
|
)
|
|
assert resp.status_code == 200
|
|
body = resp.json()
|
|
assert body["grants"] == []
|
|
assert body["revokes"] == ["write"]
|
|
assert "model.skills.write" not in body["effective"]
|
|
|
|
def test_overrides_invalid_permission_rejected(self, client, storage):
|
|
_seed_builtin_admin(storage)
|
|
resp = client.put(
|
|
"/v1/api/admin/roles/builtin-admin/overrides",
|
|
json={"grant": ["totally.fake.perm"], "revoke": []},
|
|
)
|
|
assert resp.status_code == 400
|
|
assert "invalid" in resp.json()["error"].lower()
|
|
|
|
def test_overrides_disjoint_grant_revoke_rejected(self, client, storage):
|
|
_seed_builtin_admin(storage)
|
|
resp = client.put(
|
|
"/v1/api/admin/roles/builtin-admin/overrides",
|
|
json={"grant": ["approve"], "revoke": ["approve"]},
|
|
)
|
|
assert resp.status_code == 400
|
|
|
|
def test_overrides_non_builtin_rejected(self, client, storage):
|
|
storage.create_role(
|
|
role_id="custom-1",
|
|
name="custom",
|
|
display_name="Custom",
|
|
permissions="read",
|
|
builtin=False,
|
|
)
|
|
resp = client.put(
|
|
"/v1/api/admin/roles/custom-1/overrides",
|
|
json={"grant": ["write"], "revoke": []},
|
|
)
|
|
assert resp.status_code == 400
|
|
assert "builtin" in resp.json()["error"].lower()
|
|
|
|
def test_overrides_no_op_grant_and_revoke_normalize(self, client, storage):
|
|
# A grant of a perm already in baseline AND a revoke of a perm not
|
|
# in baseline both have zero behavioural effect; the endpoint
|
|
# strips them rather than persisting redundant rows.
|
|
_seed_builtin_admin(storage, "read,write,admin.roles")
|
|
resp = client.put(
|
|
"/v1/api/admin/roles/builtin-admin/overrides",
|
|
json={
|
|
"grant": ["read", "model.skills.write"],
|
|
"revoke": ["tools.approve"],
|
|
},
|
|
)
|
|
assert resp.status_code == 200
|
|
body = resp.json()
|
|
# Only the meaningful delta survived.
|
|
assert body["grants"] == ["model.skills.write"]
|
|
assert body["revokes"] == []
|
|
|
|
def test_overrides_lockout_guard_blocks_last_admin_revoke(self, client, storage):
|
|
_seed_builtin_admin(storage, "read,admin.roles")
|
|
resp = client.put(
|
|
"/v1/api/admin/roles/builtin-admin/overrides",
|
|
json={"grant": [], "revoke": ["admin.roles"]},
|
|
)
|
|
assert resp.status_code == 409
|
|
assert "admin.roles" in resp.json()["error"]
|
|
# Verify the override was NOT applied — the user must still be admin.
|
|
assert "admin.roles" in storage.get_user_permissions("test-admin")
|
|
|
|
def test_overrides_lockout_guard_permits_revoke_when_other_admin_exists(self, client, storage):
|
|
_seed_builtin_admin(storage, "read,admin.roles")
|
|
# Second role on a different user that also carries admin.roles —
|
|
# revoking from builtin-admin no longer locks the deployment out.
|
|
storage.create_role(
|
|
role_id="custom-admin",
|
|
name="custom-admin",
|
|
display_name="Custom Admin",
|
|
permissions="read,admin.roles",
|
|
builtin=False,
|
|
)
|
|
storage.assign_role("user-1", "custom-admin")
|
|
resp = client.put(
|
|
"/v1/api/admin/roles/builtin-admin/overrides",
|
|
json={"grant": [], "revoke": ["admin.roles"]},
|
|
)
|
|
assert resp.status_code == 200
|
|
|
|
def test_list_roles_includes_overlay_fields(self, client, storage):
|
|
_seed_builtin_admin(storage, "read,admin.roles")
|
|
client.put(
|
|
"/v1/api/admin/roles/builtin-admin/overrides",
|
|
json={"grant": ["model.skills.write"], "revoke": []},
|
|
)
|
|
resp = client.get("/v1/api/admin/roles")
|
|
roles = resp.json()["roles"]
|
|
# Find builtin-admin in the listing
|
|
row = next(r for r in roles if r["role_id"] == "builtin-admin")
|
|
assert row["grants"] == ["model.skills.write"]
|
|
assert row["revokes"] == []
|
|
assert "model.skills.write" in row["effective"]
|
|
|
|
def test_overrides_lockout_guard_blocks_grant_removal(self, client, storage):
|
|
# F-1: PUT-replace semantics mean an existing grant of admin.roles
|
|
# on a role whose baseline lacks it is silently dropped when the
|
|
# new payload omits it. Old guard only fired on explicit revokes
|
|
# and missed this path entirely — concrete cluster-bricking scenario.
|
|
# Setup: only builtin-operator users hold admin.roles, via overlay grant.
|
|
storage.create_role(
|
|
role_id="builtin-operator",
|
|
name="operator",
|
|
display_name="Operator",
|
|
permissions="read,write", # baseline lacks admin.roles
|
|
builtin=True,
|
|
)
|
|
# Grant admin.roles to operator via overlay, then unassign builtin-admin
|
|
# from the test user so operator is the only path to admin.roles.
|
|
storage.set_role_overrides("builtin-operator", {"admin.roles"}, set())
|
|
storage.assign_role("test-admin", "builtin-operator")
|
|
# The test-admin user keeps builtin-admin assigned by _seed_builtin_admin
|
|
# which would normally hold admin.roles — but we seed without it so the
|
|
# only source is the overlay on builtin-operator.
|
|
if storage.get_role("builtin-admin") is None:
|
|
storage.create_role(
|
|
role_id="builtin-admin",
|
|
name="admin",
|
|
display_name="Admin",
|
|
permissions="read,write", # baseline lacks admin.roles
|
|
builtin=True,
|
|
)
|
|
storage.assign_role("test-admin", "builtin-admin")
|
|
# Sanity: admin.roles only reachable via operator's overlay
|
|
assert "admin.roles" in storage.get_user_permissions("test-admin")
|
|
# The lockout-triggering call: Reset operator's overrides (drops
|
|
# the admin.roles grant). Old guard short-circuited because
|
|
# revoke=[] doesn't contain "admin.roles"; new guard simulates
|
|
# the post-PUT effective set on the target role.
|
|
resp = client.put(
|
|
"/v1/api/admin/roles/builtin-operator/overrides",
|
|
json={"grant": [], "revoke": []},
|
|
)
|
|
assert resp.status_code == 409, resp.json()
|
|
assert "admin.roles" in resp.json()["error"]
|
|
# Override was NOT applied — admin.roles still reachable.
|
|
assert "admin.roles" in storage.get_user_permissions("test-admin")
|
|
|
|
def test_assign_role_blocks_escalation_via_overlay_grant(self, storage, client):
|
|
# F-2 reframed. Simulates the attack path where a previous
|
|
# admin.roles holder injected an overlay grant on a builtin
|
|
# role, then a separate admin.users holder (who does NOT hold
|
|
# the granted perm) tries to assign that role to a new user.
|
|
# Without this fix the assign-time subset check would read the
|
|
# baseline column and miss the overlay, silently escalating
|
|
# the assignee.
|
|
#
|
|
# Operator's baseline is unchanged production default
|
|
# ("read,write" — no model.skills.write). The overlay grant
|
|
# below is the simulated attack step, not the system default.
|
|
_seed_builtin_admin(storage, "read,write,admin.roles,admin.users")
|
|
storage.create_role(
|
|
role_id="builtin-operator",
|
|
name="operator",
|
|
display_name="Operator",
|
|
permissions="read,write", # production default
|
|
builtin=True,
|
|
)
|
|
storage.set_role_overrides(
|
|
"builtin-operator", {"model.skills.write"}, set()
|
|
) # simulated prior poisoning by an admin.roles holder
|
|
|
|
# The harness AuthResult holds admin.roles + admin.users + many
|
|
# admin.* perms but NOT model.skills.write. Assigning operator
|
|
# — whose POST-OVERLAY effective set in this test scenario
|
|
# contains model.skills.write — must 403, because the assignee
|
|
# would otherwise gain a perm the assigner doesn't hold.
|
|
resp = client.post(
|
|
"/v1/api/admin/users/user-1/roles",
|
|
json={"role_id": "builtin-operator"},
|
|
)
|
|
assert resp.status_code == 403
|
|
assert "permissions you do not hold" in resp.json()["error"]
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Tests — Role assignments
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
class TestRoleAssignments:
|
|
def test_list_user_roles_empty(self, client):
|
|
resp = client.get("/v1/api/admin/users/user-1/roles")
|
|
assert resp.status_code == 200
|
|
assert resp.json()["roles"] == []
|
|
|
|
def test_assign_role(self, client):
|
|
create_resp = client.post("/v1/api/admin/roles", json=_role_payload())
|
|
role_id = create_resp.json()["role_id"]
|
|
|
|
resp = client.post(
|
|
"/v1/api/admin/users/user-1/roles",
|
|
json={"role_id": role_id},
|
|
)
|
|
assert resp.status_code == 200
|
|
assert resp.json()["status"] == "ok"
|
|
|
|
# Verify listed
|
|
list_resp = client.get("/v1/api/admin/users/user-1/roles")
|
|
roles = list_resp.json()["roles"]
|
|
assert len(roles) >= 1
|
|
|
|
def test_assign_role_missing_role_id(self, client):
|
|
resp = client.post(
|
|
"/v1/api/admin/users/user-1/roles",
|
|
json={},
|
|
)
|
|
assert resp.status_code == 400
|
|
assert "role_id" in resp.json()["error"].lower()
|
|
|
|
def test_unassign_role(self, client):
|
|
create_resp = client.post("/v1/api/admin/roles", json=_role_payload())
|
|
role_id = create_resp.json()["role_id"]
|
|
|
|
# Assign first
|
|
client.post(
|
|
"/v1/api/admin/users/user-1/roles",
|
|
json={"role_id": role_id},
|
|
)
|
|
|
|
# Now unassign
|
|
resp = client.delete(f"/v1/api/admin/users/user-1/roles/{role_id}")
|
|
assert resp.status_code == 200
|
|
assert resp.json()["status"] == "ok"
|
|
|
|
# Verify removed
|
|
list_resp = client.get("/v1/api/admin/users/user-1/roles")
|
|
assert list_resp.json()["roles"] == []
|
|
|
|
def test_unassign_nonexistent(self, client):
|
|
resp = client.delete("/v1/api/admin/users/user-1/roles/nonexistent")
|
|
assert resp.status_code == 404
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Tests — Orgs
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
class TestOrgs:
|
|
def test_list_empty(self, client):
|
|
resp = client.get("/v1/api/admin/orgs")
|
|
assert resp.status_code == 200
|
|
assert resp.json()["orgs"] == []
|
|
|
|
def test_get_org(self, client, storage):
|
|
storage.create_org(
|
|
org_id="org-1",
|
|
name="acme",
|
|
display_name="Acme Corp",
|
|
settings='{"theme": "dark"}',
|
|
)
|
|
resp = client.get("/v1/api/admin/orgs/org-1")
|
|
assert resp.status_code == 200
|
|
org = resp.json()
|
|
assert org["org_id"] == "org-1"
|
|
assert org["name"] == "acme"
|
|
assert org["display_name"] == "Acme Corp"
|
|
|
|
def test_get_org_not_found(self, client):
|
|
resp = client.get("/v1/api/admin/orgs/nonexistent")
|
|
assert resp.status_code == 404
|
|
|
|
def test_update_org(self, client, storage):
|
|
storage.create_org(org_id="org-1", name="acme", display_name="Acme Corp")
|
|
|
|
resp = client.put(
|
|
"/v1/api/admin/orgs/org-1",
|
|
json={"display_name": "Acme Inc.", "settings": '{"theme": "light"}'},
|
|
)
|
|
assert resp.status_code == 200
|
|
org = resp.json()
|
|
assert org["display_name"] == "Acme Inc."
|
|
assert org["settings"] == '{"theme": "light"}'
|
|
|
|
def test_update_org_not_found(self, client):
|
|
resp = client.put(
|
|
"/v1/api/admin/orgs/nonexistent",
|
|
json={"display_name": "Nope"},
|
|
)
|
|
assert resp.status_code == 404
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Tests — Tool policies
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
class TestPolicies:
|
|
def test_list_empty(self, client):
|
|
resp = client.get("/v1/api/admin/policies")
|
|
assert resp.status_code == 200
|
|
assert resp.json()["policies"] == []
|
|
|
|
def test_create_policy(self, client):
|
|
resp = client.post("/v1/api/admin/policies", json=_policy_payload())
|
|
assert resp.status_code == 200
|
|
policy = resp.json()
|
|
assert policy["name"] == "Allow bash"
|
|
assert policy["tool_pattern"] == "bash_*"
|
|
assert policy["action"] == "allow"
|
|
assert policy["priority"] == 10
|
|
assert "policy_id" in policy
|
|
assert "created" in policy
|
|
|
|
def test_create_policy_missing_name(self, client):
|
|
resp = client.post("/v1/api/admin/policies", json=_policy_payload(name=""))
|
|
assert resp.status_code == 400
|
|
assert "name" in resp.json()["error"].lower()
|
|
|
|
def test_create_policy_missing_tool_pattern(self, client):
|
|
resp = client.post(
|
|
"/v1/api/admin/policies",
|
|
json=_policy_payload(tool_pattern=""),
|
|
)
|
|
assert resp.status_code == 400
|
|
assert "tool_pattern" in resp.json()["error"].lower()
|
|
|
|
def test_create_policy_invalid_action(self, client):
|
|
resp = client.post(
|
|
"/v1/api/admin/policies",
|
|
json=_policy_payload(action="yolo"),
|
|
)
|
|
assert resp.status_code == 400
|
|
assert "action" in resp.json()["error"].lower()
|
|
|
|
def test_list_after_create(self, client):
|
|
client.post("/v1/api/admin/policies", json=_policy_payload())
|
|
resp = client.get("/v1/api/admin/policies")
|
|
assert resp.status_code == 200
|
|
policies = resp.json()["policies"]
|
|
assert len(policies) == 1
|
|
assert policies[0]["name"] == "Allow bash"
|
|
|
|
def test_update_policy(self, client):
|
|
create_resp = client.post("/v1/api/admin/policies", json=_policy_payload())
|
|
policy_id = create_resp.json()["policy_id"]
|
|
|
|
resp = client.put(
|
|
f"/v1/api/admin/policies/{policy_id}",
|
|
json={"name": "Deny bash", "action": "deny", "priority": 20},
|
|
)
|
|
assert resp.status_code == 200
|
|
policy = resp.json()
|
|
assert policy["name"] == "Deny bash"
|
|
assert policy["action"] == "deny"
|
|
assert policy["priority"] == 20
|
|
|
|
def test_update_policy_invalid_action(self, client):
|
|
create_resp = client.post("/v1/api/admin/policies", json=_policy_payload())
|
|
policy_id = create_resp.json()["policy_id"]
|
|
|
|
resp = client.put(
|
|
f"/v1/api/admin/policies/{policy_id}",
|
|
json={"action": "nope"},
|
|
)
|
|
assert resp.status_code == 400
|
|
assert "action" in resp.json()["error"].lower()
|
|
|
|
def test_update_policy_not_found(self, client):
|
|
resp = client.put(
|
|
"/v1/api/admin/policies/nonexistent",
|
|
json={"name": "Nope"},
|
|
)
|
|
assert resp.status_code == 404
|
|
|
|
def test_delete_policy(self, client):
|
|
create_resp = client.post("/v1/api/admin/policies", json=_policy_payload())
|
|
policy_id = create_resp.json()["policy_id"]
|
|
|
|
resp = client.delete(f"/v1/api/admin/policies/{policy_id}")
|
|
assert resp.status_code == 200
|
|
assert resp.json()["status"] == "ok"
|
|
|
|
# Verify gone
|
|
list_resp = client.get("/v1/api/admin/policies")
|
|
assert list_resp.json()["policies"] == []
|
|
|
|
def test_delete_policy_not_found(self, client):
|
|
resp = client.delete("/v1/api/admin/policies/nonexistent")
|
|
assert resp.status_code == 404
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Tests — Usage
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
class TestUsage:
|
|
def test_usage_defaults(self, client):
|
|
"""Query usage with no params — should return summary and breakdown."""
|
|
resp = client.get("/v1/api/admin/usage")
|
|
assert resp.status_code == 200
|
|
data = resp.json()
|
|
assert "summary" in data
|
|
assert "breakdown" in data
|
|
# Summary is a list with at least one row
|
|
assert isinstance(data["summary"], list)
|
|
assert len(data["summary"]) >= 1
|
|
# All-zeros when no data
|
|
assert data["summary"][0]["prompt_tokens"] == 0
|
|
|
|
def test_usage_with_data(self, client, storage):
|
|
"""Seed usage events and verify they appear in the query."""
|
|
storage.record_usage_event(
|
|
event_id="evt-1",
|
|
user_id="user-1",
|
|
model="gpt-5",
|
|
prompt_tokens=100,
|
|
completion_tokens=50,
|
|
tool_calls_count=2,
|
|
)
|
|
storage.record_usage_event(
|
|
event_id="evt-2",
|
|
user_id="user-1",
|
|
model="gpt-5",
|
|
prompt_tokens=200,
|
|
completion_tokens=75,
|
|
tool_calls_count=1,
|
|
)
|
|
resp = client.get("/v1/api/admin/usage")
|
|
assert resp.status_code == 200
|
|
summary = resp.json()["summary"]
|
|
assert summary[0]["prompt_tokens"] == 300
|
|
assert summary[0]["completion_tokens"] == 125
|
|
assert summary[0]["tool_calls_count"] == 3
|
|
|
|
def test_usage_with_filters(self, client, storage):
|
|
storage.record_usage_event(
|
|
event_id="evt-f1",
|
|
user_id="user-a",
|
|
model="gpt-5",
|
|
prompt_tokens=100,
|
|
completion_tokens=10,
|
|
)
|
|
storage.record_usage_event(
|
|
event_id="evt-f2",
|
|
user_id="user-b",
|
|
model="claude-4",
|
|
prompt_tokens=200,
|
|
completion_tokens=20,
|
|
)
|
|
resp = client.get("/v1/api/admin/usage?user_id=user-a")
|
|
assert resp.status_code == 200
|
|
summary = resp.json()["summary"]
|
|
assert summary[0]["prompt_tokens"] == 100
|
|
|
|
resp2 = client.get("/v1/api/admin/usage?model=claude-4")
|
|
assert resp2.status_code == 200
|
|
summary2 = resp2.json()["summary"]
|
|
assert summary2[0]["prompt_tokens"] == 200
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Tests — Audit
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
class TestAudit:
|
|
def test_audit_empty(self, client):
|
|
resp = client.get("/v1/api/admin/audit")
|
|
assert resp.status_code == 200
|
|
data = resp.json()
|
|
assert data["events"] == []
|
|
assert data["total"] == 0
|
|
|
|
def test_audit_populated_by_mutations(self, client):
|
|
"""Creating a role should produce an audit event."""
|
|
client.post("/v1/api/admin/roles", json=_role_payload())
|
|
|
|
resp = client.get("/v1/api/admin/audit")
|
|
assert resp.status_code == 200
|
|
data = resp.json()
|
|
assert data["total"] >= 1
|
|
actions = [e["action"] for e in data["events"]]
|
|
assert "role.create" in actions
|
|
|
|
def test_audit_filter_by_action(self, client):
|
|
# Create a role and a policy to produce different audit actions
|
|
client.post("/v1/api/admin/roles", json=_role_payload())
|
|
client.post("/v1/api/admin/policies", json=_policy_payload())
|
|
|
|
resp = client.get("/v1/api/admin/audit?action=policy.create")
|
|
assert resp.status_code == 200
|
|
data = resp.json()
|
|
assert data["total"] >= 1
|
|
assert all(e["action"] == "policy.create" for e in data["events"])
|
|
|
|
def test_audit_filter_by_user_id(self, client):
|
|
client.post("/v1/api/admin/roles", json=_role_payload())
|
|
|
|
resp = client.get("/v1/api/admin/audit?user_id=test-admin")
|
|
assert resp.status_code == 200
|
|
data = resp.json()
|
|
assert data["total"] >= 1
|
|
assert all(e["user_id"] == "test-admin" for e in data["events"])
|
|
|
|
def test_audit_pagination(self, client):
|
|
# Create several resources to produce multiple audit events
|
|
for i in range(5):
|
|
client.post(
|
|
"/v1/api/admin/roles",
|
|
json=_role_payload(name=f"role-{i}"),
|
|
)
|
|
|
|
resp = client.get("/v1/api/admin/audit?limit=2&offset=0")
|
|
assert resp.status_code == 200
|
|
data = resp.json()
|
|
assert len(data["events"]) == 2
|
|
assert data["total"] >= 5
|
|
|
|
resp2 = client.get("/v1/api/admin/audit?limit=2&offset=2")
|
|
assert resp2.status_code == 200
|
|
data2 = resp2.json()
|
|
assert len(data2["events"]) == 2
|
|
# The two pages should not overlap
|
|
ids_page1 = {e["event_id"] for e in data["events"]}
|
|
ids_page2 = {e["event_id"] for e in data2["events"]}
|
|
assert ids_page1.isdisjoint(ids_page2)
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Tests — User self-deletion guard
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
class TestUserSelfDeletion:
|
|
def test_cannot_delete_self(self, client):
|
|
"""Admin should not be able to delete their own account."""
|
|
resp = client.delete("/v1/api/admin/users/test-admin")
|
|
assert resp.status_code == 400
|
|
assert "own account" in resp.json()["error"].lower()
|
|
|
|
def test_can_delete_other_user(self, client):
|
|
resp = client.delete("/v1/api/admin/users/user-1")
|
|
assert resp.status_code == 200
|
|
assert resp.json()["status"] == "ok"
|