Compare commits

..

8 Commits

Author SHA1 Message Date
Patrick Buckley 77c0a7736b Bump version to 0.4.0 and update security docs
- Version bump in __init__.py, pyproject.toml, api-reference.md
- security.md: document JWT aud/iss claims, login rate limiting,
  secure cookie defaults (24h, Secure flag), CORS restriction,
  service JWT auto-rotation, secret strength validation, and
  proxy auth forwarding via service tokens (not user JWT forwarding)
2026-03-04 20:45:00 -08:00
Patrick Buckley 872e1770e6 Feature/code dedup (#25)
* Add JWT auth security hardening (6 fixes)

- Secure cookie flag: make_set_cookie defaults Secure=True, max_age=24h
- Login brute-force protection: LoginRateLimiter (5 attempts/5min per key)
- JWT aud/iss claims: create_jwt/validate_jwt support audience validation
- Service JWT auto-rotation: ServiceTokenManager with 1h expiry, 80% refresh
- CORS restriction: configurable via TURNSTONE_CORS_ORIGINS env var
- JWT secret strength: warning on secrets shorter than 32 chars
- Hard fail for bridge/console when TURNSTONE_JWT_SECRET is missing

* Refactor duplicated code into shared utilities and fix 3 UI bugs

Code deduplication (~235 net lines removed):
- Extract AuthMiddleware + 4 auth endpoint handlers to core/auth.py
- Create core/web_helpers.py (require_storage_or_503, read_json_or_400,
  parse_cors_origins, cors_middleware)
- Extract add_redis_args/broker_from_args to mq/broker.py
- Extract add_log_args/configure_logging_from_args to core/log.py
- Remove dead _CSS/_JS loads, duplicate states dict, _read_json helper,
  unused required_role(), duplicate detect_model() wrapper

Bug fixes:
- Fix console proxy forwarding user's JWT_AUD_CONSOLE token to server
  nodes (use ServiceTokenManager with JWT_AUD_SERVER instead)
- Fix login form autofill: wrap inputs in <form>, add name attributes,
  set type=submit on button
- Fix SSE reconnecting flash: add onopen handler to clear status
  immediately on connection (not waiting for first message)
- Fix chat scroll: add min-height:0 to flex containers, overflow:hidden
  on body to constrain viewport height

* Address CI typecheck failure and Copilot review feedback

- Fix mypy arg-type: use Any for jwt.decode options (PyJWT stubs vary)
- Bridge SSE loops: use event_hooks for auth header refresh on reconnect
  instead of static headers that go stale after token rotation
- Login form: remove javascript:void(0) action (CSP anti-pattern)
- Use JWT_AUD_SERVER/JWT_AUD_CONSOLE constants instead of string literals
  in middleware builder calls to prevent drift
2026-03-04 20:35:21 -08:00
Patrick Buckley a6e929b0a0 Add channel integrations with Discord adapter and atomic session resu… (#24)
* Add channel integrations with Discord adapter and atomic session resume (#24)

Bidirectional channel adapter framework connecting external messaging
platforms to turnstone workstreams via Redis MQ. Discord ships as the
first adapter; the protocol supports future Slack/Teams integrations.

Channel framework:
- ChannelAdapter protocol and ChannelRouter for channel↔workstream mapping
- AsyncRedisBroker with single dispatch loop and per-channel ordered workers
- channel_routes table (migration 003) for persistent route storage
- 9 new StorageBackend methods (4 channel_user + 5 channel_route CRUD)
- Unified turnstone-channel gateway entry point, loads adapters by config
- Message chunking, approval formatting, plan review formatting

Discord adapter:
- discord.py v2.4+ bot with thread-per-@mention model
- Slash commands: /link (modal), /unlink, /ask, /status, /close
- Persistent button views for tool approval and plan review
- Streaming responses via edit-in-place (1.5s interval)
- Stale route detection and atomic session resume via resume_session field
- SessionResumedEvent confirmation back to channel
- Auto-approve support (blanket + per-tool list)

Atomic session resume:
- resume_session field on CreateWorkstreamMessage for single-request resume
- Server resumes session during POST /v1/api/workstreams/new atomically
- Bridge emits SessionResumedEvent to per-workstream channel
- WorkstreamCreatedEvent extended with resumed/session_id/message_count
- Server UI dashboardResumeSession simplified to single request
- Pruned sessions fall back gracefully to fresh start

Service auth:
- Bridge and console auto-mint service JWTs from TURNSTONE_JWT_SECRET
- Bridge: approve scope (1 week). Console collector: read. Proxy: write.

Console admin:
- Channels tab with per-user view, force-link modal, unlink
- 3 admin API endpoints for channel user management
- Styled confirm modals replacing browser confirm() dialogs

Bug fixes:
- AsyncRedisBroker: replaced per-channel listener tasks with single
  dispatch loop + per-channel queue workers (fixes message stealing race)
- Bridge: approval/plan review dedup guard prevents SSE reconnect duplicates
- Bridge: _active_sends tracked for initial messages (fixes missing
  TurnCompleteEvent and unfinalized streaming messages)
- Bridge: HTTP calls moved outside lock scope in approval handlers
- Bridge: _handle_send cleans up _active_sends on HTTP/server errors
- Formatter: reads server SSE format (func_name/preview) with fallback

Docs, SDK, tests:
- docs/channels.md setup guide, architecture diagram 16
- Updated api-reference.md, architecture.md, console.md, docker.md
- Python SDK: resume_session param on create_workstream (async + sync)
- TypeScript SDK: updated CreateWorkstreamRequest/Response interfaces
- OpenAPI schema: resume_session request, resumed/message_count response
- 91 new tests (19 storage, 15 broker, 22 protocol, 6 routing,
  18 discord, 12 resume flow) — 1120 total passing

* Fix CI lint/typecheck failures and address Copilot review feedback (#24)

Lint: fix import ordering, remove unused imports, use contextlib.suppress.
Mypy: explicit postgresql dialect import, add discord module overrides for
optional-dependency CI environments.
Copilot: fix double-escaping in admin confirm modals, return resolved
session_id from server resume response, fix channel_routes diagram schema,
use atomic setdefault for routing locks, add post-insert race guard in
admin channel create, support SSE format in auto-approve check, update
identity linking note in architecture diagram.

* Fix remaining mypy call-arg errors for discord.py optional dependency

Add type: ignore[call-arg] on Modal(title=) and Cog(name=) class
definitions that fail when discord.py is not installed in CI.
2026-03-04 13:02:58 -08:00
Patrick Buckley 047680d669 Add user identity, JWT auth, and admin console UI (#23)
* Add user identity, JWT auth, and admin console UI (#23)

JWT-based authentication with three token types: config-file (hmac,
backward-compat), API tokens (ts_ prefix, SHA-256 hashed), and JWTs
(HS256, 24h expiry). Username:password login via bcrypt. Hierarchical
scopes: read < write < approve.

New tables: users (username, password_hash), api_tokens (token_hash,
scopes, expires), channel_users (future channel integrations). user_id
column added to sessions and workstreams for attribution.

Console owns admin CRUD (6 endpoints under /api/admin/). Server
validates JWTs locally with shared signing secret. Public /api/auth/setup
endpoint for first-time admin creation (atomic, only works with zero
users). turnstone-admin CLI for user/token management.

Admin console UI: Users and Tokens tabs with full CRUD modals, scope
badges, token show-once with clipboard copy, keyboard accessibility
(focus traps, Escape, arrow key tabs, ARIA roles).

Login UI redesigned: username:password primary, token toggle for legacy,
setup wizard auto-detected via /api/auth/status. Python + TypeScript
SDKs updated with login(username, password), authStatus(), setup().

New docs/security.md + diagram 15-auth-architecture.puml. All existing
docs updated. OpenAPI specs include all new endpoints. 64 new tests
(1023 total). Dependencies: PyJWT, bcrypt.

* Fix auth bugs, XSS vector, and doc inaccuracies from PR #23 review

Address Copilot review feedback: escape double quotes in escapeHtml()
to prevent XSS in HTML attributes, add JWT validation fallback so
config tokens containing dots still work, add user_id to
AuthLoginResponse schema, return created field from admin_create_user,
and correct five documentation files to match actual API behavior.
2026-03-04 09:12:18 -08:00
Patrick Buckley 0fd0ad3b2d Add structured logging with structlog and context propagation
Replace ad-hoc logging.basicConfig() calls across all 6 entry points with
a centralized configure_logging() function backed by structlog. JSON output
when stderr is not a TTY (production/Docker), colored console output otherwise.

- New turnstone/core/log.py: configure_logging(), get_logger(), contextvars
  for node_id/ws_id/user_id/request_id auto-injected into every log event
- All entry points (server, bridge, console, sim, cli, migrate) call
  configure_logging() with --log-level and --log-format CLI flags
- Server operational print() calls replaced with structured log.info()
- LogContextMiddleware sets request_id + ws_id per HTTP request with
  token-based reset to prevent context leaking across requests
- Bridge _run_in_context() helper propagates ctx_node_id to child threads
- Env var overrides: TURNSTONE_LOG_LEVEL, TURNSTONE_LOG_FORMAT
- 18 new tests (959 total passing)
2026-03-04 06:47:30 -08:00
Patrick Buckley f3dba836dd Add Git LFS requirement note to README, Diagram PNGs are stored in LFS; git-lfs must be installed for cloning them. 2026-03-04 06:14:27 -08:00
Patrick Buckley 7adda343fc Update docs and diagrams for cluster-scale schema changes
- StorageBackend protocol: document 5 new workstream methods (26 total)
- Session ID: 12-char hex → 32-char full UUID in API reference
- /health endpoint: add node_id field to response docs
- sessions table: document node_id and ws_id columns
- Bridge node_id: document server-owned identity with /health retrieval
- Regenerate storage architecture PNG from updated PlantUML
2026-03-04 06:12:33 -08:00
Patrick Buckley a20a058c59 Add cluster-scale schema, fix console proxy UX, harden SDK sync runner (#22)
* Add cluster-scale schema, fix console proxy UX, harden SDK sync runner

Schema redesign for multi-node deployments:
- New `workstreams` table with node_id, state, lifecycle tracking
- Add node_id + ws_id columns to sessions table with indexes
- Full UUID (32 hex) for session_id and ws_id (was truncated 12/8)
- Server generates and owns node_id, bridge retrieves via /health
- Bridge retries with exponential backoff, fatal on auth errors
- WorkstreamManager persists workstreams and state changes to storage
- /health endpoint exposes node_id for bridge discovery

Console proxy UX fixes:
- Remove duplicate turnstone branding from proxy banner
- Same-tab navigation for Open Node UI and workstream deep links

SDK _SyncRunner fix:
- Sentinel pattern for StopAsyncIteration across thread boundary

Remove misplaced PNGs from docs/diagrams/ (correct copies in png/ subdir).

* Address PR #22 review feedback

- Fix CLI session_factory signature (ws_id param) — CI typecheck failure
- First-phase eviction in create() now calls _cleanup_ui + record_eviction
- close() persists "closed" state to storage via update_workstream_state
- Fix noqa comment in test to pragma: no cover
2026-03-04 06:04:59 -08:00
94 changed files with 11903 additions and 738 deletions
+1
View File
@@ -22,6 +22,7 @@ OPENAI_API_KEY=sk-...
# -- Authentication ------------------------------------------------------------
# TURNSTONE_AUTH_ENABLED=true
# TURNSTONE_AUTH_TOKEN=your-secret-token
# TURNSTONE_JWT_SECRET=python -c "import secrets; print(secrets.token_hex(32))"
# -- Ports ---------------------------------------------------------------------
# SERVER_PORT=8080
+1 -1
View File
@@ -34,7 +34,7 @@ RUN useradd --create-home --shell /bin/bash turnstone
# Install the wheel with all optional extras
COPY --from=builder /build/wheels/*.whl /tmp/wheels/
RUN pip install --no-cache-dir "$(ls /tmp/wheels/*.whl)[mq,console,sim,postgres]" \
RUN pip install --no-cache-dir "$(ls /tmp/wheels/*.whl)[mq,console,sim,postgres,discord]" \
&& rm -rf /tmp/wheels
# Health check script (stdlib only, no pip deps needed)
+1
View File
@@ -412,6 +412,7 @@ Per-workstream metrics are labeled by `ws_id` (bounded to 10 max workstreams).
- Redis (for message queue bridge — `pip install turnstone[mq]`)
- Anthropic provider (optional — `pip install turnstone[anthropic]`)
- PostgreSQL (optional, for production — `pip install turnstone[postgres]`)
- [Git LFS](https://git-lfs.com/) (for cloning — diagram PNGs are stored in LFS)
## License
+41
View File
@@ -109,9 +109,11 @@ services:
- SKIP_PERMISSIONS=${SKIP_PERMISSIONS:-}
- TURNSTONE_AUTH_ENABLED=${TURNSTONE_AUTH_ENABLED:-}
- TURNSTONE_AUTH_TOKEN=${TURNSTONE_AUTH_TOKEN:-}
- TURNSTONE_JWT_SECRET=${TURNSTONE_JWT_SECRET:-}
- MODEL=${MODEL:-}
- TURNSTONE_DB_BACKEND=${DB_BACKEND:-sqlite}
- TURNSTONE_DB_URL=${DATABASE_URL:-}
- TURNSTONE_NODE_ID=${TURNSTONE_NODE_ID:-}
extra_hosts:
- "host.docker.internal:host-gateway"
networks:
@@ -148,6 +150,7 @@ services:
environment:
- REDIS_PASSWORD=${REDIS_PASSWORD:-}
- TURNSTONE_AUTH_TOKEN=${TURNSTONE_AUTH_TOKEN:-}
- TURNSTONE_JWT_SECRET=${TURNSTONE_JWT_SECRET:-}
networks:
- turnstone-net
depends_on:
@@ -177,6 +180,9 @@ services:
- REDIS_PASSWORD=${REDIS_PASSWORD:-}
- TURNSTONE_AUTH_ENABLED=${TURNSTONE_AUTH_ENABLED:-}
- TURNSTONE_AUTH_TOKEN=${TURNSTONE_AUTH_TOKEN:-}
- TURNSTONE_JWT_SECRET=${TURNSTONE_JWT_SECRET:-}
- TURNSTONE_DB_BACKEND=${DB_BACKEND:-sqlite}
- TURNSTONE_DB_URL=${DATABASE_URL:-}
networks:
- turnstone-net
depends_on:
@@ -190,6 +196,41 @@ services:
start_period: 10s
restart: unless-stopped
# -------------------------------------------------------------------
# turnstone-channel — Channel gateway (Discord, Slack, etc.)
# Requires TURNSTONE_DISCORD_TOKEN to enable Discord adapter
# -------------------------------------------------------------------
channel:
build:
context: .
dockerfile: Dockerfile
profiles:
- production
command:
- sh
- -c
- >-
turnstone-channel
--redis-host=redis
--redis-port=6379
$${TURNSTONE_DISCORD_GUILD:+--discord-guild $$TURNSTONE_DISCORD_GUILD}
environment:
- TURNSTONE_DISCORD_TOKEN=${TURNSTONE_DISCORD_TOKEN:-}
- TURNSTONE_DISCORD_GUILD=${TURNSTONE_DISCORD_GUILD:-0}
- REDIS_PASSWORD=${REDIS_PASSWORD:-}
- TURNSTONE_JWT_SECRET=${TURNSTONE_JWT_SECRET:-}
- TURNSTONE_DB_BACKEND=${DB_BACKEND:-sqlite}
- TURNSTONE_DB_URL=${DATABASE_URL:-}
networks:
- turnstone-net
depends_on:
redis:
condition: service_healthy
postgres:
condition: service_healthy
required: false
restart: unless-stopped
# -------------------------------------------------------------------
# turnstone-sim — Multi-node cluster simulator (no LLM needed)
# Start with: docker compose --profile sim up
+183 -12
View File
@@ -56,6 +56,170 @@ console.log(result.content);
---
## Authentication
When auth is enabled (`[auth].enabled = true` or `TURNSTONE_AUTH_ENABLED=1`), all API endpoints except public paths require a valid token.
### Sending Credentials
Include a token in one of two ways:
- **Bearer header**: `Authorization: Bearer <token>`
- **Cookie**: `turnstone_auth=<token>` (set automatically by the login endpoint)
The server accepts three token types:
| Type | Format | Example |
|------|--------|---------|
| JWT | Base64 segments separated by dots | `eyJhbG...` |
| API token | `ts_` prefix + 64 hex chars | `ts_a1b2c3d4...` |
| Config token | Arbitrary string from `config.toml` | `my-secret-token` |
JWTs are the recommended credential for browser sessions. API tokens are suitable for programmatic access and CI/CD. Config tokens are a simple option for single-node deployments.
### `POST /v1/api/auth/login`
Authenticate with credentials and receive a JWT. Accepts two credential formats:
**Username + password:**
```json
{"username": "alice", "password": "hunter2"}
```
**API token:**
```json
{"token": "ts_a1b2c3d4e5f6..."}
```
**Response (success):** `200`
```json
{
"status": "ok",
"role": "full",
"scopes": "approve,read,write",
"jwt": "eyJhbGciOiJIUzI1NiIs...",
"user_id": "u_abc123"
}
```
The response also sets a `turnstone_auth` HttpOnly cookie containing the JWT.
**Response (failure):** `401`
```json
{"error": "Invalid credentials"}
```
---
### `POST /v1/api/auth/logout`
Clears the `turnstone_auth` cookie. No request body required.
**Response:** `200`
```json
{"status": "ok"}
```
The response includes a `Set-Cookie` header that expires the auth cookie.
---
### `GET /v1/api/auth/status`
Returns the current authentication state. Works with or without a valid token.
**Response (authenticated):** `200`
```json
{
"authenticated": true,
"user_id": "u_abc123",
"scopes": ["approve", "read", "write"],
"source": "jwt"
}
```
**Response (not authenticated):** `200`
```json
{
"authenticated": false,
"user_id": null,
"scopes": [],
"source": null
}
```
**Response (auth disabled):** `200`
```json
{
"authenticated": false,
"auth_enabled": false
}
```
---
### `POST /v1/api/auth/setup`
Creates the first admin user when no users exist in the database. This is a
public endpoint (no authentication required) that only succeeds when auth is
enabled and the user database is empty. Both the server and console expose
this endpoint.
**Request body:**
```json
{
"username": "admin",
"display_name": "Admin",
"password": "strongpass"
}
```
| Field | Type | Required | Validation |
|----------------|--------|----------|-----------------------------|
| `username` | string | yes | 1-64 ASCII characters |
| `display_name` | string | yes | Non-empty |
| `password` | string | yes | Minimum 8 characters |
**Response (success):** `200`
```json
{
"status": "ok",
"user_id": "u_abc123",
"username": "admin",
"role": "full",
"scopes": "approve,read,write",
"jwt": "eyJhbGciOiJIUzI1NiIs..."
}
```
The response also sets a `turnstone_auth` HttpOnly cookie containing the JWT.
**Response (already set up):** `409`
```json
{"error": "Setup already completed"}
```
Returned when one or more users already exist in the database.
**Response (auth disabled):** `400`
```json
{"error": "Auth is not enabled"}
```
---
## Endpoints
### `GET /`
@@ -394,12 +558,14 @@ Each session object:
| Field | Type | Description |
|-----------------|-------------|--------------------------------------------|
| `session_id` | string | Unique 12-char hex session identifier |
| `session_id` | string | Unique 32-char hex UUID session identifier |
| `alias` | string/null | User-assigned short name |
| `title` | string/null | LLM-generated title |
| `created` | string | ISO timestamp of session creation |
| `updated` | string | ISO timestamp of last message |
| `message_count` | int | Number of messages in the session |
| `node_id` | string/null | Server node that created the session |
| `ws_id` | string/null | Workstream the session belongs to |
---
@@ -550,22 +716,25 @@ Creates a new workstream. The server supports up to 10 concurrent workstreams.
All fields are optional. The body can be empty or an empty JSON object.
| Field | Type | Default | Description |
|----------------|--------|---------|------------------------------------------------|
| `name` | string | auto | Workstream display name |
| `model` | string | default | Model alias from the registry (`[models.*]`) |
| `auto_approve` | bool | false | Auto-approve all tool calls for this workstream |
| Field | Type | Default | Description |
|------------------|--------|---------|----------------------------------------------------------------|
| `name` | string | auto | Workstream display name |
| `model` | string | default | Model alias from the registry (`[models.*]`) |
| `auto_approve` | bool | false | Auto-approve all tool calls for this workstream |
| `resume_session` | string | "" | Session ID to resume atomically during creation (empty = fresh)|
**Response (success):**
```json
{"ws_id": "ghi789", "name": "ws-3"}
{"ws_id": "ghi789", "name": "ws-3", "resumed": false, "message_count": 0}
```
| Field | Type | Description |
|---------|--------|------------------------------------|
| `ws_id` | string | Unique ID of the new workstream |
| `name` | string | Auto-generated workstream name |
| Field | Type | Description |
|-----------------|--------|-----------------------------------------------------|
| `ws_id` | string | Unique ID of the new workstream |
| `name` | string | Auto-generated workstream name |
| `resumed` | bool | Whether a previous session was successfully resumed |
| `message_count` | int | Number of messages in the resumed session (0 if fresh) |
**Error (limit reached):**
@@ -692,7 +861,8 @@ liveness probes.
```json
{
"status": "ok",
"version": "0.3.0",
"version": "0.4.0",
"node_id": "worker-01_a3f2",
"uptime_seconds": 3614.72,
"model": "llama-3.1-70b-instruct",
"workstreams": {
@@ -714,6 +884,7 @@ liveness probes.
|-------|------|-------------|
| `status` | string | `"ok"` or `"degraded"` (degraded when backend unreachable) |
| `version` | string | turnstone server version |
| `node_id` | string | Server-generated node identity (`{hostname}_{4hex}`) |
| `uptime_seconds` | number | Seconds since the server process started |
| `model` | string | Model name detected or configured at startup |
| `workstreams.total` | integer | Total active workstreams |
+140 -5
View File
@@ -21,6 +21,8 @@ plugs in.
| `turnstone-bridge` | `turnstone.mq.bridge` | Bridge | Message queue ↔ HTTP API bridge |
| `turnstone-console` | `turnstone.console.server` | ClusterCollector | Cluster dashboard (aggregates all nodes) |
| `turnstone-eval` | `turnstone.eval` | `NullUI` | Headless evaluation and prompt optimization |
| `turnstone-channel` | `turnstone.channels.cli` | ChannelAdapter | Channel gateway (Discord, Slack, etc.) via Redis MQ |
| `turnstone-admin` | `turnstone.core.admin_cli` | — | Offline user and API token management |
---
@@ -75,6 +77,12 @@ turnstone/
collector.py ClusterCollector — aggregates state from all nodes via Redis + HTTP
server.py Cluster dashboard HTTP server + SSE + CLI entry point
static/ Cluster dashboard web UI (page-specific HTML, CSS, JS)
channels/
cli.py Unified channel gateway entry point (turnstone-channel)
_protocol.py ChannelAdapter protocol, ChannelEvent dataclass
_routing.py ChannelRouter — channel/thread ↔ workstream mapping via MQ
_config.py Base ChannelConfig dataclass
discord/ Discord adapter (bot, cog, views, streaming, config)
shared_static/ Shared design system (base.css, auth.js, theme.js, toast.js, utils.js, kb.js)
ui/
colors.py ANSI color constants with NO_COLOR support
@@ -713,7 +721,7 @@ and are the single source of truth for both backends and Alembic migrations.
| Method | Purpose |
|--------|---------|
| `register_session(session_id, title)` | Create a sessions row (no-op if exists) |
| `register_session(session_id, title, node_id, ws_id)` | Create a sessions row (no-op if exists) |
| `save_message(session_id, role, content, ...)` | Log a message to conversations |
| `load_session_messages(session_id)` | Reconstruct OpenAI message format from DB rows |
| `list_sessions(limit)` | List sessions with >=1 message, ordered by updated DESC |
@@ -725,6 +733,11 @@ and are the single source of truth for both backends and Alembic migrations.
| `set_session_alias(session_id, alias)` | Set user-friendly alias (returns False if taken) |
| `get_session_name(session_id)` | Return alias if set, else title, else None |
| `update_session_title(session_id, title)` | Set/update LLM-generated title |
| `register_workstream(ws_id, node_id, name, state)` | Create a workstreams row (no-op if exists) |
| `update_workstream_state(ws_id, state)` | Update workstream state and bump timestamp |
| `update_workstream_name(ws_id, name)` | Update workstream display name |
| `delete_workstream(ws_id)` | Delete a workstream row |
| `list_workstreams(node_id, limit)` | List workstreams, optionally by node |
| `kv_get(key)` / `kv_set(key, value)` / `kv_delete(key)` | Generic key-value store (backs memories table) |
| `kv_list()` / `kv_search(query)` | List or search key-value pairs |
| `search_history(query, limit)` | Full-text search (FTS5 on SQLite, tsvector on PostgreSQL) |
@@ -745,9 +758,11 @@ Environment variables: `TURNSTONE_DB_BACKEND`, `TURNSTONE_DB_URL`, `TURNSTONE_DB
### Session Persistence and Resume
Each `ChatSession` generates a 12-char hex `_session_id` on creation and
registers it in the `sessions` table. Messages are saved to `conversations`
as they happen via `save_message()`.
Each `ChatSession` generates a full 32-char hex UUID `_session_id` on creation
and registers it in the `sessions` table with the server's `node_id` and the
owning `ws_id`. Messages are saved to `conversations` as they happen via
`save_message()`. Workstreams are persisted to the `workstreams` table on
creation, with state changes tracked via `update_workstream_state()`.
**Auto-titling:** After the first complete exchange (user message + assistant
response), a background thread calls the LLM with a title-generation prompt
@@ -910,6 +925,98 @@ limits using a token-bucket algorithm. Each IP gets a `TokenBucket` with
---
## User Identity and Authentication
Turnstone supports three authentication mechanisms, unified behind an
`AuthResult` dataclass that carries `user_id`, `scopes`, and `token_source`:
1. **Config-file tokens** — static secrets in `config.toml` `[[auth.tokens]]`
or the `TURNSTONE_AUTH_TOKEN` env var. Validated in-memory via
`hmac.compare_digest`. Map to scopes through their role (`read` or `full`).
2. **API tokens** — database-backed, prefixed `ts_`, stored as SHA-256 hashes
in the `api_tokens` table. Can be exchanged for JWTs via
`POST /v1/api/auth/login`.
3. **JWTs** — short-lived HMAC-SHA256 session tokens (default 24h) issued after
successful credential validation. Contain `sub` (user_id), `scopes`, and
`src` (origin) in claims.
### Scope Model
Three hierarchical scopes control endpoint access:
| Scope | Grants | Endpoints |
|-------|--------|-----------|
| `read` | SSE streams, workstream listing, sessions | GET endpoints |
| `write` | `read` + send, command, workstream create/close | POST to `/api/send`, `/api/command`, etc. |
| `approve` | `write` + tool approval, admin operations | POST to `/api/approve`, `/api/admin/*` |
### Middleware Flow
`AuthMiddleware` (ASGI) intercepts every request:
1. **Public path check**`/`, `/static/*`, `/shared/*`, `/health`,
`/metrics`, `/openapi.json`, `/docs`, `/api/auth/*`, and `/api/auth/setup`
are always allowed.
2. **Token extraction**`Authorization: Bearer <token>` header first, then
`turnstone_auth` cookie as fallback.
3. **Token type detection** — dots in the token indicate JWT; `ts_` prefix
indicates API token; otherwise config-file token.
4. **Validation** — JWT signature check, API token hash lookup in storage, or
config-token hmac comparison.
5. **Scope check**`required_scope(method, path)` determines the minimum
scope; the request is rejected with 403 if the token lacks it.
6. **Context propagation** — on success, `ctx_user_id` is set so structured
logging includes the authenticated identity on every log event.
### Architecture Split
- **Console** is the auth management hub — it hosts the admin endpoints for
creating users, issuing API tokens, and managing channel mappings. User
records and token hashes live in the shared storage backend. The console
dashboard includes an **admin panel** (Users and Tokens tabs) for managing
credentials through the browser.
- **Server** is a JWT validator only — it validates tokens on each request but
never creates users or tokens. Both processes share the same `jwt_secret`
(via `TURNSTONE_JWT_SECRET` env var or `[auth].jwt_secret` config).
- **First-time setup** — both server and console expose
`POST /v1/api/auth/setup`, a public endpoint that creates the initial admin
user when no users exist. This avoids the chicken-and-egg problem of needing
`approve` scope to create the first user via `/api/admin/users`.
### Auth Storage Tables
Three tables in `storage/_schema.py` support identity:
```sql
users
user_id TEXT PRIMARY KEY
username TEXT NOT NULL UNIQUE
display_name TEXT NOT NULL
password_hash TEXT NOT NULL -- bcrypt
created TEXT NOT NULL
api_tokens
token_id TEXT PRIMARY KEY
token_hash TEXT NOT NULL UNIQUE -- SHA-256 of raw token
token_prefix TEXT NOT NULL -- first 8 chars for display
user_id TEXT NOT NULL
name TEXT NOT NULL -- human-readable label
scopes TEXT NOT NULL -- comma-separated
created TEXT NOT NULL
expires TEXT -- optional expiry timestamp
channel_users
channel_type TEXT NOT NULL -- e.g. "slack", "discord"
channel_user_id TEXT NOT NULL -- platform-specific user ID
user_id TEXT NOT NULL -- FK to users
PRIMARY KEY (channel_type, channel_user_id)
```
See [docs/security.md](security.md) for full security details including token
lifecycle, password hashing, and deployment hardening.
---
## Threading Model
### CLI
@@ -1037,7 +1144,10 @@ a response or the approval timeout (default 3600s / 1 hour) expires.
`ws_id` for active sends. When the global SSE reports `ws_state → idle` for a tracked
workstream, the bridge emits a synthetic `TurnCompleteEvent` with the correlation ID.
**Multi-node routing:** Each bridge has a `node_id` (defaults to hostname) and BLPOPs
**Multi-node routing:** Each bridge retrieves its `node_id` from the server's
`/health` endpoint on startup (with exponential backoff retry). The server
generates the `node_id` (`{hostname}_{4hex}`) and is the sole authority for
node identity. The bridge BLPOPs
from both `turnstone:inbound:{node_id}` (directed, priority) and `turnstone:inbound` (shared).
Messages with `target_node` set are pushed to the target's per-node queue. Messages
for existing workstreams are auto-routed via `turnstone:ws:{ws_id}` ownership keys in Redis.
@@ -1152,3 +1262,28 @@ with TurnstoneServer("http://localhost:8080", token="tok_xxx") as client:
result = client.send_and_wait("Hello!", ws.ws_id)
print(result.content)
```
---
## Channel Integrations
> See also: [Channel Integrations guide](channels.md)
The `turnstone-channel` gateway bridges external messaging platforms
(Discord, Slack, Teams) to the turnstone cluster via Redis MQ. Each
platform adapter implements the `ChannelAdapter` protocol and translates
between platform-native events and turnstone MQ messages.
The `ChannelRouter` manages bidirectional routing: it maps platform
channel/thread IDs to turnstone workstream IDs, handles workstream
creation and stale-route recovery, and resolves platform users to
turnstone identities via the `channel_users` table. When an evicted
workstream is reactivated, the router uses atomic session resume via the
`resume_session` field on `CreateWorkstreamMessage` — the server resumes
the old session during workstream creation in a single HTTP request,
eliminating ordering fragility. The bridge emits a `SessionResumedEvent`
to confirm success.
Discord ships as the first adapter. See [channels.md](channels.md) for
setup instructions, configuration reference, and the adapter development
guide.
+272
View File
@@ -0,0 +1,272 @@
# Channel Integrations
The `turnstone-channel` gateway connects external messaging platforms to
turnstone workstreams via Redis MQ. Each platform adapter translates
platform-native events (messages, button clicks, slash commands) into
turnstone MQ messages, and renders workstream output back into the
platform's UI.
Discord ships as the first adapter. The adapter protocol is designed for
future Slack and Teams integrations.
---
## Architecture
```
Discord Gateway
|
v
turnstone-channel (Discord adapter)
|
v
Redis MQ
|
v
turnstone-bridge ──> turnstone-server
```
Key components:
- **ChannelAdapter protocol** (`turnstone/channels/_protocol.py`) — generic
interface for any messaging platform. Defines `start()`, `stop()`,
`send()`, `edit_message()`, `send_approval_request()`,
`send_plan_review()`, and `create_thread()`.
- **ChannelRouter** (`turnstone/channels/_routing.py`) — maps
channel/thread IDs to turnstone workstream IDs. Handles workstream
creation via MQ, stale route detection, and user identity resolution.
- **AsyncRedisBroker** (`turnstone/mq/async_broker.py`) — async Redis
client compatible with discord.py's event loop. Used by the router for
pub/sub and queue operations.
- **channel_users table** — maps `(channel_type, channel_user_id)` to a
turnstone `user_id`. Messages from unlinked users are silently dropped.
- **channel_routes table** — persistent channel-to-workstream mappings.
Survives bot restarts. Stale routes (evicted workstreams) are detected
and refreshed on the next message.
---
## Discord Setup
### 1. Create a Discord Application
1. Go to https://discord.com/developers/applications
2. Click **New Application** and give it a name
3. Navigate to the **Bot** tab and click **Reset Token** to generate a
bot token. Copy it immediately — it is shown only once.
4. On the same **Bot** tab, scroll down to **Privileged Gateway Intents**
and enable **MESSAGE CONTENT INTENT**
5. Navigate to **OAuth2 > URL Generator**
6. Under **Scopes**, check `bot` and `applications.commands`
7. Under **Bot Permissions**, check:
- View Channels
- Send Messages
- Send Messages in Threads
- Create Public Threads
- Read Message History
- Add Reactions
- Embed Links
8. Copy the generated URL, open it in a browser, and add the bot to your
Discord server
### 2. Configure Turnstone
**Environment variables** (recommended for Docker):
```bash
TURNSTONE_DISCORD_TOKEN=your-bot-token-here
TURNSTONE_DISCORD_GUILD=123456789 # optional, restrict to one guild
```
**CLI flags** (bare-metal):
```bash
turnstone-channel \
--discord-token "your-bot-token" \
--discord-guild 123456789 \
--redis-host localhost \
--redis-port 6379
```
**Docker Compose** (production profile):
```bash
# In .env file:
TURNSTONE_DISCORD_TOKEN=your-bot-token
TURNSTONE_DISCORD_GUILD=123456789
```
Then start the stack:
```bash
docker compose --profile production up
```
The `channel` service starts automatically when
`TURNSTONE_DISCORD_TOKEN` is set.
### 3. Link User Accounts
Discord users must link their account to a turnstone user before they can
interact with the bot. Unlinked users' messages are silently ignored.
1. The user must have a turnstone API token — created via the admin panel
or `turnstone-admin create-token`
2. In Discord, the user runs `/link`. A modal appears prompting for the
API token (the token is never visible in Discord audit logs because it
is submitted via modal, not as a slash command argument).
3. The token is validated against the database. If valid, a
`channel_users` mapping is created.
4. The user can now @mention the bot or use slash commands.
An admin can also force-link or unlink users via the console admin panel
(Admin > Channels tab).
---
## Usage
### Conversations
- **@mention** the bot in any allowed channel to start a new conversation.
The bot creates a Discord thread from the message and a turnstone
workstream behind it.
- All subsequent messages in the thread are routed to the same workstream.
- The bot streams responses via message edits, updated approximately every
1.5 seconds.
- If the workstream is evicted for capacity, the next message in the
thread auto-creates a new workstream and atomically resumes the
previous session via the `resume_session` field on
`CreateWorkstreamMessage`. The server resumes the session during
workstream creation (same HTTP request), and the bridge emits a
`SessionResumedEvent` back to the channel. The thread receives a
*"Session resumed: {name} ({count} messages restored)"* confirmation.
### Slash Commands
| Command | Description |
|---------|-------------|
| `/link` | Link Discord account to turnstone (opens modal for API token) |
| `/unlink` | Unlink Discord account |
| `/ask <message>` | Create a new thread and workstream with an initial message |
| `/status` | Show workstream info for the current thread (ephemeral) |
| `/close` | Close the workstream, delete the route, and archive the thread |
### Tool Approvals
When manual approval is enabled (the default), tool calls are displayed as
an orange embed with:
- Tool name and argument preview
- **Approve** (green), **Reject** (red), **Always Approve** (gray) buttons
- Only linked users can interact with approval buttons
- The approval decision is forwarded through MQ to the bridge, which
relays it to the server
Buttons use static `custom_id` values so they survive bot restarts.
Correlation data (`ws_id`, `correlation_id`) is stored in the embed footer.
**Auto-approval:** When `auto_approve` is true (via `--auto-approve`), or when
all tools in the request match the `auto_approve_tools` list in the adapter
config, the bot auto-responds with approval and posts a
"*Tool auto-approved.*" notice to the thread instead of showing buttons. The
`auto_approve_tools` list is set via the `ChannelConfig.auto_approve_tools`
field (useful for allowing specific tools like `bash` or `read_file` while
still requiring manual approval for others).
### Plan Reviews
Plan review requests are displayed as a blue embed with:
- **Approve Plan** (green) button — approves the plan with empty feedback
- **Request Changes** (gray) button — opens a modal for feedback text
(up to 2000 characters)
- Feedback is forwarded through MQ as a `PlanFeedbackMessage`
---
## Configuration Reference
| CLI Flag | Env Var | Default | Description |
|----------|---------|---------|-------------|
| `--discord-token` | `TURNSTONE_DISCORD_TOKEN` | — | Bot token (required to enable Discord) |
| `--discord-guild` | — | `0` (all guilds) | Restrict to a single Discord guild |
| `--discord-channels` | — | empty (all) | Comma-separated channel IDs to allow |
| `--redis-host` | `REDIS_HOST` | `localhost` | Redis host |
| `--redis-port` | — | `6379` | Redis port |
| `--redis-password` | `REDIS_PASSWORD` | — | Redis password |
| `--redis-db` | — | `0` | Redis DB number |
| `--model` | — | server default | Default model for new workstreams |
| `--auto-approve` | — | `false` | Auto-approve ALL tool calls (skips approval buttons entirely) |
| `--log-level` | `TURNSTONE_LOG_LEVEL` | `INFO` | Log level |
| `--log-format` | `TURNSTONE_LOG_FORMAT` | `auto` | Log format (`auto`/`json`/`text`) |
---
## User Identity
- The `channel_users` table maps `(channel_type, channel_user_id)` to a
turnstone `user_id`
- Self-service linking via the `/link` slash command (modal input, not
visible in Discord audit logs)
- Admin can force-link or unlink via the console admin panel (Admin >
Channels tab). Unlinking uses a styled confirmation modal.
- Unlinked users' messages are silently dropped
- A user can be linked across multiple platforms (e.g. Discord + Slack)
See [Security: Database Schema](security.md#database-schema) for the
`channel_users` table definition.
---
## Workstream Lifecycle
1. **Creation**@mention or `/ask` creates a Discord thread and a
turnstone workstream. The `ChannelRouter` persists the mapping in the
`channel_routes` table.
2. **Active** — messages are routed bidirectionally. The bot streams
responses via message edits (updated every ~1.5 seconds).
3. **Eviction** — the server evicts an idle workstream for capacity. The
route is preserved and the thread stays open.
4. **Reactivation** — the next message in the thread detects the stale
route (no MQ owner), looks up the old session via
`get_session_id_by_ws()`, and creates a new workstream with
`resume_session` set atomically on the `CreateWorkstreamMessage`. The
server resumes the session during creation (no separate command
needed). The bridge emits a `SessionResumedEvent` to the channel, and
the thread displays *"Session resumed: {name} ({count} messages
restored)"*. If the old session was pruned, the workstream starts
fresh with no error.
5. **Close**`/close` command closes the workstream via MQ, deletes the
route, unsubscribes from events, and archives the Discord thread.
---
## Adding New Adapters
The `ChannelAdapter` protocol defines the interface any platform adapter
must implement:
```python
class ChannelAdapter(Protocol):
channel_type: str
async def start(self) -> None: ...
async def stop(self) -> None: ...
async def send(self, channel_id: str, content: str) -> str: ...
async def edit_message(self, channel_id: str, message_id: str, content: str) -> None: ...
async def send_approval_request(self, channel_id: str, ws_id: str, correlation_id: str, items: list[dict]) -> None: ...
async def send_plan_review(self, channel_id: str, ws_id: str, correlation_id: str, content: str) -> None: ...
async def create_thread(self, parent_channel_id: str, name: str, message_id: str = "") -> str: ...
```
To add a new platform:
1. Create `turnstone/channels/<platform>/` package
2. Implement the `ChannelAdapter` protocol
3. Add a `--<platform>-token` flag and detection logic in
`turnstone/channels/cli.py`
4. Add the optional dependency in `pyproject.toml` (e.g.
`turnstone[slack]`)
See `turnstone/channels/discord/` as a reference implementation.
+147 -4
View File
@@ -148,7 +148,7 @@ Single node detail with all its workstreams.
### `POST /v1/api/cluster/workstreams/new`
Create a new workstream on a target node. Dispatches a `CreateWorkstreamMessage` through the Redis MQ pipeline — the bridge on the target node picks it up and creates the workstream on the server. Requires `"full"` auth role.
Create a new workstream on a target node. Dispatches a `CreateWorkstreamMessage` through the Redis MQ pipeline — the bridge on the target node picks it up and creates the workstream on the server. Requires `write` scope.
Request:
@@ -207,6 +207,96 @@ Keepalive comments (`: keepalive\n\n`) are sent every 5 seconds. Clients should
}
```
### Admin API
User and token management endpoints. All admin endpoints require `approve` scope, except for the setup endpoint which is public.
#### `POST /v1/api/auth/setup`
Creates the first admin user when no users exist. Public endpoint (no auth required). Returns a JWT and sets a session cookie. Returns `409` if users already exist. See [Security: First-time setup](security.md#first-time-setup) for full details.
#### `POST /v1/api/admin/users`
Create a new user.
```json
{
"username": "alice",
"password": "s3cret",
"scopes": ["read", "write"]
}
```
#### `GET /v1/api/admin/users`
List all users.
```json
{
"users": [
{"user_id": "u_abc123", "username": "alice", "scopes": ["read", "write"], "created": "2026-03-01T12:00:00Z"}
]
}
```
#### `DELETE /v1/api/admin/users/{user_id}`
Delete a user and revoke all their tokens.
#### `POST /v1/api/admin/users/{user_id}/tokens`
Create an API token for the given user. Returns a `ts_`-prefixed token string that can be used for Bearer auth or passed to `client.login(token="ts_xxx")`.
```json
{
"name": "CI pipeline",
"scopes": ["read", "write"]
}
```
#### `GET /v1/api/admin/users/{user_id}/tokens`
List active tokens for a user (token strings are not returned, only metadata).
#### `DELETE /v1/api/admin/tokens/{token_id}`
Revoke a specific API token.
### Channel links
| Method | Path | Description |
|--------|------|-------------|
| GET | `/v1/api/admin/users/{user_id}/channels` | List channel links for a user |
| POST | `/v1/api/admin/users/{user_id}/channels` | Link a channel account (channel_type, channel_user_id) |
| DELETE | `/v1/api/admin/channels/{channel_type}/{channel_user_id}` | Unlink a channel account |
These endpoints manage the `channel_users` table mappings that connect external platform identities (e.g. Discord user IDs) to turnstone users. See [Channel Integrations](channels.md) for details on the linking flow.
#### `GET /v1/api/auth/status`
Public endpoint for login UI state detection. Returns auth configuration, not
current-user identity.
```json
{
"auth_enabled": true,
"has_users": true,
"setup_required": false
}
```
### Auth Scopes
The auth system uses three scopes instead of the earlier read/full role model:
| Scope | Grants |
|-------|--------|
| `read` | Read-only access: dashboards, workstream lists, SSE streams, health |
| `write` | Send messages, create/close workstreams, approve tool calls |
| `approve` | Admin operations: manage users and API tokens |
Scopes are cumulative — a user with `approve` scope can also perform `write` and `read` operations.
---
## Reverse Proxy
@@ -240,13 +330,13 @@ SSE streams (`/v1/api/events`, `/v1/api/events/global`) are proxied by creating
### Authentication
The proxy forwards requests to server nodes using the console's `--auth-token`. The console's own auth middleware also checks proxy routes — `POST` requests to proxy write endpoints (`/v1/api/send`, `/v1/api/approve`, etc.) require the `"full"` auth role, preventing read-only tokens from escalating to write operations.
The proxy forwards the user's JWT to upstream server nodes — it extracts the token from the incoming request's cookie (or `Authorization` header) and adds it as a `Bearer` header on the proxied request. Since all services share the same `TURNSTONE_JWT_SECRET`, the user's JWT is valid on every node without re-authentication. The console's own auth middleware also checks proxy routes — `POST` requests to proxy write endpoints (`/v1/api/send`, `/v1/api/approve`, etc.) require `write` scope, preventing read-only tokens from escalating via proxy. The static `--auth-token` / `proxy_auth_token` is used as a fallback when no user JWT is present.
---
## Browser Dashboard
The web UI has four views, toggled client-side:
The web UI has five views, toggled client-side:
### 1. Cluster Overview (landing)
@@ -276,7 +366,60 @@ Triggered by the "+ new" header button. A modal dialog with:
On submit, `POST /v1/api/cluster/workstreams/new` dispatches the creation request. A toast confirms success; the SSE stream delivers the `ws_created` event to update the dashboard.
All four views receive live updates via SSE — state cards update counts, node rows update metrics, workstream rows update state indicators.
All five views receive live updates via SSE — state cards update counts, node rows update metrics, workstream rows update state indicators.
### 5. Admin Panel
Accessed via the "admin" button in the header (visible when authenticated
with `approve` scope). Provides user, API token, and channel link management
with three tabs:
**Users tab:**
- Grid table listing all users (username, display name, role, creation date)
- "Create User" button opens a modal with fields for username, display name,
and password (validated: username 1-64 ASCII, password min 8 characters)
- Delete button on each row opens a styled confirmation modal before
removing the user and cascading to revoke all their tokens
**Tokens tab:**
- User selector dropdown to pick which user's tokens to manage
- Grid table listing tokens for the selected user (name, prefix, scopes,
creation date)
- Scope badges rendered as colored pills for visual clarity
- "Create Token" button opens a modal with fields for token name and scope
checkboxes
- On creation, a "Token Created" modal displays the raw `ts_`-prefixed
token with a copy button. The token is shown once and cannot be retrieved
again.
- Revoke button on each row opens a styled confirmation modal before
deleting the token
**Channels tab:**
- User selector dropdown to pick which user's channel links to manage
- Grid table listing linked channel accounts for the selected user
(channel type, channel user ID, creation date)
- "Link Channel" button opens a modal with fields for channel type
(e.g. `discord`) and the platform user ID
- Unlink button on each row opens a styled confirmation modal before
removing the channel mapping
- Admins can force-link users who have not self-linked via `/link` in
Discord
**Accessibility:**
- Full keyboard navigation: focus traps in modals, Escape to close, arrow
keys for tab switching
- Responsive layout with column hiding at 700px breakpoint
**First-time setup:**
The console also exposes `POST /v1/api/auth/setup` for first-time
bootstrap. When no users exist, the setup wizard calls this public endpoint
to create the initial admin user and receive a JWT in one step. See
[Security: First-time setup](security.md#first-time-setup) for details.
---
-3
View File
@@ -1,3 +0,0 @@
version https://git-lfs.github.com/spec/v1
oid sha256:d5a2bd1c55ac8cf3b777a8decb6f3bb3d063c10c8f3a9e63457079830e48f456
size 162310
-3
View File
@@ -1,3 +0,0 @@
version https://git-lfs.github.com/spec/v1
oid sha256:09cee5819bb7820641a466ed29a53f1b479bc2ded83079fc798c7410bf741a62
size 329703
-3
View File
@@ -1,3 +0,0 @@
version https://git-lfs.github.com/spec/v1
oid sha256:b3f76042c8046560502fa3132351821d56e8526b13d38d7be8b839fcf3d5f648
size 373463
-3
View File
@@ -1,3 +0,0 @@
version https://git-lfs.github.com/spec/v1
oid sha256:fca0957b54101ce5c2b2e06d639b04dc5fff733f2641af0d732c49ea86883882
size 279397
-3
View File
@@ -1,3 +0,0 @@
version https://git-lfs.github.com/spec/v1
oid sha256:06fb9faa29c11395c6fc54ebddc79994b000dee78d56e0c13cb689fd6a82e37a
size 237255
-3
View File
@@ -1,3 +0,0 @@
version https://git-lfs.github.com/spec/v1
oid sha256:13b2da77312e5abb44c81aabb7f0addccab31d9bc7e8ef2f1c3563ef985ed503
size 186941
+20 -3
View File
@@ -13,7 +13,7 @@ skinparam class {
' -- Protocol --
interface "StorageBackend" as SB <<protocol>> {
+register_session(session_id, title)
+register_session(session_id, title, node_id, ws_id)
+save_message(session_id, role, content, ...)
+load_session_messages(session_id) → list[dict]
+list_sessions(limit) → list
@@ -25,6 +25,11 @@ interface "StorageBackend" as SB <<protocol>> {
+set_session_alias(session_id, alias) → bool
+get_session_name(session_id) → str | None
+update_session_title(session_id, title)
+register_workstream(ws_id, node_id, name, state)
+update_workstream_state(ws_id, state)
+update_workstream_name(ws_id, name)
+delete_workstream(ws_id) → bool
+list_workstreams(node_id, limit) → list
+kv_get(key) → str | None
+kv_set(key, value) → str | None
+kv_delete(key) → bool
@@ -32,6 +37,11 @@ interface "StorageBackend" as SB <<protocol>> {
+kv_search(query) → list[(str, str)]
+search_history(query, limit) → list
+search_history_recent(limit) → list
+create_user(user_id, username, display_name, pw_hash)
+get_user(user_id) / get_user_by_username(username)
+list_users() / delete_user(user_id)
+create_api_token(...) / get_api_token_by_hash(hash)
+list_api_tokens(user_id) / delete_api_token(id)
+close()
}
@@ -58,8 +68,12 @@ class "_schema.py" as Schema <<schema>> {
+metadata: MetaData
+memories: Table
+conversations: Table
+sessions: Table
+sessions: Table (node_id, ws_id, user_id)
+workstreams: Table (node_id, user_id, state)
+session_config: Table
+users: Table (username, password_hash)
+api_tokens: Table (token_hash, scopes)
+channel_users: Table (channel_type)
--
SQLAlchemy Core
Single source of truth
@@ -76,6 +90,7 @@ class "_migrate.py" as Migrate <<migration>> {
class "migrations/" as Versions <<migration>> {
001_initial_schema.py
002_user_identity.py
}
' -- Registry --
@@ -94,9 +109,11 @@ class "memory.py" as Facade <<facade>> {
+register_session()
+save_message()
+load_session_messages()
+register_workstream()
+update_workstream_state()
+save_memory() / delete_memory()
+search_memories()
+... (all 18 functions)
+... (all 22 functions)
--
Thin delegation to
get_storage()
+179
View File
@@ -0,0 +1,179 @@
@startuml
!theme plain
title Turnstone — Authentication Architecture
skinparam class {
BackgroundColor<<core>> #E8EAF6
BackgroundColor<<jwt>> #C8E6C9
BackgroundColor<<storage>> #B3E5FC
BackgroundColor<<endpoint>> #FFE0B2
BackgroundColor<<scope>> #F3E5F5
}
' -- Core Auth --
class "AuthConfig" as AC <<core>> {
+enabled: bool
+tokens: dict[str, str]
+check(token) → role | None
--
Static config-file tokens
hmac.compare_digest
}
class "AuthResult" as AR <<core>> {
+user_id: str
+scopes: frozenset[str]
+token_source: str
+has_scope(scope) → bool
}
class "check_request()" as CR <<core>> {
auth_config, method, path,
auth_header, cookie_header,
jwt_secret, storage
→ (allowed, status, msg, AuthResult)
--
1. Auth disabled → allow
2. Public path → allow
3. Extract Bearer / cookie
4. Detect token type
5. Validate → AuthResult
6. Check scope vs path
}
' -- Token Types --
class "JWT (HS256)" as JWT <<jwt>> {
sub: user_id
scopes: "read,write,approve"
src: "password" | "database"
iat, exp (24h default)
--
Detected by: contains "."
Validated locally
No DB call
}
class "API Token" as AT <<jwt>> {
Format: ts_ + 64 hex
Stored: SHA-256 hash
--
Detected by: starts with "ts_"
Lookup by hash in DB
Expiry check
}
class "Config Token" as CT <<core>> {
Raw value in memory
Role: "read" | "full"
--
Detected by: fallback
hmac.compare_digest
No DB needed
}
' -- Scopes --
class "Scope Hierarchy" as SH <<scope>> {
read: {read}
write: {read, write}
approve: {read, write, approve}
--
GET → read
POST write paths → write
POST /api/approve → approve
/api/admin/* → approve
}
' -- Storage --
class "users" as UT <<storage>> {
user_id (PK)
username (unique)
display_name
password_hash (bcrypt)
created
}
class "api_tokens" as TT <<storage>> {
token_id (PK)
token_hash (SHA-256, unique)
token_prefix
user_id → users
name, scopes
created, expires
}
' -- Endpoints --
class "POST /api/auth/login" as Login <<endpoint>> {
{username, password}
OR {token: "ts_xxx"}
→ {jwt, role, scopes, user_id}
--
Sets HttpOnly cookie
}
class "GET /api/auth/status" as Status <<endpoint>> {
→ {auth_enabled, has_users,
setup_required}
--
Public (no auth)
Drives UI setup wizard
}
class "POST /api/auth/setup" as Setup <<endpoint>> {
{username, display_name, password}
→ {jwt, user_id, scopes}
--
Public (no auth)
Only when zero users exist
Returns 409 if already set up
}
class "Admin API (Console)" as Admin <<endpoint>> {
POST/GET/DELETE users
POST/GET tokens
DELETE tokens/{id}
--
Requires approve scope
}
' -- Relationships --
CR --> AC : config tokens
CR --> JWT : validate
CR --> AT : hash lookup
CR --> CT : hmac check
CR --> AR : returns
CR --> SH : checks
Login --> JWT : issues
Login --> UT : verify password
Login --> TT : verify API token
Setup --> UT : create first user
Setup --> JWT : issues
AT --> TT : lookup by hash
Admin --> UT : CRUD
Admin --> TT : CRUD
AR --> SH : scopes from
JWT ..> AR : produces
AT ..> AR : produces
CT ..> AR : produces
note right of CR
**Middleware Flow**
AuthMiddleware on every request:
1. Extract token from header/cookie
2. Detect type (JWT / ts_ / config)
3. Validate → AuthResult
4. Set ctx_user_id for logging
5. Store auth_result in scope state
end note
note bottom of SH
**Console** owns admin endpoints
**Server** validates JWT + config only
Both share JWT signing secret
end note
@enduml
+211
View File
@@ -0,0 +1,211 @@
@startuml
!theme plain
title Turnstone — Channel Integration Architecture
skinparam class {
BackgroundColor<<platform>> #E1BEE7
BackgroundColor<<service>> #E8EAF6
BackgroundColor<<mq>> #FFCDD2
BackgroundColor<<bridge>> #C8E6C9
BackgroundColor<<server>> #FFE0B2
BackgroundColor<<storage>> #B3E5FC
}
' -- External Platforms --
class "Discord" as Discord <<platform>> {
Gateway WebSocket (v10)
Message events
Interaction callbacks (buttons)
Thread-per-workstream
--
discord.py 2.x
asyncio event loop
}
class "Slack (future)" as Slack <<platform>> {
Socket Mode / Events API
Block Kit messages
--
Planned integration
}
class "Teams (future)" as Teams <<platform>> {
Bot Framework
Adaptive Cards
--
Planned integration
}
' -- Channel Service --
class "turnstone-channel" as ChannelService <<service>> {
entry point: turnstone-channel
--
One process per platform
asyncio event loop
Structured logging (structlog)
--log-level, --log-format
}
class "DiscordBot" as Bot <<service>> {
+on_message(msg)
+on_interaction(interaction)
+run(token)
--
discord.py Client
Receives message events
Sends replies + embeds
Creates threads for workstreams
Renders approval buttons
}
class "ChannelRouter" as Router <<service>> {
+resolve_route(platform, channel_id)
→ ws_id | None
+register_route(channel_id, ws_id)
+resolve_identity(platform, platform_user_id)
→ user_id | None
--
Maps channels → workstreams
Maps platform users → turnstone users
Caches routes in memory
}
class "AsyncRedisBroker" as Broker <<service>> {
+push_inbound(msg)
+subscribe(ws_id) → AsyncIterator
+subscribe_global() → AsyncIterator
+push_response(correlation_id, msg)
--
redis.asyncio client
Pub/sub + queue operations
}
' -- Redis MQ --
class "Redis MQ" as Redis <<mq>> {
turnstone:inbound (LIST)
turnstone:events:{ws_id} (PUBSUB)
turnstone:events:global (PUBSUB)
turnstone:resp:{corr_id} (LIST)
--
Shared message bus
Same queues as bridge protocol
}
' -- Bridge + Server --
class "turnstone-bridge" as Bridge <<bridge>> {
BLPOP turnstone:inbound
Drive server via HTTP
Relay SSE → Redis pub/sub
--
Owns workstream lifecycle
Auto-approve / manual approve
}
class "turnstone-server" as Server <<server>> {
POST /v1/api/send
POST /v1/api/approve
POST /v1/api/workstreams/new
GET /v1/api/events?ws_id=
--
LLM execution + tool use
SSE event stream
}
' -- Storage --
class "channel_users" as CU <<storage>> {
channel_user_id (PK)
platform: "discord" | "slack"
platform_user_id
user_id → users
linked_at
--
/link command creates row
Resolved on each inbound message
}
class "channel_routes" as CR <<storage>> {
channel_type (PK)
channel_id (PK)
ws_id
node_id
created
--
Maps platform channels
to turnstone workstreams
}
' -- Relationships --
Discord --> Bot : gateway\nevents
Bot --> Router : on_message\non_interaction
Router --> Broker : SendMessage\nApproveMessage
Router --> CU : resolve identity
Router --> CR : resolve / register route
Broker --> Redis : RPUSH inbound\nRPUSH resp:{id}
Redis --> Bridge : BLPOP inbound
Bridge --> Server : HTTP API
Server --> Bridge : SSE events
Bridge --> Redis : PUBLISH events:{ws_id}\nPUBLISH events:global
Redis --> Broker : SUBSCRIBE events:{ws_id}
Broker --> Bot : event stream
Bot --> Discord : reply / embed\nbutton callback
Slack .[hidden]. Discord
Teams .[hidden]. Slack
ChannelService --> Bot : creates + runs
ChannelService --> Router : creates
ChannelService --> Broker : creates
' -- Notes --
note right of Bot
**Inbound Flow**
1. Discord message arrives via gateway
2. Bot.on_message() fires
3. ChannelRouter resolves channel → ws_id
(or creates new workstream)
4. ChannelRouter resolves platform user → user_id
via channel_users table
5. Broker.push_inbound(SendMessage)
6. Bridge pops from Redis, drives server
**Session Resume (evicted workstreams)**
1. Stale route detected (no MQ owner)
2. Old session looked up via get_session_id_by_ws()
3. CreateWorkstreamMessage sent with
resume_session=<old_session_id>
4. Server resumes atomically during creation
5. Bridge emits SessionResumedEvent → thread
end note
note right of Broker
**Outbound Flow**
1. Server emits SSE events
2. Bridge relays to Redis events:{ws_id}
3. Broker.subscribe(ws_id) yields events
4. Bot formats and sends to Discord thread
end note
note bottom of CR
**Approval Flow**
1. ApprovalRequestEvent arrives via events:{ws_id}
2. Bot renders Discord buttons (Approve / Deny)
3. User clicks button → on_interaction()
4. Router builds ApproveMessage
5. Broker.push_response(correlation_id, msg)
6. Bridge pops from resp:{id}, calls POST /api/approve
end note
note bottom of CU
**Identity Linking**
1. User runs /link in Discord
2. Bot opens modal requesting API token
3. User submits ts_... API token
4. Bot validates token against storage
5. On success, inserts channel_users row
6. Subsequent messages carry user_id
7. AuthResult scopes applied by server
end note
@enduml
@@ -1,3 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:0c615984373b4893b6cc5755604f137541e9d391862122746a4fcbae63543563
size 201041
oid sha256:733aa17cbfdab60a601cac6adf439c657dd3535e3d6c33c69c2ef93ba8ec5989
size 251042
@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:f4b2a2010335f986511c8dabaf49ec046ac02e577f9bc9924897e045f860bb13
size 248808
@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:47b106bbcb1041fe4122007065c6cc85605348b42fc7140287194cf25e42e095
size 318036
+37 -3
View File
@@ -27,6 +27,7 @@ Console dashboard: http://localhost:8090
| `server` | 8080 | default | Web UI + chat workstreams + LLM |
| `bridge` | — | default | Redis-to-HTTP bridge (multi-node routing) |
| `console` | 8090 | default | Cluster dashboard |
| `channel` | — | production | Channel gateway (Discord, Slack, etc.) |
| `sim` | — | sim | Multi-node cluster simulator |
## Profiles
@@ -37,6 +38,12 @@ Console dashboard: http://localhost:8090
docker compose up
```
**Production** — adds PostgreSQL and the channel gateway. Requires `POSTGRES_PASSWORD` and (for Discord) `TURNSTONE_DISCORD_TOKEN`:
```bash
docker compose --profile production up
```
**Sim** — adds the simulator. Can run alongside the full stack or standalone with just Redis and the console:
```bash
@@ -84,8 +91,35 @@ All configuration is via environment variables in `.env` (copy from `.env.exampl
| Variable | Default | Description |
|----------|---------|-------------|
| `TURNSTONE_AUTH_ENABLED` | — | Set to `1` to require Bearer token auth |
| `TURNSTONE_AUTH_TOKEN` | — | Shared auth token for server/bridge/console |
| `TURNSTONE_AUTH_ENABLED` | — | Set to `1` to require authentication |
| `TURNSTONE_AUTH_TOKEN` | — | Config-file token for server/bridge/console (backward compat, works alongside JWT) |
| `TURNSTONE_JWT_SECRET` | — | Secret key for signing JWTs (required when using user identity / JWT auth) |
### Database
| Variable | Default | Description |
|----------|---------|-------------|
| `TURNSTONE_DB_BACKEND` | `sqlite` | Storage backend: `sqlite` or `postgresql` |
| `TURNSTONE_DB_URL` | — | Database URL (e.g. `postgresql://user:pass@db:5432/turnstone`). For SQLite, defaults to `/data/.turnstone.db` |
The database stores workstream history, user accounts, and API tokens. When using JWT auth, a database backend is required for user storage.
> **First-time setup:** After deploying with auth enabled, create an initial admin user by running `turnstone-admin create-user` inside the container:
>
> ```bash
> docker compose exec server turnstone-admin create-user --username admin --name "Admin"
> ```
>
> You will be prompted to set a password. Use it to log in via the UI or SDK, then create additional users through the admin API. Pass `--token --scopes read,write,approve` to also generate an initial API token.
### Channel Gateway
| Variable | Default | Description |
|----------|---------|-------------|
| `TURNSTONE_DISCORD_TOKEN` | — | Discord bot token (required to enable Discord adapter) |
| `TURNSTONE_DISCORD_GUILD` | `0` | Restrict to a single Discord guild (0 = all guilds) |
The channel service runs in the `production` profile. When `TURNSTONE_DISCORD_TOKEN` is set, the Discord adapter connects to the Discord Gateway and routes messages through Redis MQ to the bridge and server. See [Channel Integrations](channels.md) for full setup instructions including Discord application creation and user account linking.
### Simulator
@@ -128,7 +162,7 @@ docker compose build
docker compose build --no-cache
```
All five entry points are installed in a single image: `turnstone-server`, `turnstone-bridge`, `turnstone-console`, `turnstone-sim`, `turnstone-eval`.
All entry points are installed in a single image: `turnstone-server`, `turnstone-bridge`, `turnstone-console`, `turnstone-channel`, `turnstone-admin`, `turnstone-sim`, `turnstone-eval`.
## Cleanup
+64 -15
View File
@@ -15,8 +15,10 @@ The Python SDK is included in the `turnstone` package — no extra install requi
```python
from turnstone.sdk import TurnstoneServer
# Synchronous client
with TurnstoneServer("http://localhost:8080", token="tok_xxx") as client:
# Synchronous client — login with username/password
with TurnstoneServer("http://localhost:8080") as client:
client.login(username="alice", password="s3cret")
# Create a workstream
ws = client.create_workstream(name="Analysis")
@@ -33,6 +35,15 @@ with TurnstoneServer("http://localhost:8080", token="tok_xxx") as client:
client.close_workstream(ws.ws_id)
```
Alternatively, authenticate with an API token:
```python
with TurnstoneServer("http://localhost:8080") as client:
client.login(token="ts_abc123...")
ws = client.create_workstream(name="CI run")
result = client.send_and_wait("Run the test suite.", ws.ws_id)
```
### Async Client
```python
@@ -40,7 +51,8 @@ import asyncio
from turnstone.sdk import AsyncTurnstoneServer
async def main():
async with AsyncTurnstoneServer("http://localhost:8080", token="tok_xxx") as client:
async with AsyncTurnstoneServer("http://localhost:8080") as client:
await client.login(username="alice", password="s3cret")
ws = await client.create_workstream(name="demo")
async for event in client.stream_events(ws.ws_id):
if event.type == "content":
@@ -67,8 +79,10 @@ Both `TurnstoneServer` (sync) and `AsyncTurnstoneServer` (async) expose:
| | `stream_global_events()` | `Iterator[ServerEvent]` |
| **High-level** | `send_and_wait(message, ws_id, *, timeout, on_event)` | `TurnResult` |
| **Sessions** | `list_sessions()` | `ListSessionsResponse` |
| **Auth** | `login(token)` | `AuthLoginResponse` |
| **Auth** | `login(username=..., password=...)` | `AuthLoginResponse` |
| | `login(token="ts_xxx")` | `AuthLoginResponse` |
| | `logout()` | `StatusResponse` |
| | `auth_status()` | `AuthStatusResponse` |
| **Health** | `health()` | `HealthResponse` |
### Console Client API
@@ -83,7 +97,8 @@ Both `TurnstoneConsole` (sync) and `AsyncTurnstoneConsole` (async) expose:
| | `node_detail(node_id)` | `NodeDetailResponse` |
| | `create_workstream(*, node_id, name, model, initial_message)` | `ConsoleCreateWsResponse` |
| **Streaming** | `stream_cluster_events()` | `Iterator[ClusterEvent]` |
| **Auth** | `login(token)` / `logout()` | `AuthLoginResponse` / `StatusResponse` |
| **Auth** | `login(username=..., password=...)` / `login(token="ts_xxx")` | `AuthLoginResponse` |
| | `logout()` | `StatusResponse` |
| **Health** | `health()` | `ConsoleHealthResponse` |
### Event Types
@@ -165,10 +180,11 @@ Located at `sdk/typescript/`. Zero runtime dependencies for browsers; uses nativ
```typescript
import { TurnstoneServer } from "@turnstone/sdk";
const client = new TurnstoneServer({
baseUrl: "http://localhost:8080",
token: "tok_xxx",
});
const client = new TurnstoneServer({ baseUrl: "http://localhost:8080" });
// Login with username/password or API token
await client.login({ username: "alice", password: "s3cret" });
// or: await client.login({ token: "ts_abc123..." });
// Create workstream and send message
const ws = await client.createWorkstream({ name: "demo" });
@@ -188,16 +204,14 @@ for await (const event of client.streamEvents(ws.ws_id)) {
```typescript
import { TurnstoneConsole } from "@turnstone/sdk";
const console = new TurnstoneConsole({
baseUrl: "http://localhost:8081",
token: "tok_xxx",
});
const client = new TurnstoneConsole({ baseUrl: "http://localhost:8090" });
await client.login({ username: "alice", password: "s3cret" });
const overview = await console.overview();
const overview = await client.overview();
console.log(`Nodes: ${overview.nodes}, Workstreams: ${overview.workstreams}`);
// Stream cluster events
for await (const event of console.clusterEvents()) {
for await (const event of client.clusterEvents()) {
console.log(event.type, event);
}
```
@@ -256,3 +270,38 @@ sdk/typescript/ TypeScript SDK (npm package)
The Python SDK reuses Pydantic models from `turnstone/api/` directly — no schema duplication. The TypeScript SDK has hand-written interfaces matching those models.
Both SDKs follow the same design: typed methods for REST endpoints, async iterators for SSE streams, and a high-level `send_and_wait` method for simple request-response patterns.
---
## Authentication
When auth is enabled on the server, the SDK handles JWT-based authentication automatically.
### Login Flow
There are two ways to authenticate:
1. **Username + password** — calls `POST /v1/api/auth/login` with credentials. The server validates against the user database and returns a JWT.
2. **API token** — calls `POST /v1/api/auth/login` with a `ts_`-prefixed token string. The server looks up the token, resolves the associated user, and returns a JWT.
In both cases the server returns the JWT in the response body and as a `Set-Cookie` header. The SDK extracts the JWT and includes it as a `Bearer` token in the `Authorization` header on all subsequent requests.
```python
# Username + password
client.login(username="alice", password="s3cret")
# API token (created via admin API or turnstone-admin CLI)
client.login(token="ts_abc123...")
```
### Token Lifecycle
- JWTs have a configurable expiry (default: 24 hours).
- `client.auth_status()` returns the current user identity and scopes without refreshing the token.
- `client.logout()` clears the stored JWT from the client.
- If a request returns 401, the SDK raises `TurnstoneAPIError` — the caller is responsible for re-authenticating.
### Backward Compatibility
The config-file token (`TURNSTONE_AUTH_TOKEN`) still works as a simple Bearer token for environments that do not use the user/JWT system. When the server receives a non-JWT Bearer token, it falls back to the legacy token check.
+450
View File
@@ -0,0 +1,450 @@
# Security and Authentication
Turnstone uses a layered authentication system with three token types,
hierarchical scopes, and a split architecture where the console manages
credentials while individual server nodes validate JWTs locally.
---
## Token Types
### Config-file tokens
Static tokens defined in `config.toml` or the `TURNSTONE_AUTH_TOKEN`
environment variable. Validated in-memory using `hmac.compare_digest`
(timing-safe). Each token maps to a role that determines its scopes.
```toml
[[auth.tokens]]
value = "tok_legacy"
role = "full" # full → {read, write, approve}
```
Role mappings: `"read"``{read}`, `"full"``{read, write, approve}`.
Config tokens are sent directly as `Authorization: Bearer tok_legacy`
on every request. No JWT exchange is needed.
### API tokens
Database-backed tokens prefixed with `ts_`. Created via the admin CLI
(`turnstone-admin create-token`) or the console admin API. Stored as
SHA-256 hashes — the raw token is shown exactly once at creation and
never persisted in plaintext.
```
$ turnstone-admin create-token --user abc123 --scopes read,write --name "CI bot"
Token created: ts_a1b2c3d4e5f6...
(save this — it will not be shown again)
```
API tokens can be used directly as `Bearer ts_xxx` headers or exchanged
for a JWT via the login endpoint.
### JWTs
Short-lived session tokens (24 hours by default). Issued after
authenticating with username/password or by exchanging an API token.
HS256-signed with a shared secret. Validated locally on every service
node — no database call per request.
Claims:
| Claim | Description |
|-------|-------------|
| `sub` | User ID |
| `scopes` | Comma-separated scope list (`read,write,approve`) |
| `src` | Token source (`password`, `api_token`, `config`) |
| `iss` | Issuer — always `turnstone` |
| `aud` | Audience — `turnstone-server` or `turnstone-console` |
| `iat` | Issued-at timestamp |
| `exp` | Expiry timestamp |
The `aud` claim prevents cross-service token reuse — a JWT issued for the
console cannot be used to authenticate against a server node, and vice versa.
Tokens without an `aud` claim are accepted during the rollout window when
`audience` validation is not specified.
---
## Scope Model
Scopes are hierarchical — higher scopes imply all lower ones.
| Scope | Grants | Implies |
|-------|--------|---------|
| `read` | View workstreams, sessions, history | — |
| `write` | Send messages, create/close workstreams | `read` |
| `approve` | Approve tool calls, admin endpoints | `read`, `write` |
### Path-to-scope mapping
| Method | Path pattern | Required scope |
|--------|-------------|----------------|
| GET | Any protected path | `read` |
| POST | `/api/send`, `/api/plan`, `/api/command` | `write` |
| POST | `/api/workstreams/new`, `/api/workstreams/close` | `write` |
| POST | `/api/cluster/workstreams/new` | `write` |
| POST | `/api/approve` | `approve` |
| Any | `/api/admin/*` | `approve` |
Public paths bypass authentication entirely: `/`, `/health`, `/metrics`,
`/static/*`, `/shared/*`, `/docs`, `/openapi.json`, `/api/auth/login`,
`/api/auth/logout`, `/api/auth/status`, `/api/auth/setup`.
---
## Login Flows
### Username and password
```
POST /v1/api/auth/login
Content-Type: application/json
{"username": "admin", "password": "s3cret"}
```
Returns a JWT in the response body and sets an `HttpOnly` session cookie.
### API token exchange
```
POST /v1/api/auth/login
Content-Type: application/json
{"token": "ts_a1b2c3d4e5f6..."}
```
The API token is hashed, looked up in the database, and exchanged for a
JWT with the token's scopes. This is the recommended flow for SDKs and
automated clients that need cookie-based sessions.
### Config-file tokens (direct)
Config tokens are validated per-request via `hmac.compare_digest`. No
login exchange is needed — include the token as a `Bearer` header:
```
Authorization: Bearer tok_legacy
```
### First-time setup
When no users exist in the database:
1. `GET /v1/api/auth/status` returns `{"setup_required": true}`
2. The UI presents a setup wizard
3. `POST /v1/api/auth/setup` creates the first admin user and returns a
JWT in one atomic step (no auth required — this is a public endpoint)
4. The endpoint returns `409 Conflict` if setup has already been completed
(i.e. users already exist in the database)
5. Subsequent admin requests require `approve` scope
The `/api/auth/setup` endpoint is available on both the server and
console. It validates input before creating the user:
- **username**: 1-64 ASCII characters
- **display_name**: required (non-empty)
- **password**: minimum 8 characters
```
POST /v1/api/auth/setup
Content-Type: application/json
{"username": "admin", "display_name": "Admin", "password": "strongpass"}
```
Response:
```json
{
"status": "ok",
"user_id": "u_abc123",
"username": "admin",
"role": "full",
"scopes": "approve,read,write",
"jwt": "eyJhbGciOiJIUzI1NiIs..."
}
```
The response also sets an `HttpOnly` session cookie containing the JWT,
so the browser is immediately authenticated after setup completes.
---
## Token Detection Order
The auth middleware inspects the `Authorization: Bearer <token>` header
and classifies the token:
1. **Contains `.`** → JWT → validate HS256 signature and expiry
2. **Starts with `ts_`** → API token → SHA-256 hash, database lookup
3. **Otherwise** → config-file token → `hmac.compare_digest` against
each configured token
If a session cookie is present and no `Authorization` header is sent,
the cookie value is treated as a JWT (step 1).
---
## Password Storage
Passwords are hashed with **bcrypt** using a random salt per password.
Plaintext passwords are only accepted over HTTPS in production
deployments.
---
## Cookie Security
| Attribute | Value | Purpose |
|-----------|-------|---------|
| `HttpOnly` | `true` | Prevents JavaScript access |
| `SameSite` | `Lax` | CSRF protection |
| `Path` | `/` | Available to all routes |
| `Max-Age` | 24 hours | Matches JWT expiry |
| `Secure` | `true` (default) | Always set unless explicitly disabled for dev |
---
## JWT Configuration
| Setting | Config key | Env var | Default |
|---------|-----------|---------|---------|
| Signing secret | `[auth] jwt_secret` | `TURNSTONE_JWT_SECRET` | Auto-generated ephemeral (warning logged) |
| Expiry | `[auth] jwt_expiry_hours` | — | 24 hours |
| Algorithm | — | — | HS256 (not configurable) |
| Minimum secret length | — | — | 32 characters (warning if shorter) |
All service nodes that need to validate JWTs must share the same signing
secret. If no secret is configured, an ephemeral key is generated at
startup and a warning is logged — JWTs will not survive restarts or work
across nodes.
The bridge and console **require** `TURNSTONE_JWT_SECRET` when no
`--auth-token` is provided. They exit with an error if the secret is
missing, since ephemeral secrets would silently break inter-service
communication.
---
## Admin API Endpoints
All admin endpoints require `approve` scope.
### Users
| Method | Path | Description |
|--------|------|-------------|
| POST | `/v1/api/admin/users` | Create user (username, display_name, password) |
| GET | `/v1/api/admin/users` | List all users |
| DELETE | `/v1/api/admin/users/{user_id}` | Delete user and cascade tokens |
### API tokens
| Method | Path | Description |
|--------|------|-------------|
| POST | `/v1/api/admin/users/{user_id}/tokens` | Create API token (returns raw value once) |
| GET | `/v1/api/admin/users/{user_id}/tokens` | List tokens (prefix only, no hashes) |
| DELETE | `/v1/api/admin/tokens/{token_id}` | Revoke token |
---
## CLI Administration
The `turnstone-admin` command provides offline user and token management:
```
turnstone-admin create-user --username admin --name "Admin" [--password] [--token]
turnstone-admin create-token --user <user_id> --scopes read,write --name "CI bot"
turnstone-admin list-users
turnstone-admin list-tokens
turnstone-admin revoke-token <token_id>
```
When `--password` is omitted, the CLI prompts interactively. When
`--token` is passed to `create-user`, an API token is created alongside
the user and printed to stdout.
---
## Database Schema
```sql
CREATE TABLE users (
user_id TEXT PRIMARY KEY,
username TEXT NOT NULL UNIQUE,
display_name TEXT NOT NULL,
password_hash TEXT NOT NULL,
created TEXT NOT NULL
);
CREATE TABLE api_tokens (
token_id TEXT PRIMARY KEY,
token_hash TEXT NOT NULL, -- SHA-256 of raw token
token_prefix TEXT NOT NULL, -- first 8 chars for display
user_id TEXT NOT NULL REFERENCES users(user_id),
name TEXT NOT NULL,
scopes TEXT NOT NULL, -- comma-separated
created TEXT NOT NULL,
expires TEXT -- nullable, ISO 8601
);
CREATE UNIQUE INDEX ix_api_tokens_hash ON api_tokens(token_hash);
CREATE TABLE channel_users (
channel_type TEXT NOT NULL,
channel_user_id TEXT NOT NULL,
user_id TEXT NOT NULL REFERENCES users(user_id),
created TEXT NOT NULL,
PRIMARY KEY (channel_type, channel_user_id)
);
```
The `sessions` and `workstreams` tables have a nullable `user_id`
column for attribution when auth is enabled.
---
## Revocation
- **API tokens**: Deleting a token via the admin API or CLI prevents new
JWTs from being issued with that token. Existing JWTs derived from the
token remain valid until they expire (at most 24 hours).
- **Config-file tokens**: Remove the token from `config.toml` and
restart the service. No JWTs are involved, so revocation is immediate.
- **JWTs**: Cannot be individually revoked. Rely on short expiry (24h)
and revoke the underlying credential to prevent renewal.
---
## Architecture
```
Console (cluster-wide) Server (per-node)
┌──────────────────────┐ ┌──────────────────────┐
│ User/Token CRUD (DB) │ │ JWT validation only │
│ Login: creds → JWT │ │ (shared signing key) │
│ Admin API endpoints │ │ Config tokens: hmac │
│ Storage: users, │ │ No auth DB needed │
│ api_tokens tables │ │ │
└──────────────────────┘ └──────────────────────┘
```
The console owns the credential database and handles all user/token
CRUD. Individual server nodes only need the JWT signing secret to
validate session tokens. Config-file tokens are validated locally
without any database.
### Proxy auth forwarding
When the console proxies requests to server nodes (via `/node/{id}/...`
routes), it uses a dedicated **service proxy token** with
`aud: turnstone-server` and `write` scope. The user's console JWT
(which has `aud: turnstone-console`) is **not** forwarded — it would be
rejected by the server's audience validation.
The proxy token is managed by a `ServiceTokenManager` that auto-rotates
1-hour JWTs, refreshing at 80% of lifetime. If `--auth-token` is
provided, that static token is used instead.
### Service-to-service authentication
The bridge and console collector use `ServiceTokenManager` for
auto-rotating JWTs when communicating with server nodes:
| Service | Identity | Scope | Purpose |
|---------|----------|-------|---------|
| Bridge | `bridge` | `approve` | Tool approval proxy, message relay |
| Console collector | `console-collector` | `read` | Node health polling |
| Console proxy | `console-proxy` | `write` | Proxied API calls |
All service tokens use `aud: turnstone-server` and 1-hour expiry with
automatic refresh. The bridge injects auth headers per-request via httpx
event hooks to ensure rotated tokens are picked up on SSE reconnects.
---
## Configuration Reference
### config.toml
```toml
[auth]
enabled = true
jwt_secret = "your-secret-key-here"
jwt_expiry_hours = 24
[[auth.tokens]]
value = "tok_legacy"
role = "full"
```
### Environment variables
| Variable | Description |
|----------|-------------|
| `TURNSTONE_AUTH_ENABLED=1` | Enable authentication |
| `TURNSTONE_AUTH_TOKEN=tok_xxx` | Register a config-file token with `full` access |
| `TURNSTONE_JWT_SECRET=xxx` | JWT signing secret (must match across nodes) |
| `TURNSTONE_CORS_ORIGINS=` | CORS allowed origins (comma-separated; empty = same-origin only) |
---
## Login Rate Limiting
The `/api/auth/login` endpoint is protected by a dedicated
`LoginRateLimiter` (separate from the general API rate limiter).
Limits are enforced per-IP and per-username with a sliding window:
- **5 attempts** per **5-minute window** per key
- Failed logins record against both `ip:{client_ip}` and `user:{username}`
- Returns `429 Too Many Requests` with `Retry-After` header when exceeded
- Successful logins do not consume the budget
---
## CORS Policy
By default, no CORS headers are sent (same-origin only). To allow
cross-origin requests, set `TURNSTONE_CORS_ORIGINS`:
```bash
# Allow specific origins
TURNSTONE_CORS_ORIGINS=https://app.example.com,https://admin.example.com
# Allow all origins (development only)
TURNSTONE_CORS_ORIGINS=*
```
When the variable is empty or unset, the CORS middleware is not added
and browsers enforce same-origin policy.
---
## Security Properties
- **Timing-safe comparison** for config-file tokens via
`hmac.compare_digest` — no timing side-channel.
- **Hash-based lookup** for API tokens — the database stores only
SHA-256 hashes, eliminating timing attacks on token comparison.
- **Local JWT validation** — no network call or database query needed
per request on server nodes.
- **One-time display** of raw API tokens at creation. The plaintext is
never stored; `token_hash` never appears in API responses or logs.
- **Structured logging audit trail**`ctx_user_id` is set on every
authenticated request and injected into all log events.
- **Scope enforcement** at the middleware layer before any handler
executes. Path-to-scope mapping is defined statically.
- **JWT audience isolation** — server and console JWTs have distinct
`aud` claims, preventing cross-service token reuse.
- **Login brute-force protection** — per-IP and per-username rate
limiting on the login endpoint.
- **Secure cookies by default**`Secure` flag set unconditionally;
24-hour max-age matches JWT expiry.
- **CORS restriction** — no CORS headers by default (same-origin only).
- **Service JWT auto-rotation** — 1-hour expiry with transparent
refresh, eliminating long-lived static tokens for inter-service auth.
- **Secret strength validation** — warning logged when JWT secret is
shorter than 32 characters.
+25 -1
View File
@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
[project]
name = "turnstone"
version = "0.3.5"
version = "0.4.0"
description = "Multi-node AI orchestration platform with tool use, agent routing, and cluster simulation."
readme = "README.md"
license = "BUSL-1.1"
@@ -32,6 +32,9 @@ dependencies = [
"pydantic>=2.0",
"sqlalchemy>=2.0",
"alembic>=1.14",
"structlog>=24.1",
"PyJWT>=2.8",
"bcrypt>=4.0",
]
[project.urls]
@@ -47,6 +50,7 @@ console = ["redis>=7.2"]
sim = ["redis>=7.2"]
anthropic = ["anthropic>=0.39"]
postgres = ["psycopg[binary]>=3.2"]
discord = ["discord.py>=2.4", "redis>=7.2"]
[project.scripts]
@@ -56,6 +60,8 @@ turnstone-server = "turnstone.server:main"
turnstone-bridge = "turnstone.mq.bridge:main"
turnstone-console = "turnstone.console.server:main"
turnstone-sim = "turnstone.sim.cli:main"
turnstone-admin = "turnstone.admin:main"
turnstone-channel = "turnstone.channels.cli:main"
[tool.hatch.build.targets.wheel]
include = [
@@ -128,10 +134,28 @@ ignore_missing_imports = true
module = ["sqlalchemy", "sqlalchemy.*", "alembic", "alembic.*"]
ignore_missing_imports = true
[[tool.mypy.overrides]]
module = ["structlog", "structlog.*"]
ignore_missing_imports = true
[[tool.mypy.overrides]]
module = ["jwt", "jwt.*"]
ignore_missing_imports = true
[[tool.mypy.overrides]]
module = ["anthropic", "anthropic.*"]
ignore_missing_imports = true
[[tool.mypy.overrides]]
module = ["discord", "discord.*"]
ignore_missing_imports = true
[[tool.mypy.overrides]]
module = ["turnstone.channels.discord.*"]
disallow_subclassing_any = false
disallow_untyped_decorators = false
warn_unused_ignores = false
[[tool.mypy.overrides]]
module = "tests.*"
disallow_untyped_defs = false
+29 -3
View File
@@ -2,6 +2,8 @@ import { BaseClient, type ClientOptions } from "./base.js";
import type { ClusterEvent } from "./events.js";
import type {
AuthLoginResponse,
AuthSetupResponse,
AuthStatusResponse,
ClusterNodesResponse,
ClusterOverviewResponse,
ClusterWorkstreamsResponse,
@@ -70,9 +72,33 @@ export class TurnstoneConsole extends BaseClient {
// -- Auth -----------------------------------------------------------------
async login(token: string): Promise<AuthLoginResponse> {
return this.request("POST", "/v1/api/auth/login", {
json: { token },
async login(opts: {
token?: string;
username?: string;
password?: string;
}): Promise<AuthLoginResponse> {
const body =
opts.username && opts.password
? { username: opts.username, password: opts.password }
: { token: opts.token ?? "" };
return this.request("POST", "/v1/api/auth/login", { json: body });
}
async authStatus(): Promise<AuthStatusResponse> {
return this.request("GET", "/v1/api/auth/status");
}
async setup(opts: {
username: string;
displayName: string;
password: string;
}): Promise<AuthSetupResponse> {
return this.request("POST", "/v1/api/auth/setup", {
json: {
username: opts.username,
display_name: opts.displayName,
password: opts.password,
},
});
}
+2
View File
@@ -90,6 +90,8 @@ export type {
HealthResponse,
AuthLoginRequest,
AuthLoginResponse,
AuthStatusResponse,
AuthSetupResponse,
StatusResponse,
ErrorResponse,
ClusterOverviewResponse,
+29 -3
View File
@@ -2,6 +2,8 @@ import { BaseClient, type ClientOptions } from "./base.js";
import type { ServerEvent } from "./events.js";
import type {
AuthLoginResponse,
AuthSetupResponse,
AuthStatusResponse,
CreateWorkstreamRequest,
CreateWorkstreamResponse,
DashboardResponse,
@@ -184,9 +186,33 @@ export class TurnstoneServer extends BaseClient {
// -- Auth -----------------------------------------------------------------
async login(token: string): Promise<AuthLoginResponse> {
return this.request("POST", "/v1/api/auth/login", {
json: { token },
async login(opts: {
token?: string;
username?: string;
password?: string;
}): Promise<AuthLoginResponse> {
const body =
opts.username && opts.password
? { username: opts.username, password: opts.password }
: { token: opts.token ?? "" };
return this.request("POST", "/v1/api/auth/login", { json: body });
}
async authStatus(): Promise<AuthStatusResponse> {
return this.request("GET", "/v1/api/auth/status");
}
async setup(opts: {
username: string;
displayName: string;
password: string;
}): Promise<AuthSetupResponse> {
return this.request("POST", "/v1/api/auth/setup", {
json: {
username: opts.username,
display_name: opts.displayName,
password: opts.password,
},
});
}
+22
View File
@@ -17,6 +17,24 @@ export interface AuthLoginRequest {
export interface AuthLoginResponse {
status: string;
role: string;
scopes?: string;
jwt?: string;
user_id?: string;
}
export interface AuthStatusResponse {
auth_enabled: boolean;
has_users: boolean;
setup_required: boolean;
}
export interface AuthSetupResponse {
status: string;
user_id: string;
username: string;
role: string;
scopes: string;
jwt?: string;
}
// ---------------------------------------------------------------------------
@@ -53,11 +71,15 @@ export interface CreateWorkstreamRequest {
name?: string;
model?: string;
auto_approve?: boolean;
resume_session?: string;
}
export interface CreateWorkstreamResponse {
ws_id: string;
name: string;
resumed?: boolean;
session_id?: string;
message_count?: number;
}
export interface CloseWorkstreamRequest {
+181
View File
@@ -0,0 +1,181 @@
"""Tests for turnstone.mq.async_broker.AsyncRedisBroker."""
from __future__ import annotations
import asyncio
import contextlib
from unittest.mock import AsyncMock, MagicMock, patch
import pytest
from turnstone.mq.async_broker import AsyncRedisBroker
@pytest.fixture
def broker() -> AsyncRedisBroker:
return AsyncRedisBroker(host="localhost", port=6379, db=0, prefix="test", response_ttl=120)
@pytest.fixture
def mock_redis() -> AsyncMock:
"""Return a mock Redis client with common async methods."""
r = AsyncMock()
r.rpush = AsyncMock()
r.publish = AsyncMock()
r.expire = AsyncMock()
r.get = AsyncMock(return_value=None)
r.set = AsyncMock()
r.delete = AsyncMock()
r.blpop = AsyncMock(return_value=None)
ps = AsyncMock()
ps.subscribe = AsyncMock()
ps.unsubscribe = AsyncMock()
ps.close = AsyncMock()
ps.get_message = AsyncMock(return_value=None)
r.pubsub = MagicMock(return_value=ps)
return r
def _inject_redis(broker: AsyncRedisBroker, mock_redis: AsyncMock) -> None:
"""Inject a mock Redis client into the broker, simulating connect()."""
broker._redis = mock_redis
broker._pubsub = mock_redis.pubsub()
class TestConstructor:
def test_stores_config(self) -> None:
b = AsyncRedisBroker(host="h", port=1234, db=2, prefix="pfx", password="pw")
assert b._host == "h"
assert b._port == 1234
assert b._db == 2
assert b._prefix == "pfx"
assert b._password == "pw"
assert b._redis is None
def test_defaults(self) -> None:
b = AsyncRedisBroker()
assert b._host == "localhost"
assert b._port == 6379
assert b._prefix == "turnstone"
class TestConnect:
@pytest.mark.anyio
async def test_creates_connection(self) -> None:
b = AsyncRedisBroker()
mock_r = AsyncMock()
mock_r.pubsub = MagicMock(return_value=AsyncMock())
with patch("redis.asyncio.Redis", return_value=mock_r):
await b.connect()
assert b._redis is mock_r
assert b._pubsub is not None
@pytest.mark.anyio
async def test_connect_idempotent(
self, broker: AsyncRedisBroker, mock_redis: AsyncMock
) -> None:
_inject_redis(broker, mock_redis)
old = broker._redis
await broker.connect()
assert broker._redis is old
class TestPushInbound:
@pytest.mark.anyio
async def test_shared_queue(self, broker: AsyncRedisBroker, mock_redis: AsyncMock) -> None:
_inject_redis(broker, mock_redis)
await broker.push_inbound('{"type":"send"}')
mock_redis.rpush.assert_awaited_once_with("test:inbound", '{"type":"send"}')
@pytest.mark.anyio
async def test_per_node_queue(self, broker: AsyncRedisBroker, mock_redis: AsyncMock) -> None:
_inject_redis(broker, mock_redis)
await broker.push_inbound('{"type":"send"}', node_id="node-1")
mock_redis.rpush.assert_awaited_once_with("test:inbound:node-1", '{"type":"send"}')
class TestPublishOutbound:
@pytest.mark.anyio
async def test_publishes(self, broker: AsyncRedisBroker, mock_redis: AsyncMock) -> None:
_inject_redis(broker, mock_redis)
await broker.publish_outbound("test:events:global", '{"event":"data"}')
mock_redis.publish.assert_awaited_once_with("test:events:global", '{"event":"data"}')
class TestPushResponse:
@pytest.mark.anyio
async def test_rpush_and_expire(self, broker: AsyncRedisBroker, mock_redis: AsyncMock) -> None:
_inject_redis(broker, mock_redis)
await broker.push_response("req-123", '{"ok":true}')
mock_redis.rpush.assert_awaited_once_with("test:resp:req-123", '{"ok":true}')
mock_redis.expire.assert_awaited_once_with("test:resp:req-123", 120)
class TestSubscribe:
@pytest.mark.anyio
async def test_creates_task(self, broker: AsyncRedisBroker, mock_redis: AsyncMock) -> None:
_inject_redis(broker, mock_redis)
await broker.subscribe("test:events:global", lambda msg: None)
assert "test:events:global" in broker._callbacks
assert broker._listener_task is not None
assert isinstance(broker._listener_task, asyncio.Task)
# Clean up.
broker._listener_task.cancel()
with contextlib.suppress(asyncio.CancelledError):
await broker._listener_task
class TestUnsubscribe:
@pytest.mark.anyio
async def test_cancels_task(self, broker: AsyncRedisBroker, mock_redis: AsyncMock) -> None:
_inject_redis(broker, mock_redis)
await broker.subscribe("test:events:ch", lambda msg: None)
assert "test:events:ch" in broker._callbacks
await broker.unsubscribe("test:events:ch")
assert "test:events:ch" not in broker._callbacks
class TestRoutingPrimitives:
@pytest.mark.anyio
async def test_get_ws_owner(self, broker: AsyncRedisBroker, mock_redis: AsyncMock) -> None:
_inject_redis(broker, mock_redis)
mock_redis.get.return_value = "node-1"
result = await broker.get_ws_owner("ws-abc")
mock_redis.get.assert_awaited_once_with("test:ws:ws-abc")
assert result == "node-1"
@pytest.mark.anyio
async def test_set_ws_owner(self, broker: AsyncRedisBroker, mock_redis: AsyncMock) -> None:
_inject_redis(broker, mock_redis)
await broker.set_ws_owner("ws-abc", "node-2")
mock_redis.set.assert_awaited_once_with("test:ws:ws-abc", "node-2")
@pytest.mark.anyio
async def test_set_ws_owner_with_ttl(
self, broker: AsyncRedisBroker, mock_redis: AsyncMock
) -> None:
_inject_redis(broker, mock_redis)
await broker.set_ws_owner("ws-abc", "node-2", ttl=300)
mock_redis.set.assert_awaited_once_with("test:ws:ws-abc", "node-2", ex=300)
@pytest.mark.anyio
async def test_del_ws_owner(self, broker: AsyncRedisBroker, mock_redis: AsyncMock) -> None:
_inject_redis(broker, mock_redis)
await broker.del_ws_owner("ws-abc")
mock_redis.delete.assert_awaited_once_with("test:ws:ws-abc")
class TestClose:
@pytest.mark.anyio
async def test_cancels_tasks_and_closes(
self, broker: AsyncRedisBroker, mock_redis: AsyncMock
) -> None:
_inject_redis(broker, mock_redis)
await broker.subscribe("ch1", lambda m: None)
assert len(broker._callbacks) == 1
assert broker._listener_task is not None
await broker.close()
assert len(broker._callbacks) == 0
assert broker._listener_task is None
assert broker._redis is None
assert broker._pubsub is None
+386 -77
View File
@@ -1,7 +1,9 @@
"""Tests for turnstone.core.auth — bearer token authentication and cookies."""
import os
from unittest.mock import patch
import queue
import threading
from unittest.mock import MagicMock, patch
import pytest
@@ -15,7 +17,7 @@ from turnstone.core.auth import (
load_auth_config,
make_clear_cookie,
make_set_cookie,
required_role,
required_scope,
)
# ---------------------------------------------------------------------------
@@ -87,69 +89,59 @@ class TestIsPublicPath:
# ---------------------------------------------------------------------------
class TestRequiredRole:
class TestRequiredScope:
def test_get_api_needs_read(self):
assert required_role("GET", "/api/workstreams") == "read"
assert required_scope("GET", "/api/workstreams") == "read"
def test_get_events_needs_read(self):
assert required_role("GET", "/api/events") == "read"
assert required_scope("GET", "/api/events") == "read"
def test_get_dashboard_needs_read(self):
assert required_role("GET", "/api/dashboard") == "read"
def test_post_send_needs_write(self):
assert required_scope("POST", "/api/send") == "write"
def test_post_send_needs_full(self):
assert required_role("POST", "/api/send") == "full"
def test_post_approve_needs_approve(self):
assert required_scope("POST", "/api/approve") == "approve"
def test_post_approve_needs_full(self):
assert required_role("POST", "/api/approve") == "full"
def test_post_plan_needs_write(self):
assert required_scope("POST", "/api/plan") == "write"
def test_post_plan_needs_full(self):
assert required_role("POST", "/api/plan") == "full"
def test_post_command_needs_write(self):
assert required_scope("POST", "/api/command") == "write"
def test_post_command_needs_full(self):
assert required_role("POST", "/api/command") == "full"
def test_post_workstreams_new_needs_write(self):
assert required_scope("POST", "/api/workstreams/new") == "write"
def test_post_workstreams_new_needs_full(self):
assert required_role("POST", "/api/workstreams/new") == "full"
def test_post_workstreams_close_needs_write(self):
assert required_scope("POST", "/api/workstreams/close") == "write"
def test_post_workstreams_close_needs_full(self):
assert required_role("POST", "/api/workstreams/close") == "full"
def test_all_write_paths_need_full(self):
def test_all_write_paths_need_write(self):
for path in WRITE_PATHS:
assert required_role("POST", path) == "full"
scope = required_scope("POST", path)
assert scope in ("write", "approve"), f"{path} should need write or approve"
def test_post_unknown_path_needs_read(self):
assert required_role("POST", "/api/unknown") == "read"
assert required_scope("POST", "/api/unknown") == "read"
def test_v1_post_send_needs_full(self):
assert required_role("POST", "/v1/api/send") == "full"
def test_v1_post_send_needs_write(self):
assert required_scope("POST", "/v1/api/send") == "write"
def test_v1_post_approve_needs_full(self):
assert required_role("POST", "/v1/api/approve") == "full"
def test_v1_post_approve_needs_approve(self):
assert required_scope("POST", "/v1/api/approve") == "approve"
def test_v1_get_workstreams_needs_read(self):
assert required_role("GET", "/v1/api/workstreams") == "read"
assert required_scope("GET", "/v1/api/workstreams") == "read"
def test_v1_post_cluster_ws_new_needs_full(self):
assert required_role("POST", "/v1/api/cluster/workstreams/new") == "full"
def test_v1_post_cluster_ws_new_needs_write(self):
assert required_scope("POST", "/v1/api/cluster/workstreams/new") == "write"
def test_v1_all_write_paths_need_full(self):
for path in WRITE_PATHS:
v1_path = "/v1" + path
assert required_role("POST", v1_path) == "full", f"{v1_path} should need full"
def test_proxy_v1_send_needs_write(self):
assert required_scope("POST", "/node/node-a/v1/api/send") == "write"
def test_proxy_v1_send_needs_full(self):
assert required_role("POST", "/node/node-a/v1/api/send") == "full"
def test_proxy_v1_approve_needs_full(self):
assert required_role("POST", "/node/node-a/v1/api/approve") == "full"
def test_proxy_v1_cluster_ws_new_needs_full(self):
assert required_role("POST", "/node/node-a/v1/api/cluster/workstreams/new") == "full"
def test_proxy_v1_approve_needs_approve(self):
assert required_scope("POST", "/node/node-a/v1/api/approve") == "approve"
def test_proxy_v1_read_endpoint_needs_read(self):
assert required_role("GET", "/node/node-a/v1/api/workstreams") == "read"
assert required_scope("GET", "/node/node-a/v1/api/workstreams") == "read"
# ---------------------------------------------------------------------------
@@ -268,12 +260,24 @@ class TestMakeSetCookie:
def test_max_age_default(self):
val = make_set_cookie("tok_abc")
assert "Max-Age=2592000" in val # 30 days
assert "Max-Age=86400" in val # 24 hours (matches JWT expiry)
def test_max_age_custom(self):
val = make_set_cookie("tok_abc", max_age=3600)
assert "Max-Age=3600" in val
def test_secure_default(self):
val = make_set_cookie("tok_abc")
assert "; Secure" in val
def test_secure_false(self):
val = make_set_cookie("tok_abc", secure=False)
assert "; Secure" not in val
def test_secure_true(self):
val = make_set_cookie("tok_abc", secure=True)
assert "; Secure" in val
class TestMakeClearCookie:
def test_max_age_zero(self):
@@ -306,68 +310,78 @@ class TestCheckRequest:
)
def test_disabled_allows_all(self, disabled):
allowed, status, msg = check_request(disabled, "POST", "/api/send", None)
allowed, status, msg, _result = check_request(disabled, "POST", "/api/send", None)
assert allowed is True
assert status == 200
def test_disabled_allows_no_header(self, disabled):
allowed, status, msg = check_request(disabled, "GET", "/api/workstreams", None)
allowed, status, msg, _result = check_request(disabled, "GET", "/api/workstreams", None)
assert allowed is True
def test_public_path_no_token_ok(self, enabled):
allowed, status, msg = check_request(enabled, "GET", "/health", None)
allowed, status, msg, _result = check_request(enabled, "GET", "/health", None)
assert allowed is True
assert status == 200
def test_public_root_no_token_ok(self, enabled):
allowed, status, msg = check_request(enabled, "GET", "/", None)
allowed, status, msg, _result = check_request(enabled, "GET", "/", None)
assert allowed is True
def test_public_static_no_token_ok(self, enabled):
allowed, status, msg = check_request(enabled, "GET", "/static/style.css", None)
allowed, status, msg, _result = check_request(enabled, "GET", "/static/style.css", None)
assert allowed is True
def test_api_no_token_401(self, enabled):
allowed, status, msg = check_request(enabled, "GET", "/api/workstreams", None)
allowed, status, msg, _result = check_request(enabled, "GET", "/api/workstreams", None)
assert allowed is False
assert status == 401
assert "Unauthorized" in msg
def test_api_invalid_token_401(self, enabled):
allowed, status, msg = check_request(
allowed, status, msg, _result = check_request(
enabled, "GET", "/api/workstreams", "Bearer wrong_token"
)
assert allowed is False
assert status == 401
def test_api_read_token_ok(self, enabled):
allowed, status, msg = check_request(enabled, "GET", "/api/workstreams", "Bearer tok_read")
allowed, status, msg, _result = check_request(
enabled, "GET", "/api/workstreams", "Bearer tok_read"
)
assert allowed is True
assert status == 200
def test_api_full_token_ok(self, enabled):
allowed, status, msg = check_request(enabled, "GET", "/api/workstreams", "Bearer tok_full")
allowed, status, msg, _result = check_request(
enabled, "GET", "/api/workstreams", "Bearer tok_full"
)
assert allowed is True
def test_write_read_token_403(self, enabled):
allowed, status, msg = check_request(enabled, "POST", "/api/send", "Bearer tok_read")
allowed, status, msg, _result = check_request(
enabled, "POST", "/api/send", "Bearer tok_read"
)
assert allowed is False
assert status == 403
assert "Forbidden" in msg
def test_write_full_token_ok(self, enabled):
allowed, status, msg = check_request(enabled, "POST", "/api/send", "Bearer tok_full")
allowed, status, msg, _result = check_request(
enabled, "POST", "/api/send", "Bearer tok_full"
)
assert allowed is True
assert status == 200
def test_approve_read_token_403(self, enabled):
allowed, status, msg = check_request(enabled, "POST", "/api/approve", "Bearer tok_read")
allowed, status, msg, _result = check_request(
enabled, "POST", "/api/approve", "Bearer tok_read"
)
assert allowed is False
assert status == 403
def test_proxy_write_read_token_403(self, enabled):
"""Read tokens cannot escalate to write ops via proxy routes."""
allowed, status, msg = check_request(
allowed, status, msg, _result = check_request(
enabled, "POST", "/node/node-a/api/send", "Bearer tok_read"
)
assert allowed is False
@@ -375,7 +389,7 @@ class TestCheckRequest:
def test_proxy_write_trailing_slash_read_token_403(self, enabled):
"""Trailing slash must not bypass write-role check on proxy routes."""
allowed, status, msg = check_request(
allowed, status, msg, _result = check_request(
enabled, "POST", "/node/node-a/api/send/", "Bearer tok_read"
)
assert allowed is False
@@ -383,20 +397,22 @@ class TestCheckRequest:
def test_direct_write_trailing_slash_read_token_403(self, enabled):
"""Trailing slash must not bypass write-role check on direct routes."""
allowed, status, msg = check_request(enabled, "POST", "/api/send/", "Bearer tok_read")
allowed, status, msg, _result = check_request(
enabled, "POST", "/api/send/", "Bearer tok_read"
)
assert allowed is False
assert status == 403
def test_proxy_write_full_token_ok(self, enabled):
"""Full tokens pass through proxy write routes."""
allowed, status, msg = check_request(
allowed, status, msg, _result = check_request(
enabled, "POST", "/node/node-a/api/send", "Bearer tok_full"
)
assert allowed is True
def test_proxy_v1_write_read_token_403(self, enabled):
"""Read tokens cannot escalate to write ops via v1 proxy routes."""
allowed, status, msg = check_request(
allowed, status, msg, _result = check_request(
enabled, "POST", "/node/node-a/v1/api/send", "Bearer tok_read"
)
assert allowed is False
@@ -404,14 +420,14 @@ class TestCheckRequest:
def test_proxy_v1_write_full_token_ok(self, enabled):
"""Full tokens pass through v1 proxy write routes."""
allowed, status, msg = check_request(
allowed, status, msg, _result = check_request(
enabled, "POST", "/node/node-a/v1/api/send", "Bearer tok_full"
)
assert allowed is True
def test_proxy_v1_cluster_ws_new_read_403(self, enabled):
"""Read tokens cannot create workstreams via v1 proxy."""
allowed, status, msg = check_request(
allowed, status, msg, _result = check_request(
enabled,
"POST",
"/node/node-a/v1/api/cluster/workstreams/new",
@@ -422,25 +438,27 @@ class TestCheckRequest:
def test_proxy_read_endpoint_read_token_ok(self, enabled):
"""Read tokens can access proxy read endpoints."""
allowed, status, msg = check_request(
allowed, status, msg, _result = check_request(
enabled, "GET", "/node/node-a/api/workstreams", "Bearer tok_read"
)
assert allowed is True
def test_console_create_ws_read_token_403(self, enabled):
"""Read tokens cannot create workstreams."""
allowed, status, msg = check_request(
allowed, status, msg, _result = check_request(
enabled, "POST", "/api/cluster/workstreams/new", "Bearer tok_read"
)
assert allowed is False
assert status == 403
def test_approve_full_token_ok(self, enabled):
allowed, status, msg = check_request(enabled, "POST", "/api/approve", "Bearer tok_full")
allowed, status, msg, _result = check_request(
enabled, "POST", "/api/approve", "Bearer tok_full"
)
assert allowed is True
def test_no_auth_header_string(self, enabled):
allowed, status, msg = check_request(enabled, "GET", "/api/dashboard", "")
allowed, status, msg, _result = check_request(enabled, "GET", "/api/dashboard", "")
assert allowed is False
assert status == 401
@@ -461,7 +479,7 @@ class TestCheckRequestWithCookie:
)
def test_cookie_fallback_when_no_bearer(self, enabled):
allowed, status, _ = check_request(
allowed, status, _, _r = check_request(
enabled,
"GET",
"/api/workstreams",
@@ -473,7 +491,7 @@ class TestCheckRequestWithCookie:
def test_bearer_takes_precedence_over_cookie(self, enabled):
# Bearer is full, cookie is read — Bearer should win
allowed, status, _ = check_request(
allowed, status, _, _r = check_request(
enabled,
"POST",
"/api/send",
@@ -483,7 +501,7 @@ class TestCheckRequestWithCookie:
assert allowed is True
def test_invalid_cookie_401(self, enabled):
allowed, status, _ = check_request(
allowed, status, _, _r = check_request(
enabled,
"GET",
"/api/workstreams",
@@ -494,7 +512,7 @@ class TestCheckRequestWithCookie:
assert status == 401
def test_cookie_read_on_write_403(self, enabled):
allowed, status, _ = check_request(
allowed, status, _, _r = check_request(
enabled,
"POST",
"/api/send",
@@ -505,7 +523,7 @@ class TestCheckRequestWithCookie:
assert status == 403
def test_cookie_full_on_write_ok(self, enabled):
allowed, status, _ = check_request(
allowed, status, _, _r = check_request(
enabled,
"POST",
"/api/send",
@@ -515,7 +533,7 @@ class TestCheckRequestWithCookie:
assert allowed is True
def test_no_cookie_no_bearer_401(self, enabled):
allowed, status, _ = check_request(
allowed, status, _, _r = check_request(
enabled,
"GET",
"/api/workstreams",
@@ -526,7 +544,7 @@ class TestCheckRequestWithCookie:
assert status == 401
def test_login_path_public(self, enabled):
allowed, status, _ = check_request(
allowed, status, _, _r = check_request(
enabled,
"POST",
"/api/auth/login",
@@ -535,7 +553,7 @@ class TestCheckRequestWithCookie:
assert allowed is True
def test_logout_path_public(self, enabled):
allowed, status, _ = check_request(
allowed, status, _, _r = check_request(
enabled,
"POST",
"/api/auth/logout",
@@ -552,12 +570,28 @@ class TestCheckRequestWithCookie:
class TestLoadAuthConfig:
"""Tests for load_auth_config with mocked config + env vars."""
def test_default_disabled(self):
def test_default_enabled(self):
with patch("turnstone.core.config.load_config", return_value={}):
cfg = load_auth_config()
assert cfg.enabled is False
assert cfg.enabled is True
assert cfg.tokens == {}
def test_explicit_disable(self):
with (
patch("turnstone.core.config.load_config", return_value={"enabled": False}),
patch.dict(os.environ, {}, clear=True),
):
cfg = load_auth_config()
assert cfg.enabled is False
def test_env_disable(self):
with (
patch("turnstone.core.config.load_config", return_value={}),
patch.dict(os.environ, {"TURNSTONE_AUTH_ENABLED": "0"}, clear=True),
):
cfg = load_auth_config()
assert cfg.enabled is False
def test_config_file_tokens(self):
mock_cfg = {
"enabled": True,
@@ -705,6 +739,7 @@ class TestServerAuth:
enabled=True,
tokens={"tok_full": "full", "tok_read": "read"},
),
cors_origins=["*"],
)
cls.client = TestClient(app, raise_server_exceptions=False)
@@ -1040,3 +1075,277 @@ class TestConsoleLogin:
self.test_client.post("/v1/api/auth/logout")
resp = self.test_client.get("/v1/api/cluster/overview")
assert resp.status_code == 401
# ---------------------------------------------------------------------------
# Security hardening tests
# ---------------------------------------------------------------------------
class TestLoginRateLimiter:
def test_allows_under_limit(self):
from turnstone.core.auth import LoginRateLimiter
limiter = LoginRateLimiter(max_attempts=3, window_seconds=60)
for _ in range(3):
ok, _ = limiter.check("ip:1.2.3.4")
assert ok
limiter.record("ip:1.2.3.4")
# 4th should be blocked (3 recorded)
ok, retry = limiter.check("ip:1.2.3.4")
assert not ok
assert retry > 0
def test_different_keys_independent(self):
from turnstone.core.auth import LoginRateLimiter
limiter = LoginRateLimiter(max_attempts=2, window_seconds=60)
limiter.record("ip:a")
limiter.record("ip:a")
ok_a, _ = limiter.check("ip:a")
ok_b, _ = limiter.check("ip:b")
assert not ok_a
assert ok_b
def test_cleanup(self):
from turnstone.core.auth import LoginRateLimiter
limiter = LoginRateLimiter(max_attempts=1, window_seconds=60)
limiter.record("ip:old")
removed = limiter.cleanup(max_age=0.0)
assert removed == 1
ok, _ = limiter.check("ip:old")
assert ok
def test_max_keys_protection(self):
from turnstone.core.auth import LoginRateLimiter
limiter = LoginRateLimiter(max_attempts=5, window_seconds=60)
limiter.MAX_KEYS = 2
limiter.record("a")
limiter.record("b")
limiter.record("c") # should be silently dropped (at capacity)
assert "c" not in limiter._attempts
class TestJWTAudienceIssuer:
SECRET = "test-secret-that-is-at-least-32-chars"
def test_create_jwt_includes_iss(self):
import jwt as pyjwt
from turnstone.core.auth import JWT_ISSUER, create_jwt
token = create_jwt("user1", frozenset({"read"}), "test", self.SECRET)
payload = pyjwt.decode(
token, self.SECRET, algorithms=["HS256"], options={"verify_aud": False}
)
assert payload["iss"] == JWT_ISSUER
def test_create_jwt_with_audience(self):
import jwt as pyjwt
from turnstone.core.auth import JWT_AUD_SERVER, create_jwt
token = create_jwt(
"user1", frozenset({"read"}), "test", self.SECRET, audience=JWT_AUD_SERVER
)
payload = pyjwt.decode(token, self.SECRET, algorithms=["HS256"], audience=JWT_AUD_SERVER)
assert payload["aud"] == JWT_AUD_SERVER
def test_validate_jwt_wrong_audience_rejected(self):
from turnstone.core.auth import JWT_AUD_CONSOLE, JWT_AUD_SERVER, create_jwt, validate_jwt
token = create_jwt(
"user1", frozenset({"read"}), "test", self.SECRET, audience=JWT_AUD_SERVER
)
result = validate_jwt(token, self.SECRET, audience=JWT_AUD_CONSOLE)
assert result is None
def test_validate_jwt_correct_audience_accepted(self):
from turnstone.core.auth import JWT_AUD_SERVER, create_jwt, validate_jwt
token = create_jwt(
"user1", frozenset({"read"}), "test", self.SECRET, audience=JWT_AUD_SERVER
)
result = validate_jwt(token, self.SECRET, audience=JWT_AUD_SERVER)
assert result is not None
assert result.user_id == "user1"
def test_validate_jwt_no_audience_backward_compat(self):
from turnstone.core.auth import create_jwt, validate_jwt
# Token without aud claim should be accepted when audience="" (backward compat)
token = create_jwt("user1", frozenset({"read"}), "test", self.SECRET)
result = validate_jwt(token, self.SECRET, audience="")
assert result is not None
class TestServiceTokenManager:
SECRET = "test-secret-that-is-at-least-32-chars"
def test_auto_mints_on_first_access(self):
from turnstone.core.auth import ServiceTokenManager
mgr = ServiceTokenManager(
user_id="svc",
scopes=frozenset({"read"}),
source="test",
secret=self.SECRET,
)
token = mgr.token
assert token # non-empty
assert isinstance(token, str)
def test_bearer_header_format(self):
from turnstone.core.auth import ServiceTokenManager
mgr = ServiceTokenManager(
user_id="svc",
scopes=frozenset({"read"}),
source="test",
secret=self.SECRET,
)
header = mgr.bearer_header
assert "Authorization" in header
assert header["Authorization"].startswith("Bearer ")
def test_token_stable_within_window(self):
from turnstone.core.auth import ServiceTokenManager
mgr = ServiceTokenManager(
user_id="svc",
scopes=frozenset({"read"}),
source="test",
secret=self.SECRET,
expiry_hours=1,
)
t1 = mgr.token
t2 = mgr.token
assert t1 == t2
def test_token_rotates_near_expiry(self):
from turnstone.core.auth import ServiceTokenManager
mgr = ServiceTokenManager(
user_id="svc",
scopes=frozenset({"read"}),
source="test",
secret=self.SECRET,
expiry_hours=1,
)
_ = mgr.token # initial mint
# Simulate expiry by backdating _expires_at
mgr._expires_at = 0.0
t2 = mgr.token
# Token was re-minted (even if payload matches within same second,
# the internal state was refreshed)
assert t2 # non-empty, valid token
assert mgr._expires_at > 0.0 # was refreshed
def test_audience_included(self):
import jwt as pyjwt
from turnstone.core.auth import JWT_AUD_SERVER, ServiceTokenManager
mgr = ServiceTokenManager(
user_id="svc",
scopes=frozenset({"read"}),
source="test",
secret=self.SECRET,
audience=JWT_AUD_SERVER,
)
payload = pyjwt.decode(
mgr.token, self.SECRET, algorithms=["HS256"], audience=JWT_AUD_SERVER
)
assert payload["aud"] == JWT_AUD_SERVER
class TestIsSecureRequest:
def test_https_scheme(self):
from turnstone.core.auth import is_secure_request
assert is_secure_request({}, scheme="https") is True
def test_http_scheme(self):
from turnstone.core.auth import is_secure_request
assert is_secure_request({}, scheme="http") is False
def test_x_forwarded_proto_https(self):
from turnstone.core.auth import is_secure_request
assert is_secure_request({"x-forwarded-proto": "https"}, scheme="http") is True
def test_x_forwarded_proto_http(self):
from turnstone.core.auth import is_secure_request
assert is_secure_request({"x-forwarded-proto": "http"}, scheme="http") is False
class TestSecretStrength:
def test_short_secret_warns(self, caplog):
import logging
from turnstone.core.auth import _MIN_SECRET_LENGTH
with caplog.at_level(logging.WARNING, logger="turnstone.core.auth"):
import turnstone.core.auth as auth_mod
old = os.environ.get("TURNSTONE_JWT_SECRET", "")
os.environ["TURNSTONE_JWT_SECRET"] = "short"
try:
secret = auth_mod.load_jwt_secret()
assert secret == "short"
assert any(str(_MIN_SECRET_LENGTH) in r.message for r in caplog.records)
finally:
if old:
os.environ["TURNSTONE_JWT_SECRET"] = old
else:
os.environ.pop("TURNSTONE_JWT_SECRET", None)
class TestCorsConfigurable:
"""Verify CORS middleware is only added when origins are configured."""
def test_no_cors_origins_no_cors_headers(self):
"""Without cors_origins, no Access-Control headers."""
from starlette.testclient import TestClient
import turnstone.server as srv_mod
app = srv_mod.create_app(
workstreams=MagicMock(),
global_queue=queue.Queue(),
global_listeners=[],
global_listeners_lock=threading.Lock(),
skip_permissions=False,
auth_config=AuthConfig(enabled=False),
)
client = TestClient(app)
resp = client.get("/health", headers={"Origin": "http://evil.com"})
assert "Access-Control-Allow-Origin" not in resp.headers
client.close()
def test_cors_origins_set(self):
"""With cors_origins, CORS headers are present."""
from starlette.testclient import TestClient
import turnstone.server as srv_mod
app = srv_mod.create_app(
workstreams=MagicMock(),
global_queue=queue.Queue(),
global_listeners=[],
global_listeners_lock=threading.Lock(),
skip_permissions=False,
auth_config=AuthConfig(enabled=False),
cors_origins=["http://example.com"],
)
client = TestClient(app)
resp = client.get(
"/health",
headers={"Origin": "http://example.com"},
)
assert resp.headers.get("Access-Control-Allow-Origin") == "http://example.com"
client.close()
+357
View File
@@ -0,0 +1,357 @@
"""Tests for user identity, API tokens, JWT, and scoped auth."""
from __future__ import annotations
import time
import pytest
from turnstone.core.auth import (
AuthConfig,
AuthResult,
_authenticate_token,
check_request,
create_jwt,
generate_token,
hash_password,
hash_token,
parse_scopes,
required_scope,
token_prefix,
validate_jwt,
verify_password,
)
# ---------------------------------------------------------------------------
# AuthResult
# ---------------------------------------------------------------------------
class TestAuthResult:
def test_frozen(self):
r = AuthResult(user_id="u1", scopes=frozenset({"read"}), token_source="config")
with pytest.raises(AttributeError):
r.user_id = "u2" # type: ignore[misc]
def test_has_scope(self):
r = AuthResult(user_id="", scopes=frozenset({"read", "write"}), token_source="config")
assert r.has_scope("read")
assert r.has_scope("write")
assert not r.has_scope("approve")
def test_empty_scopes(self):
r = AuthResult(user_id="", scopes=frozenset(), token_source="config")
assert not r.has_scope("read")
# ---------------------------------------------------------------------------
# Token generation and hashing
# ---------------------------------------------------------------------------
class TestTokenHelpers:
def test_generate_token_format(self):
tok = generate_token()
assert tok.startswith("ts_")
assert len(tok) == 3 + 64 # ts_ + 64 hex chars
def test_generate_token_unique(self):
tokens = {generate_token() for _ in range(10)}
assert len(tokens) == 10
def test_hash_token_deterministic(self):
assert hash_token("ts_abc") == hash_token("ts_abc")
def test_hash_token_hex(self):
h = hash_token("test")
assert len(h) == 64 # SHA-256 hex
int(h, 16) # valid hex
def test_token_prefix(self):
assert token_prefix("ts_abcdefgh1234") == "ts_abcde"
# ---------------------------------------------------------------------------
# Password hashing (bcrypt)
# ---------------------------------------------------------------------------
class TestPasswordHashing:
def test_hash_and_verify(self):
pw = "hunter2"
hashed = hash_password(pw)
assert verify_password(pw, hashed)
def test_wrong_password(self):
hashed = hash_password("correct")
assert not verify_password("wrong", hashed)
def test_hash_is_different_each_time(self):
h1 = hash_password("same")
h2 = hash_password("same")
assert h1 != h2 # different salts
# ---------------------------------------------------------------------------
# Scope parsing
# ---------------------------------------------------------------------------
class TestParseScopes:
def test_single_scope(self):
assert parse_scopes("read") == frozenset({"read"})
def test_hierarchy_write(self):
assert parse_scopes("write") == frozenset({"read", "write"})
def test_hierarchy_approve(self):
assert parse_scopes("approve") == frozenset({"read", "write", "approve"})
def test_comma_separated(self):
assert parse_scopes("read,write") == frozenset({"read", "write"})
def test_redundant_scopes(self):
# approve already includes read,write
assert parse_scopes("read,approve") == frozenset({"read", "write", "approve"})
def test_empty_string(self):
assert parse_scopes("") == frozenset()
def test_invalid_scope_filtered(self):
assert parse_scopes("bogus") == frozenset()
def test_mixed_valid_invalid(self):
assert parse_scopes("read,bogus,approve") == frozenset({"read", "write", "approve"})
# ---------------------------------------------------------------------------
# JWT create / validate
# ---------------------------------------------------------------------------
class TestJWT:
SECRET = "test-secret-key-for-jwt"
def test_round_trip(self):
scopes = frozenset({"read", "write"})
token = create_jwt("user123", scopes, "database", self.SECRET, expiry_hours=1)
result = validate_jwt(token, self.SECRET)
assert result is not None
assert result.user_id == "user123"
assert result.scopes == frozenset({"read", "write"})
def test_expired_token(self):
import jwt
payload = {
"sub": "user1",
"scopes": "read",
"src": "database",
"iat": int(time.time()) - 7200,
"exp": int(time.time()) - 3600,
}
token = jwt.encode(payload, self.SECRET, algorithm="HS256")
assert validate_jwt(token, self.SECRET) is None
def test_invalid_signature(self):
token = create_jwt("user1", frozenset({"read"}), "db", self.SECRET)
assert validate_jwt(token, "wrong-secret") is None
def test_malformed_token(self):
assert validate_jwt("not.a.jwt", self.SECRET) is None
def test_contains_dots(self):
"""JWTs contain dots, used for detection."""
token = create_jwt("u1", frozenset({"read"}), "db", self.SECRET)
assert "." in token
# ---------------------------------------------------------------------------
# required_scope
# ---------------------------------------------------------------------------
class TestRequiredScope:
def test_get_read(self):
assert required_scope("GET", "/api/workstreams") == "read"
def test_post_write(self):
assert required_scope("POST", "/api/send") == "write"
def test_post_approve(self):
assert required_scope("POST", "/api/approve") == "approve"
def test_admin_prefix(self):
assert required_scope("GET", "/api/admin/users") == "approve"
assert required_scope("POST", "/api/admin/users") == "approve"
assert required_scope("DELETE", "/api/admin/users/abc") == "approve"
def test_versioned_path(self):
assert required_scope("POST", "/v1/api/send") == "write"
assert required_scope("POST", "/v1/api/approve") == "approve"
def test_proxy_write(self):
assert required_scope("POST", "/node/n1/api/send") == "write"
def test_proxy_approve(self):
assert required_scope("POST", "/node/n1/api/approve") == "approve"
# ---------------------------------------------------------------------------
# _authenticate_token
# ---------------------------------------------------------------------------
class TestAuthenticateToken:
def test_config_token_read(self):
cfg = AuthConfig(enabled=True, tokens={"tok_read": "read"})
result = _authenticate_token("tok_read", cfg)
assert result is not None
assert result.scopes == frozenset({"read"})
assert result.token_source == "config"
def test_config_token_full(self):
cfg = AuthConfig(enabled=True, tokens={"tok_full": "full"})
result = _authenticate_token("tok_full", cfg)
assert result is not None
assert result.scopes == frozenset({"read", "write", "approve"})
def test_jwt_token(self):
secret = "test-secret"
jwt_tok = create_jwt("user1", frozenset({"read", "write"}), "db", secret)
cfg = AuthConfig(enabled=True)
result = _authenticate_token(jwt_tok, cfg, jwt_secret=secret)
assert result is not None
assert result.user_id == "user1"
assert result.token_source == "db"
def test_api_token_with_storage(self):
"""API tokens are looked up by hash in storage."""
raw = generate_token()
class MockStorage:
def get_api_token_by_hash(self, token_hash):
expected = hash_token(raw)
if token_hash == expected:
return {
"token_id": "tid",
"token_prefix": "ts_abcde",
"user_id": "user1",
"name": "test",
"scopes": "read,write",
"created": "2026-01-01T00:00:00",
}
return None
cfg = AuthConfig(enabled=True)
result = _authenticate_token(raw, cfg, storage=MockStorage())
assert result is not None
assert result.user_id == "user1"
assert result.has_scope("write")
assert result.token_source == "database"
def test_api_token_expired(self):
"""Expired API tokens are rejected."""
raw = generate_token()
class MockStorage:
def get_api_token_by_hash(self, token_hash):
return {
"token_id": "tid",
"token_prefix": "ts_abcde",
"user_id": "user1",
"name": "test",
"scopes": "read",
"created": "2020-01-01T00:00:00",
"expires": "2020-01-02T00:00:00",
}
cfg = AuthConfig(enabled=True)
result = _authenticate_token(raw, cfg, storage=MockStorage())
assert result is None
def test_unknown_token(self):
cfg = AuthConfig(enabled=True, tokens={"tok": "full"})
result = _authenticate_token("unknown", cfg)
assert result is None
# ---------------------------------------------------------------------------
# check_request with scopes
# ---------------------------------------------------------------------------
class TestCheckRequestScopes:
def test_config_read_on_write_403(self):
cfg = AuthConfig(enabled=True, tokens={"tok_read": "read"})
allowed, status, msg, _ = check_request(cfg, "POST", "/api/send", "Bearer tok_read")
assert not allowed
assert status == 403
assert "write" in msg
def test_config_read_on_approve_403(self):
cfg = AuthConfig(enabled=True, tokens={"tok_read": "read"})
allowed, status, msg, _ = check_request(cfg, "POST", "/api/approve", "Bearer tok_read")
assert not allowed
assert status == 403
assert "approve" in msg
def test_config_full_on_approve_ok(self):
cfg = AuthConfig(enabled=True, tokens={"tok_full": "full"})
allowed, status, msg, result = check_request(cfg, "POST", "/api/approve", "Bearer tok_full")
assert allowed
assert result is not None
assert result.has_scope("approve")
def test_jwt_with_scopes(self):
secret = "test"
jwt_tok = create_jwt("u1", frozenset({"read", "write"}), "db", secret)
cfg = AuthConfig(enabled=True)
allowed, status, msg, result = check_request(
cfg,
"POST",
"/api/send",
f"Bearer {jwt_tok}",
jwt_secret=secret,
)
assert allowed
assert result is not None
assert result.user_id == "u1"
def test_jwt_insufficient_scope(self):
secret = "test"
jwt_tok = create_jwt("u1", frozenset({"read"}), "db", secret)
cfg = AuthConfig(enabled=True)
allowed, status, msg, _ = check_request(
cfg,
"POST",
"/api/send",
f"Bearer {jwt_tok}",
jwt_secret=secret,
)
assert not allowed
assert status == 403
def test_admin_path_requires_approve(self):
cfg = AuthConfig(enabled=True, tokens={"tok_read": "read"})
allowed, status, msg, _ = check_request(
cfg,
"GET",
"/v1/api/admin/users",
"Bearer tok_read",
)
assert not allowed
assert status == 403
def test_backward_compat_role_full(self):
"""Config tokens with role='full' get all scopes."""
cfg = AuthConfig(enabled=True, tokens={"tok_full": "full"})
allowed, _, _, result = check_request(
cfg,
"GET",
"/v1/api/admin/users",
"Bearer tok_full",
)
assert allowed
assert result is not None
assert result.has_scope("approve")
+328
View File
@@ -0,0 +1,328 @@
"""Tests for the Discord channel adapter (bot, cog, views, config, CLI)."""
from __future__ import annotations
import asyncio
import sys
from unittest.mock import AsyncMock, MagicMock, patch
import pytest
discord = pytest.importorskip("discord")
# ---------------------------------------------------------------------------
# Helpers
# ---------------------------------------------------------------------------
def _run(coro):
"""Run an async coroutine in a fresh event loop (no pytest-asyncio needed)."""
return asyncio.run(coro)
def _make_message(*, bot=False, guild=True, content="hello", channel=None):
"""Build a mock ``discord.Message``."""
msg = MagicMock(spec=discord.Message)
msg.author = MagicMock()
msg.author.bot = bot
msg.author.id = 12345
msg.content = content
msg.guild = MagicMock() if guild else None
msg.channel = channel or MagicMock()
msg.mentions = []
return msg
def _make_interaction(*, footer_text=None, has_embeds=True):
"""Build a mock ``discord.Interaction``."""
interaction = MagicMock(spec=discord.Interaction)
interaction.user = MagicMock()
interaction.user.id = 67890
interaction.response = MagicMock()
interaction.response.send_message = AsyncMock()
if has_embeds and footer_text is not None:
embed = MagicMock()
embed.footer.text = footer_text
interaction.message = MagicMock()
interaction.message.embeds = [embed]
elif not has_embeds:
interaction.message = MagicMock()
interaction.message.embeds = []
else:
interaction.message = None
return interaction
# ---------------------------------------------------------------------------
# DiscordConfig
# ---------------------------------------------------------------------------
class TestDiscordConfig:
"""Tests for DiscordConfig default and custom values."""
def test_defaults(self):
from turnstone.channels.discord.config import DiscordConfig
cfg = DiscordConfig()
assert cfg.bot_token == ""
assert cfg.guild_id == 0
assert cfg.allowed_channels == []
assert cfg.thread_auto_archive == 1440
assert cfg.max_message_length == 2000
assert cfg.streaming_edit_interval == 1.5
# Inherited from ChannelConfig
assert cfg.redis_host == "localhost"
assert cfg.redis_port == 6379
assert cfg.model == ""
assert cfg.auto_approve is False
def test_custom_values(self):
from turnstone.channels.discord.config import DiscordConfig
cfg = DiscordConfig(
bot_token="tok_123",
guild_id=999,
allowed_channels=[1, 2, 3],
thread_auto_archive=60,
max_message_length=4000,
streaming_edit_interval=0.5,
model="gpt-5",
auto_approve=True,
)
assert cfg.bot_token == "tok_123"
assert cfg.guild_id == 999
assert cfg.allowed_channels == [1, 2, 3]
assert cfg.thread_auto_archive == 60
assert cfg.max_message_length == 4000
assert cfg.streaming_edit_interval == 0.5
assert cfg.model == "gpt-5"
assert cfg.auto_approve is True
# ---------------------------------------------------------------------------
# StreamingMessage
# ---------------------------------------------------------------------------
class TestStreamingMessage:
"""Tests for the StreamingMessage helper in bot.py."""
def test_append_accumulates(self):
from turnstone.channels.discord.bot import StreamingMessage
channel = MagicMock()
channel.send = AsyncMock()
sm = StreamingMessage(channel=channel, edit_interval=999.0)
_run(sm.append("hello "))
_run(sm.append("world"))
assert "".join(sm._buffer) == "hello world"
def test_finalize_sends_when_no_prior_message(self):
from turnstone.channels.discord.bot import StreamingMessage
channel = MagicMock()
channel.send = AsyncMock()
sm = StreamingMessage(channel=channel, edit_interval=999.0)
_run(sm.append("hello"))
_run(sm.finalize())
channel.send.assert_awaited_once_with("hello")
def test_finalize_edits_existing_message(self):
from turnstone.channels.discord.bot import StreamingMessage
channel = MagicMock()
sent_msg = MagicMock()
sent_msg.edit = AsyncMock()
channel.send = AsyncMock(return_value=sent_msg)
sm = StreamingMessage(channel=channel, edit_interval=0.0)
# First append triggers flush (interval=0) which creates the message.
_run(sm.append("hi"))
assert sm._message is sent_msg
_run(sm.append(" there"))
_run(sm.finalize())
# finalize edits the existing message with full content.
sent_msg.edit.assert_awaited_with(content="hi there")
def test_finalize_chunks_long_content(self):
from turnstone.channels.discord.bot import StreamingMessage
channel = MagicMock()
channel.send = AsyncMock()
sm = StreamingMessage(channel=channel, max_length=10, edit_interval=999.0)
# Content longer than max_length should be chunked on finalize.
_run(sm.append("a" * 25))
_run(sm.finalize())
# Should have sent multiple chunks via channel.send.
assert channel.send.await_count >= 2
def test_finalize_empty_is_noop(self):
from turnstone.channels.discord.bot import StreamingMessage
channel = MagicMock()
channel.send = AsyncMock()
sm = StreamingMessage(channel=channel)
_run(sm.finalize())
channel.send.assert_not_awaited()
# ---------------------------------------------------------------------------
# MessageCog._on_message
# ---------------------------------------------------------------------------
class TestMessageCog:
"""Tests for the MessageCog on_message filtering logic."""
def _make_cog(self):
"""Build a MessageCog with a fully mocked bot and TurnstoneBot."""
from turnstone.channels.discord.cog import MessageCog
bot = MagicMock()
bot.user = MagicMock()
bot.user.id = 99999
bot.user.mentioned_in = MagicMock(return_value=False)
ts = MagicMock()
ts._is_allowed_channel = MagicMock(return_value=True)
ts.storage = MagicMock()
ts.router = MagicMock()
ts.router.resolve_user = AsyncMock(return_value="u_abc")
ts.router.send_message = AsyncMock()
ts.config = MagicMock()
ts._ws_tasks = {}
bot.turnstone = ts
cog = MessageCog(bot)
return cog, ts, bot
def test_ignores_bot_messages(self):
cog, ts, _bot = self._make_cog()
msg = _make_message(bot=True)
_run(cog._on_message(msg))
# No router interaction means the message was ignored.
ts.router.send_message.assert_not_awaited()
def test_ignores_own_messages(self):
cog, ts, bot = self._make_cog()
msg = _make_message(bot=False)
msg.author = bot.user # message from ourselves
_run(cog._on_message(msg))
ts.router.send_message.assert_not_awaited()
def test_ignores_dms(self):
cog, ts, _bot = self._make_cog()
msg = _make_message(guild=False)
_run(cog._on_message(msg))
ts.router.send_message.assert_not_awaited()
def test_ignores_non_allowed_channels(self):
cog, ts, _bot = self._make_cog()
ts._is_allowed_channel = MagicMock(return_value=False)
thread = MagicMock(spec=discord.Thread)
thread.id = 111
thread.parent_id = 222
msg = _make_message(channel=thread)
_run(cog._on_message(msg))
ts.router.send_message.assert_not_awaited()
# ---------------------------------------------------------------------------
# _parse_footer (views.py)
# ---------------------------------------------------------------------------
class TestParseFooter:
"""Tests for _parse_footer in views.py."""
def test_valid_footer(self):
from turnstone.channels.discord.views import _parse_footer
interaction = _make_interaction(footer_text="ws_abc|corr_123")
result = _parse_footer(interaction)
assert result == ("ws_abc", "corr_123")
def test_footer_with_pipe_in_correlation(self):
from turnstone.channels.discord.views import _parse_footer
interaction = _make_interaction(footer_text="ws_abc|corr|extra")
result = _parse_footer(interaction)
# split("|", 1) means the second part includes everything after first pipe.
assert result == ("ws_abc", "corr|extra")
def test_no_message_returns_none(self):
from turnstone.channels.discord.views import _parse_footer
interaction = MagicMock()
interaction.message = None
assert _parse_footer(interaction) is None
def test_no_embeds_returns_none(self):
from turnstone.channels.discord.views import _parse_footer
interaction = _make_interaction(has_embeds=False)
assert _parse_footer(interaction) is None
def test_empty_footer_returns_none(self):
from turnstone.channels.discord.views import _parse_footer
# Build an interaction whose embed has footer.text = None.
interaction = MagicMock(spec=discord.Interaction)
embed = MagicMock()
embed.footer.text = None
interaction.message = MagicMock()
interaction.message.embeds = [embed]
assert _parse_footer(interaction) is None
def test_footer_without_pipe_returns_none(self):
from turnstone.channels.discord.views import _parse_footer
interaction = _make_interaction(footer_text="no_pipe_here")
# footer text has no "|" separator
embed = MagicMock()
embed.footer.text = "no_pipe_here"
interaction.message.embeds = [embed]
assert _parse_footer(interaction) is None
# ---------------------------------------------------------------------------
# CLI main() — no adapter configured
# ---------------------------------------------------------------------------
class TestChannelCLI:
"""Tests for the channel CLI entry point."""
def test_exits_without_adapter_token(self):
from turnstone.channels.cli import main
with (
patch.object(sys, "argv", ["turnstone-channel"]),
patch.dict("os.environ", {}, clear=True),
pytest.raises(SystemExit) as exc_info,
):
main()
assert exc_info.value.code == 1
+203
View File
@@ -0,0 +1,203 @@
"""Tests for turnstone.channels._protocol and turnstone.channels._formatter."""
from __future__ import annotations
from turnstone.channels._formatter import (
chunk_message,
format_approval_request,
format_plan_review,
truncate,
)
from turnstone.channels._protocol import ChannelEvent
# ---------------------------------------------------------------------------
# ChannelEvent
# ---------------------------------------------------------------------------
class TestChannelEvent:
def test_construction(self) -> None:
evt = ChannelEvent(
channel_type="discord",
channel_id="ch-1",
channel_user_id="u-42",
message="hello",
parent_channel_id="parent",
metadata={"key": "val"},
)
assert evt.channel_type == "discord"
assert evt.channel_id == "ch-1"
assert evt.channel_user_id == "u-42"
assert evt.message == "hello"
assert evt.parent_channel_id == "parent"
assert evt.metadata == {"key": "val"}
def test_defaults(self) -> None:
evt = ChannelEvent(
channel_type="slack",
channel_id="ch-2",
channel_user_id="u-7",
message="hi",
)
assert evt.parent_channel_id == ""
assert evt.metadata == {}
def test_metadata_independence(self) -> None:
"""Default metadata dicts are independent across instances."""
a = ChannelEvent(channel_type="x", channel_id="1", channel_user_id="u", message="m")
b = ChannelEvent(channel_type="x", channel_id="2", channel_user_id="u", message="m")
a.metadata["key"] = "val"
assert "key" not in b.metadata
# ---------------------------------------------------------------------------
# chunk_message
# ---------------------------------------------------------------------------
class TestChunkMessage:
def test_empty_string(self) -> None:
assert chunk_message("") == [""]
def test_under_limit(self) -> None:
assert chunk_message("short text", max_length=100) == ["short text"]
def test_exactly_at_limit(self) -> None:
text = "a" * 50
assert chunk_message(text, max_length=50) == [text]
def test_splits_at_newline(self) -> None:
text = "line one\nline two\nline three"
chunks = chunk_message(text, max_length=18)
assert len(chunks) >= 2
# The split should happen at a newline boundary within the text.
# Reassembled chunks (with newline separators) should cover all content.
rejoined = "\n".join(chunks)
assert "line one" in rejoined
assert "line three" in rejoined
def test_splits_at_word_boundary(self) -> None:
text = "word1 word2 word3 word4"
chunks = chunk_message(text, max_length=12)
assert len(chunks) >= 2
# No chunk should start with a space (lstrip handles newlines).
for chunk in chunks:
assert not chunk.startswith("\n")
def test_hard_splits(self) -> None:
text = "a" * 30
chunks = chunk_message(text, max_length=10)
assert len(chunks) == 3
assert "".join(chunks) == text
def test_code_block_spanning_boundary(self) -> None:
text = "before\n```\ncode line 1\ncode line 2\ncode line 3\n```\nafter"
chunks = chunk_message(text, max_length=30)
assert len(chunks) >= 2
# If a chunk opens a code block without closing it, the chunker
# should close it and reopen in the next chunk.
for chunk in chunks:
fence_count = chunk.count("```")
assert fence_count % 2 == 0, f"Unmatched code fence in chunk: {chunk!r}"
def test_multiple_code_blocks(self) -> None:
text = "```\nblock1\n```\ntext\n```\nblock2\n```"
chunks = chunk_message(text, max_length=20)
for chunk in chunks:
fence_count = chunk.count("```")
assert fence_count % 2 == 0, f"Unmatched code fence in chunk: {chunk!r}"
def test_custom_max_length(self) -> None:
text = "hello world"
chunks = chunk_message(text, max_length=5)
assert len(chunks) >= 2
assert chunks[0] == "hello"
def test_very_long_single_line(self) -> None:
text = "x" * 5000
chunks = chunk_message(text, max_length=2000)
assert len(chunks) == 3
total = "".join(chunks)
assert total == text
# ---------------------------------------------------------------------------
# format_approval_request
# ---------------------------------------------------------------------------
class TestFormatApprovalRequest:
def test_single_tool(self) -> None:
items = [{"function": {"name": "read_file", "arguments": "/etc/hosts"}}]
result = format_approval_request(items)
assert "Tool approval required" in result
assert "`read_file`" in result
def test_multiple_tools(self) -> None:
items = [
{"function": {"name": "tool_a", "arguments": "arg1"}},
{"function": {"name": "tool_b", "arguments": "arg2"}},
]
result = format_approval_request(items)
assert "`tool_a`" in result
assert "`tool_b`" in result
def test_long_arguments_truncated(self) -> None:
long_args = "x" * 500
items = [{"function": {"name": "fn", "arguments": long_args}}]
result = format_approval_request(items)
# The result should be shorter than the original args.
assert len(result) < 500
def test_server_sse_format(self) -> None:
"""Items from the server SSE use func_name/preview, not function.name."""
items = [
{
"call_id": "c1",
"func_name": "bash",
"preview": "ls -la",
"header": "Execute: ls -la",
"needs_approval": True,
}
]
result = format_approval_request(items)
assert "`bash`" in result
assert "Execute: ls -la" in result
def test_server_sse_format_no_header(self) -> None:
items = [{"func_name": "read_file", "preview": "/etc/hosts"}]
result = format_approval_request(items)
assert "`read_file`" in result
assert "/etc/hosts" in result
# ---------------------------------------------------------------------------
# format_plan_review
# ---------------------------------------------------------------------------
class TestFormatPlanReview:
def test_format(self) -> None:
result = format_plan_review("Step 1: do stuff")
assert result.startswith("**Plan review requested:**")
assert "Step 1: do stuff" in result
# ---------------------------------------------------------------------------
# truncate
# ---------------------------------------------------------------------------
class TestTruncate:
def test_short_text_unchanged(self) -> None:
assert truncate("hello", max_length=200) == "hello"
def test_long_text_truncated(self) -> None:
text = "a" * 300
result = truncate(text, max_length=200)
assert len(result) == 200
assert result.endswith("\u2026")
def test_exactly_at_limit(self) -> None:
text = "b" * 200
assert truncate(text, max_length=200) == text
+107
View File
@@ -0,0 +1,107 @@
"""Tests for turnstone.channels._routing.ChannelRouter."""
from __future__ import annotations
import json
from unittest.mock import AsyncMock, MagicMock
import pytest
from turnstone.channels._routing import ChannelRouter
@pytest.fixture
def mock_broker() -> AsyncMock:
"""Return a mock AsyncRedisBroker."""
broker = AsyncMock()
broker._prefix = "test"
broker.push_inbound = AsyncMock()
broker.push_response = AsyncMock()
broker.subscribe = AsyncMock()
broker.unsubscribe = AsyncMock()
return broker
@pytest.fixture
def mock_storage() -> MagicMock:
"""Return a mock StorageBackend."""
storage = MagicMock()
storage.get_channel_user = MagicMock(return_value=None)
storage.get_channel_route = MagicMock(return_value=None)
storage.get_channel_route_by_ws = MagicMock(return_value=None)
storage.create_channel_route = MagicMock()
storage.delete_channel_route = MagicMock(return_value=True)
return storage
@pytest.fixture
def router(mock_broker: AsyncMock, mock_storage: MagicMock) -> ChannelRouter:
return ChannelRouter(broker=mock_broker, storage=mock_storage)
class TestResolveUser:
@pytest.mark.anyio
async def test_linked_user(self, router: ChannelRouter, mock_storage: MagicMock) -> None:
mock_storage.get_channel_user.return_value = {"user_id": "usr-1", "channel_user_id": "d-42"}
result = await router.resolve_user("discord", "d-42")
assert result == "usr-1"
mock_storage.get_channel_user.assert_called_once_with("discord", "d-42")
@pytest.mark.anyio
async def test_unlinked_user(self, router: ChannelRouter, mock_storage: MagicMock) -> None:
mock_storage.get_channel_user.return_value = None
result = await router.resolve_user("slack", "s-99")
assert result is None
class TestSendMessage:
@pytest.mark.anyio
async def test_pushes_send_message(self, router: ChannelRouter, mock_broker: AsyncMock) -> None:
cid = await router.send_message("ws-1", "hello world")
assert isinstance(cid, str)
assert len(cid) > 0
mock_broker.push_inbound.assert_awaited_once()
raw = mock_broker.push_inbound.call_args[0][0]
payload = json.loads(raw)
assert payload["type"] == "send"
assert payload["ws_id"] == "ws-1"
assert payload["message"] == "hello world"
assert payload["correlation_id"] == cid
class TestSendApproval:
@pytest.mark.anyio
async def test_pushes_to_response_queue(
self, router: ChannelRouter, mock_broker: AsyncMock
) -> None:
await router.send_approval("ws-1", "corr-abc", approved=True, feedback="ok")
mock_broker.push_response.assert_awaited_once()
queue_name = mock_broker.push_response.call_args[0][0]
assert queue_name == "corr-abc"
raw = mock_broker.push_response.call_args[0][1]
payload = json.loads(raw)
assert payload["type"] == "approve"
assert payload["approved"] is True
assert payload["ws_id"] == "ws-1"
class TestSendPlanFeedback:
@pytest.mark.anyio
async def test_pushes_to_response_queue(
self, router: ChannelRouter, mock_broker: AsyncMock
) -> None:
await router.send_plan_feedback("ws-2", "corr-xyz", "looks good")
mock_broker.push_response.assert_awaited_once()
raw = mock_broker.push_response.call_args[0][1]
payload = json.loads(raw)
assert payload["type"] == "plan_feedback"
assert payload["feedback"] == "looks good"
class TestDeleteRoute:
@pytest.mark.anyio
async def test_calls_storage_delete(
self, router: ChannelRouter, mock_storage: MagicMock
) -> None:
await router.delete_route("discord", "ch-123")
mock_storage.delete_channel_route.assert_called_once_with("discord", "ch-123")
+136
View File
@@ -0,0 +1,136 @@
"""Tests for channel_users and channel_routes storage CRUD."""
from __future__ import annotations
import pytest
from turnstone.core.storage._sqlite import SQLiteBackend
@pytest.fixture
def db(tmp_path):
"""Fresh SQLite backend for each test."""
backend = SQLiteBackend(str(tmp_path / "test.db"))
return backend
class TestChannelUserCRUD:
"""Tests for channel_users table operations."""
def test_create_and_get(self, db):
db.create_channel_user("discord", "12345", "u_abc")
result = db.get_channel_user("discord", "12345")
assert result is not None
assert result["channel_type"] == "discord"
assert result["channel_user_id"] == "12345"
assert result["user_id"] == "u_abc"
assert "created" in result
def test_get_nonexistent(self, db):
assert db.get_channel_user("discord", "99999") is None
def test_create_duplicate_noop(self, db):
db.create_channel_user("discord", "12345", "u_abc")
db.create_channel_user("discord", "12345", "u_different")
result = db.get_channel_user("discord", "12345")
assert result is not None
assert result["user_id"] == "u_abc" # first write wins
def test_same_user_different_channels(self, db):
db.create_channel_user("discord", "d_123", "u_abc")
db.create_channel_user("slack", "s_456", "u_abc")
d = db.get_channel_user("discord", "d_123")
s = db.get_channel_user("slack", "s_456")
assert d is not None and d["user_id"] == "u_abc"
assert s is not None and s["user_id"] == "u_abc"
def test_list_by_user(self, db):
db.create_channel_user("discord", "d_123", "u_abc")
db.create_channel_user("slack", "s_456", "u_abc")
db.create_channel_user("discord", "d_999", "u_other")
results = db.list_channel_users_by_user("u_abc")
assert len(results) == 2
types = {r["channel_type"] for r in results}
assert types == {"discord", "slack"}
def test_list_by_user_empty(self, db):
assert db.list_channel_users_by_user("u_nobody") == []
def test_delete(self, db):
db.create_channel_user("discord", "12345", "u_abc")
assert db.delete_channel_user("discord", "12345") is True
assert db.get_channel_user("discord", "12345") is None
def test_delete_nonexistent(self, db):
assert db.delete_channel_user("discord", "99999") is False
def test_delete_user_cascades_channel_users(self, db):
"""Deleting a turnstone user should cascade to channel_users."""
db.create_user("u_abc", "admin", "Admin", "hash123")
db.create_channel_user("discord", "12345", "u_abc")
db.delete_user("u_abc")
assert db.get_channel_user("discord", "12345") is None
class TestChannelRouteCRUD:
"""Tests for channel_routes table operations."""
def test_create_and_get(self, db):
db.create_channel_route("discord", "thread_123", "ws_abc", "node_1")
result = db.get_channel_route("discord", "thread_123")
assert result is not None
assert result["channel_type"] == "discord"
assert result["channel_id"] == "thread_123"
assert result["ws_id"] == "ws_abc"
assert result["node_id"] == "node_1"
assert "created" in result
def test_get_nonexistent(self, db):
assert db.get_channel_route("discord", "thread_999") is None
def test_create_duplicate_noop(self, db):
db.create_channel_route("discord", "thread_123", "ws_abc")
db.create_channel_route("discord", "thread_123", "ws_different")
result = db.get_channel_route("discord", "thread_123")
assert result is not None
assert result["ws_id"] == "ws_abc" # first write wins
def test_default_empty_node_id(self, db):
db.create_channel_route("discord", "thread_123", "ws_abc")
result = db.get_channel_route("discord", "thread_123")
assert result is not None
assert result["node_id"] == ""
def test_get_by_ws(self, db):
db.create_channel_route("discord", "thread_123", "ws_abc", "node_1")
result = db.get_channel_route_by_ws("ws_abc")
assert result is not None
assert result["channel_id"] == "thread_123"
assert result["ws_id"] == "ws_abc"
def test_get_by_ws_nonexistent(self, db):
assert db.get_channel_route_by_ws("ws_nobody") is None
def test_delete(self, db):
db.create_channel_route("discord", "thread_123", "ws_abc")
assert db.delete_channel_route("discord", "thread_123") is True
assert db.get_channel_route("discord", "thread_123") is None
def test_delete_nonexistent(self, db):
assert db.delete_channel_route("discord", "thread_999") is False
def test_multiple_channels_same_type(self, db):
db.create_channel_route("discord", "thread_1", "ws_1")
db.create_channel_route("discord", "thread_2", "ws_2")
r1 = db.get_channel_route("discord", "thread_1")
r2 = db.get_channel_route("discord", "thread_2")
assert r1 is not None and r1["ws_id"] == "ws_1"
assert r2 is not None and r2["ws_id"] == "ws_2"
def test_different_channel_types(self, db):
db.create_channel_route("discord", "thread_1", "ws_1")
db.create_channel_route("slack", "channel_1", "ws_2")
d = db.get_channel_route("discord", "thread_1")
s = db.get_channel_route("slack", "channel_1")
assert d is not None and d["ws_id"] == "ws_1"
assert s is not None and s["ws_id"] == "ws_2"
+214
View File
@@ -0,0 +1,214 @@
"""Tests for turnstone.core.log — structured logging configuration."""
from __future__ import annotations
import json
import logging
import structlog
from turnstone.core.log import (
configure_logging,
ctx_node_id,
ctx_request_id,
ctx_user_id,
ctx_ws_id,
get_logger,
)
class TestConfigureLogging:
"""Test configure_logging() sets up handlers and formatters."""
def setup_method(self):
# Reset structlog and stdlib between tests
structlog.reset_defaults()
root = logging.getLogger()
root.handlers.clear()
root.setLevel(logging.WARNING)
# Reset context vars
for var in (ctx_node_id, ctx_ws_id, ctx_user_id, ctx_request_id):
var.set("")
def test_sets_root_handler(self):
configure_logging(level="INFO", json_output=False, service="test")
root = logging.getLogger()
assert len(root.handlers) == 1
assert root.level == logging.INFO
def test_level_debug(self):
configure_logging(level="DEBUG", json_output=False)
root = logging.getLogger()
assert root.level == logging.DEBUG
def test_level_warning(self):
configure_logging(level="WARNING", json_output=False)
root = logging.getLogger()
assert root.level == logging.WARNING
def test_json_output(self, capsys):
configure_logging(level="INFO", json_output=True, service="test-svc")
log = logging.getLogger("test.json_output")
log.info("hello world")
captured = capsys.readouterr()
# JSON goes to stderr
line = captured.err.strip()
data = json.loads(line)
assert data["event"] == "hello world"
assert data["level"] == "info"
assert data["service"] == "test-svc"
assert "timestamp" in data
def test_console_output(self, capsys):
configure_logging(level="INFO", json_output=False)
log = logging.getLogger("test.console_output")
log.info("console hello")
captured = capsys.readouterr()
assert "console hello" in captured.err
def test_quiet_third_party(self):
configure_logging(level="DEBUG", json_output=False)
for name in ("httpx", "httpcore", "openai", "anthropic", "uvicorn.access"):
assert logging.getLogger(name).level == logging.WARNING
def test_replaces_existing_handlers(self):
root = logging.getLogger()
# Count existing handlers (pytest may add its own)
before = len(root.handlers)
root.addHandler(logging.StreamHandler())
root.addHandler(logging.StreamHandler())
assert len(root.handlers) == before + 2
configure_logging(level="INFO", json_output=False)
# configure_logging clears all and adds exactly 1
assert len(root.handlers) == 1
def test_env_var_level_override(self, monkeypatch):
monkeypatch.setenv("TURNSTONE_LOG_LEVEL", "ERROR")
configure_logging(level="DEBUG", json_output=False)
root = logging.getLogger()
assert root.level == logging.ERROR
def test_env_var_format_json(self, monkeypatch, capsys):
monkeypatch.setenv("TURNSTONE_LOG_FORMAT", "json")
configure_logging(level="INFO", service="test")
log = logging.getLogger("test.env_json")
log.info("env json test")
captured = capsys.readouterr()
data = json.loads(captured.err.strip())
assert data["event"] == "env json test"
def test_env_var_format_text(self, monkeypatch, capsys):
monkeypatch.setenv("TURNSTONE_LOG_FORMAT", "text")
configure_logging(level="INFO", json_output=True) # json_output overridden by env
log = logging.getLogger("test.env_text")
log.info("env text test")
captured = capsys.readouterr()
# Should NOT be JSON
line = captured.err.strip()
assert "env text test" in line
# Verify it's not JSON
try:
json.loads(line)
is_json = True
except json.JSONDecodeError:
is_json = False
assert not is_json
class TestContextInjection:
"""Test that context variables appear in log output."""
def setup_method(self):
structlog.reset_defaults()
root = logging.getLogger()
root.handlers.clear()
root.setLevel(logging.WARNING)
for var in (ctx_node_id, ctx_ws_id, ctx_user_id, ctx_request_id):
var.set("")
def test_node_id_in_output(self, capsys):
configure_logging(level="INFO", json_output=True)
ctx_node_id.set("worker-01_a3f2")
log = logging.getLogger("test.ctx")
log.info("ctx test")
data = json.loads(capsys.readouterr().err.strip())
assert data["node_id"] == "worker-01_a3f2"
def test_ws_id_in_output(self, capsys):
configure_logging(level="INFO", json_output=True)
ctx_ws_id.set("abc123")
log = logging.getLogger("test.ctx")
log.info("ws test")
data = json.loads(capsys.readouterr().err.strip())
assert data["ws_id"] == "abc123"
def test_empty_context_omitted(self, capsys):
configure_logging(level="INFO", json_output=True)
# All context vars are empty string (default)
log = logging.getLogger("test.ctx")
log.info("empty ctx")
data = json.loads(capsys.readouterr().err.strip())
assert "node_id" not in data
assert "ws_id" not in data
assert "user_id" not in data
assert "request_id" not in data
def test_multiple_context_vars(self, capsys):
configure_logging(level="INFO", json_output=True)
ctx_node_id.set("node-1")
ctx_ws_id.set("ws-2")
ctx_request_id.set("req-3")
log = logging.getLogger("test.ctx")
log.info("multi ctx")
data = json.loads(capsys.readouterr().err.strip())
assert data["node_id"] == "node-1"
assert data["ws_id"] == "ws-2"
assert data["request_id"] == "req-3"
assert "user_id" not in data
class TestGetLogger:
"""Test get_logger() returns a usable bound logger."""
def setup_method(self):
structlog.reset_defaults()
root = logging.getLogger()
root.handlers.clear()
root.setLevel(logging.WARNING)
def test_get_logger_returns_bound_logger(self):
configure_logging(level="INFO", json_output=False)
log = get_logger("test.bound")
assert log is not None
def test_get_logger_outputs(self, capsys):
configure_logging(level="INFO", json_output=True)
log = get_logger("test.bound")
log.info("bound logger test", extra_key="extra_val")
data = json.loads(capsys.readouterr().err.strip())
assert data["event"] == "bound logger test"
assert data["extra_key"] == "extra_val"
class TestServiceField:
"""Test that service name is injected when configured."""
def setup_method(self):
structlog.reset_defaults()
root = logging.getLogger()
root.handlers.clear()
root.setLevel(logging.WARNING)
def test_service_present(self, capsys):
configure_logging(level="INFO", json_output=True, service="myservice")
log = logging.getLogger("test.svc")
log.info("svc test")
data = json.loads(capsys.readouterr().err.strip())
assert data["service"] == "myservice"
def test_no_service_when_empty(self, capsys):
configure_logging(level="INFO", json_output=True)
log = logging.getLogger("test.svc")
log.info("no svc")
data = json.loads(capsys.readouterr().err.strip())
assert "service" not in data
+2 -2
View File
@@ -530,7 +530,7 @@ class TestWorkstreamModelParam:
captured_alias = None
def factory(ui: Any, model_alias: str | None = None) -> Any:
def factory(ui: Any, model_alias: str | None = None, ws_id: str | None = None) -> Any:
nonlocal captured_alias
captured_alias = model_alias
mock_session = MagicMock()
@@ -544,7 +544,7 @@ class TestWorkstreamModelParam:
def test_create_without_model(self) -> None:
captured_alias = None
def factory(ui: Any, model_alias: str | None = None) -> Any:
def factory(ui: Any, model_alias: str | None = None, ws_id: str | None = None) -> Any:
nonlocal captured_alias
captured_alias = model_alias
mock_session = MagicMock()
+112
View File
@@ -0,0 +1,112 @@
"""Tests for the atomic workstream resumption flow.
Covers CreateWorkstreamMessage resume_session field, SessionResumedEvent,
WorkstreamCreatedEvent resumed fields, and server endpoint handling.
"""
from __future__ import annotations
import json
from turnstone.mq.protocol import (
CreateWorkstreamMessage,
SessionResumedEvent,
WorkstreamCreatedEvent,
)
# ---------------------------------------------------------------------------
# Protocol tests
# ---------------------------------------------------------------------------
class TestCreateWorkstreamMessageResumeField:
def test_resume_session_defaults_empty(self) -> None:
msg = CreateWorkstreamMessage(name="test")
assert msg.resume_session == ""
def test_resume_session_set(self) -> None:
msg = CreateWorkstreamMessage(name="test", resume_session="sess-abc")
assert msg.resume_session == "sess-abc"
def test_resume_session_serializes(self) -> None:
msg = CreateWorkstreamMessage(resume_session="sess-xyz")
data = json.loads(msg.to_json())
assert data["resume_session"] == "sess-xyz"
def test_resume_session_deserializes(self) -> None:
msg = CreateWorkstreamMessage(resume_session="sess-123")
raw = msg.to_json()
from turnstone.mq.protocol import InboundMessage
restored = InboundMessage.from_json(raw)
assert getattr(restored, "resume_session", "") == "sess-123"
class TestWorkstreamCreatedEventResumeFields:
def test_default_not_resumed(self) -> None:
event = WorkstreamCreatedEvent(ws_id="ws-1", name="test")
assert event.resumed is False
assert event.session_id == ""
assert event.message_count == 0
def test_resumed_fields(self) -> None:
event = WorkstreamCreatedEvent(
ws_id="ws-1", name="test", resumed=True, session_id="s-1", message_count=42
)
assert event.resumed is True
assert event.session_id == "s-1"
assert event.message_count == 42
def test_serializes_resumed_fields(self) -> None:
event = WorkstreamCreatedEvent(
ws_id="ws-1", resumed=True, session_id="s-1", message_count=10
)
data = json.loads(event.to_json())
assert data["resumed"] is True
assert data["session_id"] == "s-1"
assert data["message_count"] == 10
def test_deserializes_resumed_fields(self) -> None:
event = WorkstreamCreatedEvent(
ws_id="ws-1", resumed=True, session_id="s-1", message_count=5
)
from turnstone.mq.protocol import OutboundEvent
restored = OutboundEvent.from_json(event.to_json())
assert isinstance(restored, WorkstreamCreatedEvent)
assert restored.resumed is True
assert restored.session_id == "s-1"
assert restored.message_count == 5
class TestSessionResumedEvent:
def test_defaults(self) -> None:
event = SessionResumedEvent(ws_id="ws-1")
assert event.type == "session_resumed"
assert event.session_id == ""
assert event.message_count == 0
assert event.name == ""
def test_with_values(self) -> None:
event = SessionResumedEvent(
ws_id="ws-1", session_id="s-abc", message_count=25, name="My Chat"
)
assert event.session_id == "s-abc"
assert event.message_count == 25
assert event.name == "My Chat"
def test_round_trip(self) -> None:
event = SessionResumedEvent(ws_id="ws-1", session_id="s-abc", message_count=10, name="Chat")
from turnstone.mq.protocol import OutboundEvent
restored = OutboundEvent.from_json(event.to_json())
assert isinstance(restored, SessionResumedEvent)
assert restored.session_id == "s-abc"
assert restored.message_count == 10
assert restored.name == "Chat"
def test_registered_in_outbound_registry(self) -> None:
from turnstone.mq.protocol import _OUTBOUND_REGISTRY
assert "session_resumed" in _OUTBOUND_REGISTRY
assert _OUTBOUND_REGISTRY["session_resumed"] is SessionResumedEvent
+15
View File
@@ -49,6 +49,21 @@ def test_sync_runner_iter():
runner.close()
def test_sync_runner_iter_empty():
"""_SyncRunner.run_iter handles empty async generator via sentinel."""
runner = _SyncRunner()
try:
async def _empty():
return
yield # pragma: no cover # makes this an async generator
items = list(runner.run_iter(_empty()))
assert items == []
finally:
runner.close()
# ---------------------------------------------------------------------------
# TurnstoneServer (sync)
# ---------------------------------------------------------------------------
+55
View File
@@ -249,6 +249,61 @@ class TestSearch:
assert len(results) == 1
# -- Workstream operations -----------------------------------------------------
class TestWorkstreams:
def test_register_and_list(self, backend):
backend.register_workstream("ws1", node_id="node-a", name="first")
backend.register_workstream("ws2", node_id="node-a", name="second")
rows = backend.list_workstreams()
assert len(rows) == 2
ws_ids = {r[0] for r in rows}
assert ws_ids == {"ws1", "ws2"}
def test_register_idempotent(self, backend):
backend.register_workstream("ws1", name="first")
backend.register_workstream("ws1", name="overwrite")
rows = backend.list_workstreams()
assert len(rows) == 1
assert rows[0][2] == "first" # name preserved from first insert
def test_update_state(self, backend):
backend.register_workstream("ws1")
backend.update_workstream_state("ws1", "running")
rows = backend.list_workstreams()
assert rows[0][3] == "running"
def test_update_name(self, backend):
backend.register_workstream("ws1", name="old")
backend.update_workstream_name("ws1", "new")
rows = backend.list_workstreams()
assert rows[0][2] == "new"
def test_delete(self, backend):
backend.register_workstream("ws1")
assert backend.delete_workstream("ws1") is True
assert backend.list_workstreams() == []
assert backend.delete_workstream("ws1") is False
def test_list_by_node(self, backend):
backend.register_workstream("ws1", node_id="node-a")
backend.register_workstream("ws2", node_id="node-b")
rows = backend.list_workstreams(node_id="node-a")
assert len(rows) == 1
assert rows[0][0] == "ws1"
def test_session_with_ws_id(self, backend):
backend.register_workstream("ws1", node_id="node-a")
backend.register_session("s1", node_id="node-a", ws_id="ws1")
backend.save_message("s1", "user", "hello")
rows = backend.list_sessions()
assert len(rows) == 1
# Columns: sid, alias, title, created, updated, count, node_id, ws_id
assert rows[0][6] == "node-a"
assert rows[0][7] == "ws1"
# -- Lifecycle -----------------------------------------------------------------
+169
View File
@@ -0,0 +1,169 @@
"""Tests for user identity storage operations (SQLite backend)."""
from __future__ import annotations
import pytest
from turnstone.core.storage._sqlite import SQLiteBackend
@pytest.fixture()
def db(tmp_path):
"""Create a fresh SQLite backend for each test."""
return SQLiteBackend(str(tmp_path / "test.db"))
class TestUserCRUD:
def test_create_and_get(self, db):
db.create_user("u1", "admin", "Admin User", "$2b$hash")
user = db.get_user("u1")
assert user is not None
assert user["user_id"] == "u1"
assert user["username"] == "admin"
assert user["display_name"] == "Admin User"
assert user["password_hash"] == "$2b$hash"
def test_get_nonexistent(self, db):
assert db.get_user("missing") is None
def test_get_by_username(self, db):
db.create_user("u1", "admin", "Admin", "$2b$hash")
user = db.get_user_by_username("admin")
assert user is not None
assert user["user_id"] == "u1"
def test_get_by_username_nonexistent(self, db):
assert db.get_user_by_username("nope") is None
def test_create_duplicate_noop(self, db):
db.create_user("u1", "admin", "First", "$2b$hash1")
db.create_user("u1", "admin2", "Second", "$2b$hash2")
user = db.get_user("u1")
assert user is not None
assert user["display_name"] == "First"
def test_list_users(self, db):
db.create_user("u1", "admin", "Admin", "$2b$h1")
db.create_user("u2", "reader", "Reader", "$2b$h2")
users = db.list_users()
assert len(users) == 2
assert "password_hash" not in users[0]
def test_delete_user(self, db):
db.create_user("u1", "admin", "Admin", "$2b$hash")
assert db.delete_user("u1")
assert db.get_user("u1") is None
def test_delete_nonexistent(self, db):
assert not db.delete_user("missing")
def test_delete_cascades_tokens(self, db):
db.create_user("u1", "admin", "Admin", "$2b$hash")
db.create_api_token("t1", "hash1", "ts_abcde", "u1", "tok1", "read,write")
db.create_api_token("t2", "hash2", "ts_fghij", "u1", "tok2", "read")
assert len(db.list_api_tokens("u1")) == 2
db.delete_user("u1")
assert len(db.list_api_tokens("u1")) == 0
class TestApiTokenCRUD:
def test_create_and_lookup_by_hash(self, db):
db.create_user("u1", "admin", "Admin", "$2b$hash")
db.create_api_token("t1", "tokenhash123", "ts_abcde", "u1", "My Token", "read,write")
tok = db.get_api_token_by_hash("tokenhash123")
assert tok is not None
assert tok["token_id"] == "t1"
assert tok["user_id"] == "u1"
assert tok["scopes"] == "read,write"
def test_lookup_missing_hash(self, db):
assert db.get_api_token_by_hash("nonexistent") is None
def test_list_tokens_excludes_hash(self, db):
db.create_user("u1", "admin", "Admin", "$2b$hash")
db.create_api_token("t1", "secret_hash", "ts_abcde", "u1", "tok1", "read")
tokens = db.list_api_tokens("u1")
assert len(tokens) == 1
assert "token_hash" not in tokens[0]
assert tokens[0]["token_prefix"] == "ts_abcde"
def test_list_tokens_by_user(self, db):
db.create_user("u1", "admin", "Admin", "$2b$hash")
db.create_user("u2", "reader", "Reader", "$2b$hash")
db.create_api_token("t1", "h1", "ts_a", "u1", "tok1", "read")
db.create_api_token("t2", "h2", "ts_b", "u2", "tok2", "read")
assert len(db.list_api_tokens("u1")) == 1
assert len(db.list_api_tokens("u2")) == 1
def test_delete_token(self, db):
db.create_user("u1", "admin", "Admin", "$2b$hash")
db.create_api_token("t1", "h1", "ts_a", "u1", "tok1", "read")
assert db.delete_api_token("t1")
assert db.get_api_token_by_hash("h1") is None
def test_delete_nonexistent_token(self, db):
assert not db.delete_api_token("missing")
def test_token_with_expiry(self, db):
db.create_user("u1", "admin", "Admin", "$2b$hash")
db.create_api_token(
"t1",
"h1",
"ts_a",
"u1",
"tok1",
"read",
expires="2030-01-01T00:00:00",
)
tok = db.get_api_token_by_hash("h1")
assert tok is not None
assert tok["expires"] == "2030-01-01T00:00:00"
def test_token_without_expiry(self, db):
db.create_user("u1", "admin", "Admin", "$2b$hash")
db.create_api_token("t1", "h1", "ts_a", "u1", "tok1", "read")
tok = db.get_api_token_by_hash("h1")
assert tok is not None
assert "expires" not in tok
class TestSessionWorkstreamUserId:
def test_register_session_with_user_id(self, db):
db.register_session("s1", user_id="u1")
# Verify via raw SQL that user_id is stored
import sqlalchemy as sa
from turnstone.core.storage._schema import sessions
with db._engine.connect() as conn:
row = conn.execute(
sa.select(sessions.c.user_id).where(sessions.c.session_id == "s1")
).fetchone()
assert row is not None
assert row[0] == "u1"
def test_register_workstream_with_user_id(self, db):
db.register_workstream("ws1", user_id="u1")
import sqlalchemy as sa
from turnstone.core.storage._schema import workstreams
with db._engine.connect() as conn:
row = conn.execute(
sa.select(workstreams.c.user_id).where(workstreams.c.ws_id == "ws1")
).fetchone()
assert row is not None
assert row[0] == "u1"
def test_register_session_without_user_id(self, db):
db.register_session("s1")
import sqlalchemy as sa
from turnstone.core.storage._schema import sessions
with db._engine.connect() as conn:
row = conn.execute(
sa.select(sessions.c.user_id).where(sessions.c.session_id == "s1")
).fetchone()
assert row is not None
assert row[0] is None
+5 -3
View File
@@ -20,7 +20,7 @@ class FakeSession:
self.messages = []
def _fake_factory(ui, model_alias=None):
def _fake_factory(ui, model_alias=None, ws_id=None):
return FakeSession()
@@ -440,8 +440,10 @@ class TestManagerThreadSafety:
for t in threads:
t.join()
assert mgr.count == 5
assert len(errors) == 5
# All threads resolved (created or rejected)
assert len(created) + len(errors) == 10
# Never exceeded capacity
assert mgr.count <= 5
def test_concurrent_switch(self):
"""Concurrent switches should not corrupt state."""
+1 -1
View File
@@ -1,3 +1,3 @@
"""turnstone - Multi-node AI orchestration platform with tool use, agent routing, and cluster simulation."""
__version__ = "0.3.5"
__version__ = "0.4.0"
+186
View File
@@ -0,0 +1,186 @@
"""CLI admin commands for user and token management.
Entry point: turnstone-admin
"""
from __future__ import annotations
import argparse
import os
import sys
import uuid
from typing import Any
def _get_storage() -> Any:
"""Initialize and return the storage backend."""
from turnstone.core.storage import init_storage
db_backend = os.environ.get("TURNSTONE_DB_BACKEND", "sqlite")
db_url = os.environ.get("TURNSTONE_DB_URL", "")
db_path = os.environ.get("TURNSTONE_DB_PATH", "")
return init_storage(db_backend, path=db_path, url=db_url)
def _cmd_create_user(args: argparse.Namespace) -> None:
import getpass
from turnstone.core.auth import (
generate_token,
hash_password,
hash_token,
is_valid_username,
token_prefix,
)
if not is_valid_username(args.username):
print("Error: invalid username (1-64 chars: letters, digits, . _ -)", file=sys.stderr)
sys.exit(1)
storage = _get_storage()
user_id = uuid.uuid4().hex
# Prompt for password
password = args.password
if not password:
password = getpass.getpass("Password: ")
confirm = getpass.getpass("Confirm password: ")
if password != confirm:
print("Error: passwords do not match", file=sys.stderr)
sys.exit(1)
pw_hash = hash_password(password)
storage.create_user(user_id, args.username, args.name, pw_hash)
print(f"Created user: {user_id}")
print(f" Username: {args.username}")
print(f" Name: {args.name}")
if args.token:
scopes = args.scopes or "read,write,approve"
raw = generate_token()
tid = uuid.uuid4().hex
storage.create_api_token(
token_id=tid,
token_hash=hash_token(raw),
token_prefix=token_prefix(raw),
user_id=user_id,
name="initial",
scopes=scopes,
)
print(f"\n Token: {raw}")
print(f" Token ID: {tid}")
print(f" Scopes: {scopes}")
print(" (Save this token now — it cannot be retrieved again)")
def _cmd_create_token(args: argparse.Namespace) -> None:
from turnstone.core.auth import generate_token, hash_token, token_prefix
storage = _get_storage()
if storage.get_user(args.user) is None:
print(f"Error: user {args.user} not found", file=sys.stderr)
sys.exit(1)
expires = None
if args.expires_days:
from datetime import UTC, datetime, timedelta
expires = (datetime.now(UTC) + timedelta(days=args.expires_days)).strftime(
"%Y-%m-%dT%H:%M:%S"
)
raw = generate_token()
tid = uuid.uuid4().hex
storage.create_api_token(
token_id=tid,
token_hash=hash_token(raw),
token_prefix=token_prefix(raw),
user_id=args.user,
name=args.name or "",
scopes=args.scopes,
expires=expires,
)
print(f"Token: {raw}")
print(f" ID: {tid}")
print(f" Scopes: {args.scopes}")
if expires:
print(f" Expires: {expires}")
print(" (Save this token now — it cannot be retrieved again)")
def _cmd_list_users(args: argparse.Namespace) -> None:
storage = _get_storage()
users = storage.list_users()
if not users:
print("No users found.")
return
for u in users:
print(f" {u['user_id'][:12]}.. {u['display_name']} ({u['created']})")
def _cmd_list_tokens(args: argparse.Namespace) -> None:
storage = _get_storage()
tokens = storage.list_api_tokens(args.user)
if not tokens:
print(f"No tokens found for user {args.user}.")
return
for t in tokens:
exp = f" expires={t['expires']}" if t.get("expires") else ""
print(
f" {t['token_id'][:12]}.. {t['token_prefix']}.. scopes={t['scopes']}"
f" name={t['name']}{exp}"
)
def _cmd_revoke_token(args: argparse.Namespace) -> None:
storage = _get_storage()
if storage.delete_api_token(args.token_id):
print(f"Revoked token {args.token_id}")
else:
print("Token not found", file=sys.stderr)
sys.exit(1)
def main() -> None:
"""Entry point for turnstone-admin CLI."""
parser = argparse.ArgumentParser(
prog="turnstone-admin",
description="Turnstone user and token administration",
)
sub = parser.add_subparsers(dest="command")
p_cu = sub.add_parser("create-user", help="Create a new user")
p_cu.add_argument("--username", required=True, help="Login username")
p_cu.add_argument("--name", required=True, help="Display name")
p_cu.add_argument("--password", default="", help="Password (prompted if not provided)")
p_cu.add_argument("--token", action="store_true", help="Also create an initial API token")
p_cu.add_argument("--scopes", default="read,write,approve", help="Scopes for initial token")
p_ct = sub.add_parser("create-token", help="Create an API token for a user")
p_ct.add_argument("--user", required=True, help="User ID")
p_ct.add_argument("--name", default="", help="Human label for the token")
p_ct.add_argument("--scopes", default="read,write", help="Comma-separated scopes")
p_ct.add_argument("--expires-days", type=int, default=None, help="Days until expiry")
sub.add_parser("list-users", help="List all users")
p_lt = sub.add_parser("list-tokens", help="List tokens for a user")
p_lt.add_argument("--user", required=True, help="User ID")
p_rt = sub.add_parser("revoke-token", help="Revoke an API token")
p_rt.add_argument("--token-id", required=True, help="Token ID to revoke")
args = parser.parse_args()
if not args.command:
parser.print_help()
sys.exit(1)
dispatch = {
"create-user": _cmd_create_user,
"create-token": _cmd_create_token,
"list-users": _cmd_list_users,
"list-tokens": _cmd_list_tokens,
"revoke-token": _cmd_revoke_token,
}
dispatch[args.command](args)
+81
View File
@@ -20,8 +20,17 @@ from turnstone.api.openapi import EndpointSpec, QueryParam, build_openapi
from turnstone.api.schemas import (
AuthLoginRequest,
AuthLoginResponse,
AuthSetupRequest,
AuthSetupResponse,
AuthStatusResponse,
CreateTokenRequest,
CreateTokenResponse,
CreateUserRequest,
ErrorResponse,
ListTokensResponse,
ListUsersResponse,
StatusResponse,
UserInfo,
)
CONSOLE_ENDPOINTS: list[EndpointSpec] = [
@@ -103,6 +112,22 @@ CONSOLE_ENDPOINTS: list[EndpointSpec] = [
error_codes=[401],
tags=["Auth"],
),
EndpointSpec(
"/v1/api/auth/setup",
"POST",
"Create first admin user",
request_model=AuthSetupRequest,
response_model=AuthSetupResponse,
error_codes=[400, 409, 503],
tags=["Auth"],
),
EndpointSpec(
"/v1/api/auth/status",
"GET",
"Return auth state",
response_model=AuthStatusResponse,
tags=["Auth"],
),
EndpointSpec(
"/v1/api/auth/logout",
"POST",
@@ -110,6 +135,53 @@ CONSOLE_ENDPOINTS: list[EndpointSpec] = [
response_model=StatusResponse,
tags=["Auth"],
),
# --- Admin ---
EndpointSpec(
"/v1/api/admin/users",
"GET",
"List all users",
response_model=ListUsersResponse,
tags=["Admin"],
),
EndpointSpec(
"/v1/api/admin/users",
"POST",
"Create a user",
request_model=CreateUserRequest,
response_model=UserInfo,
tags=["Admin"],
),
EndpointSpec(
"/v1/api/admin/users/{user_id}",
"DELETE",
"Delete a user and their tokens",
response_model=StatusResponse,
error_codes=[404],
tags=["Admin"],
),
EndpointSpec(
"/v1/api/admin/users/{user_id}/tokens",
"GET",
"List tokens for a user",
response_model=ListTokensResponse,
tags=["Admin"],
),
EndpointSpec(
"/v1/api/admin/users/{user_id}/tokens",
"POST",
"Create an API token (raw token shown once)",
request_model=CreateTokenRequest,
response_model=CreateTokenResponse,
tags=["Admin"],
),
EndpointSpec(
"/v1/api/admin/tokens/{token_id}",
"DELETE",
"Revoke an API token",
response_model=StatusResponse,
error_codes=[404],
tags=["Admin"],
),
# --- Observability ---
EndpointSpec(
"/health",
@@ -125,6 +197,15 @@ _ALL_MODELS: list[type[BaseModel]] = [
StatusResponse,
AuthLoginRequest,
AuthLoginResponse,
AuthSetupRequest,
AuthSetupResponse,
AuthStatusResponse,
CreateUserRequest,
UserInfo,
ListUsersResponse,
CreateTokenRequest,
CreateTokenResponse,
ListTokensResponse,
ClusterOverviewResponse,
ClusterNodesResponse,
ClusterWorkstreamsResponse,
+113 -3
View File
@@ -35,13 +35,123 @@ class StatusResponse(BaseModel):
class AuthLoginRequest(BaseModel):
"""POST /v1/api/auth/login request body."""
"""POST /v1/api/auth/login request body.
token: str = Field(description="Bearer token to authenticate")
Either username+password or token must be provided.
"""
username: str = Field(default="", description="Login username")
password: str = Field(default="", description="Login password")
token: str = Field(default="", description="Legacy: bearer token to authenticate")
class AuthLoginResponse(BaseModel):
"""POST /v1/api/auth/login success response."""
status: str = Field(default="ok")
role: str = Field(description="Assigned role", examples=["full", "read"])
user_id: str = Field(default="", description="Authenticated user ID")
role: str = Field(description="Legacy role", examples=["full", "read"])
scopes: str = Field(
default="", description="Comma-separated scopes", examples=["read,write,approve"]
)
jwt: str = Field(default="", description="JWT session token (if JWT auth is configured)")
# ---------------------------------------------------------------------------
# Admin — User identity + API tokens
# ---------------------------------------------------------------------------
class CreateUserRequest(BaseModel):
"""POST /v1/api/admin/users request body."""
username: str = Field(description="Login username (unique)")
display_name: str = Field(description="Human-readable display name")
password: str = Field(description="Initial password")
class UserInfo(BaseModel):
"""User record (no password_hash)."""
user_id: str
username: str
display_name: str
created: str
class ListUsersResponse(BaseModel):
"""GET /v1/api/admin/users response."""
users: list[UserInfo]
class CreateTokenRequest(BaseModel):
"""POST /v1/api/admin/users/{user_id}/tokens request body."""
name: str = Field(default="", description="Human label for the token")
scopes: str = Field(
default="read,write,approve",
description="Comma-separated scopes: read, write, approve",
)
expires_days: int | None = Field(
default=None,
description="Days until expiry (null = no expiry)",
)
class TokenInfo(BaseModel):
"""Token metadata (never includes the hash or raw token)."""
token_id: str
token_prefix: str
name: str
scopes: str
created: str
expires: str | None = None
class CreateTokenResponse(BaseModel):
"""POST /v1/api/admin/users/{user_id}/tokens response (raw token shown once)."""
token: str = Field(description="Raw API token — save this, it cannot be retrieved again")
token_id: str
token_prefix: str
scopes: str
class ListTokensResponse(BaseModel):
"""GET /v1/api/admin/users/{user_id}/tokens response."""
tokens: list[TokenInfo]
# ---------------------------------------------------------------------------
# Auth — Setup + status
# ---------------------------------------------------------------------------
class AuthSetupRequest(BaseModel):
"""POST /v1/api/auth/setup request body."""
username: str = Field(description="Login username (1-64 ASCII characters)")
display_name: str = Field(description="Display name")
password: str = Field(description="Password (minimum 8 characters)")
class AuthSetupResponse(BaseModel):
"""POST /v1/api/auth/setup success response."""
status: str = Field(default="ok")
user_id: str
username: str
role: str = Field(default="full")
scopes: str = Field(default="approve,read,write")
jwt: str = Field(default="", description="JWT session token")
class AuthStatusResponse(BaseModel):
"""GET /v1/api/auth/status response."""
auth_enabled: bool
has_users: bool
setup_required: bool
+7
View File
@@ -39,11 +39,18 @@ class CreateWorkstreamRequest(BaseModel):
name: str = Field(default="", description="Workstream display name (auto-generated if empty)")
model: str = Field(default="", description="Model alias from registry")
auto_approve: bool = Field(default=False, description="Auto-approve all tool calls")
resume_session: str = Field(
default="",
description="Session ID to resume atomically during creation (empty = fresh start)",
)
class CreateWorkstreamResponse(BaseModel):
ws_id: str = Field(description="Unique ID of the new workstream")
name: str = Field(description="Assigned workstream name")
resumed: bool = Field(default=False, description="Whether a previous session was resumed")
session_id: str = Field(default="", description="Resolved session ID (set when resumed)")
message_count: int = Field(default=0, description="Number of messages in the resumed session")
class CloseWorkstreamRequest(BaseModel):
+22
View File
@@ -11,6 +11,9 @@ if TYPE_CHECKING:
from turnstone.api.schemas import (
AuthLoginRequest,
AuthLoginResponse,
AuthSetupRequest,
AuthSetupResponse,
AuthStatusResponse,
ErrorResponse,
StatusResponse,
)
@@ -137,6 +140,22 @@ SERVER_ENDPOINTS: list[EndpointSpec] = [
error_codes=[401],
tags=["Auth"],
),
EndpointSpec(
"/v1/api/auth/setup",
"POST",
"Create first admin user",
request_model=AuthSetupRequest,
response_model=AuthSetupResponse,
error_codes=[400, 409, 503],
tags=["Auth"],
),
EndpointSpec(
"/v1/api/auth/status",
"GET",
"Return auth state",
response_model=AuthStatusResponse,
tags=["Auth"],
),
EndpointSpec(
"/v1/api/auth/logout",
"POST",
@@ -159,6 +178,9 @@ _ALL_MODELS: list[type[BaseModel]] = [
StatusResponse,
AuthLoginRequest,
AuthLoginResponse,
AuthSetupRequest,
AuthSetupResponse,
AuthStatusResponse,
SendRequest,
SendResponse,
ApproveRequest,
+15
View File
@@ -0,0 +1,15 @@
"""Shared channel infrastructure for turnstone communication integrations.
Provides the :class:`ChannelAdapter` protocol, the :class:`ChannelEvent`
normalized event type, the :class:`ChannelRouter` for workstream mapping,
and shared formatting / configuration utilities.
"""
from turnstone.channels._protocol import ChannelAdapter, ChannelEvent
from turnstone.channels._routing import ChannelRouter
__all__ = [
"ChannelAdapter",
"ChannelEvent",
"ChannelRouter",
]
+23
View File
@@ -0,0 +1,23 @@
"""Base configuration shared by all channel adapters."""
from __future__ import annotations
from dataclasses import dataclass, field
@dataclass
class ChannelConfig:
"""Base configuration shared by all channel adapters.
Individual adapters extend this with platform-specific fields (tokens,
guild IDs, etc.).
"""
redis_host: str = "localhost"
redis_port: int = 6379
redis_db: int = 0
redis_password: str | None = None
prefix: str = "turnstone"
model: str = ""
auto_approve: bool = False
auto_approve_tools: list[str] = field(default_factory=list)
+116
View File
@@ -0,0 +1,116 @@
"""Message formatting utilities for channel adapters.
Handles chunking long messages for platforms with character limits, formatting
tool-approval requests, and plan-review prompts.
"""
from __future__ import annotations
from typing import Any
def chunk_message(text: str, max_length: int = 2000) -> list[str]:
"""Split *text* into chunks that fit within *max_length*.
Respects code-block boundaries: if a fenced code block (````` ```)
spans a chunk boundary the current chunk is closed with ````` ``` ``
and the next chunk reopens it. Prefers splitting at newline
boundaries, then word boundaries, then hard-splits.
"""
if len(text) <= max_length:
return [text]
chunks: list[str] = []
remaining = text
in_code_block = False
while remaining:
if len(remaining) <= max_length:
chunks.append(remaining)
break
# Reserve space for a closing ``` if we're inside a code block.
limit = max_length - 4 if in_code_block else max_length
limit = max(limit, 1)
candidate = remaining[:limit]
# Prefer a newline boundary.
split_idx = candidate.rfind("\n")
if split_idx <= 0:
# Fall back to a word boundary.
split_idx = candidate.rfind(" ")
if split_idx <= 0:
# Hard split.
split_idx = limit
chunk = remaining[:split_idx]
remaining = remaining[split_idx:].lstrip("\n")
# Track code-block fences in this chunk.
fence_count = chunk.count("```")
block_open = in_code_block
if fence_count % 2 != 0:
in_code_block = not in_code_block
# If we end inside a code block, close it in this chunk and
# reopen in the next.
if in_code_block:
chunk += "\n```"
remaining = "```\n" + remaining
in_code_block = False
elif block_open and fence_count % 2 != 0:
# We were inside a code block and the chunk closed it
# properly -- nothing extra needed.
pass
chunks.append(chunk)
return chunks
def format_approval_request(items: list[dict[str, Any]]) -> str:
"""Format tool-approval *items* into a human-readable message.
Items use the server's SSE format: ``func_name``, ``preview``,
``approval_label``, ``header``. Falls back to the nested
``function.name`` format for compatibility.
"""
lines: list[str] = ["**Tool approval required:**"]
for item in items:
# Server SSE format: top-level func_name / preview
name = item.get("func_name") or item.get("approval_label", "")
if not name:
# Fallback: nested function.name (SDK / older format)
func = item.get("function", {})
name = func.get("name", "unknown")
preview = item.get("preview", "")
if not preview:
args = item.get("function", {}).get("arguments", "")
if isinstance(args, dict):
import json
args = json.dumps(args, ensure_ascii=False)
preview = str(args)
preview = truncate(preview)
header = item.get("header", "")
if header:
lines.append(f"\u2022 `{name}`: {header}")
elif preview:
lines.append(f"\u2022 `{name}`: {preview}")
else:
lines.append(f"\u2022 `{name}`")
return "\n".join(lines)
def format_plan_review(content: str) -> str:
"""Format a plan-review prompt with a header."""
return f"**Plan review requested:**\n\n{content}"
def truncate(text: str, max_length: int = 200) -> str:
"""Truncate *text* to *max_length*, appending an ellipsis if trimmed."""
if len(text) <= max_length:
return text
return text[: max_length - 1] + "\u2026"
+70
View File
@@ -0,0 +1,70 @@
"""Channel adapter protocol and normalized event type.
Defines the :class:`ChannelEvent` data class for inbound events and the
:class:`ChannelAdapter` structural protocol that all bidirectional channel
adapters (Discord, Slack, etc.) must satisfy.
"""
from __future__ import annotations
from dataclasses import dataclass, field
from typing import Any, Protocol, runtime_checkable
@dataclass
class ChannelEvent:
"""Normalized inbound event from any channel."""
channel_type: str # "discord", "slack"
channel_id: str # thread/channel ID
channel_user_id: str # platform user ID
message: str
parent_channel_id: str = "" # main channel (for thread creation)
metadata: dict[str, Any] = field(default_factory=dict)
@runtime_checkable
class ChannelAdapter(Protocol):
"""Protocol for bidirectional channel adapters."""
channel_type: str
async def start(self) -> None:
"""Connect to the platform and begin listening for events."""
...
async def stop(self) -> None:
"""Disconnect and release resources."""
...
async def send(self, channel_id: str, content: str) -> str:
"""Send a message to a channel. Returns the platform message ID."""
...
async def edit_message(self, channel_id: str, message_id: str, content: str) -> None:
"""Edit an existing message in a channel."""
...
async def send_approval_request(
self,
channel_id: str,
ws_id: str,
correlation_id: str,
items: list[dict[str, Any]],
) -> None:
"""Send an interactive tool-approval prompt to a channel."""
...
async def send_plan_review(
self,
channel_id: str,
ws_id: str,
correlation_id: str,
content: str,
) -> None:
"""Send a plan-review prompt to a channel."""
...
async def create_thread(self, parent_channel_id: str, name: str, message_id: str = "") -> str:
"""Create a thread under a parent channel. Returns the new thread ID."""
...
+304
View File
@@ -0,0 +1,304 @@
"""Channel router -- maps external channels/threads to turnstone workstreams.
:class:`ChannelRouter` uses the async Redis broker for MQ communication and
the storage backend for persistent channel-to-workstream mappings.
"""
from __future__ import annotations
import asyncio
from typing import TYPE_CHECKING
from turnstone.core.log import get_logger
from turnstone.mq.protocol import (
ApproveMessage,
CreateWorkstreamMessage,
OutboundEvent,
PlanFeedbackMessage,
SendMessage,
WorkstreamClosedEvent,
WorkstreamCreatedEvent,
)
if TYPE_CHECKING:
from turnstone.core.storage import StorageBackend
from turnstone.mq.async_broker import AsyncRedisBroker
log = get_logger(__name__)
_WS_CREATE_TIMEOUT = 30.0 # seconds
class ChannelRouter:
"""Manage channel-to-workstream routing and MQ message dispatch.
Parameters
----------
broker:
An :class:`AsyncRedisBroker` used for pub/sub and queue operations.
storage:
A :class:`StorageBackend` instance for persistent route lookups.
All storage calls are synchronous and will be wrapped in
:func:`asyncio.to_thread`.
"""
def __init__(
self,
broker: AsyncRedisBroker,
storage: StorageBackend,
*,
auto_approve: bool = False,
auto_approve_tools: list[str] | None = None,
) -> None:
self._broker = broker
self._storage = storage
self._auto_approve = auto_approve
self._auto_approve_tools: list[str] = auto_approve_tools or []
self._pending: dict[str, asyncio.Event] = {}
self._pending_results: dict[str, str] = {}
self._global_task: asyncio.Task[None] | None = None
self._create_locks: dict[str, asyncio.Lock] = {}
# -- lifecycle -----------------------------------------------------------
async def start(self) -> None:
"""Subscribe to global events for workstream lifecycle."""
channel = f"{self._broker._prefix}:events:global"
await self._broker.subscribe(channel, self._on_global_event)
log.info("channel_router.started", channel=channel)
async def stop(self) -> None:
"""Unsubscribe and clean up pending state."""
channel = f"{self._broker._prefix}:events:global"
await self._broker.unsubscribe(channel)
# Wake any waiters so they don't hang forever.
for evt in self._pending.values():
evt.set()
self._pending.clear()
self._pending_results.clear()
log.info("channel_router.stopped")
# -- event handler -------------------------------------------------------
async def _on_global_event(self, raw: str) -> None:
"""Handle events on the global pub/sub channel.
Exceptions are caught so the broker listener task stays alive.
"""
try:
event = OutboundEvent.from_json(raw)
if isinstance(event, WorkstreamCreatedEvent):
cid = event.correlation_id
if cid in self._pending:
self._pending_results[cid] = event.ws_id
self._pending[cid].set()
log.debug(
"channel_router.ws_created",
ws_id=event.ws_id,
correlation_id=cid,
)
elif isinstance(event, WorkstreamClosedEvent):
ws_id = event.ws_id
route = await asyncio.to_thread(self._storage.get_channel_route_by_ws, ws_id)
if route:
# Don't delete the route — the workstream may have been evicted
# and the thread can reactivate it. Route cleanup only happens
# via explicit /close or delete_route().
log.info(
"channel_router.ws_closed_route_kept",
ws_id=ws_id,
channel_type=route["channel_type"],
channel_id=route["channel_id"],
)
except Exception:
log.exception("channel_router.global_event_error")
# -- workstream management -----------------------------------------------
async def get_or_create_workstream(
self,
channel_type: str,
channel_id: str,
name: str = "",
model: str = "",
initial_message: str = "",
) -> tuple[str, bool]:
"""Look up or create a workstream for a channel.
Returns ``(ws_id, is_new)`` where *is_new* is ``True`` when a new
workstream was created.
A per-channel lock prevents duplicate workstreams when concurrent
messages arrive for the same channel before the first creation
completes.
"""
key = f"{channel_type}:{channel_id}"
lock = self._create_locks.setdefault(key, asyncio.Lock())
old_ws_id: str | None = None
async with lock:
# 1. Check for existing route.
old_ws_id = ""
route = await asyncio.to_thread(
self._storage.get_channel_route, channel_type, channel_id
)
if route:
# Verify the workstream is still alive (owned by a bridge node).
owner = await self._broker.get_ws_owner(route["ws_id"])
if owner:
return route["ws_id"], False
# Workstream was evicted/closed — capture old ws_id for session
# resume, then remove the stale route.
old_ws_id = route["ws_id"]
await asyncio.to_thread(
self._storage.delete_channel_route, channel_type, channel_id
)
log.info(
"channel_router.stale_route_cleared",
ws_id=old_ws_id,
channel_type=channel_type,
channel_id=channel_id,
)
# 2. Look up old session for atomic resume (if stale route).
resume_session = ""
if old_ws_id:
old_sid: str | None = await asyncio.to_thread(
self._storage.get_session_id_by_ws, old_ws_id
)
resume_session = old_sid or ""
# 3. Create via MQ with atomic resume.
msg = CreateWorkstreamMessage(
name=name,
model=model,
initial_message="" if resume_session else initial_message,
resume_session=resume_session,
auto_approve=self._auto_approve,
auto_approve_tools=list(self._auto_approve_tools),
)
cid = msg.correlation_id
waiter = asyncio.Event()
self._pending[cid] = waiter
await self._broker.push_inbound(msg.to_json())
log.info(
"channel_router.creating_workstream",
correlation_id=cid,
channel_type=channel_type,
channel_id=channel_id,
resume_session=resume_session or None,
)
try:
await asyncio.wait_for(waiter.wait(), timeout=_WS_CREATE_TIMEOUT)
except TimeoutError:
self._pending.pop(cid, None)
self._pending_results.pop(cid, None)
raise
ws_id = self._pending_results.pop(cid, "")
self._pending.pop(cid, None)
if not ws_id:
msg_err = "workstream creation returned empty ws_id"
raise RuntimeError(msg_err)
# 4. Persist the route.
await asyncio.to_thread(
self._storage.create_channel_route, channel_type, channel_id, ws_id
)
log.info(
"channel_router.route_created",
ws_id=ws_id,
channel_type=channel_type,
channel_id=channel_id,
)
return ws_id, True
# -- user resolution -----------------------------------------------------
async def resolve_user(self, channel_type: str, channel_user_id: str) -> str | None:
"""Resolve an external platform user to a turnstone ``user_id``.
Returns ``None`` if no mapping exists.
"""
result = await asyncio.to_thread(
self._storage.get_channel_user, channel_type, channel_user_id
)
if result is None:
return None
return result.get("user_id")
# -- message dispatch ----------------------------------------------------
async def send_message(self, ws_id: str, message: str) -> str:
"""Push a :class:`SendMessage` to the broker.
Returns the ``correlation_id`` of the submitted message.
"""
msg = SendMessage(
ws_id=ws_id,
message=message,
auto_approve=self._auto_approve,
auto_approve_tools=list(self._auto_approve_tools),
)
await self._broker.push_inbound(msg.to_json())
log.debug("channel_router.send_message", ws_id=ws_id, correlation_id=msg.correlation_id)
return msg.correlation_id
async def send_approval(
self,
ws_id: str,
correlation_id: str,
approved: bool,
feedback: str = "",
always: bool = False,
) -> None:
"""Push an :class:`ApproveMessage` to the broker response queue."""
msg = ApproveMessage(
ws_id=ws_id,
request_id=correlation_id,
approved=approved,
feedback=feedback or None,
always=always,
)
await self._broker.push_response(correlation_id, msg.to_json())
log.debug(
"channel_router.send_approval",
ws_id=ws_id,
correlation_id=correlation_id,
approved=approved,
)
async def send_plan_feedback(self, ws_id: str, correlation_id: str, feedback: str) -> None:
"""Push a :class:`PlanFeedbackMessage` to the broker response queue."""
msg = PlanFeedbackMessage(
ws_id=ws_id,
request_id=correlation_id,
feedback=feedback,
)
await self._broker.push_response(correlation_id, msg.to_json())
log.debug(
"channel_router.send_plan_feedback",
ws_id=ws_id,
correlation_id=correlation_id,
)
# -- route management ----------------------------------------------------
async def delete_route(self, channel_type: str, channel_id: str) -> None:
"""Remove a channel-to-workstream mapping."""
deleted = await asyncio.to_thread(
self._storage.delete_channel_route, channel_type, channel_id
)
log.info(
"channel_router.delete_route",
channel_type=channel_type,
channel_id=channel_id,
deleted=deleted,
)
+137
View File
@@ -0,0 +1,137 @@
"""Unified channel gateway entry point.
Launches one or more channel adapters (Discord, Slack, etc.) connected to
the turnstone cluster via Redis MQ. Currently supports Discord; future
adapters will be added as additional ``--*-token`` flags.
Run as: ``turnstone-channel --discord-token $TURNSTONE_DISCORD_TOKEN``
"""
from __future__ import annotations
import os
import sys
def main() -> None:
"""Parse arguments, initialize storage and broker, and run adapters."""
import argparse
parser = argparse.ArgumentParser(
description="turnstone channel gateway — bridges messaging platforms to the turnstone cluster"
)
# -- Redis ---------------------------------------------------------------
from turnstone.mq.broker import add_redis_args
add_redis_args(parser)
# -- Discord -------------------------------------------------------------
parser.add_argument(
"--discord-token",
default=os.environ.get("TURNSTONE_DISCORD_TOKEN", ""),
help="Discord bot token (default: $TURNSTONE_DISCORD_TOKEN)",
)
parser.add_argument(
"--discord-guild",
type=int,
default=0,
help="Restrict to a single Discord guild (0 = all, default: %(default)s)",
)
parser.add_argument(
"--discord-channels",
default="",
help="Comma-separated list of allowed Discord channel IDs (default: all)",
)
# -- Workstream defaults -------------------------------------------------
parser.add_argument(
"--model",
default="",
help="Default model for new workstreams (default: server default)",
)
parser.add_argument(
"--auto-approve",
action="store_true",
help="Auto-approve all tool calls",
)
# -- Logging -------------------------------------------------------------
from turnstone.core.log import add_log_args
add_log_args(parser)
args = parser.parse_args()
# -- Logging setup -------------------------------------------------------
from turnstone.core.log import configure_logging_from_args
configure_logging_from_args(args, "channel")
from turnstone.core.log import get_logger
log = get_logger(__name__)
# -- Storage -------------------------------------------------------------
from turnstone.core.storage._registry import init_storage
db_backend = os.environ.get("TURNSTONE_DB_BACKEND", "sqlite")
db_url = os.environ.get("TURNSTONE_DB_URL", "")
db_path = os.environ.get("TURNSTONE_DB_PATH", "")
init_storage(
backend=db_backend,
url=db_url,
path=db_path,
)
# -- Broker --------------------------------------------------------------
from turnstone.mq.broker import async_broker_from_args
broker = async_broker_from_args(args)
# -- Adapter selection ---------------------------------------------------
adapters_configured = False
if args.discord_token:
adapters_configured = True
if not adapters_configured:
print(
"Error: no channel adapters configured. "
"Set --discord-token or $TURNSTONE_DISCORD_TOKEN.",
file=sys.stderr,
)
sys.exit(1)
# -- Run -----------------------------------------------------------------
if args.discord_token:
from turnstone.channels.discord.bot import TurnstoneBot
from turnstone.channels.discord.config import DiscordConfig
from turnstone.core.storage._registry import get_storage
allowed_channels: list[int] = []
if args.discord_channels:
allowed_channels = [
int(c.strip()) for c in args.discord_channels.split(",") if c.strip()
]
config = DiscordConfig(
redis_host=args.redis_host,
redis_port=args.redis_port,
redis_db=args.redis_db,
redis_password=args.redis_password,
model=args.model,
auto_approve=args.auto_approve,
bot_token=args.discord_token,
guild_id=args.discord_guild,
allowed_channels=allowed_channels,
)
bot = TurnstoneBot(config, broker, get_storage())
log.info("channel.starting", adapter="discord", guild_id=config.guild_id)
bot.run()
if __name__ == "__main__":
main()
+9
View File
@@ -0,0 +1,9 @@
"""Discord channel adapter for turnstone."""
from turnstone.channels.discord.bot import TurnstoneBot
from turnstone.channels.discord.config import DiscordConfig
__all__ = [
"DiscordConfig",
"TurnstoneBot",
]
+357
View File
@@ -0,0 +1,357 @@
"""Discord bot adapter — connects Discord threads to turnstone workstreams.
:class:`TurnstoneBot` extends ``discord.ext.commands.Bot`` and manages the
lifecycle of event subscriptions, streaming message edits, and interactive
approval / plan-review views.
"""
from __future__ import annotations
import asyncio
import time
from dataclasses import dataclass, field
from typing import TYPE_CHECKING
from turnstone.channels._formatter import chunk_message
from turnstone.channels._routing import ChannelRouter
from turnstone.core.log import get_logger
from turnstone.mq.protocol import (
ApprovalRequestEvent,
ContentEvent,
ErrorEvent,
OutboundEvent,
PlanReviewEvent,
SessionResumedEvent,
TurnCompleteEvent,
)
if TYPE_CHECKING:
import discord
from discord.ext import commands
from turnstone.channels.discord.config import DiscordConfig
from turnstone.core.storage._protocol import StorageBackend
from turnstone.mq.async_broker import AsyncRedisBroker
log = get_logger(__name__)
# ---------------------------------------------------------------------------
# StreamingMessage helper
# ---------------------------------------------------------------------------
@dataclass
class StreamingMessage:
"""Accumulates streamed content and periodically edits a Discord message.
Discord rate-limits message edits, so we batch content updates and flush
at a configurable interval. On finalize we send any remaining content
and chunk if the total exceeds the platform limit.
"""
channel: discord.abc.Messageable
max_length: int = 2000
edit_interval: float = 1.5
_message: discord.Message | None = field(default=None, init=False, repr=False)
_buffer: list[str] = field(default_factory=list, init=False, repr=False)
_last_edit: float = field(default=0.0, init=False, repr=False)
async def append(self, text: str) -> None:
"""Add *text* to the buffer and edit the message if the interval has elapsed."""
self._buffer.append(text)
now = time.monotonic()
if now - self._last_edit >= self.edit_interval:
await self._flush()
async def finalize(self) -> None:
"""Flush any remaining buffered content, chunking if necessary."""
content = "".join(self._buffer)
if not content:
return
if self._message is not None:
# Final edit — may need chunking if content grew beyond the limit.
chunks = chunk_message(content, self.max_length)
try:
await self._message.edit(content=chunks[0])
except Exception:
log.debug("streaming_message.edit_failed_on_finalize")
# Any overflow chunks are sent as new messages.
for chunk in chunks[1:]:
await self.channel.send(chunk)
else:
# Never sent an initial message — send all chunks now.
for chunk in chunk_message(content, self.max_length):
await self.channel.send(chunk)
async def _flush(self) -> None:
"""Edit or create the message with the current buffer contents."""
content = "".join(self._buffer)
if not content:
return
# Truncate to max_length for the in-progress edit (finalize handles overflow).
display = content[: self.max_length]
try:
if self._message is None:
self._message = await self.channel.send(display)
else:
await self._message.edit(content=display)
except Exception:
log.debug("streaming_message.flush_failed")
self._last_edit = time.monotonic()
# ---------------------------------------------------------------------------
# TurnstoneBot
# ---------------------------------------------------------------------------
class TurnstoneBot:
"""Discord bot that bridges Discord threads to turnstone workstreams.
Parameters
----------
config:
Discord-specific configuration.
broker:
Async Redis broker for MQ communication.
storage:
Storage backend for persistent route / user lookups.
"""
channel_type: str = "discord"
def __init__(
self,
config: DiscordConfig,
broker: AsyncRedisBroker,
storage: StorageBackend,
) -> None:
import discord
from discord.ext import commands
self.config = config
self.broker = broker
self.storage = storage
self.router = ChannelRouter(
broker,
storage,
auto_approve=config.auto_approve,
auto_approve_tools=list(config.auto_approve_tools),
)
self._subscribed_ws: set[str] = set()
self._streaming: dict[str, StreamingMessage] = {}
intents = discord.Intents.default()
intents.message_content = True
self._bot: commands.Bot = commands.Bot(
command_prefix="!ts ",
intents=intents,
help_command=None,
)
# Attach ourselves so cogs can access the TurnstoneBot instance.
self._bot.turnstone = self # type: ignore[attr-defined]
# Register lifecycle hooks.
self._bot.setup_hook = self._setup_hook # type: ignore[method-assign]
@self._bot.event
async def on_ready() -> None:
await self._on_ready()
# -- lifecycle -----------------------------------------------------------
async def _setup_hook(self) -> None:
"""Called by discord.py after login but before connecting to the gateway."""
from turnstone.channels.discord.cog import MessageCog
from turnstone.channels.discord.views import ApprovalView, PlanReviewView
await self.broker.connect()
await self.router.start()
msg_cog = MessageCog(self._bot)
await self._bot.add_cog(msg_cog._cog)
# Register persistent views so button callbacks survive restarts.
self._bot.add_view(ApprovalView(self)._view)
self._bot.add_view(PlanReviewView(self)._view)
log.info("discord.setup_hook_complete")
async def _on_ready(self) -> None:
"""Sync slash commands and recover existing routes."""
import discord
bot = self._bot
log.info("discord.ready", user=str(bot.user), guild_count=len(bot.guilds))
if self.config.guild_id:
guild = discord.Object(id=self.config.guild_id)
bot.tree.copy_global_to(guild=guild)
await bot.tree.sync(guild=guild)
log.info("discord.commands_synced", guild_id=self.config.guild_id)
else:
await bot.tree.sync()
log.info("discord.commands_synced_global")
await self._recover_routes()
async def _recover_routes(self) -> None:
"""Re-subscribe to event channels for existing discord routes.
Queries the storage backend for all channel routes of type ``discord``
and subscribes to each workstream's event channel.
"""
routes = await asyncio.to_thread(self.storage.list_channel_routes_by_type, "discord")
for route in routes:
ws_id = route["ws_id"]
channel_id = int(route["channel_id"])
channel = self._bot.get_channel(channel_id)
if channel is not None:
await self.subscribe_ws(ws_id, channel) # type: ignore[arg-type]
log.info("discord.route_recovered", ws_id=ws_id, channel_id=channel_id)
else:
log.warning(
"discord.route_recovery_channel_missing",
ws_id=ws_id,
channel_id=channel_id,
)
# -- subscription management ---------------------------------------------
async def subscribe_ws(
self,
ws_id: str,
thread: discord.abc.Messageable,
) -> None:
"""Subscribe to workstream events and dispatch them to *thread*."""
if ws_id in self._subscribed_ws:
return
channel = f"{self.broker._prefix}:events:{ws_id}"
async def _callback(raw: str) -> None:
await self._on_ws_event(ws_id, thread, raw)
await self.broker.subscribe(channel, _callback)
self._subscribed_ws.add(ws_id)
log.info("discord.subscribed", ws_id=ws_id)
async def unsubscribe_ws(self, ws_id: str) -> None:
"""Cancel the subscription for *ws_id* and clean up streaming state."""
channel = f"{self.broker._prefix}:events:{ws_id}"
await self.broker.unsubscribe(channel)
self._subscribed_ws.discard(ws_id)
self._streaming.pop(ws_id, None)
log.info("discord.unsubscribed", ws_id=ws_id)
# -- event dispatch ------------------------------------------------------
async def _on_ws_event(
self,
ws_id: str,
thread: discord.abc.Messageable,
raw: str,
) -> None:
"""Handle an outbound event for a subscribed workstream."""
import discord
from turnstone.channels._formatter import format_approval_request, format_plan_review
from turnstone.channels.discord.views import ApprovalView, PlanReviewView
event = OutboundEvent.from_json(raw)
if isinstance(event, ContentEvent):
sm = self._streaming.get(ws_id)
if sm is None:
sm = StreamingMessage(
channel=thread,
max_length=self.config.max_message_length,
edit_interval=self.config.streaming_edit_interval,
)
self._streaming[ws_id] = sm
await sm.append(event.text)
elif isinstance(event, ApprovalRequestEvent):
if self.config.auto_approve or self._should_auto_approve(event):
await self.router.send_approval(ws_id, event.correlation_id, approved=True)
await thread.send("*Tool auto-approved.*")
else:
text = format_approval_request(event.items)
embed = discord.Embed(
title="Tool Approval Required",
description=text,
color=discord.Color.orange(),
)
embed.set_footer(text=f"{ws_id}|{event.correlation_id}")
await thread.send(embed=embed, view=ApprovalView(self)._view)
elif isinstance(event, PlanReviewEvent):
text = format_plan_review(event.content)
embed = discord.Embed(
title="Plan Review",
description=text,
color=discord.Color.blue(),
)
embed.set_footer(text=f"{ws_id}|{event.correlation_id}")
await thread.send(embed=embed, view=PlanReviewView(self)._view)
elif isinstance(event, TurnCompleteEvent):
sm = self._streaming.pop(ws_id, None)
if sm is not None:
await sm.finalize()
elif isinstance(event, SessionResumedEvent):
name = event.name or "previous session"
count = event.message_count
await thread.send(f"*Session resumed: {name} ({count} messages restored)*")
elif isinstance(event, ErrorEvent):
safe_msg = event.message[:500] if event.message else "An error occurred"
await thread.send(f"**Error:** {safe_msg}")
# -- helpers -------------------------------------------------------------
def _should_auto_approve(self, event: ApprovalRequestEvent) -> bool:
"""Return True if all tools in *event.items* are in the auto-approve list."""
allowed = self.config.auto_approve_tools
if not allowed or not event.items:
return False
for item in event.items:
# Support both server SSE format (func_name) and OpenAI format (function.name).
name = (
item.get("func_name")
or item.get("approval_label")
or item.get("function", {}).get("name", "")
)
if name not in allowed:
return False
return True
def _is_allowed_channel(self, channel_id: int) -> bool:
"""Return True if *channel_id* is in the allowed list (or list is empty)."""
if not self.config.allowed_channels:
return True
return channel_id in self.config.allowed_channels
def run(self, **kwargs: object) -> None:
"""Start the bot (blocking). Pass-through to ``commands.Bot.run``."""
self._bot.run(self.config.bot_token, log_handler=None, **kwargs) # type: ignore[arg-type]
async def start(self) -> None:
"""Start the bot (async). Use this for multi-adapter ``asyncio.gather``."""
await self._bot.start(self.config.bot_token, reconnect=True)
async def stop(self) -> None:
"""Disconnect the bot and clean up subscriptions."""
for ws_id in list(self._subscribed_ws):
await self.unsubscribe_ws(ws_id)
await self.router.stop()
await self.broker.close()
await self._bot.close()
+399
View File
@@ -0,0 +1,399 @@
"""Message handling cog for the Discord channel adapter.
Handles ``on_message`` events and slash commands (``/link``, ``/unlink``,
``/ask``, ``/status``, ``/close``).
"""
from __future__ import annotations
import asyncio
from typing import TYPE_CHECKING
from turnstone.core.log import get_logger
from turnstone.mq.protocol import CloseWorkstreamMessage
if TYPE_CHECKING:
import discord
from discord.ext import commands
from turnstone.channels.discord.bot import TurnstoneBot
log = get_logger(__name__)
_THREAD_NAME_MAX = 100
class MessageCog:
"""Cog that processes messages and registers slash commands.
Accessed via ``bot.turnstone`` to reach the :class:`TurnstoneBot` wrapper.
"""
def __init__(self, bot: commands.Bot) -> None:
import discord
from discord import app_commands
from discord.ext import commands as _commands
self.bot = bot
self.ts: TurnstoneBot = bot.turnstone # type: ignore[attr-defined]
# -- Cog wiring (manual since we can't use decorators with guarded imports) --
# We build the cog dynamically so discord.py's import is fully deferred.
cog_self = self
class _Cog(_commands.Cog, name="Turnstone"): # type: ignore[call-arg]
"""Turnstone Discord integration."""
@_commands.Cog.listener()
async def on_message(self_cog: _Cog, message: discord.Message) -> None: # noqa: N805
await cog_self._on_message(message)
@app_commands.command(name="link", description="Link your Discord account to Turnstone")
async def link(self_cog: _Cog, interaction: discord.Interaction) -> None: # noqa: N805
await interaction.response.send_modal(cog_self._link_modal_cls(cog_self))
@app_commands.command(
name="unlink", description="Unlink your Discord account from Turnstone"
)
async def unlink(self_cog: _Cog, interaction: discord.Interaction) -> None: # noqa: N805
await cog_self._cmd_unlink(interaction)
@app_commands.command(name="ask", description="Start a new Turnstone workstream")
@app_commands.describe(message="Your message to the assistant")
async def ask(self_cog: _Cog, interaction: discord.Interaction, message: str) -> None: # noqa: N805
await cog_self._cmd_ask(interaction, message)
@app_commands.command(name="status", description="Show workstream status")
async def status(self_cog: _Cog, interaction: discord.Interaction) -> None: # noqa: N805
await cog_self._cmd_status(interaction)
@app_commands.command(name="close", description="Close the current workstream")
async def close(self_cog: _Cog, interaction: discord.Interaction) -> None: # noqa: N805
await cog_self._cmd_close(interaction)
self._cog = _Cog()
# -- Modal for /link (avoids token appearing in slash command audit logs) --
class _LinkTokenModal(discord.ui.Modal, title="Link Account"): # type: ignore[call-arg]
token: discord.ui.TextInput[_LinkTokenModal] = discord.ui.TextInput(
label="API Token",
style=discord.TextStyle.short,
placeholder="Paste your ts_... API token",
required=True,
)
def __init__(modal_self, cog: MessageCog) -> None: # noqa: N805
super().__init__()
modal_self._cog = cog
async def on_submit(modal_self, interaction: discord.Interaction) -> None: # noqa: N805
await modal_self._cog._cmd_link(interaction, str(modal_self.token))
self._link_modal_cls = _LinkTokenModal
# -- on_message ----------------------------------------------------------
async def _on_message(self, message: discord.Message) -> None:
"""Route incoming messages to existing workstream threads."""
import discord
# Ignore self and other bots.
if message.author == self.bot.user or message.author.bot:
return
# Ignore DMs.
if message.guild is None:
return
channel = message.channel
# --- Message in a Discord Thread ---
if isinstance(channel, discord.Thread):
parent_id = channel.parent_id or 0
if not self.ts._is_allowed_channel(parent_id):
return
# Check if this thread has an existing route.
route = await asyncio.to_thread(
self.ts.storage.get_channel_route, "discord", str(channel.id)
)
if route is None:
# Not our thread — ignore.
return
# Resolve user.
user_id = await self.ts.router.resolve_user("discord", str(message.author.id))
if user_id is None:
return
# Use get_or_create_workstream so stale routes (evicted ws) are
# auto-refreshed with a new workstream.
try:
ws_id, is_new = await self.ts.router.get_or_create_workstream(
"discord",
str(channel.id),
name=channel.name or "",
initial_message="",
)
except (TimeoutError, RuntimeError):
log.warning("discord.ws_reactivation_failed", thread_id=channel.id)
return
if is_new:
await channel.send("*Workstream reactivated.*")
# Ensure subscription is active (handles bot restart recovery).
if ws_id not in self.ts._subscribed_ws:
await self.ts.subscribe_ws(ws_id, channel)
await self.ts.router.send_message(ws_id, message.content)
log.debug(
"discord.message_routed",
ws_id=ws_id,
thread_id=channel.id,
author=str(message.author),
)
return
# --- @mention in a non-thread channel ---
if self.bot.user is not None and self.bot.user.mentioned_in(message):
if not self.ts._is_allowed_channel(channel.id):
return
user_id = await self.ts.router.resolve_user("discord", str(message.author.id))
if user_id is None:
return
# Strip the mention from the message text.
content = message.content
if self.bot.user is not None:
content = content.replace(f"<@{self.bot.user.id}>", "").strip()
content = content.replace(f"<@!{self.bot.user.id}>", "").strip()
if not content:
content = "Hello"
# Create a thread from the message.
thread_name = content[:_THREAD_NAME_MAX] if len(content) > _THREAD_NAME_MAX else content
thread = await message.create_thread(
name=thread_name,
auto_archive_duration=self.ts.config.thread_auto_archive, # type: ignore[arg-type]
)
# Create workstream with the initial message.
ws_id, _is_new = await self.ts.router.get_or_create_workstream(
channel_type="discord",
channel_id=str(thread.id),
name=thread_name,
model=self.ts.config.model,
initial_message=content,
)
await self.ts.subscribe_ws(ws_id, thread)
log.info(
"discord.workstream_created",
ws_id=ws_id,
thread_id=thread.id,
author=str(message.author),
)
# -- slash commands ------------------------------------------------------
async def _cmd_link(self, interaction: discord.Interaction, token: str) -> None:
"""Link a Discord user to a turnstone account via API token."""
from turnstone.core.auth import hash_token
# Check if already linked.
existing = await asyncio.to_thread(
self.ts.storage.get_channel_user, "discord", str(interaction.user.id)
)
if existing:
await interaction.response.send_message(
"Your Discord account is already linked. Use `/unlink` first.",
ephemeral=True,
)
return
token_hash = hash_token(token)
token_record = await asyncio.to_thread(self.ts.storage.get_api_token_by_hash, token_hash)
if token_record is None:
await interaction.response.send_message(
"Invalid token. Please provide a valid Turnstone API token.",
ephemeral=True,
)
return
user_id = token_record.get("user_id", "")
if not user_id:
await interaction.response.send_message(
"Token has no associated user.",
ephemeral=True,
)
return
await asyncio.to_thread(
self.ts.storage.create_channel_user,
"discord",
str(interaction.user.id),
user_id,
)
await interaction.response.send_message("Account linked!", ephemeral=True)
log.info(
"discord.user_linked",
discord_user=str(interaction.user),
user_id=user_id,
)
async def _cmd_unlink(self, interaction: discord.Interaction) -> None:
"""Remove the channel user mapping for the calling Discord user."""
deleted = await asyncio.to_thread(
self.ts.storage.delete_channel_user,
"discord",
str(interaction.user.id),
)
if deleted:
await interaction.response.send_message("Account unlinked.", ephemeral=True)
log.info("discord.user_unlinked", discord_user=str(interaction.user))
else:
await interaction.response.send_message(
"No linked account found.",
ephemeral=True,
)
async def _cmd_ask(self, interaction: discord.Interaction, message: str) -> None:
"""Create a new thread and workstream with an initial message."""
import discord
user_id = await self.ts.router.resolve_user("discord", str(interaction.user.id))
if user_id is None:
await interaction.response.send_message(
"Your Discord account is not linked. Use `/link` first.",
ephemeral=True,
)
return
await interaction.response.defer()
channel = interaction.channel
if channel is None:
await interaction.followup.send("Cannot determine channel.", ephemeral=True)
return
thread_name = message[:_THREAD_NAME_MAX] if len(message) > _THREAD_NAME_MAX else message
# Create a standalone thread in the current channel.
if isinstance(channel, discord.TextChannel):
thread = await channel.create_thread(
name=thread_name,
auto_archive_duration=self.ts.config.thread_auto_archive, # type: ignore[arg-type]
type=discord.ChannelType.public_thread,
)
else:
await interaction.followup.send(
"Cannot create a thread in this channel type.",
ephemeral=True,
)
return
ws_id, _is_new = await self.ts.router.get_or_create_workstream(
channel_type="discord",
channel_id=str(thread.id),
name=thread_name,
model=self.ts.config.model,
initial_message=message,
)
await self.ts.subscribe_ws(ws_id, thread)
await interaction.followup.send(
f"Workstream started in {thread.mention}",
ephemeral=True,
)
log.info(
"discord.ask_workstream_created",
ws_id=ws_id,
thread_id=thread.id,
author=str(interaction.user),
)
async def _cmd_status(self, interaction: discord.Interaction) -> None:
"""Show workstream status for the current thread."""
import discord
channel = interaction.channel
if not isinstance(channel, discord.Thread):
await interaction.response.send_message(
"This command can only be used inside a thread.",
ephemeral=True,
)
return
route = await asyncio.to_thread(
self.ts.storage.get_channel_route, "discord", str(channel.id)
)
if route is None:
await interaction.response.send_message(
"No workstream is associated with this thread.",
ephemeral=True,
)
return
ws_id = route["ws_id"]
node_id = route.get("node_id", "")
created = route.get("created", "")
embed = discord.Embed(
title="Workstream Status",
color=discord.Color.green(),
)
embed.add_field(name="Workstream ID", value=ws_id, inline=False)
if node_id:
embed.add_field(name="Node", value=node_id, inline=True)
if created:
embed.add_field(name="Created", value=created, inline=True)
await interaction.response.send_message(embed=embed, ephemeral=True)
async def _cmd_close(self, interaction: discord.Interaction) -> None:
"""Close the workstream and archive the thread."""
import discord
channel = interaction.channel
if not isinstance(channel, discord.Thread):
await interaction.response.send_message(
"This command can only be used inside a thread.",
ephemeral=True,
)
return
route = await asyncio.to_thread(
self.ts.storage.get_channel_route, "discord", str(channel.id)
)
if route is None:
await interaction.response.send_message(
"No workstream is associated with this thread.",
ephemeral=True,
)
return
ws_id = route["ws_id"]
# Close via MQ.
msg = CloseWorkstreamMessage(ws_id=ws_id)
await self.ts.broker.push_inbound(msg.to_json())
# Delete route and unsubscribe.
await self.ts.router.delete_route("discord", str(channel.id))
await self.ts.unsubscribe_ws(ws_id)
await interaction.response.send_message("Workstream closed.")
log.info("discord.workstream_closed", ws_id=ws_id, thread_id=channel.id)
# Archive the thread.
try:
await channel.edit(archived=True)
except discord.Forbidden:
log.warning("discord.archive_forbidden", thread_id=channel.id)
+23
View File
@@ -0,0 +1,23 @@
"""Discord-specific configuration."""
from __future__ import annotations
from dataclasses import dataclass, field
from turnstone.channels._config import ChannelConfig
@dataclass
class DiscordConfig(ChannelConfig):
"""Configuration for the Discord channel adapter.
Extends :class:`ChannelConfig` with Discord-specific settings such as
the bot token, guild restriction, and streaming parameters.
"""
bot_token: str = ""
guild_id: int = 0 # 0 = all guilds
allowed_channels: list[int] = field(default_factory=list) # empty = all
thread_auto_archive: int = 1440 # minutes (24h)
max_message_length: int = 2000
streaming_edit_interval: float = 1.5 # seconds between message edits
+320
View File
@@ -0,0 +1,320 @@
"""Persistent interactive views for Discord approval and plan review.
These views use static ``custom_id`` values so they survive bot restarts.
Correlation information (``ws_id`` and ``correlation_id``) is stored in the
embed footer of the message the view is attached to.
"""
from __future__ import annotations
from typing import TYPE_CHECKING
from turnstone.core.log import get_logger
if TYPE_CHECKING:
import discord
from turnstone.channels.discord.bot import TurnstoneBot
log = get_logger(__name__)
def _parse_footer(interaction: discord.Interaction) -> tuple[str, str] | None:
"""Extract ``(ws_id, correlation_id)`` from the first embed's footer."""
if not interaction.message or not interaction.message.embeds:
return None
footer = interaction.message.embeds[0].footer.text
if not footer or "|" not in footer:
return None
parts = footer.split("|", 1)
return parts[0], parts[1]
async def _disable_buttons(interaction: discord.Interaction, label: str) -> None:
"""Edit the message to disable all buttons and append a result label."""
import discord
if interaction.message is None:
return
view = discord.ui.View()
for item in interaction.message.components or []:
for child in item.children: # type: ignore[union-attr]
button: discord.ui.Button[discord.ui.View] = discord.ui.Button(
label=getattr(child, "label", ""),
style=discord.ButtonStyle.secondary,
disabled=True,
custom_id=getattr(child, "custom_id", None),
)
view.add_item(button)
embed = interaction.message.embeds[0] if interaction.message.embeds else None
if embed is not None:
embed.color = discord.Color.greyple()
embed.title = f"{embed.title} - {label}"
await interaction.message.edit(embed=embed, view=view)
# ---------------------------------------------------------------------------
# ApprovalView
# ---------------------------------------------------------------------------
class ApprovalView:
"""Persistent view with Approve / Reject / Always Approve buttons."""
def __init__(self, bot: TurnstoneBot) -> None:
import discord
self.bot = bot
view_self = self
class _View(discord.ui.View):
def __init__(inner_self) -> None: # noqa: N805
super().__init__(timeout=None)
@discord.ui.button(
label="Approve",
style=discord.ButtonStyle.green,
custom_id="ts:approve",
)
async def approve(
inner_self, # noqa: N805
interaction: discord.Interaction,
button: discord.ui.Button[_View],
) -> None:
await view_self._handle(interaction, approved=True, always=False)
@discord.ui.button(
label="Reject",
style=discord.ButtonStyle.red,
custom_id="ts:reject",
)
async def reject(
inner_self, # noqa: N805
interaction: discord.Interaction,
button: discord.ui.Button[_View],
) -> None:
await view_self._handle(interaction, approved=False, always=False)
@discord.ui.button(
label="Always Approve",
style=discord.ButtonStyle.secondary,
custom_id="ts:always",
)
async def always_approve(
inner_self, # noqa: N805
interaction: discord.Interaction,
button: discord.ui.Button[_View],
) -> None:
await view_self._handle(interaction, approved=True, always=True)
self._view = _View()
async def _handle(
self,
interaction: discord.Interaction,
*,
approved: bool,
always: bool,
) -> None:
"""Process an approval button click."""
parsed = _parse_footer(interaction)
if parsed is None:
await interaction.response.send_message(
"Could not determine workstream context.",
ephemeral=True,
)
return
ws_id, correlation_id = parsed
# Verify user is linked. Scope enforcement (approve) happens
# server-side when the bridge executes the tool approval.
user_id = await self.bot.router.resolve_user("discord", str(interaction.user.id))
if user_id is None:
await interaction.response.send_message(
"Your Discord account is not linked. Use `/link` first.",
ephemeral=True,
)
return
# Defer before doing async work so Discord doesn't time out.
await interaction.response.defer(ephemeral=True)
await self.bot.router.send_approval(
ws_id=ws_id,
correlation_id=correlation_id,
approved=approved,
always=always,
)
label = "Always Approved" if always else ("Approved" if approved else "Rejected")
await _disable_buttons(interaction, label)
await interaction.followup.send(
f"Tool execution **{label.lower()}**.",
ephemeral=True,
)
log.info(
"discord.approval_response",
ws_id=ws_id,
correlation_id=correlation_id,
approved=approved,
always=always,
)
# ---------------------------------------------------------------------------
# PlanReviewView
# ---------------------------------------------------------------------------
class PlanReviewView:
"""Persistent view with Approve Plan / Request Changes buttons."""
def __init__(self, bot: TurnstoneBot) -> None:
import discord
self.bot = bot
view_self = self
class _FeedbackModal(discord.ui.Modal, title="Request Changes"): # type: ignore[call-arg]
feedback: discord.ui.TextInput[_FeedbackModal] = discord.ui.TextInput(
label="Feedback",
style=discord.TextStyle.paragraph,
placeholder="Describe the changes you'd like...",
required=True,
max_length=2000,
)
def __init__(modal_self, ws_id: str, correlation_id: str) -> None: # noqa: N805
super().__init__()
modal_self.ws_id = ws_id
modal_self.correlation_id = correlation_id
async def on_submit(modal_self, interaction: discord.Interaction) -> None: # noqa: N805
await view_self._send_feedback(
interaction,
modal_self.ws_id,
modal_self.correlation_id,
str(modal_self.feedback),
)
self._modal_cls = _FeedbackModal
class _View(discord.ui.View):
def __init__(inner_self) -> None: # noqa: N805
super().__init__(timeout=None)
@discord.ui.button(
label="Approve Plan",
style=discord.ButtonStyle.green,
custom_id="ts:plan_approve",
)
async def approve_plan(
inner_self, # noqa: N805
interaction: discord.Interaction,
button: discord.ui.Button[_View],
) -> None:
await view_self._handle_approve(interaction)
@discord.ui.button(
label="Request Changes",
style=discord.ButtonStyle.secondary,
custom_id="ts:plan_changes",
)
async def request_changes(
inner_self, # noqa: N805
interaction: discord.Interaction,
button: discord.ui.Button[_View],
) -> None:
await view_self._handle_changes(interaction)
self._view = _View()
async def _handle_approve(self, interaction: discord.Interaction) -> None:
"""Approve the plan (empty feedback = approved)."""
parsed = _parse_footer(interaction)
if parsed is None:
await interaction.response.send_message(
"Could not determine workstream context.",
ephemeral=True,
)
return
ws_id, correlation_id = parsed
user_id = await self.bot.router.resolve_user("discord", str(interaction.user.id))
if user_id is None:
await interaction.response.send_message(
"Your Discord account is not linked. Use `/link` first.",
ephemeral=True,
)
return
# Defer before doing async work so Discord doesn't time out.
await interaction.response.defer(ephemeral=True)
await self.bot.router.send_plan_feedback(
ws_id=ws_id,
correlation_id=correlation_id,
feedback="",
)
await _disable_buttons(interaction, "Approved")
await interaction.followup.send("Plan **approved**.", ephemeral=True)
log.info(
"discord.plan_approved",
ws_id=ws_id,
correlation_id=correlation_id,
)
async def _handle_changes(self, interaction: discord.Interaction) -> None:
"""Open a modal for feedback text."""
parsed = _parse_footer(interaction)
if parsed is None:
await interaction.response.send_message(
"Could not determine workstream context.",
ephemeral=True,
)
return
ws_id, correlation_id = parsed
modal = self._modal_cls(ws_id, correlation_id)
await interaction.response.send_modal(modal)
async def _send_feedback(
self,
interaction: discord.Interaction,
ws_id: str,
correlation_id: str,
feedback: str,
) -> None:
"""Send plan feedback after the modal is submitted."""
user_id = await self.bot.router.resolve_user("discord", str(interaction.user.id))
if user_id is None:
await interaction.response.send_message(
"Your Discord account is not linked. Use `/link` first.",
ephemeral=True,
)
return
# Defer before doing async work so Discord doesn't time out.
await interaction.response.defer(ephemeral=True)
await self.bot.router.send_plan_feedback(
ws_id=ws_id,
correlation_id=correlation_id,
feedback=feedback,
)
await interaction.followup.send(
"Feedback submitted. The plan will be revised.",
ephemeral=True,
)
log.info(
"discord.plan_changes_requested",
ws_id=ws_id,
correlation_id=correlation_id,
)
+7 -1
View File
@@ -828,6 +828,10 @@ def main() -> None:
apply_config(parser, ["api", "model", "session", "tools", "console", "auth", "mcp", "database"])
args = parser.parse_args()
from turnstone.core.log import configure_logging
configure_logging(level="WARNING", service="cli")
# Initialize storage backend
from turnstone.core.storage import init_storage
@@ -891,7 +895,9 @@ def main() -> None:
mcp_client = create_mcp_client(getattr(args, "mcp_config", None))
# Session factory — captures shared config for creating workstream sessions
def session_factory(ui: SessionUI | None, model_alias: str | None = None) -> ChatSession:
def session_factory(
ui: SessionUI | None, model_alias: str | None = None, ws_id: str | None = None
) -> ChatSession:
assert ui is not None, "session_factory requires a non-None UI"
r_client, r_model, r_cfg = registry.resolve(model_alias)
return ChatSession(
+445 -125
View File
@@ -30,8 +30,6 @@ import httpx
from sse_starlette import EventSourceResponse
from starlette.applications import Starlette
from starlette.middleware import Middleware
from starlette.middleware.cors import CORSMiddleware
from starlette.requests import Request
from starlette.responses import HTMLResponse, JSONResponse, Response
from starlette.routing import Mount, Route
from starlette.staticfiles import StaticFiles
@@ -39,12 +37,14 @@ from starlette.staticfiles import StaticFiles
from turnstone.api.console_spec import build_console_spec
from turnstone.api.docs import make_docs_handler, make_openapi_handler
from turnstone.console.collector import ClusterCollector
from turnstone.mq.broker import RedisBroker
from turnstone.core.auth import JWT_AUD_CONSOLE, AuthMiddleware
if TYPE_CHECKING:
from collections.abc import AsyncGenerator
from starlette.types import ASGIApp, Receive, Scope, Send
from starlette.requests import Request
from turnstone.mq.broker import RedisBroker
log = logging.getLogger("turnstone.console.server")
@@ -55,15 +55,11 @@ log = logging.getLogger("turnstone.console.server")
_STATIC_DIR = Path(__file__).parent / "static"
_SHARED_DIR = Path(__file__).parent.parent / "shared_static"
_HTML = ""
_CSS = ""
_JS = ""
def _load_static() -> None:
global _HTML, _CSS, _JS
global _HTML
_HTML = (_STATIC_DIR / "index.html").read_text(encoding="utf-8")
_CSS = (_STATIC_DIR / "style.css").read_text(encoding="utf-8")
_JS = (_STATIC_DIR / "app.js").read_text(encoding="utf-8")
# ---------------------------------------------------------------------------
@@ -90,35 +86,6 @@ def _parse_int(
# ---------------------------------------------------------------------------
class AuthMiddleware:
"""ASGI middleware that enforces bearer-token / cookie authentication."""
def __init__(self, app: ASGIApp) -> None:
self.app = app
async def __call__(self, scope: Scope, receive: Receive, send: Send) -> None:
if scope["type"] != "http":
await self.app(scope, receive, send)
return
request = Request(scope)
if request.method == "OPTIONS":
await self.app(scope, receive, send)
return
from turnstone.core.auth import check_request
auth_config = request.app.state.auth_config
method = request.method
path = request.url.path
auth_header = request.headers.get("Authorization")
cookie_header = request.headers.get("Cookie")
allowed, status, msg = check_request(auth_config, method, path, auth_header, cookie_header)
if not allowed:
response = JSONResponse({"error": msg}, status_code=status)
await response(scope, receive, send)
return
await self.app(scope, receive, send)
# ---------------------------------------------------------------------------
# Proxy helpers
# ---------------------------------------------------------------------------
@@ -150,10 +117,6 @@ _CONSOLE_BANNER_TEMPLATE = (
'<div style="background:#111827;border-bottom:1px solid rgba(229,160,66,0.3);'
"padding:6px 20px;font-family:'IBM Plex Mono',monospace;font-size:12px;"
'display:flex;align-items:center;gap:12px;position:relative;z-index:9999">'
'<span style="color:#e5a042;font-weight:700;font-size:13px;'
"font-family:'Outfit',sans-serif;letter-spacing:0.02em\">"
"turnstone</span>"
'<span style="color:#3b4463">\u2502</span>'
'<a href="/" style="color:#8a93ad;text-decoration:none;font-weight:500;'
'padding:2px 0" '
"onmouseover=\"this.style.color='#e5a042'\" "
@@ -171,6 +134,26 @@ _CONSOLE_PROXY_STYLE = "<style>.dashboard-overlay{top:32px!important}</style>"
_VALID_NODE_ID = re.compile(r"^[a-zA-Z0-9._-]+$")
def _proxy_auth_headers(request: Request) -> dict[str, str]:
"""Build auth headers for proxied requests to upstream servers.
Uses the service proxy token (``JWT_AUD_SERVER``) so the upstream node
accepts the request. The user's console-audience JWT is *not* forwarded
it would be rejected by the server's audience validation.
"""
# Prefer the auto-rotating ServiceTokenManager when available
mgr = getattr(request.app.state, "proxy_token_mgr", None)
if mgr is not None:
return dict(mgr.bearer_header)
# Fall back to static proxy_auth_token (e.g. from --auth-token)
static_token = getattr(request.app.state, "proxy_auth_token", "")
if static_token:
return {"Authorization": f"Bearer {static_token}"}
return {}
def _get_server_url(request: Request, node_id: str) -> str | None:
"""Resolve node_id to its server_url via the collector."""
if not node_id or not _VALID_NODE_ID.match(node_id) or len(node_id) > 256:
@@ -302,28 +285,31 @@ async def health(request: Request) -> JSONResponse:
async def auth_login(request: Request) -> Response:
from turnstone.core.auth import make_set_cookie
"""Authenticate via username:password or legacy token, return JWT."""
from turnstone.core.auth import handle_auth_login
try:
body: dict[str, Any] = await request.json()
except (ValueError, json.JSONDecodeError):
return JSONResponse({"error": "Invalid JSON body"}, status_code=400)
token = body.get("token", "")
auth_config = request.app.state.auth_config
role = auth_config.check(token)
if role:
response = JSONResponse({"status": "ok", "role": role})
response.headers["Set-Cookie"] = make_set_cookie(token)
return response
return JSONResponse({"error": "Invalid token"}, status_code=401)
return await handle_auth_login(request, JWT_AUD_CONSOLE)
async def auth_logout(request: Request) -> Response:
from turnstone.core.auth import make_clear_cookie
"""POST /v1/api/auth/logout — clear auth cookie."""
from turnstone.core.auth import handle_auth_logout
response = JSONResponse({"status": "ok"})
response.headers["Set-Cookie"] = make_clear_cookie()
return response
return await handle_auth_logout(request)
async def auth_status(request: Request) -> Response:
"""GET /v1/api/auth/status — public endpoint for login UI state detection."""
from turnstone.core.auth import handle_auth_status
return await handle_auth_status(request)
async def auth_setup(request: Request) -> Response:
"""POST /v1/api/auth/setup — create first admin user (public, one-time only)."""
from turnstone.core.auth import handle_auth_setup
return await handle_auth_setup(request, JWT_AUD_CONSOLE)
# ---------------------------------------------------------------------------
@@ -339,10 +325,11 @@ async def create_workstream(request: Request) -> JSONResponse:
- ``node_id`` omitted or ``"auto"`` console picks the node with most headroom
- ``node_id`` set to ``"pool"`` pushed to the shared queue for any bridge
"""
try:
body: dict[str, Any] = await request.json()
except (ValueError, json.JSONDecodeError):
return JSONResponse({"error": "Invalid JSON body"}, status_code=400)
from turnstone.core.web_helpers import read_json_or_400
body = await read_json_or_400(request)
if isinstance(body, JSONResponse):
return body
broker: RedisBroker = request.app.state.broker
collector: ClusterCollector = request.app.state.collector
@@ -427,7 +414,7 @@ async def proxy_index(request: Request) -> Response:
safe_node = urllib.parse.quote(node_id, safe="")
prefix = f"/node/{safe_node}"
try:
resp = await client.get(f"{server_url}/")
resp = await client.get(f"{server_url}/", headers=_proxy_auth_headers(request))
if resp.status_code < 200 or resp.status_code >= 300:
log.debug("Upstream %s returned status %s", node_id, resp.status_code)
return JSONResponse(
@@ -464,7 +451,10 @@ async def proxy_static(request: Request) -> Response:
client: httpx.AsyncClient = request.app.state.proxy_client
try:
resp = await client.get(f"{server_url}/static/{path}")
resp = await client.get(
f"{server_url}/static/{path}",
headers=_proxy_auth_headers(request),
)
return Response(
content=resp.content,
status_code=resp.status_code,
@@ -485,7 +475,10 @@ async def proxy_shared_static(request: Request) -> Response:
client: httpx.AsyncClient = request.app.state.proxy_client
try:
resp = await client.get(f"{server_url}/shared/{path}")
resp = await client.get(
f"{server_url}/shared/{path}",
headers=_proxy_auth_headers(request),
)
return Response(
content=resp.content,
status_code=resp.status_code,
@@ -537,7 +530,7 @@ async def _proxy_get(request: Request, server_url: str, path: str) -> Response:
if request.url.query:
target += f"?{request.url.query}"
try:
resp = await client.get(target)
resp = await client.get(target, headers=_proxy_auth_headers(request))
return Response(
content=resp.content,
status_code=resp.status_code,
@@ -559,11 +552,9 @@ async def _proxy_post(
if request.url.query:
target += f"?{request.url.query}"
try:
resp = await client.post(
target,
content=body,
headers={"Content-Type": content_type},
)
post_headers = {"Content-Type": content_type}
post_headers.update(_proxy_auth_headers(request))
resp = await client.post(target, content=body, headers=post_headers)
return Response(
content=resp.content,
status_code=resp.status_code,
@@ -583,12 +574,13 @@ async def _proxy_sse(
target += f"?{request.url.query}"
sse_client: httpx.AsyncClient = request.app.state.proxy_sse_client
sse_auth = _proxy_auth_headers(request)
async def sse_generator() -> AsyncGenerator[dict[str, str], None]:
from httpx_sse import aconnect_sse
try:
async with aconnect_sse(sse_client, "GET", target) as source:
async with aconnect_sse(sse_client, "GET", target, headers=sse_auth) as source:
if source.response.status_code != 200:
log.debug(
"SSE proxy received status %s from %s",
@@ -636,6 +628,256 @@ async def _lifespan(app: Starlette) -> AsyncGenerator[None, None]:
app.state.broker.close()
# ---------------------------------------------------------------------------
# App factory
# ---------------------------------------------------------------------------
# Admin API endpoints — user + token management
# ---------------------------------------------------------------------------
async def admin_list_users(request: Request) -> JSONResponse:
"""GET /v1/api/admin/users — list all users."""
from turnstone.core.web_helpers import require_storage_or_503
storage, err = require_storage_or_503(request)
if err:
return err
return JSONResponse({"users": storage.list_users()})
async def admin_create_user(request: Request) -> JSONResponse:
"""POST /v1/api/admin/users — create a new user."""
import uuid
from turnstone.core.auth import hash_password
from turnstone.core.web_helpers import require_storage_or_503
storage, err = require_storage_or_503(request)
if err:
return err
from turnstone.core.web_helpers import read_json_or_400
body = await read_json_or_400(request)
if isinstance(body, JSONResponse):
return body
username = body.get("username", "").strip()
display_name = body.get("display_name", "").strip()
password = body.get("password", "")
from turnstone.core.auth import is_valid_username
if not is_valid_username(username):
return JSONResponse(
{"error": "Invalid username (1-64 chars: letters, digits, . _ -)"},
status_code=400,
)
if not display_name:
return JSONResponse({"error": "display_name is required"}, status_code=400)
if not password or len(password) < 8:
return JSONResponse({"error": "Password must be at least 8 characters"}, status_code=400)
# Check username uniqueness
if storage.get_user_by_username(username) is not None:
return JSONResponse({"error": "Username already taken"}, status_code=409)
user_id = uuid.uuid4().hex
pw_hash = hash_password(password)
storage.create_user(user_id, username, display_name, pw_hash)
# Read back to get the storage-canonical created timestamp
user = storage.get_user(user_id)
return JSONResponse(
{
"user_id": user["user_id"],
"username": user["username"],
"display_name": user["display_name"],
"created": user["created"],
}
)
async def admin_delete_user(request: Request) -> JSONResponse:
"""DELETE /v1/api/admin/users/{user_id} — delete user + cascade tokens."""
from turnstone.core.web_helpers import require_storage_or_503
storage, err = require_storage_or_503(request)
if err:
return err
user_id = request.path_params["user_id"]
if storage.delete_user(user_id):
return JSONResponse({"status": "ok"})
return JSONResponse({"error": "User not found"}, status_code=404)
async def admin_list_tokens(request: Request) -> JSONResponse:
"""GET /v1/api/admin/users/{user_id}/tokens — list tokens for a user."""
from turnstone.core.web_helpers import require_storage_or_503
storage, err = require_storage_or_503(request)
if err:
return err
user_id = request.path_params["user_id"]
return JSONResponse({"tokens": storage.list_api_tokens(user_id)})
async def admin_create_token(request: Request) -> JSONResponse:
"""POST /v1/api/admin/users/{user_id}/tokens — create API token."""
import uuid
from turnstone.core.auth import generate_token, hash_token, token_prefix
from turnstone.core.web_helpers import require_storage_or_503
storage, err = require_storage_or_503(request)
if err:
return err
user_id = request.path_params["user_id"]
# Verify user exists
if storage.get_user(user_id) is None:
return JSONResponse({"error": "User not found"}, status_code=404)
try:
body: dict[str, Any] = await request.json()
except (ValueError, json.JSONDecodeError):
body = {}
name = body.get("name", "")
scopes = body.get("scopes", "read,write,approve")
expires_days = body.get("expires_days")
# Validate scopes
from turnstone.core.auth import VALID_SCOPES
requested = {s.strip() for s in scopes.split(",") if s.strip()}
if not requested or not requested.issubset(VALID_SCOPES):
return JSONResponse(
{"error": "Invalid scopes (allowed: read, write, approve)"}, status_code=400
)
expires: str | None = None
if expires_days is not None:
from datetime import UTC, datetime, timedelta
expires = (datetime.now(UTC) + timedelta(days=int(expires_days))).strftime(
"%Y-%m-%dT%H:%M:%S"
)
raw = generate_token()
tid = uuid.uuid4().hex
storage.create_api_token(
token_id=tid,
token_hash=hash_token(raw),
token_prefix=token_prefix(raw),
user_id=user_id,
name=name,
scopes=scopes,
expires=expires,
)
return JSONResponse(
{
"token": raw,
"token_id": tid,
"token_prefix": token_prefix(raw),
"scopes": scopes,
}
)
async def admin_revoke_token(request: Request) -> JSONResponse:
"""DELETE /v1/api/admin/tokens/{token_id} — revoke an API token."""
from turnstone.core.web_helpers import require_storage_or_503
storage, err = require_storage_or_503(request)
if err:
return err
token_id = request.path_params["token_id"]
if storage.delete_api_token(token_id):
return JSONResponse({"status": "ok"})
return JSONResponse({"error": "Token not found"}, status_code=404)
# ---------------------------------------------------------------------------
# Admin: Channel user mapping
# ---------------------------------------------------------------------------
async def admin_list_channels(request: Request) -> JSONResponse:
"""GET /v1/api/admin/users/{user_id}/channels — list channel links for a user."""
from turnstone.core.web_helpers import require_storage_or_503
storage, err = require_storage_or_503(request)
if err:
return err
user_id = request.path_params["user_id"]
channels = storage.list_channel_users_by_user(user_id)
return JSONResponse({"channels": channels})
async def admin_create_channel(request: Request) -> JSONResponse:
"""POST /v1/api/admin/users/{user_id}/channels — link a channel account."""
from turnstone.core.web_helpers import require_storage_or_503
storage, err = require_storage_or_503(request)
if err:
return err
user_id = request.path_params["user_id"]
from turnstone.core.web_helpers import read_json_or_400
body = await read_json_or_400(request)
if isinstance(body, JSONResponse):
return body
channel_type = body.get("channel_type", "").strip().lower()
channel_user_id = body.get("channel_user_id", "").strip()
if not channel_type:
return JSONResponse({"error": "channel_type is required"}, status_code=400)
if not channel_user_id:
return JSONResponse({"error": "channel_user_id is required"}, status_code=400)
if len(channel_type) > 64 or len(channel_user_id) > 256:
return JSONResponse({"error": "Value too long"}, status_code=400)
# Verify user exists
if storage.get_user(user_id) is None:
return JSONResponse({"error": "User not found"}, status_code=404)
# Check for existing mapping
existing = storage.get_channel_user(channel_type, channel_user_id)
if existing is not None:
return JSONResponse(
{"error": f"Channel user already linked to user {existing['user_id']}"},
status_code=409,
)
storage.create_channel_user(channel_type, channel_user_id, user_id)
result = storage.get_channel_user(channel_type, channel_user_id)
if result is None:
return JSONResponse({"error": "Failed to create channel mapping"}, status_code=500)
# Guard against race: another request may have claimed this channel_user_id.
if result.get("user_id") != user_id:
return JSONResponse(
{"error": f"Channel user already linked to user {result['user_id']}"},
status_code=409,
)
return JSONResponse(result)
async def admin_delete_channel(request: Request) -> JSONResponse:
"""DELETE /v1/api/admin/channels/{channel_type}/{channel_user_id} — unlink."""
from turnstone.core.web_helpers import require_storage_or_503
storage, err = require_storage_or_503(request)
if err:
return err
channel_type = request.path_params["channel_type"]
channel_user_id = request.path_params["channel_user_id"]
if storage.delete_channel_user(channel_type, channel_user_id):
return JSONResponse({"status": "ok"})
return JSONResponse({"error": "Channel link not found"}, status_code=404)
# ---------------------------------------------------------------------------
# App factory
# ---------------------------------------------------------------------------
@@ -646,7 +888,11 @@ def create_app(
collector: ClusterCollector,
broker: RedisBroker,
auth_config: Any,
jwt_secret: str = "",
auth_storage: Any = None,
proxy_auth_token: str = "",
proxy_token_mgr: Any = None,
cors_origins: list[str] | None = None,
) -> Starlette:
"""Build the Starlette ASGI application for the console dashboard."""
_spec = build_console_spec()
@@ -667,6 +913,30 @@ def create_app(
Route("/api/cluster/events", cluster_events_sse),
Route("/api/auth/login", auth_login, methods=["POST"]),
Route("/api/auth/logout", auth_logout, methods=["POST"]),
Route("/api/auth/status", auth_status),
Route("/api/auth/setup", auth_setup, methods=["POST"]),
Route("/api/admin/users", admin_list_users),
Route("/api/admin/users", admin_create_user, methods=["POST"]),
Route("/api/admin/users/{user_id}", admin_delete_user, methods=["DELETE"]),
Route("/api/admin/users/{user_id}/tokens", admin_list_tokens),
Route(
"/api/admin/users/{user_id}/tokens", admin_create_token, methods=["POST"]
),
Route("/api/admin/tokens/{token_id}", admin_revoke_token, methods=["DELETE"]),
Route(
"/api/admin/users/{user_id}/channels",
admin_list_channels,
),
Route(
"/api/admin/users/{user_id}/channels",
admin_create_channel,
methods=["POST"],
),
Route(
"/api/admin/channels/{channel_type}/{channel_user_id}",
admin_delete_channel,
methods=["DELETE"],
),
],
),
Route("/health", health),
@@ -682,24 +952,34 @@ def create_app(
Route("/node/{node_id}/api/{path:path}", proxy_api, methods=["GET", "POST"]),
Route("/node/{node_id}/{path:path}", proxy_non_api),
],
middleware=[
Middleware(
CORSMiddleware,
allow_origins=["*"],
allow_methods=["GET", "POST", "OPTIONS"],
allow_headers=["Content-Type", "Authorization"],
),
Middleware(AuthMiddleware),
],
middleware=_build_console_middleware(cors_origins),
lifespan=_lifespan,
)
app.state.collector = collector
app.state.broker = broker
app.state.auth_config = auth_config
app.state.jwt_secret = jwt_secret
app.state.auth_storage = auth_storage
app.state.proxy_auth_token = proxy_auth_token
app.state.proxy_token_mgr = proxy_token_mgr
from turnstone.core.auth import LoginRateLimiter
app.state.login_limiter = LoginRateLimiter()
return app
def _build_console_middleware(cors_origins: list[str] | None = None) -> list[Middleware]:
"""Build the middleware stack with optional CORS."""
stack: list[Middleware] = []
if cors_origins:
from turnstone.core.web_helpers import cors_middleware
stack.append(cors_middleware(cors_origins))
stack.append(Middleware(AuthMiddleware, jwt_audience=JWT_AUD_CONSOLE))
return stack
# ---------------------------------------------------------------------------
# Main entry point
# ---------------------------------------------------------------------------
@@ -727,40 +1007,18 @@ def main() -> None:
default=8090,
help="Port to listen on (default: 8090)",
)
parser.add_argument(
"--redis-host",
default="localhost",
help="Redis host (default: localhost)",
)
parser.add_argument(
"--redis-port",
type=int,
default=6379,
help="Redis port (default: 6379)",
)
parser.add_argument(
"--redis-password",
default=os.environ.get("REDIS_PASSWORD"),
help="Redis password (default: $REDIS_PASSWORD)",
)
parser.add_argument(
"--redis-db",
type=int,
default=0,
help="Redis DB number (default: 0)",
)
from turnstone.mq.broker import add_redis_args
add_redis_args(parser)
parser.add_argument(
"--poll-interval",
type=float,
default=10.0,
help="Node polling interval in seconds (default: 10)",
)
parser.add_argument(
"--log-level",
default="INFO",
choices=["DEBUG", "INFO", "WARNING", "ERROR"],
help="Log level (default: INFO)",
)
from turnstone.core.log import add_log_args
add_log_args(parser)
parser.add_argument(
"--auth-token",
default=os.environ.get("TURNSTONE_AUTH_TOKEN", ""),
@@ -772,41 +1030,103 @@ def main() -> None:
apply_config(parser, ["console", "redis", "auth"])
args = parser.parse_args()
logging.basicConfig(
level=getattr(logging, args.log_level),
format="%(asctime)s %(name)s %(levelname)s %(message)s",
)
from turnstone.core.log import configure_logging_from_args
broker = RedisBroker(
host=args.redis_host,
port=args.redis_port,
db=args.redis_db,
password=args.redis_password,
)
configure_logging_from_args(args, "console")
from turnstone.mq.broker import broker_from_args
broker = broker_from_args(args)
# If no explicit auth token is provided, use a ServiceTokenManager
# so collector JWTs auto-rotate. A shared JWT secret is required for
# multi-service deployments — ephemeral secrets differ per process.
collector_token = args.auth_token
collector_token_mgr = None
if not collector_token:
_jwt_secret = os.environ.get("TURNSTONE_JWT_SECRET", "")
if not _jwt_secret:
log.error(
"TURNSTONE_JWT_SECRET is not set and no --auth-token provided. "
"The console cannot authenticate to server nodes. Set TURNSTONE_JWT_SECRET "
"to a shared secret (at least 32 characters) or pass --auth-token."
)
raise SystemExit(1)
from turnstone.core.auth import JWT_AUD_SERVER, ServiceTokenManager
collector_token_mgr = ServiceTokenManager(
user_id="console-collector",
scopes=frozenset({"read"}),
source="console",
secret=_jwt_secret,
audience=JWT_AUD_SERVER,
expiry_hours=1,
)
collector_token = collector_token_mgr.token
log.info("console.collector_jwt_minted")
collector = ClusterCollector(
broker=broker,
poll_interval=args.poll_interval,
auth_token=args.auth_token,
auth_token=collector_token,
)
collector.start()
_load_static()
from turnstone.core.auth import load_auth_config
from turnstone.core.auth import load_auth_config, load_jwt_secret
auth_config = load_auth_config()
jwt_secret = load_jwt_secret() if auth_config.enabled else ""
# Initialize storage for user/token management (optional — requires DB config)
auth_storage = None
try:
from turnstone.core.storage import init_storage
db_backend = os.environ.get("TURNSTONE_DB_BACKEND", "sqlite")
db_url = os.environ.get("TURNSTONE_DB_URL", "")
db_path = os.environ.get("TURNSTONE_DB_PATH", "")
auth_storage = init_storage(db_backend, path=db_path, url=db_url)
except Exception:
log.info("Console storage not available — admin API disabled, JWT-only auth")
# If no explicit auth token is provided, use a ServiceTokenManager
# so proxy JWTs auto-rotate.
proxy_token = args.auth_token
proxy_token_mgr = None
if not proxy_token and jwt_secret:
from turnstone.core.auth import JWT_AUD_SERVER, ServiceTokenManager
proxy_token_mgr = ServiceTokenManager(
user_id="console-proxy",
scopes=frozenset({"write"}),
source="console",
secret=jwt_secret,
audience=JWT_AUD_SERVER,
expiry_hours=1,
)
proxy_token = proxy_token_mgr.token
log.info("console.proxy_jwt_minted")
from turnstone.core.web_helpers import parse_cors_origins
cors_origins = parse_cors_origins()
app = create_app(
collector=collector,
broker=broker,
auth_config=auth_config,
proxy_auth_token=args.auth_token,
jwt_secret=jwt_secret,
auth_storage=auth_storage,
proxy_auth_token=proxy_token,
proxy_token_mgr=proxy_token_mgr,
cors_origins=cors_origins,
)
print(f"turnstone console running on http://{args.host}:{args.port}")
log.info("Console starting on http://%s:%s", args.host, args.port)
if auth_config.enabled:
print(f"Auth: enabled ({len(auth_config.tokens)} token(s) configured)")
log.info("Auth: enabled (%d config token(s))", len(auth_config.tokens))
print("Press Ctrl+C to stop.")
import uvicorn
+800
View File
@@ -0,0 +1,800 @@
/* Admin panel — user & token management for turnstone console */
var _adminTab = "users";
var _adminUsers = [];
var _adminTokenUserId = "";
var _adminChannelUserId = "";
var _lastCreatedToken = "";
var _cuTrapHandler = null;
var _ctTrapHandler = null;
var _tcTrapHandler = null;
var _ccTrapHandler = null;
var _cfTrapHandler = null;
var _confirmCallbackFn = null;
var _confirmTriggerEl = null;
// ---------------------------------------------------------------------------
// View switching (called from app.js showOverview/drillDown pattern)
// ---------------------------------------------------------------------------
function showAdmin() {
/* global currentView */
currentView = "admin";
document.getElementById("view-overview").style.display = "none";
document.getElementById("view-node").style.display = "none";
document.getElementById("view-filtered").style.display = "none";
document.getElementById("view-admin").style.display = "";
document.getElementById("breadcrumb").style.display = "";
document.getElementById("breadcrumb-label").textContent = "Admin";
document.getElementById("main").scrollTop = 0;
history.pushState({ view: "admin" }, "");
loadAdminUsers();
}
function switchAdminTab(tab) {
_adminTab = tab;
var tabs = document.querySelectorAll(".admin-tab");
for (var i = 0; i < tabs.length; i++) {
var isActive = tabs[i].getAttribute("data-tab") === tab;
tabs[i].classList.toggle("active", isActive);
tabs[i].setAttribute("aria-selected", isActive ? "true" : "false");
tabs[i].setAttribute("tabindex", isActive ? "0" : "-1");
}
document.getElementById("admin-users").style.display =
tab === "users" ? "" : "none";
document.getElementById("admin-tokens").style.display =
tab === "tokens" ? "" : "none";
document.getElementById("admin-channels").style.display =
tab === "channels" ? "" : "none";
if (tab === "users") loadAdminUsers();
if (tab === "tokens") _populateTokenUserSelect();
if (tab === "channels") _populateChannelUserSelect();
}
// ---------------------------------------------------------------------------
// Users
// ---------------------------------------------------------------------------
function loadAdminUsers() {
authFetch("/v1/api/admin/users")
.then(function (r) {
if (!r.ok) throw new Error("Failed to load users");
return r.json();
})
.then(function (data) {
_adminUsers = data.users || [];
_renderUsers(_adminUsers);
_populateTokenUserSelect();
})
.catch(function () {
document.getElementById("admin-users-table").innerHTML =
'<div class="dashboard-empty">Failed to load users</div>';
});
}
function _renderUsers(users) {
var container = document.getElementById("admin-users-table");
if (!users.length) {
container.innerHTML =
'<div class="dashboard-empty">No users yet. Create one to get started.</div>';
return;
}
var html = "";
for (var i = 0; i < users.length; i++) {
var u = users[i];
html +=
'<div class="admin-row" role="listitem">' +
'<span class="admin-col admin-col-username">' +
escapeHtml(u.username) +
"</span>" +
'<span class="admin-col admin-col-name">' +
escapeHtml(u.display_name) +
"</span>" +
'<span class="admin-col admin-col-created">' +
escapeHtml(u.created || "").slice(0, 10) +
"</span>" +
'<span class="admin-col admin-col-actions">' +
'<button class="admin-btn-danger" data-delete-user="' +
escapeHtml(u.user_id) +
'" data-username="' +
escapeHtml(u.username) +
'" title="Delete user">delete</button>' +
"</span>" +
"</div>";
}
container.innerHTML = html;
// Bind delete buttons via delegation (avoids inline JS injection)
var btns = container.querySelectorAll("[data-delete-user]");
for (var j = 0; j < btns.length; j++) {
btns[j].addEventListener("click", function () {
confirmDeleteUser(
this.getAttribute("data-delete-user"),
this.getAttribute("data-username"),
);
});
}
}
function confirmDeleteUser(userId, username) {
showConfirmModal(
"Delete User",
"Delete user \u2018" +
username +
"\u2019 and all their tokens and channel links? This cannot be undone.",
"Delete",
function () {
authFetch("/v1/api/admin/users/" + encodeURIComponent(userId), {
method: "DELETE",
})
.then(function (r) {
if (!r.ok) throw new Error("Delete failed");
showToast("User '" + username + "' deleted");
loadAdminUsers();
})
.catch(function () {
showToast("Failed to delete user");
});
},
);
}
// ---------------------------------------------------------------------------
// Tokens
// ---------------------------------------------------------------------------
function _populateTokenUserSelect() {
var sel = document.getElementById("admin-token-user");
var current = sel.value;
sel.innerHTML = '<option value="">Select user...</option>';
for (var i = 0; i < _adminUsers.length; i++) {
var u = _adminUsers[i];
var opt = document.createElement("option");
opt.value = u.user_id;
opt.textContent = u.username + " (" + u.display_name + ")";
sel.appendChild(opt);
}
if (current) sel.value = current;
}
function loadAdminTokens() {
var userId = document.getElementById("admin-token-user").value;
_adminTokenUserId = userId;
if (!userId) {
document.getElementById("admin-tokens-table").innerHTML =
'<div class="dashboard-empty">Select a user to view tokens</div>';
return;
}
authFetch("/v1/api/admin/users/" + encodeURIComponent(userId) + "/tokens")
.then(function (r) {
if (!r.ok) throw new Error("Failed to load tokens");
return r.json();
})
.then(function (data) {
_renderTokens(data.tokens || []);
})
.catch(function () {
document.getElementById("admin-tokens-table").innerHTML =
'<div class="dashboard-empty">Failed to load tokens</div>';
});
}
function _renderTokens(tokens) {
var container = document.getElementById("admin-tokens-table");
if (!tokens.length) {
container.innerHTML =
'<div class="dashboard-empty">No tokens for this user</div>';
return;
}
var html = "";
for (var i = 0; i < tokens.length; i++) {
var t = tokens[i];
var expires = t.expires ? escapeHtml(t.expires).slice(0, 10) : "\u2014";
html +=
'<div class="admin-row" role="listitem">' +
'<span class="admin-col admin-col-prefix"><code>' +
escapeHtml(t.token_prefix) +
"\u2026</code></span>" +
'<span class="admin-col admin-col-tname">' +
escapeHtml(t.name || "\u2014") +
"</span>" +
'<span class="admin-col admin-col-scopes">' +
_renderScopeBadges(t.scopes) +
"</span>" +
'<span class="admin-col admin-col-created">' +
escapeHtml(t.created || "").slice(0, 10) +
"</span>" +
'<span class="admin-col admin-col-expires">' +
expires +
"</span>" +
'<span class="admin-col admin-col-actions">' +
'<button class="admin-btn-danger" data-revoke-token="' +
escapeHtml(t.token_id) +
'" title="Revoke token">revoke</button>' +
"</span>" +
"</div>";
}
container.innerHTML = html;
// Bind revoke buttons via delegation (avoids inline JS injection)
var rbtns = container.querySelectorAll("[data-revoke-token]");
for (var j = 0; j < rbtns.length; j++) {
rbtns[j].addEventListener("click", function () {
confirmRevokeToken(this.getAttribute("data-revoke-token"));
});
}
}
function _renderScopeBadges(scopes) {
if (!scopes) return "";
var parts = scopes.split(",");
var html = "";
for (var i = 0; i < parts.length; i++) {
var s = parts[i].trim();
if (!s) continue;
var cls = "scope-badge";
if (s === "approve") cls += " scope-approve";
else if (s === "write") cls += " scope-write";
html += '<span class="' + cls + '">' + escapeHtml(s) + "</span>";
}
return html;
}
function confirmRevokeToken(tokenId) {
showConfirmModal(
"Revoke Token",
"Revoke this API token? Existing JWTs issued from it will remain valid until they expire (max 24h). This cannot be undone.",
"Revoke",
function () {
authFetch("/v1/api/admin/tokens/" + encodeURIComponent(tokenId), {
method: "DELETE",
})
.then(function (r) {
if (!r.ok) throw new Error("Revoke failed");
showToast("Token revoked");
loadAdminTokens();
})
.catch(function () {
showToast("Failed to revoke token");
});
},
);
}
// ---------------------------------------------------------------------------
// Channels
// ---------------------------------------------------------------------------
function _populateChannelUserSelect() {
var sel = document.getElementById("admin-channel-user");
var current = sel.value;
sel.innerHTML = '<option value="">Select user...</option>';
for (var i = 0; i < _adminUsers.length; i++) {
var u = _adminUsers[i];
var opt = document.createElement("option");
opt.value = u.user_id;
opt.textContent = u.username + " (" + u.display_name + ")";
sel.appendChild(opt);
}
if (current) sel.value = current;
}
function loadAdminChannels() {
var userId = document.getElementById("admin-channel-user").value;
_adminChannelUserId = userId;
if (!userId) {
document.getElementById("admin-channels-table").innerHTML =
'<div class="dashboard-empty">Select a user to view channel links</div>';
return;
}
document.getElementById("admin-channels-table").innerHTML =
'<div class="dashboard-empty">Loading channel links...</div>';
authFetch("/v1/api/admin/users/" + encodeURIComponent(userId) + "/channels")
.then(function (r) {
if (!r.ok) throw new Error("Failed to load channels");
return r.json();
})
.then(function (data) {
_renderChannels(data.channels || []);
})
.catch(function () {
document.getElementById("admin-channels-table").innerHTML =
'<div class="dashboard-empty">Failed to load channel links</div>';
});
}
function _renderChannels(channels) {
var container = document.getElementById("admin-channels-table");
if (!channels.length) {
container.innerHTML =
'<div class="dashboard-empty">No channel links for this user</div>';
return;
}
var html = "";
for (var i = 0; i < channels.length; i++) {
var c = channels[i];
html +=
'<div class="admin-row" role="listitem">' +
'<span class="admin-col admin-col-chtype"><span class="scope-badge scope-channel">' +
escapeHtml(c.channel_type) +
"</span></span>" +
'<span class="admin-col admin-col-chuid"><code>' +
escapeHtml(c.channel_user_id) +
"</code></span>" +
'<span class="admin-col admin-col-created">' +
escapeHtml(c.created || "").slice(0, 10) +
"</span>" +
'<span class="admin-col admin-col-actions">' +
'<button class="admin-btn-danger" data-unlink-type="' +
escapeHtml(c.channel_type) +
'" data-unlink-uid="' +
escapeHtml(c.channel_user_id) +
'" title="Unlink channel account">unlink</button>' +
"</span>" +
"</div>";
}
container.innerHTML = html;
var btns = container.querySelectorAll("[data-unlink-type]");
for (var j = 0; j < btns.length; j++) {
btns[j].addEventListener("click", function () {
confirmUnlinkChannel(
this.getAttribute("data-unlink-type"),
this.getAttribute("data-unlink-uid"),
);
});
}
}
function confirmUnlinkChannel(channelType, channelUserId) {
showConfirmModal(
"Unlink Channel",
"Unlink " +
channelType +
" account \u2018" +
channelUserId +
"\u2019? The user will need to re-link via /link to interact with the bot.",
"Unlink",
function () {
authFetch(
"/v1/api/admin/channels/" +
encodeURIComponent(channelType) +
"/" +
encodeURIComponent(channelUserId),
{ method: "DELETE" },
)
.then(function (r) {
if (!r.ok) throw new Error("Unlink failed");
showToast("Channel account unlinked");
loadAdminChannels();
})
.catch(function () {
showToast("Failed to unlink channel account");
});
},
);
}
// ---------------------------------------------------------------------------
// Create Channel Link Modal
// ---------------------------------------------------------------------------
function showCreateChannelModal() {
if (!_adminChannelUserId) {
showToast("Select a user first");
return;
}
var overlay = document.getElementById("create-channel-overlay");
overlay.style.display = "flex";
document.getElementById("create-channel-error").style.display = "none";
document.getElementById("cc-type").value = "discord";
document.getElementById("cc-uid").value = "";
document.getElementById("cc-submit").disabled = false;
document.getElementById("cc-submit").textContent = "Link";
_ccTrapHandler = _installTrap("create-channel-overlay", "create-channel-box");
setTimeout(function () {
document.getElementById("cc-uid").focus();
}, 50);
}
function hideCreateChannelModal() {
document.getElementById("create-channel-overlay").style.display = "none";
_ccTrapHandler = _removeTrap(_ccTrapHandler);
var trigger = document.querySelector("#admin-channels .admin-action-btn");
if (trigger) trigger.focus();
}
function submitCreateChannel() {
var channelType = document.getElementById("cc-type").value;
var channelUserId = (document.getElementById("cc-uid").value || "").trim();
var errEl = document.getElementById("create-channel-error");
if (!channelUserId)
return _showModalError(errEl, "External user ID is required");
var btn = document.getElementById("cc-submit");
btn.disabled = true;
btn.textContent = "Linking\u2026";
authFetch(
"/v1/api/admin/users/" +
encodeURIComponent(_adminChannelUserId) +
"/channels",
{
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({
channel_type: channelType,
channel_user_id: channelUserId,
}),
},
)
.then(function (r) {
if (r.status === 409)
return r.json().then(function (d) {
throw new Error(d.error || "Already linked");
});
if (!r.ok)
return r.json().then(function (d) {
throw new Error(d.error || "Failed");
});
return r.json();
})
.then(function () {
hideCreateChannelModal();
showToast("Channel account linked");
loadAdminChannels();
})
.catch(function (err) {
btn.disabled = false;
btn.textContent = "Link";
_showModalError(errEl, err.message || "Failed to link channel account");
});
}
// ---------------------------------------------------------------------------
// Create User Modal
// ---------------------------------------------------------------------------
function showCreateUserModal() {
var overlay = document.getElementById("create-user-overlay");
overlay.style.display = "flex";
document.getElementById("create-user-error").style.display = "none";
document.getElementById("cu-username").value = "";
document.getElementById("cu-displayname").value = "";
document.getElementById("cu-password").value = "";
document.getElementById("cu-confirm").value = "";
document.getElementById("cu-submit").disabled = false;
document.getElementById("cu-submit").textContent = "Create";
_cuTrapHandler = _installTrap("create-user-overlay", "create-user-box");
setTimeout(function () {
document.getElementById("cu-username").focus();
}, 50);
}
function hideCreateUserModal() {
document.getElementById("create-user-overlay").style.display = "none";
_cuTrapHandler = _removeTrap(_cuTrapHandler);
var trigger = document.querySelector("#admin-users .admin-action-btn");
if (trigger) trigger.focus();
}
function submitCreateUser() {
var username = (document.getElementById("cu-username").value || "").trim();
var displayName = (
document.getElementById("cu-displayname").value || ""
).trim();
var password = document.getElementById("cu-password").value || "";
var confirm = document.getElementById("cu-confirm").value || "";
var errEl = document.getElementById("create-user-error");
if (!username) return _showModalError(errEl, "Username is required");
if (!displayName) return _showModalError(errEl, "Display name is required");
if (!password) return _showModalError(errEl, "Password is required");
if (password.length < 8)
return _showModalError(errEl, "Password must be at least 8 characters");
if (password !== confirm)
return _showModalError(errEl, "Passwords do not match");
var btn = document.getElementById("cu-submit");
btn.disabled = true;
btn.textContent = "Creating\u2026";
authFetch("/v1/api/admin/users", {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({
username: username,
display_name: displayName,
password: password,
}),
})
.then(function (r) {
if (r.status === 409) throw new Error("Username already taken");
if (!r.ok)
return r.json().then(function (d) {
throw new Error(d.error || "Failed");
});
return r.json();
})
.then(function () {
hideCreateUserModal();
showToast("User '" + username + "' created");
loadAdminUsers();
})
.catch(function (err) {
btn.disabled = false;
btn.textContent = "Create";
_showModalError(errEl, err.message || "Failed to create user");
});
}
// ---------------------------------------------------------------------------
// Create Token Modal
// ---------------------------------------------------------------------------
function showCreateTokenModal() {
if (!_adminTokenUserId) {
showToast("Select a user first");
return;
}
var overlay = document.getElementById("create-token-overlay");
overlay.style.display = "flex";
document.getElementById("create-token-error").style.display = "none";
document.getElementById("ct-name").value = "";
document.getElementById("ct-scopes").value = "read,write,approve";
document.getElementById("ct-expires").value = "";
document.getElementById("ct-submit").disabled = false;
document.getElementById("ct-submit").textContent = "Create";
_ctTrapHandler = _installTrap("create-token-overlay", "create-token-box");
setTimeout(function () {
document.getElementById("ct-name").focus();
}, 50);
}
function hideCreateTokenModal() {
document.getElementById("create-token-overlay").style.display = "none";
_ctTrapHandler = _removeTrap(_ctTrapHandler);
var trigger = document.querySelector("#admin-tokens .admin-action-btn");
if (trigger) trigger.focus();
}
function submitCreateToken() {
var name = (document.getElementById("ct-name").value || "").trim();
var scopes = document.getElementById("ct-scopes").value;
var expiresDays = document.getElementById("ct-expires").value;
var errEl = document.getElementById("create-token-error");
var btn = document.getElementById("ct-submit");
btn.disabled = true;
btn.textContent = "Creating\u2026";
var body = { name: name, scopes: scopes };
if (expiresDays) body.expires_days = parseInt(expiresDays, 10);
authFetch(
"/v1/api/admin/users/" + encodeURIComponent(_adminTokenUserId) + "/tokens",
{
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify(body),
},
)
.then(function (r) {
if (!r.ok)
return r.json().then(function (d) {
throw new Error(d.error || "Failed");
});
return r.json();
})
.then(function (data) {
hideCreateTokenModal();
_lastCreatedToken = data.token;
showTokenCreatedModal(data.token);
loadAdminTokens();
})
.catch(function (err) {
btn.disabled = false;
btn.textContent = "Create";
_showModalError(errEl, err.message || "Failed to create token");
});
}
// ---------------------------------------------------------------------------
// Token Created Modal (show-once)
// ---------------------------------------------------------------------------
function showTokenCreatedModal(token) {
document.getElementById("token-created-value").textContent = token;
document.getElementById("token-created-overlay").style.display = "flex";
_tcTrapHandler = _installTrap("token-created-overlay", "token-created-box");
}
function hideTokenCreatedModal() {
document.getElementById("token-created-overlay").style.display = "none";
_tcTrapHandler = _removeTrap(_tcTrapHandler);
_lastCreatedToken = "";
var trigger = document.querySelector("#admin-tokens .admin-action-btn");
if (trigger) trigger.focus();
}
function copyCreatedToken() {
if (!_lastCreatedToken) return;
if (navigator.clipboard) {
navigator.clipboard.writeText(_lastCreatedToken).then(function () {
showToast("Token copied to clipboard");
});
} else {
// Fallback: select the text
var el = document.getElementById("token-created-value");
var range = document.createRange();
range.selectNodeContents(el);
var sel = window.getSelection();
sel.removeAllRanges();
sel.addRange(range);
showToast("Select and copy the token");
}
}
// ---------------------------------------------------------------------------
// Modal focus trap + keyboard
// ---------------------------------------------------------------------------
function _modalFocusTrap(boxId) {
return function (e) {
if (e.key === "Tab") {
var box = document.getElementById(boxId);
if (!box) return;
var focusable = box.querySelectorAll(
"input:not([disabled]), select:not([disabled]), button:not([disabled])",
);
var visible = [];
for (var i = 0; i < focusable.length; i++) {
if (focusable[i].offsetParent !== null) visible.push(focusable[i]);
}
if (visible.length === 0) return;
var first = visible[0];
var last = visible[visible.length - 1];
if (e.shiftKey) {
if (document.activeElement === first) {
e.preventDefault();
last.focus();
}
} else {
if (document.activeElement === last) {
e.preventDefault();
first.focus();
}
}
}
};
}
function _installTrap(overlayId, boxId, trapRef) {
var overlay = document.getElementById(overlayId);
if (overlay) {
overlay.onclick = function (e) {
if (e.target === overlay) {
if (overlayId === "create-user-overlay") hideCreateUserModal();
else if (overlayId === "create-token-overlay") hideCreateTokenModal();
else if (overlayId === "token-created-overlay") hideTokenCreatedModal();
else if (overlayId === "create-channel-overlay")
hideCreateChannelModal();
else if (overlayId === "confirm-overlay") hideConfirmModal();
}
};
}
document.body.style.overflow = "hidden";
var handler = _modalFocusTrap(boxId);
document.addEventListener("keydown", handler);
return handler;
}
function _removeTrap(handler) {
if (handler) document.removeEventListener("keydown", handler);
document.body.style.overflow = "";
return null;
}
// Global Escape key for admin modals
document.addEventListener("keydown", function (e) {
if (e.key !== "Escape") return;
var cu = document.getElementById("create-user-overlay");
if (cu && cu.style.display !== "none") {
e.preventDefault();
hideCreateUserModal();
return;
}
var ct = document.getElementById("create-token-overlay");
if (ct && ct.style.display !== "none") {
e.preventDefault();
hideCreateTokenModal();
return;
}
var tc = document.getElementById("token-created-overlay");
if (tc && tc.style.display !== "none") {
e.preventDefault();
hideTokenCreatedModal();
return;
}
var cc = document.getElementById("create-channel-overlay");
if (cc && cc.style.display !== "none") {
e.preventDefault();
hideCreateChannelModal();
return;
}
var cf = document.getElementById("confirm-overlay");
if (cf && cf.style.display !== "none") {
e.preventDefault();
hideConfirmModal();
return;
}
});
// Tab arrow key navigation
(function () {
var tablist = document.querySelector(".admin-tabs");
if (!tablist) return;
tablist.addEventListener("keydown", function (e) {
if (e.key !== "ArrowLeft" && e.key !== "ArrowRight") return;
var tabOrder = ["users", "tokens", "channels"];
var idx = tabOrder.indexOf(_adminTab);
if (e.key === "ArrowRight") idx = (idx + 1) % tabOrder.length;
else idx = (idx - 1 + tabOrder.length) % tabOrder.length;
switchAdminTab(tabOrder[idx]);
var btn = document.querySelector(
'.admin-tab[data-tab="' + tabOrder[idx] + '"]',
);
if (btn) btn.focus();
});
})();
// ---------------------------------------------------------------------------
// Confirm Modal (reusable styled replacement for confirm())
// ---------------------------------------------------------------------------
function showConfirmModal(title, message, actionLabel, callback) {
_confirmCallbackFn = callback;
_confirmTriggerEl = document.activeElement;
document.getElementById("confirm-title").textContent = title;
document.getElementById("confirm-message").textContent = message;
var btn = document.getElementById("confirm-submit");
btn.textContent = actionLabel;
btn.disabled = false;
var overlay = document.getElementById("confirm-overlay");
overlay.style.display = "flex";
_cfTrapHandler = _installTrap("confirm-overlay", "confirm-box");
setTimeout(function () {
btn.focus();
}, 50);
}
function hideConfirmModal() {
document.getElementById("confirm-overlay").style.display = "none";
_cfTrapHandler = _removeTrap(_cfTrapHandler);
if (
_confirmTriggerEl &&
_confirmTriggerEl.focus &&
_confirmTriggerEl.isConnected
) {
_confirmTriggerEl.focus();
}
_confirmCallbackFn = null;
_confirmTriggerEl = null;
}
function _confirmCallback() {
var fn = _confirmCallbackFn;
_confirmCallbackFn = null;
var btn = document.getElementById("confirm-submit");
if (btn) btn.disabled = true;
if (fn) fn();
hideConfirmModal();
}
// ---------------------------------------------------------------------------
// Helpers
// ---------------------------------------------------------------------------
function _showModalError(el, msg) {
el.textContent = msg;
el.style.display = "block";
}
+29 -10
View File
@@ -52,12 +52,14 @@ function connectSSE() {
}
evtSource = new EventSource("/v1/api/cluster/events");
var statusBar = document.getElementById("status-bar");
evtSource.onmessage = function (e) {
evtSource.onopen = function () {
retryDelay = 1000;
statusBar.classList.remove("disconnected");
statusBar.textContent = "";
var csb = document.getElementById("cluster-status-bar");
if (csb) csb.classList.remove("stale");
};
evtSource.onmessage = function (e) {
try {
var data = JSON.parse(e.data);
handleClusterEvent(data);
@@ -68,6 +70,9 @@ function connectSSE() {
evtSource.onerror = function () {
evtSource.close();
evtSource = null;
// Don't show reconnecting state if login overlay is visible
var loginOverlay = document.getElementById("login-overlay");
if (loginOverlay && loginOverlay.style.display !== "none") return;
statusBar.textContent = "Reconnecting\u2026";
statusBar.classList.add("disconnected");
var csb = document.getElementById("cluster-status-bar");
@@ -126,6 +131,8 @@ function showOverview() {
document.getElementById("view-overview").style.display = "";
document.getElementById("view-node").style.display = "none";
document.getElementById("view-filtered").style.display = "none";
var adminView = document.getElementById("view-admin");
if (adminView) adminView.style.display = "none";
document.getElementById("breadcrumb").style.display = "none";
document.getElementById("main").scrollTop = 0;
loadOverview();
@@ -637,6 +644,8 @@ function drillDownToNode(nodeId, serverUrl) {
document.getElementById("view-overview").style.display = "none";
document.getElementById("view-node").style.display = "";
document.getElementById("view-filtered").style.display = "none";
var adminView = document.getElementById("view-admin");
if (adminView) adminView.style.display = "none";
document.getElementById("breadcrumb").style.display = "";
document.getElementById("breadcrumb-label").textContent = nodeId;
var link = document.getElementById("node-link");
@@ -685,6 +694,8 @@ function drillDownByState(state) {
document.getElementById("view-overview").style.display = "none";
document.getElementById("view-node").style.display = "none";
document.getElementById("view-filtered").style.display = "";
var adminView = document.getElementById("view-admin");
if (adminView) adminView.style.display = "none";
document.getElementById("breadcrumb").style.display = "";
var sd = STATE_DISPLAY[state] || STATE_DISPLAY.idle;
document.getElementById("breadcrumb-label").textContent =
@@ -703,6 +714,8 @@ function drillDownByNode(nodeId) {
document.getElementById("view-overview").style.display = "none";
document.getElementById("view-node").style.display = "none";
document.getElementById("view-filtered").style.display = "";
var adminView = document.getElementById("view-admin");
if (adminView) adminView.style.display = "none";
document.getElementById("breadcrumb").style.display = "";
document.getElementById("breadcrumb-label").textContent = nodeId;
document.getElementById("filtered-title").textContent =
@@ -884,14 +897,11 @@ function renderWsTable(container, wsList) {
row.classList.add("has-link");
(function (nodeId, wsId) {
row.onclick = function () {
window.open(
window.location.href =
"/node/" +
encodeURIComponent(nodeId) +
"/?ws_id=" +
encodeURIComponent(wsId),
"_blank",
"noopener",
);
encodeURIComponent(nodeId) +
"/?ws_id=" +
encodeURIComponent(wsId);
};
row.onkeydown = function (e) {
if (e.key === "Enter" || e.key === " ") {
@@ -918,6 +928,8 @@ window.addEventListener("popstate", function (e) {
return;
}
if (e.state.view === "overview") showOverview();
else if (e.state.view === "admin" && typeof showAdmin === "function")
showAdmin();
else if (e.state.view === "node" && e.state.nodeId)
drillDownToNode(e.state.nodeId, e.state.serverUrl);
else if (e.state.view === "filtered" && e.state.filter) {
@@ -1085,8 +1097,15 @@ document.addEventListener("keydown", function (e) {
});
// --- Init ---
// SSE connects after auth is confirmed — either via onLoginSuccess after
// login, or after the first successful data load (page refresh with valid cookie).
var _sseStarted = false;
function _ensureSSE() {
if (!_sseStarted) {
_sseStarted = true;
connectSSE();
}
}
history.replaceState({ view: "overview" }, "");
initLogin();
connectSSE();
loadOverview();
// Try loading — if auth required, login overlay will show
+163 -1
View File
@@ -16,6 +16,7 @@
<span id="cluster-summary" aria-live="polite"></span>
<span id="status-bar" role="status" aria-live="polite"></span>
<button id="new-ws-btn" class="header-btn header-btn-accent" onclick="showNewWsModal()" title="Create workstream">+ new</button>
<button id="admin-btn" class="header-btn" onclick="showAdmin()" title="User &amp; token administration">admin</button>
<button id="logout-btn" class="header-btn" onclick="logout()" style="display:none">logout</button>
<button id="theme-toggle" class="header-btn" onclick="toggleTheme()" aria-label="Toggle light/dark theme">&#9790;</button>
</div>
@@ -51,7 +52,7 @@
<span class="dash-col dash-col-ctx">CTX</span>
</div>
<div id="node-ws-table" class="dash-table" role="group" aria-label="Workstreams" aria-live="polite"></div>
<a id="node-link" class="node-link" target="_blank" rel="noopener">Open node UI</a>
<a id="node-link" class="node-link">Open node UI</a>
</div>
<!-- FILTERED WORKSTREAMS -->
@@ -72,6 +73,76 @@
<div id="filtered-ws-table" class="dash-table" role="group" aria-label="Workstreams" aria-live="polite"></div>
<div id="filtered-pagination" class="pagination"></div>
</div>
<!-- ADMIN PANEL -->
<div id="view-admin" style="display:none">
<div class="admin-tabs" role="tablist">
<button id="tab-users" class="admin-tab active" data-tab="users" role="tab" aria-selected="true" aria-controls="admin-users" tabindex="0" onclick="switchAdminTab('users')">Users</button>
<button id="tab-tokens" class="admin-tab" data-tab="tokens" role="tab" aria-selected="false" aria-controls="admin-tokens" tabindex="-1" onclick="switchAdminTab('tokens')">Tokens</button>
<button id="tab-channels" class="admin-tab" data-tab="channels" role="tab" aria-selected="false" aria-controls="admin-channels" tabindex="-1" onclick="switchAdminTab('channels')">Channels</button>
</div>
<!-- Users Tab -->
<div id="admin-users" class="admin-panel" role="tabpanel" aria-labelledby="tab-users">
<div class="admin-toolbar">
<span class="section-header" style="margin:0">USERS</span>
<button class="admin-action-btn" onclick="showCreateUserModal()">+ Create user</button>
</div>
<div class="admin-colheaders" aria-hidden="true">
<span class="admin-col admin-col-username">USERNAME</span>
<span class="admin-col admin-col-name">DISPLAY NAME</span>
<span class="admin-col admin-col-created">CREATED</span>
<span class="admin-col admin-col-actions">ACTIONS</span>
</div>
<div id="admin-users-table" role="list" aria-label="Users" aria-live="polite">
<div class="dashboard-empty">Loading users...</div>
</div>
</div>
<!-- Tokens Tab -->
<div id="admin-tokens" class="admin-panel" role="tabpanel" aria-labelledby="tab-tokens" style="display:none">
<div class="admin-toolbar">
<span class="section-header" style="margin:0">TOKENS</span>
<label for="admin-token-user" class="sr-only">Filter tokens by user</label>
<select id="admin-token-user" onchange="loadAdminTokens()">
<option value="">Select user...</option>
</select>
<button class="admin-action-btn" onclick="showCreateTokenModal()">+ Create token</button>
</div>
<div class="admin-colheaders" aria-hidden="true">
<span class="admin-col admin-col-prefix">PREFIX</span>
<span class="admin-col admin-col-tname">NAME</span>
<span class="admin-col admin-col-scopes">SCOPES</span>
<span class="admin-col admin-col-created">CREATED</span>
<span class="admin-col admin-col-expires">EXPIRES</span>
<span class="admin-col admin-col-actions">ACTIONS</span>
</div>
<div id="admin-tokens-table" role="list" aria-label="Tokens" aria-live="polite">
<div class="dashboard-empty">Select a user to view tokens</div>
</div>
</div>
<!-- Channels Tab -->
<div id="admin-channels" class="admin-panel" role="tabpanel" aria-labelledby="tab-channels" style="display:none">
<div class="admin-toolbar">
<span class="section-header" style="margin:0">CHANNEL LINKS</span>
<label for="admin-channel-user" class="sr-only">Filter channels by user</label>
<select id="admin-channel-user" onchange="loadAdminChannels()">
<option value="">Select user...</option>
</select>
<button class="admin-action-btn" onclick="showCreateChannelModal()">+ Link channel</button>
</div>
<div class="admin-colheaders" aria-hidden="true">
<span class="admin-col admin-col-chtype">CHANNEL</span>
<span class="admin-col admin-col-chuid">EXTERNAL ID</span>
<span class="admin-col admin-col-created">LINKED</span>
<span class="admin-col admin-col-actions">ACTIONS</span>
</div>
<div id="admin-channels-table" role="list" aria-label="Channel links" aria-live="polite">
<div class="dashboard-empty">Select a user to view channel links</div>
</div>
</div>
</div>
</div>
<div id="cluster-status-bar" role="region" aria-label="Cluster status">
@@ -123,6 +194,97 @@ window.TURNSTONE_KB_SHORTCUTS = [
</div>
</div>
<!-- Create User Modal -->
<div id="create-user-overlay" style="display:none" role="dialog" aria-modal="true" aria-labelledby="create-user-title">
<div id="create-user-box" class="admin-modal">
<h2 id="create-user-title">Create User</h2>
<div id="create-user-error" role="alert" aria-live="assertive"></div>
<label for="cu-username">Username</label>
<input id="cu-username" type="text" placeholder="login username" autocomplete="off" spellcheck="false">
<label for="cu-displayname">Display name</label>
<input id="cu-displayname" type="text" placeholder="Full name" autocomplete="off">
<label for="cu-password">Password</label>
<input id="cu-password" type="password" placeholder="Minimum 8 characters" autocomplete="new-password">
<label for="cu-confirm">Confirm password</label>
<input id="cu-confirm" type="password" placeholder="Confirm password" autocomplete="new-password">
<div class="modal-buttons">
<button class="modal-cancel" onclick="hideCreateUserModal()">Cancel</button>
<button id="cu-submit" class="modal-submit" onclick="submitCreateUser()">Create</button>
</div>
</div>
</div>
<!-- Create Token Modal -->
<div id="create-token-overlay" style="display:none" role="dialog" aria-modal="true" aria-labelledby="create-token-title">
<div id="create-token-box" class="admin-modal">
<h2 id="create-token-title">Create API Token</h2>
<div id="create-token-error" role="alert" aria-live="assertive"></div>
<label for="ct-name">Token name <span class="label-hint">optional</span></label>
<input id="ct-name" type="text" placeholder="e.g. CI pipeline, bridge-prod" autocomplete="off">
<label for="ct-scopes">Scopes</label>
<select id="ct-scopes">
<option value="read,write,approve">Full access (read, write, approve)</option>
<option value="read,write">Read + write</option>
<option value="read">Read only</option>
</select>
<label for="ct-expires">Expiry <span class="label-hint">optional</span></label>
<select id="ct-expires">
<option value="">Never</option>
<option value="30">30 days</option>
<option value="90">90 days</option>
<option value="365">1 year</option>
</select>
<div class="modal-buttons">
<button class="modal-cancel" onclick="hideCreateTokenModal()">Cancel</button>
<button id="ct-submit" class="modal-submit" onclick="submitCreateToken()">Create</button>
</div>
</div>
</div>
<!-- Token Created (show-once) Modal -->
<div id="token-created-overlay" style="display:none" role="dialog" aria-modal="true" aria-labelledby="token-created-title">
<div id="token-created-box" class="admin-modal">
<h2 id="token-created-title">Token Created</h2>
<p class="token-created-warning">Copy this token now. It will not be shown again.</p>
<div id="token-created-value" class="token-display"></div>
<div class="modal-buttons">
<button class="modal-submit" onclick="copyCreatedToken()">Copy to clipboard</button>
<button class="modal-cancel" onclick="hideTokenCreatedModal()">Done</button>
</div>
</div>
</div>
<!-- Confirm Action Modal (reusable) -->
<div id="confirm-overlay" style="display:none" role="alertdialog" aria-modal="true" aria-labelledby="confirm-title" aria-describedby="confirm-message">
<div id="confirm-box" class="admin-modal">
<h2 id="confirm-title">Confirm</h2>
<p id="confirm-message" style="color:var(--fg-dim);margin:12px 0 20px;line-height:1.5"></p>
<div class="modal-buttons">
<button class="modal-cancel" onclick="hideConfirmModal()">Cancel</button>
<button id="confirm-submit" class="modal-submit" style="background:var(--red);border-color:var(--red)" onclick="_confirmCallback()">Confirm</button>
</div>
</div>
</div>
<!-- Create Channel Link Modal -->
<div id="create-channel-overlay" style="display:none" role="dialog" aria-modal="true" aria-labelledby="create-channel-title">
<div id="create-channel-box" class="admin-modal">
<h2 id="create-channel-title">Link Channel Account</h2>
<div id="create-channel-error" role="alert" aria-live="assertive"></div>
<label for="cc-type">Channel type</label>
<select id="cc-type">
<option value="discord">Discord</option>
</select>
<label for="cc-uid">External user ID <span class="label-hint">the user's ID on the platform</span></label>
<input id="cc-uid" type="text" placeholder="e.g. 123456789012345678" autocomplete="off" spellcheck="false">
<div class="modal-buttons">
<button class="modal-cancel" onclick="hideCreateChannelModal()">Cancel</button>
<button id="cc-submit" class="modal-submit" onclick="submitCreateChannel()">Link</button>
</div>
</div>
</div>
<script src="/static/admin.js"></script>
<script src="/static/app.js"></script>
</body>
</html>
+296
View File
@@ -23,6 +23,7 @@
========================================================================== */
#main {
flex: 1;
min-height: 0;
overflow-y: auto;
padding: 20px 24px;
padding-bottom: 60px;
@@ -678,6 +679,298 @@
#header h1 { font-size: 13px; }
}
/* ==========================================================================
Admin panel
========================================================================== */
.admin-tabs {
display: flex;
gap: 2px;
margin-bottom: 16px;
border-bottom: 1px solid var(--border);
padding-bottom: 0;
}
.admin-tab {
background: none;
border: none;
border-bottom: 2px solid transparent;
color: var(--fg-dim);
font-family: var(--font-display);
font-size: 11px;
font-weight: 600;
text-transform: uppercase;
letter-spacing: 0.08em;
padding: 8px 16px 10px;
cursor: pointer;
transition: color 0.15s, border-color 0.15s;
}
.admin-tab:hover { color: var(--fg); }
.admin-tab.active {
color: var(--accent);
border-bottom-color: var(--accent);
}
.admin-toolbar {
display: flex;
align-items: center;
gap: 12px;
margin-bottom: 12px;
}
.admin-action-btn {
margin-left: auto;
background: var(--accent);
color: var(--bg);
border: none;
border-radius: var(--radius-sm);
font-family: var(--font-display);
font-size: 11px;
font-weight: 600;
padding: 6px 14px;
cursor: pointer;
letter-spacing: 0.02em;
transition: filter 0.15s;
}
.admin-action-btn:hover { filter: brightness(1.1); }
.admin-action-btn:focus-visible { outline: 2px solid var(--fg-bright); outline-offset: 2px; }
.admin-action-btn:disabled { opacity: 0.4; cursor: not-allowed; }
.admin-toolbar select {
background: var(--bg);
color: var(--fg);
border: 1px solid var(--border-strong);
border-radius: var(--radius-sm);
font: inherit;
font-size: 12px;
padding: 5px 30px 5px 10px;
min-width: 180px;
appearance: none;
-webkit-appearance: none;
background-image: url("data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' width='12' height='8' viewBox='0 0 12 8'%3E%3Cpath d='M1 1l5 5 5-5' stroke='%238a93ad' stroke-width='1.5' fill='none' stroke-linecap='round'/%3E%3C/svg%3E");
background-repeat: no-repeat;
background-position: right 10px center;
}
.admin-toolbar select:focus { border-color: var(--accent); outline: none; box-shadow: 0 0 0 3px var(--accent-dim); }
/* Admin table grid */
.admin-colheaders {
display: grid;
padding: 0 12px;
margin-bottom: 4px;
}
.admin-colheaders .admin-col {
font-family: var(--font-display);
font-size: 10px;
font-weight: 600;
text-transform: uppercase;
letter-spacing: 0.08em;
color: var(--fg-dim);
}
.admin-row {
display: grid;
padding: 8px 12px;
align-items: center;
border-radius: var(--radius-sm);
transition: background 0.1s;
}
.admin-row:nth-child(even) { background: var(--row-alt, rgba(255,255,255,0.015)); }
.admin-row:hover { background: var(--bg-highlight); }
.admin-col { font-size: 12px; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; }
.admin-col code { font-family: var(--font-mono); font-size: 11px; color: var(--fg-dim); }
/* Users grid: USERNAME | NAME | CREATED | ACTIONS */
#admin-users .admin-colheaders,
#admin-users .admin-row {
grid-template-columns: 140px 1fr 100px 80px;
}
/* Tokens grid: PREFIX | NAME | SCOPES | CREATED | EXPIRES | ACTIONS */
#admin-tokens .admin-colheaders,
#admin-tokens .admin-row {
grid-template-columns: 100px 1fr 160px 100px 100px 80px;
}
/* Channels grid: CHANNEL | EXTERNAL ID | LINKED | ACTIONS */
#admin-channels .admin-colheaders,
#admin-channels .admin-row {
grid-template-columns: 100px 1fr 100px 80px;
}
/* Scope badges */
.scope-badge {
display: inline-block;
font-family: var(--font-display);
font-size: 9px;
font-weight: 600;
text-transform: uppercase;
letter-spacing: 0.06em;
padding: 1px 6px;
border-radius: 2px;
margin-right: 3px;
background: var(--bg-highlight);
color: var(--fg-dim);
border: 1px solid var(--border);
}
.scope-write { color: var(--cyan); border-color: rgba(103, 232, 249, 0.2); }
.scope-approve { color: var(--accent); border-color: var(--accent-dim); }
.scope-channel { color: var(--magenta); border-color: rgba(192, 132, 252, 0.25); }
/* Action buttons */
.admin-btn-danger {
background: none;
border: 1px solid var(--red);
color: var(--red);
font-family: var(--font-display);
font-size: 10px;
font-weight: 500;
padding: 2px 8px;
border-radius: var(--radius-sm);
cursor: pointer;
opacity: 0.8;
transition: opacity 0.15s, background 0.15s;
}
.admin-btn-danger:hover { opacity: 1; background: rgba(248, 113, 113, 0.1); }
.admin-btn-danger:focus-visible { outline: 2px solid var(--red); outline-offset: 2px; }
/* Admin modals (reuse new-ws-overlay pattern) */
.admin-modal {
background: var(--bg-surface);
border: 1px solid var(--border-strong);
border-radius: var(--radius);
padding: 32px;
width: 380px;
max-width: 90vw;
box-shadow: 0 24px 48px -12px rgba(0, 0, 0, 0.5), 0 0 80px -20px var(--accent-dim);
position: relative;
}
.admin-modal::before {
content: '';
position: absolute;
top: -1px; left: 20%; right: 20%;
height: 2px;
background: linear-gradient(90deg, transparent, var(--accent), transparent);
border-radius: 1px;
}
.admin-modal h2 {
font-family: var(--font-display);
font-size: 15px;
font-weight: 700;
color: var(--accent);
margin-bottom: 16px;
letter-spacing: 0.02em;
}
.admin-modal label {
display: block;
font-family: var(--font-display);
font-size: 10px;
font-weight: 600;
text-transform: uppercase;
letter-spacing: 0.08em;
color: var(--fg-dim);
margin-bottom: 5px;
margin-top: 12px;
}
.admin-modal label:first-of-type { margin-top: 0; }
.admin-modal input, .admin-modal select {
width: 100%;
padding: 9px 12px;
background: var(--bg);
border: 1px solid var(--border-strong);
border-radius: var(--radius-sm);
color: var(--fg);
font: inherit;
font-size: 13px;
transition: border-color 0.15s, box-shadow 0.15s;
}
.admin-modal input:focus, .admin-modal select:focus {
border-color: var(--accent);
outline: none;
box-shadow: 0 0 0 3px var(--accent-dim);
}
.admin-modal input::placeholder { color: var(--fg-dim); opacity: 0.6; }
.admin-modal [role="alert"] { color: var(--red); font-size: 12px; margin-bottom: 8px; display: none; }
.modal-buttons { display: flex; gap: 10px; margin-top: 20px; }
.modal-cancel {
flex: 1;
padding: 9px;
background: var(--bg-highlight);
color: var(--fg);
border: 1px solid var(--border-strong);
border-radius: var(--radius-sm);
font: inherit;
font-family: var(--font-display);
font-size: 12px;
font-weight: 500;
cursor: pointer;
transition: background 0.15s;
}
.modal-cancel:hover { background: var(--bg-elevated); }
.modal-cancel:focus-visible { outline: 2px solid var(--accent); outline-offset: 2px; }
.modal-submit {
flex: 1;
padding: 9px;
background: var(--accent);
color: var(--bg);
border: none;
border-radius: var(--radius-sm);
font: inherit;
font-family: var(--font-display);
font-size: 12px;
font-weight: 600;
cursor: pointer;
transition: filter 0.15s;
}
.modal-submit:hover { filter: brightness(1.1); }
.modal-submit:focus-visible { outline: 2px solid var(--fg-bright); outline-offset: 2px; }
.modal-submit:disabled { opacity: 0.4; cursor: not-allowed; filter: none; }
#create-user-overlay, #create-token-overlay, #token-created-overlay, #create-channel-overlay, #confirm-overlay {
position: fixed;
inset: 0;
background: rgba(0, 0, 0, 0.7);
backdrop-filter: blur(6px);
-webkit-backdrop-filter: blur(6px);
display: flex;
align-items: center;
justify-content: center;
z-index: 500;
}
/* Token display (show-once) */
.token-created-warning {
font-family: var(--font-display);
font-size: 11px;
color: var(--yellow);
margin-bottom: 12px;
}
.token-display {
font-family: var(--font-mono);
font-size: 11px;
color: var(--fg-bright);
background: var(--bg);
padding: 12px;
border-radius: var(--radius-sm);
border: 1px solid var(--border-strong);
word-break: break-all;
user-select: all;
}
@media (max-width: 700px) {
#admin-users .admin-colheaders, #admin-users .admin-row {
grid-template-columns: 100px 1fr 80px;
}
.admin-col-created { display: none; }
#admin-tokens .admin-colheaders, #admin-tokens .admin-row {
grid-template-columns: 80px 1fr 100px 80px;
}
.admin-col-created, .admin-col-expires { display: none; }
#admin-channels .admin-colheaders, #admin-channels .admin-row {
grid-template-columns: 80px 1fr 80px;
}
#admin-channels .admin-col-created { display: none; }
}
/* ==========================================================================
Reduced motion console-specific
========================================================================== */
@@ -688,4 +981,7 @@
.node-link, .dash-cell-node, .pagination button { transition: none; }
.dash-row.has-link::after, .node-group-header::before { transition: none; }
#new-ws-box select, #new-ws-box input, #new-ws-buttons button { transition: none; }
.admin-tab, .admin-row, .admin-btn-danger { transition: none; }
.admin-action-btn, .modal-cancel, .modal-submit { transition: none; }
.admin-modal input, .admin-modal select { transition: none; }
}
+788 -66
View File
@@ -1,42 +1,106 @@
"""Bearer token authentication and authorization for turnstone HTTP servers.
Opt-in via the ``[auth]`` section in ``config.toml``. When auth is disabled
(the default), all requests pass through unchecked. When enabled, API
requests must include a valid ``Authorization: Bearer <token>`` header or
a ``turnstone_auth`` cookie (set via the ``/v1/api/auth/login`` endpoint).
Each token has a role: ``"read"`` or ``"full"``.
Supports three token types:
1. **Config-file tokens** static tokens in ``config.toml`` or the
``TURNSTONE_AUTH_TOKEN`` env var. Validated in-memory via
``hmac.compare_digest``. Map to scopes via their role.
2. **API tokens** database-backed, prefixed ``ts_``, stored as SHA-256
hashes. Exchanged for JWTs via ``/api/auth/login``.
3. **JWTs** short-lived session tokens issued after API token validation.
Validated locally via shared HMAC-SHA256 secret. Contain user_id and
scopes in claims.
Public paths (``/``, ``/static/*``, ``/shared/*``, ``/health``, ``/metrics``,
``/openapi.json``, ``/docs``, ``/api/auth/login``, ``/api/auth/logout``) are
always accessible without authentication. Paths under ``/v1/`` are
normalised by stripping the version prefix before classification so that
``/v1/api/send`` maps to ``/api/send`` in the path lists.
always accessible without authentication.
"""
from __future__ import annotations
import hashlib
import hmac
import json
import logging
import os
import re
import secrets
import threading
import time
import uuid
from dataclasses import dataclass, field
from typing import TYPE_CHECKING, Any
if TYPE_CHECKING:
from starlette.requests import Request
from starlette.responses import Response
from starlette.types import ASGIApp, Receive, Scope, Send
log = logging.getLogger(__name__)
# ---------------------------------------------------------------------------
# Public / write path classification
# Constants
# ---------------------------------------------------------------------------
AUTH_COOKIE = "turnstone_auth"
TOKEN_PREFIX = "ts_"
TOKEN_BYTES = 32 # 64 hex chars after prefix
JWT_ISSUER = "turnstone"
JWT_AUD_SERVER = "turnstone-server"
JWT_AUD_CONSOLE = "turnstone-console"
_MIN_SECRET_LENGTH = 32 # 256 bits minimum for HMAC-SHA256
VALID_SCOPES: frozenset[str] = frozenset({"read", "write", "approve"})
_USERNAME_RE = re.compile(r"^[a-zA-Z0-9._-]+$")
USERNAME_MAX_LEN = 64
def is_valid_username(username: str) -> bool:
"""Return True if *username* contains only safe characters (letters, digits, `.`, `_`, `-`)."""
return (
bool(username)
and len(username) <= USERNAME_MAX_LEN
and _USERNAME_RE.match(username) is not None
)
# Hierarchical: each scope implies all lower scopes.
SCOPE_HIERARCHY: dict[str, frozenset[str]] = {
"read": frozenset({"read"}),
"write": frozenset({"read", "write"}),
"approve": frozenset({"read", "write", "approve"}),
}
# Map old role names to scope sets.
_ROLE_TO_SCOPES: dict[str, frozenset[str]] = {
"read": frozenset({"read"}),
"full": frozenset({"read", "write", "approve"}),
}
# ---------------------------------------------------------------------------
# Path classification
# ---------------------------------------------------------------------------
PUBLIC_PATHS: frozenset[str] = frozenset(
{"/", "/health", "/metrics", "/openapi.json", "/docs", "/api/auth/login", "/api/auth/logout"}
{
"/",
"/health",
"/metrics",
"/openapi.json",
"/docs",
"/api/auth/login",
"/api/auth/logout",
"/api/auth/status",
"/api/auth/setup",
}
)
PUBLIC_PREFIXES: tuple[str, ...] = ("/static/", "/shared/")
WRITE_PATHS: frozenset[str] = frozenset(
{
"/api/send",
"/api/approve",
"/api/plan",
"/api/command",
"/api/workstreams/new",
@@ -45,16 +109,37 @@ WRITE_PATHS: frozenset[str] = frozenset(
}
)
APPROVE_PATHS: frozenset[str] = frozenset({"/api/approve"})
ADMIN_PREFIX = "/api/admin/"
def _strip_version_prefix(path: str) -> str:
"""Strip ``/v1`` prefix for path classification -- keeps path lists unversioned."""
"""Strip ``/v1`` prefix for path classification."""
if path.startswith("/v1/"):
return path[3:]
return path
# ---------------------------------------------------------------------------
# AuthConfig
# AuthResult
# ---------------------------------------------------------------------------
@dataclass(frozen=True)
class AuthResult:
"""Result of successful authentication."""
user_id: str # empty string for config-file tokens
scopes: frozenset[str]
token_source: str # "config", "jwt", "database"
def has_scope(self, scope: str) -> bool:
"""Return True if this result includes *scope*."""
return scope in self.scopes
# ---------------------------------------------------------------------------
# AuthConfig (unchanged from before — static config-file tokens)
# ---------------------------------------------------------------------------
@@ -66,7 +151,7 @@ class AuthConfig:
tokens: dict[str, str] = field(default_factory=dict) # token_value → role
def check(self, token: str | None) -> str | None:
"""Return the role (``"read"`` or ``"full"``) for a valid token, or *None*."""
"""Return the role for a valid config token, or *None*."""
if not token:
return None
for known_token, role in self.tokens.items():
@@ -75,6 +160,146 @@ class AuthConfig:
return None
# ---------------------------------------------------------------------------
# Token generation and hashing
# ---------------------------------------------------------------------------
def generate_token() -> str:
"""Generate a new API token: ``ts_`` + 64 hex chars (32 random bytes)."""
return TOKEN_PREFIX + secrets.token_hex(TOKEN_BYTES)
def hash_token(token: str) -> str:
"""Return the SHA-256 hex digest of *token*."""
return hashlib.sha256(token.encode("utf-8")).hexdigest()
def token_prefix(token: str) -> str:
"""Return the first 8 characters of a raw token (for display in listings)."""
return token[:8]
# ---------------------------------------------------------------------------
# Password hashing (bcrypt)
# ---------------------------------------------------------------------------
def hash_password(password: str) -> str:
"""Hash a password with bcrypt. Returns the hash as a string."""
import bcrypt
return bcrypt.hashpw(password.encode("utf-8"), bcrypt.gensalt()).decode("utf-8")
def verify_password(password: str, password_hash: str) -> bool:
"""Verify a password against a bcrypt hash."""
import bcrypt
return bcrypt.checkpw(password.encode("utf-8"), password_hash.encode("utf-8"))
def parse_scopes(scopes_str: str) -> frozenset[str]:
"""Parse comma-separated scopes and expand via hierarchy.
``"approve"`` expands to ``{"read", "write", "approve"}``.
"""
raw = {s.strip() for s in scopes_str.split(",") if s.strip()}
expanded: set[str] = set()
for scope in raw:
expanded |= SCOPE_HIERARCHY.get(scope, frozenset({scope}))
return frozenset(expanded & VALID_SCOPES)
# ---------------------------------------------------------------------------
# JWT helpers
# ---------------------------------------------------------------------------
def load_jwt_secret() -> str:
"""Load JWT signing secret from env or config, or auto-generate."""
secret = os.environ.get("TURNSTONE_JWT_SECRET", "").strip()
if not secret:
from turnstone.core.config import load_config
auth_cfg = load_config("auth")
secret = str(auth_cfg.get("jwt_secret", "")).strip()
if not secret:
# Auto-generate an ephemeral secret
secret = secrets.token_hex(32)
log.warning(
"No JWT secret configured — using ephemeral secret (tokens will not survive restart)"
)
return secret
if len(secret) < _MIN_SECRET_LENGTH:
log.warning(
"JWT secret is shorter than %d characters — consider using a stronger secret",
_MIN_SECRET_LENGTH,
)
return secret
def create_jwt(
user_id: str,
scopes: frozenset[str],
source: str,
secret: str,
expiry_hours: int = 24,
audience: str = "",
) -> str:
"""Create a signed JWT with user identity and scopes."""
import jwt
now = int(time.time())
payload: dict[str, Any] = {
"sub": user_id,
"scopes": ",".join(sorted(scopes)),
"src": source,
"iss": JWT_ISSUER,
"iat": now,
"exp": now + expiry_hours * 3600,
}
if audience:
payload["aud"] = audience
return jwt.encode(payload, secret, algorithm="HS256")
def validate_jwt(token: str, secret: str, audience: str = "") -> AuthResult | None:
"""Validate a JWT and return an AuthResult, or None on failure.
When *audience* is non-empty the ``aud`` claim is verified. Tokens
without an ``aud`` claim are accepted when *audience* is empty (backward
compatibility during the rollout window).
"""
import jwt
decode_opts: Any = None
if not audience:
decode_opts = {"verify_aud": False}
try:
payload = jwt.decode(
token,
secret,
algorithms=["HS256"],
audience=audience if audience else None,
options=decode_opts,
)
except jwt.InvalidTokenError:
return None
user_id = payload.get("sub", "")
scopes_str = payload.get("scopes", "")
source = payload.get("src", "jwt")
return AuthResult(
user_id=user_id,
scopes=parse_scopes(scopes_str),
token_source=source,
)
# ---------------------------------------------------------------------------
# Loading
# ---------------------------------------------------------------------------
@@ -83,24 +308,28 @@ class AuthConfig:
def load_auth_config() -> AuthConfig:
"""Build :class:`AuthConfig` from ``config.toml`` ``[auth]`` + env vars.
Auth is **enabled by default**. Set ``[auth] enabled = false`` or
``TURNSTONE_AUTH_ENABLED=0`` to disable.
Config format::
[auth]
enabled = true
enabled = false # opt out
[[auth.tokens]]
value = "tok_abc123"
role = "full"
Environment variable fallbacks:
Environment variables:
- ``TURNSTONE_AUTH_ENABLED=1`` enables auth
- ``TURNSTONE_AUTH_ENABLED=0`` disables auth
- ``TURNSTONE_AUTH_ENABLED=1`` enables auth (default)
- ``TURNSTONE_AUTH_TOKEN=<token>`` registers a single full-access token
"""
from turnstone.core.config import load_config
auth_cfg = load_config("auth")
enabled = bool(auth_cfg.get("enabled", False))
enabled = bool(auth_cfg.get("enabled", True))
tokens: dict[str, str] = {}
# Tokens from config file (TOML array-of-tables)
@@ -110,16 +339,19 @@ def load_auth_config() -> AuthConfig:
if value and role in ("read", "full"):
tokens[value] = role
# Environment variable fallbacks
if os.environ.get("TURNSTONE_AUTH_ENABLED", "").strip() in ("1", "true", "yes"):
# Environment variable overrides
env_enabled = os.environ.get("TURNSTONE_AUTH_ENABLED", "").strip().lower()
if env_enabled in ("1", "true", "yes"):
enabled = True
elif env_enabled in ("0", "false", "no"):
enabled = False
env_token = os.environ.get("TURNSTONE_AUTH_TOKEN", "").strip()
if env_token:
tokens[env_token] = "full"
if enabled and not tokens:
log.warning("Auth enabled but no tokens configured — all API requests will be rejected")
log.info("Auth enabled (no config tokens — use /api/auth/setup or turnstone-admin)")
return AuthConfig(enabled=enabled, tokens=tokens)
@@ -137,37 +369,55 @@ def is_public_path(path: str) -> bool:
return any(normalized.startswith(prefix) for prefix in PUBLIC_PREFIXES)
def required_role(method: str, path: str) -> str:
"""Return the minimum role needed for *method* + *path*.
def required_scope(method: str, path: str) -> str:
"""Return the minimum scope needed for *method* + *path*.
Returns ``"full"`` for state-modifying POST endpoints, ``"read"`` otherwise.
Handles console proxy routes (``/node/{id}/api/...``) by extracting the
proxied path and checking it against ``WRITE_PATHS``.
Returns ``"approve"`` for the approve endpoint and admin paths,
``"write"`` for other state-modifying POST endpoints, ``"read"`` otherwise.
"""
normalized = _strip_version_prefix(path)
normalized = normalized.rstrip("/") if normalized != "/" else normalized
# Admin endpoints require approve scope
if normalized.startswith(ADMIN_PREFIX):
return "approve"
# Approve endpoint
if method == "POST" and normalized in APPROVE_PATHS:
return "approve"
# Write endpoints
if method == "POST" and normalized in WRITE_PATHS:
return "full"
return "write"
# Console proxy routes: /node/{node_id}/api/{tail} or /node/{node_id}/v1/api/{tail}
if method == "POST" and normalized.startswith("/node/"):
parts = normalized.split("/", 4) # ['', 'node', '{id}', 'api'|'v1', ...]
if len(parts) >= 5:
if parts[3] == "api":
proxied_path = "/api/" + parts[4]
if proxied_path in WRITE_PATHS:
return "full"
elif parts[3] == "v1":
# /node/{id}/v1/api/{tail} — re-split the remainder
remainder = parts[4] # "api/send" etc.
if remainder.startswith("api/"):
proxied_path = "/api/" + remainder[4:]
if proxied_path in WRITE_PATHS:
return "full"
proxied = _extract_proxied_path(normalized)
if proxied:
if proxied in APPROVE_PATHS:
return "approve"
if proxied in WRITE_PATHS:
return "write"
return "read"
def _extract_proxied_path(normalized: str) -> str | None:
"""Extract the inner API path from a console proxy route."""
parts = normalized.split("/", 4) # ['', 'node', '{id}', 'api'|'v1', ...]
if len(parts) < 5:
return None
if parts[3] == "api":
return "/api/" + parts[4]
if parts[3] == "v1":
remainder = parts[4]
if remainder.startswith("api/"):
return "/api/" + remainder[4:]
return None
# ---------------------------------------------------------------------------
# Request checking — single entry point for HTTP handlers
# Request checking
# ---------------------------------------------------------------------------
@@ -177,36 +427,112 @@ def check_request(
path: str,
auth_header: str | None,
cookie_header: str | None = None,
) -> tuple[bool, int, str]:
*,
jwt_secret: str = "",
jwt_audience: str = "",
storage: Any = None,
) -> tuple[bool, int, str, AuthResult | None]:
"""Validate a request against the auth config.
Checks ``Authorization: Bearer <token>`` first, then falls back to the
``turnstone_auth`` cookie (set by ``/api/auth/login``).
``turnstone_auth`` cookie. Token types are auto-detected:
Returns ``(allowed, status_code, message)``.
On success: ``(True, 200, "")``.
On failure: ``(False, 401|403, "error message")``.
- Contains ``.`` JWT (validated with *jwt_secret*)
- Starts with ``ts_`` API token (looked up in *storage* by hash)
- Otherwise config-file token (hmac check)
Returns ``(allowed, status_code, message, auth_result)``.
"""
if not auth_config.enabled:
return True, 200, ""
return True, 200, "", None
if is_public_path(path):
return True, 200, ""
return True, 200, "", None
# Try Bearer header first, then cookie
token = _extract_bearer(auth_header)
if token is None:
token = _extract_cookie(cookie_header, AUTH_COOKIE)
# Extract token from header or cookie
raw_token = _extract_bearer(auth_header)
if raw_token is None:
raw_token = _extract_cookie(cookie_header, AUTH_COOKIE)
if not raw_token:
return False, 401, "Unauthorized: missing or invalid token", None
# Authenticate
result = _authenticate_token(
raw_token, auth_config, jwt_secret=jwt_secret, jwt_audience=jwt_audience, storage=storage
)
if result is None:
return False, 401, "Unauthorized: missing or invalid token", None
# Check scope
needed = required_scope(method, path)
if not result.has_scope(needed):
return False, 403, f"Forbidden: token lacks '{needed}' scope", None
return True, 200, "", result
def _authenticate_token(
token: str,
auth_config: AuthConfig,
*,
jwt_secret: str = "",
jwt_audience: str = "",
storage: Any = None,
) -> AuthResult | None:
"""Identify token type and authenticate it."""
# 1. JWT (contains dots) — attempt validation, fall through on failure
if "." in token and jwt_secret:
try:
jwt_result = validate_jwt(token, jwt_secret, audience=jwt_audience)
except Exception:
jwt_result = None
if jwt_result is not None:
return jwt_result
# 2. API token (starts with ts_) — look up in storage
if token.startswith(TOKEN_PREFIX) and storage is not None:
return _authenticate_api_token(token, storage)
# 3. Config-file token (hmac comparison)
role = auth_config.check(token)
if role is not None:
scopes = _ROLE_TO_SCOPES.get(role, frozenset({"read"}))
return AuthResult(user_id="", scopes=scopes, token_source="config")
if role is None:
return False, 401, "Unauthorized: missing or invalid token"
return None
needed = required_role(method, path)
if needed == "full" and role != "full":
return False, 403, "Forbidden: read-only token cannot access this endpoint"
return True, 200, ""
def _authenticate_api_token(token: str, storage: Any) -> AuthResult | None:
"""Validate an API token against the database."""
tok_hash = hash_token(token)
row = storage.get_api_token_by_hash(tok_hash)
if row is None:
return None
# Check expiry
expires = row.get("expires")
if expires:
from datetime import UTC, datetime
now = datetime.now(UTC)
try:
exp_dt = datetime.fromisoformat(expires).replace(tzinfo=UTC)
except (ValueError, TypeError):
return None # malformed expiry → treat as expired
if exp_dt < now:
return None
return AuthResult(
user_id=row["user_id"],
scopes=parse_scopes(row["scopes"]),
token_source="database",
)
# ---------------------------------------------------------------------------
# Token extraction helpers
# ---------------------------------------------------------------------------
def _extract_bearer(header: str | None) -> str | None:
@@ -220,10 +546,7 @@ def _extract_bearer(header: str | None) -> str | None:
def _extract_cookie(cookie_header: str | None, name: str) -> str | None:
"""Extract a named value from a ``Cookie`` header.
Assumes token values are simple ASCII (no URL-encoding).
"""
"""Extract a named value from a ``Cookie`` header."""
if not cookie_header:
return None
for pair in cookie_header.split(";"):
@@ -240,14 +563,15 @@ def _extract_cookie(cookie_header: str | None, name: str) -> str | None:
# ---------------------------------------------------------------------------
def make_set_cookie(token: str, max_age: int = 86400 * 30, secure: bool = False) -> str:
def make_set_cookie(token: str, max_age: int = 86400, *, secure: bool | None = None) -> str:
"""Return a ``Set-Cookie`` header value that stores the auth token.
Set *secure* to ``True`` when serving over HTTPS to add the ``Secure``
flag (prevents cookie from being sent over plain HTTP).
When *secure* is ``None`` (default) the ``Secure`` flag is set
unconditionally. Pass ``secure=False`` only for plaintext development.
*max_age* defaults to 24 hours to match the default JWT expiry.
"""
val = f"{AUTH_COOKIE}={token}; Path=/; HttpOnly; SameSite=Lax; Max-Age={max_age}"
if secure:
if secure is None or secure:
val += "; Secure"
return val
@@ -255,3 +579,401 @@ def make_set_cookie(token: str, max_age: int = 86400 * 30, secure: bool = False)
def make_clear_cookie() -> str:
"""Return a ``Set-Cookie`` header value that expires the auth cookie."""
return f"{AUTH_COOKIE}=; Path=/; HttpOnly; SameSite=Lax; Max-Age=0"
def is_secure_request(headers: dict[str, str], scheme: str = "") -> bool:
"""Return ``True`` if the request arrived over HTTPS.
Checks the URL scheme and the ``X-Forwarded-Proto`` header (set by
reverse proxies and load balancers).
"""
if scheme == "https":
return True
proto = headers.get("x-forwarded-proto", "")
return proto.lower() == "https"
# ---------------------------------------------------------------------------
# Login rate limiter
# ---------------------------------------------------------------------------
class LoginRateLimiter:
"""Sliding-window rate limiter for login attempts.
Tracks per-key (IP or username) attempt timestamps and rejects when
*max_attempts* are exceeded within *window_seconds*.
"""
MAX_KEYS: int = 50_000
def __init__(self, max_attempts: int = 5, window_seconds: int = 300) -> None:
self._max_attempts = max_attempts
self._window = window_seconds
self._attempts: dict[str, list[float]] = {}
self._lock = threading.Lock()
def check(self, key: str) -> tuple[bool, int]:
"""Return ``(allowed, retry_after_seconds)``.
Does **not** record a new attempt call :meth:`record` after a
failed login so successful logins don't consume the budget.
"""
now = time.monotonic()
with self._lock:
timestamps = self._attempts.get(key)
if timestamps is None:
return True, 0
# Prune expired
cutoff = now - self._window
timestamps[:] = [t for t in timestamps if t > cutoff]
if not timestamps:
del self._attempts[key]
return True, 0
if len(timestamps) >= self._max_attempts:
retry_after = int(timestamps[0] - cutoff) + 1
return False, max(retry_after, 1)
return True, 0
def record(self, key: str) -> None:
"""Record a failed login attempt."""
now = time.monotonic()
with self._lock:
if len(self._attempts) >= self.MAX_KEYS and key not in self._attempts:
return # prevent memory exhaustion
self._attempts.setdefault(key, []).append(now)
def cleanup(self, max_age: float = 600.0) -> int:
"""Remove stale entries older than *max_age* seconds."""
now = time.monotonic()
cutoff = now - max_age
with self._lock:
stale = [k for k, ts in self._attempts.items() if all(t <= cutoff for t in ts)]
for k in stale:
del self._attempts[k]
return len(stale)
# ---------------------------------------------------------------------------
# Service token manager (auto-rotating JWTs for service-to-service auth)
# ---------------------------------------------------------------------------
class ServiceTokenManager:
"""Auto-rotating service JWT. Thread-safe.
The :attr:`token` property returns a valid JWT, re-minting transparently
when the current token is within *refresh_margin* of expiry.
"""
def __init__(
self,
user_id: str,
scopes: frozenset[str],
source: str,
secret: str,
audience: str = "",
expiry_hours: int = 1,
refresh_margin: float = 0.2,
) -> None:
self._user_id = user_id
self._scopes = scopes
self._source = source
self._secret = secret
self._audience = audience
self._expiry_hours = expiry_hours
self._margin_seconds = expiry_hours * 3600 * refresh_margin
self._token: str = ""
self._expires_at: float = 0.0
self._lock = threading.Lock()
def _mint(self) -> None:
self._token = create_jwt(
user_id=self._user_id,
scopes=self._scopes,
source=self._source,
secret=self._secret,
expiry_hours=self._expiry_hours,
audience=self._audience,
)
self._expires_at = time.time() + self._expiry_hours * 3600
log.debug("Service JWT minted for %s (expires in %dh)", self._user_id, self._expiry_hours)
@property
def token(self) -> str:
"""Return current token, re-minting if near expiry."""
with self._lock:
if time.time() >= self._expires_at - self._margin_seconds:
self._mint()
return self._token
@property
def bearer_header(self) -> dict[str, str]:
"""Return an ``Authorization`` header dict with the current token."""
return {"Authorization": f"Bearer {self.token}"}
# ---------------------------------------------------------------------------
# Shared ASGI middleware
# ---------------------------------------------------------------------------
class AuthMiddleware:
"""ASGI middleware that enforces bearer-token / cookie authentication.
Parameterized by *jwt_audience* so the same class serves both the node
server (``JWT_AUD_SERVER``) and the console (``JWT_AUD_CONSOLE``).
"""
def __init__(self, app: ASGIApp, jwt_audience: str = "") -> None:
self.app = app
self._jwt_audience = jwt_audience
async def __call__(self, scope: Scope, receive: Receive, send: Send) -> None:
if scope["type"] != "http":
await self.app(scope, receive, send)
return
from starlette.requests import Request
from starlette.responses import JSONResponse
request = Request(scope)
# Skip auth for CORS preflight — CORSMiddleware handles it
if request.method == "OPTIONS":
await self.app(scope, receive, send)
return
auth_config = request.app.state.auth_config
jwt_secret = getattr(request.app.state, "jwt_secret", "")
storage = getattr(request.app.state, "auth_storage", None)
method = request.method
path = request.url.path
auth_header = request.headers.get("Authorization")
cookie_header = request.headers.get("Cookie")
allowed, status, msg, auth_result = check_request(
auth_config,
method,
path,
auth_header,
cookie_header,
jwt_secret=jwt_secret,
jwt_audience=self._jwt_audience,
storage=storage,
)
if not allowed:
response = JSONResponse({"error": msg}, status_code=status)
await response(scope, receive, send)
return
# Set user_id in log context and stash auth result for handlers
if auth_result and auth_result.user_id:
from turnstone.core.log import ctx_user_id
ctx_user_id.set(auth_result.user_id)
if "state" not in scope:
scope["state"] = {}
scope["state"]["auth_result"] = auth_result
await self.app(scope, receive, send)
# ---------------------------------------------------------------------------
# Shared auth endpoint handlers
# ---------------------------------------------------------------------------
async def handle_auth_login(request: Request, audience: str) -> Response:
"""Shared ``POST /api/auth/login`` handler.
Authenticates via username:password or legacy token exchange, returning
a JWT and setting the auth cookie. *audience* selects the JWT ``aud``
claim (``JWT_AUD_SERVER`` or ``JWT_AUD_CONSOLE``).
"""
from starlette.responses import JSONResponse
try:
body: dict[str, Any] = await request.json()
except (ValueError, json.JSONDecodeError):
return JSONResponse({"error": "Invalid JSON body"}, status_code=400)
auth_config = request.app.state.auth_config
jwt_secret = getattr(request.app.state, "jwt_secret", "")
storage = getattr(request.app.state, "auth_storage", None)
login_limiter: LoginRateLimiter | None = getattr(request.app.state, "login_limiter", None)
username = body.get("username", "")
client_ip = request.client.host if request.client else "unknown"
# Check login rate limits (per-IP and per-username)
if login_limiter is not None:
ip_ok, ip_retry = login_limiter.check(f"ip:{client_ip}")
if not ip_ok:
return JSONResponse(
{"error": "Too many login attempts"},
status_code=429,
headers={"Retry-After": str(ip_retry)},
)
if username:
user_ok, user_retry = login_limiter.check(f"user:{username}")
if not user_ok:
return JSONResponse(
{"error": "Too many login attempts"},
status_code=429,
headers={"Retry-After": str(user_retry)},
)
result: AuthResult | None = None
password = body.get("password", "")
if username and password and storage is not None:
user = storage.get_user_by_username(username)
if user and verify_password(password, user["password_hash"]):
result = AuthResult(
user_id=user["user_id"],
scopes=frozenset({"read", "write", "approve"}),
token_source="password",
)
elif body.get("token"):
result = _authenticate_token(
body["token"],
auth_config,
jwt_secret=jwt_secret,
jwt_audience=audience,
storage=storage,
)
if result is None:
# Record failed attempt for rate limiting
if login_limiter is not None:
login_limiter.record(f"ip:{client_ip}")
if username:
login_limiter.record(f"user:{username}")
return JSONResponse({"error": "Invalid credentials"}, status_code=401)
jwt_token = ""
if jwt_secret:
jwt_token = create_jwt(
user_id=result.user_id,
scopes=result.scopes,
source=result.token_source,
secret=jwt_secret,
audience=audience,
)
role = "full" if result.has_scope("write") else "read"
scopes_str = ",".join(sorted(result.scopes))
resp_body: dict[str, str] = {"status": "ok", "role": role, "scopes": scopes_str}
if jwt_token:
resp_body["jwt"] = jwt_token
if result.user_id:
resp_body["user_id"] = result.user_id
secure = is_secure_request(dict(request.headers), request.url.scheme)
response = JSONResponse(resp_body)
cookie_value = jwt_token if jwt_token else body.get("token", "")
if cookie_value:
response.headers["Set-Cookie"] = make_set_cookie(cookie_value, secure=secure)
return response
async def handle_auth_logout(request: Request) -> Response:
"""Shared ``POST /api/auth/logout`` handler — clear auth cookie."""
from starlette.responses import JSONResponse
response = JSONResponse({"status": "ok"})
response.headers["Set-Cookie"] = make_clear_cookie()
return response
async def handle_auth_status(request: Request) -> Response:
"""Shared ``GET /api/auth/status`` handler — login UI state detection."""
from starlette.responses import JSONResponse
auth_config = request.app.state.auth_config
storage = getattr(request.app.state, "auth_storage", None)
has_users = False
if storage is not None:
try:
users = storage.list_users()
has_users = len(users) > 0
except Exception:
pass
return JSONResponse(
{
"auth_enabled": auth_config.enabled,
"has_users": has_users,
"setup_required": auth_config.enabled and not has_users,
}
)
async def handle_auth_setup(request: Request, audience: str) -> Response:
"""Shared ``POST /api/auth/setup`` handler — create first admin user.
Only works when zero users exist. Returns JWT on success.
"""
from starlette.responses import JSONResponse
storage = getattr(request.app.state, "auth_storage", None)
jwt_secret = getattr(request.app.state, "jwt_secret", "")
if storage is None:
return JSONResponse({"error": "Storage not available"}, status_code=503)
try:
body: dict[str, Any] = await request.json()
except (ValueError, json.JSONDecodeError):
return JSONResponse({"error": "Invalid JSON body"}, status_code=400)
username = body.get("username", "").strip()
display_name = body.get("display_name", "").strip()
password = body.get("password", "")
if not is_valid_username(username):
return JSONResponse(
{"error": "Invalid username (1-64 chars: letters, digits, . _ -)"},
status_code=400,
)
if not display_name:
return JSONResponse({"error": "display_name is required"}, status_code=400)
if len(password) < 8:
return JSONResponse({"error": "Password must be at least 8 characters"}, status_code=400)
user_id = uuid.uuid4().hex
pw_hash = hash_password(password)
# Atomic: insert only if no users exist (prevents TOCTOU race)
try:
created = storage.create_first_user(user_id, username, display_name, pw_hash)
except Exception:
return JSONResponse({"error": "Storage error"}, status_code=503)
if not created:
return JSONResponse({"error": "Setup already completed"}, status_code=409)
scopes = frozenset({"read", "write", "approve"})
jwt_token = ""
if jwt_secret:
jwt_token = create_jwt(
user_id=user_id,
scopes=scopes,
source="password",
secret=jwt_secret,
audience=audience,
)
resp_body: dict[str, str] = {
"status": "ok",
"user_id": user_id,
"username": username,
"role": "full",
"scopes": ",".join(sorted(scopes)),
}
if jwt_token:
resp_body["jwt"] = jwt_token
secure = is_secure_request(dict(request.headers), request.url.scheme)
response = JSONResponse(resp_body)
if jwt_token:
response.headers["Set-Cookie"] = make_set_cookie(jwt_token, secure=secure)
return response
+185
View File
@@ -0,0 +1,185 @@
"""Structured logging configuration for all Turnstone services."""
from __future__ import annotations
import logging
import os
import sys
from contextvars import ContextVar
from typing import Any
import structlog
# ---------------------------------------------------------------------------
# Context variables — set in request handlers, workstream managers, etc.
# Any non-empty value is automatically injected into every log event.
# ---------------------------------------------------------------------------
ctx_node_id: ContextVar[str] = ContextVar("node_id", default="")
ctx_ws_id: ContextVar[str] = ContextVar("ws_id", default="")
ctx_user_id: ContextVar[str] = ContextVar("user_id", default="")
ctx_request_id: ContextVar[str] = ContextVar("request_id", default="")
_CONTEXT_VARS: list[tuple[ContextVar[str], str]] = [
(ctx_node_id, "node_id"),
(ctx_ws_id, "ws_id"),
(ctx_user_id, "user_id"),
(ctx_request_id, "request_id"),
]
# Third-party loggers that are noisy at INFO level.
_QUIET_LOGGERS = ("httpx", "httpcore", "openai", "anthropic", "uvicorn.access")
# ---------------------------------------------------------------------------
# Processors
# ---------------------------------------------------------------------------
def _inject_context(_logger: Any, _method: str, event_dict: dict[str, Any]) -> dict[str, Any]:
"""Add non-empty context variables to every log event."""
for var, key in _CONTEXT_VARS:
val = var.get("")
if val:
event_dict[key] = val
return event_dict
def _add_service(service: str) -> structlog.types.Processor:
"""Return a processor that stamps *service* onto every event."""
def _processor(_logger: Any, _method: str, event_dict: dict[str, Any]) -> dict[str, Any]:
event_dict["service"] = service
return event_dict
return _processor # type: ignore[return-value]
# ---------------------------------------------------------------------------
# Public API
# ---------------------------------------------------------------------------
def configure_logging(
level: str = "INFO",
*,
json_output: bool | None = None,
service: str = "",
) -> None:
"""Configure structured logging for a Turnstone service.
Call this once, early in each entry-point's ``main()``.
Parameters
----------
level:
Log level name (``DEBUG``, ``INFO``, ``WARNING``, ``ERROR``,
``CRITICAL``). The ``TURNSTONE_LOG_LEVEL`` env var, if set,
overrides this.
json_output:
Force JSON (``True``) or console (``False``) output. ``None``
auto-detects: JSON when stderr is not a TTY. The
``TURNSTONE_LOG_FORMAT`` env var (``json`` / ``text``) overrides.
service:
Service name added to every log line (e.g. ``"server"``).
"""
# Env-var overrides -------------------------------------------------------
env_level = os.environ.get("TURNSTONE_LOG_LEVEL", "").upper()
if env_level:
level = env_level
env_fmt = os.environ.get("TURNSTONE_LOG_FORMAT", "").lower()
if env_fmt in ("json", "text"):
json_output = env_fmt == "json"
elif json_output is None:
json_output = not sys.stderr.isatty()
# Shared processor chain --------------------------------------------------
processors: list[structlog.types.Processor] = [
structlog.contextvars.merge_contextvars,
_inject_context, # type: ignore[list-item]
structlog.stdlib.add_log_level,
structlog.stdlib.add_logger_name,
structlog.processors.TimeStamper(fmt="iso"),
structlog.processors.StackInfoRenderer(),
structlog.processors.format_exc_info,
structlog.processors.UnicodeDecoder(),
]
if service:
processors.append(_add_service(service))
# Renderer ----------------------------------------------------------------
if json_output:
renderer: structlog.types.Processor = structlog.processors.JSONRenderer()
else:
renderer = structlog.dev.ConsoleRenderer(colors=sys.stderr.isatty())
# structlog config (for structlog.get_logger()) ---------------------------
structlog.configure(
processors=[
*processors,
structlog.stdlib.ProcessorFormatter.wrap_for_formatter,
],
logger_factory=structlog.stdlib.LoggerFactory(),
wrapper_class=structlog.stdlib.BoundLogger,
cache_logger_on_first_use=True,
)
# stdlib handler (for logging.getLogger()) --------------------------------
# foreign_pre_chain runs on events from stdlib loggers (not structlog).
formatter = structlog.stdlib.ProcessorFormatter(
foreign_pre_chain=processors,
processors=[
structlog.stdlib.ProcessorFormatter.remove_processors_meta,
renderer,
],
)
handler = logging.StreamHandler(sys.stderr)
handler.setFormatter(formatter)
root = logging.getLogger()
root.handlers.clear()
root.addHandler(handler)
root.setLevel(getattr(logging, level.upper(), logging.INFO))
# Quiet noisy third-party loggers -----------------------------------------
for name in _QUIET_LOGGERS:
logging.getLogger(name).setLevel(logging.WARNING)
def get_logger(name: str) -> structlog.stdlib.BoundLogger:
"""Return a structlog bound logger backed by the stdlib."""
result: structlog.stdlib.BoundLogger = structlog.get_logger(name)
return result
# ---------------------------------------------------------------------------
# CLI helpers
# ---------------------------------------------------------------------------
def add_log_args(parser: Any) -> None:
"""Add ``--log-level`` and ``--log-format`` arguments to *parser*."""
parser.add_argument(
"--log-level",
default="INFO",
choices=["DEBUG", "INFO", "WARNING", "ERROR"],
help="Log level (default: %(default)s)",
)
parser.add_argument(
"--log-format",
default="auto",
choices=["auto", "json", "text"],
help="Log output format (default: auto — JSON when stderr is not a TTY)",
)
def configure_logging_from_args(args: Any, service: str) -> None:
"""Call :func:`configure_logging` using parsed CLI arguments."""
configure_logging(
level=args.log_level,
json_output={"json": True, "text": False}.get(args.log_format),
service=service,
)
+38 -2
View File
@@ -24,10 +24,15 @@ def normalize_key(key: str) -> str:
# -- Core session operations ---------------------------------------------------
def register_session(session_id: str, title: str | None = None) -> None:
def register_session(
session_id: str,
title: str | None = None,
node_id: str | None = None,
ws_id: str | None = None,
) -> None:
"""Create a sessions row for a new session (no-op if already exists)."""
with contextlib.suppress(Exception):
get_storage().register_session(session_id, title)
get_storage().register_session(session_id, title, node_id=node_id, ws_id=ws_id)
def save_message(
@@ -146,6 +151,37 @@ def update_session_title(session_id: str, title: str) -> None:
get_storage().update_session_title(session_id, title)
# -- Workstream operations -----------------------------------------------------
def register_workstream(
ws_id: str, node_id: str | None = None, name: str = "", state: str = "idle"
) -> None:
"""Persist a new workstream (no-op if already exists)."""
with contextlib.suppress(Exception):
get_storage().register_workstream(ws_id, node_id, name, state)
def update_workstream_state(ws_id: str, state: str) -> None:
"""Update a workstream's state."""
with contextlib.suppress(Exception):
get_storage().update_workstream_state(ws_id, state)
def update_workstream_name(ws_id: str, name: str) -> None:
"""Update a workstream's display name."""
with contextlib.suppress(Exception):
get_storage().update_workstream_name(ws_id, name)
def list_workstreams(node_id: str | None = None, limit: int = 100) -> list[Any]:
"""List workstreams, optionally filtered by node_id."""
try:
return get_storage().list_workstreams(node_id, limit)
except Exception:
return []
# -- Key-value store (memories) ------------------------------------------------
+9 -5
View File
@@ -116,6 +116,8 @@ class ChatSession:
registry: ModelRegistry | None = None,
model_alias: str | None = None,
health_monitor: BackendHealthMonitor | None = None,
node_id: str | None = None,
ws_id: str | None = None,
):
self.client = client
self.model = model
@@ -148,9 +150,11 @@ class ChatSession:
self.show_reasoning = True
self.debug = False
self.auto_approve = False
self._session_id = uuid.uuid4().hex[:12]
self._node_id = node_id
self._ws_id = ws_id
self._session_id = uuid.uuid4().hex
self._title_generated = False
register_session(self._session_id)
register_session(self._session_id, node_id=self._node_id, ws_id=self._ws_id)
self._read_files: set[str] = set()
self.messages: list[dict[str, Any]] = []
self._last_usage: dict[str, int] | None = None
@@ -2678,9 +2682,9 @@ class ChatSession:
self._read_files.clear()
self._last_usage = None
self._msg_tokens = []
self._session_id = uuid.uuid4().hex[:12]
self._session_id = uuid.uuid4().hex
self._title_generated = False
register_session(self._session_id)
register_session(self._session_id, node_id=self._node_id, ws_id=self._ws_id)
self._save_config()
self.ui.on_info("New session started.")
@@ -2690,7 +2694,7 @@ class ChatSession:
self.ui.on_info("No saved sessions.")
else:
lines = ["Sessions:\n"]
for sid, alias, title, _created, updated, count in rows:
for sid, alias, title, _created, updated, count, *_extra in rows:
display_name = alias or sid
display_title = f" {title}" if title else ""
marker = " *" if sid == self._session_id else " "
+3 -1
View File
@@ -78,7 +78,9 @@ if __name__ == "__main__":
url = os.environ.get("TURNSTONE_DB_URL", "")
path = os.environ.get("TURNSTONE_DB_PATH", "")
logging.basicConfig(level=logging.INFO)
from turnstone.core.log import configure_logging
configure_logging(level="INFO", service="migrate")
init_storage(backend, path=path, url=url, run_migrations=True)
log.info("Migrations complete")
get_storage().close()
+502 -3
View File
@@ -9,11 +9,14 @@ from typing import Any
import sqlalchemy as sa
from turnstone.core.storage._schema import (
api_tokens,
conversations,
memories,
metadata,
session_config,
sessions,
users,
workstreams,
)
from turnstone.core.storage._sqlite import _reconstruct_messages
@@ -37,7 +40,14 @@ class PostgreSQLBackend:
# -- Core session operations -----------------------------------------------
def register_session(self, session_id: str, title: str | None = None) -> None:
def register_session(
self,
session_id: str,
title: str | None = None,
node_id: str | None = None,
ws_id: str | None = None,
user_id: str | None = None,
) -> None:
now = datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%S")
with self._engine.connect() as conn:
# Use dialect-neutral upsert pattern
@@ -47,7 +57,15 @@ class PostgreSQLBackend:
if not existing:
conn.execute(
sa.insert(sessions),
{"session_id": session_id, "title": title, "created": now, "updated": now},
{
"session_id": session_id,
"title": title,
"node_id": node_id,
"ws_id": ws_id,
"user_id": user_id,
"created": now,
"updated": now,
},
)
conn.commit()
@@ -106,7 +124,8 @@ class PostgreSQLBackend:
sa.text(
"SELECT s.session_id, s.alias, s.title, s.created, s.updated, "
"(SELECT COUNT(*) FROM conversations c "
" WHERE c.session_id = s.session_id) "
" WHERE c.session_id = s.session_id), "
"s.node_id, s.ws_id "
"FROM sessions s "
"WHERE EXISTS "
" (SELECT 1 FROM conversations c WHERE c.session_id = s.session_id) "
@@ -331,6 +350,80 @@ class PostgreSQLBackend:
).fetchall()
return [(str(r[0]), str(r[1])) for r in rows]
# -- Workstream operations -------------------------------------------------
def register_workstream(
self,
ws_id: str,
node_id: str | None = None,
name: str = "",
state: str = "idle",
user_id: str | None = None,
) -> None:
now = datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%S")
with self._engine.connect() as conn:
existing = conn.execute(
sa.select(workstreams.c.ws_id).where(workstreams.c.ws_id == ws_id)
).fetchone()
if not existing:
conn.execute(
sa.insert(workstreams),
{
"ws_id": ws_id,
"node_id": node_id,
"user_id": user_id,
"name": name,
"state": state,
"created": now,
"updated": now,
},
)
conn.commit()
def update_workstream_state(self, ws_id: str, state: str) -> None:
now = datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%S")
with self._engine.connect() as conn:
conn.execute(
sa.update(workstreams)
.where(workstreams.c.ws_id == ws_id)
.values(state=state, updated=now)
)
conn.commit()
def update_workstream_name(self, ws_id: str, name: str) -> None:
now = datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%S")
with self._engine.connect() as conn:
conn.execute(
sa.update(workstreams)
.where(workstreams.c.ws_id == ws_id)
.values(name=name, updated=now)
)
conn.commit()
def delete_workstream(self, ws_id: str) -> bool:
with self._engine.connect() as conn:
result = conn.execute(sa.delete(workstreams).where(workstreams.c.ws_id == ws_id))
conn.commit()
return result.rowcount > 0
def list_workstreams(self, node_id: str | None = None, limit: int = 100) -> list[Any]:
with self._engine.connect() as conn:
q = (
sa.select(
workstreams.c.ws_id,
workstreams.c.node_id,
workstreams.c.name,
workstreams.c.state,
workstreams.c.created,
workstreams.c.updated,
)
.order_by(workstreams.c.updated.desc())
.limit(limit)
)
if node_id is not None:
q = q.where(workstreams.c.node_id == node_id)
return list(conn.execute(q).fetchall())
# -- Conversation search ---------------------------------------------------
def search_history(self, query: str, limit: int = 20) -> list[Any]:
@@ -380,6 +473,412 @@ class PostgreSQLBackend:
).fetchall()
)
# -- Session lookup by workstream ------------------------------------------
def get_session_id_by_ws(self, ws_id: str) -> str | None:
with self._engine.connect() as conn:
row = conn.execute(
sa.select(sessions.c.session_id).where(sessions.c.ws_id == ws_id)
).fetchone()
return str(row[0]) if row else None
# -- User identity operations -----------------------------------------------
def create_user(
self, user_id: str, username: str, display_name: str, password_hash: str
) -> None:
now = datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%S")
with self._engine.connect() as conn:
existing = conn.execute(
sa.select(users.c.user_id).where(users.c.user_id == user_id)
).fetchone()
if not existing:
conn.execute(
sa.insert(users),
{
"user_id": user_id,
"username": username,
"display_name": display_name,
"password_hash": password_hash,
"created": now,
},
)
conn.commit()
def create_first_user(
self, user_id: str, username: str, display_name: str, password_hash: str
) -> bool:
"""Atomically create a user only if no users exist. Returns True if created."""
now = datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%S")
with self._engine.connect() as conn:
result = conn.execute(
sa.text(
"INSERT INTO users (user_id, username, display_name, password_hash, created) "
"SELECT :user_id, :username, :display_name, :password_hash, :created "
"WHERE NOT EXISTS (SELECT 1 FROM users)"
),
{
"user_id": user_id,
"username": username,
"display_name": display_name,
"password_hash": password_hash,
"created": now,
},
)
conn.commit()
return result.rowcount > 0
def get_user(self, user_id: str) -> dict[str, str] | None:
with self._engine.connect() as conn:
row = conn.execute(
sa.select(
users.c.user_id,
users.c.username,
users.c.display_name,
users.c.password_hash,
users.c.created,
).where(users.c.user_id == user_id)
).fetchone()
if row:
return {
"user_id": row[0],
"username": row[1],
"display_name": row[2],
"password_hash": row[3],
"created": row[4],
}
return None
def get_user_by_username(self, username: str) -> dict[str, str] | None:
with self._engine.connect() as conn:
row = conn.execute(
sa.select(
users.c.user_id,
users.c.username,
users.c.display_name,
users.c.password_hash,
users.c.created,
).where(users.c.username == username)
).fetchone()
if row:
return {
"user_id": row[0],
"username": row[1],
"display_name": row[2],
"password_hash": row[3],
"created": row[4],
}
return None
def list_users(self) -> list[dict[str, str]]:
with self._engine.connect() as conn:
rows = conn.execute(
sa.select(
users.c.user_id,
users.c.username,
users.c.display_name,
users.c.created,
).order_by(users.c.created.desc())
).fetchall()
return [
{"user_id": r[0], "username": r[1], "display_name": r[2], "created": r[3]}
for r in rows
]
def delete_user(self, user_id: str) -> bool:
from turnstone.core.storage._schema import channel_users
with self._engine.connect() as conn:
conn.execute(sa.delete(channel_users).where(channel_users.c.user_id == user_id))
conn.execute(sa.delete(api_tokens).where(api_tokens.c.user_id == user_id))
result = conn.execute(sa.delete(users).where(users.c.user_id == user_id))
conn.commit()
return result.rowcount > 0
def create_api_token(
self,
token_id: str,
token_hash: str,
token_prefix: str,
user_id: str,
name: str,
scopes: str,
expires: str | None = None,
) -> None:
now = datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%S")
with self._engine.connect() as conn:
conn.execute(
sa.insert(api_tokens),
{
"token_id": token_id,
"token_hash": token_hash,
"token_prefix": token_prefix,
"user_id": user_id,
"name": name,
"scopes": scopes,
"created": now,
"expires": expires,
},
)
conn.commit()
def get_api_token_by_hash(self, token_hash: str) -> dict[str, str] | None:
with self._engine.connect() as conn:
row = conn.execute(
sa.select(
api_tokens.c.token_id,
api_tokens.c.token_prefix,
api_tokens.c.user_id,
api_tokens.c.name,
api_tokens.c.scopes,
api_tokens.c.created,
api_tokens.c.expires,
).where(api_tokens.c.token_hash == token_hash)
).fetchone()
if row:
result: dict[str, str] = {
"token_id": row[0],
"token_prefix": row[1],
"user_id": row[2],
"name": row[3],
"scopes": row[4],
"created": row[5],
}
if row[6] is not None:
result["expires"] = row[6]
return result
return None
def list_api_tokens(self, user_id: str) -> list[dict[str, str]]:
with self._engine.connect() as conn:
rows = conn.execute(
sa.select(
api_tokens.c.token_id,
api_tokens.c.token_prefix,
api_tokens.c.user_id,
api_tokens.c.name,
api_tokens.c.scopes,
api_tokens.c.created,
api_tokens.c.expires,
)
.where(api_tokens.c.user_id == user_id)
.order_by(api_tokens.c.created.desc())
).fetchall()
result = []
for r in rows:
entry: dict[str, str] = {
"token_id": r[0],
"token_prefix": r[1],
"user_id": r[2],
"name": r[3],
"scopes": r[4],
"created": r[5],
}
if r[6] is not None:
entry["expires"] = r[6]
result.append(entry)
return result
def delete_api_token(self, token_id: str) -> bool:
with self._engine.connect() as conn:
result = conn.execute(sa.delete(api_tokens).where(api_tokens.c.token_id == token_id))
conn.commit()
return result.rowcount > 0
# -- Channel user mapping ---------------------------------------------------
def create_channel_user(self, channel_type: str, channel_user_id: str, user_id: str) -> None:
from sqlalchemy.dialects import postgresql
from turnstone.core.storage._schema import channel_users
now = datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%S")
with self._engine.connect() as conn:
conn.execute(
postgresql.insert(channel_users)
.values(
channel_type=channel_type,
channel_user_id=channel_user_id,
user_id=user_id,
created=now,
)
.on_conflict_do_nothing()
)
conn.commit()
def get_channel_user(self, channel_type: str, channel_user_id: str) -> dict[str, str] | None:
from turnstone.core.storage._schema import channel_users
with self._engine.connect() as conn:
row = conn.execute(
sa.select(
channel_users.c.channel_type,
channel_users.c.channel_user_id,
channel_users.c.user_id,
channel_users.c.created,
).where(
(channel_users.c.channel_type == channel_type)
& (channel_users.c.channel_user_id == channel_user_id)
)
).fetchone()
if row:
return {
"channel_type": row[0],
"channel_user_id": row[1],
"user_id": row[2],
"created": row[3],
}
return None
def list_channel_users_by_user(self, user_id: str) -> list[dict[str, str]]:
from turnstone.core.storage._schema import channel_users
with self._engine.connect() as conn:
rows = conn.execute(
sa.select(
channel_users.c.channel_type,
channel_users.c.channel_user_id,
channel_users.c.user_id,
channel_users.c.created,
)
.where(channel_users.c.user_id == user_id)
.order_by(channel_users.c.created.desc())
).fetchall()
return [
{
"channel_type": r[0],
"channel_user_id": r[1],
"user_id": r[2],
"created": r[3],
}
for r in rows
]
def delete_channel_user(self, channel_type: str, channel_user_id: str) -> bool:
from turnstone.core.storage._schema import channel_users
with self._engine.connect() as conn:
result = conn.execute(
sa.delete(channel_users).where(
(channel_users.c.channel_type == channel_type)
& (channel_users.c.channel_user_id == channel_user_id)
)
)
conn.commit()
return result.rowcount > 0
# -- Channel routing -------------------------------------------------------
def create_channel_route(
self, channel_type: str, channel_id: str, ws_id: str, node_id: str = ""
) -> None:
from sqlalchemy.dialects import postgresql
from turnstone.core.storage._schema import channel_routes
now = datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%S")
with self._engine.connect() as conn:
conn.execute(
postgresql.insert(channel_routes)
.values(
channel_type=channel_type,
channel_id=channel_id,
ws_id=ws_id,
node_id=node_id,
created=now,
)
.on_conflict_do_nothing()
)
conn.commit()
def get_channel_route(self, channel_type: str, channel_id: str) -> dict[str, str] | None:
from turnstone.core.storage._schema import channel_routes
with self._engine.connect() as conn:
row = conn.execute(
sa.select(
channel_routes.c.channel_type,
channel_routes.c.channel_id,
channel_routes.c.ws_id,
channel_routes.c.node_id,
channel_routes.c.created,
).where(
(channel_routes.c.channel_type == channel_type)
& (channel_routes.c.channel_id == channel_id)
)
).fetchone()
if row:
return {
"channel_type": row[0],
"channel_id": row[1],
"ws_id": row[2],
"node_id": row[3],
"created": row[4],
}
return None
def get_channel_route_by_ws(self, ws_id: str) -> dict[str, str] | None:
from turnstone.core.storage._schema import channel_routes
with self._engine.connect() as conn:
row = conn.execute(
sa.select(
channel_routes.c.channel_type,
channel_routes.c.channel_id,
channel_routes.c.ws_id,
channel_routes.c.node_id,
channel_routes.c.created,
).where(channel_routes.c.ws_id == ws_id)
).fetchone()
if row:
return {
"channel_type": row[0],
"channel_id": row[1],
"ws_id": row[2],
"node_id": row[3],
"created": row[4],
}
return None
def list_channel_routes_by_type(self, channel_type: str) -> list[dict[str, str]]:
from turnstone.core.storage._schema import channel_routes
with self._engine.connect() as conn:
rows = conn.execute(
sa.select(
channel_routes.c.channel_type,
channel_routes.c.channel_id,
channel_routes.c.ws_id,
channel_routes.c.node_id,
channel_routes.c.created,
)
.where(channel_routes.c.channel_type == channel_type)
.order_by(channel_routes.c.created.desc())
).fetchall()
return [
{
"channel_type": r[0],
"channel_id": r[1],
"ws_id": r[2],
"node_id": r[3],
"created": r[4],
}
for r in rows
]
def delete_channel_route(self, channel_type: str, channel_id: str) -> bool:
from turnstone.core.storage._schema import channel_routes
with self._engine.connect() as conn:
result = conn.execute(
sa.delete(channel_routes).where(
(channel_routes.c.channel_type == channel_type)
& (channel_routes.c.channel_id == channel_id)
)
)
conn.commit()
return result.rowcount > 0
# -- Lifecycle -------------------------------------------------------------
def close(self) -> None:
+140 -1
View File
@@ -15,7 +15,14 @@ class StorageBackend(Protocol):
# -- Core session operations -----------------------------------------------
def register_session(self, session_id: str, title: str | None = None) -> None:
def register_session(
self,
session_id: str,
title: str | None = None,
node_id: str | None = None,
ws_id: str | None = None,
user_id: str | None = None,
) -> None:
"""Create a sessions row for a new session (no-op if already exists)."""
...
@@ -100,6 +107,35 @@ class StorageBackend(Protocol):
"""Search key-value pairs by query. Returns matching (key, value) pairs."""
...
# -- Workstream operations -------------------------------------------------
def register_workstream(
self,
ws_id: str,
node_id: str | None = None,
name: str = "",
state: str = "idle",
user_id: str | None = None,
) -> None:
"""Create a workstreams row (no-op if already exists)."""
...
def update_workstream_state(self, ws_id: str, state: str) -> None:
"""Update a workstream's state and bump updated timestamp."""
...
def update_workstream_name(self, ws_id: str, name: str) -> None:
"""Update a workstream's display name."""
...
def delete_workstream(self, ws_id: str) -> bool:
"""Delete a workstream. Returns True on success."""
...
def list_workstreams(self, node_id: str | None = None, limit: int = 100) -> list[Any]:
"""List workstreams, optionally filtered by node_id."""
...
# -- Conversation search ---------------------------------------------------
def search_history(self, query: str, limit: int = 20) -> list[Any]:
@@ -110,6 +146,109 @@ class StorageBackend(Protocol):
"""Return most recent conversation messages."""
...
# -- User identity operations -----------------------------------------------
def create_user(
self, user_id: str, username: str, display_name: str, password_hash: str
) -> None:
"""Create a user row. No-op if user_id already exists."""
...
def create_first_user(
self, user_id: str, username: str, display_name: str, password_hash: str
) -> bool:
"""Atomically create a user only if no users exist. Returns True if created."""
...
def get_user(self, user_id: str) -> dict[str, str] | None:
"""Return user dict {user_id, username, display_name, password_hash, created} or None."""
...
def get_user_by_username(self, username: str) -> dict[str, str] | None:
"""Lookup user by username. Returns same dict as get_user or None."""
...
def list_users(self) -> list[dict[str, str]]:
"""Return all users ordered by created DESC."""
...
def delete_user(self, user_id: str) -> bool:
"""Delete user and cascade-delete all their tokens. Returns True if existed."""
...
def create_api_token(
self,
token_id: str,
token_hash: str,
token_prefix: str,
user_id: str,
name: str,
scopes: str,
expires: str | None = None,
) -> None:
"""Store a hashed API token."""
...
def get_api_token_by_hash(self, token_hash: str) -> dict[str, str] | None:
"""Lookup token by SHA-256 hash. Returns dict with all columns or None."""
...
def list_api_tokens(self, user_id: str) -> list[dict[str, str]]:
"""List tokens for a user (no hash in results, prefix only)."""
...
def delete_api_token(self, token_id: str) -> bool:
"""Revoke/delete a token by ID. Returns True if existed."""
...
# -- Channel user mapping ---------------------------------------------------
def create_channel_user(self, channel_type: str, channel_user_id: str, user_id: str) -> None:
"""Map an external channel user to a turnstone user_id. No-op if exists."""
...
def get_channel_user(self, channel_type: str, channel_user_id: str) -> dict[str, str] | None:
"""Lookup turnstone user for a channel user. Returns dict or None."""
...
def list_channel_users_by_user(self, user_id: str) -> list[dict[str, str]]:
"""List all channel mappings for a turnstone user."""
...
def delete_channel_user(self, channel_type: str, channel_user_id: str) -> bool:
"""Remove a channel user mapping. Returns True if existed."""
...
# -- Session lookup by workstream ------------------------------------------
def get_session_id_by_ws(self, ws_id: str) -> str | None:
"""Find the session_id associated with a workstream. Returns None if not found."""
...
# -- Channel routing -------------------------------------------------------
def create_channel_route(
self, channel_type: str, channel_id: str, ws_id: str, node_id: str = ""
) -> None:
"""Map a channel/thread to a workstream. No-op if exists."""
...
def get_channel_route(self, channel_type: str, channel_id: str) -> dict[str, str] | None:
"""Lookup workstream for a channel/thread."""
...
def get_channel_route_by_ws(self, ws_id: str) -> dict[str, str] | None:
"""Reverse lookup: find channel/thread for a workstream."""
...
def list_channel_routes_by_type(self, channel_type: str) -> list[dict[str, str]]:
"""List all routes for a channel type, ordered by created DESC."""
...
def delete_channel_route(self, channel_type: str, channel_id: str) -> bool:
"""Remove a channel route. Returns True if existed."""
...
# -- Lifecycle -------------------------------------------------------------
def close(self) -> None:
+83
View File
@@ -38,6 +38,9 @@ sessions = sa.Table(
sa.Column("session_id", sa.Text, primary_key=True),
sa.Column("alias", sa.Text, unique=True),
sa.Column("title", sa.Text),
sa.Column("node_id", sa.Text),
sa.Column("ws_id", sa.Text),
sa.Column("user_id", sa.Text),
sa.Column("created", sa.Text, nullable=False),
sa.Column("updated", sa.Text, nullable=False),
)
@@ -45,6 +48,25 @@ sessions = sa.Table(
# Additional indexes on sessions (name-based to avoid duplication with SA's auto-index)
sa.Index("idx_sessions_alias", sessions.c.alias)
sa.Index("idx_sessions_updated", sessions.c.updated)
sa.Index("idx_sessions_node_id", sessions.c.node_id)
sa.Index("idx_sessions_ws_id", sessions.c.ws_id)
sa.Index("idx_sessions_user_id", sessions.c.user_id)
workstreams = sa.Table(
"workstreams",
metadata,
sa.Column("ws_id", sa.Text, primary_key=True),
sa.Column("node_id", sa.Text),
sa.Column("user_id", sa.Text),
sa.Column("name", sa.Text, nullable=False, server_default=""),
sa.Column("state", sa.Text, nullable=False, server_default="idle"),
sa.Column("created", sa.Text, nullable=False),
sa.Column("updated", sa.Text, nullable=False),
)
sa.Index("idx_workstreams_node_id", workstreams.c.node_id)
sa.Index("idx_workstreams_state", workstreams.c.state)
sa.Index("idx_workstreams_user_id", workstreams.c.user_id)
session_config = sa.Table(
"session_config",
@@ -54,3 +76,64 @@ session_config = sa.Table(
sa.Column("value", sa.Text),
sa.PrimaryKeyConstraint("session_id", "key"),
)
# ---------------------------------------------------------------------------
# User identity tables
# ---------------------------------------------------------------------------
users = sa.Table(
"users",
metadata,
sa.Column("user_id", sa.Text, primary_key=True),
sa.Column("username", sa.Text, nullable=False, unique=True),
sa.Column("display_name", sa.Text, nullable=False),
sa.Column("password_hash", sa.Text, nullable=False),
sa.Column("created", sa.Text, nullable=False),
)
sa.Index("idx_users_username", users.c.username)
api_tokens = sa.Table(
"api_tokens",
metadata,
sa.Column("token_id", sa.Text, primary_key=True),
sa.Column("token_hash", sa.Text, nullable=False, unique=True),
sa.Column("token_prefix", sa.Text, nullable=False),
sa.Column("user_id", sa.Text, nullable=False),
sa.Column("name", sa.Text, nullable=False, server_default=""),
sa.Column("scopes", sa.Text, nullable=False),
sa.Column("created", sa.Text, nullable=False),
sa.Column("expires", sa.Text),
)
sa.Index("idx_api_tokens_user_id", api_tokens.c.user_id)
sa.Index("idx_api_tokens_token_hash", api_tokens.c.token_hash)
channel_users = sa.Table(
"channel_users",
metadata,
sa.Column("channel_type", sa.Text, nullable=False),
sa.Column("channel_user_id", sa.Text, nullable=False),
sa.Column("user_id", sa.Text, nullable=False),
sa.Column("created", sa.Text, nullable=False),
sa.PrimaryKeyConstraint("channel_type", "channel_user_id"),
)
sa.Index("idx_channel_users_user_id", channel_users.c.user_id)
# ---------------------------------------------------------------------------
# Channel routing tables
# ---------------------------------------------------------------------------
channel_routes = sa.Table(
"channel_routes",
metadata,
sa.Column("channel_type", sa.Text, nullable=False),
sa.Column("channel_id", sa.Text, nullable=False),
sa.Column("ws_id", sa.Text, nullable=False),
sa.Column("node_id", sa.Text, nullable=False, server_default=""),
sa.Column("created", sa.Text, nullable=False),
sa.PrimaryKeyConstraint("channel_type", "channel_id"),
)
sa.Index("idx_channel_routes_ws", channel_routes.c.ws_id)
+488 -3
View File
@@ -11,11 +11,14 @@ from typing import Any
import sqlalchemy as sa
from turnstone.core.storage._schema import (
api_tokens,
conversations,
memories,
metadata,
session_config,
sessions,
users,
workstreams,
)
log = logging.getLogger(__name__)
@@ -81,12 +84,27 @@ class SQLiteBackend:
# -- Core session operations -----------------------------------------------
def register_session(self, session_id: str, title: str | None = None) -> None:
def register_session(
self,
session_id: str,
title: str | None = None,
node_id: str | None = None,
ws_id: str | None = None,
user_id: str | None = None,
) -> None:
now = datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%S")
with self._engine.connect() as conn:
conn.execute(
sa.insert(sessions).prefix_with("OR IGNORE"),
{"session_id": session_id, "title": title, "created": now, "updated": now},
{
"session_id": session_id,
"title": title,
"node_id": node_id,
"ws_id": ws_id,
"user_id": user_id,
"created": now,
"updated": now,
},
)
conn.commit()
@@ -159,7 +177,8 @@ class SQLiteBackend:
sa.text(
"SELECT s.session_id, s.alias, s.title, s.created, s.updated, "
"(SELECT COUNT(*) FROM conversations c "
" WHERE c.session_id = s.session_id) "
" WHERE c.session_id = s.session_id), "
"s.node_id, s.ws_id "
"FROM sessions s "
"WHERE EXISTS "
" (SELECT 1 FROM conversations c WHERE c.session_id = s.session_id) "
@@ -400,6 +419,76 @@ class SQLiteBackend:
).fetchall()
return [(str(r[0]), str(r[1])) for r in rows]
# -- Workstream operations -------------------------------------------------
def register_workstream(
self,
ws_id: str,
node_id: str | None = None,
name: str = "",
state: str = "idle",
user_id: str | None = None,
) -> None:
now = datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%S")
with self._engine.connect() as conn:
conn.execute(
sa.insert(workstreams).prefix_with("OR IGNORE"),
{
"ws_id": ws_id,
"node_id": node_id,
"user_id": user_id,
"name": name,
"state": state,
"created": now,
"updated": now,
},
)
conn.commit()
def update_workstream_state(self, ws_id: str, state: str) -> None:
now = datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%S")
with self._engine.connect() as conn:
conn.execute(
sa.update(workstreams)
.where(workstreams.c.ws_id == ws_id)
.values(state=state, updated=now)
)
conn.commit()
def update_workstream_name(self, ws_id: str, name: str) -> None:
now = datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%S")
with self._engine.connect() as conn:
conn.execute(
sa.update(workstreams)
.where(workstreams.c.ws_id == ws_id)
.values(name=name, updated=now)
)
conn.commit()
def delete_workstream(self, ws_id: str) -> bool:
with self._engine.connect() as conn:
result = conn.execute(sa.delete(workstreams).where(workstreams.c.ws_id == ws_id))
conn.commit()
return result.rowcount > 0
def list_workstreams(self, node_id: str | None = None, limit: int = 100) -> list[Any]:
with self._engine.connect() as conn:
q = (
sa.select(
workstreams.c.ws_id,
workstreams.c.node_id,
workstreams.c.name,
workstreams.c.state,
workstreams.c.created,
workstreams.c.updated,
)
.order_by(workstreams.c.updated.desc())
.limit(limit)
)
if node_id is not None:
q = q.where(workstreams.c.node_id == node_id)
return list(conn.execute(q).fetchall())
# -- Conversation search ---------------------------------------------------
def search_history(self, query: str, limit: int = 20) -> list[Any]:
@@ -444,6 +533,402 @@ class SQLiteBackend:
).fetchall()
)
# -- User identity operations -----------------------------------------------
def create_user(
self, user_id: str, username: str, display_name: str, password_hash: str
) -> None:
now = datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%S")
with self._engine.connect() as conn:
conn.execute(
sa.insert(users).prefix_with("OR IGNORE"),
{
"user_id": user_id,
"username": username,
"display_name": display_name,
"password_hash": password_hash,
"created": now,
},
)
conn.commit()
def create_first_user(
self, user_id: str, username: str, display_name: str, password_hash: str
) -> bool:
"""Atomically create a user only if no users exist. Returns True if created."""
now = datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%S")
with self._engine.connect() as conn:
result = conn.execute(
sa.text(
"INSERT INTO users (user_id, username, display_name, password_hash, created) "
"SELECT :user_id, :username, :display_name, :password_hash, :created "
"WHERE NOT EXISTS (SELECT 1 FROM users)"
),
{
"user_id": user_id,
"username": username,
"display_name": display_name,
"password_hash": password_hash,
"created": now,
},
)
conn.commit()
return result.rowcount > 0
def get_user(self, user_id: str) -> dict[str, str] | None:
with self._engine.connect() as conn:
row = conn.execute(
sa.select(
users.c.user_id,
users.c.username,
users.c.display_name,
users.c.password_hash,
users.c.created,
).where(users.c.user_id == user_id)
).fetchone()
if row:
return {
"user_id": row[0],
"username": row[1],
"display_name": row[2],
"password_hash": row[3],
"created": row[4],
}
return None
def get_user_by_username(self, username: str) -> dict[str, str] | None:
with self._engine.connect() as conn:
row = conn.execute(
sa.select(
users.c.user_id,
users.c.username,
users.c.display_name,
users.c.password_hash,
users.c.created,
).where(users.c.username == username)
).fetchone()
if row:
return {
"user_id": row[0],
"username": row[1],
"display_name": row[2],
"password_hash": row[3],
"created": row[4],
}
return None
def list_users(self) -> list[dict[str, str]]:
with self._engine.connect() as conn:
rows = conn.execute(
sa.select(
users.c.user_id,
users.c.username,
users.c.display_name,
users.c.created,
).order_by(users.c.created.desc())
).fetchall()
return [
{"user_id": r[0], "username": r[1], "display_name": r[2], "created": r[3]}
for r in rows
]
def delete_user(self, user_id: str) -> bool:
from turnstone.core.storage._schema import channel_users
with self._engine.connect() as conn:
conn.execute(sa.delete(channel_users).where(channel_users.c.user_id == user_id))
conn.execute(sa.delete(api_tokens).where(api_tokens.c.user_id == user_id))
result = conn.execute(sa.delete(users).where(users.c.user_id == user_id))
conn.commit()
return result.rowcount > 0
def create_api_token(
self,
token_id: str,
token_hash: str,
token_prefix: str,
user_id: str,
name: str,
scopes: str,
expires: str | None = None,
) -> None:
now = datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%S")
with self._engine.connect() as conn:
conn.execute(
sa.insert(api_tokens),
{
"token_id": token_id,
"token_hash": token_hash,
"token_prefix": token_prefix,
"user_id": user_id,
"name": name,
"scopes": scopes,
"created": now,
"expires": expires,
},
)
conn.commit()
def get_api_token_by_hash(self, token_hash: str) -> dict[str, str] | None:
with self._engine.connect() as conn:
row = conn.execute(
sa.select(
api_tokens.c.token_id,
api_tokens.c.token_prefix,
api_tokens.c.user_id,
api_tokens.c.name,
api_tokens.c.scopes,
api_tokens.c.created,
api_tokens.c.expires,
).where(api_tokens.c.token_hash == token_hash)
).fetchone()
if row:
result: dict[str, str] = {
"token_id": row[0],
"token_prefix": row[1],
"user_id": row[2],
"name": row[3],
"scopes": row[4],
"created": row[5],
}
if row[6] is not None:
result["expires"] = row[6]
return result
return None
def list_api_tokens(self, user_id: str) -> list[dict[str, str]]:
with self._engine.connect() as conn:
rows = conn.execute(
sa.select(
api_tokens.c.token_id,
api_tokens.c.token_prefix,
api_tokens.c.user_id,
api_tokens.c.name,
api_tokens.c.scopes,
api_tokens.c.created,
api_tokens.c.expires,
)
.where(api_tokens.c.user_id == user_id)
.order_by(api_tokens.c.created.desc())
).fetchall()
result = []
for r in rows:
entry: dict[str, str] = {
"token_id": r[0],
"token_prefix": r[1],
"user_id": r[2],
"name": r[3],
"scopes": r[4],
"created": r[5],
}
if r[6] is not None:
entry["expires"] = r[6]
result.append(entry)
return result
def delete_api_token(self, token_id: str) -> bool:
with self._engine.connect() as conn:
result = conn.execute(sa.delete(api_tokens).where(api_tokens.c.token_id == token_id))
conn.commit()
return result.rowcount > 0
# -- Session lookup by workstream ------------------------------------------
def get_session_id_by_ws(self, ws_id: str) -> str | None:
with self._engine.connect() as conn:
row = conn.execute(
sa.select(sessions.c.session_id).where(sessions.c.ws_id == ws_id)
).fetchone()
return str(row[0]) if row else None
# -- Channel user mapping ---------------------------------------------------
def create_channel_user(self, channel_type: str, channel_user_id: str, user_id: str) -> None:
from turnstone.core.storage._schema import channel_users
now = datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%S")
with self._engine.connect() as conn:
conn.execute(
sa.insert(channel_users).prefix_with("OR IGNORE"),
{
"channel_type": channel_type,
"channel_user_id": channel_user_id,
"user_id": user_id,
"created": now,
},
)
conn.commit()
def get_channel_user(self, channel_type: str, channel_user_id: str) -> dict[str, str] | None:
from turnstone.core.storage._schema import channel_users
with self._engine.connect() as conn:
row = conn.execute(
sa.select(
channel_users.c.channel_type,
channel_users.c.channel_user_id,
channel_users.c.user_id,
channel_users.c.created,
).where(
(channel_users.c.channel_type == channel_type)
& (channel_users.c.channel_user_id == channel_user_id)
)
).fetchone()
if row:
return {
"channel_type": row[0],
"channel_user_id": row[1],
"user_id": row[2],
"created": row[3],
}
return None
def list_channel_users_by_user(self, user_id: str) -> list[dict[str, str]]:
from turnstone.core.storage._schema import channel_users
with self._engine.connect() as conn:
rows = conn.execute(
sa.select(
channel_users.c.channel_type,
channel_users.c.channel_user_id,
channel_users.c.user_id,
channel_users.c.created,
)
.where(channel_users.c.user_id == user_id)
.order_by(channel_users.c.created.desc())
).fetchall()
return [
{
"channel_type": r[0],
"channel_user_id": r[1],
"user_id": r[2],
"created": r[3],
}
for r in rows
]
def delete_channel_user(self, channel_type: str, channel_user_id: str) -> bool:
from turnstone.core.storage._schema import channel_users
with self._engine.connect() as conn:
result = conn.execute(
sa.delete(channel_users).where(
(channel_users.c.channel_type == channel_type)
& (channel_users.c.channel_user_id == channel_user_id)
)
)
conn.commit()
return result.rowcount > 0
# -- Channel routing -------------------------------------------------------
def create_channel_route(
self, channel_type: str, channel_id: str, ws_id: str, node_id: str = ""
) -> None:
from turnstone.core.storage._schema import channel_routes
now = datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%S")
with self._engine.connect() as conn:
conn.execute(
sa.insert(channel_routes).prefix_with("OR IGNORE"),
{
"channel_type": channel_type,
"channel_id": channel_id,
"ws_id": ws_id,
"node_id": node_id,
"created": now,
},
)
conn.commit()
def get_channel_route(self, channel_type: str, channel_id: str) -> dict[str, str] | None:
from turnstone.core.storage._schema import channel_routes
with self._engine.connect() as conn:
row = conn.execute(
sa.select(
channel_routes.c.channel_type,
channel_routes.c.channel_id,
channel_routes.c.ws_id,
channel_routes.c.node_id,
channel_routes.c.created,
).where(
(channel_routes.c.channel_type == channel_type)
& (channel_routes.c.channel_id == channel_id)
)
).fetchone()
if row:
return {
"channel_type": row[0],
"channel_id": row[1],
"ws_id": row[2],
"node_id": row[3],
"created": row[4],
}
return None
def get_channel_route_by_ws(self, ws_id: str) -> dict[str, str] | None:
from turnstone.core.storage._schema import channel_routes
with self._engine.connect() as conn:
row = conn.execute(
sa.select(
channel_routes.c.channel_type,
channel_routes.c.channel_id,
channel_routes.c.ws_id,
channel_routes.c.node_id,
channel_routes.c.created,
).where(channel_routes.c.ws_id == ws_id)
).fetchone()
if row:
return {
"channel_type": row[0],
"channel_id": row[1],
"ws_id": row[2],
"node_id": row[3],
"created": row[4],
}
return None
def list_channel_routes_by_type(self, channel_type: str) -> list[dict[str, str]]:
from turnstone.core.storage._schema import channel_routes
with self._engine.connect() as conn:
rows = conn.execute(
sa.select(
channel_routes.c.channel_type,
channel_routes.c.channel_id,
channel_routes.c.ws_id,
channel_routes.c.node_id,
channel_routes.c.created,
)
.where(channel_routes.c.channel_type == channel_type)
.order_by(channel_routes.c.created.desc())
).fetchall()
return [
{
"channel_type": r[0],
"channel_id": r[1],
"ws_id": r[2],
"node_id": r[3],
"created": r[4],
}
for r in rows
]
def delete_channel_route(self, channel_type: str, channel_id: str) -> bool:
from turnstone.core.storage._schema import channel_routes
with self._engine.connect() as conn:
result = conn.execute(
sa.delete(channel_routes).where(
(channel_routes.c.channel_type == channel_type)
& (channel_routes.c.channel_id == channel_id)
)
)
conn.commit()
return result.rowcount > 0
# -- Lifecycle -------------------------------------------------------------
def close(self) -> None:
@@ -45,11 +45,28 @@ def upgrade() -> None:
sa.Column("session_id", sa.Text, primary_key=True),
sa.Column("alias", sa.Text, unique=True),
sa.Column("title", sa.Text),
sa.Column("node_id", sa.Text),
sa.Column("ws_id", sa.Text),
sa.Column("created", sa.Text, nullable=False),
sa.Column("updated", sa.Text, nullable=False),
)
op.create_index("idx_sessions_alias", "sessions", ["alias"])
op.create_index("idx_sessions_updated", "sessions", ["updated"])
op.create_index("idx_sessions_node_id", "sessions", ["node_id"])
op.create_index("idx_sessions_ws_id", "sessions", ["ws_id"])
# workstreams — persistent workstream lifecycle tracking
op.create_table(
"workstreams",
sa.Column("ws_id", sa.Text, primary_key=True),
sa.Column("node_id", sa.Text),
sa.Column("name", sa.Text, nullable=False, server_default=""),
sa.Column("state", sa.Text, nullable=False, server_default="idle"),
sa.Column("created", sa.Text, nullable=False),
sa.Column("updated", sa.Text, nullable=False),
)
op.create_index("idx_workstreams_node_id", "workstreams", ["node_id"])
op.create_index("idx_workstreams_state", "workstreams", ["state"])
# session_config — per-session LLM parameters
op.create_table(
@@ -63,6 +80,7 @@ def upgrade() -> None:
def downgrade() -> None:
op.drop_table("session_config")
op.drop_table("workstreams")
op.drop_table("sessions")
op.drop_table("conversations")
op.drop_table("memories")
@@ -0,0 +1,70 @@
"""User identity and API tokens.
Revision ID: 002
Revises: 001
Create Date: 2026-03-04
"""
import sqlalchemy as sa
from alembic import op
revision = "002"
down_revision = "001"
branch_labels = None
depends_on = None
def upgrade() -> None:
# --- New tables ---
op.create_table(
"users",
sa.Column("user_id", sa.Text, primary_key=True),
sa.Column("username", sa.Text, nullable=False, unique=True),
sa.Column("display_name", sa.Text, nullable=False),
sa.Column("password_hash", sa.Text, nullable=False),
sa.Column("created", sa.Text, nullable=False),
)
op.create_index("idx_users_username", "users", ["username"])
op.create_table(
"api_tokens",
sa.Column("token_id", sa.Text, primary_key=True),
sa.Column("token_hash", sa.Text, nullable=False, unique=True),
sa.Column("token_prefix", sa.Text, nullable=False),
sa.Column("user_id", sa.Text, nullable=False),
sa.Column("name", sa.Text, nullable=False, server_default=""),
sa.Column("scopes", sa.Text, nullable=False),
sa.Column("created", sa.Text, nullable=False),
sa.Column("expires", sa.Text),
)
op.create_index("idx_api_tokens_user_id", "api_tokens", ["user_id"])
op.create_index("idx_api_tokens_token_hash", "api_tokens", ["token_hash"])
op.create_table(
"channel_users",
sa.Column("channel_type", sa.Text, nullable=False),
sa.Column("channel_user_id", sa.Text, nullable=False),
sa.Column("user_id", sa.Text, nullable=False),
sa.Column("created", sa.Text, nullable=False),
sa.PrimaryKeyConstraint("channel_type", "channel_user_id"),
)
op.create_index("idx_channel_users_user_id", "channel_users", ["user_id"])
# --- Add user_id to existing tables ---
op.add_column("sessions", sa.Column("user_id", sa.Text))
op.create_index("idx_sessions_user_id", "sessions", ["user_id"])
op.add_column("workstreams", sa.Column("user_id", sa.Text))
op.create_index("idx_workstreams_user_id", "workstreams", ["user_id"])
def downgrade() -> None:
op.drop_index("idx_workstreams_user_id", "workstreams")
op.drop_column("workstreams", "user_id")
op.drop_index("idx_sessions_user_id", "sessions")
op.drop_column("sessions", "user_id")
op.drop_table("channel_users")
op.drop_table("api_tokens")
op.drop_table("users")
@@ -0,0 +1,32 @@
"""Channel routing table.
Revision ID: 003
Revises: 002
Create Date: 2026-03-04
"""
import sqlalchemy as sa
from alembic import op
revision = "003"
down_revision = "002"
branch_labels = None
depends_on = None
def upgrade() -> None:
op.create_table(
"channel_routes",
sa.Column("channel_type", sa.Text, nullable=False),
sa.Column("channel_id", sa.Text, nullable=False),
sa.Column("ws_id", sa.Text, nullable=False),
sa.Column("node_id", sa.Text, nullable=False, server_default=""),
sa.Column("created", sa.Text, nullable=False),
sa.PrimaryKeyConstraint("channel_type", "channel_id"),
)
op.create_index("idx_channel_routes_ws", "channel_routes", ["ws_id"])
def downgrade() -> None:
op.drop_index("idx_channel_routes_ws", "channel_routes")
op.drop_table("channel_routes")
+75
View File
@@ -0,0 +1,75 @@
"""Starlette web-request helpers shared across HTTP servers."""
from __future__ import annotations
import json
import os
from typing import TYPE_CHECKING, Any
if TYPE_CHECKING:
from starlette.middleware import Middleware
from starlette.requests import Request
from starlette.responses import JSONResponse
async def read_json_or_400(request: Request) -> dict[str, Any] | JSONResponse:
"""Parse a JSON request body, returning a 400 response on failure.
Callers should check ``isinstance(result, JSONResponse)`` and return
it early when the parse fails::
body = await read_json_or_400(request)
if isinstance(body, JSONResponse):
return body
"""
from starlette.responses import JSONResponse as _JSONResponse
try:
body: dict[str, Any] = await request.json()
return body
except (ValueError, json.JSONDecodeError):
return _JSONResponse({"error": "Invalid JSON body"}, status_code=400)
def require_storage_or_503(
request: Request,
) -> tuple[Any, JSONResponse | None]:
"""Return ``(storage, None)`` or ``(None, JSONResponse(503))``.
Usage::
storage, err = require_storage_or_503(request)
if err:
return err
"""
from starlette.responses import JSONResponse as _JSONResponse
storage = getattr(request.app.state, "auth_storage", None)
if storage is None:
return None, _JSONResponse({"error": "Storage not available"}, status_code=503)
return storage, None
def parse_cors_origins() -> list[str] | None:
"""Parse ``TURNSTONE_CORS_ORIGINS`` env var into a list of origin strings.
Returns ``None`` when the variable is unset or empty (meaning: no CORS
middleware, same-origin only).
"""
cors_env = os.environ.get("TURNSTONE_CORS_ORIGINS", "").strip()
if not cors_env:
return None
return [o.strip() for o in cors_env.split(",") if o.strip()]
def cors_middleware(origins: list[str]) -> Middleware:
"""Build a Starlette ``CORSMiddleware`` entry for the given *origins*."""
from starlette.middleware import Middleware as _Middleware
from starlette.middleware.cors import CORSMiddleware
return _Middleware(
CORSMiddleware,
allow_origins=origins,
allow_methods=["GET", "POST", "OPTIONS"],
allow_headers=["Content-Type", "Authorization"],
)
+50 -15
View File
@@ -40,7 +40,7 @@ class WorkstreamState(enum.Enum):
@dataclass
class Workstream:
id: str = field(default_factory=lambda: uuid.uuid4().hex[:8])
id: str = field(default_factory=lambda: uuid.uuid4().hex)
name: str = ""
state: WorkstreamState = WorkstreamState.IDLE
session: ChatSession | None = None
@@ -65,25 +65,29 @@ class WorkstreamManager:
def __init__(
self,
session_factory: Callable[[SessionUI | None, str | None], ChatSession],
session_factory: Callable[[SessionUI | None, str | None, str | None], ChatSession],
*,
max_workstreams: int = 10,
node_id: str | None = None,
):
"""
Args:
session_factory: callable(ui, model_alias) -> ChatSession.
session_factory: callable(ui, model_alias, ws_id) -> ChatSession.
Captures shared config (registry, temperature, ) so the
manager can create sessions without knowing those details.
*model_alias* selects a model from the registry (None = default).
*ws_id* links the session to its workstream in storage.
max_workstreams: Maximum number of concurrent workstreams. When at
capacity, ``create()`` will auto-evict the oldest IDLE
workstream before raising.
node_id: Server node identity (persisted with workstreams).
"""
if max_workstreams < 1:
raise ValueError(f"max_workstreams must be >= 1, got {max_workstreams}")
self._session_factory: Callable[[SessionUI | None, str | None], ChatSession] = (
self._session_factory: Callable[[SessionUI | None, str | None, str | None], ChatSession] = (
session_factory
)
self._node_id = node_id
self._max_workstreams: int = max_workstreams
self._workstreams: dict[str, Workstream] = {}
self._order: list[str] = [] # creation order
@@ -121,29 +125,54 @@ class WorkstreamManager:
model: Optional model alias from the registry. ``None`` uses the
default model.
"""
evicted_ws: Workstream | None = None
# Fast-fail capacity check (avoids expensive session creation when full).
first_evicted: Workstream | None = None
with self._lock:
if len(self._workstreams) >= self._max_workstreams:
evicted_ws = self._evict_oldest_idle_locked()
if evicted_ws is None:
first_evicted = self._evict_oldest_idle_locked()
if first_evicted is None:
raise RuntimeError(f"All {self._max_workstreams} workstreams are active")
# Cleanup first-phase eviction outside the lock (may trigger callbacks).
if first_evicted is not None:
self._cleanup_ui(first_evicted)
self._last_evicted = first_evicted
from turnstone.core.metrics import metrics as _m1
_m1.record_eviction()
# Create workstream and session outside the lock (session creation is
# expensive — involves LLM client setup and DB writes).
ws = Workstream(name=name)
if ui_factory:
ws.ui = ui_factory(ws.id)
ws.session = self._session_factory(ws.ui, model)
ws.session = self._session_factory(ws.ui, model, ws.id)
# Authoritative insert under lock with re-check (another thread may
# have filled capacity while we were unlocked).
second_evicted: Workstream | None = None
with self._lock:
if len(self._workstreams) >= self._max_workstreams:
second_evicted = self._evict_oldest_idle_locked()
if second_evicted is None:
raise RuntimeError(f"All {self._max_workstreams} workstreams are active")
self._workstreams[ws.id] = ws
self._order.append(ws.id)
if self._active_id is None:
self._active_id = ws.id
# UI cleanup for the evicted workstream must happen outside the lock
# because it may trigger callbacks.
self._last_evicted = evicted_ws
if evicted_ws is not None:
self._cleanup_ui(evicted_ws)
from turnstone.core.metrics import metrics
metrics.record_eviction()
# Persist to storage only after successful insertion
from turnstone.core.memory import register_workstream
register_workstream(ws.id, node_id=self._node_id, name=ws.name)
# Cleanup second-phase eviction outside the lock.
if second_evicted is not None:
self._cleanup_ui(second_evicted)
self._last_evicted = second_evicted
from turnstone.core.metrics import metrics as _m2
_m2.record_eviction()
return ws
# -- eviction helpers ---------------------------------------------------
@@ -197,6 +226,9 @@ class WorkstreamManager:
self._active_id = self._order[0]
# Unblock any waiting approval/plan events so worker thread can exit
self._cleanup_ui(ws)
from turnstone.core.memory import update_workstream_state
update_workstream_state(ws_id, "closed")
return True
# -- lookup -------------------------------------------------------------
@@ -260,6 +292,9 @@ class WorkstreamManager:
ws.state = state
ws.last_active = time.monotonic()
ws.error_message = error_msg
from turnstone.core.memory import update_workstream_state
update_workstream_state(ws_id, state.value)
if self._on_state_change:
self._on_state_change(ws_id, state)
+16 -1
View File
@@ -8,4 +8,19 @@ commands and subscribe to progress.
from turnstone.mq.broker import MessageBroker, RedisBroker
from turnstone.mq.client import TurnResult, TurnstoneClient
__all__ = ["MessageBroker", "RedisBroker", "TurnstoneClient", "TurnResult"]
__all__ = [
"AsyncRedisBroker",
"MessageBroker",
"RedisBroker",
"TurnstoneClient",
"TurnResult",
]
def __getattr__(name: str) -> object:
if name == "AsyncRedisBroker":
from turnstone.mq.async_broker import AsyncRedisBroker
return AsyncRedisBroker
msg = f"module {__name__!r} has no attribute {name!r}"
raise AttributeError(msg)
+314
View File
@@ -0,0 +1,314 @@
"""Async Redis message broker.
Provides :class:`AsyncRedisBroker`, an asyncio-native counterpart to
:class:`~turnstone.mq.broker.RedisBroker`. Uses ``redis.asyncio`` for all I/O
and manages pub/sub listeners as :class:`asyncio.Task` instances.
"""
from __future__ import annotations
import asyncio
import contextlib
import json
from typing import TYPE_CHECKING, Any
if TYPE_CHECKING:
from collections.abc import Callable
import redis.asyncio as _aredis_t
class AsyncRedisBroker:
"""Async Redis-backed message broker using lists (queues) and pub/sub.
This is the asyncio equivalent of :class:`~turnstone.mq.broker.RedisBroker`.
All methods are coroutines and must be awaited.
Queue keys:
``{prefix}:inbound`` shared inbound command queue
``{prefix}:inbound:{node_id}`` per-node directed queue
``{prefix}:resp:{request_id}`` per-request response queues
Routing keys:
``{prefix}:ws:{ws_id}`` workstream ownership (string)
``{prefix}:node:{node_id}`` node heartbeat + metadata (string/JSON)
Pub/sub channels:
``{prefix}:events:global`` global event channel
``{prefix}:events:{ws_id}`` per-workstream event channel
``{prefix}:events:cluster`` cluster-wide state changes
"""
def __init__(
self,
host: str = "localhost",
port: int = 6379,
db: int = 0,
prefix: str = "turnstone",
password: str | None = None,
response_ttl: int = 600,
) -> None:
self._host = host
self._port = port
self._db = db
self._password = password
self._prefix = prefix
self._response_ttl = response_ttl
self._redis: _aredis_t.Redis[str] | None = None
self._pubsub: _aredis_t.client.PubSub | None = None
self._tasks: dict[str, asyncio.Task[None]] = {}
self._callbacks: dict[str, Callable[[str], Any]] = {}
self._queues: dict[str, asyncio.Queue[str]] = {}
self._workers: dict[str, asyncio.Task[None]] = {}
self._listener_task: asyncio.Task[None] | None = None
# -- connection ----------------------------------------------------------
async def connect(self) -> None:
"""Create the async Redis connection.
This is called lazily before first use if the connection has not yet
been established.
"""
if self._redis is not None:
return
import redis.asyncio as aioredis
self._redis = aioredis.Redis(
host=self._host,
port=self._port,
db=self._db,
password=self._password,
decode_responses=True,
retry_on_timeout=True,
)
self._pubsub = self._redis.pubsub(ignore_subscribe_messages=True)
async def _ensure_connected(self) -> None:
"""Ensure the Redis connection is established."""
if self._redis is None:
await self.connect()
@property
def _r(self) -> _aredis_t.Redis[str]:
"""Return the Redis client, assuming it is connected."""
if self._redis is None:
msg = "Broker not connected — call connect() first"
raise RuntimeError(msg)
return self._redis
@property
def _ps(self) -> _aredis_t.client.PubSub:
"""Return the pub/sub client, assuming it is connected."""
if self._pubsub is None:
msg = "Broker not connected — call connect() first"
raise RuntimeError(msg)
return self._pubsub
# -- inbound queue -------------------------------------------------------
async def push_inbound(self, message: str, node_id: str = "") -> None:
"""Push a message onto the inbound queue.
If *node_id* is set, pushes to the per-node queue for directed
routing. Otherwise pushes to the shared queue.
"""
await self._ensure_connected()
if node_id:
await self._r.rpush(f"{self._prefix}:inbound:{node_id}", message)
else:
await self._r.rpush(f"{self._prefix}:inbound", message)
# -- outbound pub/sub ----------------------------------------------------
async def publish_outbound(self, channel: str, event: str) -> None:
"""Publish an event to an outbound channel."""
await self._ensure_connected()
await self._r.publish(channel, event)
async def subscribe(self, channel: str, callback: Callable[[str], Any]) -> None:
"""Subscribe to a pub/sub channel.
The *callback* receives the message string for each published event.
It may be a regular function or an async coroutine.
All subscriptions share a single listener task that dispatches
messages to the correct callback based on the channel name.
"""
await self._ensure_connected()
await self._ps.subscribe(channel)
self._callbacks[channel] = callback
# Per-channel queue + worker ensures ordered delivery within a channel
# while allowing different channels to process concurrently.
q: asyncio.Queue[str] = asyncio.Queue()
self._queues[channel] = q
self._workers[channel] = asyncio.create_task(self._channel_worker(channel, q))
# Start the shared listener task if not already running.
if self._listener_task is None or self._listener_task.done():
self._listener_task = asyncio.create_task(self._dispatch_loop())
async def _dispatch_loop(self) -> None:
"""Single listener that routes pub/sub messages to per-channel queues.
Each channel has its own queue + worker task, ensuring ordered
delivery within a channel while allowing different channels to
process concurrently.
"""
import logging
_log = logging.getLogger("turnstone.mq.async_broker")
try:
while self._callbacks:
msg = await self._ps.get_message(
ignore_subscribe_messages=True,
timeout=0.1,
)
if msg is None:
# Yield control so cancellation can be delivered.
await asyncio.sleep(0)
continue
if msg["type"] == "message":
ch = msg.get("channel", "")
q = self._queues.get(ch)
if q is not None:
q.put_nowait(msg["data"])
_log.debug("Dispatch loop exiting — no active callbacks")
except asyncio.CancelledError:
return
async def _channel_worker(self, channel: str, q: asyncio.Queue[str]) -> None:
"""Process messages for a single channel sequentially."""
import logging
_log = logging.getLogger("turnstone.mq.async_broker")
try:
while True:
data = await q.get()
cb = self._callbacks.get(channel)
if cb is not None:
try:
result = cb(data)
if asyncio.iscoroutine(result):
await result
except Exception:
_log.exception("Listener callback error on %s", channel)
except asyncio.CancelledError:
return
async def unsubscribe(self, channel: str) -> None:
"""Unsubscribe from a channel and cancel its worker."""
await self._ensure_connected()
self._callbacks.pop(channel, None)
self._queues.pop(channel, None)
worker = self._workers.pop(channel, None)
if worker is not None:
worker.cancel()
with contextlib.suppress(asyncio.CancelledError):
await worker
# Legacy per-channel task cleanup (in case any remain).
task = self._tasks.pop(channel, None)
if task is not None:
task.cancel()
with contextlib.suppress(asyncio.CancelledError):
await task
await self._ps.unsubscribe(channel)
# -- response queues -----------------------------------------------------
async def push_response(self, queue_name: str, message: str) -> None:
"""Push a response onto a named response queue."""
await self._ensure_connected()
key = f"{self._prefix}:resp:{queue_name}"
await self._r.rpush(key, message)
await self._r.expire(key, self._response_ttl)
async def pop_response(self, queue_name: str, timeout: float = 300) -> str | None:
"""Pop from a named response queue. Returns ``None`` on timeout."""
await self._ensure_connected()
key = f"{self._prefix}:resp:{queue_name}"
result = await self._r.blpop(key, timeout=int(timeout))
return result[1] if result else None
# -- routing primitives --------------------------------------------------
async def get_ws_owner(self, ws_id: str) -> str | None:
"""Look up the node that owns a workstream."""
await self._ensure_connected()
return await self._r.get(f"{self._prefix}:ws:{ws_id}")
async def set_ws_owner(self, ws_id: str, node_id: str, ttl: int = 0) -> None:
"""Register which node owns a workstream."""
await self._ensure_connected()
key = f"{self._prefix}:ws:{ws_id}"
if ttl > 0:
await self._r.set(key, node_id, ex=ttl)
else:
await self._r.set(key, node_id)
async def del_ws_owner(self, ws_id: str) -> None:
"""Remove workstream ownership."""
await self._ensure_connected()
await self._r.delete(f"{self._prefix}:ws:{ws_id}")
async def register_node(self, node_id: str, metadata: dict[str, Any], ttl: int = 60) -> None:
"""Register or refresh a node's heartbeat with metadata."""
await self._ensure_connected()
key = f"{self._prefix}:node:{node_id}"
await self._r.set(key, json.dumps(metadata), ex=ttl)
async def list_nodes(self) -> list[dict[str, Any]]:
"""List all active nodes (those with unexpired heartbeats)."""
await self._ensure_connected()
pattern = f"{self._prefix}:node:*"
prefix_len = len(f"{self._prefix}:node:")
nodes: list[dict[str, Any]] = []
async for key in self._r.scan_iter(match=pattern, count=100):
raw = await self._r.get(key)
if raw:
try:
meta: dict[str, Any] = json.loads(raw)
except json.JSONDecodeError:
meta = {}
meta["node_id"] = key[prefix_len:]
nodes.append(meta)
return nodes
# -- lifecycle -----------------------------------------------------------
async def close(self) -> None:
"""Cancel all listener tasks and close the Redis connection."""
self._callbacks.clear()
self._queues.clear()
# Cancel per-channel workers.
for worker in self._workers.values():
worker.cancel()
for worker in self._workers.values():
with contextlib.suppress(asyncio.CancelledError):
await worker
self._workers.clear()
# Cancel the shared dispatch loop.
if self._listener_task is not None:
if not self._listener_task.done():
self._listener_task.cancel()
with contextlib.suppress(asyncio.CancelledError):
await self._listener_task
self._listener_task = None
# Legacy per-channel tasks.
for task in self._tasks.values():
task.cancel()
for task in self._tasks.values():
with contextlib.suppress(asyncio.CancelledError):
await task
self._tasks.clear()
if self._pubsub is not None:
with contextlib.suppress(Exception):
await self._pubsub.close()
self._pubsub = None
if self._redis is not None:
await self._redis.close()
self._redis = None
+249 -104
View File
@@ -13,7 +13,6 @@ import contextlib
import json
import logging
import os
import socket
import threading
import time
import uuid
@@ -35,6 +34,7 @@ from turnstone.mq.protocol import (
OutboundEvent,
PlanReviewEvent,
ReasoningEvent,
SessionResumedEvent,
StateChangeEvent,
StatusEvent,
StreamEndEvent,
@@ -57,18 +57,6 @@ log = logging.getLogger("turnstone.mq.bridge")
DEFAULT_SAFE_TOOLS = frozenset(["read_file", "search", "man", "remember", "recall", "forget"])
def _default_node_id() -> str:
"""Generate a default node_id: ``{hostname}_{4hex}``, or a UUID on failure."""
suffix = uuid.uuid4().hex[:4]
try:
host = socket.gethostname()
if host and host != "localhost":
return f"{host}_{suffix}"
except OSError:
pass
return uuid.uuid4().hex[:12]
class Bridge:
"""Connects a message broker to turnstone-server's HTTP API.
@@ -88,21 +76,26 @@ class Bridge:
node_id: str = "",
heartbeat_ttl: int = 60,
auth_token: str = "",
token_manager: Any = None,
) -> None:
self._server_url = server_url.rstrip("/")
self._broker = broker or RedisBroker()
self._approval_timeout = approval_timeout
self._prefix = prefix
self._node_id = node_id or _default_node_id()
self._node_id = node_id # resolved in run() from server /health
self._heartbeat_ttl = heartbeat_ttl
self._started_at = time.time()
self._auth_token = auth_token
self._token_manager = token_manager # ServiceTokenManager (auto-rotating)
# Shared httpx client for short-lived POST requests (main thread only)
headers: dict[str, str] = {}
if auth_token:
headers["Authorization"] = f"Bearer {auth_token}"
self._http = httpx.Client(base_url=self._server_url, timeout=30, headers=headers)
# Shared httpx client for short-lived POST requests (main thread only).
# Auth headers refreshed per-request via event hook so auto-rotating
# tokens are picked up transparently.
self._http = httpx.Client(
base_url=self._server_url,
timeout=30,
event_hooks={"request": [self._inject_auth]},
)
# Protected by _lock — accessed from main, global SSE, and per-ws SSE threads
self._lock = threading.Lock()
@@ -110,19 +103,89 @@ class Bridge:
self._ws_auto_approve: dict[str, bool] = {}
self._ws_approve_tools: dict[str, set[str]] = {}
self._active_sends: dict[str, str] = {} # ws_id → correlation_id
self._pending_approvals: dict[str, str] = {} # ws_id → request_id
self._pending_plan_reviews: dict[str, str] = {} # ws_id → request_id
self._running = True
@property
def _auth_headers(self) -> dict[str, str]:
"""Return current Authorization header, auto-rotating if managed."""
if self._token_manager is not None:
return dict(self._token_manager.bearer_header)
if self._auth_token:
return {"Authorization": f"Bearer {self._auth_token}"}
return {}
def _inject_auth(self, request: httpx.Request) -> None:
"""httpx event hook: inject current auth header into each request."""
headers = self._auth_headers
for k, v in headers.items():
request.headers[k] = v
# -- thread context helper ------------------------------------------------
def _run_in_context(self, fn: Callable[..., Any], *args: Any) -> Callable[[], None]:
"""Return a callable that sets ``ctx_node_id`` before invoking *fn*."""
node_id = self._node_id
def _wrapper() -> None:
from turnstone.core.log import ctx_node_id
ctx_node_id.set(node_id)
fn(*args)
return _wrapper
# -- public entry point --------------------------------------------------
def _fetch_node_id(self) -> str:
"""Retrieve node_id from server /health with exponential backoff.
Raises ``SystemExit`` if the server is unreachable after 5 attempts.
"""
delays = [1, 2, 4, 8, 16]
for attempt, delay in enumerate(delays, 1):
try:
resp = self._http.get("/health")
if 400 <= resp.status_code < 500:
log.critical("Server returned %d — check auth_token/config", resp.status_code)
raise SystemExit(1)
resp.raise_for_status()
data = resp.json()
nid = data.get("node_id", "")
if nid:
return str(nid)
log.warning("Server /health missing node_id (attempt %d/%d)", attempt, len(delays))
except Exception as exc:
log.warning(
"Failed to fetch node_id from server (attempt %d/%d): %s",
attempt,
len(delays),
exc,
)
if attempt < len(delays):
time.sleep(delay)
log.critical(
"Could not retrieve node_id from server after %d attempts — exiting", len(delays)
)
raise SystemExit(1)
def run(self) -> None:
"""Block until shutdown (KeyboardInterrupt)."""
if not self._node_id:
self._node_id = self._fetch_node_id()
from turnstone.core.log import ctx_node_id
ctx_node_id.set(self._node_id)
log.info("Bridge starting — node=%s server=%s", self._node_id, self._server_url)
self._recover_workstreams()
heartbeat_t = threading.Thread(target=self._heartbeat_loop, daemon=True)
heartbeat_t = threading.Thread(
target=self._run_in_context(self._heartbeat_loop), daemon=True
)
heartbeat_t.start()
global_t = threading.Thread(target=self._global_sse_loop, daemon=True)
global_t = threading.Thread(target=self._run_in_context(self._global_sse_loop), daemon=True)
global_t.start()
try:
@@ -226,7 +289,7 @@ class Bridge:
# Auto-create workstream if needed
if not ws_id:
ws_id = self._create_ws_on_server(
ws_id, _resumed = self._create_ws_on_server(
name=name,
auto_approve=auto_approve,
auto_approve_tools=auto_approve_tools,
@@ -245,8 +308,17 @@ class Bridge:
with self._lock:
self._active_sends[ws_id] = msg.correlation_id
resp = self._http.post("/v1/api/send", json={"message": message, "ws_id": ws_id})
data = resp.json()
try:
resp = self._http.post("/v1/api/send", json={"message": message, "ws_id": ws_id})
data = resp.json()
except Exception:
with self._lock:
self._active_sends.pop(ws_id, None)
raise
if data.get("status") != "ok":
with self._lock:
self._active_sends.pop(ws_id, None)
self._publish_ws(
ws_id,
@@ -289,14 +361,23 @@ class Bridge:
auto_approve_tools = getattr(msg, "auto_approve_tools", [])
model = getattr(msg, "model", "")
initial_message = getattr(msg, "initial_message", "")
ws_id = self._create_ws_on_server(
resume_session = getattr(msg, "resume_session", "")
ws_id, resumed = self._create_ws_on_server(
name=name,
auto_approve=auto_approve,
auto_approve_tools=auto_approve_tools,
correlation_id=msg.correlation_id,
model=model,
resume_session=resume_session,
)
if ws_id and initial_message:
# Send initial_message only when no session was actually resumed.
# Use the server's `resumed` response (not just the intent) so that
# a pruned/missing session falls back to sending the initial message.
if ws_id and initial_message and not resumed:
# Track the send so the global SSE handler emits TurnCompleteEvent
# when the workstream returns to idle.
with self._lock:
self._active_sends[ws_id] = msg.correlation_id
try:
resp = self._http.post(
"/v1/api/send", json={"message": initial_message, "ws_id": ws_id}
@@ -304,8 +385,12 @@ class Bridge:
data = resp.json()
if data.get("error"):
log.warning("Initial message failed for ws %s: %s", ws_id, data["error"])
with self._lock:
self._active_sends.pop(ws_id, None)
except Exception as exc:
log.warning("Initial message send failed for ws %s: %s", ws_id, exc)
with self._lock:
self._active_sends.pop(ws_id, None)
def _handle_close_ws(self, msg: InboundMessage) -> None:
ws_id = getattr(msg, "ws_id", "")
@@ -350,12 +435,15 @@ class Bridge:
auto_approve_tools: list[str],
correlation_id: str,
model: str = "",
) -> str:
"""Create a workstream on the server. Returns ws_id or empty on error."""
resume_session: str = "",
) -> tuple[str, bool]:
"""Create a workstream on the server. Returns (ws_id, resumed)."""
try:
payload: dict[str, Any] = {"name": name, "auto_approve": auto_approve}
if model:
payload["model"] = model
if resume_session:
payload["resume_session"] = resume_session
resp = self._http.post(
"/v1/api/workstreams/new",
json=payload,
@@ -369,9 +457,10 @@ class Bridge:
detail=data["error"],
)
)
return ""
return "", False
ws_id: str = data["ws_id"]
ws_name = data.get("name", "")
resumed = data.get("resumed", False)
self._broker.set_ws_owner(ws_id, self._node_id)
@@ -383,11 +472,16 @@ class Bridge:
self._start_ws_sse(ws_id)
resolved_session_id = data.get("session_id", "") if resumed else ""
self._publish_global(
WorkstreamCreatedEvent(
ws_id=ws_id,
name=ws_name,
correlation_id=correlation_id,
resumed=resumed,
session_id=resolved_session_id,
message_count=data.get("message_count", 0),
)
)
self._publish_cluster(
@@ -395,9 +489,24 @@ class Bridge:
ws_id=ws_id,
name=ws_name,
correlation_id=correlation_id,
node_id=self._node_id,
)
)
return ws_id
# Emit per-workstream resume confirmation.
if resumed:
self._publish_ws(
ws_id,
SessionResumedEvent(
ws_id=ws_id,
correlation_id=correlation_id,
session_id=resolved_session_id,
message_count=data.get("message_count", 0),
name=ws_name,
),
)
return ws_id, resumed
except Exception as exc:
self._publish_global(
AckEvent(
@@ -406,7 +515,7 @@ class Bridge:
detail=str(exc),
)
)
return ""
return "", False
# -- SSE consumption -----------------------------------------------------
@@ -414,18 +523,18 @@ class Bridge:
with self._lock:
if ws_id in self._ws_threads and self._ws_threads[ws_id].is_alive():
return
t = threading.Thread(target=self._ws_sse_loop, args=(ws_id,), daemon=True)
t = threading.Thread(target=self._run_in_context(self._ws_sse_loop, ws_id), daemon=True)
self._ws_threads[ws_id] = t
t.start()
def _ws_sse_loop(self, ws_id: str) -> None:
"""Consume per-workstream SSE and forward events."""
# Each SSE thread gets its own httpx client (not thread-safe to share)
sse_headers: dict[str, str] = {}
if self._auth_token:
sse_headers["Authorization"] = f"Bearer {self._auth_token}"
# Each SSE thread gets its own httpx client (not thread-safe to share).
# Use event hooks so auth headers refresh on each reconnect.
with httpx.Client(
base_url=self._server_url, timeout=None, headers=sse_headers
base_url=self._server_url,
timeout=None,
event_hooks={"request": [self._inject_auth]},
) as sse_client:
while self._running:
try:
@@ -495,21 +604,31 @@ class Bridge:
"""Handle an approval request — auto-approve or forward to client."""
items = data.get("items", [])
# Check if all tools can be auto-approved
# Read flags under lock, then release before any HTTP calls.
with self._lock:
if self._ws_auto_approve.get(ws_id):
self._api_approve(ws_id, approved=True)
return
auto = self._ws_auto_approve.get(ws_id, False)
approve_set = self._ws_approve_tools.get(ws_id, DEFAULT_SAFE_TOOLS)
if auto:
self._api_approve(ws_id, approved=True)
return
tool_names = {it.get("func_name", "") for it in items if it.get("needs_approval")}
if tool_names and tool_names.issubset(approve_set):
self._api_approve(ws_id, approved=True)
return
# Skip if an approval is already pending for this workstream (SSE
# reconnects re-inject the pending approval, causing duplicates).
with self._lock:
if ws_id in self._pending_approvals:
log.debug("Skipping duplicate approval for ws %s", ws_id)
return
request_id = uuid.uuid4().hex[:12]
self._pending_approvals[ws_id] = request_id
# Forward to client — spawn a thread so we don't block SSE consumption
request_id = uuid.uuid4().hex[:12]
self._publish_ws(
ws_id,
ApprovalRequestEvent(
@@ -520,30 +639,42 @@ class Bridge:
)
def _wait_approval() -> None:
raw_resp = self._broker.pop_response(request_id, timeout=self._approval_timeout)
if raw_resp:
resp_msg = InboundMessage.from_json(raw_resp)
approved = getattr(resp_msg, "approved", False)
feedback = getattr(resp_msg, "feedback", None)
always = getattr(resp_msg, "always", False)
self._api_approve(ws_id, approved=approved, feedback=feedback)
if always:
with self._lock:
self._ws_auto_approve[ws_id] = True
else:
log.warning("Approval timeout for ws %s — denying", ws_id)
self._api_approve(ws_id, approved=False, feedback="Approval timed out")
try:
raw_resp = self._broker.pop_response(request_id, timeout=self._approval_timeout)
if raw_resp:
resp_msg = InboundMessage.from_json(raw_resp)
approved = getattr(resp_msg, "approved", False)
feedback = getattr(resp_msg, "feedback", None)
always = getattr(resp_msg, "always", False)
self._api_approve(ws_id, approved=approved, feedback=feedback)
if always:
with self._lock:
self._ws_auto_approve[ws_id] = True
else:
log.warning("Approval timeout for ws %s — denying", ws_id)
self._api_approve(ws_id, approved=False, feedback="Approval timed out")
finally:
with self._lock:
self._pending_approvals.pop(ws_id, None)
threading.Thread(target=_wait_approval, daemon=True).start()
threading.Thread(target=self._run_in_context(_wait_approval), daemon=True).start()
def _handle_plan_review(self, ws_id: str, data: dict[str, Any]) -> None:
"""Handle a plan review request — auto-approve or forward to client."""
with self._lock:
if self._ws_auto_approve.get(ws_id):
self._http.post("/v1/api/plan", json={"feedback": "", "ws_id": ws_id})
auto = self._ws_auto_approve.get(ws_id, False)
# Skip if a plan review is already pending (SSE reconnect guard).
if not auto and ws_id in self._pending_plan_reviews:
log.debug("Skipping duplicate plan review for ws %s", ws_id)
return
if not auto:
request_id = uuid.uuid4().hex[:12]
self._pending_plan_reviews[ws_id] = request_id
if auto:
self._http.post("/v1/api/plan", json={"feedback": "", "ws_id": ws_id})
return
request_id = uuid.uuid4().hex[:12]
self._publish_ws(
ws_id,
PlanReviewEvent(
@@ -554,16 +685,20 @@ class Bridge:
)
def _wait_plan() -> None:
raw_resp = self._broker.pop_response(request_id, timeout=self._approval_timeout)
if raw_resp:
resp_msg = InboundMessage.from_json(raw_resp)
feedback = getattr(resp_msg, "feedback", "")
self._http.post("/v1/api/plan", json={"feedback": feedback, "ws_id": ws_id})
else:
log.warning("Plan review timeout for ws %s — rejecting", ws_id)
self._http.post("/v1/api/plan", json={"feedback": "reject", "ws_id": ws_id})
try:
raw_resp = self._broker.pop_response(request_id, timeout=self._approval_timeout)
if raw_resp:
resp_msg = InboundMessage.from_json(raw_resp)
feedback = getattr(resp_msg, "feedback", "")
self._http.post("/v1/api/plan", json={"feedback": feedback, "ws_id": ws_id})
else:
log.warning("Plan review timeout for ws %s — rejecting", ws_id)
self._http.post("/v1/api/plan", json={"feedback": "reject", "ws_id": ws_id})
finally:
with self._lock:
self._pending_plan_reviews.pop(ws_id, None)
threading.Thread(target=_wait_plan, daemon=True).start()
threading.Thread(target=self._run_in_context(_wait_plan), daemon=True).start()
def _api_approve(
self,
@@ -579,12 +714,12 @@ class Bridge:
# -- global SSE ----------------------------------------------------------
def _global_sse_loop(self) -> None:
# Own httpx client for the long-lived SSE connection
sse_headers: dict[str, str] = {}
if self._auth_token:
sse_headers["Authorization"] = f"Bearer {self._auth_token}"
# Own httpx client for the long-lived SSE connection.
# Use event hooks so auth headers refresh on each reconnect.
with httpx.Client(
base_url=self._server_url, timeout=None, headers=sse_headers
base_url=self._server_url,
timeout=None,
event_hooks={"request": [self._inject_auth]},
) as sse_client:
while self._running:
try:
@@ -703,20 +838,9 @@ def main() -> None:
default="http://localhost:8080",
help="turnstone-server URL (default: %(default)s)",
)
parser.add_argument(
"--redis-host", default="localhost", help="Redis host (default: %(default)s)"
)
parser.add_argument(
"--redis-port", type=int, default=6379, help="Redis port (default: %(default)s)"
)
parser.add_argument(
"--redis-password",
default=os.environ.get("REDIS_PASSWORD"),
help="Redis password (default: $REDIS_PASSWORD)",
)
parser.add_argument(
"--redis-db", type=int, default=0, help="Redis DB number (default: %(default)s)"
)
from turnstone.mq.broker import add_redis_args
add_redis_args(parser)
parser.add_argument(
"--approval-timeout",
type=float,
@@ -734,12 +858,9 @@ def main() -> None:
default=60,
help="Heartbeat TTL in seconds (default: %(default)s)",
)
parser.add_argument(
"--log-level",
default="INFO",
choices=["DEBUG", "INFO", "WARNING", "ERROR"],
help="Log level (default: %(default)s)",
)
from turnstone.core.log import add_log_args
add_log_args(parser)
parser.add_argument(
"--auth-token",
default=os.environ.get("TURNSTONE_AUTH_TOKEN", ""),
@@ -750,24 +871,48 @@ def main() -> None:
apply_config(parser, ["bridge", "redis", "auth"])
args = parser.parse_args()
logging.basicConfig(
level=getattr(logging, args.log_level),
format="%(asctime)s %(name)s %(levelname)s %(message)s",
)
from turnstone.core.log import configure_logging_from_args
configure_logging_from_args(args, "bridge")
from turnstone.mq.broker import broker_from_args
broker = broker_from_args(args)
# If no explicit auth token is provided, use a ServiceTokenManager
# so bridge JWTs auto-rotate (1-hour expiry, refreshed at 80%).
# A shared JWT secret is required for multi-service deployments —
# ephemeral secrets differ per process and break inter-service auth.
auth_token = args.auth_token
token_manager = None
if not auth_token:
jwt_secret = os.environ.get("TURNSTONE_JWT_SECRET", "")
if not jwt_secret:
log.error(
"TURNSTONE_JWT_SECRET is not set and no --auth-token provided. "
"The bridge cannot authenticate to the server. Set TURNSTONE_JWT_SECRET "
"to a shared secret (at least 32 characters) or pass --auth-token."
)
raise SystemExit(1)
from turnstone.core.auth import JWT_AUD_SERVER, ServiceTokenManager
token_manager = ServiceTokenManager(
user_id="bridge",
scopes=frozenset({"approve"}),
source="bridge",
secret=jwt_secret,
audience=JWT_AUD_SERVER,
expiry_hours=1,
)
log.info("bridge.jwt_minted")
broker = RedisBroker(
host=args.redis_host,
port=args.redis_port,
db=args.redis_db,
password=args.redis_password,
)
bridge = Bridge(
server_url=args.server_url,
broker=broker,
approval_timeout=args.approval_timeout,
node_id=args.node_id,
heartbeat_ttl=args.heartbeat_ttl,
auth_token=args.auth_token,
auth_token=auth_token,
token_manager=token_manager,
)
bridge.run()
+55
View File
@@ -242,3 +242,58 @@ class RedisBroker:
with contextlib.suppress(Exception):
self._pubsub.close()
self._pool.disconnect()
# ---------------------------------------------------------------------------
# CLI helpers (shared across bridge, console, channels)
# ---------------------------------------------------------------------------
def add_redis_args(parser: Any) -> None:
"""Add ``--redis-host``, ``--redis-port``, ``--redis-password``, ``--redis-db``."""
import os
parser.add_argument(
"--redis-host",
default=os.environ.get("REDIS_HOST", "localhost"),
help="Redis host (default: $REDIS_HOST or localhost)",
)
parser.add_argument(
"--redis-port",
type=int,
default=int(os.environ.get("REDIS_PORT", "6379")),
help="Redis port (default: %(default)s)",
)
parser.add_argument(
"--redis-password",
default=os.environ.get("REDIS_PASSWORD"),
help="Redis password (default: $REDIS_PASSWORD)",
)
parser.add_argument(
"--redis-db",
type=int,
default=0,
help="Redis DB number (default: %(default)s)",
)
def broker_from_args(args: Any) -> RedisBroker:
"""Create a :class:`RedisBroker` from parsed CLI arguments."""
return RedisBroker(
host=args.redis_host,
port=args.redis_port,
db=args.redis_db,
password=args.redis_password,
)
def async_broker_from_args(args: Any) -> Any:
"""Create an :class:`AsyncRedisBroker` from parsed CLI arguments."""
from turnstone.mq.async_broker import AsyncRedisBroker
return AsyncRedisBroker(
host=args.redis_host,
port=args.redis_port,
db=args.redis_db,
password=args.redis_password,
)
+16
View File
@@ -95,6 +95,7 @@ class CreateWorkstreamMessage(InboundMessage):
target_node: str = ""
model: str = ""
initial_message: str = ""
resume_session: str = ""
@dataclass
@@ -273,6 +274,10 @@ class WorkstreamCreatedEvent(OutboundEvent):
type: str = "ws_created"
name: str = ""
node_id: str = ""
resumed: bool = False
session_id: str = ""
message_count: int = 0
@dataclass
@@ -330,6 +335,16 @@ class NodeListEvent(OutboundEvent):
nodes: list[dict[str, Any]] = field(default_factory=list)
@dataclass
class SessionResumedEvent(OutboundEvent):
"""Confirmation that a session was resumed during workstream creation."""
type: str = "session_resumed"
session_id: str = ""
message_count: int = 0
name: str = ""
@dataclass
class ClusterStateEvent(OutboundEvent):
"""Workstream state change with node attribution for cluster dashboard."""
@@ -395,6 +410,7 @@ _OUTBOUND_REGISTRY: dict[str, type[OutboundEvent]] = {
ErrorEvent,
InfoEvent,
NodeListEvent,
SessionResumedEvent,
ClusterStateEvent,
]
}
+16 -11
View File
@@ -12,11 +12,20 @@ import threading
from typing import TYPE_CHECKING, Any, TypeVar
if TYPE_CHECKING:
import concurrent.futures
from collections.abc import AsyncIterator, Coroutine, Iterator
T = TypeVar("T")
_STOP = object()
async def _safe_anext(agen: AsyncIterator[T]) -> T | object:
"""Advance *agen* without raising StopAsyncIteration across a thread boundary."""
try:
return await agen.__anext__()
except StopAsyncIteration:
return _STOP
class _SyncRunner:
"""Run async coroutines synchronously via a persistent background loop."""
@@ -43,16 +52,12 @@ class _SyncRunner:
def run_iter(self, async_gen: AsyncIterator[T]) -> Iterator[T]:
"""Synchronously iterate over an async generator."""
loop = self._ensure_loop()
try:
while True:
awaitable = async_gen.__anext__()
future: concurrent.futures.Future[T] = asyncio.run_coroutine_threadsafe(
awaitable, # type: ignore[arg-type]
loop,
)
yield future.result()
except StopAsyncIteration:
return
while True:
future = asyncio.run_coroutine_threadsafe(_safe_anext(async_gen), loop)
result = future.result()
if result is _STOP:
return
yield result # type: ignore[misc]
def close(self) -> None:
"""Shut down the background event loop."""
+51 -5
View File
@@ -21,7 +21,12 @@ from turnstone.api.console_schemas import (
ConsoleHealthResponse,
NodeDetailResponse,
)
from turnstone.api.schemas import AuthLoginResponse, StatusResponse
from turnstone.api.schemas import (
AuthLoginResponse,
AuthSetupResponse,
AuthStatusResponse,
StatusResponse,
)
from turnstone.sdk._base import _BaseClient
from turnstone.sdk._sync import _SyncRunner
from turnstone.sdk.events import ClusterEvent
@@ -125,14 +130,47 @@ class AsyncTurnstoneConsole(_BaseClient):
# -- auth ----------------------------------------------------------------
async def login(self, token: str) -> AuthLoginResponse:
async def login(
self,
token: str = "",
*,
username: str = "",
password: str = "",
) -> AuthLoginResponse:
"""Authenticate via API token or username:password."""
if username and password:
body: dict[str, str] = {"username": username, "password": password}
else:
body = {"token": token}
return await self._request(
"POST",
"/v1/api/auth/login",
json_body={"token": token},
json_body=body,
response_model=AuthLoginResponse,
)
async def auth_status(self) -> AuthStatusResponse:
"""Get auth status (public -- no auth required)."""
return await self._request("GET", "/v1/api/auth/status", response_model=AuthStatusResponse)
async def setup(
self,
username: str,
display_name: str,
password: str,
) -> AuthSetupResponse:
"""First-time setup: create initial admin user (public, one-time only)."""
return await self._request(
"POST",
"/v1/api/auth/setup",
json_body={
"username": username,
"display_name": display_name,
"password": password,
},
response_model=AuthSetupResponse,
)
async def logout(self) -> StatusResponse:
return await self._request("POST", "/v1/api/auth/logout", response_model=StatusResponse)
@@ -217,8 +255,16 @@ class TurnstoneConsole:
# -- auth ----------------------------------------------------------------
def login(self, token: str) -> AuthLoginResponse:
return self._runner.run(self._async.login(token))
def login(
self, token: str = "", *, username: str = "", password: str = ""
) -> AuthLoginResponse:
return self._runner.run(self._async.login(token, username=username, password=password))
def auth_status(self) -> AuthStatusResponse:
return self._runner.run(self._async.auth_status())
def setup(self, username: str, display_name: str, password: str) -> AuthSetupResponse:
return self._runner.run(self._async.setup(username, display_name, password))
def logout(self) -> StatusResponse:
return self._runner.run(self._async.logout())
+58 -6
View File
@@ -16,7 +16,12 @@ import asyncio
import contextlib
from typing import TYPE_CHECKING, Any
from turnstone.api.schemas import AuthLoginResponse, StatusResponse
from turnstone.api.schemas import (
AuthLoginResponse,
AuthSetupResponse,
AuthStatusResponse,
StatusResponse,
)
from turnstone.api.server_schemas import (
CreateWorkstreamResponse,
DashboardResponse,
@@ -71,6 +76,7 @@ class AsyncTurnstoneServer(_BaseClient):
name: str = "",
model: str = "",
auto_approve: bool = False,
resume_session: str = "",
) -> CreateWorkstreamResponse:
body: dict[str, Any] = {}
if name:
@@ -79,6 +85,8 @@ class AsyncTurnstoneServer(_BaseClient):
body["model"] = model
if auto_approve:
body["auto_approve"] = True
if resume_session:
body["resume_session"] = resume_session
return await self._request(
"POST",
"/v1/api/workstreams/new",
@@ -213,14 +221,47 @@ class AsyncTurnstoneServer(_BaseClient):
# -- auth ----------------------------------------------------------------
async def login(self, token: str) -> AuthLoginResponse:
async def login(
self,
token: str = "",
*,
username: str = "",
password: str = "",
) -> AuthLoginResponse:
"""Authenticate via API token or username:password."""
if username and password:
body: dict[str, str] = {"username": username, "password": password}
else:
body = {"token": token}
return await self._request(
"POST",
"/v1/api/auth/login",
json_body={"token": token},
json_body=body,
response_model=AuthLoginResponse,
)
async def auth_status(self) -> AuthStatusResponse:
"""Get auth status (public -- no auth required)."""
return await self._request("GET", "/v1/api/auth/status", response_model=AuthStatusResponse)
async def setup(
self,
username: str,
display_name: str,
password: str,
) -> AuthSetupResponse:
"""First-time setup: create initial admin user (public, one-time only)."""
return await self._request(
"POST",
"/v1/api/auth/setup",
json_body={
"username": username,
"display_name": display_name,
"password": password,
},
response_model=AuthSetupResponse,
)
async def logout(self) -> StatusResponse:
return await self._request("POST", "/v1/api/auth/logout", response_model=StatusResponse)
@@ -266,9 +307,12 @@ class TurnstoneServer:
name: str = "",
model: str = "",
auto_approve: bool = False,
resume_session: str = "",
) -> CreateWorkstreamResponse:
return self._runner.run(
self._async.create_workstream(name=name, model=model, auto_approve=auto_approve)
self._async.create_workstream(
name=name, model=model, auto_approve=auto_approve, resume_session=resume_session
)
)
def close_workstream(self, ws_id: str) -> StatusResponse:
@@ -326,8 +370,16 @@ class TurnstoneServer:
# -- auth ----------------------------------------------------------------
def login(self, token: str) -> AuthLoginResponse:
return self._runner.run(self._async.login(token))
def login(
self, token: str = "", *, username: str = "", password: str = ""
) -> AuthLoginResponse:
return self._runner.run(self._async.login(token, username=username, password=password))
def auth_status(self) -> AuthStatusResponse:
return self._runner.run(self._async.auth_status())
def setup(self, username: str, display_name: str, password: str) -> AuthSetupResponse:
return self._runner.run(self._async.setup(username, display_name, password))
def logout(self) -> StatusResponse:
return self._runner.run(self._async.logout())
+222 -119
View File
@@ -19,10 +19,12 @@ import json
import logging
import os
import queue
import socket
import sys
import textwrap
import threading
import time
import uuid
from contextlib import asynccontextmanager
from pathlib import Path
from typing import TYPE_CHECKING, Any
@@ -30,7 +32,6 @@ from typing import TYPE_CHECKING, Any
from sse_starlette import EventSourceResponse
from starlette.applications import Starlette
from starlette.middleware import Middleware
from starlette.middleware.cors import CORSMiddleware
from starlette.requests import Request
from starlette.responses import HTMLResponse, JSONResponse, Response
from starlette.routing import Mount, Route
@@ -39,6 +40,7 @@ from starlette.staticfiles import StaticFiles
from turnstone import __version__
from turnstone.api.docs import make_docs_handler, make_openapi_handler
from turnstone.api.server_spec import build_server_spec
from turnstone.core.auth import JWT_AUD_SERVER, AuthMiddleware
from turnstone.core.metrics import metrics as _metrics
from turnstone.core.ratelimit import resolve_client_ip
from turnstone.core.session import ChatSession, SessionUI # noqa: F401
@@ -59,8 +61,6 @@ log = logging.getLogger(__name__)
_STATIC_DIR = Path(__file__).parent / "ui" / "static"
_SHARED_DIR = Path(__file__).parent / "shared_static"
_HTML = (_STATIC_DIR / "index.html").read_text(encoding="utf-8")
_CSS = (_STATIC_DIR / "style.css").read_text(encoding="utf-8")
_JS = (_STATIC_DIR / "app.js").read_text(encoding="utf-8")
# ---------------------------------------------------------------------------
@@ -335,36 +335,6 @@ def _build_history(
# ---------------------------------------------------------------------------
class AuthMiddleware:
"""Check auth tokens on every HTTP request."""
def __init__(self, app: ASGIApp) -> None:
self.app = app
async def __call__(self, scope: Scope, receive: Receive, send: Send) -> None:
if scope["type"] != "http":
await self.app(scope, receive, send)
return
request = Request(scope)
# Skip auth for CORS preflight — CORSMiddleware handles it
if request.method == "OPTIONS":
await self.app(scope, receive, send)
return
from turnstone.core.auth import check_request
auth_config = request.app.state.auth_config
method = request.method
path = request.url.path
auth_header = request.headers.get("Authorization")
cookie_header = request.headers.get("Cookie")
allowed, status, msg = check_request(auth_config, method, path, auth_header, cookie_header)
if not allowed:
response = JSONResponse({"error": msg}, status_code=status)
await response(scope, receive, send)
return
await self.app(scope, receive, send)
class RateLimitMiddleware:
"""Per-IP token-bucket rate limiting."""
@@ -433,20 +403,40 @@ class MetricsMiddleware:
)
class LogContextMiddleware:
"""Set structlog context variables (request_id, ws_id) per request."""
def __init__(self, app: ASGIApp) -> None:
self.app = app
async def __call__(self, scope: Scope, receive: Receive, send: Send) -> None:
if scope["type"] != "http":
await self.app(scope, receive, send)
return
import structlog
from turnstone.core.log import ctx_request_id, ctx_ws_id
rid = uuid.uuid4().hex[:8]
tok_rid = ctx_request_id.set(rid)
# Extract ws_id from query params if present
request = Request(scope)
ws_id = request.query_params.get("ws_id", "")
tok_ws = ctx_ws_id.set(ws_id) if ws_id else None
try:
await self.app(scope, receive, send)
finally:
ctx_request_id.reset(tok_rid)
if tok_ws is not None:
ctx_ws_id.reset(tok_ws)
structlog.contextvars.clear_contextvars()
# ---------------------------------------------------------------------------
# Helpers
# ---------------------------------------------------------------------------
async def _read_json(request: Request) -> dict[str, Any]:
"""Read JSON body from request, returning {} on invalid/missing JSON."""
try:
body: dict[str, Any] = await request.json()
return body
except (ValueError, json.JSONDecodeError):
return {}
# ---------------------------------------------------------------------------
# Helper — workstream lookup (replaces self._get_ws on the old handler)
# ---------------------------------------------------------------------------
@@ -653,31 +643,33 @@ async def list_sessions_endpoint(request: Request) -> JSONResponse:
"created": created,
"updated": updated,
"message_count": count,
"node_id": node_id,
"ws_id": ws_id,
}
for sid, alias, title, created, updated, count in rows
for sid, alias, title, created, updated, count, node_id, ws_id in rows
]
return JSONResponse({"sessions": sessions})
def _count_ws_states(wss: list[Workstream]) -> dict[str, int]:
"""Count workstream states for health/metrics endpoints."""
counts = dict.fromkeys(("idle", "thinking", "running", "attention", "error"), 0)
for ws in wss:
counts[ws.state.value] = counts.get(ws.state.value, 0) + 1
return counts
async def health(request: Request) -> JSONResponse:
"""GET /health — server health status."""
mgr: WorkstreamManager = request.app.state.workstreams
wss = mgr.list_all()
states: dict[str, int] = {
"idle": 0,
"thinking": 0,
"running": 0,
"attention": 0,
"error": 0,
}
for ws in wss:
state = ws.state.value
states[state] = states.get(state, 0) + 1
states = _count_ws_states(wss)
monitor = getattr(request.app.state, "health_monitor", None)
backend_ok = monitor.is_healthy if monitor else True
data: dict[str, Any] = {
"status": "ok" if backend_ok else "degraded",
"version": __version__,
"node_id": getattr(request.app.state, "node_id", ""),
"uptime_seconds": round(time.monotonic() - _metrics.start_time, 2),
"model": _metrics.model,
"workstreams": {"total": len(wss), **states},
@@ -693,17 +685,9 @@ async def metrics_endpoint(request: Request) -> Response:
"""GET /metrics — Prometheus text exposition format."""
mgr: WorkstreamManager = request.app.state.workstreams
wss = mgr.list_all()
states: dict[str, int] = {
"idle": 0,
"thinking": 0,
"running": 0,
"attention": 0,
"error": 0,
}
states = _count_ws_states(wss)
ws_data = []
for ws in wss:
state = ws.state.value
states[state] = states.get(state, 0) + 1
ui: WebUI = ws.ui # type: ignore[assignment]
with ui._ws_lock:
ws_data.append(
@@ -728,7 +712,11 @@ async def metrics_endpoint(request: Request) -> Response:
async def send_message(request: Request) -> JSONResponse:
"""POST /v1/api/send — send a user message to the workstream."""
body = await _read_json(request)
from turnstone.core.web_helpers import read_json_or_400
body = await read_json_or_400(request)
if isinstance(body, JSONResponse):
return body
message = body.get("message", "").strip()
ws_id = body.get("ws_id")
if not message:
@@ -768,7 +756,11 @@ async def send_message(request: Request) -> JSONResponse:
async def approve(request: Request) -> JSONResponse:
"""POST /v1/api/approve — approve or deny a tool call."""
body = await _read_json(request)
from turnstone.core.web_helpers import read_json_or_400
body = await read_json_or_400(request)
if isinstance(body, JSONResponse):
return body
approved = body.get("approved", False)
feedback = body.get("feedback")
always = body.get("always", False)
@@ -785,7 +777,11 @@ async def approve(request: Request) -> JSONResponse:
async def plan_feedback(request: Request) -> JSONResponse:
"""POST /v1/api/plan — respond to a plan review."""
body = await _read_json(request)
from turnstone.core.web_helpers import read_json_or_400
body = await read_json_or_400(request)
if isinstance(body, JSONResponse):
return body
feedback = body.get("feedback", "")
ws_id = body.get("ws_id")
mgr = request.app.state.workstreams
@@ -798,7 +794,11 @@ async def plan_feedback(request: Request) -> JSONResponse:
async def command(request: Request) -> JSONResponse:
"""POST /v1/api/command — execute a slash command."""
body = await _read_json(request)
from turnstone.core.web_helpers import read_json_or_400
body = await read_json_or_400(request)
if isinstance(body, JSONResponse):
return body
cmd = body.get("command", "").strip()
ws_id = body.get("ws_id")
if not cmd:
@@ -839,7 +839,11 @@ async def command(request: Request) -> JSONResponse:
async def create_workstream(request: Request) -> JSONResponse:
"""POST /v1/api/workstreams/new — create a new workstream."""
body = await _read_json(request)
from turnstone.core.web_helpers import read_json_or_400
body = await read_json_or_400(request)
if isinstance(body, JSONResponse):
return body
mgr: WorkstreamManager = request.app.state.workstreams
skip: bool = request.app.state.skip_permissions
try:
@@ -864,14 +868,47 @@ async def create_workstream(request: Request) -> JSONResponse:
"reason": "evicted",
}
)
return JSONResponse({"ws_id": ws.id, "name": ws.name})
# Atomic session resume during creation.
resumed = False
message_count = 0
session_id = ""
resume_session_id = body.get("resume_session", "")
if resume_session_id and ws.session is not None:
from turnstone.core.memory import get_session_name, resolve_session
target_id = resolve_session(resume_session_id)
if target_id and ws.session.resume_session(target_id):
resumed = True
session_id = target_id
message_count = len(ws.session.messages)
ws.name = get_session_name(target_id) or ws.name
ui = ws.ui
if isinstance(ui, WebUI):
ui._enqueue({"type": "clear_ui"})
history = _build_history(ws.session)
if history:
ui._enqueue({"type": "history", "messages": history})
return JSONResponse(
{
"ws_id": ws.id,
"name": ws.name,
"resumed": resumed,
"session_id": session_id,
"message_count": message_count,
}
)
except RuntimeError as e:
return JSONResponse({"error": str(e)}, status_code=400)
async def close_workstream(request: Request) -> JSONResponse:
"""POST /v1/api/workstreams/close — close a workstream."""
body = await _read_json(request)
from turnstone.core.web_helpers import read_json_or_400
body = await read_json_or_400(request)
if isinstance(body, JSONResponse):
return body
ws_id = str(body.get("ws_id", ""))
mgr = request.app.state.workstreams
if mgr.close(ws_id):
@@ -880,39 +917,31 @@ async def close_workstream(request: Request) -> JSONResponse:
async def auth_login(request: Request) -> Response:
"""POST /v1/api/auth/login — authenticate with a token."""
from turnstone.core.auth import make_set_cookie
"""POST /v1/api/auth/login — authenticate and return JWT."""
from turnstone.core.auth import handle_auth_login
body = await _read_json(request)
token = body.get("token", "")
auth_config = request.app.state.auth_config
role = auth_config.check(token)
if role:
response = JSONResponse({"status": "ok", "role": role})
response.headers["Set-Cookie"] = make_set_cookie(token)
return response
return JSONResponse({"error": "Invalid token"}, status_code=401)
return await handle_auth_login(request, JWT_AUD_SERVER)
async def auth_logout(request: Request) -> Response:
"""POST /v1/api/auth/logout — clear auth cookie."""
from turnstone.core.auth import make_clear_cookie
from turnstone.core.auth import handle_auth_logout
response = JSONResponse({"status": "ok"})
response.headers["Set-Cookie"] = make_clear_cookie()
return response
return await handle_auth_logout(request)
# ---------------------------------------------------------------------------
# Model auto-detection (shared with cli.py)
# ---------------------------------------------------------------------------
async def auth_status(request: Request) -> Response:
"""GET /v1/api/auth/status — public endpoint for login UI state detection."""
from turnstone.core.auth import handle_auth_status
return await handle_auth_status(request)
def detect_model(client: Any, provider: str = "openai") -> tuple[str, int | None]:
"""Auto-detect model — delegates to :func:`turnstone.core.model_registry.detect_model`."""
from turnstone.core.model_registry import detect_model as _detect
async def auth_setup(request: Request) -> Response:
"""POST /v1/api/auth/setup — create first admin user (public, one-time only)."""
from turnstone.core.auth import handle_auth_setup
return _detect(client, provider=provider)
return await handle_auth_setup(request, JWT_AUD_SERVER)
# ---------------------------------------------------------------------------
@@ -1003,6 +1032,25 @@ async def _lifespan(app: Starlette) -> AsyncGenerator[None, None]:
# ---------------------------------------------------------------------------
def _build_middleware(cors_origins: list[str] | None = None) -> list[Middleware]:
"""Build the middleware stack with optional CORS."""
stack: list[Middleware] = [
Middleware(LogContextMiddleware),
Middleware(MetricsMiddleware),
]
if cors_origins:
from turnstone.core.web_helpers import cors_middleware
stack.append(cors_middleware(cors_origins))
stack.extend(
[
Middleware(AuthMiddleware, jwt_audience=JWT_AUD_SERVER),
Middleware(RateLimitMiddleware),
]
)
return stack
def create_app(
*,
workstreams: WorkstreamManager,
@@ -1011,11 +1059,15 @@ def create_app(
global_listeners_lock: threading.Lock,
skip_permissions: bool,
auth_config: Any,
jwt_secret: str = "",
auth_storage: Any = None,
health_monitor: Any = None,
rate_limiter: Any = None,
mcp_client: Any = None,
registry: Any = None,
idle_timeout: int = 0,
node_id: str = "",
cors_origins: list[str] | None = None,
) -> Starlette:
"""Create and configure the Starlette ASGI application."""
_spec = build_server_spec()
@@ -1041,6 +1093,8 @@ def create_app(
Route("/api/workstreams/close", close_workstream, methods=["POST"]),
Route("/api/auth/login", auth_login, methods=["POST"]),
Route("/api/auth/logout", auth_logout, methods=["POST"]),
Route("/api/auth/status", auth_status),
Route("/api/auth/setup", auth_setup, methods=["POST"]),
],
),
Route("/health", health),
@@ -1050,17 +1104,7 @@ def create_app(
Mount("/static", app=StaticFiles(directory=str(_STATIC_DIR)), name="static"),
Mount("/shared", app=StaticFiles(directory=str(_SHARED_DIR)), name="shared"),
],
middleware=[
Middleware(MetricsMiddleware),
Middleware(
CORSMiddleware,
allow_origins=["*"],
allow_methods=["GET", "POST", "OPTIONS"],
allow_headers=["Content-Type", "Authorization"],
),
Middleware(AuthMiddleware),
Middleware(RateLimitMiddleware),
],
middleware=_build_middleware(cors_origins),
lifespan=_lifespan,
)
app.state.workstreams = workstreams
@@ -1069,11 +1113,18 @@ def create_app(
app.state.global_listeners_lock = global_listeners_lock
app.state.skip_permissions = skip_permissions
app.state.auth_config = auth_config
app.state.jwt_secret = jwt_secret
app.state.auth_storage = auth_storage
app.state.health_monitor = health_monitor
app.state.rate_limiter = rate_limiter
app.state.mcp_client = mcp_client
app.state.registry = registry
app.state.idle_timeout = idle_timeout
app.state.node_id = node_id
from turnstone.core.auth import LoginRateLimiter
app.state.login_limiter = LoginRateLimiter()
return app
@@ -1269,6 +1320,9 @@ def main() -> None:
default=60.0,
help="Circuit breaker cooldown in seconds (default: 60)",
)
from turnstone.core.log import add_log_args
add_log_args(parser)
from turnstone.core.config import apply_config
apply_config(
@@ -1277,6 +1331,10 @@ def main() -> None:
)
args = parser.parse_args()
from turnstone.core.log import configure_logging_from_args
configure_logging_from_args(args, "server")
# Initialize storage backend
from turnstone.core.storage import init_storage
@@ -1312,13 +1370,15 @@ def main() -> None:
model = args.model
detected_ctx = None
else:
from turnstone.core.model_registry import detect_model
model, detected_ctx = detect_model(client, provider=provider_name)
# Use detected context window when the user hasn't explicitly set one
context_window = args.context_window
if detected_ctx and context_window == 131072: # default unchanged
context_window = detected_ctx
print(f"Context window: {context_window:,} (detected from backend)")
log.info("Context window: %s (detected from backend)", f"{context_window:,}")
# Build model registry (reads [models.*] sections from config.toml)
from turnstone.core.model_registry import load_model_registry
@@ -1364,8 +1424,30 @@ def main() -> None:
global_listeners_lock = threading.Lock()
WebUI._global_queue = global_queue
# Server-owned node identity
def _default_node_id() -> str:
"""Generate a node_id: ``{hostname}_{4hex}``, or a UUID on failure."""
suffix = uuid.uuid4().hex[:4]
try:
host = socket.gethostname()
if host and host != "localhost":
return f"{host}_{suffix}"
except OSError:
pass
return uuid.uuid4().hex[:12]
_node_id = os.environ.get("TURNSTONE_NODE_ID") or _default_node_id()
from turnstone.core.log import ctx_node_id
ctx_node_id.set(_node_id)
# Session factory — captures shared config
def session_factory(ui: SessionUI | None, model_alias: str | None = None) -> ChatSession:
def session_factory(
ui: SessionUI | None,
model_alias: str | None = None,
ws_id: str | None = None,
) -> ChatSession:
assert ui is not None
r_client, r_model, r_cfg = registry.resolve(model_alias)
return ChatSession(
@@ -1386,10 +1468,14 @@ def main() -> None:
registry=registry,
model_alias=model_alias or registry.default,
health_monitor=health_monitor,
node_id=_node_id,
ws_id=ws_id,
)
# Create workstream manager and initial workstream
manager = WorkstreamManager(session_factory, max_workstreams=args.max_workstreams)
manager = WorkstreamManager(
session_factory, max_workstreams=args.max_workstreams, node_id=_node_id
)
WebUI._workstream_mgr = manager
ws = manager.create(
name="default",
@@ -1406,24 +1492,30 @@ def main() -> None:
target_id = resolve_session(args.resume)
if not target_id:
print(f"Session not found: {args.resume}")
log.error("Session not found: %s", args.resume)
sys.exit(1)
if not ws.session.resume_session(target_id):
print(f"Session '{args.resume}' has no messages.")
log.error("Session '%s' has no messages.", args.resume)
sys.exit(1)
print(f"Resumed session {target_id} ({len(ws.session.messages)} messages)")
log.info("Resumed session %s (%d messages)", target_id, len(ws.session.messages))
# Record detected model in metrics
_metrics.model = model
# Auth config
from turnstone.core.auth import load_auth_config
from turnstone.core.auth import load_auth_config, load_jwt_secret
from turnstone.core.storage import get_storage
auth_config = load_auth_config()
jwt_secret = load_jwt_secret() if auth_config.enabled else ""
if auth_config.enabled:
print(f"Auth: enabled ({len(auth_config.tokens)} token(s) configured)")
log.info("Auth: enabled (%d config token(s))", len(auth_config.tokens))
# Build the ASGI app
from turnstone.core.web_helpers import parse_cors_origins
cors_origins = parse_cors_origins()
app = create_app(
workstreams=manager,
global_queue=global_queue,
@@ -1431,25 +1523,30 @@ def main() -> None:
global_listeners_lock=global_listeners_lock,
skip_permissions=args.skip_permissions,
auth_config=auth_config,
jwt_secret=jwt_secret,
auth_storage=get_storage(),
health_monitor=health_monitor,
rate_limiter=rate_limiter,
mcp_client=mcp_client,
registry=registry,
idle_timeout=args.workstream_idle_timeout,
node_id=_node_id,
cors_origins=cors_origins,
)
print(f"turnstone web server running on http://{args.host}:{args.port}")
print(f"Model: {model}")
log.info("Server starting on http://%s:%s", args.host, args.port)
log.info("Model: %s", model)
if registry.count > 1:
others = [a for a in registry.list_aliases() if a != registry.default]
print(f"Models: {registry.default} (default), {', '.join(others)}")
log.info("Models: %s (default), %s", registry.default, ", ".join(others))
if mcp_client:
mcp_tools = mcp_client.get_tools()
if mcp_tools:
print(f"MCP tools: {len(mcp_tools)} from {mcp_client.server_count} server(s)")
print(
f"Health monitor: probe every {args.health_probe_interval}s, "
f"circuit breaker threshold={args.circuit_breaker_threshold}"
log.info("MCP tools: %d from %d server(s)", len(mcp_tools), mcp_client.server_count)
log.info(
"Health monitor: probe every %ss, circuit breaker threshold=%s",
args.health_probe_interval,
args.circuit_breaker_threshold,
)
if rate_limiter.enabled:
proxy_info = (
@@ -1457,8 +1554,14 @@ def main() -> None:
if args.ratelimit_trusted_proxies
else ""
)
print(f"Rate limiter: {args.ratelimit_rps} req/s, burst={args.ratelimit_burst}{proxy_info}")
print(f"Max workstreams: {args.max_workstreams}")
log.info(
"Rate limiter: %s req/s, burst=%s%s",
args.ratelimit_rps,
args.ratelimit_burst,
proxy_info,
)
log.info("Max workstreams: %s", args.max_workstreams)
log.info("Node ID: %s", _node_id)
print("Press Ctrl+C to stop.")
import uvicorn
+303 -70
View File
@@ -1,10 +1,17 @@
/* Shared auth system turnstone design system
Configure: window.TURNSTONE_AUTH_TITLE (default "turnstone")
Hooks: window.onLoginSuccess() and window.onLogout() */
Hooks: window.onLoginSuccess() and window.onLogout()
Flows:
1. Check /v1/api/auth/status detect if setup is needed
2. If setup_required show first-time setup wizard (create admin user)
3. If auth_enabled + has_users show login (username:password)
4. Legacy: token-based login still supported via toggle */
var _AUTH_TITLE = window.TURNSTONE_AUTH_TITLE || "turnstone";
var _loginTrapHandler = null;
var _loginBusy = false;
var _authMode = "login"; // "login", "setup", "token"
async function authFetch(url, opts) {
var maxRetries = 2;
@@ -22,6 +29,10 @@ async function authFetch(url, opts) {
});
continue;
}
// Successful auth — ensure logout button and SSE connection
var _lb = document.getElementById("logout-btn");
if (_lb) _lb.style.display = "";
if (typeof _ensureSSE === "function") _ensureSSE();
return r;
}
}
@@ -33,30 +44,134 @@ function initLogin() {
overlay.setAttribute("role", "dialog");
overlay.setAttribute("aria-modal", "true");
overlay.setAttribute("aria-labelledby", "login-title");
overlay.innerHTML =
'<div id="login-box">' +
overlay.innerHTML = _buildLoginHTML();
document.body.appendChild(overlay);
_bindLoginEvents();
}
function _buildLoginHTML() {
return (
'<form id="login-box">' +
'<h2 id="login-title">' +
escapeHtml(_AUTH_TITLE) +
"</h2>" +
'<div id="login-subtitle" class="login-subtitle"></div>' +
'<div id="login-error" role="alert" aria-live="assertive"></div>' +
'<label for="login-token" class="sr-only">Auth token</label>' +
'<input id="login-token" type="password" placeholder="Enter auth token" autocomplete="off">' +
'<button id="login-submit">Sign in</button>' +
"</div>";
document.body.appendChild(overlay);
document.getElementById("login-submit").onclick = submitLogin;
document
.getElementById("login-token")
.addEventListener("keydown", function (e) {
if (e.key === "Enter") submitLogin();
if (e.key === "Escape") {
var errEl = document.getElementById("login-error");
if (errEl && errEl.style.display !== "none") {
errEl.style.display = "none";
errEl.textContent = "";
}
}
// --- Setup mode fields ---
'<div id="setup-fields" style="display:none">' +
'<label for="setup-username" class="login-label">Username</label>' +
'<input id="setup-username" name="username" type="text" placeholder="admin" autocomplete="username" spellcheck="false">' +
'<label for="setup-displayname" class="login-label">Display name</label>' +
'<input id="setup-displayname" name="display_name" type="text" placeholder="Administrator" autocomplete="name">' +
'<label for="setup-password" class="login-label">Password</label>' +
'<input id="setup-password" name="password" type="password" placeholder="Choose a strong password" autocomplete="new-password">' +
'<label for="setup-confirm" class="login-label">Confirm password</label>' +
'<input id="setup-confirm" name="confirm" type="password" placeholder="Confirm password" autocomplete="new-password">' +
"</div>" +
// --- Login mode fields ---
'<div id="login-fields">' +
'<label for="login-username" class="login-label">Username</label>' +
'<input id="login-username" name="username" type="text" placeholder="Username" autocomplete="username" spellcheck="false">' +
'<label for="login-password" class="login-label">Password</label>' +
'<input id="login-password" name="password" type="password" placeholder="Password" autocomplete="current-password">' +
"</div>" +
// --- Token mode fields ---
'<div id="token-fields" style="display:none">' +
'<label for="login-token" class="login-label">Auth token</label>' +
'<input id="login-token" name="token" type="password" placeholder="Enter auth token" autocomplete="off">' +
"</div>" +
'<button id="login-submit" type="submit">Sign in</button>' +
// --- Mode toggle ---
'<div id="login-toggle" class="login-toggle">' +
'<button id="toggle-token" class="login-link" type="button">Use token instead</button>' +
"</div>" +
"</form>"
);
}
function _bindLoginEvents() {
// Handle form submission (button click, Enter key, and password manager fill)
document.getElementById("login-box").addEventListener("submit", function (e) {
e.preventDefault();
_handleSubmit();
});
// Escape key clears errors
var inputs = document.querySelectorAll("#login-box input");
for (var i = 0; i < inputs.length; i++) {
inputs[i].addEventListener("keydown", function (e) {
if (e.key === "Escape") _clearError();
});
}
// Mode toggle
document.getElementById("toggle-token").onclick = function () {
if (_authMode === "login") {
_switchMode("token");
} else if (_authMode === "token") {
_switchMode("login");
}
};
}
function _switchMode(mode) {
_authMode = mode;
var setupFields = document.getElementById("setup-fields");
var loginFields = document.getElementById("login-fields");
var tokenFields = document.getElementById("token-fields");
var toggleDiv = document.getElementById("login-toggle");
var toggleBtn = document.getElementById("toggle-token");
var subtitle = document.getElementById("login-subtitle");
var btn = document.getElementById("login-submit");
setupFields.style.display = "none";
loginFields.style.display = "none";
tokenFields.style.display = "none";
_clearError();
if (mode === "setup") {
setupFields.style.display = "";
toggleDiv.style.display = "none";
subtitle.textContent = "Create the first admin account";
btn.textContent = "Create account";
setTimeout(function () {
document.getElementById("setup-username").focus();
}, 50);
} else if (mode === "login") {
loginFields.style.display = "";
toggleDiv.style.display = "";
toggleBtn.textContent = "Use token instead";
subtitle.textContent = "";
btn.textContent = "Sign in";
setTimeout(function () {
document.getElementById("login-username").focus();
}, 50);
} else if (mode === "token") {
tokenFields.style.display = "";
toggleDiv.style.display = "";
toggleBtn.textContent = "Use password instead";
subtitle.textContent = "";
btn.textContent = "Sign in";
setTimeout(function () {
document.getElementById("login-token").focus();
}, 50);
}
}
function _clearError() {
var errEl = document.getElementById("login-error");
if (errEl && errEl.style.display !== "none") {
errEl.style.display = "none";
errEl.textContent = "";
}
}
function _showError(msg) {
var errEl = document.getElementById("login-error");
if (errEl) {
errEl.textContent = msg;
errEl.style.display = "block";
}
}
function showLogin() {
@@ -66,26 +181,42 @@ function showLogin() {
document.body.style.overflow = "hidden";
var logoutBtn = document.getElementById("logout-btn");
if (logoutBtn) logoutBtn.style.display = "none";
var errEl = document.getElementById("login-error");
if (errEl) {
errEl.style.display = "none";
errEl.textContent = "";
}
setTimeout(function () {
var inp = document.getElementById("login-token");
if (inp) {
inp.value = "";
inp.focus();
}
}, 50);
_clearError();
// Check auth status to determine mode
fetch("/v1/api/auth/status")
.then(function (r) {
return r.json();
})
.then(function (data) {
if (data.setup_required) {
_switchMode("setup");
} else {
_switchMode("login");
}
})
.catch(function () {
// Fallback to login mode
_switchMode("login");
});
// Keyboard trap
if (_loginTrapHandler)
document.removeEventListener("keydown", _loginTrapHandler);
_loginTrapHandler = function (e) {
if (e.key === "Tab") {
var box = document.getElementById("login-box");
var focusable = box.querySelectorAll("input, button");
var first = focusable[0];
var last = focusable[focusable.length - 1];
var focusable = box.querySelectorAll(
'input:not([style*="display: none"]):not([style*="display:none"]), button:not([style*="display: none"]):not([style*="display:none"])',
);
// Filter to visible elements
var visible = [];
for (var i = 0; i < focusable.length; i++) {
if (focusable[i].offsetParent !== null) visible.push(focusable[i]);
}
if (visible.length === 0) return;
var first = visible[0];
var last = visible[visible.length - 1];
if (e.shiftKey) {
if (document.activeElement === first) {
e.preventDefault();
@@ -112,26 +243,59 @@ function hideLogin() {
}
}
function submitLogin() {
function _handleSubmit() {
if (_loginBusy) return;
var token = (document.getElementById("login-token").value || "").trim();
if (!token) {
var errEl = document.getElementById("login-error");
if (errEl) {
errEl.textContent = "Token is required";
errEl.style.display = "block";
}
document.getElementById("login-token").focus();
if (_authMode === "setup") return _submitSetup();
if (_authMode === "token") return _submitToken();
return _submitLogin();
}
function _submitLogin() {
var username = (document.getElementById("login-username").value || "").trim();
var password = document.getElementById("login-password").value || "";
if (!username) {
_showError("Username is required");
return;
}
if (!password) {
_showError("Password is required");
return;
}
_loginBusy = true;
var btn = document.getElementById("login-submit");
var inp = document.getElementById("login-token");
btn.disabled = true;
btn.textContent = "Signing in\u2026";
inp.disabled = true;
_setBusy(true);
fetch("/v1/api/auth/login", {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({ username: username, password: password }),
})
.then(function (r) {
if (r.status === 401 || r.status === 403) throw new Error("invalid");
if (!r.ok) throw new Error("server");
return r.json();
})
.then(function () {
_setBusy(false);
_onSuccess();
})
.catch(function (err) {
_setBusy(false);
_showError(
err.message === "invalid"
? "Invalid username or password"
: "Connection failed \u2014 try again",
);
});
}
function _submitToken() {
var token = (document.getElementById("login-token").value || "").trim();
if (!token) {
_showError("Token is required");
return;
}
_setBusy(true);
fetch("/v1/api/auth/login", {
method: "POST",
headers: { "Content-Type": "application/json" },
@@ -143,31 +307,100 @@ function submitLogin() {
return r.json();
})
.then(function () {
_loginBusy = false;
btn.disabled = false;
btn.textContent = "Sign in";
inp.disabled = false;
hideLogin();
var logoutBtn = document.getElementById("logout-btn");
if (logoutBtn) logoutBtn.style.display = "";
if (typeof window.onLoginSuccess === "function") window.onLoginSuccess();
_setBusy(false);
_onSuccess();
})
.catch(function (err) {
_loginBusy = false;
btn.disabled = false;
btn.textContent = "Sign in";
inp.disabled = false;
var errEl = document.getElementById("login-error");
if (errEl) {
errEl.textContent =
err.message === "invalid"
? "Invalid token"
: "Connection failed \u2014 try again";
errEl.style.display = "block";
}
_setBusy(false);
_showError(
err.message === "invalid"
? "Invalid token"
: "Connection failed \u2014 try again",
);
});
}
function _submitSetup() {
var username = (document.getElementById("setup-username").value || "").trim();
var displayName = (
document.getElementById("setup-displayname").value || ""
).trim();
var password = document.getElementById("setup-password").value || "";
var confirm = document.getElementById("setup-confirm").value || "";
if (!username) {
_showError("Username is required");
return;
}
if (!displayName) {
_showError("Display name is required");
return;
}
if (!password) {
_showError("Password is required");
return;
}
if (password.length < 8) {
_showError("Password must be at least 8 characters");
return;
}
if (password !== confirm) {
_showError("Passwords do not match");
return;
}
_setBusy(true, "Creating account\u2026");
// Use the public setup endpoint (creates user + returns JWT in one step)
fetch("/v1/api/auth/setup", {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({
username: username,
display_name: displayName,
password: password,
}),
})
.then(function (r) {
if (r.status === 409) throw new Error("Setup already completed");
if (!r.ok)
return r.json().then(function (d) {
throw new Error(d.error || "Failed to create account");
});
return r.json();
})
.then(function () {
_setBusy(false);
_onSuccess();
})
.catch(function (err) {
_setBusy(false);
_showError(err.message || "Setup failed \u2014 try again");
});
}
function _setBusy(busy, label) {
_loginBusy = busy;
var btn = document.getElementById("login-submit");
var inputs = document.querySelectorAll("#login-box input");
btn.disabled = busy;
if (busy) {
btn.textContent = label || "Signing in\u2026";
} else {
btn.textContent = _authMode === "setup" ? "Create account" : "Sign in";
}
for (var i = 0; i < inputs.length; i++) {
inputs[i].disabled = busy;
}
}
function _onSuccess() {
hideLogin();
var logoutBtn = document.getElementById("logout-btn");
if (logoutBtn) logoutBtn.style.display = "";
if (typeof window.onLoginSuccess === "function") window.onLoginSuccess();
}
function logout() {
fetch("/v1/api/auth/logout", { method: "POST" }).then(function () {
if (typeof window.onLogout === "function") window.onLogout();
+41 -6
View File
@@ -90,6 +90,7 @@ html, body {
body {
display: flex;
flex-direction: column;
overflow: hidden;
background-image:
radial-gradient(ellipse at 20% 0%, rgba(229, 160, 66, 0.03) 0%, transparent 50%),
radial-gradient(ellipse at 80% 100%, rgba(103, 232, 249, 0.02) 0%, transparent 50%);
@@ -295,12 +296,11 @@ body {
background: linear-gradient(90deg, transparent, var(--accent), transparent);
border-radius: 1px;
}
#login-box h2 {
#login-box > h2 {
font-family: var(--font-display);
color: var(--accent);
font-size: 16px;
font-weight: 700;
margin-bottom: 20px;
letter-spacing: 0.02em;
}
#login-box input {
@@ -317,7 +317,7 @@ body {
}
#login-box input:focus-visible { border-color: var(--accent); outline: none; box-shadow: 0 0 0 3px var(--accent-dim); }
#login-box input::placeholder { color: var(--fg-dim); opacity: 0.6; }
#login-box button {
#login-submit {
width: 100%;
padding: 11px;
background: var(--accent);
@@ -332,9 +332,44 @@ body {
transition: filter 0.15s;
letter-spacing: 0.02em;
}
#login-box button:hover { filter: brightness(1.1); }
#login-box button:focus-visible { outline: 2px solid var(--fg); outline-offset: 2px; }
#login-box button:disabled { opacity: 0.4; cursor: not-allowed; filter: none; }
#login-submit:hover { filter: brightness(1.1); }
#login-submit:focus-visible { outline: 2px solid var(--fg); outline-offset: 2px; }
#login-submit:disabled { opacity: 0.4; cursor: not-allowed; filter: none; }
.login-subtitle {
font-family: var(--font-display);
font-size: 11px;
color: var(--fg-dim);
margin-bottom: 18px;
letter-spacing: 0.02em;
min-height: 14px;
}
.login-label {
display: block;
font-family: var(--font-display);
font-size: 10px;
font-weight: 600;
text-transform: uppercase;
letter-spacing: 0.08em;
color: var(--fg-dim);
margin-bottom: 5px;
}
.login-toggle {
text-align: center;
margin-top: 14px;
}
.login-link {
background: none;
border: none;
color: var(--fg-dim);
font-family: var(--font-display);
font-size: 11px;
cursor: pointer;
padding: 4px 8px;
letter-spacing: 0.02em;
transition: color 0.15s;
}
.login-link:hover { color: var(--accent); }
.login-link:focus-visible { outline: 1px solid var(--accent); outline-offset: 2px; }
#login-error { color: var(--red); font-size: 12px; margin-bottom: 8px; display: none; }
@media (max-width: 380px) { #login-box { padding: 28px 20px; } }
+1 -1
View File
@@ -3,7 +3,7 @@
function escapeHtml(text) {
var el = document.createElement("span");
el.textContent = text;
return el.innerHTML;
return el.innerHTML.replace(/'/g, "&#39;").replace(/"/g, "&quot;");
}
function formatTokens(n) {
+9 -13
View File
@@ -81,18 +81,15 @@ def main() -> None:
default=1,
help="Nodes to kill per interval (default: 1)",
)
parser.add_argument("--redis-host", default="localhost")
parser.add_argument("--redis-port", type=int, default=6379)
parser.add_argument("--redis-password", default=None)
parser.add_argument("--redis-db", type=int, default=0)
from turnstone.mq.broker import add_redis_args
add_redis_args(parser)
parser.add_argument("--prefix", default="turnstone")
parser.add_argument("--seed", type=int, default=None, help="Random seed for reproducibility")
parser.add_argument("--metrics-file", default="", help="Write JSON metrics to file")
parser.add_argument(
"--log-level",
default="INFO",
choices=["DEBUG", "INFO", "WARNING", "ERROR"],
)
from turnstone.core.log import add_log_args
add_log_args(parser)
args = parser.parse_args()
@@ -116,10 +113,9 @@ def main() -> None:
metrics_file=args.metrics_file,
)
logging.basicConfig(
level=getattr(logging, args.log_level),
format="%(asctime)s %(name)s %(levelname)s %(message)s",
)
from turnstone.core.log import configure_logging_from_args
configure_logging_from_args(args, "sim")
try:
asyncio.run(_run(config))
+15 -13
View File
@@ -536,15 +536,20 @@ function connectContentSSE(wsId) {
contentEvtSource = new EventSource(
"/v1/api/events?ws_id=" + encodeURIComponent(wsId),
);
contentEvtSource.onmessage = function (e) {
contentEvtSource.onopen = function () {
contentRetryDelay = 1000;
statusBar.classList.remove("disconnected");
statusBar.textContent = "";
};
contentEvtSource.onmessage = function (e) {
var data = JSON.parse(e.data);
handleEvent(data);
};
contentEvtSource.onerror = function () {
contentEvtSource.close();
contentEvtSource = null;
var loginOverlay = document.getElementById("login-overlay");
if (loginOverlay && loginOverlay.style.display !== "none") return;
statusBar.textContent = "Reconnecting\u2026";
statusBar.classList.add("disconnected");
// Raw fetch (not authFetch) — need to inspect status before throwing
@@ -860,9 +865,10 @@ function dashboardResumeSession(sessionId) {
authFetch("/v1/api/workstreams/new", {
method: "POST",
headers: { "Content-Type": "application/json" },
body: "{}",
body: JSON.stringify({ resume_session: sessionId }),
})
.then(function (r) {
if (!r.ok) throw new Error("HTTP " + r.status);
return r.json();
})
.then(function (data) {
@@ -870,16 +876,10 @@ function dashboardResumeSession(sessionId) {
workstreams[data.ws_id] = { name: data.name, state: "idle" };
switchTab(data.ws_id);
hideDashboard();
authFetch("/v1/api/command", {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({
ws_id: data.ws_id,
command: "/resume " + sessionId,
}),
}).catch(function (err) {
addErrorMessage("Failed to resume: " + err.message);
});
// Resume handled atomically by server — history arrives via SSE.
})
.catch(function (err) {
showToast("Failed to resume session", "error");
});
}
function dashboardNewChat() {
@@ -937,8 +937,10 @@ function connectGlobalSSE() {
globalEvtSource = null;
}
globalEvtSource = new EventSource("/v1/api/events/global");
globalEvtSource.onmessage = function (e) {
globalEvtSource.onopen = function () {
globalRetryDelay = 1000;
};
globalEvtSource.onmessage = function (e) {
var data = JSON.parse(e.data);
if (data.type === "ws_state") {
updateTabIndicator(data.ws_id, data.state, {
+1
View File
@@ -187,6 +187,7 @@
========================================================================== */
#messages {
flex: 1;
min-height: 0;
overflow-y: auto;
padding: 20px;
display: flex;