Patrick Buckley
62d2a0fe6a
* fix: remove non-auth support from bootstrap wizard Auth is now mandatory for all deployments. Remove the TURNSTONE_AUTH_ENABLED toggle and make JWT_SECRET and AUTH_TOKEN required in the wizard's system prompt. * fix: remove auth disable support from runtime and infra Remove AuthConfig.enabled field — auth is always on. Drop TURNSTONE_AUTH_ENABLED env var, config toggle, and the check_request bypass. Update compose.yaml, Helm chart, Terraform, docs, and tests to match. * feat: deprecate config tokens, require JWT secret, prefer JWT auth Phase 1 of config-token removal: - load_jwt_secret() now exits with error if no secret is configured (was: silently auto-generated ephemeral secret) - _authenticate_token() logs deprecation warning on config token use - CLI /cluster commands use ServiceTokenManager when JWT secret is set - turnstone-admin tls-list uses ServiceTokenManager when JWT secret is set - Update bootstrap wizard, docker.md, security.md to mark TURNSTONE_AUTH_TOKEN as deprecated and JWT_SECRET as required - Console test fixtures use auth token + headers (auth always enforced) * feat: add service scope for inter-service JWT auth Add "service" to VALID_SCOPES and SCOPE_HIERARCHY. Service tokens bypass require_permission() RBAC checks, replacing the old empty-user-id bypass that config tokens relied on. All ServiceTokenManager instances that need admin access now include "service" in their scopes (console proxy, channel gateway, CLI, admin CLI). Read-only services (collector, notification) unchanged. * feat: phase 2 config token deprecation - SDK doc examples now show API tokens (ts_) instead of config tokens - Remove _get_config_token() from admin CLI (dead code) - Block config token exchange in handle_auth_login — only password and API token login allowed - Update login tests to use password-based auth instead of config token exchange * feat: phase 3 — remove config tokens entirely Complete removal of config-file token authentication: - Delete AuthConfig.tokens, check(), _ROLE_TO_SCOPES, hmac dispatch branch, and config token loading from load_auth_config() - Remove auth_config parameter from _authenticate_token() and check_request() — callers updated throughout - Remove TURNSTONE_AUTH_TOKEN from compose.yaml, Helm charts, Terraform, turnstone.example.toml - Remove --auth-token CLI flags from turnstone, turnstone-admin, and turnstone-console - Simplify console main() — always use ServiceTokenManager (no fallback to static tokens) - Delete config-token-specific tests, rewrite check_request and integration tests to use JWT auth with proper audience claims - Remove all config token references from docs (security.md, docker.md, sdk.md, console.md, architecture.md, bootstrap prompt) * fix: address code review findings - Fix 33 broken tests: add JWT auth to test_api_versioning, test_console_routing_proxy, test_tls_admin, test_tls_manager, test_server_live (jwt_secret + audience-scoped auth headers) - Add TestRequirePermissionServiceScope: 4 tests covering the service scope RBAC bypass path - Remove stale comments referencing config tokens in auth.py and console/server.py - Remove dead proxy_auth_token parameter from console create_app() and static token fallback in _proxy_auth_headers() - Remove TURNSTONE_AUTH_TOKEN from env.py scrub list * fix: address Copilot review — JWT audience, compose require secret - CLI /cluster: add audience=JWT_AUD_CONSOLE to ServiceTokenManager (console validates audience, JWTs without it were rejected) - Admin CLI tls-list: same audience fix - compose.yaml: TURNSTONE_JWT_SECRET now uses :? to fail fast if unset - SDK console: fix default port from 8081 to 8090 * test: add auth enforcement tests for TLS admin endpoints 5 new tests: unauthenticated requests return 401 (list, renew, delete), read-only-scoped requests return 403 (renew, delete). Closes the TLS auth enforcement test gap noted in PROGRESS.md. * fix: address remaining Copilot review feedback - Fix token_source="config" → "test" in TLS test fixtures - Fix AuthResult.token_source docstring to include service origins - Require TURNSTONE_JWT_SECRET in cluster compose profile (:?) - Helm: add auth.jwtSecret + auth.existingSecret values, wire TURNSTONE_JWT_SECRET into secret.yaml and both deployments - Terraform: replace auth_token with jwt_secret variable + secret, remove orphaned auth_token resources and IAM reference - Remove [[auth.tokens]] from security.md config example * fix: address full code review — 10 findings Critical: - Terraform: replace concat(common_env, auth_env) with common_env (auth_env local was removed but still referenced) - Channel gateway: remove hmac static token auth from _check_auth(), use JWT-only validation. Remove --auth-token CLI arg from channel - Rebalancer: add token_manager support so migration requests carry JWT auth (was sending unauthenticated POST to /internal/migrate) Major: - Guard _permissions_to_scopes() against "service" privilege escalation from DB role permissions - Remove dead AuthConfig class, load_auth_config(), and all auth_config parameters from create_app() signatures - Helm: inject JWT secret for both inline and existingSecret paths Minor: - Remove dead auth_token param from ClusterCollector - Remove empty TestLoadAuthConfig class - Short JWT secret now exits instead of warning - Compose: add generation command comment above JWT_SECRET - Clean stale config token references from 6 doc files - Clean stale AUTH_TOKEN reference from bootstrap wizard prompt * fix: remove remaining stale config token references from docs - channels.md: remove --auth-token from options table - oidc.md: remove "config-file tokens still work" claim - security.md: remove config token section, fix JWT secret docs (now required/exits, no ephemeral fallback), remove hmac from ASCII diagram, remove --auth-token reference
5.3 KiB
Docker Deployment
Docker Compose stack for running the full turnstone platform.
Quick Start
# Copy and edit environment config
cp .env.example .env
# Full stack (needs an LLM API on the host)
docker compose up
Console dashboard: http://localhost:8090
See also: Deployment diagram
Services
| Service | Port | Profile | Description |
|---|---|---|---|
server |
8080 | default | Web UI + chat workstreams + LLM |
console |
8090 | default | Cluster dashboard |
channel |
— | production | Channel gateway (Discord, Slack, etc.) |
server-1…server-10 |
— | cluster | 10-node server fleet (PostgreSQL required) |
Profiles
Default (no flag) — starts server and console. Requires an OpenAI-compatible LLM API running on the host (default: http://localhost:8000/v1).
docker compose up
Production — adds PostgreSQL and the channel gateway. Requires POSTGRES_PASSWORD and (for Discord) TURNSTONE_DISCORD_TOKEN:
docker compose --profile production up
Cluster — 10-node server fleet sharing PostgreSQL. Access all nodes via the console at :8090. Requires POSTGRES_PASSWORD:
docker compose --profile cluster up
Configuration
All configuration is via environment variables in .env (copy from .env.example):
LLM Backend
| Variable | Default | Description |
|---|---|---|
LLM_BASE_URL |
http://host.docker.internal:8000/v1 |
OpenAI-compatible API URL |
OPENAI_API_KEY |
dummy |
API key (dummy for local servers) |
TAVILY_API_KEY |
— | Web search API key (only needed for local/vLLM models; Anthropic and OpenAI search models use native search) |
Server
| Variable | Default | Description |
|---|---|---|
SERVER_PORT |
8080 |
Host port mapping |
SKIP_PERMISSIONS |
— | Set to any value to auto-approve all tools |
Console
| Variable | Default | Description |
|---|---|---|
CONSOLE_PORT |
8090 |
Host port mapping |
Auth
Auth is always enabled. TURNSTONE_JWT_SECRET is required.
| Variable | Default | Description |
|---|---|---|
TURNSTONE_JWT_SECRET |
— | Secret key for signing JWTs (required) |
Database
| Variable | Default | Description |
|---|---|---|
TURNSTONE_DB_BACKEND |
sqlite |
Storage backend: sqlite or postgresql |
TURNSTONE_DB_URL |
— | Database URL (e.g. postgresql://user:pass@db:5432/turnstone). For SQLite, defaults to /data/.turnstone.db |
TURNSTONE_DB_POOL_SIZE |
2 |
PostgreSQL connection pool size per process (default: 2 base + 3 overflow = 5 max) |
The database stores workstream history, user accounts, and API tokens. When using JWT auth, a database backend is required for user storage.
Large clusters: Each turnstone process maintains a small connection pool (5 max). At hundreds of nodes this adds up — use PgBouncer in transaction pooling mode between turnstone and PostgreSQL.
First-time setup: After deploying with auth enabled, create an initial admin user by running
turnstone-admin create-userinside the container:docker compose exec server turnstone-admin create-user --username admin --name "Admin"You will be prompted to set a password. Use it to log in via the UI or SDK, then create additional users through the admin API. Pass
--token --scopes read,write,approveto also generate an initial API token.
Channel Gateway
| Variable | Default | Description |
|---|---|---|
TURNSTONE_DISCORD_TOKEN |
— | Discord bot token (required to enable Discord adapter) |
TURNSTONE_DISCORD_GUILD |
0 |
Restrict to a single Discord guild (0 = all guilds) |
The channel service runs in the production profile. When TURNSTONE_DISCORD_TOKEN is set, the Discord adapter connects to the Discord Gateway and routes messages to the server via HTTP. See Channel Integrations for full setup instructions including Discord application creation and user account linking.
Scaling
For multi-node testing, use the cluster profile which provides 10 server instances with unique node IDs (node-1 through node-10), resource limits, and shared PostgreSQL:
POSTGRES_PASSWORD=secret docker compose --profile cluster up
The default server also runs alongside the cluster nodes (11 total). All nodes are accessible via the console dashboard at :8090.
For production clusters beyond ~50 nodes, add PgBouncer between turnstone services and PostgreSQL. See PgBouncer Connection Pooling for Docker Compose and Helm configuration.
Volumes
| Volume | Mount | Purpose |
|---|---|---|
turnstone-data |
/data |
SQLite database (.turnstone.db) |
Building
The image uses a multi-stage Dockerfile:
# Build all services
docker compose build
# Rebuild without cache
docker compose build --no-cache
All entry points are installed in a single image: turnstone-server, turnstone-console, turnstone-channel, turnstone-admin, turnstone-eval.
Cleanup
# Stop and remove containers
docker compose down
# Stop, remove containers and volumes
docker compose down -v