* Add channel integrations with Discord adapter and atomic session resume (#24) Bidirectional channel adapter framework connecting external messaging platforms to turnstone workstreams via Redis MQ. Discord ships as the first adapter; the protocol supports future Slack/Teams integrations. Channel framework: - ChannelAdapter protocol and ChannelRouter for channel↔workstream mapping - AsyncRedisBroker with single dispatch loop and per-channel ordered workers - channel_routes table (migration 003) for persistent route storage - 9 new StorageBackend methods (4 channel_user + 5 channel_route CRUD) - Unified turnstone-channel gateway entry point, loads adapters by config - Message chunking, approval formatting, plan review formatting Discord adapter: - discord.py v2.4+ bot with thread-per-@mention model - Slash commands: /link (modal), /unlink, /ask, /status, /close - Persistent button views for tool approval and plan review - Streaming responses via edit-in-place (1.5s interval) - Stale route detection and atomic session resume via resume_session field - SessionResumedEvent confirmation back to channel - Auto-approve support (blanket + per-tool list) Atomic session resume: - resume_session field on CreateWorkstreamMessage for single-request resume - Server resumes session during POST /v1/api/workstreams/new atomically - Bridge emits SessionResumedEvent to per-workstream channel - WorkstreamCreatedEvent extended with resumed/session_id/message_count - Server UI dashboardResumeSession simplified to single request - Pruned sessions fall back gracefully to fresh start Service auth: - Bridge and console auto-mint service JWTs from TURNSTONE_JWT_SECRET - Bridge: approve scope (1 week). Console collector: read. Proxy: write. Console admin: - Channels tab with per-user view, force-link modal, unlink - 3 admin API endpoints for channel user management - Styled confirm modals replacing browser confirm() dialogs Bug fixes: - AsyncRedisBroker: replaced per-channel listener tasks with single dispatch loop + per-channel queue workers (fixes message stealing race) - Bridge: approval/plan review dedup guard prevents SSE reconnect duplicates - Bridge: _active_sends tracked for initial messages (fixes missing TurnCompleteEvent and unfinalized streaming messages) - Bridge: HTTP calls moved outside lock scope in approval handlers - Bridge: _handle_send cleans up _active_sends on HTTP/server errors - Formatter: reads server SSE format (func_name/preview) with fallback Docs, SDK, tests: - docs/channels.md setup guide, architecture diagram 16 - Updated api-reference.md, architecture.md, console.md, docker.md - Python SDK: resume_session param on create_workstream (async + sync) - TypeScript SDK: updated CreateWorkstreamRequest/Response interfaces - OpenAPI schema: resume_session request, resumed/message_count response - 91 new tests (19 storage, 15 broker, 22 protocol, 6 routing, 18 discord, 12 resume flow) — 1120 total passing * Fix CI lint/typecheck failures and address Copilot review feedback (#24) Lint: fix import ordering, remove unused imports, use contextlib.suppress. Mypy: explicit postgresql dialect import, add discord module overrides for optional-dependency CI environments. Copilot: fix double-escaping in admin confirm modals, return resolved session_id from server resume response, fix channel_routes diagram schema, use atomic setdefault for routing locks, add post-insert race guard in admin channel create, support SSE format in auto-approve check, update identity linking note in architecture diagram. * Fix remaining mypy call-arg errors for discord.py optional dependency Add type: ignore[call-arg] on Modal(title=) and Cog(name=) class definitions that fail when discord.py is not installed in CI.
6.0 KiB
Docker Deployment
Docker Compose stack for running the full turnstone platform or the simulator.
Quick Start
# Copy and edit environment config
cp .env.example .env
# Full stack (needs an LLM API on the host)
docker compose up
# Simulator only (no LLM needed)
docker compose --profile sim up redis console sim
Console dashboard: http://localhost:8090
See also: Deployment diagram
Services
| Service | Port | Profile | Description |
|---|---|---|---|
redis |
6379 | default | Message broker, pub/sub, node registry |
server |
8080 | default | Web UI + chat workstreams + LLM |
bridge |
— | default | Redis-to-HTTP bridge (multi-node routing) |
console |
8090 | default | Cluster dashboard |
channel |
— | production | Channel gateway (Discord, Slack, etc.) |
sim |
— | sim | Multi-node cluster simulator |
Profiles
Default (no flag) — starts redis, server, bridge, console. Requires an OpenAI-compatible LLM API running on the host (default: http://localhost:8000/v1).
docker compose up
Production — adds PostgreSQL and the channel gateway. Requires POSTGRES_PASSWORD and (for Discord) TURNSTONE_DISCORD_TOKEN:
docker compose --profile production up
Sim — adds the simulator. Can run alongside the full stack or standalone with just Redis and the console:
# Sim + console (no LLM needed)
docker compose --profile sim up redis console sim
# Everything including sim
docker compose --profile sim up
Configuration
All configuration is via environment variables in .env (copy from .env.example):
LLM Backend
| Variable | Default | Description |
|---|---|---|
LLM_BASE_URL |
http://host.docker.internal:8000/v1 |
OpenAI-compatible API URL |
OPENAI_API_KEY |
dummy |
API key (dummy for local servers) |
TAVILY_API_KEY |
— | Web search API key (only needed for local/vLLM models; Anthropic and OpenAI search models use native search) |
Redis
| Variable | Default | Description |
|---|---|---|
REDIS_PASSWORD |
— | Redis auth password (empty = no auth) |
REDIS_PORT |
6379 |
Host port mapping |
Server
| Variable | Default | Description |
|---|---|---|
SERVER_PORT |
8080 |
Host port mapping |
SKIP_PERMISSIONS |
— | Set to any value to auto-approve all tools |
Console
| Variable | Default | Description |
|---|---|---|
CONSOLE_PORT |
8090 |
Host port mapping |
CONSOLE_POLL_INTERVAL |
10 |
Node polling interval (seconds) |
Auth
| Variable | Default | Description |
|---|---|---|
TURNSTONE_AUTH_ENABLED |
— | Set to 1 to require authentication |
TURNSTONE_AUTH_TOKEN |
— | Config-file token for server/bridge/console (backward compat, works alongside JWT) |
TURNSTONE_JWT_SECRET |
— | Secret key for signing JWTs (required when using user identity / JWT auth) |
Database
| Variable | Default | Description |
|---|---|---|
TURNSTONE_DB_BACKEND |
sqlite |
Storage backend: sqlite or postgresql |
TURNSTONE_DB_URL |
— | Database URL (e.g. postgresql://user:pass@db:5432/turnstone). For SQLite, defaults to /data/.turnstone.db |
The database stores workstream history, user accounts, and API tokens. When using JWT auth, a database backend is required for user storage.
First-time setup: After deploying with auth enabled, create an initial admin user by running
turnstone-admin create-userinside the container:docker compose exec server turnstone-admin create-user --username admin --name "Admin"You will be prompted to set a password. Use it to log in via the UI or SDK, then create additional users through the admin API. Pass
--token --scopes read,write,approveto also generate an initial API token.
Channel Gateway
| Variable | Default | Description |
|---|---|---|
TURNSTONE_DISCORD_TOKEN |
— | Discord bot token (required to enable Discord adapter) |
TURNSTONE_DISCORD_GUILD |
0 |
Restrict to a single Discord guild (0 = all guilds) |
The channel service runs in the production profile. When TURNSTONE_DISCORD_TOKEN is set, the Discord adapter connects to the Discord Gateway and routes messages through Redis MQ to the bridge and server. See Channel Integrations for full setup instructions including Discord application creation and user account linking.
Simulator
| Variable | Default | Description |
|---|---|---|
SIM_NODES |
100 |
Number of simulated nodes |
SIM_SCENARIO |
steady |
Scenario: steady, burst, node_failure, directed, lifecycle |
SIM_DURATION |
60 |
Duration in seconds |
SIM_MPS |
5.0 |
Messages per second (steady scenario) |
SIM_LOG_LEVEL |
INFO |
Log verbosity |
SIM_SEED |
— | Random seed for reproducibility |
SIM_METRICS_FILE |
— | Write JSON report to file |
Scaling
Scale to multiple server/bridge pairs:
docker compose up --scale server=3 --scale bridge=3
Each bridge auto-generates a unique node ID from its container hostname. When scaling server, remove the host port mapping (or use a reverse proxy) to avoid port conflicts.
Volumes
| Volume | Mount | Purpose |
|---|---|---|
redis-data |
/data |
Redis persistence |
turnstone-data |
/data |
SQLite database (.turnstone.db) |
Building
The image uses a multi-stage Dockerfile:
# Build all services
docker compose build
# Rebuild without cache
docker compose build --no-cache
All entry points are installed in a single image: turnstone-server, turnstone-bridge, turnstone-console, turnstone-channel, turnstone-admin, turnstone-sim, turnstone-eval.
Cleanup
# Stop and remove containers
docker compose down
# Stop, remove containers and volumes
docker compose down -v