mirror of https://github.com/turnstonelabs/turnstone.git synced 2026-08-12 23:12:23 -06:00

Files

T

Patrick Buckley 471d1a3311 docs: audit documentation for 1.4 / 1.5 state

Systematic pass over every doc under docs/, the root-level README /
QUICKSTART / CONTRIBUTING, and the PlantUML diagrams.  Memory and docs
had drifted against the code since 1.2 — this catches them up to the
1.4.0 release and the 1.5.0a1 experimental line.

User-facing fixes
- README: fix broken docs/mcp.md link (→ mcp-registry.md); channel
  gateway entry reflects shipped Discord + Slack adapters instead of
  "Slack/Teams planned"; diagrams table mentions both.
- QUICKSTART: docs/*.md relative links were wrong from the repo root;
  wizard version bumped from 0.5.4.
- CONTRIBUTING: add dev extra plus the ruff / mypy / pytest commands
  we actually expect before push.

Reference docs
- architecture.md: 19 tool schemas (was 15), 18 admin tabs (was 14),
  turnstone-bootstrap added to entry-points table, OpenAI provider
  file split (chat/responses/common) documented, 38 SDK event
  dataclasses (was 27 and referenced deleted mq/protocol.py), Slack
  adapter + multi-adapter gateway, plan_agent/task_agent naming,
  governance admin-panel rewrite.
- api-reference.md: full attachment endpoints (POST/GET/content/
  DELETE on /v1/api/workstreams/{ws_id}/attachments) plus the
  multipart mode on POST /v1/api/workstreams/new.
- channels.md: Slack Setup section (Socket Mode app creation, OAuth
  scopes, tokens), Slack CLI/env reference in config table, combined-
  adapter architecture diagram.
- console.md: 18-tab listing (was 13) with Channels/Models/Nodes/TLS
  descriptions and ConfigStore live-edit note.
- docker.md: Slack env vars block; image entry-point list now
  includes turnstone / turnstone-bootstrap.
- sdk.md: attachments methods on the server client, attachments
  example (upload-then-send and at-creation), event count fixed.
- releasing.md: four-track table (stable/1.0, 1.3, 1.4 + main 1.5);
  promotion workflow uses 1.5 / 1.6 numbering.
- settings.md: plan_model / task_model / plan_effort / task_effort
  overrides section.
- governance.md: skill naming (/skill, `skill` field — not /template),
  Prompts/Judge tabs called out.
- security.md: two-token-types wording; src claim values match the
  AuthResult source strings actually emitted.
- mcp-registry.md: SDK package name is @turnstone/sdk.
- tools.md: plan / task renamed to plan_agent / task_agent in the
  section headings and summary table; primary-key table matched.
- design/consistent-hash-ring.md: dead direct-http-transport.md
  pointer redirected to architecture.md.

Diagrams
- 02-package-structure: drop phantom chat.py entry point, add admin
  and bootstrap, add slack/bot.py, rename channels/gateway.py →
  channels/cli.py.
- 16-channel-architecture: Slack is no longer "(future)", add a
  SlackBot class and the slack-bolt Socket Mode edges; wire the new
  bot into ChannelService.  PNGs regenerated from both puml sources.

2026-04-16 16:01:24 -07:00

6.2 KiB

Raw Blame History

Docker Deployment

Docker Compose stack for running the full turnstone platform.

Quick Start

# Copy and edit environment config
cp .env.example .env

# Full stack (needs an LLM API on the host)
docker compose up

Console dashboard: http://localhost:8090

See also: Deployment diagram

Services

Service	Port	Profile	Description
`server`	8080	default	Web UI + chat workstreams + LLM
`console`	8090	default	Cluster dashboard
`channel`	—	production	Channel gateway (Discord and/or Slack adapters)
`server-1`…`server-10`	—	cluster	10-node server fleet (PostgreSQL required)

Profiles

Default (no flag) — starts server and console. Requires an OpenAI-compatible LLM API running on the host (default: http://localhost:8000/v1).

docker compose up

Production — adds PostgreSQL and the channel gateway. Requires POSTGRES_PASSWORD and (for Discord) TURNSTONE_DISCORD_TOKEN:

docker compose --profile production up

Cluster — 10-node server fleet sharing PostgreSQL. Access all nodes via the console at :8090. Requires POSTGRES_PASSWORD:

docker compose --profile cluster up

Configuration

All configuration is via environment variables in .env (copy from .env.example):

LLM Backend

Variable	Default	Description
`LLM_BASE_URL`	`http://host.docker.internal:8000/v1`	OpenAI-compatible API URL
`OPENAI_API_KEY`	`dummy`	API key (`dummy` for local servers)
`TAVILY_API_KEY`	—	Web search API key (only needed for local/vLLM models; Anthropic and OpenAI search models use native search)

Server

Variable	Default	Description
`SERVER_PORT`	`8080`	Host port mapping
`SKIP_PERMISSIONS`	—	Set to any value to auto-approve all tools

Console

Variable	Default	Description
`CONSOLE_PORT`	`8090`	Host port mapping

Auth

Auth is always enabled. TURNSTONE_JWT_SECRET is required.

Variable	Default	Description
`TURNSTONE_JWT_SECRET`	—	Secret key for signing JWTs (required)

Database

Variable	Default	Description
`TURNSTONE_DB_BACKEND`	`sqlite`	Storage backend: `sqlite` or `postgresql`
`TURNSTONE_DB_URL`	—	Database URL (e.g. `postgresql+psycopg://user:pass@postgres:5432/turnstone`). For SQLite, defaults to `/data/.turnstone.db`
`TURNSTONE_DB_POOL_SIZE`	`2`	PostgreSQL connection pool size per process (default: 2 base + 3 overflow = 5 max)
`POSTGRES_USER`	`turnstone`	PostgreSQL container username (used in default `TURNSTONE_DB_URL` for cluster/channel)
`POSTGRES_PASSWORD`	—	PostgreSQL container password (required for production and cluster profiles)

The database stores workstream history, user accounts, and API tokens. When using JWT auth, a database backend is required for user storage.

Upgrading from <1.3.0a4: Earlier versions used DB_BACKEND and DATABASE_URL in .env, which compose.yaml mapped to the TURNSTONE_-prefixed names internally. These short aliases have been removed. Rename DB_BACKEND → TURNSTONE_DB_BACKEND and DATABASE_URL → TURNSTONE_DB_URL in your .env file.

Large clusters: Each turnstone process maintains a small connection pool (5 max). At hundreds of nodes this adds up — use PgBouncer in transaction pooling mode between turnstone and PostgreSQL.

First-time setup: After deploying with auth enabled, create an initial admin user by running turnstone-admin create-user inside the container:
docker compose exec server turnstone-admin create-user --username admin --name "Admin"
You will be prompted to set a password. Use it to log in via the UI or SDK, then create additional users through the admin API. Pass --token --scopes read,write,approve to also generate an initial API token.

Channel Gateway

Variable	Default	Description
`TURNSTONE_DISCORD_TOKEN`	—	Discord bot token (required to enable Discord adapter)
`TURNSTONE_DISCORD_GUILD`	`0`	Restrict to a single Discord guild (0 = all guilds)
`TURNSTONE_SLACK_TOKEN`	—	Slack Bot User OAuth token `xoxb-…` (required to enable Slack adapter)
`TURNSTONE_SLACK_APP_TOKEN`	—	Slack App-Level token `xapp-…` (required with `TURNSTONE_SLACK_TOKEN`)
`TURNSTONE_SLACK_CHANNELS`	—	Comma-separated Slack channel IDs to allow (empty = all)
`TURNSTONE_SLACK_SLASH_COMMAND`	`/turnstone`	Slash command registered in the Slack app

The channel service runs in the production profile. When TURNSTONE_DISCORD_TOKEN or the Slack pair is set the gateway starts the corresponding adapter; both can run in one process. See Channel Integrations for platform app setup and user account linking.

Scaling

For multi-node testing, use the cluster profile which provides 10 server instances with unique node IDs (node-1 through node-10), resource limits, and shared PostgreSQL:

POSTGRES_PASSWORD=secret docker compose --profile cluster up

The default server also runs alongside the cluster nodes (11 total). All nodes are accessible via the console dashboard at :8090.

For production clusters beyond ~50 nodes, add PgBouncer between turnstone services and PostgreSQL. See PgBouncer Connection Pooling for Docker Compose and Helm configuration.

Volumes

Volume	Mount	Purpose
`turnstone-data`	`/data`	SQLite database (`.turnstone.db`)

Building

The image uses a multi-stage Dockerfile:

# Build all services
docker compose build

# Rebuild without cache
docker compose build --no-cache

All entry points are installed in a single image: turnstone, turnstone-server, turnstone-console, turnstone-channel, turnstone-admin, turnstone-eval, and turnstone-bootstrap.

Cleanup

# Stop and remove containers
docker compose down

# Stop, remove containers and volumes
docker compose down -v

6.2 KiB Raw Blame History