mirror of
https://github.com/turnstonelabs/turnstone.git
synced 2026-08-12 23:12:23 -06:00
471d1a3311
Systematic pass over every doc under docs/, the root-level README /
QUICKSTART / CONTRIBUTING, and the PlantUML diagrams. Memory and docs
had drifted against the code since 1.2 — this catches them up to the
1.4.0 release and the 1.5.0a1 experimental line.
User-facing fixes
- README: fix broken docs/mcp.md link (→ mcp-registry.md); channel
gateway entry reflects shipped Discord + Slack adapters instead of
"Slack/Teams planned"; diagrams table mentions both.
- QUICKSTART: docs/*.md relative links were wrong from the repo root;
wizard version bumped from 0.5.4.
- CONTRIBUTING: add dev extra plus the ruff / mypy / pytest commands
we actually expect before push.
Reference docs
- architecture.md: 19 tool schemas (was 15), 18 admin tabs (was 14),
turnstone-bootstrap added to entry-points table, OpenAI provider
file split (chat/responses/common) documented, 38 SDK event
dataclasses (was 27 and referenced deleted mq/protocol.py), Slack
adapter + multi-adapter gateway, plan_agent/task_agent naming,
governance admin-panel rewrite.
- api-reference.md: full attachment endpoints (POST/GET/content/
DELETE on /v1/api/workstreams/{ws_id}/attachments) plus the
multipart mode on POST /v1/api/workstreams/new.
- channels.md: Slack Setup section (Socket Mode app creation, OAuth
scopes, tokens), Slack CLI/env reference in config table, combined-
adapter architecture diagram.
- console.md: 18-tab listing (was 13) with Channels/Models/Nodes/TLS
descriptions and ConfigStore live-edit note.
- docker.md: Slack env vars block; image entry-point list now
includes turnstone / turnstone-bootstrap.
- sdk.md: attachments methods on the server client, attachments
example (upload-then-send and at-creation), event count fixed.
- releasing.md: four-track table (stable/1.0, 1.3, 1.4 + main 1.5);
promotion workflow uses 1.5 / 1.6 numbering.
- settings.md: plan_model / task_model / plan_effort / task_effort
overrides section.
- governance.md: skill naming (/skill, `skill` field — not /template),
Prompts/Judge tabs called out.
- security.md: two-token-types wording; src claim values match the
AuthResult source strings actually emitted.
- mcp-registry.md: SDK package name is @turnstone/sdk.
- tools.md: plan / task renamed to plan_agent / task_agent in the
section headings and summary table; primary-key table matched.
- design/consistent-hash-ring.md: dead direct-http-transport.md
pointer redirected to architecture.md.
Diagrams
- 02-package-structure: drop phantom chat.py entry point, add admin
and bootstrap, add slack/bot.py, rename channels/gateway.py →
channels/cli.py.
- 16-channel-architecture: Slack is no longer "(future)", add a
SlackBot class and the slack-bolt Socket Mode edges; wire the new
bot into ChannelService. PNGs regenerated from both puml sources.
245 lines
6.4 KiB
Plaintext
245 lines
6.4 KiB
Plaintext
@startuml
|
|
!theme plain
|
|
title Turnstone — Channel Integration Architecture
|
|
|
|
skinparam class {
|
|
BackgroundColor<<platform>> #E1BEE7
|
|
BackgroundColor<<service>> #E8EAF6
|
|
BackgroundColor<<server>> #FFE0B2
|
|
BackgroundColor<<storage>> #B3E5FC
|
|
}
|
|
|
|
' -- External Platforms --
|
|
class "Discord" as Discord <<platform>> {
|
|
Gateway WebSocket (v10)
|
|
Message events
|
|
Interaction callbacks (buttons)
|
|
Thread-per-workstream
|
|
--
|
|
discord.py 2.x
|
|
asyncio event loop
|
|
}
|
|
|
|
class "Slack" as Slack <<platform>> {
|
|
Socket Mode WebSocket
|
|
Block Kit messages
|
|
Slash command (default /turnstone)
|
|
DM + channel events
|
|
--
|
|
slack-bolt (Python)
|
|
asyncio event loop
|
|
}
|
|
|
|
class "Teams (future)" as Teams <<platform>> {
|
|
Bot Framework
|
|
Adaptive Cards
|
|
--
|
|
Planned integration
|
|
}
|
|
|
|
' -- Channel Service --
|
|
class "turnstone-channel" as ChannelService <<service>> {
|
|
entry point: turnstone-channel
|
|
--
|
|
One process — hosts one or more adapters
|
|
asyncio event loop
|
|
Structured logging (structlog)
|
|
--log-level, --log-format
|
|
--
|
|
POST /v1/api/notify (HTTP)
|
|
GET /health
|
|
}
|
|
|
|
class "SlackBot" as SlackBot <<service>> {
|
|
+on_message(event)
|
|
+on_action(action) (Block Kit buttons)
|
|
+send(channel_id, content)
|
|
+send_notification(channel_id, content, ws_id)
|
|
+run(bot_token, app_token)
|
|
--
|
|
slack-bolt AsyncApp
|
|
Socket Mode client
|
|
Per-user channel sessions via slash command
|
|
DM routing without slash command
|
|
}
|
|
|
|
class "DiscordBot" as Bot <<service>> {
|
|
+on_message(msg)
|
|
+on_interaction(interaction)
|
|
+send(channel_id, content)
|
|
+send_notification(channel_id, content, ws_id)
|
|
+run(token)
|
|
--
|
|
discord.py Client
|
|
Receives message events
|
|
Sends replies + embeds
|
|
Creates threads for workstreams
|
|
Renders approval buttons
|
|
escape_mentions() on send
|
|
--
|
|
_notify_ws_map: msg_id -> (ws_id, user_id)
|
|
_notify_reply_channels: ws_id -> (dm, user_id)
|
|
}
|
|
|
|
class "ChannelRouter" as Router <<service>> {
|
|
+resolve_route(platform, channel_id)
|
|
-> ws_id | None
|
|
+register_route(channel_id, ws_id)
|
|
+resolve_identity(platform, platform_user_id)
|
|
-> user_id | None
|
|
--
|
|
Maps channels -> workstreams
|
|
Maps platform users -> turnstone users
|
|
Caches routes in memory
|
|
}
|
|
|
|
' -- Server --
|
|
class "turnstone-server" as Server <<server>> {
|
|
POST /v1/api/send
|
|
POST /v1/api/approve
|
|
POST /v1/api/workstreams/new
|
|
GET /v1/api/events?ws_id=
|
|
--
|
|
LLM execution + tool use
|
|
SSE event stream
|
|
--
|
|
notify tool: _exec_notify()
|
|
ServiceTokenManager (JWT)
|
|
}
|
|
|
|
' -- Storage --
|
|
class "channel_users" as CU <<storage>> {
|
|
channel_user_id (PK)
|
|
platform: "discord" | "slack"
|
|
platform_user_id
|
|
user_id -> users
|
|
linked_at
|
|
--
|
|
/link command creates row
|
|
Resolved on each inbound message
|
|
}
|
|
|
|
class "channel_routes" as CR <<storage>> {
|
|
channel_type (PK)
|
|
channel_id (PK)
|
|
ws_id
|
|
node_id
|
|
created
|
|
--
|
|
Maps platform channels
|
|
to turnstone workstreams
|
|
}
|
|
|
|
class "services" as SVC <<storage>> {
|
|
service_type (PK)
|
|
service_id (PK)
|
|
url
|
|
last_heartbeat
|
|
created
|
|
--
|
|
Heartbeat every 30s
|
|
Stale after 120s
|
|
ON CONFLICT DO UPDATE
|
|
}
|
|
|
|
' -- Relationships --
|
|
Discord --> Bot : gateway\nevents
|
|
Bot --> Router : on_message\non_interaction
|
|
Router --> CU : resolve identity
|
|
Router --> CR : resolve / register route
|
|
|
|
Router --> Server : POST /v1/api/send\nPOST /v1/api/approve\nPOST /v1/api/workstreams/new
|
|
Bot --> Server : GET /v1/api/events?ws_id=\n(SSE via httpx-sse)
|
|
Server --> Bot : SSE event stream
|
|
|
|
Bot --> Discord : reply / embed\nbutton callback
|
|
|
|
Slack --> SlackBot : socket-mode\nevents
|
|
SlackBot --> Router : on_message / on_action
|
|
SlackBot --> Server : POST /v1/api/send\nGET /v1/api/events?ws_id=
|
|
SlackBot --> Slack : post / update\nBlock Kit button callbacks
|
|
|
|
Teams .[hidden]. Slack
|
|
|
|
ChannelService --> Bot : creates + runs
|
|
ChannelService --> SlackBot : creates + runs
|
|
ChannelService --> Router : creates
|
|
ChannelService --> SVC : register / heartbeat /\nderegister
|
|
|
|
' -- Notification path (direct HTTP) --
|
|
Server --> ChannelService : POST /v1/api/notify\n(JWT: aud=turnstone-channel)
|
|
Server --> SVC : list_services("channel",\nmax_age_seconds=120)
|
|
|
|
' -- Notes --
|
|
note right of Bot
|
|
**Inbound Flow**
|
|
1. Discord message arrives via gateway
|
|
2. Bot.on_message() fires
|
|
3. ChannelRouter resolves channel -> ws_id
|
|
(or creates new workstream)
|
|
4. ChannelRouter resolves platform user -> user_id
|
|
via channel_users table
|
|
5. Router sends POST /v1/api/send to server
|
|
|
|
**Workstream Resume (evicted workstreams)**
|
|
1. Stale route detected (no active SSE listener)
|
|
2. Existing ws_id reused directly from route
|
|
3. POST /v1/api/workstreams/new with
|
|
resume_ws=<ws_id>
|
|
4. Server resumes atomically during creation
|
|
5. SSE emits WorkstreamResumedEvent -> thread
|
|
end note
|
|
|
|
note right of Server
|
|
**Outbound Flow**
|
|
1. Server emits SSE events on
|
|
GET /v1/api/events?ws_id=
|
|
2. Bot subscribes via httpx-sse
|
|
3. Bot formats and sends to Discord thread
|
|
end note
|
|
|
|
note bottom of CR
|
|
**Approval Flow**
|
|
1. ApprovalRequestEvent arrives via SSE
|
|
2. Bot renders Discord buttons (Approve / Deny)
|
|
3. User clicks button -> on_interaction()
|
|
4. Router builds ApproveMessage
|
|
5. Router sends POST /v1/api/approve to server
|
|
end note
|
|
|
|
note bottom of CU
|
|
**Identity Linking**
|
|
1. User runs /link in Discord
|
|
2. Bot opens modal requesting API token
|
|
3. User submits ts_... API token
|
|
4. Bot validates token against storage
|
|
5. On success, inserts channel_users row
|
|
6. Subsequent messages carry user_id
|
|
7. AuthResult scopes applied by server
|
|
end note
|
|
|
|
note bottom of SVC
|
|
**Notification Flow** (direct HTTP)
|
|
1. LLM calls notify tool -> _prepare_notify()
|
|
2. _exec_notify() checks rate limit (5/turn)
|
|
3. Queries services table for healthy gateways
|
|
4. Mints JWT (aud: turnstone-channel) via
|
|
ServiceTokenManager
|
|
5. POSTs to first healthy gateway (incl. ws_id)
|
|
6. Gateway validates JWT, resolves target
|
|
7. adapter.send_notification() -> Discord API
|
|
(tracks msg_id -> ws_id for reply routing)
|
|
8. On failure: retry up to 3x (1s, 3s backoff)
|
|
9. SSRF: only http(s) URLs allowed
|
|
|
|
**Bidirectional DM Replies**
|
|
1. User replies to notification DM
|
|
2. Bot looks up ws_id from _notify_ws_map
|
|
3. Verifies author == notification recipient
|
|
4. Routes reply via router.send_message()
|
|
5. Response forwarded to DM on TurnCompleteEvent
|
|
6. Response tracked for multi-turn conversation
|
|
end note
|
|
|
|
@enduml
|