Add channel integrations with Discord adapter and atomic session resu… (#24)

* Add channel integrations with Discord adapter and atomic session resume (#24)

Bidirectional channel adapter framework connecting external messaging
platforms to turnstone workstreams via Redis MQ. Discord ships as the
first adapter; the protocol supports future Slack/Teams integrations.

Channel framework:
- ChannelAdapter protocol and ChannelRouter for channel↔workstream mapping
- AsyncRedisBroker with single dispatch loop and per-channel ordered workers
- channel_routes table (migration 003) for persistent route storage
- 9 new StorageBackend methods (4 channel_user + 5 channel_route CRUD)
- Unified turnstone-channel gateway entry point, loads adapters by config
- Message chunking, approval formatting, plan review formatting

Discord adapter:
- discord.py v2.4+ bot with thread-per-@mention model
- Slash commands: /link (modal), /unlink, /ask, /status, /close
- Persistent button views for tool approval and plan review
- Streaming responses via edit-in-place (1.5s interval)
- Stale route detection and atomic session resume via resume_session field
- SessionResumedEvent confirmation back to channel
- Auto-approve support (blanket + per-tool list)

Atomic session resume:
- resume_session field on CreateWorkstreamMessage for single-request resume
- Server resumes session during POST /v1/api/workstreams/new atomically
- Bridge emits SessionResumedEvent to per-workstream channel
- WorkstreamCreatedEvent extended with resumed/session_id/message_count
- Server UI dashboardResumeSession simplified to single request
- Pruned sessions fall back gracefully to fresh start

Service auth:
- Bridge and console auto-mint service JWTs from TURNSTONE_JWT_SECRET
- Bridge: approve scope (1 week). Console collector: read. Proxy: write.

Console admin:
- Channels tab with per-user view, force-link modal, unlink
- 3 admin API endpoints for channel user management
- Styled confirm modals replacing browser confirm() dialogs

Bug fixes:
- AsyncRedisBroker: replaced per-channel listener tasks with single
  dispatch loop + per-channel queue workers (fixes message stealing race)
- Bridge: approval/plan review dedup guard prevents SSE reconnect duplicates
- Bridge: _active_sends tracked for initial messages (fixes missing
  TurnCompleteEvent and unfinalized streaming messages)
- Bridge: HTTP calls moved outside lock scope in approval handlers
- Bridge: _handle_send cleans up _active_sends on HTTP/server errors
- Formatter: reads server SSE format (func_name/preview) with fallback

Docs, SDK, tests:
- docs/channels.md setup guide, architecture diagram 16
- Updated api-reference.md, architecture.md, console.md, docker.md
- Python SDK: resume_session param on create_workstream (async + sync)
- TypeScript SDK: updated CreateWorkstreamRequest/Response interfaces
- OpenAPI schema: resume_session request, resumed/message_count response
- 91 new tests (19 storage, 15 broker, 22 protocol, 6 routing,
  18 discord, 12 resume flow) — 1120 total passing

* Fix CI lint/typecheck failures and address Copilot review feedback (#24)

Lint: fix import ordering, remove unused imports, use contextlib.suppress.
Mypy: explicit postgresql dialect import, add discord module overrides for
optional-dependency CI environments.
Copilot: fix double-escaping in admin confirm modals, return resolved
session_id from server resume response, fix channel_routes diagram schema,
use atomic setdefault for routing locks, add post-insert race guard in
admin channel create, support SSE format in auto-approve check, update
identity linking note in architecture diagram.

* Fix remaining mypy call-arg errors for discord.py optional dependency

Add type: ignore[call-arg] on Modal(title=) and Cog(name=) class
definitions that fail when discord.py is not installed in CI.
This commit is contained in:
Patrick Buckley
2026-03-04 13:02:58 -08:00
committed by GitHub
parent 047680d669
commit a6e929b0a0
45 changed files with 5017 additions and 98 deletions
+1 -1
View File
@@ -34,7 +34,7 @@ RUN useradd --create-home --shell /bin/bash turnstone
# Install the wheel with all optional extras
COPY --from=builder /build/wheels/*.whl /tmp/wheels/
RUN pip install --no-cache-dir "$(ls /tmp/wheels/*.whl)[mq,console,sim,postgres]" \
RUN pip install --no-cache-dir "$(ls /tmp/wheels/*.whl)[mq,console,sim,postgres,discord]" \
&& rm -rf /tmp/wheels
# Health check script (stdlib only, no pip deps needed)
+35
View File
@@ -196,6 +196,41 @@ services:
start_period: 10s
restart: unless-stopped
# -------------------------------------------------------------------
# turnstone-channel — Channel gateway (Discord, Slack, etc.)
# Requires TURNSTONE_DISCORD_TOKEN to enable Discord adapter
# -------------------------------------------------------------------
channel:
build:
context: .
dockerfile: Dockerfile
profiles:
- production
command:
- sh
- -c
- >-
turnstone-channel
--redis-host=redis
--redis-port=6379
$${TURNSTONE_DISCORD_GUILD:+--discord-guild $$TURNSTONE_DISCORD_GUILD}
environment:
- TURNSTONE_DISCORD_TOKEN=${TURNSTONE_DISCORD_TOKEN:-}
- TURNSTONE_DISCORD_GUILD=${TURNSTONE_DISCORD_GUILD:-0}
- REDIS_PASSWORD=${REDIS_PASSWORD:-}
- TURNSTONE_JWT_SECRET=${TURNSTONE_JWT_SECRET:-}
- TURNSTONE_DB_BACKEND=${DB_BACKEND:-sqlite}
- TURNSTONE_DB_URL=${DATABASE_URL:-}
networks:
- turnstone-net
depends_on:
redis:
condition: service_healthy
postgres:
condition: service_healthy
required: false
restart: unless-stopped
# -------------------------------------------------------------------
# turnstone-sim — Multi-node cluster simulator (no LLM needed)
# Start with: docker compose --profile sim up
+13 -10
View File
@@ -716,22 +716,25 @@ Creates a new workstream. The server supports up to 10 concurrent workstreams.
All fields are optional. The body can be empty or an empty JSON object.
| Field | Type | Default | Description |
|----------------|--------|---------|------------------------------------------------|
| `name` | string | auto | Workstream display name |
| `model` | string | default | Model alias from the registry (`[models.*]`) |
| `auto_approve` | bool | false | Auto-approve all tool calls for this workstream |
| Field | Type | Default | Description |
|------------------|--------|---------|----------------------------------------------------------------|
| `name` | string | auto | Workstream display name |
| `model` | string | default | Model alias from the registry (`[models.*]`) |
| `auto_approve` | bool | false | Auto-approve all tool calls for this workstream |
| `resume_session` | string | "" | Session ID to resume atomically during creation (empty = fresh)|
**Response (success):**
```json
{"ws_id": "ghi789", "name": "ws-3"}
{"ws_id": "ghi789", "name": "ws-3", "resumed": false, "message_count": 0}
```
| Field | Type | Description |
|---------|--------|------------------------------------|
| `ws_id` | string | Unique ID of the new workstream |
| `name` | string | Auto-generated workstream name |
| Field | Type | Description |
|-----------------|--------|-----------------------------------------------------|
| `ws_id` | string | Unique ID of the new workstream |
| `name` | string | Auto-generated workstream name |
| `resumed` | bool | Whether a previous session was successfully resumed |
| `message_count` | int | Number of messages in the resumed session (0 if fresh) |
**Error (limit reached):**
+33
View File
@@ -21,6 +21,8 @@ plugs in.
| `turnstone-bridge` | `turnstone.mq.bridge` | Bridge | Message queue ↔ HTTP API bridge |
| `turnstone-console` | `turnstone.console.server` | ClusterCollector | Cluster dashboard (aggregates all nodes) |
| `turnstone-eval` | `turnstone.eval` | `NullUI` | Headless evaluation and prompt optimization |
| `turnstone-channel` | `turnstone.channels.cli` | ChannelAdapter | Channel gateway (Discord, Slack, etc.) via Redis MQ |
| `turnstone-admin` | `turnstone.core.admin_cli` | — | Offline user and API token management |
---
@@ -75,6 +77,12 @@ turnstone/
collector.py ClusterCollector — aggregates state from all nodes via Redis + HTTP
server.py Cluster dashboard HTTP server + SSE + CLI entry point
static/ Cluster dashboard web UI (page-specific HTML, CSS, JS)
channels/
cli.py Unified channel gateway entry point (turnstone-channel)
_protocol.py ChannelAdapter protocol, ChannelEvent dataclass
_routing.py ChannelRouter — channel/thread ↔ workstream mapping via MQ
_config.py Base ChannelConfig dataclass
discord/ Discord adapter (bot, cog, views, streaming, config)
shared_static/ Shared design system (base.css, auth.js, theme.js, toast.js, utils.js, kb.js)
ui/
colors.py ANSI color constants with NO_COLOR support
@@ -1254,3 +1262,28 @@ with TurnstoneServer("http://localhost:8080", token="tok_xxx") as client:
result = client.send_and_wait("Hello!", ws.ws_id)
print(result.content)
```
---
## Channel Integrations
> See also: [Channel Integrations guide](channels.md)
The `turnstone-channel` gateway bridges external messaging platforms
(Discord, Slack, Teams) to the turnstone cluster via Redis MQ. Each
platform adapter implements the `ChannelAdapter` protocol and translates
between platform-native events and turnstone MQ messages.
The `ChannelRouter` manages bidirectional routing: it maps platform
channel/thread IDs to turnstone workstream IDs, handles workstream
creation and stale-route recovery, and resolves platform users to
turnstone identities via the `channel_users` table. When an evicted
workstream is reactivated, the router uses atomic session resume via the
`resume_session` field on `CreateWorkstreamMessage` — the server resumes
the old session during workstream creation in a single HTTP request,
eliminating ordering fragility. The bridge emits a `SessionResumedEvent`
to confirm success.
Discord ships as the first adapter. See [channels.md](channels.md) for
setup instructions, configuration reference, and the adapter development
guide.
+272
View File
@@ -0,0 +1,272 @@
# Channel Integrations
The `turnstone-channel` gateway connects external messaging platforms to
turnstone workstreams via Redis MQ. Each platform adapter translates
platform-native events (messages, button clicks, slash commands) into
turnstone MQ messages, and renders workstream output back into the
platform's UI.
Discord ships as the first adapter. The adapter protocol is designed for
future Slack and Teams integrations.
---
## Architecture
```
Discord Gateway
|
v
turnstone-channel (Discord adapter)
|
v
Redis MQ
|
v
turnstone-bridge ──> turnstone-server
```
Key components:
- **ChannelAdapter protocol** (`turnstone/channels/_protocol.py`) — generic
interface for any messaging platform. Defines `start()`, `stop()`,
`send()`, `edit_message()`, `send_approval_request()`,
`send_plan_review()`, and `create_thread()`.
- **ChannelRouter** (`turnstone/channels/_routing.py`) — maps
channel/thread IDs to turnstone workstream IDs. Handles workstream
creation via MQ, stale route detection, and user identity resolution.
- **AsyncRedisBroker** (`turnstone/mq/async_broker.py`) — async Redis
client compatible with discord.py's event loop. Used by the router for
pub/sub and queue operations.
- **channel_users table** — maps `(channel_type, channel_user_id)` to a
turnstone `user_id`. Messages from unlinked users are silently dropped.
- **channel_routes table** — persistent channel-to-workstream mappings.
Survives bot restarts. Stale routes (evicted workstreams) are detected
and refreshed on the next message.
---
## Discord Setup
### 1. Create a Discord Application
1. Go to https://discord.com/developers/applications
2. Click **New Application** and give it a name
3. Navigate to the **Bot** tab and click **Reset Token** to generate a
bot token. Copy it immediately — it is shown only once.
4. On the same **Bot** tab, scroll down to **Privileged Gateway Intents**
and enable **MESSAGE CONTENT INTENT**
5. Navigate to **OAuth2 > URL Generator**
6. Under **Scopes**, check `bot` and `applications.commands`
7. Under **Bot Permissions**, check:
- View Channels
- Send Messages
- Send Messages in Threads
- Create Public Threads
- Read Message History
- Add Reactions
- Embed Links
8. Copy the generated URL, open it in a browser, and add the bot to your
Discord server
### 2. Configure Turnstone
**Environment variables** (recommended for Docker):
```bash
TURNSTONE_DISCORD_TOKEN=your-bot-token-here
TURNSTONE_DISCORD_GUILD=123456789 # optional, restrict to one guild
```
**CLI flags** (bare-metal):
```bash
turnstone-channel \
--discord-token "your-bot-token" \
--discord-guild 123456789 \
--redis-host localhost \
--redis-port 6379
```
**Docker Compose** (production profile):
```bash
# In .env file:
TURNSTONE_DISCORD_TOKEN=your-bot-token
TURNSTONE_DISCORD_GUILD=123456789
```
Then start the stack:
```bash
docker compose --profile production up
```
The `channel` service starts automatically when
`TURNSTONE_DISCORD_TOKEN` is set.
### 3. Link User Accounts
Discord users must link their account to a turnstone user before they can
interact with the bot. Unlinked users' messages are silently ignored.
1. The user must have a turnstone API token — created via the admin panel
or `turnstone-admin create-token`
2. In Discord, the user runs `/link`. A modal appears prompting for the
API token (the token is never visible in Discord audit logs because it
is submitted via modal, not as a slash command argument).
3. The token is validated against the database. If valid, a
`channel_users` mapping is created.
4. The user can now @mention the bot or use slash commands.
An admin can also force-link or unlink users via the console admin panel
(Admin > Channels tab).
---
## Usage
### Conversations
- **@mention** the bot in any allowed channel to start a new conversation.
The bot creates a Discord thread from the message and a turnstone
workstream behind it.
- All subsequent messages in the thread are routed to the same workstream.
- The bot streams responses via message edits, updated approximately every
1.5 seconds.
- If the workstream is evicted for capacity, the next message in the
thread auto-creates a new workstream and atomically resumes the
previous session via the `resume_session` field on
`CreateWorkstreamMessage`. The server resumes the session during
workstream creation (same HTTP request), and the bridge emits a
`SessionResumedEvent` back to the channel. The thread receives a
*"Session resumed: {name} ({count} messages restored)"* confirmation.
### Slash Commands
| Command | Description |
|---------|-------------|
| `/link` | Link Discord account to turnstone (opens modal for API token) |
| `/unlink` | Unlink Discord account |
| `/ask <message>` | Create a new thread and workstream with an initial message |
| `/status` | Show workstream info for the current thread (ephemeral) |
| `/close` | Close the workstream, delete the route, and archive the thread |
### Tool Approvals
When manual approval is enabled (the default), tool calls are displayed as
an orange embed with:
- Tool name and argument preview
- **Approve** (green), **Reject** (red), **Always Approve** (gray) buttons
- Only linked users can interact with approval buttons
- The approval decision is forwarded through MQ to the bridge, which
relays it to the server
Buttons use static `custom_id` values so they survive bot restarts.
Correlation data (`ws_id`, `correlation_id`) is stored in the embed footer.
**Auto-approval:** When `auto_approve` is true (via `--auto-approve`), or when
all tools in the request match the `auto_approve_tools` list in the adapter
config, the bot auto-responds with approval and posts a
"*Tool auto-approved.*" notice to the thread instead of showing buttons. The
`auto_approve_tools` list is set via the `ChannelConfig.auto_approve_tools`
field (useful for allowing specific tools like `bash` or `read_file` while
still requiring manual approval for others).
### Plan Reviews
Plan review requests are displayed as a blue embed with:
- **Approve Plan** (green) button — approves the plan with empty feedback
- **Request Changes** (gray) button — opens a modal for feedback text
(up to 2000 characters)
- Feedback is forwarded through MQ as a `PlanFeedbackMessage`
---
## Configuration Reference
| CLI Flag | Env Var | Default | Description |
|----------|---------|---------|-------------|
| `--discord-token` | `TURNSTONE_DISCORD_TOKEN` | — | Bot token (required to enable Discord) |
| `--discord-guild` | — | `0` (all guilds) | Restrict to a single Discord guild |
| `--discord-channels` | — | empty (all) | Comma-separated channel IDs to allow |
| `--redis-host` | `REDIS_HOST` | `localhost` | Redis host |
| `--redis-port` | — | `6379` | Redis port |
| `--redis-password` | `REDIS_PASSWORD` | — | Redis password |
| `--redis-db` | — | `0` | Redis DB number |
| `--model` | — | server default | Default model for new workstreams |
| `--auto-approve` | — | `false` | Auto-approve ALL tool calls (skips approval buttons entirely) |
| `--log-level` | `TURNSTONE_LOG_LEVEL` | `INFO` | Log level |
| `--log-format` | `TURNSTONE_LOG_FORMAT` | `auto` | Log format (`auto`/`json`/`text`) |
---
## User Identity
- The `channel_users` table maps `(channel_type, channel_user_id)` to a
turnstone `user_id`
- Self-service linking via the `/link` slash command (modal input, not
visible in Discord audit logs)
- Admin can force-link or unlink via the console admin panel (Admin >
Channels tab). Unlinking uses a styled confirmation modal.
- Unlinked users' messages are silently dropped
- A user can be linked across multiple platforms (e.g. Discord + Slack)
See [Security: Database Schema](security.md#database-schema) for the
`channel_users` table definition.
---
## Workstream Lifecycle
1. **Creation**@mention or `/ask` creates a Discord thread and a
turnstone workstream. The `ChannelRouter` persists the mapping in the
`channel_routes` table.
2. **Active** — messages are routed bidirectionally. The bot streams
responses via message edits (updated every ~1.5 seconds).
3. **Eviction** — the server evicts an idle workstream for capacity. The
route is preserved and the thread stays open.
4. **Reactivation** — the next message in the thread detects the stale
route (no MQ owner), looks up the old session via
`get_session_id_by_ws()`, and creates a new workstream with
`resume_session` set atomically on the `CreateWorkstreamMessage`. The
server resumes the session during creation (no separate command
needed). The bridge emits a `SessionResumedEvent` to the channel, and
the thread displays *"Session resumed: {name} ({count} messages
restored)"*. If the old session was pruned, the workstream starts
fresh with no error.
5. **Close**`/close` command closes the workstream via MQ, deletes the
route, unsubscribes from events, and archives the Discord thread.
---
## Adding New Adapters
The `ChannelAdapter` protocol defines the interface any platform adapter
must implement:
```python
class ChannelAdapter(Protocol):
channel_type: str
async def start(self) -> None: ...
async def stop(self) -> None: ...
async def send(self, channel_id: str, content: str) -> str: ...
async def edit_message(self, channel_id: str, message_id: str, content: str) -> None: ...
async def send_approval_request(self, channel_id: str, ws_id: str, correlation_id: str, items: list[dict]) -> None: ...
async def send_plan_review(self, channel_id: str, ws_id: str, correlation_id: str, content: str) -> None: ...
async def create_thread(self, parent_channel_id: str, name: str, message_id: str = "") -> str: ...
```
To add a new platform:
1. Create `turnstone/channels/<platform>/` package
2. Implement the `ChannelAdapter` protocol
3. Add a `--<platform>-token` flag and detection logic in
`turnstone/channels/cli.py`
4. Add the optional dependency in `pyproject.toml` (e.g.
`turnstone[slack]`)
See `turnstone/channels/discord/` as a reference implementation.
+28 -4
View File
@@ -262,6 +262,16 @@ List active tokens for a user (token strings are not returned, only metadata).
Revoke a specific API token.
### Channel links
| Method | Path | Description |
|--------|------|-------------|
| GET | `/v1/api/admin/users/{user_id}/channels` | List channel links for a user |
| POST | `/v1/api/admin/users/{user_id}/channels` | Link a channel account (channel_type, channel_user_id) |
| DELETE | `/v1/api/admin/channels/{channel_type}/{channel_user_id}` | Unlink a channel account |
These endpoints manage the `channel_users` table mappings that connect external platform identities (e.g. Discord user IDs) to turnstone users. See [Channel Integrations](channels.md) for details on the linking flow.
#### `GET /v1/api/auth/status`
Public endpoint for login UI state detection. Returns auth configuration, not
@@ -361,15 +371,16 @@ All five views receive live updates via SSE — state cards update counts, node
### 5. Admin Panel
Accessed via the "admin" button in the header (visible when authenticated
with `approve` scope). Provides user and API token management with two tabs:
with `approve` scope). Provides user, API token, and channel link management
with three tabs:
**Users tab:**
- Grid table listing all users (username, display name, role, creation date)
- "Create User" button opens a modal with fields for username, display name,
and password (validated: username 1-64 ASCII, password min 8 characters)
- Delete button on each row removes the user and cascades to revoke all
their tokens
- Delete button on each row opens a styled confirmation modal before
removing the user and cascading to revoke all their tokens
**Tokens tab:**
@@ -382,7 +393,20 @@ with `approve` scope). Provides user and API token management with two tabs:
- On creation, a "Token Created" modal displays the raw `ts_`-prefixed
token with a copy button. The token is shown once and cannot be retrieved
again.
- Revoke button on each row deletes the token
- Revoke button on each row opens a styled confirmation modal before
deleting the token
**Channels tab:**
- User selector dropdown to pick which user's channel links to manage
- Grid table listing linked channel accounts for the selected user
(channel type, channel user ID, creation date)
- "Link Channel" button opens a modal with fields for channel type
(e.g. `discord`) and the platform user ID
- Unlink button on each row opens a styled confirmation modal before
removing the channel mapping
- Admins can force-link users who have not self-linked via `/link` in
Discord
**Accessibility:**
+211
View File
@@ -0,0 +1,211 @@
@startuml
!theme plain
title Turnstone — Channel Integration Architecture
skinparam class {
BackgroundColor<<platform>> #E1BEE7
BackgroundColor<<service>> #E8EAF6
BackgroundColor<<mq>> #FFCDD2
BackgroundColor<<bridge>> #C8E6C9
BackgroundColor<<server>> #FFE0B2
BackgroundColor<<storage>> #B3E5FC
}
' -- External Platforms --
class "Discord" as Discord <<platform>> {
Gateway WebSocket (v10)
Message events
Interaction callbacks (buttons)
Thread-per-workstream
--
discord.py 2.x
asyncio event loop
}
class "Slack (future)" as Slack <<platform>> {
Socket Mode / Events API
Block Kit messages
--
Planned integration
}
class "Teams (future)" as Teams <<platform>> {
Bot Framework
Adaptive Cards
--
Planned integration
}
' -- Channel Service --
class "turnstone-channel" as ChannelService <<service>> {
entry point: turnstone-channel
--
One process per platform
asyncio event loop
Structured logging (structlog)
--log-level, --log-format
}
class "DiscordBot" as Bot <<service>> {
+on_message(msg)
+on_interaction(interaction)
+run(token)
--
discord.py Client
Receives message events
Sends replies + embeds
Creates threads for workstreams
Renders approval buttons
}
class "ChannelRouter" as Router <<service>> {
+resolve_route(platform, channel_id)
→ ws_id | None
+register_route(channel_id, ws_id)
+resolve_identity(platform, platform_user_id)
→ user_id | None
--
Maps channels → workstreams
Maps platform users → turnstone users
Caches routes in memory
}
class "AsyncRedisBroker" as Broker <<service>> {
+push_inbound(msg)
+subscribe(ws_id) → AsyncIterator
+subscribe_global() → AsyncIterator
+push_response(correlation_id, msg)
--
redis.asyncio client
Pub/sub + queue operations
}
' -- Redis MQ --
class "Redis MQ" as Redis <<mq>> {
turnstone:inbound (LIST)
turnstone:events:{ws_id} (PUBSUB)
turnstone:events:global (PUBSUB)
turnstone:resp:{corr_id} (LIST)
--
Shared message bus
Same queues as bridge protocol
}
' -- Bridge + Server --
class "turnstone-bridge" as Bridge <<bridge>> {
BLPOP turnstone:inbound
Drive server via HTTP
Relay SSE → Redis pub/sub
--
Owns workstream lifecycle
Auto-approve / manual approve
}
class "turnstone-server" as Server <<server>> {
POST /v1/api/send
POST /v1/api/approve
POST /v1/api/workstreams/new
GET /v1/api/events?ws_id=
--
LLM execution + tool use
SSE event stream
}
' -- Storage --
class "channel_users" as CU <<storage>> {
channel_user_id (PK)
platform: "discord" | "slack"
platform_user_id
user_id → users
linked_at
--
/link command creates row
Resolved on each inbound message
}
class "channel_routes" as CR <<storage>> {
channel_type (PK)
channel_id (PK)
ws_id
node_id
created
--
Maps platform channels
to turnstone workstreams
}
' -- Relationships --
Discord --> Bot : gateway\nevents
Bot --> Router : on_message\non_interaction
Router --> Broker : SendMessage\nApproveMessage
Router --> CU : resolve identity
Router --> CR : resolve / register route
Broker --> Redis : RPUSH inbound\nRPUSH resp:{id}
Redis --> Bridge : BLPOP inbound
Bridge --> Server : HTTP API
Server --> Bridge : SSE events
Bridge --> Redis : PUBLISH events:{ws_id}\nPUBLISH events:global
Redis --> Broker : SUBSCRIBE events:{ws_id}
Broker --> Bot : event stream
Bot --> Discord : reply / embed\nbutton callback
Slack .[hidden]. Discord
Teams .[hidden]. Slack
ChannelService --> Bot : creates + runs
ChannelService --> Router : creates
ChannelService --> Broker : creates
' -- Notes --
note right of Bot
**Inbound Flow**
1. Discord message arrives via gateway
2. Bot.on_message() fires
3. ChannelRouter resolves channel → ws_id
(or creates new workstream)
4. ChannelRouter resolves platform user → user_id
via channel_users table
5. Broker.push_inbound(SendMessage)
6. Bridge pops from Redis, drives server
**Session Resume (evicted workstreams)**
1. Stale route detected (no MQ owner)
2. Old session looked up via get_session_id_by_ws()
3. CreateWorkstreamMessage sent with
resume_session=<old_session_id>
4. Server resumes atomically during creation
5. Bridge emits SessionResumedEvent → thread
end note
note right of Broker
**Outbound Flow**
1. Server emits SSE events
2. Bridge relays to Redis events:{ws_id}
3. Broker.subscribe(ws_id) yields events
4. Bot formats and sends to Discord thread
end note
note bottom of CR
**Approval Flow**
1. ApprovalRequestEvent arrives via events:{ws_id}
2. Bot renders Discord buttons (Approve / Deny)
3. User clicks button → on_interaction()
4. Router builds ApproveMessage
5. Broker.push_response(correlation_id, msg)
6. Bridge pops from resp:{id}, calls POST /api/approve
end note
note bottom of CU
**Identity Linking**
1. User runs /link in Discord
2. Bot opens modal requesting API token
3. User submits ts_... API token
4. Bot validates token against storage
5. On success, inserts channel_users row
6. Subsequent messages carry user_id
7. AuthResult scopes applied by server
end note
@enduml
@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:47b106bbcb1041fe4122007065c6cc85605348b42fc7140287194cf25e42e095
size 318036
+17 -1
View File
@@ -27,6 +27,7 @@ Console dashboard: http://localhost:8090
| `server` | 8080 | default | Web UI + chat workstreams + LLM |
| `bridge` | — | default | Redis-to-HTTP bridge (multi-node routing) |
| `console` | 8090 | default | Cluster dashboard |
| `channel` | — | production | Channel gateway (Discord, Slack, etc.) |
| `sim` | — | sim | Multi-node cluster simulator |
## Profiles
@@ -37,6 +38,12 @@ Console dashboard: http://localhost:8090
docker compose up
```
**Production** — adds PostgreSQL and the channel gateway. Requires `POSTGRES_PASSWORD` and (for Discord) `TURNSTONE_DISCORD_TOKEN`:
```bash
docker compose --profile production up
```
**Sim** — adds the simulator. Can run alongside the full stack or standalone with just Redis and the console:
```bash
@@ -105,6 +112,15 @@ The database stores workstream history, user accounts, and API tokens. When usin
>
> You will be prompted to set a password. Use it to log in via the UI or SDK, then create additional users through the admin API. Pass `--token --scopes read,write,approve` to also generate an initial API token.
### Channel Gateway
| Variable | Default | Description |
|----------|---------|-------------|
| `TURNSTONE_DISCORD_TOKEN` | — | Discord bot token (required to enable Discord adapter) |
| `TURNSTONE_DISCORD_GUILD` | `0` | Restrict to a single Discord guild (0 = all guilds) |
The channel service runs in the `production` profile. When `TURNSTONE_DISCORD_TOKEN` is set, the Discord adapter connects to the Discord Gateway and routes messages through Redis MQ to the bridge and server. See [Channel Integrations](channels.md) for full setup instructions including Discord application creation and user account linking.
### Simulator
| Variable | Default | Description |
@@ -146,7 +162,7 @@ docker compose build
docker compose build --no-cache
```
All five entry points are installed in a single image: `turnstone-server`, `turnstone-bridge`, `turnstone-console`, `turnstone-sim`, `turnstone-eval`.
All entry points are installed in a single image: `turnstone-server`, `turnstone-bridge`, `turnstone-console`, `turnstone-channel`, `turnstone-admin`, `turnstone-sim`, `turnstone-eval`.
## Cleanup
+12
View File
@@ -50,6 +50,7 @@ console = ["redis>=7.2"]
sim = ["redis>=7.2"]
anthropic = ["anthropic>=0.39"]
postgres = ["psycopg[binary]>=3.2"]
discord = ["discord.py>=2.4", "redis>=7.2"]
[project.scripts]
@@ -60,6 +61,7 @@ turnstone-bridge = "turnstone.mq.bridge:main"
turnstone-console = "turnstone.console.server:main"
turnstone-sim = "turnstone.sim.cli:main"
turnstone-admin = "turnstone.admin:main"
turnstone-channel = "turnstone.channels.cli:main"
[tool.hatch.build.targets.wheel]
include = [
@@ -144,6 +146,16 @@ ignore_missing_imports = true
module = ["anthropic", "anthropic.*"]
ignore_missing_imports = true
[[tool.mypy.overrides]]
module = ["discord", "discord.*"]
ignore_missing_imports = true
[[tool.mypy.overrides]]
module = ["turnstone.channels.discord.*"]
disallow_subclassing_any = false
disallow_untyped_decorators = false
warn_unused_ignores = false
[[tool.mypy.overrides]]
module = "tests.*"
disallow_untyped_defs = false
+4
View File
@@ -71,11 +71,15 @@ export interface CreateWorkstreamRequest {
name?: string;
model?: string;
auto_approve?: boolean;
resume_session?: string;
}
export interface CreateWorkstreamResponse {
ws_id: string;
name: string;
resumed?: boolean;
session_id?: string;
message_count?: number;
}
export interface CloseWorkstreamRequest {
+181
View File
@@ -0,0 +1,181 @@
"""Tests for turnstone.mq.async_broker.AsyncRedisBroker."""
from __future__ import annotations
import asyncio
import contextlib
from unittest.mock import AsyncMock, MagicMock, patch
import pytest
from turnstone.mq.async_broker import AsyncRedisBroker
@pytest.fixture
def broker() -> AsyncRedisBroker:
return AsyncRedisBroker(host="localhost", port=6379, db=0, prefix="test", response_ttl=120)
@pytest.fixture
def mock_redis() -> AsyncMock:
"""Return a mock Redis client with common async methods."""
r = AsyncMock()
r.rpush = AsyncMock()
r.publish = AsyncMock()
r.expire = AsyncMock()
r.get = AsyncMock(return_value=None)
r.set = AsyncMock()
r.delete = AsyncMock()
r.blpop = AsyncMock(return_value=None)
ps = AsyncMock()
ps.subscribe = AsyncMock()
ps.unsubscribe = AsyncMock()
ps.close = AsyncMock()
ps.get_message = AsyncMock(return_value=None)
r.pubsub = MagicMock(return_value=ps)
return r
def _inject_redis(broker: AsyncRedisBroker, mock_redis: AsyncMock) -> None:
"""Inject a mock Redis client into the broker, simulating connect()."""
broker._redis = mock_redis
broker._pubsub = mock_redis.pubsub()
class TestConstructor:
def test_stores_config(self) -> None:
b = AsyncRedisBroker(host="h", port=1234, db=2, prefix="pfx", password="pw")
assert b._host == "h"
assert b._port == 1234
assert b._db == 2
assert b._prefix == "pfx"
assert b._password == "pw"
assert b._redis is None
def test_defaults(self) -> None:
b = AsyncRedisBroker()
assert b._host == "localhost"
assert b._port == 6379
assert b._prefix == "turnstone"
class TestConnect:
@pytest.mark.anyio
async def test_creates_connection(self) -> None:
b = AsyncRedisBroker()
mock_r = AsyncMock()
mock_r.pubsub = MagicMock(return_value=AsyncMock())
with patch("redis.asyncio.Redis", return_value=mock_r):
await b.connect()
assert b._redis is mock_r
assert b._pubsub is not None
@pytest.mark.anyio
async def test_connect_idempotent(
self, broker: AsyncRedisBroker, mock_redis: AsyncMock
) -> None:
_inject_redis(broker, mock_redis)
old = broker._redis
await broker.connect()
assert broker._redis is old
class TestPushInbound:
@pytest.mark.anyio
async def test_shared_queue(self, broker: AsyncRedisBroker, mock_redis: AsyncMock) -> None:
_inject_redis(broker, mock_redis)
await broker.push_inbound('{"type":"send"}')
mock_redis.rpush.assert_awaited_once_with("test:inbound", '{"type":"send"}')
@pytest.mark.anyio
async def test_per_node_queue(self, broker: AsyncRedisBroker, mock_redis: AsyncMock) -> None:
_inject_redis(broker, mock_redis)
await broker.push_inbound('{"type":"send"}', node_id="node-1")
mock_redis.rpush.assert_awaited_once_with("test:inbound:node-1", '{"type":"send"}')
class TestPublishOutbound:
@pytest.mark.anyio
async def test_publishes(self, broker: AsyncRedisBroker, mock_redis: AsyncMock) -> None:
_inject_redis(broker, mock_redis)
await broker.publish_outbound("test:events:global", '{"event":"data"}')
mock_redis.publish.assert_awaited_once_with("test:events:global", '{"event":"data"}')
class TestPushResponse:
@pytest.mark.anyio
async def test_rpush_and_expire(self, broker: AsyncRedisBroker, mock_redis: AsyncMock) -> None:
_inject_redis(broker, mock_redis)
await broker.push_response("req-123", '{"ok":true}')
mock_redis.rpush.assert_awaited_once_with("test:resp:req-123", '{"ok":true}')
mock_redis.expire.assert_awaited_once_with("test:resp:req-123", 120)
class TestSubscribe:
@pytest.mark.anyio
async def test_creates_task(self, broker: AsyncRedisBroker, mock_redis: AsyncMock) -> None:
_inject_redis(broker, mock_redis)
await broker.subscribe("test:events:global", lambda msg: None)
assert "test:events:global" in broker._callbacks
assert broker._listener_task is not None
assert isinstance(broker._listener_task, asyncio.Task)
# Clean up.
broker._listener_task.cancel()
with contextlib.suppress(asyncio.CancelledError):
await broker._listener_task
class TestUnsubscribe:
@pytest.mark.anyio
async def test_cancels_task(self, broker: AsyncRedisBroker, mock_redis: AsyncMock) -> None:
_inject_redis(broker, mock_redis)
await broker.subscribe("test:events:ch", lambda msg: None)
assert "test:events:ch" in broker._callbacks
await broker.unsubscribe("test:events:ch")
assert "test:events:ch" not in broker._callbacks
class TestRoutingPrimitives:
@pytest.mark.anyio
async def test_get_ws_owner(self, broker: AsyncRedisBroker, mock_redis: AsyncMock) -> None:
_inject_redis(broker, mock_redis)
mock_redis.get.return_value = "node-1"
result = await broker.get_ws_owner("ws-abc")
mock_redis.get.assert_awaited_once_with("test:ws:ws-abc")
assert result == "node-1"
@pytest.mark.anyio
async def test_set_ws_owner(self, broker: AsyncRedisBroker, mock_redis: AsyncMock) -> None:
_inject_redis(broker, mock_redis)
await broker.set_ws_owner("ws-abc", "node-2")
mock_redis.set.assert_awaited_once_with("test:ws:ws-abc", "node-2")
@pytest.mark.anyio
async def test_set_ws_owner_with_ttl(
self, broker: AsyncRedisBroker, mock_redis: AsyncMock
) -> None:
_inject_redis(broker, mock_redis)
await broker.set_ws_owner("ws-abc", "node-2", ttl=300)
mock_redis.set.assert_awaited_once_with("test:ws:ws-abc", "node-2", ex=300)
@pytest.mark.anyio
async def test_del_ws_owner(self, broker: AsyncRedisBroker, mock_redis: AsyncMock) -> None:
_inject_redis(broker, mock_redis)
await broker.del_ws_owner("ws-abc")
mock_redis.delete.assert_awaited_once_with("test:ws:ws-abc")
class TestClose:
@pytest.mark.anyio
async def test_cancels_tasks_and_closes(
self, broker: AsyncRedisBroker, mock_redis: AsyncMock
) -> None:
_inject_redis(broker, mock_redis)
await broker.subscribe("ch1", lambda m: None)
assert len(broker._callbacks) == 1
assert broker._listener_task is not None
await broker.close()
assert len(broker._callbacks) == 0
assert broker._listener_task is None
assert broker._redis is None
assert broker._pubsub is None
+328
View File
@@ -0,0 +1,328 @@
"""Tests for the Discord channel adapter (bot, cog, views, config, CLI)."""
from __future__ import annotations
import asyncio
import sys
from unittest.mock import AsyncMock, MagicMock, patch
import pytest
discord = pytest.importorskip("discord")
# ---------------------------------------------------------------------------
# Helpers
# ---------------------------------------------------------------------------
def _run(coro):
"""Run an async coroutine in a fresh event loop (no pytest-asyncio needed)."""
return asyncio.run(coro)
def _make_message(*, bot=False, guild=True, content="hello", channel=None):
"""Build a mock ``discord.Message``."""
msg = MagicMock(spec=discord.Message)
msg.author = MagicMock()
msg.author.bot = bot
msg.author.id = 12345
msg.content = content
msg.guild = MagicMock() if guild else None
msg.channel = channel or MagicMock()
msg.mentions = []
return msg
def _make_interaction(*, footer_text=None, has_embeds=True):
"""Build a mock ``discord.Interaction``."""
interaction = MagicMock(spec=discord.Interaction)
interaction.user = MagicMock()
interaction.user.id = 67890
interaction.response = MagicMock()
interaction.response.send_message = AsyncMock()
if has_embeds and footer_text is not None:
embed = MagicMock()
embed.footer.text = footer_text
interaction.message = MagicMock()
interaction.message.embeds = [embed]
elif not has_embeds:
interaction.message = MagicMock()
interaction.message.embeds = []
else:
interaction.message = None
return interaction
# ---------------------------------------------------------------------------
# DiscordConfig
# ---------------------------------------------------------------------------
class TestDiscordConfig:
"""Tests for DiscordConfig default and custom values."""
def test_defaults(self):
from turnstone.channels.discord.config import DiscordConfig
cfg = DiscordConfig()
assert cfg.bot_token == ""
assert cfg.guild_id == 0
assert cfg.allowed_channels == []
assert cfg.thread_auto_archive == 1440
assert cfg.max_message_length == 2000
assert cfg.streaming_edit_interval == 1.5
# Inherited from ChannelConfig
assert cfg.redis_host == "localhost"
assert cfg.redis_port == 6379
assert cfg.model == ""
assert cfg.auto_approve is False
def test_custom_values(self):
from turnstone.channels.discord.config import DiscordConfig
cfg = DiscordConfig(
bot_token="tok_123",
guild_id=999,
allowed_channels=[1, 2, 3],
thread_auto_archive=60,
max_message_length=4000,
streaming_edit_interval=0.5,
model="gpt-5",
auto_approve=True,
)
assert cfg.bot_token == "tok_123"
assert cfg.guild_id == 999
assert cfg.allowed_channels == [1, 2, 3]
assert cfg.thread_auto_archive == 60
assert cfg.max_message_length == 4000
assert cfg.streaming_edit_interval == 0.5
assert cfg.model == "gpt-5"
assert cfg.auto_approve is True
# ---------------------------------------------------------------------------
# StreamingMessage
# ---------------------------------------------------------------------------
class TestStreamingMessage:
"""Tests for the StreamingMessage helper in bot.py."""
def test_append_accumulates(self):
from turnstone.channels.discord.bot import StreamingMessage
channel = MagicMock()
channel.send = AsyncMock()
sm = StreamingMessage(channel=channel, edit_interval=999.0)
_run(sm.append("hello "))
_run(sm.append("world"))
assert "".join(sm._buffer) == "hello world"
def test_finalize_sends_when_no_prior_message(self):
from turnstone.channels.discord.bot import StreamingMessage
channel = MagicMock()
channel.send = AsyncMock()
sm = StreamingMessage(channel=channel, edit_interval=999.0)
_run(sm.append("hello"))
_run(sm.finalize())
channel.send.assert_awaited_once_with("hello")
def test_finalize_edits_existing_message(self):
from turnstone.channels.discord.bot import StreamingMessage
channel = MagicMock()
sent_msg = MagicMock()
sent_msg.edit = AsyncMock()
channel.send = AsyncMock(return_value=sent_msg)
sm = StreamingMessage(channel=channel, edit_interval=0.0)
# First append triggers flush (interval=0) which creates the message.
_run(sm.append("hi"))
assert sm._message is sent_msg
_run(sm.append(" there"))
_run(sm.finalize())
# finalize edits the existing message with full content.
sent_msg.edit.assert_awaited_with(content="hi there")
def test_finalize_chunks_long_content(self):
from turnstone.channels.discord.bot import StreamingMessage
channel = MagicMock()
channel.send = AsyncMock()
sm = StreamingMessage(channel=channel, max_length=10, edit_interval=999.0)
# Content longer than max_length should be chunked on finalize.
_run(sm.append("a" * 25))
_run(sm.finalize())
# Should have sent multiple chunks via channel.send.
assert channel.send.await_count >= 2
def test_finalize_empty_is_noop(self):
from turnstone.channels.discord.bot import StreamingMessage
channel = MagicMock()
channel.send = AsyncMock()
sm = StreamingMessage(channel=channel)
_run(sm.finalize())
channel.send.assert_not_awaited()
# ---------------------------------------------------------------------------
# MessageCog._on_message
# ---------------------------------------------------------------------------
class TestMessageCog:
"""Tests for the MessageCog on_message filtering logic."""
def _make_cog(self):
"""Build a MessageCog with a fully mocked bot and TurnstoneBot."""
from turnstone.channels.discord.cog import MessageCog
bot = MagicMock()
bot.user = MagicMock()
bot.user.id = 99999
bot.user.mentioned_in = MagicMock(return_value=False)
ts = MagicMock()
ts._is_allowed_channel = MagicMock(return_value=True)
ts.storage = MagicMock()
ts.router = MagicMock()
ts.router.resolve_user = AsyncMock(return_value="u_abc")
ts.router.send_message = AsyncMock()
ts.config = MagicMock()
ts._ws_tasks = {}
bot.turnstone = ts
cog = MessageCog(bot)
return cog, ts, bot
def test_ignores_bot_messages(self):
cog, ts, _bot = self._make_cog()
msg = _make_message(bot=True)
_run(cog._on_message(msg))
# No router interaction means the message was ignored.
ts.router.send_message.assert_not_awaited()
def test_ignores_own_messages(self):
cog, ts, bot = self._make_cog()
msg = _make_message(bot=False)
msg.author = bot.user # message from ourselves
_run(cog._on_message(msg))
ts.router.send_message.assert_not_awaited()
def test_ignores_dms(self):
cog, ts, _bot = self._make_cog()
msg = _make_message(guild=False)
_run(cog._on_message(msg))
ts.router.send_message.assert_not_awaited()
def test_ignores_non_allowed_channels(self):
cog, ts, _bot = self._make_cog()
ts._is_allowed_channel = MagicMock(return_value=False)
thread = MagicMock(spec=discord.Thread)
thread.id = 111
thread.parent_id = 222
msg = _make_message(channel=thread)
_run(cog._on_message(msg))
ts.router.send_message.assert_not_awaited()
# ---------------------------------------------------------------------------
# _parse_footer (views.py)
# ---------------------------------------------------------------------------
class TestParseFooter:
"""Tests for _parse_footer in views.py."""
def test_valid_footer(self):
from turnstone.channels.discord.views import _parse_footer
interaction = _make_interaction(footer_text="ws_abc|corr_123")
result = _parse_footer(interaction)
assert result == ("ws_abc", "corr_123")
def test_footer_with_pipe_in_correlation(self):
from turnstone.channels.discord.views import _parse_footer
interaction = _make_interaction(footer_text="ws_abc|corr|extra")
result = _parse_footer(interaction)
# split("|", 1) means the second part includes everything after first pipe.
assert result == ("ws_abc", "corr|extra")
def test_no_message_returns_none(self):
from turnstone.channels.discord.views import _parse_footer
interaction = MagicMock()
interaction.message = None
assert _parse_footer(interaction) is None
def test_no_embeds_returns_none(self):
from turnstone.channels.discord.views import _parse_footer
interaction = _make_interaction(has_embeds=False)
assert _parse_footer(interaction) is None
def test_empty_footer_returns_none(self):
from turnstone.channels.discord.views import _parse_footer
# Build an interaction whose embed has footer.text = None.
interaction = MagicMock(spec=discord.Interaction)
embed = MagicMock()
embed.footer.text = None
interaction.message = MagicMock()
interaction.message.embeds = [embed]
assert _parse_footer(interaction) is None
def test_footer_without_pipe_returns_none(self):
from turnstone.channels.discord.views import _parse_footer
interaction = _make_interaction(footer_text="no_pipe_here")
# footer text has no "|" separator
embed = MagicMock()
embed.footer.text = "no_pipe_here"
interaction.message.embeds = [embed]
assert _parse_footer(interaction) is None
# ---------------------------------------------------------------------------
# CLI main() — no adapter configured
# ---------------------------------------------------------------------------
class TestChannelCLI:
"""Tests for the channel CLI entry point."""
def test_exits_without_adapter_token(self):
from turnstone.channels.cli import main
with (
patch.object(sys, "argv", ["turnstone-channel"]),
patch.dict("os.environ", {}, clear=True),
pytest.raises(SystemExit) as exc_info,
):
main()
assert exc_info.value.code == 1
+203
View File
@@ -0,0 +1,203 @@
"""Tests for turnstone.channels._protocol and turnstone.channels._formatter."""
from __future__ import annotations
from turnstone.channels._formatter import (
chunk_message,
format_approval_request,
format_plan_review,
truncate,
)
from turnstone.channels._protocol import ChannelEvent
# ---------------------------------------------------------------------------
# ChannelEvent
# ---------------------------------------------------------------------------
class TestChannelEvent:
def test_construction(self) -> None:
evt = ChannelEvent(
channel_type="discord",
channel_id="ch-1",
channel_user_id="u-42",
message="hello",
parent_channel_id="parent",
metadata={"key": "val"},
)
assert evt.channel_type == "discord"
assert evt.channel_id == "ch-1"
assert evt.channel_user_id == "u-42"
assert evt.message == "hello"
assert evt.parent_channel_id == "parent"
assert evt.metadata == {"key": "val"}
def test_defaults(self) -> None:
evt = ChannelEvent(
channel_type="slack",
channel_id="ch-2",
channel_user_id="u-7",
message="hi",
)
assert evt.parent_channel_id == ""
assert evt.metadata == {}
def test_metadata_independence(self) -> None:
"""Default metadata dicts are independent across instances."""
a = ChannelEvent(channel_type="x", channel_id="1", channel_user_id="u", message="m")
b = ChannelEvent(channel_type="x", channel_id="2", channel_user_id="u", message="m")
a.metadata["key"] = "val"
assert "key" not in b.metadata
# ---------------------------------------------------------------------------
# chunk_message
# ---------------------------------------------------------------------------
class TestChunkMessage:
def test_empty_string(self) -> None:
assert chunk_message("") == [""]
def test_under_limit(self) -> None:
assert chunk_message("short text", max_length=100) == ["short text"]
def test_exactly_at_limit(self) -> None:
text = "a" * 50
assert chunk_message(text, max_length=50) == [text]
def test_splits_at_newline(self) -> None:
text = "line one\nline two\nline three"
chunks = chunk_message(text, max_length=18)
assert len(chunks) >= 2
# The split should happen at a newline boundary within the text.
# Reassembled chunks (with newline separators) should cover all content.
rejoined = "\n".join(chunks)
assert "line one" in rejoined
assert "line three" in rejoined
def test_splits_at_word_boundary(self) -> None:
text = "word1 word2 word3 word4"
chunks = chunk_message(text, max_length=12)
assert len(chunks) >= 2
# No chunk should start with a space (lstrip handles newlines).
for chunk in chunks:
assert not chunk.startswith("\n")
def test_hard_splits(self) -> None:
text = "a" * 30
chunks = chunk_message(text, max_length=10)
assert len(chunks) == 3
assert "".join(chunks) == text
def test_code_block_spanning_boundary(self) -> None:
text = "before\n```\ncode line 1\ncode line 2\ncode line 3\n```\nafter"
chunks = chunk_message(text, max_length=30)
assert len(chunks) >= 2
# If a chunk opens a code block without closing it, the chunker
# should close it and reopen in the next chunk.
for chunk in chunks:
fence_count = chunk.count("```")
assert fence_count % 2 == 0, f"Unmatched code fence in chunk: {chunk!r}"
def test_multiple_code_blocks(self) -> None:
text = "```\nblock1\n```\ntext\n```\nblock2\n```"
chunks = chunk_message(text, max_length=20)
for chunk in chunks:
fence_count = chunk.count("```")
assert fence_count % 2 == 0, f"Unmatched code fence in chunk: {chunk!r}"
def test_custom_max_length(self) -> None:
text = "hello world"
chunks = chunk_message(text, max_length=5)
assert len(chunks) >= 2
assert chunks[0] == "hello"
def test_very_long_single_line(self) -> None:
text = "x" * 5000
chunks = chunk_message(text, max_length=2000)
assert len(chunks) == 3
total = "".join(chunks)
assert total == text
# ---------------------------------------------------------------------------
# format_approval_request
# ---------------------------------------------------------------------------
class TestFormatApprovalRequest:
def test_single_tool(self) -> None:
items = [{"function": {"name": "read_file", "arguments": "/etc/hosts"}}]
result = format_approval_request(items)
assert "Tool approval required" in result
assert "`read_file`" in result
def test_multiple_tools(self) -> None:
items = [
{"function": {"name": "tool_a", "arguments": "arg1"}},
{"function": {"name": "tool_b", "arguments": "arg2"}},
]
result = format_approval_request(items)
assert "`tool_a`" in result
assert "`tool_b`" in result
def test_long_arguments_truncated(self) -> None:
long_args = "x" * 500
items = [{"function": {"name": "fn", "arguments": long_args}}]
result = format_approval_request(items)
# The result should be shorter than the original args.
assert len(result) < 500
def test_server_sse_format(self) -> None:
"""Items from the server SSE use func_name/preview, not function.name."""
items = [
{
"call_id": "c1",
"func_name": "bash",
"preview": "ls -la",
"header": "Execute: ls -la",
"needs_approval": True,
}
]
result = format_approval_request(items)
assert "`bash`" in result
assert "Execute: ls -la" in result
def test_server_sse_format_no_header(self) -> None:
items = [{"func_name": "read_file", "preview": "/etc/hosts"}]
result = format_approval_request(items)
assert "`read_file`" in result
assert "/etc/hosts" in result
# ---------------------------------------------------------------------------
# format_plan_review
# ---------------------------------------------------------------------------
class TestFormatPlanReview:
def test_format(self) -> None:
result = format_plan_review("Step 1: do stuff")
assert result.startswith("**Plan review requested:**")
assert "Step 1: do stuff" in result
# ---------------------------------------------------------------------------
# truncate
# ---------------------------------------------------------------------------
class TestTruncate:
def test_short_text_unchanged(self) -> None:
assert truncate("hello", max_length=200) == "hello"
def test_long_text_truncated(self) -> None:
text = "a" * 300
result = truncate(text, max_length=200)
assert len(result) == 200
assert result.endswith("\u2026")
def test_exactly_at_limit(self) -> None:
text = "b" * 200
assert truncate(text, max_length=200) == text
+107
View File
@@ -0,0 +1,107 @@
"""Tests for turnstone.channels._routing.ChannelRouter."""
from __future__ import annotations
import json
from unittest.mock import AsyncMock, MagicMock
import pytest
from turnstone.channels._routing import ChannelRouter
@pytest.fixture
def mock_broker() -> AsyncMock:
"""Return a mock AsyncRedisBroker."""
broker = AsyncMock()
broker._prefix = "test"
broker.push_inbound = AsyncMock()
broker.push_response = AsyncMock()
broker.subscribe = AsyncMock()
broker.unsubscribe = AsyncMock()
return broker
@pytest.fixture
def mock_storage() -> MagicMock:
"""Return a mock StorageBackend."""
storage = MagicMock()
storage.get_channel_user = MagicMock(return_value=None)
storage.get_channel_route = MagicMock(return_value=None)
storage.get_channel_route_by_ws = MagicMock(return_value=None)
storage.create_channel_route = MagicMock()
storage.delete_channel_route = MagicMock(return_value=True)
return storage
@pytest.fixture
def router(mock_broker: AsyncMock, mock_storage: MagicMock) -> ChannelRouter:
return ChannelRouter(broker=mock_broker, storage=mock_storage)
class TestResolveUser:
@pytest.mark.anyio
async def test_linked_user(self, router: ChannelRouter, mock_storage: MagicMock) -> None:
mock_storage.get_channel_user.return_value = {"user_id": "usr-1", "channel_user_id": "d-42"}
result = await router.resolve_user("discord", "d-42")
assert result == "usr-1"
mock_storage.get_channel_user.assert_called_once_with("discord", "d-42")
@pytest.mark.anyio
async def test_unlinked_user(self, router: ChannelRouter, mock_storage: MagicMock) -> None:
mock_storage.get_channel_user.return_value = None
result = await router.resolve_user("slack", "s-99")
assert result is None
class TestSendMessage:
@pytest.mark.anyio
async def test_pushes_send_message(self, router: ChannelRouter, mock_broker: AsyncMock) -> None:
cid = await router.send_message("ws-1", "hello world")
assert isinstance(cid, str)
assert len(cid) > 0
mock_broker.push_inbound.assert_awaited_once()
raw = mock_broker.push_inbound.call_args[0][0]
payload = json.loads(raw)
assert payload["type"] == "send"
assert payload["ws_id"] == "ws-1"
assert payload["message"] == "hello world"
assert payload["correlation_id"] == cid
class TestSendApproval:
@pytest.mark.anyio
async def test_pushes_to_response_queue(
self, router: ChannelRouter, mock_broker: AsyncMock
) -> None:
await router.send_approval("ws-1", "corr-abc", approved=True, feedback="ok")
mock_broker.push_response.assert_awaited_once()
queue_name = mock_broker.push_response.call_args[0][0]
assert queue_name == "corr-abc"
raw = mock_broker.push_response.call_args[0][1]
payload = json.loads(raw)
assert payload["type"] == "approve"
assert payload["approved"] is True
assert payload["ws_id"] == "ws-1"
class TestSendPlanFeedback:
@pytest.mark.anyio
async def test_pushes_to_response_queue(
self, router: ChannelRouter, mock_broker: AsyncMock
) -> None:
await router.send_plan_feedback("ws-2", "corr-xyz", "looks good")
mock_broker.push_response.assert_awaited_once()
raw = mock_broker.push_response.call_args[0][1]
payload = json.loads(raw)
assert payload["type"] == "plan_feedback"
assert payload["feedback"] == "looks good"
class TestDeleteRoute:
@pytest.mark.anyio
async def test_calls_storage_delete(
self, router: ChannelRouter, mock_storage: MagicMock
) -> None:
await router.delete_route("discord", "ch-123")
mock_storage.delete_channel_route.assert_called_once_with("discord", "ch-123")
+136
View File
@@ -0,0 +1,136 @@
"""Tests for channel_users and channel_routes storage CRUD."""
from __future__ import annotations
import pytest
from turnstone.core.storage._sqlite import SQLiteBackend
@pytest.fixture
def db(tmp_path):
"""Fresh SQLite backend for each test."""
backend = SQLiteBackend(str(tmp_path / "test.db"))
return backend
class TestChannelUserCRUD:
"""Tests for channel_users table operations."""
def test_create_and_get(self, db):
db.create_channel_user("discord", "12345", "u_abc")
result = db.get_channel_user("discord", "12345")
assert result is not None
assert result["channel_type"] == "discord"
assert result["channel_user_id"] == "12345"
assert result["user_id"] == "u_abc"
assert "created" in result
def test_get_nonexistent(self, db):
assert db.get_channel_user("discord", "99999") is None
def test_create_duplicate_noop(self, db):
db.create_channel_user("discord", "12345", "u_abc")
db.create_channel_user("discord", "12345", "u_different")
result = db.get_channel_user("discord", "12345")
assert result is not None
assert result["user_id"] == "u_abc" # first write wins
def test_same_user_different_channels(self, db):
db.create_channel_user("discord", "d_123", "u_abc")
db.create_channel_user("slack", "s_456", "u_abc")
d = db.get_channel_user("discord", "d_123")
s = db.get_channel_user("slack", "s_456")
assert d is not None and d["user_id"] == "u_abc"
assert s is not None and s["user_id"] == "u_abc"
def test_list_by_user(self, db):
db.create_channel_user("discord", "d_123", "u_abc")
db.create_channel_user("slack", "s_456", "u_abc")
db.create_channel_user("discord", "d_999", "u_other")
results = db.list_channel_users_by_user("u_abc")
assert len(results) == 2
types = {r["channel_type"] for r in results}
assert types == {"discord", "slack"}
def test_list_by_user_empty(self, db):
assert db.list_channel_users_by_user("u_nobody") == []
def test_delete(self, db):
db.create_channel_user("discord", "12345", "u_abc")
assert db.delete_channel_user("discord", "12345") is True
assert db.get_channel_user("discord", "12345") is None
def test_delete_nonexistent(self, db):
assert db.delete_channel_user("discord", "99999") is False
def test_delete_user_cascades_channel_users(self, db):
"""Deleting a turnstone user should cascade to channel_users."""
db.create_user("u_abc", "admin", "Admin", "hash123")
db.create_channel_user("discord", "12345", "u_abc")
db.delete_user("u_abc")
assert db.get_channel_user("discord", "12345") is None
class TestChannelRouteCRUD:
"""Tests for channel_routes table operations."""
def test_create_and_get(self, db):
db.create_channel_route("discord", "thread_123", "ws_abc", "node_1")
result = db.get_channel_route("discord", "thread_123")
assert result is not None
assert result["channel_type"] == "discord"
assert result["channel_id"] == "thread_123"
assert result["ws_id"] == "ws_abc"
assert result["node_id"] == "node_1"
assert "created" in result
def test_get_nonexistent(self, db):
assert db.get_channel_route("discord", "thread_999") is None
def test_create_duplicate_noop(self, db):
db.create_channel_route("discord", "thread_123", "ws_abc")
db.create_channel_route("discord", "thread_123", "ws_different")
result = db.get_channel_route("discord", "thread_123")
assert result is not None
assert result["ws_id"] == "ws_abc" # first write wins
def test_default_empty_node_id(self, db):
db.create_channel_route("discord", "thread_123", "ws_abc")
result = db.get_channel_route("discord", "thread_123")
assert result is not None
assert result["node_id"] == ""
def test_get_by_ws(self, db):
db.create_channel_route("discord", "thread_123", "ws_abc", "node_1")
result = db.get_channel_route_by_ws("ws_abc")
assert result is not None
assert result["channel_id"] == "thread_123"
assert result["ws_id"] == "ws_abc"
def test_get_by_ws_nonexistent(self, db):
assert db.get_channel_route_by_ws("ws_nobody") is None
def test_delete(self, db):
db.create_channel_route("discord", "thread_123", "ws_abc")
assert db.delete_channel_route("discord", "thread_123") is True
assert db.get_channel_route("discord", "thread_123") is None
def test_delete_nonexistent(self, db):
assert db.delete_channel_route("discord", "thread_999") is False
def test_multiple_channels_same_type(self, db):
db.create_channel_route("discord", "thread_1", "ws_1")
db.create_channel_route("discord", "thread_2", "ws_2")
r1 = db.get_channel_route("discord", "thread_1")
r2 = db.get_channel_route("discord", "thread_2")
assert r1 is not None and r1["ws_id"] == "ws_1"
assert r2 is not None and r2["ws_id"] == "ws_2"
def test_different_channel_types(self, db):
db.create_channel_route("discord", "thread_1", "ws_1")
db.create_channel_route("slack", "channel_1", "ws_2")
d = db.get_channel_route("discord", "thread_1")
s = db.get_channel_route("slack", "channel_1")
assert d is not None and d["ws_id"] == "ws_1"
assert s is not None and s["ws_id"] == "ws_2"
+112
View File
@@ -0,0 +1,112 @@
"""Tests for the atomic workstream resumption flow.
Covers CreateWorkstreamMessage resume_session field, SessionResumedEvent,
WorkstreamCreatedEvent resumed fields, and server endpoint handling.
"""
from __future__ import annotations
import json
from turnstone.mq.protocol import (
CreateWorkstreamMessage,
SessionResumedEvent,
WorkstreamCreatedEvent,
)
# ---------------------------------------------------------------------------
# Protocol tests
# ---------------------------------------------------------------------------
class TestCreateWorkstreamMessageResumeField:
def test_resume_session_defaults_empty(self) -> None:
msg = CreateWorkstreamMessage(name="test")
assert msg.resume_session == ""
def test_resume_session_set(self) -> None:
msg = CreateWorkstreamMessage(name="test", resume_session="sess-abc")
assert msg.resume_session == "sess-abc"
def test_resume_session_serializes(self) -> None:
msg = CreateWorkstreamMessage(resume_session="sess-xyz")
data = json.loads(msg.to_json())
assert data["resume_session"] == "sess-xyz"
def test_resume_session_deserializes(self) -> None:
msg = CreateWorkstreamMessage(resume_session="sess-123")
raw = msg.to_json()
from turnstone.mq.protocol import InboundMessage
restored = InboundMessage.from_json(raw)
assert getattr(restored, "resume_session", "") == "sess-123"
class TestWorkstreamCreatedEventResumeFields:
def test_default_not_resumed(self) -> None:
event = WorkstreamCreatedEvent(ws_id="ws-1", name="test")
assert event.resumed is False
assert event.session_id == ""
assert event.message_count == 0
def test_resumed_fields(self) -> None:
event = WorkstreamCreatedEvent(
ws_id="ws-1", name="test", resumed=True, session_id="s-1", message_count=42
)
assert event.resumed is True
assert event.session_id == "s-1"
assert event.message_count == 42
def test_serializes_resumed_fields(self) -> None:
event = WorkstreamCreatedEvent(
ws_id="ws-1", resumed=True, session_id="s-1", message_count=10
)
data = json.loads(event.to_json())
assert data["resumed"] is True
assert data["session_id"] == "s-1"
assert data["message_count"] == 10
def test_deserializes_resumed_fields(self) -> None:
event = WorkstreamCreatedEvent(
ws_id="ws-1", resumed=True, session_id="s-1", message_count=5
)
from turnstone.mq.protocol import OutboundEvent
restored = OutboundEvent.from_json(event.to_json())
assert isinstance(restored, WorkstreamCreatedEvent)
assert restored.resumed is True
assert restored.session_id == "s-1"
assert restored.message_count == 5
class TestSessionResumedEvent:
def test_defaults(self) -> None:
event = SessionResumedEvent(ws_id="ws-1")
assert event.type == "session_resumed"
assert event.session_id == ""
assert event.message_count == 0
assert event.name == ""
def test_with_values(self) -> None:
event = SessionResumedEvent(
ws_id="ws-1", session_id="s-abc", message_count=25, name="My Chat"
)
assert event.session_id == "s-abc"
assert event.message_count == 25
assert event.name == "My Chat"
def test_round_trip(self) -> None:
event = SessionResumedEvent(ws_id="ws-1", session_id="s-abc", message_count=10, name="Chat")
from turnstone.mq.protocol import OutboundEvent
restored = OutboundEvent.from_json(event.to_json())
assert isinstance(restored, SessionResumedEvent)
assert restored.session_id == "s-abc"
assert restored.message_count == 10
assert restored.name == "Chat"
def test_registered_in_outbound_registry(self) -> None:
from turnstone.mq.protocol import _OUTBOUND_REGISTRY
assert "session_resumed" in _OUTBOUND_REGISTRY
assert _OUTBOUND_REGISTRY["session_resumed"] is SessionResumedEvent
+7
View File
@@ -39,11 +39,18 @@ class CreateWorkstreamRequest(BaseModel):
name: str = Field(default="", description="Workstream display name (auto-generated if empty)")
model: str = Field(default="", description="Model alias from registry")
auto_approve: bool = Field(default=False, description="Auto-approve all tool calls")
resume_session: str = Field(
default="",
description="Session ID to resume atomically during creation (empty = fresh start)",
)
class CreateWorkstreamResponse(BaseModel):
ws_id: str = Field(description="Unique ID of the new workstream")
name: str = Field(description="Assigned workstream name")
resumed: bool = Field(default=False, description="Whether a previous session was resumed")
session_id: str = Field(default="", description="Resolved session ID (set when resumed)")
message_count: int = Field(default=0, description="Number of messages in the resumed session")
class CloseWorkstreamRequest(BaseModel):
+15
View File
@@ -0,0 +1,15 @@
"""Shared channel infrastructure for turnstone communication integrations.
Provides the :class:`ChannelAdapter` protocol, the :class:`ChannelEvent`
normalized event type, the :class:`ChannelRouter` for workstream mapping,
and shared formatting / configuration utilities.
"""
from turnstone.channels._protocol import ChannelAdapter, ChannelEvent
from turnstone.channels._routing import ChannelRouter
__all__ = [
"ChannelAdapter",
"ChannelEvent",
"ChannelRouter",
]
+23
View File
@@ -0,0 +1,23 @@
"""Base configuration shared by all channel adapters."""
from __future__ import annotations
from dataclasses import dataclass, field
@dataclass
class ChannelConfig:
"""Base configuration shared by all channel adapters.
Individual adapters extend this with platform-specific fields (tokens,
guild IDs, etc.).
"""
redis_host: str = "localhost"
redis_port: int = 6379
redis_db: int = 0
redis_password: str | None = None
prefix: str = "turnstone"
model: str = ""
auto_approve: bool = False
auto_approve_tools: list[str] = field(default_factory=list)
+116
View File
@@ -0,0 +1,116 @@
"""Message formatting utilities for channel adapters.
Handles chunking long messages for platforms with character limits, formatting
tool-approval requests, and plan-review prompts.
"""
from __future__ import annotations
from typing import Any
def chunk_message(text: str, max_length: int = 2000) -> list[str]:
"""Split *text* into chunks that fit within *max_length*.
Respects code-block boundaries: if a fenced code block (````` ```)
spans a chunk boundary the current chunk is closed with ````` ``` ``
and the next chunk reopens it. Prefers splitting at newline
boundaries, then word boundaries, then hard-splits.
"""
if len(text) <= max_length:
return [text]
chunks: list[str] = []
remaining = text
in_code_block = False
while remaining:
if len(remaining) <= max_length:
chunks.append(remaining)
break
# Reserve space for a closing ``` if we're inside a code block.
limit = max_length - 4 if in_code_block else max_length
limit = max(limit, 1)
candidate = remaining[:limit]
# Prefer a newline boundary.
split_idx = candidate.rfind("\n")
if split_idx <= 0:
# Fall back to a word boundary.
split_idx = candidate.rfind(" ")
if split_idx <= 0:
# Hard split.
split_idx = limit
chunk = remaining[:split_idx]
remaining = remaining[split_idx:].lstrip("\n")
# Track code-block fences in this chunk.
fence_count = chunk.count("```")
block_open = in_code_block
if fence_count % 2 != 0:
in_code_block = not in_code_block
# If we end inside a code block, close it in this chunk and
# reopen in the next.
if in_code_block:
chunk += "\n```"
remaining = "```\n" + remaining
in_code_block = False
elif block_open and fence_count % 2 != 0:
# We were inside a code block and the chunk closed it
# properly -- nothing extra needed.
pass
chunks.append(chunk)
return chunks
def format_approval_request(items: list[dict[str, Any]]) -> str:
"""Format tool-approval *items* into a human-readable message.
Items use the server's SSE format: ``func_name``, ``preview``,
``approval_label``, ``header``. Falls back to the nested
``function.name`` format for compatibility.
"""
lines: list[str] = ["**Tool approval required:**"]
for item in items:
# Server SSE format: top-level func_name / preview
name = item.get("func_name") or item.get("approval_label", "")
if not name:
# Fallback: nested function.name (SDK / older format)
func = item.get("function", {})
name = func.get("name", "unknown")
preview = item.get("preview", "")
if not preview:
args = item.get("function", {}).get("arguments", "")
if isinstance(args, dict):
import json
args = json.dumps(args, ensure_ascii=False)
preview = str(args)
preview = truncate(preview)
header = item.get("header", "")
if header:
lines.append(f"\u2022 `{name}`: {header}")
elif preview:
lines.append(f"\u2022 `{name}`: {preview}")
else:
lines.append(f"\u2022 `{name}`")
return "\n".join(lines)
def format_plan_review(content: str) -> str:
"""Format a plan-review prompt with a header."""
return f"**Plan review requested:**\n\n{content}"
def truncate(text: str, max_length: int = 200) -> str:
"""Truncate *text* to *max_length*, appending an ellipsis if trimmed."""
if len(text) <= max_length:
return text
return text[: max_length - 1] + "\u2026"
+70
View File
@@ -0,0 +1,70 @@
"""Channel adapter protocol and normalized event type.
Defines the :class:`ChannelEvent` data class for inbound events and the
:class:`ChannelAdapter` structural protocol that all bidirectional channel
adapters (Discord, Slack, etc.) must satisfy.
"""
from __future__ import annotations
from dataclasses import dataclass, field
from typing import Any, Protocol, runtime_checkable
@dataclass
class ChannelEvent:
"""Normalized inbound event from any channel."""
channel_type: str # "discord", "slack"
channel_id: str # thread/channel ID
channel_user_id: str # platform user ID
message: str
parent_channel_id: str = "" # main channel (for thread creation)
metadata: dict[str, Any] = field(default_factory=dict)
@runtime_checkable
class ChannelAdapter(Protocol):
"""Protocol for bidirectional channel adapters."""
channel_type: str
async def start(self) -> None:
"""Connect to the platform and begin listening for events."""
...
async def stop(self) -> None:
"""Disconnect and release resources."""
...
async def send(self, channel_id: str, content: str) -> str:
"""Send a message to a channel. Returns the platform message ID."""
...
async def edit_message(self, channel_id: str, message_id: str, content: str) -> None:
"""Edit an existing message in a channel."""
...
async def send_approval_request(
self,
channel_id: str,
ws_id: str,
correlation_id: str,
items: list[dict[str, Any]],
) -> None:
"""Send an interactive tool-approval prompt to a channel."""
...
async def send_plan_review(
self,
channel_id: str,
ws_id: str,
correlation_id: str,
content: str,
) -> None:
"""Send a plan-review prompt to a channel."""
...
async def create_thread(self, parent_channel_id: str, name: str, message_id: str = "") -> str:
"""Create a thread under a parent channel. Returns the new thread ID."""
...
+304
View File
@@ -0,0 +1,304 @@
"""Channel router -- maps external channels/threads to turnstone workstreams.
:class:`ChannelRouter` uses the async Redis broker for MQ communication and
the storage backend for persistent channel-to-workstream mappings.
"""
from __future__ import annotations
import asyncio
from typing import TYPE_CHECKING
from turnstone.core.log import get_logger
from turnstone.mq.protocol import (
ApproveMessage,
CreateWorkstreamMessage,
OutboundEvent,
PlanFeedbackMessage,
SendMessage,
WorkstreamClosedEvent,
WorkstreamCreatedEvent,
)
if TYPE_CHECKING:
from turnstone.core.storage import StorageBackend
from turnstone.mq.async_broker import AsyncRedisBroker
log = get_logger(__name__)
_WS_CREATE_TIMEOUT = 30.0 # seconds
class ChannelRouter:
"""Manage channel-to-workstream routing and MQ message dispatch.
Parameters
----------
broker:
An :class:`AsyncRedisBroker` used for pub/sub and queue operations.
storage:
A :class:`StorageBackend` instance for persistent route lookups.
All storage calls are synchronous and will be wrapped in
:func:`asyncio.to_thread`.
"""
def __init__(
self,
broker: AsyncRedisBroker,
storage: StorageBackend,
*,
auto_approve: bool = False,
auto_approve_tools: list[str] | None = None,
) -> None:
self._broker = broker
self._storage = storage
self._auto_approve = auto_approve
self._auto_approve_tools: list[str] = auto_approve_tools or []
self._pending: dict[str, asyncio.Event] = {}
self._pending_results: dict[str, str] = {}
self._global_task: asyncio.Task[None] | None = None
self._create_locks: dict[str, asyncio.Lock] = {}
# -- lifecycle -----------------------------------------------------------
async def start(self) -> None:
"""Subscribe to global events for workstream lifecycle."""
channel = f"{self._broker._prefix}:events:global"
await self._broker.subscribe(channel, self._on_global_event)
log.info("channel_router.started", channel=channel)
async def stop(self) -> None:
"""Unsubscribe and clean up pending state."""
channel = f"{self._broker._prefix}:events:global"
await self._broker.unsubscribe(channel)
# Wake any waiters so they don't hang forever.
for evt in self._pending.values():
evt.set()
self._pending.clear()
self._pending_results.clear()
log.info("channel_router.stopped")
# -- event handler -------------------------------------------------------
async def _on_global_event(self, raw: str) -> None:
"""Handle events on the global pub/sub channel.
Exceptions are caught so the broker listener task stays alive.
"""
try:
event = OutboundEvent.from_json(raw)
if isinstance(event, WorkstreamCreatedEvent):
cid = event.correlation_id
if cid in self._pending:
self._pending_results[cid] = event.ws_id
self._pending[cid].set()
log.debug(
"channel_router.ws_created",
ws_id=event.ws_id,
correlation_id=cid,
)
elif isinstance(event, WorkstreamClosedEvent):
ws_id = event.ws_id
route = await asyncio.to_thread(self._storage.get_channel_route_by_ws, ws_id)
if route:
# Don't delete the route — the workstream may have been evicted
# and the thread can reactivate it. Route cleanup only happens
# via explicit /close or delete_route().
log.info(
"channel_router.ws_closed_route_kept",
ws_id=ws_id,
channel_type=route["channel_type"],
channel_id=route["channel_id"],
)
except Exception:
log.exception("channel_router.global_event_error")
# -- workstream management -----------------------------------------------
async def get_or_create_workstream(
self,
channel_type: str,
channel_id: str,
name: str = "",
model: str = "",
initial_message: str = "",
) -> tuple[str, bool]:
"""Look up or create a workstream for a channel.
Returns ``(ws_id, is_new)`` where *is_new* is ``True`` when a new
workstream was created.
A per-channel lock prevents duplicate workstreams when concurrent
messages arrive for the same channel before the first creation
completes.
"""
key = f"{channel_type}:{channel_id}"
lock = self._create_locks.setdefault(key, asyncio.Lock())
old_ws_id: str | None = None
async with lock:
# 1. Check for existing route.
old_ws_id = ""
route = await asyncio.to_thread(
self._storage.get_channel_route, channel_type, channel_id
)
if route:
# Verify the workstream is still alive (owned by a bridge node).
owner = await self._broker.get_ws_owner(route["ws_id"])
if owner:
return route["ws_id"], False
# Workstream was evicted/closed — capture old ws_id for session
# resume, then remove the stale route.
old_ws_id = route["ws_id"]
await asyncio.to_thread(
self._storage.delete_channel_route, channel_type, channel_id
)
log.info(
"channel_router.stale_route_cleared",
ws_id=old_ws_id,
channel_type=channel_type,
channel_id=channel_id,
)
# 2. Look up old session for atomic resume (if stale route).
resume_session = ""
if old_ws_id:
old_sid: str | None = await asyncio.to_thread(
self._storage.get_session_id_by_ws, old_ws_id
)
resume_session = old_sid or ""
# 3. Create via MQ with atomic resume.
msg = CreateWorkstreamMessage(
name=name,
model=model,
initial_message="" if resume_session else initial_message,
resume_session=resume_session,
auto_approve=self._auto_approve,
auto_approve_tools=list(self._auto_approve_tools),
)
cid = msg.correlation_id
waiter = asyncio.Event()
self._pending[cid] = waiter
await self._broker.push_inbound(msg.to_json())
log.info(
"channel_router.creating_workstream",
correlation_id=cid,
channel_type=channel_type,
channel_id=channel_id,
resume_session=resume_session or None,
)
try:
await asyncio.wait_for(waiter.wait(), timeout=_WS_CREATE_TIMEOUT)
except TimeoutError:
self._pending.pop(cid, None)
self._pending_results.pop(cid, None)
raise
ws_id = self._pending_results.pop(cid, "")
self._pending.pop(cid, None)
if not ws_id:
msg_err = "workstream creation returned empty ws_id"
raise RuntimeError(msg_err)
# 4. Persist the route.
await asyncio.to_thread(
self._storage.create_channel_route, channel_type, channel_id, ws_id
)
log.info(
"channel_router.route_created",
ws_id=ws_id,
channel_type=channel_type,
channel_id=channel_id,
)
return ws_id, True
# -- user resolution -----------------------------------------------------
async def resolve_user(self, channel_type: str, channel_user_id: str) -> str | None:
"""Resolve an external platform user to a turnstone ``user_id``.
Returns ``None`` if no mapping exists.
"""
result = await asyncio.to_thread(
self._storage.get_channel_user, channel_type, channel_user_id
)
if result is None:
return None
return result.get("user_id")
# -- message dispatch ----------------------------------------------------
async def send_message(self, ws_id: str, message: str) -> str:
"""Push a :class:`SendMessage` to the broker.
Returns the ``correlation_id`` of the submitted message.
"""
msg = SendMessage(
ws_id=ws_id,
message=message,
auto_approve=self._auto_approve,
auto_approve_tools=list(self._auto_approve_tools),
)
await self._broker.push_inbound(msg.to_json())
log.debug("channel_router.send_message", ws_id=ws_id, correlation_id=msg.correlation_id)
return msg.correlation_id
async def send_approval(
self,
ws_id: str,
correlation_id: str,
approved: bool,
feedback: str = "",
always: bool = False,
) -> None:
"""Push an :class:`ApproveMessage` to the broker response queue."""
msg = ApproveMessage(
ws_id=ws_id,
request_id=correlation_id,
approved=approved,
feedback=feedback or None,
always=always,
)
await self._broker.push_response(correlation_id, msg.to_json())
log.debug(
"channel_router.send_approval",
ws_id=ws_id,
correlation_id=correlation_id,
approved=approved,
)
async def send_plan_feedback(self, ws_id: str, correlation_id: str, feedback: str) -> None:
"""Push a :class:`PlanFeedbackMessage` to the broker response queue."""
msg = PlanFeedbackMessage(
ws_id=ws_id,
request_id=correlation_id,
feedback=feedback,
)
await self._broker.push_response(correlation_id, msg.to_json())
log.debug(
"channel_router.send_plan_feedback",
ws_id=ws_id,
correlation_id=correlation_id,
)
# -- route management ----------------------------------------------------
async def delete_route(self, channel_type: str, channel_id: str) -> None:
"""Remove a channel-to-workstream mapping."""
deleted = await asyncio.to_thread(
self._storage.delete_channel_route, channel_type, channel_id
)
log.info(
"channel_router.delete_route",
channel_type=channel_type,
channel_id=channel_id,
deleted=deleted,
)
+174
View File
@@ -0,0 +1,174 @@
"""Unified channel gateway entry point.
Launches one or more channel adapters (Discord, Slack, etc.) connected to
the turnstone cluster via Redis MQ. Currently supports Discord; future
adapters will be added as additional ``--*-token`` flags.
Run as: ``turnstone-channel --discord-token $TURNSTONE_DISCORD_TOKEN``
"""
from __future__ import annotations
import os
import sys
def main() -> None:
"""Parse arguments, initialize storage and broker, and run adapters."""
import argparse
parser = argparse.ArgumentParser(
description="turnstone channel gateway — bridges messaging platforms to the turnstone cluster"
)
# -- Redis ---------------------------------------------------------------
parser.add_argument(
"--redis-host",
default=os.environ.get("REDIS_HOST", "localhost"),
help="Redis host (default: $REDIS_HOST or localhost)",
)
parser.add_argument(
"--redis-port",
type=int,
default=int(os.environ.get("REDIS_PORT", "6379")),
help="Redis port (default: %(default)s)",
)
parser.add_argument(
"--redis-password",
default=os.environ.get("REDIS_PASSWORD"),
help="Redis password (default: $REDIS_PASSWORD)",
)
parser.add_argument(
"--redis-db",
type=int,
default=0,
help="Redis DB number (default: %(default)s)",
)
# -- Discord -------------------------------------------------------------
parser.add_argument(
"--discord-token",
default=os.environ.get("TURNSTONE_DISCORD_TOKEN", ""),
help="Discord bot token (default: $TURNSTONE_DISCORD_TOKEN)",
)
parser.add_argument(
"--discord-guild",
type=int,
default=0,
help="Restrict to a single Discord guild (0 = all, default: %(default)s)",
)
parser.add_argument(
"--discord-channels",
default="",
help="Comma-separated list of allowed Discord channel IDs (default: all)",
)
# -- Workstream defaults -------------------------------------------------
parser.add_argument(
"--model",
default="",
help="Default model for new workstreams (default: server default)",
)
parser.add_argument(
"--auto-approve",
action="store_true",
help="Auto-approve all tool calls",
)
# -- Logging -------------------------------------------------------------
parser.add_argument(
"--log-level",
default="INFO",
choices=["DEBUG", "INFO", "WARNING", "ERROR"],
help="Log level (default: %(default)s)",
)
parser.add_argument(
"--log-format",
default="auto",
choices=["auto", "json", "text"],
help="Log output format (default: auto -- JSON when stderr is not a TTY)",
)
args = parser.parse_args()
# -- Logging setup -------------------------------------------------------
from turnstone.core.log import configure_logging
configure_logging(
level=args.log_level,
json_output={"json": True, "text": False}.get(args.log_format),
service="channel",
)
from turnstone.core.log import get_logger
log = get_logger(__name__)
# -- Storage -------------------------------------------------------------
from turnstone.core.storage._registry import init_storage
db_backend = os.environ.get("TURNSTONE_DB_BACKEND", "sqlite")
db_url = os.environ.get("TURNSTONE_DB_URL", "")
db_path = os.environ.get("TURNSTONE_DB_PATH", "")
init_storage(
backend=db_backend,
url=db_url,
path=db_path,
)
# -- Broker --------------------------------------------------------------
from turnstone.mq.async_broker import AsyncRedisBroker
broker = AsyncRedisBroker(
host=args.redis_host,
port=args.redis_port,
db=args.redis_db,
password=args.redis_password,
)
# -- Adapter selection ---------------------------------------------------
adapters_configured = False
if args.discord_token:
adapters_configured = True
if not adapters_configured:
print(
"Error: no channel adapters configured. "
"Set --discord-token or $TURNSTONE_DISCORD_TOKEN.",
file=sys.stderr,
)
sys.exit(1)
# -- Run -----------------------------------------------------------------
if args.discord_token:
from turnstone.channels.discord.bot import TurnstoneBot
from turnstone.channels.discord.config import DiscordConfig
from turnstone.core.storage._registry import get_storage
allowed_channels: list[int] = []
if args.discord_channels:
allowed_channels = [
int(c.strip()) for c in args.discord_channels.split(",") if c.strip()
]
config = DiscordConfig(
redis_host=args.redis_host,
redis_port=args.redis_port,
redis_db=args.redis_db,
redis_password=args.redis_password,
model=args.model,
auto_approve=args.auto_approve,
bot_token=args.discord_token,
guild_id=args.discord_guild,
allowed_channels=allowed_channels,
)
bot = TurnstoneBot(config, broker, get_storage())
log.info("channel.starting", adapter="discord", guild_id=config.guild_id)
bot.run()
if __name__ == "__main__":
main()
+9
View File
@@ -0,0 +1,9 @@
"""Discord channel adapter for turnstone."""
from turnstone.channels.discord.bot import TurnstoneBot
from turnstone.channels.discord.config import DiscordConfig
__all__ = [
"DiscordConfig",
"TurnstoneBot",
]
+357
View File
@@ -0,0 +1,357 @@
"""Discord bot adapter — connects Discord threads to turnstone workstreams.
:class:`TurnstoneBot` extends ``discord.ext.commands.Bot`` and manages the
lifecycle of event subscriptions, streaming message edits, and interactive
approval / plan-review views.
"""
from __future__ import annotations
import asyncio
import time
from dataclasses import dataclass, field
from typing import TYPE_CHECKING
from turnstone.channels._formatter import chunk_message
from turnstone.channels._routing import ChannelRouter
from turnstone.core.log import get_logger
from turnstone.mq.protocol import (
ApprovalRequestEvent,
ContentEvent,
ErrorEvent,
OutboundEvent,
PlanReviewEvent,
SessionResumedEvent,
TurnCompleteEvent,
)
if TYPE_CHECKING:
import discord
from discord.ext import commands
from turnstone.channels.discord.config import DiscordConfig
from turnstone.core.storage._protocol import StorageBackend
from turnstone.mq.async_broker import AsyncRedisBroker
log = get_logger(__name__)
# ---------------------------------------------------------------------------
# StreamingMessage helper
# ---------------------------------------------------------------------------
@dataclass
class StreamingMessage:
"""Accumulates streamed content and periodically edits a Discord message.
Discord rate-limits message edits, so we batch content updates and flush
at a configurable interval. On finalize we send any remaining content
and chunk if the total exceeds the platform limit.
"""
channel: discord.abc.Messageable
max_length: int = 2000
edit_interval: float = 1.5
_message: discord.Message | None = field(default=None, init=False, repr=False)
_buffer: list[str] = field(default_factory=list, init=False, repr=False)
_last_edit: float = field(default=0.0, init=False, repr=False)
async def append(self, text: str) -> None:
"""Add *text* to the buffer and edit the message if the interval has elapsed."""
self._buffer.append(text)
now = time.monotonic()
if now - self._last_edit >= self.edit_interval:
await self._flush()
async def finalize(self) -> None:
"""Flush any remaining buffered content, chunking if necessary."""
content = "".join(self._buffer)
if not content:
return
if self._message is not None:
# Final edit — may need chunking if content grew beyond the limit.
chunks = chunk_message(content, self.max_length)
try:
await self._message.edit(content=chunks[0])
except Exception:
log.debug("streaming_message.edit_failed_on_finalize")
# Any overflow chunks are sent as new messages.
for chunk in chunks[1:]:
await self.channel.send(chunk)
else:
# Never sent an initial message — send all chunks now.
for chunk in chunk_message(content, self.max_length):
await self.channel.send(chunk)
async def _flush(self) -> None:
"""Edit or create the message with the current buffer contents."""
content = "".join(self._buffer)
if not content:
return
# Truncate to max_length for the in-progress edit (finalize handles overflow).
display = content[: self.max_length]
try:
if self._message is None:
self._message = await self.channel.send(display)
else:
await self._message.edit(content=display)
except Exception:
log.debug("streaming_message.flush_failed")
self._last_edit = time.monotonic()
# ---------------------------------------------------------------------------
# TurnstoneBot
# ---------------------------------------------------------------------------
class TurnstoneBot:
"""Discord bot that bridges Discord threads to turnstone workstreams.
Parameters
----------
config:
Discord-specific configuration.
broker:
Async Redis broker for MQ communication.
storage:
Storage backend for persistent route / user lookups.
"""
channel_type: str = "discord"
def __init__(
self,
config: DiscordConfig,
broker: AsyncRedisBroker,
storage: StorageBackend,
) -> None:
import discord
from discord.ext import commands
self.config = config
self.broker = broker
self.storage = storage
self.router = ChannelRouter(
broker,
storage,
auto_approve=config.auto_approve,
auto_approve_tools=list(config.auto_approve_tools),
)
self._subscribed_ws: set[str] = set()
self._streaming: dict[str, StreamingMessage] = {}
intents = discord.Intents.default()
intents.message_content = True
self._bot: commands.Bot = commands.Bot(
command_prefix="!ts ",
intents=intents,
help_command=None,
)
# Attach ourselves so cogs can access the TurnstoneBot instance.
self._bot.turnstone = self # type: ignore[attr-defined]
# Register lifecycle hooks.
self._bot.setup_hook = self._setup_hook # type: ignore[method-assign]
@self._bot.event
async def on_ready() -> None:
await self._on_ready()
# -- lifecycle -----------------------------------------------------------
async def _setup_hook(self) -> None:
"""Called by discord.py after login but before connecting to the gateway."""
from turnstone.channels.discord.cog import MessageCog
from turnstone.channels.discord.views import ApprovalView, PlanReviewView
await self.broker.connect()
await self.router.start()
msg_cog = MessageCog(self._bot)
await self._bot.add_cog(msg_cog._cog)
# Register persistent views so button callbacks survive restarts.
self._bot.add_view(ApprovalView(self)._view)
self._bot.add_view(PlanReviewView(self)._view)
log.info("discord.setup_hook_complete")
async def _on_ready(self) -> None:
"""Sync slash commands and recover existing routes."""
import discord
bot = self._bot
log.info("discord.ready", user=str(bot.user), guild_count=len(bot.guilds))
if self.config.guild_id:
guild = discord.Object(id=self.config.guild_id)
bot.tree.copy_global_to(guild=guild)
await bot.tree.sync(guild=guild)
log.info("discord.commands_synced", guild_id=self.config.guild_id)
else:
await bot.tree.sync()
log.info("discord.commands_synced_global")
await self._recover_routes()
async def _recover_routes(self) -> None:
"""Re-subscribe to event channels for existing discord routes.
Queries the storage backend for all channel routes of type ``discord``
and subscribes to each workstream's event channel.
"""
routes = await asyncio.to_thread(self.storage.list_channel_routes_by_type, "discord")
for route in routes:
ws_id = route["ws_id"]
channel_id = int(route["channel_id"])
channel = self._bot.get_channel(channel_id)
if channel is not None:
await self.subscribe_ws(ws_id, channel) # type: ignore[arg-type]
log.info("discord.route_recovered", ws_id=ws_id, channel_id=channel_id)
else:
log.warning(
"discord.route_recovery_channel_missing",
ws_id=ws_id,
channel_id=channel_id,
)
# -- subscription management ---------------------------------------------
async def subscribe_ws(
self,
ws_id: str,
thread: discord.abc.Messageable,
) -> None:
"""Subscribe to workstream events and dispatch them to *thread*."""
if ws_id in self._subscribed_ws:
return
channel = f"{self.broker._prefix}:events:{ws_id}"
async def _callback(raw: str) -> None:
await self._on_ws_event(ws_id, thread, raw)
await self.broker.subscribe(channel, _callback)
self._subscribed_ws.add(ws_id)
log.info("discord.subscribed", ws_id=ws_id)
async def unsubscribe_ws(self, ws_id: str) -> None:
"""Cancel the subscription for *ws_id* and clean up streaming state."""
channel = f"{self.broker._prefix}:events:{ws_id}"
await self.broker.unsubscribe(channel)
self._subscribed_ws.discard(ws_id)
self._streaming.pop(ws_id, None)
log.info("discord.unsubscribed", ws_id=ws_id)
# -- event dispatch ------------------------------------------------------
async def _on_ws_event(
self,
ws_id: str,
thread: discord.abc.Messageable,
raw: str,
) -> None:
"""Handle an outbound event for a subscribed workstream."""
import discord
from turnstone.channels._formatter import format_approval_request, format_plan_review
from turnstone.channels.discord.views import ApprovalView, PlanReviewView
event = OutboundEvent.from_json(raw)
if isinstance(event, ContentEvent):
sm = self._streaming.get(ws_id)
if sm is None:
sm = StreamingMessage(
channel=thread,
max_length=self.config.max_message_length,
edit_interval=self.config.streaming_edit_interval,
)
self._streaming[ws_id] = sm
await sm.append(event.text)
elif isinstance(event, ApprovalRequestEvent):
if self.config.auto_approve or self._should_auto_approve(event):
await self.router.send_approval(ws_id, event.correlation_id, approved=True)
await thread.send("*Tool auto-approved.*")
else:
text = format_approval_request(event.items)
embed = discord.Embed(
title="Tool Approval Required",
description=text,
color=discord.Color.orange(),
)
embed.set_footer(text=f"{ws_id}|{event.correlation_id}")
await thread.send(embed=embed, view=ApprovalView(self)._view)
elif isinstance(event, PlanReviewEvent):
text = format_plan_review(event.content)
embed = discord.Embed(
title="Plan Review",
description=text,
color=discord.Color.blue(),
)
embed.set_footer(text=f"{ws_id}|{event.correlation_id}")
await thread.send(embed=embed, view=PlanReviewView(self)._view)
elif isinstance(event, TurnCompleteEvent):
sm = self._streaming.pop(ws_id, None)
if sm is not None:
await sm.finalize()
elif isinstance(event, SessionResumedEvent):
name = event.name or "previous session"
count = event.message_count
await thread.send(f"*Session resumed: {name} ({count} messages restored)*")
elif isinstance(event, ErrorEvent):
safe_msg = event.message[:500] if event.message else "An error occurred"
await thread.send(f"**Error:** {safe_msg}")
# -- helpers -------------------------------------------------------------
def _should_auto_approve(self, event: ApprovalRequestEvent) -> bool:
"""Return True if all tools in *event.items* are in the auto-approve list."""
allowed = self.config.auto_approve_tools
if not allowed or not event.items:
return False
for item in event.items:
# Support both server SSE format (func_name) and OpenAI format (function.name).
name = (
item.get("func_name")
or item.get("approval_label")
or item.get("function", {}).get("name", "")
)
if name not in allowed:
return False
return True
def _is_allowed_channel(self, channel_id: int) -> bool:
"""Return True if *channel_id* is in the allowed list (or list is empty)."""
if not self.config.allowed_channels:
return True
return channel_id in self.config.allowed_channels
def run(self, **kwargs: object) -> None:
"""Start the bot (blocking). Pass-through to ``commands.Bot.run``."""
self._bot.run(self.config.bot_token, log_handler=None, **kwargs) # type: ignore[arg-type]
async def start(self) -> None:
"""Start the bot (async). Use this for multi-adapter ``asyncio.gather``."""
await self._bot.start(self.config.bot_token, reconnect=True)
async def stop(self) -> None:
"""Disconnect the bot and clean up subscriptions."""
for ws_id in list(self._subscribed_ws):
await self.unsubscribe_ws(ws_id)
await self.router.stop()
await self.broker.close()
await self._bot.close()
+399
View File
@@ -0,0 +1,399 @@
"""Message handling cog for the Discord channel adapter.
Handles ``on_message`` events and slash commands (``/link``, ``/unlink``,
``/ask``, ``/status``, ``/close``).
"""
from __future__ import annotations
import asyncio
from typing import TYPE_CHECKING
from turnstone.core.log import get_logger
from turnstone.mq.protocol import CloseWorkstreamMessage
if TYPE_CHECKING:
import discord
from discord.ext import commands
from turnstone.channels.discord.bot import TurnstoneBot
log = get_logger(__name__)
_THREAD_NAME_MAX = 100
class MessageCog:
"""Cog that processes messages and registers slash commands.
Accessed via ``bot.turnstone`` to reach the :class:`TurnstoneBot` wrapper.
"""
def __init__(self, bot: commands.Bot) -> None:
import discord
from discord import app_commands
from discord.ext import commands as _commands
self.bot = bot
self.ts: TurnstoneBot = bot.turnstone # type: ignore[attr-defined]
# -- Cog wiring (manual since we can't use decorators with guarded imports) --
# We build the cog dynamically so discord.py's import is fully deferred.
cog_self = self
class _Cog(_commands.Cog, name="Turnstone"): # type: ignore[call-arg]
"""Turnstone Discord integration."""
@_commands.Cog.listener()
async def on_message(self_cog: _Cog, message: discord.Message) -> None: # noqa: N805
await cog_self._on_message(message)
@app_commands.command(name="link", description="Link your Discord account to Turnstone")
async def link(self_cog: _Cog, interaction: discord.Interaction) -> None: # noqa: N805
await interaction.response.send_modal(cog_self._link_modal_cls(cog_self))
@app_commands.command(
name="unlink", description="Unlink your Discord account from Turnstone"
)
async def unlink(self_cog: _Cog, interaction: discord.Interaction) -> None: # noqa: N805
await cog_self._cmd_unlink(interaction)
@app_commands.command(name="ask", description="Start a new Turnstone workstream")
@app_commands.describe(message="Your message to the assistant")
async def ask(self_cog: _Cog, interaction: discord.Interaction, message: str) -> None: # noqa: N805
await cog_self._cmd_ask(interaction, message)
@app_commands.command(name="status", description="Show workstream status")
async def status(self_cog: _Cog, interaction: discord.Interaction) -> None: # noqa: N805
await cog_self._cmd_status(interaction)
@app_commands.command(name="close", description="Close the current workstream")
async def close(self_cog: _Cog, interaction: discord.Interaction) -> None: # noqa: N805
await cog_self._cmd_close(interaction)
self._cog = _Cog()
# -- Modal for /link (avoids token appearing in slash command audit logs) --
class _LinkTokenModal(discord.ui.Modal, title="Link Account"): # type: ignore[call-arg]
token: discord.ui.TextInput[_LinkTokenModal] = discord.ui.TextInput(
label="API Token",
style=discord.TextStyle.short,
placeholder="Paste your ts_... API token",
required=True,
)
def __init__(modal_self, cog: MessageCog) -> None: # noqa: N805
super().__init__()
modal_self._cog = cog
async def on_submit(modal_self, interaction: discord.Interaction) -> None: # noqa: N805
await modal_self._cog._cmd_link(interaction, str(modal_self.token))
self._link_modal_cls = _LinkTokenModal
# -- on_message ----------------------------------------------------------
async def _on_message(self, message: discord.Message) -> None:
"""Route incoming messages to existing workstream threads."""
import discord
# Ignore self and other bots.
if message.author == self.bot.user or message.author.bot:
return
# Ignore DMs.
if message.guild is None:
return
channel = message.channel
# --- Message in a Discord Thread ---
if isinstance(channel, discord.Thread):
parent_id = channel.parent_id or 0
if not self.ts._is_allowed_channel(parent_id):
return
# Check if this thread has an existing route.
route = await asyncio.to_thread(
self.ts.storage.get_channel_route, "discord", str(channel.id)
)
if route is None:
# Not our thread — ignore.
return
# Resolve user.
user_id = await self.ts.router.resolve_user("discord", str(message.author.id))
if user_id is None:
return
# Use get_or_create_workstream so stale routes (evicted ws) are
# auto-refreshed with a new workstream.
try:
ws_id, is_new = await self.ts.router.get_or_create_workstream(
"discord",
str(channel.id),
name=channel.name or "",
initial_message="",
)
except (TimeoutError, RuntimeError):
log.warning("discord.ws_reactivation_failed", thread_id=channel.id)
return
if is_new:
await channel.send("*Workstream reactivated.*")
# Ensure subscription is active (handles bot restart recovery).
if ws_id not in self.ts._subscribed_ws:
await self.ts.subscribe_ws(ws_id, channel)
await self.ts.router.send_message(ws_id, message.content)
log.debug(
"discord.message_routed",
ws_id=ws_id,
thread_id=channel.id,
author=str(message.author),
)
return
# --- @mention in a non-thread channel ---
if self.bot.user is not None and self.bot.user.mentioned_in(message):
if not self.ts._is_allowed_channel(channel.id):
return
user_id = await self.ts.router.resolve_user("discord", str(message.author.id))
if user_id is None:
return
# Strip the mention from the message text.
content = message.content
if self.bot.user is not None:
content = content.replace(f"<@{self.bot.user.id}>", "").strip()
content = content.replace(f"<@!{self.bot.user.id}>", "").strip()
if not content:
content = "Hello"
# Create a thread from the message.
thread_name = content[:_THREAD_NAME_MAX] if len(content) > _THREAD_NAME_MAX else content
thread = await message.create_thread(
name=thread_name,
auto_archive_duration=self.ts.config.thread_auto_archive, # type: ignore[arg-type]
)
# Create workstream with the initial message.
ws_id, _is_new = await self.ts.router.get_or_create_workstream(
channel_type="discord",
channel_id=str(thread.id),
name=thread_name,
model=self.ts.config.model,
initial_message=content,
)
await self.ts.subscribe_ws(ws_id, thread)
log.info(
"discord.workstream_created",
ws_id=ws_id,
thread_id=thread.id,
author=str(message.author),
)
# -- slash commands ------------------------------------------------------
async def _cmd_link(self, interaction: discord.Interaction, token: str) -> None:
"""Link a Discord user to a turnstone account via API token."""
from turnstone.core.auth import hash_token
# Check if already linked.
existing = await asyncio.to_thread(
self.ts.storage.get_channel_user, "discord", str(interaction.user.id)
)
if existing:
await interaction.response.send_message(
"Your Discord account is already linked. Use `/unlink` first.",
ephemeral=True,
)
return
token_hash = hash_token(token)
token_record = await asyncio.to_thread(self.ts.storage.get_api_token_by_hash, token_hash)
if token_record is None:
await interaction.response.send_message(
"Invalid token. Please provide a valid Turnstone API token.",
ephemeral=True,
)
return
user_id = token_record.get("user_id", "")
if not user_id:
await interaction.response.send_message(
"Token has no associated user.",
ephemeral=True,
)
return
await asyncio.to_thread(
self.ts.storage.create_channel_user,
"discord",
str(interaction.user.id),
user_id,
)
await interaction.response.send_message("Account linked!", ephemeral=True)
log.info(
"discord.user_linked",
discord_user=str(interaction.user),
user_id=user_id,
)
async def _cmd_unlink(self, interaction: discord.Interaction) -> None:
"""Remove the channel user mapping for the calling Discord user."""
deleted = await asyncio.to_thread(
self.ts.storage.delete_channel_user,
"discord",
str(interaction.user.id),
)
if deleted:
await interaction.response.send_message("Account unlinked.", ephemeral=True)
log.info("discord.user_unlinked", discord_user=str(interaction.user))
else:
await interaction.response.send_message(
"No linked account found.",
ephemeral=True,
)
async def _cmd_ask(self, interaction: discord.Interaction, message: str) -> None:
"""Create a new thread and workstream with an initial message."""
import discord
user_id = await self.ts.router.resolve_user("discord", str(interaction.user.id))
if user_id is None:
await interaction.response.send_message(
"Your Discord account is not linked. Use `/link` first.",
ephemeral=True,
)
return
await interaction.response.defer()
channel = interaction.channel
if channel is None:
await interaction.followup.send("Cannot determine channel.", ephemeral=True)
return
thread_name = message[:_THREAD_NAME_MAX] if len(message) > _THREAD_NAME_MAX else message
# Create a standalone thread in the current channel.
if isinstance(channel, discord.TextChannel):
thread = await channel.create_thread(
name=thread_name,
auto_archive_duration=self.ts.config.thread_auto_archive, # type: ignore[arg-type]
type=discord.ChannelType.public_thread,
)
else:
await interaction.followup.send(
"Cannot create a thread in this channel type.",
ephemeral=True,
)
return
ws_id, _is_new = await self.ts.router.get_or_create_workstream(
channel_type="discord",
channel_id=str(thread.id),
name=thread_name,
model=self.ts.config.model,
initial_message=message,
)
await self.ts.subscribe_ws(ws_id, thread)
await interaction.followup.send(
f"Workstream started in {thread.mention}",
ephemeral=True,
)
log.info(
"discord.ask_workstream_created",
ws_id=ws_id,
thread_id=thread.id,
author=str(interaction.user),
)
async def _cmd_status(self, interaction: discord.Interaction) -> None:
"""Show workstream status for the current thread."""
import discord
channel = interaction.channel
if not isinstance(channel, discord.Thread):
await interaction.response.send_message(
"This command can only be used inside a thread.",
ephemeral=True,
)
return
route = await asyncio.to_thread(
self.ts.storage.get_channel_route, "discord", str(channel.id)
)
if route is None:
await interaction.response.send_message(
"No workstream is associated with this thread.",
ephemeral=True,
)
return
ws_id = route["ws_id"]
node_id = route.get("node_id", "")
created = route.get("created", "")
embed = discord.Embed(
title="Workstream Status",
color=discord.Color.green(),
)
embed.add_field(name="Workstream ID", value=ws_id, inline=False)
if node_id:
embed.add_field(name="Node", value=node_id, inline=True)
if created:
embed.add_field(name="Created", value=created, inline=True)
await interaction.response.send_message(embed=embed, ephemeral=True)
async def _cmd_close(self, interaction: discord.Interaction) -> None:
"""Close the workstream and archive the thread."""
import discord
channel = interaction.channel
if not isinstance(channel, discord.Thread):
await interaction.response.send_message(
"This command can only be used inside a thread.",
ephemeral=True,
)
return
route = await asyncio.to_thread(
self.ts.storage.get_channel_route, "discord", str(channel.id)
)
if route is None:
await interaction.response.send_message(
"No workstream is associated with this thread.",
ephemeral=True,
)
return
ws_id = route["ws_id"]
# Close via MQ.
msg = CloseWorkstreamMessage(ws_id=ws_id)
await self.ts.broker.push_inbound(msg.to_json())
# Delete route and unsubscribe.
await self.ts.router.delete_route("discord", str(channel.id))
await self.ts.unsubscribe_ws(ws_id)
await interaction.response.send_message("Workstream closed.")
log.info("discord.workstream_closed", ws_id=ws_id, thread_id=channel.id)
# Archive the thread.
try:
await channel.edit(archived=True)
except discord.Forbidden:
log.warning("discord.archive_forbidden", thread_id=channel.id)
+23
View File
@@ -0,0 +1,23 @@
"""Discord-specific configuration."""
from __future__ import annotations
from dataclasses import dataclass, field
from turnstone.channels._config import ChannelConfig
@dataclass
class DiscordConfig(ChannelConfig):
"""Configuration for the Discord channel adapter.
Extends :class:`ChannelConfig` with Discord-specific settings such as
the bot token, guild restriction, and streaming parameters.
"""
bot_token: str = ""
guild_id: int = 0 # 0 = all guilds
allowed_channels: list[int] = field(default_factory=list) # empty = all
thread_auto_archive: int = 1440 # minutes (24h)
max_message_length: int = 2000
streaming_edit_interval: float = 1.5 # seconds between message edits
+320
View File
@@ -0,0 +1,320 @@
"""Persistent interactive views for Discord approval and plan review.
These views use static ``custom_id`` values so they survive bot restarts.
Correlation information (``ws_id`` and ``correlation_id``) is stored in the
embed footer of the message the view is attached to.
"""
from __future__ import annotations
from typing import TYPE_CHECKING
from turnstone.core.log import get_logger
if TYPE_CHECKING:
import discord
from turnstone.channels.discord.bot import TurnstoneBot
log = get_logger(__name__)
def _parse_footer(interaction: discord.Interaction) -> tuple[str, str] | None:
"""Extract ``(ws_id, correlation_id)`` from the first embed's footer."""
if not interaction.message or not interaction.message.embeds:
return None
footer = interaction.message.embeds[0].footer.text
if not footer or "|" not in footer:
return None
parts = footer.split("|", 1)
return parts[0], parts[1]
async def _disable_buttons(interaction: discord.Interaction, label: str) -> None:
"""Edit the message to disable all buttons and append a result label."""
import discord
if interaction.message is None:
return
view = discord.ui.View()
for item in interaction.message.components or []:
for child in item.children: # type: ignore[union-attr]
button: discord.ui.Button[discord.ui.View] = discord.ui.Button(
label=getattr(child, "label", ""),
style=discord.ButtonStyle.secondary,
disabled=True,
custom_id=getattr(child, "custom_id", None),
)
view.add_item(button)
embed = interaction.message.embeds[0] if interaction.message.embeds else None
if embed is not None:
embed.color = discord.Color.greyple()
embed.title = f"{embed.title} - {label}"
await interaction.message.edit(embed=embed, view=view)
# ---------------------------------------------------------------------------
# ApprovalView
# ---------------------------------------------------------------------------
class ApprovalView:
"""Persistent view with Approve / Reject / Always Approve buttons."""
def __init__(self, bot: TurnstoneBot) -> None:
import discord
self.bot = bot
view_self = self
class _View(discord.ui.View):
def __init__(inner_self) -> None: # noqa: N805
super().__init__(timeout=None)
@discord.ui.button(
label="Approve",
style=discord.ButtonStyle.green,
custom_id="ts:approve",
)
async def approve(
inner_self, # noqa: N805
interaction: discord.Interaction,
button: discord.ui.Button[_View],
) -> None:
await view_self._handle(interaction, approved=True, always=False)
@discord.ui.button(
label="Reject",
style=discord.ButtonStyle.red,
custom_id="ts:reject",
)
async def reject(
inner_self, # noqa: N805
interaction: discord.Interaction,
button: discord.ui.Button[_View],
) -> None:
await view_self._handle(interaction, approved=False, always=False)
@discord.ui.button(
label="Always Approve",
style=discord.ButtonStyle.secondary,
custom_id="ts:always",
)
async def always_approve(
inner_self, # noqa: N805
interaction: discord.Interaction,
button: discord.ui.Button[_View],
) -> None:
await view_self._handle(interaction, approved=True, always=True)
self._view = _View()
async def _handle(
self,
interaction: discord.Interaction,
*,
approved: bool,
always: bool,
) -> None:
"""Process an approval button click."""
parsed = _parse_footer(interaction)
if parsed is None:
await interaction.response.send_message(
"Could not determine workstream context.",
ephemeral=True,
)
return
ws_id, correlation_id = parsed
# Verify user is linked. Scope enforcement (approve) happens
# server-side when the bridge executes the tool approval.
user_id = await self.bot.router.resolve_user("discord", str(interaction.user.id))
if user_id is None:
await interaction.response.send_message(
"Your Discord account is not linked. Use `/link` first.",
ephemeral=True,
)
return
# Defer before doing async work so Discord doesn't time out.
await interaction.response.defer(ephemeral=True)
await self.bot.router.send_approval(
ws_id=ws_id,
correlation_id=correlation_id,
approved=approved,
always=always,
)
label = "Always Approved" if always else ("Approved" if approved else "Rejected")
await _disable_buttons(interaction, label)
await interaction.followup.send(
f"Tool execution **{label.lower()}**.",
ephemeral=True,
)
log.info(
"discord.approval_response",
ws_id=ws_id,
correlation_id=correlation_id,
approved=approved,
always=always,
)
# ---------------------------------------------------------------------------
# PlanReviewView
# ---------------------------------------------------------------------------
class PlanReviewView:
"""Persistent view with Approve Plan / Request Changes buttons."""
def __init__(self, bot: TurnstoneBot) -> None:
import discord
self.bot = bot
view_self = self
class _FeedbackModal(discord.ui.Modal, title="Request Changes"): # type: ignore[call-arg]
feedback: discord.ui.TextInput[_FeedbackModal] = discord.ui.TextInput(
label="Feedback",
style=discord.TextStyle.paragraph,
placeholder="Describe the changes you'd like...",
required=True,
max_length=2000,
)
def __init__(modal_self, ws_id: str, correlation_id: str) -> None: # noqa: N805
super().__init__()
modal_self.ws_id = ws_id
modal_self.correlation_id = correlation_id
async def on_submit(modal_self, interaction: discord.Interaction) -> None: # noqa: N805
await view_self._send_feedback(
interaction,
modal_self.ws_id,
modal_self.correlation_id,
str(modal_self.feedback),
)
self._modal_cls = _FeedbackModal
class _View(discord.ui.View):
def __init__(inner_self) -> None: # noqa: N805
super().__init__(timeout=None)
@discord.ui.button(
label="Approve Plan",
style=discord.ButtonStyle.green,
custom_id="ts:plan_approve",
)
async def approve_plan(
inner_self, # noqa: N805
interaction: discord.Interaction,
button: discord.ui.Button[_View],
) -> None:
await view_self._handle_approve(interaction)
@discord.ui.button(
label="Request Changes",
style=discord.ButtonStyle.secondary,
custom_id="ts:plan_changes",
)
async def request_changes(
inner_self, # noqa: N805
interaction: discord.Interaction,
button: discord.ui.Button[_View],
) -> None:
await view_self._handle_changes(interaction)
self._view = _View()
async def _handle_approve(self, interaction: discord.Interaction) -> None:
"""Approve the plan (empty feedback = approved)."""
parsed = _parse_footer(interaction)
if parsed is None:
await interaction.response.send_message(
"Could not determine workstream context.",
ephemeral=True,
)
return
ws_id, correlation_id = parsed
user_id = await self.bot.router.resolve_user("discord", str(interaction.user.id))
if user_id is None:
await interaction.response.send_message(
"Your Discord account is not linked. Use `/link` first.",
ephemeral=True,
)
return
# Defer before doing async work so Discord doesn't time out.
await interaction.response.defer(ephemeral=True)
await self.bot.router.send_plan_feedback(
ws_id=ws_id,
correlation_id=correlation_id,
feedback="",
)
await _disable_buttons(interaction, "Approved")
await interaction.followup.send("Plan **approved**.", ephemeral=True)
log.info(
"discord.plan_approved",
ws_id=ws_id,
correlation_id=correlation_id,
)
async def _handle_changes(self, interaction: discord.Interaction) -> None:
"""Open a modal for feedback text."""
parsed = _parse_footer(interaction)
if parsed is None:
await interaction.response.send_message(
"Could not determine workstream context.",
ephemeral=True,
)
return
ws_id, correlation_id = parsed
modal = self._modal_cls(ws_id, correlation_id)
await interaction.response.send_modal(modal)
async def _send_feedback(
self,
interaction: discord.Interaction,
ws_id: str,
correlation_id: str,
feedback: str,
) -> None:
"""Send plan feedback after the modal is submitted."""
user_id = await self.bot.router.resolve_user("discord", str(interaction.user.id))
if user_id is None:
await interaction.response.send_message(
"Your Discord account is not linked. Use `/link` first.",
ephemeral=True,
)
return
# Defer before doing async work so Discord doesn't time out.
await interaction.response.defer(ephemeral=True)
await self.bot.router.send_plan_feedback(
ws_id=ws_id,
correlation_id=correlation_id,
feedback=feedback,
)
await interaction.followup.send(
"Feedback submitted. The plan will be revised.",
ephemeral=True,
)
log.info(
"discord.plan_changes_requested",
ws_id=ws_id,
correlation_id=correlation_id,
)
+121 -2
View File
@@ -985,6 +985,79 @@ async def admin_revoke_token(request: Request) -> JSONResponse:
return JSONResponse({"error": "Token not found"}, status_code=404)
# ---------------------------------------------------------------------------
# Admin: Channel user mapping
# ---------------------------------------------------------------------------
async def admin_list_channels(request: Request) -> JSONResponse:
"""GET /v1/api/admin/users/{user_id}/channels — list channel links for a user."""
storage = getattr(request.app.state, "auth_storage", None)
if storage is None:
return JSONResponse({"error": "Storage not available"}, status_code=503)
user_id = request.path_params["user_id"]
channels = storage.list_channel_users_by_user(user_id)
return JSONResponse({"channels": channels})
async def admin_create_channel(request: Request) -> JSONResponse:
"""POST /v1/api/admin/users/{user_id}/channels — link a channel account."""
storage = getattr(request.app.state, "auth_storage", None)
if storage is None:
return JSONResponse({"error": "Storage not available"}, status_code=503)
user_id = request.path_params["user_id"]
try:
body: dict[str, Any] = await request.json()
except (ValueError, json.JSONDecodeError):
return JSONResponse({"error": "Invalid JSON body"}, status_code=400)
channel_type = body.get("channel_type", "").strip().lower()
channel_user_id = body.get("channel_user_id", "").strip()
if not channel_type:
return JSONResponse({"error": "channel_type is required"}, status_code=400)
if not channel_user_id:
return JSONResponse({"error": "channel_user_id is required"}, status_code=400)
if len(channel_type) > 64 or len(channel_user_id) > 256:
return JSONResponse({"error": "Value too long"}, status_code=400)
# Verify user exists
if storage.get_user(user_id) is None:
return JSONResponse({"error": "User not found"}, status_code=404)
# Check for existing mapping
existing = storage.get_channel_user(channel_type, channel_user_id)
if existing is not None:
return JSONResponse(
{"error": f"Channel user already linked to user {existing['user_id']}"},
status_code=409,
)
storage.create_channel_user(channel_type, channel_user_id, user_id)
result = storage.get_channel_user(channel_type, channel_user_id)
if result is None:
return JSONResponse({"error": "Failed to create channel mapping"}, status_code=500)
# Guard against race: another request may have claimed this channel_user_id.
if result.get("user_id") != user_id:
return JSONResponse(
{"error": f"Channel user already linked to user {result['user_id']}"},
status_code=409,
)
return JSONResponse(result)
async def admin_delete_channel(request: Request) -> JSONResponse:
"""DELETE /v1/api/admin/channels/{channel_type}/{channel_user_id} — unlink."""
storage = getattr(request.app.state, "auth_storage", None)
if storage is None:
return JSONResponse({"error": "Storage not available"}, status_code=503)
channel_type = request.path_params["channel_type"]
channel_user_id = request.path_params["channel_user_id"]
if storage.delete_channel_user(channel_type, channel_user_id):
return JSONResponse({"status": "ok"})
return JSONResponse({"error": "Channel link not found"}, status_code=404)
# ---------------------------------------------------------------------------
# App factory
# ---------------------------------------------------------------------------
@@ -1028,6 +1101,20 @@ def create_app(
"/api/admin/users/{user_id}/tokens", admin_create_token, methods=["POST"]
),
Route("/api/admin/tokens/{token_id}", admin_revoke_token, methods=["DELETE"]),
Route(
"/api/admin/users/{user_id}/channels",
admin_list_channels,
),
Route(
"/api/admin/users/{user_id}/channels",
admin_create_channel,
methods=["POST"],
),
Route(
"/api/admin/channels/{channel_type}/{channel_user_id}",
admin_delete_channel,
methods=["DELETE"],
),
],
),
Route("/health", health),
@@ -1156,10 +1243,27 @@ def main() -> None:
password=args.redis_password,
)
# If no explicit auth token is provided, mint a service JWT using the
# shared secret so the collector can poll server nodes.
collector_token = args.auth_token
if not collector_token:
_jwt_secret = os.environ.get("TURNSTONE_JWT_SECRET", "")
if _jwt_secret:
from turnstone.core.auth import create_jwt
collector_token = create_jwt(
user_id="console-collector",
scopes=frozenset({"read"}),
source="console",
secret=_jwt_secret,
expiry_hours=168, # 1 week — console restarts refresh
)
log.info("console.collector_jwt_minted")
collector = ClusterCollector(
broker=broker,
poll_interval=args.poll_interval,
auth_token=args.auth_token,
auth_token=collector_token,
)
collector.start()
@@ -1182,13 +1286,28 @@ def main() -> None:
except Exception:
log.info("Console storage not available — admin API disabled, JWT-only auth")
# If no explicit auth token is provided, mint a service JWT using the
# shared secret so the console can proxy requests to the server.
proxy_token = args.auth_token
if not proxy_token and jwt_secret:
from turnstone.core.auth import create_jwt
proxy_token = create_jwt(
user_id="console-proxy",
scopes=frozenset({"write"}),
source="console",
secret=jwt_secret,
expiry_hours=168, # 1 week — console restarts refresh
)
log.info("console.jwt_minted")
app = create_app(
collector=collector,
broker=broker,
auth_config=auth_config,
jwt_secret=jwt_secret,
auth_storage=auth_storage,
proxy_auth_token=args.auth_token,
proxy_auth_token=proxy_token,
)
log.info("Console starting on http://%s:%s", args.host, args.port)
+297 -21
View File
@@ -3,10 +3,15 @@
var _adminTab = "users";
var _adminUsers = [];
var _adminTokenUserId = "";
var _adminChannelUserId = "";
var _lastCreatedToken = "";
var _cuTrapHandler = null;
var _ctTrapHandler = null;
var _tcTrapHandler = null;
var _ccTrapHandler = null;
var _cfTrapHandler = null;
var _confirmCallbackFn = null;
var _confirmTriggerEl = null;
// ---------------------------------------------------------------------------
// View switching (called from app.js showOverview/drillDown pattern)
@@ -39,9 +44,12 @@ function switchAdminTab(tab) {
tab === "users" ? "" : "none";
document.getElementById("admin-tokens").style.display =
tab === "tokens" ? "" : "none";
document.getElementById("admin-channels").style.display =
tab === "channels" ? "" : "none";
if (tab === "users") loadAdminUsers();
if (tab === "tokens") _populateTokenUserSelect();
if (tab === "channels") _populateChannelUserSelect();
}
// ---------------------------------------------------------------------------
@@ -109,18 +117,26 @@ function _renderUsers(users) {
}
function confirmDeleteUser(userId, username) {
if (!confirm("Delete user '" + username + "' and all their tokens?")) return;
authFetch("/v1/api/admin/users/" + encodeURIComponent(userId), {
method: "DELETE",
})
.then(function (r) {
if (!r.ok) throw new Error("Delete failed");
showToast("User '" + username + "' deleted");
loadAdminUsers();
})
.catch(function () {
showToast("Failed to delete user");
});
showConfirmModal(
"Delete User",
"Delete user \u2018" +
username +
"\u2019 and all their tokens and channel links? This cannot be undone.",
"Delete",
function () {
authFetch("/v1/api/admin/users/" + encodeURIComponent(userId), {
method: "DELETE",
})
.then(function (r) {
if (!r.ok) throw new Error("Delete failed");
showToast("User '" + username + "' deleted");
loadAdminUsers();
})
.catch(function () {
showToast("Failed to delete user");
});
},
);
}
// ---------------------------------------------------------------------------
@@ -224,17 +240,213 @@ function _renderScopeBadges(scopes) {
}
function confirmRevokeToken(tokenId) {
if (!confirm("Revoke this token? This cannot be undone.")) return;
authFetch("/v1/api/admin/tokens/" + encodeURIComponent(tokenId), {
method: "DELETE",
})
showConfirmModal(
"Revoke Token",
"Revoke this API token? Existing JWTs issued from it will remain valid until they expire (max 24h). This cannot be undone.",
"Revoke",
function () {
authFetch("/v1/api/admin/tokens/" + encodeURIComponent(tokenId), {
method: "DELETE",
})
.then(function (r) {
if (!r.ok) throw new Error("Revoke failed");
showToast("Token revoked");
loadAdminTokens();
})
.catch(function () {
showToast("Failed to revoke token");
});
},
);
}
// ---------------------------------------------------------------------------
// Channels
// ---------------------------------------------------------------------------
function _populateChannelUserSelect() {
var sel = document.getElementById("admin-channel-user");
var current = sel.value;
sel.innerHTML = '<option value="">Select user...</option>';
for (var i = 0; i < _adminUsers.length; i++) {
var u = _adminUsers[i];
var opt = document.createElement("option");
opt.value = u.user_id;
opt.textContent = u.username + " (" + u.display_name + ")";
sel.appendChild(opt);
}
if (current) sel.value = current;
}
function loadAdminChannels() {
var userId = document.getElementById("admin-channel-user").value;
_adminChannelUserId = userId;
if (!userId) {
document.getElementById("admin-channels-table").innerHTML =
'<div class="dashboard-empty">Select a user to view channel links</div>';
return;
}
document.getElementById("admin-channels-table").innerHTML =
'<div class="dashboard-empty">Loading channel links...</div>';
authFetch("/v1/api/admin/users/" + encodeURIComponent(userId) + "/channels")
.then(function (r) {
if (!r.ok) throw new Error("Revoke failed");
showToast("Token revoked");
loadAdminTokens();
if (!r.ok) throw new Error("Failed to load channels");
return r.json();
})
.then(function (data) {
_renderChannels(data.channels || []);
})
.catch(function () {
showToast("Failed to revoke token");
document.getElementById("admin-channels-table").innerHTML =
'<div class="dashboard-empty">Failed to load channel links</div>';
});
}
function _renderChannels(channels) {
var container = document.getElementById("admin-channels-table");
if (!channels.length) {
container.innerHTML =
'<div class="dashboard-empty">No channel links for this user</div>';
return;
}
var html = "";
for (var i = 0; i < channels.length; i++) {
var c = channels[i];
html +=
'<div class="admin-row" role="listitem">' +
'<span class="admin-col admin-col-chtype"><span class="scope-badge scope-channel">' +
escapeHtml(c.channel_type) +
"</span></span>" +
'<span class="admin-col admin-col-chuid"><code>' +
escapeHtml(c.channel_user_id) +
"</code></span>" +
'<span class="admin-col admin-col-created">' +
escapeHtml(c.created || "").slice(0, 10) +
"</span>" +
'<span class="admin-col admin-col-actions">' +
'<button class="admin-btn-danger" data-unlink-type="' +
escapeHtml(c.channel_type) +
'" data-unlink-uid="' +
escapeHtml(c.channel_user_id) +
'" title="Unlink channel account">unlink</button>' +
"</span>" +
"</div>";
}
container.innerHTML = html;
var btns = container.querySelectorAll("[data-unlink-type]");
for (var j = 0; j < btns.length; j++) {
btns[j].addEventListener("click", function () {
confirmUnlinkChannel(
this.getAttribute("data-unlink-type"),
this.getAttribute("data-unlink-uid"),
);
});
}
}
function confirmUnlinkChannel(channelType, channelUserId) {
showConfirmModal(
"Unlink Channel",
"Unlink " +
channelType +
" account \u2018" +
channelUserId +
"\u2019? The user will need to re-link via /link to interact with the bot.",
"Unlink",
function () {
authFetch(
"/v1/api/admin/channels/" +
encodeURIComponent(channelType) +
"/" +
encodeURIComponent(channelUserId),
{ method: "DELETE" },
)
.then(function (r) {
if (!r.ok) throw new Error("Unlink failed");
showToast("Channel account unlinked");
loadAdminChannels();
})
.catch(function () {
showToast("Failed to unlink channel account");
});
},
);
}
// ---------------------------------------------------------------------------
// Create Channel Link Modal
// ---------------------------------------------------------------------------
function showCreateChannelModal() {
if (!_adminChannelUserId) {
showToast("Select a user first");
return;
}
var overlay = document.getElementById("create-channel-overlay");
overlay.style.display = "flex";
document.getElementById("create-channel-error").style.display = "none";
document.getElementById("cc-type").value = "discord";
document.getElementById("cc-uid").value = "";
document.getElementById("cc-submit").disabled = false;
document.getElementById("cc-submit").textContent = "Link";
_ccTrapHandler = _installTrap("create-channel-overlay", "create-channel-box");
setTimeout(function () {
document.getElementById("cc-uid").focus();
}, 50);
}
function hideCreateChannelModal() {
document.getElementById("create-channel-overlay").style.display = "none";
_ccTrapHandler = _removeTrap(_ccTrapHandler);
var trigger = document.querySelector("#admin-channels .admin-action-btn");
if (trigger) trigger.focus();
}
function submitCreateChannel() {
var channelType = document.getElementById("cc-type").value;
var channelUserId = (document.getElementById("cc-uid").value || "").trim();
var errEl = document.getElementById("create-channel-error");
if (!channelUserId)
return _showModalError(errEl, "External user ID is required");
var btn = document.getElementById("cc-submit");
btn.disabled = true;
btn.textContent = "Linking\u2026";
authFetch(
"/v1/api/admin/users/" +
encodeURIComponent(_adminChannelUserId) +
"/channels",
{
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({
channel_type: channelType,
channel_user_id: channelUserId,
}),
},
)
.then(function (r) {
if (r.status === 409)
return r.json().then(function (d) {
throw new Error(d.error || "Already linked");
});
if (!r.ok)
return r.json().then(function (d) {
throw new Error(d.error || "Failed");
});
return r.json();
})
.then(function () {
hideCreateChannelModal();
showToast("Channel account linked");
loadAdminChannels();
})
.catch(function (err) {
btn.disabled = false;
btn.textContent = "Link";
_showModalError(errEl, err.message || "Failed to link channel account");
});
}
@@ -261,6 +473,8 @@ function showCreateUserModal() {
function hideCreateUserModal() {
document.getElementById("create-user-overlay").style.display = "none";
_cuTrapHandler = _removeTrap(_cuTrapHandler);
var trigger = document.querySelector("#admin-users .admin-action-btn");
if (trigger) trigger.focus();
}
function submitCreateUser() {
@@ -339,6 +553,8 @@ function showCreateTokenModal() {
function hideCreateTokenModal() {
document.getElementById("create-token-overlay").style.display = "none";
_ctTrapHandler = _removeTrap(_ctTrapHandler);
var trigger = document.querySelector("#admin-tokens .admin-action-btn");
if (trigger) trigger.focus();
}
function submitCreateToken() {
@@ -396,6 +612,8 @@ function hideTokenCreatedModal() {
document.getElementById("token-created-overlay").style.display = "none";
_tcTrapHandler = _removeTrap(_tcTrapHandler);
_lastCreatedToken = "";
var trigger = document.querySelector("#admin-tokens .admin-action-btn");
if (trigger) trigger.focus();
}
function copyCreatedToken() {
@@ -458,6 +676,9 @@ function _installTrap(overlayId, boxId, trapRef) {
if (overlayId === "create-user-overlay") hideCreateUserModal();
else if (overlayId === "create-token-overlay") hideCreateTokenModal();
else if (overlayId === "token-created-overlay") hideTokenCreatedModal();
else if (overlayId === "create-channel-overlay")
hideCreateChannelModal();
else if (overlayId === "confirm-overlay") hideConfirmModal();
}
};
}
@@ -494,6 +715,18 @@ document.addEventListener("keydown", function (e) {
hideTokenCreatedModal();
return;
}
var cc = document.getElementById("create-channel-overlay");
if (cc && cc.style.display !== "none") {
e.preventDefault();
hideCreateChannelModal();
return;
}
var cf = document.getElementById("confirm-overlay");
if (cf && cf.style.display !== "none") {
e.preventDefault();
hideConfirmModal();
return;
}
});
// Tab arrow key navigation
@@ -502,7 +735,7 @@ document.addEventListener("keydown", function (e) {
if (!tablist) return;
tablist.addEventListener("keydown", function (e) {
if (e.key !== "ArrowLeft" && e.key !== "ArrowRight") return;
var tabOrder = ["users", "tokens"];
var tabOrder = ["users", "tokens", "channels"];
var idx = tabOrder.indexOf(_adminTab);
if (e.key === "ArrowRight") idx = (idx + 1) % tabOrder.length;
else idx = (idx - 1 + tabOrder.length) % tabOrder.length;
@@ -514,6 +747,49 @@ document.addEventListener("keydown", function (e) {
});
})();
// ---------------------------------------------------------------------------
// Confirm Modal (reusable styled replacement for confirm())
// ---------------------------------------------------------------------------
function showConfirmModal(title, message, actionLabel, callback) {
_confirmCallbackFn = callback;
_confirmTriggerEl = document.activeElement;
document.getElementById("confirm-title").textContent = title;
document.getElementById("confirm-message").textContent = message;
var btn = document.getElementById("confirm-submit");
btn.textContent = actionLabel;
btn.disabled = false;
var overlay = document.getElementById("confirm-overlay");
overlay.style.display = "flex";
_cfTrapHandler = _installTrap("confirm-overlay", "confirm-box");
setTimeout(function () {
btn.focus();
}, 50);
}
function hideConfirmModal() {
document.getElementById("confirm-overlay").style.display = "none";
_cfTrapHandler = _removeTrap(_cfTrapHandler);
if (
_confirmTriggerEl &&
_confirmTriggerEl.focus &&
_confirmTriggerEl.isConnected
) {
_confirmTriggerEl.focus();
}
_confirmCallbackFn = null;
_confirmTriggerEl = null;
}
function _confirmCallback() {
var fn = _confirmCallbackFn;
_confirmCallbackFn = null;
var btn = document.getElementById("confirm-submit");
if (btn) btn.disabled = true;
if (fn) fn();
hideConfirmModal();
}
// ---------------------------------------------------------------------------
// Helpers
// ---------------------------------------------------------------------------
+56 -4
View File
@@ -77,12 +77,13 @@
<!-- ADMIN PANEL -->
<div id="view-admin" style="display:none">
<div class="admin-tabs" role="tablist">
<button class="admin-tab active" data-tab="users" role="tab" aria-selected="true" aria-controls="admin-users" tabindex="0" onclick="switchAdminTab('users')">Users</button>
<button class="admin-tab" data-tab="tokens" role="tab" aria-selected="false" aria-controls="admin-tokens" tabindex="-1" onclick="switchAdminTab('tokens')">Tokens</button>
<button id="tab-users" class="admin-tab active" data-tab="users" role="tab" aria-selected="true" aria-controls="admin-users" tabindex="0" onclick="switchAdminTab('users')">Users</button>
<button id="tab-tokens" class="admin-tab" data-tab="tokens" role="tab" aria-selected="false" aria-controls="admin-tokens" tabindex="-1" onclick="switchAdminTab('tokens')">Tokens</button>
<button id="tab-channels" class="admin-tab" data-tab="channels" role="tab" aria-selected="false" aria-controls="admin-channels" tabindex="-1" onclick="switchAdminTab('channels')">Channels</button>
</div>
<!-- Users Tab -->
<div id="admin-users" class="admin-panel" role="tabpanel">
<div id="admin-users" class="admin-panel" role="tabpanel" aria-labelledby="tab-users">
<div class="admin-toolbar">
<span class="section-header" style="margin:0">USERS</span>
<button class="admin-action-btn" onclick="showCreateUserModal()">+ Create user</button>
@@ -99,7 +100,7 @@
</div>
<!-- Tokens Tab -->
<div id="admin-tokens" class="admin-panel" role="tabpanel" style="display:none">
<div id="admin-tokens" class="admin-panel" role="tabpanel" aria-labelledby="tab-tokens" style="display:none">
<div class="admin-toolbar">
<span class="section-header" style="margin:0">TOKENS</span>
<label for="admin-token-user" class="sr-only">Filter tokens by user</label>
@@ -120,6 +121,27 @@
<div class="dashboard-empty">Select a user to view tokens</div>
</div>
</div>
<!-- Channels Tab -->
<div id="admin-channels" class="admin-panel" role="tabpanel" aria-labelledby="tab-channels" style="display:none">
<div class="admin-toolbar">
<span class="section-header" style="margin:0">CHANNEL LINKS</span>
<label for="admin-channel-user" class="sr-only">Filter channels by user</label>
<select id="admin-channel-user" onchange="loadAdminChannels()">
<option value="">Select user...</option>
</select>
<button class="admin-action-btn" onclick="showCreateChannelModal()">+ Link channel</button>
</div>
<div class="admin-colheaders" aria-hidden="true">
<span class="admin-col admin-col-chtype">CHANNEL</span>
<span class="admin-col admin-col-chuid">EXTERNAL ID</span>
<span class="admin-col admin-col-created">LINKED</span>
<span class="admin-col admin-col-actions">ACTIONS</span>
</div>
<div id="admin-channels-table" role="list" aria-label="Channel links" aria-live="polite">
<div class="dashboard-empty">Select a user to view channel links</div>
</div>
</div>
</div>
</div>
@@ -232,6 +254,36 @@ window.TURNSTONE_KB_SHORTCUTS = [
</div>
</div>
<!-- Confirm Action Modal (reusable) -->
<div id="confirm-overlay" style="display:none" role="alertdialog" aria-modal="true" aria-labelledby="confirm-title" aria-describedby="confirm-message">
<div id="confirm-box" class="admin-modal">
<h2 id="confirm-title">Confirm</h2>
<p id="confirm-message" style="color:var(--fg-dim);margin:12px 0 20px;line-height:1.5"></p>
<div class="modal-buttons">
<button class="modal-cancel" onclick="hideConfirmModal()">Cancel</button>
<button id="confirm-submit" class="modal-submit" style="background:var(--red);border-color:var(--red)" onclick="_confirmCallback()">Confirm</button>
</div>
</div>
</div>
<!-- Create Channel Link Modal -->
<div id="create-channel-overlay" style="display:none" role="dialog" aria-modal="true" aria-labelledby="create-channel-title">
<div id="create-channel-box" class="admin-modal">
<h2 id="create-channel-title">Link Channel Account</h2>
<div id="create-channel-error" role="alert" aria-live="assertive"></div>
<label for="cc-type">Channel type</label>
<select id="cc-type">
<option value="discord">Discord</option>
</select>
<label for="cc-uid">External user ID <span class="label-hint">the user's ID on the platform</span></label>
<input id="cc-uid" type="text" placeholder="e.g. 123456789012345678" autocomplete="off" spellcheck="false">
<div class="modal-buttons">
<button class="modal-cancel" onclick="hideCreateChannelModal()">Cancel</button>
<button id="cc-submit" class="modal-submit" onclick="submitCreateChannel()">Link</button>
</div>
</div>
</div>
<script src="/static/admin.js"></script>
<script src="/static/app.js"></script>
</body>
+12 -1
View File
@@ -789,6 +789,12 @@
grid-template-columns: 100px 1fr 160px 100px 100px 80px;
}
/* Channels grid: CHANNEL | EXTERNAL ID | LINKED | ACTIONS */
#admin-channels .admin-colheaders,
#admin-channels .admin-row {
grid-template-columns: 100px 1fr 100px 80px;
}
/* Scope badges */
.scope-badge {
display: inline-block;
@@ -806,6 +812,7 @@
}
.scope-write { color: var(--cyan); border-color: rgba(103, 232, 249, 0.2); }
.scope-approve { color: var(--accent); border-color: var(--accent-dim); }
.scope-channel { color: var(--magenta); border-color: rgba(192, 132, 252, 0.25); }
/* Action buttons */
.admin-btn-danger {
@@ -917,7 +924,7 @@
.modal-submit:focus-visible { outline: 2px solid var(--fg-bright); outline-offset: 2px; }
.modal-submit:disabled { opacity: 0.4; cursor: not-allowed; filter: none; }
#create-user-overlay, #create-token-overlay, #token-created-overlay {
#create-user-overlay, #create-token-overlay, #token-created-overlay, #create-channel-overlay, #confirm-overlay {
position: fixed;
inset: 0;
background: rgba(0, 0, 0, 0.7);
@@ -957,6 +964,10 @@
grid-template-columns: 80px 1fr 100px 80px;
}
.admin-col-created, .admin-col-expires { display: none; }
#admin-channels .admin-colheaders, #admin-channels .admin-row {
grid-template-columns: 80px 1fr 80px;
}
#admin-channels .admin-col-created { display: none; }
}
/* ==========================================================================
+203
View File
@@ -473,6 +473,15 @@ class PostgreSQLBackend:
).fetchall()
)
# -- Session lookup by workstream ------------------------------------------
def get_session_id_by_ws(self, ws_id: str) -> str | None:
with self._engine.connect() as conn:
row = conn.execute(
sa.select(sessions.c.session_id).where(sessions.c.ws_id == ws_id)
).fetchone()
return str(row[0]) if row else None
# -- User identity operations -----------------------------------------------
def create_user(
@@ -676,6 +685,200 @@ class PostgreSQLBackend:
conn.commit()
return result.rowcount > 0
# -- Channel user mapping ---------------------------------------------------
def create_channel_user(self, channel_type: str, channel_user_id: str, user_id: str) -> None:
from sqlalchemy.dialects import postgresql
from turnstone.core.storage._schema import channel_users
now = datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%S")
with self._engine.connect() as conn:
conn.execute(
postgresql.insert(channel_users)
.values(
channel_type=channel_type,
channel_user_id=channel_user_id,
user_id=user_id,
created=now,
)
.on_conflict_do_nothing()
)
conn.commit()
def get_channel_user(self, channel_type: str, channel_user_id: str) -> dict[str, str] | None:
from turnstone.core.storage._schema import channel_users
with self._engine.connect() as conn:
row = conn.execute(
sa.select(
channel_users.c.channel_type,
channel_users.c.channel_user_id,
channel_users.c.user_id,
channel_users.c.created,
).where(
(channel_users.c.channel_type == channel_type)
& (channel_users.c.channel_user_id == channel_user_id)
)
).fetchone()
if row:
return {
"channel_type": row[0],
"channel_user_id": row[1],
"user_id": row[2],
"created": row[3],
}
return None
def list_channel_users_by_user(self, user_id: str) -> list[dict[str, str]]:
from turnstone.core.storage._schema import channel_users
with self._engine.connect() as conn:
rows = conn.execute(
sa.select(
channel_users.c.channel_type,
channel_users.c.channel_user_id,
channel_users.c.user_id,
channel_users.c.created,
)
.where(channel_users.c.user_id == user_id)
.order_by(channel_users.c.created.desc())
).fetchall()
return [
{
"channel_type": r[0],
"channel_user_id": r[1],
"user_id": r[2],
"created": r[3],
}
for r in rows
]
def delete_channel_user(self, channel_type: str, channel_user_id: str) -> bool:
from turnstone.core.storage._schema import channel_users
with self._engine.connect() as conn:
result = conn.execute(
sa.delete(channel_users).where(
(channel_users.c.channel_type == channel_type)
& (channel_users.c.channel_user_id == channel_user_id)
)
)
conn.commit()
return result.rowcount > 0
# -- Channel routing -------------------------------------------------------
def create_channel_route(
self, channel_type: str, channel_id: str, ws_id: str, node_id: str = ""
) -> None:
from sqlalchemy.dialects import postgresql
from turnstone.core.storage._schema import channel_routes
now = datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%S")
with self._engine.connect() as conn:
conn.execute(
postgresql.insert(channel_routes)
.values(
channel_type=channel_type,
channel_id=channel_id,
ws_id=ws_id,
node_id=node_id,
created=now,
)
.on_conflict_do_nothing()
)
conn.commit()
def get_channel_route(self, channel_type: str, channel_id: str) -> dict[str, str] | None:
from turnstone.core.storage._schema import channel_routes
with self._engine.connect() as conn:
row = conn.execute(
sa.select(
channel_routes.c.channel_type,
channel_routes.c.channel_id,
channel_routes.c.ws_id,
channel_routes.c.node_id,
channel_routes.c.created,
).where(
(channel_routes.c.channel_type == channel_type)
& (channel_routes.c.channel_id == channel_id)
)
).fetchone()
if row:
return {
"channel_type": row[0],
"channel_id": row[1],
"ws_id": row[2],
"node_id": row[3],
"created": row[4],
}
return None
def get_channel_route_by_ws(self, ws_id: str) -> dict[str, str] | None:
from turnstone.core.storage._schema import channel_routes
with self._engine.connect() as conn:
row = conn.execute(
sa.select(
channel_routes.c.channel_type,
channel_routes.c.channel_id,
channel_routes.c.ws_id,
channel_routes.c.node_id,
channel_routes.c.created,
).where(channel_routes.c.ws_id == ws_id)
).fetchone()
if row:
return {
"channel_type": row[0],
"channel_id": row[1],
"ws_id": row[2],
"node_id": row[3],
"created": row[4],
}
return None
def list_channel_routes_by_type(self, channel_type: str) -> list[dict[str, str]]:
from turnstone.core.storage._schema import channel_routes
with self._engine.connect() as conn:
rows = conn.execute(
sa.select(
channel_routes.c.channel_type,
channel_routes.c.channel_id,
channel_routes.c.ws_id,
channel_routes.c.node_id,
channel_routes.c.created,
)
.where(channel_routes.c.channel_type == channel_type)
.order_by(channel_routes.c.created.desc())
).fetchall()
return [
{
"channel_type": r[0],
"channel_id": r[1],
"ws_id": r[2],
"node_id": r[3],
"created": r[4],
}
for r in rows
]
def delete_channel_route(self, channel_type: str, channel_id: str) -> bool:
from turnstone.core.storage._schema import channel_routes
with self._engine.connect() as conn:
result = conn.execute(
sa.delete(channel_routes).where(
(channel_routes.c.channel_type == channel_type)
& (channel_routes.c.channel_id == channel_id)
)
)
conn.commit()
return result.rowcount > 0
# -- Lifecycle -------------------------------------------------------------
def close(self) -> None:
+48
View File
@@ -201,6 +201,54 @@ class StorageBackend(Protocol):
"""Revoke/delete a token by ID. Returns True if existed."""
...
# -- Channel user mapping ---------------------------------------------------
def create_channel_user(self, channel_type: str, channel_user_id: str, user_id: str) -> None:
"""Map an external channel user to a turnstone user_id. No-op if exists."""
...
def get_channel_user(self, channel_type: str, channel_user_id: str) -> dict[str, str] | None:
"""Lookup turnstone user for a channel user. Returns dict or None."""
...
def list_channel_users_by_user(self, user_id: str) -> list[dict[str, str]]:
"""List all channel mappings for a turnstone user."""
...
def delete_channel_user(self, channel_type: str, channel_user_id: str) -> bool:
"""Remove a channel user mapping. Returns True if existed."""
...
# -- Session lookup by workstream ------------------------------------------
def get_session_id_by_ws(self, ws_id: str) -> str | None:
"""Find the session_id associated with a workstream. Returns None if not found."""
...
# -- Channel routing -------------------------------------------------------
def create_channel_route(
self, channel_type: str, channel_id: str, ws_id: str, node_id: str = ""
) -> None:
"""Map a channel/thread to a workstream. No-op if exists."""
...
def get_channel_route(self, channel_type: str, channel_id: str) -> dict[str, str] | None:
"""Lookup workstream for a channel/thread."""
...
def get_channel_route_by_ws(self, ws_id: str) -> dict[str, str] | None:
"""Reverse lookup: find channel/thread for a workstream."""
...
def list_channel_routes_by_type(self, channel_type: str) -> list[dict[str, str]]:
"""List all routes for a channel type, ordered by created DESC."""
...
def delete_channel_route(self, channel_type: str, channel_id: str) -> bool:
"""Remove a channel route. Returns True if existed."""
...
# -- Lifecycle -------------------------------------------------------------
def close(self) -> None:
+17
View File
@@ -120,3 +120,20 @@ channel_users = sa.Table(
)
sa.Index("idx_channel_users_user_id", channel_users.c.user_id)
# ---------------------------------------------------------------------------
# Channel routing tables
# ---------------------------------------------------------------------------
channel_routes = sa.Table(
"channel_routes",
metadata,
sa.Column("channel_type", sa.Text, nullable=False),
sa.Column("channel_id", sa.Text, nullable=False),
sa.Column("ws_id", sa.Text, nullable=False),
sa.Column("node_id", sa.Text, nullable=False, server_default=""),
sa.Column("created", sa.Text, nullable=False),
sa.PrimaryKeyConstraint("channel_type", "channel_id"),
)
sa.Index("idx_channel_routes_ws", channel_routes.c.ws_id)
+197
View File
@@ -732,6 +732,203 @@ class SQLiteBackend:
conn.commit()
return result.rowcount > 0
# -- Session lookup by workstream ------------------------------------------
def get_session_id_by_ws(self, ws_id: str) -> str | None:
with self._engine.connect() as conn:
row = conn.execute(
sa.select(sessions.c.session_id).where(sessions.c.ws_id == ws_id)
).fetchone()
return str(row[0]) if row else None
# -- Channel user mapping ---------------------------------------------------
def create_channel_user(self, channel_type: str, channel_user_id: str, user_id: str) -> None:
from turnstone.core.storage._schema import channel_users
now = datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%S")
with self._engine.connect() as conn:
conn.execute(
sa.insert(channel_users).prefix_with("OR IGNORE"),
{
"channel_type": channel_type,
"channel_user_id": channel_user_id,
"user_id": user_id,
"created": now,
},
)
conn.commit()
def get_channel_user(self, channel_type: str, channel_user_id: str) -> dict[str, str] | None:
from turnstone.core.storage._schema import channel_users
with self._engine.connect() as conn:
row = conn.execute(
sa.select(
channel_users.c.channel_type,
channel_users.c.channel_user_id,
channel_users.c.user_id,
channel_users.c.created,
).where(
(channel_users.c.channel_type == channel_type)
& (channel_users.c.channel_user_id == channel_user_id)
)
).fetchone()
if row:
return {
"channel_type": row[0],
"channel_user_id": row[1],
"user_id": row[2],
"created": row[3],
}
return None
def list_channel_users_by_user(self, user_id: str) -> list[dict[str, str]]:
from turnstone.core.storage._schema import channel_users
with self._engine.connect() as conn:
rows = conn.execute(
sa.select(
channel_users.c.channel_type,
channel_users.c.channel_user_id,
channel_users.c.user_id,
channel_users.c.created,
)
.where(channel_users.c.user_id == user_id)
.order_by(channel_users.c.created.desc())
).fetchall()
return [
{
"channel_type": r[0],
"channel_user_id": r[1],
"user_id": r[2],
"created": r[3],
}
for r in rows
]
def delete_channel_user(self, channel_type: str, channel_user_id: str) -> bool:
from turnstone.core.storage._schema import channel_users
with self._engine.connect() as conn:
result = conn.execute(
sa.delete(channel_users).where(
(channel_users.c.channel_type == channel_type)
& (channel_users.c.channel_user_id == channel_user_id)
)
)
conn.commit()
return result.rowcount > 0
# -- Channel routing -------------------------------------------------------
def create_channel_route(
self, channel_type: str, channel_id: str, ws_id: str, node_id: str = ""
) -> None:
from turnstone.core.storage._schema import channel_routes
now = datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%S")
with self._engine.connect() as conn:
conn.execute(
sa.insert(channel_routes).prefix_with("OR IGNORE"),
{
"channel_type": channel_type,
"channel_id": channel_id,
"ws_id": ws_id,
"node_id": node_id,
"created": now,
},
)
conn.commit()
def get_channel_route(self, channel_type: str, channel_id: str) -> dict[str, str] | None:
from turnstone.core.storage._schema import channel_routes
with self._engine.connect() as conn:
row = conn.execute(
sa.select(
channel_routes.c.channel_type,
channel_routes.c.channel_id,
channel_routes.c.ws_id,
channel_routes.c.node_id,
channel_routes.c.created,
).where(
(channel_routes.c.channel_type == channel_type)
& (channel_routes.c.channel_id == channel_id)
)
).fetchone()
if row:
return {
"channel_type": row[0],
"channel_id": row[1],
"ws_id": row[2],
"node_id": row[3],
"created": row[4],
}
return None
def get_channel_route_by_ws(self, ws_id: str) -> dict[str, str] | None:
from turnstone.core.storage._schema import channel_routes
with self._engine.connect() as conn:
row = conn.execute(
sa.select(
channel_routes.c.channel_type,
channel_routes.c.channel_id,
channel_routes.c.ws_id,
channel_routes.c.node_id,
channel_routes.c.created,
).where(channel_routes.c.ws_id == ws_id)
).fetchone()
if row:
return {
"channel_type": row[0],
"channel_id": row[1],
"ws_id": row[2],
"node_id": row[3],
"created": row[4],
}
return None
def list_channel_routes_by_type(self, channel_type: str) -> list[dict[str, str]]:
from turnstone.core.storage._schema import channel_routes
with self._engine.connect() as conn:
rows = conn.execute(
sa.select(
channel_routes.c.channel_type,
channel_routes.c.channel_id,
channel_routes.c.ws_id,
channel_routes.c.node_id,
channel_routes.c.created,
)
.where(channel_routes.c.channel_type == channel_type)
.order_by(channel_routes.c.created.desc())
).fetchall()
return [
{
"channel_type": r[0],
"channel_id": r[1],
"ws_id": r[2],
"node_id": r[3],
"created": r[4],
}
for r in rows
]
def delete_channel_route(self, channel_type: str, channel_id: str) -> bool:
from turnstone.core.storage._schema import channel_routes
with self._engine.connect() as conn:
result = conn.execute(
sa.delete(channel_routes).where(
(channel_routes.c.channel_type == channel_type)
& (channel_routes.c.channel_id == channel_id)
)
)
conn.commit()
return result.rowcount > 0
# -- Lifecycle -------------------------------------------------------------
def close(self) -> None:
@@ -0,0 +1,32 @@
"""Channel routing table.
Revision ID: 003
Revises: 002
Create Date: 2026-03-04
"""
import sqlalchemy as sa
from alembic import op
revision = "003"
down_revision = "002"
branch_labels = None
depends_on = None
def upgrade() -> None:
op.create_table(
"channel_routes",
sa.Column("channel_type", sa.Text, nullable=False),
sa.Column("channel_id", sa.Text, nullable=False),
sa.Column("ws_id", sa.Text, nullable=False),
sa.Column("node_id", sa.Text, nullable=False, server_default=""),
sa.Column("created", sa.Text, nullable=False),
sa.PrimaryKeyConstraint("channel_type", "channel_id"),
)
op.create_index("idx_channel_routes_ws", "channel_routes", ["ws_id"])
def downgrade() -> None:
op.drop_index("idx_channel_routes_ws", "channel_routes")
op.drop_table("channel_routes")
+16 -1
View File
@@ -8,4 +8,19 @@ commands and subscribe to progress.
from turnstone.mq.broker import MessageBroker, RedisBroker
from turnstone.mq.client import TurnResult, TurnstoneClient
__all__ = ["MessageBroker", "RedisBroker", "TurnstoneClient", "TurnResult"]
__all__ = [
"AsyncRedisBroker",
"MessageBroker",
"RedisBroker",
"TurnstoneClient",
"TurnResult",
]
def __getattr__(name: str) -> object:
if name == "AsyncRedisBroker":
from turnstone.mq.async_broker import AsyncRedisBroker
return AsyncRedisBroker
msg = f"module {__name__!r} has no attribute {name!r}"
raise AttributeError(msg)
+314
View File
@@ -0,0 +1,314 @@
"""Async Redis message broker.
Provides :class:`AsyncRedisBroker`, an asyncio-native counterpart to
:class:`~turnstone.mq.broker.RedisBroker`. Uses ``redis.asyncio`` for all I/O
and manages pub/sub listeners as :class:`asyncio.Task` instances.
"""
from __future__ import annotations
import asyncio
import contextlib
import json
from typing import TYPE_CHECKING, Any
if TYPE_CHECKING:
from collections.abc import Callable
import redis.asyncio as _aredis_t
class AsyncRedisBroker:
"""Async Redis-backed message broker using lists (queues) and pub/sub.
This is the asyncio equivalent of :class:`~turnstone.mq.broker.RedisBroker`.
All methods are coroutines and must be awaited.
Queue keys:
``{prefix}:inbound`` shared inbound command queue
``{prefix}:inbound:{node_id}`` per-node directed queue
``{prefix}:resp:{request_id}`` per-request response queues
Routing keys:
``{prefix}:ws:{ws_id}`` workstream ownership (string)
``{prefix}:node:{node_id}`` node heartbeat + metadata (string/JSON)
Pub/sub channels:
``{prefix}:events:global`` global event channel
``{prefix}:events:{ws_id}`` per-workstream event channel
``{prefix}:events:cluster`` cluster-wide state changes
"""
def __init__(
self,
host: str = "localhost",
port: int = 6379,
db: int = 0,
prefix: str = "turnstone",
password: str | None = None,
response_ttl: int = 600,
) -> None:
self._host = host
self._port = port
self._db = db
self._password = password
self._prefix = prefix
self._response_ttl = response_ttl
self._redis: _aredis_t.Redis[str] | None = None
self._pubsub: _aredis_t.client.PubSub | None = None
self._tasks: dict[str, asyncio.Task[None]] = {}
self._callbacks: dict[str, Callable[[str], Any]] = {}
self._queues: dict[str, asyncio.Queue[str]] = {}
self._workers: dict[str, asyncio.Task[None]] = {}
self._listener_task: asyncio.Task[None] | None = None
# -- connection ----------------------------------------------------------
async def connect(self) -> None:
"""Create the async Redis connection.
This is called lazily before first use if the connection has not yet
been established.
"""
if self._redis is not None:
return
import redis.asyncio as aioredis
self._redis = aioredis.Redis(
host=self._host,
port=self._port,
db=self._db,
password=self._password,
decode_responses=True,
retry_on_timeout=True,
)
self._pubsub = self._redis.pubsub(ignore_subscribe_messages=True)
async def _ensure_connected(self) -> None:
"""Ensure the Redis connection is established."""
if self._redis is None:
await self.connect()
@property
def _r(self) -> _aredis_t.Redis[str]:
"""Return the Redis client, assuming it is connected."""
if self._redis is None:
msg = "Broker not connected — call connect() first"
raise RuntimeError(msg)
return self._redis
@property
def _ps(self) -> _aredis_t.client.PubSub:
"""Return the pub/sub client, assuming it is connected."""
if self._pubsub is None:
msg = "Broker not connected — call connect() first"
raise RuntimeError(msg)
return self._pubsub
# -- inbound queue -------------------------------------------------------
async def push_inbound(self, message: str, node_id: str = "") -> None:
"""Push a message onto the inbound queue.
If *node_id* is set, pushes to the per-node queue for directed
routing. Otherwise pushes to the shared queue.
"""
await self._ensure_connected()
if node_id:
await self._r.rpush(f"{self._prefix}:inbound:{node_id}", message)
else:
await self._r.rpush(f"{self._prefix}:inbound", message)
# -- outbound pub/sub ----------------------------------------------------
async def publish_outbound(self, channel: str, event: str) -> None:
"""Publish an event to an outbound channel."""
await self._ensure_connected()
await self._r.publish(channel, event)
async def subscribe(self, channel: str, callback: Callable[[str], Any]) -> None:
"""Subscribe to a pub/sub channel.
The *callback* receives the message string for each published event.
It may be a regular function or an async coroutine.
All subscriptions share a single listener task that dispatches
messages to the correct callback based on the channel name.
"""
await self._ensure_connected()
await self._ps.subscribe(channel)
self._callbacks[channel] = callback
# Per-channel queue + worker ensures ordered delivery within a channel
# while allowing different channels to process concurrently.
q: asyncio.Queue[str] = asyncio.Queue()
self._queues[channel] = q
self._workers[channel] = asyncio.create_task(self._channel_worker(channel, q))
# Start the shared listener task if not already running.
if self._listener_task is None or self._listener_task.done():
self._listener_task = asyncio.create_task(self._dispatch_loop())
async def _dispatch_loop(self) -> None:
"""Single listener that routes pub/sub messages to per-channel queues.
Each channel has its own queue + worker task, ensuring ordered
delivery within a channel while allowing different channels to
process concurrently.
"""
import logging
_log = logging.getLogger("turnstone.mq.async_broker")
try:
while self._callbacks:
msg = await self._ps.get_message(
ignore_subscribe_messages=True,
timeout=0.1,
)
if msg is None:
# Yield control so cancellation can be delivered.
await asyncio.sleep(0)
continue
if msg["type"] == "message":
ch = msg.get("channel", "")
q = self._queues.get(ch)
if q is not None:
q.put_nowait(msg["data"])
_log.debug("Dispatch loop exiting — no active callbacks")
except asyncio.CancelledError:
return
async def _channel_worker(self, channel: str, q: asyncio.Queue[str]) -> None:
"""Process messages for a single channel sequentially."""
import logging
_log = logging.getLogger("turnstone.mq.async_broker")
try:
while True:
data = await q.get()
cb = self._callbacks.get(channel)
if cb is not None:
try:
result = cb(data)
if asyncio.iscoroutine(result):
await result
except Exception:
_log.exception("Listener callback error on %s", channel)
except asyncio.CancelledError:
return
async def unsubscribe(self, channel: str) -> None:
"""Unsubscribe from a channel and cancel its worker."""
await self._ensure_connected()
self._callbacks.pop(channel, None)
self._queues.pop(channel, None)
worker = self._workers.pop(channel, None)
if worker is not None:
worker.cancel()
with contextlib.suppress(asyncio.CancelledError):
await worker
# Legacy per-channel task cleanup (in case any remain).
task = self._tasks.pop(channel, None)
if task is not None:
task.cancel()
with contextlib.suppress(asyncio.CancelledError):
await task
await self._ps.unsubscribe(channel)
# -- response queues -----------------------------------------------------
async def push_response(self, queue_name: str, message: str) -> None:
"""Push a response onto a named response queue."""
await self._ensure_connected()
key = f"{self._prefix}:resp:{queue_name}"
await self._r.rpush(key, message)
await self._r.expire(key, self._response_ttl)
async def pop_response(self, queue_name: str, timeout: float = 300) -> str | None:
"""Pop from a named response queue. Returns ``None`` on timeout."""
await self._ensure_connected()
key = f"{self._prefix}:resp:{queue_name}"
result = await self._r.blpop(key, timeout=int(timeout))
return result[1] if result else None
# -- routing primitives --------------------------------------------------
async def get_ws_owner(self, ws_id: str) -> str | None:
"""Look up the node that owns a workstream."""
await self._ensure_connected()
return await self._r.get(f"{self._prefix}:ws:{ws_id}")
async def set_ws_owner(self, ws_id: str, node_id: str, ttl: int = 0) -> None:
"""Register which node owns a workstream."""
await self._ensure_connected()
key = f"{self._prefix}:ws:{ws_id}"
if ttl > 0:
await self._r.set(key, node_id, ex=ttl)
else:
await self._r.set(key, node_id)
async def del_ws_owner(self, ws_id: str) -> None:
"""Remove workstream ownership."""
await self._ensure_connected()
await self._r.delete(f"{self._prefix}:ws:{ws_id}")
async def register_node(self, node_id: str, metadata: dict[str, Any], ttl: int = 60) -> None:
"""Register or refresh a node's heartbeat with metadata."""
await self._ensure_connected()
key = f"{self._prefix}:node:{node_id}"
await self._r.set(key, json.dumps(metadata), ex=ttl)
async def list_nodes(self) -> list[dict[str, Any]]:
"""List all active nodes (those with unexpired heartbeats)."""
await self._ensure_connected()
pattern = f"{self._prefix}:node:*"
prefix_len = len(f"{self._prefix}:node:")
nodes: list[dict[str, Any]] = []
async for key in self._r.scan_iter(match=pattern, count=100):
raw = await self._r.get(key)
if raw:
try:
meta: dict[str, Any] = json.loads(raw)
except json.JSONDecodeError:
meta = {}
meta["node_id"] = key[prefix_len:]
nodes.append(meta)
return nodes
# -- lifecycle -----------------------------------------------------------
async def close(self) -> None:
"""Cancel all listener tasks and close the Redis connection."""
self._callbacks.clear()
self._queues.clear()
# Cancel per-channel workers.
for worker in self._workers.values():
worker.cancel()
for worker in self._workers.values():
with contextlib.suppress(asyncio.CancelledError):
await worker
self._workers.clear()
# Cancel the shared dispatch loop.
if self._listener_task is not None:
if not self._listener_task.done():
self._listener_task.cancel()
with contextlib.suppress(asyncio.CancelledError):
await self._listener_task
self._listener_task = None
# Legacy per-channel tasks.
for task in self._tasks.values():
task.cancel()
for task in self._tasks.values():
with contextlib.suppress(asyncio.CancelledError):
await task
self._tasks.clear()
if self._pubsub is not None:
with contextlib.suppress(Exception):
await self._pubsub.close()
self._pubsub = None
if self._redis is not None:
await self._redis.close()
self._redis = None
+132 -40
View File
@@ -34,6 +34,7 @@ from turnstone.mq.protocol import (
OutboundEvent,
PlanReviewEvent,
ReasoningEvent,
SessionResumedEvent,
StateChangeEvent,
StatusEvent,
StreamEndEvent,
@@ -97,6 +98,8 @@ class Bridge:
self._ws_auto_approve: dict[str, bool] = {}
self._ws_approve_tools: dict[str, set[str]] = {}
self._active_sends: dict[str, str] = {} # ws_id → correlation_id
self._pending_approvals: dict[str, str] = {} # ws_id → request_id
self._pending_plan_reviews: dict[str, str] = {} # ws_id → request_id
self._running = True
# -- thread context helper ------------------------------------------------
@@ -266,7 +269,7 @@ class Bridge:
# Auto-create workstream if needed
if not ws_id:
ws_id = self._create_ws_on_server(
ws_id, _resumed = self._create_ws_on_server(
name=name,
auto_approve=auto_approve,
auto_approve_tools=auto_approve_tools,
@@ -285,8 +288,17 @@ class Bridge:
with self._lock:
self._active_sends[ws_id] = msg.correlation_id
resp = self._http.post("/v1/api/send", json={"message": message, "ws_id": ws_id})
data = resp.json()
try:
resp = self._http.post("/v1/api/send", json={"message": message, "ws_id": ws_id})
data = resp.json()
except Exception:
with self._lock:
self._active_sends.pop(ws_id, None)
raise
if data.get("status") != "ok":
with self._lock:
self._active_sends.pop(ws_id, None)
self._publish_ws(
ws_id,
@@ -329,14 +341,23 @@ class Bridge:
auto_approve_tools = getattr(msg, "auto_approve_tools", [])
model = getattr(msg, "model", "")
initial_message = getattr(msg, "initial_message", "")
ws_id = self._create_ws_on_server(
resume_session = getattr(msg, "resume_session", "")
ws_id, resumed = self._create_ws_on_server(
name=name,
auto_approve=auto_approve,
auto_approve_tools=auto_approve_tools,
correlation_id=msg.correlation_id,
model=model,
resume_session=resume_session,
)
if ws_id and initial_message:
# Send initial_message only when no session was actually resumed.
# Use the server's `resumed` response (not just the intent) so that
# a pruned/missing session falls back to sending the initial message.
if ws_id and initial_message and not resumed:
# Track the send so the global SSE handler emits TurnCompleteEvent
# when the workstream returns to idle.
with self._lock:
self._active_sends[ws_id] = msg.correlation_id
try:
resp = self._http.post(
"/v1/api/send", json={"message": initial_message, "ws_id": ws_id}
@@ -344,8 +365,12 @@ class Bridge:
data = resp.json()
if data.get("error"):
log.warning("Initial message failed for ws %s: %s", ws_id, data["error"])
with self._lock:
self._active_sends.pop(ws_id, None)
except Exception as exc:
log.warning("Initial message send failed for ws %s: %s", ws_id, exc)
with self._lock:
self._active_sends.pop(ws_id, None)
def _handle_close_ws(self, msg: InboundMessage) -> None:
ws_id = getattr(msg, "ws_id", "")
@@ -390,12 +415,15 @@ class Bridge:
auto_approve_tools: list[str],
correlation_id: str,
model: str = "",
) -> str:
"""Create a workstream on the server. Returns ws_id or empty on error."""
resume_session: str = "",
) -> tuple[str, bool]:
"""Create a workstream on the server. Returns (ws_id, resumed)."""
try:
payload: dict[str, Any] = {"name": name, "auto_approve": auto_approve}
if model:
payload["model"] = model
if resume_session:
payload["resume_session"] = resume_session
resp = self._http.post(
"/v1/api/workstreams/new",
json=payload,
@@ -409,9 +437,10 @@ class Bridge:
detail=data["error"],
)
)
return ""
return "", False
ws_id: str = data["ws_id"]
ws_name = data.get("name", "")
resumed = data.get("resumed", False)
self._broker.set_ws_owner(ws_id, self._node_id)
@@ -423,11 +452,16 @@ class Bridge:
self._start_ws_sse(ws_id)
resolved_session_id = data.get("session_id", "") if resumed else ""
self._publish_global(
WorkstreamCreatedEvent(
ws_id=ws_id,
name=ws_name,
correlation_id=correlation_id,
resumed=resumed,
session_id=resolved_session_id,
message_count=data.get("message_count", 0),
)
)
self._publish_cluster(
@@ -435,9 +469,24 @@ class Bridge:
ws_id=ws_id,
name=ws_name,
correlation_id=correlation_id,
node_id=self._node_id,
)
)
return ws_id
# Emit per-workstream resume confirmation.
if resumed:
self._publish_ws(
ws_id,
SessionResumedEvent(
ws_id=ws_id,
correlation_id=correlation_id,
session_id=resolved_session_id,
message_count=data.get("message_count", 0),
name=ws_name,
),
)
return ws_id, resumed
except Exception as exc:
self._publish_global(
AckEvent(
@@ -446,7 +495,7 @@ class Bridge:
detail=str(exc),
)
)
return ""
return "", False
# -- SSE consumption -----------------------------------------------------
@@ -535,21 +584,31 @@ class Bridge:
"""Handle an approval request — auto-approve or forward to client."""
items = data.get("items", [])
# Check if all tools can be auto-approved
# Read flags under lock, then release before any HTTP calls.
with self._lock:
if self._ws_auto_approve.get(ws_id):
self._api_approve(ws_id, approved=True)
return
auto = self._ws_auto_approve.get(ws_id, False)
approve_set = self._ws_approve_tools.get(ws_id, DEFAULT_SAFE_TOOLS)
if auto:
self._api_approve(ws_id, approved=True)
return
tool_names = {it.get("func_name", "") for it in items if it.get("needs_approval")}
if tool_names and tool_names.issubset(approve_set):
self._api_approve(ws_id, approved=True)
return
# Skip if an approval is already pending for this workstream (SSE
# reconnects re-inject the pending approval, causing duplicates).
with self._lock:
if ws_id in self._pending_approvals:
log.debug("Skipping duplicate approval for ws %s", ws_id)
return
request_id = uuid.uuid4().hex[:12]
self._pending_approvals[ws_id] = request_id
# Forward to client — spawn a thread so we don't block SSE consumption
request_id = uuid.uuid4().hex[:12]
self._publish_ws(
ws_id,
ApprovalRequestEvent(
@@ -560,30 +619,42 @@ class Bridge:
)
def _wait_approval() -> None:
raw_resp = self._broker.pop_response(request_id, timeout=self._approval_timeout)
if raw_resp:
resp_msg = InboundMessage.from_json(raw_resp)
approved = getattr(resp_msg, "approved", False)
feedback = getattr(resp_msg, "feedback", None)
always = getattr(resp_msg, "always", False)
self._api_approve(ws_id, approved=approved, feedback=feedback)
if always:
with self._lock:
self._ws_auto_approve[ws_id] = True
else:
log.warning("Approval timeout for ws %s — denying", ws_id)
self._api_approve(ws_id, approved=False, feedback="Approval timed out")
try:
raw_resp = self._broker.pop_response(request_id, timeout=self._approval_timeout)
if raw_resp:
resp_msg = InboundMessage.from_json(raw_resp)
approved = getattr(resp_msg, "approved", False)
feedback = getattr(resp_msg, "feedback", None)
always = getattr(resp_msg, "always", False)
self._api_approve(ws_id, approved=approved, feedback=feedback)
if always:
with self._lock:
self._ws_auto_approve[ws_id] = True
else:
log.warning("Approval timeout for ws %s — denying", ws_id)
self._api_approve(ws_id, approved=False, feedback="Approval timed out")
finally:
with self._lock:
self._pending_approvals.pop(ws_id, None)
threading.Thread(target=self._run_in_context(_wait_approval), daemon=True).start()
def _handle_plan_review(self, ws_id: str, data: dict[str, Any]) -> None:
"""Handle a plan review request — auto-approve or forward to client."""
with self._lock:
if self._ws_auto_approve.get(ws_id):
self._http.post("/v1/api/plan", json={"feedback": "", "ws_id": ws_id})
auto = self._ws_auto_approve.get(ws_id, False)
# Skip if a plan review is already pending (SSE reconnect guard).
if not auto and ws_id in self._pending_plan_reviews:
log.debug("Skipping duplicate plan review for ws %s", ws_id)
return
if not auto:
request_id = uuid.uuid4().hex[:12]
self._pending_plan_reviews[ws_id] = request_id
if auto:
self._http.post("/v1/api/plan", json={"feedback": "", "ws_id": ws_id})
return
request_id = uuid.uuid4().hex[:12]
self._publish_ws(
ws_id,
PlanReviewEvent(
@@ -594,14 +665,18 @@ class Bridge:
)
def _wait_plan() -> None:
raw_resp = self._broker.pop_response(request_id, timeout=self._approval_timeout)
if raw_resp:
resp_msg = InboundMessage.from_json(raw_resp)
feedback = getattr(resp_msg, "feedback", "")
self._http.post("/v1/api/plan", json={"feedback": feedback, "ws_id": ws_id})
else:
log.warning("Plan review timeout for ws %s — rejecting", ws_id)
self._http.post("/v1/api/plan", json={"feedback": "reject", "ws_id": ws_id})
try:
raw_resp = self._broker.pop_response(request_id, timeout=self._approval_timeout)
if raw_resp:
resp_msg = InboundMessage.from_json(raw_resp)
feedback = getattr(resp_msg, "feedback", "")
self._http.post("/v1/api/plan", json={"feedback": feedback, "ws_id": ws_id})
else:
log.warning("Plan review timeout for ws %s — rejecting", ws_id)
self._http.post("/v1/api/plan", json={"feedback": "reject", "ws_id": ws_id})
finally:
with self._lock:
self._pending_plan_reviews.pop(ws_id, None)
threading.Thread(target=self._run_in_context(_wait_plan), daemon=True).start()
@@ -810,13 +885,30 @@ def main() -> None:
db=args.redis_db,
password=args.redis_password,
)
# If no explicit auth token is provided, mint a service JWT using the
# shared secret so the bridge can authenticate to the server.
auth_token = args.auth_token
if not auth_token:
jwt_secret = os.environ.get("TURNSTONE_JWT_SECRET", "")
if jwt_secret:
from turnstone.core.auth import create_jwt
auth_token = create_jwt(
user_id="bridge",
scopes=frozenset({"approve"}),
source="bridge",
secret=jwt_secret,
expiry_hours=168, # 1 week — bridge restarts refresh
)
log.info("bridge.jwt_minted")
bridge = Bridge(
server_url=args.server_url,
broker=broker,
approval_timeout=args.approval_timeout,
node_id=args.node_id,
heartbeat_ttl=args.heartbeat_ttl,
auth_token=args.auth_token,
auth_token=auth_token,
)
bridge.run()
+16
View File
@@ -95,6 +95,7 @@ class CreateWorkstreamMessage(InboundMessage):
target_node: str = ""
model: str = ""
initial_message: str = ""
resume_session: str = ""
@dataclass
@@ -273,6 +274,10 @@ class WorkstreamCreatedEvent(OutboundEvent):
type: str = "ws_created"
name: str = ""
node_id: str = ""
resumed: bool = False
session_id: str = ""
message_count: int = 0
@dataclass
@@ -330,6 +335,16 @@ class NodeListEvent(OutboundEvent):
nodes: list[dict[str, Any]] = field(default_factory=list)
@dataclass
class SessionResumedEvent(OutboundEvent):
"""Confirmation that a session was resumed during workstream creation."""
type: str = "session_resumed"
session_id: str = ""
message_count: int = 0
name: str = ""
@dataclass
class ClusterStateEvent(OutboundEvent):
"""Workstream state change with node attribution for cluster dashboard."""
@@ -395,6 +410,7 @@ _OUTBOUND_REGISTRY: dict[str, type[OutboundEvent]] = {
ErrorEvent,
InfoEvent,
NodeListEvent,
SessionResumedEvent,
ClusterStateEvent,
]
}
+7 -1
View File
@@ -76,6 +76,7 @@ class AsyncTurnstoneServer(_BaseClient):
name: str = "",
model: str = "",
auto_approve: bool = False,
resume_session: str = "",
) -> CreateWorkstreamResponse:
body: dict[str, Any] = {}
if name:
@@ -84,6 +85,8 @@ class AsyncTurnstoneServer(_BaseClient):
body["model"] = model
if auto_approve:
body["auto_approve"] = True
if resume_session:
body["resume_session"] = resume_session
return await self._request(
"POST",
"/v1/api/workstreams/new",
@@ -304,9 +307,12 @@ class TurnstoneServer:
name: str = "",
model: str = "",
auto_approve: bool = False,
resume_session: str = "",
) -> CreateWorkstreamResponse:
return self._runner.run(
self._async.create_workstream(name=name, model=model, auto_approve=auto_approve)
self._async.create_workstream(
name=name, model=model, auto_approve=auto_approve, resume_session=resume_session
)
)
def close_workstream(self, ws_id: str) -> StatusResponse:
+30 -1
View File
@@ -917,7 +917,36 @@ async def create_workstream(request: Request) -> JSONResponse:
"reason": "evicted",
}
)
return JSONResponse({"ws_id": ws.id, "name": ws.name})
# Atomic session resume during creation.
resumed = False
message_count = 0
session_id = ""
resume_session_id = body.get("resume_session", "")
if resume_session_id and ws.session is not None:
from turnstone.core.memory import get_session_name, resolve_session
target_id = resolve_session(resume_session_id)
if target_id and ws.session.resume_session(target_id):
resumed = True
session_id = target_id
message_count = len(ws.session.messages)
ws.name = get_session_name(target_id) or ws.name
ui = ws.ui
if isinstance(ui, WebUI):
ui._enqueue({"type": "clear_ui"})
history = _build_history(ws.session)
if history:
ui._enqueue({"type": "history", "messages": history})
return JSONResponse(
{
"ws_id": ws.id,
"name": ws.name,
"resumed": resumed,
"session_id": session_id,
"message_count": message_count,
}
)
except RuntimeError as e:
return JSONResponse({"error": str(e)}, status_code=400)
+6 -11
View File
@@ -862,9 +862,10 @@ function dashboardResumeSession(sessionId) {
authFetch("/v1/api/workstreams/new", {
method: "POST",
headers: { "Content-Type": "application/json" },
body: "{}",
body: JSON.stringify({ resume_session: sessionId }),
})
.then(function (r) {
if (!r.ok) throw new Error("HTTP " + r.status);
return r.json();
})
.then(function (data) {
@@ -872,16 +873,10 @@ function dashboardResumeSession(sessionId) {
workstreams[data.ws_id] = { name: data.name, state: "idle" };
switchTab(data.ws_id);
hideDashboard();
authFetch("/v1/api/command", {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({
ws_id: data.ws_id,
command: "/resume " + sessionId,
}),
}).catch(function (err) {
addErrorMessage("Failed to resume: " + err.message);
});
// Resume handled atomically by server — history arrives via SSE.
})
.catch(function (err) {
showToast("Failed to resume session", "error");
});
}
function dashboardNewChat() {