Compare commits

...

11 Commits

Author SHA1 Message Date
Patrick Buckley 28ef63a10c fix: revalidate frontend assets across builds 2026-08-12 19:00:31 -07:00
Patrick Buckley 998271b016 feat: convert large pastes to attachments 2026-08-12 00:27:38 -07:00
Patrick Buckley 6eae1c3954 fix(providers): support OpenAI v3 HTTPX2 transport 2026-08-11 23:30:44 -07:00
Patrick Buckley d961af5c45 Remove unreachable MCP owner branch 2026-08-11 22:03:10 -07:00
Patrick Buckley 0599d72625 Apply MCP formatting 2026-08-11 22:03:10 -07:00
Patrick Buckley 2b51b2f8fa Resolve FastMCP settings before lifecycle tests 2026-08-11 22:03:10 -07:00
Patrick Buckley 8517b42ec1 Handle partial MCP resource discovery 2026-08-11 22:03:10 -07:00
Patrick Buckley ed6286ab63 fix(deps): constrain OpenAI SDK below v3 2026-08-11 21:58:21 -07:00
Patrick Buckley 3252f3fd95 fix(memory): preserve replay identity and validation 2026-08-11 21:58:21 -07:00
Patrick Buckley cc84f9d176 fix(memory): harden project scope authorization and consistency 2026-08-11 21:58:21 -07:00
Copilot d2a6c2852e Stabilize context-overflow compaction test in Python 3.11 CI (#1006)
* Stabilize overflow compaction test expectations

Co-authored-by: eous <13773563+eous@users.noreply.github.com>

* Fix ObservedRLock compatibility with Python 3.14 Condition

Co-authored-by: eous <13773563+eous@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: eous <13773563+eous@users.noreply.github.com>
2026-08-11 05:46:02 -07:00
85 changed files with 6311 additions and 1521 deletions
+1 -1
View File
@@ -55,7 +55,7 @@
{
"description": "LLM SDKs — always review manually",
"groupName": "LLM SDKs",
"matchPackageNames": ["openai", "anthropic", "mcp"],
"matchPackageNames": ["openai", "httpx2", "anthropic", "mcp"],
"schedule": ["before 9am on Monday"],
"automerge": false
},
+26
View File
@@ -18,6 +18,12 @@ Earlier stable lines (`stable/1.6`, `stable/1.5`) are frozen.
### Added
- **Large plain-text pastes become attachments.** Pasting text longer than the
fixed 2,000-character threshold stages `pasted-text.txt` across all five
attachment-capable create/send composers. Clipboard files retain priority,
text above the 512 KiB upload ceiling stays inline, identical synthesized
pastes collapse to one chip, and rejected attachment sends keep the staged
message and files so they can be corrected or retried.
- **`server_parses_reasoning` model capability.** Declare it on a model
definition whose backend segregates reasoning into its own channel (a
vLLM launched with a reasoning parser, a commercial provider): the
@@ -152,6 +158,16 @@ Earlier stable lines (`stable/1.6`, `stable/1.5`) are frozen.
### Changed
- **OpenAI SDK v3 and its HTTPX2 default transport are now supported (#1009).**
Chat Completions and Responses streams normalize native HTTPX2 connection
deaths through the same retry boundary as legacy HTTPX-backed providers,
including failures observed after safe cross-thread client closure during a
model-registry reload. The OpenAI v3 runtime escape hatch for explicitly
injected legacy HTTPX clients remains supported. OpenAI connections now
follow HTTPX2's operating-system trust store by default; deployments that
relied on a modified `certifi` bundle must install that CA in the system
store or set `SSL_CERT_FILE` / `SSL_CERT_DIR`.
- **Log event rename: `drain_stream.post_finish_blip` is now
`stream.post_finish_blip`; its `usage_captured` field is retained.** The
single-shot drain normalizes mid-body transport deaths through the same
@@ -217,6 +233,16 @@ Earlier stable lines (`stable/1.6`, `stable/1.5`) are frozen.
### Fixed
- **MCP servers may expose resources without resource templates, or templates
without concrete resources (#993).** The MCP handshake has one aggregate
`resources` capability, but implementations are not required to support both
list methods. An exact JSON-RPC `Method not found` response from either
method is now treated as an empty half-catalog during static and per-user
discovery and refresh, while every other error still fails closed. A failed
static registration also tears down its transport and publishes none of its
staged tools/resources/prompts, eliminating the live callable “ghost” tools
that could remain after the UI reported registration failure.
- **A cancelled judge, guard, or compaction call can now stop before its
request goes out (#972).** Previously it could not: `model_turn` refused
to *re-issue* an abandoned call after a mid-stream death, but nothing
+26 -16
View File
@@ -1758,16 +1758,19 @@ Status code: `403`
### `GET /v1/api/memories`
List structured memories with optional filters. Requires `read` scope.
List structured memories with optional filters. Requires `read` scope. Without
`scope`, returns only `global` plus the authenticated caller's `user`
namespace. The public endpoint accepts `global`, `workstream`, and `user`;
explicit workstream access is owner-bound.
**Query parameters:**
| Parameter | Type | Required | Default | Description |
|------------|--------|----------|---------|------------------------------|
| `type` | string | no | `""` | Filter by memory type (user, project, feedback, reference) |
| `type` | string | no | `""` | Filter by memory type (user, general, feedback, reference) |
| `scope` | string | no | `""` | Filter by scope (global, workstream, user) |
| `scope_id` | string | no | `""` | Scope qualifier. Auto-resolved for `scope=user` when auth is active. |
| `limit` | int | no | `100` | Max results (capped at 200) |
| `limit` | int | no | `100` | Max results (1-200) |
**Response:**
@@ -1778,7 +1781,7 @@ List structured memories with optional filters. Requires `read` scope.
"memory_id": "a1b2c3d4-e5f6-...",
"name": "project_architecture",
"description": "Core architecture patterns",
"type": "project",
"type": "general",
"scope": "global",
"scope_id": "",
"content": "The project uses a hexagonal architecture...",
@@ -1795,7 +1798,8 @@ List structured memories with optional filters. Requires `read` scope.
### `POST /v1/api/memories`
Save or upsert a structured memory. Requires `write` scope. Returns `201` on
create, `200` on update.
create, `200` on update. Every write must include a non-empty, non-whitespace
`description`; content-only updates are rejected.
**Request body:**
@@ -1804,7 +1808,7 @@ create, `200` on update.
"name": "deployment_process",
"content": "Deploy via GitHub Actions. Staging auto-deploys on push to main.",
"description": "CI/CD deployment workflow",
"type": "project",
"type": "general",
"scope": "global",
"scope_id": ""
}
@@ -1814,8 +1818,8 @@ create, `200` on update.
|--------------|--------|----------|-------------|--------------------------------------|
| `name` | string | yes | -- | Memory name (max 256 chars) |
| `content` | string | yes | -- | Memory content (max 65536 chars) |
| `description`| string | no | `""` | Short description for search ranking |
| `type` | string | no | `"project"` | One of: user, project, feedback, reference |
| `description`| string | yes | -- | Non-empty relevance summary, required on create and update |
| `type` | string | no | unset | user, general, feedback, or reference |
| `scope` | string | no | `"global"` | One of: global, workstream, user |
| `scope_id` | string | no | `""` | Scope qualifier (auto-resolved for user scope) |
@@ -1826,7 +1830,7 @@ create, `200` on update.
"memory_id": "a1b2c3d4-e5f6-...",
"name": "deployment_process",
"description": "CI/CD deployment workflow",
"type": "project",
"type": "general",
"scope": "global",
"scope_id": "",
"content": "Deploy via GitHub Actions...",
@@ -1839,21 +1843,25 @@ create, `200` on update.
| Status | Condition |
|--------|--------------------------------------------------------|
| 400 | Missing name, empty content, invalid type/scope, name too long, content too long |
| 400 | Invalid input, public scope, scope ID, or limit |
| 403 | Cross-user or non-owner workstream access |
| 404 | Explicit workstream does not exist |
| 500 | Storage mutation failed |
---
### `POST /v1/api/memories/search`
Search memories by query. Uses POST for the request body but is non-mutating
(requires only `read` scope).
(requires only `read` scope). An omitted scope searches only `global` plus the
authenticated caller's `user` namespace.
**Request body:**
```json
{
"query": "authentication",
"type": "project",
"type": "general",
"scope": "",
"limit": 20
}
@@ -1865,7 +1873,7 @@ Search memories by query. Uses POST for the request body but is non-mutating
| `type` | string | no | `""` | Filter by type |
| `scope` | string | no | `""` | Filter by scope |
| `scope_id` | string | no | `""` | Filter by scope ID |
| `limit` | int | no | `20` | Max results (capped at 50) |
| `limit` | int | no | `20` | Max results (1-50) |
**Response:**
@@ -1876,7 +1884,7 @@ Search memories by query. Uses POST for the request body but is non-mutating
"memory_id": "a1b2c3d4-e5f6-...",
"name": "auth_patterns",
"description": "Authentication architecture",
"type": "project",
"type": "general",
"scope": "global",
"scope_id": "",
"content": "JWT tokens with HS256...",
@@ -1894,7 +1902,9 @@ Search memories by query. Uses POST for the request body but is non-mutating
### `DELETE /v1/api/memories/{name}`
Delete a memory by name and scope. Requires `write` scope.
Delete a memory by name and scope. Requires `write` scope. The delete returns
success only for the row atomically removed and records the authenticated
actor in the audit log.
**Path parameters:**
@@ -1978,7 +1988,7 @@ Get a single memory by ID. Requires `admin.memories` permission.
"memory_id": "a1b2c3d4-e5f6-...",
"name": "project_architecture",
"description": "Core architecture patterns",
"type": "project",
"type": "general",
"scope": "global",
"scope_id": "",
"content": "The project uses...",
+14
View File
@@ -429,6 +429,20 @@ Files require a non-empty initial task so the first turn consumes the staged
attachments. The console shell does not currently expose a fork action; use the
node's standalone workstream UI or the create API's `resume_ws` field.
### Large pasted text
Browser composers turn plain text longer than 2,000 Unicode code points into a
`text/plain` attachment named `pasted-text.txt`. A paste exactly at the
threshold stays inline. This applies to the interactive and coordinator send
boxes, the console home launcher, and the node dashboard and new-workstream
composers.
Clipboard files take priority over clipboard text. Text larger than the 512 KiB
attachment ceiling also stays inline, so the browser does not discard it before
a rejected upload. Attachments require a companion message and cannot be sent
as live-turn interjections; a busy composer preserves its message and chips for
an idle retry.
### Saved and filtered sessions
Saved coordinator and interactive sessions share one list with kind and persona
+1 -1
View File
@@ -77,7 +77,7 @@ or MCP config can do adds to it. Current members:
| `delete_workstream` | wind-down | Hard-delete one child. Requires approval. |
| `list_nodes` | discover | Enumerate live cluster nodes + capabilities. |
| `skills` (action=find) | discover | Browse the skill catalog; opt-in `kind` filter narrows by audience. |
| `memory` | persist | Durable orchestration memory (`coordinator` scope, per-user — survives across coordinator sessions). |
| `memory` | persist | Durable acting-user orchestration memory (`coordinator`), plus shared memory when attached to a project. |
| `notify` | broadcast | Post a status update to a human channel at a narrative beat. |
| `tasks` | plan | Orchestrator-only scratchpad. Children don't see it. |
+44 -38
View File
@@ -20,58 +20,61 @@ participant "SDK Client\n(sdk/)" as SDK <<sdk>>
== Phase 1: Tool Path (session.send) ==
Session -> Session : _prepare_tool_calls()\nparse memory(action=...)
Session -> Session : pin acting principal\nparse memory(action=...)
note right
Tool schema: 4 actions
save, search, delete, list
Tool schema: 5 actions
save, get, search, delete, list
Auto-approved (no approval needed)
end note
Session -> Session : resolve live project access\nselect exact/inherited scope
Session -> Session : _exec_memory(item)
alt action = save
Session -> Facade : save_structured_memory(\nname, content, description,\nmem_type, scope, scope_id)
Session -> Session : require non-empty description
Session -> Facade : save_structured_memory_strict(\n..., require_active_project)
Facade -> Facade : normalize_key(name)
Facade -> Storage : create_structured_memory()
alt unique constraint violation
Storage --> Facade : IntegrityError
Facade -> Storage : get_structured_memory_by_name()
Storage --> Facade : existing row
Facade -> Storage : update_structured_memory()
end
Storage --> Facade : memory_id
Facade --> Session : (memory_id, old_content)
Session -> Session : _init_system_messages()\nrefresh BM25 context
Facade -> Storage : guarded atomic upsert\nON CONFLICT ... RETURNING
Storage --> Facade : (saved row, was_update)
Facade --> Session : saved row
Session -> Session : invalidate prefix/cache\naudit acting principal
end
alt action = get
Session -> Facade : get_structured_memory_by_name_strict()
Facade -> Storage : exact scoped-name lookup
Storage --> Session : full row / not found
end
alt action = search
Session -> Facade : search_structured_memories(\nquery, mem_type, scope,\nscope_id, limit)
Facade -> Storage : search_structured_memories()
Session -> Storage : search exact scope or\nactor-visible scope union
Storage --> Session : matched rows
end
alt action = delete
Session -> Facade : delete_structured_memory(\nname, scope, scope_id)
Facade -> Storage : delete_structured_memory()
Storage --> Session : bool (existed)
Session -> Session : _init_system_messages()\nrefresh BM25 context
Session -> Facade : delete_structured_memory_returning_strict()
Facade -> Storage : DELETE ... RETURNING
Storage --> Session : deleted row / not found
Session -> Session : invalidate + audit\nmark prefix dirty
end
== Phase 2: BM25 Relevance Injection ==
Session -> Session : _init_system_messages()\nevery conversation turn
Session -> Session : resolve acting principal\nand live project ACL
Session -> Session : _list_visible_memories(\nlimit=fetch_limit)
note right
**Scope resolution:**
1. global scope (always)
2. workstream scope (ws_id)
3. user scope (user_id, if auth)
Combined and deduplicated.
Interactive: global + workstream
+ acting user + readable project
Coordinator: acting user's coordinator
+ readable project
end note
Session -> Facade : list_structured_memories()\nper scope
Facade -> Storage : list_structured_memories()
Session -> Facade : list_visible_structured_memories()
Facade -> Storage : one visibility-union query
Storage --> Session : up to fetch_limit rows
Session -> Relevance : extract_recent_context(\nmessages, max_messages=3)
@@ -103,28 +106,31 @@ Session -> Session : inject into\nsystem message
== Phase 3: Server API Path ==
SDK -> API : GET /v1/api/memories\n?type=project&limit=20
API -> Facade : list_structured_memories()
Facade -> Storage : list_structured_memories()
SDK -> API : GET /v1/api/memories\n?type=general&limit=20
API -> API : bind scope to caller\ndefault global + caller user
API -> Storage : list visible rows
Storage --> API : rows
API --> SDK : {"memories": [...], "total": N}
SDK -> API : POST /v1/api/memories\n{name, content, ...}
API -> API : validate type, scope,\nname length, content length
API -> Facade : save_structured_memory()
Facade -> Storage : create / update
SDK -> API : POST /v1/api/memories\n{name, content, description, ...}
API -> API : validate type, scope,\nname/content/description
API -> API : reject internal scopes\nowner-bind workstream scope
API -> Facade : save_structured_memory_strict()
Facade -> Storage : atomic upsert
Storage --> API : memory row
API -> API : record_audit(actor)
API --> SDK : 201 (created) / 200 (updated)
SDK -> API : POST /v1/api/memories/search\n{query, type, ...}
API -> Facade : search_structured_memories()
Facade -> Storage : search_structured_memories()
API -> API : bind scope to caller
API -> Storage : search visible rows
Storage --> API : matched rows
API --> SDK : {"memories": [...], "total": N}
SDK -> API : DELETE /v1/api/memories/{name}\n?scope=global
API -> Facade : delete_structured_memory()
Facade -> Storage : delete row
API -> Facade : delete_structured_memory_returning_strict()
Facade -> Storage : DELETE ... RETURNING
API -> API : record_audit(actor)
API --> SDK : {"status": "ok"}
== Phase 4: Console Admin Path ==
@@ -141,7 +147,7 @@ Storage --> Admin : memory row
Admin --> SDK : memory JSON
SDK -> Admin : DELETE /v1/api/admin/memories/{id}
Admin -> Storage : delete_structured_memory_by_id()
Admin -> Storage : delete_structured_memory_by_id_returning()
Admin -> Admin : record_audit(\n"memory.delete")
Admin --> SDK : {"status": "ok"}
+2 -2
View File
@@ -1,3 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:bd7fe8bf5c2b56b075453a316e54d61214b0ae912517d9cad6c1e88785aac722
size 300010
oid sha256:137d6c91a34695c820d8b0a33fd753e79165604aa92bf2ac8480d3744b2ef844
size 305199
+11 -8
View File
@@ -177,7 +177,8 @@ Runs a complete multi-turn conversation:
1. Appends the user message.
2. Checks `_cancelled` event — stops if set (timeout cleanup).
3. Calls the model API (non-streaming).
3. Calls the model through the production streaming provider path and drains
the result.
4. If tool calls are returned, executes them (with stdout suppressed) and
logs each call to `self.tool_call_log`.
5. Repeats up to `max_turns` or until the model responds without tool calls.
@@ -190,14 +191,15 @@ Parallel tool calls are capped at 10 per turn to prevent degenerate repetition.
Each test runs in a `ThreadPoolExecutor(max_workers=1)` with a per-test
timeout (`--test-timeout`). Each attempt gets its own `OpenAI` client with
a matching httpx read timeout. On timeout, three layers of defense prevent
zombie connections:
a matching per-read HTTP transport timeout. Because a trickling stream can
continually reset that read timeout, three layers bound the harness and stop
follow-on work:
1. **httpx timeout**: Per-request read timeout aborts the HTTP call and
releases the server slot.
1. **Executor wall clock**: The harness stops waiting after `--test-timeout`.
2. **`_cancelled` event**: Prevents the orphan thread from starting new turns.
3. **`run_client.close()`**: Closes the connection pool to abort any
in-flight request.
3. **`run_client.close()`**: Retires the connection pool and prevents reuse.
HTTPX2 does not promise that cross-thread client closure immediately aborts
an active body read; that read unwinds on its next wire event or read timeout.
### Retry Logic
@@ -214,7 +216,8 @@ Each test case runs in isolation:
1. A fresh temp directory is created.
2. Setup files are written to the temp directory.
3. The working directory is changed to the temp directory.
4. A per-attempt `OpenAI` client is created with httpx timeout matching `--test-timeout`.
4. A per-attempt `OpenAI` client is created with an HTTP transport timeout
matching `--test-timeout`.
5. A new `HeadlessSession` is created with the current developer prompt.
6. `send_headless()` runs the user prompt through the conversation loop.
7. The tool log is scored against expected actions.
+94 -42
View File
@@ -20,7 +20,7 @@ Each memory has three dimensions:
| Type | Purpose |
|-------------|------------------------------------------------------------|
| `user` | User preferences, conventions, working style |
| `project` | Project-specific knowledge, architecture, patterns |
| `general` | General knowledge, architecture, patterns |
| `feedback` | Corrections, lessons learned, things to avoid |
| `reference` | Reference material, documentation, specifications |
@@ -31,25 +31,37 @@ Each memory has three dimensions:
| `global` | Visible to all workstreams and users |
| `workstream` | Visible only within the originating workstream |
| `user` | Follows the authenticated user across workstreams |
| `coordinator` | Coordinator sessions only; follows the user across coordinators |
| `coordinator` | Coordinator sessions only; follows the acting user |
| `project` | Shared by workstreams attached to one active project |
A memory's identity is the tuple `(name, scope, scope_id)`. Saving a memory
with the same identity upserts -- updating content while preserving the ID.
### Coordinator scope
### Inherited target and coordinator scope
Coordinator sessions are isolated to a single scope: `coordinator`, keyed by
the coordinator's creator `user_id`. It is durable -- every coordinator
session the same user runs (including concurrent ones) shares one
orchestration namespace, so procedures and lessons survive close/reopen.
Name-based operations use one inherited target when `scope` is omitted:
- An attached active project selects `project` for `save`, `get`, and
`delete`.
- Read-only project access permits `get`, but `save` and `delete` fail. They do
not fall back to a broader namespace.
- Without a project, interactive sessions select `global`; coordinator
sessions select `coordinator`.
A valid explicit scope selects exactly that scope. `search` and `list` are the
only actions that span every visible scope when `scope` is omitted.
Each coordinator's private `coordinator` namespace is keyed by the acting
user's `user_id`. It is durable -- every coordinator session that user runs
(including concurrent ones) shares one orchestration namespace, so procedures
and lessons survive close/reopen.
Isolation is bidirectional and enforced by session kind, not by secrecy of
the scope id:
- A coordinator session can read and write **only** `coordinator`-scope rows.
It never sees `global`/`workstream`/`user` memories, so content written by
interactive sessions (which routinely ingest untrusted MCP/attachment
output) cannot reach a coordinator's system message.
- A coordinator session sees its acting user's `coordinator` scope and, when
attached, the shared `project` scope. It never sees
`global`/`workstream`/`user` memories.
- Interactive sessions -- including a coordinator's own children, which share
its `user_id` -- are rejected from the `coordinator` scope on every memory
action. Children cannot plant rows the parent coordinator would read.
@@ -64,12 +76,13 @@ coordinator cannot be constructed, so the scope id is always a real user.
On every conversation turn, the system:
1. Fetches up to `fetch_limit` memories visible in the current scope
2. Extracts context from the last 3 user messages
3. Scores memories against that context using a BM25 index
4. Injects the top `relevance_k` memories into the system message as
1. Resolves the acting principal and their live project access
2. Fetches up to `fetch_limit` memories across that visibility envelope
3. Extracts context from the last 3 user messages
4. Scores memories against that context using a BM25 index
5. Injects the top `relevance_k` memories into the system message as
`<memories>` XML tags
5. Appends a hint telling the model how many memories are in scope
6. Appends a hint telling the model how many memories are in scope
This means the model always has its most relevant memories available without
explicit recall -- but can still use `memory(action='search')` for deeper
@@ -107,19 +120,23 @@ All fields are optional. Defaults are shown above.
## Tool Usage
The `memory` tool supports four actions:
The `memory` tool supports five actions:
### save
Store or update a memory.
Every save is a complete write for the relevance summary: `description` must
be supplied and contain non-whitespace text on both creation and update.
Content-only updates are rejected.
```json
{
"action": "save",
"name": "project_architecture",
"content": "The project uses a hexagonal architecture with...",
"description": "Core architecture patterns",
"type": "project",
"type": "general",
"scope": "global"
}
```
@@ -128,9 +145,26 @@ Store or update a memory.
|---------------|----------|-------------|------------------------------------------|
| `name` | yes | -- | Snake_case identifier (max 256 chars) |
| `content` | yes | -- | Memory content (max `max_content` chars) |
| `description` | no | `""` | Short description for relevance matching |
| `type` | no | `"project"` | One of: user, project, feedback, reference |
| `scope` | no | `"global"` | One of: global, workstream, user |
| `description` | yes | -- | Non-empty relevance summary, required on create and update |
| `type` | no | `"general"` | One of: user, general, feedback, reference |
| `scope` | no | inherited | Kind-valid scope; see inherited target above |
### get
Retrieve the full content of one memory by name.
```json
{
"action": "get",
"name": "project_architecture",
"scope": "project"
}
```
| Parameter | Required | Default | Description |
|-----------|----------|-----------|----------------------------|
| `name` | yes | -- | Memory name to retrieve |
| `scope` | no | inherited | Exact scope to query |
### search
@@ -140,7 +174,7 @@ Find memories by query (BM25 full-text search).
{
"action": "search",
"query": "authentication patterns",
"type": "project",
"type": "general",
"limit": 10
}
```
@@ -167,7 +201,7 @@ Remove a memory by name.
| Parameter | Required | Default | Description |
|------------|----------|------------|--------------------------|
| `name` | yes | -- | Memory name to delete |
| `scope` | no | `"global"` | Scope of the memory |
| `scope` | no | inherited | Exact scope to delete |
### list
@@ -197,6 +231,12 @@ Four endpoints on the server for programmatic memory access.
List memories with optional filters.
Without `scope`, the response is restricted to `global` plus the authenticated
caller's `user` namespace. The public API accepts only `global`, `user`, and
`workstream`; internal `project` and `coordinator` namespaces remain available
through the session tool and admin API. Explicit `workstream` access requires
its persisted owner (or a service token).
**Query parameters:**
| Parameter | Type | Required | Default | Description |
@@ -204,10 +244,10 @@ List memories with optional filters.
| `type` | string | no | `""` | Filter by memory type |
| `scope` | string | no | `""` | Filter by scope |
| `scope_id` | string | no | `""` | Filter by scope ID |
| `limit` | int | no | `100` | Max results (capped at 200) |
| `limit` | int | no | `100` | Max results (1-200) |
When `scope=user` and `scope_id` is omitted, the authenticated user's ID is
used automatically.
When `scope=user`, the authenticated user's ID is used automatically and a
different supplied ID is rejected. `scope=workstream` requires `scope_id`.
**Response:** `200`
@@ -218,7 +258,7 @@ used automatically.
"memory_id": "a1b2c3d4-e5f6-...",
"name": "project_architecture",
"description": "Core architecture patterns",
"type": "project",
"type": "general",
"scope": "global",
"scope_id": "",
"content": "The project uses a hexagonal architecture...",
@@ -236,6 +276,9 @@ used automatically.
Save or upsert a structured memory.
`description` is mandatory for both creates and updates and must contain
non-whitespace text. The API rejects content-only updates.
**Request body:**
```json
@@ -243,7 +286,7 @@ Save or upsert a structured memory.
"name": "deployment_process",
"content": "Deploy via GitHub Actions. Staging auto-deploys on push to main.",
"description": "CI/CD deployment workflow",
"type": "project",
"type": "general",
"scope": "global",
"scope_id": ""
}
@@ -253,8 +296,8 @@ Save or upsert a structured memory.
|--------------|--------|----------|-------------|--------------------------------------|
| `name` | string | yes | -- | Memory name (max 256 chars) |
| `content` | string | yes | -- | Memory content (max 65536 chars) |
| `description`| string | no | `""` | Short description for search ranking |
| `type` | string | no | `"project"` | One of: user, project, feedback, reference |
| `description`| string | yes | -- | Non-empty relevance summary, required on create and update |
| `type` | string | no | unset | user, general, feedback, or reference |
| `scope` | string | no | `"global"` | One of: global, workstream, user |
| `scope_id` | string | no | `""` | Scope qualifier (auto-resolved for user scope) |
@@ -265,7 +308,7 @@ Save or upsert a structured memory.
"memory_id": "a1b2c3d4-e5f6-...",
"name": "deployment_process",
"description": "CI/CD deployment workflow",
"type": "project",
"type": "general",
"scope": "global",
"scope_id": "",
"content": "Deploy via GitHub Actions...",
@@ -281,7 +324,10 @@ same `(name, scope, scope_id)` already existed.
| Status | Condition |
|--------|------------------------------------|
| 400 | Missing name, empty content, invalid type/scope, content too long |
| 400 | Invalid input, scope, scope ID, or limit |
| 403 | Cross-user or non-owner workstream access |
| 404 | Explicit workstream does not exist |
| 500 | Storage mutation failed |
---
@@ -290,12 +336,15 @@ same `(name, scope, scope_id)` already existed.
Search memories by query. Uses POST for the request body but is non-mutating
(requires only `read` scope).
An omitted scope searches the same caller-bound `global` + `user` envelope as
the list endpoint. It never means every row in the table.
**Request body:**
```json
{
"query": "authentication",
"type": "project",
"type": "general",
"scope": "",
"scope_id": "",
"limit": 20
@@ -308,7 +357,7 @@ Search memories by query. Uses POST for the request body but is non-mutating
| `type` | string | no | `""` | Filter by type |
| `scope` | string | no | `""` | Filter by scope |
| `scope_id` | string | no | `""` | Filter by scope ID |
| `limit` | int | no | `20` | Max results (capped at 50) |
| `limit` | int | no | `20` | Max results (1-50) |
**Response:** `200`
@@ -319,7 +368,7 @@ Search memories by query. Uses POST for the request body but is non-mutating
"memory_id": "a1b2c3d4-e5f6-...",
"name": "auth_patterns",
"description": "Authentication architecture",
"type": "project",
"type": "general",
"scope": "global",
"scope_id": "",
"content": "JWT tokens with HS256...",
@@ -337,6 +386,9 @@ Search memories by query. Uses POST for the request body but is non-mutating
Delete a memory by name and scope.
Deletes are atomic: the row used for the success result and audit event is the
row actually removed. A storage failure returns `500`, not a false `404`.
**Path parameters:**
| Parameter | Type | Description |
@@ -391,7 +443,7 @@ List memories across all scopes (no automatic scope resolution).
"memory_id": "a1b2c3d4-e5f6-...",
"name": "project_architecture",
"description": "Core architecture patterns",
"type": "project",
"type": "general",
"scope": "global",
"scope_id": "",
"content": "The project uses...",
@@ -440,7 +492,7 @@ Get a single memory by ID.
"memory_id": "a1b2c3d4-e5f6-...",
"name": "project_architecture",
"description": "Core architecture patterns",
"type": "project",
"type": "general",
"scope": "global",
"scope_id": "",
"content": "The project uses...",
@@ -497,13 +549,13 @@ with TurnstoneServer("http://localhost:8080", token="tok_xxx") as client:
"api_conventions",
"All endpoints use /v1/ prefix. JSON responses.",
description="API design patterns",
mem_type="project",
mem_type="general",
scope="global",
)
print(mem.memory_id)
# Search memories
results = client.search_memories("authentication", mem_type="project", limit=10)
results = client.search_memories("authentication", mem_type="general", limit=10)
for m in results.memories:
print(f"{m['name']}: {m['description']}")
@@ -524,7 +576,7 @@ with TurnstoneConsole("http://localhost:9090", token="tok_xxx") as admin:
result = admin.list_memories(scope="global", limit=100)
# Search
result = admin.search_memories("architecture", mem_type="project")
result = admin.search_memories("architecture", mem_type="general")
# Get by ID
mem = admin.get_memory("a1b2c3d4-e5f6-...")
@@ -548,14 +600,14 @@ const mem = await client.saveMemory({
name: "api_conventions",
content: "All endpoints use /v1/ prefix. JSON responses.",
description: "API design patterns",
type: "project",
type: "general",
scope: "global",
});
// Search memories
const results = await client.searchMemories({
query: "authentication",
type: "project",
type: "general",
limit: 10,
});
+25 -10
View File
@@ -152,7 +152,7 @@ into its successor's trajectory.
**Auto-approved** (no user confirmation needed at runtime):
- `read_file` -- reads files, no side effects
- `search` -- grep-style search, no side effects
- `memory` -- structured persistent memory (save/search/delete/list)
- `memory` -- structured persistent memory (save/get/search/delete/list)
- `recall` -- searches conversation history
- `notify` -- sends notifications to linked channels (time-sensitive, auto-approved for urgency)
@@ -433,7 +433,7 @@ Delegate a general-purpose task to an autonomous sub-agent.
|-----------|--------|----------|-------------|
| `prompt` | string | yes | Complete task description for the sub-agent. |
- **What it does**: Spawns a sub-agent that inherits the `TASK_AGENT_TOOLS` set (read, write, edit, search, bash, web tools, memory tools). The sub-agent runs autonomously to completion. Use for work that requires file modifications or command execution.
- **What it does**: Spawns a sub-agent that inherits the `TASK_AGENT_TOOLS` set (read, write, edit, search, bash, and web tools). The sub-agent runs autonomously to completion. Use for work that requires file modifications or command execution.
- **Auto-approve**: No -- requires user confirmation.
- **Agent availability**: Top-level only.
@@ -447,18 +447,26 @@ Structured persistent memory across sessions with typed, scoped entries.
| Parameter | Type | Required | Description |
|---------------|---------|----------|-------------|
| `action` | string | yes | `save`, `search`, `delete`, or `list`. |
| `name` | string | save/delete | Short snake_case identifier for the memory. |
| `action` | string | yes | `save`, `get`, `search`, `delete`, or `list`. |
| `name` | string | save/get/delete | Short snake_case identifier for the memory. |
| `content` | string | save | Memory content to store. |
| `description` | string | no | Short description for relevance matching (recommended for `save`). |
| `type` | string | no | Memory type: `user`, `project`, `feedback`, or `reference`. Default: `project`. |
| `scope` | string | no | Memory scope: `global`, `workstream`, or `user`. Default: `global`. |
| `description` | string | save | Non-empty description for relevance matching; required on create and update. |
| `type` | string | no | Memory type: `user`, `general`, `feedback`, or `reference`. Default: `general`. |
| `scope` | string | no | Memory scope: `global`, `workstream`, `user`, `coordinator`, or `project`. See defaults below. |
| `query` | string | search | Search query for finding memories. |
| `limit` | integer | no | Max results for `search` or `list`. Default: 20. |
- **What it does**: Manages structured persistent memories in the database. Memories persist across sessions, have a type classification (user preferences, project knowledge, feedback, reference material) and a scope (global across all workstreams, private to a workstream, or following a user). Relevant memories are included in the system prompt on startup.
- **What it does**: Manages structured persistent memories in the database.
Memories persist across sessions, have a type classification, and live in a
role-specific visible scope. Unscoped `save`/`get`/`delete` resolve to one
target: the attached active project, otherwise `global` for an interactive
session or `coordinator` for a coordinator. Read-only project access permits
`get` but makes `save`/`delete` fail without falling back. A valid explicit
scope selects exactly that scope. Unscoped `search`/`list` cover all visible
scopes; use the displayed scope when following a result with `get` or
`delete`.
- **Auto-approve**: Yes.
- **Agent availability**: Not available to sub-agents (top-level only).
- **Agent availability**: Not available to task agents.
---
@@ -473,7 +481,7 @@ Search conversation history for past messages and tool results.
- **What it does**: Searches conversation history across sessions using FTS5 full-text search. Returns matching messages, tool calls, and tool results with timestamps and workstream context.
- **Auto-approve**: Yes.
- **Agent availability**: Not available to sub-agents (top-level only).
- **Agent availability**: Not available to task agents.
---
@@ -871,6 +879,13 @@ capabilities for the `resources` capability. For servers that declare it:
2. `list_resource_templates` fetches URI templates (parameterized patterns like
`db://tables/{table}/rows/{id}`).
The protocol advertises both lists through one aggregate `resources`
capability, so a server may implement only one of them. If either request
returns the JSON-RPC `Method not found` code (`-32601`), turnstone treats that
half of the catalog as empty and keeps the other half; authentication,
validation, transport, and all other discovery errors still fail the
connection or refresh.
Both are stored as `{uri, name, description, mimeType, server}` dicts and
merged into a unified catalog.
+11 -1
View File
@@ -23,9 +23,14 @@ classifiers = [
"Topic :: Scientific/Engineering :: Artificial Intelligence",
]
dependencies = [
"openai>=2.45", # GPT-5.6: typed reasoning.mode, prompt_cache_options, and cache_write_tokens
# Version 3 moves the default transport to HTTPX2. Keep major upgrades
# deliberate because the stream retry boundary depends on that contract.
"openai>=3,<4",
"anthropic>=0.117", # tracks the release current at claude-opus-5 onboarding; hard runtime floor is still 0.105 (mid-conversation system blocks) — Opus 5 itself needs no new SDK surface (model ids are opaque strings; "refusal" has been in the StopReason literal since ~0.95). Raise this when adopting fast mode / server-side fallbacks / advisor / mid-conversation tool changes, which DO need newer typed params.
"httpx>=0.28",
# Direct because the provider boundary catches this exception family;
# OpenAI v3's transitive dependency alone is not an import contract.
"httpx2>=2.7,<3",
"mcp>=1.27,<2", # v2 is a breaking rewrite (2.0.0a1 live 2026-06-11; stable ~2026-07-27) — streamablehttp_client removed, 2-tuple transport, snake_case types; migrate deliberately
"starlette>=1.3.1", # CVE-2026-54282 (path->authority host spoof) + CVE-2026-54283 (url-encoded form DoS); supersedes the PYSEC-2026-161 host-header path-injection floor
"uvicorn>=0.34",
@@ -106,6 +111,11 @@ markers = [
"e2e_recovery: opt-in end-to-end SSE recovery harness (real server + real SSE consumers, scripted provider — NOT live, no LLM backend needed); tens of seconds each. CI lanes run ``-m 'not live and not e2e_recovery'``; select with ``-m e2e_recovery``.",
]
filterwarnings = [
# The MCP v1 FastMCP integration fixture rebuilds its incomplete generic
# Settings model before construction. Keep the new pydantic-settings 2.15
# diagnostic fatal so removing that compatibility step cannot regress
# into a warning hidden in the full-suite summary.
"error:Field 'lifespan' has an incomplete definition",
# mcp v1 deprecates streamablehttp_client for an entry point whose call
# shape only settles in v2 — adoption rides the deliberate v2 migration
# (pin capped <2); silence exactly this message until then.
+458
View File
@@ -0,0 +1,458 @@
#!/usr/bin/env python3
"""Browser regression for the two-layer frontend cache contract.
This harness serves a versioned entry module which imports the real,
unversioned ``shared/interactive.js`` module. It loads an old pane build in a
real Chrome profile, switches the server to a same-version replacement whose
pane has the same byte length and mtime, then performs a normal browser reload
without clearing or disabling the HTTP cache.
The old fixture advertises ``user_turn=0&tool_turn=0`` in its EventSource URL;
the current source advertises both capabilities as ``1``. A passing run
therefore proves both layers of the contract:
* the package-versioned entry URL revalidates and may return 304; and
* its unversioned transitive pane import revalidates by content and returns the
current bytes rather than surviving from the prior build.
The page also loads representative KaTeX, Highlight.js, and HLS.js assets.
Their installed version directories are discovered from ``shared_static`` at
runtime, so a normal vendor-version bump requires no harness edit; reload must
reuse them under the immutable policy.
Usage::
uv run python scripts/asset_cache_e2e.py
"""
from __future__ import annotations
import contextlib
import json
import os
import shutil
import socket
import sys
import tempfile
import threading
import time
from dataclasses import dataclass, field
from pathlib import Path
from typing import Any
ROOT = Path(__file__).resolve().parents[1]
if str(ROOT) not in sys.path:
sys.path.insert(0, str(ROOT))
from recovery_e2e import CDP, _find_chrome, _launch_chrome, _page_ws_url # noqa: E402
from turnstone import __version__ # noqa: E402
from turnstone.core.web_helpers import ( # noqa: E402
RevalidatingStaticFiles,
version_html,
)
_PAGE_HTML = """<!doctype html>
<html>
<head>
<meta charset="utf-8">
<title>ASSET-CACHE-PENDING</title>
<!-- VENDORED_ASSET_TAGS -->
<script>
window.__assetCacheUrls = [];
class RecordingEventSource {
static CONNECTING = 0;
static OPEN = 1;
static CLOSED = 2;
constructor(url) {
this.url = String(url);
this.readyState = RecordingEventSource.CONNECTING;
window.__assetCacheUrls.push(this.url);
}
close() {
this.readyState = RecordingEventSource.CLOSED;
}
}
window.EventSource = RecordingEventSource;
window.addEventListener("error", (event) => {
document.title = "ASSET-CACHE-FAILED-" + String(event.message || "script").slice(0, 80);
});
window.addEventListener("unhandledrejection", (event) => {
document.title = "ASSET-CACHE-FAILED-" + String(event.reason || "promise").slice(0, 80);
});
</script>
</head>
<body>
<main id="pane"></main>
<script type="module" src="/static/asset_cache_boot.js"
onerror="document.title='ASSET-CACHE-FAILED-module-load'"></script>
</body>
</html>
"""
_BOOT_JS = """import { InteractivePane } from "/shared/interactive.js";
const fixtureWorkstreamId = "00000000-0000-0000-0000-000000000001";
const pane = new InteractivePane(fixtureWorkstreamId, { base: "" });
document.getElementById("pane").appendChild(pane.el);
pane.connectSSE(fixtureWorkstreamId);
const url = window.__assetCacheUrls.at(-1) || "";
const generation = url.includes("user_turn=1") && url.includes("tool_turn=1")
? "CURRENT"
: url.includes("user_turn=0") && url.includes("tool_turn=0")
? "OLD"
: "FAILED-CAPABILITIES";
window.__assetCacheResult = { generation, url };
document.title = "ASSET-CACHE-" + generation;
"""
@dataclass
class CacheState:
phase: str = "old"
requests: list[dict[str, Any]] = field(default_factory=list)
lock: threading.Lock = field(default_factory=threading.Lock)
def record(self, item: dict[str, Any]) -> None:
with self.lock:
self.requests.append(item)
def matching(self, phase: str, path: str) -> list[dict[str, Any]]:
with self.lock:
return [
item for item in self.requests if item["phase"] == phase and item["path"] == path
]
class SwitchingStaticFiles:
"""Select one immutable build snapshot at request dispatch time."""
def __init__(self, state: CacheState, old_dir: Path, current_dir: Path) -> None:
self._state = state
self._apps = {
"old": RevalidatingStaticFiles(directory=str(old_dir)),
"current": RevalidatingStaticFiles(directory=str(current_dir)),
}
async def __call__(self, scope: Any, receive: Any, send: Any) -> None:
await self._apps[self._state.phase](scope, receive, send)
class RecordingApp:
"""Record static HTTP validators and status without perturbing streaming."""
def __init__(self, app: Any, state: CacheState) -> None:
self._app = app
self._state = state
async def __call__(self, scope: Any, receive: Any, send: Any) -> None:
path = str(scope.get("path", ""))
if scope.get("type") != "http" or not path.startswith(("/static/", "/shared/")):
await self._app(scope, receive, send)
return
phase = self._state.phase
request_headers = {
key.decode("latin-1").lower(): value.decode("latin-1")
for key, value in scope.get("headers", [])
}
async def record_send(message: dict[str, Any]) -> None:
if message["type"] == "http.response.start":
response_headers = {
key.decode("latin-1").lower(): value.decode("latin-1")
for key, value in message.get("headers", [])
}
self._state.record(
{
"phase": phase,
"path": path,
"query": scope.get("query_string", b"").decode("latin-1"),
"if_none_match": request_headers.get("if-none-match"),
"status": message["status"],
"etag": response_headers.get("etag"),
"cache_control": response_headers.get("cache-control"),
}
)
await send(message)
await self._app(scope, receive, record_send)
def _old_interactive(current: bytes) -> bytes:
old = current
for needle, replacement in (
(b'"user_turn=1"', b'"user_turn=0"'),
(b'"&tool_turn=1"', b'"&tool_turn=0"'),
):
if old.count(needle) != 1:
raise RuntimeError(f"expected exactly one {needle.decode()} capability literal")
old = old.replace(needle, replacement, 1)
if len(old) != len(current):
raise AssertionError("old and current pane fixtures must have equal byte length")
return old
def _discover_vendor_assets(source_shared: Path) -> tuple[str, ...]:
selected = (
("katex", "katex.min.css"),
("katex", "katex.min.js"),
("hljs", "highlight.min.js"),
("hls", "hls.min.js"),
)
paths = []
for library, filename in selected:
matches = sorted(source_shared.glob(f"{library}-*/{filename}"))
if not matches:
raise RuntimeError(f"no vendored {library} asset named {filename} was found")
paths.extend(f"/shared/{match.relative_to(source_shared).as_posix()}" for match in matches)
return tuple(paths)
def _vendor_tags(vendor_paths: tuple[str, ...]) -> str:
tags = []
for path in vendor_paths:
if path.endswith(".css"):
tags.append(f'<link rel="stylesheet" href="{path}">')
else:
tags.append(f'<script src="{path}"></script>')
return "\n ".join(tags)
def _prepare_builds(scratch: Path) -> tuple[Path, Path, Path, tuple[str, ...]]:
import turnstone
package_dir = Path(turnstone.__file__).resolve().parent
source_shared = package_dir / "shared_static"
vendor_paths = _discover_vendor_assets(source_shared)
old_shared = scratch / "old" / "shared"
current_shared = scratch / "current" / "shared"
static_dir = scratch / "static"
shutil.copytree(source_shared, old_shared)
shutil.copytree(source_shared, current_shared)
static_dir.mkdir()
(static_dir / "asset_cache_boot.js").write_text(_BOOT_JS, encoding="utf-8")
current_asset = current_shared / "interactive.js"
old_asset = old_shared / "interactive.js"
current = current_asset.read_bytes()
old_asset.write_bytes(_old_interactive(current))
# Reproduce the metadata collision which defeated Starlette's default ETag.
fixed_mtime_ns = 1_700_000_000_123_456_789
for asset in (old_asset, current_asset):
os.utime(asset, ns=(fixed_mtime_ns, fixed_mtime_ns))
old_stat = old_asset.stat()
current_stat = current_asset.stat()
if (old_stat.st_size, old_stat.st_mtime_ns) != (
current_stat.st_size,
current_stat.st_mtime_ns,
):
raise AssertionError("pane fixture size/mtime collision was not preserved")
return old_shared, current_shared, static_dir, vendor_paths
def _make_app(
state: CacheState,
old_dir: Path,
current_dir: Path,
static_dir: Path,
vendor_paths: tuple[str, ...],
) -> Any:
from starlette.applications import Starlette
from starlette.responses import HTMLResponse
from starlette.routing import Mount, Route
async def page(_request: Any) -> HTMLResponse:
page_html = _PAGE_HTML.replace("<!-- VENDORED_ASSET_TAGS -->", _vendor_tags(vendor_paths))
return HTMLResponse(
version_html(page_html),
headers={"Cache-Control": "no-store"},
)
app = Starlette(
routes=[
Route("/asset-cache-e2e", page),
Mount(
"/static",
app=RevalidatingStaticFiles(directory=str(static_dir)),
),
Mount(
"/shared",
app=SwitchingStaticFiles(state, old_dir, current_dir),
),
]
)
return RecordingApp(app, state)
def _start_server(app: Any) -> tuple[Any, threading.Thread, socket.socket, str]:
import uvicorn
sock = socket.socket()
sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
sock.bind(("127.0.0.1", 0))
sock.listen(128)
port = int(sock.getsockname()[1])
server = uvicorn.Server(
uvicorn.Config(app, host="127.0.0.1", port=port, log_level="warning", lifespan="off")
)
thread = threading.Thread(
target=server.run,
kwargs={"sockets": [sock]},
name="asset-cache-e2e-server",
daemon=True,
)
thread.start()
deadline = time.monotonic() + 10
while not server.started and thread.is_alive() and time.monotonic() < deadline:
time.sleep(0.05)
if not server.started:
server.should_exit = True
thread.join(timeout=2)
sock.close()
raise RuntimeError("asset cache test server did not start")
return server, thread, sock, f"http://127.0.0.1:{port}"
def _wait_for_generation(cdp: CDP, expected: str, timeout: float = 20) -> dict[str, str]:
deadline = time.monotonic() + timeout
last_title = ""
while time.monotonic() < deadline:
last_title = cdp.title()
result = cdp.evaluate("window.__assetCacheResult || null")
if isinstance(result, dict) and result.get("generation") == expected:
return {"generation": str(result["generation"]), "url": str(result["url"])}
if last_title.startswith("ASSET-CACHE-FAILED"):
raise RuntimeError(last_title)
time.sleep(0.1)
raise TimeoutError(f"expected {expected}, last title was {last_title!r}")
def _one(state: CacheState, phase: str, path: str) -> dict[str, Any]:
requests = state.matching(phase, path)
if len(requests) != 1:
raise AssertionError(f"expected one {phase} request for {path}, got {requests!r}")
return requests[0]
def _verify_trace(state: CacheState, vendor_paths: tuple[str, ...]) -> tuple[str, list[str]]:
entry_path = "/static/asset_cache_boot.js"
pane_path = "/shared/interactive.js"
old_entry = _one(state, "old", entry_path)
current_entry = _one(state, "current", entry_path)
old_pane = _one(state, "old", pane_path)
current_pane = _one(state, "current", pane_path)
expected_query = f"v={__version__}"
if old_entry["query"] != expected_query or current_entry["query"] != expected_query:
raise AssertionError("entry URL did not retain the same package version across builds")
if old_entry["status"] != 200 or old_pane["status"] != 200:
raise AssertionError("old build did not populate the browser cache")
if current_entry["status"] != 304 or not current_entry["if_none_match"]:
raise AssertionError(f"versioned entry did not revalidate to 304: {current_entry!r}")
if current_pane["status"] != 200 or not current_pane["if_none_match"]:
raise AssertionError(
f"transitive pane did not revalidate to current bytes: {current_pane!r}"
)
if old_pane["etag"] == current_pane["etag"]:
raise AssertionError("content-derived pane validators did not change")
if current_pane["cache_control"] != "no-cache":
raise AssertionError("transitive pane lost its revalidation policy")
vendor_trace = []
immutable = "public, max-age=31536000, immutable"
for path in vendor_paths:
old_vendor = _one(state, "old", path)
if old_vendor["query"] or old_vendor["status"] != 200:
raise AssertionError(f"versioned vendor URL was rewritten or failed: {old_vendor!r}")
if old_vendor["cache_control"] != immutable:
raise AssertionError(f"versioned vendor asset was not immutable: {old_vendor!r}")
revisits = state.matching("current", path)
if revisits:
if len(revisits) != 1 or revisits[0]["status"] not in (200, 304):
raise AssertionError(f"unexpected vendor reload trace: {revisits!r}")
if revisits[0]["cache_control"] != immutable:
raise AssertionError(f"vendor reload lost immutable policy: {revisits[0]!r}")
vendor_trace.append(f"{path}: revisited-{revisits[0]['status']}")
else:
vendor_trace.append(f"{path}: cache-hit")
verdict = f"ASSET-CACHE-READY-entry304-pane200-user1-tool1-vendor{len(vendor_paths)}"
return verdict, vendor_trace
def _stop_process(proc: Any) -> None:
if proc.poll() is not None:
return
proc.terminate()
with contextlib.suppress(Exception):
proc.wait(timeout=5)
if proc.poll() is None:
proc.kill()
with contextlib.suppress(Exception):
proc.wait(timeout=2)
def main() -> int:
chrome = _find_chrome()
if not chrome:
print("ASSET-CACHE-FAILED-no-chrome")
return 2
with tempfile.TemporaryDirectory(prefix="turnstone-asset-cache-e2e-") as raw_scratch:
scratch = Path(raw_scratch)
old_dir, current_dir, static_dir, vendor_paths = _prepare_builds(scratch)
state = CacheState()
server, server_thread, sock, base_url = _start_server(
_make_app(state, old_dir, current_dir, static_dir, vendor_paths)
)
chrome_proc = None
cdp = None
try:
chrome_proc, cdp_port = _launch_chrome(chrome, scratch / "chrome-profile")
cdp = CDP(_page_ws_url(cdp_port))
cdp.cmd("Page.enable")
cdp.cmd("Runtime.enable")
cdp.cmd("Network.enable")
cdp.cmd("Page.navigate", {"url": f"{base_url}/asset-cache-e2e"})
old_result = _wait_for_generation(cdp, "OLD")
state.phase = "current"
cdp.cmd("Page.reload", {"ignoreCache": False})
current_result = _wait_for_generation(cdp, "CURRENT")
if "user_turn=0" not in old_result["url"] or "tool_turn=0" not in old_result["url"]:
raise AssertionError(f"old pane did not expose old capabilities: {old_result!r}")
if (
"user_turn=1" not in current_result["url"]
or "tool_turn=1" not in current_result["url"]
):
raise AssertionError(
f"reloaded pane did not expose current capabilities: {current_result!r}"
)
verdict, vendor_trace = _verify_trace(state, vendor_paths)
cdp.evaluate(f"document.title = {json.dumps(verdict)}")
print(verdict)
print(f" old EventSource: {old_result['url']}")
print(f" current EventSource: {current_result['url']}")
for item in vendor_trace:
print(f" vendor: {item}")
return 0
except Exception as exc:
print(f"ASSET-CACHE-FAILED-{type(exc).__name__}: {exc}")
return 1
finally:
if cdp is not None:
cdp.close()
if chrome_proc is not None:
_stop_process(chrome_proc)
server.should_exit = True
server_thread.join(timeout=10)
with contextlib.suppress(OSError):
sock.close()
if __name__ == "__main__":
raise SystemExit(main())
+189 -2
View File
@@ -74,6 +74,21 @@ Attachments harness (/attachments/livepass.html): the composer attachment
thumbnail crop/size, the native audio-control fit at the constrained
height, the snippet contrast, and how a long filename behaves at the
340px chip cap.
Paste-over-HTTP harness (/paste/livepass.html): the REAL Composer's
large-text paste path, exercised with a browser-generated, trusted paste
event on an explicitly non-secure HTTP origin. No
``navigator.clipboard`` stub, synthetic ``ClipboardEvent``, or
secure-context override is involved. The page requires
``window.isSecureContext === false``, ``event.isTrusted``, a ``text/plain``
clipboard item, canceled inline insertion, and an exact
``pasted-text.txt`` File round-trip before stamping ``PASTE-HTTP-READY``.
It fails closed as ``PASTE-HTTP-FAILED-<reason>``. Serve beyond loopback,
open the page by a LAN address (localhost and 127.0.0.1 are treated as
trustworthy origins by browsers), then use the browser's normal Copy and
Paste commands:
python3 scripts/livepass.py --serve 8950 --bind 0.0.0.0
Task-agent harness (/taskagent/livepass.html): the task_agent card a task
agent's sub-tool steps nested under its conversation row, driven through the
REAL InteractivePane.handleEvent (parent tool_pending/tool_info -> child
@@ -1031,6 +1046,165 @@ ATTACH_TEMPLATE = """<!doctype html>
"""
# --------------------------------------------------------------------------
# Native paste-over-HTTP harness. Unlike the copy-affordance harness, this
# must never stub clipboard access or dispatch a script-created paste event:
# its purpose is to prove that the production ClipboardEvent path still sees
# user-agent clipboard data on a non-secure origin.
# --------------------------------------------------------------------------
PASTE_TEMPLATE = """<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
<title>PASTE-HTTP-BOOTING</title>
<link rel="stylesheet" href="shared/base.css" />
<link rel="stylesheet" href="shared/ui-base.css" />
<link rel="stylesheet" href="shared/chat.css" />
<style>
body {
margin: 0; padding: 32px;
background: var(--bg); color: var(--fg);
font-family: var(--font-sans, system-ui, sans-serif);
}
main { width: min(760px, 100%); margin: 0 auto; }
#paste-source {
box-sizing: border-box; width: 100%; height: 80px;
}
#composer-mount, #probe-status { margin-top: 16px; }
#probe-status { white-space: pre-wrap; }
</style>
</head>
<body>
<main>
<h1>Native paste on plain HTTP</h1>
<p>
This passes only when a trusted paste exposes clipboard text to the
real Composer on a non-secure HTTP origin.
</p>
<p>1. Select this 2001-character fixture, then copy it normally.</p>
<textarea id="paste-source" aria-label="Paste test source"></textarea>
<button id="select-source" type="button">Select source text</button>
<p>2. Focus the composer and paste normally.</p>
<div id="composer-mount"></div>
<pre id="probe-status" role="status" aria-live="polite">Booting</pre>
</main>
<script type="module">
import { Composer } from "./shared/composer.js";
import { PASTE_ATTACHMENT_CHARS } from "./shared/composer_paste_text.js";
const prefix = "TURNSTONE-PASTE-HTTP:";
const seedText =
prefix + "x".repeat(PASTE_ATTACHMENT_CHARS + 1 - prefix.length);
const source = document.getElementById("paste-source");
const status = document.getElementById("probe-status");
source.value = seedText;
let copyTrusted = false;
let paste = null;
let attachment = null;
let fileSettled = false;
let failed = false;
function paint() {
status.textContent = document.title + "\\n" + JSON.stringify({
origin: location.origin,
secureContext: window.isSecureContext,
copyTrusted: copyTrusted,
paste: paste,
attachment: attachment,
}, null, 2);
}
function fail(reason) {
if (failed) return;
failed = true;
document.title = "PASTE-HTTP-FAILED-" + reason;
paint();
}
function finish() {
if (failed || !paste || !fileSettled) return;
const checks = [
["copy-untrusted", copyTrusted],
["paste-untrusted", paste.trusted],
["no-clipboard-data", paste.hasClipboardData],
["no-text-plain", paste.hasPlainText],
["paste-not-canceled", paste.defaultPrevented],
["attach-count", attachment.calls === 1],
["filename", attachment.name === "pasted-text.txt"],
["mime", attachment.type === "text/plain"],
["size", attachment.size === new Blob([seedText]).size],
["content", attachment.contentMatches],
["inline-insert", paste.inputValue === ""],
];
const firstFailure = checks.find(function (entry) { return !entry[1]; });
if (firstFailure) return fail(firstFailure[0]);
document.title = "PASTE-HTTP-READY";
paint();
}
document.addEventListener("copy", function (event) {
copyTrusted = event.isTrusted;
paint();
}, true);
document.addEventListener("paste", function (event) {
const observed = {
trusted: event.isTrusted,
hasClipboardData: !!event.clipboardData,
hasPlainText: !!event.clipboardData &&
Array.from(event.clipboardData.types || []).includes("text/plain"),
};
setTimeout(function () {
paste = Object.assign(observed, {
defaultPrevented: event.defaultPrevented,
inputValue: composer.inputEl.value,
});
if (!attachment) fail("no-attachment");
else finish();
}, 0);
});
const composer = new Composer(document.getElementById("composer-mount"), {
onSend: function () {},
placeholder: "Paste the copied fixture here…",
attachments: {
onAttach: function (file) {
attachment = {
calls: attachment ? attachment.calls + 1 : 1,
name: file.name,
type: file.type,
size: file.size,
contentMatches: false,
};
file.text().then(function (text) {
attachment.contentMatches = text === seedText;
fileSettled = true;
finish();
}, function () { fail("file-read"); });
return true;
},
},
});
document.getElementById("select-source").addEventListener("click", function () {
source.focus();
source.select();
});
if (location.protocol !== "http:") fail("not-http");
else if (window.isSecureContext) fail("secure-context");
else {
document.title = "PASTE-HTTP-WAITING";
paint();
}
</script>
</body>
</html>
"""
# --------------------------------------------------------------------------
# Task-agent harness — the task_agent card: a task agent's sub-tool steps
# nested under its conversation row. Driven through the REAL
@@ -1958,6 +2132,12 @@ def build(out: Path) -> None:
(att / "livepass.html").write_text(ATTACH_TEMPLATE, encoding="utf-8")
print(f"{att}/livepass.html — composer chips + message attachment pills")
paste = out / "paste"
paste.mkdir(parents=True, exist_ok=True)
symlink(paste / "shared", ROOT / "turnstone/shared_static")
(paste / "livepass.html").write_text(PASTE_TEMPLATE, encoding="utf-8")
print(f"{paste}/livepass.html — trusted native paste on insecure HTTP")
ta = out / "taskagent"
ta.mkdir(parents=True, exist_ok=True)
symlink(ta / "shared", ROOT / "turnstone/shared_static")
@@ -2218,6 +2398,12 @@ def main() -> None:
ap = argparse.ArgumentParser(description=__doc__.splitlines()[0])
ap.add_argument("--out", type=Path, default=Path("/tmp/livepass"))
ap.add_argument("--serve", type=int, metavar="PORT")
ap.add_argument(
"--bind",
default="127.0.0.1",
metavar="HOST",
help="listen address for --serve (use 0.0.0.0 for a manual insecure-origin pass)",
)
ap.add_argument("--perf", action="store_true", help="run the perf baseline and exit")
ap.add_argument(
"--perf-n",
@@ -2235,8 +2421,9 @@ def main() -> None:
import functools
handler = functools.partial(_HarnessHandler, directory=str(args.out))
print(f"serving {args.out} on http://localhost:{args.serve}/ — Ctrl+C stops")
http.server.ThreadingHTTPServer(("127.0.0.1", args.serve), handler).serve_forever()
display_host = "localhost" if args.bind == "127.0.0.1" else args.bind
print(f"serving {args.out} on http://{display_host}:{args.serve}/ — Ctrl+C stops")
http.server.ThreadingHTTPServer((args.bind, args.serve), handler).serve_forever()
if __name__ == "__main__":
+2 -2
View File
@@ -185,8 +185,8 @@ esac
echo ""
echo "NOTE: If you added a NEW library (not just updating a version), also update"
echo " the _ASSET_RE regex in turnstone/core/web_helpers.py — its negative lookahead"
echo " skips vendored directories to avoid double-versioning static asset URLs."
echo " _VERSIONED_VENDOR_DIR in turnstone/core/web_helpers.py — it controls both"
echo " HTML version rewriting and immutable static-response caching."
echo ""
echo "Verify the update:"
echo " git diff --stat"
+1 -1
View File
@@ -2,7 +2,7 @@
"openapi": "3.1.0",
"info": {
"title": "turnstone Console API",
"version": "1.8.0a6",
"version": "1.8.0a7",
"description": "Cluster-wide visibility and control across all turnstone nodes."
},
"paths": {
+168 -16
View File
@@ -2,7 +2,7 @@
"openapi": "3.1.0",
"info": {
"title": "turnstone Server API",
"version": "1.8.0a6",
"version": "1.8.0a7",
"description": "Single-node workstream management, chat interaction, and real-time streaming."
},
"paths": {
@@ -1869,7 +1869,7 @@
},
"/v1/api/memories": {
"get": {
"summary": "List structured memories",
"summary": "List structured memories. Without a scope, returns global plus the authenticated user's memories; workstream scope is owner-bound.",
"operationId": "v1_api_memories_get",
"tags": [
"Memories"
@@ -1891,7 +1891,7 @@
"schema": {
"type": "string"
},
"description": "Filter by scope"
"description": "Filter by public scope: global, workstream, or user"
},
{
"name": "scope_id",
@@ -1923,6 +1923,46 @@
}
}
}
},
"400": {
"description": "Error 400",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ErrorResponse"
}
}
}
},
"403": {
"description": "Error 403",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ErrorResponse"
}
}
}
},
"404": {
"description": "Error 404",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ErrorResponse"
}
}
}
},
"500": {
"description": "Error 500",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ErrorResponse"
}
}
}
}
}
},
@@ -1962,13 +2002,43 @@
}
}
}
},
"403": {
"description": "Error 403",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ErrorResponse"
}
}
}
},
"404": {
"description": "Error 404",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ErrorResponse"
}
}
}
},
"500": {
"description": "Error 500",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ErrorResponse"
}
}
}
}
}
}
},
"/v1/api/memories/search": {
"post": {
"summary": "Search structured memories by query",
"summary": "Search structured memories by query. Without a scope, searches global plus the authenticated user's memories.",
"operationId": "v1_api_memories_search_post",
"tags": [
"Memories"
@@ -1993,6 +2063,46 @@
}
}
}
},
"400": {
"description": "Error 400",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ErrorResponse"
}
}
}
},
"403": {
"description": "Error 403",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ErrorResponse"
}
}
}
},
"404": {
"description": "Error 404",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ErrorResponse"
}
}
}
},
"500": {
"description": "Error 500",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ErrorResponse"
}
}
}
}
}
}
@@ -2043,6 +2153,26 @@
}
}
},
"400": {
"description": "Error 400",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ErrorResponse"
}
}
}
},
"403": {
"description": "Error 403",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ErrorResponse"
}
}
}
},
"404": {
"description": "Error 404",
"content": {
@@ -2052,6 +2182,16 @@
}
}
}
},
"500": {
"description": "Error 500",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ErrorResponse"
}
}
}
}
}
}
@@ -3854,32 +3994,42 @@
"properties": {
"name": {
"description": "Memory identifier (normalized to snake_case)",
"maxLength": 256,
"minLength": 1,
"title": "Name",
"type": "string"
},
"content": {
"description": "Memory content",
"maxLength": 65536,
"minLength": 1,
"title": "Content",
"type": "string"
},
"description": {
"default": "",
"description": "Short description for relevance matching",
"description": "Required non-empty description used for relevance matching",
"minLength": 1,
"title": "Description",
"type": "string"
},
"type": {
"default": "general",
"description": "Memory type",
"enum": [
"user",
"general",
"feedback",
"reference"
"anyOf": [
{
"enum": [
"user",
"general",
"feedback",
"reference"
],
"type": "string"
},
{
"type": "null"
}
],
"title": "Type",
"type": "string"
"default": null,
"description": "Memory type; omission preserves it on update and defaults on insert",
"title": "Type"
},
"scope": {
"default": "global",
@@ -3901,7 +4051,8 @@
},
"required": [
"name",
"content"
"content",
"description"
],
"title": "SaveMemoryRequest",
"type": "object"
@@ -3995,6 +4146,7 @@
"properties": {
"query": {
"description": "Search query text",
"minLength": 1,
"title": "Query",
"type": "string"
},
+8 -1
View File
@@ -394,7 +394,14 @@ export class TurnstoneServer extends BaseClient {
}
async saveMemory(opts: SaveMemoryRequest): Promise<MemoryInfo> {
return this.request("POST", "/v1/api/memories", { json: opts });
if (typeof opts.description !== "string" || !opts.description.trim()) {
throw new TypeError(
"memory description is required and must be non-empty",
);
}
return this.request("POST", "/v1/api/memories", {
json: { ...opts, description: opts.description.trim() },
});
}
async searchMemories(
+1 -1
View File
@@ -894,7 +894,7 @@ export interface WorkstreamsOptions {
export interface SaveMemoryRequest {
name: string;
content: string;
description?: string;
description: string;
type?: "user" | "general" | "feedback" | "reference";
scope?: "global" | "workstream" | "user";
scope_id?: string;
+37
View File
@@ -75,6 +75,43 @@ describe("TurnstoneServer", () => {
});
});
it("saveMemory requires and normalizes the description", async () => {
const fetchFn = mockFetch({
memory_id: "m1",
name: "deployment_process",
description: "Production deployment workflow",
type: "general",
scope: "global",
scope_id: "",
content: "Deploy from main",
created: "2026-08-11T00:00:00",
updated: "2026-08-11T00:00:00",
});
const client = new TurnstoneServer({
baseUrl: "http://test",
fetch: fetchFn,
});
await client.saveMemory({
name: "deployment_process",
content: "Deploy from main",
description: " Production deployment workflow ",
});
const [, init] = (fetchFn as ReturnType<typeof vi.fn>).mock.calls[0];
expect(JSON.parse(init.body)).toMatchObject({
description: "Production deployment workflow",
});
await expect(
client.saveMemory({
name: "deployment_process",
content: "Deploy from main",
description: " ",
}),
).rejects.toThrow("description is required");
expect(fetchFn).toHaveBeenCalledTimes(1);
});
it("send posts correct payload", async () => {
const fetchFn = mockFetch({ status: "ok" });
const client = new TurnstoneServer({
+3 -5
View File
@@ -1,10 +1,8 @@
"""Shared mock factory for ``events_replay`` tests.
Both interactive (:func:`turnstone.server._interactive_events_replay`)
and coord (:func:`turnstone.console.server._coord_events_replay`) drive
the same shared preamble at
:func:`turnstone.core.session_replay.session_replay_preamble`. Their
test suites share the underlying mock surface (session.model,
Interactive and coordinator replay tests exercise the same shared preamble at
:func:`turnstone.core.session_replay.session_replay_preamble` plus their
kind-specific tails. Their test suites share the underlying mock surface (session.model,
session.model_alias, session._last_usage, ui._pending_*, ui._ws_lock,
counters); this module is the single home for that shape so a future
field add lands once.
+14
View File
@@ -15,6 +15,20 @@ from unittest.mock import MagicMock
import pytest
def pytest_sessionstart(session: pytest.Session) -> None:
"""Resolve MCP v1's generic FastMCP settings model for test servers.
MCP 1.29.0 defines ``Settings`` before ``FastMCP``, so its ``lifespan``
annotation remains an unresolved forward reference after import. Rebuild
once, after the module is fully loaded, before any integration fixture
constructs a FastMCP server. Pydantic's public hook is a no-op once the SDK
ships a complete model.
"""
from mcp.server.fastmcp.server import Settings as FastMCPSettings
FastMCPSettings.model_rebuild()
def stop_loop_thread(loop: asyncio.AbstractEventLoop, thread: threading.Thread) -> None:
"""Fully tear down a ``loop.run_forever``-in-a-thread test loop.
+4
View File
@@ -91,6 +91,8 @@ class TestServerVersioning:
def test_shared_static_unversioned(self, client):
resp = client.get("/shared/base.css")
assert resp.status_code == 200
assert resp.headers["cache-control"] == "no-cache"
assert resp.headers["etag"]
class TestConsoleVersioning:
@@ -147,3 +149,5 @@ class TestConsoleVersioning:
resp = client.get("/static/app.js")
body = resp.text
assert "/v1/api/cluster" in body
assert resp.headers["cache-control"] == "no-cache"
assert resp.headers["etag"]
+3
View File
@@ -225,6 +225,9 @@ class _ObservedRLock:
def release(self) -> None:
self._lock.release()
def locked(self) -> bool:
return self._lock.locked()
def __enter__(self):
self.acquire()
return self
+253
View File
@@ -0,0 +1,253 @@
"""Behavior and wiring tests for large-paste attachment conversion."""
from __future__ import annotations
import json
import shutil
import subprocess
from pathlib import Path
import pytest
_ROOT = Path(__file__).resolve().parent.parent
_HELPER = _ROOT / "turnstone/shared_static/composer_paste_text.js"
_COMPOSER = _ROOT / "turnstone/shared_static/composer.js"
_ATTACHMENTS = _ROOT / "turnstone/shared_static/composer_attachments.js"
_INTERACTIVE = _ROOT / "turnstone/shared_static/interactive.js"
_UI_APP = _ROOT / "turnstone/ui/static/app.js"
_CONSOLE_APP = _ROOT / "turnstone/console/static/app.js"
_COORDINATOR = _ROOT / "turnstone/console/static/coordinator/coordinator.js"
def test_paste_text_helper_behavior(tmp_path: Path) -> None:
"""Execute the real ESM and pin its fixed threshold, byte cap, and dedup."""
if shutil.which("node") is None:
pytest.skip("node binary not available on PATH")
module = tmp_path / "composer_paste_text.mjs"
module.write_text(_HELPER.read_text(encoding="utf-8"), encoding="utf-8")
script = tmp_path / "paste_harness.mjs"
module_url = json.dumps(module.as_uri())
script.write_text(
f"const moduleUrl = {module_url};\n"
+ r"""
import { Blob, File } from "node:buffer";
globalThis.Blob = Blob;
globalThis.File = File;
globalThis.window = globalThis;
function check(condition, message) {
if (!condition) throw new Error(message);
}
const paste = await import(moduleUrl);
check(paste.PASTE_ATTACHMENT_CHARS === 2000, "fixed threshold drifted");
check(
paste.pasteTextToFile("x".repeat(1999)) === null,
"below-threshold text converted",
);
const exactText = "x".repeat(2000);
const exact = paste.pasteTextToFile(exactText);
check(exact === null, "exact-threshold text converted");
const convertedText = "x".repeat(2001);
const converted = paste.pasteTextToFile(convertedText);
check(converted instanceof File, "above-threshold text did not convert");
check(converted.name === "pasted-text.txt", "filename drifted");
check(converted.type === "text/plain", "MIME drifted");
check(converted.size === 2001, "byte size drifted");
check(
(await converted.text()) === convertedText,
"file content did not round-trip",
);
check(
paste.pasteTextToFile("") === null,
"empty text converted",
);
check(
paste.pasteTextToFile("😀".repeat(2000)) === null,
"UTF-16 code units were counted as characters",
);
check(
paste.pasteTextToFile("😀".repeat(2001)) instanceof File,
"Unicode code points above the threshold did not convert",
);
const cjkAtCap = "".repeat(Math.floor(paste.TEXT_ATTACHMENT_MAX_BYTES / 3));
check(
paste.pasteTextToFile(cjkAtCap) instanceof File,
"text within the byte ceiling did not convert",
);
check(
paste.pasteTextToFile(cjkAtCap + "") === null,
"multibyte text escaped the byte ceiling",
);
const sameText = "same".repeat(501);
const sameA = paste.pasteTextToFile(sameText);
const sameB = paste.pasteTextToFile(sameText);
const different = paste.pasteTextToFile("size".repeat(501));
check(
paste.isDuplicatePastedTextFile(sameB, [sameA]),
"identical synthesized pastes did not deduplicate",
);
check(
!paste.isDuplicatePastedTextFile(different, [sameA]),
"different same-size pastes were deduplicated",
);
check(
!paste.isDuplicatePastedTextFile(
new File([sameText], "ordinary.txt", { type: "text/plain" }),
[sameA],
),
"ordinary user files entered synthesized-paste dedup",
);
""",
encoding="utf-8",
)
proc = subprocess.run(
["node", str(script)],
capture_output=True,
text=True,
timeout=15,
)
assert proc.returncode == 0, f"paste harness failed:\n{proc.stderr}\n{proc.stdout}"
def test_attachment_snapshot_reports_in_flight_upload(tmp_path: Path) -> None:
"""A synthesized paste cannot disappear from a send-time snapshot while
its immediate upload is still resolving."""
if shutil.which("node") is None:
pytest.skip("node binary not available on PATH")
script = tmp_path / "attachment_snapshot_harness.mjs"
module_url = json.dumps(_ATTACHMENTS.as_uri())
script.write_text(
f"const moduleUrl = {module_url};\n"
+ r"""
import { File } from "node:buffer";
globalThis.File = File;
globalThis.window = globalThis;
function fakeElement() {
return {
children: [],
dataset: {},
appendChild(child) {
this.children.push(child);
return child;
},
addEventListener() {},
querySelector() { return null; },
setAttribute() {},
remove() {},
};
}
globalThis.document = { createElement: () => fakeElement() };
let resolveUpload;
const uploadResponse = new Promise((resolve) => { resolveUpload = resolve; });
const attachmentsModule = await import(moduleUrl);
const controller = attachmentsModule.createAttachmentController({
chipsEl: fakeElement(),
getWsId: () => "ws-1",
authFetch: () => uploadResponse,
});
let snap = controller.snapshot();
if (snap.uploading || snap.attachment_ids.length)
throw new Error("empty controller reported an upload");
controller.upload(new File(["large paste"], "pasted-text.txt", {
type: "text/plain",
}));
snap = controller.snapshot();
if (!snap.uploading)
throw new Error("in-flight placeholder was omitted from snapshot state");
if (snap.attachments.length || snap.attachment_ids.length)
throw new Error("placeholder escaped into stable attachment arrays");
resolveUpload({
ok: true,
status: 200,
json: () => Promise.resolve({
attachment_id: "attachment-1",
filename: "pasted-text.txt",
size_bytes: 11,
mime_type: "text/plain",
kind: "text",
}),
});
await new Promise((resolve) => setTimeout(resolve, 0));
snap = controller.snapshot();
if (snap.uploading)
throw new Error("settled upload remained marked in flight");
if (snap.attachment_ids.join(",") !== "attachment-1")
throw new Error("settled upload was not sendable: " + snap.attachment_ids);
""",
encoding="utf-8",
)
proc = subprocess.run(
["node", str(script)],
capture_output=True,
text=True,
timeout=15,
)
assert proc.returncode == 0, (
f"attachment snapshot harness failed:\n{proc.stderr}\n{proc.stdout}"
)
def test_paste_text_wiring_guard_rails() -> None:
"""Guard every surface around the behavior-tested shared helper."""
helper = _HELPER.read_text(encoding="utf-8")
composer = _COMPOSER.read_text(encoding="utf-8")
attachments = _ATTACHMENTS.read_text(encoding="utf-8")
interactive = _INTERACTIVE.read_text(encoding="utf-8")
ui_app = _UI_APP.read_text(encoding="utf-8")
console_app = _CONSOLE_APP.read_text(encoding="utf-8")
coordinator = _COORDINATOR.read_text(encoding="utf-8")
assert "window.TurnstonePasteText" in helper
assert "PASTE_ATTACHMENT_CHARS = 2000" in helper
assert "paste_attachment_chars" not in helper
assert "loadPasteThresholdChars" not in helper
assert 'from "./composer_paste_text.js"' in composer
assert "pasteTextToFile(text" in composer
assert "2000" not in composer, "the threshold belongs in the shared helper"
assert composer.index("if (uploaded > 0)") < composer.index("pasteTextToFile(text")
assert "if (accepted !== false) e.preventDefault();" in composer
assert "if (pending.has(info.attachment_id))" in attachments
assert "pending.delete(placeholderId);" in attachments
assert "uploading: uploading" in attachments
assert "function _handleComposerPaste(" in ui_app
assert "if (files.length > 0)" in ui_app
assert "if (!textFile || addFiles([textFile]) === false) return false;" in ui_app
assert "_handleComposerPaste(event, _newWsAddFiles)" in ui_app
assert "_handleComposerPaste(e, _addDashboardFiles)" in ui_app
assert "initEl.onpaste" in ui_app
assert ui_app.count("Add a message to send with this attachment.") == 2
assert "_loadPasteAttachmentSetting" not in ui_app
assert "return _homeStageFile(file);" in console_app
assert "isDuplicatePastedTextFile(file, _homeStagedFiles)" in console_app
assert "Add a message to send with this attachment." in console_app
assert "_loadPasteAttachmentSetting" not in console_app
for pane in (interactive, coordinator):
assert "Add a message to send with this attachment." in pane
assert "Attachments can't be sent while the assistant is working." in pane
assert "if (snap.uploading)" in pane
assert "Wait for attachments to finish uploading before sending." in pane
assert "!attachments.isEmpty()" in pane or "!this.attachments.isEmpty()" in pane
assert "loadPasteThresholdChars" not in coordinator
for page in (
_ROOT / "turnstone/ui/static/index.html",
_ROOT / "turnstone/console/static/index.html",
_ROOT / "turnstone/console/static/coordinator/index.html",
):
body = page.read_text(encoding="utf-8")
assert "/shared/composer_paste_text.js" in body, page
assert body.index("/shared/composer_paste_text.js") < body.index("/shared/composer.js"), (
page
)
+163
View File
@@ -1587,6 +1587,168 @@ class TestConsoleProxy:
mock_collector.get_node_detail.return_value = None
resp = client.get("/node/unknown/static/app.js")
assert resp.status_code == 404
assert resp.headers["cache-control"] == "no-store"
@pytest.mark.parametrize("mount", ["static", "shared"])
@pytest.mark.parametrize(
"suffix",
[
"%2e%2e/%2e%2e/v1/api/workstreams/private/history",
"%2E%2E/%2E%2E/v1/api/workstreams/private/history",
"%2e%2e%2f%2e%2e%2fv1%2fapi%2fworkstreams",
"%5c..%5c..%5cv1%5capi%5cworkstreams",
],
)
def test_proxy_static_rejects_encoded_traversal_before_upstream(self, mount, suffix):
from starlette.applications import Starlette
from starlette.routing import Route
from starlette.testclient import TestClient
from turnstone.console import server as csrv
handler = csrv.proxy_static if mount == "static" else csrv.proxy_shared_static
app = Starlette(
routes=[Route(f"/node/{{node_id}}/{mount}/{{path:path}}", endpoint=handler)]
)
app.state.proxy_client = MagicMock()
with TestClient(app) as client:
resp = client.get(f"/node/node-a/{mount}/katex-0.18.4/{suffix}")
assert resp.status_code == 400
assert resp.headers["cache-control"] == "no-store"
app.state.proxy_client.get.assert_not_called()
def test_proxy_static_percent_encodes_each_valid_path_segment(self, monkeypatch):
from types import SimpleNamespace
import httpx
from turnstone.console import server as csrv
upstream = httpx.Response(
200,
content=b"asset",
request=httpx.Request("GET", "http://n:1/static/nested/asset"),
)
calls = []
async def _mock_get(url, *, headers):
calls.append((url, headers))
return upstream
proxy_client = MagicMock(spec=httpx.AsyncClient)
proxy_client.get = MagicMock(side_effect=_mock_get)
request = SimpleNamespace(
app=SimpleNamespace(state=SimpleNamespace(proxy_client=proxy_client)),
path_params={"node_id": "node-a", "path": "nested/asset name?#.js"},
headers={},
)
monkeypatch.setattr(csrv, "_proxy_auth_headers", lambda request: {})
monkeypatch.setattr(csrv, "_get_server_url", lambda request, node_id: "http://n:1")
resp = asyncio.run(csrv.proxy_static(request))
assert calls == [("http://n:1/static/nested/asset%20name%3F%23.js", {})]
assert resp.status_code == 200
assert resp.headers["cache-control"] == "no-cache"
@pytest.mark.parametrize(
("handler_name", "mount", "path"),
[
("proxy_static", "static", "app.js"),
("proxy_shared_static", "shared", "interactive.js"),
],
)
def test_proxy_static_forwards_conditional_request_and_validators(
self, monkeypatch, handler_name, mount, path
):
from types import SimpleNamespace
import httpx
from turnstone.console import server as csrv
upstream = httpx.Response(
304,
headers={
"etag": '"current-build"',
"last-modified": "Wed, 12 Aug 2026 12:00:00 GMT",
},
request=httpx.Request("GET", f"http://n:1/{mount}/{path}"),
)
calls = []
async def _mock_get(url, *, headers):
calls.append((url, headers))
return upstream
proxy_client = MagicMock(spec=httpx.AsyncClient)
proxy_client.get = MagicMock(side_effect=_mock_get)
request = SimpleNamespace(
app=SimpleNamespace(state=SimpleNamespace(proxy_client=proxy_client)),
path_params={"node_id": "node-a", "path": path},
headers={"if-none-match": '"current-build"'},
)
monkeypatch.setattr(csrv, "_proxy_auth_headers", lambda request: {"X-Auth": "test"})
monkeypatch.setattr(csrv, "_get_server_url", lambda request, node_id: "http://n:1")
resp = asyncio.run(getattr(csrv, handler_name)(request))
assert calls == [
(
f"http://n:1/{mount}/{path}",
{"X-Auth": "test", "if-none-match": '"current-build"'},
)
]
assert resp.status_code == 304
assert resp.headers["cache-control"] == "no-cache"
assert resp.headers["etag"] == '"current-build"'
assert resp.headers["last-modified"] == "Wed, 12 Aug 2026 12:00:00 GMT"
@pytest.mark.parametrize("status_code", [404, 500])
def test_proxy_vendor_static_error_is_never_immutable(self, status_code):
import httpx
from turnstone.console.server import _proxy_static_response
upstream = httpx.Response(
status_code,
content=b"transient failure",
request=httpx.Request("GET", "http://n:1/shared/katex-0.18.4/missing.css"),
)
resp = _proxy_static_response(upstream, "katex-0.18.4/missing.css")
assert resp.status_code == status_code
assert resp.headers["cache-control"] == "no-store"
@pytest.mark.parametrize(
("upstream_policy", "expected"),
[
("no-store", "no-store"),
("private, max-age=600", "private, max-age=600"),
("public, max-age=0, must-revalidate", "public, max-age=0, must-revalidate"),
("public, max-age=60", "public, max-age=60"),
],
)
def test_proxy_vendor_static_preserves_stricter_upstream_policy(
self, upstream_policy, expected
):
import httpx
from turnstone.console.server import _proxy_static_response
upstream = httpx.Response(
200,
content=b"asset",
headers={"cache-control": upstream_policy},
request=httpx.Request("GET", "http://n:1/shared/katex-0.18.4/katex.js"),
)
resp = _proxy_static_response(upstream, "katex-0.18.4/katex.js")
assert resp.headers["cache-control"] == expected
def test_proxy_api_unknown_node_returns_404(self, client, mock_collector):
mock_collector.get_node_detail.return_value = None
@@ -2447,6 +2609,7 @@ class TestProxySharedStatic:
client = TestClient(app, raise_server_exceptions=False, headers=_TEST_AUTH_HEADERS)
resp = client.get("/node/unknown/shared/base.css")
assert resp.status_code == 404
assert resp.headers["cache-control"] == "no-store"
client.close()
+14 -27
View File
@@ -2102,39 +2102,35 @@ def test_coord_cancel_cascade_failure_does_not_fail_owner_cancel(storage):
from tests._replay_helpers import make_replay_mocks as _make_coord_replay_mocks # noqa: E402
def test_coord_events_replay_yields_connected_first():
"""Pre-status-bar coord replay only re-injected pending_approval +
pending_plan_review. Post-status-bar parity with interactive
yields ``connected`` first so the dashboard's status bar populates
the model cell before any history arrives mirrors the
interactive replay (turnstone/server.py:_interactive_events_replay)."""
from turnstone.console.server import _coord_events_replay
def test_coord_shared_preamble_yields_connected_first():
"""The shared preamble gives coordinator streams the same bootstrap."""
from turnstone.core.session_replay import session_replay_preamble
ws, ui, request = _make_coord_replay_mocks()
out = list(_coord_events_replay(ws, ui, request))
ws, ui, _request = _make_coord_replay_mocks()
out = list(session_replay_preamble(ws.session, ui))
assert out[0]["type"] == "connected"
assert out[0]["model"] == "gpt-5"
assert out[0]["model_alias"] == "default"
assert out[0]["skip_permissions"] is False
def test_coord_events_replay_includes_status_only_when_last_usage_present():
def test_coord_shared_preamble_includes_status_only_when_last_usage_present():
"""The ``status`` event populates the per-tab token-usage bar on
resume. Skipped when ``session._last_usage`` is None (a freshly-
created coordinator that hasn't completed a turn) — matches
interactive behaviour."""
from turnstone.console.server import _coord_events_replay
from turnstone.core.session_replay import session_replay_preamble
ws, ui, request = _make_coord_replay_mocks()
out = list(_coord_events_replay(ws, ui, request))
ws, ui, _request = _make_coord_replay_mocks()
out = list(session_replay_preamble(ws.session, ui))
assert "status" not in {ev["type"] for ev in out}
def test_coord_events_replay_status_payload_shape():
def test_coord_shared_preamble_status_payload_shape():
"""When ``last_usage`` exists, the replayed ``status`` event carries
every field the dashboard's updateStatusBar() reads — same shape
SessionUI.on_status emits live."""
from turnstone.console.server import _coord_events_replay
from turnstone.core.session_replay import session_replay_preamble
ws, ui, request = _make_coord_replay_mocks(
last_usage={
@@ -2146,7 +2142,7 @@ def test_coord_events_replay_status_payload_shape():
_ws_turn_tool_calls=3,
_ws_messages=7,
)
out = list(_coord_events_replay(ws, ui, request))
out = list(session_replay_preamble(ws.session, ui))
status = next(ev for ev in out if ev["type"] == "status")
assert status["prompt_tokens"] == 40000
assert status["completion_tokens"] == 6310
@@ -2174,12 +2170,7 @@ def test_coord_events_replay_skips_session_block_when_no_session():
def test_coord_events_replay_yields_pending_approval():
"""The lifted coord ``events_replay`` callback yields, after the
connected preamble, the pending approval (if any). Pre-lift coord
pushed it onto the listener queue via ``put_nowait``; the lift
restructures as a generator the lifted body iterates and yields as
``data:`` lines, but the payload identity is preserved. Pure-read
never mutates ``ui``."""
"""The coord replay tail yields pending approval without mutation."""
from turnstone.console.server import _coord_events_replay
ws, ui, request = _make_coord_replay_mocks(
@@ -2188,8 +2179,6 @@ def test_coord_events_replay_yields_pending_approval():
out = list(_coord_events_replay(ws, ui, request))
types = [ev["type"] for ev in out]
# Status preamble is yielded first (no last_usage → no status); the
# pending-approval re-injection then matches the pre-lift body.
assert types[0] == "connected"
assert "approve_request" in types
@@ -2246,9 +2235,7 @@ def test_coord_events_replay_skips_verdict_replay_without_pending_approval():
def test_coord_events_replay_yields_only_connected_when_no_pending():
"""A workstream with a session but no pending approval / plan
review and no last_usage yields just the ``connected`` preamble.
The lifted body falls through to the live loop immediately after."""
"""Without controls or usage, replay contains only the preamble."""
from turnstone.console.server import _coord_events_replay
ws, ui, request = _make_coord_replay_mocks()
+23 -3
View File
@@ -26,6 +26,8 @@ def client():
def test_valid_ws_id_injects_data_attr(client):
from turnstone import __version__
ws_id = "a" * 32
resp = client.get(f"/coordinator/{ws_id}")
assert resp.status_code == 200
@@ -35,9 +37,27 @@ def test_valid_ws_id_injects_data_attr(client):
assert f'data-ws-id="{ws_id}"' in body
# Template placeholder is fully substituted.
assert "{{WS_ID}}" not in body
# Sanity: the shared static imports are wired.
assert "/shared/base.css" in body
assert "/static/coordinator/coordinator.js" in body
# First-party tags are versioned; version-named vendor assets stay stable.
assert f"/shared/base.css?v={__version__}" in body
assert f"/static/coordinator/coordinator.css?v={__version__}" in body
assert "/shared/katex-0.18.4/katex.min.css?v=" not in body
# Inline module imports are outside version_html's src/href boundary. The
# static route's no-cache policy makes this URL revalidate on every reload.
assert 'from "/static/coordinator/coordinator.js"' in body
def test_coordinator_page_revalidates_with_etag(client):
ws_id = "b" * 32
first = client.get(f"/coordinator/{ws_id}")
assert first.headers["cache-control"] == "no-cache"
assert first.headers["etag"]
unchanged = client.get(
f"/coordinator/{ws_id}", headers={"If-None-Match": first.headers["etag"]}
)
assert unchanged.status_code == 304
assert unchanged.headers["cache-control"] == "no-cache"
assert unchanged.headers["etag"] == first.headers["etag"]
def test_non_hex_ws_id_returns_400(client):
+1
View File
@@ -2261,6 +2261,7 @@ def test_prepare_and_write_path_refuse_in_the_same_words(coord_session):
item = sess._prepare_tool(_tc("tasks", args))
assert "error" in item, args
expected = sess._coord_tool_error("call-1", "tasks", f"{action}: {authoritative['error']}")
expected["_principal_id"] = sess._tool_prepare_principal_id()
assert item == expected, (args, item["error"])
+39 -23
View File
@@ -9,6 +9,8 @@ provider_blocks, trailing-citation fold).
from __future__ import annotations
import httpx
import httpx2
import pytest
from turnstone.core.providers import (
@@ -418,11 +420,18 @@ class TestTransportGuarded:
"""``transport_guarded`` — drain's conversion rule for consumers that
keep streaming semantics (the interactive loop)."""
@pytest.mark.parametrize("exc_name", ["ReadError", "RemoteProtocolError"])
def test_pre_finish_transport_error_becomes_retryable_incomplete(self, exc_name):
import httpx
exc_cls = getattr(httpx, exc_name)
@pytest.mark.parametrize(
"exc_cls",
[
httpx.ReadError,
httpx.RemoteProtocolError,
httpx2.ReadError,
httpx2.RemoteProtocolError,
],
ids=["httpx-read", "httpx-protocol", "httpx2-read", "httpx2-protocol"],
)
def test_pre_finish_transport_error_becomes_retryable_incomplete(self, exc_cls):
exc_name = exc_cls.__name__
def chunks():
yield StreamChunk(content_delta="partial")
@@ -434,14 +443,17 @@ class TestTransportGuarded:
next(it)
assert isinstance(excinfo.value.__cause__, exc_cls)
def test_message_byte_matches_drains_shape(self):
@pytest.mark.parametrize(
"exc_cls",
[httpx.ReadError, httpx2.ReadError],
ids=["httpx", "httpx2"],
)
def test_message_byte_matches_drains_shape(self, exc_cls):
# The wrapper and the drain are the SAME conversion rule; any test
# or log filter pinned to drain's message must match the wrapper's.
import httpx
def chunks():
yield StreamChunk(content_delta="partial")
raise httpx.ReadError("[SSL] record layer failure (_ssl.c:2590)")
raise exc_cls("[SSL] record layer failure (_ssl.c:2590)")
with pytest.raises(IncompleteStreamError) as guarded:
list(transport_guarded(chunks()))
@@ -449,17 +461,20 @@ class TestTransportGuarded:
drain_stream(chunks())
assert str(guarded.value) == str(drained.value)
def test_post_finish_blip_ends_stream_cleanly(self, caplog):
@pytest.mark.parametrize(
"exc_cls",
[httpx.ReadError, httpx2.ReadError],
ids=["httpx", "httpx2"],
)
def test_post_finish_blip_ends_stream_cleanly(self, caplog, exc_cls):
# The generation already completed — the blip only cost trailing
# metadata, so the stream ends instead of raising.
import logging
import httpx
def chunks():
yield StreamChunk(content_delta="done")
yield StreamChunk(finish_reason="stop")
raise httpx.ReadError("late blip")
raise exc_cls("late blip")
with caplog.at_level(logging.WARNING, logger="turnstone.core.providers._protocol"):
out = list(transport_guarded(chunks()))
@@ -502,32 +517,33 @@ class TestErrorPropagation:
# The generation completed (finish reason in hand) — a trailing
# transport blip forfeits only trailing metadata (here: the usage
# chunk), never the completed result.
import httpx
def chunks():
yield StreamChunk(content_delta="whole answer")
yield StreamChunk(finish_reason="stop")
raise httpx.ReadError("late blip")
raise httpx2.ReadError("late blip")
result = drain_stream(chunks())
assert result.content == "whole answer"
assert result.finish_reason == "stop"
assert result.usage is None
def test_httpx_transport_error_becomes_retryable_incomplete(self):
@pytest.mark.parametrize(
"exc_cls",
[httpx.RemoteProtocolError, httpx2.RemoteProtocolError],
ids=["httpx", "httpx2"],
)
def test_transport_error_becomes_retryable_incomplete(self, exc_cls):
# Streaming moves the body read out of the SDK's wrapped request:
# a mid-body wire failure surfaces as a raw httpx.TransportError
# no retry predicate recognizes. The drain re-raises it (chained,
# a mid-body wire failure surfaces as a raw HTTPX-family TransportError
# no retry predicate recognizes. The drain re-raises it (chained,
# message preserved) as the retryable IncompleteStreamError.
import httpx
def chunks():
yield StreamChunk(content_delta="partial")
raise httpx.RemoteProtocolError("peer closed connection")
raise exc_cls("peer closed connection")
with pytest.raises(IncompleteStreamError, match="RemoteProtocolError") as excinfo:
drain_stream(chunks())
assert isinstance(excinfo.value.__cause__, httpx.RemoteProtocolError)
assert isinstance(excinfo.value.__cause__, exc_cls)
def test_mid_stream_exception_propagates_verbatim(self):
# Retry/deadline/fallback policy is the caller's — the drain adds
+2 -1
View File
@@ -256,6 +256,7 @@ class TestWorldSeeding:
"memory": [
{
"name": "proj-context",
"description": "Project deployment context",
"content": "acme-api: staging tracks main.",
"type": "reference",
}
@@ -2752,7 +2753,7 @@ class TestRunResourceLifecycle:
assert not is_storage_initialized()
def test_a_hung_generation_is_bounded_by_the_wall_clock(self, monkeypatch):
"""The per-request httpx timeout cannot bound a STREAM — a
"""The per-request HTTP transport timeout cannot bound a STREAM — a
trickling response resets the read timeout indefinitely so
without the executor wall clock a hung generation occupies a run
slot forever and is scored as a body regression when the sweep
+89 -2
View File
@@ -1209,8 +1209,8 @@ def test_deferred_send_settle_protocol_pins() -> None:
assert "deferred: !!data.deferred" in composer_queue
assert "attachedCount: (data.attached_ids || []).length" in composer_queue
assert "ctx.busyIsOptimistic()" in composer_queue
assert composer_queue.count("ctx.optimisticEl.remove()") >= 2, (
"both the retro-convert and queue_full arms must clear the optimistic bubble"
assert composer_queue.count("ctx.optimisticEl.remove()") >= 3, (
"retro-convert, queue_full, and attachments_busy must clear false optimistic bubbles"
)
# The missed-edge settle: a non-deferred chip binding onto an
# already-idle pane missed its only sweep — the post-bind promote
@@ -1233,6 +1233,10 @@ def test_deferred_send_settle_protocol_pins() -> None:
assert "settleSendResponse(" not in src, f"{name}: must not bypass the fetch stage"
assert "busyIsOptimistic" in src, name
assert "paneIsBusy" in src, f"{name}: the missed-edge settle needs the live flag"
assert "mergeRejectedComposerText" in src, f"{name}: refused text must be restored"
assert src.count("restoreInput:") == 2, (
f"{name}: composer send and edit-resend both need refusal restoration"
)
assert 'setBusy(true, "optimistic")' in src, f"{name}: optimistic flip must stamp"
assert "parsePriority(" in src, f"{name}: shared !!! parse"
assert 'case "message_dispatched"' in src, f"{name}: settle event not consumed"
@@ -1332,6 +1336,89 @@ console.log("settle matrix OK");
assert proc.returncode == 0, f"settle harness failed:\n{proc.stderr}\n{proc.stdout}"
def test_stale_idle_refusals_restore_input(tmp_path) -> None:
"""A stale local idle state must not render either busy refusal as
delivered or discard its companion text. Text entered during the POST is
retained after the rejected text, and an SSE idle that already arrived
prevents the old optimistic busy state from being reasserted."""
import shutil
import subprocess
if shutil.which("node") is None:
pytest.skip("node binary not available on PATH")
helper = _ROOT / "turnstone/shared_static/composer_queue.js"
script = tmp_path / "attachments_busy_harness.mjs"
script.write_text(
rf"""const {{ mergeRejectedComposerText, settleSendResponse }} =
await import("file://{helper}");
function run(status, optimisticBusy) {{
const calls = [];
let composerValue = "typed during request";
const optimisticEl = {{
isConnected: true,
dataset: {{}},
remove: () => calls.push("remove-optimistic"),
}};
settleSendResponse(
{{ remove: () => calls.push("remove-queued") }},
{{ status }},
{{
queuedEl: null,
optimisticEl,
isBusy: false,
setBusy: (value) => calls.push("busy:" + value),
busyIsOptimistic: () => optimisticBusy,
paneIsBusy: () => optimisticBusy,
restoreInput: () => {{
composerValue = mergeRejectedComposerText("rejected", composerValue);
calls.push("restore");
}},
renderError: () => calls.push("error"),
consumeAttachments: () => calls.push("consume"),
}},
);
return {{ calls, composerValue }};
}}
for (const [status, expectedBusy] of [
["attachments_busy", "busy:true"],
["cross_user_interjection", "busy:false"],
["queue_full", "busy:false"],
]) {{
let result = run(status, true);
if (result.composerValue !== "rejected\ntyped during request")
throw new Error(status + " companion/current text merge drifted: " + result.composerValue);
for (const call of ["remove-optimistic", "restore", expectedBusy, "error"]) {{
if (!result.calls.includes(call))
throw new Error(status + " missing stale-idle settlement " + call + ": " + result.calls);
}}
if (result.calls.includes("consume") || result.calls.includes("remove-queued"))
throw new Error(status + " attachments/chip state was consumed: " + result.calls);
result = run(status, false);
if (result.calls.some((call) => call.startsWith("busy:")))
throw new Error(status + " overwrote a raced SSE state: " + result.calls);
}}
if (
mergeRejectedComposerText("rejected", "rejected\nlater") !==
"rejected\nrejected\nlater"
)
throw new Error("independently typed matching text was discarded");
if (mergeRejectedComposerText("rejected", "") !== "rejected")
throw new Error("empty composer did not restore rejected text");
""",
encoding="utf-8",
)
proc = subprocess.run(
["node", str(script)],
capture_output=True,
text=True,
timeout=15,
)
assert proc.returncode == 0, f"attachments_busy harness failed:\n{proc.stderr}\n{proc.stdout}"
def test_accepted_tool_event_recorded_only_when_painted() -> None:
"""An unpainted accepted tool_result must stay replayable.
+69
View File
@@ -0,0 +1,69 @@
"""Regression guards for the native paste-over-HTTP livepass."""
from __future__ import annotations
import importlib.util
import re
import shutil
import subprocess
from pathlib import Path
from typing import Any
import pytest
_ROOT = Path(__file__).resolve().parent.parent
_SCRIPT = _ROOT / "scripts/livepass.py"
def _load_livepass() -> Any:
spec = importlib.util.spec_from_file_location("livepass_paste_script", _SCRIPT)
assert spec is not None and spec.loader is not None
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)
return module
def test_paste_livepass_builds_real_clipboard_event_path(tmp_path: Path) -> None:
livepass = _load_livepass()
livepass.build(tmp_path)
page_path = tmp_path / "paste/livepass.html"
page = page_path.read_text(encoding="utf-8")
assert (tmp_path / "paste/shared").resolve() == _ROOT / "turnstone/shared_static"
assert 'import { Composer } from "./shared/composer.js";' in page
assert "PASTE_ATTACHMENT_CHARS" in page
assert "new Composer(" in page
assert "event.isTrusted" in page
assert "event.clipboardData" in page
assert "window.isSecureContext" in page
assert 'attachment.name === "pasted-text.txt"' in page
assert 'attachment.type === "text/plain"' in page
assert 'document.title = "PASTE-HTTP-READY"' in page
assert 'document.title = "PASTE-HTTP-FAILED-" + reason' in page
# These shortcuts would make the harness green without exercising the
# browser's native clipboard event and therefore invalidate its purpose.
assert "navigator.clipboard" not in page
assert "new ClipboardEvent" not in page
assert ".dispatchEvent(" not in page
assert "__pasteProbe" not in page
def test_generated_paste_module_is_valid_javascript(tmp_path: Path) -> None:
if shutil.which("node") is None:
pytest.skip("node binary not available on PATH")
livepass = _load_livepass()
livepass.build(tmp_path)
page = (tmp_path / "paste/livepass.html").read_text(encoding="utf-8")
match = re.search(r'<script type="module">\s*(.*?)\s*</script>', page, re.S)
assert match is not None
module = tmp_path / "paste_livepass.mjs"
module.write_text(match.group(1), encoding="utf-8")
result = subprocess.run(
["node", "--check", str(module)],
capture_output=True,
text=True,
timeout=15,
)
assert result.returncode == 0, result.stderr
+69
View File
@@ -4069,6 +4069,75 @@ class TestStaticNotificationRefresh:
"the hung sibling must be cancelled and reaped inside the scope"
)
def test_resources_refresh_accepts_missing_template_method(self) -> None:
"""Push/manual refresh shares the per-method compatibility behavior."""
from mcp import McpError
mgr = MCPClientManager({})
session = MagicMock()
session.list_resources = AsyncMock(
return_value=mcp_types.ListResourcesResult(
resources=[
mcp_types.Resource(
uri="res://fresh",
name="fresh",
mimeType="text/plain",
)
]
)
)
session.list_resource_templates = AsyncMock(
side_effect=McpError(
mcp_types.ErrorData(
code=mcp_types.METHOD_NOT_FOUND,
message="templates disabled",
)
)
)
state = _seed_static_state(
mgr,
"srv",
session=session,
supports_resources=True,
resources=[{"uri": "res://old", "server": "srv"}],
)
mgr._rebuild_resources()
asyncio.run(mgr._refresh_server_resources("srv"))
assert [r["uri"] for r in state.resources] == ["res://fresh"]
assert [r["uri"] for r in mgr.get_resources()] == ["res://fresh"]
def test_resources_refresh_rejects_message_only_method_match(self) -> None:
"""Only -32601 is normalized; a lookalike error leaves catalog intact."""
from mcp import McpError
mgr = MCPClientManager({})
session = MagicMock()
session.list_resources = AsyncMock(return_value=mcp_types.ListResourcesResult(resources=[]))
session.list_resource_templates = AsyncMock(
side_effect=McpError(
mcp_types.ErrorData(
code=mcp_types.INVALID_PARAMS,
message="Method not found",
)
)
)
old_resources = [{"uri": "res://old", "server": "srv"}]
state = _seed_static_state(
mgr,
"srv",
session=session,
supports_resources=True,
resources=old_resources,
)
with pytest.raises(McpError) as raised:
asyncio.run(mgr._refresh_server_resources("srv"))
assert raised.value.error.code == mcp_types.INVALID_PARAMS
assert state.resources is old_resources
def test_reap_bounded_reraises_external_cancel(self) -> None:
"""An EXTERNAL cancel delivered during the reap window must be
HONOURED (re-raised), not swallowed else a shutdown/cancel of
+392
View File
@@ -26,7 +26,9 @@ from contextlib import asynccontextmanager
from typing import Any
from unittest.mock import AsyncMock, MagicMock, patch
import mcp.types as mcp_types
import pytest
from mcp import McpError
from turnstone.core.mcp_client import MCPClientManager
@@ -155,6 +157,348 @@ class TestTransportOwnerLifecycle:
assert state.owner_task is None
assert state.close_requested is None
def test_tools_only_connect_skips_unchanged_catalog_listeners(self, running_loop_mgr) -> None:
"""A tools-only registration must not rebuild every live chat twice."""
mgr, loop, _ = running_loop_mgr
patches: dict[str, Any] = {}
_fake_transport_and_session(patches)
notifications: list[str] = []
mgr.add_listener(lambda: notifications.append("tools"))
mgr.add_resource_listener(lambda: notifications.append("resources"))
mgr.add_prompt_listener(lambda: notifications.append("prompts"))
with (
patch("turnstone.core.mcp_client.stdio_client", patches["stdio_client"]),
patch("turnstone.core.mcp_client.ClientSession", patches["ClientSession"]),
):
_run(loop, mgr._connect_one_locked("srv", mgr._server_configs["srv"]))
assert notifications == ["tools"]
_run(loop, mgr._teardown_static_session("srv"))
def test_connect_notifies_when_dropped_capability_clears_stale_catalog(
self, running_loop_mgr
) -> None:
"""Capability loss clears and broadcasts genuinely stale catalogs."""
mgr, loop, _ = running_loop_mgr
state = mgr._ensure_static_state("srv")
state.resources = [
{
"uri": "res://stale",
"name": "stale",
"description": "",
"mimeType": "text/plain",
"server": "srv",
}
]
state.prompts = [
{
"name": "mcp__srv__stale",
"original_name": "stale",
"server": "srv",
"description": "",
"arguments": [],
}
]
mgr._rebuild_resources()
mgr._rebuild_prompts()
patches: dict[str, Any] = {}
_fake_transport_and_session(patches)
notifications: list[str] = []
mgr.add_listener(lambda: notifications.append("tools"))
mgr.add_resource_listener(lambda: notifications.append("resources"))
mgr.add_prompt_listener(lambda: notifications.append("prompts"))
with (
patch("turnstone.core.mcp_client.stdio_client", patches["stdio_client"]),
patch("turnstone.core.mcp_client.ClientSession", patches["ClientSession"]),
):
_run(loop, mgr._connect_one_locked("srv", mgr._server_configs["srv"]))
assert state.resources == []
assert state.prompts == []
assert mgr.get_resources() == []
assert mgr.get_prompts() == []
assert notifications == ["tools", "resources", "prompts"]
_run(loop, mgr._teardown_static_session("srv"))
def test_static_connect_accepts_resources_without_template_method(
self, running_loop_mgr
) -> None:
"""The aggregate resources capability does not require both list methods.
A server with concrete resources but no templates still publishes its
tools/resources and retains a usable live session when the unsupported
half returns the protocol's exact METHOD_NOT_FOUND code.
"""
mgr, loop, _ = running_loop_mgr
patches: dict[str, Any] = {}
fake = _fake_transport_and_session(patches)
session = fake["session"]
session.get_server_capabilities = MagicMock(
return_value=mcp_types.ServerCapabilities(
tools=mcp_types.ToolsCapability(listChanged=False),
resources=mcp_types.ResourcesCapability(listChanged=False),
)
)
session.list_tools = AsyncMock(
return_value=mcp_types.ListToolsResult(
tools=[
mcp_types.Tool(
name="echo",
description="Echo input",
inputSchema={"type": "object", "properties": {}},
)
]
)
)
session.list_resources = AsyncMock(
return_value=mcp_types.ListResourcesResult(
resources=[
mcp_types.Resource(
uri="res://one",
name="one",
mimeType="text/plain",
)
]
)
)
session.list_resource_templates = AsyncMock(
side_effect=McpError(
mcp_types.ErrorData(
code=mcp_types.METHOD_NOT_FOUND,
message="templates disabled",
)
)
)
with (
patch("turnstone.core.mcp_client.stdio_client", patches["stdio_client"]),
patch("turnstone.core.mcp_client.ClientSession", patches["ClientSession"]),
):
_run(loop, mgr._connect_one_locked("srv", mgr._server_configs["srv"]))
state = mgr._static_servers["srv"]
assert state.session is session
assert state.owner_task is not None and not state.owner_task.done()
assert mgr.is_mcp_tool("mcp__srv__echo") is True
assert {r["uri"] for r in mgr.get_resources()} == {"res://one"}
_run(loop, mgr._teardown_static_session("srv"))
def test_failed_add_is_atomic_and_rejects_message_only_method_match(
self, running_loop_mgr
) -> None:
"""A later discovery failure cannot leave a callable ghost tool.
The error deliberately says ``Method not found`` but carries
INVALID_PARAMS: compatibility is classified by JSON-RPC code only.
"""
mgr, _loop, _ = running_loop_mgr
patches: dict[str, Any] = {}
fake = _fake_transport_and_session(patches)
session = fake["session"]
session.get_server_capabilities = MagicMock(
return_value=mcp_types.ServerCapabilities(
tools=mcp_types.ToolsCapability(listChanged=False),
resources=mcp_types.ResourcesCapability(listChanged=False),
)
)
session.list_tools = AsyncMock(
return_value=mcp_types.ListToolsResult(
tools=[
mcp_types.Tool(
name="ghost",
description="Must never publish",
inputSchema={"type": "object", "properties": {}},
)
]
)
)
session.list_resources = AsyncMock(
side_effect=McpError(
mcp_types.ErrorData(
code=mcp_types.INVALID_PARAMS,
message="Method not found",
)
)
)
with (
patch("turnstone.core.mcp_client.stdio_client", patches["stdio_client"]),
patch("turnstone.core.mcp_client.ClientSession", patches["ClientSession"]),
):
result = mgr.add_server_sync(
"new",
{"type": "stdio", "command": "fake-cmd"},
timeout=5,
)
assert result["connected"] is False
assert "Method not found" in result["error"]
assert "new" not in mgr._server_configs
state = mgr._static_servers["new"]
assert state.session is None
assert state.owner_task is None
assert state.close_requested is None
assert mgr.is_mcp_tool("mcp__new__ghost") is False
assert mgr.get_tools() == []
assert mgr.get_resources() == []
assert fake["events"] == [
"transport_enter",
"session_enter",
"session_exit",
"transport_exit",
]
def test_timed_out_add_waits_for_atomic_rollback(self, running_loop_mgr) -> None:
"""Returning timeout must not strand an unconfigured live transport."""
mgr, _loop, _ = running_loop_mgr
patches: dict[str, Any] = {}
fake = _fake_transport_and_session(patches)
session = fake["session"]
session.get_server_capabilities = MagicMock(
return_value=mcp_types.ServerCapabilities(
tools=mcp_types.ToolsCapability(listChanged=False),
resources=mcp_types.ResourcesCapability(listChanged=False),
)
)
session.list_tools = AsyncMock(
return_value=mcp_types.ListToolsResult(
tools=[
mcp_types.Tool(
name="ghost",
description="Must never publish",
inputSchema={"type": "object", "properties": {}},
)
]
)
)
async def _park_resources() -> Any:
await asyncio.sleep(3600)
session.list_resources = _park_resources
with (
patch("turnstone.core.mcp_client.stdio_client", patches["stdio_client"]),
patch("turnstone.core.mcp_client.ClientSession", patches["ClientSession"]),
):
result = mgr.add_server_sync(
"new",
{"type": "stdio", "command": "fake-cmd"},
timeout=0.05,
)
assert result == {
"connected": False,
"tools": 0,
"resources": 0,
"prompts": 0,
"error": "MCP server 'new' registration timed out",
}
assert "new" not in mgr._server_configs
state = mgr._static_servers["new"]
assert state.session is None
assert state.owner_task is None
assert state.close_requested is None
assert state.tools == []
assert state.resources == []
assert state.prompts == []
assert mgr.is_mcp_tool("mcp__new__ghost") is False
assert fake["events"] == [
"transport_enter",
"session_enter",
"session_exit",
"transport_exit",
]
def test_add_reports_late_success_when_connect_suppresses_timeout(
self, running_loop_mgr
) -> None:
"""The sync result reflects the definitive loop-side outcome.
A dependency can suppress cancellation. In that case ``asyncio.timeout``
cannot honestly report a timeout, so the registration is a late success
rather than a failed result with a live ghost catalog.
"""
mgr, _loop, _ = running_loop_mgr
async def _late_success(name: str, _cfg: dict[str, Any]) -> None:
# Model the completion-vs-timeout race: the connect finishes and
# commits after the sync caller's deadline but before its
# cancellation can make the operation fail.
with contextlib.suppress(asyncio.CancelledError):
await asyncio.sleep(3600)
state = mgr._ensure_static_state(name)
state.session = MagicMock()
state.tools = [
{
"type": "function",
"function": {
"name": f"mcp__{name}__late",
"description": "late",
"parameters": {"type": "object", "properties": {}},
},
}
]
mgr._rebuild_tools()
with patch.object(mgr, "_connect_one_locked", side_effect=_late_success):
result = mgr.add_server_sync(
"new",
{"type": "stdio", "command": "fake-cmd"},
timeout=0.05,
)
assert result == {
"connected": True,
"tools": 1,
"resources": 0,
"prompts": 0,
"error": "",
}
assert mgr._server_configs["new"] == {"type": "stdio", "command": "fake-cmd"}
state = mgr._static_servers["new"]
assert state.session is not None
assert len(state.tools) == 1
assert mgr.is_mcp_tool("mcp__new__late") is True
def test_blocked_loop_cannot_return_failure_before_add_outcome(self, running_loop_mgr) -> None:
"""A synchronous listener stall may delay success, never expose a ghost."""
mgr, _loop, _ = running_loop_mgr
async def _publish_then_notify(name: str, _cfg: dict[str, Any]) -> None:
state = mgr._ensure_static_state(name)
state.session = MagicMock()
state.tools = [
{
"type": "function",
"function": {
"name": f"mcp__{name}__live",
"description": "live",
"parameters": {"type": "object", "properties": {}},
},
}
]
mgr._rebuild_tools()
mgr.add_listener(lambda: time.sleep(0.1))
started = time.monotonic()
with patch.object(mgr, "_connect_one_locked", side_effect=_publish_then_notify):
result = mgr.add_server_sync(
"new",
{"type": "stdio", "command": "fake-cmd"},
timeout=0.01,
)
elapsed = time.monotonic() - started
assert elapsed >= 0.08
assert result["connected"] is True
assert result["error"] == ""
assert "new" in mgr._server_configs
assert mgr._static_servers["new"].session is not None
assert mgr.is_mcp_tool("mcp__new__live") is True
def test_owner_death_evicts_session(self, running_loop_mgr) -> None:
"""Trigger-A observer: the transport collapsing under a live session
(owner task dies without a requested close) evicts the session so the
@@ -236,6 +580,54 @@ class TestTransportOwnerLifecycle:
# The owner unwound its cms despite dying mid-discovery.
assert fake["events"][-2:] == ["session_exit", "transport_exit"]
def test_owner_death_same_turn_as_discovery_cannot_publish_catalog(
self, running_loop_mgr
) -> None:
"""When discovery and owner death tie, transport death wins."""
mgr, _loop, _ = running_loop_mgr
patches: dict[str, Any] = {}
fake = _fake_transport_and_session(patches)
async def _cancel_owner_and_return_tool() -> mcp_types.ListToolsResult:
owner = mgr._static_servers["new"].owner_task
assert owner is not None
owner.cancel()
# Let the owner consume cancellation before this discovery task
# returns, putting both futures in the completed set observed by
# ``_await_owner_discovery``.
await asyncio.sleep(0)
return mcp_types.ListToolsResult(
tools=[
mcp_types.Tool(
name="ghost",
description="Must never publish",
inputSchema={"type": "object", "properties": {}},
)
]
)
fake["session"].list_tools = AsyncMock(side_effect=_cancel_owner_and_return_tool)
with (
patch("turnstone.core.mcp_client.stdio_client", patches["stdio_client"]),
patch("turnstone.core.mcp_client.ClientSession", patches["ClientSession"]),
):
result = mgr.add_server_sync(
"new",
{"type": "stdio", "command": "fake-cmd"},
timeout=5,
)
assert result["connected"] is False
assert "died during discovery" in result["error"]
assert "new" not in mgr._server_configs
state = mgr._static_servers["new"]
assert state.session is None
assert state.owner_task is None
assert state.tools == []
assert mgr.is_mcp_tool("mcp__new__ghost") is False
assert fake["events"][-2:] == ["session_exit", "transport_exit"]
def test_base_exception_escape_resolves_waiter_and_propagates(self, running_loop_mgr) -> None:
"""A BaseException-derived escape that is neither CancelledError nor
Exception/group (a library control-flow escape; SystemExit and
+61
View File
@@ -1414,6 +1414,67 @@ def test_pool_resource_discovery_on_connect_via_real_streamable_http(
assert {r["uri"] for r in mgr._user_resources["user-1"]} == {"res://test/1", "res://test/2"}
@pytest.mark.parametrize(
("unsupported_method", "expected_uri", "is_template"),
[
("templates", "res://concrete", False),
("resources", "res://items/{id}", True),
],
)
def test_pool_resource_discovery_accepts_either_list_method_unsupported(
running_loop_mgr: Any,
monkeypatch: pytest.MonkeyPatch,
unsupported_method: str,
expected_uri: str,
is_template: bool,
) -> None:
"""Real SDK boundary: either half of resource discovery may be absent.
The server advertises the aggregate resources capability and exposes a
tool plus exactly one resource-list method. A wire-level -32601 from the
other method is an empty half-catalog, not a failed pool registration.
"""
mgr, loop, _ = running_loop_mgr
unsupported = {
"jsonrpc": "2.0",
"id": 0,
"error": {"code": -32601, "message": "Method not found"},
}
concrete = _list_resources_payload([_resource_spec("res://concrete")])
templates = {
"jsonrpc": "2.0",
"id": 3,
"result": {
"resourceTemplates": [
{
"uriTemplate": "res://items/{id}",
"name": "item",
"mimeType": "text/plain",
}
]
},
}
handler = _make_jsonrpc_handler(
init_response=_init_response_with_caps(resources=True),
list_tools_response=_list_tools_payload([_tool_spec("echo")]),
list_resources_response=unsupported if unsupported_method == "resources" else concrete,
list_resource_templates_response=(
unsupported if unsupported_method == "templates" else templates
),
)
_build_mock_transport_factory(mgr, monkeypatch, handler)
_patch_tcp_probe(mgr, monkeypatch)
entry = _connect_pool(mgr, loop, user_id="user-1", server_name="pool-srv")
assert entry.session is not None
assert mgr.is_mcp_tool("mcp__pool-srv__echo", user_id="user-1") is True
assert entry.resources is not None
assert [(r["uri"], bool(r.get("template"))) for r in entry.resources] == [
(expected_uri, is_template)
]
def test_pool_prompt_discovery_on_connect_via_real_streamable_http(
running_loop_mgr: Any, monkeypatch: pytest.MonkeyPatch
) -> None:
+133 -18
View File
@@ -121,7 +121,7 @@ def _seed_memory(storage, name="test_key", content="test content", **kw):
storage.create_structured_memory(
mid,
name,
kw.get("description", ""),
kw.get("description", "Seeded memory"),
kw.get("mem_type", "general"),
kw.get("scope", "global"),
kw.get("scope_id", ""),
@@ -130,6 +130,20 @@ def _seed_memory(storage, name="test_key", content="test content", **kw):
return mid
def _seed_workstream(storage, ws_id: str = "ws1", user_id: str = "test-user") -> None:
storage.register_workstream(ws_id, user_id=user_id)
def _save_body(name: str, content: str, **overrides: Any) -> dict[str, Any]:
body: dict[str, Any] = {
"name": name,
"content": content,
"description": f"Description for {name}",
}
body.update(overrides)
return body
# ===========================================================================
# Server endpoint tests
# ===========================================================================
@@ -158,6 +172,7 @@ class TestServerListMemories:
assert r.json()["memories"][0]["name"] == "a"
def test_filter_by_scope(self, server_client, storage):
_seed_workstream(storage)
_seed_memory(storage, "a", "x", scope="global")
_seed_memory(storage, "b", "y", scope="workstream", scope_id="ws1")
r = server_client.get("/v1/api/memories?scope=workstream&scope_id=ws1")
@@ -174,12 +189,38 @@ class TestServerListMemories:
r = server_client.get("/v1/api/memories?limit=abc")
assert r.status_code == 400
def test_unscoped_list_is_caller_bound(self, server_client, storage):
_seed_memory(storage, "global_visible", "g")
_seed_memory(storage, "own_visible", "u", scope="user", scope_id="test-user")
_seed_memory(storage, "victim_user", "secret", scope="user", scope_id="victim")
_seed_memory(storage, "victim_coord", "secret", scope="coordinator", scope_id="victim")
_seed_memory(storage, "private_project", "secret", scope="project", scope_id="p1")
r = server_client.get("/v1/api/memories")
assert r.status_code == 200
assert {row["name"] for row in r.json()["memories"]} == {
"global_visible",
"own_visible",
}
def test_internal_scopes_are_rejected(self, server_client):
for scope in ("coordinator", "project", "bogus"):
r = server_client.get(f"/v1/api/memories?scope={scope}&scope_id=victim")
assert r.status_code == 400
def test_workstream_scope_is_owner_bound(self, server_client, storage):
_seed_workstream(storage, "victim-ws", "victim")
_seed_memory(storage, "secret", "x", scope="workstream", scope_id="victim-ws")
r = server_client.get("/v1/api/memories?scope=workstream&scope_id=victim-ws")
assert r.status_code == 403
class TestServerSaveMemory:
def test_create(self, server_client):
r = server_client.post(
"/v1/api/memories",
json={"name": "my_key", "content": "my content"},
json=_save_body("my_key", "my content"),
)
assert r.status_code == 201
data = r.json()
@@ -191,21 +232,23 @@ class TestServerSaveMemory:
def test_upsert(self, server_client):
server_client.post(
"/v1/api/memories",
json={"name": "key", "content": "v1"},
json=_save_body("key", "v1"),
)
r = server_client.post(
"/v1/api/memories",
json={"name": "key", "content": "v2"},
json=_save_body("key", "v2", description="Updated key description"),
)
assert r.status_code == 200
assert r.json()["content"] == "v2"
def test_with_type_and_scope(self, server_client):
def test_with_type_and_scope(self, server_client, storage):
_seed_workstream(storage)
r = server_client.post(
"/v1/api/memories",
json={
"name": "feedback_key",
"content": "data",
"description": "Feedback memory",
"type": "feedback",
"scope": "workstream",
"scope_id": "ws1",
@@ -216,17 +259,30 @@ class TestServerSaveMemory:
assert r.json()["scope"] == "workstream"
def test_missing_name(self, server_client):
r = server_client.post("/v1/api/memories", json={"content": "data"})
r = server_client.post(
"/v1/api/memories", json={"content": "data", "description": "Missing name"}
)
assert r.status_code == 400
def test_missing_content(self, server_client):
r = server_client.post("/v1/api/memories", json={"name": "k"})
r = server_client.post(
"/v1/api/memories", json={"name": "k", "description": "Missing content"}
)
assert r.status_code == 400
@pytest.mark.parametrize("description", [None, "", " "])
def test_missing_or_empty_description(self, server_client, description):
r = server_client.post(
"/v1/api/memories",
json={"name": "k", "content": "c", "description": description},
)
assert r.status_code == 400
assert "description is required" in r.json()["error"]
def test_invalid_type(self, server_client):
r = server_client.post(
"/v1/api/memories",
json={"name": "k", "content": "c", "type": "bogus"},
json=_save_body("k", "c", type="bogus"),
)
assert r.status_code == 400
assert "invalid type" in r.json()["error"]
@@ -234,7 +290,7 @@ class TestServerSaveMemory:
def test_invalid_scope(self, server_client):
r = server_client.post(
"/v1/api/memories",
json={"name": "k", "content": "c", "scope": "bogus"},
json=_save_body("k", "c", scope="bogus"),
)
assert r.status_code == 400
assert "invalid scope" in r.json()["error"]
@@ -242,7 +298,7 @@ class TestServerSaveMemory:
def test_content_too_large(self, server_client):
r = server_client.post(
"/v1/api/memories",
json={"name": "k", "content": "x" * 70000},
json=_save_body("k", "x" * 70000),
)
assert r.status_code == 400
assert "limit" in r.json()["error"]
@@ -250,18 +306,32 @@ class TestServerSaveMemory:
def test_name_normalisation(self, server_client):
r = server_client.post(
"/v1/api/memories",
json={"name": "My-Key Name", "content": "data"},
json=_save_body("My-Key Name", "data"),
)
assert r.status_code == 201
assert r.json()["name"] == "my_key_name"
def test_create_and_update_are_audited(self, server_client, storage):
first = server_client.post(
"/v1/api/memories",
json=_save_body("audit_me", "v1"),
)
second = server_client.post(
"/v1/api/memories",
json=_save_body("audit_me", "v2", description="Updated audit memory"),
)
assert first.status_code == 201
assert second.status_code == 200
assert len(storage.list_audit_events(action="memory.save", user_id="test-user")) == 1
assert len(storage.list_audit_events(action="memory.update", user_id="test-user")) == 1
class TestServerUserScopeSecurity:
def test_user_scope_binds_to_auth(self, server_client):
"""User scope auto-resolves scope_id from authenticated user."""
r = server_client.post(
"/v1/api/memories",
json={"name": "priv", "content": "secret", "scope": "user"},
json=_save_body("priv", "secret", scope="user"),
)
assert r.status_code == 201
assert r.json()["scope_id"] == "test-user"
@@ -270,7 +340,7 @@ class TestServerUserScopeSecurity:
"""Cannot access another user's memories via scope_id."""
r = server_client.post(
"/v1/api/memories",
json={"name": "x", "content": "y", "scope": "user", "scope_id": "other-user"},
json=_save_body("x", "y", scope="user", scope_id="other-user"),
)
assert r.status_code == 403
@@ -278,7 +348,7 @@ class TestServerUserScopeSecurity:
"""Passing own user_id as scope_id is allowed."""
r = server_client.post(
"/v1/api/memories",
json={"name": "x", "content": "y", "scope": "user", "scope_id": "test-user"},
json=_save_body("x", "y", scope="user", scope_id="test-user"),
)
assert r.status_code == 201
@@ -298,7 +368,7 @@ class TestServerScopeScopeIdValidation:
def test_save_global_with_scope_id_rejected(self, server_client):
r = server_client.post(
"/v1/api/memories",
json={"name": "k", "content": "c", "scope": "global", "scope_id": "ws1"},
json=_save_body("k", "c", scope="global", scope_id="ws1"),
)
assert r.status_code == 400
assert "scope_id" in r.json()["error"]
@@ -306,15 +376,16 @@ class TestServerScopeScopeIdValidation:
def test_save_workstream_without_scope_id_rejected(self, server_client):
r = server_client.post(
"/v1/api/memories",
json={"name": "k", "content": "c", "scope": "workstream"},
json=_save_body("k", "c", scope="workstream"),
)
assert r.status_code == 400
assert "scope_id is required" in r.json()["error"]
def test_save_workstream_with_scope_id_ok(self, server_client):
def test_save_workstream_with_scope_id_ok(self, server_client, storage):
_seed_workstream(storage)
r = server_client.post(
"/v1/api/memories",
json={"name": "k", "content": "c", "scope": "workstream", "scope_id": "ws1"},
json=_save_body("k", "c", scope="workstream", scope_id="ws1"),
)
assert r.status_code == 201
@@ -380,6 +451,21 @@ class TestServerSearchMemories:
r = server_client.post("/v1/api/memories/search", json={})
assert r.status_code == 400
def test_unscoped_search_is_caller_bound(self, server_client, storage):
_seed_memory(storage, "own", "needle", scope="user", scope_id="test-user")
_seed_memory(storage, "victim", "needle", scope="user", scope_id="victim")
_seed_memory(storage, "project", "needle", scope="project", scope_id="p1")
r = server_client.post("/v1/api/memories/search", json={"query": "needle"})
assert r.status_code == 200
assert {row["name"] for row in r.json()["memories"]} == {"own"}
def test_internal_scope_is_rejected(self, server_client):
r = server_client.post(
"/v1/api/memories/search",
json={"query": "x", "scope": "project", "scope_id": "p1"},
)
assert r.status_code == 400
class TestServerDeleteMemory:
def test_delete(self, server_client, storage):
@@ -393,6 +479,7 @@ class TestServerDeleteMemory:
assert r.status_code == 404
def test_delete_scoped(self, server_client, storage):
_seed_workstream(storage)
_seed_memory(storage, "k", "data", scope="workstream", scope_id="ws1")
# Wrong scope → not found
r = server_client.delete("/v1/api/memories/k")
@@ -405,6 +492,14 @@ class TestServerDeleteMemory:
r = server_client.delete("/v1/api/memories/k?scope=bogus")
assert r.status_code == 400
def test_delete_is_audited(self, server_client, storage):
mid = _seed_memory(storage, "audited")
r = server_client.delete("/v1/api/memories/audited")
assert r.status_code == 200
events = storage.list_audit_events(action="memory.delete", user_id="test-user")
assert len(events) == 1
assert events[0]["resource_id"] == mid
# ===========================================================================
# Console admin endpoint tests
@@ -492,6 +587,26 @@ class TestAdminDeleteMemory:
r = admin_client.delete("/v1/api/admin/memories/nonexistent-id")
assert r.status_code == 404
def test_no_audit_or_success_when_atomic_delete_misses(
self, admin_client, storage, monkeypatch
):
mid = _seed_memory(storage, "still_here")
monkeypatch.setattr(storage, "delete_structured_memory_by_id_returning", lambda _mid: None)
r = admin_client.delete(f"/v1/api/admin/memories/{mid}")
assert r.status_code == 404
assert storage.get_structured_memory(mid) is not None
assert storage.list_audit_events(action="memory.delete") == []
def test_storage_failure_is_500(self, admin_client, storage, monkeypatch):
def _raise(_memory_id):
raise RuntimeError("db down")
monkeypatch.setattr(storage, "delete_structured_memory_by_id_returning", _raise)
r = admin_client.delete("/v1/api/admin/memories/m1")
assert r.status_code == 500
# ===========================================================================
# Storage: delete_structured_memory_by_id
+42 -6
View File
@@ -1,7 +1,9 @@
"""Tests for turnstone.core.memory_relevance — scoring, formatting, context extraction."""
from typing import Any
from unittest.mock import patch
from turnstone.core import auth
from turnstone.core.memory_relevance import (
MemoryConfig,
build_memory_context,
@@ -298,6 +300,11 @@ def _make_session(fetch_limit: int = 5, relevance_k: int = 3, **kwargs: object):
)
def _execute_prepared_tool(session: Any, item: dict[str, Any]) -> tuple[str, str]:
item.setdefault("_principal_id", session._tool_prepare_principal_id())
return item["execute"](item)
class TestCompositionCandidateSelection:
"""Verify the query-aware candidate set in _init_system_messages."""
@@ -520,9 +527,11 @@ class TestMemorySearchToolExecution:
"""Multi-word query returns rows where ANY term matches — not all."""
from turnstone.core.memory import save_structured_memory
save_structured_memory("postgres_notes", "host=localhost port=5432")
save_structured_memory("redis_notes", "host=redis port=6379")
save_structured_memory("unrelated", "completely different")
save_structured_memory(
"postgres_notes", "host=localhost port=5432", description="Postgres notes"
)
save_structured_memory("redis_notes", "host=redis port=6379", description="Redis notes")
save_structured_memory("unrelated", "completely different", description="Unrelated notes")
session = _make_session()
item = session._prepare_memory(
@@ -532,12 +541,39 @@ class TestMemorySearchToolExecution:
# Sanity: prepare returned a search-ready dispatch (not an error item)
assert item.get("action") == "search"
call_id, msg = session._exec_memory(item)
call_id, msg = _execute_prepared_tool(session, item)
assert call_id == "call-1"
assert "postgres_notes" in msg
# Other memories don't match any query term
assert "unrelated" not in msg
def test_search_and_list_guidance_carries_the_displayed_scope(self, tmp_db, monkeypatch):
"""Follow-up guidance must not drop a project result's scope."""
from turnstone.core.memory import save_structured_memory
save_structured_memory(
"july_digest",
"project day digest",
description="July project digest",
scope="project",
scope_id="p1",
)
monkeypatch.setattr(
auth,
"resolve_project_access",
lambda *_a, **_k: auth.ProjectAccess(True, True, "P", "active"),
)
session = _make_session(user_id="u1", project_id="p1")
for args in (
{"action": "search", "query": "digest"},
{"action": "list"},
):
item = session._prepare_memory("call-1", args)
_, msg = _execute_prepared_tool(session, item)
assert "[general:project] july_digest" in msg
assert "call memory(action='get') with the displayed name and scope" in msg
class TestPerTurnSearchCache:
"""The per-turn cache spares redundant SQL across mid-turn rebuilds."""
@@ -545,7 +581,7 @@ class TestPerTurnSearchCache:
def test_repeated_search_in_same_turn_hits_cache(self, tmp_db):
from turnstone.core.memory import save_structured_memory
save_structured_memory("hello_mem", "alpha beta gamma")
save_structured_memory("hello_mem", "alpha beta gamma", description="Greeting memory")
session = _make_session()
with patch(
"turnstone.core.session.search_visible_structured_memories",
@@ -560,7 +596,7 @@ class TestPerTurnSearchCache:
def test_user_turn_invalidates_cache(self, tmp_db):
from turnstone.core.memory import save_structured_memory
save_structured_memory("hello_mem", "alpha")
save_structured_memory("hello_mem", "alpha", description="Greeting memory")
session = _make_session()
with patch(
"turnstone.core.session.search_visible_structured_memories",
+390 -81
View File
@@ -1,11 +1,4 @@
"""Phase 4: the ``project`` memory scope.
Covers construction-time access resolution (``_project_id`` / ``_project_writable``)
and its effect on recall ``_visible_scopes`` / ``_resolve_scope_id`` /
``_validate_scope`` for both interactive and coordinator sessions. The ACL is
monkeypatched (it is unit-tested in ``test_project_storage.py``); here we assert
the session wiring around it.
"""
"""Actor-scoped, live ``project`` memory authorization."""
from __future__ import annotations
@@ -35,10 +28,32 @@ def _session(**kwargs: Any) -> ChatSession:
return ChatSession(**defaults)
class TestConstructionResolvesProjectAccess:
"""Construction resolves the attached project through a single
``resolve_project_access`` call; recall is gated on read access AND a
non-archived project."""
def _execute_prepared_tool(
session: ChatSession,
item: dict[str, Any],
) -> tuple[str, str | list[dict[str, Any]]]:
item.setdefault("_principal_id", session._tool_prepare_principal_id())
return item["execute"](item)
def _project_session(
monkeypatch: pytest.MonkeyPatch,
*,
writable: bool = True,
kind: WorkstreamKind = WorkstreamKind.INTERACTIVE,
user_id: str = "u1",
) -> ChatSession:
"""Construct an attached session whose live ACL stays controllable."""
monkeypatch.setattr(
auth,
"resolve_project_access",
lambda *_a, **_k: auth.ProjectAccess(True, writable, "P", "active"),
)
return _session(user_id=user_id, ws_id="ws1", kind=kind, project_id="p1")
class TestLiveProjectAccess:
"""Each access snapshot resolves the attachment for the current actor."""
def _access(self, can_read: bool, can_write: bool, state: str = "active") -> object:
return auth.ProjectAccess(can_read, can_write, "P", state)
@@ -48,9 +63,10 @@ class TestConstructionResolvesProjectAccess:
auth, "resolve_project_access", lambda *a, **k: self._access(True, True)
)
s = _session(user_id="u1", project_id="p1")
assert s._project_id == "p1"
assert s._project_writable is True
assert s._project_name == "P"
access = s._memory_access()
assert access.project_id == "p1"
assert access.project_writable is True
assert access.project_name == "P"
def test_read_only_member(self, monkeypatch: pytest.MonkeyPatch) -> None:
# Read access but no write (e.g. a non-member reading a public project).
@@ -58,16 +74,19 @@ class TestConstructionResolvesProjectAccess:
auth, "resolve_project_access", lambda *a, **k: self._access(True, False)
)
s = _session(user_id="u1", project_id="p1")
assert s._project_id == "p1"
assert s._project_writable is False
access = s._memory_access()
assert access.project_id == "p1"
assert access.project_writable is False
def test_denied_without_access(self, monkeypatch: pytest.MonkeyPatch) -> None:
monkeypatch.setattr(
auth, "resolve_project_access", lambda *a, **k: self._access(False, False)
)
s = _session(user_id="u1", project_id="p1")
assert s._project_id == ""
assert s._project_writable is False
access = s._memory_access()
assert access.attached_project_id == "p1"
assert access.project_id == ""
assert access.project_writable is False
def test_archived_project_not_recalled(self, monkeypatch: pytest.MonkeyPatch) -> None:
# Full access but archived → not recalled (the owner still reaches it via
@@ -76,13 +95,17 @@ class TestConstructionResolvesProjectAccess:
auth, "resolve_project_access", lambda *a, **k: self._access(True, True, "archived")
)
s = _session(user_id="u1", project_id="p1")
assert s._project_id == ""
assert s._project_writable is False
access = s._memory_access()
assert access.attached_project_id == "p1"
assert access.project_id == ""
assert access.project_writable is False
def test_no_project_id_is_inert(self) -> None:
s = _session(user_id="u1")
assert s._project_id == ""
assert s._project_writable is False
access = s._memory_access()
assert access.attached_project_id == ""
assert access.project_id == ""
assert access.project_writable is False
def test_unauthenticated_never_resolves(self, monkeypatch: pytest.MonkeyPatch) -> None:
# Even if the ACL would allow it, an empty user_id short-circuits before
@@ -91,13 +114,49 @@ class TestConstructionResolvesProjectAccess:
auth, "resolve_project_access", lambda *a, **k: self._access(True, True)
)
s = _session(user_id="", project_id="p1")
assert s._project_id == ""
access = s._memory_access()
assert access.attached_project_id == "p1"
assert access.project_id == ""
def test_project_display_name_uses_explicit_principal(
self, monkeypatch: pytest.MonkeyPatch
) -> None:
seen: list[tuple[str, str]] = []
def _resolve(principal_id: str, project_id: str) -> object:
seen.append((principal_id, project_id))
return self._access(True, True)
monkeypatch.setattr(auth, "resolve_project_access", _resolve)
s = _session(user_id="owner", project_id="p1")
s._acting_user_id = "stale-turn-actor"
seen.clear() # Ignore constructor-time system-context composition.
assert s.project_name_for_principal("reconnecting-viewer") == "P"
assert seen == [("reconnecting-viewer", "p1")]
def test_project_display_name_does_not_fallback_for_empty_principal(
self, monkeypatch: pytest.MonkeyPatch
) -> None:
seen: list[tuple[str, str]] = []
def _resolve(principal_id: str, project_id: str) -> object:
seen.append((principal_id, project_id))
return self._access(True, True)
monkeypatch.setattr(auth, "resolve_project_access", _resolve)
s = _session(user_id="owner", project_id="p1")
seen.clear()
assert s.project_name_for_principal("") == ""
assert seen == []
class TestProjectRecall:
def test_interactive_visible_scopes_includes_project(self) -> None:
s = _session(user_id="u1", ws_id="ws1")
s._project_id = "p1"
def test_interactive_visible_scopes_includes_project(
self, monkeypatch: pytest.MonkeyPatch
) -> None:
s = _project_session(monkeypatch)
scopes = s._visible_scopes()
assert ("project", "p1") in scopes
assert ("global", "") in scopes
@@ -107,9 +166,10 @@ class TestProjectRecall:
s = _session(user_id="u1", ws_id="ws1")
assert all(scope != "project" for scope, _ in s._visible_scopes())
def test_coordinator_adds_project_keeps_isolation(self) -> None:
s = _session(user_id="u1", kind=WorkstreamKind.COORDINATOR)
s._project_id = "p1"
def test_coordinator_adds_project_keeps_isolation(
self, monkeypatch: pytest.MonkeyPatch
) -> None:
s = _project_session(monkeypatch, kind=WorkstreamKind.COORDINATOR)
scopes = s._visible_scopes()
assert ("coordinator", "u1") in scopes
assert ("project", "p1") in scopes
@@ -118,25 +178,24 @@ class TestProjectRecall:
def test_visible_scopes_omits_empty_project(self) -> None:
s = _session(user_id="u1", ws_id="ws1")
s._project_id = ""
assert all(scope != "project" for scope, _ in s._visible_scopes())
class TestProjectScopeResolutionAndValidation:
def test_resolve_scope_id_project(self) -> None:
s = _session(user_id="u1")
s._project_id = "p1"
def test_resolve_scope_id_project(self, monkeypatch: pytest.MonkeyPatch) -> None:
s = _project_session(monkeypatch)
assert s._resolve_scope_id("project") == "p1"
def test_validate_requires_attachment(self) -> None:
def test_validate_requires_attachment(self, monkeypatch: pytest.MonkeyPatch) -> None:
s = _session(user_id="u1")
assert s._validate_scope("project", "cid") is not None # not attached → rejected
s._project_id = "p1"
assert s._validate_scope("project", "cid") is None
attached = _project_session(monkeypatch)
assert attached._validate_scope("project", "cid") is None
def test_coordinator_allows_project_rejects_global(self) -> None:
s = _session(user_id="u1", kind=WorkstreamKind.COORDINATOR)
s._project_id = "p1"
def test_coordinator_allows_project_rejects_global(
self, monkeypatch: pytest.MonkeyPatch
) -> None:
s = _project_session(monkeypatch, kind=WorkstreamKind.COORDINATOR)
assert s._validate_scope("project", "cid") is None # project allowed for coord
assert s._validate_scope("global", "cid") is not None # global still rejected
@@ -172,78 +231,328 @@ class TestProjectInSystemContext:
class TestProjectWriteGate:
"""The save AND delete memory paths block writes to a project the session
can read but not write (a read-only member of a public project). Construction
resolves ``_project_writable``; these drive the preparer to assert the gate
actually fires (the resolution-level check lives in
``TestConstructionResolvesProjectAccess``)."""
resolves live access; these drive the preparer to assert the gate actually
fires (the resolution-level check lives in ``TestLiveProjectAccess``)."""
def _attached(self, *, writable: bool) -> ChatSession:
s = _session(user_id="u1")
s._project_id = "p1"
s._project_writable = writable
return s
def _attached(self, monkeypatch: pytest.MonkeyPatch, *, writable: bool) -> ChatSession:
return _project_session(monkeypatch, writable=writable)
def test_save_blocked_when_read_only(self) -> None:
s = self._attached(writable=False)
def test_save_blocked_when_read_only(self, monkeypatch: pytest.MonkeyPatch) -> None:
s = self._attached(monkeypatch, writable=False)
out = s._prepare_memory(
"cid", {"action": "save", "scope": "project", "name": "k", "content": "v"}
"cid",
{
"action": "save",
"scope": "project",
"name": "k",
"content": "v",
"description": "Test memory",
},
)
assert "read-only access to this project" in out.get("error", "")
assert "read-only access to the attached project" in out.get("error", "")
def test_save_allowed_when_writable(self) -> None:
s = self._attached(writable=True)
def test_save_allowed_when_writable(self, monkeypatch: pytest.MonkeyPatch) -> None:
s = self._attached(monkeypatch, writable=True)
out = s._prepare_memory(
"cid", {"action": "save", "scope": "project", "name": "k", "content": "v"}
"cid",
{
"action": "save",
"scope": "project",
"name": "k",
"content": "v",
"description": "Test memory",
},
)
assert "error" not in out
assert out.get("execute") is not None # would proceed to the save exec
def test_delete_blocked_when_read_only(self) -> None:
s = self._attached(writable=False)
def test_delete_blocked_when_read_only(self, monkeypatch: pytest.MonkeyPatch) -> None:
s = self._attached(monkeypatch, writable=False)
out = s._prepare_memory("cid", {"action": "delete", "scope": "project", "name": "k"})
assert "read-only access to this project" in out.get("error", "")
assert "read-only access to the attached project" in out.get("error", "")
def test_delete_allowed_when_writable(self) -> None:
s = self._attached(writable=True)
def test_delete_allowed_when_writable(self, monkeypatch: pytest.MonkeyPatch) -> None:
s = self._attached(monkeypatch, writable=True)
out = s._prepare_memory("cid", {"action": "delete", "scope": "project", "name": "k"})
assert "error" not in out
assert out.get("execute") is not None
class TestProjectDefaultSaveScope:
"""A writable attached project becomes the DEFAULT save scope (both kinds);
a read-only or unattached session keeps the kind default."""
class TestActingPrincipalProjectAuthority:
@staticmethod
def _tool_call(call_id: str, **arguments: Any) -> dict[str, Any]:
import json
def test_writable_project_is_default(self) -> None:
s = _session(user_id="u1")
s._project_id = "p1"
s._project_writable = True
return {
"id": call_id,
"function": {"name": "memory", "arguments": json.dumps(arguments)},
}
def test_guest_cannot_inherit_owner_project_access(
self, tmp_db: str, monkeypatch: pytest.MonkeyPatch
) -> None:
def resolve(user_id: str, _project_id: str, **_kwargs: Any) -> auth.ProjectAccess:
if user_id == "owner":
return auth.ProjectAccess(True, True, "Owner Project", "active")
return auth.ProjectAccess(False, False, "", "")
monkeypatch.setattr(auth, "resolve_project_access", resolve)
session = _session(user_id="owner", ws_id="shared", project_id="p1")
session.bind_acting_user("guest")
assert all(scope != "project" for scope, _ in session._visible_scopes())
for action in ("get", "save", "delete"):
arguments: dict[str, Any] = {
"action": action,
"name": "owner_secret",
"scope": "project",
}
if action == "save":
arguments["content"] = "guest write"
arguments["description"] = "Guest write attempt"
item = session._prepare_tool(self._tool_call(action, **arguments))
assert item["_principal_id"] == "guest"
assert "error" in item
assert "acting user cannot access" in item["error"]
def test_project_delete_revalidates_prepared_principal_and_live_acl(
self, tmp_db: str, monkeypatch: pytest.MonkeyPatch
) -> None:
from turnstone.core.memory import (
get_structured_memory_by_name,
save_structured_memory,
)
guest_access = {"value": auth.ProjectAccess(True, True, "Shared", "active")}
def resolve(user_id: str, _project_id: str, **_kwargs: Any) -> auth.ProjectAccess:
if user_id == "guest":
return guest_access["value"]
return auth.ProjectAccess(True, True, "Shared", "active")
monkeypatch.setattr(auth, "resolve_project_access", resolve)
save_structured_memory(
"shared_secret",
"keep",
description="Shared project secret",
scope="project",
scope_id="p1",
)
session = _session(user_id="owner", ws_id="shared", project_id="p1")
session.bind_acting_user("guest")
item = session._prepare_tool(
self._tool_call(
"delete",
action="delete",
name="shared_secret",
scope="project",
)
)
assert "error" not in item
assert item["_principal_id"] == "guest"
guest_access["value"] = auth.ProjectAccess(False, False, "", "")
session.bind_acting_user("owner")
_, message = _execute_prepared_tool(session, item)
assert "acting user cannot access" in message
assert get_structured_memory_by_name("shared_secret", "project", "p1") is not None
def test_archived_project_is_removed_from_live_visibility(
self, tmp_db: str, monkeypatch: pytest.MonkeyPatch
) -> None:
project_state = {"value": "active"}
monkeypatch.setattr(
auth,
"resolve_project_access",
lambda *_args, **_kwargs: auth.ProjectAccess(
True, True, "Shared", project_state["value"]
),
)
session = _session(user_id="owner", ws_id="shared", project_id="p1")
assert ("project", "p1") in session._visible_scopes()
project_state["value"] = "archived"
assert all(scope != "project" for scope, _ in session._visible_scopes())
item = session._prepare_memory(
"get", {"action": "get", "name": "anything", "scope": "project"}
)
assert "error" in item
assert "active attached project" in item["error"]
class TestProjectDefaultSaveScope:
"""An attachment is the inherited target even when it is read-only."""
def test_writable_project_is_default(self, monkeypatch: pytest.MonkeyPatch) -> None:
s = _project_session(monkeypatch)
assert s._default_memory_scope() == "project"
def test_read_only_project_keeps_kind_default(self) -> None:
s = _session(user_id="u1")
s._project_id = "p1"
s._project_writable = False
assert s._default_memory_scope() == "global"
def test_read_only_project_remains_inherited_target(
self, monkeypatch: pytest.MonkeyPatch
) -> None:
s = _project_session(monkeypatch, writable=False)
assert s._default_memory_scope() == "project"
def test_no_project_keeps_kind_default(self) -> None:
assert _session(user_id="u1")._default_memory_scope() == "global"
def test_coordinator_writable_project_is_default(self) -> None:
s = _session(user_id="u1", kind=WorkstreamKind.COORDINATOR)
s._project_id = "p1"
s._project_writable = True
def test_coordinator_writable_project_is_default(self, monkeypatch: pytest.MonkeyPatch) -> None:
s = _project_session(monkeypatch, kind=WorkstreamKind.COORDINATOR)
assert s._default_memory_scope() == "project"
def test_coordinator_without_project_is_coordinator(self) -> None:
s = _session(user_id="u1", kind=WorkstreamKind.COORDINATOR)
assert s._default_memory_scope() == "coordinator"
def test_save_without_scope_lands_in_project(self) -> None:
def test_save_without_scope_lands_in_project(self, monkeypatch: pytest.MonkeyPatch) -> None:
# End-to-end: an unscoped save in a writable-project session resolves to
# scope=project / scope_id=project_id (not the global default).
s = _session(user_id="u1")
s._project_id = "p1"
s._project_writable = True
out = s._prepare_memory("cid", {"action": "save", "name": "k", "content": "v"})
s = _project_session(monkeypatch)
out = s._prepare_memory(
"cid",
{
"action": "save",
"name": "k",
"content": "v",
"description": "Test memory",
},
)
assert out.get("scope") == "project"
assert out.get("scope_id") == "p1"
class TestProjectDefaultGetDeleteScope:
"""An attached project is the inherited get/delete target.
This aligns the name-based lifecycle: a memory saved without an explicit
scope can be fetched or removed the same way while the workstream remains
attached to that project.
"""
@staticmethod
def _attached(
monkeypatch: pytest.MonkeyPatch,
*,
writable: bool = True,
kind: WorkstreamKind = WorkstreamKind.INTERACTIVE,
) -> ChatSession:
return _project_session(monkeypatch, writable=writable, kind=kind)
def test_get_without_scope_targets_project(self, monkeypatch: pytest.MonkeyPatch) -> None:
# Read access is sufficient for the inherited get target; writability
# only controls save/delete.
s = self._attached(monkeypatch, writable=False)
item = s._prepare_memory("cid", {"action": "get", "name": "k"})
assert item["scopes_to_try"] == [("project", "p1")]
def test_delete_without_scope_targets_writable_project(
self, monkeypatch: pytest.MonkeyPatch
) -> None:
s = self._attached(monkeypatch)
item = s._prepare_memory("cid", {"action": "delete", "name": "k"})
assert item["scopes_to_try"] == [("project", "p1")]
def test_delete_without_scope_rejects_read_only_project(
self, monkeypatch: pytest.MonkeyPatch
) -> None:
s = self._attached(monkeypatch, writable=False)
item = s._prepare_memory("cid", {"action": "delete", "name": "k"})
assert "read-only access to the attached project" in item.get("error", "")
def test_read_only_project_does_not_block_explicit_other_scope(
self, monkeypatch: pytest.MonkeyPatch
) -> None:
s = self._attached(monkeypatch, writable=False)
item = s._prepare_memory(
"cid",
{"action": "delete", "name": "k", "scope": "global"},
)
assert "error" not in item
assert item["scopes_to_try"] == [("global", "")]
def test_coordinator_inherits_project_too(self, monkeypatch: pytest.MonkeyPatch) -> None:
s = self._attached(monkeypatch, kind=WorkstreamKind.COORDINATOR)
get_item = s._prepare_memory("get", {"action": "get", "name": "k"})
delete_item = s._prepare_memory("delete", {"action": "delete", "name": "k"})
assert get_item["scopes_to_try"] == [("project", "p1")]
assert delete_item["scopes_to_try"] == [("project", "p1")]
def test_unscoped_get_and_delete_round_trip_project_memory(
self, tmp_db: str, monkeypatch: pytest.MonkeyPatch
) -> None:
from turnstone.core.memory import (
get_structured_memory_by_name,
save_structured_memory,
)
row, _ = save_structured_memory(
"july_digest",
"full digest",
description="July digest",
scope="project",
scope_id="p1",
)
assert row is not None
s = self._attached(monkeypatch)
get_item = s._prepare_memory("get", {"action": "get", "name": "july_digest"})
_, get_msg = _execute_prepared_tool(s, get_item)
assert "[general:project] july_digest" in get_msg
assert "full digest" in get_msg
delete_item = s._prepare_memory("delete", {"action": "delete", "name": "july_digest"})
_, delete_msg = _execute_prepared_tool(s, delete_item)
assert "Deleted memory 'july_digest' (scope=project)" in delete_msg
assert get_structured_memory_by_name("july_digest", "project", "p1") is None
def test_wrong_explicit_scope_hints_at_attached_project(
self, tmp_db: str, monkeypatch: pytest.MonkeyPatch
) -> None:
from turnstone.core.memory import save_structured_memory
row, _ = save_structured_memory(
"july_digest",
"full digest",
description="July digest",
scope="project",
scope_id="p1",
)
assert row is not None
s = self._attached(monkeypatch)
for action in ("get", "delete"):
item = s._prepare_memory(
action,
{"action": action, "name": "july_digest", "scope": "global"},
)
_, msg = _execute_prepared_tool(s, item)
assert "not found (scope=global)" in msg
assert "exists in scope='project'" in msg
assert "retry with scope='project'" in msg
def test_project_default_miss_hints_at_other_visible_scope(
self, tmp_db: str, monkeypatch: pytest.MonkeyPatch
) -> None:
from turnstone.core.memory import save_structured_memory
row, _ = save_structured_memory(
"shared_runbook",
"global content",
description="Shared runbook",
scope="global",
)
assert row is not None
s = self._attached(monkeypatch)
for action in ("get", "delete"):
item = s._prepare_memory(
action,
{"action": action, "name": "shared_runbook"},
)
_, msg = _execute_prepared_tool(s, item)
assert "not found (scope=project)" in msg
assert "exists in scope='global'" in msg
assert "retry with scope='global'" in msg
+3 -3
View File
@@ -68,9 +68,9 @@ class TestProjectStore:
# a sibling project's nor other scopes' rows.
backend.create_project("p1", "A", "u1")
backend.create_project("p2", "B", "u1")
backend.create_structured_memory("m1", "k", "", "general", "project", "p1", "v")
backend.create_structured_memory("m2", "k", "", "general", "project", "p2", "v")
backend.create_structured_memory("m3", "k", "", "general", "user", "u1", "v")
backend.create_structured_memory("m1", "k", "Test memory", "general", "project", "p1", "v")
backend.create_structured_memory("m2", "k", "Test memory", "general", "project", "p2", "v")
backend.create_structured_memory("m3", "k", "Test memory", "general", "user", "u1", "v")
assert backend.delete_project("p1")
assert backend.get_structured_memory("m1") is None # purged
assert backend.get_structured_memory("m2") is not None # sibling project intact
+56
View File
@@ -361,6 +361,62 @@ async def test_logout():
assert resp.status == "ok"
# ---------------------------------------------------------------------------
# Memories
# ---------------------------------------------------------------------------
@pytest.mark.anyio
async def test_save_memory_requires_and_sends_description():
captured: dict = {}
def handler(request: httpx.Request) -> httpx.Response:
captured.update(json.loads(request.content))
return _json_response(
{
"memory_id": "m1",
"name": "deployment_process",
"description": captured["description"],
"type": "general",
"scope": "global",
"scope_id": "",
"content": "Deploy from main",
"created": "2026-08-11T00:00:00",
"updated": "2026-08-11T00:00:00",
},
status=201,
)
transport = httpx.MockTransport(handler)
async with httpx.AsyncClient(transport=transport, base_url="http://test") as hc:
client = AsyncTurnstoneServer(httpx_client=hc)
memory = await client.save_memory(
"deployment_process",
"Deploy from main",
description=" Production deployment workflow ",
)
assert captured["description"] == "Production deployment workflow"
assert memory.description == "Production deployment workflow"
@pytest.mark.anyio
@pytest.mark.parametrize("description", [None, "", " "])
async def test_save_memory_rejects_empty_description(description):
def unexpected_request(_request: httpx.Request) -> httpx.Response:
raise AssertionError("invalid memory must not reach the server")
transport = httpx.MockTransport(unexpected_request)
async with httpx.AsyncClient(transport=transport, base_url="http://test") as hc:
client = AsyncTurnstoneServer(httpx_client=hc)
with pytest.raises(ValueError, match="description is required"):
await client.save_memory(
"deployment_process",
"Deploy from main",
description=description, # type: ignore[arg-type]
)
# ---------------------------------------------------------------------------
# Health
# ---------------------------------------------------------------------------
+219 -56
View File
@@ -1,17 +1,19 @@
"""Offline pins of the SDK boundary behaviors the #937 retry design rests on.
Three facts, each probed against the REAL SDKs over mock/loopback
Four facts, each probed against the REAL SDKs over mock/loopback
transports (no network, no live backend):
1. The OpenAI SDK's ``max_retries`` covers request time only — a
mid-BODY death produces no re-request, and the raw ``httpx.ReadError``
escapes the chunk iterator unwrapped.
2. The Anthropic ``messages.stream()`` helper propagates the same shape.
3. Closing an httpx-backed SDK client from another thread while a read is
blocked (the ``ModelRegistry.reload()`` shape) surfaces as an
``httpx.TransportError`` on the blocked ``next()`` which is why
``_stream_response``'s mid-stream re-issue ladder re-resolves the
registry binding before re-creating.
1. OpenAI v3's ``max_retries`` covers request time only — a mid-BODY death
produces no re-request, and the raw ``httpx2.ReadError`` escapes both Chat
Completions and Responses chunk iterators unwrapped.
2. OpenAI v3's runtime-only legacy-client path preserves the old ``httpx``
exception family when an application explicitly injects that client.
3. The Anthropic ``messages.stream()`` helper propagates the ``httpx`` shape.
4. Closing an OpenAI v3 default client from another thread while a read is
blocked (the ``ModelRegistry.reload()`` shape) completes safely; a later
wire release surfaces as an ``httpx2.TransportError`` on the blocked
``next()``. The production ``transport_guarded`` seam must normalize it
before the retry gate.
If an SDK/httpx upgrade changes any of these, the ``transport_guarded``
conversion (and the retry gate consuming it) must be re-verified these
@@ -27,6 +29,7 @@ import time
import anthropic
import httpx
import httpx2
import openai
import pytest
@@ -39,6 +42,12 @@ CHAT_CHUNK = (
'"finish_reason":null}]}' + LF + LF
)
RESPONSES_EVENT = (
'data: {"type":"response.output_text.delta","sequence_number":0,'
'"item_id":"item_1","output_index":0,"content_index":0,'
'"delta":"hello","logprobs":[]}' + LF + LF
)
ANTHROPIC_EVENTS = (
"event: message_start"
+ LF
@@ -71,6 +80,17 @@ class _DyingStream(httpx.SyncByteStream):
raise httpx.ReadError("[SSL] record layer failure (_ssl.c:2590)")
class _Httpx2DyingStream(httpx2.SyncByteStream):
"""HTTPX2 response body: one SSE payload, then a wire death."""
def __init__(self, payload: bytes) -> None:
self._payload = payload
def __iter__(self):
yield self._payload
raise httpx2.ReadError("[SSL] record layer failure (_ssl.c:2590)")
def _dying_transport(payload: str, requests: list) -> httpx.MockTransport:
def handler(request: httpx.Request) -> httpx.Response:
requests.append(request)
@@ -84,12 +104,70 @@ def _dying_transport(payload: str, requests: list) -> httpx.MockTransport:
return httpx.MockTransport(handler)
def test_openai_midbody_death_is_unwrapped_readerror_and_no_rerequest():
def _httpx2_dying_transport(payload: str, requests: list) -> httpx2.MockTransport:
def handler(request: httpx2.Request) -> httpx2.Response:
requests.append(request)
return httpx2.Response(
200,
headers={"content-type": "text/event-stream"},
stream=_Httpx2DyingStream(payload.encode()),
request=request,
)
return httpx2.MockTransport(handler)
def test_openai_v3_chat_midbody_death_is_unwrapped_httpx2_error_and_no_rerequest():
requests: list = []
client = openai.OpenAI(
api_key="probe",
base_url="http://probe.invalid/v1",
http_client=httpx.Client(transport=_dying_transport(CHAT_CHUNK, requests)),
http_client=httpx2.Client(transport=_httpx2_dying_transport(CHAT_CHUNK, requests)),
max_retries=2,
)
stream = client.chat.completions.create(
model="m", messages=[{"role": "user", "content": "hi"}], stream=True
)
texts = []
with pytest.raises(httpx2.ReadError) as excinfo:
for chunk in stream:
if chunk.choices and chunk.choices[0].delta.content:
texts.append(chunk.choices[0].delta.content)
# The retry gate matches on the class NAME; pin the exact identity the
# SDK lets escape, and that it is the HTTPX2 transport family.
assert type(excinfo.value).__name__ == "ReadError"
assert isinstance(excinfo.value, httpx2.TransportError)
assert texts == ["hello"] # the request succeeded; the BODY died
assert len(requests) == 1 # max_retries never re-requested mid-body
def test_openai_v3_responses_midbody_death_is_unwrapped_httpx2_error_and_no_rerequest():
requests: list = []
client = openai.OpenAI(
api_key="probe",
base_url="http://probe.invalid/v1",
http_client=httpx2.Client(transport=_httpx2_dying_transport(RESPONSES_EVENT, requests)),
max_retries=2,
)
stream = client.responses.create(model="m", input="hi", stream=True)
texts = []
with pytest.raises(httpx2.ReadError) as excinfo:
for event in stream:
if event.type == "response.output_text.delta":
texts.append(event.delta)
assert type(excinfo.value).__name__ == "ReadError"
assert isinstance(excinfo.value, httpx2.TransportError)
assert texts == ["hello"]
assert len(requests) == 1
def test_openai_v3_legacy_httpx_midbody_death_keeps_legacy_error_family():
requests: list = []
legacy_http_client = httpx.Client(transport=_dying_transport(CHAT_CHUNK, requests))
client = openai.OpenAI(
api_key="probe",
base_url="http://probe.invalid/v1",
http_client=legacy_http_client, # type: ignore[arg-type]
max_retries=2,
)
stream = client.chat.completions.create(
@@ -100,12 +178,10 @@ def test_openai_midbody_death_is_unwrapped_readerror_and_no_rerequest():
for chunk in stream:
if chunk.choices and chunk.choices[0].delta.content:
texts.append(chunk.choices[0].delta.content)
# The retry gate matches on the class NAME; pin the exact identity the
# SDK lets escape, and that it is the httpx transport family.
assert type(excinfo.value).__name__ == "ReadError"
assert isinstance(excinfo.value, httpx.TransportError)
assert texts == ["hello"] # the request succeeded; the BODY died
assert len(requests) == 1 # max_retries never re-requested mid-body
assert texts == ["hello"]
assert len(requests) == 1
def test_anthropic_midbody_death_is_unwrapped_readerror_and_no_rerequest():
@@ -136,13 +212,59 @@ def test_anthropic_midbody_death_is_unwrapped_readerror_and_no_rerequest():
assert len(requests) == 1
def test_cross_thread_client_close_surfaces_transport_error_to_blocked_read():
"""The ``ModelRegistry.reload()`` shape: an admin-thread ``client.close()``
under a worker blocked in ``next()`` must surface as an
``httpx.TransportError`` (so ``transport_guarded`` converts it and the
retry gate passes) not as a plain ``RuntimeError`` the gate would
treat as fatal."""
body = f"{len(CHAT_CHUNK):x}" + CRLF + CHAT_CHUNK + CRLF
@pytest.mark.parametrize("surface", ["chat", "responses"])
def test_closed_v3_default_client_creation_stays_sdk_wrapped_and_unarmed(surface: str):
"""A re-create on the client closed by reload is still a creation error.
OpenAI v3 must wrap the default HTTPX2 client's ``RuntimeError`` as its
retryable ``APIConnectionError`` before either adapter can arm the stream.
This preserves the creation-vs-mid-stream classifier while the original
normalized stream death remains available to the outer re-issue ladder.
"""
from turnstone.core.providers import create_provider
client = openai.OpenAI(
api_key="probe",
base_url="http://probe.invalid/v1",
max_retries=0,
)
client.close()
cancel_ref: list = []
provider = create_provider("openai-compatible", api_surface=surface)
with pytest.raises(openai.APIConnectionError) as excinfo:
provider.create_streaming(
client=client,
model="m",
messages=[{"role": "user", "content": "hi"}],
cancel_ref=cancel_ref,
)
assert cancel_ref == []
assert type(excinfo.value).__name__ in provider.retryable_error_names
assert type(excinfo.value.__cause__) is RuntimeError
@pytest.mark.parametrize(
("surface", "payload"),
[("chat", CHAT_CHUNK), ("responses", RESPONSES_EVENT)],
)
def test_cross_thread_v3_default_client_close_then_wire_release_is_normalized(
surface: str, payload: str
):
"""The ``ModelRegistry.reload()`` shape stays safe at the retry seam.
OpenAI v3's synchronous HTTPX2 client does not promise that cross-thread
``close()`` itself interrupts a blocked body read. Pin the behavior
Turnstone needs instead: closing from the admin thread completes safely
while a worker is in ``next()``, and the subsequent wire release reaches
``transport_guarded`` as a provider-retryable ``IncompleteStreamError``
for both OpenAI streaming adapters.
"""
from turnstone.core.providers import create_provider, transport_guarded
from turnstone.core.providers._protocol import IncompleteStreamError
body = f"{len(payload):x}" + CRLF + payload + CRLF
response_head = (
"HTTP/1.1 200 OK"
+ CRLF
@@ -153,59 +275,100 @@ def test_cross_thread_client_close_surfaces_transport_error_to_blocked_read():
+ CRLF
)
listener = socket.create_server(("127.0.0.1", 0))
listener.settimeout(10.0)
port = listener.getsockname()[1]
client_closed = threading.Event()
release_peer = threading.Event()
reader_blocked = threading.Event()
close_done = threading.Event()
first_content: list[str] = []
reader_errors: list[BaseException] = []
closer_errors: list[BaseException] = []
server_errors: list[BaseException] = []
def serve() -> None:
conn, _ = listener.accept()
conn.recv(65536)
conn.sendall((response_head + body).encode())
# Hold the connection (no second chunk) so the reader blocks, and
# release only once the client has been closed under it.
client_closed.wait(timeout=10.0)
conn.close()
try:
conn, _ = listener.accept()
with conn:
conn.settimeout(10.0)
conn.recv(65536)
conn.sendall((response_head + body).encode())
# Keep the peer open through close_done: any reader error
# before release_peer is therefore caused by close(), not EOF.
if not release_peer.wait(timeout=15.0):
raise AssertionError("peer release was never signalled")
except BaseException as exc:
server_errors.append(exc)
client = openai.OpenAI(api_key="probe", base_url=f"http://127.0.0.1:{port}/v1", max_retries=0)
client = openai.OpenAI(
api_key="probe",
base_url=f"http://127.0.0.1:{port}/v1",
max_retries=0,
timeout=5.0,
)
def read_stream() -> None:
try:
provider = create_provider("openai-compatible", api_surface=surface)
chunks = provider.create_streaming(
client=client,
model="m",
messages=[{"role": "user", "content": "hi"}],
)
it = transport_guarded(chunks)
first_content.append(next(it).content_delta)
reader_blocked.set()
next(it)
except BaseException as exc:
reader_errors.append(exc)
def closer() -> None:
reader_blocked.wait(timeout=10.0)
time.sleep(0.5) # let the reader enter the blocking socket read
client.close()
client_closed.set()
try:
if not reader_blocked.wait(timeout=10.0):
raise AssertionError("reader never reached the blocked body read")
time.sleep(0.5) # let the reader enter the blocking socket read
client.close()
except BaseException as exc:
closer_errors.append(exc)
finally:
close_done.set()
server_thread = threading.Thread(target=serve)
reader_thread = threading.Thread(target=read_stream)
closer_thread = threading.Thread(target=closer)
server_thread.start()
reader_thread.start()
closer_thread.start()
try:
stream = client.chat.completions.create(
model="m", messages=[{"role": "user", "content": "hi"}], stream=True
)
it = iter(stream)
first = next(it) # the one sent chunk arrives; the wire then idles
assert first.choices[0].delta.content == "hello"
reader_blocked.set()
with pytest.raises(httpx.TransportError) as excinfo:
next(it) # blocked read, killed by the cross-thread close()
# Platform/timing-dependent: the killed read surfaces as ReadError
# (EBADF from the blocked recv) or, where the reader observes EOF
# first, RemoteProtocolError (chunked body never terminated). Both
# are TransportError members of _BACKEND_STREAM_EXC_NAMES, so
# transport_guarded converts either and the retry gate passes — the
# property this pin exists for.
assert type(excinfo.value).__name__ in {"ReadError", "RemoteProtocolError"}
assert reader_blocked.wait(timeout=10.0)
assert close_done.wait(timeout=10.0)
assert closer_errors == []
# The server has not closed its peer yet, proving close() completed
# safely rather than merely returning after an EOF unblocked it.
assert server_errors == []
release_peer.set()
reader_thread.join(timeout=10.0)
finally:
# Unblock and join both threads on every exit path so nothing
# Unblock and join every thread on every exit path so nothing
# outlives the test (leaked-thread guard).
reader_blocked.set()
client_closed.set()
release_peer.set()
closer_thread.join(timeout=10.0)
reader_thread.join(timeout=10.0)
server_thread.join(timeout=10.0)
listener.close()
with contextlib.suppress(Exception):
client.close()
assert first_content == ["hello"]
assert len(reader_errors) == 1
exc = reader_errors[0]
assert isinstance(exc, IncompleteStreamError)
assert any(name in str(exc) for name in ("ReadError", "RemoteProtocolError"))
cause = exc.__cause__
assert isinstance(cause, httpx2.TransportError)
assert type(cause).__name__ in {"ReadError", "RemoteProtocolError"}
assert server_errors == []
assert not closer_thread.is_alive()
assert not reader_thread.is_alive()
assert not server_thread.is_alive()
@@ -240,7 +403,7 @@ class TestEagerAppendContract:
requests: list = []
client = openai.OpenAI(
api_key="probe",
http_client=httpx.Client(transport=_dying_transport(CHAT_CHUNK, requests)),
http_client=httpx2.Client(transport=_httpx2_dying_transport(CHAT_CHUNK, requests)),
)
self._armed_at_return(OpenAIChatCompletionsProvider(), client)
assert len(requests) == 1 # the HTTP call happened inside create
@@ -251,7 +414,7 @@ class TestEagerAppendContract:
requests: list = []
client = openai.OpenAI(
api_key="probe",
http_client=httpx.Client(transport=_dying_transport(CHAT_CHUNK, requests)),
http_client=httpx2.Client(transport=_httpx2_dying_transport(RESPONSES_EVENT, requests)),
)
self._armed_at_return(OpenAIResponsesProvider(), client)
assert len(requests) == 1
+9 -11
View File
@@ -1435,28 +1435,26 @@ class TestInteractiveEventsLifted:
section.
"""
def test_events_replay_yields_connected_first(self):
"""Pre-lift ``events_sse`` yielded a ``connected`` event
first (model + skip_permissions). The lifted callback
preserves the order so client SSE handlers that key on
the connected event for state setup keep working."""
from turnstone.server import _interactive_events_replay
def test_shared_preamble_yields_connected_first(self):
"""The shared handler's preamble preserves connected-event shape."""
from turnstone.core.session_replay import session_replay_preamble
ws, ui, request = _make_interactive_replay_mocks()
out = list(_interactive_events_replay(ws, ui, request))
out = list(session_replay_preamble(ws.session, ui, project_name="Visible Project"))
assert out[0]["type"] == "connected"
assert out[0]["model"] == "gpt-5"
assert out[0]["model_alias"] == "default"
assert out[0]["project_name"] == "Visible Project"
assert out[0]["skip_permissions"] is False
def test_events_replay_includes_status_only_when_last_usage_present(self):
def test_shared_preamble_includes_status_only_when_last_usage_present(self):
"""The ``status`` event populates the per-tab token-usage
bar on resume. Skipped when ``session._last_usage`` is None
(a freshly-created workstream that hasn't completed a turn)."""
from turnstone.server import _interactive_events_replay
from turnstone.core.session_replay import session_replay_preamble
ws, ui, request = _make_interactive_replay_mocks()
out = list(_interactive_events_replay(ws, ui, request))
ws, ui, _request = _make_interactive_replay_mocks()
out = list(session_replay_preamble(ws.session, ui))
assert "status" not in {ev["type"] for ev in out}
def test_events_replay_yields_pending_approval_then_verdicts(self):
+403 -72
View File
@@ -29,7 +29,12 @@ from turnstone.core.model_turn import (
provider_extra_params,
serialized_tool_chars,
)
from turnstone.core.session import _IMAGE_EXTENSIONS, _IMAGE_SIZE_CAP, ChatSession
from turnstone.core.session import (
_IMAGE_EXTENSIONS,
_IMAGE_SIZE_CAP,
_MEMORY_MIXED_BATCH_ERROR,
ChatSession,
)
from turnstone.core.trajectory import (
Role,
Turn,
@@ -129,6 +134,15 @@ def _make_session(
return session
def _execute_prepared_tool(
session: ChatSession,
item: dict[str, Any],
) -> tuple[str, str | list[dict[str, Any]]]:
"""Mirror the dispatch boundary for tests that call a preparer directly."""
item.setdefault("_principal_id", session._tool_prepare_principal_id())
return item["execute"](item)
@contextlib.contextmanager
def _send_with_mocks(session, responses, mock_execute, **extra_patches):
"""Stand up the mock context that the queued-message ``send()`` tests share.
@@ -458,12 +472,20 @@ class TestTaskExec:
"func_name": "task_agent",
"needs_approval": True,
"execute": execute,
"_needs_origin_context": True,
"_requires_fresh_system_prefix": True,
}
def prepare(_tool_call):
seen["prepare"] = session._tool_prepare_principal_id()
return item
judge = MagicMock(side_effect=evaluate_intent)
with (
patch.object(session, "_safe_prepare_tool", return_value=item),
patch.object(session, "_safe_prepare_tool", side_effect=prepare),
patch.object(session, "_evaluate_intent", judge),
patch.object(session.ui, "approve_tools", side_effect=approve_tools),
patch.object(session, "_ensure_system_prefix_fresh") as ensure_system_prefix,
):
session._execute_tools(
[{"id": "c1", "function": {"name": "task_agent", "arguments": "{}"}}],
@@ -471,9 +493,14 @@ class TestTaskExec:
my_generation=generation,
)
assert seen["prepare"] == "user-a"
assert seen["worker"] == "user-a"
assert seen["generation"] == generation
assert seen["event"] is generation_event
ensure_system_prefix.assert_called_once_with(
principal_id="user-a",
origin_generation=generation,
)
assert judge.call_args.kwargs["principal_id"] == "user-a"
assert seen["execution_item"] is not item
approval_witness = seen["approval_item"]["_approval_cancel_witness"]
@@ -6199,6 +6226,96 @@ class TestSafePrepareTool:
assert "RuntimeError" in output
class TestToolBatchPolicy:
@staticmethod
def _tool_calls(count: int) -> list[dict[str, Any]]:
return [
{
"id": f"call_{index}",
"function": {"name": "state_tool", "arguments": "{}"},
}
for index in range(count)
]
def test_mixed_read_write_batch_is_rejected(self, tmp_db):
session = _make_session()
executed: list[str] = []
def execute(item):
executed.append(item["call_id"])
return item["call_id"], "unexpected"
items = [
{
"call_id": "call_0",
"func_name": "state_tool",
"execute": execute,
"needs_approval": False,
"_batch_policy": {
"group": "state",
"access": "write",
"mixed_access_error": "Error: state reads and writes cannot run together",
"serialize": True,
},
},
{
"call_id": "call_1",
"func_name": "state_tool",
"execute": execute,
"needs_approval": False,
"_batch_policy": {
"group": "state",
"access": "read",
"mixed_access_error": "Error: state reads and writes cannot run together",
},
},
]
with (
patch.object(session, "_safe_prepare_tool", side_effect=items),
patch.object(session.ui, "approve_tools", return_value=(True, None)),
):
results, _ = session._execute_tools(self._tool_calls(2))
assert executed == []
assert all("cannot run together" in str(result) for _, result in results)
def test_write_batch_executes_serially_in_model_order(self, tmp_db):
session = _make_session()
executed: list[str] = []
def execute(item):
executed.append(item["call_id"])
return item["call_id"], "ok"
items = [
{
"call_id": f"call_{index}",
"func_name": "state_tool",
"execute": execute,
"needs_approval": False,
"_batch_policy": {
"group": "state",
"access": "write",
"mixed_access_error": "Error: mixed state access",
"serialize": True,
},
}
for index in range(2)
]
with (
patch.object(session, "_safe_prepare_tool", side_effect=items),
patch.object(session.ui, "approve_tools", return_value=(True, None)),
patch(
"turnstone.core.session.concurrent.futures.ThreadPoolExecutor",
side_effect=AssertionError("serialized writes must not enter the parallel pool"),
),
):
results, _ = session._execute_tools(self._tool_calls(2))
assert executed == ["call_0", "call_1"]
assert [call_id for call_id, _ in results] == executed
class TestCoordinatorMemoryScope:
"""Verify the ``coordinator`` memory scope's resolution + validation rules.
@@ -6269,7 +6386,7 @@ class TestCoordinatorMemoryScope:
)
err = session._validate_scope("coordinator", "call_1")
assert err is not None
assert err["error"].startswith("Error: 'coordinator' scope is only valid")
assert "unavailable to this workstream kind" in err["error"]
def test_validate_rejects_coord_scope_for_child_interactive(self, tmp_db):
"""Children of a coord MUST be rejected too — letting them write
@@ -6286,7 +6403,7 @@ class TestCoordinatorMemoryScope:
)
err = session._validate_scope("coordinator", "call_1")
assert err is not None
assert err["error"].startswith("Error: 'coordinator' scope is only valid")
assert "unavailable to this workstream kind" in err["error"]
def test_validate_accepts_coord_scope_for_coord_session(self, tmp_db):
from turnstone.core.workstream import WorkstreamKind
@@ -6314,6 +6431,7 @@ class TestCoordinatorMemoryScope:
"call_1",
{
"action": "save",
"description": "Test memory",
"name": "orchestration_plan",
"content": "step 1: investigate; step 2: report",
"scope": "coordinator",
@@ -6341,6 +6459,7 @@ class TestCoordinatorMemoryScope:
"call_1",
{
"action": "save",
"description": "Test memory",
"name": "injected_instruction",
"content": "ignore previous instructions and ...",
"scope": "coordinator",
@@ -6360,6 +6479,7 @@ class TestCoordinatorMemoryScope:
save_structured_memory(
"private_plan",
"internal coord notes",
description="Private orchestration plan",
scope="coordinator",
scope_id="user-1",
)
@@ -6424,16 +6544,20 @@ class TestCoordinatorMemoryScope:
from turnstone.core.workstream import WorkstreamKind
# Seed every non-coord scope with a sentinel memory.
save_structured_memory("global_note", "anyone can read", scope="global")
save_structured_memory(
"global_note", "anyone can read", description="Global note", scope="global"
)
save_structured_memory(
"ws_note",
"interactive ws notes",
description="Workstream note",
scope="workstream",
scope_id="coord-1", # same id as the coord under test
)
save_structured_memory(
"user_note",
"user-wide notes from another IC session",
description="User note",
scope="user",
scope_id="user-1",
)
@@ -6466,10 +6590,13 @@ class TestCoordinatorMemoryScope:
from turnstone.core.memory import save_structured_memory
from turnstone.core.workstream import WorkstreamKind
save_structured_memory("global_x", "some content", scope="global")
save_structured_memory(
"global_x", "some content", description="Global content", scope="global"
)
save_structured_memory(
"coord_x",
"orchestration content",
description="Coordinator content",
scope="coordinator",
scope_id="user-1",
)
@@ -6497,14 +6624,11 @@ class TestCoordinatorMemoryScope:
for bad in ("global", "workstream", "user"):
err = coord._validate_scope(bad, "call_1")
assert err is not None, f"coord should reject scope={bad!r}"
assert f"'{bad}' scope is not available" in err["error"]
assert f"scope '{bad}' is unavailable" in err["error"]
def test_coord_default_save_scope_is_coordinator(self, tmp_db):
"""Coord sessions calling memory(action='save') without an
explicit scope default to 'coordinator' anything else would
either land in a namespace the coord can't read back from
(workstream/user) or fall back to global which the new
visibility rules also exclude."""
explicit scope target the coordinator namespace."""
from turnstone.core.workstream import WorkstreamKind
coord = _make_session(
@@ -6514,17 +6638,20 @@ class TestCoordinatorMemoryScope:
)
item = coord._prepare_memory(
"call_1",
{"action": "save", "name": "auto_scope", "content": "x"},
{
"action": "save",
"name": "auto_scope",
"content": "x",
"description": "Automatic scope test",
},
)
assert "error" not in item
assert item["scope"] == "coordinator"
assert item["scope_id"] == "user-1"
def test_coord_implicit_walk_only_coordinator(self, tmp_db):
def test_coord_inherited_get_targets_only_coordinator(self, tmp_db):
"""Coord ``memory(action='get')`` with no explicit scope must
walk only the coordinator scope the IC walk
(workstream user global) would be wasted lookups against
rows the coord can't see."""
target only the coordinator scope."""
from turnstone.core.workstream import WorkstreamKind
coord = _make_session(
@@ -6539,10 +6666,8 @@ class TestCoordinatorMemoryScope:
assert "error" not in item
assert [s for s, _ in item["scopes_to_try"]] == ["coordinator"]
def test_ic_implicit_walk_unchanged(self, tmp_db):
"""Interactive sessions retain the narrowest-to-widest walk:
workstream user global. Coord scope is excluded IC
sessions can't see/write it anyway."""
def test_ic_unscoped_get_uses_single_global_target(self, tmp_db):
"""Without a project, interactive save/get/delete all inherit global."""
from turnstone.core.workstream import WorkstreamKind
ic = _make_session(
@@ -6555,8 +6680,7 @@ class TestCoordinatorMemoryScope:
{"action": "get", "name": "anything"},
)
assert "error" not in item
scopes = [s for s, _ in item["scopes_to_try"]]
assert scopes == ["workstream", "user", "global"]
assert item["scopes_to_try"] == [("global", "")]
def test_coord_memory_persists_across_sessions(self, tmp_db):
"""End-to-end through the real save lane: a memory saved by one
@@ -6575,13 +6699,14 @@ class TestCoordinatorMemoryScope:
"call_1",
{
"action": "save",
"description": "Test memory",
"name": "deploy_runbook",
"content": "drain node before rotating certs",
"scope": "coordinator",
},
)
assert "error" not in item
result = item["execute"](item)
result = _execute_prepared_tool(first, item)
assert "Saved" in str(result) or "saved" in str(result).lower()
# Brand-new coordinator session, new ws_id, same user.
@@ -6595,7 +6720,7 @@ class TestCoordinatorMemoryScope:
{"action": "get", "name": "deploy_runbook"},
)
assert "error" not in get_item
out = str(get_item["execute"](get_item))
out = str(_execute_prepared_tool(second, get_item))
assert "drain node before rotating certs" in out
def test_coordinator_session_requires_user_id(self, tmp_db):
@@ -6636,11 +6761,17 @@ class TestCoordinatorMemoryScope:
coord._user_id = "" # simulate a constructor-bypassing double
err = coord._validate_scope("coordinator", "call_1")
assert err is not None
assert "requires authenticated user identity" in err["error"]
assert "requires an authenticated acting user" in err["error"]
assert coord._coordinator_scope_id() == ""
item = coord._prepare_memory(
"call_1",
{"action": "save", "name": "x", "content": "y", "scope": "coordinator"},
{
"action": "save",
"name": "x",
"content": "y",
"description": "Authentication backstop test",
"scope": "coordinator",
},
)
assert "error" in item
@@ -6654,6 +6785,7 @@ class TestCoordinatorMemoryScope:
save_structured_memory(
"other_users_row",
"must not leak",
description="Another user's row",
scope="coordinator",
scope_id="user-9",
)
@@ -6682,7 +6814,48 @@ class TestMemoryToolAudit:
return get_storage().list_audit_events(action=action)
def test_save_new_emits_memory_save(self, tmp_db):
def test_preparer_declares_generic_batch_policy(self, tmp_db):
session = _make_session(ws_id="ws-1", user_id="user-1")
saved = session._prepare_memory(
"save-call",
{
"action": "save",
"name": "fact_one",
"content": "alpha content",
"description": "Alpha fact",
},
)
fetched = session._prepare_memory(
"get-call",
{"action": "get", "name": "fact_one"},
)
assert saved["_batch_policy"] == {
"group": "memory",
"access": "write",
"mixed_access_error": _MEMORY_MIXED_BATCH_ERROR,
"serialize": True,
}
assert fetched["_batch_policy"] == {
"group": "memory",
"access": "read",
"mixed_access_error": _MEMORY_MIXED_BATCH_ERROR,
"serialize": False,
}
def test_unstamped_executor_does_not_inherit_session_actor(self, tmp_db):
session = _make_session(ws_id="ws-1", user_id="user-1")
item = session._prepare_memory(
"get-call",
{"action": "get", "name": "private_fact", "scope": "user"},
)
_call_id, message = item["execute"](item)
assert "requires an authenticated acting user" in message
@pytest.mark.parametrize("description", [None, "", " "])
def test_save_requires_non_empty_description(self, tmp_db, description):
session = _make_session(ws_id="ws-1", user_id="user-1")
item = session._prepare_memory(
"call_1",
@@ -6690,12 +6863,27 @@ class TestMemoryToolAudit:
"action": "save",
"name": "fact_one",
"content": "alpha content",
"description": description,
},
)
assert "error" in item
assert "description' must be non-empty" in item["error"]
def test_save_new_emits_memory_save(self, tmp_db):
session = _make_session(ws_id="ws-1", user_id="user-1")
item = session._prepare_memory(
"call_1",
{
"action": "save",
"description": "Test memory",
"name": "fact_one",
"content": "alpha content",
"scope": "user",
"type": "reference",
},
)
assert "error" not in item
session._exec_memory(item)
_execute_prepared_tool(session, item)
rows = self._audit_rows("memory.save")
assert len(rows) == 1
@@ -6712,6 +6900,71 @@ class TestMemoryToolAudit:
# The "create" path must NOT also stamp an update row.
assert self._audit_rows("memory.update") == []
def test_prepared_user_save_stays_bound_to_acting_principal(self, tmp_db):
from turnstone.core.memory import get_structured_memory_by_name
session = _make_session(ws_id="shared", user_id="owner")
session.bind_acting_user("guest")
item = session._prepare_tool(
{
"id": "call_1",
"function": {
"name": "memory",
"arguments": json.dumps(
{
"action": "save",
"description": "Test memory",
"name": "private_note",
"content": "guest content",
"scope": "user",
}
),
},
}
)
assert item["_principal_id"] == "guest"
assert item["scope_id"] == "guest"
session.bind_acting_user("owner")
_, message = _execute_prepared_tool(session, item)
assert "Saved memory" in message
assert get_structured_memory_by_name("private_note", "user", "guest") is not None
assert get_structured_memory_by_name("private_note", "user", "owner") is None
rows = self._audit_rows("memory.save")
assert len(rows) == 1
assert rows[0]["user_id"] == "guest"
def test_guest_user_get_does_not_probe_owner_namespace(self, tmp_db):
from turnstone.core.memory import save_structured_memory
save_structured_memory(
"owner_secret",
"must not leak",
description="Owner secret",
scope="user",
scope_id="owner",
)
session = _make_session(ws_id="shared", user_id="owner")
session.bind_acting_user("guest")
item = session._prepare_tool(
{
"id": "call_1",
"function": {
"name": "memory",
"arguments": json.dumps(
{"action": "get", "name": "owner_secret", "scope": "user"}
),
},
}
)
_, message = _execute_prepared_tool(session, item)
assert "not found" in message
assert "must not leak" not in message
assert "exists in scope" not in message
def test_save_global_scope_emits_empty_scope_id(self, tmp_db):
"""Global memories have no scope_id — the audit row's detail
must still carry the key (with value ``""``) so a forensic
@@ -6722,13 +6975,14 @@ class TestMemoryToolAudit:
"call_1",
{
"action": "save",
"description": "Test memory",
"name": "fact_global",
"content": "shared content",
"scope": "global",
},
)
assert "error" not in item
session._exec_memory(item)
_execute_prepared_tool(session, item)
rows = self._audit_rows("memory.save")
assert len(rows) == 1
@@ -6744,13 +6998,14 @@ class TestMemoryToolAudit:
"call_x",
{
"action": "save",
"description": "Test memory",
"name": "fact_one",
"content": content,
"scope": "user",
"type": "reference",
},
)
session._exec_memory(item)
_execute_prepared_tool(session, item)
saves = self._audit_rows("memory.save")
updates = self._audit_rows("memory.update")
@@ -6765,20 +7020,21 @@ class TestMemoryToolAudit:
"call_1",
{
"action": "save",
"description": "Test memory",
"name": "fact_one",
"content": "alpha",
"scope": "user",
"type": "reference",
},
)
session._exec_memory(save_item)
_execute_prepared_tool(session, save_item)
saved_memory_id = self._audit_rows("memory.save")[0]["resource_id"]
delete_item = session._prepare_memory(
"call_2",
{"action": "delete", "name": "fact_one", "scope": "user"},
)
_, msg = session._exec_memory(delete_item)
_, msg = _execute_prepared_tool(session, delete_item)
assert "Deleted memory" in msg
rows = self._audit_rows("memory.delete")
@@ -6797,23 +7053,70 @@ class TestMemoryToolAudit:
"call_1",
{"action": "delete", "name": "no_such_mem", "scope": "user"},
)
_, msg = session._exec_memory(delete_item)
_, msg = _execute_prepared_tool(session, delete_item)
assert "not found" in msg
assert self._audit_rows("memory.delete") == []
def test_committed_delete_is_truthful_and_next_prefix_refresh_fails_closed(self, tmp_db):
from turnstone.core.memory import get_structured_memory_by_name, save_structured_memory
save_structured_memory("doomed", "value", description="Memory to delete", scope="global")
session = _make_session(ws_id="ws-1", user_id="user-1")
item = session._prepare_memory(
"call_1", {"action": "delete", "name": "doomed", "scope": "global"}
)
_, message = _execute_prepared_tool(session, item)
assert "Deleted memory 'doomed'" in message
assert get_structured_memory_by_name("doomed", "global", "") is None
assert len(self._audit_rows("memory.delete")) == 1
assert session._system_prefix_dirty is True
with (
patch.object(
session,
"_init_system_messages",
side_effect=RuntimeError("composition failed"),
),
pytest.raises(RuntimeError, match="composition failed"),
):
session._ensure_system_prefix_fresh()
def test_storage_failure_is_not_reported_as_not_found(self, tmp_db):
from turnstone.core.storage import get_storage
session = _make_session(ws_id="ws-1", user_id="user-1")
storage = get_storage()
operations = (
(
"get_structured_memory_by_name",
{"action": "get", "name": "key", "scope": "global"},
),
(
"delete_structured_memory_returning",
{"action": "delete", "name": "key", "scope": "global"},
),
)
for method_name, arguments in operations:
item = session._prepare_memory("call_1", arguments)
with patch.object(storage, method_name, side_effect=RuntimeError("db down")):
_, message = _execute_prepared_tool(session, item)
assert "storage operation failed" in message
assert "not found" not in message
def test_reads_emit_no_audit(self, tmp_db):
session = _make_session(ws_id="ws-1", user_id="user-1")
session._exec_memory(
session._prepare_memory(
"call_save",
{
"action": "save",
"name": "fact_one",
"content": "alpha",
"scope": "user",
},
)
save_item = session._prepare_memory(
"call_save",
{
"action": "save",
"description": "Test memory",
"name": "fact_one",
"content": "alpha",
"scope": "user",
},
)
_execute_prepared_tool(session, save_item)
for spec in (
{"action": "get", "name": "fact_one", "scope": "user"},
@@ -6822,7 +7125,7 @@ class TestMemoryToolAudit:
):
item = session._prepare_memory("call_read", spec)
assert "error" not in item
session._exec_memory(item)
_execute_prepared_tool(session, item)
# Only the save above should have audited.
save_count = len(self._audit_rows("memory.save"))
@@ -6842,6 +7145,7 @@ class TestMemoryToolAudit:
"call_1",
{
"action": "save",
"description": "Test memory",
"name": "fact_one",
"content": "alpha",
"scope": "user",
@@ -6851,7 +7155,7 @@ class TestMemoryToolAudit:
"turnstone.core.audit.record_audit",
side_effect=RuntimeError("audit storage exploded"),
):
_, msg = session._exec_memory(item)
_, msg = _execute_prepared_tool(session, item)
assert "Saved memory 'fact_one'" in msg
# The save itself still landed.
from turnstone.core.memory import get_structured_memory_by_name
@@ -6863,10 +7167,10 @@ class TestPerKindToolVariants:
"""Verify the ``kind_variants`` metadata applies per-kind tool overrides.
Each kind sees only the tool surface it can actually use the
coord sees ``scope`` enum ``["coordinator"]`` and a coord-flavored
description; the IC sees ``["global", "workstream", "user"]`` and
the existing IC-flavored description. The union ``TOOLS`` list
keeps the full schema for introspection / docs / eval catalogs.
coord sees coordinator/project scopes and a coord-flavored description;
the IC sees global/workstream/user/project and the IC-flavored description.
The union ``TOOLS`` list keeps the full schema for introspection / docs /
eval catalogs.
"""
def test_coord_memory_tool_has_coord_only_scope_enum(self):
@@ -6877,6 +7181,10 @@ class TestPerKindToolVariants:
# v1.7: a coordinator attached to a project also reads/writes the shared
# 'project' scope, alongside its isolated 'coordinator' namespace.
assert scope["enum"] == ["coordinator", "project"]
scope_desc = scope["description"]
assert "Save/get/delete without scope target project when attached" in scope_desc
assert "otherwise coordinator" in scope_desc
assert "valid explicit scope selects exactly that scope" in scope_desc
def test_coord_memory_tool_description_mentions_orchestration(self):
from turnstone.core.tools import COORDINATOR_TOOLS
@@ -6896,6 +7204,10 @@ class TestPerKindToolVariants:
scope = memory["function"]["parameters"]["properties"]["scope"]
# v1.7: 'project' is offered (usable when the workstream is attached).
assert scope["enum"] == ["global", "workstream", "user", "project"]
scope_desc = scope["description"]
assert "Save/get/delete without scope target project when attached" in scope_desc
assert "otherwise global" in scope_desc
assert "valid explicit scope selects exactly that scope" in scope_desc
def test_ic_memory_tool_description_omits_coord_scope(self):
from turnstone.core.tools import INTERACTIVE_TOOLS
@@ -7024,7 +7336,12 @@ class TestMemoryAccessTouch:
def _save(name: str, content: str) -> None:
from turnstone.core.memory import save_structured_memory
save_structured_memory(name, content, scope="global")
save_structured_memory(
name,
content,
description=f"Test memory for {name}",
scope="global",
)
@staticmethod
def _empty_session() -> ChatSession:
@@ -7134,7 +7451,7 @@ class TestMemoryAccessTouch:
self._save("kafka_runbook", "restart the kafka broker pods")
item = session._prepare_memory("call_1", {"action": "search", "query": "kafka"})
assert "error" not in item
session._exec_memory(item)
_execute_prepared_tool(session, item)
assert self._access_count("kafka_runbook") == 1
def test_get_action_touches_fetched_memory(self, tmp_db):
@@ -7144,7 +7461,7 @@ class TestMemoryAccessTouch:
"call_1", {"action": "get", "name": "kafka_runbook", "scope": "global"}
)
assert "error" not in item
session._exec_memory(item)
_execute_prepared_tool(session, item)
assert self._access_count("kafka_runbook") == 1
def test_get_miss_touches_nothing(self, tmp_db):
@@ -7153,7 +7470,7 @@ class TestMemoryAccessTouch:
item = session._prepare_memory(
"call_1", {"action": "get", "name": "no_such_mem", "scope": "global"}
)
_, msg = session._exec_memory(item)
_, msg = _execute_prepared_tool(session, item)
assert "not found" in msg
# The existing row must not be collaterally touched by a miss.
assert self._access_count("kafka_runbook") == 0
@@ -7162,7 +7479,7 @@ class TestMemoryAccessTouch:
session = self._empty_session()
self._save("kafka_runbook", "restart the kafka broker pods")
item = session._prepare_memory("call_1", {"action": "list"})
session._exec_memory(item)
_execute_prepared_tool(session, item)
assert self._access_count("kafka_runbook") == 0
def test_save_action_does_not_touch_access_count(self, tmp_db):
@@ -7174,9 +7491,15 @@ class TestMemoryAccessTouch:
session = self._empty_session()
item = session._prepare_memory(
"call_1",
{"action": "save", "name": "kafka_runbook", "content": "x", "scope": "global"},
{
"action": "save",
"name": "kafka_runbook",
"content": "x",
"description": "Kafka runbook",
"scope": "global",
},
)
session._exec_memory(item)
_execute_prepared_tool(session, item)
assert self._access_count("kafka_runbook") == 0
def test_save_through_exec_does_not_recompose_prefix(self, tmp_db):
@@ -7202,13 +7525,14 @@ class TestMemoryAccessTouch:
"call_1",
{
"action": "save",
"description": "Test memory",
"name": "kafka_scaling",
"content": "restart kafka and scale the broker pods cluster",
"scope": "global",
},
)
assert "error" not in item
session._exec_memory(item)
_execute_prepared_tool(session, item)
# 1. Prefix byte-for-byte unchanged -> no prompt-cache bust.
after = "\n".join(m["content"] for m in session.system_messages if m["role"] == "system")
@@ -7226,11 +7550,8 @@ class TestMemoryAccessTouch:
)
assert '<memory name="kafka_scaling"' in recomposed
def test_save_through_tool_preserves_omitted_overwrites_explicit(self, tmp_db):
"""The None-sentinel flows through _prepare_memory -> _exec_memory: a
content-only re-save keeps the stored type/description, while an
explicit field overwrites it. Guards the _prepare_memory omit->None
logic that the storage-level tests don't exercise."""
def test_save_through_tool_requires_and_updates_description(self, tmp_db):
"""Every tool save describes the row; an omitted type stays preserved."""
from turnstone.core.memory import get_structured_memory_by_name
session = self._empty_session()
@@ -7246,48 +7567,58 @@ class TestMemoryAccessTouch:
},
)
assert "error" not in item
session._exec_memory(item)
_execute_prepared_tool(session, item)
# Content-only re-save (omits type/description) -> both preserved.
# An update supplies a fresh description while omitting type.
item2 = session._prepare_memory(
"c2", {"action": "save", "name": "digest", "content": "v2", "scope": "global"}
"c2",
{
"action": "save",
"name": "digest",
"content": "v2",
"description": "revised daily digest",
"scope": "global",
},
)
session._exec_memory(item2)
_execute_prepared_tool(session, item2)
mem = get_structured_memory_by_name("digest", "global", "")
assert mem is not None
assert mem["content"] == "v2"
assert mem["type"] == "reference"
assert mem["description"] == "daily digest"
assert mem["description"] == "revised daily digest"
# An invalid/typo'd type is treated as unset -> stored type preserved,
# not silently downgraded to "general".
# Invalid/typo'd types fail preparation and do not mutate the row.
item_bad = session._prepare_memory(
"c2b",
{
"action": "save",
"description": "Test memory",
"name": "digest",
"content": "v2b",
"type": "nonsense",
"scope": "global",
},
)
session._exec_memory(item_bad)
assert "error" in item_bad
assert "invalid memory type" in item_bad["error"]
mem = get_structured_memory_by_name("digest", "global", "")
assert mem is not None
assert mem["type"] == "reference" # invalid type ignored, not downgraded
assert mem["content"] == "v2"
assert mem["type"] == "reference"
# An explicit field -> overwrites (the behaviour the None-sentinel enables).
item3 = session._prepare_memory(
"c3",
{
"action": "save",
"description": "Test memory",
"name": "digest",
"content": "v3",
"type": "general",
"scope": "global",
},
)
session._exec_memory(item3)
_execute_prepared_tool(session, item3)
mem = get_structured_memory_by_name("digest", "global", "")
assert mem is not None
assert mem["type"] == "general"
+2 -2
View File
@@ -1,6 +1,6 @@
"""Tests for :meth:`ChatSession._format_backend_error`.
The helper turns bare backend-boundary exceptions (httpx ``ReadTimeout``,
The helper turns bare backend-boundary exceptions (HTTPX/HTTPX2 ``ReadTimeout``,
OpenAI SDK ``APITimeoutError`` / ``APIConnectionError`` /
``NotFoundError`` / ``RateLimitError`` / ``AuthenticationError``) into
operator-actionable messages that include the provider, base URL, and
@@ -225,7 +225,7 @@ def test_rate_limit_with_overflow_phrasing_is_not_mislabeled_overflow():
def _stream_death_exemplars() -> list[BaseException]:
"""One realistic instance per name in ``_BACKEND_STREAM_EXC_NAMES``:
the normalized shape the guarded iterators raise, plus the raw httpx
the normalized shape the guarded iterators raise, plus the raw HTTPX-family
names for any future unguarded path."""
from turnstone.core.providers import IncompleteStreamError
+4 -3
View File
@@ -124,9 +124,10 @@ def test_nonfork_resume_rebinds_project_memory_context_before_recomposition(tmp_
assert session.resume("target-ws") is True
assert session.ws_id == "target-ws"
assert session._project_id == "target-project"
assert session._project_name == "Target Project"
assert session._project_writable is True
access = session._memory_access()
assert access.project_id == "target-project"
assert access.project_name == "Target Project"
assert access.project_writable is True
assert ("project", "target-project") in session._visible_scopes()
assert ("project", "source-project") not in session._visible_scopes()
assert stale_cache_key not in session._mem_search_cache
+2
View File
@@ -46,6 +46,7 @@ _ESM_BUNDLES = [
_SHARED / "auth.js",
_SHARED / "renderer.js",
_SHARED / "status_bar.js",
_SHARED / "composer_paste_text.js",
_SHARED / "composer.js",
_SHARED / "composer_attachments.js",
_SHARED / "composer_queue.js",
@@ -73,6 +74,7 @@ _ESM_NO_VAR_BUNDLES = [
_SHARED / "toast.js",
_SHARED / "kb.js",
_SHARED / "auth.js",
_SHARED / "composer_paste_text.js",
_SHARED / "interactive.js",
_SHARED / "conversation.js",
_SHARED / "preview.js",
+9 -8
View File
@@ -6,6 +6,7 @@ hint pattern, and the skill catalog disclosure in system messages.
from __future__ import annotations
import threading
from typing import Any
from unittest.mock import MagicMock, patch
@@ -1375,15 +1376,14 @@ class TestSkillCatalogDisclosure:
session._tools = []
session._client_type = ClientType.CLI
session._username = ""
# _init_system_messages renders the attached project into the Session
# Context; this __new__-built session skips __init__'s project resolution,
# so seed the (unattached) defaults it reads.
session._project_name = ""
session._project_id = ""
session._project_writable = False
# This __new__-built session skips __init__'s attachment setup.
session._memory_attached_project_id = ""
session._system_prefix_lock = threading.RLock()
session._system_prefix_dirty = True
session._system_prefix_signature = None
session._kind = "interactive"
# Persona snapshot attrs (set by __init__, bypassed here) — legacy
# defaults: no override, unrestricted tools, MCP + memory on.
# Persona snapshot attrs (set by __init__, bypassed here): open
# defaults with no override, unrestricted tools, MCP + memory on.
session._persona_name = ""
session._persona_prompt = ""
session._persona_tools = None
@@ -1393,6 +1393,7 @@ class TestSkillCatalogDisclosure:
session._memory_config = MagicMock()
session._memory_config.fetch_limit = 0
session._user_id = "test-user"
session._acting_user_id = ""
# _init_system_messages -> _recompute_shared_state reads the session
# owner (_mcp_user_id) to decide shared-workstream framing; __init__
# normally sets it from user_id, so seed it here for the __new__ build.
+93 -6
View File
@@ -53,6 +53,7 @@ def _fake_request(
headers: dict[str, str] | None = None,
query: dict[str, str] | None = None,
path_params: dict[str, str] | None = None,
user_id: str = "viewer-1",
) -> Request:
"""Construct a Starlette ``Request`` for the events handler.
@@ -80,7 +81,9 @@ def _fake_request(
async def _recv() -> dict[str, Any]: # noqa: RUF029 — async signature required
return {"type": "http.disconnect"}
return Request(scope, receive=_recv)
request = Request(scope, receive=_recv)
request.state.auth_result = SimpleNS(user_id=user_id)
return request
# ---------------------------------------------------------------------------
@@ -673,6 +676,87 @@ def test_handler_emits_retry_on_first_yield() -> None:
assert 2500 <= retry <= 4500, f"retry {retry} outside jitter band [2500, 4500]"
def test_handler_resolves_project_off_loop_and_preserves_legacy_replay_callback() -> None:
"""Project metadata uses the viewer off-loop; replay keeps its 3-arg API."""
from turnstone.core.session_replay import (
request_replay_project_name,
session_replay_preamble,
)
event_loop_thread = threading.get_ident()
resolver_threads: list[int] = []
principals: list[str] = []
callback_calls: list[tuple[Any, Any, Any]] = []
session = MagicMock()
session.model = "test"
session.model_alias = ""
session._last_usage = None
def _project_name(principal_id: str) -> str:
resolver_threads.append(threading.get_ident())
principals.append(principal_id)
return "Visible Project"
session.project_name_for_principal.side_effect = _project_name
def _replay(ws: Any, ui: Any, request: Any) -> Any:
callback_calls.append((ws, ui, request))
yield from session_replay_preamble(
ws.session,
ui,
project_name=request_replay_project_name(request),
)
yield {"type": "tail"}
ui = _make_ui()
_, blob = _drain_handler_yields(
ui,
session=session,
events_replay=_replay,
max_yields=4,
)
assert principals == ["viewer-1"]
assert resolver_threads and resolver_threads[0] != event_loop_thread
assert len(callback_calls) == 1
assert '"type": "connected"' in blob
assert '"project_name": "Visible Project"' in blob
assert '"type": "tail"' in blob
def test_handler_project_lookup_failure_keeps_connected_event() -> None:
"""Optional project metadata fails closed without losing bootstrap."""
from turnstone.core.session_replay import (
request_replay_project_name,
session_replay_preamble,
)
session = MagicMock()
session.model = "test"
session.model_alias = ""
session._last_usage = None
session.project_name_for_principal.side_effect = RuntimeError("storage unavailable")
def _replay(ws: Any, ui: Any, request: Any) -> Any:
yield from session_replay_preamble(
ws.session,
ui,
project_name=request_replay_project_name(request),
)
ui = _make_ui()
_, blob = _drain_handler_yields(
ui,
session=session,
events_replay=_replay,
max_yields=3,
)
assert '"type": "connected"' in blob
assert '"project_name": ""' in blob
def test_handler_replay_ok_skips_snapshot_emits_id(monkeypatch: Any) -> None:
"""``Last-Event-ID`` + buffer covers gap → emit buffered events
with SSE ``id:`` field, SKIP the in-progress snapshot (it would
@@ -982,6 +1066,11 @@ class _HandoffSession:
self.context_window = 1000
self.reasoning_effort = "low"
self._last_usage = {"prompt_tokens": 12, "completion_tokens": 0}
self.project_name_principals: list[str] = []
def project_name_for_principal(self, principal_id: str) -> str:
self.project_name_principals.append(principal_id)
return ""
def register_listener_for_history_handoff(
self,
@@ -1093,11 +1182,8 @@ def test_handoff_cursor_replay_keeps_preamble_without_pending_control_duplicate(
session = _HandoffSession(ui)
def _full_replay(_ws: Any, _ui: Any, _request: Any) -> Any:
# The full replay's own preamble half is what the replay_ok path
# must NOT re-run; the lifted body calls the shared
# session_replay_preamble directly instead (no per-kind hook).
yield {"type": "connected", "model": "test"}
yield {"type": "status", "total_tokens": 12}
# The kind-specific tail must NOT run on replay_ok; the lifted body
# calls the shared preamble directly and the ring owns controls.
yield {"type": "approve_request", "items": [{"call_id": "duplicate"}]}
_, blob = _drain_handler_yields(
@@ -1110,6 +1196,7 @@ def test_handoff_cursor_replay_keeps_preamble_without_pending_control_duplicate(
)
assert session.calls == [(session.token, 0)]
assert session.project_name_principals == ["viewer-1"]
assert '"type": "connected"' in blob
assert '"type": "status"' in blob
assert '"type": "state_change"' in blob
+72 -30
View File
@@ -1,5 +1,7 @@
"""Tests for turnstone.core.memory — structured memory facade functions."""
import pytest
from turnstone.core.memory import (
count_structured_memories,
delete_structured_memory,
@@ -7,31 +9,71 @@ from turnstone.core.memory import (
list_structured_memories,
normalize_key,
save_structured_memory,
save_structured_memory_strict,
search_structured_memories,
)
def _save(name, content, **kwargs):
kwargs.setdefault("description", "test memory description")
return save_structured_memory(name, content, **kwargs)
class TestSaveStructuredMemory:
@pytest.mark.parametrize("save", [save_structured_memory, save_structured_memory_strict])
@pytest.mark.parametrize("description", [None, "", " "])
def test_description_is_required(self, tmp_db, description, save):
with pytest.raises(ValueError, match="description is required"):
save("test_key", "hello world", description=description)
def test_best_effort_save_still_swallows_storage_failures(self, tmp_db, monkeypatch):
from turnstone.core import memory as memory_mod
class _BoomStorage:
def upsert_structured_memory(self, *_args, **_kwargs):
raise RuntimeError("simulated storage failure")
monkeypatch.setattr(memory_mod, "get_storage", lambda: _BoomStorage())
assert save_structured_memory(
"test_key",
"hello world",
description="Test memory",
) == (None, False)
def test_best_effort_save_does_not_misclassify_backend_value_error(self, tmp_db, monkeypatch):
from turnstone.core import memory as memory_mod
class _BoomStorage:
def upsert_structured_memory(self, *_args, **_kwargs):
raise ValueError("backend decode failure")
monkeypatch.setattr(memory_mod, "get_storage", lambda: _BoomStorage())
assert save_structured_memory(
"test_key",
"hello world",
description="Test memory",
) == (None, False)
def test_save_new(self, tmp_db):
row, was_update = save_structured_memory("test_key", "hello world")
row, was_update = _save("test_key", "hello world")
assert row and row["memory_id"]
assert was_update is False
def test_save_upsert(self, tmp_db):
row1, was_update1 = save_structured_memory("test_key", "first")
row2, was_update2 = save_structured_memory("test_key", "second")
row1, was_update1 = _save("test_key", "first")
row2, was_update2 = _save("test_key", "second")
assert was_update1 is False
assert was_update2 is True
assert row2 and row1 and row2["memory_id"] == row1["memory_id"] # same row
assert row2["content"] == "second"
def test_save_normalizes_key(self, tmp_db):
save_structured_memory("My-Key", "value")
_save("My-Key", "value")
mems = list_structured_memories()
assert any(m["name"] == "my_key" for m in mems)
def test_save_with_type_and_scope(self, tmp_db):
save_structured_memory("k", "v", mem_type="user", scope="workstream", scope_id="ws1")
_save("k", "v", mem_type="user", scope="workstream", scope_id="ws1")
mems = list_structured_memories(scope="workstream", scope_id="ws1")
assert len(mems) == 1
assert mems[0]["type"] == "user"
@@ -39,14 +81,14 @@ class TestSaveStructuredMemory:
class TestDeleteStructuredMemory:
def test_delete_existing(self, tmp_db):
save_structured_memory("mykey", "val")
_save("mykey", "val")
assert delete_structured_memory("mykey")
def test_delete_nonexistent(self, tmp_db):
assert not delete_structured_memory("nope")
def test_delete_normalizes_key(self, tmp_db):
save_structured_memory("my_key", "val")
_save("my_key", "val")
assert delete_structured_memory("My-Key")
@@ -55,25 +97,25 @@ class TestListStructuredMemories:
assert list_structured_memories() == []
def test_list_returns_saved(self, tmp_db):
save_structured_memory("a", "alpha")
save_structured_memory("b", "beta")
_save("a", "alpha")
_save("b", "beta")
mems = list_structured_memories()
assert len(mems) == 2
class TestSearchStructuredMemories:
def test_search_finds_match(self, tmp_db):
save_structured_memory("db_host", "localhost", description="database hostname")
save_structured_memory("api_url", "http://example.com")
_save("db_host", "localhost", description="database hostname")
_save("api_url", "http://example.com")
results = search_structured_memories("database")
assert len(results) >= 1
assert any(r["name"] == "db_host" for r in results)
def test_multiword_or_matches_partial(self, tmp_db):
"""OR-of-terms: memory matching only 1 of 3 query terms is returned."""
save_structured_memory("postgres_config", "host=localhost port=5432")
save_structured_memory("redis_config", "host=redis port=6379")
save_structured_memory("unrelated", "nothing relevant here")
_save("postgres_config", "host=localhost port=5432")
_save("redis_config", "host=redis port=6379")
_save("unrelated", "nothing relevant here")
# "postgres missing_word_a missing_word_b": only postgres_config matches "postgres"
results = search_structured_memories("postgres missing_word_a missing_word_b")
@@ -83,9 +125,9 @@ class TestSearchStructuredMemories:
def test_multiword_or_multiple_partial_matches(self, tmp_db):
"""Multiple memories each matching different terms are all returned."""
save_structured_memory("key_alpha", "alpha content here")
save_structured_memory("key_beta", "beta content here")
save_structured_memory("key_other", "completely different")
_save("key_alpha", "alpha content here")
_save("key_beta", "beta content here")
_save("key_other", "completely different")
results = search_structured_memories("alpha beta")
names = {r["name"] for r in results}
@@ -95,9 +137,9 @@ class TestSearchStructuredMemories:
def test_search_scope_filtering_preserved(self, tmp_db):
"""Search with scope filter only returns memories in that scope."""
save_structured_memory("ws1_fact", "alpha info", scope="workstream", scope_id="ws1")
save_structured_memory("ws2_fact", "alpha info", scope="workstream", scope_id="ws2")
save_structured_memory("global_fact", "alpha info", scope="global")
_save("ws1_fact", "alpha info", scope="workstream", scope_id="ws1")
_save("ws2_fact", "alpha info", scope="workstream", scope_id="ws2")
_save("global_fact", "alpha info", scope="global")
results = search_structured_memories("alpha", scope="workstream", scope_id="ws1")
names = {r["name"] for r in results}
@@ -108,7 +150,7 @@ class TestSearchStructuredMemories:
class TestGetStructuredMemoryByName:
def test_get_existing(self, tmp_db):
save_structured_memory("my_mem", "full content here that is quite long")
_save("my_mem", "full content here that is quite long")
mem = get_structured_memory_by_name("my_mem", "global", "")
assert mem is not None
assert mem["content"] == "full content here that is quite long"
@@ -118,12 +160,12 @@ class TestGetStructuredMemoryByName:
assert get_structured_memory_by_name("nope", "global", "") is None
def test_get_wrong_scope(self, tmp_db):
save_structured_memory("ws_mem", "data", scope="workstream", scope_id="ws1")
_save("ws_mem", "data", scope="workstream", scope_id="ws1")
assert get_structured_memory_by_name("ws_mem", "global", "") is None
assert get_structured_memory_by_name("ws_mem", "workstream", "ws1") is not None
def test_get_normalizes_key(self, tmp_db):
save_structured_memory("My-Key", "value")
_save("My-Key", "value")
mem = get_structured_memory_by_name("My-Key", "global", "")
assert mem is not None
assert mem["name"] == "my_key"
@@ -134,8 +176,8 @@ class TestCountStructuredMemories:
assert count_structured_memories() == 0
def test_count_after_save(self, tmp_db):
save_structured_memory("a", "1")
save_structured_memory("b", "2")
_save("a", "1")
_save("b", "2")
assert count_structured_memories() == 2
@@ -154,11 +196,11 @@ class TestScopeIsolation:
def _seed(self):
"""Create memories across multiple scopes."""
save_structured_memory("global_note", "visible to all", scope="global")
save_structured_memory("ws1_note", "belongs to ws1", scope="workstream", scope_id="ws1")
save_structured_memory("ws2_note", "belongs to ws2", scope="workstream", scope_id="ws2")
save_structured_memory("u1_note", "belongs to user1", scope="user", scope_id="u1")
save_structured_memory("u2_note", "belongs to user2", scope="user", scope_id="u2")
_save("global_note", "visible to all", scope="global")
_save("ws1_note", "belongs to ws1", scope="workstream", scope_id="ws1")
_save("ws2_note", "belongs to ws2", scope="workstream", scope_id="ws2")
_save("u1_note", "belongs to user1", scope="user", scope_id="u1")
_save("u2_note", "belongs to user2", scope="user", scope_id="u2")
@staticmethod
def _list_visible(ws_id: str, user_id: str, mem_type: str = "", limit: int = 50):
+212 -54
View File
@@ -2,6 +2,15 @@
class TestCreateAndGet:
def test_create_requires_non_empty_description(self, backend):
import pytest
for description in (None, "", " "):
with pytest.raises(ValueError, match="description is required"):
backend.create_structured_memory(
"m1", "test_key", description, "general", "global", "", "data"
)
def test_create_and_get_by_id(self, backend):
backend.create_structured_memory("m1", "test_key", "desc", "general", "global", "", "data")
mem = backend.get_structured_memory("m1")
@@ -43,34 +52,44 @@ class TestSaveUpsert:
import pytest
import sqlalchemy as sa
backend.create_structured_memory("m1", "dup", "", "general", "global", "", "a")
backend.create_structured_memory("m1", "dup", "Test memory", "general", "global", "", "a")
with pytest.raises(sa.exc.IntegrityError):
backend.create_structured_memory("m2", "dup", "", "general", "global", "", "b")
backend.create_structured_memory(
"m2", "dup", "Test memory", "general", "global", "", "b"
)
def test_save_same_key_updates_in_place(self, backend):
from turnstone.core.memory import save_structured_memory
row1, was_update1 = save_structured_memory("upsert_key", "v1", scope="global")
row1, was_update1 = save_structured_memory(
"upsert_key", "v1", description="first description", scope="global"
)
assert row1 and was_update1 is False # inserted
row2, was_update2 = save_structured_memory("upsert_key", "v2", scope="global")
row2, was_update2 = save_structured_memory(
"upsert_key", "v2", description="updated description", scope="global"
)
assert row2 and was_update2 is True # updated in place
assert row2["memory_id"] == row1["memory_id"] # same row, not a duplicate
assert row2["content"] == "v2"
names = [r["name"] for r in backend.list_structured_memories(scope="global")]
assert names.count("upsert_key") == 1
def test_save_same_key_preserves_description_and_type_on_default_resave(self, backend):
from turnstone.core.memory import save_structured_memory
def test_save_same_key_requires_and_updates_description(self, backend):
from turnstone.core.memory import save_structured_memory, save_structured_memory_strict
save_structured_memory(
"meta_key", "c1", description="orig desc", mem_type="fact", scope="global"
)
# A re-save that omits description/type (defaults) must not clobber them.
save_structured_memory("meta_key", "c2", scope="global")
# Every update must describe the revised memory; type can still be omitted.
import pytest
with pytest.raises(ValueError, match="description is required"):
save_structured_memory_strict("meta_key", "c2", description=None, scope="global")
save_structured_memory("meta_key", "c2", description="revised description", scope="global")
row = backend.get_structured_memory_by_name("meta_key", "global", "")
assert row["content"] == "c2"
assert row["description"] == "orig desc"
assert row["description"] == "revised description"
assert row["type"] == "fact"
def test_upsert_method_updates_in_place_no_raise(self, backend):
@@ -89,20 +108,75 @@ class TestSaveUpsert:
names = [r["name"] for r in backend.list_structured_memories(scope="global")]
assert names.count("k") == 1
def test_upsert_none_preserves_explicit_overwrites(self, backend):
"""None description/type keep the stored value on conflict; an explicit
value (including "" / "general") overwrites it."""
def test_upsert_requires_description_and_preserves_omitted_type(self, backend):
"""Description is mandatory; an omitted type keeps the stored value."""
import pytest
backend.create_structured_memory("m1", "k", "keepdesc", "fact", "global", "", "v1")
# None -> preserve stored description/type (a content-only save).
row, _ = backend.upsert_structured_memory("m2", "k", None, None, "global", "", "v2")
with pytest.raises(ValueError, match="description is required"):
backend.upsert_structured_memory("m2", "k", None, None, "global", "", "v2")
with pytest.raises(ValueError, match="description is required"):
backend.upsert_structured_memory("m2", "k", " ", None, "global", "", "v2")
row, _ = backend.upsert_structured_memory(
"m2", "k", "new description", None, "global", "", "v2"
)
assert row["content"] == "v2"
assert row["description"] == "keepdesc"
assert row["description"] == "new description"
assert row["type"] == "fact"
# Explicit "" / "general" -> overwrite.
row2, _ = backend.upsert_structured_memory("m3", "k", "", "general", "global", "", "v3")
assert row2["description"] == ""
row2, _ = backend.upsert_structured_memory(
"m3", "k", "final description", "general", "global", "", "v3"
)
assert row2["description"] == "final description"
assert row2["type"] == "general"
def test_active_project_guard_accepts_only_active_project(self, backend):
import pytest
backend.create_project("active", "Active", "u1")
row, was_update = backend.upsert_structured_memory(
"m1",
"guarded",
"guarded description",
None,
"project",
"active",
"value",
require_active_project=True,
)
assert row["scope_id"] == "active"
assert was_update is False
backend.create_project("archived", "Archived", "u1", state="archived")
for project_id in ("archived", "missing"):
with pytest.raises(ValueError, match="missing, archived"):
backend.upsert_structured_memory(
f"m-{project_id}",
"guarded",
"guarded description",
None,
"project",
project_id,
"value",
require_active_project=True,
)
assert backend.get_structured_memory_by_name("guarded", "project", project_id) is None
def test_active_project_guard_rejects_non_project_scope(self, backend):
import pytest
with pytest.raises(ValueError, match="requires project scope"):
backend.upsert_structured_memory(
"m1",
"guarded",
"guarded description",
None,
"global",
"",
"value",
require_active_project=True,
)
class TestDelete:
def test_delete_existing(self, backend):
@@ -118,63 +192,119 @@ class TestDelete:
assert not backend.delete_structured_memory("k", "global", "")
assert backend.delete_structured_memory("k", "workstream", "ws1")
def test_delete_returning_is_atomic_and_truthful(self, backend):
backend.create_structured_memory(
"m1", "k", "description", "reference", "user", "u1", "data"
)
deleted = backend.delete_structured_memory_returning("k", "user", "u1")
assert deleted is not None
assert deleted["memory_id"] == "m1"
assert deleted["description"] == "description"
assert deleted["type"] == "reference"
assert backend.get_structured_memory("m1") is None
assert backend.delete_structured_memory_returning("k", "user", "u1") is None
def test_delete_by_id_returning_is_atomic_and_truthful(self, backend):
backend.create_structured_memory("m1", "k", "Test memory", "general", "global", "", "data")
deleted = backend.delete_structured_memory_by_id_returning("m1")
assert deleted is not None
assert deleted["name"] == "k"
assert backend.get_structured_memory("m1") is None
assert backend.delete_structured_memory_by_id_returning("m1") is None
class TestFindScopes:
def test_finds_only_requested_same_name_scopes(self, backend):
backend.create_structured_memory("m1", "same", "Test memory", "general", "global", "", "g")
backend.create_structured_memory(
"m2", "same", "Test memory", "general", "user", "u1", "own"
)
backend.create_structured_memory(
"m3", "same", "Test memory", "general", "user", "victim", "secret"
)
backend.create_structured_memory(
"m4", "other", "Test memory", "general", "workstream", "ws1", "other"
)
found = backend.find_structured_memory_scopes(
"same", [("global", ""), ("user", "u1"), ("workstream", "ws1")]
)
assert set(found) == {("global", ""), ("user", "u1")}
class TestList:
def test_list_all(self, backend):
backend.create_structured_memory("m1", "a", "", "general", "global", "", "1")
backend.create_structured_memory("m2", "b", "", "user", "global", "", "2")
backend.create_structured_memory("m1", "a", "Test memory", "general", "global", "", "1")
backend.create_structured_memory("m2", "b", "Test memory", "user", "global", "", "2")
mems = backend.list_structured_memories()
assert len(mems) == 2
def test_list_by_type(self, backend):
backend.create_structured_memory("m1", "a", "", "general", "global", "", "1")
backend.create_structured_memory("m2", "b", "", "user", "global", "", "2")
backend.create_structured_memory("m1", "a", "Test memory", "general", "global", "", "1")
backend.create_structured_memory("m2", "b", "Test memory", "user", "global", "", "2")
mems = backend.list_structured_memories(mem_type="user")
assert len(mems) == 1
assert mems[0]["name"] == "b"
def test_list_by_scope(self, backend):
backend.create_structured_memory("m1", "a", "", "general", "global", "", "1")
backend.create_structured_memory("m2", "b", "", "general", "workstream", "ws1", "2")
backend.create_structured_memory("m1", "a", "Test memory", "general", "global", "", "1")
backend.create_structured_memory(
"m2", "b", "Test memory", "general", "workstream", "ws1", "2"
)
mems = backend.list_structured_memories(scope="workstream")
assert len(mems) == 1
def test_list_respects_limit(self, backend):
for i in range(10):
backend.create_structured_memory(f"m{i}", f"k{i}", "", "general", "global", "", f"{i}")
backend.create_structured_memory(
f"m{i}", f"k{i}", "Test memory", "general", "global", "", f"{i}"
)
mems = backend.list_structured_memories(limit=3)
assert len(mems) == 3
class TestSearch:
def test_search_by_name(self, backend):
backend.create_structured_memory("m1", "database_config", "", "general", "global", "", "pg")
backend.create_structured_memory("m2", "api_key", "", "general", "global", "", "secret")
backend.create_structured_memory(
"m1", "database_config", "Test memory", "general", "global", "", "pg"
)
backend.create_structured_memory(
"m2", "api_key", "Test memory", "general", "global", "", "secret"
)
results = backend.search_structured_memories("database")
assert len(results) == 1
assert results[0]["name"] == "database_config"
def test_search_by_content(self, backend):
backend.create_structured_memory("m1", "a", "", "general", "global", "", "postgresql host")
backend.create_structured_memory(
"m1", "a", "Test memory", "general", "global", "", "postgresql host"
)
results = backend.search_structured_memories("postgresql")
assert len(results) == 1
def test_search_empty_lists_all(self, backend):
backend.create_structured_memory("m1", "a", "", "general", "global", "", "1")
backend.create_structured_memory("m2", "b", "", "general", "global", "", "2")
backend.create_structured_memory("m1", "a", "Test memory", "general", "global", "", "1")
backend.create_structured_memory("m2", "b", "Test memory", "general", "global", "", "2")
results = backend.search_structured_memories("")
assert len(results) == 2
class TestCount:
def test_count_all(self, backend):
backend.create_structured_memory("m1", "a", "", "general", "global", "", "1")
backend.create_structured_memory("m2", "b", "", "general", "global", "", "2")
backend.create_structured_memory("m1", "a", "Test memory", "general", "global", "", "1")
backend.create_structured_memory("m2", "b", "Test memory", "general", "global", "", "2")
assert backend.count_structured_memories() == 2
def test_count_by_scope(self, backend):
backend.create_structured_memory("m1", "a", "", "general", "global", "", "1")
backend.create_structured_memory("m2", "b", "", "general", "workstream", "ws1", "2")
backend.create_structured_memory("m1", "a", "Test memory", "general", "global", "", "1")
backend.create_structured_memory(
"m2", "b", "Test memory", "general", "workstream", "ws1", "2"
)
assert backend.count_structured_memories(scope="global") == 1
assert backend.count_structured_memories(scope="workstream") == 1
@@ -184,8 +314,12 @@ class TestSearchOrOfTerms:
def test_single_matching_term_in_multi_word_query(self, backend):
"""Memory with content 'apple' found when query is 'apple banana cherry'."""
backend.create_structured_memory("m1", "apple_mem", "", "general", "global", "", "apple")
backend.create_structured_memory("m2", "other_mem", "", "general", "global", "", "grape")
backend.create_structured_memory(
"m1", "apple_mem", "Test memory", "general", "global", "", "apple"
)
backend.create_structured_memory(
"m2", "other_mem", "Test memory", "general", "global", "", "grape"
)
results = backend.search_structured_memories("apple banana cherry")
names = {r["name"] for r in results}
@@ -194,10 +328,18 @@ class TestSearchOrOfTerms:
def test_partial_overlap_across_memories(self, backend):
"""Each memory matches one of three terms; all three are returned."""
backend.create_structured_memory("m1", "alpha_doc", "", "general", "global", "", "alpha")
backend.create_structured_memory("m2", "beta_doc", "", "general", "global", "", "beta")
backend.create_structured_memory("m3", "gamma_doc", "", "general", "global", "", "gamma")
backend.create_structured_memory("m4", "unrelated", "", "general", "global", "", "delta")
backend.create_structured_memory(
"m1", "alpha_doc", "Test memory", "general", "global", "", "alpha"
)
backend.create_structured_memory(
"m2", "beta_doc", "Test memory", "general", "global", "", "beta"
)
backend.create_structured_memory(
"m3", "gamma_doc", "Test memory", "general", "global", "", "gamma"
)
backend.create_structured_memory(
"m4", "unrelated", "Test memory", "general", "global", "", "delta"
)
results = backend.search_structured_memories("alpha beta gamma")
names = {r["name"] for r in results}
@@ -209,12 +351,14 @@ class TestSearchOrOfTerms:
def test_scope_filter_preserved(self, backend):
"""OR-of-terms search still respects scope / scope_id filters."""
backend.create_structured_memory(
"m1", "ws1_note", "", "general", "workstream", "ws1", "info"
"m1", "ws1_note", "Test memory", "general", "workstream", "ws1", "info"
)
backend.create_structured_memory(
"m2", "ws2_note", "", "general", "workstream", "ws2", "info"
"m2", "ws2_note", "Test memory", "general", "workstream", "ws2", "info"
)
backend.create_structured_memory(
"m3", "global_note", "Test memory", "general", "global", "", "info"
)
backend.create_structured_memory("m3", "global_note", "", "general", "global", "", "info")
results = backend.search_structured_memories("info", scope="workstream", scope_id="ws1")
names = {r["name"] for r in results}
@@ -224,9 +368,11 @@ class TestSearchOrOfTerms:
def test_term_cap_normalizes_unbounded_query(self, backend):
"""A multi-KB query collapses to <= MAX terms (de-dupe + length filter)."""
backend.create_structured_memory("m1", "alpha_doc", "", "general", "global", "", "alpha")
backend.create_structured_memory(
"m2", "other_doc", "", "general", "global", "", "irrelevant"
"m1", "alpha_doc", "Test memory", "general", "global", "", "alpha"
)
backend.create_structured_memory(
"m2", "other_doc", "Test memory", "general", "global", "", "irrelevant"
)
# Build a noisy query: same word repeated, plus 1-char tokens that
@@ -241,10 +387,18 @@ class TestVisibleStructuredMemories:
"""Single-query union helpers used by the composition path."""
def test_list_visible_unions_global_workstream_user(self, backend):
backend.create_structured_memory("m1", "g_note", "", "general", "global", "", "g")
backend.create_structured_memory("m2", "ws_note", "", "general", "workstream", "ws1", "w")
backend.create_structured_memory("m3", "u_note", "", "general", "user", "u1", "u")
backend.create_structured_memory("m4", "other_ws", "", "general", "workstream", "ws2", "x")
backend.create_structured_memory(
"m1", "g_note", "Test memory", "general", "global", "", "g"
)
backend.create_structured_memory(
"m2", "ws_note", "Test memory", "general", "workstream", "ws1", "w"
)
backend.create_structured_memory(
"m3", "u_note", "Test memory", "general", "user", "u1", "u"
)
backend.create_structured_memory(
"m4", "other_ws", "Test memory", "general", "workstream", "ws2", "x"
)
scopes = [("global", ""), ("workstream", "ws1"), ("user", "u1")]
rows = backend.list_visible_structured_memories(scopes)
@@ -252,12 +406,14 @@ class TestVisibleStructuredMemories:
assert names == {"g_note", "ws_note", "u_note"} # ws2 excluded
def test_search_visible_unions_scopes_and_terms(self, backend):
backend.create_structured_memory("m1", "g_alpha", "", "general", "global", "", "alpha")
backend.create_structured_memory(
"m2", "ws_beta", "", "general", "workstream", "ws1", "beta"
"m1", "g_alpha", "Test memory", "general", "global", "", "alpha"
)
backend.create_structured_memory(
"m3", "ws_other", "", "general", "workstream", "ws2", "alpha"
"m2", "ws_beta", "Test memory", "general", "workstream", "ws1", "beta"
)
backend.create_structured_memory(
"m3", "ws_other", "Test memory", "general", "workstream", "ws2", "alpha"
)
scopes = [("global", ""), ("workstream", "ws1")]
@@ -268,7 +424,9 @@ class TestVisibleStructuredMemories:
assert "ws_other" not in names # ws2 -> outside visibility
def test_visible_helpers_handle_empty_scopes(self, backend):
backend.create_structured_memory("m1", "anything", "", "general", "global", "", "x")
backend.create_structured_memory(
"m1", "anything", "Test memory", "general", "global", "", "x"
)
assert backend.list_visible_structured_memories([]) == []
assert backend.search_visible_structured_memories("x", []) == []
@@ -288,7 +446,7 @@ class TestStableOrderingOnTimestampTies:
# batch lands them in the same second.
for mid in ("zebra_id", "apple_id", "mango_id"):
backend.create_structured_memory(
mid, f"name_{mid}", "", "general", "global", "", "shared content"
mid, f"name_{mid}", "Test memory", "general", "global", "", "shared content"
)
import sqlalchemy as sa
+1
View File
@@ -225,6 +225,7 @@ class TestContextOverflowRecovery:
patch.object(session, "_stream_response", side_effect=mock_stream_response),
patch.object(session, "_compact_messages", compact_mock),
patch.object(session, "_full_messages", return_value=[]),
patch.object(session, "_over_soft", return_value=False),
patch.object(session, "_update_token_table"),
patch.object(session, "_print_status_line"),
patch.object(session, "_emit_state"),
+145
View File
@@ -2,6 +2,10 @@
from __future__ import annotations
import os
import pytest
class TestVersionHtml:
def test_app_css_gets_version(self):
@@ -54,6 +58,13 @@ class TestVersionHtml:
result = version_html(html)
assert result == html # unchanged
def test_vendor_prefix_lookalike_is_still_versioned(self):
from turnstone.core.web_helpers import version_html
html = '<script src="/shared/hls-2-player.js"></script>'
result = version_html(html)
assert "/shared/hls-2-player.js?v=" in result
def test_external_urls_not_modified(self):
from turnstone.core.web_helpers import version_html
@@ -116,6 +127,140 @@ class TestVersionHtml:
assert result == html # unchanged — already has query string
class TestRevalidatingStaticFiles:
def test_same_versioned_url_revalidates_after_asset_changes(self, tmp_path):
from starlette.applications import Starlette
from starlette.routing import Mount
from starlette.testclient import TestClient
from turnstone import __version__
from turnstone.core.web_helpers import RevalidatingStaticFiles
asset = tmp_path / "app.js"
old_body = b"export const generation = 'old';"
new_body = b"export const generation = 'new';"
assert len(old_body) == len(new_body)
asset.write_bytes(old_body)
original_stat = asset.stat()
app = Starlette(
routes=[Mount("/static", app=RevalidatingStaticFiles(directory=str(tmp_path)))]
)
with TestClient(app) as client:
url = f"/static/app.js?v={__version__}"
first = client.get(url)
assert first.status_code == 200
assert first.headers["cache-control"] == "no-cache"
assert "last-modified" not in first.headers
old_etag = first.headers["etag"]
asset.write_bytes(new_body)
os.utime(
asset,
ns=(original_stat.st_atime_ns, original_stat.st_mtime_ns),
)
changed_stat = asset.stat()
assert changed_stat.st_size == original_stat.st_size
assert changed_stat.st_mtime_ns == original_stat.st_mtime_ns
changed = client.get(url, headers={"If-None-Match": old_etag})
assert changed.status_code == 200
assert changed.content == new_body
assert changed.headers["etag"] != old_etag
assert changed.headers["cache-control"] == "no-cache"
assert "last-modified" not in changed.headers
stale_date_only = client.get(
url,
headers={"If-Modified-Since": "Wed, 31 Dec 9999 23:59:59 GMT"},
)
assert stale_date_only.status_code == 200
assert stale_date_only.content == new_body
unchanged = client.get(url, headers={"If-None-Match": changed.headers["etag"]})
assert unchanged.status_code == 304
assert unchanged.headers["cache-control"] == "no-cache"
def test_version_named_vendor_asset_is_immutable(self, tmp_path):
from starlette.applications import Starlette
from starlette.routing import Mount
from starlette.testclient import TestClient
from turnstone.core.web_helpers import RevalidatingStaticFiles
vendor_dir = tmp_path / "katex-0.18.4"
vendor_dir.mkdir()
(vendor_dir / "katex.min.css").write_text(".katex {}", encoding="utf-8")
app = Starlette(
routes=[Mount("/shared", app=RevalidatingStaticFiles(directory=str(tmp_path)))]
)
with TestClient(app) as client:
resp = client.get("/shared/katex-0.18.4/katex.min.css")
assert resp.status_code == 200
assert resp.headers["cache-control"] == "public, max-age=31536000, immutable"
assert resp.headers["etag"]
def test_missing_asset_is_not_cached(self, tmp_path):
from starlette.applications import Starlette
from starlette.routing import Mount
from starlette.testclient import TestClient
from turnstone.core.web_helpers import RevalidatingStaticFiles
app = Starlette(
routes=[Mount("/shared", app=RevalidatingStaticFiles(directory=str(tmp_path)))]
)
with TestClient(app) as client:
resp = client.get("/shared/not-deployed-yet.js")
assert resp.status_code == 404
assert resp.headers["cache-control"] == "no-store"
class TestStaticAssetCacheControl:
@pytest.mark.parametrize(
("path", "expected"),
[
("interactive.js", "no-cache"),
("hls-2-player.js", "no-cache"),
("katex-0.18.4/katex.min.css", "public, max-age=31536000, immutable"),
("hljs-11.11.1/highlight.min.js", "public, max-age=31536000, immutable"),
("katex-0.18.4/../private.json", "no-store"),
(r"katex-0.18.4\..\private.json", "no-store"),
("nested//asset.js", "no-store"),
],
)
def test_policy_requires_a_canonical_exact_vendor_path(self, path, expected):
from turnstone.core.web_helpers import static_asset_cache_control
assert static_asset_cache_control(path) == expected
def test_every_packaged_versioned_vendor_directory_uses_the_shared_policy(self):
import re
from pathlib import Path
import turnstone
from turnstone.core.web_helpers import static_asset_cache_control, version_html
shared_dir = Path(turnstone.__file__).resolve().parent / "shared_static"
versioned_dir = re.compile(r"^[a-z][a-z0-9_-]*-\d+(?:\.\d+)+$")
vendor_dirs = sorted(
path.name
for path in shared_dir.iterdir()
if path.is_dir() and versioned_dir.fullmatch(path.name)
)
assert vendor_dirs
for directory in vendor_dirs:
asset_path = f"{directory}/asset.js"
assert static_asset_cache_control(asset_path) == "public, max-age=31536000, immutable"
html = f'<script src="/shared/{asset_path}"></script>'
assert version_html(html) == html
class TestLatin1SafeFilename:
"""Content-Disposition filename sanitizer — must yield a value that is
both latin-1 encodable (Starlette) and control-char free (h11)."""
+19 -5
View File
@@ -753,10 +753,20 @@ MemoryScope = Literal["global", "workstream", "user"]
class SaveMemoryRequest(BaseModel):
name: str = Field(description="Memory identifier (normalized to snake_case)")
content: str = Field(description="Memory content", max_length=65536)
description: str = Field(default="", description="Short description for relevance matching")
type: MemoryType = Field(default="general", description="Memory type")
name: str = Field(
description="Memory identifier (normalized to snake_case)",
min_length=1,
max_length=256,
)
content: str = Field(description="Memory content", min_length=1, max_length=65536)
description: str = Field(
description="Required non-empty description used for relevance matching",
min_length=1,
)
type: MemoryType | None = Field(
default=None,
description="Memory type; omission preserves it on update and defaults on insert",
)
scope: MemoryScope = Field(default="global", description="Memory scope")
scope_id: str = Field(
default="",
@@ -765,6 +775,8 @@ class SaveMemoryRequest(BaseModel):
@model_validator(mode="after")
def _validate_scope_scope_id(self) -> SaveMemoryRequest:
if not self.description.strip():
raise ValueError("description is required and must be non-empty")
scope_id = self.scope_id.strip()
if self.scope == "global" and scope_id:
raise ValueError("scope_id is not allowed with global scope")
@@ -795,7 +807,7 @@ MemoryScopeFilter = Literal["", "global", "workstream", "user"]
class SearchMemoriesRequest(BaseModel):
query: str = Field(description="Search query text")
query: str = Field(description="Search query text", min_length=1)
type: MemoryTypeFilter = Field(default="", description="Filter by memory type")
scope: MemoryScopeFilter = Field(default="", description="Filter by scope")
scope_id: str = Field(default="", description="Filter by scope_id")
@@ -808,6 +820,8 @@ class SearchMemoriesRequest(BaseModel):
raise ValueError("scope_id is not allowed with global scope")
if scope_id and not self.scope:
raise ValueError("scope is required when scope_id is provided")
if self.scope == "workstream" and not scope_id:
raise ValueError("scope_id is required for workstream scope")
return self
+7 -5
View File
@@ -496,16 +496,17 @@ SERVER_ENDPOINTS: list[EndpointSpec] = [
EndpointSpec(
"/v1/api/memories",
"GET",
"List structured memories",
"List structured memories. Without a scope, returns global plus the authenticated user's memories; workstream scope is owner-bound.",
response_model=ListMemoriesResponse,
query_params=[
QueryParam("type", "Filter by memory type"),
QueryParam("scope", "Filter by scope"),
QueryParam("scope", "Filter by public scope: global, workstream, or user"),
QueryParam("scope_id", "Filter by scope identifier"),
QueryParam(
"limit", "Max results (default 100, max 200)", schema_type="integer", default=100
),
],
error_codes=[400, 403, 404, 500],
tags=["Memories"],
),
EndpointSpec(
@@ -514,15 +515,16 @@ SERVER_ENDPOINTS: list[EndpointSpec] = [
"Save (upsert) a structured memory",
request_model=SaveMemoryRequest,
response_model=MemoryInfo,
error_codes=[400],
error_codes=[400, 403, 404, 500],
tags=["Memories"],
),
EndpointSpec(
"/v1/api/memories/search",
"POST",
"Search structured memories by query",
"Search structured memories by query. Without a scope, searches global plus the authenticated user's memories.",
request_model=SearchMemoriesRequest,
response_model=ListMemoriesResponse,
error_codes=[400, 403, 404, 500],
tags=["Memories"],
),
EndpointSpec(
@@ -534,7 +536,7 @@ SERVER_ENDPOINTS: list[EndpointSpec] = [
QueryParam("scope", "Scope (default: global)"),
QueryParam("scope_id", "Scope identifier"),
],
error_codes=[404],
error_codes=[400, 403, 404, 500],
tags=["Memories"],
),
# --- Admin settings ---
+87 -57
View File
@@ -14,6 +14,7 @@ import argparse
import asyncio
import contextlib
import functools
import hashlib
import json
import logging
import math
@@ -38,7 +39,6 @@ from starlette.background import BackgroundTask
from starlette.middleware import Middleware
from starlette.responses import HTMLResponse, JSONResponse, Response, StreamingResponse
from starlette.routing import Mount, Route
from starlette.staticfiles import StaticFiles
from turnstone.api.console_spec import build_console_spec
from turnstone.api.docs import make_docs_handler, make_openapi_handler
@@ -76,7 +76,10 @@ from turnstone.core.model_registry import (
from turnstone.core.model_registry import MODEL_AUTH_MODES as _MODEL_AUTH_MODES
from turnstone.core.rendezvous import NoAvailableNodeError, NodeRef
from turnstone.core.rerank_calibrate import canonical_caps_value
from turnstone.core.session_replay import session_replay_preamble
from turnstone.core.session_replay import (
request_replay_project_name,
session_replay_preamble,
)
from turnstone.core.session_routes import (
AttachmentUploadHelpers,
CoordOnlyVerbHandlers,
@@ -107,8 +110,12 @@ from turnstone.core.skill_field_validation import SKILL_RUNTIME_CONFIG_FIELDS
from turnstone.core.skill_kind import SkillKind
from turnstone.core.skill_parser import MAX_SKILL_DESCRIPTION_LEN
from turnstone.core.web_helpers import (
RevalidatingStaticFiles,
is_safe_static_asset_path,
read_json_or_400,
require_storage_or_503,
static_asset_cache_control,
version_html,
)
from turnstone.core.workstream import (
Workstream,
@@ -144,10 +151,6 @@ _HTML_ETAG = ""
def _load_static() -> None:
import hashlib
from turnstone.core.web_helpers import version_html
global _HTML, _HTML_ETAG
_HTML = version_html((_STATIC_DIR / "index.html").read_text(encoding="utf-8"))
_HTML_ETAG = '"' + hashlib.md5(_HTML.encode()).hexdigest()[:16] + '"' # noqa: S324
@@ -3029,52 +3032,66 @@ async def proxy_index(request: Request) -> Response:
return JSONResponse({"error": "Node unreachable"}, status_code=502)
async def proxy_static(request: Request) -> Response:
"""GET /node/{node_id}/static/{path} — proxy static files."""
def _proxy_static_request_headers(request: Request) -> dict[str, str]:
"""Build upstream headers for a cache-aware static asset request."""
headers = _proxy_auth_headers(request)
for name in ("if-none-match", "if-modified-since"):
value = request.headers.get(name)
if value:
headers[name] = value
return headers
def _proxy_static_response(resp: httpx.Response, path: str) -> Response:
"""Preserve upstream validators and apply the local static cache policy."""
cache_control = "no-store"
if resp.status_code in (200, 304):
cache_control = resp.headers.get("cache-control") or static_asset_cache_control(path)
headers = {"Cache-Control": cache_control}
for name in ("content-type", "etag", "last-modified"):
value = resp.headers.get(name)
if value:
headers[name] = value
return Response(content=resp.content, status_code=resp.status_code, headers=headers)
def _static_proxy_error(message: str, status_code: int) -> JSONResponse:
return JSONResponse(
{"error": message}, status_code=status_code, headers={"Cache-Control": "no-store"}
)
async def _proxy_static_mount(request: Request, mount: str) -> Response:
"""Proxy one validated static mount without URL-normalization ambiguity."""
node_id = request.path_params["node_id"]
path = request.path_params["path"]
if not is_safe_static_asset_path(path):
return _static_proxy_error("Invalid static asset path", 400)
server_url = _get_server_url(request, node_id)
if not server_url:
return JSONResponse({"error": "Node not found"}, status_code=404)
return _static_proxy_error("Node not found", 404)
encoded_path = "/".join(urllib.parse.quote(segment, safe="") for segment in path.split("/"))
client: httpx.AsyncClient = request.app.state.proxy_client
try:
resp = await client.get(
f"{server_url}/static/{path}",
headers=_proxy_auth_headers(request),
)
return Response(
content=resp.content,
status_code=resp.status_code,
media_type=resp.headers.get("content-type", "application/octet-stream"),
f"{server_url}/{mount}/{encoded_path}",
headers=_proxy_static_request_headers(request),
)
return _proxy_static_response(resp, path)
except httpx.HTTPError as exc:
log.debug("Proxy static error for %s/%s: %s", node_id, path, exc)
return JSONResponse({"error": "Node unreachable"}, status_code=502)
log.debug("Proxy %s error for %s/%s: %s", mount, node_id, path, exc)
return _static_proxy_error("Node unreachable", 502)
async def proxy_static(request: Request) -> Response:
"""GET /node/{node_id}/static/{path} — proxy static files."""
return await _proxy_static_mount(request, "static")
async def proxy_shared_static(request: Request) -> Response:
"""GET /node/{node_id}/shared/{path} — proxy shared static files."""
node_id = request.path_params["node_id"]
path = request.path_params["path"]
server_url = _get_server_url(request, node_id)
if not server_url:
return JSONResponse({"error": "Node not found"}, status_code=404)
client: httpx.AsyncClient = request.app.state.proxy_client
try:
resp = await client.get(
f"{server_url}/shared/{path}",
headers=_proxy_auth_headers(request),
)
return Response(
content=resp.content,
status_code=resp.status_code,
media_type=resp.headers.get("content-type", "application/octet-stream"),
)
except httpx.HTTPError as exc:
log.debug("Proxy shared static error for %s/%s: %s", node_id, path, exc)
return JSONResponse({"error": "Node unreachable"}, status_code=502)
return await _proxy_static_mount(request, "shared")
# Auth endpoints the console handles locally instead of forwarding to
@@ -3687,20 +3704,15 @@ def _audit_retry_coordinator(
def _coord_events_replay(
ws: Workstream,
ui: Any,
request: Request, # noqa: ARG001 — coord replay doesn't need request context
request: Request,
) -> Iterable[dict[str, Any]]:
"""Initial SSE replay payload for coord ``events`` connections.
Yields, in order:
1. ``connected`` + optional ``status`` via the shared
:func:`turnstone.core.session_replay.session_replay_preamble`
so the dashboard's status bar populates before any live tick.
Same payload shape interactive uses.
2. Pending approval prompt (if any) and the cached LLM verdicts
that fired since it surfaced. Without this replay a refresh
loses the judge chip on the pending approval until the
operator re-invokes the action.
Yields ``connected`` plus optional ``status``, then the pending approval
prompt (if any) and cached LLM verdicts that fired since it surfaced. The
shared handler resolves viewer-specific project metadata off-loop before
invoking this callback. Without the control replay a refresh loses the
judge chip until the operator re-invokes the action.
Coord still skips conversation history the dashboard fetches it
via a separate ``GET /history`` endpoint and doesn't want a
@@ -3708,7 +3720,11 @@ def _coord_events_replay(
Pure read never mutates ``ui`` / ``ws`` / ``session``.
"""
yield from session_replay_preamble(ws.session, ui)
yield from session_replay_preamble(
ws.session,
ui,
project_name=request_replay_project_name(request),
)
# EVERY live approval cycle replays (parallel task agents can have
# several outstanding), each card followed once by the cached LLM
@@ -3963,14 +3979,18 @@ async def coordinator_page(request: Request) -> Response:
if not template_path.is_file():
return JSONResponse({"error": "coordinator UI template missing"}, status_code=500)
try:
body = template_path.read_text(encoding="utf-8")
body = version_html(template_path.read_text(encoding="utf-8"))
except OSError:
return JSONResponse({"error": "failed to read coordinator UI template"}, status_code=500)
# Inject the ws_id as an HTML attribute. ws_id passed the
# ``_VALID_WS_ID_RE`` gate above (hex only) so there's nothing
# to HTML-escape; leave the replacement simple.
body = body.replace("{{WS_ID}}", ws_id)
return Response(body, media_type="text/html; charset=utf-8")
etag = '"' + hashlib.md5(body.encode()).hexdigest()[:16] + '"' # noqa: S324
headers = {"Cache-Control": "no-cache", "ETag": etag}
if request.headers.get("If-None-Match") == etag:
return Response(status_code=304, headers=headers)
return HTMLResponse(body, headers=headers)
_CHILDREN_PAGE_LIMIT = 200
@@ -9544,12 +9564,14 @@ async def admin_delete_memory(request: Request) -> JSONResponse:
return err
memory_id = request.path_params["memory_id"]
existing = storage.get_structured_memory(memory_id)
try:
existing = storage.delete_structured_memory_by_id_returning(memory_id)
except Exception:
log.warning("memory.admin_delete_failed memory_id=%s", memory_id, exc_info=True)
return JSONResponse({"error": "Failed to delete memory"}, status_code=500)
if not existing:
return JSONResponse({"error": "Memory not found"}, status_code=404)
storage.delete_structured_memory_by_id(memory_id)
audit_uid, ip = _audit_context(request)
record_audit(
storage,
@@ -16467,8 +16489,16 @@ def create_app(
Route("/metrics", console_metrics_endpoint),
Route("/openapi.json", _openapi_handler),
Route("/docs", _docs_handler),
Mount("/static", app=StaticFiles(directory=str(_STATIC_DIR)), name="static"),
Mount("/shared", app=StaticFiles(directory=str(_SHARED_DIR)), name="shared"),
Mount(
"/static",
app=RevalidatingStaticFiles(directory=str(_STATIC_DIR)),
name="static",
),
Mount(
"/shared",
app=RevalidatingStaticFiles(directory=str(_SHARED_DIR)),
name="shared",
),
# Coordinator one-pane UI — the route serves a single
# index.html template with the ws_id injected via data-ws-id
# so coordinator.js can pull it without an extra round-trip.
+15 -8
View File
@@ -1491,12 +1491,20 @@ function _homeRenderChips() {
}
function _homeStageFile(file) {
if (!file) return;
if (!file) return false;
const paste = window.TurnstonePasteText;
if (
paste &&
paste.isDuplicatePastedTextFile &&
paste.isDuplicatePastedTextFile(file, _homeStagedFiles)
) {
return true;
}
if (_homeStagedFiles.length >= _HOME_MAX_FILES) {
_homeShowError(
"At most " + _HOME_MAX_FILES + " attachments per coordinator",
);
return;
return false;
}
if (!_homeIsAttachmentAllowed(file)) {
_homeShowError(
@@ -1504,17 +1512,18 @@ function _homeStageFile(file) {
file.name +
" (allowed: png/jpeg/gif/webp images, text)",
);
return;
return false;
}
const isImage = (file.type || "").indexOf("image/") === 0;
const cap = isImage ? _HOME_IMAGE_CAP : _HOME_TEXT_CAP;
if (file.size > cap) {
_homeShowError(file.name + " exceeds the " + _homeFormatSize(cap) + " cap");
return;
return false;
}
_homeShowError("");
_homeStagedFiles.push(file);
_homeRenderChips();
return true;
}
function _homeClearStagedFiles() {
@@ -1807,7 +1816,7 @@ function _mountHomeCoordComposer() {
},
attachments: {
onAttach: function (file) {
_homeStageFile(file);
return _homeStageFile(file);
},
},
dragDrop: { targetEl: mount, dropClass: "home-coord-drop" },
@@ -1962,9 +1971,7 @@ function submitHomeCoord(textFromComposer) {
// pending storage rows until the GC sweep. Require text whenever
// attachments are staged so the first turn always picks them up.
if (files.length > 0 && !(task || "").trim()) {
_homeShowError(
"Add a task message — attachments need an initial turn to dispatch on.",
);
_homeShowError("Add a message to send with this attachment.");
return;
}
const shared = {
@@ -50,6 +50,7 @@ import {
acceptUserTurnEvent,
clientSendMaySettleForViewer,
createQueueController,
mergeRejectedComposerText,
mintClientSendId,
parsePriority,
postAndSettleSend,
@@ -2656,9 +2657,34 @@ function createCoordinatorPane(root, wsId, opts) {
function coordSend() {
const text = composer.value;
const trimmed = (text || "").trim();
if (!trimmed) return false;
if (!trimmed) {
if (!attachments.isEmpty()) {
appendText("info", "Add a message to send with this attachment.", {
label: "info",
});
}
return false;
}
// The live-turn interjection queue is text-only. Keep the input and chips
// intact instead of optimistically clearing them before attachments_busy.
if (busy && !attachments.isEmpty()) {
appendText(
"info",
"Attachments can't be sent while the assistant is working. Wait for it to finish, then send again.",
{ label: "info" },
);
return false;
}
const snap = attachments.snapshot();
if (snap.uploading) {
appendText(
"info",
"Wait for attachments to finish uploading before sending.",
{ label: "info" },
);
return false;
}
let queuedEl = null;
let optimisticEl = null;
@@ -2734,6 +2760,9 @@ function createCoordinatorPane(root, wsId, opts) {
setBusy: (b) => setBusy(b),
busyIsOptimistic: () => busy && busySource === "optimistic",
paneIsBusy: () => busy,
restoreInput: () => {
composer.value = mergeRejectedComposerText(trimmed, composer.value);
},
renderError: (msg) => appendText("error", msg, { label: "error" }),
consumeAttachments: (attached, droppedIds) =>
attachments.consume(attached, droppedIds),
@@ -4278,6 +4307,12 @@ function createCoordinatorPane(root, wsId, opts) {
setBusy: (value) => setBusy(value),
busyIsOptimistic: () => busy && busySource === "optimistic",
paneIsBusy: () => busy,
restoreInput: () => {
composer.value = mergeRejectedComposerText(
editText,
composer.value,
);
},
renderError: (message) =>
appendText("error", message, { label: "error" }),
consumeAttachments: () => {},
@@ -43,6 +43,7 @@
<script type="module" src="/shared/toast.js"></script>
<script type="module" src="/shared/auth.js"></script>
<script type="module" src="/shared/kb.js"></script>
<script type="module" src="/shared/composer_paste_text.js"></script>
<script type="module" src="/shared/composer.js"></script>
<script type="module" src="/shared/composer_attachments.js"></script>
<script type="module" src="/shared/composer_queue.js"></script>
+1
View File
@@ -4116,6 +4116,7 @@
<script type="module" src="/shared/models.js"></script>
<script type="module" src="/shared/skills.js"></script>
<script type="module" src="/shared/project_creator.js"></script>
<script type="module" src="/shared/composer_paste_text.js"></script>
<script type="module" src="/shared/composer.js"></script>
<!-- coordinator-pane deps (step 4): the controller builds its chrome + uses
the shared composer/renderer stack; load before the shell module -->
+256 -111
View File
@@ -507,6 +507,45 @@ def _cap_server_prompts(server_name: str, prompts: list[Any]) -> list[Any]:
return prompts[:_MAX_PROMPTS_PER_SERVER]
async def _list_resources_compatible(session: Any, server_name: str) -> Any:
"""List concrete resources, treating an unsupported method as empty.
MCP exposes one aggregate ``resources`` capability for both concrete
resources and resource templates. Servers may legitimately implement only
one of the two list methods, so the capability bit alone cannot tell us
which request is supported. Only the protocol's exact METHOD_NOT_FOUND code
is normalized; every other error remains a real discovery failure.
"""
try:
return await session.list_resources()
except McpError as exc:
if exc.error.code != mcp_types.METHOD_NOT_FOUND:
raise
log.debug(
"MCP server '%s' does not implement resources/list; treating it as empty",
server_name,
)
return mcp_types.ListResourcesResult(resources=[])
async def _list_resource_templates_compatible(session: Any, server_name: str) -> Any:
"""List resource templates, treating an unsupported method as empty.
See :func:`_list_resources_compatible` for why the aggregate capability
requires per-method probing and exact JSON-RPC error classification.
"""
try:
return await session.list_resource_templates()
except McpError as exc:
if exc.error.code != mcp_types.METHOD_NOT_FOUND:
raise
log.debug(
"MCP server '%s' does not implement resources/templates/list; treating it as empty",
server_name,
)
return mcp_types.ListResourceTemplatesResult(resourceTemplates=[])
# ---------------------------------------------------------------------------
# Per-server state containers
# ---------------------------------------------------------------------------
@@ -2336,96 +2375,127 @@ class MCPClientManager:
state.close_requested = close_requested
state.session = session
# Check push notification support for each capability
caps = session.get_server_capabilities()
# Capability flags and all three catalogs are STAGED until discovery
# succeeds in full. A later resource/prompt failure must not publish a
# callable tool backed by a registration that add_server_sync reports
# as failed. The live owner/session must likewise be torn down before
# the error escapes.
try:
caps = session.get_server_capabilities()
tools_cap = getattr(caps, "tools", None) if caps else None
state.supports_list_changed = bool(getattr(tools_cap, "listChanged", False))
tools_cap = getattr(caps, "tools", None) if caps else None
supports_list_changed = bool(getattr(tools_cap, "listChanged", False))
resources_cap = getattr(caps, "resources", None) if caps else None
state.supports_resources = resources_cap is not None
state.supports_resource_list_changed = bool(getattr(resources_cap, "listChanged", False))
resources_cap = getattr(caps, "resources", None) if caps else None
supports_resources = resources_cap is not None
supports_resource_list_changed = bool(getattr(resources_cap, "listChanged", False))
prompts_cap = getattr(caps, "prompts", None) if caps else None
state.supports_prompts = prompts_cap is not None
state.supports_prompt_list_changed = bool(getattr(prompts_cap, "listChanged", False))
prompts_cap = getattr(caps, "prompts", None) if caps else None
supports_prompts = prompts_cap is not None
supports_prompt_list_changed = bool(getattr(prompts_cap, "listChanged", False))
# Discover tools. Discovery runs in THIS caller task while the
# transport is hosted by the owner, so a transport collapse
# mid-discovery cancels the OWNER, not us — ``_await_owner_discovery``
# races the owner so that death surfaces as a prompt ConnectionError
# instead of hanging to the caller-side attempt timeout.
result = await self._await_owner_discovery(owner, session.list_tools())
capped = _cap_server_tools(name, result.tools)
server_tools: list[dict[str, Any]] = [_mcp_to_openai(name, tool) for tool in capped]
# Discover tools. Discovery runs in THIS caller task while the
# transport is hosted by the owner, so a transport collapse
# mid-discovery cancels the OWNER, not us — ``_await_owner_discovery``
# races the owner so that death surfaces as a prompt ConnectionError
# instead of hanging to the caller-side attempt timeout.
result = await self._await_owner_discovery(owner, session.list_tools())
capped = _cap_server_tools(name, result.tools)
server_tools: list[dict[str, Any]] = [_mcp_to_openai(name, tool) for tool in capped]
state.tools = server_tools
self._rebuild_tools()
# Discover resources
resource_count = 0
if resources_cap is not None:
# Discover resources. The protocol advertises the pair with one
# aggregate capability, but either list method may independently be
# absent; the compatibility wrappers normalize only -32601.
server_resources: list[dict[str, Any]] = []
res_result = await self._await_owner_discovery(owner, session.list_resources())
# Capped like the tools list above (and like the pool twins): a
# misbehaving server must not balloon the shared node's merged
# catalogs.
for r in _cap_server_resources(name, res_result.resources):
server_resources.append(
{
"uri": str(r.uri),
"name": r.name or "",
"description": r.description or "",
"mimeType": r.mimeType or "",
"server": name,
}
if resources_cap is not None:
res_result = await self._await_owner_discovery(
owner, _list_resources_compatible(session, name)
)
# Also include resource templates (catalog-only — not directly
# readable via read_resource since they contain URI placeholders)
tmpl_result = await self._await_owner_discovery(
owner, session.list_resource_templates()
)
for t in _cap_server_resource_templates(name, tmpl_result.resourceTemplates):
server_resources.append(
{
"uri": str(t.uriTemplate),
"name": t.name or "",
"description": t.description or "",
"mimeType": t.mimeType or "",
"server": name,
"template": True,
}
# Capped like the tools list above (and like the pool twins): a
# misbehaving server must not balloon the shared node's merged
# catalogs.
for r in _cap_server_resources(name, res_result.resources):
server_resources.append(
{
"uri": str(r.uri),
"name": r.name or "",
"description": r.description or "",
"mimeType": r.mimeType or "",
"server": name,
}
)
# Templates are catalog-only — not directly readable via
# read_resource since they contain URI placeholders.
tmpl_result = await self._await_owner_discovery(
owner, _list_resource_templates_compatible(session, name)
)
resource_count = len(server_resources)
state.resources = server_resources
self._rebuild_resources()
for t in _cap_server_resource_templates(name, tmpl_result.resourceTemplates):
server_resources.append(
{
"uri": str(t.uriTemplate),
"name": t.name or "",
"description": t.description or "",
"mimeType": t.mimeType or "",
"server": name,
"template": True,
}
)
# Discover prompts
prompt_count = 0
if prompts_cap is not None:
# Discover prompts.
server_prompts: list[dict[str, Any]] = []
prompt_result = await self._await_owner_discovery(owner, session.list_prompts())
for p in _cap_server_prompts(name, prompt_result.prompts):
server_prompts.append(
{
"name": f"mcp__{name}__{p.name}",
"original_name": p.name,
"server": name,
"description": p.description or "",
"arguments": [
{
"name": a.name,
"description": a.description or "",
"required": a.required or False,
}
for a in (p.arguments or [])
],
}
)
prompt_count = len(server_prompts)
state.prompts = server_prompts
if prompts_cap is not None:
prompt_result = await self._await_owner_discovery(owner, session.list_prompts())
for p in _cap_server_prompts(name, prompt_result.prompts):
server_prompts.append(
{
"name": f"mcp__{name}__{p.name}",
"original_name": p.name,
"server": name,
"description": p.description or "",
"arguments": [
{
"name": a.name,
"description": a.description or "",
"required": a.required or False,
}
for a in (p.arguments or [])
],
}
)
# The owner done-callback can evict the session in the same loop
# turn that the final discovery call completes. Revalidate the
# exact wiring immediately before commit; there are no awaits from
# this check through publication, so a callable catalog can never
# be installed behind a dead/replaced transport.
if owner.done() or state.owner_task is not owner or state.session is not session:
raise ConnectionError(f"MCP server '{name}' transport died before catalog commit")
except BaseException:
await self._teardown_static_session(name)
raise
# Publish the staged connection and catalogs only after every enabled
# discovery method succeeded. Successful reconnects also clear a stale
# resource/prompt catalog when the server drops that capability.
resources_changed = state.resources != server_resources
prompts_changed = state.prompts != server_prompts
state.supports_list_changed = supports_list_changed
state.supports_resources = supports_resources
state.supports_resource_list_changed = supports_resource_list_changed
state.supports_prompts = supports_prompts
state.supports_prompt_list_changed = supports_prompt_list_changed
state.tools = server_tools
state.resources = server_resources
state.prompts = server_prompts
self._rebuild_tools()
if resources_changed:
self._rebuild_resources()
if prompts_changed:
self._rebuild_prompts()
resource_count = len(server_resources)
prompt_count = len(server_prompts)
push_parts: list[str] = []
if state.supports_list_changed:
push_parts.append("tools")
@@ -2776,23 +2846,27 @@ class MCPClientManager:
call.cancel()
await asyncio.gather(call, return_exceptions=True)
raise
if call.done():
if call.cancelled():
# The discovery future was cancelled out from under us (an
# SDK-internal cancellation shape, not this method's own reap)
# — the transport can no longer answer, which is the same
# failure class as the owner dying. Convert instead of leaking
# a bare CancelledError the caller would misread as its own
# cancellation.
raise ConnectionError("MCP discovery request cancelled by transport failure")
return call.result() # normal result, or the real discovery error
# Owner finished first — the transport died under discovery. Reap the
# discovery future (gather absorbs its cancellation outcome; a caller
# cancellation arriving DURING the reap propagates instead — honoring
# the cancel beats reporting the dead transport).
call.cancel()
await asyncio.gather(call, return_exceptions=True)
raise ConnectionError("MCP transport owner died during discovery")
# If both futures completed in the same scheduling turn, owner death
# wins. Returning a successful list result after its backing transport
# has already exited would let the caller publish a catalog with no
# live session. Gathering a completed call also retrieves any exception
# it carried, avoiding an unobserved-task warning.
if owner.done():
if not call.done():
call.cancel()
await asyncio.gather(call, return_exceptions=True)
raise ConnectionError("MCP transport owner died during discovery")
# FIRST_COMPLETED returned normally and the owner is not done, so the
# discovery future is necessarily the completed member of the pair.
if call.cancelled():
# The discovery future was cancelled out from under us (an
# SDK-internal cancellation shape, not this method's own reap)
# — the transport can no longer answer, which is the same
# failure class as the owner dying. Convert instead of leaking
# a bare CancelledError the caller would misread as its own
# cancellation.
raise ConnectionError("MCP discovery request cancelled by transport failure")
return call.result() # normal result, or the real discovery error
async def _connect_one_pool(
self,
@@ -2985,10 +3059,7 @@ class MCPClientManager:
# ordering is irrelevant.
res_result, tmpl_result = await self._await_owner_discovery(
owner,
asyncio.gather(
session.list_resources(),
session.list_resource_templates(),
),
self._list_resource_pair(session, server_name),
)
except asyncio.CancelledError:
task = asyncio.current_task()
@@ -3068,6 +3139,15 @@ class MCPClientManager:
}
)
# Mirror the static commit guard. A transport owner can finish in the
# same scheduling turn as the final discovery response; never publish
# that response into the per-user maps after its session was evicted.
if owner.done() or entry.owner_task is not owner or entry.session is not session:
await self._teardown_pool_entry(key)
raise ConnectionError(
f"MCP pool server '{server_name}' transport died before catalog commit"
)
entry.tools = server_tools
entry.resources = server_resources if resources_cap is not None else None
entry.prompts = server_prompts if prompts_cap is not None else None
@@ -4320,7 +4400,7 @@ class MCPClientManager:
)
return added, removed
async def _list_resource_pair(self, session: Any) -> tuple[Any, Any]:
async def _list_resource_pair(self, session: Any, server_name: str) -> tuple[Any, Any]:
"""``list_resources`` + ``list_resource_templates`` in one bounded RTT.
The ONE copy of the paired-list protocol for both refresh twins
@@ -4329,8 +4409,10 @@ class MCPClientManager:
timeout budget and target disjoint catalogs (resources vs.
templates), so ordering is irrelevant.
Fail-FAST on the first real error a fast, meaningful failure
(an auth or method rejection) must surface as ITSELF, not be
Each method independently treats an exact JSON-RPC METHOD_NOT_FOUND as
an empty half-catalog. Fail-FAST on the first remaining real error a
fast, meaningful failure (for example auth or invalid params) must
surface as ITSELF, not be
masked behind a hung sibling's eventual ``TimeoutError`` — but
with the surviving sibling CANCELLED and REAPED inside this
scope before the error re-raises: bare fail-fast ``gather``
@@ -4350,8 +4432,10 @@ class MCPClientManager:
task, GC-reaped) rather than held onto.
"""
async with asyncio.timeout(self._CONNECT_TIMEOUT):
res_task = asyncio.create_task(session.list_resources())
tmpl_task = asyncio.create_task(session.list_resource_templates())
res_task = asyncio.create_task(_list_resources_compatible(session, server_name))
tmpl_task = asyncio.create_task(
_list_resource_templates_compatible(session, server_name)
)
try:
res_result, tmpl_result = await asyncio.gather(res_task, tmpl_task)
except BaseException:
@@ -4424,7 +4508,7 @@ class MCPClientManager:
user_id, server_name = key
old_uris = {r["uri"] for r in (entry.resources or []) if not r.get("template")}
res_result, tmpl_result = await self._list_resource_pair(session)
res_result, tmpl_result = await self._list_resource_pair(session, server_name)
if self._user_pool_entries.get(key) is not entry:
# Entry replaced mid-flight — stale result, discard.
return [], []
@@ -5035,7 +5119,7 @@ class MCPClientManager:
# turn the second list_resource_templates() call into AttributeError.
session = state.session
res_result, tmpl_result = await self._list_resource_pair(session)
res_result, tmpl_result = await self._list_resource_pair(session, name)
server_resources: list[dict[str, Any]] = []
# Capped like the pool twin — a misbehaving server's push must not
@@ -5687,15 +5771,76 @@ class MCPClientManager:
"error": "MCP event loop not running",
}
# Add to config so _refresh_all can reconnect on failure
# Add to config so _refresh_all can reconnect on failure.
self._server_configs[name] = cfg
future = asyncio.run_coroutine_threadsafe(self._connect_one(name, cfg), self._loop)
def _clear_failed_add_state() -> None:
"""Remove this registration's config and every public catalog."""
if self._server_configs.get(name) is cfg:
self._server_configs.pop(name, None)
failed_state = self._static_servers.get(name)
tools_changed = bool(failed_state and failed_state.tools)
resources_changed = bool(failed_state and failed_state.resources)
prompts_changed = bool(failed_state and failed_state.prompts)
if failed_state is not None:
failed_state.tools = []
failed_state.resources = []
failed_state.prompts = []
failed_state.supports_list_changed = False
failed_state.supports_resources = False
failed_state.supports_resource_list_changed = False
failed_state.supports_prompts = False
failed_state.supports_prompt_list_changed = False
if tools_changed:
self._rebuild_tools()
if resources_changed:
self._rebuild_resources()
if prompts_changed:
self._rebuild_prompts()
async def _add() -> None:
# Keep failure rollback under the SAME per-name lock as connect.
# The timeout is loop-owned: the sync bridge waits for a definitive
# outcome instead of racing a second wall-clock deadline against
# rollback. If the loop is temporarily blocked, this may report a
# late success, but it can never report failure while a published
# session/catalog remains live.
async with self._static_connect_lock_for(name):
try:
async with asyncio.timeout(timeout):
await self._connect_one_locked(name, cfg)
except BaseException as exc:
try:
await self._teardown_static_session(name)
finally:
# A newer concurrent add may already have replaced the
# config while waiting on this lock. Remove only the
# registration this coroutine owns; its successor will
# run after this cleanup releases the lock.
_clear_failed_add_state()
if isinstance(exc, TimeoutError):
raise TimeoutError(f"MCP server '{name}' registration timed out") from None
raise
future = asyncio.run_coroutine_threadsafe(_add(), self._loop)
try:
future.result(timeout=timeout)
# ``_add`` owns both its deadline and rollback. A caller-side
# timeout here could return a failed result while cleanup is merely
# queued on a stalled loop, exposing a live unconfigured tool.
future.result()
except concurrent.futures.TimeoutError:
return {
"connected": False,
"tools": 0,
"resources": 0,
"prompts": 0,
"error": f"MCP server '{name}' registration timed out",
}
except Exception as exc:
# Remove from configs on failure
self._server_configs.pop(name, None)
# The loop-side rollback normally removed this exact registration;
# retain a defensive cleanup for failures before _add could start.
if self._server_configs.get(name) is cfg:
self._server_configs.pop(name, None)
return {"connected": False, "tools": 0, "resources": 0, "prompts": 0, "error": str(exc)}
state = self._static_servers.get(name)
+98 -10
View File
@@ -855,13 +855,22 @@ def search_history_recent(limit: int = 20, *, user_id: str | None = None) -> lis
# -- Structured memories -------------------------------------------------------
def _require_memory_description(description: str) -> str:
"""Return a normalized description or raise the public validation error."""
if not isinstance(description, str) or not (normalized := description.strip()):
raise ValueError("memory description is required and must be non-empty")
return normalized
def save_structured_memory(
name: str,
content: str,
description: str | None = None,
description: str,
mem_type: str | None = None,
scope: str = "global",
scope_id: str = "",
*,
require_active_project: bool = False,
) -> tuple[dict[str, str] | None, bool]:
"""Save a structured memory as a single atomic upsert by name+scope+scope_id.
@@ -871,23 +880,65 @@ def save_structured_memory(
:meth:`StorageBackend.upsert_structured_memory`) -- no preceding read, no
IntegrityError round-trip, no TOCTOU window. ``(row, was_update)`` comes
straight from that upsert (this passes a fresh ``memory_id``, so a differing
returned id means an existing row was updated in place). A ``None``
description / ``mem_type`` means "leave unset" -- the column default applies
on insert and the stored value is kept on conflict.
returned id means an existing row was updated in place). ``description``
is required and must contain non-whitespace text for both inserts and
updates. A ``None`` ``mem_type`` keeps the stored value on an update and
uses the column default on insert.
"""
import uuid
name = normalize_key(name)
# Validate outside the best-effort storage boundary. Backend/driver
# ``ValueError`` instances remain operational failures; only this explicit
# caller-input check propagates.
normalized_description = _require_memory_description(description)
try:
row, was_update = get_storage().upsert_structured_memory(
str(uuid.uuid4()), name, description, mem_type, scope, scope_id, content
return save_structured_memory_strict(
name,
content,
description=normalized_description,
mem_type=mem_type,
scope=scope,
scope_id=scope_id,
require_active_project=require_active_project,
)
return (row, was_update) if row else (None, False)
except Exception:
log.warning("Failed to save structured memory name=%s", name, exc_info=True)
return None, False
def save_structured_memory_strict(
name: str,
content: str,
description: str,
mem_type: str | None = None,
scope: str = "global",
scope_id: str = "",
*,
require_active_project: bool = False,
) -> tuple[dict[str, str], bool]:
"""Strict structured-memory upsert for mutation-facing boundaries.
Unlike :func:`save_structured_memory`, storage failures propagate so an
API or tool cannot report a database outage as an ordinary failed/not-found
result. Best-effort internal callers keep using the facade.
"""
import uuid
normalized = normalize_key(name)
normalized_description = _require_memory_description(description)
row, was_update = get_storage().upsert_structured_memory(
str(uuid.uuid4()),
normalized,
normalized_description,
mem_type,
scope,
scope_id,
content,
require_active_project=require_active_project,
)
if not row:
raise RuntimeError("structured memory upsert returned no row")
return row, was_update
def get_structured_memory_by_name(
name: str, scope: str = "global", scope_id: str = ""
) -> dict[str, str] | None:
@@ -900,6 +951,13 @@ def get_structured_memory_by_name(
return None
def get_structured_memory_by_name_strict(
name: str, scope: str = "global", scope_id: str = ""
) -> dict[str, str] | None:
"""Strict scoped-name lookup; storage failures propagate."""
return get_storage().get_structured_memory_by_name(normalize_key(name), scope, scope_id)
def delete_structured_memory(name: str, scope: str = "global", scope_id: str = "") -> bool:
"""Delete a structured memory by name+scope. Returns True if existed."""
name = normalize_key(name)
@@ -919,6 +977,36 @@ def delete_structured_memory_by_id(memory_id: str) -> bool:
return False
def delete_structured_memory_returning_strict(
name: str, scope: str = "global", scope_id: str = ""
) -> dict[str, str] | None:
"""Atomically delete and return one scoped-name memory.
Storage failures propagate. A ``None`` return therefore means only that
no matching row existed at the mutation point.
"""
return get_storage().delete_structured_memory_returning(normalize_key(name), scope, scope_id)
def delete_structured_memory_by_id_returning_strict(
memory_id: str,
) -> dict[str, str] | None:
"""Atomically delete and return one memory by id; failures propagate."""
return get_storage().delete_structured_memory_by_id_returning(memory_id)
def find_structured_memory_scopes(
name: str,
scopes: list[tuple[str, str]],
) -> list[tuple[str, str]]:
"""Find visible same-name scope pairs in one metadata-only query."""
try:
return get_storage().find_structured_memory_scopes(normalize_key(name), scopes)
except Exception:
log.warning("Failed to find structured memory scopes name=%s", name, exc_info=True)
return []
def list_structured_memories(
mem_type: str = "",
scope: str = "",
+5 -5
View File
@@ -452,11 +452,11 @@ class OpenAIChatCompletionsProvider:
# operator-declared ``finish_reason_optional`` capability: on a
# server that never sends finish reasons, a stream that ended
# CLEANLY — the SDK ends iteration on [DONE]; an abrupt connection
# death raises httpx.TransportError out of this generator — after
# delivering output is a completed generation. Everywhere else a
# clean finish-less end is indistinguishable from a generation
# that died behind a clean-closing proxy/ASGI layer, so the shim
# stays DISARMED and the drain's complete-or-error gate raises
# death raises the active HTTP client's TransportError out of this
# generator — after delivering output is a completed generation.
# Everywhere else a clean finish-less end is indistinguishable
# from a generation that died behind a clean-closing proxy/ASGI layer,
# so the shim stays DISARMED and the drain's complete-or-error gate raises
# (retryable) instead of blessing possibly-truncated text.
# Reasoning counts as delivered output — a thinking model that
# spent its budget before emitting content is still a completed
+13 -7
View File
@@ -228,10 +228,12 @@ def transport_guarded(chunks: Iterator[StreamChunk]) -> Iterator[StreamChunk]:
Streaming moves the body read out of the SDK's
``APIConnectionError``-wrapped request into raw iteration, so a
mid-body wire death (connection drop, TLS record failure, read
timeout) surfaces as a bare ``httpx.TransportError`` no retry
predicate recognizes. This wrapper is that conversion rule made
reusable for consumers that keep streaming semantics (the
interactive loop); :func:`drain_stream` applies the same rule for
timeout) surfaces as a bare transport error no retry predicate
recognizes. OpenAI v3's default client raises ``httpx2`` errors;
Anthropic, Turnstone's own HTTP clients, and the OpenAI v3 legacy-client
escape hatch raise ``httpx`` errors. This wrapper is the one conversion
rule for both families, reusable by consumers that keep streaming
semantics (the interactive loop); :func:`drain_stream` applies it for
the single-shot lanes.
- A ``TransportError`` BEFORE any finish reason re-raises (chained)
@@ -243,7 +245,11 @@ def transport_guarded(chunks: Iterator[StreamChunk]) -> Iterator[StreamChunk]:
- Everything else chunks, exhaustion, non-transport exceptions
passes through untouched.
"""
import httpx # noqa: PLC0415 — heavyweight; deferred off the type-module import path
# Both libraries are heavyweight; keep them off this module's dataclass-
# only import path. ``httpx`` remains Turnstone's application transport,
# while ``httpx2`` is the OpenAI v3 default transport.
import httpx # noqa: PLC0415
import httpx2 # noqa: PLC0415
finish_seen = False
usage_seen = False
@@ -253,7 +259,7 @@ def transport_guarded(chunks: Iterator[StreamChunk]) -> Iterator[StreamChunk]:
sc = next(iterator)
except StopIteration:
return
except httpx.TransportError as exc:
except (httpx.TransportError, httpx2.TransportError) as exc:
if finish_seen:
# usage_captured distinguishes "completed result kept but
# its spend went missing from usage accounting" (the chat
@@ -348,7 +354,7 @@ def drain_stream(
Raises whatever the underlying stream raises retry/deadline/fallback
policy stays with the caller, exactly as with the old non-streaming
transport EXCEPT httpx transport failures, normalized by
transport EXCEPT HTTPX/HTTPX2 transport failures, normalized by
:func:`transport_guarded` (the one conversion rule, shared with the
interactive loop): a mid-body death before the finish reason is
re-raised (chained) as :class:`IncompleteStreamError`, restoring the
+714 -542
View File
File diff suppressed because it is too large Load Diff
+13 -4
View File
@@ -22,9 +22,17 @@ if TYPE_CHECKING:
from turnstone.core.session import ChatSession
def request_replay_project_name(request: Any) -> str:
"""Return project metadata pre-resolved by the shared SSE route."""
value = getattr(getattr(request, "state", None), "_session_replay_project_name", "")
return value if isinstance(value, str) else ""
def session_replay_preamble(
session: ChatSession | None,
ui: Any,
*,
project_name: str = "",
) -> Iterable[dict[str, Any]]:
"""Yield ``connected`` + optional ``status`` events for an SSE replay.
@@ -33,7 +41,8 @@ def session_replay_preamble(
through to the kind-specific tail.
- ``connected`` carries ``model`` / ``model_alias`` / ``skip_permissions``
so the per-tab status bar populates the model cell before any
history arrives.
history arrives. The caller supplies the project display name already
resolved for the authenticated connection principal.
- ``status`` only fires when ``session._last_usage`` exists (a
session that has completed at least one turn). The payload shape
matches :meth:`SessionUI.on_status` so live ticks and replays use
@@ -48,9 +57,9 @@ def session_replay_preamble(
"type": "connected",
"model": session.model,
"model_alias": session.model_alias or "",
# The attached project's display name (""=none) so the composer can
# paint its "has a project" badge on connect, beside the model chip.
"project_name": getattr(session, "_project_name", "") or "",
# The visible attached-project display name (""=none) so the composer
# can paint its project badge on connect, beside the model chip.
"project_name": project_name,
"skip_permissions": getattr(ui, "auto_approve", False),
}
+52 -27
View File
@@ -173,15 +173,11 @@ class EventsReplay(Protocol):
live event loop starts. Each yielded dict gets JSON-serialised
and sent as a single ``data:`` line to the client.
Interactive yields four things on connect: ``connected`` (model +
skip_permissions), ``status`` (token usage + context %, only when
``session._last_usage`` exists), ``history`` (replayed conversation),
and ``pending_approval`` + cached intent verdicts. Coord yields
just one: ``pending_approval`` (the rest aren't needed because
coord's dashboard fetches history via a separate ``/history``
endpoint and doesn't render the per-tab status bar). Kinds that
don't need any pre-replay wire ``None`` and the live loop starts
immediately.
The shared handler pre-resolves viewer-specific project metadata onto the
request before calling this stable three-argument callback. Production
callbacks emit ``connected`` plus optional ``status``, then pending controls
and cached verdicts. Conversation history stays on the separate REST
endpoint. Kinds without replay wire ``None``.
"""
def __call__(self, ws: Workstream, ui: Any, request: Request) -> Iterable[dict[str, Any]]:
@@ -444,13 +440,11 @@ class SessionEndpointConfig:
# wires ``None`` and lets the cluster collector handle the
# transition via ``CoordinatorAdapter.emit_rehydrated``.
open_post_load: OpenPostLoad | None = None
# (ws, ui, request) -> Iterable[dict]. Kind-specific initial
# SSE replay payload the lifted ``events`` body yields after
# registering the per-UI listener queue but before the live
# event loop. Both production kinds yield connected + optional status,
# followed by pending approval controls and cached verdicts. Conversation
# history stays on the separate REST ``/history`` bootstrap. Kinds that
# don't need pre-replay wire ``None``.
# (ws, ui, request) -> Iterable[dict]. Kind-specific initial SSE replay
# payload the lifted ``events`` body yields before the live event loop.
# The shared handler pins viewer-specific project metadata on ``request``;
# production callbacks combine it with connected/status and pending
# controls. Conversation history stays on the separate REST bootstrap.
events_replay: EventsReplay | None = None
# (request) -> Executor for the SSE live-loop's blocking
# ``queue.get`` wait. Interactive returns the dedicated
@@ -2298,15 +2292,15 @@ def make_open_handler(
def make_events_handler(cfg: SessionEndpointConfig) -> Handler:
"""Lifted body for ``GET {prefix}/{ws_id}/events`` — per-workstream SSE.
Both kinds share the SSE plumbing: register the per-UI listener
queue, run the kind-specific initial replay (``cfg.events_replay``,
typically ``connected`` + ``status`` + ``history`` + pending
approval / plan on interactive; just pending approval / plan on
coord), then drain the queue forever until either the workstream
closes (``ws_closed`` event) or the client disconnects.
Both kinds share the SSE plumbing: resolve viewer-specific project
metadata off-loop, register the per-UI listener queue, run the configured
initial replay (``cfg.events_replay``), then drain the queue forever until
either the workstream closes (``ws_closed`` event) or the client
disconnects. Cursor-only replay emits the shared connected/status preamble
directly because it deliberately skips kind-specific pending controls.
The kind-specific divergence is captured entirely by
``cfg.events_replay``. The live-loop body, the listener
The kind-specific replay divergence is captured by
``cfg.events_replay``. The cursor preamble, live-loop body, listener
registration, the ``ws_closed`` exit, the disconnect detection,
and the SSE-connect/disconnect metric recording are uniform.
@@ -2432,6 +2426,33 @@ def make_events_handler(cfg: SessionEndpointConfig) -> Handler:
# ``ui`` is a ``SessionUIBase`` subclass, so the cast is
# tightening the type, not weakening it.
ui_base = cast("SessionUIBase", ui)
from turnstone.core.web_helpers import auth_user_id
# Pin the authenticated viewer once per connection. Resolve optional
# project presentation metadata off the event loop and before listener
# registration, so database latency cannot stall unrelated async work
# or let this listener's queue accumulate while it waits.
replay_principal_id = auth_user_id(request).strip()
replay_project_name = ""
session = getattr(ws, "session", None)
project_name_for_principal = getattr(session, "project_name_for_principal", None)
if callable(project_name_for_principal):
try:
resolved_project_name = await asyncio.to_thread(
project_name_for_principal,
replay_principal_id,
)
if isinstance(resolved_project_name, str):
replay_project_name = resolved_project_name
except Exception:
# Project metadata is optional presentation context. Fail
# closed to no badge without dropping the SSE bootstrap.
log.debug(
"ws.events.project_name_failed ws=%s",
ws_id[:8],
exc_info=True,
)
request.state._session_replay_project_name = replay_project_name
replay_status: str
replay_events: list[dict[str, Any]] = []
lost_count = 0
@@ -2675,7 +2696,11 @@ def make_events_handler(cfg: SessionEndpointConfig) -> Handler:
# contract lives in session_replay_preamble alone (it
# no-ops on a detached session internally).
try:
for ev in session_replay_preamble(ws.session, ui):
for ev in session_replay_preamble(
ws.session,
ui,
project_name=replay_project_name,
):
yield {"data": json.dumps(ev)}
except Exception:
log.debug(
@@ -2745,8 +2770,8 @@ def make_events_handler(cfg: SessionEndpointConfig) -> Handler:
)
}
# Replay phase — stream the kind-specific initial
# payload one event at a time so the client sees
# Replay phase — stream the kind-specific initial payload
# one event at a time so the client sees
# the first byte immediately. Pre-building into
# a list would block time-to-first-byte until the
# entire replay materialized AND let the listener
+85 -20
View File
@@ -4746,6 +4746,9 @@ class PostgreSQLBackend(_KeyedAttachmentSaveWrappers):
scope_id: str,
content: str,
) -> None:
if description is None or not description.strip():
raise ValueError("memory description is required and must be non-empty")
description = description.strip()
now = datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%S")
with self._conn() as conn:
conn.execute(
@@ -4770,19 +4773,24 @@ class PostgreSQLBackend(_KeyedAttachmentSaveWrappers):
self,
memory_id: str,
name: str,
description: str | None,
description: str,
mem_type: str | None,
scope: str,
scope_id: str,
content: str,
*,
require_active_project: bool = False,
) -> tuple[dict[str, str], bool]:
from sqlalchemy.dialects.postgresql import insert as pg_insert
if description is None or not description.strip():
raise ValueError("memory description is required and must be non-empty")
description = description.strip()
now = datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%S")
insert_stmt = pg_insert(structured_memories).values(
memory_id=memory_id,
name=name,
description="" if description is None else description,
description=description,
type="general" if mem_type is None else mem_type,
scope=scope,
scope_id=scope_id,
@@ -4792,16 +4800,14 @@ class PostgreSQLBackend(_KeyedAttachmentSaveWrappers):
last_accessed=now,
access_count=0,
)
# On conflict, refresh content + timestamps. description/type are
# overwritten only when the caller supplied them; None means "unset" ->
# keep the stored value. created and access_count are left untouched.
# On conflict, refresh content, description, and timestamps. A None type means
# "unset" -> keep the stored value. created/access_count stay untouched.
set_: dict[str, Any] = {
"content": insert_stmt.excluded.content,
"updated": now,
"last_accessed": now,
}
if description is not None:
set_["description"] = insert_stmt.excluded.description
set_["description"] = insert_stmt.excluded.description
if mem_type is not None:
set_["type"] = insert_stmt.excluded.type
stmt = insert_stmt.on_conflict_do_update(
@@ -4809,6 +4815,22 @@ class PostgreSQLBackend(_KeyedAttachmentSaveWrappers):
set_=set_,
).returning(structured_memories)
with self._conn() as conn:
if require_active_project:
if scope != "project" or not scope_id:
raise ValueError("active-project guard requires project scope")
project = conn.execute(
sa.select(projects.c.project_id)
.where(
sa.and_(
projects.c.project_id == scope_id,
projects.c.state == "active",
)
)
.with_for_update(read=True)
).fetchone()
if project is None:
conn.rollback()
raise ValueError("project is missing, archived, or no longer writable")
row = conn.execute(stmt).fetchone()
conn.commit()
if row is None: # unreachable: ON CONFLICT DO UPDATE returns one row
@@ -4841,26 +4863,58 @@ class PostgreSQLBackend(_KeyedAttachmentSaveWrappers):
def delete_structured_memory(
self, name: str, scope: str = "global", scope_id: str = ""
) -> bool:
with self._conn() as conn:
result = conn.execute(
sa.delete(structured_memories).where(
sa.and_(
structured_memories.c.name == name,
structured_memories.c.scope == scope,
structured_memories.c.scope_id == scope_id,
)
return self.delete_structured_memory_returning(name, scope, scope_id) is not None
def delete_structured_memory_returning(
self, name: str, scope: str = "global", scope_id: str = ""
) -> dict[str, str] | None:
stmt = (
sa.delete(structured_memories)
.where(
sa.and_(
structured_memories.c.name == name,
structured_memories.c.scope == scope,
structured_memories.c.scope_id == scope_id,
)
)
.returning(structured_memories)
)
with self._conn() as conn:
row = conn.execute(stmt).fetchone()
conn.commit()
return result.rowcount > 0
return dict(row._mapping) if row is not None else None
def delete_structured_memory_by_id(self, memory_id: str) -> bool:
return self.delete_structured_memory_by_id_returning(memory_id) is not None
def delete_structured_memory_by_id_returning(self, memory_id: str) -> dict[str, str] | None:
with self._conn() as conn:
result = conn.execute(
sa.delete(structured_memories).where(structured_memories.c.memory_id == memory_id)
)
row = conn.execute(
sa.delete(structured_memories)
.where(structured_memories.c.memory_id == memory_id)
.returning(structured_memories)
).fetchone()
conn.commit()
return result.rowcount > 0
return dict(row._mapping) if row is not None else None
def find_structured_memory_scopes(
self,
name: str,
scopes: list[tuple[str, str]],
) -> list[tuple[str, str]]:
if not scopes:
return []
with self._conn() as conn:
scope_clauses, params = self._build_scope_or_clause(scopes)
rows = conn.execute(
sa.text(
"SELECT scope, scope_id FROM structured_memories "
f"WHERE name = :name AND ({scope_clauses}) "
"ORDER BY scope, scope_id"
),
{**params, "name": name},
).fetchall()
return [(str(row.scope), str(row.scope_id)) for row in rows]
def list_structured_memories(
self,
@@ -5905,6 +5959,17 @@ class PostgreSQLBackend(_KeyedAttachmentSaveWrappers):
def delete_project(self, project_id: str) -> bool:
with self._conn() as conn:
# Serialize with guarded project-memory upserts. If a writer got
# the row first, its memory is committed before our purge; if this
# delete wins, the later writer's active-project check finds no row.
project = conn.execute(
sa.select(projects.c.project_id)
.where(projects.c.project_id == project_id)
.with_for_update()
).fetchone()
if project is None:
conn.rollback()
return False
# No FK cascade in the schema family, so purge the project's scoped
# memory + member rows explicitly (same transaction) before the
# project row — honouring the "destroys the container AND its scoped
+31 -6
View File
@@ -829,34 +829,41 @@ class StorageBackend(Protocol):
scope_id: str,
content: str,
) -> None:
"""Create a structured memory record."""
"""Create a structured memory record with a non-empty description."""
...
def upsert_structured_memory(
self,
memory_id: str,
name: str,
description: str | None,
description: str,
mem_type: str | None,
scope: str,
scope_id: str,
content: str,
*,
require_active_project: bool = False,
) -> tuple[dict[str, str], bool]:
"""Insert a structured memory, or update it in place on a
``(name, scope, scope_id)`` conflict.
Atomic ``INSERT ... ON CONFLICT DO UPDATE ... RETURNING`` no
IntegrityError round-trip, race-safe under concurrent saves of the same
key. ``description`` / ``mem_type`` of ``None`` mean "unset": the
column default ("" / "general") is used on insert and the stored value
is kept on conflict; a non-``None`` value (including "" or "general") is
written.
key. ``description`` must contain non-whitespace text on every insert
or update. A ``mem_type`` of ``None`` means "unset": the column default
is used on insert and the stored value is kept on conflict.
Returns ``(row, was_update)`` (like Django's ``update_or_create``): the
full saved row, and ``True`` when an existing row was updated rather
than inserted. Callers MUST supply a fresh unique ``memory_id`` it is
compared against the returned row's id to tell INSERT from UPDATE, so a
reused id would report ``was_update=False`` on a real update.
When ``require_active_project`` is true, ``scope`` must be
``"project"`` and the backend must verify that the referenced project
still exists and is active in the same transaction as the upsert. The
project row is locked where the backend supports row locks so a
concurrent project delete cannot leave an orphaned memory behind.
"""
...
@@ -876,10 +883,28 @@ class StorageBackend(Protocol):
"""Delete a structured memory by (name, scope, scope_id). Returns True if existed."""
...
def delete_structured_memory_returning(
self, name: str, scope: str = "global", scope_id: str = ""
) -> dict[str, str] | None:
"""Atomically delete and return a memory selected by its scoped name."""
...
def delete_structured_memory_by_id(self, memory_id: str) -> bool:
"""Delete a structured memory by its primary key. Returns True if existed."""
...
def delete_structured_memory_by_id_returning(self, memory_id: str) -> dict[str, str] | None:
"""Atomically delete and return a memory selected by primary key."""
...
def find_structured_memory_scopes(
self,
name: str,
scopes: list[tuple[str, str]],
) -> list[tuple[str, str]]:
"""Return visible scope pairs containing ``name`` in one small query."""
...
def list_structured_memories(
self,
mem_type: str = "",
+79 -20
View File
@@ -4818,6 +4818,9 @@ class SQLiteBackend(_KeyedAttachmentSaveWrappers):
scope_id: str,
content: str,
) -> None:
if description is None or not description.strip():
raise ValueError("memory description is required and must be non-empty")
description = description.strip()
now = datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%S")
with self._conn() as conn:
conn.execute(
@@ -4842,19 +4845,24 @@ class SQLiteBackend(_KeyedAttachmentSaveWrappers):
self,
memory_id: str,
name: str,
description: str | None,
description: str,
mem_type: str | None,
scope: str,
scope_id: str,
content: str,
*,
require_active_project: bool = False,
) -> tuple[dict[str, str], bool]:
from sqlalchemy.dialects.sqlite import insert as sqlite_insert
if description is None or not description.strip():
raise ValueError("memory description is required and must be non-empty")
description = description.strip()
now = datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%S")
insert_stmt = sqlite_insert(structured_memories).values(
memory_id=memory_id,
name=name,
description="" if description is None else description,
description=description,
type="general" if mem_type is None else mem_type,
scope=scope,
scope_id=scope_id,
@@ -4864,16 +4872,14 @@ class SQLiteBackend(_KeyedAttachmentSaveWrappers):
last_accessed=now,
access_count=0,
)
# On conflict, refresh content + timestamps. description/type are
# overwritten only when the caller supplied them; None means "unset" ->
# keep the stored value. created and access_count are left untouched.
# On conflict, refresh content, description, and timestamps. A None type means
# "unset" -> keep the stored value. created/access_count stay untouched.
set_: dict[str, Any] = {
"content": insert_stmt.excluded.content,
"updated": now,
"last_accessed": now,
}
if description is not None:
set_["description"] = insert_stmt.excluded.description
set_["description"] = insert_stmt.excluded.description
if mem_type is not None:
set_["type"] = insert_stmt.excluded.type
stmt = insert_stmt.on_conflict_do_update(
@@ -4881,6 +4887,24 @@ class SQLiteBackend(_KeyedAttachmentSaveWrappers):
set_=set_,
).returning(structured_memories)
with self._conn() as conn:
if require_active_project:
if scope != "project" or not scope_id:
raise ValueError("active-project guard requires project scope")
# SQLite has no row locks. Taking the writer lock before the
# existence check serializes this transaction with
# ``delete_project`` (which uses the same prologue).
conn.execute(sa.text("BEGIN IMMEDIATE"))
project = conn.execute(
sa.select(projects.c.project_id).where(
sa.and_(
projects.c.project_id == scope_id,
projects.c.state == "active",
)
)
).fetchone()
if project is None:
conn.rollback()
raise ValueError("project is missing, archived, or no longer writable")
row = conn.execute(stmt).fetchone()
conn.commit()
if row is None: # unreachable: ON CONFLICT DO UPDATE returns one row
@@ -4913,26 +4937,58 @@ class SQLiteBackend(_KeyedAttachmentSaveWrappers):
def delete_structured_memory(
self, name: str, scope: str = "global", scope_id: str = ""
) -> bool:
with self._conn() as conn:
result = conn.execute(
sa.delete(structured_memories).where(
sa.and_(
structured_memories.c.name == name,
structured_memories.c.scope == scope,
structured_memories.c.scope_id == scope_id,
)
return self.delete_structured_memory_returning(name, scope, scope_id) is not None
def delete_structured_memory_returning(
self, name: str, scope: str = "global", scope_id: str = ""
) -> dict[str, str] | None:
stmt = (
sa.delete(structured_memories)
.where(
sa.and_(
structured_memories.c.name == name,
structured_memories.c.scope == scope,
structured_memories.c.scope_id == scope_id,
)
)
.returning(structured_memories)
)
with self._conn() as conn:
row = conn.execute(stmt).fetchone()
conn.commit()
return result.rowcount > 0
return dict(row._mapping) if row is not None else None
def delete_structured_memory_by_id(self, memory_id: str) -> bool:
return self.delete_structured_memory_by_id_returning(memory_id) is not None
def delete_structured_memory_by_id_returning(self, memory_id: str) -> dict[str, str] | None:
with self._conn() as conn:
result = conn.execute(
sa.delete(structured_memories).where(structured_memories.c.memory_id == memory_id)
)
row = conn.execute(
sa.delete(structured_memories)
.where(structured_memories.c.memory_id == memory_id)
.returning(structured_memories)
).fetchone()
conn.commit()
return result.rowcount > 0
return dict(row._mapping) if row is not None else None
def find_structured_memory_scopes(
self,
name: str,
scopes: list[tuple[str, str]],
) -> list[tuple[str, str]]:
if not scopes:
return []
with self._conn() as conn:
scope_clauses, params = self._build_scope_or_clause(scopes)
rows = conn.execute(
sa.text(
"SELECT scope, scope_id FROM structured_memories "
f"WHERE name = :name AND ({scope_clauses}) "
"ORDER BY scope, scope_id"
),
{**params, "name": name},
).fetchall()
return [(str(row.scope), str(row.scope_id)) for row in rows]
def list_structured_memories(
self,
@@ -5961,6 +6017,9 @@ class SQLiteBackend(_KeyedAttachmentSaveWrappers):
def delete_project(self, project_id: str) -> bool:
with self._conn() as conn:
# Serialize with guarded project-memory upserts before inspecting
# or deleting the container.
conn.execute(sa.text("BEGIN IMMEDIATE"))
# No FK cascade in the schema family, so purge the project's scoped
# memory + member rows explicitly (same transaction) before the
# project row — honouring the "destroys the container AND its scoped
+5 -6
View File
@@ -64,12 +64,11 @@ def _apply_kind_variant(tool: dict[str, Any], kind: str, meta: dict[str, Any]) -
"""Return a kind-specific copy of ``tool`` with description / params overridden.
Each kind sees only the surface it can actually use for ``memory``,
coord sessions get a description + scope enum that mention only the
``coordinator`` scope, while interactive sessions get a description
+ scope enum that omit ``coordinator`` entirely. This keeps the
LLM contract tight: the model never sees enum values it can't use,
and never reads description sentences explaining why a scope is
forbidden.
coord sessions get ``coordinator`` plus the attach-dependent ``project``
scope, while interactive sessions get global/workstream/user plus
``project``. This keeps the LLM contract tight: the model never sees enum
values it can't use, and never reads description sentences explaining why
a scope is forbidden.
No-op (returns the input tool unchanged) when the tool has no
``kind_variants`` metadata or no entry for ``kind``. Otherwise
+107 -2
View File
@@ -3,15 +3,23 @@
from __future__ import annotations
import contextlib
import hashlib
import json
import os
import re
from functools import cached_property
from typing import TYPE_CHECKING, Any
from starlette.datastructures import Headers
from starlette.exceptions import HTTPException
from starlette.responses import FileResponse, PlainTextResponse, Response
from starlette.staticfiles import StaticFiles
if TYPE_CHECKING:
from starlette.middleware import Middleware
from starlette.requests import Request
from starlette.responses import JSONResponse
from starlette.types import Scope
def latin1_safe_filename(name: str, *, fallback: str = "attachment") -> str:
@@ -463,6 +471,103 @@ def cors_middleware(origins: list[str]) -> Middleware:
# Static asset cache-busting
# ---------------------------------------------------------------------------
# Keep this directory definition shared by the HTML rewriter and static response
# policy. These directories already carry their library version in the URL,
# so their contents can be cached immutably; every other first-party asset must
# revalidate because ES-module imports do not inherit the entry module's ?v=.
_VERSIONED_VENDOR_DIR = r"(?:katex|hljs|hls|mermaid)-\d+(?:\.\d+)+"
_VERSIONED_VENDOR_PATH_RE = re.compile(rf"^{_VERSIONED_VENDOR_DIR}/")
def is_safe_static_asset_path(path: str) -> bool:
"""Return whether *path* is a canonical mount-relative asset path.
Starlette decodes percent escapes before populating ``{path:path}``, and
HTTPX normalizes dot segments when it builds an outbound URL. Reject the
ambiguous forms before a console proxy can leave its static mount.
"""
return (
bool(path)
and "\\" not in path
and all(segment not in {"", ".", ".."} for segment in path.split("/"))
)
def static_asset_cache_control(path: str) -> str:
"""Return the cache policy for an asset path relative to its mount."""
if not is_safe_static_asset_path(path):
return "no-store"
if _VERSIONED_VENDOR_PATH_RE.match(path):
return "public, max-age=31536000, immutable"
return "no-cache"
class RevalidatingStaticFiles(StaticFiles):
"""StaticFiles with correctness-first caching for first-party assets.
First-party files use a content-derived ETag and no Last-Modified header.
``no-cache`` therefore revalidates by content on reload, even when two
builds share a package version, byte length, and filesystem timestamp.
Version-named vendor directories retain immutable caching.
"""
@cached_property
def _content_etags(self) -> dict[str, tuple[tuple[int, int, int], str]]:
"""Lazily allocate the bounded per-static-tree validator cache."""
return {}
def _content_etag(
self,
full_path: str | os.PathLike[str],
stat_result: os.stat_result,
) -> str:
path = os.fspath(full_path)
signature = (stat_result.st_mtime_ns, stat_result.st_ctime_ns, stat_result.st_size)
cached = self._content_etags.get(path)
if cached is not None and cached[0] == signature:
return cached[1]
with open(path, "rb") as asset:
digest = hashlib.file_digest(asset, "sha256").hexdigest()
etag = f'"sha256-{digest}"'
self._content_etags[path] = (signature, etag)
return etag
def file_response(
self,
full_path: str | os.PathLike[str],
stat_result: os.stat_result,
scope: Scope,
status_code: int = 200,
) -> Response:
asset_path = self.get_path(scope)
if static_asset_cache_control(asset_path) != "no-cache":
return super().file_response(full_path, stat_result, scope, status_code)
request_headers = Headers(scope=scope)
response = FileResponse(full_path, status_code=status_code, stat_result=stat_result)
etag = self._content_etag(full_path, stat_result)
response.headers["ETag"] = etag
del response.headers["Last-Modified"]
if self.is_not_modified(response.headers, request_headers):
return Response(status_code=304, headers={"ETag": etag})
return response
async def get_response(self, path: str, scope: Scope) -> Response:
try:
response = await super().get_response(path, scope)
except HTTPException as exc:
if exc.status_code != 404:
raise
return PlainTextResponse(
"Not Found", status_code=404, headers={"Cache-Control": "no-store"}
)
cache_control = (
"no-store" if response.status_code >= 400 else static_asset_cache_control(path)
)
response.headers["Cache-Control"] = cache_control
return response
# Matches src="/static/..." and href="/shared/..." (and vice-versa) but skips
# vendored libraries whose directory names already contain a version number
# (e.g. katex-0.16.44/, hljs-11.11.1/) and URLs that already have a query
@@ -470,7 +575,7 @@ def cors_middleware(origins: list[str]) -> Middleware:
_ASSET_RE = re.compile(
r'(?P<attr>(?:src|href)=")'
r"(?P<path>/(?:static|shared)/)"
r"(?!(?:katex|hljs|hls|mermaid)-\d)"
rf"(?!{_VERSIONED_VENDOR_DIR}/)"
r'(?P<file>[^"?]+)"'
)
@@ -480,7 +585,7 @@ def version_html(html: str) -> str:
Vendored libraries with version-bearing directory names are skipped.
URLs that already contain a query string are left unchanged.
Called once at startup when loading HTML into memory.
Safe to call either at startup or while rendering a dynamic template.
"""
from turnstone import __version__
+8 -8
View File
@@ -232,7 +232,7 @@ class HeadlessSession(ChatSession):
- auto_approve is always True
- Tool calls are recorded into a structured log
- All stdout output is suppressed
- send_headless() uses non-streaming API
- send_headless() drains the production streaming provider path
"""
def __init__(
@@ -307,8 +307,8 @@ class HeadlessSession(ChatSession):
) -> list[dict[str, Any]]:
"""Run a complete conversation turn headlessly.
Uses non-streaming API calls. Captures all tool calls into
self.tool_call_log.
Drains production streaming provider calls into single-shot results.
Captures all tool calls into ``self.tool_call_log``.
Returns the tool call log: list of dicts with keys:
tool: str, args: dict, result: str (truncated), ok: bool,
@@ -533,9 +533,9 @@ def run_with_lifecycle(
past the teardown (which closes only the last one built).
* *drive* (returned by ``build_session``) submitted to a
one-worker executor and bounded by ``future.result(test_timeout)``.
The per-request httpx timeout cannot bound a STREAM: a trickling
response resets the read timeout indefinitely, so without the
wall clock a hung generation occupies a run slot forever. On
The per-request HTTP transport timeout cannot bound a STREAM: a
trickling response resets the read timeout indefinitely, so without
the wall clock a hung generation occupies a run slot forever. On
timeout the session is DROPPED, not closed: the shutdown did not
wait, so the worker is still inside the drive, and ``close()``'s
bounded shell-join would trade a bounded leak for a blocked
@@ -736,8 +736,8 @@ def _run_single_test(
)
def _build_client() -> Any:
# Per-attempt client with request-level timeout so httpx aborts
# the HTTP request itself — no zombie connections on the server.
# Per-attempt client with a per-read timeout. The executor wall clock
# above separately bounds a trickling stream that keeps resetting it.
return OpenAI(
base_url=client.base_url,
api_key=client.api_key,
+5 -6
View File
@@ -259,9 +259,8 @@ def _tasks_action_enum() -> frozenset[str]:
#
# Neither half is a literal here. The action VOCABULARY comes from the
# tool's own schema (:func:`_tasks_action_enum`) and the READ half is
# ``ChatSession._TASKS_READ_ACTIONS``, production's own classifier
# the one its parallel-batch guard and approval path rule on. Mutating
# is the remainder, so an action added to the schema counts as a
# ``_TASKS_READ_ACTIONS``, production's own preparer classifier.
# Mutating is the remainder, so an action added to the schema counts as a
# mutation until production classifies it as a read: a new write can
# never be silently dropped from the bookkeeping test, and the drift
# that IS possible (a new read) is caught statically by
@@ -824,7 +823,7 @@ def _seed_world(storage: Any, case: dict[str, Any]) -> None:
saved, _was_update = save_structured_memory(
row["name"],
row["content"],
row.get("description"),
row["description"],
row.get("type"),
scope=row.get("scope", "global"),
scope_id=row.get("scope_id", ""),
@@ -1969,7 +1968,7 @@ def _check_world_is_seedable(case: dict[str, Any]) -> str | None:
Recognized keys only (``memory`` / ``nodes``) an unrecognized key
is a silent no-op seed, which reads as "seeded" while leaving the
hollow world the block exists to fill. Memory rows need non-empty
string ``name`` and ``content`` (the production upsert's own
string ``name``, ``description``, and ``content`` (the production upsert's own
requirements, surfaced at authoring time); node rows need a
non-empty string ``node_id``.
"""
@@ -1984,7 +1983,7 @@ def _check_world_is_seedable(case: dict[str, Any]) -> str | None:
for i, row in enumerate(world.get("memory", ())):
if not isinstance(row, dict):
return f"world.memory[{i}] must be a dict"
for field in ("name", "content"):
for field in ("name", "content", "description"):
v = row.get(field)
if not isinstance(v, str) or not v.strip():
return f"world.memory[{i}].{field} must be a non-empty string"
+2
View File
@@ -95,6 +95,7 @@ NUDGE_CELLS: list[dict[str, Any]] = [
"memory": [
{
"name": "acme-api-project",
"description": "Acme API repository and deployment context",
"type": "reference",
"content": (
"acme-api: FastAPI service. Repo layout: "
@@ -105,6 +106,7 @@ NUDGE_CELLS: list[dict[str, Any]] = [
},
{
"name": "auth-backend-migration-status",
"description": "Current authentication migration status",
"content": (
"migrations/007_auth_backend.sql applied on the "
"staging replica; auth service suite green "
+6 -4
View File
@@ -525,19 +525,21 @@ class AsyncTurnstoneServer(_BaseClient):
name: str,
content: str,
*,
description: str = "",
description: str,
mem_type: str = "general",
scope: str = "global",
scope_id: str = "",
) -> MemoryInfo:
description = (description or "").strip()
if not description:
raise ValueError("memory description is required and must be non-empty")
body: dict[str, Any] = {
"name": name,
"content": content,
"description": description,
"type": mem_type,
"scope": scope,
}
if description:
body["description"] = description
if scope_id:
body["scope_id"] = scope_id
return await self._request(
@@ -868,7 +870,7 @@ class TurnstoneServer:
name: str,
content: str,
*,
description: str = "",
description: str,
mem_type: str = "general",
scope: str = "global",
scope_id: str = "",
+237 -66
View File
@@ -42,7 +42,6 @@ from starlette.middleware import Middleware
from starlette.requests import Request
from starlette.responses import HTMLResponse, JSONResponse, Response, StreamingResponse
from starlette.routing import Mount, Route
from starlette.staticfiles import StaticFiles
from turnstone import __version__
from turnstone.api.docs import make_docs_handler, make_openapi_handler
@@ -78,7 +77,10 @@ from turnstone.core.session_manager import (
STALE_CREATE_SWEEP_INTERVAL_SECONDS,
SessionManager,
)
from turnstone.core.session_replay import session_replay_preamble
from turnstone.core.session_replay import (
request_replay_project_name,
session_replay_preamble,
)
from turnstone.core.session_routes import (
AttachmentUploadHelpers,
CreatePreCommitError,
@@ -111,6 +113,7 @@ from turnstone.core.session_ui_base import (
)
from turnstone.core.tools import TOOLS # noqa: F401 — available for introspection
from turnstone.core.trajectory import final_assistant_text
from turnstone.core.web_helpers import RevalidatingStaticFiles
from turnstone.core.web_helpers import version_html as _version_html
from turnstone.core.workstream import (
Workstream,
@@ -801,12 +804,9 @@ def _interactive_events_replay(
) -> Iterable[dict[str, Any]]:
"""Initial SSE replay payload for interactive ``events`` connections.
Yields a ``connected`` event (model + skip_permissions), a
``status`` event with the workstream's last token usage + context %
(when a turn has completed), and the pending approval prompt + cached
intent verdicts (if a prompt is pending). The lifted
``make_events_handler`` body delegates that yield sequence to this
callback so the kind-specific shape stays in this module.
Yields the shared ``connected`` / ``status`` preamble, followed by pending
approval prompts and cached intent verdicts. The lifted handler resolves
viewer-specific project metadata off-loop before invoking this callback.
Conversation history is NOT replayed over SSE: the frontend fetches
it via ``GET /history`` on page load and re-fetches on the
@@ -821,10 +821,11 @@ def _interactive_events_replay(
# session can still be detached on the close-then-reopen path.
return
# Connected + status preamble — same shape coord replays and the
# lifted reconnect path use; one shared function, no per-kind
# wrapper, so a future field add cannot land on one surface only.
yield from session_replay_preamble(ws.session, ui)
yield from session_replay_preamble(
ws.session,
ui,
project_name=request_replay_project_name(request),
)
# Pending approval re-injection (so a reconnecting tab sees the
# prompt) + cached LLM verdicts received since the prompt fired.
@@ -3495,31 +3496,157 @@ def _resolve_user_scope_id(
return uid, None
def _resolve_workstream_memory_scope_id(
request: Request,
scope_id: str,
) -> tuple[str, JSONResponse | None]:
"""Bind REST workstream-memory access to the authenticated owner."""
from turnstone.core.storage._registry import get_storage
resolved = scope_id.strip()
if not resolved:
return "", JSONResponse(
{"error": "scope_id is required for workstream scope"},
status_code=400,
)
owner = get_storage().get_workstream_owner(resolved)
if owner is None:
return "", JSONResponse({"error": "Workstream not found"}, status_code=404)
caller = _auth_user_id(request)
if "service" not in _auth_scopes(request) and (not caller or owner != caller):
return "", JSONResponse(
{"error": "Cannot access another user's workstream memories"},
status_code=403,
)
return resolved, None
def _resolve_rest_memory_scope(
request: Request,
scope: str,
scope_id: str,
*,
allow_empty: bool,
) -> tuple[str, str, JSONResponse | None]:
"""Validate a public memory scope and bind its caller-controlled id."""
normalized_scope = scope.strip().lower()
normalized_id = scope_id.strip()
if not normalized_scope and allow_empty:
err = _validate_scope_scope_id(normalized_scope, normalized_id)
return normalized_scope, normalized_id, err
if normalized_scope not in _VALID_MEMORY_SCOPES:
return (
"",
"",
JSONResponse(
{
"error": (
f"invalid scope: {normalized_scope}; "
f"must be one of {sorted(_VALID_MEMORY_SCOPES)}"
)
},
status_code=400,
),
)
if normalized_scope == "user":
normalized_id, err = _resolve_user_scope_id(request, normalized_id)
if err:
return "", "", err
elif normalized_scope == "workstream":
normalized_id, err = _resolve_workstream_memory_scope_id(request, normalized_id)
if err:
return "", "", err
err = _validate_scope_scope_id(
normalized_scope,
normalized_id,
require_scope_id=True,
)
return normalized_scope, normalized_id, err
def _rest_visible_memory_scopes(request: Request) -> list[tuple[str, str]]:
"""Default public read envelope: global plus the caller's user scope."""
scopes = [("global", "")]
uid = _auth_user_id(request)
if uid:
scopes.append(("user", uid))
return scopes
def _audit_rest_memory_mutation(
request: Request,
action: str,
row: dict[str, str],
) -> None:
"""Record one authenticated REST/SDK memory mutation."""
from turnstone.core.audit import record_audit
from turnstone.core.storage._registry import get_storage
uid, ip = _audit_context(request)
record_audit(
get_storage(),
uid,
action,
"memory",
row["memory_id"],
{
"name": row["name"],
"scope": row["scope"],
"scope_id": row["scope_id"],
"type": row["type"],
"surface": "rest",
},
ip,
)
async def list_memories(request: Request) -> JSONResponse:
"""GET /v1/api/memories — list memories with optional filters."""
from turnstone.core.memory import list_structured_memories
from turnstone.core.storage._registry import get_storage
mem_type = request.query_params.get("type", "")
mem_type = request.query_params.get("type", "").strip().lower()
scope = request.query_params.get("scope", "")
scope_id = request.query_params.get("scope_id", "")
if mem_type and mem_type not in _VALID_MEMORY_TYPES:
return JSONResponse({"error": f"invalid type: {mem_type}"}, status_code=400)
try:
limit = min(int(request.query_params.get("limit", "100")), 200)
limit = int(request.query_params.get("limit", "100"))
except (ValueError, TypeError):
return JSONResponse({"error": "limit must be an integer"}, status_code=400)
err = _validate_scope_scope_id(scope, scope_id)
if not 1 <= limit <= 200:
return JSONResponse({"error": "limit must be between 1 and 200"}, status_code=400)
scope, scope_id, err = _resolve_rest_memory_scope(
request,
scope,
scope_id,
allow_empty=True,
)
if err:
return err
if scope == "user":
scope_id, err = _resolve_user_scope_id(request, scope_id)
if err:
return err
rows = list_structured_memories(mem_type=mem_type, scope=scope, scope_id=scope_id, limit=limit)
try:
storage = get_storage()
if scope:
rows = storage.list_structured_memories(
mem_type=mem_type,
scope=scope,
scope_id=scope_id,
limit=limit,
)
else:
rows = storage.list_visible_structured_memories(
_rest_visible_memory_scopes(request),
mem_type=mem_type,
limit=limit,
)
except Exception:
log.warning("memory.rest_list_failed", exc_info=True)
return JSONResponse({"error": "Memory storage unavailable"}, status_code=500)
return JSONResponse({"memories": rows, "total": len(rows)})
async def save_memory(request: Request) -> JSONResponse:
"""POST /v1/api/memories — save (upsert) a structured memory."""
from turnstone.core.memory import save_structured_memory
from turnstone.core.memory import save_structured_memory_strict
from turnstone.core.web_helpers import read_json_or_400
body = await read_json_or_400(request)
@@ -3536,12 +3663,15 @@ async def save_memory(request: Request) -> JSONResponse:
{"error": f"content exceeds {_MAX_MEMORY_CONTENT} character limit"},
status_code=400,
)
# None (field omitted) means "leave unset": the upsert keeps the stored
# value on update and defaults on insert; an explicit value overwrites.
raw_desc = body.get("description")
description = None if raw_desc is None else str(raw_desc)
description = "" if raw_desc is None else str(raw_desc).strip()
if not description:
return JSONResponse(
{"error": "description is required and must be non-empty"},
status_code=400,
)
raw_type = body.get("type")
mem_type = None if raw_type is None else str(raw_type)
mem_type = None if raw_type is None else str(raw_type).strip().lower()
scope = str(body.get("scope", "global"))
scope_id = str(body.get("scope_id", ""))
if mem_type is not None and mem_type not in _VALID_MEMORY_TYPES:
@@ -3549,24 +3679,31 @@ async def save_memory(request: Request) -> JSONResponse:
{"error": f"invalid type: {mem_type}; must be one of {sorted(_VALID_MEMORY_TYPES)}"},
status_code=400,
)
if scope not in _VALID_MEMORY_SCOPES:
return JSONResponse(
{"error": f"invalid scope: {scope}; must be one of {sorted(_VALID_MEMORY_SCOPES)}"},
status_code=400,
)
if scope == "user":
scope_id, err = _resolve_user_scope_id(request, scope_id)
if err:
return err
err = _validate_scope_scope_id(scope, scope_id, require_scope_id=True)
scope, scope_id, err = _resolve_rest_memory_scope(
request,
scope,
scope_id,
allow_empty=False,
)
if err:
return err
# The upsert RETURNINGs the full saved row, so no follow-up read is needed.
row, was_update = save_structured_memory(
name, content, description=description, mem_type=mem_type, scope=scope, scope_id=scope_id
)
if not row:
try:
row, was_update = save_structured_memory_strict(
name,
content,
description=description,
mem_type=mem_type,
scope=scope,
scope_id=scope_id,
)
except Exception:
log.warning("memory.rest_save_failed", name=name, exc_info=True)
return JSONResponse({"error": "Failed to save memory"}, status_code=500)
_audit_rest_memory_mutation(
request,
"memory.update" if was_update else "memory.save",
row,
)
return JSONResponse(row, status_code=200 if was_update else 201)
@@ -3575,7 +3712,7 @@ async def search_memories(request: Request) -> JSONResponse:
Uses POST for the request body but requires only read scope (non-mutating).
"""
from turnstone.core.memory import search_structured_memories as search_fn
from turnstone.core.storage._registry import get_storage
from turnstone.core.web_helpers import read_json_or_400
body = await read_json_or_400(request)
@@ -3584,46 +3721,72 @@ async def search_memories(request: Request) -> JSONResponse:
query = str(body.get("query", "")).strip()
if not query:
return JSONResponse({"error": "query is required"}, status_code=400)
mem_type = str(body.get("type", ""))
mem_type = str(body.get("type", "")).strip().lower()
scope = str(body.get("scope", ""))
scope_id = str(body.get("scope_id", ""))
try:
limit = min(int(body.get("limit", 20)), 50)
limit = int(body.get("limit", 20))
except (ValueError, TypeError):
return JSONResponse({"error": "limit must be an integer"}, status_code=400)
err = _validate_scope_scope_id(scope, scope_id)
if mem_type and mem_type not in _VALID_MEMORY_TYPES:
return JSONResponse({"error": f"invalid type: {mem_type}"}, status_code=400)
if not 1 <= limit <= 50:
return JSONResponse({"error": "limit must be between 1 and 50"}, status_code=400)
scope, scope_id, err = _resolve_rest_memory_scope(
request,
scope,
scope_id,
allow_empty=True,
)
if err:
return err
if scope == "user":
scope_id, err = _resolve_user_scope_id(request, scope_id)
if err:
return err
rows = search_fn(query, mem_type=mem_type, scope=scope, scope_id=scope_id, limit=limit)
try:
storage = get_storage()
if scope:
rows = storage.search_structured_memories(
query,
mem_type=mem_type,
scope=scope,
scope_id=scope_id,
limit=limit,
)
else:
rows = storage.search_visible_structured_memories(
query,
_rest_visible_memory_scopes(request),
mem_type=mem_type,
limit=limit,
)
except Exception:
log.warning("memory.rest_search_failed", exc_info=True)
return JSONResponse({"error": "Memory storage unavailable"}, status_code=500)
return JSONResponse({"memories": rows, "total": len(rows)})
async def delete_memory_endpoint(request: Request) -> JSONResponse:
"""DELETE /v1/api/memories/{name} — delete a memory by name and scope."""
from turnstone.core.memory import delete_structured_memory, normalize_key
from turnstone.core.memory import delete_structured_memory_returning_strict, normalize_key
name = normalize_key(request.path_params["name"])
scope = request.query_params.get("scope", "global")
if scope not in _VALID_MEMORY_SCOPES:
return JSONResponse(
{"error": f"invalid scope: {scope}; must be one of {sorted(_VALID_MEMORY_SCOPES)}"},
status_code=400,
)
scope_id = request.query_params.get("scope_id", "")
if scope == "user":
scope_id, err = _resolve_user_scope_id(request, scope_id)
if err:
return err
err = _validate_scope_scope_id(scope, scope_id, require_scope_id=True)
scope, scope_id, err = _resolve_rest_memory_scope(
request,
scope,
scope_id,
allow_empty=False,
)
if err:
return err
if delete_structured_memory(name, scope, scope_id):
return JSONResponse({"status": "ok", "name": name})
return JSONResponse({"error": f"Memory '{name}' not found"}, status_code=404)
try:
deleted = delete_structured_memory_returning_strict(name, scope, scope_id)
except Exception:
log.warning("memory.rest_delete_failed", name=name, exc_info=True)
return JSONResponse({"error": "Failed to delete memory"}, status_code=500)
if deleted is None:
return JSONResponse({"error": f"Memory '{name}' not found"}, status_code=404)
_audit_rest_memory_mutation(request, "memory.delete", deleted)
return JSONResponse({"status": "ok", "name": name})
# ---------------------------------------------------------------------------
@@ -5639,8 +5802,16 @@ def create_app(
Route("/metrics", metrics_endpoint),
Route("/openapi.json", _openapi_handler),
Route("/docs", _docs_handler),
Mount("/static", app=StaticFiles(directory=str(_STATIC_DIR)), name="static"),
Mount("/shared", app=StaticFiles(directory=str(_SHARED_DIR)), name="shared"),
Mount(
"/static",
app=RevalidatingStaticFiles(directory=str(_STATIC_DIR)),
name="static",
),
Mount(
"/shared",
app=RevalidatingStaticFiles(directory=str(_SHARED_DIR)),
name="shared",
),
],
middleware=_build_middleware(cors_origins),
lifespan=_lifespan,
+20 -3
View File
@@ -12,7 +12,8 @@
* Event surface (caller-provided callbacks):
* onSend(text) Enter / Send-button required
* onStop() Stop button click required iff stopBtn=true
* onAttach(file) file picker change OR paste image OR drop
* onAttach(file) file picker change OR paste image/text OR drop;
* synchronous false preserves native text paste
*
* Imperative API on the returned instance:
* value current textarea value (getter / setter)
@@ -34,6 +35,8 @@
* whole pane) broader than the composer itself so users can drop
* anywhere onto the pane to attach.
*/
import { pasteTextToFile } from "./composer_paste_text.js";
var ATTACH_DEFAULT_ACCEPT =
"image/png,image/jpeg,image/gif,image/webp,application/pdf,text/*," +
"audio/wav,audio/mpeg,audio/ogg,audio/flac,audio/mp4,audio/aac,audio/webm," +
@@ -62,7 +65,7 @@ function makeButton(opts) {
* @param {Object} opts configuration:
* onSend: (text) => void
* onStop: () => void
* attachments: { onAttach: (file) => void, accept?: string } | null
* attachments: { onAttach: (file) => void|false, accept?: string } | null
* stopBtn: boolean (default false)
* queueWhileBusy: boolean (default false) busy state shows "Queue"
* instead of disabling the send button
@@ -484,7 +487,21 @@ Composer.prototype._wireEvents = function () {
}
}
}
if (uploaded > 0) e.preventDefault();
if (uploaded > 0) {
e.preventDefault();
return;
}
var text = e.clipboardData
? e.clipboardData.getData("text/plain")
: "";
var textFile = pasteTextToFile(text);
if (textFile) {
// A synchronous false lets pre-create staging reject safely (for
// example, its file-count cap) and preserve the native inline paste.
// Immediate-upload controllers return undefined and remain accepted.
var accepted = opts.attachments.onAttach(textFile);
if (accepted !== false) e.preventDefault();
}
});
}
@@ -23,9 +23,10 @@
* clearChips() drop all chips + map entries (no DELETE).
* rehydrate() pull the server-side pending list (page
* reload / tab switch).
* snapshot() {attachments, attachment_ids} of stable
* chips only (skips in-flight placeholders),
* ready to feed into a /send body.
* snapshot() {attachments, attachment_ids, uploading}; the
* arrays contain stable chips only, while the flag
* lets senders refuse before an in-flight
* placeholder is silently omitted.
* consume(attached_ids,
* dropped_ids?) strip chips for ids the server reserved;
* surface a toast if any were dropped.
@@ -327,6 +328,14 @@ export function createAttachmentController(opts) {
// user dismissed would attach an untracked element (not in the
// map, so coordSend wouldn't include it) and confuse them.
if (!pending.has(placeholderId)) return;
// Content-addressed uploads are idempotent. If this response resolves to
// an attachment already represented by another chip, discard only this
// upload's placeholder so identical pastes visibly deduplicate too.
if (pending.has(info.attachment_id)) {
_removeChipDom(placeholderId);
pending.delete(placeholderId);
return;
}
_replaceMapKey(pending, placeholderId, info.attachment_id, info);
var chip = _findChip(placeholderId);
@@ -454,13 +463,20 @@ export function createAttachmentController(opts) {
function snapshot() {
var attachments = [];
var ids = [];
var uploading = false;
pending.forEach(function (info, id) {
if (info && !info.uploading) {
attachments.push(info);
ids.push(id);
} else if (info && info.uploading) {
uploading = true;
}
});
return { attachments: attachments, attachment_ids: ids };
return {
attachments: attachments,
attachment_ids: ids,
uploading: uploading,
};
}
function consume(attachedIds, droppedIds) {
@@ -0,0 +1,59 @@
/* composer_paste_text.js shared large-paste attachment policy.
*
* DOM-free so the decision can be exercised directly under Node. Paste event
* handlers stay surface-owned: they must preserve each composer's existing
* file-first staging path and call preventDefault() only after this helper
* returns a File.
*/
export const PASTE_ATTACHMENT_CHARS = 2000;
// Keep the byte ceiling aligned with core/attachments.py:TEXT_DOC_SIZE_CAP.
export const TEXT_ATTACHMENT_MAX_BYTES = 512 * 1024;
const PASTED_TEXT_FILENAME = "pasted-text.txt";
const PASTED_TEXT_MIME = "text/plain";
const pastedTextSources = new WeakMap();
function hasMoreThanCharacters(text, thresholdChars) {
let count = 0;
// String iteration counts Unicode code points, matching Python's len()
// more closely than UTF-16 String.length (which counts emoji twice).
for (const _character of text) {
count += 1;
if (count > thresholdChars) return true;
}
return false;
}
export function pasteTextToFile(text) {
if (typeof text !== "string" || text.length === 0) return null;
if (!hasMoreThanCharacters(text, PASTE_ATTACHMENT_CHARS)) return null;
// Blob.size is the UTF-8 byte count browsers use for the synthesized File.
// Check it before the caller suppresses the native paste: over-cap text must
// remain inline instead of becoming a guaranteed 413 with no textarea copy.
if (new Blob([text]).size > TEXT_ATTACHMENT_MAX_BYTES) return null;
const file = new File([text], PASTED_TEXT_FILENAME, {
type: PASTED_TEXT_MIME,
});
pastedTextSources.set(file, text);
return file;
}
export function isDuplicatePastedTextFile(file, existingFiles) {
const source = pastedTextSources.get(file);
if (source === undefined) return false;
for (const existing of existingFiles || []) {
if (pastedTextSources.get(existing) === source) return true;
}
return false;
}
// Classic node/console bundles consume the same module at boot/event time.
if (typeof window !== "undefined") {
window.TurnstonePasteText = {
isDuplicatePastedTextFile,
pasteTextToFile,
};
}
+38 -6
View File
@@ -507,6 +507,18 @@ export function parsePriority(text) {
return { displayText: text, priority: "notice" };
}
// Restore a server-refused send without overwriting text entered during the
// POST round-trip. The rejected message stays first (its original chronology).
// Settlement is one-shot, so never infer duplicate restoration from content:
// the user may independently type the same text while the request is pending.
export function mergeRejectedComposerText(rejectedText, currentText) {
var rejected = rejectedText == null ? "" : String(rejectedText);
var current = currentText == null ? "" : String(currentText);
if (!rejected) return current;
if (!current) return rejected;
return rejected + "\n" + current;
}
// Mint one opaque browser correlation token. This is deliberately not a
// delivery/idempotency key; the server may accept the same value on multiple
// distinct turns, whose event ids remain authoritative.
@@ -678,6 +690,8 @@ export function acceptUserTurnEvent(evt, host) {
// has since asserted it (see the panes' busySource stamp)
// paneIsBusy(): the pane's LIVE busy flag (not the send-time snapshot)
// — drives the missed-edge settle below
// restoreInput(): restore rejected companion text without overwriting input
// entered during the POST round-trip
// renderError(msg): pane error row
// consumeAttachments(attached_ids, dropped_ids): composer chip sync
//
@@ -693,11 +707,15 @@ export function acceptUserTurnEvent(evt, host) {
// post-bind settle promotes it (see the inline contract).
// queue_full — the send was NEVER accepted (interjection cap, deferred-
// list saturation, or drain-spawn failure): remove the optimistic
// bubble too — leaving it renders loss as delivery — and restore busy
// under the same guard (no worker and no drain may exist to ever emit
// a state event; leaving busy strands the composer in Stop mode).
// busy / attachments_busy / cross_user_interjection / unknown-ok —
// the panes' historical shapes, verbatim.
// bubble too — leaving it renders loss as delivery — restore the input,
// and restore busy under the same guard (no worker and no drain may exist
// to ever emit a state event; leaving busy strands the composer in Stop
// mode).
// attachments_busy / cross_user_interjection — remove the false sent bubble,
// restore the companion text, and preserve attachment chips. The former
// proves server busy; the latter can also be a retained-input refusal with
// no worker, so it clears only this send's still-optimistic busy stamp.
// busy / unknown-ok — historical behavior.
export function settleSendResponse(queue, data, ctx) {
// Normalize a null / non-object 2xx body once, here at the shared
// chokepoint, so neither pane's call site has to guard it (interactive
@@ -787,11 +805,19 @@ export function settleSendResponse(queue, data, ctx) {
ctx.optimisticEl.remove();
if (ctx.busyIsOptimistic()) ctx.setBusy(false);
}
if (typeof ctx.restoreInput === "function") ctx.restoreInput();
ctx.renderError("Message queue full. Please wait.");
return;
}
if (status === "attachments_busy") {
if (ctx.queuedEl) queue.remove(ctx.queuedEl);
if (ctx.optimisticEl && ctx.optimisticEl.isConnected)
ctx.optimisticEl.remove();
if (typeof ctx.restoreInput === "function") ctx.restoreInput();
// The server just proved a worker owns the slot. Replace this send's
// optimistic source stamp with a server stamp; an idle SSE that already
// arrived would have cleared it and therefore fails this guard.
if (ctx.busyIsOptimistic()) ctx.setBusy(true);
ctx.renderError(
"Attachments can't be sent while the assistant is working. " +
"Send a text-only message now, or wait and resend with attachments.",
@@ -800,12 +826,18 @@ export function settleSendResponse(queue, data, ctx) {
}
if (status === "cross_user_interjection") {
if (ctx.queuedEl) queue.remove(ctx.queuedEl);
if (ctx.optimisticEl && ctx.optimisticEl.isConnected)
ctx.optimisticEl.remove();
if (typeof ctx.restoreInput === "function") ctx.restoreInput();
// A 409 can also come from retained foreign queued input with no live
// worker, so it does not prove busy. Undo only this send's optimistic
// stamp; a newer SSE state remains authoritative in either direction.
if (ctx.busyIsOptimistic()) ctx.setBusy(false);
ctx.renderError(
data.error ||
"Another participant's turn is in progress. Wait for it to " +
"finish, then send your message.",
);
if (!ctx.isBusy) ctx.setBusy(false);
return;
}
// Unknown / "ok" status (e.g. the stale-busy race: the client
+35 -1
View File
@@ -52,6 +52,7 @@ import {
acceptUserTurnEvent,
clientSendMaySettleForViewer,
createQueueController,
mergeRejectedComposerText,
mintClientSendId,
parsePriority,
postAndSettleSend,
@@ -2887,6 +2888,12 @@ class Pane {
busyIsOptimistic: () =>
this.busy && this.busySource === "optimistic",
paneIsBusy: () => this.busy,
restoreInput: () => {
this.composer.value = mergeRejectedComposerText(
editText,
this.composer.value,
);
},
renderError: (message) => this.addErrorMessage(message),
consumeAttachments: () => {},
},
@@ -4708,7 +4715,12 @@ class Pane {
sendMessage() {
const text = this.inputEl.value.trim();
if (!text) return;
if (!text) {
if (!this.attachments.isEmpty()) {
this.addInfoMessage("Add a message to send with this attachment.");
}
return;
}
if (text.startsWith("/")) {
if (this.busy) {
@@ -4798,11 +4810,27 @@ class Pane {
return;
}
// Attachments cannot ride the live turn's text-only interjection queue.
// Refuse before the optimistic clear so the user's companion text and
// staged chips remain ready to send once the worker is idle.
if (this.busy && !this.attachments.isEmpty()) {
this.addInfoMessage(
"Attachments can't be sent while the assistant is working. Wait for it to finish, then send again.",
);
return;
}
const isBusy = this.busy;
let queuedEl = null;
let optimisticEl = null;
const clientSendId = mintClientSendId();
const snap = this.attachments.snapshot();
if (snap.uploading) {
this.addInfoMessage(
"Wait for attachments to finish uploading before sending.",
);
return;
}
// Display-only strip of the !!! prefix (the server re-parses it
// authoritatively); shared parse so the settle helper's retro-convert
@@ -4874,6 +4902,12 @@ class Pane {
setBusy: (b) => this.setBusy(b),
busyIsOptimistic: () => this.busy && this.busySource === "optimistic",
paneIsBusy: () => this.busy,
restoreInput: () => {
this.composer.value = mergeRejectedComposerText(
text,
this.composer.value,
);
},
renderError: (msg) => this.addErrorMessage(msg),
consumeAttachments: (attached, droppedIds) =>
this.attachments.consume(attached, droppedIds),
+9 -7
View File
@@ -1,6 +1,6 @@
{
"name": "memory",
"description": "Persistent memory across sessions. Actions: 'save' stores a memory, 'get' retrieves full content by name, 'search' finds memories by query, 'delete' removes a memory, 'list' shows all memories. Use 'get' to read full content — search/list truncate previews to 200 chars. Memories have a type (user/general/feedback/reference) and a scope.",
"description": "Persistent memory across sessions. Actions: 'save' stores a memory and always requires a non-empty description, 'get' retrieves full content by name, 'search' finds memories by query, 'delete' removes a memory, 'list' shows all memories. Use 'get' to read full content — search/list truncate previews to 200 chars. Memories have a type (user/general/feedback/reference) and a scope. Save/get/delete inherit one target: an attached active project, otherwise the session-kind default. A read-only project permits get but makes inherited save/delete fail; they never fall back to another scope. Pass the displayed scope explicitly when following a search/list result from another scope.",
"parameters": {
"type": "object",
"properties": {
@@ -11,6 +11,7 @@
},
"name": {
"type": "string",
"maxLength": 256,
"description": "Memory identifier (required for 'save', 'get', and 'delete'). Short snake_case key."
},
"content": {
@@ -19,7 +20,8 @@
},
"description": {
"type": "string",
"description": "Short description for relevance matching (recommended for 'save')."
"minLength": 1,
"description": "Required non-empty description for relevance matching on every 'save' (create or update)."
},
"type": {
"type": "string",
@@ -29,7 +31,7 @@
"scope": {
"type": "string",
"enum": ["global", "workstream", "user", "coordinator", "project"],
"description": "Memory scope. 'global' = shared across everything; 'workstream' = private to this workstream; 'user' = follows the user across workstreams; 'project' = the shared bucket of the project this session is attached to (available only when attached, writable only with project write access). Default: the attached project when you can write it, otherwise 'global'."
"description": "Memory scope. 'global' = shared across everything; 'workstream' = private to this workstream; 'user' = follows the acting user across workstreams; 'project' = the shared bucket of the active project this session is attached to. Save/get/delete without scope inherit exactly one target: project when attached, otherwise the session-kind default. A read-only project permits get but rejects save/delete without falling back. Search/list without scope use all visible scopes. A valid explicit scope selects exactly that scope."
},
"query": {
"type": "string",
@@ -46,20 +48,20 @@
"interactive": true,
"kind_variants": {
"interactive": {
"description": "Persistent memory across sessions. Actions: 'save' stores a memory, 'get' retrieves full content by name, 'search' finds memories by query, 'delete' removes a memory, 'list' shows all memories. Use 'get' to read full content — search/list truncate previews to 200 chars. Memories have a type (user/general/feedback/reference) and a scope: 'global' (shared everywhere), 'workstream' (this workstream only), 'user' (follows you across workstreams), and 'project' (the shared bucket of an attached project). When this workstream is attached to a project you can write, new memories default to the project; otherwise to 'global'.",
"description": "Persistent memory across sessions. Actions: 'save' stores a memory and always requires a non-empty description, 'get' retrieves full content by name, 'search' finds memories by query, 'delete' removes a memory, 'list' shows all memories. Use 'get' to read full content — search/list truncate previews to 200 chars. Memories have a type (user/general/feedback/reference) and a scope: 'global' (shared everywhere), 'workstream' (this workstream only), 'user' (follows the acting user across workstreams), and 'project' (the shared bucket of an attached project). Unscoped save/get/delete target project when attached, otherwise global. A read-only project permits get but rejects save/delete without falling back. Pass the displayed scope explicitly when following a search/list result from another scope.",
"parameter_overrides": {
"scope": {
"enum": ["global", "workstream", "user", "project"],
"description": "Memory scope. 'global' = shared across everything; 'workstream' = private to this workstream; 'user' = follows the user across workstreams; 'project' = the shared bucket of the project this workstream is attached to (available only when attached, and writable only with project write access). Default: the attached project when you can write it, otherwise 'global'."
"description": "Memory scope. 'global' = shared across everything; 'workstream' = private to this workstream; 'user' = follows the acting user across workstreams; 'project' = the shared bucket of the active project this workstream is attached to. Save/get/delete without scope target project when attached, otherwise global. A read-only project permits get but rejects save/delete without falling back. Search/list without scope use all visible scopes. A valid explicit scope selects exactly that scope."
}
}
},
"coordinator": {
"description": "Persistent orchestration memory shared by all of your user's coordinator sessions. Actions: 'save' stores a memory, 'get' retrieves full content by name, 'search' finds memories by query, 'delete' removes a memory, 'list' shows all memories. Use 'get' to read full content — search/list truncate previews to 200 chars. Memories have a type (user/general/feedback/reference). Coordinator memories survive across coordinator sessions — save orchestration knowledge worth keeping (recurring procedures, environment facts, lessons from past runs). They are NOT visible to child workstreams.",
"description": "Persistent orchestration memory for the acting user. Actions: 'save' stores a memory and always requires a non-empty description, 'get' retrieves full content by name, 'search' finds memories by query, 'delete' removes a memory, 'list' shows all memories. Use 'get' to read full content — search/list truncate previews to 200 chars. Coordinator memories survive across that user's coordinator sessions and are NOT visible to child workstreams. Unscoped save/get/delete target project when attached, otherwise coordinator. A read-only project permits get but rejects save/delete without falling back. Pass the displayed scope explicitly when following a search/list result from another scope.",
"parameter_overrides": {
"scope": {
"enum": ["coordinator", "project"],
"description": "'coordinator' = the per-user orchestration namespace, durable across coordinator sessions; 'project' = the shared bucket of the project this coordinator is attached to (available only when attached, writable only with project write access). Default: the attached project when you can write it, otherwise 'coordinator'."
"description": "'coordinator' = the acting user's private orchestration namespace, durable across their coordinator sessions; 'project' = the shared bucket of the active project this coordinator is attached to. Save/get/delete without scope target project when attached, otherwise coordinator. A read-only project permits get but rejects save/delete without falling back. Search/list without scope use all visible scopes. A valid explicit scope selects exactly that scope."
}
}
}
+77 -21
View File
@@ -329,6 +329,35 @@ function _isAttachmentAllowed(file) {
return false;
}
// File clipboard items preserve their existing priority. Only when none were
// staged do we consider synthesizing a text attachment through the shared
// module bridge. Returns true when the native paste was handled.
function _handleComposerPaste(event, addFiles) {
if (!event.clipboardData) return false;
const items = event.clipboardData.items || [];
const files = [];
for (let i = 0; i < items.length; i++) {
if (items[i].kind === "file") {
const file = items[i].getAsFile();
if (file) files.push(file);
}
}
if (files.length > 0) {
event.preventDefault();
addFiles(files);
return true;
}
const paste = window.TurnstonePasteText;
if (!paste || !paste.pasteTextToFile) return false;
const textFile = paste.pasteTextToFile(
event.clipboardData.getData("text/plain"),
);
if (!textFile || addFiles([textFile]) === false) return false;
event.preventDefault();
return true;
}
// In-dialog error strip (sh-alert). Empty message clears + hides; a set
// message also scrolls into view — the alert sits at the top of the
// scrollable body while the submit lives in the pinned foot.
@@ -344,13 +373,23 @@ function _newWsError(msg) {
}
function _newWsAddFiles(files) {
let handled = false;
for (let i = 0; i < files.length; i++) {
const f = files[i];
const paste = window.TurnstonePasteText;
if (
paste &&
paste.isDuplicatePastedTextFile &&
paste.isDuplicatePastedTextFile(f, _newWsStagedFiles)
) {
handled = true;
continue;
}
if (_newWsStagedFiles.length >= _NEW_WS_MAX_FILES) {
_newWsError(
"At most " + _NEW_WS_MAX_FILES + " attachments per workstream",
);
return;
return handled;
}
if (!_isAttachmentAllowed(f)) {
_newWsError(
@@ -358,18 +397,20 @@ function _newWsAddFiles(files) {
f.name +
" (allowed: png/jpeg/gif/webp images, text)",
);
return;
return handled;
}
const isImage = (f.type || "").indexOf("image/") === 0;
const cap = isImage ? _NEW_WS_IMAGE_CAP : _NEW_WS_TEXT_CAP;
if (f.size > cap) {
_newWsError(f.name + " exceeds the " + _formatAttachSize(cap) + " cap");
return;
return handled;
}
_newWsStagedFiles.push(f);
handled = true;
}
_newWsError("");
_newWsRenderChips();
return handled;
}
function newWorkstream() {
@@ -465,7 +506,14 @@ function showNewWsModal(forkFromWsId) {
document.getElementById("new-ws-name").value = "";
const initEl = document.getElementById("new-ws-initial-message");
if (initEl) initEl.value = "";
if (initEl) {
initEl.value = "";
initEl.onpaste = function (event) {
// Forks inherit history and intentionally have no attachment lane.
if (_forkFromWsId) return;
_handleComposerPaste(event, _newWsAddFiles);
};
}
_newWsError("");
// Reset attachment staging. Forks don't carry attachments —
@@ -827,6 +875,11 @@ function submitNewWs() {
const persona = personaEl ? personaEl.value : "";
const initEl = document.getElementById("new-ws-initial-message");
const initial_message = initEl ? initEl.value.trim() : "";
const staged = _forkFromWsId ? [] : _newWsStagedFiles.slice();
if (staged.length > 0 && !initial_message) {
_newWsError("Add a message to send with this attachment.");
return;
}
if (name) body.name = name;
// Forks DELIBERATELY inherit their source's model + judge: the selects are
// hidden for a fork (showNewWsModal) and never sent here — matching the
@@ -855,7 +908,6 @@ function submitNewWs() {
window.TurnstoneHatch.setBusy(dlg, true);
let fetchOpts;
const staged = _forkFromWsId ? [] : _newWsStagedFiles.slice();
if (staged.length > 0) {
const form = new FormData();
form.append("meta", JSON.stringify(body));
@@ -1567,13 +1619,23 @@ function _renderDashboardChips() {
}
function _addDashboardFiles(files) {
let handled = false;
for (let i = 0; i < files.length; i++) {
const f = files[i];
const paste = window.TurnstonePasteText;
if (
paste &&
paste.isDuplicatePastedTextFile &&
paste.isDuplicatePastedTextFile(f, _dashboardStagedFiles)
) {
handled = true;
continue;
}
if (_dashboardStagedFiles.length >= _DASH_MAX_FILES) {
_dashboardError(
"At most " + _DASH_MAX_FILES + " attachments per workstream",
);
return;
return handled;
}
// Drag-drop bypasses the <input accept="..."> filter, so re-check
// against the server's allowlist before the upload roundtrip.
@@ -1583,7 +1645,7 @@ function _addDashboardFiles(files) {
f.name +
" (allowed: png/jpeg/gif/webp images, text)",
);
return;
return handled;
}
const isImage = (f.type || "").indexOf("image/") === 0;
const cap = isImage ? _DASH_IMAGE_CAP : _DASH_TEXT_CAP;
@@ -1591,12 +1653,14 @@ function _addDashboardFiles(files) {
_dashboardError(
f.name + " exceeds the " + _formatAttachSize(cap) + " cap",
);
return;
return handled;
}
_dashboardStagedFiles.push(f);
handled = true;
}
_renderDashboardChips();
_refreshDashboardSubmitLabel();
return handled;
}
let _dashboardErrorTimer = null;
@@ -1772,6 +1836,10 @@ function dashboardSubmit() {
const btn = document.getElementById("dashboard-submit-btn");
const text = input.value.trim();
const staged = _dashboardStagedFiles.slice();
if (staged.length > 0 && !text) {
_dashboardError("Add a message to send with this attachment.");
return;
}
const body = {};
const model = document.getElementById("dashboard-model").value.trim();
@@ -2131,19 +2199,7 @@ function _announce(text) {
});
input.addEventListener("input", _refreshDashboardSubmitLabel);
input.addEventListener("paste", function (e) {
if (!e.clipboardData) return;
const items = e.clipboardData.items || [];
const pasted = [];
for (let i = 0; i < items.length; i++) {
if (items[i].kind === "file") {
const f = items[i].getAsFile();
if (f) pasted.push(f);
}
}
if (pasted.length) {
e.preventDefault();
_addDashboardFiles(pasted);
}
_handleComposerPaste(e, _addDashboardFiles);
});
if (attachBtn && attachInput) {
+1
View File
@@ -664,6 +664,7 @@
<script type="module" src="/shared/models.js"></script>
<script type="module" src="/shared/skills.js"></script>
<script type="module" src="/shared/project_creator.js"></script>
<script type="module" src="/shared/composer_paste_text.js"></script>
<script type="module" src="/shared/composer.js"></script>
<script type="module" src="/shared/composer_attachments.js"></script>
<script type="module" src="/shared/composer_queue.js"></script>
Generated
+57 -6
View File
@@ -9,7 +9,8 @@ resolution-markers = [
"python_full_version == '3.13.*' and sys_platform == 'win32'",
"python_full_version < '3.13' and sys_platform == 'win32'",
"python_full_version == '3.13.*' and sys_platform != 'win32'",
"python_full_version < '3.13' and sys_platform != 'win32'",
"python_full_version == '3.12.*' and sys_platform == 'emscripten'",
"(python_full_version < '3.12' and sys_platform == 'emscripten') or (python_full_version < '3.13' and sys_platform != 'emscripten' and sys_platform != 'win32')",
]
[[package]]
@@ -972,6 +973,19 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/7e/f5/f66802a942d491edb555dd61e3a9961140fd64c90bce1eafd741609d334d/httpcore-1.0.9-py3-none-any.whl", hash = "sha256:2d400746a40668fc9dec9810239072b40b4484b640a8c38fd654a024c7a1bf55", size = 78784, upload-time = "2025-04-24T22:06:20.566Z" },
]
[[package]]
name = "httpcore2"
version = "2.10.0"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "h11", marker = "python_full_version != '3.12.*' or sys_platform != 'emscripten'" },
{ name = "truststore", marker = "python_full_version != '3.12.*' or sys_platform != 'emscripten'" },
]
sdist = { url = "https://files.pythonhosted.org/packages/a9/83/a896fc59940fc5a6e2aff3a4be1d92fa890112936803b331cae75a993c34/httpcore2-2.10.0.tar.gz", hash = "sha256:13c0cc3d1919d4f28457f60cd2c2abe04113a8af184ccf1142811beba936f9dc", size = 67427, upload-time = "2026-08-09T09:11:32.123Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/e5/4f/d149104195a35e2853a2fc203a8e3477747e58c80e17dda686dace174383/httpcore2-2.10.0-py3-none-any.whl", hash = "sha256:7df06cfb34070cae4f7c89be69dc1095eca138e9704ceffb98d25c1912ab6f01", size = 83000, upload-time = "2026-08-09T09:11:29.555Z" },
]
[[package]]
name = "httpx"
version = "0.28.1"
@@ -996,6 +1010,32 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/d2/fd/6668e5aec43ab844de6fc74927e155a3b37bf40d7c3790e49fc0406b6578/httpx_sse-0.4.3-py3-none-any.whl", hash = "sha256:0ac1c9fe3c0afad2e0ebb25a934a59f4c7823b60792691f779fad2c5568830fc", size = 8960, upload-time = "2025-10-10T21:48:21.158Z" },
]
[[package]]
name = "httpx2"
version = "2.10.0"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "anyio", marker = "sys_platform != 'emscripten'" },
{ name = "httpcore2", marker = "sys_platform != 'emscripten'" },
{ name = "httpx2-jsfetch", marker = "python_full_version >= '3.12' and sys_platform == 'emscripten'" },
{ name = "idna" },
{ name = "truststore", marker = "sys_platform != 'emscripten'" },
{ name = "typing-extensions", marker = "python_full_version < '3.13'" },
]
sdist = { url = "https://files.pythonhosted.org/packages/bd/3d/f9a8c07a3884f3e5b26205e8436a18b3af61c5d53192c3bea235574dbbec/httpx2-2.10.0.tar.gz", hash = "sha256:8741d7329fe2c7885fc9ceb61c8217acfb87a85f75723714b89ebf7ad7196338", size = 98749, upload-time = "2026-08-09T09:11:33.24Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/b9/6d/a637d52449d98a6892d9a4dc0262587afdb6a66f201871842dce5a97b1c1/httpx2-2.10.0-py3-none-any.whl", hash = "sha256:5e3194a432701e1cc6f69a8b1b2fa199ef907013fede8d9a09a2c5b7b8141a18", size = 94355, upload-time = "2026-08-09T09:11:30.882Z" },
]
[[package]]
name = "httpx2-jsfetch"
version = "1.0"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/cd/c4/0e5636363151a2a1795e0a77617168b9ca438e1748ec05fc9b5687f93d64/httpx2_jsfetch-1.0.tar.gz", hash = "sha256:70a0e3eabfef7cce5ad9c629f7d01ca05e418f586646f4ddf14782e4c1454c60", size = 6872, upload-time = "2026-08-07T00:13:07.492Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/9b/43/832f631d32e4f1211caa2ba368317739fe71f0b8530e4c9d15dc454bac2a/httpx2_jsfetch-1.0-py3-none-any.whl", hash = "sha256:cb916b707601e69a07721aabc8f3f6659be3a6893bc1ff5c6f9e02241df2da32", size = 6382, upload-time = "2026-08-07T00:13:06.567Z" },
]
[[package]]
name = "idna"
version = "3.18"
@@ -1568,21 +1608,21 @@ wheels = [
[[package]]
name = "openai"
version = "2.53.0"
version = "3.0.0"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "anyio" },
{ name = "distro" },
{ name = "httpx" },
{ name = "httpx2" },
{ name = "jiter" },
{ name = "pydantic" },
{ name = "sniffio" },
{ name = "tqdm" },
{ name = "typing-extensions" },
]
sdist = { url = "https://files.pythonhosted.org/packages/ef/cf/36e3e7235fdf6d125c052acc0970924611b17a20a4fe580596faf4566a65/openai-2.53.0.tar.gz", hash = "sha256:baf5802ad08980e1d9d561e1b996e800c8bcd14af5847c6d0e7a5cc59e4d4116", size = 1099435, upload-time = "2026-08-03T21:42:01.664Z" }
sdist = { url = "https://files.pythonhosted.org/packages/54/8c/2f500e8be09d1ae98c530467962535198b02cd4550cd418bbbaedc8b2910/openai-3.0.0.tar.gz", hash = "sha256:ffd00ef1678d70957e1f1ed98d5bfcf1d661f41ea4482f22e7d0144a66435a49", size = 1123740, upload-time = "2026-08-12T01:55:50.849Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/78/0f/cc6afea3542a5142c5d8fc8211c5e059a8375105d004a41dfa2c7948dbb0/openai-2.53.0-py3-none-any.whl", hash = "sha256:c694ffc747a3c4d1663ef2b07b811315a476164ee5efa3a993967349ebca7618", size = 1659829, upload-time = "2026-08-03T21:41:59.581Z" },
{ url = "https://files.pythonhosted.org/packages/7b/0d/9850e7eddb5e66da4439ed503e78e09ad1fd0195e6df51e4236c75763581/openai-3.0.0-py3-none-any.whl", hash = "sha256:8d32ac3a6647a66910d6cb8a64f0fa5a6c823604b6e82db83d9d055c6709bd51", size = 1665775, upload-time = "2026-08-12T01:55:48.678Z" },
]
[[package]]
@@ -2568,6 +2608,15 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/f9/1c/01bfd571a64e7f270e6bab5e33777debe0edc56759233ce84f27dec92d14/tqdm-4.70.0-py3-none-any.whl", hash = "sha256:7f585706bfddbdebf89daac705b2dfcc16890130727d3197ca62c732b4310953", size = 80184, upload-time = "2026-07-27T11:33:13.167Z" },
]
[[package]]
name = "truststore"
version = "0.10.4"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/53/a3/1585216310e344e8102c22482f6060c7a6ea0322b63e026372e6dcefcfd6/truststore-0.10.4.tar.gz", hash = "sha256:9d91bd436463ad5e4ee4aba766628dd6cd7010cf3e2461756b3303710eebc301", size = 26169, upload-time = "2025-08-12T18:49:02.73Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/19/97/56608b2249fe206a67cd573bc93cd9896e1efb9e98bce9c163bcdc704b88/truststore-0.10.4-py3-none-any.whl", hash = "sha256:adaeaecf1cbb5f4de3b1959b42d41f6fab57b2b1666adb59e89cb0b53361d981", size = 18660, upload-time = "2025-08-12T18:49:01.46Z" },
]
[[package]]
name = "turnstone"
version = "1.8.0a7"
@@ -2581,6 +2630,7 @@ dependencies = [
{ name = "cryptography" },
{ name = "httpx" },
{ name = "httpx-sse" },
{ name = "httpx2" },
{ name = "lacme" },
{ name = "mcp" },
{ name = "openai" },
@@ -2636,10 +2686,11 @@ requires-dist = [
{ name = "discord-py", marker = "extra == 'discord'", specifier = ">=2.4" },
{ name = "httpx", specifier = ">=0.28" },
{ name = "httpx-sse", specifier = ">=0.4" },
{ name = "httpx2", specifier = ">=2.7,<3" },
{ name = "lacme", specifier = ">=1.0.5" },
{ name = "mcp", specifier = ">=1.27,<2" },
{ name = "mypy", marker = "extra == 'dev'", specifier = ">=1.14" },
{ name = "openai", specifier = ">=2.45" },
{ name = "openai", specifier = ">=3,<4" },
{ name = "pillow", specifier = ">=10" },
{ name = "psycopg", extras = ["binary"], specifier = ">=3.2" },
{ name = "pydantic", specifier = ">=2.0" },