feat: [memory] REST API endpoints + SDK methods + docs (#56)

* feat: [memory] REST API endpoints + SDK methods + docs

Server API (4 endpoints):
- GET /v1/api/memories — list with type/scope/scope_id/limit filters
- POST /v1/api/memories — save (upsert) with validation
- POST /v1/api/memories/search — search by query (read scope)
- DELETE /v1/api/memories/{name} — delete by name+scope

Console admin API (4 endpoints):
- GET /v1/api/admin/memories — list all memories
- GET /v1/api/admin/memories/search — search with ?q= param
- GET /v1/api/admin/memories/{memory_id} — get by ID
- DELETE /v1/api/admin/memories/{memory_id} — delete by ID with audit

Storage: add delete_structured_memory_by_id, add mem_type filter to
count_structured_memories. Auth: memory DELETE requires write scope,
admin.memories permission added to valid set + builtin-admin role.

Python SDK: list_memories, save_memory, search_memories, delete_memory
on both server (async+sync) and console (async+sync) clients.

TypeScript SDK: matching methods + types on both clients.

Pydantic schemas with Literal type/scope validation, OpenAPI endpoint
specs on both servers. 33 endpoint tests + 8 auth scope tests.

Docs: docs/memory.md feature guide, api-reference.md endpoint docs,
23-memory-architecture.puml diagram.

Also fixes stray `total: int` on CreateChannelUserRequest.

* fix: [memory] address PR review — cross-user scope, schema types, snapshots

Security: user-scoped memory endpoints now bind scope_id to the
authenticated user's identity.  Providing a mismatched scope_id
returns 403, preventing cross-user memory access on all 4 server
endpoints.

Schema: MemoryInfo response uses MemoryType/MemoryScope Literals.
SearchMemoriesRequest uses filter Literals (empty string allowed).
Limit query params declare schema_type="integer" for correct OpenAPI.

Regenerate sdk/typescript/openapi-{server,console}.json snapshots.
Update count_structured_memories docstring for mem_type param.
Fix fallback response to use normalized name after save.

6 new security tests for user-scope access control.
This commit is contained in:
Patrick Buckley
2026-03-14 02:28:47 -07:00
committed by GitHub
parent 2888e8ce0a
commit 67f43a7ee0
27 changed files with 7049 additions and 105 deletions
+264
View File
@@ -940,6 +940,270 @@ Status code: `403`
---
### `GET /v1/api/memories`
List structured memories with optional filters. Requires `read` scope.
**Query parameters:**
| Parameter | Type | Required | Default | Description |
|------------|--------|----------|---------|------------------------------|
| `type` | string | no | `""` | Filter by memory type (user, project, feedback, reference) |
| `scope` | string | no | `""` | Filter by scope (global, workstream, user) |
| `scope_id` | string | no | `""` | Scope qualifier. Auto-resolved for `scope=user` when auth is active. |
| `limit` | int | no | `100` | Max results (capped at 200) |
**Response:**
```json
{
"memories": [
{
"memory_id": "a1b2c3d4-e5f6-...",
"name": "project_architecture",
"description": "Core architecture patterns",
"type": "project",
"scope": "global",
"scope_id": "",
"content": "The project uses a hexagonal architecture...",
"created": "2026-03-10T10:00:00",
"updated": "2026-03-12T14:30:00"
}
],
"total": 1
}
```
---
### `POST /v1/api/memories`
Save or upsert a structured memory. Requires `write` scope. Returns `201` on
create, `200` on update.
**Request body:**
```json
{
"name": "deployment_process",
"content": "Deploy via GitHub Actions. Staging auto-deploys on push to main.",
"description": "CI/CD deployment workflow",
"type": "project",
"scope": "global",
"scope_id": ""
}
```
| Field | Type | Required | Default | Description |
|--------------|--------|----------|-------------|--------------------------------------|
| `name` | string | yes | -- | Memory name (max 256 chars) |
| `content` | string | yes | -- | Memory content (max 65536 chars) |
| `description`| string | no | `""` | Short description for search ranking |
| `type` | string | no | `"project"` | One of: user, project, feedback, reference |
| `scope` | string | no | `"global"` | One of: global, workstream, user |
| `scope_id` | string | no | `""` | Scope qualifier (auto-resolved for user scope) |
**Response (created):** `201`
```json
{
"memory_id": "a1b2c3d4-e5f6-...",
"name": "deployment_process",
"description": "CI/CD deployment workflow",
"type": "project",
"scope": "global",
"scope_id": "",
"content": "Deploy via GitHub Actions...",
"created": "2026-03-14T10:00:00",
"updated": "2026-03-14T10:00:00"
}
```
**Error responses:**
| Status | Condition |
|--------|--------------------------------------------------------|
| 400 | Missing name, empty content, invalid type/scope, name too long, content too long |
---
### `POST /v1/api/memories/search`
Search memories by query. Uses POST for the request body but is non-mutating
(requires only `read` scope).
**Request body:**
```json
{
"query": "authentication",
"type": "project",
"scope": "",
"limit": 20
}
```
| Field | Type | Required | Default | Description |
|------------|--------|----------|---------|--------------------------------|
| `query` | string | yes | -- | Search query |
| `type` | string | no | `""` | Filter by type |
| `scope` | string | no | `""` | Filter by scope |
| `scope_id` | string | no | `""` | Filter by scope ID |
| `limit` | int | no | `20` | Max results (capped at 50) |
**Response:**
```json
{
"memories": [
{
"memory_id": "a1b2c3d4-e5f6-...",
"name": "auth_patterns",
"description": "Authentication architecture",
"type": "project",
"scope": "global",
"scope_id": "",
"content": "JWT tokens with HS256...",
"created": "2026-03-10T10:00:00",
"updated": "2026-03-12T14:30:00"
}
],
"total": 1
}
```
**Error:** `400` with `{"error": "query is required"}` if `query` is empty.
---
### `DELETE /v1/api/memories/{name}`
Delete a memory by name and scope. Requires `write` scope.
**Path parameters:**
| Parameter | Type | Description |
|-----------|--------|----------------------|
| `name` | string | Memory name |
**Query parameters:**
| Parameter | Type | Required | Default | Description |
|------------|--------|----------|------------|---------------------|
| `scope` | string | no | `"global"` | Scope of the memory |
| `scope_id` | string | no | `""` | Scope qualifier |
**Response (success):** `200`
```json
{"status": "ok", "name": "deployment_process"}
```
**Error (not found):** `404`
```json
{"error": "Memory 'deployment_process' not found"}
```
---
### `GET /v1/api/admin/memories` (Console)
List structured memories across all scopes. Requires `admin.memories`
permission.
**Query parameters:**
| Parameter | Type | Required | Default | Description |
|------------|--------|----------|---------|------------------------------|
| `type` | string | no | `""` | Filter by type |
| `scope` | string | no | `""` | Filter by scope |
| `scope_id` | string | no | `""` | Filter by scope ID |
| `limit` | int | no | `100` | Max results (capped at 200) |
**Response:** `200` -- same schema as `GET /v1/api/memories`.
---
### `GET /v1/api/admin/memories/search` (Console)
Search memories by query. Requires `admin.memories` permission.
**Query parameters:**
| Parameter | Type | Required | Default | Description |
|------------|--------|----------|---------|-------------------------------|
| `q` | string | yes | -- | Search query |
| `type` | string | no | `""` | Filter by type |
| `scope` | string | no | `""` | Filter by scope |
| `scope_id` | string | no | `""` | Filter by scope ID |
| `limit` | int | no | `20` | Max results (capped at 50) |
**Response:** `200` -- same schema as `GET /v1/api/memories`.
**Error:** `400` with `{"error": "q is required"}` if `q` is empty.
---
### `GET /v1/api/admin/memories/{memory_id}` (Console)
Get a single memory by ID. Requires `admin.memories` permission.
**Path parameters:**
| Parameter | Type | Description |
|-------------|--------|------------------------|
| `memory_id` | string | Memory UUID |
**Response (success):** `200`
```json
{
"memory_id": "a1b2c3d4-e5f6-...",
"name": "project_architecture",
"description": "Core architecture patterns",
"type": "project",
"scope": "global",
"scope_id": "",
"content": "The project uses...",
"created": "2026-03-10T10:00:00",
"updated": "2026-03-12T14:30:00"
}
```
**Error (not found):** `404`
```json
{"error": "Memory not found"}
```
---
### `DELETE /v1/api/admin/memories/{memory_id}` (Console)
Delete a memory by ID. Records an audit event (`memory.delete`). Requires
`admin.memories` permission.
**Path parameters:**
| Parameter | Type | Description |
|-------------|--------|------------------------|
| `memory_id` | string | Memory UUID |
**Response (success):** `200`
```json
{"status": "ok"}
```
**Error (not found):** `404`
```json
{"error": "Memory not found"}
```
---
### `GET /v1/api/admin/verdicts` (Console)
List intent validation verdicts from the `intent_verdicts` table. This endpoint
+1 -1
View File
@@ -47,7 +47,7 @@ turnstone/
watch.py WatchRunner daemon — periodic command polling, condition DSL, result dispatch
judge.py Intent validation — heuristic rules + LLM judge, advisory verdicts
model_registry.py ModelRegistry — named model configs, lazy client creation, fallback routing
memory.py Persistence facade (delegates to storage backend)
memory.py Persistence facade + structured memory API (delegates to storage backend)
storage/ Pluggable storage: StorageBackend protocol, SQLite + PostgreSQL
metrics.py Prometheus-compatible metrics collector (MetricsCollector)
healthcheck.py BackendHealthMonitor — periodic probe + circuit breaker
+159
View File
@@ -0,0 +1,159 @@
@startuml
!theme plain
title Turnstone — Structured Memory Architecture
skinparam participant {
BackgroundColor<<session>> #C8E6C9
BackgroundColor<<facade>> #FFE0B2
BackgroundColor<<storage>> #B3E5FC
BackgroundColor<<api>> #E8EAF6
BackgroundColor<<sdk>> #F5F5F5
}
participant "ChatSession\n(session.py)" as Session <<session>>
participant "MemoryFacade\n(memory.py)" as Facade <<facade>>
participant "MemoryRelevance\n(memory_relevance.py)" as Relevance <<facade>>
participant "StorageBackend\n(SQLite)" as Storage <<storage>>
participant "Server API\n(server.py)" as API <<api>>
participant "Console Admin\n(console/server.py)" as Admin <<api>>
participant "SDK Client\n(sdk/)" as SDK <<sdk>>
== Phase 1: Tool Path (session.send) ==
Session -> Session : _prepare_tool_calls()\nparse memory(action=...)
note right
Tool schema: 4 actions
save, search, delete, list
Auto-approved (no approval needed)
end note
Session -> Session : _exec_memory(item)
alt action = save
Session -> Facade : save_structured_memory(\nname, content, description,\nmem_type, scope, scope_id)
Facade -> Facade : normalize_key(name)
Facade -> Storage : create_structured_memory()
alt unique constraint violation
Storage --> Facade : IntegrityError
Facade -> Storage : get_structured_memory_by_name()
Storage --> Facade : existing row
Facade -> Storage : update_structured_memory()
end
Storage --> Facade : memory_id
Facade --> Session : (memory_id, old_content)
Session -> Session : _init_system_messages()\nrefresh BM25 context
end
alt action = search
Session -> Facade : search_structured_memories(\nquery, mem_type, scope,\nscope_id, limit)
Facade -> Storage : search_structured_memories()
Storage --> Session : matched rows
end
alt action = delete
Session -> Facade : delete_structured_memory(\nname, scope, scope_id)
Facade -> Storage : delete_structured_memory()
Storage --> Session : bool (existed)
Session -> Session : _init_system_messages()\nrefresh BM25 context
end
== Phase 2: BM25 Relevance Injection ==
Session -> Session : _init_system_messages()\nevery conversation turn
Session -> Session : _get_visible_memories(\nlimit=fetch_limit)
note right
**Scope resolution:**
1. global scope (always)
2. workstream scope (ws_id)
3. user scope (user_id, if auth)
Combined and deduplicated.
end note
Session -> Facade : list_structured_memories()\nper scope
Facade -> Storage : list_structured_memories()
Storage --> Session : up to fetch_limit rows
Session -> Relevance : extract_recent_context(\nmessages, max_messages=3)
Relevance --> Session : user text context
Session -> Relevance : score_memories(\nmemories, context,\nk=relevance_k)
note right
**BM25 scoring:**
Index over name + description
+ content[:200] for each memory.
Returns top-k by relevance.
Empty query returns most recent k.
end note
Relevance --> Session : top-k memories
Session -> Relevance : build_memory_context(\nrelevant_memories)
note right
Formats as XML block:
<memories>
<memory name="..." type="..."
scope="..." description="...">
content (max 500 chars)
</memory>
</memories>
end note
Relevance --> Session : XML string
Session -> Session : inject into\nsystem message
== Phase 3: Server API Path ==
SDK -> API : GET /v1/api/memories\n?type=project&limit=20
API -> Facade : list_structured_memories()
Facade -> Storage : list_structured_memories()
Storage --> API : rows
API --> SDK : {"memories": [...], "total": N}
SDK -> API : POST /v1/api/memories\n{name, content, ...}
API -> API : validate type, scope,\nname length, content length
API -> Facade : save_structured_memory()
Facade -> Storage : create / update
Storage --> API : memory row
API --> SDK : 201 (created) / 200 (updated)
SDK -> API : POST /v1/api/memories/search\n{query, type, ...}
API -> Facade : search_structured_memories()
Facade -> Storage : search_structured_memories()
Storage --> API : matched rows
API --> SDK : {"memories": [...], "total": N}
SDK -> API : DELETE /v1/api/memories/{name}\n?scope=global
API -> Facade : delete_structured_memory()
Facade -> Storage : delete row
API --> SDK : {"status": "ok"}
== Phase 4: Console Admin Path ==
SDK -> Admin : GET /v1/api/admin/memories\n?type=&scope=&limit=
Admin -> Admin : require_permission(\n"admin.memories")
Admin -> Storage : list_structured_memories()
Storage --> Admin : rows
Admin --> SDK : {"memories": [...], "total": N}
SDK -> Admin : GET /v1/api/admin/memories/{id}
Admin -> Storage : get_structured_memory(id)
Storage --> Admin : memory row
Admin --> SDK : memory JSON
SDK -> Admin : DELETE /v1/api/admin/memories/{id}
Admin -> Storage : delete_structured_memory_by_id()
Admin -> Admin : record_audit(\n"memory.delete")
Admin --> SDK : {"status": "ok"}
== Configuration ==
note over Session, Relevance
**MemoryConfig** (from [memory] in config.toml):
relevance_k = 5 -- top-k memories per turn
fetch_limit = 50 -- max memories fetched for scoring
max_content = 32768 -- max content length per memory
nudge_cooldown = 300 -- seconds between metacognitive nudges
nudges = true -- enable/disable memory nudges
end note
@enduml
@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:c89628ed917dfd576c1af75c68fe5fed9beadaaee9dcea7aa7a1643867c4f1b9
size 344323
+569
View File
@@ -0,0 +1,569 @@
# Structured Memory
> See also: [Memory Architecture diagram](diagrams/png/23-memory-architecture.png)
The structured memory system gives the AI persistent, typed, scoped memories
that survive across sessions and workstreams. Memories are automatically
surfaced in the system message via BM25 relevance scoring, so the model has
contextual recall without explicit search.
## Overview
Each memory has three dimensions:
- **Type** -- categorizes the memory's purpose
- **Scope** -- controls visibility boundaries
- **Name** -- unique identifier within a scope (snake_case, normalized)
### Memory types
| Type | Purpose |
|-------------|------------------------------------------------------------|
| `user` | User preferences, conventions, working style |
| `project` | Project-specific knowledge, architecture, patterns |
| `feedback` | Corrections, lessons learned, things to avoid |
| `reference` | Reference material, documentation, specifications |
### Memory scopes
| Scope | Visibility |
|--------------|-----------------------------------------------------------|
| `global` | Visible to all workstreams and users |
| `workstream` | Visible only within the originating workstream |
| `user` | Follows the authenticated user across workstreams |
A memory's identity is the tuple `(name, scope, scope_id)`. Saving a memory
with the same identity upserts -- updating content while preserving the ID.
### BM25 relevance injection
On every conversation turn, the system:
1. Fetches up to `fetch_limit` memories visible in the current scope
2. Extracts context from the last 3 user messages
3. Scores memories against that context using a BM25 index
4. Injects the top `relevance_k` memories into the system message as
`<memories>` XML tags
5. Appends a hint telling the model how many memories are in scope
This means the model always has its most relevant memories available without
explicit recall -- but can still use `memory(action='search')` for deeper
lookup.
### Nudges
The metacognition layer can nudge the model to save memories at appropriate
moments (e.g., after a correction or when resuming a workstream). Nudges are
rate-limited by `nudge_cooldown` and can be disabled entirely.
---
## Configuration
### config.toml
```toml
[memory]
relevance_k = 5 # top-k memories injected per turn
fetch_limit = 50 # max memories fetched from storage for scoring
max_content = 32768 # max content length per memory (characters)
nudge_cooldown = 300 # minimum seconds between memory nudges
nudges = true # enable/disable metacognitive nudges
```
All fields are optional. Defaults are shown above.
---
## Tool Usage
The `memory` tool supports four actions:
### save
Store or update a memory.
```json
{
"action": "save",
"name": "project_architecture",
"content": "The project uses a hexagonal architecture with...",
"description": "Core architecture patterns",
"type": "project",
"scope": "global"
}
```
| Parameter | Required | Default | Description |
|---------------|----------|-------------|------------------------------------------|
| `name` | yes | -- | Snake_case identifier (max 256 chars) |
| `content` | yes | -- | Memory content (max `max_content` chars) |
| `description` | no | `""` | Short description for relevance matching |
| `type` | no | `"project"` | One of: user, project, feedback, reference |
| `scope` | no | `"global"` | One of: global, workstream, user |
### search
Find memories by query (BM25 full-text search).
```json
{
"action": "search",
"query": "authentication patterns",
"type": "project",
"limit": 10
}
```
| Parameter | Required | Default | Description |
|-----------|----------|---------|--------------------------------------|
| `query` | yes | -- | Search query |
| `type` | no | `""` | Filter by type |
| `scope` | no | `""` | Filter by scope |
| `limit` | no | `20` | Max results (capped at 50) |
### delete
Remove a memory by name.
```json
{
"action": "delete",
"name": "outdated_pattern",
"scope": "global"
}
```
| Parameter | Required | Default | Description |
|------------|----------|------------|--------------------------|
| `name` | yes | -- | Memory name to delete |
| `scope` | no | `"global"` | Scope of the memory |
### list
List all memories with optional filters.
```json
{
"action": "list",
"type": "feedback",
"limit": 50
}
```
| Parameter | Required | Default | Description |
|-----------|----------|---------|----------------------------|
| `type` | no | `""` | Filter by type |
| `scope` | no | `""` | Filter by scope |
| `limit` | no | `20` | Max results (capped at 50) |
---
## Server API
Four endpoints on the server for programmatic memory access.
### `GET /v1/api/memories`
List memories with optional filters.
**Query parameters:**
| Parameter | Type | Required | Default | Description |
|------------|--------|----------|---------|------------------------------|
| `type` | string | no | `""` | Filter by memory type |
| `scope` | string | no | `""` | Filter by scope |
| `scope_id` | string | no | `""` | Filter by scope ID |
| `limit` | int | no | `100` | Max results (capped at 200) |
When `scope=user` and `scope_id` is omitted, the authenticated user's ID is
used automatically.
**Response:** `200`
```json
{
"memories": [
{
"memory_id": "a1b2c3d4-e5f6-...",
"name": "project_architecture",
"description": "Core architecture patterns",
"type": "project",
"scope": "global",
"scope_id": "",
"content": "The project uses a hexagonal architecture...",
"created": "2026-03-10T10:00:00",
"updated": "2026-03-12T14:30:00"
}
],
"total": 1
}
```
---
### `POST /v1/api/memories`
Save or upsert a structured memory.
**Request body:**
```json
{
"name": "deployment_process",
"content": "Deploy via GitHub Actions. Staging auto-deploys on push to main.",
"description": "CI/CD deployment workflow",
"type": "project",
"scope": "global",
"scope_id": ""
}
```
| Field | Type | Required | Default | Description |
|--------------|--------|----------|-------------|--------------------------------------|
| `name` | string | yes | -- | Memory name (max 256 chars) |
| `content` | string | yes | -- | Memory content (max 65536 chars) |
| `description`| string | no | `""` | Short description for search ranking |
| `type` | string | no | `"project"` | One of: user, project, feedback, reference |
| `scope` | string | no | `"global"` | One of: global, workstream, user |
| `scope_id` | string | no | `""` | Scope qualifier (auto-resolved for user scope) |
**Response (created):** `201`
```json
{
"memory_id": "a1b2c3d4-e5f6-...",
"name": "deployment_process",
"description": "CI/CD deployment workflow",
"type": "project",
"scope": "global",
"scope_id": "",
"content": "Deploy via GitHub Actions...",
"created": "2026-03-14T10:00:00",
"updated": "2026-03-14T10:00:00"
}
```
**Response (updated):** `200` -- same schema, returned when a memory with the
same `(name, scope, scope_id)` already existed.
**Errors:**
| Status | Condition |
|--------|------------------------------------|
| 400 | Missing name, empty content, invalid type/scope, content too long |
---
### `POST /v1/api/memories/search`
Search memories by query. Uses POST for the request body but is non-mutating
(requires only `read` scope).
**Request body:**
```json
{
"query": "authentication",
"type": "project",
"scope": "",
"scope_id": "",
"limit": 20
}
```
| Field | Type | Required | Default | Description |
|------------|--------|----------|---------|--------------------------------|
| `query` | string | yes | -- | Search query |
| `type` | string | no | `""` | Filter by type |
| `scope` | string | no | `""` | Filter by scope |
| `scope_id` | string | no | `""` | Filter by scope ID |
| `limit` | int | no | `20` | Max results (capped at 50) |
**Response:** `200`
```json
{
"memories": [
{
"memory_id": "a1b2c3d4-e5f6-...",
"name": "auth_patterns",
"description": "Authentication architecture",
"type": "project",
"scope": "global",
"scope_id": "",
"content": "JWT tokens with HS256...",
"created": "2026-03-10T10:00:00",
"updated": "2026-03-12T14:30:00"
}
],
"total": 1
}
```
---
### `DELETE /v1/api/memories/{name}`
Delete a memory by name and scope.
**Path parameters:**
| Parameter | Type | Description |
|-----------|--------|----------------------|
| `name` | string | Memory name |
**Query parameters:**
| Parameter | Type | Required | Default | Description |
|------------|--------|----------|------------|---------------------|
| `scope` | string | no | `"global"` | Scope of the memory |
| `scope_id` | string | no | `""` | Scope qualifier |
**Response (success):** `200`
```json
{"status": "ok", "name": "deployment_process"}
```
**Response (not found):** `404`
```json
{"error": "Memory 'deployment_process' not found"}
```
---
## Console Admin API
Four admin endpoints for cross-workstream memory management. All require the
`admin.memories` permission.
### `GET /v1/api/admin/memories`
List memories across all scopes (no automatic scope resolution).
**Query parameters:**
| Parameter | Type | Required | Default | Description |
|------------|--------|----------|---------|------------------------------|
| `type` | string | no | `""` | Filter by type |
| `scope` | string | no | `""` | Filter by scope |
| `scope_id` | string | no | `""` | Filter by scope ID |
| `limit` | int | no | `100` | Max results (capped at 200) |
**Response:** `200`
```json
{
"memories": [
{
"memory_id": "a1b2c3d4-e5f6-...",
"name": "project_architecture",
"description": "Core architecture patterns",
"type": "project",
"scope": "global",
"scope_id": "",
"content": "The project uses...",
"created": "2026-03-10T10:00:00",
"updated": "2026-03-12T14:30:00"
}
],
"total": 1
}
```
---
### `GET /v1/api/admin/memories/search`
Search memories by query (uses query parameters, not POST body).
**Query parameters:**
| Parameter | Type | Required | Default | Description |
|------------|--------|----------|---------|-------------------------------|
| `q` | string | yes | -- | Search query |
| `type` | string | no | `""` | Filter by type |
| `scope` | string | no | `""` | Filter by scope |
| `scope_id` | string | no | `""` | Filter by scope ID |
| `limit` | int | no | `20` | Max results (capped at 50) |
**Response:** `200` -- same schema as `GET /v1/api/admin/memories`.
---
### `GET /v1/api/admin/memories/{memory_id}`
Get a single memory by ID.
**Path parameters:**
| Parameter | Type | Description |
|-------------|--------|------------------------|
| `memory_id` | string | Memory UUID |
**Response (success):** `200`
```json
{
"memory_id": "a1b2c3d4-e5f6-...",
"name": "project_architecture",
"description": "Core architecture patterns",
"type": "project",
"scope": "global",
"scope_id": "",
"content": "The project uses...",
"created": "2026-03-10T10:00:00",
"updated": "2026-03-12T14:30:00"
}
```
**Response (not found):** `404`
```json
{"error": "Memory not found"}
```
---
### `DELETE /v1/api/admin/memories/{memory_id}`
Delete a memory by ID. Records an audit event (`memory.delete`).
**Path parameters:**
| Parameter | Type | Description |
|-------------|--------|------------------------|
| `memory_id` | string | Memory UUID |
**Response (success):** `200`
```json
{"status": "ok"}
```
**Response (not found):** `404`
```json
{"error": "Memory not found"}
```
---
## SDK
### Python
The server SDK uses `mem_type` (not `type`) to avoid shadowing the Python
builtin.
```python
from turnstone.sdk import TurnstoneServer
with TurnstoneServer("http://localhost:8080", token="tok_xxx") as client:
# Save a memory
mem = client.save_memory(
"api_conventions",
"All endpoints use /v1/ prefix. JSON responses.",
description="API design patterns",
mem_type="project",
scope="global",
)
print(mem.memory_id)
# Search memories
results = client.search_memories("authentication", mem_type="project", limit=10)
for m in results.memories:
print(f"{m['name']}: {m['description']}")
# List memories
all_mems = client.list_memories(mem_type="feedback", limit=50)
# Delete a memory
client.delete_memory("api_conventions", scope="global")
```
Console admin SDK:
```python
from turnstone.sdk import TurnstoneConsole
with TurnstoneConsole("http://localhost:9090", token="tok_xxx") as admin:
# List all memories (admin view, no scope auto-resolution)
result = admin.list_memories(scope="global", limit=100)
# Search
result = admin.search_memories("architecture", mem_type="project")
# Get by ID
mem = admin.get_memory("a1b2c3d4-e5f6-...")
# Delete by ID
admin.delete_memory("a1b2c3d4-e5f6-...")
```
### TypeScript
```typescript
import { TurnstoneServer } from "@turnstone/sdk";
const client = new TurnstoneServer({
baseUrl: "http://localhost:8080",
token: "tok_xxx",
});
// Save a memory
const mem = await client.saveMemory({
name: "api_conventions",
content: "All endpoints use /v1/ prefix. JSON responses.",
description: "API design patterns",
type: "project",
scope: "global",
});
// Search memories
const results = await client.searchMemories({
query: "authentication",
type: "project",
limit: 10,
});
// List memories
const all = await client.listMemories({ type: "feedback", limit: 50 });
// Delete a memory
await client.deleteMemory("api_conventions", { scope: "global" });
```
Console admin SDK:
```typescript
import { TurnstoneConsole } from "@turnstone/sdk";
const admin = new TurnstoneConsole({
baseUrl: "http://localhost:9090",
token: "tok_xxx",
});
// List, search, get, delete by ID
const mems = await admin.listMemories({ scope: "global" });
const found = await admin.searchMemories({ q: "auth", limit: 20 });
const one = await admin.getMemory("a1b2c3d4-e5f6-...");
await admin.deleteMemory("a1b2c3d4-e5f6-...");
```
---
## Storage
Memories are stored in the `structured_memories` table (migration 013).
The unique constraint on `(name, scope, scope_id)` ensures upsert semantics.
The name is normalized on save: lowercased, hyphens and spaces replaced with
underscores.
## Architecture
See [Memory Architecture diagram](diagrams/png/23-memory-architecture.png) for
the full data flow covering the session tool path, API path, admin path, and
BM25 relevance injection.
File diff suppressed because it is too large Load Diff
+559 -61
View File
@@ -2,7 +2,7 @@
"openapi": "3.1.0",
"info": {
"title": "turnstone Server API",
"version": "0.4.2",
"version": "0.6.0",
"description": "Single-node workstream management, chat interaction, and real-time streaming."
},
"paths": {
@@ -10,7 +10,9 @@
"get": {
"summary": "List active workstreams",
"operationId": "v1_api_workstreams_get",
"tags": ["Workstreams"],
"tags": [
"Workstreams"
],
"responses": {
"200": {
"description": "Success",
@@ -29,7 +31,9 @@
"get": {
"summary": "Dashboard with workstream details and aggregates",
"operationId": "v1_api_dashboard_get",
"tags": ["Workstreams"],
"tags": [
"Workstreams"
],
"responses": {
"200": {
"description": "Success",
@@ -48,7 +52,9 @@
"post": {
"summary": "Create a new workstream",
"operationId": "v1_api_workstreams_new_post",
"tags": ["Workstreams"],
"tags": [
"Workstreams"
],
"requestBody": {
"required": true,
"content": {
@@ -87,7 +93,9 @@
"post": {
"summary": "Close a workstream",
"operationId": "v1_api_workstreams_close_post",
"tags": ["Workstreams"],
"tags": [
"Workstreams"
],
"requestBody": {
"required": true,
"content": {
@@ -126,7 +134,9 @@
"post": {
"summary": "Send a user message",
"operationId": "v1_api_send_post",
"tags": ["Chat"],
"tags": [
"Chat"
],
"requestBody": {
"required": true,
"content": {
@@ -175,7 +185,9 @@
"post": {
"summary": "Approve or deny a tool call",
"operationId": "v1_api_approve_post",
"tags": ["Chat"],
"tags": [
"Chat"
],
"requestBody": {
"required": true,
"content": {
@@ -214,7 +226,9 @@
"post": {
"summary": "Respond to a plan review",
"operationId": "v1_api_plan_post",
"tags": ["Chat"],
"tags": [
"Chat"
],
"requestBody": {
"required": true,
"content": {
@@ -253,7 +267,9 @@
"post": {
"summary": "Execute a slash command",
"operationId": "v1_api_command_post",
"tags": ["Chat"],
"tags": [
"Chat"
],
"requestBody": {
"required": true,
"content": {
@@ -302,7 +318,9 @@
"post": {
"summary": "Cancel the active generation in a workstream",
"operationId": "v1_api_cancel_post",
"tags": ["Chat"],
"tags": [
"Chat"
],
"requestBody": {
"required": true,
"content": {
@@ -351,7 +369,9 @@
"get": {
"summary": "Per-workstream SSE event stream",
"operationId": "v1_api_events_get",
"tags": ["Streaming"],
"tags": [
"Streaming"
],
"description": "Opens a Server-Sent Events stream scoped to a single workstream. Returns text/event-stream. See API reference for event types.",
"parameters": [
{
@@ -385,7 +405,9 @@
"get": {
"summary": "Global SSE event stream",
"operationId": "v1_api_events_global_get",
"tags": ["Streaming"],
"tags": [
"Streaming"
],
"description": "Global Server-Sent Events stream for state-change broadcasts across all workstreams. Returns text/event-stream.",
"responses": {
"200": {
@@ -398,7 +420,9 @@
"get": {
"summary": "List saved workstreams",
"operationId": "v1_api_workstreams_saved_get",
"tags": ["Workstreams"],
"tags": [
"Workstreams"
],
"responses": {
"200": {
"description": "Success",
@@ -417,7 +441,9 @@
"post": {
"summary": "Authenticate with a token",
"operationId": "v1_api_auth_login_post",
"tags": ["Auth"],
"tags": [
"Auth"
],
"requestBody": {
"required": true,
"content": {
@@ -456,7 +482,9 @@
"post": {
"summary": "Create first admin user",
"operationId": "v1_api_auth_setup_post",
"tags": ["Auth"],
"tags": [
"Auth"
],
"requestBody": {
"required": true,
"content": {
@@ -515,7 +543,9 @@
"get": {
"summary": "Return auth state",
"operationId": "v1_api_auth_status_get",
"tags": ["Auth"],
"tags": [
"Auth"
],
"responses": {
"200": {
"description": "Success",
@@ -534,7 +564,9 @@
"post": {
"summary": "Clear auth cookie",
"operationId": "v1_api_auth_logout_post",
"tags": ["Auth"],
"tags": [
"Auth"
],
"responses": {
"200": {
"description": "Success",
@@ -549,11 +581,202 @@
}
}
},
"/v1/api/memories": {
"get": {
"summary": "List structured memories",
"operationId": "v1_api_memories_get",
"tags": [
"Memories"
],
"parameters": [
{
"name": "type",
"in": "query",
"required": false,
"schema": {
"type": "string"
},
"description": "Filter by memory type"
},
{
"name": "scope",
"in": "query",
"required": false,
"schema": {
"type": "string"
},
"description": "Filter by scope"
},
{
"name": "scope_id",
"in": "query",
"required": false,
"schema": {
"type": "string"
},
"description": "Filter by scope identifier"
},
{
"name": "limit",
"in": "query",
"required": false,
"schema": {
"type": "integer",
"default": 100
},
"description": "Max results (default 100, max 200)"
}
],
"responses": {
"200": {
"description": "Success",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ListMemoriesResponse"
}
}
}
}
}
},
"post": {
"summary": "Save (upsert) a structured memory",
"operationId": "v1_api_memories_post",
"tags": [
"Memories"
],
"requestBody": {
"required": true,
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/SaveMemoryRequest"
}
}
}
},
"responses": {
"200": {
"description": "Success",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/MemoryInfo"
}
}
}
},
"400": {
"description": "Error 400",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ErrorResponse"
}
}
}
}
}
}
},
"/v1/api/memories/search": {
"post": {
"summary": "Search structured memories by query",
"operationId": "v1_api_memories_search_post",
"tags": [
"Memories"
],
"requestBody": {
"required": true,
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/SearchMemoriesRequest"
}
}
}
},
"responses": {
"200": {
"description": "Success",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ListMemoriesResponse"
}
}
}
}
}
}
},
"/v1/api/memories/{name}": {
"delete": {
"summary": "Delete a structured memory by name and scope",
"operationId": "v1_api_memories_{name}_delete",
"tags": [
"Memories"
],
"parameters": [
{
"name": "name",
"in": "path",
"required": true,
"schema": {
"type": "string"
}
},
{
"name": "scope",
"in": "query",
"required": false,
"schema": {
"type": "string"
},
"description": "Scope (default: global)"
},
{
"name": "scope_id",
"in": "query",
"required": false,
"schema": {
"type": "string"
},
"description": "Scope identifier"
}
],
"responses": {
"200": {
"description": "Success",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/StatusResponse"
}
}
}
},
"404": {
"description": "Error 404",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ErrorResponse"
}
}
}
}
}
}
},
"/health": {
"get": {
"summary": "Server health check",
"operationId": "health_get",
"tags": ["Observability"],
"tags": [
"Observability"
],
"responses": {
"200": {
"description": "Success",
@@ -580,7 +803,9 @@
"type": "string"
}
},
"required": ["error"],
"required": [
"error"
],
"title": "ErrorResponse",
"type": "object"
},
@@ -589,7 +814,9 @@
"properties": {
"status": {
"default": "ok",
"examples": ["ok"],
"examples": [
"ok"
],
"title": "Status",
"type": "string"
}
@@ -638,14 +865,19 @@
},
"role": {
"description": "Legacy role",
"examples": ["full", "read"],
"examples": [
"full",
"read"
],
"title": "Role",
"type": "string"
},
"scopes": {
"default": "",
"description": "Comma-separated scopes",
"examples": ["read,write,approve"],
"examples": [
"read,write,approve"
],
"title": "Scopes",
"type": "string"
},
@@ -656,7 +888,9 @@
"type": "string"
}
},
"required": ["role"],
"required": [
"role"
],
"title": "AuthLoginResponse",
"type": "object"
},
@@ -679,7 +913,11 @@
"type": "string"
}
},
"required": ["username", "display_name", "password"],
"required": [
"username",
"display_name",
"password"
],
"title": "AuthSetupRequest",
"type": "object"
},
@@ -716,7 +954,10 @@
"type": "string"
}
},
"required": ["user_id", "username"],
"required": [
"user_id",
"username"
],
"title": "AuthSetupResponse",
"type": "object"
},
@@ -736,7 +977,11 @@
"type": "boolean"
}
},
"required": ["auth_enabled", "has_users", "setup_required"],
"required": [
"auth_enabled",
"has_users",
"setup_required"
],
"title": "AuthStatusResponse",
"type": "object"
},
@@ -753,7 +998,10 @@
"type": "string"
}
},
"required": ["message", "ws_id"],
"required": [
"message",
"ws_id"
],
"title": "SendRequest",
"type": "object"
},
@@ -761,12 +1009,17 @@
"properties": {
"status": {
"description": "'ok' or 'busy'",
"examples": ["ok", "busy"],
"examples": [
"ok",
"busy"
],
"title": "Status",
"type": "string"
}
},
"required": ["status"],
"required": [
"status"
],
"title": "SendResponse",
"type": "object"
},
@@ -802,7 +1055,10 @@
"type": "string"
}
},
"required": ["approved", "ws_id"],
"required": [
"approved",
"ws_id"
],
"title": "ApproveRequest",
"type": "object"
},
@@ -819,7 +1075,10 @@
"type": "string"
}
},
"required": ["feedback", "ws_id"],
"required": [
"feedback",
"ws_id"
],
"title": "PlanFeedbackRequest",
"type": "object"
},
@@ -836,7 +1095,10 @@
"type": "string"
}
},
"required": ["command", "ws_id"],
"required": [
"command",
"ws_id"
],
"title": "CommandRequest",
"type": "object"
},
@@ -848,7 +1110,9 @@
"type": "string"
}
},
"required": ["ws_id"],
"required": [
"ws_id"
],
"title": "CancelRequest",
"type": "object"
},
@@ -886,7 +1150,7 @@
},
"ws_template": {
"default": "",
"description": "Workstream template name (behavioral profile applied at creation)",
"description": "Workstream template name to apply defaults from",
"title": "Ws Template",
"type": "string"
}
@@ -919,7 +1183,10 @@
"type": "integer"
}
},
"required": ["ws_id", "name"],
"required": [
"ws_id",
"name"
],
"title": "CreateWorkstreamResponse",
"type": "object"
},
@@ -931,7 +1198,9 @@
"type": "string"
}
},
"required": ["ws_id"],
"required": [
"ws_id"
],
"title": "CloseWorkstreamRequest",
"type": "object"
},
@@ -945,7 +1214,9 @@
"type": "array"
}
},
"required": ["workstreams"],
"required": [
"workstreams"
],
"title": "ListWorkstreamsResponse",
"type": "object"
},
@@ -964,7 +1235,11 @@
"type": "string"
}
},
"required": ["id", "name", "state"],
"required": [
"id",
"name",
"state"
],
"title": "WorkstreamInfo",
"type": "object"
},
@@ -981,7 +1256,10 @@
"$ref": "#/components/schemas/DashboardAggregate"
}
},
"required": ["workstreams", "aggregate"],
"required": [
"workstreams",
"aggregate"
],
"title": "DashboardResponse",
"type": "object"
},
@@ -1081,7 +1359,11 @@
"type": "string"
}
},
"required": ["id", "name", "state"],
"required": [
"id",
"name",
"state"
],
"title": "DashboardWorkstream",
"type": "object"
},
@@ -1095,7 +1377,9 @@
"type": "array"
}
},
"required": ["workstreams"],
"required": [
"workstreams"
],
"title": "ListSavedWorkstreamsResponse",
"type": "object"
},
@@ -1142,14 +1426,22 @@
"type": "integer"
}
},
"required": ["ws_id", "created", "updated", "message_count"],
"required": [
"ws_id",
"created",
"updated",
"message_count"
],
"title": "SavedWorkstreamInfo",
"type": "object"
},
"HealthResponse": {
"properties": {
"status": {
"examples": ["ok", "degraded"],
"examples": [
"ok",
"degraded"
],
"title": "Status",
"type": "string"
},
@@ -1202,10 +1494,39 @@
"default": null
}
},
"required": ["status"],
"required": [
"status"
],
"title": "HealthResponse",
"type": "object"
},
"BackendStatus": {
"properties": {
"status": {
"examples": [
"up",
"down"
],
"title": "Status",
"type": "string"
},
"circuit_state": {
"examples": [
"closed",
"open",
"half_open"
],
"title": "Circuit State",
"type": "string"
}
},
"required": [
"status",
"circuit_state"
],
"title": "BackendStatus",
"type": "object"
},
"McpStatus": {
"properties": {
"servers": {
@@ -1227,23 +1548,6 @@
"title": "McpStatus",
"type": "object"
},
"BackendStatus": {
"properties": {
"status": {
"examples": ["up", "down"],
"title": "Status",
"type": "string"
},
"circuit_state": {
"examples": ["closed", "open", "half_open"],
"title": "Circuit State",
"type": "string"
}
},
"required": ["status", "circuit_state"],
"title": "BackendStatus",
"type": "object"
},
"WorkstreamCounts": {
"properties": {
"total": {
@@ -1279,6 +1583,200 @@
},
"title": "WorkstreamCounts",
"type": "object"
},
"SaveMemoryRequest": {
"properties": {
"name": {
"description": "Memory identifier (normalized to snake_case)",
"title": "Name",
"type": "string"
},
"content": {
"description": "Memory content",
"maxLength": 65536,
"title": "Content",
"type": "string"
},
"description": {
"default": "",
"description": "Short description for relevance matching",
"title": "Description",
"type": "string"
},
"type": {
"default": "project",
"description": "Memory type",
"enum": [
"user",
"project",
"feedback",
"reference"
],
"title": "Type",
"type": "string"
},
"scope": {
"default": "global",
"description": "Memory scope",
"enum": [
"global",
"workstream",
"user"
],
"title": "Scope",
"type": "string"
},
"scope_id": {
"default": "",
"description": "Scope identifier (ws_id for workstream, user_id for user scope)",
"title": "Scope Id",
"type": "string"
}
},
"required": [
"name",
"content"
],
"title": "SaveMemoryRequest",
"type": "object"
},
"MemoryInfo": {
"properties": {
"memory_id": {
"title": "Memory Id",
"type": "string"
},
"name": {
"title": "Name",
"type": "string"
},
"description": {
"default": "",
"title": "Description",
"type": "string"
},
"type": {
"enum": [
"user",
"project",
"feedback",
"reference"
],
"title": "Type",
"type": "string"
},
"scope": {
"enum": [
"global",
"workstream",
"user"
],
"title": "Scope",
"type": "string"
},
"scope_id": {
"default": "",
"title": "Scope Id",
"type": "string"
},
"content": {
"title": "Content",
"type": "string"
},
"created": {
"title": "Created",
"type": "string"
},
"updated": {
"title": "Updated",
"type": "string"
}
},
"required": [
"memory_id",
"name",
"type",
"scope",
"content",
"created",
"updated"
],
"title": "MemoryInfo",
"type": "object"
},
"ListMemoriesResponse": {
"properties": {
"memories": {
"items": {
"$ref": "#/components/schemas/MemoryInfo"
},
"title": "Memories",
"type": "array"
},
"total": {
"default": 0,
"title": "Total",
"type": "integer"
}
},
"required": [
"memories"
],
"title": "ListMemoriesResponse",
"type": "object"
},
"SearchMemoriesRequest": {
"properties": {
"query": {
"description": "Search query text",
"title": "Query",
"type": "string"
},
"type": {
"default": "",
"description": "Filter by memory type",
"enum": [
"",
"user",
"project",
"feedback",
"reference"
],
"title": "Type",
"type": "string"
},
"scope": {
"default": "",
"description": "Filter by scope",
"enum": [
"",
"global",
"workstream",
"user"
],
"title": "Scope",
"type": "string"
},
"scope_id": {
"default": "",
"description": "Filter by scope_id",
"title": "Scope Id",
"type": "string"
},
"limit": {
"default": 20,
"description": "Max results (1-50)",
"maximum": 50,
"minimum": 1,
"title": "Limit",
"type": "integer"
}
},
"required": [
"query"
],
"title": "SearchMemoriesRequest",
"type": "object"
}
}
}
+36
View File
@@ -1,6 +1,9 @@
import { BaseClient, type ClientOptions } from "./base.js";
import type { ClusterEvent } from "./events.js";
import type {
AdminListMemoriesOptions,
AdminMemoryInfo,
AdminSearchMemoriesOptions,
AuditQueryOptions,
AuditResponse,
AuthLoginResponse,
@@ -18,6 +21,7 @@ import type {
CreateScheduleRequest,
CreateTemplateOptions,
CreateWsTemplateOptions,
ListAdminMemoriesResponse,
ListScheduleRunsResponse,
ListSchedulesResponse,
NodeDetailResponse,
@@ -343,4 +347,36 @@ export class TurnstoneConsole extends BaseClient {
if (opts?.offset !== undefined) params.offset = String(opts.offset);
return this.request("GET", "/v1/api/admin/audit", { params });
}
// -- Admin: Memories ------------------------------------------------------
async listMemories(
opts?: AdminListMemoriesOptions,
): Promise<ListAdminMemoriesResponse> {
const params: Record<string, string | number> = {};
if (opts?.type) params.type = opts.type;
if (opts?.scope) params.scope = opts.scope;
if (opts?.scope_id) params.scope_id = opts.scope_id;
if (opts?.limit !== undefined) params.limit = opts.limit;
return this.request("GET", "/v1/api/admin/memories", { params });
}
async searchMemories(
opts: AdminSearchMemoriesOptions,
): Promise<ListAdminMemoriesResponse> {
const params: Record<string, string | number> = { q: opts.q };
if (opts.type) params.type = opts.type;
if (opts.scope) params.scope = opts.scope;
if (opts.scope_id) params.scope_id = opts.scope_id;
if (opts.limit !== undefined) params.limit = opts.limit;
return this.request("GET", "/v1/api/admin/memories/search", { params });
}
async getMemory(memoryId: string): Promise<AdminMemoryInfo> {
return this.request("GET", `/v1/api/admin/memories/${memoryId}`);
}
async deleteMemory(memoryId: string): Promise<StatusResponse> {
return this.request("DELETE", `/v1/api/admin/memories/${memoryId}`);
}
}
+11
View File
@@ -143,6 +143,17 @@ export type {
SendAndWaitOptions,
NodesOptions,
WorkstreamsOptions,
// Memory types
SaveMemoryRequest,
MemoryInfo,
ListMemoriesResponse,
SearchMemoriesRequest,
ListMemoriesOptions,
DeleteMemoryOptions,
AdminMemoryInfo,
ListAdminMemoriesResponse,
AdminListMemoriesOptions,
AdminSearchMemoriesOptions,
} from "./types.js";
// SSE parser (for advanced usage)
+39
View File
@@ -7,9 +7,15 @@ import type {
CreateWorkstreamRequest,
CreateWorkstreamResponse,
DashboardResponse,
DeleteMemoryOptions,
HealthResponse,
ListMemoriesOptions,
ListMemoriesResponse,
ListSavedWorkstreamsResponse,
ListWorkstreamsResponse,
MemoryInfo,
SaveMemoryRequest,
SearchMemoriesRequest,
SendAndWaitOptions,
SendResponse,
StatusResponse,
@@ -190,6 +196,39 @@ export class TurnstoneServer extends BaseClient {
return this.request("GET", "/v1/api/workstreams/saved");
}
// -- Memories -------------------------------------------------------------
async listMemories(
opts?: ListMemoriesOptions,
): Promise<ListMemoriesResponse> {
const params: Record<string, string | number> = {};
if (opts?.type) params.type = opts.type;
if (opts?.scope) params.scope = opts.scope;
if (opts?.scope_id) params.scope_id = opts.scope_id;
if (opts?.limit !== undefined) params.limit = opts.limit;
return this.request("GET", "/v1/api/memories", { params });
}
async saveMemory(opts: SaveMemoryRequest): Promise<MemoryInfo> {
return this.request("POST", "/v1/api/memories", { json: opts });
}
async searchMemories(
opts: SearchMemoriesRequest,
): Promise<ListMemoriesResponse> {
return this.request("POST", "/v1/api/memories/search", { json: opts });
}
async deleteMemory(
name: string,
opts?: DeleteMemoryOptions,
): Promise<StatusResponse> {
const params: Record<string, string> = {};
if (opts?.scope) params.scope = opts.scope;
if (opts?.scope_id) params.scope_id = opts.scope_id;
return this.request("DELETE", `/v1/api/memories/${name}`, { params });
}
// -- Auth -----------------------------------------------------------------
async login(opts: {
+84
View File
@@ -644,5 +644,89 @@ export interface WorkstreamsOptions {
per_page?: number;
}
// -- Server API: Memories ---------------------------------------------------
export interface SaveMemoryRequest {
name: string;
content: string;
description?: string;
type?: "user" | "project" | "feedback" | "reference";
scope?: "global" | "workstream" | "user";
scope_id?: string;
}
export interface MemoryInfo {
memory_id: string;
name: string;
description: string;
type: string;
scope: string;
scope_id: string;
content: string;
created: string;
updated: string;
}
export interface ListMemoriesResponse {
memories: MemoryInfo[];
total: number;
}
export interface SearchMemoriesRequest {
query: string;
type?: string;
scope?: string;
scope_id?: string;
limit?: number;
}
export interface ListMemoriesOptions {
type?: string;
scope?: string;
scope_id?: string;
limit?: number;
}
export interface DeleteMemoryOptions {
scope?: string;
scope_id?: string;
}
// -- Console API: Admin Memories --------------------------------------------
export interface AdminMemoryInfo {
memory_id: string;
name: string;
description: string;
type: string;
scope: string;
scope_id: string;
content: string;
created: string;
updated: string;
last_accessed: string;
access_count: number;
}
export interface ListAdminMemoriesResponse {
memories: AdminMemoryInfo[];
total: number;
}
export interface AdminListMemoriesOptions {
type?: string;
scope?: string;
scope_id?: string;
limit?: number;
}
export interface AdminSearchMemoriesOptions {
q: string;
type?: string;
scope?: string;
scope_id?: string;
limit?: number;
}
// Re-export event types for convenience
export type { ServerEvent, ClusterEvent } from "./events.js";
+26
View File
@@ -143,6 +143,32 @@ class TestRequiredScope:
def test_proxy_v1_read_endpoint_needs_read(self):
assert required_scope("GET", "/node/node-a/v1/api/workstreams") == "read"
# Memory endpoints
def test_get_memories_needs_read(self):
assert required_scope("GET", "/api/memories") == "read"
def test_post_memories_needs_write(self):
assert required_scope("POST", "/api/memories") == "write"
def test_post_memories_search_needs_read(self):
"""Search via POST is non-mutating — requires only read scope."""
assert required_scope("POST", "/api/memories/search") == "read"
def test_delete_memory_needs_write(self):
assert required_scope("DELETE", "/api/memories/my_key") == "write"
def test_v1_post_memories_needs_write(self):
assert required_scope("POST", "/v1/api/memories") == "write"
def test_v1_delete_memory_needs_write(self):
assert required_scope("DELETE", "/v1/api/memories/test_key") == "write"
def test_admin_memories_needs_approve(self):
assert required_scope("GET", "/api/admin/memories") == "approve"
def test_admin_memory_delete_needs_approve(self):
assert required_scope("DELETE", "/api/admin/memories/some-id") == "approve"
# ---------------------------------------------------------------------------
# TestAuthConfig
+421
View File
@@ -0,0 +1,421 @@
"""Tests for memory API endpoints (server + console admin)."""
from __future__ import annotations
from typing import TYPE_CHECKING, Any
import pytest
from starlette.applications import Starlette
from starlette.middleware import Middleware
from starlette.middleware.base import BaseHTTPMiddleware
from starlette.routing import Mount, Route
from starlette.testclient import TestClient
if TYPE_CHECKING:
from starlette.requests import Request
from starlette.responses import Response
from turnstone.console.server import (
admin_delete_memory,
admin_get_memory,
admin_list_memories,
admin_search_memories,
)
from turnstone.core.auth import AuthResult
from turnstone.core.storage._sqlite import SQLiteBackend
from turnstone.server import (
delete_memory_endpoint,
list_memories,
save_memory,
search_memories,
)
# ---------------------------------------------------------------------------
# Auth bypass middleware
# ---------------------------------------------------------------------------
class _InjectAuthMiddleware(BaseHTTPMiddleware):
async def dispatch(self, request: Request, call_next: Any) -> Response:
request.state.auth_result = AuthResult(
user_id="test-user",
scopes=frozenset({"approve"}),
token_source="config",
permissions=frozenset(
{
"read",
"write",
"approve",
"admin.memories",
}
),
)
return await call_next(request)
# ---------------------------------------------------------------------------
# Fixtures
# ---------------------------------------------------------------------------
@pytest.fixture
def storage(tmp_path):
return SQLiteBackend(str(tmp_path / "test.db"))
@pytest.fixture
def server_client(storage):
"""TestClient wired to server memory endpoints."""
import turnstone.core.storage._registry as reg
old = reg._storage
reg._storage = storage
app = Starlette(
routes=[
Mount(
"/v1",
routes=[
Route("/api/memories", list_memories),
Route("/api/memories", save_memory, methods=["POST"]),
Route("/api/memories/search", search_memories, methods=["POST"]),
Route("/api/memories/{name}", delete_memory_endpoint, methods=["DELETE"]),
],
),
],
middleware=[Middleware(_InjectAuthMiddleware)],
)
yield TestClient(app)
reg._storage = old
@pytest.fixture
def admin_client(storage):
"""TestClient wired to console admin memory endpoints."""
app = Starlette(
routes=[
Mount(
"/v1",
routes=[
Route("/api/admin/memories", admin_list_memories),
Route("/api/admin/memories/search", admin_search_memories),
Route("/api/admin/memories/{memory_id}", admin_get_memory),
Route(
"/api/admin/memories/{memory_id}",
admin_delete_memory,
methods=["DELETE"],
),
],
),
],
middleware=[Middleware(_InjectAuthMiddleware)],
)
app.state.auth_storage = storage
return TestClient(app)
def _seed_memory(storage, name="test_key", content="test content", **kw):
"""Helper to insert a memory directly into storage."""
import uuid
mid = kw.pop("memory_id", str(uuid.uuid4()))
storage.create_structured_memory(
mid,
name,
kw.get("description", ""),
kw.get("mem_type", "project"),
kw.get("scope", "global"),
kw.get("scope_id", ""),
content,
)
return mid
# ===========================================================================
# Server endpoint tests
# ===========================================================================
class TestServerListMemories:
def test_empty(self, server_client):
r = server_client.get("/v1/api/memories")
assert r.status_code == 200
data = r.json()
assert data["memories"] == []
assert data["total"] == 0
def test_with_data(self, server_client, storage):
_seed_memory(storage, "key_a", "content a")
_seed_memory(storage, "key_b", "content b")
r = server_client.get("/v1/api/memories")
assert r.status_code == 200
assert r.json()["total"] == 2
def test_filter_by_type(self, server_client, storage):
_seed_memory(storage, "a", "x", mem_type="user")
_seed_memory(storage, "b", "y", mem_type="project")
r = server_client.get("/v1/api/memories?type=user")
assert r.json()["total"] == 1
assert r.json()["memories"][0]["name"] == "a"
def test_filter_by_scope(self, server_client, storage):
_seed_memory(storage, "a", "x", scope="global")
_seed_memory(storage, "b", "y", scope="workstream", scope_id="ws1")
r = server_client.get("/v1/api/memories?scope=workstream&scope_id=ws1")
assert r.json()["total"] == 1
assert r.json()["memories"][0]["name"] == "b"
def test_limit(self, server_client, storage):
for i in range(5):
_seed_memory(storage, f"k{i}", f"v{i}")
r = server_client.get("/v1/api/memories?limit=2")
assert r.json()["total"] == 2
def test_invalid_limit(self, server_client):
r = server_client.get("/v1/api/memories?limit=abc")
assert r.status_code == 400
class TestServerSaveMemory:
def test_create(self, server_client):
r = server_client.post(
"/v1/api/memories",
json={"name": "my_key", "content": "my content"},
)
assert r.status_code == 201
data = r.json()
assert data["name"] == "my_key"
assert data["content"] == "my content"
assert data["type"] == "project"
assert data["scope"] == "global"
def test_upsert(self, server_client):
server_client.post(
"/v1/api/memories",
json={"name": "key", "content": "v1"},
)
r = server_client.post(
"/v1/api/memories",
json={"name": "key", "content": "v2"},
)
assert r.status_code == 200
assert r.json()["content"] == "v2"
def test_with_type_and_scope(self, server_client):
r = server_client.post(
"/v1/api/memories",
json={
"name": "feedback_key",
"content": "data",
"type": "feedback",
"scope": "workstream",
"scope_id": "ws1",
},
)
assert r.status_code == 201
assert r.json()["type"] == "feedback"
assert r.json()["scope"] == "workstream"
def test_missing_name(self, server_client):
r = server_client.post("/v1/api/memories", json={"content": "data"})
assert r.status_code == 400
def test_missing_content(self, server_client):
r = server_client.post("/v1/api/memories", json={"name": "k"})
assert r.status_code == 400
def test_invalid_type(self, server_client):
r = server_client.post(
"/v1/api/memories",
json={"name": "k", "content": "c", "type": "bogus"},
)
assert r.status_code == 400
assert "invalid type" in r.json()["error"]
def test_invalid_scope(self, server_client):
r = server_client.post(
"/v1/api/memories",
json={"name": "k", "content": "c", "scope": "bogus"},
)
assert r.status_code == 400
assert "invalid scope" in r.json()["error"]
def test_content_too_large(self, server_client):
r = server_client.post(
"/v1/api/memories",
json={"name": "k", "content": "x" * 70000},
)
assert r.status_code == 400
assert "limit" in r.json()["error"]
def test_name_normalisation(self, server_client):
r = server_client.post(
"/v1/api/memories",
json={"name": "My-Key Name", "content": "data"},
)
assert r.status_code == 201
assert r.json()["name"] == "my_key_name"
class TestServerUserScopeSecurity:
def test_user_scope_binds_to_auth(self, server_client):
"""User scope auto-resolves scope_id from authenticated user."""
r = server_client.post(
"/v1/api/memories",
json={"name": "priv", "content": "secret", "scope": "user"},
)
assert r.status_code == 201
assert r.json()["scope_id"] == "test-user"
def test_user_scope_rejects_cross_user(self, server_client):
"""Cannot access another user's memories via scope_id."""
r = server_client.post(
"/v1/api/memories",
json={"name": "x", "content": "y", "scope": "user", "scope_id": "other-user"},
)
assert r.status_code == 403
def test_user_scope_allows_own_scope_id(self, server_client):
"""Passing own user_id as scope_id is allowed."""
r = server_client.post(
"/v1/api/memories",
json={"name": "x", "content": "y", "scope": "user", "scope_id": "test-user"},
)
assert r.status_code == 201
def test_list_rejects_cross_user(self, server_client):
r = server_client.get("/v1/api/memories?scope=user&scope_id=other-user")
assert r.status_code == 403
def test_delete_rejects_cross_user(self, server_client, storage):
_seed_memory(storage, "k", "v", scope="user", scope_id="other-user")
r = server_client.delete("/v1/api/memories/k?scope=user&scope_id=other-user")
assert r.status_code == 403
class TestServerSearchMemories:
def test_search(self, server_client, storage):
_seed_memory(storage, "db_config", "postgresql host", description="database")
_seed_memory(storage, "api_key", "secret_value")
r = server_client.post(
"/v1/api/memories/search",
json={"query": "database"},
)
assert r.status_code == 200
assert r.json()["total"] == 1
assert r.json()["memories"][0]["name"] == "db_config"
def test_no_results(self, server_client, storage):
_seed_memory(storage, "a", "b")
r = server_client.post(
"/v1/api/memories/search",
json={"query": "nonexistent_xyz"},
)
assert r.status_code == 200
assert r.json()["total"] == 0
def test_missing_query(self, server_client):
r = server_client.post("/v1/api/memories/search", json={})
assert r.status_code == 400
class TestServerDeleteMemory:
def test_delete(self, server_client, storage):
_seed_memory(storage, "doomed")
r = server_client.delete("/v1/api/memories/doomed")
assert r.status_code == 200
assert r.json()["status"] == "ok"
def test_not_found(self, server_client):
r = server_client.delete("/v1/api/memories/nope")
assert r.status_code == 404
def test_delete_scoped(self, server_client, storage):
_seed_memory(storage, "k", "data", scope="workstream", scope_id="ws1")
# Wrong scope → not found
r = server_client.delete("/v1/api/memories/k")
assert r.status_code == 404
# Correct scope → success
r = server_client.delete("/v1/api/memories/k?scope=workstream&scope_id=ws1")
assert r.status_code == 200
def test_invalid_scope(self, server_client):
r = server_client.delete("/v1/api/memories/k?scope=bogus")
assert r.status_code == 400
# ===========================================================================
# Console admin endpoint tests
# ===========================================================================
class TestAdminListMemories:
def test_empty(self, admin_client):
r = admin_client.get("/v1/api/admin/memories")
assert r.status_code == 200
assert r.json()["memories"] == []
def test_with_data(self, admin_client, storage):
_seed_memory(storage, "a", "1")
_seed_memory(storage, "b", "2")
r = admin_client.get("/v1/api/admin/memories")
assert r.json()["total"] == 2
def test_filter(self, admin_client, storage):
_seed_memory(storage, "a", "1", mem_type="user")
_seed_memory(storage, "b", "2", mem_type="project")
r = admin_client.get("/v1/api/admin/memories?type=user")
assert r.json()["total"] == 1
class TestAdminSearchMemories:
def test_search(self, admin_client, storage):
_seed_memory(storage, "db_config", "pg host", description="database")
_seed_memory(storage, "other", "unrelated")
r = admin_client.get("/v1/api/admin/memories/search?q=database")
assert r.status_code == 200
assert r.json()["total"] == 1
def test_missing_query(self, admin_client):
r = admin_client.get("/v1/api/admin/memories/search")
assert r.status_code == 400
class TestAdminGetMemory:
def test_found(self, admin_client, storage):
mid = _seed_memory(storage, "k", "content")
r = admin_client.get(f"/v1/api/admin/memories/{mid}")
assert r.status_code == 200
assert r.json()["name"] == "k"
def test_not_found(self, admin_client):
r = admin_client.get("/v1/api/admin/memories/nonexistent-id")
assert r.status_code == 404
class TestAdminDeleteMemory:
def test_delete(self, admin_client, storage):
mid = _seed_memory(storage, "doomed", "data")
r = admin_client.delete(f"/v1/api/admin/memories/{mid}")
assert r.status_code == 200
assert r.json()["status"] == "ok"
# Verify it's gone
assert storage.get_structured_memory(mid) is None
def test_not_found(self, admin_client):
r = admin_client.delete("/v1/api/admin/memories/nonexistent-id")
assert r.status_code == 404
# ===========================================================================
# Storage: delete_structured_memory_by_id
# ===========================================================================
class TestDeleteByIdStorage:
def test_delete_existing(self, storage):
storage.create_structured_memory("m1", "k", "d", "project", "global", "", "data")
assert storage.delete_structured_memory_by_id("m1")
assert storage.get_structured_memory("m1") is None
def test_delete_nonexistent(self, storage):
assert not storage.delete_structured_memory_by_id("nope")
+24 -1
View File
@@ -501,4 +501,27 @@ class ListChannelUsersResponse(BaseModel):
class CreateChannelUserRequest(BaseModel):
channel_type: str = Field(..., description="Channel type (e.g. discord, slack)")
channel_user_id: str = Field(..., description="External channel user identifier")
total: int
# ---------------------------------------------------------------------------
# Admin: Memories
# ---------------------------------------------------------------------------
class AdminMemoryInfo(BaseModel):
memory_id: str
name: str
description: str = ""
type: str
scope: str
scope_id: str = ""
content: str
created: str
updated: str
last_accessed: str = ""
access_count: int = 0
class ListAdminMemoriesResponse(BaseModel):
memories: list[AdminMemoryInfo]
total: int = 0
+48
View File
@@ -8,6 +8,7 @@ if TYPE_CHECKING:
from pydantic import BaseModel
from turnstone.api.console_schemas import (
AdminMemoryInfo,
AssignRoleRequest,
AuditEventInfo,
ChannelUserInfo,
@@ -23,6 +24,7 @@ from turnstone.api.console_schemas import (
CreateRoleRequest,
CreateToolPolicyRequest,
CreateWsTemplateRequest,
ListAdminMemoriesResponse,
ListAuditEventsResponse,
ListChannelUsersResponse,
ListOrgsResponse,
@@ -572,6 +574,50 @@ CONSOLE_ENDPOINTS: list[EndpointSpec] = [
],
tags=["Admin"],
),
# --- Admin: Memories ---
EndpointSpec(
"/v1/api/admin/memories",
"GET",
"List structured memories",
response_model=ListAdminMemoriesResponse,
query_params=[
QueryParam("type", "Filter by memory type"),
QueryParam("scope", "Filter by scope"),
QueryParam("scope_id", "Filter by scope identifier"),
QueryParam("limit", "Page size (max 200)", schema_type="integer", default=100),
],
tags=["Admin"],
),
EndpointSpec(
"/v1/api/admin/memories/search",
"GET",
"Search memories by query",
response_model=ListAdminMemoriesResponse,
query_params=[
QueryParam("q", "Search query", required=True),
QueryParam("type", "Filter by memory type"),
QueryParam("scope", "Filter by scope"),
QueryParam("scope_id", "Filter by scope identifier"),
QueryParam("limit", "Max results", schema_type="integer", default=20),
],
tags=["Admin"],
),
EndpointSpec(
"/v1/api/admin/memories/{memory_id}",
"GET",
"Get a single memory by ID",
response_model=AdminMemoryInfo,
error_codes=[404],
tags=["Admin"],
),
EndpointSpec(
"/v1/api/admin/memories/{memory_id}",
"DELETE",
"Delete a memory by ID",
response_model=StatusResponse,
error_codes=[404],
tags=["Admin"],
),
# --- Observability ---
EndpointSpec(
"/health",
@@ -636,6 +682,8 @@ _ALL_MODELS: list[type[BaseModel]] = [
ListAuditEventsResponse,
VerdictInfo,
ListVerdictsResponse,
AdminMemoryInfo,
ListAdminMemoriesResponse,
]
+51
View File
@@ -2,6 +2,8 @@
from __future__ import annotations
from typing import Literal
from pydantic import BaseModel, Field
# ---------------------------------------------------------------------------
@@ -163,3 +165,52 @@ class HealthResponse(BaseModel):
workstreams: WorkstreamCounts = WorkstreamCounts()
backend: BackendStatus | None = None
mcp: McpStatus | None = None
# ---------------------------------------------------------------------------
# Memories
# ---------------------------------------------------------------------------
MemoryType = Literal["user", "project", "feedback", "reference"]
MemoryScope = Literal["global", "workstream", "user"]
class SaveMemoryRequest(BaseModel):
name: str = Field(description="Memory identifier (normalized to snake_case)")
content: str = Field(description="Memory content", max_length=65536)
description: str = Field(default="", description="Short description for relevance matching")
type: MemoryType = Field(default="project", description="Memory type")
scope: MemoryScope = Field(default="global", description="Memory scope")
scope_id: str = Field(
default="",
description="Scope identifier (ws_id for workstream, user_id for user scope)",
)
class MemoryInfo(BaseModel):
memory_id: str
name: str
description: str = ""
type: MemoryType
scope: MemoryScope
scope_id: str = ""
content: str
created: str
updated: str
class ListMemoriesResponse(BaseModel):
memories: list[MemoryInfo]
total: int = 0
MemoryTypeFilter = Literal["", "user", "project", "feedback", "reference"]
MemoryScopeFilter = Literal["", "global", "workstream", "user"]
class SearchMemoriesRequest(BaseModel):
query: str = Field(description="Search query text")
type: MemoryTypeFilter = Field(default="", description="Filter by memory type")
scope: MemoryScopeFilter = Field(default="", description="Filter by scope")
scope_id: str = Field(default="", description="Filter by scope_id")
limit: int = Field(default=20, description="Max results (1-50)", ge=1, le=50)
+53
View File
@@ -26,9 +26,13 @@ from turnstone.api.server_schemas import (
CreateWorkstreamResponse,
DashboardResponse,
HealthResponse,
ListMemoriesResponse,
ListSavedWorkstreamsResponse,
ListWorkstreamsResponse,
MemoryInfo,
PlanFeedbackRequest,
SaveMemoryRequest,
SearchMemoriesRequest,
SendRequest,
SendResponse,
)
@@ -173,6 +177,51 @@ SERVER_ENDPOINTS: list[EndpointSpec] = [
response_model=StatusResponse,
tags=["Auth"],
),
# --- Memories ---
EndpointSpec(
"/v1/api/memories",
"GET",
"List structured memories",
response_model=ListMemoriesResponse,
query_params=[
QueryParam("type", "Filter by memory type"),
QueryParam("scope", "Filter by scope"),
QueryParam("scope_id", "Filter by scope identifier"),
QueryParam(
"limit", "Max results (default 100, max 200)", schema_type="integer", default=100
),
],
tags=["Memories"],
),
EndpointSpec(
"/v1/api/memories",
"POST",
"Save (upsert) a structured memory",
request_model=SaveMemoryRequest,
response_model=MemoryInfo,
error_codes=[400],
tags=["Memories"],
),
EndpointSpec(
"/v1/api/memories/search",
"POST",
"Search structured memories by query",
request_model=SearchMemoriesRequest,
response_model=ListMemoriesResponse,
tags=["Memories"],
),
EndpointSpec(
"/v1/api/memories/{name}",
"DELETE",
"Delete a structured memory by name and scope",
response_model=StatusResponse,
query_params=[
QueryParam("scope", "Scope (default: global)"),
QueryParam("scope_id", "Scope identifier"),
],
error_codes=[404],
tags=["Memories"],
),
# --- Observability ---
EndpointSpec(
"/health",
@@ -204,6 +253,10 @@ _ALL_MODELS: list[type[BaseModel]] = [
DashboardResponse,
ListSavedWorkstreamsResponse,
HealthResponse,
SaveMemoryRequest,
MemoryInfo,
ListMemoriesResponse,
SearchMemoriesRequest,
]
+124
View File
@@ -1511,6 +1511,7 @@ _VALID_PERMISSIONS = frozenset(
"admin.watches",
"admin.ws_templates",
"admin.judge",
"admin.memories",
"tools.approve",
"workstreams.create",
"workstreams.close",
@@ -2607,6 +2608,120 @@ async def admin_list_verdicts(request: Request) -> JSONResponse:
return JSONResponse({"verdicts": verdicts, "total": total})
# ---------------------------------------------------------------------------
# Admin: Memories
# ---------------------------------------------------------------------------
async def admin_list_memories(request: Request) -> JSONResponse:
"""GET /v1/api/admin/memories — list structured memories with filters."""
from turnstone.core.auth import require_permission
from turnstone.core.web_helpers import require_storage_or_503
storage, err = require_storage_or_503(request)
if err:
return err
err = require_permission(request, "admin.memories")
if err:
return err
mem_type = request.query_params.get("type", "")
scope = request.query_params.get("scope", "")
scope_id = request.query_params.get("scope_id", "")
try:
limit = min(int(request.query_params.get("limit", "100")), 200)
except (ValueError, TypeError):
return JSONResponse({"error": "limit must be an integer"}, status_code=400)
rows = storage.list_structured_memories(
mem_type=mem_type, scope=scope, scope_id=scope_id, limit=limit
)
total = storage.count_structured_memories(mem_type=mem_type, scope=scope, scope_id=scope_id)
return JSONResponse({"memories": rows, "total": total})
async def admin_search_memories(request: Request) -> JSONResponse:
"""GET /v1/api/admin/memories/search — search memories by query."""
from turnstone.core.auth import require_permission
from turnstone.core.web_helpers import require_storage_or_503
storage, err = require_storage_or_503(request)
if err:
return err
err = require_permission(request, "admin.memories")
if err:
return err
query = request.query_params.get("q", "").strip()
if not query:
return JSONResponse({"error": "q is required"}, status_code=400)
mem_type = request.query_params.get("type", "")
scope = request.query_params.get("scope", "")
scope_id = request.query_params.get("scope_id", "")
try:
limit = min(int(request.query_params.get("limit", "20")), 50)
except (ValueError, TypeError):
return JSONResponse({"error": "limit must be an integer"}, status_code=400)
rows = storage.search_structured_memories(
query, mem_type=mem_type, scope=scope, scope_id=scope_id, limit=limit
)
return JSONResponse({"memories": rows, "total": len(rows)})
async def admin_get_memory(request: Request) -> JSONResponse:
"""GET /v1/api/admin/memories/{memory_id} — get a single memory."""
from turnstone.core.auth import require_permission
from turnstone.core.web_helpers import require_storage_or_503
storage, err = require_storage_or_503(request)
if err:
return err
err = require_permission(request, "admin.memories")
if err:
return err
memory_id = request.path_params["memory_id"]
mem = storage.get_structured_memory(memory_id)
if not mem:
return JSONResponse({"error": "Memory not found"}, status_code=404)
return JSONResponse(mem)
async def admin_delete_memory(request: Request) -> JSONResponse:
"""DELETE /v1/api/admin/memories/{memory_id} — delete a memory by ID."""
from turnstone.core.audit import record_audit
from turnstone.core.auth import require_permission
from turnstone.core.web_helpers import require_storage_or_503
storage, err = require_storage_or_503(request)
if err:
return err
err = require_permission(request, "admin.memories")
if err:
return err
memory_id = request.path_params["memory_id"]
existing = storage.get_structured_memory(memory_id)
if not existing:
return JSONResponse({"error": "Memory not found"}, status_code=404)
storage.delete_structured_memory_by_id(memory_id)
audit_uid, ip = _audit_context(request)
record_audit(
storage,
audit_uid,
"memory.delete",
"memory",
memory_id,
{"name": existing.get("name", ""), "scope": existing.get("scope", "")},
ip,
)
return JSONResponse({"status": "ok"})
# ---------------------------------------------------------------------------
# App factory
# ---------------------------------------------------------------------------
@@ -2748,6 +2863,15 @@ def create_app(
"/api/admin/ws-templates/{ws_template_id}/versions",
admin_list_ws_template_versions,
),
# Governance: Memories
Route("/api/admin/memories", admin_list_memories),
Route("/api/admin/memories/search", admin_search_memories),
Route("/api/admin/memories/{memory_id}", admin_get_memory),
Route(
"/api/admin/memories/{memory_id}",
admin_delete_memory,
methods=["DELETE"],
),
# Governance: Usage & Audit
Route("/api/admin/usage", admin_usage),
Route("/api/admin/audit", admin_audit),
+4
View File
@@ -163,6 +163,7 @@ WRITE_PATHS: frozenset[str] = frozenset(
"/api/workstreams/new",
"/api/workstreams/close",
"/api/cluster/workstreams/new",
"/api/memories",
}
)
@@ -465,6 +466,9 @@ def required_scope(method: str, path: str) -> str:
and normalized.endswith("/cancel")
):
return "write"
# Memory delete: /api/memories/{name}
if method == "DELETE" and normalized.startswith("/api/memories/"):
return "write"
# Console proxy routes: /node/{node_id}/api/{tail} or /node/{node_id}/v1/api/{tail}
if method == "POST" and normalized.startswith("/node/"):
+13 -3
View File
@@ -295,6 +295,14 @@ def delete_structured_memory(name: str, scope: str = "global", scope_id: str = "
return False
def delete_structured_memory_by_id(memory_id: str) -> bool:
"""Delete a structured memory by its primary key. Returns True if existed."""
try:
return get_storage().delete_structured_memory_by_id(memory_id)
except Exception:
return False
def list_structured_memories(
mem_type: str = "",
scope: str = "",
@@ -326,9 +334,11 @@ def search_structured_memories(
return []
def count_structured_memories(scope: str = "", scope_id: str = "") -> int:
"""Count structured memories with optional scope filter."""
def count_structured_memories(mem_type: str = "", scope: str = "", scope_id: str = "") -> int:
"""Count structured memories with optional type/scope filter."""
try:
return get_storage().count_structured_memories(scope=scope, scope_id=scope_id)
return get_storage().count_structured_memories(
mem_type=mem_type, scope=scope, scope_id=scope_id
)
except Exception:
return 0
+13 -1
View File
@@ -2180,6 +2180,14 @@ class PostgreSQLBackend:
conn.commit()
return result.rowcount > 0
def delete_structured_memory_by_id(self, memory_id: str) -> bool:
with self._engine.connect() as conn:
result = conn.execute(
sa.delete(structured_memories).where(structured_memories.c.memory_id == memory_id)
)
conn.commit()
return result.rowcount > 0
def list_structured_memories(
self,
mem_type: str = "",
@@ -2244,9 +2252,13 @@ class PostgreSQLBackend:
).fetchall()
return [dict(r._mapping) for r in rows]
def count_structured_memories(self, scope: str = "", scope_id: str = "") -> int:
def count_structured_memories(
self, mem_type: str = "", scope: str = "", scope_id: str = ""
) -> int:
with self._engine.connect() as conn:
q = sa.select(sa.func.count()).select_from(structured_memories)
if mem_type:
q = q.where(structured_memories.c.type == mem_type)
if scope:
q = q.where(structured_memories.c.scope == scope)
if scope_id:
+8 -2
View File
@@ -106,6 +106,10 @@ class StorageBackend(Protocol):
"""Delete a structured memory by (name, scope, scope_id). Returns True if existed."""
...
def delete_structured_memory_by_id(self, memory_id: str) -> bool:
"""Delete a structured memory by its primary key. Returns True if existed."""
...
def list_structured_memories(
self,
mem_type: str = "",
@@ -127,8 +131,10 @@ class StorageBackend(Protocol):
"""Search structured memories by query. Returns matching memory dicts."""
...
def count_structured_memories(self, scope: str = "", scope_id: str = "") -> int:
"""Count structured memories with optional scope filter."""
def count_structured_memories(
self, mem_type: str = "", scope: str = "", scope_id: str = ""
) -> int:
"""Count structured memories with optional type and scope filters."""
...
# -- Workstream operations -------------------------------------------------
+13 -1
View File
@@ -2204,6 +2204,14 @@ class SQLiteBackend:
conn.commit()
return result.rowcount > 0
def delete_structured_memory_by_id(self, memory_id: str) -> bool:
with self._engine.connect() as conn:
result = conn.execute(
sa.delete(structured_memories).where(structured_memories.c.memory_id == memory_id)
)
conn.commit()
return result.rowcount > 0
def list_structured_memories(
self,
mem_type: str = "",
@@ -2268,9 +2276,13 @@ class SQLiteBackend:
).fetchall()
return [dict(r._mapping) for r in rows]
def count_structured_memories(self, scope: str = "", scope_id: str = "") -> int:
def count_structured_memories(
self, mem_type: str = "", scope: str = "", scope_id: str = ""
) -> int:
with self._engine.connect() as conn:
q = sa.select(sa.func.count()).select_from(structured_memories)
if mem_type:
q = q.where(structured_memories.c.type == mem_type)
if scope:
q = q.where(structured_memories.c.scope == scope)
if scope_id:
@@ -56,6 +56,15 @@ def upgrade() -> None:
)
op.drop_table("memories")
# Grant admin.memories permission to the built-in admin role
conn.execute(
sa.text(
"UPDATE roles SET permissions = permissions || ',admin.memories' "
"WHERE role_id = 'builtin-admin' "
"AND permissions NOT LIKE '%admin.memories%'"
)
)
def downgrade() -> None:
op.create_table(
+95
View File
@@ -14,12 +14,14 @@ from __future__ import annotations
from typing import TYPE_CHECKING, Any
from turnstone.api.console_schemas import (
AdminMemoryInfo,
ClusterNodesResponse,
ClusterOverviewResponse,
ClusterSnapshotResponse,
ClusterWorkstreamsResponse,
ConsoleCreateWsResponse,
ConsoleHealthResponse,
ListAdminMemoriesResponse,
ListAuditEventsResponse,
ListOrgsResponse,
ListPromptTemplatesResponse,
@@ -568,6 +570,67 @@ class AsyncTurnstoneConsole(_BaseClient):
"GET", "/v1/api/admin/audit", params=params, response_model=ListAuditEventsResponse
)
# -- governance: memories ------------------------------------------------
async def list_memories(
self,
*,
mem_type: str = "",
scope: str = "",
scope_id: str = "",
limit: int = 100,
) -> ListAdminMemoriesResponse:
params: dict[str, Any] = {"limit": limit}
if mem_type:
params["type"] = mem_type
if scope:
params["scope"] = scope
if scope_id:
params["scope_id"] = scope_id
return await self._request(
"GET",
"/v1/api/admin/memories",
params=params,
response_model=ListAdminMemoriesResponse,
)
async def search_memories(
self,
query: str,
*,
mem_type: str = "",
scope: str = "",
scope_id: str = "",
limit: int = 20,
) -> ListAdminMemoriesResponse:
params: dict[str, Any] = {"q": query, "limit": limit}
if mem_type:
params["type"] = mem_type
if scope:
params["scope"] = scope
if scope_id:
params["scope_id"] = scope_id
return await self._request(
"GET",
"/v1/api/admin/memories/search",
params=params,
response_model=ListAdminMemoriesResponse,
)
async def get_memory(self, memory_id: str) -> AdminMemoryInfo:
return await self._request(
"GET",
f"/v1/api/admin/memories/{memory_id}",
response_model=AdminMemoryInfo,
)
async def delete_memory(self, memory_id: str) -> StatusResponse:
return await self._request(
"DELETE",
f"/v1/api/admin/memories/{memory_id}",
response_model=StatusResponse,
)
class TurnstoneConsole:
"""Synchronous client for the turnstone console API.
@@ -897,6 +960,38 @@ class TurnstoneConsole:
)
)
# -- governance: memories ------------------------------------------------
def list_memories(
self, *, mem_type: str = "", scope: str = "", scope_id: str = "", limit: int = 100
) -> ListAdminMemoriesResponse:
return self._runner.run(
self._async.list_memories(
mem_type=mem_type, scope=scope, scope_id=scope_id, limit=limit
)
)
def search_memories(
self,
query: str,
*,
mem_type: str = "",
scope: str = "",
scope_id: str = "",
limit: int = 20,
) -> ListAdminMemoriesResponse:
return self._runner.run(
self._async.search_memories(
query, mem_type=mem_type, scope=scope, scope_id=scope_id, limit=limit
)
)
def get_memory(self, memory_id: str) -> AdminMemoryInfo:
return self._runner.run(self._async.get_memory(memory_id))
def delete_memory(self, memory_id: str) -> StatusResponse:
return self._runner.run(self._async.delete_memory(memory_id))
# -- lifecycle -----------------------------------------------------------
def close(self) -> None:
+139
View File
@@ -26,8 +26,10 @@ from turnstone.api.server_schemas import (
CreateWorkstreamResponse,
DashboardResponse,
HealthResponse,
ListMemoriesResponse,
ListSavedWorkstreamsResponse,
ListWorkstreamsResponse,
MemoryInfo,
SendResponse,
)
from turnstone.sdk._base import _BaseClient
@@ -235,6 +237,91 @@ class AsyncTurnstoneServer(_BaseClient):
"GET", "/v1/api/workstreams/saved", response_model=ListSavedWorkstreamsResponse
)
# -- memories ------------------------------------------------------------
async def list_memories(
self,
*,
mem_type: str = "",
scope: str = "",
scope_id: str = "",
limit: int = 100,
) -> ListMemoriesResponse:
params: dict[str, Any] = {"limit": limit}
if mem_type:
params["type"] = mem_type
if scope:
params["scope"] = scope
if scope_id:
params["scope_id"] = scope_id
return await self._request(
"GET", "/v1/api/memories", params=params, response_model=ListMemoriesResponse
)
async def save_memory(
self,
name: str,
content: str,
*,
description: str = "",
mem_type: str = "project",
scope: str = "global",
scope_id: str = "",
) -> MemoryInfo:
body: dict[str, Any] = {
"name": name,
"content": content,
"type": mem_type,
"scope": scope,
}
if description:
body["description"] = description
if scope_id:
body["scope_id"] = scope_id
return await self._request(
"POST", "/v1/api/memories", json_body=body, response_model=MemoryInfo
)
async def search_memories(
self,
query: str,
*,
mem_type: str = "",
scope: str = "",
scope_id: str = "",
limit: int = 20,
) -> ListMemoriesResponse:
body: dict[str, Any] = {"query": query, "limit": limit}
if mem_type:
body["type"] = mem_type
if scope:
body["scope"] = scope
if scope_id:
body["scope_id"] = scope_id
return await self._request(
"POST",
"/v1/api/memories/search",
json_body=body,
response_model=ListMemoriesResponse,
)
async def delete_memory(
self,
name: str,
*,
scope: str = "global",
scope_id: str = "",
) -> StatusResponse:
params: dict[str, Any] = {"scope": scope}
if scope_id:
params["scope_id"] = scope_id
return await self._request(
"DELETE",
f"/v1/api/memories/{name}",
params=params,
response_model=StatusResponse,
)
# -- auth ----------------------------------------------------------------
async def login(
@@ -394,6 +481,58 @@ class TurnstoneServer:
def list_saved_workstreams(self) -> ListSavedWorkstreamsResponse:
return self._runner.run(self._async.list_saved_workstreams())
# -- memories ------------------------------------------------------------
def list_memories(
self, *, mem_type: str = "", scope: str = "", scope_id: str = "", limit: int = 100
) -> ListMemoriesResponse:
return self._runner.run(
self._async.list_memories(
mem_type=mem_type, scope=scope, scope_id=scope_id, limit=limit
)
)
def save_memory(
self,
name: str,
content: str,
*,
description: str = "",
mem_type: str = "project",
scope: str = "global",
scope_id: str = "",
) -> MemoryInfo:
return self._runner.run(
self._async.save_memory(
name,
content,
description=description,
mem_type=mem_type,
scope=scope,
scope_id=scope_id,
)
)
def search_memories(
self,
query: str,
*,
mem_type: str = "",
scope: str = "",
scope_id: str = "",
limit: int = 20,
) -> ListMemoriesResponse:
return self._runner.run(
self._async.search_memories(
query, mem_type=mem_type, scope=scope, scope_id=scope_id, limit=limit
)
)
def delete_memory(
self, name: str, *, scope: str = "global", scope_id: str = ""
) -> StatusResponse:
return self._runner.run(self._async.delete_memory(name, scope=scope, scope_id=scope_id))
# -- auth ----------------------------------------------------------------
def login(
+164
View File
@@ -1359,6 +1359,166 @@ async def cancel_watch(request: Request) -> JSONResponse:
return JSONResponse({"status": "ok", "watch_id": watch_id})
# ---------------------------------------------------------------------------
# Memory endpoints
# ---------------------------------------------------------------------------
_VALID_MEMORY_TYPES = frozenset({"user", "project", "feedback", "reference"})
_VALID_MEMORY_SCOPES = frozenset({"global", "workstream", "user"})
_MAX_MEMORY_CONTENT = 65536 # hard upper bound; server may enforce lower via config
def _resolve_user_scope_id(
request: Request, provided_scope_id: str = ""
) -> tuple[str, JSONResponse | None]:
"""Resolve and validate scope_id for user-scoped memory.
Always binds to the authenticated user's identity. If a scope_id is
provided and doesn't match, returns 403 to prevent cross-user access.
"""
auth = getattr(getattr(request, "state", None), "auth_result", None)
uid: str = getattr(auth, "user_id", "") or ""
if not uid:
return "", JSONResponse(
{"error": "User scope requires authentication with a user identity"},
status_code=400,
)
if provided_scope_id and provided_scope_id != uid:
return "", JSONResponse(
{"error": "Cannot access another user's memories"},
status_code=403,
)
return uid, None
async def list_memories(request: Request) -> JSONResponse:
"""GET /v1/api/memories — list memories with optional filters."""
from turnstone.core.memory import list_structured_memories
mem_type = request.query_params.get("type", "")
scope = request.query_params.get("scope", "")
scope_id = request.query_params.get("scope_id", "")
try:
limit = min(int(request.query_params.get("limit", "100")), 200)
except (ValueError, TypeError):
return JSONResponse({"error": "limit must be an integer"}, status_code=400)
if scope == "user":
scope_id, err = _resolve_user_scope_id(request, scope_id)
if err:
return err
rows = list_structured_memories(mem_type=mem_type, scope=scope, scope_id=scope_id, limit=limit)
return JSONResponse({"memories": rows, "total": len(rows)})
async def save_memory(request: Request) -> JSONResponse:
"""POST /v1/api/memories — save (upsert) a structured memory."""
from turnstone.core.memory import save_structured_memory
from turnstone.core.web_helpers import read_json_or_400
body = await read_json_or_400(request)
if isinstance(body, JSONResponse):
return body
name = str(body.get("name", "")).strip()
content = str(body.get("content", "")).strip()
if not name or len(name) > 256:
return JSONResponse({"error": "name is required (max 256 characters)"}, status_code=400)
if not content:
return JSONResponse({"error": "content is required"}, status_code=400)
if len(content) > _MAX_MEMORY_CONTENT:
return JSONResponse(
{"error": f"content exceeds {_MAX_MEMORY_CONTENT} character limit"},
status_code=400,
)
description = str(body.get("description", ""))
mem_type = str(body.get("type", "project"))
scope = str(body.get("scope", "global"))
scope_id = str(body.get("scope_id", ""))
if mem_type not in _VALID_MEMORY_TYPES:
return JSONResponse(
{"error": f"invalid type: {mem_type}; must be one of {sorted(_VALID_MEMORY_TYPES)}"},
status_code=400,
)
if scope not in _VALID_MEMORY_SCOPES:
return JSONResponse(
{"error": f"invalid scope: {scope}; must be one of {sorted(_VALID_MEMORY_SCOPES)}"},
status_code=400,
)
if scope == "user":
scope_id, err = _resolve_user_scope_id(request, scope_id)
if err:
return err
# save_structured_memory normalises the name internally
from turnstone.core.memory import normalize_key
normalized_name = normalize_key(name)
memory_id, old_content = save_structured_memory(
name, content, description=description, mem_type=mem_type, scope=scope, scope_id=scope_id
)
if not memory_id:
return JSONResponse({"error": "Failed to save memory"}, status_code=500)
from turnstone.core.storage._registry import get_storage
storage = get_storage()
mem = storage.get_structured_memory(memory_id) if storage else None
if not mem:
return JSONResponse(
{"memory_id": memory_id, "name": normalized_name, "status": "saved"},
status_code=201,
)
status_code = 200 if old_content is not None else 201
return JSONResponse(mem, status_code=status_code)
async def search_memories(request: Request) -> JSONResponse:
"""POST /v1/api/memories/search — search memories by query.
Uses POST for the request body but requires only read scope (non-mutating).
"""
from turnstone.core.memory import search_structured_memories as search_fn
from turnstone.core.web_helpers import read_json_or_400
body = await read_json_or_400(request)
if isinstance(body, JSONResponse):
return body
query = str(body.get("query", "")).strip()
if not query:
return JSONResponse({"error": "query is required"}, status_code=400)
mem_type = str(body.get("type", ""))
scope = str(body.get("scope", ""))
scope_id = str(body.get("scope_id", ""))
try:
limit = min(int(body.get("limit", 20)), 50)
except (ValueError, TypeError):
return JSONResponse({"error": "limit must be an integer"}, status_code=400)
if scope == "user":
scope_id, err = _resolve_user_scope_id(request, scope_id)
if err:
return err
rows = search_fn(query, mem_type=mem_type, scope=scope, scope_id=scope_id, limit=limit)
return JSONResponse({"memories": rows, "total": len(rows)})
async def delete_memory_endpoint(request: Request) -> JSONResponse:
"""DELETE /v1/api/memories/{name} — delete a memory by name and scope."""
from turnstone.core.memory import delete_structured_memory, normalize_key
name = normalize_key(request.path_params["name"])
scope = request.query_params.get("scope", "global")
if scope not in _VALID_MEMORY_SCOPES:
return JSONResponse(
{"error": f"invalid scope: {scope}; must be one of {sorted(_VALID_MEMORY_SCOPES)}"},
status_code=400,
)
scope_id = request.query_params.get("scope_id", "")
if scope == "user":
scope_id, err = _resolve_user_scope_id(request, scope_id)
if err:
return err
if delete_structured_memory(name, scope, scope_id):
return JSONResponse({"status": "ok", "name": name})
return JSONResponse({"error": f"Memory '{name}' not found"}, status_code=404)
async def auth_login(request: Request) -> Response:
"""POST /v1/api/auth/login — authenticate and return JWT."""
from turnstone.core.auth import handle_auth_login
@@ -1544,6 +1704,10 @@ def create_app(
Route("/api/workstreams/close", close_workstream, methods=["POST"]),
Route("/api/watches", list_watches),
Route("/api/watches/{watch_id}/cancel", cancel_watch, methods=["POST"]),
Route("/api/memories", list_memories),
Route("/api/memories", save_memory, methods=["POST"]),
Route("/api/memories/search", search_memories, methods=["POST"]),
Route("/api/memories/{name}", delete_memory_endpoint, methods=["DELETE"]),
Route("/api/auth/login", auth_login, methods=["POST"]),
Route("/api/auth/logout", auth_logout, methods=["POST"]),
Route("/api/auth/status", auth_status),