turnstone/docs/memory.md

Structured Memory

See also: Memory Architecture diagram

The structured memory system gives the AI persistent, typed, scoped memories that survive across sessions and workstreams. Memories are automatically surfaced in the system message via BM25 relevance scoring, so the model has contextual recall without explicit search.

Overview

Each memory has three dimensions:

• Type -- categorizes the memory's purpose
• Scope -- controls visibility boundaries
• Name -- unique identifier within a scope (snake_case, normalized)

Memory types

Type	Purpose
user	User preferences, conventions, working style
project	Project-specific knowledge, architecture, patterns
feedback	Corrections, lessons learned, things to avoid
reference	Reference material, documentation, specifications

Memory scopes

Scope	Visibility
global	Visible to all workstreams and users
workstream	Visible only within the originating workstream
user	Follows the authenticated user across workstreams
coordinator	Coordinator sessions only; follows the user across coordinators

A memory's identity is the tuple (name, scope, scope_id). Saving a memory with the same identity upserts -- updating content while preserving the ID.

Coordinator scope

Coordinator sessions are isolated to a single scope: coordinator, keyed by the coordinator's creator user_id. It is durable -- every coordinator session the same user runs (including concurrent ones) shares one orchestration namespace, so procedures and lessons survive close/reopen.

Isolation is bidirectional and enforced by session kind, not by secrecy of the scope id:

• A coordinator session can read and write only coordinator-scope rows. It never sees global/workstream/user memories, so content written by interactive sessions (which routinely ingest untrusted MCP/attachment output) cannot reach a coordinator's system message.
• Interactive sessions -- including a coordinator's own children, which share its user_id -- are rejected from the coordinator scope on every memory action. Children cannot plant rows the parent coordinator would read.
• The REST memory API (/v1/api/memories) does not accept the coordinator scope at all; the scope is written exclusively through a coordinator session's own memory tool.

Coordinator sessions require an authenticated user identity -- an anonymous coordinator cannot be constructed, so the scope id is always a real user.

BM25 relevance injection

On every conversation turn, the system:

1. Fetches up to fetch_limit memories visible in the current scope
2. Extracts context from the last 3 user messages
3. Scores memories against that context using a BM25 index
4. Injects the top relevance_k memories into the system message as <memories> XML tags
5. Appends a hint telling the model how many memories are in scope

This means the model always has its most relevant memories available without explicit recall -- but can still use memory(action='search') for deeper lookup.

The persona memory lever gates this pathway: a workstream whose persona turns memory off receives no relevance injection at all -- the steps above run only when memory is enabled for the session. See Personas.

Nudges

The metacognition layer can nudge the model to save memories at appropriate moments (e.g., after a correction or when resuming a workstream). Nudges are rate-limited by nudge_cooldown and can be disabled entirely.

---

Configuration

config.toml

[memory]
relevance_k = 5          # top-k memories injected per turn
fetch_limit = 50          # max memories fetched from storage for scoring
max_content = 32768       # max content length per memory (characters)
nudge_cooldown = 300      # minimum seconds between memory nudges
nudges = true             # enable/disable metacognitive nudges

All fields are optional. Defaults are shown above.

---

Tool Usage

The memory tool supports four actions:

save

Store or update a memory.

{
  "action": "save",
  "name": "project_architecture",
  "content": "The project uses a hexagonal architecture with...",
  "description": "Core architecture patterns",
  "type": "project",
  "scope": "global"
}

Parameter	Required	Default	Description
name	yes	--	Snake_case identifier (max 256 chars)
content	yes	--	Memory content (max max_content chars)
description	no	""	Short description for relevance matching
type	no	"project"	One of: user, project, feedback, reference
scope	no	"global"	One of: global, workstream, user

search

Find memories by query (BM25 full-text search).

{
  "action": "search",
  "query": "authentication patterns",
  "type": "project",
  "limit": 10
}

Parameter	Required	Default	Description
query	yes	--	Search query
type	no	""	Filter by type
scope	no	""	Filter by scope
limit	no	20	Max results (capped at 50)

delete

Remove a memory by name.

{
  "action": "delete",
  "name": "outdated_pattern",
  "scope": "global"
}

Parameter	Required	Default	Description
name	yes	--	Memory name to delete
scope	no	"global"	Scope of the memory

list

List all memories with optional filters.

{
  "action": "list",
  "type": "feedback",
  "limit": 50
}

Parameter	Required	Default	Description
type	no	""	Filter by type
scope	no	""	Filter by scope
limit	no	20	Max results (capped at 50)

---

Server API

Four endpoints on the server for programmatic memory access.

GET /v1/api/memories

List memories with optional filters.

Query parameters:

Parameter	Type	Required	Default	Description
type	string	no	""	Filter by memory type
scope	string	no	""	Filter by scope
scope_id	string	no	""	Filter by scope ID
limit	int	no	100	Max results (capped at 200)

When scope=user and scope_id is omitted, the authenticated user's ID is used automatically.

Response: 200

{
  "memories": [
    {
      "memory_id": "a1b2c3d4-e5f6-...",
      "name": "project_architecture",
      "description": "Core architecture patterns",
      "type": "project",
      "scope": "global",
      "scope_id": "",
      "content": "The project uses a hexagonal architecture...",
      "created": "2026-03-10T10:00:00",
      "updated": "2026-03-12T14:30:00"
    }
  ],
  "total": 1
}

---

POST /v1/api/memories

Save or upsert a structured memory.

Request body:

{
  "name": "deployment_process",
  "content": "Deploy via GitHub Actions. Staging auto-deploys on push to main.",
  "description": "CI/CD deployment workflow",
  "type": "project",
  "scope": "global",
  "scope_id": ""
}

Field	Type	Required	Default	Description
name	string	yes	--	Memory name (max 256 chars)
content	string	yes	--	Memory content (max 65536 chars)
description	string	no	""	Short description for search ranking
type	string	no	"project"	One of: user, project, feedback, reference
scope	string	no	"global"	One of: global, workstream, user
scope_id	string	no	""	Scope qualifier (auto-resolved for user scope)

Response (created): 201

{
  "memory_id": "a1b2c3d4-e5f6-...",
  "name": "deployment_process",
  "description": "CI/CD deployment workflow",
  "type": "project",
  "scope": "global",
  "scope_id": "",
  "content": "Deploy via GitHub Actions...",
  "created": "2026-03-14T10:00:00",
  "updated": "2026-03-14T10:00:00"
}

Response (updated): 200 -- same schema, returned when a memory with the same (name, scope, scope_id) already existed.

Errors:

Status	Condition
400	Missing name, empty content, invalid type/scope, content too long

---

POST /v1/api/memories/search

Search memories by query. Uses POST for the request body but is non-mutating (requires only read scope).

Request body:

{
  "query": "authentication",
  "type": "project",
  "scope": "",
  "scope_id": "",
  "limit": 20
}

Field	Type	Required	Default	Description
query	string	yes	--	Search query
type	string	no	""	Filter by type
scope	string	no	""	Filter by scope
scope_id	string	no	""	Filter by scope ID
limit	int	no	20	Max results (capped at 50)

Response: 200

{
  "memories": [
    {
      "memory_id": "a1b2c3d4-e5f6-...",
      "name": "auth_patterns",
      "description": "Authentication architecture",
      "type": "project",
      "scope": "global",
      "scope_id": "",
      "content": "JWT tokens with HS256...",
      "created": "2026-03-10T10:00:00",
      "updated": "2026-03-12T14:30:00"
    }
  ],
  "total": 1
}

---

DELETE /v1/api/memories/{name}

Delete a memory by name and scope.

Path parameters:

Parameter	Type	Description
name	string	Memory name

Query parameters:

Parameter	Type	Required	Default	Description
scope	string	no	"global"	Scope of the memory
scope_id	string	no	""	Scope qualifier

Response (success): 200

{"status": "ok", "name": "deployment_process"}

Response (not found): 404

{"error": "Memory 'deployment_process' not found"}

---

Console Admin API

Four admin endpoints for cross-workstream memory management. All require the admin.memories permission.

GET /v1/api/admin/memories

List memories across all scopes (no automatic scope resolution).

Query parameters:

Parameter	Type	Required	Default	Description
type	string	no	""	Filter by type
scope	string	no	""	Filter by scope
scope_id	string	no	""	Filter by scope ID
limit	int	no	100	Max results (capped at 200)

Response: 200

{
  "memories": [
    {
      "memory_id": "a1b2c3d4-e5f6-...",
      "name": "project_architecture",
      "description": "Core architecture patterns",
      "type": "project",
      "scope": "global",
      "scope_id": "",
      "content": "The project uses...",
      "created": "2026-03-10T10:00:00",
      "updated": "2026-03-12T14:30:00"
    }
  ],
  "total": 1
}

---

GET /v1/api/admin/memories/search

Search memories by query (uses query parameters, not POST body).

Query parameters:

Parameter	Type	Required	Default	Description
q	string	yes	--	Search query
type	string	no	""	Filter by type
scope	string	no	""	Filter by scope
scope_id	string	no	""	Filter by scope ID
limit	int	no	20	Max results (capped at 50)

Response: 200 -- same schema as GET /v1/api/admin/memories.

---

GET /v1/api/admin/memories/{memory_id}

Get a single memory by ID.

Path parameters:

Parameter	Type	Description
memory_id	string	Memory UUID

Response (success): 200

{
  "memory_id": "a1b2c3d4-e5f6-...",
  "name": "project_architecture",
  "description": "Core architecture patterns",
  "type": "project",
  "scope": "global",
  "scope_id": "",
  "content": "The project uses...",
  "created": "2026-03-10T10:00:00",
  "updated": "2026-03-12T14:30:00"
}

Response (not found): 404

{"error": "Memory not found"}

---

DELETE /v1/api/admin/memories/{memory_id}

Delete a memory by ID. Records an audit event (memory.delete).

Path parameters:

Parameter	Type	Description
memory_id	string	Memory UUID

Response (success): 200

{"status": "ok"}

Response (not found): 404

{"error": "Memory not found"}

---

SDK

Python

The server SDK uses mem_type (not type) to avoid shadowing the Python builtin.

from turnstone.sdk import TurnstoneServer

with TurnstoneServer("http://localhost:8080", token="tok_xxx") as client:
    # Save a memory
    mem = client.save_memory(
        "api_conventions",
        "All endpoints use /v1/ prefix. JSON responses.",
        description="API design patterns",
        mem_type="project",
        scope="global",
    )
    print(mem.memory_id)

    # Search memories
    results = client.search_memories("authentication", mem_type="project", limit=10)
    for m in results.memories:
        print(f"{m['name']}: {m['description']}")

    # List memories
    all_mems = client.list_memories(mem_type="feedback", limit=50)

    # Delete a memory
    client.delete_memory("api_conventions", scope="global")

Console admin SDK:

from turnstone.sdk import TurnstoneConsole

with TurnstoneConsole("http://localhost:9090", token="tok_xxx") as admin:
    # List all memories (admin view, no scope auto-resolution)
    result = admin.list_memories(scope="global", limit=100)

    # Search
    result = admin.search_memories("architecture", mem_type="project")

    # Get by ID
    mem = admin.get_memory("a1b2c3d4-e5f6-...")

    # Delete by ID
    admin.delete_memory("a1b2c3d4-e5f6-...")

TypeScript

import { TurnstoneServer } from "@turnstone/sdk";

const client = new TurnstoneServer({
  baseUrl: "http://localhost:8080",
  token: "tok_xxx",
});

// Save a memory
const mem = await client.saveMemory({
  name: "api_conventions",
  content: "All endpoints use /v1/ prefix. JSON responses.",
  description: "API design patterns",
  type: "project",
  scope: "global",
});

// Search memories
const results = await client.searchMemories({
  query: "authentication",
  type: "project",
  limit: 10,
});

// List memories
const all = await client.listMemories({ type: "feedback", limit: 50 });

// Delete a memory
await client.deleteMemory("api_conventions", { scope: "global" });

Console admin SDK:

import { TurnstoneConsole } from "@turnstone/sdk";

const admin = new TurnstoneConsole({
  baseUrl: "http://localhost:9090",
  token: "tok_xxx",
});

// List, search, get, delete by ID
const mems = await admin.listMemories({ scope: "global" });
const found = await admin.searchMemories({ q: "auth", limit: 20 });
const one = await admin.getMemory("a1b2c3d4-e5f6-...");
await admin.deleteMemory("a1b2c3d4-e5f6-...");

---

Storage

Memories are stored in the structured_memories table (migration 013). The unique constraint on (name, scope, scope_id) ensures upsert semantics. The name is normalized on save: lowercased, hyphens and spaces replaced with underscores.

Architecture

See Memory Architecture diagram for the full data flow covering the session tool path, API path, admin path, and BM25 relevance injection.