Compare commits

...

22 Commits

Author SHA1 Message Date
Patrick Buckley 0397640567 feat(tls): support cross-host mTLS with lacme 1.2 2026-08-13 02:19:45 -07:00
Patrick Buckley 28ef63a10c fix: revalidate frontend assets across builds 2026-08-12 19:00:31 -07:00
Patrick Buckley 998271b016 feat: convert large pastes to attachments 2026-08-12 00:27:38 -07:00
Patrick Buckley 6eae1c3954 fix(providers): support OpenAI v3 HTTPX2 transport 2026-08-11 23:30:44 -07:00
Patrick Buckley d961af5c45 Remove unreachable MCP owner branch 2026-08-11 22:03:10 -07:00
Patrick Buckley 0599d72625 Apply MCP formatting 2026-08-11 22:03:10 -07:00
Patrick Buckley 2b51b2f8fa Resolve FastMCP settings before lifecycle tests 2026-08-11 22:03:10 -07:00
Patrick Buckley 8517b42ec1 Handle partial MCP resource discovery 2026-08-11 22:03:10 -07:00
Patrick Buckley ed6286ab63 fix(deps): constrain OpenAI SDK below v3 2026-08-11 21:58:21 -07:00
Patrick Buckley 3252f3fd95 fix(memory): preserve replay identity and validation 2026-08-11 21:58:21 -07:00
Patrick Buckley cc84f9d176 fix(memory): harden project scope authorization and consistency 2026-08-11 21:58:21 -07:00
Copilot d2a6c2852e Stabilize context-overflow compaction test in Python 3.11 CI (#1006)
* Stabilize overflow compaction test expectations

Co-authored-by: eous <13773563+eous@users.noreply.github.com>

* Fix ObservedRLock compatibility with Python 3.14 Condition

Co-authored-by: eous <13773563+eous@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: eous <13773563+eous@users.noreply.github.com>
2026-08-11 05:46:02 -07:00
Patrick Buckley cf2811fceb chore: bump version to 1.8.0a7 2026-08-11 04:19:12 -07:00
Patrick Buckley 480a1426b3 Fail-closed history-commit handoff (#1005)
* fix(session): fail-closed history-commit handoff (#981)

The deleted-workstream discovery is now a terminal, ws_id-keyed latch:
keyed conversation commits refuse admission once the durable parent is
gone (convergence finalizers and force-abandon are exempt), history
handoff refuses to mint a proof token so /history fails closed with a
503 instead of silently wiping the pane, and the SSE stream carries a
workstream_gone resync reason. Discarded commits leave a forensic log
of commit keys and roles, never content.

Conversation rows gain a commit_key (migration 071): keyed saves are
idempotent under retry, validated against the full commit identity, and
refused when they would cross a workstream deletion. The prune orphan
category now requires a NULL alias plus a two-hour updated grace, with
cutoffs computed at discovery time and carried into both dialects'
rechecks.

The mid-turn interjection queue is owner-partitioned with no per-site
mode flags: pops take the acting principal's and unowned rows, other
participants' rows are structurally retained, and enforcement lives at
queue admission plus the shared before_spawn gates. The retraction
ledger is bounded by open pop windows: pops open a window atomically
with the queue delete, restores close their ids atomically with the
ledger consume, every other exit closes through one helper, and misses
for unheld ids record nothing. The workstream-gone latch refuses
unattended wakes at all three gates (watcher spawn, claim, delivery
pre-pop), and the retry dispatcher regained its pre-envelope
cancel/error convergence net.

Persistence-state reporting derives through the session bound to each
UI instead of a registry lookup by id that failed open to healthy
during tombstone retention. The dashboard roster no longer re-inserts
ghost entries from trailing activity events, the history tool-outcome
scan tolerates interleaved non-turn rows, and the shared
handoff-deadline handle owns its own retirement.

Single-sourced across call sites: keyed-commit row values, attachment
save wrappers, tail-truncation and conflict-resolution bodies for both
storage dialects; worker-slot lifecycle field sets; the direct-commit
admission frame; queued-row layout accessors; the string-aware comment
stripper shared by every JS harness suite.

Refs #981 #964

* fix(session): sweep handoff fixes to their sibling surfaces

The interactive replay loop treated a system row as a tool-batch
boundary, so every tool result after an interleaved row vanished from
that pane while the coordinator rendered the same history correctly.
Only a conversational turn ends the batch window now, matching the
shared outcome index.

Accepted user turns clear the composer's attachment chips on the same
viewer policy that settles optimistic bubbles rather than on having
matched a local bubble, so a workstream created with an upload no
longer keeps a chip for an attachment the create dispatch already
consumed. The coordinator's raced-Stop arm emits the stream-end hook it
inherits alongside the idle state, leaving no unfinalized bubble or
unflushed tool output. Ending a session surfaces a failure toast when
the request never lands or answers with a non-JSON body.

The per-second persistence reconcile now probes each session without
blocking: a workstream whose generation and handoff locks are held is
skipped until the next pass instead of contending the locks every
commit needs. The one-shot repair that gates workstream creation at
capacity keeps a definite probe — it has no next pass, and the sessions
likeliest to be contended are the ones whose unresolved journals
emptied its candidate list.

Single-sourced: the attachment lane builds its conversation row through
the shared commit-identity builder; the ordinary worker exit releases
its slot through the lifecycle owner; both operator surfaces snapshot
their counters through one non-consuming helper; the replay preamble
loses its per-kind wrappers and its config hook; the browser harness
suites share one brace walker; and each in-flight history attempt is
one record carrying both its abort controller and its deadline.

Refs #981 #964
2026-08-11 04:18:36 -07:00
Patrick Buckley f4fd7e1f67 fix(security): classify outbound addresses by what they reach (GHSA-wm4f-79pw-pfr9) (#1003)
* fix(security): classify outbound addresses by what they reach (GHSA-wm4f-79pw-pfr9)

Five guards screened outbound URLs and each hand-rolled its own address
normalization and policy tests, so each had a different hole. An IPv6
transition address carries an IPv4 destination in its low bits and
`ipaddress` classifies the wrapper, not the destination: 64:ff9b::a9fe:a9fe
reports is_global because 64:ff9b::/96 is global unicast, while a NAT64
gateway routes it to the cloud metadata endpoint. CGNAT (100.64.0.0/10) is
neither is_private nor is_global, so a denylist built on is_private missed
it with no gateway involved at all.

Add turnstone/core/ip_classify.py as the single classifier. One function
returns exactly one policy lane — PUBLIC, PRIVATE (operator-approvable) or
NEVER — and every guard branches on the lane rather than re-deriving it.
Two overlapping booleans would make a verdict depend on which one a caller
tested first; several addresses are simultaneously globally routable and
metadata-reaching.

- Decode transition addresses per RFC 6052 §2.2 (NAT64 well-known and
  local-use prefixes, 6to4, Teredo, IPv4-mapped, IPv4-compatible) and judge
  them by the IPv4 they reach. The local-use prefix does not say which
  layout its gateway uses, so every length it can carry is decoded and the
  worst result classified.
- Share hostname resolution too. The five copies had already drifted on
  which failures they caught, and getaddrinfo raises UnicodeError — not an
  OSError — from the IDNA encoder.
- Resolution failure is a refusal, not a pass: the fetch resolves again, so
  an authority answering the guard with SERVFAIL and the fetch with an
  internal address would otherwise switch the guard off for that hop.
- Screen every redirect hop in every mode. allow_private_origin widens which
  lanes are acceptable rather than turning screening off, and the permission
  is revoked after any hop that is not wholly private.
- Cleartext http is allowed only for a hostname that RESOLVES to loopback.
  *.localhost is ordinary DNS, and trusting the name put an OIDC token
  exchange on the wire in the clear.
- Screen doctor and console-probe URLs through the classifier. Both used a
  host.startswith("169.254.") string test that never resolved, so any DNS
  name pointing at the metadata service passed and its body was returned to
  the model.
- Add known vendor metadata prefixes the stdlib does not flag, and place
  deprecated IPv6 site-local outside the public lane.

The operator's private-network opt-in still admits the whole home lab,
including IPv6 loopback, CGNAT and split-horizon hosts. Metadata,
link-local, multicast, unspecified and reserved addresses stay refused
regardless of the opt-in, including as a redirect target from an approved
private origin — the settings help and docs now say so.

Reported by @tonghuaroot.

* fix(security): close Azure/Oracle metadata gap and restore dual-stack origins

Review follow-ups on the address-classification rework.

Azure's host-agent endpoint (168.63.129.16) and Oracle Cloud's metadata
endpoint (192.0.0.192) sit in ordinary unicast space, so the stdlib reported
them as globally routable and both classified PUBLIC — reachable with no
opt-in at all, a worse position than the RFC 1918 host beside them, and
directly contradicting the "metadata stays refused even with the opt-in"
guarantee the settings help and docs now advertise. Both join the shared
vendor list.

Revoking the private-hop permission on the ORIGIN hop broke the case
`_screen_tool_url` deliberately admits: a dual-stack or split-horizon
home-lab host answering with both a LAN and a public record was approved,
then refused on its own `302 /login` — one hop was all it ever got. Track
the approved HOST instead, so redirects that stay on it remain covered while
a redirect to any other private host is still refused once the chain is no
longer wholly private.

Also:

- Try several registry candidates for the collector-scope probe instead of
  abandoning it when the first is unresolvable, which also stopped a healthy
  registry from logging as malformed.
- Bound the probe's name resolution with an explicit timeout matching the
  2s the httpx connect deadline used to provide; it runs before the console
  lifespan yields and getaddrinfo has no timeout of its own.
- Route doctor and the console probe through `web.screen_url` rather than
  keeping a third and fourth copy of parse/resolve/classify/fold, which had
  already diverged on default port and empty-hostname wording. An empty
  hostname no longer reports as a cloud-metadata refusal.
- Give `screen_url` a scheme-aware default port.
- Stop doubling the word "hostname" in the OAuth resolution refusal.
- Correct the `_screen_tool_url` docstring: it described `private_origin` as
  requiring every record to be private, which the mixed-record decision
  reversed, and `private_block` as a property of a refusal when it reports
  the lane on the success path too.
- Make the preview tests' screening stub opt-in rather than autouse — as a
  module-wide fixture it also stubbed the tests whose subject IS the screen,
  so one of them would have passed even if screening refused everything.
  Verified the module now passes with all name resolution blocked.

* fix(security): refuse mixed-record private origins instead of exempting them

The previous commit let an approved private origin redirect to itself by
exempting its hostname from the chain-wide revocation. That exemption was
wrong three ways: it was captured once and never cleared, so a public hop
could steer the fetcher back into the approved host at a path of its
choosing — reopening the private -> public -> private bypass; it was
re-entrant across same-host redirects with fresh DNS each time, so a
self-redirecting host could walk arbitrary internal addresses; and it
matched on bare hostname, so it spanned every port on the approved box.
All three were reproduced against the parent commit, which refuses them.

Delete the exemption rather than repair it. The case it existed for — a
dual-stack host answering with both a LAN and a public record — is now
refused where it is actually decidable, in `_screen_tool_url`, with the
remedy in the message: point the tool at the LAN address directly. A
granted chain therefore always starts wholly private, so the fetch guard
needs no notion of an approved host and stays one unconditional rule.

That the accommodation could not be expressed safely in the guard is the
signal: the connection may land on either record, so approving such a host
never described where the fetch would go.

Also from the same review:

- Walk the whole service registry for a collector-scope probe candidate
  instead of the first three, and split the outcome into three log lines,
  so entries that are merely unreachable stop raising the malformed-registry
  alarm and skipping the boot check cluster-wide.
- Stop the candidate walk on a resolver timeout. `asyncio.timeout` bounds
  the await, not the work, so continuing left one parked thread per timed-out
  candidate on the shared executor.
- Move the metadata-hostname denylist into `ip_classify` and enforce it in
  `screen_url`, so doctor and the console probe inherit it instead of each
  keeping a copy.
- Drop the scheme-aware default port: a numeric service does not change
  which addresses resolution returns, and classification reads only those.
  `parsed.port` is still touched so an out-of-range value refuses.
- Correct the vendor-metadata comment, which generalized a claim true of
  Azure's and Oracle's addresses to Alibaba's CGNAT one.
- Rename a test class that was still named for the rule it no longer tests.
2026-08-11 02:18:03 -07:00
renovate[bot] cbce6a16a6 chore(deps): update ghcr.io/astral-sh/uv docker tag to v0.12.3 2026-08-10 21:05:02 -07:00
github-actions[bot] 9211c4fb29 chore: download vendored JS files 2026-08-10 21:04:25 -07:00
renovate[bot] 8e9df1fc1b chore(deps): update dependency katex to v0.18.4 2026-08-10 21:04:25 -07:00
renovate[bot] ca6db54bcc chore(deps): lock file maintenance 2026-08-10 18:56:19 -07:00
Patrick Buckley 766223e774 feat(judge): parallelize batch evaluations (#991) 2026-08-08 23:56:15 -07:00
Patrick Buckley 98e96ab5f3 Add per-alias model concurrency admission (#990)
* feat(models): add per-alias concurrency admission

Add registry-backed FIFO admission limits with queue-aware deadlines and full-stream leases. Expose max_concurrency through storage, admin configuration, OpenAPI, documentation, and diagrams, with role and live backend count coverage.

* fix(api): omit null concurrency schema default

Keep max_concurrency optional for presence-keyed updates without advertising a null default for its non-null integer OpenAPI shape.
2026-08-08 22:01:04 -07:00
Patrick Buckley 7a06f5e8bc refactor(session): make ModelLane the provider boundary (#979) (#989)
* refactor(session): make ModelLane the provider boundary (#979)

## Summary

This closes the model-lane ownership gap left by #832: `ChatSession` no longer stores raw provider/client handles. `ResolvedModelBinding` now carries the provider, client, model, capabilities, registry generation, and backend-auth configuration as one coherent snapshot.

- Atomically rebind existing sessions after model-registry changes while pinning each in-flight send, fallback, judge, output guard, task agent, title, compaction, perception, and voice operation to its initiating principal and binding.
- Fence UI publication, canonical trajectory folds, durable writes, streams, retries, child scopes, and judge work by generation. Stop can hand off to a successor without accepting late state; cancelled tools retain typed effect receipts, and concurrent approval batches resolve by exact cycle or call.
- Make create, fork, open, close, and delete race-safe with hidden `creating` reservations, incarnation-aware state tails, and an ACL-rechecked transaction that clones checkpoint-bounded history, configuration, project/persona state, and attachment references.
- Extend REST/OpenAPI and Python/TypeScript SDK contracts for create/fork inputs, routed-create metadata, live-workstream probes, targeted approvals, and structured cancellation results.
- Update architecture, storage, authentication, judge, channel, console, API, and SDK documentation, including regenerated architecture diagrams and OpenAPI artifacts.

## Validation

- SQLite suite: 11,188 passed, 9 skipped, 10 deselected
- PostgreSQL suite: 11,195 passed, 2 skipped, 10 deselected
- Live backend: 3 passed
- SSE recovery: 6 passed; browser recovery harness passed all scenarios
- Ruff: clean; 595 files correctly formatted
- mypy: 243 source files clean
- TypeScript: typecheck/build and 35 tests passed
- OpenAPI artifacts fresh; all 14 changed diagrams reproduce byte-for-byte
- `git diff --check` and Git LFS integrity clean

Closes #979.

* fix(deps): update nanoid for GHSA-2v37-7h3g-55p8

Refresh the transitive lock entry admitted by PostCSS so the TypeScript security gate no longer resolves the vulnerable custom-generator implementation.

Validation:
- npm ci
- npm audit --audit-level=moderate: 0 vulnerabilities
- TypeScript typecheck and build
- TypeScript tests: 35 passed

* fix(test): assert canonical model registry URLs

Replace prefix checks with exact canonical base URL assertions so the tests do not model incomplete URL validation.

Validation: tests/test_model_registry.py (185 passed); Ruff check/format; mypy.
2026-08-08 16:13:35 -07:00
394 changed files with 80957 additions and 11584 deletions
+3
View File
@@ -42,6 +42,9 @@
# CONSOLE_HTTPS_PORT=8443 # Caddy (dashboard HTTPS)
# POSTGRES_PORT=5432 # exposed for bare-metal host joins
# POSTGRES_BIND=127.0.0.1 # set 0.0.0.0 to let another machine join
# TURNSTONE_HOST_IP=127.0.0.1 # dev-stack bind address for cross-host joins
# TURNSTONE_CONSOLE_HTTP_BIND=127.0.0.1 # production TLS-overlay ACME/API bind
# TURNSTONE_ACME_EXTERNAL_URL=http://192.0.2.1:8090/acme # routable ACME base; bracket IPv6; include /acme
# -- Workspace ----------------------------------------------------------------
# Bind-mount a host directory the model can read/write at /workspace:
+1 -1
View File
@@ -55,7 +55,7 @@
{
"description": "LLM SDKs — always review manually",
"groupName": "LLM SDKs",
"matchPackageNames": ["openai", "anthropic", "mcp"],
"matchPackageNames": ["openai", "httpx2", "anthropic", "mcp"],
"schedule": ["before 9am on Monday"],
"automerge": false
},
+3 -1
View File
@@ -19,6 +19,8 @@ docker-compose.override.yml
.ruff_cache/
.pytest_cache/
*.db
*.db-shm
*.db-wal
.plan.md
.plan-*.md
.hypothesis/
@@ -29,4 +31,4 @@ tools/skill_audit_analysis/output/
design_ideas/
.claude/
docs/design/
/.idea
/.idea
+45
View File
@@ -18,6 +18,12 @@ Earlier stable lines (`stable/1.6`, `stable/1.5`) are frozen.
### Added
- **Large plain-text pastes become attachments.** Pasting text longer than the
fixed 2,000-character threshold stages `pasted-text.txt` across all five
attachment-capable create/send composers. Clipboard files retain priority,
text above the 512 KiB upload ceiling stays inline, identical synthesized
pastes collapse to one chip, and rejected attachment sends keep the staged
message and files so they can be corrected or retried.
- **`server_parses_reasoning` model capability.** Declare it on a model
definition whose backend segregates reasoning into its own channel (a
vLLM launched with a reasoning parser, a commercial provider): the
@@ -152,6 +158,35 @@ Earlier stable lines (`stable/1.6`, `stable/1.5`) are frozen.
### Changed
- **lacme 1.2.0 and HTTPX2 now back the core mTLS path.** TLS/ACME is a typed
core dependency rather than optional-import-era code; renewal and admin
clients use lacme's public close/stop lifecycle. Consoles can set
`TURNSTONE_ACME_EXTERNAL_URL` to a routable responder base ending in `/acme`,
so nodes on another host receive usable directory and follow-up URLs during
enrollment. Signing routes now require a dedicated, purpose-confined rotating
service JWT, and HTTPX2 pins that credential to canonical resources on
configured responder origins. Internal and external console identities use
separate persistence namespaces; cluster identities are reused only after
key, SAN, validity, EKU, and active-root verification. Responder-side keyless
CSR results cannot overwrite managed identities, failed live reloads restore
the last usable bundle, certificate lifetime stays at 48 hours with a
12-hour renewal cadence, and cancellation drains renewal clients before
propagating. Operator-supplied IPv4 and IPv6 literals are passed to lacme as
typed IP identifiers and issued as IP SANs; unexpired legacy certificates
containing `DNS:<ip>` are reissued instead of being reused as an invalid IP
identity
([#1011](https://github.com/turnstonelabs/turnstone/issues/1011)).
- **OpenAI SDK v3 and its HTTPX2 default transport are now supported (#1009).**
Chat Completions and Responses streams normalize native HTTPX2 connection
deaths through the same retry boundary as legacy HTTPX-backed providers,
including failures observed after safe cross-thread client closure during a
model-registry reload. The OpenAI v3 runtime escape hatch for explicitly
injected legacy HTTPX clients remains supported. OpenAI connections now
follow HTTPX2's operating-system trust store by default; deployments that
relied on a modified `certifi` bundle must install that CA in the system
store or set `SSL_CERT_FILE` / `SSL_CERT_DIR`.
- **Log event rename: `drain_stream.post_finish_blip` is now
`stream.post_finish_blip`; its `usage_captured` field is retained.** The
single-shot drain normalizes mid-body transport deaths through the same
@@ -217,6 +252,16 @@ Earlier stable lines (`stable/1.6`, `stable/1.5`) are frozen.
### Fixed
- **MCP servers may expose resources without resource templates, or templates
without concrete resources (#993).** The MCP handshake has one aggregate
`resources` capability, but implementations are not required to support both
list methods. An exact JSON-RPC `Method not found` response from either
method is now treated as an empty half-catalog during static and per-user
discovery and refresh, while every other error still fails closed. A failed
static registration also tears down its transport and publishes none of its
staged tools/resources/prompts, eliminating the live callable “ghost” tools
that could remain after the UI reported registration failure.
- **A cancelled judge, guard, or compaction call can now stop before its
request goes out (#972).** Previously it could not: `model_turn` refused
to *re-issue* an abandoned call after a mid-stream death, but nothing
+1 -1
View File
@@ -8,7 +8,7 @@ FROM python:3.14-slim
LABEL org.opencontainers.image.title="turnstone" \
org.opencontainers.image.description="Multi-node AI orchestration platform"
COPY --from=ghcr.io/astral-sh/uv:0.12.1 /uv /usr/local/bin/uv
COPY --from=ghcr.io/astral-sh/uv:0.12.3 /uv /usr/local/bin/uv
# Remove the slim image's man page exclusion so man-db has actual content
RUN rm -f /etc/dpkg/dpkg.cfg.d/docker
+14 -5
View File
@@ -53,8 +53,9 @@
# The node registers in Postgres, auto-enrolls its mTLS cert from the console's
# ACME endpoint (when the cluster runs mTLS), and the console collector reaches
# it back via host.docker.internal. To join from ANOTHER machine, set
# TURNSTONE_HOST_IP to this host's LAN IP and use it in the URLs above (and the
# node's TURNSTONE_ADVERTISE_URL = the NODE host's IP) — see docs/docker.md.
# TURNSTONE_HOST_IP to this host's LAN IP and set TURNSTONE_ACME_EXTERNAL_URL to
# http://<this-host-ip>:8090/acme. Use the same host IP in the node's URLs above
# (and the NODE host's IP in TURNSTONE_ADVERTISE_URL) — see docs/docker.md.
# =============================================================================
name: turnstone
@@ -154,9 +155,11 @@ services:
# Publishes the console's plain-HTTP listener so a bare-metal node can reach
# the ACME endpoint, fetch the CA, and enroll its cert (the console serves
# HTTP here even under mTLS). Bound to 127.0.0.1 by default; setting
# TURNSTONE_HOST_IP exposes the WHOLE console HTTP API — including the
# cert-issuing ACME endpoint — on that interface, so the JWT secret's
# strength is the only gate. Browsers use Caddy :8443, never this port.
# TURNSTONE_HOST_IP exposes the WHOLE console HTTP API on that interface.
# ACME signing routes require a dedicated short-lived service JWT, but the
# listener and bearer token are still plain HTTP: bind only a trusted LAN
# or VPN interface and restrict it to enrolling nodes. Browsers use Caddy
# :8443, never this port.
ports:
- "${TURNSTONE_HOST_IP:-127.0.0.1}:8090:8090"
environment:
@@ -164,6 +167,9 @@ services:
TURNSTONE_DB_BACKEND: *db-backend
TURNSTONE_DB_URL: *db-url
TURNSTONE_CONSOLE_URL: http://console:8090
# Separate from TURNSTONE_CONSOLE_URL: this is the canonical responder
# base embedded in ACME directory/order URLs for cross-host enrollment.
TURNSTONE_ACME_EXTERNAL_URL: "${TURNSTONE_ACME_EXTERNAL_URL:-}"
extra_hosts:
- "host.docker.internal:host-gateway"
networks:
@@ -305,6 +311,9 @@ services:
SKIP_PERMISSIONS: ${SKIP_PERMISSIONS:-}
TURNSTONE_NODE_ID: node-1
TURNSTONE_ADVERTISE_URL: http://node-1:8080
# Lets the authenticated ACME client follow the console's canonical LAN
# URLs without trusting destinations learned from the public directory.
TURNSTONE_ACME_EXTERNAL_URL: "${TURNSTONE_ACME_EXTERNAL_URL:-}"
extra_hosts:
- "host.docker.internal:host-gateway"
networks:
+9 -5
View File
@@ -38,10 +38,18 @@ services:
condition: service_completed_successfully
volumes:
- tls-certs:/certs:ro
# The production base keeps :8090 private. The TLS overlay publishes it on
# localhost for same-host enrollment; use a trusted LAN/VPN address for a
# remote node and firewall it to that node.
ports:
- "${TURNSTONE_CONSOLE_HTTP_BIND:-127.0.0.1}:8090:8090"
environment:
TURNSTONE_TLS_ENABLED: "true"
TURNSTONE_TLS_SANS: "console"
TURNSTONE_CONSOLE_URL: "http://console:8090"
# Canonical ACME responder base advertised to enrolling nodes. Set this
# when they reach the console through a different host/address.
TURNSTONE_ACME_EXTERNAL_URL: "${TURNSTONE_ACME_EXTERNAL_URL:-}"
command:
- turnstone-console
- --host=0.0.0.0
@@ -58,11 +66,7 @@ services:
environment:
TURNSTONE_TLS_ENABLED: "true"
TURNSTONE_TLS_SANS: "server"
# Disable healthcheck — server serves HTTPS with mTLS which the
# stdlib healthcheck script can't satisfy. The base compose
# healthcheck uses plain HTTP which won't work on an HTTPS listener.
healthcheck:
disable: true
TURNSTONE_ACME_EXTERNAL_URL: "${TURNSTONE_ACME_EXTERNAL_URL:-}"
# Channel: TLS
channel:
+23 -8
View File
@@ -15,11 +15,14 @@ counterpart to the quick `turnstone-server …` invocation in
## Cluster-side prerequisite
The compose stack must publish Postgres, the console's ACME endpoint, and SearxNG
on an address the bare-metal host can reach. Start it with `TURNSTONE_HOST_IP`
set to the compose host's LAN IP (default `127.0.0.1` keeps everything host-local):
on an address the bare-metal host can reach. Use a trusted LAN or VPN interface,
firewall it to the joining node, and advertise the same reachable ACME endpoint
(default `127.0.0.1` keeps everything host-local):
```bash
TURNSTONE_HOST_IP=<compose-host-ip> docker compose up -d
TURNSTONE_HOST_IP=<compose-host-ip> \
TURNSTONE_ACME_EXTERNAL_URL=http://<compose-host-ip>:8090/acme \
docker compose up -d
```
## Install (run as root on the bare-metal host)
@@ -65,8 +68,20 @@ journalctl -u turnstone-server -f # watch it register + (if the cluster
shared settings (the database). If the cluster runs mTLS, the node auto-enrolls a
cert from the console's ACME endpoint and re-advertises itself over `https://`.
> **mTLS + cross-host caveat:** a node on a *different* host than the console
> currently can't complete ACME enrollment — the console advertises an
> unroutable in-container address in its ACME directory
> ([turnstonelabs/lacme#22](https://github.com/turnstonelabs/lacme/issues/22)).
> Same-host bare-metal nodes, and any node in a non-mTLS cluster, are unaffected.
For a node on a different host, `TURNSTONE_ACME_EXTERNAL_URL` is required on the
console and should also be set in the node drop-in. It is the full, externally
reachable responder base
(including `/acme`) that the console embeds in the ACME protocol's follow-up
URLs and that the node trusts as an enrollment-credential destination. The
node's `TURNSTONE_CONSOLE_URL` should point at the same host and port, without
the `/acme` suffix.
For mTLS, `TURNSTONE_ADVERTISE_URL` may use a resolvable DNS hostname or a
literal IP address. Turnstone enrolls literals as IP SANs. Bracket IPv6 literals
inside URLs, for example `http://[2001:db8::10]:8080`; do not use wildcard,
unspecified, or scoped addresses as certificate identities. Restart the node
after changing its advertised identity so it enrolls a matching certificate.
The dedicated service JWT authenticates enrollment but the direct `:8090`
bootstrap is still plain HTTP/TOFU. Use HTTPS through an independently trusted
proxy when the network itself is not trusted.
@@ -4,16 +4,20 @@
# they live here; the JWT secret + DB URL live in /etc/turnstone/config.toml.
#
# Addresses below use RFC 5737 documentation IPs — replace them:
# <this-host> = the bare-metal host's own LAN IP (what the console dials back)
# <this-host> = the bare-metal host's own reachable DNS name or IP address
# (its mTLS identity and the address peers dial)
# <compose-host> = the host running the cluster / docker-compose stack, started
# with TURNSTONE_HOST_IP=<compose-host> so :8090 and :8081 are
# published on its LAN interface (see docs/docker.md).
# with TURNSTONE_HOST_IP=<compose-host> and
# TURNSTONE_ACME_EXTERNAL_URL=http://<compose-host>:8090/acme
# so enrollment links and published ports are reachable
# (see docs/docker.md).
[Service]
# Unique node id (defaults to the hostname if unset).
Environment=TURNSTONE_NODE_ID=host-1
# The address peers + the console collector dial back. Auto-upgrades to https://
# once the node enrolls its mTLS cert.
# once the node enrolls its mTLS cert. IPv6 literals require URL brackets, for
# example http://[2001:db8::10]:8080.
Environment=TURNSTONE_ADVERTISE_URL=http://192.0.2.10:8080
# The cluster console's reachable plain-HTTP ACME/API endpoint. A bare-metal node
@@ -21,5 +25,9 @@ Environment=TURNSTONE_ADVERTISE_URL=http://192.0.2.10:8080
# port; turnstone-server honors this for cert enrollment.
Environment=TURNSTONE_CONSOLE_URL=http://192.0.2.1:8090
# Trusted canonical responder base. This pins where the node may send its
# enrollment JWT; it must match the console-side value (a literal IP is fine).
Environment=TURNSTONE_ACME_EXTERNAL_URL=http://192.0.2.1:8090/acme
# The cluster's published SearxNG, for the web_search tool.
Environment=TURNSTONE_SEARXNG_URL=http://192.0.2.1:8081
+545 -135
View File
File diff suppressed because it is too large Load Diff
+641 -196
View File
File diff suppressed because it is too large Load Diff
+22 -16
View File
@@ -195,13 +195,17 @@ both and the gateway hosts both adapters in one process.
- All subsequent messages in the thread are routed to the same workstream.
- The bot streams responses via message edits, updated approximately every
1.5 seconds.
- If the workstream is evicted for capacity, the next message in the
thread auto-creates a new workstream and atomically resumes the
previous workstream via the `resume_ws` field on
`CreateWorkstreamMessage`. The server resumes the workstream during
creation (same HTTP request), and the server emits a
`WorkstreamResumedEvent` back to the channel. The thread receives a
*"Resumed: {name} ({count} messages restored)"* confirmation.
- If a persisted channel route is no longer active on its owning node, the
router asks the create endpoint to fork the old workstream into a new ID via
`resume_ws`. The saved source can still resolve normally; its
checkpoint-bounded history, configuration, persona, effective project, and
attachment references are cloned before the channel route is repointed. The
old route remains durable until the replacement (and any initial message)
succeeds. If the create endpoint returns the ordinary
source-not-found response *and* a fresh authoritative storage lookup confirms
that the source is gone, the router retries once without `resume_ws` and
starts a fresh conversation. Other access, conflict, routing, and storage
failures remain visible rather than silently discarding history.
### Slash Commands
@@ -284,15 +288,17 @@ See [Security: Database Schema](security.md#database-schema) for the
`channel_routes` table.
2. **Active** — messages are routed bidirectionally. The bot streams
responses via message edits (updated every ~1.5 seconds).
3. **Eviction** — the server evicts an idle workstream for capacity. The
route is preserved and the thread stays open.
4. **Reactivation** — the next message in the thread detects the stale
route and creates a new workstream with the old `ws_id`
as `resume_ws` on the creation request. The server resumes
the workstream during creation (no separate command or reverse lookup
needed). The channel receives a `WorkstreamResumedEvent`, and
the thread displays *"Resumed: {name} ({count} messages restored)"*.
If the old workstream was pruned, a fresh one starts with no error.
3. **Eviction** — the server evicts an idle workstream for capacity. Its saved
source row and channel route remain durable, and the thread stays open.
4. **Reactivation** — the next message resolves the saved route and probes
whether that workstream is live on its owning node. If it is not, the router
creates a distinct workstream with the old `ws_id` as `resume_ws`. The
create response confirms the fork and message count; there is no separate
resume command or channel-specific resumed event. Only after the replacement
succeeds does the router swap the persisted route. If the source was deleted
or pruned, an exact source-not-found response plus a second authoritative
storage miss triggers one fresh-create retry; other fork failures leave the
old route intact and are surfaced normally.
5. **Close**`/close` command closes the workstream via HTTP, deletes the
route, unsubscribes from events, and archives the Discord thread.
+107 -36
View File
@@ -174,17 +174,32 @@ Request:
{
"node_id": "db-west-04",
"name": "perf-analysis",
"model": "gpt-5"
"model": "gpt-5",
"project_id": "proj_analytics",
"initial_message": "Profile the slow query"
}
```
All fields are optional:
- `node_id` — targeting mode:
- **omitted or `"auto"`** — console picks the reachable node with the most available capacity (max_ws - ws_total) and proxies the request to it.
- **`"pool"`** — console picks a reachable node with available capacity using round-robin selection.
- **`"pool"`** — compatibility alias for automatic placement on the reachable node with the most headroom.
- **specific node ID** — proxies the request to that node directly.
- `name` — workstream display name. Auto-generated if omitted.
- `model` — model alias from the target node's registry. Uses the node's default model if omitted.
- `judge_model` — optional judge-model alias for this workstream.
- `initial_message` — first message dispatched after the workstream is published.
- `skill` — enabled profile/skill to snapshot onto a fresh workstream.
- `persona` — enabled persona slug; empty uses the interactive default.
- `project_id` — project to attach, subject to the target node's membership gate.
- `resume_ws` — source ID to **fork** atomically into a new workstream. The
source remains unchanged; its checkpoint-bounded history, configuration,
persona, project, and attachment references are copied transactionally.
The endpoint also accepts the same multipart create shape as a node: one
JSON-encoded `meta` field plus up to ten `file` parts. Files require an
`initial_message` in the dashboard launcher. Files cannot be combined with
`resume_ws`; fork first and upload on the new workstream.
Response:
@@ -196,7 +211,19 @@ Response:
}
```
The response confirms the workstream creation request was proxied to the target node. A `ws_created` event on the cluster SSE stream confirms the workstream was actually created.
The response is returned only after the target node has durably published the
workstream. Its hidden `creating` reservation has already crossed to `idle`,
and the node emitted `ws_created` before any initial-message state event. The
cluster SSE event may therefore arrive before or after the HTTP response;
clients should reconcile both by the returned `correlation_id`/workstream ID
rather than treating them as two creates.
For safety, the console masks most target-node failures as the opaque `502`
shape `{"error":"Dispatch to node <node_id> failed"}` instead of reflecting
arbitrary node text or retry-triggering 401/429 responses. The coded
`server.require_project` refusal is the exception and remains a `400` with
actionable wording. Consult the target node's logs for the underlying create
correlation when a reachable node returns a masked 502.
### `GET /v1/api/cluster/events`
@@ -310,8 +337,8 @@ The auth system uses three scopes instead of the earlier read/full role model:
| Scope | Grants |
|-------|--------|
| `read` | Read-only access: dashboards, workstream lists, SSE streams, health |
| `write` | Send messages, create/close workstreams, approve tool calls |
| `approve` | Admin operations: manage users and API tokens |
| `write` | Non-approval mutations: send, create/open/close/delete, cancel, attachments, rewind, and retry |
| `approve` | Tool-approval and admin HTTP surfaces (with their additional RBAC permission checks) |
Scopes are cumulative — a user with `approve` scope can also perform `write` and `read` operations.
@@ -348,52 +375,87 @@ SSE streams (`/v1/api/workstreams/{ws_id}/events`, `/v1/api/events/global`) are
### Authentication
The proxy mints a short-lived (5-minute) JWT per request carrying the real user's `user_id`, `scopes`, and `permissions` with `aud: turnstone-server`. The user's console JWT (`aud: turnstone-console`) cannot be forwarded directly — it would be rejected by the server's audience validation — so the console re-signs a new server-audience JWT from the validated `AuthResult`. This preserves audit attribution (the upstream server sees the real user, not a service identity) and enforces scope narrowing as defense in depth (a read-only console user's proxied request carries only `read` scope). The JWT `src` claim is set to `"console-proxy"` for audit traceability. When no user context is available (auth disabled), the proxy falls back to a `ServiceTokenManager` with service identity `console-proxy`. The static `--auth-token` / `proxy_auth_token` is used as a final fallback.
The proxy mints a short-lived (5-minute) JWT per request carrying the real user's `user_id`, `scopes`, and `permissions` with `aud: turnstone-server`. The user's console JWT (`aud: turnstone-console`) cannot be forwarded directly — it would be rejected by the server's audience validation — so the console re-signs a new server-audience JWT from the validated `AuthResult`. This preserves audit attribution (the upstream server sees the real user, not a service identity) and enforces scope narrowing as defense in depth (a read-only console user's proxied request carries only `read` scope). Ordinary users are re-minted with `src="console-proxy"`; coordinator tokens retain `src="coordinator"` plus `coord_ws_id`, and only the validated console service identity with `service` scope retains `src="console"` for trusted owner forwarding. When no user context is available, the proxy falls back to a `ServiceTokenManager` identity `console-proxy` carrying `src="console"` and `{read, write, approve, service}` scopes. The static `--auth-token` / `proxy_auth_token` is used as a final fallback.
---
## Browser Dashboard
The web UI has five views, toggled client-side:
The console uses an L-shaped application shell: a collapsible navigation rail,
a tab bar, and a pane host. On mobile the rail becomes an off-canvas drawer.
The rail is fed by the cluster SSE snapshot and shows:
### 1. Cluster Overview (landing)
- state/count filters and the live compute-node list, including version drift;
- active coordinator and interactive workstreams, nested under their
coordinator parent and grouped by project when project metadata is visible;
- permission-filtered Manage groups that open the singleton Admin pane.
- **State cards** — 5 clickable cards (running, thinking, attention, idle, error) with count and colored top border. Clicking filters to that state.
- **Aggregate bar** — total tokens and tool calls across the cluster.
- **Node table** — columns: NODE, WS, RUN, ATTN, TOKENS, VER, LOAD. Sorted by activity. Clickable rows drill down to node detail. Version column shows per-node version; hidden on mobile.
- **Version drift indicator** — when nodes report different versions, the status bar shows a yellow "DRIFT" warning with a tooltip listing all versions. Node groups show "mixed" with a yellow badge when their members disagree.
- **"+ new" button** — opens the workstream creation modal (see below).
Coordinator and interactive conversations open as tabs inside the same shell.
Interactive panes use the owning node's console proxy, so users do not need
direct network access to compute-node ports. Split-right and split-down actions
can display several panes at once. Closing a pane removes only that tab; use the
pane menu's explicit close or delete action to change the workstream lifecycle.
### 2. Node Drill-down
### Dashboard pane
Breadcrumb: `Cluster > db-west-04`. Shows the node's workstreams in a table matching the per-node dashboard layout (STATE, NAME, MODEL, NODE, TASK, TOKENS, CTX) with activity sub-lines. Includes a link to the node's proxied server UI.
The home view is coordinator-first. It contains the persistent workstream
launcher plus the saved-sessions list. Selecting a state count opens the
filtered workstream table inside the same Dashboard pane; selecting a compute
node opens its proxied node surface. Cluster SSE updates keep rail state,
workstream rows, and tab state glyphs synchronized.
**Proxy deep-linking:** Clicking a workstream row opens the node's server UI in a new tab via the proxy at `/node/{node_id}/?ws_id=<id>`, which auto-selects that workstream. Users do not need direct network access to the server node.
### Workstream launcher
### 3. Filtered Workstreams
The landing-page composer starts a workstream with an optional initial task and
attachments. When the caller can create both kinds, a Coordinator / Interactive
toggle selects the target kind. Its options include:
Breadcrumb: `Cluster > Running` or `Cluster > db-west-04`. Server-side paginated workstream table. NODE column values are clickable to filter further. Pagination controls at bottom. Workstream rows use proxy deep-links.
### 4. Workstream Creation Modal
Triggered by the "+ new" header button. A modal dialog with:
- **Node selector** — dropdown with three targeting modes: "Auto (best available)" picks the node with the most headroom, "General pool (any node)" picks a node with available capacity using round-robin, or a specific node from the list (showing capacity).
- **Node placement** — "Least loaded" picks the reachable node with the most
headroom, or "Specific node" pins the create to a node from the live list.
- **Persona** — optional dropdown listing the enabled personas for the workstream kind. Sets the system-message composition and capability envelope at creation, snapshotted server-side; empty uses the kind's default. Picking one requires no `persona.*` permission.
- **Profile** — optional dropdown listing enabled skills. Applies the skill's model, auto-approve policy, token budget, and other behavioral settings at creation time.
- **Skill** — optional dropdown listing enabled skills. Applies the skill's model, auto-approve policy, token budget, and other behavioral settings at creation time.
- **Project** — optional project filing. Private projects require owner/member access. A coordinator child inherits its parent's project unless explicitly routed to another attachable project.
- **Name** — optional text input. Auto-generated if left empty.
- **Model** — optional text input for a model alias from the target node's registry.
- **Judge Model** — optional text input for the judge model alias (overrides the default judge model for this workstream).
- **Model** — optional selector populated from the target model registry.
- **Judge Model** — optional selector for the judge alias (overrides the default
judge model for this workstream).
Keyboard shortcuts: Ctrl+Shift+R (refresh title), Ctrl+Shift+E (edit title), Ctrl+Shift+F (fork), Ctrl+Shift+X (delete). Press ? for full shortcut help.
Interactive launches additionally expose node strategy / node selection.
Submitting uses `POST /v1/api/cluster/workstreams/new`; coordinator launches use
the console's coordinator create surface. A toast confirms the committed
create, while SSE updates the dashboard and opens the resulting pane.
On submit, `POST /v1/api/cluster/workstreams/new` dispatches the creation request. A toast confirms success; the SSE stream delivers the `ws_created` event to update the dashboard.
Files require a non-empty initial task so the first turn consumes the staged
attachments. The console shell does not currently expose a fork action; use the
node's standalone workstream UI or the create API's `resume_ws` field.
All five views receive live updates via SSE — state cards update counts, node rows update metrics, workstream rows update state indicators.
### Large pasted text
The browser maintains a local `clusterState` object that mirrors the cluster snapshot. It is initialized from the SSE `snapshot` event on connect (or via `GET /v1/api/cluster/snapshot` on initial page load) and updated incrementally by SSE events. View navigation reads from local state — no API round-trips needed after the initial snapshot.
Browser composers turn plain text longer than 2,000 Unicode code points into a
`text/plain` attachment named `pasted-text.txt`. A paste exactly at the
threshold stays inline. This applies to the interactive and coordinator send
boxes, the console home launcher, and the node dashboard and new-workstream
composers.
### 5. Admin Panel
Clipboard files take priority over clipboard text. Text larger than the 512 KiB
attachment ceiling also stays inline, so the browser does not discard it before
a rejected upload. Attachments require a companion message and cannot be sent
as live-turn interjections; a busy composer preserves its message and chips for
an idle retry.
### Saved and filtered sessions
Saved coordinator and interactive sessions share one list with kind and persona
labels, filtering, pagination, and multi-select deletion. Opening a saved
coordinator rehydrates it in the console; opening a saved interactive session
resolves its node, calls `open`, and then connects the node-proxied pane.
The filtered live table carries STATE, NAME, MODEL, NODE, TASK, TOKENS, and CTX
columns. The browser maintains a local `clusterState` initialized from the
cluster snapshot and updated incrementally by SSE; the filtered view normally
renders from that state without another API round trip.
### Admin pane
Accessed via the "admin" button in the header (visible when authenticated
with `approve` scope). Provides user, API token, channel link, MCP server,
@@ -405,8 +467,14 @@ Audit tabs, and [Settings](settings.md) for the database-backed
configuration editor.
The **Channels** tab links users to either a Discord or Slack account
via a per-row channel-type selector. The **Models** tab is a CRUD
editor for `model_definitions`, the **Nodes** tab edits per-node
via a per-row channel-type selector. The **Models** tab is a CRUD
editor for `model_definitions`, including static and dynamic backend-auth
modes and a per-process **Max concurrent generations** limit for each alias
(`0` means unlimited). The limit is shared by every model-backed role using
that alias and a streaming generation holds its slot through the full decode.
Model edits rebind existing workstreams at their next send while
in-flight requests keep their original definition snapshot; see
[Settings](settings.md#model-definition-reloads) for the full contract. The **Nodes** tab edits per-node
metadata, and the **TLS** tab manages CA and leaf certificates for the
internal mTLS fabric. The **Settings** tab edits ConfigStore values
live; edits apply without restart.
@@ -504,7 +572,7 @@ Run history is automatically pruned (runs older than 90 days) approximately once
| Mode | Behavior |
|------|----------|
| `auto` | Picks the reachable node with the most available capacity |
| `pool` | Picks a reachable node with available capacity using round-robin |
| `pool` | Compatibility alias for the reachable node with the most headroom |
| `all` | Fan-out to all reachable nodes (capped at `max_fan_out`, default 20) |
| `<node_id>` | Targets a specific node by ID |
@@ -664,4 +732,7 @@ turnstone-server --port 8080
turnstone-console --port 8090
```
Open `http://localhost:8090` for the cluster dashboard. Create workstreams via the "+ new" button. Click any workstream to open the proxied server UI — no direct access to server ports required.
Open `http://localhost:8090` for the cluster dashboard. Create workstreams from
the persistent Dashboard launcher. Selecting a workstream opens a coordinator
or node-proxied interactive pane in the console shell — no direct access to
server ports is required.
+57 -15
View File
@@ -37,7 +37,7 @@ schema changes.
| # | Action | Operation |
|---|------------------------------|-------------------------------------------------------------|
| 1 | Create | `POST /v1/api/workstreams/new` |
| 2 | Subscribe to events | `GET /v1/api/workstreams/{ws_id}/events` (SSE) |
| 2 | Bootstrap history + subscribe | `GET .../history`, then `GET .../events` (SSE) |
| 3 | Send a user message | `POST /v1/api/workstreams/{ws_id}/send` |
| 4 | Inspect children | `GET /v1/api/workstreams/{ws_id}/children` |
| 5 | Inspect one workstream | `GET /v1/api/cluster/ws/{ws_id}/detail` |
@@ -91,14 +91,34 @@ subscribers (step 2) see the session warm up as token traffic starts.
---
## 2. Subscribe to the per-coordinator event stream
## 2. Bootstrap history, then subscribe to the event stream
Read and render history before opening the initial stream:
```http
GET /v1/api/workstreams/{ws_id}/events HTTP/1.1
GET /v1/api/workstreams/{ws_id}/history?limit=100 HTTP/1.1
Authorization: Bearer <token>
```
For a loaded coordinator, `messages` is the requested tail of one total
accepted conversation-row prefix: user, assistant, tool, and system rows,
including projected compaction checkpoints and cancellation-generated markers.
The response's optional `cursor` and `handoff_token` belong to that exact
render. Pass both once on the initial stream URL:
```http
GET /v1/api/workstreams/{ws_id}/events?last_event_id={cursor}&history_token={handoff_token}&user_turn=1&tool_turn=1 HTTP/1.1
Accept: text/event-stream
Authorization: Bearer <token>
```
Omit either query parameter when its history field is `null`. A handoff token
is opaque and process-local: do not parse, persist, or reuse it. Admission of a
later conversation row changes the token; durable acknowledgement does not. If
history returns `503 {"error":"History temporarily unavailable"}`, the response
is not authoritative: retain the current transcript, do not open a tokenless
replacement stream, and retry the read.
One persistent SSE connection per browser tab / SDK caller — the
console fans each event out to every listener queue (cap 500 events
per queue, put_nowait drop on overflow). Events come in flat JSON
@@ -110,10 +130,10 @@ with a `type` field. The recurring shapes a UI has to handle:
| `reasoning` | Reasoning-token stream chunk (when the model exposes it) | `text` |
| `content` | Assistant-content stream chunk | `text` |
| `stream_end` | End of a single provider stream | — |
| `tool_result` | A tool call completed (success or error) | `call_id`, `name`, `output`, `is_error?` |
| `tool_result` | A tool call completed; capable panes also receive the accepted-history replacement | `call_id`, `name`, `output`, `is_error?`, `accepted?`, `_event_id?`, `preview?`, `effect_status?` |
| `tool_output_chunk` | Streaming tool output (e.g. long bash command) | `call_id`, `chunk` |
| `approve_request` | One or more tool calls need operator approval | `items: [{call_id, header, preview, func_name, approval_label, needs_approval}]` |
| `approval_resolved` | Operator answered the approval prompt | `approved`, `feedback` |
| `approve_request` | One approval cycle needs operator action; several cycles may coexist | `cycle_id`, `items: [{call_id, header, preview, func_name, approval_label, needs_approval}]` |
| `approval_resolved` | One identified approval cycle was answered | `cycle_id`, `call_ids`, `approved`, `feedback`, `always` |
| `state_change` | Worker-thread state transition (also re-emitted with the current state on every fresh subscribe so refresh-mid-stream restores composer mode) | `state``running`, `thinking`, `attention`, `idle`, `error` |
| `in_progress_snapshot` | One-shot replay of the in-progress turn's content + reasoning when this client connects mid-stream | `content`, `reasoning` |
| `status` | Token usage + context-window snapshot (fires on every streaming tick) | `prompt_tokens`, `completion_tokens`, `total_tokens`, `context_window`, `pct`, `effort`, `cache_creation_tokens`, `cache_read_tokens` |
@@ -127,10 +147,11 @@ with a `type` field. The recurring shapes a UI has to handle:
| `wait_started` / `wait_progress` / `wait_ended` | `wait_for_workstream` tool lifecycle (see §6) | `call_id`, `ws_ids`, `elapsed`, `results`, `complete` |
| `batch_started` / `batch_ended` | `spawn_batch` / `close_all_children` tool lifecycle | `call_id`, `op`, `total`/`succeeded`/`denied`/`closed`/`failed`/`skipped` |
| `info` / `error` | Operational messages | `message` |
| `history_resync` | The rendered history token no longer names the accepted row prefix | `ws_id`, `reason` |
**Reconnection contract:** a freshly-opened SSE connection receives
the current snapshot of any pending tool approval (`approve_request`
is re-sent if unresolved), any in-flight `wait_*` / `batch_*`
one `approve_request` snapshot for every unresolved approval cycle, keyed by
the same stable `cycle_id`, plus any in-flight `wait_*` / `batch_*`
indicator, the worker's current `state_change`, and an
`in_progress_snapshot` carrying any partial content / reasoning the
model has produced for the in-progress turn — so a tab refresh
@@ -138,6 +159,11 @@ mid-approval, mid-tool-execution, or mid-stream restores both the
correct composer mode and the partial assistant text without waiting
for the response to complete.
`history_resync` is stronger than a numeric replay gap. The server closes that
stream; fetch and render `/history` again, then open a new stream with its new
cursor/token pair. The API and SDK expose these primitives but deliberately do
not choose a reconnect policy for callers.
---
## 3. Send the first user message
@@ -324,24 +350,35 @@ uses the cascade-mutation shape and how it differs from the
The `approve` endpoint is what resolves an `approve_request` SSE
event. The coordinator's worker thread is blocked inside
`ui.approve_tools` waiting for this POST.
`ui.approve_tools` waiting for this POST. Parallel task agents can leave
several approval cycles live at once, so current clients echo the event's
`cycle_id` (or a member `call_id`). A selector-less request resolves the oldest
cycle for compatibility.
```http
POST /v1/api/workstreams/{ws_id}/approve
{"approved": true, "feedback": null, "always": false}
{"approved": true, "feedback": null, "always": false, "cycle_id": "cycle_789"}
{"approved": false, "feedback": "spawn count looks too high — try 3 not 10"}
{"approved": true, "feedback": null, "always": true} // always-approve this tool name
{"approved": true, "feedback": null, "always": true} // remember this cycle's tool names
```
`cancel` drops the coordinator's in-flight generation and, for a
coordinator, auto-cascades the cancel to its direct children:
Success returns `{"status": "ok", "cycle_id": "cycle_789"}`. A stale selector
returns `409` with the currently oldest cycle/call IDs. `always` remembers only
the tool names in the cycle that actually resolved; it does not enable blanket
approval.
`cancel` requests cooperative cancellation of the coordinator's in-flight
generation and auto-cascades to its direct children:
`cancel_workstream` is dispatched through the routing proxy for
every direct child in the registry. The coordinator itself is left
idle and open for a fresh `send`:
every direct child in the registry. The HTTP acknowledgement is immediate;
the worker becomes idle after unwinding. Pass `{"force": true}` only to release
a wedged worker slot immediately. The coordinator itself remains open for a
fresh `send`:
```http
POST /v1/api/workstreams/{ws_id}/cancel
{}
{"status": "ok", "dropped": {}}
```
---
@@ -361,6 +398,11 @@ disconnect. The row is reopenable via
`POST /v1/api/workstreams/{ws_id}/open` so long as it hasn't been
deleted.
If any accepted live conversation row still requires persistence
reconciliation, close returns `409 {"error":"workstream has unresolved
persistence"}`. The coordinator remains loaded, its journal is retained, and
no history is discarded; retry after storage recovers.
---
## Further reading
+1 -1
View File
@@ -77,7 +77,7 @@ or MCP config can do adds to it. Current members:
| `delete_workstream` | wind-down | Hard-delete one child. Requires approval. |
| `list_nodes` | discover | Enumerate live cluster nodes + capabilities. |
| `skills` (action=find) | discover | Browse the skill catalog; opt-in `kind` filter narrows by audience. |
| `memory` | persist | Durable orchestration memory (`coordinator` scope, per-user — survives across coordinator sessions). |
| `memory` | persist | Durable acting-user orchestration memory (`coordinator`), plus shared memory when attached to a project. |
| `notify` | broadcast | Post a status update to a human channel at a narrative beat. |
| `tasks` | plan | Orchestrator-only scratchpad. Children don't see it. |
+11 -9
View File
@@ -13,7 +13,7 @@ cloud "LLM Providers" as llm {
component [OpenAI-compatible API\n(OpenAI, vLLM, llama.cpp)] as llm_openai
component [Anthropic Messages API] as llm_anthropic
}
database "SQLite\n(.turnstone.db)" as sqlite
database "SQLite / PostgreSQL\n(durable state)" as storage
' Turnstone System Boundary
package "Turnstone Platform" {
@@ -33,24 +33,26 @@ eval_user --> eval : Python API
' Internal connections
cli --> llm : LLM Provider API\n(via provider adapters)
cli --> sqlite : SQLite
cli --> storage : persistence
server --> llm : LLM Provider API\n(via provider adapters)
server --> sqlite : SQLite
server --> storage : persistence
eval --> llm : LLM Provider API\n(non-streaming)
eval --> sqlite : SQLite
eval --> storage : persistence
console --> server : HTTP proxy\n(hash-ring bucket lookup,\nproxy /node/{id}/* traffic)
console --> server : HTTP routing/UI proxy + cluster SSE\n(FNV-1a rendezvous placement,\nproxy /node/{id}/* traffic)
channel --> server : HTTP + SSE\n(POST /v1/api/workstreams/{ws_id}/send,\nGET /v1/api/workstreams/{ws_id}/events)
channel --> console : multi-node route/create/live/send/approve
channel --> server : direct mode + owning-node SSE\n(POST /v1/api/workstreams/{ws_id}/send,\nGET /v1/api/workstreams/{ws_id}/events)
' Notes
note right of console
Multi-node router:
- Hash-ring bucket lookup
- FNV-1a rendezvous placement
- Proxies create/send/approve
- Direct SSE from client to node
- HTTP polling for dashboard
- Collector aggregates node SSE
- Browser dashboard receives console SSE fanout
- /node/{id} proxies pane HTTP + SSE
end note
@enduml
+37 -10
View File
@@ -18,6 +18,7 @@ skinparam component {
package "Entry Points" <<Rectangle>> {
component [cli.py\nturnstone] as cli <<entry>>
component [server.py\nturnstone-server] as server <<entry>>
component [console/server.py\nturnstone-console] as consoleentry <<entry>>
component [eval.py\nturnstone-eval] as eval <<entry>>
component [admin.py\nturnstone-admin] as admin <<entry>>
component [bootstrap.py\nturnstone-bootstrap] as bootstrap <<entry>>
@@ -25,9 +26,16 @@ package "Entry Points" <<Rectangle>> {
' Core engine
package "turnstone/core/" <<Rectangle>> {
component [session.py\nChatSession, SessionUI] as session <<core>>
component [session.py\nChatSession, SessionUI\ngeneration-fenced turn loop] as session <<core>>
component [session_manager.py\nSessionManager\nshared lifecycle invariants] as sessionmanager <<core>>
component [adapters/\ninteractive + coordinator\nconstruction/event policies] as adapters <<core>>
component [model_turn.py\nModelLane, model_turn()\nlower / sample / re-ingest] as modelturn <<core>>
component [trajectory.py\ncanonical Turn IR] as trajectory <<core>>
component [lowering.py\nprovider-wire lowering] as lowering <<core>>
component [state_writer.py\nordered durable state tail] as statewriter <<core>>
component [model_backend_auth.py\nper-call backend credentials] as modelauth <<core>>
component [providers/\nLLMProvider, OpenAI, Anthropic, Google] as providers <<core>>
component [workstream.py\nWorkstreamManager] as workstream <<core>>
component [workstream.py\nWorkstream types + state] as workstream <<core>>
component [tools.py\nTool loader] as tools <<core>>
component [memory.py\nPersistence facade] as memory <<core>>
component [storage/\nStorageBackend protocol\nSQLite + PostgreSQL] as storage <<core>>
@@ -79,18 +87,18 @@ package "turnstone/api/" <<Rectangle>> {
package "turnstone/sdk/" <<Rectangle>> {
component [server.py\nTurnstoneServer (sync+async)] as sdkserver <<sdk>>
component [console.py\nTurnstoneConsole (sync+async)] as sdkconsole <<sdk>>
component [events.py\n27 SSE event types] as sdkevents <<sdk>>
component [events.py\nTyped SSE event stream] as sdkevents <<sdk>>
component [_base.py\nhttpx client base] as sdkbase <<sdk>>
}
' Tool schemas
package "turnstone/tools/" <<Rectangle>> {
component [*.json\n19 tool schemas] as schemas <<artifact>>
component [*.json\nBuilt-in tool schemas] as schemas <<artifact>>
}
' Entry point dependencies
cli --> session
cli --> workstream
cli --> sessionmanager
cli --> config
cli --> memory
cli --> colors
@@ -99,7 +107,8 @@ cli --> spinner
cli --> tools
server --> session
server --> workstream
server --> sessionmanager
server --> adapters
server --> config
server --> memory
server --> metrics
@@ -113,11 +122,26 @@ eval --> memory
eval --> config
eval --> tools
consoleentry --> sessionmanager
consoleentry --> adapters
consoleentry --> consoleserver
admin --> auth
bootstrap --> providers
' Core internal deps
session --> providers
sessionmanager --> workstream
sessionmanager --> adapters
sessionmanager --> storage
adapters --> session : constructs
session --> modelturn
session --> trajectory
session --> lowering
session --> statewriter
session --> modelauth
modelturn --> providers
modelturn --> trajectory
modelturn --> lowering
session --> tools
session --> memory
memory --> storage
@@ -129,6 +153,7 @@ session --> mcp : optional
session --> toolsearch : optional
session --> registry : optional
registry --> providers
modelturn --> registry : coherent snapshot
healthcheck --> metrics
mcp --> config
registry --> config
@@ -138,15 +163,17 @@ tools --> schemas
gateway --> discordbot
gateway --> slackbot
gateway --> router
discordbot --> sdkserver : HTTP + SSE
slackbot --> sdkserver : HTTP + SSE
discordbot --> sdkserver : direct HTTP + node SSE
slackbot --> sdkserver : direct HTTP + node SSE
router --> sdkserver : single-node/direct mode
router --> sdkconsole : multi-node route/create/live
router --> storage : channel_routes
' Console dependencies
consoleserver --> collector
consoleserver --> config
consoleserver --> auth
collector --> server : HTTP polling
collector --> server : discovery HTTP + cluster SSE aggregation
' API dependencies
serverspec --> openapi
+141 -25
View File
@@ -32,7 +32,7 @@ class "TerminalUI" as TerminalUI {
class "WorkstreamTerminalUI" as WsTermUI {
- _output_buffer: list[tuple]
- ws_id: str
- manager: WorkstreamManager
- manager: SessionManager
+ flush_buffer()
--
Buffers output when workstream
@@ -41,14 +41,14 @@ class "WorkstreamTerminalUI" as WsTermUI {
class "WebUI" as WebUI {
- _listeners: list[Queue]
- _approval_event: Event
- _approval_cycles: dict[str, ApprovalCycle]
- _ws_prompt_tokens: int
- _ws_tool_calls: dict
+ resolve_approval(approved, feedback)
+ resolve_approval(approved, feedback, cycle_id?, call_id?)
--
Enqueues JSON events for SSE.
Blocks on threading.Event for
approval.
Concurrent approval cycles each own
a threading.Event and result slot.
SSE handlers bridge Queue to
async via run_in_executor().
--
@@ -126,25 +126,76 @@ class "ModelCapabilities" as ModelCaps <<frozen>> {
+ supports_reasoning_replay: bool
}
class "ModelLane" as ModelLane <<frozen>> {
+ provider: LLMProvider
+ client: Any
+ model: str
+ alias: str
+ capabilities: ModelCapabilities
+ extra_params: dict | None
+ registry: ModelRegistry | None
+ admission: ModelAdmission | None
+ backend_auth_config: ModelConfig | None
+ backend_auth_resolver: Callable | None
}
class "ResolvedModelBinding" as ResolvedBinding <<frozen>> {
+ lane: ModelLane
+ config: ModelConfig | None
+ registry_generation: int
}
class "ModelTurnResult" as ModelTurnResult <<frozen>> {
+ turn: Turn
+ tool_calls: list[dict]
+ finish_reason: str
+ usage: UsageInfo | None
+ wire_msgs: list[dict] | None
+ producer: str
+ serving_model: str
}
class "model_turn()" as ModelTurnFn {
Turn IR → lower → provider stream
→ drain → canonical assistant Turn
--
core/model_turn.py
}
class "Backend auth resolver" as BackendAuth {
+ resolve_model_backend_auth_token(...)
--
Resolves static / Entra OBO /
Entra app / RFC 8693 per call.
Dynamic failure can fail closed.
--
core/model_backend_auth.py
}
' ChatSession
class "ChatSession" as ChatSession {
- client: Any
- provider: LLMProvider
- model: str
- _model_binding: ResolvedModelBinding
- _model_binding_lock: Lock
- ui: SessionUI
- messages: list[dict]
- messages: list[Turn]
- _msg_tokens: list[int]
- _ws_id: str
- _mcp_client: MCPClientManager | None
- _tool_search: ToolSearchManager | None
- _registry: ModelRegistry | None
- _generation: int
- _cancel_event: Event
- _durability_next_ticket: int
+ model_alias: str | None {property}
- _tools: list[dict]
- _task_tools: list[dict]
- _read_files: set[str]
- system_messages: list[dict]
--
+ send(user_input: str)
+ send(user_input: str, ..., acting_user_id: str | None)
+ cancel()
+ compact_now() → bool
+ fork_from_storage(source_ws_id, principal_id, ...)
+ handle_command(command: str)
+ resume(ws_id: str)
- _save_config()
@@ -162,8 +213,10 @@ class "ChatSession" as ChatSession {
- _rebuild_tool_search()
+ close()
- _run_agent(messages, tools, ...) → str
- _compact_messages(auto: bool)
- _full_messages() → list[dict]
- _compact_messages(auto: bool, my_generation: int)
- _commit_for_generation(generation, commit)
- _publish_for_generation(generation, publish)
- _full_messages() → list[Turn]
- _update_token_table(msg)
- _emit_state(state: str)
- _generate_title()
@@ -180,15 +233,36 @@ class "HeadlessSession" as HeadlessSession {
records all tool calls
}
' WorkstreamManager
class "WorkstreamManager" as WsMgr {
- _session_factory: Callable[[SessionUI], ChatSession]
' SessionManager
interface "SessionKindAdapter" as KindAdapter <<Protocol>> {
+ kind: WorkstreamKind
+ build_ui(ws) → SessionUI
+ build_session(ws, ...) → ChatSession
+ cleanup_ui(ws)
}
interface "SessionEventEmitter" as EventEmitter <<Protocol>> {
+ emit_created(ws)
+ emit_rehydrated(ws)
+ emit_state(ws, state)
+ emit_closed(ws_id, reason, name)
}
class "SessionManager" as SessionMgr {
- _adapter: SessionKindAdapter
- _storage: StorageBackend
- _workstreams: dict[str, Workstream]
- _pending_creates: dict[str, Workstream]
- _retiring_ids: set[str]
- _state_writer: StateWriter | None
- _order: list[str]
- _active_id: str
- _on_state_change: Callable
--
+ create(name, ui_factory) → Workstream
+ create(user_id, name, ..., defer_emit_created) → Workstream
+ commit_create(ws) → bool
+ discard(ws, ...) → bool
+ open(ws_id) → Workstream | None
+ delete(ws_id) → bool
+ close(ws_id)
+ get(ws_id) → Workstream
+ get_active() → Workstream
@@ -203,11 +277,17 @@ class "Workstream" as Ws <<dataclass>> {
+ id: str
+ name: str
+ state: WorkstreamState
+ session: ChatSession
+ ui: SessionUI
+ worker_thread: Thread
+ session: ChatSession | None
+ ui: SessionUI | None
+ worker_thread: Thread | None
+ error_message: str
+ last_active: float
+ kind: WorkstreamKind
+ user_id: str
+ parent_ws_id: str | None
+ project_id: str | None
- _fork_reservation_token: str
- _closed: bool
- _lock: Lock
}
@@ -278,12 +358,13 @@ class "ModelRegistry" as ModelReg {
- _models: dict[str, ModelConfig]
- _clients: dict[str, Any]
- _providers: dict[str, LLMProvider]
- _admissions: dict[str, ModelAdmission]
- _client_lock: Lock
+ default: str
+ fallback: list[str]
+ agent_model: str | None
--
+ resolve(alias) → (client, model, config)
+ resolve_binding(alias) → (client, model, config, provider, admission, generation)
+ get_client(alias) → Any
+ get_provider(alias) → LLMProvider
+ has_alias(alias) → bool
@@ -297,6 +378,22 @@ class "ModelRegistry" as ModelReg {
core/model_registry.py
}
class "ModelAdmission" as ModelAdmission {
- alias: str
- _limit: int
- _in_flight: int
- _waiters: deque
+ acquire(cancel_ref) → AdmissionLease
+ set_limit(limit)
+ snapshot() → AdmissionSnapshot
--
Per-process FIFO generation gate.
Stable across alias hot reloads;
queue time is deadline credit.
--
core/admission.py
}
class "ModelConfig" as ModelCfg <<frozen>> {
+ alias: str
+ provider: str
@@ -306,6 +403,10 @@ class "ModelConfig" as ModelCfg <<frozen>> {
+ temperature: float | None
+ max_tokens: int | None
+ reasoning_effort: str | None
+ max_concurrency: int
+ auth_mode: str
+ obo_audience: str
+ obo_scopes: str
}
' Circuit breaker state
@@ -375,22 +476,35 @@ LLMProvider <|.. AnthropicProv
OpenAIProv <|-- GoogleProv
ChatSession --> SessionUI : uses
ChatSession --> LLMProvider : delegates LLM calls
ChatSession --> ResolvedBinding : owns coherent snapshot
ChatSession --> ModelTurnFn : every model-backed role
ChatSession --> MCPMgr : optional
ChatSession --o ToolSearchMgr : _tool_search
ChatSession --> ModelReg : optional
ChatSession <|-- HeadlessSession
WsMgr --> "*" Ws : manages
SessionMgr --> "*" Ws : manages
SessionMgr --> KindAdapter : delegates construction
SessionMgr --> EventEmitter : lifecycle fan-out
Ws --> "1" ChatSession : wraps
Ws --> "1" SessionUI : wraps
Ws --> "1" WsState : has
WsMgr ..> ChatSession : creates via\nsession_factory(ui, model_alias)
KindAdapter ..> ChatSession : constructs
ModelReg --> "*" ModelCfg : holds
ModelReg --> "*" LLMProvider : caches
ModelReg --> "*" ModelAdmission : owns per alias
LLMProvider --> ModelCaps : returns
ModelReg --> ResolvedBinding : resolves atomically
ResolvedBinding --> ModelLane
ModelLane --> LLMProvider
ModelLane --> ModelCaps
ModelLane --> ModelCfg : auth/config snapshot
ModelLane --> ModelAdmission : admission lease
ModelTurnFn --> ModelLane
ModelTurnFn --> ModelTurnResult
ModelTurnFn ..> BackendAuth : per-call resolver
ChatSession --> HealthMon : checks circuit
HealthMon --> "1" CircuitState : has
@@ -403,7 +517,9 @@ note bottom of ChatSession
Provider-agnostic — delegates all LLM
communication to LLMProvider adapters.
core/session.py (~2700 lines)
Every live/durable publication is fenced by
its generation. Model calls use immutable lanes;
provider-wire mutation stays at lowering.
end note
@enduml
+158 -170
View File
@@ -1,183 +1,171 @@
@startuml
!theme plain
title Turnstone — Conversation Turn Lifecycle
title Turnstone — Generation-Fenced Conversation Turn
skinparam sequenceArrowThickness 1.5
skinparam sequenceLifeLineBackgroundColor #F5F5F5
participant "User /\nHTTP Client" as User
participant "ChatSession" as CS
participant "SessionUI" as UI
participant "LLMProvider\n(OpenAI / Anthropic)" as LLM
participant "Tool Executor\n(ThreadPool)" as TP
database "SQLite" as DB
participant "HTTP / CLI\ncaller" as User
participant "SessionManager" as Manager
participant "ChatSession" as Session
participant "SessionUIBase" as UI
participant "Accepted-row handoff\n(total live prefix)" as Handoff
participant "model_turn()\n+ lowering" as Plant
participant "ModelAdmission\n(per alias)" as Admission
participant "LLM provider" as Provider
participant "Tool workers" as Tools
database "StorageBackend\n(SQLite / PostgreSQL)" as Storage
== User Input ==
== Admission and generation claim ==
User -> CS : send(user_input)
activate CS
CS -> CS : messages.append({role: "user", content: input})
CS -> DB : save_message(ws_id, "user", input)
== LLM Call Loop ==
group loop [while tool_calls present]
CS -> UI : on_turn_start()
note right of UI
SessionUIBase resets the per-turn inflight
buffers (_ws_inflight_content / reasoning /
seq) that fuel the SSE in_progress_snapshot
event for mid-stream refresh resume.
end note
CS -> UI : on_state_change("thinking")
CS -> UI : on_thinking_start()
CS -> LLM : provider.create_streaming(\n client, model, messages, tools, ...)\n (normalized StreamChunk iterator)
activate LLM
note right of CS
Retry up to 3× on transient errors:
RateLimitError, APITimeoutError,
APIConnectionError, InternalServerError,
ServiceUnavailableError, APIError
Backoff: 1s, 2s, 4s
end note
== Streaming Response ==
loop for each chunk in stream
LLM --> CS : delta
note right of CS
on_thinking_stop() called on first
delta token via _stop_spinner_once()
end note
alt reasoning_content present
CS -> UI : on_reasoning_token(text)
else content present
CS -> UI : on_content_token(text)
else tool_call delta
CS -> CS : accumulate in tool_calls_acc
else info_delta present
CS -> UI : on_info(text)\n(e.g. server-side web search status)
end
end
note right of CS
**Cancellation checkpoint:**
_check_cancelled() runs per chunk.
If cancel_event is set, raises
GenerationCancelled — preserves
partial content, emits idle state.
end note
LLM --> CS : stream complete (usage stats)
deactivate LLM
CS -> UI : on_thinking_stop() (no-op guard: already called by _stop_spinner_once)
CS -> UI : on_stream_end()
CS -> CS : _update_token_table()\ncalibrate chars_per_token ratio
CS -> CS : messages.append(assistant_msg)
CS -> UI : on_turn_committed()
note right of UI
Drops the per-turn inflight buffers — the
assistant message is now in the history
list, so the in_progress_snapshot must
not re-render it during the next tool-
execution window or the next streaming turn.
end note
CS -> DB : save_message(ws_id, "assistant", content)
CS -> DB : save_message(ws_id, "tool_call", ...) ×N
== Tool Dispatch (if tool_calls) ==
alt no tool_calls
CS -> UI : on_status(usage, context_window, effort)
opt prompt_tokens > context_window × auto_compact_pct
CS -> CS : _compact_messages(auto=True)
CS -> LLM : Non-streaming summarization call
CS -> CS : Replace messages with [summary]
end
opt first exchange & no title
CS -> CS : Background thread: _generate_title()
end
CS -> UI : on_state_change("idle")
CS --> User : return
else has tool_calls
CS -> UI : on_state_change("running")
== Phase 1: Prepare ==
CS -> CS : [_prepare_tool(tc) for tc in tool_calls]\nParse JSON args, validate,\nbuild preview + header
== Phase 2: Approve ==
CS -> UI : on_state_change("attention")
CS -> UI : approve_tools(items)
activate UI
note right of UI
TerminalUI: input() prompt
WebUI: _approval_event.wait()
NullUI: returns (True, None)
end note
UI --> CS : (approved: bool, feedback: str?)
deactivate UI
CS -> UI : on_state_change("running")
== Phase 3: Execute ==
CS -> TP : ThreadPoolExecutor(max_workers=4)\nrun_one(item) for each tool
activate TP
note right of TP
Parallel execution:
bash → Popen + line-by-line streaming
read_file → open().read() or base64 image
search → grep subprocess
edit_file → string replace
task → _run_agent() sub-loop
web_fetch → httpx + LLM summarize
web_search → provider-native or SearxNG fallback
memory/recall → SQLite
end note
note right of TP
bash: on_tool_output_chunk(call_id, line)
called per stdout line,
then on_tool_result(call_id, name, output, is_error).
is_error=True when execution failed.
call_id routes chunks/results to correct
tool div during parallel execution.
Other tools: on_tool_result() only.
end note
TP --> CS : [(call_id, output), ...]
deactivate TP
loop for each result
CS -> CS : messages.append({role: "tool", ...})
CS -> DB : save_message(ws_id, "tool_result", ...)
end
opt user_feedback from approval
CS -> CS : messages.append({role: "user", content: feedback})
end
note right of CS : Loop back for next LLM call
else GenerationCancelled
CS -> CS : Preserve partial content\nor roll back incomplete tools
CS -> UI : on_info("[Generation cancelled]")
CS -> UI : on_state_change("idle")
CS --> User : return (no re-raise)
end
User -> Manager : dispatch send on one Workstream
Manager -> Session : bind_acting_user(principal)\nsend(text, attachments, send_id)
activate Session
Session -> Session : refresh immutable ResolvedModelBinding
opt token budget exhausted
Session -> UI : approve_tools(__budget_override__)
note right of UI
This gate precedes a generation claim but carries
a monotonic cancellation witness. Stop cannot be
mistaken for a budget-policy denial.
end note
end
deactivate CS
Session -> Session : _claim_generation() → generation N\ninstall fresh cancel event
Session -> Session : plan memory / participant context
Session -> Handoff : admit USER row\ncommit_key + prefix revision
Session -> Storage : ordered durable batch:\nappend canonical user Turn + metadata
note over Session, Handoff
Every accepted conversation row enters this lane before durability:
USER, ASSISTANT, TOOL, SYSTEM, compaction checkpoints, and cancellation
markers. Admission shares the handoff lock with its live UI transition
or history_resync repair event.
end note
note over Handoff, Storage
_commit_for_generation(N) admits bounded live mutations under the
generation lock, then executes immutable persistence closures in FIFO
ticket order. A force successor either follows the whole commit or
prevents it. /history projects durable prefix + pending journal suffix;
durable ACK removes the pending copy without changing the prefix revision.
end note
opt already over the hard context ceiling
Session -> Session : compact before first model call\n(preserve the new user turn)
end
== Model / tool loop ==
loop until final answer and no queued input
Session -> UI : on_turn_start()\nreset per-stream replay buffers
Session -> UI : state = thinking\non_thinking_start()
Session -> Session : _stream_response(N)\nretry + fallback policy
Session -> Plant : model_turn(active ModelLane, Turns,\n tools, cancel_ref, on_chunk)
activate Plant
Plant -> Plant : canonical Turns → provider wire\nrestore ids + repair + lane-specific fold
Plant -> Plant : materialize attachment refs\n(nested perception before outer slot)
Plant -> Admission : acquire(cancel_ref)
activate Admission
Plant -> Plant : resolve per-call backend credential\nfrom lane's pinned ModelConfig
Plant -> Provider : create_streaming(...)
activate Provider
loop normalized stream chunks
Provider --> Plant : StreamChunk
Plant --> Session : on_chunk(StreamChunk)
Session -> Session : check cancel event + generation N
Session -> UI : reasoning / content / info token
end
Provider --> Plant : finish + usage + native blocks
deactivate Provider
Plant -> Plant : drain + re-ingest assistant Turn\nwith serving-lane provenance
Plant -> Admission : release before retry backoff
deactivate Admission
Plant --> Session : ModelTurnResult
deactivate Plant
Session -> UI : on_stream_end()
Session -> Session : generation-fenced result commit:\nappend assistant Turn + token accounting
Session -> UI : on_turn_committed()
Session -> Handoff : admit ASSISTANT row\ncommit_key + prefix revision
Session -> Storage : ordered durable assistant row\n(content + tool mirror + native lane)
alt no tool calls
opt over soft threshold
Session -> Session : cooperative / end-of-turn compaction
Session -> Handoff : admit SYSTEM/source=compaction\ncheckpoint projection
Session -> Storage : append checkpoint summary marker\nwith source watermark
note right of Storage
Full history remains durable. Resume loads
[summary] + rows after the checkpoint.
end note
opt model stopped for compaction
Session -> Handoff : admit USER/source=compaction_resume row
Session -> Storage : append synthetic compaction_resume Turn
end
end
alt queued messages drained
Session -> Handoff : admit combined queued USER row
Session -> Storage : append combined queued user Turn
else truly complete
Session -> UI : state = idle
end
else tool calls present
Session -> UI : state = running
Session -> Session : prepare items + previews\nattach cancellation witnesses
opt one or more items require a human
Session -> UI : approve_tools(items)\nregister independent ApprovalCycle
note right of UI
Parallel agents may own concurrent cycles.
cycle_id / call_id routes exactly one decision;
Smart Approvals may clear qualifying items.
end note
User -> UI : approve / deny selected cycle
UI --> Session : decision + optional feedback
end
Session -> Tools : execute admitted items in parallel
activate Tools
Tools --> UI : chunks + result card\nwith effect disposition
Tools --> Session : outputs / errors / effect statuses
deactivate Tools
Session -> Session : output-guard evaluation\nthen generation N re-check
opt compaction owed before result sizing
Session -> Session : compact, preserving assistant tool-call Turn
Session -> Handoff : admit SYSTEM/source=compaction checkpoint
Session -> Storage : append checkpoint marker
end
Session -> Session : one generation-fenced batch:\nappend all Tool Turns, advisories, feedback
Session -> Handoff : admit FIFO TOOL rows\ncommit keys + prefix revisions
Session -> Storage : FIFO durable tool rows + metadata
end
end
== Stop / force-successor boundary ==
User -> Session : cancel()
Session -> Session : atomically set generation event; snapshot\nmain stream, child scopes, judges, subprocesses
Session -> Provider : close live stream handle
Session -> Tools : abort child scopes + kill subprocess groups
Session -> UI : resolve only cancelled operation's\napproval cycles
opt cancellation produced accepted conversation rows
Session -> Handoff : admit partial ASSISTANT and/or\nsynthesized TOOL cancellation markers
Session -> Storage : idempotent keyed cancellation rows
end
note over Session, Storage
Every later publish/commit checks generation ownership. An abandoned
worker may unwind, but cannot append Turns, overwrite state, resolve a
successor approval, or repaint the successor UI. Observed tool effects
are preserved as controller-authored cancellation receipts; unreviewed
tool bytes are not laundered into model context.
end note
deactivate Session
@enduml
+86 -103
View File
@@ -1,134 +1,117 @@
@startuml
!theme plain
title Turnstone — Tool Execution Pipeline (Three Phases)
title Turnstone — Tool Pipeline: Prepare, Approve, Execute, Fold
start
partition "Phase 1: Prepare" #E8F5E9 {
:Receive tool_calls list from LLM response;
partition "Phase 1 Prepare and assess" #E8F5E9 {
:Receive tool calls from one assistant Turn;
:Capture the generation's cancel event\nand acting principal;
while (more tool_calls?) is (yes)
:Extract call_id, func_name, raw_args;
if (json.loads(raw_args) succeeds?) then (yes)
:parsed_args = JSON dict;
while (more tool calls?) is (yes)
:Parse arguments and dispatch to\nthe tool-specific preparer;
if (preparation succeeds?) then (yes)
:Build item: call_id, name, header, preview,\nneeds_approval, execute closure;
else (no)
:Fallback 1: regex extraction;
if (regex found keys?) then (yes)
:parsed_args = extracted dict;
else (no)
:Fallback 2: bare string →\nPRIMARY_KEY_MAP[func_name];
endif
:Build an error item for this call only;\nkeep sibling calls valid;
endif
:Dispatch to _prepare_{func_name}();
note right
**Dispatch table (16 built-in + tool_search):**
┌───────────────┬──────────────────┐
│ Tool │ Needs Approval? │
├───────────────┼──────────────────┤
│ bash │ ✓ Yes │
│ read_file │ ✗ Auto-approve │
│ write_file │ ✓ Yes │
│ edit_file │ ✓ Yes │
│ search │ ✗ Auto-approve │
│ diff_file │ ✗ Auto-approve │
│ web_fetch │ ✗ Auto-approve │
│ web_search │ ✗ Auto-approve │
│ tool_search │ ✗ Auto-approve │
│ task_agent │ ✓ Yes │
│ memory │ ✗ Auto-approve │
│ recall │ ✗ Auto-approve │
│ notify │ ✗ Auto-approve │
│ watch │ ✓ create only │
│ skill │ ✓ load only │
│ read_resource │ ✓ Yes │
│ use_prompt │ ✓ Yes │
├───────────────┼──────────────────┤
│ mcp__* │ ✓ Yes (external) │
└───────────────┴──────────────────┘
end note
:Build item dict:
{call_id, func_name, header,
preview, needs_approval,
approval_label, execute: Callable};
:Attach operation-local cancellation witness\nand pinned principal;
endwhile (no)
:Reject only unsafe ordering shapes\n(for example tasks read + write in one batch);
:Run heuristic intent assessment immediately;
:Start generation-pinned LLM judge in background;
:Stamp one immutable Smart Approval\nsettings snapshot on the batch;
note right
Preparation is per-call isolated: one bad preparer
becomes one error Tool Turn rather than orphaning the
assistant's entire tool-call set.
end note
}
partition "Phase 2: Approve" #FFF3E0 {
if (any items need approval?) then (yes)
:_emit_state("attention");
:ui.approve_tools(items);
partition "Phase 2 Approval cycle" #FFF3E0 {
:Apply explicit bypasses:\nskill / always / policy / blanket;
if (Smart Approvals enabled?) then (yes)
:Wait within the batch's bounded judge deadline;
:Auto-approve only LLM approve verdicts\nat or above the captured threshold;
endif
if (human-gated items remain?) then (yes)
:Acquire approval-publication lease;
:Register independent ApprovalCycle\n(cycle_id, call_ids, event, result);
:Publish approve_request + heuristic verdicts;
note right
**auto_approve check is handled
internally by ui.approve_tools()**
**TerminalUI**: Print headers/previews,
prompt [y/n/a, optional message]
If user chose "always":
Add pending tool names to auto_approve_tools
(auto-approve these tool types going forward)
**WebUI**: Enqueue approve_request,
block on _approval_event.wait()
**NullUI**: Return (True, None)
Parallel task agents can hold several cycles at once.
A decision selects one cycle_id / call_id (or the oldest
cycle for a legacy selector-less client). Double resolve
is a guarded no-op; one cycle cannot wake a sibling.
end note
if (user approved?) then (yes)
:_emit_state("running");
else (denied)
:Mark all pending items as denied;
:denial_msg = "Denied by user";
:_emit_state("running");
if (operator approves?) then (yes)
:Record decision and optional feedback;
else (denies / policy blocks)
:Mark only pending items denied;\nEffectStatus = none;
endif
else (all auto-approved)
:ui enqueues tool_info event\n(no blocking);
:Publish approval_resolved;\nunregister this cycle;
else (all bypassed / auto-approved)
:Publish tool_info with the exact\nauto-approve reason per item;
endif
if (owning operation cancelled?) then (yes)
:Cancel only cycles carrying that witness;
:Stage every unstarted call as\nEffectStatus = none;
stop
endif
}
partition "Phase 3: Execute" #E3F2FD {
:_check_cancelled();
note right: Cancellation checkpoint:\nraises GenerationCancelled if\ncancel event is set
if (single tool call?) then (yes)
:Execute sequentially:\nrun_one(items[0]);
else (multiple)
:Execute in parallel:\nThreadPoolExecutor(max_workers=4)\npool.map(run_one, items);
partition "Phase 3 Execute" #E3F2FD {
:Generation + cancellation checkpoint;
if (batch requires serial ordering?) then (yes)
:Execute in provider order;
else (no)
:Execute via bounded ThreadPoolExecutor;
endif
note right
**run_one(item):**
if item.error → return error string
if item.denied → return denial message
else → item["execute"](item)
├─ _exec_bash: subprocess.run(["bash", script.sh])
├─ _exec_read_file: open().readlines() or _exec_read_image (base64)
├─ _exec_write_file: makedirs + write
├─ _exec_edit_file: find_occurrences + replace
├─ _exec_search: grep subprocess
├─ _exec_web_fetch: httpx.get + LLM summary
├─ _exec_web_search: SearxNG JSON GET (fallback for local models)
├─ _exec_tool_search: BM25 search + expand_visible()
├─ _exec_task: _run_agent(TASK_AGENT_TOOLS)
├─ _exec_notify: HTTP POST to channel gateway
├─ _exec_memory: structured memory save/search/delete/list
├─ _exec_recall: conversation history FTS5 search
├─ _exec_read_resource: MCPClientManager.read_resource_sync()
├─ _exec_use_prompt: MCPClientManager.get_prompt_sync()
└─ _exec_mcp_tool: MCPClientManager.call_tool_sync()
Each worker marks its call started only after the final
generation/cancel check. A missing result after that edge is
conservatively unknown; an unstarted call is definitively none.
end note
:Collect results: [(call_id, output), ...];
:Stream tool chunks to the matching call card;
:Capture result / error / preview and effect disposition;
:_truncate_output() on each result\n(max context_window × chars_per_token × 0.5 chars\ndefault: ~context_window × 2 chars);
if (Stop interrupts execution?) then (yes)
:Abort child model scopes and subprocess groups;
:Synthesize cancellation receipts;
note right
EffectStatus vocabulary:
committed / none / unknown /
partial / rolled_back.
:bash: ui.on_tool_output_chunk(call_id, line) per stdout line;
:ui.on_tool_result(call_id, name, output, is_error) for each;
Observed but unreviewed bytes are omitted from the
model-facing receipt; effect truth is retained.
end note
endif
}
:Return (results, user_feedback);
partition "Phase 4 — Guard and atomic fold" #F3E5F5 {
if (compaction already owed?) then (yes)
:Compact before sizing/folding results;\npreserve the assistant tool-call Turn;
endif
:Truncate each result against the remaining shared budget;
:Run heuristic + optional LLM output guard;
:Re-check generation after guard work;
:Under one generation commit, append the complete\nTool Turn block + advisories + feedback;
:Persist rows and effect/preview metadata\non the ordered durability lane;
:Return results to the next model turn;
}
stop
@enduml
+76 -12
View File
@@ -3,6 +3,7 @@
title Turnstone — Workstream State Machine
skinparam state {
BackgroundColor<<lifecycle>> #ECEFF1
BackgroundColor<<idle>> #E8F5E9
BackgroundColor<<thinking>> #E3F2FD
BackgroundColor<<running>> #FFF3E0
@@ -10,13 +11,18 @@ skinparam state {
BackgroundColor<<error>> #FFCDD2
}
state "CREATING (persisted only)" as creating <<lifecycle>> : Hidden durable reservation.\nNot returned by ordinary list/open/history.
state "IDLE" as idle <<idle>> : Waiting for user input.\nNo active LLM call or tool execution.
state "THINKING" as thinking <<thinking>> : LLM streaming response.\nTokens flowing (reasoning + content).
state "RUNNING" as running <<running>> : Tools executing.\nThreadPoolExecutor active.
state "ATTENTION" as attention <<attention>> : Blocked on user action.\nTool approval needed.
state "ERROR" as error <<error>> : Exception occurred.\nRecoverable on next send().
state "CLOSED (persisted only)" as closed <<lifecycle>> : Unloaded, explicitly reopenable row.\nNot a live WorkstreamState member.
[*] --> idle : Session created
[*] --> creating : register exact incarnation\nstate="creating"
creating --> idle : finalize + publish create\nemit ws_created
creating --> [*] : immediate exact-token rollback\n(no lifecycle birth emitted)
creating --> [*] : stale >2h recovery\natomic hard delete; no close event
idle --> thinking : send() called\n_emit_state("thinking")
@@ -38,6 +44,22 @@ running --> error : Exception during\ntool execution
error --> thinking : New send() call\n_emit_state("thinking")
idle --> closed : close / eviction\n[journal reconciled]
error --> closed : close\n[journal reconciled]
thinking --> closed : close\n[journal reconciled]
running --> closed : close\n[journal reconciled]
attention --> closed : close\n[journal reconciled]
closed --> [*] : hard delete
closed --> idle : open / rehydrate
note right of closed
Before every soft-close / eviction transition,
the total accepted conversation-row journal must
be durably reconciled. An unresolved row makes an
explicit close return HTTP 409 (eviction refuses),
and the workstream remains loaded in its live state.
end note
thinking --> idle : cancel() called\nstream aborted\n_emit_state("idle")
running --> idle : cancel() called\n_emit_state("idle")
@@ -45,33 +67,75 @@ running --> idle : cancel() called\n_emit_state("idle")
attention --> idle : cancel() unblocks\napproval wait\n_emit_state("idle")
note left of idle
**Cancel escalation:**
1. **Cooperative**: cancel() sets event + closes
SDK stream → worker exits at next checkpoint
2. **Force**: force=true abandons the worker
thread, emits stream_end immediately.
Orphaned thread still kills subprocesses
but skips message mutations (generation
counter prevents stale writes).
**Generation-scoped Stop:**
• Sets the active generation event.
• Closes its SDK stream; aborts child model
scopes and judges; kills subprocess groups.
• Sweeps every approval cycle owned by the
cancelled workstream operation.
• Every later send/model live or durable commit
re-checks generation ownership.
**force=true:** also abandons the stuck worker
slot and emits stream_end + IDLE immediately.
An orphaned send/model generation may unwind
but cannot publish into a successor generation.
Quick slash-command workers are a best-effort
escape hatch: without generation checkpoints,
one may finish an in-place mutation concurrently.
**Capacity eviction:** an IDLE candidate is only
a hint. Per-ID + object lifecycle lanes and the
workstream lock revalidate it as worker- and
send-barrier-free,
then install a terminal claim before slot swap.
end note
note right of thinking
**Emitted via:**
session._emit_state(state)
→ ui.on_state_change(state)
→ SessionManager state tail
**Propagation:**
• WebUI → global SSE queue (ws_state)
• Console → HTTP polling picks up state
• CLI → WorkstreamManager.set_state()
• Console → cluster event / HTTP state
• CLI → SessionManager.set_state()
Non-terminal persistence may use StateWriter;
a per-id tail orders storage + subscribers and
prevents a late state from overwriting CLOSED.
end note
note left of attention
**Blocking mechanisms:**
• TerminalUI: input() prompt
• WebUI: threading.Event.wait()
• WebUI: one Event per ApprovalCycle
• ChannelBot: SSE event + Discord button
• NullUI: auto-approve (never reaches)
end note
note right of creating
CREATING and CLOSED are storage lifecycle
values, not members of WorkstreamState. The
live enum remains IDLE / THINKING / RUNNING /
ATTENTION / ERROR.
**Crash-abandoned CREATING recovery:**
• Boot pass, then every 5 min even when idle
eviction is disabled.
• Only rows >2h old; manager loaded/pending
IDs and live remote owners are protected.
• The current stable node ID is not a live-owner
exemption, allowing restart recovery.
• Unknown liveness/storage fails closed. Deletion
is atomic across dependents and attachment refs.
• Tokenless legacy/corrupt rows are locked,
reaped, and logged with a warning.
A loaded hard delete closes publication, drains
admitted session durability + state tails, then
conditionally removes the exact durable token.
end note
@enduml
+2 -2
View File
@@ -31,7 +31,7 @@ node "Docker Host" as host {
Command: turnstone-console
--port 8090
Depends: server
Hash-ring router for
FNV-1a rendezvous router for
multi-node clusters
end note
}
@@ -69,7 +69,7 @@ apiclient --> server : HTTP + SSE\nport 8080
' Internal connections
server --> llm_api : OpenAI API\n(HTTPS/HTTP)
console --> server : HTTP proxy\n(hash-ring lookup,\nproxy /node/{id}/*)
console --> server : HTTP proxy\n(FNV-1a rendezvous placement,\nproxy /node/{id}/*)
' Database connections (production/cluster profiles)
server ..> pgbouncer : PostgreSQL\n(pool_size=2)
+18 -2
View File
@@ -32,7 +32,8 @@ package "turnstone/sdk/ (Python)" {
+ approve()
+ command()
+ cancel(ws_id)
+ stream_events(ws_id)
+ get_history(ws_id, limit) → WorkstreamHistoryResponse
+ stream_events(ws_id, last_event_id?, history_token?)
+ stream_global_events()
+ send_and_wait()
+ list_saved_workstreams()
@@ -87,6 +88,13 @@ package "turnstone/sdk/ (Python)" {
+ ok: bool
}
class WorkstreamHistoryResponse <<type>> {
+ ws_id: str
+ messages: list[dict]
+ cursor: int | None
+ handoff_token: str | None
}
class ServerEvent <<event>> {
+ type: str
+ ws_id: str
@@ -105,6 +113,7 @@ package "turnstone/sdk/ (Python)" {
TurnstoneConsole --> AsyncTurnstoneConsole : wraps
TurnstoneConsole --> _SyncRunner : uses
AsyncTurnstoneServer ..> TurnResult : returns
AsyncTurnstoneServer ..> WorkstreamHistoryResponse : renders before SSE
AsyncTurnstoneServer ..> ServerEvent : yields
AsyncTurnstoneConsole ..> ClusterEvent : yields
}
@@ -122,7 +131,8 @@ package "sdk/typescript/ (TypeScript)" {
class "TurnstoneServer" as TSServer <<ts>> {
+ listWorkstreams()
+ send()
+ streamEvents()
+ getHistory() → WorkstreamHistoryResponse
+ streamEvents(cursor?, token?)
+ sendAndWait()
...
}
@@ -154,4 +164,10 @@ note right of AsyncTurnstoneServer
(no type duplication)
end note
note bottom of ServerEvent
history_resync is a typed repair signal.
SDKs expose the REST cursor/token handshake but
never refetch, render, or reconnect automatically.
end note
@enduml
+150 -129
View File
@@ -1,170 +1,191 @@
@startuml
!theme plain
title Turnstone — Storage Architecture
title Turnstone — Storage, Deferred Create, Fork, and Checkpoint Architecture
skinparam class {
BackgroundColor<<protocol>> #E8EAF6
BackgroundColor<<sqlite>> #C8E6C9
BackgroundColor<<postgres>> #B3E5FC
BackgroundColor<<facade>> #FFF9C4
BackgroundColor<<migration>> #FFE0B2
BackgroundColor<<lifecycle>> #FFF9C4
BackgroundColor<<schema>> #F3E5F5
BackgroundColor<<helper>> #FFE0B2
}
' -- Protocol --
interface "StorageBackend" as SB <<protocol>> {
+save_message(ws_id, role, content, ...)
+load_messages(ws_id) → list[dict]
+register_workstream(ws_id, node_id, name, state)
+update_workstream_state(ws_id, state)
+update_workstream_name(ws_id, name)
+set_workstream_alias(ws_id, alias) → bool
+update_workstream_title(ws_id, title)
+resolve_workstream(alias_or_id) → str | None
+delete_workstream(ws_id) → bool
+prune_workstreams(retention_days) → (int, int)
+list_workstreams(node_id, limit, *, parent_ws_id, kind, user_id) → list
+save_workstream_config(ws_id, config)
+load_workstream_config(ws_id) → dict
+kv_get(key) → str | None
+kv_set(key, value) → str | None
+kv_delete(key) → bool
+kv_list() → list[(str, str)]
+kv_search(query) → list[(str, str)]
+search_history(query, limit) → list
+search_history_recent(limit) → list
+create_user(user_id, username, display_name, pw_hash)
+get_user(user_id) / get_user_by_username(username)
+list_users() / delete_user(user_id)
+create_api_token(...) / get_api_token_by_hash(hash)
+list_api_tokens(user_id) / delete_api_token(id)
+close()
}
' -- Backends --
class "SQLiteBackend" as SQLite <<sqlite>> {
-_engine: sa.Engine
-_fts5_available: bool
+__init__(path: str)
interface "StorageBackend" as Storage <<protocol>> {
+ load_message_turns(ws_id, checkpointed=True) → list[Turn]
+ save_message(ws_id, role, content, metadata...)
+ clone_workstream(source, destination, principal, expected_session) → ForkCloneSnapshot
--
FTS5 full-text search
Default pool, check_same_thread=False
+ register_workstream(..., state, reservation_token) → bool
+ ensure_workstream_incarnation_snapshot(ws_id) → row + token
+ finalize_deferred_create(ws_id, token, config...) → bool
+ publish_deferred_create(ws_id, token) → bool
+ delete_workstream_if_fork_reserved(ws_id, token) → bool
+ delete_stale_creating_reservations(...) → list[ws_id]
+ update_workstream_state(ws_id, state)
+ delete_workstream(ws_id) → bool
--
+ attachment / project / memory / auth / governance APIs
}
class "SQLiteBackend" as SQLite <<sqlite>> {
- _engine: sa.Engine
- _fts5_available: bool
--
Fork clone: BEGIN IMMEDIATE
FTS5 refresh in same transaction
}
class "PostgreSQLBackend" as PG <<postgres>> {
-_engine: sa.Engine
+__init__(url: str, pool_size: int = 2,\n max_overflow: int = 3)
- _engine: sa.Engine
--
tsvector + ILIKE search
Connection pooling (5 max per process)
Fork clone: SERIALIZABLE + row locks
Retry SQLSTATE 40001 / 40P01
DML success uses RETURNING rows
}
' -- Schema --
class "_schema.py" as Schema <<schema>> {
+metadata: MetaData
+memories: Table
+conversations: Table
+workstreams: Table (node_id, alias, title,\n state, skill_id)
+workstream_config: Table
+users: Table (username, password_hash)
+api_tokens: Table (token_hash, scopes)
+channel_users: Table (channel_type)
+scheduled_tasks: Table (..., skill)
class "_utils.py" as Utils <<helper>> {
+ reconstruct_turns(rows) → list[Turn]
+ recover_trajectory(turns) → list[Turn]
+ reconstruct_turns_checkpointed(...)
+ retain_attachment_refs(conn, ids)
+ release_attachment_refs(conn, ids)
+ clone_workstream_transaction(...) → ForkCloneSnapshot
}
class "ForkCloneExpectation" as Expectation <<lifecycle>> {
+ persona_config
+ project_id / name / writable
+ source_reservation_token
+ destination_reservation_token
}
class "ForkCloneSnapshot" as Snapshot <<lifecycle>> {
+ turns: tuple[Turn, ...]
+ config: dict[str, str]
+ project_id: str | None
}
class "workstreams" as Workstreams <<schema>> {
ws_id PK
state: creating | live state | closed
user_id, node_id, kind, parent_ws_id
project_id, persona, alias, title
}
class "conversations" as Conversations <<schema>> {
canonical persisted Turn rows
provider_data + tool_calls mirror
event_id, source, is_error, meta
attachment-id ref list
--
SQLAlchemy Core
Single source of truth
compaction marker:
source="compaction"
meta.watermark=<folded row id>
}
' -- Migration --
class "_migrate.py" as Migrate <<migration>> {
+run_migrations(storage, backend)
-_bootstrap_existing_sqlite()
--
Programmatic Alembic
Auto-bootstrap existing DBs
class "workstream_config" as WorkstreamConfig <<schema>> {
PK (ws_id, key)
stamped persona/session config
private durable incarnation fence:
__fork_destination_reservation
}
class "migrations/" as Versions <<migration>> {
001_initial_schema.py
002_user_identity.py
class "workstream_attachments" as Attachments <<schema>> {
content-addressed blob
attachment_id, bytes, kind
refcount
}
' -- Registry --
class "_registry.py" as Registry {
-_storage: StorageBackend | None
+init_storage(backend, path, url) → StorageBackend
+get_storage() → StorageBackend
+reset_storage()
--
Auto-initializes SQLite
if not configured
class "projects + project_members" as Projects <<schema>> {
visibility / owner / membership
active project-memory envelope
}
' -- Facade --
class "memory.py" as Facade <<facade>> {
+save_message()
+load_messages()
+register_workstream()
+update_workstream_state()
+save_workstream_config()
+save_memory() / delete_memory()
+search_memories()
+... (all delegated functions)
--
Thin delegation to
get_storage()
Silent failure behavior
class "SessionManager" as Manager <<lifecycle>> {
+ create(..., defer_emit_created)
+ commit_create(ws)
+ discard(ws)
+ reap_stale_creating_reservations(max_age=2h)
+ open / close / delete
}
' -- Consumers --
class "session.py\nChatSession" as Session {
class "ChatSession" as Session <<lifecycle>> {
+ append canonical Turns
+ compact / resume checkpoint
+ fork_from_storage(...)
}
class "server.py\nWeb UI" as Server {
}
SQLite ..|> Storage
PG ..|> Storage
SQLite --> Utils
PG --> Utils
class "cli.py\nTerminal" as CLI {
}
Storage --> Workstreams
Storage --> Conversations
Storage --> WorkstreamConfig
Storage --> Attachments
Storage --> Projects
' -- Relationships --
SQLite ..|> SB
PG ..|> SB
Manager --> Storage : lifecycle reservation + state
Session --> Storage : turn durability + resume
Session --> Expectation : construction witness
Storage --> Snapshot : atomic clone result
Expectation --> Utils : checked inside transaction
Utils --> Snapshot : builds
SQLite --> Schema : uses
PG --> Schema : uses
note right of Manager
**Deferred create publication**
1. INSERT workstream as state="creating" and store a fresh
private token in the same transaction.
2. Construct UI/session and run attachment/fork gates while
ordinary list/open/history reads exclude the row.
3. finalize_deferred_create atomically applies config/alias.
4. publish_deferred_create compare-and-swaps creating → idle.
5. Only then emit ws_created.
Registry --> SB : creates
Registry --> Migrate : calls
Migrate --> Versions : applies
Migrate --> Schema : references
Facade --> Registry : get_storage()
Session --> Facade : imports
Server --> Facade : imports
CLI --> Facade : imports
' -- Config --
note right of Registry
[database]
backend = "sqlite" | "postgresql"
url = "postgresql+psycopg://..."
path = ".turnstone.db"
pool_size = 2 (+ 3 overflow)
Any normal prepublication failure immediately calls exact token-checked
deletion. The token survives publication as the row's incarnation fence:
rollback or later hard delete can never ABA-delete a replacement row.
A legacy row acquires the same private token atomically when rehydrate,
delete, or fork preflight takes its authoritative snapshot. Loaded hard
delete drains admitted session durability before its token-checked delete.
end note
note bottom of SQLite
Default backend.
Zero-config for
single-node / dev.
note left of Manager
**Crash-abandoned hidden-create recovery**
• Boot pass; long-lived processes repeat every 5 min,
even when ordinary idle eviction is disabled.
• Candidates remain state="creating", are >2h old,
and are absent from the manager loaded/pending set.
• Live remote owners are protected. The current stable
node ID does not self-protect, enabling restart recovery.
• Unknown liveness or storage failure deletes nothing.
• One transaction rechecks state, age, and token, then
hard-deletes dependents and releases attachment refs.
• Tokenless legacy/corrupt rows use their locked durable
row as the incarnation fence and log a warning.
• Retention pruning excludes creating rows. Recovery never
closes or publishes them as live WorkstreamState values.
end note
note bottom of PG
Production backend.
Multi-node / Docker default.
Use PgBouncer (transaction mode)
for clusters > 50 nodes.
note bottom of Utils
**Atomic fork clone**
• Reject a provisional source; compare the source incarnation captured
by canonical preflight; re-authorize project visibility and compare the
live session envelope inside the transaction.
• Require a same-owner, empty destination still in creating state
with the exact reservation token.
• Copy the checkpoint-bounded canonical trajectory and config;
retain every referenced attachment or roll everything back.
• Preserve/rebase a valid compaction checkpoint watermark and
return the exact snapshot installed into the live destination.
end note
note bottom of Conversations
Full transcript rows are never deleted by compaction. Normal resume
loads the latest valid [summary] + rows after its watermark; audit and
export can request the full marker-free history.
end note
@enduml
+129 -166
View File
@@ -1,190 +1,153 @@
@startuml
!theme plain
title Turnstone — Authentication Architecture
title Turnstone — User Authentication and Model-Backend Credentials
skinparam class {
BackgroundColor<<core>> #E8EAF6
BackgroundColor<<jwt>> #C8E6C9
BackgroundColor<<token>> #C8E6C9
BackgroundColor<<storage>> #B3E5FC
BackgroundColor<<endpoint>> #FFE0B2
BackgroundColor<<scope>> #F3E5F5
BackgroundColor<<runtime>> #FFE0B2
BackgroundColor<<model>> #F3E5F5
}
' -- Core Auth --
class "AuthConfig" as AC <<core>> {
+enabled: bool
+tokens: dict[str, str]
+check(token) → role | None
--
Static config-file tokens
hmac.compare_digest
package "Request identity" {
class "AuthMiddleware / check_request()" as RequestAuth <<core>> {
Extract bearer or HttpOnly cookie
Validate audience + expiry
Check scope / permission
Publish AuthResult in request state
}
class "AuthResult" as AuthResult <<core>> {
+ user_id: str
+ scopes: frozenset[str]
+ permissions: frozenset[str]
+ token_source: str
}
class "JWT" as JWT <<token>> {
HS256, sub, aud, iat, exp
console proxy mints short-lived
server-audience identity
}
class "API / config token" as ApiToken <<token>> {
ts_* token: SHA-256 DB lookup
config token: constant-time compare
}
class "users / roles / api_tokens" as UserTables <<storage>> {
password hash + token hash
role-derived permissions
}
}
class "AuthResult" as AR <<core>> {
+user_id: str
+scopes: frozenset[str]
+token_source: str
+has_scope(scope) → bool
package "Immutable model binding" {
class "ModelRegistry" as Registry <<model>> {
+ resolve_binding(alias)
+ generation: int
--
Atomically resolves client, provider,
model, ModelConfig, generation.
}
class "ModelConfig snapshot" as ModelConfig <<model>> {
+ alias / provider / endpoint / static key
+ auth_mode
+ obo_audience
+ obo_scopes
--
static | entra_obo | entra_app | rfc8693_obo
}
class "ModelLane" as Lane <<model>> {
+ client / provider / model / capabilities
+ backend_auth_config: ModelConfig
+ backend_auth_resolver: Callable
}
class "Model definitions" as ModelTable <<storage>> {
DB + config-file definitions
encrypted protected fields
}
}
class "check_request()" as CR <<core>> {
auth_config, method, path,
auth_header, cookie_header,
jwt_secret, storage
→ (allowed, status, msg, AuthResult)
--
1. Auth disabled → allow
2. Public path → allow
3. Extract Bearer / cookie
4. Detect token type
5. Validate → AuthResult
6. Check scope vs path
package "Per-call credential resolution" {
class "resolve_model_backend_auth_token()" as Resolver <<runtime>> {
+ alias + pinned ModelConfig
+ initiating principal_id
+ ConfigStore + mint client
→ dynamic token | None | fail closed
}
class "Model mint client" as Mint <<runtime>> {
+ mint_model_obo_token_sync(...)
+ mint_app_token_sync(...)
--
Cached by alias / principal / grant leg;
retains refusal cause for diagnostics.
}
class "OIDC / OBO protected state" as OBOState <<storage>> {
encrypted user refresh credential
deployment Fernet key
configured grant profile
}
class "lane_call_client()" as CallClient <<runtime>> {
cancel check before mint
resolve once per plant call
cancel check after mint
client.with_options(api_key=token)
}
class "Provider SDK request" as ProviderCall <<runtime>> {
Anthropic: x-api-key
OpenAI-style: Authorization Bearer
}
}
' -- Token Types --
class "JWT (HS256)" as JWT <<jwt>> {
sub: user_id
scopes: "read,write,approve"
src: "password" | "database"
iat, exp (24h default)
--
Detected by: contains "."
Validated locally
No DB call
}
RequestAuth --> JWT : validates
RequestAuth --> ApiToken : validates
RequestAuth --> UserTables : lookup + permissions
RequestAuth --> AuthResult : returns
class "API Token" as AT <<jwt>> {
Format: ts_ + 64 hex
Stored: SHA-256 hash
--
Detected by: starts with "ts_"
Lookup by hash in DB
Expiry check
}
ModelTable --> Registry : load / hot reload
Registry --> ModelConfig : immutable snapshot
Registry --> Lane : coherent binding
class "Config Token" as CT <<core>> {
Raw value in memory
Role: "read" | "full"
--
Detected by: fallback
hmac.compare_digest
No DB needed
}
AuthResult --> Resolver : initiating principal
Lane --> Resolver : callable + pinned config
Resolver --> Mint : dynamic modes only
Mint --> OBOState : decrypt / grant policy
CallClient --> Lane
CallClient --> Resolver
CallClient --> ProviderCall : cloned SDK client
' -- Scopes --
class "Scope Hierarchy" as SH <<scope>> {
read: {read}
write: {read, write}
approve: {read, write, approve}
--
GET → read
POST write paths → write
POST /api/workstreams/{ws_id}/approve → approve
/api/admin/* → approve
}
' -- Storage --
class "users" as UT <<storage>> {
user_id (PK)
username (unique)
display_name
password_hash (bcrypt)
created
}
class "api_tokens" as TT <<storage>> {
token_id (PK)
token_hash (SHA-256, unique)
token_prefix
user_id → users
name, scopes
created, expires
}
' -- Endpoints --
class "POST /api/auth/login" as Login <<endpoint>> {
{username, password}
OR {token: "ts_xxx"}
→ {jwt, role, scopes, user_id}
--
Sets HttpOnly cookie
}
class "GET /api/auth/status" as Status <<endpoint>> {
→ {auth_enabled, has_users,
setup_required}
--
Public (no auth)
Drives UI setup wizard
}
class "POST /api/auth/setup" as Setup <<endpoint>> {
{username, display_name, password}
→ {jwt, user_id, scopes}
--
Public (no auth)
Only when zero users exist
Returns 409 if already set up
}
class "Admin API (Console)" as Admin <<endpoint>> {
POST/GET/DELETE users
POST/GET tokens
DELETE tokens/{id}
--
Requires approve scope
}
' -- Relationships --
CR --> AC : config tokens
CR --> JWT : validate
CR --> AT : hash lookup
CR --> CT : hmac check
CR --> AR : returns
CR --> SH : checks
Login --> JWT : issues
Login --> UT : verify password
Login --> TT : verify API token
Setup --> UT : create first user
Setup --> JWT : issues
AT --> TT : lookup by hash
Admin --> UT : CRUD
Admin --> TT : CRUD
AR --> SH : scopes from
JWT ..> AR : produces
AT ..> AR : produces
CT ..> AR : produces
note right of CR
**Middleware Flow**
AuthMiddleware on every request:
1. Extract token from header/cookie
2. Detect type (JWT / ts_ / config)
3. Validate → AuthResult
4. Set ctx_user_id for logging
5. Store auth_result in scope state
note right of Resolver
**Mode policy**
• static: return None; registry client's explicit key remains.
• entra_obo / rfc8693_obo: require an effective principal. HTTP
turns pin the authenticated initiator; single-user internal lanes
may use their session owner. Never borrow another generation's identity.
• entra_app: use deployment app identity, no user required.
• rfc8693_obo alone sends obo_scopes; each dynamic mode is paired
with its required Entra or RFC 8693 grant profile.
end note
note bottom of SH
**Console** owns admin endpoints
**Server** validates JWT + config only
Both share JWT signing secret
note bottom of CallClient
Dynamic credentials are minted at dispatch, not cached in the registry
snapshot. Endpoint, audience, scopes, auth mode, and static-key presence stay
pinned to the same ModelConfig generation as the SDK client. The global
model.auth_fail_closed policy is read live on every mint. A Stop that wins
before or during mint prevents model bytes from being sent afterward.
end note
note left of JWT
**Console Proxy Token Minting**
When proxying requests to server nodes:
1. Console AuthMiddleware validates user JWT (aud: turnstone-console)
2. Proxy mints new JWT (aud: turnstone-server)
with real user_id, scopes, permissions
3. src: "console-proxy" for audit traceability
4. 5-minute expiry (fresh per request)
5. Fallback: ServiceTokenManager if no user context
note bottom of ProviderCall
If minting fails, a configured fail-closed deployment or a keyless alias
raises BackendAuthUnavailableError. A dynamic alias with an explicit static
key may fall back only when policy allows. Authentication refusal is not a
backend-health failure and does not walk to a static fallback model.
end note
@enduml
+30 -14
View File
@@ -82,10 +82,12 @@ class "DiscordBot" as Bot <<service>> {
}
class "ChannelRouter" as Router <<service>> {
+resolve_route(platform, channel_id)
-> ws_id | None
+register_route(channel_id, ws_id)
+resolve_identity(platform, platform_user_id)
+get_or_create_workstream(channel_type, channel_id)
+_is_ws_live(ws_id)
+send_message(ws_id, message)
+send_approval(ws_id, ...)
+lookup_ws_id(channel_type, channel_id)
+resolve_user(channel_type, channel_user_id)
-> user_id | None
--
Maps channels -> workstreams
@@ -93,6 +95,16 @@ class "ChannelRouter" as Router <<service>> {
Caches routes in memory
}
class "turnstone-console router" as ConsoleRouter <<server>> {
POST /v1/api/route/workstreams/new
GET /v1/api/route/workstreams/{ws_id}/live
POST /v1/api/route/workstreams/{ws_id}/send
POST /v1/api/route/workstreams/{ws_id}/approve
GET /v1/api/route?ws_id=...
--
Multi-node rendezvous + durable overrides
}
' -- Server --
class "turnstone-server" as Server <<server>> {
POST /v1/api/workstreams/{ws_id}/send
@@ -148,7 +160,9 @@ Bot --> Router : on_message\non_interaction
Router --> CU : resolve identity
Router --> CR : resolve / register route
Router --> Server : POST /v1/api/workstreams/{ws_id}/send\nPOST /v1/api/workstreams/{ws_id}/approve\nPOST /v1/api/workstreams/new
Router --> Server : single-node/direct mode\ncreate + send + approve
Router --> ConsoleRouter : multi-node mode\nroute create/live/send/approve/lookup
ConsoleRouter --> Server : routed HTTP to owning node
Bot --> Server : GET /v1/api/workstreams/{ws_id}/events\n(SSE via httpx-sse)
Server --> Bot : SSE event stream
@@ -156,7 +170,7 @@ Bot --> Discord : reply / embed\nbutton callback
Slack --> SlackBot : socket-mode\nevents
SlackBot --> Router : on_message / on_action
SlackBot --> Server : POST /v1/api/workstreams/{ws_id}/send\nGET /v1/api/workstreams/{ws_id}/events
SlackBot --> Server : owning-node SSE after route lookup
SlackBot --> Slack : post / update\nBlock Kit button callbacks
Teams .[hidden]. Slack
@@ -175,19 +189,21 @@ note right of Bot
**Inbound Flow**
1. Discord message arrives via gateway
2. Bot.on_message() fires
3. ChannelRouter resolves channel -> ws_id
(or creates new workstream)
3. ChannelRouter gets or creates channel -> ws_id
(direct server or multi-node console router)
4. ChannelRouter resolves platform user -> user_id
via channel_users table
5. Router sends POST /v1/api/workstreams/{ws_id}/send to server
5. Router sends through the configured server/console SDK
**Workstream Resume (evicted workstreams)**
1. Stale route detected (no active SSE listener)
2. Existing ws_id reused directly from route
**Stale-route recovery (evicted workstreams)**
1. Route health check reports the old ws unavailable
2. Existing ws_id becomes the fork source
3. POST /v1/api/workstreams/new with
resume_ws=<ws_id>
4. Server resumes atomically during creation
5. SSE emits WorkstreamResumedEvent -> thread
4. Server atomically clones source history/config/
persona/project/attachment refs into a new ws_id
5. Router stores the new destination route; source is unchanged
6. If the source was pruned, retry one fresh create
end note
note right of Server
+2 -2
View File
@@ -11,7 +11,7 @@ skinparam participant {
participant "ChatSession\n(session.py)" as Session <<session>>
participant "WatchRunner\n(watch.py)" as Runner <<server>>
participant "StorageBackend\n(SQLite)" as Storage <<storage>>
participant "StorageBackend\n(SQLite / PostgreSQL)" as Storage <<storage>>
participant "WebUI / SSE\n(server.py)" as UI <<ui>>
== Create Phase ==
@@ -130,7 +130,7 @@ note right : action="cancel" (auto-approve)
note over Runner, Storage
**Startup:**
1. WatchRunner created in main() with storage + node_id
2. restore_fn closure captures WorkstreamManager
2. restore_fn closure captures SessionManager
3. Initial workstream: session.set_watch_runner(runner)
4. _lifespan(): runner.start() — daemon thread begins
+103 -195
View File
@@ -1,218 +1,126 @@
@startuml
!theme plain
title Turnstone — Intent Validation (Judge) Architecture
title Turnstone — Intent Judge, Concurrent Approval Cycles, and Output Guard
skinparam participant {
BackgroundColor<<session>> #C8E6C9
BackgroundColor<<judge>> #FFE0B2
BackgroundColor<<storage>> #B3E5FC
BackgroundColor<<ui>> #E8EAF6
BackgroundColor<<fs>> #F5F5F5
}
skinparam sequenceArrowThickness 1.5
skinparam sequenceLifeLineBackgroundColor #F5F5F5
participant "ChatSession\n(session.py)" as Session <<session>>
participant "IntentJudge\n(judge.py)" as Judge <<judge>>
participant "LLM Provider\n(provider)" as LLM <<judge>>
participant "StorageBackend\n(SQLite)" as Storage <<storage>>
participant "WebUI / SSE\n(server.py)" as UI <<ui>>
participant "Filesystem" as FS <<fs>>
participant "ChatSession\ngeneration N" as Session
participant "SessionUIBase" as UI
participant "IntentJudge" as Judge
participant "model_turn()\n(pinned ModelLane)" as Model
participant "Operator / client" as Operator
participant "OutputGuardJudge" as Guard
database "StorageBackend" as Storage
== Tool Call Requires Approval ==
== Intent assessment begins during preparation ==
Session -> Session : _prepare_tool_calls()
note right
Tool calls parsed from
LLM response. Auto-approved
tools dispatched immediately.
Remaining items need approval.
Session -> Session : prepare each tool item independently\nattach principal + cancel witness
Session -> Judge : evaluate(items, callback, cancel_ref)
activate Judge
Judge -> Judge : synchronous heuristic verdict\nfor each call (first matching rule)
Judge --> Session : heuristic verdicts + daemon cancel event
Session -> UI : cache / publish heuristic assessments
Session -> Storage : persist heuristic intent verdicts
note over Judge, Model
The judge owns an immutable resolved binding. Registry/config generations
are freshness watermarks: an effective lane change replaces the judge for
the next batch, while in-flight work keeps the lane it started with.
Dynamic backend auth is resolved for this batch's initiating principal.
parallel_evaluations (1-16) sets per-batch worker width; the model alias's
admission gate remains the process-wide generation ceiling.
end note
Session -> Session : _evaluate_intent(pending_items)
== Tier 1: Heuristic (synchronous, sub-ms) ==
Session -> Judge : evaluate(items, messages, callback)
Judge -> Judge : evaluate_heuristic()\nfor each item
note right
**36 rules (first match wins):**
Critical (0.90, deny): rm /, mkfs,
dd, pipe-to-shell, chmod 777 /,
write/edit /etc/ .ssh/,
download-then-execute chains
High (0.80, review): sudo, kill -9,
destructive git, DROP TABLE,
secrets, HTTP mutations, ssh/scp,
browser+data-export, transitive
install, control-plane mutation
Medium (0.70, review): content
ingestion, interpreter exec,
cloud CLI mutations, pkg install,
write_file, MCP tools, docker ops
Low (0.85, approve): read_file,
list_directory, search, recall,
tool_search, read_resource,
web_search, read-only bash
Default: medium, 0.50, review
end note
Judge --> Session : heuristic_verdicts[]
Session -> Session : attach _heuristic_verdict\nto each pending item
Session -> UI : SSE: approve_request\n{items: [{verdict: ...}],\n judge_pending: true}
note right
Heuristic verdict displayed
immediately as risk badge.
Spinner shown while LLM
judge evaluates.
end note
Session -> Storage : create_intent_verdict()\nfor each heuristic verdict
== Tier 2: LLM Judge (daemon thread, async) ==
Judge -> Judge : spawn daemon thread\n"intent-judge"
note over Judge, LLM
**Context preparation:**
1. FIFO-truncate conversation history
to max_context_ratio of context window
2. Append tool call details as user message
3. System prompt defines judge role + JSON schema
end note
loop up to 3 turns (timeout budget)
Judge -> LLM : model_turn(lane, judge_turns,\ntools=[read_file, list_directory])\nvia drained create_streaming
LLM --> Judge : ModelTurnResult
alt tool_calls present (turn < 3)
Judge -> Judge : _exec_read_only_tool()
note right
**Security hardening:**
Blocked: /etc/, /root/,
/proc/, /sys/, /dev/,
.ssh, .gnupg, .aws,
*.pem, *.key, *.p12
File cap: 32KB
Dir cap: 200 entries
end note
Judge -> FS : read_file / list_directory
FS --> Judge : file contents
Judge -> Judge : append tool result\nto judge_messages
else text response (final verdict)
Judge -> Judge : _parse_verdict()
note right
**4-stage JSON parsing:**
1. Direct JSON.loads
2. Markdown code block
3. Brace-counting
4. Regex field extraction
end note
par LLM judge daemon coordinator
Judge -> Judge : start min(batch size, parallel_evaluations,\npositive alias capacity) workers
loop each worker claims one independent call
Judge -> Model : model_turn(judge lane, canonical Turns,\nread-only evidence tools, cancel_ref)
Model --> Judge : ModelTurnResult
alt evidence tool requested
Judge -> Judge : execute bounded read_file / list_directory
else verdict text
Judge -> Judge : parse + arbitrate against heuristic
end
Judge --> UI : on_intent_verdict(verdict, judge generation)
UI -> Storage : persist LLM verdict / audit update
end
else approval path continues
Session -> UI : approve_tools(items) with one\nSmart Approval config snapshot
end
== Tier 3: Arbitration ==
== Policy, Smart Approval, and human gate ==
Judge -> Judge : compare confidence:\nLLM vs heuristic
note right
Only deliver LLM verdict
if confidence > heuristic.
Otherwise heuristic stands.
end note
alt LLM confidence > heuristic confidence
Judge -> Session : callback(llm_verdict)
Session -> UI : SSE: intent_verdict\n{tier: "llm", ...}
note right
UI replaces heuristic badge
with LLM verdict. Spinner
resolves to final assessment.
end note
Session -> Storage : create_intent_verdict()\nfor LLM verdict
UI -> UI : apply explicit policy / skill / always / blanket bypasses
opt Smart Approvals enabled
UI -> UI : wait within captured deadline for this batch's verdicts
UI -> UI : auto-approve only recommendation=approve\nand confidence >= captured threshold
UI -> Storage : persist auto-approval reason and decision
end
== User Decision ==
UI -> Session : resolve_approval(\napproved, feedback)
Session -> Storage : update_intent_verdict(\nverdict_id, user_decision)
note right
All tracked verdicts
(heuristic + LLM) updated
with "approved" or "denied".
Swap-and-clear avoids racing
with daemon judge thread.
end note
== Tool Execution ==
Session -> Session : _execute_tools()
note right
Tools execute with
user approval.
end note
== Output Guard (synchronous, time-budgeted) ==
Session -> Session : _evaluate_output()\nfor each tool result
note right
**Priority-ordered checks (5s budget):**
P1: Prompt injection (role injection,
override phrases, instruction tags)
P2: Credential leakage (API keys,
PEM blocks, connection strings)
P3: Encoded payloads (data URIs,
hex shellcode)
P4: Adversarial URLs (cloud metadata,
credential query params)
P5: System info disclosure (private
IPs, sensitive paths)
Annotates + optionally redacts.
Does NOT gate.
end note
alt output_warning flags detected
Session -> UI : SSE: output_warning\n{call_id, risk_level, flags,\nfunc_name, redacted}
note right
Credential values replaced
with [REDACTED:<type>] before
output enters conversation.
sanitized text excluded from
SSE payload (defense in depth).
end note
UI -> Storage : record_output_assessment()\nfire-and-forget persistence
note right
Stored: flags, risk_level,
annotations, output_length,
redacted (bool). Raw tool
output is never stored.
end note
alt human-gated items remain
UI -> UI : acquire publication lease; register ApprovalCycle\n(cycle_id, call_ids, event, result, witnesses)
UI -> Operator : approve_request with cycle_id + item verdicts
Operator -> UI : approve / deny by cycle_id or call_id
UI -> UI : atomically claim exactly one unresolved cycle
UI -> Operator : approval_resolved
UI --> Session : decision + optional feedback
UI -> Storage : stamp tracked verdicts with operator decision
else every item bypassed / auto-approved
UI -> Operator : tool_info with exact auto_approve_reason
UI --> Session : approved
end
== Lifecycle ==
note right of UI
Parallel task agents may register several ApprovalCycles. Each cycle owns
its own Event and result slot. A legacy selector-less decision targets the
oldest cycle; double resolution is a no-op. Cached LLM verdicts carry their
judge generation, so reused provider call ids cannot satisfy a new cycle.
end note
note over Session, Judge
**Lazy initialization:**
IntentJudge created on first approval if judge_config.enabled.
Re-uses session's provider/client by default (self-consistency).
Cross-model: separate provider/client from [judge] config.
== Cancellation boundary ==
**Sub-agent exemption:**
Task sub-agents skip intent validation entirely.
opt Stop / close / force-successor
Session -> Judge : abort all judge events owned by the cancelled operation
Session -> UI : resolve_all_approvals(False, "cancelled")
UI -> UI : block new admission leases; wait for admitted bundles;\nclaim only cycles whose cancellation witness is aborted
UI -> Operator : one cancelled resolution per claimed cycle
note over Session, UI
A Stop can win before cycle registration, during publication, or while a
click resolves. The witness + admission sweep makes exactly one terminal
outcome visible; a successor generation's new cycle is not swept.
end note
end
**Output guard:**
Runs when judge_config.output_guard is true (default).
Credential redaction when judge_config.redact_secrets is true.
note over Judge
Normal operator resolution does not necessarily cancel judge inference.
With cancel_on_approval=false, the daemon finishes and late verdicts remain
auditable. With it enabled, the batch event stops remaining judge work.
end note
**Storage:**
intent_verdicts table (migration 012), output_assessments table
(migration 022). Both queryable via admin API endpoints
(requires admin.judge permission). Skills store risk_level,
scan_report, scan_version for install-time risk assessment.
deactivate Judge
== Tool output guard ==
Session -> Session : execute admitted tools; truncate each result
Session -> Guard : evaluate(result, tool context, cancel event)
activate Guard
Guard -> Guard : heuristic checks first
opt LLM guard enabled and time remains
Guard -> Model : model_turn(output-guard lane, bounded prompt, cancel_ref)
Model --> Guard : structured verdict
end
Guard --> Session : assessment / redaction / warning
deactivate Guard
Session -> Session : re-check generation N before folding result
Session -> UI : output warning (no raw secret payload)
Session -> Storage : persist assessment + guarded Tool Turn metadata
note over Guard, Storage
Output-guard objects also pin model/config lanes. Replacement retires the
old object but lets admitted evaluations drain before its private client is
closed. A cancelled or superseded evaluation cannot fold into the successor
trajectory. Raw pre-redaction secrets are never stored in assessment rows.
end note
@enduml
+45 -39
View File
@@ -13,65 +13,68 @@ skinparam participant {
participant "ChatSession\n(session.py)" as Session <<session>>
participant "MemoryFacade\n(memory.py)" as Facade <<facade>>
participant "MemoryRelevance\n(memory_relevance.py)" as Relevance <<facade>>
participant "StorageBackend\n(SQLite)" as Storage <<storage>>
participant "StorageBackend\n(SQLite / PostgreSQL)" as Storage <<storage>>
participant "Server API\n(server.py)" as API <<api>>
participant "Console Admin\n(console/server.py)" as Admin <<api>>
participant "SDK Client\n(sdk/)" as SDK <<sdk>>
== Phase 1: Tool Path (session.send) ==
Session -> Session : _prepare_tool_calls()\nparse memory(action=...)
Session -> Session : pin acting principal\nparse memory(action=...)
note right
Tool schema: 4 actions
save, search, delete, list
Tool schema: 5 actions
save, get, search, delete, list
Auto-approved (no approval needed)
end note
Session -> Session : resolve live project access\nselect exact/inherited scope
Session -> Session : _exec_memory(item)
alt action = save
Session -> Facade : save_structured_memory(\nname, content, description,\nmem_type, scope, scope_id)
Session -> Session : require non-empty description
Session -> Facade : save_structured_memory_strict(\n..., require_active_project)
Facade -> Facade : normalize_key(name)
Facade -> Storage : create_structured_memory()
alt unique constraint violation
Storage --> Facade : IntegrityError
Facade -> Storage : get_structured_memory_by_name()
Storage --> Facade : existing row
Facade -> Storage : update_structured_memory()
end
Storage --> Facade : memory_id
Facade --> Session : (memory_id, old_content)
Session -> Session : _init_system_messages()\nrefresh BM25 context
Facade -> Storage : guarded atomic upsert\nON CONFLICT ... RETURNING
Storage --> Facade : (saved row, was_update)
Facade --> Session : saved row
Session -> Session : invalidate prefix/cache\naudit acting principal
end
alt action = get
Session -> Facade : get_structured_memory_by_name_strict()
Facade -> Storage : exact scoped-name lookup
Storage --> Session : full row / not found
end
alt action = search
Session -> Facade : search_structured_memories(\nquery, mem_type, scope,\nscope_id, limit)
Facade -> Storage : search_structured_memories()
Session -> Storage : search exact scope or\nactor-visible scope union
Storage --> Session : matched rows
end
alt action = delete
Session -> Facade : delete_structured_memory(\nname, scope, scope_id)
Facade -> Storage : delete_structured_memory()
Storage --> Session : bool (existed)
Session -> Session : _init_system_messages()\nrefresh BM25 context
Session -> Facade : delete_structured_memory_returning_strict()
Facade -> Storage : DELETE ... RETURNING
Storage --> Session : deleted row / not found
Session -> Session : invalidate + audit\nmark prefix dirty
end
== Phase 2: BM25 Relevance Injection ==
Session -> Session : _init_system_messages()\nevery conversation turn
Session -> Session : resolve acting principal\nand live project ACL
Session -> Session : _list_visible_memories(\nlimit=fetch_limit)
note right
**Scope resolution:**
1. global scope (always)
2. workstream scope (ws_id)
3. user scope (user_id, if auth)
Combined and deduplicated.
Interactive: global + workstream
+ acting user + readable project
Coordinator: acting user's coordinator
+ readable project
end note
Session -> Facade : list_structured_memories()\nper scope
Facade -> Storage : list_structured_memories()
Session -> Facade : list_visible_structured_memories()
Facade -> Storage : one visibility-union query
Storage --> Session : up to fetch_limit rows
Session -> Relevance : extract_recent_context(\nmessages, max_messages=3)
@@ -103,28 +106,31 @@ Session -> Session : inject into\nsystem message
== Phase 3: Server API Path ==
SDK -> API : GET /v1/api/memories\n?type=project&limit=20
API -> Facade : list_structured_memories()
Facade -> Storage : list_structured_memories()
SDK -> API : GET /v1/api/memories\n?type=general&limit=20
API -> API : bind scope to caller\ndefault global + caller user
API -> Storage : list visible rows
Storage --> API : rows
API --> SDK : {"memories": [...], "total": N}
SDK -> API : POST /v1/api/memories\n{name, content, ...}
API -> API : validate type, scope,\nname length, content length
API -> Facade : save_structured_memory()
Facade -> Storage : create / update
SDK -> API : POST /v1/api/memories\n{name, content, description, ...}
API -> API : validate type, scope,\nname/content/description
API -> API : reject internal scopes\nowner-bind workstream scope
API -> Facade : save_structured_memory_strict()
Facade -> Storage : atomic upsert
Storage --> API : memory row
API -> API : record_audit(actor)
API --> SDK : 201 (created) / 200 (updated)
SDK -> API : POST /v1/api/memories/search\n{query, type, ...}
API -> Facade : search_structured_memories()
Facade -> Storage : search_structured_memories()
API -> API : bind scope to caller
API -> Storage : search visible rows
Storage --> API : matched rows
API --> SDK : {"memories": [...], "total": N}
SDK -> API : DELETE /v1/api/memories/{name}\n?scope=global
API -> Facade : delete_structured_memory()
Facade -> Storage : delete row
API -> Facade : delete_structured_memory_returning_strict()
Facade -> Storage : DELETE ... RETURNING
API -> API : record_audit(actor)
API --> SDK : {"status": "ok"}
== Phase 4: Console Admin Path ==
@@ -141,7 +147,7 @@ Storage --> Admin : memory row
Admin --> SDK : memory JSON
SDK -> Admin : DELETE /v1/api/admin/memories/{id}
Admin -> Storage : delete_structured_memory_by_id()
Admin -> Storage : delete_structured_memory_by_id_returning()
Admin -> Admin : record_audit(\n"memory.delete")
Admin --> SDK : {"status": "ok"}
+8 -6
View File
@@ -13,7 +13,7 @@ skinparam participant {
participant "Server\n(main)" as Server <<session>>
participant "ConfigStore\n(config_store.py)" as Store <<config>>
participant "SettingsRegistry\n(settings_registry.py)" as Registry <<config>>
participant "StorageBackend\n(SQLite)" as Storage <<storage>>
participant "StorageBackend\n(SQLite / PostgreSQL)" as Storage <<storage>>
participant "Console Admin\n(console/server.py)" as Admin <<api>>
participant "SDK Client\n(sdk/)" as SDK <<sdk>>
participant "ChatSession\n(session.py)" as Session <<session>>
@@ -57,9 +57,10 @@ else key not in cache
Store --> Session : default value
end
note right of Session
Settings are captured once
at workstream creation.
Not re-read on every turn.
Most session settings are captured once
at workstream creation. Documented live readers
(including model.auth_fail_closed per mint)
apply immediately.
end note
== Phase 3: Admin API — List / Schema ==
@@ -128,8 +129,9 @@ Store -> Storage : get_system_settings_bulk(node_id)
Storage --> Store : all settings
Store -> Store : rebuild cache,\nswap atomically,\nincrement _version
note right
Existing sessions: unchanged
(frozen at creation time).
Most existing-session settings are unchanged
(frozen at creation time); documented
live readers apply immediately.
New sessions: pick up
updated values immediately.
end note
+5 -5
View File
@@ -88,7 +88,7 @@
<rect x="300" y="151" width="160" height="2" fill="#161b22"/>
<text x="380" y="178" text-anchor="middle" fill="#e6edf3" font-size="11" font-weight="600">Console</text>
<line x1="318" y1="190" x2="442" y2="190" stroke="#30363d" stroke-width="1"/>
<text x="380" y="208" text-anchor="middle" fill="#8b949e" font-size="9">hash-ring router</text>
<text x="380" y="208" text-anchor="middle" fill="#8b949e" font-size="9">FNV-1a rendezvous router</text>
<text x="380" y="223" text-anchor="middle" fill="#8b949e" font-size="9">cluster dashboard</text>
<text x="380" y="238" text-anchor="middle" fill="#8b949e" font-size="9">reverse proxy</text>
<line x1="318" y1="250" x2="442" y2="250" stroke="#30363d" stroke-width="1"/>
@@ -115,7 +115,7 @@
<rect x="608" y="162" width="184" height="28" rx="3" fill="#1c2128" stroke="#30363d" stroke-width="1"/>
<text x="700" y="180" text-anchor="middle" fill="#8b949e" font-size="9">turnstone-server :8080</text>
<!-- Tools label -->
<text x="700" y="206" text-anchor="middle" fill="#484f58" font-size="8">19 tools + MCP</text>
<text x="700" y="206" text-anchor="middle" fill="#484f58" font-size="8">model lanes + tools / MCP</text>
</g>
<!-- Node B -->
@@ -130,7 +130,7 @@
<rect x="608" y="282" width="184" height="28" rx="3" fill="#1c2128" stroke="#30363d" stroke-width="1"/>
<text x="700" y="300" text-anchor="middle" fill="#8b949e" font-size="9">turnstone-server :8080</text>
<!-- Tools label -->
<text x="700" y="326" text-anchor="middle" fill="#484f58" font-size="8">19 tools + MCP</text>
<text x="700" y="326" text-anchor="middle" fill="#484f58" font-size="8">model lanes + tools / MCP</text>
</g>
<!-- ==================== LLM PROVIDERS ==================== -->
@@ -171,7 +171,7 @@
<rect x="590" y="450" width="220" height="5" fill="#bc8cff"/>
<rect x="590" y="453" width="220" height="2" fill="#161b22"/>
<text x="700" y="476" text-anchor="middle" fill="#e6edf3" font-size="11" font-weight="600">PostgreSQL / SQLite</text>
<text x="700" y="492" text-anchor="middle" fill="#8b949e" font-size="9">conversations, memory, auth</text>
<text x="700" y="492" text-anchor="middle" fill="#8b949e" font-size="9">workstreams, turns, config, auth</text>
</g>
<text x="700" y="444" text-anchor="middle" fill="#bc8cff" font-size="9" font-weight="600" letter-spacing="2">STORAGE</text>
@@ -239,7 +239,7 @@
<!-- Routing rules at bottom, left-aligned -->
<text x="44" y="456" fill="#30363d" font-size="9" font-weight="600" letter-spacing="1">ROUTING</text>
<circle cx="44" cy="474" r="3" fill="#3fb950" opacity="0.6"/>
<text x="54" y="477" fill="#484f58" font-size="9">control plane: client &#x2192; console &#x2192; server node (hash-ring bucket lookup)</text>
<text x="54" y="477" fill="#484f58" font-size="9">control plane: client &#x2192; console &#x2192; server node (FNV-1a rendezvous placement)</text>
<circle cx="44" cy="494" r="3" fill="#58a6ff" opacity="0.6"/>
<text x="54" y="497" fill="#484f58" font-size="9">data plane: client &#x2192; server node (direct SSE, node_url from create response)</text>
<circle cx="44" cy="514" r="3" fill="#f47067" opacity="0.6"/>

Before

Width:  |  Height:  |  Size: 16 KiB

After

Width:  |  Height:  |  Size: 16 KiB

+2 -2
View File
@@ -1,3 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:881a8b9bce67b5af9a52d5e50deaa72351cd99c76f18aad5caeb2b61131ca1af
size 119798
oid sha256:b8c1460440784f07e30afea32d4ee17687627df46a24003d761ad79c2676a361
size 169499
+2 -2
View File
@@ -1,3 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:95dd5ebc899a1261d516686a5aa3319a7f45015d411302825fa28afbfc82e1ce
size 326766
oid sha256:66847ccdf10ef2bd04e93bc0d3924a56ce28462ec9e76a383b53aee4500755e8
size 631799
+2 -2
View File
@@ -1,3 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:9857db23fe3c4316d492073aac69c7e7558b1abe3b95ad7756d4a5933bd0ece7
size 620214
oid sha256:c74e99c530c3a8af9ab35b1e4d8c4fef0ea35c0c04cc35da7cf3588e71382057
size 661175
+2 -2
View File
@@ -1,3 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:d9c7769a600c38e6387390e6c42db8152e0f80c31d17b2218f7f636b71c7b868
size 355459
oid sha256:6261604cc8b75878a8704308929ea64d121cf26547019543fbe1b21cbe700415
size 189791
+2 -2
View File
@@ -1,3 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:23ca090b5656baaf70820cbe4ab6c27f0a3a02e18b4db0695614cf9489c23980
size 281440
oid sha256:1b3b7b745f6006ee73d4b31fa598faa0d71ffb74ce349ab08eb3ce09ded506c3
size 266294
+2 -2
View File
@@ -1,3 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:04d2069a9b5155ad1e7d842147fd78535ad9106d6856520439c33a9868a47499
size 156694
oid sha256:33dddd8cd8b53fa464cc8a4c899896fee32035d63e0669fe327969e3356349c7
size 329815
+2 -2
View File
@@ -1,3 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:a872556d111185f4531d1b68ee892b4ce5042d7ccf277e2cad08beb6932c9803
size 191144
oid sha256:16e3f3bfa0a6af637f7a9fb6765d594eb598428679c88a429c096c3dbae931e4
size 181185
+2 -2
View File
@@ -1,3 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:e7c3e40c10425d721f833390ae3531c09af501157fd3142531ba4eba86ff719d
size 197112
oid sha256:59f14b835665244f3d32981b6c1ac4c4380393a83cc519e271831622aa3f261a
size 197433
@@ -1,3 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:b047cdc318c505f0f0895a65e14c5cc7552716053055cca57fa0a77db150e618
size 255458
oid sha256:1a510eeaabf4ed8dab3b268c8f6bb5b7fef629a664361f7fb5614a6b498db36e
size 294415
+2 -2
View File
@@ -1,3 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:af5ab3126bf685afe68e24bc4b0ed97371d0ebdb77bf4d76c0331ab120580cc0
size 248809
oid sha256:ea86c6b6c68ed96f6fd18543d2e7a873f4715332cc3a7d4df167392f668a2de7
size 232403
@@ -1,3 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:ae4f79fb22600106f8cb0af4ba5586bb26ea5d57e27ef382fdc59b6549fdbd21
size 415473
oid sha256:edf02b97e1e1287ebba9e74b9858474dcda42e5542656e505ea133a5b2416f47
size 402992
+2 -2
View File
@@ -1,3 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:96176a09e65e90dadc32d5e9ed778423842be89204d2cf382225f53a90cfaf01
size 258547
oid sha256:aa9ca9a367c79159a26d1ec544b20fc7118a49082d72f7ee0edbaa85608d49fc
size 238991
+2 -2
View File
@@ -1,3 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:79a690c466a5d6f6d4292d78a27b9474e9e9c1373fa80e17dfe37700238c8af8
size 382508
oid sha256:636a6b2fc1075e4863421e68b99efe7f6f6f62cedbcff36ef0934c055f39fd46
size 281161
+2 -2
View File
@@ -1,3 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:c89628ed917dfd576c1af75c68fe5fed9beadaaee9dcea7aa7a1643867c4f1b9
size 344323
oid sha256:137d6c91a34695c820d8b0a33fd753e79165604aa92bf2ac8480d3744b2ef844
size 305199
@@ -1,3 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:06fe076f0835a891e00afc804fd1805196ebde9fc0d34998c7873e87287f982b
size 346887
oid sha256:0455e0dec36ebb8bcfdadcf327a1dd24ddbdcdd8df211c08918180842497727e
size 318681
+33 -7
View File
@@ -102,13 +102,26 @@ the cluster runs mTLS; harmless otherwise), and `TURNSTONE_SEARXNG_URL` points
are the dev-stack defaults — match whatever you set in `.env` if you changed them.
To let a server on a **different** machine join, start the stack with
`TURNSTONE_HOST_IP=<this host's LAN IP>` — that binds PostgreSQL, the console
ACME endpoint, and SearxNG to that interface. Then on the remote box set the
three URLs above to that IP, and set `TURNSTONE_ADVERTISE_URL` to the **remote**
box's own IP (the address the console dials back). **Set a strong
`POSTGRES_PASSWORD` first** — `TURNSTONE_HOST_IP` exposes the database (and every
user account + API-token hash in it), the console API, and the unauthenticated
SearxNG to your network.
both `TURNSTONE_HOST_IP=<this host's LAN IP>` and
`TURNSTONE_ACME_EXTERNAL_URL=http://<this host's LAN IP>:8090/acme`. The first
binds PostgreSQL, the console ACME endpoint, and SearxNG to that interface; the
second makes every URL in the ACME directory routable from the remote node (the
full value must include the `/acme` mount). Set the same
`TURNSTONE_ACME_EXTERNAL_URL` on the remote node so its authenticated ACME
client can pin that credential destination. Set `TURNSTONE_CONSOLE_URL` and
`TURNSTONE_SEARXNG_URL` to the compose host's IP, but set
`TURNSTONE_ADVERTISE_URL=http://192.0.2.10:8080` to the **remote** box's own
address. A resolvable DNS name works too. IPv6 literals must be bracketed in
URLs, for example `http://[2001:db8::10]:8080`; Turnstone enrolls literal
addresses as IP SANs rather than numeric DNS SANs.
Use a trusted LAN or VPN address and firewall `:8090` to enrolling nodes. ACME
signing routes require a dedicated short-lived service JWT, but direct bootstrap
is still plain HTTP/TOFU: a bearer token provides authentication, not transport
confidentiality or protection from an active on-path attacker. **Set a strong
`POSTGRES_PASSWORD` first** — `TURNSTONE_HOST_IP` also exposes the database (and
every user account + API-token hash in it), the console API, and the
unauthenticated SearxNG to your network.
To run the bare-metal node as a hardened, persistent service instead of by hand,
use the systemd units in [`deploy/systemd/`](../deploy/systemd/).
@@ -148,6 +161,11 @@ certs via the console's ACME endpoint:
docker compose -f turnstone/deploy/compose.yaml -f deploy/docker-compose.tls.yml up
```
The overlay publishes the console's plain-HTTP bootstrap/API port on
`TURNSTONE_CONSOLE_HTTP_BIND` (default `127.0.0.1`). For a cross-host node, set
that to a trusted LAN/VPN address, set `TURNSTONE_ACME_EXTERNAL_URL` to the same
address plus `/acme`, and firewall the port to enrolling nodes.
See [tls.md](tls.md) for details.
## Configuration
@@ -186,6 +204,12 @@ overrides.
> put [PgBouncer](pgbouncer.md) (transaction pooling) between turnstone and
> PostgreSQL.
> **Lifecycle upgrade:** the release that introduces hidden deferred-create
> reservations must be deployed as a coordinated cohort across every server
> sharing PostgreSQL; older processes do not understand `state='creating'`.
> Drain create traffic until the cohort is upgraded. See
> [PgBouncer: deferred workstream creation](pgbouncer.md#upgrade-note-deferred-workstream-creation).
### Ports
Both stacks publish Caddy (dashboard) and PostgreSQL; the dev stack additionally
@@ -200,6 +224,8 @@ Caddy or proxied by the console:
| `POSTGRES_PORT` | `5432` | Host port for PostgreSQL (for bare-metal joins) |
| `SEARXNG_API_PORT` | `8081` | Host port for the SearxNG API a bare-metal node's `web_search` dials (dev stack) |
| `TURNSTONE_HOST_IP` | `127.0.0.1` | Interface PostgreSQL, the console ACME endpoint, and SearxNG bind on (dev stack). Set to this host's LAN IP so a bare-metal node on **another machine** can reach them — set a strong `POSTGRES_PASSWORD` first (it also exposes the DB and the unauthenticated SearxNG to your network). |
| `TURNSTONE_CONSOLE_HTTP_BIND` | `127.0.0.1` | Production TLS-overlay interface for the console's plain-HTTP bootstrap/API listener. Use only a trusted LAN/VPN address and firewall it to enrolling nodes. |
| `TURNSTONE_ACME_EXTERNAL_URL` | request-derived | Canonical externally reachable ACME responder base, including the final `/acme` mount (for example `http://192.0.2.1:8090/acme`). Set it on the console and clients for cross-host mTLS: the console advertises it, while clients pin it as an allowed enrollment-JWT destination. A reverse-proxy prefix is supported only when the proxy maps it to Turnstone's internal `/acme` mount. |
| `POSTGRES_BIND` | `127.0.0.1` | Production stack (`turnstone/deploy/compose.yaml`) only: interface PostgreSQL binds on; set to the host's LAN IP for remote joins. |
### Channel gateway
+11 -8
View File
@@ -177,7 +177,8 @@ Runs a complete multi-turn conversation:
1. Appends the user message.
2. Checks `_cancelled` event — stops if set (timeout cleanup).
3. Calls the model API (non-streaming).
3. Calls the model through the production streaming provider path and drains
the result.
4. If tool calls are returned, executes them (with stdout suppressed) and
logs each call to `self.tool_call_log`.
5. Repeats up to `max_turns` or until the model responds without tool calls.
@@ -190,14 +191,15 @@ Parallel tool calls are capped at 10 per turn to prevent degenerate repetition.
Each test runs in a `ThreadPoolExecutor(max_workers=1)` with a per-test
timeout (`--test-timeout`). Each attempt gets its own `OpenAI` client with
a matching httpx read timeout. On timeout, three layers of defense prevent
zombie connections:
a matching per-read HTTP transport timeout. Because a trickling stream can
continually reset that read timeout, three layers bound the harness and stop
follow-on work:
1. **httpx timeout**: Per-request read timeout aborts the HTTP call and
releases the server slot.
1. **Executor wall clock**: The harness stops waiting after `--test-timeout`.
2. **`_cancelled` event**: Prevents the orphan thread from starting new turns.
3. **`run_client.close()`**: Closes the connection pool to abort any
in-flight request.
3. **`run_client.close()`**: Retires the connection pool and prevents reuse.
HTTPX2 does not promise that cross-thread client closure immediately aborts
an active body read; that read unwinds on its next wire event or read timeout.
### Retry Logic
@@ -214,7 +216,8 @@ Each test case runs in isolation:
1. A fresh temp directory is created.
2. Setup files are written to the temp directory.
3. The working directory is changed to the temp directory.
4. A per-attempt `OpenAI` client is created with httpx timeout matching `--test-timeout`.
4. A per-attempt `OpenAI` client is created with an HTTP transport timeout
matching `--test-timeout`.
5. A new `HeadlessSession` is created with the current developer prompt.
6. `send_headless()` runs the user prompt through the conversation loop.
7. The tool log is scored against expected actions.
+8 -5
View File
@@ -16,15 +16,18 @@ The permission model has two layers:
2. **Permissions** (granular) — named permission strings checked per-endpoint by
`require_permission()`.
**Built-in roles** (seeded by migration 008):
**Built-in roles** (seeded by migration 008 and extended by later feature
migrations):
| Role | Permissions |
|------|-------------|
| admin | read, write, approve, admin.users, admin.roles, admin.orgs, admin.policies, admin.skills, admin.audit, admin.usage, admin.schedules, admin.watches, tools.approve, workstreams.create, workstreams.close |
| operator | read, write, workstreams.create, workstreams.close |
| admin | Admin-default baseline: ordinary admin, lifecycle, tool-approval, coordinator, project, and persona capabilities. Explicit opt-in capabilities such as `model.skills.write` remain ungranted. |
| operator | read, write, workstreams.create, workstreams.close, conversation.modify |
| viewer | read |
Custom roles can be created with any subset of the valid permissions.
Custom roles can be created with any subset of the valid permissions. Built-in
role permission overrides can grant or revoke individual capabilities, so the
admin console is authoritative for the effective set on a deployment.
The `persona.create` / `persona.read` / `persona.write` family gates
persona administration; migration `063` seeds all three onto
`builtin-admin`, and any role can be granted them through the standard
@@ -67,7 +70,7 @@ etc.) since workstream templates were merged into the skills system in v0.8.0.
workstreams, concatenated in alphabetical order by name. Use name prefixes
(e.g. `01-safety`, `02-style`) to control ordering.
- **Explicit selection**: `--skill <name>` CLI flag, `skill` field on
`POST /v1/api/workstreams/new`, console creation modal dropdown, scheduled task
`POST /v1/api/workstreams/new`, console launcher dropdown, scheduled task
config, and channel adapter config. An explicit skill *replaces* defaults.
- **Variables**: Three built-in placeholders resolved at load time:
`{{model}}` (active model name), `{{ws_id}}` (workstream ID),
+103 -64
View File
@@ -17,7 +17,9 @@ evaluation:
read-only tool access. Runs on a daemon thread and delivers its verdict
progressively.
The verdict is purely advisory -- the user always makes the final decision.
The verdict is advisory by default. The opt-in Smart Approvals mode can use a
completed, high-confidence LLM `approve` verdict to make the decision
automatically under the fail-closed rules below.
The heuristic verdict is attached to the `approve_request` SSE event immediately.
The LLM verdict arrives later via an `intent_verdict` SSE event, allowing the
@@ -28,55 +30,77 @@ persisted to the `intent_verdicts` table for audit and future calibration.
## Configuration
### config.toml
### Server and console
```toml
[judge]
enabled = true
model = "" # empty = same as session model
provider = "" # empty = same as session provider
base_url = ""
api_key = ""
smart_approvals = false # auto-approve high-confidence "approve" LLM verdicts (opt-in)
confidence_threshold = 0.95 # Smart Approvals auto-approve bar (LLM recommendation=approve)
max_context_ratio = 0.5 # max % of judge context window for history
timeout = 120.0 # seconds (generous for local models)
read_only_tools = true # judge can use read_file/list_directory
cancel_on_approval = false # stop judging remaining tool calls once user decides
Server and console workstreams read database-backed `judge.*` settings from
the settings registry. Edit them at **Admin → Judge** or through the admin
settings API; changes take effect for the next judge batch without a restart.
The principal settings are:
```text
judge.enabled = true
judge.model = "" # empty = same alias as the session
judge.smart_approvals = false # opt-in automatic approval
judge.confidence_threshold = 0.95 # Smart Approvals confidence bar
judge.max_context_ratio = 0.5 # fraction of judge context used for history
judge.timeout = 120.0 # per judge turn and Smart Approvals wait
judge.parallel_evaluations = 1 # concurrent calls within one batch, 1-16
judge.read_only_tools = true # permit read_file/list_directory evidence
judge.cancel_on_approval = false # stop unfinished calls when the gate resolves
```
`parallel_evaluations = 1` preserves serial evaluation. Raising it reduces the
latency of wide tool-call batches. The selected judge model alias's
`max_concurrency` remains the process-wide generation ceiling, so it can reduce
the actual overlap across judge batches and other roles using that alias.
### Smart Approvals
With `smart_approvals = true` (off by default) a tool call is approved
automatically — no operator prompt — when the intent judge's **LLM** verdict
recommends `approve` with confidence at or above `confidence_threshold`. Every
other outcome still reaches a human: `review` / `deny` recommendations,
confidence below the threshold, judge errors or timeouts (`llm_fallback`), and
any call the deterministic heuristic rules explicitly flagged `deny` or
`critical`. That heuristic floor blocks only those explicit danger verdicts — it
is **not** a general "never lower the heuristic" rule: the heuristic's default
for an unmatched tool is `review`, and letting a confident LLM `approve` upgrade
a `review` is exactly what Smart Approvals is for. Only `deny` / `critical`
findings are off-limits to auto-approval. Requires the judge to be enabled;
auto-approved calls are tagged `smart_approval` in the dashboard and audit trail.
Smart Approvals applies to the web and coordinator surfaces, not the interactive
CLI.
With `smart_approvals = true` (off by default), a pending batch is approved
automatically — no operator prompt — only when **every** call has a completed
LLM verdict recommending `approve` at or above `confidence_threshold`. The
decision is batch-atomic: one uncertain sibling sends the entire parallel batch
to a human rather than executing the safe-looking subset piecemeal.
All fields are optional. The judge is enabled by default; use `enabled = false`
(or `--no-judge` on the command line) to disable it.
Every other outcome reaches a human: `review` / `deny` recommendations,
confidence below the threshold, judge errors or timeouts (`llm_fallback`), a
missing/duplicate call ID, an unjudged sibling, and any call the deterministic
heuristic rules explicitly flagged `deny` or `critical`. That heuristic floor
blocks only explicit danger verdicts — it is **not** a general "never lower the
heuristic" rule. The heuristic's default for an unmatched tool is `review`, and
letting a confident LLM upgrade that default is the feature's purpose.
The Smart Approvals enabled flag, threshold, and bounded verdict wait are
captured as one immutable snapshot when each gate batch starts. A settings
reload takes effect on the next batch, while concurrent main-loop and
task-agent gates cannot mix fields from different reload generations. Stop
wakes a batch still waiting for verdicts and is linearized against the final
auto-approval commit: if Stop wins, no `smart_approval` decision or audit row
is recorded for tools that did not cross the gate.
The verdict wait is capped by the snapshot's `judge.timeout`; the judge may
continue evaluating advisory verdicts after that gate falls back to a human.
Requires the judge to be enabled. Auto-approved calls are tagged
`smart_approval` in the dashboard and audit trail. Smart Approvals applies to
the web and coordinator surfaces, not the interactive CLI.
The judge is enabled by default. Disable `judge.enabled` in the admin Judge
settings, or use `--no-judge` in the interactive CLI.
### CLI flags
```
--judge / --no-judge Enable/disable (default: enabled)
--judge-model MODEL Model for judge
--judge-provider PROVIDER Provider for judge
--judge-model ALIAS Registered model alias for judge
--judge-timeout SECONDS LLM judge timeout (default: 120)
--judge-parallel-evaluations N Concurrent evaluations per batch, 1-16 (default: 1)
--judge-confidence FLOAT Confidence threshold, 0-1 (default: 0.95)
```
(Smart Approvals is configured via `[judge] smart_approvals` / the admin Judge
settings, not a CLI flag — the interactive CLI prompts for approval directly.)
The same five values can be placed in the CLI's `config.toml` `[judge]`
section. Smart Approvals is configured through the server/console admin Judge
settings, not a CLI flag—the interactive CLI prompts for approval directly.
CLI flags override `config.toml` values.
@@ -87,20 +111,19 @@ CLI flags override `config.toml` values.
- **Default (self-consistency)**: When `model` is empty, the session model
evaluates its own tool calls. Research shows self-consistency achieves
comparable accuracy to multi-agent debate at a fraction of the cost.
- **Cross-model**: Use a different model for the judge (e.g. local model for
the session, commercial model for the judge). Set `model` and `provider`
in the `[judge]` config section, or use `--judge-model` / `--judge-provider`
CLI flags.
- **Cross-provider**: When both `model` and `provider` are set, the judge
creates its own LLM client. You can optionally specify `base_url` and
`api_key` for non-default endpoints.
- **Google models**: The judge supports `google` as a provider. Note that
read-only tools are disabled for Google models (the Gemini API requires
`thought_signature` in tool call round-trips which the judge's normalized
format does not preserve).
- **Cross-model**: Register the desired model in the Models tab, then set
`judge.model` to that alias (or pass `--judge-model ALIAS` to the CLI).
- **Cross-provider**: A model alias carries its provider, endpoint, and
credential configuration together, so a judge alias may use a different
provider from the session without separate judge connection settings.
- **Google models**: The judge supports `google` aliases, including read-only
evidence tools. Provider-native reasoning state such as Gemini
`thought_signature` stays attached to the pinned model lane across evidence
turns.
The judge creates a fresh HTTP client for each evaluation run and closes it
when done, avoiding stale connection issues across runs.
The judge creates one fresh HTTP client per active batch worker and closes each
when that worker finishes, avoiding cross-thread client sharing and stale
connections across runs.
If the LLM judge fails or returns no verdict, a fallback verdict with tier
`llm_fallback` is delivered via the callback, ensuring the UI always receives
@@ -232,17 +255,25 @@ calls for approval, it calls `_evaluate_intent()` which:
4. Attaches each heuristic verdict to its item as `_heuristic_verdict`
5. The daemon thread runs the LLM judge and delivers results via `ui.on_intent_verdict()`
The daemon evaluates items sequentially, so a large parallel batch can outlive
its approval gate. With `cancel_on_approval = false` (the default) the daemon
runs every item to completion: verdicts that land after the operator decided
still stream to the UI and persist, stamped with the decision. The daemon is
aborted only when the next tool batch supersedes it or the session closes —
then each unfinished item degrades to an `llm_fallback` verdict. With
`cancel_on_approval = true` the abort additionally fires the moment the gate
resolves, trading verdict completeness for inference savings — recommended
when the judge shares a single local inference backend with the session model,
where a large batch's remaining judge calls would otherwise compete with the
next turn's completion.
The daemon coordinates up to `parallel_evaluations` independent workers for
one batch. Completed verdicts stream to the UI as workers finish, and every call
still receives exactly one LLM or `llm_fallback` verdict. The default of 1 keeps
the historical serial behavior; a higher value collapses a wide batch toward
`ceil(batch size / workers)` judge-call intervals. A smaller positive model
alias capacity also bounds the worker count, avoiding surplus threads queued at
the same admission gate.
With `cancel_on_approval = false` (the default) the daemon runs every item to
completion: verdicts that land after the operator decided still stream to the
UI and persist, stamped with the decision. A newer main-loop batch, session
close, or explicit Stop retires the old generation; unfinished items degrade
to `llm_fallback` verdicts. A judge/model binding or parallelism edit prevents
reuse on the next batch, while already-started calls stay pinned to the binding
and worker count they began with. With `cancel_on_approval = true`, an ordinary
gate decision additionally aborts unfinished work, trading verdict completeness
for inference savings—recommended when the judge shares a single local
inference backend with the session model. Explicit Stop always cancels every
live judge generation, regardless of this preference.
Verdicts that arrive after a *newer batch* has replaced the judge generation
are withheld from the live surfaces (a reused call_id must never ride a stale
@@ -258,6 +289,15 @@ siblings would otherwise make each other's verdicts look stale); per-cycle
generation checks enforce staleness instead, and `judge.cancel_on_approval`
fires per gate exactly like the main loop.
Several parallel task agents can therefore leave several approval cycles live
on one workstream. Each cycle owns its event, result, verdict set, and
`cycle_id`; a decision targets exactly one cycle by `cycle_id` or member
`call_id` (selector-less legacy clients resolve the oldest). Workstream Stop or
close performs a workstream-wide denial sweep over all cycles belonging to the
cancelled operation. A force-cancel successor's newly registered cycle carries
a fresh operation witness and is not accidentally denied by the predecessor's
late sweep.
---
## Storage and Audit
@@ -415,13 +455,12 @@ Redaction types: `api_key`, `private_key`, `password`, `secret`.
### Configuration
```toml
[judge]
output_guard = true # enable output evaluation (default)
redact_secrets = true # auto-redact detected credentials (default)
```text
judge.output_guard = true # enable output evaluation (default)
judge.redact_secrets = true # auto-redact detected credentials (default)
```
Configurable at runtime via the admin Settings tab.
Configure both at runtime through the admin Judge settings.
### Merge semantics (heuristic + LLM judge)
+94 -42
View File
@@ -20,7 +20,7 @@ Each memory has three dimensions:
| Type | Purpose |
|-------------|------------------------------------------------------------|
| `user` | User preferences, conventions, working style |
| `project` | Project-specific knowledge, architecture, patterns |
| `general` | General knowledge, architecture, patterns |
| `feedback` | Corrections, lessons learned, things to avoid |
| `reference` | Reference material, documentation, specifications |
@@ -31,25 +31,37 @@ Each memory has three dimensions:
| `global` | Visible to all workstreams and users |
| `workstream` | Visible only within the originating workstream |
| `user` | Follows the authenticated user across workstreams |
| `coordinator` | Coordinator sessions only; follows the user across coordinators |
| `coordinator` | Coordinator sessions only; follows the acting user |
| `project` | Shared by workstreams attached to one active project |
A memory's identity is the tuple `(name, scope, scope_id)`. Saving a memory
with the same identity upserts -- updating content while preserving the ID.
### Coordinator scope
### Inherited target and coordinator scope
Coordinator sessions are isolated to a single scope: `coordinator`, keyed by
the coordinator's creator `user_id`. It is durable -- every coordinator
session the same user runs (including concurrent ones) shares one
orchestration namespace, so procedures and lessons survive close/reopen.
Name-based operations use one inherited target when `scope` is omitted:
- An attached active project selects `project` for `save`, `get`, and
`delete`.
- Read-only project access permits `get`, but `save` and `delete` fail. They do
not fall back to a broader namespace.
- Without a project, interactive sessions select `global`; coordinator
sessions select `coordinator`.
A valid explicit scope selects exactly that scope. `search` and `list` are the
only actions that span every visible scope when `scope` is omitted.
Each coordinator's private `coordinator` namespace is keyed by the acting
user's `user_id`. It is durable -- every coordinator session that user runs
(including concurrent ones) shares one orchestration namespace, so procedures
and lessons survive close/reopen.
Isolation is bidirectional and enforced by session kind, not by secrecy of
the scope id:
- A coordinator session can read and write **only** `coordinator`-scope rows.
It never sees `global`/`workstream`/`user` memories, so content written by
interactive sessions (which routinely ingest untrusted MCP/attachment
output) cannot reach a coordinator's system message.
- A coordinator session sees its acting user's `coordinator` scope and, when
attached, the shared `project` scope. It never sees
`global`/`workstream`/`user` memories.
- Interactive sessions -- including a coordinator's own children, which share
its `user_id` -- are rejected from the `coordinator` scope on every memory
action. Children cannot plant rows the parent coordinator would read.
@@ -64,12 +76,13 @@ coordinator cannot be constructed, so the scope id is always a real user.
On every conversation turn, the system:
1. Fetches up to `fetch_limit` memories visible in the current scope
2. Extracts context from the last 3 user messages
3. Scores memories against that context using a BM25 index
4. Injects the top `relevance_k` memories into the system message as
1. Resolves the acting principal and their live project access
2. Fetches up to `fetch_limit` memories across that visibility envelope
3. Extracts context from the last 3 user messages
4. Scores memories against that context using a BM25 index
5. Injects the top `relevance_k` memories into the system message as
`<memories>` XML tags
5. Appends a hint telling the model how many memories are in scope
6. Appends a hint telling the model how many memories are in scope
This means the model always has its most relevant memories available without
explicit recall -- but can still use `memory(action='search')` for deeper
@@ -107,19 +120,23 @@ All fields are optional. Defaults are shown above.
## Tool Usage
The `memory` tool supports four actions:
The `memory` tool supports five actions:
### save
Store or update a memory.
Every save is a complete write for the relevance summary: `description` must
be supplied and contain non-whitespace text on both creation and update.
Content-only updates are rejected.
```json
{
"action": "save",
"name": "project_architecture",
"content": "The project uses a hexagonal architecture with...",
"description": "Core architecture patterns",
"type": "project",
"type": "general",
"scope": "global"
}
```
@@ -128,9 +145,26 @@ Store or update a memory.
|---------------|----------|-------------|------------------------------------------|
| `name` | yes | -- | Snake_case identifier (max 256 chars) |
| `content` | yes | -- | Memory content (max `max_content` chars) |
| `description` | no | `""` | Short description for relevance matching |
| `type` | no | `"project"` | One of: user, project, feedback, reference |
| `scope` | no | `"global"` | One of: global, workstream, user |
| `description` | yes | -- | Non-empty relevance summary, required on create and update |
| `type` | no | `"general"` | One of: user, general, feedback, reference |
| `scope` | no | inherited | Kind-valid scope; see inherited target above |
### get
Retrieve the full content of one memory by name.
```json
{
"action": "get",
"name": "project_architecture",
"scope": "project"
}
```
| Parameter | Required | Default | Description |
|-----------|----------|-----------|----------------------------|
| `name` | yes | -- | Memory name to retrieve |
| `scope` | no | inherited | Exact scope to query |
### search
@@ -140,7 +174,7 @@ Find memories by query (BM25 full-text search).
{
"action": "search",
"query": "authentication patterns",
"type": "project",
"type": "general",
"limit": 10
}
```
@@ -167,7 +201,7 @@ Remove a memory by name.
| Parameter | Required | Default | Description |
|------------|----------|------------|--------------------------|
| `name` | yes | -- | Memory name to delete |
| `scope` | no | `"global"` | Scope of the memory |
| `scope` | no | inherited | Exact scope to delete |
### list
@@ -197,6 +231,12 @@ Four endpoints on the server for programmatic memory access.
List memories with optional filters.
Without `scope`, the response is restricted to `global` plus the authenticated
caller's `user` namespace. The public API accepts only `global`, `user`, and
`workstream`; internal `project` and `coordinator` namespaces remain available
through the session tool and admin API. Explicit `workstream` access requires
its persisted owner (or a service token).
**Query parameters:**
| Parameter | Type | Required | Default | Description |
@@ -204,10 +244,10 @@ List memories with optional filters.
| `type` | string | no | `""` | Filter by memory type |
| `scope` | string | no | `""` | Filter by scope |
| `scope_id` | string | no | `""` | Filter by scope ID |
| `limit` | int | no | `100` | Max results (capped at 200) |
| `limit` | int | no | `100` | Max results (1-200) |
When `scope=user` and `scope_id` is omitted, the authenticated user's ID is
used automatically.
When `scope=user`, the authenticated user's ID is used automatically and a
different supplied ID is rejected. `scope=workstream` requires `scope_id`.
**Response:** `200`
@@ -218,7 +258,7 @@ used automatically.
"memory_id": "a1b2c3d4-e5f6-...",
"name": "project_architecture",
"description": "Core architecture patterns",
"type": "project",
"type": "general",
"scope": "global",
"scope_id": "",
"content": "The project uses a hexagonal architecture...",
@@ -236,6 +276,9 @@ used automatically.
Save or upsert a structured memory.
`description` is mandatory for both creates and updates and must contain
non-whitespace text. The API rejects content-only updates.
**Request body:**
```json
@@ -243,7 +286,7 @@ Save or upsert a structured memory.
"name": "deployment_process",
"content": "Deploy via GitHub Actions. Staging auto-deploys on push to main.",
"description": "CI/CD deployment workflow",
"type": "project",
"type": "general",
"scope": "global",
"scope_id": ""
}
@@ -253,8 +296,8 @@ Save or upsert a structured memory.
|--------------|--------|----------|-------------|--------------------------------------|
| `name` | string | yes | -- | Memory name (max 256 chars) |
| `content` | string | yes | -- | Memory content (max 65536 chars) |
| `description`| string | no | `""` | Short description for search ranking |
| `type` | string | no | `"project"` | One of: user, project, feedback, reference |
| `description`| string | yes | -- | Non-empty relevance summary, required on create and update |
| `type` | string | no | unset | user, general, feedback, or reference |
| `scope` | string | no | `"global"` | One of: global, workstream, user |
| `scope_id` | string | no | `""` | Scope qualifier (auto-resolved for user scope) |
@@ -265,7 +308,7 @@ Save or upsert a structured memory.
"memory_id": "a1b2c3d4-e5f6-...",
"name": "deployment_process",
"description": "CI/CD deployment workflow",
"type": "project",
"type": "general",
"scope": "global",
"scope_id": "",
"content": "Deploy via GitHub Actions...",
@@ -281,7 +324,10 @@ same `(name, scope, scope_id)` already existed.
| Status | Condition |
|--------|------------------------------------|
| 400 | Missing name, empty content, invalid type/scope, content too long |
| 400 | Invalid input, scope, scope ID, or limit |
| 403 | Cross-user or non-owner workstream access |
| 404 | Explicit workstream does not exist |
| 500 | Storage mutation failed |
---
@@ -290,12 +336,15 @@ same `(name, scope, scope_id)` already existed.
Search memories by query. Uses POST for the request body but is non-mutating
(requires only `read` scope).
An omitted scope searches the same caller-bound `global` + `user` envelope as
the list endpoint. It never means every row in the table.
**Request body:**
```json
{
"query": "authentication",
"type": "project",
"type": "general",
"scope": "",
"scope_id": "",
"limit": 20
@@ -308,7 +357,7 @@ Search memories by query. Uses POST for the request body but is non-mutating
| `type` | string | no | `""` | Filter by type |
| `scope` | string | no | `""` | Filter by scope |
| `scope_id` | string | no | `""` | Filter by scope ID |
| `limit` | int | no | `20` | Max results (capped at 50) |
| `limit` | int | no | `20` | Max results (1-50) |
**Response:** `200`
@@ -319,7 +368,7 @@ Search memories by query. Uses POST for the request body but is non-mutating
"memory_id": "a1b2c3d4-e5f6-...",
"name": "auth_patterns",
"description": "Authentication architecture",
"type": "project",
"type": "general",
"scope": "global",
"scope_id": "",
"content": "JWT tokens with HS256...",
@@ -337,6 +386,9 @@ Search memories by query. Uses POST for the request body but is non-mutating
Delete a memory by name and scope.
Deletes are atomic: the row used for the success result and audit event is the
row actually removed. A storage failure returns `500`, not a false `404`.
**Path parameters:**
| Parameter | Type | Description |
@@ -391,7 +443,7 @@ List memories across all scopes (no automatic scope resolution).
"memory_id": "a1b2c3d4-e5f6-...",
"name": "project_architecture",
"description": "Core architecture patterns",
"type": "project",
"type": "general",
"scope": "global",
"scope_id": "",
"content": "The project uses...",
@@ -440,7 +492,7 @@ Get a single memory by ID.
"memory_id": "a1b2c3d4-e5f6-...",
"name": "project_architecture",
"description": "Core architecture patterns",
"type": "project",
"type": "general",
"scope": "global",
"scope_id": "",
"content": "The project uses...",
@@ -497,13 +549,13 @@ with TurnstoneServer("http://localhost:8080", token="tok_xxx") as client:
"api_conventions",
"All endpoints use /v1/ prefix. JSON responses.",
description="API design patterns",
mem_type="project",
mem_type="general",
scope="global",
)
print(mem.memory_id)
# Search memories
results = client.search_memories("authentication", mem_type="project", limit=10)
results = client.search_memories("authentication", mem_type="general", limit=10)
for m in results.memories:
print(f"{m['name']}: {m['description']}")
@@ -524,7 +576,7 @@ with TurnstoneConsole("http://localhost:9090", token="tok_xxx") as admin:
result = admin.list_memories(scope="global", limit=100)
# Search
result = admin.search_memories("architecture", mem_type="project")
result = admin.search_memories("architecture", mem_type="general")
# Get by ID
mem = admin.get_memory("a1b2c3d4-e5f6-...")
@@ -548,14 +600,14 @@ const mem = await client.saveMemory({
name: "api_conventions",
content: "All endpoints use /v1/ prefix. JSON responses.",
description: "API design patterns",
type: "project",
type: "general",
scope: "global",
});
// Search memories
const results = await client.searchMemories({
query: "authentication",
type: "project",
type: "general",
limit: 10,
});
+8 -5
View File
@@ -124,11 +124,14 @@ allow_private_network = true
or via `TURNSTONE_OIDC_ALLOW_PRIVATE_NETWORK=true` (the env var wins
when both are set).
The opt-in admits private-range (RFC 1918), unique-local, CGNAT
(100.64/10 — tailnets), and loopback addresses. Link-local, multicast,
and reserved ranges stay refused even with the opt-in — cloud metadata
services (169.254.169.254) live there, and no legitimate IdP does. The
HTTPS requirement and the same-origin endpoint checks are unaffected.
The opt-in admits private-range (RFC 1918), unique-local, site-local,
CGNAT (100.64/10, where overlay VPNs commonly assign hosts), and
loopback addresses. Link-local, multicast, reserved ranges and known
cloud-metadata endpoints stay refused even with the opt-in — no
legitimate IdP lives there. An address is judged by what it actually
reaches, so an IPv6 transition address (NAT64, 6to4, Teredo) wrapping
an internal IPv4 is treated exactly as that IPv4 would be. The HTTPS
requirement and the same-origin endpoint checks are unaffected.
This knob only affects the login-flow IdP configured here. OAuth
endpoints advertised by remote MCP servers are untrusted input and are
+2 -2
View File
@@ -46,8 +46,8 @@ reads only the stamp:
`creative_mode` set are converted by migration `063` into full
`writer` stamps, so they resume as writing sessions rather than as
legacy defaults.
- Forking (`resume_ws` on create) resumes the source's stamped persona; the
fork does not re-resolve.
- Forking (`resume_ws` on create) clones the source's stamped persona into the
new workstream; the fork does not re-resolve it.
## Seed personas
+44 -8
View File
@@ -15,11 +15,19 @@ down to a small number of real database connections.
## Why PgBouncer works well with turnstone
All turnstone database operations are short-burst queries: acquire a
connection, execute 13 statements, commit, release. No operation holds
a connection for more than a few milliseconds. This makes **transaction
pooling mode** ideal — PgBouncer assigns a real connection only for the
duration of each transaction, then returns it to the pool.
Most turnstone database operations are short-burst queries: acquire a
connection, execute a small transaction, commit, release. Workstream forks are
the deliberate exception: they clone the source's checkpoint-bounded history
and configuration and retain its attachment references in one transaction.
PostgreSQL runs that clone at `SERIALIZABLE` isolation and retries serialization
or deadlock conflicts as a whole. A large fork can therefore hold its assigned
server connection longer than an ordinary message write.
This still makes **transaction pooling mode** the right fit — no operation
depends on server-session state, and PgBouncer returns the connection as soon
as the transaction finishes. Size and monitor the server pool with concurrent
fork traffic in mind rather than assuming every transaction completes in a few
milliseconds.
| Cluster size | Client connections (max) | PgBouncer server connections needed |
|--------------|------------------------|-------------------------------------|
@@ -143,9 +151,11 @@ PgBouncer (which then multiplexes to PostgreSQL):
| `TURNSTONE_DB_URL` | — | Connection URL (point at PgBouncer, not PostgreSQL directly) |
The default pool of 2 + 3 overflow = 5 connections per process is
intentionally small to support large clusters. You should not need to
increase this — turnstone's database operations are all short-burst
context-managed queries that hold connections for milliseconds.
intentionally small to support large clusters. Most deployments should not
need to increase it. If operators create many large forks concurrently, watch
PgBouncer's `cl_waiting` and PostgreSQL transaction latency before changing
the per-process pool; adding client-side connections cannot help once the
PgBouncer server pool is saturated.
SQLAlchemy `pool_pre_ping` is enabled, so stale connections (e.g. after
PgBouncer restarts) are automatically detected and replaced.
@@ -177,6 +187,32 @@ Key metrics to watch:
- **`sv_active`** — active server (PostgreSQL) connections. Should stay
below PostgreSQL `max_connections`.
Short `cl_waiting` spikes during large workstream forks can be normal. Sustained
waiters accompanied by long serializable transactions indicate fork/storage
load, not an SSE or HTTP client-pool problem.
---
## Upgrade note: deferred workstream creation
The workstream lifecycle now uses durable, hidden `state='creating'`
reservations while session construction, upload validation, and optional fork
cloning complete. Older server processes do not understand that private state:
against the same database they may resolve, list, open, or prune a reservation
before its new owner publishes it.
For the upgrade that introduces deferred creation, drain create traffic and
upgrade all server processes sharing the database as one cohort. Do not resume
creates until no older server process remains. The change needs no manual
schema migration, but it is not safe to treat mixed lifecycle implementations
as an ordinary rolling-upgrade state.
A `creating` row should be transient and absent from normal APIs and cluster
events. If one persists after a process crash, inspect the corresponding
`ws.create.*` and `session_mgr.commit_create.*` logs before cleanup. Do not
promote it to `idle` manually: its history, configuration, attachment
references, or lifecycle publication may be incomplete.
---
## Troubleshooting
+138 -29
View File
@@ -50,6 +50,7 @@ with TurnstoneServer("http://localhost:8080") as client:
import asyncio
from turnstone.sdk import AsyncTurnstoneServer
async def main():
async with AsyncTurnstoneServer("http://localhost:8080") as client:
await client.login(username="alice", password="s3cret")
@@ -58,6 +59,7 @@ async def main():
if event.type == "content":
print(event.text, end="", flush=True)
asyncio.run(main())
```
@@ -69,17 +71,18 @@ Both `TurnstoneServer` (sync) and `AsyncTurnstoneServer` (async) expose:
|----------|--------|---------|
| **Workstreams** | `list_workstreams()` | `ListWorkstreamsResponse` |
| | `dashboard()` | `DashboardResponse` |
| | `create_workstream(*, name, model, auto_approve, skill, persona, initial_message, attachments)` | `CreateWorkstreamResponse` |
| | `create_workstream(*, name, model, auto_approve, resume_ws, skill, persona, initial_message, project_id, attachments, ...)` | `CreateWorkstreamResponse` |
| | `close_workstream(ws_id)` | `StatusResponse` |
| **Attachments** | `upload_attachment(ws_id, filename, data, *, mime_type=...)` | `UploadAttachmentResponse` |
| | `list_attachments(ws_id)` | `ListAttachmentsResponse` |
| | `get_attachment_content(ws_id, attachment_id)` | `bytes` |
| | `delete_attachment(ws_id, attachment_id)` | `StatusResponse` |
| **Chat** | `send(message, ws_id)` | `SendResponse` |
| | `approve(*, ws_id, approved, feedback, always)` | `StatusResponse` |
| **Chat** | `send(message, ws_id, *, attachment_ids=None, client_send_id=None)` | `SendResponse` |
| | `approve(*, ws_id, approved, feedback, always, cycle_id, call_id)` | `ApproveResponse` |
| | `command(*, ws_id, command)` | `StatusResponse` |
| | `cancel(ws_id, *, force=False)` | `StatusResponse` |
| **Streaming** | `stream_events(ws_id)` | `Iterator[ServerEvent]` |
| | `cancel(ws_id, *, force=False)` | `CancelResponse` |
| **History** | `get_history(ws_id, *, limit=100)` | `WorkstreamHistoryResponse` |
| **Streaming** | `stream_events(ws_id, *, last_event_id=None, history_token=None)` | `Iterator[ServerEvent]` |
| | `stream_global_events()` | `Iterator[ServerEvent]` |
| **High-level** | `send_and_wait(message, ws_id, *, timeout, on_event)` | `TurnResult` |
| **Saved** | `list_saved_workstreams()` | `ListSavedWorkstreamsResponse` |
@@ -100,7 +103,7 @@ Both `TurnstoneConsole` (sync) and `AsyncTurnstoneConsole` (async) expose:
| | `workstreams(*, state, node, search, sort, page, per_page)` | `ClusterWorkstreamsResponse` |
| | `node_detail(node_id)` | `NodeDetailResponse` |
| | `snapshot()` | `ClusterSnapshotResponse` |
| | `create_workstream(*, node_id, name, model, initial_message, skill, persona)` | `ConsoleCreateWsResponse` |
| | `create_workstream(*, node_id, name, model, initial_message, skill, persona, resume_ws)` | `ConsoleCreateWsResponse` |
| **Schedules** | `list_schedules()` | `ListSchedulesResponse` |
| | `create_schedule(*, name, schedule_type, initial_message, ...)` | `ScheduleInfo` |
| | `get_schedule(task_id)` | `ScheduleInfo` |
@@ -125,12 +128,12 @@ SSE events are deserialized into typed dataclasses. Use `event.type` to discrimi
| Type | Class | Key Fields |
|------|-------|------------|
| `connected` | `ConnectedEvent` | `model`, `model_alias`, `skip_permissions` |
| `history` | `HistoryEvent` | `messages` |
| `user_turn` | `UserTurnEvent` | `ws_id`, `content`, `attachments`, `sender`, `source`, `client_send_ids`, `_event_id` |
| `content` | `ContentEvent` | `text` |
| `reasoning` | `ReasoningEvent` | `text` |
| `tool_info` | `ToolInfoEvent` | `items` |
| `approve_request` | `ApproveRequestEvent` | `items` |
| `tool_result` | `ToolResultEvent` | `call_id`, `name`, `output`, `is_error` |
| `approve_request` | `ApproveRequestEvent` | `cycle_id`, `items` |
| `tool_result` | `ToolResultEvent` | `call_id`, `name`, `output`, `is_error`, `preview`, `accepted`, `effect_status`, `_event_id` |
| `tool_output_chunk` | `ToolOutputChunkEvent` | `call_id`, `chunk` |
| `status` | `StatusEvent` | `prompt_tokens`, `total_tokens`, `pct`, `effort`, `cache_creation_tokens`, `cache_read_tokens` |
| `error` | `ErrorEvent` | `message` |
@@ -138,14 +141,92 @@ SSE events are deserialized into typed dataclasses. Use `event.type` to discrimi
| `stream_end` | `StreamEndEvent` | — |
| `state_change` | `StateChangeEvent` | `state``running`/`thinking`/`attention`/`idle`/`error` |
| `in_progress_snapshot` | `InProgressSnapshotEvent` | `content`, `reasoning` (one-shot mid-stream refresh resume) |
| `approval_resolved` | `ApprovalResolvedEvent` | `approved`, `feedback` |
| `approval_resolved` | `ApprovalResolvedEvent` | `cycle_id`, `call_ids`, `approved`, `feedback`, `always` |
| `cancelled` | `CancelledEvent` | — |
| `history_resync` | `HistoryResyncEvent` | `reason`, optional `ws_id` |
The Python server `send()` and console `coordinator_send()` methods accept an
optional `client_send_id`; TypeScript `send()` accepts the equivalent
`options.clientSendId`. Values match `[A-Za-z0-9_-]{1,128}`. The value is an
opaque optimistic-UI correlation token, not an idempotency key: reusing it
still creates distinct accepted turns and events.
Every upgraded listener on the shared workstream receives `UserTurnEvent`.
Originating panes use `client_send_ids` only to settle the exact optimistic
bubble, while peers render the accepted row once by `_event_id`. A
`message_queued` event carrying the token can establish acceptance even if the
POST acknowledgement is lost. History projects the same correlation alongside
the accepted user row. These tokens are not credentials: when sender and viewer
identities are both known, only a matching sender may settle local optimistic
state; a peer event still renders its canonical row.
The typed projection is negotiated with `?user_turn=1` on the per-workstream
SSE URL. Python `stream_events()` / `send_and_wait()` and TypeScript
`streamEvents()` / `sendAndWait()` set it automatically. Raw consumers that
omit it receive a backward-compatible `replay_truncated` repair signal instead
of the user row and must rebuild from `/history`; its pre-row cursor keeps the
repair retryable if that history request fails.
The browser-only final-tool upsert capability is `?tool_turn=1`. The bundled
Python and TypeScript SDK streaming helpers and channel adapters intentionally
do not negotiate it yet: they retain the executor-receipt `tool_result`
contract and do not own a transcript reducer. `ToolResultEvent` can deserialize
the accepted fields for direct/custom capable clients. Raw capable clients must
deduplicate `_event_id` and replace the newest matching call occurrence; raw
incapable clients receive the pre-row `tool_turn_projection_unsupported` repair
frame and rebuild from history. That staging deliberately prices in two costs
for incapable consumers. A raw client that treats every `replay_truncated`
frame as a rebuild trigger refetches `/history` once per accepted tool row —
one fetch per tool call on a long agentic turn; a client that wants tool
results incrementally should negotiate `tool_turn=1` and reduce, and the
bundled helpers (which ignore the frame rather than rebuild) stay correct
because their receipt-only view never depends on the accepted projection.
Second, only the accepted event carries post-execution output transforms, so a
receipt-rendering consumer (for example, a channel adapter posting the
executor receipt into a thread) keeps the pre-transform text; the accepted
projection is a transcript-consistency mechanism, not a wire confidentiality
boundary — see the API reference note on the preliminary `tool_result`.
Current servers bootstrap conversation history through
`GET /v1/api/workstreams/{ws_id}/history` before the SSE stream; they do not
emit a `history` event. `HistoryEvent` remains deserializable only for
compatibility with older servers. `get_history()` exposes the current REST
bootstrap response, including its optional cursor and one-shot handoff token.
### Caller-managed history handoff
The SDK supplies typed handshake primitives but intentionally does not own a
transcript renderer or reconnect policy. After rendering a successful history
response, pass its cursor and token to exactly one initial stream:
```python
from turnstone.sdk import HistoryResyncEvent
history = client.get_history(ws_id)
render(history.messages)
for event in client.stream_events(
ws_id,
last_event_id=history.cursor,
history_token=history.handoff_token,
):
if isinstance(event, HistoryResyncEvent):
# Stop this stream. The caller chooses when to fetch, render, and
# reconnect with a new history response.
break
apply_live_event(event)
```
`history_resync` means numeric replay cannot prove that the rendered limited
tail came from the same total accepted conversation-row prefix. Stop the
stream, fetch and render history again, and use only the new cursor/token pair.
A 503 history response raises `TurnstoneAPIError`; it is not authoritative, so
retain any existing transcript and do not open a tokenless replacement stream.
**Global events** (from `stream_global_events()`):
| Type | Class | Key Fields |
|------|-------|------------|
| `ws_state` | `WsStateEvent` | `ws_id`, `state`, `tokens`, `activity` |
| `ws_state` | `WsStateEvent` | `ws_id`, `state`, `tokens`, `activity`, `persistence_state` |
| `ws_activity` | `WsActivityEvent` | `ws_id`, `activity`, `activity_state` |
| `ws_rename` | `WsRenameEvent` | `ws_id`, `name` |
| `ws_closed` | `WsClosedEvent` | `ws_id` |
@@ -156,24 +237,30 @@ SSE events are deserialized into typed dataclasses. Use `event.type` to discrimi
|------|-------|------------|
| `node_joined` | `NodeJoinedEvent` | `node_id` |
| `node_lost` | `NodeLostEvent` | `node_id` |
| `cluster_state` | `ClusterStateEvent` | `ws_id`, `node_id`, `state`, `tokens` |
| `ws_created` | `ClusterWsCreatedEvent` | `ws_id`, `node_id`, `name` |
| `cluster_state` | `ClusterStateEvent` | `ws_id`, `node_id`, `state`, `tokens`, `persistence_state` |
| `ws_created` | `ClusterWsCreatedEvent` | `ws_id`, `node_id`, `name`, `persistence_state` |
| `ws_closed` | `ClusterWsClosedEvent` | `ws_id` |
| `ws_rename` | `ClusterWsRenameEvent` | `ws_id`, `name` |
| `snapshot` | `ClusterSnapshotEvent` | `nodes`, `overview`, `timestamp` |
Operator-facing workstream rows and rich state events expose only the sanitized
`persistence_state`: `healthy`, `pending`, `retrying`, or `conflict`. SDK types
treat it as optional for compatibility with older nodes; an omitted value means
`healthy`. Retry counts, storage errors, commit keys, and conversation content
are never part of this status surface.
### TurnResult
The `send_and_wait()` method returns a `TurnResult` that aggregates the full response:
```python
result = client.send_and_wait("Hello", ws_id, timeout=60)
result.content # Full text response
result.reasoning # Chain-of-thought (if shown)
result.tool_results # List of (tool_name, output) tuples
result.errors # Any error messages
result.ok # True if no errors and not timed out
result.timed_out # True if timeout expired
result.content # Full text response
result.reasoning # Chain-of-thought (if shown)
result.tool_results # List of (tool_name, output) tuples
result.errors # Any error messages
result.ok # True if no errors and not timed out
result.timed_out # True if timeout expired
```
### Attachments
@@ -183,9 +270,7 @@ Upload files to a workstream and attach them to the next user turn:
```python
# Upload separately, then send a message — attachments auto-attach
with open("screenshot.png", "rb") as f:
att = client.upload_attachment(ws.ws_id, "screenshot.png",
f.read(),
mime_type="image/png")
att = client.upload_attachment(ws.ws_id, "screenshot.png", f.read(), mime_type="image/png")
client.send("What's wrong in this screenshot?", ws.ws_id)
# Or attach at workstream-creation time (multipart upload)
@@ -195,9 +280,7 @@ with open("notes.txt", "rb") as f:
ws = client.create_workstream(
name="triage",
initial_message="Summarize the notes",
attachments=[AttachmentUpload(data=f.read(),
filename="notes.txt",
mime_type="text/plain")],
attachments=[AttachmentUpload(data=f.read(), filename="notes.txt", mime_type="text/plain")],
)
```
@@ -206,6 +289,26 @@ Limits: images ≤ 4 MiB (png/jpeg/gif/webp), text ≤ 512 KiB (UTF-8),
client so cluster-routed callers bind attachments to the owning node
before the request lands.
### Forking a workstream
`resume_ws` is the API's compatibility name for an atomic fork. It creates a
new workstream ID while the source remains unchanged:
```python
fork = client.create_workstream(
resume_ws=ws.ws_id,
name="analysis-branch",
initial_message="Try the alternative plan.",
)
assert fork.resumed
```
The server transaction clones the source's checkpoint-bounded history, saved
session configuration, persona, project, and attachment references. Do not
combine `resume_ws` with `attachments`; fork first, then upload to the new ID.
To rehydrate the original ID rather than branch it, call the server's
`POST /v1/api/workstreams/{ws_id}/open` endpoint.
### Error Handling
Non-2xx responses raise `TurnstoneAPIError`:
@@ -217,7 +320,7 @@ try:
client.send("hi", "bad_ws_id")
except TurnstoneAPIError as e:
print(e.status_code) # 404
print(e.message) # "Unknown workstream"
print(e.message) # "Unknown workstream"
```
---
@@ -242,8 +345,14 @@ const ws = await client.createWorkstream({ name: "demo" });
const result = await client.sendAndWait("Hello!", ws.ws_id);
console.log(result.content);
// Stream events
for await (const event of client.streamEvents(ws.ws_id)) {
// Render history, then use its one-shot hints on the initial stream.
const history = await client.getHistory(ws.ws_id);
render(history.messages);
for await (const event of client.streamEvents(ws.ws_id, {
lastEventId: history.cursor ?? undefined,
historyToken: history.handoff_token ?? undefined,
})) {
if (event.type === "history_resync") break; // caller refetches and reconnects
if (event.type === "content") {
process.stdout.write(event.text);
}
@@ -319,7 +428,7 @@ turnstone/sdk/ Python SDK (sub-package)
_base.py Shared httpx async client, auth, error handling
_sync.py Background event loop for sync wrappers
_types.py TurnResult + TurnstoneAPIError
events.py 38 SSE event dataclasses with type registry
events.py Typed SSE event dataclasses with type registry
server.py AsyncTurnstoneServer + TurnstoneServer
console.py AsyncTurnstoneConsole + TurnstoneConsole
+53 -20
View File
@@ -64,15 +64,17 @@ Scopes are hierarchical — higher scopes imply all lower ones.
### Path-to-scope mapping
| Method | Path pattern | Required scope |
|--------|-------------|----------------|
| GET | Any protected path | `read` |
| POST | `/api/command` | `write` |
| POST | `/api/workstreams/new`, `/api/cluster/workstreams/new` | `write` |
| POST | `/api/workstreams/{ws_id}/{send,cancel,close,delete,open,refresh-title,title,attachments}` | `write` |
| DELETE | `/api/workstreams/{ws_id}/send` (dequeue), `/api/workstreams/{ws_id}/attachments/{attachment_id}` | `write` |
| POST | `/api/workstreams/{ws_id}/approve` | `approve` |
| Any | `/api/admin/*` | `approve` |
| Method | Path pattern | Required scope | Additional RBAC gate |
|--------|-------------|----------------|----------------------|
| GET | Any protected path | `read` | Endpoint-specific where documented |
| POST | `/api/command` | `write` | Project tenancy on the target workstream |
| POST | `/api/workstreams/new`, `/api/cluster/workstreams/new` | `write` | `workstreams.create` or `admin.coordinator` |
| POST | `/api/workstreams/{ws_id}/close` | `write` | `workstreams.close` or `admin.coordinator` |
| POST | `/api/workstreams/{ws_id}/approve` | `approve` | `tools.approve` or `admin.coordinator` |
| POST | `/api/workstreams/{ws_id}/{rewind,retry}` | `write` | `conversation.modify` |
| POST | Other `/api/workstreams/{ws_id}/...` mutation endpoints | `write` | Project tenancy and endpoint-specific gates |
| DELETE | `/api/workstreams/{ws_id}/send` (dequeue), `/api/workstreams/{ws_id}/attachments/{attachment_id}` | `write` | Project tenancy on the target workstream |
| Any | `/api/admin/*` | `approve` | Matching `admin.*` permission |
Public paths bypass authentication entirely: `/`, `/health`, `/metrics`,
`/static/*`, `/shared/*`, `/docs`, `/openapi.json`, `/api/auth/login`,
@@ -84,7 +86,7 @@ Public paths bypass authentication entirely: `/`, `/health`, `/metrics`,
> See also: [Governance documentation](governance.md)
Scopes provide coarse endpoint-level access control. For finer-grained
enforcement, the governance layer adds 15 named permissions checked
enforcement, the governance layer adds named permissions checked
per-endpoint by `require_permission()`. Permissions are bundled into
roles; users are assigned roles via the `user_roles` join table.
@@ -98,8 +100,8 @@ Three built-in roles are seeded by migration 008:
| Role | Permissions |
|------|-------------|
| admin | All 15 permissions |
| operator | read, write, workstreams.create, workstreams.close |
| admin | Admin-default baseline (all ordinary admin and lifecycle permissions; explicitly opt-in capabilities remain ungranted) |
| operator | read, write, workstreams.create, workstreams.close, conversation.modify |
| viewer | read |
Custom roles can be created with any subset of the valid permissions.
@@ -107,6 +109,34 @@ Role creation and update validate permissions against a static allowlist.
Self-assignment is blocked, and assigning a role requires the caller to
hold a superset of the target role's permissions.
### Workstream lifecycle and project boundaries
The remote `/api/command` endpoint is conversation-local. It refuses
`/new`, `/workstreams`, `/resume`, and `/delete` because those local-CLI
helpers enumerate or mutate storage outside the HTTP resource gates. Remote
clients use the dedicated create, open, close, and delete endpoints instead;
`/rewind` and `/retry` have their own path-keyed, `conversation.modify`-gated
endpoints.
Passing `resume_ws` to create is an atomic **fork**, not an in-place resume.
It requires the ordinary create capability and source visibility. A private
project source is visible only to its workstream creator, project owner/member,
or authorized service-to-service cluster plumbing; denials use a not-found
response so guessed IDs do not become an existence oracle. The caller must also
be allowed to attach a new workstream to the source's current project. The
destination always inherits that effective project — a caller-supplied
`project_id` cannot re-file or declassify the conversation.
The canonical preflight atomically captures (and, for a legacy row, installs) a
private source-incarnation fence. The storage transaction compares that source
fence, rejects provisional sources, repeats the ACL/project check, and verifies
the persona/project construction snapshot, destination ownership and
incarnation, emptiness, and every referenced attachment before committing. A
source replacement, membership, project, persona, or destination-incarnation
race aborts the whole fork. Concurrent source-history writes serialize wholly
before or after the snapshot; no mixed or partially authorized history or
attachment reference becomes visible.
---
## Login Flows
@@ -441,12 +471,15 @@ Each proxied request gets a fresh JWT (5-minute expiry). This ensures:
- **Permission forwarding** — granular RBAC permissions from the
console JWT are carried through to the server.
The JWT `src` claim is set to `"console-proxy"`, allowing servers to
distinguish proxied requests from direct logins in audit logs.
For ordinary users the JWT `src` claim is set to `"console-proxy"`, allowing
servers to distinguish proxied requests from direct logins in audit logs.
Coordinator tokens retain `src="coordinator"` and their signed `coord_ws_id`;
the console service identity retains `src="console"` only when its validated
token also carries the unassignable `service` scope.
When no user context is available (auth disabled, or internal requests),
the proxy falls back to a `ServiceTokenManager` with service identity
`console-proxy` and full scopes.
the proxy falls back to a `ServiceTokenManager` with identity `console-proxy`,
`src="console"`, and `{read, write, approve, service}` scopes.
### Service-to-service authentication
@@ -455,8 +488,8 @@ JWTs when communicating with server nodes:
| Service | Identity | Scope | Audience | Purpose |
|---------|----------|-------|----------|---------|
| Console collector | `console-collector` | `read` | `turnstone-server` | Node health polling |
| Console proxy (fallback) | `console-proxy` | `approve` | `turnstone-server` | Proxied API calls when no user context |
| Console collector | `console-collector` | `read`, `service` | `turnstone-server` | Node health polling and global event collection |
| Console proxy (fallback) | `console-proxy` | `read`, `write`, `approve`, `service` | `turnstone-server` | Proxied API calls when no user context |
| Channel notify | `system` | `write` | `turnstone-channel` | Notification delivery to channel gateway |
Service tokens use 1-hour expiry with automatic refresh via
@@ -468,8 +501,8 @@ When the console creates a workstream (the normal path), the
authenticated user's `user_id` is forwarded in the HTTP payload when
calling the server's `POST /v1/api/workstreams/new`. The server
accepts a `user_id` from the request body **only when the caller is a
trusted service** — identified by `token_source` matching
`console-proxy` or `console`. Regular API callers cannot
trusted service** — identified by `token_source="console"` together with the
unassignable `service` scope. `console-proxy`, coordinator, and regular API callers cannot
override `user_id`; the server always uses their JWT identity.
Note that the channel gateway uses a distinct JWT audience
+79 -10
View File
@@ -54,6 +54,38 @@ When a per-model override is `NULL` (empty in the UI), the global default is
used. Switching models via `/model <alias>` re-resolves sampling parameters
from the new model's overrides or global defaults.
### Per-model concurrency
Each model definition may set `max_concurrency` to limit simultaneous model
generations for that alias in one Turnstone process. `0` or an omitted value
means unlimited. The gate is shared by every role using the alias—interactive
turns, coordinators, task agents, judges, output guards, perception, compaction,
and title generation—and a streaming generation holds its slot until the
stream is fully drained or closed.
Admission is strictly per alias. Two aliases remain independent even when they
point to the same URL; Turnstone does not infer shared capacity from endpoint
text. Queue time is excluded from judge/output-guard deadline accounting, and
each retry releases its slot before backoff and reacquires for the next wire
attempt. The cap is local to each process, not cluster-wide; account for the
number of nodes targeting the same inference server. Direct STT/TTS protocol
calls and Cohere/Jina reranking do not currently consume this generation cap.
### Judge batch parallelism
`judge.parallel_evaluations` controls how many independent tool calls from one
approval batch the intent judge evaluates concurrently. It is an integer from
1 through 16 and defaults to 1, preserving serial evaluation until an operator
opts into wider fan-out. Changes are hot-read at the next batch; work already
in flight keeps its captured worker count.
This is a per-batch fan-out setting, not another backend capacity limit. The
judge model alias's `max_concurrency` gate still caps total generations across
all judge batches and every other role using that alias. Actual overlap is
therefore bounded by the batch size, `judge.parallel_evaluations`, and available
alias admission slots. A smaller positive alias cap also narrows the batch's
worker pool so excess judge threads do not queue ahead of later alias traffic.
### Model backend authentication
Model definitions support four backend credential modes:
@@ -134,6 +166,15 @@ delegated-mode rows and memo entries. `entra_app` rows belong to the shared
revocation, an already-minted app bearer remains usable until its recorded
expiry.
Each model call resolves its dynamic credential against the immutable model
definition snapshot that supplied that call's provider, client, endpoint, and
model ID. An admin edit can therefore never pair an old `base_url` with a new
audience, grant mode, or static-key fallback input. The principal and token
remain per-call/live; the connection and model-owned auth configuration move
together as one binding on the next operation. The deployment-wide
`model.auth_fail_closed` switch is intentionally read live on every mint, so an
operator can tighten fallback policy immediately without rebuilding sessions.
`obo_audience` and `obo_scopes` are literal and capped at 2048 characters
each. Environment-variable expansion is deliberately not applied, so the
allow-list decision cannot vary by node or expand beyond the persisted
@@ -217,7 +258,7 @@ initialization:
| `mcp` | config_path, registry_url |
| `ratelimit` | enabled, requests_per_second, burst, trusted_proxies |
| `health` | backend_probe_interval, backend_probe_timeout, circuit_breaker_threshold, circuit_breaker_cooldown |
| `judge` | enabled, model, provider, base_url, api_key, confidence_threshold, max_context_ratio, timeout, read_only_tools, output_guard, redact_secrets, cancel_on_approval |
| `judge` | enabled, model, smart_approvals, confidence_threshold, max_context_ratio, timeout, parallel_evaluations, read_only_tools, output_guard, output_guard_budget_seconds, output_guard_llm, output_guard_model, output_guard_llm_timeout, redact_secrets, cancel_on_approval |
| `interface` | close_tab_action, theme |
| `skills` | discovery_url |
| `memory` | relevance_k, fetch_limit, max_content, nudge_cooldown, nudges |
@@ -394,13 +435,11 @@ Reset a setting to its registry default by removing it from storage.
## Secret Settings
Settings with `is_secret=True` (currently only `judge.api_key`) are blocked
from the write API with a `403` response. This prevents accidental exposure
through the admin UI or audit logs. Secret settings must be configured via
`config.toml` or environment variables.
The list endpoint masks secret values: stored secrets appear as `"***"`
rather than their actual value.
The registry currently defines no production secret system setting. The generic
machinery nevertheless treats any future `is_secret=True` entry as write-only:
list and write responses return `"***"`, and submitting that sentinel preserves
the stored value. Model API keys are fields on model definitions—not
`judge.*` system settings—and use the Models tab's separate write-only flow.
---
@@ -421,10 +460,40 @@ reload.
**Behavior after reload:**
- New workstreams pick up updated values immediately (via `session_factory`)
- Existing sessions keep their frozen configuration (settings are captured at
workstream creation time, not read on every turn)
- Most workstream/session settings remain the snapshot captured at creation or
resume. Component docs call out deliberate live-read exceptions; for
example, Smart Approval settings are snapshotted coherently at the start of
each approval batch.
- Settings marked `restart_required=True` need a server restart to take effect
### Model-definition reloads
The Models tab has a separate live-reload contract from ordinary ConfigStore
settings. Existing sessions remember the concrete registry generation that
supplied their active alias and re-resolve that alias at the start of the next
send. Endpoint, provider, backend model ID, capabilities, extra parameters, and
backend-auth configuration are replaced as one immutable binding. In-flight
turns, judges, and task agents finish or cancel against the binding they
started with; an admin edit never tears one request across two definitions.
The alias's admission gate is retained and resized in place, so a concurrency
edit preserves in-flight accounting and does not reset cached judges or the
output-guard rate limiter.
Sampling and other saved workstream configuration remain workstream state. A
model-definition edit does not silently rewrite a live workstream's chosen
temperature, reasoning effort, max tokens, skill, or persona. Use
`/model <alias>` (or create/fork a workstream) when an explicit session-level
model switch is intended.
If a live workstream's alias is deleted, its next send first attempts the
configured fallback chain. Without a usable fallback, the operator-facing
error names the removed alias and points interactive users to `/model`; adding
the alias back causes the next send to rebind without a process restart. If a
replacement client cannot be constructed, Turnstone logs one
`session.model_refresh_client_construction_failed` warning per registry
generation and retries only after another model reload, avoiding a rebuild
storm on every send.
---
## Migration from config.toml
+106 -24
View File
@@ -1,7 +1,7 @@
---
name: import-conversation-history
description: Use this skill when the user wants to import or migrate conversation history from another LLM chat or coding tool (e.g. ChatGPT, Claude.ai, Cursor, Copilot Chat, Aider, Gemini, a custom JSON export) into Turnstone. The skill teaches Turnstone's destination contracts — workstream identity, the OpenAI-shaped message rows, tool-call/result pairing, provider-fidelity blobs, attachments, and archive-vs-resumable choice — so the agent can map any source format onto them. Trigger phrases: "import my chats", "migrate this transcript into Turnstone", "bring my Claude.ai history over", "load this export as a workstream".
version: 1.0.0
version: 1.1.0
---
# Importing Conversation History into Turnstone
@@ -12,7 +12,7 @@ Source formats vary; the destination does not. Your job is to translate whatever
Two questions to settle with the user before writing anything:
1. **Archive or resumable?** An archive ("saved" workstream — `state="closed"`) is read-only history. A resumable workstream (`state="idle"`) lets the user continue the conversation; this only works cleanly when the source LLM matches a Turnstone-supported provider/model and tool definitions still resolve.
1. **Archive or resumable?** An archive is left closed and is read-only history. A resumable import is also kept closed and unloaded while rows are written, then explicitly opened after validation; this only works cleanly when the source LLM matches a Turnstone-supported provider/model and tool definitions still resolve.
2. **One workstream per source thread, or merge?** Default to one-to-one unless the user explicitly asks to merge.
Default to **archive** when in doubt — resuming a foreign transcript with mismatched tool schemas or stale provider signatures will fail at the next turn.
@@ -25,13 +25,13 @@ Two tables carry the conversation:
| Column | Required | Notes |
|---|---|---|
| `ws_id` | yes | 32-char lowercase hex. Auto-generate with `secrets.token_hex(16)` if you don't already have one. **First 4 hex chars are the routing bucket** — see "Identity & Routing" below. |
| `ws_id` | yes | 32-char lowercase hex. Auto-generate with `secrets.token_hex(16)` if you don't already have one. The router hashes the **full ID** — see "Identity & Routing" below. |
| `name` | yes | Short title. Pull from source thread title; fall back to first ~60 chars of first user message. |
| `state` | yes | `"closed"` for archive, `"idle"` for resumable. Never set `"running"` on import. |
| `state` | yes | Register as `"closed"` while importing. Leave it closed for an archive; explicitly open it after commit for a resumable import. Never set `"running"` or `"creating"` directly. |
| `kind` | yes | `"interactive"` for normal threads. Do NOT use `"coordinator"` for imports — that's reserved for cluster-spawned coordinator workstreams. |
| `parent_ws_id` | no | Leave NULL. Only set if you're importing a coordinator-spawned subtree and re-parenting it; rare. |
| `user_id` | yes | Owner. Must exist in `users`; importer must know which Turnstone user owns the imported history. |
| `node_id` | yes (multi-node) | Denormalized cache of the node that owns this `ws_id`'s bucket. Single-node deployments can leave it NULL or set it to the only node. |
| `node_id` | no | Nullable creation-time service/liveness hint. It is not the routing key or durable owner and may become stale after membership changes. Let a routed create stamp it; a direct shared-storage import may leave it NULL. |
| `alias` | no | Human-typeable short name. Optional; must be unique cluster-wide if set. |
| `title` | no | Auto-titled later by the LLM; safe to leave NULL on import. |
| `skill_id`, `skill_version` | yes | Default `""` and `0` unless the source thread was scoped to a Turnstone skill. |
@@ -55,25 +55,65 @@ The internal format is **OpenAI-shaped**, even when the source was Anthropic or
## Identity & Routing (`ws_id`)
- `ws_id` is **32-char lowercase hex** (i.e. `secrets.token_hex(16)`).
- The **routing bucket** is `int(ws_id[:4], 16)` — the first 4 hex chars place this workstream on a specific node via the consistent hash ring.
- For multi-node imports: either insert through the console's routing proxy (which forwards to the owning node), or generate `ws_id`s and write directly to each node's database in batches grouped by bucket.
- For single-node imports: bucket math is irrelevant; any `ws_id` works.
- Ordinary placement is rendezvous (Highest Random Weight, HRW) selection over
the **full `ws_id`** and the current live server set. For each node, Turnstone
computes 32-bit FNV-1a over the node ID, a NUL separator, and the full
workstream ID; it then applies the node weight and selects the highest score.
A live per-workstream override takes precedence.
- The live set comes from recent `services` heartbeats. Placement can therefore
change when nodes join, leave, change weight, or an override changes. There
is no stable prefix-derived placement to pre-compute or persist.
- `workstreams.node_id` is stamped at creation and is not updated as HRW
placement changes. It supports display and liveness-safe cleanup; the console
router does not use it as the ordinary ownership decision.
- For multi-node imports, create through the console routing proxy when the
lifecycle must be published, or write the history once through the cluster's
configured **shared storage backend**. Never partition rows across node-local
databases by ID prefix or by a one-time HRW result: a later membership change
can route the same full ID to another node.
- For single-node imports, HRW placement is degenerate; any valid `ws_id` works.
- **Do not reuse the source platform's IDs as `ws_id`** unless they happen to be 32-char hex. Generate fresh; if you need the old ID for traceability, store it in `workstream_config` under a key like `import.source_id`.
## Recommended Import Path
Three options, in order of preference:
### 1. Storage protocol (recommended for full history)
### 1. Quiesced storage import (recommended for full history)
Use `turnstone.core.storage.Storage.save_messages_bulk(rows)`. This is the canonical bulk-insert primitive and bypasses the LLM round-trip entirely.
Use the current `turnstone.core.storage.StorageBackend` protocol against the
same shared backend as the cluster. The destination must remain absent from all
in-memory session managers while rows are changing: a loaded `ChatSession`
holds its own trajectory and will not observe conversation rows inserted behind
it.
The safe sequence is:
1. Normalize and validate the complete source transcript before writing.
2. Call `register_workstream(..., state="closed")` and require a `True` return;
`False` means the caller-selected ID already exists, so abort rather than
appending to an unrelated workstream.
3. Insert the ordered conversation rows and attachment references.
4. Load the saved rows back and run the validation checklist below.
5. Leave an archive closed. For a resumable import, only now invoke the normal
`POST /v1/api/workstreams/{ws_id}/open` endpoint on the currently routed
node so the session hydrates from the complete transcript.
Do **not** create the destination through the web/SDK create endpoint before a
direct bulk import. Create publishes an empty live session. If that already
happened, close the workstream and confirm the manager-authoritative live probe
returns false before writing, then explicitly open it again after validation.
For attachment-free history, `save_messages_bulk(rows)` is the canonical
single-transaction insert primitive and bypasses the LLM round-trip entirely.
New attachment bytes require the per-row path described under
[Attachments](#attachments).
```python
from turnstone.core.storage import get_storage # construct via the same path the server uses
from turnstone.core.storage import get_storage # initialized by the host/import entry point
storage = get_storage(...) # see turnstone.core.storage.__init__ for the project's wiring
storage = get_storage()
storage.create_workstream( # or whatever the project's exposed creator is — check turnstone/core/storage/_protocol.py
inserted = storage.register_workstream(
ws_id=ws_id,
user_id=user_id,
name=name,
@@ -81,6 +121,8 @@ storage.create_workstream( # or whatever the project's exposed creator is — c
kind="interactive",
...
)
if not inserted:
raise RuntimeError(f"destination already exists: {ws_id}")
storage.save_messages_bulk([
{"ws_id": ws_id, "role": "user", "content": "Hello"},
@@ -94,7 +136,19 @@ storage.save_messages_bulk([
])
```
`save_messages_bulk` handles `timestamp` and the workstream's `updated` column internally, so you don't need to compute them per row. **Verify the exact creator signature** by reading `turnstone/core/storage/_protocol.py` — table layout has shifted across migrations and the Storage protocol is the source of truth.
`save_messages_bulk` handles `timestamp` and the workstream's `updated` column
internally, so you don't need to compute them per row. Verify the exact
`register_workstream` and message signatures in
`turnstone/core/storage/_protocol.py`; the Storage protocol, not the physical
table layout, is the source of truth.
**Multi-node note:** this path assumes `get_storage()` is connected to the
cluster's shared backend. Do not open a node-local database selected from the
current HRW result, and do not pre-create a live session through the console
routing proxy. After the shared-storage import commits, resolve the current
route and open the closed workstream on that node. Any stored `node_id`
describes creation-time placement, not a permanent shard that should receive a
separate copy.
### 2. SDK `create_workstream(resume_ws=...)` (when the source is already a Turnstone workstream)
@@ -181,27 +235,48 @@ If the source thread had image or file attachments:
- **Size limits**: images ≤ 4 MiB, text documents ≤ 512 KiB. Reject or downsample anything bigger.
- **Allowed types**: server validates magic bytes for images and UTF-8-decodes for text. Binary blobs that aren't images won't pass.
- **Lifecycle**: pending → reserved → consumed. For imports, the cleanest path is to upload as pending and immediately consume by attaching to the relevant `conversations.id`.
- **Blob identity**: `attachment_id` is the lowercase SHA-256 hex digest of the
bytes. `workstream_attachments` stores that content-addressed blob and its
refcount; it has no workstream or message foreign key.
- **Message link**: the sole message-to-blob link is the ordered JSON ID list in
`conversations.attachments`.
- **No persisted staging lifecycle**: pending upload bytes live only in a
node's in-memory attachment buffer. The old persisted
`pending → reserved → consumed` lifecycle does not apply to storage imports.
Two import paths:
For new attachment bytes, preserve row order by calling `save_message()` for
each turn. It returns the `conversations.id`; for every attachment referenced by
that turn, call `save_attachment()` with its content hash and bytes, then call
`set_message_attachments(ws_id, message_id, ordered_ids)`. Each
`save_attachment()` call accounts for one reference, while
`set_message_attachments()` records the ordered link.
1. **Bulk-insert + post-attach**: insert messages first, get back the assistant/user `conversations.id`, then write `workstream_attachments` rows linking the file to `message_id`.
2. **SDK multipart create**: `create_workstream(attachments=[...], initial_message=...)` for the *first* turn only — the server reserves and consumes them onto that turn. Doesn't help for mid-thread attachments.
`save_messages_bulk(..., attachment_ids=[...])` is appropriate only when those
content-addressed blobs already exist: the bulk transaction retains their
references and writes the ordered lists. Do not first call `save_attachment()`
for a new reference and then pass the same reference to `save_messages_bulk()`;
both paths retain it and would double-count the refcount.
For full-history imports with multiple attachments at different turns, path (1) is the only option.
SDK multipart create remains useful only for attachments on a new first turn;
it publishes a live session and is not the full-history import path.
## Validation Checklist
Before declaring success, verify:
- [ ] `ws_id` is 32-char lowercase hex.
- [ ] `workstreams` row exists with the right `user_id`, `state`, `kind`.
- [ ] The workstream remained closed and absent from every live manager while rows were written; archives stay closed and resumable imports are opened only after validation.
- [ ] `workstreams` row exists with the right `user_id` and `kind`.
- [ ] Conversation rows are inserted **in order** (autoincrement `id` will reflect insert order).
- [ ] Every assistant `tool_calls[].id` has a matching `role="tool"` row with the same `tool_call_id`.
- [ ] `tool_calls[].function.arguments` is a JSON-encoded **string**, not a parsed object.
- [ ] First message is typically `role="user"` (not `system`) — Turnstone composes its own system prompt at runtime.
- [ ] No empty assistant rows (`content=NULL` AND `tool_calls=NULL` is invalid).
- [ ] If multi-node: the `ws_id`'s bucket maps to a node that exists; `workstreams.node_id` matches.
- [ ] Every attachment ID is the SHA-256 of its stored bytes; each turn's ordered IDs are in `conversations.attachments`, and blob refcounts match message references.
- [ ] If multi-node: the row is in shared storage and the node selected by
`ConsoleRouter.route(ws_id)` from the current live set can load it.
`workstreams.node_id`, when present, is treated as a creation-time hint rather
than asserted equal to the current HRW result.
- [ ] Round-trip test: run `Storage.load_messages(ws_id)` and confirm the reconstructed list matches what you inserted (modulo timestamps).
## Anti-patterns
@@ -211,15 +286,20 @@ Before declaring success, verify:
- **Don't fabricate `tool_call_id`s without re-pairing.** Mismatched ids silently break the replay chain on the next turn.
- **Don't skip the `tool_name` field on `role="tool"` rows.** Some load paths use it for display and audit; NULL there will render as "unknown tool".
- **Don't write through the LLM (`send()` per turn) for full history.** It's expensive, rewrites assistant turns, and rate-limits will bite long imports.
- **Don't shard imported rows by an ID prefix or a one-time HRW result.** HRW
uses the full ID and live membership; placement may move. In a cluster, write
one copy to shared storage and let request routing select the live node.
## Quick Reference
| Task | Path |
|---|---|
| Generate ws_id | `secrets.token_hex(16)` |
| Bulk insert messages | `Storage.save_messages_bulk(rows)` |
| Multi-node placement | Full-ID 32-bit FNV-1a HRW over live servers; store rows once in shared storage |
| Bulk insert attachment-free messages | `Storage.save_messages_bulk(rows)` |
| Attach new bytes | `save_message()``save_attachment()` per reference → `set_message_attachments()` |
| Archive (read-only) | `state="closed"`, skip `provider_data` |
| Resumable | `state="idle"`, populate `provider_data` if same provider |
| Resumable | Register closed, import and validate while unloaded, then explicitly open; populate `provider_data` if same provider |
| Tool call id | OpenAI shape: `{"id": ..., "type": "function", "function": {"name": ..., "arguments": "<json string>"}}` |
| Tool result row | `role="tool"`, `tool_name`, `tool_call_id`, `content` |
| Source role → Turnstone role | See "Role Mapping" table |
@@ -228,6 +308,8 @@ Before declaring success, verify:
## Files to read before writing the importer
- `turnstone/core/storage/_schema.py` — authoritative table definitions.
- `turnstone/core/storage/_protocol.py``save_message`, `save_messages_bulk`, `load_messages` signatures.
- `turnstone/core/storage/_protocol.py``register_workstream`, message, attachment, and load signatures.
- `turnstone/core/rendezvous.py` — authoritative full-ID FNV-1a HRW scoring.
- `turnstone/console/router.py` — live-node discovery, override precedence, and routing behavior.
- `turnstone/core/session.py` (around the message-save section) — how the runtime constructs in-memory message dicts; mirror this shape on import to round-trip cleanly.
- `turnstone/api/server_schemas.py` — Pydantic shapes for the SDK paths if you go through HTTP.
+65 -17
View File
@@ -54,15 +54,11 @@ docker compose exec caddy \
cat /data/caddy/pki/authorities/local/root.crt # import into your OS/browser
```
**Can Caddy get its cert from the console's internal CA instead?** Technically
yes — the console exposes a real ACME directory (`/acme/directory`) with
auto-approval, so Caddy's `tls { ca http://console:8090/acme/directory }` would
mint a cert for any name. It's not recommended as the default: lacme's ACME
responder is built for turnstone's own client (interop with Caddy's client is
unverified), it couples Caddy startup to the console, and the browser must trust
a private CA either way — so it buys nothing over `tls internal`. For a publicly
trusted cert (no warning), point Caddy at Let's Encrypt with a real domain
instead.
**Can Caddy get its cert from the console's internal CA instead?** Not directly.
The console's ACME signing routes require Turnstone's rotating enrollment JWT,
which a standard Caddy ACME issuer does not attach. Keep `tls internal`, or use a
public ACME CA for a publicly trusted certificate. An authenticated gateway or
Caddy plugin would be required to use Turnstone's responder.
---
@@ -106,13 +102,15 @@ An mTLS listener rejects plain-HTTP probes at the socket, so
it presents the node's own cert as the client cert and pins the cluster
CA, using the PEM files the server writes at boot under
`$TURNSTONE_TLS_PEM_DIR` (default `<tmpdir>/turnstone-tls`). The probe
dials `localhost` for the TLS attempt — the internal CA issues DNS SANs
only, so a literal-IP URL would fail verification. Cert renewal rewrites
dials `localhost` for the TLS attempt, which every service certificate carries
as a DNS SAN. Cert renewal rewrites
the PEM dir alongside the live listener swap, so the probe's client cert
never outlives the served cert. With TLS disabled the plain probe succeeds
and the PEM directory is never consulted. On bare metal with multiple
nodes per host, set `TURNSTONE_TLS_PEM_DIR` per node (each boot clears
stale `lacme-pem-*` dirs under its root).
The production TLS Compose overlay inherits this healthcheck from the base
service; it remains enabled under mTLS.
---
@@ -125,6 +123,13 @@ stale `lacme-pem-*` dirs under its root).
| `tls.enabled` | `false` | Master switch for internal mTLS |
| `tls.acme_directory` | `""` | External ACME CA URL for console frontend cert |
### ACME topology environment
| Variable | Default | Description |
|----------|---------|-------------|
| `TURNSTONE_ACME_EXTERNAL_URL` | request-derived | Canonical externally reachable responder base, including `/acme` (for example `http://192.0.2.1:8090/acme`). Set it on the console so advertised URLs are routable and on in-cluster clients so their enrollment JWT is allowed only at that configured destination. A public path prefix is valid only when a reverse proxy maps it to Turnstone's internal `/acme` mount. |
| `TURNSTONE_CONSOLE_HTTP_BIND` | `127.0.0.1` | Production TLS-overlay bind for the console's plain-HTTP bootstrap/API port. For cross-host enrollment, use a trusted LAN/VPN interface and firewall it to enrolling nodes. |
### Bootstrap Config (config.toml)
These are needed before storage is available:
@@ -144,7 +149,7 @@ sslkey = "" # path to client key
| CA common name | "Turnstone CA" | |
| CA validity | 10 years | |
| Cert validity | 48 hours | Short-lived, auto-renewed |
| Renewal interval | 24 hours | Half of validity |
| Renewal interval | 12 hours | Leaves retry headroom before expiry |
| ACME auto-approve | true | Internal network, no challenge validation |
---
@@ -177,10 +182,14 @@ turnstone-admin tls-ca-cert --out ca.pem --console-url http://console:8080
# Request a cert for a domain
turnstone-admin tls-issue worker-1.internal --out /certs --console-url http://console:8080
# List issued certs
# List managed cluster certs
turnstone-admin tls-list --console-url http://console:8080
```
`tls-ca-cert` preserves the supplied scheme. An `https://` console URL is
verified with the system trust store; an explicitly supplied `http://` URL is
TOFU and prints a fingerprint that must be checked out of band.
### Console URL Discovery
If `--console-url` is not provided, the CLI discovers it from the `services`
@@ -193,7 +202,9 @@ table in the shared database. The console registers itself on startup.
The **TLS** tab in the console admin panel (System group) shows:
- CA status (common name, certificate count)
- Certificate table (domain, SANs, issued, expires)
- Force-renew and delete actions per certificate
- Force-renew for the console-owned internal identity; remote nodes renew and
hot-reload their own keys
- Delete for expired, remotely managed certificate rows
---
@@ -248,8 +259,15 @@ const client = new TurnstoneServer({
`TURNSTONE_CONSOLE_URL` (a bare-metal node outside the compose network can't
resolve the in-cluster `console` name, so it points this at the console's
published ACME endpoint)
3. Fetches CA root cert from `http://console/acme/ca.pem` (plain HTTP, TOFU)
4. Requests a service cert via ACME (plain HTTP, JWS-signed). The cert's
3. Fetches the CA root from the configured console scheme. Direct deployments
use `http://console/acme/ca.pem` (plain HTTP, TOFU); an explicitly configured
HTTPS proxy is preserved and verified with the system trust store.
4. Requests a service cert via ACME with a dedicated, short-lived Turnstone
service JWT pinned to configured responder origins. lacme emits ACME JWS
messages, but its lightweight responder deliberately does not validate their
signatures or nonces; the service JWT is the enrollment authorization gate.
Direct HTTP bootstrap therefore still requires a trusted LAN/VPN (or an
independently trusted HTTPS proxy). The cert's
primary domain / SAN is the node's **advertised host** (the host of
`TURNSTONE_ADVERTISE_URL`, e.g. `node-1`) — the name peers actually dial,
not the container hostname. This makes mTLS hostname verification succeed
@@ -264,7 +282,12 @@ const client = new TurnstoneServer({
1. Read `tls.enabled` from ConfigStore
2. Initialize CA (load from DB or generate new root key)
3. Mount ACME responder at `/acme` (serves `/ca.pem` natively)
3. Mount ACME responder at `/acme` (serves `/ca.pem` natively). When
`TURNSTONE_ACME_EXTERNAL_URL` is set, use it for every advertised directory,
order, authorization, and certificate URL; otherwise derive URLs from each
request as before. Directory, nonce, and CA bootstrap resources stay public;
account/order/challenge/finalization/certificate routes require the dedicated
enrollment service JWT.
4. Issue console certs (internal + optional frontend)
5. Start CA-direct auto-renewal (no network, signs directly), scoped to the
console's own cert, plus a periodic GC that reclaims cert rows for
@@ -290,6 +313,15 @@ fronted under a second hostname). Symptom if this is wrong: the console
dashboard shows nodes as unreachable and `openssl s_client` reports the served
cert's SANs don't include the dialed name.
The advertised host and extra SANs may be DNS names or literal IPv4/IPv6
addresses. Turnstone converts IP literals to typed ACME identifiers so the
certificate contains `IPAddress` SANs that normal IP hostname verification can
use; DNS spelling is preserved. Bracket an IPv6 address when it appears in a URL
(for example `TURNSTONE_ADVERTISE_URL=http://[2001:db8::10]:8080`), but use the
bare address in `TURNSTONE_TLS_SANS`. Unspecified bind addresses (`0.0.0.0` and
`::`) and scoped IPv6 addresses such as `fe80::1%eth0` are not certificate
identities. Restart after changing the advertised identity or extra SANs.
### "No console service found"
The console registers itself in the `services` table on startup. If the console
@@ -297,6 +329,22 @@ hasn't started or the registration expired (1 hour TTL), nodes can't discover
it. Set `TURNSTONE_CONSOLE_URL` to a reachable console address (this is also how
a bare-metal node that can't resolve the in-cluster `console` name enrolls).
### Cross-host ACME links point at the container
For a node on another host, publish port 8090 on a reachable interface and set
`TURNSTONE_ACME_EXTERNAL_URL` on the console **and in-cluster nodes** to that full
responder base, including `/acme` (for example
`http://192.0.2.1:8090/acme`). The console advertises it; clients use it as a
trusted enrollment-token destination. Keep
`TURNSTONE_CONSOLE_URL=http://console:8090` for in-cluster service discovery.
A remote node whose `TURNSTONE_CONSOLE_URL` already names the public origin can
derive the same `/acme` base, but setting both values explicitly avoids drift.
Bind only a trusted LAN/VPN interface and firewall it to enrolling nodes. The
JWT authenticates the client, but a direct plain-HTTP bootstrap remains TOFU and
does not resist an active on-path attacker. If the network is untrusted, expose
the responder through an independently trusted HTTPS proxy instead.
### Browser HTTPS to the console
The console serves plain HTTP (it's the ACME bootstrap endpoint — see
+75 -24
View File
@@ -1,9 +1,10 @@
# Tools Reference
turnstone exposes 17 built-in tools plus any number of external MCP tools to the
LLM via the OpenAI function-calling interface. Built-in tools are defined as JSON
files under `turnstone/tools/` and loaded at startup by `turnstone/core/tools.py`.
MCP tools are discovered from configured MCP servers at startup by
Turnstone exposes a role-specific built-in tool surface plus any configured MCP
tools through provider-native or OpenAI-compatible function calling. Built-in
schemas live under `turnstone/tools/` and are loaded by
`turnstone/core/tools.py`; metadata selects the interactive, coordinator, and
task-agent subsets. MCP tools are discovered from configured servers by
`turnstone/core/mcp_client.py`.
---
@@ -50,10 +51,10 @@ set lives in `_META_KEYS` in `turnstone/core/tools.py`):
| Name | Description |
|---------------------|-------------|
| `TOOLS` | All 29 loaded built-in tool definitions (interactive + coordinator union). Sessions send a kind-specific subset (`INTERACTIVE_TOOLS` or `COORDINATOR_TOOLS`). |
| `TOOLS` | The complete loaded built-in union. Sessions send a kind-specific subset (`INTERACTIVE_TOOLS` or `COORDINATOR_TOOLS`). |
| `TASK_AGENT_TOOLS` | Tools with `task_agent: true` -- available to task sub-agents. Includes write operations. |
| `TASK_AUTO_TOOLS` | Set of all tool names with `auto_approve: true` -- used by task-agent sub-sessions to skip confirmation for matching available tools. |
| `BUILTIN_TOOL_NAMES`| Frozenset of all 29 built-in tool names (interactive + coordinator union). Used by tool search to distinguish always-on tools from deferrable MCP tools. |
| `BUILTIN_TOOL_NAMES`| Frozenset of the built-in union. Used by tool search to distinguish built-ins from deferrable MCP tools. |
| `PRIMARY_KEY_MAP` | Dict mapping tool name to its `primary_key` parameter name. |
---
@@ -62,7 +63,10 @@ set lives in `_META_KEYS` in `turnstone/core/tools.py`):
> See also: [Tool Pipeline diagram](diagrams/png/05-tool-pipeline.png)
Tool execution follows a three-phase pipeline inside `ChatSession._execute_tools()`:
Tool handling spans a four-phase pipeline. `ChatSession._execute_tools()` owns
prepare, approval, and execution (phases 13); after it returns, the owning
conversation loop guards the observed results and folds them into the
trajectory (phase 4).
### Phase 1: Prepare
@@ -71,9 +75,8 @@ Tool execution follows a three-phase pipeline inside `ChatSession._execute_tools
- Parses the JSON arguments (with fallback for malformed JSON).
- If JSON parsing fails entirely, uses `PRIMARY_KEY_MAP` to map a bare string
to the correct parameter.
- Dispatches to the matching `_prepare_{func_name}()` handler. There are 17
built-in tools plus `tool_search` (synthetic, client-side BM25 fallback) and
the generic `_prepare_mcp_tool()` handler for MCP tools.
- Dispatches to the matching `_prepare_{func_name}()` handler, the synthetic
`tool_search` fallback, or the generic `_prepare_mcp_tool()` handler.
- Validates arguments and builds a preview dict containing:
- `call_id`, `func_name`, `header`, `preview` (for display)
- `needs_approval` (bool)
@@ -82,7 +85,10 @@ Tool execution follows a three-phase pipeline inside `ChatSession._execute_tools
### Phase 2: Approve
All prepared items are sent to the UI via `ui.approve_tools(items)`.
Prepared items are sent to the UI via `ui.approve_tools(items)`. Several
parallel task agents may leave independent `ApprovalCycle` objects pending on
one workstream; each round owns a `cycle_id`, event, result, and verdict set.
Remote clients resolve the exact round by `cycle_id` (or a member `call_id`).
- The UI displays each tool's header and preview to the user.
- Items where `needs_approval` is `False` (auto-approved tools) are shown
@@ -94,6 +100,10 @@ All prepared items are sent to the UI via `ui.approve_tools(items)`.
prompt). This is per-tool, not blanket.
- If `auto_approve` is `True` on the session (via `--skip-permissions` or workstream
template), all tools are approved automatically.
- When Smart Approvals are enabled, one immutable judge/settings snapshot is
stamped onto the whole batch. The batch auto-approves only when every gated
item has a qualifying verdict; partial or mixed qualification fails closed to
the human prompt. Stop is linearized against that terminal decision.
### Phase 3: Execute
@@ -113,6 +123,28 @@ Each item's `execute` callable is invoked:
denials are tracked separately. This removes the need for text-prefix heuristics.
Other tools deliver results atomically via
`ui.on_tool_result(call_id, name, output, is_error=...)` only.
Stop propagates to child model scopes, judges, tracked subprocess groups, and
the approval cycles owned by the cancelled operation. Calls that definitely
never started receive `EffectStatus.none`; an interrupted call whose external
outcome was not observed receives `unknown`, `partial`, or `rolled_back` as
appropriate. These typed receipts preserve effect truth across storage/replay
without exposing unreviewed model output as a tool result.
### Phase 4: Guard and atomic fold
After `_execute_tools()` returns, the main `send()` loop compacts/truncates
completed results to the remaining shared budget and then runs the heuristic
and optional LLM output guard. The task-agent loop deliberately guards the
observed raw output before applying its size cap, so truncation cannot hide a
sensitive result from that check.
After guard work, the owning loop rechecks generation ownership. On the main
conversation path, one generation-fenced commit appends the complete
tool-result block, advisories, feedback, and queued user turns; its durable
records run in FIFO order outside the lifecycle lock. A force-cancelled
predecessor can therefore finish external cleanup, but cannot fold late results
into its successor's trajectory.
---
## Tool Approval Flow
@@ -120,7 +152,7 @@ Each item's `execute` callable is invoked:
**Auto-approved** (no user confirmation needed at runtime):
- `read_file` -- reads files, no side effects
- `search` -- grep-style search, no side effects
- `memory` -- structured persistent memory (save/search/delete/list)
- `memory` -- structured persistent memory (save/get/search/delete/list)
- `recall` -- searches conversation history
- `notify` -- sends notifications to linked channels (time-sensitive, auto-approved for urgency)
@@ -295,7 +327,7 @@ Fetch a URL and extract specific information from it.
| `url` | string | yes | The URL to fetch (must start with `http://` or `https://`). |
| `question` | string | yes | What to extract or answer from the page content. |
- **What it does**: Fetches the URL, strips HTML to plain text, and uses the LLM to extract the answer to the question from the page content. Every redirect hop is SSRF-screened before it is requested. Private/internal addresses are refused by default; enable `tools.allow_private_network` (console Settings → Tools) to make them approvable for self-hosted setups whose services live on the local network — the approval prompt marks such requests, and a public site redirecting into private space is refused regardless.
- **What it does**: Fetches the URL, strips HTML to plain text, and uses the LLM to extract the answer to the question from the page content. Every redirect hop is SSRF-screened before it is requested. Private/internal addresses are refused by default; enable `tools.allow_private_network` (console Settings → Tools) to make them approvable for self-hosted setups whose services live on the local network — the approval prompt marks such requests, and a public site redirecting into private space is refused regardless. Cloud metadata endpoints and link-local, multicast and reserved addresses are refused even with the opt-in enabled, including as a redirect target from a private address you approved. An address is judged by what it actually reaches, so an IPv6 transition address (NAT64, 6to4, Teredo) wrapping an internal IPv4 is treated exactly as that IPv4 would be.
- **Auto-approve**: No -- requires user confirmation (makes network requests).
- **Agent availability**: `task_agent`.
@@ -401,7 +433,7 @@ Delegate a general-purpose task to an autonomous sub-agent.
|-----------|--------|----------|-------------|
| `prompt` | string | yes | Complete task description for the sub-agent. |
- **What it does**: Spawns a sub-agent that inherits the `TASK_AGENT_TOOLS` set (read, write, edit, search, bash, web tools, memory tools). The sub-agent runs autonomously to completion. Use for work that requires file modifications or command execution.
- **What it does**: Spawns a sub-agent that inherits the `TASK_AGENT_TOOLS` set (read, write, edit, search, bash, and web tools). The sub-agent runs autonomously to completion. Use for work that requires file modifications or command execution.
- **Auto-approve**: No -- requires user confirmation.
- **Agent availability**: Top-level only.
@@ -415,18 +447,26 @@ Structured persistent memory across sessions with typed, scoped entries.
| Parameter | Type | Required | Description |
|---------------|---------|----------|-------------|
| `action` | string | yes | `save`, `search`, `delete`, or `list`. |
| `name` | string | save/delete | Short snake_case identifier for the memory. |
| `action` | string | yes | `save`, `get`, `search`, `delete`, or `list`. |
| `name` | string | save/get/delete | Short snake_case identifier for the memory. |
| `content` | string | save | Memory content to store. |
| `description` | string | no | Short description for relevance matching (recommended for `save`). |
| `type` | string | no | Memory type: `user`, `project`, `feedback`, or `reference`. Default: `project`. |
| `scope` | string | no | Memory scope: `global`, `workstream`, or `user`. Default: `global`. |
| `description` | string | save | Non-empty description for relevance matching; required on create and update. |
| `type` | string | no | Memory type: `user`, `general`, `feedback`, or `reference`. Default: `general`. |
| `scope` | string | no | Memory scope: `global`, `workstream`, `user`, `coordinator`, or `project`. See defaults below. |
| `query` | string | search | Search query for finding memories. |
| `limit` | integer | no | Max results for `search` or `list`. Default: 20. |
- **What it does**: Manages structured persistent memories in the database. Memories persist across sessions, have a type classification (user preferences, project knowledge, feedback, reference material) and a scope (global across all workstreams, private to a workstream, or following a user). Relevant memories are included in the system prompt on startup.
- **What it does**: Manages structured persistent memories in the database.
Memories persist across sessions, have a type classification, and live in a
role-specific visible scope. Unscoped `save`/`get`/`delete` resolve to one
target: the attached active project, otherwise `global` for an interactive
session or `coordinator` for a coordinator. Read-only project access permits
`get` but makes `save`/`delete` fail without falling back. A valid explicit
scope selects exactly that scope. Unscoped `search`/`list` cover all visible
scopes; use the displayed scope when following a result with `get` or
`delete`.
- **Auto-approve**: Yes.
- **Agent availability**: Not available to sub-agents (top-level only).
- **Agent availability**: Not available to task agents.
---
@@ -441,7 +481,7 @@ Search conversation history for past messages and tool results.
- **What it does**: Searches conversation history across sessions using FTS5 full-text search. Returns matching messages, tool calls, and tool results with timestamps and workstream context.
- **Auto-approve**: Yes.
- **Agent availability**: Not available to sub-agents (top-level only).
- **Agent availability**: Not available to task agents.
---
@@ -577,7 +617,11 @@ pre-configure skills at workstream creation.
---
## Summary Table
## Interactive Tool Summary
This table describes the ordinary interactive surface. Coordinator sessions
receive their delegation/lifecycle tools instead, and task agents receive the
metadata-selected `TASK_AGENT_TOOLS` subset.
| Tool | Category | Auto-approve | task_agent | primary_key |
|--------------|------------|--------------|------------|-------------|
@@ -698,7 +742,7 @@ MCP-compatible service.
3. **Schema conversion**: Each MCP tool's `inputSchema` is converted to OpenAI
function-calling format. The tool name is prefixed: `mcp__{server}__{tool}`.
4. **Merging**: MCP tools are appended after the 17 built-in tools via
4. **Merging**: MCP tools are appended after the role's built-in tools via
`merge_mcp_tools()`. Built-in tools appear first, giving them natural LLM priority.
When dynamic tool search is active, MCP tools are deferred rather than directly
visible -- the model discovers them via search as needed (see
@@ -835,6 +879,13 @@ capabilities for the `resources` capability. For servers that declare it:
2. `list_resource_templates` fetches URI templates (parameterized patterns like
`db://tables/{table}/rows/{id}`).
The protocol advertises both lists through one aggregate `resources`
capability, so a server may implement only one of them. If either request
returns the JSON-RPC `Method not found` code (`-32601`), turnstone treats that
half of the catalog as empty and keeps the other half; authentication,
validation, transport, and all other discovery errors still fail the
connection or refresh.
Both are stored as `{uri, name, description, mimeType, server}` dicts and
merged into a unified catalog.
+16 -8
View File
@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
[project]
name = "turnstone"
version = "1.8.0a6"
version = "1.8.0a7"
description = "Multi-node AI orchestration platform with tool use, agent routing, and cluster simulation."
readme = "README.md"
license = "Apache-2.0"
@@ -23,9 +23,14 @@ classifiers = [
"Topic :: Scientific/Engineering :: Artificial Intelligence",
]
dependencies = [
"openai>=2.45", # GPT-5.6: typed reasoning.mode, prompt_cache_options, and cache_write_tokens
# Version 3 moves the default transport to HTTPX2. Keep major upgrades
# deliberate because the stream retry boundary depends on that contract.
"openai>=3,<4",
"anthropic>=0.117", # tracks the release current at claude-opus-5 onboarding; hard runtime floor is still 0.105 (mid-conversation system blocks) — Opus 5 itself needs no new SDK surface (model ids are opaque strings; "refusal" has been in the StopReason literal since ~0.95). Raise this when adopting fast mode / server-side fallbacks / advisor / mid-conversation tool changes, which DO need newer typed params.
"httpx>=0.28",
# Direct because the provider boundary catches this exception family;
# OpenAI v3's transitive dependency alone is not an import contract.
"httpx2>=2.7,<3",
"mcp>=1.27,<2", # v2 is a breaking rewrite (2.0.0a1 live 2026-06-11; stable ~2026-07-27) — streamablehttp_client removed, 2-tuple transport, snake_case types; migrate deliberately
"starlette>=1.3.1", # CVE-2026-54282 (path->authority host spoof) + CVE-2026-54283 (url-encoded form DoS); supersedes the PYSEC-2026-161 host-header path-injection floor
"uvicorn>=0.34",
@@ -42,7 +47,9 @@ dependencies = [
"PyJWT>=2.8",
"bcrypt>=4.0",
"cryptography>=48.0.1", # GHSA-537c-gmf6-5ccf: PyPI wheels <48.0.1 bundle a vulnerable statically-linked OpenSSL (2026-06-09 secadv)
"lacme>=1.0.5",
# Core mTLS/ACME contract. 1.1 moves lacme onto HTTPX2; 1.2 adds typed IP
# identifiers and retains the external responder URL for cross-host enrollment.
"lacme>=1.2,<2",
"python-frontmatter>=1.0",
"pypdfium2>=4", # PDF text-extract + rasterize for models without native PDF input (core/pdf.py)
"pillow>=10", # PNG encoding for the PDF->images rasterize fallback (vision models, core/pdf.py)
@@ -88,7 +95,7 @@ include = [
"turnstone/console/static/coordinator/*.js",
"turnstone/shared_static/*.css",
"turnstone/shared_static/*.js",
"turnstone/shared_static/katex-0.18.1/**/*",
"turnstone/shared_static/katex-0.18.4/**/*",
"turnstone/shared_static/hljs-11.11.1/**/*",
"turnstone/shared_static/mermaid-11.16.1/**/*",
"turnstone/shared_static/hls-1.6.17/**/*",
@@ -106,6 +113,11 @@ markers = [
"e2e_recovery: opt-in end-to-end SSE recovery harness (real server + real SSE consumers, scripted provider — NOT live, no LLM backend needed); tens of seconds each. CI lanes run ``-m 'not live and not e2e_recovery'``; select with ``-m e2e_recovery``.",
]
filterwarnings = [
# The MCP v1 FastMCP integration fixture rebuilds its incomplete generic
# Settings model before construction. Keep the new pydantic-settings 2.15
# diagnostic fatal so removing that compatibility step cannot regress
# into a warning hidden in the full-suite summary.
"error:Field 'lifespan' has an incomplete definition",
# mcp v1 deprecates streamablehttp_client for an entry point whose call
# shape only settles in v2 — adoption rides the deliberate v2 migration
# (pin capped <2); silence exactly this message until then.
@@ -187,10 +199,6 @@ ignore_missing_imports = true
module = ["frontmatter", "frontmatter.*"]
ignore_missing_imports = true
[[tool.mypy.overrides]]
module = ["lacme", "lacme.*"]
ignore_missing_imports = true
[[tool.mypy.overrides]]
module = ["pypdfium2", "pypdfium2.*"]
ignore_missing_imports = true
+458
View File
@@ -0,0 +1,458 @@
#!/usr/bin/env python3
"""Browser regression for the two-layer frontend cache contract.
This harness serves a versioned entry module which imports the real,
unversioned ``shared/interactive.js`` module. It loads an old pane build in a
real Chrome profile, switches the server to a same-version replacement whose
pane has the same byte length and mtime, then performs a normal browser reload
without clearing or disabling the HTTP cache.
The old fixture advertises ``user_turn=0&tool_turn=0`` in its EventSource URL;
the current source advertises both capabilities as ``1``. A passing run
therefore proves both layers of the contract:
* the package-versioned entry URL revalidates and may return 304; and
* its unversioned transitive pane import revalidates by content and returns the
current bytes rather than surviving from the prior build.
The page also loads representative KaTeX, Highlight.js, and HLS.js assets.
Their installed version directories are discovered from ``shared_static`` at
runtime, so a normal vendor-version bump requires no harness edit; reload must
reuse them under the immutable policy.
Usage::
uv run python scripts/asset_cache_e2e.py
"""
from __future__ import annotations
import contextlib
import json
import os
import shutil
import socket
import sys
import tempfile
import threading
import time
from dataclasses import dataclass, field
from pathlib import Path
from typing import Any
ROOT = Path(__file__).resolve().parents[1]
if str(ROOT) not in sys.path:
sys.path.insert(0, str(ROOT))
from recovery_e2e import CDP, _find_chrome, _launch_chrome, _page_ws_url # noqa: E402
from turnstone import __version__ # noqa: E402
from turnstone.core.web_helpers import ( # noqa: E402
RevalidatingStaticFiles,
version_html,
)
_PAGE_HTML = """<!doctype html>
<html>
<head>
<meta charset="utf-8">
<title>ASSET-CACHE-PENDING</title>
<!-- VENDORED_ASSET_TAGS -->
<script>
window.__assetCacheUrls = [];
class RecordingEventSource {
static CONNECTING = 0;
static OPEN = 1;
static CLOSED = 2;
constructor(url) {
this.url = String(url);
this.readyState = RecordingEventSource.CONNECTING;
window.__assetCacheUrls.push(this.url);
}
close() {
this.readyState = RecordingEventSource.CLOSED;
}
}
window.EventSource = RecordingEventSource;
window.addEventListener("error", (event) => {
document.title = "ASSET-CACHE-FAILED-" + String(event.message || "script").slice(0, 80);
});
window.addEventListener("unhandledrejection", (event) => {
document.title = "ASSET-CACHE-FAILED-" + String(event.reason || "promise").slice(0, 80);
});
</script>
</head>
<body>
<main id="pane"></main>
<script type="module" src="/static/asset_cache_boot.js"
onerror="document.title='ASSET-CACHE-FAILED-module-load'"></script>
</body>
</html>
"""
_BOOT_JS = """import { InteractivePane } from "/shared/interactive.js";
const fixtureWorkstreamId = "00000000-0000-0000-0000-000000000001";
const pane = new InteractivePane(fixtureWorkstreamId, { base: "" });
document.getElementById("pane").appendChild(pane.el);
pane.connectSSE(fixtureWorkstreamId);
const url = window.__assetCacheUrls.at(-1) || "";
const generation = url.includes("user_turn=1") && url.includes("tool_turn=1")
? "CURRENT"
: url.includes("user_turn=0") && url.includes("tool_turn=0")
? "OLD"
: "FAILED-CAPABILITIES";
window.__assetCacheResult = { generation, url };
document.title = "ASSET-CACHE-" + generation;
"""
@dataclass
class CacheState:
phase: str = "old"
requests: list[dict[str, Any]] = field(default_factory=list)
lock: threading.Lock = field(default_factory=threading.Lock)
def record(self, item: dict[str, Any]) -> None:
with self.lock:
self.requests.append(item)
def matching(self, phase: str, path: str) -> list[dict[str, Any]]:
with self.lock:
return [
item for item in self.requests if item["phase"] == phase and item["path"] == path
]
class SwitchingStaticFiles:
"""Select one immutable build snapshot at request dispatch time."""
def __init__(self, state: CacheState, old_dir: Path, current_dir: Path) -> None:
self._state = state
self._apps = {
"old": RevalidatingStaticFiles(directory=str(old_dir)),
"current": RevalidatingStaticFiles(directory=str(current_dir)),
}
async def __call__(self, scope: Any, receive: Any, send: Any) -> None:
await self._apps[self._state.phase](scope, receive, send)
class RecordingApp:
"""Record static HTTP validators and status without perturbing streaming."""
def __init__(self, app: Any, state: CacheState) -> None:
self._app = app
self._state = state
async def __call__(self, scope: Any, receive: Any, send: Any) -> None:
path = str(scope.get("path", ""))
if scope.get("type") != "http" or not path.startswith(("/static/", "/shared/")):
await self._app(scope, receive, send)
return
phase = self._state.phase
request_headers = {
key.decode("latin-1").lower(): value.decode("latin-1")
for key, value in scope.get("headers", [])
}
async def record_send(message: dict[str, Any]) -> None:
if message["type"] == "http.response.start":
response_headers = {
key.decode("latin-1").lower(): value.decode("latin-1")
for key, value in message.get("headers", [])
}
self._state.record(
{
"phase": phase,
"path": path,
"query": scope.get("query_string", b"").decode("latin-1"),
"if_none_match": request_headers.get("if-none-match"),
"status": message["status"],
"etag": response_headers.get("etag"),
"cache_control": response_headers.get("cache-control"),
}
)
await send(message)
await self._app(scope, receive, record_send)
def _old_interactive(current: bytes) -> bytes:
old = current
for needle, replacement in (
(b'"user_turn=1"', b'"user_turn=0"'),
(b'"&tool_turn=1"', b'"&tool_turn=0"'),
):
if old.count(needle) != 1:
raise RuntimeError(f"expected exactly one {needle.decode()} capability literal")
old = old.replace(needle, replacement, 1)
if len(old) != len(current):
raise AssertionError("old and current pane fixtures must have equal byte length")
return old
def _discover_vendor_assets(source_shared: Path) -> tuple[str, ...]:
selected = (
("katex", "katex.min.css"),
("katex", "katex.min.js"),
("hljs", "highlight.min.js"),
("hls", "hls.min.js"),
)
paths = []
for library, filename in selected:
matches = sorted(source_shared.glob(f"{library}-*/{filename}"))
if not matches:
raise RuntimeError(f"no vendored {library} asset named {filename} was found")
paths.extend(f"/shared/{match.relative_to(source_shared).as_posix()}" for match in matches)
return tuple(paths)
def _vendor_tags(vendor_paths: tuple[str, ...]) -> str:
tags = []
for path in vendor_paths:
if path.endswith(".css"):
tags.append(f'<link rel="stylesheet" href="{path}">')
else:
tags.append(f'<script src="{path}"></script>')
return "\n ".join(tags)
def _prepare_builds(scratch: Path) -> tuple[Path, Path, Path, tuple[str, ...]]:
import turnstone
package_dir = Path(turnstone.__file__).resolve().parent
source_shared = package_dir / "shared_static"
vendor_paths = _discover_vendor_assets(source_shared)
old_shared = scratch / "old" / "shared"
current_shared = scratch / "current" / "shared"
static_dir = scratch / "static"
shutil.copytree(source_shared, old_shared)
shutil.copytree(source_shared, current_shared)
static_dir.mkdir()
(static_dir / "asset_cache_boot.js").write_text(_BOOT_JS, encoding="utf-8")
current_asset = current_shared / "interactive.js"
old_asset = old_shared / "interactive.js"
current = current_asset.read_bytes()
old_asset.write_bytes(_old_interactive(current))
# Reproduce the metadata collision which defeated Starlette's default ETag.
fixed_mtime_ns = 1_700_000_000_123_456_789
for asset in (old_asset, current_asset):
os.utime(asset, ns=(fixed_mtime_ns, fixed_mtime_ns))
old_stat = old_asset.stat()
current_stat = current_asset.stat()
if (old_stat.st_size, old_stat.st_mtime_ns) != (
current_stat.st_size,
current_stat.st_mtime_ns,
):
raise AssertionError("pane fixture size/mtime collision was not preserved")
return old_shared, current_shared, static_dir, vendor_paths
def _make_app(
state: CacheState,
old_dir: Path,
current_dir: Path,
static_dir: Path,
vendor_paths: tuple[str, ...],
) -> Any:
from starlette.applications import Starlette
from starlette.responses import HTMLResponse
from starlette.routing import Mount, Route
async def page(_request: Any) -> HTMLResponse:
page_html = _PAGE_HTML.replace("<!-- VENDORED_ASSET_TAGS -->", _vendor_tags(vendor_paths))
return HTMLResponse(
version_html(page_html),
headers={"Cache-Control": "no-store"},
)
app = Starlette(
routes=[
Route("/asset-cache-e2e", page),
Mount(
"/static",
app=RevalidatingStaticFiles(directory=str(static_dir)),
),
Mount(
"/shared",
app=SwitchingStaticFiles(state, old_dir, current_dir),
),
]
)
return RecordingApp(app, state)
def _start_server(app: Any) -> tuple[Any, threading.Thread, socket.socket, str]:
import uvicorn
sock = socket.socket()
sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
sock.bind(("127.0.0.1", 0))
sock.listen(128)
port = int(sock.getsockname()[1])
server = uvicorn.Server(
uvicorn.Config(app, host="127.0.0.1", port=port, log_level="warning", lifespan="off")
)
thread = threading.Thread(
target=server.run,
kwargs={"sockets": [sock]},
name="asset-cache-e2e-server",
daemon=True,
)
thread.start()
deadline = time.monotonic() + 10
while not server.started and thread.is_alive() and time.monotonic() < deadline:
time.sleep(0.05)
if not server.started:
server.should_exit = True
thread.join(timeout=2)
sock.close()
raise RuntimeError("asset cache test server did not start")
return server, thread, sock, f"http://127.0.0.1:{port}"
def _wait_for_generation(cdp: CDP, expected: str, timeout: float = 20) -> dict[str, str]:
deadline = time.monotonic() + timeout
last_title = ""
while time.monotonic() < deadline:
last_title = cdp.title()
result = cdp.evaluate("window.__assetCacheResult || null")
if isinstance(result, dict) and result.get("generation") == expected:
return {"generation": str(result["generation"]), "url": str(result["url"])}
if last_title.startswith("ASSET-CACHE-FAILED"):
raise RuntimeError(last_title)
time.sleep(0.1)
raise TimeoutError(f"expected {expected}, last title was {last_title!r}")
def _one(state: CacheState, phase: str, path: str) -> dict[str, Any]:
requests = state.matching(phase, path)
if len(requests) != 1:
raise AssertionError(f"expected one {phase} request for {path}, got {requests!r}")
return requests[0]
def _verify_trace(state: CacheState, vendor_paths: tuple[str, ...]) -> tuple[str, list[str]]:
entry_path = "/static/asset_cache_boot.js"
pane_path = "/shared/interactive.js"
old_entry = _one(state, "old", entry_path)
current_entry = _one(state, "current", entry_path)
old_pane = _one(state, "old", pane_path)
current_pane = _one(state, "current", pane_path)
expected_query = f"v={__version__}"
if old_entry["query"] != expected_query or current_entry["query"] != expected_query:
raise AssertionError("entry URL did not retain the same package version across builds")
if old_entry["status"] != 200 or old_pane["status"] != 200:
raise AssertionError("old build did not populate the browser cache")
if current_entry["status"] != 304 or not current_entry["if_none_match"]:
raise AssertionError(f"versioned entry did not revalidate to 304: {current_entry!r}")
if current_pane["status"] != 200 or not current_pane["if_none_match"]:
raise AssertionError(
f"transitive pane did not revalidate to current bytes: {current_pane!r}"
)
if old_pane["etag"] == current_pane["etag"]:
raise AssertionError("content-derived pane validators did not change")
if current_pane["cache_control"] != "no-cache":
raise AssertionError("transitive pane lost its revalidation policy")
vendor_trace = []
immutable = "public, max-age=31536000, immutable"
for path in vendor_paths:
old_vendor = _one(state, "old", path)
if old_vendor["query"] or old_vendor["status"] != 200:
raise AssertionError(f"versioned vendor URL was rewritten or failed: {old_vendor!r}")
if old_vendor["cache_control"] != immutable:
raise AssertionError(f"versioned vendor asset was not immutable: {old_vendor!r}")
revisits = state.matching("current", path)
if revisits:
if len(revisits) != 1 or revisits[0]["status"] not in (200, 304):
raise AssertionError(f"unexpected vendor reload trace: {revisits!r}")
if revisits[0]["cache_control"] != immutable:
raise AssertionError(f"vendor reload lost immutable policy: {revisits[0]!r}")
vendor_trace.append(f"{path}: revisited-{revisits[0]['status']}")
else:
vendor_trace.append(f"{path}: cache-hit")
verdict = f"ASSET-CACHE-READY-entry304-pane200-user1-tool1-vendor{len(vendor_paths)}"
return verdict, vendor_trace
def _stop_process(proc: Any) -> None:
if proc.poll() is not None:
return
proc.terminate()
with contextlib.suppress(Exception):
proc.wait(timeout=5)
if proc.poll() is None:
proc.kill()
with contextlib.suppress(Exception):
proc.wait(timeout=2)
def main() -> int:
chrome = _find_chrome()
if not chrome:
print("ASSET-CACHE-FAILED-no-chrome")
return 2
with tempfile.TemporaryDirectory(prefix="turnstone-asset-cache-e2e-") as raw_scratch:
scratch = Path(raw_scratch)
old_dir, current_dir, static_dir, vendor_paths = _prepare_builds(scratch)
state = CacheState()
server, server_thread, sock, base_url = _start_server(
_make_app(state, old_dir, current_dir, static_dir, vendor_paths)
)
chrome_proc = None
cdp = None
try:
chrome_proc, cdp_port = _launch_chrome(chrome, scratch / "chrome-profile")
cdp = CDP(_page_ws_url(cdp_port))
cdp.cmd("Page.enable")
cdp.cmd("Runtime.enable")
cdp.cmd("Network.enable")
cdp.cmd("Page.navigate", {"url": f"{base_url}/asset-cache-e2e"})
old_result = _wait_for_generation(cdp, "OLD")
state.phase = "current"
cdp.cmd("Page.reload", {"ignoreCache": False})
current_result = _wait_for_generation(cdp, "CURRENT")
if "user_turn=0" not in old_result["url"] or "tool_turn=0" not in old_result["url"]:
raise AssertionError(f"old pane did not expose old capabilities: {old_result!r}")
if (
"user_turn=1" not in current_result["url"]
or "tool_turn=1" not in current_result["url"]
):
raise AssertionError(
f"reloaded pane did not expose current capabilities: {current_result!r}"
)
verdict, vendor_trace = _verify_trace(state, vendor_paths)
cdp.evaluate(f"document.title = {json.dumps(verdict)}")
print(verdict)
print(f" old EventSource: {old_result['url']}")
print(f" current EventSource: {current_result['url']}")
for item in vendor_trace:
print(f" vendor: {item}")
return 0
except Exception as exc:
print(f"ASSET-CACHE-FAILED-{type(exc).__name__}: {exc}")
return 1
finally:
if cdp is not None:
cdp.close()
if chrome_proc is not None:
_stop_process(chrome_proc)
server.should_exit = True
server_thread.join(timeout=10)
with contextlib.suppress(OSError):
sock.close()
if __name__ == "__main__":
raise SystemExit(main())
+189 -2
View File
@@ -74,6 +74,21 @@ Attachments harness (/attachments/livepass.html): the composer attachment
thumbnail crop/size, the native audio-control fit at the constrained
height, the snippet contrast, and how a long filename behaves at the
340px chip cap.
Paste-over-HTTP harness (/paste/livepass.html): the REAL Composer's
large-text paste path, exercised with a browser-generated, trusted paste
event on an explicitly non-secure HTTP origin. No
``navigator.clipboard`` stub, synthetic ``ClipboardEvent``, or
secure-context override is involved. The page requires
``window.isSecureContext === false``, ``event.isTrusted``, a ``text/plain``
clipboard item, canceled inline insertion, and an exact
``pasted-text.txt`` File round-trip before stamping ``PASTE-HTTP-READY``.
It fails closed as ``PASTE-HTTP-FAILED-<reason>``. Serve beyond loopback,
open the page by a LAN address (localhost and 127.0.0.1 are treated as
trustworthy origins by browsers), then use the browser's normal Copy and
Paste commands:
python3 scripts/livepass.py --serve 8950 --bind 0.0.0.0
Task-agent harness (/taskagent/livepass.html): the task_agent card a task
agent's sub-tool steps nested under its conversation row, driven through the
REAL InteractivePane.handleEvent (parent tool_pending/tool_info -> child
@@ -1031,6 +1046,165 @@ ATTACH_TEMPLATE = """<!doctype html>
"""
# --------------------------------------------------------------------------
# Native paste-over-HTTP harness. Unlike the copy-affordance harness, this
# must never stub clipboard access or dispatch a script-created paste event:
# its purpose is to prove that the production ClipboardEvent path still sees
# user-agent clipboard data on a non-secure origin.
# --------------------------------------------------------------------------
PASTE_TEMPLATE = """<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
<title>PASTE-HTTP-BOOTING</title>
<link rel="stylesheet" href="shared/base.css" />
<link rel="stylesheet" href="shared/ui-base.css" />
<link rel="stylesheet" href="shared/chat.css" />
<style>
body {
margin: 0; padding: 32px;
background: var(--bg); color: var(--fg);
font-family: var(--font-sans, system-ui, sans-serif);
}
main { width: min(760px, 100%); margin: 0 auto; }
#paste-source {
box-sizing: border-box; width: 100%; height: 80px;
}
#composer-mount, #probe-status { margin-top: 16px; }
#probe-status { white-space: pre-wrap; }
</style>
</head>
<body>
<main>
<h1>Native paste on plain HTTP</h1>
<p>
This passes only when a trusted paste exposes clipboard text to the
real Composer on a non-secure HTTP origin.
</p>
<p>1. Select this 2001-character fixture, then copy it normally.</p>
<textarea id="paste-source" aria-label="Paste test source"></textarea>
<button id="select-source" type="button">Select source text</button>
<p>2. Focus the composer and paste normally.</p>
<div id="composer-mount"></div>
<pre id="probe-status" role="status" aria-live="polite">Booting</pre>
</main>
<script type="module">
import { Composer } from "./shared/composer.js";
import { PASTE_ATTACHMENT_CHARS } from "./shared/composer_paste_text.js";
const prefix = "TURNSTONE-PASTE-HTTP:";
const seedText =
prefix + "x".repeat(PASTE_ATTACHMENT_CHARS + 1 - prefix.length);
const source = document.getElementById("paste-source");
const status = document.getElementById("probe-status");
source.value = seedText;
let copyTrusted = false;
let paste = null;
let attachment = null;
let fileSettled = false;
let failed = false;
function paint() {
status.textContent = document.title + "\\n" + JSON.stringify({
origin: location.origin,
secureContext: window.isSecureContext,
copyTrusted: copyTrusted,
paste: paste,
attachment: attachment,
}, null, 2);
}
function fail(reason) {
if (failed) return;
failed = true;
document.title = "PASTE-HTTP-FAILED-" + reason;
paint();
}
function finish() {
if (failed || !paste || !fileSettled) return;
const checks = [
["copy-untrusted", copyTrusted],
["paste-untrusted", paste.trusted],
["no-clipboard-data", paste.hasClipboardData],
["no-text-plain", paste.hasPlainText],
["paste-not-canceled", paste.defaultPrevented],
["attach-count", attachment.calls === 1],
["filename", attachment.name === "pasted-text.txt"],
["mime", attachment.type === "text/plain"],
["size", attachment.size === new Blob([seedText]).size],
["content", attachment.contentMatches],
["inline-insert", paste.inputValue === ""],
];
const firstFailure = checks.find(function (entry) { return !entry[1]; });
if (firstFailure) return fail(firstFailure[0]);
document.title = "PASTE-HTTP-READY";
paint();
}
document.addEventListener("copy", function (event) {
copyTrusted = event.isTrusted;
paint();
}, true);
document.addEventListener("paste", function (event) {
const observed = {
trusted: event.isTrusted,
hasClipboardData: !!event.clipboardData,
hasPlainText: !!event.clipboardData &&
Array.from(event.clipboardData.types || []).includes("text/plain"),
};
setTimeout(function () {
paste = Object.assign(observed, {
defaultPrevented: event.defaultPrevented,
inputValue: composer.inputEl.value,
});
if (!attachment) fail("no-attachment");
else finish();
}, 0);
});
const composer = new Composer(document.getElementById("composer-mount"), {
onSend: function () {},
placeholder: "Paste the copied fixture here…",
attachments: {
onAttach: function (file) {
attachment = {
calls: attachment ? attachment.calls + 1 : 1,
name: file.name,
type: file.type,
size: file.size,
contentMatches: false,
};
file.text().then(function (text) {
attachment.contentMatches = text === seedText;
fileSettled = true;
finish();
}, function () { fail("file-read"); });
return true;
},
},
});
document.getElementById("select-source").addEventListener("click", function () {
source.focus();
source.select();
});
if (location.protocol !== "http:") fail("not-http");
else if (window.isSecureContext) fail("secure-context");
else {
document.title = "PASTE-HTTP-WAITING";
paint();
}
</script>
</body>
</html>
"""
# --------------------------------------------------------------------------
# Task-agent harness — the task_agent card: a task agent's sub-tool steps
# nested under its conversation row. Driven through the REAL
@@ -1958,6 +2132,12 @@ def build(out: Path) -> None:
(att / "livepass.html").write_text(ATTACH_TEMPLATE, encoding="utf-8")
print(f"{att}/livepass.html — composer chips + message attachment pills")
paste = out / "paste"
paste.mkdir(parents=True, exist_ok=True)
symlink(paste / "shared", ROOT / "turnstone/shared_static")
(paste / "livepass.html").write_text(PASTE_TEMPLATE, encoding="utf-8")
print(f"{paste}/livepass.html — trusted native paste on insecure HTTP")
ta = out / "taskagent"
ta.mkdir(parents=True, exist_ok=True)
symlink(ta / "shared", ROOT / "turnstone/shared_static")
@@ -2218,6 +2398,12 @@ def main() -> None:
ap = argparse.ArgumentParser(description=__doc__.splitlines()[0])
ap.add_argument("--out", type=Path, default=Path("/tmp/livepass"))
ap.add_argument("--serve", type=int, metavar="PORT")
ap.add_argument(
"--bind",
default="127.0.0.1",
metavar="HOST",
help="listen address for --serve (use 0.0.0.0 for a manual insecure-origin pass)",
)
ap.add_argument("--perf", action="store_true", help="run the perf baseline and exit")
ap.add_argument(
"--perf-n",
@@ -2235,8 +2421,9 @@ def main() -> None:
import functools
handler = functools.partial(_HarnessHandler, directory=str(args.out))
print(f"serving {args.out} on http://localhost:{args.serve}/ — Ctrl+C stops")
http.server.ThreadingHTTPServer(("127.0.0.1", args.serve), handler).serve_forever()
display_host = "localhost" if args.bind == "127.0.0.1" else args.bind
print(f"serving {args.out} on http://{display_host}:{args.serve}/ — Ctrl+C stops")
http.server.ThreadingHTTPServer((args.bind, args.serve), handler).serve_forever()
if __name__ == "__main__":
+686 -268
View File
File diff suppressed because it is too large Load Diff
+2 -2
View File
@@ -185,8 +185,8 @@ esac
echo ""
echo "NOTE: If you added a NEW library (not just updating a version), also update"
echo " the _ASSET_RE regex in turnstone/core/web_helpers.py — its negative lookahead"
echo " skips vendored directories to avoid double-versioning static asset URLs."
echo " _VERSIONED_VENDOR_DIR in turnstone/core/web_helpers.py — it controls both"
echo " HTML version rewriting and immutable static-response caching."
echo ""
echo "Verify the update:"
echo " git diff --stat"
File diff suppressed because it is too large Load Diff
+477 -42
View File
@@ -2,7 +2,7 @@
"openapi": "3.1.0",
"info": {
"title": "turnstone Server API",
"version": "1.8.0a5",
"version": "1.8.0a7",
"description": "Single-node workstream management, chat interaction, and real-time streaming."
},
"paths": {
@@ -55,7 +55,7 @@
"tags": [
"Workstreams"
],
"description": "Accepts two content types. Default is `application/json` with a `CreateWorkstreamRequest` body. Alternatively, `multipart/form-data` with one `meta` field (JSON-encoded `CreateWorkstreamRequest` shape) plus zero-or-more `file` parts saves each file as an attachment under the new workstream. When `initial_message` is also set, attachments are resolved onto that turn before the worker thread dispatches; otherwise they remain pending for a follow-up `POST /v1/api/workstreams/{ws_id}/send`.",
"description": "Accepts two content types. Default is `application/json` with a `CreateWorkstreamRequest` body. Alternatively, `multipart/form-data` with one `meta` field (JSON-encoded `CreateWorkstreamRequest` shape) plus zero-or-more `file` parts saves each file as an attachment under the new workstream. When `initial_message` is also set, attachments are resolved onto that turn before the worker thread dispatches; otherwise they remain pending for a follow-up `POST /v1/api/workstreams/{ws_id}/send`. Setting `resume_ws` atomically forks the visible source history, configuration, project, persona, and attachment references into a distinct destination; it does not reopen or mutate the source. Attachments and `resume_ws` cannot be combined. Creation stays unpublished until validation and the optional fork transaction complete.",
"requestBody": {
"required": true,
"content": {
@@ -87,6 +87,26 @@
}
}
},
"403": {
"description": "Error 403",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ErrorResponse"
}
}
}
},
"404": {
"description": "Error 404",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ErrorResponse"
}
}
}
},
"409": {
"description": "Error 409",
"content": {
@@ -106,6 +126,36 @@
}
}
}
},
"429": {
"description": "Error 429",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ErrorResponse"
}
}
}
},
"500": {
"description": "Error 500",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ErrorResponse"
}
}
}
},
"503": {
"description": "Error 503",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ErrorResponse"
}
}
}
}
}
}
@@ -117,6 +167,7 @@
"tags": [
"Workstreams"
],
"description": "Unloads the live workstream while preserving storage. Returns 409 when any accepted live conversation row still requires persistence reconciliation; the workstream remains loaded and its history journal is retained.",
"parameters": [
{
"name": "ws_id",
@@ -167,6 +218,16 @@
}
}
}
},
"409": {
"description": "Error 409",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ErrorResponse"
}
}
}
}
}
}
@@ -335,7 +396,7 @@
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/StatusResponse"
"$ref": "#/components/schemas/ApproveResponse"
}
}
}
@@ -349,6 +410,16 @@
}
}
}
},
"409": {
"description": "Error 409",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ErrorResponse"
}
}
}
}
}
}
@@ -442,7 +513,7 @@
}
],
"requestBody": {
"required": true,
"required": false,
"content": {
"application/json": {
"schema": {
@@ -457,7 +528,7 @@
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/StatusResponse"
"$ref": "#/components/schemas/CancelResponse"
}
}
}
@@ -492,6 +563,7 @@
"tags": [
"Chat"
],
"description": "Claims the workstream mutation slot, durably truncates the requested tail, then emits clear_ui. Concurrent sends are ordered after the cut; a storage failure returns 503 without changing live history.",
"parameters": [
{
"name": "ws_id",
@@ -542,6 +614,16 @@
}
}
}
},
"503": {
"description": "Error 503",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ErrorResponse"
}
}
}
}
}
}
@@ -553,6 +635,7 @@
"tags": [
"Chat"
],
"description": "Uses one workstream worker claim for the durable truncation and the replacement generation, so another send cannot enter between them. A storage failure returns 503 without changing live history.",
"parameters": [
{
"name": "ws_id",
@@ -593,6 +676,16 @@
}
}
}
},
"503": {
"description": "Error 503",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ErrorResponse"
}
}
}
}
}
}
@@ -604,7 +697,7 @@
"tags": [
"Streaming"
],
"description": "Opens a Server-Sent Events stream scoped to a single workstream. Returns text/event-stream. See API reference for event types.",
"description": "Opens a Server-Sent Events stream scoped to a single workstream. After rendering REST history, pass its opaque handoff_token once as ?history_token=; it names the exact accepted conversation-row prefix used for that render. A history_resync event closes this stream and requires a fresh history read; numeric event replay is not a substitute. Native Last-Event-ID reconnects take priority. Pass ?user_turn=1 to opt into typed accepted-user events; otherwise those rows become a backward-compatible strong-repair frame. Pass ?tool_turn=1 to receive the final accepted tool row as a typed tool_result with accepted=true; without it, accepted tool rows use the same pre-row strong-repair projection. Returns text/event-stream. See API reference for event types.",
"parameters": [
{
"name": "ws_id",
@@ -613,6 +706,42 @@
"schema": {
"type": "string"
}
},
{
"name": "last_event_id",
"in": "query",
"required": false,
"schema": {
"type": "integer"
},
"description": "Numeric per-workstream event cursor for manual reconnects."
},
{
"name": "history_token",
"in": "query",
"required": false,
"schema": {
"type": "string"
},
"description": "Opaque one-shot token naming the accepted prefix rendered from REST history."
},
{
"name": "user_turn",
"in": "query",
"required": false,
"schema": {
"type": "integer"
},
"description": "Set to 1 to receive typed user_turn events instead of history-repair frames."
},
{
"name": "tool_turn",
"in": "query",
"required": false,
"schema": {
"type": "integer"
},
"description": "Set to 1 to receive final accepted tool_result projections."
}
],
"responses": {
@@ -912,7 +1041,7 @@
"tags": [
"Workstreams"
],
"description": "Returns the tail of the conversation in OpenAI-like message format. Persisted-but-not-loaded workstreams (closed / evicted) serve history without rehydrating. Lifted from the coord-only surface in the Stage 2 history/detail verb lift \u2014 interactive previously only exposed history through the SSE replay on ``/events``.",
"description": "Returns the tail of the conversation in OpenAI-like message format. Persisted-but-not-loaded workstreams (closed / evicted) are rehydrated before history is served so every successful response participates in the REST-to-SSE handoff. Lifted from the coord-only surface in the Stage 2 history/detail verb lift \u2014 interactive previously only exposed history through the SSE replay on ``/events``. Messages are the requested limit-bounded tail of one authoritative total accepted conversation-row prefix: user, assistant, tool, and system rows, including projected compaction checkpoints and cancellation markers. The opaque handoff_token names the exact prefix used for the render and is passed once on initial SSE registration. Admission of a later row changes the token; durable acknowledgement does not. If the durable prefix cannot be loaded, the endpoint returns 503 with `History temporarily unavailable`; that response is not authoritative and supplies no usable handoff token.",
"parameters": [
{
"name": "ws_id",
@@ -1740,7 +1869,7 @@
},
"/v1/api/memories": {
"get": {
"summary": "List structured memories",
"summary": "List structured memories. Without a scope, returns global plus the authenticated user's memories; workstream scope is owner-bound.",
"operationId": "v1_api_memories_get",
"tags": [
"Memories"
@@ -1762,7 +1891,7 @@
"schema": {
"type": "string"
},
"description": "Filter by scope"
"description": "Filter by public scope: global, workstream, or user"
},
{
"name": "scope_id",
@@ -1794,6 +1923,46 @@
}
}
}
},
"400": {
"description": "Error 400",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ErrorResponse"
}
}
}
},
"403": {
"description": "Error 403",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ErrorResponse"
}
}
}
},
"404": {
"description": "Error 404",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ErrorResponse"
}
}
}
},
"500": {
"description": "Error 500",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ErrorResponse"
}
}
}
}
}
},
@@ -1833,13 +2002,43 @@
}
}
}
},
"403": {
"description": "Error 403",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ErrorResponse"
}
}
}
},
"404": {
"description": "Error 404",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ErrorResponse"
}
}
}
},
"500": {
"description": "Error 500",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ErrorResponse"
}
}
}
}
}
}
},
"/v1/api/memories/search": {
"post": {
"summary": "Search structured memories by query",
"summary": "Search structured memories by query. Without a scope, searches global plus the authenticated user's memories.",
"operationId": "v1_api_memories_search_post",
"tags": [
"Memories"
@@ -1864,6 +2063,46 @@
}
}
}
},
"400": {
"description": "Error 400",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ErrorResponse"
}
}
}
},
"403": {
"description": "Error 403",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ErrorResponse"
}
}
}
},
"404": {
"description": "Error 404",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ErrorResponse"
}
}
}
},
"500": {
"description": "Error 500",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ErrorResponse"
}
}
}
}
}
}
@@ -1914,6 +2153,26 @@
}
}
},
"400": {
"description": "Error 400",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ErrorResponse"
}
}
}
},
"403": {
"description": "Error 403",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ErrorResponse"
}
}
}
},
"404": {
"description": "Error 404",
"content": {
@@ -1923,6 +2182,16 @@
}
}
}
},
"500": {
"description": "Error 500",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ErrorResponse"
}
}
}
}
}
}
@@ -2264,6 +2533,22 @@
"title": "Message",
"type": "string"
},
"client_send_id": {
"anyOf": [
{
"maxLength": 128,
"minLength": 1,
"pattern": "^[A-Za-z0-9_-]+$",
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"description": "Opaque browser correlation token echoed in the accepted `user_turn` event and history row. It is not an idempotency key; repeated sends with the same value remain distinct turns.",
"title": "Client Send Id"
},
"attachment_ids": {
"anyOf": [
{
@@ -2396,6 +2681,32 @@
"description": "Auto-approve the tools in this batch going forward",
"title": "Always",
"type": "boolean"
},
"cycle_id": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"description": "Resolve this exact approval cycle",
"title": "Cycle Id"
},
"call_id": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"description": "Resolve the approval cycle containing this tool call",
"title": "Call Id"
}
},
"required": [
@@ -2404,10 +2715,35 @@
"title": "ApproveRequest",
"type": "object"
},
"ApproveResponse": {
"properties": {
"status": {
"default": "ok",
"description": "Request outcome",
"title": "Status",
"type": "string"
},
"cycle_id": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"description": "Approval cycle that was resolved, or null when none was pending",
"title": "Cycle Id"
}
},
"title": "ApproveResponse",
"type": "object"
},
"CommandRequest": {
"properties": {
"command": {
"description": "Slash command (e.g. /clear, /new, /resume)",
"description": "Workstream-local slash command (for example /clear or /instructions). Lifecycle commands such as /new and /resume are local-CLI-only; remote clients use the dedicated workstream endpoints.",
"title": "Command",
"type": "string"
},
@@ -2436,6 +2772,24 @@
"title": "CancelRequest",
"type": "object"
},
"CancelResponse": {
"properties": {
"status": {
"default": "ok",
"description": "Request outcome",
"title": "Status",
"type": "string"
},
"dropped": {
"additionalProperties": true,
"description": "Best-effort, credential-redacted snapshot of pending work affected by cancellation; keys are omitted when not observable",
"title": "Dropped",
"type": "object"
}
},
"title": "CancelResponse",
"type": "object"
},
"RewindRequest": {
"properties": {
"turns": {
@@ -2465,15 +2819,43 @@
"title": "Model",
"type": "string"
},
"judge_model": {
"default": "",
"description": "Optional judge model alias for this workstream. Empty uses the server's configured judge model.",
"title": "Judge Model",
"type": "string"
},
"auto_approve": {
"default": false,
"description": "Auto-approve all tool calls",
"title": "Auto Approve",
"type": "boolean"
},
"auto_approve_tools": {
"anyOf": [
{
"type": "string"
},
{
"items": {
"type": "string"
},
"type": "array"
}
],
"default": "",
"description": "Tool names to auto-approve even when auto_approve is false, accepted as either a comma-separated string or an array of strings.",
"title": "Auto Approve Tools"
},
"user_id": {
"default": "",
"description": "Optional workstream owner override. Honored only for trusted service identities (currently the console); ordinary callers remain bound to their authenticated user id.",
"title": "User Id",
"type": "string"
},
"resume_ws": {
"default": "",
"description": "Workstream ID to resume atomically during creation (empty = fresh start)",
"description": "Source workstream ID or alias to fork atomically into the new workstream (empty = fresh start)",
"title": "Resume Ws",
"type": "string"
},
@@ -2510,7 +2892,7 @@
},
"client_type": {
"default": "",
"description": "Client surface type (web, cli, chat). Defaults to web for server-created sessions.",
"description": "Client surface type (web, cli, chat, scheduled). Defaults to web for server-created sessions.",
"title": "Client Type",
"type": "string"
},
@@ -2584,13 +2966,13 @@
},
"resumed": {
"default": false,
"description": "Whether a previous workstream was resumed",
"description": "Whether the requested source was successfully forked",
"title": "Resumed",
"type": "boolean"
},
"message_count": {
"default": 0,
"description": "Number of messages in the resumed workstream",
"description": "Number of messages cloned into the new workstream",
"title": "Message Count",
"type": "integer"
},
@@ -2603,21 +2985,13 @@
"type": "array"
},
"initial_message_status": {
"anyOf": [
{
"enum": [
"queue_full",
"refused_closed"
],
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"description": "Present ONLY when the workstream was created but its initial_message could not be delivered: 'queue_full' (a raced live worker's interjection queue was at capacity \u2014 resend via /send; any uploads stay staged) or 'refused_closed' (the workstream was closed mid-create). Absent whenever the message was dispatched.",
"title": "Initial Message Status"
"enum": [
"queue_full",
"refused_closed"
],
"title": "Initial Message Status",
"type": "string"
}
},
"required": [
@@ -2711,6 +3085,18 @@
],
"default": null,
"title": "Project Id"
},
"persistence_state": {
"default": "healthy",
"description": "Sanitized durable-history status for the loaded workstream: healthy, pending its first save, retrying automatically, or blocked by a permanent commit conflict. Older servers and unloaded rows default to healthy.",
"enum": [
"healthy",
"pending",
"retrying",
"conflict"
],
"title": "Persistence State",
"type": "string"
}
},
"required": [
@@ -2744,6 +3130,18 @@
"$ref": "#/components/schemas/WorkstreamKind",
"default": "interactive"
},
"persistence_state": {
"default": "healthy",
"description": "Sanitized durable-history status: healthy, pending its first save, retrying automatically, or blocked by a permanent commit conflict.",
"enum": [
"healthy",
"pending",
"retrying",
"conflict"
],
"title": "Persistence State",
"type": "string"
},
"pending_approval": {
"default": false,
"description": "True when at least one approval cycle is live (a gate thread parked awaiting an operator approve/deny). Mirrors the same field on ``DashboardWorkstream`` / cluster live projections so a freshly-loaded chat tab can render the inline approval gate from the detail snapshot before SSE replay arrives.",
@@ -2883,7 +3281,7 @@
"type": "string"
},
"messages": {
"description": "Tail of the workstream's message history, projected to the canonical render shape (``role`` may be ``system`` for operator-context turns; flat tool_calls with verdict / output_assessment; top-level source / attachments / reasoning; derived denied / is_error / pending). Bounded by the ``limit`` query parameter (default 100, max 500).",
"description": "Requested limit-bounded tail of one authoritative total accepted conversation-row prefix, projected to the canonical render shape. Roles include ``user``, ``assistant``, ``tool``, and ``system``; compaction checkpoints project as ``role=system, source=compaction`` and cancellation-generated assistant/tool markers appear when present. The projection also carries flat tool_calls with verdict / output_assessment; top-level source / attachments / reasoning; and derived denied / is_error / pending. Bounded by the ``limit`` query parameter (default 100, max 500).",
"items": {
"additionalProperties": true,
"type": "object"
@@ -2903,6 +3301,19 @@
"default": null,
"description": "SSE resume cursor (a ``Last-Event-ID`` value). Non-null only when the trailing turn is an executing in-flight tool batch that the live ring buffer can replay: ``messages`` then omits that turn and the client opens its initial SSE with this cursor so the existing delta replay fast-forwards the in-flight turn (tool calls, results, prompts) instead of the lossy synthetic snapshot. Null on every other read \u2014 the client connects fresh.",
"title": "Cursor"
},
"handoff_token": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"description": "Opaque token naming the exact accepted conversation-row prefix used for this render. Present only while the workstream is loaded. A client that renders this response passes the token once as the initial event stream's ``history_token`` query parameter; the server atomically validates it while registering the listener. Admission of a later row changes the token; durable acknowledgement does not. Clients must not inspect, persist, or reuse it for later reconnects.",
"title": "Handoff Token"
}
},
"required": [
@@ -3060,6 +3471,18 @@
"default": null,
"title": "Project Id"
},
"persistence_state": {
"default": "healthy",
"description": "Sanitized durable-history status for this live row. Contains no storage error, commit key, retry count, or conversation content.",
"enum": [
"healthy",
"pending",
"retrying",
"conflict"
],
"title": "Persistence State",
"type": "string"
},
"pending_approval_details": {
"description": "Inline approval payload for the coordinator children-tree UI: EVERY live approval cycle, oldest first \u2014 parallel task agents gate concurrently, so a workstream can hold several prompts at once. Each entry carries the cycle's items + per-call_id LLM verdict cache so a coord can render approve/deny buttons + judge pill without a separate per-child round-trip; resolve each with its ``cycle_id``. Empty when no approval is pending. Also surfaced (verbatim) on ``GET /v1/api/cluster/ws/live`` via the ``_CLUSTER_WS_LIVE_KEYS`` projection. Replaces 1.6's ``pending_approval_detail`` single-object field (breaking, 1.7).",
"items": {
@@ -3571,32 +3994,42 @@
"properties": {
"name": {
"description": "Memory identifier (normalized to snake_case)",
"maxLength": 256,
"minLength": 1,
"title": "Name",
"type": "string"
},
"content": {
"description": "Memory content",
"maxLength": 65536,
"minLength": 1,
"title": "Content",
"type": "string"
},
"description": {
"default": "",
"description": "Short description for relevance matching",
"description": "Required non-empty description used for relevance matching",
"minLength": 1,
"title": "Description",
"type": "string"
},
"type": {
"default": "general",
"description": "Memory type",
"enum": [
"user",
"general",
"feedback",
"reference"
"anyOf": [
{
"enum": [
"user",
"general",
"feedback",
"reference"
],
"type": "string"
},
{
"type": "null"
}
],
"title": "Type",
"type": "string"
"default": null,
"description": "Memory type; omission preserves it on update and defaults on insert",
"title": "Type"
},
"scope": {
"default": "global",
@@ -3618,7 +4051,8 @@
},
"required": [
"name",
"content"
"content",
"description"
],
"title": "SaveMemoryRequest",
"type": "object"
@@ -3712,6 +4146,7 @@
"properties": {
"query": {
"description": "Search query text",
"minLength": 1,
"title": "Query",
"type": "string"
},
+75 -167
View File
@@ -13,40 +13,6 @@
"vitest": "^4.1"
}
},
"node_modules/@emnapi/core": {
"version": "2.0.0-alpha.3",
"resolved": "https://registry.npmjs.org/@emnapi/core/-/core-2.0.0-alpha.3.tgz",
"integrity": "sha512-AZypUeJ/yByuxyS7BlSNRDOMLMlROYtjYdIAuBmJssVz1UJDSeYxLrdizhXCFYhedC5bqd/ASy8EuNXbVVXp9g==",
"dev": true,
"license": "MIT",
"optional": true,
"dependencies": {
"@emnapi/wasi-threads": "2.0.1",
"tslib": "^2.4.0"
}
},
"node_modules/@emnapi/runtime": {
"version": "2.0.0-alpha.3",
"resolved": "https://registry.npmjs.org/@emnapi/runtime/-/runtime-2.0.0-alpha.3.tgz",
"integrity": "sha512-hFPAhMUjJD9BSyCANEISPOogeXC9Zo9ZQl7L6vKnaVsMkCtzznaW/naYypeyl0Gv5rYfWYsZbpixTMpjDJzQeA==",
"dev": true,
"license": "MIT",
"optional": true,
"dependencies": {
"tslib": "^2.4.0"
}
},
"node_modules/@emnapi/wasi-threads": {
"version": "2.0.1",
"resolved": "https://registry.npmjs.org/@emnapi/wasi-threads/-/wasi-threads-2.0.1.tgz",
"integrity": "sha512-9DsSk+o5NBX0CCJT8s0EROGSGxjR/tKu6aBTaVyq+SjAEQH4XcdcRxPBRzsBLizTTJ49MJjF+jgu3qnO9GLQcQ==",
"dev": true,
"license": "MIT",
"optional": true,
"dependencies": {
"tslib": "^2.4.0"
}
},
"node_modules/@jridgewell/sourcemap-codec": {
"version": "1.5.5",
"resolved": "https://registry.npmjs.org/@jridgewell/sourcemap-codec/-/sourcemap-codec-1.5.5.tgz",
@@ -54,32 +20,10 @@
"dev": true,
"license": "MIT"
},
"node_modules/@napi-rs/wasm-runtime": {
"version": "1.2.2",
"resolved": "https://registry.npmjs.org/@napi-rs/wasm-runtime/-/wasm-runtime-1.2.2.tgz",
"integrity": "sha512-JfB4kuJQjaoHuCTseIINHtHWeJnvgEcxjwA5t/Y00ZgaOO1Crz3fjT/p8kT28zA/Caz7oiUMn3d6H2yOVCVwuw==",
"dev": true,
"license": "MIT",
"optional": true,
"dependencies": {
"@tybys/wasm-util": "^0.10.3"
},
"engines": {
"node": "^20.19.0 || ^22.13.0 || >=23.5.0"
},
"funding": {
"type": "github",
"url": "https://github.com/sponsors/Brooooooklyn"
},
"peerDependencies": {
"@emnapi/core": "^1.7.1 || ^2.0.0-alpha.3",
"@emnapi/runtime": "^1.7.1 || ^2.0.0-alpha.3"
}
},
"node_modules/@oxc-project/types": {
"version": "0.142.0",
"resolved": "https://registry.npmjs.org/@oxc-project/types/-/types-0.142.0.tgz",
"integrity": "sha512-7W+2q5AKQVU36fkaryontrHn3YDt1RyUYXatw9i5H8ocYe2sPKSFB6eS8WNPeRKiN1qAWWZUPm7gwFzJGrccqQ==",
"version": "0.143.0",
"resolved": "https://registry.npmjs.org/@oxc-project/types/-/types-0.143.0.tgz",
"integrity": "sha512-u6JZdLBTLotrNC9Vd6vPssINdzcCzleKAH6EJKImQb7GtYvX5keN2dxkoK44stCc4tffE6QQRtZTXVSzsLUlWA==",
"dev": true,
"license": "MIT",
"funding": {
@@ -87,9 +31,9 @@
}
},
"node_modules/@rolldown/binding-android-arm64": {
"version": "1.2.1",
"resolved": "https://registry.npmjs.org/@rolldown/binding-android-arm64/-/binding-android-arm64-1.2.1.tgz",
"integrity": "sha512-02hOeOSryYxVrOIphmLAsqnCJWxwlzFk+pEt/N/i6OgT3lShHO7xGCU5cpgchRDHboAEbSjzgGh+O/u1GswQmA==",
"version": "1.2.3",
"resolved": "https://registry.npmjs.org/@rolldown/binding-android-arm64/-/binding-android-arm64-1.2.3.tgz",
"integrity": "sha512-zrJtHDcaZJ1Fp7xf4hNl+7seH9Cn/N5TwLYkhgXREtBwAd/jaqW3uqeHxpDugJLVICWg4eW44kOQEGJ1r6jCGw==",
"cpu": [
"arm64"
],
@@ -104,9 +48,9 @@
}
},
"node_modules/@rolldown/binding-darwin-arm64": {
"version": "1.2.1",
"resolved": "https://registry.npmjs.org/@rolldown/binding-darwin-arm64/-/binding-darwin-arm64-1.2.1.tgz",
"integrity": "sha512-fMsTOnN0OjFm3CyppWPitKnc8UlliVARUULW6cfU6AIqjdtgmSFWSk9vecHzZduv/yMWIHDlRhM1e8Iff9uAfA==",
"version": "1.2.3",
"resolved": "https://registry.npmjs.org/@rolldown/binding-darwin-arm64/-/binding-darwin-arm64-1.2.3.tgz",
"integrity": "sha512-ieIiibVCp0tX7TLu2cafoNPv8wJyYi01ekXpbf8q2j7F4rGAhhXb/eQh7ge9DRBY78GwmRQtvjZDux7EDbA8kA==",
"cpu": [
"arm64"
],
@@ -121,9 +65,9 @@
}
},
"node_modules/@rolldown/binding-darwin-x64": {
"version": "1.2.1",
"resolved": "https://registry.npmjs.org/@rolldown/binding-darwin-x64/-/binding-darwin-x64-1.2.1.tgz",
"integrity": "sha512-1wjKdz/XLGKHaTNHjQveQ/B23TKx4ItAqm1JbyVuvNPc4Ze0Fb48s49TAd/2zcplPl8okE/UbTgmlVfwT7eFeQ==",
"version": "1.2.3",
"resolved": "https://registry.npmjs.org/@rolldown/binding-darwin-x64/-/binding-darwin-x64-1.2.3.tgz",
"integrity": "sha512-Zh9tCon19eDXJoihx0rqKhMUlMYqzwj3aPsSuHmI4RWZh62dWUL+DJN4C5YQya5TcQBJU/Fe8+rY0jhXTQITqA==",
"cpu": [
"x64"
],
@@ -138,9 +82,9 @@
}
},
"node_modules/@rolldown/binding-freebsd-x64": {
"version": "1.2.1",
"resolved": "https://registry.npmjs.org/@rolldown/binding-freebsd-x64/-/binding-freebsd-x64-1.2.1.tgz",
"integrity": "sha512-Fa0jHR07E7YBN4vOEsbVf2briYNsuOowfLJaXULZM0ldMlaCaj2LJgLMbMe4iacRyZmvR8efFhgR9wKuGclQUg==",
"version": "1.2.3",
"resolved": "https://registry.npmjs.org/@rolldown/binding-freebsd-x64/-/binding-freebsd-x64-1.2.3.tgz",
"integrity": "sha512-nGbJWewA1wrXXZiQhjAT5rhibGfns5ZNkDVqxsO6zJ3f3YvpoDNNmGMSbbhLuXKjNScaBJVOAboztAWVespQMg==",
"cpu": [
"x64"
],
@@ -155,9 +99,9 @@
}
},
"node_modules/@rolldown/binding-linux-arm-gnueabihf": {
"version": "1.2.1",
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-arm-gnueabihf/-/binding-linux-arm-gnueabihf-1.2.1.tgz",
"integrity": "sha512-pzkgu1SSHGgRRyRZ4fbmSgmajbVt+epaLP99NDjFft69v/ypfTi6swBMiVdh2EkQ0OSnHE1lZDM7DRGkyAzUpA==",
"version": "1.2.3",
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-arm-gnueabihf/-/binding-linux-arm-gnueabihf-1.2.3.tgz",
"integrity": "sha512-QNniJr5Kml0kDEB98jiDOJjXNroxIIi0IXIbdYzY26Xt1pVbeP62+KnoIZLwirOymX/0jDk/2gI/bNUv7A7OIw==",
"cpu": [
"arm"
],
@@ -172,9 +116,9 @@
}
},
"node_modules/@rolldown/binding-linux-arm64-gnu": {
"version": "1.2.1",
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-arm64-gnu/-/binding-linux-arm64-gnu-1.2.1.tgz",
"integrity": "sha512-QI5SEDY8cbiYWHx0VO4vIc3UlS6a32vXHjU8Qy/17adEmZIPuByJg13UEvo9c/UCiUkdcVWY83C+b+JrwnNyUg==",
"version": "1.2.3",
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-arm64-gnu/-/binding-linux-arm64-gnu-1.2.3.tgz",
"integrity": "sha512-TkqEAcmmvH3I/q4114NB4RVt6241Dao48pF45uLcFGrwAaIn0iITgTAKP/dLjbN0R4buJjGb91+UHSoFmpgIWw==",
"cpu": [
"arm64"
],
@@ -192,9 +136,9 @@
}
},
"node_modules/@rolldown/binding-linux-arm64-musl": {
"version": "1.2.1",
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-arm64-musl/-/binding-linux-arm64-musl-1.2.1.tgz",
"integrity": "sha512-Sm41FyCeXqmYcERoYOCbGIL5hNfd8w9LQ7Y61Bev48HkcjaJqV/iiVOaiDxjVTRMS+QKrZmD8cfPt4uMVnvM+A==",
"version": "1.2.3",
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-arm64-musl/-/binding-linux-arm64-musl-1.2.3.tgz",
"integrity": "sha512-NHqjnxpsndf4MPymxteFAWHHfkTL8HjWh1KB7z23ofZ6QO2euONuxDXjat69dKZRALnGypg8k8SsK8vZJoXv1Q==",
"cpu": [
"arm64"
],
@@ -212,9 +156,9 @@
}
},
"node_modules/@rolldown/binding-linux-ppc64-gnu": {
"version": "1.2.1",
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-ppc64-gnu/-/binding-linux-ppc64-gnu-1.2.1.tgz",
"integrity": "sha512-2x+WhXTGl9yJYPbltW/BSEPTVz9OIWQyER4N+gJEDWkkn904eRcBzELqh/Hf7K0w/ubGbKNMv0ZC+94QK/IFEg==",
"version": "1.2.3",
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-ppc64-gnu/-/binding-linux-ppc64-gnu-1.2.3.tgz",
"integrity": "sha512-6tbrbwfz5GB9DQ4Jwo6hy9v+vR31xZlvzZ6n5Xut6Hhx5PvrA9q/HsK8KMaYQp063iqZGXwNvZtYNLD7EM/x0w==",
"cpu": [
"ppc64"
],
@@ -232,9 +176,9 @@
}
},
"node_modules/@rolldown/binding-linux-s390x-gnu": {
"version": "1.2.1",
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-s390x-gnu/-/binding-linux-s390x-gnu-1.2.1.tgz",
"integrity": "sha512-eEjmQpuRQayHPWWnywaWHkFT3ToPbP3RYy42VVd/B9aBGDA+Ol25EIWHxKQST3IiWJjikCWUF7KtbfqwZrzVwQ==",
"version": "1.2.3",
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-s390x-gnu/-/binding-linux-s390x-gnu-1.2.3.tgz",
"integrity": "sha512-oyuXxXmoZHjXC917IAPFAAv4wWAa0cM9afk8nx1+9/jNNOX1uPf8yDA6p7G0RypOfw/X0PQt5IfoquY1um+zSg==",
"cpu": [
"s390x"
],
@@ -252,9 +196,9 @@
}
},
"node_modules/@rolldown/binding-linux-x64-gnu": {
"version": "1.2.1",
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-x64-gnu/-/binding-linux-x64-gnu-1.2.1.tgz",
"integrity": "sha512-/Orga1fZYkLc/56jBICcHrKchl8Z2UKdDSr3LG9ToWO1lQ6a4Livk9Xz+9WN91zsz5QR3XQz2NNoSDEvP6qadw==",
"version": "1.2.3",
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-x64-gnu/-/binding-linux-x64-gnu-1.2.3.tgz",
"integrity": "sha512-TytMwF2KVGqP2tgd0I1OY0PAv78dZRAYcF5ssDzjM34SUXCED3uXvSd5+lHoC0bTD6eEdFz7LdQNCO1y0oVk9w==",
"cpu": [
"x64"
],
@@ -272,9 +216,9 @@
}
},
"node_modules/@rolldown/binding-linux-x64-musl": {
"version": "1.2.1",
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-x64-musl/-/binding-linux-x64-musl-1.2.1.tgz",
"integrity": "sha512-xxBJRL+0q0Kce7orznGWLuylHDY65vuARXZRpX+hPdv+DqK2c3NlCsVA98tlWzWNEE7yPqA/1NQ5nnCrj49Y5A==",
"version": "1.2.3",
"resolved": "https://registry.npmjs.org/@rolldown/binding-linux-x64-musl/-/binding-linux-x64-musl-1.2.3.tgz",
"integrity": "sha512-/E9m3qstrJFVPoULV25mVQblSNExY2+kBsYe4sy0Tn0yOOgJ8wZbZt3KnRbF/XeU2Gl1STKUQnDNTqhIE5MD4A==",
"cpu": [
"x64"
],
@@ -292,9 +236,9 @@
}
},
"node_modules/@rolldown/binding-openharmony-arm64": {
"version": "1.2.1",
"resolved": "https://registry.npmjs.org/@rolldown/binding-openharmony-arm64/-/binding-openharmony-arm64-1.2.1.tgz",
"integrity": "sha512-M6AdXIXw3s+/8XpKMzdGDEXGS1S7kwUsy+rcTIUIOx5Ge4nXKCtAFHFV9YKkXvGcC5WMoTjAteLzlsQROVI0Yw==",
"version": "1.2.3",
"resolved": "https://registry.npmjs.org/@rolldown/binding-openharmony-arm64/-/binding-openharmony-arm64-1.2.3.tgz",
"integrity": "sha512-Kr0OcsoQI816i6HOl3vFHpd1K0eZyh76zgfj4c1nTyaTsd5r2Mj1lwM4R90y/qaCfmTn9eHy0SKwi98eitRxug==",
"cpu": [
"arm64"
],
@@ -308,26 +252,10 @@
"node": "^20.19.0 || >=22.12.0"
}
},
"node_modules/@rolldown/binding-wasm32-wasi": {
"version": "1.2.1",
"resolved": "https://registry.npmjs.org/@rolldown/binding-wasm32-wasi/-/binding-wasm32-wasi-1.2.1.tgz",
"integrity": "sha512-/TX0SoRGojHzSAHpfVBbavRVSazg5U3h3Y3VXfcc0cdugq6kxdqw8LPGFiPr+/7gE/60zRcsOY2Vi9b9eT0jww==",
"dev": true,
"license": "MIT",
"optional": true,
"dependencies": {
"@emnapi/core": "2.0.0-alpha.3",
"@emnapi/runtime": "2.0.0-alpha.3",
"@napi-rs/wasm-runtime": "^1.2.0"
},
"engines": {
"node": "^20.19.0 || ^22.13.0 || >=23.5.0"
}
},
"node_modules/@rolldown/binding-win32-arm64-msvc": {
"version": "1.2.1",
"resolved": "https://registry.npmjs.org/@rolldown/binding-win32-arm64-msvc/-/binding-win32-arm64-msvc-1.2.1.tgz",
"integrity": "sha512-EvRrivJieyHG+AO9lleZWgq+g0+S7oV2C51yuqlcyU/R9net+sI4Pj0F+lUoP2bEr6TWX3SqFaaS0SzfLxSzkw==",
"version": "1.2.3",
"resolved": "https://registry.npmjs.org/@rolldown/binding-win32-arm64-msvc/-/binding-win32-arm64-msvc-1.2.3.tgz",
"integrity": "sha512-hOtMwTqnME+/gJcH/PCZ0wn0zPUjiWOgkHpxbSJpfGKMezHltx1S7/k1SitzVa7Ww2cqrDDaFbZEhcJZO8o+Jw==",
"cpu": [
"arm64"
],
@@ -342,9 +270,9 @@
}
},
"node_modules/@rolldown/binding-win32-x64-msvc": {
"version": "1.2.1",
"resolved": "https://registry.npmjs.org/@rolldown/binding-win32-x64-msvc/-/binding-win32-x64-msvc-1.2.1.tgz",
"integrity": "sha512-Z4eCmn5QJ/5+azF9knpLWKfVd9aidn0mAe9TpJgvBLId9Ax3t0+JVxBmT25Bv7NBbVW1TZyKjQjQReouMeH5UQ==",
"version": "1.2.3",
"resolved": "https://registry.npmjs.org/@rolldown/binding-win32-x64-msvc/-/binding-win32-x64-msvc-1.2.3.tgz",
"integrity": "sha512-ekcqMMkI2PlhYnfzQnB/cEdYUVVJViWvoUyLrbzgDoi3Snfc1mVBwdnc306ufA5ejy8JSPjT2RlW1nQSjW7efg==",
"cpu": [
"x64"
],
@@ -372,17 +300,6 @@
"dev": true,
"license": "MIT"
},
"node_modules/@tybys/wasm-util": {
"version": "0.10.3",
"resolved": "https://registry.npmjs.org/@tybys/wasm-util/-/wasm-util-0.10.3.tgz",
"integrity": "sha512-F3fo1MYrRJYL3zER0OUOmkutjr1Vp23m7OsSgp7nq4SP6OqX6C/56XFIPAl5bt3zaBRjmW7SGz3u/6LwFpYcOg==",
"dev": true,
"license": "MIT",
"optional": true,
"dependencies": {
"tslib": "^2.4.0"
}
},
"node_modules/@types/chai": {
"version": "5.2.3",
"resolved": "https://registry.npmjs.org/@types/chai/-/chai-5.2.3.tgz",
@@ -1242,9 +1159,9 @@
}
},
"node_modules/nanoid": {
"version": "3.3.16",
"resolved": "https://registry.npmjs.org/nanoid/-/nanoid-3.3.16.tgz",
"integrity": "sha512-bzlKTyNJ7+LdGIIwy8ijFpIqEQIvafahV7eYykJ8Cvh42EdJeODoJ6gUJXpQJvej1BddH8OqTXZNE/KfbWAu8Q==",
"version": "3.3.18",
"resolved": "https://registry.npmjs.org/nanoid/-/nanoid-3.3.18.tgz",
"integrity": "sha512-DTg4MJbGMWkfi6VZFdNt2/caMbQy4Ou+Op/hJQvGEWcnVfoA1QA+xzRKAzw9jD6+GVOOeYr/mIcuDSdug6F6+w==",
"dev": true,
"funding": [
{
@@ -1302,9 +1219,9 @@
}
},
"node_modules/postcss": {
"version": "8.5.25",
"resolved": "https://registry.npmjs.org/postcss/-/postcss-8.5.25.tgz",
"integrity": "sha512-DTPx3RWSSnWyzLxQnlH0rJP+EW5ekl16ZU4/psbIhA0e53kJfdgaN5vKM+xP7yJtXVu+nfdVFmlgFDEKAe4Pyw==",
"version": "8.5.26",
"resolved": "https://registry.npmjs.org/postcss/-/postcss-8.5.26.tgz",
"integrity": "sha512-u82N74LFzG8ca+dD8puPnplTXoGH4fTPpVGuIbt36G3qvNlkvfD0lEAZSxaly3KX8TS/L1A1gsCEmvKmBcVbkQ==",
"dev": true,
"funding": [
{
@@ -1322,7 +1239,7 @@
],
"license": "MIT",
"dependencies": {
"nanoid": "^3.3.16",
"nanoid": "^3.3.17",
"picocolors": "^1.1.1",
"source-map-js": "^1.2.1"
},
@@ -1331,13 +1248,13 @@
}
},
"node_modules/rolldown": {
"version": "1.2.1",
"resolved": "https://registry.npmjs.org/rolldown/-/rolldown-1.2.1.tgz",
"integrity": "sha512-4FKJhg8d3OiyQOA6Q1Q0hoFFpW9/OoX+VsHzpECsdsIZoOArrAK90gl59YK/Z+gnDel45bgJZK03ozH/9bCqEw==",
"version": "1.2.3",
"resolved": "https://registry.npmjs.org/rolldown/-/rolldown-1.2.3.tgz",
"integrity": "sha512-rn9wpmxplLf7NLNyCk9FyWh3FM43DbY8jOzCdEPzH7uflhTftRbCEpqi6Ly2osgoU8OwObtmavMbWLaWy4LX7A==",
"dev": true,
"license": "MIT",
"dependencies": {
"@oxc-project/types": "=0.142.0",
"@oxc-project/types": "=0.143.0",
"@rolldown/pluginutils": "^1.0.0"
},
"bin": {
@@ -1347,21 +1264,20 @@
"node": "^20.19.0 || >=22.12.0"
},
"optionalDependencies": {
"@rolldown/binding-android-arm64": "1.2.1",
"@rolldown/binding-darwin-arm64": "1.2.1",
"@rolldown/binding-darwin-x64": "1.2.1",
"@rolldown/binding-freebsd-x64": "1.2.1",
"@rolldown/binding-linux-arm-gnueabihf": "1.2.1",
"@rolldown/binding-linux-arm64-gnu": "1.2.1",
"@rolldown/binding-linux-arm64-musl": "1.2.1",
"@rolldown/binding-linux-ppc64-gnu": "1.2.1",
"@rolldown/binding-linux-s390x-gnu": "1.2.1",
"@rolldown/binding-linux-x64-gnu": "1.2.1",
"@rolldown/binding-linux-x64-musl": "1.2.1",
"@rolldown/binding-openharmony-arm64": "1.2.1",
"@rolldown/binding-wasm32-wasi": "1.2.1",
"@rolldown/binding-win32-arm64-msvc": "1.2.1",
"@rolldown/binding-win32-x64-msvc": "1.2.1"
"@rolldown/binding-android-arm64": "1.2.3",
"@rolldown/binding-darwin-arm64": "1.2.3",
"@rolldown/binding-darwin-x64": "1.2.3",
"@rolldown/binding-freebsd-x64": "1.2.3",
"@rolldown/binding-linux-arm-gnueabihf": "1.2.3",
"@rolldown/binding-linux-arm64-gnu": "1.2.3",
"@rolldown/binding-linux-arm64-musl": "1.2.3",
"@rolldown/binding-linux-ppc64-gnu": "1.2.3",
"@rolldown/binding-linux-s390x-gnu": "1.2.3",
"@rolldown/binding-linux-x64-gnu": "1.2.3",
"@rolldown/binding-linux-x64-musl": "1.2.3",
"@rolldown/binding-openharmony-arm64": "1.2.3",
"@rolldown/binding-win32-arm64-msvc": "1.2.3",
"@rolldown/binding-win32-x64-msvc": "1.2.3"
}
},
"node_modules/siginfo": {
@@ -1439,14 +1355,6 @@
"node": ">=14.0.0"
}
},
"node_modules/tslib": {
"version": "2.8.1",
"resolved": "https://registry.npmjs.org/tslib/-/tslib-2.8.1.tgz",
"integrity": "sha512-oJFu94HQb+KVduSUQL7wnpmqnfmLsOA/nAh6b6EH0wCEoK0/mPeXU6c3wKDV83MkOuHPRHtSXKKU99IBazS/2w==",
"dev": true,
"license": "0BSD",
"optional": true
},
"node_modules/typescript": {
"version": "7.0.2",
"resolved": "https://registry.npmjs.org/typescript/-/typescript-7.0.2.tgz",
@@ -1483,16 +1391,16 @@
}
},
"node_modules/vite": {
"version": "8.2.0",
"resolved": "https://registry.npmjs.org/vite/-/vite-8.2.0.tgz",
"integrity": "sha512-pn+CFpM0lwDeKwmOq1ZaBK/9sjorZcgqxki6MbY/jPEVd9vichIlmlD4HmQ5wdP5EgqQCFRaACBxMC7uEGc6lQ==",
"version": "8.2.1",
"resolved": "https://registry.npmjs.org/vite/-/vite-8.2.1.tgz",
"integrity": "sha512-EU/eS7BH3XROHh2YnBefjM6DBKA6ZeMZEYQbj7NLWg5wHYlhB8B/Mayd5XsgWq+NFYccDOTemRpdETWR6Ka/lw==",
"dev": true,
"license": "MIT",
"dependencies": {
"lightningcss": "^1.33.0",
"picomatch": "^4.0.5",
"postcss": "^8.5.23",
"rolldown": "~1.2.0",
"postcss": "^8.5.25",
"rolldown": "~1.2.1",
"tinyglobby": "^0.2.17"
},
"bin": {
+12 -6
View File
@@ -18,8 +18,6 @@ import type {
ConsoleCreateWsRequest,
ConsoleCreateWsResponse,
ConsoleHealthResponse,
CreateWorkstreamRequest,
CreateWorkstreamResponse,
ListAttachmentsResponse,
CreateMcpServerRequest,
CreatePolicyOptions,
@@ -39,6 +37,9 @@ import type {
McpServerDetail,
RegistryInstallRequest,
RegistrySearchResponse,
RouteCreateRequest,
RouteCreateResponse,
RouteLiveResponse,
SkillDiscoverResponse,
SkillInfo,
SkillInstallRequest,
@@ -154,10 +155,8 @@ export class TurnstoneConsole extends BaseClient {
* owning node directly.
*/
async routeCreateWorkstream(
opts?: CreateWorkstreamRequest & { target_node?: string },
): Promise<
CreateWorkstreamResponse & { node_url?: string; node_id?: string }
> {
opts?: RouteCreateRequest,
): Promise<RouteCreateResponse> {
const attachments = opts?.attachments;
if (attachments && attachments.length > 0) {
// The console's multipart route_create routes by `?ws_id=` only —
@@ -192,6 +191,13 @@ export class TurnstoneConsole extends BaseClient {
});
}
async routeWorkstreamLive(wsId: string): Promise<RouteLiveResponse> {
return this.request(
"GET",
`/v1/api/route/workstreams/${encodeURIComponent(wsId)}/live`,
);
}
async routeUploadAttachment(
wsId: string,
file: AttachmentUpload,
+60 -1
View File
@@ -1,4 +1,8 @@
import type { ClusterOverviewResponse, ClusterSnapshotNode } from "./types.js";
import type {
ClusterOverviewResponse,
ClusterSnapshotNode,
ConversationPersistenceState,
} from "./types.js";
// ---------------------------------------------------------------------------
// Server SSE events
@@ -35,6 +39,37 @@ export interface HistoryEvent {
messages: Array<Record<string, unknown>>;
}
/**
* The REST history rendered by the caller no longer names the live accepted
* row prefix. Stop this stream, refetch and render history, then open a new
* stream with its cursor and one-shot token. The SDK does not do this
* automatically.
*/
export interface HistoryResyncEvent {
type: "history_resync";
/** Present on registration-time handoff mismatches; implied by a scoped stream. */
ws_id?: string;
reason: string;
}
/** One accepted user row, projected live to every workstream consumer. */
export interface UserTurnEvent {
type: "user_turn";
ws_id?: string;
content: string;
attachments?: Array<{
attachment_id: string;
kind: string;
filename: string;
mime_type: string;
}>;
sender?: string;
source?: string;
/** Optimistic-browser correlation only; not delivery idempotency. */
client_send_ids: string[];
_event_id?: number;
}
export interface ThinkingStartEvent {
type: "thinking_start";
}
@@ -112,6 +147,12 @@ export interface ToolResultEvent {
name: string;
output: string;
is_error?: boolean;
preview?: Record<string, unknown>;
/** True only for the final guarded row accepted into conversation history. */
accepted?: boolean;
effect_status?: string;
/** Monotonic accepted-row identity; present for projection-capable clients. */
_event_id?: number;
}
export interface ToolOutputChunkEvent {
@@ -210,6 +251,8 @@ export interface WsStateEvent {
context_ratio: number;
activity: string;
activity_state: string;
/** Defaults to `healthy` when omitted by an older node. */
persistence_state?: ConversationPersistenceState;
/** Full assistant response text — populated on idle transitions only. */
content?: string;
}
@@ -237,6 +280,8 @@ export interface WsClosedEvent {
export type ServerEvent =
| ConnectedEvent
| HistoryEvent
| HistoryResyncEvent
| UserTurnEvent
| ThinkingStartEvent
| ThinkingStopEvent
| ContentEvent
@@ -284,6 +329,8 @@ export interface ClusterStateEvent {
context_ratio: number;
activity: string;
activity_state: string;
/** Defaults to `healthy` when omitted by an older node. */
persistence_state?: ConversationPersistenceState;
}
export interface ClusterWsCreatedEvent {
@@ -291,6 +338,8 @@ export interface ClusterWsCreatedEvent {
ws_id: string;
node_id: string;
name: string;
/** Defaults to `healthy` when omitted by an older node. */
persistence_state?: ConversationPersistenceState;
}
export interface ClusterWsClosedEvent {
@@ -374,3 +423,13 @@ export function isApprovalResolvedEvent(
export function isCancelledEvent(e: ServerEvent): e is CancelledEvent {
return e.type === "cancelled";
}
export function isHistoryResyncEvent(
e: ServerEvent,
): e is HistoryResyncEvent {
return e.type === "history_resync";
}
export function isUserTurnEvent(e: ServerEvent): e is UserTurnEvent {
return e.type === "user_turn";
}
+13
View File
@@ -30,6 +30,8 @@ export type {
ClusterEvent,
ConnectedEvent,
HistoryEvent,
HistoryResyncEvent,
UserTurnEvent,
ThinkingStartEvent,
ThinkingStopEvent,
ContentEvent,
@@ -71,19 +73,27 @@ export {
isApproveRequestEvent,
isApprovalResolvedEvent,
isCancelledEvent,
isHistoryResyncEvent,
isUserTurnEvent,
} from "./events.js";
// Request/response types
export type {
ConversationPersistenceState,
SendRequest,
SendResponse,
ApproveRequest,
ApproveResponse,
CancelRequest,
CancelResponse,
CommandRequest,
CreateWorkstreamRequest,
CreateWorkstreamResponse,
CloseWorkstreamRequest,
WorkstreamInfo,
ListWorkstreamsResponse,
WorkstreamHistoryResponse,
StreamEventsOptions,
DashboardWorkstream,
DashboardAggregate,
DashboardResponse,
@@ -110,6 +120,9 @@ export type {
NodeDetailResponse,
ConsoleCreateWsRequest,
ConsoleCreateWsResponse,
RouteCreateRequest,
RouteCreateResponse,
RouteLiveResponse,
ConsoleHealthResponse,
CreateScheduleRequest,
UpdateScheduleRequest,
+54 -6
View File
@@ -3,9 +3,11 @@ import type { ServerEvent } from "./events.js";
import type {
AttachmentContent,
AttachmentUpload,
ApproveResponse,
AuthLoginResponse,
AuthSetupResponse,
AuthStatusResponse,
CancelResponse,
CreateWorkstreamRequest,
CreateWorkstreamResponse,
DashboardResponse,
@@ -23,8 +25,10 @@ import type {
SendResponse,
SkillSummary,
StatusResponse,
StreamEventsOptions,
TurnResult,
UploadAttachmentResponse,
WorkstreamHistoryResponse,
} from "./types.js";
function generateWsId(): string {
@@ -111,12 +115,15 @@ export class TurnstoneServer extends BaseClient {
async send(
message: string,
wsId: string,
opts?: { attachmentIds?: string[] },
opts?: { attachmentIds?: string[]; clientSendId?: string },
): Promise<SendResponse> {
const body: Record<string, unknown> = { message };
if (opts?.attachmentIds !== undefined) {
body.attachment_ids = opts.attachmentIds;
}
if (opts?.clientSendId !== undefined) {
body.client_send_id = opts.clientSendId;
}
return this.request(
"POST",
`/v1/api/workstreams/${encodeURIComponent(wsId)}/send`,
@@ -173,7 +180,7 @@ export class TurnstoneServer extends BaseClient {
cycleId?: string;
/** Alternative selector: any call_id inside the target cycle. */
callId?: string;
}): Promise<StatusResponse> {
}): Promise<ApproveResponse> {
return this.request(
"POST",
`/v1/api/workstreams/${encodeURIComponent(opts.wsId)}/approve`,
@@ -201,7 +208,7 @@ export class TurnstoneServer extends BaseClient {
async cancel(
wsId: string,
opts?: { force?: boolean },
): Promise<StatusResponse> {
): Promise<CancelResponse> {
const body: Record<string, unknown> = {};
if (opts?.force) body.force = true;
return this.request(
@@ -229,11 +236,45 @@ export class TurnstoneServer extends BaseClient {
);
}
// -- History ---------------------------------------------------------------
/**
* Return the requested tail of the authoritative total accepted row prefix.
* A 503 is non-authoritative and must not replace an existing transcript.
*/
async getHistory(
wsId: string,
opts?: { limit?: number },
): Promise<WorkstreamHistoryResponse> {
return this.request(
"GET",
`/v1/api/workstreams/${encodeURIComponent(wsId)}/history`,
{ params: { limit: opts?.limit ?? 100 } },
);
}
// -- Streaming ------------------------------------------------------------
async *streamEvents(wsId: string): AsyncIterableIterator<ServerEvent> {
/**
* Open one caller-managed event stream. Pass history hints only after fully
* rendering the corresponding `getHistory()` response. On `history_resync`,
* stop this iterator, refetch and render history, then open a new stream with
* the new hints. No automatic reconnect or transcript repair is performed.
*/
async *streamEvents(
wsId: string,
opts?: StreamEventsOptions,
): AsyncIterableIterator<ServerEvent> {
const params: Record<string, string | number> = { user_turn: 1 };
if (opts?.lastEventId !== undefined) {
params.last_event_id = opts.lastEventId;
}
if (opts?.historyToken) {
params.history_token = opts.historyToken;
}
yield* this.streamSSE<ServerEvent>(
`/v1/api/workstreams/${encodeURIComponent(wsId)}/events`,
params,
);
}
@@ -275,7 +316,7 @@ export class TurnstoneServer extends BaseClient {
// Start consuming the per-workstream SSE stream first
const events = this.streamSSE<ServerEvent>(
`/v1/api/workstreams/${encodeURIComponent(wsId)}/events`,
undefined,
{ user_turn: 1 },
controller.signal,
);
@@ -353,7 +394,14 @@ export class TurnstoneServer extends BaseClient {
}
async saveMemory(opts: SaveMemoryRequest): Promise<MemoryInfo> {
return this.request("POST", "/v1/api/memories", { json: opts });
if (typeof opts.description !== "string" || !opts.description.trim()) {
throw new TypeError(
"memory description is required and must be non-empty",
);
}
return this.request("POST", "/v1/api/memories", {
json: { ...opts, description: opts.description.trim() },
});
}
async searchMemories(
+94 -5
View File
@@ -2,6 +2,13 @@
// Shared types
// ---------------------------------------------------------------------------
/** Sanitized operator-visible state of accepted conversation persistence. */
export type ConversationPersistenceState =
| "healthy"
| "pending"
| "retrying"
| "conflict";
export interface ErrorResponse {
error: string;
}
@@ -56,6 +63,11 @@ export interface SendRequest {
* workstream are auto-consumed; an empty list disables auto-consume.
*/
attachment_ids?: string[];
/**
* Opaque optimistic-send correlation echoed by user_turn/history.
* Reusing it does not collapse or deduplicate accepted turns.
*/
client_send_id?: string;
}
export interface SendResponse {
@@ -116,7 +128,26 @@ export interface ApproveRequest {
approved: boolean;
feedback?: string | null;
always?: boolean;
ws_id: string;
/** Resolve exactly this approval cycle. */
cycle_id?: string | null;
/** Resolve the approval cycle containing this tool call. */
call_id?: string | null;
}
export interface ApproveResponse {
status: string;
/** The cycle resolved by the request, or null when none was pending. */
cycle_id: string | null;
}
export interface CancelRequest {
force?: boolean;
}
export interface CancelResponse {
status: string;
/** Credential-redacted snapshot of pending work affected by cancellation. */
dropped: Record<string, unknown>;
}
export interface CommandRequest {
@@ -128,7 +159,20 @@ export interface CreateWorkstreamRequest {
name?: string;
model?: string;
auto_approve?: boolean;
/** Tool names accepted as a CSV string or array; blanks are removed server-side. */
auto_approve_tools?: string | string[];
/** Override judge model alias for this workstream. */
judge_model?: string;
/**
* Owner override for trusted service identities. Ordinary callers remain
* bound to their authenticated principal.
*/
user_id?: string;
resume_ws?: string;
/** Completion-notification targets as JSON text or structured target objects. */
notify_targets?: string | Array<Record<string, string>>;
/** Client surface label such as web, cli, chat, or scheduled. */
client_type?: string;
skill?: string;
/**
* Persona name (slug) to create the workstream with. Resolved and
@@ -193,6 +237,8 @@ export interface WorkstreamInfo {
parent_ws_id: string | null;
user_id: string;
project_id: string | null;
/** Defaults to `healthy` when omitted by an older node. */
persistence_state?: ConversationPersistenceState;
}
export interface ListWorkstreamsResponse {
@@ -208,14 +254,33 @@ export interface WorkstreamDetailResponse {
state: string;
user_id: string;
kind: string;
/** Defaults to `healthy` when omitted by an older node. */
persistence_state?: ConversationPersistenceState;
}
export interface WorkstreamHistoryResponse {
ws_id: string;
// Tail of the workstream's reconstructed message history
// (provider-fidelity OpenAI-like shape). Bounded by the ?limit=
// query param (default 100, max 500).
/**
* Requested limit-bounded tail of the authoritative total accepted
* conversation-row prefix.
* Roles include user, assistant, tool, and system; projected compaction and
* cancellation markers participate in the same prefix.
*/
messages: Record<string, unknown>[];
/** Initial event-ring cursor returned by the history projection, if needed. */
cursor: number | null;
/**
* Opaque one-shot token naming the exact live prefix used for this render.
* Null for a workstream that is not currently loaded.
*/
handoff_token: string | null;
}
export interface StreamEventsOptions {
/** Initial event-ring cursor, normally copied from `getHistory()`. */
lastEventId?: number;
/** One-shot live-prefix token, copied only from the history just rendered. */
historyToken?: string;
}
export interface DashboardWorkstream {
@@ -232,6 +297,8 @@ export interface DashboardWorkstream {
node?: string;
model?: string;
model_alias?: string;
/** Defaults to `healthy` when omitted by an older node. */
persistence_state?: ConversationPersistenceState;
}
export interface DashboardAggregate {
@@ -498,6 +565,8 @@ export interface ClusterWorkstreamInfo {
activity?: string;
activity_state?: string;
tool_calls?: number;
/** Defaults to `healthy` when omitted by an older node. */
persistence_state?: ConversationPersistenceState;
}
export interface ClusterWorkstreamsResponse {
@@ -541,7 +610,11 @@ export interface ConsoleCreateWsRequest {
skill?: string;
/** Persona slug — resolved and snapshotted at creation. */
persona?: string;
/** Project to attach the workstream to. */
project_id?: string;
resume_ws?: string;
/** Override judge model alias for this workstream. */
judge_model?: string;
}
export interface ConsoleCreateWsResponse {
@@ -550,6 +623,22 @@ export interface ConsoleCreateWsResponse {
target_node: string;
}
export interface RouteCreateRequest extends CreateWorkstreamRequest {
/** Pin placement to this node by generating a matching rendezvous key. */
target_node?: string;
}
export interface RouteCreateResponse extends CreateWorkstreamResponse {
node_url: string;
node_id: string;
routing_strategy: "rendezvous" | "target_node" | "resume";
}
export interface RouteLiveResponse {
ws_id: string;
live: boolean;
}
export interface ConsoleHealthResponse {
status: string;
service: string;
@@ -805,7 +894,7 @@ export interface WorkstreamsOptions {
export interface SaveMemoryRequest {
name: string;
content: string;
description?: string;
description: string;
type?: "user" | "general" | "feedback" | "reference";
scope?: "global" | "workstream" | "user";
scope_id?: string;
+67
View File
@@ -62,6 +62,58 @@ describe("TurnstoneConsole", () => {
expect(url).toContain("page=2");
});
it("createWorkstream sends the live cluster-create contract", async () => {
const fetchFn = mockFetch({
status: "ok",
correlation_id: "ws-new",
target_node: "node-a",
});
const client = new TurnstoneConsole({
baseUrl: "http://test",
fetch: fetchFn,
});
await client.createWorkstream({
node_id: "node-a",
project_id: "project-42",
judge_model: "judge-fast",
});
const [, init] = (fetchFn as ReturnType<typeof vi.fn>).mock.calls[0];
expect(JSON.parse(init.body)).toEqual({
node_id: "node-a",
project_id: "project-42",
judge_model: "judge-fast",
});
});
it("routeCreateWorkstream returns placement metadata", async () => {
const fetchFn = mockFetch({
ws_id: "ws-new",
name: "routed",
node_url: "http://node-a:8080",
node_id: "node-a",
routing_strategy: "target_node",
});
const client = new TurnstoneConsole({
baseUrl: "http://test",
fetch: fetchFn,
});
const response = await client.routeCreateWorkstream({
name: "routed",
target_node: "node-a",
client_type: "scheduled",
notify_targets: [{ channel_type: "slack", channel_id: "C123" }],
});
expect(response.node_id).toBe("node-a");
expect(response.routing_strategy).toBe("target_node");
const [, init] = (fetchFn as ReturnType<typeof vi.fn>).mock.calls[0];
expect(JSON.parse(init.body)).toMatchObject({
client_type: "scheduled",
notify_targets: [{ channel_type: "slack", channel_id: "C123" }],
});
});
it("routeCreateWorkstream rejects attachments + target_node", async () => {
const fetchFn = vi.fn().mockResolvedValue(
new Response("{}", {
@@ -84,6 +136,21 @@ describe("TurnstoneConsole", () => {
expect(fetchFn).not.toHaveBeenCalled();
});
it("routeWorkstreamLive returns the non-mutating liveness probe", async () => {
const fetchFn = mockFetch({ ws_id: "saved/ws", live: true });
const client = new TurnstoneConsole({
baseUrl: "http://test",
fetch: fetchFn,
});
const response = await client.routeWorkstreamLive("saved/ws");
expect(response).toEqual({ ws_id: "saved/ws", live: true });
const [url, init] = (fetchFn as ReturnType<typeof vi.fn>).mock.calls[0];
expect(url).toContain("/v1/api/route/workstreams/saved%2Fws/live");
expect(init.method).toBe("GET");
});
it("health returns parsed response", async () => {
const fetchFn = mockFetch({
status: "ok",
+45
View File
@@ -8,6 +8,8 @@ import {
isApproveRequestEvent,
isApprovalResolvedEvent,
isReasoningEvent,
isHistoryResyncEvent,
isUserTurnEvent,
} from "../src/events.js";
import type { ServerEvent } from "../src/events.js";
@@ -44,6 +46,26 @@ describe("event type guards", () => {
expect(isToolResultEvent(e)).toBe(true);
});
it("carries accepted tool projection metadata", () => {
const e: ServerEvent = {
type: "tool_result",
call_id: "c-final",
name: "open_preview",
output: "guarded\nscalar",
is_error: true,
preview: { kind: "html", attachment_id: "preview-1" },
accepted: true,
effect_status: "unknown",
_event_id: 42,
};
expect(isToolResultEvent(e)).toBe(true);
if (!isToolResultEvent(e)) throw new Error("tool result type guard failed");
expect(e.accepted).toBe(true);
expect(e.preview).toEqual({ kind: "html", attachment_id: "preview-1" });
expect(e.effect_status).toBe("unknown");
expect(e._event_id).toBe(42);
});
it("isWsStateEvent", () => {
const e: ServerEvent = {
type: "ws_state",
@@ -53,6 +75,7 @@ describe("event type guards", () => {
context_ratio: 0,
activity: "",
activity_state: "",
persistence_state: "retrying",
};
expect(isWsStateEvent(e)).toBe(true);
});
@@ -70,4 +93,26 @@ describe("event type guards", () => {
};
expect(isApprovalResolvedEvent(e)).toBe(true);
});
it("isHistoryResyncEvent", () => {
const e: ServerEvent = {
type: "history_resync",
ws_id: "ws1",
reason: "handoff_mismatch",
};
expect(isHistoryResyncEvent(e)).toBe(true);
expect(isContentEvent(e)).toBe(false);
});
it("isUserTurnEvent", () => {
const e: ServerEvent = {
type: "user_turn",
content: "hello",
sender: "user-1",
client_send_ids: ["browser-send"],
_event_id: 17,
};
expect(isUserTurnEvent(e)).toBe(true);
expect(isContentEvent(e)).toBe(false);
});
});
+154 -2
View File
@@ -58,11 +58,58 @@ describe("TurnstoneServer", () => {
baseUrl: "http://test",
fetch: fetchFn,
});
const resp = await client.createWorkstream({ name: "Analysis" });
const resp = await client.createWorkstream({
name: "Analysis",
judge_model: "judge-fast",
client_type: "scheduled",
notify_targets: [{ channel_type: "slack", channel_id: "C123" }],
});
expect(resp.ws_id).toBe("ws_new");
const [, init] = (fetchFn as ReturnType<typeof vi.fn>).mock.calls[0];
expect(JSON.parse(init.body)).toEqual({ name: "Analysis" });
expect(JSON.parse(init.body)).toEqual({
name: "Analysis",
judge_model: "judge-fast",
client_type: "scheduled",
notify_targets: [{ channel_type: "slack", channel_id: "C123" }],
});
});
it("saveMemory requires and normalizes the description", async () => {
const fetchFn = mockFetch({
memory_id: "m1",
name: "deployment_process",
description: "Production deployment workflow",
type: "general",
scope: "global",
scope_id: "",
content: "Deploy from main",
created: "2026-08-11T00:00:00",
updated: "2026-08-11T00:00:00",
});
const client = new TurnstoneServer({
baseUrl: "http://test",
fetch: fetchFn,
});
await client.saveMemory({
name: "deployment_process",
content: "Deploy from main",
description: " Production deployment workflow ",
});
const [, init] = (fetchFn as ReturnType<typeof vi.fn>).mock.calls[0];
expect(JSON.parse(init.body)).toMatchObject({
description: "Production deployment workflow",
});
await expect(
client.saveMemory({
name: "deployment_process",
content: "Deploy from main",
description: " ",
}),
).rejects.toThrow("description is required");
expect(fetchFn).toHaveBeenCalledTimes(1);
});
it("send posts correct payload", async () => {
@@ -78,6 +125,111 @@ describe("TurnstoneServer", () => {
expect(JSON.parse(init.body)).toEqual({ message: "Hello" });
});
it("send threads the optional browser correlation token", async () => {
const fetchFn = mockFetch({ status: "ok" });
const client = new TurnstoneServer({
baseUrl: "http://test",
fetch: fetchFn,
});
await client.send("Hello", "ws1", { clientSendId: "browser-send_1" });
const [, init] = (fetchFn as ReturnType<typeof vi.fn>).mock.calls[0];
expect(JSON.parse(init.body)).toEqual({
message: "Hello",
client_send_id: "browser-send_1",
});
});
it("approve selects a cycle without duplicating ws_id in the body", async () => {
const fetchFn = mockFetch({ status: "ok", cycle_id: "cycle-1" });
const client = new TurnstoneServer({
baseUrl: "http://test",
fetch: fetchFn,
});
const response = await client.approve({
wsId: "ws1",
approved: false,
cycleId: "cycle-1",
callId: "call-1",
});
const [url, init] = (fetchFn as ReturnType<typeof vi.fn>).mock.calls[0];
expect(url).toBe("http://test/v1/api/workstreams/ws1/approve");
expect(JSON.parse(init.body)).toEqual({
approved: false,
cycle_id: "cycle-1",
call_id: "call-1",
});
expect(response.cycle_id).toBe("cycle-1");
});
it("cancel preserves the dropped-work snapshot", async () => {
const fetchFn = mockFetch({
status: "cancelled",
dropped: { tool_calls: ["call-1"] },
});
const client = new TurnstoneServer({
baseUrl: "http://test",
fetch: fetchFn,
});
const response = await client.cancel("ws1", { force: true });
const [, init] = (fetchFn as ReturnType<typeof vi.fn>).mock.calls[0];
expect(JSON.parse(init.body)).toEqual({ force: true });
expect(response.dropped).toEqual({ tool_calls: ["call-1"] });
});
it("getHistory returns the cursor and one-shot handoff token", async () => {
const fetchFn = mockFetch({
ws_id: "ws1",
messages: [{ role: "system", source: "compaction", content: "summary" }],
cursor: 0,
handoff_token: "epoch.7",
});
const client = new TurnstoneServer({
baseUrl: "http://test",
fetch: fetchFn,
});
const history = await client.getHistory("ws1", { limit: 42 });
expect(history.cursor).toBe(0);
expect(history.handoff_token).toBe("epoch.7");
const [url] = (fetchFn as ReturnType<typeof vi.fn>).mock.calls[0];
expect(url).toBe("http://test/v1/api/workstreams/ws1/history?limit=42");
});
it("streamEvents forwards caller-managed initial history hints", async () => {
const fetchFn = vi
.fn()
.mockResolvedValue(
new Response(
'data: {"type":"history_resync","ws_id":"ws1","reason":"handoff_mismatch"}\n\n',
{ status: 200, headers: { "content-type": "text/event-stream" } },
),
);
const client = new TurnstoneServer({
baseUrl: "http://test",
fetch: fetchFn,
});
const events = [];
for await (const event of client.streamEvents("ws1", {
lastEventId: 0,
historyToken: "epoch.7",
})) {
events.push(event);
}
expect(events).toEqual([
{ type: "history_resync", ws_id: "ws1", reason: "handoff_mismatch" },
]);
const [url] = (fetchFn as ReturnType<typeof vi.fn>).mock.calls[0];
expect(url).toBe(
"http://test/v1/api/workstreams/ws1/events?user_turn=1&last_event_id=0&history_token=epoch.7",
);
});
it("injects auth header when token provided", async () => {
const fetchFn = mockFetch({ workstreams: [] });
const client = new TurnstoneServer({
+16 -2
View File
@@ -21,6 +21,8 @@ from turnstone.console.collector import ClusterCollector
from turnstone.console.coordinator_adapter import CoordinatorAdapter
from turnstone.console.coordinator_ui import ConsoleCoordinatorUI
from turnstone.core.auth import AuthResult
from turnstone.core.model_registry import ModelConfig
from turnstone.core.providers import ModelCapabilities
from turnstone.core.session_manager import SessionManager
if TYPE_CHECKING:
@@ -72,9 +74,21 @@ class _FakeConfigStore:
def _fake_registry() -> MagicMock:
"""MagicMock whose ``.resolve()`` succeeds so the 503 gate passes."""
"""MagicMock whose legacy and atomic binding resolutions both succeed."""
client = MagicMock()
cfg = ModelConfig(
alias="default",
base_url="https://example.invalid/v1",
api_key="test",
model="gpt-4",
)
provider = MagicMock()
provider.provider_name = "openai"
provider.get_capabilities.return_value = ModelCapabilities()
reg = MagicMock()
reg.resolve.return_value = (MagicMock(), "gpt-4", MagicMock(), 0)
reg.default = "default"
reg.resolve.return_value = (client, cfg.model, cfg, 0)
reg.resolve_binding.return_value = (client, cfg.model, cfg, provider, 0)
return reg
+3
View File
@@ -71,6 +71,9 @@ def patch_session_storage(
calls: list[str] = []
class _Stub:
def get_workstream(self, ws_id: str) -> None:
return None
def is_watch_active(self, watch_id: str) -> bool:
calls.append(watch_id)
if raise_on_is_active:
+135
View File
@@ -43,3 +43,138 @@ def demodulize(path: Path) -> str:
)
src = re.sub(r"^export\s*\{[^}]*\};\s*$", "", src, flags=re.M)
return src
def slice_braced_block(source: str, anchor: int) -> str | None:
"""Slice the ``{ … }`` block starting at/just after ``anchor``.
THE brace walker every JS harness suite shares (the comment-AND-
string-aware superset of the per-suite predecessors, which disagreed
on comment handling and window bounds the same source
reorganization could pass one suite's structural pin while breaking
the other's with a slice-dependent failure). Comment awareness makes
it correct on raw AND pre-stripped input alike. Returns ``None``
when no ``{`` opens within 200 chars of ``anchor`` (a missing brace
must not silently slice some later unrelated block) or the block is
unterminated.
"""
start = source.find("{", anchor)
if start == -1 or start - anchor > 200:
return None
depth = 0
quote = ""
escaped = False
line_comment = False
block_comment = False
i = start
while i < len(source):
ch = source[i]
nxt = source[i + 1] if i + 1 < len(source) else ""
if line_comment:
if ch == "\n":
line_comment = False
elif block_comment:
if ch == "*" and nxt == "/":
block_comment = False
i += 1
elif quote:
if escaped:
escaped = False
elif ch == "\\":
escaped = True
elif ch == quote:
quote = ""
elif ch == "/" and nxt == "/":
line_comment = True
i += 1
elif ch == "/" and nxt == "*":
block_comment = True
i += 1
elif ch in {'"', "'", "`"}:
quote = ch
elif ch == "{":
depth += 1
elif ch == "}":
depth -= 1
if depth == 0:
return source[start : i + 1]
i += 1
return None
def extract_braced(source: str, signature: str) -> str:
"""Extract one JS function/method (signature included) — raising form.
``signature`` must end at its opening ``{``. The loud sibling of
:func:`slice_braced_block` for suites that treat a missing or
unterminated function as a hard failure rather than a skip.
"""
start = source.index(signature)
brace = start + len(signature) - 1
if source[brace] != "{":
raise AssertionError(f"signature does not end at an opening brace: {signature}")
block = slice_braced_block(source, brace)
if block is None:
raise AssertionError(f"unterminated JavaScript function: {signature}")
return source[start:brace] + block
def strip_js_comments(source: str) -> str:
"""Strip ``//`` and ``/* */`` comments for source-pattern assertions —
the single implementation every JS harness suite shares.
STRING-AWARE and OFFSET-PRESERVING (comments become spaces, byte
length identical): a ``//`` inside a string literal (``"https://…"``)
is content, not a comment a string-blind scanner truncates the rest
of the line, and pattern pins then silently assert against corrupted
text (a ``not in`` guard passes vacuously after the pattern it
polices was reintroduced). Length preservation keeps downstream
offset math (brace walkers, ``.index`` comparisons) valid. This is
the strict superset of every per-suite predecessor, hoisted so the
suites cannot diverge again.
Limitation regex literals (``/pattern/flags``) are not detected: a
``//`` inside one would be misread as a line comment. Safe for
every region currently scanned; extend the tracker before scanning a
region with regex literals.
"""
out: list[str] = []
n = len(source)
i = 0
in_str: str | None = None
while i < n:
ch = source[i]
if in_str:
out.append(ch)
if ch == "\\" and i + 1 < n:
out.append(source[i + 1])
i += 2
continue
if ch == in_str:
in_str = None
i += 1
continue
# Line comment: replace with spaces up to newline (preserve
# length so downstream offset math still works).
if ch == "/" and i + 1 < n and source[i + 1] == "/":
j = source.find("\n", i)
if j == -1:
j = n
out.append(" " * (j - i))
i = j
continue
# Block comment: replace with spaces up to closing */.
if ch == "/" and i + 1 < n and source[i + 1] == "*":
j = source.find("*/", i + 2)
if j == -1:
out.append(" " * (n - i))
i = n
continue
out.append(" " * (j + 2 - i))
i = j + 2
continue
if ch in ('"', "'", "`"):
in_str = ch
out.append(ch)
i += 1
return "".join(out)
+7 -2
View File
@@ -34,7 +34,12 @@ import re
from pathlib import Path
from typing import Any
from tests._session_helpers import RecordingUI, make_session, scripted_provider
from tests._session_helpers import (
RecordingUI,
make_session,
replace_session_lane,
scripted_provider,
)
from turnstone.core.providers._protocol import StreamChunk, ToolCallDelta, UsageInfo
from turnstone.core.trajectory import Turn
@@ -171,7 +176,7 @@ def run_scenario(name: str) -> dict[str, Any]:
# exponential delays in a unit run. The retry-notice transform in
# test_832_parity hardcodes the matching "0s" wording.
session._RETRY_BASE_DELAY = 0
session._provider = scripted_provider(SCENARIOS[name])
replace_session_lane(session, provider=scripted_provider(SCENARIOS[name]))
pre_fold = "msgs" in inspect.signature(type(session)._stream_response).parameters
record: dict[str, Any] = {"scenario": name}
+3 -5
View File
@@ -1,10 +1,8 @@
"""Shared mock factory for ``events_replay`` tests.
Both interactive (:func:`turnstone.server._interactive_events_replay`)
and coord (:func:`turnstone.console.server._coord_events_replay`) drive
the same shared preamble at
:func:`turnstone.core.session_replay.session_replay_preamble`. Their
test suites share the underlying mock surface (session.model,
Interactive and coordinator replay tests exercise the same shared preamble at
:func:`turnstone.core.session_replay.session_replay_preamble` plus their
kind-specific tails. Their test suites share the underlying mock surface (session.model,
session.model_alias, session._last_usage, ui._pending_*, ui._ws_lock,
counters); this module is the single home for that shape so a future
field add lands once.
+59 -4
View File
@@ -15,12 +15,13 @@ collect it as a test file — it's an importable utility, not a test.
from __future__ import annotations
import dataclasses
import json
from types import SimpleNamespace
from typing import Any
from unittest.mock import MagicMock
from turnstone.core.model_turn import ModelTurnResult
from turnstone.core.model_turn import ModelTurnResult, resolve_model_binding
from turnstone.core.providers import ModelCapabilities, StreamChunk, ToolCallDelta, UsageInfo
from turnstone.core.session import ChatSession
from turnstone.core.session_ui_base import SessionUIBase
@@ -35,6 +36,46 @@ class NullUI(SessionUIBase):
super().__init__()
_UNCHANGED = object()
def replace_session_lane(
session: Any,
*,
provider: Any = _UNCHANGED,
client: Any = _UNCHANGED,
model: Any = _UNCHANGED,
alias: Any = _UNCHANGED,
capabilities: Any = _UNCHANGED,
) -> Any:
"""Atomically replace selected facets of a test session's model lane.
Production sessions deliberately expose no mutable raw provider/client
slots. Tests that install a scripted provider use this one helper so their
setup follows the same whole-lane replacement rule as registry rebinding.
"""
binding = session._model_binding
old_lane = binding.lane
next_provider = old_lane.provider if provider is _UNCHANGED else provider
next_model = old_lane.model if model is _UNCHANGED else model
if capabilities is _UNCHANGED:
next_capabilities = old_lane.capabilities
if provider is not _UNCHANGED:
next_capabilities = next_provider.get_capabilities(next_model)
else:
next_capabilities = capabilities
lane = dataclasses.replace(
old_lane,
provider=next_provider,
client=old_lane.client if client is _UNCHANGED else client,
model=next_model,
alias=old_lane.alias if alias is _UNCHANGED else alias,
capabilities=next_capabilities,
)
session._model_binding = dataclasses.replace(binding, lane=lane)
return lane
def make_session(**kwargs: Any) -> ChatSession:
"""Build a ChatSession with minimal defaults; tests override
individual fields via kwargs."""
@@ -48,6 +89,20 @@ def make_session(**kwargs: Any) -> ChatSession:
"tool_timeout": 30,
}
defaults.update(kwargs)
registry = defaults.get("registry")
model_alias = defaults.get("model_alias")
if registry is not None and model_alias and defaults.get("model_binding") is None:
binding = resolve_model_binding(
registry,
model_alias,
config_store=defaults.get("config_store"),
)
defaults["client"] = binding.lane.client
defaults["model"] = binding.lane.model
defaults["registry_generation"] = binding.registry_generation
defaults["model_binding"] = binding
if "context_window" not in kwargs and binding.config is not None:
defaults["context_window"] = binding.config.context_window
return ChatSession(**defaults)
@@ -576,15 +631,15 @@ def arm_session(
return iter(nxt) if not hasattr(nxt, "__next__") else nxt
provider.create_streaming = MagicMock(side_effect=_create)
session._provider = provider
replace_session_lane(session, provider=provider)
return provider
def scripted_provider(chunks: list[StreamChunk]) -> MagicMock:
"""Provider fake replaying *chunks*, arming ``cancel_ref`` eagerly.
Assign to ``session._provider`` (never mutate a resolved provider
the create_provider singleton rule above). Each call returns a FRESH
Install with :func:`replace_session_lane` (never mutate a resolved
provider the create_provider singleton rule above). Each call returns a FRESH
iterator over the same script so ladder tests re-drive it; the armed
handle is appended per call, matching the one-handle-per-create
behavior of every real adapter.
+2 -2
View File
@@ -134,9 +134,9 @@ class BrowserlikeSSEClient:
# -- connection lifecycle ------------------------------------------------
def _events_path(self, cursor: str | None) -> str:
path = f"/v1/api/workstreams/{self._ws_id}/events"
path = f"/v1/api/workstreams/{self._ws_id}/events?user_turn=1&tool_turn=1"
if cursor is not None:
path += f"?last_event_id={cursor}"
path += f"&last_event_id={cursor}"
return path
def connect(self, *, native: bool = False, rcvbuf: int | None = None) -> None:
+105
View File
@@ -249,6 +249,7 @@ class RecoveryServer:
# reconnect actually reached the reborn node's real endpoint.
self.global_events_requests = 0
self._history_fail_remaining = 0
self._history_tokenless_remaining = 0
self._history_delay_ms = 0
# A thin pure-ASGI fault layer wrapping the REAL app (the production
# app itself is untouched): count + optionally delay/fail
@@ -281,6 +282,26 @@ class RecoveryServer:
)
await send({"type": "http.response.body", "body": b'{"error": "injected"}'})
return
if self._history_tokenless_remaining > 0:
# Old/malformed server simulation for the strong
# handoff-repair latch. A 200 without handoff_token is
# not proof and must never authorize EventSource.
self._history_tokenless_remaining -= 1
self.history_ok += 1
await send(
{
"type": "http.response.start",
"status": 200,
"headers": [(b"content-type", b"application/json")],
}
)
await send(
{
"type": "http.response.body",
"body": (b'{"ws_id":"compat","messages":[],"cursor":null}'),
}
)
return
# Successful-RESPONSE counter, distinct from the arrival
# bump above. A scenario asserting that a render was
@@ -459,6 +480,85 @@ class RecoveryServer:
result: int | None = get_storage().get_max_event_id(ws_id)
return result
def emit_idle_edge(self, ws_id: str) -> int:
"""Publish one real per-workstream ``state_change: idle`` event.
Recovery scenarios use this test-server pulse when they need an
organic-settle-equivalent edge without admitting another user row.
A normal ``/send`` is not a neutral trigger: it publishes a live
``user_turn``, starts model work, and changes the transcript/counts
these scenarios use to isolate the stale-history backstop.
"""
ws = self._manager.get(ws_id)
ui = ws.ui if ws is not None else None
if not isinstance(ui, SessionUIBase):
raise AssertionError(f"emit_idle_edge: ws {ws_id} has no session UI")
before = ui._event_id
ui.on_state_change("idle")
if ui._event_id != before + 1:
raise AssertionError(
f"emit_idle_edge: expected one event after {before}, got {ui._event_id}"
)
return ui._event_id
def emit_history_resync(self, ws_id: str, reason: str = "recovery_probe") -> int:
"""Publish the real strong repair frame through the ordered UI lane."""
ws = self._manager.get(ws_id)
ui = ws.ui if ws is not None else None
if not isinstance(ui, SessionUIBase):
raise AssertionError(f"emit_history_resync: ws {ws_id} has no session UI")
before = ui._event_id
ui.on_history_resync(reason)
if ui._event_id != before + 1:
raise AssertionError(
f"emit_history_resync: expected one event after {before}, got {ui._event_id}"
)
return ui._event_id
def emit_tool_pending(self, ws_id: str, call_id: str) -> int:
"""Publish a live ``tool_pending`` phase without persisting a turn.
This drives the coordinator's event-owned ``liveToolCalls`` gate in
isolation. ``on_agent_step`` is the production hook that emits this
exact envelope; the synthetic item is intentionally not added to
history, so a later authoritative repaint must remove its DOM shell.
"""
ws = self._manager.get(ws_id)
ui = ws.ui if ws is not None else None
if not isinstance(ui, SessionUIBase):
raise AssertionError(f"emit_tool_pending: ws {ws_id} has no session UI")
before = ui._event_id
ui.on_agent_step(
"",
{
"call_id": call_id,
"func_name": "recovery_probe",
"approval_label": "recovery probe",
"header": "recovery render-gate probe",
"needs_approval": False,
},
)
if ui._event_id != before + 1:
raise AssertionError(
f"emit_tool_pending: expected one event after {before}, got {ui._event_id}"
)
return ui._event_id
def emit_tool_result(self, ws_id: str, call_id: str) -> int:
"""Resolve a tool pulse emitted by :meth:`emit_tool_pending`."""
ws = self._manager.get(ws_id)
ui = ws.ui if ws is not None else None
if not isinstance(ui, SessionUIBase):
raise AssertionError(f"emit_tool_result: ws {ws_id} has no session UI")
before = ui._event_id
ui.on_tool_result(call_id, "recovery_probe", "probe complete")
if ui._event_id != before + 1:
raise AssertionError(
f"emit_tool_result: expected one event after {before}, got {ui._event_id}"
)
return ui._event_id
def fetch_history(self, ws_id: str) -> dict[str, Any]:
r = self._http.get(
f"/v1/api/workstreams/{ws_id}/history",
@@ -501,6 +601,11 @@ class RecoveryServer:
failed refetch the #890 guard-before-wipe must survive."""
self._history_fail_remaining = count
def tokenless_history(self, count: int) -> None:
"""Make the next history responses 200 without a handoff proof."""
self._history_tokenless_remaining = count
def delay_history(self, ms: int) -> None:
"""Hold each ``GET …/history`` ``ms`` ms before forwarding (0
clears). Opens the clear_ui-refetch quiesce window that the row
+125
View File
@@ -0,0 +1,125 @@
"""Scripted-PostgreSQL fakes shared by the storage race-test modules.
One implementation of the scripted connection/result pair and the keyed-save
three-way dispatch, so a backend statement-sequence or signature change is
updated once. The two hand-rolled twins had already diverged before the
round-4 review folded them here: the truncation copy grew a ``SET LOCAL``
arm and ``fetchall``/``scalar`` the prune copy lacked.
"""
from __future__ import annotations
from typing import Any
from turnstone.core.storage import AttachmentWrite
def make_attachment(
attachment_id: str,
content: bytes,
*,
filename: str | None = None,
mime_type: str = "text/plain",
kind: str = "text",
) -> AttachmentWrite:
return AttachmentWrite(
attachment_id=attachment_id,
filename=filename or f"{attachment_id[0]}.txt",
mime_type=mime_type,
size_bytes=len(content),
kind=kind,
content=content,
)
def save_keyed(
backend: Any,
ws_id: str,
kind: str,
*,
content: str,
commit_key: str,
attachments: list[AttachmentWrite] | None = None,
tool_content: str | None = None,
tool_name: str = "read_file",
tool_call_id: str = "call-keyed",
) -> int:
"""Three-way plain/user/tool keyed-save dispatch.
The per-module literals (content, commit keys, attachment multiplicity)
stay at the call sites this owns only the method dispatch, so a
signature change on the three save entry points is threaded once.
"""
if kind == "plain":
return int(backend.save_message(ws_id, "assistant", content, commit_key=commit_key))
if kind == "user":
return int(
backend.save_user_message_with_attachments(
ws_id,
content,
attachments or [],
commit_key=commit_key,
)
)
return int(
backend.save_tool_message_with_attachments(
ws_id,
tool_content if tool_content is not None else content,
tool_name,
tool_call_id,
attachments or [],
commit_key=commit_key,
)
)
class ScriptedPostgresResult:
def __init__(
self,
*,
row: Any | None = None,
rows: list[Any] | None = None,
scalar_value: Any | None = None,
) -> None:
self._row = row
self._rows = rows or []
self._scalar_value = scalar_value
def fetchone(self) -> Any | None:
return self._row
def fetchall(self) -> list[Any]:
return self._rows
def scalar(self) -> Any | None:
return self._scalar_value
def scalar_one_or_none(self) -> Any | None:
return self._scalar_value
class ScriptedPostgresConnection:
def __init__(self, results: list[ScriptedPostgresResult]) -> None:
self._results = results
self.statements: list[Any] = []
self.commits = 0
self.rollbacks = 0
def execute(self, statement: Any, *_args: Any, **_kwargs: Any) -> ScriptedPostgresResult:
self.statements.append(statement)
# Session-scoped tuning (the truncation lock_timeout bound) is not part
# of the scripted result sequence; record it and return an empty result.
if str(statement).startswith("SET LOCAL "):
return ScriptedPostgresResult()
if not self._results:
raise AssertionError("unexpected PostgreSQL statement")
return self._results.pop(0)
def commit(self) -> None:
self.commits += 1
def rollback(self) -> None:
self.rollbacks += 1
def assert_consumed(self) -> None:
assert not self._results, f"unconsumed scripted results: {len(self._results)}"
+14
View File
@@ -15,6 +15,20 @@ from unittest.mock import MagicMock
import pytest
def pytest_sessionstart(session: pytest.Session) -> None:
"""Resolve MCP v1's generic FastMCP settings model for test servers.
MCP 1.29.0 defines ``Settings`` before ``FastMCP``, so its ``lifespan``
annotation remains an unresolved forward reference after import. Rebuild
once, after the module is fully loaded, before any integration fixture
constructs a FastMCP server. Pydantic's public hook is a no-op once the SDK
ships a complete model.
"""
from mcp.server.fastmcp.server import Settings as FastMCPSettings
FastMCPSettings.model_rebuild()
def stop_loop_thread(loop: asyncio.AbstractEventLoop, thread: threading.Thread) -> None:
"""Fully tear down a ``loop.run_forever``-in-a-thread test loop.
+7 -2
View File
@@ -30,7 +30,12 @@ from tests._parity_832 import (
run_scenario,
write_fixture,
)
from tests._session_helpers import RecordingUI, make_session, scripted_provider
from tests._session_helpers import (
RecordingUI,
make_session,
replace_session_lane,
scripted_provider,
)
from turnstone.core.providers._protocol import StreamChunk, UsageInfo
from turnstone.core.trajectory import Turn
@@ -129,7 +134,7 @@ class TestDisplayCommitMirror:
ui = RecordingUI()
session = make_session(ui=ui)
session._RETRY_BASE_DELAY = 0
session._provider = scripted_provider(chunks)
replace_session_lane(session, provider=scripted_provider(chunks))
session.messages.append(Turn.user("hi"))
result = session._stream_response(0)
displayed = "".join(d for k, d in ui.events if k == "content")
+174
View File
@@ -92,6 +92,7 @@ def _seed_model_def(
obo_audience: str = "",
obo_scopes: str = "",
capabilities: str = "{}",
max_concurrency: int = 0,
) -> None:
"""Insert a model definition row directly via the storage API."""
storage.create_model_definition(
@@ -108,6 +109,7 @@ def _seed_model_def(
auth_mode=auth_mode,
obo_audience=obo_audience,
obo_scopes=obo_scopes,
max_concurrency=max_concurrency,
)
@@ -176,6 +178,92 @@ def test_helper_preserves_object_identity(storage: SQLiteBackend) -> None:
assert id(state.coord_registry) == before
def test_concurrent_refresh_cannot_install_older_snapshot_last(
storage: SQLiteBackend,
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""The strict load and in-place reload form one serialized operation.
The first caller captures an older snapshot and pauses inside the loader.
The second caller represents a later committed CRUD write. It must block
before loading until the first install completes, then install the newer
snapshot last. Without the outer refresh lock, the second reload wins
temporarily and the released first caller rolls the registry backward.
"""
from turnstone.console import server as server_module
class _TrackingLock:
def __init__(self) -> None:
self._lock = threading.Lock()
self._attempt_guard = threading.Lock()
self._attempts = 0
self.second_attempted = threading.Event()
def __enter__(self) -> _TrackingLock:
with self._attempt_guard:
self._attempts += 1
if self._attempts == 2:
self.second_attempted.set()
self._lock.acquire()
return self
def __exit__(self, *_exc: object) -> None:
self._lock.release()
tracking_lock = _TrackingLock()
monkeypatch.setattr(server_module, "_COORD_REGISTRY_REFRESH_LOCK", tracking_lock)
first_load_entered = threading.Event()
release_first_load = threading.Event()
second_load_entered = threading.Event()
call_guard = threading.Lock()
call_count = 0
def _load_snapshot(**_kwargs: Any) -> ModelRegistry:
nonlocal call_count
with call_guard:
call_count += 1
call_number = call_count
if call_number == 1:
first_load_entered.set()
assert release_first_load.wait(timeout=5), "test did not release older snapshot"
return _make_registry(alias="local", model="older-snapshot")
second_load_entered.set()
return _make_registry(alias="local", model="newer-snapshot")
monkeypatch.setattr("turnstone.core.model_registry.load_model_registry", _load_snapshot)
state = SimpleNamespace(
coord_registry=_make_registry(alias="local", model="initial"),
coord_registry_error="",
)
errors: list[BaseException] = []
def _run_refresh() -> None:
try:
server_module._refresh_coord_registry(state, storage)
except BaseException as exc: # pragma: no cover - diagnostic capture
errors.append(exc)
older = threading.Thread(target=_run_refresh, daemon=True)
newer = threading.Thread(target=_run_refresh, daemon=True)
older.start()
assert first_load_entered.wait(timeout=5), "older refresh never reached loader"
newer.start()
second_attempted = tracking_lock.second_attempted.wait(timeout=5)
loaded_while_older_blocked = second_load_entered.is_set()
release_first_load.set()
older.join(timeout=5)
newer.join(timeout=5)
assert second_attempted, "newer refresh never attempted the serialization lock"
assert not loaded_while_older_blocked
assert not older.is_alive()
assert not newer.is_alive()
assert errors == []
assert call_count == 2
assert state.coord_registry.get_config("local").model == "newer-snapshot"
def test_helper_noop_when_coord_registry_none(storage: SQLiteBackend) -> None:
"""Console boot with no model rows leaves coord_registry = None.
The helper must not 500 in that state CRUD that lands the FIRST
@@ -2345,6 +2433,91 @@ def test_update_endpoint_refreshes_registry(storage: SQLiteBackend) -> None:
assert registry.get_config("local").model == "new-model"
def test_create_and_update_max_concurrency_refresh_registry(storage: SQLiteBackend) -> None:
_seed_model_def(storage, definition_id="m1", alias="local", model="m")
registry = _make_registry(alias="local", model="m")
client = _make_client(storage, registry)
created = client.post(
"/v1/api/admin/model-definitions",
json={"alias": "limited", "model": "m2", "max_concurrency": 3},
)
assert created.status_code == 200, created.text
assert created.json()["max_concurrency"] == 3
assert registry.get_config("limited").max_concurrency == 3
definition_id = created.json()["definition_id"]
updated = client.put(
f"/v1/api/admin/model-definitions/{definition_id}",
json={"max_concurrency": 0},
)
assert updated.status_code == 200, updated.text
assert updated.json()["max_concurrency"] == 0
assert registry.get_config("limited").max_concurrency == 0
def test_update_omission_preserves_max_concurrency(storage: SQLiteBackend) -> None:
_seed_model_def(
storage,
definition_id="m1",
alias="local",
model="m",
max_concurrency=2,
)
registry = _make_registry(alias="local", model="m")
client = _make_client(storage, registry)
resp = client.put(
"/v1/api/admin/model-definitions/m1",
json={"model": "m2"},
)
assert resp.status_code == 200, resp.text
assert resp.json()["max_concurrency"] == 2
assert storage.get_model_definition("m1")["max_concurrency"] == 2
@pytest.mark.parametrize("invalid", [None, True, "1", 1.0, -1, 2_147_483_648])
def test_create_rejects_invalid_max_concurrency(
storage: SQLiteBackend,
invalid: Any,
) -> None:
client = _make_client(storage, _make_registry())
resp = client.post(
"/v1/api/admin/model-definitions",
json={"alias": "invalid", "model": "m", "max_concurrency": invalid},
)
assert resp.status_code == 400, resp.text
assert "max_concurrency" in resp.json()["error"]
assert storage.get_model_definition_by_alias("invalid") is None
@pytest.mark.parametrize("invalid", [None, True, "1", 1.0, -1, 2_147_483_648])
def test_update_rejects_invalid_max_concurrency(
storage: SQLiteBackend,
invalid: Any,
) -> None:
_seed_model_def(
storage,
definition_id="m1",
alias="local",
model="m",
max_concurrency=2,
)
client = _make_client(storage, _make_registry(alias="local", model="m"))
resp = client.put(
"/v1/api/admin/model-definitions/m1",
json={"max_concurrency": invalid},
)
assert resp.status_code == 400, resp.text
assert "max_concurrency" in resp.json()["error"]
assert storage.get_model_definition("m1")["max_concurrency"] == 2
def test_update_endpoint_skips_refresh_on_empty_body(
storage: SQLiteBackend, monkeypatch: pytest.MonkeyPatch
) -> None:
@@ -3723,6 +3896,7 @@ def test_every_mutable_column_probes_its_classification(
"base_url": "https://other.example/v1",
"api_key": "sk-new",
"context_window": 4096,
"max_concurrency": 2,
"capabilities": {"note": "probe"},
# Arm direction: the loop seeds THIS row disabled, so the probe is the
# gated false→true flip rather than the carved-out disarm.
+4
View File
@@ -91,6 +91,8 @@ class TestServerVersioning:
def test_shared_static_unversioned(self, client):
resp = client.get("/shared/base.css")
assert resp.status_code == 200
assert resp.headers["cache-control"] == "no-cache"
assert resp.headers["etag"]
class TestConsoleVersioning:
@@ -147,3 +149,5 @@ class TestConsoleVersioning:
resp = client.get("/static/app.js")
body = resp.text
assert "/v1/api/cluster" in body
assert resp.headers["cache-control"] == "no-cache"
assert resp.headers["etag"]
+250 -105
View File
@@ -17,6 +17,9 @@ from pathlib import Path
import pytest
from tests._js_harness_helpers import slice_braced_block
from tests._js_harness_helpers import strip_js_comments as _strip_js_comments
_APP_JS = Path(__file__).resolve().parent.parent / "turnstone/ui/static/app.js"
_INTERACTIVE_JS = Path(__file__).resolve().parent.parent / "turnstone/shared_static/interactive.js"
_MCP_ERROR_JS = Path(__file__).resolve().parent.parent / "turnstone/shared_static/mcp_error.js"
@@ -44,6 +47,71 @@ def _pane_method_offset(body: str, name: str) -> int:
return m.start()
def test_close_workstream_maps_unresolved_history_to_plain_retry_copy() -> None:
body = _APP_JS.read_text(encoding="utf-8")
start = body.index("function closeWorkstream(wsId)")
end = body.index("// 10. Dashboard", start)
close = body[start:end]
assert "result.status === 409" in close
assert (
"Conversation history is still being saved. Try ending the session again shortly." in close
)
assert "delete workstreams[wsId]" in close, "successful close behavior must remain intact"
@pytest.mark.parametrize("bundle", [_APP_JS, _CONSOLE_APP_JS], ids=["node", "console"])
def test_dashboard_persistence_badges_render_sanitized_operator_states(bundle: Path) -> None:
"""Both dashboards render the same three non-healthy journal states."""
body = bundle.read_text(encoding="utf-8")
display_anchor = body.index("const PERSISTENCE_DISPLAY =")
display_body = _slice_balanced_body(body, display_anchor)
helper_body = _slice_function_body(body, "appendPersistenceStatus")
assert display_body is not None
assert helper_body is not None
script = f"""
const PERSISTENCE_DISPLAY = {display_body};
const document = {{
createElement: function (tag) {{
return {{
tag: tag, dataset: {{}}, attrs: {{}}, className: "", textContent: "", title: "",
setAttribute: function (name, value) {{ this.attrs[name] = value; }},
}};
}},
}};
function appendPersistenceStatus(container, ws) {helper_body}
function probe(state) {{
const container = {{ children: [], appendChild: function (el) {{ this.children.push(el); }} }};
appendPersistenceStatus(container, {{ persistence_state: state }});
return container.children[0] || null;
}}
console.log(JSON.stringify({{
pending: probe("pending"),
retrying: probe("retrying"),
conflict: probe("conflict"),
healthy: probe("healthy"),
}}));
"""
try:
proc = subprocess.run(
["node", "-e", script],
capture_output=True,
text=True,
timeout=15,
)
except FileNotFoundError:
pytest.skip("node binary not available on PATH")
assert proc.returncode == 0, proc.stderr
rendered = json.loads(proc.stdout)
assert rendered["pending"]["textContent"] == "History save pending"
assert rendered["retrying"]["textContent"] == "History save retrying"
assert rendered["conflict"]["textContent"] == "History save blocked"
assert rendered["conflict"]["dataset"]["state"] == "conflict"
assert "Operator intervention is required" in rendered["conflict"]["title"]
assert rendered["healthy"] is None
def test_switch_tab_opens_an_interactive_pane() -> None:
"""In the L-shell ``switchTab`` is a thin shim onto the PaneManager: it
opens/focuses the session as an interactive pane. The split-pane
@@ -258,6 +326,101 @@ def test_refetch_history_seeds_resume_cursor_only_on_initial_connect() -> None:
)
def test_initial_history_handoff_token_is_one_shot_and_resyncs_on_mismatch() -> None:
"""The opaque /history handoff belongs only to the next SSE bootstrap.
It must survive a hidden-tab deferral, compose with a valid cursor of 0,
and be consumed only after an EventSource is constructed. A server-side
revision mismatch takes the full REST-history path; it must never try to
heal a missing committed row with numeric ring replay.
"""
body = _INTERACTIVE_JS.read_text(encoding="utf-8")
start = body.index(" connectSSE(wsId) {")
end = body.index(" _onVisibilityChange() {", start)
connect = body[start:end]
hidden = connect.index("if (document.hidden)")
capability = connect.index('"user_turn=1"')
handoff_query = connect.index('"history_token="')
construct = connect.index("new EventSource(evtUrl)")
consume = connect.index("this._historyHandoffToken = null;", construct)
assert capability < hidden < handoff_query < construct < consume, (
"the bootstrap token must survive hidden-tab deferral and be consumed "
"only by a successfully constructed EventSource"
)
assert 'evtUrl += "?last_event_id="' in connect
assert '(evtUrl.includes("?") ? "&" : "?")' in connect, (
"history_token must compose with ?last_event_id=0 instead of replacing it"
)
assert connect.count('"user_turn=1"') == 1
refetch_start = body.index("async _refetchHistory(")
refetch_end = body.index("_beginReplayQuiesce(", refetch_start)
refetch = body[refetch_start:refetch_end]
assert re.search(
r"if\s*\(seedCursor\)\s*\{\s*this\._historyHandoffToken\s*=\s*"
r"typeof data\.handoff_token === \"string\"",
refetch,
), "only a seeded history fetch may arm the initial SSE handoff"
mismatch = body.index('case "history_resync"')
truncated = body.index('case "replay_truncated"', mismatch)
mismatch_case = body[mismatch:truncated]
assert "this._historyRepair.begin(this.wsId);" in mismatch_case
assert "last_event_id" not in mismatch_case
# Once the server says the rendered history revision is stale, every
# reconnect chokepoint must fail closed until a new response has rendered
# and supplied its proof. A failed fetch schedules one capped retry; it
# must not fall through to a cursorless/tokenless EventSource. The latch,
# budget, backoff, and parked prompt moved into the shared controller
# (history_handoff.createHistoryHandoffRepair) — those are pinned there,
# once; what stays pinned HERE is the pane's use of it.
repair_guard = connect.index("if (this._historyRepair.isRepairing(wsId))")
assert repair_guard < handoff_query < construct
guard_end = connect.index("if (this._historyHandoffToken != null)", repair_guard)
guard = connect[repair_guard:guard_end]
assert "this._historyRepair.schedule();" in guard
assert "return;" in guard
load_start = body.index("_loadHistoryThenConnect(wsId, manualAttempt = false)")
load_end = body.index("async _refetchHistory(", load_start)
load = body[load_start:load_end]
# Admission before any work, then the budget charge, then exactly one
# handover of the verdict; the non-repair tail keeps its own reconnect.
admit = load.index("this._historyRepair.admitAttempt(manualAttempt)")
start_attempt = load.index("this._historyRepair.startAttempt(manualAttempt,", admit)
settle = load.index("this._historyRepair.settle({", start_attempt)
ordinary = load.index("// Ordinary first paint", settle)
assert admit < start_attempt < settle < ordinary
assert "hasToken: this._historyHandoffToken != null" in load[settle:ordinary]
assert "this.connectSSE(wsId);" not in load[settle:ordinary]
assert "return;" in load[settle:ordinary]
# Both terminal paths invalidate the in-flight load and kill the timer;
# a late retry/fetch settlement cannot resurrect the pane.
assert body.count("pane._historyRepair.clear();") >= 2
# The strong repair attempt has a logical 15s deadline, not merely an
# AbortController timeout: authFetch's Retry-After sleep is not abort-aware
# and old runtimes can lack AbortController entirely. The pane still owns
# this per-attempt bound (the coordinator bounds every /history centrally
# instead), and hands the controller a teardown that expires and settles
# the race so no detached pane waits for the deadline.
assert "Promise.race([" in load
assert "createHistoryHandoffDeadline(" in load
assert "deadlineHandle.promise" in load
assert "HISTORY_HANDOFF_FETCH_TIMEOUT_MS" in load
assert "if (repairAttempt && repairAttempt.expired) return;" in refetch
teardown = load[start_attempt:settle]
assert "deadlineHandle.dispose({ expire: true, resolve: true })" in teardown, (
"the mid-flight teardown must expire AND settle the race through the "
"module's dispose() — direct state-slot pokes are the drift the "
"shared handle exists to prevent."
)
assert "repairCtrl.abort()" in teardown
def test_shared_utils_no_longer_defines_replay_advisories_after_tool() -> None:
"""Operator context (interjections / guard findings / nudges) no longer
rides the tool envelope it is first-class ``{"role": "system"}`` rows
@@ -708,6 +871,22 @@ def test_audio_roles_gated_to_openai_sdk_providers() -> None:
assert '_providerCarriesAudio((md && md.provider) || "openai")' in body
def test_judge_integer_settings_render_as_bounded_number_inputs() -> None:
"""The Judge tab has a custom schema renderer separate from Settings.
Integer settings must not fall through to its text-input branch: doing so
drops the registry's step/min/max affordances for parallel_evaluations.
"""
governance = _CONSOLE_GOVERNANCE_JS.read_text(encoding="utf-8")
start = governance.index("function renderJudgeSettings()")
end = governance.index("\nfunction saveJudgeSetting(", start)
body = governance[start:end]
assert 's.type === "float" || s.type === "int"' in body
assert '(s.type === "int" ? "1" : "0.01")' in body
assert "s.min_value" in body
assert "s.max_value" in body
# Tile keys that are deliberately NOT ``ModelCapabilities`` fields.
# ``supports_rerank`` is a registry-level flag read off the model row.
_NON_DATACLASS_TILES = {"supports_rerank"}
@@ -909,6 +1088,29 @@ def test_model_response_controls_are_capability_driven_and_sparse() -> None:
assert 'apiSurfEl.addEventListener("change", _onModelFieldChange)' in admin
def test_model_max_concurrency_form_round_trips_strict_integer() -> None:
html = _CONSOLE_INDEX.read_text(encoding="utf-8")
admin = _CONSOLE_ADMIN_JS.read_text(encoding="utf-8")
assert 'id="model-max-concurrency"' in html
assert 'max="2147483647"' in html
assert "0 = unlimited" in html
create = _slice_function_body(admin, "showCreateModelModal")
edit = _slice_function_body(admin, "showEditModelModal")
render = _slice_function_body(admin, "_renderModels")
assert create is not None and edit is not None and render is not None
assert 'getElementById("model-max-concurrency").value = "0"' in create
assert "m.max_concurrency != null ? m.max_concurrency : 0" in edit
# submitCreateModel is longer than the balanced-slice helper's bounded
# window; these names are unique to that form path, so whole-file pins are
# both stable and unambiguous.
assert "Number.isInteger(maxConcurrency)" in admin
assert "maxConcurrency > 2147483647" in admin
assert "form.max_concurrency = maxConcurrency" in admin
assert 'overrides.push("limit=" + m.max_concurrency)' in render
def test_shared_utils_defines_set_safe_html_helper() -> None:
"""``setSafeHtml`` in ``shared/utils.js`` is the single audited entry
point for installing trusted HTML strings into a DOM element outside
@@ -1135,46 +1337,15 @@ def test_phase8_consent_url_prefix_check_in_click_handler() -> None:
# cleanly into a standalone node invocation.
def _slice_balanced_body(body: str, anchor: int) -> str | None:
"""Slice ``body`` from ``anchor`` (which must point at or just before
the opening ``{`` of a block) up to and including the matching ``}``.
Tracks brace depth + string state so the slice is robust to comment
growth and arbitrary body reorganisation. Returns ``None`` if the
matching brace isn't found within a reasonable window.
Used to slice JS handler / function bodies for static assertions
without committing to a fixed character window."""
n = len(body)
i = body.find("{", anchor)
if i == -1 or i - anchor > 200:
return None
depth = 0
in_str: str | None = None
start = i
# 12000: connectSSE reached ~7950 chars during the 2026-07 SSE
# recovery campaign (cursor-override + capture-rationale comments);
# the window exists to bound a runaway scan, not to cap legitimate
# method growth — keep it comfortably above the largest real body.
while i < n and i - start < 12000:
ch = body[i]
if in_str:
if ch == "\\" and i + 1 < n:
i += 2
continue
if ch == in_str:
in_str = None
i += 1
continue
if ch in ('"', "'", "`"):
in_str = ch
elif ch == "{":
depth += 1
elif ch == "}":
depth -= 1
if depth == 0:
return body[start : i + 1]
i += 1
return None
# ``_slice_balanced_body`` is the shared comment-and-string-aware brace
# walker from tests/_js_harness_helpers (imported at the top of this
# file). Comment awareness is a strict superset of the old string-only
# local: most callers here pre-strip, where the two agree exactly, and a
# few pass raw source (the persistence-badge and beforeunload pins),
# where the shared walker is the more correct of the two — braces inside
# comments no longer inflate its depth count. One implementation means
# a walker fix lands once for every suite.
_slice_balanced_body = slice_braced_block
def _slice_listener_body(body: str, event_name: str) -> str | None:
@@ -1679,66 +1850,10 @@ def test_dead_sse_defensive_reconnect_registered() -> None:
# the contract so a future refactor can't silently regress it.
def _strip_js_comments(src: str) -> str:
"""Strip ``//`` and ``/* */`` comments while preserving string
literal contents (``"..."``, ``'...'``, `` `...` ``) and keeping
byte length identical (comments replaced with spaces).
Limitation does NOT detect regex literals (``/pattern/flags``).
A ``//`` inside a regex like ``/abc//`` would be misread as the
start of a line comment. Safe today because the regions we scan
(SSE-handler ``onerror`` bodies, ``connectSSE`` /
``connectGlobalSSE`` function bodies) don't contain regex
literals; if a future caller wants to scan a region with regex
literals, extend the tracker first.
Motivation: ``_slice_balanced_body`` doesn't skip comments, so an
apostrophe inside a comment (``can't``, ``don't``) opens a fake
string state that swallows braces until the next ``'``. The new
onerror handlers carry these comments routinely; stripping
comments before brace-walking removes the hazard without
re-architecting the existing slice helper.
"""
out: list[str] = []
n = len(src)
i = 0
in_str: str | None = None
while i < n:
ch = src[i]
if in_str:
out.append(ch)
if ch == "\\" and i + 1 < n:
out.append(src[i + 1])
i += 2
continue
if ch == in_str:
in_str = None
i += 1
continue
# Line comment: replace with spaces up to newline (preserve
# length so downstream offset math still works).
if ch == "/" and i + 1 < n and src[i + 1] == "/":
j = src.find("\n", i)
if j == -1:
j = n
out.append(" " * (j - i))
i = j
continue
# Block comment: replace with spaces up to closing */.
if ch == "/" and i + 1 < n and src[i + 1] == "*":
j = src.find("*/", i + 2)
if j == -1:
out.append(" " * (n - i))
i = n
continue
out.append(" " * (j + 2 - i))
i = j + 2
continue
if ch in ('"', "'", "`"):
in_str = ch
out.append(ch)
i += 1
return "".join(out)
# ``_strip_js_comments`` is the shared string-aware, offset-preserving
# stripper from tests/_js_harness_helpers (imported at the top of this
# file) — one implementation for every suite, so the string-blind /
# offset-destroying per-suite variants cannot diverge again.
def _onerror_block(body: str, anchor_substring: str) -> str | None:
@@ -1843,6 +1958,26 @@ def test_connectglobalsse_onerror_preserves_native_reconnect() -> None:
assert passed, f"connectGlobalSSE.onerror regressed: {reason}"
def test_ws_activity_never_reinserts_a_closed_workstream() -> None:
"""A trailing ``ws_activity`` for a closed workstream must not re-create
a skeletal roster entry: its dashboard row can outlive ``ws_closed``
until the next REST-driven repaint, and a ghost entry both suppresses
the empty-state transition (``showDashboard``) and paints a nameless
rail tab until a full resync. The arm is membership-gated like
``ws_rename`` read the entry, mutate it in place only when it exists,
and never assign into the roster map."""
body = _strip_js_comments(_APP_JS.read_text(encoding="utf-8"))
start = body.index('data.type === "ws_activity"')
end = body.index('data.type === "ws_rename"', start)
arm = body[start:end]
assert "workstreams[data.ws_id] =" not in arm, (
"ws_activity assigns into the roster map — a trailing event for a "
"closed workstream would re-insert a ghost entry"
)
assert "const roster = workstreams[data.ws_id];" in arm
assert "if (roster)" in arm
def test_coord_connectsse_onerror_preserves_native_reconnect() -> None:
"""Coordinator's connectSSE has the same contract — without the
guard the coord's per-ws SSE silently drops events on any blip."""
@@ -2280,7 +2415,11 @@ def test_coord_truncated_resync_is_full_fresh_connect_with_churn_limit() -> None
assert "clearTimeout(truncatedResyncTimer)" in teardown.group(1)
# (6) the dead-stream flow: teardown first, refs reset, seeded refetch,
# guarded .finally reconnect, deferred latch superseded.
flow = re.search(r"function loadHistoryThenReconnect\(\)\s*\{(.*?)\n \}", body, re.S)
flow = re.search(
r"function loadHistoryThenReconnect\(manualAttempt = false\)\s*\{(.*?)\n \}",
body,
re.S,
)
assert flow is not None, "loadHistoryThenReconnect not found"
f = flow.group(1)
assert f.index("suspendStream();") < f.index("refetchHistory(true)")
@@ -2296,8 +2435,13 @@ def test_coord_truncated_resync_is_full_fresh_connect_with_churn_limit() -> None
# this pin only keeps the fix from being "simplified" away.
assert "lastEventId = null;" in f
assert f.index("lastEventId = null;") < f.index("refetchHistory(true)")
assert ".finally(" in f
assert f.index("refetchHistory(true)") < f.index(".finally(")
# The settle rides the outcome-threaded terminal .then (a rendered
# tokenless 200 downgrades to the tokenless bootstrap; failures retry),
# with rejections normalized ahead of it — LOUDLY (round-4 review: a
# bare `.catch(() => undefined)` silently swallowed render throws).
assert 'console.error("history load/render failed"' in f
assert ".then((outcome) => {" in f
assert f.index("refetchHistory(true)") < f.index('console.error("history load/render failed"')
assert "if (visHandler) connectSSE();" in f
# (10) heal-time sidebar refresh: once, on the successful render only
# (record cleared), only on the cursor-SEEDED heal (the cursorless
@@ -2878,8 +3022,9 @@ def test_strict_picker_requires_explicit_pick() -> None:
def _slice_top_level_fn(body: str, header: str) -> str:
"""Slice a top-level ``function`` body from ``header`` to the next
column-0 ``function`` declaration (or EOF). Unlike
``_slice_balanced_body`` this has no fixed-size window, so it is safe
for large functions like ``showNewWsModal``. Nested (indented)
``_slice_balanced_body`` this needs no balanced braces at all, so it
survives a body the walker would refuse (an unterminated block, or a
regex literal the walker misreads as a comment). Nested (indented)
``function () {}`` expressions never match the ``\\nfunction `` bound,
so the slice stops at the next top-level function."""
start = body.index(header)
+39
View File
@@ -68,6 +68,45 @@ def test_discard_is_scope_checked() -> None:
assert buf.get(entry.attachment_id, ws_id="ws1", user_id="u1") is None
def test_consume_all_consumes_present_subset_and_reports_missing() -> None:
"""Survivors are consumed exactly once even when a sibling is missing.
Deliberate pin update: the old all-or-nothing contract left every
surviving reference staged when one handle expired, letting the same
uploads be attached again after the turn that owned their bytes already
committed (the double-spend the atomic transfer exists to prevent).
"""
buf = AttachmentBuffer()
first = _stage(buf, content=b"first")
second = _stage(buf, content=b"second")
_stage(buf, content=b"first", ws="other", user="u1")
missing = hashlib.sha256(b"missing").hexdigest()
consumed = buf.consume_all(
[first.attachment_id, missing, second.attachment_id],
ws_id="ws1",
user_id="u1",
)
assert consumed == {first.attachment_id, second.attachment_id}
assert buf.get(first.attachment_id, ws_id="ws1", user_id="u1") is None
assert buf.get(second.attachment_id, ws_id="ws1", user_id="u1") is None
# Scope isolation: another workstream's staging of the same bytes survives.
assert buf.get(first.attachment_id, ws_id="other", user_id="u1") is not None
# A second consume finds nothing — the ownership reference is one-shot,
# and duplicate handles in one call consume it only once.
assert (
buf.consume_all(
[first.attachment_id, first.attachment_id, second.attachment_id],
ws_id="ws1",
user_id="u1",
)
== frozenset()
)
assert buf.consume_all([], ws_id="ws1", user_id="u1") == frozenset()
def test_ttl_eviction_on_access() -> None:
clock = [0.0]
buf = AttachmentBuffer(ttl_seconds=10.0, clock=lambda: clock[0])
+460 -39
View File
@@ -13,6 +13,9 @@ from unittest.mock import MagicMock
import pytest
from turnstone.core import audio
from turnstone.core.deadline import DeadlineCancelledError, StreamAbortRef
from turnstone.core.model_backend_auth import BackendAuthUnavailableError
from turnstone.core.providers._openai_chat import OpenAIChatCompletionsProvider
class _Cfg:
@@ -46,6 +49,10 @@ class _FakeRegistry:
self._alias = alias
self._cfg = cfg
self._client = client
self.default = alias
self.generation = 0
self.resolve_binding_calls = 0
self._provider = OpenAIChatCompletionsProvider()
def has_alias(self, alias: str) -> bool:
return alias == self._alias
@@ -55,10 +62,21 @@ class _FakeRegistry:
raise ValueError(alias)
return self._cfg
def resolve(self, alias: str | None = None):
def resolve_binding(self, alias: str | None = None):
if alias not in (None, self._alias):
raise ValueError(alias)
return self._client, self._cfg.model, self._cfg, 0
self.resolve_binding_calls += 1
return self._client, self._cfg.model, self._cfg, self._provider, self.generation
def _response_manager(*, parsed=None, body: bytes = b""):
response = MagicMock()
response.parse.return_value = parsed
response.read.return_value = body
manager = MagicMock()
manager.__enter__.return_value = response
manager.__exit__.return_value = False
return manager, response
# ---------------------------------------------------------------------------
@@ -162,7 +180,8 @@ class TestResolveRoleAlias:
class TestTranscribe:
def test_calls_audio_transcriptions_and_returns_text(self):
client = MagicMock()
client.audio.transcriptions.create.return_value = MagicMock(text=" hello world ")
manager, _response = _response_manager(parsed=MagicMock(text=" hello world "))
client.audio.transcriptions.with_streaming_response.create.return_value = manager
reg = _FakeRegistry("voice", _Cfg("gpt-4o-mini-transcribe"), client)
res = audio.transcribe(
registry=reg, alias="voice", data=b"RIFFfake", filename="speech.webm"
@@ -170,29 +189,35 @@ class TestTranscribe:
assert res.transcript == "hello world"
assert res.model_alias == "voice"
assert res.model == "gpt-4o-mini-transcribe"
kwargs = client.audio.transcriptions.create.call_args.kwargs
kwargs = client.audio.transcriptions.with_streaming_response.create.call_args.kwargs
assert kwargs["model"] == "gpt-4o-mini-transcribe"
assert kwargs["file"] == ("speech.webm", b"RIFFfake")
def test_prompt_forwarded_when_set(self):
client = MagicMock()
client.audio.transcriptions.create.return_value = MagicMock(text="ok")
manager, _response = _response_manager(parsed=MagicMock(text="ok"))
client.audio.transcriptions.with_streaming_response.create.return_value = manager
reg = _FakeRegistry("voice", _Cfg("whisper-1"), client)
audio.transcribe(
registry=reg, alias="voice", data=b"x", filename="a.wav", prompt="ACME jargon"
)
assert client.audio.transcriptions.create.call_args.kwargs["prompt"] == "ACME jargon"
kwargs = client.audio.transcriptions.with_streaming_response.create.call_args.kwargs
assert kwargs["prompt"] == "ACME jargon"
def test_prompt_omitted_when_blank(self):
client = MagicMock()
client.audio.transcriptions.create.return_value = MagicMock(text="ok")
manager, _response = _response_manager(parsed=MagicMock(text="ok"))
client.audio.transcriptions.with_streaming_response.create.return_value = manager
reg = _FakeRegistry("voice", _Cfg("whisper-1"), client)
audio.transcribe(registry=reg, alias="voice", data=b"x", filename="a.wav")
assert "prompt" not in client.audio.transcriptions.create.call_args.kwargs
kwargs = client.audio.transcriptions.with_streaming_response.create.call_args.kwargs
assert "prompt" not in kwargs
def test_backend_failure_raises_backend_error(self):
client = MagicMock()
client.audio.transcriptions.create.side_effect = RuntimeError("boom")
client.audio.transcriptions.with_streaming_response.create.side_effect = RuntimeError(
"boom"
)
reg = _FakeRegistry("voice", _Cfg("whisper-1"), client)
with pytest.raises(audio.AudioBackendError):
audio.transcribe(registry=reg, alias="voice", data=b"x", filename="a.wav")
@@ -201,15 +226,17 @@ class TestTranscribe:
monkeypatch.setattr(audio, "_to_wav_16k_mono", lambda data: data)
client = MagicMock()
msg = MagicMock(content=" the transcript ")
client.chat.completions.create.return_value = MagicMock(choices=[MagicMock(message=msg)])
manager, _response = _response_manager(parsed=MagicMock(choices=[MagicMock(message=msg)]))
client.chat.completions.with_streaming_response.create.return_value = manager
reg = _FakeRegistry("omni", _Cfg("gemma-omni", {"supports_audio_input": True}), client)
res = audio.transcribe(
registry=reg, alias="omni", data=b"webmbytes", filename="speech.webm"
)
assert res.transcript == "the transcript"
# The dedicated transcription endpoint is NOT used for an omni model.
client.audio.transcriptions.create.assert_not_called()
parts = client.chat.completions.create.call_args.kwargs["messages"][0]["content"]
client.audio.transcriptions.with_streaming_response.create.assert_not_called()
kwargs = client.chat.completions.with_streaming_response.create.call_args.kwargs
parts = kwargs["messages"][0]["content"]
# Prompt precedes the audio part — the order Gemma documents for transcription.
assert [p["type"] for p in parts] == ["text", "input_audio"]
# The clip is transcoded to wav regardless of the upload container.
@@ -222,14 +249,16 @@ class TestTranscribe:
def test_omni_prompt_override_used(self, monkeypatch):
monkeypatch.setattr(audio, "_to_wav_16k_mono", lambda data: data)
client = MagicMock()
client.chat.completions.create.return_value = MagicMock(
choices=[MagicMock(message=MagicMock(content="x"))]
manager, _response = _response_manager(
parsed=MagicMock(choices=[MagicMock(message=MagicMock(content="x"))])
)
client.chat.completions.with_streaming_response.create.return_value = manager
reg = _FakeRegistry("omni", _Cfg("gemma-omni", {"supports_audio_input": True}), client)
audio.transcribe(
registry=reg, alias="omni", data=b"x", filename="a.wav", prompt="custom instruction"
)
parts = client.chat.completions.create.call_args.kwargs["messages"][0]["content"]
kwargs = client.chat.completions.with_streaming_response.create.call_args.kwargs
parts = kwargs["messages"][0]["content"]
text_part = next(p for p in parts if p["type"] == "text")
assert text_part["text"] == "custom instruction"
@@ -245,38 +274,211 @@ class TestTranscribe:
)
with pytest.raises(audio.AudioUnavailableError, match="OpenAI-compatible provider"):
audio.transcribe(registry=reg, alias="omni", data=b"x", filename="a.webm")
client.chat.completions.create.assert_not_called()
client.chat.completions.with_streaming_response.create.assert_not_called()
def test_dedicated_endpoint_uses_authenticated_clone_and_pinned_config(self):
base_client = MagicMock()
call_client = MagicMock()
base_client.with_options.return_value = call_client
manager, _response = _response_manager(parsed=MagicMock(text="hello"))
call_client.audio.transcriptions.with_streaming_response.create.return_value = manager
cfg = _Cfg("whisper-1")
resolver = MagicMock(return_value="minted-token")
result = audio.transcribe(
registry=_FakeRegistry("voice", cfg, base_client),
alias="voice",
data=b"x",
filename="a.wav",
backend_auth_resolver=resolver,
)
assert result.transcript == "hello"
resolver.assert_called_once_with("voice", cfg)
base_client.with_options.assert_called_once_with(api_key="minted-token")
base_client.audio.transcriptions.with_streaming_response.create.assert_not_called()
base_client.close.assert_not_called()
call_client.close.assert_not_called()
def test_omni_endpoint_uses_authenticated_clone_and_pinned_config(self, monkeypatch):
monkeypatch.setattr(audio, "_to_wav_16k_mono", lambda data: data)
base_client = MagicMock()
call_client = MagicMock()
base_client.with_options.return_value = call_client
manager, _response = _response_manager(
parsed=MagicMock(choices=[MagicMock(message=MagicMock(content="hello from omni"))])
)
call_client.chat.completions.with_streaming_response.create.return_value = manager
cfg = _Cfg("omni", {"supports_audio_input": True})
resolver = MagicMock(return_value="minted-token")
result = audio.transcribe(
registry=_FakeRegistry("voice", cfg, base_client),
alias="voice",
data=b"x",
filename="a.webm",
backend_auth_resolver=resolver,
)
assert result.transcript == "hello from omni"
resolver.assert_called_once_with("voice", cfg)
base_client.with_options.assert_called_once_with(api_key="minted-token")
base_client.chat.completions.with_streaming_response.create.assert_not_called()
base_client.close.assert_not_called()
call_client.close.assert_not_called()
def test_abort_during_response_parse_closes_handle_and_propagates(self):
client = MagicMock()
ref = StreamAbortRef()
manager, response = _response_manager()
def _abort_while_parsing():
ref.abort()
return MagicMock(text="too late")
response.parse.side_effect = _abort_while_parsing
client.audio.transcriptions.with_streaming_response.create.return_value = manager
with pytest.raises(DeadlineCancelledError):
audio.transcribe(
registry=_FakeRegistry("voice", _Cfg("whisper-1"), client),
alias="voice",
data=b"x",
filename="a.wav",
cancel_ref=ref,
)
response.close.assert_called()
def test_abort_after_transcription_manager_creation_prevents_dispatch(self):
client = MagicMock()
ref = StreamAbortRef()
manager, response = _response_manager(parsed=MagicMock(text="too late"))
def _create_manager(**_kwargs):
ref.abort()
return manager
client.audio.transcriptions.with_streaming_response.create.side_effect = _create_manager
with pytest.raises(DeadlineCancelledError):
audio.transcribe(
registry=_FakeRegistry("voice", _Cfg("whisper-1"), client),
alias="voice",
data=b"x",
filename="a.wav",
cancel_ref=ref,
)
manager.__enter__.assert_not_called()
response.parse.assert_not_called()
def test_abort_after_omni_manager_creation_prevents_dispatch(self, monkeypatch):
monkeypatch.setattr(audio, "_to_wav_16k_mono", lambda data: data)
client = MagicMock()
ref = StreamAbortRef()
manager, response = _response_manager(
parsed=MagicMock(choices=[MagicMock(message=MagicMock(content="too late"))])
)
def _create_manager(**_kwargs):
ref.abort()
return manager
client.chat.completions.with_streaming_response.create.side_effect = _create_manager
with pytest.raises(DeadlineCancelledError):
audio.transcribe(
registry=_FakeRegistry(
"omni", _Cfg("gemma-omni", {"supports_audio_input": True}), client
),
alias="omni",
data=b"x",
filename="a.webm",
cancel_ref=ref,
)
manager.__enter__.assert_not_called()
response.parse.assert_not_called()
class TestSynthesize:
def test_calls_audio_speech_and_returns_bytes(self):
client = MagicMock()
speech = MagicMock()
speech.read.return_value = b"RIFF...wavbytes"
client.audio.speech.create.return_value = speech
manager, _response = _response_manager(body=b"RIFF...wavbytes")
client.audio.speech.with_streaming_response.create.return_value = manager
reg = _FakeRegistry("voice", _Cfg("gpt-4o-mini-tts"), client)
res = audio.synthesize(registry=reg, alias="voice", text="hi", voice="nova")
assert res.audio_bytes == b"RIFF...wavbytes"
assert res.media_type == "audio/mpeg"
assert res.model_alias == "voice"
kwargs = client.audio.speech.create.call_args.kwargs
kwargs = client.audio.speech.with_streaming_response.create.call_args.kwargs
assert kwargs["voice"] == "nova"
assert kwargs["input"] == "hi"
def test_default_voice_when_empty(self):
client = MagicMock()
client.audio.speech.create.return_value = MagicMock(read=lambda: b"a")
manager, _response = _response_manager(body=b"a")
client.audio.speech.with_streaming_response.create.return_value = manager
reg = _FakeRegistry("voice", _Cfg("gpt-4o-mini-tts"), client)
audio.synthesize(registry=reg, alias="voice", text="hi", voice="")
assert client.audio.speech.create.call_args.kwargs["voice"] == "alloy"
kwargs = client.audio.speech.with_streaming_response.create.call_args.kwargs
assert kwargs["voice"] == "alloy"
def test_backend_failure_raises_backend_error(self):
client = MagicMock()
client.audio.speech.create.side_effect = RuntimeError("down")
client.audio.speech.with_streaming_response.create.side_effect = RuntimeError("down")
reg = _FakeRegistry("voice", _Cfg("gpt-4o-mini-tts"), client)
with pytest.raises(audio.AudioBackendError):
audio.synthesize(registry=reg, alias="voice", text="hi", voice="nova")
def test_uses_authenticated_clone_and_pinned_config(self):
base_client = MagicMock()
call_client = MagicMock()
base_client.with_options.return_value = call_client
manager, _response = _response_manager(body=b"voice")
call_client.audio.speech.with_streaming_response.create.return_value = manager
cfg = _Cfg("gpt-4o-mini-tts")
resolver = MagicMock(return_value="minted-token")
result = audio.synthesize(
registry=_FakeRegistry("voice", cfg, base_client),
alias="voice",
text="hello",
voice="alloy",
backend_auth_resolver=resolver,
)
assert result.audio_bytes == b"voice"
resolver.assert_called_once_with("voice", cfg)
base_client.with_options.assert_called_once_with(api_key="minted-token")
base_client.audio.speech.with_streaming_response.create.assert_not_called()
base_client.close.assert_not_called()
call_client.close.assert_not_called()
def test_abort_after_speech_manager_creation_prevents_dispatch(self):
client = MagicMock()
ref = StreamAbortRef()
manager, response = _response_manager(body=b"too late")
def _create_manager(**_kwargs):
ref.abort()
return manager
client.audio.speech.with_streaming_response.create.side_effect = _create_manager
with pytest.raises(DeadlineCancelledError):
audio.synthesize(
registry=_FakeRegistry("voice", _Cfg("gpt-4o-mini-tts"), client),
alias="voice",
text="hello",
voice="alloy",
cancel_ref=ref,
)
manager.__enter__.assert_not_called()
response.read.assert_not_called()
class TestOpenAIAudioModelsKnown:
"""The current OpenAI STT/TTS lineup is registered in the static capability
@@ -314,9 +516,7 @@ class TestOpenAIAudioModelsKnown:
class TestTranscribeCached:
"""The memoized, non-raising transcribe used by the no-native-audio wire
fallback. Caching an STT result is an audio-domain concern, so it lives here
next to ``transcribe`` rather than bundled with PDF text extraction."""
"""Memoized STT for the no-native-audio wire fallback."""
def _result(self, text: str):
return audio.TranscriptionResult(transcript=text, model_alias="w", model="m")
@@ -324,13 +524,14 @@ class TestTranscribeCached:
def test_memoizes_by_alias_and_hash(self, monkeypatch):
audio._clear_transcript_cache_for_test()
calls = []
reg = _FakeRegistry("w", _Cfg("whisper-1"), MagicMock())
def fake(*, registry, alias, data, filename):
def fake(binding, **kwargs):
calls.append(1)
return self._result("hello world")
monkeypatch.setattr(audio, "transcribe", fake)
kw = dict(registry=object(), alias="w", content_hash="h1", data=b"x", filename="a.wav")
monkeypatch.setattr(audio, "_transcribe_binding", fake)
kw = dict(registry=reg, alias="w", content_hash="h1", data=b"x", filename="a.wav")
assert audio.transcribe_cached(**kw) == "hello world"
assert audio.transcribe_cached(**kw) == "hello world"
assert len(calls) == 1 # second served from cache
@@ -338,17 +539,182 @@ class TestTranscribeCached:
def test_backend_failure_returns_empty_and_is_not_cached(self, monkeypatch):
audio._clear_transcript_cache_for_test()
calls = []
reg = _FakeRegistry("w", _Cfg("whisper-1"), MagicMock())
def boom(*, registry, alias, data, filename):
def boom(binding, **kwargs):
calls.append(1)
raise audio.AudioBackendError("down")
monkeypatch.setattr(audio, "transcribe", boom)
kw = dict(registry=object(), alias="w", content_hash="h2", data=b"x", filename="a.wav")
monkeypatch.setattr(audio, "_transcribe_binding", boom)
kw = dict(registry=reg, alias="w", content_hash="h2", data=b"x", filename="a.wav")
assert audio.transcribe_cached(**kw) == ""
audio.transcribe_cached(**kw)
assert len(calls) == 2 # failure not cached -> retried
@pytest.mark.parametrize("failure_seam", ["resolve", "transcribe"])
def test_abort_during_backend_failure_propagates_cancellation(
self,
monkeypatch,
failure_seam,
):
audio._clear_transcript_cache_for_test()
ref = StreamAbortRef()
reg = _FakeRegistry("w", _Cfg("whisper-1"), MagicMock())
if failure_seam == "resolve":
def fail_resolve(**_kwargs):
ref.abort()
raise audio.AudioUnavailableError("gone")
monkeypatch.setattr(audio, "_resolve_audio_binding", fail_resolve)
else:
def fail_transcribe(_binding, **_kwargs):
ref.abort()
raise audio.AudioBackendError("down")
monkeypatch.setattr(audio, "_transcribe_binding", fail_transcribe)
with pytest.raises(DeadlineCancelledError):
audio.transcribe_cached(
registry=reg,
alias="w",
content_hash="cancelled-failure",
data=b"x",
filename="a.wav",
cancel_ref=ref,
)
assert audio._transcript_cache == {}
def test_disappeared_alias_returns_empty_before_backend_dispatch(self, monkeypatch):
audio._clear_transcript_cache_for_test()
transcribe = MagicMock()
monkeypatch.setattr(audio, "_transcribe_binding", transcribe)
reg = _FakeRegistry("live", _Cfg("whisper-1"), MagicMock())
result = audio.transcribe_cached(
registry=reg,
alias="removed",
content_hash="gone",
data=b"x",
filename="a.wav",
)
assert result == ""
transcribe.assert_not_called()
assert audio._transcript_cache == {}
def test_pre_aborted_unknown_alias_propagates_cancellation(self):
audio._clear_transcript_cache_for_test()
ref = StreamAbortRef()
ref.abort()
with pytest.raises(DeadlineCancelledError):
audio.transcribe_cached(
registry=_FakeRegistry("live", _Cfg("whisper-1"), MagicMock()),
alias="removed",
content_hash="gone",
data=b"x",
filename="a.wav",
cancel_ref=ref,
)
assert audio._transcript_cache == {}
def test_cache_isolated_by_principal_and_registry_generation(self, monkeypatch):
audio._clear_transcript_cache_for_test()
calls = []
reg = _FakeRegistry("w", _Cfg("whisper-1"), MagicMock())
def fake(binding, **kwargs):
calls.append((binding.registry_generation, kwargs["data"]))
return self._result(f"result-{len(calls)}")
monkeypatch.setattr(audio, "_transcribe_binding", fake)
common = dict(
registry=reg,
alias="w",
content_hash="same",
data=b"x",
filename="a.wav",
)
assert audio.transcribe_cached(**common, principal_id="user-a") == "result-1"
assert audio.transcribe_cached(**common, principal_id="user-b") == "result-2"
assert audio.transcribe_cached(**common, principal_id="user-a") == "result-1"
reg.generation = 1
assert audio.transcribe_cached(**common, principal_id="user-a") == "result-3"
assert calls == [(0, b"x"), (0, b"x"), (1, b"x")]
def test_racing_empty_result_never_clobbers_real_transcript(self, monkeypatch):
audio._clear_transcript_cache_for_test()
reg = _FakeRegistry("w", _Cfg("whisper-1"), MagicMock())
def racing_empty(binding, **kwargs):
key = (
"user-a",
binding.lane.alias,
binding.registry_generation,
"race",
)
with audio._transcript_lock:
audio._transcript_cache[key] = "real from racer"
return self._result("")
monkeypatch.setattr(audio, "_transcribe_binding", racing_empty)
result = audio.transcribe_cached(
registry=reg,
alias="w",
content_hash="race",
data=b"x",
filename="a.wav",
principal_id="user-a",
)
assert result == "real from racer"
assert audio._transcript_cache[("user-a", "w", 0, "race")] == "real from racer"
def test_pre_aborted_cache_hit_propagates_cancellation(self, monkeypatch):
audio._clear_transcript_cache_for_test()
reg = _FakeRegistry("w", _Cfg("whisper-1"), MagicMock())
monkeypatch.setattr(
audio,
"_transcribe_binding",
lambda binding, **kwargs: self._result("cached"),
)
common = dict(
registry=reg,
alias="w",
content_hash="same",
data=b"x",
filename="a.wav",
principal_id="user-a",
)
assert audio.transcribe_cached(**common) == "cached"
ref = StreamAbortRef()
ref.abort()
with pytest.raises(DeadlineCancelledError):
audio.transcribe_cached(**common, cancel_ref=ref)
def test_backend_auth_refusal_is_not_swallowed(self):
audio._clear_transcript_cache_for_test()
def refuse(alias, cfg):
raise BackendAuthUnavailableError("unavailable")
with pytest.raises(BackendAuthUnavailableError):
audio.transcribe_cached(
registry=_FakeRegistry("w", _Cfg("whisper-1"), MagicMock()),
alias="w",
content_hash="h",
data=b"x",
filename="a.wav",
principal_id="user-a",
backend_auth_resolver=refuse,
)
# ---------------------------------------------------------------------------
# Omni chat request shaping — transcode + thinking-off + token cap
@@ -396,9 +762,10 @@ class TestOmniChatCall:
def test_sends_thinking_off_and_token_cap(self, monkeypatch):
monkeypatch.setattr(audio, "_to_wav_16k_mono", lambda data: data)
client = MagicMock()
client.chat.completions.create.return_value = MagicMock(
choices=[MagicMock(message=MagicMock(content="hi"))]
manager, _response = _response_manager(
parsed=MagicMock(choices=[MagicMock(message=MagicMock(content="hi"))])
)
client.chat.completions.with_streaming_response.create.return_value = manager
cfg = _Cfg(
"gemma-omni",
{
@@ -413,7 +780,7 @@ class TestOmniChatCall:
data=b"webmbytes",
filename="speech.webm",
)
kwargs = client.chat.completions.create.call_args.kwargs
kwargs = client.chat.completions.with_streaming_response.create.call_args.kwargs
assert kwargs["extra_body"]["chat_template_kwargs"]["enable_thinking"] is False
assert kwargs["max_tokens"] == audio._OMNI_STT_MAX_TOKENS
@@ -528,10 +895,64 @@ class TestTranscribeStream:
def test_whisper_alias_emits_single_chunk(self):
client = MagicMock()
client.audio.transcriptions.create.return_value = MagicMock(text=" full transcript ")
manager, _response = _response_manager(parsed=MagicMock(text=" full transcript "))
client.audio.transcriptions.with_streaming_response.create.return_value = manager
cfg = _Cfg("whisper-1") # name inference -> dedicated endpoint, no chat stream
gen = audio.transcribe_stream(
registry=_FakeRegistry("w", cfg, client), alias="w", data=b"x"
)
registry = _FakeRegistry("w", cfg, client)
gen = audio.transcribe_stream(registry=registry, alias="w", data=b"x")
assert list(gen) == ["full transcript"]
assert registry.resolve_binding_calls == 1
client.chat.completions.create.assert_not_called()
def test_omni_stream_uses_authenticated_clone_and_abort_closes_handle(self, monkeypatch):
monkeypatch.setattr(audio, "_to_wav_16k_mono", lambda data: data)
base_client = MagicMock()
call_client = MagicMock()
base_client.with_options.return_value = call_client
stream = MagicMock()
stream.__iter__.return_value = iter([_stream_chunk("hello")])
call_client.chat.completions.create.return_value = stream
cfg = _Cfg("omni", {"supports_audio_input": True})
resolver = MagicMock(return_value="minted-token")
ref = StreamAbortRef()
deltas = audio.transcribe_stream(
registry=_FakeRegistry("omni", cfg, base_client),
alias="omni",
data=b"x",
backend_auth_resolver=resolver,
cancel_ref=ref,
)
resolver.assert_called_once_with("omni", cfg)
base_client.with_options.assert_called_once_with(api_key="minted-token")
base_client.chat.completions.create.assert_not_called()
ref.abort()
with pytest.raises(DeadlineCancelledError):
list(deltas)
stream.close.assert_called()
base_client.close.assert_not_called()
call_client.close.assert_not_called()
def test_abort_during_final_omni_request_shaping_prevents_dispatch(self, monkeypatch):
monkeypatch.setattr(audio, "_to_wav_16k_mono", lambda data: data)
client = MagicMock()
ref = StreamAbortRef()
def _abort_in_final_shaping(_cfg):
ref.abort()
return {}
monkeypatch.setattr(audio, "_omni_chat_extra_body", _abort_in_final_shaping)
with pytest.raises(DeadlineCancelledError):
audio.transcribe_stream(
registry=_FakeRegistry(
"omni", _Cfg("gemma-omni", {"supports_audio_input": True}), client
),
alias="omni",
data=b"x",
cancel_ref=ref,
)
client.chat.completions.create.assert_not_called()
+178
View File
@@ -11,6 +11,8 @@ from turnstone.core.auth import (
AUTH_COOKIE,
AUTH_COOKIE_CONSOLE,
AUTH_COOKIE_SERVER,
JWT_AUD_CONSOLE,
TLS_ACME_TOKEN_SOURCE,
WRITE_PATHS,
_extract_bearer,
_extract_cookie,
@@ -53,6 +55,17 @@ class TestIsPublicPath:
def test_shared_js_public(self):
assert is_public_path("/shared/utils.js") is True
@pytest.mark.parametrize("path", ["/acme/directory", "/acme/new-nonce", "/acme/ca.pem"])
def test_acme_bootstrap_resources_public(self, path):
assert is_public_path(path) is True
@pytest.mark.parametrize(
"path",
["/acme/new-account", "/acme/new-order", "/acme/authz/order-1", "/acme/cert/1"],
)
def test_acme_signing_resources_not_public(self, path):
assert is_public_path(path) is False
def test_api_workstreams_not_public(self):
assert is_public_path("/api/workstreams") is False
@@ -129,6 +142,12 @@ class TestRequiredScope:
def test_get_api_needs_read(self):
assert required_scope("GET", "/api/workstreams") == "read"
@pytest.mark.parametrize(
"path", ["/acme/new-account", "/acme/new-order", "/acme/finalize/order-1"]
)
def test_acme_signing_requires_service(self, path):
assert required_scope("POST", path) == "service"
def test_get_events_needs_read(self):
assert required_scope("GET", "/api/events") == "read"
@@ -479,6 +498,146 @@ class TestCheckRequest:
)
assert allowed is True
def test_acme_signing_no_token_401(self):
allowed, status, _msg, _result = check_request(
"POST", "/acme/new-order", None, cookie_name=AUTH_COOKIE_CONSOLE
)
assert allowed is False
assert status == 401
def test_acme_rejects_generic_service_token(self):
token = create_jwt(
"node-1",
frozenset({"service"}),
"console",
self._SECRET,
audience=JWT_AUD_CONSOLE,
)
allowed, status, msg, _result = check_request(
"POST",
"/acme/new-order",
f"Bearer {token}",
jwt_secret=self._SECRET,
jwt_audience=JWT_AUD_CONSOLE,
cookie_name=AUTH_COOKIE_CONSOLE,
)
assert allowed is False
assert status == 403
assert "ACME enrollment" in msg
def test_acme_rejects_enrollment_token_without_service_scope(self):
token = create_jwt(
"node-1",
frozenset({"read"}),
TLS_ACME_TOKEN_SOURCE,
self._SECRET,
audience=JWT_AUD_CONSOLE,
)
allowed, status, _msg, _result = check_request(
"POST",
"/acme/new-order",
f"Bearer {token}",
jwt_secret=self._SECRET,
jwt_audience=JWT_AUD_CONSOLE,
cookie_name=AUTH_COOKIE_CONSOLE,
)
assert allowed is False
assert status == 403
def test_acme_accepts_purpose_specific_service_token(self):
token = create_jwt(
"node-1",
frozenset({"service"}),
TLS_ACME_TOKEN_SOURCE,
self._SECRET,
audience=JWT_AUD_CONSOLE,
)
allowed, status, _msg, result = check_request(
"POST",
"/acme/new-order",
f"Bearer {token}",
jwt_secret=self._SECRET,
jwt_audience=JWT_AUD_CONSOLE,
cookie_name=AUTH_COOKIE_CONSOLE,
)
assert allowed is True
assert status == 200
assert result is not None and result.token_source == TLS_ACME_TOKEN_SOURCE
@pytest.mark.parametrize(
("method", "path"),
[
("GET", "/v1/api/workstreams"),
("POST", "/v1/api/workstreams/new"),
("GET", "/v1/api/admin/users"),
("POST", "/v1/api/admin/tls/certs/node-1/renew"),
("GET", "/v1/node/node-1/api/workstreams"),
],
)
def test_enrollment_token_rejected_outside_acme(self, method, path):
token = create_jwt(
"node-1",
frozenset({"service"}),
TLS_ACME_TOKEN_SOURCE,
self._SECRET,
audience=JWT_AUD_CONSOLE,
)
allowed, status, msg, result = check_request(
method,
path,
f"Bearer {token}",
jwt_secret=self._SECRET,
jwt_audience=JWT_AUD_CONSOLE,
cookie_name=AUTH_COOKIE_CONSOLE,
)
assert allowed is False
assert status == 403
assert "enrollment token" in msg
assert result is None
def test_versioned_acme_accepts_purpose_specific_service_token(self):
token = create_jwt(
"node-1",
frozenset({"service"}),
TLS_ACME_TOKEN_SOURCE,
self._SECRET,
audience=JWT_AUD_CONSOLE,
)
allowed, status, _msg, result = check_request(
"POST",
"/v1/acme/new-order",
f"Bearer {token}",
jwt_secret=self._SECRET,
jwt_audience=JWT_AUD_CONSOLE,
cookie_name=AUTH_COOKIE_CONSOLE,
)
assert allowed is True
assert status == 200
assert result is not None and result.token_source == TLS_ACME_TOKEN_SOURCE
def test_acme_rejects_wrong_audience(self):
token = create_jwt(
"node-1",
frozenset({"service"}),
TLS_ACME_TOKEN_SOURCE,
self._SECRET,
audience="another-service",
)
allowed, status, _msg, _result = check_request(
"POST",
"/acme/new-order",
f"Bearer {token}",
jwt_secret=self._SECRET,
jwt_audience=JWT_AUD_CONSOLE,
cookie_name=AUTH_COOKIE_CONSOLE,
)
assert allowed is False
assert status == 401
def test_api_no_token_401(self):
allowed, status, msg, _result = check_request(
"GET", "/api/workstreams", None, cookie_name=AUTH_COOKIE_SERVER
@@ -1406,6 +1565,25 @@ class TestConsoleLogin:
)
assert resp.status_code == 401
def test_login_enrollment_token_rejected(self):
token = create_jwt(
"node-1",
frozenset({"service"}),
TLS_ACME_TOKEN_SOURCE,
self._jwt_secret,
audience=JWT_AUD_CONSOLE,
)
resp = self.test_client.post(
"/v1/api/auth/login",
json={"token": token},
)
assert resp.status_code == 401
assert resp.json() == {"error": "Invalid credentials"}
assert "jwt" not in resp.json()
assert AUTH_COOKIE_CONSOLE not in resp.headers.get("set-cookie", "")
def test_login_password_ok(self):
resp = self.test_client.post(
"/v1/api/auth/login",
+14 -2
View File
@@ -21,11 +21,19 @@ import pytest
from tests._proc_helpers import pid_alive as _pid_alive
from tests._proc_helpers import poll_until as _wait_until
from tests._session_helpers import make_session
from turnstone.core.session import _active_shell_owner
from turnstone.core.storage import get_storage
@pytest.fixture
def session():
def session(tmp_db):
s = make_session()
get_storage().register_workstream(
s.ws_id,
user_id=s._user_id,
kind=s._kind,
parent_ws_id=s._parent_ws_id,
)
yield s
s.close()
@@ -751,9 +759,11 @@ def test_task_agent_shells_are_owner_scoped_and_reaped(session, monkeypatch):
seen = {}
def fake_run_agent(agent_turns, label="task", **kwargs):
owner = _active_shell_owner.get()
seen["owner"] = owner
out = _start_background(session, "sleep 60", call_id="sub-bash")
seen["start_output"] = out
agent_shells = session._background_shells.shells(owner="task-1")
agent_shells = session._background_shells.shells(owner=owner)
seen["agent_shells"] = list(agent_shells)
seen["pid"] = agent_shells[0].pid if agent_shells else None
# The sub-agent's shell is invisible to the main scope.
@@ -763,6 +773,8 @@ def test_task_agent_shells_are_owner_scoped_and_reaped(session, monkeypatch):
monkeypatch.setattr(session, "_run_agent", fake_run_agent)
call_id, result = session._exec_task({"call_id": "task-1", "prompt": "start a server"})
assert "agent done" in result
assert seen["owner"].startswith("task_agent:task-1:")
assert seen["owner"] != "task-1"
assert seen["agent_shells"], "shell spawned inside the agent must carry its owner"
# Scope honesty in the start message: the sub-agent must not promise its
# caller a server that dies the moment it returns.
+3653 -22
View File
File diff suppressed because it is too large Load Diff
+266 -4
View File
@@ -6,6 +6,7 @@ from unittest.mock import AsyncMock, MagicMock
import pytest
from turnstone.api.console_schemas import RouteCreateResponse
from turnstone.channels._routing import ChannelRouter
from turnstone.sdk._types import TurnstoneAPIError
@@ -17,6 +18,7 @@ def mock_storage() -> MagicMock:
storage.get_channel_user = MagicMock(return_value=None)
storage.get_channel_route = MagicMock(return_value=None)
storage.get_channel_route_by_ws = MagicMock(return_value=None)
storage.resolve_workstream = MagicMock(side_effect=lambda ws_id: ws_id)
storage.create_channel_route = MagicMock()
storage.delete_channel_route = MagicMock(return_value=True)
return storage
@@ -124,6 +126,45 @@ class TestDeleteRoute:
mock_storage.delete_channel_route.assert_called_once_with("discord", "ch-123")
class TestWorkstreamLiveness:
@pytest.mark.anyio
async def test_direct_mode_uses_manager_authoritative_active_list(
self,
router: ChannelRouter,
monkeypatch: pytest.MonkeyPatch,
) -> None:
assert router._server is not None
mock_list = AsyncMock(
return_value=MagicMock(
workstreams=[
MagicMock(ws_id="other", state="idle"),
MagicMock(ws_id="ws-live", state="running"),
MagicMock(ws_id="ws-creating", state="creating"),
]
)
)
monkeypatch.setattr(router._server, "list_workstreams", mock_list)
assert await router._is_ws_live("ws-live") is True
assert await router._is_ws_live("ws-cold") is False
assert await router._is_ws_live("ws-creating") is False
assert mock_list.await_count == 3
@pytest.mark.anyio
async def test_console_mode_uses_routed_live_probe(
self,
console_router: ChannelRouter,
monkeypatch: pytest.MonkeyPatch,
) -> None:
assert console_router._console is not None
mock_live = AsyncMock(side_effect=[MagicMock(live=True), MagicMock(live=False)])
monkeypatch.setattr(console_router._console, "route_workstream_live", mock_live)
assert await console_router._is_ws_live("ws-live") is True
assert await console_router._is_ws_live("ws-cold") is False
assert [item.args[0] for item in mock_live.await_args_list] == ["ws-live", "ws-cold"]
class TestGetOrCreateWorkstream:
@pytest.mark.anyio
async def test_creates_new_workstream_via_server(
@@ -151,7 +192,13 @@ class TestGetOrCreateWorkstream:
) -> None:
assert console_router._console is not None
mock_create = AsyncMock(
return_value={"ws_id": "ws-new", "name": "test", "node_url": "http://node1:8080/v1"}
return_value=RouteCreateResponse(
ws_id="ws-new",
name="test",
node_url="http://node1:8080/v1",
node_id="node-1",
routing_strategy="rendezvous",
)
)
monkeypatch.setattr(console_router._console, "route_create_workstream", mock_create)
ws_id, is_new = await console_router.get_or_create_workstream(
@@ -175,7 +222,7 @@ class TestGetOrCreateWorkstream:
"channel_type": "discord",
"channel_id": "ch-1",
}
monkeypatch.setattr(router, "_is_ws_alive", AsyncMock(return_value=True))
monkeypatch.setattr(router, "_is_ws_live", AsyncMock(return_value=True))
ws_id, is_new = await router.get_or_create_workstream("discord", "ch-1")
assert ws_id == "ws-old"
assert is_new is False
@@ -192,8 +239,8 @@ class TestGetOrCreateWorkstream:
"channel_type": "discord",
"channel_id": "ch-1",
}
# Alive check returns False — ws is not alive.
monkeypatch.setattr(router, "_is_ws_alive", AsyncMock(return_value=False))
# The durable source exists but is no longer loaded on the node.
monkeypatch.setattr(router, "_is_ws_live", AsyncMock(return_value=False))
# Server create returns a resumed workstream.
assert router._server is not None
mock_create = AsyncMock()
@@ -211,6 +258,221 @@ class TestGetOrCreateWorkstream:
call_kwargs = mock_create.call_args[1]
assert call_kwargs["resume_ws"] == "ws-stale"
@pytest.mark.anyio
async def test_missing_stale_source_retries_fresh_via_server(
self,
router: ChannelRouter,
mock_storage: MagicMock,
monkeypatch: pytest.MonkeyPatch,
) -> None:
mock_storage.get_channel_route.return_value = {
"ws_id": "ws-pruned",
"channel_type": "discord",
"channel_id": "ch-1",
}
mock_storage.resolve_workstream.side_effect = None
mock_storage.resolve_workstream.return_value = None
assert router._server is not None
mock_create = AsyncMock(
side_effect=[
TurnstoneAPIError(404, "Workstream not found"),
MagicMock(ws_id="ws-fresh", name="test"),
]
)
mock_send = AsyncMock()
monkeypatch.setattr(router._server, "create_workstream", mock_create)
monkeypatch.setattr(router._server, "send", mock_send)
ws_id, is_new = await router.get_or_create_workstream(
"discord",
"ch-1",
name="test",
initial_message="hello",
)
assert (ws_id, is_new) == ("ws-fresh", True)
assert [item.kwargs["resume_ws"] for item in mock_create.await_args_list] == [
"ws-pruned",
"",
]
mock_send.assert_awaited_once_with("hello", "ws-fresh")
mock_storage.create_channel_route.assert_called_once_with("discord", "ch-1", "ws-fresh")
@pytest.mark.anyio
async def test_missing_stale_source_retries_fresh_via_console(
self,
console_router: ChannelRouter,
mock_storage: MagicMock,
monkeypatch: pytest.MonkeyPatch,
) -> None:
mock_storage.get_channel_route.return_value = {
"ws_id": "ws-pruned",
"channel_type": "slack",
"channel_id": "ch-1",
}
mock_storage.resolve_workstream.side_effect = None
mock_storage.resolve_workstream.return_value = None
assert console_router._console is not None
mock_create = AsyncMock(
side_effect=[
TurnstoneAPIError(404, "Workstream not found"),
RouteCreateResponse(
ws_id="ws-fresh",
name="test",
node_url="http://node2:8080/v1",
node_id="node-2",
routing_strategy="rendezvous",
),
]
)
mock_send = AsyncMock()
monkeypatch.setattr(console_router._console, "route_create_workstream", mock_create)
monkeypatch.setattr(console_router._console, "route_send", mock_send)
ws_id, is_new = await console_router.get_or_create_workstream(
"slack",
"ch-1",
name="test",
initial_message="hello",
)
assert (ws_id, is_new) == ("ws-fresh", True)
assert [item.kwargs["resume_ws"] for item in mock_create.await_args_list] == [
"ws-pruned",
"",
]
mock_send.assert_awaited_once_with("hello", "ws-fresh")
assert console_router._node_urls["ws-fresh"] == "http://node2:8080/v1"
mock_storage.create_channel_route.assert_called_once_with("slack", "ch-1", "ws-fresh")
@pytest.mark.anyio
@pytest.mark.parametrize(
("status_code", "message"),
[
(404, "Workstream not found"),
(403, "Forbidden"),
(503, "Storage unavailable"),
(409, "Fork source is no longer available"),
(404, "Project not found"),
],
ids=["masked-acl", "forbidden", "operational", "conflict", "other-not-found"],
)
@pytest.mark.parametrize("via_console", [False, True], ids=["server", "console"])
async def test_stale_source_does_not_retry_other_failures(
self,
router: ChannelRouter,
console_router: ChannelRouter,
mock_storage: MagicMock,
monkeypatch: pytest.MonkeyPatch,
status_code: int,
message: str,
via_console: bool,
) -> None:
selected = console_router if via_console else router
mock_storage.get_channel_route.return_value = {
"ws_id": "ws-stale",
"channel_type": "discord",
"channel_id": "ch-1",
}
monkeypatch.setattr(selected, "_is_ws_live", AsyncMock(return_value=False))
mock_create = AsyncMock(side_effect=TurnstoneAPIError(status_code, message))
if selected._console is not None:
monkeypatch.setattr(selected._console, "route_create_workstream", mock_create)
else:
assert selected._server is not None
monkeypatch.setattr(selected._server, "create_workstream", mock_create)
with pytest.raises(TurnstoneAPIError) as exc_info:
await selected.get_or_create_workstream("discord", "ch-1")
assert exc_info.value.status_code == status_code
assert exc_info.value.message == message
mock_create.assert_awaited_once()
mock_storage.delete_channel_route.assert_not_called()
mock_storage.create_channel_route.assert_not_called()
@pytest.mark.anyio
async def test_fresh_retry_is_attempted_only_once(
self,
router: ChannelRouter,
mock_storage: MagicMock,
monkeypatch: pytest.MonkeyPatch,
) -> None:
mock_storage.get_channel_route.return_value = {
"ws_id": "ws-pruned",
"channel_type": "discord",
"channel_id": "ch-1",
}
mock_storage.resolve_workstream.side_effect = None
mock_storage.resolve_workstream.return_value = None
assert router._server is not None
error = TurnstoneAPIError(404, "Workstream not found")
mock_create = AsyncMock(side_effect=[error, error])
monkeypatch.setattr(router._server, "create_workstream", mock_create)
with pytest.raises(TurnstoneAPIError, match="Workstream not found"):
await router.get_or_create_workstream("discord", "ch-1")
assert [item.kwargs["resume_ws"] for item in mock_create.await_args_list] == [
"ws-pruned",
"",
]
mock_storage.create_channel_route.assert_not_called()
@pytest.mark.anyio
async def test_storage_lookup_failure_preserves_route(
self,
router: ChannelRouter,
mock_storage: MagicMock,
monkeypatch: pytest.MonkeyPatch,
) -> None:
mock_storage.get_channel_route.return_value = {
"ws_id": "ws-existing",
"channel_type": "discord",
"channel_id": "ch-1",
}
mock_storage.resolve_workstream.side_effect = RuntimeError("storage offline")
assert router._server is not None
mock_create = AsyncMock()
monkeypatch.setattr(router._server, "create_workstream", mock_create)
with pytest.raises(RuntimeError, match="storage offline"):
await router.get_or_create_workstream("discord", "ch-1")
mock_storage.delete_channel_route.assert_not_called()
mock_create.assert_not_awaited()
@pytest.mark.anyio
@pytest.mark.parametrize("via_console", [False, True], ids=["server", "console"])
async def test_live_probe_failure_preserves_route_without_creating(
self,
router: ChannelRouter,
console_router: ChannelRouter,
mock_storage: MagicMock,
monkeypatch: pytest.MonkeyPatch,
via_console: bool,
) -> None:
selected = console_router if via_console else router
mock_storage.get_channel_route.return_value = {
"ws_id": "ws-existing",
"channel_type": "discord",
"channel_id": "ch-1",
}
probe_error = TurnstoneAPIError(503, "route uncertain")
monkeypatch.setattr(selected, "_is_ws_live", AsyncMock(side_effect=probe_error))
mock_create = AsyncMock()
if selected._console is not None:
monkeypatch.setattr(selected._console, "route_create_workstream", mock_create)
else:
assert selected._server is not None
monkeypatch.setattr(selected._server, "create_workstream", mock_create)
with pytest.raises(TurnstoneAPIError, match="route uncertain"):
await selected.get_or_create_workstream("discord", "ch-1")
mock_storage.delete_channel_route.assert_not_called()
mock_create.assert_not_awaited()
@pytest.mark.anyio
async def test_sends_initial_message_for_new_workstream(
self,
+3
View File
@@ -62,9 +62,11 @@ class _FakeConnect:
def __init__(self, queue: list[_FakeEventSource]) -> None:
self._queue = queue
self.call_count = 0
self.calls: list[tuple[tuple[object, ...], dict[str, object]]] = []
def __call__(self, *args, **kwargs): # noqa: ANN001, ANN204
self.call_count += 1
self.calls.append((args, kwargs))
if not self._queue:
raise asyncio.CancelledError
return self._queue.pop(0)
@@ -128,6 +130,7 @@ class TestStaleRoute:
on_event.assert_not_awaited()
# No reconnect after 404.
assert fake_connect.call_count == 1
assert fake_connect.calls[0][1]["params"] == {"user_turn": 1}
assert _fast_sleep == []
def test_on_stale_exception_still_exits(self, monkeypatch, _fast_sleep):
+229 -15
View File
@@ -26,7 +26,9 @@ import json
import pytest
from tests._session_helpers import make_session
from turnstone.core.trajectory import turns_from_dicts
from turnstone.core.session import _SummaryResult
from turnstone.core.storage._utils import _fork_turn_insert_row
from turnstone.core.trajectory import PROVENANCE_META_KEY, TurnProvenance, turns_from_dicts
def _marker_meta(watermark: int | None) -> str | None:
@@ -247,7 +249,11 @@ def test_compaction_persists_checkpoint_and_resume_is_bounded(tmp_db, mock_opena
sess._ws_id = ws
sess.messages = turns_from_dicts(history)
sess._msg_tokens = [1] * len(history)
with patch.object(sess, "_summarize_blocks", return_value="DENSE SUMMARY"):
with patch.object(
sess,
"_summarize_blocks",
return_value=_SummaryResult(text="DENSE SUMMARY", producer="summary-producer"),
):
assert sess._compact_messages(auto=False) is True
# Conversation continues after the compaction.
@@ -263,6 +269,140 @@ def test_compaction_persists_checkpoint_and_resume_is_bounded(tmp_db, mock_opena
assert not any(t.startswith("turn ") for t in texts) # full history NOT reloaded
def test_marker_watermark_read_blip_recovers_on_retry(tmp_db, mock_openai_client):
"""Round-3 review pin: a transient watermark-read failure aborts that
persist attempt the journal classifies the marker row retrying with NO
meta bytes memoized so the healthy retry re-reads and commits WITH a
watermark. Memoizing the failed read as "absent" would durably commit a
permanently checkpoint-less marker: full-history rehydration and an
immediate re-compaction on every reopen."""
from unittest.mock import patch
from turnstone.core.memory import register_workstream, save_message
from turnstone.core.storage._registry import get_storage
ws = "wsBLIP"
register_workstream(ws, user_id="u1", name="t")
history = [
{"role": "user" if i % 2 == 0 else "assistant", "content": f"turn {i}"} for i in range(6)
]
for h in history:
save_message(ws, h["role"], h["content"])
sess = make_session(client=mock_openai_client, context_window=10_000, max_tokens=1_000)
sess._ws_id = ws
sess.messages = turns_from_dicts(history)
sess._msg_tokens = [1] * len(history)
st = get_storage()
real_watermark = st.get_compaction_watermark
reads: list[int] = []
def _flaky_watermark(ws_id: str, preserve_tail: int = 0) -> int | None:
reads.append(preserve_tail)
if len(reads) == 1:
raise RuntimeError("transient watermark blip")
return real_watermark(ws_id, preserve_tail)
from turnstone.core.session import ConversationPersistenceError
with (
patch.object(
sess,
"_summarize_blocks",
return_value=_SummaryResult(text="DENSE SUMMARY", producer="summary-producer"),
),
patch.object(st, "get_compaction_watermark", side_effect=_flaky_watermark),
):
# The first persist attempt dies at the read and surfaces like any
# initial durability failure; the in-memory compaction stays applied
# and the marker row is retained in the journal, nothing durable yet.
with pytest.raises(ConversationPersistenceError):
sess._compact_messages(auto=False)
assert sess.conversation_persistence_status()["state"] == "retrying"
assert st.get_compaction_checkpoint(ws) is None
sess._reconcile_pending_conversation_commits(_force_retry=True)
assert sess.conversation_persistence_status()["state"] == "healthy"
checkpoint = st.get_compaction_checkpoint(ws)
assert checkpoint is not None
assert checkpoint == real_watermark(ws, 0)
def test_compaction_summary_producer_survives_storage_round_trip(
storage_backend, mock_openai_client
):
"""Final summary producer and model provenance are durable metadata.
A compaction marker has no provider-native payload, so its producer belongs
in the marker's ``summary_producer`` meta field. Checkpoint reconstruction
maps that object to the summary Turn's ``meta.extra["source_meta"]`` while
retaining the accepted model alias/backend/generation/principal tuple in
the well-known provenance envelope.
"""
st = storage_backend
ws = _register(st, "ws-summary-producer")
history = [
{"role": "user" if i % 2 == 0 else "assistant", "content": f"turn {i}"} for i in range(6)
]
for message in history:
st.save_message(ws, message["role"], message["content"])
sess = make_session(client=mock_openai_client, context_window=10_000, max_tokens=1_000)
sess._ws_id = ws
sess.messages = turns_from_dicts(history)
sess._msg_tokens = [1] * len(history)
provenance = TurnProvenance(
model_alias="summary-alias",
backend_model_id="summary-kernel",
registry_generation=12,
acting_principal_id="user-alice",
)
with pytest.MonkeyPatch.context() as monkeypatch:
monkeypatch.setattr(
sess,
"_summarize_blocks",
lambda *_args, **_kwargs: _SummaryResult(
text="DENSE SUMMARY",
producer="final-summary-producer",
provenance=provenance,
),
)
assert sess._compact_messages(auto=False) is True
marker = next(
message
for message in st.load_messages(ws, include_compaction=True)
if message.get("_source") == "compaction"
)
assert marker["_source_meta"]["summary_producer"] == "final-summary-producer"
assert "_provenance" not in marker
assert "user-alice" not in json.dumps(marker)
loaded = st.load_message_turns(ws)
assert [turn.text for turn in loaded[:2]] == ["[Conversation summary]", "DENSE SUMMARY"]
assert "source_meta" not in loaded[0].meta.extra
assert loaded[1].meta.extra["source_meta"]["summary_producer"] == "final-summary-producer"
assert loaded[1].meta.extra[PROVENANCE_META_KEY] == provenance.to_meta()
fork_row, _attachment_ids = _fork_turn_insert_row(
loaded[1],
"compaction-provenance-fork",
"2026-08-09T00:00:00",
)
fork_meta = json.loads(fork_row["meta"])
assert fork_meta["summary_producer"] == "final-summary-producer"
assert fork_meta[PROVENANCE_META_KEY] == provenance.to_meta()
reopened = make_session(client=mock_openai_client, context_window=10_000, max_tokens=1_000)
assert reopened.resume(ws) is True
assert reopened.messages[1].text == "DENSE SUMMARY"
assert (
reopened.messages[1].meta.extra["source_meta"]["summary_producer"]
== "final-summary-producer"
)
assert reopened.messages[1].meta.extra[PROVENANCE_META_KEY] == provenance.to_meta()
# ---------------------------------------------------------------------------
# Malformed / edge-case markers — the watermark guards and the empty tail
# ---------------------------------------------------------------------------
@@ -446,11 +586,12 @@ def test_persist_truncation_uncompacted_matches_plain_tail_delete(tmp_db, mock_o
assert st.count_messages(ws) == 3
def test_persist_truncation_skips_delete_when_count_unavailable(tmp_db, mock_openai_client):
"""count_messages==0 (the storage-error sentinel) must NOT delete — a wrong
truncation would lose user history."""
def test_persist_truncation_propagates_atomic_storage_failure(tmp_db, mock_openai_client):
"""The strict backend error must reach the failure-atomic session caller."""
from unittest.mock import patch
import pytest
from turnstone.core.memory import get_storage, register_workstream, save_message
ws = "wsCnt"
@@ -460,25 +601,98 @@ def test_persist_truncation_skips_delete_when_count_unavailable(tmp_db, mock_ope
st = get_storage()
sess = make_session(client=mock_openai_client)
sess._ws_id = ws
with patch("turnstone.core.session.count_messages", return_value=0):
with (
patch.object(
st,
"truncate_messages_tail",
side_effect=RuntimeError("injected atomic truncation failure"),
),
pytest.raises(RuntimeError, match="injected atomic truncation failure"),
):
sess._persist_truncation(2)
assert st.count_messages(ws) == 4 # nothing deleted
def test_persist_truncation_skips_delete_when_floor_unavailable(tmp_db, mock_openai_client):
"""get_compaction_floor==-1 (the storage-error sentinel) must NOT delete — a 0
floor on a compacted ws could otherwise drop the marker on an over-deep trim."""
def test_persist_truncation_zero_is_a_storage_noop(tmp_db, mock_openai_client):
"""A no-op plan never opens the strict backend transaction."""
from unittest.mock import patch
from turnstone.core.memory import get_storage, register_workstream, save_message
from turnstone.core.memory import get_storage, register_workstream
ws = "wsFloor"
register_workstream(ws, user_id="u1", name="t")
for i in range(4):
save_message(ws, "user", f"m{i}")
st = get_storage()
sess = make_session(client=mock_openai_client)
sess._ws_id = ws
with patch("turnstone.core.session.get_compaction_floor", return_value=-1):
sess._persist_truncation(2)
assert st.count_messages(ws) == 4 # nothing deleted
with patch.object(st, "truncate_messages_tail") as truncate:
assert sess._persist_truncation(0) == 0
truncate.assert_not_called()
def test_watermark_reads_inside_the_marker_persist_not_before_the_commit(
tmp_db, mock_openai_client
):
"""The boundary is cut in the ordered durable batch, at persist time.
A pre-commit snapshot can undercount the durable prefix whenever accepted
rows are still pending in the journal when compaction is admitted (they
land, FIFO, before the marker's persist executes). Reading inside the
persist closure names exactly the summarized prefix; memoization keeps a
keyed lost-ACK retry byte-identical.
"""
from unittest.mock import patch
from turnstone.core.memory import register_workstream, save_message
from turnstone.core.storage._registry import get_storage
ws = "wsWatermarkOrder"
register_workstream(ws, user_id="u1", name="t")
history = [
{"role": "user" if i % 2 == 0 else "assistant", "content": f"turn {i}"} for i in range(6)
]
for h in history:
save_message(ws, h["role"], h["content"])
sess = make_session(client=mock_openai_client, context_window=10_000, max_tokens=1_000)
sess._ws_id = ws
sess.messages = turns_from_dicts(history)
sess._msg_tokens = [1] * len(history)
order: list[str] = []
real_journal = sess._journal_conversation_row_locked
# The persist closure reads the backend directly (bypassing the memory
# wrapper's error-to-None coercion), so the spy sits on the storage method.
st = get_storage()
real_watermark = st.get_compaction_watermark
def _journal_spy(**kwargs):
order.append("journal")
return real_journal(**kwargs)
def _watermark_spy(ws_id, preserve_tail=0):
order.append("watermark")
return real_watermark(ws_id, preserve_tail)
with (
patch.object(sess, "_journal_conversation_row_locked", side_effect=_journal_spy),
patch.object(st, "get_compaction_watermark", side_effect=_watermark_spy),
patch.object(
sess,
"_summarize_blocks",
return_value=_SummaryResult(text="DENSE SUMMARY", producer="summary-producer"),
),
):
assert sess._compact_messages(auto=False) is True
assert order == ["journal", "watermark"], order
# The durable marker's boundary covers every row persisted before it.
from turnstone.core.storage import get_storage
rows = get_storage().load_messages(ws, repair=False, include_compaction=True)
marker = next(r for r in rows if r.get("_source") == "compaction")
watermark = (marker.get("_source_meta") or {}).get("watermark")
plain_ids = [r["id"] for r in rows if r.get("_source") != "compaction" if "id" in r]
assert isinstance(watermark, int)
if plain_ids:
assert watermark >= max(plain_ids)
+47 -24
View File
@@ -33,6 +33,7 @@ import pytest
from tests._session_helpers import make_session
from turnstone.core.session import COMPACTION_SOURCE, COMPACTION_SUMMARY_LABEL
from turnstone.core.storage import get_storage
from turnstone.core.trajectory import turns_from_dicts
@@ -42,17 +43,34 @@ def session(tmp_db, mock_openai_client):
the summary output reserve is tiny and the carry budget is easy to compute
(reserve=100, margin=500, spare=9_400, budget=min(2_500, 9_400)=2_500
tokens 10_000 chars at the uncalibrated 4.0 chars/token)."""
return make_session(
s = make_session(
client=mock_openai_client,
context_window=10_000,
compact_max_tokens=100,
max_tokens=1_000,
tool_timeout=10,
)
_register_session_workstream(s)
return s
def _register_session_workstream(session):
"""Give direct ChatSession fixtures their production parent row."""
get_storage().register_workstream(
session.ws_id,
user_id=session._user_id,
kind=session._kind,
parent_ws_id=session._parent_ws_id,
)
return session
def _stub_summary(text: str = "DENSE"):
return SimpleNamespace(content=text, finish_reason="stop")
return SimpleNamespace(
content=text,
finish_reason="stop",
producer="test-summary-provider",
)
# ---------------------------------------------------------------------------
@@ -353,7 +371,7 @@ class TestWindDownSpill:
spare // 2, so two oversize carries land truncated to the shared
budget instead of stacking two solo quarter-window allowances on top
of the half-window summary reserve."""
s = make_session(client=mock_openai_client, tool_timeout=10)
s = _register_session_workstream(make_session(client=mock_openai_client, tool_timeout=10))
per_carry = s._carry_budget_chars(2)
ask = "ASK-HEAD " + "a" * (per_carry * 2) + " ASK-TAIL"
spill = "PLAN-HEAD " + "b" * (per_carry * 2) + " PLAN-TAIL"
@@ -378,10 +396,11 @@ class TestWindDownSpill:
def test_do_auto_compact_forwards_carry_spill(self, session):
"""The end-of-turn site passes carry_spill=stopped_to_compact through
_do_auto_compact pin the forwarding."""
generation = session._claim_generation()
with patch.object(session, "_compact_messages", return_value=True) as cm:
session._do_auto_compact(my_generation=3, carry_spill=True)
session._do_auto_compact(my_generation=generation, carry_spill=True)
assert cm.call_args.kwargs["carry_spill"] is True
assert cm.call_args.kwargs["my_generation"] == 3
assert cm.call_args.kwargs["my_generation"] == generation
# ---------------------------------------------------------------------------
@@ -427,16 +446,18 @@ def _coord_client(tasks=None, children=None) -> MagicMock:
def _coord_session(mock_openai_client, *, coord_client=..., **kwargs):
from turnstone.core.workstream import WorkstreamKind
return make_session(
client=mock_openai_client,
context_window=10_000,
compact_max_tokens=100,
max_tokens=1_000,
tool_timeout=10,
kind=WorkstreamKind.COORDINATOR,
user_id="u1",
coord_client=_coord_client() if coord_client is ... else coord_client,
**kwargs,
return _register_session_workstream(
make_session(
client=mock_openai_client,
context_window=10_000,
compact_max_tokens=100,
max_tokens=1_000,
tool_timeout=10,
kind=WorkstreamKind.COORDINATOR,
user_id="u1",
coord_client=_coord_client() if coord_client is ... else coord_client,
**kwargs,
)
)
@@ -488,8 +509,8 @@ class TestCoordinatorHandles:
fails and they must revisit that trade rather than stack both.
"""
coord = _coord_session(mock_openai_client)
interactive = make_session(
client=mock_openai_client, context_window=10_000, tool_timeout=10
interactive = _register_session_workstream(
make_session(client=mock_openai_client, context_window=10_000, tool_timeout=10)
)
prompts = []
for s in (coord, interactive):
@@ -512,12 +533,14 @@ class TestCoordinatorHandles:
children, so the reads are skipped entirely not merely rendered
empty and its summary is what it was before this existed."""
client = _coord_client()
s = make_session(
client=mock_openai_client,
context_window=10_000,
compact_max_tokens=100,
tool_timeout=10,
coord_client=client, # present but irrelevant: kind decides
s = _register_session_workstream(
make_session(
client=mock_openai_client,
context_window=10_000,
compact_max_tokens=100,
tool_timeout=10,
coord_client=client, # present but irrelevant: kind decides
)
)
text = _compact(s)
assert "## Handles" not in text
@@ -655,7 +678,7 @@ class TestCoordinatorHandles:
s = _coord_session(mock_openai_client)
s._ws_id = "ws-coord"
with (
patch("turnstone.core.session.get_compaction_watermark", return_value=7),
patch.object(get_storage(), "get_compaction_watermark", return_value=7),
patch("turnstone.core.session.save_message") as saved,
):
_compact(s)
+253
View File
@@ -0,0 +1,253 @@
"""Behavior and wiring tests for large-paste attachment conversion."""
from __future__ import annotations
import json
import shutil
import subprocess
from pathlib import Path
import pytest
_ROOT = Path(__file__).resolve().parent.parent
_HELPER = _ROOT / "turnstone/shared_static/composer_paste_text.js"
_COMPOSER = _ROOT / "turnstone/shared_static/composer.js"
_ATTACHMENTS = _ROOT / "turnstone/shared_static/composer_attachments.js"
_INTERACTIVE = _ROOT / "turnstone/shared_static/interactive.js"
_UI_APP = _ROOT / "turnstone/ui/static/app.js"
_CONSOLE_APP = _ROOT / "turnstone/console/static/app.js"
_COORDINATOR = _ROOT / "turnstone/console/static/coordinator/coordinator.js"
def test_paste_text_helper_behavior(tmp_path: Path) -> None:
"""Execute the real ESM and pin its fixed threshold, byte cap, and dedup."""
if shutil.which("node") is None:
pytest.skip("node binary not available on PATH")
module = tmp_path / "composer_paste_text.mjs"
module.write_text(_HELPER.read_text(encoding="utf-8"), encoding="utf-8")
script = tmp_path / "paste_harness.mjs"
module_url = json.dumps(module.as_uri())
script.write_text(
f"const moduleUrl = {module_url};\n"
+ r"""
import { Blob, File } from "node:buffer";
globalThis.Blob = Blob;
globalThis.File = File;
globalThis.window = globalThis;
function check(condition, message) {
if (!condition) throw new Error(message);
}
const paste = await import(moduleUrl);
check(paste.PASTE_ATTACHMENT_CHARS === 2000, "fixed threshold drifted");
check(
paste.pasteTextToFile("x".repeat(1999)) === null,
"below-threshold text converted",
);
const exactText = "x".repeat(2000);
const exact = paste.pasteTextToFile(exactText);
check(exact === null, "exact-threshold text converted");
const convertedText = "x".repeat(2001);
const converted = paste.pasteTextToFile(convertedText);
check(converted instanceof File, "above-threshold text did not convert");
check(converted.name === "pasted-text.txt", "filename drifted");
check(converted.type === "text/plain", "MIME drifted");
check(converted.size === 2001, "byte size drifted");
check(
(await converted.text()) === convertedText,
"file content did not round-trip",
);
check(
paste.pasteTextToFile("") === null,
"empty text converted",
);
check(
paste.pasteTextToFile("😀".repeat(2000)) === null,
"UTF-16 code units were counted as characters",
);
check(
paste.pasteTextToFile("😀".repeat(2001)) instanceof File,
"Unicode code points above the threshold did not convert",
);
const cjkAtCap = "".repeat(Math.floor(paste.TEXT_ATTACHMENT_MAX_BYTES / 3));
check(
paste.pasteTextToFile(cjkAtCap) instanceof File,
"text within the byte ceiling did not convert",
);
check(
paste.pasteTextToFile(cjkAtCap + "") === null,
"multibyte text escaped the byte ceiling",
);
const sameText = "same".repeat(501);
const sameA = paste.pasteTextToFile(sameText);
const sameB = paste.pasteTextToFile(sameText);
const different = paste.pasteTextToFile("size".repeat(501));
check(
paste.isDuplicatePastedTextFile(sameB, [sameA]),
"identical synthesized pastes did not deduplicate",
);
check(
!paste.isDuplicatePastedTextFile(different, [sameA]),
"different same-size pastes were deduplicated",
);
check(
!paste.isDuplicatePastedTextFile(
new File([sameText], "ordinary.txt", { type: "text/plain" }),
[sameA],
),
"ordinary user files entered synthesized-paste dedup",
);
""",
encoding="utf-8",
)
proc = subprocess.run(
["node", str(script)],
capture_output=True,
text=True,
timeout=15,
)
assert proc.returncode == 0, f"paste harness failed:\n{proc.stderr}\n{proc.stdout}"
def test_attachment_snapshot_reports_in_flight_upload(tmp_path: Path) -> None:
"""A synthesized paste cannot disappear from a send-time snapshot while
its immediate upload is still resolving."""
if shutil.which("node") is None:
pytest.skip("node binary not available on PATH")
script = tmp_path / "attachment_snapshot_harness.mjs"
module_url = json.dumps(_ATTACHMENTS.as_uri())
script.write_text(
f"const moduleUrl = {module_url};\n"
+ r"""
import { File } from "node:buffer";
globalThis.File = File;
globalThis.window = globalThis;
function fakeElement() {
return {
children: [],
dataset: {},
appendChild(child) {
this.children.push(child);
return child;
},
addEventListener() {},
querySelector() { return null; },
setAttribute() {},
remove() {},
};
}
globalThis.document = { createElement: () => fakeElement() };
let resolveUpload;
const uploadResponse = new Promise((resolve) => { resolveUpload = resolve; });
const attachmentsModule = await import(moduleUrl);
const controller = attachmentsModule.createAttachmentController({
chipsEl: fakeElement(),
getWsId: () => "ws-1",
authFetch: () => uploadResponse,
});
let snap = controller.snapshot();
if (snap.uploading || snap.attachment_ids.length)
throw new Error("empty controller reported an upload");
controller.upload(new File(["large paste"], "pasted-text.txt", {
type: "text/plain",
}));
snap = controller.snapshot();
if (!snap.uploading)
throw new Error("in-flight placeholder was omitted from snapshot state");
if (snap.attachments.length || snap.attachment_ids.length)
throw new Error("placeholder escaped into stable attachment arrays");
resolveUpload({
ok: true,
status: 200,
json: () => Promise.resolve({
attachment_id: "attachment-1",
filename: "pasted-text.txt",
size_bytes: 11,
mime_type: "text/plain",
kind: "text",
}),
});
await new Promise((resolve) => setTimeout(resolve, 0));
snap = controller.snapshot();
if (snap.uploading)
throw new Error("settled upload remained marked in flight");
if (snap.attachment_ids.join(",") !== "attachment-1")
throw new Error("settled upload was not sendable: " + snap.attachment_ids);
""",
encoding="utf-8",
)
proc = subprocess.run(
["node", str(script)],
capture_output=True,
text=True,
timeout=15,
)
assert proc.returncode == 0, (
f"attachment snapshot harness failed:\n{proc.stderr}\n{proc.stdout}"
)
def test_paste_text_wiring_guard_rails() -> None:
"""Guard every surface around the behavior-tested shared helper."""
helper = _HELPER.read_text(encoding="utf-8")
composer = _COMPOSER.read_text(encoding="utf-8")
attachments = _ATTACHMENTS.read_text(encoding="utf-8")
interactive = _INTERACTIVE.read_text(encoding="utf-8")
ui_app = _UI_APP.read_text(encoding="utf-8")
console_app = _CONSOLE_APP.read_text(encoding="utf-8")
coordinator = _COORDINATOR.read_text(encoding="utf-8")
assert "window.TurnstonePasteText" in helper
assert "PASTE_ATTACHMENT_CHARS = 2000" in helper
assert "paste_attachment_chars" not in helper
assert "loadPasteThresholdChars" not in helper
assert 'from "./composer_paste_text.js"' in composer
assert "pasteTextToFile(text" in composer
assert "2000" not in composer, "the threshold belongs in the shared helper"
assert composer.index("if (uploaded > 0)") < composer.index("pasteTextToFile(text")
assert "if (accepted !== false) e.preventDefault();" in composer
assert "if (pending.has(info.attachment_id))" in attachments
assert "pending.delete(placeholderId);" in attachments
assert "uploading: uploading" in attachments
assert "function _handleComposerPaste(" in ui_app
assert "if (files.length > 0)" in ui_app
assert "if (!textFile || addFiles([textFile]) === false) return false;" in ui_app
assert "_handleComposerPaste(event, _newWsAddFiles)" in ui_app
assert "_handleComposerPaste(e, _addDashboardFiles)" in ui_app
assert "initEl.onpaste" in ui_app
assert ui_app.count("Add a message to send with this attachment.") == 2
assert "_loadPasteAttachmentSetting" not in ui_app
assert "return _homeStageFile(file);" in console_app
assert "isDuplicatePastedTextFile(file, _homeStagedFiles)" in console_app
assert "Add a message to send with this attachment." in console_app
assert "_loadPasteAttachmentSetting" not in console_app
for pane in (interactive, coordinator):
assert "Add a message to send with this attachment." in pane
assert "Attachments can't be sent while the assistant is working." in pane
assert "if (snap.uploading)" in pane
assert "Wait for attachments to finish uploading before sending." in pane
assert "!attachments.isEmpty()" in pane or "!this.attachments.isEmpty()" in pane
assert "loadPasteThresholdChars" not in coordinator
for page in (
_ROOT / "turnstone/ui/static/index.html",
_ROOT / "turnstone/console/static/index.html",
_ROOT / "turnstone/console/static/coordinator/index.html",
):
body = page.read_text(encoding="utf-8")
assert "/shared/composer_paste_text.js" in body, page
assert body.index("/shared/composer_paste_text.js") < body.index("/shared/composer.js"), (
page
)

Some files were not shown because too many files have changed in this diff Show More