Compare commits

...

15 Commits

Author SHA1 Message Date
Patrick Buckley 7000ef03a8 chore: bump version to 1.6.9 2026-06-17 18:21:01 -07:00
Patrick Buckley 0ae0db55f3 test: address Copilot review on the leaked-thread guard
- The guard snapshotted live threads by `Thread.ident`, but idents are
  recycled after a thread exits — a new leaked thread reusing an exited
  thread's ident would be mistaken for pre-existing and missed (false
  negative). Snapshot the Thread OBJECTS and compare by identity instead.
- Fix the `serve` fixture docstring: the factory returns the ephemeral
  port, not the server.
2026-06-17 18:16:34 -07:00
Patrick Buckley 8256e7441a test: eliminate leaked-thread test pollution + guard against it
Background daemons, event loops, and test servers that outlived their test
bled into later tests' captured output — an intermittent "I/O operation on
closed file" heisenbug, and the same class behind a past multi-day CI-hang
investigation.

- conftest: a fail-on-leak autouse guard (`_no_leaked_threads`) snapshots
  threads at setup and fails any test that leaves one running past teardown,
  with an `allow_thread_leak` opt-out — so the next leak is caught in minutes,
  not days. Plus `logging.raiseExceptions = False` to mute the benign
  logging-vs-capture-teardown race, and shared loop/server teardown helpers
  (`stop_loop_thread`, `serve_until_exit`).

- collector (PRODUCT FIX): the node-discovery loop slept uninterruptibly, so
  `ClusterCollector.stop()` couldn't join the `console-discovery` thread until
  the full interval elapsed — a real shutdown hang in production (up to
  `discovery_interval`). It now sleeps on an interruptible Event that `stop()`
  sets and `start()` clears.

- test fixtures: docker_healthcheck's HTTP servers, the MCP background event
  loops (shutdown_default_executor + close), and the FastMCP uvicorn upstreams
  (timeout_graceful_shutdown=0 + force_exit) now tear down cleanly instead of
  leaking.

Full non-live suite: 7456 passed, 0 closed-file errors, 0 leaked threads, and
~1.5 min faster (the leaks were dragging it).
2026-06-17 18:16:34 -07:00
Patrick Buckley 1f121c739c feat(coordinator): port Regenerate/Edit title to coordinators
Coordinators carry LLM/auto titles like interactive workstreams but had no
way to regenerate or rename them. Port the interactive "Refresh title" (LLM
regenerate) + "Edit title" (manual alias) dropdown actions by lifting the
two handlers — the last shared verbs that weren't yet lifted — and opting
coordinators in.

- session_routes.py: add make_refresh_title_handler / make_set_title_handler
  factories (cfg pattern, mirroring make_close_handler). set_title resolves
  the workstream BEFORE the alias write and 404s when the kind has no
  tenant_check storage gate and the in-memory manager doesn't own it:
  set_workstream_alias is a global, kind-unscoped UPDATE, so this prevents
  an operator renaming a workstream the coord manager doesn't own (e.g. an
  interactive ws via the coord route) and the silent-200 on a bogus id.
- server.py: re-point the interactive bundle to the lifted handlers; drop
  the standalone refresh_workstream_title / set_workstream_title.
- console/server.py: wire refresh_title / set_title into the coord bundle
  (gated by the existing admin.coordinator operator check).
- shell.js: enable titleVerbs on the coordinator pane's tab menu; the
  base-aware lane posts to the console-origin coord routes.

Tests: coord refresh/set-title (regenerate, operator-gate, 404 unknown,
alias store + broadcast, empty, conflict, cross-kind reject); interactive
title tests re-pointed to the lifted handlers for lift-parity; shell.js
coord-menu assertion.
2026-06-17 18:16:34 -07:00
Patrick Buckley 026acbf907 fix(coordinator): address Copilot review on title persistence
Three points from the PR #676 Copilot review:

- _coord_display_name ran on a lifecycle-event path and called
  get_workstream_display_name → get_storage(), which auto-initializes a
  SQLite .turnstone.db in the CWD when storage isn't initialized yet —
  a stray-file footgun on early-startup / unit-test paths. Add
  is_storage_initialized() to the storage registry and skip the DB read
  (fall back to ws.name) when storage isn't up. (Copilot's "skip when
  ws.name is non-synthetic" suggestion would have broken alias > title >
  name, so guard on init state instead.)

- Document, on SessionUIBase, that on_aux_usage (storage/metrics, no
  _ws_lock state) and on_rename (queue/locked fan-out) are safe to call
  from a concurrent auxiliary thread — the title-gen thread now runs
  during streaming, and these are the only two UI hooks it touches. No
  behavior change: the methods were already thread-safe (the same path
  task_agent sub-agents use); the contract just didn't say so. Add a
  matching note at the title-trigger site.

- Note in _coordinator_rows that the secondary `title` field is
  best-effort for a live coord outside the limit=200 window (the
  user-visible `name` stays correct via the uncapped bulk lookup, and
  the window is unreachable in practice — live coords are max_active-
  bounded and sort to the top of updated DESC).
2026-06-17 18:16:34 -07:00
Patrick Buckley 49013a5593 fix(coordinator): persist + eagerly generate workstream titles
Coordinator workstream LLM titles were written to workstreams.title but
never read back, and were rarely generated in the first place:

- Read path: the dashboard's `_coordinator_rows` builder hardcoded
  title="" and used the synthetic `ws.name`, so a generated title (or a
  user alias) reverted to `ws-xxxx` on every refresh. Interactive rows
  resolve via get_workstream_display_name, so the gap was coord-only.
- Write path: the auto-title trigger only fired on a tool-call-free
  assistant turn, which coordinators (near-constant tool use) seldom
  reach — so the title almost never generated.

Read path:
- Project `title` + `alias` in list_workstreams (appended after user_id so
  existing positional fallbacks stay valid). `_coordinator_rows` resolves
  the display name (alias > title > name) for both lanes — live names via
  the bulk get_workstream_display_names (exact ids, no row cap), persisted
  rows from their own _mapping.
- Seed the console pseudo-node fan-out with the resolved display name so a
  rehydrated coordinator shows its title in the live tree immediately
  (one bulk lookup instead of an N+1 over mgr.list_all()).

Write path:
- Fire auto-title right after the user turn is recorded in send(), gated on
  a real (non-wake, non-empty) user message, instead of waiting for the
  terminal tool-call-free turn. Applies to interactive + coordinator.
- Snapshot self.messages in _generate_title since it can now run
  concurrently with the streaming turn.
2026-06-17 18:16:34 -07:00
Patrick Buckley ac68efba45 chore: bump version to 1.6.8 2026-06-17 01:33:33 -07:00
Patrick Buckley ea5b727ae9 fix(audio): omni STT transcode + thinking-off, with streaming
Speech-to-text against an omni chat model (e.g. Gemma-4 on vLLM) was
broken end to end:

- The browser records webm/opus, but the omni chat lane only decodes
  wav/mp3 (it sniffs the bytes), so every clip came back 400 "Invalid
  or unsupported audio file". Transcode the upload to 16 kHz mono WAV
  with ffmpeg first, hardened against the untrusted blob:
  -protocol_whitelist pipe (no file:/http: SSRF), -vn, and a duration cap.
- The chat STT path calls the raw client and so bypasses the provider's
  request shaping. It now forces enable_thinking=false (via the model's
  thinking_param): leaving reasoning on costs ~11x latency and returns
  empty content on some clips. The prompt precedes the audio part (the
  order Gemma documents for transcription) and max_tokens is capped.

Add a streaming variant: POST .../speech-to-text/stream returns the
transcript as plain-text deltas and the composer fills them in live
(~0.3s to first word). The blocking stream is driven from one worker
thread that owns and closes the upstream connection.

Drop the gemma skip_special_tokens server-compat workaround: the vLLM
bug it patched is fixed upstream, and a stale shim can corrupt output.

The node image now installs ffmpeg; rebuild to run this live.
2026-06-17 01:23:51 -07:00
Patrick Buckley 87e189ae7d fix(tls): stub backoff via a _sleep seam, not the global asyncio.sleep
The test-postgres failure on test_init_retries_exhausted_raises surfaced the
root cause: sleeps held 2275x 0.1 instead of [1.0, 2.0]. Those 0.1s came from
a concurrent background poller doing asyncio.sleep(0.1) on anyio's shared
(persistent) event loop — the tls retry tests patched the *global*
asyncio.sleep, which intercepted that poller too.

- Before: the stub didn't yield, so the poller busy-looped and monopolized
  the loop -> the test hung (the CI-only "after 92%" hang on 3.12+).
- The earlier "make the stub yield" change converted the hang into this
  flood (the poller spins instead of blocking), which is what exposed it.

Fix: route init()'s backoff through TLSClient._sleep so the tests stub that
method in isolation and never touch the global asyncio.sleep. Tasks sharing
the loop are no longer affected; schedule assertions are unchanged.

The deeper fragility this exploited — a leaked, un-cancelled background poller
surviving on the shared test loop — is left as a follow-up.
2026-06-17 01:23:51 -07:00
Patrick Buckley 795193fa00 fix(deadline): prefer a ready result over a same-window deadline/cancel
run_with_deadline checked the deadline/cancel before reading the result
queue, so a call that completed in the same scheduling window could be
reported as a spurious timeout. Drain the queue first.

Also from review:
- test_validate_regex_pattern stubs run_with_deadline, so the probe regex
  never runs — use a benign pattern instead of a real backtracking literal
  (the literal tripped a ReDoS scanner).
- output_guard_judge docstring: reference IntentJudge._parse_verdict instead
  of brittle judge.py line numbers.

The _runner BaseException catch is intentional and kept: it relays (not
swallows) whatever fn() raises to the caller via the queue; narrowing to
Exception would let a BaseException escape the worker so the caller never
gets a value, degrading the no-hang guarantee.
2026-06-17 01:23:51 -07:00
Patrick Buckley 155fbb1427 ci: cap the suite jobs at 20 minutes
A hung run otherwise rides GitHub's 6-hour default with -v streaming the
whole time (the source of the multi-GB job logs). Cap test and test-postgres
at 20 minutes so a flaky hang fails fast instead of bleeding hours.
2026-06-17 01:23:51 -07:00
Patrick Buckley 7e3ec8dea5 test(tls): yield in the asyncio.sleep stub (suspected CI-hang fix)
CI hung on test_init_retries_transient_failure (the new -v output named it:
its nodeid printed, no PASSED, the job rode to cancellation). It is the first
retry test that actually awaits the stubbed asyncio.sleep — the earlier tests
raise before sleeping — which points straight at the stub.

The stub returned without ever suspending, so the retry run completed in one
event-loop step with no checkpoint; that is fragile under the async test
runner and is the suspected cause (3.12/3.13/3.14 only — never reproduced on
3.11 or locally). Capture the real asyncio.sleep before patching and await
sleep(0) in the stub so it still yields, keeping the no-real-delay behavior
and the backoff-schedule assertions. Same fix in the discovery-failure test.
2026-06-17 01:23:51 -07:00
Patrick Buckley 6560f1ec4f fix(judge): daemon-thread call deadlines; raise local-model timeouts
The judges and the regex ReDoS probe ran a blocking call on a
ThreadPoolExecutor and abandoned the worker with shutdown(wait=False) on
timeout or cancel. concurrent.futures joins every executor worker from an
atexit hook regardless of wait=False, so a wedged call could pin
interpreter exit — and hang the test suite at shutdown.

Add turnstone/core/deadline.py::run_with_deadline: run a blocking callable
on a daemon thread bounded by a wall-clock timeout and an optional cancel
event. A daemon worker is never joined at exit, so abandoning one is safe.

Migrate three sites onto it:
- OutputGuardJudge.evaluate()
- IntentJudge._evaluate_single / _run_judge — this also removes
  _ExecutorPoisonedError and the executor-restart dance: per-call daemon
  threads can't poison a shared single-slot pool, so a timeout now returns
  None and the caller delivers one fallback verdict.
- console/server.py _validate_regex_pattern (regex ReDoS probe)

Also:
- Double the default judge LLM timeouts for slower local models:
  judge.timeout 60->120s and judge.output_guard_llm_timeout 30->60s
  (settings registry, JudgeConfig dataclass, --judge-timeout CLI default,
  class docstring, docs). Correct a stale doc that described the per-turn
  timeout as a total budget across turns.
- Raise the regex probe bound 0.5->3.0s so a legitimately complex pattern
  isn't false-flagged as catastrophic backtracking.
- CI: run pytest with -v instead of -q so a hang names the offending test
  instead of riding the job timeout.
- Tests: cover deadline.py and the regex validator; move test_judge.py off
  fixed sleeps onto the existing _wait_for helper.
2026-06-17 01:23:51 -07:00
Patrick Buckley 093239d614 chore: bump version to 1.6.7 2026-06-16 04:13:57 -07:00
Patrick Buckley f5ab26b4dc fix(deps): bump cryptography + starlette for security advisories
- cryptography >=48.0.1 (resolved 49.0.0): PyPI wheels <48.0.1 bundle a
  vulnerable statically-linked OpenSSL (GHSA-537c-gmf6-5ccf, 2026-06-09 secadv).
- starlette >=1.3.1: CVE-2026-54282 (path->authority host spoof via
  request.url reconstruction) + CVE-2026-54283 (url-encoded form DoS —
  max_fields / max_part_size silently ignored for x-www-form-urlencoded).

Full non-live suite green on the bumped deps.
2026-06-16 04:13:57 -07:00
47 changed files with 1865 additions and 477 deletions
+9 -2
View File
@@ -35,6 +35,9 @@ jobs:
test:
runs-on: ubuntu-latest
# Cap a hung run at 20 min instead of riding GitHub's 6-hour default
# (a flaky-hang run otherwise streams -v output for hours).
timeout-minutes: 20
strategy:
matrix:
python-version: ["3.11", "3.12", "3.13"]
@@ -51,7 +54,10 @@ jobs:
with:
node-version: "24"
- run: pip install -e ".[test]"
- run: pytest tests/ -m "not live" --cov=turnstone --cov-report=term-missing --cov-report=xml -q
# -v lists each test id as it starts (pytest prints the nodeid at
# logstart), so a hang names the culprit on the last line instead of
# riding the job timeout with only a trail of "..." dots.
- run: pytest tests/ -m "not live" --cov=turnstone --cov-report=term-missing --cov-report=xml -v
- uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
if: always()
with:
@@ -60,6 +66,7 @@ jobs:
test-postgres:
runs-on: ubuntu-latest
timeout-minutes: 20
services:
postgres:
image: postgres:18
@@ -83,7 +90,7 @@ jobs:
with:
node-version: "24"
- run: pip install -e ".[test]"
- run: pytest tests/ -m "not live" --storage-backend=postgresql -q
- run: pytest tests/ -m "not live" --storage-backend=postgresql -v
env:
TURNSTONE_TEST_PG_URL: postgresql+psycopg://postgres:postgres@localhost:5432/turnstone_test
+3 -1
View File
@@ -17,8 +17,10 @@ RUN rm -f /etc/dpkg/dpkg.cfg.d/docker
# ripgrep is the preferred backend for the search tool — natively bounds
# per-line, per-file, and per-filesize so pathological inputs (minified
# bundles, training-data JSONL with multi-MB single records) can't OOM us.
# ffmpeg transcodes omni STT uploads (browser webm/opus) to the 16 kHz mono
# WAV the omni chat-audio lane decodes.
RUN apt-get update && apt-get upgrade -y && apt-get install -y --no-install-recommends \
libpq5 git curl jq man-db manpages procps file ripgrep \
libpq5 git curl jq man-db manpages procps file ripgrep ffmpeg \
&& rm -rf /var/lib/apt/lists/*
# Node.js LTS (for npx-based MCP servers like @modelcontextprotocol/server-github)
+6 -5
View File
@@ -40,7 +40,7 @@ api_key = ""
smart_approvals = false # auto-approve high-confidence "approve" LLM verdicts (opt-in)
confidence_threshold = 0.95 # Smart Approvals auto-approve bar (LLM recommendation=approve)
max_context_ratio = 0.5 # max % of judge context window for history
timeout = 60.0 # seconds (generous for local models)
timeout = 120.0 # seconds (generous for local models)
read_only_tools = true # judge can use read_file/list_directory
cancel_on_approval = false # stop judging remaining tool calls once user decides
```
@@ -71,7 +71,7 @@ All fields are optional. The judge is enabled by default; use `enabled = false`
--judge / --no-judge Enable/disable (default: enabled)
--judge-model MODEL Model for judge
--judge-provider PROVIDER Provider for judge
--judge-timeout SECONDS LLM judge timeout (default: 60)
--judge-timeout SECONDS LLM judge timeout (default: 120)
--judge-confidence FLOAT Confidence threshold, 0-1 (default: 0.95)
```
@@ -193,9 +193,10 @@ Security hardening blocks access to sensitive paths:
### Timeout
The `timeout` setting (default 60 seconds) is a total budget across all judge
turns. Time is decremented after each LLM call. If the budget expires mid-turn,
the judge attempts to parse whatever partial response is available.
The `timeout` setting (default 120 seconds) applies **per turn**, not as a total
budget across turns — each of the up to 5 turns gets a fresh budget, so a slow
earlier turn doesn't starve later ones. If a turn's budget expires, the judge
attempts to parse whatever partial response is available.
---
+7 -4
View File
@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
[project]
name = "turnstone"
version = "1.6.6"
version = "1.6.9"
description = "Multi-node AI orchestration platform with tool use, agent routing, and cluster simulation."
readme = "README.md"
license = "Apache-2.0"
@@ -27,7 +27,7 @@ dependencies = [
"anthropic>=0.108", # claude-fable-5 support; hard runtime floor is 0.105 (mid-conversation system blocks)
"httpx>=0.28",
"mcp>=1.27,<2", # v2 is a breaking rewrite (2.0.0a1 live 2026-06-11; stable ~2026-07-27) — streamablehttp_client removed, 2-tuple transport, snake_case types; migrate deliberately
"starlette>=1.0.1", # PYSEC-2026-161: host-header path-injection in URL reconstruction (auth-bypass on apps comparing reconstructed URL paths)
"starlette>=1.3.1", # CVE-2026-54282 (path->authority host spoof) + CVE-2026-54283 (url-encoded form DoS); supersedes the PYSEC-2026-161 host-header path-injection floor
"uvicorn>=0.34",
"sse-starlette>=2.0",
"httpx-sse>=0.4",
@@ -39,7 +39,7 @@ dependencies = [
"structlog>=24.1",
"PyJWT>=2.8",
"bcrypt>=4.0",
"cryptography>=42",
"cryptography>=48.0.1", # GHSA-537c-gmf6-5ccf: PyPI wheels <48.0.1 bundle a vulnerable statically-linked OpenSSL (2026-06-09 secadv)
"lacme>=1.0.5",
"python-frontmatter>=1.0",
"pypdfium2>=4", # PDF text-extract + rasterize for models without native PDF input (core/pdf.py)
@@ -95,7 +95,10 @@ include = [
[tool.pytest.ini_options]
testpaths = ["tests"]
markers = ["live: requires a running LLM backend"]
markers = [
"live: requires a running LLM backend",
"allow_thread_leak: test intentionally leaves a background thread running (opts out of the leaked-thread guard)",
]
filterwarnings = [
# mcp v1 deprecates streamablehttp_client for an entry point whose call
# shape only settles in v2 — adoption rides the deliberate v2 migration
+101
View File
@@ -1,17 +1,118 @@
from __future__ import annotations
import asyncio
import contextlib
import logging
import os
import threading
import time
from typing import TYPE_CHECKING, Any
from unittest.mock import MagicMock
import pytest
def stop_loop_thread(loop: asyncio.AbstractEventLoop, thread: threading.Thread) -> None:
"""Fully tear down a ``loop.run_forever``-in-a-thread test loop.
Shuts the loop's default executor down ON the loop (joining its worker
threads — the ``asyncio_N`` threads that otherwise leak past the test),
then stops the loop, joins the thread, and closes the loop. Use in the
``finally`` of a background-loop fixture so nothing outlives the test.
"""
with contextlib.suppress(Exception):
asyncio.run_coroutine_threadsafe(loop.shutdown_default_executor(), loop).result(timeout=5)
loop.call_soon_threadsafe(loop.stop)
thread.join(timeout=5)
with contextlib.suppress(Exception):
loop.close()
def serve_until_exit(server: Any) -> None:
"""Run a uvicorn ``Server`` on a fresh event loop until it exits.
The thread target for an in-thread test upstream: when ``server.serve()``
returns (the fixture set ``server.should_exit`` / ``force_exit``), the loop
is closed so it doesn't leak past the fixture.
"""
loop = asyncio.new_event_loop()
asyncio.set_event_loop(loop)
try:
loop.run_until_complete(server.serve())
finally:
# Cancel + drain anything the app left pending (e.g. sse_starlette's
# shutdown watcher) so loop.close() doesn't warn "Task was destroyed
# but it is pending".
pending = asyncio.all_tasks(loop)
for task in pending:
task.cancel()
if pending:
with contextlib.suppress(Exception):
loop.run_until_complete(asyncio.gather(*pending, return_exceptions=True))
loop.close()
if TYPE_CHECKING:
from collections.abc import Iterator
from turnstone.core.mcp_client import MCPClientManager, StaticServerState
from turnstone.core.mcp_crypto import MCPTokenCipher
from turnstone.core.oidc import OIDCConfig
# A background daemon (e.g. title generation) can log into pytest's per-test
# capture as it is torn down — a benign "I/O operation on closed file" handler
# error. Don't let the logging module turn that race into noisy stderr
# tracebacks. (Process-global, test-only — product runtime keeps the default.)
logging.raiseExceptions = False
# Threads a test leaves running after teardown bleed into LATER tests' captured
# output (the "I/O operation on closed file" heisenbug) and, worse, can wedge
# the whole run (a leaked event loop / server that never stops). This grace
# lets a legitimately-finishing quick daemon settle before we judge a leak.
_THREAD_LEAK_GRACE = 5.0
@pytest.fixture(autouse=True)
def _no_leaked_threads(request: pytest.FixtureRequest) -> Iterator[None]:
"""Fail a test that leaves a background thread running past teardown.
Snapshots the live threads at setup; at teardown, gives any NEW thread a
short grace to finish, then fails listing those still alive — so a leak is
caught here instead of as a heisenbug days later. Opt out with
``@pytest.mark.allow_thread_leak`` (e.g. module-scoped servers in the live
suite).
"""
if request.node.get_closest_marker("allow_thread_leak"):
yield
return
# Snapshot the Thread OBJECTS, not their idents: Thread.ident is recycled
# after a thread exits, so an ident-based snapshot could mistake a new
# leaked thread (reusing an exited thread's ident) for a pre-existing one.
before = set(threading.enumerate())
yield
main = threading.main_thread()
current = threading.current_thread()
# One deadline shared across all joined threads — a deliberate TOTAL
# teardown budget (not per-thread), so a pathological test can't stall
# teardown by N×grace. A genuine never-stopping leak exhausts it and fails.
deadline = time.monotonic() + _THREAD_LEAK_GRACE
leaked = []
for t in threading.enumerate():
if t in before or t is main or t is current or not t.is_alive():
continue
t.join(timeout=max(0.0, deadline - time.monotonic()))
if t.is_alive():
leaked.append(t.name)
if leaked:
pytest.fail(
f"test left background threads running after teardown: {leaked}. "
"Stop them in teardown (shut down servers / close event loops / join "
"threads), or mark @pytest.mark.allow_thread_leak if intentional."
)
def make_mcp_token_cipher() -> MCPTokenCipher:
"""Build a single-key MCP token cipher for tests.
+202 -5
View File
@@ -7,6 +7,7 @@ helper code runs end-to-end without a network call.
from __future__ import annotations
import shutil
from unittest.mock import MagicMock
import pytest
@@ -18,11 +19,16 @@ class _Cfg:
"""Stand-in for ModelConfig — only the fields audio.py reads."""
def __init__(
self, model: str, capabilities: dict | None = None, provider: str = "openai"
self,
model: str,
capabilities: dict | None = None,
provider: str = "openai",
server_compat: dict | None = None,
) -> None:
self.model = model
self.capabilities = capabilities or {}
self.provider = provider
self.server_compat = server_compat or {}
class _FakeConfigStore:
@@ -191,7 +197,8 @@ class TestTranscribe:
with pytest.raises(audio.AudioBackendError):
audio.transcribe(registry=reg, alias="voice", data=b"x", filename="a.wav")
def test_omni_model_transcribes_via_chat(self):
def test_omni_model_transcribes_via_chat(self, monkeypatch):
monkeypatch.setattr(audio, "_to_wav_16k_mono", lambda data: data)
client = MagicMock()
msg = MagicMock(content=" the transcript ")
client.chat.completions.create.return_value = MagicMock(choices=[MagicMock(message=msg)])
@@ -202,15 +209,18 @@ class TestTranscribe:
assert res.transcript == "the transcript"
# The dedicated transcription endpoint is NOT used for an omni model.
client.audio.transcriptions.create.assert_not_called()
# Audio rides as an input_audio chat part; format comes from the filename.
parts = client.chat.completions.create.call_args.kwargs["messages"][0]["content"]
# Prompt precedes the audio part — the order Gemma documents for transcription.
assert [p["type"] for p in parts] == ["text", "input_audio"]
# The clip is transcoded to wav regardless of the upload container.
audio_part = next(p for p in parts if p["type"] == "input_audio")
assert audio_part["input_audio"]["format"] == "webm"
assert audio_part["input_audio"]["format"] == "wav"
# A blank prompt falls back to the omni STT default instruction.
text_part = next(p for p in parts if p["type"] == "text")
assert "Only output the transcription" in text_part["text"]
def test_omni_prompt_override_used(self):
def test_omni_prompt_override_used(self, monkeypatch):
monkeypatch.setattr(audio, "_to_wav_16k_mono", lambda data: data)
client = MagicMock()
client.chat.completions.create.return_value = MagicMock(
choices=[MagicMock(message=MagicMock(content="x"))]
@@ -338,3 +348,190 @@ class TestTranscribeCached:
assert audio.transcribe_cached(**kw) == ""
audio.transcribe_cached(**kw)
assert len(calls) == 2 # failure not cached -> retried
# ---------------------------------------------------------------------------
# Omni chat request shaping — transcode + thinking-off + token cap
# ---------------------------------------------------------------------------
class TestOmniChatExtraBody:
"""``_omni_chat_extra_body`` re-applies what the raw-client STT path skips."""
_THINKING = {"thinking_mode": "manual", "thinking_param": "enable_thinking"}
def test_disables_thinking_via_model_param(self):
cfg = _Cfg("gemma", dict(self._THINKING))
assert audio._omni_chat_extra_body(cfg) == {
"chat_template_kwargs": {"enable_thinking": False}
}
def test_thinking_off_wins_over_operator_flag(self):
cfg = _Cfg(
"gemma",
dict(self._THINKING),
server_compat={"extra_body": {"chat_template_kwargs": {"enable_thinking": True}}},
)
# STT never wants reasoning, even if an operator stored thinking on.
assert audio._omni_chat_extra_body(cfg)["chat_template_kwargs"]["enable_thinking"] is False
def test_forwards_operator_server_compat_extra_body(self):
cfg = _Cfg(
"model",
dict(self._THINKING),
server_compat={"extra_body": {"reasoning_format": "auto"}},
)
extra = audio._omni_chat_extra_body(cfg)
assert extra["reasoning_format"] == "auto"
assert extra["chat_template_kwargs"] == {"enable_thinking": False}
def test_empty_for_non_thinking_model(self):
cfg = _Cfg("omni", {"supports_audio_input": True})
assert audio._omni_chat_extra_body(cfg) == {}
class TestOmniChatCall:
"""The omni chat call carries the thinking-off extra_body and a token cap."""
def test_sends_thinking_off_and_token_cap(self, monkeypatch):
monkeypatch.setattr(audio, "_to_wav_16k_mono", lambda data: data)
client = MagicMock()
client.chat.completions.create.return_value = MagicMock(
choices=[MagicMock(message=MagicMock(content="hi"))]
)
cfg = _Cfg(
"gemma-omni",
{
"supports_audio_input": True,
"thinking_mode": "manual",
"thinking_param": "enable_thinking",
},
)
audio.transcribe(
registry=_FakeRegistry("omni", cfg, client),
alias="omni",
data=b"webmbytes",
filename="speech.webm",
)
kwargs = client.chat.completions.create.call_args.kwargs
assert kwargs["extra_body"]["chat_template_kwargs"]["enable_thinking"] is False
assert kwargs["max_tokens"] == audio._OMNI_STT_MAX_TOKENS
class TestTranscode:
"""``_to_wav_16k_mono`` normalizes any container to 16 kHz mono WAV via ffmpeg."""
def _stereo_wav_44k(self) -> bytes:
import io
import wave
buf = io.BytesIO()
with wave.open(buf, "wb") as w:
w.setnchannels(2)
w.setsampwidth(2)
w.setframerate(44100)
w.writeframes(b"\x00\x01\x00\x01" * 4410) # 0.1 s of stereo
return buf.getvalue()
@pytest.mark.skipif(shutil.which("ffmpeg") is None, reason="ffmpeg not installed")
def test_transcodes_to_16k_mono(self):
import io
import wave
out = audio._to_wav_16k_mono(self._stereo_wav_44k())
with wave.open(io.BytesIO(out), "rb") as w:
assert w.getnchannels() == 1
assert w.getframerate() == 16000
@pytest.mark.skipif(shutil.which("ffmpeg") is None, reason="ffmpeg not installed")
def test_undecodable_bytes_raise_backend_error(self):
with pytest.raises(audio.AudioBackendError):
audio._to_wav_16k_mono(b"this is not audio at all")
def test_missing_ffmpeg_raises_backend_error(self, monkeypatch):
def _no_ffmpeg(*a, **k):
raise FileNotFoundError("ffmpeg")
monkeypatch.setattr(audio.subprocess, "run", _no_ffmpeg)
with pytest.raises(audio.AudioBackendError, match="ffmpeg is not installed"):
audio._to_wav_16k_mono(b"x")
def test_invokes_ffmpeg_with_hardened_argv(self, monkeypatch):
# Covers the argv shaping even on a CI image without ffmpeg installed.
captured = {}
def _fake_run(cmd, **kwargs):
captured["cmd"] = cmd
captured["input"] = kwargs.get("input")
return MagicMock(returncode=0, stdout=b"RIFF....WAVE", stderr=b"")
monkeypatch.setattr(audio.subprocess, "run", _fake_run)
assert audio._to_wav_16k_mono(b"rawclip") == b"RIFF....WAVE"
cmd = captured["cmd"]
assert cmd[0] == "ffmpeg"
assert captured["input"] == b"rawclip"
# SSRF/decompression-bomb hardening + the 16 kHz mono normalization.
assert cmd[cmd.index("-protocol_whitelist") + 1] == "pipe"
assert "-vn" in cmd
assert cmd[cmd.index("-ac") + 1] == "1"
assert cmd[cmd.index("-ar") + 1] == "16000"
assert cmd[cmd.index("-f") + 1] == "wav"
def test_nonzero_returncode_raises_backend_error(self, monkeypatch):
monkeypatch.setattr(
audio.subprocess,
"run",
lambda *a, **k: MagicMock(returncode=1, stdout=b"", stderr=b"boom"),
)
with pytest.raises(audio.AudioBackendError, match="Audio transcode failed"):
audio._to_wav_16k_mono(b"x")
def _stream_chunk(content):
return MagicMock(choices=[MagicMock(delta=MagicMock(content=content))])
class TestTranscribeStream:
"""``transcribe_stream`` yields content deltas; resolve/transcode are eager."""
def test_streams_chat_deltas_with_thinking_off(self, monkeypatch):
monkeypatch.setattr(audio, "_to_wav_16k_mono", lambda data: data)
client = MagicMock()
client.chat.completions.create.return_value = iter(
[_stream_chunk("and so"), _stream_chunk(None), _stream_chunk(" my fellow americans")]
)
cfg = _Cfg(
"gemma-omni",
{
"supports_audio_input": True,
"thinking_mode": "manual",
"thinking_param": "enable_thinking",
},
)
gen = audio.transcribe_stream(
registry=_FakeRegistry("omni", cfg, client), alias="omni", data=b"webmbytes"
)
# Empty/None deltas are skipped; the rest stream through in order.
assert list(gen) == ["and so", " my fellow americans"]
kwargs = client.chat.completions.create.call_args.kwargs
assert kwargs["stream"] is True
assert kwargs["extra_body"]["chat_template_kwargs"]["enable_thinking"] is False
def test_non_audio_provider_raises_before_streaming(self):
client = MagicMock()
cfg = _Cfg("gemma", {"supports_audio_input": True}, provider="anthropic-compatible")
with pytest.raises(audio.AudioUnavailableError, match="OpenAI-compatible provider"):
audio.transcribe_stream(
registry=_FakeRegistry("omni", cfg, client), alias="omni", data=b"x"
)
client.chat.completions.create.assert_not_called()
def test_whisper_alias_emits_single_chunk(self):
client = MagicMock()
client.audio.transcriptions.create.return_value = MagicMock(text=" full transcript ")
cfg = _Cfg("whisper-1") # name inference -> dedicated endpoint, no chat stream
gen = audio.transcribe_stream(
registry=_FakeRegistry("w", cfg, client), alias="w", data=b"x"
)
assert list(gen) == ["full transcript"]
client.chat.completions.create.assert_not_called()
+53
View File
@@ -85,6 +85,59 @@ def test_emit_created_calls_collector_with_coord_fields() -> None:
)
def test_emit_created_seeds_resolved_display_name(tmp_path: Any) -> None:
"""The collector seed uses the resolved display name (alias > title >
name), not the synthetic ``ws.name``. A coordinator carrying a
persisted LLM auto-title (written by ``update_workstream_title``) then
shows that title in the live cluster tree instead of reverting to
``ws-xxxx``. Regression guard for the adapter half of the
coordinator-title-persistence fix — the server-side ``_coordinator_rows``
half is pinned in test_coordinator_endpoints.py."""
from turnstone.core.storage import init_storage, reset_storage
reset_storage()
backend = init_storage("sqlite", path=str(tmp_path / "adapter.db"), run_migrations=False)
try:
# Titled coordinator → the title surfaces over the placeholder name.
backend.register_workstream(
"coord-1",
node_id="console",
user_id="u1",
name="ws-c0c0",
kind=WorkstreamKind.COORDINATOR,
)
backend.update_workstream_title("coord-1", "Investigate the title bug")
adapter, collector = _make_adapter()
adapter.emit_created(_make_ws(name="ws-c0c0"))
assert (
collector.emit_console_ws_created.call_args.kwargs["name"]
== "Investigate the title bug"
)
# A user alias outranks the auto-title (alias > title > name).
assert backend.set_workstream_alias("coord-1", "Pinned name")
collector.emit_console_ws_created.reset_mock()
adapter._fanout_console_ws_created(_make_ws(name="ws-c0c0"))
assert collector.emit_console_ws_created.call_args.kwargs["name"] == "Pinned name"
finally:
reset_storage()
def test_coord_display_name_skips_uninitialized_storage() -> None:
"""_coord_display_name runs on a lifecycle-event path and must NOT trip
get_storage()'s SQLite auto-init (a stray .turnstone.db in the CWD) when
storage isn't initialized — it falls back to the placeholder ws.name and
leaves storage untouched."""
from turnstone.console.coordinator_adapter import _coord_display_name
from turnstone.core.storage import is_storage_initialized, reset_storage
reset_storage()
assert not is_storage_initialized()
assert _coord_display_name(_make_ws(name="ws-abcd")) == "ws-abcd"
# The resolution did not auto-initialize storage as a side effect.
assert not is_storage_initialized()
def test_emit_state_calls_collector_state() -> None:
"""Post-rich-payload, emit_state passes tokens / context_ratio /
activity / activity_state / content kwargs read from ws.ui's
+168
View File
@@ -65,8 +65,10 @@ from turnstone.core.session_routes import (
make_history_handler,
make_list_handler,
make_open_handler,
make_refresh_title_handler,
make_saved_handler,
make_send_handler,
make_set_title_handler,
)
from turnstone.core.workstream import WorkstreamKind
@@ -204,6 +206,16 @@ def _make_client(
),
methods=["POST"],
),
Route(
"/v1/api/workstreams/{ws_id}/refresh-title",
make_refresh_title_handler(_coord_endpoint_config),
methods=["POST"],
),
Route(
"/v1/api/workstreams/{ws_id}/title",
make_set_title_handler(_coord_endpoint_config),
methods=["POST"],
),
Route(
"/v1/api/workstreams/{ws_id}/history",
make_history_handler(_coord_endpoint_config),
@@ -370,6 +382,114 @@ def test_unresolvable_alias_returns_503(storage):
_COORD_HEADERS = {"X-Test-User": "user-1", "X-Test-Perms": "admin.coordinator"}
# ---------------------------------------------------------------------------
# Title verbs — refresh-title (LLM regenerate) + set title (manual alias),
# ported to coordinators via the lifted make_refresh_title_handler /
# make_set_title_handler factories so both kinds share one body.
# ---------------------------------------------------------------------------
def test_coord_refresh_title_triggers_regeneration(storage):
mgr = _build_mgr(storage)
ws = mgr.create(user_id="user-1", name="c1")
client = _make_client(storage, coord_mgr=mgr, registry=_fake_registry())
resp = client.post(f"/v1/api/workstreams/{ws.id}/refresh-title", headers=_COORD_HEADERS)
assert resp.status_code == 200
# The lifted handler resolves the current display name and asks the
# live session to regenerate a (different) title in the background.
ws.session.request_title_refresh.assert_called_once_with("c1")
def test_coord_refresh_title_requires_operator_permission(storage):
mgr = _build_mgr(storage)
ws = mgr.create(user_id="user-1", name="c1")
client = _make_client(storage, coord_mgr=mgr, registry=_fake_registry())
resp = client.post(
f"/v1/api/workstreams/{ws.id}/refresh-title",
headers={"X-Test-User": "user-1", "X-Test-Perms": "read"},
)
assert resp.status_code == 403
ws.session.request_title_refresh.assert_not_called()
def test_coord_refresh_title_unknown_ws_404(storage):
mgr = _build_mgr(storage)
client = _make_client(storage, coord_mgr=mgr, registry=_fake_registry())
resp = client.post(
"/v1/api/workstreams/" + ("0" * 32) + "/refresh-title", headers=_COORD_HEADERS
)
assert resp.status_code == 404
def test_coord_set_title_stores_alias_and_broadcasts(storage):
from turnstone.core.memory import get_workstream_display_name
mgr = _build_mgr(storage)
ws = mgr.create(user_id="user-1", name="c1")
client = _make_client(storage, coord_mgr=mgr, registry=_fake_registry())
resp = client.post(
f"/v1/api/workstreams/{ws.id}/title",
json={"title": "Nightly migration sweep"},
headers=_COORD_HEADERS,
)
assert resp.status_code == 200
assert resp.json()["title"] == "Nightly migration sweep"
# Stored as the alias (outranks the auto-title) ...
assert get_workstream_display_name(ws.id) == "Nightly migration sweep"
# ... and broadcast live to the dashboard via the session UI.
ws.session.ui.on_rename.assert_called_once_with("Nightly migration sweep")
def test_coord_set_title_empty_400(storage):
mgr = _build_mgr(storage)
ws = mgr.create(user_id="user-1", name="c1")
client = _make_client(storage, coord_mgr=mgr, registry=_fake_registry())
resp = client.post(
f"/v1/api/workstreams/{ws.id}/title", json={"title": " "}, headers=_COORD_HEADERS
)
assert resp.status_code == 400
def test_coord_set_title_alias_conflict_409(storage):
mgr = _build_mgr(storage)
first = mgr.create(user_id="user-1", name="c1")
second = mgr.create(user_id="user-1", name="c2")
storage.set_workstream_alias(first.id, "taken")
client = _make_client(storage, coord_mgr=mgr, registry=_fake_registry())
resp = client.post(
f"/v1/api/workstreams/{second.id}/title", json={"title": "taken"}, headers=_COORD_HEADERS
)
assert resp.status_code == 409
def test_coord_set_title_rejects_unowned_ws_404(storage):
"""An admin.coordinator operator can't rename a workstream the coord
manager doesn't own (here a cross-kind interactive row) via the coord
/title route: set_workstream_alias is a global kind-unscoped UPDATE, so
the handler 404s on the in-memory coord lookup BEFORE writing — no
silent 200, no cross-kind alias write."""
from turnstone.core.memory import get_workstream_display_name
mgr = _build_mgr(storage)
# An interactive-kind row in storage, NOT held by coord_mgr.
storage.register_workstream(
"i" * 32,
node_id="node-1",
user_id="user-1",
name="interactive-ws",
kind=WorkstreamKind.INTERACTIVE,
)
client = _make_client(storage, coord_mgr=mgr, registry=_fake_registry())
resp = client.post(
f"/v1/api/workstreams/{'i' * 32}/title",
json={"title": "hijacked"},
headers=_COORD_HEADERS,
)
assert resp.status_code == 404
# The interactive ws's display name is untouched — the alias write never fired.
assert get_workstream_display_name("i" * 32) == "interactive-ws"
def test_active_list_row_shape_includes_unified_fields(storage):
"""Stage 2 list-verb-lift parity regression — coord active-list row
carries the always-include fields (ws_id, name, state, kind,
@@ -2433,6 +2553,54 @@ def test_coordinator_rows_persisted_cluster_wide(storage):
assert {r["name"] for r in rows} == {"alice-closed", "bob-closed", "orphan-closed"}
def test_coordinator_rows_surface_persisted_title(storage):
"""Regression for the coordinator-title-persistence bug.
The LLM auto-title (``update_workstream_title``) and the user alias
(``set_workstream_alias``) live only in ``workstreams.title`` /
``workstreams.alias``. ``_coordinator_rows`` must resolve the
display name ``alias > title > name`` from the persisted row for BOTH
lanes — the in-memory ``ws.name`` is the synthetic ``ws-xxxx``
placeholder. Before the fix the read path hardcoded ``title=""`` and
used ``ws.name`` / the ``name`` column, so a generated title was
written but never read back: it reverted to ``ws-xxxx`` on every
dashboard refresh."""
from turnstone.console.server import _coordinator_rows
from turnstone.core.workstream import WorkstreamKind
mgr = _build_mgr(storage)
# In-memory lane: a LIVE coordinator titled after creation. The
# manager assigned the placeholder ``ws.name``; the title is in the DB.
live = mgr.create(user_id="alice", name="ws-abcd")
storage.update_workstream_title(live.id, "Refactor the auth layer")
# Persisted lane: a closed coordinator (evicted from the manager)
# carrying BOTH a title and a user alias — the alias must win.
storage.register_workstream(
"f" * 32,
node_id="console",
user_id="bob",
name="ws-f0f0",
state="closed",
kind=WorkstreamKind.COORDINATOR,
parent_ws_id=None,
)
storage.update_workstream_title("f" * 32, "auto-generated title")
assert storage.set_workstream_alias("f" * 32, "Bob's pinned name")
request = _persisted_rows_request(storage, mgr, "alice", frozenset({"read"}))
rows = {r["id"]: r for r in _coordinator_rows(request)}
live_row = rows[live.id]
assert live_row["name"] == "Refactor the auth layer"
assert live_row["title"] == "Refactor the auth layer"
closed_row = rows["f" * 32]
assert closed_row["name"] == "Bob's pinned name" # alias > title > name
assert closed_row["title"] == "auto-generated title"
# ---------------------------------------------------------------------------
# Stage 2 P1.5 — coord attachment surface parity with interactive
# ---------------------------------------------------------------------------
+66
View File
@@ -0,0 +1,66 @@
"""Tests for turnstone.core.deadline.run_with_deadline.
The load-bearing property is the daemon worker: on timeout or cancel the call
is abandoned, and the abandoned thread must be a daemon so it can never block
interpreter exit (the bug that motivated the helper a non-daemon
ThreadPoolExecutor worker is joined by concurrent.futures' atexit hook).
"""
from __future__ import annotations
import threading
import time
import pytest
from turnstone.core.deadline import (
DeadlineCancelledError,
DeadlineExceededError,
run_with_deadline,
)
def test_returns_result_on_success() -> None:
assert run_with_deadline(lambda: 42, timeout=1.0) == 42
def test_reraises_callable_exception() -> None:
def boom() -> None:
raise ValueError("upstream failed")
with pytest.raises(ValueError, match="upstream failed"):
run_with_deadline(boom, timeout=1.0)
def test_timeout_returns_promptly_and_abandons_a_daemon_worker() -> None:
# The worker sleeps far past the deadline; the call must return promptly
# via DeadlineExceededError, and the abandoned worker must be a daemon so
# it cannot pin interpreter exit.
start = time.monotonic()
with pytest.raises(DeadlineExceededError):
run_with_deadline(lambda: time.sleep(2.0), timeout=0.2, poll=0.05, thread_name="dl-timeout")
assert time.monotonic() - start < 1.0
stragglers = [t for t in threading.enumerate() if t.name == "dl-timeout" and not t.daemon]
assert stragglers == [], f"non-daemon worker survived: {stragglers}"
def test_cancel_returns_promptly() -> None:
cancel = threading.Event()
def _fire() -> None:
time.sleep(0.1)
cancel.set()
threading.Thread(target=_fire, daemon=True).start()
start = time.monotonic()
with pytest.raises(DeadlineCancelledError):
run_with_deadline(
lambda: time.sleep(2.0),
timeout=10.0,
cancel_event=cancel,
poll=0.05,
thread_name="dl-cancel",
)
assert time.monotonic() - start < 1.0
stragglers = [t for t in threading.enumerate() if t.name == "dl-cancel" and not t.daemon]
assert stragglers == [], f"non-daemon worker survived: {stragglers}"
+40 -21
View File
@@ -52,13 +52,32 @@ class _Handler(http.server.BaseHTTPRequestHandler):
pass
def _serve(handler_cls, ssl_context: ssl.SSLContext | None = None) -> int:
"""Start a daemon-thread HTTP(S) server on an ephemeral port."""
httpd = http.server.HTTPServer(("127.0.0.1", 0), handler_cls)
if ssl_context is not None:
httpd.socket = ssl_context.wrap_socket(httpd.socket, server_side=True)
threading.Thread(target=httpd.serve_forever, daemon=True).start()
return httpd.server_address[1]
@pytest.fixture
def serve():
"""Factory that starts an HTTP(S) server on an ephemeral port and returns
that port.
Every server it starts is shut down + its serve_forever thread joined at
teardown, so the thread never outlives the test (which would otherwise bleed
into a later test's captured output / leak the listener).
"""
started: list[tuple[http.server.HTTPServer, threading.Thread]] = []
def _factory(handler_cls, ssl_context: ssl.SSLContext | None = None) -> int:
httpd = http.server.HTTPServer(("127.0.0.1", 0), handler_cls)
if ssl_context is not None:
httpd.socket = ssl_context.wrap_socket(httpd.socket, server_side=True)
thread = threading.Thread(target=httpd.serve_forever, daemon=True)
thread.start()
started.append((httpd, thread))
return httpd.server_address[1]
yield _factory
for httpd, thread in started:
httpd.shutdown() # break the serve_forever loop
httpd.server_close() # release the listening socket
thread.join(timeout=5)
@pytest.fixture
@@ -90,29 +109,29 @@ def mtls_setup(tmp_path):
# ── Plain HTTP (mTLS disabled — the default deployment) ─────────────────────
def test_plain_http_ok():
def test_plain_http_ok(serve):
"""Default path: plain probe succeeds, PEM dir never consulted."""
port = _serve(_Handler)
port = serve(_Handler)
result = run_healthcheck(f"http://127.0.0.1:{port}/health")
assert result.returncode == 0, result.stderr
def test_plain_http_degraded_is_healthy():
def test_plain_http_degraded_is_healthy(serve):
"""'degraded' (backend down, server up) still counts as container-healthy."""
class Degraded(_Handler):
payload = {"status": "degraded"}
port = _serve(Degraded)
port = serve(Degraded)
result = run_healthcheck(f"http://127.0.0.1:{port}/health")
assert result.returncode == 0, result.stderr
def test_plain_http_bad_status_fails():
def test_plain_http_bad_status_fails(serve):
class Bad(_Handler):
payload = {"status": "error"}
port = _serve(Bad)
port = serve(Bad)
result = run_healthcheck(f"http://127.0.0.1:{port}/health")
assert result.returncode == 1
assert "unhealthy payload" in result.stderr
@@ -128,40 +147,40 @@ def test_server_down_fails():
# ── mTLS (tls.enabled) ───────────────────────────────────────────────────────
def test_mtls_probe_with_pem_dir(mtls_setup):
def test_mtls_probe_with_pem_dir(mtls_setup, serve):
"""The regression case: mTLS node + plain-HTTP probe URL.
The plain attempt is rejected at the socket; the script must fall back
to HTTPS with the node cert as client cert and report healthy."""
pem_root, server_ctx = mtls_setup
port = _serve(_Handler, ssl_context=server_ctx)
port = serve(_Handler, ssl_context=server_ctx)
result = run_healthcheck(f"http://127.0.0.1:{port}/health", pem_root=pem_root)
assert result.returncode == 0, result.stderr
def test_mtls_probe_without_pems_fails(mtls_setup):
def test_mtls_probe_without_pems_fails(mtls_setup, serve):
"""mTLS node but no PEM material on disk: the probe must fail."""
_, server_ctx = mtls_setup
port = _serve(_Handler, ssl_context=server_ctx)
port = serve(_Handler, ssl_context=server_ctx)
result = run_healthcheck(f"http://127.0.0.1:{port}/health", pem_root=None)
assert result.returncode == 1
assert "Health check failed" in result.stderr
def test_mtls_unhealthy_payload_fails(mtls_setup):
def test_mtls_unhealthy_payload_fails(mtls_setup, serve):
"""A reachable mTLS server with a bad payload is still unhealthy."""
pem_root, server_ctx = mtls_setup
class Bad(_Handler):
payload = {"status": "error"}
port = _serve(Bad, ssl_context=server_ctx)
port = serve(Bad, ssl_context=server_ctx)
result = run_healthcheck(f"http://127.0.0.1:{port}/health", pem_root=pem_root)
assert result.returncode == 1
assert "unhealthy payload" in result.stderr
def test_mtls_incomplete_pem_dir_fails(mtls_setup, tmp_path):
def test_mtls_incomplete_pem_dir_fails(mtls_setup, tmp_path, serve):
"""A PEM dir missing the key is skipped, not half-used."""
_, server_ctx = mtls_setup
incomplete = tmp_path / "incomplete-root"
@@ -170,7 +189,7 @@ def test_mtls_incomplete_pem_dir_fails(mtls_setup, tmp_path):
(d / "fullchain.pem").write_text("not a cert")
(d / "ca.pem").write_text("not a cert")
port = _serve(_Handler, ssl_context=server_ctx)
port = serve(_Handler, ssl_context=server_ctx)
result = run_healthcheck(f"http://127.0.0.1:{port}/health", pem_root=incomplete)
assert result.returncode == 1
+44 -61
View File
@@ -5,7 +5,6 @@ from __future__ import annotations
import json
import threading
import time
from concurrent.futures import ThreadPoolExecutor
from pathlib import Path
from typing import Any
from unittest.mock import MagicMock
@@ -120,8 +119,7 @@ class TestVerdictParsing:
[{"role": "user", "content": "Run echo hello"}],
callback_results.append,
)
# Wait for daemon thread
time.sleep(0.5)
_wait_for(callback_results, 1)
assert len(heuristics) == 1
assert heuristics[0].tier == "heuristic"
@@ -184,14 +182,12 @@ class TestErrorHandling:
provider = _make_mock_provider(side_effect=RuntimeError("API error"))
judge = _make_judge(provider)
with ThreadPoolExecutor(max_workers=1) as pool:
result = judge._evaluate_single(
_make_item(),
[{"role": "user", "content": "test"}],
cancel_event=None,
executor=pool,
client=MagicMock(),
)
result = judge._evaluate_single(
_make_item(),
[{"role": "user", "content": "test"}],
cancel_event=None,
client=MagicMock(),
)
assert result is None
def test_provider_error_heuristic_still_returned(self):
@@ -209,7 +205,7 @@ class TestErrorHandling:
[{"role": "user", "content": "test"}],
callback_results.append,
)
time.sleep(0.5)
_wait_for(callback_results, 1)
assert len(heuristics) == 1
assert heuristics[0].tier == "heuristic"
@@ -233,20 +229,19 @@ class TestErrorHandling:
[{"role": "user", "content": "test"}],
callback_results.append,
)
time.sleep(0.5)
_wait_for(callback_results, 1)
assert len(callback_results) == 1
assert callback_results[0].tier == "llm_fallback"
def test_executor_poison_delivers_fallback(self):
"""An _ExecutorPoisonedError (a judge-call timeout poisoning the
single-worker executor) restarts the executor AND still delivers one
fallback for the interrupted item the twin of the generic-exception
path, and load-bearing for Smart Approvals' batch-completeness wait."""
from turnstone.core.judge import _ExecutorPoisonedError
def test_evaluate_single_none_delivers_fallback(self):
"""A judge-call timeout now surfaces as ``_evaluate_single`` returning
None (the executor-poison restart dance is gone); the daemon must still
deliver exactly one fallback for that item Smart Approvals waits on
the full verdict set before gating, so a silently-skipped item would
block that wait until its timeout."""
judge = _make_judge()
judge._evaluate_single = MagicMock( # type: ignore[method-assign]
side_effect=_ExecutorPoisonedError()
return_value=None
)
callback_results: list[IntentVerdict] = []
judge.evaluate(
@@ -254,7 +249,7 @@ class TestErrorHandling:
[{"role": "user", "content": "test"}],
callback_results.append,
)
time.sleep(0.5)
_wait_for(callback_results, 1)
assert len(callback_results) == 1
assert callback_results[0].tier == "llm_fallback"
@@ -266,14 +261,12 @@ class TestErrorHandling:
result_mock.content = ""
judge = _make_judge(provider)
with ThreadPoolExecutor(max_workers=1) as pool:
result = judge._evaluate_single(
_make_item(),
[{"role": "user", "content": "test"}],
cancel_event=None,
executor=pool,
client=MagicMock(),
)
result = judge._evaluate_single(
_make_item(),
[{"role": "user", "content": "test"}],
cancel_event=None,
client=MagicMock(),
)
assert result is None
def test_empty_content_length_stop_no_retry(self):
@@ -285,14 +278,12 @@ class TestErrorHandling:
result_mock.finish_reason = "length"
judge = _make_judge(provider)
with ThreadPoolExecutor(max_workers=1) as pool:
result = judge._evaluate_single(
_make_item(),
[{"role": "user", "content": "test"}],
cancel_event=None,
executor=pool,
client=MagicMock(),
)
result = judge._evaluate_single(
_make_item(),
[{"role": "user", "content": "test"}],
cancel_event=None,
client=MagicMock(),
)
assert result is None
# Should have been called exactly once — no retries
assert provider.create_completion.call_count == 1
@@ -404,14 +395,12 @@ class TestMultiTurnToolUse:
provider.create_completion.side_effect = [turn1, turn2]
judge = _make_judge(provider)
with ThreadPoolExecutor(max_workers=1) as pool:
verdict = judge._evaluate_single(
_make_item(),
[{"role": "user", "content": "test"}],
cancel_event=None,
executor=pool,
client=MagicMock(),
)
verdict = judge._evaluate_single(
_make_item(),
[{"role": "user", "content": "test"}],
cancel_event=None,
client=MagicMock(),
)
assert verdict is not None
assert verdict.tier == "llm"
assert provider.create_completion.call_count == 2
@@ -454,14 +443,12 @@ class TestMultiTurnToolUse:
]
judge = _make_judge(provider)
with ThreadPoolExecutor(max_workers=1) as pool:
judge._evaluate_single(
_make_item(),
[{"role": "user", "content": "test"}],
cancel_event=None,
executor=pool,
client=MagicMock(),
)
judge._evaluate_single(
_make_item(),
[{"role": "user", "content": "test"}],
cancel_event=None,
client=MagicMock(),
)
# Should have called create_completion exactly _JUDGE_MAX_TURNS times
assert provider.create_completion.call_count == 5
@@ -507,7 +494,7 @@ class TestConfidenceArbitration:
[{"role": "user", "content": "Run echo hello"}],
callback_results.append,
)
time.sleep(0.5)
_wait_for(callback_results, 1)
assert len(heuristics) == 1
assert heuristics[0].confidence == 0.85
@@ -527,7 +514,7 @@ class TestConfidenceArbitration:
[{"role": "user", "content": "Run echo hello"}],
callback_results.append,
)
time.sleep(0.5)
_wait_for(callback_results, 1)
assert len(heuristics) == 1
# LLM verdict is always delivered regardless of confidence comparison
@@ -966,11 +953,7 @@ class TestModelAliasResolution:
[{"role": "user", "content": "delegate the audit"}],
callback_results.append,
)
# Wait for daemon thread.
for _ in range(20):
if callback_results:
break
time.sleep(0.1)
_wait_for(callback_results, 1)
assert callback_results, "judge never delivered a verdict"
assert callback_results[0].tier == "llm"
+15 -7
View File
@@ -38,7 +38,7 @@ import uvicorn
from mcp.server.fastmcp import FastMCP
from starlette.middleware.base import BaseHTTPMiddleware
from tests.conftest import make_mcp_token_cipher
from tests.conftest import make_mcp_token_cipher, serve_until_exit, stop_loop_thread
from turnstone.core.mcp_client import MCPClientManager
from turnstone.core.mcp_crypto import MCPTokenStore
from turnstone.core.mcp_oauth import TokenLookupResult
@@ -187,7 +187,14 @@ def _build_server(port: int, behaviour: dict[str, Any]) -> uvicorn.Server:
app = mcp.streamable_http_app()
app.add_middleware(BehaviorMiddleware, behaviour=behaviour)
config = uvicorn.Config(app, host="127.0.0.1", port=port, log_level="warning", access_log=False)
config = uvicorn.Config(
app,
host="127.0.0.1",
port=port,
log_level="warning",
access_log=False,
timeout_graceful_shutdown=0, # don't block teardown on a held-open stream
)
return uvicorn.Server(config)
@@ -215,9 +222,7 @@ def upstream():
server = _build_server(port, behaviour)
def _run() -> None:
loop = asyncio.new_event_loop()
asyncio.set_event_loop(loop)
loop.run_until_complete(server.serve())
serve_until_exit(server)
t = threading.Thread(target=_run, daemon=True, name="phase6-upstream")
t.start()
@@ -225,7 +230,11 @@ def upstream():
_wait_ready(port)
yield f"http://127.0.0.1:{port}/mcp", behaviour
finally:
# should_exit alone triggers a GRACEFUL shutdown that can wait forever
# on a held-open streamable-http stream; force_exit skips that wait so
# serve() returns and the upstream thread doesn't leak past the test.
server.should_exit = True
server.force_exit = True
t.join(timeout=5)
@@ -311,8 +320,7 @@ def running_loop_mgr():
with contextlib.suppress(Exception):
asyncio.run_coroutine_threadsafe(_drain(mgr), loop).result(timeout=2)
loop.call_soon_threadsafe(loop.stop)
thread.join(timeout=2)
stop_loop_thread(loop, thread)
# ---------------------------------------------------------------------------
+2 -3
View File
@@ -34,7 +34,7 @@ from unittest.mock import AsyncMock, MagicMock
import httpx
import pytest
from tests.conftest import make_mcp_token_cipher
from tests.conftest import make_mcp_token_cipher, stop_loop_thread
from turnstone.core.mcp_client import (
MCPClientManager,
_AuthCapture,
@@ -131,8 +131,7 @@ def running_loop_mgr():
with contextlib.suppress(Exception):
asyncio.run_coroutine_threadsafe(_drain(mgr), loop).result(timeout=2)
loop.call_soon_threadsafe(loop.stop)
thread.join(timeout=2)
stop_loop_thread(loop, thread)
def _run_on_loop(loop: asyncio.AbstractEventLoop, coro: Any) -> Any:
+15 -7
View File
@@ -26,7 +26,7 @@ import uvicorn
from mcp.server.fastmcp import FastMCP
from starlette.middleware.base import BaseHTTPMiddleware
from tests.conftest import make_mcp_token_cipher
from tests.conftest import make_mcp_token_cipher, serve_until_exit, stop_loop_thread
from turnstone.core.mcp_client import MCPClientManager
from turnstone.core.mcp_crypto import MCPTokenStore
from turnstone.core.mcp_oauth import TokenLookupResult
@@ -130,7 +130,14 @@ def _build_server(port: int, behaviour: dict[str, Any]) -> uvicorn.Server:
app = mcp.streamable_http_app()
app.add_middleware(BehaviorMiddleware, behaviour=behaviour)
config = uvicorn.Config(app, host="127.0.0.1", port=port, log_level="warning", access_log=False)
config = uvicorn.Config(
app,
host="127.0.0.1",
port=port,
log_level="warning",
access_log=False,
timeout_graceful_shutdown=0, # don't block teardown on a held-open stream
)
return uvicorn.Server(config)
@@ -152,9 +159,7 @@ def upstream():
server = _build_server(port, behaviour)
def _run() -> None:
loop = asyncio.new_event_loop()
asyncio.set_event_loop(loop)
loop.run_until_complete(server.serve())
serve_until_exit(server)
t = threading.Thread(target=_run, daemon=True, name="phase7b-prompt-upstream")
t.start()
@@ -162,7 +167,11 @@ def upstream():
_wait_ready(port)
yield f"http://127.0.0.1:{port}/mcp", behaviour
finally:
# should_exit alone triggers a GRACEFUL shutdown that can wait forever
# on a held-open streamable-http stream; force_exit skips that wait so
# serve() returns and the upstream thread doesn't leak past the test.
server.should_exit = True
server.force_exit = True
t.join(timeout=5)
@@ -248,8 +257,7 @@ def running_loop_mgr():
with contextlib.suppress(Exception):
asyncio.run_coroutine_threadsafe(_drain(mgr), loop).result(timeout=2)
loop.call_soon_threadsafe(loop.stop)
thread.join(timeout=2)
stop_loop_thread(loop, thread)
def _seed_pool_prompt_map(
@@ -26,7 +26,7 @@ import uvicorn
from mcp.server.fastmcp import FastMCP
from starlette.middleware.base import BaseHTTPMiddleware
from tests.conftest import make_mcp_token_cipher
from tests.conftest import make_mcp_token_cipher, serve_until_exit, stop_loop_thread
from turnstone.core.mcp_client import MCPClientManager
from turnstone.core.mcp_crypto import MCPTokenStore
from turnstone.core.mcp_oauth import TokenLookupResult
@@ -139,7 +139,14 @@ def _build_server(port: int, behaviour: dict[str, Any]) -> uvicorn.Server:
app = mcp.streamable_http_app()
app.add_middleware(BehaviorMiddleware, behaviour=behaviour)
config = uvicorn.Config(app, host="127.0.0.1", port=port, log_level="warning", access_log=False)
config = uvicorn.Config(
app,
host="127.0.0.1",
port=port,
log_level="warning",
access_log=False,
timeout_graceful_shutdown=0, # don't block teardown on a held-open stream
)
return uvicorn.Server(config)
@@ -161,9 +168,7 @@ def upstream():
server = _build_server(port, behaviour)
def _run() -> None:
loop = asyncio.new_event_loop()
asyncio.set_event_loop(loop)
loop.run_until_complete(server.serve())
serve_until_exit(server)
t = threading.Thread(target=_run, daemon=True, name="phase7b-resource-upstream")
t.start()
@@ -171,7 +176,11 @@ def upstream():
_wait_ready(port)
yield f"http://127.0.0.1:{port}/mcp", behaviour
finally:
# should_exit alone triggers a GRACEFUL shutdown that can wait forever
# on a held-open streamable-http stream; force_exit skips that wait so
# serve() returns and the upstream thread doesn't leak past the test.
server.should_exit = True
server.force_exit = True
t.join(timeout=5)
@@ -257,8 +266,7 @@ def running_loop_mgr():
with contextlib.suppress(Exception):
asyncio.run_coroutine_threadsafe(_drain(mgr), loop).result(timeout=2)
loop.call_soon_threadsafe(loop.stop)
thread.join(timeout=2)
stop_loop_thread(loop, thread)
def _seed_pool_resource_map(
+2 -3
View File
@@ -25,7 +25,7 @@ from unittest.mock import AsyncMock, MagicMock
import pytest
from tests.conftest import make_mcp_token_cipher
from tests.conftest import make_mcp_token_cipher, stop_loop_thread
from turnstone.core.mcp_client import MCPClientManager, PoolEntryState
from turnstone.core.mcp_crypto import MCPTokenStore
from turnstone.core.storage._sqlite import SQLiteBackend
@@ -123,8 +123,7 @@ def running_loop_mgr():
with contextlib.suppress(Exception):
asyncio.run_coroutine_threadsafe(_drain(mgr), loop).result(timeout=2)
loop.call_soon_threadsafe(loop.stop)
thread.join(timeout=2)
stop_loop_thread(loop, thread)
def _run_on_loop(loop: asyncio.AbstractEventLoop, coro: Any) -> Any:
+20
View File
@@ -220,6 +220,26 @@ class TestEvaluateFailurePaths:
# Cancel should return promptly, well below the 10s timeout.
assert elapsed < 2.0, f"cancel returned in {elapsed:.2f}s, expected < 2.0s"
def test_timeout_leaves_no_nondaemon_straggler(self) -> None:
# Regression: evaluate() abandons a slow upstream call on timeout, but
# the worker must be a *daemon* so it can never pin interpreter exit.
# The old ThreadPoolExecutor worker was non-daemon and got joined by
# concurrent.futures' atexit hook, hanging the whole test run at
# shutdown. See turnstone/core/deadline.py.
judge = _make_judge(
content='{"risk_level":"medium","flags":[],"reasoning":""}',
timeout=1.0,
delay=5.0,
)
v = judge.evaluate("payload", call_id="c1")
assert v.error == "timeout"
stragglers = [
t
for t in threading.enumerate()
if t.name.startswith("output-guard-judge") and not t.daemon
]
assert stragglers == [], f"non-daemon worker survived evaluate(): {stragglers}"
class TestAliasResolution:
def test_unknown_alias_falls_back_to_session_model(self) -> None:
+20 -13
View File
@@ -19,7 +19,8 @@ class TestSuggestProfile:
p = suggest_profile("vllm", "google/gemma-4-31B-it")
assert p["capabilities"]["thinking_mode"] == "manual"
assert p["capabilities"]["thinking_param"] == "enable_thinking"
assert p["server_compat"]["extra_body"]["skip_special_tokens"] is False
# No bug-workaround extra_body — gemma-4 needs only the thinking param.
assert "extra_body" not in p["server_compat"]
def test_vllm_gemma3(self) -> None:
p = suggest_profile("vllm", "google/gemma-3-27b-it")
@@ -147,14 +148,14 @@ class TestMergeServerCompat:
result = merge_server_compat(None, {"extra_body": {"skip_special_tokens": False}})
assert result == {"skip_special_tokens": False}
def test_full_vllm_gemma_compat_no_base(self) -> None:
"""vLLM workaround forwards on its own."""
def test_full_server_compat_extra_body_no_base(self) -> None:
"""A server workaround (e.g. llama.cpp reasoning_format) forwards on its own."""
compat = {
"server_type": "vllm",
"extra_body": {"skip_special_tokens": False},
"server_type": "llama.cpp",
"extra_body": {"reasoning_format": "auto"},
}
result = merge_server_compat(None, compat)
assert result == {"skip_special_tokens": False}
assert result == {"reasoning_format": "auto"}
def test_operator_chat_template_kwargs_only(self) -> None:
"""Operator can set chat_template_kwargs explicitly without seeding the base."""
@@ -210,21 +211,27 @@ class TestEndToEndRequestShaping:
"""Compose both layers — session builds extra_params, provider applies thinking."""
def test_vllm_gemma_full_flow(self) -> None:
"""Session forwards server workarounds, provider adds thinking param."""
"""Gemma now needs only the thinking param — no server workaround."""
caps = ModelCapabilities(thinking_mode="manual", thinking_param="enable_thinking")
server_compat = {
"server_type": "vllm",
"extra_body": {"skip_special_tokens": False},
}
server_compat = {"server_type": "vllm"}
# Step 1: session forwards (no auto-injection of reasoning_effort).
extra_params = merge_server_compat(None, server_compat)
# Step 2: provider injects thinking param into chat_template_kwargs.
extra_body = dict(extra_params)
OpenAIChatCompletionsProvider._apply_thinking_mode(extra_body, caps)
assert extra_body == {"chat_template_kwargs": {"enable_thinking": True}}
def test_server_workaround_composes_with_thinking(self) -> None:
"""A top-level server workaround forwards alongside the injected thinking param."""
caps = ModelCapabilities(thinking_mode="manual", thinking_param="enable_thinking")
compat = {"server_type": "llama.cpp", "extra_body": {"reasoning_format": "auto"}}
extra_body = dict(merge_server_compat(None, compat))
OpenAIChatCompletionsProvider._apply_thinking_mode(extra_body, caps)
assert extra_body == {
"chat_template_kwargs": {"enable_thinking": True},
"skip_special_tokens": False,
"reasoning_format": "auto",
}
def test_granite_thinking_key(self) -> None:
@@ -292,7 +299,7 @@ class TestProbeIntegration:
assert result["server_type"] == "vllm"
assert result["suggested_capabilities"]["thinking_mode"] == "manual"
assert result["suggested_capabilities"]["thinking_param"] == "enable_thinking"
assert result["suggested_server_compat"]["extra_body"]["skip_special_tokens"] is False
assert "extra_body" not in result["suggested_server_compat"]
def test_detect_non_thinking_no_suggested_capabilities(self) -> None:
"""Non-thinking vLLM model gets server_compat but no capabilities suggestion."""
+81
View File
@@ -150,6 +150,27 @@ def _send_with_mocks(session, responses, mock_execute, **extra_patches):
yield save_msg
def _capturing_thread_cls():
"""Return a no-op ``threading.Thread`` stand-in plus the list it records
each constructed thread's ``target`` into.
Patched over ``session.threading.Thread`` so a test can assert WHICH
callable was scheduled (e.g. ``_generate_title``) without the thread
actually running ``start()`` is a no-op, so no background LLM call
fires.
"""
started: list = []
class _CaptureThread:
def __init__(self, *a, target=None, **kw):
started.append(target)
def start(self):
pass
return _CaptureThread, started
def _user_pending(session) -> list[tuple[str, str]]:
"""Return user-channel queued nudges as ``(type, text)`` tuples.
@@ -1010,6 +1031,66 @@ class TestTitleRetry:
# Restore for cleanup
session._ws_id = original_ws_id
def test_title_fires_after_send_not_after_tool_free_turn(self, tmp_db):
"""Auto-title fires right after the user turn is recorded, BEFORE
tools run it no longer waits for a tool-call-free assistant
turn. Coordinators spend nearly every turn in tool calls and may
never reach that terminal text turn, so the old end-of-turn
trigger almost never fired for them (the timing half of the
coordinator-title bug)."""
session = _make_session()
assert session._title_generated is False
# The assistant's opening turn is ALL tool calls — under the old
# trigger no title would generate until a later text-only turn.
responses = [
{
"role": "assistant",
"content": "working",
"tool_calls": [
{
"id": "c1",
"type": "function",
"function": {"name": "echo", "arguments": "{}"},
}
],
},
{"role": "assistant", "content": "done"},
]
capture_cls, started = _capturing_thread_cls()
def mock_execute(_tool_calls):
# The title must already be scheduled by the time tools run.
assert session._title_generated is True
return [("c1", "ok")], None
with (
_send_with_mocks(session, responses, mock_execute),
patch("turnstone.core.session.threading.Thread", capture_cls),
):
session.send("refactor the auth layer")
assert session._title_generated is True
assert session._generate_title in started
def test_title_not_generated_for_blank_or_wake_send(self, tmp_db):
"""Blank input and synthetic wake sends don't burn the one-shot
auto-title ``_generate_title`` needs first-user-message text,
and a wake carries none."""
capture_cls, started = _capturing_thread_cls()
def mock_execute(_tool_calls):
return [], None
for user_input, kwargs in ((" ", {}), ("a real message", {"from_wake": True})):
session = _make_session()
with (
_send_with_mocks(session, [{"role": "assistant", "content": "ok"}], mock_execute),
patch("turnstone.core.session.threading.Thread", capture_cls),
):
session.send(user_input, **kwargs)
assert session._generate_title not in started
assert session._title_generated is False
class TestLiveConfigUpdate:
"""ConfigStore-backed sessions pick up settings changes at point-of-use."""
+21
View File
@@ -602,6 +602,27 @@ def test_step7_tab_menu_wired_per_persona() -> None:
)
def test_coordinator_tab_menu_enables_title_verbs() -> None:
"""Coordinators carry LLM/auto titles like interactive workstreams, so
their tab dropdown must surface Refresh/Edit title convTabMenu's
``titleVerbs`` block, POSTed to the console-origin coord
``refresh-title`` / ``title`` routes via the base-aware lane (default
base ""). Scoped to the coordinator registerType block so it can't
pass on the interactive pane's long-standing ``titleVerbs``."""
shell = _SHELL_JS.read_text(encoding="utf-8")
start = shell.index('registerType("coordinator"')
tail = shell[start:]
nxt = tail.find("registerType(", 1) # bound at the next pane registration
coord_block = tail[:nxt] if nxt != -1 else tail
assert "pane._ctl.closeSession()" in coord_block, (
"sanity: the extracted block is the coordinator pane"
)
assert "convTabMenu(" in coord_block, "the coordinator pane must wire a tab menu"
assert "titleVerbs: true" in coord_block, (
"the coordinator tab menu must enable titleVerbs (Refresh/Edit title)"
)
def test_tab_menu_base_aware_verb_lane() -> None:
"""Lifecycle round 2: a proxied interactive pane's tab menu must act on the
pane's OWN transport base, not the console origin — the globals lane only
+9 -7
View File
@@ -96,10 +96,8 @@ def _make_flaky_client(monkeypatch, failures: int):
"""TLSClient whose CA fetch fails ``failures`` times, then succeeds.
Returns (client, calls, sleeps) mutable lists recording each CA-fetch
attempt and each backoff delay (asyncio.sleep is stubbed out).
attempt and each backoff delay (the client's backoff sleep is stubbed).
"""
import asyncio
from turnstone.core.tls import TLSClient
client = TLSClient(
@@ -119,11 +117,14 @@ def _make_flaky_client(monkeypatch, failures: int):
pass
async def fake_sleep(delay):
# Stub the client's own _sleep seam, NOT the global asyncio.sleep:
# patching the global also intercepts any concurrent task sharing the
# event loop, which corrupted a background poller and hung CI.
sleeps.append(delay)
monkeypatch.setattr(client, "_fetch_ca_cert", flaky_fetch)
monkeypatch.setattr(client, "_request_cert", ok_request)
monkeypatch.setattr(asyncio, "sleep", fake_sleep)
monkeypatch.setattr(client, "_sleep", fake_sleep)
return client, calls, sleeps
@@ -175,8 +176,6 @@ async def test_init_retries_exhausted_raises(monkeypatch):
@pytest.mark.anyio
async def test_init_retries_discovery_failure(monkeypatch):
"""Console discovery (not-yet-registered console) is retried too."""
import asyncio
from turnstone.core.tls import TLSClient
client = TLSClient(storage=get_storage(), hostnames=["node-1"])
@@ -191,10 +190,13 @@ async def test_init_retries_discovery_failure(monkeypatch):
async def ok():
pass
async def fake_sleep(_delay):
pass
monkeypatch.setattr(client, "_discover_console_url", flaky_discover)
monkeypatch.setattr(client, "_fetch_ca_cert", ok)
monkeypatch.setattr(client, "_request_cert", ok)
monkeypatch.setattr(asyncio, "sleep", lambda _: ok())
monkeypatch.setattr(client, "_sleep", fake_sleep)
await client.init(attempts=2)
assert attempts == [1, 2]
+40
View File
@@ -0,0 +1,40 @@
"""Tests for turnstone.console.server._validate_regex_pattern.
The catastrophic-backtracking branch is verified by simulating the deadline
firing rather than running a real ReDoS regex a genuine runaway pattern would
leave a CPU-pinned daemon worker for the rest of the suite. The daemon-abandon
mechanism itself is covered in tests/test_deadline.py.
"""
from __future__ import annotations
from turnstone.console.server import _validate_regex_pattern
from turnstone.core.deadline import DeadlineExceededError
def test_valid_pattern_returns_none() -> None:
assert _validate_regex_pattern(r"\d{3}-\d{4}") is None
def test_invalid_pattern_returns_error() -> None:
msg = _validate_regex_pattern(r"(unclosed")
assert msg is not None
assert msg.startswith("Invalid regex")
def test_catastrophic_backtracking_returns_message(monkeypatch) -> None:
def _deadline(*_args, **_kwargs):
raise DeadlineExceededError
monkeypatch.setattr("turnstone.console.server.run_with_deadline", _deadline)
# The pattern is arbitrary — run_with_deadline is stubbed to raise, so the
# probe never runs; a real backtracking literal here would only trip CodeQL.
assert _validate_regex_pattern(r"\w+") == "Regex appears to have catastrophic backtracking"
def test_probe_error_returns_generic_message(monkeypatch) -> None:
def _err(*_args, **_kwargs):
raise RuntimeError("boom")
monkeypatch.setattr("turnstone.console.server.run_with_deadline", _err)
assert _validate_regex_pattern(r"abc") == "Regex caused an error during test"
+17 -5
View File
@@ -31,16 +31,17 @@ from turnstone.core.session_routes import (
make_export_handler,
make_history_handler,
make_open_handler,
make_refresh_title_handler,
make_retry_handler,
make_rewind_handler,
make_set_title_handler,
)
from turnstone.core.storage._sqlite import SQLiteBackend
from turnstone.core.workstream import WorkstreamKind
from turnstone.server import (
_interactive_tenant_check,
delete_workstream_endpoint,
list_interface_settings,
refresh_workstream_title,
set_workstream_title,
update_interface_setting,
)
@@ -113,6 +114,18 @@ def delete_client(_inject_storage):
@pytest.fixture
def title_client(_inject_storage):
# Build the lifted refresh/set-title handlers the same way server.py
# wires the interactive bundle — same SessionEndpointConfig
# (manager_lookup + _interactive_tenant_check) so the tests exercise
# the production resolution path (mgr fast-path → storage ownership).
mock_mgr = MagicMock()
cfg = SessionEndpointConfig(
permission_gate=None,
manager_lookup=lambda _r: (mock_mgr, None),
tenant_check=_interactive_tenant_check,
not_found_label="Workstream not found",
audit_action_prefix="workstream",
)
app = Starlette(
routes=[
Mount(
@@ -120,12 +133,12 @@ def title_client(_inject_storage):
routes=[
Route(
"/api/workstreams/{ws_id}/title",
set_workstream_title,
make_set_title_handler(cfg),
methods=["POST"],
),
Route(
"/api/workstreams/{ws_id}/refresh-title",
refresh_workstream_title,
make_refresh_title_handler(cfg),
methods=["POST"],
),
],
@@ -133,7 +146,6 @@ def title_client(_inject_storage):
],
middleware=[Middleware(_InjectAuthMiddleware)],
)
mock_mgr = MagicMock()
app.state.workstreams = mock_mgr
return TestClient(app), mock_mgr
+1 -1
View File
@@ -1,3 +1,3 @@
"""turnstone - Multi-node AI orchestration platform with tool use, agent routing, and cluster simulation."""
__version__ = "1.6.6"
__version__ = "1.6.9"
+2 -2
View File
@@ -1022,8 +1022,8 @@ def main() -> None:
"--judge-timeout",
dest="judge_timeout",
type=float,
default=60.0,
help="LLM judge timeout in seconds (default: 60)",
default=120.0,
help="LLM judge timeout in seconds (default: 120)",
)
judge_group.add_argument(
"--judge-confidence",
+12 -1
View File
@@ -94,6 +94,11 @@ class ClusterCollector:
self._nodes: dict[str, NodeSnapshot] = {}
self._running = False
self._threads: list[threading.Thread] = []
# Wakes the discovery loop out of its inter-scan sleep so ``stop()``
# can join it promptly instead of blocking up to ``discovery_interval``
# (a long interval would otherwise leave the thread sleeping past
# join's timeout — a leaked background thread).
self._discovery_wake = threading.Event()
# SSE fan-out to browser clients
self._listeners: list[queue.Queue[dict[str, Any]]] = []
@@ -135,6 +140,9 @@ class ClusterCollector:
def start(self) -> None:
"""Start background threads."""
self._running = True
# Clear the shutdown wake so a restarted collector (stop() set it) sleeps
# the full interval again instead of busy-spinning the discovery loop.
self._discovery_wake.clear()
# Subscribe to the ``services`` channel for reactive node discovery.
# NOTIFY-driven wake-ups bring new-node visibility from up-to-60 s
# (next discovery tick) down to ~500 ms on Postgres; the 60 s
@@ -161,6 +169,7 @@ class ClusterCollector:
its ``finally`` cleanup (cancel tasks, close AsyncClient).
"""
self._running = False
self._discovery_wake.set() # wake the discovery loop out of its sleep
if self._notify_unsubscribe is not None:
with contextlib.suppress(Exception):
self._notify_unsubscribe()
@@ -417,7 +426,9 @@ class ClusterCollector:
pass # already logged by storage layer
except Exception:
log.exception("Node discovery error")
time.sleep(self._discovery_interval)
# Interruptible inter-scan sleep — ``stop()`` sets the event to
# wake us immediately instead of blocking out the full interval.
self._discovery_wake.wait(self._discovery_interval)
def _discover_nodes(self) -> None:
"""Query the service registry and update the node map."""
+32 -3
View File
@@ -23,6 +23,8 @@ from turnstone.core.child_event_bus import ChildEventBus
from turnstone.core.child_source import ClusterChildSource
from turnstone.core.children_registry import ChildrenRegistry
from turnstone.core.log import get_logger
from turnstone.core.memory import get_workstream_display_name, get_workstream_display_names
from turnstone.core.storage import is_storage_initialized
from turnstone.core.workstream import Workstream, WorkstreamKind, WorkstreamState
if TYPE_CHECKING:
@@ -37,6 +39,28 @@ if TYPE_CHECKING:
log = get_logger(__name__)
def _coord_display_name(ws: Workstream) -> str:
"""Resolve a coordinator's display name (``alias > title > name``).
``ws.name`` is the synthetic ``ws-xxxx`` placeholder; the persisted
auto-title (``update_workstream_title``) and user alias live only in
the DB. Seeding the collector with the resolved name means a
rehydrated coordinator shows its title in the live cluster tree
immediately, rather than reverting to ``ws-xxxx`` until a (for
coordinators, rarely-firing) ``on_rename`` event arrives.
Skips the DB read when storage isn't initialized: this runs on a
lifecycle-event path, and a display-name resolution must never trip
``get_storage``'s SQLite auto-init side effect (a stray
``.turnstone.db``) before the host has called ``init_storage`` (the
real cluster always does so at startup this only bites early /
test call paths). The placeholder ``ws.name`` is the right fallback.
"""
if not is_storage_initialized():
return ws.name
return get_workstream_display_name(ws.id) or ws.name
class CoordinatorAdapter:
"""Bridges SessionManager to the console's coordinator transport."""
@@ -132,7 +156,7 @@ class CoordinatorAdapter:
try:
self._collector.emit_console_ws_created(
ws.id,
name=ws.name,
name=_coord_display_name(ws),
user_id=ws.user_id,
kind=ws.kind.value,
state=ws.state.value,
@@ -466,11 +490,16 @@ class CoordinatorAdapter:
# creates happened before the collector was wired up and their
# rows never showed on the snapshot. (Coord-specific — interactive
# has no analogous pseudo-node.)
for ws in mgr.list_all():
coords = mgr.list_all()
# One round-trip for every coordinator's display name instead of a
# per-``ws`` ``_coord_display_name`` lookup (N+1); cold path, but
# the bulk helper is right there.
seed_names = get_workstream_display_names([ws.id for ws in coords])
for ws in coords:
try:
collector.emit_console_ws_created(
ws.id,
name=ws.name,
name=seed_names.get(ws.id) or ws.name,
user_id=ws.user_id or "",
kind=WorkstreamKind.COORDINATOR.value,
state=ws.state.value,
+82 -37
View File
@@ -55,6 +55,8 @@ from turnstone.core.auth import (
jwt_version_slot,
require_permission,
)
from turnstone.core.deadline import DeadlineExceededError, run_with_deadline
from turnstone.core.memory import get_workstream_display_names
from turnstone.core.rendezvous import NoAvailableNodeError
from turnstone.core.session_replay import session_replay_preamble
from turnstone.core.session_routes import (
@@ -74,9 +76,11 @@ from turnstone.core.session_routes import (
make_history_handler,
make_list_handler,
make_open_handler,
make_refresh_title_handler,
make_retry_handler,
make_rewind_handler,
make_send_handler,
make_set_title_handler,
make_unified_saved_handler,
register_coord_verbs,
register_session_routes,
@@ -852,6 +856,14 @@ def _coordinator_rows(request: Request) -> list[dict[str, Any]]:
In-memory wins on ws_id conflict so live state stays authoritative
for active sessions.
Display name resolves ``alias > title > name`` from the persisted
row for BOTH lanes. ``ws.name`` on the in-memory Workstream is the
synthetic ``ws-xxxx`` placeholder; the LLM auto-title
(``update_workstream_title``) and the user alias
(``set_workstream_alias``) live only in the DB, so without the
persisted lookup the live lane would show ``ws-xxxx`` and the
auto-title would never survive a dashboard refresh.
Trusted-team visibility (post-#400): the cluster dashboard shows
every coordinator regardless of caller identity; ``user_id`` is
surfaced on each row as display metadata.
@@ -869,6 +881,61 @@ def _coordinator_rows(request: Request) -> list[dict[str, Any]]:
val = getattr(sess, name, "") if sess else ""
return val if isinstance(val, str) else ""
# Persisted coordinator rows serve two purposes: (1) surface
# closed / error / deleted coordinators the manager has already
# evicted from ``self._workstreams``, and (2) supply the persisted
# display name (``alias > title > name``) for the LIVE coordinators
# too — ``ws.name`` is the synthetic placeholder. Cluster-wide
# (trusted-team visibility). Indexed by ws_id so both lanes resolve
# the same way.
storage = getattr(request.app.state, "auth_storage", None)
persisted: list[Any] = []
if storage is not None:
try:
persisted = storage.list_workstreams(
kind=WorkstreamKind.COORDINATOR,
user_id=None,
limit=200,
)
except Exception:
log.debug("cluster_workstreams.coord_persisted_failed", exc_info=True)
persisted = []
# SQLAlchemy Row — access via _mapping so future SELECT reorders /
# new columns don't silently corrupt the projection (per the
# storage-protocol guidance on list_workstreams). Test doubles must
# expose the same ._mapping attribute.
meta: dict[str, Any] = {}
for row in persisted:
m = row._mapping
rid = m.get("ws_id") or ""
if rid:
meta[rid] = m
# Live coordinators resolve their display name through the bulk
# helper keyed on their EXACT ids (one round-trip, no row cap) rather
# than the ``limit=200`` ``meta`` map: a live coord that has dropped
# below the 200-row ``updated DESC`` window would otherwise revert to
# its synthetic ``ws.name``. Closed/evicted rows (the persisted lane
# below) already carry alias/title in their own ``_mapping``.
live_display = get_workstream_display_names([ws.id for ws in wss]) if wss else {}
def _display_name(ws_id: str, fallback: str) -> str:
m = meta.get(ws_id)
if m is None:
return fallback
return m.get("alias") or m.get("title") or m.get("name") or fallback
def _title(ws_id: str) -> str:
# Best-effort: the secondary ``title`` field is sourced from the
# ``limit=200`` ``meta`` map, so a live coord outside that window
# reports ``""`` here. The user-visible ``name`` stays correct
# (resolved via the uncapped ``live_display`` above, and the UI
# renders ``title || name``); the empty title is harmless and the
# window is unreachable in practice (live coords are bounded by
# ``max_active`` and sort to the top of ``updated DESC``).
m = meta.get(ws_id)
return str(m.get("title") or "") if m is not None else ""
rows: list[dict[str, Any]] = []
seen: set[str] = set()
for ws in wss:
@@ -876,9 +943,9 @@ def _coordinator_rows(request: Request) -> list[dict[str, Any]]:
rows.append(
{
"id": ws.id,
"name": ws.name,
"name": live_display.get(ws.id) or ws.name,
"state": ws.state.value,
"title": "",
"title": _title(ws.id),
"node": "console",
"server_url": "",
"model": _str_sess_attr(sess, "model"),
@@ -895,30 +962,7 @@ def _coordinator_rows(request: Request) -> list[dict[str, Any]]:
)
seen.add(ws.id)
# Second lane — persisted coordinator rows, used to surface
# closed / error / deleted coordinators the manager has already
# evicted from ``self._workstreams``. Cluster-wide (trusted-team
# visibility).
storage = getattr(request.app.state, "auth_storage", None)
if storage is None:
return rows
try:
persisted = storage.list_workstreams(
kind=WorkstreamKind.COORDINATOR,
user_id=None,
limit=200,
)
except Exception:
log.debug("cluster_workstreams.coord_persisted_failed", exc_info=True)
return rows
for row in persisted:
# SQLAlchemy Row — access via _mapping so future SELECT reorders
# / new columns don't silently corrupt the projection (per the
# storage-protocol guidance on list_workstreams). Test doubles
# must expose the same ._mapping attribute; positional indexing
# was removed because it hard-coded column offsets that drift
# with migrations.
m = row._mapping
row_id = m.get("ws_id") or ""
if not row_id or row_id in seen:
@@ -927,9 +971,9 @@ def _coordinator_rows(request: Request) -> list[dict[str, Any]]:
rows.append(
{
"id": row_id,
"name": m.get("name") or f"coord-{row_id[:4]}",
"name": _display_name(row_id, f"coord-{row_id[:4]}"),
"state": str(m.get("state") or "idle"),
"title": "",
"title": _title(row_id),
"node": "console",
"server_url": "",
"model": "",
@@ -11530,16 +11574,15 @@ def _validate_regex_pattern(pattern: str, flags: int = 0) -> str | None:
compiled.search(s)
try:
from concurrent.futures import ThreadPoolExecutor
from concurrent.futures import TimeoutError as FuturesTimeout
pool = ThreadPoolExecutor(max_workers=1)
try:
pool.submit(_probe).result(timeout=0.5)
except FuturesTimeout:
return "Regex appears to have catastrophic backtracking"
finally:
pool.shutdown(wait=False, cancel_futures=True)
# Daemon worker: a catastrophically-backtracking regex must be
# abandonable without pinning a non-daemon thread that would hang
# interpreter exit (a ThreadPoolExecutor worker is joined at exit).
# Budget is generous — a legitimately complex pattern can take a second
# or two on the probe strings; only exponential blowup (which sails past
# any few-second bound) should trip the catastrophic-backtracking guard.
run_with_deadline(_probe, timeout=3.0, poll=0.1, thread_name="regex-redos-probe")
except DeadlineExceededError:
return "Regex appears to have catastrophic backtracking"
except Exception:
return "Regex caused an error during test"
return None
@@ -12979,6 +13022,8 @@ def create_app(
audit_emit=_audit_close_coordinator,
supports_close_reason=False,
),
refresh_title=make_refresh_title_handler(coord_endpoint_config), # lifted: shared body
set_title=make_set_title_handler(coord_endpoint_config), # lifted: shared body
send=make_send_handler(coord_endpoint_config), # lifted: shared body (P1.5)
dequeue=make_dequeue_handler(coord_endpoint_config), # lifted: shared body
approve=make_approve_handler(coord_endpoint_config), # lifted: shared body
+197 -18
View File
@@ -15,11 +15,16 @@ backend is surfaced as a typed error the endpoint maps to 503 / 502.
from __future__ import annotations
import subprocess
import threading
from dataclasses import dataclass
from typing import Any
from typing import TYPE_CHECKING, Any
from turnstone.core.log import get_logger
from turnstone.core.server_compat import merge_server_compat
if TYPE_CHECKING:
from collections.abc import Iterator
log = get_logger(__name__)
@@ -83,6 +88,22 @@ _OMNI_STT_PROMPT = (
"seven, and write 3 instead of three"
)
# Bound the omni STT decode. Gemma caps audio at 30 s and a 30 s transcript is
# well under this, so the cap only catches a pathological runaway — it never
# truncates a real transcript.
_OMNI_STT_MAX_TOKENS = 1024
# Hard limit on the ffmpeg transcode subprocess (seconds).
_FFMPEG_TIMEOUT_S = 30
# Cap the decoded audio duration so a crafted clip can't expand into an
# unbounded decode (the upload itself is already size-capped at the endpoint).
_MAX_AUDIO_SECONDS = 300
# Per-request timeout for the streaming STT chat call — bounds a hung backend
# (the whole transcription is ~1 s; this only catches a stalled stream).
_OMNI_STT_TIMEOUT_S = 60
@dataclass(frozen=True)
class TranscriptionResult:
@@ -185,30 +206,115 @@ def _serves_transcription_endpoint(cfg: Any, model: str) -> bool:
return _infer_audio_capability(model, "stt")
def _transcribe_via_chat(client: Any, model: str, data: bytes, filename: str, prompt: str) -> str:
def _to_wav_16k_mono(data: bytes) -> bytes:
"""Decode any ffmpeg-readable audio container to 16 kHz mono PCM WAV.
Browsers record webm/opus (or ogg/mp4); the omni chat lane vLLM in
particular only decodes wav/mp3 and sniffs the bytes, so the raw upload is
rejected as an "Invalid or unsupported audio file". ffmpeg reads the
container from the byte stream (no reliance on the filename) and resamples to
the 16 kHz mono PCM the model documents. Raises :class:`AudioBackendError`
(the endpoint maps it to 502) if ffmpeg is missing or the bytes don't decode.
"""
# ffmpeg reads only the piped bytes (-protocol_whitelist pipe) so a crafted
# container can't open file:/http: references (SSRF / local file read); -vn
# drops video streams and -t bounds the decode against a decompression bomb.
try:
proc = subprocess.run(
[
"ffmpeg",
"-hide_banner",
"-loglevel",
"error",
"-protocol_whitelist",
"pipe",
"-i",
"pipe:0",
"-vn",
"-t",
str(_MAX_AUDIO_SECONDS),
"-ac",
"1",
"-ar",
"16000",
"-f",
"wav",
"pipe:1",
],
input=data,
capture_output=True,
timeout=_FFMPEG_TIMEOUT_S,
)
except FileNotFoundError as exc:
raise AudioBackendError("ffmpeg is not installed; cannot transcode audio") from exc
except subprocess.TimeoutExpired as exc:
raise AudioBackendError("Audio transcode timed out") from exc
if proc.returncode != 0 or not proc.stdout:
detail = proc.stderr.decode("utf-8", "replace").strip()
raise AudioBackendError(f"Audio transcode failed: {detail[-200:] or 'no output'}")
return proc.stdout
def _omni_chat_extra_body(cfg: Any) -> dict[str, Any]:
"""Build the chat ``extra_body`` for an omni STT call.
The STT path calls the raw client, so it bypasses the provider's request
shaping. Reuse ``merge_server_compat`` to forward any operator-stored
``server_compat["extra_body"]``, then force **thinking OFF** via the model's
own ``thinking_param``: transcription needs no reasoning, and leaving it on
multiplies latency ~10x and (on some chat templates) empties the content.
The override is applied last so it wins over any operator thinking flag.
"""
server_compat = getattr(cfg, "server_compat", None)
extra = merge_server_compat(None, server_compat) if isinstance(server_compat, dict) else {}
caps = getattr(cfg, "capabilities", None) or {}
thinking_param = caps.get("thinking_param")
if thinking_param and caps.get("thinking_mode") in ("manual", "adaptive"):
extra.setdefault("chat_template_kwargs", {})[thinking_param] = False
return extra
def _omni_chat_messages(prompt: str, audio_b64: str) -> list[dict[str, Any]]:
"""The single user turn for an omni STT chat call: the prompt precedes the
audio part the order Gemma documents for transcription."""
return [
{
"role": "user",
"content": [
{"type": "text", "text": prompt},
{"type": "input_audio", "input_audio": {"data": audio_b64, "format": "wav"}},
],
}
]
def _transcribe_via_chat(
client: Any,
model: str,
data: bytes,
prompt: str,
*,
extra_body: dict[str, Any] | None = None,
max_tokens: int = _OMNI_STT_MAX_TOKENS,
) -> str:
"""Transcribe by handing the clip to an omni *chat* model as ``input_audio``.
For models that accept audio in chat (``supports_audio_input``) but don't
serve ``/audio/transcriptions``. The instruction ``prompt`` steers the model
to emit only the transcript. The audio format is taken from the upload's
filename extension (the same shape the attachment wire path uses).
serve ``/audio/transcriptions``. The clip is transcoded to 16 kHz mono WAV
first (browsers record webm/opus, which the chat lane can't decode). The
instruction ``prompt`` precedes the audio part the order Gemma documents
for transcription and ``extra_body`` carries the thinking-off / server
compat params the raw-client path would otherwise skip.
"""
import base64
name = filename or "speech.webm"
fmt = name.rsplit(".", 1)[-1].lower() if "." in name else "wav"
audio_b64 = base64.b64encode(data).decode("ascii")
wav = _to_wav_16k_mono(data)
audio_b64 = base64.b64encode(wav).decode("ascii")
resp = client.chat.completions.create(
model=model,
messages=[
{
"role": "user",
"content": [
{"type": "text", "text": prompt},
{"type": "input_audio", "input_audio": {"data": audio_b64, "format": fmt}},
],
}
],
messages=_omni_chat_messages(prompt, audio_b64),
max_tokens=max_tokens,
extra_body=extra_body or None,
)
choices = getattr(resp, "choices", None) or []
if not choices:
@@ -261,13 +367,86 @@ def transcribe(
transcript = (getattr(resp, "text", "") or "").strip()
else:
transcript = _transcribe_via_chat(
client, model, data, filename, prompt or _OMNI_STT_PROMPT
client,
model,
data,
prompt or _OMNI_STT_PROMPT,
extra_body=_omni_chat_extra_body(cfg),
)
except AudioBackendError:
# Transcode errors already carry an actionable message — keep it.
raise
except Exception as exc:
raise AudioBackendError(f"Transcription backend failed: {exc}") from exc
return TranscriptionResult(transcript=transcript, model_alias=alias, model=model)
def _iter_stream_deltas(stream: Any) -> Iterator[str]:
"""Yield non-empty content deltas from an OpenAI streaming chat response.
Owns the stream's lifecycle: exhausting or closing this generator releases
the underlying HTTP connection, so an abandoned stream can't leak it.
"""
try:
for chunk in stream:
choices = getattr(chunk, "choices", None) or []
if not choices:
continue
delta = getattr(choices[0].delta, "content", None)
if delta:
yield delta
finally:
close = getattr(stream, "close", None)
if callable(close):
close()
def transcribe_stream(*, registry: Any, alias: str, data: bytes, prompt: str = "") -> Iterator[str]:
"""Stream transcript content deltas for the STT role alias.
Resolve, transcode, and opening the streaming-chat request all run eagerly
(before the returned generator yields its first delta) so the caller can
surface a clean 503 / 502; only the token iteration is deferred. A
whisper-style endpoint alias has no chat stream, so it emits the whole
transcript as a single chunk.
"""
try:
client, model, cfg = registry.resolve(alias)
except Exception as exc: # unknown/removed alias
raise AudioUnavailableError(f"STT model alias {alias!r} is not available") from exc
if not _provider_carries_audio(cfg):
raise AudioUnavailableError(
f"STT model alias {alias!r} (provider {getattr(cfg, 'provider', 'unknown')!r}) "
"can't transcribe audio — audio roles require an OpenAI-compatible provider."
)
if _serves_transcription_endpoint(cfg, model):
# Whisper-style endpoint: no chat stream — emit the whole transcript once.
text = transcribe(
registry=registry, alias=alias, data=data, filename="speech.webm", prompt=prompt
).transcript
return iter([text] if text else [])
caps = getattr(cfg, "capabilities", None) or {}
if not caps.get("supports_audio_input"):
raise AudioUnavailableError(f"STT model alias {alias!r} cannot transcribe audio")
import base64
wav = _to_wav_16k_mono(data)
audio_b64 = base64.b64encode(wav).decode("ascii")
try:
stream = client.chat.completions.create(
model=model,
messages=_omni_chat_messages(prompt or _OMNI_STT_PROMPT, audio_b64),
max_tokens=_OMNI_STT_MAX_TOKENS,
extra_body=_omni_chat_extra_body(cfg) or None,
stream=True,
timeout=_OMNI_STT_TIMEOUT_S,
)
except Exception as exc:
raise AudioBackendError(f"Transcription backend failed: {exc}") from exc
return _iter_stream_deltas(stream)
# -- transcript memoization (no-native-audio wire fallback) -------------------
# Caching an STT result is an audio-domain concern, so it lives here next to
# ``transcribe``. The wire resolver re-materializes every attachment on every
+94
View File
@@ -0,0 +1,94 @@
"""Run a blocking call under a wall-clock deadline on a daemon thread.
The motivating constraint comes from the judges (:mod:`turnstone.core.judge`,
:mod:`turnstone.core.output_guard_judge`): an upstream LLM call must be
*abandonable* the instant its timeout or cancel fires, without the abandoned
call being able to block process or interpreter exit.
A :class:`~concurrent.futures.ThreadPoolExecutor` worker is **non-daemon**, and
``concurrent.futures`` joins every executor worker from an ``atexit`` hook
(``_python_exit``) regardless of ``shutdown(wait=False)``. So an upstream call
wedged with no socket timeout hangs interpreter shutdown forever which is
exactly how a single slow judge call can deadlock a whole test run at exit.
A **daemon** worker is never joined at exit, so abandoning one is always safe:
the call keeps running until it returns or the process dies, whichever comes
first, and never pins shutdown.
"""
from __future__ import annotations
import queue
import threading
import time
from typing import TYPE_CHECKING, TypeVar
if TYPE_CHECKING:
from collections.abc import Callable
_T = TypeVar("_T")
class DeadlineExceededError(Exception):
"""The call did not complete before its wall-clock deadline."""
class DeadlineCancelledError(Exception):
"""The cancel event fired before the call completed."""
def run_with_deadline(
fn: Callable[[], _T],
*,
timeout: float,
cancel_event: threading.Event | None = None,
poll: float = 1.0,
thread_name: str = "deadline-worker",
) -> _T:
"""Run ``fn()`` on a daemon thread, bounded by ``timeout``/``cancel_event``.
Returns ``fn()``'s result, or re-raises whatever ``fn`` raised. Raises
:class:`DeadlineExceededError` if ``timeout`` seconds elapse first, or
:class:`DeadlineCancelledError` if ``cancel_event`` fires first. On either
abort the worker thread is abandoned; being a daemon it cannot block
process or interpreter exit.
``poll`` bounds how often ``cancel_event`` is checked (and thus the worst-
case latency from a cancel to this function returning).
"""
box: queue.Queue[tuple[bool, object]] = queue.Queue(maxsize=1)
def _runner() -> None:
try:
box.put((True, fn()))
except BaseException as exc: # noqa: BLE001 - relayed to the caller verbatim
box.put((False, exc))
threading.Thread(target=_runner, name=thread_name, daemon=True).start()
deadline = time.monotonic() + timeout
while True:
# Prefer a result that has already arrived over a deadline or cancel
# firing in the same scheduling window — otherwise a completed call
# could be reported as a spurious timeout/cancel under jitter.
try:
ok, payload = box.get_nowait()
except queue.Empty:
pass
else:
if ok:
return payload # type: ignore[return-value] # ok=True ⇒ payload is _T
raise payload # type: ignore[misc] # ok=False ⇒ payload is the raised exc
if cancel_event is not None and cancel_event.is_set():
raise DeadlineCancelledError
remaining = deadline - time.monotonic()
if remaining <= 0:
raise DeadlineExceededError
try:
ok, payload = box.get(timeout=min(remaining, poll))
except queue.Empty:
continue
if ok:
return payload # type: ignore[return-value]
raise payload # type: ignore[misc]
+33 -50
View File
@@ -15,11 +15,16 @@ import re
import threading
import time
import uuid
from concurrent.futures import ThreadPoolExecutor
from dataclasses import dataclass, field
from functools import partial
from pathlib import Path
from typing import TYPE_CHECKING, Any
from turnstone.core.deadline import (
DeadlineCancelledError,
DeadlineExceededError,
run_with_deadline,
)
from turnstone.core.log import get_logger
if TYPE_CHECKING:
@@ -76,8 +81,8 @@ class JudgeConfig:
"""Configuration for the intent validation judge.
The *timeout* value applies **per turn**, not as a total budget across
all turns. With the default of 60 s and a maximum of 5 turns, a
single tool-call evaluation can take up to 300 s in the worst case
all turns. With the default of 120 s and a maximum of 5 turns, a
single tool-call evaluation can take up to 600 s in the worst case
(e.g. a multi-turn tool-use exchange with a slow local model).
"""
@@ -86,13 +91,13 @@ class JudgeConfig:
smart_approvals: bool = False # auto-approve high-confidence "approve" LLM verdicts
confidence_threshold: float = 0.95 # Smart Approvals auto-approve bar (recommendation=approve)
max_context_ratio: float = 0.5
timeout: float = 60.0 # per-turn timeout in seconds (see class docstring)
timeout: float = 120.0 # per-turn timeout in seconds (see class docstring)
read_only_tools: bool = True
output_guard: bool = True
output_guard_budget_seconds: float = 30.0 # wall-clock budget for output_guard regex scan
output_guard_llm: bool = False # enable LLM stage on tool output (issue #560 mitigation #1)
output_guard_model: str = "" # alias for the LLM stage; empty = inherit session model
output_guard_llm_timeout: float = 30.0 # wall-clock budget for the LLM stage
output_guard_llm_timeout: float = 60.0 # wall-clock budget for the LLM stage
redact_secrets: bool = True
# True = the approval gate's resolution aborts remaining evaluations
# (saves inference; undone items degrade to ``llm_fallback`` verdicts
@@ -881,10 +886,6 @@ If you used read_file to check a target, cite what you found."""
# ---------------------------------------------------------------------------
class _ExecutorPoisonedError(Exception):
"""Raised when a timeout leaves the executor's worker thread stuck."""
class IntentJudge:
"""Session-scoped LLM judge for intent validation.
@@ -1054,7 +1055,6 @@ class IntentJudge:
verdicts are delivered.
"""
client = self._create_client()
executor = ThreadPoolExecutor(max_workers=1, thread_name_prefix="judge-api")
try:
for idx, (item, h_verdict) in enumerate(zip(items, heuristic_verdicts, strict=True)):
if cancel_event and cancel_event.is_set():
@@ -1071,7 +1071,6 @@ class IntentJudge:
item,
messages,
cancel_event,
executor,
client,
)
if llm_verdict:
@@ -1116,17 +1115,6 @@ class IntentJudge:
"judge cancelled before evaluating this call",
)
return
except _ExecutorPoisonedError:
executor.shutdown(wait=False, cancel_futures=True)
executor = ThreadPoolExecutor(max_workers=1, thread_name_prefix="judge-api")
# Deliver a fallback for the interrupted item so every
# call still gets exactly one verdict. Smart Approvals
# waits on the full set before gating; a silently-
# skipped item would otherwise block that wait until
# its timeout (and the advisory UI would miss a chip).
self._deliver_fallbacks(
[item], [h_verdict], callback, "judge executor restarted"
)
except Exception:
log.exception(
"Judge evaluation failed for %s",
@@ -1134,7 +1122,6 @@ class IntentJudge:
)
self._deliver_fallbacks([item], [h_verdict], callback, "judge evaluation error")
finally:
executor.shutdown(wait=False, cancel_futures=True)
try:
if hasattr(client, "close"):
client.close()
@@ -1172,7 +1159,6 @@ class IntentJudge:
item: dict[str, Any],
messages: list[dict[str, Any]],
cancel_event: threading.Event | None,
executor: ThreadPoolExecutor,
client: Any,
) -> IntentVerdict | None:
"""Run LLM judge for a single tool call. Returns verdict or None."""
@@ -1237,32 +1223,29 @@ class IntentJudge:
# models aren't penalised for slow earlier turns.
per_call_timeout = max(self._config.timeout, 5.0) # at least 5s
try:
future = executor.submit(
self._provider.create_completion,
client=client,
model=self._model,
messages=judge_messages,
tools=None if is_last_turn else tools,
max_tokens=2048,
temperature=0.0,
reasoning_effort="medium",
# Each turn runs on its own daemon worker (1s cancel polling).
# A timeout or cancel abandons the call without pinning a
# non-daemon thread that would block interpreter exit — the old
# single-slot ThreadPoolExecutor left a stuck worker that
# poisoned the pool, which is why the restart dance existed.
result = run_with_deadline(
partial(
self._provider.create_completion,
client=client,
model=self._model,
messages=judge_messages,
tools=None if is_last_turn else tools,
max_tokens=2048,
temperature=0.0,
reasoning_effort="medium",
),
timeout=per_call_timeout,
cancel_event=cancel_event,
thread_name="judge-api",
)
# Poll in 1s increments so we notice cancellation promptly
# instead of blocking for the full per_call_timeout.
deadline = time.monotonic() + per_call_timeout
while True:
remaining = deadline - time.monotonic()
if cancel_event and cancel_event.is_set():
future.cancel()
return None
if remaining <= 0:
raise TimeoutError
try:
result = future.result(timeout=min(remaining, 1.0))
break
except TimeoutError:
pass # loop back to check remaining/cancel
except TimeoutError:
except DeadlineCancelledError:
return None
except DeadlineExceededError:
log.info("judge.turn.timeout", turn=turn + 1, timeout=per_call_timeout)
# Safety net: if we have a partial result from a previous turn,
# try to parse a verdict from it before giving up.
@@ -1277,7 +1260,7 @@ class IntentJudge:
if verdict:
log.info("judge.verdict.from_partial", turn=turn + 1)
return verdict
raise _ExecutorPoisonedError from None
return None
except Exception as e:
log.info("judge.turn.failed", turn=turn + 1, error=str(e))
return None
+38 -40
View File
@@ -12,13 +12,13 @@ Design:
a static tool result doesn't benefit from multi-turn — the text is
already in hand.
- JSON-in-content verdict. 4-strategy parser inlined from
:class:`IntentJudge` (``judge.py:1603-1659``).
- ``ThreadPoolExecutor`` + ``future.result(timeout=)`` with 1 s
cancel-event polling. The executor is owned explicitly with
``shutdown(wait=False, cancel_futures=True)`` so a timeout or
cancellation returns promptly even if the worker thread is still
blocked on the upstream LLM call. This mirrors
:meth:`IntentJudge._run_judge`'s pattern at ``judge.py:1117-1118``.
:meth:`IntentJudge._parse_verdict`.
- Wall-clock deadline via :func:`turnstone.core.deadline.run_with_deadline`,
which runs the call on a *daemon* worker and polls the cancel event each
second. A timeout or cancel abandons the call rather than waiting it out,
and the daemon worker can never block process or interpreter exit unlike
a ``ThreadPoolExecutor`` worker, which ``concurrent.futures`` joins from an
``atexit`` hook regardless of ``shutdown(wait=False)``.
- HTTP client is lazy-init + reused across evaluations on a single
judge instance. Session-side model swaps drop the entire
:class:`OutputGuardJudge` (``session.py:1733``/``:2136``), which
@@ -39,11 +39,15 @@ import json
import re
import time
import uuid
from concurrent.futures import ThreadPoolExecutor
from dataclasses import dataclass
from typing import TYPE_CHECKING, Any
from turnstone.core import fence
from turnstone.core.deadline import (
DeadlineCancelledError,
DeadlineExceededError,
run_with_deadline,
)
from turnstone.core.log import get_logger
if TYPE_CHECKING:
@@ -370,9 +374,10 @@ class OutputGuardJudge:
by :meth:`_user_prompt`. Callers that don't have a particular
field leave it at its default the prompt skips empty sections.
Timeout enforcement is real wall-clock: the executor is shut
down with ``wait=False, cancel_futures=True`` on the timeout /
cancel path, so a hung upstream LLM call does not block return.
Timeout enforcement is real wall-clock: the upstream call runs on a
daemon worker via :func:`~turnstone.core.deadline.run_with_deadline`
and is abandoned on the timeout / cancel path, so a hung upstream LLM
call neither blocks return nor pins interpreter exit.
"""
if not output:
return OutputJudgeVerdict(
@@ -407,15 +412,16 @@ class OutputGuardJudge:
verdict_id, call_id, start, f"client_create_failed: {type(e).__name__}"
)
# Explicit executor lifetime — the `with ... as ex:` form's
# implicit shutdown(wait=True) would block return until the
# upstream call completed, defeating the wall-clock timeout.
# Mirror IntentJudge's pattern at judge.py:1117-1118.
ex = ThreadPoolExecutor(max_workers=1, thread_name_prefix="output-guard-judge")
# Run the upstream call on a *daemon* worker bounded by a real
# wall-clock deadline: a timeout or cancel abandons the call instead of
# waiting it out, and because the worker is a daemon an abandoned call
# can never block process or interpreter exit. (A ThreadPoolExecutor
# worker is non-daemon, and concurrent.futures joins it from an atexit
# hook regardless of shutdown(wait=False) — so a wedged upstream call
# would otherwise hang shutdown.)
try:
try:
future = ex.submit(
self._provider.create_completion,
result = run_with_deadline(
lambda: self._provider.create_completion(
client=client,
model=self._model,
messages=judge_messages,
@@ -423,27 +429,19 @@ class OutputGuardJudge:
max_tokens=512,
temperature=0.0,
reasoning_effort="low",
)
deadline = time.monotonic() + timeout
while True:
if cancel_event is not None and cancel_event.is_set():
future.cancel()
return self._error_verdict(verdict_id, call_id, start, "cancelled")
remaining = deadline - time.monotonic()
if remaining <= 0:
future.cancel()
return self._error_verdict(verdict_id, call_id, start, "timeout")
try:
result = future.result(timeout=min(remaining, 1.0))
break
except TimeoutError:
continue
except Exception as e:
return self._error_verdict(
verdict_id, call_id, start, f"provider_error: {type(e).__name__}"
)
finally:
ex.shutdown(wait=False, cancel_futures=True)
),
timeout=timeout,
cancel_event=cancel_event,
thread_name="output-guard-judge",
)
except DeadlineCancelledError:
return self._error_verdict(verdict_id, call_id, start, "cancelled")
except DeadlineExceededError:
return self._error_verdict(verdict_id, call_id, start, "timeout")
except Exception as e:
return self._error_verdict(
verdict_id, call_id, start, f"provider_error: {type(e).__name__}"
)
content = (getattr(result, "content", "") or "").strip()
if not content:
+6 -9
View File
@@ -14,10 +14,12 @@ request shaping. This module separates three concerns:
Stored under ``server_compat`` because it's an endpoint property,
not a model property.
3. **Server workarounds** ``extra_body`` overrides like
``skip_special_tokens=false`` are properties of the *server* (vLLM
bug workaround). These stay in ``server_compat`` and get merged
into the request's ``extra_body`` at call time.
3. **Server workarounds** ``extra_body`` overrides like llama.cpp's
``reasoning_format`` are properties of the *server*, not the model.
These stay in ``server_compat`` and get merged into the request's
``extra_body`` at call time. Reserve these for stable server config:
bug-workaround flags for fast-moving open models go stale the moment
the upstream bug is fixed, so we don't carry them speculatively.
Profiles are *suggestions* only. The admin UI auto-fills them on
Detect; the operator has final say, and the stored DB config is what
@@ -45,11 +47,6 @@ _PROFILES: dict[str, dict[str, Any]] = {
},
"server_compat": {
"server_type": "vllm",
# Workaround: vLLM strips special tokens before the Gemma4
# reasoning parser sees them. skip_special_tokens=false
# preserves <|channel> / <channel|> markers so reasoning
# content is extracted correctly.
"extra_body": {"skip_special_tokens": False},
},
},
"vllm-qwen-thinking": {
+26 -6
View File
@@ -2425,10 +2425,15 @@ class ChatSession:
ws_id = self._ws_id # Capture before async work
log.info("ws.title.gen_start", ws_id=ws_id[:8])
try:
# Gather first user message and first assistant reply
# Gather first user message and first assistant reply.
# Snapshot ``self.messages`` (C-level atomic copy under the
# GIL): this runs in a background thread that may now fire
# while the main ``send`` loop is still streaming and
# appending turns, so iterating the live list directly could
# raise "list changed size during iteration".
user_msg = ""
asst_msg = ""
for m in self.messages:
for m in list(self.messages):
content = m.text # joins text blocks; multipart attachments contribute none
if m.role is Role.USER and not user_msg:
user_msg = content[:300]
@@ -4150,6 +4155,25 @@ class ChatSession:
# legacy per-message ``_reminders`` side-channel splice.
self._emit_pending_user_nudges()
# Auto-title from the opening user message — fire NOW rather than
# waiting for the assistant's final tool-call-free turn. The old
# trigger sat in the ``not tool_calls`` branch of the loop below;
# coordinators spend nearly every turn in tool calls and may never
# reach that terminal text turn, so the title almost never
# generated for them. Gate on a real user message: synthetic wake
# sends carry no content and ``_generate_title`` would no-op on the
# empty/attachment-only case anyway (it needs first-user-message
# text). Concurrency: this background thread runs alongside the
# streaming turn started below, but safely — it snapshots
# ``self.messages`` for iteration, and the only UI it touches is
# ``on_aux_usage`` (storage/metrics, no ``_ws_lock`` state) and
# ``on_rename`` (queue/locked fan-out), both documented
# auxiliary-thread-safe on ``SessionUIBase``; the provider + client
# handle concurrent requests (the same path ``task_agent`` uses).
if not self._title_generated and user_input.strip() and not from_wake:
self._title_generated = True
threading.Thread(target=self._generate_title, daemon=True).start()
# A fresh session composed its system prefix at __init__ with an empty
# history, so memory selection fell back to recency (no query, no rerank).
# Recompose once the first real user message exists so the opening turn
@@ -4292,10 +4316,6 @@ class ChatSession:
self._compact_messages(auto=True)
# Update status bar with post-compaction token counts
self._print_status_line()
# Auto-title session after first exchange
if not self._title_generated:
self._title_generated = True
threading.Thread(target=self._generate_title, daemon=True).start()
# Flush any queued messages that weren't injected
# (no tool calls → no advisory seam to inject at).
# If anything drained, the model hasn't seen those
+118 -3
View File
@@ -493,9 +493,8 @@ class SharedSessionVerbHandlers:
"""Bundle of HTTP handler callables for verbs both kinds expose.
All handlers are optional; ``None`` skips that route. One bundle
describes either kind coord omits ``delete`` / ``refresh_title``
/ ``set_title`` / attachments; interactive populates every
interaction verb post-Stage-2.
describes either kind coord omits ``delete``; interactive
populates every interaction verb post-Stage-2.
"""
# Listing
@@ -972,6 +971,122 @@ def make_close_handler(
return close
def make_refresh_title_handler(cfg: SessionEndpointConfig) -> Handler:
"""Lifted body for ``POST {prefix}/{ws_id}/refresh-title``.
Regenerates the workstream title via a background LLM call
(:meth:`ChatSession.request_title_refresh`). Both kinds share the
auth mgr ws-lookup request sequence; the session must be live
in memory (``mgr.get``, not ``open``) since the refresh runs on the
loaded :class:`ChatSession`. The current display name is passed so
the generator is steered toward a *different* title on a manual
refresh.
"""
async def refresh_title(request: Request) -> Response:
import asyncio
from turnstone.core.memory import get_workstream_display_name
if cfg.permission_gate is not None:
err = cfg.permission_gate(request)
if err is not None:
return err
mgr_opt, err503 = cfg.manager_lookup(request)
if err503 is not None:
return err503
# See ``make_approve_handler`` for the cast rationale.
mgr = cast("SessionManager", mgr_opt)
ws_id = request.path_params.get("ws_id", "")
if cfg.tenant_check is not None:
err_tenant = await asyncio.to_thread(cfg.tenant_check, request, ws_id, mgr)
if err_tenant is not None:
return err_tenant
ws = mgr.get(ws_id)
if ws is None or ws.session is None:
return JSONResponse({"error": cfg.not_found_label}, status_code=404)
current_title = await asyncio.to_thread(get_workstream_display_name, ws_id) or ""
ws.session.request_title_refresh(current_title)
return JSONResponse({"status": "ok"})
return refresh_title
def make_set_title_handler(cfg: SessionEndpointConfig) -> Handler:
"""Lifted body for ``POST {prefix}/{ws_id}/title``.
Sets a user-chosen title manually. Stored as the workstream *alias*
so it outranks the LLM auto-title in the display fallback chain
(``alias > title > name``). Both kinds share the auth validate
``set_workstream_alias`` ``on_rename`` sequence. Returns 409 when
the name collides with another workstream's alias.
Behavior matches the pre-lift interactive handler: the alias is set
against storage regardless of whether the session is loaded (so a
saved/closed workstream can still be renamed), and the live
``on_rename`` broadcast fires only when the session is in memory.
"""
async def set_title(request: Request) -> Response:
import asyncio
from turnstone.core.memory import set_workstream_alias
from turnstone.core.web_helpers import read_json_or_400
if cfg.permission_gate is not None:
err = cfg.permission_gate(request)
if err is not None:
return err
mgr_opt, err503 = cfg.manager_lookup(request)
if err503 is not None:
return err503
mgr = cast("SessionManager", mgr_opt)
ws_id = request.path_params.get("ws_id", "")
if not ws_id:
return JSONResponse({"error": "ws_id is required"}, status_code=400)
if cfg.tenant_check is not None:
err_tenant = await asyncio.to_thread(cfg.tenant_check, request, ws_id, mgr)
if err_tenant is not None:
return err_tenant
# Resolve the workstream BEFORE writing the alias. ``set_workstream_alias``
# is a global, kind-unscoped UPDATE keyed on ``ws_id`` alone (it returns
# True even on a 0-row match), so a kind that has no ``tenant_check``
# storage gate (coord — the in-memory manager is its existence + kind
# authority) must 404 here, or an operator could rename a workstream this
# manager doesn't own (e.g. an interactive ws via the coord route) and a
# bogus id would silently 200. Interactive keeps ``tenant_check`` as its
# existence gate, so this stays skipped there and a non-loaded
# saved/closed ws still renames.
ws = mgr.get(ws_id)
if cfg.tenant_check is None and ws is None:
return JSONResponse({"error": cfg.not_found_label}, status_code=404)
body = await read_json_or_400(request)
if isinstance(body, JSONResponse):
return body
title = str(body.get("title", "")).strip()
if not title:
return JSONResponse({"error": "title is required"}, status_code=400)
title = title[:80]
if not await asyncio.to_thread(set_workstream_alias, ws_id, title):
return JSONResponse(
{"error": "That name is already used by another workstream"},
status_code=409,
)
if ws is not None and ws.session is not None and ws.session.ui is not None:
ws.session.ui.on_rename(title)
return JSONResponse({"status": "ok", "title": title})
return set_title
CancelAuditEmitter = Callable[
["Request", str, "Workstream", bool],
None,
+12 -2
View File
@@ -201,8 +201,18 @@ class SessionUIBase:
methods (and the approval blocking helpers that live on
subclasses); HTTP handlers drive ``_register_listener`` /
``_unregister_listener`` / ``resolve_approval`` from the event
loop. All shared state is guarded by ``_listeners_lock`` or
``threading.Event`` primitives.
loop. All shared state is guarded by ``_listeners_lock`` /
``_ws_lock`` or ``threading.Event`` primitives.
Two ``on_*`` methods are additionally safe to call from a
*concurrent* auxiliary thread (e.g. background title generation in
``ChatSession._generate_title``, or ``task_agent`` sub-agents), even
while the worker thread is mid-stream: :meth:`on_aux_usage` (a
storage ``usage_event`` write + thread-safe metric counters it
touches none of the ``_ws_lock``-guarded inflight state
:meth:`on_status`/token writers mutate) and :meth:`on_rename` (a
queue / locked fan-out). Keep those two free of unguarded
``_ws_*`` writes so the auxiliary-thread guarantee holds.
"""
def __init__(self, ws_id: str = "", user_id: str = "") -> None:
+2 -2
View File
@@ -577,7 +577,7 @@ def _build_registry() -> dict[str, SettingDef]:
SettingDef(
"judge.timeout",
"float",
60.0,
120.0,
"Judge evaluation timeout in seconds",
"judge",
min_value=5.0,
@@ -641,7 +641,7 @@ def _build_registry() -> dict[str, SettingDef]:
SettingDef(
"judge.output_guard_llm_timeout",
"float",
30.0,
60.0,
"Wall-clock budget for the output-guard LLM judge call",
"judge",
min_value=1.0,
+2
View File
@@ -8,6 +8,7 @@ from turnstone.core.storage._registry import (
StorageUnavailableError,
get_storage,
init_storage,
is_storage_initialized,
reset_storage,
)
@@ -17,5 +18,6 @@ __all__ = [
"StorageUnavailableError",
"get_storage",
"init_storage",
"is_storage_initialized",
"reset_storage",
]
+6
View File
@@ -1052,6 +1052,12 @@ class PostgreSQLBackend:
workstreams.c.skill_id,
workstreams.c.skill_version,
workstreams.c.user_id,
# Appended after ``user_id`` so positional fallbacks in
# consumers (``_coord_children_row`` et al.) that index
# up to row[9] stay valid; ``_coordinator_rows`` reads
# these by name to surface the persisted display title.
workstreams.c.title,
workstreams.c.alias,
)
.order_by(workstreams.c.updated.desc())
.limit(limit)
+3 -1
View File
@@ -679,7 +679,9 @@ class StorageBackend(Protocol):
Returns a list of SQLAlchemy ``Row`` objects. **Prefer dict access
via ``row._mapping[<col>]``**; positional indexing is brittle against
future SELECT reorders and against new columns appearing in the
tail (the select currently ends with ``user_id``).
tail (the select currently ends with ``user_id, title, alias``
``title``/``alias`` were appended after ``user_id`` so existing
positional fallbacks that index up to row[9] stay valid).
"""
...
+13
View File
@@ -124,6 +124,19 @@ def get_storage() -> StorageBackend:
return _storage
def is_storage_initialized() -> bool:
"""Return True when the storage singleton has been initialized.
Lets callers on lifecycle / early-startup paths consult storage
without tripping :func:`get_storage`'s SQLite auto-init side effect
(which would create ``.turnstone.db`` in the CWD). Use this to guard
a best-effort read that should simply be skipped before the host has
called :func:`init_storage` never as a substitute for the explicit
init the app's startup performs.
"""
return _storage is not None
def reset_storage() -> None:
"""Close and clear the storage backend singleton (for tests)."""
global _storage
+6
View File
@@ -1209,6 +1209,12 @@ class SQLiteBackend:
workstreams.c.skill_id,
workstreams.c.skill_version,
workstreams.c.user_id,
# Appended after ``user_id`` so positional fallbacks in
# consumers (``_coord_children_row`` et al.) that index
# up to row[9] stay valid; ``_coordinator_rows`` reads
# these by name to surface the persisted display title.
workstreams.c.title,
workstreams.c.alias,
)
.order_by(workstreams.c.updated.desc())
.limit(limit)
+12 -3
View File
@@ -294,8 +294,6 @@ class TLSClient:
Discovery, CA fetch, and cert request are all idempotent, so the whole
sequence is retried as a unit.
"""
import asyncio
if attempts < 1:
# range(1, attempts + 1) would be empty: init() would return
# "successfully" with no CA and no cert.
@@ -321,7 +319,18 @@ class TLSClient:
delay_seconds=delay,
error=f"{type(exc).__name__}: {exc}",
)
await asyncio.sleep(delay)
await self._sleep(delay)
async def _sleep(self, delay: float) -> None:
"""Backoff sleep behind a seam so tests can stub it in isolation.
Patching the module-global ``asyncio.sleep`` would also intercept it
for every other task sharing the event loop; routing the retry backoff
through a method keeps test stubs from corrupting concurrent tasks.
"""
import asyncio
await asyncio.sleep(delay)
def _discover_console_url(self) -> str:
"""Look up the console URL from the services table."""
+108 -71
View File
@@ -39,7 +39,7 @@ from sse_starlette import EventSourceResponse
from starlette.applications import Starlette
from starlette.middleware import Middleware
from starlette.requests import Request
from starlette.responses import HTMLResponse, JSONResponse, Response
from starlette.responses import HTMLResponse, JSONResponse, Response, StreamingResponse
from starlette.routing import Mount, Route
from starlette.staticfiles import StaticFiles
@@ -77,10 +77,12 @@ from turnstone.core.session_routes import (
make_history_handler,
make_list_handler,
make_open_handler,
make_refresh_title_handler,
make_retry_handler,
make_rewind_handler,
make_saved_handler,
make_send_handler,
make_set_title_handler,
register_session_routes,
)
from turnstone.core.session_ui_base import (
@@ -1290,6 +1292,100 @@ async def speech_to_text(request: Request) -> JSONResponse:
)
async def speech_to_text_stream(request: Request) -> Response:
"""POST /v1/api/workstreams/{ws_id}/speech-to-text/stream — stream the
transcript as plain-text deltas for lower perceived latency than the JSON
``speech-to-text`` endpoint. Resolve/transcode failures surface as
503 / 502 before any bytes are sent; once streaming begins the body is
best-effort (a mid-stream backend error just ends the partial stream)."""
from turnstone.core.audio import (
AudioBackendError,
AudioUnavailableError,
resolve_role_alias,
transcribe_stream,
)
from turnstone.core.web_helpers import read_multipart_file_or_400
ws_id = request.path_params.get("ws_id", "")
if not ws_id:
return JSONResponse({"error": "ws_id is required"}, status_code=400)
_user_id, err = _require_ws_access(request, ws_id)
if err:
return err
registry = getattr(request.app.state, "registry", None)
config_store = getattr(request.app.state, "config_store", None)
alias = resolve_role_alias(config_store=config_store, registry=registry, role="stt")
if not alias:
return JSONResponse(
{
"error": (
"Speech-to-text is not configured. Assign an STT model role in Models → Roles."
)
},
status_code=503,
)
got = await read_multipart_file_or_400(request, field="audio", max_bytes=_STT_UPLOAD_CAP)
if isinstance(got, JSONResponse):
return got
_filename, _claimed_mime, data = got
if not data:
return JSONResponse({"error": "Empty audio upload"}, status_code=400)
stt_prompt = ""
if config_store is not None:
stt_prompt = (config_store.get("audio.stt_prompt") or "").strip()
# Resolve + transcode + open the stream eagerly (off the event loop) so the
# common failures map to a clean status before any bytes are sent.
try:
deltas = await asyncio.to_thread(
transcribe_stream, registry=registry, alias=alias, data=data, prompt=stt_prompt
)
except AudioUnavailableError as exc:
return JSONResponse({"error": str(exc)}, status_code=503)
except AudioBackendError:
log.warning("speech_to_text_stream.backend_failed", exc_info=True)
return JSONResponse({"error": "Speech transcription backend failed"}, status_code=502)
# Drive the blocking stream from one worker thread that owns (and closes)
# the upstream connection, handing deltas to the loop via a queue. A client
# disconnect sets ``stop`` so the thread releases the connection promptly
# instead of being pinned mid-``next()`` (which can't be cancelled).
async def _body() -> AsyncGenerator[bytes, None]:
loop = asyncio.get_running_loop()
queue: asyncio.Queue[bytes | None] = asyncio.Queue()
stop = threading.Event()
def _pump() -> None:
try:
for delta in deltas:
if stop.is_set():
break
loop.call_soon_threadsafe(queue.put_nowait, delta.encode("utf-8"))
except Exception:
# Mid-stream backend failure: end the partial stream (logged).
log.warning("speech_to_text_stream.mid_stream_failed", exc_info=True)
finally:
close = getattr(deltas, "close", None)
if callable(close):
close()
loop.call_soon_threadsafe(queue.put_nowait, None)
loop.run_in_executor(None, _pump)
try:
while True:
chunk = await queue.get()
if chunk is None:
break
yield chunk
finally:
stop.set()
return StreamingResponse(_body(), media_type="text/plain; charset=utf-8")
async def text_to_speech(request: Request) -> Response:
"""POST /v1/api/tts — synthesize assistant text into playable audio."""
from turnstone.core.audio import (
@@ -2215,74 +2311,6 @@ async def delete_workstream_endpoint(request: Request) -> JSONResponse:
return JSONResponse({"error": "Delete failed"}, status_code=500)
async def refresh_workstream_title(request: Request, ws_id: str = "") -> JSONResponse:
"""POST /v1/api/workstreams/{ws_id}/refresh-title — regenerate workstream title via LLM."""
from turnstone.core.log import get_logger
from turnstone.core.memory import get_workstream_display_name
log = get_logger(__name__)
ws_id = request.path_params.get("ws_id", "")
log.info("ws.title.refresh_requested", ws_id=ws_id[:8] if ws_id else "empty")
mgr = request.app.state.workstreams
_owner, err = _require_ws_access(request, ws_id, mgr=mgr)
if err:
return err
ws = mgr.get(ws_id)
if not ws or not ws.session:
log.warning(
"ws.title.refresh_failed",
ws_id=ws_id[:8] if ws_id else "empty",
reason="workstream_not_found",
)
return JSONResponse({"error": "Workstream not found or not active"}, status_code=404)
# Fetch current title so the LLM can generate something different
current_title = get_workstream_display_name(ws_id) or ""
log.info("ws.title.refresh_triggered", ws_id=ws_id[:8], current_title=current_title[:50])
ws.session.request_title_refresh(current_title)
return JSONResponse({"status": "ok"})
async def set_workstream_title(request: Request, ws_id: str = "") -> JSONResponse:
"""POST /v1/api/workstreams/{ws_id}/title — set workstream title manually.
Stores the user-chosen title as the workstream *alias* so it takes
priority over the LLM auto-generated title in the display name
fallback chain (alias -> title -> name).
"""
from turnstone.core.log import get_logger
from turnstone.core.memory import set_workstream_alias
from turnstone.core.web_helpers import read_json_or_400
log = get_logger(__name__)
ws_id = request.path_params.get("ws_id", "")
log.info("ws.title.set_requested", ws_id=ws_id[:8] if ws_id else "empty")
if not ws_id:
return JSONResponse({"error": "ws_id is required"}, status_code=400)
mgr = request.app.state.workstreams
_owner, err = _require_ws_access(request, ws_id, mgr=mgr)
if err:
return err
body = await read_json_or_400(request)
if isinstance(body, JSONResponse):
return body
title = str(body.get("title", "")).strip()
if not title:
return JSONResponse({"error": "title is required"}, status_code=400)
title = title[:80]
if not set_workstream_alias(ws_id, title):
log.warning("ws.title.set_alias_conflict", ws_id=ws_id[:8], title=title[:50])
return JSONResponse(
{"error": "That name is already used by another workstream"},
status_code=409,
)
log.info("ws.title.set_alias_updated", ws_id=ws_id[:8])
ws = mgr.get(ws_id)
if ws and ws.session and ws.session.ui:
ws.session.ui.on_rename(title)
log.info("ws.title.set_success", ws_id=ws_id[:8], title=title)
return JSONResponse({"status": "ok", "title": title})
def _auth_user_id(request: Request) -> str:
"""Return the authenticated user's id (empty string when absent).
@@ -3865,6 +3893,8 @@ def create_app(
history_handler = make_history_handler(interactive_endpoint_config)
export_handler = make_export_handler(interactive_endpoint_config)
detail_handler = make_detail_handler(interactive_endpoint_config)
refresh_title_handler = make_refresh_title_handler(interactive_endpoint_config)
set_title_handler = make_set_title_handler(interactive_endpoint_config)
v1_routes: list[Any] = [
Route("/api/events/global", global_events_sse),
]
@@ -3879,8 +3909,8 @@ def create_app(
detail=detail_handler, # lifted: shared body (interactive feature gain)
open=open_handler, # lifted: shared body
close=close_handler, # lifted: shared body
refresh_title=refresh_workstream_title,
set_title=set_workstream_title,
refresh_title=refresh_title_handler, # lifted: shared body
set_title=set_title_handler, # lifted: shared body
send=send_handler, # lifted: shared body (P1.5)
dequeue=dequeue_handler, # lifted (P1.5) — DELETE /send
approve=approve_handler, # lifted: shared body
@@ -3901,6 +3931,13 @@ def create_app(
methods=["POST"],
)
)
v1_routes.append(
Route(
"/api/workstreams/{ws_id}/speech-to-text/stream",
speech_to_text_stream,
methods=["POST"],
)
)
v1_routes.append(Route("/api/tts", text_to_speech, methods=["POST"]))
app = Starlette(
+46 -21
View File
@@ -1580,32 +1580,62 @@ class Pane {
this._micBtn.classList.add("is-busy");
}
voiceAnnounce("Transcribing…");
const resetMic = () => {
if (this._micBtn && !this._micDenied) {
this._micBtn.disabled = !!this.busy;
this._micBtn.classList.remove("is-busy");
}
};
authFetch(
this._base +
"/v1/api/workstreams/" +
encodeURIComponent(this.wsId) +
"/speech-to-text",
"/speech-to-text/stream",
{ method: "POST", body: fd },
)
.then((r) => r.json().then((body) => ({ ok: r.ok, body })))
.then((res) => {
if (!res.ok) {
showToast(
(res.body && res.body.error) || "Transcription failed",
"error",
);
.then(async (r) => {
if (!r.ok) {
let msg = "Transcription failed";
try {
const body = await r.json();
if (body && body.error) msg = body.error;
} catch (_e) {
/* non-JSON error body */
}
showToast(msg, "error");
return;
}
const text = (res.body && res.body.transcript) || "";
if (text && this.inputEl) {
const cur = this.inputEl.value || "";
this.inputEl.value = cur
? cur.replace(/\s*$/, "") + " " + text
: text;
// Stream transcript deltas into the composer as they arrive (first word
// in ~0.3s) instead of waiting for the whole transcript.
const reader = r.body.getReader();
const decoder = new TextDecoder();
let started = false;
let got = false;
while (true) {
const { done, value } = await reader.read();
if (done) break;
const chunk = decoder.decode(value, { stream: true });
if (!chunk || !this.inputEl) continue;
got = true;
if (!started) {
// Read the composer's value now (not before the await) so text the
// user typed while transcribing isn't clobbered.
const cur = this.inputEl.value || "";
this.inputEl.value = cur
? cur.replace(/\s*$/, "") + " " + chunk
: chunk;
started = true;
} else {
this.inputEl.value += chunk;
}
// Drive the composer's auto-resize + send-enable listeners.
this.inputEl.dispatchEvent(new Event("input", { bubbles: true }));
this.inputEl.focus();
}
if (got) {
if (this.inputEl) this.inputEl.focus();
voiceAnnounce("Transcript added to message.");
} else {
showToast("No speech detected", "error");
}
})
.catch((err) => {
@@ -1614,12 +1644,7 @@ class Pane {
"error",
);
})
.finally(() => {
if (this._micBtn && !this._micDenied) {
this._micBtn.disabled = !!this.busy;
this._micBtn.classList.remove("is-busy");
}
});
.finally(resetMic);
}
_addTtsAction(el) {
+5
View File
@@ -837,6 +837,11 @@ async function mountShell() {
});
pane.tabMenu = () =>
convTabMenu(pane, pm, id, {
// Coordinators carry titles like interactive workstreams now:
// surface Refresh/Edit title. The default base ("") targets the
// console origin, where the coord refresh-title / title routes
// are mounted (same base coordinator.js posts every verb to).
titleVerbs: true,
closeSession: () => {
if (pane._ctl && pane._ctl.closeSession) pane._ctl.closeSession();
},
Generated
+53 -56
View File
@@ -595,61 +595,58 @@ wheels = [
[[package]]
name = "cryptography"
version = "48.0.0"
version = "49.0.0"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "cffi", marker = "platform_python_implementation != 'PyPy'" },
]
sdist = { url = "https://files.pythonhosted.org/packages/9f/a9/db8f313fdcd85d767d4973515e1db101f9c71f95fced83233de224673757/cryptography-48.0.0.tar.gz", hash = "sha256:5c3932f4436d1cccb036cb0eaef46e6e2db91035166f1ad6505c3c9d5a635920", size = 832984, upload-time = "2026-05-04T22:59:38.133Z" }
sdist = { url = "https://files.pythonhosted.org/packages/1f/99/d1c90d6041656cc6ee229dc99cd67fd0cd5aec3c5f7d72fffc27cc750054/cryptography-49.0.0.tar.gz", hash = "sha256:f89660a348f4f78a92366240a61404e337586ef7f5909a2fef59ca88ef505493", size = 854345, upload-time = "2026-06-12T20:02:30.512Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/df/3d/01f6dd9190170a5a241e0e98c2d04be3664a9e6f5b9b872cde63aff1c3dd/cryptography-48.0.0-cp311-abi3-macosx_10_9_universal2.whl", hash = "sha256:0c558d2cdffd8f4bbb30fc7134c74d2ca9a476f830bb053074498fbc86f41ed6", size = 8001587, upload-time = "2026-05-04T22:57:36.803Z" },
{ url = "https://files.pythonhosted.org/packages/b2/6e/e90527eef33f309beb811cf7c982c3aeffcce8e3edb178baa4ca3ae4a6fa/cryptography-48.0.0-cp311-abi3-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:f5333311663ea94f75dd408665686aaf426563556bb5283554a3539177e03b8c", size = 4690433, upload-time = "2026-05-04T22:57:40.373Z" },
{ url = "https://files.pythonhosted.org/packages/90/04/673510ed51ddff56575f306cf1617d80411ee76831ccd3097599140efdfe/cryptography-48.0.0-cp311-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:7995ef305d7165c3f11ae07f2517e5a4f1d5c18da1376a0a9ed496336b69e5f3", size = 4710620, upload-time = "2026-05-04T22:57:42.935Z" },
{ url = "https://files.pythonhosted.org/packages/14/d5/e9c4ef932c8d800490c34d8bd589d64a31d5890e27ec9e9ad532be893294/cryptography-48.0.0-cp311-abi3-manylinux_2_28_aarch64.whl", hash = "sha256:40ba1f85eaa6959837b1d51c9767e230e14612eea4ef110ee8854ada22da1bf5", size = 4696283, upload-time = "2026-05-04T22:57:45.294Z" },
{ url = "https://files.pythonhosted.org/packages/0c/29/174b9dfb60b12d59ecfc6cfa04bc88c21b42a54f01b8aae09bb6e51e4c7f/cryptography-48.0.0-cp311-abi3-manylinux_2_28_ppc64le.whl", hash = "sha256:369a6348999f94bbd53435c894377b20ab95f25a9065c283570e70150d8abc3c", size = 5296573, upload-time = "2026-05-04T22:57:47.933Z" },
{ url = "https://files.pythonhosted.org/packages/95/38/0d29a6fd7d0d1373f0c0c88a04ba20e359b257753ac497564cd660fc1d55/cryptography-48.0.0-cp311-abi3-manylinux_2_28_x86_64.whl", hash = "sha256:a0e692c683f4df67815a2d258b324e66f4738bd7a96a218c826dce4f4bd05d8f", size = 4743677, upload-time = "2026-05-04T22:57:50.067Z" },
{ url = "https://files.pythonhosted.org/packages/30/be/eef653013d5c63b6a490529e0316f9ac14a37602965d4903efed1399f32b/cryptography-48.0.0-cp311-abi3-manylinux_2_31_armv7l.whl", hash = "sha256:18349bbc56f4743c8b12dc32e2bccb2cf83ee8b69a3bba74ef8ae857e26b3d25", size = 4330808, upload-time = "2026-05-04T22:57:52.301Z" },
{ url = "https://files.pythonhosted.org/packages/84/9e/500463e87abb7a0a0f9f256ec21123ecde0a7b5541a15e840ea54551fd81/cryptography-48.0.0-cp311-abi3-manylinux_2_34_aarch64.whl", hash = "sha256:7e8eac43dfca5c4cccc6dad9a80504436fca53bb9bc3100a2386d730fbe6b602", size = 4695941, upload-time = "2026-05-04T22:57:54.603Z" },
{ url = "https://files.pythonhosted.org/packages/e3/dc/7303087450c2ec9e7fbb750e17c2abfbc658f23cbd0e54009509b7cc4091/cryptography-48.0.0-cp311-abi3-manylinux_2_34_ppc64le.whl", hash = "sha256:9ccdac7d40688ecb5a3b4a604b8a88c8002e3442d6c60aead1db2a89a041560c", size = 5252579, upload-time = "2026-05-04T22:57:57.207Z" },
{ url = "https://files.pythonhosted.org/packages/d0/c0/7101d3b7215edcdc90c45da544961fd8ed2d6448f77577460fa75a8443f7/cryptography-48.0.0-cp311-abi3-manylinux_2_34_x86_64.whl", hash = "sha256:bd72e68b06bb1e96913f97dd4901119bc17f39d4586a5adf2d3e47bc2b9d58b5", size = 4743326, upload-time = "2026-05-04T22:57:59.535Z" },
{ url = "https://files.pythonhosted.org/packages/ac/d8/5b833bad13016f562ab9d063d68199a4bd121d18458e439515601d3357ec/cryptography-48.0.0-cp311-abi3-musllinux_1_2_aarch64.whl", hash = "sha256:59baa2cb386c4f0b9905bd6eb4c2a79a69a128408fd31d32ca4d7102d4156321", size = 4826672, upload-time = "2026-05-04T22:58:01.996Z" },
{ url = "https://files.pythonhosted.org/packages/98/e1/7074eb8bf3c135558c73fc2bcf0f5633f912e6fb87e868a55c454080ef09/cryptography-48.0.0-cp311-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:9249e3cd978541d665967ac2cb2787fd6a62bddf1e75b3e347a594d7dacf4f74", size = 4972574, upload-time = "2026-05-04T22:58:03.968Z" },
{ url = "https://files.pythonhosted.org/packages/04/70/e5a1b41d325f797f39427aa44ef8baf0be500065ab6d8e10369d850d4a4f/cryptography-48.0.0-cp311-abi3-win32.whl", hash = "sha256:9c459db21422be75e2809370b829a87eb37f74cd785fc4aa9ea1e5f43b47cda4", size = 3294868, upload-time = "2026-05-04T22:58:06.467Z" },
{ url = "https://files.pythonhosted.org/packages/f4/ac/8ac51b4a5fc5932eb7ee5c517ba7dc8cd834f0048962b6b352f00f41ebf9/cryptography-48.0.0-cp311-abi3-win_amd64.whl", hash = "sha256:5b012212e08b8dd5edc78ef54da83dd9892fd9105323b3993eff6bea65dc21d7", size = 3817107, upload-time = "2026-05-04T22:58:08.845Z" },
{ url = "https://files.pythonhosted.org/packages/6b/84/70e3feea9feea87fd7cbe77efb2712ae1e3e6edf10749dc6e95f4e60e455/cryptography-48.0.0-cp314-cp314t-macosx_10_9_universal2.whl", hash = "sha256:3cb07a3ed6431663cd321ea8a000a1314c74211f823e4177fefa2255e057d1ec", size = 7986556, upload-time = "2026-05-04T22:58:11.172Z" },
{ url = "https://files.pythonhosted.org/packages/89/6e/18e07a618bb5442ba10cf4df16e99c071365528aa570dfcb8c02e25a303b/cryptography-48.0.0-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:8c7378637d7d88016fa6791c159f698b3d3eed28ebf844ac36b9dc04a14dae18", size = 4684776, upload-time = "2026-05-04T22:58:13.712Z" },
{ url = "https://files.pythonhosted.org/packages/be/6a/4ea3b4c6c6759794d5ee2103c304a5076dc4b19ae1f9fe47dba439e159e9/cryptography-48.0.0-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:cc90c0b39b2e3c65ef52c804b72e3c58f8a04ab2a1871272798e5f9572c17d20", size = 4698121, upload-time = "2026-05-04T22:58:16.448Z" },
{ url = "https://files.pythonhosted.org/packages/2f/59/6ff6ad6cae03bb887da2a5860b2c9805f8dac969ef01ce563336c49bd1d1/cryptography-48.0.0-cp314-cp314t-manylinux_2_28_aarch64.whl", hash = "sha256:76341972e1eff8b4bea859f09c0d3e64b96ce931b084f9b9b7db8ef364c30eff", size = 4690042, upload-time = "2026-05-04T22:58:18.544Z" },
{ url = "https://files.pythonhosted.org/packages/ca/b4/fc334ed8cfd705aca282fe4d8f5ae64a8e0f74932e9feecb344610cf6e4d/cryptography-48.0.0-cp314-cp314t-manylinux_2_28_ppc64le.whl", hash = "sha256:55b7718303bf06a5753dcdccf2f3945cf18ad7bffde41b61226e4db31ab89a9c", size = 5282526, upload-time = "2026-05-04T22:58:20.75Z" },
{ url = "https://files.pythonhosted.org/packages/11/08/9f8c5386cc4cd90d8255c7cdd0f5baf459a08502a09de30dc51f553d38dc/cryptography-48.0.0-cp314-cp314t-manylinux_2_28_x86_64.whl", hash = "sha256:a64697c641c7b1b2178e573cbc31c7c6684cd56883a478d75143dbb7118036db", size = 4733116, upload-time = "2026-05-04T22:58:23.627Z" },
{ url = "https://files.pythonhosted.org/packages/b8/77/99307d7574045699f8805aa500fa0fb83422d115b5400a064ddd306d7750/cryptography-48.0.0-cp314-cp314t-manylinux_2_31_armv7l.whl", hash = "sha256:561215ea3879cb1cbbf272867e2efda62476f240fb58c64de6b393ae19246741", size = 4316030, upload-time = "2026-05-04T22:58:25.581Z" },
{ url = "https://files.pythonhosted.org/packages/fd/36/a608b98337af3cb2aff4818e406649d30572b7031918b04c87d979495348/cryptography-48.0.0-cp314-cp314t-manylinux_2_34_aarch64.whl", hash = "sha256:ad64688338ed4bc1a6618076ba75fd7194a5f1797ac60b47afe926285adb3166", size = 4689640, upload-time = "2026-05-04T22:58:27.747Z" },
{ url = "https://files.pythonhosted.org/packages/dd/a6/825010a291b4438aecc1f568bc428189fc1175515223632477c07dc0a6df/cryptography-48.0.0-cp314-cp314t-manylinux_2_34_ppc64le.whl", hash = "sha256:906cbf0670286c6e0044156bc7d4af9cbb0ef6db9f73e52c3ec56ba6bdde5336", size = 5237657, upload-time = "2026-05-04T22:58:29.848Z" },
{ url = "https://files.pythonhosted.org/packages/b9/09/4e76a09b4caa29aad535ddc806f5d4c5d01885bd978bd984fbc6ca032cae/cryptography-48.0.0-cp314-cp314t-manylinux_2_34_x86_64.whl", hash = "sha256:ea8990436d914540a40ab24b6a77c0969695ed52f4a4874c5137ccf7045a7057", size = 4732362, upload-time = "2026-05-04T22:58:32.009Z" },
{ url = "https://files.pythonhosted.org/packages/18/78/444fa04a77d0cb95f417dda20d450e13c56ba8e5220fc892a1658f44f882/cryptography-48.0.0-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:c18684a7f0cc9a3cb60328f496b8e3372def7c5d2df39ac267878b05565aaaae", size = 4819580, upload-time = "2026-05-04T22:58:34.254Z" },
{ url = "https://files.pythonhosted.org/packages/38/85/ea67067c70a1fd4be2c63d35eeed82658023021affccc7b17705f8527dd2/cryptography-48.0.0-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:9be5aafa5736574f8f15f262adc81b2a9869e2cfe9014d52a44633905b40d52c", size = 4963283, upload-time = "2026-05-04T22:58:36.376Z" },
{ url = "https://files.pythonhosted.org/packages/75/54/cc6d0f3deac3e81c7f847e8a189a12b6cdd65059b43dad25d4316abd849a/cryptography-48.0.0-cp314-cp314t-win32.whl", hash = "sha256:c17dfe85494deaeddc5ce251aebd1d60bbe6afc8b62071bb0b469431a000124f", size = 3270954, upload-time = "2026-05-04T22:58:38.791Z" },
{ url = "https://files.pythonhosted.org/packages/49/67/cc947e288c0758a4e5473d1dcb743037ab7785541265a969240b8885441a/cryptography-48.0.0-cp314-cp314t-win_amd64.whl", hash = "sha256:27241b1dc9962e056062a8eef1991d02c3a24569c95975bd2322a8a52c6e5e12", size = 3797313, upload-time = "2026-05-04T22:58:40.746Z" },
{ url = "https://files.pythonhosted.org/packages/f2/63/61d4a4e1c6b6bab6ce1e213cd36a24c415d90e76d78c5eb8577c5541d2e8/cryptography-48.0.0-cp39-abi3-macosx_10_9_universal2.whl", hash = "sha256:58d00498e8933e4a194f3076aee1b4a97dfec1a6da444535755822fe5d8b0b86", size = 7983482, upload-time = "2026-05-04T22:58:43.769Z" },
{ url = "https://files.pythonhosted.org/packages/d5/ac/f5b5995b87770c693e2596559ffafe195b4033a57f14a82268a2842953f3/cryptography-48.0.0-cp39-abi3-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:614d0949f4790582d2cc25553abd09dd723025f0c0e7c67376a1d77196743d6e", size = 4683266, upload-time = "2026-05-04T22:58:46.064Z" },
{ url = "https://files.pythonhosted.org/packages/ec/c6/8b14f67e18338fbc4adb76f66c001f5c3610b3e2d1837f268f47a347dbbb/cryptography-48.0.0-cp39-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:7ce4bfae76319a532a2dc68f82cc32f5676ee792a983187dac07183690e5c66f", size = 4696228, upload-time = "2026-05-04T22:58:48.22Z" },
{ url = "https://files.pythonhosted.org/packages/ea/73/f808fbae9514bd91b47875b003f13e284c8c6bdfd904b7944e803937eec1/cryptography-48.0.0-cp39-abi3-manylinux_2_28_aarch64.whl", hash = "sha256:2eb992bbd4661238c5a397594c83f5b4dc2bc5b848c365c8f991b6780efcc5c7", size = 4689097, upload-time = "2026-05-04T22:58:50.9Z" },
{ url = "https://files.pythonhosted.org/packages/93/01/d86632d7d28db8ae83221995752eeb6639ffb374c2d22955648cf8d52797/cryptography-48.0.0-cp39-abi3-manylinux_2_28_ppc64le.whl", hash = "sha256:22a5cb272895dce158b2cacdfdc3debd299019659f42947dbdac6f32d68fe832", size = 5283582, upload-time = "2026-05-04T22:58:53.017Z" },
{ url = "https://files.pythonhosted.org/packages/02/e1/50edc7a50334807cc4791fc4a0ce7468b4a1416d9138eab358bfc9a3d70b/cryptography-48.0.0-cp39-abi3-manylinux_2_28_x86_64.whl", hash = "sha256:2b4d59804e8408e2fea7d1fbaf218e5ec984325221db76e6a241a9abd6cdd95c", size = 4730479, upload-time = "2026-05-04T22:58:55.611Z" },
{ url = "https://files.pythonhosted.org/packages/6f/af/99a582b1b1641ff5911ac559beb45097cf79efd4ead4657f578ef1af2d47/cryptography-48.0.0-cp39-abi3-manylinux_2_31_armv7l.whl", hash = "sha256:984a20b0f62a26f48a3396c72e4bc34c66e356d356bf370053066b3b6d54634a", size = 4326481, upload-time = "2026-05-04T22:58:57.607Z" },
{ url = "https://files.pythonhosted.org/packages/90/ee/89aa26a06ef0a7d7611788ffd571a7c50e368cc6a4d5eef8b4884e866edb/cryptography-48.0.0-cp39-abi3-manylinux_2_34_aarch64.whl", hash = "sha256:5a5ed8fde7a1d09376ca0b40e68cd59c69fe23b1f9768bd5824f54681626032a", size = 4688713, upload-time = "2026-05-04T22:59:00.077Z" },
{ url = "https://files.pythonhosted.org/packages/70/ba/bcb1b0bb7a33d4c7c0c4d4c7874b4a62ae4f56113a5f4baefa362dfb1f0f/cryptography-48.0.0-cp39-abi3-manylinux_2_34_ppc64le.whl", hash = "sha256:8cd666227ef7af430aa5914a9910e0ddd703e75f039cef0825cd0da71b6b711a", size = 5238165, upload-time = "2026-05-04T22:59:02.317Z" },
{ url = "https://files.pythonhosted.org/packages/c9/70/ca4003b1ce5ca3dc3186ada51908c8a9b9ff7d5cab83cc0d43ee14ec144f/cryptography-48.0.0-cp39-abi3-manylinux_2_34_x86_64.whl", hash = "sha256:9071196d81abc88b3516ac8cdfad32e2b66dd4a5393a8e68a961e9161ddc6239", size = 4729947, upload-time = "2026-05-04T22:59:05.255Z" },
{ url = "https://files.pythonhosted.org/packages/44/a0/4ec7cf774207905aef1a8d11c3750d5a1db805eb380ee4e16df317870128/cryptography-48.0.0-cp39-abi3-musllinux_1_2_aarch64.whl", hash = "sha256:1e2d54c8be6152856a36f0882ab231e70f8ec7f14e93cf87db8a2ed056bf160c", size = 4822059, upload-time = "2026-05-04T22:59:07.802Z" },
{ url = "https://files.pythonhosted.org/packages/1e/75/a2e55f99c16fcac7b5d6c1eb19ad8e00799854d6be5ca845f9259eae1681/cryptography-48.0.0-cp39-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:a5da777e32ffed6f85a7b2b3f7c5cbc88c146bfcd0a1d7baf5fcc6c52ee35dd4", size = 4960575, upload-time = "2026-05-04T22:59:09.851Z" },
{ url = "https://files.pythonhosted.org/packages/b8/23/6e6f32143ab5d8b36ca848a502c4bcd477ae75b9e1677e3530d669062578/cryptography-48.0.0-cp39-abi3-win32.whl", hash = "sha256:77a2ccbbe917f6710e05ba9adaa25fb5075620bf3ea6fb751997875aff4ae4bd", size = 3279117, upload-time = "2026-05-04T22:59:12.019Z" },
{ url = "https://files.pythonhosted.org/packages/9d/9a/0fea98a70cf1749d41d738836f6349d97945f7c89433a259a6c2642eefeb/cryptography-48.0.0-cp39-abi3-win_amd64.whl", hash = "sha256:16cd65b9330583e4619939b3a3843eec1e6e789744bb01e7c7e2e62e33c239c8", size = 3792100, upload-time = "2026-05-04T22:59:14.884Z" },
{ url = "https://files.pythonhosted.org/packages/be/d2/024b5e06be9d44cb021fb0e1a03d34d63989cf56a0fe62f3dfbab695b9b4/cryptography-48.0.0-pp311-pypy311_pp73-macosx_11_0_arm64.whl", hash = "sha256:84cf79f0dc8b36ac5da873481716e87aef31fcfa0444f9e1d8b4b2cece142855", size = 3950391, upload-time = "2026-05-04T22:59:17.415Z" },
{ url = "https://files.pythonhosted.org/packages/bc/17/3861e17c56fa0fd37491a14a8673fdb77c57fc5693cafe745ea8b06dba75/cryptography-48.0.0-pp311-pypy311_pp73-manylinux_2_28_aarch64.whl", hash = "sha256:fdfef35d751d510fcef5252703621574364fec16418c4a1e5e1055248401054b", size = 4637126, upload-time = "2026-05-04T22:59:20.197Z" },
{ url = "https://files.pythonhosted.org/packages/f0/0a/7e226dbff530f21480727eb764973a7bff2b912f8e15cd4f129e71b56d1d/cryptography-48.0.0-pp311-pypy311_pp73-manylinux_2_28_x86_64.whl", hash = "sha256:0890f502ddf7d9c6426129c3f49f5c0a39278ed7cd6322c8755ffca6ee675a13", size = 4667270, upload-time = "2026-05-04T22:59:22.647Z" },
{ url = "https://files.pythonhosted.org/packages/3b/f2/5a72274ca9f1b2a8b44a662ee0bf1b435909deb473d6f97bcd035bcdbc71/cryptography-48.0.0-pp311-pypy311_pp73-manylinux_2_34_aarch64.whl", hash = "sha256:ecde28a596bead48b0cfd2a1b4416c3d43074c2d785e3a398d7ec1fc4d0f7fbb", size = 4636797, upload-time = "2026-05-04T22:59:24.912Z" },
{ url = "https://files.pythonhosted.org/packages/b4/e1/48cedb2fe63626e91ded1edad159e2a4fb8b6906c4425eb7749673077ce7/cryptography-48.0.0-pp311-pypy311_pp73-manylinux_2_34_x86_64.whl", hash = "sha256:4defde8685ae324a9eb9d818717e93b4638ef67070ac9bc15b8ca85f63048355", size = 4666800, upload-time = "2026-05-04T22:59:27.474Z" },
{ url = "https://files.pythonhosted.org/packages/a2/ca/7e8365deec19afb2b2c7be7c1c0aa8f99633b54e90c570999acda93260fc/cryptography-48.0.0-pp311-pypy311_pp73-win_amd64.whl", hash = "sha256:db63bf618e5dea46c07de12e900fe1cdd2541e6dc9dbae772a70b7d4d4765f6a", size = 3739536, upload-time = "2026-05-04T22:59:29.61Z" },
{ url = "https://files.pythonhosted.org/packages/9b/22/adf66990e63584a68dfb50c24f48a125c07b1699899381c8151e63ed458c/cryptography-49.0.0-cp311-abi3-macosx_11_0_arm64.whl", hash = "sha256:966fe0e9c67490071f14c0d2b1cb2dfb3023c5ce39457343931415f08382f2db", size = 4032100, upload-time = "2026-06-12T20:02:32.143Z" },
{ url = "https://files.pythonhosted.org/packages/09/41/3797cfaf69cae04a13ee78ebd83f0678d9c02b4779d21ce24445326f1a69/cryptography-49.0.0-cp311-abi3-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:36d1709f992593689b45bda411498d62c6e365f2ca00b84657d4dadd24de16db", size = 4692978, upload-time = "2026-06-12T20:01:21.305Z" },
{ url = "https://files.pythonhosted.org/packages/e6/8b/43011f7ebe515a8aa20d61f290a326cd890c2e738e16e59eaff8d9c3a412/cryptography-49.0.0-cp311-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:0e959b578856a3924bc0cbb710fc12c387b9412a951389f3ca61704a9e25f325", size = 4716422, upload-time = "2026-06-12T20:01:48.566Z" },
{ url = "https://files.pythonhosted.org/packages/4a/91/01ce7303a4579e6d3a6abef01bd322848e9ea7a219adcabc5048b9033571/cryptography-49.0.0-cp311-abi3-manylinux_2_28_aarch64.whl", hash = "sha256:53ecee2e23f7169b6117e99fc8a944e5e50f79e69758a83b52a00cb98ab2b2d2", size = 4700503, upload-time = "2026-06-12T20:02:47.091Z" },
{ url = "https://files.pythonhosted.org/packages/62/99/a2c95cf8293f07491e9e27c20cc4dcd18176d944e674679adeb1d0173fd6/cryptography-49.0.0-cp311-abi3-manylinux_2_28_ppc64le.whl", hash = "sha256:2eda353d8a27bcbcaa4cbed18994a74ab4d19a2ca897db188ea269ab9b71419b", size = 5309779, upload-time = "2026-06-12T20:02:08.987Z" },
{ url = "https://files.pythonhosted.org/packages/20/2c/0622f20ff02b2ef32558733443805dc82fd4c275be01b2d19d14676f3a1b/cryptography-49.0.0-cp311-abi3-manylinux_2_28_x86_64.whl", hash = "sha256:2afe9051da7ae7bd5905da5a949280c7d2bb75682e188f650a9d0f2756b834c6", size = 4749683, upload-time = "2026-06-12T20:02:03.335Z" },
{ url = "https://files.pythonhosted.org/packages/a3/5b/c5246635d5fd3b64e0d45ae10e99fd32fe9676a79915ccfe5a61ba9af1a5/cryptography-49.0.0-cp311-abi3-manylinux_2_31_armv7l.whl", hash = "sha256:0b82e28ee398a386f0807bba7884d30f25218855690f45115831bcce5d90822c", size = 4337874, upload-time = "2026-06-12T20:02:54.323Z" },
{ url = "https://files.pythonhosted.org/packages/6d/88/05563c7fe2e914e87d1a536d06fe83e66b4e1d95cb593e05aea375531da8/cryptography-49.0.0-cp311-abi3-manylinux_2_34_aarch64.whl", hash = "sha256:ccac2bfebc306b862133e3bb71f3f6ee8bb525240089b2d952e4144b3a6d5da7", size = 4700283, upload-time = "2026-06-12T20:01:34.822Z" },
{ url = "https://files.pythonhosted.org/packages/c4/b6/d7696e4e890d6ae1469935164c9e5215c557671cb78d6e3f458ccceaa632/cryptography-49.0.0-cp311-abi3-manylinux_2_34_ppc64le.whl", hash = "sha256:d0527ce944105f257f605a827d6ebead966c752038b6e8656abb9c5edee6fc68", size = 5265844, upload-time = "2026-06-12T20:01:24.09Z" },
{ url = "https://files.pythonhosted.org/packages/a9/3c/f3ad17eecc1a57b0ba236dc01f90e783c51f4a2f35f64777cc4f47a184b2/cryptography-49.0.0-cp311-abi3-manylinux_2_34_x86_64.whl", hash = "sha256:cbc77da8c523d5abd028635ba850a6966fcee2c82e2bf65a41d1d8afe0f98be9", size = 4749290, upload-time = "2026-06-12T20:01:30.848Z" },
{ url = "https://files.pythonhosted.org/packages/4f/01/339573cf1023163a400b0b5d16f6d507de413b9f60be6fd1b77feeaf6737/cryptography-49.0.0-cp311-abi3-musllinux_1_2_aarch64.whl", hash = "sha256:b87e65d263b3e5d3bb92a57e2a6638e2f31110fa7aa890c7b2dbba42248d0a3f", size = 4834612, upload-time = "2026-06-12T20:01:29.246Z" },
{ url = "https://files.pythonhosted.org/packages/71/fd/577302e213a1be9468f92d1afef66fcf1ef83d516819d9992ca547f592bd/cryptography-49.0.0-cp311-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:66ec79c3904820572d7e987abdf304281f141d37ad9a489b8e97066e7b9b6459", size = 4980804, upload-time = "2026-06-12T20:01:42.853Z" },
{ url = "https://files.pythonhosted.org/packages/1f/09/f42b1d190c5ba75f72062a387f8030d1d75f6ab035788f1d9c4b01de6525/cryptography-49.0.0-cp311-abi3-win_amd64.whl", hash = "sha256:e5dfc1e64de5677cec922ffa8da89c546d0415bf6efdf081842e5d44c84e1f0e", size = 3810026, upload-time = "2026-06-12T20:02:39.262Z" },
{ url = "https://files.pythonhosted.org/packages/ec/9e/db72b3ae7fc9cfad53e630e56c6ae83b9b6ff0bf3718ffb8012d20b3aabf/cryptography-49.0.0-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:73a205dce83953d131a4aa1e0fd917a2fd1c5b1eef251e9d7152efefcbf5caf7", size = 4013892, upload-time = "2026-06-12T20:02:10.735Z" },
{ url = "https://files.pythonhosted.org/packages/86/12/c48a424f38db03027be9f7ed5c7dc5de9933dbee992865f98b13727a009d/cryptography-49.0.0-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:196ecd6a36e4e9aa10270393bb98d8df88fccee0bf1e5128b91ae4eb4375896d", size = 4678835, upload-time = "2026-06-12T20:02:48.743Z" },
{ url = "https://files.pythonhosted.org/packages/68/28/8a3ad4653662c93fc44dc4e5d8fd374c25c42e07b34bbfbadf49cf57a5a8/cryptography-49.0.0-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:7abcee80084cda3f7691f3eb1ce480d8df49cec637b429aa35986c1de71738aa", size = 4697239, upload-time = "2026-06-12T20:02:56.03Z" },
{ url = "https://files.pythonhosted.org/packages/a8/b2/2193fc74f81aee4f9b62733133b73b5176718932ed8f2e4b03fa040480a6/cryptography-49.0.0-cp314-cp314t-manylinux_2_28_aarch64.whl", hash = "sha256:4ae387c9cb68ea569ca17e490d66d8142b81c3cc814bf179974b7d146e490bbb", size = 4685593, upload-time = "2026-06-12T20:02:50.666Z" },
{ url = "https://files.pythonhosted.org/packages/47/f1/1d3eaa243bfc5de4a187b22aa8c048b3e4980bfbe830ac46e6bac2e66947/cryptography-49.0.0-cp314-cp314t-manylinux_2_28_ppc64le.whl", hash = "sha256:f37d847238971164fdbc68ade6f6574aecc9c0af714190e2083429ff68f4ce9d", size = 5289961, upload-time = "2026-06-12T20:01:46.468Z" },
{ url = "https://files.pythonhosted.org/packages/58/39/2d51306721330c486495853eda1c567880ff036de15a14c4b74f399934af/cryptography-49.0.0-cp314-cp314t-manylinux_2_28_x86_64.whl", hash = "sha256:c2bc30226390d60ea19d9f82b19db005fe0452154a23c1c410c12ea801e43561", size = 4731145, upload-time = "2026-06-12T20:02:16.832Z" },
{ url = "https://files.pythonhosted.org/packages/17/50/983e838c7fd0d87fd8c969bcdd328edaf5f756e38df5281637424c155873/cryptography-49.0.0-cp314-cp314t-manylinux_2_31_armv7l.whl", hash = "sha256:07cab27cc7b7e0fd28e5e26bb9eeedde5c135c868b46de4a27845abe94af6122", size = 4321719, upload-time = "2026-06-12T20:02:52.611Z" },
{ url = "https://files.pythonhosted.org/packages/a7/f5/8f571d7e27c55bce9f76f026143bcb1e040a4233149ecca0bea5fa5dd5f7/cryptography-49.0.0-cp314-cp314t-manylinux_2_34_aarch64.whl", hash = "sha256:b20133d204d2bb56ba047642199603876c872026ca53e79c35b83772ab2cc505", size = 4685209, upload-time = "2026-06-12T20:02:07.282Z" },
{ url = "https://files.pythonhosted.org/packages/e7/84/0e27016a6fc5a0886f797018b26aa42f40c09a82332bff77822a451deaaa/cryptography-49.0.0-cp314-cp314t-manylinux_2_34_ppc64le.whl", hash = "sha256:b970c6da94d5bb18629db453d14f2a1300f6bf59b61e9b82377931ef95504866", size = 5246285, upload-time = "2026-06-12T20:01:32.439Z" },
{ url = "https://files.pythonhosted.org/packages/11/2d/5e1fb307cb5931881516b464c98774b3f2c36b5d4bb9a2830253cf553cad/cryptography-49.0.0-cp314-cp314t-manylinux_2_34_x86_64.whl", hash = "sha256:d8ecde755e2e91bf773fc94e8c9d730cd7f2007004cb492263a794ec3899a1c8", size = 4730441, upload-time = "2026-06-12T20:02:01.469Z" },
{ url = "https://files.pythonhosted.org/packages/e4/c0/bff5a02ee731d207d6a1ed51732549d8c53d2bc8da1d10ec6f2844201d68/cryptography-49.0.0-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:e3fb64c420688e5319ae25113a354015abbd8dffbfbc41781a1ea66fc7622ac3", size = 4815869, upload-time = "2026-06-12T20:01:36.574Z" },
{ url = "https://files.pythonhosted.org/packages/b9/26/814681d14248d95d73d5c3eea0c39a94eb8302df966f670a2c60de90974b/cryptography-49.0.0-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:32703d93296f5c1f4b53349ad3a250c2cae0fdecd3a3dd5d47e616d8d616af27", size = 4960948, upload-time = "2026-06-12T20:02:18.688Z" },
{ url = "https://files.pythonhosted.org/packages/4c/fe/93ecac273d3738939d023612ad12cca9a3740a5345d69fda04134c43fd96/cryptography-49.0.0-cp314-cp314t-win_amd64.whl", hash = "sha256:33cd0565932807baddb67b96dbee92f2c374b5c89dee09fd74079aeb8c8dba61", size = 3799153, upload-time = "2026-06-12T20:01:39.059Z" },
{ url = "https://files.pythonhosted.org/packages/19/2a/5bb823f5bedcf80718cea7fbc95ec5515cca3769633c4b01a32be7f30e7c/cryptography-49.0.0-cp39-abi3-macosx_11_0_arm64.whl", hash = "sha256:ec5e529fb80935c94fe7b729f9972b50e351a0e6b50aa294fd5cabb109fcc29a", size = 4025947, upload-time = "2026-06-12T20:01:25.745Z" },
{ url = "https://files.pythonhosted.org/packages/3d/df/40577043ca124e17012f408ddddaeb213b856336ac82ddb3bc915f39e29f/cryptography-49.0.0-cp39-abi3-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:f78ff2c9ed8dc2d036b0f4d640e22522213d047c1b14e61205a7e55c80a494d4", size = 4692429, upload-time = "2026-06-12T20:01:53.628Z" },
{ url = "https://files.pythonhosted.org/packages/2c/99/2d13299eb3dd27b02dcfaafcc91d6b5cb3329f7cbd6d8f51921acd566c1a/cryptography-49.0.0-cp39-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:35b151772baff2c74cba7fa290ceaff4c3b11c0c881eb93eb5dbc05a7cfbba18", size = 4700968, upload-time = "2026-06-12T20:02:45.383Z" },
{ url = "https://files.pythonhosted.org/packages/a5/4d/9c0cd02f95e2602dd5e563da149ee0830abef3537be8b34dc56281ebe27a/cryptography-49.0.0-cp39-abi3-manylinux_2_28_aarch64.whl", hash = "sha256:0f21641cf4b30fca7aee061ced0ec7ad7b073518088b7c9969a297c0ae796c69", size = 4697758, upload-time = "2026-06-12T20:01:41.13Z" },
{ url = "https://files.pythonhosted.org/packages/24/01/186c825898477d77e2324d5360fefe622ff1d8d1963ec0554e2cada8ec77/cryptography-49.0.0-cp39-abi3-manylinux_2_28_ppc64le.whl", hash = "sha256:9e82dcc8e56052715fb18b2429e3bca4823b1629136a2084fc45a9a5cecb9b64", size = 5298863, upload-time = "2026-06-12T20:02:24.579Z" },
{ url = "https://files.pythonhosted.org/packages/b8/7b/62cbbab75d0659865bf0273790031544a0b16c8072d258f9428dcd8190dc/cryptography-49.0.0-cp39-abi3-manylinux_2_28_x86_64.whl", hash = "sha256:6f2debedf9ca60cf1d5bd466475638af5130f89965605cd818484d19987d3a21", size = 4735983, upload-time = "2026-06-12T20:01:50.14Z" },
{ url = "https://files.pythonhosted.org/packages/6c/72/3e798c064bc39e471008075d0f9bc9daf77a80879c092e4a8e170c585ed4/cryptography-49.0.0-cp39-abi3-manylinux_2_31_armv7l.whl", hash = "sha256:8c25ceb16df5b9435f3f6a9829204985b0e0cbee3b48aacd432c7d2c850b44d9", size = 4334173, upload-time = "2026-06-12T20:01:44.743Z" },
{ url = "https://files.pythonhosted.org/packages/f0/ee/6fca21d1ac73e06f8bef71940abfd4d2f6472b4bca284d770f32bd4086f6/cryptography-49.0.0-cp39-abi3-manylinux_2_34_aarch64.whl", hash = "sha256:28d8b15e6275f12c8a207dc309dfa957903c927d08d0cc937ee3f63f200693cc", size = 4697298, upload-time = "2026-06-12T20:02:20.918Z" },
{ url = "https://files.pythonhosted.org/packages/67/d0/a5fcd3515f0bae49a7b6d0413cc1bdccdcc1fc0047037a0d480642cdc5d6/cryptography-49.0.0-cp39-abi3-manylinux_2_34_ppc64le.whl", hash = "sha256:6fc361c34fb6aac015ce19435876635e5c6d21db31998b0920f675f131e043b8", size = 5254338, upload-time = "2026-06-12T20:02:22.737Z" },
{ url = "https://files.pythonhosted.org/packages/a0/84/84fe36f19caf857d61cb7fc9c63035a47ffabd84ea12d1d393148efa3615/cryptography-49.0.0-cp39-abi3-manylinux_2_34_x86_64.whl", hash = "sha256:2400ef9c9e2299a25614eb1dea3db54a69b1349efd043bfac9c67630d136df36", size = 4735650, upload-time = "2026-06-12T20:02:41.389Z" },
{ url = "https://files.pythonhosted.org/packages/6c/a0/db537264e234f7273a73ec020873d6d6b39dfd8a53db78b550ca8320440e/cryptography-49.0.0-cp39-abi3-musllinux_1_2_aarch64.whl", hash = "sha256:67e1d20ad9ef3a563c59ef22e7a8a0b8210bd26604369ea4a30a7c66aefe504e", size = 4834820, upload-time = "2026-06-12T20:01:51.847Z" },
{ url = "https://files.pythonhosted.org/packages/93/77/8df9eb486495979bccecd1062e2eaf435250e84437040295b57d09048b0b/cryptography-49.0.0-cp39-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:42b0684e0e40cf26122427802486f6d93aea593612603a94fbf260c7eb1e9c1b", size = 4967968, upload-time = "2026-06-12T20:02:12.524Z" },
{ url = "https://files.pythonhosted.org/packages/c2/e6/f60198ea8d9dfa15fff9ed4ca02ce362f6eadd9ba757dcc50634c4257b63/cryptography-49.0.0-cp39-abi3-win_amd64.whl", hash = "sha256:026ac7423e6fa66872d3bf889be5974507da3944f866f704fa200eadacd00001", size = 3785547, upload-time = "2026-06-12T20:02:26.847Z" },
{ url = "https://files.pythonhosted.org/packages/63/d3/4a83af35d65e3fad632c926fad684c193ea4398569ccb0bbbc7fe8f5dc9a/cryptography-49.0.0-pp311-pypy311_pp73-macosx_11_0_arm64.whl", hash = "sha256:fc1e275c2f1d97b1a6450b8b0ea3ebfa6e087a611c2b26cb2404d48588abab7b", size = 3993685, upload-time = "2026-06-12T20:02:14.883Z" },
{ url = "https://files.pythonhosted.org/packages/d6/a7/f9dac0ab7f80368c56993a7bf638ef9935f825c91902798481fac0898138/cryptography-49.0.0-pp311-pypy311_pp73-manylinux_2_28_aarch64.whl", hash = "sha256:c83782480a4a9da4d0feb51950131ba32e12e70813848b3343f6e18c28a66838", size = 4676239, upload-time = "2026-06-12T20:02:28.793Z" },
{ url = "https://files.pythonhosted.org/packages/d7/70/2ba3769dd0ae167e2f33dfa9592d45db6ff9a61d62ca1a5b3d1bdd09068f/cryptography-49.0.0-pp311-pypy311_pp73-manylinux_2_28_x86_64.whl", hash = "sha256:b39efa323140595abd3ecca8529d321ae50f55f3aa3ba9cc81ea56a6011953d5", size = 4715584, upload-time = "2026-06-12T20:01:27.495Z" },
{ url = "https://files.pythonhosted.org/packages/94/64/2923570ac1c0bd3a737aa366ac3abbbbde273042308b8cde95e2364a6e6a/cryptography-49.0.0-pp311-pypy311_pp73-manylinux_2_34_aarch64.whl", hash = "sha256:b47db11c2c3525083296069b98ac5221907455e989ae0c2e3008bde851921615", size = 4675885, upload-time = "2026-06-12T20:01:55.49Z" },
{ url = "https://files.pythonhosted.org/packages/ab/f8/614dc7e051418cfe53d55173c1e24c6b0085e89996fe90508c2fdf769aef/cryptography-49.0.0-pp311-pypy311_pp73-manylinux_2_34_x86_64.whl", hash = "sha256:084ef1af862eb07ec46d25f68689f2102a9fc0e05ce7b80f14f5fe51e4eef0f6", size = 4715449, upload-time = "2026-06-12T20:02:05.469Z" },
{ url = "https://files.pythonhosted.org/packages/aa/50/a9caea39ad19c431c1a3f8a31114df65b260cdfe67786b6c7e7c040c4c44/cryptography-49.0.0-pp311-pypy311_pp73-win_amd64.whl", hash = "sha256:be9fcb48a55f023493482827d4f459bd263cc20efde64f204b97c123201850c6", size = 3783731, upload-time = "2026-06-12T20:02:43.319Z" },
]
[[package]]
@@ -2352,15 +2349,15 @@ wheels = [
[[package]]
name = "starlette"
version = "1.2.1"
version = "1.3.1"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "anyio" },
{ name = "typing-extensions", marker = "python_full_version < '3.13'" },
]
sdist = { url = "https://files.pythonhosted.org/packages/25/44/ec35f1b6e83094b997da438a02c8c9b0ade2b1e84cfc48bd4656780760a6/starlette-1.2.1.tar.gz", hash = "sha256:9b9b5ebb992e67d6093741e63c2f59e4f6fff986f81163c087867bd7b924b3f6", size = 2701854, upload-time = "2026-05-31T01:07:51.847Z" }
sdist = { url = "https://files.pythonhosted.org/packages/eb/e3/7c1dc7381d9f8ab7d854328ebfa884e62cb3f3d8549ddfd37c7814f42afa/starlette-1.3.1.tar.gz", hash = "sha256:05d0213193f2fbaae60e2ecb593b4add4262ad4e46536b54abe36f11a71724e0", size = 2703240, upload-time = "2026-06-12T09:23:11.602Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/1c/54/196d0c1db10af76baa4f64894448505d60d3cdf70ef92cbb35f46a4e4c71/starlette-1.2.1-py3-none-any.whl", hash = "sha256:4de0082d08c8f6764a85a54cf1120d6939507a19905c7768acad2a9f875d2b89", size = 73350, upload-time = "2026-05-31T01:07:50.09Z" },
{ url = "https://files.pythonhosted.org/packages/ec/bb/2799cc2ede3ed41131f8975621e7213dfc7ef4acbbaadfa440f32500c370/starlette-1.3.1-py3-none-any.whl", hash = "sha256:c7372aae11c3c3f26a42df7bd626cec2f47d03483d261d369516a615a53714c6", size = 73632, upload-time = "2026-06-12T09:23:10.017Z" },
]
[[package]]
@@ -2440,7 +2437,7 @@ wheels = [
[[package]]
name = "turnstone"
version = "1.6.6"
version = "1.6.9"
source = { editable = "." }
dependencies = [
{ name = "alembic" },
@@ -2498,7 +2495,7 @@ requires-dist = [
{ name = "anthropic", specifier = ">=0.108" },
{ name = "bcrypt", specifier = ">=4.0" },
{ name = "croniter", specifier = ">=3.0" },
{ name = "cryptography", specifier = ">=42" },
{ name = "cryptography", specifier = ">=48.0.1" },
{ name = "discord-py", marker = "extra == 'discord'", specifier = ">=2.4" },
{ name = "httpx", specifier = ">=0.28" },
{ name = "httpx-sse", specifier = ">=0.4" },
@@ -2519,7 +2516,7 @@ requires-dist = [
{ name = "slack-bolt", marker = "extra == 'test'", specifier = ">=1.18" },
{ name = "sqlalchemy", specifier = ">=2.0" },
{ name = "sse-starlette", specifier = ">=2.0" },
{ name = "starlette", specifier = ">=1.0.1" },
{ name = "starlette", specifier = ">=1.3.1" },
{ name = "structlog", specifier = ">=24.1" },
{ name = "turnstone", extras = ["discord", "slack"], marker = "extra == 'all'" },
{ name = "uvicorn", specifier = ">=0.34" },