The job was named "test", colliding with core CI's "test" matrix so the PR
checks list showed two "test (3.11)" rows. Rename it to "understone" so the
example's checks read unambiguously (understone (3.11) / (3.13)).
The door-game example is a standalone package (no turnstone-core
dependency) that the root suite does not collect — its
testpaths are scoped to ["tests"], so the example's 419 tests, ruff,
and mypy gates never ran in CI.
Add a path-filtered workflow that installs the example and runs its
full gate (pytest + ruff check + ruff format --check + mypy) whenever
examples/door-game (or this workflow) changes, across the example's
declared Python floor and ceiling (3.11, 3.13). Pinned action SHAs and
contents:read permissions match the existing CI workflows.
The Anthropic SDK provider was the lone first-class provider gated behind
an optional extra, while OpenAI ships in core and Google rides the
OpenAI-compatible path. Fold anthropic, psycopg (postgres), croniter
(console), and lacme (tls) into the base dependency set so a default
`pip install turnstone` yields a complete single- or multi-node
deployment; only the Discord/Slack channel gateways stay optional.
- pyproject: four extras → base deps; `all` is now discord+slack; drop the
redundant croniter from the `test` extra; regenerate uv.lock.
- ci: the postgres test job installs `.[test]` (psycopg is base now).
- providers: `_ensure_anthropic` becomes a thin SDK accessor for
`create_client`; drop the now-redundant eager import-guard calls from
the streaming/completion hot path (anthropic is always present).
- bootstrap: import anthropic directly.
- tests/docs: drop the anthropic importorskips and stale extra-install hints.
The pyjwt 2.12.1 advisory (\"weak encryption\") is disputed by the
supplier — the key length is the calling application's
responsibility, not the library's. Turnstone generates its JWT
signing keys via the standard ``secrets`` module at
operator-controlled strength (see ``turnstone/core/auth.py``), so the
advisory does not apply to this codebase.
No fix version is available — pyjwt 2.12.1 is the current PyPI
latest as of 2026-05-21. Adding ``--ignore-vuln PYSEC-2025-183``
with the rationale documented in-line so a future reviewer can
re-evaluate when an upstream fix or a non-disputed re-issue lands.
The advisory was published between main's last CI pass (2026-05-19)
and the interactive-cleanup PR's CI run (2026-05-21); main's
security job will fail next push without this fix.
* fix(renderer): handle LaTeX-style \(...\) and \[...\] math delimiters
The browser renderer at turnstone/shared_static/renderer.js only
recognized TeX-style $...$ / $$...$$ delimiters. Most modern LLMs
(GPT-5 / o-series, Claude with reasoning effort) emit LaTeX-style
\(...\) for inline math and \[...\] for display by default — those
slipped through as raw text in the coord + interactive WebUIs,
making KaTeX appear "broken when nested inside a markdown block"
(actually broken everywhere, the surrounding markdown just made
the failure noticeable).
Added a second pass for each delimiter style alongside the
existing $...$ / $$...$$ patterns. Both styles now feed the same
mathBlocks / inlineMaths placeholder pipeline so all the existing
nested-block handling (lists, blockquotes, tables, bold, headings,
details, post-render KaTeX markup) Just Works.
Edge cases verified by the new test_renderer_js.py harness:
- \(...\) inside inline code stays literal
- \(...\) inside fenced code blocks stays literal
- Solo \[ with no closing \] doesn't trigger spurious math
- Markdown links [text](url) untouched (regex uses \[ \], not [ ])
- Mixed TeX + LaTeX delimiters in one message both render
The harness drives renderer.js through Node via vm.runInThisContext
with stubbed document/katex globals — first JS-side regression
guard for the renderer; previously it had no test coverage at all.
* fix(renderer): apply Copilot feedback on PR #425
Three review items from Copilot:
1. Display-math sentinel could leak through inline-code spans.
The original ordering ran $$...$$ / \[...\] extraction BEFORE
inline code, so a backtick span around math (e.g. `$$x$$` or
`\[x\]`) had its delimiters consumed by the math regex and
replaced with \x00MB…\x00. Inline code then captured the
sentinel; restore order put MB after IC, leaving the null-byte
placeholder visible inside the rendered <code>. Reorder: inline
code first, then display math, then inline math. Code spans
now seal their content before any math regex sees it. The
reverse edge case (math containing backticks, e.g. \verb|`x`|)
is much rarer and KaTeX rejects \verb anyway.
2. Inline LaTeX-style \(...\) regex used [\s\S]+? which allowed
newlines, so an unterminated \( on one line would eat the
next paragraph until it found a closing \). Aligned with the
existing $...$ behavior by switching to [^\n]+? — display
math (\[...\] / $$...$$) stays multi-line by design.
3. tests/test_renderer_js.py was guarded with a node-availability
skip, but CI's test + test-postgres jobs didn't explicitly
install Node, so the suite would have silently no-op'd if the
runner image dropped Node. Added actions/setup-node@v5 to
both jobs.
Four new regression tests cover the leak (both delimiter styles
inside backticks must stay literal) and the cross-paragraph span
(both \(...\) and $...$ must not eat newlines).
Remove the weekly Trivy scan job and the .trivyignore exclusion file.
The scanner has been flagging base-image CVEs that require no action
on our part (upstream-only fixes) and has provided no actionable
signal, while breaking CI on an ongoing basis.
* chore(deps): update dependency hls.js to v1.6.16
* chore: download vendored hls.js files + add hls to workflow detection loop
The wheel-completeness check failed on the Renovate bump because
vendor-js.yml only iterated katex/hljs/mermaid — so hls.js PRs
never got their files auto-downloaded. Adding hls to the loop so
future Renovate bumps are merge-ready without manual intervention.
Also running the update now to fix this specific PR.
---------
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Patrick Buckley <buckleypm@gmail.com>
* fix: include prompt .md files in wheel, add wheel-completeness CI (#289)
Prompt markdown files were missing from PyPI wheels since the modular
prompts refactor, causing FileNotFoundError on startup for pip-installed
users. Add the missing include pattern and a new CI job that diffs
source-tree data files against wheel contents so omissions are caught
before merge.
* fix: sanitise ALLOW patterns in wheel-completeness check
Strip blank lines and leading whitespace from the allowlist before
passing to grep -vFxf so empty patterns cannot silently match all lines.
Multiple CI completions for the same commit (tag push + branch push)
caused duplicate publish and docker runs. Concurrency group keyed on
head_sha ensures only one publish runs per commit.
* chore: release infrastructure for dual-track stable/experimental
CI/CD changes for the 1.0 release:
- Gate PyPI publish and Docker publish on CI success via workflow_run
- Add docker-publish.yml: builds and pushes to GHCR with smart tagging
(stable gets :X.Y.Z/:X.Y/:stable/:latest, pre-release gets :experimental)
- Add stable/* and v* tags to CI and docker-scan triggers
- Remove stale [mq] extra and types-redis from CI (Redis MQ deleted)
- Remove stale redis from Renovate package rules
Release tooling:
- scripts/release.sh: bump version, uv lock, commit, tag (with --push)
- docs/releasing.md: documents stable/experimental workflow
Docker:
- Add /workspace mount point (WORKSPACE_MOUNT env var, defaults to empty volume)
- Update .env.example: remove stale Redis/auth-token refs, add workspace/model/discord
README:
- Remove beta warning, add hero image and release tracks table
* fix: derive release tag from git instead of workflow_run.head_branch
Use git tag --points-at HEAD after checkout to resolve the release
tag instead of relying on workflow_run.head_branch, which may not
reliably be the tag name for tag-triggered CI runs. Both publish
and docker-publish workflows now skip cleanly when no v* tag exists
at the checked-out commit.
Add vendor-js workflow that triggers on Renovate PRs touching
pyproject.toml — detects version changes, runs update-vendored-js.sh,
and commits the actual files back to the PR branch. Supports manual
dispatch via pr_number input for one-off runs.
* feat: add PostgreSQL CI integration tests
Add --storage-backend pytest option and shared storage_backend fixture
in conftest.py that creates SQLiteBackend or PostgreSQLBackend based
on the flag. Migrate 13 storage test files to use shared fixture
instead of local SQLiteBackend fixtures.
Add test-postgres CI job with PostgreSQL 17 service container that
runs the full test suite against real PostgreSQL.
* fix: use TRUNCATE CASCADE for PG cleanup, wrap in try/finally
TRUNCATE is faster than per-table DELETE and resets autoincrement
sequences. try/except ensures reset_storage() always runs even if
cleanup fails due to a corrupted connection from a failing test.
* fix: document _engine coupling in PG cleanup comment
uv lock --check fails after version bump because the lockfile is stale.
pip-audit --strict fails because turnstone 0.7.0 isn't on PyPI yet.
Fix: regenerate uv.lock, and audit only third-party deps via
uv export --no-emit-project piped to pip-audit -r.
* ci: add GitHub Release creation on tag push
* refactor: rename plan tool to create_plan
Rename plan → create_plan to resolve cross-provider tool selection
failures. Models consistently treated "plan" as a reasoning concept
rather than a callable tool. The new name is an unambiguous verb+noun
action. Also rename the parameter from prompt → goal for clarity,
add web_search to the default system prompt tool patterns
* feat: eval harness improvements inspired by autoresearch patterns
Major enhancements to turnstone-eval:
- Per-test timeout (--test-timeout, default 300s) and suite timeout
(--suite-timeout) prevent stuck runs from blocking the suite
- Fast-fail skips remaining runs after ceil(n/2) consecutive zeros
- Summary table with colored PASS/WEAK/FAIL and append-only TSV output
- Progress reporting with running pass rate, token count, and ETA
- Parallel test execution via ProcessPoolExecutor (--parallel N)
- Per-role model assignment: test/optimizer/observer can use different
models and providers (--optimizer-model, --observer-model, etc.)
with auto-detection from base URL
- Improved optimizer and observer system prompts with structured
failure-mode diagnosis, keep/discard rules, and trend analysis
- Fixed token counting (prompt tokens use last-turn value, not sum)
- Added math-calculation and web-search-query test cases
- Fixed multi-file-edit test (both files now contain the target string)
* fix: address Copilot review feedback
- Revert prompt token counting to sum (reflects billed usage)
- Add tool_args to fast-fail skipped run dicts for schema consistency
- Align approval_label with func_name ("create_plan")
- Add timeout to future.result() in parallel path (test_timeout + 30s)
- Document thread-leak trade-off on serial timeout path
* Add test coverage reporting and pre-commit hooks
- Add pytest-cov to test dependencies, configure coverage in pyproject.toml
(branch coverage, static asset omission, standard exclusion patterns)
- CI test job now runs with --cov and uploads coverage XML as artifact
- Add .pre-commit-config.yaml with ruff (check + format) and mypy hooks
Baseline coverage: 41% (482 tests, Python 3.13)
* Fix Copilot review: add redis dep for pre-commit mypy, explicit --cov target
- Add redis>=7.2 to mypy pre-commit hook additional_dependencies so
mypy can resolve redis imports in isolated pre-commit environments
- Use --cov=turnstone instead of bare --cov to explicitly scope coverage
* Quality overhaul: code tooling, CI/CD, architecture diagrams, UI redesign, and legacy cleanup
- Add ruff (lint+format) and mypy (strict) with zero errors across 37 source files
- Add GitHub Actions CI (lint, typecheck, test matrix 3.11/3.12/3.13) and PyPI publish workflow
- Create 12 PlantUML architecture diagrams with PNG renders covering all subsystems
- Refresh README and docs with badges, diagram links, and current descriptions
- Refactor test_server_live.py with mock streaming helpers for deterministic CI testing
- Update dependencies to current versions (openai>=2.24, httpx>=0.28, redis>=7.2)
Console dashboard:
- Move state indicators from top cards to fixed bottom status bar with cluster metrics
- Replace flat 50-node list with hostname-prefix grouped nodes (expand/collapse, up to 1000)
- Apply "Instrument Panel" visual redesign: IBM Plex Mono + Outfit fonts, warm amber accent,
LED glow state indicators, deep charcoal surfaces, WCAG AA contrast compliance
- Add render cache, stale indicator, active filter highlight, loading states
Server web UI:
- Apply matching Instrument Panel aesthetic for visual consistency with console
- Fix branding (pcode → turnstone), extract inline styles to CSS classes
- Rename pcode localStorage keys and history state to turnstone
Legacy cleanup:
- Remove persona-model-specific --persona flag and /persona slash command
- Remove model_identity from chat_template_kwargs (vLLM-specific mechanism)
- Refactor plan agent to use standard developer message instead of model_identity
- Remove dead code (unused date/has_tools variables, noqa suppressions)
* Fix CI typecheck: add mypy overrides for optional sympy/numpy imports
The math sandbox optionally imports sympy and numpy at runtime (try/except
ImportError). In CI these packages are not installed, so mypy raises
import-not-found rather than import-untyped. Add mypy overrides to
ignore missing imports for these optional dependencies.
* Fix Copilot review findings: ARIA role, status bar cache, and pulse opacity
- Change #node-table from role="tree" to role="list" and group elements
from role="treeitem" to role="listitem" (proper ARIA semantics)
- Include currentView and currentFilter.state in renderStatusBar cache key
so active pill highlight updates when switching views
- Align pulse animation to 0.35 opacity (already applied in CSS)
Patrick Buckley