Commit Graph

149 Commits

Author SHA1 Message Date
Patrick Buckley 6eae1c3954 fix(providers): support OpenAI v3 HTTPX2 transport 2026-08-11 23:30:44 -07:00
Patrick Buckley ed6286ab63 fix(deps): constrain OpenAI SDK below v3 2026-08-11 21:58:21 -07:00
Patrick Buckley cf2811fceb chore: bump version to 1.8.0a7 2026-08-11 04:19:12 -07:00
renovate[bot] ca6db54bcc chore(deps): lock file maintenance 2026-08-10 18:56:19 -07:00
Patrick Buckley f138784ba3 chore: bump version to 1.8.0a6 2026-08-06 15:48:17 -07:00
renovate[bot] 1a4f411cd5 chore(deps): lock file maintenance 2026-08-03 11:38:31 -07:00
Patrick Buckley 166b46cda4 chore: bump version to 1.8.0a5 2026-07-29 22:49:25 -07:00
renovate[bot] a184494fe7 chore(deps): lock file maintenance 2026-07-27 00:45:01 -07:00
Patrick Buckley 96834496c4 feat(anthropic): onboard claude-opus-5
The capabilities row is a copy of claude-opus-4-8 — 1M context, 128K output,
adaptive thinking, the full low..max effort ladder, mid-conversation system
messages. Two of the model's documented breaking changes are unreachable from
this lane and stay that way only while thinking_mode is "adaptive": thinking is
on by default when the param is omitted, and disabling it is a 400 at effort
xhigh or max. We never omit it and never emit "disabled", so both are recorded
at the row rather than defended against.

The third needed work. A safety classifier can decline with
stop_reason="refusal" on a successful HTTP 200, with content either empty or
partial, and an unmapped value fell through _normalize_finish_reason as a
literal string. The drain gate only raises on an ABSENT finish reason, so a
declined turn landed as a complete result with nothing to notice it: the
interactive lane matched neither of its two warn arms and stayed silent, and a
sub-agent handed the declined partial up to its parent as though it were
finished synthesis. Normalizing onto content_filter routes the decline into the
arms both lanes already have for that state — the operator gets the warning,
and the sub-agent stops rather than passing the fragment on.

The raw stop reason is logged where it is still in hand: normalization is lossy
and a classifier decline is otherwise indistinguishable from an ordinary
content filter. The gate is on the RAW value rather than
(normalized != raw), which is true for end_turn and tool_use as well and would
fire on every turn in every lane.

The anthropic floor moves to 0.117 to track the release current at onboarding.
The model needs no new SDK surface — ids are opaque strings and "refusal" has
been in the StopReason literal since ~0.95 — so this is hygiene; raise it again
when adopting fast mode, server-side fallbacks, advisor, or mid-conversation
tool changes, which do need newer typed params.
2026-07-25 01:12:04 -07:00
Patrick Buckley 51bb525b27 chore: bump version to 1.8.0a4 2026-07-21 15:30:35 -07:00
Patrick Buckley a16e6d66d8 chore: bump version to 1.8.0a3 2026-07-20 07:38:27 -07:00
renovate[bot] 5ae963cb49 chore(deps): lock file maintenance 2026-07-20 07:21:40 -07:00
renovate[bot] 7b1f77dda7 chore(deps): lock file maintenance 2026-07-15 21:07:39 -07:00
Patrick Buckley 0e6a99e0f1 chore: bump version to 1.8.0a2 2026-07-14 11:43:44 -07:00
Patrick Buckley 6f991d6aff chore: bump version to 1.8.0a1 2026-07-13 22:41:36 -07:00
Patrick Buckley b450b9ad20 fix(providers): align GPT-5.6 with the GA API surface
- every 5.6 tier accepts effort "max" and reasoning.mode
  "standard"/"pro" (GA docs: pro is a request mode on any GPT-5.6
  model) -- drop the Sol-only gating
- GPT-5.6 deprecates prompt_cache_retention; send
  prompt_cache_options={"ttl": "30m"} (its only supported lifetime)
  and keep the 24h retention policy for pre-5.6 models
- never inject commercial cache params into local lanes: dropped from
  the Chat Completions lane (which serves only openai-compatible and
  google) and gated off the compat-pinned Responses lane -- a gpt-5*
  served-model name is not an OpenAI account
- account cache writes: usage *_tokens_details.cache_write_tokens
  flows into cache_creation_tokens (5.6 bills writes at 1.25x the
  uncached input rate)
- drop non-string verbosity/reasoning_mode overrides with a warning
  instead of raising on unhashable capability-JSON values
- keep ModelCapabilities' public positional prefix stable by appending
  the verbosity/pro fields at the tail; pin it with a constructor test
- openai floor 2.44 -> 2.45, the first release with the typed
  prompt_cache_options kwarg
2026-07-10 15:59:40 -07:00
Patrick Buckley 47f908c9e0 feat(providers): add OpenAI GPT-5.6 (Sol/Terra/Luna) support
Onboard the GPT-5.6 family (GA 2026-07-09) to the OpenAI Responses lane.

- Capability rows for gpt-5.6 (= Sol alias/catch-all), gpt-5.6-terra, and
  gpt-5.6-luna: 1.05M context, 128K output, tool_search/vision/pdf/reasoning
  replay, default effort medium, temperature only at effort=none.
- "max" reasoning effort, Sol-only; Terra/Luna cap at xhigh (the knob's "max"
  snaps to the xhigh ceiling). First commercial OpenAI use of "max" — the
  ordinal knob already ranked it, so no effort-ladder change was needed.
- Verbosity and pro mode as operator-declared capability fields
  (supports_verbosity/verbosity, supports_pro_mode/reasoning_mode), merged
  from the model-definition capabilities JSON and emitted on the Responses
  wire as text.verbosity and reasoning.mode. Both are gated by a supports
  flag plus an enum guard that drops unknown values with a warning. Pro mode
  is Sol-only. There is no gpt-5.6-pro model — "pro" is the reasoning.mode
  param, not a separate model id.
- Raise the openai floor to >=2.44 for the 5.6 Responses params.

Unit and wire-golden tests cover the rows, max->xhigh snapping, the two
levers, and the enum guards. Validated live against the OpenAI API: gpt-5.6
accepts the model id, effort "max", text.verbosity, and reasoning.mode="pro".
2026-07-09 18:48:51 -07:00
renovate[bot] 5fded65b82 chore(deps): lock file maintenance 2026-07-06 18:20:14 -07:00
renovate[bot] 3cf94dd80f chore(deps): lock file maintenance 2026-07-06 03:09:51 -07:00
Patrick Buckley 801d5dfb59 chore: bump version to 1.7.0rc1 2026-07-05 02:03:18 -07:00
Patrick Buckley a352b20786 chore: bump version to 1.7.0a7 2026-07-05 02:02:16 -07:00
Patrick Buckley c71cc749d9 chore: bump version to 1.7.0a6 2026-07-01 21:07:52 -07:00
renovate[bot] 76241ab703 chore(deps): lock file maintenance 2026-07-01 19:47:04 -07:00
Patrick Buckley bbadd00ac0 chore: bump version to 1.7.0a5 2026-06-28 04:12:06 -07:00
Patrick Buckley 2b0b1cf73e chore: bump version to 1.7.0a4 2026-06-27 17:20:21 -07:00
Patrick Buckley 0e1972e6aa chore: bump version to 1.7.0a3 2026-06-27 01:22:33 -07:00
renovate[bot] af98fe434a chore(deps): lock file maintenance 2026-06-25 19:17:12 -07:00
Patrick Buckley 2b3b212971 Add altair + vl-convert-python viz stack (#685)
* feat(deps): add altair + vl-convert-python viz stack

The standard, ui://-ready visualization stack: one Vega-Lite spec renders to
static SVG via vl-convert (a bundled Rust renderer — no browser, GDAL, or
chromium) and drops into vega-embed for interactive ui:// panels. The first
consumer is the civic-records choropleth map; future ui:// surfaces build on
the same stack.

The dependency closure is fully permissive (BSD-3 + the OFL font + MIT/ISC JS) —
clean for Apache-2.0 and commercial use. Adds a mypy override for the untyped
vl_convert wheel.

* fix(deps): bump pydantic-settings to 2.14.2 (GHSA-4xgf-cpjx-pc3j)

Clears the pip-audit --strict advisory on the transitive pydantic-settings
2.14.1. Pinned as an explicit security floor in [project.dependencies]
(matching the starlette/cryptography CVE-floor convention) even though it is
transitive-only, so the floor is documented and survives re-resolution.

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* chore: regenerate uv.lock to pass lock check

---------

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
2026-06-21 15:39:41 -07:00
Patrick Buckley fd5b710437 chore: bump version to 1.7.0a2 2026-06-16 04:39:26 -07:00
Patrick Buckley e562d04e8b fix(deps): enforce cryptography + starlette security floors
main's lockfile was already on the patched versions (cryptography 49.0.0,
starlette 1.3.1) via renovate, but the pyproject floors (>=42, >=1.0.1) still
permitted a regression to vulnerable versions. Raise the floors to match
stable/1.6's v1.6.7 security fix:
- cryptography >=48.0.1 (GHSA-537c-gmf6-5ccf — bundled OpenSSL vulnerable <48.0.1)
- starlette >=1.3.1 (CVE-2026-54282 host spoof + CVE-2026-54283 url-encoded form DoS)
2026-06-16 04:39:26 -07:00
Patrick Buckley 471d94b27f feat(attachments): rasterize PDF to page images for vision models without native PDF
A vision-capable model that can't ingest PDF natively now gets the PDF
rendered to one image per page instead of extracted text; falls back to
text extraction when rendering yields nothing.

- core/pdf.py: rasterize_pdf via pypdfium2 render + Pillow PNG (page-capped
  at 10, never raises)
- session._wire_content_part: pdf + !supports_pdf + supports_vision ->
  rasterized image parts; else text extraction
- trajectory.resolve_attachment_parts: a placeholder can now expand to a
  list of parts (1->N); the resolve_attachments callback return type widened
  to dict[str, Any] across the provider protocol + 4 providers
- pyproject: pillow dependency
- tests: rasterize_pdf, vision-rasterize gate path, 1->N materialization
2026-06-16 00:48:14 -07:00
Patrick Buckley addb8d0be8 feat(attachments): capability-gated client-side fallback (pdf->text, audio->transcript)
When the active model can't ingest a kind natively, the wire resolver
converts it client-side instead of sending a part the model can't read.
Per-kind ownership, no shared machinery: PDF text-extraction is a
pure-local PDF concern; audio transcription is an STT concern memoized
in the audio domain.

- core/pdf.py: extract_pdf_text via pypdfium2 (pure-local, no network, no
  cache — re-run per build; page-capped)
- core/audio.py: transcribe_cached — non-raising, memoized by
  (alias, content-hash); backend failures not cached
- session._wire_content_part: per-kind dispatch — native where the model
  supports the kind (supports_pdf / supports_audio_input), else fallback;
  display/export resolve natively so no conversion fires on a render
- image left ungated (pre-existing behavior unchanged)
- pyproject: pypdfium2 dependency + mypy untyped-import override
- tests: pdf extraction, transcript memoization, per-kind gate dispatch
2026-06-16 00:48:14 -07:00
renovate[bot] 94e385e91f chore(deps): lock file maintenance 2026-06-15 02:49:32 -07:00
Patrick Buckley 1497c392e4 chore: cap mcp <2 ahead of the v2 breaking rewrite
mcp 2.0.0a1 shipped 2026-06-11 (stable targeted ~2026-07-27). v2 removes
streamablehttp_client, changes the transport tuple arity, and renames
mcp.types fields to snake_case — all of which our client imports. The
maintainers' release note asks downstream packages to add an upper
bound now (their worked example is this exact constraint). Floor stays
at 1.27: nothing newer adds anything our surface needs, and the #2147
shutdown busy-loop we wrap remains unfixed at every released version.
Resolution is unchanged (1.27.2); lockfile re-pinned metadata only.
2026-06-11 20:42:34 -07:00
Patrick Buckley a137bffa25 chore: bump version to 1.7.0a1 2026-06-10 22:21:43 -07:00
Patrick Buckley 23007e3ac5 chore: bump version to 1.6.0 2026-06-10 22:21:03 -07:00
Patrick Buckley 160062235a chore: bump version to 1.6.0rc2 2026-06-10 13:35:47 -07:00
Patrick Buckley 319b73d046 chore: bump version to 1.6.0rc1 2026-06-09 13:39:19 -07:00
Patrick Buckley 91c4afb2d0 chore: sync uv.lock with anthropic>=0.108 floor + docstring nit
CI lock-check failed: the Fable 5 commit raised the anthropic floor to
>=0.108 in pyproject.toml but uv.lock still recorded >=0.39 / resolved
0.107.1. Regenerate the lock: anthropic 0.107.1 -> 0.108.0, specifier
0.39 -> 0.108 (no transitive changes).

Also address the Copilot review nit: the operator-instruction trust
declaration docstring wrote the fence marker as <system-reminder_<nonce>>;
align it to the emitted and project-standard <system-reminder_{nonce}>
notation.
2026-06-09 10:50:58 -07:00
Patrick Buckley a5bc37058e chore: bump version to 1.6.0a12 2026-06-08 10:12:29 -07:00
renovate[bot] 3003e935f4 chore(deps): lock file maintenance 2026-06-07 22:44:37 -07:00
Patrick Buckley 87d1ea8530 chore: bump version to 1.6.0a11 2026-06-04 18:40:02 -07:00
Patrick Buckley b3c1b9c9e0 build: promote anthropic, postgres, console, tls to core dependencies
The Anthropic SDK provider was the lone first-class provider gated behind
an optional extra, while OpenAI ships in core and Google rides the
OpenAI-compatible path. Fold anthropic, psycopg (postgres), croniter
(console), and lacme (tls) into the base dependency set so a default
`pip install turnstone` yields a complete single- or multi-node
deployment; only the Discord/Slack channel gateways stay optional.

- pyproject: four extras → base deps; `all` is now discord+slack; drop the
  redundant croniter from the `test` extra; regenerate uv.lock.
- ci: the postgres test job installs `.[test]` (psycopg is base now).
- providers: `_ensure_anthropic` becomes a thin SDK accessor for
  `create_client`; drop the now-redundant eager import-guard calls from
  the streaming/completion hot path (anthropic is always present).
- bootstrap: import anthropic directly.
- tests/docs: drop the anthropic importorskips and stale extra-install hints.
2026-06-04 11:03:13 -07:00
renovate[bot] ef2b18b62f chore(deps): lock file maintenance 2026-06-03 18:56:09 -07:00
renovate[bot] e8211f12a0 chore(deps): update dependency aiohttp to v3.14.0 [security] 2026-06-03 18:41:51 -07:00
Patrick Buckley 87c51b43ca chore: bump version to 1.6.0a10 2026-06-01 21:36:36 -07:00
Patrick Buckley 40b49c1d6c chore: bump version to 1.6.0a9 2026-05-31 20:45:06 -07:00
Patrick Buckley 110d44b07e refactor(tools): remove man, math, and plan_agent built-in tools
`man` and `math` duplicated capabilities already reachable through
`bash`; `plan_agent` is better expressed as a `task_agent` running a
planning skill, and carried a large amount of special-case machinery
(plan-review gate, refinement loop, per-kind model routing). Removing
all three shrinks the tool surface and cuts per-call token cost.

Also removed, as dead-once-the-tools-are-gone:
- the `math` sandbox executor (`turnstone.core.sandbox`) and its
  `[sandbox]` extra; the eval analyst now runs bash-only
- the read-only `AGENT_TOOLS` sub-agent tool set and the `agent`
  tool-metadata key (`task_agent`/`TASK_AGENT_TOOLS` retained)
- the plan-review protocol end to end: the `on_plan_review` UI hook,
  `resolve_plan`, `POST /v1/api/plan` + `POST /v1/api/route/plan`,
  the `plan_review`/`plan_resolved` SSE events, and their Python SDK /
  TypeScript SDK / OpenAPI / frontend / Discord+Slack bindings
- the `model.plan_alias` / `model.plan_effort` settings and the
  registry `plan_model` / `plan_effort` routing fields

TOOLS 31->28, TASK_AGENT_TOOLS 13->11; COORDINATOR_TOOLS unchanged.

BREAKING CHANGE: removes the `man`, `math`, `plan_agent` tools, the
plan-review SSE/HTTP/SDK surface, and the plan_* model-routing settings
from the experimental 1.6 line.
2026-05-31 19:54:43 -07:00
Patrick Buckley 1728a4c0af feat(web-search): replace Tavily/DuckDuckGo backends with self-hosted SearxNG
Drop the Tavily and DuckDuckGo (ddgs) web_search backends for a single
self-hosted SearxNG service bundled into the docker-compose stacks.

Core:
- New SearXNGClient + _format_searxng; rewrite resolve_web_search_client to
  (backend, searxng_url, searxng_engines, ...). MCP backend + oauth_user guard
  unchanged. _resolve_search_client follows storage -> toml -> env -> default
  precedence (explicit "" disables, via ConfigStore.stored_keys()).
- Drop the Tavily-era topic=finance (no SearxNG category); topic is now
  general/news.

Settings/config:
- Remove tools.tavily_api_key, get_tavily_key, $TAVILY_API_KEY, [api].tavily_key.
- Add tools.searxng_url (default http://searxng:8080) + tools.searxng_engines,
  with get_searxng_url/get_searxng_engines.

Compose + bundled config:
- Internal-only searxng service (no published API port, :ro config, /healthz
  healthcheck, persistent searxng-cache volume) in both stacks; bundle
  turnstone/deploy/searxng/settings.yml (JSON output on, limiter off).
- Caddy serves the SearxNG web UI on :8444 (dev: localhost-only; prod: opt-in).
- bootstrap extractor + wheel packaging updated.

Deps: drop the ddg extra + ddgs mypy override (regenerates uv.lock, removing the
lxml/h2/brotli transitives).

Docs: tools/docker/architecture/openshell + diagrams + config example + CHANGELOG;
docs/docker.md carries the AGPL-3.0 §13 operator note.

BREAKING: tools.web_search_backend no longer accepts "tavily"/"ddg";
tools.tavily_api_key and the ddg extra are removed. Run the bundled SearxNG (ships
in the compose stacks) or set TURNSTONE_SEARXNG_URL to an external instance.

Closes #545
2026-05-31 19:03:16 -07:00
Patrick Buckley e322b69c8b chore: bump version to 1.6.0a8 2026-05-31 00:58:44 -07:00