feat(providers): anthropic-compatible lane for local /v1/messages servers

Add provider id "anthropic-compatible": the existing AnthropicProvider
pointed at Anthropic-compatible local servers (vLLM /v1/messages),
mirroring the openai/openai-compatible split. Registry-only — configured
via the admin Models tab or [models.*] toml, not exposed on the bare
--provider flag, so the CLI/server prod-URL defaults are unreachable for
the lane and real-Anthropic behavior is untouched.

Lane behavior (live-verified against vLLM 0.22.1rc1 + DeepSeek-V4-Flash):
- Capability defaults replace the Claude static table: token_param
  max_tokens, thinking_mode none, web_search/tool_search/vision off,
  reasoning replay on. vLLM rejects Anthropic server-side tool types
  (tools require input_schema) and ignores the thinking request param,
  so neither is sent; thinking blocks still stream back and round-trip
  through the native lane verbatim.
- Reasoning toggles via server_compat extra_body chat_template_kwargs
  (first-class vLLM request field; request-level keys beat server
  defaults). _build_thinking_and_kwargs forwards non-internal
  extra_params as SDK extra_body; thinking_budget_tokens stays internal.
- No temperature force: thinking_mode none skips the Claude-only
  temperature=1.0 requirement.

Admin UI: provider option + URL placeholder (base_url without /v1 — the
SDK appends /v1/messages); the server-compat section shows only the
extra-body field for the lane. thinking_mode round-trips through the
form dropdown for every provider except anthropic-compatible, where it
stays in the raw capabilities JSON — the edit-load lift and save restore
use the same predicate so stored overrides are never silently dropped.

Docs: architecture.md gains the lane subsection incl. verified quirks
(thinking param dropped by vLLM; stop_sequences cut inside thinking and
report end_turn; usage has no cache fields; images need a multimodal
model; mid-conversation system turns are per-model opt-in).

Negative-tested: removing the _INTERNAL_EXTRA_PARAMS exclusion fails
test_internal_keys_not_leaked; the live test drives a streamed turn with
the chat_template_kwargs toggle and asserts no reasoning deltas.
This commit is contained in:
Patrick Buckley
2026-06-11 19:54:05 -07:00
parent 7aba631201
commit 8f0115ee2e
11 changed files with 615 additions and 55 deletions
+56 -1
View File
@@ -702,7 +702,7 @@ agent_model = "claude"
Each `[models.*]` entry produces a `ModelConfig` with a `provider` field
(default: `"openai"`). Supported values: `"openai"`, `"anthropic"`, `"google"`,
and `"openai-compatible"`.
`"openai-compatible"`, and `"anthropic-compatible"`.
**Per-model sampling overrides:** Each model can specify `temperature`,
`max_tokens`, and `reasoning_effort` to override the global defaults from
@@ -765,6 +765,61 @@ model = "qwen-3.5-vl"
supports_vision = true
```
**Anthropic-compatible local servers (vLLM `/v1/messages`):** the
`"anthropic-compatible"` provider drives local servers that expose
Anthropic's Messages API for arbitrary checkpoints — vLLM's
`/v1/messages` endpoint, which requires a release with thinking-block
support in the Anthropic endpoint (post-2026-02-28; verified against
v0.22.1rc1). The lane reuses `AnthropicProvider` in compat mode: same
wire translation as the real Anthropic lane, but every model resolves to
the `_ANTHROPIC_COMPAT_DEFAULT` capabilities (200K context, 64K output,
`token_param=max_tokens`, `thinking_mode=none`, no native
web_search/tool_search, no vision) — the static Claude table never
applies to local checkpoints. `base_url` is the server root WITHOUT
`/v1` (the Anthropic SDK appends `/v1/messages`); a trailing `/v1`
pasted out of openai-compatible habit is stripped automatically. Set a
placeholder `api_key` (e.g. `"dummy"`) for unauthenticated servers. Tool calling
needs the server started with `--enable-auto-tool-choice
--tool-call-parser <family>` plus the matching reasoning parser.
Per-model capability overrides opt in to what the checkpoint actually
supports:
```toml
[models.vllm-claude]
provider = "anthropic-compatible"
base_url = "http://localhost:8000" # no /v1 — the SDK appends /v1/messages
api_key = "dummy"
model = "deepseek-ai/DeepSeek-V4-Flash"
[models.vllm-claude.capabilities]
supports_vision = true # multimodal checkpoints only
supports_mid_conversation_system = true # template-dependent
context_window = 131072
```
The reasoning toggle does NOT use Anthropic's `thinking` request param.
Toggle it through the chat template instead: set `{"chat_template_kwargs":
{"thinking": false}}` as extra body params in the admin Models
server-compat section (for this provider the section shows only the
extra-body field — server type, API surface, and thinking mode are
openai-compatible-only knobs); the provider forwards it via the SDK's
`extra_body`.
Verified quirks of vLLM's Anthropic endpoint:
* The `thinking` request param is silently dropped — use
`chat_template_kwargs` (above) to control reasoning.
* `stop_sequences` cut the raw stream wherever the text appears —
including inside thinking — and report `end_turn` with
`stop_sequence=None`. Turnstone does not send stop sequences from
this provider.
* No cache telemetry: `usage` carries input/output token counts only
(no `cache_creation_input_tokens` / `cache_read_input_tokens`).
* Images require a multimodal checkpoint — text-only models return a
500 on image blocks, so `supports_vision` stays opt-in per model.
* Mid-conversation `role: "system"` turns are template-dependent —
opt in per model via `supports_mid_conversation_system`.
**Database model definitions:** On server entry points, models can also be
defined in the `model_definitions` table (admin Models tab). DB models support
the same per-model sampling overrides. Config.toml models override DB models
+19
View File
@@ -164,6 +164,25 @@ class TestProbeModelEndpoint:
assert result["server_type"] == "anthropic"
assert result["context_window"] == 1000000
@patch("turnstone.core.providers.create_client")
def test_anthropic_compatible_server_type(self, mock_cc: MagicMock) -> None:
m = _mock_model("deepseek-ai/DeepSeek-V4-Flash")
mock_cc.return_value = _mock_client(m)
result = probe_model_endpoint("anthropic-compatible", "http://localhost:8000", "dummy")
assert result["reachable"] is True
assert result["server_type"] == "anthropic-compatible"
assert result["context_window"] is None
@patch("turnstone.core.providers.create_client")
def test_anthropic_compatible_max_model_len(self, mock_cc: MagicMock) -> None:
m = _mock_model("deepseek-ai/DeepSeek-V4-Flash", max_model_len=131072)
mock_cc.return_value = _mock_client(m)
result = probe_model_endpoint("anthropic-compatible", "http://localhost:8000", "dummy")
assert result["context_window"] == 131072
assert result["server_type"] == "anthropic-compatible"
@patch("turnstone.core.providers.create_client")
def test_connection_failure(self, mock_cc: MagicMock) -> None:
mock_cc.side_effect = OSError("Connection refused")
+18
View File
@@ -142,6 +142,24 @@ def test_create_emits_models_changed(storage: SQLiteBackend) -> None:
assert collector.emit_models_changed.call_count == 1
def test_create_accepts_anthropic_compatible_provider(storage: SQLiteBackend) -> None:
"""anthropic-compatible passes the _MODEL_PROVIDERS enum check."""
client, collector = _make_client(storage)
resp = client.post(
"/v1/api/admin/model-definitions",
json={
"alias": "vllm-messages",
"model": "deepseek-ai/DeepSeek-V4-Flash",
"provider": "anthropic-compatible",
"base_url": "http://localhost:8000",
"api_key": "dummy",
"context_window": 131072,
},
)
assert resp.status_code == 200, resp.text
assert collector.emit_models_changed.call_count == 1
def test_update_emits_models_changed(storage: SQLiteBackend) -> None:
_seed(storage, definition_id="m1", alias="local")
client, collector = _make_client(storage)
+347
View File
@@ -0,0 +1,347 @@
"""Tests for the ``anthropic-compatible`` provider lane.
Local servers (vLLM) expose Anthropic's ``/v1/messages`` wire surface for
arbitrary checkpoints. The lane reuses ``AnthropicProvider`` with
``compat=True``: identical message translation, but capabilities come from
``_ANTHROPIC_COMPAT_DEFAULT`` for every model (the static Claude table
never applies), native server-side tools are not injected, and operator
``server_compat["extra_body"]`` overrides ride the Anthropic SDK's
``extra_body`` — the channel for vLLM's ``chat_template_kwargs`` reasoning
toggle.
"""
from __future__ import annotations
import os
import sys
from typing import Any
from unittest.mock import MagicMock, patch
import pytest
from tests._session_helpers import make_session as _make_session
from turnstone.core.providers._anthropic import AnthropicProvider
# ---------------------------------------------------------------------------
# Helpers
# ---------------------------------------------------------------------------
def _capture_client() -> MagicMock:
"""Build a fake Anthropic client whose ``messages.stream`` records kwargs."""
stream_ctx = MagicMock()
stream_ctx.__enter__ = MagicMock(return_value=iter([]))
stream_ctx.__exit__ = MagicMock(return_value=False)
client = MagicMock()
client.messages.stream.return_value = stream_ctx
return client
_WEB_SEARCH_FUNCTION_TOOL = {
"type": "function",
"function": {
"name": "web_search",
"description": "Search the web",
"parameters": {
"type": "object",
"properties": {"query": {"type": "string"}},
"required": ["query"],
},
},
}
# ===========================================================================
# TestCompatCapabilities
# ===========================================================================
class TestCompatCapabilities:
"""Capability resolution on the compat lane."""
def test_compat_capability_defaults(self) -> None:
provider = AnthropicProvider(compat=True)
caps = provider.get_capabilities("deepseek-ai/DeepSeek-V4-Flash")
assert caps.token_param == "max_tokens"
assert caps.thinking_mode == "none"
assert caps.supports_web_search is False
assert caps.supports_tool_search is False
assert caps.supports_vision is False
assert caps.supports_reasoning_replay is True
assert caps.supports_temperature is True
def test_claude_id_does_not_pick_up_static_table(self) -> None:
"""A Claude-named local checkpoint must not inherit Claude API caps."""
provider = AnthropicProvider(compat=True)
caps = provider.get_capabilities("claude-opus-4-6")
assert caps.context_window == 200000
assert caps.thinking_mode == "none"
assert caps.supports_web_search is False
# The real lane still resolves the static entry.
real_caps = AnthropicProvider().get_capabilities("claude-opus-4-6")
assert real_caps.context_window == 1000000
assert real_caps.thinking_mode == "adaptive"
# ===========================================================================
# TestCompatWireShape
# ===========================================================================
class TestCompatWireShape:
"""Body-inspecting tests on the kwargs handed to ``messages.stream``."""
def setup_method(self) -> None:
self.provider = AnthropicProvider(compat=True)
@patch("turnstone.core.providers._anthropic._ensure_anthropic")
def test_compat_no_web_search_swap_no_temp_force(self, mock_ensure: MagicMock) -> None:
"""No native web_search swap, no temperature=1 forcing, max_tokens param."""
client = _capture_client()
list(
self.provider.create_streaming(
client=client,
model="deepseek-ai/DeepSeek-V4-Flash",
messages=[{"role": "user", "content": "hi"}],
tools=[_WEB_SEARCH_FUNCTION_TOOL],
temperature=0.6,
)
)
kwargs = client.messages.stream.call_args[1]
sent_tools = kwargs["tools"]
assert sent_tools == [
{
"name": "web_search",
"description": "Search the web",
"input_schema": {
"type": "object",
"properties": {"query": {"type": "string"}},
"required": ["query"],
},
}
]
assert all(t.get("type") != "web_search_20250305" for t in sent_tools)
assert kwargs["temperature"] == 0.6
assert "thinking" not in kwargs
assert "max_tokens" in kwargs
assert "max_completion_tokens" not in kwargs
@patch("turnstone.core.providers._anthropic._ensure_anthropic")
def test_extra_params_passthrough_to_extra_body(self, mock_ensure: MagicMock) -> None:
"""server_compat extra_body (chat_template_kwargs) reaches the SDK."""
client = _capture_client()
list(
self.provider.create_streaming(
client=client,
model="deepseek-ai/DeepSeek-V4-Flash",
messages=[{"role": "user", "content": "hi"}],
extra_params={"chat_template_kwargs": {"thinking": False}},
)
)
kwargs = client.messages.stream.call_args[1]
assert kwargs["extra_body"] == {"chat_template_kwargs": {"thinking": False}}
@patch("turnstone.core.providers._anthropic._ensure_anthropic")
def test_internal_keys_not_leaked(self, mock_ensure: MagicMock) -> None:
"""Real-lane request bodies stay byte-identical with thinking overrides.
``thinking_budget_tokens`` is consumed by ``_reasoning_params`` and
must never surface as wire ``extra_body`` — a leaked key would change
every real-Anthropic request that threads a thinking override.
Negative-tested: fails when the ``_INTERNAL_EXTRA_PARAMS`` exclusion
is removed from ``_build_thinking_and_kwargs``.
"""
provider = AnthropicProvider()
client = _capture_client()
list(
provider.create_streaming(
client=client,
model="claude-sonnet-4-5",
messages=[{"role": "user", "content": "hi"}],
extra_params={"thinking_budget_tokens": 2048},
)
)
kwargs = client.messages.stream.call_args[1]
assert "extra_body" not in kwargs
assert kwargs["thinking"] == {"type": "enabled", "budget_tokens": 2048}
# ===========================================================================
# TestCompatFactory
# ===========================================================================
class TestCompatFactory:
"""create_provider / create_client routing for the compat lane."""
def test_create_provider_anthropic_compatible(self) -> None:
from turnstone.core.providers import create_provider
provider = create_provider("anthropic-compatible")
assert provider.provider_name == "anthropic-compatible"
assert provider is not create_provider("anthropic")
assert create_provider("anthropic-compatible") is provider
@patch("turnstone.core.providers._anthropic._ensure_anthropic")
def test_create_client_anthropic_compatible(self, mock_ensure: MagicMock) -> None:
"""base_url forwards verbatim; empty api_key is omitted entirely."""
from turnstone.core.providers import create_client
mock_anthropic_cls = MagicMock()
mock_mod = MagicMock()
mock_mod.Anthropic = mock_anthropic_cls
mock_ensure.return_value = mock_mod
create_client("anthropic-compatible", base_url="http://vllm-host:8000", api_key="")
mock_anthropic_cls.assert_called_once_with(base_url="http://vllm-host:8000")
@patch("turnstone.core.providers._anthropic._ensure_anthropic")
def test_create_client_strips_v1_suffix(self, mock_ensure: MagicMock) -> None:
"""A /v1-suffixed base_url (openai-compatible muscle memory) is
normalized for the compat lane — the SDK appends /v1/... itself,
so the verbatim URL would request /v1/v1/messages and 404."""
from turnstone.core.providers import create_client
mock_anthropic_cls = MagicMock()
mock_mod = MagicMock()
mock_mod.Anthropic = mock_anthropic_cls
mock_ensure.return_value = mock_mod
for suffixed in ("http://vllm-host:8000/v1", "http://vllm-host:8000/v1/"):
mock_anthropic_cls.reset_mock()
create_client("anthropic-compatible", base_url=suffixed, api_key="")
mock_anthropic_cls.assert_called_once_with(base_url="http://vllm-host:8000")
# A base_url that strips to nothing stays verbatim so the typo
# fails loudly in httpx instead of silently targeting the SDK's
# prod default.
mock_anthropic_cls.reset_mock()
create_client("anthropic-compatible", base_url="/v1", api_key="")
mock_anthropic_cls.assert_called_once_with(base_url="/v1")
@patch("turnstone.core.providers._anthropic._ensure_anthropic")
def test_create_client_real_lane_base_url_untouched(self, mock_ensure: MagicMock) -> None:
"""The real anthropic lane forwards base_url verbatim — the /v1
normalization is compat-lane-only."""
from turnstone.core.providers import create_client
mock_anthropic_cls = MagicMock()
mock_mod = MagicMock()
mock_mod.Anthropic = mock_anthropic_cls
mock_ensure.return_value = mock_mod
create_client("anthropic", base_url="http://proxy:9000/v1", api_key="k")
mock_anthropic_cls.assert_called_once_with(api_key="k", base_url="http://proxy:9000/v1")
# ===========================================================================
# TestCliScope
# ===========================================================================
class TestCliScope:
def test_cli_rejects_compat_provider_id(self, monkeypatch: pytest.MonkeyPatch) -> None:
"""The lane is registry-only — the CLI --provider flag does not grow."""
from turnstone import cli
monkeypatch.setattr(sys, "argv", ["turnstone", "--provider", "anthropic-compatible"])
with pytest.raises(SystemExit) as excinfo:
cli.main()
assert excinfo.value.code == 2
# ===========================================================================
# TestCompatSessionPlumbing
# ===========================================================================
class TestCompatSessionPlumbing:
"""ChatSession capability merge + extra_params gate for the lane."""
def test_per_model_capability_override_merge(self, tmp_db: Any) -> None:
"""Per-model capabilities win over _ANTHROPIC_COMPAT_DEFAULT fields."""
from turnstone.core.model_registry import ModelConfig, ModelRegistry
from turnstone.core.providers import create_provider
cfg = ModelConfig(
alias="vllm-messages",
base_url="http://localhost:8000",
api_key="dummy",
model="deepseek-ai/DeepSeek-V4-Flash",
provider="anthropic-compatible",
capabilities={"supports_mid_conversation_system": True, "context_window": 131072},
)
registry = ModelRegistry(models={"vllm-messages": cfg}, default="vllm-messages")
session = _make_session(registry=registry, model_alias="vllm-messages")
provider = create_provider("anthropic-compatible")
caps = session._resolve_capabilities(
provider, "deepseek-ai/DeepSeek-V4-Flash", "vllm-messages"
)
assert caps.supports_mid_conversation_system is True
assert caps.context_window == 131072
# Untouched fields keep the compat-lane defaults.
assert caps.token_param == "max_tokens"
assert caps.thinking_mode == "none"
assert caps.supports_web_search is False
assert caps.supports_vision is False
def test_session_extra_params_gate(self, tmp_db: Any) -> None:
"""server_compat extra_body forwards for the compat lane, not real Anthropic."""
from turnstone.core.model_registry import ModelConfig, ModelRegistry
from turnstone.core.providers import create_provider
session = _make_session(reasoning_effort="medium")
cfg = ModelConfig(
alias="vllm-messages",
base_url="http://localhost:8000",
api_key="dummy",
model="deepseek-ai/DeepSeek-V4-Flash",
provider="anthropic-compatible",
server_compat={"extra_body": {"chat_template_kwargs": {"thinking": False}}},
)
session._registry = ModelRegistry(models={"vllm-messages": cfg}, default="vllm-messages")
session._model_alias = "vllm-messages"
session._provider = create_provider("anthropic-compatible")
assert session._provider_extra_params() == {"chat_template_kwargs": {"thinking": False}}
session._provider = create_provider("anthropic")
assert session._provider_extra_params() is None
# ===========================================================================
# TestLiveCompatStream
# ===========================================================================
@pytest.mark.live
@pytest.mark.skipif(
not os.environ.get("TURNSTONE_LIVE_ANTHROPIC_COMPAT_URL"),
reason="TURNSTONE_LIVE_ANTHROPIC_COMPAT_URL not set",
)
class TestLiveCompatStream:
"""One real streamed turn against a vLLM /v1/messages endpoint."""
def test_live_compat_streamed_turn(self) -> None:
from turnstone.core.providers import create_client, create_provider
base_url = os.environ["TURNSTONE_LIVE_ANTHROPIC_COMPAT_URL"]
model = os.environ.get(
"TURNSTONE_LIVE_ANTHROPIC_COMPAT_MODEL", "deepseek-ai/DeepSeek-V4-Flash"
)
client = create_client("anthropic-compatible", base_url=base_url, api_key="dummy")
provider = create_provider("anthropic-compatible")
chunks = list(
provider.create_streaming(
client=client,
model=model,
messages=[{"role": "user", "content": "Reply with the single word: pong"}],
max_tokens=64,
extra_params={"chat_template_kwargs": {"thinking": False}},
)
)
content = "".join(c.content_delta or "" for c in chunks)
assert content.strip()
assert any(c.finish_reason for c in chunks)
assert any(c.usage is not None for c in chunks)
assert not any(c.reasoning_delta for c in chunks)
+3 -1
View File
@@ -10210,7 +10210,9 @@ async def admin_import_mcp_config(request: Request) -> JSONResponse:
# ---------------------------------------------------------------------------
_MODEL_ALIAS_RE = re.compile(r"^[a-zA-Z0-9._-]+$")
_MODEL_PROVIDERS = frozenset({"openai", "anthropic", "openai-compatible", "google", "xai"})
_MODEL_PROVIDERS = frozenset(
{"openai", "anthropic", "openai-compatible", "anthropic-compatible", "google", "xai"}
)
_REASONING_EFFORT_CHOICES = frozenset(
{"", "none", "minimal", "low", "medium", "high", "xhigh", "max"}
)
+53 -19
View File
@@ -5596,7 +5596,8 @@ function _renderModels(items) {
? "model-provider-anthropic"
: m.provider === "google"
? "model-provider-google"
: m.provider === "openai-compatible"
: m.provider === "openai-compatible" ||
m.provider === "anthropic-compatible"
? "model-provider-compat"
: "model-provider-openai";
@@ -5896,12 +5897,16 @@ function showEditModelModal(definitionId) {
? capsObj.server_compat
: {};
// Only extract thinking_mode into the dropdown when the UI can
// represent it ("manual" or ""). Values like "adaptive" (Anthropic-
// only) stay in the raw capabilities JSON so they aren't silently
// lost on save.
// represent it ("manual" or "") AND the provider round-trips the
// dropdown on save (every provider except anthropic-compatible —
// see submitCreateModel). Unrepresentable values like "adaptive"
// and anthropic-compatible rows keep thinking_mode in the raw
// capabilities JSON so it isn't silently lost on save.
const tmVal = capsObj.thinking_mode || "";
const tmRepresentable = tmVal === "" || tmVal === "manual";
if (tmRepresentable) {
const tmCaptured =
tmRepresentable && (m.provider || "openai") !== "anthropic-compatible";
if (tmCaptured) {
document.getElementById("model-thinking-mode").value = tmVal;
document.getElementById("model-thinking-param").value =
capsObj.thinking_param || "";
@@ -5945,7 +5950,7 @@ function showEditModelModal(definitionId) {
// Remove structured fields from capabilities display — only delete
// thinking_mode/thinking_param when the UI successfully captured them.
delete capsObj.server_compat;
if (tmRepresentable) {
if (tmCaptured) {
delete capsObj.thinking_mode;
delete capsObj.thinking_param;
}
@@ -6017,10 +6022,16 @@ function submitCreateModel() {
}
}
// Thinking mode → capabilities (provider uses this to inject
// the correct chat_template_kwargs param automatically).
const providerVal = document.getElementById("model-provider").value;
// Thinking mode → capabilities. thinking_mode round-trips through the
// dropdown for every provider EXCEPT anthropic-compatible, where it
// stays in the raw capabilities JSON (mirroring the edit-load lift):
// that lane hides the dropdown row and drives reasoning via extra-body
// chat_template_kwargs, so a lingering dropdown value must never be
// persisted.
const thinkingMode = document.getElementById("model-thinking-mode").value;
if (thinkingMode) {
if (providerVal !== "anthropic-compatible" && thinkingMode) {
caps.thinking_mode = thinkingMode;
// Preserve thinking_param so Granite/DeepSeek "thinking" key
// isn't silently reverted to the default "enable_thinking".
@@ -6028,20 +6039,25 @@ function submitCreateModel() {
if (savedParam) caps.thinking_param = savedParam;
}
// Build server_compat from structured fields. Only meaningful for
// openai-compatible aliases — for other providers the section is hidden
// Build server_compat from structured fields. Only meaningful for the
// compat lanes (openai-compatible: all fields; anthropic-compatible: the
// extra-body JSON only) — for other providers the section is hidden
// but the form values can linger after a provider switch, so gate the
// whole block on the active provider to keep persisted state honest.
const serverCompat = {};
const providerVal = document.getElementById("model-provider").value;
const ebEl = document.getElementById("model-extra-body");
ebEl.removeAttribute("aria-invalid");
ebEl.style.borderColor = "";
if (providerVal === "openai-compatible") {
const serverType = document.getElementById("model-server-type").value;
if (serverType) serverCompat.server_type = serverType;
const apiSurface = document.getElementById("model-api-surface").value;
if (apiSurface) serverCompat.api_surface = apiSurface;
if (
providerVal === "openai-compatible" ||
providerVal === "anthropic-compatible"
) {
if (providerVal === "openai-compatible") {
const serverType = document.getElementById("model-server-type").value;
if (serverType) serverCompat.server_type = serverType;
const apiSurface = document.getElementById("model-api-surface").value;
if (apiSurface) serverCompat.api_surface = apiSurface;
}
const ebText = ebEl.value.trim();
if (ebText) {
try {
@@ -6516,7 +6532,11 @@ function _modelCapsRefreshBaseline() {
const provider = document.getElementById("model-provider").value;
const modelName = document.getElementById("model-name").value.trim();
const banner = document.getElementById("model-autofill");
if (!modelName || provider === "openai-compatible") {
if (
!modelName ||
provider === "openai-compatible" ||
provider === "anthropic-compatible"
) {
_modelCapsBaseline = {};
banner.hidden = true;
_modelRenderTiles();
@@ -6578,6 +6598,10 @@ const _providerDefaults = {
urlPlaceholder: "e.g. https://your-provider.com/v1",
modelPlaceholder: "GLM5",
},
"anthropic-compatible": {
urlPlaceholder: "e.g. http://your-vllm-host:8000",
modelPlaceholder: "deepseek-ai/DeepSeek-V4-Flash",
},
};
/* Update placeholders when provider changes. */
@@ -6592,8 +6616,18 @@ function _applyProviderDefaults() {
if (scSection) {
// hidden attr, not style.display — `.hatch [hidden]` is !important and
// an inline display can never un-hide it.
scSection.hidden = provider !== "openai-compatible";
scSection.hidden =
provider !== "openai-compatible" && provider !== "anthropic-compatible";
}
// Within the section, server type / API surface / thinking mode are
// openai-compatible knobs — the anthropic-compatible lane is configured
// through the extra-body JSON alone, so collapse the section to just
// that field.
const hideOpenaiOnlyRows = provider === "anthropic-compatible";
["model-server-fields-row", "model-thinking-mode-row"].forEach(function (id) {
const row = document.getElementById(id);
if (row) row.hidden = hideOpenaiOnlyRows;
});
}
/* Populate the model name datalist with known model prefixes for the
+25 -20
View File
@@ -1577,6 +1577,7 @@
<option value="anthropic">anthropic</option>
<option value="google">google</option>
<option value="openai-compatible">openai-compatible</option>
<option value="anthropic-compatible">anthropic-compatible</option>
</select>
</div>
<div>
@@ -1673,7 +1674,7 @@
<div id="model-server-compat-section" hidden>
<div class="sh-section">Server compatibility</div>
<div class="field-pair">
<div class="field-pair" id="model-server-fields-row">
<div>
<label for="model-server-type"
>Server type
@@ -1702,28 +1703,32 @@
</select>
</div>
</div>
<label for="model-thinking-mode"
>Thinking mode
<span class="label-hint">chat-template reasoning</span></label
>
<select id="model-thinking-mode">
<option value="">None</option>
<option value="manual">Enabled</option>
</select>
<div id="model-thinking-param-row" hidden>
<label for="model-thinking-param"
>Template param name
<div id="model-thinking-mode-row">
<label for="model-thinking-mode"
>Thinking mode
<span class="label-hint"
>Granite/DeepSeek use "thinking"</span
>chat-template reasoning</span
></label
>
<input
type="text"
id="model-thinking-param"
class="sh-mono"
value="enable_thinking"
placeholder="enable_thinking"
/>
<select id="model-thinking-mode">
<option value="">None</option>
<option value="manual">Enabled</option>
</select>
<div id="model-thinking-param-row" hidden>
<label for="model-thinking-param"
>Template param name
<span class="label-hint"
>Granite/DeepSeek use "thinking"</span
></label
>
<input
type="text"
id="model-thinking-param"
class="sh-mono"
value="enable_thinking"
placeholder="enable_thinking"
/>
</div>
</div>
<label for="model-extra-body"
>Extra body params
+8
View File
@@ -817,6 +817,14 @@ def probe_model_endpoint(
if known is not None:
result["context_window"] = known["context_window"]
result["server_type"] = "xai"
elif provider == "anthropic-compatible":
# No static table for local models; vLLM exposes max_model_len
# as an extra field the Anthropic SDK preserves (extra="allow").
if inspect_obj is not None:
max_len = inspect_obj.model_dump().get("max_model_len")
if isinstance(max_len, int) and max_len > 0:
result["context_window"] = max_len
result["server_type"] = "anthropic-compatible"
else:
# OpenAI-compatible path
_detect_openai_compat(result, inspect_obj, inspect_id, base_url)
+31 -8
View File
@@ -46,6 +46,7 @@ _openai_provider = OpenAIResponsesProvider()
_openai_compat_provider = OpenAIChatCompletionsProvider()
_xai_provider = XAIProvider()
_anthropic_provider: LLMProvider | None = None
_anthropic_compat_provider: LLMProvider | None = None
_google_provider: LLMProvider | None = None
@@ -67,8 +68,13 @@ def create_provider(
endpoints like Mistral cloud, or local servers that expose the
Responses surface).
Ignored for non-OpenAI providers. ``provider_name="openai"`` always
uses the Responses API regardless of *api_surface*.
Ignored for non-OpenAI providers — both Anthropic lanes
(``"anthropic"`` and ``"anthropic-compatible"``) talk to the
Messages API regardless of *api_surface*.
``provider_name="anthropic-compatible"`` returns the Anthropic
adapter in compat mode (local servers exposing ``/v1/messages``,
e.g. vLLM). ``provider_name="openai"`` always uses the Responses
API regardless of *api_surface*.
Note: the ``OpenAIResponsesProvider`` singleton is reused for both
cloud OpenAI and ``openai-compatible`` + responses, so its
@@ -77,7 +83,7 @@ def create_provider(
must read ``ModelConfig.provider`` and ``server_compat["api_surface"]``
rather than ``provider.provider_name``.
"""
global _anthropic_provider, _google_provider # noqa: PLW0603
global _anthropic_provider, _anthropic_compat_provider, _google_provider # noqa: PLW0603
if provider_name == "openai":
return _openai_provider
if provider_name == "openai-compatible":
@@ -98,6 +104,13 @@ def create_provider(
_anthropic_provider = AnthropicProvider()
return _anthropic_provider
if provider_name == "anthropic-compatible":
with _provider_lock:
if _anthropic_compat_provider is None:
from turnstone.core.providers._anthropic import AnthropicProvider
_anthropic_compat_provider = AnthropicProvider(compat=True)
return _anthropic_compat_provider
if provider_name == "google":
with _provider_lock:
if _google_provider is None:
@@ -107,7 +120,7 @@ def create_provider(
return _google_provider
raise ValueError(
f"Unknown provider: {provider_name!r}. "
"Supported: openai, anthropic, google, openai-compatible, xai"
"Supported: openai, anthropic, google, openai-compatible, anthropic-compatible, xai"
)
@@ -133,19 +146,28 @@ def create_client(provider_name: str, *, base_url: str, api_key: str) -> Any:
if base_url:
return OpenAI(base_url=base_url, api_key=resolved_key)
return OpenAI(api_key=resolved_key)
if provider_name == "anthropic":
if provider_name in ("anthropic", "anthropic-compatible"):
from turnstone.core.providers._anthropic import _ensure_anthropic
anthropic = _ensure_anthropic()
kwargs: dict[str, str] = {}
if resolved_key is not None:
kwargs["api_key"] = resolved_key
if provider_name == "anthropic-compatible" and base_url:
# The Anthropic SDK appends /v1/... to base_url, so a
# /v1-suffixed URL (the openai-compatible convention) would
# request /v1/v1/messages and 404. Tolerate the suffix.
# Keep the verbatim value when stripping would empty it
# (base_url of exactly "/v1") so the typo still fails loudly
# instead of silently retargeting the SDK's prod default.
stripped = base_url.rstrip("/").removesuffix("/v1")
base_url = stripped or base_url
if base_url and base_url != "https://api.anthropic.com":
kwargs["base_url"] = base_url
return anthropic.Anthropic(**kwargs)
raise ValueError(
f"Unknown provider: {provider_name!r}. "
"Supported: openai, anthropic, google, openai-compatible, xai"
"Supported: openai, anthropic, google, openai-compatible, anthropic-compatible, xai"
)
@@ -153,11 +175,12 @@ def lookup_model_capabilities(provider: str, model: str) -> dict[str, Any] | Non
"""Return static capabilities for a known model, or ``None`` if unknown.
The returned dict has JSON-friendly values (tuples converted to lists).
Returns ``None`` for ``openai-compatible`` (no static table for local models).
Returns ``None`` for ``openai-compatible`` and ``anthropic-compatible``
(no static table for local models).
"""
import dataclasses
if provider == "openai-compatible":
if provider in ("openai-compatible", "anthropic-compatible"):
return None
prov = create_provider(provider)
caps = prov.get_capabilities(model)
+46 -2
View File
@@ -71,6 +71,11 @@ _WEB_SEARCH_TOOL_TYPE = "web_search_20250305"
# Tool search: server-side BM25 tool discovery for deferred tools
_TOOL_SEARCH_TOOL_TYPE = "tool_search_tool_bm25"
# extra_params keys consumed internally (``_reasoning_params``) — never
# forwarded to the wire ``extra_body``. Keeps real-Anthropic request
# bodies byte-identical when a caller threads thinking overrides.
_INTERNAL_EXTRA_PARAMS = frozenset({"thinking_budget_tokens"})
# -- model capabilities -------------------------------------------------------
_ANTHROPIC_DEFAULT = ModelCapabilities(
@@ -83,6 +88,27 @@ _ANTHROPIC_DEFAULT = ModelCapabilities(
supports_reasoning_replay=True,
)
# Anthropic-compatible local servers (vLLM's /v1/messages endpoint):
# token_param must be "max_tokens" (the only token param the endpoint
# accepts); the "thinking" request param is not consumed by vLLM — the
# reasoning toggle rides chat_template_kwargs via extra_body, so
# thinking_mode stays "none"; supports_reasoning_replay stays True even
# so, because the endpoint emits and round-trips thinking blocks whenever
# the chat template enables reasoning (the request param is simply not
# the switch); native web_search / tool_search server-tool types 400 on
# vLLM (tools require input_schema); vision is opt-in per model.
# supports_temperature stays True via the dataclass default.
_ANTHROPIC_COMPAT_DEFAULT = ModelCapabilities(
context_window=200000,
max_output_tokens=64000,
token_param="max_tokens",
thinking_mode="none",
supports_web_search=False,
supports_tool_search=False,
supports_vision=False,
supports_reasoning_replay=True,
)
_ANTHROPIC_CAPABILITIES: dict[str, ModelCapabilities] = {
# Fable 5: same wire surface as opus-4-8 (adaptive-only thinking, no
# sampling params, prefill rejected) with one extra constraint — an
@@ -239,13 +265,24 @@ ANTHROPIC_REASONING_BLOCK_TYPES = frozenset({"thinking", "redacted_thinking"})
class AnthropicProvider:
"""Provider for Anthropic's Messages API with native streaming."""
"""Provider for Anthropic's Messages API with native streaming.
``compat=True`` serves Anthropic-compatible local servers (vLLM's
``/v1/messages``): same wire translation, but capabilities come from
``_ANTHROPIC_COMPAT_DEFAULT`` for every model — the static Claude
table never applies to local checkpoints.
"""
def __init__(self, *, compat: bool = False) -> None:
self._compat = compat
@property
def provider_name(self) -> str:
return "anthropic"
return "anthropic-compatible" if self._compat else "anthropic"
def get_capabilities(self, model: str) -> ModelCapabilities:
if self._compat:
return _ANTHROPIC_COMPAT_DEFAULT
return _lookup_capabilities(model, _ANTHROPIC_CAPABILITIES, _ANTHROPIC_DEFAULT)
# -- web search tool injection -------------------------------------------
@@ -352,6 +389,13 @@ class AnthropicProvider:
if effort:
kwargs["output_config"] = {"effort": effort}
# Operator server_compat extra_body overrides (e.g. chat_template_kwargs
# for anthropic-compatible local servers) ride the SDK's extra_body.
if extra_params:
wire_extra = {k: v for k, v in extra_params.items() if k not in _INTERNAL_EXTRA_PARAMS}
if wire_extra:
kwargs["extra_body"] = wire_extra
return kwargs
# -- message conversion --------------------------------------------------
+9 -4
View File
@@ -3135,7 +3135,10 @@ class ChatSession:
Forwards operator-supplied ``server_compat["extra_body"]`` overrides
(``skip_special_tokens``, ``reasoning_format``, or explicit
``chat_template_kwargs``) to the OpenAI SDK ``extra_body``. Operators
``chat_template_kwargs``) to the OpenAI SDK ``extra_body`` on the
OpenAI-shaped lanes, and to the Anthropic SDK ``extra_body`` on the
anthropic-compatible lane (the channel for vLLM's
``chat_template_kwargs`` reasoning toggle). Operators
running gpt-oss-style local templates that consume ``reasoning_effort``
from ``chat_template_kwargs`` should set it explicitly under
``server_compat["extra_body"]["chat_template_kwargs"]``.
@@ -3151,9 +3154,11 @@ class ChatSession:
from turnstone.core.server_compat import merge_server_compat
prov = provider or self._provider
# Only OpenAI-shaped providers consume extra_body. Anthropic/Google
# have their own param paths handled inside their providers.
if prov.provider_name not in ("openai", "openai-compatible"):
# extra_body consumers: the OpenAI-shaped providers, plus the
# anthropic-compatible lane (server_compat extra_body rides the
# Anthropic SDK's extra_body). Real Anthropic and Google keep
# their own param paths handled inside their providers.
if prov.provider_name not in ("openai", "openai-compatible", "anthropic-compatible"):
return None
extra = merge_server_compat(None, self._get_server_compat(model_alias))
return extra or None