mirror of
https://github.com/turnstonelabs/turnstone.git
synced 2026-08-12 23:12:23 -06:00
fix(providers): the session effort level always reaches the local-lane wire
Local lanes dropped the knob's graded value unless the operator declared
reasoning_effort_values (and, on the template channel, an effort key) —
picking Max sent a bare thinking toggle and the effort select
degenerated into seven positions that all meant 'on'. The user's
setting now always rides:
- openai-compatible: the flat reasoning_effort param carries the knob
verbatim (effort_passthrough on the lane default); declared values
still snap ordinally, and a declared effort_param still claims the
template channel and suppresses the flat param.
- anthropic-compatible: the graded value rides chat_template_kwargs
alongside the toggle whenever reasoning control is engaged — under
the operator's effort_param, else the conventional fallback key
(reasoning_effort); templates that don't reference the kwarg ignore
it. thinking_mode=none still injects nothing.
- Commercial lanes untouched: empty declared values still mean 'no
effort control' (o1-mini) and the ordinal snap is unchanged.
Golden writer now pins ensure_ascii=False: the baselines' literal em
dashes came from a hand edit (03f82521) the default-escaping writer
could never reproduce — regens no longer churn unrelated lines.
This commit is contained in:
+16
-7
@@ -824,12 +824,17 @@ the levers live in the chat template, reached through
|
||||
`true` — the same contract as the real lane's manual mode. ("Always
|
||||
on" / `thinking_mode = "adaptive"` instead always sends `true`: the
|
||||
model self-regulates, so the knob never force-disables — mirroring
|
||||
the native adaptive branch.) Templates
|
||||
with a graded effort key (gpt-oss-style) additionally set
|
||||
`effort_param` (e.g. `"reasoning_effort"`), plus optional
|
||||
`reasoning_effort_values` / `default_reasoning_effort` to validate
|
||||
the knob before it reaches the template; without declared values the
|
||||
knob is forwarded as-is. The knob is ordinal, and validation
|
||||
the native adaptive branch.) The graded effort value always rides
|
||||
alongside the toggle: under `effort_param` when the operator names
|
||||
the template's key, else under the conventional fallback key
|
||||
(`reasoning_effort`) on the anthropic-compatible lane — the user's
|
||||
effort setting always reaches the wire, and a template that doesn't
|
||||
reference the kwarg ignores it. On the openai-compatible lane the
|
||||
undeclared-key case rides the flat top-level `reasoning_effort`
|
||||
param instead (the documented compat field), forwarded verbatim.
|
||||
Optional `reasoning_effort_values` / `default_reasoning_effort`
|
||||
validate the knob before it reaches the server; without declared
|
||||
values the knob is forwarded as-is. The knob is ordinal, and validation
|
||||
respects that: an off-list knob value rounds UP onto the declared
|
||||
list and a value above the ceiling rides the ceiling
|
||||
(`snap_reasoning_effort`) — asking for more effort than the model
|
||||
@@ -858,7 +863,11 @@ the levers live in the chat template, reached through
|
||||
toggle unconditionally `true` whenever thinking mode was enabled. A
|
||||
stored per-model `reasoning_effort = "none"` now disables thinking
|
||||
on such models — pick any real level (or clear the override) to keep
|
||||
it on.
|
||||
it on. Also since 1.7.0a7 the effort level itself always reaches the
|
||||
wire on the local lanes (previously dropped unless
|
||||
`reasoning_effort_values` was declared): flat `reasoning_effort` on
|
||||
openai-compatible, the `effort_param`-or-fallback template key on
|
||||
anthropic-compatible when reasoning control is engaged.
|
||||
* **Operator pin (static).** Entries under `{"chat_template_kwargs":
|
||||
...}` in the admin Models extra-body field ride the SDK's
|
||||
`extra_body` unconditionally and win over the knob mapping on key
|
||||
|
||||
@@ -33,7 +33,7 @@
|
||||
"tool_call_id": "call_1"
|
||||
},
|
||||
{
|
||||
"content": "Tool execution was cancelled. Outcome UNKNOWN \u2014 this call may have begun executing before the generation was stopped; do not assume it did not run, and reconcile before re-issuing it.",
|
||||
"content": "Tool execution was cancelled. Outcome UNKNOWN — this call may have begun executing before the generation was stopped; do not assume it did not run, and reconcile before re-issuing it.",
|
||||
"role": "tool",
|
||||
"tool_call_id": "call_2"
|
||||
},
|
||||
|
||||
@@ -20,7 +20,7 @@
|
||||
]
|
||||
},
|
||||
{
|
||||
"content": "Tool execution was cancelled. Outcome UNKNOWN \u2014 this call may have begun executing before the generation was stopped; do not assume it did not run, and reconcile before re-issuing it.",
|
||||
"content": "Tool execution was cancelled. Outcome UNKNOWN — this call may have begun executing before the generation was stopped; do not assume it did not run, and reconcile before re-issuing it.",
|
||||
"role": "tool",
|
||||
"tool_call_id": "call_1"
|
||||
}
|
||||
|
||||
@@ -20,7 +20,7 @@
|
||||
]
|
||||
},
|
||||
{
|
||||
"content": "Tool execution was cancelled. Outcome UNKNOWN \u2014 this call may have begun executing before the generation was stopped; do not assume it did not run, and reconcile before re-issuing it.",
|
||||
"content": "Tool execution was cancelled. Outcome UNKNOWN — this call may have begun executing before the generation was stopped; do not assume it did not run, and reconcile before re-issuing it.",
|
||||
"role": "tool",
|
||||
"tool_call_id": "call_1"
|
||||
}
|
||||
|
||||
@@ -43,6 +43,7 @@
|
||||
}
|
||||
],
|
||||
"model": "gpt-4o-mini",
|
||||
"reasoning_effort": "medium",
|
||||
"stream": true,
|
||||
"stream_options": {
|
||||
"include_usage": true
|
||||
|
||||
@@ -18,6 +18,7 @@
|
||||
}
|
||||
],
|
||||
"model": "gpt-4o-mini",
|
||||
"reasoning_effort": "medium",
|
||||
"stream": true,
|
||||
"stream_options": {
|
||||
"include_usage": true
|
||||
|
||||
@@ -26,6 +26,7 @@
|
||||
}
|
||||
],
|
||||
"model": "gpt-4o-mini",
|
||||
"reasoning_effort": "medium",
|
||||
"stream": true,
|
||||
"stream_options": {
|
||||
"include_usage": true
|
||||
|
||||
@@ -26,6 +26,7 @@
|
||||
}
|
||||
],
|
||||
"model": "gpt-4o-mini",
|
||||
"reasoning_effort": "medium",
|
||||
"stream": true,
|
||||
"stream_options": {
|
||||
"include_usage": true
|
||||
|
||||
@@ -34,6 +34,7 @@
|
||||
}
|
||||
],
|
||||
"model": "gpt-4o-mini",
|
||||
"reasoning_effort": "medium",
|
||||
"stream": true,
|
||||
"stream_options": {
|
||||
"include_usage": true
|
||||
|
||||
@@ -15,6 +15,7 @@
|
||||
}
|
||||
],
|
||||
"model": "gpt-4o-mini",
|
||||
"reasoning_effort": "medium",
|
||||
"stream": true,
|
||||
"stream_options": {
|
||||
"include_usage": true
|
||||
|
||||
@@ -30,6 +30,7 @@
|
||||
}
|
||||
],
|
||||
"model": "gpt-4o-mini",
|
||||
"reasoning_effort": "medium",
|
||||
"stream": true,
|
||||
"stream_options": {
|
||||
"include_usage": true
|
||||
|
||||
@@ -26,6 +26,7 @@
|
||||
}
|
||||
],
|
||||
"model": "gpt-4o-mini",
|
||||
"reasoning_effort": "medium",
|
||||
"stream": true,
|
||||
"stream_options": {
|
||||
"include_usage": true
|
||||
|
||||
@@ -368,8 +368,8 @@ def test_effort_ladder_parses_string_capabilities(storage: SQLiteBackend) -> Non
|
||||
body = _get_models(_make_client(storage))
|
||||
ladder = {r["value"]: r["effective"] for r in body["models"][0]["effort_ladder"]}
|
||||
assert ladder["none"] == "off"
|
||||
assert ladder["medium"] == "on"
|
||||
assert ladder["max"] == "on"
|
||||
assert ladder["medium"] == "on+medium"
|
||||
assert ladder["max"] == "on+max"
|
||||
|
||||
|
||||
def test_effort_ladder_key_survives_malformed_capabilities(
|
||||
|
||||
@@ -44,7 +44,7 @@ def test_valid_request_returns_ladder() -> None:
|
||||
assert resp.status_code == 200, resp.text
|
||||
ladder = {r["value"]: r["effective"] for r in resp.json()["ladder"]}
|
||||
assert ladder["none"] == "off"
|
||||
assert ladder["high"] == "on"
|
||||
assert ladder["high"] == "on+high"
|
||||
|
||||
|
||||
def test_api_surface_switches_projection() -> None:
|
||||
|
||||
@@ -21,12 +21,17 @@ def _as_map(ladder: list[dict[str, str]]) -> dict[str, str]:
|
||||
|
||||
|
||||
class TestLocalLanes:
|
||||
def test_qwen_style_toggle_only_two_groups(self) -> None:
|
||||
"""Toggle-only model: none=off, everything else one 'on' group."""
|
||||
def test_toggle_engaged_carries_graded_value_per_position(self) -> None:
|
||||
"""No declared effort key: the toggle rides the knob AND the graded
|
||||
value is forwarded under the fallback template key — the user's
|
||||
effort setting always reaches the wire (a template that doesn't
|
||||
reference the kwarg ignores it), so every position is distinct."""
|
||||
caps = ModelCapabilities(thinking_mode="manual", thinking_param="enable_thinking")
|
||||
eff = _as_map(effort_ladder("anthropic-compatible", caps))
|
||||
assert eff["none"] == "off"
|
||||
assert {eff[k] for k in KNOB_VALUES if k != "none"} == {"on"}
|
||||
assert eff["minimal"] == "on+minimal"
|
||||
assert eff["max"] == "on+max"
|
||||
assert len({eff[k] for k in KNOB_VALUES}) == len(KNOB_VALUES)
|
||||
|
||||
def test_freeform_effort_param_forwards_each_value(self) -> None:
|
||||
"""deepseek-style config: toggle + verbatim effort per position."""
|
||||
|
||||
@@ -37,7 +37,10 @@ import pytest
|
||||
|
||||
from tests._wire_capture import RecordingClient
|
||||
from turnstone.core.providers import create_provider
|
||||
from turnstone.core.providers._protocol import ModelCapabilities
|
||||
from turnstone.core.providers._protocol import (
|
||||
EFFORT_TEMPLATE_FALLBACK_PARAM,
|
||||
ModelCapabilities,
|
||||
)
|
||||
from turnstone.core.providers.effort_ladder import KNOB_VALUES, effort_ladder
|
||||
|
||||
# Above the largest manual-mode thinking budget (max: 65536) so the
|
||||
@@ -271,14 +274,21 @@ def _wire_payload(shape: Shape, knob: str) -> dict[str, Any]:
|
||||
return dict(client.captured["payload"])
|
||||
|
||||
|
||||
def _effort_wire_subset(payload: dict[str, Any], caps: ModelCapabilities) -> dict[str, Any]:
|
||||
def _effort_wire_subset(payload: dict[str, Any], shape: Shape) -> dict[str, Any]:
|
||||
"""Every effort-related lever in *payload*, normalized across lanes.
|
||||
|
||||
Keys: ``thinking`` (native Anthropic param), ``output_effort``
|
||||
(Anthropic ``output_config.effort``), ``flat`` (Chat Completions
|
||||
``reasoning_effort`` / Responses ``reasoning.effort``), ``toggle``
|
||||
and ``template_effort`` (``extra_body.chat_template_kwargs``).
|
||||
and ``template_effort`` (``extra_body.chat_template_kwargs`` — the
|
||||
graded key is ``caps.effort_param``, else the fallback template key
|
||||
on the anthropic-compatible lane, whose only effort channel is the
|
||||
template).
|
||||
"""
|
||||
caps = shape.caps
|
||||
effort_key = caps.effort_param or (
|
||||
EFFORT_TEMPLATE_FALLBACK_PARAM if shape.provider == "anthropic-compatible" else ""
|
||||
)
|
||||
subset: dict[str, Any] = {}
|
||||
if "thinking" in payload:
|
||||
subset["thinking"] = payload["thinking"]
|
||||
@@ -293,13 +303,13 @@ def _effort_wire_subset(payload: dict[str, Any], caps: ModelCapabilities) -> dic
|
||||
extra_body = payload.get("extra_body")
|
||||
ctk = extra_body.get("chat_template_kwargs") if isinstance(extra_body, dict) else None
|
||||
if isinstance(ctk, dict):
|
||||
known = {caps.thinking_param, caps.effort_param} - {""}
|
||||
known = {caps.thinking_param, effort_key} - {""}
|
||||
unexpected = set(ctk) - known
|
||||
assert not unexpected, f"unexpected chat_template_kwargs keys: {unexpected}"
|
||||
if caps.thinking_param in ctk:
|
||||
subset["toggle"] = ctk[caps.thinking_param]
|
||||
if caps.effort_param and caps.effort_param in ctk:
|
||||
subset["template_effort"] = ctk[caps.effort_param]
|
||||
if effort_key and effort_key in ctk:
|
||||
subset["template_effort"] = ctk[effort_key]
|
||||
return subset
|
||||
|
||||
|
||||
@@ -350,12 +360,12 @@ def _decode_template(provider: str, caps: ModelCapabilities, token: str) -> dict
|
||||
parts = parts[1:]
|
||||
if parts:
|
||||
assert len(parts) == 1, f"unparseable ladder token: {token!r}"
|
||||
if caps.effort_param:
|
||||
if caps.effort_param or provider == "anthropic-compatible":
|
||||
# Declared graded key, or the anthropic-compatible fallback
|
||||
# template key — that lane has no flat channel, so a graded
|
||||
# part there is always template-borne.
|
||||
expected["template_effort"] = parts[0]
|
||||
else:
|
||||
# The anthropic-compatible lane has no flat channel, so a
|
||||
# bare effort part there could only be the template key.
|
||||
assert provider != "anthropic-compatible", token
|
||||
expected["flat"] = parts[0]
|
||||
return expected
|
||||
|
||||
@@ -372,7 +382,7 @@ def test_ladder_tokens_match_wire(shape: Shape) -> None:
|
||||
assert [row["value"] for row in ladder] == list(KNOB_VALUES)
|
||||
for row in ladder:
|
||||
knob, token = row["value"], row["effective"]
|
||||
observed = _effort_wire_subset(_wire_payload(shape, knob), shape.caps)
|
||||
observed = _effort_wire_subset(_wire_payload(shape, knob), shape)
|
||||
expected = _decode_token(shape, token)
|
||||
assert observed == expected, (
|
||||
f"{shape.id}/knob={knob}: ladder says {token!r} which decodes to "
|
||||
@@ -387,9 +397,7 @@ def test_equal_tokens_iff_equal_wire(shape: Shape) -> None:
|
||||
row["value"]: row["effective"]
|
||||
for row in effort_ladder(shape.provider, shape.caps, shape.api_surface)
|
||||
}
|
||||
subsets = {
|
||||
knob: _effort_wire_subset(_wire_payload(shape, knob), shape.caps) for knob in KNOB_VALUES
|
||||
}
|
||||
subsets = {knob: _effort_wire_subset(_wire_payload(shape, knob), shape) for knob in KNOB_VALUES}
|
||||
for a, b in itertools.combinations(KNOB_VALUES, 2):
|
||||
same_token = tokens[a] == tokens[b]
|
||||
same_wire = subsets[a] == subsets[b]
|
||||
|
||||
@@ -217,27 +217,39 @@ class TestCompatReasoningControl:
|
||||
return client.messages.stream.call_args[1]
|
||||
|
||||
def test_manual_toggle_on(self) -> None:
|
||||
"""Any non-none effort turns the template toggle on; wire stays clean."""
|
||||
"""Any non-none effort turns the toggle on AND carries the graded
|
||||
value under the fallback key — the user's effort setting always
|
||||
reaches the wire; a template that doesn't reference the kwarg
|
||||
ignores it."""
|
||||
kwargs = self._stream_kwargs(self._MANUAL_CAPS, "medium")
|
||||
assert kwargs["extra_body"] == {"chat_template_kwargs": {"enable_thinking": True}}
|
||||
assert kwargs["extra_body"] == {
|
||||
"chat_template_kwargs": {"enable_thinking": True, "reasoning_effort": "medium"}
|
||||
}
|
||||
assert "thinking" not in kwargs
|
||||
assert kwargs["temperature"] == 0.6 # never forced to 1.0 on compat
|
||||
|
||||
@pytest.mark.parametrize("knob", ["none", ""])
|
||||
def test_manual_toggle_off(self, knob: str) -> None:
|
||||
"""Effort "none"/empty disables thinking — native manual-mode parity."""
|
||||
"""Effort "none"/empty disables thinking — native manual-mode parity;
|
||||
no effort key rides when thinking is off."""
|
||||
kwargs = self._stream_kwargs(self._MANUAL_CAPS, knob)
|
||||
assert kwargs["extra_body"] == {"chat_template_kwargs": {"enable_thinking": False}}
|
||||
assert "thinking" not in kwargs
|
||||
|
||||
def test_adaptive_always_on(self) -> None:
|
||||
"""Adaptive never knob-disables — native-adaptive contract, no native dict."""
|
||||
"""Adaptive never knob-disables — native-adaptive contract, no native
|
||||
dict; the graded value rides for on-positions only."""
|
||||
caps = dataclasses.replace(self._MANUAL_CAPS, thinking_mode="adaptive")
|
||||
for knob in ("high", "none"):
|
||||
kwargs = self._stream_kwargs(caps, knob)
|
||||
assert kwargs["extra_body"] == {"chat_template_kwargs": {"enable_thinking": True}}
|
||||
assert "thinking" not in kwargs
|
||||
assert kwargs["temperature"] == 0.6
|
||||
kwargs = self._stream_kwargs(caps, "high")
|
||||
assert kwargs["extra_body"] == {
|
||||
"chat_template_kwargs": {"enable_thinking": True, "reasoning_effort": "high"}
|
||||
}
|
||||
assert "thinking" not in kwargs
|
||||
assert kwargs["temperature"] == 0.6
|
||||
kwargs = self._stream_kwargs(caps, "none")
|
||||
assert kwargs["extra_body"] == {"chat_template_kwargs": {"enable_thinking": True}}
|
||||
assert "thinking" not in kwargs
|
||||
assert kwargs["temperature"] == 0.6
|
||||
|
||||
def test_default_caps_inject_nothing(self) -> None:
|
||||
"""Untouched compat defaults (thinking_mode=none) keep today's wire."""
|
||||
@@ -305,7 +317,9 @@ class TestCompatReasoningControl:
|
||||
)
|
||||
kwargs = self._stream_kwargs(caps, "high")
|
||||
assert "output_config" not in kwargs
|
||||
assert kwargs["extra_body"] == {"chat_template_kwargs": {"enable_thinking": True}}
|
||||
assert kwargs["extra_body"] == {
|
||||
"chat_template_kwargs": {"enable_thinking": True, "reasoning_effort": "high"}
|
||||
}
|
||||
|
||||
def test_create_completion_same_injection(self) -> None:
|
||||
"""The non-streaming path shares _build_thinking_and_kwargs."""
|
||||
|
||||
@@ -2095,13 +2095,19 @@ class TestOpenAIParameterGating:
|
||||
def setup_method(self) -> None:
|
||||
self.provider = OpenAIProvider()
|
||||
|
||||
def test_unknown_model_no_reasoning_effort(self) -> None:
|
||||
"""Unknown/local models should NOT receive top-level reasoning_effort."""
|
||||
def test_local_model_effort_forwarded_verbatim(self) -> None:
|
||||
"""Local-lane models receive the session knob verbatim on the flat
|
||||
param (effort_passthrough) — the user's effort setting always
|
||||
reaches the wire; "none" stays omitted (nothing to disable
|
||||
beyond the template toggle)."""
|
||||
caps = self.provider.get_capabilities("my-local-model")
|
||||
kwargs: dict[str, Any] = {}
|
||||
apply_temperature_and_effort(kwargs, caps, temperature=0.7, reasoning_effort="medium")
|
||||
assert "reasoning_effort" not in kwargs
|
||||
assert kwargs["reasoning_effort"] == "medium"
|
||||
assert kwargs["temperature"] == 0.7
|
||||
kwargs = {}
|
||||
apply_temperature_and_effort(kwargs, caps, temperature=0.7, reasoning_effort="none")
|
||||
assert "reasoning_effort" not in kwargs
|
||||
|
||||
def test_gpt5_no_temperature_has_reasoning_effort(self) -> None:
|
||||
"""GPT-5 base: no temperature, reasoning_effort sent."""
|
||||
|
||||
@@ -85,7 +85,10 @@ def _assert_golden(name: str, payload: dict[str, Any]) -> None:
|
||||
norm = _normalize(payload)
|
||||
if _UPDATE:
|
||||
GOLDEN_DIR.mkdir(parents=True, exist_ok=True)
|
||||
path.write_text(json.dumps(norm, indent=2, sort_keys=True) + "\n")
|
||||
# ensure_ascii=False keeps non-ASCII (em dashes in repair
|
||||
# messages) literal, matching the existing baselines — a regen
|
||||
# must not churn unrelated lines into \uXXXX escapes.
|
||||
path.write_text(json.dumps(norm, indent=2, sort_keys=True, ensure_ascii=False) + "\n")
|
||||
return
|
||||
assert path.exists(), f"missing golden {name!r}; run UPDATE_WIRE_GOLDENS=1 to baseline"
|
||||
assert norm == json.loads(path.read_text()), f"wire-payload drift for {name!r}"
|
||||
|
||||
@@ -13,6 +13,7 @@ from typing import TYPE_CHECKING, Any
|
||||
|
||||
from turnstone.core.attachments import safe_attachment_label
|
||||
from turnstone.core.providers._protocol import (
|
||||
EFFORT_TEMPLATE_FALLBACK_PARAM,
|
||||
CompletionResult,
|
||||
ModelCapabilities,
|
||||
StreamChunk,
|
||||
@@ -420,8 +421,17 @@ class AnthropicProvider:
|
||||
# schema (silently dropped) and must not have temperature forced
|
||||
# to 1.0 — reasoning control rides chat_template_kwargs instead,
|
||||
# folded into extra_params here so the extra_body forwarding
|
||||
# below carries it to the wire.
|
||||
extra_params = merge_reasoning_template_kwargs(caps, reasoning_effort, extra_params)
|
||||
# below carries it to the wire. This lane has no flat effort
|
||||
# param, so the graded value falls back to the conventional
|
||||
# template key when the operator engaged reasoning control
|
||||
# without naming one — the session effort setting always
|
||||
# reaches the wire.
|
||||
extra_params = merge_reasoning_template_kwargs(
|
||||
caps,
|
||||
reasoning_effort,
|
||||
extra_params,
|
||||
fallback_effort_param=EFFORT_TEMPLATE_FALLBACK_PARAM,
|
||||
)
|
||||
elif caps.thinking_mode == "adaptive":
|
||||
thinking_dict: dict[str, Any] = {"type": "adaptive"}
|
||||
if caps.thinking_display:
|
||||
|
||||
@@ -291,12 +291,17 @@ OPENAI_DEFAULT = ModelCapabilities()
|
||||
# collision with a cloud model id ("gpt-5.5-my-finetune", "o3-distill")
|
||||
# would inherit that model's sampling and effort contract, none of which
|
||||
# holds for the box actually serving the name. Every local model gets
|
||||
# the plain dataclass defaults — the same fallthrough unmatched names
|
||||
# always got — and anything beyond them is declared by the operator on
|
||||
# the plain dataclass defaults plus ``effort_passthrough`` — the session
|
||||
# effort knob is forwarded verbatim on the flat ``reasoning_effort``
|
||||
# param (the documented compat field; tolerant servers ignore it, vLLM
|
||||
# maps it into the template where supported) unless the operator
|
||||
# declares ``reasoning_effort_values`` (then the ordinal snap applies)
|
||||
# or an ``effort_param`` (then the template channel claims the value
|
||||
# and the flat param is suppressed). Everything else is declared on
|
||||
# the model definition (capabilities JSON + server_compat), mirroring
|
||||
# ``_ANTHROPIC_COMPAT_DEFAULT`` and ``lookup_model_capabilities``'s
|
||||
# "no static table for local models" contract.
|
||||
OPENAI_COMPAT_DEFAULT = ModelCapabilities()
|
||||
OPENAI_COMPAT_DEFAULT = ModelCapabilities(effort_passthrough=True)
|
||||
|
||||
|
||||
def lookup_openai_capabilities(model: str) -> ModelCapabilities:
|
||||
|
||||
@@ -95,6 +95,13 @@ class ModelCapabilities:
|
||||
effort_levels: tuple[str, ...] = ()
|
||||
reasoning_effort_values: tuple[str, ...] = ()
|
||||
default_reasoning_effort: str = "medium"
|
||||
# Local-lane contract: with NO declared ``reasoning_effort_values``
|
||||
# the session knob is forwarded VERBATIM instead of omitted — the
|
||||
# user's effort setting always reaches the wire, and the serving
|
||||
# box is the authority on what it means. Commercial rows leave
|
||||
# this False: there, an empty values list means the model has no
|
||||
# effort control at all (o1-mini) and the param must be omitted.
|
||||
effort_passthrough: bool = False
|
||||
supports_web_search: bool = False
|
||||
supports_tool_search: bool = False
|
||||
supports_vision: bool = False
|
||||
@@ -191,8 +198,18 @@ def resolve_reasoning_effort(caps: ModelCapabilities, reasoning_effort: str) ->
|
||||
charge of a knob that promises off. Models without a declared
|
||||
``"none"`` get the param omitted, and ``"none"`` is never a snap
|
||||
target for other knob positions.
|
||||
|
||||
With no declared values, ``caps.effort_passthrough`` (local lanes)
|
||||
forwards the knob verbatim — the user's effort setting always
|
||||
reaches the wire and the serving box decides what it means; without
|
||||
the flag an empty list means "no effort control" and the param is
|
||||
omitted (commercial rows).
|
||||
"""
|
||||
if not caps.reasoning_effort_values or not reasoning_effort:
|
||||
if not reasoning_effort:
|
||||
return None
|
||||
if not caps.reasoning_effort_values:
|
||||
if caps.effort_passthrough and reasoning_effort != "none":
|
||||
return reasoning_effort
|
||||
return None
|
||||
if reasoning_effort == "none":
|
||||
return "none" if "none" in caps.reasoning_effort_values else None
|
||||
@@ -222,21 +239,35 @@ def flat_effort_suppressed(caps: ModelCapabilities) -> bool:
|
||||
return bool(caps.effort_param)
|
||||
|
||||
|
||||
# Default chat-template key for the graded effort value on lanes whose
|
||||
# ONLY effort channel is ``chat_template_kwargs`` (anthropic-compatible —
|
||||
# vLLM's /v1/messages schema has no flat ``reasoning_effort`` param).
|
||||
# Injected when the operator engaged template reasoning control
|
||||
# (thinking_mode manual/adaptive) without naming an effort key: the
|
||||
# user's effort setting must reach the wire regardless, a template that
|
||||
# doesn't reference the kwarg ignores it, and ``caps.effort_param``
|
||||
# overrides the name for templates that grade under something else
|
||||
# (e.g. "reasoning").
|
||||
EFFORT_TEMPLATE_FALLBACK_PARAM = "reasoning_effort"
|
||||
|
||||
|
||||
def reasoning_template_kwargs(
|
||||
caps: ModelCapabilities,
|
||||
reasoning_effort: str,
|
||||
*,
|
||||
fallback_effort_param: str = "",
|
||||
) -> dict[str, Any]:
|
||||
"""``chat_template_kwargs`` entries carrying reasoning control.
|
||||
|
||||
On local model servers the reasoning levers live in the chat template:
|
||||
a boolean toggle (``caps.thinking_param``) and an optional graded
|
||||
effort key (``caps.effort_param``, gpt-oss-style templates). The
|
||||
session effort knob drives both, mirroring the native Anthropic
|
||||
contracts: ``"manual"`` maps ``"none"``/empty to an explicit
|
||||
``false`` (``_reasoning_params`` parity — the knob is the switch),
|
||||
while ``"adaptive"`` always sends ``true`` (the model self-regulates;
|
||||
the native adaptive branch never lets the knob force-disable
|
||||
thinking). The effort value is validated via
|
||||
a boolean toggle (``caps.thinking_param``) and a graded effort key
|
||||
(``caps.effort_param``, else *fallback_effort_param* on lanes with no
|
||||
flat effort channel). The session effort knob drives both, mirroring
|
||||
the native Anthropic contracts: ``"manual"`` maps ``"none"``/empty to
|
||||
an explicit ``false`` (``_reasoning_params`` parity — the knob is the
|
||||
switch), while ``"adaptive"`` always sends ``true`` (the model
|
||||
self-regulates; the native adaptive branch never lets the knob
|
||||
force-disable thinking). The effort value is validated via
|
||||
``resolve_reasoning_effort`` when the model declares
|
||||
``reasoning_effort_values`` (off-list knob values round up onto the
|
||||
declared list, capped at its ceiling); with no declared values the
|
||||
@@ -249,14 +280,21 @@ def reasoning_template_kwargs(
|
||||
updates[caps.thinking_param] = effort_on
|
||||
elif caps.thinking_mode == "adaptive":
|
||||
updates[caps.thinking_param] = True
|
||||
if caps.effort_param and effort_on:
|
||||
# A declared effort_param is an operator opt-in at any thinking_mode
|
||||
# (gpt-oss-style boxes grade without a toggle). The FALLBACK key
|
||||
# rides only when template reasoning control is engaged — a
|
||||
# thinking_mode="none" box keeps its inject-nothing contract.
|
||||
effort_param = caps.effort_param or (
|
||||
fallback_effort_param if caps.thinking_mode in ("manual", "adaptive") else ""
|
||||
)
|
||||
if effort_param and effort_on:
|
||||
effort = (
|
||||
resolve_reasoning_effort(caps, reasoning_effort)
|
||||
if caps.reasoning_effort_values
|
||||
else reasoning_effort
|
||||
)
|
||||
if effort:
|
||||
updates[caps.effort_param] = effort
|
||||
updates[effort_param] = effort
|
||||
return updates
|
||||
|
||||
|
||||
@@ -264,6 +302,8 @@ def merge_reasoning_template_kwargs(
|
||||
caps: ModelCapabilities,
|
||||
reasoning_effort: str,
|
||||
extra_params: dict[str, Any] | None,
|
||||
*,
|
||||
fallback_effort_param: str = "",
|
||||
) -> dict[str, Any] | None:
|
||||
"""Merge ``reasoning_template_kwargs`` into a copy of the extra params.
|
||||
|
||||
@@ -274,7 +314,9 @@ def merge_reasoning_template_kwargs(
|
||||
``server_compat`` pin beats the knob mapping. The caller's dict
|
||||
(and its ``chat_template_kwargs`` sub-dict) is never mutated.
|
||||
"""
|
||||
updates = reasoning_template_kwargs(caps, reasoning_effort)
|
||||
updates = reasoning_template_kwargs(
|
||||
caps, reasoning_effort, fallback_effort_param=fallback_effort_param
|
||||
)
|
||||
if not updates:
|
||||
return dict(extra_params) if extra_params else extra_params
|
||||
merged = dict(extra_params) if extra_params else {}
|
||||
|
||||
@@ -40,6 +40,7 @@ from turnstone.core.providers._anthropic import (
|
||||
_map_reasoning_to_effort,
|
||||
)
|
||||
from turnstone.core.providers._protocol import (
|
||||
EFFORT_TEMPLATE_FALLBACK_PARAM,
|
||||
ModelCapabilities,
|
||||
flat_effort_suppressed,
|
||||
reasoning_template_kwargs,
|
||||
@@ -151,12 +152,17 @@ def _effective_anthropic(caps: ModelCapabilities, knob: str) -> str:
|
||||
|
||||
def _effective_local(provider_name: str, caps: ModelCapabilities, knob: str) -> str:
|
||||
"""Chat lanes — mirrors ``merge_reasoning_template_kwargs`` (+ flat param)."""
|
||||
updates = reasoning_template_kwargs(caps, knob)
|
||||
# The anthropic-compatible request path passes the fallback effort
|
||||
# key (its only effort channel is chat_template_kwargs); the chat
|
||||
# lanes carry effort on the flat param instead.
|
||||
fallback = EFFORT_TEMPLATE_FALLBACK_PARAM if provider_name == "anthropic-compatible" else ""
|
||||
updates = reasoning_template_kwargs(caps, knob, fallback_effort_param=fallback)
|
||||
effort_param = caps.effort_param or fallback
|
||||
parts: list[str] = []
|
||||
if caps.thinking_param in updates:
|
||||
parts.append("on" if updates[caps.thinking_param] else "off")
|
||||
if caps.effort_param and caps.effort_param in updates:
|
||||
parts.append(str(updates[caps.effort_param]))
|
||||
if effort_param and effort_param in updates:
|
||||
parts.append(str(updates[effort_param]))
|
||||
# Chat-completions providers also send the flat param unless a
|
||||
# declared effort_param claims the template channel — the same
|
||||
# ``flat_effort_suppressed`` predicate ``apply_temperature_and_effort``
|
||||
|
||||
Reference in New Issue
Block a user