ewilliams28/turnstone
1
0
Fork 0
mirror of https://github.com/turnstonelabs/turnstone.git synced 2026-08-12 23:12:23 -06:00
Code Issues Packages Projects Releases Wiki Activity

fix(model-turn): one sampling-knob assignment scheme — alias > config > model definition > omit

Browse Source 
Round-2 review fixes. The round-1 de-pinning collided with
ConfigStore.get's default-on-miss semantics: the registry defaults
(temperature 1.0, effort "medium") were manufactured onto every
store-backed lane's wire, making the documented "unset -> omit"
terminal unreachable. Unset is now representable end to end, and one
scheme governs every lane: per-model alias value > operator-stored
global setting > in-code model definition (effort only: caps
declaration) > field omitted, inference engine's default rules.

- settings_registry: model.temperature default None, model.reasoning_effort
  default "" — the registered defaults ARE the unset sentinels, so the
  admin UI and the wire agree. Admin webux renders nullable floats blank
  ("(inherit model default)") and maps blank-save to reset; the "" effort
  choice reads "(inherit)".
- model_turn: resolve_temperature_setting/resolve_effort_setting are the
  ONE pair of operator-rung resolvers, shared by resolve_lane, both
  session factories, and the /model switch (the 4th-copy mirror is gone;
  the switch no longer leaks the previous model's override on store-less
  sessions). The caps rung moved out of the lane into model_turn's
  effective computation, below a new request-shaped default_reasoning_effort
  parameter (utility + output guard pass "low": budget coherence with
  their small token caps, not sampling policy — any operator or
  model-definition value beats it). The hidden "medium" terminal is gone.
- providers: Protocol + all adapters take reasoning_effort: str | None =
  None (the Protocol-signature "medium" was the same manufactured pin one
  layer down); ModelCapabilities.default_reasoning_effort defaults "" —
  commercial rows all declare theirs explicitly, so only local lanes and
  Anthropic change, both to match their real serving defaults (Anthropic
  manual-thinking models no longer get implicit thinking-on-medium).
  reasoning_template_kwargs distinguishes unset (inject nothing; template
  default rules) from the explicit "none" off-switch. apply_temperature
  skips temperature unless reasoning is EXPLICITLY off on none-declaring
  models (unset leaves the server default in charge, possibly reasoning-on).
- session: ctor takes temperature: float | None / reasoning_effort:
  str | None = None; _save_config/resume round-trip unset as "" (the
  str(None) era guarded); _run_agent relays session temperature AND
  effort on the same-alias fall-through only (a task alias's configured
  knobs stay reachable in both directions).
- optimizer: the five meta lanes are decoupled from --temperature/
  --reasoning-effort (test-model knobs, per their documented meaning);
  registry-less meta lanes omit both fields.
- cli: --temperature/--reasoning-effort default unset and fall through
  the model config instead of pinning 0.5/"medium" for every CLI session.
- cleanup from the review's below-cap findings: dead resolve_server_type
  deleted (tests re-pointed at _server_type_of), stale ChatSession
  comments in _openai_responses fixed, _store_get_or_none extracted,
  eval system-turn conversion hoisted out of the per-turn loop, dead
  _provider_extra_params patch removed, test_perception uses the shared
  mock_completion_result, effort_ladder uses apply_capability_overrides
  instead of a SimpleNamespace fake config.

Wire goldens regenerated: the only drift is the manufactured "medium"
effort vanishing from unset-effort requests (Responses reasoning.effort,
Chat/Google reasoning_effort, Anthropic output_config.effort) — pure
removals, no additions. Ladder tests now fake ConfigStore with the REAL
get() semantics (registry default on miss) so a forgiving fake can't
mask this class of bug again.
This commit is contained in:
Patrick Buckley
2026-07-13 06:25:34 -07:00
parent 09fd17f2da
commit b6391d1f90
64 changed files with 482 additions and 400 deletions
Download Patch File Download Diff File Expand all files Collapse all files
tests/data/wire_payloads/anthropic_hoist__mid_orphan.json
-3
View File
@@ -51,9 +51,6 @@
}
],
"model": "claude-sonnet-4-6",
"output_config": {
"effort": "medium"
},
"temperature": 1.0,
"thinking": {
"type": "adaptive"
tests/data/wire_payloads/anthropic_hoist__multipart.json
-3
View File
@@ -23,9 +23,6 @@
}
],
"model": "claude-sonnet-4-6",
"output_config": {
"effort": "medium"
},
"temperature": 1.0,
"thinking": {
"type": "adaptive"
tests/data/wire_payloads/anthropic_hoist__native_orphan.json
-3
View File
@@ -43,9 +43,6 @@
}
],
"model": "claude-sonnet-4-6",
"output_config": {
"effort": "medium"
},
"temperature": 1.0,
"thinking": {
"type": "adaptive"
tests/data/wire_payloads/anthropic_hoist__native_reasoning.json
-3
View File
@@ -42,9 +42,6 @@
}
],
"model": "claude-sonnet-4-6",
"output_config": {
"effort": "medium"
},
"temperature": 1.0,
"thinking": {
"type": "adaptive"
tests/data/wire_payloads/anthropic_hoist__operator_system.json
-3
View File
@@ -35,9 +35,6 @@
}
],
"model": "claude-sonnet-4-6",
"output_config": {
"effort": "medium"
},
"system": "Output-guard: deploy output looked clean.",
"temperature": 1.0,
"thinking": {
tests/data/wire_payloads/anthropic_hoist__text.json
-3
View File
@@ -23,9 +23,6 @@
}
],
"model": "claude-sonnet-4-6",
"output_config": {
"effort": "medium"
},
"temperature": 1.0,
"thinking": {
"type": "adaptive"
tests/data/wire_payloads/anthropic_hoist__toolcall_complete.json
-3
View File
@@ -42,9 +42,6 @@
}
],
"model": "claude-sonnet-4-6",
"output_config": {
"effort": "medium"
},
"temperature": 1.0,
"thinking": {
"type": "adaptive"
tests/data/wire_payloads/anthropic_hoist__trailing_orphan.json
-3
View File
@@ -34,9 +34,6 @@
}
],
"model": "claude-sonnet-4-6",
"output_config": {
"effort": "medium"
},
"temperature": 1.0,
"thinking": {
"type": "adaptive"
tests/data/wire_payloads/anthropic_native__mid_orphan.json
-3
View File
@@ -51,9 +51,6 @@
}
],
"model": "claude-opus-4-8",
"output_config": {
"effort": "medium"
},
"thinking": {
"display": "summarized",
"type": "adaptive"
tests/data/wire_payloads/anthropic_native__multipart.json
-3
View File
@@ -23,9 +23,6 @@
}
],
"model": "claude-opus-4-8",
"output_config": {
"effort": "medium"
},
"thinking": {
"display": "summarized",
"type": "adaptive"
tests/data/wire_payloads/anthropic_native__native_orphan.json
-3
View File
@@ -43,9 +43,6 @@
}
],
"model": "claude-opus-4-8",
"output_config": {
"effort": "medium"
},
"thinking": {
"display": "summarized",
"type": "adaptive"
tests/data/wire_payloads/anthropic_native__native_reasoning.json
-3
View File
@@ -42,9 +42,6 @@
}
],
"model": "claude-opus-4-8",
"output_config": {
"effort": "medium"
},
"thinking": {
"display": "summarized",
"type": "adaptive"
tests/data/wire_payloads/anthropic_native__operator_system.json
-3
View File
@@ -39,9 +39,6 @@
}
],
"model": "claude-opus-4-8",
"output_config": {
"effort": "medium"
},
"thinking": {
"display": "summarized",
"type": "adaptive"
tests/data/wire_payloads/anthropic_native__text.json
-3
View File
@@ -23,9 +23,6 @@
}
],
"model": "claude-opus-4-8",
"output_config": {
"effort": "medium"
},
"thinking": {
"display": "summarized",
"type": "adaptive"
tests/data/wire_payloads/anthropic_native__toolcall_complete.json
-3
View File
@@ -42,9 +42,6 @@
}
],
"model": "claude-opus-4-8",
"output_config": {
"effort": "medium"
},
"thinking": {
"display": "summarized",
"type": "adaptive"
tests/data/wire_payloads/anthropic_native__trailing_orphan.json
-3
View File
@@ -34,9 +34,6 @@
}
],
"model": "claude-opus-4-8",
"output_config": {
"effort": "medium"
},
"thinking": {
"display": "summarized",
"type": "adaptive"
tests/data/wire_payloads/google__mid_orphan.json
-1
View File
@@ -43,7 +43,6 @@
}
],
"model": "gemini-2.5-pro",
"reasoning_effort": "medium",
"stream": true,
"stream_options": {
"include_usage": true
tests/data/wire_payloads/google__multipart.json
-1
View File
@@ -18,7 +18,6 @@
}
],
"model": "gemini-2.5-pro",
"reasoning_effort": "medium",
"stream": true,
"stream_options": {
"include_usage": true
tests/data/wire_payloads/google__native_orphan.json
-1
View File
@@ -26,7 +26,6 @@
}
],
"model": "gemini-2.5-pro",
"reasoning_effort": "medium",
"stream": true,
"stream_options": {
"include_usage": true
tests/data/wire_payloads/google__native_reasoning.json
-1
View File
@@ -26,7 +26,6 @@
}
],
"model": "gemini-2.5-pro",
"reasoning_effort": "medium",
"stream": true,
"stream_options": {
"include_usage": true
tests/data/wire_payloads/google__operator_system.json
-1
View File
@@ -34,7 +34,6 @@
}
],
"model": "gemini-2.5-pro",
"reasoning_effort": "medium",
"stream": true,
"stream_options": {
"include_usage": true
tests/data/wire_payloads/google__text.json
-1
View File
@@ -15,7 +15,6 @@
}
],
"model": "gemini-2.5-pro",
"reasoning_effort": "medium",
"stream": true,
"stream_options": {
"include_usage": true
tests/data/wire_payloads/google__toolcall_complete.json
-1
View File
@@ -30,7 +30,6 @@
}
],
"model": "gemini-2.5-pro",
"reasoning_effort": "medium",
"stream": true,
"stream_options": {
"include_usage": true
tests/data/wire_payloads/google__trailing_orphan.json
-1
View File
@@ -26,7 +26,6 @@
}
],
"model": "gemini-2.5-pro",
"reasoning_effort": "medium",
"stream": true,
"stream_options": {
"include_usage": true
tests/data/wire_payloads/openai_chat__mid_orphan.json
-1
View File
@@ -43,7 +43,6 @@
}
],
"model": "gpt-4o-mini",
"reasoning_effort": "medium",
"stream": true,
"stream_options": {
"include_usage": true
tests/data/wire_payloads/openai_chat__multipart.json
-1
View File
@@ -18,7 +18,6 @@
}
],
"model": "gpt-4o-mini",
"reasoning_effort": "medium",
"stream": true,
"stream_options": {
"include_usage": true
tests/data/wire_payloads/openai_chat__native_orphan.json
-1
View File
@@ -26,7 +26,6 @@
}
],
"model": "gpt-4o-mini",
"reasoning_effort": "medium",
"stream": true,
"stream_options": {
"include_usage": true
tests/data/wire_payloads/openai_chat__native_reasoning.json
-1
View File
@@ -26,7 +26,6 @@
}
],
"model": "gpt-4o-mini",
"reasoning_effort": "medium",
"stream": true,
"stream_options": {
"include_usage": true
tests/data/wire_payloads/openai_chat__operator_system.json
-1
View File
@@ -34,7 +34,6 @@
}
],
"model": "gpt-4o-mini",
"reasoning_effort": "medium",
"stream": true,
"stream_options": {
"include_usage": true
tests/data/wire_payloads/openai_chat__text.json
-1
View File
@@ -15,7 +15,6 @@
}
],
"model": "gpt-4o-mini",
"reasoning_effort": "medium",
"stream": true,
"stream_options": {
"include_usage": true
tests/data/wire_payloads/openai_chat__toolcall_complete.json
-1
View File
@@ -30,7 +30,6 @@
}
],
"model": "gpt-4o-mini",
"reasoning_effort": "medium",
"stream": true,
"stream_options": {
"include_usage": true
tests/data/wire_payloads/openai_chat__trailing_orphan.json
-1
View File
@@ -26,7 +26,6 @@
}
],
"model": "gpt-4o-mini",
"reasoning_effort": "medium",
"stream": true,
"stream_options": {
"include_usage": true
tests/data/wire_payloads/openai_responses__mid_orphan.json
-3
View File
@@ -39,9 +39,6 @@
"max_output_tokens": 4096,
"model": "gpt-5",
"prompt_cache_retention": "24h",
"reasoning": {
"effort": "medium"
},
"store": false,
"stream": true,
"tools": [
tests/data/wire_payloads/openai_responses__multipart.json
-3
View File
@@ -21,9 +21,6 @@
"max_output_tokens": 4096,
"model": "gpt-5",
"prompt_cache_retention": "24h",
"reasoning": {
"effort": "medium"
},
"store": false,
"stream": true
}
tests/data/wire_payloads/openai_responses__native_orphan.json
-3
View File
@@ -28,9 +28,6 @@
"max_output_tokens": 4096,
"model": "gpt-5",
"prompt_cache_retention": "24h",
"reasoning": {
"effort": "medium"
},
"store": false,
"stream": true,
"tools": [
tests/data/wire_payloads/openai_responses__native_reasoning.json
-3
View File
@@ -28,9 +28,6 @@
"max_output_tokens": 4096,
"model": "gpt-5",
"prompt_cache_retention": "24h",
"reasoning": {
"effort": "medium"
},
"store": false,
"stream": true,
"tools": [
tests/data/wire_payloads/openai_responses__operator_system.json
-3
View File
@@ -29,9 +29,6 @@
"max_output_tokens": 4096,
"model": "gpt-5",
"prompt_cache_retention": "24h",
"reasoning": {
"effort": "medium"
},
"store": false,
"stream": true,
"tools": [
tests/data/wire_payloads/openai_responses__text.json
-3
View File
@@ -22,9 +22,6 @@
"max_output_tokens": 4096,
"model": "gpt-5",
"prompt_cache_retention": "24h",
"reasoning": {
"effort": "medium"
},
"store": false,
"stream": true
}
tests/data/wire_payloads/openai_responses__toolcall_complete.json
-3
View File
@@ -28,9 +28,6 @@
"max_output_tokens": 4096,
"model": "gpt-5",
"prompt_cache_retention": "24h",
"reasoning": {
"effort": "medium"
},
"store": false,
"stream": true,
"tools": [
tests/data/wire_payloads/openai_responses__trailing_orphan.json
-3
View File
@@ -23,9 +23,6 @@
"max_output_tokens": 4096,
"model": "gpt-5",
"prompt_cache_retention": "24h",
"reasoning": {
"effort": "medium"
},
"store": false,
"stream": true,
"tools": [
tests/test_model_registry.py
+6 -5
View File
@@ -1188,8 +1188,11 @@ class TestSessionModelCommand:
assert session.max_tokens == 2048
assert session.reasoning_effort == "high"
def test_model_switch_none_params_reverts_to_global(self) -> None:
"""Switching to a model with no overrides reverts to global defaults."""
def test_model_switch_none_params_reverts_to_unset(self) -> None:
"""Switching to a model with no overrides re-resolves the knobs for
the NEW alias: nothing configured → unset (wire omission). The old
model's override must not leak onto the new lane — pre-scheme, a
store-less session kept the stale 1.5 forever."""
reg = ModelRegistry(
models={
"hot": ModelConfig("hot", "x", "x", "hot-model", temperature=1.5),
@@ -1199,10 +1202,8 @@ class TestSessionModelCommand:
)
session = _make_session(registry=reg, model_alias="hot")
session.temperature = 1.5 # as set by per-model override
# Without a config_store, fallback keeps current value (CLI sessions).
# With a config_store, it would revert to the global default.
session.handle_command("/model plain")
assert session.temperature == 1.5 # no config_store → keeps current
assert session.temperature is None
def test_model_switch_unknown_alias(self) -> None:
reg = ModelRegistry(
tests/test_model_turn.py
+94 -15
View File
@@ -69,6 +69,28 @@ def _lane(provider: _FakeProvider, **kw: Any) -> ModelLane:
return ModelLane(provider=provider, client=object(), model="m", **kw)
def _real_semantics_store(**stored: Any) -> SimpleNamespace:
"""A ConfigStore fake with the REAL ``get()`` semantics.
A stored key returns its value; a never-stored key returns the
SETTINGS registry default — which for the sampling keys IS the unset
sentinel (``None`` / ``""``). The old fakes returned ``None`` on any
miss, which masked the default-on-miss collision the round-2 review
caught: never fake a store rung more forgiving than the real one.
"""
from turnstone.core.settings_registry import SETTINGS
def _get(key: str, default: Any = ...) -> Any:
if key in stored:
return stored[key]
if default is not ...:
return default
defn = SETTINGS.get(key)
return defn.default if defn else None
return SimpleNamespace(get=_get)
def test_model_turn_lowers_turns_and_threads_lane_config() -> None:
caps = ModelCapabilities(max_output_tokens=1234)
extra = {"chat_template_kwargs": {"enable_thinking": True}}
@@ -388,11 +410,12 @@ def test_temperature_unresolved_passes_none_and_wire_omits_it() -> None:
def test_resolve_lane_global_config_store_rung() -> None:
# ModelConfig.temperature=None means "use the global default from
# ConfigStore" (the documented ladder) — resolve_lane climbs it.
# The global rung fires only when the operator actually STORED a
# value; the registry default is the unset sentinel (None), so an
# untouched install resolves None → the wire omits the field.
provider = _FakeProvider([])
registry = _fake_registry(temperature=None)
store = SimpleNamespace(get=lambda key: 1.0 if key == "model.temperature" else None)
store = _real_semantics_store(**{"model.temperature": 1.0})
lane = resolve_lane(provider, object(), "m", alias="ali", registry=registry, config_store=store)
assert lane.temperature == 1.0
# The per-model value wins over the global rung.
@@ -401,34 +424,90 @@ def test_resolve_lane_global_config_store_rung() -> None:
provider, object(), "m", alias="ali", registry=registry2, config_store=store
)
assert lane2.temperature == 0.3
# Never-stored global → None (the round-2 headline: ConfigStore.get
# must NOT manufacture a wire value on a miss).
lane3 = resolve_lane(
provider,
object(),
"m",
alias="ali",
registry=_fake_registry(temperature=None),
config_store=_real_semantics_store(),
)
assert lane3.temperature is None
def test_resolve_lane_reasoning_effort_ladder() -> None:
# Effort rides the same ladder as temperature with a caps terminal:
# per-model config -> global setting -> caps default_reasoning_effort.
def test_resolve_lane_reasoning_effort_operator_rungs() -> None:
# The lane carries the OPERATOR rungs only: per-model config → stored
# global setting → None. The in-code model definition (caps default)
# applies at the model_turn call, so an operator-silent lane stays
# None — the assignment scheme's "if not set, we don't send it".
provider = _FakeProvider([])
# Per-model config wins.
reg = _fake_registry()
reg.get_config.return_value.reasoning_effort = "high"
lane = resolve_lane(provider, object(), "m", alias="ali", registry=reg)
assert lane.reasoning_effort == "high"
# Global setting rung.
# Stored global setting rung.
reg2 = _fake_registry()
reg2.get_config.return_value.reasoning_effort = None
store = SimpleNamespace(get=lambda key: "low" if key == "model.reasoning_effort" else None)
store = _real_semantics_store(**{"model.reasoning_effort": "low"})
lane2 = resolve_lane(provider, object(), "m", alias="ali", registry=reg2, config_store=store)
assert lane2.reasoning_effort == "low"
# Terminal: the lane capabilities' per-provider default.
lane3 = resolve_lane(provider, object(), "m")
assert lane3.reasoning_effort == ModelCapabilities().default_reasoning_effort
# Empty string at any rung is the unset sentinel (a valid settings
# choice meaning fall through) — with a never-stored global (registry
# default IS "") the lane stays operator-silent.
reg3 = _fake_registry()
reg3.get_config.return_value.reasoning_effort = ""
lane3 = resolve_lane(
provider, object(), "m", alias="ali", registry=reg3, config_store=_real_semantics_store()
)
assert lane3.reasoning_effort is None
# Bare lane (no registry, no store): None.
assert resolve_lane(provider, object(), "m").reasoning_effort is None
def test_model_turn_effort_unresolved_falls_to_caps_default() -> None:
# No caller value, bare lane -> the caps default reaches the provider
# (effort always resolves concrete; wire omission is temperature-only).
def test_model_turn_effort_lower_rungs() -> None:
# Below the lane's operator rungs, model_turn applies: caller
# request-shaped default → in-code model definition (caps) → None
# (wire omission — no hidden "medium" constant anywhere).
# Bare hand-built lane: nothing anywhere → the provider receives None.
provider = _FakeProvider([CompletionResult(content="")])
model_turn(_lane(provider), [Turn.user("x")])
assert provider.calls[0]["reasoning_effort"] == "medium"
assert provider.calls[0]["reasoning_effort"] is None
# In-code model definition rung: a declared caps default applies.
caps = ModelCapabilities(default_reasoning_effort="high")
provider2 = _FakeProvider([CompletionResult(content="")])
model_turn(_lane(provider2, capabilities=caps), [Turn.user("x")])
assert provider2.calls[0]["reasoning_effort"] == "high"
# A caller request-shaped default (budget-coupled lanes: utility,
# output guard) beats the model-generic caps default…
provider3 = _FakeProvider([CompletionResult(content="")])
model_turn(
_lane(provider3, capabilities=caps), [Turn.user("x")], default_reasoning_effort="low"
)
assert provider3.calls[0]["reasoning_effort"] == "low"
# …loses to an operator value on the lane…
provider4 = _FakeProvider([CompletionResult(content="")])
model_turn(
_lane(provider4, capabilities=caps, reasoning_effort="xhigh"),
[Turn.user("x")],
default_reasoning_effort="low",
)
assert provider4.calls[0]["reasoning_effort"] == "xhigh"
# …and to an explicit relay (the "none" knob stays distinct from unset).
provider5 = _FakeProvider([CompletionResult(content="")])
model_turn(
_lane(provider5, capabilities=caps),
[Turn.user("x")],
reasoning_effort="none",
default_reasoning_effort="low",
)
assert provider5.calls[0]["reasoning_effort"] == "none"
def test_model_turn_fetches_config_once_per_call() -> None:
tests/test_perception.py
+6 -10
View File
@@ -2,11 +2,11 @@
from __future__ import annotations
from types import SimpleNamespace
from typing import TYPE_CHECKING, Any
import pytest
from tests._session_helpers import mock_completion_result
from turnstone.core import perception
if TYPE_CHECKING:
@@ -44,20 +44,16 @@ class _StubProvider:
messages: list[dict[str, Any]],
resolve_attachments: Any = None,
**_: Any,
) -> SimpleNamespace:
) -> Any:
self.calls += 1
self.last_messages = messages
self.last_resolve = resolve_attachments
if self.calls <= self._fail_times:
raise RuntimeError("backend down")
return SimpleNamespace(
content=self._content,
tool_calls=None,
finish_reason="stop",
usage=None,
provider_blocks=[],
reasoning="",
)
# Shared field inventory: when model_turn's re-ingest reads a new
# CompletionResult field, mock_completion_result is the ONE
# definition to extend and this suite moves with it.
return mock_completion_result(self._content)
@pytest.fixture(autouse=True)
tests/test_provider_anthropic_compat.py
+17 -6
View File
@@ -151,7 +151,9 @@ class TestCompatWireShape:
must never surface as wire ``extra_body`` — a leaked key would change
every real-Anthropic request that threads a thinking override.
Negative-tested: fails when the ``_INTERNAL_EXTRA_PARAMS`` exclusion
is removed from ``_build_thinking_and_kwargs``.
is removed from ``_build_thinking_and_kwargs``. The effort knob is
explicit: unset effort means thinking OFF (the budget override
modifies a thinking block, it never creates one).
"""
provider = AnthropicProvider()
client = _capture_client()
@@ -160,6 +162,7 @@ class TestCompatWireShape:
client=client,
model="claude-sonnet-4-5",
messages=[{"role": "user", "content": "hi"}],
reasoning_effort="medium",
extra_params={"thinking_budget_tokens": 2048},
)
)
@@ -228,14 +231,22 @@ class TestCompatReasoningControl:
assert "thinking" not in kwargs
assert kwargs["temperature"] == 0.6 # never forced to 1.0 on compat
@pytest.mark.parametrize("knob", ["none", ""])
def test_manual_toggle_off(self, knob: str) -> None:
"""Effort "none"/empty disables thinking — native manual-mode parity;
no effort key rides when thinking is off."""
kwargs = self._stream_kwargs(self._MANUAL_CAPS, knob)
def test_manual_toggle_explicit_off(self) -> None:
"""The explicit "none" knob disables thinking — native manual-mode
parity; no effort key rides when thinking is off."""
kwargs = self._stream_kwargs(self._MANUAL_CAPS, "none")
assert kwargs["extra_body"] == {"chat_template_kwargs": {"enable_thinking": False}}
assert "thinking" not in kwargs
def test_manual_unset_injects_nothing(self) -> None:
"""An UNSET knob (no rung of the assignment scheme resolved a
value) injects no toggle at all — the template's own default
rules, matching "if not set, we don't send it". Distinct from
the explicit "none" off-switch above."""
kwargs = self._stream_kwargs(self._MANUAL_CAPS, "")
assert "extra_body" not in kwargs
assert "thinking" not in kwargs
def test_adaptive_always_on(self) -> None:
"""Adaptive never knob-disables — native-adaptive contract, no native
dict; the graded value rides for on-positions only."""
tests/test_session.py
+3 -1
View File
@@ -268,8 +268,10 @@ class TestChatSessionConstruction:
assert session.reasoning_effort == "high"
def test_default_reasoning_effort(self, tmp_db):
# Unset by default: no rung of the assignment scheme spoke, so the
# wire omits the effort param (no hidden "medium" constructor pin).
session = _make_session()
assert session.reasoning_effort == "medium"
assert session.reasoning_effort is None
# ---------------------------------------------------------------------------
tests/test_session_replay_reasoning.py
+4 -4
View File
@@ -634,10 +634,10 @@ class TestUtilityCompletionPassesFlag:
mock_provider.create_completion = capture_completion
session._provider = mock_provider
caps = ModelCapabilities(max_output_tokens=0, supports_reasoning_replay=True)
with (
patch.object(session, "_get_capabilities", return_value=caps),
patch.object(session, "_provider_extra_params", return_value=None),
):
# No _provider_extra_params patch: _utility_completion resolves
# extra_params inside resolve_lane (module seam), which a
# session-attribute patch cannot intercept.
with patch.object(session, "_get_capabilities", return_value=caps):
session._utility_completion(
[Turn.user("summarize")],
max_tokens=512,
tests/test_session_synth_reasoning_block.py
+18 -33
View File
@@ -30,7 +30,7 @@ from types import SimpleNamespace
from typing import Any
from tests._session_helpers import make_session as _make_session
from turnstone.core.model_turn import resolve_server_type, synth_reasoning_block
from turnstone.core.model_turn import _server_type_of, synth_reasoning_block
from turnstone.core.providers._anthropic import (
ANTHROPIC_VALID_BLOCK_TYPES,
AnthropicProvider,
@@ -78,7 +78,7 @@ class TestSynthReasoningBlock:
assert out[0]["source"] == "vllm"
def test_synth_handles_registry_exception(self) -> None:
# resolve_server_type silently returns "" on any lookup error
# The defensive config fetch degrades to None on any lookup error
# — synth still fires but omits the source field.
class BrokenRegistry:
def get_config(self, alias: str) -> Any:
@@ -299,47 +299,32 @@ class TestStreamResponseSynthBlockIntegration:
assert provider_content[0]["source"] == "vllm"
class TestResolveServerType:
"""Direct unit tests for ``model_turn.resolve_server_type``, the one
reader of ``server_compat.server_type``."""
class TestServerTypeOf:
"""Direct unit tests for ``model_turn._server_type_of``, the one
reader of ``server_compat.server_type`` (the Phase 5 vLLM gate and
the synth-block source tagging both go through it)."""
def test_returns_empty_when_no_registry(self) -> None:
assert resolve_server_type(None, "some-alias") == ""
def test_returns_empty_when_no_alias(self) -> None:
registry = SimpleNamespace(
get_config=lambda alias: SimpleNamespace(capabilities={}, server_compat={})
)
assert resolve_server_type(registry, "") == ""
def test_returns_server_type_when_present(self) -> None:
def test_reads_server_type_when_present(self) -> None:
# Mirrors production ModelConfig shape: server_compat lives at
# the top-level dataclass field, NOT inside capabilities. Both
# model_registry loader paths pop("server_compat") out of caps
# before construction (see model_registry.py:401, 485).
registry = SimpleNamespace(
get_config=lambda alias: SimpleNamespace(
capabilities={},
server_compat={"server_type": "llama.cpp"},
)
cfg = SimpleNamespace(
capabilities={},
server_compat={"server_type": "llama.cpp"},
)
assert resolve_server_type(registry, "local-model") == "llama.cpp"
assert _server_type_of(cfg) == "llama.cpp"
def test_returns_empty_when_server_compat_missing(self) -> None:
registry = SimpleNamespace(
get_config=lambda alias: SimpleNamespace(
capabilities={"context_window": 32768},
server_compat={},
)
cfg = SimpleNamespace(
capabilities={"context_window": 32768},
server_compat={},
)
assert resolve_server_type(registry, "local-model") == ""
assert _server_type_of(cfg) == ""
def test_returns_empty_on_exception(self) -> None:
class BrokenRegistry:
def get_config(self, alias: str) -> Any:
raise RuntimeError("boom")
assert resolve_server_type(BrokenRegistry(), "x") == ""
def test_returns_empty_on_non_dict_server_compat(self) -> None:
assert _server_type_of(SimpleNamespace(server_compat=None)) == ""
assert _server_type_of(SimpleNamespace()) == ""
class TestFinalizeProviderBlocks:
turnstone/cli.py
+28 -8
View File
@@ -301,7 +301,7 @@ class TerminalUI(SessionUI):
total_tok = usage["prompt_tokens"] + usage["completion_tokens"]
pct = total_tok / context_window * 100 if context_window > 0 else 0
parts = [f"{total_tok:,} / {context_window:,} tokens ({pct:.0f}%)"]
if effort != "medium":
if effort and effort != "medium":
parts.append(f"reasoning: {effort}")
sys.stdout.write(f"\n {DIM}[{' · '.join(parts)}]{RESET}\n")
sys.stdout.flush()
@@ -998,8 +998,11 @@ def main() -> None:
parser.add_argument(
"--temperature",
type=float,
default=0.5,
help="Sampling temperature (default: 0.5)",
default=None,
help=(
"Sampling temperature (default: the model's configured value, "
"else the field is omitted and the serving default applies)"
),
)
parser.add_argument(
"--max-tokens",
@@ -1015,9 +1018,12 @@ def main() -> None:
)
parser.add_argument(
"--reasoning-effort",
default="medium",
choices=["none", "minimal", "low", "medium", "high", "xhigh", "max"],
help="Reasoning effort level (default: medium)",
default="",
choices=["", "none", "minimal", "low", "medium", "high", "xhigh", "max"],
help=(
"Reasoning effort level (default: the model's configured or "
"declared value, else the serving default)"
),
)
parser.add_argument(
"--provider",
@@ -1258,16 +1264,30 @@ def main() -> None:
) -> ChatSession:
assert ui is not None, "session_factory requires a non-None UI"
del project_id
from turnstone.core.model_turn import (
resolve_effort_setting,
resolve_temperature_setting,
)
r_client, r_model, r_cfg = registry.resolve(model_alias)
# An explicit CLI flag is the user speaking; otherwise the knobs
# ride the shared assignment scheme (the CLI has no ConfigStore,
# so the rungs are the model config, then unset = wire omission).
eff_temperature = (
args.temperature
if args.temperature is not None
else resolve_temperature_setting(r_cfg, None)
)
eff_effort = args.reasoning_effort or resolve_effort_setting(r_cfg, None)
return ChatSession(
client=r_client,
model=r_model,
ui=ui,
instructions=args.instructions,
temperature=args.temperature,
temperature=eff_temperature,
max_tokens=args.max_tokens,
tool_timeout=args.tool_timeout,
reasoning_effort=args.reasoning_effort,
reasoning_effort=eff_effort,
context_window=r_cfg.context_window,
compact_max_tokens=args.compact_max_tokens,
auto_compact_pct=args.auto_compact_pct,
turnstone/console/session_factory.py
+7 -14
View File
@@ -23,6 +23,7 @@ from typing import TYPE_CHECKING
from turnstone.console.coordinator_alias import resolve_coordinator_alias
from turnstone.core.log import get_logger
from turnstone.core.model_turn import resolve_effort_setting, resolve_temperature_setting
from turnstone.core.session import ChatSession
from turnstone.core.workstream import WorkstreamKind
from turnstone.prompts import ClientType
@@ -168,25 +169,17 @@ def build_console_session_factory(
e,
)
eff_temperature = (
r_cfg.temperature
if r_cfg.temperature is not None
else config_store.get("model.temperature")
)
# Sampling knobs ride the shared assignment scheme (alias >
# stored config > unset); the coordinator role setting slots in
# as a role rung. Unset means the wire omits the field.
eff_temperature = resolve_temperature_setting(r_cfg, config_store)
eff_max_tokens = (
r_cfg.max_tokens
if r_cfg.max_tokens is not None
else config_store.get("model.max_tokens")
)
# Coordinator has its own effort setting; fall back to model-level
# override, then global default.
eff_reasoning_effort = (
r_cfg.reasoning_effort
if r_cfg.reasoning_effort is not None
else (
config_store.get("coordinator.reasoning_effort")
or config_store.get("model.reasoning_effort")
)
eff_reasoning_effort = resolve_effort_setting(
r_cfg, config_store, role_key="coordinator.reasoning_effort"
)
coord_client = coord_client_factory(ws_id or "", uid)
turnstone/console/static/admin.js
+23 -1
View File
@@ -22,7 +22,11 @@ const ALIAS_SETTING_KEYS = [
// opposed to "no value" — distinct from the literal "none" choice (e.g.
// reasoning_effort="none" actually disables reasoning, very different
// from leaving it unset).
const INHERIT_EMPTY_LABEL_KEYS = ["model.task_effort"];
const INHERIT_EMPTY_LABEL_KEYS = [
"model.task_effort",
"model.reasoning_effort",
"coordinator.reasoning_effort",
];
// ---------------------------------------------------------------------------
// Admin information architecture — the single source of truth for the rail's
@@ -4274,6 +4278,13 @@ function _renderSettingRow(item) {
item.max_value !== null && item.max_value !== undefined
? ' max="' + item.max_value + '"'
: "";
// A null registry default means "unset = inherit" (e.g.
// model.temperature): blank is a saveable state, not a validation
// error — the save handler maps it to reset-to-default.
const nullableAttr =
item.default_value === null
? ' data-nullable="1" placeholder="(inherit model default)"'
: "";
html +=
'<input type="number" data-setting-key="' +
escapedKey +
@@ -4286,6 +4297,7 @@ function _renderSettingRow(item) {
'"' +
minAttr +
maxAttr +
nullableAttr +
">";
} else {
// str
@@ -4448,6 +4460,16 @@ function _saveSettingValue(key) {
value = inp.checked;
} else if (inp.type === "number") {
if (inp.value === "") {
if (inp.getAttribute("data-nullable") === "1") {
// Blank on a nullable-default setting means "inherit": clear any
// stored override (reset), or nothing to do if already default.
if (document.querySelector('[data-reset-key="' + key + '"]')) {
_resetSetting(key);
} else if (saveBtn) {
saveBtn.classList.remove("visible");
}
return;
}
showToast("Value is required");
return;
}
turnstone/core/model_turn.py
+123 -73
View File
@@ -121,12 +121,24 @@ def resolve_capabilities(
if cfg is ...:
cfg = _get_config_or_none(registry, alias)
if cfg is not None:
overrides_raw = getattr(cfg, "capabilities", None)
if isinstance(overrides_raw, dict) and overrides_raw:
names = {f.name for f in fields(type(caps))}
overrides = {k: v for k, v in overrides_raw.items() if k in names}
if overrides:
caps = replace(caps, **overrides)
caps = apply_capability_overrides(caps, getattr(cfg, "capabilities", None))
return caps
def apply_capability_overrides(caps: ModelCapabilities, overrides_raw: Any) -> ModelCapabilities:
"""Field-filtered merge of an operator ``capabilities`` dict onto *caps*.
The ONE merge shared by the request path (:func:`resolve_capabilities`)
and the admin-UI effort-ladder projection — callers holding a raw
overrides dict use this directly instead of faking a ModelConfig.
Unknown keys are ignored (the registry accepts free-form dicts) and a
non-dict value degrades to "no overrides".
"""
if isinstance(overrides_raw, dict) and overrides_raw:
names = {f.name for f in fields(type(caps))}
overrides = {k: v for k, v in overrides_raw.items() if k in names}
if overrides:
caps = replace(caps, **overrides)
return caps
@@ -161,9 +173,10 @@ def _server_type_of(cfg: Any) -> str:
Reads ``cfg.server_compat`` (the dedicated dataclass field hoisted by
both model_registry loader paths) — NOT ``cfg.capabilities``. The ONE
reader of the field path: :func:`resolve_server_type` and the Phase 5
gate in :func:`maybe_attach_vllm_chat_reasoning` both go through here,
so a loader shape change cannot desync them.
reader of the field path: the Phase 5 gate in
:func:`maybe_attach_vllm_chat_reasoning` and the synth-block source
tagging in :func:`synth_reasoning_block` both go through here, so a
loader shape change cannot desync them.
"""
sc = getattr(cfg, "server_compat", None)
if isinstance(sc, dict):
@@ -171,14 +184,65 @@ def _server_type_of(cfg: Any) -> str:
return ""
def resolve_server_type(registry: ModelRegistry | None, alias: str) -> str:
"""``server_compat.server_type`` for an alias (``""`` on any miss).
def _store_get_or_none(config_store: Any, key: str) -> Any | None:
"""Best-effort ConfigStore read for a lane rung: value or ``None``.
Best-effort lookup — synth-block source tagging is informational,
never load-bearing.
A broken store degrades the rung to unset — settings lookups must
never crash lane resolution. The registered defaults for the
sampling keys ARE the unset sentinels (``None`` / ``""``), so a
never-stored key falls through the ladder instead of manufacturing
a wire value (``ConfigStore.get`` returns the SettingDef default on
a miss, never ``None`` for a registered key — the admin UI shows
that same default, so sentinel defaults keep the UI and the wire in
agreement).
"""
cfg = _get_config_or_none(registry, alias)
return _server_type_of(cfg) if cfg is not None else ""
try:
return config_store.get(key)
except Exception:
log.debug("config_store %s lookup failed", key, exc_info=True)
return None
def resolve_temperature_setting(cfg: Any | None, config_store: Any | None) -> float | None:
"""The operator rungs of the temperature assignment scheme.
``ModelConfig.temperature`` (the alias's per-model value; ``0.0`` is
a valid explicit choice) → stored global ``model.temperature`` →
``None``. ``None`` means no operator spoke: the field is omitted
from the wire and the inference engine's own default applies. Code
never supplies a number here — the ONE resolver shared by
:func:`resolve_lane`, the session factories, and the ``/model``
switch, so every surface samples identically on the same alias.
"""
temperature = getattr(cfg, "temperature", None) if cfg is not None else None
if temperature is None and config_store is not None:
temperature = _store_get_or_none(config_store, "model.temperature")
return temperature
def resolve_effort_setting(
cfg: Any | None,
config_store: Any | None,
*,
role_key: str = "",
) -> str | None:
"""The operator rungs of the reasoning-effort assignment scheme.
``ModelConfig.reasoning_effort`` → *role_key* setting (a role-scoped
override such as ``coordinator.reasoning_effort``) → stored global
``model.reasoning_effort`` → ``None``. Empty string at any rung
means "unset" (it is a valid settings choice meaning fall through),
distinct from the explicit ``"none"`` effort value. The in-code
model-definition rung (``caps.default_reasoning_effort``) applies at
the call site (:func:`model_turn`), NOT here — a lane's resolved
effort is operator intent only.
"""
effort = getattr(cfg, "reasoning_effort", None) if cfg is not None else None
if not effort and config_store is not None and role_key:
effort = _store_get_or_none(config_store, role_key)
if not effort and config_store is not None:
effort = _store_get_or_none(config_store, "model.reasoning_effort")
return effort or None
def resolve_replay_reasoning_to_model(
@@ -270,19 +334,18 @@ class ModelLane:
the lane runs outside the registry (then every registry-backed pass
degrades to its documented miss behavior).
*temperature* is the lane's inherited sampling temperature — the ladder
``ModelConfig.temperature`` → global ``model.temperature`` setting →
``None`` (field omitted from the wire; server default). House rule:
code never pins a temperature; callers pass an explicit value only when
relaying an operator/user-resolved knob (the session's own value on the
session-model lanes). Sub-agent, judge, and single-shot lanes inherit
their OWN model's ladder.
*reasoning_effort* rides the same ladder with a different terminal:
``ModelConfig.reasoning_effort`` → global ``model.reasoning_effort``
setting → the lane capabilities' ``default_reasoning_effort``. Effort
always resolves to a concrete value (it gates thinking modes, so wire
omission is not the unset semantics the way it is for temperature).
*temperature* / *reasoning_effort* are the lane's OPERATOR-resolved
sampling knobs — the assignment scheme's operator rungs only
(per-model alias value → stored global setting → ``None``; see
:func:`resolve_temperature_setting` / :func:`resolve_effort_setting`).
``None`` means no operator spoke. For temperature that is terminal:
the field is omitted from the wire and the inference engine's own
default applies. For effort, :func:`model_turn` applies two more
rungs below the lane — a caller-supplied request-shaped default,
then the in-code model definition (``caps.default_reasoning_effort``)
— before omitting. House rule: code never pins either knob; callers
pass an explicit value only when relaying an operator/user-resolved
knob (the session's own value on the session-model lanes).
"""
provider: LLMProvider
@@ -315,14 +378,16 @@ def resolve_lane(
pass. ``...`` (the sentinel default) means "resolve for me" —
``None`` is a valid resolved value for *extra_params*.
The lane temperature climbs the documented ladder
(``ModelConfig.temperature`` docstring: "None = use global default
from ConfigStore"): the alias's per-model value, else the operator's
global ``model.temperature`` when *config_store* is supplied, else
``None`` — which the providers translate to omitting the field so the
SERVER default applies. This mirrors the session_factory / ``/model``
switch resolution so a judge or single-shot lane samples exactly like
the main loop on the same model.
The lane sampling knobs resolve through the shared operator rungs
(:func:`resolve_temperature_setting` / :func:`resolve_effort_setting`
— the SAME resolvers the session factories and the ``/model`` switch
use, so a judge or single-shot lane samples exactly like the main
loop on the same model): the alias's per-model value, else the
operator's stored global setting when *config_store* is supplied,
else ``None``. The registered defaults for both settings are the
unset sentinels, so an untouched install resolves ``None`` — the
providers then OMIT the field and the inference engine's own default
applies.
All resolved facets read ONE defensively-fetched ModelConfig, so a
registry hot-reload mid-resolution cannot mix config generations, and
@@ -336,28 +401,6 @@ def resolve_lane(
if extra_params is ...
else extra_params
)
temperature = getattr(cfg, "temperature", None) if cfg is not None else None
if temperature is None and config_store is not None:
try:
temperature = config_store.get("model.temperature")
except Exception:
# Best-effort global rung — a broken store degrades to the
# server default, never crashes lane resolution.
log.debug("config_store model.temperature lookup failed", exc_info=True)
temperature = None
# Effort ladder: per-model config → global setting → the lane caps'
# per-provider default. Empty string at any rung means "unset" (it is
# a valid settings choice meaning fall through), distinct from the
# explicit "none" effort value.
effort = getattr(cfg, "reasoning_effort", None) if cfg is not None else None
if not effort and config_store is not None:
try:
effort = config_store.get("model.reasoning_effort") or None
except Exception:
log.debug("config_store model.reasoning_effort lookup failed", exc_info=True)
effort = None
if not effort:
effort = caps.default_reasoning_effort if caps is not None else None
return ModelLane(
provider=provider,
client=client,
@@ -366,8 +409,8 @@ def resolve_lane(
capabilities=caps,
extra_params=extra,
registry=registry,
temperature=temperature,
reasoning_effort=effort or None,
temperature=resolve_temperature_setting(cfg, config_store),
reasoning_effort=resolve_effort_setting(cfg, config_store),
)
@@ -572,6 +615,7 @@ def model_turn(
max_tokens: int = 4096,
temperature: float | None = None,
reasoning_effort: str | None = None,
default_reasoning_effort: str | None = None,
mint: Callable[[str], str] | None = None,
wire_id_map: dict[str, str] | None = None,
resolve_attachments: Callable[[list[str]], dict[str, Any]] | None = None,
@@ -589,9 +633,14 @@ def model_turn(
so a caller's retry loop just calls again.
*temperature* / *reasoning_effort* ``None`` (the defaults) inherit the
lane's resolved ladder (per-model config → global setting → server
default for temperature / caps default for effort). House rule: never
pin either in code; pass an explicit value only to relay an
lane's operator-resolved knobs (per-model config → stored global
setting). When no operator spoke, temperature is OMITTED from the
wire (the inference engine's default rules); effort falls through
*default_reasoning_effort* — a request-shaped default for lanes whose
token budget constrains thinking (title gen, the output guard), which
any operator value beats — then the in-code model definition
(``caps.default_reasoning_effort``), then omission. House rule: never
pin either knob in code; pass an explicit value only to relay an
operator- or user-resolved knob (the session's own knobs, a CLI flag).
*resolve_attachments* materializes by-reference ``AttachmentRef``
@@ -631,18 +680,19 @@ def model_turn(
wire_id_map if wire_id_map is not None else {},
)
wire = maybe_attach_vllm_chat_reasoning(wire, lane.provider, lane.registry, lane.alias, cfg=cfg)
# Effort always resolves to a concrete value (ladder terminal = caps
# default); the trailing fallback covers a hand-built lane that skipped
# resolve_lane. Temperature may stay None — the providers then OMIT
# the field so the server default applies (house rule: a Python-level
# constant anywhere on this path is a hidden pin). Direct keyword call
# (not **kwargs) so strict mypy checks the module's most important
# invocation against the Protocol.
# The effort assignment scheme's lower rungs: explicit relay → lane
# (operator) → caller's request-shaped default → in-code model
# definition → None. None/unset knobs are OMITTED from the wire so
# the inference engine's default rules (house rule: a Python-level
# constant anywhere on this path is a hidden pin). Direct keyword
# call (not **kwargs) so strict mypy checks the module's most
# important invocation against the Protocol.
effective_effort = (
reasoning_effort
if reasoning_effort is not None
else lane.reasoning_effort
or (lane.capabilities.default_reasoning_effort if lane.capabilities else "medium")
or lane.reasoning_effort
or default_reasoning_effort
or (lane.capabilities.default_reasoning_effort if lane.capabilities else None)
or None
)
result = lane.provider.create_completion(
client=lane.client,
turnstone/core/output_guard_judge.py
+7 -1
View File
@@ -540,7 +540,12 @@ class OutputGuardJudge:
config_store=self._config_store,
)
# Temperature deliberately not pinned (house rule) — the lane
# inherits the guard model's configured temperature.
# inherits the guard model's configured temperature. The effort
# default is request shape, not a pin: screening runs inside a
# 512-token cap, and an unconstrained thinking pass would consume
# the whole budget and return empty content (the stage would
# silently no-op to heuristic-only). Any operator or
# model-definition effort value beats it.
try:
result = run_with_deadline(
lambda: model_turn(
@@ -548,6 +553,7 @@ class OutputGuardJudge:
judge_turns,
tools=None,
max_tokens=512,
default_reasoning_effort="low",
),
timeout=timeout,
cancel_event=cancel_event,
turnstone/core/providers/_anthropic.py
+5 -5
View File
@@ -275,7 +275,7 @@ _ANTHROPIC_CAPABILITIES: dict[str, ModelCapabilities] = {
def _map_reasoning_to_effort(
reasoning_effort: str,
reasoning_effort: str | None,
valid_levels: tuple[str, ...],
) -> str | None:
"""Map turnstone reasoning_effort to Anthropic effort parameter.
@@ -404,7 +404,7 @@ class AnthropicProvider:
def _build_thinking_and_kwargs(
self,
caps: ModelCapabilities,
reasoning_effort: str,
reasoning_effort: str | None,
extra_params: dict[str, Any] | None,
max_tokens: int,
temperature: float | None,
@@ -828,7 +828,7 @@ class AnthropicProvider:
def _reasoning_params(
self,
reasoning_effort: str,
reasoning_effort: str | None,
extra_params: dict[str, Any] | None,
max_tokens: int = 4096,
) -> dict[str, Any]:
@@ -865,7 +865,7 @@ class AnthropicProvider:
tools: list[dict[str, Any]] | None = None,
max_tokens: int = 4096,
temperature: float | None = None,
reasoning_effort: str = "medium",
reasoning_effort: str | None = None,
extra_params: dict[str, Any] | None = None,
deferred_names: frozenset[str] | None = None,
cancel_ref: list[Any] | None = None,
@@ -1079,7 +1079,7 @@ class AnthropicProvider:
tools: list[dict[str, Any]] | None = None,
max_tokens: int = 4096,
temperature: float | None = None,
reasoning_effort: str = "medium",
reasoning_effort: str | None = None,
extra_params: dict[str, Any] | None = None,
deferred_names: frozenset[str] | None = None,
capabilities: ModelCapabilities | None = None,
turnstone/core/providers/_openai_chat.py
+3 -3
View File
@@ -145,7 +145,7 @@ class OpenAIChatCompletionsProvider:
self,
extra_params: dict[str, Any] | None,
caps: ModelCapabilities,
reasoning_effort: str,
reasoning_effort: str | None,
) -> dict[str, Any] | None:
"""Build the final ``extra_body``, injecting reasoning params if needed.
@@ -178,7 +178,7 @@ class OpenAIChatCompletionsProvider:
tools: list[dict[str, Any]] | None = None,
max_tokens: int = 4096,
temperature: float | None = None,
reasoning_effort: str = "medium",
reasoning_effort: str | None = None,
extra_params: dict[str, Any] | None = None,
deferred_names: frozenset[str] | None = None,
cancel_ref: list[Any] | None = None,
@@ -313,7 +313,7 @@ class OpenAIChatCompletionsProvider:
tools: list[dict[str, Any]] | None = None,
max_tokens: int = 4096,
temperature: float | None = None,
reasoning_effort: str = "medium",
reasoning_effort: str | None = None,
extra_params: dict[str, Any] | None = None,
deferred_names: frozenset[str] | None = None,
capabilities: ModelCapabilities | None = None,
turnstone/core/providers/_openai_common.py
+8 -5
View File
@@ -362,7 +362,7 @@ def apply_temperature(
kwargs: dict[str, Any],
caps: ModelCapabilities,
temperature: float | None,
reasoning_effort: str,
reasoning_effort: str | None,
) -> None:
"""Conditionally add temperature to *kwargs*.
@@ -373,14 +373,17 @@ def apply_temperature(
- Models with ``supports_temperature=False`` (GPT-5 base, O-series)
never receive temperature.
- Models that list ``"none"`` in their effort values (GPT-5.1/5.2)
only receive temperature when reasoning is inactive.
only receive temperature when reasoning is EXPLICITLY off. An
unset effort (``None``/empty) leaves the server default in charge,
which may have reasoning active (gpt-5.5/5.6 default medium) — so
unset skips temperature too.
"""
if temperature is None:
return
if not caps.supports_temperature:
return
if "none" in caps.reasoning_effort_values and reasoning_effort not in ("none", ""):
return # Skip temperature when reasoning is active
if "none" in caps.reasoning_effort_values and reasoning_effort != "none":
return # Skip temperature unless reasoning is explicitly off
kwargs["temperature"] = temperature
@@ -388,7 +391,7 @@ def apply_temperature_and_effort(
kwargs: dict[str, Any],
caps: ModelCapabilities,
temperature: float | None,
reasoning_effort: str,
reasoning_effort: str | None,
) -> None:
"""Conditionally add temperature and reasoning_effort to *kwargs*.
turnstone/core/providers/_openai_responses.py
+10 -9
View File
@@ -355,7 +355,7 @@ class OpenAIResponsesProvider:
tools: list[dict[str, Any]] | None,
max_tokens: int,
temperature: float | None,
reasoning_effort: str,
reasoning_effort: str | None,
deferred_names: frozenset[str] | None,
capabilities: ModelCapabilities | None = None,
replay_reasoning_to_model: bool = True,
@@ -372,10 +372,10 @@ class OpenAIResponsesProvider:
turns (the SDK's ``ResponseReasoningItemParam`` shape).
The AND-gate against ``caps.supports_reasoning_replay`` lives
upstream in ``ChatSession._resolve_replay_reasoning_to_model``
(single source of truth across providers). Production callers
always thread the session-resolved flag, so this method trusts
the bool it receives.
upstream in ``model_turn.resolve_replay_reasoning_to_model``
(single source of truth across providers; the session wrapper
delegates there). Production callers always thread the resolved
flag, so this method trusts the bool it receives.
"""
caps = capabilities or self.get_capabilities(model)
@@ -481,7 +481,7 @@ class OpenAIResponsesProvider:
tools: list[dict[str, Any]] | None = None,
max_tokens: int = 4096,
temperature: float | None = None,
reasoning_effort: str = "medium",
reasoning_effort: str | None = None,
extra_params: dict[str, Any] | None = None,
deferred_names: frozenset[str] | None = None,
cancel_ref: list[Any] | None = None,
@@ -491,8 +491,9 @@ class OpenAIResponsesProvider:
# AND ``_convert_messages`` round-tripping stored reasoning
# items as input. The AND-gate against
# ``caps.supports_reasoning_replay`` lives in
# ``ChatSession._resolve_replay_reasoning_to_model`` — single
# source of truth across providers.
# ``model_turn.resolve_replay_reasoning_to_model`` — single
# source of truth across providers (the session wrapper
# delegates there).
replay_reasoning_to_model: bool = True,
extra_headers: dict[str, str] | None = None,
resolve_attachments: Callable[[list[str]], dict[str, Any]] | None = None,
@@ -677,7 +678,7 @@ class OpenAIResponsesProvider:
tools: list[dict[str, Any]] | None = None,
max_tokens: int = 4096,
temperature: float | None = None,
reasoning_effort: str = "medium",
reasoning_effort: str | None = None,
extra_params: dict[str, Any] | None = None,
deferred_names: frozenset[str] | None = None,
capabilities: ModelCapabilities | None = None,
turnstone/core/providers/_protocol.py
+32 -19
View File
@@ -100,7 +100,13 @@ class ModelCapabilities:
supports_effort: bool = False
effort_levels: tuple[str, ...] = ()
reasoning_effort_values: tuple[str, ...] = ()
default_reasoning_effort: str = "medium"
# The model definition's own default effort — the in-code rung of the
# assignment scheme (alias > stored config > this > omit). Empty =
# the definition declares no default: the effort param is omitted and
# the serving side's own default rules. Commercial rows declare
# their documented defaults explicitly; local lanes stay silent so an
# unconfigured box keeps its template/server behavior.
default_reasoning_effort: str = ""
# Local-lane contract: with NO declared ``reasoning_effort_values``
# the session knob is forwarded VERBATIM instead of omitted — the
# user's effort setting always reaches the wire, and the serving
@@ -203,9 +209,12 @@ def snap_reasoning_effort(reasoning_effort: str, declared: tuple[str, ...]) -> s
return min(at_or_above)[1] if at_or_above else max(rankable)[1]
def resolve_reasoning_effort(caps: ModelCapabilities, reasoning_effort: str) -> str | None:
def resolve_reasoning_effort(caps: ModelCapabilities, reasoning_effort: str | None) -> str | None:
"""Return the validated reasoning effort value, or ``None`` to omit.
``None``/empty input means no rung of the assignment scheme resolved
a value: the param is omitted and the serving side's default rules.
Declared values match verbatim; off-list knob values round UP onto
the declared list, capped at its ceiling (``snap_reasoning_effort``).
``default_reasoning_effort`` is the last resort for values the
@@ -273,7 +282,7 @@ EFFORT_TEMPLATE_FALLBACK_PARAM = "reasoning_effort"
def reasoning_template_kwargs(
caps: ModelCapabilities,
reasoning_effort: str,
reasoning_effort: str | None,
*,
fallback_effort_param: str = "",
) -> dict[str, Any]:
@@ -282,22 +291,26 @@ def reasoning_template_kwargs(
On local model servers the reasoning levers live in the chat template:
a boolean toggle (``caps.thinking_param``) and a graded effort key
(``caps.effort_param``, else *fallback_effort_param* on lanes with no
flat effort channel). The session effort knob drives both, mirroring
the native Anthropic contracts: ``"manual"`` maps ``"none"``/empty to
an explicit ``false`` (``_reasoning_params`` parity — the knob is the
switch), while ``"adaptive"`` always sends ``true`` (the model
self-regulates; the native adaptive branch never lets the knob
force-disable thinking). The effort value is validated via
``resolve_reasoning_effort`` when the model declares
``reasoning_effort_values`` (off-list knob values round up onto the
declared list, capped at its ceiling); with no declared values the
knob is forwarded as-is and the template is the authority on
validity.
flat effort channel). The effort knob drives both, mirroring the
native Anthropic contracts: ``"manual"`` maps a concrete level to an
explicit ``true`` and the explicit ``"none"`` knob to an explicit
``false`` (``_reasoning_params`` parity — the knob is the switch),
while an UNSET knob (``None``/empty: no rung of the assignment scheme
resolved a value) injects nothing — the template's own default rules.
``"adaptive"`` always sends ``true`` (the model self-regulates; the
native adaptive branch never lets the knob force-disable thinking).
The effort value is validated via ``resolve_reasoning_effort`` when
the model declares ``reasoning_effort_values`` (off-list knob values
round up onto the declared list, capped at its ceiling); with no
declared values the knob is forwarded as-is and the template is the
authority on validity.
"""
updates: dict[str, Any] = {}
effort_on = bool(reasoning_effort) and reasoning_effort != "none"
explicit_off = reasoning_effort == "none"
effort_on = bool(reasoning_effort) and not explicit_off
if caps.thinking_mode == "manual":
updates[caps.thinking_param] = effort_on
if effort_on or explicit_off:
updates[caps.thinking_param] = effort_on
elif caps.thinking_mode == "adaptive":
updates[caps.thinking_param] = True
# A declared effort_param is an operator opt-in at any thinking_mode
@@ -320,7 +333,7 @@ def reasoning_template_kwargs(
def merge_reasoning_template_kwargs(
caps: ModelCapabilities,
reasoning_effort: str,
reasoning_effort: str | None,
extra_params: dict[str, Any] | None,
*,
fallback_effort_param: str = "",
@@ -422,7 +435,7 @@ class LLMProvider(Protocol):
tools: list[dict[str, Any]] | None = None,
max_tokens: int = 4096,
temperature: float | None = None,
reasoning_effort: str = "medium",
reasoning_effort: str | None = None,
extra_params: dict[str, Any] | None = None,
deferred_names: frozenset[str] | None = None,
cancel_ref: list[Any] | None = None,
@@ -470,7 +483,7 @@ class LLMProvider(Protocol):
tools: list[dict[str, Any]] | None = None,
max_tokens: int = 4096,
temperature: float | None = None,
reasoning_effort: str = "medium",
reasoning_effort: str | None = None,
extra_params: dict[str, Any] | None = None,
deferred_names: frozenset[str] | None = None,
capabilities: ModelCapabilities | None = None,
turnstone/core/providers/_xai.py
+1 -1
View File
@@ -165,7 +165,7 @@ class XAIProvider(OpenAIResponsesProvider):
tools: list[dict[str, Any]] | None,
max_tokens: int,
temperature: float | None,
reasoning_effort: str,
reasoning_effort: str | None,
deferred_names: frozenset[str] | None,
capabilities: ModelCapabilities | None = None,
replay_reasoning_to_model: bool = True,
turnstone/core/providers/effort_ladder.py
+3 -7
View File
@@ -95,21 +95,17 @@ def effort_ladder_for_model(
"""Ladder for a stored model row: provider defaults + operator overrides.
Delegates the override merge to
:func:`turnstone.core.model_turn.resolve_capabilities` — the ONE
:func:`turnstone.core.model_turn.apply_capability_overrides` — the ONE
field-filtered merge every lane's requests use — so the admin-UI
effort projection can never drift from the wire. Callers holding a
raw DB row must parse the ``capabilities`` JSON string first
(``model_registry`` does the same) — this function takes a dict.
"""
from types import SimpleNamespace
from turnstone.core.model_turn import resolve_capabilities
from turnstone.core.model_turn import apply_capability_overrides
from turnstone.core.providers import create_provider
provider = create_provider(provider_name, api_surface=api_surface or None)
caps = resolve_capabilities(
provider, model, "", None, cfg=SimpleNamespace(capabilities=capability_overrides or {})
)
caps = apply_capability_overrides(provider.get_capabilities(model), capability_overrides or {})
return effort_ladder(provider_name, caps, api_surface)
turnstone/core/session.py
+45 -31
View File
@@ -131,8 +131,10 @@ from turnstone.core.model_turn import (
model_turn,
provider_extra_params,
resolve_capabilities,
resolve_effort_setting,
resolve_lane,
resolve_replay_reasoning_to_model,
resolve_temperature_setting,
)
from turnstone.core.nudge_queue import (
QUIET_CHANNEL,
@@ -1291,10 +1293,10 @@ class ChatSession:
model: str,
ui: SessionUI,
instructions: str | None,
temperature: float,
temperature: float | None,
max_tokens: int,
tool_timeout: int,
reasoning_effort: str = "medium",
reasoning_effort: str | None = None,
context_window: int = 32768,
compact_max_tokens: int = 32768,
auto_compact_pct: float = DEFAULT_AUTO_COMPACT_PCT,
@@ -2129,8 +2131,10 @@ class ChatSession:
config = {
"model": self.model,
"model_alias": self._model_alias or "",
"temperature": str(self.temperature),
"reasoning_effort": self.reasoning_effort,
# Unset knobs persist as "" — None means "no operator spoke"
# (wire omission) and must survive the resume round-trip.
"temperature": "" if self.temperature is None else str(self.temperature),
"reasoning_effort": self.reasoning_effort or "",
"max_tokens": str(self.max_tokens),
"instructions": self.instructions or "",
"skill": self._skill_name or "",
@@ -3503,9 +3507,12 @@ class ChatSession:
self.model,
)
if "temperature" in config:
self.temperature = float(config["temperature"])
# "" = unset (wire omission); "None" guards rows written
# by the brief str(None) era of _save_config.
raw_temp = config["temperature"]
self.temperature = float(raw_temp) if raw_temp not in (None, "", "None") else None
if "reasoning_effort" in config:
self.reasoning_effort = config["reasoning_effort"]
self.reasoning_effort = config["reasoning_effort"] or None
if "max_tokens" in config:
self.max_tokens = int(config["max_tokens"])
if "instructions" in config:
@@ -4771,10 +4778,14 @@ class ChatSession:
— the same operator/registry-resolved value the main turn uses — rather
than a hard-coded constant: utility calls should not silently override an
explicit ``[models.*]`` temperature. ``reasoning_effort`` ``None``
inherits the lane's ladder (per-model config → global setting → caps
default) — callers relaying the session's user-facing effort knob
(web-fetch extraction) pass it explicitly. extra_params resolve
inside the lane from the same single config fetch as the rest.
inherits the lane's operator rungs, then the request-shaped
``default_reasoning_effort="low"``: these calls run inside small
token budgets, and an unconstrained thinking pass can consume the
whole budget and return empty content (the #676 signature) — any
operator or model-definition value still beats the default.
Callers relaying the session's user-facing effort knob (web-fetch
extraction) pass it explicitly. extra_params resolve inside the
lane from the same single config fetch as the rest.
"""
caps = self._get_capabilities()
clamped = min(max_tokens, caps.max_output_tokens) if caps.max_output_tokens else max_tokens
@@ -4793,6 +4804,7 @@ class ChatSession:
max_tokens=clamped,
temperature=self.temperature if temperature is None else temperature,
reasoning_effort=reasoning_effort,
default_reasoning_effort="low",
)
# Utility completions (title gen, compaction, web-fetch extraction)
# bypass the streaming on_status path — record their usage so the
@@ -7085,7 +7097,9 @@ class ChatSession:
if not self._last_usage:
return
usage: dict[str, Any] = {**self._last_usage, "model": self.model}
self.ui.on_status(usage, self.context_window, self.reasoning_effort)
# "" = no effort resolved anywhere (the wire omitted the param);
# the UI protocol keeps a plain str.
self.ui.on_status(usage, self.context_window, self.reasoning_effort or "")
# -- Conversation compaction ------------------------------------------------
@@ -15077,15 +15091,17 @@ class ChatSession:
# operator flags (replay-reasoning, Phase 5 vLLM attach) re-resolve
# inside ``model_turn`` through the carried registry, so mid-session
# admin toggles keep applying exactly as they did pre-extraction.
# Temperature follows the AGENT model's own ladder (its ModelConfig,
# else the global ``model.temperature``) — relaying the session
# model's value here would make a task alias's configured
# temperature unreachable (house rule: the model's configuration is
# the source of truth).
# Session-knob relay is SAME-LANE ONLY: on the fall-through (agent
# runs the session's own model) the workstream/user-resolved session
# temperature and effort apply to sub-agent calls exactly as they do
# to the main loop; on a distinct task alias neither relays — a
# relay there would make the task alias's own configured knobs
# unreachable (the model's configuration is the source of truth).
# Agent trajectories stay excluded from the persistence/replay
# contract — history is in-memory, rebuilt per ``_run_agent``
# invocation; the native lane carried here serves the WITHIN-RUN
# reasoning continuity of the agent's own tool loop.
same_lane = (agent_alias or "") == (self._model_alias or "")
lane = resolve_lane(
agent_provider,
agent_client,
@@ -15116,7 +15132,9 @@ class ChatSession:
turns,
tools=_tools,
max_tokens=self.max_tokens,
reasoning_effort=reasoning_effort or self.reasoning_effort,
temperature=self.temperature if same_lane else None,
reasoning_effort=reasoning_effort
or (self.reasoning_effort if same_lane else None),
mint=mint,
wire_id_map=wire_id_map,
)
@@ -17023,25 +17041,20 @@ class ChatSession:
self.context_window = cfg.context_window
if not self._manual_tool_truncation:
self.tool_truncation = int(cfg.context_window * self._chars_per_token * 0.5)
# Apply per-model sampling overrides, falling back to global
# defaults — mirrors session_factory() resolution logic so
# switching away from a model with overrides doesn't leak them.
# Re-resolve the sampling knobs for the new alias through the
# SAME shared resolvers session_factory uses, so switching
# away from a model with overrides doesn't leak them and
# every surface samples identically on the same alias.
# Unset resolves to None (wire omission), replacing any prior
# model's value.
cs = self._config_store
self.temperature = (
cfg.temperature
if cfg.temperature is not None
else (cs.get("model.temperature") if cs else self.temperature)
)
self.temperature = resolve_temperature_setting(cfg, cs)
self.max_tokens = (
cfg.max_tokens
if cfg.max_tokens is not None
else (cs.get("model.max_tokens") if cs else self.max_tokens)
)
self.reasoning_effort = (
cfg.reasoning_effort
if cfg.reasoning_effort is not None
else (cs.get("model.reasoning_effort") if cs else self.reasoning_effort)
)
self.reasoning_effort = resolve_effort_setting(cfg, cs)
self._init_system_messages()
self._save_config()
self.ui.on_info(f"Switched to {cyan(arg)}: {model_name}")
@@ -17060,7 +17073,8 @@ class ChatSession:
valid = ("low", "medium", "high")
aliases = {"med": "medium", "lo": "low", "hi": "high"}
if not arg:
self.ui.on_info(f"Reasoning effort: {cyan(self.reasoning_effort)}")
shown = self.reasoning_effort or "model default"
self.ui.on_info(f"Reasoning effort: {cyan(shown)}")
else:
value = aliases.get(arg.lower(), arg.lower())
if value in valid:
turnstone/core/settings_registry.py
+16 -11
View File
@@ -55,15 +55,17 @@ def _build_registry() -> dict[str, SettingDef]:
SettingDef(
"model.temperature",
"float",
1.0,
"Default sampling temperature (overridden by per-model settings)",
None,
"Global sampling temperature (empty = inherit each model's own default)",
"model",
min_value=0.0,
max_value=2.0,
help="Default sampling temperature for models without a per-model override. "
"Controls randomness in responses. Lower values (0.0\u20130.3) give focused, "
"deterministic output; higher values (0.7\u20131.5) make responses more creative "
"and varied. Per-model overrides can be set in the Models tab.",
help="Global sampling temperature for models without a per-model override. "
"When empty (the default), the request omits the field entirely and the "
"model's own serving default applies \u2014 recommended for modern models, "
"which ship tuned sampling defaults and often reject explicit values. "
"Set a number only to force one temperature everywhere; per-model "
"overrides can be set in the Models tab.",
reference_url="https://arxiv.org/abs/1904.09751",
),
SettingDef(
@@ -80,14 +82,17 @@ def _build_registry() -> dict[str, SettingDef]:
SettingDef(
"model.reasoning_effort",
"str",
"medium",
"Default reasoning effort (overridden by per-model settings)",
"",
"Global reasoning effort (empty = inherit each model's own default)",
"model",
choices=["", "none", "minimal", "low", "medium", "high", "xhigh", "max"],
help="Default reasoning effort for models without a per-model override. "
help="Global reasoning effort for models without a per-model override. "
"Controls how much internal \u2018thinking\u2019 the model does before responding. "
"Higher effort improves quality on complex tasks but is slower and uses more "
"tokens. Per-model overrides can be set in the Models tab.",
"When empty (the default), each model's own declared or serving-side "
"default applies. \u2018none\u2019 explicitly disables reasoning where the model "
"supports that; higher effort improves quality on complex tasks but is "
"slower and uses more tokens. Per-model overrides can be set in the "
"Models tab.",
),
SettingDef(
"model.task_alias",
turnstone/eval/core.py
+6 -3
View File
@@ -314,6 +314,11 @@ class HeadlessSession(ChatSession):
registry=self._registry,
capabilities=self._get_capabilities(),
)
# System prompts live as wire dicts on the session; bridge them to
# Turn IR once — they are invariant for the run (only __init__ /
# set_skill recompose them, both before send_headless). Only the
# growing ``self.messages`` concatenation happens per turn.
system_turns = turns_from_dicts(self.system_messages)
for turn in range(max_turns):
if self._cancelled.is_set():
@@ -323,9 +328,7 @@ class HeadlessSession(ChatSession):
_log(f"{log_prefix} turn {turn}: calling API...", dim=True)
t0 = time.monotonic()
# System prompts live as wire dicts on the session; bridge them to
# Turn IR so the whole trajectory lowers through the shared seam.
turns = turns_from_dicts(self.system_messages) + self.messages
turns = system_turns + self.messages
if self._cancelled.is_set():
break
turnstone/optimizer.py
+11 -36
View File
@@ -232,8 +232,6 @@ def _diversify_prompts(
cases: list[dict[str, Any]],
n_variants: int,
provider: LLMProvider | None = None,
temperature: float | None = None,
reasoning_effort: str | None = None,
) -> dict[str, list[str]]:
"""Generate paraphrased prompt variants for each test case.
@@ -241,8 +239,10 @@ def _diversify_prompts(
user_prompt is always included as the first variant.
"""
prov = provider or create_provider("openai")
# Temperature is not pinned (house rule) — sampling diversity for the
# paraphraser belongs in the diversifier model's own configuration.
# Sampling is not pinned and NOT coupled to run_optimization's
# --temperature/--reasoning-effort (those knobs belong to the model
# under test): this registry-less lane omits both fields and the
# diversifier model's serving defaults rule.
lane = resolve_lane(prov, client, model)
result: dict[str, list[str]] = {}
@@ -302,8 +302,6 @@ def _diversify_prompts(
lane,
[Turn.system(DIVERSIFIER_SYSTEM), Turn.user(user_content)],
max_tokens=8192,
temperature=temperature,
reasoning_effort=reasoning_effort,
)
raw = (cr.content or "").strip()
# Strip reasoning tags
@@ -365,8 +363,6 @@ def _observe_and_update_optimizer(
optimizer_system: str,
iterations: list[dict[str, Any]],
provider: LLMProvider | None = None,
temperature: float | None = None,
reasoning_effort: str | None = None,
) -> str:
"""Analyze optimizer behavior and return a modified OPTIMIZER_SYSTEM."""
parts: list[str] = []
@@ -456,8 +452,6 @@ def _observe_and_update_optimizer(
resolve_lane(prov, client, model),
[Turn.system(OBSERVER_SYSTEM), Turn.user(user_content)],
max_tokens=8192,
temperature=temperature,
reasoning_effort=reasoning_effort,
)
result = cr.content or optimizer_system
@@ -666,8 +660,6 @@ def _run_analyst(
test_cases: list[dict[str, Any]],
iteration_result: dict[str, Any],
provider: LLMProvider | None = None,
temperature: float | None = None,
reasoning_effort: str | None = None,
optimize_tools: bool = False,
tool_overrides: dict[str, dict[str, Any]] | None = None,
) -> str:
@@ -771,8 +763,8 @@ def _run_analyst(
]
# Multi-turn loop: let the analyst call tools up to 5 rounds.
# Temperature is not pinned (house rule) — the analyst model's own
# configuration governs sampling.
# Sampling is not pinned and not coupled to the test model's knobs —
# this registry-less lane omits both fields (serving defaults rule).
lane = resolve_lane(prov, client, model)
max_turns = 5
for _turn in range(max_turns):
@@ -781,8 +773,6 @@ def _run_analyst(
turns,
tools=_ANALYST_TOOLS,
max_tokens=8192,
temperature=temperature,
reasoning_effort=reasoning_effort,
)
# Same degenerate-repetition cap as before (shared guard — see
@@ -865,8 +855,6 @@ def _propose_tool_overrides(
iteration_result: dict[str, Any],
analyst_output: str,
provider: LLMProvider | None = None,
temperature: float | None = None,
reasoning_effort: str | None = None,
) -> dict[str, dict[str, Any]]:
"""Propose tool description overrides based on failure analysis.
@@ -922,8 +910,6 @@ def _propose_tool_overrides(
resolve_lane(prov, client, model),
[Turn.system(TOOL_OPTIMIZER_SYSTEM), Turn.user(user_content)],
max_tokens=8192,
temperature=temperature,
reasoning_effort=reasoning_effort,
)
raw = (cr.content or "").strip()
@@ -985,8 +971,6 @@ def _propose_prompt_modification(
history: list[dict[str, Any]],
optimizer_system: str = OPTIMIZER_SYSTEM,
provider: LLMProvider | None = None,
temperature: float | None = None,
reasoning_effort: str | None = None,
parent_scores: dict[str, float] | None = None,
analyst_output: str = "",
tool_overrides: dict[str, dict[str, Any]] | None = None,
@@ -1067,8 +1051,6 @@ def _propose_prompt_modification(
resolve_lane(prov, client, model),
[Turn.system(optimizer_system), Turn.user(user_content)],
max_tokens=16384,
temperature=temperature,
reasoning_effort=reasoning_effort,
)
new_prompt = cr.content or current_prompt
@@ -1425,8 +1407,6 @@ def run_optimization(
cases=cases,
n_variants=diversify,
provider=div_provider,
temperature=temperature,
reasoning_effort=reasoning_effort,
)
results["meta"]["diversify"] = diversify
results["meta"]["prompt_variants"] = prompt_variants
@@ -1560,8 +1540,6 @@ def run_optimization(
current_optimizer_system,
results["iterations"],
provider=obs_provider,
temperature=temperature,
reasoning_effort=reasoning_effort,
)
if new_opt != current_optimizer_system:
opt_diff = _simple_diff(current_optimizer_system, new_opt)
@@ -1601,8 +1579,6 @@ def run_optimization(
test_cases=opt_cases,
iteration_result=opt_result,
provider=ana_provider,
temperature=temperature,
reasoning_effort=reasoning_effort,
optimize_tools=optimize_tools,
tool_overrides=selected.tool_overrides or None,
)
@@ -1629,8 +1605,6 @@ def run_optimization(
iteration_result=opt_result,
analyst_output=analyst_output,
provider=tool_opt_provider,
temperature=temperature,
reasoning_effort=reasoning_effort,
)
if new_tool_overrides != selected.tool_overrides:
added_tools = set(new_tool_overrides) - set(selected.tool_overrides)
@@ -1678,8 +1652,6 @@ def run_optimization(
history=results["iterations"],
optimizer_system=current_optimizer_system,
provider=opt_provider,
temperature=temperature,
reasoning_effort=reasoning_effort,
parent_scores=parent_scores,
analyst_output=analyst_output,
tool_overrides=new_tool_overrides or None,
@@ -1897,7 +1869,9 @@ def main() -> None:
"--temperature",
type=float,
default=0.7,
help="Sampling temperature (default: 0.7)",
help="Sampling temperature for the model under test (default: 0.7; "
"meta lanes — diversifier/observer/analyst/optimizers — inherit "
"their own model's serving defaults)",
)
parser.add_argument(
"--max-tokens",
@@ -1909,7 +1883,8 @@ def main() -> None:
"--reasoning-effort",
default="medium",
choices=["low", "medium", "high"],
help="Reasoning effort (default: medium)",
help="Reasoning effort for the model under test (default: medium; "
"meta lanes inherit their own model's defaults)",
)
parser.add_argument(
"--context-window",
turnstone/server.py
+6 -11
View File
@@ -58,6 +58,7 @@ from turnstone.core.auth import (
from turnstone.core.idle_nudge_watcher import wake_workstream_if_pending
from turnstone.core.log import get_logger
from turnstone.core.metrics import metrics as _metrics
from turnstone.core.model_turn import resolve_effort_setting, resolve_temperature_setting
from turnstone.core.ratelimit import resolve_client_ip
from turnstone.core.session import ChatSession, GenerationCancelled, SessionUI # noqa: F401
from turnstone.core.session_manager import SessionManager
@@ -5021,22 +5022,16 @@ def main() -> None:
except Exception as e:
log.warning("Failed to resolve judge_model %r: %s", judge_model, e)
# Per-model sampling overrides take priority over global defaults
eff_temperature = (
r_cfg.temperature
if r_cfg.temperature is not None
else config_store.get("model.temperature")
)
# Sampling knobs ride the shared assignment scheme (alias > stored
# config > unset); unset means the wire omits the field and the
# inference engine's own default rules.
eff_temperature = resolve_temperature_setting(r_cfg, config_store)
eff_max_tokens = (
r_cfg.max_tokens
if r_cfg.max_tokens is not None
else config_store.get("model.max_tokens")
)
eff_reasoning_effort = (
r_cfg.reasoning_effort
if r_cfg.reasoning_effort is not None
else config_store.get("model.reasoning_effort")
)
eff_reasoning_effort = resolve_effort_setting(r_cfg, config_store)
return ChatSession(
client=r_client,