feat: per-model sampling parameters (temperature, max_tokens, reasoni… (#350)

* feat: per-model sampling parameters (temperature, max_tokens, reasoning_effort)

Model sampling parameters were global-only settings applied uniformly to
all models. Different models have fundamentally different requirements
(o-series needs no temperature, Anthropic needs temp=1.0 with thinking,
local models may need different max_tokens). This adds per-model overrides
with global fallback so each model definition can specify its own defaults.

Migration 036 adds nullable temperature, max_tokens, reasoning_effort
columns to model_definitions. NULL inherits the global default from
ConfigStore. The session factory and /model switch command both resolve
per-model override → global fallback consistently.

The admin UI model create/edit modal now has dedicated form fields for
these parameters with client-side validation, a visual section divider,
and per-model override hints in the model table rows.

Removes vestigial model.name and model.context_window global settings
(now handled per-model by the model registry) with startup warnings for
existing config.toml users.

* fix: defensive parsing for config.toml per-model sampling params

Wrap temperature/max_tokens conversions in try/except with range
validation. Invalid values log a warning and fall back to None
(inherit global default) instead of aborting registry load.
This commit is contained in:
Patrick Buckley
2026-04-13 17:14:58 -07:00
committed by GitHub
parent a793d009fd
commit 934cb075d6
24 changed files with 591 additions and 55 deletions
+30 -3
View File
@@ -696,6 +696,26 @@ agent_model = "claude"
Each `[models.*]` entry produces a `ModelConfig` with a `provider` field
(default: `"openai"`). Supported values: `"openai"`, `"anthropic"`, `"google"`,
and `"openai-compatible"`.
**Per-model sampling overrides:** Each model can specify `temperature`,
`max_tokens`, and `reasoning_effort` to override the global defaults from
ConfigStore. When unset (`NULL`), the global default is used.
```toml
[models.local]
base_url = "http://localhost:8000/v1"
model = "qwen3-32b"
temperature = 0.7
max_tokens = 8192
[models.o3]
base_url = "https://api.openai.com/v1"
api_key = "sk-..."
model = "o3"
reasoning_effort = "high"
# temperature omitted — uses global default
```
An optional `[models.*.capabilities]` sub-table overrides per-model
`ModelCapabilities` flags (useful for local models whose capabilities
cannot be detected programmatically):
@@ -709,9 +729,15 @@ model = "qwen-3.5-vl"
supports_vision = true
```
**Database model definitions:** On server entry points, models can also be
defined in the `model_definitions` table (admin Models tab). DB models support
the same per-model sampling overrides. Config.toml models override DB models
with the same alias in-memory (the DB rows are never modified).
**Lifecycle:**
1. `load_model_registry()` reads `[models.*]` sections from config.toml and
builds a `"default"` entry from CLI `--base-url`/`--model`/`--api-key` args
1. `load_model_registry()` loads DB model definitions (if storage available),
then overlays `[models.*]` from config.toml, then builds a `"default"` entry
from CLI `--base-url`/`--model`/`--api-key` args
2. The registry is passed to the session factory closure in both `cli.py` and
`server.py`; each workstream resolves its model on creation
3. `ModelRegistry.get_client()` lazily creates SDK client instances via
@@ -720,7 +746,8 @@ supports_vision = true
4. `ModelRegistry.get_provider()` lazily creates `LLMProvider` instances via
`create_provider()` (also cached and thread-safe)
5. `/model` command shows available models; `/model <alias>` switches the
active workstream's client, model, and context window
active workstream's client, model, context window, and per-model sampling
parameters
6. `_create_stream_with_retry()` tries the primary model, then each fallback
alias in order if the primary is unreachable
7. `_run_agent()` resolves `registry.agent_model` (if set) for plan/task
+4 -1
View File
@@ -295,7 +295,7 @@ class "ModelRegistry" as ModelReg {
--
Thread-safe lazy client + provider
creation. Loaded by load_model_registry()
from CLI args + [models.*] config.
from DB + [models.*] config + CLI args.
--
core/model_registry.py
}
@@ -306,6 +306,9 @@ class "ModelConfig" as ModelCfg <<frozen>> {
+ base_url: str
+ model: str
+ context_window: int
+ temperature: float | None
+ max_tokens: int | None
+ reasoning_effort: str | None
}
' Circuit breaker state
+2 -2
View File
@@ -1,3 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:6471e611beebf647f3a191eb16588571a404cc52a43067883a2b6f06dd936376
size 594676
oid sha256:474b900448ec04d1117b48a2b55614524721b2f04ac4bda66170bd0a06aae0f2
size 624573
+28 -3
View File
@@ -36,6 +36,31 @@ users to the admin Settings API.
---
## Per-Model Sampling Overrides
The global `model.temperature`, `model.max_tokens`, and `model.reasoning_effort`
settings serve as cluster-wide defaults. Individual models can override these
via per-model settings in the `model_definitions` table (admin Models tab).
Resolution order for sampling parameters:
| Priority | Source |
|----------|--------|
| 1 (highest) | Per-model override (set in Models tab) |
| 2 | Global default (set in Settings tab) |
| 3 | Registry default (code) |
When a per-model override is `NULL` (empty in the UI), the global default is
used. Switching models via `/model <alias>` re-resolves sampling parameters
from the new model's overrides or global defaults.
**Removed settings:** `model.name` and `model.context_window` have been removed
from ConfigStore. Model names and context windows are now configured per-model
in the Models tab. A startup warning is logged if these keys appear in
`config.toml`.
---
## Bootstrap vs ConfigStore
**Bootstrap settings** are required before storage is available (database
@@ -49,12 +74,12 @@ connection, Redis, auth secrets, server bind address). These stay in
| Auth | `[auth]` | config.toml / env |
| Console bind | `[console]` | config.toml / env |
**ConfigStore settings** (51 settings) are loaded from the database after
storage initialization:
**ConfigStore settings** are loaded from the database after storage
initialization:
| Section | Settings |
|---------|----------|
| `model` | name, temperature, max_tokens, reasoning_effort, context_window |
| `model` | default_alias, temperature, max_tokens, reasoning_effort |
| `session` | instructions, retention_days, compact_max_tokens, auto_compact_pct |
| `tools` | timeout, truncation, agent_max_turns, skip_permissions, search, search_threshold, search_max_results |
| `server` | workstream_idle_timeout, max_workstreams |
+5 -5
View File
@@ -87,8 +87,8 @@ class TestSetGetRoundTrip:
assert store.get("tools.skip_permissions") is False
def test_str(self, store):
store.set("model.name", "gpt-5")
assert store.get("model.name") == "gpt-5"
store.set("model.default_alias", "gpt5-prod")
assert store.get("model.default_alias") == "gpt5-prod"
# ---------------------------------------------------------------------------
@@ -165,10 +165,10 @@ class TestStoredKeys:
assert store.stored_keys() == frozenset()
store.set("tools.timeout", 30)
assert store.stored_keys() == frozenset({"tools.timeout"})
store.set("model.name", "gpt-5")
assert store.stored_keys() == frozenset({"tools.timeout", "model.name"})
store.set("model.default_alias", "gpt5-prod")
assert store.stored_keys() == frozenset({"tools.timeout", "model.default_alias"})
store.delete("tools.timeout")
assert store.stored_keys() == frozenset({"model.name"})
assert store.stored_keys() == frozenset({"model.default_alias"})
# ---------------------------------------------------------------------------
+51
View File
@@ -161,3 +161,54 @@ class TestModelDefinitionStorage:
assert m["capabilities"] == "{}"
assert m["enabled"] is True
assert m["created_by"] == ""
# Per-model sampling params default to None (use global default)
assert m["temperature"] is None
assert m["max_tokens"] is None
assert m["reasoning_effort"] is None
def test_create_with_sampling_params(self, db: SQLiteBackend) -> None:
did = _make_id()
db.create_model_definition(
definition_id=did,
alias="sampling",
model="gpt-5",
temperature=0.7,
max_tokens=8192,
reasoning_effort="high",
)
m = db.get_model_definition(did)
assert m is not None
assert m["temperature"] == 0.7
assert m["max_tokens"] == 8192
assert m["reasoning_effort"] == "high"
def test_create_with_zero_temperature(self, db: SQLiteBackend) -> None:
"""temperature=0.0 is a valid override, distinct from None."""
did = _make_id()
db.create_model_definition(
definition_id=did, alias="zero-temp", model="o3", temperature=0.0
)
m = db.get_model_definition(did)
assert m is not None
assert m["temperature"] == 0.0
def test_update_sampling_params(self, db: SQLiteBackend) -> None:
did = _make_id()
db.create_model_definition(definition_id=did, alias="upd-samp", model="gpt-5")
db.update_model_definition(did, temperature=1.2, max_tokens=4096, reasoning_effort="low")
m = db.get_model_definition(did)
assert m is not None
assert m["temperature"] == 1.2
assert m["max_tokens"] == 4096
assert m["reasoning_effort"] == "low"
def test_clear_sampling_params(self, db: SQLiteBackend) -> None:
"""Setting sampling params to None clears them back to global default."""
did = _make_id()
db.create_model_definition(
definition_id=did, alias="clear-samp", model="gpt-5", temperature=0.9
)
db.update_model_definition(did, temperature=None)
m = db.get_model_definition(did)
assert m is not None
assert m["temperature"] is None
+116
View File
@@ -51,6 +51,31 @@ class TestModelConfig:
cfg = ModelConfig(alias="test", base_url="http://x", api_key="sk-secret-key", model="m")
assert "sk-secret-key" not in repr(cfg)
def test_sampling_params_default_none(self) -> None:
cfg = ModelConfig(alias="x", base_url="x", api_key="x", model="x")
assert cfg.temperature is None
assert cfg.max_tokens is None
assert cfg.reasoning_effort is None
def test_sampling_params_set(self) -> None:
cfg = ModelConfig(
alias="x",
base_url="x",
api_key="x",
model="x",
temperature=0.7,
max_tokens=8192,
reasoning_effort="high",
)
assert cfg.temperature == 0.7
assert cfg.max_tokens == 8192
assert cfg.reasoning_effort == "high"
def test_zero_temperature_distinct_from_none(self) -> None:
cfg = ModelConfig(alias="x", base_url="x", api_key="x", model="x", temperature=0.0)
assert cfg.temperature == 0.0
assert cfg.temperature is not None
# ---------------------------------------------------------------------------
# ModelRegistry
@@ -483,6 +508,58 @@ class TestLoadModelRegistryWithDB:
reg = load_model_registry("http://x/v1", "x", "x", storage=storage)
assert reg.get_config("caps-model").capabilities == {"supports_vision": True}
def test_db_sampling_params_loaded(self) -> None:
"""Per-model sampling params from DB are carried in ModelConfig."""
storage = _MockStorage(
[
{
"alias": "hot-model",
"model": "m",
"provider": "openai",
"base_url": "",
"api_key": "",
"context_window": 32768,
"capabilities": "{}",
"enabled": True,
"temperature": 1.5,
"max_tokens": 4096,
"reasoning_effort": "high",
}
]
)
with patch("turnstone.core.model_registry.load_config", return_value={}):
reg = load_model_registry("http://x/v1", "x", "x", storage=storage)
cfg = reg.get_config("hot-model")
assert cfg.temperature == 1.5
assert cfg.max_tokens == 4096
assert cfg.reasoning_effort == "high"
def test_db_sampling_params_null_means_none(self) -> None:
"""NULL sampling params in DB map to None (use global default)."""
storage = _MockStorage(
[
{
"alias": "null-model",
"model": "m",
"provider": "openai",
"base_url": "",
"api_key": "",
"context_window": 32768,
"capabilities": "{}",
"enabled": True,
"temperature": None,
"max_tokens": None,
"reasoning_effort": None,
}
]
)
with patch("turnstone.core.model_registry.load_config", return_value={}):
reg = load_model_registry("http://x/v1", "x", "x", storage=storage)
cfg = reg.get_config("null-model")
assert cfg.temperature is None
assert cfg.max_tokens is None
assert cfg.reasoning_effort is None
def test_db_default_alias_not_clobbered(self) -> None:
"""DB model with alias='default' is not overwritten by CLI args."""
storage = _MockStorage(
@@ -688,6 +765,45 @@ class TestSessionModelCommand:
assert session.context_window == 64000
assert "Switched to" in session.ui.infos[-1]
def test_model_switch_applies_sampling_params(self) -> None:
reg = ModelRegistry(
models={
"default": ModelConfig("default", "x", "x", "default-model"),
"hot": ModelConfig(
"hot",
"y",
"y",
"hot-model",
temperature=1.5,
max_tokens=2048,
reasoning_effort="high",
),
},
default="default",
)
session = _make_session(registry=reg, model_alias="default")
assert session.temperature == 0.5 # initial global default
session.handle_command("/model hot")
assert session.temperature == 1.5
assert session.max_tokens == 2048
assert session.reasoning_effort == "high"
def test_model_switch_none_params_reverts_to_global(self) -> None:
"""Switching to a model with no overrides reverts to global defaults."""
reg = ModelRegistry(
models={
"hot": ModelConfig("hot", "x", "x", "hot-model", temperature=1.5),
"plain": ModelConfig("plain", "y", "y", "plain-model"),
},
default="hot",
)
session = _make_session(registry=reg, model_alias="hot")
session.temperature = 1.5 # as set by per-model override
# Without a config_store, fallback keeps current value (CLI sessions).
# With a config_store, it would revert to the global default.
session.handle_command("/model plain")
assert session.temperature == 1.5 # no config_store → keeps current
def test_model_switch_unknown_alias(self) -> None:
reg = ModelRegistry(
models={"default": ModelConfig("default", "x", "x", "test-model")},
+3 -3
View File
@@ -69,7 +69,7 @@ class TestValidateValueCoercion:
validate_value("tools.timeout", None)
def test_str(self):
assert validate_value("model.name", "gpt-5") == "gpt-5"
assert validate_value("model.default_alias", "gpt5-prod") == "gpt5-prod"
assert validate_value("session.instructions", "be nice") == "be nice"
@@ -143,10 +143,10 @@ class TestSerializeDeserialize:
def test_str_round_trip(self):
v = "hello world"
assert deserialize_value("model.name", serialize_value(v)) == v
assert deserialize_value("model.default_alias", serialize_value(v)) == v
def test_str_round_trip_empty(self):
assert deserialize_value("model.name", serialize_value("")) == ""
assert deserialize_value("model.default_alias", serialize_value("")) == ""
# ---------------------------------------------------------------------------
+9
View File
@@ -812,6 +812,9 @@ class ModelDefinitionInfo(BaseModel):
context_window: int = 32768
capabilities: str = "{}"
enabled: bool = True
temperature: float | None = None
max_tokens: int | None = None
reasoning_effort: str | None = None
source: str = ""
created_by: str = ""
created: str = ""
@@ -827,6 +830,9 @@ class CreateModelDefinitionRequest(BaseModel):
context_window: int = 32768
capabilities: dict[str, Any] = Field(default_factory=dict)
enabled: bool = True
temperature: float | None = None
max_tokens: int | None = None
reasoning_effort: str | None = None
class UpdateModelDefinitionRequest(BaseModel):
@@ -838,6 +844,9 @@ class UpdateModelDefinitionRequest(BaseModel):
context_window: int | None = None
capabilities: dict[str, Any] | None = None
enabled: bool | None = None
temperature: float | None = None
max_tokens: int | None = None
reasoning_effort: str | None = None
class ListModelDefinitionsResponse(BaseModel):
+89
View File
@@ -5235,6 +5235,9 @@ async def admin_import_mcp_config(request: Request) -> JSONResponse:
_MODEL_ALIAS_RE = re.compile(r"^[a-zA-Z0-9._-]+$")
_MODEL_PROVIDERS = frozenset({"openai", "anthropic", "openai-compatible", "google"})
_REASONING_EFFORT_CHOICES = frozenset(
{"", "none", "minimal", "low", "medium", "high", "xhigh", "max"}
)
# Keep in sync with turnstone.core.providers._google.GOOGLE_DEFAULT_BASE_URL
_PROVIDER_DEFAULT_URLS: dict[str, str] = {
"openai": "https://api.openai.com/v1",
@@ -5347,12 +5350,18 @@ async def admin_list_model_definitions(request: Request) -> JSONResponse:
model_name = ""
provider = "openai"
context_window = 0
cfg_temperature = None
cfg_max_tokens = None
cfg_reasoning_effort = None
for node_models in node_statuses.values():
nm = node_models.get(alias)
if nm:
model_name = nm.get("model", "")
provider = nm.get("provider", "openai")
context_window = nm.get("context_window", 0)
cfg_temperature = nm.get("temperature")
cfg_max_tokens = nm.get("max_tokens")
cfg_reasoning_effort = nm.get("reasoning_effort")
break
result.append(
{
@@ -5365,6 +5374,9 @@ async def admin_list_model_definitions(request: Request) -> JSONResponse:
"context_window": context_window,
"capabilities": "{}",
"enabled": True,
"temperature": cfg_temperature,
"max_tokens": cfg_max_tokens,
"reasoning_effort": cfg_reasoning_effort,
"source": "config",
"created_by": "",
"created": "",
@@ -5454,6 +5466,36 @@ async def admin_create_model_definition(request: Request) -> JSONResponse:
capabilities = json.dumps(caps) if isinstance(caps, dict) else "{}"
enabled = bool(body.get("enabled", True))
# Per-model sampling overrides (None = use global default)
temperature: float | None = None
if body.get("temperature") is not None:
try:
temperature = float(body["temperature"])
except (ValueError, TypeError):
return JSONResponse({"error": "temperature must be a number"}, status_code=400)
if not 0.0 <= temperature <= 2.0:
return JSONResponse(
{"error": "temperature must be between 0.0 and 2.0"}, status_code=400
)
max_tokens: int | None = None
if body.get("max_tokens") is not None:
try:
max_tokens = int(body["max_tokens"])
except (ValueError, TypeError):
return JSONResponse({"error": "max_tokens must be an integer"}, status_code=400)
if max_tokens < 1:
return JSONResponse({"error": "max_tokens must be >= 1"}, status_code=400)
reasoning_effort: str | None = None
if body.get("reasoning_effort") is not None:
reasoning_effort = str(body["reasoning_effort"]).strip()
if reasoning_effort and reasoning_effort not in _REASONING_EFFORT_CHOICES:
return JSONResponse(
{"error": f"Invalid reasoning_effort: {reasoning_effort!r}"},
status_code=400,
)
if not reasoning_effort:
reasoning_effort = None
storage.create_model_definition(
definition_id=definition_id,
alias=alias,
@@ -5465,6 +5507,9 @@ async def admin_create_model_definition(request: Request) -> JSONResponse:
capabilities=capabilities,
enabled=enabled,
created_by=audit_uid,
temperature=temperature,
max_tokens=max_tokens,
reasoning_effort=reasoning_effort,
)
record_audit(
@@ -5573,6 +5618,50 @@ async def admin_update_model_definition(request: Request) -> JSONResponse:
if "enabled" in body:
updates["enabled"] = bool(body["enabled"])
# Per-model sampling overrides — explicit null clears to "use global default"
if "temperature" in body:
raw_temp = body["temperature"]
if raw_temp is None:
updates["temperature"] = None
else:
try:
temp_val = float(raw_temp)
except (ValueError, TypeError):
return JSONResponse({"error": "temperature must be a number"}, status_code=400)
if not 0.0 <= temp_val <= 2.0:
return JSONResponse(
{"error": "temperature must be between 0.0 and 2.0"},
status_code=400,
)
updates["temperature"] = temp_val
if "max_tokens" in body:
raw_mt = body["max_tokens"]
if raw_mt is None:
updates["max_tokens"] = None
else:
try:
mt_val = int(raw_mt)
except (ValueError, TypeError):
return JSONResponse({"error": "max_tokens must be an integer"}, status_code=400)
if mt_val < 1:
return JSONResponse({"error": "max_tokens must be >= 1"}, status_code=400)
updates["max_tokens"] = mt_val
if "reasoning_effort" in body:
raw_re = body["reasoning_effort"]
if raw_re is None:
updates["reasoning_effort"] = None
else:
re_val = str(raw_re).strip()
if not re_val:
updates["reasoning_effort"] = None
elif re_val not in _REASONING_EFFORT_CHOICES:
return JSONResponse(
{"error": f"Invalid reasoning_effort: {re_val!r}"},
status_code=400,
)
else:
updates["reasoning_effort"] = re_val
if updates:
storage.update_model_definition(definition_id, **updates)
+57
View File
@@ -4447,6 +4447,24 @@ function _renderModels(items) {
colAlias.appendChild(document.createTextNode(" "));
colAlias.appendChild(defBadge);
}
// Per-model sampling override indicators
var overrides = [];
if (m.temperature != null) overrides.push("temp=" + m.temperature);
if (m.max_tokens != null) overrides.push("max_tok=" + m.max_tokens);
if (m.reasoning_effort != null)
overrides.push("effort=" + m.reasoning_effort);
if (overrides.length) {
var ovrSpan = document.createElement("span");
ovrSpan.className = "model-overrides-hint";
ovrSpan.textContent = overrides.join(", ");
ovrSpan.title = "Per-model overrides (override global defaults)";
ovrSpan.setAttribute(
"aria-label",
"Per-model overrides: " + overrides.join(", "),
);
colAlias.appendChild(document.createElement("br"));
colAlias.appendChild(ovrSpan);
}
row.appendChild(colAlias);
// Model ID
@@ -4593,6 +4611,9 @@ function showCreateModelModal() {
document.getElementById("model-api-key").value = "";
document.getElementById("model-api-key").placeholder = "sk-...";
document.getElementById("model-ctx-window").value = "0";
document.getElementById("model-temperature").value = "";
document.getElementById("model-max-tokens").value = "";
document.getElementById("model-reasoning-effort").value = "";
document.getElementById("model-capabilities").value = "";
document.getElementById("model-enabled").checked = true;
document.getElementById("model-detect-result").style.display = "none";
@@ -4626,6 +4647,12 @@ function showEditModelModal(definitionId) {
"\u2022\u2022\u2022 (leave blank to keep existing)";
document.getElementById("model-ctx-window").value =
m.context_window != null ? m.context_window : 0;
document.getElementById("model-temperature").value =
m.temperature != null ? m.temperature : "";
document.getElementById("model-max-tokens").value =
m.max_tokens != null ? m.max_tokens : "";
document.getElementById("model-reasoning-effort").value =
m.reasoning_effort != null ? m.reasoning_effort : "";
// Parse capabilities JSON for display
var caps = m.capabilities || "{}";
try {
@@ -4689,6 +4716,36 @@ function submitCreateModel() {
enabled: document.getElementById("model-enabled").checked,
};
// Per-model sampling overrides — null when empty (use global default)
var tempVal = document.getElementById("model-temperature").value.trim();
if (tempVal !== "") {
var t = parseFloat(tempVal);
if (isNaN(t) || t < 0 || t > 2) {
_showModelError("Temperature must be between 0 and 2");
return;
}
form.temperature = t;
} else {
form.temperature = null;
}
var mtVal = document.getElementById("model-max-tokens").value.trim();
if (mtVal !== "") {
var mt = parseInt(mtVal, 10);
if (isNaN(mt) || mt < 1) {
_showModelError("Max tokens must be at least 1");
return;
}
form.max_tokens = mt;
} else {
form.max_tokens = null;
}
var reVal = document.getElementById("model-reasoning-effort").value;
if (reVal !== "") {
form.reasoning_effort = reVal;
} else {
form.reasoning_effort = null;
}
var apiKey = document.getElementById("model-api-key").value;
if (apiKey) form.api_key = apiKey;
+16
View File
@@ -1551,6 +1551,22 @@ window.TURNSTONE_KB_SHORTCUTS = [
<input type="password" id="model-api-key" placeholder="sk-..." autocomplete="off">
<label for="model-ctx-window">Context Window <span style="font-weight:400;text-transform:none">(0 = auto-detect from model)</span></label>
<input type="number" id="model-ctx-window" value="0" min="0">
<div class="modal-section-divider" role="separator">Sampling Defaults</div>
<label for="model-temperature">Temperature <span style="font-weight:400;text-transform:none">(empty = use global default)</span></label>
<input type="number" id="model-temperature" placeholder="Global default" step="0.1" min="0" max="2">
<label for="model-max-tokens">Max Tokens <span style="font-weight:400;text-transform:none">(empty = use global default)</span></label>
<input type="number" id="model-max-tokens" placeholder="Global default" min="1">
<label for="model-reasoning-effort">Reasoning Effort <span style="font-weight:400;text-transform:none">(empty = use global default)</span></label>
<select id="model-reasoning-effort">
<option value="">Global default</option>
<option value="none">none</option>
<option value="minimal">minimal</option>
<option value="low">low</option>
<option value="medium">medium</option>
<option value="high">high</option>
<option value="xhigh">xhigh</option>
<option value="max">max</option>
</select>
<label for="model-capabilities">Capabilities <span style="font-weight:400;text-transform:none">(JSON)</span></label>
<textarea id="model-capabilities" rows="3" placeholder='{"supports_vision": true}' style="font-family:var(--font-mono);font-size:11px"></textarea>
<div style="display:flex;gap:20px;margin-top:14px">
+6
View File
@@ -2471,6 +2471,12 @@ h3.skill-spec-heading { font-size: inherit; margin-block: 0; }
.model-provider-google{color:var(--green);border-color:rgba(52,211,153,.2)}
.model-provider-compat{color:var(--fg-dim);border-color:var(--border-strong)}
/* Per-model override hints */
.model-overrides-hint{font-size:10px;color:var(--fg-dim);font-family:var(--font-mono);letter-spacing:.02em}
/* Modal section divider for field groups */
.modal-section-divider{font-family:var(--font-display);font-size:9px;font-weight:600;text-transform:uppercase;letter-spacing:.1em;color:var(--fg-dim);margin:16px 0 4px;padding-top:12px;border-top:1px solid var(--border)}
/* Model source badge */
.scope-db{color:var(--blue);border-color:rgba(56,189,248,.2)}
+18
View File
@@ -266,3 +266,21 @@ def warn_migrated_settings() -> None:
config_key,
key,
)
# Warn about removed settings whose config.toml keys are now ignored.
# model.name → use model definitions (Models tab); model.context_window
# → set per-model in the Models tab (context_window column).
removed_settings: dict[str, str] = {
"model.name": "Use model definitions in the Models tab instead.",
"model.context_window": "Set per-model in the Models tab instead.",
}
for key, guidance in removed_settings.items():
section, config_key = key.split(".", 1)
section_data = cfg.get(section, {})
if isinstance(section_data, dict) and config_key in section_data:
log.warning(
"config.toml [%s] %s has been removed and will be ignored. %s",
section,
config_key,
guidance,
)
+46
View File
@@ -35,6 +35,10 @@ class ModelConfig:
provider: str = "openai"
capabilities: dict[str, Any] = field(default_factory=dict)
source: str = "" # "config", "db", or "" (CLI default)
# Per-model sampling overrides (None = use global default from ConfigStore)
temperature: float | None = None
max_tokens: int | None = None
reasoning_effort: str | None = None
# ---------------------------------------------------------------------------
@@ -268,6 +272,10 @@ def load_model_registry(
# 0 = auto-detect: inherit CLI-detected context_window,
# same fallback chain as config.toml models
row_ctx = row.get("context_window", 0) or context_window
# Per-model sampling overrides (None = use global default)
row_temperature = row.get("temperature")
row_max_tokens = row.get("max_tokens")
row_reasoning_effort = row.get("reasoning_effort")
configs[alias] = ModelConfig(
alias=alias,
base_url=row_base_url,
@@ -277,6 +285,11 @@ def load_model_registry(
provider=row_provider,
capabilities=caps,
source="db",
temperature=float(row_temperature) if row_temperature is not None else None,
max_tokens=int(row_max_tokens) if row_max_tokens is not None else None,
reasoning_effort=row_reasoning_effort
if row_reasoning_effort is not None
else None,
)
except Exception:
log.warning("Failed to load model definitions from storage", exc_info=True)
@@ -290,6 +303,36 @@ def load_model_registry(
log.warning("Model entry '%s' has no model name, skipping", alias)
continue
entry_base_url = _resolve_env_vars(entry.get("base_url", base_url))
# Per-model sampling overrides from config.toml — invalid values
# are logged and treated as None (inherit global default).
entry_temp: float | None = None
entry_max_tokens: int | None = None
entry_effort: str | None = None
raw_temp = entry.get("temperature")
if raw_temp is not None:
try:
entry_temp = float(raw_temp)
if not 0.0 <= entry_temp <= 2.0:
log.warning(
"Model '%s' temperature %.2f out of range [0, 2], ignoring",
alias,
entry_temp,
)
entry_temp = None
except (ValueError, TypeError):
log.warning("Model '%s' has invalid temperature %r, ignoring", alias, raw_temp)
raw_mt = entry.get("max_tokens")
if raw_mt is not None:
try:
entry_max_tokens = int(raw_mt)
if entry_max_tokens < 1:
log.warning("Model '%s' max_tokens %d < 1, ignoring", alias, entry_max_tokens)
entry_max_tokens = None
except (ValueError, TypeError):
log.warning("Model '%s' has invalid max_tokens %r, ignoring", alias, raw_mt)
raw_effort = entry.get("reasoning_effort")
if raw_effort is not None:
entry_effort = str(raw_effort)
configs[alias] = ModelConfig(
alias=alias,
base_url=entry_base_url,
@@ -301,6 +344,9 @@ def load_model_registry(
if isinstance(entry.get("capabilities"), dict)
else {},
source="config",
temperature=entry_temp,
max_tokens=entry_max_tokens,
reasoning_effort=entry_effort,
)
# 3. Ensure a "default" entry from CLI args (only if not already defined
+19
View File
@@ -6924,6 +6924,25 @@ class ChatSession:
self.context_window = cfg.context_window
if not self._manual_tool_truncation:
self.tool_truncation = int(cfg.context_window * self._chars_per_token * 0.5)
# Apply per-model sampling overrides, falling back to global
# defaults — mirrors session_factory() resolution logic so
# switching away from a model with overrides doesn't leak them.
cs = self._config_store
self.temperature = (
cfg.temperature
if cfg.temperature is not None
else (cs.get("model.temperature") if cs else self.temperature)
)
self.max_tokens = (
cfg.max_tokens
if cfg.max_tokens is not None
else (cs.get("model.max_tokens") if cs else self.max_tokens)
)
self.reasoning_effort = (
cfg.reasoning_effort
if cfg.reasoning_effort is not None
else (cs.get("model.reasoning_effort") if cs else self.reasoning_effort)
)
self._init_system_messages()
self._save_config()
self.ui.on_info(f"Switched to {cyan(arg)}: {model_name}")
+14 -29
View File
@@ -35,14 +35,6 @@ def _build_registry() -> dict[str, SettingDef]:
"""Build the settings registry from declarative definitions."""
defs: list[SettingDef] = [
# -- model ----------------------------------------------------------
SettingDef(
"model.name",
"str",
"",
"Default model name (empty = use provider default)",
"model",
help="Which AI model to use for conversations. Leave empty to use the provider's default.",
),
SettingDef(
"model.default_alias",
"str",
@@ -58,45 +50,38 @@ def _build_registry() -> dict[str, SettingDef]:
"model.temperature",
"float",
0.5,
"Sampling temperature (ignored by models that don't support it, e.g. o-series)",
"Default sampling temperature (overridden by per-model settings)",
"model",
min_value=0.0,
max_value=2.0,
help="Controls randomness in responses. Lower values (0.0\u20130.3) give focused, "
"deterministic output; higher values (0.7\u20131.5) make responses more creative and varied.",
help="Default sampling temperature for models without a per-model override. "
"Controls randomness in responses. Lower values (0.0\u20130.3) give focused, "
"deterministic output; higher values (0.7\u20131.5) make responses more creative "
"and varied. Per-model overrides can be set in the Models tab.",
reference_url="https://arxiv.org/abs/1904.09751",
),
SettingDef(
"model.max_tokens",
"int",
32768,
"Max output tokens per response",
"Default max output tokens (overridden by per-model settings)",
"model",
min_value=1,
help="Upper limit on how long each response can be. One token is roughly 4 characters "
"of English text. Higher values allow longer responses but cost more.",
help="Default max output tokens for models without a per-model override. "
"Upper limit on how long each response can be. One token is roughly 4 characters "
"of English text. Per-model overrides can be set in the Models tab.",
),
SettingDef(
"model.reasoning_effort",
"str",
"medium",
"Reasoning effort level (only applies to models with reasoning support)",
"Default reasoning effort (overridden by per-model settings)",
"model",
choices=["", "none", "minimal", "low", "medium", "high", "xhigh", "max"],
help="How much internal \u2018thinking\u2019 the model does before responding. Higher effort "
"improves quality on complex tasks but is slower and uses more tokens. Not all models "
"support this \u2014 it is silently ignored when unsupported.",
),
SettingDef(
"model.context_window",
"int",
0,
"Context window size in tokens (0 = auto-detect from model)",
"model",
min_value=0,
help="How much conversation history the model can see at once, measured in tokens "
"(~4 characters each). Set to 0 to auto-detect from the model. Only override this "
"if auto-detection fails (common with local models).",
help="Default reasoning effort for models without a per-model override. "
"Controls how much internal \u2018thinking\u2019 the model does before responding. "
"Higher effort improves quality on complex tasks but is slower and uses more "
"tokens. Per-model overrides can be set in the Models tab.",
),
# -- session --------------------------------------------------------
SettingDef(
+6
View File
@@ -3084,6 +3084,9 @@ class PostgreSQLBackend:
capabilities: str = "{}",
enabled: bool = True,
created_by: str = "",
temperature: float | None = None,
max_tokens: int | None = None,
reasoning_effort: str | None = None,
) -> None:
from sqlalchemy.dialects import postgresql
@@ -3101,6 +3104,9 @@ class PostgreSQLBackend:
context_window=context_window,
capabilities=capabilities,
enabled=1 if enabled else 0,
temperature=temperature,
max_tokens=max_tokens,
reasoning_effort=reasoning_effort,
created_by=created_by,
created=now,
updated=now,
+3
View File
@@ -1070,6 +1070,9 @@ class StorageBackend(Protocol):
capabilities: str = "{}",
enabled: bool = True,
created_by: str = "",
temperature: float | None = None,
max_tokens: int | None = None,
reasoning_effort: str | None = None,
) -> None:
"""Create a model definition. No-op if definition_id already exists."""
...
+3
View File
@@ -587,6 +587,9 @@ model_definitions = sa.Table(
sa.Column("context_window", sa.Integer, nullable=False, server_default="32768"),
sa.Column("capabilities", sa.Text, nullable=False, server_default="{}"),
sa.Column("enabled", sa.Integer, nullable=False, server_default="1"),
sa.Column("temperature", sa.Float, nullable=True),
sa.Column("max_tokens", sa.Integer, nullable=True),
sa.Column("reasoning_effort", sa.Text, nullable=True),
sa.Column("created_by", sa.Text, nullable=False, server_default=""),
sa.Column("created", sa.Text, nullable=False),
sa.Column("updated", sa.Text, nullable=False),
+6
View File
@@ -3147,6 +3147,9 @@ class SQLiteBackend:
capabilities: str = "{}",
enabled: bool = True,
created_by: str = "",
temperature: float | None = None,
max_tokens: int | None = None,
reasoning_effort: str | None = None,
) -> None:
now = datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%S")
@@ -3163,6 +3166,9 @@ class SQLiteBackend:
"context_window": context_window,
"capabilities": capabilities,
"enabled": 1 if enabled else 0,
"temperature": temperature,
"max_tokens": max_tokens,
"reasoning_effort": reasoning_effort,
"created_by": created_by,
"created": now,
"updated": now,
+3
View File
@@ -106,6 +106,9 @@ MODEL_DEFINITION_MUTABLE = frozenset(
"context_window",
"capabilities",
"enabled",
"temperature",
"max_tokens",
"reasoning_effort",
}
)
PROMPT_POLICY_MUTABLE = frozenset({"name", "content", "tool_gate", "priority", "enabled"})
@@ -0,0 +1,32 @@
"""Add per-model sampling parameters to model_definitions.
Adds nullable temperature, max_tokens, and reasoning_effort columns
so each model can override the global defaults. NULL means "inherit
the cluster-wide setting from system_settings".
Revision ID: 036
Revises: 035
Create Date: 2026-04-13
"""
import sqlalchemy as sa
from alembic import op
revision = "036"
down_revision = "035"
branch_labels = None
depends_on = None
def upgrade() -> None:
with op.batch_alter_table("model_definitions") as batch:
batch.add_column(sa.Column("temperature", sa.Float, nullable=True))
batch.add_column(sa.Column("max_tokens", sa.Integer, nullable=True))
batch.add_column(sa.Column("reasoning_effort", sa.Text, nullable=True))
def downgrade() -> None:
with op.batch_alter_table("model_definitions") as batch:
batch.drop_column("reasoning_effort")
batch.drop_column("max_tokens")
batch.drop_column("temperature")
+25 -9
View File
@@ -2774,6 +2774,9 @@ def internal_model_status(request: Request) -> JSONResponse:
"source": cfg.source,
"context_window": cfg.context_window,
"enabled": True,
"temperature": cfg.temperature,
"max_tokens": cfg.max_tokens,
"reasoning_effort": cfg.reasoning_effort,
}
return JSONResponse({"models": models})
@@ -3406,9 +3409,8 @@ def main() -> None:
client = create_client(provider_name, base_url=base_url, api_key=api_key)
cs_model = config_store.get("model.name")
cli_model = args.model
effective_model = cli_model or cs_model or None
effective_model = cli_model or None
if effective_model:
model = effective_model
detected_ctx = None
@@ -3422,13 +3424,10 @@ def main() -> None:
# entry and relies on DB / config.toml models instead.
model = ""
# Use detected context window, fall back to ConfigStore override or 32768
cfg_ctx = config_store.get("model.context_window")
# Use detected context window, fall back to 32768
if detected_ctx:
context_window = detected_ctx
log.info("Context window: %s (detected from backend)", f"{context_window:,}")
elif cfg_ctx: # 0 = auto-detect (no override)
context_window = cfg_ctx
else:
context_window = 32768
@@ -3595,15 +3594,32 @@ def main() -> None:
except Exception as e:
log.warning("Failed to resolve judge_model %r: %s", judge_model, e)
# Per-model sampling overrides take priority over global defaults
eff_temperature = (
r_cfg.temperature
if r_cfg.temperature is not None
else config_store.get("model.temperature")
)
eff_max_tokens = (
r_cfg.max_tokens
if r_cfg.max_tokens is not None
else config_store.get("model.max_tokens")
)
eff_reasoning_effort = (
r_cfg.reasoning_effort
if r_cfg.reasoning_effort is not None
else config_store.get("model.reasoning_effort")
)
return ChatSession(
client=r_client,
model=r_model,
ui=ui,
instructions=config_store.get("session.instructions") or None,
temperature=config_store.get("model.temperature"),
max_tokens=config_store.get("model.max_tokens"),
temperature=eff_temperature,
max_tokens=eff_max_tokens,
tool_timeout=config_store.get("tools.timeout"),
reasoning_effort=config_store.get("model.reasoning_effort"),
reasoning_effort=eff_reasoning_effort,
context_window=r_cfg.context_window,
compact_max_tokens=config_store.get("session.compact_max_tokens"),
auto_compact_pct=config_store.get("session.auto_compact_pct"),